2023-11-08T16:11:45.490 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-08T16:11:45.491 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} --verbose --owner scheduled_vshankar@teuthology --name vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595 -- /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595/orig.config.yaml 2023-11-08T16:11:45.538 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-08T16:11:45.626 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595 branch: wip-vshankar-testing-20231106.073650 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} email: vshankar@redhat.com first_in_suite: false job_id: '7449595' kernel: flavor: default kdb: true sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20231106.073650 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - inconsistent rstat on inode - Error recovering journal sha1: 85bdb240f5cf2a758acc9b247951fbc8f5799bc1 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 85bdb240f5cf2a758acc9b247951fbc8f5799bc1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20231106.073650 sha1: 85bdb240f5cf2a758acc9b247951fbc8f5799bc1 owner: scheduled_vshankar@teuthology priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 8308 sha1: 85bdb240f5cf2a758acc9b247951fbc8f5799bc1 sleep_before_teardown: 0 subset: 430/1024 suite: fs suite_branch: wip-vshankar-testing-20231106.073650 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 85bdb240f5cf2a758acc9b247951fbc8f5799bc1 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEb/Zd1RkyJlvHItW402GsLGrVK/DxAP1UQ3Ffdb0JiiZSX55PYP8qQj1NkQP07dHaUwrlKgI7Xouph9A5trxkU= smithi093.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBITVhNVsh9QJgG0rtGpBJz4qGMcOK6+9C0lzvZOaEmIqMbNTLzfFp3YuUcIX2Eq0w7oJ2Tjqx/svsh6/pCfWsGY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_data_scan teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 6899cd26fceddb2fec83dc1a1349394b28c8998e timestamp: 2023-11-06_10:33:49 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714 2023-11-08T16:11:45.627 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa; will attempt to use it 2023-11-08T16:11:45.628 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa/tasks 2023-11-08T16:11:45.628 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-08T16:11:45.630 INFO:teuthology.task.internal:Checking packages... 2023-11-08T16:11:45.657 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '85bdb240f5cf2a758acc9b247951fbc8f5799bc1' 2023-11-08T16:11:45.658 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-08T16:11:45.658 INFO:teuthology.packaging:ref: None 2023-11-08T16:11:45.659 INFO:teuthology.packaging:tag: None 2023-11-08T16:11:45.659 INFO:teuthology.packaging:branch: wip-vshankar-testing-20231106.073650 2023-11-08T16:11:45.659 INFO:teuthology.packaging:sha1: 85bdb240f5cf2a758acc9b247951fbc8f5799bc1 2023-11-08T16:11:45.660 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20231106.073650 2023-11-08T16:11:45.804 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7083.g85bdb240 2023-11-08T16:11:45.805 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-08T16:11:45.810 INFO:teuthology.task.internal:no buildpackages task found 2023-11-08T16:11:45.811 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-08T16:11:45.816 INFO:teuthology.task.internal:Saving configuration 2023-11-08T16:11:45.836 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-08T16:11:45.842 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T16:11:45.867 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 16:06:48.927014', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEb/Zd1RkyJlvHItW402GsLGrVK/DxAP1UQ3Ffdb0JiiZSX55PYP8qQj1NkQP07dHaUwrlKgI7Xouph9A5trxkU='} 2023-11-08T16:11:45.888 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 16:06:48.924968', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBITVhNVsh9QJgG0rtGpBJz4qGMcOK6+9C0lzvZOaEmIqMbNTLzfFp3YuUcIX2Eq0w7oJ2Tjqx/svsh6/pCfWsGY='} 2023-11-08T16:11:45.889 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-08T16:11:45.894 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-08T16:11:45.894 INFO:teuthology.task.internal:roles: ubuntu@smithi093.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-08T16:11:45.895 INFO:teuthology.run_tasks:Running task console_log... 2023-11-08T16:11:45.973 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f035a855b80>, signals=[15]) 2023-11-08T16:11:45.974 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-08T16:11:45.980 INFO:teuthology.task.internal:Opening connections... 2023-11-08T16:11:45.980 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:11:45.981 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T16:11:46.065 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:11:46.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T16:11:46.152 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-08T16:11:46.160 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2023-11-08T16:11:46.179 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2023-11-08T16:11:46.180 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2023-11-08T16:11:46.236 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2023-11-08T16:11:46.236 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="9" 2023-11-08T16:11:46.236 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2023-11-08T16:11:46.237 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2023-11-08T16:11:46.237 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="9" 2023-11-08T16:11:46.237 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el9" 2023-11-08T16:11:46.237 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-08T16:11:46.238 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2023-11-08T16:11:46.238 INFO:teuthology.orchestra.run.smithi081.stdout:LOGO="fedora-logo-icon" 2023-11-08T16:11:46.238 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-08T16:11:46.239 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2023-11-08T16:11:46.239 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-08T16:11:46.239 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-08T16:11:46.240 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-08T16:11:46.241 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2023-11-08T16:11:46.263 DEBUG:teuthology.orchestra.run.smithi093:> uname -m 2023-11-08T16:11:46.280 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64 2023-11-08T16:11:46.280 DEBUG:teuthology.orchestra.run.smithi093:> cat /etc/os-release 2023-11-08T16:11:46.336 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="CentOS Stream" 2023-11-08T16:11:46.337 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="9" 2023-11-08T16:11:46.337 INFO:teuthology.orchestra.run.smithi093.stdout:ID="centos" 2023-11-08T16:11:46.337 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="rhel fedora" 2023-11-08T16:11:46.338 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="9" 2023-11-08T16:11:46.338 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el9" 2023-11-08T16:11:46.338 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-08T16:11:46.338 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31" 2023-11-08T16:11:46.338 INFO:teuthology.orchestra.run.smithi093.stdout:LOGO="fedora-logo-icon" 2023-11-08T16:11:46.339 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-08T16:11:46.339 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://centos.org/" 2023-11-08T16:11:46.339 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-08T16:11:46.339 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-08T16:11:46.339 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-08T16:11:46.340 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2023-11-08T16:11:46.371 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-08T16:11:46.380 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-08T16:11:46.396 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-08T16:11:46.396 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2023-11-08T16:11:46.399 DEBUG:teuthology.orchestra.run.smithi093:> test '!' -e /home/ubuntu/cephtest 2023-11-08T16:11:46.414 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-08T16:11:46.420 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-08T16:11:46.421 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2023-11-08T16:11:46.456 DEBUG:teuthology.orchestra.run.smithi093:> test -z $(ls -A /var/lib/ceph) 2023-11-08T16:11:46.789 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-08T16:11:46.845 INFO:teuthology.run_tasks:Running task kernel... 2023-11-08T16:11:46.859 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'} 2023-11-08T16:11:46.860 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}}, timeout 300 2023-11-08T16:11:46.861 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-08T16:11:46.862 DEBUG:teuthology.orchestra.run.smithi093:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-08T16:11:46.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:11:46.879 DEBUG:teuthology.task.kernel:sha1 for mon.b is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T16:11:46.879 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-11-08T16:11:46.879 DEBUG:teuthology.orchestra.run.smithi093:> uname -r 2023-11-08T16:11:46.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:11:46.881 DEBUG:teuthology.task.kernel:sha1 for mon.a is 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T16:11:46.881 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5b73517e0e2fa132ea79290e5fc06ea5349b2288"... 2023-11-08T16:11:46.881 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2023-11-08T16:11:46.936 INFO:teuthology.orchestra.run.smithi093.stdout:5.14.0-361.el9.x86_64 2023-11-08T16:11:46.936 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T16:11:46.937 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-08T16:11:46.937 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T16:11:46.941 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-361.el9.x86_64 2023-11-08T16:11:46.941 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T16:11:46.941 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-08T16:11:46.941 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T16:11:47.080 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum upgrade -y linux-firmware 2023-11-08T16:11:47.148 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum upgrade -y linux-firmware 2023-11-08T16:11:49.136 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - AppStream 15 MB/s | 6.3 MB 00:00 2023-11-08T16:11:49.167 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS-9 - AppStream 16 MB/s | 6.3 MB 00:00 2023-11-08T16:11:50.493 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS-9 - BaseOS 7.2 MB/s | 2.0 MB 00:00 2023-11-08T16:11:50.770 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - BaseOS 3.4 MB/s | 2.0 MB 00:00 2023-11-08T16:11:52.085 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 8.0 MB/s | 7.9 MB 00:00 2023-11-08T16:11:55.587 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 18 MB 00:02 2023-11-08T16:11:59.837 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 6.0 MB/s | 5.8 MB 00:00 2023-11-08T16:12:01.255 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 15 kB 00:00 2023-11-08T16:12:03.823 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 8.0 MB/s | 19 MB 00:02 2023-11-08T16:12:08.531 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2023-11-08T16:12:10.921 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2023-11-08T16:12:10.930 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2023-11-08T16:12:10.930 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2023-11-08T16:12:10.930 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2023-11-08T16:12:10.930 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2023-11-08T16:12:10.931 INFO:teuthology.orchestra.run.smithi081.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-11-08T16:12:10.931 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:12:10.931 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2023-11-08T16:12:10.932 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2023-11-08T16:12:10.932 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 1 Package 2023-11-08T16:12:10.932 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:12:10.932 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 326 M 2023-11-08T16:12:10.933 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2023-11-08T16:12:15.032 INFO:teuthology.orchestra.run.smithi081.stdout:linux-firmware-20230926-140.el9.noarch.rpm 79 MB/s | 326 MB 00:04 2023-11-08T16:12:15.033 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2023-11-08T16:12:15.034 INFO:teuthology.orchestra.run.smithi081.stdout:Total 79 MB/s | 326 MB 00:04 2023-11-08T16:12:15.034 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2023-11-08T16:12:15.118 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2023-11-08T16:12:15.118 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2023-11-08T16:12:17.240 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2023-11-08T16:12:17.241 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2023-11-08T16:12:22.889 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2023-11-08T16:12:23.323 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-08T16:12:23.413 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T16:12:24.365 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T16:12:24.366 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-08T16:12:24.744 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T16:12:24.744 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:12:24.744 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2023-11-08T16:12:24.745 INFO:teuthology.orchestra.run.smithi081.stdout: linux-firmware-20230926-140.el9.noarch 2023-11-08T16:12:24.745 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:12:24.745 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2023-11-08T16:12:25.013 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on mon.a... 2023-11-08T16:12:25.014 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T16:12:25.155 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ 2023-11-08T16:12:25.156 DEBUG:teuthology.orchestra.run.smithi081:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-08T16:12:25.158 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi081.front.sepia.ceph.com to complete... 2023-11-08T16:12:36.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08 16:12:36 URL:https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [320585487/320585487] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-08T16:12:36.050 INFO:teuthology.orchestra.run.smithi081.stderr:FINISHED --2023-11-08 16:12:36-- 2023-11-08T16:12:36.050 INFO:teuthology.orchestra.run.smithi081.stderr:Total wall clock time: 11s 2023-11-08T16:12:36.050 INFO:teuthology.orchestra.run.smithi081.stderr:Downloaded: 1 files, 306M in 11s (28.9 MB/s) 2023-11-08T16:12:36.051 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on mon.a... 2023-11-08T16:12:36.052 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-08T16:12:37.328 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS Stream 9 - BaseOS 174 kB/s | 7.9 MB 00:46 2023-11-08T16:12:37.898 INFO:teuthology.orchestra.run.smithi081.stdout:Verifying... ######################################## 2023-11-08T16:12:37.947 INFO:teuthology.orchestra.run.smithi081.stdout:Preparing... ######################################## 2023-11-08T16:12:38.028 INFO:teuthology.orchestra.run.smithi081.stdout:Updating / installing... 2023-11-08T16:12:40.762 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 18 MB 00:02 2023-11-08T16:12:45.171 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-11-08T16:12:45.645 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 5.8 MB 00:01 2023-11-08T16:12:46.988 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 15 kB 00:00 2023-11-08T16:12:51.583 INFO:teuthology.orchestra.run.smithi093.stdout:Extra Packages for Enterprise Linux 4.4 MB/s | 19 MB 00:04 2023-11-08T16:12:56.364 INFO:teuthology.orchestra.run.smithi093.stdout:lab-extras 5.9 kB/s | 1.7 kB 00:00 2023-11-08T16:12:58.666 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-11-08T16:12:58.678 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-11-08T16:12:58.678 INFO:teuthology.orchestra.run.smithi093.stdout: Package Arch Version Repository Size 2023-11-08T16:12:58.678 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-11-08T16:12:58.679 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrading: 2023-11-08T16:12:58.679 INFO:teuthology.orchestra.run.smithi093.stdout: linux-firmware noarch 20230926-140.el9 CentOS-BaseOS 326 M 2023-11-08T16:12:58.679 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:12:58.679 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-11-08T16:12:58.679 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-11-08T16:12:58.680 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrade 1 Package 2023-11-08T16:12:58.680 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:12:58.680 INFO:teuthology.orchestra.run.smithi093.stdout:Total download size: 326 M 2023-11-08T16:12:58.680 INFO:teuthology.orchestra.run.smithi093.stdout:Downloading Packages: 2023-11-08T16:13:00.666 INFO:teuthology.orchestra.run.smithi081.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-08T16:13:02.340 INFO:teuthology.orchestra.run.smithi093.stdout:linux-firmware-20230926-140.el9.noarch.rpm 89 MB/s | 326 MB 00:03 2023-11-08T16:13:02.340 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-11-08T16:13:02.341 INFO:teuthology.orchestra.run.smithi093.stdout:Total 89 MB/s | 326 MB 00:03 2023-11-08T16:13:02.341 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction check 2023-11-08T16:13:02.438 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction check succeeded. 2023-11-08T16:13:02.438 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction test 2023-11-08T16:13:04.544 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction test succeeded. 2023-11-08T16:13:04.545 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction 2023-11-08T16:13:10.035 INFO:teuthology.orchestra.run.smithi093.stdout: Preparing : 1/1 2023-11-08T16:13:10.478 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-08T16:13:10.562 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T16:13:11.494 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T16:13:11.495 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : linux-firmware-20230926-140.el9.noarch 1/2 2023-11-08T16:13:11.880 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-08T16:13:11.881 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:13:11.881 INFO:teuthology.orchestra.run.smithi093.stdout:Upgraded: 2023-11-08T16:13:11.881 INFO:teuthology.orchestra.run.smithi093.stdout: linux-firmware-20230926-140.el9.noarch 2023-11-08T16:13:11.882 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:13:11.882 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-11-08T16:13:12.159 INFO:teuthology.task.kernel:Downloading kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on mon.b... 2023-11-08T16:13:12.159 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5b73517e0e2fa132ea79290e5fc06ea5349b2288 2023-11-08T16:13:12.311 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ 2023-11-08T16:13:12.311 DEBUG:teuthology.orchestra.run.smithi093:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-08T16:13:12.314 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi093.front.sepia.ceph.com to complete... 2023-11-08T16:13:21.827 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-08T16:13:21.828 DEBUG:teuthology.orchestra.run.smithi081:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-08T16:13:21.866 INFO:teuthology.orchestra.run.smithi081.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:13:21.866 INFO:teuthology.orchestra.run.smithi081.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:13:21.866 INFO:teuthology.orchestra.run.smithi081.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:13:21.867 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:13:21.867 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-11-08T16:13:21.867 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-11-08T16:13:21.867 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-11-08T16:13:21.867 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-11-08T16:13:21.868 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-08T16:13:21.868 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-08T16:13:21.868 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-08T16:13:21.868 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-08T16:13:21.869 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-08T16:13:21.869 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-08T16:13:21.869 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-08T16:13:21.869 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-08T16:13:21.869 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-08T16:13:21.870 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-08T16:13:21.870 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-08T16:13:21.870 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-08T16:13:21.870 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-08T16:13:21.870 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-08T16:13:21.871 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-08T16:13:21.871 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-08T16:13:21.871 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-08T16:13:21.871 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-08T16:13:21.871 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-08T16:13:21.872 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-08T16:13:21.872 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-11-08T16:13:21.872 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-11-08T16:13:21.872 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-11-08T16:13:21.872 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-08T16:13:21.873 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-11-08T16:13:21.873 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-08T16:13:21.873 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-08T16:13:21.873 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-11-08T16:13:21.873 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-11-08T16:13:21.874 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-11-08T16:13:21.874 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-11-08T16:13:21.874 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-11-08T16:13:21.874 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-11-08T16:13:21.874 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-11-08T16:13:21.875 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-11-08T16:13:21.875 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-08T16:13:21.875 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-11-08T16:13:21.875 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-11-08T16:13:21.875 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-11-08T16:13:21.876 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-11-08T16:13:21.876 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-11-08T16:13:21.876 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-11-08T16:13:21.876 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-11-08T16:13:21.876 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-11-08T16:13:21.877 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-11-08T16:13:21.877 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-11-08T16:13:21.877 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-11-08T16:13:21.877 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-11-08T16:13:21.877 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-11-08T16:13:21.878 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-11-08T16:13:21.878 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-11-08T16:13:21.878 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-11-08T16:13:21.878 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-11-08T16:13:21.878 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-11-08T16:13:21.879 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-11-08T16:13:21.879 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-11-08T16:13:21.879 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-11-08T16:13:21.879 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-11-08T16:13:21.879 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-11-08T16:13:21.880 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-11-08T16:13:21.880 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-11-08T16:13:21.880 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-11-08T16:13:21.880 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-11-08T16:13:21.880 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-11-08T16:13:21.881 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-11-08T16:13:21.881 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-11-08T16:13:21.881 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-11-08T16:13:21.881 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-11-08T16:13:21.881 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-11-08T16:13:21.882 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-11-08T16:13:21.882 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-11-08T16:13:21.882 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-11-08T16:13:21.882 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-11-08T16:13:21.882 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-11-08T16:13:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-11-08T16:13:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-11-08T16:13:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-11-08T16:13:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-11-08T16:13:21.883 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-11-08T16:13:21.884 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-11-08T16:13:21.884 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-08T16:13:21.884 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-11-08T16:13:21.884 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-11-08T16:13:21.884 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-11-08T16:13:21.885 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-11-08T16:13:21.885 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-11-08T16:13:21.885 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-11-08T16:13:21.885 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-11-08T16:13:21.885 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-11-08T16:13:21.885 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-11-08T16:13:21.886 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-11-08T16:13:21.886 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-11-08T16:13:21.886 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-11-08T16:13:21.886 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-11-08T16:13:21.887 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-11-08T16:13:21.887 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-11-08T16:13:21.887 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-11-08T16:13:21.887 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-11-08T16:13:21.887 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-11-08T16:13:21.888 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-11-08T16:13:21.888 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-11-08T16:13:21.888 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-11-08T16:13:21.888 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-11-08T16:13:21.888 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-11-08T16:13:21.889 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-11-08T16:13:21.889 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-08T16:13:21.889 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-08T16:13:21.889 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-11-08T16:13:21.889 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-11-08T16:13:21.889 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-11-08T16:13:21.890 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-11-08T16:13:21.890 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-11-08T16:13:21.890 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-11-08T16:13:21.890 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-11-08T16:13:21.891 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-11-08T16:13:21.891 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-11-08T16:13:21.891 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-11-08T16:13:21.891 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-11-08T16:13:21.891 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-11-08T16:13:21.892 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-11-08T16:13:21.892 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-08T16:13:21.892 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-08T16:13:21.892 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-11-08T16:13:21.892 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-11-08T16:13:21.893 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-11-08T16:13:21.893 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-11-08T16:13:21.893 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-11-08T16:13:21.893 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-11-08T16:13:21.893 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-11-08T16:13:21.894 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-11-08T16:13:21.894 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-11-08T16:13:21.894 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-11-08T16:13:21.894 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-11-08T16:13:21.894 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-11-08T16:13:21.895 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-11-08T16:13:21.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-11-08T16:13:21.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-11-08T16:13:21.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-11-08T16:13:21.928 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-11-08T16:13:21.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-11-08T16:13:21.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-11-08T16:13:21.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-11-08T16:13:21.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-11-08T16:13:21.929 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-11-08T16:13:21.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-11-08T16:13:21.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-11-08T16:13:21.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-11-08T16:13:21.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-11-08T16:13:21.930 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-11-08T16:13:21.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-08T16:13:21.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-08T16:13:21.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-11-08T16:13:21.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-11-08T16:13:21.931 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-11-08T16:13:21.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-11-08T16:13:21.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-11-08T16:13:21.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-11-08T16:13:21.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-11-08T16:13:21.932 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-11-08T16:13:21.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-11-08T16:13:21.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-11-08T16:13:21.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-11-08T16:13:21.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-08T16:13:21.933 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-11-08T16:13:21.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-11-08T16:13:21.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-11-08T16:13:21.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-11-08T16:13:21.934 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-11-08T16:13:21.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-11-08T16:13:21.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-11-08T16:13:21.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-08T16:13:21.935 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-11-08T16:13:21.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-11-08T16:13:21.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-11-08T16:13:21.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-11-08T16:13:21.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-11-08T16:13:21.936 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-11-08T16:13:21.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-08T16:13:21.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-08T16:13:21.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-08T16:13:21.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-11-08T16:13:21.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-08T16:13:21.937 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-11-08T16:13:21.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-08T16:13:21.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-08T16:13:21.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-08T16:13:21.938 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-08T16:13:21.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-08T16:13:21.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-08T16:13:21.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-11-08T16:13:21.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-11-08T16:13:21.939 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-11-08T16:13:21.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-11-08T16:13:21.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-11-08T16:13:21.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-11-08T16:13:21.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-11-08T16:13:21.940 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-08T16:13:21.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-08T16:13:21.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-08T16:13:21.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-08T16:13:21.941 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-08T16:13:21.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-11-08T16:13:21.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-11-08T16:13:21.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-11-08T16:13:21.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-08T16:13:21.942 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-08T16:13:21.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-08T16:13:21.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-08T16:13:21.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-08T16:13:21.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-08T16:13:21.943 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-11-08T16:13:21.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-11-08T16:13:21.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-08T16:13:21.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-11-08T16:13:21.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-11-08T16:13:21.944 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-11-08T16:13:21.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-11-08T16:13:21.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-08T16:13:21.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-08T16:13:21.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-08T16:13:21.945 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-11-08T16:13:21.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-11-08T16:13:21.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-11-08T16:13:21.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-08T16:13:21.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-11-08T16:13:21.946 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-11-08T16:13:21.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-11-08T16:13:21.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-08T16:13:21.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-11-08T16:13:21.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-11-08T16:13:21.947 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-08T16:13:21.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-11-08T16:13:21.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-08T16:13:21.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-11-08T16:13:21.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-11-08T16:13:21.948 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-11-08T16:13:21.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-11-08T16:13:21.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-11-08T16:13:21.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-11-08T16:13:21.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-11-08T16:13:21.949 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-11-08T16:13:21.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-11-08T16:13:21.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-11-08T16:13:21.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-11-08T16:13:21.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-11-08T16:13:21.950 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-11-08T16:13:21.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-11-08T16:13:21.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-11-08T16:13:21.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-11-08T16:13:21.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-08T16:13:21.951 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-11-08T16:13:21.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-11-08T16:13:21.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-08T16:13:21.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-11-08T16:13:21.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-11-08T16:13:21.952 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-11-08T16:13:21.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-11-08T16:13:21.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-11-08T16:13:21.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-11-08T16:13:21.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-11-08T16:13:21.953 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-11-08T16:13:21.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-11-08T16:13:21.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-11-08T16:13:21.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-11-08T16:13:21.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-08T16:13:21.954 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-08T16:13:21.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-11-08T16:13:21.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-11-08T16:13:21.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-11-08T16:13:21.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-11-08T16:13:21.955 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-11-08T16:13:21.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-11-08T16:13:21.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-11-08T16:13:21.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-11-08T16:13:21.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-08T16:13:21.956 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-08T16:13:21.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-11-08T16:13:21.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-11-08T16:13:21.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-11-08T16:13:21.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-11-08T16:13:21.957 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-11-08T16:13:21.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-11-08T16:13:21.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-11-08T16:13:21.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-11-08T16:13:21.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-11-08T16:13:21.958 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-08T16:13:21.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-11-08T16:13:21.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-08T16:13:21.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-08T16:13:21.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-08T16:13:21.959 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-08T16:13:21.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-08T16:13:21.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-08T16:13:21.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-08T16:13:21.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-08T16:13:21.960 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-08T16:13:21.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-11-08T16:13:21.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-11-08T16:13:21.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-11-08T16:13:21.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-08T16:13:21.961 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-11-08T16:13:21.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-11-08T16:13:21.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-11-08T16:13:21.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-11-08T16:13:21.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-11-08T16:13:21.962 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-08T16:13:21.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-11-08T16:13:21.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-11-08T16:13:21.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-11-08T16:13:21.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-11-08T16:13:21.963 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-11-08T16:13:21.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-11-08T16:13:21.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-11-08T16:13:21.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-11-08T16:13:21.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-11-08T16:13:21.964 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-11-08T16:13:21.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-11-08T16:13:21.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-11-08T16:13:21.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-11-08T16:13:21.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-11-08T16:13:21.965 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-11-08T16:13:21.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-11-08T16:13:21.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-11-08T16:13:21.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-11-08T16:13:21.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-11-08T16:13:21.966 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-08T16:13:21.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-11-08T16:13:21.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-11-08T16:13:21.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-11-08T16:13:21.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-11-08T16:13:21.967 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-11-08T16:13:21.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-11-08T16:13:21.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-11-08T16:13:21.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-11-08T16:13:21.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-11-08T16:13:21.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-11-08T16:13:21.968 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-11-08T16:13:21.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-11-08T16:13:21.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-11-08T16:13:21.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-11-08T16:13:21.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-11-08T16:13:21.969 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-11-08T16:13:21.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-11-08T16:13:21.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-11-08T16:13:21.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-11-08T16:13:21.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-08T16:13:21.970 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-11-08T16:13:21.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-11-08T16:13:21.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-08T16:13:21.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-11-08T16:13:21.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-11-08T16:13:21.971 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-11-08T16:13:21.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-11-08T16:13:21.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-11-08T16:13:21.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-11-08T16:13:21.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-11-08T16:13:21.972 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-11-08T16:13:21.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-11-08T16:13:21.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-11-08T16:13:21.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-11-08T16:13:21.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-11-08T16:13:21.973 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-11-08T16:13:21.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-11-08T16:13:21.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-08T16:13:21.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-11-08T16:13:21.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-11-08T16:13:21.974 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-11-08T16:13:21.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-11-08T16:13:21.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-11-08T16:13:21.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-11-08T16:13:21.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-11-08T16:13:21.975 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-11-08T16:13:21.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-11-08T16:13:21.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-11-08T16:13:21.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-11-08T16:13:21.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-11-08T16:13:21.976 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-11-08T16:13:21.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-11-08T16:13:21.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-11-08T16:13:21.977 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-11-08T16:13:21.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-11-08T16:13:21.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-11-08T16:13:21.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-11-08T16:13:21.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-11-08T16:13:21.978 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-11-08T16:13:21.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-11-08T16:13:21.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-11-08T16:13:21.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-11-08T16:13:21.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-11-08T16:13:21.979 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-11-08T16:13:21.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-11-08T16:13:21.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-11-08T16:13:21.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-11-08T16:13:21.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-11-08T16:13:21.980 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-11-08T16:13:21.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-11-08T16:13:21.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-11-08T16:13:21.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-11-08T16:13:21.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-11-08T16:13:21.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-11-08T16:13:21.981 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-11-08T16:13:21.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-11-08T16:13:21.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-11-08T16:13:21.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-11-08T16:13:21.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-11-08T16:13:21.982 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-11-08T16:13:21.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-11-08T16:13:21.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-11-08T16:13:21.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-11-08T16:13:21.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-11-08T16:13:21.983 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-11-08T16:13:21.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-11-08T16:13:21.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-08T16:13:21.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-08T16:13:21.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-08T16:13:21.984 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-08T16:13:21.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-08T16:13:21.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-08T16:13:21.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-08T16:13:21.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-08T16:13:21.985 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-08T16:13:21.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-08T16:13:21.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-08T16:13:21.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-08T16:13:21.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-11-08T16:13:21.986 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-11-08T16:13:21.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-11-08T16:13:21.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-08T16:13:21.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-11-08T16:13:21.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-11-08T16:13:21.987 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-11-08T16:13:21.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-11-08T16:13:21.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-11-08T16:13:21.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-11-08T16:13:21.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-11-08T16:13:21.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-11-08T16:13:21.988 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-11-08T16:13:21.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-11-08T16:13:21.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-11-08T16:13:21.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-11-08T16:13:21.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-11-08T16:13:21.989 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-11-08T16:13:21.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-11-08T16:13:21.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-11-08T16:13:21.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-11-08T16:13:21.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-11-08T16:13:21.990 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-11-08T16:13:21.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-11-08T16:13:21.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-11-08T16:13:21.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-11-08T16:13:21.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-11-08T16:13:21.991 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-11-08T16:13:21.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-11-08T16:13:21.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-11-08T16:13:21.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-11-08T16:13:21.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-08T16:13:21.992 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-08T16:13:21.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-11-08T16:13:21.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-08T16:13:21.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-08T16:13:21.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-08T16:13:21.993 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-08T16:13:21.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-08T16:13:21.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-08T16:13:21.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-08T16:13:21.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-08T16:13:21.994 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-08T16:13:21.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-08T16:13:21.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-08T16:13:21.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-08T16:13:21.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-08T16:13:21.995 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-08T16:13:21.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-08T16:13:21.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-08T16:13:21.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-08T16:13:21.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-08T16:13:21.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-08T16:13:21.996 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-08T16:13:21.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-08T16:13:21.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-08T16:13:21.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-08T16:13:21.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-11-08T16:13:21.997 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-11-08T16:13:21.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-08T16:13:21.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-11-08T16:13:21.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-11-08T16:13:21.998 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-11-08T16:13:21.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-11-08T16:13:21.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-08T16:13:21.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-11-08T16:13:21.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-08T16:13:21.999 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-08T16:13:22.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-08T16:13:22.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-08T16:13:22.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-08T16:13:22.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-11-08T16:13:22.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-11-08T16:13:22.000 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-08T16:13:22.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-11-08T16:13:22.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-08T16:13:22.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-11-08T16:13:22.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-08T16:13:22.001 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-11-08T16:13:22.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-08T16:13:22.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-11-08T16:13:22.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-11-08T16:13:22.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-08T16:13:22.002 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-11-08T16:13:22.003 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-08T16:13:22.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-11-08T16:13:22.005 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-08T16:13:22.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-11-08T16:13:22.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-08T16:13:22.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-11-08T16:13:22.006 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-08T16:13:22.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-11-08T16:13:22.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-11-08T16:13:22.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-08T16:13:22.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-08T16:13:22.007 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-11-08T16:13:22.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-11-08T16:13:22.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-11-08T16:13:22.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-08T16:13:22.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-11-08T16:13:22.008 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-11-08T16:13:22.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-11-08T16:13:22.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-11-08T16:13:22.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-08T16:13:22.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-11-08T16:13:22.009 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-08T16:13:22.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-08T16:13:22.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-08T16:13:22.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-11-08T16:13:22.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-08T16:13:22.010 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-11-08T16:13:22.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-11-08T16:13:22.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-11-08T16:13:22.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-08T16:13:22.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-11-08T16:13:22.011 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-11-08T16:13:22.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-11-08T16:13:22.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-11-08T16:13:22.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-08T16:13:22.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-11-08T16:13:22.012 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-11-08T16:13:22.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-11-08T16:13:22.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-11-08T16:13:22.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-11-08T16:13:22.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-11-08T16:13:22.013 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-08T16:13:22.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-08T16:13:22.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-08T16:13:22.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-08T16:13:22.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-08T16:13:22.014 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-08T16:13:22.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-08T16:13:22.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-08T16:13:22.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-11-08T16:13:22.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-11-08T16:13:22.015 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-11-08T16:13:22.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-11-08T16:13:22.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-11-08T16:13:22.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-11-08T16:13:22.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-11-08T16:13:22.016 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-11-08T16:13:22.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-11-08T16:13:22.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-11-08T16:13:22.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-11-08T16:13:22.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-11-08T16:13:22.017 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-11-08T16:13:22.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-11-08T16:13:22.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-11-08T16:13:22.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-11-08T16:13:22.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-11-08T16:13:22.018 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-11-08T16:13:22.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-11-08T16:13:22.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-11-08T16:13:22.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-11-08T16:13:22.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-11-08T16:13:22.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-08T16:13:22.019 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-11-08T16:13:22.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-11-08T16:13:22.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-11-08T16:13:22.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-11-08T16:13:22.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-11-08T16:13:22.020 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-11-08T16:13:22.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-11-08T16:13:22.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-11-08T16:13:22.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-11-08T16:13:22.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-11-08T16:13:22.021 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-11-08T16:13:22.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-11-08T16:13:22.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-11-08T16:13:22.022 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-11-08T16:13:22.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-11-08T16:13:22.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-11-08T16:13:22.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-11-08T16:13:22.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-11-08T16:13:22.023 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-11-08T16:13:22.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-11-08T16:13:22.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-11-08T16:13:22.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-11-08T16:13:22.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-11-08T16:13:22.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-08T16:13:22.024 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-11-08T16:13:22.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-11-08T16:13:22.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-11-08T16:13:22.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-11-08T16:13:22.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-11-08T16:13:22.025 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-11-08T16:13:22.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-11-08T16:13:22.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-08T16:13:22.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-08T16:13:22.026 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-11-08T16:13:22.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-11-08T16:13:22.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-11-08T16:13:22.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-11-08T16:13:22.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-11-08T16:13:22.027 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-11-08T16:13:22.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-08T16:13:22.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-08T16:13:22.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-11-08T16:13:22.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-11-08T16:13:22.028 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-11-08T16:13:22.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-11-08T16:13:22.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-11-08T16:13:22.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-08T16:13:22.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-11-08T16:13:22.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-08T16:13:22.029 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-08T16:13:22.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-08T16:13:22.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-11-08T16:13:22.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-11-08T16:13:22.030 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-11-08T16:13:22.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-11-08T16:13:22.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-11-08T16:13:22.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-08T16:13:22.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-11-08T16:13:22.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-11-08T16:13:22.031 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-11-08T16:13:22.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-11-08T16:13:22.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-11-08T16:13:22.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-08T16:13:22.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-08T16:13:22.032 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-08T16:13:22.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-11-08T16:13:22.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-08T16:13:22.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-11-08T16:13:22.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-11-08T16:13:22.033 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-11-08T16:13:22.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-11-08T16:13:22.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-08T16:13:22.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-08T16:13:22.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-08T16:13:22.034 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-08T16:13:22.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-11-08T16:13:22.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-08T16:13:22.035 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-08T16:13:22.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-08T16:13:22.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-08T16:13:22.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-11-08T16:13:22.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-08T16:13:22.036 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-11-08T16:13:22.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-08T16:13:22.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-11-08T16:13:22.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-08T16:13:22.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-11-08T16:13:22.037 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-08T16:13:22.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-11-08T16:13:22.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-08T16:13:22.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-08T16:13:22.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-08T16:13:22.038 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-11-08T16:13:22.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-11-08T16:13:22.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-08T16:13:22.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-08T16:13:22.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-08T16:13:22.039 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-08T16:13:22.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-08T16:13:22.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-11-08T16:13:22.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-08T16:13:22.040 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-08T16:13:22.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-11-08T16:13:22.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-08T16:13:22.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-08T16:13:22.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-08T16:13:22.041 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-08T16:13:22.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-08T16:13:22.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-11-08T16:13:22.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-08T16:13:22.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-08T16:13:22.042 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-11-08T16:13:22.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-11-08T16:13:22.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-08T16:13:22.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-11-08T16:13:22.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-08T16:13:22.043 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-08T16:13:22.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-08T16:13:22.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-08T16:13:22.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-08T16:13:22.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-08T16:13:22.044 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-08T16:13:22.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-08T16:13:22.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-08T16:13:22.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-08T16:13:22.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-11-08T16:13:22.045 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-11-08T16:13:22.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-08T16:13:22.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-11-08T16:13:22.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-11-08T16:13:22.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-08T16:13:22.046 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-08T16:13:22.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-08T16:13:22.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-08T16:13:22.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-08T16:13:22.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-08T16:13:22.047 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-11-08T16:13:22.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-08T16:13:22.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-08T16:13:22.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-11-08T16:13:22.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-11-08T16:13:22.048 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-11-08T16:13:22.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-08T16:13:22.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-11-08T16:13:22.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-11-08T16:13:22.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-11-08T16:13:22.049 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-08T16:13:22.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-11-08T16:13:22.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-08T16:13:22.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-11-08T16:13:22.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-08T16:13:22.050 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-11-08T16:13:22.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-08T16:13:22.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-11-08T16:13:22.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-08T16:13:22.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-11-08T16:13:22.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-08T16:13:22.051 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-11-08T16:13:22.052 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-08T16:13:22.052 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-11-08T16:13:22.052 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-08T16:13:22.052 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-08T16:13:22.052 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-08T16:13:22.053 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-08T16:13:22.053 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-08T16:13:22.053 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-11-08T16:13:22.053 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-11-08T16:13:22.053 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-08T16:13:22.054 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-08T16:13:22.054 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-08T16:13:22.054 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-11-08T16:13:22.054 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-08T16:13:22.054 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-08T16:13:22.055 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-08T16:13:22.055 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-08T16:13:22.055 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-08T16:13:22.055 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-08T16:13:22.055 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-11-08T16:13:22.056 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-08T16:13:22.056 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-11-08T16:13:22.056 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-08T16:13:22.056 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-08T16:13:22.056 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-11-08T16:13:22.057 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-08T16:13:22.057 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-08T16:13:22.057 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-11-08T16:13:22.057 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-08T16:13:22.057 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-11-08T16:13:22.058 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-08T16:13:22.058 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-11-08T16:13:22.058 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-08T16:13:22.058 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-08T16:13:22.058 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-11-08T16:13:22.059 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-08T16:13:22.059 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-08T16:13:22.059 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-08T16:13:22.059 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-08T16:13:22.059 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-08T16:13:22.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-11-08T16:13:22.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-08T16:13:22.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-11-08T16:13:22.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-08T16:13:22.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-08T16:13:22.060 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-08T16:13:22.061 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-08T16:13:22.061 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-11-08T16:13:22.061 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-08T16:13:22.061 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-11-08T16:13:22.062 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-08T16:13:22.062 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-11-08T16:13:22.062 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-08T16:13:22.062 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-11-08T16:13:22.062 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-08T16:13:22.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-08T16:13:22.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-11-08T16:13:22.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-08T16:13:22.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-08T16:13:22.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-08T16:13:22.063 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-08T16:13:22.064 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-11-08T16:13:22.064 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-08T16:13:22.064 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-08T16:13:22.064 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-08T16:13:22.064 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-11-08T16:13:22.065 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-08T16:13:22.065 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-08T16:13:22.065 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-08T16:13:22.065 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-08T16:13:22.065 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-11-08T16:13:22.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-08T16:13:22.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-11-08T16:13:22.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-08T16:13:22.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-11-08T16:13:22.066 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-08T16:13:22.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-08T16:13:22.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-11-08T16:13:22.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-08T16:13:22.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-08T16:13:22.067 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-11-08T16:13:22.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-08T16:13:22.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-11-08T16:13:22.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-11-08T16:13:22.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-08T16:13:22.068 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-11-08T16:13:22.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-08T16:13:22.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-11-08T16:13:22.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-11-08T16:13:22.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-08T16:13:22.069 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-11-08T16:13:22.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-11-08T16:13:22.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-11-08T16:13:22.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-11-08T16:13:22.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-08T16:13:22.070 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-08T16:13:22.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-08T16:13:22.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-11-08T16:13:22.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-11-08T16:13:22.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-11-08T16:13:22.071 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-11-08T16:13:22.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-11-08T16:13:22.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-08T16:13:22.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-08T16:13:22.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-11-08T16:13:22.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-11-08T16:13:22.072 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-08T16:13:22.073 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-08T16:13:22.073 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-08T16:13:22.073 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-11-08T16:13:22.073 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-11-08T16:13:22.073 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-11-08T16:13:22.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-11-08T16:13:22.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-11-08T16:13:22.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-11-08T16:13:22.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-11-08T16:13:22.074 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-11-08T16:13:22.075 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-11-08T16:13:22.075 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-11-08T16:13:22.075 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-11-08T16:13:22.075 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-11-08T16:13:22.075 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-11-08T16:13:22.076 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-11-08T16:13:22.076 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-11-08T16:13:22.076 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-11-08T16:13:22.076 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-11-08T16:13:22.076 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-11-08T16:13:22.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-08T16:13:22.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-11-08T16:13:22.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-08T16:13:22.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-08T16:13:22.077 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-08T16:13:22.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-08T16:13:22.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-11-08T16:13:22.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-11-08T16:13:22.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-11-08T16:13:22.078 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-11-08T16:13:22.079 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-11-08T16:13:22.079 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-11-08T16:13:22.079 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-11-08T16:13:22.079 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-11-08T16:13:22.079 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-11-08T16:13:22.079 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-11-08T16:13:22.080 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-08T16:13:22.080 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-08T16:13:22.080 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-08T16:13:22.080 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-11-08T16:13:22.080 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-08T16:13:22.081 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-11-08T16:13:22.081 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-11-08T16:13:22.081 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-11-08T16:13:22.081 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-11-08T16:13:22.081 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-08T16:13:22.082 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-11-08T16:13:22.082 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-11-08T16:13:22.082 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-11-08T16:13:22.082 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-08T16:13:22.082 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-11-08T16:13:22.083 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-11-08T16:13:22.083 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-11-08T16:13:22.083 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-11-08T16:13:22.083 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-11-08T16:13:22.083 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-11-08T16:13:22.084 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-11-08T16:13:22.084 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-11-08T16:13:22.084 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-11-08T16:13:22.084 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-11-08T16:13:22.084 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-11-08T16:13:22.084 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-11-08T16:13:22.085 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-11-08T16:13:22.085 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-11-08T16:13:22.085 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-08T16:13:22.085 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-11-08T16:13:22.085 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-08T16:13:22.086 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-11-08T16:13:22.086 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-11-08T16:13:22.086 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-11-08T16:13:22.086 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-11-08T16:13:22.086 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-08T16:13:22.087 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-11-08T16:13:22.087 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-11-08T16:13:22.087 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-08T16:13:22.087 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-11-08T16:13:22.087 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-11-08T16:13:22.088 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-08T16:13:22.088 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-11-08T16:13:22.088 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-08T16:13:22.088 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-08T16:13:22.088 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-08T16:13:22.088 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-08T16:13:22.089 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-08T16:13:22.089 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-08T16:13:22.089 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-08T16:13:22.089 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-08T16:13:22.089 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-08T16:13:22.090 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-11-08T16:13:22.090 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-11-08T16:13:22.090 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-08T16:13:22.090 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-11-08T16:13:22.090 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-08T16:13:22.091 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-08T16:13:22.091 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-08T16:13:22.091 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-08T16:13:22.091 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-08T16:13:22.091 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-08T16:13:22.092 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-08T16:13:22.092 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-08T16:13:22.092 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-08T16:13:22.092 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-11-08T16:13:22.092 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-11-08T16:13:22.093 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-08T16:13:22.093 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-08T16:13:22.093 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-11-08T16:13:22.093 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-11-08T16:13:22.093 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-08T16:13:22.094 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-08T16:13:22.094 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-11-08T16:13:22.094 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-11-08T16:13:22.094 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-11-08T16:13:22.094 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-11-08T16:13:22.095 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-11-08T16:13:22.095 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-11-08T16:13:22.095 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-08T16:13:22.095 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-08T16:13:22.095 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-08T16:13:22.096 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-08T16:13:22.096 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-08T16:13:22.096 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-08T16:13:22.096 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-08T16:13:22.096 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-08T16:13:22.097 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-08T16:13:22.097 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-08T16:13:22.097 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-11-08T16:13:22.097 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-08T16:13:22.097 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-08T16:13:22.098 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-08T16:13:22.098 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-08T16:13:22.098 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-08T16:13:22.098 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-08T16:13:22.098 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-11-08T16:13:22.098 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-08T16:13:22.099 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-08T16:13:22.099 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-08T16:13:22.099 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-08T16:13:22.099 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-08T16:13:22.099 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-08T16:13:22.100 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-08T16:13:22.100 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-08T16:13:22.100 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-08T16:13:22.100 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-08T16:13:22.100 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-08T16:13:22.101 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-08T16:13:22.101 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-11-08T16:13:22.101 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-11-08T16:13:22.101 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-11-08T16:13:22.101 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-11-08T16:13:22.102 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-11-08T16:13:22.102 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-11-08T16:13:22.102 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-11-08T16:13:22.102 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-11-08T16:13:22.102 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-08T16:13:22.103 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-11-08T16:13:22.103 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-11-08T16:13:22.103 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-08T16:13:22.103 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-08T16:13:22.103 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-11-08T16:13:22.103 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-11-08T16:13:22.104 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-08T16:13:22.104 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-11-08T16:13:22.104 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-11-08T16:13:22.104 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-08T16:13:22.104 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-11-08T16:13:22.105 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-11-08T16:13:22.105 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-11-08T16:13:22.105 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-08T16:13:22.105 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-11-08T16:13:22.105 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-08T16:13:22.106 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-11-08T16:13:22.106 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-11-08T16:13:22.106 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-08T16:13:22.106 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-11-08T16:13:22.106 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-08T16:13:22.106 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-08T16:13:22.107 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-08T16:13:22.107 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-08T16:13:22.107 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-08T16:13:22.107 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-11-08T16:13:22.107 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-11-08T16:13:22.108 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-08T16:13:22.108 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-08T16:13:22.108 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-08T16:13:22.108 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-08T16:13:22.108 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-11-08T16:13:22.109 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-11-08T16:13:22.109 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-08T16:13:22.109 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-11-08T16:13:22.109 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-11-08T16:13:22.109 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-08T16:13:22.110 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-08T16:13:22.110 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-11-08T16:13:22.110 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-11-08T16:13:22.110 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-11-08T16:13:22.110 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-11-08T16:13:22.110 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-11-08T16:13:22.111 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-11-08T16:13:22.111 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-11-08T16:13:22.111 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-11-08T16:13:22.111 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-11-08T16:13:22.111 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-11-08T16:13:22.112 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-08T16:13:22.112 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-08T16:13:22.112 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-11-08T16:13:22.112 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-11-08T16:13:22.112 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-11-08T16:13:22.113 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-08T16:13:22.113 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-11-08T16:13:22.113 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-11-08T16:13:22.113 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-11-08T16:13:22.113 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-08T16:13:22.113 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-11-08T16:13:22.114 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-11-08T16:13:22.114 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-08T16:13:22.114 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-08T16:13:22.114 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-08T16:13:22.114 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-11-08T16:13:22.115 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-08T16:13:22.115 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-11-08T16:13:22.115 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-08T16:13:22.115 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-11-08T16:13:22.115 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-11-08T16:13:22.116 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-08T16:13:22.116 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-11-08T16:13:22.116 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-11-08T16:13:22.116 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-11-08T16:13:22.116 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-08T16:13:22.116 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-08T16:13:22.117 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-11-08T16:13:22.117 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-08T16:13:22.117 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-11-08T16:13:22.117 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-11-08T16:13:22.117 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-08T16:13:22.118 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-08T16:13:22.118 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-08T16:13:22.118 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-08T16:13:22.118 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-08T16:13:22.118 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-11-08T16:13:22.119 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-11-08T16:13:22.119 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-11-08T16:13:22.119 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-11-08T16:13:22.119 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-08T16:13:22.119 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-11-08T16:13:22.120 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-08T16:13:22.120 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-11-08T16:13:22.120 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-11-08T16:13:22.120 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-11-08T16:13:22.120 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-11-08T16:13:22.121 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-11-08T16:13:22.121 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-08T16:13:22.121 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-11-08T16:13:22.121 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-08T16:13:22.121 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-11-08T16:13:22.121 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-11-08T16:13:22.122 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-11-08T16:13:22.122 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-11-08T16:13:22.122 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-11-08T16:13:22.122 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-11-08T16:13:22.123 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-08T16:13:22.123 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-11-08T16:13:22.123 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-11-08T16:13:22.123 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-11-08T16:13:22.123 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-08T16:13:22.124 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-08T16:13:22.124 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-08T16:13:22.124 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-08T16:13:22.124 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-08T16:13:22.124 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-08T16:13:22.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-08T16:13:22.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-08T16:13:22.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-08T16:13:22.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-08T16:13:22.125 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-08T16:13:22.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-08T16:13:22.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-08T16:13:22.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-08T16:13:22.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-08T16:13:22.126 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-11-08T16:13:22.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-11-08T16:13:22.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-08T16:13:22.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-11-08T16:13:22.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-11-08T16:13:22.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-11-08T16:13:22.127 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-11-08T16:13:22.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-11-08T16:13:22.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-08T16:13:22.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-11-08T16:13:22.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-11-08T16:13:22.128 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-08T16:13:22.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-11-08T16:13:22.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-11-08T16:13:22.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-08T16:13:22.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-11-08T16:13:22.129 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-08T16:13:22.130 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-11-08T16:13:22.130 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-11-08T16:13:22.130 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-11-08T16:13:22.130 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-11-08T16:13:22.130 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-11-08T16:13:22.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-11-08T16:13:22.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-11-08T16:13:22.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-11-08T16:13:22.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-11-08T16:13:22.131 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-08T16:13:22.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-11-08T16:13:22.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-11-08T16:13:22.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-11-08T16:13:22.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-08T16:13:22.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-11-08T16:13:22.132 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-11-08T16:13:22.133 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-11-08T16:13:22.133 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-11-08T16:13:22.133 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-11-08T16:13:22.133 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-11-08T16:13:22.133 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-11-08T16:13:22.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-11-08T16:13:22.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-11-08T16:13:22.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-11-08T16:13:22.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-11-08T16:13:22.134 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-08T16:13:22.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-11-08T16:13:22.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-11-08T16:13:22.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-11-08T16:13:22.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-08T16:13:22.135 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-08T16:13:22.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-11-08T16:13:22.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-11-08T16:13:22.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-08T16:13:22.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-11-08T16:13:22.136 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-11-08T16:13:22.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-11-08T16:13:22.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-11-08T16:13:22.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-11-08T16:13:22.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-11-08T16:13:22.137 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-11-08T16:13:22.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-08T16:13:22.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-11-08T16:13:22.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-11-08T16:13:22.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-11-08T16:13:22.138 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-08T16:13:22.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-08T16:13:22.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-11-08T16:13:22.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-08T16:13:22.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-08T16:13:22.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-08T16:13:22.139 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-11-08T16:13:22.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-11-08T16:13:22.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-11-08T16:13:22.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-11-08T16:13:22.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-08T16:13:22.140 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-11-08T16:13:22.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-08T16:13:22.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-08T16:13:22.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-08T16:13:22.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-11-08T16:13:22.141 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-11-08T16:13:22.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-11-08T16:13:22.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-11-08T16:13:22.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-11-08T16:13:22.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-11-08T16:13:22.142 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-11-08T16:13:22.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-11-08T16:13:22.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-11-08T16:13:22.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-11-08T16:13:22.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-11-08T16:13:22.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-11-08T16:13:22.143 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-11-08T16:13:22.144 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-11-08T16:13:22.144 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-08T16:13:22.144 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-11-08T16:13:22.144 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-11-08T16:13:22.144 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-08T16:13:22.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-08T16:13:22.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-11-08T16:13:22.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-08T16:13:22.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-11-08T16:13:22.145 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-11-08T16:13:22.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-08T16:13:22.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-11-08T16:13:22.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-11-08T16:13:22.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-08T16:13:22.146 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-08T16:13:22.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-08T16:13:22.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-08T16:13:22.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-08T16:13:22.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-08T16:13:22.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-08T16:13:22.147 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-11-08T16:13:22.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-08T16:13:22.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-08T16:13:22.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-08T16:13:22.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-08T16:13:22.148 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-08T16:13:22.149 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-11-08T16:13:22.149 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-11-08T16:13:22.149 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-08T16:13:22.149 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-08T16:13:22.149 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-08T16:13:22.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-11-08T16:13:22.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-11-08T16:13:22.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-08T16:13:22.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-08T16:13:22.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-11-08T16:13:22.150 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-08T16:13:22.151 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-11-08T16:13:22.151 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-11-08T16:13:22.151 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-08T16:13:22.151 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-11-08T16:13:22.151 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-11-08T16:13:22.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-11-08T16:13:22.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-11-08T16:13:22.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-11-08T16:13:22.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-11-08T16:13:22.152 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-08T16:13:22.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-11-08T16:13:22.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-11-08T16:13:22.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-11-08T16:13:22.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-11-08T16:13:22.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-11-08T16:13:22.153 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-08T16:13:22.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-08T16:13:22.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-11-08T16:13:22.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-11-08T16:13:22.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-11-08T16:13:22.154 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-11-08T16:13:22.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-11-08T16:13:22.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-11-08T16:13:22.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-08T16:13:22.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-08T16:13:22.155 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-08T16:13:22.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-08T16:13:22.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-08T16:13:22.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-08T16:13:22.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-08T16:13:22.156 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-08T16:13:22.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-08T16:13:22.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-08T16:13:22.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-08T16:13:22.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-11-08T16:13:22.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-11-08T16:13:22.157 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-08T16:13:22.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-08T16:13:22.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-08T16:13:22.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-11-08T16:13:22.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-11-08T16:13:22.158 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-08T16:13:22.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-08T16:13:22.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-11-08T16:13:22.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-11-08T16:13:22.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-08T16:13:22.159 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-08T16:13:22.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-11-08T16:13:22.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-11-08T16:13:22.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-11-08T16:13:22.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-08T16:13:22.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-08T16:13:22.160 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-08T16:13:22.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-08T16:13:22.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-11-08T16:13:22.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-08T16:13:22.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-11-08T16:13:22.161 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-08T16:13:22.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-08T16:13:22.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-08T16:13:22.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-11-08T16:13:22.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-11-08T16:13:22.162 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-11-08T16:13:22.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-08T16:13:22.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-11-08T16:13:22.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-11-08T16:13:22.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-11-08T16:13:22.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-11-08T16:13:22.163 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-08T16:13:22.164 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-11-08T16:13:22.164 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-11-08T16:13:22.164 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-08T16:13:22.164 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-11-08T16:13:22.164 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-11-08T16:13:22.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-11-08T16:13:22.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-11-08T16:13:22.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-11-08T16:13:22.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-11-08T16:13:22.165 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-11-08T16:13:22.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-11-08T16:13:22.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-11-08T16:13:22.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-11-08T16:13:22.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-11-08T16:13:22.166 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-11-08T16:13:22.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-11-08T16:13:22.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-11-08T16:13:22.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-11-08T16:13:22.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-11-08T16:13:22.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-11-08T16:13:22.167 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-11-08T16:13:22.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-11-08T16:13:22.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-11-08T16:13:22.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-11-08T16:13:22.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-08T16:13:22.168 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-11-08T16:13:22.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-11-08T16:13:22.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-11-08T16:13:22.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-11-08T16:13:22.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-11-08T16:13:22.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-11-08T16:13:22.169 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-11-08T16:13:22.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-11-08T16:13:22.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-11-08T16:13:22.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-11-08T16:13:22.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-11-08T16:13:22.170 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-11-08T16:13:22.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-11-08T16:13:22.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-11-08T16:13:22.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-11-08T16:13:22.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-11-08T16:13:22.171 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-11-08T16:13:22.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-11-08T16:13:22.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-11-08T16:13:22.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-11-08T16:13:22.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-11-08T16:13:22.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-11-08T16:13:22.172 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-11-08T16:13:22.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-11-08T16:13:22.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-11-08T16:13:22.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-11-08T16:13:22.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-11-08T16:13:22.173 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-11-08T16:13:22.174 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-11-08T16:13:22.174 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-08T16:13:22.174 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-11-08T16:13:22.174 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-08T16:13:22.174 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-11-08T16:13:22.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-08T16:13:22.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-11-08T16:13:22.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-11-08T16:13:22.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-11-08T16:13:22.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-11-08T16:13:22.175 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-11-08T16:13:22.176 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-11-08T16:13:22.176 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-11-08T16:13:22.176 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-11-08T16:13:22.176 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-11-08T16:13:22.176 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-11-08T16:13:22.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-11-08T16:13:22.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-11-08T16:13:22.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-11-08T16:13:22.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-11-08T16:13:22.177 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-11-08T16:13:22.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-11-08T16:13:22.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-11-08T16:13:22.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-11-08T16:13:22.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-11-08T16:13:22.178 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-11-08T16:13:22.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-11-08T16:13:22.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-11-08T16:13:22.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-11-08T16:13:22.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-11-08T16:13:22.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-11-08T16:13:22.179 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-11-08T16:13:22.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-11-08T16:13:22.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-11-08T16:13:22.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-11-08T16:13:22.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-11-08T16:13:22.180 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-11-08T16:13:22.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-11-08T16:13:22.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-11-08T16:13:22.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-11-08T16:13:22.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-11-08T16:13:22.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-11-08T16:13:22.181 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-11-08T16:13:22.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-11-08T16:13:22.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-11-08T16:13:22.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-11-08T16:13:22.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-11-08T16:13:22.182 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-11-08T16:13:22.183 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-11-08T16:13:22.183 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-11-08T16:13:22.183 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-11-08T16:13:22.183 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-11-08T16:13:22.183 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-11-08T16:13:22.184 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-11-08T16:13:22.184 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-11-08T16:13:22.184 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-11-08T16:13:22.184 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-11-08T16:13:22.184 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-11-08T16:13:22.185 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-11-08T16:13:22.185 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-11-08T16:13:22.185 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-11-08T16:13:22.185 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-11-08T16:13:22.185 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-11-08T16:13:22.185 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-11-08T16:13:22.186 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-11-08T16:13:22.186 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-08T16:13:22.186 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-11-08T16:13:22.186 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-11-08T16:13:22.186 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-11-08T16:13:22.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-08T16:13:22.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-11-08T16:13:22.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-08T16:13:22.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-11-08T16:13:22.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-08T16:13:22.187 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-11-08T16:13:22.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-08T16:13:22.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-08T16:13:22.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-08T16:13:22.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-11-08T16:13:22.188 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-11-08T16:13:22.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-11-08T16:13:22.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-11-08T16:13:22.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-11-08T16:13:22.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-11-08T16:13:22.189 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-11-08T16:13:22.190 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-11-08T16:13:22.190 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-11-08T16:13:22.190 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-11-08T16:13:22.190 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-11-08T16:13:22.190 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-11-08T16:13:22.191 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-11-08T16:13:22.191 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-11-08T16:13:22.191 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-11-08T16:13:22.191 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-11-08T16:13:22.191 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-11-08T16:13:22.191 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-11-08T16:13:22.192 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-11-08T16:13:22.192 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-11-08T16:13:22.192 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-11-08T16:13:22.192 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-11-08T16:13:22.192 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-11-08T16:13:22.193 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-11-08T16:13:22.193 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-11-08T16:13:22.193 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-11-08T16:13:22.193 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-11-08T16:13:22.193 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-11-08T16:13:22.194 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-11-08T16:13:22.194 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-08T16:13:22.194 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-11-08T16:13:22.194 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-11-08T16:13:22.194 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-11-08T16:13:22.195 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-11-08T16:13:22.195 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-11-08T16:13:22.195 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-11-08T16:13:22.195 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-11-08T16:13:22.195 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-11-08T16:13:22.195 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-11-08T16:13:22.196 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-11-08T16:13:22.196 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-11-08T16:13:22.196 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-11-08T16:13:22.196 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-11-08T16:13:22.196 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-11-08T16:13:22.197 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-11-08T16:13:22.197 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-11-08T16:13:22.197 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-11-08T16:13:22.197 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-08T16:13:22.197 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-11-08T16:13:22.197 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-11-08T16:13:22.198 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-11-08T16:13:22.198 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-11-08T16:13:22.198 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-11-08T16:13:22.198 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-11-08T16:13:22.198 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-11-08T16:13:22.199 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-11-08T16:13:22.199 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-11-08T16:13:22.199 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-11-08T16:13:22.199 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-11-08T16:13:22.199 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-11-08T16:13:22.200 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-11-08T16:13:22.200 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-08T16:13:22.200 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-11-08T16:13:22.200 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-11-08T16:13:22.200 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-11-08T16:13:22.201 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-11-08T16:13:22.201 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-11-08T16:13:22.201 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-11-08T16:13:22.201 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-11-08T16:13:22.201 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-11-08T16:13:22.201 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-11-08T16:13:22.202 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-11-08T16:13:22.202 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-11-08T16:13:22.202 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-11-08T16:13:22.202 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-11-08T16:13:22.202 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-11-08T16:13:22.203 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-11-08T16:13:22.203 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-11-08T16:13:22.203 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-11-08T16:13:22.203 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-11-08T16:13:22.203 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-11-08T16:13:22.204 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-11-08T16:13:22.204 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-11-08T16:13:22.204 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-11-08T16:13:22.204 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-08T16:13:22.204 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-08T16:13:22.204 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-08T16:13:22.205 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-08T16:13:22.205 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-08T16:13:22.205 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-08T16:13:22.205 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-08T16:13:22.205 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-08T16:13:22.206 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-08T16:13:22.206 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-08T16:13:22.206 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-08T16:13:22.206 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-08T16:13:22.206 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-08T16:13:22.207 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-08T16:13:22.207 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-08T16:13:22.207 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-08T16:13:22.207 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-08T16:13:22.207 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-08T16:13:22.207 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-08T16:13:22.209 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-08T16:13:22.209 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-11-08T16:13:22.210 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-08T16:13:22.210 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-08T16:13:22.210 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-08T16:13:22.210 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-11-08T16:13:22.210 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-11-08T16:13:22.210 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-11-08T16:13:22.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-11-08T16:13:22.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-11-08T16:13:22.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-11-08T16:13:22.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-11-08T16:13:22.211 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-11-08T16:13:22.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-11-08T16:13:22.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-11-08T16:13:22.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-11-08T16:13:22.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-11-08T16:13:22.212 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-11-08T16:13:22.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-11-08T16:13:22.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-11-08T16:13:22.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-11-08T16:13:22.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-11-08T16:13:22.213 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-11-08T16:13:22.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-11-08T16:13:22.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-11-08T16:13:22.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-08T16:13:22.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-11-08T16:13:22.214 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-08T16:13:22.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-11-08T16:13:22.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-11-08T16:13:22.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-11-08T16:13:22.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-11-08T16:13:22.215 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-11-08T16:13:22.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-11-08T16:13:22.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-11-08T16:13:22.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-11-08T16:13:22.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-11-08T16:13:22.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-11-08T16:13:22.216 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-11-08T16:13:22.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-11-08T16:13:22.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-08T16:13:22.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-08T16:13:22.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-08T16:13:22.217 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-08T16:13:22.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-08T16:13:22.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-08T16:13:22.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-08T16:13:22.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-08T16:13:22.218 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-08T16:13:22.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-08T16:13:22.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-08T16:13:22.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-08T16:13:22.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-08T16:13:22.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-08T16:13:22.219 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-08T16:13:22.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-08T16:13:22.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-08T16:13:22.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-08T16:13:22.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-08T16:13:22.220 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-08T16:13:22.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-08T16:13:22.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-11-08T16:13:22.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-11-08T16:13:22.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-11-08T16:13:22.221 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-11-08T16:13:22.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-11-08T16:13:22.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-11-08T16:13:22.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-11-08T16:13:22.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-11-08T16:13:22.222 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-11-08T16:13:22.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-11-08T16:13:22.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-11-08T16:13:22.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-11-08T16:13:22.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-11-08T16:13:22.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-11-08T16:13:22.223 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-11-08T16:13:22.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-11-08T16:13:22.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-08T16:13:22.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-11-08T16:13:22.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-11-08T16:13:22.224 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-11-08T16:13:22.225 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-11-08T16:13:22.225 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-11-08T16:13:22.225 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-08T16:13:22.225 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-11-08T16:13:22.225 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-11-08T16:13:22.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-11-08T16:13:22.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-08T16:13:22.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-08T16:13:22.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-08T16:13:22.226 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-08T16:13:22.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-08T16:13:22.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-08T16:13:22.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-08T16:13:22.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-08T16:13:22.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-08T16:13:22.227 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-08T16:13:22.228 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-08T16:13:22.228 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-08T16:13:22.228 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-08T16:13:22.228 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-08T16:13:22.228 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-08T16:13:22.229 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-08T16:13:22.229 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-08T16:13:22.229 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-08T16:13:22.229 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-08T16:13:22.229 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-08T16:13:22.229 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-08T16:13:22.230 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-08T16:13:22.230 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-08T16:13:22.230 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-08T16:13:22.230 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-11-08T16:13:22.230 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-11-08T16:13:22.231 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-08T16:13:22.231 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-11-08T16:13:22.231 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-11-08T16:13:22.231 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-11-08T16:13:22.231 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-11-08T16:13:22.232 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-08T16:13:22.232 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-11-08T16:13:22.232 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-11-08T16:13:22.232 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-11-08T16:13:22.232 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-11-08T16:13:22.232 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-11-08T16:13:22.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-11-08T16:13:22.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-11-08T16:13:22.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-11-08T16:13:22.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-11-08T16:13:22.233 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-11-08T16:13:22.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-11-08T16:13:22.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-11-08T16:13:22.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-11-08T16:13:22.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-11-08T16:13:22.234 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-08T16:13:22.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-08T16:13:22.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-08T16:13:22.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-08T16:13:22.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-08T16:13:22.235 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-08T16:13:22.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-08T16:13:22.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-08T16:13:22.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-08T16:13:22.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-08T16:13:22.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-08T16:13:22.236 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-08T16:13:22.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-08T16:13:22.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-08T16:13:22.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-08T16:13:22.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-11-08T16:13:22.237 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-08T16:13:22.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-08T16:13:22.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-08T16:13:22.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-08T16:13:22.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-08T16:13:22.238 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-08T16:13:22.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-08T16:13:22.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-08T16:13:22.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-08T16:13:22.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-08T16:13:22.239 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-08T16:13:22.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-08T16:13:22.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-08T16:13:22.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-08T16:13:22.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-11-08T16:13:22.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-11-08T16:13:22.240 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-08T16:13:22.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-08T16:13:22.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-08T16:13:22.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-08T16:13:22.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-08T16:13:22.241 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-08T16:13:22.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-08T16:13:22.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-08T16:13:22.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-08T16:13:22.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-08T16:13:22.242 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-08T16:13:22.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-08T16:13:22.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-08T16:13:22.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-11-08T16:13:22.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-08T16:13:22.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-11-08T16:13:22.243 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-11-08T16:13:22.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-08T16:13:22.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-08T16:13:22.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-11-08T16:13:22.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-11-08T16:13:22.244 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-08T16:13:22.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-08T16:13:22.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-11-08T16:13:22.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-11-08T16:13:22.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-08T16:13:22.245 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-11-08T16:13:22.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-08T16:13:22.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-08T16:13:22.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-11-08T16:13:22.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-11-08T16:13:22.246 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-11-08T16:13:22.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-08T16:13:22.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-11-08T16:13:22.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-08T16:13:22.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-11-08T16:13:22.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-11-08T16:13:22.247 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-11-08T16:13:22.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-11-08T16:13:22.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-11-08T16:13:22.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-11-08T16:13:22.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-11-08T16:13:22.248 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-11-08T16:13:22.249 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-11-08T16:13:22.249 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-11-08T16:13:22.249 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-11-08T16:13:22.249 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-11-08T16:13:22.249 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-11-08T16:13:22.250 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-11-08T16:13:22.250 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-08T16:13:22.250 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-08T16:13:22.250 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-08T16:13:22.250 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-11-08T16:13:22.251 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-11-08T16:13:22.251 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-11-08T16:13:22.251 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-11-08T16:13:22.251 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-08T16:13:22.251 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-11-08T16:13:22.251 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-11-08T16:13:22.252 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-08T16:13:22.252 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-11-08T16:13:22.252 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-11-08T16:13:22.252 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-08T16:13:22.252 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-11-08T16:13:22.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-08T16:13:22.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-11-08T16:13:22.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-08T16:13:22.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-08T16:13:22.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-11-08T16:13:22.253 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-11-08T16:13:22.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-11-08T16:13:22.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-11-08T16:13:22.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-11-08T16:13:22.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-11-08T16:13:22.254 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-11-08T16:13:22.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-11-08T16:13:22.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-11-08T16:13:22.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-11-08T16:13:22.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-11-08T16:13:22.255 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-11-08T16:13:22.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-11-08T16:13:22.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-11-08T16:13:22.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-11-08T16:13:22.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-11-08T16:13:22.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-11-08T16:13:22.256 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-11-08T16:13:22.257 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-11-08T16:13:22.257 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-11-08T16:13:22.257 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-11-08T16:13:22.257 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-11-08T16:13:22.257 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-11-08T16:13:22.258 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-11-08T16:13:22.258 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-11-08T16:13:22.258 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-11-08T16:13:22.258 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-11-08T16:13:22.258 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-11-08T16:13:22.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-11-08T16:13:22.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-11-08T16:13:22.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-11-08T16:13:22.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-11-08T16:13:22.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-11-08T16:13:22.259 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-11-08T16:13:22.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-11-08T16:13:22.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-11-08T16:13:22.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-11-08T16:13:22.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-11-08T16:13:22.260 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-11-08T16:13:22.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-11-08T16:13:22.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-11-08T16:13:22.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-11-08T16:13:22.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-11-08T16:13:22.261 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-11-08T16:13:22.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-11-08T16:13:22.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-11-08T16:13:22.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-11-08T16:13:22.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-11-08T16:13:22.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-11-08T16:13:22.262 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-11-08T16:13:22.263 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-11-08T16:13:22.263 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-11-08T16:13:22.265 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-11-08T16:13:22.265 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-11-08T16:13:22.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-11-08T16:13:22.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-11-08T16:13:22.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-11-08T16:13:22.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-11-08T16:13:22.266 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-11-08T16:13:22.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-11-08T16:13:22.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-11-08T16:13:22.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-11-08T16:13:22.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-11-08T16:13:22.267 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-11-08T16:13:22.268 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-11-08T16:13:22.268 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-11-08T16:13:22.268 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-11-08T16:13:22.268 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-11-08T16:13:22.268 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-11-08T16:13:22.269 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-11-08T16:13:22.269 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-11-08T16:13:22.269 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-11-08T16:13:22.269 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-11-08T16:13:22.269 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-11-08T16:13:22.269 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-11-08T16:13:22.270 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-11-08T16:13:22.270 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-11-08T16:13:22.270 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-11-08T16:13:22.270 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-11-08T16:13:22.270 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-11-08T16:13:22.271 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-11-08T16:13:22.271 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-11-08T16:13:22.271 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-11-08T16:13:22.271 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-11-08T16:13:22.271 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-11-08T16:13:22.272 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-11-08T16:13:22.272 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-11-08T16:13:22.272 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-11-08T16:13:22.272 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-11-08T16:13:22.272 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-11-08T16:13:22.273 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-11-08T16:13:22.273 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-11-08T16:13:22.273 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-11-08T16:13:22.273 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-11-08T16:13:22.273 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-11-08T16:13:22.273 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-11-08T16:13:22.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-11-08T16:13:22.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-11-08T16:13:22.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-11-08T16:13:22.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-11-08T16:13:22.274 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-11-08T16:13:22.275 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-11-08T16:13:22.275 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-11-08T16:13:22.275 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-11-08T16:13:22.275 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-11-08T16:13:22.275 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-11-08T16:13:22.276 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-11-08T16:13:22.276 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-11-08T16:13:22.276 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-11-08T16:13:22.276 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-11-08T16:13:22.276 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-11-08T16:13:22.277 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-11-08T16:13:22.277 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-11-08T16:13:22.277 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-11-08T16:13:22.277 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-11-08T16:13:22.277 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-11-08T16:13:22.278 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-08T16:13:22.278 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-08T16:13:22.278 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-11-08T16:13:22.278 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-11-08T16:13:22.278 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-08T16:13:22.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-11-08T16:13:22.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-11-08T16:13:22.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-11-08T16:13:22.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-11-08T16:13:22.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-11-08T16:13:22.279 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-11-08T16:13:22.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-08T16:13:22.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-08T16:13:22.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-11-08T16:13:22.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-08T16:13:22.280 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-08T16:13:22.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-11-08T16:13:22.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-08T16:13:22.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-11-08T16:13:22.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-11-08T16:13:22.281 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-11-08T16:13:22.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-08T16:13:22.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-11-08T16:13:22.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-11-08T16:13:22.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-11-08T16:13:22.282 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-11-08T16:13:22.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-11-08T16:13:22.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-11-08T16:13:22.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-11-08T16:13:22.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-11-08T16:13:22.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.alias.bin 2023-11-08T16:13:22.283 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-11-08T16:13:22.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-11-08T16:13:22.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-11-08T16:13:22.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-11-08T16:13:22.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-11-08T16:13:22.284 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-11-08T16:13:22.285 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-11-08T16:13:22.285 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-11-08T16:13:22.285 INFO:teuthology.orchestra.run.smithi081.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-11-08T16:13:22.286 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:13:22.286 DEBUG:teuthology.orchestra.run.smithi081:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-08T16:13:22.329 INFO:teuthology.orchestra.run.smithi081.stdout:postinstall scriptlet (using /bin/sh): 2023-11-08T16:13:22.329 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-11-08T16:13:22.329 INFO:teuthology.orchestra.run.smithi081.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T16:13:22.330 INFO:teuthology.orchestra.run.smithi081.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T16:13:22.330 INFO:teuthology.orchestra.run.smithi081.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:13:22.330 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T16:13:22.330 INFO:teuthology.orchestra.run.smithi081.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T16:13:22.330 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2023-11-08T16:13:22.331 INFO:teuthology.orchestra.run.smithi081.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-08T16:13:22.331 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-08T16:13:22.331 INFO:teuthology.orchestra.run.smithi081.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-11-08T16:13:22.331 INFO:teuthology.orchestra.run.smithi081.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-08T16:13:22.331 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:13:22.332 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2023-11-08T16:13:22.332 INFO:teuthology.orchestra.run.smithi081.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-08T16:13:22.332 INFO:teuthology.orchestra.run.smithi081.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-08T16:13:22.332 INFO:teuthology.orchestra.run.smithi081.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:13:22.332 INFO:teuthology.orchestra.run.smithi081.stdout:fi 2023-11-08T16:13:22.334 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -qi grub2-tools 2023-11-08T16:13:22.410 INFO:teuthology.orchestra.run.smithi081.stdout:Name : grub2-tools 2023-11-08T16:13:22.411 INFO:teuthology.orchestra.run.smithi081.stdout:Epoch : 1 2023-11-08T16:13:22.411 INFO:teuthology.orchestra.run.smithi081.stdout:Version : 2.06 2023-11-08T16:13:22.411 INFO:teuthology.orchestra.run.smithi081.stdout:Release : 68.el9 2023-11-08T16:13:22.411 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: x86_64 2023-11-08T16:13:22.412 INFO:teuthology.orchestra.run.smithi081.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-08T16:13:22.412 INFO:teuthology.orchestra.run.smithi081.stdout:Group : Unspecified 2023-11-08T16:13:22.412 INFO:teuthology.orchestra.run.smithi081.stdout:Size : 8264254 2023-11-08T16:13:22.412 INFO:teuthology.orchestra.run.smithi081.stdout:License : GPLv3+ 2023-11-08T16:13:22.412 INFO:teuthology.orchestra.run.smithi081.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-08T16:13:22.413 INFO:teuthology.orchestra.run.smithi081.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-08T16:13:22.413 INFO:teuthology.orchestra.run.smithi081.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-08T16:13:22.413 INFO:teuthology.orchestra.run.smithi081.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-08T16:13:22.413 INFO:teuthology.orchestra.run.smithi081.stdout:Packager : builder@centos.org 2023-11-08T16:13:22.413 INFO:teuthology.orchestra.run.smithi081.stdout:Vendor : CentOS 2023-11-08T16:13:22.414 INFO:teuthology.orchestra.run.smithi081.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-08T16:13:22.414 INFO:teuthology.orchestra.run.smithi081.stdout:Summary : Support tools for GRUB. 2023-11-08T16:13:22.414 INFO:teuthology.orchestra.run.smithi081.stdout:Description : 2023-11-08T16:13:22.414 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:13:22.414 INFO:teuthology.orchestra.run.smithi081.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-08T16:13:22.415 INFO:teuthology.orchestra.run.smithi081.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-08T16:13:22.415 INFO:teuthology.orchestra.run.smithi081.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-08T16:13:22.415 INFO:teuthology.orchestra.run.smithi081.stdout:hardware devices. 2023-11-08T16:13:22.415 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:13:22.415 INFO:teuthology.orchestra.run.smithi081.stdout:This subpackage provides tools for support of all platforms. 2023-11-08T16:13:22.418 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-08T16:13:22.419 INFO:teuthology.task.kernel:Updating grub on smithi081 to boot 6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:13:22.419 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-08T16:13:23.000 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2023-11-08T16:13:23.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08 16:13:23 URL:https://3.chacra.ceph.com/r/kernel/testing/5b73517e0e2fa132ea79290e5fc06ea5349b2288/centos/9/flavors/default/x86_64/kernel-6.6.0_rc5_g5b73517e0e2f-1.x86_64.rpm [320585487/320585487] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-08T16:13:23.200 INFO:teuthology.orchestra.run.smithi093.stderr:FINISHED --2023-11-08 16:13:23-- 2023-11-08T16:13:23.200 INFO:teuthology.orchestra.run.smithi093.stderr:Total wall clock time: 11s 2023-11-08T16:13:23.200 INFO:teuthology.orchestra.run.smithi093.stderr:Downloaded: 1 files, 306M in 11s (28.9 MB/s) 2023-11-08T16:13:23.201 INFO:teuthology.task.kernel:Installing kernel 5b73517e0e2fa132ea79290e5fc06ea5349b2288 on mon.b... 2023-11-08T16:13:23.202 DEBUG:teuthology.orchestra.run.smithi093:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-08T16:13:24.543 INFO:teuthology.orchestra.run.smithi081.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-08T16:13:24.577 INFO:teuthology.orchestra.run.smithi081.stderr:done 2023-11-08T16:13:24.579 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2023-11-08T16:13:24.634 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.6KUb6FTmEQ 2023-11-08T16:13:24.635 DEBUG:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6KUb6FTmEQ 2023-11-08T16:13:24.715 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.6KUb6FTmEQ 2023-11-08T16:13:24.857 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.6KUb6FTmEQ is 6KB 2023-11-08T16:13:24.906 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.6KUb6FTmEQ 2023-11-08T16:13:24.920 DEBUG:teuthology.orchestra.run.smithi081:> sudo /bin/ls /boot/loader/entries || true 2023-11-08T16:13:24.985 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-08T16:13:24.985 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-08T16:13:24.986 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-08T16:13:24.986 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f.conf 2023-11-08T16:13:24.987 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:13:25.054 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying... ######################################## 2023-11-08T16:13:25.142 DEBUG:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2023-11-08T16:13:25.178 INFO:teuthology.orchestra.run.smithi093.stdout:Preparing... ######################################## 2023-11-08T16:13:25.257 INFO:teuthology.orchestra.run.smithi093.stdout:Updating / installing... 2023-11-08T16:13:32.352 INFO:teuthology.orchestra.run.smithi093.stdout:kernel-6.6.0_rc5_g5b73517e0e2f-1 ######################################## 2023-11-08T16:13:47.005 INFO:teuthology.orchestra.run.smithi093.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-08T16:13:55.145 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-11-08T16:13:55.145 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-11-08T16:13:55.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T16:14:07.956 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi093.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-08T16:14:07.956 DEBUG:teuthology.orchestra.run.smithi093:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-08T16:14:08.030 INFO:teuthology.orchestra.run.smithi093.stdout:/boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:14:08.030 INFO:teuthology.orchestra.run.smithi093.stdout:/boot/config-6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:14:08.030 INFO:teuthology.orchestra.run.smithi093.stdout:/boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:14:08.030 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:14:08.031 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel 2023-11-08T16:14:08.031 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch 2023-11-08T16:14:08.031 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86 2023-11-08T16:14:08.031 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto 2023-11-08T16:14:08.031 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-08T16:14:08.032 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-08T16:14:08.032 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-08T16:14:08.032 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-08T16:14:08.032 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-08T16:14:08.032 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-08T16:14:08.033 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-08T16:14:08.033 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-08T16:14:08.033 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-08T16:14:08.033 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-08T16:14:08.033 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-08T16:14:08.034 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-08T16:14:08.034 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-08T16:14:08.034 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-08T16:14:08.034 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-08T16:14:08.034 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-08T16:14:08.035 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-08T16:14:08.035 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-08T16:14:08.035 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-08T16:14:08.035 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-08T16:14:08.035 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel 2023-11-08T16:14:08.036 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu 2023-11-08T16:14:08.036 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce 2023-11-08T16:14:08.036 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-08T16:14:08.036 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm 2023-11-08T16:14:08.036 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-08T16:14:08.037 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-08T16:14:08.037 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/arch/x86/kvm/kvm.ko 2023-11-08T16:14:08.037 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto 2023-11-08T16:14:08.037 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ansi_cprng.ko 2023-11-08T16:14:08.037 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/anubis.ko 2023-11-08T16:14:08.037 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/arc4.ko 2023-11-08T16:14:08.038 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx 2023-11-08T16:14:08.038 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_memcpy.ko 2023-11-08T16:14:08.038 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_pq.ko 2023-11-08T16:14:08.038 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-08T16:14:08.038 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_tx.ko 2023-11-08T16:14:08.039 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/async_xor.ko 2023-11-08T16:14:08.039 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/async_tx/raid6test.ko 2023-11-08T16:14:08.039 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authenc.ko 2023-11-08T16:14:08.039 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/authencesn.ko 2023-11-08T16:14:08.039 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blake2b_generic.ko 2023-11-08T16:14:08.040 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_common.ko 2023-11-08T16:14:08.040 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/blowfish_generic.ko 2023-11-08T16:14:08.040 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/camellia_generic.ko 2023-11-08T16:14:08.040 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast5_generic.ko 2023-11-08T16:14:08.040 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast6_generic.ko 2023-11-08T16:14:08.041 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cast_common.ko 2023-11-08T16:14:08.041 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/ccm.ko 2023-11-08T16:14:08.041 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cmac.ko 2023-11-08T16:14:08.041 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crc32_generic.ko 2023-11-08T16:14:08.041 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_engine.ko 2023-11-08T16:14:08.042 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/crypto_user.ko 2023-11-08T16:14:08.042 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/cts.ko 2023-11-08T16:14:08.042 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/des_generic.ko 2023-11-08T16:14:08.042 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/dh_generic.ko 2023-11-08T16:14:08.042 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/echainiv.ko 2023-11-08T16:14:08.043 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/essiv.ko 2023-11-08T16:14:08.043 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/fcrypt.ko 2023-11-08T16:14:08.043 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/khazad.ko 2023-11-08T16:14:08.043 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4.ko 2023-11-08T16:14:08.043 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/lz4hc.ko 2023-11-08T16:14:08.044 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/md4.ko 2023-11-08T16:14:08.044 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/michael_mic.ko 2023-11-08T16:14:08.044 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcbc.ko 2023-11-08T16:14:08.044 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/pcrypt.ko 2023-11-08T16:14:08.044 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/rmd160.ko 2023-11-08T16:14:08.045 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/seed.ko 2023-11-08T16:14:08.045 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/serpent_generic.ko 2023-11-08T16:14:08.045 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tcrypt.ko 2023-11-08T16:14:08.045 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/tea.ko 2023-11-08T16:14:08.045 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_common.ko 2023-11-08T16:14:08.046 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/twofish_generic.ko 2023-11-08T16:14:08.046 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/vmac.ko 2023-11-08T16:14:08.046 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/wp512.ko 2023-11-08T16:14:08.046 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xcbc.ko 2023-11-08T16:14:08.046 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xor.ko 2023-11-08T16:14:08.047 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/crypto/xxhash_generic.ko 2023-11-08T16:14:08.047 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers 2023-11-08T16:14:08.047 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi 2023-11-08T16:14:08.047 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-08T16:14:08.047 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/acpi_pad.ko 2023-11-08T16:14:08.047 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/custom_method.ko 2023-11-08T16:14:08.048 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/ec_sys.ko 2023-11-08T16:14:08.048 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/platform_profile.ko 2023-11-08T16:14:08.048 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbs.ko 2023-11-08T16:14:08.048 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/sbshc.ko 2023-11-08T16:14:08.048 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/acpi/video.ko 2023-11-08T16:14:08.049 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata 2023-11-08T16:14:08.049 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/acard-ahci.ko 2023-11-08T16:14:08.049 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ahci_platform.ko 2023-11-08T16:14:08.049 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/ata_generic.ko 2023-11-08T16:14:08.049 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/libahci_platform.ko 2023-11-08T16:14:08.050 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_acpi.ko 2023-11-08T16:14:08.050 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ali.ko 2023-11-08T16:14:08.050 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_amd.ko 2023-11-08T16:14:08.050 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_artop.ko 2023-11-08T16:14:08.050 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atiixp.ko 2023-11-08T16:14:08.050 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_atp867x.ko 2023-11-08T16:14:08.051 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd640.ko 2023-11-08T16:14:08.051 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cmd64x.ko 2023-11-08T16:14:08.051 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_cypress.ko 2023-11-08T16:14:08.051 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_efar.ko 2023-11-08T16:14:08.051 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt366.ko 2023-11-08T16:14:08.052 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt37x.ko 2023-11-08T16:14:08.052 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-08T16:14:08.052 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-08T16:14:08.052 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it8213.ko 2023-11-08T16:14:08.052 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_it821x.ko 2023-11-08T16:14:08.053 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_jmicron.ko 2023-11-08T16:14:08.053 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_marvell.ko 2023-11-08T16:14:08.053 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_mpiix.ko 2023-11-08T16:14:08.053 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_netcell.ko 2023-11-08T16:14:08.053 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ninja32.ko 2023-11-08T16:14:08.054 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87410.ko 2023-11-08T16:14:08.054 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_ns87415.ko 2023-11-08T16:14:08.054 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_oldpiix.ko 2023-11-08T16:14:08.054 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_opti.ko 2023-11-08T16:14:08.054 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_optidma.ko 2023-11-08T16:14:08.054 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pcmcia.ko 2023-11-08T16:14:08.055 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-08T16:14:08.055 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-08T16:14:08.055 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_piccolo.ko 2023-11-08T16:14:08.055 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_rdc.ko 2023-11-08T16:14:08.055 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sch.ko 2023-11-08T16:14:08.056 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_serverworks.ko 2023-11-08T16:14:08.056 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sil680.ko 2023-11-08T16:14:08.057 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sis.ko 2023-11-08T16:14:08.058 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_sl82c105.ko 2023-11-08T16:14:08.058 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_triflex.ko 2023-11-08T16:14:08.058 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pata_via.ko 2023-11-08T16:14:08.058 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/pdc_adma.ko 2023-11-08T16:14:08.058 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_inic162x.ko 2023-11-08T16:14:08.059 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_mv.ko 2023-11-08T16:14:08.059 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_nv.ko 2023-11-08T16:14:08.059 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_promise.ko 2023-11-08T16:14:08.059 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_qstor.ko 2023-11-08T16:14:08.059 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil.ko 2023-11-08T16:14:08.060 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sil24.ko 2023-11-08T16:14:08.060 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sis.ko 2023-11-08T16:14:08.060 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_svw.ko 2023-11-08T16:14:08.060 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_sx4.ko 2023-11-08T16:14:08.060 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_uli.ko 2023-11-08T16:14:08.060 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_via.ko 2023-11-08T16:14:08.061 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ata/sata_vsc.ko 2023-11-08T16:14:08.061 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay 2023-11-08T16:14:08.061 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/auxdisplay/ks0108.ko 2023-11-08T16:14:08.061 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base 2023-11-08T16:14:08.061 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap 2023-11-08T16:14:08.062 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-08T16:14:08.062 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-08T16:14:08.062 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma 2023-11-08T16:14:08.062 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/bcma/bcma.ko 2023-11-08T16:14:08.062 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block 2023-11-08T16:14:08.063 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe 2023-11-08T16:14:08.063 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/aoe/aoe.ko 2023-11-08T16:14:08.063 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/brd.ko 2023-11-08T16:14:08.063 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd 2023-11-08T16:14:08.063 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/drbd/drbd.ko 2023-11-08T16:14:08.064 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/floppy.ko 2023-11-08T16:14:08.064 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/loop.ko 2023-11-08T16:14:08.064 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx 2023-11-08T16:14:08.064 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-08T16:14:08.064 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/nbd.ko 2023-11-08T16:14:08.065 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk 2023-11-08T16:14:08.065 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/null_blk/null_blk.ko 2023-11-08T16:14:08.065 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/pktcdvd.ko 2023-11-08T16:14:08.065 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/rbd.ko 2023-11-08T16:14:08.065 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/virtio_blk.ko 2023-11-08T16:14:08.066 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback 2023-11-08T16:14:08.066 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-08T16:14:08.066 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/xen-blkfront.ko 2023-11-08T16:14:08.067 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram 2023-11-08T16:14:08.067 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/block/zram/zram.ko 2023-11-08T16:14:08.067 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char 2023-11-08T16:14:08.067 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hangcheck-timer.ko 2023-11-08T16:14:08.068 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random 2023-11-08T16:14:08.068 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-08T16:14:08.068 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-08T16:14:08.068 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-08T16:14:08.069 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/via-rng.ko 2023-11-08T16:14:08.069 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-08T16:14:08.069 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi 2023-11-08T16:14:08.069 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-08T16:14:08.070 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-08T16:14:08.070 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-08T16:14:08.070 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-08T16:14:08.070 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-08T16:14:08.071 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-08T16:14:08.071 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/lp.ko 2023-11-08T16:14:08.071 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave 2023-11-08T16:14:08.072 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/mwave/mwave.ko 2023-11-08T16:14:08.072 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/ppdev.ko 2023-11-08T16:14:08.072 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tlclk.ko 2023-11-08T16:14:08.072 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm 2023-11-08T16:14:08.072 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm.ko 2023-11-08T16:14:08.073 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-08T16:14:08.073 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-08T16:14:08.073 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-08T16:14:08.073 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-08T16:14:08.074 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-08T16:14:08.074 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/uv_mmtimer.ko 2023-11-08T16:14:08.074 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/char/virtio_console.ko 2023-11-08T16:14:08.074 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq 2023-11-08T16:14:08.075 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-08T16:14:08.075 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-08T16:14:08.075 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-08T16:14:08.075 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-08T16:14:08.075 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-08T16:14:08.075 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-08T16:14:08.076 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto 2023-11-08T16:14:08.076 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp 2023-11-08T16:14:08.076 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-08T16:14:08.076 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/ccp/ccp.ko 2023-11-08T16:14:08.077 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel 2023-11-08T16:14:08.077 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat 2023-11-08T16:14:08.077 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common 2023-11-08T16:14:08.077 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-08T16:14:08.077 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-08T16:14:08.077 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-08T16:14:08.078 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-aes.ko 2023-11-08T16:14:08.078 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/padlock-sha.ko 2023-11-08T16:14:08.078 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio 2023-11-08T16:14:08.078 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-08T16:14:08.078 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca 2023-11-08T16:14:08.079 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dca/dca.ko 2023-11-08T16:14:08.079 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq 2023-11-08T16:14:08.079 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-08T16:14:08.079 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma 2023-11-08T16:14:08.079 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw 2023-11-08T16:14:08.080 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-08T16:14:08.080 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat 2023-11-08T16:14:08.080 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-08T16:14:08.080 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac 2023-11-08T16:14:08.080 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/amd64_edac.ko 2023-11-08T16:14:08.080 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/e752x_edac.ko 2023-11-08T16:14:08.081 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/edac_mce_amd.ko 2023-11-08T16:14:08.081 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3000_edac.ko 2023-11-08T16:14:08.081 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i3200_edac.ko 2023-11-08T16:14:08.081 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5100_edac.ko 2023-11-08T16:14:08.082 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i5400_edac.ko 2023-11-08T16:14:08.082 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7300_edac.ko 2023-11-08T16:14:08.082 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i7core_edac.ko 2023-11-08T16:14:08.082 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/i82975x_edac.ko 2023-11-08T16:14:08.083 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/ie31200_edac.ko 2023-11-08T16:14:08.083 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/sb_edac.ko 2023-11-08T16:14:08.083 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/edac/x38_edac.ko 2023-11-08T16:14:08.083 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware 2023-11-08T16:14:08.083 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/edd.ko 2023-11-08T16:14:08.083 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-08T16:14:08.084 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio 2023-11-08T16:14:08.084 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-ich.ko 2023-11-08T16:14:08.084 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-08T16:14:08.084 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid 2023-11-08T16:14:08.084 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-appleir.ko 2023-11-08T16:14:08.085 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-aureal.ko 2023-11-08T16:14:08.085 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-axff.ko 2023-11-08T16:14:08.085 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-dr.ko 2023-11-08T16:14:08.085 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elecom.ko 2023-11-08T16:14:08.085 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-elo.ko 2023-11-08T16:14:08.085 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-emsff.ko 2023-11-08T16:14:08.086 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gaff.ko 2023-11-08T16:14:08.086 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gt683r.ko 2023-11-08T16:14:08.086 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-gyration.ko 2023-11-08T16:14:08.086 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-08T16:14:08.087 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-08T16:14:08.087 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-holtekff.ko 2023-11-08T16:14:08.087 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-hyperv.ko 2023-11-08T16:14:08.087 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-icade.ko 2023-11-08T16:14:08.087 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-keytouch.ko 2023-11-08T16:14:08.088 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-kye.ko 2023-11-08T16:14:08.088 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lcpower.ko 2023-11-08T16:14:08.088 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-led.ko 2023-11-08T16:14:08.088 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-lenovo.ko 2023-11-08T16:14:08.088 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-08T16:14:08.089 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-08T16:14:08.089 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-multitouch.ko 2023-11-08T16:14:08.089 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-ortek.ko 2023-11-08T16:14:08.090 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-penmount.ko 2023-11-08T16:14:08.090 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-petalynx.ko 2023-11-08T16:14:08.090 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-picolcd.ko 2023-11-08T16:14:08.090 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-pl.ko 2023-11-08T16:14:08.090 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-plantronics.ko 2023-11-08T16:14:08.091 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-primax.ko 2023-11-08T16:14:08.091 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-rmi.ko 2023-11-08T16:14:08.091 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-08T16:14:08.091 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-common.ko 2023-11-08T16:14:08.091 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-08T16:14:08.092 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-08T16:14:08.092 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-08T16:14:08.092 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-08T16:14:08.092 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-08T16:14:08.092 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-08T16:14:08.092 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-08T16:14:08.093 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-08T16:14:08.093 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-08T16:14:08.093 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-roccat.ko 2023-11-08T16:14:08.093 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-saitek.ko 2023-11-08T16:14:08.093 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-samsung.ko 2023-11-08T16:14:08.093 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-08T16:14:08.094 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sjoy.ko 2023-11-08T16:14:08.094 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sony.ko 2023-11-08T16:14:08.094 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-speedlink.ko 2023-11-08T16:14:08.095 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-steelseries.ko 2023-11-08T16:14:08.095 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-sunplus.ko 2023-11-08T16:14:08.095 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-08T16:14:08.095 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tivo.ko 2023-11-08T16:14:08.095 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-tmff.ko 2023-11-08T16:14:08.096 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-topseed.ko 2023-11-08T16:14:08.096 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-twinhan.ko 2023-11-08T16:14:08.096 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-uclogic.ko 2023-11-08T16:14:08.096 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-waltop.ko 2023-11-08T16:14:08.097 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-wiimote.ko 2023-11-08T16:14:08.097 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-xinmo.ko 2023-11-08T16:14:08.097 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zpff.ko 2023-11-08T16:14:08.097 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/hid-zydacron.ko 2023-11-08T16:14:08.098 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/uhid.ko 2023-11-08T16:14:08.098 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hid/wacom.ko 2023-11-08T16:14:08.098 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv 2023-11-08T16:14:08.098 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_balloon.ko 2023-11-08T16:14:08.099 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_utils.ko 2023-11-08T16:14:08.099 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hv/hv_vmbus.ko 2023-11-08T16:14:08.099 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon 2023-11-08T16:14:08.099 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru.ko 2023-11-08T16:14:08.100 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/abituguru3.ko 2023-11-08T16:14:08.100 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-08T16:14:08.100 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7414.ko 2023-11-08T16:14:08.100 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ad7418.ko 2023-11-08T16:14:08.101 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adc128d818.ko 2023-11-08T16:14:08.101 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1025.ko 2023-11-08T16:14:08.101 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1026.ko 2023-11-08T16:14:08.101 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1029.ko 2023-11-08T16:14:08.102 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm1031.ko 2023-11-08T16:14:08.102 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adm9240.ko 2023-11-08T16:14:08.102 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ads7828.ko 2023-11-08T16:14:08.102 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7410.ko 2023-11-08T16:14:08.103 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7411.ko 2023-11-08T16:14:08.103 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7462.ko 2023-11-08T16:14:08.103 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7470.ko 2023-11-08T16:14:08.103 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7475.ko 2023-11-08T16:14:08.103 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/adt7x10.ko 2023-11-08T16:14:08.104 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/amc6821.ko 2023-11-08T16:14:08.104 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/applesmc.ko 2023-11-08T16:14:08.104 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asb100.ko 2023-11-08T16:14:08.104 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asc7621.ko 2023-11-08T16:14:08.104 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-08T16:14:08.104 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/atxp1.ko 2023-11-08T16:14:08.105 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/coretemp.ko 2023-11-08T16:14:08.105 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-08T16:14:08.105 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/dme1737.ko 2023-11-08T16:14:08.105 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds1621.ko 2023-11-08T16:14:08.105 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ds620.ko 2023-11-08T16:14:08.106 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc1403.ko 2023-11-08T16:14:08.106 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/emc6w201.ko 2023-11-08T16:14:08.106 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71805f.ko 2023-11-08T16:14:08.106 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f71882fg.ko 2023-11-08T16:14:08.107 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/f75375s.ko 2023-11-08T16:14:08.107 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fam15h_power.ko 2023-11-08T16:14:08.107 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/fschmd.ko 2023-11-08T16:14:08.107 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g760a.ko 2023-11-08T16:14:08.107 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/g762.ko 2023-11-08T16:14:08.107 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl518sm.ko 2023-11-08T16:14:08.108 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/gl520sm.ko 2023-11-08T16:14:08.108 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-08T16:14:08.108 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5500_temp.ko 2023-11-08T16:14:08.108 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/i5k_amb.ko 2023-11-08T16:14:08.108 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmaem.ko 2023-11-08T16:14:08.109 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ibmpex.ko 2023-11-08T16:14:08.109 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina209.ko 2023-11-08T16:14:08.109 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ina2xx.ko 2023-11-08T16:14:08.109 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/it87.ko 2023-11-08T16:14:08.109 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k10temp.ko 2023-11-08T16:14:08.109 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/k8temp.ko 2023-11-08T16:14:08.110 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lineage-pem.ko 2023-11-08T16:14:08.110 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm63.ko 2023-11-08T16:14:08.110 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm73.ko 2023-11-08T16:14:08.110 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm75.ko 2023-11-08T16:14:08.110 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm77.ko 2023-11-08T16:14:08.111 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm78.ko 2023-11-08T16:14:08.111 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm80.ko 2023-11-08T16:14:08.111 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm83.ko 2023-11-08T16:14:08.111 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm85.ko 2023-11-08T16:14:08.111 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm87.ko 2023-11-08T16:14:08.111 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm90.ko 2023-11-08T16:14:08.112 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm92.ko 2023-11-08T16:14:08.112 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm93.ko 2023-11-08T16:14:08.112 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95234.ko 2023-11-08T16:14:08.112 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95241.ko 2023-11-08T16:14:08.112 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/lm95245.ko 2023-11-08T16:14:08.113 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc2945.ko 2023-11-08T16:14:08.113 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4151.ko 2023-11-08T16:14:08.113 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4215.ko 2023-11-08T16:14:08.113 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4222.ko 2023-11-08T16:14:08.113 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4245.ko 2023-11-08T16:14:08.113 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4260.ko 2023-11-08T16:14:08.114 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/ltc4261.ko 2023-11-08T16:14:08.114 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max16065.ko 2023-11-08T16:14:08.114 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1619.ko 2023-11-08T16:14:08.114 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max1668.ko 2023-11-08T16:14:08.114 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max197.ko 2023-11-08T16:14:08.115 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6639.ko 2023-11-08T16:14:08.115 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6650.ko 2023-11-08T16:14:08.115 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/max6697.ko 2023-11-08T16:14:08.115 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/mcp3021.ko 2023-11-08T16:14:08.115 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6683.ko 2023-11-08T16:14:08.116 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775-core.ko 2023-11-08T16:14:08.116 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct6775.ko 2023-11-08T16:14:08.116 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/nct7802.ko 2023-11-08T16:14:08.116 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87360.ko 2023-11-08T16:14:08.116 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pc87427.ko 2023-11-08T16:14:08.117 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pcf8591.ko 2023-11-08T16:14:08.117 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus 2023-11-08T16:14:08.117 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-08T16:14:08.117 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-08T16:14:08.118 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-08T16:14:08.118 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-08T16:14:08.118 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-08T16:14:08.118 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-08T16:14:08.119 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-08T16:14:08.119 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-08T16:14:08.119 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-08T16:14:08.119 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-08T16:14:08.119 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-08T16:14:08.120 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-08T16:14:08.120 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/powr1220.ko 2023-11-08T16:14:08.120 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5627.ko 2023-11-08T16:14:08.120 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch5636.ko 2023-11-08T16:14:08.121 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-08T16:14:08.121 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht15.ko 2023-11-08T16:14:08.121 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sht21.ko 2023-11-08T16:14:08.121 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/shtc1.ko 2023-11-08T16:14:08.121 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/sis5595.ko 2023-11-08T16:14:08.122 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47b397.ko 2023-11-08T16:14:08.122 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m1.ko 2023-11-08T16:14:08.122 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/smsc47m192.ko 2023-11-08T16:14:08.122 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/thmc50.ko 2023-11-08T16:14:08.123 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp102.ko 2023-11-08T16:14:08.123 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp103.ko 2023-11-08T16:14:08.123 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp401.ko 2023-11-08T16:14:08.123 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/tmp421.ko 2023-11-08T16:14:08.124 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via-cputemp.ko 2023-11-08T16:14:08.124 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/via686a.ko 2023-11-08T16:14:08.124 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt1211.ko 2023-11-08T16:14:08.124 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/vt8231.ko 2023-11-08T16:14:08.124 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627ehf.ko 2023-11-08T16:14:08.125 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83627hf.ko 2023-11-08T16:14:08.125 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83781d.ko 2023-11-08T16:14:08.125 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83791d.ko 2023-11-08T16:14:08.125 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83792d.ko 2023-11-08T16:14:08.126 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83793.ko 2023-11-08T16:14:08.126 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83795.ko 2023-11-08T16:14:08.126 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l785ts.ko 2023-11-08T16:14:08.126 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/hwmon/w83l786ng.ko 2023-11-08T16:14:08.127 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c 2023-11-08T16:14:08.127 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos 2023-11-08T16:14:08.127 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-08T16:14:08.127 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-08T16:14:08.128 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses 2023-11-08T16:14:08.128 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-08T16:14:08.128 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-08T16:14:08.128 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-08T16:14:08.129 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-08T16:14:08.129 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-08T16:14:08.129 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-08T16:14:08.130 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-08T16:14:08.130 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-08T16:14:08.130 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-08T16:14:08.130 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-08T16:14:08.130 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-08T16:14:08.131 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-08T16:14:08.131 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-08T16:14:08.131 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-08T16:14:08.131 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-08T16:14:08.131 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-08T16:14:08.132 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-08T16:14:08.132 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-08T16:14:08.132 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-08T16:14:08.132 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-08T16:14:08.132 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-08T16:14:08.133 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-08T16:14:08.133 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-08T16:14:08.133 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-dev.ko 2023-11-08T16:14:08.133 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-mux.ko 2023-11-08T16:14:08.133 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-08T16:14:08.133 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-smbus.ko 2023-11-08T16:14:08.134 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/i2c/i2c-stub.ko 2023-11-08T16:14:08.134 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband 2023-11-08T16:14:08.134 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core 2023-11-08T16:14:08.134 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-08T16:14:08.135 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_core.ko 2023-11-08T16:14:08.135 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-08T16:14:08.135 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-08T16:14:08.135 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-08T16:14:08.135 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-08T16:14:08.135 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-08T16:14:08.136 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw 2023-11-08T16:14:08.136 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4 2023-11-08T16:14:08.136 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-08T16:14:08.136 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4 2023-11-08T16:14:08.137 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-08T16:14:08.137 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5 2023-11-08T16:14:08.137 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-08T16:14:08.137 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca 2023-11-08T16:14:08.137 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-08T16:14:08.138 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp 2023-11-08T16:14:08.138 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib 2023-11-08T16:14:08.138 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-08T16:14:08.138 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser 2023-11-08T16:14:08.138 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-08T16:14:08.139 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert 2023-11-08T16:14:08.139 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-08T16:14:08.139 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp 2023-11-08T16:14:08.139 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-08T16:14:08.139 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt 2023-11-08T16:14:08.140 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-08T16:14:08.140 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input 2023-11-08T16:14:08.140 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport 2023-11-08T16:14:08.140 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-08T16:14:08.140 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-08T16:14:08.140 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/gameport/gameport.ko 2023-11-08T16:14:08.141 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/joydev.ko 2023-11-08T16:14:08.141 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard 2023-11-08T16:14:08.141 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-08T16:14:08.141 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse 2023-11-08T16:14:08.141 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/appletouch.ko 2023-11-08T16:14:08.142 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/bcm5974.ko 2023-11-08T16:14:08.142 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/cyapatp.ko 2023-11-08T16:14:08.142 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-08T16:14:08.142 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/sermouse.ko 2023-11-08T16:14:08.142 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-08T16:14:08.143 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-08T16:14:08.143 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-08T16:14:08.143 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4 2023-11-08T16:14:08.143 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-08T16:14:08.143 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio 2023-11-08T16:14:08.144 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/altera_ps2.ko 2023-11-08T16:14:08.144 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/arc_ps2.ko 2023-11-08T16:14:08.144 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-08T16:14:08.144 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/serio/serio_raw.ko 2023-11-08T16:14:08.145 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/input/sparse-keymap.ko 2023-11-08T16:14:08.145 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu 2023-11-08T16:14:08.145 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd 2023-11-08T16:14:08.145 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-08T16:14:08.145 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds 2023-11-08T16:14:08.145 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-blinkm.ko 2023-11-08T16:14:08.146 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lm3530.ko 2023-11-08T16:14:08.146 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-lp3944.ko 2023-11-08T16:14:08.146 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/leds-ss4200.ko 2023-11-08T16:14:08.146 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger 2023-11-08T16:14:08.147 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-08T16:14:08.147 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-08T16:14:08.147 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-08T16:14:08.147 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-08T16:14:08.148 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-08T16:14:08.148 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-08T16:14:08.148 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-08T16:14:08.148 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-08T16:14:08.149 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md 2023-11-08T16:14:08.149 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache 2023-11-08T16:14:08.149 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/bcache/bcache.ko 2023-11-08T16:14:08.149 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-bio-prison.ko 2023-11-08T16:14:08.150 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache-smq.ko 2023-11-08T16:14:08.150 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-cache.ko 2023-11-08T16:14:08.150 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-crypt.ko 2023-11-08T16:14:08.151 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-delay.ko 2023-11-08T16:14:08.151 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-flakey.ko 2023-11-08T16:14:08.151 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-log-userspace.ko 2023-11-08T16:14:08.151 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-multipath.ko 2023-11-08T16:14:08.151 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-queue-length.ko 2023-11-08T16:14:08.152 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-raid.ko 2023-11-08T16:14:08.152 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-round-robin.ko 2023-11-08T16:14:08.152 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-service-time.ko 2023-11-08T16:14:08.152 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-switch.ko 2023-11-08T16:14:08.153 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-thin-pool.ko 2023-11-08T16:14:08.153 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/dm-verity.ko 2023-11-08T16:14:08.153 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/faulty.ko 2023-11-08T16:14:08.153 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/linear.ko 2023-11-08T16:14:08.153 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/multipath.ko 2023-11-08T16:14:08.153 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data 2023-11-08T16:14:08.154 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-08T16:14:08.154 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid0.ko 2023-11-08T16:14:08.154 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid1.ko 2023-11-08T16:14:08.154 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid10.ko 2023-11-08T16:14:08.154 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/md/raid456.ko 2023-11-08T16:14:08.155 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message 2023-11-08T16:14:08.155 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion 2023-11-08T16:14:08.155 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptbase.ko 2023-11-08T16:14:08.155 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptctl.ko 2023-11-08T16:14:08.155 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptfc.ko 2023-11-08T16:14:08.156 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptlan.ko 2023-11-08T16:14:08.156 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptsas.ko 2023-11-08T16:14:08.156 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptscsih.ko 2023-11-08T16:14:08.156 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/message/fusion/mptspi.ko 2023-11-08T16:14:08.157 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd 2023-11-08T16:14:08.157 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_ich.ko 2023-11-08T16:14:08.157 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/lpc_sch.ko 2023-11-08T16:14:08.157 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/mfd-core.ko 2023-11-08T16:14:08.158 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/sm501.ko 2023-11-08T16:14:08.158 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/viperboard.ko 2023-11-08T16:14:08.158 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/vx855.ko 2023-11-08T16:14:08.159 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/mfd/wl1273-core.ko 2023-11-08T16:14:08.159 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc 2023-11-08T16:14:08.159 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl 2023-11-08T16:14:08.159 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-08T16:14:08.160 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds9802als.ko 2023-11-08T16:14:08.160 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/apds990x.ko 2023-11-08T16:14:08.160 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/bh1770glc.ko 2023-11-08T16:14:08.160 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710 2023-11-08T16:14:08.161 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/cb710/cb710.ko 2023-11-08T16:14:08.161 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom 2023-11-08T16:14:08.161 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/at24.ko 2023-11-08T16:14:08.161 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-08T16:14:08.162 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-08T16:14:08.162 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/eeprom/max6875.ko 2023-11-08T16:14:08.162 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/enclosure.ko 2023-11-08T16:14:08.162 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/hpilo.ko 2023-11-08T16:14:08.163 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29003.ko 2023-11-08T16:14:08.163 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/isl29020.ko 2023-11-08T16:14:08.163 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d 2023-11-08T16:14:08.163 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-08T16:14:08.163 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-08T16:14:08.164 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei 2023-11-08T16:14:08.164 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-me.ko 2023-11-08T16:14:08.164 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei-txe.ko 2023-11-08T16:14:08.164 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/mei/mei.ko 2023-11-08T16:14:08.165 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru 2023-11-08T16:14:08.165 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-08T16:14:08.165 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp 2023-11-08T16:14:08.165 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-08T16:14:08.165 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-08T16:14:08.166 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-08T16:14:08.166 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_7xx1.ko 2023-11-08T16:14:08.166 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tifm_core.ko 2023-11-08T16:14:08.166 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/tsl2550.ko 2023-11-08T16:14:08.166 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_balloon.ko 2023-11-08T16:14:08.167 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci 2023-11-08T16:14:08.167 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-08T16:14:08.167 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net 2023-11-08T16:14:08.167 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding 2023-11-08T16:14:08.168 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/bonding/bonding.ko 2023-11-08T16:14:08.168 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa 2023-11-08T16:14:08.168 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53 2023-11-08T16:14:08.168 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-08T16:14:08.169 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-08T16:14:08.169 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-08T16:14:08.169 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx 2023-11-08T16:14:08.169 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-08T16:14:08.170 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/dummy.ko 2023-11-08T16:14:08.170 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/eql.ko 2023-11-08T16:14:08.170 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet 2023-11-08T16:14:08.170 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com 2023-11-08T16:14:08.170 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-08T16:14:08.171 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-08T16:14:08.171 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-08T16:14:08.171 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-08T16:14:08.171 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390 2023-11-08T16:14:08.172 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-08T16:14:08.172 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-08T16:14:08.172 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-08T16:14:08.172 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-08T16:14:08.172 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec 2023-11-08T16:14:08.173 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-08T16:14:08.173 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere 2023-11-08T16:14:08.173 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-08T16:14:08.173 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon 2023-11-08T16:14:08.173 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-08T16:14:08.174 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera 2023-11-08T16:14:08.174 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-08T16:14:08.174 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd 2023-11-08T16:14:08.174 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-08T16:14:08.175 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-08T16:14:08.175 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-08T16:14:08.175 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros 2023-11-08T16:14:08.175 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx 2023-11-08T16:14:08.176 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-08T16:14:08.176 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-08T16:14:08.176 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-08T16:14:08.176 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-08T16:14:08.176 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-08T16:14:08.177 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx 2023-11-08T16:14:08.177 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-08T16:14:08.177 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-08T16:14:08.177 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom 2023-11-08T16:14:08.178 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-08T16:14:08.178 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-08T16:14:08.178 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-08T16:14:08.178 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-08T16:14:08.179 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-08T16:14:08.179 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet 2023-11-08T16:14:08.179 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-08T16:14:08.179 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-08T16:14:08.179 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade 2023-11-08T16:14:08.180 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna 2023-11-08T16:14:08.180 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-08T16:14:08.180 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio 2023-11-08T16:14:08.180 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-08T16:14:08.181 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-08T16:14:08.181 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-08T16:14:08.181 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-08T16:14:08.181 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-08T16:14:08.182 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-08T16:14:08.182 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-08T16:14:08.182 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-08T16:14:08.182 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-08T16:14:08.183 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-08T16:14:08.183 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco 2023-11-08T16:14:08.183 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic 2023-11-08T16:14:08.183 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-08T16:14:08.184 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec 2023-11-08T16:14:08.184 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip 2023-11-08T16:14:08.184 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-08T16:14:08.184 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-08T16:14:08.185 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-08T16:14:08.185 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-08T16:14:08.185 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-08T16:14:08.185 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-08T16:14:08.186 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink 2023-11-08T16:14:08.186 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-08T16:14:08.186 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-08T16:14:08.187 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/dnet.ko 2023-11-08T16:14:08.187 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex 2023-11-08T16:14:08.187 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet 2023-11-08T16:14:08.187 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-08T16:14:08.188 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ethoc.ko 2023-11-08T16:14:08.188 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/fealnx.ko 2023-11-08T16:14:08.188 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel 2023-11-08T16:14:08.188 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-08T16:14:08.189 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000 2023-11-08T16:14:08.189 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-08T16:14:08.189 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e 2023-11-08T16:14:08.189 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-08T16:14:08.189 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k 2023-11-08T16:14:08.190 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-08T16:14:08.190 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e 2023-11-08T16:14:08.190 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-08T16:14:08.190 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf 2023-11-08T16:14:08.191 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-08T16:14:08.191 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb 2023-11-08T16:14:08.191 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-08T16:14:08.191 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf 2023-11-08T16:14:08.191 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-08T16:14:08.191 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-08T16:14:08.192 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-08T16:14:08.192 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-08T16:14:08.192 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-08T16:14:08.192 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/jme.ko 2023-11-08T16:14:08.192 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell 2023-11-08T16:14:08.193 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-08T16:14:08.193 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-08T16:14:08.193 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-08T16:14:08.194 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox 2023-11-08T16:14:08.194 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-08T16:14:08.194 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-08T16:14:08.194 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-08T16:14:08.195 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-08T16:14:08.195 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-08T16:14:08.195 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-08T16:14:08.195 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel 2023-11-08T16:14:08.195 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-08T16:14:08.196 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom 2023-11-08T16:14:08.196 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-08T16:14:08.196 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-08T16:14:08.196 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi 2023-11-08T16:14:08.196 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-08T16:14:08.197 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-08T16:14:08.197 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion 2023-11-08T16:14:08.197 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-08T16:14:08.197 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia 2023-11-08T16:14:08.198 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-08T16:14:08.198 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines 2023-11-08T16:14:08.198 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-08T16:14:08.198 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-08T16:14:08.198 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic 2023-11-08T16:14:08.199 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-08T16:14:08.199 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-08T16:14:08.199 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-08T16:14:08.199 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-08T16:14:08.200 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-08T16:14:08.200 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc 2023-11-08T16:14:08.200 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-08T16:14:08.200 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek 2023-11-08T16:14:08.200 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-08T16:14:08.201 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-08T16:14:08.201 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-08T16:14:08.201 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-08T16:14:08.201 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker 2023-11-08T16:14:08.201 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-08T16:14:08.202 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc 2023-11-08T16:14:08.202 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-08T16:14:08.202 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan 2023-11-08T16:14:08.203 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-08T16:14:08.203 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis 2023-11-08T16:14:08.203 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-08T16:14:08.203 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-08T16:14:08.203 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc 2023-11-08T16:14:08.203 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-08T16:14:08.204 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-08T16:14:08.204 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-08T16:14:08.204 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-08T16:14:08.204 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro 2023-11-08T16:14:08.204 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-08T16:14:08.205 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-08T16:14:08.205 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-08T16:14:08.205 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun 2023-11-08T16:14:08.205 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-08T16:14:08.206 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-08T16:14:08.206 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-08T16:14:08.206 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-08T16:14:08.206 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti 2023-11-08T16:14:08.207 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-08T16:14:08.207 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti 2023-11-08T16:14:08.207 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-08T16:14:08.207 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via 2023-11-08T16:14:08.208 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-08T16:14:08.208 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-08T16:14:08.208 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet 2023-11-08T16:14:08.208 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-08T16:14:08.209 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-08T16:14:08.209 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom 2023-11-08T16:14:08.209 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-08T16:14:08.209 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/geneve.ko 2023-11-08T16:14:08.209 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv 2023-11-08T16:14:08.210 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-08T16:14:08.210 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154 2023-11-08T16:14:08.210 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-08T16:14:08.210 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ifb.ko 2023-11-08T16:14:08.211 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan 2023-11-08T16:14:08.211 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-08T16:14:08.211 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvlan.ko 2023-11-08T16:14:08.211 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/macvtap.ko 2023-11-08T16:14:08.212 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio 2023-11-08T16:14:08.212 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio.ko 2023-11-08T16:14:08.212 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-08T16:14:08.212 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-08T16:14:08.212 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-08T16:14:08.213 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/mii.ko 2023-11-08T16:14:08.213 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/net_failover.ko 2023-11-08T16:14:08.213 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/netconsole.ko 2023-11-08T16:14:08.213 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/nlmon.ko 2023-11-08T16:14:08.213 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs 2023-11-08T16:14:08.214 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-08T16:14:08.214 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-08T16:14:08.214 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy 2023-11-08T16:14:08.214 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/amd.ko 2023-11-08T16:14:08.215 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-08T16:14:08.215 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-08T16:14:08.215 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-08T16:14:08.215 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/bcm87xx.ko 2023-11-08T16:14:08.216 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/broadcom.ko 2023-11-08T16:14:08.216 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/cicada.ko 2023-11-08T16:14:08.216 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/davicom.ko 2023-11-08T16:14:08.216 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/dp83640.ko 2023-11-08T16:14:08.217 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/et1011c.ko 2023-11-08T16:14:08.217 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/icplus.ko 2023-11-08T16:14:08.217 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/lxt.ko 2023-11-08T16:14:08.217 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/marvell.ko 2023-11-08T16:14:08.218 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/micrel.ko 2023-11-08T16:14:08.218 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/national.ko 2023-11-08T16:14:08.218 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/phylink.ko 2023-11-08T16:14:08.218 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/qsemi.ko 2023-11-08T16:14:08.219 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/realtek.ko 2023-11-08T16:14:08.219 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/smsc.ko 2023-11-08T16:14:08.219 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/ste10Xp.ko 2023-11-08T16:14:08.219 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/phy/vitesse.ko 2023-11-08T16:14:08.219 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp 2023-11-08T16:14:08.220 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-08T16:14:08.220 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_async.ko 2023-11-08T16:14:08.220 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-08T16:14:08.220 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-08T16:14:08.220 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-08T16:14:08.221 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-08T16:14:08.221 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppoe.ko 2023-11-08T16:14:08.221 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pppox.ko 2023-11-08T16:14:08.221 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/ppp/pptp.ko 2023-11-08T16:14:08.221 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip 2023-11-08T16:14:08.222 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slhc.ko 2023-11-08T16:14:08.222 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/slip/slip.ko 2023-11-08T16:14:08.222 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/sungem_phy.ko 2023-11-08T16:14:08.222 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tap.ko 2023-11-08T16:14:08.222 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team 2023-11-08T16:14:08.223 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team.ko 2023-11-08T16:14:08.223 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-08T16:14:08.223 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-08T16:14:08.223 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-08T16:14:08.223 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_random.ko 2023-11-08T16:14:08.224 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-08T16:14:08.224 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/tun.ko 2023-11-08T16:14:08.224 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/veth.ko 2023-11-08T16:14:08.224 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/virtio_net.ko 2023-11-08T16:14:08.225 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3 2023-11-08T16:14:08.225 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-08T16:14:08.225 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan 2023-11-08T16:14:08.225 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/vxlan/vxlan.ko 2023-11-08T16:14:08.225 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback 2023-11-08T16:14:08.225 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-08T16:14:08.226 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/net/xen-netfront.ko 2023-11-08T16:14:08.226 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb 2023-11-08T16:14:08.226 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ntb/ntb.ko 2023-11-08T16:14:08.226 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme 2023-11-08T16:14:08.227 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host 2023-11-08T16:14:08.227 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme-core.ko 2023-11-08T16:14:08.227 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/nvme/host/nvme.ko 2023-11-08T16:14:08.227 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport 2023-11-08T16:14:08.227 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport.ko 2023-11-08T16:14:08.228 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_cs.ko 2023-11-08T16:14:08.228 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_pc.ko 2023-11-08T16:14:08.228 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/parport/parport_serial.ko 2023-11-08T16:14:08.228 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci 2023-11-08T16:14:08.228 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug 2023-11-08T16:14:08.229 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-08T16:14:08.229 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie 2023-11-08T16:14:08.229 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-08T16:14:08.229 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pci/xen-pcifront.ko 2023-11-08T16:14:08.229 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia 2023-11-08T16:14:08.230 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/i82092.ko 2023-11-08T16:14:08.230 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/pd6729.ko 2023-11-08T16:14:08.230 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-08T16:14:08.230 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl 2023-11-08T16:14:08.230 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel 2023-11-08T16:14:08.231 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-08T16:14:08.231 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform 2023-11-08T16:14:08.231 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86 2023-11-08T16:14:08.231 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-08T16:14:08.231 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/acerhdf.ko 2023-11-08T16:14:08.232 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-08T16:14:08.232 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-08T16:14:08.232 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-08T16:14:08.232 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-08T16:14:08.232 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-08T16:14:08.233 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-08T16:14:08.233 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-08T16:14:08.233 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-08T16:14:08.233 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-08T16:14:08.233 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/hdaps.ko 2023-11-08T16:14:08.234 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel 2023-11-08T16:14:08.234 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-08T16:14:08.234 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/intel_ips.ko 2023-11-08T16:14:08.234 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-08T16:14:08.235 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-08T16:14:08.235 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-08T16:14:08.235 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-08T16:14:08.235 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-08T16:14:08.235 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-08T16:14:08.235 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-08T16:14:08.236 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-08T16:14:08.236 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-08T16:14:08.236 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/platform/x86/wmi.ko 2023-11-08T16:14:08.236 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap 2023-11-08T16:14:08.236 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-08T16:14:08.237 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-08T16:14:08.237 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps 2023-11-08T16:14:08.237 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients 2023-11-08T16:14:08.237 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-08T16:14:08.237 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-08T16:14:08.238 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/clients/pps_parport.ko 2023-11-08T16:14:08.238 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/pps/pps_core.ko 2023-11-08T16:14:08.238 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp 2023-11-08T16:14:08.238 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp.ko 2023-11-08T16:14:08.239 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ptp/ptp_kvm.ko 2023-11-08T16:14:08.239 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc 2023-11-08T16:14:08.239 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-08T16:14:08.239 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-08T16:14:08.239 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-08T16:14:08.240 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-08T16:14:08.240 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-08T16:14:08.240 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-08T16:14:08.240 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-08T16:14:08.240 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-08T16:14:08.241 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-08T16:14:08.241 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-08T16:14:08.241 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-em3027.ko 2023-11-08T16:14:08.241 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-08T16:14:08.241 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-08T16:14:08.241 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-08T16:14:08.242 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-08T16:14:08.242 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-08T16:14:08.242 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-08T16:14:08.242 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-max6900.ko 2023-11-08T16:14:08.242 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-08T16:14:08.243 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-08T16:14:08.243 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-08T16:14:08.243 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-08T16:14:08.243 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-08T16:14:08.243 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-08T16:14:08.244 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-08T16:14:08.244 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-08T16:14:08.244 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-08T16:14:08.244 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-08T16:14:08.244 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-08T16:14:08.245 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-08T16:14:08.245 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/rtc/rtc-x1205.ko 2023-11-08T16:14:08.245 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi 2023-11-08T16:14:08.245 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-9xxx.ko 2023-11-08T16:14:08.245 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-sas.ko 2023-11-08T16:14:08.246 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/3w-xxxx.ko 2023-11-08T16:14:08.246 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/BusLogic.ko 2023-11-08T16:14:08.246 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/a100u2w.ko 2023-11-08T16:14:08.246 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid 2023-11-08T16:14:08.246 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-08T16:14:08.247 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/advansys.ko 2023-11-08T16:14:08.247 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx 2023-11-08T16:14:08.247 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-08T16:14:08.247 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-08T16:14:08.247 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/am53c974.ko 2023-11-08T16:14:08.248 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr 2023-11-08T16:14:08.248 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-08T16:14:08.248 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/atp870u.ko 2023-11-08T16:14:08.248 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi 2023-11-08T16:14:08.248 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-08T16:14:08.249 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa 2023-11-08T16:14:08.249 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bfa/bfa.ko 2023-11-08T16:14:08.249 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc 2023-11-08T16:14:08.249 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-08T16:14:08.249 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i 2023-11-08T16:14:08.250 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-08T16:14:08.250 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ch.ko 2023-11-08T16:14:08.250 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor 2023-11-08T16:14:08.250 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-08T16:14:08.250 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi 2023-11-08T16:14:08.251 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-08T16:14:08.251 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-08T16:14:08.251 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-08T16:14:08.251 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-08T16:14:08.252 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-08T16:14:08.252 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dc395x.ko 2023-11-08T16:14:08.252 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler 2023-11-08T16:14:08.252 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-08T16:14:08.252 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-08T16:14:08.252 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-08T16:14:08.253 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-08T16:14:08.253 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/dmx3191d.ko 2023-11-08T16:14:08.253 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r 2023-11-08T16:14:08.253 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-08T16:14:08.253 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/esp_scsi.ko 2023-11-08T16:14:08.254 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe 2023-11-08T16:14:08.254 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-08T16:14:08.254 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-08T16:14:08.254 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic 2023-11-08T16:14:08.255 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/fnic/fnic.ko 2023-11-08T16:14:08.255 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hpsa.ko 2023-11-08T16:14:08.255 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hptiop.ko 2023-11-08T16:14:08.255 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/hv_storvsc.ko 2023-11-08T16:14:08.255 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/initio.ko 2023-11-08T16:14:08.256 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ipr.ko 2023-11-08T16:14:08.256 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ips.ko 2023-11-08T16:14:08.256 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci 2023-11-08T16:14:08.256 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/isci/isci.ko 2023-11-08T16:14:08.256 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-08T16:14:08.257 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-08T16:14:08.257 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc 2023-11-08T16:14:08.257 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libfc/libfc.ko 2023-11-08T16:14:08.257 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi.ko 2023-11-08T16:14:08.257 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-08T16:14:08.258 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas 2023-11-08T16:14:08.258 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/libsas/libsas.ko 2023-11-08T16:14:08.258 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc 2023-11-08T16:14:08.258 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-08T16:14:08.258 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid 2023-11-08T16:14:08.259 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid.ko 2023-11-08T16:14:08.259 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-08T16:14:08.259 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-08T16:14:08.259 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-08T16:14:08.259 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas 2023-11-08T16:14:08.260 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-08T16:14:08.260 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas 2023-11-08T16:14:08.260 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-08T16:14:08.260 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/mvumi.ko 2023-11-08T16:14:08.260 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001 2023-11-08T16:14:08.261 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-08T16:14:08.261 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/pmcraid.ko 2023-11-08T16:14:08.261 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla1280.ko 2023-11-08T16:14:08.261 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx 2023-11-08T16:14:08.261 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-08T16:14:08.262 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-08T16:14:08.262 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx 2023-11-08T16:14:08.262 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-08T16:14:08.262 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/raid_class.ko 2023-11-08T16:14:08.262 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_debug.ko 2023-11-08T16:14:08.263 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-08T16:14:08.263 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-08T16:14:08.263 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-08T16:14:08.263 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-08T16:14:08.263 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-08T16:14:08.264 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/ses.ko 2023-11-08T16:14:08.264 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/st.ko 2023-11-08T16:14:08.264 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/stex.ko 2023-11-08T16:14:08.264 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2 2023-11-08T16:14:08.264 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-08T16:14:08.265 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/virtio_scsi.ko 2023-11-08T16:14:08.265 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-08T16:14:08.265 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/scsi/wd719x.ko 2023-11-08T16:14:08.265 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb 2023-11-08T16:14:08.266 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ssb/ssb.ko 2023-11-08T16:14:08.266 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target 2023-11-08T16:14:08.266 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi 2023-11-08T16:14:08.266 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-08T16:14:08.266 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback 2023-11-08T16:14:08.267 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-08T16:14:08.267 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_file.ko 2023-11-08T16:14:08.267 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_iblock.ko 2023-11-08T16:14:08.267 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_mod.ko 2023-11-08T16:14:08.267 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_pscsi.ko 2023-11-08T16:14:08.268 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/target_core_user.ko 2023-11-08T16:14:08.268 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc 2023-11-08T16:14:08.268 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-08T16:14:08.268 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal 2023-11-08T16:14:08.269 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel 2023-11-08T16:14:08.269 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal 2023-11-08T16:14:08.269 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-08T16:14:08.269 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-08T16:14:08.269 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-08T16:14:08.269 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-08T16:14:08.270 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-08T16:14:08.270 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-08T16:14:08.270 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-08T16:14:08.270 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-08T16:14:08.270 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-08T16:14:08.271 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-08T16:14:08.271 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-08T16:14:08.271 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-08T16:14:08.271 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-08T16:14:08.271 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-08T16:14:08.272 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-08T16:14:08.272 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty 2023-11-08T16:14:08.272 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless 2023-11-08T16:14:08.272 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-08T16:14:08.272 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_gsm.ko 2023-11-08T16:14:08.273 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/n_hdlc.ko 2023-11-08T16:14:08.273 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/nozomi.ko 2023-11-08T16:14:08.273 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial 2023-11-08T16:14:08.273 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250 2023-11-08T16:14:08.273 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-08T16:14:08.274 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/arc_uart.ko 2023-11-08T16:14:08.274 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm 2023-11-08T16:14:08.274 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-08T16:14:08.274 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs 2023-11-08T16:14:08.274 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core 2023-11-08T16:14:08.275 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-08T16:14:08.275 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host 2023-11-08T16:14:08.275 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-08T16:14:08.275 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio 2023-11-08T16:14:08.275 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio.ko 2023-11-08T16:14:08.276 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_aec.ko 2023-11-08T16:14:08.276 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_cif.ko 2023-11-08T16:14:08.276 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_pci_generic.ko 2023-11-08T16:14:08.276 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/uio/uio_sercos3.ko 2023-11-08T16:14:08.276 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb 2023-11-08T16:14:08.277 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class 2023-11-08T16:14:08.277 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-acm.ko 2023-11-08T16:14:08.277 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-08T16:14:08.277 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usblp.ko 2023-11-08T16:14:08.277 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/class/usbtmc.ko 2023-11-08T16:14:08.277 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host 2023-11-08T16:14:08.278 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-08T16:14:08.278 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image 2023-11-08T16:14:08.278 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/mdc800.ko 2023-11-08T16:14:08.278 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/image/microtek.ko 2023-11-08T16:14:08.279 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc 2023-11-08T16:14:08.279 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/adutux.ko 2023-11-08T16:14:08.279 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/appledisplay.ko 2023-11-08T16:14:08.279 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi26.ko 2023-11-08T16:14:08.279 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/emi62.ko 2023-11-08T16:14:08.280 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ezusb.ko 2023-11-08T16:14:08.280 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/idmouse.ko 2023-11-08T16:14:08.280 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/iowarrior.ko 2023-11-08T16:14:08.280 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-08T16:14:08.280 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/ldusb.ko 2023-11-08T16:14:08.280 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/legousbtower.ko 2023-11-08T16:14:08.281 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga 2023-11-08T16:14:08.281 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-08T16:14:08.281 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-08T16:14:08.281 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usb3503.ko 2023-11-08T16:14:08.281 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usblcd.ko 2023-11-08T16:14:08.282 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-08T16:14:08.282 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/uss720.ko 2023-11-08T16:14:08.282 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/misc/yurex.ko 2023-11-08T16:14:08.282 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy 2023-11-08T16:14:08.282 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/phy/phy-generic.ko 2023-11-08T16:14:08.283 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial 2023-11-08T16:14:08.283 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/aircable.ko 2023-11-08T16:14:08.283 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ark3116.ko 2023-11-08T16:14:08.283 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-08T16:14:08.283 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ch341.ko 2023-11-08T16:14:08.284 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cp210x.ko 2023-11-08T16:14:08.284 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cyberjack.ko 2023-11-08T16:14:08.284 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-08T16:14:08.284 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-08T16:14:08.284 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/empeg.ko 2023-11-08T16:14:08.284 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-08T16:14:08.285 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-08T16:14:08.285 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-08T16:14:08.285 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/io_ti.ko 2023-11-08T16:14:08.285 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipaq.ko 2023-11-08T16:14:08.285 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ipw.ko 2023-11-08T16:14:08.286 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ir-usb.ko 2023-11-08T16:14:08.286 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-08T16:14:08.286 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan.ko 2023-11-08T16:14:08.286 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-08T16:14:08.287 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-08T16:14:08.287 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-08T16:14:08.287 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mct_u232.ko 2023-11-08T16:14:08.287 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7720.ko 2023-11-08T16:14:08.287 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/mos7840.ko 2023-11-08T16:14:08.288 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/navman.ko 2023-11-08T16:14:08.288 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/omninet.ko 2023-11-08T16:14:08.288 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/opticon.ko 2023-11-08T16:14:08.288 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/option.ko 2023-11-08T16:14:08.288 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/oti6858.ko 2023-11-08T16:14:08.289 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/pl2303.ko 2023-11-08T16:14:08.289 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcaux.ko 2023-11-08T16:14:08.289 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/qcserial.ko 2023-11-08T16:14:08.289 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/quatech2.ko 2023-11-08T16:14:08.289 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/safe_serial.ko 2023-11-08T16:14:08.290 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/sierra.ko 2023-11-08T16:14:08.290 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-08T16:14:08.290 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ssu100.ko 2023-11-08T16:14:08.290 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/symbolserial.ko 2023-11-08T16:14:08.290 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-08T16:14:08.291 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-08T16:14:08.291 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_debug.ko 2023-11-08T16:14:08.291 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-08T16:14:08.291 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/visor.ko 2023-11-08T16:14:08.291 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/whiteheat.ko 2023-11-08T16:14:08.291 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-08T16:14:08.292 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage 2023-11-08T16:14:08.292 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/uas.ko 2023-11-08T16:14:08.292 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-08T16:14:08.292 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-08T16:14:08.293 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-08T16:14:08.293 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-08T16:14:08.293 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-08T16:14:08.293 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-08T16:14:08.293 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-08T16:14:08.293 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-karma.ko 2023-11-08T16:14:08.294 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-08T16:14:08.294 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-08T16:14:08.294 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-08T16:14:08.294 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-08T16:14:08.294 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-08T16:14:08.295 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/storage/usb-storage.ko 2023-11-08T16:14:08.295 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip 2023-11-08T16:14:08.295 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-08T16:14:08.295 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-08T16:14:08.295 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-08T16:14:08.296 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio 2023-11-08T16:14:08.296 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci 2023-11-08T16:14:08.296 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-08T16:14:08.296 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-08T16:14:08.297 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio.ko 2023-11-08T16:14:08.297 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-08T16:14:08.297 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost 2023-11-08T16:14:08.297 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost.ko 2023-11-08T16:14:08.297 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-08T16:14:08.298 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_net.ko 2023-11-08T16:14:08.298 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/vhost/vhost_scsi.ko 2023-11-08T16:14:08.298 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video 2023-11-08T16:14:08.298 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight 2023-11-08T16:14:08.298 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/apple_bl.ko 2023-11-08T16:14:08.299 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/lcd.ko 2023-11-08T16:14:08.299 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-08T16:14:08.299 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio 2023-11-08T16:14:08.299 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio.ko 2023-11-08T16:14:08.299 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_balloon.ko 2023-11-08T16:14:08.300 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_mmio.ko 2023-11-08T16:14:08.300 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci.ko 2023-11-08T16:14:08.300 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-08T16:14:08.300 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-08T16:14:08.300 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/virtio/virtio_ring.ko 2023-11-08T16:14:08.301 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1 2023-11-08T16:14:08.301 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters 2023-11-08T16:14:08.301 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2482.ko 2023-11-08T16:14:08.301 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/masters/ds2490.ko 2023-11-08T16:14:08.301 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves 2023-11-08T16:14:08.302 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-08T16:14:08.302 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-08T16:14:08.302 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-08T16:14:08.302 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-08T16:14:08.302 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-08T16:14:08.303 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-08T16:14:08.303 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-08T16:14:08.303 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-08T16:14:08.303 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-08T16:14:08.303 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-08T16:14:08.304 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-08T16:14:08.304 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/w1/wire.ko 2023-11-08T16:14:08.304 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog 2023-11-08T16:14:08.304 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-08T16:14:08.304 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-08T16:14:08.305 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-08T16:14:08.305 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/hpwdt.ko 2023-11-08T16:14:08.305 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/i6300esb.ko 2023-11-08T16:14:08.305 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-08T16:14:08.306 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-08T16:14:08.306 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ib700wdt.ko 2023-11-08T16:14:08.306 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ibmasr.ko 2023-11-08T16:14:08.306 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-08T16:14:08.306 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-08T16:14:08.307 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/it87_wdt.ko 2023-11-08T16:14:08.307 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/machzwd.ko 2023-11-08T16:14:08.307 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/nv_tco.ko 2023-11-08T16:14:08.307 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-08T16:14:08.307 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-08T16:14:08.308 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-08T16:14:08.308 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-08T16:14:08.308 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/softdog.ko 2023-11-08T16:14:08.308 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-08T16:14:08.308 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/via_wdt.ko 2023-11-08T16:14:08.309 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-08T16:14:08.309 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-08T16:14:08.309 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-08T16:14:08.309 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/wdt_pci.ko 2023-11-08T16:14:08.309 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/watchdog/xen_wdt.ko 2023-11-08T16:14:08.309 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen 2023-11-08T16:14:08.310 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-08T16:14:08.310 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-evtchn.ko 2023-11-08T16:14:08.310 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntalloc.ko 2023-11-08T16:14:08.310 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-gntdev.ko 2023-11-08T16:14:08.310 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback 2023-11-08T16:14:08.311 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-08T16:14:08.311 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xen-privcmd.ko 2023-11-08T16:14:08.311 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs 2023-11-08T16:14:08.311 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-08T16:14:08.311 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs 2023-11-08T16:14:08.312 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p 2023-11-08T16:14:08.312 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/9p/9p.ko 2023-11-08T16:14:08.312 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs 2023-11-08T16:14:08.312 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/affs/affs.ko 2023-11-08T16:14:08.313 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs 2023-11-08T16:14:08.313 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/befs/befs.ko 2023-11-08T16:14:08.313 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/binfmt_misc.ko 2023-11-08T16:14:08.313 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs 2023-11-08T16:14:08.313 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/btrfs/btrfs.ko 2023-11-08T16:14:08.314 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles 2023-11-08T16:14:08.314 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cachefiles/cachefiles.ko 2023-11-08T16:14:08.314 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph 2023-11-08T16:14:08.314 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ceph/ceph.ko 2023-11-08T16:14:08.314 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda 2023-11-08T16:14:08.315 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/coda/coda.ko 2023-11-08T16:14:08.315 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs 2023-11-08T16:14:08.315 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/cramfs/cramfs.ko 2023-11-08T16:14:08.315 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm 2023-11-08T16:14:08.315 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/dlm/dlm.ko 2023-11-08T16:14:08.316 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs 2023-11-08T16:14:08.316 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-08T16:14:08.316 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs 2023-11-08T16:14:08.316 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/f2fs/f2fs.ko 2023-11-08T16:14:08.316 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat 2023-11-08T16:14:08.316 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/fat.ko 2023-11-08T16:14:08.317 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/msdos.ko 2023-11-08T16:14:08.317 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fat/vfat.ko 2023-11-08T16:14:08.317 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache 2023-11-08T16:14:08.317 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fscache/fscache.ko 2023-11-08T16:14:08.317 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse 2023-11-08T16:14:08.318 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/cuse.ko 2023-11-08T16:14:08.318 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/fuse/fuse.ko 2023-11-08T16:14:08.318 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2 2023-11-08T16:14:08.318 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/gfs2/gfs2.ko 2023-11-08T16:14:08.319 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs 2023-11-08T16:14:08.319 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfs/hfs.ko 2023-11-08T16:14:08.319 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus 2023-11-08T16:14:08.319 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/hfsplus/hfsplus.ko 2023-11-08T16:14:08.319 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs 2023-11-08T16:14:08.320 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/isofs/isofs.ko 2023-11-08T16:14:08.320 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs 2023-11-08T16:14:08.320 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/jfs/jfs.ko 2023-11-08T16:14:08.320 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd 2023-11-08T16:14:08.320 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/lockd/lockd.ko 2023-11-08T16:14:08.321 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix 2023-11-08T16:14:08.321 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/minix/minix.ko 2023-11-08T16:14:08.321 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs 2023-11-08T16:14:08.321 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/netfs/netfs.ko 2023-11-08T16:14:08.321 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs 2023-11-08T16:14:08.322 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout 2023-11-08T16:14:08.322 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-08T16:14:08.322 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout 2023-11-08T16:14:08.322 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-08T16:14:08.323 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout 2023-11-08T16:14:08.323 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-08T16:14:08.323 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfs.ko 2023-11-08T16:14:08.323 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv3.ko 2023-11-08T16:14:08.323 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs/nfsv4.ko 2023-11-08T16:14:08.323 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common 2023-11-08T16:14:08.324 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/grace.ko 2023-11-08T16:14:08.324 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfs_common/nfs_acl.ko 2023-11-08T16:14:08.324 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd 2023-11-08T16:14:08.324 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nfsd/nfsd.ko 2023-11-08T16:14:08.324 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2 2023-11-08T16:14:08.325 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nilfs2/nilfs2.ko 2023-11-08T16:14:08.325 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls 2023-11-08T16:14:08.325 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-celtic.ko 2023-11-08T16:14:08.325 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-centeuro.ko 2023-11-08T16:14:08.325 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-croatian.ko 2023-11-08T16:14:08.326 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-cyrillic.ko 2023-11-08T16:14:08.326 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-gaelic.ko 2023-11-08T16:14:08.326 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-greek.ko 2023-11-08T16:14:08.326 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-iceland.ko 2023-11-08T16:14:08.326 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-inuit.ko 2023-11-08T16:14:08.327 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-roman.ko 2023-11-08T16:14:08.327 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-romanian.ko 2023-11-08T16:14:08.327 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/mac-turkish.ko 2023-11-08T16:14:08.327 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1250.ko 2023-11-08T16:14:08.327 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1251.ko 2023-11-08T16:14:08.328 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp1255.ko 2023-11-08T16:14:08.328 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp737.ko 2023-11-08T16:14:08.328 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp775.ko 2023-11-08T16:14:08.328 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp850.ko 2023-11-08T16:14:08.328 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp852.ko 2023-11-08T16:14:08.328 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp855.ko 2023-11-08T16:14:08.329 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp857.ko 2023-11-08T16:14:08.329 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp860.ko 2023-11-08T16:14:08.329 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp861.ko 2023-11-08T16:14:08.329 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp862.ko 2023-11-08T16:14:08.329 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp863.ko 2023-11-08T16:14:08.330 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp864.ko 2023-11-08T16:14:08.330 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp865.ko 2023-11-08T16:14:08.330 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp866.ko 2023-11-08T16:14:08.330 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp869.ko 2023-11-08T16:14:08.330 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp874.ko 2023-11-08T16:14:08.331 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp932.ko 2023-11-08T16:14:08.331 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp936.ko 2023-11-08T16:14:08.331 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp949.ko 2023-11-08T16:14:08.331 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_cp950.ko 2023-11-08T16:14:08.331 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_euc-jp.ko 2023-11-08T16:14:08.332 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-1.ko 2023-11-08T16:14:08.332 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-13.ko 2023-11-08T16:14:08.332 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-14.ko 2023-11-08T16:14:08.332 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-15.ko 2023-11-08T16:14:08.332 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-2.ko 2023-11-08T16:14:08.333 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-3.ko 2023-11-08T16:14:08.333 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-4.ko 2023-11-08T16:14:08.333 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-5.ko 2023-11-08T16:14:08.333 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-6.ko 2023-11-08T16:14:08.333 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-7.ko 2023-11-08T16:14:08.333 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_iso8859-9.ko 2023-11-08T16:14:08.334 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-r.ko 2023-11-08T16:14:08.334 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-ru.ko 2023-11-08T16:14:08.334 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_koi8-u.ko 2023-11-08T16:14:08.334 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-08T16:14:08.334 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/nls/nls_utf8.ko 2023-11-08T16:14:08.335 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2 2023-11-08T16:14:08.335 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster 2023-11-08T16:14:08.335 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-08T16:14:08.335 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm 2023-11-08T16:14:08.335 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-08T16:14:08.336 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs 2023-11-08T16:14:08.336 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-08T16:14:08.336 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2.ko 2023-11-08T16:14:08.336 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-08T16:14:08.336 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-08T16:14:08.337 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-08T16:14:08.337 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs 2023-11-08T16:14:08.337 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/overlayfs/overlay.ko 2023-11-08T16:14:08.337 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore 2023-11-08T16:14:08.337 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/pstore/ramoops.ko 2023-11-08T16:14:08.338 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs 2023-11-08T16:14:08.338 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/reiserfs/reiserfs.ko 2023-11-08T16:14:08.338 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs 2023-11-08T16:14:08.338 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/romfs/romfs.ko 2023-11-08T16:14:08.338 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb 2023-11-08T16:14:08.339 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client 2023-11-08T16:14:08.339 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/client/cifs.ko 2023-11-08T16:14:08.339 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common 2023-11-08T16:14:08.339 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_arc4.ko 2023-11-08T16:14:08.339 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/smb/common/cifs_md4.ko 2023-11-08T16:14:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs 2023-11-08T16:14:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/squashfs/squashfs.ko 2023-11-08T16:14:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv 2023-11-08T16:14:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/sysv/sysv.ko 2023-11-08T16:14:08.340 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf 2023-11-08T16:14:08.341 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/udf/udf.ko 2023-11-08T16:14:08.341 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs 2023-11-08T16:14:08.341 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/ufs/ufs.ko 2023-11-08T16:14:08.341 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs 2023-11-08T16:14:08.342 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/fs/xfs/xfs.ko 2023-11-08T16:14:08.342 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel 2023-11-08T16:14:08.342 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu 2023-11-08T16:14:08.342 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/rcu/rcutorture.ko 2023-11-08T16:14:08.342 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/torture.ko 2023-11-08T16:14:08.343 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace 2023-11-08T16:14:08.343 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-08T16:14:08.343 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib 2023-11-08T16:14:08.343 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/asn1_encoder.ko 2023-11-08T16:14:08.343 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc-itu-t.ko 2023-11-08T16:14:08.343 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crc8.ko 2023-11-08T16:14:08.344 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto 2023-11-08T16:14:08.344 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libarc4.ko 2023-11-08T16:14:08.344 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/crypto/libdes.ko 2023-11-08T16:14:08.344 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/libcrc32c.ko 2023-11-08T16:14:08.345 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lru_cache.ko 2023-11-08T16:14:08.345 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4 2023-11-08T16:14:08.345 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4_compress.ko 2023-11-08T16:14:08.345 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/lz4/lz4hc_compress.ko 2023-11-08T16:14:08.345 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math 2023-11-08T16:14:08.346 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/math/cordic.ko 2023-11-08T16:14:08.346 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6 2023-11-08T16:14:08.346 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/raid6/raid6_pq.ko 2023-11-08T16:14:08.346 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon 2023-11-08T16:14:08.346 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-08T16:14:08.347 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_bm.ko 2023-11-08T16:14:08.347 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_fsm.ko 2023-11-08T16:14:08.347 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/ts_kmp.ko 2023-11-08T16:14:08.347 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd 2023-11-08T16:14:08.347 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/lib/zstd/zstd_compress.ko 2023-11-08T16:14:08.347 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm 2023-11-08T16:14:08.348 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/mm/hwpoison-inject.ko 2023-11-08T16:14:08.348 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net 2023-11-08T16:14:08.348 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan 2023-11-08T16:14:08.348 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/6lowpan.ko 2023-11-08T16:14:08.348 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_dest.ko 2023-11-08T16:14:08.349 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_fragment.ko 2023-11-08T16:14:08.349 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_hop.ko 2023-11-08T16:14:08.349 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-08T16:14:08.349 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_mobility.ko 2023-11-08T16:14:08.349 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_routing.ko 2023-11-08T16:14:08.350 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/6lowpan/nhc_udp.ko 2023-11-08T16:14:08.350 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802 2023-11-08T16:14:08.350 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/garp.ko 2023-11-08T16:14:08.350 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/mrp.ko 2023-11-08T16:14:08.350 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/p8022.ko 2023-11-08T16:14:08.351 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/psnap.ko 2023-11-08T16:14:08.351 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/802/stp.ko 2023-11-08T16:14:08.351 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q 2023-11-08T16:14:08.351 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/8021q/8021q.ko 2023-11-08T16:14:08.351 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p 2023-11-08T16:14:08.352 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet.ko 2023-11-08T16:14:08.352 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_fd.ko 2023-11-08T16:14:08.352 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_rdma.ko 2023-11-08T16:14:08.352 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/9p/9pnet_virtio.ko 2023-11-08T16:14:08.352 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv 2023-11-08T16:14:08.353 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/batman-adv/batman-adv.ko 2023-11-08T16:14:08.353 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge 2023-11-08T16:14:08.353 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/br_netfilter.ko 2023-11-08T16:14:08.353 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/bridge.ko 2023-11-08T16:14:08.353 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter 2023-11-08T16:14:08.354 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-08T16:14:08.354 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-08T16:14:08.354 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-08T16:14:08.354 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-08T16:14:08.354 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-08T16:14:08.355 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-08T16:14:08.355 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-08T16:14:08.355 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-08T16:14:08.355 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-08T16:14:08.355 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-08T16:14:08.356 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-08T16:14:08.356 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-08T16:14:08.356 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-08T16:14:08.356 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-08T16:14:08.356 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-08T16:14:08.357 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-08T16:14:08.357 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-08T16:14:08.357 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-08T16:14:08.357 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-08T16:14:08.357 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-08T16:14:08.358 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/ebtables.ko 2023-11-08T16:14:08.358 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-08T16:14:08.358 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-08T16:14:08.358 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-08T16:14:08.358 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph 2023-11-08T16:14:08.359 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ceph/libceph.ko 2023-11-08T16:14:08.359 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core 2023-11-08T16:14:08.359 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/failover.ko 2023-11-08T16:14:08.359 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/core/pktgen.ko 2023-11-08T16:14:08.359 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp 2023-11-08T16:14:08.359 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp.ko 2023-11-08T16:14:08.360 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_diag.ko 2023-11-08T16:14:08.360 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv4.ko 2023-11-08T16:14:08.360 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dccp/dccp_ipv6.ko 2023-11-08T16:14:08.360 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver 2023-11-08T16:14:08.360 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dns_resolver/dns_resolver.ko 2023-11-08T16:14:08.361 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa 2023-11-08T16:14:08.361 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/dsa_core.ko 2023-11-08T16:14:08.361 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_brcm.ko 2023-11-08T16:14:08.361 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_dsa.ko 2023-11-08T16:14:08.361 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_none.ko 2023-11-08T16:14:08.362 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/dsa/tag_trailer.ko 2023-11-08T16:14:08.362 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154 2023-11-08T16:14:08.362 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan 2023-11-08T16:14:08.362 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-08T16:14:08.363 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154.ko 2023-11-08T16:14:08.363 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-08T16:14:08.363 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4 2023-11-08T16:14:08.363 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ah4.ko 2023-11-08T16:14:08.363 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/esp4.ko 2023-11-08T16:14:08.363 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/fou.ko 2023-11-08T16:14:08.364 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/gre.ko 2023-11-08T16:14:08.364 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/inet_diag.ko 2023-11-08T16:14:08.364 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_gre.ko 2023-11-08T16:14:08.364 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_tunnel.ko 2023-11-08T16:14:08.364 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ip_vti.ko 2023-11-08T16:14:08.365 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipcomp.ko 2023-11-08T16:14:08.365 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/ipip.ko 2023-11-08T16:14:08.365 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter 2023-11-08T16:14:08.365 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-08T16:14:08.365 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-08T16:14:08.366 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-08T16:14:08.366 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-08T16:14:08.366 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-08T16:14:08.366 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-08T16:14:08.366 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-08T16:14:08.367 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-08T16:14:08.367 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-08T16:14:08.367 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-08T16:14:08.367 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-08T16:14:08.367 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-08T16:14:08.368 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-08T16:14:08.368 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-08T16:14:08.368 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-08T16:14:08.368 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-08T16:14:08.368 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-08T16:14:08.369 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-08T16:14:08.369 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-08T16:14:08.369 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-08T16:14:08.369 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-08T16:14:08.369 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_bic.ko 2023-11-08T16:14:08.370 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_dctcp.ko 2023-11-08T16:14:08.370 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_diag.ko 2023-11-08T16:14:08.370 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_highspeed.ko 2023-11-08T16:14:08.370 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_htcp.ko 2023-11-08T16:14:08.370 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_hybla.ko 2023-11-08T16:14:08.371 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_illinois.ko 2023-11-08T16:14:08.371 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_lp.ko 2023-11-08T16:14:08.371 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_scalable.ko 2023-11-08T16:14:08.371 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_vegas.ko 2023-11-08T16:14:08.371 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_veno.ko 2023-11-08T16:14:08.372 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_westwood.ko 2023-11-08T16:14:08.372 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tcp_yeah.ko 2023-11-08T16:14:08.372 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/tunnel4.ko 2023-11-08T16:14:08.372 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_diag.ko 2023-11-08T16:14:08.372 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/udp_tunnel.ko 2023-11-08T16:14:08.373 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-08T16:14:08.373 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6 2023-11-08T16:14:08.373 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ah6.ko 2023-11-08T16:14:08.373 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/esp6.ko 2023-11-08T16:14:08.373 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/fou6.ko 2023-11-08T16:14:08.374 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_tunnel.ko 2023-11-08T16:14:08.374 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-08T16:14:08.374 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ip6_vti.ko 2023-11-08T16:14:08.374 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/ipcomp6.ko 2023-11-08T16:14:08.374 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter 2023-11-08T16:14:08.375 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-08T16:14:08.375 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-08T16:14:08.375 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-08T16:14:08.375 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-08T16:14:08.375 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-08T16:14:08.376 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-08T16:14:08.376 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-08T16:14:08.376 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-08T16:14:08.376 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-08T16:14:08.376 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-08T16:14:08.377 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-08T16:14:08.377 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-08T16:14:08.377 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-08T16:14:08.377 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-08T16:14:08.377 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-08T16:14:08.378 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-08T16:14:08.378 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-08T16:14:08.378 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-08T16:14:08.378 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-08T16:14:08.378 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-08T16:14:08.379 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-08T16:14:08.379 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-08T16:14:08.379 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-08T16:14:08.379 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-08T16:14:08.379 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/sit.ko 2023-11-08T16:14:08.380 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/tunnel6.ko 2023-11-08T16:14:08.380 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-08T16:14:08.380 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key 2023-11-08T16:14:08.380 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/key/af_key.ko 2023-11-08T16:14:08.380 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp 2023-11-08T16:14:08.381 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_core.ko 2023-11-08T16:14:08.381 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-08T16:14:08.381 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_eth.ko 2023-11-08T16:14:08.381 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip.ko 2023-11-08T16:14:08.381 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ip6.ko 2023-11-08T16:14:08.382 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_netlink.ko 2023-11-08T16:14:08.382 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/l2tp/l2tp_ppp.ko 2023-11-08T16:14:08.382 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc 2023-11-08T16:14:08.382 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/llc/llc.ko 2023-11-08T16:14:08.382 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154 2023-11-08T16:14:08.383 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mac802154/mac802154.ko 2023-11-08T16:14:08.383 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls 2023-11-08T16:14:08.383 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/mpls/mpls_gso.ko 2023-11-08T16:14:08.383 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter 2023-11-08T16:14:08.383 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset 2023-11-08T16:14:08.383 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set.ko 2023-11-08T16:14:08.384 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-08T16:14:08.384 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-08T16:14:08.384 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-08T16:14:08.384 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-08T16:14:08.384 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-08T16:14:08.385 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-08T16:14:08.385 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-08T16:14:08.385 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-08T16:14:08.385 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-08T16:14:08.385 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-08T16:14:08.386 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-08T16:14:08.386 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-08T16:14:08.386 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-08T16:14:08.386 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-08T16:14:08.386 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-08T16:14:08.387 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs 2023-11-08T16:14:08.387 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-08T16:14:08.387 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-08T16:14:08.387 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-08T16:14:08.387 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-08T16:14:08.388 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-08T16:14:08.388 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-08T16:14:08.388 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-08T16:14:08.388 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-08T16:14:08.388 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-08T16:14:08.389 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-08T16:14:08.389 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-08T16:14:08.389 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-08T16:14:08.389 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-08T16:14:08.389 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-08T16:14:08.390 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conncount.ko 2023-11-08T16:14:08.390 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack.ko 2023-11-08T16:14:08.390 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-08T16:14:08.390 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-08T16:14:08.390 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-08T16:14:08.391 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-08T16:14:08.391 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-08T16:14:08.391 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-08T16:14:08.391 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-08T16:14:08.391 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-08T16:14:08.391 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-08T16:14:08.392 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-08T16:14:08.392 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-08T16:14:08.392 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-08T16:14:08.392 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-08T16:14:08.393 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table.ko 2023-11-08T16:14:08.393 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-08T16:14:08.393 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_log_syslog.ko 2023-11-08T16:14:08.393 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat.ko 2023-11-08T16:14:08.393 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-08T16:14:08.393 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-08T16:14:08.394 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_irc.ko 2023-11-08T16:14:08.394 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_sip.ko 2023-11-08T16:14:08.394 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-08T16:14:08.394 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-08T16:14:08.395 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nf_tables.ko 2023-11-08T16:14:08.395 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink.ko 2023-11-08T16:14:08.395 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-08T16:14:08.395 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_log.ko 2023-11-08T16:14:08.395 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-08T16:14:08.396 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-08T16:14:08.396 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_chain_nat.ko 2023-11-08T16:14:08.396 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_compat.ko 2023-11-08T16:14:08.396 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_ct.ko 2023-11-08T16:14:08.396 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-08T16:14:08.397 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fib.ko 2023-11-08T16:14:08.397 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-08T16:14:08.397 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_hash.ko 2023-11-08T16:14:08.397 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_limit.ko 2023-11-08T16:14:08.397 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_log.ko 2023-11-08T16:14:08.397 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_masq.ko 2023-11-08T16:14:08.398 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_nat.ko 2023-11-08T16:14:08.398 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_numgen.ko 2023-11-08T16:14:08.398 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_queue.ko 2023-11-08T16:14:08.398 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_quota.ko 2023-11-08T16:14:08.399 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_redir.ko 2023-11-08T16:14:08.399 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject.ko 2023-11-08T16:14:08.399 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_reject_inet.ko 2023-11-08T16:14:08.399 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_socket.ko 2023-11-08T16:14:08.399 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/nft_tunnel.ko 2023-11-08T16:14:08.400 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_AUDIT.ko 2023-11-08T16:14:08.400 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-08T16:14:08.400 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-08T16:14:08.400 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-08T16:14:08.400 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_CT.ko 2023-11-08T16:14:08.401 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_DSCP.ko 2023-11-08T16:14:08.401 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HL.ko 2023-11-08T16:14:08.401 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_HMARK.ko 2023-11-08T16:14:08.401 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-08T16:14:08.402 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LED.ko 2023-11-08T16:14:08.402 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_LOG.ko 2023-11-08T16:14:08.402 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-08T16:14:08.402 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NETMAP.ko 2023-11-08T16:14:08.402 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFLOG.ko 2023-11-08T16:14:08.403 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-08T16:14:08.403 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_RATEEST.ko 2023-11-08T16:14:08.403 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-08T16:14:08.403 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_SECMARK.ko 2023-11-08T16:14:08.403 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-08T16:14:08.404 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-08T16:14:08.404 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TEE.ko 2023-11-08T16:14:08.404 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TPROXY.ko 2023-11-08T16:14:08.404 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_TRACE.ko 2023-11-08T16:14:08.404 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_addrtype.ko 2023-11-08T16:14:08.405 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_bpf.ko 2023-11-08T16:14:08.405 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cgroup.ko 2023-11-08T16:14:08.405 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cluster.ko 2023-11-08T16:14:08.405 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_comment.ko 2023-11-08T16:14:08.405 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connbytes.ko 2023-11-08T16:14:08.406 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlabel.ko 2023-11-08T16:14:08.406 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connlimit.ko 2023-11-08T16:14:08.406 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_connmark.ko 2023-11-08T16:14:08.406 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_conntrack.ko 2023-11-08T16:14:08.406 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_cpu.ko 2023-11-08T16:14:08.407 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dccp.ko 2023-11-08T16:14:08.407 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_devgroup.ko 2023-11-08T16:14:08.407 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_dscp.ko 2023-11-08T16:14:08.407 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ecn.ko 2023-11-08T16:14:08.407 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_esp.ko 2023-11-08T16:14:08.408 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hashlimit.ko 2023-11-08T16:14:08.408 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_helper.ko 2023-11-08T16:14:08.408 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_hl.ko 2023-11-08T16:14:08.408 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipcomp.ko 2023-11-08T16:14:08.408 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_iprange.ko 2023-11-08T16:14:08.409 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_ipvs.ko 2023-11-08T16:14:08.409 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_l2tp.ko 2023-11-08T16:14:08.409 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_length.ko 2023-11-08T16:14:08.409 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_limit.ko 2023-11-08T16:14:08.409 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mac.ko 2023-11-08T16:14:08.410 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_mark.ko 2023-11-08T16:14:08.410 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_multiport.ko 2023-11-08T16:14:08.410 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nat.ko 2023-11-08T16:14:08.410 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_nfacct.ko 2023-11-08T16:14:08.410 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_osf.ko 2023-11-08T16:14:08.411 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_owner.ko 2023-11-08T16:14:08.411 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_physdev.ko 2023-11-08T16:14:08.411 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_pkttype.ko 2023-11-08T16:14:08.411 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_policy.ko 2023-11-08T16:14:08.411 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_quota.ko 2023-11-08T16:14:08.411 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_rateest.ko 2023-11-08T16:14:08.412 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_realm.ko 2023-11-08T16:14:08.412 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_recent.ko 2023-11-08T16:14:08.412 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_sctp.ko 2023-11-08T16:14:08.412 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_set.ko 2023-11-08T16:14:08.412 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_socket.ko 2023-11-08T16:14:08.413 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_state.ko 2023-11-08T16:14:08.413 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_statistic.ko 2023-11-08T16:14:08.413 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_string.ko 2023-11-08T16:14:08.413 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_tcpmss.ko 2023-11-08T16:14:08.413 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_time.ko 2023-11-08T16:14:08.414 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netfilter/xt_u32.ko 2023-11-08T16:14:08.414 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink 2023-11-08T16:14:08.414 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/netlink/netlink_diag.ko 2023-11-08T16:14:08.414 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh 2023-11-08T16:14:08.414 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/nsh/nsh.ko 2023-11-08T16:14:08.415 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch 2023-11-08T16:14:08.415 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/openvswitch.ko 2023-11-08T16:14:08.415 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-geneve.ko 2023-11-08T16:14:08.415 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-gre.ko 2023-11-08T16:14:08.415 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/openvswitch/vport-vxlan.ko 2023-11-08T16:14:08.416 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet 2023-11-08T16:14:08.416 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/packet/af_packet_diag.ko 2023-11-08T16:14:08.416 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds 2023-11-08T16:14:08.416 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds.ko 2023-11-08T16:14:08.416 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_rdma.ko 2023-11-08T16:14:08.417 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/rds/rds_tcp.ko 2023-11-08T16:14:08.417 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched 2023-11-08T16:14:08.417 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_csum.ko 2023-11-08T16:14:08.417 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_gact.ko 2023-11-08T16:14:08.417 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_ipt.ko 2023-11-08T16:14:08.418 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_mirred.ko 2023-11-08T16:14:08.418 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_nat.ko 2023-11-08T16:14:08.418 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_pedit.ko 2023-11-08T16:14:08.418 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_police.ko 2023-11-08T16:14:08.418 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_simple.ko 2023-11-08T16:14:08.419 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_skbedit.ko 2023-11-08T16:14:08.419 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/act_vlan.ko 2023-11-08T16:14:08.419 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_basic.ko 2023-11-08T16:14:08.419 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_bpf.ko 2023-11-08T16:14:08.419 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_flow.ko 2023-11-08T16:14:08.420 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_fw.ko 2023-11-08T16:14:08.420 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_route.ko 2023-11-08T16:14:08.420 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/cls_u32.ko 2023-11-08T16:14:08.420 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_cmp.ko 2023-11-08T16:14:08.420 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_ipset.ko 2023-11-08T16:14:08.421 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_meta.ko 2023-11-08T16:14:08.421 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_nbyte.ko 2023-11-08T16:14:08.421 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_text.ko 2023-11-08T16:14:08.421 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/em_u32.ko 2023-11-08T16:14:08.421 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_choke.ko 2023-11-08T16:14:08.421 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_codel.ko 2023-11-08T16:14:08.422 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_drr.ko 2023-11-08T16:14:08.422 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_fq.ko 2023-11-08T16:14:08.422 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_gred.ko 2023-11-08T16:14:08.422 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hfsc.ko 2023-11-08T16:14:08.422 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_hhf.ko 2023-11-08T16:14:08.423 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_htb.ko 2023-11-08T16:14:08.423 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_ingress.ko 2023-11-08T16:14:08.423 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio.ko 2023-11-08T16:14:08.423 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_mqprio_lib.ko 2023-11-08T16:14:08.423 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_multiq.ko 2023-11-08T16:14:08.424 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_netem.ko 2023-11-08T16:14:08.424 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_pie.ko 2023-11-08T16:14:08.424 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_plug.ko 2023-11-08T16:14:08.424 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_prio.ko 2023-11-08T16:14:08.424 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_qfq.ko 2023-11-08T16:14:08.425 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_red.ko 2023-11-08T16:14:08.425 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfb.ko 2023-11-08T16:14:08.425 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_sfq.ko 2023-11-08T16:14:08.425 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_tbf.ko 2023-11-08T16:14:08.425 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sched/sch_teql.ko 2023-11-08T16:14:08.426 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp 2023-11-08T16:14:08.426 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp.ko 2023-11-08T16:14:08.426 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sctp/sctp_diag.ko 2023-11-08T16:14:08.426 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc 2023-11-08T16:14:08.426 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss 2023-11-08T16:14:08.427 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-08T16:14:08.427 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-08T16:14:08.427 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/sunrpc.ko 2023-11-08T16:14:08.427 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma 2023-11-08T16:14:08.427 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-08T16:14:08.428 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc 2023-11-08T16:14:08.428 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/diag.ko 2023-11-08T16:14:08.428 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/tipc/tipc.ko 2023-11-08T16:14:08.428 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix 2023-11-08T16:14:08.428 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/unix/unix_diag.ko 2023-11-08T16:14:08.429 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock 2023-11-08T16:14:08.429 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-08T16:14:08.429 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-08T16:14:08.429 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock.ko 2023-11-08T16:14:08.429 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-08T16:14:08.430 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-08T16:14:08.430 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm 2023-11-08T16:14:08.430 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-08T16:14:08.430 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security 2023-11-08T16:14:08.431 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys 2023-11-08T16:14:08.431 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys 2023-11-08T16:14:08.431 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-08T16:14:08.431 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys 2023-11-08T16:14:08.431 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/security/keys/trusted-keys/trusted.ko 2023-11-08T16:14:08.432 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt 2023-11-08T16:14:08.432 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib 2023-11-08T16:14:08.432 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/kernel/virt/lib/irqbypass.ko 2023-11-08T16:14:08.432 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias 2023-11-08T16:14:08.432 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.alias.bin 2023-11-08T16:14:08.433 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin 2023-11-08T16:14:08.433 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.alias.bin 2023-11-08T16:14:08.433 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.bin 2023-11-08T16:14:08.433 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.builtin.modinfo 2023-11-08T16:14:08.433 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep 2023-11-08T16:14:08.434 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.dep.bin 2023-11-08T16:14:08.434 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.devname 2023-11-08T16:14:08.434 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.order 2023-11-08T16:14:08.434 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.softdep 2023-11-08T16:14:08.434 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols 2023-11-08T16:14:08.435 INFO:teuthology.orchestra.run.smithi093.stdout:/lib/modules/6.6.0-rc5-g5b73517e0e2f/modules.symbols.bin 2023-11-08T16:14:08.435 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:14:08.436 DEBUG:teuthology.orchestra.run.smithi093:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-08T16:14:08.459 INFO:teuthology.orchestra.run.smithi093.stdout:postinstall scriptlet (using /bin/sh): 2023-11-08T16:14:08.460 INFO:teuthology.orchestra.run.smithi093.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f -a -r /boot/System.map-6.6.0-rc5-g5b73517e0e2f ]; then 2023-11-08T16:14:08.460 INFO:teuthology.orchestra.run.smithi093.stdout:cp /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T16:14:08.460 INFO:teuthology.orchestra.run.smithi093.stdout:cp /boot/System.map-6.6.0-rc5-g5b73517e0e2f /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T16:14:08.460 INFO:teuthology.orchestra.run.smithi093.stdout:rm -f /boot/vmlinuz-6.6.0-rc5-g5b73517e0e2f /boot/System.map-6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:14:08.461 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/installkernel 6.6.0-rc5-g5b73517e0e2f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T16:14:08.461 INFO:teuthology.orchestra.run.smithi093.stdout:rm -f /boot/.vmlinuz-6.6.0-rc5-g5b73517e0e2f-rpm /boot/.System.map-6.6.0-rc5-g5b73517e0e2f-rpm 2023-11-08T16:14:08.461 INFO:teuthology.orchestra.run.smithi093.stdout:fi 2023-11-08T16:14:08.461 INFO:teuthology.orchestra.run.smithi093.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-08T16:14:08.461 INFO:teuthology.orchestra.run.smithi093.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-08T16:14:08.462 INFO:teuthology.orchestra.run.smithi093.stdout:new-kernel-pkg --remove 6.6.0-rc5-g5b73517e0e2f --rminitrd --initrdfile=/boot/initramfs-6.6.0-rc5-g5b73517e0e2f.img 2023-11-08T16:14:08.462 INFO:teuthology.orchestra.run.smithi093.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-08T16:14:08.462 INFO:teuthology.orchestra.run.smithi093.stdout:kernel-install remove 6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:14:08.462 INFO:teuthology.orchestra.run.smithi093.stdout:fi 2023-11-08T16:14:08.463 INFO:teuthology.orchestra.run.smithi093.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-08T16:14:08.463 INFO:teuthology.orchestra.run.smithi093.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-08T16:14:08.463 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/update-bootloader --remove 6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:14:08.463 INFO:teuthology.orchestra.run.smithi093.stdout:fi 2023-11-08T16:14:08.464 DEBUG:teuthology.orchestra.run.smithi093:> sudo rpm -qi grub2-tools 2023-11-08T16:14:08.541 INFO:teuthology.orchestra.run.smithi093.stdout:Name : grub2-tools 2023-11-08T16:14:08.542 INFO:teuthology.orchestra.run.smithi093.stdout:Epoch : 1 2023-11-08T16:14:08.542 INFO:teuthology.orchestra.run.smithi093.stdout:Version : 2.06 2023-11-08T16:14:08.542 INFO:teuthology.orchestra.run.smithi093.stdout:Release : 68.el9 2023-11-08T16:14:08.543 INFO:teuthology.orchestra.run.smithi093.stdout:Architecture: x86_64 2023-11-08T16:14:08.543 INFO:teuthology.orchestra.run.smithi093.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-08T16:14:08.543 INFO:teuthology.orchestra.run.smithi093.stdout:Group : Unspecified 2023-11-08T16:14:08.543 INFO:teuthology.orchestra.run.smithi093.stdout:Size : 8264254 2023-11-08T16:14:08.543 INFO:teuthology.orchestra.run.smithi093.stdout:License : GPLv3+ 2023-11-08T16:14:08.544 INFO:teuthology.orchestra.run.smithi093.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-08T16:14:08.544 INFO:teuthology.orchestra.run.smithi093.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-08T16:14:08.544 INFO:teuthology.orchestra.run.smithi093.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-08T16:14:08.544 INFO:teuthology.orchestra.run.smithi093.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-08T16:14:08.544 INFO:teuthology.orchestra.run.smithi093.stdout:Packager : builder@centos.org 2023-11-08T16:14:08.544 INFO:teuthology.orchestra.run.smithi093.stdout:Vendor : CentOS 2023-11-08T16:14:08.545 INFO:teuthology.orchestra.run.smithi093.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-08T16:14:08.545 INFO:teuthology.orchestra.run.smithi093.stdout:Summary : Support tools for GRUB. 2023-11-08T16:14:08.545 INFO:teuthology.orchestra.run.smithi093.stdout:Description : 2023-11-08T16:14:08.545 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:14:08.545 INFO:teuthology.orchestra.run.smithi093.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-08T16:14:08.546 INFO:teuthology.orchestra.run.smithi093.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-08T16:14:08.546 INFO:teuthology.orchestra.run.smithi093.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-08T16:14:08.546 INFO:teuthology.orchestra.run.smithi093.stdout:hardware devices. 2023-11-08T16:14:08.546 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:14:08.546 INFO:teuthology.orchestra.run.smithi093.stdout:This subpackage provides tools for support of all platforms. 2023-11-08T16:14:08.548 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-08T16:14:08.548 INFO:teuthology.task.kernel:Updating grub on smithi093 to boot 6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:14:08.548 DEBUG:teuthology.orchestra.run.smithi093:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-08T16:14:09.290 INFO:teuthology.orchestra.run.smithi093.stderr:Generating grub configuration file ... 2023-11-08T16:14:10.804 INFO:teuthology.orchestra.run.smithi093.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-08T16:14:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:done 2023-11-08T16:14:10.825 DEBUG:teuthology.orchestra.run.smithi093:> mktemp 2023-11-08T16:14:10.840 INFO:teuthology.orchestra.run.smithi093.stdout:/tmp/tmp.QjXVV3Wz0y 2023-11-08T16:14:10.840 DEBUG:teuthology.orchestra.run.smithi093:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.QjXVV3Wz0y 2023-11-08T16:14:10.909 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 0666 /tmp/tmp.QjXVV3Wz0y 2023-11-08T16:14:11.038 DEBUG:teuthology.orchestra.remote:smithi093:/tmp/tmp.QjXVV3Wz0y is 6KB 2023-11-08T16:14:11.085 DEBUG:teuthology.orchestra.run.smithi093:> rm -fr /tmp/tmp.QjXVV3Wz0y 2023-11-08T16:14:11.099 DEBUG:teuthology.orchestra.run.smithi093:> sudo /bin/ls /boot/loader/entries || true 2023-11-08T16:14:11.164 INFO:teuthology.orchestra.run.smithi093.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-08T16:14:11.164 INFO:teuthology.orchestra.run.smithi093.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-08T16:14:11.165 INFO:teuthology.orchestra.run.smithi093.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-08T16:14:11.165 INFO:teuthology.orchestra.run.smithi093.stdout:15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f.conf 2023-11-08T16:14:11.166 DEBUG:teuthology.orchestra.run.smithi093:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:14:11.310 DEBUG:teuthology.orchestra.run.smithi093:> sudo shutdown -r now 2023-11-08T16:14:41.342 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-11-08T16:14:41.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-08T16:14:41.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T16:14:55.162 DEBUG:teuthology.orchestra.remote:timed out 2023-11-08T16:15:04.171 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-11-08T16:15:04.172 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T16:15:11.609 DEBUG:teuthology.orchestra.run.smithi081:> true 2023-11-08T16:15:12.284 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-11-08T16:15:12.284 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-rc5-g5b73517e0e2f"... 2023-11-08T16:15:12.284 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2023-11-08T16:15:12.324 INFO:teuthology.orchestra.run.smithi081.stdout:6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:15:12.325 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:15:12.325 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-08T16:15:12.325 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-08T16:15:13.327 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-08T16:15:13.327 DEBUG:teuthology.orchestra.run.smithi081:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-08T16:15:13.455 INFO:teuthology.orchestra.run.smithi081.stdout:ttyS1 2023-11-08T16:15:13.503 DEBUG:teuthology.parallel:result is None 2023-11-08T16:15:41.347 DEBUG:teuthology.orchestra.remote:timed out 2023-11-08T16:15:50.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-08T16:15:50.350 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T16:15:51.298 DEBUG:teuthology.orchestra.run.smithi093:> true 2023-11-08T16:15:51.900 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-08T16:15:51.901 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-rc5-g5b73517e0e2f"... 2023-11-08T16:15:51.901 DEBUG:teuthology.orchestra.run.smithi093:> uname -r 2023-11-08T16:15:51.945 INFO:teuthology.orchestra.run.smithi093.stdout:6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:15:51.945 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-rc5-g5b73517e0e2f vs 6.6.0-rc5-g5b73517e0e2f 2023-11-08T16:15:51.946 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-08T16:15:51.946 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-08T16:15:52.947 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-08T16:15:52.948 DEBUG:teuthology.orchestra.run.smithi093:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-08T16:15:53.166 INFO:teuthology.orchestra.run.smithi093.stdout:ttyS1 2023-11-08T16:15:53.228 DEBUG:teuthology.parallel:result is None 2023-11-08T16:15:53.229 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-08T16:15:53.236 INFO:teuthology.task.internal:Creating test directory... 2023-11-08T16:15:53.237 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-08T16:15:53.248 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-08T16:15:53.299 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-08T16:15:53.306 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-08T16:15:53.315 INFO:teuthology.task.internal:Creating archive directory... 2023-11-08T16:15:53.315 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-08T16:15:53.321 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-08T16:15:53.405 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-08T16:15:53.413 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-08T16:15:53.414 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-08T16:15:53.423 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-08T16:15:53.537 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T16:15:53.547 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T16:15:53.576 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T16:15:53.594 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-08T16:15:53.598 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-08T16:15:53.606 INFO:teuthology.task.internal:Configuring sudo... 2023-11-08T16:15:53.607 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-08T16:15:53.617 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-08T16:15:53.709 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-08T16:15:53.719 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-08T16:15:53.719 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-08T16:15:53.728 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-08T16:15:53.779 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T16:15:53.866 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T16:15:53.973 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T16:15:54.059 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T16:15:54.166 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:15:54.167 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-08T16:15:54.266 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T16:15:54.347 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-08T16:15:54.434 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T16:15:54.503 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-08T16:15:54.641 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:15:54.642 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-08T16:15:54.720 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2023-11-08T16:15:54.729 DEBUG:teuthology.orchestra.run.smithi093:> sudo service rsyslog restart 2023-11-08T16:15:54.870 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T16:15:54.892 INFO:teuthology.orchestra.run.smithi093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T16:15:55.146 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-08T16:15:55.154 INFO:teuthology.task.internal:Starting timer... 2023-11-08T16:15:55.155 INFO:teuthology.run_tasks:Running task pcp... 2023-11-08T16:15:55.171 INFO:teuthology.run_tasks:Running task selinux... 2023-11-08T16:15:55.183 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2023-11-08T16:15:55.320 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2023-11-08T16:15:55.325 DEBUG:teuthology.orchestra.run.smithi093:> sudo service auditd rotate 2023-11-08T16:15:55.443 INFO:teuthology.orchestra.run.smithi093.stdout:Rotating logs: 2023-11-08T16:15:55.447 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-08T16:15:55.447 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2023-11-08T16:15:55.518 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2023-11-08T16:15:55.519 DEBUG:teuthology.orchestra.run.smithi093:> /usr/sbin/getenforce 2023-11-08T16:15:55.579 INFO:teuthology.orchestra.run.smithi093.stdout:Permissive 2023-11-08T16:15:55.580 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi081.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi093.front.sepia.ceph.com': 'permissive'} 2023-11-08T16:15:55.580 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T16:15:55.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:15:55.684 DEBUG:teuthology.orchestra.run.smithi093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T16:15:55.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:15:55.776 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-08T16:15:55.776 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2023-11-08T16:15:55.891 DEBUG:teuthology.orchestra.run.smithi093:> sudo /usr/sbin/setenforce permissive 2023-11-08T16:15:55.985 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-08T16:15:55.998 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-08T16:15:56.009 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-08T16:15:56.133 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-08T16:15:56.151 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-08T16:15:56.153 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi081.front.sepia.ceph.com,smithi093.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-08T16:25:48.130 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi081.front.sepia.ceph.com'), Remote(name='ubuntu@smithi093.front.sepia.ceph.com')] 2023-11-08T16:25:48.132 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-11-08T16:25:48.133 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T16:25:48.274 DEBUG:teuthology.orchestra.run.smithi081:> true 2023-11-08T16:25:48.436 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2023-11-08T16:25:48.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-08T16:25:48.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-08T16:25:48.568 DEBUG:teuthology.orchestra.run.smithi093:> true 2023-11-08T16:25:48.719 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-11-08T16:25:48.720 INFO:teuthology.run_tasks:Running task clock... 2023-11-08T16:25:48.734 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-08T16:25:48.735 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-08T16:25:48.735 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T16:25:48.744 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-08T16:25:48.745 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T16:25:48.844 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-08T16:25:48.848 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-08T16:25:48.898 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-08T16:25:48.901 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-08T16:25:48.990 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2023-11-08T16:25:48.991 INFO:teuthology.orchestra.run.smithi093.stderr:sudo: ntpd: command not found 2023-11-08T16:25:49.031 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2023-11-08T16:25:49.033 INFO:teuthology.orchestra.run.smithi093.stdout:506 Cannot talk to daemon 2023-11-08T16:25:49.081 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-08T16:25:49.085 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-08T16:25:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-08T16:25:49.137 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-08T16:25:49.255 INFO:teuthology.orchestra.run.smithi093.stderr:bash: line 1: ntpq: command not found 2023-11-08T16:25:49.262 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2023-11-08T16:25:49.264 INFO:teuthology.orchestra.run.smithi093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T16:25:49.264 INFO:teuthology.orchestra.run.smithi093.stdout:=============================================================================== 2023-11-08T16:25:49.264 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T16:25:49.265 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T16:25:49.265 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T16:25:49.265 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T16:25:49.270 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T16:25:49.270 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2023-11-08T16:25:49.270 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T16:25:49.270 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T16:25:49.271 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T16:25:49.271 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T16:25:49.272 INFO:teuthology.run_tasks:Running task install... 2023-11-08T16:25:49.283 DEBUG:teuthology.task.install:project ceph 2023-11-08T16:25:49.284 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '85bdb240f5cf2a758acc9b247951fbc8f5799bc1'}} 2023-11-08T16:25:49.284 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '85bdb240f5cf2a758acc9b247951fbc8f5799bc1'} 2023-11-08T16:25:49.284 INFO:teuthology.task.install:Using flavor: default 2023-11-08T16:25:49.290 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-08T16:25:49.291 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-08T16:25:49.291 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '85bdb240f5cf2a758acc9b247951fbc8f5799bc1', 'tag': None, 'wait_for_package': False} 2023-11-08T16:25:49.292 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=85bdb240f5cf2a758acc9b247951fbc8f5799bc1 2023-11-08T16:25:49.295 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '85bdb240f5cf2a758acc9b247951fbc8f5799bc1', 'tag': None, 'wait_for_package': False} 2023-11-08T16:25:49.295 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=85bdb240f5cf2a758acc9b247951fbc8f5799bc1 2023-11-08T16:25:49.435 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231106.073650/85bdb240f5cf2a758acc9b247951fbc8f5799bc1/centos/9/flavors/default/ 2023-11-08T16:25:49.436 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7083.g85bdb240 2023-11-08T16:25:49.588 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231106.073650/85bdb240f5cf2a758acc9b247951fbc8f5799bc1/centos/9/flavors/default/ 2023-11-08T16:25:49.588 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7083.g85bdb240 2023-11-08T16:25:49.704 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231106.073650/85bdb240f5cf2a758acc9b247951fbc8f5799bc1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231106.073650/85bdb240f5cf2a758acc9b247951fbc8f5799bc1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231106.073650/85bdb240f5cf2a758acc9b247951fbc8f5799bc1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-08T16:25:49.704 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:25:49.705 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-08T16:25:49.806 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-08T16:25:49.806 DEBUG:teuthology.orchestra.run.smithi093:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/85bdb240f5cf2a758acc9b247951fbc8f5799bc1/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-08T16:25:49.948 DEBUG:teuthology.orchestra.run.smithi093:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-08T16:25:50.029 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231106.073650/85bdb240f5cf2a758acc9b247951fbc8f5799bc1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231106.073650/85bdb240f5cf2a758acc9b247951fbc8f5799bc1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing-20231106.073650/85bdb240f5cf2a758acc9b247951fbc8f5799bc1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-08T16:25:50.030 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:25:50.030 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-08T16:25:50.082 DEBUG:teuthology.orchestra.run.smithi093:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-08T16:25:50.131 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-08T16:25:50.131 DEBUG:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/85bdb240f5cf2a758acc9b247951fbc8f5799bc1/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-08T16:25:50.173 INFO:teuthology.orchestra.run.smithi093.stdout:check_obsoletes = 1 2023-11-08T16:25:50.177 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum clean all 2023-11-08T16:25:50.274 DEBUG:teuthology.orchestra.run.smithi081:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-08T16:25:50.407 DEBUG:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-08T16:25:50.499 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2023-11-08T16:25:50.503 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum clean all 2023-11-08T16:25:50.532 INFO:teuthology.orchestra.run.smithi093.stdout:72 files removed 2023-11-08T16:25:50.574 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-08T16:25:50.850 INFO:teuthology.orchestra.run.smithi081.stdout:72 files removed 2023-11-08T16:25:50.888 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-08T16:25:51.986 INFO:teuthology.orchestra.run.smithi093.stdout:ceph packages for x86_64 86 kB/s | 78 kB 00:00 2023-11-08T16:25:52.299 INFO:teuthology.orchestra.run.smithi081.stdout:ceph packages for x86_64 86 kB/s | 78 kB 00:00 2023-11-08T16:25:52.978 INFO:teuthology.orchestra.run.smithi093.stdout:ceph noarch packages 11 kB/s | 9.5 kB 00:00 2023-11-08T16:25:53.124 INFO:teuthology.orchestra.run.smithi081.stdout:ceph noarch packages 13 kB/s | 9.5 kB 00:00 2023-11-08T16:25:53.528 INFO:teuthology.orchestra.run.smithi093.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2023-11-08T16:25:53.726 INFO:teuthology.orchestra.run.smithi081.stdout:ceph source packages 3.6 kB/s | 1.9 kB 00:00 2023-11-08T16:25:54.143 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS-9 - AppStream 12 MB/s | 6.3 MB 00:00 2023-11-08T16:25:54.279 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - AppStream 13 MB/s | 6.3 MB 00:00 2023-11-08T16:25:55.707 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS-9 - BaseOS 6.1 MB/s | 2.0 MB 00:00 2023-11-08T16:25:56.013 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - BaseOS 4.7 MB/s | 2.0 MB 00:00 2023-11-08T16:25:56.744 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 7.9 MB 00:00 2023-11-08T16:25:59.004 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS Stream 9 - AppStream 17 MB/s | 18 MB 00:01 2023-11-08T16:26:00.800 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 7.9 MB 00:04 2023-11-08T16:26:02.985 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 19 MB/s | 18 MB 00:00 2023-11-08T16:26:03.213 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS Stream 9 - CRB 11 MB/s | 5.8 MB 00:00 2023-11-08T16:26:05.006 INFO:teuthology.orchestra.run.smithi093.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 15 kB 00:00 2023-11-08T16:26:07.488 INFO:teuthology.orchestra.run.smithi093.stdout:Extra Packages for Enterprise Linux 8.5 MB/s | 19 MB 00:02 2023-11-08T16:26:08.674 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 5.8 MB 00:02 2023-11-08T16:26:10.311 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 15 kB 00:00 2023-11-08T16:26:12.840 INFO:teuthology.orchestra.run.smithi093.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2023-11-08T16:26:13.205 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 7.2 MB/s | 19 MB 00:02 2023-11-08T16:26:15.655 INFO:teuthology.orchestra.run.smithi093.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-08T16:26:15.657 INFO:teuthology.orchestra.run.smithi093.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-08T16:26:15.678 INFO:teuthology.orchestra.run.smithi093.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-08T16:26:15.679 INFO:teuthology.orchestra.run.smithi093.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-08T16:26:15.680 INFO:teuthology.orchestra.run.smithi093.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-08T16:26:15.682 INFO:teuthology.orchestra.run.smithi093.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-08T16:26:15.684 INFO:teuthology.orchestra.run.smithi093.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-08T16:26:15.688 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-08T16:26:15.689 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-08T16:26:15.766 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-11-08T16:26:15.790 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================ 2023-11-08T16:26:15.790 INFO:teuthology.orchestra.run.smithi093.stdout: Package Arch Version Repository Size 2023-11-08T16:26:15.791 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================ 2023-11-08T16:26:15.791 INFO:teuthology.orchestra.run.smithi093.stdout:Installing: 2023-11-08T16:26:15.791 INFO:teuthology.orchestra.run.smithi093.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-08T16:26:15.791 INFO:teuthology.orchestra.run.smithi093.stdout: ceph x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 6.5 k 2023-11-08T16:26:15.791 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 5.2 M 2023-11-08T16:26:15.792 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 858 k 2023-11-08T16:26:15.792 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 140 k 2023-11-08T16:26:15.792 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 1.5 M 2023-11-08T16:26:15.792 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 132 k 2023-11-08T16:26:15.793 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 1.8 M 2023-11-08T16:26:15.793 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 7.4 M 2023-11-08T16:26:15.793 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 50 k 2023-11-08T16:26:15.793 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 8.7 M 2023-11-08T16:26:15.793 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 41 M 2023-11-08T16:26:15.794 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 262 k 2023-11-08T16:26:15.794 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 272 k 2023-11-08T16:26:15.794 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 216 k 2023-11-08T16:26:15.794 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 18 k 2023-11-08T16:26:15.794 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-11-08T16:26:15.795 INFO:teuthology.orchestra.run.smithi093.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-08T16:26:15.795 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-08T16:26:15.795 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 31 k 2023-11-08T16:26:15.795 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2 x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 718 k 2023-11-08T16:26:15.795 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 126 k 2023-11-08T16:26:15.796 INFO:teuthology.orchestra.run.smithi093.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-08T16:26:15.796 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-11-08T16:26:15.796 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-11-08T16:26:15.796 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-08T16:26:15.796 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-08T16:26:15.797 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-08T16:26:15.797 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 162 k 2023-11-08T16:26:15.797 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 321 k 2023-11-08T16:26:15.797 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 297 k 2023-11-08T16:26:15.798 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 99 k 2023-11-08T16:26:15.798 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 84 k 2023-11-08T16:26:15.798 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 3.0 M 2023-11-08T16:26:15.798 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 169 k 2023-11-08T16:26:15.798 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-11-08T16:26:15.799 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrading: 2023-11-08T16:26:15.799 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-08T16:26:15.799 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-08T16:26:15.799 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-08T16:26:15.799 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-08T16:26:15.800 INFO:teuthology.orchestra.run.smithi093.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-08T16:26:15.800 INFO:teuthology.orchestra.run.smithi093.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-08T16:26:15.800 INFO:teuthology.orchestra.run.smithi093.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-08T16:26:15.800 INFO:teuthology.orchestra.run.smithi093.stdout: librados2 x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 3.3 M 2023-11-08T16:26:15.800 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1 x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 3.0 M 2023-11-08T16:26:15.801 INFO:teuthology.orchestra.run.smithi093.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-08T16:26:15.801 INFO:teuthology.orchestra.run.smithi093.stdout:Installing dependencies: 2023-11-08T16:26:15.801 INFO:teuthology.orchestra.run.smithi093.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-11-08T16:26:15.801 INFO:teuthology.orchestra.run.smithi093.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-08T16:26:15.801 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 19 M 2023-11-08T16:26:15.802 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 23 k 2023-11-08T16:26:15.802 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 2.1 M 2023-11-08T16:26:15.802 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 243 k 2023-11-08T16:26:15.802 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 4.4 M 2023-11-08T16:26:15.802 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 18 M 2023-11-08T16:26:15.803 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 15 k 2023-11-08T16:26:15.803 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 24 k 2023-11-08T16:26:15.803 INFO:teuthology.orchestra.run.smithi093.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-08T16:26:15.803 INFO:teuthology.orchestra.run.smithi093.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-08T16:26:15.803 INFO:teuthology.orchestra.run.smithi093.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-08T16:26:15.804 INFO:teuthology.orchestra.run.smithi093.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-08T16:26:15.804 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-08T16:26:15.805 INFO:teuthology.orchestra.run.smithi093.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-08T16:26:15.805 INFO:teuthology.orchestra.run.smithi093.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-08T16:26:15.806 INFO:teuthology.orchestra.run.smithi093.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-08T16:26:15.806 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 162 k 2023-11-08T16:26:15.806 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-08T16:26:15.806 INFO:teuthology.orchestra.run.smithi093.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-08T16:26:15.807 INFO:teuthology.orchestra.run.smithi093.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-08T16:26:15.807 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-08T16:26:15.807 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-11-08T16:26:15.807 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1 x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 476 k 2023-11-08T16:26:15.807 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-08T16:26:15.808 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2 x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 4.8 M 2023-11-08T16:26:15.808 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-08T16:26:15.808 INFO:teuthology.orchestra.run.smithi093.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-08T16:26:15.808 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-08T16:26:15.808 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-08T16:26:15.809 INFO:teuthology.orchestra.run.smithi093.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-08T16:26:15.809 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-08T16:26:15.809 INFO:teuthology.orchestra.run.smithi093.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-08T16:26:15.809 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-08T16:26:15.809 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-08T16:26:15.810 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-08T16:26:15.810 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-08T16:26:15.810 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-08T16:26:15.810 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 45 k 2023-11-08T16:26:15.810 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 127 k 2023-11-08T16:26:15.811 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-08T16:26:15.811 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-08T16:26:15.811 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-08T16:26:15.811 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-08T16:26:15.811 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-11-08T16:26:15.812 INFO:teuthology.orchestra.run.smithi093.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-11-08T16:26:15.812 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-11-08T16:26:15.812 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-08T16:26:15.812 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-08T16:26:15.812 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-08T16:26:15.813 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-08T16:26:15.813 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-08T16:26:15.813 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-08T16:26:15.813 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-08T16:26:15.813 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-08T16:26:15.814 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-08T16:26:15.814 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-08T16:26:15.814 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-08T16:26:15.814 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-08T16:26:15.814 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-08T16:26:15.815 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-08T16:26:15.815 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-08T16:26:15.815 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-08T16:26:15.815 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-08T16:26:15.815 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-08T16:26:15.816 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-08T16:26:15.816 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-08T16:26:15.816 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-08T16:26:15.816 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-08T16:26:15.816 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-08T16:26:15.817 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-08T16:26:15.817 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-08T16:26:15.817 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-08T16:26:15.817 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-11-08T16:26:15.817 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-08T16:26:15.818 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-08T16:26:15.818 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-08T16:26:15.818 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-08T16:26:15.818 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-08T16:26:15.818 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-08T16:26:15.819 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-08T16:26:15.819 INFO:teuthology.orchestra.run.smithi093.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-08T16:26:15.819 INFO:teuthology.orchestra.run.smithi093.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-08T16:26:15.819 INFO:teuthology.orchestra.run.smithi093.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-08T16:26:15.819 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-08T16:26:15.819 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-08T16:26:15.820 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-08T16:26:15.820 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-08T16:26:15.820 INFO:teuthology.orchestra.run.smithi093.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-08T16:26:15.820 INFO:teuthology.orchestra.run.smithi093.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-08T16:26:15.820 INFO:teuthology.orchestra.run.smithi093.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-08T16:26:15.821 INFO:teuthology.orchestra.run.smithi093.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-08T16:26:15.821 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-08T16:26:15.821 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-11-08T16:26:15.821 INFO:teuthology.orchestra.run.smithi093.stdout:Installing weak dependencies: 2023-11-08T16:26:15.821 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-08T16:26:15.822 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:26:15.822 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-11-08T16:26:15.822 INFO:teuthology.orchestra.run.smithi093.stdout:============================================================================================ 2023-11-08T16:26:15.822 INFO:teuthology.orchestra.run.smithi093.stdout:Install 132 Packages 2023-11-08T16:26:15.822 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrade 10 Packages 2023-11-08T16:26:15.823 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:26:15.823 INFO:teuthology.orchestra.run.smithi093.stdout:Total download size: 200 M 2023-11-08T16:26:15.824 INFO:teuthology.orchestra.run.smithi093.stdout:Downloading Packages: 2023-11-08T16:26:16.792 INFO:teuthology.orchestra.run.smithi093.stdout:(1/142): ceph-18.0.0-7083.g85bdb240.el9.x86_64. 18 kB/s | 6.5 kB 00:00 2023-11-08T16:26:17.125 INFO:teuthology.orchestra.run.smithi093.stdout:(2/142): ceph-base-18.0.0-7083.g85bdb240.el9.x8 7.5 MB/s | 5.2 MB 00:00 2023-11-08T16:26:17.176 INFO:teuthology.orchestra.run.smithi093.stdout:(3/142): ceph-immutable-object-cache-18.0.0-708 2.8 MB/s | 140 kB 00:00 2023-11-08T16:26:17.334 INFO:teuthology.orchestra.run.smithi093.stdout:(4/142): ceph-mds-18.0.0-7083.g85bdb240.el9.x86 13 MB/s | 2.1 MB 00:00 2023-11-08T16:26:17.460 INFO:teuthology.orchestra.run.smithi093.stdout:(5/142): ceph-mgr-18.0.0-7083.g85bdb240.el9.x86 12 MB/s | 1.5 MB 00:00 2023-11-08T16:26:17.818 INFO:teuthology.orchestra.run.smithi093.stdout:(6/142): ceph-mon-18.0.0-7083.g85bdb240.el9.x86 12 MB/s | 4.4 MB 00:00 2023-11-08T16:26:18.127 INFO:teuthology.orchestra.run.smithi093.stdout:(7/142): ceph-fuse-18.0.0-7083.g85bdb240.el9.x8 644 kB/s | 858 kB 00:01 2023-11-08T16:26:18.497 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2023-11-08T16:26:19.236 INFO:teuthology.orchestra.run.smithi093.stdout:(8/142): ceph-osd-18.0.0-7083.g85bdb240.el9.x86 13 MB/s | 18 MB 00:01 2023-11-08T16:26:19.302 INFO:teuthology.orchestra.run.smithi093.stdout:(9/142): ceph-selinux-18.0.0-7083.g85bdb240.el9 368 kB/s | 24 kB 00:00 2023-11-08T16:26:21.347 INFO:teuthology.orchestra.run.smithi081.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-08T16:26:21.349 INFO:teuthology.orchestra.run.smithi081.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-08T16:26:21.370 INFO:teuthology.orchestra.run.smithi081.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-08T16:26:21.372 INFO:teuthology.orchestra.run.smithi081.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-08T16:26:21.373 INFO:teuthology.orchestra.run.smithi081.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-08T16:26:21.375 INFO:teuthology.orchestra.run.smithi081.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-08T16:26:21.377 INFO:teuthology.orchestra.run.smithi081.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-08T16:26:21.381 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-08T16:26:21.382 INFO:teuthology.orchestra.run.smithi081.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-08T16:26:21.458 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2023-11-08T16:26:21.480 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================ 2023-11-08T16:26:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2023-11-08T16:26:21.481 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================ 2023-11-08T16:26:21.481 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2023-11-08T16:26:21.481 INFO:teuthology.orchestra.run.smithi081.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-08T16:26:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 6.5 k 2023-11-08T16:26:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 5.2 M 2023-11-08T16:26:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 858 k 2023-11-08T16:26:21.482 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 140 k 2023-11-08T16:26:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 1.5 M 2023-11-08T16:26:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 132 k 2023-11-08T16:26:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 1.8 M 2023-11-08T16:26:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 7.4 M 2023-11-08T16:26:21.483 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 50 k 2023-11-08T16:26:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 8.7 M 2023-11-08T16:26:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 41 M 2023-11-08T16:26:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 262 k 2023-11-08T16:26:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 272 k 2023-11-08T16:26:21.484 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 216 k 2023-11-08T16:26:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 18 k 2023-11-08T16:26:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel x86_64 0.189-3.el9 appstream 22 k 2023-11-08T16:26:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-08T16:26:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-08T16:26:21.485 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 31 k 2023-11-08T16:26:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 718 k 2023-11-08T16:26:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 126 k 2023-11-08T16:26:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-08T16:26:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel x86_64 2.37.4-15.el9 appstream 26 k 2023-11-08T16:26:21.486 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel x86_64 1:3.0.7-24.el9 appstream 4.1 M 2023-11-08T16:26:21.487 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-08T16:26:21.487 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-08T16:26:21.487 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-08T16:26:21.487 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 162 k 2023-11-08T16:26:21.487 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 321 k 2023-11-08T16:26:21.488 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 297 k 2023-11-08T16:26:21.488 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 99 k 2023-11-08T16:26:21.488 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 84 k 2023-11-08T16:26:21.488 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 3.0 M 2023-11-08T16:26:21.488 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 169 k 2023-11-08T16:26:21.489 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 appstream 86 k 2023-11-08T16:26:21.489 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrading: 2023-11-08T16:26:21.489 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-08T16:26:21.489 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-08T16:26:21.489 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-08T16:26:21.490 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-08T16:26:21.490 INFO:teuthology.orchestra.run.smithi081.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-08T16:26:21.490 INFO:teuthology.orchestra.run.smithi081.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-08T16:26:21.490 INFO:teuthology.orchestra.run.smithi081.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-08T16:26:21.490 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 3.3 M 2023-11-08T16:26:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 3.0 M 2023-11-08T16:26:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-08T16:26:21.492 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2023-11-08T16:26:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2023-11-08T16:26:21.492 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-08T16:26:21.493 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 19 M 2023-11-08T16:26:21.493 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 23 k 2023-11-08T16:26:21.493 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 2.1 M 2023-11-08T16:26:21.493 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 243 k 2023-11-08T16:26:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 4.4 M 2023-11-08T16:26:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 18 M 2023-11-08T16:26:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7083.g85bdb240.el9 ceph-noarch 15 k 2023-11-08T16:26:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 24 k 2023-11-08T16:26:21.494 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-08T16:26:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-08T16:26:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-08T16:26:21.495 INFO:teuthology.orchestra.run.smithi081.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-08T16:26:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-08T16:26:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-08T16:26:21.496 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-08T16:26:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-08T16:26:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 162 k 2023-11-08T16:26:21.497 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-08T16:26:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-08T16:26:21.498 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-08T16:26:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-08T16:26:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2023-11-08T16:26:21.499 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 476 k 2023-11-08T16:26:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-08T16:26:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 4.8 M 2023-11-08T16:26:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-08T16:26:21.500 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-08T16:26:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-08T16:26:21.501 INFO:teuthology.orchestra.run.smithi081.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-08T16:26:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-08T16:26:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-08T16:26:21.502 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-08T16:26:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-08T16:26:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-08T16:26:21.503 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-08T16:26:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-08T16:26:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-08T16:26:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 45 k 2023-11-08T16:26:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common x86_64 2:18.0.0-7083.g85bdb240.el9 ceph 127 k 2023-11-08T16:26:21.504 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-08T16:26:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-08T16:26:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-08T16:26:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-08T16:26:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2023-11-08T16:26:21.505 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel x86_64 3.9.17-2.el9 CentOS-AppStream 205 k 2023-11-08T16:26:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth noarch 1:2.23.3-1.el9 epel 197 k 2023-11-08T16:26:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-08T16:26:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-08T16:26:21.506 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-08T16:26:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-08T16:26:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-08T16:26:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-08T16:26:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-08T16:26:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-08T16:26:21.507 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-08T16:26:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-08T16:26:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-08T16:26:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-08T16:26:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-08T16:26:21.508 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-08T16:26:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-08T16:26:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-08T16:26:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-08T16:26:21.509 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-08T16:26:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-08T16:26:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-08T16:26:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-08T16:26:21.510 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-08T16:26:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-08T16:26:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-08T16:26:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-08T16:26:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-08T16:26:21.511 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2023-11-08T16:26:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-08T16:26:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-08T16:26:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-08T16:26:21.512 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-08T16:26:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-08T16:26:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-08T16:26:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-08T16:26:21.513 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-08T16:26:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-08T16:26:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-08T16:26:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-08T16:26:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-08T16:26:21.514 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-08T16:26:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-08T16:26:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-08T16:26:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-08T16:26:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-08T16:26:21.515 INFO:teuthology.orchestra.run.smithi081.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-08T16:26:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-08T16:26:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel x86_64 1.2.11-40.el9 appstream 45 k 2023-11-08T16:26:21.516 INFO:teuthology.orchestra.run.smithi081.stdout:Installing weak dependencies: 2023-11-08T16:26:21.516 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-08T16:26:21.517 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:26:21.517 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2023-11-08T16:26:21.517 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================================ 2023-11-08T16:26:21.517 INFO:teuthology.orchestra.run.smithi081.stdout:Install 132 Packages 2023-11-08T16:26:21.517 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade 10 Packages 2023-11-08T16:26:21.518 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:26:21.518 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 200 M 2023-11-08T16:26:21.519 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2023-11-08T16:26:22.086 INFO:teuthology.orchestra.run.smithi093.stdout:(10/142): ceph-test-18.0.0-7083.g85bdb240.el9.x 15 MB/s | 41 MB 00:02 2023-11-08T16:26:22.144 INFO:teuthology.orchestra.run.smithi093.stdout:(11/142): cephfs-mirror-18.0.0-7083.g85bdb240.e 3.7 MB/s | 216 kB 00:00 2023-11-08T16:26:22.186 INFO:teuthology.orchestra.run.smithi093.stdout:(12/142): libcephfs-devel-18.0.0-7083.g85bdb240 759 kB/s | 31 kB 00:00 2023-11-08T16:26:22.303 INFO:teuthology.orchestra.run.smithi093.stdout:(13/142): libcephfs2-18.0.0-7083.g85bdb240.el9. 6.1 MB/s | 718 kB 00:00 2023-11-08T16:26:22.362 INFO:teuthology.orchestra.run.smithi093.stdout:(14/142): libcephsqlite-18.0.0-7083.g85bdb240.e 2.8 MB/s | 162 kB 00:00 2023-11-08T16:26:22.429 INFO:teuthology.orchestra.run.smithi093.stdout:(15/142): librados-devel-18.0.0-7083.g85bdb240. 1.9 MB/s | 126 kB 00:00 2023-11-08T16:26:22.487 INFO:teuthology.orchestra.run.smithi093.stdout:(16/142): libradosstriper1-18.0.0-7083.g85bdb24 8.1 MB/s | 476 kB 00:00 2023-11-08T16:26:22.896 INFO:teuthology.orchestra.run.smithi093.stdout:(17/142): librgw2-18.0.0-7083.g85bdb240.el9.x86 12 MB/s | 4.8 MB 00:00 2023-11-08T16:26:23.021 INFO:teuthology.orchestra.run.smithi093.stdout:(18/142): python3-ceph-argparse-18.0.0-7083.g85 364 kB/s | 45 kB 00:00 2023-11-08T16:26:23.074 INFO:teuthology.orchestra.run.smithi093.stdout:(19/142): python3-ceph-common-18.0.0-7083.g85bd 2.4 MB/s | 127 kB 00:00 2023-11-08T16:26:23.132 INFO:teuthology.orchestra.run.smithi093.stdout:(20/142): python3-cephfs-18.0.0-7083.g85bdb240. 2.7 MB/s | 162 kB 00:00 2023-11-08T16:26:23.199 INFO:teuthology.orchestra.run.smithi093.stdout:(21/142): python3-rados-18.0.0-7083.g85bdb240.e 4.8 MB/s | 321 kB 00:00 2023-11-08T16:26:23.267 INFO:teuthology.orchestra.run.smithi093.stdout:(22/142): python3-rbd-18.0.0-7083.g85bdb240.el9 4.4 MB/s | 297 kB 00:00 2023-11-08T16:26:23.317 INFO:teuthology.orchestra.run.smithi093.stdout:(23/142): python3-rgw-18.0.0-7083.g85bdb240.el9 2.0 MB/s | 99 kB 00:00 2023-11-08T16:26:23.367 INFO:teuthology.orchestra.run.smithi093.stdout:(24/142): rbd-fuse-18.0.0-7083.g85bdb240.el9.x8 1.7 MB/s | 84 kB 00:00 2023-11-08T16:26:23.634 INFO:teuthology.orchestra.run.smithi093.stdout:(25/142): rbd-mirror-18.0.0-7083.g85bdb240.el9. 11 MB/s | 3.0 MB 00:00 2023-11-08T16:26:23.693 INFO:teuthology.orchestra.run.smithi093.stdout:(26/142): rbd-nbd-18.0.0-7083.g85bdb240.el9.x86 2.9 MB/s | 169 kB 00:00 2023-11-08T16:26:23.732 INFO:teuthology.orchestra.run.smithi081.stdout:(1/142): ceph-18.0.0-7083.g85bdb240.el9.x86_64. 4.6 kB/s | 6.5 kB 00:01 2023-11-08T16:26:23.760 INFO:teuthology.orchestra.run.smithi093.stdout:(27/142): ceph-grafana-dashboards-18.0.0-7083.g 353 kB/s | 23 kB 00:00 2023-11-08T16:26:23.835 INFO:teuthology.orchestra.run.smithi093.stdout:(28/142): ceph-mgr-cephadm-18.0.0-7083.g85bdb24 1.7 MB/s | 132 kB 00:00 2023-11-08T16:26:24.110 INFO:teuthology.orchestra.run.smithi093.stdout:(29/142): ceph-mgr-dashboard-18.0.0-7083.g85bdb 6.4 MB/s | 1.8 MB 00:00 2023-11-08T16:26:24.665 INFO:teuthology.orchestra.run.smithi081.stdout:(2/142): ceph-fuse-18.0.0-7083.g85bdb240.el9.x8 921 kB/s | 858 kB 00:00 2023-11-08T16:26:24.752 INFO:teuthology.orchestra.run.smithi093.stdout:(30/142): ceph-mgr-diskprediction-local-18.0.0- 12 MB/s | 7.4 MB 00:00 2023-11-08T16:26:24.811 INFO:teuthology.orchestra.run.smithi093.stdout:(31/142): ceph-mgr-modules-core-18.0.0-7083.g85 4.1 MB/s | 243 kB 00:00 2023-11-08T16:26:24.861 INFO:teuthology.orchestra.run.smithi093.stdout:(32/142): ceph-mgr-rook-18.0.0-7083.g85bdb240.e 1.0 MB/s | 50 kB 00:00 2023-11-08T16:26:24.911 INFO:teuthology.orchestra.run.smithi093.stdout:(33/142): ceph-prometheus-alerts-18.0.0-7083.g8 305 kB/s | 15 kB 00:00 2023-11-08T16:26:24.961 INFO:teuthology.orchestra.run.smithi093.stdout:(34/142): ceph-volume-18.0.0-7083.g85bdb240.el9 5.2 MB/s | 262 kB 00:00 2023-11-08T16:26:25.020 INFO:teuthology.orchestra.run.smithi093.stdout:(35/142): cephadm-18.0.0-7083.g85bdb240.el9.noa 4.6 MB/s | 272 kB 00:00 2023-11-08T16:26:25.070 INFO:teuthology.orchestra.run.smithi093.stdout:(36/142): cephfs-top-18.0.0-7083.g85bdb240.el9. 368 kB/s | 18 kB 00:00 2023-11-08T16:26:25.090 INFO:teuthology.orchestra.run.smithi081.stdout:(3/142): ceph-immutable-object-cache-18.0.0-708 331 kB/s | 140 kB 00:00 2023-11-08T16:26:25.254 INFO:teuthology.orchestra.run.smithi093.stdout:(37/142): bison-3.7.4-5.el9.x86_64.rpm 4.9 MB/s | 920 kB 00:00 2023-11-08T16:26:25.279 INFO:teuthology.orchestra.run.smithi093.stdout:(38/142): boost-program-options-1.75.0-8.el9.x8 4.3 MB/s | 106 kB 00:00 2023-11-08T16:26:25.322 INFO:teuthology.orchestra.run.smithi093.stdout:(39/142): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 307 kB 00:00 2023-11-08T16:26:25.455 INFO:teuthology.orchestra.run.smithi093.stdout:(40/142): ceph-radosgw-18.0.0-7083.g85bdb240.el 1.2 MB/s | 8.7 MB 00:07 2023-11-08T16:26:25.480 INFO:teuthology.orchestra.run.smithi093.stdout:(41/142): flexiblas-3.0.4-8.el9.x86_64.rpm 200 kB/s | 32 kB 00:00 2023-11-08T16:26:25.581 INFO:teuthology.orchestra.run.smithi093.stdout:(42/142): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2023-11-08T16:26:25.606 INFO:teuthology.orchestra.run.smithi093.stdout:(43/142): flexiblas-openblas-openmp-3.0.4-8.el9 154 kB/s | 17 kB 00:00 2023-11-08T16:26:25.639 INFO:teuthology.orchestra.run.smithi093.stdout:(44/142): libaio-devel-0.3.111-13.el9.x86_64.rp 315 kB/s | 10 kB 00:00 2023-11-08T16:26:25.665 INFO:teuthology.orchestra.run.smithi093.stdout:(45/142): jq-1.6-16.el9.x86_64.rpm 3.2 MB/s | 186 kB 00:00 2023-11-08T16:26:25.690 INFO:teuthology.orchestra.run.smithi093.stdout:(46/142): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2023-11-08T16:26:25.757 INFO:teuthology.orchestra.run.smithi093.stdout:(47/142): librdkafka-1.6.1-102.el9.x86_64.rpm 7.2 MB/s | 661 kB 00:00 2023-11-08T16:26:25.799 INFO:teuthology.orchestra.run.smithi093.stdout:(48/142): libtool-2.4.6-45.el9.x86_64.rpm 5.5 MB/s | 578 kB 00:00 2023-11-08T16:26:25.824 INFO:teuthology.orchestra.run.smithi093.stdout:(49/142): libzstd-devel-1.5.1-2.el9.x86_64.rpm 757 kB/s | 46 kB 00:00 2023-11-08T16:26:25.858 INFO:teuthology.orchestra.run.smithi093.stdout:(50/142): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-11-08T16:26:25.874 INFO:teuthology.orchestra.run.smithi081.stdout:(4/142): ceph-base-18.0.0-7083.g85bdb240.el9.x8 1.5 MB/s | 5.2 MB 00:03 2023-11-08T16:26:25.891 INFO:teuthology.orchestra.run.smithi093.stdout:(51/142): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2023-11-08T16:26:25.917 INFO:teuthology.orchestra.run.smithi093.stdout:(52/142): openblas-0.3.21-2.el9.x86_64.rpm 611 kB/s | 34 kB 00:00 2023-11-08T16:26:25.949 INFO:teuthology.orchestra.run.smithi081.stdout:(5/142): ceph-mds-18.0.0-7083.g85bdb240.el9.x86 2.4 MB/s | 2.1 MB 00:00 2023-11-08T16:26:26.140 INFO:teuthology.orchestra.run.smithi093.stdout:(53/142): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2023-11-08T16:26:26.158 INFO:teuthology.orchestra.run.smithi081.stdout:(6/142): ceph-mgr-18.0.0-7083.g85bdb240.el9.x86 5.2 MB/s | 1.5 MB 00:00 2023-11-08T16:26:26.192 INFO:teuthology.orchestra.run.smithi093.stdout:(54/142): postgresql-13.11-1.el9.x86_64.rpm 5.8 MB/s | 1.5 MB 00:00 2023-11-08T16:26:26.243 INFO:teuthology.orchestra.run.smithi093.stdout:(55/142): postgresql-contrib-13.11-1.el9.x86_64 7.8 MB/s | 811 kB 00:00 2023-11-08T16:26:26.268 INFO:teuthology.orchestra.run.smithi093.stdout:(56/142): postgresql-private-libs-13.11-1.el9.x 1.9 MB/s | 134 kB 00:00 2023-11-08T16:26:26.418 INFO:teuthology.orchestra.run.smithi093.stdout:(57/142): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2023-11-08T16:26:26.570 INFO:teuthology.orchestra.run.smithi093.stdout:(58/142): python3-babel-2.9.1-2.el9.noarch.rpm 20 MB/s | 5.8 MB 00:00 2023-11-08T16:26:26.602 INFO:teuthology.orchestra.run.smithi093.stdout:(59/142): python3-devel-3.9.17-2.el9.x86_64.rpm 1.2 MB/s | 205 kB 00:00 2023-11-08T16:26:26.635 INFO:teuthology.orchestra.run.smithi093.stdout:(60/142): python3-jsonpatch-1.21-16.el9.noarch. 765 kB/s | 24 kB 00:00 2023-11-08T16:26:26.669 INFO:teuthology.orchestra.run.smithi093.stdout:(61/142): python3-jinja2-2.11.3-4.el9.noarch.rp 3.4 MB/s | 228 kB 00:00 2023-11-08T16:26:26.694 INFO:teuthology.orchestra.run.smithi093.stdout:(62/142): python3-jsonpointer-2.0-4.el9.noarch. 323 kB/s | 18 kB 00:00 2023-11-08T16:26:26.720 INFO:teuthology.orchestra.run.smithi093.stdout:(63/142): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-08T16:26:26.745 INFO:teuthology.orchestra.run.smithi093.stdout:(64/142): python3-markupsafe-1.1.1-12.el9.x86_6 655 kB/s | 32 kB 00:00 2023-11-08T16:26:26.870 INFO:teuthology.orchestra.run.smithi093.stdout:(65/142): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 4.8 MB 00:00 2023-11-08T16:26:26.895 INFO:teuthology.orchestra.run.smithi093.stdout:(66/142): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 217 kB 00:00 2023-11-08T16:26:26.929 INFO:teuthology.orchestra.run.smithi093.stdout:(67/142): python3-prettytable-0.7.2-27.el9.noar 1.3 MB/s | 41 kB 00:00 2023-11-08T16:26:26.954 INFO:teuthology.orchestra.run.smithi093.stdout:(68/142): python3-oauthlib-3.1.1-5.el9.noarch.r 3.0 MB/s | 174 kB 00:00 2023-11-08T16:26:26.980 INFO:teuthology.orchestra.run.smithi093.stdout:(69/142): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 132 kB 00:00 2023-11-08T16:26:27.005 INFO:teuthology.orchestra.run.smithi093.stdout:(70/142): python3-pyasn1-modules-0.4.8-6.el9.no 4.3 MB/s | 211 kB 00:00 2023-11-08T16:26:27.048 INFO:teuthology.orchestra.run.smithi093.stdout:(71/142): python3-requests-oauthlib-1.3.0-12.el 661 kB/s | 43 kB 00:00 2023-11-08T16:26:27.116 INFO:teuthology.orchestra.run.smithi081.stdout:(7/142): ceph-common-18.0.0-7083.g85bdb240.el9. 4.0 MB/s | 19 MB 00:04 2023-11-08T16:26:27.216 INFO:teuthology.orchestra.run.smithi081.stdout:(8/142): ceph-mon-18.0.0-7083.g85bdb240.el9.x86 3.5 MB/s | 4.4 MB 00:01 2023-11-08T16:26:27.314 INFO:teuthology.orchestra.run.smithi093.stdout:(72/142): python3-scipy-1.6.2-8.el9.x86_64.rpm 51 MB/s | 16 MB 00:00 2023-11-08T16:26:27.325 INFO:teuthology.orchestra.run.smithi081.stdout:(9/142): ceph-selinux-18.0.0-7083.g85bdb240.el9 224 kB/s | 24 kB 00:00 2023-11-08T16:26:27.339 INFO:teuthology.orchestra.run.smithi093.stdout:(73/142): python3-toml-0.10.2-6.el9.noarch.rpm 139 kB/s | 39 kB 00:00 2023-11-08T16:26:27.390 INFO:teuthology.orchestra.run.smithi093.stdout:(74/142): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2023-11-08T16:26:27.415 INFO:teuthology.orchestra.run.smithi093.stdout:(75/142): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 300 kB 00:00 2023-11-08T16:26:27.440 INFO:teuthology.orchestra.run.smithi093.stdout:(76/142): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2023-11-08T16:26:27.466 INFO:teuthology.orchestra.run.smithi093.stdout:(77/142): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2023-11-08T16:26:27.516 INFO:teuthology.orchestra.run.smithi093.stdout:(78/142): libgfortran-11.4.1-2.3.el9.x86_64.rpm 11 MB/s | 803 kB 00:00 2023-11-08T16:26:27.550 INFO:teuthology.orchestra.run.smithi093.stdout:(79/142): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2023-11-08T16:26:27.583 INFO:teuthology.orchestra.run.smithi093.stdout:(80/142): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.6 MB/s | 241 kB 00:00 2023-11-08T16:26:27.608 INFO:teuthology.orchestra.run.smithi093.stdout:(81/142): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2023-11-08T16:26:27.634 INFO:teuthology.orchestra.run.smithi093.stdout:(82/142): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2023-11-08T16:26:27.659 INFO:teuthology.orchestra.run.smithi093.stdout:(83/142): python3-requests-2.25.1-7.el9.noarch. 2.3 MB/s | 113 kB 00:00 2023-11-08T16:26:27.693 INFO:teuthology.orchestra.run.smithi093.stdout:(84/142): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-11-08T16:26:27.893 INFO:teuthology.orchestra.run.smithi093.stdout:(85/142): python3-cryptography-36.0.1-4.el9.x86 5.3 MB/s | 1.2 MB 00:00 2023-11-08T16:26:28.252 INFO:teuthology.orchestra.run.smithi093.stdout:(86/142): ceph-common-18.0.0-7083.g85bdb240.el9 1.6 MB/s | 19 MB 00:11 2023-11-08T16:26:28.293 INFO:teuthology.orchestra.run.smithi093.stdout:(87/142): elfutils-libelf-devel-0.189-3.el9.x86 56 kB/s | 22 kB 00:00 2023-11-08T16:26:28.460 INFO:teuthology.orchestra.run.smithi093.stdout:(88/142): libuuid-devel-2.37.4-15.el9.x86_64.rp 154 kB/s | 26 kB 00:00 2023-11-08T16:26:28.625 INFO:teuthology.orchestra.run.smithi081.stdout:(10/142): ceph-osd-18.0.0-7083.g85bdb240.el9.x8 7.2 MB/s | 18 MB 00:02 2023-11-08T16:26:28.709 INFO:teuthology.orchestra.run.smithi081.stdout:(11/142): cephfs-mirror-18.0.0-7083.g85bdb240.e 2.6 MB/s | 216 kB 00:00 2023-11-08T16:26:28.719 INFO:teuthology.orchestra.run.smithi093.stdout:(89/142): librabbitmq-0.11.0-7.el9.x86_64.rpm 97 kB/s | 45 kB 00:00 2023-11-08T16:26:28.867 INFO:teuthology.orchestra.run.smithi081.stdout:(12/142): ceph-radosgw-18.0.0-7083.g85bdb240.el 5.0 MB/s | 8.7 MB 00:01 2023-11-08T16:26:28.884 INFO:teuthology.orchestra.run.smithi081.stdout:(13/142): libcephfs-devel-18.0.0-7083.g85bdb240 178 kB/s | 31 kB 00:00 2023-11-08T16:26:28.968 INFO:teuthology.orchestra.run.smithi081.stdout:(14/142): libcephsqlite-18.0.0-7083.g85bdb240.e 1.9 MB/s | 162 kB 00:00 2023-11-08T16:26:29.043 INFO:teuthology.orchestra.run.smithi081.stdout:(15/142): libcephfs2-18.0.0-7083.g85bdb240.el9. 4.0 MB/s | 718 kB 00:00 2023-11-08T16:26:29.078 INFO:teuthology.orchestra.run.smithi093.stdout:(90/142): python3-pytz-2021.1-5.el9.noarch.rpm 142 kB/s | 51 kB 00:00 2023-11-08T16:26:29.252 INFO:teuthology.orchestra.run.smithi081.stdout:(16/142): librados-devel-18.0.0-7083.g85bdb240. 447 kB/s | 126 kB 00:00 2023-11-08T16:26:29.294 INFO:teuthology.orchestra.run.smithi081.stdout:(17/142): libradosstriper1-18.0.0-7083.g85bdb24 1.9 MB/s | 476 kB 00:00 2023-11-08T16:26:29.403 INFO:teuthology.orchestra.run.smithi081.stdout:(18/142): python3-ceph-argparse-18.0.0-7083.g85 419 kB/s | 45 kB 00:00 2023-11-08T16:26:29.469 INFO:teuthology.orchestra.run.smithi093.stdout:(91/142): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 220 kB/s | 86 kB 00:00 2023-11-08T16:26:29.495 INFO:teuthology.orchestra.run.smithi081.stdout:(19/142): python3-ceph-common-18.0.0-7083.g85bd 1.4 MB/s | 127 kB 00:00 2023-11-08T16:26:29.578 INFO:teuthology.orchestra.run.smithi081.stdout:(20/142): python3-cephfs-18.0.0-7083.g85bdb240. 1.9 MB/s | 162 kB 00:00 2023-11-08T16:26:29.745 INFO:teuthology.orchestra.run.smithi081.stdout:(21/142): python3-rados-18.0.0-7083.g85bdb240.e 1.9 MB/s | 321 kB 00:00 2023-11-08T16:26:29.794 INFO:teuthology.orchestra.run.smithi093.stdout:(92/142): zlib-devel-1.2.11-40.el9.x86_64.rpm 139 kB/s | 45 kB 00:00 2023-11-08T16:26:29.912 INFO:teuthology.orchestra.run.smithi081.stdout:(22/142): librgw2-18.0.0-7083.g85bdb240.el9.x86 7.3 MB/s | 4.8 MB 00:00 2023-11-08T16:26:29.920 INFO:teuthology.orchestra.run.smithi093.stdout:(93/142): openssl-devel-3.0.7-24.el9.x86_64.rpm 2.8 MB/s | 4.1 MB 00:01 2023-11-08T16:26:29.954 INFO:teuthology.orchestra.run.smithi081.stdout:(23/142): python3-rbd-18.0.0-7083.g85bdb240.el9 1.4 MB/s | 297 kB 00:00 2023-11-08T16:26:29.988 INFO:teuthology.orchestra.run.smithi081.stdout:(24/142): python3-rgw-18.0.0-7083.g85bdb240.el9 1.3 MB/s | 99 kB 00:00 2023-11-08T16:26:30.021 INFO:teuthology.orchestra.run.smithi081.stdout:(25/142): rbd-fuse-18.0.0-7083.g85bdb240.el9.x8 1.2 MB/s | 84 kB 00:00 2023-11-08T16:26:30.028 INFO:teuthology.orchestra.run.smithi093.stdout:(94/142): automake-1.16.2-8.el9.noarch.rpm 295 kB/s | 689 kB 00:02 2023-11-08T16:26:30.122 INFO:teuthology.orchestra.run.smithi081.stdout:(26/142): rbd-nbd-18.0.0-7083.g85bdb240.el9.x86 1.7 MB/s | 169 kB 00:00 2023-11-08T16:26:30.197 INFO:teuthology.orchestra.run.smithi081.stdout:(27/142): ceph-grafana-dashboards-18.0.0-7083.g 315 kB/s | 23 kB 00:00 2023-11-08T16:26:30.329 INFO:teuthology.orchestra.run.smithi093.stdout:(95/142): libarrow-doc-9.0.0-9.el9.noarch.rpm 81 kB/s | 24 kB 00:00 2023-11-08T16:26:30.330 INFO:teuthology.orchestra.run.smithi081.stdout:(28/142): ceph-mgr-cephadm-18.0.0-7083.g85bdb24 1.0 MB/s | 132 kB 00:00 2023-11-08T16:26:30.422 INFO:teuthology.orchestra.run.smithi081.stdout:(29/142): rbd-mirror-18.0.0-7083.g85bdb240.el9. 6.9 MB/s | 3.0 MB 00:00 2023-11-08T16:26:30.437 INFO:teuthology.orchestra.run.smithi093.stdout:(96/142): gperftools-libs-2.9.1-2.el9.x86_64.rp 481 kB/s | 309 kB 00:00 2023-11-08T16:26:30.462 INFO:teuthology.orchestra.run.smithi093.stdout:(97/142): liboath-2.6.7-2.el9.x86_64.rpm 366 kB/s | 49 kB 00:00 2023-11-08T16:26:30.521 INFO:teuthology.orchestra.run.smithi093.stdout:(98/142): libunwind-1.6.2-1.el9.x86_64.rpm 814 kB/s | 67 kB 00:00 2023-11-08T16:26:30.671 INFO:teuthology.orchestra.run.smithi093.stdout:(99/142): python3-asyncssh-2.13.2-3.el9.noarch. 3.6 MB/s | 548 kB 00:00 2023-11-08T16:26:30.681 INFO:teuthology.orchestra.run.smithi081.stdout:(30/142): ceph-mgr-dashboard-18.0.0-7083.g85bdb 5.0 MB/s | 1.8 MB 00:00 2023-11-08T16:26:30.747 INFO:teuthology.orchestra.run.smithi093.stdout:(100/142): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.9 MB/s | 837 kB 00:00 2023-11-08T16:26:30.772 INFO:teuthology.orchestra.run.smithi093.stdout:(101/142): python3-bcrypt-3.2.2-1.el9.x86_64.rp 436 kB/s | 43 kB 00:00 2023-11-08T16:26:30.815 INFO:teuthology.orchestra.run.smithi081.stdout:(31/142): ceph-mgr-modules-core-18.0.0-7083.g85 1.8 MB/s | 243 kB 00:00 2023-11-08T16:26:30.872 INFO:teuthology.orchestra.run.smithi093.stdout:(102/142): libarrow-9.0.0-9.el9.x86_64.rpm 5.0 MB/s | 4.4 MB 00:00 2023-11-08T16:26:30.898 INFO:teuthology.orchestra.run.smithi093.stdout:(103/142): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2023-11-08T16:26:30.898 INFO:teuthology.orchestra.run.smithi081.stdout:(32/142): ceph-mgr-rook-18.0.0-7083.g85bdb240.e 608 kB/s | 50 kB 00:00 2023-11-08T16:26:30.923 INFO:teuthology.orchestra.run.smithi093.stdout:(104/142): python3-cachetools-4.2.4-1.el9.noarc 184 kB/s | 32 kB 00:00 2023-11-08T16:26:30.957 INFO:teuthology.orchestra.run.smithi093.stdout:(105/142): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2023-11-08T16:26:30.982 INFO:teuthology.orchestra.run.smithi081.stdout:(33/142): ceph-prometheus-alerts-18.0.0-7083.g8 183 kB/s | 15 kB 00:00 2023-11-08T16:26:30.990 INFO:teuthology.orchestra.run.smithi093.stdout:(106/142): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2023-11-08T16:26:31.015 INFO:teuthology.orchestra.run.smithi093.stdout:(107/142): python3-google-auth-2.23.3-1.el9.noa 2.1 MB/s | 197 kB 00:00 2023-11-08T16:26:31.041 INFO:teuthology.orchestra.run.smithi093.stdout:(108/142): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2023-11-08T16:26:31.066 INFO:teuthology.orchestra.run.smithi093.stdout:(109/142): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2023-11-08T16:26:31.074 INFO:teuthology.orchestra.run.smithi081.stdout:(34/142): ceph-volume-18.0.0-7083.g85bdb240.el9 2.8 MB/s | 262 kB 00:00 2023-11-08T16:26:31.091 INFO:teuthology.orchestra.run.smithi093.stdout:(110/142): python3-jaraco-collections-3.0.0-8.e 309 kB/s | 23 kB 00:00 2023-11-08T16:26:31.116 INFO:teuthology.orchestra.run.smithi093.stdout:(111/142): python3-jaraco-functools-3.5.0-2.el9 260 kB/s | 19 kB 00:00 2023-11-08T16:26:31.142 INFO:teuthology.orchestra.run.smithi093.stdout:(112/142): python3-jaraco-text-3.2.0-6.el9.noar 263 kB/s | 20 kB 00:00 2023-11-08T16:26:31.191 INFO:teuthology.orchestra.run.smithi081.stdout:(35/142): cephadm-18.0.0-7083.g85bdb240.el9.noa 2.3 MB/s | 272 kB 00:00 2023-11-08T16:26:31.233 INFO:teuthology.orchestra.run.smithi081.stdout:(36/142): cephfs-top-18.0.0-7083.g85bdb240.el9. 445 kB/s | 18 kB 00:00 2023-11-08T16:26:31.250 INFO:teuthology.orchestra.run.smithi093.stdout:(113/142): python3-jwt+crypto-2.4.0-1.el9.noarc 57 kB/s | 9.0 kB 00:00 2023-11-08T16:26:31.267 INFO:teuthology.orchestra.run.smithi093.stdout:(114/142): python3-jwt-2.4.0-1.el9.noarch.rpm 273 kB/s | 41 kB 00:00 2023-11-08T16:26:31.318 INFO:teuthology.orchestra.run.smithi093.stdout:(115/142): python3-kubernetes-26.1.0-2.el9.noar 5.9 MB/s | 1.0 MB 00:00 2023-11-08T16:26:31.334 INFO:teuthology.orchestra.run.smithi093.stdout:(116/142): python3-logutils-0.3.5-21.el9.noarch 558 kB/s | 46 kB 00:00 2023-11-08T16:26:31.360 INFO:teuthology.orchestra.run.smithi093.stdout:(117/142): python3-more-itertools-8.12.0-2.el9. 863 kB/s | 79 kB 00:00 2023-11-08T16:26:31.393 INFO:teuthology.orchestra.run.smithi093.stdout:(118/142): python3-natsort-7.1.1-5.el9.noarch.r 774 kB/s | 58 kB 00:00 2023-11-08T16:26:31.427 INFO:teuthology.orchestra.run.smithi093.stdout:(119/142): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2023-11-08T16:26:31.433 INFO:teuthology.orchestra.run.smithi081.stdout:(37/142): ceph-mgr-diskprediction-local-18.0.0- 7.3 MB/s | 7.4 MB 00:01 2023-11-08T16:26:31.452 INFO:teuthology.orchestra.run.smithi093.stdout:(120/142): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2023-11-08T16:26:31.477 INFO:teuthology.orchestra.run.smithi093.stdout:(121/142): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2023-11-08T16:26:31.503 INFO:teuthology.orchestra.run.smithi093.stdout:(122/142): python3-repoze-lru-0.7-16.el9.noarch 410 kB/s | 31 kB 00:00 2023-11-08T16:26:31.554 INFO:teuthology.orchestra.run.smithi093.stdout:(123/142): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2023-11-08T16:26:31.587 INFO:teuthology.orchestra.run.smithi093.stdout:(124/142): python3-tempora-5.0.0-2.el9.noarch.r 431 kB/s | 36 kB 00:00 2023-11-08T16:26:31.592 INFO:teuthology.orchestra.run.smithi081.stdout:(38/142): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 920 kB 00:00 2023-11-08T16:26:31.612 INFO:teuthology.orchestra.run.smithi093.stdout:(125/142): python3-rsa-4.9-2.el9.noarch.rpm 443 kB/s | 59 kB 00:00 2023-11-08T16:26:31.617 INFO:teuthology.orchestra.run.smithi081.stdout:(39/142): boost-program-options-1.75.0-8.el9.x8 582 kB/s | 106 kB 00:00 2023-11-08T16:26:31.637 INFO:teuthology.orchestra.run.smithi093.stdout:(126/142): python3-typing-extensions-4.4.0-2.el 621 kB/s | 51 kB 00:00 2023-11-08T16:26:31.651 INFO:teuthology.orchestra.run.smithi081.stdout:(40/142): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 307 kB 00:00 2023-11-08T16:26:31.668 INFO:teuthology.orchestra.run.smithi081.stdout:(41/142): flexiblas-3.0.4-8.el9.x86_64.rpm 671 kB/s | 32 kB 00:00 2023-11-08T16:26:31.670 INFO:teuthology.orchestra.run.smithi093.stdout:(127/142): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2023-11-08T16:26:31.696 INFO:teuthology.orchestra.run.smithi093.stdout:(128/142): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2023-11-08T16:26:31.738 INFO:teuthology.orchestra.run.smithi093.stdout:(129/142): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2023-11-08T16:26:31.760 INFO:teuthology.orchestra.run.smithi081.stdout:(42/142): flexiblas-openblas-openmp-3.0.4-8.el9 192 kB/s | 17 kB 00:00 2023-11-08T16:26:31.763 INFO:teuthology.orchestra.run.smithi093.stdout:(130/142): python3-zc-lockfile-2.0-10.el9.noarc 218 kB/s | 20 kB 00:00 2023-11-08T16:26:31.797 INFO:teuthology.orchestra.run.smithi093.stdout:(131/142): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2023-11-08T16:26:31.835 INFO:teuthology.orchestra.run.smithi081.stdout:(43/142): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2023-11-08T16:26:31.861 INFO:teuthology.orchestra.run.smithi081.stdout:(44/142): jq-1.6-16.el9.x86_64.rpm 1.9 MB/s | 186 kB 00:00 2023-11-08T16:26:31.886 INFO:teuthology.orchestra.run.smithi081.stdout:(45/142): libaio-devel-0.3.111-13.el9.x86_64.rp 208 kB/s | 10 kB 00:00 2023-11-08T16:26:31.897 INFO:teuthology.orchestra.run.smithi093.stdout:(132/142): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2023-11-08T16:26:31.912 INFO:teuthology.orchestra.run.smithi081.stdout:(46/142): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2023-11-08T16:26:31.931 INFO:teuthology.orchestra.run.smithi093.stdout:(133/142): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-11-08T16:26:31.954 INFO:teuthology.orchestra.run.smithi081.stdout:(47/142): librdkafka-1.6.1-102.el9.x86_64.rpm 9.9 MB/s | 661 kB 00:00 2023-11-08T16:26:31.988 INFO:teuthology.orchestra.run.smithi081.stdout:(48/142): libtool-2.4.6-45.el9.x86_64.rpm 8.2 MB/s | 578 kB 00:00 2023-11-08T16:26:32.013 INFO:teuthology.orchestra.run.smithi081.stdout:(49/142): libzstd-devel-1.5.1-2.el9.x86_64.rpm 876 kB/s | 46 kB 00:00 2023-11-08T16:26:32.023 INFO:teuthology.orchestra.run.smithi093.stdout:(134/142): NetworkManager-1.45.6-1.el9.x86_64.r 25 MB/s | 2.2 MB 00:00 2023-11-08T16:26:32.055 INFO:teuthology.orchestra.run.smithi081.stdout:(50/142): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2023-11-08T16:26:32.089 INFO:teuthology.orchestra.run.smithi081.stdout:(51/142): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.9 MB/s | 217 kB 00:00 2023-11-08T16:26:32.098 INFO:teuthology.orchestra.run.smithi093.stdout:(135/142): NetworkManager-libnm-1.45.6-1.el9.x8 24 MB/s | 1.8 MB 00:00 2023-11-08T16:26:32.114 INFO:teuthology.orchestra.run.smithi081.stdout:(52/142): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 34 kB 00:00 2023-11-08T16:26:32.132 INFO:teuthology.orchestra.run.smithi093.stdout:(136/142): NetworkManager-team-1.45.6-1.el9.x86 1.3 MB/s | 42 kB 00:00 2023-11-08T16:26:32.173 INFO:teuthology.orchestra.run.smithi093.stdout:(137/142): NetworkManager-tui-1.45.6-1.el9.x86_ 5.9 MB/s | 245 kB 00:00 2023-11-08T16:26:32.207 INFO:teuthology.orchestra.run.smithi093.stdout:(138/142): acl-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 69 kB 00:00 2023-11-08T16:26:32.241 INFO:teuthology.orchestra.run.smithi093.stdout:(139/142): libacl-2.3.1-4.el9.x86_64.rpm 678 kB/s | 22 kB 00:00 2023-11-08T16:26:32.281 INFO:teuthology.orchestra.run.smithi081.stdout:(53/142): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2023-11-08T16:26:32.324 INFO:teuthology.orchestra.run.smithi093.stdout:(140/142): librbd1-18.0.0-7083.g85bdb240.el9.x8 5.7 MB/s | 3.0 MB 00:00 2023-11-08T16:26:32.339 INFO:teuthology.orchestra.run.smithi081.stdout:(54/142): postgresql-13.11-1.el9.x86_64.rpm 7.1 MB/s | 1.5 MB 00:00 2023-11-08T16:26:32.358 INFO:teuthology.orchestra.run.smithi093.stdout:(141/142): xfsdump-3.1.12-4.el9.x86_64.rpm 2.7 MB/s | 321 kB 00:00 2023-11-08T16:26:32.382 INFO:teuthology.orchestra.run.smithi081.stdout:(55/142): postgresql-contrib-13.11-1.el9.x86_64 9.0 MB/s | 811 kB 00:00 2023-11-08T16:26:32.407 INFO:teuthology.orchestra.run.smithi081.stdout:(56/142): postgresql-private-libs-13.11-1.el9.x 2.2 MB/s | 134 kB 00:00 2023-11-08T16:26:32.557 INFO:teuthology.orchestra.run.smithi081.stdout:(57/142): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2023-11-08T16:26:32.699 INFO:teuthology.orchestra.run.smithi081.stdout:(58/142): python3-babel-2.9.1-2.el9.noarch.rpm 21 MB/s | 5.8 MB 00:00 2023-11-08T16:26:32.724 INFO:teuthology.orchestra.run.smithi081.stdout:(59/142): python3-devel-3.9.17-2.el9.x86_64.rpm 1.3 MB/s | 205 kB 00:00 2023-11-08T16:26:32.758 INFO:teuthology.orchestra.run.smithi081.stdout:(60/142): python3-jsonpatch-1.21-16.el9.noarch. 761 kB/s | 24 kB 00:00 2023-11-08T16:26:32.792 INFO:teuthology.orchestra.run.smithi081.stdout:(61/142): python3-jinja2-2.11.3-4.el9.noarch.rp 3.4 MB/s | 228 kB 00:00 2023-11-08T16:26:32.817 INFO:teuthology.orchestra.run.smithi081.stdout:(62/142): python3-jsonpointer-2.0-4.el9.noarch. 323 kB/s | 18 kB 00:00 2023-11-08T16:26:32.842 INFO:teuthology.orchestra.run.smithi081.stdout:(63/142): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-08T16:26:32.867 INFO:teuthology.orchestra.run.smithi081.stdout:(64/142): python3-markupsafe-1.1.1-12.el9.x86_6 654 kB/s | 32 kB 00:00 2023-11-08T16:26:33.001 INFO:teuthology.orchestra.run.smithi081.stdout:(65/142): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2023-11-08T16:26:33.026 INFO:teuthology.orchestra.run.smithi081.stdout:(66/142): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2023-11-08T16:26:33.060 INFO:teuthology.orchestra.run.smithi081.stdout:(67/142): python3-prettytable-0.7.2-27.el9.noar 1.3 MB/s | 41 kB 00:00 2023-11-08T16:26:33.094 INFO:teuthology.orchestra.run.smithi081.stdout:(68/142): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2023-11-08T16:26:33.119 INFO:teuthology.orchestra.run.smithi081.stdout:(69/142): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2023-11-08T16:26:33.153 INFO:teuthology.orchestra.run.smithi081.stdout:(70/142): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2023-11-08T16:26:33.178 INFO:teuthology.orchestra.run.smithi081.stdout:(71/142): python3-requests-oauthlib-1.3.0-12.el 762 kB/s | 43 kB 00:00 2023-11-08T16:26:33.462 INFO:teuthology.orchestra.run.smithi081.stdout:(72/142): python3-scipy-1.6.2-8.el9.x86_64.rpm 51 MB/s | 16 MB 00:00 2023-11-08T16:26:33.486 INFO:teuthology.orchestra.run.smithi081.stdout:(73/142): python3-toml-0.10.2-6.el9.noarch.rpm 133 kB/s | 39 kB 00:00 2023-11-08T16:26:33.548 INFO:teuthology.orchestra.run.smithi081.stdout:(74/142): uuid-1.6.2-55.el9.x86_64.rpm 937 kB/s | 56 kB 00:00 2023-11-08T16:26:33.587 INFO:teuthology.orchestra.run.smithi081.stdout:(75/142): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 300 kB 00:00 2023-11-08T16:26:33.600 INFO:teuthology.orchestra.run.smithi093.stdout:(142/142): librados2-18.0.0-7083.g85bdb240.el9. 1.8 MB/s | 3.3 MB 00:01 2023-11-08T16:26:33.612 INFO:teuthology.orchestra.run.smithi081.stdout:(76/142): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.0 MB/s | 63 kB 00:00 2023-11-08T16:26:33.613 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-11-08T16:26:33.614 INFO:teuthology.orchestra.run.smithi093.stdout:Total 11 MB/s | 200 MB 00:17 2023-11-08T16:26:33.629 INFO:teuthology.orchestra.run.smithi081.stdout:(77/142): fuse-2.9.9-15.el9.x86_64.rpm 1.9 MB/s | 79 kB 00:00 2023-11-08T16:26:33.671 INFO:teuthology.orchestra.run.smithi081.stdout:(78/142): libgfortran-11.4.1-2.3.el9.x86_64.rpm 14 MB/s | 803 kB 00:00 2023-11-08T16:26:33.705 INFO:teuthology.orchestra.run.smithi081.stdout:(79/142): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.8 MB/s | 195 kB 00:00 2023-11-08T16:26:33.730 INFO:teuthology.orchestra.run.smithi081.stdout:(80/142): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 241 kB 00:00 2023-11-08T16:26:33.756 INFO:teuthology.orchestra.run.smithi081.stdout:(81/142): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 103 kB 00:00 2023-11-08T16:26:33.781 INFO:teuthology.orchestra.run.smithi081.stdout:(82/142): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2023-11-08T16:26:33.806 INFO:teuthology.orchestra.run.smithi081.stdout:(83/142): python3-requests-2.25.1-7.el9.noarch. 2.3 MB/s | 113 kB 00:00 2023-11-08T16:26:33.840 INFO:teuthology.orchestra.run.smithi081.stdout:(84/142): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-11-08T16:26:34.015 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction check 2023-11-08T16:26:34.040 INFO:teuthology.orchestra.run.smithi081.stdout:(85/142): python3-cryptography-36.0.1-4.el9.x86 5.3 MB/s | 1.2 MB 00:00 2023-11-08T16:26:34.091 INFO:teuthology.orchestra.run.smithi081.stdout:(86/142): automake-1.16.2-8.el9.noarch.rpm 2.7 MB/s | 689 kB 00:00 2023-11-08T16:26:34.116 INFO:teuthology.orchestra.run.smithi081.stdout:(87/142): elfutils-libelf-devel-0.189-3.el9.x86 301 kB/s | 22 kB 00:00 2023-11-08T16:26:34.215 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction check succeeded. 2023-11-08T16:26:34.215 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction test 2023-11-08T16:26:34.283 INFO:teuthology.orchestra.run.smithi081.stdout:(88/142): libuuid-devel-2.37.4-15.el9.x86_64.rp 154 kB/s | 26 kB 00:00 2023-11-08T16:26:34.308 INFO:teuthology.orchestra.run.smithi081.stdout:(89/142): librabbitmq-0.11.0-7.el9.x86_64.rpm 209 kB/s | 45 kB 00:00 2023-11-08T16:26:34.375 INFO:teuthology.orchestra.run.smithi081.stdout:(90/142): python3-pytz-2021.1-5.el9.noarch.rpm 774 kB/s | 51 kB 00:00 2023-11-08T16:26:34.458 INFO:teuthology.orchestra.run.smithi081.stdout:(91/142): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.0 MB/s | 86 kB 00:00 2023-11-08T16:26:34.584 INFO:teuthology.orchestra.run.smithi081.stdout:(92/142): zlib-devel-1.2.11-40.el9.x86_64.rpm 363 kB/s | 45 kB 00:00 2023-11-08T16:26:35.225 INFO:teuthology.orchestra.run.smithi081.stdout:(93/142): gperftools-libs-2.9.1-2.el9.x86_64.rp 482 kB/s | 309 kB 00:00 2023-11-08T16:26:35.693 INFO:teuthology.orchestra.run.smithi081.stdout:(94/142): ceph-test-18.0.0-7083.g85bdb240.el9.x 4.9 MB/s | 41 MB 00:08 2023-11-08T16:26:35.993 INFO:teuthology.orchestra.run.smithi081.stdout:(95/142): openssl-devel-3.0.7-24.el9.x86_64.rpm 2.4 MB/s | 4.1 MB 00:01 2023-11-08T16:26:36.021 INFO:teuthology.orchestra.run.smithi081.stdout:(96/142): libarrow-doc-9.0.0-9.el9.noarch.rpm 74 kB/s | 24 kB 00:00 2023-11-08T16:26:36.324 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction test succeeded. 2023-11-08T16:26:36.328 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction 2023-11-08T16:26:36.488 INFO:teuthology.orchestra.run.smithi081.stdout:(97/142): libarrow-9.0.0-9.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2023-11-08T16:26:36.513 INFO:teuthology.orchestra.run.smithi081.stdout:(98/142): liboath-2.6.7-2.el9.x86_64.rpm 93 kB/s | 49 kB 00:00 2023-11-08T16:26:36.541 INFO:teuthology.orchestra.run.smithi081.stdout:(99/142): libunwind-1.6.2-1.el9.x86_64.rpm 130 kB/s | 67 kB 00:00 2023-11-08T16:26:36.872 INFO:teuthology.orchestra.run.smithi081.stdout:(100/142): parquet-libs-9.0.0-9.el9.x86_64.rpm 2.1 MB/s | 837 kB 00:00 2023-11-08T16:26:36.897 INFO:teuthology.orchestra.run.smithi081.stdout:(101/142): python3-bcrypt-3.2.2-1.el9.x86_64.rp 121 kB/s | 43 kB 00:00 2023-11-08T16:26:37.056 INFO:teuthology.orchestra.run.smithi081.stdout:(102/142): python3-cachetools-4.2.4-1.el9.noarc 176 kB/s | 32 kB 00:00 2023-11-08T16:26:37.082 INFO:teuthology.orchestra.run.smithi081.stdout:(103/142): python3-certifi-2021.10.8-2.el9.noar 82 kB/s | 15 kB 00:00 2023-11-08T16:26:37.115 INFO:teuthology.orchestra.run.smithi081.stdout:(104/142): python3-asyncssh-2.13.2-3.el9.noarch 911 kB/s | 548 kB 00:00 2023-11-08T16:26:37.257 INFO:teuthology.orchestra.run.smithi081.stdout:(105/142): python3-cheroot-8.6.0-4.el9.noarch.r 859 kB/s | 172 kB 00:00 2023-11-08T16:26:37.316 INFO:teuthology.orchestra.run.smithi081.stdout:(106/142): python3-google-auth-2.23.3-1.el9.noa 986 kB/s | 197 kB 00:00 2023-11-08T16:26:37.399 INFO:teuthology.orchestra.run.smithi081.stdout:(107/142): python3-cherrypy-18.6.1-2.el9.noarch 1.1 MB/s | 358 kB 00:00 2023-11-08T16:26:37.416 INFO:teuthology.orchestra.run.smithi081.stdout:(108/142): python3-jaraco-8.2.1-3.el9.noarch.rp 67 kB/s | 11 kB 00:00 2023-11-08T16:26:37.475 INFO:teuthology.orchestra.run.smithi081.stdout:(109/142): python3-jaraco-classes-3.2.1-5.el9.n 112 kB/s | 18 kB 00:00 2023-11-08T16:26:37.525 INFO:teuthology.orchestra.run.smithi081.stdout:(110/142): python3-jaraco-collections-3.0.0-8.e 185 kB/s | 23 kB 00:00 2023-11-08T16:26:37.567 INFO:teuthology.orchestra.run.smithi081.stdout:(111/142): python3-jaraco-functools-3.5.0-2.el9 129 kB/s | 19 kB 00:00 2023-11-08T16:26:37.626 INFO:teuthology.orchestra.run.smithi081.stdout:(112/142): python3-jaraco-text-3.2.0-6.el9.noar 131 kB/s | 20 kB 00:00 2023-11-08T16:26:37.709 INFO:teuthology.orchestra.run.smithi081.stdout:(113/142): python3-jwt+crypto-2.4.0-1.el9.noarc 49 kB/s | 9.0 kB 00:00 2023-11-08T16:26:37.735 INFO:teuthology.orchestra.run.smithi081.stdout:(114/142): python3-jwt-2.4.0-1.el9.noarch.rpm 245 kB/s | 41 kB 00:00 2023-11-08T16:26:37.893 INFO:teuthology.orchestra.run.smithi081.stdout:(115/142): python3-more-itertools-8.12.0-2.el9. 500 kB/s | 79 kB 00:00 2023-11-08T16:26:37.918 INFO:teuthology.orchestra.run.smithi081.stdout:(116/142): python3-logutils-0.3.5-21.el9.noarch 222 kB/s | 46 kB 00:00 2023-11-08T16:26:37.985 INFO:teuthology.orchestra.run.smithi081.stdout:(117/142): python3-kubernetes-26.1.0-2.el9.noar 2.8 MB/s | 1.0 MB 00:00 2023-11-08T16:26:38.094 INFO:teuthology.orchestra.run.smithi081.stdout:(118/142): python3-natsort-7.1.1-5.el9.noarch.r 289 kB/s | 58 kB 00:00 2023-11-08T16:26:38.144 INFO:teuthology.orchestra.run.smithi081.stdout:(119/142): python3-portend-3.1.0-2.el9.noarch.r 104 kB/s | 16 kB 00:00 2023-11-08T16:26:38.178 INFO:teuthology.orchestra.run.smithi081.stdout:(120/142): python3-pecan-1.4.2-3.el9.noarch.rpm 1.0 MB/s | 272 kB 00:00 2023-11-08T16:26:38.228 INFO:teuthology.orchestra.run.smithi081.stdout:(121/142): python3-pyOpenSSL-21.0.0-1.el9.noarc 672 kB/s | 90 kB 00:00 2023-11-08T16:26:38.320 INFO:teuthology.orchestra.run.smithi081.stdout:(122/142): python3-repoze-lru-0.7-16.el9.noarch 176 kB/s | 31 kB 00:00 2023-11-08T16:26:38.362 INFO:teuthology.orchestra.run.smithi081.stdout:(123/142): python3-rsa-4.9-2.el9.noarch.rpm 444 kB/s | 59 kB 00:00 2023-11-08T16:26:38.396 INFO:teuthology.orchestra.run.smithi081.stdout:(124/142): python3-routes-2.5.1-5.el9.noarch.rp 869 kB/s | 188 kB 00:00 2023-11-08T16:26:38.446 INFO:teuthology.orchestra.run.smithi081.stdout:(125/142): python3-tempora-5.0.0-2.el9.noarch.r 286 kB/s | 36 kB 00:00 2023-11-08T16:26:38.496 INFO:teuthology.orchestra.run.smithi081.stdout:(126/142): python3-typing-extensions-4.4.0-2.el 383 kB/s | 51 kB 00:00 2023-11-08T16:26:38.588 INFO:teuthology.orchestra.run.smithi081.stdout:(127/142): python3-webob-1.8.7-6.el9.noarch.rpm 1.2 MB/s | 230 kB 00:00 2023-11-08T16:26:38.613 INFO:teuthology.orchestra.run.smithi081.stdout:(128/142): python3-websocket-client-1.2.3-2.el9 537 kB/s | 90 kB 00:00 2023-11-08T16:26:38.755 INFO:teuthology.orchestra.run.smithi081.stdout:(129/142): python3-werkzeug-2.0.3-3.el9.1.noarc 1.6 MB/s | 427 kB 00:00 2023-11-08T16:26:38.781 INFO:teuthology.orchestra.run.smithi081.stdout:(130/142): python3-zc-lockfile-2.0-10.el9.noarc 104 kB/s | 20 kB 00:00 2023-11-08T16:26:38.900 INFO:teuthology.orchestra.run.smithi081.stdout:(131/142): re2-20211101-3.el9.x86_64.rpm 671 kB/s | 192 kB 00:00 2023-11-08T16:26:39.242 INFO:teuthology.orchestra.run.smithi081.stdout:(132/142): thrift-0.15.0-2.el9.x86_64.rpm 3.3 MB/s | 1.6 MB 00:00 2023-11-08T16:26:39.275 INFO:teuthology.orchestra.run.smithi081.stdout:(133/142): libacl-devel-2.3.1-4.el9.x86_64.rpm 2.0 MB/s | 68 kB 00:00 2023-11-08T16:26:39.367 INFO:teuthology.orchestra.run.smithi081.stdout:(134/142): NetworkManager-1.45.6-1.el9.x86_64.r 25 MB/s | 2.2 MB 00:00 2023-11-08T16:26:39.451 INFO:teuthology.orchestra.run.smithi081.stdout:(135/142): NetworkManager-libnm-1.45.6-1.el9.x8 21 MB/s | 1.8 MB 00:00 2023-11-08T16:26:39.484 INFO:teuthology.orchestra.run.smithi081.stdout:(136/142): NetworkManager-team-1.45.6-1.el9.x86 1.3 MB/s | 42 kB 00:00 2023-11-08T16:26:39.568 INFO:teuthology.orchestra.run.smithi081.stdout:(137/142): librados2-18.0.0-7083.g85bdb240.el9. 4.2 MB/s | 3.3 MB 00:00 2023-11-08T16:26:39.602 INFO:teuthology.orchestra.run.smithi081.stdout:(138/142): NetworkManager-tui-1.45.6-1.el9.x86_ 2.1 MB/s | 245 kB 00:00 2023-11-08T16:26:39.627 INFO:teuthology.orchestra.run.smithi081.stdout:(139/142): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-11-08T16:26:39.652 INFO:teuthology.orchestra.run.smithi081.stdout:(140/142): libacl-2.3.1-4.el9.x86_64.rpm 451 kB/s | 22 kB 00:00 2023-11-08T16:26:39.686 INFO:teuthology.orchestra.run.smithi081.stdout:(141/142): xfsdump-3.1.12-4.el9.x86_64.rpm 5.4 MB/s | 321 kB 00:00 2023-11-08T16:26:39.786 INFO:teuthology.orchestra.run.smithi081.stdout:(142/142): librbd1-18.0.0-7083.g85bdb240.el9.x8 3.4 MB/s | 3.0 MB 00:00 2023-11-08T16:26:39.794 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2023-11-08T16:26:39.794 INFO:teuthology.orchestra.run.smithi081.stdout:Total 11 MB/s | 200 MB 00:18 2023-11-08T16:26:40.183 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2023-11-08T16:26:40.383 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2023-11-08T16:26:40.384 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2023-11-08T16:26:42.325 INFO:teuthology.orchestra.run.smithi093.stdout: Preparing : 1/1 2023-11-08T16:26:42.452 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2023-11-08T16:26:42.488 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2023-11-08T16:26:42.576 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2023-11-08T16:26:42.628 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2023-11-08T16:26:42.630 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2023-11-08T16:26:42.814 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/152 2023-11-08T16:26:43.242 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/152 2023-11-08T16:26:43.295 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librados2-2:18.0.0-7083.g85bdb240.el9.x86_64 6/152 2023-11-08T16:26:43.452 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-2:18.0.0-7083.g85bdb240.el9.x86_64 6/152 2023-11-08T16:26:43.488 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs2-2:18.0.0-7083.g85bdb240.el9.x86_64 7/152 2023-11-08T16:26:43.611 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephfs2-2:18.0.0-7083.g85bdb240.el9.x86_64 7/152 2023-11-08T16:26:43.672 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rados-2:18.0.0-7083.g85bdb240.el9.x86_64 8/152 2023-11-08T16:26:43.707 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/152 2023-11-08T16:26:43.957 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/152 2023-11-08T16:26:44.020 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/152 2023-11-08T16:26:44.051 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephsqlite-2:18.0.0-7083.g85bdb240.el9.x86_64 12/152 2023-11-08T16:26:44.206 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7083.g85bdb240.el9.x86_64 12/152 2023-11-08T16:26:44.235 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libradosstriper1-2:18.0.0-7083.g85bdb240.el9.x86 13/152 2023-11-08T16:26:44.319 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7083.g85bdb240.el9.x86 13/152 2023-11-08T16:26:44.651 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/152 2023-11-08T16:26:44.798 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-08T16:26:44.919 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-08T16:26:45.095 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T16:26:45.152 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T16:26:45.557 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T16:26:45.599 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/152 2023-11-08T16:26:45.646 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2023-11-08T16:26:45.743 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2023-11-08T16:26:45.817 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/152 2023-11-08T16:26:45.847 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/152 2023-11-08T16:26:45.898 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/152 2023-11-08T16:26:45.994 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/152 2023-11-08T16:26:46.039 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/152 2023-11-08T16:26:46.073 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/152 2023-11-08T16:26:46.178 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2023-11-08T16:26:46.219 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-common-2:18.0.0-7083.g85bdb240.el9. 27/152 2023-11-08T16:26:46.269 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-argparse-2:18.0.0-7083.g85bdb240.el 28/152 2023-11-08T16:26:46.372 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cephfs-2:18.0.0-7083.g85bdb240.el9.x86_6 29/152 2023-11-08T16:26:46.547 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/152 2023-11-08T16:26:46.612 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/152 2023-11-08T16:26:47.316 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/152 2023-11-08T16:26:47.434 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/152 2023-11-08T16:26:47.489 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/152 2023-11-08T16:26:47.529 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librados-devel-2:18.0.0-7083.g85bdb240.el9.x86_6 35/152 2023-11-08T16:26:47.567 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/152 2023-11-08T16:26:47.715 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/152 2023-11-08T16:26:47.752 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/152 2023-11-08T16:26:47.819 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/152 2023-11-08T16:26:47.901 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/152 2023-11-08T16:26:47.953 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/152 2023-11-08T16:26:47.991 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/152 2023-11-08T16:26:48.068 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/152 2023-11-08T16:26:48.194 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/152 2023-11-08T16:26:48.269 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/152 2023-11-08T16:26:48.403 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2023-11-08T16:26:48.469 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/152 2023-11-08T16:26:48.476 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2023-11-08T16:26:48.518 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/152 2023-11-08T16:26:48.565 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/152 2023-11-08T16:26:48.644 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2023-11-08T16:26:48.712 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/152 2023-11-08T16:26:48.752 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/152 2023-11-08T16:26:48.753 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2023-11-08T16:26:48.799 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/152 2023-11-08T16:26:48.833 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/152 2023-11-08T16:26:48.868 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/152 2023-11-08T16:26:48.871 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/152 2023-11-08T16:26:48.913 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/152 2023-11-08T16:26:49.056 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/152 2023-11-08T16:26:49.116 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : automake-1.16.2-8.el9.noarch 56/152 2023-11-08T16:26:49.194 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/152 2023-11-08T16:26:49.275 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/152 2023-11-08T16:26:49.277 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/152 2023-11-08T16:26:49.315 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librados2-2:18.0.0-7083.g85bdb240.el9.x86_64 6/152 2023-11-08T16:26:49.480 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:18.0.0-7083.g85bdb240.el9.x86_64 6/152 2023-11-08T16:26:49.508 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs2-2:18.0.0-7083.g85bdb240.el9.x86_64 7/152 2023-11-08T16:26:49.550 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/152 2023-11-08T16:26:49.610 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/152 2023-11-08T16:26:49.633 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephfs2-2:18.0.0-7083.g85bdb240.el9.x86_64 7/152 2023-11-08T16:26:49.697 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rados-2:18.0.0-7083.g85bdb240.el9.x86_64 8/152 2023-11-08T16:26:49.742 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 9/152 2023-11-08T16:26:49.787 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/152 2023-11-08T16:26:49.837 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/152 2023-11-08T16:26:49.886 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/152 2023-11-08T16:26:49.926 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/152 2023-11-08T16:26:49.970 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/152 2023-11-08T16:26:49.976 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 10/152 2023-11-08T16:26:50.013 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/152 2023-11-08T16:26:50.030 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-13.11-1.el9.x86_64 11/152 2023-11-08T16:26:50.054 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephsqlite-2:18.0.0-7083.g85bdb240.el9.x86_64 12/152 2023-11-08T16:26:50.061 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/152 2023-11-08T16:26:50.155 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/152 2023-11-08T16:26:50.195 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7083.g85bdb240.el9.x86_64 12/152 2023-11-08T16:26:50.216 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/152 2023-11-08T16:26:50.239 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libradosstriper1-2:18.0.0-7083.g85bdb240.el9.x86 13/152 2023-11-08T16:26:50.263 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/152 2023-11-08T16:26:50.322 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7083.g85bdb240.el9.x86 13/152 2023-11-08T16:26:50.390 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/152 2023-11-08T16:26:50.426 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/152 2023-11-08T16:26:50.461 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/152 2023-11-08T16:26:50.523 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/152 2023-11-08T16:26:50.564 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/152 2023-11-08T16:26:50.674 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 14/152 2023-11-08T16:26:50.710 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/152 2023-11-08T16:26:50.758 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/152 2023-11-08T16:26:50.828 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-08T16:26:50.897 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/152 2023-11-08T16:26:50.916 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 15/152 2023-11-08T16:26:50.972 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2023-11-08T16:26:51.125 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T16:26:51.172 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T16:26:51.572 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 16/152 2023-11-08T16:26:51.611 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 17/152 2023-11-08T16:26:51.666 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.7-2.el9.x86_64 18/152 2023-11-08T16:26:51.764 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 19/152 2023-11-08T16:26:51.825 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 20/152 2023-11-08T16:26:51.857 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 21/152 2023-11-08T16:26:51.910 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 22/152 2023-11-08T16:26:51.955 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2023-11-08T16:26:52.007 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 23/152 2023-11-08T16:26:52.013 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2023-11-08T16:26:52.046 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2023-11-08T16:26:52.075 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/152 2023-11-08T16:26:52.109 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/152 2023-11-08T16:26:52.173 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2023-11-08T16:26:52.213 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/152 2023-11-08T16:26:52.222 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/152 2023-11-08T16:26:52.255 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-common-2:18.0.0-7083.g85bdb240.el9. 27/152 2023-11-08T16:26:52.308 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ceph-argparse-2:18.0.0-7083.g85bdb240.el 28/152 2023-11-08T16:26:52.422 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cephfs-2:18.0.0-7083.g85bdb240.el9.x86_6 29/152 2023-11-08T16:26:52.444 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/152 2023-11-08T16:26:52.477 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/152 2023-11-08T16:26:52.569 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/152 2023-11-08T16:26:52.602 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 30/152 2023-11-08T16:26:52.666 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 31/152 2023-11-08T16:26:53.362 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rsa-4.9-2.el9.noarch 32/152 2023-11-08T16:26:53.379 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/152 2023-11-08T16:26:53.455 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 33/152 2023-11-08T16:26:53.500 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 34/152 2023-11-08T16:26:53.521 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/152 2023-11-08T16:26:53.538 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librados-devel-2:18.0.0-7083.g85bdb240.el9.x86_6 35/152 2023-11-08T16:26:53.569 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 36/152 2023-11-08T16:26:53.712 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : re2-1:20211101-3.el9.x86_64 37/152 2023-11-08T16:26:53.747 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 38/152 2023-11-08T16:26:53.811 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 39/152 2023-11-08T16:26:53.896 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 40/152 2023-11-08T16:26:53.931 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 41/152 2023-11-08T16:26:53.968 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 42/152 2023-11-08T16:26:54.031 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 43/152 2023-11-08T16:26:54.110 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 44/152 2023-11-08T16:26:54.164 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 45/152 2023-11-08T16:26:54.357 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 46/152 2023-11-08T16:26:54.405 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 47/152 2023-11-08T16:26:54.460 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 48/152 2023-11-08T16:26:54.587 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 49/152 2023-11-08T16:26:54.622 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-google-auth-1:2.23.3-1.el9.noarch 50/152 2023-11-08T16:26:54.668 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 51/152 2023-11-08T16:26:54.710 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 52/152 2023-11-08T16:26:54.740 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 53/152 2023-11-08T16:26:54.784 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 54/152 2023-11-08T16:26:54.914 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 55/152 2023-11-08T16:26:54.969 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : automake-1.16.2-8.el9.noarch 56/152 2023-11-08T16:26:55.039 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-ply-3.11-14.el9.noarch 57/152 2023-11-08T16:26:55.069 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/152 2023-11-08T16:26:55.119 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 58/152 2023-11-08T16:26:55.171 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/152 2023-11-08T16:26:55.211 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/152 2023-11-08T16:26:55.241 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : jq-1.6-16.el9.x86_64 93/152 2023-11-08T16:26:55.282 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2023-11-08T16:26:55.389 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 59/152 2023-11-08T16:26:55.448 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 60/152 2023-11-08T16:26:55.619 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 61/152 2023-11-08T16:26:55.643 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/152 2023-11-08T16:26:55.666 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 62/152 2023-11-08T16:26:55.691 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 63/152 2023-11-08T16:26:55.692 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librbd1-2:18.0.0-7083.g85bdb240.el9.x86_64 96/152 2023-11-08T16:26:55.729 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 64/152 2023-11-08T16:26:55.771 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 65/152 2023-11-08T16:26:55.813 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-2:18.0.0-7083.g85bdb240.el9.x86_64 96/152 2023-11-08T16:26:55.824 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : fuse-2.9.9-15.el9.x86_64 66/152 2023-11-08T16:26:55.842 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rbd-2:18.0.0-7083.g85bdb240.el9.x86_64 97/152 2023-11-08T16:26:55.880 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 67/152 2023-11-08T16:26:55.897 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/152 2023-11-08T16:26:55.945 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : uuid-1.6.2-55.el9.x86_64 68/152 2023-11-08T16:26:55.998 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 69/152 2023-11-08T16:26:56.042 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 70/152 2023-11-08T16:26:56.166 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 71/152 2023-11-08T16:26:56.206 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 72/152 2023-11-08T16:26:56.249 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/152 2023-11-08T16:26:56.314 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/152 2023-11-08T16:26:56.353 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/152 2023-11-08T16:26:56.378 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/152 2023-11-08T16:26:56.405 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librgw2-2:18.0.0-7083.g85bdb240.el9.x86_64 100/152 2023-11-08T16:26:56.480 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librgw2-2:18.0.0-7083.g85bdb240.el9.x86_64 100/152 2023-11-08T16:26:56.496 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/152 2023-11-08T16:26:56.548 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rgw-2:18.0.0-7083.g85bdb240.el9.x86_64 101/152 2023-11-08T16:26:56.550 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 77/152 2023-11-08T16:26:56.691 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 78/152 2023-11-08T16:26:56.756 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2023-11-08T16:26:57.732 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2023-11-08T16:26:57.780 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2023-11-08T16:26:57.814 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2023-11-08T16:26:57.936 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2023-11-08T16:26:57.982 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 84/152 2023-11-08T16:26:58.212 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-0.3.21-2.el9.x86_64 85/152 2023-11-08T16:26:58.246 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 86/152 2023-11-08T16:26:58.339 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 87/152 2023-11-08T16:26:59.049 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 102/152 2023-11-08T16:26:59.086 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 102/152 2023-11-08T16:26:59.147 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 88/152 2023-11-08T16:26:59.309 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 89/152 2023-11-08T16:26:59.706 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 102/152 2023-11-08T16:26:59.756 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-base-2:18.0.0-7083.g85bdb240.el9.x86_64 103/152 2023-11-08T16:26:59.958 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-base-2:18.0.0-7083.g85bdb240.el9.x86_64 103/152 2023-11-08T16:26:59.958 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-08T16:26:59.958 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-08T16:26:59.959 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:00.008 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-selinux-2:18.0.0-7083.g85bdb240.el9.x86_64 104/152 2023-11-08T16:27:00.891 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 90/152 2023-11-08T16:27:00.984 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 91/152 2023-11-08T16:27:01.031 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 92/152 2023-11-08T16:27:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : jq-1.6-16.el9.x86_64 93/152 2023-11-08T16:27:01.118 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2023-11-08T16:27:01.483 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 95/152 2023-11-08T16:27:01.515 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : librbd1-2:18.0.0-7083.g85bdb240.el9.x86_64 96/152 2023-11-08T16:27:01.626 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:18.0.0-7083.g85bdb240.el9.x86_64 96/152 2023-11-08T16:27:01.654 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rbd-2:18.0.0-7083.g85bdb240.el9.x86_64 97/152 2023-11-08T16:27:01.718 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 98/152 2023-11-08T16:27:02.194 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 99/152 2023-11-08T16:27:02.226 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librgw2-2:18.0.0-7083.g85bdb240.el9.x86_64 100/152 2023-11-08T16:27:02.343 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librgw2-2:18.0.0-7083.g85bdb240.el9.x86_64 100/152 2023-11-08T16:27:02.370 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-rgw-2:18.0.0-7083.g85bdb240.el9.x86_64 101/152 2023-11-08T16:27:04.802 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 102/152 2023-11-08T16:27:04.841 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 102/152 2023-11-08T16:27:05.461 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 102/152 2023-11-08T16:27:05.503 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-base-2:18.0.0-7083.g85bdb240.el9.x86_64 103/152 2023-11-08T16:27:05.673 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-base-2:18.0.0-7083.g85bdb240.el9.x86_64 103/152 2023-11-08T16:27:05.673 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-08T16:27:05.674 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-08T16:27:05.674 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:05.721 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-selinux-2:18.0.0-7083.g85bdb240.el9.x86_64 104/152 2023-11-08T16:27:12.475 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7083.g85bdb240.el9.x86_64 104/152 2023-11-08T16:27:12.476 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /sys 2023-11-08T16:27:12.476 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /proc 2023-11-08T16:27:12.476 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /mnt 2023-11-08T16:27:12.476 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /var/tmp 2023-11-08T16:27:12.477 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /home 2023-11-08T16:27:12.477 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /root 2023-11-08T16:27:12.477 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /tmp 2023-11-08T16:27:12.477 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:12.683 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-volume-2:18.0.0-7083.g85bdb240.el9.noarch 105/152 2023-11-08T16:27:12.732 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-volume-2:18.0.0-7083.g85bdb240.el9.noarch 105/152 2023-11-08T16:27:12.732 INFO:teuthology.orchestra.run.smithi093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:12.733 INFO:teuthology.orchestra.run.smithi093.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-08T16:27:12.733 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:14.419 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-osd-2:18.0.0-7083.g85bdb240.el9.x86_64 106/152 2023-11-08T16:27:14.505 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-osd-2:18.0.0-7083.g85bdb240.el9.x86_64 106/152 2023-11-08T16:27:14.505 INFO:teuthology.orchestra.run.smithi093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:14.505 INFO:teuthology.orchestra.run.smithi093.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-08T16:27:14.506 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-08T16:27:14.506 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-08T16:27:14.506 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:14.727 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mds-2:18.0.0-7083.g85bdb240.el9.x86_64 107/152 2023-11-08T16:27:14.800 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mds-2:18.0.0-7083.g85bdb240.el9.x86_64 107/152 2023-11-08T16:27:14.801 INFO:teuthology.orchestra.run.smithi093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:14.801 INFO:teuthology.orchestra.run.smithi093.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-08T16:27:14.801 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-08T16:27:14.801 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-08T16:27:14.802 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:15.286 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mon-2:18.0.0-7083.g85bdb240.el9.x86_64 108/152 2023-11-08T16:27:15.363 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mon-2:18.0.0-7083.g85bdb240.el9.x86_64 108/152 2023-11-08T16:27:15.363 INFO:teuthology.orchestra.run.smithi093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:15.364 INFO:teuthology.orchestra.run.smithi093.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-08T16:27:15.364 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-08T16:27:15.364 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-08T16:27:15.364 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:15.935 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephadm-2:18.0.0-7083.g85bdb240.el9.noarch 109/152 2023-11-08T16:27:15.997 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephadm-2:18.0.0-7083.g85bdb240.el9.noarch 109/152 2023-11-08T16:27:16.047 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7083.g85bdb240.e 110/152 2023-11-08T16:27:16.159 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7083.g85bdb240. 111/152 2023-11-08T16:27:16.208 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7083.g85bdb240.el9.noa 112/152 2023-11-08T16:27:16.531 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7083.g85bdb240.el9.noa 112/152 2023-11-08T16:27:16.567 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7083.g85bdb240.el9.n 113/152 2023-11-08T16:27:17.693 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7083.g85bdb240.el9.n 113/152 2023-11-08T16:27:17.717 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7083.g85b 114/152 2023-11-08T16:27:17.858 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7083.g85b 114/152 2023-11-08T16:27:18.026 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7083.g85bdb240.el 115/152 2023-11-08T16:27:18.061 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-2:18.0.0-7083.g85bdb240.el9.x86_64 116/152 2023-11-08T16:27:18.131 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7083.g85bdb240.el9.x86_64 116/152 2023-11-08T16:27:18.131 INFO:teuthology.orchestra.run.smithi093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:18.132 INFO:teuthology.orchestra.run.smithi093.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-08T16:27:18.132 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-08T16:27:18.132 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-08T16:27:18.133 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:18.203 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-rook-2:18.0.0-7083.g85bdb240.el9.noarch 117/152 2023-11-08T16:27:18.242 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7083.g85bdb240.el9.noarch 117/152 2023-11-08T16:27:18.431 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7083.g85bdb240.el9.x86_64 104/152 2023-11-08T16:27:18.431 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /sys 2023-11-08T16:27:18.432 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /proc 2023-11-08T16:27:18.432 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /mnt 2023-11-08T16:27:18.432 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /var/tmp 2023-11-08T16:27:18.432 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /home 2023-11-08T16:27:18.432 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /root 2023-11-08T16:27:18.433 INFO:teuthology.orchestra.run.smithi081.stdout:skipping the directory /tmp 2023-11-08T16:27:18.433 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:18.643 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-volume-2:18.0.0-7083.g85bdb240.el9.noarch 105/152 2023-11-08T16:27:18.693 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-volume-2:18.0.0-7083.g85bdb240.el9.noarch 105/152 2023-11-08T16:27:18.694 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:18.694 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-08T16:27:18.694 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:19.131 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-2:18.0.0-7083.g85bdb240.el9.x86_64 118/152 2023-11-08T16:27:19.162 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-radosgw-2:18.0.0-7083.g85bdb240.el9.x86_64 119/152 2023-11-08T16:27:19.232 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7083.g85bdb240.el9.x86_64 119/152 2023-11-08T16:27:19.232 INFO:teuthology.orchestra.run.smithi093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:19.232 INFO:teuthology.orchestra.run.smithi093.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-08T16:27:19.232 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-08T16:27:19.233 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-08T16:27:19.233 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:19.305 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7083.g85bdb 120/152 2023-11-08T16:27:19.379 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7083.g85bdb 120/152 2023-11-08T16:27:19.379 INFO:teuthology.orchestra.run.smithi093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:19.379 INFO:teuthology.orchestra.run.smithi093.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-08T16:27:19.379 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:19.447 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 121/152 2023-11-08T16:27:19.524 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 121/152 2023-11-08T16:27:19.524 INFO:teuthology.orchestra.run.smithi093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:19.524 INFO:teuthology.orchestra.run.smithi093.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-08T16:27:19.524 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-08T16:27:19.525 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-08T16:27:19.525 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:19.874 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 122/152 2023-11-08T16:27:19.949 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 122/152 2023-11-08T16:27:19.949 INFO:teuthology.orchestra.run.smithi093.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:19.949 INFO:teuthology.orchestra.run.smithi093.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-08T16:27:19.949 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-08T16:27:19.950 INFO:teuthology.orchestra.run.smithi093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-08T16:27:19.950 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:20.395 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-osd-2:18.0.0-7083.g85bdb240.el9.x86_64 106/152 2023-11-08T16:27:20.493 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-osd-2:18.0.0-7083.g85bdb240.el9.x86_64 106/152 2023-11-08T16:27:20.493 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:20.493 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-08T16:27:20.494 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-08T16:27:20.494 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-08T16:27:20.494 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:20.729 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mds-2:18.0.0-7083.g85bdb240.el9.x86_64 107/152 2023-11-08T16:27:20.804 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mds-2:18.0.0-7083.g85bdb240.el9.x86_64 107/152 2023-11-08T16:27:20.804 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:20.804 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-08T16:27:20.804 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-08T16:27:20.805 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-08T16:27:20.805 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:21.313 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mon-2:18.0.0-7083.g85bdb240.el9.x86_64 108/152 2023-11-08T16:27:21.384 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mon-2:18.0.0-7083.g85bdb240.el9.x86_64 108/152 2023-11-08T16:27:21.384 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:21.384 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-08T16:27:21.384 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-08T16:27:21.385 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-08T16:27:21.385 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:21.988 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephadm-2:18.0.0-7083.g85bdb240.el9.noarch 109/152 2023-11-08T16:27:22.024 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephadm-2:18.0.0-7083.g85bdb240.el9.noarch 109/152 2023-11-08T16:27:22.070 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7083.g85bdb240.e 110/152 2023-11-08T16:27:22.164 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7083.g85bdb240. 111/152 2023-11-08T16:27:22.210 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7083.g85bdb240.el9.noa 112/152 2023-11-08T16:27:22.528 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7083.g85bdb240.el9.noa 112/152 2023-11-08T16:27:22.569 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7083.g85bdb240.el9.n 113/152 2023-11-08T16:27:23.691 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7083.g85bdb240.el9.n 113/152 2023-11-08T16:27:23.720 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7083.g85b 114/152 2023-11-08T16:27:23.860 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7083.g85b 114/152 2023-11-08T16:27:24.038 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7083.g85bdb240.el 115/152 2023-11-08T16:27:24.072 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-2:18.0.0-7083.g85bdb240.el9.x86_64 116/152 2023-11-08T16:27:24.113 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-test-2:18.0.0-7083.g85bdb240.el9.x86_64 123/152 2023-11-08T16:27:24.146 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7083.g85bdb240.el9.x86_64 116/152 2023-11-08T16:27:24.146 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:24.147 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-08T16:27:24.147 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-08T16:27:24.147 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-08T16:27:24.147 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:24.169 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 124/152 2023-11-08T16:27:24.197 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-mgr-rook-2:18.0.0-7083.g85bdb240.el9.noarch 117/152 2023-11-08T16:27:24.210 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-nbd-2:18.0.0-7083.g85bdb240.el9.x86_64 125/152 2023-11-08T16:27:24.237 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7083.g85bdb240.el9.noarch 117/152 2023-11-08T16:27:24.476 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/152 2023-11-08T16:27:24.592 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/152 2023-11-08T16:27:24.655 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 128/152 2023-11-08T16:27:24.701 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/152 2023-11-08T16:27:24.731 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/152 2023-11-08T16:27:24.762 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs-devel-2:18.0.0-7083.g85bdb240.el9.x86_ 131/152 2023-11-08T16:27:24.806 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 132/152 2023-11-08T16:27:24.921 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 133/152 2023-11-08T16:27:25.002 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 134/152 2023-11-08T16:27:25.040 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 135/152 2023-11-08T16:27:25.099 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-2:18.0.0-7083.g85bdb240.el9.x86_64 118/152 2023-11-08T16:27:25.132 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-radosgw-2:18.0.0-7083.g85bdb240.el9.x86_64 119/152 2023-11-08T16:27:25.212 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7083.g85bdb240.el9.x86_64 119/152 2023-11-08T16:27:25.213 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:25.213 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-08T16:27:25.213 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-08T16:27:25.214 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-08T16:27:25.214 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:25.274 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7083.g85bdb 120/152 2023-11-08T16:27:25.346 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7083.g85bdb 120/152 2023-11-08T16:27:25.347 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:25.347 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-08T16:27:25.347 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:25.417 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 121/152 2023-11-08T16:27:25.498 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 121/152 2023-11-08T16:27:25.498 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:25.499 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-08T16:27:25.499 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-08T16:27:25.499 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-08T16:27:25.499 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:25.868 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 122/152 2023-11-08T16:27:25.939 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 122/152 2023-11-08T16:27:25.940 INFO:teuthology.orchestra.run.smithi081.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-08T16:27:25.940 INFO:teuthology.orchestra.run.smithi081.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-08T16:27:25.940 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-08T16:27:25.940 INFO:teuthology.orchestra.run.smithi081.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-08T16:27:25.941 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:25.978 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T16:27:26.029 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T16:27:26.087 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T16:27:26.150 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-top-2:18.0.0-7083.g85bdb240.el9.noarch 137/152 2023-11-08T16:27:27.528 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 138/152 2023-11-08T16:27:27.642 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 139/152 2023-11-08T16:27:27.691 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 140/152 2023-11-08T16:27:27.825 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : flex-2.6.4-9.el9.x86_64 141/152 2023-11-08T16:27:27.861 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : bison-3.7.4-5.el9.x86_64 142/152 2023-11-08T16:27:27.899 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 143/152 2023-11-08T16:27:27.899 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-08T16:27:28.051 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-08T16:27:28.082 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 145/152 2023-11-08T16:27:28.115 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 146/152 2023-11-08T16:27:28.148 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/152 2023-11-08T16:27:28.187 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T16:27:28.188 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T16:27:28.498 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T16:27:28.498 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-08T16:27:28.670 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-08T16:27:28.701 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-08T16:27:28.701 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-08T16:27:28.839 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-08T16:27:28.963 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T16:27:29.277 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T16:27:29.278 INFO:teuthology.orchestra.run.smithi093.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-08T16:27:29.278 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:30.057 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-test-2:18.0.0-7083.g85bdb240.el9.x86_64 123/152 2023-11-08T16:27:30.113 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 124/152 2023-11-08T16:27:30.146 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : rbd-nbd-2:18.0.0-7083.g85bdb240.el9.x86_64 125/152 2023-11-08T16:27:30.410 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : elfutils-libelf-devel-0.189-3.el9.x86_64 126/152 2023-11-08T16:27:30.538 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 127/152 2023-11-08T16:27:30.606 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : ceph-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 128/152 2023-11-08T16:27:30.655 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/152 2023-11-08T16:27:30.686 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 130/152 2023-11-08T16:27:30.716 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libcephfs-devel-2:18.0.0-7083.g85bdb240.el9.x86_ 131/152 2023-11-08T16:27:30.760 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 132/152 2023-11-08T16:27:30.857 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 133/152 2023-11-08T16:27:30.936 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 134/152 2023-11-08T16:27:30.978 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 135/152 2023-11-08T16:27:31.945 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T16:27:31.992 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T16:27:32.049 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 136/152 2023-11-08T16:27:32.113 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : cephfs-top-2:18.0.0-7083.g85bdb240.el9.noarch 137/152 2023-11-08T16:27:33.464 INFO:teuthology.orchestra.run.smithi081.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 138/152 2023-11-08T16:27:33.568 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : openssl-devel-1:3.0.7-24.el9.x86_64 139/152 2023-11-08T16:27:33.616 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 140/152 2023-11-08T16:27:33.751 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : flex-2.6.4-9.el9.x86_64 141/152 2023-11-08T16:27:33.795 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : bison-3.7.4-5.el9.x86_64 142/152 2023-11-08T16:27:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 143/152 2023-11-08T16:27:33.832 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-08T16:27:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-2:18.0.0-7083.g85bdb240.el9.x86_64 1/152 2023-11-08T16:27:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-base-2:18.0.0-7083.g85bdb240.el9.x86_64 2/152 2023-11-08T16:27:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 3/152 2023-11-08T16:27:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 4/152 2023-11-08T16:27:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7083.g85bdb 5/152 2023-11-08T16:27:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mds-2:18.0.0-7083.g85bdb240.el9.x86_64 6/152 2023-11-08T16:27:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-2:18.0.0-7083.g85bdb240.el9.x86_64 7/152 2023-11-08T16:27:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mon-2:18.0.0-7083.g85bdb240.el9.x86_64 8/152 2023-11-08T16:27:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-osd-2:18.0.0-7083.g85bdb240.el9.x86_64 9/152 2023-11-08T16:27:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-radosgw-2:18.0.0-7083.g85bdb240.el9.x86_64 10/152 2023-11-08T16:27:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-selinux-2:18.0.0-7083.g85bdb240.el9.x86_64 11/152 2023-11-08T16:27:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-test-2:18.0.0-7083.g85bdb240.el9.x86_64 12/152 2023-11-08T16:27:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 13/152 2023-11-08T16:27:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs-devel-2:18.0.0-7083.g85bdb240.el9.x86_ 14/152 2023-11-08T16:27:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs2-2:18.0.0-7083.g85bdb240.el9.x86_64 15/152 2023-11-08T16:27:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephsqlite-2:18.0.0-7083.g85bdb240.el9.x86_64 16/152 2023-11-08T16:27:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados-devel-2:18.0.0-7083.g85bdb240.el9.x86_6 17/152 2023-11-08T16:27:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libradosstriper1-2:18.0.0-7083.g85bdb240.el9.x86 18/152 2023-11-08T16:27:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librgw2-2:18.0.0-7083.g85bdb240.el9.x86_64 19/152 2023-11-08T16:27:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7083.g85bdb240.el 20/152 2023-11-08T16:27:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-common-2:18.0.0-7083.g85bdb240.el9. 21/152 2023-11-08T16:27:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cephfs-2:18.0.0-7083.g85bdb240.el9.x86_6 22/152 2023-11-08T16:27:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rados-2:18.0.0-7083.g85bdb240.el9.x86_64 23/152 2023-11-08T16:27:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rbd-2:18.0.0-7083.g85bdb240.el9.x86_64 24/152 2023-11-08T16:27:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rgw-2:18.0.0-7083.g85bdb240.el9.x86_64 25/152 2023-11-08T16:27:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 26/152 2023-11-08T16:27:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 27/152 2023-11-08T16:27:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-nbd-2:18.0.0-7083.g85bdb240.el9.x86_64 28/152 2023-11-08T16:27:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7083.g85bdb240. 29/152 2023-11-08T16:27:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7083.g85bdb240.el9.noa 30/152 2023-11-08T16:27:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7083.g85bdb240.el9.n 31/152 2023-11-08T16:27:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7083.g85b 32/152 2023-11-08T16:27:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7083.g85bdb240.el 33/152 2023-11-08T16:27:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7083.g85bdb240.el9.noarch 34/152 2023-11-08T16:27:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7083.g85bdb240.e 35/152 2023-11-08T16:27:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-volume-2:18.0.0-7083.g85bdb240.el9.noarch 36/152 2023-11-08T16:27:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephadm-2:18.0.0-7083.g85bdb240.el9.noarch 37/152 2023-11-08T16:27:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-top-2:18.0.0-7083.g85bdb240.el9.noarch 38/152 2023-11-08T16:27:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/152 2023-11-08T16:27:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/152 2023-11-08T16:27:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/152 2023-11-08T16:27:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/152 2023-11-08T16:27:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/152 2023-11-08T16:27:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/152 2023-11-08T16:27:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : jq-1.6-16.el9.x86_64 45/152 2023-11-08T16:27:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/152 2023-11-08T16:27:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/152 2023-11-08T16:27:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 48/152 2023-11-08T16:27:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 49/152 2023-11-08T16:27:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 50/152 2023-11-08T16:27:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 51/152 2023-11-08T16:27:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 52/152 2023-11-08T16:27:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 53/152 2023-11-08T16:27:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 54/152 2023-11-08T16:27:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-13.11-1.el9.x86_64 55/152 2023-11-08T16:27:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 56/152 2023-11-08T16:27:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 57/152 2023-11-08T16:27:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 58/152 2023-11-08T16:27:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 59/152 2023-11-08T16:27:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 60/152 2023-11-08T16:27:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 61/152 2023-11-08T16:27:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 62/152 2023-11-08T16:27:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 63/152 2023-11-08T16:27:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 64/152 2023-11-08T16:27:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 65/152 2023-11-08T16:27:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 66/152 2023-11-08T16:27:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 67/152 2023-11-08T16:27:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 68/152 2023-11-08T16:27:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/152 2023-11-08T16:27:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/152 2023-11-08T16:27:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/152 2023-11-08T16:27:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 72/152 2023-11-08T16:27:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 73/152 2023-11-08T16:27:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 74/152 2023-11-08T16:27:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 75/152 2023-11-08T16:27:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 76/152 2023-11-08T16:27:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 77/152 2023-11-08T16:27:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/152 2023-11-08T16:27:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/152 2023-11-08T16:27:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/152 2023-11-08T16:27:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/152 2023-11-08T16:27:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ply-3.11-14.el9.noarch 82/152 2023-11-08T16:27:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 83/152 2023-11-08T16:27:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 84/152 2023-11-08T16:27:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 85/152 2023-11-08T16:27:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 86/152 2023-11-08T16:27:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : automake-1.16.2-8.el9.noarch 87/152 2023-11-08T16:27:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 88/152 2023-11-08T16:27:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/152 2023-11-08T16:27:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/152 2023-11-08T16:27:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/152 2023-11-08T16:27:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/152 2023-11-08T16:27:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/152 2023-11-08T16:27:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/152 2023-11-08T16:27:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/152 2023-11-08T16:27:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/152 2023-11-08T16:27:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/152 2023-11-08T16:27:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/152 2023-11-08T16:27:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/152 2023-11-08T16:27:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/152 2023-11-08T16:27:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/152 2023-11-08T16:27:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/152 2023-11-08T16:27:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/152 2023-11-08T16:27:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/152 2023-11-08T16:27:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/152 2023-11-08T16:27:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/152 2023-11-08T16:27:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 107/152 2023-11-08T16:27:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/152 2023-11-08T16:27:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/152 2023-11-08T16:27:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/152 2023-11-08T16:27:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/152 2023-11-08T16:27:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/152 2023-11-08T16:27:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/152 2023-11-08T16:27:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/152 2023-11-08T16:27:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/152 2023-11-08T16:27:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/152 2023-11-08T16:27:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/152 2023-11-08T16:27:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/152 2023-11-08T16:27:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/152 2023-11-08T16:27:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/152 2023-11-08T16:27:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/152 2023-11-08T16:27:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/152 2023-11-08T16:27:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/152 2023-11-08T16:27:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/152 2023-11-08T16:27:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/152 2023-11-08T16:27:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/152 2023-11-08T16:27:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/152 2023-11-08T16:27:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/152 2023-11-08T16:27:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/152 2023-11-08T16:27:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/152 2023-11-08T16:27:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/152 2023-11-08T16:27:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/152 2023-11-08T16:27:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-2:18.0.0-7083.g85bdb240.el9.x86_64 133/152 2023-11-08T16:27:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/152 2023-11-08T16:27:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-2:18.0.0-7083.g85bdb240.el9.x86_64 135/152 2023-11-08T16:27:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/152 2023-11-08T16:27:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/152 2023-11-08T16:27:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/152 2023-11-08T16:27:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/152 2023-11-08T16:27:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/152 2023-11-08T16:27:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/152 2023-11-08T16:27:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/152 2023-11-08T16:27:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/152 2023-11-08T16:27:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/152 2023-11-08T16:27:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/152 2023-11-08T16:27:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/152 2023-11-08T16:27:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/152 2023-11-08T16:27:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/152 2023-11-08T16:27:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 149/152 2023-11-08T16:27:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-08T16:27:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 151/152 2023-11-08T16:27:33.960 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 144/152 2023-11-08T16:27:33.983 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 145/152 2023-11-08T16:27:34.016 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 146/152 2023-11-08T16:27:34.048 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 147/152 2023-11-08T16:27:34.088 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T16:27:34.088 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T16:27:34.431 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 148/152 2023-11-08T16:27:34.431 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-08T16:27:34.570 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 149/152 2023-11-08T16:27:34.601 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-08T16:27:34.601 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-08T16:27:34.740 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 151/152 2023-11-08T16:27:34.863 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T16:27:35.121 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T16:27:35.122 INFO:teuthology.orchestra.run.smithi081.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-08T16:27:35.122 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:35.958 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T16:27:35.958 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:35.958 INFO:teuthology.orchestra.run.smithi093.stdout:Upgraded: 2023-11-08T16:27:35.959 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-08T16:27:35.959 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-08T16:27:35.959 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-08T16:27:35.959 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-08T16:27:35.959 INFO:teuthology.orchestra.run.smithi093.stdout: acl-2.3.1-4.el9.x86_64 2023-11-08T16:27:35.960 INFO:teuthology.orchestra.run.smithi093.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-08T16:27:35.960 INFO:teuthology.orchestra.run.smithi093.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-08T16:27:35.960 INFO:teuthology.orchestra.run.smithi093.stdout: librados2-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.960 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.960 INFO:teuthology.orchestra.run.smithi093.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-08T16:27:35.961 INFO:teuthology.orchestra.run.smithi093.stdout:Installed: 2023-11-08T16:27:35.961 INFO:teuthology.orchestra.run.smithi093.stdout: automake-1.16.2-8.el9.noarch 2023-11-08T16:27:35.961 INFO:teuthology.orchestra.run.smithi093.stdout: bison-3.7.4-5.el9.x86_64 2023-11-08T16:27:35.961 INFO:teuthology.orchestra.run.smithi093.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-08T16:27:35.961 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.962 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.962 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.962 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.962 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:35.962 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.963 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.963 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.963 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:35.963 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:35.963 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:35.964 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:35.964 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:35.964 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.964 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.965 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:35.965 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.965 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.965 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.966 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:35.966 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:35.966 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.966 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:35.966 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-11-08T16:27:35.967 INFO:teuthology.orchestra.run.smithi093.stdout: flex-2.6.4-9.el9.x86_64 2023-11-08T16:27:35.967 INFO:teuthology.orchestra.run.smithi093.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-08T16:27:35.967 INFO:teuthology.orchestra.run.smithi093.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-08T16:27:35.967 INFO:teuthology.orchestra.run.smithi093.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-08T16:27:35.967 INFO:teuthology.orchestra.run.smithi093.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-08T16:27:35.968 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-08T16:27:35.968 INFO:teuthology.orchestra.run.smithi093.stdout: jq-1.6-16.el9.x86_64 2023-11-08T16:27:35.968 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-08T16:27:35.968 INFO:teuthology.orchestra.run.smithi093.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-08T16:27:35.969 INFO:teuthology.orchestra.run.smithi093.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-08T16:27:35.969 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.969 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.969 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.969 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-08T16:27:35.970 INFO:teuthology.orchestra.run.smithi093.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-08T16:27:35.970 INFO:teuthology.orchestra.run.smithi093.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-08T16:27:35.970 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-08T16:27:35.970 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-08T16:27:35.970 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.971 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.971 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-08T16:27:35.971 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.971 INFO:teuthology.orchestra.run.smithi093.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-08T16:27:35.971 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-08T16:27:35.972 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-08T16:27:35.972 INFO:teuthology.orchestra.run.smithi093.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-08T16:27:35.972 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-08T16:27:35.972 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-08T16:27:35.972 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-08T16:27:35.973 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-08T16:27:35.973 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-11-08T16:27:35.973 INFO:teuthology.orchestra.run.smithi093.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-08T16:27:35.973 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-13.11-1.el9.x86_64 2023-11-08T16:27:35.973 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-08T16:27:35.974 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-08T16:27:35.974 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-08T16:27:35.974 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-08T16:27:35.974 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-08T16:27:35.974 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-08T16:27:35.975 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-08T16:27:35.975 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.975 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.975 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.976 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-08T16:27:35.976 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-08T16:27:35.976 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-08T16:27:35.976 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-08T16:27:35.976 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-08T16:27:35.977 INFO:teuthology.orchestra.run.smithi093.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-08T16:27:35.977 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-11-08T16:27:35.977 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-08T16:27:35.977 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-08T16:27:35.977 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-08T16:27:35.978 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-08T16:27:35.978 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-08T16:27:35.978 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-08T16:27:35.978 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-08T16:27:35.979 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-08T16:27:35.979 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-08T16:27:35.979 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-08T16:27:35.979 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-08T16:27:35.979 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-08T16:27:35.980 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-08T16:27:35.980 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-08T16:27:35.980 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-08T16:27:35.980 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-08T16:27:35.980 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-08T16:27:35.981 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-08T16:27:35.981 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-08T16:27:35.981 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-08T16:27:35.981 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ply-3.11-14.el9.noarch 2023-11-08T16:27:35.981 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-08T16:27:35.982 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-08T16:27:35.982 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-08T16:27:35.982 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-08T16:27:35.982 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-08T16:27:35.982 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-08T16:27:35.983 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-08T16:27:35.983 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.983 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.983 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-08T16:27:35.983 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-08T16:27:35.984 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-08T16:27:35.984 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.984 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-08T16:27:35.984 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-08T16:27:35.984 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-08T16:27:35.985 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-08T16:27:35.985 INFO:teuthology.orchestra.run.smithi093.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-08T16:27:35.985 INFO:teuthology.orchestra.run.smithi093.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-08T16:27:35.985 INFO:teuthology.orchestra.run.smithi093.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-08T16:27:35.986 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-08T16:27:35.986 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-08T16:27:35.986 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-08T16:27:35.986 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-08T16:27:35.987 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.987 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.987 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:35.987 INFO:teuthology.orchestra.run.smithi093.stdout: re2-1:20211101-3.el9.x86_64 2023-11-08T16:27:35.987 INFO:teuthology.orchestra.run.smithi093.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-08T16:27:35.988 INFO:teuthology.orchestra.run.smithi093.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-08T16:27:35.988 INFO:teuthology.orchestra.run.smithi093.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-08T16:27:35.988 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-08T16:27:35.988 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-08T16:27:35.989 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-08T16:27:35.989 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:35.989 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-11-08T16:27:36.477 DEBUG:teuthology.parallel:result is None 2023-11-08T16:27:39.842 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-2:18.0.0-7083.g85bdb240.el9.x86_64 1/152 2023-11-08T16:27:39.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-base-2:18.0.0-7083.g85bdb240.el9.x86_64 2/152 2023-11-08T16:27:39.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 3/152 2023-11-08T16:27:39.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 4/152 2023-11-08T16:27:39.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7083.g85bdb 5/152 2023-11-08T16:27:39.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mds-2:18.0.0-7083.g85bdb240.el9.x86_64 6/152 2023-11-08T16:27:39.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-2:18.0.0-7083.g85bdb240.el9.x86_64 7/152 2023-11-08T16:27:39.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mon-2:18.0.0-7083.g85bdb240.el9.x86_64 8/152 2023-11-08T16:27:39.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-osd-2:18.0.0-7083.g85bdb240.el9.x86_64 9/152 2023-11-08T16:27:39.844 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-radosgw-2:18.0.0-7083.g85bdb240.el9.x86_64 10/152 2023-11-08T16:27:39.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-selinux-2:18.0.0-7083.g85bdb240.el9.x86_64 11/152 2023-11-08T16:27:39.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-test-2:18.0.0-7083.g85bdb240.el9.x86_64 12/152 2023-11-08T16:27:39.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 13/152 2023-11-08T16:27:39.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs-devel-2:18.0.0-7083.g85bdb240.el9.x86_ 14/152 2023-11-08T16:27:39.845 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephfs2-2:18.0.0-7083.g85bdb240.el9.x86_64 15/152 2023-11-08T16:27:39.846 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libcephsqlite-2:18.0.0-7083.g85bdb240.el9.x86_64 16/152 2023-11-08T16:27:39.846 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados-devel-2:18.0.0-7083.g85bdb240.el9.x86_6 17/152 2023-11-08T16:27:39.846 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libradosstriper1-2:18.0.0-7083.g85bdb240.el9.x86 18/152 2023-11-08T16:27:39.846 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librgw2-2:18.0.0-7083.g85bdb240.el9.x86_64 19/152 2023-11-08T16:27:39.846 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7083.g85bdb240.el 20/152 2023-11-08T16:27:39.847 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ceph-common-2:18.0.0-7083.g85bdb240.el9. 21/152 2023-11-08T16:27:39.847 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cephfs-2:18.0.0-7083.g85bdb240.el9.x86_6 22/152 2023-11-08T16:27:39.847 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rados-2:18.0.0-7083.g85bdb240.el9.x86_64 23/152 2023-11-08T16:27:39.847 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rbd-2:18.0.0-7083.g85bdb240.el9.x86_64 24/152 2023-11-08T16:27:39.847 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rgw-2:18.0.0-7083.g85bdb240.el9.x86_64 25/152 2023-11-08T16:27:39.848 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 26/152 2023-11-08T16:27:39.848 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 27/152 2023-11-08T16:27:39.848 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : rbd-nbd-2:18.0.0-7083.g85bdb240.el9.x86_64 28/152 2023-11-08T16:27:39.860 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7083.g85bdb240. 29/152 2023-11-08T16:27:39.860 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7083.g85bdb240.el9.noa 30/152 2023-11-08T16:27:39.860 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7083.g85bdb240.el9.n 31/152 2023-11-08T16:27:39.861 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7083.g85b 32/152 2023-11-08T16:27:39.861 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7083.g85bdb240.el 33/152 2023-11-08T16:27:39.861 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7083.g85bdb240.el9.noarch 34/152 2023-11-08T16:27:39.861 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7083.g85bdb240.e 35/152 2023-11-08T16:27:39.861 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : ceph-volume-2:18.0.0-7083.g85bdb240.el9.noarch 36/152 2023-11-08T16:27:39.862 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephadm-2:18.0.0-7083.g85bdb240.el9.noarch 37/152 2023-11-08T16:27:39.862 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : cephfs-top-2:18.0.0-7083.g85bdb240.el9.noarch 38/152 2023-11-08T16:27:39.862 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : bison-3.7.4-5.el9.x86_64 39/152 2023-11-08T16:27:39.862 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 40/152 2023-11-08T16:27:39.862 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flex-2.6.4-9.el9.x86_64 41/152 2023-11-08T16:27:39.863 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 42/152 2023-11-08T16:27:39.863 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 43/152 2023-11-08T16:27:39.863 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 44/152 2023-11-08T16:27:39.863 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : jq-1.6-16.el9.x86_64 45/152 2023-11-08T16:27:39.863 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 46/152 2023-11-08T16:27:39.864 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 47/152 2023-11-08T16:27:39.864 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 48/152 2023-11-08T16:27:39.864 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 49/152 2023-11-08T16:27:39.864 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 50/152 2023-11-08T16:27:39.864 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 51/152 2023-11-08T16:27:39.865 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 52/152 2023-11-08T16:27:39.865 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 53/152 2023-11-08T16:27:39.865 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 54/152 2023-11-08T16:27:39.865 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-13.11-1.el9.x86_64 55/152 2023-11-08T16:27:39.865 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 56/152 2023-11-08T16:27:39.866 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 57/152 2023-11-08T16:27:39.866 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 58/152 2023-11-08T16:27:39.866 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 59/152 2023-11-08T16:27:39.866 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 60/152 2023-11-08T16:27:39.866 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 61/152 2023-11-08T16:27:39.867 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 62/152 2023-11-08T16:27:39.867 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 63/152 2023-11-08T16:27:39.867 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 64/152 2023-11-08T16:27:39.867 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 65/152 2023-11-08T16:27:39.867 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 66/152 2023-11-08T16:27:39.868 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 67/152 2023-11-08T16:27:39.868 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 68/152 2023-11-08T16:27:39.868 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 69/152 2023-11-08T16:27:39.868 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 70/152 2023-11-08T16:27:39.868 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 71/152 2023-11-08T16:27:39.869 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 72/152 2023-11-08T16:27:39.869 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 73/152 2023-11-08T16:27:39.869 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 74/152 2023-11-08T16:27:39.869 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 75/152 2023-11-08T16:27:39.870 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 76/152 2023-11-08T16:27:39.870 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 77/152 2023-11-08T16:27:39.870 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 78/152 2023-11-08T16:27:39.871 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 79/152 2023-11-08T16:27:39.871 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 80/152 2023-11-08T16:27:39.871 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 81/152 2023-11-08T16:27:39.871 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-ply-3.11-14.el9.noarch 82/152 2023-11-08T16:27:39.871 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 83/152 2023-11-08T16:27:39.872 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 84/152 2023-11-08T16:27:39.872 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 85/152 2023-11-08T16:27:39.872 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 86/152 2023-11-08T16:27:39.872 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : automake-1.16.2-8.el9.noarch 87/152 2023-11-08T16:27:39.872 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : elfutils-libelf-devel-0.189-3.el9.x86_64 88/152 2023-11-08T16:27:39.873 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 89/152 2023-11-08T16:27:39.873 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 90/152 2023-11-08T16:27:39.873 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : openssl-devel-1:3.0.7-24.el9.x86_64 91/152 2023-11-08T16:27:39.873 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/152 2023-11-08T16:27:39.873 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 93/152 2023-11-08T16:27:39.874 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 94/152 2023-11-08T16:27:39.874 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/152 2023-11-08T16:27:39.874 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/152 2023-11-08T16:27:39.874 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/152 2023-11-08T16:27:39.874 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/152 2023-11-08T16:27:39.875 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/152 2023-11-08T16:27:39.875 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/152 2023-11-08T16:27:39.875 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/152 2023-11-08T16:27:39.875 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/152 2023-11-08T16:27:39.875 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/152 2023-11-08T16:27:39.876 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/152 2023-11-08T16:27:39.876 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/152 2023-11-08T16:27:39.876 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/152 2023-11-08T16:27:39.876 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-google-auth-1:2.23.3-1.el9.noarch 107/152 2023-11-08T16:27:39.876 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/152 2023-11-08T16:27:39.877 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/152 2023-11-08T16:27:39.877 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/152 2023-11-08T16:27:39.877 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/152 2023-11-08T16:27:39.877 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/152 2023-11-08T16:27:39.877 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/152 2023-11-08T16:27:39.877 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/152 2023-11-08T16:27:39.878 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/152 2023-11-08T16:27:39.878 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/152 2023-11-08T16:27:39.878 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/152 2023-11-08T16:27:39.878 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/152 2023-11-08T16:27:39.879 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/152 2023-11-08T16:27:39.879 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/152 2023-11-08T16:27:39.879 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/152 2023-11-08T16:27:39.879 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/152 2023-11-08T16:27:39.879 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/152 2023-11-08T16:27:39.879 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/152 2023-11-08T16:27:39.880 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/152 2023-11-08T16:27:39.880 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/152 2023-11-08T16:27:39.880 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/152 2023-11-08T16:27:39.880 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/152 2023-11-08T16:27:39.880 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/152 2023-11-08T16:27:39.881 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/152 2023-11-08T16:27:39.881 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/152 2023-11-08T16:27:39.881 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/152 2023-11-08T16:27:39.881 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:18.0.0-7083.g85bdb240.el9.x86_64 133/152 2023-11-08T16:27:39.881 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/152 2023-11-08T16:27:39.882 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:18.0.0-7083.g85bdb240.el9.x86_64 135/152 2023-11-08T16:27:39.882 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/152 2023-11-08T16:27:39.882 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/152 2023-11-08T16:27:39.882 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/152 2023-11-08T16:27:39.882 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/152 2023-11-08T16:27:39.883 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/152 2023-11-08T16:27:39.883 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/152 2023-11-08T16:27:39.883 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/152 2023-11-08T16:27:39.883 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/152 2023-11-08T16:27:39.883 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/152 2023-11-08T16:27:39.884 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/152 2023-11-08T16:27:39.884 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/152 2023-11-08T16:27:39.884 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/152 2023-11-08T16:27:39.884 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/152 2023-11-08T16:27:39.884 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 149/152 2023-11-08T16:27:39.885 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 150/152 2023-11-08T16:27:39.885 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 151/152 2023-11-08T16:27:41.953 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 152/152 2023-11-08T16:27:41.953 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:41.954 INFO:teuthology.orchestra.run.smithi081.stdout:Upgraded: 2023-11-08T16:27:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-08T16:27:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-08T16:27:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-08T16:27:41.954 INFO:teuthology.orchestra.run.smithi081.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-08T16:27:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: acl-2.3.1-4.el9.x86_64 2023-11-08T16:27:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-08T16:27:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-08T16:27:41.955 INFO:teuthology.orchestra.run.smithi081.stdout: librados2-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-08T16:27:41.956 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2023-11-08T16:27:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: automake-1.16.2-8.el9.noarch 2023-11-08T16:27:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: bison-3.7.4-5.el9.x86_64 2023-11-08T16:27:41.956 INFO:teuthology.orchestra.run.smithi081.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-08T16:27:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.957 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-immutable-object-cache-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.958 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-cephadm-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-modules-core-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:41.959 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-prometheus-alerts-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.960 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-volume-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: cephadm-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: cephfs-top-2:18.0.0-7083.g85bdb240.el9.noarch 2023-11-08T16:27:41.961 INFO:teuthology.orchestra.run.smithi081.stdout: elfutils-libelf-devel-0.189-3.el9.x86_64 2023-11-08T16:27:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: flex-2.6.4-9.el9.x86_64 2023-11-08T16:27:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-08T16:27:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-08T16:27:41.962 INFO:teuthology.orchestra.run.smithi081.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-08T16:27:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-08T16:27:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-08T16:27:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: jq-1.6-16.el9.x86_64 2023-11-08T16:27:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-08T16:27:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-08T16:27:41.963 INFO:teuthology.orchestra.run.smithi081.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-08T16:27:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: libcephsqlite-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.964 INFO:teuthology.orchestra.run.smithi081.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-08T16:27:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-08T16:27:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-08T16:27:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-08T16:27:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-08T16:27:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.965 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-08T16:27:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-08T16:27:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-08T16:27:41.966 INFO:teuthology.orchestra.run.smithi081.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-08T16:27:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-08T16:27:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-08T16:27:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-08T16:27:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-08T16:27:41.967 INFO:teuthology.orchestra.run.smithi081.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-08T16:27:41.968 INFO:teuthology.orchestra.run.smithi081.stdout: openssl-devel-1:3.0.7-24.el9.x86_64 2023-11-08T16:27:41.968 INFO:teuthology.orchestra.run.smithi081.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-08T16:27:41.968 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-13.11-1.el9.x86_64 2023-11-08T16:27:41.968 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-08T16:27:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-08T16:27:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-08T16:27:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-08T16:27:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-08T16:27:41.969 INFO:teuthology.orchestra.run.smithi081.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-08T16:27:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-08T16:27:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-argparse-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ceph-common-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.970 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cephfs-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-08T16:27:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-08T16:27:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-08T16:27:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-08T16:27:41.971 INFO:teuthology.orchestra.run.smithi081.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-08T16:27:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-08T16:27:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: python3-google-auth-1:2.23.3-1.el9.noarch 2023-11-08T16:27:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-08T16:27:41.972 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-08T16:27:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-08T16:27:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-08T16:27:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-08T16:27:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-08T16:27:41.973 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-08T16:27:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-08T16:27:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-08T16:27:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-08T16:27:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-08T16:27:41.974 INFO:teuthology.orchestra.run.smithi081.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-08T16:27:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-08T16:27:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-08T16:27:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-08T16:27:41.975 INFO:teuthology.orchestra.run.smithi081.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-08T16:27:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-08T16:27:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-08T16:27:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-08T16:27:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-08T16:27:41.976 INFO:teuthology.orchestra.run.smithi081.stdout: python3-ply-3.11-14.el9.noarch 2023-11-08T16:27:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-08T16:27:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-08T16:27:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-08T16:27:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-08T16:27:41.977 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-08T16:27:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-08T16:27:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-08T16:27:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rados-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rbd-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.978 INFO:teuthology.orchestra.run.smithi081.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-08T16:27:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-08T16:27:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-08T16:27:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rgw-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-08T16:27:41.979 INFO:teuthology.orchestra.run.smithi081.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-08T16:27:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-08T16:27:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-08T16:27:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-08T16:27:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-08T16:27:41.980 INFO:teuthology.orchestra.run.smithi081.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-08T16:27:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-08T16:27:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-08T16:27:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-08T16:27:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-08T16:27:41.981 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-mirror-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-nbd-2:18.0.0-7083.g85bdb240.el9.x86_64 2023-11-08T16:27:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: re2-1:20211101-3.el9.x86_64 2023-11-08T16:27:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-08T16:27:41.982 INFO:teuthology.orchestra.run.smithi081.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-08T16:27:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-08T16:27:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-08T16:27:41.983 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-08T16:27:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-08T16:27:41.984 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:41.984 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2023-11-08T16:27:42.549 DEBUG:teuthology.parallel:result is None 2023-11-08T16:27:42.550 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=85bdb240f5cf2a758acc9b247951fbc8f5799bc1 2023-11-08T16:27:42.689 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-08T16:27:42.746 INFO:teuthology.orchestra.run.smithi081.stdout:18.0.0-7083.g85bdb240.el9 2023-11-08T16:27:42.747 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7083.g85bdb240.el9 2023-11-08T16:27:42.747 INFO:teuthology.task.install:The correct ceph version 18.0.0-7083.g85bdb240 is installed. 2023-11-08T16:27:42.748 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=85bdb240f5cf2a758acc9b247951fbc8f5799bc1 2023-11-08T16:27:42.887 DEBUG:teuthology.orchestra.run.smithi093:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-08T16:27:42.960 INFO:teuthology.orchestra.run.smithi093.stdout:18.0.0-7083.g85bdb240.el9 2023-11-08T16:27:42.960 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7083.g85bdb240.el9 2023-11-08T16:27:42.961 INFO:teuthology.task.install:The correct ceph version 18.0.0-7083.g85bdb240 is installed. 2023-11-08T16:27:42.961 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-08T16:27:42.962 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:27:42.962 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-08T16:27:43.052 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:27:43.052 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-08T16:27:43.196 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-08T16:27:43.197 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:27:43.197 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/daemon-helper 2023-11-08T16:27:43.281 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-08T16:27:43.403 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:27:43.404 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/daemon-helper 2023-11-08T16:27:43.508 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-08T16:27:43.602 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-08T16:27:43.603 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:27:43.603 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-08T16:27:43.681 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-08T16:27:43.819 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:27:43.820 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-08T16:27:43.923 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-08T16:27:44.059 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-08T16:27:44.060 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:27:44.060 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/usr/bin/stdin-killer 2023-11-08T16:27:44.150 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-08T16:27:44.265 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:27:44.266 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/stdin-killer 2023-11-08T16:27:44.367 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-08T16:27:44.462 INFO:teuthology.run_tasks:Running task ceph... 2023-11-08T16:27:44.530 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-08T16:27:44.531 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /var/log/ceph 2023-11-08T16:27:44.539 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 777 /var/log/ceph 2023-11-08T16:27:44.642 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-08T16:27:44.643 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-08T16:27:44.647 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-08T16:27:44.742 INFO:tasks.ceph:Creating extra log directories... 2023-11-08T16:27:44.742 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-08T16:27:44.747 DEBUG:teuthology.orchestra.run.smithi093:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-08T16:27:44.857 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-08T16:27:44.857 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-08T16:27:44.858 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-08T16:27:44.858 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:27:44.859 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-08T16:27:44.901 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-08T16:27:45.113 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-08T16:27:45.189 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:27:45.190 DEBUG:teuthology.orchestra.run.smithi093:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-08T16:27:45.250 DEBUG:teuthology.orchestra.run.smithi093:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-08T16:27:45.508 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-08T16:27:45.608 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-08T16:27:45.609 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-08T16:27:45.609 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595', 'branch': 'wip-vshankar-testing-20231106.073650', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7449595', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing-20231106.073650'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'sha1': '85bdb240f5cf2a758acc9b247951fbc8f5799bc1'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '85bdb240f5cf2a758acc9b247951fbc8f5799bc1'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing-20231106.073650', 'sha1': '85bdb240f5cf2a758acc9b247951fbc8f5799bc1'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 120, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 8308, 'sha1': '85bdb240f5cf2a758acc9b247951fbc8f5799bc1', 'sleep_before_teardown': 0, 'subset': '430/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing-20231106.073650', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '85bdb240f5cf2a758acc9b247951fbc8f5799bc1', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi081.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEb/Zd1RkyJlvHItW402GsLGrVK/DxAP1UQ3Ffdb0JiiZSX55PYP8qQj1NkQP07dHaUwrlKgI7Xouph9A5trxkU=', 'smithi093.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBITVhNVsh9QJgG0rtGpBJz4qGMcOK6+9C0lzvZOaEmIqMbNTLzfFp3YuUcIX2Eq0w7oJ2Tjqx/svsh6/pCfWsGY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '5b73517e0e2fa132ea79290e5fc06ea5349b2288'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '85bdb240f5cf2a758acc9b247951fbc8f5799bc1'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'inconsistent rstat on inode', 'Error recovering journal'], 'sha1': '85bdb240f5cf2a758acc9b247951fbc8f5799bc1', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_data_scan']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '6899cd26fceddb2fec83dc1a1349394b28c8998e', 'timestamp': '2023-11-06_10:33:49', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3590714'} 2023-11-08T16:27:45.611 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-08T16:27:45.618 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-08T16:27:45.668 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/run/ceph 2023-11-08T16:27:45.705 DEBUG:teuthology.orchestra.run.smithi093:> sudo install -d -m0777 -- /var/run/ceph 2023-11-08T16:27:45.787 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:27:45.787 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2023-11-08T16:27:45.828 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-08T16:27:45.828 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2023-11-08T16:27:45.927 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-08T16:27:45.927 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T16:27:45.928 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 634 Links: 1 2023-11-08T16:27:45.928 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T16:27:45.928 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T16:27:45.928 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2023-11-08 16:27:35.568452972 +0000 2023-11-08T16:27:45.928 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2023-11-08 16:25:04.763584853 +0000 2023-11-08T16:27:45.929 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2023-11-08 16:25:04.763584853 +0000 2023-11-08T16:27:45.929 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2023-11-08 16:25:04.763584853 +0000 2023-11-08T16:27:45.929 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-08T16:27:46.017 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2023-11-08T16:27:46.017 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2023-11-08T16:27:46.017 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000410429 s, 1.2 MB/s 2023-11-08T16:27:46.021 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-08T16:27:46.074 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2023-11-08T16:27:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-08T16:27:46.118 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T16:27:46.118 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 651 Links: 1 2023-11-08T16:27:46.118 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T16:27:46.119 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T16:27:46.119 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2023-11-08 16:27:35.569452954 +0000 2023-11-08T16:27:46.119 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2023-11-08 16:25:05.770566121 +0000 2023-11-08T16:27:46.119 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2023-11-08 16:25:05.770566121 +0000 2023-11-08T16:27:46.120 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2023-11-08 16:25:05.770566121 +0000 2023-11-08T16:27:46.120 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-08T16:27:46.245 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2023-11-08T16:27:46.245 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2023-11-08T16:27:46.245 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000320602 s, 1.6 MB/s 2023-11-08T16:27:46.248 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-08T16:27:46.297 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2023-11-08T16:27:46.381 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-08T16:27:46.381 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T16:27:46.381 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 670 Links: 1 2023-11-08T16:27:46.381 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T16:27:46.382 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T16:27:46.382 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2023-11-08 16:27:35.570452936 +0000 2023-11-08T16:27:46.382 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2023-11-08 16:25:06.781547314 +0000 2023-11-08T16:27:46.382 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2023-11-08 16:25:06.781547314 +0000 2023-11-08T16:27:46.382 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2023-11-08 16:25:06.781547314 +0000 2023-11-08T16:27:46.383 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-08T16:27:46.497 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2023-11-08T16:27:46.497 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2023-11-08T16:27:46.497 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000323354 s, 1.6 MB/s 2023-11-08T16:27:46.500 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-08T16:27:46.547 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2023-11-08T16:27:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-08T16:27:46.589 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T16:27:46.590 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-11-08T16:27:46.590 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T16:27:46.590 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T16:27:46.590 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2023-11-08 16:27:35.570452936 +0000 2023-11-08T16:27:46.590 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2023-11-08 16:25:07.780528730 +0000 2023-11-08T16:27:46.591 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2023-11-08 16:25:07.780528730 +0000 2023-11-08T16:27:46.591 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2023-11-08 16:25:07.780528730 +0000 2023-11-08T16:27:46.591 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-08T16:27:46.722 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2023-11-08T16:27:46.722 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2023-11-08T16:27:46.722 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000338067 s, 1.5 MB/s 2023-11-08T16:27:46.725 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-08T16:27:46.780 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T16:27:46.781 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:27:46.781 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/scratch_devs of=/dev/stdout 2023-11-08T16:27:46.837 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-08T16:27:46.838 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_1 2023-11-08T16:27:46.889 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-08T16:27:46.890 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T16:27:46.890 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 5h/5d Inode: 634 Links: 1 2023-11-08T16:27:46.890 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T16:27:46.890 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T16:27:46.891 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-11-08 16:27:29.702200920 +0000 2023-11-08T16:27:46.891 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-11-08 16:25:05.456602009 +0000 2023-11-08T16:27:46.891 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-11-08 16:25:05.456602009 +0000 2023-11-08T16:27:46.891 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: 2023-11-08 16:25:05.456602009 +0000 2023-11-08T16:27:46.892 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-08T16:27:46.983 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-11-08T16:27:46.983 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-11-08T16:27:46.983 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000337823 s, 1.5 MB/s 2023-11-08T16:27:46.987 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-08T16:27:47.048 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_2 2023-11-08T16:27:47.101 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-08T16:27:47.102 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T16:27:47.102 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 5h/5d Inode: 650 Links: 1 2023-11-08T16:27:47.102 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T16:27:47.102 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T16:27:47.103 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-11-08 16:27:29.703200899 +0000 2023-11-08T16:27:47.103 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-11-08 16:25:06.476581058 +0000 2023-11-08T16:27:47.103 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-11-08 16:25:06.476581058 +0000 2023-11-08T16:27:47.103 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: 2023-11-08 16:25:06.476581058 +0000 2023-11-08T16:27:47.104 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-08T16:27:47.192 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-11-08T16:27:47.192 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-11-08T16:27:47.193 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000317004 s, 1.6 MB/s 2023-11-08T16:27:47.196 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-08T16:27:47.255 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_3 2023-11-08T16:27:47.306 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-08T16:27:47.307 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T16:27:47.307 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 5h/5d Inode: 669 Links: 1 2023-11-08T16:27:47.307 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T16:27:47.307 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T16:27:47.307 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-11-08 16:27:29.703200899 +0000 2023-11-08T16:27:47.308 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-11-08 16:25:07.490560229 +0000 2023-11-08T16:27:47.308 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-11-08 16:25:07.490560229 +0000 2023-11-08T16:27:47.308 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: 2023-11-08 16:25:07.490560229 +0000 2023-11-08T16:27:47.308 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-08T16:27:47.398 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-11-08T16:27:47.398 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-11-08T16:27:47.399 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000332921 s, 1.5 MB/s 2023-11-08T16:27:47.402 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-08T16:27:47.462 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_4 2023-11-08T16:27:47.511 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-08T16:27:47.512 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-08T16:27:47.512 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 5h/5d Inode: 689 Links: 1 2023-11-08T16:27:47.513 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-08T16:27:47.513 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-11-08T16:27:47.513 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-11-08 16:27:29.704200879 +0000 2023-11-08T16:27:47.514 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-11-08 16:25:08.531538846 +0000 2023-11-08T16:27:47.514 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-11-08 16:25:08.531538846 +0000 2023-11-08T16:27:47.514 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: 2023-11-08 16:25:08.531538846 +0000 2023-11-08T16:27:47.515 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-08T16:27:47.604 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-11-08T16:27:47.605 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-11-08T16:27:47.605 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000327446 s, 1.6 MB/s 2023-11-08T16:27:47.608 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-08T16:27:47.667 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T16:27:47.668 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi093.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-08T16:27:47.668 INFO:tasks.ceph:Generating config... 2023-11-08T16:27:47.669 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-08T16:27:47.670 INFO:tasks.ceph:[client] debug client = 20 2023-11-08T16:27:47.670 INFO:tasks.ceph:[client] debug ms = 1 2023-11-08T16:27:47.670 INFO:tasks.ceph:[client] pid file = 2023-11-08T16:27:47.670 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-08T16:27:47.671 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-08T16:27:47.671 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-08T16:27:47.672 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-08T16:27:47.672 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-08T16:27:47.672 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-08T16:27:47.672 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-08T16:27:47.673 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-08T16:27:47.673 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-08T16:27:47.673 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-08T16:27:47.674 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-08T16:27:47.674 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-08T16:27:47.674 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-08T16:27:47.674 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-08T16:27:47.675 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-08T16:27:47.675 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-08T16:27:47.675 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-08T16:27:47.675 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-08T16:27:47.676 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-08T16:27:47.676 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-08T16:27:47.676 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-08T16:27:47.677 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-08T16:27:47.677 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-08T16:27:47.677 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-08T16:27:47.678 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-08T16:27:47.678 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-08T16:27:47.678 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-08T16:27:47.678 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-08T16:27:47.679 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-08T16:27:47.679 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-08T16:27:47.680 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-08T16:27:47.680 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-08T16:27:47.680 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-08T16:27:47.680 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-08T16:27:47.681 INFO:tasks.ceph:Setting up mon.a... 2023-11-08T16:27:47.681 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-08T16:27:47.828 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.keyring 2023-11-08T16:27:47.836 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-08T16:27:47.970 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T16:27:48.057 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.81'), ('mon.b', '172.21.15.93'), ('mon.c', '[v2:172.21.15.93:3301,v1:172.21.15.93:6790]')] 2023-11-08T16:27:48.058 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-08T16:27:48.060 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:27:48.061 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-08T16:27:48.118 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.81 --add b 172.21.15.93 --addv c '[v2:172.21.15.93:3301,v1:172.21.15.93:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:27:48.273 INFO:teuthology.orchestra.run.smithi081.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:27:48.274 INFO:teuthology.orchestra.run.smithi081.stdout:monmaptool: generated fsid 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:27:48.274 INFO:teuthology.orchestra.run.smithi081.stdout:setting min_mon_release = pacific 2023-11-08T16:27:48.274 INFO:teuthology.orchestra.run.smithi081.stdout:epoch 0 2023-11-08T16:27:48.275 INFO:teuthology.orchestra.run.smithi081.stdout:fsid 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:27:48.275 INFO:teuthology.orchestra.run.smithi081.stdout:last_changed 2023-11-08T16:27:48.272771+0000 2023-11-08T16:27:48.275 INFO:teuthology.orchestra.run.smithi081.stdout:created 2023-11-08T16:27:48.272771+0000 2023-11-08T16:27:48.275 INFO:teuthology.orchestra.run.smithi081.stdout:min_mon_release 16 (pacific) 2023-11-08T16:27:48.276 INFO:teuthology.orchestra.run.smithi081.stdout:election_strategy: 1 2023-11-08T16:27:48.276 INFO:teuthology.orchestra.run.smithi081.stdout:0: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.a 2023-11-08T16:27:48.277 INFO:teuthology.orchestra.run.smithi081.stdout:1: [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] mon.b 2023-11-08T16:27:48.277 INFO:teuthology.orchestra.run.smithi081.stdout:2: [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] mon.c 2023-11-08T16:27:48.277 INFO:teuthology.orchestra.run.smithi081.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-08T16:27:48.279 DEBUG:teuthology.orchestra.run.smithi081:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-08T16:27:48.329 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 49b142e1-3087-4ba1-853b-235f756dae94... 2023-11-08T16:27:48.331 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:27:48.337 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:27:48.481 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2023-11-08T16:27:48.481 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2023-11-08T16:27:48.481 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:27:48.481 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2023-11-08T16:27:48.482 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.482 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2023-11-08T16:27:48.482 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.482 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2023-11-08T16:27:48.482 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.483 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2023-11-08T16:27:48.483 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2023-11-08T16:27:48.483 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.483 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2023-11-08T16:27:48.483 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2023-11-08T16:27:48.484 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.484 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:27:48.484 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2023-11-08T16:27:48.484 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.484 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:27:48.485 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:27:48.485 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.485 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2023-11-08T16:27:48.485 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.485 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2023-11-08T16:27:48.486 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2023-11-08T16:27:48.486 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:27:48.486 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:27:48.486 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2023-11-08T16:27:48.486 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:27:48.487 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2023-11-08T16:27:48.487 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:27:48.487 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:27:48.487 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2023-11-08T16:27:48.487 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.487 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:27:48.488 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.488 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2023-11-08T16:27:48.488 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.488 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2023-11-08T16:27:48.488 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.489 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2023-11-08T16:27:48.489 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2023-11-08T16:27:48.489 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2023-11-08T16:27:48.489 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:27:48.489 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:27:48.490 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.490 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2023-11-08T16:27:48.490 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2023-11-08T16:27:48.490 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.490 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2023-11-08T16:27:48.490 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2023-11-08T16:27:48.491 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:27:48.491 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.491 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2023-11-08T16:27:48.491 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2023-11-08T16:27:48.491 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.492 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2023-11-08T16:27:48.492 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2023-11-08T16:27:48.492 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2023-11-08T16:27:48.492 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2023-11-08T16:27:48.492 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.492 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2023-11-08T16:27:48.493 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2023-11-08T16:27:48.493 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.493 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:27:48.493 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.493 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2023-11-08T16:27:48.494 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.494 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2023-11-08T16:27:48.494 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.494 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:27:48.494 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2023-11-08T16:27:48.495 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.495 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2023-11-08T16:27:48.495 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2023-11-08T16:27:48.495 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2023-11-08T16:27:48.495 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2023-11-08T16:27:48.495 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2023-11-08T16:27:48.496 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2023-11-08T16:27:48.496 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2023-11-08T16:27:48.496 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2023-11-08T16:27:48.496 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2023-11-08T16:27:48.496 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:27:48.497 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2023-11-08T16:27:48.497 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2023-11-08T16:27:48.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:27:48.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2023-11-08T16:27:48.497 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:27:48.498 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:27:48.498 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2023-11-08T16:27:48.498 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:27:48.498 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.498 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2023-11-08T16:27:48.498 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:27:48.499 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2023-11-08T16:27:48.499 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:27:48.499 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:27:48.499 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:27:48.499 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:27:48.500 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2023-11-08T16:27:48.500 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.500 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2023-11-08T16:27:48.500 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:27:48.500 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:27:48.501 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2023-11-08T16:27:48.501 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:27:48.501 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2023-11-08T16:27:48.501 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2023-11-08T16:27:48.501 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:27:48.501 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2023-11-08T16:27:48.502 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2023-11-08T16:27:48.502 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:27:48.502 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.502 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:27:48.502 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:27:48.503 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:27:48.503 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.503 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:27:48.503 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:27:48.503 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:27:48.503 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.504 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2023-11-08T16:27:48.504 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:27:48.504 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.504 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:27:48.504 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2023-11-08T16:27:48.505 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:27:48.505 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2023-11-08T16:27:48.505 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2023-11-08T16:27:48.505 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2023-11-08T16:27:48.505 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2023-11-08T16:27:48.505 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:27:48.506 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:27:48.506 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2023-11-08T16:27:48.506 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2023-11-08T16:27:48.506 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:27:48.506 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2023-11-08T16:27:48.507 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:27:48.507 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:27:48.507 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2023-11-08T16:27:48.507 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2023-11-08T16:27:48.507 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2023-11-08T16:27:48.508 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:27:48.508 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:27:48.508 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:27:48.508 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:27:48.508 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2023-11-08T16:27:48.509 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2023-11-08T16:27:48.509 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2023-11-08T16:27:48.509 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:27:48.509 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2023-11-08T16:27:48.509 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = True 2023-11-08T16:27:48.509 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2023-11-08T16:27:48.510 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = True 2023-11-08T16:27:48.510 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:27:48.510 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:27:48.510 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:27:48.515 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-08T16:27:48.515 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2023-11-08T16:27:48.515 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:27:48.515 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2023-11-08T16:27:48.515 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.516 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2023-11-08T16:27:48.516 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.516 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2023-11-08T16:27:48.516 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.516 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2023-11-08T16:27:48.517 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2023-11-08T16:27:48.517 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.517 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2023-11-08T16:27:48.517 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2023-11-08T16:27:48.517 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.518 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:27:48.518 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2023-11-08T16:27:48.518 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.518 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:27:48.518 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:27:48.519 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.519 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2023-11-08T16:27:48.519 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.519 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2023-11-08T16:27:48.519 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2023-11-08T16:27:48.520 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:27:48.520 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:27:48.520 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2023-11-08T16:27:48.520 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:27:48.520 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2023-11-08T16:27:48.520 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:27:48.521 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:27:48.521 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2023-11-08T16:27:48.521 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.521 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:27:48.521 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.522 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2023-11-08T16:27:48.522 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.522 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2023-11-08T16:27:48.522 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.522 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2023-11-08T16:27:48.523 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2023-11-08T16:27:48.523 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2023-11-08T16:27:48.523 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:27:48.523 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:27:48.523 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.523 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2023-11-08T16:27:48.524 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2023-11-08T16:27:48.524 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.524 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2023-11-08T16:27:48.524 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2023-11-08T16:27:48.524 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:27:48.525 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.525 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2023-11-08T16:27:48.525 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2023-11-08T16:27:48.525 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.525 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2023-11-08T16:27:48.525 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2023-11-08T16:27:48.526 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2023-11-08T16:27:48.526 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2023-11-08T16:27:48.526 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.526 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2023-11-08T16:27:48.526 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2023-11-08T16:27:48.527 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.527 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:27:48.527 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.527 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2023-11-08T16:27:48.527 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.527 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2023-11-08T16:27:48.528 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.528 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:27:48.528 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2023-11-08T16:27:48.528 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.528 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2023-11-08T16:27:48.529 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2023-11-08T16:27:48.529 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2023-11-08T16:27:48.529 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2023-11-08T16:27:48.529 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2023-11-08T16:27:48.529 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2023-11-08T16:27:48.529 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2023-11-08T16:27:48.530 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2023-11-08T16:27:48.530 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2023-11-08T16:27:48.530 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:27:48.530 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2023-11-08T16:27:48.530 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2023-11-08T16:27:48.531 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:27:48.531 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2023-11-08T16:27:48.531 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:27:48.531 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:27:48.531 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2023-11-08T16:27:48.532 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:27:48.532 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.532 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2023-11-08T16:27:48.532 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:27:48.532 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2023-11-08T16:27:48.532 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:27:48.533 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:27:48.533 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:27:48.533 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:27:48.533 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2023-11-08T16:27:48.533 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.534 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2023-11-08T16:27:48.534 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:27:48.534 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:27:48.534 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2023-11-08T16:27:48.534 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:27:48.535 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2023-11-08T16:27:48.535 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2023-11-08T16:27:48.535 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:27:48.535 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2023-11-08T16:27:48.535 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2023-11-08T16:27:48.535 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:27:48.536 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.536 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:27:48.536 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:27:48.536 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:27:48.536 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.537 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:27:48.537 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:27:48.537 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:27:48.537 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.537 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2023-11-08T16:27:48.538 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:27:48.538 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.538 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:27:48.538 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2023-11-08T16:27:48.538 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:27:48.539 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2023-11-08T16:27:48.539 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2023-11-08T16:27:48.539 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2023-11-08T16:27:48.539 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2023-11-08T16:27:48.539 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:27:48.539 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:27:48.540 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2023-11-08T16:27:48.540 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2023-11-08T16:27:48.540 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:27:48.540 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2023-11-08T16:27:48.540 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:27:48.541 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:27:48.541 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2023-11-08T16:27:48.541 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2023-11-08T16:27:48.541 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2023-11-08T16:27:48.541 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:27:48.541 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:27:48.542 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:27:48.542 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:27:48.542 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2023-11-08T16:27:48.542 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2023-11-08T16:27:48.542 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2023-11-08T16:27:48.543 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:27:48.543 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2023-11-08T16:27:48.543 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = True 2023-11-08T16:27:48.543 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2023-11-08T16:27:48.543 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = True 2023-11-08T16:27:48.543 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:27:48.544 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:27:48.544 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:27:48.559 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-08T16:27:48.559 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-08T16:27:48.703 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-08T16:27:48.704 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:27:48.704 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-08T16:27:48.754 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:27:48.754 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-08T16:27:48.844 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:27:48.845 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:27:48.845 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-08T16:27:48.845 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T16:27:49.015 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:27:49.015 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:27:49.107 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:27:49.108 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:27:49.109 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-08T16:27:49.109 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-08T16:27:49.271 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:27:49.272 DEBUG:teuthology.orchestra.run.smithi093:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:27:49.372 INFO:tasks.ceph:Setting up mon nodes... 2023-11-08T16:27:49.372 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-08T16:27:49.373 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-08T16:27:49.552 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-08T16:27:49.559 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-08T16:27:49.762 INFO:teuthology.orchestra.run.smithi093.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-08T16:27:49.773 INFO:tasks.ceph:Setting up mds nodes... 2023-11-08T16:27:49.774 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-08T16:27:49.938 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-08T16:27:49.944 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-08T16:27:50.034 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-08T16:27:50.214 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-08T16:27:50.223 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-08T16:27:50.321 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-08T16:27:50.509 INFO:teuthology.orchestra.run.smithi093.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-08T16:27:50.520 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-08T16:27:50.611 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-08T16:27:50.794 INFO:teuthology.orchestra.run.smithi093.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-08T16:27:50.804 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-08T16:27:50.901 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-08T16:27:50.902 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-08T16:27:51.035 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-08T16:27:51.099 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-08T16:27:51.229 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-08T16:27:51.284 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-08T16:27:51.418 INFO:teuthology.orchestra.run.smithi093.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-08T16:27:51.468 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-08T16:27:51.598 INFO:teuthology.orchestra.run.smithi093.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-08T16:27:51.655 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-08T16:27:51.655 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi093.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-08T16:27:51.656 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-08T16:27:51.756 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T16:27:51.757 INFO:tasks.ceph:role: osd.0 2023-11-08T16:27:51.757 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:27:51.757 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-08T16:27:51.861 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T16:27:51.861 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T16:27:51.861 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T16:27:51.862 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T16:27:51.862 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T16:27:51.862 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2023-11-08T16:27:51.862 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T16:27:51.862 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T16:27:51.863 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T16:27:51.863 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T16:27:52.388 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2023-11-08T16:27:52.395 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2023-11-08T16:27:52.396 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-08T16:27:52.583 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-08T16:27:52.689 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-08T16:27:52.781 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T16:27:52.782 INFO:tasks.ceph:role: osd.1 2023-11-08T16:27:52.782 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:27:52.783 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-08T16:27:52.884 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T16:27:52.884 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T16:27:52.884 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T16:27:52.884 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T16:27:52.885 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T16:27:52.885 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2023-11-08T16:27:52.885 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T16:27:52.885 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T16:27:52.885 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T16:27:52.885 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T16:27:53.373 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2023-11-08T16:27:53.380 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2023-11-08T16:27:53.380 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-08T16:27:53.522 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-08T16:27:53.620 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-08T16:27:53.714 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T16:27:53.715 INFO:tasks.ceph:role: osd.2 2023-11-08T16:27:53.715 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:27:53.715 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-08T16:27:53.813 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T16:27:53.814 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T16:27:53.814 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T16:27:53.814 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T16:27:53.814 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T16:27:53.815 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2023-11-08T16:27:53.815 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T16:27:53.815 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T16:27:53.815 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T16:27:53.815 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T16:27:54.337 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2023-11-08T16:27:54.343 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2023-11-08T16:27:54.344 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-08T16:27:54.541 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-08T16:27:54.649 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-08T16:27:54.742 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-08T16:27:54.743 INFO:tasks.ceph:role: osd.3 2023-11-08T16:27:54.743 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:27:54.743 DEBUG:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-08T16:27:54.845 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T16:27:54.845 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T16:27:54.845 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T16:27:54.846 INFO:teuthology.orchestra.run.smithi081.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T16:27:54.846 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T16:27:54.846 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2023-11-08T16:27:54.846 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T16:27:54.847 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T16:27:54.847 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T16:27:54.847 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T16:27:55.305 INFO:teuthology.orchestra.run.smithi081.stdout:Discarding blocks...Done. 2023-11-08T16:27:55.311 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2023-11-08T16:27:55.311 DEBUG:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-08T16:27:55.456 DEBUG:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-08T16:27:55.547 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:27:55.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:55.690+0000 7ff65300e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-08T16:27:55.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:55.691+0000 7ff65300e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-08T16:27:55.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:55.691+0000 7ff65300e500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T16:27:55.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:55.692+0000 7ff65300e500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T16:27:55.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:55.692+0000 7ff65300e500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-08T16:27:55.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:55.698+0000 7ff65300e500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-08T16:27:58.869 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-08T16:27:58.962 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:27:59.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:59.091+0000 7f703d00d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-08T16:27:59.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:59.092+0000 7f703d00d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-08T16:27:59.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:59.092+0000 7f703d00d500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T16:27:59.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:59.092+0000 7f703d00d500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T16:27:59.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:59.092+0000 7f703d00d500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-08T16:27:59.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:27:59.094+0000 7f703d00d500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-08T16:28:02.008 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-08T16:28:02.107 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:28:02.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:02.239+0000 7f4d16e65500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-08T16:28:02.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:02.240+0000 7f4d16e65500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-08T16:28:02.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:02.240+0000 7f4d16e65500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T16:28:02.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:02.240+0000 7f4d16e65500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T16:28:02.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:02.240+0000 7f4d16e65500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-08T16:28:02.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:02.242+0000 7f4d16e65500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-08T16:28:05.183 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-08T16:28:05.283 DEBUG:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:28:05.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:05.418+0000 7f4b487ad500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-08T16:28:05.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:05.419+0000 7f4b487ad500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-08T16:28:05.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:05.419+0000 7f4b487ad500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T16:28:05.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:05.419+0000 7f4b487ad500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T16:28:05.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:05.419+0000 7f4b487ad500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-08T16:28:05.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:05.421+0000 7f4b487ad500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-08T16:28:08.593 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-08T16:28:08.692 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-08T16:28:08.794 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T16:28:08.794 INFO:tasks.ceph:role: osd.4 2023-11-08T16:28:08.794 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:28:08.795 DEBUG:teuthology.orchestra.run.smithi093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-08T16:28:08.898 INFO:teuthology.orchestra.run.smithi093.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T16:28:08.899 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T16:28:08.899 INFO:teuthology.orchestra.run.smithi093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T16:28:08.900 INFO:teuthology.orchestra.run.smithi093.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T16:28:08.900 INFO:teuthology.orchestra.run.smithi093.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T16:28:08.900 INFO:teuthology.orchestra.run.smithi093.stdout: = sunit=0 swidth=0 blks 2023-11-08T16:28:08.900 INFO:teuthology.orchestra.run.smithi093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T16:28:08.900 INFO:teuthology.orchestra.run.smithi093.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T16:28:08.901 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T16:28:08.901 INFO:teuthology.orchestra.run.smithi093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T16:28:09.423 INFO:teuthology.orchestra.run.smithi093.stdout:Discarding blocks...Done. 2023-11-08T16:28:09.431 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi093.front.sepia.ceph.com -o noatime 2023-11-08T16:28:09.431 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-08T16:28:09.613 DEBUG:teuthology.orchestra.run.smithi093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-08T16:28:09.721 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-08T16:28:09.803 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T16:28:09.803 INFO:tasks.ceph:role: osd.5 2023-11-08T16:28:09.803 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:28:09.804 DEBUG:teuthology.orchestra.run.smithi093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-08T16:28:09.892 INFO:teuthology.orchestra.run.smithi093.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T16:28:09.892 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T16:28:09.893 INFO:teuthology.orchestra.run.smithi093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T16:28:09.893 INFO:teuthology.orchestra.run.smithi093.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T16:28:09.893 INFO:teuthology.orchestra.run.smithi093.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T16:28:09.893 INFO:teuthology.orchestra.run.smithi093.stdout: = sunit=0 swidth=0 blks 2023-11-08T16:28:09.893 INFO:teuthology.orchestra.run.smithi093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T16:28:09.894 INFO:teuthology.orchestra.run.smithi093.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T16:28:09.894 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T16:28:09.894 INFO:teuthology.orchestra.run.smithi093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T16:28:10.376 INFO:teuthology.orchestra.run.smithi093.stdout:Discarding blocks...Done. 2023-11-08T16:28:10.382 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi093.front.sepia.ceph.com -o noatime 2023-11-08T16:28:10.382 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-08T16:28:10.513 DEBUG:teuthology.orchestra.run.smithi093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-08T16:28:10.604 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-08T16:28:10.678 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T16:28:10.678 INFO:tasks.ceph:role: osd.6 2023-11-08T16:28:10.678 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:28:10.679 DEBUG:teuthology.orchestra.run.smithi093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-08T16:28:10.762 INFO:teuthology.orchestra.run.smithi093.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T16:28:10.762 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T16:28:10.762 INFO:teuthology.orchestra.run.smithi093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T16:28:10.763 INFO:teuthology.orchestra.run.smithi093.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T16:28:10.763 INFO:teuthology.orchestra.run.smithi093.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T16:28:10.763 INFO:teuthology.orchestra.run.smithi093.stdout: = sunit=0 swidth=0 blks 2023-11-08T16:28:10.763 INFO:teuthology.orchestra.run.smithi093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T16:28:10.764 INFO:teuthology.orchestra.run.smithi093.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T16:28:10.764 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T16:28:10.764 INFO:teuthology.orchestra.run.smithi093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T16:28:11.266 INFO:teuthology.orchestra.run.smithi093.stdout:Discarding blocks...Done. 2023-11-08T16:28:11.273 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi093.front.sepia.ceph.com -o noatime 2023-11-08T16:28:11.274 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-08T16:28:11.450 DEBUG:teuthology.orchestra.run.smithi093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-08T16:28:11.558 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-08T16:28:11.650 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-08T16:28:11.650 INFO:tasks.ceph:role: osd.7 2023-11-08T16:28:11.651 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:28:11.651 DEBUG:teuthology.orchestra.run.smithi093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-08T16:28:11.747 INFO:teuthology.orchestra.run.smithi093.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-08T16:28:11.748 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-08T16:28:11.748 INFO:teuthology.orchestra.run.smithi093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-08T16:28:11.748 INFO:teuthology.orchestra.run.smithi093.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-08T16:28:11.748 INFO:teuthology.orchestra.run.smithi093.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-08T16:28:11.748 INFO:teuthology.orchestra.run.smithi093.stdout: = sunit=0 swidth=0 blks 2023-11-08T16:28:11.749 INFO:teuthology.orchestra.run.smithi093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-08T16:28:11.749 INFO:teuthology.orchestra.run.smithi093.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-08T16:28:11.749 INFO:teuthology.orchestra.run.smithi093.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-08T16:28:11.749 INFO:teuthology.orchestra.run.smithi093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-08T16:28:12.201 INFO:teuthology.orchestra.run.smithi093.stdout:Discarding blocks...Done. 2023-11-08T16:28:12.206 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi093.front.sepia.ceph.com -o noatime 2023-11-08T16:28:12.207 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-08T16:28:12.351 DEBUG:teuthology.orchestra.run.smithi093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-08T16:28:12.449 DEBUG:teuthology.orchestra.run.smithi093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:28:12.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:12.601+0000 7f9c936f8500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-08T16:28:12.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:12.607+0000 7f9c936f8500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-08T16:28:12.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:12.607+0000 7f9c936f8500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T16:28:12.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:12.607+0000 7f9c936f8500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T16:28:12.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:12.607+0000 7f9c936f8500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-08T16:28:12.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:12.615+0000 7f9c936f8500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-08T16:28:15.613 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:28:15.622 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:28:15.803 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-08T16:28:15.946 DEBUG:teuthology.orchestra.run.smithi093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:28:16.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:16.083+0000 7fb44bbfb500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-08T16:28:16.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:16.084+0000 7fb44bbfb500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-08T16:28:16.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:16.084+0000 7fb44bbfb500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T16:28:16.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:16.084+0000 7fb44bbfb500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T16:28:16.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:16.084+0000 7fb44bbfb500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-08T16:28:16.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:16.086+0000 7fb44bbfb500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-08T16:28:19.250 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-08T16:28:19.345 DEBUG:teuthology.orchestra.run.smithi093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:28:19.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:19.483+0000 7faad9215500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-08T16:28:19.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:19.484+0000 7faad9215500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-08T16:28:19.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:19.484+0000 7faad9215500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T16:28:19.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:19.484+0000 7faad9215500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T16:28:19.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:19.484+0000 7faad9215500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-08T16:28:19.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:19.486+0000 7faad9215500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-08T16:28:22.656 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-08T16:28:22.756 DEBUG:teuthology.orchestra.run.smithi093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:28:22.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:22.896+0000 7efc682c9500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-08T16:28:22.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:22.897+0000 7efc682c9500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-08T16:28:22.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:22.897+0000 7efc682c9500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T16:28:22.899 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:22.897+0000 7efc682c9500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T16:28:22.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:22.897+0000 7efc682c9500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-08T16:28:22.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:22.899+0000 7efc682c9500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-08T16:28:25.809 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-08T16:28:25.911 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-08T16:28:25.911 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:25.912 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-08T16:28:26.004 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:26.004 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-08T16:28:26.099 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:26.100 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-08T16:28:26.242 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:26.243 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-08T16:28:26.337 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:26.338 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-08T16:28:26.429 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:26.429 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-08T16:28:26.527 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:26.528 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-08T16:28:26.626 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:26.626 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-08T16:28:26.728 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:26.729 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-08T16:28:26.826 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:26.826 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-08T16:28:26.922 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:26.923 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-08T16:28:27.021 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:27.022 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-08T16:28:27.114 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:27.114 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-08T16:28:27.257 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:27.258 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-08T16:28:27.352 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:27.353 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-08T16:28:27.412 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:27.413 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-08T16:28:27.465 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:27.465 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-08T16:28:27.524 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:27.524 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-08T16:28:27.574 INFO:tasks.ceph:Adding keys to all mons... 2023-11-08T16:28:27.575 DEBUG:teuthology.orchestra.run.smithi081:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-08T16:28:27.584 DEBUG:teuthology.orchestra.run.smithi093:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-08T16:28:27.673 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.y] 2023-11-08T16:28:27.673 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAFt0tlNpDaIBAAtgLFNQitjQnLny6SRY7hHw== 2023-11-08T16:28:27.674 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:28:27.674 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAFt0tlL0bmNxAAlrzRL4depZdd+lBB5HN09g== 2023-11-08T16:28:27.674 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:28:27.675 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAGt0tlwo23DBAAl1sBjOJMIQRb1fH3o9XoiQ== 2023-11-08T16:28:27.675 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.0] 2023-11-08T16:28:27.675 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQALt0tl3kkwKRAAgDwD1eQYk7qjvV+/gyol/w== 2023-11-08T16:28:27.675 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.1] 2023-11-08T16:28:27.675 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAPt0tlpIh+BRAAUukXDBbd9Vev1TzIrv9sRg== 2023-11-08T16:28:27.675 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.2] 2023-11-08T16:28:27.676 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQASt0tlmzJfDhAASO7zeg/9ddmLsoOTe+MIug== 2023-11-08T16:28:27.676 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.3] 2023-11-08T16:28:27.676 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAVt0tlGoQSGRAAQsrZhrx4kpYUm3KEb6ZUGQ== 2023-11-08T16:28:27.676 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.x] 2023-11-08T16:28:27.676 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAFt0tlhZFjLRAAANBB8roHZH4z4pE4C2Aq/Q== 2023-11-08T16:28:27.677 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:28:27.677 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAGt0tl2cFWHhAAvwSSL9e5996l9MBwNQzs6A== 2023-11-08T16:28:27.677 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:28:27.677 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAGt0tl5DdQLxAA1aU7e9epGtKnZ6hkHZTnsQ== 2023-11-08T16:28:27.677 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.4] 2023-11-08T16:28:27.678 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAct0tlOS/kIxAAxC2I47B2WeGTFZxvdnmkMg== 2023-11-08T16:28:27.678 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.5] 2023-11-08T16:28:27.678 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAgt0tlnAgEBRAADIRSyu/bxiykCc2uiIt0kA== 2023-11-08T16:28:27.678 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.6] 2023-11-08T16:28:27.678 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAjt0tlrdfZHBAAHGdvBBzVwXMmy8GmsZLNbg== 2023-11-08T16:28:27.679 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.7] 2023-11-08T16:28:27.679 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAmt0tl4nN9NRAAzV9abuBfhybYCsSFnqHEYA== 2023-11-08T16:28:27.679 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2023-11-08T16:28:27.679 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ== 2023-11-08T16:28:27.679 INFO:teuthology.orchestra.run.smithi081.stdout:[client.2] 2023-11-08T16:28:27.680 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAHt0tlbQmgDRAAL/b9NwCBVT+uGxuv3VAWRw== 2023-11-08T16:28:27.680 INFO:teuthology.orchestra.run.smithi081.stdout:[client.1] 2023-11-08T16:28:27.680 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAHt0tlgn/qGBAA+WHDN1CJ1v62EbX1GEFBWA== 2023-11-08T16:28:27.680 INFO:teuthology.orchestra.run.smithi081.stdout:[client.3] 2023-11-08T16:28:27.680 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAHt0tlah+ZIxAAg53qCmUjPtG+l++2Byn1NA== 2023-11-08T16:28:27.712 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr.y] 2023-11-08T16:28:27.713 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAFt0tlNpDaIBAAtgLFNQitjQnLny6SRY7hHw== 2023-11-08T16:28:27.713 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:28:27.713 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAFt0tlL0bmNxAAlrzRL4depZdd+lBB5HN09g== 2023-11-08T16:28:27.713 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:28:27.713 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAGt0tlwo23DBAAl1sBjOJMIQRb1fH3o9XoiQ== 2023-11-08T16:28:27.714 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.0] 2023-11-08T16:28:27.714 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQALt0tl3kkwKRAAgDwD1eQYk7qjvV+/gyol/w== 2023-11-08T16:28:27.714 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.1] 2023-11-08T16:28:27.714 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAPt0tlpIh+BRAAUukXDBbd9Vev1TzIrv9sRg== 2023-11-08T16:28:27.714 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.2] 2023-11-08T16:28:27.715 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQASt0tlmzJfDhAASO7zeg/9ddmLsoOTe+MIug== 2023-11-08T16:28:27.715 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.3] 2023-11-08T16:28:27.715 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAVt0tlGoQSGRAAQsrZhrx4kpYUm3KEb6ZUGQ== 2023-11-08T16:28:27.715 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr.x] 2023-11-08T16:28:27.715 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAFt0tlhZFjLRAAANBB8roHZH4z4pE4C2Aq/Q== 2023-11-08T16:28:27.716 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:28:27.716 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAGt0tl2cFWHhAAvwSSL9e5996l9MBwNQzs6A== 2023-11-08T16:28:27.716 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:28:27.716 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAGt0tl5DdQLxAA1aU7e9epGtKnZ6hkHZTnsQ== 2023-11-08T16:28:27.716 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.4] 2023-11-08T16:28:27.717 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAct0tlOS/kIxAAxC2I47B2WeGTFZxvdnmkMg== 2023-11-08T16:28:27.717 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.5] 2023-11-08T16:28:27.717 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAgt0tlnAgEBRAADIRSyu/bxiykCc2uiIt0kA== 2023-11-08T16:28:27.717 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.6] 2023-11-08T16:28:27.717 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAjt0tlrdfZHBAAHGdvBBzVwXMmy8GmsZLNbg== 2023-11-08T16:28:27.718 INFO:teuthology.orchestra.run.smithi093.stdout:[osd.7] 2023-11-08T16:28:27.718 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAmt0tl4nN9NRAAzV9abuBfhybYCsSFnqHEYA== 2023-11-08T16:28:27.718 INFO:teuthology.orchestra.run.smithi093.stdout:[client.0] 2023-11-08T16:28:27.718 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ== 2023-11-08T16:28:27.718 INFO:teuthology.orchestra.run.smithi093.stdout:[client.2] 2023-11-08T16:28:27.719 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAHt0tlbQmgDRAAL/b9NwCBVT+uGxuv3VAWRw== 2023-11-08T16:28:27.719 INFO:teuthology.orchestra.run.smithi093.stdout:[client.1] 2023-11-08T16:28:27.719 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAHt0tlgn/qGBAA+WHDN1CJ1v62EbX1GEFBWA== 2023-11-08T16:28:27.719 INFO:teuthology.orchestra.run.smithi093.stdout:[client.3] 2023-11-08T16:28:27.719 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQAHt0tlah+ZIxAAg53qCmUjPtG+l++2Byn1NA== 2023-11-08T16:28:27.720 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T16:28:27.730 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T16:28:27.878 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T16:28:27.884 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T16:28:28.035 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T16:28:28.040 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T16:28:28.186 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:28.192 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:28.336 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:28.341 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:28.486 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:28.492 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:28.635 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:28.643 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:28.779 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T16:28:28.784 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-08T16:28:28.921 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T16:28:28.927 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T16:28:29.073 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T16:28:29.078 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-08T16:28:29.219 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:29.227 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:29.366 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:29.372 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:29.514 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:29.519 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:29.660 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:29.665 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-08T16:28:29.808 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T16:28:29.813 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T16:28:29.953 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T16:28:29.961 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T16:28:30.098 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T16:28:30.103 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T16:28:30.247 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T16:28:30.253 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-08T16:28:30.396 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-08T16:28:30.397 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-08T16:28:30.494 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T16:28:30.649 DEBUG:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-08T16:28:30.740 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-08T16:28:30.844 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T16:28:31.006 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-08T16:28:31.100 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-08T16:28:31.198 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-08T16:28:31.355 DEBUG:teuthology.orchestra.run.smithi093:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-08T16:28:31.437 DEBUG:teuthology.orchestra.run.smithi081:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:28:31.446 DEBUG:teuthology.orchestra.run.smithi093:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-08T16:28:31.501 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-08T16:28:31.568 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-08T16:28:31.569 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-08T16:28:31.579 INFO:tasks.ceph.mon.a:Started 2023-11-08T16:28:31.580 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-08T16:28:31.580 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-08T16:28:31.590 INFO:tasks.ceph.mon.b:Started 2023-11-08T16:28:31.590 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-08T16:28:31.590 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-08T16:28:31.599 INFO:tasks.ceph.mon.c:Started 2023-11-08T16:28:31.600 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-08T16:28:31.600 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-08T16:28:31.600 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-08T16:28:31.606 INFO:tasks.ceph.mgr.y:Started 2023-11-08T16:28:31.607 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-08T16:28:31.607 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-08T16:28:31.614 INFO:tasks.ceph.mgr.x:Started 2023-11-08T16:28:31.617 INFO:tasks.ceph:Setting crush tunables to default 2023-11-08T16:28:31.617 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd crush tunables default 2023-11-08T16:28:31.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.792+0000 7f59f5916640 1 Processor -- start 2023-11-08T16:28:31.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.792+0000 7f59f5916640 1 -- start start 2023-11-08T16:28:31.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.793+0000 7f59f5916640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:31.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.793+0000 7f59f5916640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f00b22c0 con 0x7f59f0160a10 2023-11-08T16:28:31.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.793+0000 7f59f5916640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f00b2e20 con 0x7f59f00d39e0 2023-11-08T16:28:31.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.793+0000 7f59f5916640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59f0160df0 con 0x7f59f00cffe0 2023-11-08T16:28:31.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.794+0000 7f59e67fc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:31.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.794+0000 7f59eeffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f59f0160a10 0x7f59f00cf8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56386/0 (socket says 172.21.15.81:56386) 2023-11-08T16:28:31.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.794+0000 7f59eeffd640 1 -- 172.21.15.81:0/2757366824 learned_addr learned my addr 172.21.15.81:0/2757366824 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:28:31.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.794+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 msgr2=0x7f59f00b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T16:28:31.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.794+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 msgr2=0x7f59f00b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:31.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.794+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:31.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.794+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-08T16:28:31.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.995+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:31.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.996+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 msgr2=0x7f59f00b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T16:28:31.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.996+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 msgr2=0x7f59f00b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:31.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.996+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:31.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:31.996+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-08T16:28:32.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:32.397+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:32.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:32.398+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 msgr2=0x7f59f00b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T16:28:32.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:32.398+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 msgr2=0x7f59f00b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:32.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:32.398+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:32.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:32.398+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-08T16:28:33.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:33.200+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:33.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:33.200+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 msgr2=0x7f59f00b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T16:28:33.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:33.200+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 msgr2=0x7f59f00b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:33.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:33.200+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:33.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:33.200+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-08T16:28:34.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.793+0000 7f59ed7fa640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 msgr2=0x7f59f00b42c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T16:28:34.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.793+0000 7f59ed7fa640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59f00cffe0 0x7f59f00b42c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:34.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.793+0000 7f59ed7fa640 1 -- 172.21.15.81:0/2757366824 >> v1:172.21.15.93:6789/0 conn(0x7f59f00d39e0 legacy=0x7f59f00cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:34.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.793+0000 7f59ed7fa640 1 -- 172.21.15.81:0/2757366824 >> v1:172.21.15.81:6789/0 conn(0x7f59f0160a10 legacy=0x7f59f00cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:34.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.793+0000 7f59ed7fa640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:34.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.794+0000 7f59ed7fa640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:34.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.794+0000 7f59ed7fa640 1 -- 172.21.15.81:0/2757366824 --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d0008d30 con 0x7f59d0006200 2023-11-08T16:28:34.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.794+0000 7f59ed7fa640 1 -- 172.21.15.81:0/2757366824 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d0008ea0 con 0x7f59d0003740 2023-11-08T16:28:34.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.794+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:34.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.794+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 msgr2=0x7f59d00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T16:28:34.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.794+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 msgr2=0x7f59d00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:34.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.794+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:34.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.794+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:34.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.795+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-08T16:28:34.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.795+0000 7f59eeffd640 1 -- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 msgr2=0x7f59d0005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-08T16:28:34.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.795+0000 7f59eeffd640 1 -- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 msgr2=0x7f59d0005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:34.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.795+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:34.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.795+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-08T16:28:34.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.996+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:34.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.996+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:34.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.997+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 msgr2=0x7f59d00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T16:28:34.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.997+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 msgr2=0x7f59d00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:35.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.997+0000 7f59eeffd640 1 -- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 msgr2=0x7f59d0005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-08T16:28:35.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.997+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:35.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.997+0000 7f59eeffd640 1 -- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 msgr2=0x7f59d0005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:35.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.997+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:35.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.997+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-08T16:28:35.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:34.997+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-11-08T16:28:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:35.398+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:35.398+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:35.398+0000 7f59eeffd640 1 -- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 msgr2=0x7f59d0005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-08T16:28:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:35.398+0000 7f59eeffd640 1 -- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 msgr2=0x7f59d0005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:35.398+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:35.398+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-08T16:28:35.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:35.399+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 msgr2=0x7f59d00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T16:28:35.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:35.399+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 msgr2=0x7f59d00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:35.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:35.399+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:35.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:35.399+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-11-08T16:28:36.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:36.200+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:36.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:36.200+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:36.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:36.201+0000 7f59eeffd640 1 -- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 msgr2=0x7f59d0005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-08T16:28:36.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:36.201+0000 7f59eeffd640 1 -- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 msgr2=0x7f59d0005b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:36.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:36.201+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:36.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:36.201+0000 7f59eeffd640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-08T16:28:36.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:36.201+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 msgr2=0x7f59d00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-08T16:28:36.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:36.201+0000 7f59e67fc640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 msgr2=0x7f59d00086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-08T16:28:36.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:36.201+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-08T16:28:36.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:36.201+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-11-08T16:28:37.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.794+0000 7f59ed7fa640 1 -- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 msgr2=0x7f59d0005b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T16:28:37.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.794+0000 7f59ed7fa640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0003740 0x7f59d0005b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:37.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.794+0000 7f59ed7fa640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 msgr2=0x7f59d00086a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T16:28:37.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.794+0000 7f59ed7fa640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.81:3300/0 conn(0x7f59d0006200 0x7f59d00086a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:37.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.794+0000 7f59ef7fe640 1 -- 172.21.15.81:0/2757366824 reap_dead start 2023-11-08T16:28:37.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.794+0000 7f59ed7fa640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59d0003b80 0x7f59d0006690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:37.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.794+0000 7f59ed7fa640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0006bd0 0x7f59d0007080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:37.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.794+0000 7f59ed7fa640 1 -- 172.21.15.81:0/2757366824 --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59f00d0fc0 con 0x7f59f00d39e0 2023-11-08T16:28:37.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.794+0000 7f59ed7fa640 1 -- 172.21.15.81:0/2757366824 --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59d0008d30 con 0x7f59d0003b80 2023-11-08T16:28:37.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.794+0000 7f59ed7fa640 1 -- 172.21.15.81:0/2757366824 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d0008ea0 con 0x7f59d0006bd0 2023-11-08T16:28:37.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.795+0000 7f59e67fc640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59d0003b80 0x7f59d0006690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:37.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.795+0000 7f59ef7fe640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0006bd0 0x7f59d0007080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:37.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.795+0000 7f59ee7fc640 1 -- 172.21.15.81:0/2757366824 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979160803 0 0) 0x7f59f00d0fc0 con 0x7f59f00d39e0 2023-11-08T16:28:37.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.795+0000 7f59ef7fe640 1 -- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59d0003b80 msgr2=0x7f59d0006690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:37.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.795+0000 7f59ef7fe640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59d0003b80 0x7f59d0006690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:37.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.795+0000 7f59ef7fe640 1 -- 172.21.15.81:0/2757366824 >> v1:172.21.15.93:6789/0 conn(0x7f59f00d39e0 legacy=0x7f59d000b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:37.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.795+0000 7f59ef7fe640 1 -- 172.21.15.81:0/2757366824 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d00087c0 con 0x7f59d0006bd0 2023-11-08T16:28:37.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.796+0000 7f59ef7fe640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0006bd0 0x7f59d0007080 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59d0003630 tx=0x7f59e802b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16870f359fd31eb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:37.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.797+0000 7f59ee7fc640 1 -- 172.21.15.81:0/2757366824 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8005d40 con 0x7f59d0006bd0 2023-11-08T16:28:37.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.797+0000 7f59ee7fc640 1 -- 172.21.15.81:0/2757366824 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59e802e070 con 0x7f59d0006bd0 2023-11-08T16:28:37.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.797+0000 7f59ee7fc640 1 -- 172.21.15.81:0/2757366824 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59e8005ea0 con 0x7f59d0006bd0 2023-11-08T16:28:37.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.798+0000 7f59f5916640 1 -- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0006bd0 msgr2=0x7f59d0007080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:37.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.798+0000 7f59f5916640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0006bd0 0x7f59d0007080 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f59d0003630 tx=0x7f59e802b5d0 comp rx=0 tx=0).stop 2023-11-08T16:28:37.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.798+0000 7f59f5916640 1 -- 172.21.15.81:0/2757366824 shutdown_connections 2023-11-08T16:28:37.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.798+0000 7f59f5916640 1 --2- 172.21.15.81:0/2757366824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d0006bd0 0x7f59d0007080 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:37.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.798+0000 7f59f5916640 1 --2- 172.21.15.81:0/2757366824 >> v2:172.21.15.93:3300/0 conn(0x7f59d0003b80 0x7f59d0006690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:37.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.798+0000 7f59f5916640 1 -- 172.21.15.81:0/2757366824 >> 172.21.15.81:0/2757366824 conn(0x7f59f00d2ec0 msgr2=0x7f59f00d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:37.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.798+0000 7f59f5916640 1 -- 172.21.15.81:0/2757366824 shutdown_connections 2023-11-08T16:28:37.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.798+0000 7f59f5916640 1 -- 172.21.15.81:0/2757366824 wait complete. 2023-11-08T16:28:37.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.801+0000 7f59f5916640 1 Processor -- start 2023-11-08T16:28:37.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.801+0000 7f59f5916640 1 -- start start 2023-11-08T16:28:37.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.802+0000 7f59f5916640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d0003b80 0x7f59f0158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:37.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.802+0000 7f59f5916640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d0006bd0 0x7f59f015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:37.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.802+0000 7f59f5916640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59f00d39e0 0x7f59f015e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:37.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.802+0000 7f59f5916640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f00c9f90 con 0x7f59d0006bd0 2023-11-08T16:28:37.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.802+0000 7f59f5916640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f00ca200 con 0x7f59d0003b80 2023-11-08T16:28:37.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.802+0000 7f59f5916640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f015a3c0 con 0x7f59f00d39e0 2023-11-08T16:28:37.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.803+0000 7f59e67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d0006bd0 0x7f59f015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:37.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.803+0000 7f59e67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d0006bd0 0x7f59f015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57520/0 (socket says 172.21.15.81:57520) 2023-11-08T16:28:37.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.803+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d0003b80 0x7f59f0158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:37.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.803+0000 7f59e67fc640 1 -- 172.21.15.81:0/315158117 learned_addr learned my addr 172.21.15.81:0/315158117 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:37.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.803+0000 7f59ef7fe640 1 --2- 172.21.15.81:0/315158117 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59f00d39e0 0x7f59f015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:37.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.803+0000 7f59e67fc640 1 -- 172.21.15.81:0/315158117 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59f00d39e0 msgr2=0x7f59f015e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:37.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.803+0000 7f59e67fc640 1 --2- 172.21.15.81:0/315158117 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59f00d39e0 0x7f59f015e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:37.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.803+0000 7f59e67fc640 1 -- 172.21.15.81:0/315158117 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d0003b80 msgr2=0x7f59f0158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:37.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.803+0000 7f59e67fc640 1 --2- 172.21.15.81:0/315158117 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d0003b80 0x7f59f0158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:37.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.803+0000 7f59e67fc640 1 -- 172.21.15.81:0/315158117 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f00b2e20 con 0x7f59d0006bd0 2023-11-08T16:28:37.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.804+0000 7f59e67fc640 1 --2- 172.21.15.81:0/315158117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d0006bd0 0x7f59f015de00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59d8006e60 tx=0x7f59d8018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:37.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.804+0000 7f59ecff9640 1 -- 172.21.15.81:0/315158117 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d801e050 con 0x7f59d0006bd0 2023-11-08T16:28:37.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.804+0000 7f59ecff9640 1 -- 172.21.15.81:0/315158117 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d801f040 con 0x7f59d0006bd0 2023-11-08T16:28:37.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.804+0000 7f59f5916640 1 -- 172.21.15.81:0/315158117 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59f015a6a0 con 0x7f59d0006bd0 2023-11-08T16:28:37.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.804+0000 7f59ecff9640 1 -- 172.21.15.81:0/315158117 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d8004030 con 0x7f59d0006bd0 2023-11-08T16:28:37.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.804+0000 7f59f5916640 1 -- 172.21.15.81:0/315158117 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59f0159600 con 0x7f59d0006bd0 2023-11-08T16:28:37.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.805+0000 7f59ecff9640 1 -- 172.21.15.81:0/315158117 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f59d8004190 con 0x7f59d0006bd0 2023-11-08T16:28:37.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.805+0000 7f59ecff9640 1 -- 172.21.15.81:0/315158117 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f59d80247a0 con 0x7f59d0006bd0 2023-11-08T16:28:37.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.806+0000 7f59f5916640 1 -- 172.21.15.81:0/315158117 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59f015abc0 con 0x7f59d0006bd0 2023-11-08T16:28:37.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.809+0000 7f59ecff9640 1 -- 172.21.15.81:0/315158117 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f59f015abc0 con 0x7f59d0006bd0 2023-11-08T16:28:37.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.881+0000 7f59f5916640 1 -- 172.21.15.81:0/315158117 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f59f004ece0 con 0x7f59d0006bd0 2023-11-08T16:28:37.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.883+0000 7f59ecff9640 1 -- 172.21.15.81:0/315158117 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f59d8033060 con 0x7f59d0006bd0 2023-11-08T16:28:37.885 INFO:teuthology.orchestra.run.smithi081.stderr:adjusted tunables profile to default 2023-11-08T16:28:37.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.884+0000 7f59f5916640 1 -- 172.21.15.81:0/315158117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d0006bd0 msgr2=0x7f59f015de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:37.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.884+0000 7f59f5916640 1 --2- 172.21.15.81:0/315158117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d0006bd0 0x7f59f015de00 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f59d8006e60 tx=0x7f59d8018f40 comp rx=0 tx=0).stop 2023-11-08T16:28:37.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.885+0000 7f59f5916640 1 -- 172.21.15.81:0/315158117 shutdown_connections 2023-11-08T16:28:37.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.885+0000 7f59f5916640 1 --2- 172.21.15.81:0/315158117 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59f00d39e0 0x7f59f015e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:37.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.885+0000 7f59f5916640 1 --2- 172.21.15.81:0/315158117 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d0006bd0 0x7f59f015de00 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:37.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.885+0000 7f59f5916640 1 --2- 172.21.15.81:0/315158117 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d0003b80 0x7f59f0158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:37.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.885+0000 7f59f5916640 1 -- 172.21.15.81:0/315158117 >> 172.21.15.81:0/315158117 conn(0x7f59f00d2ec0 msgr2=0x7f59f015c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:37.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.885+0000 7f59f5916640 1 -- 172.21.15.81:0/315158117 shutdown_connections 2023-11-08T16:28:37.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:37.885+0000 7f59f5916640 1 -- 172.21.15.81:0/315158117 wait complete. 2023-11-08T16:28:37.903 INFO:tasks.ceph:check_enable_crimson: False 2023-11-08T16:28:37.904 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-08T16:28:37.904 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:37.905 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-08T16:28:37.982 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:37.983 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-08T16:28:38.104 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:38.104 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-08T16:28:38.238 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:28:38.239 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-08T16:28:38.320 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:38.320 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-08T16:28:38.339 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:38.337+0000 7f478b837040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-08T16:28:38.346 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:38.345+0000 7fab3f21c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-08T16:28:38.403 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:38.401+0000 7f478b837040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-08T16:28:38.406 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:38.405+0000 7fab3f21c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-08T16:28:38.408 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:38.409 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-08T16:28:38.534 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:38.535 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-08T16:28:38.664 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:28:38.665 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-08T16:28:38.745 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new 6cdeacf5-8ee5-4738-8d05-2576d080248f 0 2023-11-08T16:28:38.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.976+0000 7ff07b0a9640 1 Processor -- start 2023-11-08T16:28:38.979 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.977+0000 7ff07b0a9640 1 -- start start 2023-11-08T16:28:38.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.978+0000 7ff07b0a9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff07415d240 0x7ff0740cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:38.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.978+0000 7ff07b0a9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0740d1910 0x7ff0740ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:38.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.978+0000 7ff07b0a9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0740d39e0 con 0x7ff0740d14a0 2023-11-08T16:28:38.981 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.978+0000 7ff07b0a9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0740cf020 con 0x7ff0740d1910 2023-11-08T16:28:38.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.978+0000 7ff07b0a9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0740d3c10 con 0x7ff07415d240 2023-11-08T16:28:38.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.978+0000 7ff078e45640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff07415d240 0x7ff0740cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:38.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.978+0000 7ff078e45640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff07415d240 0x7ff0740cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54746/0 (socket says 172.21.15.93:54746) 2023-11-08T16:28:38.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.978+0000 7ff078e45640 1 -- 172.21.15.93:0/1616269550 learned_addr learned my addr 172.21.15.93:0/1616269550 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:38.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.978+0000 7ff073fff640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7ff0740d14a0 0x7ff0740b34a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.93:55710/0 (socket says 172.21.15.93:55710) 2023-11-08T16:28:38.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.978+0000 7ff079646640 1 --2- 172.21.15.93:0/1616269550 >> v2:172.21.15.81:3300/0 conn(0x7ff0740d1910 0x7ff0740ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:38.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.979+0000 7ff078e45640 1 -- 172.21.15.93:0/1616269550 >> v2:172.21.15.81:3300/0 conn(0x7ff0740d1910 msgr2=0x7ff0740ceae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:38.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.979+0000 7ff078e45640 1 --2- 172.21.15.93:0/1616269550 >> v2:172.21.15.81:3300/0 conn(0x7ff0740d1910 0x7ff0740ceae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:38.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.979+0000 7ff078e45640 1 -- 172.21.15.93:0/1616269550 >> v1:172.21.15.93:6789/0 conn(0x7ff0740d14a0 legacy=0x7ff0740b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:38.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.979+0000 7ff078e45640 1 -- 172.21.15.93:0/1616269550 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0740b29c0 con 0x7ff07415d240 2023-11-08T16:28:38.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.980+0000 7ff078e45640 1 --2- 172.21.15.93:0/1616269550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff07415d240 0x7ff0740cfed0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff07408d450 tx=0x7ff06802b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fea6f8998d7e81dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:38.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.981+0000 7ff0737fe640 1 -- 172.21.15.93:0/1616269550 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068030850 con 0x7ff07415d240 2023-11-08T16:28:38.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.981+0000 7ff0737fe640 1 -- 172.21.15.93:0/1616269550 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0680309b0 con 0x7ff07415d240 2023-11-08T16:28:38.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.981+0000 7ff0737fe640 1 -- 172.21.15.93:0/1616269550 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068030d40 con 0x7ff07415d240 2023-11-08T16:28:38.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.981+0000 7ff07b0a9640 1 -- 172.21.15.93:0/1616269550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff07415d240 msgr2=0x7ff0740cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:38.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.981+0000 7ff07b0a9640 1 --2- 172.21.15.93:0/1616269550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff07415d240 0x7ff0740cfed0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff07408d450 tx=0x7ff06802b990 comp rx=0 tx=0).stop 2023-11-08T16:28:38.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.982+0000 7ff07b0a9640 1 -- 172.21.15.93:0/1616269550 shutdown_connections 2023-11-08T16:28:38.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.982+0000 7ff07b0a9640 1 --2- 172.21.15.93:0/1616269550 >> v2:172.21.15.81:3300/0 conn(0x7ff0740d1910 0x7ff0740ceae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:38.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.982+0000 7ff07b0a9640 1 --2- 172.21.15.93:0/1616269550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff07415d240 0x7ff0740cfed0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:38.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.982+0000 7ff07b0a9640 1 -- 172.21.15.93:0/1616269550 >> 172.21.15.93:0/1616269550 conn(0x7ff0740d2ec0 msgr2=0x7ff0740d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:38.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.982+0000 7ff07b0a9640 1 -- 172.21.15.93:0/1616269550 shutdown_connections 2023-11-08T16:28:38.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.982+0000 7ff07b0a9640 1 -- 172.21.15.93:0/1616269550 wait complete. 2023-11-08T16:28:38.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.984+0000 7ff07b0a9640 1 Processor -- start 2023-11-08T16:28:38.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.984+0000 7ff07b0a9640 1 -- start start 2023-11-08T16:28:38.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.985+0000 7ff07b0a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0740d14a0 0x7ff0741f0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:38.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.985+0000 7ff07b0a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0740d1910 0x7ff0740caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:38.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.985+0000 7ff07b0a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff07415d240 0x7ff0740b7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:38.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.985+0000 7ff07b0a9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740cc1a0 con 0x7ff0740d1910 2023-11-08T16:28:38.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.985+0000 7ff07b0a9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740cb590 con 0x7ff0740d14a0 2023-11-08T16:28:38.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.985+0000 7ff07b0a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0740cb870 con 0x7ff07415d240 2023-11-08T16:28:38.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.985+0000 7ff078e45640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0740d14a0 0x7ff0741f0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:38.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.985+0000 7ff079646640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff07415d240 0x7ff0740b7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:38.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.985+0000 7ff078e45640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0740d14a0 0x7ff0741f0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54756/0 (socket says 172.21.15.93:54756) 2023-11-08T16:28:38.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.985+0000 7ff078e45640 1 -- 172.21.15.93:0/3324604123 learned_addr learned my addr 172.21.15.93:0/3324604123 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:38.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff073fff640 1 --2- 172.21.15.93:0/3324604123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0740d1910 0x7ff0740caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:38.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff079646640 1 -- 172.21.15.93:0/3324604123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0740d14a0 msgr2=0x7ff0741f0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:38.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff079646640 1 --2- 172.21.15.93:0/3324604123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0740d14a0 0x7ff0741f0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:38.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff079646640 1 -- 172.21.15.93:0/3324604123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0740d1910 msgr2=0x7ff0740caad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:38.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff079646640 1 --2- 172.21.15.93:0/3324604123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0740d1910 0x7ff0740caad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:38.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff079646640 1 -- 172.21.15.93:0/3324604123 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff068009470 con 0x7ff07415d240 2023-11-08T16:28:38.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff078e45640 1 --2- 172.21.15.93:0/3324604123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0740d14a0 0x7ff0741f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:28:38.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff079646640 1 --2- 172.21.15.93:0/3324604123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff07415d240 0x7ff0740b7e00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff06c007d90 tx=0x7ff06c00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:38.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff0717fa640 1 -- 172.21.15.93:0/3324604123 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c018070 con 0x7ff07415d240 2023-11-08T16:28:38.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff0717fa640 1 -- 172.21.15.93:0/3324604123 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c00edd0 con 0x7ff07415d240 2023-11-08T16:28:38.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff0717fa640 1 -- 172.21.15.93:0/3324604123 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c003bb0 con 0x7ff07415d240 2023-11-08T16:28:38.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff07b0a9640 1 -- 172.21.15.93:0/3324604123 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0740cbb50 con 0x7ff07415d240 2023-11-08T16:28:38.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.986+0000 7ff07b0a9640 1 -- 172.21.15.93:0/3324604123 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0741f2e60 con 0x7ff07415d240 2023-11-08T16:28:38.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.987+0000 7ff0717fa640 1 -- 172.21.15.93:0/3324604123 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff06c014720 con 0x7ff07415d240 2023-11-08T16:28:38.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.987+0000 7ff0717fa640 1 -- 172.21.15.93:0/3324604123 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff06c01d8b0 con 0x7ff07415d240 2023-11-08T16:28:38.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.988+0000 7ff07b0a9640 1 -- 172.21.15.93:0/3324604123 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07404ece0 con 0x7ff07415d240 2023-11-08T16:28:38.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:38.990+0000 7ff0717fa640 1 -- 172.21.15.93:0/3324604123 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ff06c01d410 con 0x7ff07415d240 2023-11-08T16:28:39.017 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:39.014+0000 7fab3f21c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-08T16:28:39.041 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:39.039+0000 7f478b837040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-08T16:28:39.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.068+0000 7ff07b0a9640 1 -- 172.21.15.93:0/3324604123 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6cdeacf5-8ee5-4738-8d05-2576d080248f", "id": 0} v 0) v1 -- 0x7ff07406be80 con 0x7ff07415d240 2023-11-08T16:28:39.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.074+0000 7ff0717fa640 1 -- 172.21.15.93:0/3324604123 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6cdeacf5-8ee5-4738-8d05-2576d080248f", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff06c02c0b0 con 0x7ff07415d240 2023-11-08T16:28:39.077 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:39.075+0000 7fab3f21c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-08T16:28:39.078 INFO:teuthology.orchestra.run.smithi093.stdout:0 2023-11-08T16:28:39.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.076+0000 7ff07b0a9640 1 -- 172.21.15.93:0/3324604123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff07415d240 msgr2=0x7ff0740b7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.076+0000 7ff07b0a9640 1 --2- 172.21.15.93:0/3324604123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff07415d240 0x7ff0740b7e00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff06c007d90 tx=0x7ff06c00e4b0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.076+0000 7ff07b0a9640 1 -- 172.21.15.93:0/3324604123 shutdown_connections 2023-11-08T16:28:39.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.076+0000 7ff07b0a9640 1 --2- 172.21.15.93:0/3324604123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff07415d240 0x7ff0740b7e00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.076+0000 7ff07b0a9640 1 --2- 172.21.15.93:0/3324604123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0740d1910 0x7ff0740caad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.076+0000 7ff07b0a9640 1 --2- 172.21.15.93:0/3324604123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0740d14a0 0x7ff0741f0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.076+0000 7ff07b0a9640 1 -- 172.21.15.93:0/3324604123 >> 172.21.15.93:0/3324604123 conn(0x7ff0740d2ec0 msgr2=0x7ff0740b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:39.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.076+0000 7ff07b0a9640 1 -- 172.21.15.93:0/3324604123 shutdown_connections 2023-11-08T16:28:39.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.076+0000 7ff07b0a9640 1 -- 172.21.15.93:0/3324604123 wait complete. 2023-11-08T16:28:39.094 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new 1618ae25-61cf-4e42-b167-a75cccfd9d7b 1 2023-11-08T16:28:39.108 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:39.106+0000 7f478b837040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-08T16:28:39.128 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:39.127+0000 7fab3f21c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-08T16:28:39.164 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:39.162+0000 7f478b837040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-08T16:28:39.191 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:39.189+0000 7fab3f21c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-08T16:28:39.230 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:39.227+0000 7f478b837040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-08T16:28:39.240 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:39.238+0000 7fab3f21c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-08T16:28:39.281 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:39.279+0000 7f478b837040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-08T16:28:39.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.284+0000 7fbb64e58640 1 Processor -- start 2023-11-08T16:28:39.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.284+0000 7fbb64e58640 1 -- start start 2023-11-08T16:28:39.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb64e58640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb600cffe0 0x7fbb600b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb64e58640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbb600d39e0 0x7fbb600cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb64e58640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb600b22c0 con 0x7fbb60160a10 2023-11-08T16:28:39.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb64e58640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb60160df0 con 0x7fbb600cffe0 2023-11-08T16:28:39.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb64e58640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb600cf0d0 con 0x7fbb600d39e0 2023-11-08T16:28:39.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb5ed9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbb600d39e0 0x7fbb600cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb5ed9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbb600d39e0 0x7fbb600cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54768/0 (socket says 172.21.15.93:54768) 2023-11-08T16:28:39.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb5ed9d640 1 -- 172.21.15.93:0/612710418 learned_addr learned my addr 172.21.15.93:0/612710418 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:39.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb55d9b640 1 --2- 172.21.15.93:0/612710418 >> v2:172.21.15.81:3300/0 conn(0x7fbb600cffe0 0x7fbb600b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb5ed9d640 1 -- 172.21.15.93:0/612710418 >> v2:172.21.15.81:3300/0 conn(0x7fbb600cffe0 msgr2=0x7fbb600b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:39.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb5ed9d640 1 --2- 172.21.15.93:0/612710418 >> v2:172.21.15.81:3300/0 conn(0x7fbb600cffe0 0x7fbb600b42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb5ed9d640 1 -- 172.21.15.93:0/612710418 >> v1:172.21.15.93:6789/0 conn(0x7fbb60160a10 legacy=0x7fbb600cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.286+0000 7fbb5ed9d640 1 -- 172.21.15.93:0/612710418 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb600b2e20 con 0x7fbb600d39e0 2023-11-08T16:28:39.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.288+0000 7fbb5ed9d640 1 --2- 172.21.15.93:0/612710418 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb600d39e0 0x7fbb600cdb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009870 tx=0x7fbb5802b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e28937fdb39cf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:39.295 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.288+0000 7fbb5dd9b640 1 -- 172.21.15.93:0/612710418 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58031870 con 0x7fbb600d39e0 2023-11-08T16:28:39.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.288+0000 7fbb5dd9b640 1 -- 172.21.15.93:0/612710418 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb580319d0 con 0x7fbb600d39e0 2023-11-08T16:28:39.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.288+0000 7fbb5dd9b640 1 -- 172.21.15.93:0/612710418 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb58031d60 con 0x7fbb600d39e0 2023-11-08T16:28:39.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.288+0000 7fbb64e58640 1 -- 172.21.15.93:0/612710418 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb600d39e0 msgr2=0x7fbb600cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.288+0000 7fbb64e58640 1 --2- 172.21.15.93:0/612710418 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb600d39e0 0x7fbb600cdb80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009870 tx=0x7fbb5802b910 comp rx=0 tx=0).stop 2023-11-08T16:28:39.296 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.289+0000 7fbb64e58640 1 -- 172.21.15.93:0/612710418 shutdown_connections 2023-11-08T16:28:39.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.289+0000 7fbb64e58640 1 --2- 172.21.15.93:0/612710418 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb600d39e0 0x7fbb600cdb80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.289+0000 7fbb64e58640 1 --2- 172.21.15.93:0/612710418 >> v2:172.21.15.81:3300/0 conn(0x7fbb600cffe0 0x7fbb600b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.289+0000 7fbb64e58640 1 -- 172.21.15.93:0/612710418 >> 172.21.15.93:0/612710418 conn(0x7fbb600d2ec0 msgr2=0x7fbb600d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:39.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.289+0000 7fbb64e58640 1 -- 172.21.15.93:0/612710418 shutdown_connections 2023-11-08T16:28:39.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.289+0000 7fbb64e58640 1 -- 172.21.15.93:0/612710418 wait complete. 2023-11-08T16:28:39.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.290+0000 7fbb64e58640 1 Processor -- start 2023-11-08T16:28:39.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.291+0000 7fbb64e58640 1 -- start start 2023-11-08T16:28:39.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.291+0000 7fbb64e58640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600cffe0 0x7fbb600c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.291+0000 7fbb64e58640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600d39e0 0x7fbb600b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.298 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb64e58640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb60160a10 0x7fbb6014cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb64e58640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb601e5c50 con 0x7fbb600cffe0 2023-11-08T16:28:39.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb64e58640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600c8f20 con 0x7fbb60160a10 2023-11-08T16:28:39.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb64e58640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb600c9200 con 0x7fbb600d39e0 2023-11-08T16:28:39.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb55d9b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600d39e0 0x7fbb600b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.299 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb55d9b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600d39e0 0x7fbb600b1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.93:43612/0 (socket says 172.21.15.93:43612) 2023-11-08T16:28:39.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb55d9b640 1 -- 172.21.15.93:0/2157567353 learned_addr learned my addr 172.21.15.93:0/2157567353 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:39.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb5ed9d640 1 --2- 172.21.15.93:0/2157567353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb60160a10 0x7fbb6014cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb5e59c640 1 --2- 172.21.15.93:0/2157567353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600cffe0 0x7fbb600c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.300 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb55d9b640 1 -- 172.21.15.93:0/2157567353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb60160a10 msgr2=0x7fbb6014cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb55d9b640 1 --2- 172.21.15.93:0/2157567353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb60160a10 0x7fbb6014cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.301 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb55d9b640 1 -- 172.21.15.93:0/2157567353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600cffe0 msgr2=0x7fbb600c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb55d9b640 1 --2- 172.21.15.93:0/2157567353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600cffe0 0x7fbb600c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb55d9b640 1 -- 172.21.15.93:0/2157567353 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb580094d0 con 0x7fbb600d39e0 2023-11-08T16:28:39.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.292+0000 7fbb5e59c640 1 --2- 172.21.15.93:0/2157567353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600cffe0 0x7fbb600c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:28:39.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.293+0000 7fbb55d9b640 1 --2- 172.21.15.93:0/2157567353 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600d39e0 0x7fbb600b1650 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c7f0 tx=0x7fbb4800ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:39.302 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.293+0000 7fbb577fe640 1 -- 172.21.15.93:0/2157567353 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48004220 con 0x7fbb600d39e0 2023-11-08T16:28:39.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.293+0000 7fbb577fe640 1 -- 172.21.15.93:0/2157567353 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb480043a0 con 0x7fbb600d39e0 2023-11-08T16:28:39.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.293+0000 7fbb577fe640 1 -- 172.21.15.93:0/2157567353 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48007860 con 0x7fbb600d39e0 2023-11-08T16:28:39.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.293+0000 7fbb64e58640 1 -- 172.21.15.93:0/2157567353 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb600c94e0 con 0x7fbb600d39e0 2023-11-08T16:28:39.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.293+0000 7fbb64e58640 1 -- 172.21.15.93:0/2157567353 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb600c8c10 con 0x7fbb600d39e0 2023-11-08T16:28:39.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.293+0000 7fbb577fe640 1 -- 172.21.15.93:0/2157567353 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbb48007bd0 con 0x7fbb600d39e0 2023-11-08T16:28:39.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.293+0000 7fbb577fe640 1 -- 172.21.15.93:0/2157567353 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fbb480197b0 con 0x7fbb600d39e0 2023-11-08T16:28:39.304 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.294+0000 7fbb64e58640 1 -- 172.21.15.93:0/2157567353 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb601e60e0 con 0x7fbb600d39e0 2023-11-08T16:28:39.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.297+0000 7fbb577fe640 1 -- 172.21.15.93:0/2157567353 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fbb48038b40 con 0x7fbb600d39e0 2023-11-08T16:28:39.352 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:39.350+0000 7fab3f21c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-08T16:28:39.376 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.374+0000 7fbb64e58640 1 -- 172.21.15.93:0/2157567353 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1618ae25-61cf-4e42-b167-a75cccfd9d7b", "id": 1} v 0) v1 -- 0x7fbb600d0f80 con 0x7fbb600d39e0 2023-11-08T16:28:39.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.381+0000 7fbb577fe640 1 -- 172.21.15.93:0/2157567353 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1618ae25-61cf-4e42-b167-a75cccfd9d7b", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fbb48027030 con 0x7fbb600d39e0 2023-11-08T16:28:39.384 INFO:teuthology.orchestra.run.smithi093.stdout:1 2023-11-08T16:28:39.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.383+0000 7fbb64e58640 1 -- 172.21.15.93:0/2157567353 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600d39e0 msgr2=0x7fbb600b1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.383+0000 7fbb64e58640 1 --2- 172.21.15.93:0/2157567353 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600d39e0 0x7fbb600b1650 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fbb4800c7f0 tx=0x7fbb4800ccb0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.383+0000 7fbb64e58640 1 -- 172.21.15.93:0/2157567353 shutdown_connections 2023-11-08T16:28:39.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.383+0000 7fbb64e58640 1 --2- 172.21.15.93:0/2157567353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb60160a10 0x7fbb6014cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.383+0000 7fbb64e58640 1 --2- 172.21.15.93:0/2157567353 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600d39e0 0x7fbb600b1650 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.383+0000 7fbb64e58640 1 --2- 172.21.15.93:0/2157567353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600cffe0 0x7fbb600c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.383+0000 7fbb64e58640 1 -- 172.21.15.93:0/2157567353 >> 172.21.15.93:0/2157567353 conn(0x7fbb600d2ec0 msgr2=0x7fbb6015fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:39.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.383+0000 7fbb64e58640 1 -- 172.21.15.93:0/2157567353 shutdown_connections 2023-11-08T16:28:39.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.383+0000 7fbb64e58640 1 -- 172.21.15.93:0/2157567353 wait complete. 2023-11-08T16:28:39.401 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new fc50be73-b7fa-4c89-9ac0-a7568f2525b3 2 2023-11-08T16:28:39.402 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:39.400+0000 7f478b837040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-08T16:28:39.432 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:39.430+0000 7fab3f21c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-08T16:28:39.485 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:39.483+0000 7fab3f21c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-08T16:28:39.488 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:39.486+0000 7f478b837040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-08T16:28:39.544 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:39.542+0000 7fab3f21c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-08T16:28:39.545 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:39.542+0000 7f478b837040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-08T16:28:39.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.586+0000 7f0cbb59e640 1 Processor -- start 2023-11-08T16:28:39.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.586+0000 7f0cbb59e640 1 -- start start 2023-11-08T16:28:39.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbb59e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0cbc0ca6f0 0x7f0cbc0c9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cbc0c9cb0 0x7f0cbc06b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.591 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbb59e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f0cbc066250 0x7f0cbc06b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbb59e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cbc160bb0 con 0x7f0cbc0ca6f0 2023-11-08T16:28:39.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbb59e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cbc0caad0 con 0x7f0cbc066250 2023-11-08T16:28:39.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbb59e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc0719f0 con 0x7f0cbc0c9cb0 2023-11-08T16:28:39.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbad9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f0cbc066250 0x7f0cbc06b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.592 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cb3fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cbc0c9cb0 0x7f0cbc06b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbad9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f0cbc066250 0x7f0cbc06b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54782/0 (socket says 172.21.15.93:54782) 2023-11-08T16:28:39.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbad9d640 1 -- 172.21.15.93:0/2843785507 learned_addr learned my addr 172.21.15.93:0/2843785507 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:39.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cba59c640 1 --2- 172.21.15.93:0/2843785507 >> v2:172.21.15.81:3300/0 conn(0x7f0cbc0ca6f0 0x7f0cbc0c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbad9d640 1 -- 172.21.15.93:0/2843785507 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cbc0c9cb0 msgr2=0x7f0cbc06b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:39.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbad9d640 1 --2- 172.21.15.93:0/2843785507 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cbc0c9cb0 0x7f0cbc06b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbad9d640 1 -- 172.21.15.93:0/2843785507 >> v2:172.21.15.81:3300/0 conn(0x7f0cbc0ca6f0 msgr2=0x7f0cbc0c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:39.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.588+0000 7f0cbad9d640 1 --2- 172.21.15.93:0/2843785507 >> v2:172.21.15.81:3300/0 conn(0x7f0cbc0ca6f0 0x7f0cbc0c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.589+0000 7f0cbad9d640 1 -- 172.21.15.93:0/2843785507 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cbc071cd0 con 0x7f0cbc066250 2023-11-08T16:28:39.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.589+0000 7f0cb3fff640 1 --2- 172.21.15.93:0/2843785507 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cbc0c9cb0 0x7f0cbc06b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:28:39.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.589+0000 7f0cbad9d640 1 --2- 172.21.15.93:0/2843785507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cbc066250 0x7f0cbc06b600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0aeb10 tx=0x7f0cb402be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c3ee12457642745 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:39.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.590+0000 7f0cb9d9b640 1 -- 172.21.15.93:0/2843785507 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb402c9f0 con 0x7f0cbc066250 2023-11-08T16:28:39.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.590+0000 7f0cb9d9b640 1 -- 172.21.15.93:0/2843785507 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb402cb50 con 0x7f0cbc066250 2023-11-08T16:28:39.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.590+0000 7f0cb9d9b640 1 -- 172.21.15.93:0/2843785507 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cb40343f0 con 0x7f0cbc066250 2023-11-08T16:28:39.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.590+0000 7f0cbb59e640 1 -- 172.21.15.93:0/2843785507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cbc066250 msgr2=0x7f0cbc06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.590+0000 7f0cbb59e640 1 --2- 172.21.15.93:0/2843785507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cbc066250 0x7f0cbc06b600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc0aeb10 tx=0x7f0cb402be50 comp rx=0 tx=0).stop 2023-11-08T16:28:39.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.591+0000 7f0cbb59e640 1 -- 172.21.15.93:0/2843785507 shutdown_connections 2023-11-08T16:28:39.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.591+0000 7f0cbb59e640 1 --2- 172.21.15.93:0/2843785507 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cbc0c9cb0 0x7f0cbc06b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.591+0000 7f0cbb59e640 1 --2- 172.21.15.93:0/2843785507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cbc066250 0x7f0cbc06b600 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.591+0000 7f0cbb59e640 1 --2- 172.21.15.93:0/2843785507 >> v2:172.21.15.81:3300/0 conn(0x7f0cbc0ca6f0 0x7f0cbc0c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.591+0000 7f0cbb59e640 1 -- 172.21.15.93:0/2843785507 >> 172.21.15.93:0/2843785507 conn(0x7f0cbc0b0c10 msgr2=0x7f0cbc0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:39.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.591+0000 7f0cbb59e640 1 -- 172.21.15.93:0/2843785507 shutdown_connections 2023-11-08T16:28:39.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.591+0000 7f0cbb59e640 1 -- 172.21.15.93:0/2843785507 wait complete. 2023-11-08T16:28:39.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.592+0000 7f0cbb59e640 1 Processor -- start 2023-11-08T16:28:39.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.593+0000 7f0cbb59e640 1 -- start start 2023-11-08T16:28:39.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.593+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cbc066250 0x7f0cbc15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cbc0ca6f0 0x7f0cbc0b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cbb59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cbc09b460 0x7f0cbc06bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cbb59e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc0b30f0 con 0x7f0cbc09b460 2023-11-08T16:28:39.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cbb59e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc0b3340 con 0x7f0cbc066250 2023-11-08T16:28:39.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cbb59e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cbc0b3620 con 0x7f0cbc0ca6f0 2023-11-08T16:28:39.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cba59c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cbc066250 0x7f0cbc15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cba59c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cbc066250 0x7f0cbc15ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54792/0 (socket says 172.21.15.93:54792) 2023-11-08T16:28:39.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cba59c640 1 -- 172.21.15.93:0/3681318912 learned_addr learned my addr 172.21.15.93:0/3681318912 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:39.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cb3fff640 1 --2- 172.21.15.93:0/3681318912 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cbc0ca6f0 0x7f0cbc0b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cba59c640 1 -- 172.21.15.93:0/3681318912 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cbc0ca6f0 msgr2=0x7f0cbc0b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cba59c640 1 --2- 172.21.15.93:0/3681318912 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cbc0ca6f0 0x7f0cbc0b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cba59c640 1 -- 172.21.15.93:0/3681318912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cbc09b460 msgr2=0x7f0cbc06bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cbad9d640 1 --2- 172.21.15.93:0/3681318912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cbc09b460 0x7f0cbc06bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cba59c640 1 --2- 172.21.15.93:0/3681318912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cbc09b460 0x7f0cbc06bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.594+0000 7f0cba59c640 1 -- 172.21.15.93:0/3681318912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca4009400 con 0x7f0cbc066250 2023-11-08T16:28:39.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.595+0000 7f0cba59c640 1 --2- 172.21.15.93:0/3681318912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cbc066250 0x7f0cbc15ebf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d800 tx=0x7f0cac00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:39.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.595+0000 7f0cb2ffd640 1 -- 172.21.15.93:0/3681318912 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00b6c0 con 0x7f0cbc066250 2023-11-08T16:28:39.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.595+0000 7f0cb2ffd640 1 -- 172.21.15.93:0/3681318912 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cac00b820 con 0x7f0cbc066250 2023-11-08T16:28:39.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.595+0000 7f0cb2ffd640 1 -- 172.21.15.93:0/3681318912 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cac00bbb0 con 0x7f0cbc066250 2023-11-08T16:28:39.605 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.595+0000 7f0cbb59e640 1 -- 172.21.15.93:0/3681318912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb4009b30 con 0x7f0cbc066250 2023-11-08T16:28:39.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.595+0000 7f0cbb59e640 1 -- 172.21.15.93:0/3681318912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cbc0b3be0 con 0x7f0cbc066250 2023-11-08T16:28:39.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.595+0000 7f0cb2ffd640 1 -- 172.21.15.93:0/3681318912 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0cac01e760 con 0x7f0cbc066250 2023-11-08T16:28:39.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.595+0000 7f0cb2ffd640 1 -- 172.21.15.93:0/3681318912 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f0cac0148f0 con 0x7f0cbc066250 2023-11-08T16:28:39.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.596+0000 7f0cbb59e640 1 -- 172.21.15.93:0/3681318912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cbc067800 con 0x7f0cbc066250 2023-11-08T16:28:39.607 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:39.597+0000 7fab3f21c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-08T16:28:39.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.599+0000 7f0cb2ffd640 1 -- 172.21.15.93:0/3681318912 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f0cac0144d0 con 0x7f0cbc066250 2023-11-08T16:28:39.608 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:39.606+0000 7f478b837040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-08T16:28:39.665 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:39.662+0000 7f478b837040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-08T16:28:39.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.676+0000 7f0cbb59e640 1 -- 172.21.15.93:0/3681318912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fc50be73-b7fa-4c89-9ac0-a7568f2525b3", "id": 2} v 0) v1 -- 0x7f0cbc0b50a0 con 0x7f0cbc066250 2023-11-08T16:28:39.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.684+0000 7f0cb2ffd640 1 -- 172.21.15.93:0/3681318912 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fc50be73-b7fa-4c89-9ac0-a7568f2525b3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0cac014730 con 0x7f0cbc066250 2023-11-08T16:28:39.687 INFO:teuthology.orchestra.run.smithi093.stdout:2 2023-11-08T16:28:39.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.686+0000 7f0cbb59e640 1 -- 172.21.15.93:0/3681318912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cbc066250 msgr2=0x7f0cbc15ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.686+0000 7f0cbb59e640 1 --2- 172.21.15.93:0/3681318912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cbc066250 0x7f0cbc15ebf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00d800 tx=0x7f0cac00dcc0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.686+0000 7f0cbb59e640 1 -- 172.21.15.93:0/3681318912 shutdown_connections 2023-11-08T16:28:39.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.686+0000 7f0cbb59e640 1 --2- 172.21.15.93:0/3681318912 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0cbc09b460 0x7f0cbc06bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.686+0000 7f0cbb59e640 1 --2- 172.21.15.93:0/3681318912 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0cbc0ca6f0 0x7f0cbc0b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.686+0000 7f0cbb59e640 1 --2- 172.21.15.93:0/3681318912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cbc066250 0x7f0cbc15ebf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.686+0000 7f0cbb59e640 1 -- 172.21.15.93:0/3681318912 >> 172.21.15.93:0/3681318912 conn(0x7f0cbc0b0c10 msgr2=0x7f0cbc0b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:39.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.686+0000 7f0cbb59e640 1 -- 172.21.15.93:0/3681318912 shutdown_connections 2023-11-08T16:28:39.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.686+0000 7f0cbb59e640 1 -- 172.21.15.93:0/3681318912 wait complete. 2023-11-08T16:28:39.704 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new d9aad8ef-8624-40c1-9504-bd6525972f3a 3 2023-11-08T16:28:39.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.895+0000 7fc0c8373640 1 Processor -- start 2023-11-08T16:28:39.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.896+0000 7fc0c8373640 1 -- start start 2023-11-08T16:28:39.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.897+0000 7fc0c8373640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc0c0160a10 0x7fc0c00cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.897+0000 7fc0c8373640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc0c00d39e0 0x7fc0c00cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.897+0000 7fc0c8373640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c00b22c0 con 0x7fc0c00b32b0 2023-11-08T16:28:39.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.897+0000 7fc0c8373640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c0160df0 con 0x7fc0c00d39e0 2023-11-08T16:28:39.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.897+0000 7fc0c8373640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c00b2e20 con 0x7fc0c0160a10 2023-11-08T16:28:39.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.897+0000 7fc0c610f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc0c0160a10 0x7fc0c00cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.897+0000 7fc0c610f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc0c0160a10 0x7fc0c00cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54808/0 (socket says 172.21.15.93:54808) 2023-11-08T16:28:39.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.897+0000 7fc0c610f640 1 -- 172.21.15.93:0/2825453493 learned_addr learned my addr 172.21.15.93:0/2825453493 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:39.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.898+0000 7fc0c610f640 1 -- 172.21.15.93:0/2825453493 >> v2:172.21.15.81:3300/0 conn(0x7fc0c00d39e0 msgr2=0x7fc0c00cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:39.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.898+0000 7fc0c610f640 1 --2- 172.21.15.93:0/2825453493 >> v2:172.21.15.81:3300/0 conn(0x7fc0c00d39e0 0x7fc0c00cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.898+0000 7fc0c610f640 1 -- 172.21.15.93:0/2825453493 >> v1:172.21.15.93:6789/0 conn(0x7fc0c00b32b0 legacy=0x7fc0c00b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.898+0000 7fc0c510d640 1 -- 172.21.15.93:0/2825453493 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173463733 0 0) 0x7fc0c00b22c0 con 0x7fc0c00b32b0 2023-11-08T16:28:39.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.898+0000 7fc0c610f640 1 -- 172.21.15.93:0/2825453493 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c00d0fc0 con 0x7fc0c0160a10 2023-11-08T16:28:39.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.899+0000 7fc0c610f640 1 --2- 172.21.15.93:0/2825453493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0c0160a10 0x7fc0c00cf8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc0c008bcb0 tx=0x7fc0bc02b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6da4a82e73aa21a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:39.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.899+0000 7fc0c510d640 1 -- 172.21.15.93:0/2825453493 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc031860 con 0x7fc0c0160a10 2023-11-08T16:28:39.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.899+0000 7fc0c510d640 1 -- 172.21.15.93:0/2825453493 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0bc0319c0 con 0x7fc0c0160a10 2023-11-08T16:28:39.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.899+0000 7fc0c510d640 1 -- 172.21.15.93:0/2825453493 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc031d50 con 0x7fc0c0160a10 2023-11-08T16:28:39.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.900+0000 7fc0c8373640 1 -- 172.21.15.93:0/2825453493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0c0160a10 msgr2=0x7fc0c00cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.900+0000 7fc0c8373640 1 --2- 172.21.15.93:0/2825453493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0c0160a10 0x7fc0c00cf8d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc0c008bcb0 tx=0x7fc0bc02b900 comp rx=0 tx=0).stop 2023-11-08T16:28:39.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.900+0000 7fc0c8373640 1 -- 172.21.15.93:0/2825453493 shutdown_connections 2023-11-08T16:28:39.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.900+0000 7fc0c8373640 1 --2- 172.21.15.93:0/2825453493 >> v2:172.21.15.81:3300/0 conn(0x7fc0c00d39e0 0x7fc0c00cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.900+0000 7fc0c8373640 1 --2- 172.21.15.93:0/2825453493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0c0160a10 0x7fc0c00cf8d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.900+0000 7fc0c8373640 1 -- 172.21.15.93:0/2825453493 >> 172.21.15.93:0/2825453493 conn(0x7fc0c00d2ec0 msgr2=0x7fc0c00d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:39.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.900+0000 7fc0c8373640 1 -- 172.21.15.93:0/2825453493 shutdown_connections 2023-11-08T16:28:39.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.900+0000 7fc0c8373640 1 -- 172.21.15.93:0/2825453493 wait complete. 2023-11-08T16:28:39.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.903+0000 7fc0c8373640 1 Processor -- start 2023-11-08T16:28:39.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.903+0000 7fc0c8373640 1 -- start start 2023-11-08T16:28:39.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.904+0000 7fc0c8373640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc0c00b32b0 0x7fc0c01f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c8373640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0c00d39e0 0x7fc0c006bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c8373640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c0160a10 0x7fc0c0151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:39.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c8373640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c01f3c60 con 0x7fc0c0160a10 2023-11-08T16:28:39.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c8373640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c006ab80 con 0x7fc0c00d39e0 2023-11-08T16:28:39.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c8373640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c006ae60 con 0x7fc0c00b32b0 2023-11-08T16:28:39.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c590e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0c00d39e0 0x7fc0c006bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c610f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc0c00b32b0 0x7fc0c01f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c610f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc0c00b32b0 0x7fc0c01f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.93:43648/0 (socket says 172.21.15.93:43648) 2023-11-08T16:28:39.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c610f640 1 -- 172.21.15.93:0/1317886060 learned_addr learned my addr 172.21.15.93:0/1317886060 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:39.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c590e640 1 -- 172.21.15.93:0/1317886060 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc0c00b32b0 msgr2=0x7fc0c01f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c6910640 1 --2- 172.21.15.93:0/1317886060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c0160a10 0x7fc0c0151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:39.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c590e640 1 --2- 172.21.15.93:0/1317886060 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc0c00b32b0 0x7fc0c01f3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c590e640 1 -- 172.21.15.93:0/1317886060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c0160a10 msgr2=0x7fc0c0151210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:39.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c590e640 1 --2- 172.21.15.93:0/1317886060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c0160a10 0x7fc0c0151210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:39.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c590e640 1 -- 172.21.15.93:0/1317886060 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc009470 con 0x7fc0c00d39e0 2023-11-08T16:28:39.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.905+0000 7fc0c6910640 1 --2- 172.21.15.93:0/1317886060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c0160a10 0x7fc0c0151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:28:39.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.906+0000 7fc0c590e640 1 --2- 172.21.15.93:0/1317886060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0c00d39e0 0x7fc0c006bc20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000b3f0 tx=0x7fc0b000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:39.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.906+0000 7fc0b6ffd640 1 -- 172.21.15.93:0/1317886060 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0011830 con 0x7fc0c00d39e0 2023-11-08T16:28:39.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.906+0000 7fc0b6ffd640 1 -- 172.21.15.93:0/1317886060 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b0011990 con 0x7fc0c00d39e0 2023-11-08T16:28:39.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.906+0000 7fc0b6ffd640 1 -- 172.21.15.93:0/1317886060 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0011d40 con 0x7fc0c00d39e0 2023-11-08T16:28:39.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.906+0000 7fc0c8373640 1 -- 172.21.15.93:0/1317886060 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c006b140 con 0x7fc0c00d39e0 2023-11-08T16:28:39.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.906+0000 7fc0c8373640 1 -- 172.21.15.93:0/1317886060 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c006b700 con 0x7fc0c00d39e0 2023-11-08T16:28:39.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.906+0000 7fc0b6ffd640 1 -- 172.21.15.93:0/1317886060 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc0b001a700 con 0x7fc0c00d39e0 2023-11-08T16:28:39.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.906+0000 7fc0b6ffd640 1 -- 172.21.15.93:0/1317886060 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fc0b0019840 con 0x7fc0c00d39e0 2023-11-08T16:28:39.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.907+0000 7fc0c8373640 1 -- 172.21.15.93:0/1317886060 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c00b2780 con 0x7fc0c00d39e0 2023-11-08T16:28:39.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.910+0000 7fc0b6ffd640 1 -- 172.21.15.93:0/1317886060 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc0b0039b40 con 0x7fc0c00d39e0 2023-11-08T16:28:39.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.987+0000 7fc0c8373640 1 -- 172.21.15.93:0/1317886060 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d9aad8ef-8624-40c1-9504-bd6525972f3a", "id": 3} v 0) v1 -- 0x7fc0c006b8f0 con 0x7fc0c00d39e0 2023-11-08T16:28:39.998 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.995+0000 7fc0b6ffd640 1 -- 172.21.15.93:0/1317886060 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d9aad8ef-8624-40c1-9504-bd6525972f3a", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc0b0028030 con 0x7fc0c00d39e0 2023-11-08T16:28:39.998 INFO:teuthology.orchestra.run.smithi093.stdout:3 2023-11-08T16:28:40.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.996+0000 7fc0c8373640 1 -- 172.21.15.93:0/1317886060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0c00d39e0 msgr2=0x7fc0c006bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.996+0000 7fc0c8373640 1 --2- 172.21.15.93:0/1317886060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0c00d39e0 0x7fc0c006bc20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000b3f0 tx=0x7fc0b000b8b0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.996+0000 7fc0c8373640 1 -- 172.21.15.93:0/1317886060 shutdown_connections 2023-11-08T16:28:40.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.996+0000 7fc0c8373640 1 --2- 172.21.15.93:0/1317886060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0c0160a10 0x7fc0c0151210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.996+0000 7fc0c8373640 1 --2- 172.21.15.93:0/1317886060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0c00d39e0 0x7fc0c006bc20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.996+0000 7fc0c8373640 1 --2- 172.21.15.93:0/1317886060 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc0c00b32b0 0x7fc0c01f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.997+0000 7fc0c8373640 1 -- 172.21.15.93:0/1317886060 >> 172.21.15.93:0/1317886060 conn(0x7fc0c00d2ec0 msgr2=0x7fc0c015fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.997+0000 7fc0c8373640 1 -- 172.21.15.93:0/1317886060 shutdown_connections 2023-11-08T16:28:40.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:39.997+0000 7fc0c8373640 1 -- 172.21.15.93:0/1317886060 wait complete. 2023-11-08T16:28:40.016 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new 908bf7ff-953c-4d13-b343-4e552afd723e 4 2023-11-08T16:28:40.047 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:40.045+0000 7fab3f21c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-08T16:28:40.149 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:40.147+0000 7f478b837040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-08T16:28:40.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.203+0000 7fe5f2ae7640 1 Processor -- start 2023-11-08T16:28:40.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.203+0000 7fe5f2ae7640 1 -- start start 2023-11-08T16:28:40.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.205+0000 7fe5f2ae7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5ec158c90 0x7fe5ec06abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.205+0000 7fe5f2ae7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ec0cf420 con 0x7fe5ec0ca200 2023-11-08T16:28:40.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.205+0000 7fe5f2ae7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ec06a150 con 0x7fe5ec15f1f0 2023-11-08T16:28:40.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.205+0000 7fe5f2ae7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ec0b56e0 con 0x7fe5ec158c90 2023-11-08T16:28:40.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.205+0000 7fe5f1084640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fe5ec15f1f0 0x7fe5ec070a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.93:55752/0 (socket says 172.21.15.93:55752) 2023-11-08T16:28:40.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.205+0000 7fe5f1084640 1 -- 172.21.15.93:0/556625734 learned_addr learned my addr 172.21.15.93:0/556625734 (peer_addr_for_me v1:172.21.15.93:0/0) 2023-11-08T16:28:40.209 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:40.205+0000 7fab3f21c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-08T16:28:40.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.205+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 520289760 0 0) 0x7fe5ec06a150 con 0x7fe5ec15f1f0 2023-11-08T16:28:40.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.205+0000 7fe5f0883640 1 --2- 172.21.15.93:0/556625734 >> v2:172.21.15.81:3300/0 conn(0x7fe5ec158c90 0x7fe5ec06abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.205+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5dc003520 con 0x7fe5ec15f1f0 2023-11-08T16:28:40.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 <== mon.1 v1:172.21.15.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3750718593 0 0) 0x7fe5dc003520 con 0x7fe5ec15f1f0 2023-11-08T16:28:40.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 >> v2:172.21.15.81:3300/0 conn(0x7fe5ec158c90 msgr2=0x7fe5ec06abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:40.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 --2- 172.21.15.93:0/556625734 >> v2:172.21.15.81:3300/0 conn(0x7fe5ec158c90 0x7fe5ec06abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 >> v1:172.21.15.81:6789/0 conn(0x7fe5ec0ca200 legacy=0x7fe5ec15eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 --> v1:172.21.15.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec0cf670 con 0x7fe5ec15f1f0 2023-11-08T16:28:40.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 <== mon.1 v1:172.21.15.93:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fe5e40019f0 con 0x7fe5ec15f1f0 2023-11-08T16:28:40.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 >> v1:172.21.15.93:6789/0 conn(0x7fe5ec15f1f0 legacy=0x7fe5ec070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5dc003a30 0x7fe5dc003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5dc0047c0 0x7fe5dc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc005380 0x7fe5dc00b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc00c290 con 0x7fe5dc003a30 2023-11-08T16:28:40.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc00c400 con 0x7fe5dc0047c0 2023-11-08T16:28:40.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc00c5c0 con 0x7fe5dc005380 2023-11-08T16:28:40.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5ebfff640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5dc0047c0 0x7fe5dc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5f1084640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc005380 0x7fe5dc00b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.206+0000 7fe5f0883640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5dc003a30 0x7fe5dc003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.207+0000 7fe5ebfff640 1 -- 172.21.15.93:0/556625734 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc005380 msgr2=0x7fe5dc00b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:40.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.207+0000 7fe5ebfff640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc005380 0x7fe5dc00b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.207+0000 7fe5ebfff640 1 -- 172.21.15.93:0/556625734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5dc003a30 msgr2=0x7fe5dc003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:40.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.207+0000 7fe5ebfff640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5dc003a30 0x7fe5dc003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.207+0000 7fe5ebfff640 1 -- 172.21.15.93:0/556625734 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc003520 con 0x7fe5dc0047c0 2023-11-08T16:28:40.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.207+0000 7fe5f1084640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc005380 0x7fe5dc00b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:28:40.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.207+0000 7fe5f1084640 1 -- 172.21.15.93:0/556625734 reap_dead start 2023-11-08T16:28:40.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.208+0000 7fe5ebfff640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5dc0047c0 0x7fe5dc004c40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8001750 tx=0x7fe5dc0057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e2a0e289d692e93 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:40.216 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.208+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80052d0 con 0x7fe5dc0047c0 2023-11-08T16:28:40.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.208+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d802e050 con 0x7fe5dc0047c0 2023-11-08T16:28:40.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.208+0000 7fe5eb7fe640 1 -- 172.21.15.93:0/556625734 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8003830 con 0x7fe5dc0047c0 2023-11-08T16:28:40.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.209+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/556625734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5dc0047c0 msgr2=0x7fe5dc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.209+0000 7fe5f2ae7640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5dc0047c0 0x7fe5dc004c40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8001750 tx=0x7fe5dc0057f0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.209+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/556625734 shutdown_connections 2023-11-08T16:28:40.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.209+0000 7fe5f2ae7640 1 --2- 172.21.15.93:0/556625734 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5dc0047c0 0x7fe5dc004c40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.209+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/556625734 >> 172.21.15.93:0/556625734 conn(0x7fe5ec0cfbb0 msgr2=0x7fe5ec0cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:40.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.209+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/556625734 shutdown_connections 2023-11-08T16:28:40.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.209+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/556625734 wait complete. 2023-11-08T16:28:40.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.211+0000 7fe5f2ae7640 1 Processor -- start 2023-11-08T16:28:40.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.211+0000 7fe5f2ae7640 1 -- start start 2023-11-08T16:28:40.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f2ae7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc0047c0 0x7fe5ec0a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f2ae7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5ec15df80 0x7fe5ec15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f2ae7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5ec0c7f40 0x7fe5ec154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f2ae7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec15d3d0 con 0x7fe5ec15df80 2023-11-08T16:28:40.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f2ae7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec15d150 con 0x7fe5ec0c7f40 2023-11-08T16:28:40.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f2ae7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec0a8fe0 con 0x7fe5dc0047c0 2023-11-08T16:28:40.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f0883640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc0047c0 0x7fe5ec0a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f1084640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5ec0c7f40 0x7fe5ec154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f0883640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc0047c0 0x7fe5ec0a9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.93:43668/0 (socket says 172.21.15.93:43668) 2023-11-08T16:28:40.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f0883640 1 -- 172.21.15.93:0/92199465 learned_addr learned my addr 172.21.15.93:0/92199465 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:40.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5ebfff640 1 --2- 172.21.15.93:0/92199465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5ec15df80 0x7fe5ec15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f0883640 1 -- 172.21.15.93:0/92199465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5ec0c7f40 msgr2=0x7fe5ec154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f0883640 1 --2- 172.21.15.93:0/92199465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5ec0c7f40 0x7fe5ec154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f0883640 1 -- 172.21.15.93:0/92199465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5ec15df80 msgr2=0x7fe5ec15cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f0883640 1 --2- 172.21.15.93:0/92199465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5ec15df80 0x7fe5ec15cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.212+0000 7fe5f0883640 1 -- 172.21.15.93:0/92199465 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec0a92c0 con 0x7fe5dc0047c0 2023-11-08T16:28:40.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.213+0000 7fe5f0883640 1 --2- 172.21.15.93:0/92199465 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc0047c0 0x7fe5ec0a9da0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000f5a0 tx=0x7fe5e000fa60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:40.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.213+0000 7fe5e97fa640 1 -- 172.21.15.93:0/92199465 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0014870 con 0x7fe5dc0047c0 2023-11-08T16:28:40.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.213+0000 7fe5e97fa640 1 -- 172.21.15.93:0/92199465 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e00149d0 con 0x7fe5dc0047c0 2023-11-08T16:28:40.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.213+0000 7fe5e97fa640 1 -- 172.21.15.93:0/92199465 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0014d40 con 0x7fe5dc0047c0 2023-11-08T16:28:40.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.213+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/92199465 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ec0a95a0 con 0x7fe5dc0047c0 2023-11-08T16:28:40.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.213+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/92199465 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ec0c83c0 con 0x7fe5dc0047c0 2023-11-08T16:28:40.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.213+0000 7fe5e97fa640 1 -- 172.21.15.93:0/92199465 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe5e0015710 con 0x7fe5dc0047c0 2023-11-08T16:28:40.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.213+0000 7fe5e97fa640 1 -- 172.21.15.93:0/92199465 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fe5e001ebe0 con 0x7fe5dc0047c0 2023-11-08T16:28:40.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.214+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/92199465 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ec15e360 con 0x7fe5dc0047c0 2023-11-08T16:28:40.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.216+0000 7fe5e97fa640 1 -- 172.21.15.93:0/92199465 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fe5e003db40 con 0x7fe5dc0047c0 2023-11-08T16:28:40.297 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.295+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/92199465 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd new", "uuid": "908bf7ff-953c-4d13-b343-4e552afd723e", "id": 4} v 0) v1 -- 0x7fe5ec04ece0 con 0x7fe5dc0047c0 2023-11-08T16:28:40.303 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.301+0000 7fe5e97fa640 1 -- 172.21.15.93:0/92199465 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "908bf7ff-953c-4d13-b343-4e552afd723e", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe5e002c070 con 0x7fe5dc0047c0 2023-11-08T16:28:40.304 INFO:teuthology.orchestra.run.smithi093.stdout:4 2023-11-08T16:28:40.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.302+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/92199465 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc0047c0 msgr2=0x7fe5ec0a9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.302+0000 7fe5f2ae7640 1 --2- 172.21.15.93:0/92199465 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc0047c0 0x7fe5ec0a9da0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000f5a0 tx=0x7fe5e000fa60 comp rx=0 tx=0).stop 2023-11-08T16:28:40.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.302+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/92199465 shutdown_connections 2023-11-08T16:28:40.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.302+0000 7fe5f2ae7640 1 --2- 172.21.15.93:0/92199465 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5ec0c7f40 0x7fe5ec154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.302+0000 7fe5f2ae7640 1 --2- 172.21.15.93:0/92199465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5ec15df80 0x7fe5ec15cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.302+0000 7fe5f2ae7640 1 --2- 172.21.15.93:0/92199465 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5dc0047c0 0x7fe5ec0a9da0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.302+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/92199465 >> 172.21.15.93:0/92199465 conn(0x7fe5ec0cfbb0 msgr2=0x7fe5ec0b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:40.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.302+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/92199465 shutdown_connections 2023-11-08T16:28:40.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.302+0000 7fe5f2ae7640 1 -- 172.21.15.93:0/92199465 wait complete. 2023-11-08T16:28:40.318 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:40.316+0000 7f478b837040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-08T16:28:40.321 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new 1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3 5 2023-11-08T16:28:40.341 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:40.339+0000 7fab3f21c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-08T16:28:40.393 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:40.392+0000 7fab3f21c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-08T16:28:40.462 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:40.459+0000 7f478b837040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-08T16:28:40.501 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:40.500+0000 7fab3f21c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-08T16:28:40.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.509+0000 7fb0d60c3640 1 Processor -- start 2023-11-08T16:28:40.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.509+0000 7fb0d60c3640 1 -- start start 2023-11-08T16:28:40.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.510+0000 7fb0d60c3640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb0d015cf30 0x7fb0d00d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.510+0000 7fb0d60c3640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb0d00cfd90 0x7fb0d00cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.510+0000 7fb0d60c3640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d00ce0b0 con 0x7fb0d015d310 2023-11-08T16:28:40.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.510+0000 7fb0d60c3640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d00cdb70 con 0x7fb0d00cfd90 2023-11-08T16:28:40.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.510+0000 7fb0d60c3640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d00ce390 con 0x7fb0d015cf30 2023-11-08T16:28:40.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.511+0000 7fb0d50c1640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb0d015cf30 0x7fb0d00d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.511+0000 7fb0d50c1640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb0d015cf30 0x7fb0d00d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54842/0 (socket says 172.21.15.93:54842) 2023-11-08T16:28:40.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.511+0000 7fb0d50c1640 1 -- 172.21.15.93:0/2128189443 learned_addr learned my addr 172.21.15.93:0/2128189443 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:40.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.511+0000 7fb0d58c2640 1 --2- 172.21.15.93:0/2128189443 >> v2:172.21.15.81:3300/0 conn(0x7fb0d00cfd90 0x7fb0d00cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.511+0000 7fb0d50c1640 1 -- 172.21.15.93:0/2128189443 >> v2:172.21.15.81:3300/0 conn(0x7fb0d00cfd90 msgr2=0x7fb0d00cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:40.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.511+0000 7fb0d50c1640 1 --2- 172.21.15.93:0/2128189443 >> v2:172.21.15.81:3300/0 conn(0x7fb0d00cfd90 0x7fb0d00cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.511+0000 7fb0d50c1640 1 -- 172.21.15.93:0/2128189443 >> v1:172.21.15.81:6789/0 conn(0x7fb0d015d310 legacy=0x7fb0d00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:40.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.511+0000 7fb0d50c1640 1 -- 172.21.15.93:0/2128189443 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d00d1be0 con 0x7fb0d015cf30 2023-11-08T16:28:40.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.512+0000 7fb0d50c1640 1 --2- 172.21.15.93:0/2128189443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0d015cf30 0x7fb0d00d26c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009870 tx=0x7fb0cc02b9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce2649baaecf235b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:40.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.513+0000 7fb0c7fff640 1 -- 172.21.15.93:0/2128189443 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc030860 con 0x7fb0d015cf30 2023-11-08T16:28:40.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.513+0000 7fb0c7fff640 1 -- 172.21.15.93:0/2128189443 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc0309c0 con 0x7fb0d015cf30 2023-11-08T16:28:40.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.513+0000 7fb0c7fff640 1 -- 172.21.15.93:0/2128189443 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc030d50 con 0x7fb0d015cf30 2023-11-08T16:28:40.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.513+0000 7fb0d60c3640 1 -- 172.21.15.93:0/2128189443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0d015cf30 msgr2=0x7fb0d00d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.513+0000 7fb0d60c3640 1 --2- 172.21.15.93:0/2128189443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0d015cf30 0x7fb0d00d26c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc009870 tx=0x7fb0cc02b9a0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.513+0000 7fb0d60c3640 1 -- 172.21.15.93:0/2128189443 shutdown_connections 2023-11-08T16:28:40.520 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.513+0000 7fb0d60c3640 1 --2- 172.21.15.93:0/2128189443 >> v2:172.21.15.81:3300/0 conn(0x7fb0d00cfd90 0x7fb0d00cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.520 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.513+0000 7fb0d60c3640 1 --2- 172.21.15.93:0/2128189443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0d015cf30 0x7fb0d00d26c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.520 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.513+0000 7fb0d60c3640 1 -- 172.21.15.93:0/2128189443 >> 172.21.15.93:0/2128189443 conn(0x7fb0d006c040 msgr2=0x7fb0d006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:40.520 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.514+0000 7fb0d60c3640 1 -- 172.21.15.93:0/2128189443 shutdown_connections 2023-11-08T16:28:40.521 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.514+0000 7fb0d60c3640 1 -- 172.21.15.93:0/2128189443 wait complete. 2023-11-08T16:28:40.521 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:40.515+0000 7f478b837040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-08T16:28:40.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.515+0000 7fb0d60c3640 1 Processor -- start 2023-11-08T16:28:40.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.515+0000 7fb0d60c3640 1 -- start start 2023-11-08T16:28:40.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.516+0000 7fb0d60c3640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0d00cfd90 0x7fb0d00c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.516+0000 7fb0d60c3640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0d015cf30 0x7fb0d00c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.516+0000 7fb0d60c3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0d01491a0 0x7fb0d01478b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.516+0000 7fb0d60c3640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0148d10 con 0x7fb0d01491a0 2023-11-08T16:28:40.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.516+0000 7fb0d60c3640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d01f27f0 con 0x7fb0d00cfd90 2023-11-08T16:28:40.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.516+0000 7fb0d60c3640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d0148100 con 0x7fb0d015cf30 2023-11-08T16:28:40.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.516+0000 7fb0d50c1640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0d00cfd90 0x7fb0d00c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.516+0000 7fb0d48c0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0d015cf30 0x7fb0d00c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.517+0000 7fb0d50c1640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0d00cfd90 0x7fb0d00c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54856/0 (socket says 172.21.15.93:54856) 2023-11-08T16:28:40.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.517+0000 7fb0d50c1640 1 -- 172.21.15.93:0/808341985 learned_addr learned my addr 172.21.15.93:0/808341985 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:40.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.517+0000 7fb0d58c2640 1 --2- 172.21.15.93:0/808341985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0d01491a0 0x7fb0d01478b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.517+0000 7fb0d50c1640 1 -- 172.21.15.93:0/808341985 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0d015cf30 msgr2=0x7fb0d00c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.517+0000 7fb0d50c1640 1 --2- 172.21.15.93:0/808341985 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0d015cf30 0x7fb0d00c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.517+0000 7fb0d50c1640 1 -- 172.21.15.93:0/808341985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0d01491a0 msgr2=0x7fb0d01478b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.517+0000 7fb0d50c1640 1 --2- 172.21.15.93:0/808341985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0d01491a0 0x7fb0d01478b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.517+0000 7fb0d50c1640 1 -- 172.21.15.93:0/808341985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0cc0094d0 con 0x7fb0d00cfd90 2023-11-08T16:28:40.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.517+0000 7fb0d48c0640 1 --2- 172.21.15.93:0/808341985 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0d015cf30 0x7fb0d00c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:28:40.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.517+0000 7fb0d50c1640 1 --2- 172.21.15.93:0/808341985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0d00cfd90 0x7fb0d00c85d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:40.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.518+0000 7fb0c5ffb640 1 -- 172.21.15.93:0/808341985 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc039c90 con 0x7fb0d00cfd90 2023-11-08T16:28:40.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.518+0000 7fb0c5ffb640 1 -- 172.21.15.93:0/808341985 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0cc039df0 con 0x7fb0d00cfd90 2023-11-08T16:28:40.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.518+0000 7fb0d60c3640 1 -- 172.21.15.93:0/808341985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d01483e0 con 0x7fb0d00cfd90 2023-11-08T16:28:40.532 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.518+0000 7fb0c5ffb640 1 -- 172.21.15.93:0/808341985 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0cc043640 con 0x7fb0d00cfd90 2023-11-08T16:28:40.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.518+0000 7fb0d60c3640 1 -- 172.21.15.93:0/808341985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d01489a0 con 0x7fb0d00cfd90 2023-11-08T16:28:40.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.518+0000 7fb0c5ffb640 1 -- 172.21.15.93:0/808341985 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb0cc043aa0 con 0x7fb0d00cfd90 2023-11-08T16:28:40.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.518+0000 7fb0c5ffb640 1 -- 172.21.15.93:0/808341985 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7fb0cc03e070 con 0x7fb0d00cfd90 2023-11-08T16:28:40.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.519+0000 7fb0d60c3640 1 -- 172.21.15.93:0/808341985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0d0148b30 con 0x7fb0d00cfd90 2023-11-08T16:28:40.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.521+0000 7fb0c5ffb640 1 -- 172.21.15.93:0/808341985 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb0cc034070 con 0x7fb0d00cfd90 2023-11-08T16:28:40.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.598+0000 7fb0d60c3640 1 -- 172.21.15.93:0/808341985 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3", "id": 5} v 0) v1 -- 0x7fb0d0067800 con 0x7fb0d00cfd90 2023-11-08T16:28:40.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.604+0000 7fb0c5ffb640 1 -- 172.21.15.93:0/808341985 <== mon.1 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb0cc04d530 con 0x7fb0d00cfd90 2023-11-08T16:28:40.607 INFO:teuthology.orchestra.run.smithi093.stdout:5 2023-11-08T16:28:40.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.606+0000 7fb0d60c3640 1 -- 172.21.15.93:0/808341985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0d00cfd90 msgr2=0x7fb0d00c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.606+0000 7fb0d60c3640 1 --2- 172.21.15.93:0/808341985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0d00cfd90 0x7fb0d00c85d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0cc000c00 tx=0x7fb0cc004730 comp rx=0 tx=0).stop 2023-11-08T16:28:40.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.606+0000 7fb0d60c3640 1 -- 172.21.15.93:0/808341985 shutdown_connections 2023-11-08T16:28:40.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.606+0000 7fb0d60c3640 1 --2- 172.21.15.93:0/808341985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0d01491a0 0x7fb0d01478b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.606+0000 7fb0d60c3640 1 --2- 172.21.15.93:0/808341985 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0d015cf30 0x7fb0d00c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.606+0000 7fb0d60c3640 1 --2- 172.21.15.93:0/808341985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0d00cfd90 0x7fb0d00c85d0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.606+0000 7fb0d60c3640 1 -- 172.21.15.93:0/808341985 >> 172.21.15.93:0/808341985 conn(0x7fb0d006c040 msgr2=0x7fb0d015e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:40.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.606+0000 7fb0d60c3640 1 -- 172.21.15.93:0/808341985 shutdown_connections 2023-11-08T16:28:40.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.606+0000 7fb0d60c3640 1 -- 172.21.15.93:0/808341985 wait complete. 2023-11-08T16:28:40.612 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:40.611+0000 7fab3f21c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-08T16:28:40.624 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new d8989a20-6fea-4830-aa5f-5648e5867300 6 2023-11-08T16:28:40.633 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:40.629+0000 7f478b837040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-08T16:28:40.749 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:40.746+0000 7f478b837040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-08T16:28:40.770 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:40.768+0000 7fab3f21c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-08T16:28:40.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.812+0000 7f559a9e2640 1 Processor -- start 2023-11-08T16:28:40.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.812+0000 7f559a9e2640 1 -- start start 2023-11-08T16:28:40.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.814+0000 7f559a9e2640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f55940cf740 0x7f55940b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.814+0000 7f559a9e2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 0x7f55940d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.814+0000 7f559a9e2640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55940d38c0 con 0x7f55940cfb20 2023-11-08T16:28:40.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.814+0000 7f559a9e2640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55940c7900 con 0x7f55940cf740 2023-11-08T16:28:40.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.814+0000 7f559a9e2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55940ce4c0 con 0x7f55940b3560 2023-11-08T16:28:40.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.814+0000 7f55999e0640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f55940cf740 0x7f55940b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.814+0000 7f559a1e1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 0x7f55940d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.814+0000 7f55999e0640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f55940cf740 0x7f55940b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54866/0 (socket says 172.21.15.93:54866) 2023-11-08T16:28:40.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.814+0000 7f55999e0640 1 -- 172.21.15.93:0/568556212 learned_addr learned my addr 172.21.15.93:0/568556212 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:40.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.815+0000 7f559a1e1640 1 -- 172.21.15.93:0/568556212 >> v2:172.21.15.93:3300/0 conn(0x7f55940cf740 msgr2=0x7f55940b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:40.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.815+0000 7f559a1e1640 1 --2- 172.21.15.93:0/568556212 >> v2:172.21.15.93:3300/0 conn(0x7f55940cf740 0x7f55940b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.815+0000 7f559a1e1640 1 -- 172.21.15.93:0/568556212 >> v1:172.21.15.93:6789/0 conn(0x7f55940cfb20 legacy=0x7f55940b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.815+0000 7f559a1e1640 1 -- 172.21.15.93:0/568556212 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55940ce7a0 con 0x7f55940b3560 2023-11-08T16:28:40.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.815+0000 7f55999e0640 1 --2- 172.21.15.93:0/568556212 >> v2:172.21.15.93:3300/0 conn(0x7f55940cf740 0x7f55940b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:28:40.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.816+0000 7f559a1e1640 1 --2- 172.21.15.93:0/568556212 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 0x7f55940d0b80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f558c009d90 tx=0x7f558c02be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a352fa792c2b06b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:40.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.816+0000 7f55989de640 1 -- 172.21.15.93:0/568556212 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c02cbd0 con 0x7f55940b3560 2023-11-08T16:28:40.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.816+0000 7f55989de640 1 -- 172.21.15.93:0/568556212 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c02cd30 con 0x7f55940b3560 2023-11-08T16:28:40.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.816+0000 7f55989de640 1 -- 172.21.15.93:0/568556212 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c02cea0 con 0x7f55940b3560 2023-11-08T16:28:40.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.817+0000 7f559a9e2640 1 -- 172.21.15.93:0/568556212 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 msgr2=0x7f55940d0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.817+0000 7f559a9e2640 1 --2- 172.21.15.93:0/568556212 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 0x7f55940d0b80 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f558c009d90 tx=0x7f558c02be20 comp rx=0 tx=0).stop 2023-11-08T16:28:40.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.817+0000 7f559a9e2640 1 -- 172.21.15.93:0/568556212 shutdown_connections 2023-11-08T16:28:40.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.817+0000 7f559a9e2640 1 --2- 172.21.15.93:0/568556212 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 0x7f55940d0b80 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.817+0000 7f559a9e2640 1 --2- 172.21.15.93:0/568556212 >> v2:172.21.15.93:3300/0 conn(0x7f55940cf740 0x7f55940b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.817+0000 7f559a9e2640 1 -- 172.21.15.93:0/568556212 >> 172.21.15.93:0/568556212 conn(0x7f55940d2da0 msgr2=0x7f55940d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:40.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.817+0000 7f559a9e2640 1 -- 172.21.15.93:0/568556212 shutdown_connections 2023-11-08T16:28:40.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.817+0000 7f559a9e2640 1 -- 172.21.15.93:0/568556212 wait complete. 2023-11-08T16:28:40.824 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:40.817+0000 7fab3f21c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-08T16:28:40.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.819+0000 7f559a9e2640 1 Processor -- start 2023-11-08T16:28:40.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.819+0000 7f559a9e2640 1 -- start start 2023-11-08T16:28:40.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.820+0000 7f559a9e2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 0x7f55941f5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.820+0000 7f559a9e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55940cf740 0x7f55940af9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.820+0000 7f559a9e2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55940cfb20 0x7f55940aff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:40.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.820+0000 7f559a9e2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55940b6ec0 con 0x7f55940cf740 2023-11-08T16:28:40.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.820+0000 7f559a9e2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55940b71a0 con 0x7f55940cfb20 2023-11-08T16:28:40.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.820+0000 7f559a9e2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55940b7480 con 0x7f55940b3560 2023-11-08T16:28:40.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.820+0000 7f55999e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 0x7f55941f5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.820+0000 7f55999e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 0x7f55941f5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.93:43702/0 (socket says 172.21.15.93:43702) 2023-11-08T16:28:40.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.820+0000 7f55999e0640 1 -- 172.21.15.93:0/425619236 learned_addr learned my addr 172.21.15.93:0/425619236 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:40.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.820+0000 7f559a1e1640 1 --2- 172.21.15.93:0/425619236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55940cfb20 0x7f55940aff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.820+0000 7f55991df640 1 --2- 172.21.15.93:0/425619236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55940cf740 0x7f55940af9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:40.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f55999e0640 1 -- 172.21.15.93:0/425619236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55940cfb20 msgr2=0x7f55940aff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f55999e0640 1 --2- 172.21.15.93:0/425619236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55940cfb20 0x7f55940aff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f55999e0640 1 -- 172.21.15.93:0/425619236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55940cf740 msgr2=0x7f55940af9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f55999e0640 1 --2- 172.21.15.93:0/425619236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55940cf740 0x7f55940af9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f55999e0640 1 -- 172.21.15.93:0/425619236 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55900094d0 con 0x7f55940b3560 2023-11-08T16:28:40.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f559a1e1640 1 --2- 172.21.15.93:0/425619236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55940cfb20 0x7f55940aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:28:40.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f55999e0640 1 --2- 172.21.15.93:0/425619236 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 0x7f55941f5860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f559000e9d0 tx=0x7f559000ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:40.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f558a7fc640 1 -- 172.21.15.93:0/425619236 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559000ca60 con 0x7f55940b3560 2023-11-08T16:28:40.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f558a7fc640 1 -- 172.21.15.93:0/425619236 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559000cbc0 con 0x7f55940b3560 2023-11-08T16:28:40.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f558a7fc640 1 -- 172.21.15.93:0/425619236 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5590013470 con 0x7f55940b3560 2023-11-08T16:28:40.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f559a9e2640 1 -- 172.21.15.93:0/425619236 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f558c009a70 con 0x7f55940b3560 2023-11-08T16:28:40.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.821+0000 7f559a9e2640 1 -- 172.21.15.93:0/425619236 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55940b7a40 con 0x7f55940b3560 2023-11-08T16:28:40.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.822+0000 7f558a7fc640 1 -- 172.21.15.93:0/425619236 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5590005c90 con 0x7f55940b3560 2023-11-08T16:28:40.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.822+0000 7f558a7fc640 1 -- 172.21.15.93:0/425619236 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f5590013bc0 con 0x7f55940b3560 2023-11-08T16:28:40.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.822+0000 7f559a9e2640 1 -- 172.21.15.93:0/425619236 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5594067cb0 con 0x7f55940b3560 2023-11-08T16:28:40.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.825+0000 7f558a7fc640 1 -- 172.21.15.93:0/425619236 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f5590021060 con 0x7f55940b3560 2023-11-08T16:28:40.904 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.902+0000 7f559a9e2640 1 -- 172.21.15.93:0/425619236 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd new", "uuid": "d8989a20-6fea-4830-aa5f-5648e5867300", "id": 6} v 0) v1 -- 0x7f5594067800 con 0x7f55940b3560 2023-11-08T16:28:40.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.910+0000 7f558a7fc640 1 -- 172.21.15.93:0/425619236 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d8989a20-6fea-4830-aa5f-5648e5867300", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5590017070 con 0x7f55940b3560 2023-11-08T16:28:40.913 INFO:teuthology.orchestra.run.smithi093.stdout:6 2023-11-08T16:28:40.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.911+0000 7f559a9e2640 1 -- 172.21.15.93:0/425619236 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 msgr2=0x7f55941f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:40.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.911+0000 7f559a9e2640 1 --2- 172.21.15.93:0/425619236 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 0x7f55941f5860 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f559000e9d0 tx=0x7f559000ee90 comp rx=0 tx=0).stop 2023-11-08T16:28:40.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.912+0000 7f559a9e2640 1 -- 172.21.15.93:0/425619236 shutdown_connections 2023-11-08T16:28:40.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.912+0000 7f559a9e2640 1 --2- 172.21.15.93:0/425619236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55940cfb20 0x7f55940aff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.912+0000 7f559a9e2640 1 --2- 172.21.15.93:0/425619236 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55940cf740 0x7f55940af9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.912+0000 7f559a9e2640 1 --2- 172.21.15.93:0/425619236 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55940b3560 0x7f55941f5860 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:40.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.912+0000 7f559a9e2640 1 -- 172.21.15.93:0/425619236 >> 172.21.15.93:0/425619236 conn(0x7f55940d2da0 msgr2=0x7f5594160770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:40.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.912+0000 7f559a9e2640 1 -- 172.21.15.93:0/425619236 shutdown_connections 2023-11-08T16:28:40.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:40.912+0000 7f559a9e2640 1 -- 172.21.15.93:0/425619236 wait complete. 2023-11-08T16:28:40.916 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:40.913+0000 7f478b837040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-08T16:28:40.930 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --cluster ceph osd new 0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50 7 2023-11-08T16:28:40.969 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:40.966+0000 7f478b837040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-08T16:28:40.976 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:40.974+0000 7fab3f21c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-08T16:28:41.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.122+0000 7faca6b0a640 1 Processor -- start 2023-11-08T16:28:41.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.122+0000 7faca6b0a640 1 -- start start 2023-11-08T16:28:41.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.124+0000 7faca6b0a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7faca015a990 0x7faca015ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:41.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.124+0000 7faca6b0a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d1640 0x7faca00b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:41.127 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.124+0000 7faca6b0a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faca00cd250 con 0x7faca00d0aa0 2023-11-08T16:28:41.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.124+0000 7faca6b0a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faca00cd530 con 0x7faca015a990 2023-11-08T16:28:41.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.124+0000 7faca6b0a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca00cd810 con 0x7faca00d1640 2023-11-08T16:28:41.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.124+0000 7faca5b08640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7faca015a990 0x7faca015ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:41.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.124+0000 7faca5b08640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7faca015a990 0x7faca015ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54886/0 (socket says 172.21.15.93:54886) 2023-11-08T16:28:41.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.124+0000 7faca5b08640 1 -- 172.21.15.93:0/2189727331 learned_addr learned my addr 172.21.15.93:0/2189727331 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.124+0000 7faca5307640 1 --2- 172.21.15.93:0/2189727331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d1640 0x7faca00b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.125+0000 7faca5b08640 1 -- 172.21.15.93:0/2189727331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d1640 msgr2=0x7faca00b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.125+0000 7faca5b08640 1 --2- 172.21.15.93:0/2189727331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d1640 0x7faca00b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.125+0000 7faca5b08640 1 -- 172.21.15.93:0/2189727331 >> v1:172.21.15.93:6789/0 conn(0x7faca00d0aa0 legacy=0x7faca00af510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:41.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.125+0000 7faca5b08640 1 -- 172.21.15.93:0/2189727331 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faca00cdaf0 con 0x7faca015a990 2023-11-08T16:28:41.130 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.125+0000 7faca5307640 1 --2- 172.21.15.93:0/2189727331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d1640 0x7faca00b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:28:41.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.126+0000 7faca5b08640 1 --2- 172.21.15.93:0/2189727331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca015a990 0x7faca015ad70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009870 tx=0x7fac9c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62cfd8b76bfbf77f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:41.131 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.126+0000 7faca4b06640 1 -- 172.21.15.93:0/2189727331 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c030840 con 0x7faca015a990 2023-11-08T16:28:41.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.126+0000 7faca4b06640 1 -- 172.21.15.93:0/2189727331 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c0309a0 con 0x7faca015a990 2023-11-08T16:28:41.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.126+0000 7faca4b06640 1 -- 172.21.15.93:0/2189727331 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c030d30 con 0x7faca015a990 2023-11-08T16:28:41.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.127+0000 7faca6b0a640 1 -- 172.21.15.93:0/2189727331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca015a990 msgr2=0x7faca015ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:41.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.127+0000 7faca6b0a640 1 --2- 172.21.15.93:0/2189727331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca015a990 0x7faca015ad70 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009870 tx=0x7fac9c02b980 comp rx=0 tx=0).stop 2023-11-08T16:28:41.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.127+0000 7faca6b0a640 1 -- 172.21.15.93:0/2189727331 shutdown_connections 2023-11-08T16:28:41.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.127+0000 7faca6b0a640 1 --2- 172.21.15.93:0/2189727331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d1640 0x7faca00b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.127+0000 7faca6b0a640 1 --2- 172.21.15.93:0/2189727331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca015a990 0x7faca015ad70 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.127+0000 7faca6b0a640 1 -- 172.21.15.93:0/2189727331 >> 172.21.15.93:0/2189727331 conn(0x7faca006d9c0 msgr2=0x7faca006dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:41.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.127+0000 7faca6b0a640 1 -- 172.21.15.93:0/2189727331 shutdown_connections 2023-11-08T16:28:41.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.127+0000 7faca6b0a640 1 -- 172.21.15.93:0/2189727331 wait complete. 2023-11-08T16:28:41.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.129+0000 7faca6b0a640 1 Processor -- start 2023-11-08T16:28:41.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.129+0000 7faca6b0a640 1 -- start start 2023-11-08T16:28:41.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca6b0a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d0aa0 0x7faca00ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:41.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca6b0a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faca00d1640 0x7faca0159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:41.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca6b0a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca015a990 0x7faca0159580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:41.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca6b0a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca0145bf0 con 0x7faca00d1640 2023-11-08T16:28:41.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca6b0a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faca015d720 con 0x7faca015a990 2023-11-08T16:28:41.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca6b0a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faca015da00 con 0x7faca00d0aa0 2023-11-08T16:28:41.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca5b08640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d0aa0 0x7faca00ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:41.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca5b08640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d0aa0 0x7faca00ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.93:43722/0 (socket says 172.21.15.93:43722) 2023-11-08T16:28:41.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca6309640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca015a990 0x7faca0159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:41.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca5b08640 1 -- 172.21.15.93:0/365690523 learned_addr learned my addr 172.21.15.93:0/365690523 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-11-08T16:28:41.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca5307640 1 --2- 172.21.15.93:0/365690523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faca00d1640 0x7faca0159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:41.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca5b08640 1 -- 172.21.15.93:0/365690523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca015a990 msgr2=0x7faca0159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:41.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca5b08640 1 --2- 172.21.15.93:0/365690523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca015a990 0x7faca0159580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca5b08640 1 -- 172.21.15.93:0/365690523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faca00d1640 msgr2=0x7faca0159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca5b08640 1 --2- 172.21.15.93:0/365690523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faca00d1640 0x7faca0159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca5b08640 1 -- 172.21.15.93:0/365690523 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac90009400 con 0x7faca00d0aa0 2023-11-08T16:28:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.130+0000 7faca5307640 1 --2- 172.21.15.93:0/365690523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faca00d1640 0x7faca0159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:28:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.131+0000 7faca5b08640 1 --2- 172.21.15.93:0/365690523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d0aa0 0x7faca00ca4f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009870 tx=0x7fac9c004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.131+0000 7fac967fc640 1 -- 172.21.15.93:0/365690523 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c039c90 con 0x7faca00d0aa0 2023-11-08T16:28:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.131+0000 7fac967fc640 1 -- 172.21.15.93:0/365690523 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9c039e10 con 0x7faca00d0aa0 2023-11-08T16:28:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.131+0000 7fac967fc640 1 -- 172.21.15.93:0/365690523 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9c043640 con 0x7faca00d0aa0 2023-11-08T16:28:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.131+0000 7faca6b0a640 1 -- 172.21.15.93:0/365690523 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c0094d0 con 0x7faca00d0aa0 2023-11-08T16:28:41.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.131+0000 7faca6b0a640 1 -- 172.21.15.93:0/365690523 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faca015dfc0 con 0x7faca00d0aa0 2023-11-08T16:28:41.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.131+0000 7fac967fc640 1 -- 172.21.15.93:0/365690523 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fac9c043aa0 con 0x7faca00d0aa0 2023-11-08T16:28:41.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.132+0000 7fac967fc640 1 -- 172.21.15.93:0/365690523 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fac9c03e070 con 0x7faca00d0aa0 2023-11-08T16:28:41.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.132+0000 7faca6b0a640 1 -- 172.21.15.93:0/365690523 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faca00caa30 con 0x7faca00d0aa0 2023-11-08T16:28:41.143 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:41.134+0000 7f478b837040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-08T16:28:41.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.138+0000 7fac967fc640 1 -- 172.21.15.93:0/365690523 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fac9c034070 con 0x7faca00d0aa0 2023-11-08T16:28:41.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.215+0000 7faca6b0a640 1 -- 172.21.15.93:0/365690523 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50", "id": 7} v 0) v1 -- 0x7faca015e150 con 0x7faca00d0aa0 2023-11-08T16:28:41.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.225+0000 7fac967fc640 1 -- 172.21.15.93:0/365690523 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fac9c04d460 con 0x7faca00d0aa0 2023-11-08T16:28:41.228 INFO:teuthology.orchestra.run.smithi093.stdout:7 2023-11-08T16:28:41.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.227+0000 7faca6b0a640 1 -- 172.21.15.93:0/365690523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d0aa0 msgr2=0x7faca00ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:41.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.227+0000 7faca6b0a640 1 --2- 172.21.15.93:0/365690523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d0aa0 0x7faca00ca4f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fac9c009870 tx=0x7fac9c004730 comp rx=0 tx=0).stop 2023-11-08T16:28:41.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.227+0000 7faca6b0a640 1 -- 172.21.15.93:0/365690523 shutdown_connections 2023-11-08T16:28:41.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.227+0000 7faca6b0a640 1 --2- 172.21.15.93:0/365690523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faca015a990 0x7faca0159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.227+0000 7faca6b0a640 1 --2- 172.21.15.93:0/365690523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faca00d1640 0x7faca0159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.227+0000 7faca6b0a640 1 --2- 172.21.15.93:0/365690523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faca00d0aa0 0x7faca00ca4f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.227+0000 7faca6b0a640 1 -- 172.21.15.93:0/365690523 >> 172.21.15.93:0/365690523 conn(0x7faca006d9c0 msgr2=0x7faca01588d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:41.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.227+0000 7faca6b0a640 1 -- 172.21.15.93:0/365690523 shutdown_connections 2023-11-08T16:28:41.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:28:41.227+0000 7faca6b0a640 1 -- 172.21.15.93:0/365690523 wait complete. 2023-11-08T16:28:41.251 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-08T16:28:41.252 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-08T16:28:41.261 INFO:tasks.ceph.osd.0:Started 2023-11-08T16:28:41.261 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-08T16:28:41.262 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-08T16:28:41.270 INFO:tasks.ceph.osd.1:Started 2023-11-08T16:28:41.271 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-08T16:28:41.271 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-08T16:28:41.277 INFO:tasks.ceph.osd.2:Started 2023-11-08T16:28:41.277 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-08T16:28:41.278 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-08T16:28:41.285 INFO:tasks.ceph.osd.3:Started 2023-11-08T16:28:41.285 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-08T16:28:41.286 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-08T16:28:41.299 INFO:tasks.ceph.osd.4:Started 2023-11-08T16:28:41.300 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-08T16:28:41.300 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-08T16:28:41.308 INFO:tasks.ceph.osd.5:Started 2023-11-08T16:28:41.308 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-08T16:28:41.309 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-08T16:28:41.315 INFO:tasks.ceph.osd.6:Started 2023-11-08T16:28:41.315 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-08T16:28:41.316 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-08T16:28:41.320 INFO:tasks.ceph.osd.7:Started 2023-11-08T16:28:41.321 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:28:41.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.540+0000 7fa5d0832640 1 Processor -- start 2023-11-08T16:28:41.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.540+0000 7fa5d0832640 1 -- start start 2023-11-08T16:28:41.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.541+0000 7fa5d0832640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa5cc0ca480 0x7fa5cc15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:41.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.541+0000 7fa5d0832640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa5cc15ea60 0x7fa5cc15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:41.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.541+0000 7fa5d0832640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5cc06b630 con 0x7fa5cc0ade50 2023-11-08T16:28:41.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.541+0000 7fa5d0832640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc0b1860 con 0x7fa5cc15ea60 2023-11-08T16:28:41.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.541+0000 7fa5d0832640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5cc0ca910 con 0x7fa5cc0ca480 2023-11-08T16:28:41.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.541+0000 7fa5cad9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa5cc15ea60 0x7fa5cc15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:41.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.541+0000 7fa5cad9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa5cc15ea60 0x7fa5cc15eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40114/0 (socket says 172.21.15.81:40114) 2023-11-08T16:28:41.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.541+0000 7fa5cad9d640 1 -- 172.21.15.81:0/466911525 learned_addr learned my addr 172.21.15.81:0/466911525 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:41.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.541+0000 7fa5c9d9b640 1 --2- 172.21.15.81:0/466911525 >> v2:172.21.15.93:3300/0 conn(0x7fa5cc0ca480 0x7fa5cc15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:41.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.542+0000 7fa5cad9d640 1 -- 172.21.15.81:0/466911525 >> v2:172.21.15.93:3300/0 conn(0x7fa5cc0ca480 msgr2=0x7fa5cc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:41.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.542+0000 7fa5cad9d640 1 --2- 172.21.15.81:0/466911525 >> v2:172.21.15.93:3300/0 conn(0x7fa5cc0ca480 0x7fa5cc15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.542+0000 7fa5cad9d640 1 -- 172.21.15.81:0/466911525 >> v1:172.21.15.93:6789/0 conn(0x7fa5cc0ade50 legacy=0x7fa5cc15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:41.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.542+0000 7fa5cad9d640 1 -- 172.21.15.81:0/466911525 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5cc0cabb0 con 0x7fa5cc15ea60 2023-11-08T16:28:41.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.543+0000 7fa5cad9d640 1 --2- 172.21.15.81:0/466911525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc15ea60 0x7fa5cc15eef0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc073d00 tx=0x7fa5c402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2428dcab5cbe001 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:41.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.543+0000 7fa5c959a640 1 -- 172.21.15.81:0/466911525 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c402ca80 con 0x7fa5cc15ea60 2023-11-08T16:28:41.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.544+0000 7fa5c959a640 1 -- 172.21.15.81:0/466911525 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c402cbe0 con 0x7fa5cc15ea60 2023-11-08T16:28:41.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.544+0000 7fa5c959a640 1 -- 172.21.15.81:0/466911525 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c40343f0 con 0x7fa5cc15ea60 2023-11-08T16:28:41.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.544+0000 7fa5d0832640 1 -- 172.21.15.81:0/466911525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc15ea60 msgr2=0x7fa5cc15eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:41.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.544+0000 7fa5d0832640 1 --2- 172.21.15.81:0/466911525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc15ea60 0x7fa5cc15eef0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc073d00 tx=0x7fa5c402bee0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.544+0000 7fa5d0832640 1 -- 172.21.15.81:0/466911525 shutdown_connections 2023-11-08T16:28:41.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.544+0000 7fa5d0832640 1 --2- 172.21.15.81:0/466911525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc15ea60 0x7fa5cc15eef0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.544+0000 7fa5d0832640 1 --2- 172.21.15.81:0/466911525 >> v2:172.21.15.93:3300/0 conn(0x7fa5cc0ca480 0x7fa5cc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.544+0000 7fa5d0832640 1 -- 172.21.15.81:0/466911525 >> 172.21.15.81:0/466911525 conn(0x7fa5cc0cb490 msgr2=0x7fa5cc0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:41.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.544+0000 7fa5d0832640 1 -- 172.21.15.81:0/466911525 shutdown_connections 2023-11-08T16:28:41.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.544+0000 7fa5d0832640 1 -- 172.21.15.81:0/466911525 wait complete. 2023-11-08T16:28:41.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.546+0000 7fa5d0832640 1 Processor -- start 2023-11-08T16:28:41.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.546+0000 7fa5d0832640 1 -- start start 2023-11-08T16:28:41.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5d0832640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc0ade50 0x7fa5cc155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:41.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5d0832640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5cc0ca480 0x7fa5cc157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:41.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5d0832640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5cc15ea60 0x7fa5cc06bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:41.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5d0832640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc155890 con 0x7fa5cc0ade50 2023-11-08T16:28:41.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5d0832640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc1584b0 con 0x7fa5cc0ca480 2023-11-08T16:28:41.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5d0832640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5cc154590 con 0x7fa5cc15ea60 2023-11-08T16:28:41.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5ca59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc0ade50 0x7fa5cc155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:41.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5ca59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc0ade50 0x7fa5cc155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40122/0 (socket says 172.21.15.81:40122) 2023-11-08T16:28:41.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5ca59c640 1 -- 172.21.15.81:0/3186331895 learned_addr learned my addr 172.21.15.81:0/3186331895 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:41.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5cad9d640 1 --2- 172.21.15.81:0/3186331895 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5cc15ea60 0x7fa5cc06bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:41.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5c9d9b640 1 --2- 172.21.15.81:0/3186331895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5cc0ca480 0x7fa5cc157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:41.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5ca59c640 1 -- 172.21.15.81:0/3186331895 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5cc15ea60 msgr2=0x7fa5cc06bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:41.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5ca59c640 1 --2- 172.21.15.81:0/3186331895 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5cc15ea60 0x7fa5cc06bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5ca59c640 1 -- 172.21.15.81:0/3186331895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5cc0ca480 msgr2=0x7fa5cc157b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:41.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5ca59c640 1 --2- 172.21.15.81:0/3186331895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5cc0ca480 0x7fa5cc157b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.547+0000 7fa5ca59c640 1 -- 172.21.15.81:0/3186331895 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c4009b30 con 0x7fa5cc0ade50 2023-11-08T16:28:41.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.548+0000 7fa5ca59c640 1 --2- 172.21.15.81:0/3186331895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc0ade50 0x7fa5cc155350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00b3f0 tx=0x7fa5bc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:41.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.548+0000 7fa5baffd640 1 -- 172.21.15.81:0/3186331895 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc011830 con 0x7fa5cc0ade50 2023-11-08T16:28:41.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.548+0000 7fa5baffd640 1 -- 172.21.15.81:0/3186331895 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc011990 con 0x7fa5cc0ade50 2023-11-08T16:28:41.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.548+0000 7fa5baffd640 1 -- 172.21.15.81:0/3186331895 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc011d40 con 0x7fa5cc0ade50 2023-11-08T16:28:41.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.548+0000 7fa5d0832640 1 -- 172.21.15.81:0/3186331895 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5cc154870 con 0x7fa5cc0ade50 2023-11-08T16:28:41.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.548+0000 7fa5d0832640 1 -- 172.21.15.81:0/3186331895 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5cc154e30 con 0x7fa5cc0ade50 2023-11-08T16:28:41.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.549+0000 7fa5baffd640 1 -- 172.21.15.81:0/3186331895 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa5bc004320 con 0x7fa5cc0ade50 2023-11-08T16:28:41.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.549+0000 7fa5baffd640 1 -- 172.21.15.81:0/3186331895 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fa5bc01aad0 con 0x7fa5cc0ade50 2023-11-08T16:28:41.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.549+0000 7fa5d0832640 1 -- 172.21.15.81:0/3186331895 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5cc04ece0 con 0x7fa5cc0ade50 2023-11-08T16:28:41.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.552+0000 7fa5baffd640 1 -- 172.21.15.81:0/3186331895 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fa5bc038b40 con 0x7fa5cc0ade50 2023-11-08T16:28:41.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.614+0000 7fa5d0832640 1 -- 172.21.15.81:0/3186331895 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5cc0cc1d0 con 0x7fa5cc0ade50 2023-11-08T16:28:41.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.615+0000 7fa5baffd640 1 -- 172.21.15.81:0/3186331895 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fa5bc027070 con 0x7fa5cc0ade50 2023-11-08T16:28:41.617 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:28:41.618 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":10,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:28:41.218568+0000","last_up_change":"0.000000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:28:41.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.616+0000 7fa5d0832640 1 -- 172.21.15.81:0/3186331895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc0ade50 msgr2=0x7fa5cc155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:41.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.616+0000 7fa5d0832640 1 --2- 172.21.15.81:0/3186331895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc0ade50 0x7fa5cc155350 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00b3f0 tx=0x7fa5bc00b8b0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.616+0000 7fa5d0832640 1 -- 172.21.15.81:0/3186331895 shutdown_connections 2023-11-08T16:28:41.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.616+0000 7fa5d0832640 1 --2- 172.21.15.81:0/3186331895 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5cc15ea60 0x7fa5cc06bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.616+0000 7fa5d0832640 1 --2- 172.21.15.81:0/3186331895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5cc0ca480 0x7fa5cc157b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.616+0000 7fa5d0832640 1 --2- 172.21.15.81:0/3186331895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5cc0ade50 0x7fa5cc155350 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:41.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.616+0000 7fa5d0832640 1 -- 172.21.15.81:0/3186331895 >> 172.21.15.81:0/3186331895 conn(0x7fa5cc0cb490 msgr2=0x7fa5cc153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:41.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.616+0000 7fa5d0832640 1 -- 172.21.15.81:0/3186331895 shutdown_connections 2023-11-08T16:28:41.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:41.616+0000 7fa5d0832640 1 -- 172.21.15.81:0/3186331895 wait complete. 2023-11-08T16:28:41.633 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-08T16:28:41.633 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-08T16:28:41.981 INFO:tasks.ceph.osd.1.smithi081.stderr:2023-11-08T16:28:41.979+0000 7fd948210500 -1 Falling back to public interface 2023-11-08T16:28:41.984 INFO:tasks.ceph.osd.0.smithi081.stderr:2023-11-08T16:28:41.983+0000 7f821a20e500 -1 Falling back to public interface 2023-11-08T16:28:41.993 INFO:tasks.ceph.osd.3.smithi081.stderr:2023-11-08T16:28:41.991+0000 7fd46b00c500 -1 Falling back to public interface 2023-11-08T16:28:42.006 INFO:tasks.ceph.osd.2.smithi081.stderr:2023-11-08T16:28:42.004+0000 7fa7d370e500 -1 Falling back to public interface 2023-11-08T16:28:42.019 INFO:tasks.ceph.osd.5.smithi093.stderr:2023-11-08T16:28:42.017+0000 7fa50bbde500 -1 Falling back to public interface 2023-11-08T16:28:42.019 INFO:tasks.ceph.osd.4.smithi093.stderr:2023-11-08T16:28:42.017+0000 7f9923b6e500 -1 Falling back to public interface 2023-11-08T16:28:42.020 INFO:tasks.ceph.osd.5.smithi093.stderr:2023-11-08T16:28:42.017+0000 7fa50ab85640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2023-11-08T16:28:42.020 INFO:tasks.ceph.osd.5.smithi093.stderr:2023-11-08T16:28:42.017+0000 7fa50ab85640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6801/0: (98) Address already in use 2023-11-08T16:28:42.030 INFO:tasks.ceph.osd.6.smithi093.stderr:2023-11-08T16:28:42.028+0000 7f1c38e14500 -1 Falling back to public interface 2023-11-08T16:28:42.034 INFO:tasks.ceph.osd.7.smithi093.stderr:2023-11-08T16:28:42.032+0000 7f34f6a1c500 -1 Falling back to public interface 2023-11-08T16:28:42.041 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:42.039+0000 7fab3f21c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-08T16:28:42.280 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:42.278+0000 7f478b837040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-08T16:28:42.347 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:42.346+0000 7fab3f21c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-08T16:28:42.406 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:42.405+0000 7fab3f21c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-08T16:28:42.439 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-08T16:28:42.523 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:42.521+0000 7fab3f21c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-08T16:28:42.590 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:42.588+0000 7f478b837040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-08T16:28:42.615 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.613+0000 7f5c952d4640 1 Processor -- start 2023-11-08T16:28:42.615 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.613+0000 7f5c952d4640 1 -- start start 2023-11-08T16:28:42.616 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.614+0000 7f5c952d4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c9015aeb0 0x7f5c900ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:42.616 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.614+0000 7f5c952d4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5c900c90f0 0x7f5c900c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:42.616 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.614+0000 7f5c952d4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c9006cc60 con 0x7f5c9006b2b0 2023-11-08T16:28:42.616 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.614+0000 7f5c952d4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c900cb3e0 con 0x7f5c900c90f0 2023-11-08T16:28:42.617 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.614+0000 7f5c952d4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c900cae20 con 0x7f5c9015aeb0 2023-11-08T16:28:42.618 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.615+0000 7f5c8e7fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5c900c90f0 0x7f5c900c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:42.618 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.615+0000 7f5c8e7fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5c900c90f0 0x7f5c900c9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40198/0 (socket says 172.21.15.81:40198) 2023-11-08T16:28:42.618 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.615+0000 7f5c8e7fc640 1 -- 172.21.15.81:0/691616950 learned_addr learned my addr 172.21.15.81:0/691616950 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:42.618 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.615+0000 7f5c8f7fe640 1 --1- 172.21.15.81:0/691616950 >> v1:172.21.15.81:6789/0 conn(0x7f5c9006b2b0 0x7f5c9006b700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:42142/0 (socket says 172.21.15.81:42142) 2023-11-08T16:28:42.619 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.615+0000 7f5c8effd640 1 --2- 172.21.15.81:0/691616950 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c9015aeb0 0x7f5c900ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:42.619 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.615+0000 7f5c8e7fc640 1 -- 172.21.15.81:0/691616950 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c9015aeb0 msgr2=0x7f5c900ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:42.619 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.615+0000 7f5c8e7fc640 1 --2- 172.21.15.81:0/691616950 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c9015aeb0 0x7f5c900ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:42.619 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.615+0000 7f5c8e7fc640 1 -- 172.21.15.81:0/691616950 >> v1:172.21.15.81:6789/0 conn(0x7f5c9006b2b0 legacy=0x7f5c9006b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:42.619 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.615+0000 7f5c8e7fc640 1 -- 172.21.15.81:0/691616950 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c900c9a90 con 0x7f5c900c90f0 2023-11-08T16:28:42.620 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.616+0000 7f5c8e7fc640 1 --2- 172.21.15.81:0/691616950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c900c90f0 0x7f5c900c9550 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009870 tx=0x7f5c8402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66901d8e64c9e67e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:42.620 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.616+0000 7f5c8dffb640 1 -- 172.21.15.81:0/691616950 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84030840 con 0x7f5c900c90f0 2023-11-08T16:28:42.620 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.616+0000 7f5c8dffb640 1 -- 172.21.15.81:0/691616950 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c840309a0 con 0x7f5c900c90f0 2023-11-08T16:28:42.620 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.616+0000 7f5c8dffb640 1 -- 172.21.15.81:0/691616950 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84030d30 con 0x7f5c900c90f0 2023-11-08T16:28:42.623 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.617+0000 7f5c952d4640 1 -- 172.21.15.81:0/691616950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c900c90f0 msgr2=0x7f5c900c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:42.623 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.617+0000 7f5c952d4640 1 --2- 172.21.15.81:0/691616950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c900c90f0 0x7f5c900c9550 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009870 tx=0x7f5c8402b980 comp rx=0 tx=0).stop 2023-11-08T16:28:42.623 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.617+0000 7f5c952d4640 1 -- 172.21.15.81:0/691616950 shutdown_connections 2023-11-08T16:28:42.623 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.617+0000 7f5c952d4640 1 --2- 172.21.15.81:0/691616950 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c900c90f0 0x7f5c900c9550 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:42.623 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.617+0000 7f5c952d4640 1 --2- 172.21.15.81:0/691616950 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c9015aeb0 0x7f5c900ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:42.624 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.617+0000 7f5c952d4640 1 -- 172.21.15.81:0/691616950 >> 172.21.15.81:0/691616950 conn(0x7f5c900ceaa0 msgr2=0x7f5c900cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:42.624 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.617+0000 7f5c952d4640 1 -- 172.21.15.81:0/691616950 shutdown_connections 2023-11-08T16:28:42.624 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.617+0000 7f5c952d4640 1 -- 172.21.15.81:0/691616950 wait complete. 2023-11-08T16:28:42.624 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.619+0000 7f5c952d4640 1 Processor -- start 2023-11-08T16:28:42.624 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.619+0000 7f5c952d4640 1 -- start start 2023-11-08T16:28:42.625 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c952d4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c9006b2b0 0x7f5c901ee000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:42.625 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c952d4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c900c90f0 0x7f5c9015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:42.625 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c952d4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c9015aeb0 0x7f5c901564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:42.625 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c952d4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c901ef030 con 0x7f5c9006b2b0 2023-11-08T16:28:42.625 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c952d4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c901ef4c0 con 0x7f5c9015aeb0 2023-11-08T16:28:42.626 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c952d4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c901ee700 con 0x7f5c900c90f0 2023-11-08T16:28:42.626 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c8effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c9006b2b0 0x7f5c901ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:42.626 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c8effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c9006b2b0 0x7f5c901ee000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40204/0 (socket says 172.21.15.81:40204) 2023-11-08T16:28:42.626 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c8effd640 1 -- 172.21.15.81:0/290112561 learned_addr learned my addr 172.21.15.81:0/290112561 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:42.627 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c8f7fe640 1 --2- 172.21.15.81:0/290112561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c9015aeb0 0x7f5c901564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:42.627 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c8e7fc640 1 --2- 172.21.15.81:0/290112561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c900c90f0 0x7f5c9015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:42.627 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c8effd640 1 -- 172.21.15.81:0/290112561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c900c90f0 msgr2=0x7f5c9015c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:42.628 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c8effd640 1 --2- 172.21.15.81:0/290112561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c900c90f0 0x7f5c9015c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:42.628 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c8effd640 1 -- 172.21.15.81:0/290112561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c9015aeb0 msgr2=0x7f5c901564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:42.628 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c8effd640 1 --2- 172.21.15.81:0/290112561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c9015aeb0 0x7f5c901564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:42.628 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.620+0000 7f5c8effd640 1 -- 172.21.15.81:0/290112561 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c840094d0 con 0x7f5c9006b2b0 2023-11-08T16:28:42.629 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.621+0000 7f5c8effd640 1 --2- 172.21.15.81:0/290112561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c9006b2b0 0x7f5c901ee000 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000e870 tx=0x7f5c8000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:42.629 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.621+0000 7f5c777fe640 1 -- 172.21.15.81:0/290112561 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c800095c0 con 0x7f5c9006b2b0 2023-11-08T16:28:42.629 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.621+0000 7f5c777fe640 1 -- 172.21.15.81:0/290112561 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c80017c60 con 0x7f5c9006b2b0 2023-11-08T16:28:42.629 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.621+0000 7f5c952d4640 1 -- 172.21.15.81:0/290112561 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c901ee9e0 con 0x7f5c9006b2b0 2023-11-08T16:28:42.629 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.621+0000 7f5c777fe640 1 -- 172.21.15.81:0/290112561 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8000c730 con 0x7f5c9006b2b0 2023-11-08T16:28:42.629 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.621+0000 7f5c952d4640 1 -- 172.21.15.81:0/290112561 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9015d070 con 0x7f5c9006b2b0 2023-11-08T16:28:42.630 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.622+0000 7f5c777fe640 1 -- 172.21.15.81:0/290112561 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 2) v1 ==== 44567+0+0 (secure 0 0 0) 0x7f5c8000c890 con 0x7f5c9006b2b0 2023-11-08T16:28:42.631 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.622+0000 7f5c952d4640 1 -- 172.21.15.81:0/290112561 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c900743f0 con 0x7f5c9006b2b0 2023-11-08T16:28:42.631 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.622+0000 7f5c777fe640 1 -- 172.21.15.81:0/290112561 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f5c80051b40 con 0x7f5c9006b2b0 2023-11-08T16:28:42.631 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.625+0000 7f5c777fe640 1 -- 172.21.15.81:0/290112561 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f5c8001ec00 con 0x7f5c9006b2b0 2023-11-08T16:28:42.654 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:42.652+0000 7f478b837040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-08T16:28:42.691 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.690+0000 7f5c952d4640 1 -- 172.21.15.81:0/290112561 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c90159150 con 0x7f5c9006b2b0 2023-11-08T16:28:42.693 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.691+0000 7f5c777fe640 1 -- 172.21.15.81:0/290112561 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f5c80024020 con 0x7f5c9006b2b0 2023-11-08T16:28:42.693 INFO:teuthology.misc.health.smithi081.stdout: 2023-11-08T16:28:42.693 INFO:teuthology.misc.health.smithi081.stdout:{"epoch":10,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:28:41.218568+0000","last_up_change":"0.000000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:28:42.694 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.692+0000 7f5c952d4640 1 -- 172.21.15.81:0/290112561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c9006b2b0 msgr2=0x7f5c901ee000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:42.694 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.692+0000 7f5c952d4640 1 --2- 172.21.15.81:0/290112561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c9006b2b0 0x7f5c901ee000 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000e870 tx=0x7f5c8000ed30 comp rx=0 tx=0).stop 2023-11-08T16:28:42.695 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.692+0000 7f5c952d4640 1 -- 172.21.15.81:0/290112561 shutdown_connections 2023-11-08T16:28:42.695 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.692+0000 7f5c952d4640 1 --2- 172.21.15.81:0/290112561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c9015aeb0 0x7f5c901564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:42.695 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.692+0000 7f5c952d4640 1 --2- 172.21.15.81:0/290112561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c900c90f0 0x7f5c9015c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:42.695 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.692+0000 7f5c952d4640 1 --2- 172.21.15.81:0/290112561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c9006b2b0 0x7f5c901ee000 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:42.695 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.692+0000 7f5c952d4640 1 -- 172.21.15.81:0/290112561 >> 172.21.15.81:0/290112561 conn(0x7f5c900ceaa0 msgr2=0x7f5c900b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:42.696 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.692+0000 7f5c952d4640 1 -- 172.21.15.81:0/290112561 shutdown_connections 2023-11-08T16:28:42.696 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:42.692+0000 7f5c952d4640 1 -- 172.21.15.81:0/290112561 wait complete. 2023-11-08T16:28:42.710 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-08T16:28:42.767 INFO:tasks.ceph.mgr.x.smithi093.stderr:2023-11-08T16:28:42.765+0000 7f478b837040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-08T16:28:43.570 INFO:tasks.ceph.mgr.y.smithi081.stderr:2023-11-08T16:28:43.568+0000 7faae02bc640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-08T16:28:45.565 INFO:tasks.ceph.osd.1.smithi081.stderr:2023-11-08T16:28:45.563+0000 7fd948210500 -1 osd.1 0 log_to_monitors true 2023-11-08T16:28:45.589 INFO:tasks.ceph.osd.2.smithi081.stderr:2023-11-08T16:28:45.587+0000 7fa7d370e500 -1 osd.2 0 log_to_monitors true 2023-11-08T16:28:45.600 INFO:tasks.ceph.osd.4.smithi093.stderr:2023-11-08T16:28:45.598+0000 7f9923b6e500 -1 osd.4 0 log_to_monitors true 2023-11-08T16:28:45.613 INFO:tasks.ceph.osd.6.smithi093.stderr:2023-11-08T16:28:45.611+0000 7f1c38e14500 -1 osd.6 0 log_to_monitors true 2023-11-08T16:28:45.623 INFO:tasks.ceph.osd.5.smithi093.stderr:2023-11-08T16:28:45.621+0000 7fa50bbde500 -1 osd.5 0 log_to_monitors true 2023-11-08T16:28:45.632 INFO:tasks.ceph.osd.7.smithi093.stderr:2023-11-08T16:28:45.630+0000 7f34f6a1c500 -1 osd.7 0 log_to_monitors true 2023-11-08T16:28:45.778 INFO:tasks.ceph.osd.3.smithi081.stderr:2023-11-08T16:28:45.776+0000 7fd46b00c500 -1 osd.3 0 log_to_monitors true 2023-11-08T16:28:45.785 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:28:45.790 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:28:45.825 INFO:tasks.ceph.osd.0.smithi081.stderr:2023-11-08T16:28:45.823+0000 7f821a20e500 -1 osd.0 0 log_to_monitors true 2023-11-08T16:28:49.517 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-08T16:28:49.752 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.750+0000 7fd858621640 1 Processor -- start 2023-11-08T16:28:49.752 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.750+0000 7fd858621640 1 -- start start 2023-11-08T16:28:49.753 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.752+0000 7fd858621640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 0x7fd8500c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:49.754 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.752+0000 7fd858621640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85006cc60 con 0x7fd85015aeb0 2023-11-08T16:28:49.755 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.752+0000 7fd858621640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8500c9a90 con 0x7fd8500b18f0 2023-11-08T16:28:49.755 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.752+0000 7fd858621640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500cb3e0 con 0x7fd8500c90f0 2023-11-08T16:28:49.755 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.752+0000 7fd8563bd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd85015aeb0 0x7fd8500ca4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:42144/0 (socket says 172.21.15.81:42144) 2023-11-08T16:28:49.755 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.752+0000 7fd8563bd640 1 -- 172.21.15.81:0/3404567027 learned_addr learned my addr 172.21.15.81:0/3404567027 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:28:49.755 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.753+0000 7fd855bbc640 1 --2- 172.21.15.81:0/3404567027 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 0x7fd8500c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:49.756 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.753+0000 7fd8553bb640 1 -- 172.21.15.81:0/3404567027 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 868245628 0 0) 0x7fd85006cc60 con 0x7fd85015aeb0 2023-11-08T16:28:49.756 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.753+0000 7fd8553bb640 1 -- 172.21.15.81:0/3404567027 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd83c003520 con 0x7fd85015aeb0 2023-11-08T16:28:49.757 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.753+0000 7fd855bbc640 1 -- 172.21.15.81:0/3404567027 >> v1:172.21.15.93:6789/0 conn(0x7fd8500b18f0 legacy=0x7fd8500b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:49.757 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.753+0000 7fd855bbc640 1 -- 172.21.15.81:0/3404567027 >> v1:172.21.15.81:6789/0 conn(0x7fd85015aeb0 legacy=0x7fd8500ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:49.757 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.754+0000 7fd855bbc640 1 -- 172.21.15.81:0/3404567027 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8500cabb0 con 0x7fd8500c90f0 2023-11-08T16:28:49.758 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.755+0000 7fd855bbc640 1 --2- 172.21.15.81:0/3404567027 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 0x7fd8500c9550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd8400097d0 tx=0x7fd84002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=197ff6a15a324afe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:49.759 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.756+0000 7fd8553bb640 1 -- 172.21.15.81:0/3404567027 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840030800 con 0x7fd8500c90f0 2023-11-08T16:28:49.759 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.756+0000 7fd8553bb640 1 -- 172.21.15.81:0/3404567027 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd840030960 con 0x7fd8500c90f0 2023-11-08T16:28:49.760 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.756+0000 7fd8553bb640 1 -- 172.21.15.81:0/3404567027 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840030cf0 con 0x7fd8500c90f0 2023-11-08T16:28:49.760 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.757+0000 7fd858621640 1 -- 172.21.15.81:0/3404567027 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 msgr2=0x7fd8500c9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:49.760 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.757+0000 7fd858621640 1 --2- 172.21.15.81:0/3404567027 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 0x7fd8500c9550 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd8400097d0 tx=0x7fd84002b940 comp rx=0 tx=0).stop 2023-11-08T16:28:49.761 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.758+0000 7fd858621640 1 -- 172.21.15.81:0/3404567027 shutdown_connections 2023-11-08T16:28:49.761 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.758+0000 7fd858621640 1 --2- 172.21.15.81:0/3404567027 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 0x7fd8500c9550 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:49.761 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.758+0000 7fd858621640 1 -- 172.21.15.81:0/3404567027 >> 172.21.15.81:0/3404567027 conn(0x7fd8500ceaa0 msgr2=0x7fd8500cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:49.761 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.759+0000 7fd858621640 1 -- 172.21.15.81:0/3404567027 shutdown_connections 2023-11-08T16:28:49.763 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.759+0000 7fd858621640 1 -- 172.21.15.81:0/3404567027 wait complete. 2023-11-08T16:28:49.763 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.762+0000 7fd858621640 1 Processor -- start 2023-11-08T16:28:49.766 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.763+0000 7fd858621640 1 -- start start 2023-11-08T16:28:49.766 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.764+0000 7fd858621640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8500b18f0 0x7fd850159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:49.766 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.764+0000 7fd858621640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 0x7fd8500cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:49.766 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.764+0000 7fd858621640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85015aeb0 0x7fd85015bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:49.767 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.764+0000 7fd858621640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85015b7a0 con 0x7fd85015aeb0 2023-11-08T16:28:49.767 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.764+0000 7fd858621640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85015a310 con 0x7fd8500b18f0 2023-11-08T16:28:49.767 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.764+0000 7fd858621640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500cdaf0 con 0x7fd8500c90f0 2023-11-08T16:28:49.768 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.765+0000 7fd855bbc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 0x7fd8500cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:49.769 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.765+0000 7fd855bbc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 0x7fd8500cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:44290/0 (socket says 172.21.15.81:44290) 2023-11-08T16:28:49.769 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.765+0000 7fd855bbc640 1 -- 172.21.15.81:0/652099569 learned_addr learned my addr 172.21.15.81:0/652099569 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:49.769 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.765+0000 7fd8563bd640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8500b18f0 0x7fd850159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:49.769 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.765+0000 7fd856bbe640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85015aeb0 0x7fd85015bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:49.770 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.765+0000 7fd855bbc640 1 -- 172.21.15.81:0/652099569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8500b18f0 msgr2=0x7fd850159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:49.770 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.765+0000 7fd855bbc640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8500b18f0 0x7fd850159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:49.770 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.765+0000 7fd855bbc640 1 -- 172.21.15.81:0/652099569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85015aeb0 msgr2=0x7fd85015bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:49.770 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.765+0000 7fd855bbc640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85015aeb0 0x7fd85015bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:49.770 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.765+0000 7fd855bbc640 1 -- 172.21.15.81:0/652099569 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd840009400 con 0x7fd8500c90f0 2023-11-08T16:28:49.771 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.766+0000 7fd855bbc640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 0x7fd8500cd5b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd8400098c0 tx=0x7fd840004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:49.771 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.766+0000 7fd846ffd640 1 -- 172.21.15.81:0/652099569 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84003e040 con 0x7fd8500c90f0 2023-11-08T16:28:49.771 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.766+0000 7fd846ffd640 1 -- 172.21.15.81:0/652099569 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd840039c80 con 0x7fd8500c90f0 2023-11-08T16:28:49.771 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.766+0000 7fd846ffd640 1 -- 172.21.15.81:0/652099569 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd840043440 con 0x7fd8500c90f0 2023-11-08T16:28:49.772 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.768+0000 7fd858621640 1 -- 172.21.15.81:0/652099569 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8500ce2e0 con 0x7fd8500c90f0 2023-11-08T16:28:49.772 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.768+0000 7fd858621640 1 -- 172.21.15.81:0/652099569 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8500ce000 con 0x7fd8500c90f0 2023-11-08T16:28:49.772 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.770+0000 7fd858621640 1 -- 172.21.15.81:0/652099569 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8500743f0 con 0x7fd8500c90f0 2023-11-08T16:28:49.774 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.772+0000 7fd846ffd640 1 -- 172.21.15.81:0/652099569 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7fd8400423e0 con 0x7fd8500c90f0 2023-11-08T16:28:49.775 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.773+0000 7fd846ffd640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83406ae30 0x7fd83406d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:49.775 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.774+0000 7fd846ffd640 1 -- 172.21.15.81:0/652099569 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fd8400aa950 con 0x7fd8500c90f0 2023-11-08T16:28:49.776 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.775+0000 7fd8563bd640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83406ae30 0x7fd83406d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:49.778 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.777+0000 7fd8563bd640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83406ae30 0x7fd83406d2f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0010e0 tx=0x7fd84c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:49.785 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:49.784+0000 7fd846ffd640 1 -- 172.21.15.81:0/652099569 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd840076a90 con 0x7fd8500c90f0 2023-11-08T16:28:49.809 INFO:tasks.ceph.osd.7.smithi093.stderr:2023-11-08T16:28:49.807+0000 7f34f1aca640 -1 osd.7 0 waiting for initial osdmap 2023-11-08T16:28:49.821 INFO:tasks.ceph.osd.7.smithi093.stderr:2023-11-08T16:28:49.819+0000 7f34ea103640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T16:28:49.862 INFO:tasks.ceph.osd.4.smithi093.stderr:2023-11-08T16:28:49.859+0000 7f9920b11640 -1 osd.4 0 waiting for initial osdmap 2023-11-08T16:28:49.880 INFO:tasks.ceph.osd.4.smithi093.stderr:2023-11-08T16:28:49.878+0000 7f9917912640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T16:28:49.893 INFO:tasks.ceph.osd.6.smithi093.stderr:2023-11-08T16:28:49.889+0000 7f1c33ebc640 -1 osd.6 0 waiting for initial osdmap 2023-11-08T16:28:49.897 INFO:tasks.ceph.osd.6.smithi093.stderr:2023-11-08T16:28:49.895+0000 7f1c2c4f5640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T16:28:49.959 INFO:tasks.ceph.osd.5.smithi093.stderr:2023-11-08T16:28:49.957+0000 7fa507349640 -1 osd.5 0 waiting for initial osdmap 2023-11-08T16:28:49.966 INFO:tasks.ceph.osd.5.smithi093.stderr:2023-11-08T16:28:49.964+0000 7fa4ff982640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T16:28:49.989 INFO:tasks.ceph.osd.1.smithi081.stderr:2023-11-08T16:28:49.987+0000 7fd94346c640 -1 osd.1 0 waiting for initial osdmap 2023-11-08T16:28:49.996 INFO:tasks.ceph.osd.1.smithi081.stderr:2023-11-08T16:28:49.994+0000 7fd93baa5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T16:28:50.002 INFO:tasks.ceph.osd.3.smithi081.stderr:2023-11-08T16:28:50.000+0000 7fd466777640 -1 osd.3 0 waiting for initial osdmap 2023-11-08T16:28:50.017 INFO:tasks.ceph.osd.3.smithi081.stderr:2023-11-08T16:28:50.015+0000 7fd45edb0640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T16:28:50.039 INFO:tasks.ceph.osd.0.smithi081.stderr:2023-11-08T16:28:50.038+0000 7f8215330640 -1 osd.0 0 waiting for initial osdmap 2023-11-08T16:28:50.043 INFO:tasks.ceph.osd.0.smithi081.stderr:2023-11-08T16:28:50.042+0000 7f820d969640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T16:28:50.052 INFO:tasks.ceph.osd.2.smithi081.stderr:2023-11-08T16:28:50.051+0000 7fa7cee79640 -1 osd.2 0 waiting for initial osdmap 2023-11-08T16:28:50.073 INFO:tasks.ceph.osd.2.smithi081.stderr:2023-11-08T16:28:50.071+0000 7fa7c74b2640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-08T16:28:50.108 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.106+0000 7fd858621640 1 -- 172.21.15.81:0/652099569 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd85006c730 con 0x7fd8500c90f0 2023-11-08T16:28:50.110 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.108+0000 7fd846ffd640 1 -- 172.21.15.81:0/652099569 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fd840048090 con 0x7fd8500c90f0 2023-11-08T16:28:50.110 INFO:teuthology.misc.health.smithi081.stdout: 2023-11-08T16:28:50.110 INFO:teuthology.misc.health.smithi081.stdout:{"epoch":12,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:28:47.569965+0000","last_up_change":"0.000000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:28:50.112 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.110+0000 7fd858621640 1 -- 172.21.15.81:0/652099569 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83406ae30 msgr2=0x7fd83406d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:50.113 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.110+0000 7fd858621640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83406ae30 0x7fd83406d2f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0010e0 tx=0x7fd84c009040 comp rx=0 tx=0).stop 2023-11-08T16:28:50.113 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.110+0000 7fd858621640 1 -- 172.21.15.81:0/652099569 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 msgr2=0x7fd8500cd5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:50.113 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.110+0000 7fd858621640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 0x7fd8500cd5b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd8400098c0 tx=0x7fd840004730 comp rx=0 tx=0).stop 2023-11-08T16:28:50.113 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.110+0000 7fd858621640 1 -- 172.21.15.81:0/652099569 shutdown_connections 2023-11-08T16:28:50.113 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.110+0000 7fd858621640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83406ae30 0x7fd83406d2f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:50.114 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.110+0000 7fd858621640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85015aeb0 0x7fd85015bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:50.114 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.110+0000 7fd858621640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8500c90f0 0x7fd8500cd5b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:50.114 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.110+0000 7fd858621640 1 --2- 172.21.15.81:0/652099569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8500b18f0 0x7fd850159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:50.114 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.110+0000 7fd858621640 1 -- 172.21.15.81:0/652099569 >> 172.21.15.81:0/652099569 conn(0x7fd8500ceaa0 msgr2=0x7fd8500b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:50.115 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.110+0000 7fd858621640 1 -- 172.21.15.81:0/652099569 shutdown_connections 2023-11-08T16:28:50.115 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:50.111+0000 7fd858621640 1 -- 172.21.15.81:0/652099569 wait complete. 2023-11-08T16:28:50.126 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-08T16:28:56.934 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-08T16:28:57.114 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.112+0000 7f081636c640 1 Processor -- start 2023-11-08T16:28:57.115 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.112+0000 7f081636c640 1 -- start start 2023-11-08T16:28:57.116 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.113+0000 7f081636c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f081015ac20 0x7f08100cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.116 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.113+0000 7f081636c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081006cd70 con 0x7f08100ad940 2023-11-08T16:28:57.117 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.113+0000 7f081636c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08100ca500 con 0x7f081006f7c0 2023-11-08T16:28:57.117 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.113+0000 7f081636c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08100ae130 con 0x7f081015ac20 2023-11-08T16:28:57.117 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.114+0000 7f081536a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f081015ac20 0x7f08100cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.117 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.114+0000 7f081536a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f081015ac20 0x7f08100cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42222/0 (socket says 172.21.15.81:42222) 2023-11-08T16:28:57.118 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.114+0000 7f081536a640 1 -- 172.21.15.81:0/3747621595 learned_addr learned my addr 172.21.15.81:0/3747621595 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:57.119 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.114+0000 7f081536a640 1 -- 172.21.15.81:0/3747621595 >> v1:172.21.15.93:6789/0 conn(0x7f081006f7c0 legacy=0x7f0810070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:57.119 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.114+0000 7f081536a640 1 -- 172.21.15.81:0/3747621595 >> v1:172.21.15.81:6789/0 conn(0x7f08100ad940 legacy=0x7f081006b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:57.120 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.114+0000 7f081536a640 1 -- 172.21.15.81:0/3747621595 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081006d030 con 0x7f081015ac20 2023-11-08T16:28:57.120 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.115+0000 7f081536a640 1 --2- 172.21.15.81:0/3747621595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f081015ac20 0x7f08100cbaa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f080c0098a0 tx=0x7f080c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e23b43d5b8ee907 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:57.120 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.115+0000 7f0807fff640 1 -- 172.21.15.81:0/3747621595 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c031640 con 0x7f081015ac20 2023-11-08T16:28:57.120 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.116+0000 7f0807fff640 1 -- 172.21.15.81:0/3747621595 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c0317a0 con 0x7f081015ac20 2023-11-08T16:28:57.121 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.116+0000 7f0807fff640 1 -- 172.21.15.81:0/3747621595 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c031b30 con 0x7f081015ac20 2023-11-08T16:28:57.121 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.116+0000 7f081636c640 1 -- 172.21.15.81:0/3747621595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f081015ac20 msgr2=0x7f08100cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:57.121 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.116+0000 7f081636c640 1 --2- 172.21.15.81:0/3747621595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f081015ac20 0x7f08100cbaa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f080c0098a0 tx=0x7f080c02b8f0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.121 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.116+0000 7f081636c640 1 -- 172.21.15.81:0/3747621595 shutdown_connections 2023-11-08T16:28:57.121 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.116+0000 7f081636c640 1 --2- 172.21.15.81:0/3747621595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f081015ac20 0x7f08100cbaa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.122 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.116+0000 7f081636c640 1 -- 172.21.15.81:0/3747621595 >> 172.21.15.81:0/3747621595 conn(0x7f08100b04d0 msgr2=0x7f08100b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:57.122 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.116+0000 7f081636c640 1 -- 172.21.15.81:0/3747621595 shutdown_connections 2023-11-08T16:28:57.122 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.116+0000 7f081636c640 1 -- 172.21.15.81:0/3747621595 wait complete. 2023-11-08T16:28:57.124 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.118+0000 7f081636c640 1 Processor -- start 2023-11-08T16:28:57.124 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.118+0000 7f081636c640 1 -- start start 2023-11-08T16:28:57.125 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.119+0000 7f081636c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f081006f7c0 0x7f0810153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.125 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.119+0000 7f081636c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08100ad940 0x7f08100c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.125 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.119+0000 7f081636c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f081015ac20 0x7f08100c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.125 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.119+0000 7f081636c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08100c6930 con 0x7f081015ac20 2023-11-08T16:28:57.126 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.119+0000 7f081636c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08100c6c10 con 0x7f081006f7c0 2023-11-08T16:28:57.126 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.119+0000 7f081636c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08100c6ef0 con 0x7f08100ad940 2023-11-08T16:28:57.126 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.119+0000 7f0815b6b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f081015ac20 0x7f08100c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.126 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.119+0000 7f0815b6b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f081015ac20 0x7f08100c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42230/0 (socket says 172.21.15.81:42230) 2023-11-08T16:28:57.127 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.119+0000 7f0815b6b640 1 -- 172.21.15.81:0/2526653700 learned_addr learned my addr 172.21.15.81:0/2526653700 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:57.127 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.119+0000 7f0814b69640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08100ad940 0x7f08100c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.127 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f081536a640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f081006f7c0 0x7f0810153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.127 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f0815b6b640 1 -- 172.21.15.81:0/2526653700 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08100ad940 msgr2=0x7f08100c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:57.127 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f0815b6b640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08100ad940 0x7f08100c8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.128 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f0815b6b640 1 -- 172.21.15.81:0/2526653700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f081006f7c0 msgr2=0x7f0810153300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:57.128 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f0815b6b640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f081006f7c0 0x7f0810153300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.128 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f0815b6b640 1 -- 172.21.15.81:0/2526653700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c0094d0 con 0x7f081015ac20 2023-11-08T16:28:57.128 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f081536a640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f081006f7c0 0x7f0810153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:28:57.128 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f0815b6b640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f081015ac20 0x7f08100c89c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f080800c7a0 tx=0x7f080800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:57.129 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f0805ffb640 1 -- 172.21.15.81:0/2526653700 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808003b70 con 0x7f081015ac20 2023-11-08T16:28:57.129 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f0805ffb640 1 -- 172.21.15.81:0/2526653700 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0808003cf0 con 0x7f081015ac20 2023-11-08T16:28:57.129 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f0805ffb640 1 -- 172.21.15.81:0/2526653700 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0808003ea0 con 0x7f081015ac20 2023-11-08T16:28:57.129 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f081636c640 1 -- 172.21.15.81:0/2526653700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08100c71d0 con 0x7f081015ac20 2023-11-08T16:28:57.129 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.120+0000 7f081636c640 1 -- 172.21.15.81:0/2526653700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08100c5bc0 con 0x7f081015ac20 2023-11-08T16:28:57.130 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.122+0000 7f081636c640 1 -- 172.21.15.81:0/2526653700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081004ece0 con 0x7f081015ac20 2023-11-08T16:28:57.131 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.122+0000 7f0805ffb640 1 -- 172.21.15.81:0/2526653700 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f080800aa90 con 0x7f081015ac20 2023-11-08T16:28:57.131 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.123+0000 7f0805ffb640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07f406aea0 0x7f07f406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.131 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.123+0000 7f0805ffb640 1 -- 172.21.15.81:0/2526653700 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0808088f80 con 0x7f081015ac20 2023-11-08T16:28:57.131 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.124+0000 7f081536a640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07f406aea0 0x7f07f406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.132 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.124+0000 7f081536a640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07f406aea0 0x7f07f406d360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f080c036000 tx=0x7f080c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:57.132 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.129+0000 7f0805ffb640 1 -- 172.21.15.81:0/2526653700 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08080bbca0 con 0x7f081015ac20 2023-11-08T16:28:57.325 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.323+0000 7f081636c640 1 -- 172.21.15.81:0/2526653700 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08101e7320 con 0x7f081015ac20 2023-11-08T16:28:57.327 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.325+0000 7f0805ffb640 1 -- 172.21.15.81:0/2526653700 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12790 (secure 0 0 0) 0x7f0808054410 con 0x7f081015ac20 2023-11-08T16:28:57.328 INFO:teuthology.misc.health.smithi081.stdout: 2023-11-08T16:28:57.328 INFO:teuthology.misc.health.smithi081.stdout:{"epoch":17,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:28:54.878405+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:28:57.331 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.328+0000 7f081636c640 1 -- 172.21.15.81:0/2526653700 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07f406aea0 msgr2=0x7f07f406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:57.332 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.328+0000 7f081636c640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07f406aea0 0x7f07f406d360 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f080c036000 tx=0x7f080c03c040 comp rx=0 tx=0).stop 2023-11-08T16:28:57.332 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.328+0000 7f081636c640 1 -- 172.21.15.81:0/2526653700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f081015ac20 msgr2=0x7f08100c89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:57.332 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.328+0000 7f081636c640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f081015ac20 0x7f08100c89c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f080800c7a0 tx=0x7f080800cc60 comp rx=0 tx=0).stop 2023-11-08T16:28:57.332 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.329+0000 7f081636c640 1 -- 172.21.15.81:0/2526653700 shutdown_connections 2023-11-08T16:28:57.333 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.329+0000 7f081636c640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07f406aea0 0x7f07f406d360 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.333 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.329+0000 7f081636c640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f081015ac20 0x7f08100c89c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.333 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.329+0000 7f081636c640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08100ad940 0x7f08100c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.333 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.329+0000 7f081636c640 1 --2- 172.21.15.81:0/2526653700 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f081006f7c0 0x7f0810153300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.333 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.329+0000 7f081636c640 1 -- 172.21.15.81:0/2526653700 >> 172.21.15.81:0/2526653700 conn(0x7f08100b04d0 msgr2=0x7f08100af5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:57.334 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.329+0000 7f081636c640 1 -- 172.21.15.81:0/2526653700 shutdown_connections 2023-11-08T16:28:57.334 INFO:teuthology.misc.health.smithi081.stderr:2023-11-08T16:28:57.329+0000 7f081636c640 1 -- 172.21.15.81:0/2526653700 wait complete. 2023-11-08T16:28:57.346 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-08T16:28:57.346 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-08T16:28:57.347 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T16:28:57.347 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T16:28:57.352 INFO:tasks.ceph.mds.a:Started 2023-11-08T16:28:57.352 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T16:28:57.352 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T16:28:57.357 INFO:tasks.ceph.mds.c:Started 2023-11-08T16:28:57.357 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T16:28:57.358 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T16:28:57.366 INFO:tasks.ceph.mds.b:Started 2023-11-08T16:28:57.366 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T16:28:57.367 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T16:28:57.375 INFO:tasks.ceph.mds.d:Started 2023-11-08T16:28:57.375 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-08T16:28:57.376 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:28:57.506 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:28:57.511 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:28:57.526 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:28:57.527 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2023-11-08T16:28:57.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.563+0000 7f32c1c26640 1 Processor -- start 2023-11-08T16:28:57.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.564+0000 7f32c1c26640 1 -- start start 2023-11-08T16:28:57.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.565+0000 7f32c1c26640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f32bc0710d0 0x7f32bc0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.565+0000 7f32c1c26640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc0cdc20 con 0x7f32bc0cbad0 2023-11-08T16:28:57.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.565+0000 7f32c1c26640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32bc0cbeb0 con 0x7f32bc1616f0 2023-11-08T16:28:57.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.565+0000 7f32c1c26640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32bc06a0b0 con 0x7f32bc0710d0 2023-11-08T16:28:57.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.565+0000 7f32b2ffd640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f32bc1616f0 0x7f32bc0709c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:37484/0 (socket says 172.21.15.81:37484) 2023-11-08T16:28:57.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.565+0000 7f32b2ffd640 1 -- 172.21.15.81:0/1101135601 learned_addr learned my addr 172.21.15.81:0/1101135601 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:28:57.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.565+0000 7f32bbfff640 1 --2- 172.21.15.81:0/1101135601 >> v2:172.21.15.81:3300/0 conn(0x7f32bc0710d0 0x7f32bc0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448611055 0 0) 0x7f32bc0cbeb0 con 0x7f32bc1616f0 2023-11-08T16:28:57.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a8003520 con 0x7f32bc1616f0 2023-11-08T16:28:57.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 <== mon.1 v1:172.21.15.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 120513937 0 0) 0x7f32a8003520 con 0x7f32bc1616f0 2023-11-08T16:28:57.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 >> v2:172.21.15.81:3300/0 conn(0x7f32bc0710d0 msgr2=0x7f32bc0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:57.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 --2- 172.21.15.81:0/1101135601 >> v2:172.21.15.81:3300/0 conn(0x7f32bc0710d0 0x7f32bc0cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 >> v1:172.21.15.81:6789/0 conn(0x7f32bc0cbad0 legacy=0x7f32bc06a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:57.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 --> v1:172.21.15.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc161b10 con 0x7f32bc1616f0 2023-11-08T16:28:57.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 <== mon.1 v1:172.21.15.93:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f32a4001670 con 0x7f32bc1616f0 2023-11-08T16:28:57.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 >> v1:172.21.15.93:6789/0 conn(0x7f32bc1616f0 legacy=0x7f32bc0709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:57.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 --2- 172.21.15.81:0/1101135601 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32a8003aa0 0x7f32a8003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 --2- 172.21.15.81:0/1101135601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32a80047c0 0x7f32a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 --2- 172.21.15.81:0/1101135601 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32a8005380 0x7f32a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc06a0b0 con 0x7f32a80047c0 2023-11-08T16:28:57.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a800c1f0 con 0x7f32a8003aa0 2023-11-08T16:28:57.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.566+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a800c3b0 con 0x7f32a8005380 2023-11-08T16:28:57.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.567+0000 7f32bb7fe640 1 --2- 172.21.15.81:0/1101135601 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32a8003aa0 0x7f32a8003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.567+0000 7f32b2ffd640 1 --2- 172.21.15.81:0/1101135601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32a80047c0 0x7f32a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.567+0000 7f32bbfff640 1 --2- 172.21.15.81:0/1101135601 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32a8005380 0x7f32a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.567+0000 7f32bb7fe640 1 -- 172.21.15.81:0/1101135601 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32a8005380 msgr2=0x7f32a800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:57.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.567+0000 7f32bb7fe640 1 --2- 172.21.15.81:0/1101135601 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32a8005380 0x7f32a800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.567+0000 7f32bb7fe640 1 -- 172.21.15.81:0/1101135601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32a80047c0 msgr2=0x7f32a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:57.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.567+0000 7f32bb7fe640 1 --2- 172.21.15.81:0/1101135601 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32a80047c0 0x7f32a8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.567+0000 7f32bb7fe640 1 -- 172.21.15.81:0/1101135601 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a8003520 con 0x7f32a8003aa0 2023-11-08T16:28:57.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.567+0000 7f32bbfff640 1 -- 172.21.15.81:0/1101135601 reap_dead start 2023-11-08T16:28:57.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.568+0000 7f32bb7fe640 1 --2- 172.21.15.81:0/1101135601 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32a8003aa0 0x7f32a8003ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0015b0 tx=0x7f32a80057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8698f5952e20d51 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:57.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.568+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac0052d0 con 0x7f32a8003aa0 2023-11-08T16:28:57.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.568+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac02e050 con 0x7f32a8003aa0 2023-11-08T16:28:57.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.568+0000 7f32baffd640 1 -- 172.21.15.81:0/1101135601 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac003830 con 0x7f32a8003aa0 2023-11-08T16:28:57.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.569+0000 7f32c1c26640 1 -- 172.21.15.81:0/1101135601 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32a8003aa0 msgr2=0x7f32a8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:57.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.569+0000 7f32c1c26640 1 --2- 172.21.15.81:0/1101135601 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32a8003aa0 0x7f32a8003ec0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0015b0 tx=0x7f32a80057f0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.569+0000 7f32c1c26640 1 -- 172.21.15.81:0/1101135601 shutdown_connections 2023-11-08T16:28:57.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.569+0000 7f32c1c26640 1 --2- 172.21.15.81:0/1101135601 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32a8003aa0 0x7f32a8003ec0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.569+0000 7f32c1c26640 1 -- 172.21.15.81:0/1101135601 >> 172.21.15.81:0/1101135601 conn(0x7f32bc0b1600 msgr2=0x7f32bc0b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:57.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.569+0000 7f32c1c26640 1 -- 172.21.15.81:0/1101135601 shutdown_connections 2023-11-08T16:28:57.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.569+0000 7f32c1c26640 1 -- 172.21.15.81:0/1101135601 wait complete. 2023-11-08T16:28:57.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.571+0000 7f32c1c26640 1 Processor -- start 2023-11-08T16:28:57.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.571+0000 7f32c1c26640 1 -- start start 2023-11-08T16:28:57.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.572+0000 7f32c1c26640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32a8003aa0 0x7f32bc158230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.572+0000 7f32c1c26640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32bc0b4e60 0x7f32bc0c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.572+0000 7f32c1c26640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32bc0b52e0 0x7f32bc0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.572+0000 7f32c1c26640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0b3dc0 con 0x7f32bc0b4e60 2023-11-08T16:28:57.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.572+0000 7f32c1c26640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0b40a0 con 0x7f32a8003aa0 2023-11-08T16:28:57.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.572+0000 7f32c1c26640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0b4380 con 0x7f32bc0b52e0 2023-11-08T16:28:57.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.573+0000 7f32b2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32bc0b4e60 0x7f32bc0c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.573+0000 7f32b2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32bc0b4e60 0x7f32bc0c8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42282/0 (socket says 172.21.15.81:42282) 2023-11-08T16:28:57.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.573+0000 7f32b2ffd640 1 -- 172.21.15.81:0/2234230436 learned_addr learned my addr 172.21.15.81:0/2234230436 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:57.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.573+0000 7f32bbfff640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32bc0b52e0 0x7f32bc0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.573+0000 7f32bb7fe640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32a8003aa0 0x7f32bc158230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.573+0000 7f32bbfff640 1 -- 172.21.15.81:0/2234230436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32a8003aa0 msgr2=0x7f32bc158230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:57.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.573+0000 7f32bbfff640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32a8003aa0 0x7f32bc158230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.573+0000 7f32bbfff640 1 -- 172.21.15.81:0/2234230436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32bc0b4e60 msgr2=0x7f32bc0c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:57.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.573+0000 7f32bbfff640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32bc0b4e60 0x7f32bc0c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:57.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.573+0000 7f32bbfff640 1 -- 172.21.15.81:0/2234230436 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc0b4660 con 0x7f32bc0b52e0 2023-11-08T16:28:57.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.574+0000 7f32bbfff640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32bc0b52e0 0x7f32bc0c92f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0cead0 tx=0x7f32b4007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.574+0000 7f32b8ff9640 1 -- 172.21.15.81:0/2234230436 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b40046a0 con 0x7f32bc0b52e0 2023-11-08T16:28:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.574+0000 7f32b8ff9640 1 -- 172.21.15.81:0/2234230436 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b4004800 con 0x7f32bc0b52e0 2023-11-08T16:28:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.574+0000 7f32b8ff9640 1 -- 172.21.15.81:0/2234230436 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b4015660 con 0x7f32bc0b52e0 2023-11-08T16:28:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.574+0000 7f32c1c26640 1 -- 172.21.15.81:0/2234230436 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32bc0b4940 con 0x7f32bc0b52e0 2023-11-08T16:28:57.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.574+0000 7f32c1c26640 1 -- 172.21.15.81:0/2234230436 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32bc0c79d0 con 0x7f32bc0b52e0 2023-11-08T16:28:57.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.575+0000 7f32c1c26640 1 -- 172.21.15.81:0/2234230436 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32bc0b4b30 con 0x7f32bc0b52e0 2023-11-08T16:28:57.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.576+0000 7f32b8ff9640 1 -- 172.21.15.81:0/2234230436 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32b40157c0 con 0x7f32bc0b52e0 2023-11-08T16:28:57.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.577+0000 7f32b8ff9640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f329c06ae70 0x7f329c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:57.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.577+0000 7f32b8ff9640 1 -- 172.21.15.81:0/2234230436 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f32b4089800 con 0x7f32bc0b52e0 2023-11-08T16:28:57.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.577+0000 7f32bb7fe640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f329c06ae70 0x7f329c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:57.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.578+0000 7f32bb7fe640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f329c06ae70 0x7f329c06d330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f32ac033000 tx=0x7f32ac009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:57.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.583+0000 7f32b8ff9640 1 -- 172.21.15.81:0/2234230436 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32b4054c10 con 0x7f32bc0b52e0 2023-11-08T16:28:57.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:57.776+0000 7f32c1c26640 1 -- 172.21.15.81:0/2234230436 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32bc0b44c0 con 0x7f32bc0b52e0 2023-11-08T16:28:58.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.172+0000 7f32b8ff9640 1 -- 172.21.15.81:0/2234230436 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12790 (secure 0 0 0) 0x7f32bc0b44c0 con 0x7f32bc0b52e0 2023-11-08T16:28:58.176 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:28:58.176 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":17,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:28:54.878405+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:28:58.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.175+0000 7f32c1c26640 1 -- 172.21.15.81:0/2234230436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f329c06ae70 msgr2=0x7f329c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.175+0000 7f32c1c26640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f329c06ae70 0x7f329c06d330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f32ac033000 tx=0x7f32ac009030 comp rx=0 tx=0).stop 2023-11-08T16:28:58.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.175+0000 7f32c1c26640 1 -- 172.21.15.81:0/2234230436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32bc0b52e0 msgr2=0x7f32bc0c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.175+0000 7f32c1c26640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32bc0b52e0 0x7f32bc0c92f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f32bc0cead0 tx=0x7f32b4007700 comp rx=0 tx=0).stop 2023-11-08T16:28:58.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.176+0000 7f32c1c26640 1 -- 172.21.15.81:0/2234230436 shutdown_connections 2023-11-08T16:28:58.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.176+0000 7f32c1c26640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f329c06ae70 0x7f329c06d330 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.176+0000 7f32c1c26640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32bc0b52e0 0x7f32bc0c92f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.176+0000 7f32c1c26640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32bc0b4e60 0x7f32bc0c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.176+0000 7f32c1c26640 1 --2- 172.21.15.81:0/2234230436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32a8003aa0 0x7f32bc158230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.176+0000 7f32c1c26640 1 -- 172.21.15.81:0/2234230436 >> 172.21.15.81:0/2234230436 conn(0x7f32bc0b1600 msgr2=0x7f32bc0ca0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:58.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.176+0000 7f32c1c26640 1 -- 172.21.15.81:0/2234230436 shutdown_connections 2023-11-08T16:28:58.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.176+0000 7f32c1c26640 1 -- 172.21.15.81:0/2234230436 wait complete. 2023-11-08T16:28:58.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:28:58.202 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:28:58.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.404+0000 7f91722ec640 1 Processor -- start 2023-11-08T16:28:58.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.405+0000 7f91722ec640 1 -- start start 2023-11-08T16:28:58.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.406+0000 7f91722ec640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c15f280 0x7f916c0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.406+0000 7f91722ec640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c0cda10 con 0x7f916c0ce0d0 2023-11-08T16:28:58.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.406+0000 7f91722ec640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f916c06fa30 con 0x7f916c15bfc0 2023-11-08T16:28:58.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.406+0000 7f91722ec640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0ce4b0 con 0x7f916c15f280 2023-11-08T16:28:58.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.406+0000 7f916bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f916c0ce0d0 0x7f916c15eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44450/0 (socket says 172.21.15.81:44450) 2023-11-08T16:28:58.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.406+0000 7f916bfff640 1 -- 172.21.15.81:0/4034340493 learned_addr learned my addr 172.21.15.81:0/4034340493 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:28:58.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f9170889640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c15f280 0x7f916c0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769174300 0 0) 0x7f916c0cda10 con 0x7f916c0ce0d0 2023-11-08T16:28:58.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9158003520 con 0x7f916c0ce0d0 2023-11-08T16:28:58.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1217612956 0 0) 0x7f9158003520 con 0x7f916c0ce0d0 2023-11-08T16:28:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c15f280 msgr2=0x7f916c0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c15f280 0x7f916c0caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 >> v1:172.21.15.93:6789/0 conn(0x7f916c15bfc0 legacy=0x7f916c1516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c06fca0 con 0x7f916c0ce0d0 2023-11-08T16:28:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f915c001670 con 0x7f916c0ce0d0 2023-11-08T16:28:58.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 >> v1:172.21.15.81:6789/0 conn(0x7f916c0ce0d0 legacy=0x7f916c15eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9158003a10 0x7f9158003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c15f280 0x7f91580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 0x7f9158005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0ce4b0 con 0x7f9158004d00 2023-11-08T16:28:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915800bb10 con 0x7f9158003a10 2023-11-08T16:28:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.407+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915800bcd0 con 0x7f916c15f280 2023-11-08T16:28:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.408+0000 7f9170889640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 0x7f9158005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.408+0000 7f916bfff640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9158003a10 0x7f9158003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.408+0000 7f9163fff640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c15f280 0x7f91580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.408+0000 7f9170889640 1 -- 172.21.15.81:0/4034340493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c15f280 msgr2=0x7f91580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.408+0000 7f9170889640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c15f280 0x7f91580047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.408+0000 7f9170889640 1 -- 172.21.15.81:0/4034340493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9158003a10 msgr2=0x7f9158003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.408+0000 7f9170889640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9158003a10 0x7f9158003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.408+0000 7f9170889640 1 -- 172.21.15.81:0/4034340493 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9158003520 con 0x7f9158004d00 2023-11-08T16:28:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.409+0000 7f9170889640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 0x7f9158005180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f916c06b180 tx=0x7f91580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6dbea25c4669e47 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.409+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916402cb90 con 0x7f9158004d00 2023-11-08T16:28:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.409+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916400f040 con 0x7f9158004d00 2023-11-08T16:28:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.409+0000 7f916b7fe640 1 -- 172.21.15.81:0/4034340493 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164034440 con 0x7f9158004d00 2023-11-08T16:28:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.410+0000 7f91722ec640 1 -- 172.21.15.81:0/4034340493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 msgr2=0x7f9158005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.410+0000 7f91722ec640 1 --2- 172.21.15.81:0/4034340493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 0x7f9158005180 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f916c06b180 tx=0x7f91580057f0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.410+0000 7f9170889640 1 -- 172.21.15.81:0/4034340493 reap_dead start 2023-11-08T16:28:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.410+0000 7f91722ec640 1 -- 172.21.15.81:0/4034340493 shutdown_connections 2023-11-08T16:28:58.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.410+0000 7f91722ec640 1 -- 172.21.15.81:0/4034340493 >> 172.21.15.81:0/4034340493 conn(0x7f916c0cb580 msgr2=0x7f916c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:58.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.410+0000 7f91722ec640 1 -- 172.21.15.81:0/4034340493 shutdown_connections 2023-11-08T16:28:58.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.410+0000 7f91722ec640 1 -- 172.21.15.81:0/4034340493 wait complete. 2023-11-08T16:28:58.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.411+0000 7f91722ec640 1 Processor -- start 2023-11-08T16:28:58.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.412+0000 7f91722ec640 1 -- start start 2023-11-08T16:28:58.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.412+0000 7f91722ec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 0x7f916c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.412+0000 7f91722ec640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f916c0c9d90 0x7f916c140dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f91722ec640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c0ca210 0x7f916c141310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f91722ec640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c6110 con 0x7f9158004d00 2023-11-08T16:28:58.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f91722ec640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c4550 con 0x7f916c0c9d90 2023-11-08T16:28:58.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f91722ec640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f916c0c5090 con 0x7f916c0ca210 2023-11-08T16:28:58.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f916bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 0x7f916c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f916bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 0x7f916c0c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42286/0 (socket says 172.21.15.81:42286) 2023-11-08T16:28:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f916bfff640 1 -- 172.21.15.81:0/4274502478 learned_addr learned my addr 172.21.15.81:0/4274502478 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f916bfff640 1 -- 172.21.15.81:0/4274502478 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c0ca210 msgr2=0x7f916c141310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f916bfff640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c0ca210 0x7f916c141310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f916bfff640 1 -- 172.21.15.81:0/4274502478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f916c0c9d90 msgr2=0x7f916c140dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f9163fff640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f916c0c9d90 0x7f916c140dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f916bfff640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f916c0c9d90 0x7f916c140dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f916bfff640 1 -- 172.21.15.81:0/4274502478 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f916c0c5520 con 0x7f9158004d00 2023-11-08T16:28:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.413+0000 7f916bfff640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 0x7f916c0c5bd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f915c00e780 tx=0x7f915c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.414+0000 7f91697fa640 1 -- 172.21.15.81:0/4274502478 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c00ee30 con 0x7f9158004d00 2023-11-08T16:28:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.414+0000 7f91697fa640 1 -- 172.21.15.81:0/4274502478 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915c00c8d0 con 0x7f9158004d00 2023-11-08T16:28:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.414+0000 7f91697fa640 1 -- 172.21.15.81:0/4274502478 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915c00cc80 con 0x7f9158004d00 2023-11-08T16:28:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.414+0000 7f91722ec640 1 -- 172.21.15.81:0/4274502478 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916c0c4760 con 0x7f9158004d00 2023-11-08T16:28:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.414+0000 7f91722ec640 1 -- 172.21.15.81:0/4274502478 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916c0c4d20 con 0x7f9158004d00 2023-11-08T16:28:58.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.415+0000 7f91722ec640 1 -- 172.21.15.81:0/4274502478 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916c06d830 con 0x7f9158004d00 2023-11-08T16:28:58.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.416+0000 7f91697fa640 1 -- 172.21.15.81:0/4274502478 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f915c01d460 con 0x7f9158004d00 2023-11-08T16:28:58.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.417+0000 7f91697fa640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f914c06ae70 0x7f914c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.417+0000 7f91697fa640 1 -- 172.21.15.81:0/4274502478 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f915c089170 con 0x7f9158004d00 2023-11-08T16:28:58.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.417+0000 7f9163fff640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f914c06ae70 0x7f914c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.418+0000 7f9163fff640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f914c06ae70 0x7f914c06d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f916c13ec90 tx=0x7f9154009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:58.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.422+0000 7f91697fa640 1 -- 172.21.15.81:0/4274502478 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f915c054600 con 0x7f9158004d00 2023-11-08T16:28:58.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.598+0000 7f91722ec640 1 -- 172.21.15.81:0/4274502478 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f916c04ece0 con 0x7f9158004d00 2023-11-08T16:28:58.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.599+0000 7f91697fa640 1 -- 172.21.15.81:0/4274502478 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f915c0014e0 con 0x7f9158004d00 2023-11-08T16:28:58.602 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:28:58.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.601+0000 7f91722ec640 1 -- 172.21.15.81:0/4274502478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f914c06ae70 msgr2=0x7f914c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.601+0000 7f91722ec640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f914c06ae70 0x7f914c06d330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f916c13ec90 tx=0x7f9154009040 comp rx=0 tx=0).stop 2023-11-08T16:28:58.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.601+0000 7f91722ec640 1 -- 172.21.15.81:0/4274502478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 msgr2=0x7f916c0c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.601+0000 7f91722ec640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 0x7f916c0c5bd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f915c00e780 tx=0x7f915c00ec40 comp rx=0 tx=0).stop 2023-11-08T16:28:58.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.601+0000 7f91722ec640 1 -- 172.21.15.81:0/4274502478 shutdown_connections 2023-11-08T16:28:58.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.601+0000 7f91722ec640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f914c06ae70 0x7f914c06d330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.601+0000 7f91722ec640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f916c0ca210 0x7f916c141310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.601+0000 7f91722ec640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f916c0c9d90 0x7f916c140dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.601+0000 7f91722ec640 1 --2- 172.21.15.81:0/4274502478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9158004d00 0x7f916c0c5bd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.601+0000 7f91722ec640 1 -- 172.21.15.81:0/4274502478 >> 172.21.15.81:0/4274502478 conn(0x7f916c0cb580 msgr2=0x7f916c15d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:58.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.602+0000 7f91722ec640 1 -- 172.21.15.81:0/4274502478 shutdown_connections 2023-11-08T16:28:58.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.602+0000 7f91722ec640 1 -- 172.21.15.81:0/4274502478 wait complete. 2023-11-08T16:28:58.622 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:28:58.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.827+0000 7f8266d27640 1 Processor -- start 2023-11-08T16:28:58.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.827+0000 7f8266d27640 1 -- start start 2023-11-08T16:28:58.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.828+0000 7f8266d27640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f826015aac0 0x7f82600711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.828+0000 7f8266d27640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f82600cf420 0x7f826015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.828+0000 7f8266d27640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8260070470 con 0x7f82600716e0 2023-11-08T16:28:58.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.828+0000 7f8266d27640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826006f650 con 0x7f826015aac0 2023-11-08T16:28:58.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.828+0000 7f8266d27640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826006d580 con 0x7f82600cf420 2023-11-08T16:28:58.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.828+0000 7f8265d25640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f826015aac0 0x7f82600711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.828+0000 7f8265d25640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f826015aac0 0x7f82600711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42290/0 (socket says 172.21.15.81:42290) 2023-11-08T16:28:58.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.828+0000 7f8265d25640 1 -- 172.21.15.81:0/1406740158 learned_addr learned my addr 172.21.15.81:0/1406740158 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:58.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.829+0000 7f8266526640 1 --2- 172.21.15.81:0/1406740158 >> v2:172.21.15.93:3300/0 conn(0x7f82600cf420 0x7f826015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.829+0000 7f8265d25640 1 -- 172.21.15.81:0/1406740158 >> v2:172.21.15.93:3300/0 conn(0x7f82600cf420 msgr2=0x7f826015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:58.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.829+0000 7f8265d25640 1 --2- 172.21.15.81:0/1406740158 >> v2:172.21.15.93:3300/0 conn(0x7f82600cf420 0x7f826015aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.829+0000 7f8265d25640 1 -- 172.21.15.81:0/1406740158 >> v1:172.21.15.81:6789/0 conn(0x7f82600716e0 legacy=0x7f82600ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.829+0000 7f8265d25640 1 -- 172.21.15.81:0/1406740158 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826006ca00 con 0x7f826015aac0 2023-11-08T16:28:58.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.829+0000 7f8264d23640 1 -- 172.21.15.81:0/1406740158 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1192905523 0 0) 0x7f8260070470 con 0x7f82600716e0 2023-11-08T16:28:58.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.830+0000 7f8265d25640 1 --2- 172.21.15.81:0/1406740158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f826015aac0 0x7f82600711a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82600703e0 tx=0x7f825c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959aafa9e69d2a24 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:58.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.830+0000 7f8264d23640 1 -- 172.21.15.81:0/1406740158 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c030860 con 0x7f826015aac0 2023-11-08T16:28:58.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.830+0000 7f8264d23640 1 -- 172.21.15.81:0/1406740158 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c0309c0 con 0x7f826015aac0 2023-11-08T16:28:58.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.830+0000 7f8264d23640 1 -- 172.21.15.81:0/1406740158 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c030d50 con 0x7f826015aac0 2023-11-08T16:28:58.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.831+0000 7f8266d27640 1 -- 172.21.15.81:0/1406740158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f826015aac0 msgr2=0x7f82600711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.831+0000 7f8266d27640 1 --2- 172.21.15.81:0/1406740158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f826015aac0 0x7f82600711a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82600703e0 tx=0x7f825c02b9a0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.831+0000 7f8266d27640 1 -- 172.21.15.81:0/1406740158 shutdown_connections 2023-11-08T16:28:58.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.831+0000 7f8266d27640 1 --2- 172.21.15.81:0/1406740158 >> v2:172.21.15.93:3300/0 conn(0x7f82600cf420 0x7f826015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.831+0000 7f8266d27640 1 --2- 172.21.15.81:0/1406740158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f826015aac0 0x7f82600711a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.831+0000 7f8266d27640 1 -- 172.21.15.81:0/1406740158 >> 172.21.15.81:0/1406740158 conn(0x7f826006a4a0 msgr2=0x7f826006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:58.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.831+0000 7f8266d27640 1 -- 172.21.15.81:0/1406740158 shutdown_connections 2023-11-08T16:28:58.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.831+0000 7f8266d27640 1 -- 172.21.15.81:0/1406740158 wait complete. 2023-11-08T16:28:58.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.833+0000 7f8266d27640 1 Processor -- start 2023-11-08T16:28:58.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.833+0000 7f8266d27640 1 -- start start 2023-11-08T16:28:58.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.834+0000 7f8266d27640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82600716e0 0x7f82601581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.834+0000 7f8266d27640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f82600cf420 0x7f826014d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.834+0000 7f8266d27640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f826015aac0 0x7f82600ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.834+0000 7f8266d27640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82600c4730 con 0x7f82600716e0 2023-11-08T16:28:58.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.834+0000 7f8266d27640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8260158710 con 0x7f82600cf420 2023-11-08T16:28:58.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.834+0000 7f8266d27640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82600c4170 con 0x7f826015aac0 2023-11-08T16:28:58.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.834+0000 7f8265d25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82600716e0 0x7f82601581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.834+0000 7f8265d25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82600716e0 0x7f82601581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42304/0 (socket says 172.21.15.81:42304) 2023-11-08T16:28:58.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.834+0000 7f8265d25640 1 -- 172.21.15.81:0/2739495751 learned_addr learned my addr 172.21.15.81:0/2739495751 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:58.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.834+0000 7f8265524640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f82600cf420 0x7f826014d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f8266526640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f826015aac0 0x7f82600ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f8265d25640 1 -- 172.21.15.81:0/2739495751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f826015aac0 msgr2=0x7f82600ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f8265d25640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f826015aac0 0x7f82600ac8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f8265d25640 1 -- 172.21.15.81:0/2739495751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f82600cf420 msgr2=0x7f826014d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:58.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f8265d25640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f82600cf420 0x7f826014d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:58.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f8265d25640 1 -- 172.21.15.81:0/2739495751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f825c0094d0 con 0x7f82600716e0 2023-11-08T16:28:58.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f8265524640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f82600cf420 0x7f826014d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:28:58.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f8265d25640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82600716e0 0x7f82601581d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f825c000c00 tx=0x7f825c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:58.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f82567fc640 1 -- 172.21.15.81:0/2739495751 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c039a90 con 0x7f82600716e0 2023-11-08T16:28:58.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f82567fc640 1 -- 172.21.15.81:0/2739495751 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825c039bf0 con 0x7f82600716e0 2023-11-08T16:28:58.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f82567fc640 1 -- 172.21.15.81:0/2739495751 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825c0433f0 con 0x7f82600716e0 2023-11-08T16:28:58.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f8266d27640 1 -- 172.21.15.81:0/2739495751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82600c4450 con 0x7f82600716e0 2023-11-08T16:28:58.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.835+0000 7f8266d27640 1 -- 172.21.15.81:0/2739495751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82600b1e10 con 0x7f82600716e0 2023-11-08T16:28:58.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.836+0000 7f8266d27640 1 -- 172.21.15.81:0/2739495751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82600b1990 con 0x7f82600716e0 2023-11-08T16:28:58.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.837+0000 7f82567fc640 1 -- 172.21.15.81:0/2739495751 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f825c02f3b0 con 0x7f82600716e0 2023-11-08T16:28:58.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.838+0000 7f82567fc640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f824406ae70 0x7f824406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:58.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.838+0000 7f82567fc640 1 -- 172.21.15.81:0/2739495751 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f825c0ab620 con 0x7f82600716e0 2023-11-08T16:28:58.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.838+0000 7f8265524640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f824406ae70 0x7f824406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:58.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.839+0000 7f8265524640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f824406ae70 0x7f824406d330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8260157050 tx=0x7f8250009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:58.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:58.842+0000 7f82567fc640 1 -- 172.21.15.81:0/2739495751 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f82600b1990 con 0x7f82600716e0 2023-11-08T16:28:59.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.067+0000 7f8266d27640 1 -- 172.21.15.81:0/2739495751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82600b1990 con 0x7f82600716e0 2023-11-08T16:28:59.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.068+0000 7f82567fc640 1 -- 172.21.15.81:0/2739495751 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3694 (secure 0 0 0) 0x7f82600b1990 con 0x7f82600716e0 2023-11-08T16:28:59.071 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:28:59.072 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4190352668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4190352668},{"type":"v1","addr":"172.21.15.81:6835","nonce":4190352668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2636431103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2636431103},{"type":"v1","addr":"172.21.15.93:6835","nonce":2636431103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3681499690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3681499690},{"type":"v1","addr":"172.21.15.81:6837","nonce":3681499690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4270,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/2087517381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2087517381},{"type":"v1","addr":"172.21.15.93:6834","nonce":2087517381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-08T16:28:59.072 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 3 2023-11-08T16:28:59.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 -- 172.21.15.81:0/2739495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f824406ae70 msgr2=0x7f824406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f824406ae70 0x7f824406d330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8260157050 tx=0x7f8250009040 comp rx=0 tx=0).stop 2023-11-08T16:28:59.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 -- 172.21.15.81:0/2739495751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82600716e0 msgr2=0x7f82601581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82600716e0 0x7f82601581d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f825c000c00 tx=0x7f825c004730 comp rx=0 tx=0).stop 2023-11-08T16:28:59.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 -- 172.21.15.81:0/2739495751 shutdown_connections 2023-11-08T16:28:59.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f824406ae70 0x7f824406d330 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f826015aac0 0x7f82600ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f82600cf420 0x7f826014d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 --2- 172.21.15.81:0/2739495751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f82600716e0 0x7f82601581d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 -- 172.21.15.81:0/2739495751 >> 172.21.15.81:0/2739495751 conn(0x7f826006a4a0 msgr2=0x7f82601599e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:59.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 -- 172.21.15.81:0/2739495751 shutdown_connections 2023-11-08T16:28:59.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.071+0000 7f8266d27640 1 -- 172.21.15.81:0/2739495751 wait complete. 2023-11-08T16:28:59.092 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:28:59.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.299+0000 7f147e64f640 1 Processor -- start 2023-11-08T16:28:59.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.300+0000 7f147e64f640 1 -- start start 2023-11-08T16:28:59.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.301+0000 7f147e64f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14780adcd0 0x7f14780ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.301+0000 7f147e64f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14780c9a70 con 0x7f14780cc5e0 2023-11-08T16:28:59.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.301+0000 7f147e64f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14780cc3e0 con 0x7f1478070050 2023-11-08T16:28:59.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.301+0000 7f147e64f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f147806d070 con 0x7f14780adcd0 2023-11-08T16:28:59.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.301+0000 7f146ffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f14780cc5e0 0x7f14780cca40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44466/0 (socket says 172.21.15.81:44466) 2023-11-08T16:28:59.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.301+0000 7f146ffff640 1 -- 172.21.15.81:0/2495717434 learned_addr learned my addr 172.21.15.81:0/2495717434 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:28:59.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f1477fff640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14780adcd0 0x7f14780ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324553013 0 0) 0x7f14780c9a70 con 0x7f14780cc5e0 2023-11-08T16:28:59.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1464003520 con 0x7f14780cc5e0 2023-11-08T16:28:59.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 55751861 0 0) 0x7f1464003520 con 0x7f14780cc5e0 2023-11-08T16:28:59.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14780adcd0 msgr2=0x7f14780ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:59.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14780adcd0 0x7f14780ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 >> v1:172.21.15.93:6789/0 conn(0x7f1478070050 legacy=0x7f14780704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f147806d350 con 0x7f14780cc5e0 2023-11-08T16:28:59.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f1460001670 con 0x7f14780cc5e0 2023-11-08T16:28:59.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 >> v1:172.21.15.81:6789/0 conn(0x7f14780cc5e0 legacy=0x7f14780cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1464003a80 0x7f1464003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 0x7f1464004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14780adcd0 0x7f1464005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147806d070 con 0x7f14640047c0 2023-11-08T16:28:59.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f146400dd50 con 0x7f1464003a80 2023-11-08T16:28:59.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.302+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146400df10 con 0x7f14780adcd0 2023-11-08T16:28:59.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.303+0000 7f146ffff640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 0x7f1464004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.303+0000 7f1477fff640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1464003a80 0x7f1464003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.303+0000 7f146ffff640 1 -- 172.21.15.81:0/2495717434 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14780adcd0 msgr2=0x7f1464005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:59.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.303+0000 7f146ffff640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14780adcd0 0x7f1464005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.303+0000 7f146ffff640 1 -- 172.21.15.81:0/2495717434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1464003a80 msgr2=0x7f1464003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:59.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.303+0000 7f146ffff640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1464003a80 0x7f1464003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.303+0000 7f146ffff640 1 -- 172.21.15.81:0/2495717434 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1464003520 con 0x7f14640047c0 2023-11-08T16:28:59.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.303+0000 7f1477fff640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1464003a80 0x7f1464003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:28:59.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.304+0000 7f146ffff640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 0x7f1464004c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f146000c0e0 tx=0x7f14640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796eaf693dc375cd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:59.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.304+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1460005ad0 con 0x7f14640047c0 2023-11-08T16:28:59.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.304+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1460005c50 con 0x7f14640047c0 2023-11-08T16:28:59.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.304+0000 7f14777fe640 1 -- 172.21.15.81:0/2495717434 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146002c810 con 0x7f14640047c0 2023-11-08T16:28:59.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.305+0000 7f147e64f640 1 -- 172.21.15.81:0/2495717434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 msgr2=0x7f1464004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.305+0000 7f147e64f640 1 --2- 172.21.15.81:0/2495717434 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 0x7f1464004c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f146000c0e0 tx=0x7f14640057f0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.305+0000 7f147cbec640 1 -- 172.21.15.81:0/2495717434 reap_dead start 2023-11-08T16:28:59.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.305+0000 7f147e64f640 1 -- 172.21.15.81:0/2495717434 shutdown_connections 2023-11-08T16:28:59.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.305+0000 7f147e64f640 1 -- 172.21.15.81:0/2495717434 >> 172.21.15.81:0/2495717434 conn(0x7f14780cb490 msgr2=0x7f14780cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:59.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.305+0000 7f147e64f640 1 -- 172.21.15.81:0/2495717434 shutdown_connections 2023-11-08T16:28:59.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.305+0000 7f147e64f640 1 -- 172.21.15.81:0/2495717434 wait complete. 2023-11-08T16:28:59.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.307+0000 7f147e64f640 1 Processor -- start 2023-11-08T16:28:59.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.307+0000 7f147e64f640 1 -- start start 2023-11-08T16:28:59.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.308+0000 7f147e64f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 0x7f147815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.308+0000 7f147e64f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14781f4090 0x7f14780ae790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.308+0000 7f147e64f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14781f4510 0x7f14780aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.308+0000 7f147e64f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780c4620 con 0x7f14640047c0 2023-11-08T16:28:59.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.308+0000 7f147e64f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14780c74c0 con 0x7f14781f4090 2023-11-08T16:28:59.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.308+0000 7f147e64f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14780c3c80 con 0x7f14781f4510 2023-11-08T16:28:59.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.308+0000 7f1477fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 0x7f147815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.308+0000 7f1477fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 0x7f147815ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42314/0 (socket says 172.21.15.81:42314) 2023-11-08T16:28:59.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.308+0000 7f1477fff640 1 -- 172.21.15.81:0/1151253795 learned_addr learned my addr 172.21.15.81:0/1151253795 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:59.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f146ffff640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14781f4090 0x7f14780ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f147cbec640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14781f4510 0x7f14780aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f1477fff640 1 -- 172.21.15.81:0/1151253795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14781f4510 msgr2=0x7f14780aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f1477fff640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14781f4510 0x7f14780aecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f1477fff640 1 -- 172.21.15.81:0/1151253795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14781f4090 msgr2=0x7f14780ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f1477fff640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14781f4090 0x7f14780ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f1477fff640 1 -- 172.21.15.81:0/1151253795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14780c3f60 con 0x7f14640047c0 2023-11-08T16:28:59.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f146ffff640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14781f4090 0x7f14780ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:28:59.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f1477fff640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 0x7f147815ab90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f14680107e0 tx=0x7f1468010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:59.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f14757fa640 1 -- 172.21.15.81:0/1151253795 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146800eb60 con 0x7f14640047c0 2023-11-08T16:28:59.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f14757fa640 1 -- 172.21.15.81:0/1151253795 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f146800ecc0 con 0x7f14640047c0 2023-11-08T16:28:59.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f14757fa640 1 -- 172.21.15.81:0/1151253795 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146801c400 con 0x7f14640047c0 2023-11-08T16:28:59.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f147e64f640 1 -- 172.21.15.81:0/1151253795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14780c4240 con 0x7f14640047c0 2023-11-08T16:28:59.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.309+0000 7f147e64f640 1 -- 172.21.15.81:0/1151253795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14780c4a40 con 0x7f14640047c0 2023-11-08T16:28:59.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.311+0000 7f147e64f640 1 -- 172.21.15.81:0/1151253795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14780c4890 con 0x7f14640047c0 2023-11-08T16:28:59.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.311+0000 7f14757fa640 1 -- 172.21.15.81:0/1151253795 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f146801c560 con 0x7f14640047c0 2023-11-08T16:28:59.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.312+0000 7f14757fa640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f145806ae70 0x7f145806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.312+0000 7f14757fa640 1 -- 172.21.15.81:0/1151253795 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1468089830 con 0x7f14640047c0 2023-11-08T16:28:59.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.313+0000 7f146ffff640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f145806ae70 0x7f145806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.313+0000 7f146ffff640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f145806ae70 0x7f145806d330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1460034000 tx=0x7f1460009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:59.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.317+0000 7f14757fa640 1 -- 172.21.15.81:0/1151253795 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1468054cd0 con 0x7f14640047c0 2023-11-08T16:28:59.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.509+0000 7f147e64f640 1 -- 172.21.15.81:0/1151253795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1478072170 con 0x7f14640047c0 2023-11-08T16:28:59.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.510+0000 7f14757fa640 1 -- 172.21.15.81:0/1151253795 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12790 (secure 0 0 0) 0x7f14680582f0 con 0x7f14640047c0 2023-11-08T16:28:59.515 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:28:59.515 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":17,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:28:54.878405+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:28:59.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 -- 172.21.15.81:0/1151253795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f145806ae70 msgr2=0x7f145806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f145806ae70 0x7f145806d330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1460034000 tx=0x7f1460009030 comp rx=0 tx=0).stop 2023-11-08T16:28:59.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 -- 172.21.15.81:0/1151253795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 msgr2=0x7f147815ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 0x7f147815ab90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f14680107e0 tx=0x7f1468010ca0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 -- 172.21.15.81:0/1151253795 shutdown_connections 2023-11-08T16:28:59.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f145806ae70 0x7f145806d330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14781f4510 0x7f14780aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14781f4090 0x7f14780ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 --2- 172.21.15.81:0/1151253795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14640047c0 0x7f147815ab90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 -- 172.21.15.81:0/1151253795 >> 172.21.15.81:0/1151253795 conn(0x7f14780cb490 msgr2=0x7f14780c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:59.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 -- 172.21.15.81:0/1151253795 shutdown_connections 2023-11-08T16:28:59.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.513+0000 7f147e64f640 1 -- 172.21.15.81:0/1151253795 wait complete. 2023-11-08T16:28:59.537 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:28:59.538 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:28:59.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.754+0000 7f2a290da640 1 Processor -- start 2023-11-08T16:28:59.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.755+0000 7f2a290da640 1 -- start start 2023-11-08T16:28:59.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.756+0000 7f2a290da640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a2415a0e0 0x7f2a240ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.756+0000 7f2a290da640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2a240c9bf0 0x7f2a240ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.756+0000 7f2a290da640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2a240ca550 0x7f2a2415dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.756+0000 7f2a290da640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a24071980 con 0x7f2a240c9bf0 2023-11-08T16:28:59.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.756+0000 7f2a290da640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a240ce960 con 0x7f2a240ca550 2023-11-08T16:28:59.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.756+0000 7f2a290da640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240c9480 con 0x7f2a2415a0e0 2023-11-08T16:28:59.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.756+0000 7f2a1bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2a240c9bf0 0x7f2a240ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.756+0000 7f2a1bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2a240c9bf0 0x7f2a240ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42322/0 (socket says 172.21.15.81:42322) 2023-11-08T16:28:59.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.756+0000 7f2a1bfff640 1 -- 172.21.15.81:0/3195802207 learned_addr learned my addr 172.21.15.81:0/3195802207 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:59.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.756+0000 7f2a22d9d640 1 --2- 172.21.15.81:0/3195802207 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a2415a0e0 0x7f2a240ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.756+0000 7f2a2359e640 1 --2- 172.21.15.81:0/3195802207 >> v2:172.21.15.93:3300/0 conn(0x7f2a240ca550 0x7f2a2415dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.757+0000 7f2a1bfff640 1 -- 172.21.15.81:0/3195802207 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a2415a0e0 msgr2=0x7f2a240ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:59.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.757+0000 7f2a1bfff640 1 --2- 172.21.15.81:0/3195802207 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a2415a0e0 0x7f2a240ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.757+0000 7f2a1bfff640 1 -- 172.21.15.81:0/3195802207 >> v2:172.21.15.93:3300/0 conn(0x7f2a240ca550 msgr2=0x7f2a2415dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:28:59.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.757+0000 7f2a1bfff640 1 --2- 172.21.15.81:0/3195802207 >> v2:172.21.15.93:3300/0 conn(0x7f2a240ca550 0x7f2a2415dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.757+0000 7f2a1bfff640 1 -- 172.21.15.81:0/3195802207 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a2415a4c0 con 0x7f2a240c9bf0 2023-11-08T16:28:59.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.757+0000 7f2a2359e640 1 --2- 172.21.15.81:0/3195802207 >> v2:172.21.15.93:3300/0 conn(0x7f2a240ca550 0x7f2a2415dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:28:59.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.757+0000 7f2a22d9d640 1 --2- 172.21.15.81:0/3195802207 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a2415a0e0 0x7f2a240ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:28:59.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.758+0000 7f2a1bfff640 1 --2- 172.21.15.81:0/3195802207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a240c9bf0 0x7f2a240ca010 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2a240cd1f0 tx=0x7f2a0c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe901a0f5314484 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:59.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.758+0000 7f2a2259c640 1 -- 172.21.15.81:0/3195802207 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c030860 con 0x7f2a240c9bf0 2023-11-08T16:28:59.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.758+0000 7f2a2259c640 1 -- 172.21.15.81:0/3195802207 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c0309c0 con 0x7f2a240c9bf0 2023-11-08T16:28:59.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.758+0000 7f2a2259c640 1 -- 172.21.15.81:0/3195802207 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c030d50 con 0x7f2a240c9bf0 2023-11-08T16:28:59.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.759+0000 7f2a290da640 1 -- 172.21.15.81:0/3195802207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a240c9bf0 msgr2=0x7f2a240ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.759+0000 7f2a290da640 1 --2- 172.21.15.81:0/3195802207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a240c9bf0 0x7f2a240ca010 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f2a240cd1f0 tx=0x7f2a0c02b9a0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.759+0000 7f2a290da640 1 -- 172.21.15.81:0/3195802207 shutdown_connections 2023-11-08T16:28:59.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.759+0000 7f2a290da640 1 --2- 172.21.15.81:0/3195802207 >> v2:172.21.15.93:3300/0 conn(0x7f2a240ca550 0x7f2a2415dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.759+0000 7f2a290da640 1 --2- 172.21.15.81:0/3195802207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a240c9bf0 0x7f2a240ca010 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.759+0000 7f2a290da640 1 --2- 172.21.15.81:0/3195802207 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a2415a0e0 0x7f2a240ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.759+0000 7f2a290da640 1 -- 172.21.15.81:0/3195802207 >> 172.21.15.81:0/3195802207 conn(0x7f2a240cb580 msgr2=0x7f2a240cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:59.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.759+0000 7f2a290da640 1 -- 172.21.15.81:0/3195802207 shutdown_connections 2023-11-08T16:28:59.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.759+0000 7f2a290da640 1 -- 172.21.15.81:0/3195802207 wait complete. 2023-11-08T16:28:59.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.761+0000 7f2a290da640 1 Processor -- start 2023-11-08T16:28:59.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.761+0000 7f2a290da640 1 -- start start 2023-11-08T16:28:59.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a290da640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a240c9bf0 0x7f2a241f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a290da640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a240ca550 0x7f2a24159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a290da640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a2415a0e0 0x7f2a240b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a290da640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240c7530 con 0x7f2a2415a0e0 2023-11-08T16:28:59.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a290da640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240c6c20 con 0x7f2a240c9bf0 2023-11-08T16:28:59.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a290da640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a240b3f60 con 0x7f2a240ca550 2023-11-08T16:28:59.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a2359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a2415a0e0 0x7f2a240b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a2359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a2415a0e0 0x7f2a240b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42328/0 (socket says 172.21.15.81:42328) 2023-11-08T16:28:59.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a2359e640 1 -- 172.21.15.81:0/4171486870 learned_addr learned my addr 172.21.15.81:0/4171486870 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:28:59.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a22d9d640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a240c9bf0 0x7f2a241f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a1bfff640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a240ca550 0x7f2a24159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a2359e640 1 -- 172.21.15.81:0/4171486870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a240ca550 msgr2=0x7f2a24159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a2359e640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a240ca550 0x7f2a24159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a2359e640 1 -- 172.21.15.81:0/4171486870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a240c9bf0 msgr2=0x7f2a241f31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a2359e640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a240c9bf0 0x7f2a241f31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a2359e640 1 -- 172.21.15.81:0/4171486870 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c0094d0 con 0x7f2a2415a0e0 2023-11-08T16:28:59.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.762+0000 7f2a1bfff640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a240ca550 0x7f2a24159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:28:59.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.763+0000 7f2a2359e640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a2415a0e0 0x7f2a240b3610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00cef0 tx=0x7f2a1c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:59.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.763+0000 7f2a1b7fe640 1 -- 172.21.15.81:0/4171486870 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00d840 con 0x7f2a2415a0e0 2023-11-08T16:28:59.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.763+0000 7f2a1b7fe640 1 -- 172.21.15.81:0/4171486870 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a1c00d9a0 con 0x7f2a2415a0e0 2023-11-08T16:28:59.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.763+0000 7f2a1b7fe640 1 -- 172.21.15.81:0/4171486870 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1c00dd50 con 0x7f2a2415a0e0 2023-11-08T16:28:59.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.763+0000 7f2a290da640 1 -- 172.21.15.81:0/4171486870 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a240c6ee0 con 0x7f2a2415a0e0 2023-11-08T16:28:59.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.763+0000 7f2a290da640 1 -- 172.21.15.81:0/4171486870 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a241f3720 con 0x7f2a2415a0e0 2023-11-08T16:28:59.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.764+0000 7f2a290da640 1 -- 172.21.15.81:0/4171486870 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a240c76a0 con 0x7f2a2415a0e0 2023-11-08T16:28:59.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.765+0000 7f2a1b7fe640 1 -- 172.21.15.81:0/4171486870 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a1c01e3b0 con 0x7f2a2415a0e0 2023-11-08T16:28:59.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.767+0000 7f2a1b7fe640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a0406ae70 0x7f2a0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:28:59.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.767+0000 7f2a1b7fe640 1 -- 172.21.15.81:0/4171486870 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2a1c089db0 con 0x7f2a2415a0e0 2023-11-08T16:28:59.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.767+0000 7f2a22d9d640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a0406ae70 0x7f2a0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:28:59.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.767+0000 7f2a22d9d640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a0406ae70 0x7f2a0406d330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400a950 tx=0x7f2a14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:28:59.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.770+0000 7f2a1b7fe640 1 -- 172.21.15.81:0/4171486870 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a1c08e070 con 0x7f2a2415a0e0 2023-11-08T16:28:59.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.959+0000 7f2a290da640 1 -- 172.21.15.81:0/4171486870 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2a2404ece0 con 0x7f2a2415a0e0 2023-11-08T16:28:59.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.960+0000 7f2a1b7fe640 1 -- 172.21.15.81:0/4171486870 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2a1c01a0a0 con 0x7f2a2415a0e0 2023-11-08T16:28:59.963 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:28:59.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 -- 172.21.15.81:0/4171486870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a0406ae70 msgr2=0x7f2a0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a0406ae70 0x7f2a0406d330 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2a1400a950 tx=0x7f2a14008040 comp rx=0 tx=0).stop 2023-11-08T16:28:59.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 -- 172.21.15.81:0/4171486870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a2415a0e0 msgr2=0x7f2a240b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:28:59.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a2415a0e0 0x7f2a240b3610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00cef0 tx=0x7f2a1c00a710 comp rx=0 tx=0).stop 2023-11-08T16:28:59.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 -- 172.21.15.81:0/4171486870 shutdown_connections 2023-11-08T16:28:59.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a0406ae70 0x7f2a0406d330 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a2415a0e0 0x7f2a240b3610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a240ca550 0x7f2a24159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 --2- 172.21.15.81:0/4171486870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a240c9bf0 0x7f2a241f31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:28:59.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 -- 172.21.15.81:0/4171486870 >> 172.21.15.81:0/4171486870 conn(0x7f2a240cb580 msgr2=0x7f2a240ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:28:59.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 -- 172.21.15.81:0/4171486870 shutdown_connections 2023-11-08T16:28:59.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:28:59.963+0000 7f2a290da640 1 -- 172.21.15.81:0/4171486870 wait complete. 2023-11-08T16:28:59.983 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T16:29:00.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.193+0000 7f9cec9e2640 1 Processor -- start 2023-11-08T16:29:00.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.193+0000 7f9cec9e2640 1 -- start start 2023-11-08T16:29:00.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.194+0000 7f9cec9e2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9ce815aeb0 0x7f9ce80ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.194+0000 7f9cec9e2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ce80c90f0 0x7f9ce80c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.194+0000 7f9cec9e2640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9ce806b2b0 0x7f9ce806b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.194+0000 7f9cec9e2640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce80cb3e0 con 0x7f9ce815aeb0 2023-11-08T16:29:00.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.194+0000 7f9cec9e2640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce806cc60 con 0x7f9ce806b2b0 2023-11-08T16:29:00.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.194+0000 7f9cec9e2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80c9a90 con 0x7f9ce80c90f0 2023-11-08T16:29:00.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.194+0000 7f9ce659c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9ce815aeb0 0x7f9ce80ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.195+0000 7f9ce659c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9ce815aeb0 0x7f9ce80ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42334/0 (socket says 172.21.15.81:42334) 2023-11-08T16:29:00.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.195+0000 7f9ce659c640 1 -- 172.21.15.81:0/2954711820 learned_addr learned my addr 172.21.15.81:0/2954711820 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:00.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.195+0000 7f9cddd9b640 1 --2- 172.21.15.81:0/2954711820 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ce80c90f0 0x7f9ce80c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.195+0000 7f9ce6d9d640 1 --2- 172.21.15.81:0/2954711820 >> v2:172.21.15.93:3300/0 conn(0x7f9ce806b2b0 0x7f9ce806b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.195+0000 7f9ce659c640 1 -- 172.21.15.81:0/2954711820 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ce80c90f0 msgr2=0x7f9ce80c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:00.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.195+0000 7f9ce659c640 1 --2- 172.21.15.81:0/2954711820 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ce80c90f0 0x7f9ce80c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.195+0000 7f9ce659c640 1 -- 172.21.15.81:0/2954711820 >> v2:172.21.15.93:3300/0 conn(0x7f9ce806b2b0 msgr2=0x7f9ce806b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:00.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.195+0000 7f9ce659c640 1 --2- 172.21.15.81:0/2954711820 >> v2:172.21.15.93:3300/0 conn(0x7f9ce806b2b0 0x7f9ce806b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.195+0000 7f9ce659c640 1 -- 172.21.15.81:0/2954711820 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce80cae20 con 0x7f9ce815aeb0 2023-11-08T16:29:00.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.195+0000 7f9ce6d9d640 1 --2- 172.21.15.81:0/2954711820 >> v2:172.21.15.93:3300/0 conn(0x7f9ce806b2b0 0x7f9ce806b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:00.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.196+0000 7f9ce659c640 1 --2- 172.21.15.81:0/2954711820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ce815aeb0 0x7f9ce80ca4a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ce806f990 tx=0x7f9cd802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903997692922ded5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:00.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.196+0000 7f9ce5d9b640 1 -- 172.21.15.81:0/2954711820 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8004230 con 0x7f9ce815aeb0 2023-11-08T16:29:00.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.196+0000 7f9ce5d9b640 1 -- 172.21.15.81:0/2954711820 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd8004390 con 0x7f9ce815aeb0 2023-11-08T16:29:00.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.196+0000 7f9ce5d9b640 1 -- 172.21.15.81:0/2954711820 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8003810 con 0x7f9ce815aeb0 2023-11-08T16:29:00.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.197+0000 7f9cec9e2640 1 -- 172.21.15.81:0/2954711820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ce815aeb0 msgr2=0x7f9ce80ca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:00.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.197+0000 7f9cec9e2640 1 --2- 172.21.15.81:0/2954711820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ce815aeb0 0x7f9ce80ca4a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9ce806f990 tx=0x7f9cd802b8f0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.197+0000 7f9cec9e2640 1 -- 172.21.15.81:0/2954711820 shutdown_connections 2023-11-08T16:29:00.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.197+0000 7f9cec9e2640 1 --2- 172.21.15.81:0/2954711820 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ce80c90f0 0x7f9ce80c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.197+0000 7f9cec9e2640 1 --2- 172.21.15.81:0/2954711820 >> v2:172.21.15.93:3300/0 conn(0x7f9ce806b2b0 0x7f9ce806b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.197+0000 7f9cec9e2640 1 --2- 172.21.15.81:0/2954711820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ce815aeb0 0x7f9ce80ca4a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.197+0000 7f9cec9e2640 1 -- 172.21.15.81:0/2954711820 >> 172.21.15.81:0/2954711820 conn(0x7f9ce80ceaa0 msgr2=0x7f9ce80cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:00.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.197+0000 7f9cec9e2640 1 -- 172.21.15.81:0/2954711820 shutdown_connections 2023-11-08T16:29:00.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.197+0000 7f9cec9e2640 1 -- 172.21.15.81:0/2954711820 wait complete. 2023-11-08T16:29:00.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.199+0000 7f9cec9e2640 1 Processor -- start 2023-11-08T16:29:00.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.200+0000 7f9cec9e2640 1 -- start start 2023-11-08T16:29:00.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.201+0000 7f9cec9e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ce806b2b0 0x7f9ce80cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.201+0000 7f9cec9e2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce80c90f0 0x7f9ce80c5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.201+0000 7f9cec9e2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ce815aeb0 0x7f9ce814fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.201+0000 7f9cec9e2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce8150260 con 0x7f9ce806b2b0 2023-11-08T16:29:00.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.201+0000 7f9cec9e2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80c4930 con 0x7f9ce80c90f0 2023-11-08T16:29:00.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.201+0000 7f9cec9e2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce80ccc50 con 0x7f9ce815aeb0 2023-11-08T16:29:00.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.201+0000 7f9ce659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ce806b2b0 0x7f9ce80cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.201+0000 7f9ce659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ce806b2b0 0x7f9ce80cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42340/0 (socket says 172.21.15.81:42340) 2023-11-08T16:29:00.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.201+0000 7f9ce659c640 1 -- 172.21.15.81:0/3029011065 learned_addr learned my addr 172.21.15.81:0/3029011065 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:00.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9cddd9b640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce80c90f0 0x7f9ce80c5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9ce6d9d640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ce815aeb0 0x7f9ce814fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9ce659c640 1 -- 172.21.15.81:0/3029011065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ce815aeb0 msgr2=0x7f9ce814fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:00.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9ce659c640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ce815aeb0 0x7f9ce814fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9ce659c640 1 -- 172.21.15.81:0/3029011065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce80c90f0 msgr2=0x7f9ce80c5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:00.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9ce659c640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce80c90f0 0x7f9ce80c5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9ce659c640 1 -- 172.21.15.81:0/3029011065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd8009460 con 0x7f9ce806b2b0 2023-11-08T16:29:00.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9cddd9b640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce80c90f0 0x7f9ce80c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:00.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9ce659c640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ce806b2b0 0x7f9ce80cc670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8000c00 tx=0x7f9cd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:00.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9cdf7fe640 1 -- 172.21.15.81:0/3029011065 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8003c40 con 0x7f9ce806b2b0 2023-11-08T16:29:00.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9cdf7fe640 1 -- 172.21.15.81:0/3029011065 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd803f070 con 0x7f9ce806b2b0 2023-11-08T16:29:00.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.202+0000 7f9cec9e2640 1 -- 172.21.15.81:0/3029011065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce80ccf30 con 0x7f9ce806b2b0 2023-11-08T16:29:00.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.203+0000 7f9cdf7fe640 1 -- 172.21.15.81:0/3029011065 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd8004500 con 0x7f9ce806b2b0 2023-11-08T16:29:00.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.203+0000 7f9cec9e2640 1 -- 172.21.15.81:0/3029011065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce80cd4f0 con 0x7f9ce806b2b0 2023-11-08T16:29:00.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.204+0000 7f9cec9e2640 1 -- 172.21.15.81:0/3029011065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce80743f0 con 0x7f9ce806b2b0 2023-11-08T16:29:00.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.205+0000 7f9cdf7fe640 1 -- 172.21.15.81:0/3029011065 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9cd8041030 con 0x7f9ce806b2b0 2023-11-08T16:29:00.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.206+0000 7f9cdf7fe640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cc806ae70 0x7f9cc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.206+0000 7f9cdf7fe640 1 -- 172.21.15.81:0/3029011065 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9cd80ab5a0 con 0x7f9ce806b2b0 2023-11-08T16:29:00.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.206+0000 7f9cddd9b640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cc806ae70 0x7f9cc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.207+0000 7f9cddd9b640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cc806ae70 0x7f9cc806d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80b3170 tx=0x7f9cd0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:00.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.210+0000 7f9cdf7fe640 1 -- 172.21.15.81:0/3029011065 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9cd8076990 con 0x7f9ce806b2b0 2023-11-08T16:29:00.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.401+0000 7f9cec9e2640 1 -- 172.21.15.81:0/3029011065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9ce80b26d0 con 0x7f9ce806b2b0 2023-11-08T16:29:00.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.402+0000 7f9cdf7fe640 1 -- 172.21.15.81:0/3029011065 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9cd8079fb0 con 0x7f9ce806b2b0 2023-11-08T16:29:00.405 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:00.405 INFO:teuthology.orchestra.run.smithi081.stdout:[ 2023-11-08T16:29:00.406 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:29:00.406 INFO:teuthology.orchestra.run.smithi081.stdout: "poolnum": 1, 2023-11-08T16:29:00.406 INFO:teuthology.orchestra.run.smithi081.stdout: "poolname": ".mgr" 2023-11-08T16:29:00.406 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:29:00.407 INFO:teuthology.orchestra.run.smithi081.stdout:] 2023-11-08T16:29:00.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.404+0000 7f9cec9e2640 1 -- 172.21.15.81:0/3029011065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cc806ae70 msgr2=0x7f9cc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:00.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.404+0000 7f9cec9e2640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cc806ae70 0x7f9cc806d330 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ce80b3170 tx=0x7f9cd0008040 comp rx=0 tx=0).stop 2023-11-08T16:29:00.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.404+0000 7f9cec9e2640 1 -- 172.21.15.81:0/3029011065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ce806b2b0 msgr2=0x7f9ce80cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:00.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.404+0000 7f9cec9e2640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ce806b2b0 0x7f9ce80cc670 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9cd8000c00 tx=0x7f9cd80046c0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.404+0000 7f9cec9e2640 1 -- 172.21.15.81:0/3029011065 shutdown_connections 2023-11-08T16:29:00.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.404+0000 7f9cec9e2640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cc806ae70 0x7f9cc806d330 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.404+0000 7f9cec9e2640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ce815aeb0 0x7f9ce814fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.404+0000 7f9cec9e2640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ce80c90f0 0x7f9ce80c5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.404+0000 7f9cec9e2640 1 --2- 172.21.15.81:0/3029011065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9ce806b2b0 0x7f9ce80cc670 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.404+0000 7f9cec9e2640 1 -- 172.21.15.81:0/3029011065 >> 172.21.15.81:0/3029011065 conn(0x7f9ce80ceaa0 msgr2=0x7f9ce80b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:00.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.405+0000 7f9cec9e2640 1 -- 172.21.15.81:0/3029011065 shutdown_connections 2023-11-08T16:29:00.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.405+0000 7f9cec9e2640 1 -- 172.21.15.81:0/3029011065 wait complete. 2023-11-08T16:29:00.425 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T16:29:00.425 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T16:29:00.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.640+0000 7f2c63eaf640 1 Processor -- start 2023-11-08T16:29:00.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.640+0000 7f2c63eaf640 1 -- start start 2023-11-08T16:29:00.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.642+0000 7f2c63eaf640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2c5c15e170 0x7f2c5c15ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.642+0000 7f2c63eaf640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c0b0bf0 0x7f2c5c0ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.642+0000 7f2c63eaf640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5c071460 con 0x7f2c5c15e550 2023-11-08T16:29:00.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.642+0000 7f2c63eaf640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c0703c0 con 0x7f2c5c15e170 2023-11-08T16:29:00.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.642+0000 7f2c63eaf640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0706a0 con 0x7f2c5c0b0bf0 2023-11-08T16:29:00.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.642+0000 7f2c61c4b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2c5c15e170 0x7f2c5c15ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.642+0000 7f2c61c4b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2c5c15e170 0x7f2c5c15ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42352/0 (socket says 172.21.15.81:42352) 2023-11-08T16:29:00.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.642+0000 7f2c61c4b640 1 -- 172.21.15.81:0/2486225984 learned_addr learned my addr 172.21.15.81:0/2486225984 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:00.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.642+0000 7f2c6244c640 1 --2- 172.21.15.81:0/2486225984 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c0b0bf0 0x7f2c5c0ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.643+0000 7f2c61c4b640 1 -- 172.21.15.81:0/2486225984 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c0b0bf0 msgr2=0x7f2c5c0ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:00.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.643+0000 7f2c61c4b640 1 --2- 172.21.15.81:0/2486225984 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c0b0bf0 0x7f2c5c0ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.643+0000 7f2c61c4b640 1 -- 172.21.15.81:0/2486225984 >> v1:172.21.15.81:6789/0 conn(0x7f2c5c15e550 legacy=0x7f2c5c0b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:00.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.643+0000 7f2c61c4b640 1 -- 172.21.15.81:0/2486225984 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c070980 con 0x7f2c5c15e170 2023-11-08T16:29:00.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.644+0000 7f2c61c4b640 1 --2- 172.21.15.81:0/2486225984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c5c15e170 0x7f2c5c15ac80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009e50 tx=0x7f2c5802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4543f5bbcca315a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:00.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.644+0000 7f2c60c49640 1 -- 172.21.15.81:0/2486225984 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5802cb00 con 0x7f2c5c15e170 2023-11-08T16:29:00.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.644+0000 7f2c60c49640 1 -- 172.21.15.81:0/2486225984 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5802cc60 con 0x7f2c5c15e170 2023-11-08T16:29:00.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.644+0000 7f2c60c49640 1 -- 172.21.15.81:0/2486225984 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c580343f0 con 0x7f2c5c15e170 2023-11-08T16:29:00.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.645+0000 7f2c63eaf640 1 -- 172.21.15.81:0/2486225984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c5c15e170 msgr2=0x7f2c5c15ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:00.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.645+0000 7f2c63eaf640 1 --2- 172.21.15.81:0/2486225984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c5c15e170 0x7f2c5c15ac80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c58009e50 tx=0x7f2c5802bf60 comp rx=0 tx=0).stop 2023-11-08T16:29:00.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.645+0000 7f2c63eaf640 1 -- 172.21.15.81:0/2486225984 shutdown_connections 2023-11-08T16:29:00.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.645+0000 7f2c63eaf640 1 --2- 172.21.15.81:0/2486225984 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c0b0bf0 0x7f2c5c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.645+0000 7f2c63eaf640 1 --2- 172.21.15.81:0/2486225984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c5c15e170 0x7f2c5c15ac80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.645+0000 7f2c63eaf640 1 -- 172.21.15.81:0/2486225984 >> 172.21.15.81:0/2486225984 conn(0x7f2c5c0cb4e0 msgr2=0x7f2c5c0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:00.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.645+0000 7f2c63eaf640 1 -- 172.21.15.81:0/2486225984 shutdown_connections 2023-11-08T16:29:00.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.645+0000 7f2c63eaf640 1 -- 172.21.15.81:0/2486225984 wait complete. 2023-11-08T16:29:00.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.647+0000 7f2c63eaf640 1 Processor -- start 2023-11-08T16:29:00.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.647+0000 7f2c63eaf640 1 -- start start 2023-11-08T16:29:00.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c63eaf640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c0b0bf0 0x7f2c5c0c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c63eaf640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c15e170 0x7f2c5c0c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c63eaf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c5c15e550 0x7f2c5c14df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c63eaf640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0c3560 con 0x7f2c5c15e550 2023-11-08T16:29:00.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c63eaf640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0c3ec0 con 0x7f2c5c0b0bf0 2023-11-08T16:29:00.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c63eaf640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0c28e0 con 0x7f2c5c15e170 2023-11-08T16:29:00.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c6244c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c5c15e550 0x7f2c5c14df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c6244c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c5c15e550 0x7f2c5c14df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42364/0 (socket says 172.21.15.81:42364) 2023-11-08T16:29:00.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c6244c640 1 -- 172.21.15.81:0/287613150 learned_addr learned my addr 172.21.15.81:0/287613150 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:00.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c6144a640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c15e170 0x7f2c5c0c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c61c4b640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c0b0bf0 0x7f2c5c0c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c6244c640 1 -- 172.21.15.81:0/287613150 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c15e170 msgr2=0x7f2c5c0c4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:00.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c6244c640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c15e170 0x7f2c5c0c4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c6244c640 1 -- 172.21.15.81:0/287613150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c0b0bf0 msgr2=0x7f2c5c0c3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:00.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c6244c640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c0b0bf0 0x7f2c5c0c3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:00.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c6244c640 1 -- 172.21.15.81:0/287613150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c58009b30 con 0x7f2c5c15e550 2023-11-08T16:29:00.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.648+0000 7f2c61c4b640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c0b0bf0 0x7f2c5c0c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:00.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.649+0000 7f2c6244c640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c5c15e550 0x7f2c5c14df10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400d5d0 tx=0x7f2c5400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:00.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.649+0000 7f2c527fc640 1 -- 172.21.15.81:0/287613150 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c540125f0 con 0x7f2c5c15e550 2023-11-08T16:29:00.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.649+0000 7f2c527fc640 1 -- 172.21.15.81:0/287613150 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c54012750 con 0x7f2c5c15e550 2023-11-08T16:29:00.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.649+0000 7f2c527fc640 1 -- 172.21.15.81:0/287613150 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54012b00 con 0x7f2c5c15e550 2023-11-08T16:29:00.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.649+0000 7f2c63eaf640 1 -- 172.21.15.81:0/287613150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c0c2bc0 con 0x7f2c5c15e550 2023-11-08T16:29:00.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.649+0000 7f2c63eaf640 1 -- 172.21.15.81:0/287613150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c0c3180 con 0x7f2c5c15e550 2023-11-08T16:29:00.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.650+0000 7f2c63eaf640 1 -- 172.21.15.81:0/287613150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5c0c36a0 con 0x7f2c5c15e550 2023-11-08T16:29:00.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.651+0000 7f2c527fc640 1 -- 172.21.15.81:0/287613150 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c54012c60 con 0x7f2c5c15e550 2023-11-08T16:29:00.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.652+0000 7f2c527fc640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4006ae70 0x7f2c4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:00.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.652+0000 7f2c527fc640 1 -- 172.21.15.81:0/287613150 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2c540897e0 con 0x7f2c5c15e550 2023-11-08T16:29:00.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.652+0000 7f2c61c4b640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4006ae70 0x7f2c4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:00.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.653+0000 7f2c61c4b640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4006ae70 0x7f2c4006d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c58035000 tx=0x7f2c58030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:00.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.657+0000 7f2c527fc640 1 -- 172.21.15.81:0/287613150 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c54054bd0 con 0x7f2c5c15e550 2023-11-08T16:29:00.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:00.852+0000 7f2c63eaf640 1 -- 172.21.15.81:0/287613150 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2c5c0743f0 con 0x7f2c5c15e550 2023-11-08T16:29:01.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.698+0000 7f2c527fc640 1 -- 172.21.15.81:0/287613150 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2c540581f0 con 0x7f2c5c15e550 2023-11-08T16:29:01.702 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' created 2023-11-08T16:29:01.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.703+0000 7f2c63eaf640 1 -- 172.21.15.81:0/287613150 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4006ae70 msgr2=0x7f2c4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:01.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.703+0000 7f2c63eaf640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4006ae70 0x7f2c4006d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2c58035000 tx=0x7f2c58030000 comp rx=0 tx=0).stop 2023-11-08T16:29:01.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.703+0000 7f2c63eaf640 1 -- 172.21.15.81:0/287613150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c5c15e550 msgr2=0x7f2c5c14df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:01.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.703+0000 7f2c63eaf640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c5c15e550 0x7f2c5c14df10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400d5d0 tx=0x7f2c5400da90 comp rx=0 tx=0).stop 2023-11-08T16:29:01.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.703+0000 7f2c63eaf640 1 -- 172.21.15.81:0/287613150 shutdown_connections 2023-11-08T16:29:01.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.703+0000 7f2c63eaf640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4006ae70 0x7f2c4006d330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:01.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.703+0000 7f2c63eaf640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c5c15e550 0x7f2c5c14df10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.703+0000 7f2c63eaf640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c15e170 0x7f2c5c0c4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.703+0000 7f2c63eaf640 1 --2- 172.21.15.81:0/287613150 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c0b0bf0 0x7f2c5c0c3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.703+0000 7f2c63eaf640 1 -- 172.21.15.81:0/287613150 >> 172.21.15.81:0/287613150 conn(0x7f2c5c0cb4e0 msgr2=0x7f2c5c0cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.703+0000 7f2c63eaf640 1 -- 172.21.15.81:0/287613150 shutdown_connections 2023-11-08T16:29:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.704+0000 7f2c63eaf640 1 -- 172.21.15.81:0/287613150 wait complete. 2023-11-08T16:29:01.731 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T16:29:01.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.936+0000 7fdd4949f640 1 Processor -- start 2023-11-08T16:29:01.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.936+0000 7fdd4949f640 1 -- start start 2023-11-08T16:29:01.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.938+0000 7fdd4949f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdd440adbc0 0x7fdd4406a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:01.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.938+0000 7fdd4949f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4406a5f0 con 0x7fdd4415abe0 2023-11-08T16:29:01.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.938+0000 7fdd4949f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd440ce590 con 0x7fdd4406f6b0 2023-11-08T16:29:01.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.938+0000 7fdd4949f640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4406ade0 con 0x7fdd440adbc0 2023-11-08T16:29:01.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.938+0000 7fdd43fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fdd4415abe0 0x7fdd440cd7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58768/0 (socket says 172.21.15.81:58768) 2023-11-08T16:29:01.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.938+0000 7fdd43fff640 1 -- 172.21.15.81:0/2740500801 learned_addr learned my addr 172.21.15.81:0/2740500801 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:01.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.938+0000 7fdd437fe640 1 --2- 172.21.15.81:0/2740500801 >> v2:172.21.15.81:3300/0 conn(0x7fdd440adbc0 0x7fdd4406a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:01.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.939+0000 7fdd437fe640 1 -- 172.21.15.81:0/2740500801 >> v1:172.21.15.93:6789/0 conn(0x7fdd4406f6b0 legacy=0x7fdd4406fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:01.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.939+0000 7fdd437fe640 1 -- 172.21.15.81:0/2740500801 >> v1:172.21.15.81:6789/0 conn(0x7fdd4415abe0 legacy=0x7fdd440cd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:01.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.939+0000 7fdd437fe640 1 -- 172.21.15.81:0/2740500801 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4415afc0 con 0x7fdd440adbc0 2023-11-08T16:29:01.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.939+0000 7fdd42ffd640 1 -- 172.21.15.81:0/2740500801 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310620484 0 0) 0x7fdd4406a5f0 con 0x7fdd4415abe0 2023-11-08T16:29:01.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.940+0000 7fdd437fe640 1 --2- 172.21.15.81:0/2740500801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd440adbc0 0x7fdd4406a0b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdd380097a0 tx=0x7fdd3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4deacbe52c3675b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:01.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.940+0000 7fdd42ffd640 1 -- 172.21.15.81:0/2740500801 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38031830 con 0x7fdd440adbc0 2023-11-08T16:29:01.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.940+0000 7fdd42ffd640 1 -- 172.21.15.81:0/2740500801 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd38031990 con 0x7fdd440adbc0 2023-11-08T16:29:01.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.940+0000 7fdd42ffd640 1 -- 172.21.15.81:0/2740500801 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd38031d20 con 0x7fdd440adbc0 2023-11-08T16:29:01.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.941+0000 7fdd4949f640 1 -- 172.21.15.81:0/2740500801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd440adbc0 msgr2=0x7fdd4406a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:01.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.941+0000 7fdd4949f640 1 --2- 172.21.15.81:0/2740500801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd440adbc0 0x7fdd4406a0b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdd380097a0 tx=0x7fdd3802b8d0 comp rx=0 tx=0).stop 2023-11-08T16:29:01.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.941+0000 7fdd4949f640 1 -- 172.21.15.81:0/2740500801 shutdown_connections 2023-11-08T16:29:01.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.941+0000 7fdd4949f640 1 --2- 172.21.15.81:0/2740500801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd440adbc0 0x7fdd4406a0b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:01.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.941+0000 7fdd4949f640 1 -- 172.21.15.81:0/2740500801 >> 172.21.15.81:0/2740500801 conn(0x7fdd440ce840 msgr2=0x7fdd440cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:01.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.941+0000 7fdd4949f640 1 -- 172.21.15.81:0/2740500801 shutdown_connections 2023-11-08T16:29:01.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.941+0000 7fdd4949f640 1 -- 172.21.15.81:0/2740500801 wait complete. 2023-11-08T16:29:01.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.942+0000 7fdd4949f640 1 Processor -- start 2023-11-08T16:29:01.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.943+0000 7fdd4949f640 1 -- start start 2023-11-08T16:29:01.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.943+0000 7fdd4949f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdd4406f6b0 0x7fdd440c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:01.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd4949f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd440adbc0 0x7fdd440c9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:01.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd4949f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4415abe0 0x7fdd440c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:01.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd4949f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440c6b80 con 0x7fdd4415abe0 2023-11-08T16:29:01.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd4949f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440caf10 con 0x7fdd440adbc0 2023-11-08T16:29:01.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd4949f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd440cac30 con 0x7fdd4406f6b0 2023-11-08T16:29:01.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd48c9e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4415abe0 0x7fdd440c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:01.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd48c9e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4415abe0 0x7fdd440c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41608/0 (socket says 172.21.15.81:41608) 2023-11-08T16:29:01.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd48c9e640 1 -- 172.21.15.81:0/818774971 learned_addr learned my addr 172.21.15.81:0/818774971 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:01.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd43fff640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdd4406f6b0 0x7fdd440c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:01.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd437fe640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd440adbc0 0x7fdd440c9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:01.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd48c9e640 1 -- 172.21.15.81:0/818774971 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdd4406f6b0 msgr2=0x7fdd440c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:01.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd48c9e640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdd4406f6b0 0x7fdd440c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:01.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd48c9e640 1 -- 172.21.15.81:0/818774971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd440adbc0 msgr2=0x7fdd440c9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:01.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd48c9e640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd440adbc0 0x7fdd440c9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:01.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd48c9e640 1 -- 172.21.15.81:0/818774971 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd38009400 con 0x7fdd4415abe0 2023-11-08T16:29:01.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd43fff640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdd4406f6b0 0x7fdd440c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:01.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.944+0000 7fdd437fe640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd440adbc0 0x7fdd440c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:01.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.945+0000 7fdd48c9e640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4415abe0 0x7fdd440c6fa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00b970 tx=0x7fdd3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:01.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.945+0000 7fdd40ff9640 1 -- 172.21.15.81:0/818774971 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c00cb40 con 0x7fdd4415abe0 2023-11-08T16:29:01.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.945+0000 7fdd40ff9640 1 -- 172.21.15.81:0/818774971 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c00cca0 con 0x7fdd4415abe0 2023-11-08T16:29:01.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.945+0000 7fdd40ff9640 1 -- 172.21.15.81:0/818774971 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c014680 con 0x7fdd4415abe0 2023-11-08T16:29:01.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.945+0000 7fdd4949f640 1 -- 172.21.15.81:0/818774971 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd440c64c0 con 0x7fdd4415abe0 2023-11-08T16:29:01.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.945+0000 7fdd4949f640 1 -- 172.21.15.81:0/818774971 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd440ca1e0 con 0x7fdd4415abe0 2023-11-08T16:29:01.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.946+0000 7fdd4949f640 1 -- 172.21.15.81:0/818774971 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd440c6320 con 0x7fdd4415abe0 2023-11-08T16:29:01.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.947+0000 7fdd40ff9640 1 -- 172.21.15.81:0/818774971 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd3c01d3b0 con 0x7fdd4415abe0 2023-11-08T16:29:01.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.948+0000 7fdd40ff9640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd3006ae70 0x7fdd3006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:01.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.948+0000 7fdd40ff9640 1 -- 172.21.15.81:0/818774971 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fdd3c0892b0 con 0x7fdd4415abe0 2023-11-08T16:29:01.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.948+0000 7fdd43fff640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd3006ae70 0x7fdd3006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:01.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.949+0000 7fdd43fff640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd3006ae70 0x7fdd3006d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd34000f60 tx=0x7fdd34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:01.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:01.953+0000 7fdd40ff9640 1 -- 172.21.15.81:0/818774971 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdd440c6320 con 0x7fdd4415abe0 2023-11-08T16:29:02.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.144+0000 7fdd4949f640 1 -- 172.21.15.81:0/818774971 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdd440c6cc0 con 0x7fdd4415abe0 2023-11-08T16:29:02.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.705+0000 7fdd40ff9640 1 -- 172.21.15.81:0/818774971 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdd440c6cc0 con 0x7fdd4415abe0 2023-11-08T16:29:02.708 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' created 2023-11-08T16:29:02.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.708+0000 7fdd4949f640 1 -- 172.21.15.81:0/818774971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd3006ae70 msgr2=0x7fdd3006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:02.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.708+0000 7fdd4949f640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd3006ae70 0x7fdd3006d330 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fdd34000f60 tx=0x7fdd34009040 comp rx=0 tx=0).stop 2023-11-08T16:29:02.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.708+0000 7fdd4949f640 1 -- 172.21.15.81:0/818774971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4415abe0 msgr2=0x7fdd440c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:02.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.708+0000 7fdd4949f640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4415abe0 0x7fdd440c6fa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c00b970 tx=0x7fdd3c00be30 comp rx=0 tx=0).stop 2023-11-08T16:29:02.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.709+0000 7fdd4949f640 1 -- 172.21.15.81:0/818774971 shutdown_connections 2023-11-08T16:29:02.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.709+0000 7fdd4949f640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd3006ae70 0x7fdd3006d330 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:02.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.709+0000 7fdd4949f640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdd4415abe0 0x7fdd440c6fa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:02.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.709+0000 7fdd4949f640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdd440adbc0 0x7fdd440c9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:02.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.709+0000 7fdd4949f640 1 --2- 172.21.15.81:0/818774971 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdd4406f6b0 0x7fdd440c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:02.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.709+0000 7fdd4949f640 1 -- 172.21.15.81:0/818774971 >> 172.21.15.81:0/818774971 conn(0x7fdd440ce840 msgr2=0x7fdd44070210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:02.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.709+0000 7fdd4949f640 1 -- 172.21.15.81:0/818774971 shutdown_connections 2023-11-08T16:29:02.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.709+0000 7fdd4949f640 1 -- 172.21.15.81:0/818774971 wait complete. 2023-11-08T16:29:02.743 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T16:29:02.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.928+0000 7fc9aec7c640 1 Processor -- start 2023-11-08T16:29:02.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.928+0000 7fc9aec7c640 1 -- start start 2023-11-08T16:29:02.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.929+0000 7fc9aec7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 0x7fc9a80caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:02.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.929+0000 7fc9aec7c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc9a815bfc0 0x7fc9a815eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:02.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.929+0000 7fc9aec7c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a815f500 con 0x7fc9a815f0b0 2023-11-08T16:29:02.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.929+0000 7fc9aec7c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a80cda10 con 0x7fc9a815bfc0 2023-11-08T16:29:02.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.929+0000 7fc9aec7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a806fa30 con 0x7fc9a80ce0d0 2023-11-08T16:29:02.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.930+0000 7fc9aca18640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 0x7fc9a80caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:02.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.930+0000 7fc9a7fff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc9a815bfc0 0x7fc9a815eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:02.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.930+0000 7fc9aca18640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 0x7fc9a80caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:43304/0 (socket says 172.21.15.81:43304) 2023-11-08T16:29:02.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.930+0000 7fc9aca18640 1 -- 172.21.15.81:0/1210599923 learned_addr learned my addr 172.21.15.81:0/1210599923 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:02.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.930+0000 7fc9ad219640 1 --1- 172.21.15.81:0/1210599923 >> v1:172.21.15.93:6789/0 conn(0x7fc9a815f0b0 0x7fc9a81516e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50578/0 (socket says 172.21.15.81:50578) 2023-11-08T16:29:02.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.931+0000 7fc9aca18640 1 -- 172.21.15.81:0/1210599923 >> v2:172.21.15.93:3300/0 conn(0x7fc9a815bfc0 msgr2=0x7fc9a815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:02.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.931+0000 7fc9aca18640 1 --2- 172.21.15.81:0/1210599923 >> v2:172.21.15.93:3300/0 conn(0x7fc9a815bfc0 0x7fc9a815eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:02.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.931+0000 7fc9aca18640 1 -- 172.21.15.81:0/1210599923 >> v1:172.21.15.93:6789/0 conn(0x7fc9a815f0b0 legacy=0x7fc9a81516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:02.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.931+0000 7fc9aca18640 1 -- 172.21.15.81:0/1210599923 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a80ce4b0 con 0x7fc9a80ce0d0 2023-11-08T16:29:02.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.933+0000 7fc9aca18640 1 --2- 172.21.15.81:0/1210599923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 0x7fc9a80caf20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc998009870 tx=0x7fc99802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=858a20d748bdd7ab server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:02.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.933+0000 7fc9a77fe640 1 -- 172.21.15.81:0/1210599923 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998030840 con 0x7fc9a80ce0d0 2023-11-08T16:29:02.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.933+0000 7fc9a77fe640 1 -- 172.21.15.81:0/1210599923 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9980309a0 con 0x7fc9a80ce0d0 2023-11-08T16:29:02.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.933+0000 7fc9a77fe640 1 -- 172.21.15.81:0/1210599923 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc998030d30 con 0x7fc9a80ce0d0 2023-11-08T16:29:02.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.934+0000 7fc9aec7c640 1 -- 172.21.15.81:0/1210599923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 msgr2=0x7fc9a80caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:02.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.934+0000 7fc9aec7c640 1 --2- 172.21.15.81:0/1210599923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 0x7fc9a80caf20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc998009870 tx=0x7fc99802b980 comp rx=0 tx=0).stop 2023-11-08T16:29:02.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.934+0000 7fc9aec7c640 1 -- 172.21.15.81:0/1210599923 shutdown_connections 2023-11-08T16:29:02.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.934+0000 7fc9aec7c640 1 --2- 172.21.15.81:0/1210599923 >> v2:172.21.15.93:3300/0 conn(0x7fc9a815bfc0 0x7fc9a815eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:02.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.934+0000 7fc9aec7c640 1 --2- 172.21.15.81:0/1210599923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 0x7fc9a80caf20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:02.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.934+0000 7fc9aec7c640 1 -- 172.21.15.81:0/1210599923 >> 172.21.15.81:0/1210599923 conn(0x7fc9a80cb580 msgr2=0x7fc9a80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:02.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.934+0000 7fc9aec7c640 1 -- 172.21.15.81:0/1210599923 shutdown_connections 2023-11-08T16:29:02.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.934+0000 7fc9aec7c640 1 -- 172.21.15.81:0/1210599923 wait complete. 2023-11-08T16:29:02.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.936+0000 7fc9aec7c640 1 Processor -- start 2023-11-08T16:29:02.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.936+0000 7fc9aec7c640 1 -- start start 2023-11-08T16:29:02.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.937+0000 7fc9aec7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 0x7fc9a80b2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:02.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.937+0000 7fc9aec7c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9a815bfc0 0x7fc9a81f2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:02.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.937+0000 7fc9aec7c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9a815f0b0 0x7fc9a80c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:02.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.937+0000 7fc9aec7c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80c5d70 con 0x7fc9a815bfc0 2023-11-08T16:29:02.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.937+0000 7fc9aec7c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a80c4f20 con 0x7fc9a815f0b0 2023-11-08T16:29:02.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.937+0000 7fc9aec7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a8154d20 con 0x7fc9a80ce0d0 2023-11-08T16:29:02.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.938+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9a815bfc0 0x7fc9a81f2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:02.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.938+0000 7fc9a7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9a815bfc0 0x7fc9a81f2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41624/0 (socket says 172.21.15.81:41624) 2023-11-08T16:29:02.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.938+0000 7fc9a7fff640 1 -- 172.21.15.81:0/2913033166 learned_addr learned my addr 172.21.15.81:0/2913033166 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:02.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.938+0000 7fc9aca18640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 0x7fc9a80b2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:02.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.938+0000 7fc9a7fff640 1 -- 172.21.15.81:0/2913033166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 msgr2=0x7fc9a80b2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:02.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.938+0000 7fc9a7fff640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 0x7fc9a80b2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:02.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.938+0000 7fc9a7fff640 1 -- 172.21.15.81:0/2913033166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9a815f0b0 msgr2=0x7fc9a80c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:02.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.938+0000 7fc9ad219640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9a815f0b0 0x7fc9a80c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:02.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.938+0000 7fc9a7fff640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9a815f0b0 0x7fc9a80c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:02.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.938+0000 7fc9a7fff640 1 -- 172.21.15.81:0/2913033166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9980094d0 con 0x7fc9a815bfc0 2023-11-08T16:29:02.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.939+0000 7fc9a7fff640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9a815bfc0 0x7fc9a81f2470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00efc0 tx=0x7fc99c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:02.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.939+0000 7fc9a57fa640 1 -- 172.21.15.81:0/2913033166 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c009ca0 con 0x7fc9a815bfc0 2023-11-08T16:29:02.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.939+0000 7fc9a57fa640 1 -- 172.21.15.81:0/2913033166 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c00f040 con 0x7fc9a815bfc0 2023-11-08T16:29:02.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.939+0000 7fc9aec7c640 1 -- 172.21.15.81:0/2913033166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a80c6010 con 0x7fc9a815bfc0 2023-11-08T16:29:02.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.939+0000 7fc9a57fa640 1 -- 172.21.15.81:0/2913033166 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0146c0 con 0x7fc9a815bfc0 2023-11-08T16:29:02.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.939+0000 7fc9aec7c640 1 -- 172.21.15.81:0/2913033166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a80c65d0 con 0x7fc9a815bfc0 2023-11-08T16:29:02.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.940+0000 7fc9aec7c640 1 -- 172.21.15.81:0/2913033166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a80c67c0 con 0x7fc9a815bfc0 2023-11-08T16:29:02.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.941+0000 7fc9a57fa640 1 -- 172.21.15.81:0/2913033166 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc99c014820 con 0x7fc9a815bfc0 2023-11-08T16:29:02.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.942+0000 7fc9a57fa640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc99406ae70 0x7fc99406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:02.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.943+0000 7fc9a57fa640 1 -- 172.21.15.81:0/2913033166 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc99c08a600 con 0x7fc9a815bfc0 2023-11-08T16:29:02.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.943+0000 7fc9aca18640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc99406ae70 0x7fc99406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:02.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.943+0000 7fc9aca18640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc99406ae70 0x7fc99406d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc998036000 tx=0x7fc998008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:02.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:02.948+0000 7fc9a57fa640 1 -- 172.21.15.81:0/2913033166 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc99c055670 con 0x7fc9a815bfc0 2023-11-08T16:29:03.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.178+0000 7fc9aec7c640 1 -- 172.21.15.81:0/2913033166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc9a804ece0 con 0x7fc9a815bfc0 2023-11-08T16:29:03.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.773+0000 7fc9a57fa640 1 -- 172.21.15.81:0/2913033166 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc99c058c90 con 0x7fc9a815bfc0 2023-11-08T16:29:03.776 INFO:teuthology.orchestra.run.smithi081.stderr:new fs with metadata pool 2 and data pool 3 2023-11-08T16:29:03.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.777+0000 7fc9aec7c640 1 -- 172.21.15.81:0/2913033166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc99406ae70 msgr2=0x7fc99406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:03.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.777+0000 7fc9aec7c640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc99406ae70 0x7fc99406d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc998036000 tx=0x7fc998008030 comp rx=0 tx=0).stop 2023-11-08T16:29:03.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.777+0000 7fc9aec7c640 1 -- 172.21.15.81:0/2913033166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9a815bfc0 msgr2=0x7fc9a81f2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:03.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.777+0000 7fc9aec7c640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9a815bfc0 0x7fc9a81f2470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00efc0 tx=0x7fc99c00c490 comp rx=0 tx=0).stop 2023-11-08T16:29:03.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.777+0000 7fc9aec7c640 1 -- 172.21.15.81:0/2913033166 shutdown_connections 2023-11-08T16:29:03.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.777+0000 7fc9aec7c640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc99406ae70 0x7fc99406d330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:03.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.777+0000 7fc9aec7c640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9a815f0b0 0x7fc9a80c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:03.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.777+0000 7fc9aec7c640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9a815bfc0 0x7fc9a81f2470 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:03.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.777+0000 7fc9aec7c640 1 --2- 172.21.15.81:0/2913033166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9a80ce0d0 0x7fc9a80b2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:03.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.777+0000 7fc9aec7c640 1 -- 172.21.15.81:0/2913033166 >> 172.21.15.81:0/2913033166 conn(0x7fc9a80cb580 msgr2=0x7fc9a80af250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:03.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.778+0000 7fc9aec7c640 1 -- 172.21.15.81:0/2913033166 shutdown_connections 2023-11-08T16:29:03.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.778+0000 7fc9aec7c640 1 -- 172.21.15.81:0/2913033166 wait complete. 2023-11-08T16:29:03.804 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:03.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.984+0000 7facf10da640 1 Processor -- start 2023-11-08T16:29:03.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.984+0000 7facf10da640 1 -- start start 2023-11-08T16:29:03.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.986+0000 7facf10da640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7facec0d1af0 0x7facec0d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:03.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.986+0000 7facf10da640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7facec0b2c30 0x7facec0b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:03.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.986+0000 7facf10da640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec0b2450 con 0x7facec15d1a0 2023-11-08T16:29:03.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.986+0000 7facf10da640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec0ce800 con 0x7facec0b2c30 2023-11-08T16:29:03.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.986+0000 7facf10da640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec0ceae0 con 0x7facec0d1af0 2023-11-08T16:29:03.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.986+0000 7faceb59e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7facec0b2c30 0x7facec0b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:03.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.986+0000 7faceb59e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7facec0b2c30 0x7facec0b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41638/0 (socket says 172.21.15.81:41638) 2023-11-08T16:29:03.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.986+0000 7faceb59e640 1 -- 172.21.15.81:0/4087420181 learned_addr learned my addr 172.21.15.81:0/4087420181 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:03.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.987+0000 7faceb59e640 1 -- 172.21.15.81:0/4087420181 >> v2:172.21.15.93:3300/0 conn(0x7facec0d1af0 msgr2=0x7facec0d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:03.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.987+0000 7face3fff640 1 --2- 172.21.15.81:0/4087420181 >> v2:172.21.15.93:3300/0 conn(0x7facec0d1af0 0x7facec0d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:03.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.987+0000 7faceb59e640 1 --2- 172.21.15.81:0/4087420181 >> v2:172.21.15.93:3300/0 conn(0x7facec0d1af0 0x7facec0d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:03.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.987+0000 7faceb59e640 1 -- 172.21.15.81:0/4087420181 >> v1:172.21.15.93:6789/0 conn(0x7facec15d1a0 legacy=0x7facec0d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:03.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.987+0000 7faceb59e640 1 -- 172.21.15.81:0/4087420181 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec0b2710 con 0x7facec0b2c30 2023-11-08T16:29:03.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.987+0000 7face3fff640 1 --2- 172.21.15.81:0/4087420181 >> v2:172.21.15.93:3300/0 conn(0x7facec0d1af0 0x7facec0d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:03.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.988+0000 7faceb59e640 1 --2- 172.21.15.81:0/4087420181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facec0b2c30 0x7facec0b30c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7facec0b2640 tx=0x7face402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7398d6e8a02b681 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:03.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.988+0000 7facea59c640 1 -- 172.21.15.81:0/4087420181 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face402ca70 con 0x7facec0b2c30 2023-11-08T16:29:03.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.988+0000 7facea59c640 1 -- 172.21.15.81:0/4087420181 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face402cbd0 con 0x7facec0b2c30 2023-11-08T16:29:03.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.988+0000 7facea59c640 1 -- 172.21.15.81:0/4087420181 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40343f0 con 0x7facec0b2c30 2023-11-08T16:29:03.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.989+0000 7facf10da640 1 -- 172.21.15.81:0/4087420181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facec0b2c30 msgr2=0x7facec0b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:03.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.989+0000 7facf10da640 1 --2- 172.21.15.81:0/4087420181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facec0b2c30 0x7facec0b30c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7facec0b2640 tx=0x7face402bed0 comp rx=0 tx=0).stop 2023-11-08T16:29:03.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.989+0000 7facf10da640 1 -- 172.21.15.81:0/4087420181 shutdown_connections 2023-11-08T16:29:03.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.989+0000 7facf10da640 1 --2- 172.21.15.81:0/4087420181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facec0b2c30 0x7facec0b30c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:03.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.989+0000 7facf10da640 1 --2- 172.21.15.81:0/4087420181 >> v2:172.21.15.93:3300/0 conn(0x7facec0d1af0 0x7facec0d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:03.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.989+0000 7facf10da640 1 -- 172.21.15.81:0/4087420181 >> 172.21.15.81:0/4087420181 conn(0x7facec0d2e00 msgr2=0x7facec0d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:03.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.989+0000 7facf10da640 1 -- 172.21.15.81:0/4087420181 shutdown_connections 2023-11-08T16:29:03.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.989+0000 7facf10da640 1 -- 172.21.15.81:0/4087420181 wait complete. 2023-11-08T16:29:03.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.990+0000 7facf10da640 1 Processor -- start 2023-11-08T16:29:03.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.991+0000 7facf10da640 1 -- start start 2023-11-08T16:29:03.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7facf10da640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facec0b2c30 0x7facec0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:03.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7facf10da640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facec0d1af0 0x7facec14f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:03.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7facf10da640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facec15d1a0 0x7facec0c6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7facf10da640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec0b0f70 con 0x7facec0d1af0 2023-11-08T16:29:04.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7facf10da640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec0c9860 con 0x7facec0b2c30 2023-11-08T16:29:04.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7facf10da640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec14f710 con 0x7facec15d1a0 2023-11-08T16:29:04.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7face3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facec0d1af0 0x7facec14f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7face3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facec0d1af0 0x7facec14f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41644/0 (socket says 172.21.15.81:41644) 2023-11-08T16:29:04.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7face3fff640 1 -- 172.21.15.81:0/244123958 learned_addr learned my addr 172.21.15.81:0/244123958 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:04.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7faceb59e640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facec15d1a0 0x7facec0c6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7facead9d640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facec0b2c30 0x7facec0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7face3fff640 1 -- 172.21.15.81:0/244123958 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facec15d1a0 msgr2=0x7facec0c6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7face3fff640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facec15d1a0 0x7facec0c6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7face3fff640 1 -- 172.21.15.81:0/244123958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facec0b2c30 msgr2=0x7facec0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7face3fff640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facec0b2c30 0x7facec0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7face3fff640 1 -- 172.21.15.81:0/244123958 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face4009b30 con 0x7facec0d1af0 2023-11-08T16:29:04.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.992+0000 7facead9d640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facec0b2c30 0x7facec0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:04.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.993+0000 7face3fff640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facec0d1af0 0x7facec14f1d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7facd400c7f0 tx=0x7facd400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:04.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.993+0000 7face37fe640 1 -- 172.21.15.81:0/244123958 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4007820 con 0x7facec0d1af0 2023-11-08T16:29:04.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.993+0000 7face37fe640 1 -- 172.21.15.81:0/244123958 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd4007980 con 0x7facec0d1af0 2023-11-08T16:29:04.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.993+0000 7face37fe640 1 -- 172.21.15.81:0/244123958 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd4007d10 con 0x7facec0d1af0 2023-11-08T16:29:04.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.993+0000 7facf10da640 1 -- 172.21.15.81:0/244123958 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec14fc00 con 0x7facec0d1af0 2023-11-08T16:29:04.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.993+0000 7facf10da640 1 -- 172.21.15.81:0/244123958 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec0b8650 con 0x7facec0d1af0 2023-11-08T16:29:04.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.994+0000 7facf10da640 1 -- 172.21.15.81:0/244123958 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec0b1120 con 0x7facec0d1af0 2023-11-08T16:29:04.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.995+0000 7face37fe640 1 -- 172.21.15.81:0/244123958 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7facd40113f0 con 0x7facec0d1af0 2023-11-08T16:29:04.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.996+0000 7face37fe640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faccc06ae70 0x7faccc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.996+0000 7face37fe640 1 -- 172.21.15.81:0/244123958 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facd4089930 con 0x7facec0d1af0 2023-11-08T16:29:04.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.996+0000 7facead9d640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faccc06ae70 0x7faccc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:03.997+0000 7facead9d640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faccc06ae70 0x7faccc06d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7facdc003e60 tx=0x7facdc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:04.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.001+0000 7face37fe640 1 -- 172.21.15.81:0/244123958 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7facec0b1120 con 0x7facec0d1af0 2023-11-08T16:29:04.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.191+0000 7facf10da640 1 -- 172.21.15.81:0/244123958 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7facec04ece0 con 0x7facec0d1af0 2023-11-08T16:29:04.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.195+0000 7face37fe640 1 -- 172.21.15.81:0/244123958 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16458 (secure 0 0 0) 0x7facd401f090 con 0x7facec0d1af0 2023-11-08T16:29:04.198 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:04.198 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":20,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:03.180970+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:04.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 -- 172.21.15.81:0/244123958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faccc06ae70 msgr2=0x7faccc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faccc06ae70 0x7faccc06d330 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7facdc003e60 tx=0x7facdc009040 comp rx=0 tx=0).stop 2023-11-08T16:29:04.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 -- 172.21.15.81:0/244123958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facec0d1af0 msgr2=0x7facec14f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facec0d1af0 0x7facec14f1d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7facd400c7f0 tx=0x7facd400ccb0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 -- 172.21.15.81:0/244123958 shutdown_connections 2023-11-08T16:29:04.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faccc06ae70 0x7faccc06d330 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facec15d1a0 0x7facec0c6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facec0d1af0 0x7facec14f1d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 --2- 172.21.15.81:0/244123958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7facec0b2c30 0x7facec0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 -- 172.21.15.81:0/244123958 >> 172.21.15.81:0/244123958 conn(0x7facec0d2e00 msgr2=0x7facec0d13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:04.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 -- 172.21.15.81:0/244123958 shutdown_connections 2023-11-08T16:29:04.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.197+0000 7facf10da640 1 -- 172.21.15.81:0/244123958 wait complete. 2023-11-08T16:29:04.218 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.417+0000 7fa0f87f5640 1 Processor -- start 2023-11-08T16:29:04.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.417+0000 7fa0f87f5640 1 -- start start 2023-11-08T16:29:04.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f87f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f00ca480 0x7fa0f015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f87f5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa0f015ea60 0x7fa0f015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f87f5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f006b630 con 0x7fa0f00ade50 2023-11-08T16:29:04.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f87f5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f00b1860 con 0x7fa0f015ea60 2023-11-08T16:29:04.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f87f5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00ca8e0 con 0x7fa0f00ca480 2023-11-08T16:29:04.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f6d92640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa0f015ea60 0x7fa0f015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f6d92640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa0f015ea60 0x7fa0f015eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41652/0 (socket says 172.21.15.81:41652) 2023-11-08T16:29:04.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f6d92640 1 -- 172.21.15.81:0/58200959 learned_addr learned my addr 172.21.15.81:0/58200959 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:04.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f5d90640 1 --2- 172.21.15.81:0/58200959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f00ca480 0x7fa0f015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f6d92640 1 -- 172.21.15.81:0/58200959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f00ca480 msgr2=0x7fa0f015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:04.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f6d92640 1 --2- 172.21.15.81:0/58200959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f00ca480 0x7fa0f015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f6d92640 1 -- 172.21.15.81:0/58200959 >> v1:172.21.15.81:6789/0 conn(0x7fa0f00ade50 legacy=0x7fa0f015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f6d92640 1 -- 172.21.15.81:0/58200959 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f00cabb0 con 0x7fa0f015ea60 2023-11-08T16:29:04.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.419+0000 7fa0f5d90640 1 --2- 172.21.15.81:0/58200959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f00ca480 0x7fa0f015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:04.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.420+0000 7fa0f6d92640 1 --2- 172.21.15.81:0/58200959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f015ea60 0x7fa0f015eef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009870 tx=0x7fa0e802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6141c0f7c2c21a0c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:04.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.421+0000 7fa0f558f640 1 -- 172.21.15.81:0/58200959 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8030840 con 0x7fa0f015ea60 2023-11-08T16:29:04.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.421+0000 7fa0f558f640 1 -- 172.21.15.81:0/58200959 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e80309a0 con 0x7fa0f015ea60 2023-11-08T16:29:04.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.421+0000 7fa0f558f640 1 -- 172.21.15.81:0/58200959 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e8030d30 con 0x7fa0f015ea60 2023-11-08T16:29:04.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.421+0000 7fa0f87f5640 1 -- 172.21.15.81:0/58200959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f015ea60 msgr2=0x7fa0f015eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.421+0000 7fa0f87f5640 1 --2- 172.21.15.81:0/58200959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f015ea60 0x7fa0f015eef0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8009870 tx=0x7fa0e802b980 comp rx=0 tx=0).stop 2023-11-08T16:29:04.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.421+0000 7fa0f87f5640 1 -- 172.21.15.81:0/58200959 shutdown_connections 2023-11-08T16:29:04.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.421+0000 7fa0f87f5640 1 --2- 172.21.15.81:0/58200959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f015ea60 0x7fa0f015eef0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.421+0000 7fa0f87f5640 1 --2- 172.21.15.81:0/58200959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f00ca480 0x7fa0f015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.421+0000 7fa0f87f5640 1 -- 172.21.15.81:0/58200959 >> 172.21.15.81:0/58200959 conn(0x7fa0f00cb490 msgr2=0x7fa0f00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:04.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.421+0000 7fa0f87f5640 1 -- 172.21.15.81:0/58200959 shutdown_connections 2023-11-08T16:29:04.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.422+0000 7fa0f87f5640 1 -- 172.21.15.81:0/58200959 wait complete. 2023-11-08T16:29:04.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.423+0000 7fa0f87f5640 1 Processor -- start 2023-11-08T16:29:04.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.423+0000 7fa0f87f5640 1 -- start start 2023-11-08T16:29:04.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.424+0000 7fa0f87f5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f00ade50 0x7fa0f00cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.424+0000 7fa0f87f5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f00ca480 0x7fa0f01580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.424+0000 7fa0f87f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f015ea60 0x7fa0f01511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.424+0000 7fa0f87f5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00c36a0 con 0x7fa0f00ade50 2023-11-08T16:29:04.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.424+0000 7fa0f87f5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00cda40 con 0x7fa0f00ca480 2023-11-08T16:29:04.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.424+0000 7fa0f87f5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f00c2a90 con 0x7fa0f015ea60 2023-11-08T16:29:04.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.424+0000 7fa0f6591640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f00ade50 0x7fa0f00cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.424+0000 7fa0f6591640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f00ade50 0x7fa0f00cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41662/0 (socket says 172.21.15.81:41662) 2023-11-08T16:29:04.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.424+0000 7fa0f6591640 1 -- 172.21.15.81:0/3338017939 learned_addr learned my addr 172.21.15.81:0/3338017939 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:04.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0f6d92640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f015ea60 0x7fa0f01511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0f6591640 1 -- 172.21.15.81:0/3338017939 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f015ea60 msgr2=0x7fa0f01511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0f6591640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f015ea60 0x7fa0f01511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0f6591640 1 -- 172.21.15.81:0/3338017939 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f00ca480 msgr2=0x7fa0f01580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0f6591640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f00ca480 0x7fa0f01580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0f6591640 1 -- 172.21.15.81:0/3338017939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e80094d0 con 0x7fa0f00ade50 2023-11-08T16:29:04.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0f6591640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f00ade50 0x7fa0f00cd500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000c450 tx=0x7fa0e000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:04.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0eeffd640 1 -- 172.21.15.81:0/3338017939 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e00052d0 con 0x7fa0f00ade50 2023-11-08T16:29:04.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0eeffd640 1 -- 172.21.15.81:0/3338017939 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e0005430 con 0x7fa0f00ade50 2023-11-08T16:29:04.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0eeffd640 1 -- 172.21.15.81:0/3338017939 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e0012870 con 0x7fa0f00ade50 2023-11-08T16:29:04.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0f87f5640 1 -- 172.21.15.81:0/3338017939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f00c2d70 con 0x7fa0f00ade50 2023-11-08T16:29:04.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.425+0000 7fa0f87f5640 1 -- 172.21.15.81:0/3338017939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f00c3330 con 0x7fa0f00ade50 2023-11-08T16:29:04.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.427+0000 7fa0f87f5640 1 -- 172.21.15.81:0/3338017939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f00cb150 con 0x7fa0f00ade50 2023-11-08T16:29:04.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.428+0000 7fa0eeffd640 1 -- 172.21.15.81:0/3338017939 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0e00129d0 con 0x7fa0f00ade50 2023-11-08T16:29:04.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.428+0000 7fa0eeffd640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa0dc06ae70 0x7fa0dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.428+0000 7fa0eeffd640 1 -- 172.21.15.81:0/3338017939 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0e00890c0 con 0x7fa0f00ade50 2023-11-08T16:29:04.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.429+0000 7fa0f5d90640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa0dc06ae70 0x7fa0dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.429+0000 7fa0f5d90640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa0dc06ae70 0x7fa0dc06d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa0f01e5c90 tx=0x7fa0e40083d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:04.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.432+0000 7fa0eeffd640 1 -- 172.21.15.81:0/3338017939 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0f00cb150 con 0x7fa0f00ade50 2023-11-08T16:29:04.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.624+0000 7fa0f87f5640 1 -- 172.21.15.81:0/3338017939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0f00cb150 con 0x7fa0f00ade50 2023-11-08T16:29:04.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.626+0000 7fa0eeffd640 1 -- 172.21.15.81:0/3338017939 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16458 (secure 0 0 0) 0x7fa0f00cb150 con 0x7fa0f00ade50 2023-11-08T16:29:04.629 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:04.630 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":20,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:03.180970+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:04.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.629+0000 7fa0f87f5640 1 -- 172.21.15.81:0/3338017939 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa0dc06ae70 msgr2=0x7fa0dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.629+0000 7fa0f87f5640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa0dc06ae70 0x7fa0dc06d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa0f01e5c90 tx=0x7fa0e40083d0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.629+0000 7fa0f87f5640 1 -- 172.21.15.81:0/3338017939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f00ade50 msgr2=0x7fa0f00cd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.629+0000 7fa0f87f5640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f00ade50 0x7fa0f00cd500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000c450 tx=0x7fa0e000c910 comp rx=0 tx=0).stop 2023-11-08T16:29:04.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.629+0000 7fa0f87f5640 1 -- 172.21.15.81:0/3338017939 shutdown_connections 2023-11-08T16:29:04.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.629+0000 7fa0f87f5640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa0dc06ae70 0x7fa0dc06d330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.629+0000 7fa0f87f5640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0f015ea60 0x7fa0f01511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.629+0000 7fa0f87f5640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f00ca480 0x7fa0f01580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.629+0000 7fa0f87f5640 1 --2- 172.21.15.81:0/3338017939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0f00ade50 0x7fa0f00cd500 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.629+0000 7fa0f87f5640 1 -- 172.21.15.81:0/3338017939 >> 172.21.15.81:0/3338017939 conn(0x7fa0f00cb490 msgr2=0x7fa0f00b2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:04.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.630+0000 7fa0f87f5640 1 -- 172.21.15.81:0/3338017939 shutdown_connections 2023-11-08T16:29:04.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.630+0000 7fa0f87f5640 1 -- 172.21.15.81:0/3338017939 wait complete. 2023-11-08T16:29:04.652 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T16:29:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.855+0000 7f9f2f24f640 1 Processor -- start 2023-11-08T16:29:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.855+0000 7f9f2f24f640 1 -- start start 2023-11-08T16:29:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.856+0000 7f9f2f24f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9f280ccc10 0x7f9f280ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.856+0000 7f9f2f24f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f2806b710 0x7f9f2806bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.856+0000 7f9f2f24f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f280ce960 con 0x7f9f2815a0e0 2023-11-08T16:29:04.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.856+0000 7f9f2f24f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f28071980 con 0x7f9f280ccc10 2023-11-08T16:29:04.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.856+0000 7f9f2f24f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280c9480 con 0x7f9f2806b710 2023-11-08T16:29:04.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.857+0000 7f9f2d7ec640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f2806b710 0x7f9f2806bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.857+0000 7f9f2cfeb640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f9f2815a0e0 0x7f9f280c9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50592/0 (socket says 172.21.15.81:50592) 2023-11-08T16:29:04.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.857+0000 7f9f27fff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9f280ccc10 0x7f9f280ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.857+0000 7f9f2cfeb640 1 -- 172.21.15.81:0/3616999354 learned_addr learned my addr 172.21.15.81:0/3616999354 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:04.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.858+0000 7f9f277fe640 1 -- 172.21.15.81:0/3616999354 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73551717 0 0) 0x7f9f1c001110 con 0x7f9f2815a0e0 2023-11-08T16:29:04.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.858+0000 7f9f277fe640 1 -- 172.21.15.81:0/3616999354 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f18003520 con 0x7f9f2815a0e0 2023-11-08T16:29:04.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.858+0000 7f9f27fff640 1 -- 172.21.15.81:0/3616999354 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f2806b710 msgr2=0x7f9f2806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:04.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.858+0000 7f9f27fff640 1 --2- 172.21.15.81:0/3616999354 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f2806b710 0x7f9f2806bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.858+0000 7f9f27fff640 1 -- 172.21.15.81:0/3616999354 >> v1:172.21.15.93:6789/0 conn(0x7f9f2815a0e0 legacy=0x7f9f280c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.858+0000 7f9f27fff640 1 -- 172.21.15.81:0/3616999354 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2815a4c0 con 0x7f9f280ccc10 2023-11-08T16:29:04.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.860+0000 7f9f27fff640 1 --2- 172.21.15.81:0/3616999354 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f280ccc10 0x7f9f280ca300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f140097a0 tx=0x7f9f1402b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f03379fd4651c44d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:04.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.860+0000 7f9f277fe640 1 -- 172.21.15.81:0/3616999354 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14030610 con 0x7f9f280ccc10 2023-11-08T16:29:04.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.860+0000 7f9f277fe640 1 -- 172.21.15.81:0/3616999354 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f14030770 con 0x7f9f280ccc10 2023-11-08T16:29:04.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.860+0000 7f9f277fe640 1 -- 172.21.15.81:0/3616999354 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f14030ae0 con 0x7f9f280ccc10 2023-11-08T16:29:04.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.861+0000 7f9f2f24f640 1 -- 172.21.15.81:0/3616999354 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f280ccc10 msgr2=0x7f9f280ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.861+0000 7f9f2f24f640 1 --2- 172.21.15.81:0/3616999354 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f280ccc10 0x7f9f280ca300 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9f140097a0 tx=0x7f9f1402b960 comp rx=0 tx=0).stop 2023-11-08T16:29:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.861+0000 7f9f2f24f640 1 -- 172.21.15.81:0/3616999354 shutdown_connections 2023-11-08T16:29:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.861+0000 7f9f2f24f640 1 --2- 172.21.15.81:0/3616999354 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f2806b710 0x7f9f2806bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.861+0000 7f9f2f24f640 1 --2- 172.21.15.81:0/3616999354 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f280ccc10 0x7f9f280ca300 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.861+0000 7f9f2f24f640 1 -- 172.21.15.81:0/3616999354 >> 172.21.15.81:0/3616999354 conn(0x7f9f280cb580 msgr2=0x7f9f280cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.861+0000 7f9f2f24f640 1 -- 172.21.15.81:0/3616999354 shutdown_connections 2023-11-08T16:29:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.861+0000 7f9f2f24f640 1 -- 172.21.15.81:0/3616999354 wait complete. 2023-11-08T16:29:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.863+0000 7f9f2f24f640 1 Processor -- start 2023-11-08T16:29:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.863+0000 7f9f2f24f640 1 -- start start 2023-11-08T16:29:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2f24f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2806b710 0x7f9f280b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2f24f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f280ccc10 0x7f9f2814c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2f24f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f2815a0e0 0x7f9f2814cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2f24f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2814c990 con 0x7f9f2806b710 2023-11-08T16:29:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2f24f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280b2940 con 0x7f9f2815a0e0 2023-11-08T16:29:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2f24f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280b2c20 con 0x7f9f280ccc10 2023-11-08T16:29:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2cfeb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2806b710 0x7f9f280b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2cfeb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2806b710 0x7f9f280b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41670/0 (socket says 172.21.15.81:41670) 2023-11-08T16:29:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2cfeb640 1 -- 172.21.15.81:0/1766971110 learned_addr learned my addr 172.21.15.81:0/1766971110 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f27fff640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f280ccc10 0x7f9f2814c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2cfeb640 1 -- 172.21.15.81:0/1766971110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f280ccc10 msgr2=0x7f9f2814c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2cfeb640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f280ccc10 0x7f9f2814c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2cfeb640 1 -- 172.21.15.81:0/1766971110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f2815a0e0 msgr2=0x7f9f2814cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2d7ec640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f2815a0e0 0x7f9f2814cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2cfeb640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f2815a0e0 0x7f9f2814cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.864+0000 7f9f2cfeb640 1 -- 172.21.15.81:0/1766971110 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f14009400 con 0x7f9f2806b710 2023-11-08T16:29:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.865+0000 7f9f2cfeb640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2806b710 0x7f9f280b39e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00c700 tx=0x7f9f1c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.865+0000 7f9f257fa640 1 -- 172.21.15.81:0/1766971110 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c005ab0 con 0x7f9f2806b710 2023-11-08T16:29:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.865+0000 7f9f257fa640 1 -- 172.21.15.81:0/1766971110 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f1c005c10 con 0x7f9f2806b710 2023-11-08T16:29:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.865+0000 7f9f257fa640 1 -- 172.21.15.81:0/1766971110 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c001d50 con 0x7f9f2806b710 2023-11-08T16:29:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.865+0000 7f9f2f24f640 1 -- 172.21.15.81:0/1766971110 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f280b2f00 con 0x7f9f2806b710 2023-11-08T16:29:04.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.865+0000 7f9f2f24f640 1 -- 172.21.15.81:0/1766971110 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f280b34c0 con 0x7f9f2806b710 2023-11-08T16:29:04.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.866+0000 7f9f2f24f640 1 -- 172.21.15.81:0/1766971110 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f2804ece0 con 0x7f9f2806b710 2023-11-08T16:29:04.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.868+0000 7f9f257fa640 1 -- 172.21.15.81:0/1766971110 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f1c007070 con 0x7f9f2806b710 2023-11-08T16:29:04.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.869+0000 7f9f257fa640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f0c06ae70 0x7f9f0c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:04.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.869+0000 7f9f257fa640 1 -- 172.21.15.81:0/1766971110 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9f1c088a00 con 0x7f9f2806b710 2023-11-08T16:29:04.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.869+0000 7f9f27fff640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f0c06ae70 0x7f9f0c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:04.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.870+0000 7f9f27fff640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f0c06ae70 0x7f9f0c06d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f14036000 tx=0x7f9f14008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:04.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:04.875+0000 7f9f257fa640 1 -- 172.21.15.81:0/1766971110 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f1c053ab0 con 0x7f9f2806b710 2023-11-08T16:29:05.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.101+0000 7f9f2f24f640 1 -- 172.21.15.81:0/1766971110 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9f280ce960 con 0x7f9f2806b710 2023-11-08T16:29:05.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.786+0000 7f9f257fa640 1 -- 172.21.15.81:0/1766971110 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9f280ce960 con 0x7f9f2806b710 2023-11-08T16:29:05.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 -- 172.21.15.81:0/1766971110 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f0c06ae70 msgr2=0x7f9f0c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f0c06ae70 0x7f9f0c06d330 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f9f14036000 tx=0x7f9f14008030 comp rx=0 tx=0).stop 2023-11-08T16:29:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 -- 172.21.15.81:0/1766971110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2806b710 msgr2=0x7f9f280b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2806b710 0x7f9f280b39e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c00c700 tx=0x7f9f1c00cbc0 comp rx=0 tx=0).stop 2023-11-08T16:29:05.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 -- 172.21.15.81:0/1766971110 shutdown_connections 2023-11-08T16:29:05.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f0c06ae70 0x7f9f0c06d330 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:05.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f2815a0e0 0x7f9f2814cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:05.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f280ccc10 0x7f9f2814c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 --2- 172.21.15.81:0/1766971110 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f2806b710 0x7f9f280b39e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 -- 172.21.15.81:0/1766971110 >> 172.21.15.81:0/1766971110 conn(0x7f9f280cb580 msgr2=0x7f9f2815a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 -- 172.21.15.81:0/1766971110 shutdown_connections 2023-11-08T16:29:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:05.791+0000 7f9f2f24f640 1 -- 172.21.15.81:0/1766971110 wait complete. 2023-11-08T16:29:05.815 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T16:29:05.815 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T16:29:06.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.024+0000 7f9408fa5640 1 Processor -- start 2023-11-08T16:29:06.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.024+0000 7f9408fa5640 1 -- start start 2023-11-08T16:29:06.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.026+0000 7f9408fa5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f940415d4c0 0x7f94040cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:06.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.026+0000 7f9408fa5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f94040cfcc0 0x7f94040b6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:06.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.026+0000 7f9408fa5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94040cef70 con 0x7f94040d39a0 2023-11-08T16:29:06.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.026+0000 7f9408fa5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94040cec90 con 0x7f940415d4c0 2023-11-08T16:29:06.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.026+0000 7f9408fa5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94040b5e70 con 0x7f94040cfcc0 2023-11-08T16:29:06.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.026+0000 7f940259c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f940415d4c0 0x7f94040cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:06.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.026+0000 7f940259c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f940415d4c0 0x7f94040cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41674/0 (socket says 172.21.15.81:41674) 2023-11-08T16:29:06.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.026+0000 7f940259c640 1 -- 172.21.15.81:0/2166377308 learned_addr learned my addr 172.21.15.81:0/2166377308 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:06.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.027+0000 7f9402d9d640 1 --2- 172.21.15.81:0/2166377308 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f94040cfcc0 0x7f94040b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:06.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.027+0000 7f940259c640 1 -- 172.21.15.81:0/2166377308 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f94040cfcc0 msgr2=0x7f94040b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:06.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.027+0000 7f940259c640 1 --2- 172.21.15.81:0/2166377308 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f94040cfcc0 0x7f94040b6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:06.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.027+0000 7f940259c640 1 -- 172.21.15.81:0/2166377308 >> v1:172.21.15.93:6789/0 conn(0x7f94040d39a0 legacy=0x7f94040cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:06.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.027+0000 7f940259c640 1 -- 172.21.15.81:0/2166377308 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94040b6150 con 0x7f940415d4c0 2023-11-08T16:29:06.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.028+0000 7f940259c640 1 --2- 172.21.15.81:0/2166377308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f940415d4c0 0x7f94040cdb40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009870 tx=0x7f93f402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed608e6506ca9dd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:06.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.028+0000 7f940159a640 1 -- 172.21.15.81:0/2166377308 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4030840 con 0x7f940415d4c0 2023-11-08T16:29:06.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.028+0000 7f940159a640 1 -- 172.21.15.81:0/2166377308 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f40309a0 con 0x7f940415d4c0 2023-11-08T16:29:06.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.028+0000 7f940159a640 1 -- 172.21.15.81:0/2166377308 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4030d30 con 0x7f940415d4c0 2023-11-08T16:29:06.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.029+0000 7f9408fa5640 1 -- 172.21.15.81:0/2166377308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f940415d4c0 msgr2=0x7f94040cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:06.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.029+0000 7f9408fa5640 1 --2- 172.21.15.81:0/2166377308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f940415d4c0 0x7f94040cdb40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009870 tx=0x7f93f402b980 comp rx=0 tx=0).stop 2023-11-08T16:29:06.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.029+0000 7f9408fa5640 1 -- 172.21.15.81:0/2166377308 shutdown_connections 2023-11-08T16:29:06.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.029+0000 7f9408fa5640 1 --2- 172.21.15.81:0/2166377308 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f94040cfcc0 0x7f94040b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:06.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.029+0000 7f9408fa5640 1 --2- 172.21.15.81:0/2166377308 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f940415d4c0 0x7f94040cdb40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:06.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.029+0000 7f9408fa5640 1 -- 172.21.15.81:0/2166377308 >> 172.21.15.81:0/2166377308 conn(0x7f94040d2e80 msgr2=0x7f94040d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:06.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.029+0000 7f9408fa5640 1 -- 172.21.15.81:0/2166377308 shutdown_connections 2023-11-08T16:29:06.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.029+0000 7f9408fa5640 1 -- 172.21.15.81:0/2166377308 wait complete. 2023-11-08T16:29:06.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.031+0000 7f9408fa5640 1 Processor -- start 2023-11-08T16:29:06.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.031+0000 7f9408fa5640 1 -- start start 2023-11-08T16:29:06.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9408fa5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94040cfcc0 0x7f9404159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:06.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9408fa5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f94040d39a0 0x7f9404156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:06.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9408fa5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f940415d4c0 0x7f94041f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:06.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9408fa5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94040c8820 con 0x7f940415d4c0 2023-11-08T16:29:06.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9408fa5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94040c7c10 con 0x7f94040cfcc0 2023-11-08T16:29:06.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9408fa5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94040c7ef0 con 0x7f94040d39a0 2023-11-08T16:29:06.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9402d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f940415d4c0 0x7f94041f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:06.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9402d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f940415d4c0 0x7f94041f5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41688/0 (socket says 172.21.15.81:41688) 2023-11-08T16:29:06.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9402d9d640 1 -- 172.21.15.81:0/3260119590 learned_addr learned my addr 172.21.15.81:0/3260119590 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:06.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9402d9d640 1 -- 172.21.15.81:0/3260119590 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f94040d39a0 msgr2=0x7f9404156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:06.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9401d9b640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f94040d39a0 0x7f9404156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:06.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9402d9d640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f94040d39a0 0x7f9404156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:06.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9402d9d640 1 -- 172.21.15.81:0/3260119590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94040cfcc0 msgr2=0x7f9404159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:06.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9402d9d640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94040cfcc0 0x7f9404159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:06.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.032+0000 7f9402d9d640 1 -- 172.21.15.81:0/3260119590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f40094d0 con 0x7f940415d4c0 2023-11-08T16:29:06.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.033+0000 7f9402d9d640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f940415d4c0 0x7f94041f5660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00d800 tx=0x7f93fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:06.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.033+0000 7f93faffd640 1 -- 172.21.15.81:0/3260119590 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc00b6c0 con 0x7f940415d4c0 2023-11-08T16:29:06.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.033+0000 7f93faffd640 1 -- 172.21.15.81:0/3260119590 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93fc00b840 con 0x7f940415d4c0 2023-11-08T16:29:06.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.033+0000 7f9408fa5640 1 -- 172.21.15.81:0/3260119590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94040c81d0 con 0x7f940415d4c0 2023-11-08T16:29:06.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.033+0000 7f93faffd640 1 -- 172.21.15.81:0/3260119590 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93fc00bbf0 con 0x7f940415d4c0 2023-11-08T16:29:06.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.033+0000 7f9408fa5640 1 -- 172.21.15.81:0/3260119590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9404159890 con 0x7f940415d4c0 2023-11-08T16:29:06.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.034+0000 7f9408fa5640 1 -- 172.21.15.81:0/3260119590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94040d2860 con 0x7f940415d4c0 2023-11-08T16:29:06.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.035+0000 7f93faffd640 1 -- 172.21.15.81:0/3260119590 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f93fc00bd50 con 0x7f940415d4c0 2023-11-08T16:29:06.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.036+0000 7f93faffd640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93e406ae70 0x7f93e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:06.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.036+0000 7f93faffd640 1 -- 172.21.15.81:0/3260119590 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93fc089bb0 con 0x7f940415d4c0 2023-11-08T16:29:06.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.036+0000 7f940259c640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93e406ae70 0x7f93e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:06.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.037+0000 7f940259c640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93e406ae70 0x7f93e406d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f93f4036000 tx=0x7f93f403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:06.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.040+0000 7f93faffd640 1 -- 172.21.15.81:0/3260119590 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93fc054c50 con 0x7f940415d4c0 2023-11-08T16:29:06.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.269+0000 7f9408fa5640 1 -- 172.21.15.81:0/3260119590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f94040c8960 con 0x7f940415d4c0 2023-11-08T16:29:06.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.797+0000 7f93faffd640 1 -- 172.21.15.81:0/3260119590 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f94040c8960 con 0x7f940415d4c0 2023-11-08T16:29:06.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.802+0000 7f9408fa5640 1 -- 172.21.15.81:0/3260119590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93e406ae70 msgr2=0x7f93e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:06.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.802+0000 7f9408fa5640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93e406ae70 0x7f93e406d330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f93f4036000 tx=0x7f93f403b040 comp rx=0 tx=0).stop 2023-11-08T16:29:06.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.802+0000 7f9408fa5640 1 -- 172.21.15.81:0/3260119590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f940415d4c0 msgr2=0x7f94041f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:06.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.802+0000 7f9408fa5640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f940415d4c0 0x7f94041f5660 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f93fc00d800 tx=0x7f93fc00dcc0 comp rx=0 tx=0).stop 2023-11-08T16:29:06.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.802+0000 7f9408fa5640 1 -- 172.21.15.81:0/3260119590 shutdown_connections 2023-11-08T16:29:06.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.802+0000 7f9408fa5640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93e406ae70 0x7f93e406d330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:06.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.802+0000 7f9408fa5640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f940415d4c0 0x7f94041f5660 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:06.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.802+0000 7f9408fa5640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f94040d39a0 0x7f9404156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:06.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.802+0000 7f9408fa5640 1 --2- 172.21.15.81:0/3260119590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94040cfcc0 0x7f9404159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:06.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.802+0000 7f9408fa5640 1 -- 172.21.15.81:0/3260119590 >> 172.21.15.81:0/3260119590 conn(0x7f94040d2e80 msgr2=0x7f94040b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:06.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.803+0000 7f9408fa5640 1 -- 172.21.15.81:0/3260119590 shutdown_connections 2023-11-08T16:29:06.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:06.803+0000 7f9408fa5640 1 -- 172.21.15.81:0/3260119590 wait complete. 2023-11-08T16:29:06.825 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T16:29:06.825 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:29:07.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.034+0000 7f59f18bb640 1 Processor -- start 2023-11-08T16:29:07.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.034+0000 7f59f18bb640 1 -- start start 2023-11-08T16:29:07.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.036+0000 7f59f18bb640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f59ec0cc5e0 0x7f59ec0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.036+0000 7f59f18bb640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec0c9a70 con 0x7f59ec0ccfb0 2023-11-08T16:29:07.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.036+0000 7f59f18bb640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59ec0cc3e0 con 0x7f59ec0adcd0 2023-11-08T16:29:07.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.036+0000 7f59f18bb640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59ec06d070 con 0x7f59ec0cc5e0 2023-11-08T16:29:07.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.036+0000 7f59eb7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f59ec0ccfb0 0x7f59ec070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58782/0 (socket says 172.21.15.81:58782) 2023-11-08T16:29:07.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.036+0000 7f59eb7fe640 1 -- 172.21.15.81:0/2297951288 learned_addr learned my addr 172.21.15.81:0/2297951288 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:07.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.036+0000 7f59e27fc640 1 --2- 172.21.15.81:0/2297951288 >> v2:172.21.15.93:3300/0 conn(0x7f59ec0cc5e0 0x7f59ec0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.036+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1663740507 0 0) 0x7f59ec0c9a70 con 0x7f59ec0ccfb0 2023-11-08T16:29:07.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.036+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59d8005730 con 0x7f59ec0ccfb0 2023-11-08T16:29:07.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2336063681 0 0) 0x7f59d8005730 con 0x7f59ec0ccfb0 2023-11-08T16:29:07.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 >> v2:172.21.15.93:3300/0 conn(0x7f59ec0cc5e0 msgr2=0x7f59ec0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:07.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 --2- 172.21.15.81:0/2297951288 >> v2:172.21.15.93:3300/0 conn(0x7f59ec0cc5e0 0x7f59ec0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 >> v1:172.21.15.93:6789/0 conn(0x7f59ec0adcd0 legacy=0x7f59ec0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec06d350 con 0x7f59ec0ccfb0 2023-11-08T16:29:07.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f59e40019f0 con 0x7f59ec0ccfb0 2023-11-08T16:29:07.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 >> v1:172.21.15.81:6789/0 conn(0x7f59ec0ccfb0 legacy=0x7f59ec070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 --2- 172.21.15.81:0/2297951288 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d8005cb0 0x7f59d80060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 --2- 172.21.15.81:0/2297951288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d80069d0 0x7f59d8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 --2- 172.21.15.81:0/2297951288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d80074b0 0x7f59d800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec06d070 con 0x7f59d80069d0 2023-11-08T16:29:07.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59d800e360 con 0x7f59d80074b0 2023-11-08T16:29:07.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59d800e520 con 0x7f59d8005cb0 2023-11-08T16:29:07.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59e27fc640 1 --2- 172.21.15.81:0/2297951288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d80069d0 0x7f59d8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.037+0000 7f59eaffd640 1 --2- 172.21.15.81:0/2297951288 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d8005cb0 0x7f59d80060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.038+0000 7f59e27fc640 1 -- 172.21.15.81:0/2297951288 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d8005cb0 msgr2=0x7f59d80060d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:07.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.038+0000 7f59e27fc640 1 --2- 172.21.15.81:0/2297951288 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59d8005cb0 0x7f59d80060d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.038+0000 7f59e27fc640 1 -- 172.21.15.81:0/2297951288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d80074b0 msgr2=0x7f59d800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:07.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.038+0000 7f59e27fc640 1 --2- 172.21.15.81:0/2297951288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d80074b0 0x7f59d800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.038+0000 7f59e27fc640 1 -- 172.21.15.81:0/2297951288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d8005730 con 0x7f59d80069d0 2023-11-08T16:29:07.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.038+0000 7f59eb7fe640 1 -- 172.21.15.81:0/2297951288 reap_dead start 2023-11-08T16:29:07.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.039+0000 7f59e27fc640 1 --2- 172.21.15.81:0/2297951288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d80069d0 0x7f59d8006e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009a90 tx=0x7f59d8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d00dae1f241197dc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:07.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.039+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d402c6b0 con 0x7f59d80069d0 2023-11-08T16:29:07.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.039+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d402c810 con 0x7f59d80069d0 2023-11-08T16:29:07.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.039+0000 7f59ea7fc640 1 -- 172.21.15.81:0/2297951288 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d402cba0 con 0x7f59d80069d0 2023-11-08T16:29:07.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.039+0000 7f59f18bb640 1 -- 172.21.15.81:0/2297951288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d80069d0 msgr2=0x7f59d8006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.039+0000 7f59f18bb640 1 --2- 172.21.15.81:0/2297951288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d80069d0 0x7f59d8006e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009a90 tx=0x7f59d8007a00 comp rx=0 tx=0).stop 2023-11-08T16:29:07.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.040+0000 7f59f18bb640 1 -- 172.21.15.81:0/2297951288 shutdown_connections 2023-11-08T16:29:07.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.040+0000 7f59f18bb640 1 --2- 172.21.15.81:0/2297951288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59d80069d0 0x7f59d8006e50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.040+0000 7f59f18bb640 1 -- 172.21.15.81:0/2297951288 >> 172.21.15.81:0/2297951288 conn(0x7f59ec0cb490 msgr2=0x7f59ec0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:07.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.040+0000 7f59f18bb640 1 -- 172.21.15.81:0/2297951288 shutdown_connections 2023-11-08T16:29:07.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.040+0000 7f59f18bb640 1 -- 172.21.15.81:0/2297951288 wait complete. 2023-11-08T16:29:07.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.041+0000 7f59f18bb640 1 Processor -- start 2023-11-08T16:29:07.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.042+0000 7f59f18bb640 1 -- start start 2023-11-08T16:29:07.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59f18bb640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d80069d0 0x7f59ec0b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59f18bb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ec0b4120 0x7f59ec158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59f18bb640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59ec070cf0 0x7f59ec071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59f18bb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec0b37e0 con 0x7f59ec0b4120 2023-11-08T16:29:07.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59f18bb640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec1e7df0 con 0x7f59d80069d0 2023-11-08T16:29:07.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59f18bb640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59ec0716b0 con 0x7f59ec070cf0 2023-11-08T16:29:07.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59e27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ec0b4120 0x7f59ec158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59e27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ec0b4120 0x7f59ec158fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41702/0 (socket says 172.21.15.81:41702) 2023-11-08T16:29:07.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59e27fc640 1 -- 172.21.15.81:0/3965373554 learned_addr learned my addr 172.21.15.81:0/3965373554 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:07.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59e27fc640 1 -- 172.21.15.81:0/3965373554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59ec070cf0 msgr2=0x7f59ec071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59e27fc640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59ec070cf0 0x7f59ec071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59e27fc640 1 -- 172.21.15.81:0/3965373554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d80069d0 msgr2=0x7f59ec0b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59e27fc640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d80069d0 0x7f59ec0b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59e27fc640 1 -- 172.21.15.81:0/3965373554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59ec0b2b60 con 0x7f59ec0b4120 2023-11-08T16:29:07.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.043+0000 7f59e27fc640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ec0b4120 0x7f59ec158fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f59d4037040 tx=0x7f59d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:07.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.044+0000 7f59e8ff9640 1 -- 172.21.15.81:0/3965373554 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4003810 con 0x7f59ec0b4120 2023-11-08T16:29:07.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.044+0000 7f59e8ff9640 1 -- 172.21.15.81:0/3965373554 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4003970 con 0x7f59ec0b4120 2023-11-08T16:29:07.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.044+0000 7f59e8ff9640 1 -- 172.21.15.81:0/3965373554 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4003ce0 con 0x7f59ec0b4120 2023-11-08T16:29:07.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.044+0000 7f59f18bb640 1 -- 172.21.15.81:0/3965373554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ec0b2e40 con 0x7f59ec0b4120 2023-11-08T16:29:07.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.044+0000 7f59f18bb640 1 -- 172.21.15.81:0/3965373554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ec0b3400 con 0x7f59ec0b4120 2023-11-08T16:29:07.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.045+0000 7f59f18bb640 1 -- 172.21.15.81:0/3965373554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ec072170 con 0x7f59ec0b4120 2023-11-08T16:29:07.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.046+0000 7f59e8ff9640 1 -- 172.21.15.81:0/3965373554 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f59d4034440 con 0x7f59ec0b4120 2023-11-08T16:29:07.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.047+0000 7f59e8ff9640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f59cc06ae70 0x7f59cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.047+0000 7f59e8ff9640 1 -- 172.21.15.81:0/3965373554 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59d40ab610 con 0x7f59ec0b4120 2023-11-08T16:29:07.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.047+0000 7f59eaffd640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f59cc06ae70 0x7f59cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.048+0000 7f59eaffd640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f59cc06ae70 0x7f59cc06d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f59dc007fd0 tx=0x7f59dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:07.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.052+0000 7f59e8ff9640 1 -- 172.21.15.81:0/3965373554 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f59d40766d0 con 0x7f59ec0b4120 2023-11-08T16:29:07.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.280+0000 7f59f18bb640 1 -- 172.21.15.81:0/3965373554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59ec04ece0 con 0x7f59ec0b4120 2023-11-08T16:29:07.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.282+0000 7f59e8ff9640 1 -- 172.21.15.81:0/3965373554 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f59d4080070 con 0x7f59ec0b4120 2023-11-08T16:29:07.285 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:07.285 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4190352668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4190352668},{"type":"v1","addr":"172.21.15.81:6835","nonce":4190352668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2636431103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2636431103},{"type":"v1","addr":"172.21.15.93:6835","nonce":2636431103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3681499690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3681499690},{"type":"v1","addr":"172.21.15.81:6837","nonce":3681499690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:29:03.181472+0000","modified":"2023-11-08T16:29:06.272041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6834/2087517381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2087517381},{"type":"v1","addr":"172.21.15.93:6834","nonce":2087517381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T16:29:07.285 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 8 2023-11-08T16:29:07.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.284+0000 7f59f18bb640 1 -- 172.21.15.81:0/3965373554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f59cc06ae70 msgr2=0x7f59cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.284+0000 7f59f18bb640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f59cc06ae70 0x7f59cc06d330 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f59dc007fd0 tx=0x7f59dc009040 comp rx=0 tx=0).stop 2023-11-08T16:29:07.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.284+0000 7f59f18bb640 1 -- 172.21.15.81:0/3965373554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ec0b4120 msgr2=0x7f59ec158fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.284+0000 7f59f18bb640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ec0b4120 0x7f59ec158fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f59d4037040 tx=0x7f59d4004730 comp rx=0 tx=0).stop 2023-11-08T16:29:07.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.285+0000 7f59f18bb640 1 -- 172.21.15.81:0/3965373554 shutdown_connections 2023-11-08T16:29:07.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.285+0000 7f59f18bb640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f59cc06ae70 0x7f59cc06d330 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.285+0000 7f59f18bb640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59ec070cf0 0x7f59ec071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.285+0000 7f59f18bb640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59ec0b4120 0x7f59ec158fe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.285+0000 7f59f18bb640 1 --2- 172.21.15.81:0/3965373554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59d80069d0 0x7f59ec0b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.285+0000 7f59f18bb640 1 -- 172.21.15.81:0/3965373554 >> 172.21.15.81:0/3965373554 conn(0x7f59ec0cb490 msgr2=0x7f59ec159990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:07.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.285+0000 7f59f18bb640 1 -- 172.21.15.81:0/3965373554 shutdown_connections 2023-11-08T16:29:07.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.285+0000 7f59f18bb640 1 -- 172.21.15.81:0/3965373554 wait complete. 2023-11-08T16:29:07.307 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:07.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.517+0000 7f2e53dd7640 1 Processor -- start 2023-11-08T16:29:07.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.517+0000 7f2e53dd7640 1 -- start start 2023-11-08T16:29:07.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.519+0000 7f2e53dd7640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2e4c161710 0x7f2e4c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.519+0000 7f2e53dd7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c0ccd30 0x7f2e4c06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.519+0000 7f2e53dd7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c161bc0 con 0x7f2e4c15d2b0 2023-11-08T16:29:07.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.519+0000 7f2e53dd7640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c0cd1c0 con 0x7f2e4c161710 2023-11-08T16:29:07.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.519+0000 7f2e53dd7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c15d690 con 0x7f2e4c0ccd30 2023-11-08T16:29:07.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.519+0000 7f2e51b73640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2e4c15d2b0 0x7f2e4c0c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58788/0 (socket says 172.21.15.81:58788) 2023-11-08T16:29:07.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.519+0000 7f2e51b73640 1 -- 172.21.15.81:0/2852999160 learned_addr learned my addr 172.21.15.81:0/2852999160 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:07.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e52374640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c0ccd30 0x7f2e4c06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e51372640 1 --2- 172.21.15.81:0/2852999160 >> v2:172.21.15.93:3300/0 conn(0x7f2e4c161710 0x7f2e4c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1172574507 0 0) 0x7f2e4c161bc0 con 0x7f2e4c15d2b0 2023-11-08T16:29:07.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e34003520 con 0x7f2e4c15d2b0 2023-11-08T16:29:07.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 634671492 0 0) 0x7f2e34003520 con 0x7f2e4c15d2b0 2023-11-08T16:29:07.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c0ccd30 msgr2=0x7f2e4c06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:07.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c0ccd30 0x7f2e4c06ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 >> v2:172.21.15.93:3300/0 conn(0x7f2e4c161710 msgr2=0x7f2e4c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:07.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 --2- 172.21.15.81:0/2852999160 >> v2:172.21.15.93:3300/0 conn(0x7f2e4c161710 0x7f2e4c0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c06b980 con 0x7f2e4c15d2b0 2023-11-08T16:29:07.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f2e3c001670 con 0x7f2e4c15d2b0 2023-11-08T16:29:07.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 >> v1:172.21.15.81:6789/0 conn(0x7f2e4c15d2b0 legacy=0x7f2e4c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e34003a50 0x7f2e34003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c0ccd30 0x7f2e340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e34004d00 0x7f2e34005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0cd1c0 con 0x7f2e34003a50 2023-11-08T16:29:07.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c15d690 con 0x7f2e34004d00 2023-11-08T16:29:07.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.520+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3400ba30 con 0x7f2e4c0ccd30 2023-11-08T16:29:07.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.521+0000 7f2e51b73640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e34003a50 0x7f2e34003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.521+0000 7f2e51372640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c0ccd30 0x7f2e340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.521+0000 7f2e52374640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e34004d00 0x7f2e34005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.521+0000 7f2e51b73640 1 -- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c0ccd30 msgr2=0x7f2e340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:07.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.521+0000 7f2e51b73640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c0ccd30 0x7f2e340047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.521+0000 7f2e51b73640 1 -- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e34004d00 msgr2=0x7f2e34005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:07.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.521+0000 7f2e51b73640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e34004d00 0x7f2e34005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.521+0000 7f2e51b73640 1 -- 172.21.15.81:0/2852999160 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e34003520 con 0x7f2e34003a50 2023-11-08T16:29:07.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.521+0000 7f2e52374640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e34004d00 0x7f2e34005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:07.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.522+0000 7f2e51b73640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e34003a50 0x7f2e34003e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00c110 tx=0x7f2e340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a842baf2c9417c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:07.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.522+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c005ad0 con 0x7f2e34003a50 2023-11-08T16:29:07.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.522+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c005c50 con 0x7f2e34003a50 2023-11-08T16:29:07.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.522+0000 7f2e50b71640 1 -- 172.21.15.81:0/2852999160 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c007070 con 0x7f2e34003a50 2023-11-08T16:29:07.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.523+0000 7f2e53dd7640 1 -- 172.21.15.81:0/2852999160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e34003a50 msgr2=0x7f2e34003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.523+0000 7f2e53dd7640 1 --2- 172.21.15.81:0/2852999160 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e34003a50 0x7f2e34003e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00c110 tx=0x7f2e340057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.523+0000 7f2e52374640 1 -- 172.21.15.81:0/2852999160 reap_dead start 2023-11-08T16:29:07.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.523+0000 7f2e53dd7640 1 -- 172.21.15.81:0/2852999160 shutdown_connections 2023-11-08T16:29:07.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.523+0000 7f2e53dd7640 1 -- 172.21.15.81:0/2852999160 >> 172.21.15.81:0/2852999160 conn(0x7f2e4c0cb490 msgr2=0x7f2e4c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:07.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.523+0000 7f2e53dd7640 1 -- 172.21.15.81:0/2852999160 shutdown_connections 2023-11-08T16:29:07.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.523+0000 7f2e53dd7640 1 -- 172.21.15.81:0/2852999160 wait complete. 2023-11-08T16:29:07.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.525+0000 7f2e53dd7640 1 Processor -- start 2023-11-08T16:29:07.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.525+0000 7f2e53dd7640 1 -- start start 2023-11-08T16:29:07.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.526+0000 7f2e53dd7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e34003a50 0x7f2e4c15eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.526+0000 7f2e53dd7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c156da0 0x7f2e4c146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.526+0000 7f2e53dd7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c1470f0 0x7f2e4c15f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.526+0000 7f2e53dd7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c146920 con 0x7f2e4c1470f0 2023-11-08T16:29:07.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.526+0000 7f2e53dd7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0b1c50 con 0x7f2e34003a50 2023-11-08T16:29:07.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.526+0000 7f2e53dd7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c15f3f0 con 0x7f2e4c156da0 2023-11-08T16:29:07.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.526+0000 7f2e52374640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c1470f0 0x7f2e4c15f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.526+0000 7f2e52374640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c1470f0 0x7f2e4c15f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41724/0 (socket says 172.21.15.81:41724) 2023-11-08T16:29:07.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.526+0000 7f2e52374640 1 -- 172.21.15.81:0/530735955 learned_addr learned my addr 172.21.15.81:0/530735955 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:07.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e51b73640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e34003a50 0x7f2e4c15eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e51372640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c156da0 0x7f2e4c146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e52374640 1 -- 172.21.15.81:0/530735955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c156da0 msgr2=0x7f2e4c146bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e52374640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c156da0 0x7f2e4c146bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e52374640 1 -- 172.21.15.81:0/530735955 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e34003a50 msgr2=0x7f2e4c15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e52374640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e34003a50 0x7f2e4c15eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e52374640 1 -- 172.21.15.81:0/530735955 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c15f0d0 con 0x7f2e4c1470f0 2023-11-08T16:29:07.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e52374640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c1470f0 0x7f2e4c15f880 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0ce0f0 tx=0x7f2e4400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:07.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e427fc640 1 -- 172.21.15.81:0/530735955 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4400fe30 con 0x7f2e4c1470f0 2023-11-08T16:29:07.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e427fc640 1 -- 172.21.15.81:0/530735955 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e44011040 con 0x7f2e4c1470f0 2023-11-08T16:29:07.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e427fc640 1 -- 172.21.15.81:0/530735955 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e44017640 con 0x7f2e4c1470f0 2023-11-08T16:29:07.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.527+0000 7f2e53dd7640 1 -- 172.21.15.81:0/530735955 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c149b50 con 0x7f2e4c1470f0 2023-11-08T16:29:07.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.528+0000 7f2e53dd7640 1 -- 172.21.15.81:0/530735955 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c0b1970 con 0x7f2e4c1470f0 2023-11-08T16:29:07.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.529+0000 7f2e53dd7640 1 -- 172.21.15.81:0/530735955 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c04ece0 con 0x7f2e4c1470f0 2023-11-08T16:29:07.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.529+0000 7f2e427fc640 1 -- 172.21.15.81:0/530735955 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e440203b0 con 0x7f2e4c1470f0 2023-11-08T16:29:07.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.530+0000 7f2e427fc640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e3806aea0 0x7f2e3806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.531+0000 7f2e427fc640 1 -- 172.21.15.81:0/530735955 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2e44089750 con 0x7f2e4c1470f0 2023-11-08T16:29:07.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.531+0000 7f2e51b73640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e3806aea0 0x7f2e3806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.531+0000 7f2e51b73640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e3806aea0 0x7f2e3806d360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c034000 tx=0x7f2e3c007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:07.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.535+0000 7f2e427fc640 1 -- 172.21.15.81:0/530735955 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e44054800 con 0x7f2e4c1470f0 2023-11-08T16:29:07.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.727+0000 7f2e53dd7640 1 -- 172.21.15.81:0/530735955 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e4c15f530 con 0x7f2e4c1470f0 2023-11-08T16:29:07.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.729+0000 7f2e427fc640 1 -- 172.21.15.81:0/530735955 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f2e44057e20 con 0x7f2e4c1470f0 2023-11-08T16:29:07.734 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:07.735 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:04.727022+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:07.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 -- 172.21.15.81:0/530735955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e3806aea0 msgr2=0x7f2e3806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e3806aea0 0x7f2e3806d360 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c034000 tx=0x7f2e3c007000 comp rx=0 tx=0).stop 2023-11-08T16:29:07.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 -- 172.21.15.81:0/530735955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c1470f0 msgr2=0x7f2e4c15f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c1470f0 0x7f2e4c15f880 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e4c0ce0f0 tx=0x7f2e4400f4a0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 -- 172.21.15.81:0/530735955 shutdown_connections 2023-11-08T16:29:07.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e3806aea0 0x7f2e3806d360 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e4c1470f0 0x7f2e4c15f880 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e4c156da0 0x7f2e4c146bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 --2- 172.21.15.81:0/530735955 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e34003a50 0x7f2e4c15eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 -- 172.21.15.81:0/530735955 >> 172.21.15.81:0/530735955 conn(0x7f2e4c0cb490 msgr2=0x7f2e4c06c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:07.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 -- 172.21.15.81:0/530735955 shutdown_connections 2023-11-08T16:29:07.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.732+0000 7f2e53dd7640 1 -- 172.21.15.81:0/530735955 wait complete. 2023-11-08T16:29:07.756 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T16:29:07.757 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:29:07.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.967+0000 7f9dfab90640 1 Processor -- start 2023-11-08T16:29:07.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.967+0000 7f9dfab90640 1 -- start start 2023-11-08T16:29:07.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.968+0000 7f9dfab90640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9df40ca480 0x7f9df415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.968+0000 7f9dfab90640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df406b630 con 0x7f9df415ea60 2023-11-08T16:29:07.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.968+0000 7f9dfab90640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df40ca910 con 0x7f9df40ade50 2023-11-08T16:29:07.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.968+0000 7f9dfab90640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df40b1860 con 0x7f9df40ca480 2023-11-08T16:29:07.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.968+0000 7f9df3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9df40ca480 0x7f9df415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.968+0000 7f9df3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9df40ca480 0x7f9df415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41738/0 (socket says 172.21.15.81:41738) 2023-11-08T16:29:07.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.968+0000 7f9df3fff640 1 -- 172.21.15.81:0/95369198 learned_addr learned my addr 172.21.15.81:0/95369198 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:07.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.969+0000 7f9df3fff640 1 -- 172.21.15.81:0/95369198 >> v1:172.21.15.93:6789/0 conn(0x7f9df40ade50 legacy=0x7f9df415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:07.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.969+0000 7f9df3fff640 1 -- 172.21.15.81:0/95369198 >> v1:172.21.15.81:6789/0 conn(0x7f9df415ea60 legacy=0x7f9df415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.969+0000 7f9df3fff640 1 -- 172.21.15.81:0/95369198 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df40cabb0 con 0x7f9df40ca480 2023-11-08T16:29:07.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.969+0000 7f9df37fe640 1 -- 172.21.15.81:0/95369198 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915336729 0 0) 0x7f9df406b630 con 0x7f9df415ea60 2023-11-08T16:29:07.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.970+0000 7f9df3fff640 1 --2- 172.21.15.81:0/95369198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9df40ca480 0x7f9df415ad90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009870 tx=0x7f9de002ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0d4e76eb00f86b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:07.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.970+0000 7f9df37fe640 1 -- 172.21.15.81:0/95369198 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de002f840 con 0x7f9df40ca480 2023-11-08T16:29:07.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.970+0000 7f9df37fe640 1 -- 172.21.15.81:0/95369198 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de002f9a0 con 0x7f9df40ca480 2023-11-08T16:29:07.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.970+0000 7f9df37fe640 1 -- 172.21.15.81:0/95369198 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de002fd30 con 0x7f9df40ca480 2023-11-08T16:29:07.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.971+0000 7f9dfab90640 1 -- 172.21.15.81:0/95369198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9df40ca480 msgr2=0x7f9df415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.971+0000 7f9dfab90640 1 --2- 172.21.15.81:0/95369198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9df40ca480 0x7f9df415ad90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9de0009870 tx=0x7f9de002ba20 comp rx=0 tx=0).stop 2023-11-08T16:29:07.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.971+0000 7f9dfab90640 1 -- 172.21.15.81:0/95369198 shutdown_connections 2023-11-08T16:29:07.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.971+0000 7f9dfab90640 1 --2- 172.21.15.81:0/95369198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9df40ca480 0x7f9df415ad90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.971+0000 7f9dfab90640 1 -- 172.21.15.81:0/95369198 >> 172.21.15.81:0/95369198 conn(0x7f9df40cb490 msgr2=0x7f9df40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:07.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.971+0000 7f9dfab90640 1 -- 172.21.15.81:0/95369198 shutdown_connections 2023-11-08T16:29:07.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.971+0000 7f9dfab90640 1 -- 172.21.15.81:0/95369198 wait complete. 2023-11-08T16:29:07.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.973+0000 7f9dfab90640 1 Processor -- start 2023-11-08T16:29:07.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.973+0000 7f9dfab90640 1 -- start start 2023-11-08T16:29:07.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9dfab90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9df40ade50 0x7f9df41f2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9dfab90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9df40ca480 0x7f9df40b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9dfab90640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9df415ea60 0x7f9df40b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9dfab90640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df414fb70 con 0x7f9df40ade50 2023-11-08T16:29:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9dfab90640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df414f890 con 0x7f9df415ea60 2023-11-08T16:29:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9dfab90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df41f6640 con 0x7f9df40ca480 2023-11-08T16:29:07.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9df892c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9df40ade50 0x7f9df41f2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9df892c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9df40ade50 0x7f9df41f2900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41740/0 (socket says 172.21.15.81:41740) 2023-11-08T16:29:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9df892c640 1 -- 172.21.15.81:0/8154884 learned_addr learned my addr 172.21.15.81:0/8154884 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9df892c640 1 -- 172.21.15.81:0/8154884 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9df40ca480 msgr2=0x7f9df40b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9df892c640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9df40ca480 0x7f9df40b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9df892c640 1 -- 172.21.15.81:0/8154884 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9df415ea60 msgr2=0x7f9df40b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9df892c640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9df415ea60 0x7f9df40b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.974+0000 7f9df892c640 1 -- 172.21.15.81:0/8154884 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de00094d0 con 0x7f9df40ade50 2023-11-08T16:29:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.975+0000 7f9df892c640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9df40ade50 0x7f9df41f2900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c450 tx=0x7f9de800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.975+0000 7f9df17fa640 1 -- 172.21.15.81:0/8154884 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de80052d0 con 0x7f9df40ade50 2023-11-08T16:29:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.975+0000 7f9df17fa640 1 -- 172.21.15.81:0/8154884 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de8005430 con 0x7f9df40ade50 2023-11-08T16:29:07.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.975+0000 7f9df17fa640 1 -- 172.21.15.81:0/8154884 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8012870 con 0x7f9df40ade50 2023-11-08T16:29:07.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.975+0000 7f9dfab90640 1 -- 172.21.15.81:0/8154884 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df41f6920 con 0x7f9df40ade50 2023-11-08T16:29:07.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.975+0000 7f9dfab90640 1 -- 172.21.15.81:0/8154884 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df41f7120 con 0x7f9df40ade50 2023-11-08T16:29:07.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.976+0000 7f9dfab90640 1 -- 172.21.15.81:0/8154884 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df41f64a0 con 0x7f9df40ade50 2023-11-08T16:29:07.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.977+0000 7f9df17fa640 1 -- 172.21.15.81:0/8154884 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9de80129d0 con 0x7f9df40ade50 2023-11-08T16:29:07.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.978+0000 7f9df17fa640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9dd806ae70 0x7f9dd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:07.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.978+0000 7f9df17fa640 1 -- 172.21.15.81:0/8154884 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9de8089120 con 0x7f9df40ade50 2023-11-08T16:29:07.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.978+0000 7f9df3fff640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9dd806ae70 0x7f9dd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:07.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.979+0000 7f9df3fff640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9dd806ae70 0x7f9dd806d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9de0036000 tx=0x7f9de0039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:07.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:07.982+0000 7f9df17fa640 1 -- 172.21.15.81:0/8154884 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9df41f64a0 con 0x7f9df40ade50 2023-11-08T16:29:08.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.169+0000 7f9dfab90640 1 -- 172.21.15.81:0/8154884 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9df40cb170 con 0x7f9dd806ae70 2023-11-08T16:29:08.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.178+0000 7f9df17fa640 1 -- 172.21.15.81:0/8154884 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+249756 (secure 0 0 0) 0x7f9de002fb10 con 0x7f9dd806ae70 2023-11-08T16:29:08.185 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:08.186 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:29:08.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 -- 172.21.15.81:0/8154884 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9dd806ae70 msgr2=0x7f9dd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9dd806ae70 0x7f9dd806d330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9de0036000 tx=0x7f9de0039040 comp rx=0 tx=0).stop 2023-11-08T16:29:08.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 -- 172.21.15.81:0/8154884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9df40ade50 msgr2=0x7f9df41f2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9df40ade50 0x7f9df41f2900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9de800c450 tx=0x7f9de800c910 comp rx=0 tx=0).stop 2023-11-08T16:29:08.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 -- 172.21.15.81:0/8154884 shutdown_connections 2023-11-08T16:29:08.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9dd806ae70 0x7f9dd806d330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9df415ea60 0x7f9df40b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9df40ca480 0x7f9df40b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 --2- 172.21.15.81:0/8154884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9df40ade50 0x7f9df41f2900 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 -- 172.21.15.81:0/8154884 >> 172.21.15.81:0/8154884 conn(0x7f9df40cb490 msgr2=0x7f9df40b1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:08.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 -- 172.21.15.81:0/8154884 shutdown_connections 2023-11-08T16:29:08.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.183+0000 7f9dfab90640 1 -- 172.21.15.81:0/8154884 wait complete. 2023-11-08T16:29:08.205 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-08T16:29:06.548517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":219812,"kb_used_data":2472,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754908,"statfs":{"total":773094113280,"available":772869025792,"internally_reserved":0,"allocated":2531328,"data_stored":1584048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.846723"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":161,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-11-08T16:28:58.202584+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:28:58.202584+0000","last_peered":"2023-11-08T16:28:58.202584+0000","last_clean":"2023-11-08T16:28:58.202584+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:28:58.202584+0000","last_undegraded":"2023-11-08T16:28:58.202584+0000","last_fullsized":"2023-11-08T16:28:58.202584+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:02.700092+0000","last_change":"2023-11-08T16:29:02.700092+0000","last_active":"2023-11-08T16:29:02.700092+0000","last_peered":"2023-11-08T16:29:02.700092+0000","last_clean":"2023-11-08T16:29:02.700092+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:02.700092+0000","last_undegraded":"2023-11-08T16:29:02.700092+0000","last_fullsized":"2023-11-08T16:29:02.700092+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:29:01.686169+0000","last_change":"2023-11-08T16:29:01.686169+0000","last_active":"2023-11-08T16:29:01.686169+0000","last_peered":"2023-11-08T16:29:01.686169+0000","last_clean":"2023-11-08T16:29:01.686169+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:29:01.686169+0000","last_undegraded":"2023-11-08T16:29:01.686169+0000","last_fullsized":"2023-11-08T16:29:01.686169+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28324,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343516,"statfs":{"total":96636764160,"available":96607760384,"internally_reserved":0,"allocated":761856,"data_stored":640782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0629999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4730000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6359999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.728}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.746}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8220000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.875}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2949999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4609999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8680000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6339999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8330000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7789999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1160000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1930000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0880000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.042}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1679999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3759999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4550000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8839999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.25}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0950000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1469999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3380000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0169999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2969999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2190000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2749999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27184,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344656,"statfs":{"total":96636764160,"available":96608927744,"internally_reserved":0,"allocated":167936,"data_stored":50414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1040000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7250000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9159999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.4730000000000008}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.875}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28384,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343456,"statfs":{"total":96636764160,"available":96607698944,"internally_reserved":0,"allocated":761856,"data_stored":640782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0489999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:29:08.211 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:29:08.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.466+0000 7f87cbd78640 1 Processor -- start 2023-11-08T16:29:08.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.467+0000 7f87cbd78640 1 -- start start 2023-11-08T16:29:08.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.468+0000 7f87cbd78640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f87c40d3920 0x7f87c40b3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.468+0000 7f87cbd78640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c40b2450 con 0x7f87c415d1a0 2023-11-08T16:29:08.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.468+0000 7f87cbd78640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c40ce800 con 0x7f87c40d1af0 2023-11-08T16:29:08.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.468+0000 7f87cbd78640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c40d3db0 con 0x7f87c40d3920 2023-11-08T16:29:08.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.468+0000 7f87c9b14640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f87c415d1a0 0x7f87c40d13e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58808/0 (socket says 172.21.15.81:58808) 2023-11-08T16:29:08.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.468+0000 7f87c9b14640 1 -- 172.21.15.81:0/579179856 learned_addr learned my addr 172.21.15.81:0/579179856 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:08.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902530721 0 0) 0x7f87c40b2450 con 0x7f87c415d1a0 2023-11-08T16:29:08.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87b0003520 con 0x7f87c415d1a0 2023-11-08T16:29:08.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87ca315640 1 --2- 172.21.15.81:0/579179856 >> v2:172.21.15.93:3300/0 conn(0x7f87c40d3920 0x7f87c40b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:08.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2325771763 0 0) 0x7f87b0003520 con 0x7f87c415d1a0 2023-11-08T16:29:08.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 >> v2:172.21.15.93:3300/0 conn(0x7f87c40d3920 msgr2=0x7f87c40b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:08.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 --2- 172.21.15.81:0/579179856 >> v2:172.21.15.93:3300/0 conn(0x7f87c40d3920 0x7f87c40b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 >> v1:172.21.15.93:6789/0 conn(0x7f87c40d1af0 legacy=0x7f87c40b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c40ceae0 con 0x7f87c415d1a0 2023-11-08T16:29:08.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f87c00017f0 con 0x7f87c415d1a0 2023-11-08T16:29:08.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 >> v1:172.21.15.81:6789/0 conn(0x7f87c415d1a0 legacy=0x7f87c40d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87b0003a50 0x7f87b0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87b00047c0 0x7f87b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87b00052a0 0x7f87b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40d3db0 con 0x7f87b00052a0 2023-11-08T16:29:08.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b000c1a0 con 0x7f87b0003a50 2023-11-08T16:29:08.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.469+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b000c360 con 0x7f87b00047c0 2023-11-08T16:29:08.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.470+0000 7f87ca315640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87b00052a0 0x7f87b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:08.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.470+0000 7f87c9313640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87b00047c0 0x7f87b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:08.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.470+0000 7f87c9b14640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87b0003a50 0x7f87b0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:08.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.470+0000 7f87ca315640 1 -- 172.21.15.81:0/579179856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87b00047c0 msgr2=0x7f87b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:08.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.470+0000 7f87ca315640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87b00047c0 0x7f87b0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.470+0000 7f87ca315640 1 -- 172.21.15.81:0/579179856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87b0003a50 msgr2=0x7f87b0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:08.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.470+0000 7f87ca315640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87b0003a50 0x7f87b0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.470+0000 7f87ca315640 1 -- 172.21.15.81:0/579179856 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0003520 con 0x7f87b00052a0 2023-11-08T16:29:08.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.470+0000 7f87c9b14640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87b0003a50 0x7f87b0003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:08.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.471+0000 7f87ca315640 1 -- 172.21.15.81:0/579179856 reap_dead start 2023-11-08T16:29:08.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.471+0000 7f87ca315640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87b00052a0 0x7f87b000b980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f87bc007360 tx=0x7f87b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2ee93b32e94a9f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:08.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.471+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc02ccf0 con 0x7f87b00052a0 2023-11-08T16:29:08.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.471+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc02ce90 con 0x7f87b00052a0 2023-11-08T16:29:08.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.471+0000 7f87c8b12640 1 -- 172.21.15.81:0/579179856 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc003bd0 con 0x7f87b00052a0 2023-11-08T16:29:08.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.472+0000 7f87cbd78640 1 -- 172.21.15.81:0/579179856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87b00052a0 msgr2=0x7f87b000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.472+0000 7f87cbd78640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87b00052a0 0x7f87b000b980 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f87bc007360 tx=0x7f87b00057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.472+0000 7f87cbd78640 1 -- 172.21.15.81:0/579179856 shutdown_connections 2023-11-08T16:29:08.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.472+0000 7f87cbd78640 1 --2- 172.21.15.81:0/579179856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87b00052a0 0x7f87b000b980 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.472+0000 7f87cbd78640 1 -- 172.21.15.81:0/579179856 >> 172.21.15.81:0/579179856 conn(0x7f87c40d2e00 msgr2=0x7f87c40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:08.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.472+0000 7f87cbd78640 1 -- 172.21.15.81:0/579179856 shutdown_connections 2023-11-08T16:29:08.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.472+0000 7f87cbd78640 1 -- 172.21.15.81:0/579179856 wait complete. 2023-11-08T16:29:08.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.474+0000 7f87cbd78640 1 Processor -- start 2023-11-08T16:29:08.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.474+0000 7f87cbd78640 1 -- start start 2023-11-08T16:29:08.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.475+0000 7f87cbd78640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87b00052a0 0x7f87c415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.475+0000 7f87cbd78640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c40cc7b0 0x7f87c40ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.475+0000 7f87cbd78640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c40ca4c0 0x7f87c40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.475+0000 7f87cbd78640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40c9a40 con 0x7f87c40ca4c0 2023-11-08T16:29:08.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.475+0000 7f87cbd78640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c415e880 con 0x7f87b00052a0 2023-11-08T16:29:08.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.475+0000 7f87cbd78640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c415eb60 con 0x7f87c40cc7b0 2023-11-08T16:29:08.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.475+0000 7f87ca315640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c40ca4c0 0x7f87c40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:08.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.475+0000 7f87ca315640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c40ca4c0 0x7f87c40ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41756/0 (socket says 172.21.15.81:41756) 2023-11-08T16:29:08.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.475+0000 7f87ca315640 1 -- 172.21.15.81:0/2116255750 learned_addr learned my addr 172.21.15.81:0/2116255750 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:08.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.476+0000 7f87ca315640 1 -- 172.21.15.81:0/2116255750 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c40cc7b0 msgr2=0x7f87c40ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.476+0000 7f87ca315640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c40cc7b0 0x7f87c40ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.476+0000 7f87ca315640 1 -- 172.21.15.81:0/2116255750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87b00052a0 msgr2=0x7f87c415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.476+0000 7f87ca315640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87b00052a0 0x7f87c415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.476+0000 7f87ca315640 1 -- 172.21.15.81:0/2116255750 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c415ee40 con 0x7f87c40ca4c0 2023-11-08T16:29:08.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.476+0000 7f87ca315640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c40ca4c0 0x7f87c40ca970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f87bc02c760 tx=0x7f87bc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:08.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.476+0000 7f87ba7fc640 1 -- 172.21.15.81:0/2116255750 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc004670 con 0x7f87c40ca4c0 2023-11-08T16:29:08.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.476+0000 7f87ba7fc640 1 -- 172.21.15.81:0/2116255750 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87bc0047d0 con 0x7f87c40ca4c0 2023-11-08T16:29:08.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.476+0000 7f87ba7fc640 1 -- 172.21.15.81:0/2116255750 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87bc040680 con 0x7f87c40ca4c0 2023-11-08T16:29:08.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.476+0000 7f87cbd78640 1 -- 172.21.15.81:0/2116255750 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c415f120 con 0x7f87c40ca4c0 2023-11-08T16:29:08.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.476+0000 7f87cbd78640 1 -- 172.21.15.81:0/2116255750 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c40cd170 con 0x7f87c40ca4c0 2023-11-08T16:29:08.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.478+0000 7f87cbd78640 1 -- 172.21.15.81:0/2116255750 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c415f360 con 0x7f87c40ca4c0 2023-11-08T16:29:08.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.478+0000 7f87ba7fc640 1 -- 172.21.15.81:0/2116255750 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f87bc034440 con 0x7f87c40ca4c0 2023-11-08T16:29:08.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.479+0000 7f87ba7fc640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87a806aea0 0x7f87a806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.479+0000 7f87ba7fc640 1 -- 172.21.15.81:0/2116255750 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87bc03b070 con 0x7f87c40ca4c0 2023-11-08T16:29:08.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.480+0000 7f87c9b14640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87a806aea0 0x7f87a806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:08.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.480+0000 7f87c9b14640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87a806aea0 0x7f87a806d360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87c0001eb0 tx=0x7f87c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:08.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.484+0000 7f87ba7fc640 1 -- 172.21.15.81:0/2116255750 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87c415f360 con 0x7f87c40ca4c0 2023-11-08T16:29:08.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.674+0000 7f87cbd78640 1 -- 172.21.15.81:0/2116255750 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87c40d31e0 con 0x7f87a806aea0 2023-11-08T16:29:08.692 INFO:tasks.ceph.mds.d.smithi093.stderr:2023-11-08T16:29:08.691+0000 7fc8b50ea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:29:08.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.700+0000 7f87ba7fc640 1 -- 172.21.15.81:0/2116255750 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264529 (secure 0 0 0) 0x7f87c40d31e0 con 0x7f87a806aea0 2023-11-08T16:29:08.705 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:08.706 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:29:08.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.705+0000 7f87cbd78640 1 -- 172.21.15.81:0/2116255750 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87a806aea0 msgr2=0x7f87a806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.705+0000 7f87cbd78640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87a806aea0 0x7f87a806d360 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f87c0001eb0 tx=0x7f87c0009040 comp rx=0 tx=0).stop 2023-11-08T16:29:08.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.705+0000 7f87cbd78640 1 -- 172.21.15.81:0/2116255750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c40ca4c0 msgr2=0x7f87c40ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.705+0000 7f87cbd78640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c40ca4c0 0x7f87c40ca970 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f87bc02c760 tx=0x7f87bc004ab0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.705+0000 7f87cbd78640 1 -- 172.21.15.81:0/2116255750 shutdown_connections 2023-11-08T16:29:08.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.705+0000 7f87cbd78640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87a806aea0 0x7f87a806d360 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.705+0000 7f87cbd78640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c40ca4c0 0x7f87c40ca970 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.705+0000 7f87cbd78640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c40cc7b0 0x7f87c40ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.705+0000 7f87cbd78640 1 --2- 172.21.15.81:0/2116255750 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87b00052a0 0x7f87c415ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.705+0000 7f87cbd78640 1 -- 172.21.15.81:0/2116255750 >> 172.21.15.81:0/2116255750 conn(0x7f87c40d2e00 msgr2=0x7f87c415bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:08.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.706+0000 7f87cbd78640 1 -- 172.21.15.81:0/2116255750 shutdown_connections 2023-11-08T16:29:08.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.706+0000 7f87cbd78640 1 -- 172.21.15.81:0/2116255750 wait complete. 2023-11-08T16:29:08.725 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-08T16:29:08.549102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219048,"kb_used_data":2872,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755672,"statfs":{"total":773094113280,"available":772869808128,"internally_reserved":0,"allocated":2940928,"data_stored":1765004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001797"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.741028+0000","last_change":"2023-11-08T16:29:03.766177+0000","last_active":"2023-11-08T16:29:04.741028+0000","last_peered":"2023-11-08T16:29:04.741028+0000","last_clean":"2023-11-08T16:29:04.741028+0000","last_became_active":"2023-11-08T16:29:03.765758+0000","last_became_peered":"2023-11-08T16:29:03.765758+0000","last_unstale":"2023-11-08T16:29:04.741028+0000","last_undegraded":"2023-11-08T16:29:04.741028+0000","last_fullsized":"2023-11-08T16:29:04.741028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:20:02.456685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.778191+0000","last_change":"2023-11-08T16:29:03.775462+0000","last_active":"2023-11-08T16:29:03.778191+0000","last_peered":"2023-11-08T16:29:03.778191+0000","last_clean":"2023-11-08T16:29:03.778191+0000","last_became_active":"2023-11-08T16:29:03.775182+0000","last_became_peered":"2023-11-08T16:29:03.775182+0000","last_unstale":"2023-11-08T16:29:03.778191+0000","last_undegraded":"2023-11-08T16:29:03.778191+0000","last_fullsized":"2023-11-08T16:29:03.778191+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:22.242002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.775495+0000","last_change":"2023-11-08T16:29:03.771342+0000","last_active":"2023-11-08T16:29:03.775495+0000","last_peered":"2023-11-08T16:29:03.775495+0000","last_clean":"2023-11-08T16:29:03.775495+0000","last_became_active":"2023-11-08T16:29:03.770972+0000","last_became_peered":"2023-11-08T16:29:03.770972+0000","last_unstale":"2023-11-08T16:29:03.775495+0000","last_undegraded":"2023-11-08T16:29:03.775495+0000","last_fullsized":"2023-11-08T16:29:03.775495+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:11.949345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738552+0000","last_change":"2023-11-08T16:29:03.762848+0000","last_active":"2023-11-08T16:29:04.738552+0000","last_peered":"2023-11-08T16:29:04.738552+0000","last_clean":"2023-11-08T16:29:04.738552+0000","last_became_active":"2023-11-08T16:29:03.762029+0000","last_became_peered":"2023-11-08T16:29:03.762029+0000","last_unstale":"2023-11-08T16:29:04.738552+0000","last_undegraded":"2023-11-08T16:29:04.738552+0000","last_fullsized":"2023-11-08T16:29:04.738552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:27.108912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739867+0000","last_change":"2023-11-08T16:29:03.758683+0000","last_active":"2023-11-08T16:29:04.739867+0000","last_peered":"2023-11-08T16:29:04.739867+0000","last_clean":"2023-11-08T16:29:04.739867+0000","last_became_active":"2023-11-08T16:29:03.758313+0000","last_became_peered":"2023-11-08T16:29:03.758313+0000","last_unstale":"2023-11-08T16:29:04.739867+0000","last_undegraded":"2023-11-08T16:29:04.739867+0000","last_fullsized":"2023-11-08T16:29:04.739867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:42.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738354+0000","last_change":"2023-11-08T16:29:03.774526+0000","last_active":"2023-11-08T16:29:04.738354+0000","last_peered":"2023-11-08T16:29:04.738354+0000","last_clean":"2023-11-08T16:29:04.738354+0000","last_became_active":"2023-11-08T16:29:03.774248+0000","last_became_peered":"2023-11-08T16:29:03.774248+0000","last_unstale":"2023-11-08T16:29:04.738354+0000","last_undegraded":"2023-11-08T16:29:04.738354+0000","last_fullsized":"2023-11-08T16:29:04.738354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:06.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738024+0000","last_change":"2023-11-08T16:29:03.775653+0000","last_active":"2023-11-08T16:29:04.738024+0000","last_peered":"2023-11-08T16:29:04.738024+0000","last_clean":"2023-11-08T16:29:04.738024+0000","last_became_active":"2023-11-08T16:29:03.774965+0000","last_became_peered":"2023-11-08T16:29:03.774965+0000","last_unstale":"2023-11-08T16:29:04.738024+0000","last_undegraded":"2023-11-08T16:29:04.738024+0000","last_fullsized":"2023-11-08T16:29:04.738024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:27.108912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735466+0000","last_change":"2023-11-08T16:29:03.767731+0000","last_active":"2023-11-08T16:29:04.735466+0000","last_peered":"2023-11-08T16:29:04.735466+0000","last_clean":"2023-11-08T16:29:04.735466+0000","last_became_active":"2023-11-08T16:29:03.767409+0000","last_became_peered":"2023-11-08T16:29:03.767409+0000","last_unstale":"2023-11-08T16:29:04.735466+0000","last_undegraded":"2023-11-08T16:29:04.735466+0000","last_fullsized":"2023-11-08T16:29:04.735466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:44.430116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.741175+0000","last_change":"2023-11-08T16:29:03.761330+0000","last_active":"2023-11-08T16:29:04.741175+0000","last_peered":"2023-11-08T16:29:04.741175+0000","last_clean":"2023-11-08T16:29:04.741175+0000","last_became_active":"2023-11-08T16:29:03.760953+0000","last_became_peered":"2023-11-08T16:29:03.760953+0000","last_unstale":"2023-11-08T16:29:04.741175+0000","last_undegraded":"2023-11-08T16:29:04.741175+0000","last_fullsized":"2023-11-08T16:29:04.741175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:56.358445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.769270+0000","last_change":"2023-11-08T16:29:03.761817+0000","last_active":"2023-11-08T16:29:03.769270+0000","last_peered":"2023-11-08T16:29:03.769270+0000","last_clean":"2023-11-08T16:29:03.769270+0000","last_became_active":"2023-11-08T16:29:03.760974+0000","last_became_peered":"2023-11-08T16:29:03.760974+0000","last_unstale":"2023-11-08T16:29:03.769270+0000","last_undegraded":"2023-11-08T16:29:03.769270+0000","last_fullsized":"2023-11-08T16:29:03.769270+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:46:56.190555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.770410+0000","last_change":"2023-11-08T16:29:03.765935+0000","last_active":"2023-11-08T16:29:03.770410+0000","last_peered":"2023-11-08T16:29:03.770410+0000","last_clean":"2023-11-08T16:29:03.770410+0000","last_became_active":"2023-11-08T16:29:03.765176+0000","last_became_peered":"2023-11-08T16:29:03.765176+0000","last_unstale":"2023-11-08T16:29:03.770410+0000","last_undegraded":"2023-11-08T16:29:03.770410+0000","last_fullsized":"2023-11-08T16:29:03.770410+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:14.067261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.774594+0000","last_change":"2023-11-08T16:29:03.771863+0000","last_active":"2023-11-08T16:29:03.774594+0000","last_peered":"2023-11-08T16:29:03.774594+0000","last_clean":"2023-11-08T16:29:03.774594+0000","last_became_active":"2023-11-08T16:29:03.770022+0000","last_became_peered":"2023-11-08T16:29:03.770022+0000","last_unstale":"2023-11-08T16:29:03.774594+0000","last_undegraded":"2023-11-08T16:29:03.774594+0000","last_fullsized":"2023-11-08T16:29:03.774594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:56.745110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735985+0000","last_change":"2023-11-08T16:29:03.765640+0000","last_active":"2023-11-08T16:29:04.735985+0000","last_peered":"2023-11-08T16:29:04.735985+0000","last_clean":"2023-11-08T16:29:04.735985+0000","last_became_active":"2023-11-08T16:29:03.765361+0000","last_became_peered":"2023-11-08T16:29:03.765361+0000","last_unstale":"2023-11-08T16:29:04.735985+0000","last_undegraded":"2023-11-08T16:29:04.735985+0000","last_fullsized":"2023-11-08T16:29:04.735985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:04.870398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.778100+0000","last_change":"2023-11-08T16:29:03.773181+0000","last_active":"2023-11-08T16:29:03.778100+0000","last_peered":"2023-11-08T16:29:03.778100+0000","last_clean":"2023-11-08T16:29:03.778100+0000","last_became_active":"2023-11-08T16:29:03.772830+0000","last_became_peered":"2023-11-08T16:29:03.772830+0000","last_unstale":"2023-11-08T16:29:03.778100+0000","last_undegraded":"2023-11-08T16:29:03.778100+0000","last_fullsized":"2023-11-08T16:29:03.778100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:46.473609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.774631+0000","last_change":"2023-11-08T16:29:03.770534+0000","last_active":"2023-11-08T16:29:03.774631+0000","last_peered":"2023-11-08T16:29:03.774631+0000","last_clean":"2023-11-08T16:29:03.774631+0000","last_became_active":"2023-11-08T16:29:03.770195+0000","last_became_peered":"2023-11-08T16:29:03.770195+0000","last_unstale":"2023-11-08T16:29:03.774631+0000","last_undegraded":"2023-11-08T16:29:03.774631+0000","last_fullsized":"2023-11-08T16:29:03.774631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:11.949345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.737592+0000","last_change":"2023-11-08T16:29:03.758938+0000","last_active":"2023-11-08T16:29:04.737592+0000","last_peered":"2023-11-08T16:29:04.737592+0000","last_clean":"2023-11-08T16:29:04.737592+0000","last_became_active":"2023-11-08T16:29:03.758630+0000","last_became_peered":"2023-11-08T16:29:03.758630+0000","last_unstale":"2023-11-08T16:29:04.737592+0000","last_undegraded":"2023-11-08T16:29:04.737592+0000","last_fullsized":"2023-11-08T16:29:04.737592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740206+0000","last_change":"2023-11-08T16:29:03.763570+0000","last_active":"2023-11-08T16:29:04.740206+0000","last_peered":"2023-11-08T16:29:04.740206+0000","last_clean":"2023-11-08T16:29:04.740206+0000","last_became_active":"2023-11-08T16:29:03.763210+0000","last_became_peered":"2023-11-08T16:29:03.763210+0000","last_unstale":"2023-11-08T16:29:04.740206+0000","last_undegraded":"2023-11-08T16:29:04.740206+0000","last_fullsized":"2023-11-08T16:29:04.740206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:40:59.683967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401379+0000","last_change":"2023-11-08T16:29:03.765682+0000","last_active":"2023-11-08T16:29:05.401379+0000","last_peered":"2023-11-08T16:29:05.401379+0000","last_clean":"2023-11-08T16:29:05.401379+0000","last_became_active":"2023-11-08T16:29:03.765405+0000","last_became_peered":"2023-11-08T16:29:03.765405+0000","last_unstale":"2023-11-08T16:29:05.401379+0000","last_undegraded":"2023-11-08T16:29:05.401379+0000","last_fullsized":"2023-11-08T16:29:05.401379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:09.861000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740558+0000","last_change":"2023-11-08T16:29:03.759779+0000","last_active":"2023-11-08T16:29:04.740558+0000","last_peered":"2023-11-08T16:29:04.740558+0000","last_clean":"2023-11-08T16:29:04.740558+0000","last_became_active":"2023-11-08T16:29:03.759472+0000","last_became_peered":"2023-11-08T16:29:03.759472+0000","last_unstale":"2023-11-08T16:29:04.740558+0000","last_undegraded":"2023-11-08T16:29:04.740558+0000","last_fullsized":"2023-11-08T16:29:04.740558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:40:59.683967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.778624+0000","last_change":"2023-11-08T16:29:03.776902+0000","last_active":"2023-11-08T16:29:03.778624+0000","last_peered":"2023-11-08T16:29:03.778624+0000","last_clean":"2023-11-08T16:29:03.778624+0000","last_became_active":"2023-11-08T16:29:03.776528+0000","last_became_peered":"2023-11-08T16:29:03.776528+0000","last_unstale":"2023-11-08T16:29:03.778624+0000","last_undegraded":"2023-11-08T16:29:03.778624+0000","last_fullsized":"2023-11-08T16:29:03.778624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:45:23.080055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.773818+0000","last_change":"2023-11-08T16:29:03.769496+0000","last_active":"2023-11-08T16:29:03.773818+0000","last_peered":"2023-11-08T16:29:03.773818+0000","last_clean":"2023-11-08T16:29:03.773818+0000","last_became_active":"2023-11-08T16:29:03.768491+0000","last_became_peered":"2023-11-08T16:29:03.768491+0000","last_unstale":"2023-11-08T16:29:03.773818+0000","last_undegraded":"2023-11-08T16:29:03.773818+0000","last_fullsized":"2023-11-08T16:29:03.773818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:58.627262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738562+0000","last_change":"2023-11-08T16:29:03.758807+0000","last_active":"2023-11-08T16:29:04.738562+0000","last_peered":"2023-11-08T16:29:04.738562+0000","last_clean":"2023-11-08T16:29:04.738562+0000","last_became_active":"2023-11-08T16:29:03.758466+0000","last_became_peered":"2023-11-08T16:29:03.758466+0000","last_unstale":"2023-11-08T16:29:04.738562+0000","last_undegraded":"2023-11-08T16:29:04.738562+0000","last_fullsized":"2023-11-08T16:29:04.738562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:10.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735439+0000","last_change":"2023-11-08T16:29:03.774206+0000","last_active":"2023-11-08T16:29:04.735439+0000","last_peered":"2023-11-08T16:29:04.735439+0000","last_clean":"2023-11-08T16:29:04.735439+0000","last_became_active":"2023-11-08T16:29:03.771583+0000","last_became_peered":"2023-11-08T16:29:03.771583+0000","last_unstale":"2023-11-08T16:29:04.735439+0000","last_undegraded":"2023-11-08T16:29:04.735439+0000","last_fullsized":"2023-11-08T16:29:04.735439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:51.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.737820+0000","last_change":"2023-11-08T16:29:03.760386+0000","last_active":"2023-11-08T16:29:04.737820+0000","last_peered":"2023-11-08T16:29:04.737820+0000","last_clean":"2023-11-08T16:29:04.737820+0000","last_became_active":"2023-11-08T16:29:03.760050+0000","last_became_peered":"2023-11-08T16:29:03.760050+0000","last_unstale":"2023-11-08T16:29:04.737820+0000","last_undegraded":"2023-11-08T16:29:04.737820+0000","last_fullsized":"2023-11-08T16:29:04.737820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735988+0000","last_change":"2023-11-08T16:29:03.768029+0000","last_active":"2023-11-08T16:29:04.735988+0000","last_peered":"2023-11-08T16:29:04.735988+0000","last_clean":"2023-11-08T16:29:04.735988+0000","last_became_active":"2023-11-08T16:29:03.767621+0000","last_became_peered":"2023-11-08T16:29:03.767621+0000","last_unstale":"2023-11-08T16:29:04.735988+0000","last_undegraded":"2023-11-08T16:29:04.735988+0000","last_fullsized":"2023-11-08T16:29:04.735988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.776117+0000","last_change":"2023-11-08T16:29:03.772197+0000","last_active":"2023-11-08T16:29:03.776117+0000","last_peered":"2023-11-08T16:29:03.776117+0000","last_clean":"2023-11-08T16:29:03.776117+0000","last_became_active":"2023-11-08T16:29:03.771699+0000","last_became_peered":"2023-11-08T16:29:03.771699+0000","last_unstale":"2023-11-08T16:29:03.776117+0000","last_undegraded":"2023-11-08T16:29:03.776117+0000","last_fullsized":"2023-11-08T16:29:03.776117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:35:56.104991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.762780+0000","last_change":"2023-11-08T16:29:03.761215+0000","last_active":"2023-11-08T16:29:03.762780+0000","last_peered":"2023-11-08T16:29:03.762780+0000","last_clean":"2023-11-08T16:29:03.762780+0000","last_became_active":"2023-11-08T16:29:03.760855+0000","last_became_peered":"2023-11-08T16:29:03.760855+0000","last_unstale":"2023-11-08T16:29:03.762780+0000","last_undegraded":"2023-11-08T16:29:03.762780+0000","last_fullsized":"2023-11-08T16:29:03.762780+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:05.442030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736058+0000","last_change":"2023-11-08T16:29:03.776139+0000","last_active":"2023-11-08T16:29:04.736058+0000","last_peered":"2023-11-08T16:29:04.736058+0000","last_clean":"2023-11-08T16:29:04.736058+0000","last_became_active":"2023-11-08T16:29:03.775828+0000","last_became_peered":"2023-11-08T16:29:03.775828+0000","last_unstale":"2023-11-08T16:29:04.736058+0000","last_undegraded":"2023-11-08T16:29:04.736058+0000","last_fullsized":"2023-11-08T16:29:04.736058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.778013+0000","last_change":"2023-11-08T16:29:03.774003+0000","last_active":"2023-11-08T16:29:03.778013+0000","last_peered":"2023-11-08T16:29:03.778013+0000","last_clean":"2023-11-08T16:29:03.778013+0000","last_became_active":"2023-11-08T16:29:03.772344+0000","last_became_peered":"2023-11-08T16:29:03.772344+0000","last_unstale":"2023-11-08T16:29:03.778013+0000","last_undegraded":"2023-11-08T16:29:03.778013+0000","last_fullsized":"2023-11-08T16:29:03.778013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:46:56.190555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738706+0000","last_change":"2023-11-08T16:29:03.762765+0000","last_active":"2023-11-08T16:29:04.738706+0000","last_peered":"2023-11-08T16:29:04.738706+0000","last_clean":"2023-11-08T16:29:04.738706+0000","last_became_active":"2023-11-08T16:29:03.762435+0000","last_became_peered":"2023-11-08T16:29:03.762435+0000","last_unstale":"2023-11-08T16:29:04.738706+0000","last_undegraded":"2023-11-08T16:29:04.738706+0000","last_fullsized":"2023-11-08T16:29:04.738706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:58.120335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401083+0000","last_change":"2023-11-08T16:29:03.761315+0000","last_active":"2023-11-08T16:29:05.401083+0000","last_peered":"2023-11-08T16:29:05.401083+0000","last_clean":"2023-11-08T16:29:05.401083+0000","last_became_active":"2023-11-08T16:29:03.761017+0000","last_became_peered":"2023-11-08T16:29:03.761017+0000","last_unstale":"2023-11-08T16:29:05.401083+0000","last_undegraded":"2023-11-08T16:29:05.401083+0000","last_fullsized":"2023-11-08T16:29:05.401083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:46.473609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.770409+0000","last_change":"2023-11-08T16:29:03.766087+0000","last_active":"2023-11-08T16:29:03.770409+0000","last_peered":"2023-11-08T16:29:03.770409+0000","last_clean":"2023-11-08T16:29:03.770409+0000","last_became_active":"2023-11-08T16:29:03.765811+0000","last_became_peered":"2023-11-08T16:29:03.765811+0000","last_unstale":"2023-11-08T16:29:03.770409+0000","last_undegraded":"2023-11-08T16:29:03.770409+0000","last_fullsized":"2023-11-08T16:29:03.770409+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:11.949345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.771964+0000","last_change":"2023-11-08T16:29:03.770480+0000","last_active":"2023-11-08T16:29:03.771964+0000","last_peered":"2023-11-08T16:29:03.771964+0000","last_clean":"2023-11-08T16:29:03.771964+0000","last_became_active":"2023-11-08T16:29:03.770166+0000","last_became_peered":"2023-11-08T16:29:03.770166+0000","last_unstale":"2023-11-08T16:29:03.771964+0000","last_undegraded":"2023-11-08T16:29:03.771964+0000","last_fullsized":"2023-11-08T16:29:03.771964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:08.473156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736199+0000","last_change":"2023-11-08T16:29:02.745095+0000","last_active":"2023-11-08T16:29:04.736199+0000","last_peered":"2023-11-08T16:29:04.736199+0000","last_clean":"2023-11-08T16:29:04.736199+0000","last_became_active":"2023-11-08T16:29:02.744708+0000","last_became_peered":"2023-11-08T16:29:02.744708+0000","last_unstale":"2023-11-08T16:29:04.736199+0000","last_undegraded":"2023-11-08T16:29:04.736199+0000","last_fullsized":"2023-11-08T16:29:04.736199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.777993+0000","last_change":"2023-11-08T16:29:03.773134+0000","last_active":"2023-11-08T16:29:03.777993+0000","last_peered":"2023-11-08T16:29:03.777993+0000","last_clean":"2023-11-08T16:29:03.777993+0000","last_became_active":"2023-11-08T16:29:03.772787+0000","last_became_peered":"2023-11-08T16:29:03.772787+0000","last_unstale":"2023-11-08T16:29:03.777993+0000","last_undegraded":"2023-11-08T16:29:03.777993+0000","last_fullsized":"2023-11-08T16:29:03.777993+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:56.745110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.789586+0000","last_change":"2023-11-08T16:29:02.752072+0000","last_active":"2023-11-08T16:29:03.789586+0000","last_peered":"2023-11-08T16:29:03.789586+0000","last_clean":"2023-11-08T16:29:03.789586+0000","last_became_active":"2023-11-08T16:29:02.751752+0000","last_became_peered":"2023-11-08T16:29:02.751752+0000","last_unstale":"2023-11-08T16:29:03.789586+0000","last_undegraded":"2023-11-08T16:29:03.789586+0000","last_fullsized":"2023-11-08T16:29:03.789586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:21.228079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735516+0000","last_change":"2023-11-08T16:29:02.751815+0000","last_active":"2023-11-08T16:29:04.735516+0000","last_peered":"2023-11-08T16:29:04.735516+0000","last_clean":"2023-11-08T16:29:04.735516+0000","last_became_active":"2023-11-08T16:29:02.751501+0000","last_became_peered":"2023-11-08T16:29:02.751501+0000","last_unstale":"2023-11-08T16:29:04.735516+0000","last_undegraded":"2023-11-08T16:29:04.735516+0000","last_fullsized":"2023-11-08T16:29:04.735516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:13.053337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735399+0000","last_change":"2023-11-08T16:29:03.758235+0000","last_active":"2023-11-08T16:29:04.735399+0000","last_peered":"2023-11-08T16:29:04.735399+0000","last_clean":"2023-11-08T16:29:04.735399+0000","last_became_active":"2023-11-08T16:29:03.757937+0000","last_became_peered":"2023-11-08T16:29:03.757937+0000","last_unstale":"2023-11-08T16:29:04.735399+0000","last_undegraded":"2023-11-08T16:29:04.735399+0000","last_fullsized":"2023-11-08T16:29:04.735399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:45:23.080055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.737829+0000","last_change":"2023-11-08T16:29:02.741267+0000","last_active":"2023-11-08T16:29:04.737829+0000","last_peered":"2023-11-08T16:29:04.737829+0000","last_clean":"2023-11-08T16:29:04.737829+0000","last_became_active":"2023-11-08T16:29:02.740048+0000","last_became_peered":"2023-11-08T16:29:02.740048+0000","last_unstale":"2023-11-08T16:29:04.737829+0000","last_undegraded":"2023-11-08T16:29:04.737829+0000","last_fullsized":"2023-11-08T16:29:04.737829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:10.935421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.758904+0000","last_change":"2023-11-08T16:29:03.755015+0000","last_active":"2023-11-08T16:29:03.758904+0000","last_peered":"2023-11-08T16:29:03.758904+0000","last_clean":"2023-11-08T16:29:03.758904+0000","last_became_active":"2023-11-08T16:29:03.753403+0000","last_became_peered":"2023-11-08T16:29:03.753403+0000","last_unstale":"2023-11-08T16:29:03.758904+0000","last_undegraded":"2023-11-08T16:29:03.758904+0000","last_fullsized":"2023-11-08T16:29:03.758904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:56.745110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.749550+0000","last_change":"2023-11-08T16:29:02.749431+0000","last_active":"2023-11-08T16:29:03.749550+0000","last_peered":"2023-11-08T16:29:03.749550+0000","last_clean":"2023-11-08T16:29:03.749550+0000","last_became_active":"2023-11-08T16:29:02.748783+0000","last_became_peered":"2023-11-08T16:29:02.748783+0000","last_unstale":"2023-11-08T16:29:03.749550+0000","last_undegraded":"2023-11-08T16:29:03.749550+0000","last_fullsized":"2023-11-08T16:29:03.749550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:45.459685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739258+0000","last_change":"2023-11-08T16:29:03.759712+0000","last_active":"2023-11-08T16:29:04.739258+0000","last_peered":"2023-11-08T16:29:04.739258+0000","last_clean":"2023-11-08T16:29:04.739258+0000","last_became_active":"2023-11-08T16:29:03.759328+0000","last_became_peered":"2023-11-08T16:29:03.759328+0000","last_unstale":"2023-11-08T16:29:04.739258+0000","last_undegraded":"2023-11-08T16:29:04.739258+0000","last_fullsized":"2023-11-08T16:29:04.739258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:42.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739532+0000","last_change":"2023-11-08T16:29:02.749359+0000","last_active":"2023-11-08T16:29:04.739532+0000","last_peered":"2023-11-08T16:29:04.739532+0000","last_clean":"2023-11-08T16:29:04.739532+0000","last_became_active":"2023-11-08T16:29:02.748992+0000","last_became_peered":"2023-11-08T16:29:02.748992+0000","last_unstale":"2023-11-08T16:29:04.739532+0000","last_undegraded":"2023-11-08T16:29:04.739532+0000","last_fullsized":"2023-11-08T16:29:04.739532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:29.529005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.773627+0000","last_change":"2023-11-08T16:29:03.766416+0000","last_active":"2023-11-08T16:29:03.773627+0000","last_peered":"2023-11-08T16:29:03.773627+0000","last_clean":"2023-11-08T16:29:03.773627+0000","last_became_active":"2023-11-08T16:29:03.766072+0000","last_became_peered":"2023-11-08T16:29:03.766072+0000","last_unstale":"2023-11-08T16:29:03.773627+0000","last_undegraded":"2023-11-08T16:29:03.773627+0000","last_fullsized":"2023-11-08T16:29:03.773627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:08.473156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.796204+0000","last_change":"2023-11-08T16:29:02.747464+0000","last_active":"2023-11-08T16:29:03.796204+0000","last_peered":"2023-11-08T16:29:03.796204+0000","last_clean":"2023-11-08T16:29:03.796204+0000","last_became_active":"2023-11-08T16:29:02.747178+0000","last_became_peered":"2023-11-08T16:29:02.747178+0000","last_unstale":"2023-11-08T16:29:03.796204+0000","last_undegraded":"2023-11-08T16:29:03.796204+0000","last_fullsized":"2023-11-08T16:29:03.796204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:13.053337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.773483+0000","last_change":"2023-11-08T16:29:03.768953+0000","last_active":"2023-11-08T16:29:03.773483+0000","last_peered":"2023-11-08T16:29:03.773483+0000","last_clean":"2023-11-08T16:29:03.773483+0000","last_became_active":"2023-11-08T16:29:03.768671+0000","last_became_peered":"2023-11-08T16:29:03.768671+0000","last_unstale":"2023-11-08T16:29:03.773483+0000","last_undegraded":"2023-11-08T16:29:03.773483+0000","last_fullsized":"2023-11-08T16:29:03.773483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:37:46.761474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.797258+0000","last_change":"2023-11-08T16:29:02.750178+0000","last_active":"2023-11-08T16:29:03.797258+0000","last_peered":"2023-11-08T16:29:03.797258+0000","last_clean":"2023-11-08T16:29:03.797258+0000","last_became_active":"2023-11-08T16:29:02.749883+0000","last_became_peered":"2023-11-08T16:29:02.749883+0000","last_unstale":"2023-11-08T16:29:03.797258+0000","last_undegraded":"2023-11-08T16:29:03.797258+0000","last_fullsized":"2023-11-08T16:29:03.797258+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:50.401270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739830+0000","last_change":"2023-11-08T16:29:03.761268+0000","last_active":"2023-11-08T16:29:04.739830+0000","last_peered":"2023-11-08T16:29:04.739830+0000","last_clean":"2023-11-08T16:29:04.739830+0000","last_became_active":"2023-11-08T16:29:03.760923+0000","last_became_peered":"2023-11-08T16:29:03.760923+0000","last_unstale":"2023-11-08T16:29:04.739830+0000","last_undegraded":"2023-11-08T16:29:04.739830+0000","last_fullsized":"2023-11-08T16:29:04.739830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:18:29.767162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.792387+0000","last_change":"2023-11-08T16:29:02.741262+0000","last_active":"2023-11-08T16:29:03.792387+0000","last_peered":"2023-11-08T16:29:03.792387+0000","last_clean":"2023-11-08T16:29:03.792387+0000","last_became_active":"2023-11-08T16:29:02.740954+0000","last_became_peered":"2023-11-08T16:29:02.740954+0000","last_unstale":"2023-11-08T16:29:03.792387+0000","last_undegraded":"2023-11-08T16:29:03.792387+0000","last_fullsized":"2023-11-08T16:29:03.792387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:13.053337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.777829+0000","last_change":"2023-11-08T16:29:03.772717+0000","last_active":"2023-11-08T16:29:03.777829+0000","last_peered":"2023-11-08T16:29:03.777829+0000","last_clean":"2023-11-08T16:29:03.777829+0000","last_became_active":"2023-11-08T16:29:03.772441+0000","last_became_peered":"2023-11-08T16:29:03.772441+0000","last_unstale":"2023-11-08T16:29:03.777829+0000","last_undegraded":"2023-11-08T16:29:03.777829+0000","last_fullsized":"2023-11-08T16:29:03.777829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:05.442030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738807+0000","last_change":"2023-11-08T16:29:02.744749+0000","last_active":"2023-11-08T16:29:04.738807+0000","last_peered":"2023-11-08T16:29:04.738807+0000","last_clean":"2023-11-08T16:29:04.738807+0000","last_became_active":"2023-11-08T16:29:02.744383+0000","last_became_peered":"2023-11-08T16:29:02.744383+0000","last_unstale":"2023-11-08T16:29:04.738807+0000","last_undegraded":"2023-11-08T16:29:04.738807+0000","last_fullsized":"2023-11-08T16:29:04.738807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:32.723904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740184+0000","last_change":"2023-11-08T16:29:03.771356+0000","last_active":"2023-11-08T16:29:04.740184+0000","last_peered":"2023-11-08T16:29:04.740184+0000","last_clean":"2023-11-08T16:29:04.740184+0000","last_became_active":"2023-11-08T16:29:03.771082+0000","last_became_peered":"2023-11-08T16:29:03.771082+0000","last_unstale":"2023-11-08T16:29:04.740184+0000","last_undegraded":"2023-11-08T16:29:04.740184+0000","last_fullsized":"2023-11-08T16:29:04.740184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:11:11.384242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.746693+0000","last_change":"2023-11-08T16:29:02.752309+0000","last_active":"2023-11-08T16:29:03.746693+0000","last_peered":"2023-11-08T16:29:03.746693+0000","last_clean":"2023-11-08T16:29:03.746693+0000","last_became_active":"2023-11-08T16:29:02.751973+0000","last_became_peered":"2023-11-08T16:29:02.751973+0000","last_unstale":"2023-11-08T16:29:03.746693+0000","last_undegraded":"2023-11-08T16:29:03.746693+0000","last_fullsized":"2023-11-08T16:29:03.746693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.768685+0000","last_change":"2023-11-08T16:29:03.759186+0000","last_active":"2023-11-08T16:29:03.768685+0000","last_peered":"2023-11-08T16:29:03.768685+0000","last_clean":"2023-11-08T16:29:03.768685+0000","last_became_active":"2023-11-08T16:29:03.758790+0000","last_became_peered":"2023-11-08T16:29:03.758790+0000","last_unstale":"2023-11-08T16:29:03.768685+0000","last_undegraded":"2023-11-08T16:29:03.768685+0000","last_fullsized":"2023-11-08T16:29:03.768685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:08.473156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.400909+0000","last_change":"2023-11-08T16:29:02.736184+0000","last_active":"2023-11-08T16:29:05.400909+0000","last_peered":"2023-11-08T16:29:05.400909+0000","last_clean":"2023-11-08T16:29:05.400909+0000","last_became_active":"2023-11-08T16:29:02.735423+0000","last_became_peered":"2023-11-08T16:29:02.735423+0000","last_unstale":"2023-11-08T16:29:05.400909+0000","last_undegraded":"2023-11-08T16:29:05.400909+0000","last_fullsized":"2023-11-08T16:29:05.400909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:46:55.176632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738431+0000","last_change":"2023-11-08T16:29:03.773617+0000","last_active":"2023-11-08T16:29:04.738431+0000","last_peered":"2023-11-08T16:29:04.738431+0000","last_clean":"2023-11-08T16:29:04.738431+0000","last_became_active":"2023-11-08T16:29:03.772381+0000","last_became_peered":"2023-11-08T16:29:03.772381+0000","last_unstale":"2023-11-08T16:29:04.738431+0000","last_undegraded":"2023-11-08T16:29:04.738431+0000","last_fullsized":"2023-11-08T16:29:04.738431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:10.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.792658+0000","last_change":"2023-11-08T16:29:02.750733+0000","last_active":"2023-11-08T16:29:03.792658+0000","last_peered":"2023-11-08T16:29:03.792658+0000","last_clean":"2023-11-08T16:29:03.792658+0000","last_became_active":"2023-11-08T16:29:02.750390+0000","last_became_peered":"2023-11-08T16:29:02.750390+0000","last_unstale":"2023-11-08T16:29:03.792658+0000","last_undegraded":"2023-11-08T16:29:03.792658+0000","last_fullsized":"2023-11-08T16:29:03.792658+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739867+0000","last_change":"2023-11-08T16:29:03.772892+0000","last_active":"2023-11-08T16:29:04.739867+0000","last_peered":"2023-11-08T16:29:04.739867+0000","last_clean":"2023-11-08T16:29:04.739867+0000","last_became_active":"2023-11-08T16:29:03.772403+0000","last_became_peered":"2023-11-08T16:29:03.772403+0000","last_unstale":"2023-11-08T16:29:04.739867+0000","last_undegraded":"2023-11-08T16:29:04.739867+0000","last_fullsized":"2023-11-08T16:29:04.739867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:58.120335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401406+0000","last_change":"2023-11-08T16:29:02.737476+0000","last_active":"2023-11-08T16:29:05.401406+0000","last_peered":"2023-11-08T16:29:05.401406+0000","last_clean":"2023-11-08T16:29:05.401406+0000","last_became_active":"2023-11-08T16:29:02.737136+0000","last_became_peered":"2023-11-08T16:29:02.737136+0000","last_unstale":"2023-11-08T16:29:05.401406+0000","last_undegraded":"2023-11-08T16:29:05.401406+0000","last_fullsized":"2023-11-08T16:29:05.401406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:55.731186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.741221+0000","last_change":"2023-11-08T16:29:03.768059+0000","last_active":"2023-11-08T16:29:04.741221+0000","last_peered":"2023-11-08T16:29:04.741221+0000","last_clean":"2023-11-08T16:29:04.741221+0000","last_became_active":"2023-11-08T16:29:03.767846+0000","last_became_peered":"2023-11-08T16:29:03.767846+0000","last_unstale":"2023-11-08T16:29:04.741221+0000","last_undegraded":"2023-11-08T16:29:04.741221+0000","last_fullsized":"2023-11-08T16:29:04.741221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:03:32.436141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.795454+0000","last_change":"2023-11-08T16:29:02.740588+0000","last_active":"2023-11-08T16:29:03.795454+0000","last_peered":"2023-11-08T16:29:03.795454+0000","last_clean":"2023-11-08T16:29:03.795454+0000","last_became_active":"2023-11-08T16:29:02.740216+0000","last_became_peered":"2023-11-08T16:29:02.740216+0000","last_unstale":"2023-11-08T16:29:03.795454+0000","last_undegraded":"2023-11-08T16:29:03.795454+0000","last_fullsized":"2023-11-08T16:29:03.795454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740514+0000","last_change":"2023-11-08T16:29:03.755779+0000","last_active":"2023-11-08T16:29:04.740514+0000","last_peered":"2023-11-08T16:29:04.740514+0000","last_clean":"2023-11-08T16:29:04.740514+0000","last_became_active":"2023-11-08T16:29:03.755430+0000","last_became_peered":"2023-11-08T16:29:03.755430+0000","last_unstale":"2023-11-08T16:29:04.740514+0000","last_undegraded":"2023-11-08T16:29:04.740514+0000","last_fullsized":"2023-11-08T16:29:04.740514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:20:02.456685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401088+0000","last_change":"2023-11-08T16:29:02.744228+0000","last_active":"2023-11-08T16:29:05.401088+0000","last_peered":"2023-11-08T16:29:05.401088+0000","last_clean":"2023-11-08T16:29:05.401088+0000","last_became_active":"2023-11-08T16:29:02.743952+0000","last_became_peered":"2023-11-08T16:29:02.743952+0000","last_unstale":"2023-11-08T16:29:05.401088+0000","last_undegraded":"2023-11-08T16:29:05.401088+0000","last_fullsized":"2023-11-08T16:29:05.401088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:07.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740168+0000","last_change":"2023-11-08T16:29:03.759959+0000","last_active":"2023-11-08T16:29:04.740168+0000","last_peered":"2023-11-08T16:29:04.740168+0000","last_clean":"2023-11-08T16:29:04.740168+0000","last_became_active":"2023-11-08T16:29:03.759633+0000","last_became_peered":"2023-11-08T16:29:03.759633+0000","last_unstale":"2023-11-08T16:29:04.740168+0000","last_undegraded":"2023-11-08T16:29:04.740168+0000","last_fullsized":"2023-11-08T16:29:04.740168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:18.395303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.735559+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:29:03.735559+0000","last_peered":"2023-11-08T16:29:03.735559+0000","last_clean":"2023-11-08T16:29:03.735559+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:29:03.735559+0000","last_undegraded":"2023-11-08T16:29:03.735559+0000","last_fullsized":"2023-11-08T16:29:03.735559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739097+0000","last_change":"2023-11-08T16:29:02.746555+0000","last_active":"2023-11-08T16:29:04.739097+0000","last_peered":"2023-11-08T16:29:04.739097+0000","last_clean":"2023-11-08T16:29:04.739097+0000","last_became_active":"2023-11-08T16:29:02.745687+0000","last_became_peered":"2023-11-08T16:29:02.745687+0000","last_unstale":"2023-11-08T16:29:04.739097+0000","last_undegraded":"2023-11-08T16:29:04.739097+0000","last_fullsized":"2023-11-08T16:29:04.739097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:45.459685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736447+0000","last_change":"2023-11-08T16:29:03.767888+0000","last_active":"2023-11-08T16:29:04.736447+0000","last_peered":"2023-11-08T16:29:04.736447+0000","last_clean":"2023-11-08T16:29:04.736447+0000","last_became_active":"2023-11-08T16:29:03.767484+0000","last_became_peered":"2023-11-08T16:29:03.767484+0000","last_unstale":"2023-11-08T16:29:04.736447+0000","last_undegraded":"2023-11-08T16:29:04.736447+0000","last_fullsized":"2023-11-08T16:29:04.736447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:10.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738882+0000","last_change":"2023-11-08T16:29:02.746067+0000","last_active":"2023-11-08T16:29:04.738882+0000","last_peered":"2023-11-08T16:29:04.738882+0000","last_clean":"2023-11-08T16:29:04.738882+0000","last_became_active":"2023-11-08T16:29:02.745767+0000","last_became_peered":"2023-11-08T16:29:02.745767+0000","last_unstale":"2023-11-08T16:29:04.738882+0000","last_undegraded":"2023-11-08T16:29:04.738882+0000","last_fullsized":"2023-11-08T16:29:04.738882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:10.935421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.777949+0000","last_change":"2023-11-08T16:29:03.773037+0000","last_active":"2023-11-08T16:29:03.777949+0000","last_peered":"2023-11-08T16:29:03.777949+0000","last_clean":"2023-11-08T16:29:03.777949+0000","last_became_active":"2023-11-08T16:29:03.771599+0000","last_became_peered":"2023-11-08T16:29:03.771599+0000","last_unstale":"2023-11-08T16:29:03.777949+0000","last_undegraded":"2023-11-08T16:29:03.777949+0000","last_fullsized":"2023-11-08T16:29:03.777949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:35.452265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401068+0000","last_change":"2023-11-08T16:29:02.737753+0000","last_active":"2023-11-08T16:29:05.401068+0000","last_peered":"2023-11-08T16:29:05.401068+0000","last_clean":"2023-11-08T16:29:05.401068+0000","last_became_active":"2023-11-08T16:29:02.737374+0000","last_became_peered":"2023-11-08T16:29:02.737374+0000","last_unstale":"2023-11-08T16:29:05.401068+0000","last_undegraded":"2023-11-08T16:29:05.401068+0000","last_fullsized":"2023-11-08T16:29:05.401068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:10.935421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.761375+0000","last_change":"2023-11-08T16:29:03.759721+0000","last_active":"2023-11-08T16:29:03.761375+0000","last_peered":"2023-11-08T16:29:03.761375+0000","last_clean":"2023-11-08T16:29:03.761375+0000","last_became_active":"2023-11-08T16:29:03.758129+0000","last_became_peered":"2023-11-08T16:29:03.758129+0000","last_unstale":"2023-11-08T16:29:03.761375+0000","last_undegraded":"2023-11-08T16:29:03.761375+0000","last_fullsized":"2023-11-08T16:29:03.761375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:46:56.190555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738972+0000","last_change":"2023-11-08T16:29:02.742034+0000","last_active":"2023-11-08T16:29:04.738972+0000","last_peered":"2023-11-08T16:29:04.738972+0000","last_clean":"2023-11-08T16:29:04.738972+0000","last_became_active":"2023-11-08T16:29:02.741705+0000","last_became_peered":"2023-11-08T16:29:02.741705+0000","last_unstale":"2023-11-08T16:29:04.738972+0000","last_undegraded":"2023-11-08T16:29:04.738972+0000","last_fullsized":"2023-11-08T16:29:04.738972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:03.856474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.780528+0000","last_change":"2023-11-08T16:29:03.777239+0000","last_active":"2023-11-08T16:29:03.780528+0000","last_peered":"2023-11-08T16:29:03.780528+0000","last_clean":"2023-11-08T16:29:03.780528+0000","last_became_active":"2023-11-08T16:29:03.776975+0000","last_became_peered":"2023-11-08T16:29:03.776975+0000","last_unstale":"2023-11-08T16:29:03.780528+0000","last_undegraded":"2023-11-08T16:29:03.780528+0000","last_fullsized":"2023-11-08T16:29:03.780528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:35.452265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.733803+0000","last_change":"2023-11-08T16:29:02.733716+0000","last_active":"2023-11-08T16:29:03.733803+0000","last_peered":"2023-11-08T16:29:03.733803+0000","last_clean":"2023-11-08T16:29:03.733803+0000","last_became_active":"2023-11-08T16:29:02.733435+0000","last_became_peered":"2023-11-08T16:29:02.733435+0000","last_unstale":"2023-11-08T16:29:03.733803+0000","last_undegraded":"2023-11-08T16:29:03.733803+0000","last_fullsized":"2023-11-08T16:29:03.733803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739982+0000","last_change":"2023-11-08T16:29:03.764171+0000","last_active":"2023-11-08T16:29:04.739982+0000","last_peered":"2023-11-08T16:29:04.739982+0000","last_clean":"2023-11-08T16:29:04.739982+0000","last_became_active":"2023-11-08T16:29:03.762549+0000","last_became_peered":"2023-11-08T16:29:03.762549+0000","last_unstale":"2023-11-08T16:29:04.739982+0000","last_undegraded":"2023-11-08T16:29:04.739982+0000","last_fullsized":"2023-11-08T16:29:04.739982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:11:11.384242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740565+0000","last_change":"2023-11-08T16:29:02.744381+0000","last_active":"2023-11-08T16:29:04.740565+0000","last_peered":"2023-11-08T16:29:04.740565+0000","last_clean":"2023-11-08T16:29:04.740565+0000","last_became_active":"2023-11-08T16:29:02.744077+0000","last_became_peered":"2023-11-08T16:29:02.744077+0000","last_unstale":"2023-11-08T16:29:04.740565+0000","last_undegraded":"2023-11-08T16:29:04.740565+0000","last_fullsized":"2023-11-08T16:29:04.740565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:43.416193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.759205+0000","last_change":"2023-11-08T16:29:03.753962+0000","last_active":"2023-11-08T16:29:03.759205+0000","last_peered":"2023-11-08T16:29:03.759205+0000","last_clean":"2023-11-08T16:29:03.759205+0000","last_became_active":"2023-11-08T16:29:03.753317+0000","last_became_peered":"2023-11-08T16:29:03.753317+0000","last_unstale":"2023-11-08T16:29:03.759205+0000","last_undegraded":"2023-11-08T16:29:03.759205+0000","last_fullsized":"2023-11-08T16:29:03.759205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:46.473609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.736155+0000","last_change":"2023-11-08T16:29:02.745315+0000","last_active":"2023-11-08T16:29:03.736155+0000","last_peered":"2023-11-08T16:29:03.736155+0000","last_clean":"2023-11-08T16:29:03.736155+0000","last_became_active":"2023-11-08T16:29:02.744940+0000","last_became_peered":"2023-11-08T16:29:02.744940+0000","last_unstale":"2023-11-08T16:29:03.736155+0000","last_undegraded":"2023-11-08T16:29:03.736155+0000","last_fullsized":"2023-11-08T16:29:03.736155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:45:22.066131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735964+0000","last_change":"2023-11-08T16:29:03.763070+0000","last_active":"2023-11-08T16:29:04.735964+0000","last_peered":"2023-11-08T16:29:04.735964+0000","last_clean":"2023-11-08T16:29:04.735964+0000","last_became_active":"2023-11-08T16:29:03.762709+0000","last_became_peered":"2023-11-08T16:29:03.762709+0000","last_unstale":"2023-11-08T16:29:04.735964+0000","last_undegraded":"2023-11-08T16:29:04.735964+0000","last_fullsized":"2023-11-08T16:29:04.735964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:33.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.790266+0000","last_change":"2023-11-08T16:29:02.749634+0000","last_active":"2023-11-08T16:29:03.790266+0000","last_peered":"2023-11-08T16:29:03.790266+0000","last_clean":"2023-11-08T16:29:03.790266+0000","last_became_active":"2023-11-08T16:29:02.749138+0000","last_became_peered":"2023-11-08T16:29:02.749138+0000","last_unstale":"2023-11-08T16:29:03.790266+0000","last_undegraded":"2023-11-08T16:29:03.790266+0000","last_fullsized":"2023-11-08T16:29:03.790266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.778062+0000","last_change":"2023-11-08T16:29:03.773063+0000","last_active":"2023-11-08T16:29:03.778062+0000","last_peered":"2023-11-08T16:29:03.778062+0000","last_clean":"2023-11-08T16:29:03.778062+0000","last_became_active":"2023-11-08T16:29:03.772775+0000","last_became_peered":"2023-11-08T16:29:03.772775+0000","last_unstale":"2023-11-08T16:29:03.778062+0000","last_undegraded":"2023-11-08T16:29:03.778062+0000","last_fullsized":"2023-11-08T16:29:03.778062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:09.861000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738667+0000","last_change":"2023-11-08T16:29:02.745343+0000","last_active":"2023-11-08T16:29:04.738667+0000","last_peered":"2023-11-08T16:29:04.738667+0000","last_clean":"2023-11-08T16:29:04.738667+0000","last_became_active":"2023-11-08T16:29:02.744884+0000","last_became_peered":"2023-11-08T16:29:02.744884+0000","last_unstale":"2023-11-08T16:29:04.738667+0000","last_undegraded":"2023-11-08T16:29:04.738667+0000","last_fullsized":"2023-11-08T16:29:04.738667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.770763+0000","last_change":"2023-11-08T16:29:03.767447+0000","last_active":"2023-11-08T16:29:03.770763+0000","last_peered":"2023-11-08T16:29:03.770763+0000","last_clean":"2023-11-08T16:29:03.770763+0000","last_became_active":"2023-11-08T16:29:03.767116+0000","last_became_peered":"2023-11-08T16:29:03.767116+0000","last_unstale":"2023-11-08T16:29:03.770763+0000","last_undegraded":"2023-11-08T16:29:03.770763+0000","last_fullsized":"2023-11-08T16:29:03.770763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:45:47.674315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.735887+0000","last_change":"2023-11-08T16:29:02.746234+0000","last_active":"2023-11-08T16:29:03.735887+0000","last_peered":"2023-11-08T16:29:03.735887+0000","last_clean":"2023-11-08T16:29:03.735887+0000","last_became_active":"2023-11-08T16:29:02.745941+0000","last_became_peered":"2023-11-08T16:29:02.745941+0000","last_unstale":"2023-11-08T16:29:03.735887+0000","last_undegraded":"2023-11-08T16:29:03.735887+0000","last_fullsized":"2023-11-08T16:29:03.735887+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:35:55.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735534+0000","last_change":"2023-11-08T16:29:03.754330+0000","last_active":"2023-11-08T16:29:04.735534+0000","last_peered":"2023-11-08T16:29:04.735534+0000","last_clean":"2023-11-08T16:29:04.735534+0000","last_became_active":"2023-11-08T16:29:03.751108+0000","last_became_peered":"2023-11-08T16:29:03.751108+0000","last_unstale":"2023-11-08T16:29:04.735534+0000","last_undegraded":"2023-11-08T16:29:04.735534+0000","last_fullsized":"2023-11-08T16:29:04.735534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:30.542928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.733474+0000","last_change":"2023-11-08T16:29:02.737421+0000","last_active":"2023-11-08T16:29:03.733474+0000","last_peered":"2023-11-08T16:29:03.733474+0000","last_clean":"2023-11-08T16:29:03.733474+0000","last_became_active":"2023-11-08T16:29:02.737099+0000","last_became_peered":"2023-11-08T16:29:02.737099+0000","last_unstale":"2023-11-08T16:29:03.733474+0000","last_undegraded":"2023-11-08T16:29:03.733474+0000","last_fullsized":"2023-11-08T16:29:03.733474+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:07.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736361+0000","last_change":"2023-11-08T16:29:03.774533+0000","last_active":"2023-11-08T16:29:04.736361+0000","last_peered":"2023-11-08T16:29:04.736361+0000","last_clean":"2023-11-08T16:29:04.736361+0000","last_became_active":"2023-11-08T16:29:03.774212+0000","last_became_peered":"2023-11-08T16:29:03.774212+0000","last_unstale":"2023-11-08T16:29:04.736361+0000","last_undegraded":"2023-11-08T16:29:04.736361+0000","last_fullsized":"2023-11-08T16:29:04.736361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:27.108912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740429+0000","last_change":"2023-11-08T16:29:02.754015+0000","last_active":"2023-11-08T16:29:04.740429+0000","last_peered":"2023-11-08T16:29:04.740429+0000","last_clean":"2023-11-08T16:29:04.740429+0000","last_became_active":"2023-11-08T16:29:02.753688+0000","last_became_peered":"2023-11-08T16:29:02.753688+0000","last_unstale":"2023-11-08T16:29:04.740429+0000","last_undegraded":"2023-11-08T16:29:04.740429+0000","last_fullsized":"2023-11-08T16:29:04.740429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.775998+0000","last_change":"2023-11-08T16:29:03.771420+0000","last_active":"2023-11-08T16:29:03.775998+0000","last_peered":"2023-11-08T16:29:03.775998+0000","last_clean":"2023-11-08T16:29:03.775998+0000","last_became_active":"2023-11-08T16:29:03.771137+0000","last_became_peered":"2023-11-08T16:29:03.771137+0000","last_unstale":"2023-11-08T16:29:03.775998+0000","last_undegraded":"2023-11-08T16:29:03.775998+0000","last_fullsized":"2023-11-08T16:29:03.775998+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:05.442030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739672+0000","last_change":"2023-11-08T16:29:03.774156+0000","last_active":"2023-11-08T16:29:04.739672+0000","last_peered":"2023-11-08T16:29:04.739672+0000","last_clean":"2023-11-08T16:29:04.739672+0000","last_became_active":"2023-11-08T16:29:03.773842+0000","last_became_peered":"2023-11-08T16:29:03.773842+0000","last_unstale":"2023-11-08T16:29:04.739672+0000","last_undegraded":"2023-11-08T16:29:04.739672+0000","last_fullsized":"2023-11-08T16:29:04.739672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:18.395303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738809+0000","last_change":"2023-11-08T16:29:02.747528+0000","last_active":"2023-11-08T16:29:04.738809+0000","last_peered":"2023-11-08T16:29:04.738809+0000","last_clean":"2023-11-08T16:29:04.738809+0000","last_became_active":"2023-11-08T16:29:02.747113+0000","last_became_peered":"2023-11-08T16:29:02.747113+0000","last_unstale":"2023-11-08T16:29:04.738809+0000","last_undegraded":"2023-11-08T16:29:04.738809+0000","last_fullsized":"2023-11-08T16:29:04.738809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:46:55.176632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738937+0000","last_change":"2023-11-08T16:29:03.778220+0000","last_active":"2023-11-08T16:29:04.738937+0000","last_peered":"2023-11-08T16:29:04.738937+0000","last_clean":"2023-11-08T16:29:04.738937+0000","last_became_active":"2023-11-08T16:29:03.777800+0000","last_became_peered":"2023-11-08T16:29:03.777800+0000","last_unstale":"2023-11-08T16:29:04.738937+0000","last_undegraded":"2023-11-08T16:29:04.738937+0000","last_fullsized":"2023-11-08T16:29:04.738937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:18:29.767162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740343+0000","last_change":"2023-11-08T16:29:02.749398+0000","last_active":"2023-11-08T16:29:04.740343+0000","last_peered":"2023-11-08T16:29:04.740343+0000","last_clean":"2023-11-08T16:29:04.740343+0000","last_became_active":"2023-11-08T16:29:02.749084+0000","last_became_peered":"2023-11-08T16:29:02.749084+0000","last_unstale":"2023-11-08T16:29:04.740343+0000","last_undegraded":"2023-11-08T16:29:04.740343+0000","last_fullsized":"2023-11-08T16:29:04.740343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:03.856474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.779949+0000","last_change":"2023-11-08T16:29:03.776424+0000","last_active":"2023-11-08T16:29:03.779949+0000","last_peered":"2023-11-08T16:29:03.779949+0000","last_clean":"2023-11-08T16:29:03.779949+0000","last_became_active":"2023-11-08T16:29:03.774967+0000","last_became_peered":"2023-11-08T16:29:03.774967+0000","last_unstale":"2023-11-08T16:29:03.779949+0000","last_undegraded":"2023-11-08T16:29:03.779949+0000","last_fullsized":"2023-11-08T16:29:03.779949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:22.242002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736286+0000","last_change":"2023-11-08T16:29:02.733433+0000","last_active":"2023-11-08T16:29:04.736286+0000","last_peered":"2023-11-08T16:29:04.736286+0000","last_clean":"2023-11-08T16:29:04.736286+0000","last_became_active":"2023-11-08T16:29:02.733126+0000","last_became_peered":"2023-11-08T16:29:02.733126+0000","last_unstale":"2023-11-08T16:29:04.736286+0000","last_undegraded":"2023-11-08T16:29:04.736286+0000","last_fullsized":"2023-11-08T16:29:04.736286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:35:55.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736155+0000","last_change":"2023-11-08T16:29:03.770442+0000","last_active":"2023-11-08T16:29:04.736155+0000","last_peered":"2023-11-08T16:29:04.736155+0000","last_clean":"2023-11-08T16:29:04.736155+0000","last_became_active":"2023-11-08T16:29:03.770158+0000","last_became_peered":"2023-11-08T16:29:03.770158+0000","last_unstale":"2023-11-08T16:29:04.736155+0000","last_undegraded":"2023-11-08T16:29:04.736155+0000","last_fullsized":"2023-11-08T16:29:04.736155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:06.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.792000+0000","last_change":"2023-11-08T16:29:02.744925+0000","last_active":"2023-11-08T16:29:03.792000+0000","last_peered":"2023-11-08T16:29:03.792000+0000","last_clean":"2023-11-08T16:29:03.792000+0000","last_became_active":"2023-11-08T16:29:02.744574+0000","last_became_peered":"2023-11-08T16:29:02.744574+0000","last_unstale":"2023-11-08T16:29:03.792000+0000","last_undegraded":"2023-11-08T16:29:03.792000+0000","last_fullsized":"2023-11-08T16:29:03.792000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:35:55.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93500000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":70893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.522}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.47}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8690000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0190000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9610000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0979999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":217088,"data_stored":69407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.506}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7789999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7010000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1359999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3420000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.581}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98799999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.167}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3149999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":69441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0339999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3179999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1659999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.093}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.984}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3609999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":811008,"data_stored":660287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4350000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4810000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0489999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:29:08.731 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T16:29:08.732 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-08T16:29:08.732 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-08T16:29:08.733 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-08T16:29:08.733 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-08T16:29:08.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.982+0000 7f3ea0c0d640 1 Processor -- start 2023-11-08T16:29:08.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.982+0000 7f3ea0c0d640 1 -- start start 2023-11-08T16:29:08.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.984+0000 7f3ea0c0d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c15aac0 0x7f3e9c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.984+0000 7f3ea0c0d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3e9c0716e0 0x7f3e9c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.984+0000 7f3ea0c0d640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c070470 con 0x7f3e9c0cf250 2023-11-08T16:29:08.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.984+0000 7f3ea0c0d640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c06f650 con 0x7f3e9c0716e0 2023-11-08T16:29:08.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.984+0000 7f3ea0c0d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0cf6c0 con 0x7f3e9c15aac0 2023-11-08T16:29:08.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.984+0000 7f3e9b7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c15aac0 0x7f3e9c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:08.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.985+0000 7f3e9b7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c15aac0 0x7f3e9c0711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:43444/0 (socket says 172.21.15.81:43444) 2023-11-08T16:29:08.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.985+0000 7f3e9b7fe640 1 -- 172.21.15.81:0/207984445 learned_addr learned my addr 172.21.15.81:0/207984445 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:08.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.985+0000 7f3e93fff640 1 --2- 172.21.15.81:0/207984445 >> v2:172.21.15.93:3300/0 conn(0x7f3e9c0716e0 0x7f3e9c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:08.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.985+0000 7f3e9b7fe640 1 -- 172.21.15.81:0/207984445 >> v2:172.21.15.93:3300/0 conn(0x7f3e9c0716e0 msgr2=0x7f3e9c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:08.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.985+0000 7f3e9b7fe640 1 --2- 172.21.15.81:0/207984445 >> v2:172.21.15.93:3300/0 conn(0x7f3e9c0716e0 0x7f3e9c0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.985+0000 7f3e9b7fe640 1 -- 172.21.15.81:0/207984445 >> v1:172.21.15.93:6789/0 conn(0x7f3e9c0cf250 legacy=0x7f3e9c06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.985+0000 7f3e9b7fe640 1 -- 172.21.15.81:0/207984445 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c06d580 con 0x7f3e9c15aac0 2023-11-08T16:29:08.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.986+0000 7f3e9b7fe640 1 --2- 172.21.15.81:0/207984445 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c15aac0 0x7f3e9c0711a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009870 tx=0x7f3e8c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f330de49cde3eafb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:08.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.987+0000 7f3e9affd640 1 -- 172.21.15.81:0/207984445 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c030840 con 0x7f3e9c15aac0 2023-11-08T16:29:08.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.987+0000 7f3e9affd640 1 -- 172.21.15.81:0/207984445 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c0309a0 con 0x7f3e9c15aac0 2023-11-08T16:29:08.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.987+0000 7f3e9affd640 1 -- 172.21.15.81:0/207984445 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c030d30 con 0x7f3e9c15aac0 2023-11-08T16:29:08.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.988+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/207984445 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c15aac0 msgr2=0x7f3e9c0711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.988+0000 7f3ea0c0d640 1 --2- 172.21.15.81:0/207984445 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c15aac0 0x7f3e9c0711a0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c009870 tx=0x7f3e8c02b980 comp rx=0 tx=0).stop 2023-11-08T16:29:08.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.988+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/207984445 shutdown_connections 2023-11-08T16:29:08.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.988+0000 7f3ea0c0d640 1 --2- 172.21.15.81:0/207984445 >> v2:172.21.15.93:3300/0 conn(0x7f3e9c0716e0 0x7f3e9c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.988+0000 7f3ea0c0d640 1 --2- 172.21.15.81:0/207984445 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c15aac0 0x7f3e9c0711a0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.988+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/207984445 >> 172.21.15.81:0/207984445 conn(0x7f3e9c06a4a0 msgr2=0x7f3e9c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:08.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.988+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/207984445 shutdown_connections 2023-11-08T16:29:08.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.988+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/207984445 wait complete. 2023-11-08T16:29:08.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.990+0000 7f3ea0c0d640 1 Processor -- start 2023-11-08T16:29:08.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.990+0000 7f3ea0c0d640 1 -- start start 2023-11-08T16:29:08.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.991+0000 7f3ea0c0d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9c0716e0 0x7f3e9c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.991+0000 7f3ea0c0d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c0cf250 0x7f3e9c1f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.991+0000 7f3ea0c0d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c15aac0 0x7f3e9c151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:08.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.991+0000 7f3ea0c0d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c15c790 con 0x7f3e9c15aac0 2023-11-08T16:29:08.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.991+0000 7f3ea0c0d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c15c9c0 con 0x7f3e9c0716e0 2023-11-08T16:29:08.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.991+0000 7f3ea0c0d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c15cca0 con 0x7f3e9c0cf250 2023-11-08T16:29:08.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.991+0000 7f3e9bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c15aac0 0x7f3e9c151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:08.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.991+0000 7f3e9bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c15aac0 0x7f3e9c151cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41758/0 (socket says 172.21.15.81:41758) 2023-11-08T16:29:08.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.991+0000 7f3e9bfff640 1 -- 172.21.15.81:0/90512650 learned_addr learned my addr 172.21.15.81:0/90512650 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:08.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.991+0000 7f3e9bfff640 1 -- 172.21.15.81:0/90512650 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c0cf250 msgr2=0x7f3e9c1f3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.991+0000 7f3e93fff640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c0cf250 0x7f3e9c1f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:08.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.992+0000 7f3e9bfff640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c0cf250 0x7f3e9c1f3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.992+0000 7f3e9bfff640 1 -- 172.21.15.81:0/90512650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9c0716e0 msgr2=0x7f3e9c154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:08.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.992+0000 7f3e9bfff640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9c0716e0 0x7f3e9c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:08.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.992+0000 7f3e9bfff640 1 -- 172.21.15.81:0/90512650 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c0094d0 con 0x7f3e9c15aac0 2023-11-08T16:29:08.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.992+0000 7f3e9bfff640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c15aac0 0x7f3e9c151cf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c710 tx=0x7f3e9400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:08.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.992+0000 7f3e98ff9640 1 -- 172.21.15.81:0/90512650 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94006d50 con 0x7f3e9c15aac0 2023-11-08T16:29:08.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.992+0000 7f3e98ff9640 1 -- 172.21.15.81:0/90512650 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9400ce20 con 0x7f3e9c15aac0 2023-11-08T16:29:08.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.992+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/90512650 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c15cf80 con 0x7f3e9c15aac0 2023-11-08T16:29:08.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.992+0000 7f3e98ff9640 1 -- 172.21.15.81:0/90512650 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94003bb0 con 0x7f3e9c15aac0 2023-11-08T16:29:08.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.992+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/90512650 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c15d540 con 0x7f3e9c15aac0 2023-11-08T16:29:09.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.993+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/90512650 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c06b6f0 con 0x7f3e9c15aac0 2023-11-08T16:29:09.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.994+0000 7f3e98ff9640 1 -- 172.21.15.81:0/90512650 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e94003d10 con 0x7f3e9c15aac0 2023-11-08T16:29:09.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.995+0000 7f3e98ff9640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e7c06ae70 0x7f3e7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.995+0000 7f3e98ff9640 1 -- 172.21.15.81:0/90512650 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e94089150 con 0x7f3e9c15aac0 2023-11-08T16:29:09.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.995+0000 7f3e9b7fe640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e7c06ae70 0x7f3e7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.996+0000 7f3e9b7fe640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e7c06ae70 0x7f3e7c06d330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c036000 tx=0x7f3e8c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:09.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:08.999+0000 7f3e98ff9640 1 -- 172.21.15.81:0/90512650 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e94057d20 con 0x7f3e9c15aac0 2023-11-08T16:29:09.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.236+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/90512650 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3e9c067800 con 0x7f3e9c15aac0 2023-11-08T16:29:09.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.240+0000 7f3e98ff9640 1 -- 172.21.15.81:0/90512650 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171032 (secure 0 0 0) 0x7f3e940578b0 con 0x7f3e9c15aac0 2023-11-08T16:29:09.243 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:09.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.244+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/90512650 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e7c06ae70 msgr2=0x7f3e7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:09.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.244+0000 7f3ea0c0d640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e7c06ae70 0x7f3e7c06d330 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c036000 tx=0x7f3e8c030000 comp rx=0 tx=0).stop 2023-11-08T16:29:09.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.244+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/90512650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c15aac0 msgr2=0x7f3e9c151cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:09.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.244+0000 7f3ea0c0d640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c15aac0 0x7f3e9c151cf0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3e9400c710 tx=0x7f3e9400cbd0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.245+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/90512650 shutdown_connections 2023-11-08T16:29:09.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.245+0000 7f3ea0c0d640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e7c06ae70 0x7f3e7c06d330 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.245+0000 7f3ea0c0d640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e9c15aac0 0x7f3e9c151cf0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.245+0000 7f3ea0c0d640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e9c0cf250 0x7f3e9c1f3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.245+0000 7f3ea0c0d640 1 --2- 172.21.15.81:0/90512650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e9c0716e0 0x7f3e9c154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.245+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/90512650 >> 172.21.15.81:0/90512650 conn(0x7f3e9c06a4a0 msgr2=0x7f3e9c159890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:09.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.245+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/90512650 shutdown_connections 2023-11-08T16:29:09.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.245+0000 7f3ea0c0d640 1 -- 172.21.15.81:0/90512650 wait complete. 2023-11-08T16:29:09.266 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":5,"active_gid":4101,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6832","nonce":3984324735},{"type":"v1","addr":"172.21.15.81:6833","nonce":3984324735}]},"active_addr":"172.21.15.81:6833/3984324735","active_change":"2023-11-08T16:28:42.525518+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4104,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:v0.0.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":4129894172}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":3775483289}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":1061001784}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.81:0","nonce":1888441077}]}]} 2023-11-08T16:29:09.270 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-08T16:29:09.270 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-08T16:29:09.271 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:09.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.478+0000 7fad24c2e640 1 Processor -- start 2023-11-08T16:29:09.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.479+0000 7fad24c2e640 1 -- start start 2023-11-08T16:29:09.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.480+0000 7fad24c2e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fad2015aac0 0x7fad200711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.480+0000 7fad24c2e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fad200716e0 0x7fad200ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.480+0000 7fad24c2e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad200cf250 0x7fad2006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.480+0000 7fad24c2e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2006f650 con 0x7fad200716e0 2023-11-08T16:29:09.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.480+0000 7fad24c2e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad200cf6c0 con 0x7fad2015aac0 2023-11-08T16:29:09.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.480+0000 7fad24c2e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad20070470 con 0x7fad200cf250 2023-11-08T16:29:09.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.481+0000 7fad17fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fad200716e0 0x7fad200ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.481+0000 7fad17fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fad200716e0 0x7fad200ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41762/0 (socket says 172.21.15.81:41762) 2023-11-08T16:29:09.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.481+0000 7fad17fff640 1 -- 172.21.15.81:0/605568680 learned_addr learned my addr 172.21.15.81:0/605568680 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:09.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.481+0000 7fad1ffff640 1 --2- 172.21.15.81:0/605568680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad200cf250 0x7fad2006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.481+0000 7fad1f7fe640 1 --2- 172.21.15.81:0/605568680 >> v2:172.21.15.93:3300/0 conn(0x7fad2015aac0 0x7fad200711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.481+0000 7fad17fff640 1 -- 172.21.15.81:0/605568680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad200cf250 msgr2=0x7fad2006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:09.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.481+0000 7fad17fff640 1 --2- 172.21.15.81:0/605568680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad200cf250 0x7fad2006b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.481+0000 7fad17fff640 1 -- 172.21.15.81:0/605568680 >> v2:172.21.15.93:3300/0 conn(0x7fad2015aac0 msgr2=0x7fad200711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:09.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.481+0000 7fad17fff640 1 --2- 172.21.15.81:0/605568680 >> v2:172.21.15.93:3300/0 conn(0x7fad2015aac0 0x7fad200711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.481+0000 7fad17fff640 1 -- 172.21.15.81:0/605568680 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2006d580 con 0x7fad200716e0 2023-11-08T16:29:09.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.483+0000 7fad17fff640 1 --2- 172.21.15.81:0/605568680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad200716e0 0x7fad200ced10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad2006db60 tx=0x7fad0802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611fc041529081a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:09.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.483+0000 7fad1effd640 1 -- 172.21.15.81:0/605568680 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08031870 con 0x7fad200716e0 2023-11-08T16:29:09.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.483+0000 7fad1effd640 1 -- 172.21.15.81:0/605568680 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad080319d0 con 0x7fad200716e0 2023-11-08T16:29:09.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.483+0000 7fad1effd640 1 -- 172.21.15.81:0/605568680 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08031d60 con 0x7fad200716e0 2023-11-08T16:29:09.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.484+0000 7fad24c2e640 1 -- 172.21.15.81:0/605568680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad200716e0 msgr2=0x7fad200ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:09.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.484+0000 7fad24c2e640 1 --2- 172.21.15.81:0/605568680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad200716e0 0x7fad200ced10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad2006db60 tx=0x7fad0802b910 comp rx=0 tx=0).stop 2023-11-08T16:29:09.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.484+0000 7fad24c2e640 1 -- 172.21.15.81:0/605568680 shutdown_connections 2023-11-08T16:29:09.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.484+0000 7fad24c2e640 1 --2- 172.21.15.81:0/605568680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad200cf250 0x7fad2006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.484+0000 7fad24c2e640 1 --2- 172.21.15.81:0/605568680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad200716e0 0x7fad200ced10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.484+0000 7fad24c2e640 1 --2- 172.21.15.81:0/605568680 >> v2:172.21.15.93:3300/0 conn(0x7fad2015aac0 0x7fad200711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.484+0000 7fad24c2e640 1 -- 172.21.15.81:0/605568680 >> 172.21.15.81:0/605568680 conn(0x7fad2006a4a0 msgr2=0x7fad2006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:09.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.484+0000 7fad24c2e640 1 -- 172.21.15.81:0/605568680 shutdown_connections 2023-11-08T16:29:09.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.484+0000 7fad24c2e640 1 -- 172.21.15.81:0/605568680 wait complete. 2023-11-08T16:29:09.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.486+0000 7fad24c2e640 1 Processor -- start 2023-11-08T16:29:09.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.486+0000 7fad24c2e640 1 -- start start 2023-11-08T16:29:09.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad24c2e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad200716e0 0x7fad2015dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad24c2e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad200cf250 0x7fad200a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad24c2e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad2015aac0 0x7fad200aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad24c2e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200a93e0 con 0x7fad200716e0 2023-11-08T16:29:09.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad24c2e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad200ac640 con 0x7fad2015aac0 2023-11-08T16:29:09.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad24c2e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad200ac090 con 0x7fad200cf250 2023-11-08T16:29:09.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad1f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad200716e0 0x7fad2015dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad1f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad200716e0 0x7fad2015dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41768/0 (socket says 172.21.15.81:41768) 2023-11-08T16:29:09.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad1f7fe640 1 -- 172.21.15.81:0/1958566932 learned_addr learned my addr 172.21.15.81:0/1958566932 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:09.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad1f7fe640 1 -- 172.21.15.81:0/1958566932 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad200cf250 msgr2=0x7fad200a8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:09.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad1f7fe640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad200cf250 0x7fad200a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad1f7fe640 1 -- 172.21.15.81:0/1958566932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad2015aac0 msgr2=0x7fad200aa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:09.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad1f7fe640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad2015aac0 0x7fad200aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.487+0000 7fad1f7fe640 1 -- 172.21.15.81:0/1958566932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad080094d0 con 0x7fad200716e0 2023-11-08T16:29:09.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.488+0000 7fad1f7fe640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad200716e0 0x7fad2015dcc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad1000d5f0 tx=0x7fad1000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:09.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.488+0000 7fad1cff9640 1 -- 172.21.15.81:0/1958566932 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad100125f0 con 0x7fad200716e0 2023-11-08T16:29:09.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.488+0000 7fad1cff9640 1 -- 172.21.15.81:0/1958566932 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad10012750 con 0x7fad200716e0 2023-11-08T16:29:09.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.488+0000 7fad1cff9640 1 -- 172.21.15.81:0/1958566932 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad10012b00 con 0x7fad200716e0 2023-11-08T16:29:09.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.488+0000 7fad24c2e640 1 -- 172.21.15.81:0/1958566932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad200ac2f0 con 0x7fad200716e0 2023-11-08T16:29:09.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.488+0000 7fad24c2e640 1 -- 172.21.15.81:0/1958566932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad200a98d0 con 0x7fad200716e0 2023-11-08T16:29:09.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.489+0000 7fad24c2e640 1 -- 172.21.15.81:0/1958566932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad20067800 con 0x7fad200716e0 2023-11-08T16:29:09.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.490+0000 7fad1cff9640 1 -- 172.21.15.81:0/1958566932 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad10012c60 con 0x7fad200716e0 2023-11-08T16:29:09.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.491+0000 7fad1cff9640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad0006ae70 0x7fad0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.491+0000 7fad1cff9640 1 -- 172.21.15.81:0/1958566932 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad10089a90 con 0x7fad200716e0 2023-11-08T16:29:09.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.491+0000 7fad17fff640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad0006ae70 0x7fad0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.492+0000 7fad17fff640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad0006ae70 0x7fad0006d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad08036000 tx=0x7fad08030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:09.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.495+0000 7fad1cff9640 1 -- 172.21.15.81:0/1958566932 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad10054b40 con 0x7fad200716e0 2023-11-08T16:29:09.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.686+0000 7fad24c2e640 1 -- 172.21.15.81:0/1958566932 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad200cb880 con 0x7fad200716e0 2023-11-08T16:29:09.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.689+0000 7fad1cff9640 1 -- 172.21.15.81:0/1958566932 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fad10058160 con 0x7fad200716e0 2023-11-08T16:29:09.691 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:09.692 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:04.727022+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:09.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 -- 172.21.15.81:0/1958566932 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad0006ae70 msgr2=0x7fad0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:09.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad0006ae70 0x7fad0006d330 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad08036000 tx=0x7fad08030000 comp rx=0 tx=0).stop 2023-11-08T16:29:09.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 -- 172.21.15.81:0/1958566932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad200716e0 msgr2=0x7fad2015dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:09.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad200716e0 0x7fad2015dcc0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fad1000d5f0 tx=0x7fad1000dab0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 -- 172.21.15.81:0/1958566932 shutdown_connections 2023-11-08T16:29:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad0006ae70 0x7fad0006d330 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad2015aac0 0x7fad200aa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad200cf250 0x7fad200a8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 --2- 172.21.15.81:0/1958566932 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad200716e0 0x7fad2015dcc0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 -- 172.21.15.81:0/1958566932 >> 172.21.15.81:0/1958566932 conn(0x7fad2006a4a0 msgr2=0x7fad2006c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:09.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 -- 172.21.15.81:0/1958566932 shutdown_connections 2023-11-08T16:29:09.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.692+0000 7fad24c2e640 1 -- 172.21.15.81:0/1958566932 wait complete. 2023-11-08T16:29:09.713 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-08T16:29:09.714 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:09.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.921+0000 7f960e9fb640 1 Processor -- start 2023-11-08T16:29:09.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.921+0000 7f960e9fb640 1 -- start start 2023-11-08T16:29:09.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.922+0000 7f960e9fb640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f960815d490 0x7f96080d39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.923+0000 7f960e9fb640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96080d1080 0x7f96080d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.923+0000 7f960e9fb640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96080d1eb0 con 0x7f96080d1a20 2023-11-08T16:29:09.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.923+0000 7f960e9fb640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96080ce890 con 0x7f960815d490 2023-11-08T16:29:09.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.923+0000 7f960e9fb640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96080ceb70 con 0x7f96080d1080 2023-11-08T16:29:09.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.923+0000 7f960e1fa640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f96080d1a20 0x7f96080cf820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58820/0 (socket says 172.21.15.81:58820) 2023-11-08T16:29:09.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.923+0000 7f960e1fa640 1 -- 172.21.15.81:0/867018677 learned_addr learned my addr 172.21.15.81:0/867018677 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:09.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.923+0000 7f960d1f8640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96080d1080 0x7f96080d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.923+0000 7f960d9f9640 1 --2- 172.21.15.81:0/867018677 >> v2:172.21.15.93:3300/0 conn(0x7f960815d490 0x7f96080d39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.923+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366838138 0 0) 0x7f96080d1eb0 con 0x7f96080d1a20 2023-11-08T16:29:09.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.923+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95f4003520 con 0x7f96080d1a20 2023-11-08T16:29:09.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 922300963 0 0) 0x7f95f4003520 con 0x7f96080d1a20 2023-11-08T16:29:09.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96080d1080 msgr2=0x7f96080d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:09.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96080d1080 0x7f96080d14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 >> v2:172.21.15.93:3300/0 conn(0x7f960815d490 msgr2=0x7f96080d39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:09.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 --2- 172.21.15.81:0/867018677 >> v2:172.21.15.93:3300/0 conn(0x7f960815d490 0x7f96080d39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96080cee50 con 0x7f96080d1a20 2023-11-08T16:29:09.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f96000019f0 con 0x7f96080d1a20 2023-11-08T16:29:09.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 >> v1:172.21.15.81:6789/0 conn(0x7f96080d1a20 legacy=0x7f96080cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:09.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 0x7f95f4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95f40047c0 0x7f95f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96080d1080 0x7f95f4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96080ce890 con 0x7f95f4003a50 2023-11-08T16:29:09.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96080ceb70 con 0x7f95f40047c0 2023-11-08T16:29:09.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f400ba60 con 0x7f96080d1080 2023-11-08T16:29:09.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960d9f9640 1 --2- 172.21.15.81:0/867018677 >> v2:172.21.15.93:3300/0 conn(0x7f960815d490 0x7f96080d39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:09.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960d9f9640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 0x7f95f4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960e1fa640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96080d1080 0x7f95f4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.924+0000 7f960d1f8640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95f40047c0 0x7f95f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.925+0000 7f960d9f9640 1 -- 172.21.15.81:0/867018677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96080d1080 msgr2=0x7f95f4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:09.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.925+0000 7f960d9f9640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96080d1080 0x7f95f4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.925+0000 7f960d9f9640 1 -- 172.21.15.81:0/867018677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95f40047c0 msgr2=0x7f95f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:09.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.925+0000 7f960d9f9640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95f40047c0 0x7f95f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.925+0000 7f960d9f9640 1 -- 172.21.15.81:0/867018677 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f4003520 con 0x7f95f4003a50 2023-11-08T16:29:09.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.926+0000 7f960d9f9640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 0x7f95f4003e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9604006fd0 tx=0x7f95f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83df52b265e15043 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:09.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.926+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604030640 con 0x7f95f4003a50 2023-11-08T16:29:09.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.926+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96040307a0 con 0x7f95f4003a50 2023-11-08T16:29:09.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.926+0000 7f960c9f7640 1 -- 172.21.15.81:0/867018677 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604030b10 con 0x7f95f4003a50 2023-11-08T16:29:09.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.926+0000 7f960e9fb640 1 -- 172.21.15.81:0/867018677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 msgr2=0x7f95f4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:09.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.926+0000 7f960e9fb640 1 --2- 172.21.15.81:0/867018677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 0x7f95f4003e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f9604006fd0 tx=0x7f95f40057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.927+0000 7f960e1fa640 1 -- 172.21.15.81:0/867018677 reap_dead start 2023-11-08T16:29:09.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.927+0000 7f960e9fb640 1 -- 172.21.15.81:0/867018677 shutdown_connections 2023-11-08T16:29:09.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.927+0000 7f960e9fb640 1 -- 172.21.15.81:0/867018677 >> 172.21.15.81:0/867018677 conn(0x7f96080d2e50 msgr2=0x7f96080d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:09.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.927+0000 7f960e9fb640 1 -- 172.21.15.81:0/867018677 shutdown_connections 2023-11-08T16:29:09.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.927+0000 7f960e9fb640 1 -- 172.21.15.81:0/867018677 wait complete. 2023-11-08T16:29:09.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.929+0000 7f960e9fb640 1 Processor -- start 2023-11-08T16:29:09.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.929+0000 7f960e9fb640 1 -- start start 2023-11-08T16:29:09.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960e9fb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 0x7f960814a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960e9fb640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96081f37c0 0x7f96081f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960e9fb640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f960815db80 0x7f960815e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960e9fb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96081f6f60 con 0x7f95f4003a50 2023-11-08T16:29:09.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960e9fb640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96081f69a0 con 0x7f960815db80 2023-11-08T16:29:09.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960e9fb640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96081f6c80 con 0x7f96081f37c0 2023-11-08T16:29:09.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960d9f9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 0x7f960814a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960d9f9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 0x7f960814a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41788/0 (socket says 172.21.15.81:41788) 2023-11-08T16:29:09.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960d9f9640 1 -- 172.21.15.81:0/2636343752 learned_addr learned my addr 172.21.15.81:0/2636343752 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:09.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960e1fa640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f960815db80 0x7f960815e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960d9f9640 1 -- 172.21.15.81:0/2636343752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96081f37c0 msgr2=0x7f96081f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:09.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960d9f9640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96081f37c0 0x7f96081f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960d9f9640 1 -- 172.21.15.81:0/2636343752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f960815db80 msgr2=0x7f960815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:09.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960d9f9640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f960815db80 0x7f960815e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:09.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.930+0000 7f960d9f9640 1 -- 172.21.15.81:0/2636343752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96080cbac0 con 0x7f95f4003a50 2023-11-08T16:29:09.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.931+0000 7f960d9f9640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 0x7f960814a3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f960402bed0 tx=0x7f9604004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:09.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.931+0000 7f95fe7fc640 1 -- 172.21.15.81:0/2636343752 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96040399e0 con 0x7f95f4003a50 2023-11-08T16:29:09.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.931+0000 7f95fe7fc640 1 -- 172.21.15.81:0/2636343752 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f960403f040 con 0x7f95f4003a50 2023-11-08T16:29:09.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.931+0000 7f960e9fb640 1 -- 172.21.15.81:0/2636343752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96080cbda0 con 0x7f95f4003a50 2023-11-08T16:29:09.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.931+0000 7f95fe7fc640 1 -- 172.21.15.81:0/2636343752 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9604039d60 con 0x7f95f4003a50 2023-11-08T16:29:09.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.931+0000 7f960e9fb640 1 -- 172.21.15.81:0/2636343752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96081f6750 con 0x7f95f4003a50 2023-11-08T16:29:09.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.932+0000 7f960e9fb640 1 -- 172.21.15.81:0/2636343752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96081f6e90 con 0x7f95f4003a50 2023-11-08T16:29:09.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.933+0000 7f95fe7fc640 1 -- 172.21.15.81:0/2636343752 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f960402e070 con 0x7f95f4003a50 2023-11-08T16:29:09.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.934+0000 7f95fe7fc640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95ec06aea0 0x7f95ec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:09.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.934+0000 7f95fe7fc640 1 -- 172.21.15.81:0/2636343752 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96040ab8d0 con 0x7f95f4003a50 2023-11-08T16:29:09.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.934+0000 7f960d1f8640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95ec06aea0 0x7f95ec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:09.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.935+0000 7f960d1f8640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95ec06aea0 0x7f95ec06d360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009910 tx=0x7f95f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:09.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:09.938+0000 7f95fe7fc640 1 -- 172.21.15.81:0/2636343752 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9604076970 con 0x7f95f4003a50 2023-11-08T16:29:10.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.127+0000 7f960e9fb640 1 -- 172.21.15.81:0/2636343752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9608067800 con 0x7f95f4003a50 2023-11-08T16:29:10.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.130+0000 7f95fe7fc640 1 -- 172.21.15.81:0/2636343752 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f9604034070 con 0x7f95f4003a50 2023-11-08T16:29:10.132 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:10.132 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:04.727022+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:10.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.132+0000 7f960e9fb640 1 -- 172.21.15.81:0/2636343752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95ec06aea0 msgr2=0x7f95ec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.132+0000 7f960e9fb640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95ec06aea0 0x7f95ec06d360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009910 tx=0x7f95f8008040 comp rx=0 tx=0).stop 2023-11-08T16:29:10.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.132+0000 7f960e9fb640 1 -- 172.21.15.81:0/2636343752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 msgr2=0x7f960814a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.132+0000 7f960e9fb640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 0x7f960814a3f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f960402bed0 tx=0x7f9604004730 comp rx=0 tx=0).stop 2023-11-08T16:29:10.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.132+0000 7f960e9fb640 1 -- 172.21.15.81:0/2636343752 shutdown_connections 2023-11-08T16:29:10.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.132+0000 7f960e9fb640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95ec06aea0 0x7f95ec06d360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.132+0000 7f960e9fb640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f960815db80 0x7f960815e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.132+0000 7f960e9fb640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96081f37c0 0x7f96081f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.132+0000 7f960e9fb640 1 --2- 172.21.15.81:0/2636343752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95f4003a50 0x7f960814a3f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.132+0000 7f960e9fb640 1 -- 172.21.15.81:0/2636343752 >> 172.21.15.81:0/2636343752 conn(0x7f96080d2e50 msgr2=0x7f960815b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.132+0000 7f960e9fb640 1 -- 172.21.15.81:0/2636343752 shutdown_connections 2023-11-08T16:29:10.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.133+0000 7f960e9fb640 1 -- 172.21.15.81:0/2636343752 wait complete. 2023-11-08T16:29:10.152 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-08T16:29:10.153 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-08T16:29:10.153 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-08T16:29:10.154 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-08T16:29:10.155 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-08T16:29:10.155 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-08T16:29:10.156 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-08T16:29:10.157 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-08T16:29:10.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.453+0000 7f85cfb00640 1 Processor -- start 2023-11-08T16:29:10.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.453+0000 7f85cfb00640 1 -- start start 2023-11-08T16:29:10.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.455+0000 7f85cfb00640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f85c80cfd20 0x7f85c80d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.455+0000 7f85cfb00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f85c80b2210 0x7f85c80d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.455+0000 7f85cfb00640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85c80d3bb0 con 0x7f85c815cff0 2023-11-08T16:29:10.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.455+0000 7f85cfb00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80ce6d0 con 0x7f85c80b2210 2023-11-08T16:29:10.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.455+0000 7f85cfb00640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85c80ce9b0 con 0x7f85c80cfd20 2023-11-08T16:29:10.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.456+0000 7f85cd09b640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f85c80cfd20 0x7f85c80d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.456+0000 7f85cd09b640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f85c80cfd20 0x7f85c80d0ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:46516/0 (socket says 172.21.15.81:46516) 2023-11-08T16:29:10.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.456+0000 7f85cd09b640 1 -- 172.21.15.81:0/3311128168 learned_addr learned my addr 172.21.15.81:0/3311128168 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.456+0000 7fbe2bfd0640 1 Processor -- start 2023-11-08T16:29:10.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.456+0000 7f85ce09d640 1 --2- 172.21.15.81:0/3311128168 >> v2:172.21.15.81:3300/0 conn(0x7f85c80b2210 0x7f85c80d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.457+0000 7fbe2bfd0640 1 -- start start 2023-11-08T16:29:10.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.458+0000 7f85cd09b640 1 -- 172.21.15.81:0/3311128168 >> v2:172.21.15.81:3300/0 conn(0x7f85c80b2210 msgr2=0x7f85c80d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:10.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.458+0000 7f85cc89a640 1 -- 172.21.15.81:0/3311128168 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284449587 0 0) 0x7f85c80d3bb0 con 0x7f85c815cff0 2023-11-08T16:29:10.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.458+0000 7f85cd09b640 1 --2- 172.21.15.81:0/3311128168 >> v2:172.21.15.81:3300/0 conn(0x7f85c80b2210 0x7f85c80d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.458+0000 7f85cd09b640 1 -- 172.21.15.81:0/3311128168 >> v1:172.21.15.93:6789/0 conn(0x7f85c815cff0 legacy=0x7f85c80cf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.458+0000 7f85cd09b640 1 -- 172.21.15.81:0/3311128168 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c80cec90 con 0x7f85c80cfd20 2023-11-08T16:29:10.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.458+0000 7f85ce09d640 1 --2- 172.21.15.81:0/3311128168 >> v2:172.21.15.81:3300/0 conn(0x7f85c80b2210 0x7f85c80d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:10.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.458+0000 7fbe2bfd0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbe1c0e75c0 0x7fbe1c057490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.459+0000 7fbe2bfd0640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbe1c0589d0 0x7fbe1c04ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.459+0000 7fbe2bfd0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1c053500 0x7fbe1c053990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.459+0000 7fbe2bfd0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1c057e00 con 0x7fbe1c0e75c0 2023-11-08T16:29:10.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.459+0000 7fbe2bfd0640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe1c053ed0 con 0x7fbe1c0589d0 2023-11-08T16:29:10.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.459+0000 7fbe2bfd0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c0e79a0 con 0x7fbe1c053500 2023-11-08T16:29:10.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.459+0000 7fbe29d6c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbe1c0e75c0 0x7fbe1c057490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.459+0000 7fbe29d6c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbe1c0e75c0 0x7fbe1c057490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41812/0 (socket says 172.21.15.81:41812) 2023-11-08T16:29:10.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.459+0000 7fbe29d6c640 1 -- 172.21.15.81:0/308222611 learned_addr learned my addr 172.21.15.81:0/308222611 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.460+0000 7fbe2a56d640 1 --2- 172.21.15.81:0/308222611 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1c053500 0x7fbe1c053990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.460+0000 7fbe2956b640 1 --2- 172.21.15.81:0/308222611 >> v2:172.21.15.93:3300/0 conn(0x7fbe1c0589d0 0x7fbe1c04ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.460+0000 7fa06ffff640 1 Processor -- start 2023-11-08T16:29:10.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.460+0000 7fbe2a56d640 1 -- 172.21.15.81:0/308222611 >> v2:172.21.15.93:3300/0 conn(0x7fbe1c0589d0 msgr2=0x7fbe1c04ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:10.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.460+0000 7fbe2a56d640 1 --2- 172.21.15.81:0/308222611 >> v2:172.21.15.93:3300/0 conn(0x7fbe1c0589d0 0x7fbe1c04ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.460+0000 7fbe2a56d640 1 -- 172.21.15.81:0/308222611 >> v2:172.21.15.81:3300/0 conn(0x7fbe1c0e75c0 msgr2=0x7fbe1c057490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.460+0000 7fbe2a56d640 1 --2- 172.21.15.81:0/308222611 >> v2:172.21.15.81:3300/0 conn(0x7fbe1c0e75c0 0x7fbe1c057490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.460+0000 7fbe2a56d640 1 -- 172.21.15.81:0/308222611 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe1c0061e0 con 0x7fbe1c053500 2023-11-08T16:29:10.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.461+0000 7fa06ffff640 1 -- start start 2023-11-08T16:29:10.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.462+0000 7f85cd09b640 1 --2- 172.21.15.81:0/3311128168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85c80cfd20 0x7f85c80d0ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85c80b5840 tx=0x7f85b802b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3271949733deefb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.462+0000 7fbe2a56d640 1 --2- 172.21.15.81:0/308222611 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1c053500 0x7fbe1c053990 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000b60 tx=0x7fbe240a5f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a11dd4aed68178b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.462+0000 7f85cc89a640 1 -- 172.21.15.81:0/3311128168 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8030870 con 0x7f85c80cfd20 2023-11-08T16:29:10.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.462+0000 7f85cc89a640 1 -- 172.21.15.81:0/3311128168 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85b80309d0 con 0x7f85c80cfd20 2023-11-08T16:29:10.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.462+0000 7f85cc89a640 1 -- 172.21.15.81:0/3311128168 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85b8030d60 con 0x7f85c80cfd20 2023-11-08T16:29:10.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.462+0000 7fbe28d6a640 1 -- 172.21.15.81:0/308222611 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2406d030 con 0x7fbe1c053500 2023-11-08T16:29:10.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.462+0000 7fbe28d6a640 1 -- 172.21.15.81:0/308222611 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe240a6b80 con 0x7fbe1c053500 2023-11-08T16:29:10.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.462+0000 7fbe28d6a640 1 -- 172.21.15.81:0/308222611 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe240ae440 con 0x7fbe1c053500 2023-11-08T16:29:10.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.462+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0680e3140 0x7fa0680e77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.462+0000 7fa06ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa06803e950 0x7fa068058590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.463+0000 7fa06ffff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0680e7ce0 con 0x7fa068058ad0 2023-11-08T16:29:10.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.463+0000 7fa06ffff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa068057510 con 0x7fa06803e950 2023-11-08T16:29:10.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.463+0000 7fa06ffff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0680577f0 con 0x7fa0680e3140 2023-11-08T16:29:10.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.463+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/308222611 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1c053500 msgr2=0x7fbe1c053990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.463+0000 7fbe2bfd0640 1 --2- 172.21.15.81:0/308222611 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1c053500 0x7fbe1c053990 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbe24000b60 tx=0x7fbe240a5f10 comp rx=0 tx=0).stop 2023-11-08T16:29:10.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/308222611 shutdown_connections 2023-11-08T16:29:10.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fbe2bfd0640 1 --2- 172.21.15.81:0/308222611 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1c053500 0x7fbe1c053990 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fa06e7fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa06803e950 0x7fa068058590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fa06f7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa068058ad0 0x7fa068053c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58836/0 (socket says 172.21.15.81:58836) 2023-11-08T16:29:10.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fa06f7fe640 1 -- 172.21.15.81:0/950421412 learned_addr learned my addr 172.21.15.81:0/950421412 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:10.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fa06effd640 1 --2- 172.21.15.81:0/950421412 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0680e3140 0x7fa0680e77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fbe2bfd0640 1 --2- 172.21.15.81:0/308222611 >> v2:172.21.15.93:3300/0 conn(0x7fbe1c0589d0 0x7fbe1c04ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fbe2bfd0640 1 --2- 172.21.15.81:0/308222611 >> v2:172.21.15.81:3300/0 conn(0x7fbe1c0e75c0 0x7fbe1c057490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/308222611 >> 172.21.15.81:0/308222611 conn(0x7fbe1c058eb0 msgr2=0x7fbe1c0592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/308222611 shutdown_connections 2023-11-08T16:29:10.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/308222611 wait complete. 2023-11-08T16:29:10.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.464+0000 7fa06dffb640 1 -- 172.21.15.81:0/950421412 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1854189754 0 0) 0x7fa0680e7ce0 con 0x7fa068058ad0 2023-11-08T16:29:10.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.465+0000 7fa06dffb640 1 -- 172.21.15.81:0/950421412 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa058003520 con 0x7fa068058ad0 2023-11-08T16:29:10.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.465+0000 7f85cfb00640 1 -- 172.21.15.81:0/3311128168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85c80cfd20 msgr2=0x7f85c80d0ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.465+0000 7f85cfb00640 1 --2- 172.21.15.81:0/3311128168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85c80cfd20 0x7f85c80d0ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85c80b5840 tx=0x7f85b802b9b0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.465+0000 7f85cfb00640 1 -- 172.21.15.81:0/3311128168 shutdown_connections 2023-11-08T16:29:10.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.465+0000 7f85cfb00640 1 --2- 172.21.15.81:0/3311128168 >> v2:172.21.15.81:3300/0 conn(0x7f85c80b2210 0x7f85c80d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.465+0000 7f85cfb00640 1 --2- 172.21.15.81:0/3311128168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85c80cfd20 0x7f85c80d0ac0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.465+0000 7f85cfb00640 1 -- 172.21.15.81:0/3311128168 >> 172.21.15.81:0/3311128168 conn(0x7f85c80d2c50 msgr2=0x7f85c80d3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.466+0000 7fbe2bfd0640 1 Processor -- start 2023-11-08T16:29:10.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.466+0000 7fbe2bfd0640 1 -- start start 2023-11-08T16:29:10.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.465+0000 7fa06dffb640 1 -- 172.21.15.81:0/950421412 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 983275407 0 0) 0x7fa058003520 con 0x7fa068058ad0 2023-11-08T16:29:10.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.466+0000 7f019b165640 1 Processor -- start 2023-11-08T16:29:10.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.466+0000 7f019b165640 1 -- start start 2023-11-08T16:29:10.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.467+0000 7f81dd20e640 1 Processor -- start 2023-11-08T16:29:10.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.466+0000 7fa06e7fc640 1 -- 172.21.15.81:0/950421412 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0680e3140 msgr2=0x7fa0680e77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:10.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.466+0000 7f85cfb00640 1 -- 172.21.15.81:0/3311128168 shutdown_connections 2023-11-08T16:29:10.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.467+0000 7f85cfb00640 1 -- 172.21.15.81:0/3311128168 wait complete. 2023-11-08T16:29:10.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.466+0000 7fa06e7fc640 1 --2- 172.21.15.81:0/950421412 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0680e3140 0x7fa0680e77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.466+0000 7fa06e7fc640 1 -- 172.21.15.81:0/950421412 >> v1:172.21.15.81:6789/0 conn(0x7fa068058ad0 legacy=0x7fa068053c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.466+0000 7fa06e7fc640 1 -- 172.21.15.81:0/950421412 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa068057ad0 con 0x7fa06803e950 2023-11-08T16:29:10.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.466+0000 7fa06effd640 1 --2- 172.21.15.81:0/950421412 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0680e3140 0x7fa0680e77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:10.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.467+0000 7f81dd20e640 1 -- start start 2023-11-08T16:29:10.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.468+0000 7fbe2bfd0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1c053500 0x7fbe1c0e61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.468+0000 7fa06e7fc640 1 --2- 172.21.15.81:0/950421412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa06803e950 0x7fa068058590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0640097a0 tx=0x7fa06402b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b5f483ecbb02c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.468+0000 7f019b165640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f01940cebd0 0x7f01940d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.468+0000 7f019b165640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f01940b1fd0 0x7f01940cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.468+0000 7f019b165640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01940b4350 con 0x7f019415d5a0 2023-11-08T16:29:10.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.468+0000 7f019b165640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01940b4630 con 0x7f01940b1fd0 2023-11-08T16:29:10.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.468+0000 7f019b165640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01940d03f0 con 0x7f01940cebd0 2023-11-08T16:29:10.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.468+0000 7fbe2bfd0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe1c0589d0 0x7fbe1c00b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.469+0000 7fbe2bfd0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1c0e75c0 0x7fbe1c00bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.469+0000 7fbe29d6c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1c053500 0x7fbe1c0e61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.469+0000 7f81dd20e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f81d80adb40 0x7f81d80c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.469+0000 7f85cfb00640 1 Processor -- start 2023-11-08T16:29:10.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.469+0000 7f81dd20e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f81d8165940 0x7f81d806b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.469+0000 7f0199702640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f01940b1fd0 0x7f01940cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.469+0000 7f0198f01640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f019415d5a0 0x7f01940cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50644/0 (socket says 172.21.15.81:50644) 2023-11-08T16:29:10.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.469+0000 7f0198f01640 1 -- 172.21.15.81:0/1899337249 learned_addr learned my addr 172.21.15.81:0/1899337249 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:10.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f81dd20e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d80ce3f0 con 0x7f81d80adf20 2023-11-08T16:29:10.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f81d6d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f81d80adb40 0x7f81d80c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7fbe29d6c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1c053500 0x7fbe1c0e61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41836/0 (socket says 172.21.15.81:41836) 2023-11-08T16:29:10.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f81dd20e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d80cb340 con 0x7f81d80adb40 2023-11-08T16:29:10.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f81d6d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f81d80adb40 0x7f81d80c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41862/0 (socket says 172.21.15.81:41862) 2023-11-08T16:29:10.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f81dd20e640 1 -- 172.21.15.81:0/1203471473 --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d80cac80 con 0x7f81d8165940 2023-11-08T16:29:10.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f81d6d9d640 1 -- 172.21.15.81:0/1203471473 learned_addr learned my addr 172.21.15.81:0/1203471473 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7fbe29d6c640 1 -- 172.21.15.81:0/3728882396 learned_addr learned my addr 172.21.15.81:0/3728882396 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7fbe2bfd0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c04ff80 con 0x7fbe1c053500 2023-11-08T16:29:10.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c00ab40 con 0x7fbe1c0589d0 2023-11-08T16:29:10.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1c00ae20 con 0x7fbe1c0e75c0 2023-11-08T16:29:10.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7fbe2a56d640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1c0e75c0 0x7fbe1c00bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7fa06dffb640 1 -- 172.21.15.81:0/950421412 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07004ece0 con 0x7fa06803e950 2023-11-08T16:29:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f85cfb00640 1 -- start start 2023-11-08T16:29:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f81d759e640 1 --2- 172.21.15.81:0/1203471473 >> v2:172.21.15.93:3300/0 conn(0x7f81d8165940 0x7f81d806b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7fbe2956b640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe1c0589d0 0x7fbe1c00b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7fa06dffb640 1 -- 172.21.15.81:0/950421412 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06402e050 con 0x7fa06803e950 2023-11-08T16:29:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7fa06dffb640 1 -- 172.21.15.81:0/950421412 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064030730 con 0x7fa06803e950 2023-11-08T16:29:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7fbe29d6c640 1 -- 172.21.15.81:0/3728882396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1c0e75c0 msgr2=0x7fbe1c00bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7fbe29d6c640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1c0e75c0 0x7fbe1c00bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7fbe29d6c640 1 -- 172.21.15.81:0/3728882396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe1c0589d0 msgr2=0x7fbe1c00b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7fbe29d6c640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe1c0589d0 0x7fbe1c00b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7fbe29d6c640 1 -- 172.21.15.81:0/3728882396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe2404ea00 con 0x7fbe1c053500 2023-11-08T16:29:10.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7fbe2956b640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe1c0589d0 0x7fbe1c00b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:10.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.472+0000 7fbe29d6c640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1c053500 0x7fbe1c0e61d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000e860 tx=0x7fbe2000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f018bfff640 1 --2- 172.21.15.81:0/1899337249 >> v2:172.21.15.93:3300/0 conn(0x7f01940cebd0 0x7f01940d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f0199702640 1 -- 172.21.15.81:0/1899337249 >> v2:172.21.15.93:3300/0 conn(0x7f01940cebd0 msgr2=0x7f01940d3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:10.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f0199702640 1 --2- 172.21.15.81:0/1899337249 >> v2:172.21.15.93:3300/0 conn(0x7f01940cebd0 0x7f01940d3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f0199702640 1 -- 172.21.15.81:0/1899337249 >> v1:172.21.15.93:6789/0 conn(0x7f019415d5a0 legacy=0x7f01940cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.470+0000 7f0199702640 1 -- 172.21.15.81:0/1899337249 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01940ce6b0 con 0x7f01940b1fd0 2023-11-08T16:29:10.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7f81d759e640 1 -- 172.21.15.81:0/1203471473 >> v2:172.21.15.81:3300/0 conn(0x7f81d80adb40 msgr2=0x7f81d80c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:10.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7f81d759e640 1 --2- 172.21.15.81:0/1203471473 >> v2:172.21.15.81:3300/0 conn(0x7f81d80adb40 0x7f81d80c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7f81d759e640 1 -- 172.21.15.81:0/1203471473 >> v1:172.21.15.93:6789/0 conn(0x7f81d80adf20 legacy=0x7f81d8165230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:10.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7f81d759e640 1 -- 172.21.15.81:0/1203471473 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d80caf60 con 0x7f81d8165940 2023-11-08T16:29:10.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7f85cfb00640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85c80b2210 0x7f85c80c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.471+0000 7f85cfb00640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f85c80cfd20 0x7f85c815fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.472+0000 7f85cfb00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c815cff0 0x7f85c815a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.472+0000 7f85cfb00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c815fff0 con 0x7f85c815cff0 2023-11-08T16:29:10.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.472+0000 7f85cfb00640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80c72e0 con 0x7f85c80b2210 2023-11-08T16:29:10.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.472+0000 7f85cfb00640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85c80c75c0 con 0x7f85c80cfd20 2023-11-08T16:29:10.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.472+0000 7f85ce09d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c815cff0 0x7f85c815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.472+0000 7f85ce09d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c815cff0 0x7f85c815a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41870/0 (socket says 172.21.15.81:41870) 2023-11-08T16:29:10.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.472+0000 7f85ce09d640 1 -- 172.21.15.81:0/1930655589 learned_addr learned my addr 172.21.15.81:0/1930655589 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.472+0000 7f85cd09b640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f85c80cfd20 0x7f85c815fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.472+0000 7f0199702640 1 --2- 172.21.15.81:0/1899337249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01940b1fd0 0x7f01940cfd10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f01940b5980 tx=0x7f018c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4345a36894c1014f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7fbe1a7fc640 1 -- 172.21.15.81:0/3728882396 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000c940 con 0x7fbe1c053500 2023-11-08T16:29:10.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7fbe1a7fc640 1 -- 172.21.15.81:0/3728882396 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2000caa0 con 0x7fbe1c053500 2023-11-08T16:29:10.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7fbe1a7fc640 1 -- 172.21.15.81:0/3728882396 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2000ce50 con 0x7fbe1c053500 2023-11-08T16:29:10.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7fa06ffff640 1 -- 172.21.15.81:0/950421412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa06803e950 msgr2=0x7fa068058590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7fa06ffff640 1 --2- 172.21.15.81:0/950421412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa06803e950 0x7fa068058590 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa0640097a0 tx=0x7fa06402b790 comp rx=0 tx=0).stop 2023-11-08T16:29:10.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7ff08b1c0640 1 Processor -- start 2023-11-08T16:29:10.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7f85cd89c640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85c80b2210 0x7f85c80c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7f0193fff640 1 -- 172.21.15.81:0/1899337249 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c02cc90 con 0x7f01940b1fd0 2023-11-08T16:29:10.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe1c00b100 con 0x7fbe1c053500 2023-11-08T16:29:10.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe1c04b8b0 con 0x7fbe1c053500 2023-11-08T16:29:10.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7f81d759e640 1 --2- 172.21.15.81:0/1203471473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81d8165940 0x7f81d806b910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81d806d910 tx=0x7f81c802b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99bfd09b193448b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.474+0000 7f0193fff640 1 -- 172.21.15.81:0/1899337249 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f018c02cdf0 con 0x7f01940b1fd0 2023-11-08T16:29:10.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.474+0000 7f0193fff640 1 -- 172.21.15.81:0/1899337249 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f018c034660 con 0x7f01940b1fd0 2023-11-08T16:29:10.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.474+0000 7fa06ffff640 1 -- 172.21.15.81:0/950421412 shutdown_connections 2023-11-08T16:29:10.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.474+0000 7fa06ffff640 1 --2- 172.21.15.81:0/950421412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa06803e950 0x7fa068058590 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.474+0000 7fa06ffff640 1 --2- 172.21.15.81:0/950421412 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0680e3140 0x7fa0680e77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.474+0000 7fa06ffff640 1 -- 172.21.15.81:0/950421412 >> 172.21.15.81:0/950421412 conn(0x7fa068055a40 msgr2=0x7fa068055e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.474+0000 7f81d5d9b640 1 -- 172.21.15.81:0/1203471473 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8031520 con 0x7f81d8165940 2023-11-08T16:29:10.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.474+0000 7f81d5d9b640 1 -- 172.21.15.81:0/1203471473 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c8031680 con 0x7f81d8165940 2023-11-08T16:29:10.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7f85ce09d640 1 -- 172.21.15.81:0/1930655589 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f85c80cfd20 msgr2=0x7f85c815fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7f85ce09d640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f85c80cfd20 0x7f85c815fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7f85ce09d640 1 -- 172.21.15.81:0/1930655589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85c80b2210 msgr2=0x7f85c80c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7f85ce09d640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f85c80b2210 0x7f85c80c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7f85ce09d640 1 -- 172.21.15.81:0/1930655589 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85c0009950 con 0x7f85c815cff0 2023-11-08T16:29:10.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.474+0000 7f81d5d9b640 1 -- 172.21.15.81:0/1203471473 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c8031a10 con 0x7f81d8165940 2023-11-08T16:29:10.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.474+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbe1c0067a0 con 0x7fbe1c053500 2023-11-08T16:29:10.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.474+0000 7f85ce09d640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c815cff0 0x7f85c815a140 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85c000c690 tx=0x7f85c000cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.473+0000 7ff08b1c0640 1 -- start start 2023-11-08T16:29:10.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.475+0000 7f81dd20e640 1 -- 172.21.15.81:0/1203471473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81d8165940 msgr2=0x7f81d806b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.475+0000 7f81dd20e640 1 --2- 172.21.15.81:0/1203471473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81d8165940 0x7f81d806b910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f81d806d910 tx=0x7f81c802b7c0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.475+0000 7fa06ffff640 1 -- 172.21.15.81:0/950421412 shutdown_connections 2023-11-08T16:29:10.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.475+0000 7f85be7fc640 1 -- 172.21.15.81:0/1930655589 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0009bb0 con 0x7f85c815cff0 2023-11-08T16:29:10.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.475+0000 7fa06ffff640 1 -- 172.21.15.81:0/950421412 wait complete. 2023-11-08T16:29:10.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.475+0000 7f85be7fc640 1 -- 172.21.15.81:0/1930655589 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c000ce90 con 0x7f85c815cff0 2023-11-08T16:29:10.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.475+0000 7f85be7fc640 1 -- 172.21.15.81:0/1930655589 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c000bbb0 con 0x7f85c815cff0 2023-11-08T16:29:10.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7f85cfb00640 1 -- 172.21.15.81:0/1930655589 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b80094d0 con 0x7f85c815cff0 2023-11-08T16:29:10.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7f85cfb00640 1 -- 172.21.15.81:0/1930655589 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85c80c7b80 con 0x7f85c815cff0 2023-11-08T16:29:10.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7f81dd20e640 1 -- 172.21.15.81:0/1203471473 shutdown_connections 2023-11-08T16:29:10.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7f81dd20e640 1 --2- 172.21.15.81:0/1203471473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81d8165940 0x7f81d806b910 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7f81dd20e640 1 --2- 172.21.15.81:0/1203471473 >> v2:172.21.15.81:3300/0 conn(0x7f81d80adb40 0x7f81d80c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7f81dd20e640 1 -- 172.21.15.81:0/1203471473 >> 172.21.15.81:0/1203471473 conn(0x7f81d80cb490 msgr2=0x7f81d80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7ff08b1c0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0840adb40 0x7ff0840c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7ff08b1c0640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff0840adf20 0x7ff084165230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7ff08b1c0640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff084165b90 con 0x7ff084165770 2023-11-08T16:29:10.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7ff08b1c0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0840cb340 con 0x7ff0840adb40 2023-11-08T16:29:10.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7ff08b1c0640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0840ce3f0 con 0x7ff0840adf20 2023-11-08T16:29:10.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7ff088f5c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0840adb40 0x7ff0840c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.477+0000 7ff088f5c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0840adb40 0x7ff0840c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41880/0 (socket says 172.21.15.81:41880) 2023-11-08T16:29:10.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.477+0000 7ff088f5c640 1 -- 172.21.15.81:0/3859857224 learned_addr learned my addr 172.21.15.81:0/3859857224 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.477+0000 7ff08975d640 1 --1- 172.21.15.81:0/3859857224 >> v1:172.21.15.93:6789/0 conn(0x7ff084165770 0x7ff0840cc5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50666/0 (socket says 172.21.15.81:50666) 2023-11-08T16:29:10.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.477+0000 7fa06ffff640 1 Processor -- start 2023-11-08T16:29:10.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.478+0000 7fa06ffff640 1 -- start start 2023-11-08T16:29:10.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.477+0000 7ff088f5c640 1 -- 172.21.15.81:0/3859857224 >> v2:172.21.15.93:3300/0 conn(0x7ff0840adf20 msgr2=0x7ff084165230 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T16:29:10.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7f019b165640 1 -- 172.21.15.81:0/1899337249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01940b1fd0 msgr2=0x7f01940cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.478+0000 7f019b165640 1 --2- 172.21.15.81:0/1899337249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01940b1fd0 0x7f01940cfd10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f01940b5980 tx=0x7f018c02bee0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.477+0000 7ff088f5c640 1 --2- 172.21.15.81:0/3859857224 >> v2:172.21.15.93:3300/0 conn(0x7ff0840adf20 0x7ff084165230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.477+0000 7fbe1a7fc640 1 -- 172.21.15.81:0/3728882396 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbe20013470 con 0x7fbe1c053500 2023-11-08T16:29:10.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fab6e8d7640 1 Processor -- start 2023-11-08T16:29:10.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.477+0000 7ff088f5c640 1 -- 172.21.15.81:0/3859857224 >> v1:172.21.15.93:6789/0 conn(0x7ff084165770 legacy=0x7ff0840cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7f85cfb00640 1 -- 172.21.15.81:0/1930655589 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f85b0000fc0 con 0x7f85c815cff0 2023-11-08T16:29:10.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.477+0000 7ff088f5c640 1 -- 172.21.15.81:0/3859857224 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0840cac80 con 0x7ff0840adb40 2023-11-08T16:29:10.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa06803e950 0x7fa0680dc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fbe1a7fc640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbe0806ae70 0x7fbe0806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fbe1a7fc640 1 -- 172.21.15.81:0/3728882396 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe20089a10 con 0x7fbe1c053500 2023-11-08T16:29:10.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fbe1a7fc640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fbe08073090 0x7fbe080754b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fbe1a7fc640 1 -- 172.21.15.81:0/3728882396 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbe08075b60 con 0x7fbe08073090 2023-11-08T16:29:10.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fbe1a7fc640 1 -- 172.21.15.81:0/3728882396 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe20089dc0 con 0x7fbe1c053500 2023-11-08T16:29:10.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa068058ad0 0x7fa0680de2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0680e3140 0x7fa0680de7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fa06ffff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06817d4f0 con 0x7fa0680e3140 2023-11-08T16:29:10.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fa06ffff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0680ded30 con 0x7fa06803e950 2023-11-08T16:29:10.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fa06ffff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06817e2f0 con 0x7fa068058ad0 2023-11-08T16:29:10.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7f019b165640 1 -- 172.21.15.81:0/1899337249 shutdown_connections 2023-11-08T16:29:10.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7f019b165640 1 --2- 172.21.15.81:0/1899337249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01940b1fd0 0x7f01940cfd10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7f019b165640 1 --2- 172.21.15.81:0/1899337249 >> v2:172.21.15.93:3300/0 conn(0x7f01940cebd0 0x7f01940d3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.480+0000 7f019b165640 1 -- 172.21.15.81:0/1899337249 >> 172.21.15.81:0/1899337249 conn(0x7f01940d2e40 msgr2=0x7f01940d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7f81dd20e640 1 -- 172.21.15.81:0/1203471473 shutdown_connections 2023-11-08T16:29:10.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.476+0000 7f81dd20e640 1 -- 172.21.15.81:0/1203471473 wait complete. 2023-11-08T16:29:10.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.480+0000 7f019b165640 1 -- 172.21.15.81:0/1899337249 shutdown_connections 2023-11-08T16:29:10.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.478+0000 7f81dd20e640 1 Processor -- start 2023-11-08T16:29:10.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.480+0000 7f81dd20e640 1 -- start start 2023-11-08T16:29:10.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.480+0000 7fa06f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0680e3140 0x7fa0680de7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.480+0000 7fa06f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0680e3140 0x7fa0680de7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41890/0 (socket says 172.21.15.81:41890) 2023-11-08T16:29:10.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.480+0000 7fa06f7fe640 1 -- 172.21.15.81:0/3129510334 learned_addr learned my addr 172.21.15.81:0/3129510334 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.480+0000 7f019b165640 1 -- 172.21.15.81:0/1899337249 wait complete. 2023-11-08T16:29:10.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.480+0000 7ff088f5c640 1 --2- 172.21.15.81:0/3859857224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0840adb40 0x7ff0840c9060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff0841662c0 tx=0x7ff07402bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c9a1a53802808c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.480+0000 7fa06e7fc640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa068058ad0 0x7fa0680de2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7fab6e8d7640 1 -- start start 2023-11-08T16:29:10.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.480+0000 7fbe2956b640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbe0806ae70 0x7fbe0806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.480+0000 7fbe2a56d640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fbe08073090 0x7fbe080754b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7fbe2956b640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbe0806ae70 0x7fbe0806d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c03b420 tx=0x7fbe14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7fa06f7fe640 1 -- 172.21.15.81:0/3129510334 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa068058ad0 msgr2=0x7fa0680de2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7fa06f7fe640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa068058ad0 0x7fa0680de2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7fa06f7fe640 1 -- 172.21.15.81:0/3129510334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa06803e950 msgr2=0x7fa0680dc3f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-08T16:29:10.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7fa06f7fe640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa06803e950 0x7fa0680dc3f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7fa06f7fe640 1 -- 172.21.15.81:0/3129510334 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa060009400 con 0x7fa0680e3140 2023-11-08T16:29:10.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.479+0000 7f85be7fc640 1 -- 172.21.15.81:0/1930655589 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85c000bd10 con 0x7f85c815cff0 2023-11-08T16:29:10.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7f85be7fc640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f85ac06aec0 0x7f85ac06d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7f85be7fc640 1 -- 172.21.15.81:0/1930655589 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85c008a240 con 0x7f85c815cff0 2023-11-08T16:29:10.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7f85cd89c640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f85ac06aec0 0x7f85ac06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7f85be7fc640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f85ac0730e0 0x7f85ac075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7f85be7fc640 1 -- 172.21.15.81:0/1930655589 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f85ac075bb0 con 0x7f85ac0730e0 2023-11-08T16:29:10.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.481+0000 7f85be7fc640 1 -- 172.21.15.81:0/1930655589 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85c008c7e0 con 0x7f85c815cff0 2023-11-08T16:29:10.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7fbe2a56d640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fbe08073090 0x7fbe080754b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7fa06f7fe640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0680e3140 0x7fa0680de7f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa070061cc0 tx=0x7fa07004ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7f81dd20e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81d80adb40 0x7f81d815f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7fa0577fe640 1 -- 172.21.15.81:0/3129510334 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07006d030 con 0x7fa0680e3140 2023-11-08T16:29:10.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7f85cd89c640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f85ac06aec0 0x7f85ac06d380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f85c4000f60 tx=0x7f85c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7fa0577fe640 1 -- 172.21.15.81:0/3129510334 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa070068640 con 0x7fa0680e3140 2023-11-08T16:29:10.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7fa0577fe640 1 -- 172.21.15.81:0/3129510334 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07006d030 con 0x7fa0680e3140 2023-11-08T16:29:10.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7f81dd20e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81d80adf20 0x7f81d80cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7f81dd20e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f81d8165940 0x7f81d80ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.483+0000 7f81dd20e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d81624b0 con 0x7f81d80adb40 2023-11-08T16:29:10.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.483+0000 7f81dd20e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80c34a0 con 0x7f81d80adf20 2023-11-08T16:29:10.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.483+0000 7f81dd20e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d815fc60 con 0x7f81d8165940 2023-11-08T16:29:10.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.483+0000 7f81d659c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81d80adf20 0x7f81d80cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.483+0000 7f81d659c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81d80adf20 0x7f81d80cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:46578/0 (socket says 172.21.15.81:46578) 2023-11-08T16:29:10.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.483+0000 7f81d659c640 1 -- 172.21.15.81:0/2857591259 learned_addr learned my addr 172.21.15.81:0/2857591259 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.483+0000 7f81d759e640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f81d8165940 0x7f81d80ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7fab6e8d7640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fab68165940 0x7fab6816c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7fab6e8d7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab680ce3f0 con 0x7fab680adf20 2023-11-08T16:29:10.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7fab6e8d7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab680cac80 con 0x7fab680adb40 2023-11-08T16:29:10.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.482+0000 7fab6e8d7640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab680cb340 con 0x7fab68165940 2023-11-08T16:29:10.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.483+0000 7fa06ffff640 1 -- 172.21.15.81:0/3129510334 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa064009400 con 0x7fa0680e3140 2023-11-08T16:29:10.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7f85cd09b640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f85ac0730e0 0x7f85ac075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7ff083fff640 1 -- 172.21.15.81:0/3859857224 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07402caa0 con 0x7ff0840adb40 2023-11-08T16:29:10.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7ff083fff640 1 -- 172.21.15.81:0/3859857224 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07402cc00 con 0x7ff0840adb40 2023-11-08T16:29:10.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7f26c9dee640 1 Processor -- start 2023-11-08T16:29:10.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7ff083fff640 1 -- 172.21.15.81:0/3859857224 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0740343f0 con 0x7ff0840adb40 2023-11-08T16:29:10.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7fbe1a7fc640 1 -- 172.21.15.81:0/3728882396 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fbe08075b60 con 0x7fbe08073090 2023-11-08T16:29:10.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7fab67fff640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fab680adb40 0x7fab680c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50672/0 (socket says 172.21.15.81:50672) 2023-11-08T16:29:10.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7fab67fff640 1 -- 172.21.15.81:0/1457848235 learned_addr learned my addr 172.21.15.81:0/1457848235 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:10.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7fa06ffff640 1 -- 172.21.15.81:0/3129510334 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0680df270 con 0x7fa0680e3140 2023-11-08T16:29:10.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.485+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698000383 0 0) 0x7fab680cac80 con 0x7fab680adb40 2023-11-08T16:29:10.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.485+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab50003520 con 0x7fab680adb40 2023-11-08T16:29:10.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.485+0000 7f019b165640 1 Processor -- start 2023-11-08T16:29:10.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.485+0000 7ff08b1c0640 1 -- 172.21.15.81:0/3859857224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0840adb40 msgr2=0x7ff0840c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.485+0000 7ff08b1c0640 1 --2- 172.21.15.81:0/3859857224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0840adb40 0x7ff0840c9060 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff0841662c0 tx=0x7ff07402bf00 comp rx=0 tx=0).stop 2023-11-08T16:29:10.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.485+0000 7f019b165640 1 -- start start 2023-11-08T16:29:10.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.485+0000 7fa06ffff640 1 -- 172.21.15.81:0/3129510334 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa068058090 con 0x7fa0680e3140 2023-11-08T16:29:10.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7ff08b1c0640 1 -- 172.21.15.81:0/3859857224 shutdown_connections 2023-11-08T16:29:10.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7ff08b1c0640 1 --2- 172.21.15.81:0/3859857224 >> v2:172.21.15.93:3300/0 conn(0x7ff0840adf20 0x7ff084165230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7ff08b1c0640 1 --2- 172.21.15.81:0/3859857224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0840adb40 0x7ff0840c9060 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7ff08b1c0640 1 -- 172.21.15.81:0/3859857224 >> 172.21.15.81:0/3859857224 conn(0x7ff0840cb490 msgr2=0x7ff0840cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7f81d659c640 1 -- 172.21.15.81:0/2857591259 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f81d8165940 msgr2=0x7f81d80ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7f81d659c640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f81d8165940 0x7f81d80ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7f81d659c640 1 -- 172.21.15.81:0/2857591259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81d80adb40 msgr2=0x7f81d815f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7f81d659c640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81d80adb40 0x7f81d815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7f81d659c640 1 -- 172.21.15.81:0/2857591259 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c8009400 con 0x7f81d80adf20 2023-11-08T16:29:10.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.485+0000 7f81d659c640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81d80adf20 0x7f81d80cc5c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c780 tx=0x7f81d000cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7f81bf7fe640 1 -- 172.21.15.81:0/2857591259 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d000aa50 con 0x7f81d80adf20 2023-11-08T16:29:10.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7f81bf7fe640 1 -- 172.21.15.81:0/2857591259 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d000ce90 con 0x7f81d80adf20 2023-11-08T16:29:10.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 <== mon.1 v1:172.21.15.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 317780029 0 0) 0x7fab50003520 con 0x7fab680adb40 2023-11-08T16:29:10.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 >> v2:172.21.15.93:3300/0 conn(0x7fab68165940 msgr2=0x7fab6816c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:10.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7fab6ce74640 1 --2- 172.21.15.81:0/1457848235 >> v2:172.21.15.93:3300/0 conn(0x7fab68165940 0x7fab6816c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7fab66ffd640 1 --2- 172.21.15.81:0/1457848235 >> v2:172.21.15.93:3300/0 conn(0x7fab68165940 0x7fab6816c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 >> v1:172.21.15.81:6789/0 conn(0x7fab680adf20 legacy=0x7fab68165230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 --> v1:172.21.15.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab680caf60 con 0x7fab680adb40 2023-11-08T16:29:10.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 <== mon.1 v1:172.21.15.93:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fab58001670 con 0x7fab680adb40 2023-11-08T16:29:10.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 >> v1:172.21.15.93:6789/0 conn(0x7fab680adb40 legacy=0x7fab680c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7f81dd20e640 1 -- 172.21.15.81:0/2857591259 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d80c31c0 con 0x7f81d80adf20 2023-11-08T16:29:10.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f81dd20e640 1 -- 172.21.15.81:0/2857591259 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d81620d0 con 0x7f81d80adf20 2023-11-08T16:29:10.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f019b165640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01940b1fd0 0x7f01941fae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.484+0000 7f26c9dee640 1 -- start start 2023-11-08T16:29:10.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f26c37fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f26c415d5a0 0x7f26c40cf600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58848/0 (socket says 172.21.15.81:58848) 2023-11-08T16:29:10.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f019b165640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01940cebd0 0x7f019415e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f26c9dee640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26c40cebd0 0x7f26c40d3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f26c37fe640 1 -- 172.21.15.81:0/253176151 learned_addr learned my addr 172.21.15.81:0/253176151 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:10.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f019b165640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f019415d5a0 0x7f019415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f019b165640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0194160cb0 con 0x7f019415d5a0 2023-11-08T16:29:10.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f019b165640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01941fb3b0 con 0x7f01940b1fd0 2023-11-08T16:29:10.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f019b165640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019415de10 con 0x7f01940cebd0 2023-11-08T16:29:10.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f85cd09b640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f85ac0730e0 0x7f85ac075500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7f81bf7fe640 1 -- 172.21.15.81:0/2857591259 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d000ae30 con 0x7f81d80adf20 2023-11-08T16:29:10.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7f26c9dee640 1 --2- 172.21.15.81:0/253176151 >> v2:172.21.15.81:3300/0 conn(0x7f26c40b1fd0 0x7f26c40cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7f0198f01640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01940b1fd0 0x7f01941fae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7f0198f01640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01940b1fd0 0x7f01941fae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:46598/0 (socket says 172.21.15.81:46598) 2023-11-08T16:29:10.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7f0198f01640 1 -- 172.21.15.81:0/878747184 learned_addr learned my addr 172.21.15.81:0/878747184 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7ff08b1c0640 1 -- 172.21.15.81:0/3859857224 shutdown_connections 2023-11-08T16:29:10.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7ff08b1c0640 1 -- 172.21.15.81:0/3859857224 wait complete. 2023-11-08T16:29:10.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7f26c9dee640 1 -- 172.21.15.81:0/253176151 --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c40b4350 con 0x7f26c415d5a0 2023-11-08T16:29:10.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7f018bfff640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01940cebd0 0x7f019415e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7f26c9dee640 1 -- 172.21.15.81:0/253176151 --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c40b4630 con 0x7f26c40b1fd0 2023-11-08T16:29:10.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7f26c9dee640 1 -- 172.21.15.81:0/253176151 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c40d03f0 con 0x7f26c40cebd0 2023-11-08T16:29:10.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7f26c3fff640 1 --2- 172.21.15.81:0/253176151 >> v2:172.21.15.81:3300/0 conn(0x7f26c40b1fd0 0x7f26c40cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.486+0000 7fab66ffd640 1 --2- 172.21.15.81:0/1457848235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab50003aa0 0x7fab50003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7fab66ffd640 1 --2- 172.21.15.81:0/1457848235 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab500047c0 0x7fab50004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.487+0000 7fab677fe640 1 --2- 172.21.15.81:0/1457848235 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab500047c0 0x7fab50004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7fab66ffd640 1 --2- 172.21.15.81:0/1457848235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab50009970 0x7fab5000bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5000c350 con 0x7fab50009970 2023-11-08T16:29:10.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5000c4c0 con 0x7fab50003aa0 2023-11-08T16:29:10.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab5000c680 con 0x7fab500047c0 2023-11-08T16:29:10.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7f85be7fc640 1 -- 172.21.15.81:0/1930655589 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f85ac075bb0 con 0x7f85ac0730e0 2023-11-08T16:29:10.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7f26c2ffd640 1 --2- 172.21.15.81:0/253176151 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26c40cebd0 0x7f26c40d3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f81dd20e640 1 -- 172.21.15.81:0/2857591259 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f81d80ce0a0 con 0x7f81d80adf20 2023-11-08T16:29:10.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.488+0000 7fab6ce74640 1 --2- 172.21.15.81:0/1457848235 >> v2:172.21.15.93:3300/0 conn(0x7fab68165940 0x7fab6816c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:10.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7ff08b1c0640 1 Processor -- start 2023-11-08T16:29:10.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f26c27fc640 1 -- 172.21.15.81:0/253176151 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44608860 0 0) 0x7f26c40b4350 con 0x7f26c415d5a0 2023-11-08T16:29:10.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f26c27fc640 1 -- 172.21.15.81:0/253176151 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26ac003520 con 0x7f26c415d5a0 2023-11-08T16:29:10.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7ff08b1c0640 1 -- start start 2023-11-08T16:29:10.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f018bfff640 1 -- 172.21.15.81:0/878747184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01940b1fd0 msgr2=0x7f01941fae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f018bfff640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01940b1fd0 0x7f01941fae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f018bfff640 1 -- 172.21.15.81:0/878747184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f019415d5a0 msgr2=0x7f019415be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f018bfff640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f019415d5a0 0x7f019415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f018bfff640 1 -- 172.21.15.81:0/878747184 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c009a70 con 0x7f01940cebd0 2023-11-08T16:29:10.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f26c3fff640 1 -- 172.21.15.81:0/253176151 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26c40cebd0 msgr2=0x7f26c40d3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:10.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f26c3fff640 1 --2- 172.21.15.81:0/253176151 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26c40cebd0 0x7f26c40d3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f26c3fff640 1 -- 172.21.15.81:0/253176151 >> v1:172.21.15.81:6789/0 conn(0x7f26c415d5a0 legacy=0x7f26c40cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7f26c3fff640 1 -- 172.21.15.81:0/253176151 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c40ce6b0 con 0x7f26c40b1fd0 2023-11-08T16:29:10.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.489+0000 7fab67fff640 1 --2- 172.21.15.81:0/1457848235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab50003aa0 0x7fab50003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.490+0000 7f018bfff640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01940cebd0 0x7f019415e090 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f01940d3ea0 tx=0x7f017c00c9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.490+0000 7fab677fe640 1 -- 172.21.15.81:0/1457848235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab50003aa0 msgr2=0x7fab50003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:10.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.490+0000 7fab677fe640 1 --2- 172.21.15.81:0/1457848235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab50003aa0 0x7fab50003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.490+0000 7fab677fe640 1 -- 172.21.15.81:0/1457848235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab50009970 msgr2=0x7fab5000bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:10.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.490+0000 7fab677fe640 1 --2- 172.21.15.81:0/1457848235 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab50009970 0x7fab5000bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.490+0000 7fab677fe640 1 -- 172.21.15.81:0/1457848235 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50003520 con 0x7fab500047c0 2023-11-08T16:29:10.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.490+0000 7f0191ffb640 1 -- 172.21.15.81:0/878747184 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c004220 con 0x7f01940cebd0 2023-11-08T16:29:10.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.490+0000 7f0191ffb640 1 -- 172.21.15.81:0/878747184 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017c004380 con 0x7f01940cebd0 2023-11-08T16:29:10.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.490+0000 7f0191ffb640 1 -- 172.21.15.81:0/878747184 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c003810 con 0x7f01940cebd0 2023-11-08T16:29:10.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.490+0000 7f019b165640 1 -- 172.21.15.81:0/878747184 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01941609d0 con 0x7f01940cebd0 2023-11-08T16:29:10.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.491+0000 7ff08b1c0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0840adb40 0x7ff08415f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.491+0000 7ff08b1c0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0840adf20 0x7ff0840c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.491+0000 7ff08b1c0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff084165770 0x7ff0840ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.491+0000 7fa0577fe640 1 -- 172.21.15.81:0/3129510334 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa0700724c0 con 0x7fa0680e3140 2023-11-08T16:29:10.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.491+0000 7ff08b1c0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08415f5d0 con 0x7ff0840adb40 2023-11-08T16:29:10.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.491+0000 7ff08b1c0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840c55d0 con 0x7ff0840adf20 2023-11-08T16:29:10.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.491+0000 7ff08b1c0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840c4530 con 0x7ff084165770 2023-11-08T16:29:10.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.491+0000 7fab6ce74640 1 -- 172.21.15.81:0/1457848235 reap_dead start 2023-11-08T16:29:10.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7fab677fe640 1 --2- 172.21.15.81:0/1457848235 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab500047c0 0x7fab50004c40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fab5c001720 tx=0x7fab500057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21cf7af671e61316 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7f26c3fff640 1 --2- 172.21.15.81:0/253176151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26c40b1fd0 0x7f26c40cfd10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009d90 tx=0x7f26bc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47ca1eef2e6e269 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7fa0577fe640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05c06aed0 0x7fa05c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7ff08975d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff084165770 0x7ff0840ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0840adf20 0x7ff0840c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7f26c27fc640 1 -- 172.21.15.81:0/253176151 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc02cc60 con 0x7f26c40b1fd0 2023-11-08T16:29:10.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0840adf20 0x7ff0840c3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:46622/0 (socket says 172.21.15.81:46622) 2023-11-08T16:29:10.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7fa0577fe640 1 -- 172.21.15.81:0/3129510334 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa07006e070 con 0x7fa0680e3140 2023-11-08T16:29:10.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7fa0577fe640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fa05c0730f0 0x7fa05c075510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7fa0577fe640 1 -- 172.21.15.81:0/3129510334 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa05c075bc0 con 0x7fa05c0730f0 2023-11-08T16:29:10.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7fa0577fe640 1 -- 172.21.15.81:0/3129510334 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0701058a0 con 0x7fa0680e3140 2023-11-08T16:29:10.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7ff07bfff640 1 -- 172.21.15.81:0/167738415 learned_addr learned my addr 172.21.15.81:0/167738415 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7f26c27fc640 1 -- 172.21.15.81:0/253176151 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc02cdc0 con 0x7f26c40b1fd0 2023-11-08T16:29:10.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7f26c27fc640 1 -- 172.21.15.81:0/253176151 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc034640 con 0x7f26c40b1fd0 2023-11-08T16:29:10.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.493+0000 7ff088f5c640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0840adb40 0x7ff08415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.493+0000 7fa06e7fc640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fa05c0730f0 0x7fa05c075510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.493+0000 7f019b165640 1 -- 172.21.15.81:0/878747184 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01940b7610 con 0x7f01940cebd0 2023-11-08T16:29:10.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.493+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c0052d0 con 0x7fab500047c0 2023-11-08T16:29:10.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.493+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab5c02e050 con 0x7fab500047c0 2023-11-08T16:29:10.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.493+0000 7fab66ffd640 1 -- 172.21.15.81:0/1457848235 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab5c003830 con 0x7fab500047c0 2023-11-08T16:29:10.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.493+0000 7ff07bfff640 1 -- 172.21.15.81:0/167738415 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff084165770 msgr2=0x7ff0840ae960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.494+0000 7ff07bfff640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff084165770 0x7ff0840ae960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.492+0000 7f81bf7fe640 1 -- 172.21.15.81:0/2857591259 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81d0012420 con 0x7f81d80adf20 2023-11-08T16:29:10.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.494+0000 7ff07bfff640 1 -- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0840adb40 msgr2=0x7ff08415f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.494+0000 7ff07bfff640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0840adb40 0x7ff08415f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.494+0000 7fa06effd640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05c06aed0 0x7fa05c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.494+0000 7fa06e7fc640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fa05c0730f0 0x7fa05c075510 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.494+0000 7ff07bfff640 1 -- 172.21.15.81:0/167738415 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff074009b30 con 0x7ff0840adf20 2023-11-08T16:29:10.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.494+0000 7ff08975d640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff084165770 0x7ff0840ae960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:10.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.494+0000 7fab6e8d7640 1 -- 172.21.15.81:0/1457848235 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab500047c0 msgr2=0x7fab50004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.494+0000 7fab6e8d7640 1 --2- 172.21.15.81:0/1457848235 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab500047c0 0x7fab50004c40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fab5c001720 tx=0x7fab500057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.494+0000 7f26c9dee640 1 -- 172.21.15.81:0/253176151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26c40b1fd0 msgr2=0x7f26c40cfd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.495+0000 7ff07bfff640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0840adf20 0x7ff0840c3b70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00b3e0 tx=0x7ff06c00b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.495+0000 7f019b165640 1 -- 172.21.15.81:0/878747184 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f01940d06d0 con 0x7f01940cebd0 2023-11-08T16:29:10.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.495+0000 7fab6e8d7640 1 -- 172.21.15.81:0/1457848235 shutdown_connections 2023-11-08T16:29:10.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.495+0000 7fab6e8d7640 1 --2- 172.21.15.81:0/1457848235 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab500047c0 0x7fab50004c40 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fab5c001720 tx=0x7fab500057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.495+0000 7fab6e8d7640 1 -- 172.21.15.81:0/1457848235 >> 172.21.15.81:0/1457848235 conn(0x7fab680cb490 msgr2=0x7fab680cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.495+0000 7f81bf7fe640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f81c406aec0 0x7f81c406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.495+0000 7fab6e8d7640 1 -- 172.21.15.81:0/1457848235 shutdown_connections 2023-11-08T16:29:10.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.495+0000 7fab6e8d7640 1 -- 172.21.15.81:0/1457848235 wait complete. 2023-11-08T16:29:10.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.495+0000 7f81bf7fe640 1 -- 172.21.15.81:0/2857591259 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81d008a140 con 0x7f81d80adf20 2023-11-08T16:29:10.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.496+0000 7fa0577fe640 1 -- 172.21.15.81:0/3129510334 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fa05c075bc0 con 0x7fa05c0730f0 2023-11-08T16:29:10.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.495+0000 7f81d6d9d640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f81c406aec0 0x7f81c406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.496+0000 7f0191ffb640 1 -- 172.21.15.81:0/878747184 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f017c003970 con 0x7f01940cebd0 2023-11-08T16:29:10.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.494+0000 7f26c9dee640 1 --2- 172.21.15.81:0/253176151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26c40b1fd0 0x7f26c40cfd10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f26bc009d90 tx=0x7f26bc02beb0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.496+0000 7f26c9dee640 1 -- 172.21.15.81:0/253176151 shutdown_connections 2023-11-08T16:29:10.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.496+0000 7f26c9dee640 1 --2- 172.21.15.81:0/253176151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26c40b1fd0 0x7f26c40cfd10 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.496+0000 7f81bf7fe640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f81c40730e0 0x7f81c4075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.496+0000 7f81bf7fe640 1 -- 172.21.15.81:0/2857591259 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81c4075bb0 con 0x7f81c40730e0 2023-11-08T16:29:10.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.496+0000 7f81bf7fe640 1 -- 172.21.15.81:0/2857591259 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81d008a4f0 con 0x7f81d80adf20 2023-11-08T16:29:10.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.496+0000 7ff081ffb640 1 -- 172.21.15.81:0/167738415 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c011830 con 0x7ff0840adf20 2023-11-08T16:29:10.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.496+0000 7f26c9dee640 1 --2- 172.21.15.81:0/253176151 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26c40cebd0 0x7f26c40d3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.496+0000 7f26c9dee640 1 -- 172.21.15.81:0/253176151 >> 172.21.15.81:0/253176151 conn(0x7f26c40d2e40 msgr2=0x7f26c40d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7ff081ffb640 1 -- 172.21.15.81:0/167738415 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c011990 con 0x7ff0840adf20 2023-11-08T16:29:10.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7ff081ffb640 1 -- 172.21.15.81:0/167738415 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c011d60 con 0x7ff0840adf20 2023-11-08T16:29:10.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7ff08b1c0640 1 -- 172.21.15.81:0/167738415 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0840c4810 con 0x7ff0840adf20 2023-11-08T16:29:10.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7f0191ffb640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f017806af40 0x7f017806d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7f0191ffb640 1 -- 172.21.15.81:0/878747184 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f017c088940 con 0x7f01940cebd0 2023-11-08T16:29:10.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7f0191ffb640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f0178073160 0x7f0178075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7f0191ffb640 1 -- 172.21.15.81:0/878747184 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0178075c30 con 0x7f0178073160 2023-11-08T16:29:10.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7f0191ffb640 1 -- 172.21.15.81:0/878747184 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f017c033ea0 con 0x7f01940cebd0 2023-11-08T16:29:10.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7f26c9dee640 1 -- 172.21.15.81:0/253176151 shutdown_connections 2023-11-08T16:29:10.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7f26c9dee640 1 -- 172.21.15.81:0/253176151 wait complete. 2023-11-08T16:29:10.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7fa06effd640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05c06aed0 0x7fa05c06d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0680e7040 tx=0x7fa060009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.497+0000 7ff08b1c0640 1 -- 172.21.15.81:0/167738415 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0840c4dd0 con 0x7ff0840adf20 2023-11-08T16:29:10.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.498+0000 7f0199702640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f0178073160 0x7f0178075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.498+0000 7f81d6d9d640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f81c406aec0 0x7f81c406d380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006f40 tx=0x7f81cc006e90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.498+0000 7f81d759e640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f81c40730e0 0x7f81c4075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.498+0000 7ff08b1c0640 1 -- 172.21.15.81:0/167738415 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff0840ce6d0 con 0x7ff0840adf20 2023-11-08T16:29:10.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.499+0000 7f0199702640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f0178073160 0x7f0178075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.499+0000 7f0198f01640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f017806af40 0x7f017806d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.499+0000 7f81d759e640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f81c40730e0 0x7f81c4075500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.499+0000 7fab6e8d7640 1 Processor -- start 2023-11-08T16:29:10.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.500+0000 7fab6e8d7640 1 -- start start 2023-11-08T16:29:10.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.500+0000 7f26c9dee640 1 Processor -- start 2023-11-08T16:29:10.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.500+0000 7f0198f01640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f017806af40 0x7f017806d400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0184005880 tx=0x7f0184009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.500+0000 7f26c9dee640 1 -- start start 2023-11-08T16:29:10.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.501+0000 7f26c9dee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26c40b1fd0 0x7f26c40c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.501+0000 7f26c9dee640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26c40cebd0 0x7f26c41fab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.501+0000 7f26c9dee640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26c415d5a0 0x7f26c41fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.501+0000 7f26c9dee640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c4152c10 con 0x7f26c40b1fd0 2023-11-08T16:29:10.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.501+0000 7f26c9dee640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c40b7470 con 0x7f26c40cebd0 2023-11-08T16:29:10.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.501+0000 7f26c9dee640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c40b7750 con 0x7f26c415d5a0 2023-11-08T16:29:10.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.501+0000 7f0191ffb640 1 -- 172.21.15.81:0/878747184 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f0178075c30 con 0x7f0178073160 2023-11-08T16:29:10.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.501+0000 7f26c2ffd640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26c40cebd0 0x7f26c41fab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7f26c2ffd640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26c40cebd0 0x7f26c41fab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:46634/0 (socket says 172.21.15.81:46634) 2023-11-08T16:29:10.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7f26c2ffd640 1 -- 172.21.15.81:0/3419653318 learned_addr learned my addr 172.21.15.81:0/3419653318 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7f26c37fe640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26c40b1fd0 0x7f26c40c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7fab6e8d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab68156fc0 0x7fab681643b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7f26c2ffd640 1 -- 172.21.15.81:0/3419653318 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26c415d5a0 msgr2=0x7f26c41fb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7fab6e8d7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab68165280 0x7fab681648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7f26c2ffd640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26c415d5a0 0x7f26c41fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7f26c2ffd640 1 -- 172.21.15.81:0/3419653318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26c40b1fd0 msgr2=0x7f26c40c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7f26c2ffd640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26c40b1fd0 0x7f26c40c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7f26c2ffd640 1 -- 172.21.15.81:0/3419653318 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc009a70 con 0x7f26c40cebd0 2023-11-08T16:29:10.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7f26c37fe640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26c40b1fd0 0x7f26c40c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:10.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.503+0000 7fab67fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab68156fc0 0x7fab681643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.503+0000 7f26c2ffd640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26c40cebd0 0x7f26c41fab80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e840 tx=0x7f26b800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.503+0000 7fab6e8d7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab680b3110 0x7fab680b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.503+0000 7fab6e8d7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab681f78d0 con 0x7fab68156fc0 2023-11-08T16:29:10.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7ff081ffb640 1 -- 172.21.15.81:0/167738415 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff06c01a400 con 0x7ff0840adf20 2023-11-08T16:29:10.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.504+0000 7ff081ffb640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff06406ae70 0x7ff06406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.504+0000 7ff088f5c640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff06406ae70 0x7ff06406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.505+0000 7ff088f5c640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff06406ae70 0x7ff06406d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff074035000 tx=0x7ff074030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.506+0000 7ff081ffb640 1 -- 172.21.15.81:0/167738415 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff06c059230 con 0x7ff0840adf20 2023-11-08T16:29:10.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.506+0000 7ff081ffb640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7ff064073c20 0x7ff0640760c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.506+0000 7ff081ffb640 1 -- 172.21.15.81:0/167738415 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff064076790 con 0x7ff064073c20 2023-11-08T16:29:10.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.506+0000 7ff081ffb640 1 -- 172.21.15.81:0/167738415 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff06c009210 con 0x7ff0840adf20 2023-11-08T16:29:10.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.506+0000 7ff08975d640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7ff064073c20 0x7ff0640760c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.502+0000 7f81bf7fe640 1 -- 172.21.15.81:0/2857591259 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f81c4075bb0 con 0x7f81c40730e0 2023-11-08T16:29:10.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.503+0000 7f26abfff640 1 -- 172.21.15.81:0/3419653318 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b800c940 con 0x7f26c40cebd0 2023-11-08T16:29:10.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.503+0000 7f26abfff640 1 -- 172.21.15.81:0/3419653318 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b800caa0 con 0x7f26c40cebd0 2023-11-08T16:29:10.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.505+0000 7f26c9dee640 1 -- 172.21.15.81:0/3419653318 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c40b7a30 con 0x7f26c40cebd0 2023-11-08T16:29:10.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.505+0000 7f26c9dee640 1 -- 172.21.15.81:0/3419653318 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c40b7100 con 0x7f26c40cebd0 2023-11-08T16:29:10.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.505+0000 7f26abfff640 1 -- 172.21.15.81:0/3419653318 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b800ce30 con 0x7f26c40cebd0 2023-11-08T16:29:10.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.506+0000 7f26c9dee640 1 -- 172.21.15.81:0/3419653318 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f26c40d06d0 con 0x7f26c40cebd0 2023-11-08T16:29:10.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.504+0000 7fab67fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab68156fc0 0x7fab681643b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41962/0 (socket says 172.21.15.81:41962) 2023-11-08T16:29:10.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.507+0000 7ff08975d640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7ff064073c20 0x7ff0640760c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.504+0000 7fab67fff640 1 -- 172.21.15.81:0/3356581783 learned_addr learned my addr 172.21.15.81:0/3356581783 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:10.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.505+0000 7fab6ce74640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab680b3110 0x7fab680b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.507+0000 7fab677fe640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab68165280 0x7fab681648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.507+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab680c84b0 con 0x7fab68165280 2023-11-08T16:29:10.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.507+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab68164e30 con 0x7fab680b3110 2023-11-08T16:29:10.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.508+0000 7fab6ce74640 1 -- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab68165280 msgr2=0x7fab681648f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.508+0000 7fab6ce74640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab68165280 0x7fab681648f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.508+0000 7fab6ce74640 1 -- 172.21.15.81:0/3356581783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab68156fc0 msgr2=0x7fab681643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.508+0000 7fab6ce74640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab68156fc0 0x7fab681643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.508+0000 7fab6ce74640 1 -- 172.21.15.81:0/3356581783 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab680b2350 con 0x7fab680b3110 2023-11-08T16:29:10.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.508+0000 7fab6ce74640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab680b3110 0x7fab680b35a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fab680c5810 tx=0x7fab6000a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.509+0000 7fab67fff640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab68156fc0 0x7fab681643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:10.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.508+0000 7f26abfff640 1 -- 172.21.15.81:0/3419653318 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f26b8013470 con 0x7f26c40cebd0 2023-11-08T16:29:10.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.510+0000 7ff081ffb640 1 -- 172.21.15.81:0/167738415 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7ff064076790 con 0x7ff064073c20 2023-11-08T16:29:10.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.510+0000 7fab64ff9640 1 -- 172.21.15.81:0/3356581783 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab60015440 con 0x7fab680b3110 2023-11-08T16:29:10.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.510+0000 7fab64ff9640 1 -- 172.21.15.81:0/3356581783 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab600155a0 con 0x7fab680b3110 2023-11-08T16:29:10.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.510+0000 7fab64ff9640 1 -- 172.21.15.81:0/3356581783 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab60015950 con 0x7fab680b3110 2023-11-08T16:29:10.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.510+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab680b2630 con 0x7fab680b3110 2023-11-08T16:29:10.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.510+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab680b2bf0 con 0x7fab680b3110 2023-11-08T16:29:10.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.510+0000 7f26abfff640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f26b006ae70 0x7f26b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.510+0000 7f26abfff640 1 -- 172.21.15.81:0/3419653318 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26b808a190 con 0x7f26c40cebd0 2023-11-08T16:29:10.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.510+0000 7f26abfff640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7f26b0073010 0x7f26b0075430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.510+0000 7f26abfff640 1 -- 172.21.15.81:0/3419653318 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f26b0075ae0 con 0x7f26b0073010 2023-11-08T16:29:10.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.510+0000 7f26abfff640 1 -- 172.21.15.81:0/3419653318 <== mon.1 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f26b808a500 con 0x7f26c40cebd0 2023-11-08T16:29:10.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.512+0000 7f26c37fe640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f26b006ae70 0x7f26b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.511+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fab680ce0a0 con 0x7fab680b3110 2023-11-08T16:29:10.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.514+0000 7f26c37fe640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f26b006ae70 0x7f26b006d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f26b40011e0 tx=0x7f26b4007d30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.515+0000 7f26c3fff640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7f26b0073010 0x7f26b0075430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.515+0000 7fab64ff9640 1 -- 172.21.15.81:0/3356581783 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab6000d430 con 0x7fab680b3110 2023-11-08T16:29:10.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.516+0000 7f26c3fff640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7f26b0073010 0x7f26b0075430 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.516+0000 7fab64ff9640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fab5406aec0 0x7fab5406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.516+0000 7fab64ff9640 1 -- 172.21.15.81:0/3356581783 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab6008a240 con 0x7fab680b3110 2023-11-08T16:29:10.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.517+0000 7fab67fff640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fab5406aec0 0x7fab5406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.517+0000 7fab64ff9640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7fab540730e0 0x7fab54075500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:10.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.517+0000 7fab64ff9640 1 -- 172.21.15.81:0/3356581783 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab54075bb0 con 0x7fab540730e0 2023-11-08T16:29:10.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.517+0000 7fab64ff9640 1 -- 172.21.15.81:0/3356581783 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab6008a5b0 con 0x7fab680b3110 2023-11-08T16:29:10.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.517+0000 7fab677fe640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7fab540730e0 0x7fab54075500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:10.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.519+0000 7f26abfff640 1 -- 172.21.15.81:0/3419653318 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f26b0075ae0 con 0x7f26b0073010 2023-11-08T16:29:10.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.519+0000 7fab677fe640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7fab540730e0 0x7fab54075500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.519+0000 7fab67fff640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fab5406aec0 0x7fab5406d380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab58001050 tx=0x7fab58007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:10.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.521+0000 7fab64ff9640 1 -- 172.21.15.81:0/3356581783 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fab54075bb0 con 0x7fab540730e0 2023-11-08T16:29:10.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.524+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbe1c0064c0 con 0x7fbe08073090 2023-11-08T16:29:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.524+0000 7fbe1a7fc640 1 -- 172.21.15.81:0/3728882396 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fbe24071080 con 0x7fbe08073090 2023-11-08T16:29:10.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.525+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fbe08073090 msgr2=0x7fbe080754b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.525+0000 7fbe2bfd0640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fbe08073090 0x7fbe080754b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.526+0000 7f85cfb00640 1 -- 172.21.15.81:0/1930655589 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f85b0002cb0 con 0x7f85ac0730e0 2023-11-08T16:29:10.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.527+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbe0806ae70 msgr2=0x7fbe0806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.527+0000 7fbe2bfd0640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbe0806ae70 0x7fbe0806d330 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbe1c03b420 tx=0x7fbe14008040 comp rx=0 tx=0).stop 2023-11-08T16:29:10.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.527+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1c053500 msgr2=0x7fbe1c0e61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.527+0000 7fbe2bfd0640 1 --2- 172.21.15.81:0/3728882396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1c053500 0x7fbe1c0e61d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbe2000e860 tx=0x7fbe2000ed20 comp rx=0 tx=0).stop 2023-11-08T16:29:10.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.527+0000 7fbe2a56d640 1 -- 172.21.15.81:0/3728882396 reap_dead start 2023-11-08T16:29:10.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.527+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 shutdown_connections 2023-11-08T16:29:10.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.527+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 >> 172.21.15.81:0/3728882396 conn(0x7fbe1c058eb0 msgr2=0x7fbe1c0ea050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.527+0000 7f85be7fc640 1 -- 172.21.15.81:0/1930655589 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f85b0002cb0 con 0x7f85ac0730e0 2023-11-08T16:29:10.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.527+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 shutdown_connections 2023-11-08T16:29:10.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.527+0000 7fa06ffff640 1 -- 172.21.15.81:0/3129510334 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa068057db0 con 0x7fa05c0730f0 2023-11-08T16:29:10.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.528+0000 7fbe2bfd0640 1 -- 172.21.15.81:0/3728882396 wait complete. 2023-11-08T16:29:10.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.528+0000 7fa0577fe640 1 -- 172.21.15.81:0/3129510334 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa0640307d0 con 0x7fa05c0730f0 2023-11-08T16:29:10.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.528+0000 7f85cfb00640 1 -- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f85ac0730e0 msgr2=0x7f85ac075500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.528+0000 7f85cfb00640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f85ac0730e0 0x7f85ac075500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7f85cfb00640 1 -- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f85ac06aec0 msgr2=0x7f85ac06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7f85cfb00640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f85ac06aec0 0x7f85ac06d380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f85c4000f60 tx=0x7f85c4009040 comp rx=0 tx=0).stop 2023-11-08T16:29:10.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7f85cfb00640 1 -- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c815cff0 msgr2=0x7f85c815a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7f85cfb00640 1 --2- 172.21.15.81:0/1930655589 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f85c815cff0 0x7f85c815a140 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85c000c690 tx=0x7f85c000cb50 comp rx=0 tx=0).stop 2023-11-08T16:29:10.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7f85ce09d640 1 -- 172.21.15.81:0/1930655589 reap_dead start 2023-11-08T16:29:10.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7fa06ffff640 1 -- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fa05c0730f0 msgr2=0x7fa05c075510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7fa06ffff640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fa05c0730f0 0x7fa05c075510 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7f85cfb00640 1 -- 172.21.15.81:0/1930655589 shutdown_connections 2023-11-08T16:29:10.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7f85cfb00640 1 -- 172.21.15.81:0/1930655589 >> 172.21.15.81:0/1930655589 conn(0x7f85c80d2c50 msgr2=0x7f85c815ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7fa06ffff640 1 -- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05c06aed0 msgr2=0x7fa05c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7f85cfb00640 1 -- 172.21.15.81:0/1930655589 shutdown_connections 2023-11-08T16:29:10.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7f85cfb00640 1 -- 172.21.15.81:0/1930655589 wait complete. 2023-11-08T16:29:10.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7f019b165640 1 -- 172.21.15.81:0/878747184 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f019404ece0 con 0x7f0178073160 2023-11-08T16:29:10.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7fa06ffff640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05c06aed0 0x7fa05c06d390 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa0680e7040 tx=0x7fa060009190 comp rx=0 tx=0).stop 2023-11-08T16:29:10.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7fa06ffff640 1 -- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0680e3140 msgr2=0x7fa0680de7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7fa06ffff640 1 --2- 172.21.15.81:0/3129510334 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa0680e3140 0x7fa0680de7f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa070061cc0 tx=0x7fa07004ea00 comp rx=0 tx=0).stop 2023-11-08T16:29:10.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.529+0000 7fa06f7fe640 1 -- 172.21.15.81:0/3129510334 reap_dead start 2023-11-08T16:29:10.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.530+0000 7fa06ffff640 1 -- 172.21.15.81:0/3129510334 shutdown_connections 2023-11-08T16:29:10.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.530+0000 7fa06ffff640 1 -- 172.21.15.81:0/3129510334 >> 172.21.15.81:0/3129510334 conn(0x7fa068055a40 msgr2=0x7fa0680e5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.530+0000 7fa06ffff640 1 -- 172.21.15.81:0/3129510334 shutdown_connections 2023-11-08T16:29:10.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.530+0000 7f81dd20e640 1 -- 172.21.15.81:0/2857591259 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f81d80ce6d0 con 0x7f81c40730e0 2023-11-08T16:29:10.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.531+0000 7f0191ffb640 1 -- 172.21.15.81:0/878747184 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f019404ece0 con 0x7f0178073160 2023-11-08T16:29:10.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.531+0000 7f018affd640 1 -- 172.21.15.81:0/878747184 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f0178073160 msgr2=0x7f0178075580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.531+0000 7f018affd640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f0178073160 0x7f0178075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.531+0000 7f018affd640 1 -- 172.21.15.81:0/878747184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f017806af40 msgr2=0x7f017806d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.531+0000 7f81bf7fe640 1 -- 172.21.15.81:0/2857591259 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f81c80319b0 con 0x7f81c40730e0 2023-11-08T16:29:10.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.532+0000 7f81dd20e640 1 -- 172.21.15.81:0/2857591259 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f81c40730e0 msgr2=0x7f81c4075500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.532+0000 7f81dd20e640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f81c40730e0 0x7f81c4075500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.532+0000 7f81dd20e640 1 -- 172.21.15.81:0/2857591259 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f81c406aec0 msgr2=0x7f81c406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.532+0000 7f81dd20e640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f81c406aec0 0x7f81c406d380 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f81cc006f40 tx=0x7f81cc006e90 comp rx=0 tx=0).stop 2023-11-08T16:29:10.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.532+0000 7f81dd20e640 1 -- 172.21.15.81:0/2857591259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81d80adf20 msgr2=0x7f81d80cc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.532+0000 7f81dd20e640 1 --2- 172.21.15.81:0/2857591259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81d80adf20 0x7f81d80cc5c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f81d000c780 tx=0x7f81d000cc40 comp rx=0 tx=0).stop 2023-11-08T16:29:10.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.532+0000 7f81d759e640 1 -- 172.21.15.81:0/2857591259 reap_dead start 2023-11-08T16:29:10.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.532+0000 7f81dd20e640 1 -- 172.21.15.81:0/2857591259 shutdown_connections 2023-11-08T16:29:10.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.532+0000 7f81dd20e640 1 -- 172.21.15.81:0/2857591259 >> 172.21.15.81:0/2857591259 conn(0x7f81d80cb490 msgr2=0x7f81d80afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.531+0000 7f018affd640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f017806af40 0x7f017806d400 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0184005880 tx=0x7f0184009040 comp rx=0 tx=0).stop 2023-11-08T16:29:10.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.531+0000 7f018affd640 1 -- 172.21.15.81:0/878747184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01940cebd0 msgr2=0x7f019415e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.531+0000 7f018affd640 1 --2- 172.21.15.81:0/878747184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01940cebd0 0x7f019415e090 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f01940d3ea0 tx=0x7f017c00c9d0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.531+0000 7f0199702640 1 -- 172.21.15.81:0/878747184 reap_dead start 2023-11-08T16:29:10.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.532+0000 7f018affd640 1 -- 172.21.15.81:0/878747184 shutdown_connections 2023-11-08T16:29:10.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.532+0000 7f018affd640 1 -- 172.21.15.81:0/878747184 >> 172.21.15.81:0/878747184 conn(0x7f01940d2e40 msgr2=0x7f01940b5240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.533+0000 7f018affd640 1 -- 172.21.15.81:0/878747184 shutdown_connections 2023-11-08T16:29:10.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.533+0000 7f018affd640 1 -- 172.21.15.81:0/878747184 wait complete. 2023-11-08T16:29:10.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.533+0000 7f81dd20e640 1 -- 172.21.15.81:0/2857591259 shutdown_connections 2023-11-08T16:29:10.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.533+0000 7f81dd20e640 1 -- 172.21.15.81:0/2857591259 wait complete. 2023-11-08T16:29:10.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.534+0000 7fa06ffff640 1 -- 172.21.15.81:0/3129510334 wait complete. 2023-11-08T16:29:10.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.541+0000 7ff08b1c0640 1 -- 172.21.15.81:0/167738415 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff0840caf60 con 0x7ff064073c20 2023-11-08T16:29:10.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.543+0000 7ff081ffb640 1 -- 172.21.15.81:0/167738415 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ff07c0047e0 con 0x7ff064073c20 2023-11-08T16:29:10.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.544+0000 7ff08b1c0640 1 -- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7ff064073c20 msgr2=0x7ff0640760c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.544+0000 7ff08b1c0640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7ff064073c20 0x7ff0640760c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.548+0000 7ff08b1c0640 1 -- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff06406ae70 msgr2=0x7ff06406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.548+0000 7f26c9dee640 1 -- 172.21.15.81:0/3419653318 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f26c40b75b0 con 0x7f26b0073010 2023-11-08T16:29:10.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.548+0000 7ff08b1c0640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff06406ae70 0x7ff06406d330 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff074035000 tx=0x7ff074030000 comp rx=0 tx=0).stop 2023-11-08T16:29:10.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.548+0000 7ff08b1c0640 1 -- 172.21.15.81:0/167738415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0840adf20 msgr2=0x7ff0840c3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.548+0000 7ff08b1c0640 1 --2- 172.21.15.81:0/167738415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0840adf20 0x7ff0840c3b70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00b3e0 tx=0x7ff06c00b8a0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.548+0000 7ff08975d640 1 -- 172.21.15.81:0/167738415 reap_dead start 2023-11-08T16:29:10.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.548+0000 7ff08b1c0640 1 -- 172.21.15.81:0/167738415 shutdown_connections 2023-11-08T16:29:10.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.548+0000 7ff08b1c0640 1 -- 172.21.15.81:0/167738415 >> 172.21.15.81:0/167738415 conn(0x7ff0840cb490 msgr2=0x7ff08406b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.549+0000 7ff08b1c0640 1 -- 172.21.15.81:0/167738415 shutdown_connections 2023-11-08T16:29:10.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.549+0000 7ff08b1c0640 1 -- 172.21.15.81:0/167738415 wait complete. 2023-11-08T16:29:10.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.550+0000 7f26abfff640 1 -- 172.21.15.81:0/3419653318 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f26c40b75b0 con 0x7f26b0073010 2023-11-08T16:29:10.687 INFO:teuthology.orchestra.run.smithi081.stdout:55834574853 2023-11-08T16:29:10.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.554+0000 7f26c9dee640 1 -- 172.21.15.81:0/3419653318 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7f26b0073010 msgr2=0x7f26b0075430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.554+0000 7f26c9dee640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7f26b0073010 0x7f26b0075430 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.554+0000 7f26c9dee640 1 -- 172.21.15.81:0/3419653318 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f26b006ae70 msgr2=0x7f26b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.555+0000 7f26c9dee640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f26b006ae70 0x7f26b006d330 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f26b40011e0 tx=0x7f26b4007d30 comp rx=0 tx=0).stop 2023-11-08T16:29:10.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.555+0000 7f26c9dee640 1 -- 172.21.15.81:0/3419653318 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26c40cebd0 msgr2=0x7f26c41fab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.555+0000 7f26c9dee640 1 --2- 172.21.15.81:0/3419653318 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26c40cebd0 0x7f26c41fab80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f26b800e840 tx=0x7f26b800ed00 comp rx=0 tx=0).stop 2023-11-08T16:29:10.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.556+0000 7f26c3fff640 1 -- 172.21.15.81:0/3419653318 reap_dead start 2023-11-08T16:29:10.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.556+0000 7f26c9dee640 1 -- 172.21.15.81:0/3419653318 shutdown_connections 2023-11-08T16:29:10.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.556+0000 7f26c9dee640 1 -- 172.21.15.81:0/3419653318 >> 172.21.15.81:0/3419653318 conn(0x7f26c40d2e40 msgr2=0x7f26c406b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.556+0000 7f26c9dee640 1 -- 172.21.15.81:0/3419653318 shutdown_connections 2023-11-08T16:29:10.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.556+0000 7f26c9dee640 1 -- 172.21.15.81:0/3419653318 wait complete. 2023-11-08T16:29:10.691 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-08T16:29:10.692 INFO:teuthology.orchestra.run.smithi081.stdout:55834574855 2023-11-08T16:29:10.692 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-08T16:29:10.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.558+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fab680ce6d0 con 0x7fab540730e0 2023-11-08T16:29:10.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.560+0000 7fab64ff9640 1 -- 172.21.15.81:0/3356581783 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fab5c0051f0 con 0x7fab540730e0 2023-11-08T16:29:10.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.561+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7fab540730e0 msgr2=0x7fab54075500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.561+0000 7fab6e8d7640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7fab540730e0 0x7fab54075500 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:10.695 INFO:teuthology.orchestra.run.smithi081.stdout:55834574854 2023-11-08T16:29:10.695 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-08T16:29:10.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.561+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fab5406aec0 msgr2=0x7fab5406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.561+0000 7fab6e8d7640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fab5406aec0 0x7fab5406d380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fab58001050 tx=0x7fab58007040 comp rx=0 tx=0).stop 2023-11-08T16:29:10.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.561+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab680b3110 msgr2=0x7fab680b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:10.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.561+0000 7fab6e8d7640 1 --2- 172.21.15.81:0/3356581783 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab680b3110 0x7fab680b35a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fab680c5810 tx=0x7fab6000a340 comp rx=0 tx=0).stop 2023-11-08T16:29:10.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.561+0000 7fab6ce74640 1 -- 172.21.15.81:0/3356581783 reap_dead start 2023-11-08T16:29:10.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.561+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 shutdown_connections 2023-11-08T16:29:10.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.561+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 >> 172.21.15.81:0/3356581783 conn(0x7fab680cb490 msgr2=0x7fab6806b810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:10.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.561+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 shutdown_connections 2023-11-08T16:29:10.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:10.561+0000 7fab6e8d7640 1 -- 172.21.15.81:0/3356581783 wait complete. 2023-11-08T16:29:10.698 INFO:teuthology.orchestra.run.smithi081.stdout:55834574855 2023-11-08T16:29:10.699 INFO:teuthology.orchestra.run.smithi081.stdout:55834574855 2023-11-08T16:29:10.700 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-08T16:29:10.700 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-08T16:29:10.701 INFO:teuthology.orchestra.run.smithi081.stdout:55834574854 2023-11-08T16:29:10.702 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-08T16:29:10.703 INFO:teuthology.orchestra.run.smithi081.stdout:55834574853 2023-11-08T16:29:10.703 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-08T16:29:10.704 INFO:teuthology.orchestra.run.smithi081.stdout:55834574855 2023-11-08T16:29:10.704 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-08T16:29:11.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.032+0000 7f10d2a56640 1 Processor -- start 2023-11-08T16:29:11.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.033+0000 7f10d2a56640 1 -- start start 2023-11-08T16:29:11.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.035+0000 7f10d2a56640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c40f0170 0x7f10c40f2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.035+0000 7f10d2a56640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c403c040 con 0x7f10c403d4c0 2023-11-08T16:29:11.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.035+0000 7f10d2a56640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10c4058bd0 con 0x7f10c4053a20 2023-11-08T16:29:11.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.035+0000 7f10cbfff640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f10c4053a20 0x7f10c4053e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50260/0 (socket says 172.21.15.81:50260) 2023-11-08T16:29:11.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.035+0000 7f10cbfff640 1 -- 172.21.15.81:0/403618169 learned_addr learned my addr 172.21.15.81:0/403618169 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:11.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.036+0000 7f10d2a56640 1 -- 172.21.15.81:0/403618169 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40387a0 con 0x7f10c40f0170 2023-11-08T16:29:11.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.036+0000 7f10d0ff3640 1 --2- 172.21.15.81:0/403618169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c40f0170 0x7f10c40f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.036+0000 7f10caffd640 1 -- 172.21.15.81:0/403618169 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 151651376 0 0) 0x7f10c403c040 con 0x7f10c403d4c0 2023-11-08T16:29:11.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.037+0000 7f10caffd640 1 -- 172.21.15.81:0/403618169 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10bc003520 con 0x7f10c403d4c0 2023-11-08T16:29:11.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.037+0000 7f10caffd640 1 -- 172.21.15.81:0/403618169 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285903945 0 0) 0x7f10c4058bd0 con 0x7f10c4053a20 2023-11-08T16:29:11.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.037+0000 7f10caffd640 1 -- 172.21.15.81:0/403618169 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10c403c040 con 0x7f10c4053a20 2023-11-08T16:29:11.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.037+0000 7f10d0ff3640 1 -- 172.21.15.81:0/403618169 >> v1:172.21.15.93:6789/0 conn(0x7f10c4053a20 legacy=0x7f10c4053e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.037+0000 7f10d0ff3640 1 -- 172.21.15.81:0/403618169 >> v1:172.21.15.81:6789/0 conn(0x7f10c403d4c0 legacy=0x7f10c4056f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.037+0000 7f10d0ff3640 1 -- 172.21.15.81:0/403618169 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10c40079a0 con 0x7f10c40f0170 2023-11-08T16:29:11.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.039+0000 7f02321a9640 1 Processor -- start 2023-11-08T16:29:11.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.039+0000 7f10d0ff3640 1 --2- 172.21.15.81:0/403618169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c40f0170 0x7f10c40f2560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f10b80097d0 tx=0x7f10b802b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1831e52e86e0044 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.039+0000 7f02321a9640 1 -- start start 2023-11-08T16:29:11.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.040+0000 7f10caffd640 1 -- 172.21.15.81:0/403618169 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8030800 con 0x7f10c40f0170 2023-11-08T16:29:11.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.040+0000 7f10caffd640 1 -- 172.21.15.81:0/403618169 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10b8030960 con 0x7f10c40f0170 2023-11-08T16:29:11.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.040+0000 7f10caffd640 1 -- 172.21.15.81:0/403618169 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10b8030cd0 con 0x7f10c40f0170 2023-11-08T16:29:11.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.041+0000 7f10d2a56640 1 -- 172.21.15.81:0/403618169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c40f0170 msgr2=0x7f10c40f2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.041+0000 7f10d2a56640 1 --2- 172.21.15.81:0/403618169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c40f0170 0x7f10c40f2560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f10b80097d0 tx=0x7f10b802b940 comp rx=0 tx=0).stop 2023-11-08T16:29:11.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.041+0000 7f02321a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f022c0cf590 0x7f022c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.041+0000 7f02321a9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f022c15cca0 0x7f022c0c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.041+0000 7f02321a9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022c160010 con 0x7f022c0c9630 2023-11-08T16:29:11.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.041+0000 7f02321a9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022c0c9aa0 con 0x7f022c15cca0 2023-11-08T16:29:11.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.041+0000 7f02321a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0ccd00 con 0x7f022c0cf590 2023-11-08T16:29:11.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.042+0000 7f10d2a56640 1 -- 172.21.15.81:0/403618169 shutdown_connections 2023-11-08T16:29:11.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.042+0000 7f10d2a56640 1 --2- 172.21.15.81:0/403618169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c40f0170 0x7f10c40f2560 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.042+0000 7f10d2a56640 1 -- 172.21.15.81:0/403618169 >> 172.21.15.81:0/403618169 conn(0x7f10c4058ea0 msgr2=0x7f10c40592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.042+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f022c0cf590 0x7f022c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.042+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f022c0cf590 0x7f022c0cc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54458/0 (socket says 172.21.15.81:54458) 2023-11-08T16:29:11.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.042+0000 7f022b7fe640 1 -- 172.21.15.81:0/779667400 learned_addr learned my addr 172.21.15.81:0/779667400 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.043+0000 7f022affd640 1 --2- 172.21.15.81:0/779667400 >> v2:172.21.15.93:3300/0 conn(0x7f022c15cca0 0x7f022c0c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.043+0000 7f10d2a56640 1 -- 172.21.15.81:0/403618169 shutdown_connections 2023-11-08T16:29:11.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.043+0000 7f10d2a56640 1 -- 172.21.15.81:0/403618169 wait complete. 2023-11-08T16:29:11.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.043+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078261248 0 0) 0x7f022c160010 con 0x7f022c0c9630 2023-11-08T16:29:11.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.043+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0214003520 con 0x7f022c0c9630 2023-11-08T16:29:11.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 <== mon.1 v1:172.21.15.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 370905133 0 0) 0x7f0214003520 con 0x7f022c0c9630 2023-11-08T16:29:11.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f022c0cf590 msgr2=0x7f022c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:11.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f022c0cf590 0x7f022c0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 >> v2:172.21.15.93:3300/0 conn(0x7f022c15cca0 msgr2=0x7f022c0c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:11.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 --2- 172.21.15.81:0/779667400 >> v2:172.21.15.93:3300/0 conn(0x7f022c15cca0 0x7f022c0c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 --> v1:172.21.15.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c0ccfe0 con 0x7f022c0c9630 2023-11-08T16:29:11.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 <== mon.1 v1:172.21.15.93:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f02240019f0 con 0x7f022c0c9630 2023-11-08T16:29:11.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 >> v1:172.21.15.93:6789/0 conn(0x7f022c0c9630 legacy=0x7f022c06b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0214003970 0x7f0214003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f02140047c0 0x7f0214004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0214005180 0x7f021400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0c9aa0 con 0x7f0214005180 2023-11-08T16:29:11.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0ccd00 con 0x7f0214003970 2023-11-08T16:29:11.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.044+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f021400bec0 con 0x7f02140047c0 2023-11-08T16:29:11.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.045+0000 7f022b7fe640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f022c0cf590 0x7f022c0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:11.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.045+0000 7f022affd640 1 --2- 172.21.15.81:0/779667400 >> v2:172.21.15.93:3300/0 conn(0x7f022c15cca0 0x7f022c0c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:11.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.045+0000 7f022b7fe640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0214003970 0x7f0214003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.046+0000 7f10d2a56640 1 Processor -- start 2023-11-08T16:29:11.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.047+0000 7f10d2a56640 1 -- start start 2023-11-08T16:29:11.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.046+0000 7f022b7fe640 1 -- 172.21.15.81:0/779667400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f02140047c0 msgr2=0x7f0214004c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-08T16:29:11.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.046+0000 7f022bfff640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0214005180 0x7f021400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.048+0000 7f10d2a56640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10c403d4c0 0x7f10c40da6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.048+0000 7f10d2a56640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c4053a20 0x7f10c40d9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.048+0000 7f10d2a56640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c40f0170 0x7f10c4180950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.048+0000 7f10d2a56640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40e6d80 con 0x7f10c40f0170 2023-11-08T16:29:11.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.048+0000 7f10d2a56640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40e6170 con 0x7f10c403d4c0 2023-11-08T16:29:11.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.048+0000 7f10d2a56640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10c40e6450 con 0x7f10c4053a20 2023-11-08T16:29:11.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.047+0000 7f022b7fe640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f02140047c0 0x7f0214004c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.047+0000 7f022b7fe640 1 -- 172.21.15.81:0/779667400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0214005180 msgr2=0x7f021400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:11.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.047+0000 7f022b7fe640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0214005180 0x7f021400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.047+0000 7f022b7fe640 1 -- 172.21.15.81:0/779667400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c0ccfe0 con 0x7f0214003970 2023-11-08T16:29:11.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.049+0000 7f022bfff640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0214005180 0x7f021400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:11.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.049+0000 7f10cb7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c4053a20 0x7f10c40d9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.049+0000 7f10cb7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c4053a20 0x7f10c40d9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54482/0 (socket says 172.21.15.81:54482) 2023-11-08T16:29:11.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.049+0000 7f10cb7fe640 1 -- 172.21.15.81:0/160977845 learned_addr learned my addr 172.21.15.81:0/160977845 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.049+0000 7f10d0ff3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c40f0170 0x7f10c4180950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.050+0000 7f10cbfff640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10c403d4c0 0x7f10c40da6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.049+0000 7f022b7fe640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0214003970 0x7f0214003d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f021c009c70 tx=0x7f02140057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbb09ccaa14d07fd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.050+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c02c7e0 con 0x7f0214003970 2023-11-08T16:29:11.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.050+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c02c940 con 0x7f0214003970 2023-11-08T16:29:11.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.050+0000 7f022a7fc640 1 -- 172.21.15.81:0/779667400 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c02ccb0 con 0x7f0214003970 2023-11-08T16:29:11.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.050+0000 7f10cb7fe640 1 -- 172.21.15.81:0/160977845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10c403d4c0 msgr2=0x7f10c40da6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.050+0000 7f10cb7fe640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10c403d4c0 0x7f10c40da6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.050+0000 7f10cb7fe640 1 -- 172.21.15.81:0/160977845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c40f0170 msgr2=0x7f10c4180950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.051+0000 7f10cb7fe640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c40f0170 0x7f10c4180950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.051+0000 7f10cb7fe640 1 -- 172.21.15.81:0/160977845 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b8009400 con 0x7f10c4053a20 2023-11-08T16:29:11.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.051+0000 7f10d0ff3640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c40f0170 0x7f10c4180950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:11.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.051+0000 7f02321a9640 1 -- 172.21.15.81:0/779667400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0214003970 msgr2=0x7f0214003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.051+0000 7f02321a9640 1 --2- 172.21.15.81:0/779667400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0214003970 0x7f0214003d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f021c009c70 tx=0x7f02140057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.051+0000 7f022bfff640 1 -- 172.21.15.81:0/779667400 reap_dead start 2023-11-08T16:29:11.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.051+0000 7f10cb7fe640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c4053a20 0x7f10c40d9e90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c430 tx=0x7f10c000c830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.052+0000 7f02321a9640 1 -- 172.21.15.81:0/779667400 shutdown_connections 2023-11-08T16:29:11.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.052+0000 7f02321a9640 1 -- 172.21.15.81:0/779667400 >> 172.21.15.81:0/779667400 conn(0x7f022c0ceaa0 msgr2=0x7f022c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.052+0000 7f10c8ff9640 1 -- 172.21.15.81:0/160977845 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c00052d0 con 0x7f10c4053a20 2023-11-08T16:29:11.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.052+0000 7f10d2a56640 1 -- 172.21.15.81:0/160977845 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10c40e6730 con 0x7f10c4053a20 2023-11-08T16:29:11.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.052+0000 7f10d2a56640 1 -- 172.21.15.81:0/160977845 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10c40da3d0 con 0x7f10c4053a20 2023-11-08T16:29:11.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.052+0000 7f10c8ff9640 1 -- 172.21.15.81:0/160977845 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10c0005450 con 0x7f10c4053a20 2023-11-08T16:29:11.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.053+0000 7f10c8ff9640 1 -- 172.21.15.81:0/160977845 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10c0003830 con 0x7f10c4053a20 2023-11-08T16:29:11.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.054+0000 7f10d2a56640 1 -- 172.21.15.81:0/160977845 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac005250 con 0x7f10c4053a20 2023-11-08T16:29:11.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.054+0000 7f02321a9640 1 -- 172.21.15.81:0/779667400 shutdown_connections 2023-11-08T16:29:11.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.054+0000 7f02321a9640 1 -- 172.21.15.81:0/779667400 wait complete. 2023-11-08T16:29:11.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.054+0000 7f0b38675640 1 Processor -- start 2023-11-08T16:29:11.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.054+0000 7f0b38675640 1 -- start start 2023-11-08T16:29:11.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.056+0000 7f2761f0f640 1 Processor -- start 2023-11-08T16:29:11.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.056+0000 7f0b38675640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b30159170 0x7f0b300cd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.056+0000 7f2761f0f640 1 -- start start 2023-11-08T16:29:11.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.056+0000 7f10c8ff9640 1 -- 172.21.15.81:0/160977845 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f10c0003990 con 0x7f10c4053a20 2023-11-08T16:29:11.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.056+0000 7f0b38675640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f0b300cd5c0 0x7f0b3006fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.056+0000 7f0b38675640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b3006b2b0 con 0x7f0b3006ff40 2023-11-08T16:29:11.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.056+0000 7f0b38675640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b30159550 con 0x7f0b300cd5c0 2023-11-08T16:29:11.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.056+0000 7f0b38675640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3006b590 con 0x7f0b30159170 2023-11-08T16:29:11.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.057+0000 7f0b35c10640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f0b300cd5c0 0x7f0b3006fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.057+0000 7f0b35c10640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f0b300cd5c0 0x7f0b3006fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:54534/0 (socket says 172.21.15.81:54534) 2023-11-08T16:29:11.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.057+0000 7f0b35c10640 1 -- 172.21.15.81:0/380807953 learned_addr learned my addr 172.21.15.81:0/380807953 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.057+0000 7f0b36411640 1 --2- 172.21.15.81:0/380807953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b30159170 0x7f0b300cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.058+0000 7f02321a9640 1 Processor -- start 2023-11-08T16:29:11.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.058+0000 7fcc17d05640 1 Processor -- start 2023-11-08T16:29:11.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.058+0000 7f02321a9640 1 -- start start 2023-11-08T16:29:11.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.058+0000 7f10c8ff9640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10b006ae70 0x7f10b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.058+0000 7f0b36411640 1 -- 172.21.15.81:0/380807953 >> v2:172.21.15.93:3300/0 conn(0x7f0b300cd5c0 msgr2=0x7f0b3006fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:11.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.058+0000 7f0b36411640 1 --2- 172.21.15.81:0/380807953 >> v2:172.21.15.93:3300/0 conn(0x7f0b300cd5c0 0x7f0b3006fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.058+0000 7f0b36411640 1 -- 172.21.15.81:0/380807953 >> v1:172.21.15.81:6789/0 conn(0x7f0b3006ff40 legacy=0x7f0b3006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.058+0000 7f0b36411640 1 -- 172.21.15.81:0/380807953 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b3006b870 con 0x7f0b30159170 2023-11-08T16:29:11.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.058+0000 7f0b35c10640 1 --2- 172.21.15.81:0/380807953 >> v2:172.21.15.93:3300/0 conn(0x7f0b300cd5c0 0x7f0b3006fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:11.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.059+0000 7f10c8ff9640 1 -- 172.21.15.81:0/160977845 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10c0088ef0 con 0x7f10c4053a20 2023-11-08T16:29:11.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.060+0000 7f10cbfff640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10b006ae70 0x7f10b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.060+0000 7f02321a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0214003970 0x7f022c0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.060+0000 7f0b36411640 1 --2- 172.21.15.81:0/380807953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b30159170 0x7f0b300cd080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009790 tx=0x7f0b2c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d71f14ea96b3786f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.060+0000 7f02321a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f022c148b20 0x7f022c0c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.059+0000 7f2761f0f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f275c15d310 0x7f275c0d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.059+0000 7f2761f0f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d01d0 0x7f275c0b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.059+0000 7f2761f0f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f275c0b6100 con 0x7f275c0d1300 2023-11-08T16:29:11.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.059+0000 7f2761f0f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f275c0b6480 con 0x7f275c15d310 2023-11-08T16:29:11.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.059+0000 7f2761f0f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c0d1b50 con 0x7f275c0d01d0 2023-11-08T16:29:11.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.060+0000 7f275b7fe640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f275c15d310 0x7f275c0d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.060+0000 7f275affd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f275c0d1300 0x7f275c0cfac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49114/0 (socket says 172.21.15.81:49114) 2023-11-08T16:29:11.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.060+0000 7f275affd640 1 -- 172.21.15.81:0/593123143 learned_addr learned my addr 172.21.15.81:0/593123143 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:11.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7f02321a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f022c0c60b0 0x7f022c1fa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0214003970 0x7f022c0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7efd01b39640 1 Processor -- start 2023-11-08T16:29:11.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.058+0000 7fcc17d05640 1 -- start start 2023-11-08T16:29:11.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.060+0000 7fcc17d05640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcc100c9360 0x7fcc100cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.060+0000 7fcc17d05640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc100ce7a0 con 0x7fcc100c9740 2023-11-08T16:29:11.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.060+0000 7fcc17d05640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc100ccd00 con 0x7fcc1015cca0 2023-11-08T16:29:11.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.060+0000 7fcc17d05640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc100ccfe0 con 0x7fcc100c9360 2023-11-08T16:29:11.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7f10cbfff640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10b006ae70 0x7f10b006d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10cc000b60 tx=0x7f10cc068a60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7f275bfff640 1 --2- 172.21.15.81:0/593123143 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d01d0 0x7f275c0b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7f275b7fe640 1 -- 172.21.15.81:0/593123143 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d01d0 msgr2=0x7f275c0b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:11.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7f275b7fe640 1 --2- 172.21.15.81:0/593123143 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d01d0 0x7f275c0b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7f275b7fe640 1 -- 172.21.15.81:0/593123143 >> v1:172.21.15.81:6789/0 conn(0x7f275c0d1300 legacy=0x7f275c0cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0214003970 0x7f022c0c4d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54508/0 (socket says 172.21.15.81:54508) 2023-11-08T16:29:11.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7f275b7fe640 1 -- 172.21.15.81:0/593123143 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f275c0d1e30 con 0x7f275c15d310 2023-11-08T16:29:11.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7efd01b39640 1 -- start start 2023-11-08T16:29:11.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7f022affd640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f022c148b20 0x7f022c0c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7f022b7fe640 1 -- 172.21.15.81:0/341653187 learned_addr learned my addr 172.21.15.81:0/341653187 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.062+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0c6540 con 0x7f022c0c60b0 2023-11-08T16:29:11.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.062+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0b3c90 con 0x7f022c148b20 2023-11-08T16:29:11.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.062+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c1face0 con 0x7f0214003970 2023-11-08T16:29:11.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.061+0000 7fcc15aa1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcc100c9360 0x7fcc100cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.062+0000 7fcc152a0640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fcc1015cca0 0x7fcc1006b4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50270/0 (socket says 172.21.15.81:50270) 2023-11-08T16:29:11.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.062+0000 7f022bfff640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f022c0c60b0 0x7f022c1fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.062+0000 7fcc162a2640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcc100c9740 0x7fcc10159f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49124/0 (socket says 172.21.15.81:49124) 2023-11-08T16:29:11.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.062+0000 7fcc162a2640 1 -- 172.21.15.81:0/290210999 learned_addr learned my addr 172.21.15.81:0/290210999 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:11.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.062+0000 7f022bfff640 1 -- 172.21.15.81:0/341653187 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0214003970 msgr2=0x7f022c0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.062+0000 7f022bfff640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0214003970 0x7f022c0c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7efd01b39640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7efcfc15d310 0x7efcfc0d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7efd01b39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc0b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7efd01b39640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcfc0b6100 con 0x7efcfc0d1300 2023-11-08T16:29:11.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7efd01b39640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcfc0b6480 con 0x7efcfc15d310 2023-11-08T16:29:11.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7efd01b39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc0d1b50 con 0x7efcfc0d01d0 2023-11-08T16:29:11.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7fcc14a9f640 1 -- 172.21.15.81:0/290210999 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028633245 0 0) 0x7fcc100ce7a0 con 0x7fcc100c9740 2023-11-08T16:29:11.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7f022bfff640 1 -- 172.21.15.81:0/341653187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f022c148b20 msgr2=0x7f022c0c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7f022bfff640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f022c148b20 0x7f022c0c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7f022bfff640 1 -- 172.21.15.81:0/341653187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c1fafc0 con 0x7f022c0c60b0 2023-11-08T16:29:11.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7f022affd640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f022c148b20 0x7f022c0c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:11.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7fcc14a9f640 1 -- 172.21.15.81:0/290210999 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc00004630 con 0x7fcc100c9740 2023-11-08T16:29:11.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7f022bfff640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f022c0c60b0 0x7f022c1fa100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f022c0c9a10 tx=0x7f022400deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7efcfbfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc0b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7fcc15aa1640 1 -- 172.21.15.81:0/290210999 >> v1:172.21.15.93:6789/0 conn(0x7fcc1015cca0 legacy=0x7fcc1006b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7efcfbfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc0b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54518/0 (socket says 172.21.15.81:54518) 2023-11-08T16:29:11.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7efcfbfff640 1 -- 172.21.15.81:0/3893865862 learned_addr learned my addr 172.21.15.81:0/3893865862 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7fcc15aa1640 1 -- 172.21.15.81:0/290210999 >> v1:172.21.15.81:6789/0 conn(0x7fcc100c9740 legacy=0x7fcc10159f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7fcc14a9f640 1 -- 172.21.15.81:0/290210999 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 268713099 0 0) 0x7fcc00004630 con 0x7fcc100c9740 2023-11-08T16:29:11.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.063+0000 7fcc15aa1640 1 -- 172.21.15.81:0/290210999 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc100cd2c0 con 0x7fcc100c9360 2023-11-08T16:29:11.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.064+0000 7efcfb7fe640 1 --2- 172.21.15.81:0/3893865862 >> v2:172.21.15.93:3300/0 conn(0x7efcfc15d310 0x7efcfc0d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.064+0000 7f0213fff640 1 -- 172.21.15.81:0/341653187 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224003d80 con 0x7f022c0c60b0 2023-11-08T16:29:11.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.064+0000 7f0213fff640 1 -- 172.21.15.81:0/341653187 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0224005070 con 0x7f022c0c60b0 2023-11-08T16:29:11.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.064+0000 7f0213fff640 1 -- 172.21.15.81:0/341653187 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224005420 con 0x7f022c0c60b0 2023-11-08T16:29:11.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.064+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022c0b1760 con 0x7f022c0c60b0 2023-11-08T16:29:11.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.064+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022c0b1d20 con 0x7f022c0c60b0 2023-11-08T16:29:11.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.064+0000 7f275b7fe640 1 --2- 172.21.15.81:0/593123143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f275c15d310 0x7f275c0d2b70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f274c0097a0 tx=0x7f274c02b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cef1136b2ea2c2b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.064+0000 7efcfbfff640 1 -- 172.21.15.81:0/3893865862 >> v2:172.21.15.93:3300/0 conn(0x7efcfc15d310 msgr2=0x7efcfc0d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:11.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.064+0000 7efcfbfff640 1 --2- 172.21.15.81:0/3893865862 >> v2:172.21.15.93:3300/0 conn(0x7efcfc15d310 0x7efcfc0d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.064+0000 7efcfbfff640 1 -- 172.21.15.81:0/3893865862 >> v1:172.21.15.93:6789/0 conn(0x7efcfc0d1300 legacy=0x7efcfc0cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.064+0000 7efcfbfff640 1 -- 172.21.15.81:0/3893865862 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc0d1e30 con 0x7efcfc0d01d0 2023-11-08T16:29:11.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.065+0000 7f275a7fc640 1 -- 172.21.15.81:0/593123143 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c030800 con 0x7f275c15d310 2023-11-08T16:29:11.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.065+0000 7f275a7fc640 1 -- 172.21.15.81:0/593123143 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f274c030960 con 0x7f275c15d310 2023-11-08T16:29:11.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.065+0000 7f275a7fc640 1 -- 172.21.15.81:0/593123143 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f274c030cf0 con 0x7f275c15d310 2023-11-08T16:29:11.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.065+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022c0b1f40 con 0x7f022c0c60b0 2023-11-08T16:29:11.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.065+0000 7fcc15aa1640 1 --2- 172.21.15.81:0/290210999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc100c9360 0x7fcc100cc670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcc080097a0 tx=0x7fcc0802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544b81d70145ea36 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.066+0000 7efcfbfff640 1 --2- 172.21.15.81:0/3893865862 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc0b68a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009e50 tx=0x7efcf402bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e0d201ac73b3469 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.066+0000 7f2761f0f640 1 -- 172.21.15.81:0/593123143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f275c15d310 msgr2=0x7f275c0d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.066+0000 7f2761f0f640 1 --2- 172.21.15.81:0/593123143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f275c15d310 0x7f275c0d2b70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f274c0097a0 tx=0x7f274c02b940 comp rx=0 tx=0).stop 2023-11-08T16:29:11.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.067+0000 7f2761f0f640 1 -- 172.21.15.81:0/593123143 shutdown_connections 2023-11-08T16:29:11.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.067+0000 7f2761f0f640 1 --2- 172.21.15.81:0/593123143 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d01d0 0x7f275c0b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.067+0000 7f2761f0f640 1 --2- 172.21.15.81:0/593123143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f275c15d310 0x7f275c0d2b70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.067+0000 7f2761f0f640 1 -- 172.21.15.81:0/593123143 >> 172.21.15.81:0/593123143 conn(0x7f275c06aac0 msgr2=0x7f275c06aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.067+0000 7f0213fff640 1 -- 172.21.15.81:0/341653187 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0224005580 con 0x7f022c0c60b0 2023-11-08T16:29:11.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.067+0000 7fcc14a9f640 1 -- 172.21.15.81:0/290210999 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc001670 con 0x7fcc100c9360 2023-11-08T16:29:11.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.067+0000 7fcc14a9f640 1 -- 172.21.15.81:0/290210999 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0802e050 con 0x7fcc100c9360 2023-11-08T16:29:11.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.067+0000 7fcc14a9f640 1 -- 172.21.15.81:0/290210999 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc08030730 con 0x7fcc100c9360 2023-11-08T16:29:11.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.067+0000 7f2761f0f640 1 -- 172.21.15.81:0/593123143 shutdown_connections 2023-11-08T16:29:11.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.067+0000 7f2761f0f640 1 -- 172.21.15.81:0/593123143 wait complete. 2023-11-08T16:29:11.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.068+0000 7f0213fff640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f021806aef0 0x7f021806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.068+0000 7fcc17d05640 1 -- 172.21.15.81:0/290210999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc100c9360 msgr2=0x7fcc100cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.068+0000 7fcc17d05640 1 --2- 172.21.15.81:0/290210999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc100c9360 0x7fcc100cc670 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcc080097a0 tx=0x7fcc0802b790 comp rx=0 tx=0).stop 2023-11-08T16:29:11.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.068+0000 7f0213fff640 1 -- 172.21.15.81:0/341653187 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0224089010 con 0x7f022c0c60b0 2023-11-08T16:29:11.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.068+0000 7fcc17d05640 1 -- 172.21.15.81:0/290210999 shutdown_connections 2023-11-08T16:29:11.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.068+0000 7fcc17d05640 1 --2- 172.21.15.81:0/290210999 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc100c9360 0x7fcc100cc670 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.068+0000 7fcc17d05640 1 -- 172.21.15.81:0/290210999 >> 172.21.15.81:0/290210999 conn(0x7fcc100ceaa0 msgr2=0x7fcc100cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.068+0000 7f022b7fe640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f021806aef0 0x7f021806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.069+0000 7fcc17d05640 1 -- 172.21.15.81:0/290210999 shutdown_connections 2023-11-08T16:29:11.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.069+0000 7fcc17d05640 1 -- 172.21.15.81:0/290210999 wait complete. 2023-11-08T16:29:11.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.069+0000 7f022b7fe640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f021806aef0 0x7f021806d3b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f022c070cc0 tx=0x7f021c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.070+0000 7f2761f0f640 1 Processor -- start 2023-11-08T16:29:11.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.070+0000 7fe33a0ea640 1 Processor -- start 2023-11-08T16:29:11.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.070+0000 7f2761f0f640 1 -- start start 2023-11-08T16:29:11.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.070+0000 7fe33a0ea640 1 -- start start 2023-11-08T16:29:11.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.071+0000 7fe33a0ea640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe32c0e3150 0x7fe32c03ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.071+0000 7fe33a0ea640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c053590 con 0x7fe32c053f70 2023-11-08T16:29:11.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.071+0000 7fe33a0ea640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c053da0 con 0x7fe32c0e3530 2023-11-08T16:29:11.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.071+0000 7fe33a0ea640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32c057fa0 con 0x7fe32c0e3150 2023-11-08T16:29:11.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7fcc17d05640 1 Processor -- start 2023-11-08T16:29:11.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7f0b3540f640 1 -- 172.21.15.81:0/380807953 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c02d040 con 0x7f0b30159170 2023-11-08T16:29:11.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7f0b3540f640 1 -- 172.21.15.81:0/380807953 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b2c031810 con 0x7f0b30159170 2023-11-08T16:29:11.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7f0b3540f640 1 -- 172.21.15.81:0/380807953 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c031ba0 con 0x7f0b30159170 2023-11-08T16:29:11.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7efcfa7fc640 1 -- 172.21.15.81:0/3893865862 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf402cb90 con 0x7efcfc0d01d0 2023-11-08T16:29:11.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7efcfa7fc640 1 -- 172.21.15.81:0/3893865862 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcf402ccf0 con 0x7efcfc0d01d0 2023-11-08T16:29:11.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7f10c8ff9640 1 -- 172.21.15.81:0/160977845 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10c0054180 con 0x7f10c4053a20 2023-11-08T16:29:11.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7efcfa7fc640 1 -- 172.21.15.81:0/3893865862 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4034610 con 0x7efcfc0d01d0 2023-11-08T16:29:11.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7f2761f0f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f275c0d01d0 0x7f275c16ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.073+0000 7f2761f0f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d1300 0x7f275c151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.073+0000 7efd01b39640 1 -- 172.21.15.81:0/3893865862 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 msgr2=0x7efcfc0b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.073+0000 7efd01b39640 1 --2- 172.21.15.81:0/3893865862 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc0b68a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009e50 tx=0x7efcf402bff0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f0b38675640 1 -- 172.21.15.81:0/380807953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b30159170 msgr2=0x7f0b300cd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f0b38675640 1 --2- 172.21.15.81:0/380807953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b30159170 0x7f0b300cd080 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009790 tx=0x7f0b2c02b8b0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7efd01b39640 1 -- 172.21.15.81:0/3893865862 shutdown_connections 2023-11-08T16:29:11.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7efd01b39640 1 --2- 172.21.15.81:0/3893865862 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc0b68a0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7efd01b39640 1 --2- 172.21.15.81:0/3893865862 >> v2:172.21.15.93:3300/0 conn(0x7efcfc15d310 0x7efcfc0d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7efd01b39640 1 -- 172.21.15.81:0/3893865862 >> 172.21.15.81:0/3893865862 conn(0x7efcfc06aac0 msgr2=0x7efcfc06aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.073+0000 7f2761f0f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f275c15d310 0x7f275c151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f2761f0f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c15e790 con 0x7f275c0d01d0 2023-11-08T16:29:11.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7fe3390e8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe32c0e3150 0x7fe32c03ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7fe3398e9640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe32c053f70 0x7fe32c03aa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49140/0 (socket says 172.21.15.81:49140) 2023-11-08T16:29:11.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7fe3398e9640 1 -- 172.21.15.81:0/1004492166 learned_addr learned my addr 172.21.15.81:0/1004492166 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:11.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.073+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231925243 0 0) 0x7fe32c053da0 con 0x7fe32c0e3530 2023-11-08T16:29:11.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe320003520 con 0x7fe32c0e3530 2023-11-08T16:29:11.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.072+0000 7fcc17d05640 1 -- start start 2023-11-08T16:29:11.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f275affd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d1300 0x7f275c151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7fcc17d05640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcc100c9360 0x7fcc100c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7fcc17d05640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc100c9740 0x7fcc100c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7fcc17d05640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc1015cca0 0x7fcc101f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7fcc17d05640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc100c7660 con 0x7fcc100c9740 2023-11-08T16:29:11.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7efe37b61640 1 Processor -- start 2023-11-08T16:29:11.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7fcc17d05640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc100c7910 con 0x7fcc1015cca0 2023-11-08T16:29:11.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7fcc17d05640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc100c7bf0 con 0x7fcc100c9360 2023-11-08T16:29:11.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7efd01b39640 1 -- 172.21.15.81:0/3893865862 shutdown_connections 2023-11-08T16:29:11.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f0b38675640 1 -- 172.21.15.81:0/380807953 shutdown_connections 2023-11-08T16:29:11.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f0b38675640 1 --2- 172.21.15.81:0/380807953 >> v2:172.21.15.93:3300/0 conn(0x7f0b300cd5c0 0x7f0b3006fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f0b38675640 1 --2- 172.21.15.81:0/380807953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b30159170 0x7f0b300cd080 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f0b38675640 1 -- 172.21.15.81:0/380807953 >> 172.21.15.81:0/380807953 conn(0x7f0b300ceaa0 msgr2=0x7f0b300cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f275affd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d1300 0x7f275c151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54520/0 (socket says 172.21.15.81:54520) 2023-11-08T16:29:11.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f275affd640 1 -- 172.21.15.81:0/550157876 learned_addr learned my addr 172.21.15.81:0/550157876 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f275b7fe640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f275c0d01d0 0x7f275c16ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f0b38675640 1 -- 172.21.15.81:0/380807953 shutdown_connections 2023-11-08T16:29:11.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f275c15ea70 con 0x7f275c15d310 2023-11-08T16:29:11.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7efe37b61640 1 -- start start 2023-11-08T16:29:11.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f275c15ed50 con 0x7f275c0d1300 2023-11-08T16:29:11.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.075+0000 7fcc15aa1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcc100c9360 0x7fcc100c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.075+0000 7fcc15aa1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcc100c9360 0x7fcc100c6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54530/0 (socket says 172.21.15.81:54530) 2023-11-08T16:29:11.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.075+0000 7fcc15aa1640 1 -- 172.21.15.81:0/1772668199 learned_addr learned my addr 172.21.15.81:0/1772668199 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.075+0000 7fcc162a2640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc1015cca0 0x7fcc101f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.075+0000 7efe37b61640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe300d2b70 0x7efe300cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.075+0000 7efe37b61640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7efe300d0000 0x7efe300b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.075+0000 7efe37b61640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe300d0470 con 0x7efe3015d310 2023-11-08T16:29:11.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.075+0000 7efe37b61640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe300b6480 con 0x7efe300d0000 2023-11-08T16:29:11.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7efe37b61640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe300b6100 con 0x7efe300d2b70 2023-11-08T16:29:11.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7f275bfff640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f275c15d310 0x7f275c151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.075+0000 7f0b38675640 1 -- 172.21.15.81:0/380807953 wait complete. 2023-11-08T16:29:11.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7fcc152a0640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc100c9740 0x7fcc100c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 <== mon.1 v1:172.21.15.93:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3832594369 0 0) 0x7fe320003520 con 0x7fe32c0e3530 2023-11-08T16:29:11.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 >> v2:172.21.15.81:3300/0 conn(0x7fe32c0e3150 msgr2=0x7fe32c03ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:11.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7fe32bfff640 1 --2- 172.21.15.81:0/1004492166 >> v2:172.21.15.81:3300/0 conn(0x7fe32c0e3150 0x7fe32c03ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 >> v1:172.21.15.81:6789/0 conn(0x7fe32c053f70 legacy=0x7fe32c03aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 --> v1:172.21.15.93:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c03ef40 con 0x7fe32c0e3530 2023-11-08T16:29:11.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 <== mon.1 v1:172.21.15.93:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fe324001670 con 0x7fe32c0e3530 2023-11-08T16:29:11.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7f275affd640 1 -- 172.21.15.81:0/550157876 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f275c15d310 msgr2=0x7f275c151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7f275affd640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f275c15d310 0x7f275c151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7f275affd640 1 -- 172.21.15.81:0/550157876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f275c0d01d0 msgr2=0x7f275c16ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7efe350fc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe300d2b70 0x7efe300cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7efe350fc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe300d2b70 0x7efe300cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54540/0 (socket says 172.21.15.81:54540) 2023-11-08T16:29:11.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7efe350fc640 1 -- 172.21.15.81:0/888591106 learned_addr learned my addr 172.21.15.81:0/888591106 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.076+0000 7efe360fe640 1 --2- 172.21.15.81:0/888591106 >> v2:172.21.15.93:3300/0 conn(0x7efe300d0000 0x7efe300b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7f0213fff640 1 -- 172.21.15.81:0/341653187 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0224054250 con 0x7f022c0c60b0 2023-11-08T16:29:11.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.074+0000 7efd01b39640 1 -- 172.21.15.81:0/3893865862 wait complete. 2023-11-08T16:29:11.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7efd01b39640 1 Processor -- start 2023-11-08T16:29:11.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 >> v1:172.21.15.93:6789/0 conn(0x7fe32c0e3530 legacy=0x7fe32c057890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7efd01b39640 1 -- start start 2023-11-08T16:29:11.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7f275affd640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f275c0d01d0 0x7f275c16ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7fe32bfff640 1 --2- 172.21.15.81:0/1004492166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe320003970 0x7fe320003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7fe32bfff640 1 --2- 172.21.15.81:0/1004492166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe3200047c0 0x7fe320004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7fe32bfff640 1 --2- 172.21.15.81:0/1004492166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 0x7fe32000b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c057fa0 con 0x7fe320005180 2023-11-08T16:29:11.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32000bf90 con 0x7fe320003970 2023-11-08T16:29:11.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32000c150 con 0x7fe3200047c0 2023-11-08T16:29:11.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7efe350fc640 1 -- 172.21.15.81:0/888591106 >> v2:172.21.15.93:3300/0 conn(0x7efe300d0000 msgr2=0x7efe300b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:11.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7efe350fc640 1 --2- 172.21.15.81:0/888591106 >> v2:172.21.15.93:3300/0 conn(0x7efe300d0000 0x7efe300b68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.077+0000 7efe350fc640 1 -- 172.21.15.81:0/888591106 >> v1:172.21.15.81:6789/0 conn(0x7efe3015d310 legacy=0x7efe300d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7fcc15aa1640 1 -- 172.21.15.81:0/1772668199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc1015cca0 msgr2=0x7fcc101f6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7fcc15aa1640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc1015cca0 0x7fcc101f6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7efe350fc640 1 -- 172.21.15.81:0/888591106 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe300d1b50 con 0x7efe300d2b70 2023-11-08T16:29:11.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7f275affd640 1 -- 172.21.15.81:0/550157876 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c009400 con 0x7f275c0d1300 2023-11-08T16:29:11.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7fe3398e9640 1 --2- 172.21.15.81:0/1004492166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 0x7fe32000b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7fcc15aa1640 1 -- 172.21.15.81:0/1772668199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc100c9740 msgr2=0x7fcc100c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7fcc15aa1640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc100c9740 0x7fcc100c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7fcc152a0640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc100c9740 0x7fcc100c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:11.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7fcc15aa1640 1 -- 172.21.15.81:0/1772668199 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc08009400 con 0x7fcc100c9360 2023-11-08T16:29:11.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7fe3388e7640 1 --2- 172.21.15.81:0/1004492166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe3200047c0 0x7fe320004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7fcc162a2640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc1015cca0 0x7fcc101f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:11.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7f275affd640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d1300 0x7f275c151150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f274400c4d0 tx=0x7f274400c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.078+0000 7fe3398e9640 1 -- 172.21.15.81:0/1004492166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe3200047c0 msgr2=0x7fe320004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:11.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7f0b38675640 1 Processor -- start 2023-11-08T16:29:11.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7efd01b39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc15c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7efd01b39640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcfc0d1300 0x7efcfc06e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7efd01b39640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcfc15d310 0x7efcfc1e6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7efd01b39640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc06d3d0 con 0x7efcfc15d310 2023-11-08T16:29:11.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7efd01b39640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc06df50 con 0x7efcfc0d1300 2023-11-08T16:29:11.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7f2753fff640 1 -- 172.21.15.81:0/550157876 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744005ce0 con 0x7f275c0d1300 2023-11-08T16:29:11.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7efd01b39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc15cc00 con 0x7efcfc0d01d0 2023-11-08T16:29:11.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7f2753fff640 1 -- 172.21.15.81:0/550157876 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2744005e40 con 0x7f275c0d1300 2023-11-08T16:29:11.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7f2753fff640 1 -- 172.21.15.81:0/550157876 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744007270 con 0x7f275c0d1300 2023-11-08T16:29:11.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f275c15f030 con 0x7f275c0d1300 2023-11-08T16:29:11.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7efe350fc640 1 --2- 172.21.15.81:0/888591106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe300d2b70 0x7efe300cfac0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009870 tx=0x7efe1c02ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dec87badfd413f4e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.080+0000 7efcfaffd640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcfc0d1300 0x7efcfc06e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.080+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc15c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.080+0000 7efcfb7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc15c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54554/0 (socket says 172.21.15.81:54554) 2023-11-08T16:29:11.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.080+0000 7efcfb7fe640 1 -- 172.21.15.81:0/3335027587 learned_addr learned my addr 172.21.15.81:0/3335027587 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.080+0000 7fcc15aa1640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcc100c9360 0x7fcc100c6be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcc080098c0 tx=0x7fcc080046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.080+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f275c1704a0 con 0x7f275c0d1300 2023-11-08T16:29:11.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.080+0000 7fcc0e7fc640 1 -- 172.21.15.81:0/1772668199 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0803d950 con 0x7fcc100c9360 2023-11-08T16:29:11.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7f0b38675640 1 -- start start 2023-11-08T16:29:11.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.080+0000 7efcfbfff640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcfc15d310 0x7efcfc1e6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.080+0000 7fcc0e7fc640 1 -- 172.21.15.81:0/1772668199 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0803dad0 con 0x7fcc100c9360 2023-11-08T16:29:11.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.080+0000 7fcc0e7fc640 1 -- 172.21.15.81:0/1772668199 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0803de80 con 0x7fcc100c9360 2023-11-08T16:29:11.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.081+0000 7fcc17d05640 1 -- 172.21.15.81:0/1772668199 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc100c7ed0 con 0x7fcc100c9360 2023-11-08T16:29:11.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7fe3398e9640 1 --2- 172.21.15.81:0/1004492166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe3200047c0 0x7fe320004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7fe3398e9640 1 -- 172.21.15.81:0/1004492166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe320003970 msgr2=0x7fe320003d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T16:29:11.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7fe3398e9640 1 --2- 172.21.15.81:0/1004492166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe320003970 0x7fe320003d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7fe3398e9640 1 -- 172.21.15.81:0/1004492166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe320003520 con 0x7fe320005180 2023-11-08T16:29:11.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.079+0000 7fe3388e7640 1 --2- 172.21.15.81:0/1004492166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe3200047c0 0x7fe320004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:11.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.080+0000 7fe3398e9640 1 -- 172.21.15.81:0/1004492166 reap_dead start 2023-11-08T16:29:11.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.081+0000 7fcc17d05640 1 -- 172.21.15.81:0/1772668199 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc100c8490 con 0x7fcc100c9360 2023-11-08T16:29:11.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.081+0000 7efcfb7fe640 1 -- 172.21.15.81:0/3335027587 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcfc0d1300 msgr2=0x7efcfc06e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.081+0000 7efcfb7fe640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcfc0d1300 0x7efcfc06e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.081+0000 7efcfb7fe640 1 -- 172.21.15.81:0/3335027587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcfc15d310 msgr2=0x7efcfc1e6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.082+0000 7f0b38675640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b3006ff40 0x7f0b301ee1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.082+0000 7f0b38675640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b300cd5c0 0x7f0b3014e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.082+0000 7f0b38675640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b30159170 0x7f0b300b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.082+0000 7f0b38675640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b2370 con 0x7f0b30159170 2023-11-08T16:29:11.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.082+0000 7f0b38675640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b1760 con 0x7f0b3006ff40 2023-11-08T16:29:11.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.082+0000 7f0b38675640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b1a40 con 0x7f0b300cd5c0 2023-11-08T16:29:11.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.082+0000 7fe3398e9640 1 --2- 172.21.15.81:0/1004492166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 0x7fe32000b8f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe334063df0 tx=0x7fe3200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9ab2980e1e262d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.082+0000 7efcfb7fe640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcfc15d310 0x7efcfc1e6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.082+0000 7efcfb7fe640 1 -- 172.21.15.81:0/3335027587 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf4009b30 con 0x7efcfc0d01d0 2023-11-08T16:29:11.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.082+0000 7efcfbfff640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcfc15d310 0x7efcfc1e6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:11.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.083+0000 7fcc17d05640 1 -- 172.21.15.81:0/1772668199 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc100743f0 con 0x7fcc100c9360 2023-11-08T16:29:11.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.083+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33406d030 con 0x7fe320005180 2023-11-08T16:29:11.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.083+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3340a6890 con 0x7fe320005180 2023-11-08T16:29:11.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.083+0000 7fe32bfff640 1 -- 172.21.15.81:0/1004492166 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3340a6c60 con 0x7fe320005180 2023-11-08T16:29:11.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.083+0000 7efcfb7fe640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc15c6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9e0 tx=0x7efce400eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.083+0000 7f0b36c12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b30159170 0x7f0b300b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.083+0000 7f0b36c12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b30159170 0x7f0b300b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35584/0 (socket says 172.21.15.81:35584) 2023-11-08T16:29:11.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.083+0000 7f0b36c12640 1 -- 172.21.15.81:0/664620511 learned_addr learned my addr 172.21.15.81:0/664620511 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.083+0000 7f0b36411640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b3006ff40 0x7f0b301ee1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.083+0000 7efcf3fff640 1 -- 172.21.15.81:0/3335027587 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce4009670 con 0x7efcfc0d01d0 2023-11-08T16:29:11.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.083+0000 7efd01b39640 1 -- 172.21.15.81:0/3335027587 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc06d620 con 0x7efcfc0d01d0 2023-11-08T16:29:11.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7efe348fb640 1 -- 172.21.15.81:0/888591106 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c02f830 con 0x7efe300d2b70 2023-11-08T16:29:11.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7efe348fb640 1 -- 172.21.15.81:0/888591106 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1c02f990 con 0x7efe300d2b70 2023-11-08T16:29:11.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7efe348fb640 1 -- 172.21.15.81:0/888591106 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1c02fd20 con 0x7efe300d2b70 2023-11-08T16:29:11.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1004492166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 msgr2=0x7fe32000b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7fe33a0ea640 1 --2- 172.21.15.81:0/1004492166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 0x7fe32000b8f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe334063df0 tx=0x7fe3200057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7f0b35c10640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b300cd5c0 0x7f0b3014e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7efd01b39640 1 -- 172.21.15.81:0/3335027587 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc06dbe0 con 0x7efcfc0d01d0 2023-11-08T16:29:11.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7efcf3fff640 1 -- 172.21.15.81:0/3335027587 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce400c9d0 con 0x7efcfc0d01d0 2023-11-08T16:29:11.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7efcf3fff640 1 -- 172.21.15.81:0/3335027587 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce400cd80 con 0x7efcfc0d01d0 2023-11-08T16:29:11.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7f0b36c12640 1 -- 172.21.15.81:0/664620511 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b300cd5c0 msgr2=0x7f0b3014e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7f0b36c12640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b300cd5c0 0x7f0b3014e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7f0b36c12640 1 -- 172.21.15.81:0/664620511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b3006ff40 msgr2=0x7f0b301ee1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7f0b36c12640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b3006ff40 0x7f0b301ee1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7f0b36c12640 1 -- 172.21.15.81:0/664620511 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b20008550 con 0x7f0b30159170 2023-11-08T16:29:11.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1004492166 shutdown_connections 2023-11-08T16:29:11.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7fe33a0ea640 1 --2- 172.21.15.81:0/1004492166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 0x7fe32000b8f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1004492166 >> 172.21.15.81:0/1004492166 conn(0x7fe32c055a50 msgr2=0x7fe32c055e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7efe37b61640 1 -- 172.21.15.81:0/888591106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe300d2b70 msgr2=0x7efe300cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7efe37b61640 1 --2- 172.21.15.81:0/888591106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe300d2b70 0x7efe300cfac0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe1c009870 tx=0x7efe1c02ba10 comp rx=0 tx=0).stop 2023-11-08T16:29:11.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7f2753fff640 1 -- 172.21.15.81:0/550157876 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27440073d0 con 0x7f275c0d1300 2023-11-08T16:29:11.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.084+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f275c15f550 con 0x7f275c0d1300 2023-11-08T16:29:11.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7f0b36c12640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b30159170 0x7f0b300b2800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800c6b0 tx=0x7f0b2800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7efe37b61640 1 -- 172.21.15.81:0/888591106 shutdown_connections 2023-11-08T16:29:11.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7efe37b61640 1 --2- 172.21.15.81:0/888591106 >> v2:172.21.15.93:3300/0 conn(0x7efe300d0000 0x7efe300b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7efe37b61640 1 --2- 172.21.15.81:0/888591106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe300d2b70 0x7efe300cfac0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7efe37b61640 1 -- 172.21.15.81:0/888591106 >> 172.21.15.81:0/888591106 conn(0x7efe3006aac0 msgr2=0x7efe3006aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7f2753fff640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f273c06ae70 0x7f273c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.086+0000 7f2753fff640 1 -- 172.21.15.81:0/550157876 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2744088b90 con 0x7f275c0d1300 2023-11-08T16:29:11.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.086+0000 7f275b7fe640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f273c06ae70 0x7f273c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1004492166 shutdown_connections 2023-11-08T16:29:11.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1004492166 wait complete. 2023-11-08T16:29:11.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.086+0000 7efd01b39640 1 -- 172.21.15.81:0/3335027587 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce0005250 con 0x7efcfc0d01d0 2023-11-08T16:29:11.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.087+0000 7efe37b61640 1 -- 172.21.15.81:0/888591106 shutdown_connections 2023-11-08T16:29:11.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.087+0000 7f275b7fe640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f273c06ae70 0x7f273c06d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f274c036000 tx=0x7f274c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.087+0000 7efe37b61640 1 -- 172.21.15.81:0/888591106 wait complete. 2023-11-08T16:29:11.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.085+0000 7fcc0e7fc640 1 -- 172.21.15.81:0/1772668199 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc08039040 con 0x7fcc100c9360 2023-11-08T16:29:11.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.087+0000 7fcc0e7fc640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcbf406ae70 0x7fcbf406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.087+0000 7f0b26ffd640 1 -- 172.21.15.81:0/664620511 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28003b70 con 0x7f0b30159170 2023-11-08T16:29:11.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.087+0000 7f0b26ffd640 1 -- 172.21.15.81:0/664620511 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b28003cd0 con 0x7f0b30159170 2023-11-08T16:29:11.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.087+0000 7f0b26ffd640 1 -- 172.21.15.81:0/664620511 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28003b70 con 0x7f0b30159170 2023-11-08T16:29:11.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.087+0000 7fcc0e7fc640 1 -- 172.21.15.81:0/1772668199 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc080abae0 con 0x7fcc100c9360 2023-11-08T16:29:11.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.088+0000 7fcc152a0640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcbf406ae70 0x7fcbf406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.088+0000 7f0b38675640 1 -- 172.21.15.81:0/664620511 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b2c0093a0 con 0x7f0b30159170 2023-11-08T16:29:11.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.089+0000 7f0b38675640 1 -- 172.21.15.81:0/664620511 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b300b2000 con 0x7f0b30159170 2023-11-08T16:29:11.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.088+0000 7efe37b61640 1 Processor -- start 2023-11-08T16:29:11.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.089+0000 7efe37b61640 1 -- start start 2023-11-08T16:29:11.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.089+0000 7fe33a0ea640 1 Processor -- start 2023-11-08T16:29:11.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.089+0000 7fcc152a0640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcbf406ae70 0x7fcbf406d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc100b4bb0 tx=0x7fcc0400a040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.089+0000 7fe33a0ea640 1 -- start start 2023-11-08T16:29:11.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.090+0000 7efe37b61640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe300d0000 0x7efe30158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.090+0000 7efe37b61640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe300d2b70 0x7efe300cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.090+0000 7efe37b61640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe3015d310 0x7efe301562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.090+0000 7efe37b61640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe30144660 con 0x7efe300d2b70 2023-11-08T16:29:11.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.090+0000 7efe37b61640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe30144940 con 0x7efe300d0000 2023-11-08T16:29:11.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.090+0000 7efe37b61640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe30144c20 con 0x7efe3015d310 2023-11-08T16:29:11.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.090+0000 7f0b38675640 1 -- 172.21.15.81:0/664620511 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b300b2520 con 0x7f0b30159170 2023-11-08T16:29:11.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.090+0000 7efe358fd640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe300d0000 0x7efe30158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.090+0000 7efe360fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe3015d310 0x7efe301562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7efe358fd640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe300d0000 0x7efe30158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:54624/0 (socket says 172.21.15.81:54624) 2023-11-08T16:29:11.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7efe358fd640 1 -- 172.21.15.81:0/891504198 learned_addr learned my addr 172.21.15.81:0/891504198 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7efe350fc640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe300d2b70 0x7efe300cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7efe360fe640 1 -- 172.21.15.81:0/891504198 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe300d0000 msgr2=0x7efe30158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7efe360fe640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe300d0000 0x7efe30158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7efe360fe640 1 -- 172.21.15.81:0/891504198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe300d2b70 msgr2=0x7efe300cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7efe360fe640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe300d2b70 0x7efe300cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7efe360fe640 1 -- 172.21.15.81:0/891504198 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1c0094d0 con 0x7efe3015d310 2023-11-08T16:29:11.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7fe33a0ea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 0x7fe32c0e6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7fe33a0ea640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c0e43e0 0x7fe32c052260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7fe33a0ea640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe32c0527a0 0x7fe32c16f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7fe33a0ea640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c1801a0 con 0x7fe320005180 2023-11-08T16:29:11.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7fe33a0ea640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c051dd0 con 0x7fe32c0e43e0 2023-11-08T16:29:11.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7fe33a0ea640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c0511e0 con 0x7fe32c0527a0 2023-11-08T16:29:11.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.091+0000 7efe350fc640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe300d2b70 0x7efe300cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:11.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.092+0000 7efe360fe640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe3015d310 0x7efe301562a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efe2400eba0 tx=0x7efe2400c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.092+0000 7fe3388e7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c0e43e0 0x7fe32c052260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.092+0000 7fe3390e8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 0x7fe32c0e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.092+0000 7fe3390e8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 0x7fe32c0e6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35598/0 (socket says 172.21.15.81:35598) 2023-11-08T16:29:11.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.092+0000 7fe3388e7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c0e43e0 0x7fe32c052260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:54630/0 (socket says 172.21.15.81:54630) 2023-11-08T16:29:11.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.092+0000 7fe3388e7640 1 -- 172.21.15.81:0/1767904693 learned_addr learned my addr 172.21.15.81:0/1767904693 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:11.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.092+0000 7f0b26ffd640 1 -- 172.21.15.81:0/664620511 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b280050d0 con 0x7f0b30159170 2023-11-08T16:29:11.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.092+0000 7fe3398e9640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe32c0527a0 0x7fe32c16f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.093+0000 7fe3398e9640 1 -- 172.21.15.81:0/1767904693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c0e43e0 msgr2=0x7fe32c052260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.093+0000 7fe3398e9640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c0e43e0 0x7fe32c052260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.093+0000 7fe3398e9640 1 -- 172.21.15.81:0/1767904693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 msgr2=0x7fe32c0e6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.093+0000 7fe3398e9640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 0x7fe32c0e6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.093+0000 7fe3398e9640 1 -- 172.21.15.81:0/1767904693 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c0514a0 con 0x7fe32c0527a0 2023-11-08T16:29:11.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.093+0000 7fe3390e8640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 0x7fe32c0e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:11.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.093+0000 7fe3398e9640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe32c0527a0 0x7fe32c16f3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe33402cce0 tx=0x7fe334068ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.093+0000 7f0b26ffd640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1406aed0 0x7f0b1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.093+0000 7f0b26ffd640 1 -- 172.21.15.81:0/664620511 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b28089800 con 0x7f0b30159170 2023-11-08T16:29:11.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.094+0000 7f0b36411640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1406aed0 0x7f0b1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.094+0000 7f0b36411640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1406aed0 0x7f0b1406d390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c036000 tx=0x7f0b2c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.100+0000 7f0b26ffd640 1 -- 172.21.15.81:0/664620511 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b28054940 con 0x7f0b30159170 2023-11-08T16:29:11.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.100+0000 7efe2e7fc640 1 -- 172.21.15.81:0/891504198 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400cd20 con 0x7efe3015d310 2023-11-08T16:29:11.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.100+0000 7efe37b61640 1 -- 172.21.15.81:0/891504198 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe30144f00 con 0x7efe3015d310 2023-11-08T16:29:11.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.100+0000 7fcc0e7fc640 1 -- 172.21.15.81:0/1772668199 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcc08076b10 con 0x7fcc100c9360 2023-11-08T16:29:11.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.100+0000 7efe37b61640 1 -- 172.21.15.81:0/891504198 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe301441b0 con 0x7efe3015d310 2023-11-08T16:29:11.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.101+0000 7fe329ffb640 1 -- 172.21.15.81:0/1767904693 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3340bf070 con 0x7fe32c0527a0 2023-11-08T16:29:11.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.101+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1767904693 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c051780 con 0x7fe32c0527a0 2023-11-08T16:29:11.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.102+0000 7efe37b61640 1 -- 172.21.15.81:0/891504198 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe300743f0 con 0x7efe3015d310 2023-11-08T16:29:11.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.101+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1767904693 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c0d8df0 con 0x7fe32c0527a0 2023-11-08T16:29:11.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.102+0000 7efcf3fff640 1 -- 172.21.15.81:0/3335027587 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efce4013470 con 0x7efcfc0d01d0 2023-11-08T16:29:11.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.104+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1767904693 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c008480 con 0x7fe32c0527a0 2023-11-08T16:29:11.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.104+0000 7efcf3fff640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcdc06ae70 0x7efcdc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.104+0000 7efcfaffd640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcdc06ae70 0x7efcdc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.105+0000 7efcfaffd640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcdc06ae70 0x7efcdc06d330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0c64a0 tx=0x7efcec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.113+0000 7fe329ffb640 1 -- 172.21.15.81:0/1767904693 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3340710c0 con 0x7fe32c0527a0 2023-11-08T16:29:11.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.114+0000 7fe329ffb640 1 -- 172.21.15.81:0/1767904693 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33406d040 con 0x7fe32c0527a0 2023-11-08T16:29:11.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.113+0000 7efcf3fff640 1 -- 172.21.15.81:0/3335027587 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efce4058d90 con 0x7efcfc0d01d0 2023-11-08T16:29:11.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.113+0000 7efe2e7fc640 1 -- 172.21.15.81:0/891504198 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2400f040 con 0x7efe3015d310 2023-11-08T16:29:11.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.114+0000 7efe2e7fc640 1 -- 172.21.15.81:0/891504198 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe24014680 con 0x7efe3015d310 2023-11-08T16:29:11.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.114+0000 7f2753fff640 1 -- 172.21.15.81:0/550157876 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f275c15f550 con 0x7f275c0d1300 2023-11-08T16:29:11.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.125+0000 7efcf3fff640 1 -- 172.21.15.81:0/3335027587 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efce40550d0 con 0x7efcfc0d01d0 2023-11-08T16:29:11.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.128+0000 7efe2e7fc640 1 -- 172.21.15.81:0/891504198 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efe240147e0 con 0x7efe3015d310 2023-11-08T16:29:11.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.130+0000 7efe2e7fc640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efe1406ae70 0x7efe1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.130+0000 7efe2e7fc640 1 -- 172.21.15.81:0/891504198 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe2408a8a0 con 0x7efe3015d310 2023-11-08T16:29:11.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.128+0000 7fe329ffb640 1 -- 172.21.15.81:0/1767904693 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe3340aeb60 con 0x7fe32c0527a0 2023-11-08T16:29:11.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.130+0000 7fe329ffb640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe31806ae70 0x7fe31806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:11.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.130+0000 7fe329ffb640 1 -- 172.21.15.81:0/1767904693 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe334124c60 con 0x7fe32c0527a0 2023-11-08T16:29:11.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.130+0000 7fe3390e8640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe31806ae70 0x7fe31806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.131+0000 7efe358fd640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efe1406ae70 0x7efe1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:11.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.132+0000 7fe3390e8640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe31806ae70 0x7fe31806d330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe330009eb0 tx=0x7fe330009190 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.133+0000 7efe358fd640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efe1406ae70 0x7efe1406d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efe28007fd0 tx=0x7efe28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:11.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.138+0000 7efe2e7fc640 1 -- 172.21.15.81:0/891504198 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efe240558a0 con 0x7efe3015d310 2023-11-08T16:29:11.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.143+0000 7fe329ffb640 1 -- 172.21.15.81:0/1767904693 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe3340efe30 con 0x7fe32c0527a0 2023-11-08T16:29:11.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.441+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f022c0cd570 con 0x7f022c0c60b0 2023-11-08T16:29:11.445 INFO:teuthology.orchestra.run.smithi081.stdout:55834574855 2023-11-08T16:29:11.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.442+0000 7f0213fff640 1 -- 172.21.15.81:0/341653187 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0224018020 con 0x7f022c0c60b0 2023-11-08T16:29:11.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.445+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f021806aef0 msgr2=0x7f021806d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.445+0000 7f02321a9640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f021806aef0 0x7f021806d3b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f022c070cc0 tx=0x7f021c030000 comp rx=0 tx=0).stop 2023-11-08T16:29:11.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.445+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f022c0c60b0 msgr2=0x7f022c1fa100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.445+0000 7f02321a9640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f022c0c60b0 0x7f022c1fa100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f022c0c9a10 tx=0x7f022400deb0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.446+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 shutdown_connections 2023-11-08T16:29:11.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.446+0000 7f02321a9640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f021806aef0 0x7f021806d3b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.446+0000 7f02321a9640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f022c0c60b0 0x7f022c1fa100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.446+0000 7f02321a9640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f022c148b20 0x7f022c0c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.446+0000 7f02321a9640 1 --2- 172.21.15.81:0/341653187 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0214003970 0x7f022c0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.446+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 >> 172.21.15.81:0/341653187 conn(0x7f022c0ceaa0 msgr2=0x7f022c06bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.447+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 shutdown_connections 2023-11-08T16:29:11.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.447+0000 7f02321a9640 1 -- 172.21.15.81:0/341653187 wait complete. 2023-11-08T16:29:11.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.471+0000 7f10d2a56640 1 -- 172.21.15.81:0/160977845 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f10ac0057d0 con 0x7f10c4053a20 2023-11-08T16:29:11.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.472+0000 7f10c8ff9640 1 -- 172.21.15.81:0/160977845 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f10c00577a0 con 0x7f10c4053a20 2023-11-08T16:29:11.476 INFO:teuthology.orchestra.run.smithi081.stdout:55834574853 2023-11-08T16:29:11.477 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.2 2023-11-08T16:29:11.477 DEBUG:teuthology.parallel:result is None 2023-11-08T16:29:11.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 -- 172.21.15.81:0/160977845 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10b006ae70 msgr2=0x7f10b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10b006ae70 0x7f10b006d330 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f10cc000b60 tx=0x7f10cc068a60 comp rx=0 tx=0).stop 2023-11-08T16:29:11.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 -- 172.21.15.81:0/160977845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c4053a20 msgr2=0x7f10c40d9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c4053a20 0x7f10c40d9e90 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f10c000c430 tx=0x7f10c000c830 comp rx=0 tx=0).stop 2023-11-08T16:29:11.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 -- 172.21.15.81:0/160977845 shutdown_connections 2023-11-08T16:29:11.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10b006ae70 0x7f10b006d330 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10c40f0170 0x7f10c4180950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10c4053a20 0x7f10c40d9e90 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 --2- 172.21.15.81:0/160977845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10c403d4c0 0x7f10c40da6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 -- 172.21.15.81:0/160977845 >> 172.21.15.81:0/160977845 conn(0x7f10c4058ea0 msgr2=0x7f10c4058250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 -- 172.21.15.81:0/160977845 shutdown_connections 2023-11-08T16:29:11.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.477+0000 7f10d2a56640 1 -- 172.21.15.81:0/160977845 wait complete. 2023-11-08T16:29:11.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2023-11-08T16:29:11.505 DEBUG:teuthology.parallel:result is None 2023-11-08T16:29:11.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.508+0000 7f0b38675640 1 -- 172.21.15.81:0/664620511 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f0b300b2520 con 0x7f0b30159170 2023-11-08T16:29:11.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.508+0000 7f0b26ffd640 1 -- 172.21.15.81:0/664620511 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0b300b2520 con 0x7f0b30159170 2023-11-08T16:29:11.510 INFO:teuthology.orchestra.run.smithi081.stdout:55834574854 2023-11-08T16:29:11.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.510+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1767904693 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fe32c005380 con 0x7fe32c0527a0 2023-11-08T16:29:11.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.511+0000 7fe329ffb640 1 -- 172.21.15.81:0/1767904693 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe3340b3020 con 0x7fe32c0527a0 2023-11-08T16:29:11.513 INFO:teuthology.orchestra.run.smithi081.stdout:55834574854 2023-11-08T16:29:11.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.512+0000 7f0b38675640 1 -- 172.21.15.81:0/664620511 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1406aed0 msgr2=0x7f0b1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.512+0000 7f0b38675640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1406aed0 0x7f0b1406d390 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c036000 tx=0x7f0b2c03c040 comp rx=0 tx=0).stop 2023-11-08T16:29:11.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.512+0000 7f0b38675640 1 -- 172.21.15.81:0/664620511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b30159170 msgr2=0x7f0b300b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.512+0000 7f0b38675640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b30159170 0x7f0b300b2800 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b2800c6b0 tx=0x7f0b2800cb70 comp rx=0 tx=0).stop 2023-11-08T16:29:11.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.512+0000 7f0b38675640 1 -- 172.21.15.81:0/664620511 shutdown_connections 2023-11-08T16:29:11.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.512+0000 7f0b38675640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1406aed0 0x7f0b1406d390 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.512+0000 7f0b38675640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b30159170 0x7f0b300b2800 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.512+0000 7f0b38675640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b300cd5c0 0x7f0b3014e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.512+0000 7f0b38675640 1 --2- 172.21.15.81:0/664620511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b3006ff40 0x7f0b301ee1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.512+0000 7f0b38675640 1 -- 172.21.15.81:0/664620511 >> 172.21.15.81:0/664620511 conn(0x7f0b300ceaa0 msgr2=0x7f0b301596d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.512+0000 7f0b38675640 1 -- 172.21.15.81:0/664620511 shutdown_connections 2023-11-08T16:29:11.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.513+0000 7f0b38675640 1 -- 172.21.15.81:0/664620511 wait complete. 2023-11-08T16:29:11.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1767904693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe31806ae70 msgr2=0x7fe31806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe31806ae70 0x7fe31806d330 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe330009eb0 tx=0x7fe330009190 comp rx=0 tx=0).stop 2023-11-08T16:29:11.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1767904693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe32c0527a0 msgr2=0x7fe32c16f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe32c0527a0 0x7fe32c16f3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe33402cce0 tx=0x7fe334068ba0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1767904693 shutdown_connections 2023-11-08T16:29:11.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe31806ae70 0x7fe31806d330 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe32c0527a0 0x7fe32c16f3c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe32c0e43e0 0x7fe32c052260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 --2- 172.21.15.81:0/1767904693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe320005180 0x7fe32c0e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1767904693 >> 172.21.15.81:0/1767904693 conn(0x7fe32c055a50 msgr2=0x7fe32c03de70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1767904693 shutdown_connections 2023-11-08T16:29:11.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.514+0000 7fe33a0ea640 1 -- 172.21.15.81:0/1767904693 wait complete. 2023-11-08T16:29:11.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.518+0000 7fcc17d05640 1 -- 172.21.15.81:0/1772668199 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fcc10156840 con 0x7fcc100c9360 2023-11-08T16:29:11.521 INFO:teuthology.orchestra.run.smithi081.stdout:55834574855 2023-11-08T16:29:11.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.519+0000 7fcc0e7fc640 1 -- 172.21.15.81:0/1772668199 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fcc08037020 con 0x7fcc100c9360 2023-11-08T16:29:11.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.521+0000 7efe37b61640 1 -- 172.21.15.81:0/891504198 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7efe300cda60 con 0x7efe3015d310 2023-11-08T16:29:11.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.522+0000 7efe2e7fc640 1 -- 172.21.15.81:0/891504198 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efe24058ec0 con 0x7efe3015d310 2023-11-08T16:29:11.524 INFO:teuthology.orchestra.run.smithi081.stdout:55834574852 2023-11-08T16:29:11.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 -- 172.21.15.81:0/1772668199 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcbf406ae70 msgr2=0x7fcbf406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcbf406ae70 0x7fcbf406d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcc100b4bb0 tx=0x7fcc0400a040 comp rx=0 tx=0).stop 2023-11-08T16:29:11.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 -- 172.21.15.81:0/1772668199 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcc100c9360 msgr2=0x7fcc100c6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcc100c9360 0x7fcc100c6be0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fcc080098c0 tx=0x7fcc080046c0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 -- 172.21.15.81:0/1772668199 shutdown_connections 2023-11-08T16:29:11.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcbf406ae70 0x7fcbf406d330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcc1015cca0 0x7fcc101f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcc100c9740 0x7fcc100c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 --2- 172.21.15.81:0/1772668199 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcc100c9360 0x7fcc100c6be0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 -- 172.21.15.81:0/1772668199 >> 172.21.15.81:0/1772668199 conn(0x7fcc100ceaa0 msgr2=0x7fcc10159fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 -- 172.21.15.81:0/1772668199 shutdown_connections 2023-11-08T16:29:11.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.523+0000 7fcc17d05640 1 -- 172.21.15.81:0/1772668199 wait complete. 2023-11-08T16:29:11.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 -- 172.21.15.81:0/891504198 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efe1406ae70 msgr2=0x7efe1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efe1406ae70 0x7efe1406d330 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efe28007fd0 tx=0x7efe28009040 comp rx=0 tx=0).stop 2023-11-08T16:29:11.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 -- 172.21.15.81:0/891504198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe3015d310 msgr2=0x7efe301562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe3015d310 0x7efe301562a0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efe2400eba0 tx=0x7efe2400c490 comp rx=0 tx=0).stop 2023-11-08T16:29:11.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 -- 172.21.15.81:0/891504198 shutdown_connections 2023-11-08T16:29:11.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efe1406ae70 0x7efe1406d330 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe3015d310 0x7efe301562a0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe300d2b70 0x7efe300cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 --2- 172.21.15.81:0/891504198 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe300d0000 0x7efe30158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 -- 172.21.15.81:0/891504198 >> 172.21.15.81:0/891504198 conn(0x7efe3006aac0 msgr2=0x7efe30155b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 -- 172.21.15.81:0/891504198 shutdown_connections 2023-11-08T16:29:11.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.526+0000 7efe37b61640 1 -- 172.21.15.81:0/891504198 wait complete. 2023-11-08T16:29:11.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.531+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f275c15f550 con 0x7f275c0d1300 2023-11-08T16:29:11.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.532+0000 7f2753fff640 1 -- 172.21.15.81:0/550157876 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f275c15f550 con 0x7f275c0d1300 2023-11-08T16:29:11.534 INFO:teuthology.orchestra.run.smithi081.stdout:55834574854 2023-11-08T16:29:11.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.535+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f273c06ae70 msgr2=0x7f273c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.535+0000 7f2761f0f640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f273c06ae70 0x7f273c06d330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f274c036000 tx=0x7f274c008030 comp rx=0 tx=0).stop 2023-11-08T16:29:11.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.535+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d1300 msgr2=0x7f275c151150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.535+0000 7f2761f0f640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d1300 0x7f275c151150 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f274400c4d0 tx=0x7f274400c990 comp rx=0 tx=0).stop 2023-11-08T16:29:11.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.536+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 shutdown_connections 2023-11-08T16:29:11.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.536+0000 7f2761f0f640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f273c06ae70 0x7f273c06d330 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.536+0000 7f2761f0f640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f275c15d310 0x7f275c151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.536+0000 7f2761f0f640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f275c0d1300 0x7f275c151150 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.536+0000 7f2761f0f640 1 --2- 172.21.15.81:0/550157876 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f275c0d01d0 0x7f275c16ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.536+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 >> 172.21.15.81:0/550157876 conn(0x7f275c06aac0 msgr2=0x7f275c0cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.536+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 shutdown_connections 2023-11-08T16:29:11.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.536+0000 7f2761f0f640 1 -- 172.21.15.81:0/550157876 wait complete. 2023-11-08T16:29:11.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.536+0000 7efd01b39640 1 -- 172.21.15.81:0/3335027587 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7efce00057d0 con 0x7efcfc0d01d0 2023-11-08T16:29:11.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.537+0000 7efcf3fff640 1 -- 172.21.15.81:0/3335027587 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efce4017090 con 0x7efcfc0d01d0 2023-11-08T16:29:11.542 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.6 2023-11-08T16:29:11.543 INFO:teuthology.orchestra.run.smithi081.stdout:55834574855 2023-11-08T16:29:11.544 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-11-08T16:29:11.545 DEBUG:teuthology.parallel:result is None 2023-11-08T16:29:11.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 -- 172.21.15.81:0/3335027587 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcdc06ae70 msgr2=0x7efcdc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcdc06ae70 0x7efcdc06d330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efcfc0c64a0 tx=0x7efcec009040 comp rx=0 tx=0).stop 2023-11-08T16:29:11.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 -- 172.21.15.81:0/3335027587 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 msgr2=0x7efcfc15c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:11.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc15c6c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efce400e9e0 tx=0x7efce400eea0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 -- 172.21.15.81:0/3335027587 shutdown_connections 2023-11-08T16:29:11.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcdc06ae70 0x7efcdc06d330 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efcfc15d310 0x7efcfc1e6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcfc0d1300 0x7efcfc06e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 --2- 172.21.15.81:0/3335027587 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efcfc0d01d0 0x7efcfc15c6c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:11.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 -- 172.21.15.81:0/3335027587 >> 172.21.15.81:0/3335027587 conn(0x7efcfc06aac0 msgr2=0x7efcfc0cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:11.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 -- 172.21.15.81:0/3335027587 shutdown_connections 2023-11-08T16:29:11.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:11.541+0000 7efd01b39640 1 -- 172.21.15.81:0/3335027587 wait complete. 2023-11-08T16:29:11.550 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2023-11-08T16:29:11.551 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-11-08T16:29:11.551 DEBUG:teuthology.parallel:result is None 2023-11-08T16:29:11.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2023-11-08T16:29:11.556 DEBUG:teuthology.parallel:result is None 2023-11-08T16:29:11.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.5 2023-11-08T16:29:11.559 DEBUG:teuthology.parallel:result is None 2023-11-08T16:29:12.544 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-08T16:29:12.551 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-08T16:29:12.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.744+0000 7fbcf658c640 1 Processor -- start 2023-11-08T16:29:12.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.744+0000 7fbcf658c640 1 -- start start 2023-11-08T16:29:12.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.745+0000 7fedc6a7e640 1 Processor -- start 2023-11-08T16:29:12.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.745+0000 7fedc6a7e640 1 -- start start 2023-11-08T16:29:12.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.746+0000 7fbcf658c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbcf00b4f80 0x7fbcf00b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.746+0000 7fbcf658c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcf0160d40 0x7fbcf0161170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.746+0000 7fbcf658c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcf00b6540 con 0x7fbcf015cfe0 2023-11-08T16:29:12.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.746+0000 7fbcf658c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcf0161ab0 con 0x7fbcf00b4f80 2023-11-08T16:29:12.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.746+0000 7fbcf658c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf00d1060 con 0x7fbcf0160d40 2023-11-08T16:29:12.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.746+0000 7fedc6a7e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc015cfe0 0x7fedc00b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.746+0000 7fedc6a7e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fedc00b4870 0x7fedc00b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.746+0000 7fedc6a7e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc00b5210 con 0x7fedc0160d40 2023-11-08T16:29:12.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.746+0000 7fbcf5d8b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcf0160d40 0x7fbcf0161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:12.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.746+0000 7fedc6a7e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc01618c0 con 0x7fedc00b4870 2023-11-08T16:29:12.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.747+0000 7fedc6a7e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00b6540 con 0x7fedc015cfe0 2023-11-08T16:29:12.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.747+0000 7fbcf5d8b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcf0160d40 0x7fbcf0161170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54588/0 (socket says 172.21.15.81:54588) 2023-11-08T16:29:12.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.747+0000 7fbcf5d8b640 1 -- 172.21.15.81:0/3516251819 learned_addr learned my addr 172.21.15.81:0/3516251819 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:12.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.747+0000 7fbcf558a640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fbcf015cfe0 0x7fbcf00b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50294/0 (socket says 172.21.15.81:50294) 2023-11-08T16:29:12.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.747+0000 7fbcf4d89640 1 --2- 172.21.15.81:0/3516251819 >> v2:172.21.15.93:3300/0 conn(0x7fbcf00b4f80 0x7fbcf00b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:12.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.747+0000 7fedc627d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fedc0160d40 0x7fedc01611b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49160/0 (socket says 172.21.15.81:49160) 2023-11-08T16:29:12.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.747+0000 7fedc627d640 1 -- 172.21.15.81:0/341340273 learned_addr learned my addr 172.21.15.81:0/341340273 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:12.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.747+0000 7fedc527b640 1 --2- 172.21.15.81:0/341340273 >> v2:172.21.15.93:3300/0 conn(0x7fedc00b4870 0x7fedc00b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:12.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.747+0000 7fedc5a7c640 1 --2- 172.21.15.81:0/341340273 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc015cfe0 0x7fedc00b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:12.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.747+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203742546 0 0) 0x7fedc00b5210 con 0x7fedc0160d40 2023-11-08T16:29:12.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.747+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedac003520 con 0x7fedc0160d40 2023-11-08T16:29:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fbcf5d8b640 1 -- 172.21.15.81:0/3516251819 >> v2:172.21.15.93:3300/0 conn(0x7fbcf00b4f80 msgr2=0x7fbcf00b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fbcf5d8b640 1 --2- 172.21.15.81:0/3516251819 >> v2:172.21.15.93:3300/0 conn(0x7fbcf00b4f80 0x7fbcf00b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fbcf5d8b640 1 -- 172.21.15.81:0/3516251819 >> v1:172.21.15.93:6789/0 conn(0x7fbcf015cfe0 legacy=0x7fbcf00b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fbce7fff640 1 -- 172.21.15.81:0/3516251819 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554844015 0 0) 0x7fbcf00b6540 con 0x7fbcf015cfe0 2023-11-08T16:29:12.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fbcf5d8b640 1 -- 172.21.15.81:0/3516251819 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcf00d1340 con 0x7fbcf0160d40 2023-11-08T16:29:12.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3846499488 0 0) 0x7fedac003520 con 0x7fedc0160d40 2023-11-08T16:29:12.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc015cfe0 msgr2=0x7fedc00b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:12.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 --2- 172.21.15.81:0/341340273 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc015cfe0 0x7fedc00b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 >> v2:172.21.15.93:3300/0 conn(0x7fedc00b4870 msgr2=0x7fedc00b4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:12.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 --2- 172.21.15.81:0/341340273 >> v2:172.21.15.93:3300/0 conn(0x7fedc00b4870 0x7fedc00b4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc0161ab0 con 0x7fedc0160d40 2023-11-08T16:29:12.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fedb80019f0 con 0x7fedc0160d40 2023-11-08T16:29:12.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc527b640 1 --2- 172.21.15.81:0/341340273 >> v2:172.21.15.93:3300/0 conn(0x7fedc00b4870 0x7fedc00b4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:29:12.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 >> v1:172.21.15.81:6789/0 conn(0x7fedc0160d40 legacy=0x7fedc01611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 --2- 172.21.15.81:0/341340273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedac003970 0x7fedac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 --2- 172.21.15.81:0/341340273 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc015cfe0 0x7fedac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 --2- 172.21.15.81:0/341340273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedac004d00 0x7fedac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00b6540 con 0x7fedac003970 2023-11-08T16:29:12.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac00b980 con 0x7fedac004d00 2023-11-08T16:29:12.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedac00bac0 con 0x7fedc015cfe0 2023-11-08T16:29:12.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc5a7c640 1 --2- 172.21.15.81:0/341340273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedac003970 0x7fedac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:12.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc627d640 1 --2- 172.21.15.81:0/341340273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedac004d00 0x7fedac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:12.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc5a7c640 1 -- 172.21.15.81:0/341340273 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc015cfe0 msgr2=0x7fedac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:12.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc5a7c640 1 --2- 172.21.15.81:0/341340273 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc015cfe0 0x7fedac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc5a7c640 1 -- 172.21.15.81:0/341340273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedac004d00 msgr2=0x7fedac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:12.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc5a7c640 1 --2- 172.21.15.81:0/341340273 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedac004d00 0x7fedac005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.748+0000 7fedc5a7c640 1 -- 172.21.15.81:0/341340273 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedac003520 con 0x7fedac003970 2023-11-08T16:29:12.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.749+0000 7fbcf5d8b640 1 --2- 172.21.15.81:0/3516251819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcf0160d40 0x7fbcf0161170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009e50 tx=0x7fbce802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd1f6f338100f2b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:12.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.749+0000 7fbce7fff640 1 -- 172.21.15.81:0/3516251819 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce802cb00 con 0x7fbcf0160d40 2023-11-08T16:29:12.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.749+0000 7fedc5a7c640 1 --2- 172.21.15.81:0/341340273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedac003970 0x7fedac003d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009ac0 tx=0x7fedac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec3372a29a9c787 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fbce7fff640 1 -- 172.21.15.81:0/3516251819 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbce802cc60 con 0x7fbcf0160d40 2023-11-08T16:29:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fbce7fff640 1 -- 172.21.15.81:0/3516251819 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbce80343f0 con 0x7fbcf0160d40 2023-11-08T16:29:12.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc02c710 con 0x7fedac003970 2023-11-08T16:29:12.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc02c870 con 0x7fedac003970 2023-11-08T16:29:12.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fedc4a7a640 1 -- 172.21.15.81:0/341340273 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc02cc00 con 0x7fedac003970 2023-11-08T16:29:12.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fbcf658c640 1 -- 172.21.15.81:0/3516251819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcf0160d40 msgr2=0x7fbcf0161170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fbcf658c640 1 --2- 172.21.15.81:0/3516251819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcf0160d40 0x7fbcf0161170 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009e50 tx=0x7fbce802bf60 comp rx=0 tx=0).stop 2023-11-08T16:29:12.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fbcf658c640 1 -- 172.21.15.81:0/3516251819 shutdown_connections 2023-11-08T16:29:12.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fbcf658c640 1 --2- 172.21.15.81:0/3516251819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcf0160d40 0x7fbcf0161170 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fbcf658c640 1 --2- 172.21.15.81:0/3516251819 >> v2:172.21.15.93:3300/0 conn(0x7fbcf00b4f80 0x7fbcf00b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fbcf658c640 1 -- 172.21.15.81:0/3516251819 >> 172.21.15.81:0/3516251819 conn(0x7fbcf00cf870 msgr2=0x7fbcf00cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:12.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fbcf658c640 1 -- 172.21.15.81:0/3516251819 shutdown_connections 2023-11-08T16:29:12.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fbcf658c640 1 -- 172.21.15.81:0/3516251819 wait complete. 2023-11-08T16:29:12.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fedc6a7e640 1 -- 172.21.15.81:0/341340273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedac003970 msgr2=0x7fedac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fedc6a7e640 1 --2- 172.21.15.81:0/341340273 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedac003970 0x7fedac003d90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fedbc009ac0 tx=0x7fedac0057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.750+0000 7fedc627d640 1 -- 172.21.15.81:0/341340273 reap_dead start 2023-11-08T16:29:12.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.751+0000 7fedc6a7e640 1 -- 172.21.15.81:0/341340273 shutdown_connections 2023-11-08T16:29:12.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.751+0000 7fedc6a7e640 1 -- 172.21.15.81:0/341340273 >> 172.21.15.81:0/341340273 conn(0x7fedc00cf870 msgr2=0x7fedc00cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:12.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.751+0000 7fedc6a7e640 1 -- 172.21.15.81:0/341340273 shutdown_connections 2023-11-08T16:29:12.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.751+0000 7fedc6a7e640 1 -- 172.21.15.81:0/341340273 wait complete. 2023-11-08T16:29:12.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.752+0000 7fbcf658c640 1 Processor -- start 2023-11-08T16:29:12.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.752+0000 7fbcf658c640 1 -- start start 2023-11-08T16:29:12.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.752+0000 7fedc6a7e640 1 Processor -- start 2023-11-08T16:29:12.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.752+0000 7fedc6a7e640 1 -- start start 2023-11-08T16:29:12.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fbcf658c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcf00b4f80 0x7fbcf01449c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fbcf658c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcf015cfe0 0x7fbcf015d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fbcf658c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcf0160d40 0x7fbcf015df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fbcf658c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf015f310 con 0x7fbcf00b4f80 2023-11-08T16:29:12.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fbcf658c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf015e450 con 0x7fbcf015cfe0 2023-11-08T16:29:12.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fbcf658c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf00c9220 con 0x7fbcf0160d40 2023-11-08T16:29:12.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fbcf558a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcf00b4f80 0x7fbcf01449c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:12.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fbcf558a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcf00b4f80 0x7fbcf01449c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35624/0 (socket says 172.21.15.81:35624) 2023-11-08T16:29:12.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fbcf558a640 1 -- 172.21.15.81:0/3370527310 learned_addr learned my addr 172.21.15.81:0/3370527310 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:12.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fedc6a7e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedac003970 0x7fedc00cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fedc6a7e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc00ca1a0 0x7fedc00ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fedc6a7e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedc00cab60 0x7fedc015e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fedc6a7e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00cafe0 con 0x7fedc00ca1a0 2023-11-08T16:29:12.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.753+0000 7fedc6a7e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00cbd20 con 0x7fedc00cab60 2023-11-08T16:29:12.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fedc6a7e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc0154a40 con 0x7fedac003970 2023-11-08T16:29:12.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fbcf558a640 1 -- 172.21.15.81:0/3370527310 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcf0160d40 msgr2=0x7fbcf015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fbcf558a640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcf0160d40 0x7fbcf015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fbcf558a640 1 -- 172.21.15.81:0/3370527310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcf015cfe0 msgr2=0x7fbcf015d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fbcf558a640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcf015cfe0 0x7fbcf015d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fbcf558a640 1 -- 172.21.15.81:0/3370527310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce8009b30 con 0x7fbcf00b4f80 2023-11-08T16:29:12.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fedc527b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc00ca1a0 0x7fedc00ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:12.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fedc527b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc00ca1a0 0x7fedc00ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35628/0 (socket says 172.21.15.81:35628) 2023-11-08T16:29:12.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fedc527b640 1 -- 172.21.15.81:0/3011314061 learned_addr learned my addr 172.21.15.81:0/3011314061 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:12.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fbcf558a640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcf00b4f80 0x7fbcf01449c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00c4d0 tx=0x7fbcec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:12.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fedc5a7c640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedac003970 0x7fedc00cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:12.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fedc527b640 1 -- 172.21.15.81:0/3011314061 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedac003970 msgr2=0x7fedc00cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fedc527b640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedac003970 0x7fedc00cb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fedc527b640 1 -- 172.21.15.81:0/3011314061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedc00cab60 msgr2=0x7fedc015e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fedc527b640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedc00cab60 0x7fedc015e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fedc527b640 1 -- 172.21.15.81:0/3011314061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc015d510 con 0x7fedc00ca1a0 2023-11-08T16:29:12.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fbce5ffb640 1 -- 172.21.15.81:0/3370527310 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec0075a0 con 0x7fbcf00b4f80 2023-11-08T16:29:12.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fbce5ffb640 1 -- 172.21.15.81:0/3370527310 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec0047e0 con 0x7fbcf00b4f80 2023-11-08T16:29:12.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fbce5ffb640 1 -- 172.21.15.81:0/3370527310 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec004b90 con 0x7fbcf00b4f80 2023-11-08T16:29:12.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fbcf658c640 1 -- 172.21.15.81:0/3370527310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf015e690 con 0x7fbcf00b4f80 2023-11-08T16:29:12.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.754+0000 7fbcf658c640 1 -- 172.21.15.81:0/3370527310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf015ec50 con 0x7fbcf00b4f80 2023-11-08T16:29:12.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.755+0000 7fedc527b640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc00ca1a0 0x7fedc00ca620 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fedb000eb60 tx=0x7fedb000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:12.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.755+0000 7fedb67fc640 1 -- 172.21.15.81:0/3011314061 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0009750 con 0x7fedc00ca1a0 2023-11-08T16:29:12.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.755+0000 7fedb67fc640 1 -- 172.21.15.81:0/3011314061 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb000ce40 con 0x7fedc00ca1a0 2023-11-08T16:29:12.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.755+0000 7fedb67fc640 1 -- 172.21.15.81:0/3011314061 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb000ee90 con 0x7fedc00ca1a0 2023-11-08T16:29:12.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.755+0000 7fedc6a7e640 1 -- 172.21.15.81:0/3011314061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc015d7f0 con 0x7fedc00ca1a0 2023-11-08T16:29:12.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.755+0000 7fedc6a7e640 1 -- 172.21.15.81:0/3011314061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc015ddb0 con 0x7fedc00ca1a0 2023-11-08T16:29:12.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.756+0000 7fbcf658c640 1 -- 172.21.15.81:0/3370527310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf004ece0 con 0x7fbcf00b4f80 2023-11-08T16:29:12.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.757+0000 7fbce5ffb640 1 -- 172.21.15.81:0/3370527310 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbcec004cf0 con 0x7fbcf00b4f80 2023-11-08T16:29:12.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.757+0000 7fbce5ffb640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcd406aef0 0x7fbcd406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.757+0000 7fedc6a7e640 1 -- 172.21.15.81:0/3011314061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc004ece0 con 0x7fedc00ca1a0 2023-11-08T16:29:12.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.757+0000 7fedb67fc640 1 -- 172.21.15.81:0/3011314061 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fedb0013470 con 0x7fedc00ca1a0 2023-11-08T16:29:12.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.758+0000 7fbce5ffb640 1 -- 172.21.15.81:0/3370527310 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbcec088b90 con 0x7fbcf00b4f80 2023-11-08T16:29:12.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.758+0000 7fbcf4d89640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcd406aef0 0x7fbcd406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:12.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.758+0000 7fbcf4d89640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcd406aef0 0x7fbcd406d3b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbce0006a90 tx=0x7fbce0006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:12.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.758+0000 7fedb67fc640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feda406ae70 0x7feda406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:12.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.759+0000 7fedb67fc640 1 -- 172.21.15.81:0/3011314061 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedb0089bd0 con 0x7fedc00ca1a0 2023-11-08T16:29:12.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.759+0000 7fedc5a7c640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feda406ae70 0x7feda406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:12.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.759+0000 7fedc5a7c640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feda406ae70 0x7feda406d330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fedbc036000 tx=0x7fedbc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:12.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.763+0000 7fbce5ffb640 1 -- 172.21.15.81:0/3370527310 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbcec053cb0 con 0x7fbcf00b4f80 2023-11-08T16:29:12.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.767+0000 7fedb67fc640 1 -- 172.21.15.81:0/3011314061 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fedb0054c80 con 0x7fedc00ca1a0 2023-11-08T16:29:12.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.944+0000 7fbcf658c640 1 -- 172.21.15.81:0/3370527310 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbcf00d0b70 con 0x7fbcf00b4f80 2023-11-08T16:29:12.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.945+0000 7fbce5ffb640 1 -- 172.21.15.81:0/3370527310 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbcec018070 con 0x7fbcf00b4f80 2023-11-08T16:29:12.947 INFO:teuthology.orchestra.run.smithi081.stdout:55834574856 2023-11-08T16:29:12.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.947+0000 7fbcf658c640 1 -- 172.21.15.81:0/3370527310 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcd406aef0 msgr2=0x7fbcd406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.947+0000 7fbcf658c640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcd406aef0 0x7fbcd406d3b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbce0006a90 tx=0x7fbce0006a20 comp rx=0 tx=0).stop 2023-11-08T16:29:12.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.947+0000 7fbcf658c640 1 -- 172.21.15.81:0/3370527310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcf00b4f80 msgr2=0x7fbcf01449c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.947+0000 7fbcf658c640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcf00b4f80 0x7fbcf01449c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00c4d0 tx=0x7fbcec00c990 comp rx=0 tx=0).stop 2023-11-08T16:29:12.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.947+0000 7fbcf658c640 1 -- 172.21.15.81:0/3370527310 shutdown_connections 2023-11-08T16:29:12.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.947+0000 7fbcf658c640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcd406aef0 0x7fbcd406d3b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.947+0000 7fbcf658c640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcf0160d40 0x7fbcf015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.947+0000 7fbcf658c640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcf015cfe0 0x7fbcf015d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.947+0000 7fbcf658c640 1 --2- 172.21.15.81:0/3370527310 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcf00b4f80 0x7fbcf01449c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.947+0000 7fbcf658c640 1 -- 172.21.15.81:0/3370527310 >> 172.21.15.81:0/3370527310 conn(0x7fbcf00cf870 msgr2=0x7fbcf00d1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:12.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.948+0000 7fbcf658c640 1 -- 172.21.15.81:0/3370527310 shutdown_connections 2023-11-08T16:29:12.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.948+0000 7fbcf658c640 1 -- 172.21.15.81:0/3370527310 wait complete. 2023-11-08T16:29:12.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.955+0000 7fedc6a7e640 1 -- 172.21.15.81:0/3011314061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fedc00d0b70 con 0x7fedc00ca1a0 2023-11-08T16:29:12.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.955+0000 7fedb67fc640 1 -- 172.21.15.81:0/3011314061 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fedb00582a0 con 0x7fedc00ca1a0 2023-11-08T16:29:12.957 INFO:teuthology.orchestra.run.smithi081.stdout:55834574854 2023-11-08T16:29:12.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 -- 172.21.15.81:0/3011314061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feda406ae70 msgr2=0x7feda406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feda406ae70 0x7feda406d330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fedbc036000 tx=0x7fedbc030000 comp rx=0 tx=0).stop 2023-11-08T16:29:12.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 -- 172.21.15.81:0/3011314061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc00ca1a0 msgr2=0x7fedc00ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:12.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc00ca1a0 0x7fedc00ca620 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fedb000eb60 tx=0x7fedb000c520 comp rx=0 tx=0).stop 2023-11-08T16:29:12.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 -- 172.21.15.81:0/3011314061 shutdown_connections 2023-11-08T16:29:12.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feda406ae70 0x7feda406d330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedc00cab60 0x7fedc015e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc00ca1a0 0x7fedc00ca620 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 --2- 172.21.15.81:0/3011314061 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedac003970 0x7fedc00cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:12.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 -- 172.21.15.81:0/3011314061 >> 172.21.15.81:0/3011314061 conn(0x7fedc00cf870 msgr2=0x7fedc00b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:12.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 -- 172.21.15.81:0/3011314061 shutdown_connections 2023-11-08T16:29:12.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:12.958+0000 7fedc6a7e640 1 -- 172.21.15.81:0/3011314061 wait complete. 2023-11-08T16:29:12.966 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.6 2023-11-08T16:29:12.966 DEBUG:teuthology.parallel:result is None 2023-11-08T16:29:12.976 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.7 2023-11-08T16:29:12.976 DEBUG:teuthology.parallel:result is None 2023-11-08T16:29:12.977 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-08T16:29:12.977 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:29:13.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.227+0000 7fa139b75640 1 Processor -- start 2023-11-08T16:29:13.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.227+0000 7fa139b75640 1 -- start start 2023-11-08T16:29:13.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.228+0000 7fa139b75640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa13415aac0 0x7fa1340711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.228+0000 7fa139b75640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa1340cf420 0x7fa13415aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.228+0000 7fa139b75640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa134070470 con 0x7fa1340716e0 2023-11-08T16:29:13.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.228+0000 7fa139b75640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13406f650 con 0x7fa13415aac0 2023-11-08T16:29:13.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.228+0000 7fa139b75640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa13406d580 con 0x7fa1340cf420 2023-11-08T16:29:13.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.229+0000 7fa138b73640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa13415aac0 0x7fa1340711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:13.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.229+0000 7fa138b73640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa13415aac0 0x7fa1340711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35634/0 (socket says 172.21.15.81:35634) 2023-11-08T16:29:13.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.229+0000 7fa138b73640 1 -- 172.21.15.81:0/1211513683 learned_addr learned my addr 172.21.15.81:0/1211513683 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:13.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.229+0000 7fa138b73640 1 -- 172.21.15.81:0/1211513683 >> v2:172.21.15.93:3300/0 conn(0x7fa1340cf420 msgr2=0x7fa13415aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:13.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.229+0000 7fa138b73640 1 --2- 172.21.15.81:0/1211513683 >> v2:172.21.15.93:3300/0 conn(0x7fa1340cf420 0x7fa13415aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.229+0000 7fa138b73640 1 -- 172.21.15.81:0/1211513683 >> v1:172.21.15.93:6789/0 conn(0x7fa1340716e0 legacy=0x7fa1340ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:13.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.229+0000 7fa138b73640 1 -- 172.21.15.81:0/1211513683 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13406ca00 con 0x7fa13415aac0 2023-11-08T16:29:13.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.230+0000 7fa138b73640 1 --2- 172.21.15.81:0/1211513683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa13415aac0 0x7fa1340711a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa1340703e0 tx=0x7fa12802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52760d983f1652c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:13.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.230+0000 7fa1337fe640 1 -- 172.21.15.81:0/1211513683 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128031870 con 0x7fa13415aac0 2023-11-08T16:29:13.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.230+0000 7fa1337fe640 1 -- 172.21.15.81:0/1211513683 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1280319d0 con 0x7fa13415aac0 2023-11-08T16:29:13.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.230+0000 7fa1337fe640 1 -- 172.21.15.81:0/1211513683 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa128031d60 con 0x7fa13415aac0 2023-11-08T16:29:13.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.231+0000 7fa139b75640 1 -- 172.21.15.81:0/1211513683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa13415aac0 msgr2=0x7fa1340711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.231+0000 7fa139b75640 1 --2- 172.21.15.81:0/1211513683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa13415aac0 0x7fa1340711a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa1340703e0 tx=0x7fa12802b910 comp rx=0 tx=0).stop 2023-11-08T16:29:13.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.231+0000 7fa139b75640 1 -- 172.21.15.81:0/1211513683 shutdown_connections 2023-11-08T16:29:13.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.231+0000 7fa139b75640 1 --2- 172.21.15.81:0/1211513683 >> v2:172.21.15.93:3300/0 conn(0x7fa1340cf420 0x7fa13415aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.231+0000 7fa139b75640 1 --2- 172.21.15.81:0/1211513683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa13415aac0 0x7fa1340711a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.231+0000 7fa139b75640 1 -- 172.21.15.81:0/1211513683 >> 172.21.15.81:0/1211513683 conn(0x7fa13406a4a0 msgr2=0x7fa13406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:13.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.231+0000 7fa139b75640 1 -- 172.21.15.81:0/1211513683 shutdown_connections 2023-11-08T16:29:13.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.231+0000 7fa139b75640 1 -- 172.21.15.81:0/1211513683 wait complete. 2023-11-08T16:29:13.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.233+0000 7fa139b75640 1 Processor -- start 2023-11-08T16:29:13.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.233+0000 7fa139b75640 1 -- start start 2023-11-08T16:29:13.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.234+0000 7fa139b75640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1340716e0 0x7fa1340af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.234+0000 7fa139b75640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1340cf420 0x7fa1341f2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.234+0000 7fa139b75640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa13415aac0 0x7fa134145280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.234+0000 7fa139b75640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1341f31b0 con 0x7fa1340cf420 2023-11-08T16:29:13.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.234+0000 7fa139b75640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340ae920 con 0x7fa1340716e0 2023-11-08T16:29:13.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.234+0000 7fa139b75640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1340aec00 con 0x7fa13415aac0 2023-11-08T16:29:13.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.234+0000 7fa133fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1340cf420 0x7fa1341f2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:13.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.234+0000 7fa133fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1340cf420 0x7fa1341f2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35642/0 (socket says 172.21.15.81:35642) 2023-11-08T16:29:13.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.234+0000 7fa133fff640 1 -- 172.21.15.81:0/495753322 learned_addr learned my addr 172.21.15.81:0/495753322 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:13.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa139374640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa13415aac0 0x7fa134145280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:13.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa138b73640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1340716e0 0x7fa1340af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:13.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa133fff640 1 -- 172.21.15.81:0/495753322 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa13415aac0 msgr2=0x7fa134145280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa133fff640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa13415aac0 0x7fa134145280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa133fff640 1 -- 172.21.15.81:0/495753322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1340716e0 msgr2=0x7fa1340af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa133fff640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1340716e0 0x7fa1340af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa133fff640 1 -- 172.21.15.81:0/495753322 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1280094d0 con 0x7fa1340cf420 2023-11-08T16:29:13.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa133fff640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1340cf420 0x7fa1341f2c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa12000c400 tx=0x7fa12000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:13.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa1317fa640 1 -- 172.21.15.81:0/495753322 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1200035f0 con 0x7fa1340cf420 2023-11-08T16:29:13.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa1317fa640 1 -- 172.21.15.81:0/495753322 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa120003750 con 0x7fa1340cf420 2023-11-08T16:29:13.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa1317fa640 1 -- 172.21.15.81:0/495753322 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa120003b40 con 0x7fa1340cf420 2023-11-08T16:29:13.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa139b75640 1 -- 172.21.15.81:0/495753322 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1340aeee0 con 0x7fa1340cf420 2023-11-08T16:29:13.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.235+0000 7fa139b75640 1 -- 172.21.15.81:0/495753322 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1340af4a0 con 0x7fa1340cf420 2023-11-08T16:29:13.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.236+0000 7fa139b75640 1 -- 172.21.15.81:0/495753322 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa134067800 con 0x7fa1340cf420 2023-11-08T16:29:13.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.237+0000 7fa1317fa640 1 -- 172.21.15.81:0/495753322 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa120003ca0 con 0x7fa1340cf420 2023-11-08T16:29:13.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.238+0000 7fa1317fa640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa11806aea0 0x7fa11806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.238+0000 7fa1317fa640 1 -- 172.21.15.81:0/495753322 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa120088710 con 0x7fa1340cf420 2023-11-08T16:29:13.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.238+0000 7fa138b73640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa11806aea0 0x7fa11806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:13.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.239+0000 7fa138b73640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa11806aea0 0x7fa11806d360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa128036000 tx=0x7fa12803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:13.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.242+0000 7fa1317fa640 1 -- 172.21.15.81:0/495753322 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa120053900 con 0x7fa1340cf420 2023-11-08T16:29:13.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.428+0000 7fa139b75640 1 -- 172.21.15.81:0/495753322 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa13406fb40 con 0x7fa11806aea0 2023-11-08T16:29:13.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.437+0000 7fa1317fa640 1 -- 172.21.15.81:0/495753322 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264441 (secure 0 0 0) 0x7fa128031d10 con 0x7fa11806aea0 2023-11-08T16:29:13.440 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:13.443 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:29:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 -- 172.21.15.81:0/495753322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa11806aea0 msgr2=0x7fa11806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa11806aea0 0x7fa11806d360 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa128036000 tx=0x7fa12803c040 comp rx=0 tx=0).stop 2023-11-08T16:29:13.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 -- 172.21.15.81:0/495753322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1340cf420 msgr2=0x7fa1341f2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1340cf420 0x7fa1341f2c70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa12000c400 tx=0x7fa12000c8c0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 -- 172.21.15.81:0/495753322 shutdown_connections 2023-11-08T16:29:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa11806aea0 0x7fa11806d360 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa13415aac0 0x7fa134145280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1340cf420 0x7fa1341f2c70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 --2- 172.21.15.81:0/495753322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1340716e0 0x7fa1340af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 -- 172.21.15.81:0/495753322 >> 172.21.15.81:0/495753322 conn(0x7fa13406a4a0 msgr2=0x7fa1340ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 -- 172.21.15.81:0/495753322 shutdown_connections 2023-11-08T16:29:13.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.443+0000 7fa139b75640 1 -- 172.21.15.81:0/495753322 wait complete. 2023-11-08T16:29:13.464 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-08T16:29:12.550395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219096,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755624,"statfs":{"total":773094113280,"available":772869758976,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.821950"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.741028+0000","last_change":"2023-11-08T16:29:03.766177+0000","last_active":"2023-11-08T16:29:04.741028+0000","last_peered":"2023-11-08T16:29:04.741028+0000","last_clean":"2023-11-08T16:29:04.741028+0000","last_became_active":"2023-11-08T16:29:03.765758+0000","last_became_peered":"2023-11-08T16:29:03.765758+0000","last_unstale":"2023-11-08T16:29:04.741028+0000","last_undegraded":"2023-11-08T16:29:04.741028+0000","last_fullsized":"2023-11-08T16:29:04.741028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:20:02.456685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538878+0000","last_change":"2023-11-08T16:29:03.775462+0000","last_active":"2023-11-08T16:29:07.538878+0000","last_peered":"2023-11-08T16:29:07.538878+0000","last_clean":"2023-11-08T16:29:07.538878+0000","last_became_active":"2023-11-08T16:29:03.775182+0000","last_became_peered":"2023-11-08T16:29:03.775182+0000","last_unstale":"2023-11-08T16:29:07.538878+0000","last_undegraded":"2023-11-08T16:29:07.538878+0000","last_fullsized":"2023-11-08T16:29:07.538878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737457+0000","last_change":"2023-11-08T16:29:03.771342+0000","last_active":"2023-11-08T16:29:06.737457+0000","last_peered":"2023-11-08T16:29:06.737457+0000","last_clean":"2023-11-08T16:29:06.737457+0000","last_became_active":"2023-11-08T16:29:03.770972+0000","last_became_peered":"2023-11-08T16:29:03.770972+0000","last_unstale":"2023-11-08T16:29:06.737457+0000","last_undegraded":"2023-11-08T16:29:06.737457+0000","last_fullsized":"2023-11-08T16:29:06.737457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:33.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738552+0000","last_change":"2023-11-08T16:29:03.762848+0000","last_active":"2023-11-08T16:29:04.738552+0000","last_peered":"2023-11-08T16:29:04.738552+0000","last_clean":"2023-11-08T16:29:04.738552+0000","last_became_active":"2023-11-08T16:29:03.762029+0000","last_became_peered":"2023-11-08T16:29:03.762029+0000","last_unstale":"2023-11-08T16:29:04.738552+0000","last_undegraded":"2023-11-08T16:29:04.738552+0000","last_fullsized":"2023-11-08T16:29:04.738552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:27.108912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739867+0000","last_change":"2023-11-08T16:29:03.758683+0000","last_active":"2023-11-08T16:29:04.739867+0000","last_peered":"2023-11-08T16:29:04.739867+0000","last_clean":"2023-11-08T16:29:04.739867+0000","last_became_active":"2023-11-08T16:29:03.758313+0000","last_became_peered":"2023-11-08T16:29:03.758313+0000","last_unstale":"2023-11-08T16:29:04.739867+0000","last_undegraded":"2023-11-08T16:29:04.739867+0000","last_fullsized":"2023-11-08T16:29:04.739867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:42.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738354+0000","last_change":"2023-11-08T16:29:03.774526+0000","last_active":"2023-11-08T16:29:04.738354+0000","last_peered":"2023-11-08T16:29:04.738354+0000","last_clean":"2023-11-08T16:29:04.738354+0000","last_became_active":"2023-11-08T16:29:03.774248+0000","last_became_peered":"2023-11-08T16:29:03.774248+0000","last_unstale":"2023-11-08T16:29:04.738354+0000","last_undegraded":"2023-11-08T16:29:04.738354+0000","last_fullsized":"2023-11-08T16:29:04.738354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:06.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738024+0000","last_change":"2023-11-08T16:29:03.775653+0000","last_active":"2023-11-08T16:29:04.738024+0000","last_peered":"2023-11-08T16:29:04.738024+0000","last_clean":"2023-11-08T16:29:04.738024+0000","last_became_active":"2023-11-08T16:29:03.774965+0000","last_became_peered":"2023-11-08T16:29:03.774965+0000","last_unstale":"2023-11-08T16:29:04.738024+0000","last_undegraded":"2023-11-08T16:29:04.738024+0000","last_fullsized":"2023-11-08T16:29:04.738024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:27.108912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735466+0000","last_change":"2023-11-08T16:29:03.767731+0000","last_active":"2023-11-08T16:29:04.735466+0000","last_peered":"2023-11-08T16:29:04.735466+0000","last_clean":"2023-11-08T16:29:04.735466+0000","last_became_active":"2023-11-08T16:29:03.767409+0000","last_became_peered":"2023-11-08T16:29:03.767409+0000","last_unstale":"2023-11-08T16:29:04.735466+0000","last_undegraded":"2023-11-08T16:29:04.735466+0000","last_fullsized":"2023-11-08T16:29:04.735466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:44.430116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.741175+0000","last_change":"2023-11-08T16:29:03.761330+0000","last_active":"2023-11-08T16:29:04.741175+0000","last_peered":"2023-11-08T16:29:04.741175+0000","last_clean":"2023-11-08T16:29:04.741175+0000","last_became_active":"2023-11-08T16:29:03.760953+0000","last_became_peered":"2023-11-08T16:29:03.760953+0000","last_unstale":"2023-11-08T16:29:04.741175+0000","last_undegraded":"2023-11-08T16:29:04.741175+0000","last_fullsized":"2023-11-08T16:29:04.741175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:56.358445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.545117+0000","last_change":"2023-11-08T16:29:03.761817+0000","last_active":"2023-11-08T16:29:07.545117+0000","last_peered":"2023-11-08T16:29:07.545117+0000","last_clean":"2023-11-08T16:29:07.545117+0000","last_became_active":"2023-11-08T16:29:03.760974+0000","last_became_peered":"2023-11-08T16:29:03.760974+0000","last_unstale":"2023-11-08T16:29:07.545117+0000","last_undegraded":"2023-11-08T16:29:07.545117+0000","last_fullsized":"2023-11-08T16:29:07.545117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:06.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.544830+0000","last_change":"2023-11-08T16:29:03.765935+0000","last_active":"2023-11-08T16:29:07.544830+0000","last_peered":"2023-11-08T16:29:07.544830+0000","last_clean":"2023-11-08T16:29:07.544830+0000","last_became_active":"2023-11-08T16:29:03.765176+0000","last_became_peered":"2023-11-08T16:29:03.765176+0000","last_unstale":"2023-11-08T16:29:07.544830+0000","last_undegraded":"2023-11-08T16:29:07.544830+0000","last_fullsized":"2023-11-08T16:29:07.544830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.703359+0000","last_change":"2023-11-08T16:29:03.771863+0000","last_active":"2023-11-08T16:29:07.703359+0000","last_peered":"2023-11-08T16:29:07.703359+0000","last_clean":"2023-11-08T16:29:07.703359+0000","last_became_active":"2023-11-08T16:29:03.770022+0000","last_became_peered":"2023-11-08T16:29:03.770022+0000","last_unstale":"2023-11-08T16:29:07.703359+0000","last_undegraded":"2023-11-08T16:29:07.703359+0000","last_fullsized":"2023-11-08T16:29:07.703359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:22.242002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735985+0000","last_change":"2023-11-08T16:29:03.765640+0000","last_active":"2023-11-08T16:29:04.735985+0000","last_peered":"2023-11-08T16:29:04.735985+0000","last_clean":"2023-11-08T16:29:04.735985+0000","last_became_active":"2023-11-08T16:29:03.765361+0000","last_became_peered":"2023-11-08T16:29:03.765361+0000","last_unstale":"2023-11-08T16:29:04.735985+0000","last_undegraded":"2023-11-08T16:29:04.735985+0000","last_fullsized":"2023-11-08T16:29:04.735985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:04.870398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.736888+0000","last_change":"2023-11-08T16:29:03.773181+0000","last_active":"2023-11-08T16:29:06.736888+0000","last_peered":"2023-11-08T16:29:06.736888+0000","last_clean":"2023-11-08T16:29:06.736888+0000","last_became_active":"2023-11-08T16:29:03.772830+0000","last_became_peered":"2023-11-08T16:29:03.772830+0000","last_unstale":"2023-11-08T16:29:06.736888+0000","last_undegraded":"2023-11-08T16:29:06.736888+0000","last_fullsized":"2023-11-08T16:29:06.736888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:44.430116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538489+0000","last_change":"2023-11-08T16:29:03.770534+0000","last_active":"2023-11-08T16:29:07.538489+0000","last_peered":"2023-11-08T16:29:07.538489+0000","last_clean":"2023-11-08T16:29:07.538489+0000","last_became_active":"2023-11-08T16:29:03.770195+0000","last_became_peered":"2023-11-08T16:29:03.770195+0000","last_unstale":"2023-11-08T16:29:07.538489+0000","last_undegraded":"2023-11-08T16:29:07.538489+0000","last_fullsized":"2023-11-08T16:29:07.538489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:44.430116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.737592+0000","last_change":"2023-11-08T16:29:03.758938+0000","last_active":"2023-11-08T16:29:04.737592+0000","last_peered":"2023-11-08T16:29:04.737592+0000","last_clean":"2023-11-08T16:29:04.737592+0000","last_became_active":"2023-11-08T16:29:03.758630+0000","last_became_peered":"2023-11-08T16:29:03.758630+0000","last_unstale":"2023-11-08T16:29:04.737592+0000","last_undegraded":"2023-11-08T16:29:04.737592+0000","last_fullsized":"2023-11-08T16:29:04.737592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740206+0000","last_change":"2023-11-08T16:29:03.763570+0000","last_active":"2023-11-08T16:29:04.740206+0000","last_peered":"2023-11-08T16:29:04.740206+0000","last_clean":"2023-11-08T16:29:04.740206+0000","last_became_active":"2023-11-08T16:29:03.763210+0000","last_became_peered":"2023-11-08T16:29:03.763210+0000","last_unstale":"2023-11-08T16:29:04.740206+0000","last_undegraded":"2023-11-08T16:29:04.740206+0000","last_fullsized":"2023-11-08T16:29:04.740206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:40:59.683967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401379+0000","last_change":"2023-11-08T16:29:03.765682+0000","last_active":"2023-11-08T16:29:05.401379+0000","last_peered":"2023-11-08T16:29:05.401379+0000","last_clean":"2023-11-08T16:29:05.401379+0000","last_became_active":"2023-11-08T16:29:03.765405+0000","last_became_peered":"2023-11-08T16:29:03.765405+0000","last_unstale":"2023-11-08T16:29:05.401379+0000","last_undegraded":"2023-11-08T16:29:05.401379+0000","last_fullsized":"2023-11-08T16:29:05.401379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:09.861000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740558+0000","last_change":"2023-11-08T16:29:03.759779+0000","last_active":"2023-11-08T16:29:04.740558+0000","last_peered":"2023-11-08T16:29:04.740558+0000","last_clean":"2023-11-08T16:29:04.740558+0000","last_became_active":"2023-11-08T16:29:03.759472+0000","last_became_peered":"2023-11-08T16:29:03.759472+0000","last_unstale":"2023-11-08T16:29:04.740558+0000","last_undegraded":"2023-11-08T16:29:04.740558+0000","last_fullsized":"2023-11-08T16:29:04.740558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:40:59.683967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.544556+0000","last_change":"2023-11-08T16:29:03.776902+0000","last_active":"2023-11-08T16:29:07.544556+0000","last_peered":"2023-11-08T16:29:07.544556+0000","last_clean":"2023-11-08T16:29:07.544556+0000","last_became_active":"2023-11-08T16:29:03.776528+0000","last_became_peered":"2023-11-08T16:29:03.776528+0000","last_unstale":"2023-11-08T16:29:07.544556+0000","last_undegraded":"2023-11-08T16:29:07.544556+0000","last_fullsized":"2023-11-08T16:29:07.544556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:04.870398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737582+0000","last_change":"2023-11-08T16:29:03.769496+0000","last_active":"2023-11-08T16:29:06.737582+0000","last_peered":"2023-11-08T16:29:06.737582+0000","last_clean":"2023-11-08T16:29:06.737582+0000","last_became_active":"2023-11-08T16:29:03.768491+0000","last_became_peered":"2023-11-08T16:29:03.768491+0000","last_unstale":"2023-11-08T16:29:06.737582+0000","last_undegraded":"2023-11-08T16:29:06.737582+0000","last_fullsized":"2023-11-08T16:29:06.737582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738562+0000","last_change":"2023-11-08T16:29:03.758807+0000","last_active":"2023-11-08T16:29:04.738562+0000","last_peered":"2023-11-08T16:29:04.738562+0000","last_clean":"2023-11-08T16:29:04.738562+0000","last_became_active":"2023-11-08T16:29:03.758466+0000","last_became_peered":"2023-11-08T16:29:03.758466+0000","last_unstale":"2023-11-08T16:29:04.738562+0000","last_undegraded":"2023-11-08T16:29:04.738562+0000","last_fullsized":"2023-11-08T16:29:04.738562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:10.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735439+0000","last_change":"2023-11-08T16:29:03.774206+0000","last_active":"2023-11-08T16:29:04.735439+0000","last_peered":"2023-11-08T16:29:04.735439+0000","last_clean":"2023-11-08T16:29:04.735439+0000","last_became_active":"2023-11-08T16:29:03.771583+0000","last_became_peered":"2023-11-08T16:29:03.771583+0000","last_unstale":"2023-11-08T16:29:04.735439+0000","last_undegraded":"2023-11-08T16:29:04.735439+0000","last_fullsized":"2023-11-08T16:29:04.735439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:51.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.737820+0000","last_change":"2023-11-08T16:29:03.760386+0000","last_active":"2023-11-08T16:29:04.737820+0000","last_peered":"2023-11-08T16:29:04.737820+0000","last_clean":"2023-11-08T16:29:04.737820+0000","last_became_active":"2023-11-08T16:29:03.760050+0000","last_became_peered":"2023-11-08T16:29:03.760050+0000","last_unstale":"2023-11-08T16:29:04.737820+0000","last_undegraded":"2023-11-08T16:29:04.737820+0000","last_fullsized":"2023-11-08T16:29:04.737820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735988+0000","last_change":"2023-11-08T16:29:03.768029+0000","last_active":"2023-11-08T16:29:04.735988+0000","last_peered":"2023-11-08T16:29:04.735988+0000","last_clean":"2023-11-08T16:29:04.735988+0000","last_became_active":"2023-11-08T16:29:03.767621+0000","last_became_peered":"2023-11-08T16:29:03.767621+0000","last_unstale":"2023-11-08T16:29:04.735988+0000","last_undegraded":"2023-11-08T16:29:04.735988+0000","last_fullsized":"2023-11-08T16:29:04.735988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.545128+0000","last_change":"2023-11-08T16:29:03.772197+0000","last_active":"2023-11-08T16:29:07.545128+0000","last_peered":"2023-11-08T16:29:07.545128+0000","last_clean":"2023-11-08T16:29:07.545128+0000","last_became_active":"2023-11-08T16:29:03.771699+0000","last_became_peered":"2023-11-08T16:29:03.771699+0000","last_unstale":"2023-11-08T16:29:07.545128+0000","last_undegraded":"2023-11-08T16:29:07.545128+0000","last_fullsized":"2023-11-08T16:29:07.545128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538799+0000","last_change":"2023-11-08T16:29:03.761215+0000","last_active":"2023-11-08T16:29:07.538799+0000","last_peered":"2023-11-08T16:29:07.538799+0000","last_clean":"2023-11-08T16:29:07.538799+0000","last_became_active":"2023-11-08T16:29:03.760855+0000","last_became_peered":"2023-11-08T16:29:03.760855+0000","last_unstale":"2023-11-08T16:29:07.538799+0000","last_undegraded":"2023-11-08T16:29:07.538799+0000","last_fullsized":"2023-11-08T16:29:07.538799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736058+0000","last_change":"2023-11-08T16:29:03.776139+0000","last_active":"2023-11-08T16:29:04.736058+0000","last_peered":"2023-11-08T16:29:04.736058+0000","last_clean":"2023-11-08T16:29:04.736058+0000","last_became_active":"2023-11-08T16:29:03.775828+0000","last_became_peered":"2023-11-08T16:29:03.775828+0000","last_unstale":"2023-11-08T16:29:04.736058+0000","last_undegraded":"2023-11-08T16:29:04.736058+0000","last_fullsized":"2023-11-08T16:29:04.736058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737631+0000","last_change":"2023-11-08T16:29:03.774003+0000","last_active":"2023-11-08T16:29:06.737631+0000","last_peered":"2023-11-08T16:29:06.737631+0000","last_clean":"2023-11-08T16:29:06.737631+0000","last_became_active":"2023-11-08T16:29:03.772344+0000","last_became_peered":"2023-11-08T16:29:03.772344+0000","last_unstale":"2023-11-08T16:29:06.737631+0000","last_undegraded":"2023-11-08T16:29:06.737631+0000","last_fullsized":"2023-11-08T16:29:06.737631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:27.108912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738706+0000","last_change":"2023-11-08T16:29:03.762765+0000","last_active":"2023-11-08T16:29:04.738706+0000","last_peered":"2023-11-08T16:29:04.738706+0000","last_clean":"2023-11-08T16:29:04.738706+0000","last_became_active":"2023-11-08T16:29:03.762435+0000","last_became_peered":"2023-11-08T16:29:03.762435+0000","last_unstale":"2023-11-08T16:29:04.738706+0000","last_undegraded":"2023-11-08T16:29:04.738706+0000","last_fullsized":"2023-11-08T16:29:04.738706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:58.120335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401083+0000","last_change":"2023-11-08T16:29:03.761315+0000","last_active":"2023-11-08T16:29:05.401083+0000","last_peered":"2023-11-08T16:29:05.401083+0000","last_clean":"2023-11-08T16:29:05.401083+0000","last_became_active":"2023-11-08T16:29:03.761017+0000","last_became_peered":"2023-11-08T16:29:03.761017+0000","last_unstale":"2023-11-08T16:29:05.401083+0000","last_undegraded":"2023-11-08T16:29:05.401083+0000","last_fullsized":"2023-11-08T16:29:05.401083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:46.473609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.544286+0000","last_change":"2023-11-08T16:29:03.766087+0000","last_active":"2023-11-08T16:29:07.544286+0000","last_peered":"2023-11-08T16:29:07.544286+0000","last_clean":"2023-11-08T16:29:07.544286+0000","last_became_active":"2023-11-08T16:29:03.765811+0000","last_became_peered":"2023-11-08T16:29:03.765811+0000","last_unstale":"2023-11-08T16:29:07.544286+0000","last_undegraded":"2023-11-08T16:29:07.544286+0000","last_fullsized":"2023-11-08T16:29:07.544286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:44.430116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.702817+0000","last_change":"2023-11-08T16:29:03.770480+0000","last_active":"2023-11-08T16:29:07.702817+0000","last_peered":"2023-11-08T16:29:07.702817+0000","last_clean":"2023-11-08T16:29:07.702817+0000","last_became_active":"2023-11-08T16:29:03.770166+0000","last_became_peered":"2023-11-08T16:29:03.770166+0000","last_unstale":"2023-11-08T16:29:07.702817+0000","last_undegraded":"2023-11-08T16:29:07.702817+0000","last_fullsized":"2023-11-08T16:29:07.702817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:11.949345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736199+0000","last_change":"2023-11-08T16:29:02.745095+0000","last_active":"2023-11-08T16:29:04.736199+0000","last_peered":"2023-11-08T16:29:04.736199+0000","last_clean":"2023-11-08T16:29:04.736199+0000","last_became_active":"2023-11-08T16:29:02.744708+0000","last_became_peered":"2023-11-08T16:29:02.744708+0000","last_unstale":"2023-11-08T16:29:04.736199+0000","last_undegraded":"2023-11-08T16:29:04.736199+0000","last_fullsized":"2023-11-08T16:29:04.736199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737953+0000","last_change":"2023-11-08T16:29:03.773134+0000","last_active":"2023-11-08T16:29:06.737953+0000","last_peered":"2023-11-08T16:29:06.737953+0000","last_clean":"2023-11-08T16:29:06.737953+0000","last_became_active":"2023-11-08T16:29:03.772787+0000","last_became_peered":"2023-11-08T16:29:03.772787+0000","last_unstale":"2023-11-08T16:29:06.737953+0000","last_undegraded":"2023-11-08T16:29:06.737953+0000","last_fullsized":"2023-11-08T16:29:06.737953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:10.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.604720+0000","last_change":"2023-11-08T16:29:02.752072+0000","last_active":"2023-11-08T16:29:06.604720+0000","last_peered":"2023-11-08T16:29:06.604720+0000","last_clean":"2023-11-08T16:29:06.604720+0000","last_became_active":"2023-11-08T16:29:02.751752+0000","last_became_peered":"2023-11-08T16:29:02.751752+0000","last_unstale":"2023-11-08T16:29:06.604720+0000","last_undegraded":"2023-11-08T16:29:06.604720+0000","last_fullsized":"2023-11-08T16:29:06.604720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:21.228079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735516+0000","last_change":"2023-11-08T16:29:02.751815+0000","last_active":"2023-11-08T16:29:04.735516+0000","last_peered":"2023-11-08T16:29:04.735516+0000","last_clean":"2023-11-08T16:29:04.735516+0000","last_became_active":"2023-11-08T16:29:02.751501+0000","last_became_peered":"2023-11-08T16:29:02.751501+0000","last_unstale":"2023-11-08T16:29:04.735516+0000","last_undegraded":"2023-11-08T16:29:04.735516+0000","last_fullsized":"2023-11-08T16:29:04.735516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:13.053337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735399+0000","last_change":"2023-11-08T16:29:03.758235+0000","last_active":"2023-11-08T16:29:04.735399+0000","last_peered":"2023-11-08T16:29:04.735399+0000","last_clean":"2023-11-08T16:29:04.735399+0000","last_became_active":"2023-11-08T16:29:03.757937+0000","last_became_peered":"2023-11-08T16:29:03.757937+0000","last_unstale":"2023-11-08T16:29:04.735399+0000","last_undegraded":"2023-11-08T16:29:04.735399+0000","last_fullsized":"2023-11-08T16:29:04.735399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:45:23.080055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.737829+0000","last_change":"2023-11-08T16:29:02.741267+0000","last_active":"2023-11-08T16:29:04.737829+0000","last_peered":"2023-11-08T16:29:04.737829+0000","last_clean":"2023-11-08T16:29:04.737829+0000","last_became_active":"2023-11-08T16:29:02.740048+0000","last_became_peered":"2023-11-08T16:29:02.740048+0000","last_unstale":"2023-11-08T16:29:04.737829+0000","last_undegraded":"2023-11-08T16:29:04.737829+0000","last_fullsized":"2023-11-08T16:29:04.737829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:10.935421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538501+0000","last_change":"2023-11-08T16:29:03.755015+0000","last_active":"2023-11-08T16:29:07.538501+0000","last_peered":"2023-11-08T16:29:07.538501+0000","last_clean":"2023-11-08T16:29:07.538501+0000","last_became_active":"2023-11-08T16:29:03.753403+0000","last_became_peered":"2023-11-08T16:29:03.753403+0000","last_unstale":"2023-11-08T16:29:07.538501+0000","last_undegraded":"2023-11-08T16:29:07.538501+0000","last_fullsized":"2023-11-08T16:29:07.538501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:33.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.604895+0000","last_change":"2023-11-08T16:29:02.749431+0000","last_active":"2023-11-08T16:29:06.604895+0000","last_peered":"2023-11-08T16:29:06.604895+0000","last_clean":"2023-11-08T16:29:06.604895+0000","last_became_active":"2023-11-08T16:29:02.748783+0000","last_became_peered":"2023-11-08T16:29:02.748783+0000","last_unstale":"2023-11-08T16:29:06.604895+0000","last_undegraded":"2023-11-08T16:29:06.604895+0000","last_fullsized":"2023-11-08T16:29:06.604895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:45.459685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739258+0000","last_change":"2023-11-08T16:29:03.759712+0000","last_active":"2023-11-08T16:29:04.739258+0000","last_peered":"2023-11-08T16:29:04.739258+0000","last_clean":"2023-11-08T16:29:04.739258+0000","last_became_active":"2023-11-08T16:29:03.759328+0000","last_became_peered":"2023-11-08T16:29:03.759328+0000","last_unstale":"2023-11-08T16:29:04.739258+0000","last_undegraded":"2023-11-08T16:29:04.739258+0000","last_fullsized":"2023-11-08T16:29:04.739258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:42.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739532+0000","last_change":"2023-11-08T16:29:02.749359+0000","last_active":"2023-11-08T16:29:04.739532+0000","last_peered":"2023-11-08T16:29:04.739532+0000","last_clean":"2023-11-08T16:29:04.739532+0000","last_became_active":"2023-11-08T16:29:02.748992+0000","last_became_peered":"2023-11-08T16:29:02.748992+0000","last_unstale":"2023-11-08T16:29:04.739532+0000","last_undegraded":"2023-11-08T16:29:04.739532+0000","last_fullsized":"2023-11-08T16:29:04.739532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:29.529005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737622+0000","last_change":"2023-11-08T16:29:03.766416+0000","last_active":"2023-11-08T16:29:06.737622+0000","last_peered":"2023-11-08T16:29:06.737622+0000","last_clean":"2023-11-08T16:29:06.737622+0000","last_became_active":"2023-11-08T16:29:03.766072+0000","last_became_peered":"2023-11-08T16:29:03.766072+0000","last_unstale":"2023-11-08T16:29:06.737622+0000","last_undegraded":"2023-11-08T16:29:06.737622+0000","last_fullsized":"2023-11-08T16:29:06.737622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.605209+0000","last_change":"2023-11-08T16:29:02.747464+0000","last_active":"2023-11-08T16:29:06.605209+0000","last_peered":"2023-11-08T16:29:06.605209+0000","last_clean":"2023-11-08T16:29:06.605209+0000","last_became_active":"2023-11-08T16:29:02.747178+0000","last_became_peered":"2023-11-08T16:29:02.747178+0000","last_unstale":"2023-11-08T16:29:06.605209+0000","last_undegraded":"2023-11-08T16:29:06.605209+0000","last_fullsized":"2023-11-08T16:29:06.605209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:13.053337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.703654+0000","last_change":"2023-11-08T16:29:03.768953+0000","last_active":"2023-11-08T16:29:07.703654+0000","last_peered":"2023-11-08T16:29:07.703654+0000","last_clean":"2023-11-08T16:29:07.703654+0000","last_became_active":"2023-11-08T16:29:03.768671+0000","last_became_peered":"2023-11-08T16:29:03.768671+0000","last_unstale":"2023-11-08T16:29:07.703654+0000","last_undegraded":"2023-11-08T16:29:07.703654+0000","last_fullsized":"2023-11-08T16:29:07.703654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:46.473609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.604768+0000","last_change":"2023-11-08T16:29:02.750178+0000","last_active":"2023-11-08T16:29:06.604768+0000","last_peered":"2023-11-08T16:29:06.604768+0000","last_clean":"2023-11-08T16:29:06.604768+0000","last_became_active":"2023-11-08T16:29:02.749883+0000","last_became_peered":"2023-11-08T16:29:02.749883+0000","last_unstale":"2023-11-08T16:29:06.604768+0000","last_undegraded":"2023-11-08T16:29:06.604768+0000","last_fullsized":"2023-11-08T16:29:06.604768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:50.401270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739830+0000","last_change":"2023-11-08T16:29:03.761268+0000","last_active":"2023-11-08T16:29:04.739830+0000","last_peered":"2023-11-08T16:29:04.739830+0000","last_clean":"2023-11-08T16:29:04.739830+0000","last_became_active":"2023-11-08T16:29:03.760923+0000","last_became_peered":"2023-11-08T16:29:03.760923+0000","last_unstale":"2023-11-08T16:29:04.739830+0000","last_undegraded":"2023-11-08T16:29:04.739830+0000","last_fullsized":"2023-11-08T16:29:04.739830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:18:29.767162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.600133+0000","last_change":"2023-11-08T16:29:02.741262+0000","last_active":"2023-11-08T16:29:06.600133+0000","last_peered":"2023-11-08T16:29:06.600133+0000","last_clean":"2023-11-08T16:29:06.600133+0000","last_became_active":"2023-11-08T16:29:02.740954+0000","last_became_peered":"2023-11-08T16:29:02.740954+0000","last_unstale":"2023-11-08T16:29:06.600133+0000","last_undegraded":"2023-11-08T16:29:06.600133+0000","last_fullsized":"2023-11-08T16:29:06.600133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:13.053337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737160+0000","last_change":"2023-11-08T16:29:03.772717+0000","last_active":"2023-11-08T16:29:06.737160+0000","last_peered":"2023-11-08T16:29:06.737160+0000","last_clean":"2023-11-08T16:29:06.737160+0000","last_became_active":"2023-11-08T16:29:03.772441+0000","last_became_peered":"2023-11-08T16:29:03.772441+0000","last_unstale":"2023-11-08T16:29:06.737160+0000","last_undegraded":"2023-11-08T16:29:06.737160+0000","last_fullsized":"2023-11-08T16:29:06.737160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:30.542928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738807+0000","last_change":"2023-11-08T16:29:02.744749+0000","last_active":"2023-11-08T16:29:04.738807+0000","last_peered":"2023-11-08T16:29:04.738807+0000","last_clean":"2023-11-08T16:29:04.738807+0000","last_became_active":"2023-11-08T16:29:02.744383+0000","last_became_peered":"2023-11-08T16:29:02.744383+0000","last_unstale":"2023-11-08T16:29:04.738807+0000","last_undegraded":"2023-11-08T16:29:04.738807+0000","last_fullsized":"2023-11-08T16:29:04.738807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:32.723904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740184+0000","last_change":"2023-11-08T16:29:03.771356+0000","last_active":"2023-11-08T16:29:04.740184+0000","last_peered":"2023-11-08T16:29:04.740184+0000","last_clean":"2023-11-08T16:29:04.740184+0000","last_became_active":"2023-11-08T16:29:03.771082+0000","last_became_peered":"2023-11-08T16:29:03.771082+0000","last_unstale":"2023-11-08T16:29:04.740184+0000","last_undegraded":"2023-11-08T16:29:04.740184+0000","last_fullsized":"2023-11-08T16:29:04.740184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:11:11.384242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.604672+0000","last_change":"2023-11-08T16:29:02.752309+0000","last_active":"2023-11-08T16:29:06.604672+0000","last_peered":"2023-11-08T16:29:06.604672+0000","last_clean":"2023-11-08T16:29:06.604672+0000","last_became_active":"2023-11-08T16:29:02.751973+0000","last_became_peered":"2023-11-08T16:29:02.751973+0000","last_unstale":"2023-11-08T16:29:06.604672+0000","last_undegraded":"2023-11-08T16:29:06.604672+0000","last_fullsized":"2023-11-08T16:29:06.604672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.544337+0000","last_change":"2023-11-08T16:29:03.759186+0000","last_active":"2023-11-08T16:29:07.544337+0000","last_peered":"2023-11-08T16:29:07.544337+0000","last_clean":"2023-11-08T16:29:07.544337+0000","last_became_active":"2023-11-08T16:29:03.758790+0000","last_became_peered":"2023-11-08T16:29:03.758790+0000","last_unstale":"2023-11-08T16:29:07.544337+0000","last_undegraded":"2023-11-08T16:29:07.544337+0000","last_fullsized":"2023-11-08T16:29:07.544337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:30.542928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.400909+0000","last_change":"2023-11-08T16:29:02.736184+0000","last_active":"2023-11-08T16:29:05.400909+0000","last_peered":"2023-11-08T16:29:05.400909+0000","last_clean":"2023-11-08T16:29:05.400909+0000","last_became_active":"2023-11-08T16:29:02.735423+0000","last_became_peered":"2023-11-08T16:29:02.735423+0000","last_unstale":"2023-11-08T16:29:05.400909+0000","last_undegraded":"2023-11-08T16:29:05.400909+0000","last_fullsized":"2023-11-08T16:29:05.400909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:46:55.176632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738431+0000","last_change":"2023-11-08T16:29:03.773617+0000","last_active":"2023-11-08T16:29:04.738431+0000","last_peered":"2023-11-08T16:29:04.738431+0000","last_clean":"2023-11-08T16:29:04.738431+0000","last_became_active":"2023-11-08T16:29:03.772381+0000","last_became_peered":"2023-11-08T16:29:03.772381+0000","last_unstale":"2023-11-08T16:29:04.738431+0000","last_undegraded":"2023-11-08T16:29:04.738431+0000","last_fullsized":"2023-11-08T16:29:04.738431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:10.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.597523+0000","last_change":"2023-11-08T16:29:02.750733+0000","last_active":"2023-11-08T16:29:06.597523+0000","last_peered":"2023-11-08T16:29:06.597523+0000","last_clean":"2023-11-08T16:29:06.597523+0000","last_became_active":"2023-11-08T16:29:02.750390+0000","last_became_peered":"2023-11-08T16:29:02.750390+0000","last_unstale":"2023-11-08T16:29:06.597523+0000","last_undegraded":"2023-11-08T16:29:06.597523+0000","last_fullsized":"2023-11-08T16:29:06.597523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739867+0000","last_change":"2023-11-08T16:29:03.772892+0000","last_active":"2023-11-08T16:29:04.739867+0000","last_peered":"2023-11-08T16:29:04.739867+0000","last_clean":"2023-11-08T16:29:04.739867+0000","last_became_active":"2023-11-08T16:29:03.772403+0000","last_became_peered":"2023-11-08T16:29:03.772403+0000","last_unstale":"2023-11-08T16:29:04.739867+0000","last_undegraded":"2023-11-08T16:29:04.739867+0000","last_fullsized":"2023-11-08T16:29:04.739867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:58.120335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401406+0000","last_change":"2023-11-08T16:29:02.737476+0000","last_active":"2023-11-08T16:29:05.401406+0000","last_peered":"2023-11-08T16:29:05.401406+0000","last_clean":"2023-11-08T16:29:05.401406+0000","last_became_active":"2023-11-08T16:29:02.737136+0000","last_became_peered":"2023-11-08T16:29:02.737136+0000","last_unstale":"2023-11-08T16:29:05.401406+0000","last_undegraded":"2023-11-08T16:29:05.401406+0000","last_fullsized":"2023-11-08T16:29:05.401406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:55.731186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.741221+0000","last_change":"2023-11-08T16:29:03.768059+0000","last_active":"2023-11-08T16:29:04.741221+0000","last_peered":"2023-11-08T16:29:04.741221+0000","last_clean":"2023-11-08T16:29:04.741221+0000","last_became_active":"2023-11-08T16:29:03.767846+0000","last_became_peered":"2023-11-08T16:29:03.767846+0000","last_unstale":"2023-11-08T16:29:04.741221+0000","last_undegraded":"2023-11-08T16:29:04.741221+0000","last_fullsized":"2023-11-08T16:29:04.741221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:03:32.436141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.605274+0000","last_change":"2023-11-08T16:29:02.740588+0000","last_active":"2023-11-08T16:29:06.605274+0000","last_peered":"2023-11-08T16:29:06.605274+0000","last_clean":"2023-11-08T16:29:06.605274+0000","last_became_active":"2023-11-08T16:29:02.740216+0000","last_became_peered":"2023-11-08T16:29:02.740216+0000","last_unstale":"2023-11-08T16:29:06.605274+0000","last_undegraded":"2023-11-08T16:29:06.605274+0000","last_fullsized":"2023-11-08T16:29:06.605274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740514+0000","last_change":"2023-11-08T16:29:03.755779+0000","last_active":"2023-11-08T16:29:04.740514+0000","last_peered":"2023-11-08T16:29:04.740514+0000","last_clean":"2023-11-08T16:29:04.740514+0000","last_became_active":"2023-11-08T16:29:03.755430+0000","last_became_peered":"2023-11-08T16:29:03.755430+0000","last_unstale":"2023-11-08T16:29:04.740514+0000","last_undegraded":"2023-11-08T16:29:04.740514+0000","last_fullsized":"2023-11-08T16:29:04.740514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:20:02.456685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401088+0000","last_change":"2023-11-08T16:29:02.744228+0000","last_active":"2023-11-08T16:29:05.401088+0000","last_peered":"2023-11-08T16:29:05.401088+0000","last_clean":"2023-11-08T16:29:05.401088+0000","last_became_active":"2023-11-08T16:29:02.743952+0000","last_became_peered":"2023-11-08T16:29:02.743952+0000","last_unstale":"2023-11-08T16:29:05.401088+0000","last_undegraded":"2023-11-08T16:29:05.401088+0000","last_fullsized":"2023-11-08T16:29:05.401088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:07.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740168+0000","last_change":"2023-11-08T16:29:03.759959+0000","last_active":"2023-11-08T16:29:04.740168+0000","last_peered":"2023-11-08T16:29:04.740168+0000","last_clean":"2023-11-08T16:29:04.740168+0000","last_became_active":"2023-11-08T16:29:03.759633+0000","last_became_peered":"2023-11-08T16:29:03.759633+0000","last_unstale":"2023-11-08T16:29:04.740168+0000","last_undegraded":"2023-11-08T16:29:04.740168+0000","last_fullsized":"2023-11-08T16:29:04.740168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:18.395303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.735559+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:29:03.735559+0000","last_peered":"2023-11-08T16:29:03.735559+0000","last_clean":"2023-11-08T16:29:03.735559+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:29:03.735559+0000","last_undegraded":"2023-11-08T16:29:03.735559+0000","last_fullsized":"2023-11-08T16:29:03.735559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739097+0000","last_change":"2023-11-08T16:29:02.746555+0000","last_active":"2023-11-08T16:29:04.739097+0000","last_peered":"2023-11-08T16:29:04.739097+0000","last_clean":"2023-11-08T16:29:04.739097+0000","last_became_active":"2023-11-08T16:29:02.745687+0000","last_became_peered":"2023-11-08T16:29:02.745687+0000","last_unstale":"2023-11-08T16:29:04.739097+0000","last_undegraded":"2023-11-08T16:29:04.739097+0000","last_fullsized":"2023-11-08T16:29:04.739097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:45.459685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736447+0000","last_change":"2023-11-08T16:29:03.767888+0000","last_active":"2023-11-08T16:29:04.736447+0000","last_peered":"2023-11-08T16:29:04.736447+0000","last_clean":"2023-11-08T16:29:04.736447+0000","last_became_active":"2023-11-08T16:29:03.767484+0000","last_became_peered":"2023-11-08T16:29:03.767484+0000","last_unstale":"2023-11-08T16:29:04.736447+0000","last_undegraded":"2023-11-08T16:29:04.736447+0000","last_fullsized":"2023-11-08T16:29:04.736447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:10.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738882+0000","last_change":"2023-11-08T16:29:02.746067+0000","last_active":"2023-11-08T16:29:04.738882+0000","last_peered":"2023-11-08T16:29:04.738882+0000","last_clean":"2023-11-08T16:29:04.738882+0000","last_became_active":"2023-11-08T16:29:02.745767+0000","last_became_peered":"2023-11-08T16:29:02.745767+0000","last_unstale":"2023-11-08T16:29:04.738882+0000","last_undegraded":"2023-11-08T16:29:04.738882+0000","last_fullsized":"2023-11-08T16:29:04.738882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:10.935421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.736892+0000","last_change":"2023-11-08T16:29:03.773037+0000","last_active":"2023-11-08T16:29:06.736892+0000","last_peered":"2023-11-08T16:29:06.736892+0000","last_clean":"2023-11-08T16:29:06.736892+0000","last_became_active":"2023-11-08T16:29:03.771599+0000","last_became_peered":"2023-11-08T16:29:03.771599+0000","last_unstale":"2023-11-08T16:29:06.736892+0000","last_undegraded":"2023-11-08T16:29:06.736892+0000","last_fullsized":"2023-11-08T16:29:06.736892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:51.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401068+0000","last_change":"2023-11-08T16:29:02.737753+0000","last_active":"2023-11-08T16:29:05.401068+0000","last_peered":"2023-11-08T16:29:05.401068+0000","last_clean":"2023-11-08T16:29:05.401068+0000","last_became_active":"2023-11-08T16:29:02.737374+0000","last_became_peered":"2023-11-08T16:29:02.737374+0000","last_unstale":"2023-11-08T16:29:05.401068+0000","last_undegraded":"2023-11-08T16:29:05.401068+0000","last_fullsized":"2023-11-08T16:29:05.401068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:10.935421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538502+0000","last_change":"2023-11-08T16:29:03.759721+0000","last_active":"2023-11-08T16:29:07.538502+0000","last_peered":"2023-11-08T16:29:07.538502+0000","last_clean":"2023-11-08T16:29:07.538502+0000","last_became_active":"2023-11-08T16:29:03.758129+0000","last_became_peered":"2023-11-08T16:29:03.758129+0000","last_unstale":"2023-11-08T16:29:07.538502+0000","last_undegraded":"2023-11-08T16:29:07.538502+0000","last_fullsized":"2023-11-08T16:29:07.538502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:30.542928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738972+0000","last_change":"2023-11-08T16:29:02.742034+0000","last_active":"2023-11-08T16:29:04.738972+0000","last_peered":"2023-11-08T16:29:04.738972+0000","last_clean":"2023-11-08T16:29:04.738972+0000","last_became_active":"2023-11-08T16:29:02.741705+0000","last_became_peered":"2023-11-08T16:29:02.741705+0000","last_unstale":"2023-11-08T16:29:04.738972+0000","last_undegraded":"2023-11-08T16:29:04.738972+0000","last_fullsized":"2023-11-08T16:29:04.738972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:03.856474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.703155+0000","last_change":"2023-11-08T16:29:03.777239+0000","last_active":"2023-11-08T16:29:07.703155+0000","last_peered":"2023-11-08T16:29:07.703155+0000","last_clean":"2023-11-08T16:29:07.703155+0000","last_became_active":"2023-11-08T16:29:03.776975+0000","last_became_peered":"2023-11-08T16:29:03.776975+0000","last_unstale":"2023-11-08T16:29:07.703155+0000","last_undegraded":"2023-11-08T16:29:07.703155+0000","last_fullsized":"2023-11-08T16:29:07.703155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:46:56.190555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.605164+0000","last_change":"2023-11-08T16:29:02.733716+0000","last_active":"2023-11-08T16:29:06.605164+0000","last_peered":"2023-11-08T16:29:06.605164+0000","last_clean":"2023-11-08T16:29:06.605164+0000","last_became_active":"2023-11-08T16:29:02.733435+0000","last_became_peered":"2023-11-08T16:29:02.733435+0000","last_unstale":"2023-11-08T16:29:06.605164+0000","last_undegraded":"2023-11-08T16:29:06.605164+0000","last_fullsized":"2023-11-08T16:29:06.605164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739982+0000","last_change":"2023-11-08T16:29:03.764171+0000","last_active":"2023-11-08T16:29:04.739982+0000","last_peered":"2023-11-08T16:29:04.739982+0000","last_clean":"2023-11-08T16:29:04.739982+0000","last_became_active":"2023-11-08T16:29:03.762549+0000","last_became_peered":"2023-11-08T16:29:03.762549+0000","last_unstale":"2023-11-08T16:29:04.739982+0000","last_undegraded":"2023-11-08T16:29:04.739982+0000","last_fullsized":"2023-11-08T16:29:04.739982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:11:11.384242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740565+0000","last_change":"2023-11-08T16:29:02.744381+0000","last_active":"2023-11-08T16:29:04.740565+0000","last_peered":"2023-11-08T16:29:04.740565+0000","last_clean":"2023-11-08T16:29:04.740565+0000","last_became_active":"2023-11-08T16:29:02.744077+0000","last_became_peered":"2023-11-08T16:29:02.744077+0000","last_unstale":"2023-11-08T16:29:04.740565+0000","last_undegraded":"2023-11-08T16:29:04.740565+0000","last_fullsized":"2023-11-08T16:29:04.740565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:43.416193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538664+0000","last_change":"2023-11-08T16:29:03.753962+0000","last_active":"2023-11-08T16:29:07.538664+0000","last_peered":"2023-11-08T16:29:07.538664+0000","last_clean":"2023-11-08T16:29:07.538664+0000","last_became_active":"2023-11-08T16:29:03.753317+0000","last_became_peered":"2023-11-08T16:29:03.753317+0000","last_unstale":"2023-11-08T16:29:07.538664+0000","last_undegraded":"2023-11-08T16:29:07.538664+0000","last_fullsized":"2023-11-08T16:29:07.538664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:04.870398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538904+0000","last_change":"2023-11-08T16:29:02.745315+0000","last_active":"2023-11-08T16:29:07.538904+0000","last_peered":"2023-11-08T16:29:07.538904+0000","last_clean":"2023-11-08T16:29:07.538904+0000","last_became_active":"2023-11-08T16:29:02.744940+0000","last_became_peered":"2023-11-08T16:29:02.744940+0000","last_unstale":"2023-11-08T16:29:07.538904+0000","last_undegraded":"2023-11-08T16:29:07.538904+0000","last_fullsized":"2023-11-08T16:29:07.538904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:45:22.066131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735964+0000","last_change":"2023-11-08T16:29:03.763070+0000","last_active":"2023-11-08T16:29:04.735964+0000","last_peered":"2023-11-08T16:29:04.735964+0000","last_clean":"2023-11-08T16:29:04.735964+0000","last_became_active":"2023-11-08T16:29:03.762709+0000","last_became_peered":"2023-11-08T16:29:03.762709+0000","last_unstale":"2023-11-08T16:29:04.735964+0000","last_undegraded":"2023-11-08T16:29:04.735964+0000","last_fullsized":"2023-11-08T16:29:04.735964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:33.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.604918+0000","last_change":"2023-11-08T16:29:02.749634+0000","last_active":"2023-11-08T16:29:06.604918+0000","last_peered":"2023-11-08T16:29:06.604918+0000","last_clean":"2023-11-08T16:29:06.604918+0000","last_became_active":"2023-11-08T16:29:02.749138+0000","last_became_peered":"2023-11-08T16:29:02.749138+0000","last_unstale":"2023-11-08T16:29:06.604918+0000","last_undegraded":"2023-11-08T16:29:06.604918+0000","last_fullsized":"2023-11-08T16:29:06.604918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737617+0000","last_change":"2023-11-08T16:29:03.773063+0000","last_active":"2023-11-08T16:29:06.737617+0000","last_peered":"2023-11-08T16:29:06.737617+0000","last_clean":"2023-11-08T16:29:06.737617+0000","last_became_active":"2023-11-08T16:29:03.772775+0000","last_became_peered":"2023-11-08T16:29:03.772775+0000","last_unstale":"2023-11-08T16:29:06.737617+0000","last_undegraded":"2023-11-08T16:29:06.737617+0000","last_fullsized":"2023-11-08T16:29:06.737617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:06.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738667+0000","last_change":"2023-11-08T16:29:02.745343+0000","last_active":"2023-11-08T16:29:04.738667+0000","last_peered":"2023-11-08T16:29:04.738667+0000","last_clean":"2023-11-08T16:29:04.738667+0000","last_became_active":"2023-11-08T16:29:02.744884+0000","last_became_peered":"2023-11-08T16:29:02.744884+0000","last_unstale":"2023-11-08T16:29:04.738667+0000","last_undegraded":"2023-11-08T16:29:04.738667+0000","last_fullsized":"2023-11-08T16:29:04.738667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.703174+0000","last_change":"2023-11-08T16:29:03.767447+0000","last_active":"2023-11-08T16:29:07.703174+0000","last_peered":"2023-11-08T16:29:07.703174+0000","last_clean":"2023-11-08T16:29:07.703174+0000","last_became_active":"2023-11-08T16:29:03.767116+0000","last_became_peered":"2023-11-08T16:29:03.767116+0000","last_unstale":"2023-11-08T16:29:07.703174+0000","last_undegraded":"2023-11-08T16:29:07.703174+0000","last_fullsized":"2023-11-08T16:29:07.703174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:05.442030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.605062+0000","last_change":"2023-11-08T16:29:02.746234+0000","last_active":"2023-11-08T16:29:06.605062+0000","last_peered":"2023-11-08T16:29:06.605062+0000","last_clean":"2023-11-08T16:29:06.605062+0000","last_became_active":"2023-11-08T16:29:02.745941+0000","last_became_peered":"2023-11-08T16:29:02.745941+0000","last_unstale":"2023-11-08T16:29:06.605062+0000","last_undegraded":"2023-11-08T16:29:06.605062+0000","last_fullsized":"2023-11-08T16:29:06.605062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:35:55.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735534+0000","last_change":"2023-11-08T16:29:03.754330+0000","last_active":"2023-11-08T16:29:04.735534+0000","last_peered":"2023-11-08T16:29:04.735534+0000","last_clean":"2023-11-08T16:29:04.735534+0000","last_became_active":"2023-11-08T16:29:03.751108+0000","last_became_peered":"2023-11-08T16:29:03.751108+0000","last_unstale":"2023-11-08T16:29:04.735534+0000","last_undegraded":"2023-11-08T16:29:04.735534+0000","last_fullsized":"2023-11-08T16:29:04.735534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:30.542928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.605087+0000","last_change":"2023-11-08T16:29:02.737421+0000","last_active":"2023-11-08T16:29:06.605087+0000","last_peered":"2023-11-08T16:29:06.605087+0000","last_clean":"2023-11-08T16:29:06.605087+0000","last_became_active":"2023-11-08T16:29:02.737099+0000","last_became_peered":"2023-11-08T16:29:02.737099+0000","last_unstale":"2023-11-08T16:29:06.605087+0000","last_undegraded":"2023-11-08T16:29:06.605087+0000","last_fullsized":"2023-11-08T16:29:06.605087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:07.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736361+0000","last_change":"2023-11-08T16:29:03.774533+0000","last_active":"2023-11-08T16:29:04.736361+0000","last_peered":"2023-11-08T16:29:04.736361+0000","last_clean":"2023-11-08T16:29:04.736361+0000","last_became_active":"2023-11-08T16:29:03.774212+0000","last_became_peered":"2023-11-08T16:29:03.774212+0000","last_unstale":"2023-11-08T16:29:04.736361+0000","last_undegraded":"2023-11-08T16:29:04.736361+0000","last_fullsized":"2023-11-08T16:29:04.736361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:27.108912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740429+0000","last_change":"2023-11-08T16:29:02.754015+0000","last_active":"2023-11-08T16:29:04.740429+0000","last_peered":"2023-11-08T16:29:04.740429+0000","last_clean":"2023-11-08T16:29:04.740429+0000","last_became_active":"2023-11-08T16:29:02.753688+0000","last_became_peered":"2023-11-08T16:29:02.753688+0000","last_unstale":"2023-11-08T16:29:04.740429+0000","last_undegraded":"2023-11-08T16:29:04.740429+0000","last_fullsized":"2023-11-08T16:29:04.740429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.544487+0000","last_change":"2023-11-08T16:29:03.771420+0000","last_active":"2023-11-08T16:29:07.544487+0000","last_peered":"2023-11-08T16:29:07.544487+0000","last_clean":"2023-11-08T16:29:07.544487+0000","last_became_active":"2023-11-08T16:29:03.771137+0000","last_became_peered":"2023-11-08T16:29:03.771137+0000","last_unstale":"2023-11-08T16:29:07.544487+0000","last_undegraded":"2023-11-08T16:29:07.544487+0000","last_fullsized":"2023-11-08T16:29:07.544487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:33.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739672+0000","last_change":"2023-11-08T16:29:03.774156+0000","last_active":"2023-11-08T16:29:04.739672+0000","last_peered":"2023-11-08T16:29:04.739672+0000","last_clean":"2023-11-08T16:29:04.739672+0000","last_became_active":"2023-11-08T16:29:03.773842+0000","last_became_peered":"2023-11-08T16:29:03.773842+0000","last_unstale":"2023-11-08T16:29:04.739672+0000","last_undegraded":"2023-11-08T16:29:04.739672+0000","last_fullsized":"2023-11-08T16:29:04.739672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:18.395303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738809+0000","last_change":"2023-11-08T16:29:02.747528+0000","last_active":"2023-11-08T16:29:04.738809+0000","last_peered":"2023-11-08T16:29:04.738809+0000","last_clean":"2023-11-08T16:29:04.738809+0000","last_became_active":"2023-11-08T16:29:02.747113+0000","last_became_peered":"2023-11-08T16:29:02.747113+0000","last_unstale":"2023-11-08T16:29:04.738809+0000","last_undegraded":"2023-11-08T16:29:04.738809+0000","last_fullsized":"2023-11-08T16:29:04.738809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:46:55.176632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738937+0000","last_change":"2023-11-08T16:29:03.778220+0000","last_active":"2023-11-08T16:29:04.738937+0000","last_peered":"2023-11-08T16:29:04.738937+0000","last_clean":"2023-11-08T16:29:04.738937+0000","last_became_active":"2023-11-08T16:29:03.777800+0000","last_became_peered":"2023-11-08T16:29:03.777800+0000","last_unstale":"2023-11-08T16:29:04.738937+0000","last_undegraded":"2023-11-08T16:29:04.738937+0000","last_fullsized":"2023-11-08T16:29:04.738937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:18:29.767162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740343+0000","last_change":"2023-11-08T16:29:02.749398+0000","last_active":"2023-11-08T16:29:04.740343+0000","last_peered":"2023-11-08T16:29:04.740343+0000","last_clean":"2023-11-08T16:29:04.740343+0000","last_became_active":"2023-11-08T16:29:02.749084+0000","last_became_peered":"2023-11-08T16:29:02.749084+0000","last_unstale":"2023-11-08T16:29:04.740343+0000","last_undegraded":"2023-11-08T16:29:04.740343+0000","last_fullsized":"2023-11-08T16:29:04.740343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:03.856474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737570+0000","last_change":"2023-11-08T16:29:03.776424+0000","last_active":"2023-11-08T16:29:06.737570+0000","last_peered":"2023-11-08T16:29:06.737570+0000","last_clean":"2023-11-08T16:29:06.737570+0000","last_became_active":"2023-11-08T16:29:03.774967+0000","last_became_peered":"2023-11-08T16:29:03.774967+0000","last_unstale":"2023-11-08T16:29:06.737570+0000","last_undegraded":"2023-11-08T16:29:06.737570+0000","last_fullsized":"2023-11-08T16:29:06.737570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:04.870398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736286+0000","last_change":"2023-11-08T16:29:02.733433+0000","last_active":"2023-11-08T16:29:04.736286+0000","last_peered":"2023-11-08T16:29:04.736286+0000","last_clean":"2023-11-08T16:29:04.736286+0000","last_became_active":"2023-11-08T16:29:02.733126+0000","last_became_peered":"2023-11-08T16:29:02.733126+0000","last_unstale":"2023-11-08T16:29:04.736286+0000","last_undegraded":"2023-11-08T16:29:04.736286+0000","last_fullsized":"2023-11-08T16:29:04.736286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:35:55.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736155+0000","last_change":"2023-11-08T16:29:03.770442+0000","last_active":"2023-11-08T16:29:04.736155+0000","last_peered":"2023-11-08T16:29:04.736155+0000","last_clean":"2023-11-08T16:29:04.736155+0000","last_became_active":"2023-11-08T16:29:03.770158+0000","last_became_peered":"2023-11-08T16:29:03.770158+0000","last_unstale":"2023-11-08T16:29:04.736155+0000","last_undegraded":"2023-11-08T16:29:04.736155+0000","last_fullsized":"2023-11-08T16:29:04.736155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:06.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.600398+0000","last_change":"2023-11-08T16:29:02.744925+0000","last_active":"2023-11-08T16:29:06.600398+0000","last_peered":"2023-11-08T16:29:06.600398+0000","last_clean":"2023-11-08T16:29:06.600398+0000","last_became_active":"2023-11-08T16:29:02.744574+0000","last_became_peered":"2023-11-08T16:29:02.744574+0000","last_unstale":"2023-11-08T16:29:06.600398+0000","last_undegraded":"2023-11-08T16:29:06.600398+0000","last_fullsized":"2023-11-08T16:29:06.600398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:35:55.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2249999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.653}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9410000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.774}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.599}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.5369999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4980000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3050000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9409999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8959999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.536}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.962}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:29:13.469 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:29:13.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.717+0000 7facf9933640 1 Processor -- start 2023-11-08T16:29:13.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.717+0000 7facf9933640 1 -- start start 2023-11-08T16:29:13.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.718+0000 7facf9933640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facf415d2b0 0x7facf4161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.718+0000 7facf9933640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf4161c50 con 0x7facf40c9060 2023-11-08T16:29:13.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.718+0000 7facf9933640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facf406b980 con 0x7facf40ccf00 2023-11-08T16:29:13.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.718+0000 7facf9933640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf415d690 con 0x7facf415d2b0 2023-11-08T16:29:13.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf27fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7facf40c9060 0x7facf40cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49166/0 (socket says 172.21.15.81:49166) 2023-11-08T16:29:13.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf27fc640 1 -- 172.21.15.81:0/1305114808 learned_addr learned my addr 172.21.15.81:0/1305114808 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:13.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf2ffd640 1 --2- 172.21.15.81:0/1305114808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facf415d2b0 0x7facf4161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:13.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1809860310 0 0) 0x7facdc001110 con 0x7facf40c9060 2023-11-08T16:29:13.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7face0003520 con 0x7facf40c9060 2023-11-08T16:29:13.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4170312535 0 0) 0x7face0003520 con 0x7facf40c9060 2023-11-08T16:29:13.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facf415d2b0 msgr2=0x7facf4161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:13.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf1ffb640 1 --2- 172.21.15.81:0/1305114808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facf415d2b0 0x7facf4161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 >> v1:172.21.15.93:6789/0 conn(0x7facf40ccf00 legacy=0x7facf406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf406bc60 con 0x7facf40c9060 2023-11-08T16:29:13.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7facdc001800 con 0x7facf40c9060 2023-11-08T16:29:13.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 >> v1:172.21.15.81:6789/0 conn(0x7facf40c9060 legacy=0x7facf40cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.719+0000 7facf1ffb640 1 --2- 172.21.15.81:0/1305114808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7face0003a80 0x7face0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf1ffb640 1 --2- 172.21.15.81:0/1305114808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7face00047c0 0x7face0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf1ffb640 1 --2- 172.21.15.81:0/1305114808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facf415d2b0 0x7face0005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf415d690 con 0x7face0003a80 2023-11-08T16:29:13.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face000bb40 con 0x7face00047c0 2023-11-08T16:29:13.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face000bd00 con 0x7facf415d2b0 2023-11-08T16:29:13.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf2ffd640 1 --2- 172.21.15.81:0/1305114808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7face0003a80 0x7face0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:13.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf2ffd640 1 -- 172.21.15.81:0/1305114808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facf415d2b0 msgr2=0x7face0005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:13.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf2ffd640 1 --2- 172.21.15.81:0/1305114808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facf415d2b0 0x7face0005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf27fc640 1 --2- 172.21.15.81:0/1305114808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7face00047c0 0x7face0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:13.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf2ffd640 1 -- 172.21.15.81:0/1305114808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7face00047c0 msgr2=0x7face0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:13.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf2ffd640 1 --2- 172.21.15.81:0/1305114808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7face00047c0 0x7face0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.720+0000 7facf2ffd640 1 -- 172.21.15.81:0/1305114808 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face0003520 con 0x7face0003a80 2023-11-08T16:29:13.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.721+0000 7facf2ffd640 1 --2- 172.21.15.81:0/1305114808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7face0003a80 0x7face0003ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7face4009fc0 tx=0x7face00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bc840a5e114ea0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:13.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.721+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face402cb90 con 0x7face0003a80 2023-11-08T16:29:13.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.721+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face400f040 con 0x7face0003a80 2023-11-08T16:29:13.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.721+0000 7facf1ffb640 1 -- 172.21.15.81:0/1305114808 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4034440 con 0x7face0003a80 2023-11-08T16:29:13.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.722+0000 7facf9933640 1 -- 172.21.15.81:0/1305114808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7face0003a80 msgr2=0x7face0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.722+0000 7facf9933640 1 --2- 172.21.15.81:0/1305114808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7face0003a80 0x7face0003ea0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7face4009fc0 tx=0x7face00057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.722+0000 7facf37fe640 1 -- 172.21.15.81:0/1305114808 reap_dead start 2023-11-08T16:29:13.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.722+0000 7facf9933640 1 -- 172.21.15.81:0/1305114808 shutdown_connections 2023-11-08T16:29:13.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.722+0000 7facf9933640 1 -- 172.21.15.81:0/1305114808 >> 172.21.15.81:0/1305114808 conn(0x7facf40cb490 msgr2=0x7facf40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:13.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.722+0000 7facf9933640 1 -- 172.21.15.81:0/1305114808 shutdown_connections 2023-11-08T16:29:13.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.722+0000 7facf9933640 1 -- 172.21.15.81:0/1305114808 wait complete. 2023-11-08T16:29:13.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.724+0000 7facf9933640 1 Processor -- start 2023-11-08T16:29:13.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.724+0000 7facf9933640 1 -- start start 2023-11-08T16:29:13.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf9933640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7face0003a80 0x7facf40c53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf9933640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facf41f6270 0x7facf40c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf9933640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facf40b1df0 0x7facf4156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf9933640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf40b1960 con 0x7facf41f6270 2023-11-08T16:29:13.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf9933640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facf41f6a60 con 0x7face0003a80 2023-11-08T16:29:13.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf9933640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facf41f6d40 con 0x7facf40b1df0 2023-11-08T16:29:13.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facf41f6270 0x7facf40c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:13.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facf41f6270 0x7facf40c74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35672/0 (socket says 172.21.15.81:35672) 2023-11-08T16:29:13.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf27fc640 1 -- 172.21.15.81:0/525582906 learned_addr learned my addr 172.21.15.81:0/525582906 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:13.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf27fc640 1 -- 172.21.15.81:0/525582906 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facf40b1df0 msgr2=0x7facf4156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf27fc640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facf40b1df0 0x7facf4156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf27fc640 1 -- 172.21.15.81:0/525582906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7face0003a80 msgr2=0x7facf40c53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf27fc640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7face0003a80 0x7facf40c53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.725+0000 7facf27fc640 1 -- 172.21.15.81:0/525582906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf41f7020 con 0x7facf41f6270 2023-11-08T16:29:13.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.726+0000 7facf27fc640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facf41f6270 0x7facf40c74f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7facdc001270 tx=0x7facdc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:13.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.726+0000 7facebfff640 1 -- 172.21.15.81:0/525582906 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc0074e0 con 0x7facf41f6270 2023-11-08T16:29:13.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.726+0000 7facebfff640 1 -- 172.21.15.81:0/525582906 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc007640 con 0x7facf41f6270 2023-11-08T16:29:13.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.726+0000 7facebfff640 1 -- 172.21.15.81:0/525582906 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc004a00 con 0x7facf41f6270 2023-11-08T16:29:13.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.726+0000 7facf9933640 1 -- 172.21.15.81:0/525582906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facf40b15f0 con 0x7facf41f6270 2023-11-08T16:29:13.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.726+0000 7facf9933640 1 -- 172.21.15.81:0/525582906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf40c7a30 con 0x7facf41f6270 2023-11-08T16:29:13.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.727+0000 7facf9933640 1 -- 172.21.15.81:0/525582906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facf404ece0 con 0x7facf41f6270 2023-11-08T16:29:13.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.728+0000 7facebfff640 1 -- 172.21.15.81:0/525582906 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7facdc004b60 con 0x7facf41f6270 2023-11-08T16:29:13.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.729+0000 7facebfff640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7facd406ae70 0x7facd406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:13.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.729+0000 7facebfff640 1 -- 172.21.15.81:0/525582906 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7facdc088e70 con 0x7facf41f6270 2023-11-08T16:29:13.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.729+0000 7facf2ffd640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7facd406ae70 0x7facd406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:13.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.730+0000 7facf2ffd640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7facd406ae70 0x7facd406d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7face4036000 tx=0x7face402e070 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:13.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.733+0000 7facebfff640 1 -- 172.21.15.81:0/525582906 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7facdc057910 con 0x7facf41f6270 2023-11-08T16:29:13.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.919+0000 7facf9933640 1 -- 172.21.15.81:0/525582906 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7facf406c220 con 0x7facd406ae70 2023-11-08T16:29:13.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.940+0000 7facebfff640 1 -- 172.21.15.81:0/525582906 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264441 (secure 0 0 0) 0x7face402ccd0 con 0x7facd406ae70 2023-11-08T16:29:13.945 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:13.946 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:29:13.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 -- 172.21.15.81:0/525582906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7facd406ae70 msgr2=0x7facd406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7facd406ae70 0x7facd406d330 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7face4036000 tx=0x7face402e070 comp rx=0 tx=0).stop 2023-11-08T16:29:13.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 -- 172.21.15.81:0/525582906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facf41f6270 msgr2=0x7facf40c74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:13.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facf41f6270 0x7facf40c74f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7facdc001270 tx=0x7facdc00dc00 comp rx=0 tx=0).stop 2023-11-08T16:29:13.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 -- 172.21.15.81:0/525582906 shutdown_connections 2023-11-08T16:29:13.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7facd406ae70 0x7facd406d330 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7facf40b1df0 0x7facf4156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7facf41f6270 0x7facf40c74f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 --2- 172.21.15.81:0/525582906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7face0003a80 0x7facf40c53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:13.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 -- 172.21.15.81:0/525582906 >> 172.21.15.81:0/525582906 conn(0x7facf40cb490 msgr2=0x7facf40ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:13.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 -- 172.21.15.81:0/525582906 shutdown_connections 2023-11-08T16:29:13.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:13.945+0000 7facf9933640 1 -- 172.21.15.81:0/525582906 wait complete. 2023-11-08T16:29:13.966 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-08T16:29:12.550395+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219096,"kb_used_data":2920,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755624,"statfs":{"total":773094113280,"available":772869758976,"internally_reserved":0,"allocated":2990080,"data_stored":1790204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.821950"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.741028+0000","last_change":"2023-11-08T16:29:03.766177+0000","last_active":"2023-11-08T16:29:04.741028+0000","last_peered":"2023-11-08T16:29:04.741028+0000","last_clean":"2023-11-08T16:29:04.741028+0000","last_became_active":"2023-11-08T16:29:03.765758+0000","last_became_peered":"2023-11-08T16:29:03.765758+0000","last_unstale":"2023-11-08T16:29:04.741028+0000","last_undegraded":"2023-11-08T16:29:04.741028+0000","last_fullsized":"2023-11-08T16:29:04.741028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:20:02.456685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538878+0000","last_change":"2023-11-08T16:29:03.775462+0000","last_active":"2023-11-08T16:29:07.538878+0000","last_peered":"2023-11-08T16:29:07.538878+0000","last_clean":"2023-11-08T16:29:07.538878+0000","last_became_active":"2023-11-08T16:29:03.775182+0000","last_became_peered":"2023-11-08T16:29:03.775182+0000","last_unstale":"2023-11-08T16:29:07.538878+0000","last_undegraded":"2023-11-08T16:29:07.538878+0000","last_fullsized":"2023-11-08T16:29:07.538878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737457+0000","last_change":"2023-11-08T16:29:03.771342+0000","last_active":"2023-11-08T16:29:06.737457+0000","last_peered":"2023-11-08T16:29:06.737457+0000","last_clean":"2023-11-08T16:29:06.737457+0000","last_became_active":"2023-11-08T16:29:03.770972+0000","last_became_peered":"2023-11-08T16:29:03.770972+0000","last_unstale":"2023-11-08T16:29:06.737457+0000","last_undegraded":"2023-11-08T16:29:06.737457+0000","last_fullsized":"2023-11-08T16:29:06.737457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:33.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738552+0000","last_change":"2023-11-08T16:29:03.762848+0000","last_active":"2023-11-08T16:29:04.738552+0000","last_peered":"2023-11-08T16:29:04.738552+0000","last_clean":"2023-11-08T16:29:04.738552+0000","last_became_active":"2023-11-08T16:29:03.762029+0000","last_became_peered":"2023-11-08T16:29:03.762029+0000","last_unstale":"2023-11-08T16:29:04.738552+0000","last_undegraded":"2023-11-08T16:29:04.738552+0000","last_fullsized":"2023-11-08T16:29:04.738552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:27.108912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739867+0000","last_change":"2023-11-08T16:29:03.758683+0000","last_active":"2023-11-08T16:29:04.739867+0000","last_peered":"2023-11-08T16:29:04.739867+0000","last_clean":"2023-11-08T16:29:04.739867+0000","last_became_active":"2023-11-08T16:29:03.758313+0000","last_became_peered":"2023-11-08T16:29:03.758313+0000","last_unstale":"2023-11-08T16:29:04.739867+0000","last_undegraded":"2023-11-08T16:29:04.739867+0000","last_fullsized":"2023-11-08T16:29:04.739867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:42.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738354+0000","last_change":"2023-11-08T16:29:03.774526+0000","last_active":"2023-11-08T16:29:04.738354+0000","last_peered":"2023-11-08T16:29:04.738354+0000","last_clean":"2023-11-08T16:29:04.738354+0000","last_became_active":"2023-11-08T16:29:03.774248+0000","last_became_peered":"2023-11-08T16:29:03.774248+0000","last_unstale":"2023-11-08T16:29:04.738354+0000","last_undegraded":"2023-11-08T16:29:04.738354+0000","last_fullsized":"2023-11-08T16:29:04.738354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:06.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738024+0000","last_change":"2023-11-08T16:29:03.775653+0000","last_active":"2023-11-08T16:29:04.738024+0000","last_peered":"2023-11-08T16:29:04.738024+0000","last_clean":"2023-11-08T16:29:04.738024+0000","last_became_active":"2023-11-08T16:29:03.774965+0000","last_became_peered":"2023-11-08T16:29:03.774965+0000","last_unstale":"2023-11-08T16:29:04.738024+0000","last_undegraded":"2023-11-08T16:29:04.738024+0000","last_fullsized":"2023-11-08T16:29:04.738024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:27.108912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735466+0000","last_change":"2023-11-08T16:29:03.767731+0000","last_active":"2023-11-08T16:29:04.735466+0000","last_peered":"2023-11-08T16:29:04.735466+0000","last_clean":"2023-11-08T16:29:04.735466+0000","last_became_active":"2023-11-08T16:29:03.767409+0000","last_became_peered":"2023-11-08T16:29:03.767409+0000","last_unstale":"2023-11-08T16:29:04.735466+0000","last_undegraded":"2023-11-08T16:29:04.735466+0000","last_fullsized":"2023-11-08T16:29:04.735466+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:44.430116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.741175+0000","last_change":"2023-11-08T16:29:03.761330+0000","last_active":"2023-11-08T16:29:04.741175+0000","last_peered":"2023-11-08T16:29:04.741175+0000","last_clean":"2023-11-08T16:29:04.741175+0000","last_became_active":"2023-11-08T16:29:03.760953+0000","last_became_peered":"2023-11-08T16:29:03.760953+0000","last_unstale":"2023-11-08T16:29:04.741175+0000","last_undegraded":"2023-11-08T16:29:04.741175+0000","last_fullsized":"2023-11-08T16:29:04.741175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:27:56.358445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.545117+0000","last_change":"2023-11-08T16:29:03.761817+0000","last_active":"2023-11-08T16:29:07.545117+0000","last_peered":"2023-11-08T16:29:07.545117+0000","last_clean":"2023-11-08T16:29:07.545117+0000","last_became_active":"2023-11-08T16:29:03.760974+0000","last_became_peered":"2023-11-08T16:29:03.760974+0000","last_unstale":"2023-11-08T16:29:07.545117+0000","last_undegraded":"2023-11-08T16:29:07.545117+0000","last_fullsized":"2023-11-08T16:29:07.545117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:06.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.544830+0000","last_change":"2023-11-08T16:29:03.765935+0000","last_active":"2023-11-08T16:29:07.544830+0000","last_peered":"2023-11-08T16:29:07.544830+0000","last_clean":"2023-11-08T16:29:07.544830+0000","last_became_active":"2023-11-08T16:29:03.765176+0000","last_became_peered":"2023-11-08T16:29:03.765176+0000","last_unstale":"2023-11-08T16:29:07.544830+0000","last_undegraded":"2023-11-08T16:29:07.544830+0000","last_fullsized":"2023-11-08T16:29:07.544830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.703359+0000","last_change":"2023-11-08T16:29:03.771863+0000","last_active":"2023-11-08T16:29:07.703359+0000","last_peered":"2023-11-08T16:29:07.703359+0000","last_clean":"2023-11-08T16:29:07.703359+0000","last_became_active":"2023-11-08T16:29:03.770022+0000","last_became_peered":"2023-11-08T16:29:03.770022+0000","last_unstale":"2023-11-08T16:29:07.703359+0000","last_undegraded":"2023-11-08T16:29:07.703359+0000","last_fullsized":"2023-11-08T16:29:07.703359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:22.242002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735985+0000","last_change":"2023-11-08T16:29:03.765640+0000","last_active":"2023-11-08T16:29:04.735985+0000","last_peered":"2023-11-08T16:29:04.735985+0000","last_clean":"2023-11-08T16:29:04.735985+0000","last_became_active":"2023-11-08T16:29:03.765361+0000","last_became_peered":"2023-11-08T16:29:03.765361+0000","last_unstale":"2023-11-08T16:29:04.735985+0000","last_undegraded":"2023-11-08T16:29:04.735985+0000","last_fullsized":"2023-11-08T16:29:04.735985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:04.870398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.736888+0000","last_change":"2023-11-08T16:29:03.773181+0000","last_active":"2023-11-08T16:29:06.736888+0000","last_peered":"2023-11-08T16:29:06.736888+0000","last_clean":"2023-11-08T16:29:06.736888+0000","last_became_active":"2023-11-08T16:29:03.772830+0000","last_became_peered":"2023-11-08T16:29:03.772830+0000","last_unstale":"2023-11-08T16:29:06.736888+0000","last_undegraded":"2023-11-08T16:29:06.736888+0000","last_fullsized":"2023-11-08T16:29:06.736888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:44.430116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538489+0000","last_change":"2023-11-08T16:29:03.770534+0000","last_active":"2023-11-08T16:29:07.538489+0000","last_peered":"2023-11-08T16:29:07.538489+0000","last_clean":"2023-11-08T16:29:07.538489+0000","last_became_active":"2023-11-08T16:29:03.770195+0000","last_became_peered":"2023-11-08T16:29:03.770195+0000","last_unstale":"2023-11-08T16:29:07.538489+0000","last_undegraded":"2023-11-08T16:29:07.538489+0000","last_fullsized":"2023-11-08T16:29:07.538489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:44.430116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.737592+0000","last_change":"2023-11-08T16:29:03.758938+0000","last_active":"2023-11-08T16:29:04.737592+0000","last_peered":"2023-11-08T16:29:04.737592+0000","last_clean":"2023-11-08T16:29:04.737592+0000","last_became_active":"2023-11-08T16:29:03.758630+0000","last_became_peered":"2023-11-08T16:29:03.758630+0000","last_unstale":"2023-11-08T16:29:04.737592+0000","last_undegraded":"2023-11-08T16:29:04.737592+0000","last_fullsized":"2023-11-08T16:29:04.737592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740206+0000","last_change":"2023-11-08T16:29:03.763570+0000","last_active":"2023-11-08T16:29:04.740206+0000","last_peered":"2023-11-08T16:29:04.740206+0000","last_clean":"2023-11-08T16:29:04.740206+0000","last_became_active":"2023-11-08T16:29:03.763210+0000","last_became_peered":"2023-11-08T16:29:03.763210+0000","last_unstale":"2023-11-08T16:29:04.740206+0000","last_undegraded":"2023-11-08T16:29:04.740206+0000","last_fullsized":"2023-11-08T16:29:04.740206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:40:59.683967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401379+0000","last_change":"2023-11-08T16:29:03.765682+0000","last_active":"2023-11-08T16:29:05.401379+0000","last_peered":"2023-11-08T16:29:05.401379+0000","last_clean":"2023-11-08T16:29:05.401379+0000","last_became_active":"2023-11-08T16:29:03.765405+0000","last_became_peered":"2023-11-08T16:29:03.765405+0000","last_unstale":"2023-11-08T16:29:05.401379+0000","last_undegraded":"2023-11-08T16:29:05.401379+0000","last_fullsized":"2023-11-08T16:29:05.401379+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:09.861000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740558+0000","last_change":"2023-11-08T16:29:03.759779+0000","last_active":"2023-11-08T16:29:04.740558+0000","last_peered":"2023-11-08T16:29:04.740558+0000","last_clean":"2023-11-08T16:29:04.740558+0000","last_became_active":"2023-11-08T16:29:03.759472+0000","last_became_peered":"2023-11-08T16:29:03.759472+0000","last_unstale":"2023-11-08T16:29:04.740558+0000","last_undegraded":"2023-11-08T16:29:04.740558+0000","last_fullsized":"2023-11-08T16:29:04.740558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:40:59.683967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.544556+0000","last_change":"2023-11-08T16:29:03.776902+0000","last_active":"2023-11-08T16:29:07.544556+0000","last_peered":"2023-11-08T16:29:07.544556+0000","last_clean":"2023-11-08T16:29:07.544556+0000","last_became_active":"2023-11-08T16:29:03.776528+0000","last_became_peered":"2023-11-08T16:29:03.776528+0000","last_unstale":"2023-11-08T16:29:07.544556+0000","last_undegraded":"2023-11-08T16:29:07.544556+0000","last_fullsized":"2023-11-08T16:29:07.544556+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:04.870398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737582+0000","last_change":"2023-11-08T16:29:03.769496+0000","last_active":"2023-11-08T16:29:06.737582+0000","last_peered":"2023-11-08T16:29:06.737582+0000","last_clean":"2023-11-08T16:29:06.737582+0000","last_became_active":"2023-11-08T16:29:03.768491+0000","last_became_peered":"2023-11-08T16:29:03.768491+0000","last_unstale":"2023-11-08T16:29:06.737582+0000","last_undegraded":"2023-11-08T16:29:06.737582+0000","last_fullsized":"2023-11-08T16:29:06.737582+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738562+0000","last_change":"2023-11-08T16:29:03.758807+0000","last_active":"2023-11-08T16:29:04.738562+0000","last_peered":"2023-11-08T16:29:04.738562+0000","last_clean":"2023-11-08T16:29:04.738562+0000","last_became_active":"2023-11-08T16:29:03.758466+0000","last_became_peered":"2023-11-08T16:29:03.758466+0000","last_unstale":"2023-11-08T16:29:04.738562+0000","last_undegraded":"2023-11-08T16:29:04.738562+0000","last_fullsized":"2023-11-08T16:29:04.738562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:10.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735439+0000","last_change":"2023-11-08T16:29:03.774206+0000","last_active":"2023-11-08T16:29:04.735439+0000","last_peered":"2023-11-08T16:29:04.735439+0000","last_clean":"2023-11-08T16:29:04.735439+0000","last_became_active":"2023-11-08T16:29:03.771583+0000","last_became_peered":"2023-11-08T16:29:03.771583+0000","last_unstale":"2023-11-08T16:29:04.735439+0000","last_undegraded":"2023-11-08T16:29:04.735439+0000","last_fullsized":"2023-11-08T16:29:04.735439+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:51.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.737820+0000","last_change":"2023-11-08T16:29:03.760386+0000","last_active":"2023-11-08T16:29:04.737820+0000","last_peered":"2023-11-08T16:29:04.737820+0000","last_clean":"2023-11-08T16:29:04.737820+0000","last_became_active":"2023-11-08T16:29:03.760050+0000","last_became_peered":"2023-11-08T16:29:03.760050+0000","last_unstale":"2023-11-08T16:29:04.737820+0000","last_undegraded":"2023-11-08T16:29:04.737820+0000","last_fullsized":"2023-11-08T16:29:04.737820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735988+0000","last_change":"2023-11-08T16:29:03.768029+0000","last_active":"2023-11-08T16:29:04.735988+0000","last_peered":"2023-11-08T16:29:04.735988+0000","last_clean":"2023-11-08T16:29:04.735988+0000","last_became_active":"2023-11-08T16:29:03.767621+0000","last_became_peered":"2023-11-08T16:29:03.767621+0000","last_unstale":"2023-11-08T16:29:04.735988+0000","last_undegraded":"2023-11-08T16:29:04.735988+0000","last_fullsized":"2023-11-08T16:29:04.735988+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.545128+0000","last_change":"2023-11-08T16:29:03.772197+0000","last_active":"2023-11-08T16:29:07.545128+0000","last_peered":"2023-11-08T16:29:07.545128+0000","last_clean":"2023-11-08T16:29:07.545128+0000","last_became_active":"2023-11-08T16:29:03.771699+0000","last_became_peered":"2023-11-08T16:29:03.771699+0000","last_unstale":"2023-11-08T16:29:07.545128+0000","last_undegraded":"2023-11-08T16:29:07.545128+0000","last_fullsized":"2023-11-08T16:29:07.545128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538799+0000","last_change":"2023-11-08T16:29:03.761215+0000","last_active":"2023-11-08T16:29:07.538799+0000","last_peered":"2023-11-08T16:29:07.538799+0000","last_clean":"2023-11-08T16:29:07.538799+0000","last_became_active":"2023-11-08T16:29:03.760855+0000","last_became_peered":"2023-11-08T16:29:03.760855+0000","last_unstale":"2023-11-08T16:29:07.538799+0000","last_undegraded":"2023-11-08T16:29:07.538799+0000","last_fullsized":"2023-11-08T16:29:07.538799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:46:52.336621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736058+0000","last_change":"2023-11-08T16:29:03.776139+0000","last_active":"2023-11-08T16:29:04.736058+0000","last_peered":"2023-11-08T16:29:04.736058+0000","last_clean":"2023-11-08T16:29:04.736058+0000","last_became_active":"2023-11-08T16:29:03.775828+0000","last_became_peered":"2023-11-08T16:29:03.775828+0000","last_unstale":"2023-11-08T16:29:04.736058+0000","last_undegraded":"2023-11-08T16:29:04.736058+0000","last_fullsized":"2023-11-08T16:29:04.736058+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737631+0000","last_change":"2023-11-08T16:29:03.774003+0000","last_active":"2023-11-08T16:29:06.737631+0000","last_peered":"2023-11-08T16:29:06.737631+0000","last_clean":"2023-11-08T16:29:06.737631+0000","last_became_active":"2023-11-08T16:29:03.772344+0000","last_became_peered":"2023-11-08T16:29:03.772344+0000","last_unstale":"2023-11-08T16:29:06.737631+0000","last_undegraded":"2023-11-08T16:29:06.737631+0000","last_fullsized":"2023-11-08T16:29:06.737631+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:27.108912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738706+0000","last_change":"2023-11-08T16:29:03.762765+0000","last_active":"2023-11-08T16:29:04.738706+0000","last_peered":"2023-11-08T16:29:04.738706+0000","last_clean":"2023-11-08T16:29:04.738706+0000","last_became_active":"2023-11-08T16:29:03.762435+0000","last_became_peered":"2023-11-08T16:29:03.762435+0000","last_unstale":"2023-11-08T16:29:04.738706+0000","last_undegraded":"2023-11-08T16:29:04.738706+0000","last_fullsized":"2023-11-08T16:29:04.738706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:58.120335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401083+0000","last_change":"2023-11-08T16:29:03.761315+0000","last_active":"2023-11-08T16:29:05.401083+0000","last_peered":"2023-11-08T16:29:05.401083+0000","last_clean":"2023-11-08T16:29:05.401083+0000","last_became_active":"2023-11-08T16:29:03.761017+0000","last_became_peered":"2023-11-08T16:29:03.761017+0000","last_unstale":"2023-11-08T16:29:05.401083+0000","last_undegraded":"2023-11-08T16:29:05.401083+0000","last_fullsized":"2023-11-08T16:29:05.401083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:46.473609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.544286+0000","last_change":"2023-11-08T16:29:03.766087+0000","last_active":"2023-11-08T16:29:07.544286+0000","last_peered":"2023-11-08T16:29:07.544286+0000","last_clean":"2023-11-08T16:29:07.544286+0000","last_became_active":"2023-11-08T16:29:03.765811+0000","last_became_peered":"2023-11-08T16:29:03.765811+0000","last_unstale":"2023-11-08T16:29:07.544286+0000","last_undegraded":"2023-11-08T16:29:07.544286+0000","last_fullsized":"2023-11-08T16:29:07.544286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:44.430116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.702817+0000","last_change":"2023-11-08T16:29:03.770480+0000","last_active":"2023-11-08T16:29:07.702817+0000","last_peered":"2023-11-08T16:29:07.702817+0000","last_clean":"2023-11-08T16:29:07.702817+0000","last_became_active":"2023-11-08T16:29:03.770166+0000","last_became_peered":"2023-11-08T16:29:03.770166+0000","last_unstale":"2023-11-08T16:29:07.702817+0000","last_undegraded":"2023-11-08T16:29:07.702817+0000","last_fullsized":"2023-11-08T16:29:07.702817+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:11.949345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736199+0000","last_change":"2023-11-08T16:29:02.745095+0000","last_active":"2023-11-08T16:29:04.736199+0000","last_peered":"2023-11-08T16:29:04.736199+0000","last_clean":"2023-11-08T16:29:04.736199+0000","last_became_active":"2023-11-08T16:29:02.744708+0000","last_became_peered":"2023-11-08T16:29:02.744708+0000","last_unstale":"2023-11-08T16:29:04.736199+0000","last_undegraded":"2023-11-08T16:29:04.736199+0000","last_fullsized":"2023-11-08T16:29:04.736199+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737953+0000","last_change":"2023-11-08T16:29:03.773134+0000","last_active":"2023-11-08T16:29:06.737953+0000","last_peered":"2023-11-08T16:29:06.737953+0000","last_clean":"2023-11-08T16:29:06.737953+0000","last_became_active":"2023-11-08T16:29:03.772787+0000","last_became_peered":"2023-11-08T16:29:03.772787+0000","last_unstale":"2023-11-08T16:29:06.737953+0000","last_undegraded":"2023-11-08T16:29:06.737953+0000","last_fullsized":"2023-11-08T16:29:06.737953+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:10.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.604720+0000","last_change":"2023-11-08T16:29:02.752072+0000","last_active":"2023-11-08T16:29:06.604720+0000","last_peered":"2023-11-08T16:29:06.604720+0000","last_clean":"2023-11-08T16:29:06.604720+0000","last_became_active":"2023-11-08T16:29:02.751752+0000","last_became_peered":"2023-11-08T16:29:02.751752+0000","last_unstale":"2023-11-08T16:29:06.604720+0000","last_undegraded":"2023-11-08T16:29:06.604720+0000","last_fullsized":"2023-11-08T16:29:06.604720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:21.228079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735516+0000","last_change":"2023-11-08T16:29:02.751815+0000","last_active":"2023-11-08T16:29:04.735516+0000","last_peered":"2023-11-08T16:29:04.735516+0000","last_clean":"2023-11-08T16:29:04.735516+0000","last_became_active":"2023-11-08T16:29:02.751501+0000","last_became_peered":"2023-11-08T16:29:02.751501+0000","last_unstale":"2023-11-08T16:29:04.735516+0000","last_undegraded":"2023-11-08T16:29:04.735516+0000","last_fullsized":"2023-11-08T16:29:04.735516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:13.053337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735399+0000","last_change":"2023-11-08T16:29:03.758235+0000","last_active":"2023-11-08T16:29:04.735399+0000","last_peered":"2023-11-08T16:29:04.735399+0000","last_clean":"2023-11-08T16:29:04.735399+0000","last_became_active":"2023-11-08T16:29:03.757937+0000","last_became_peered":"2023-11-08T16:29:03.757937+0000","last_unstale":"2023-11-08T16:29:04.735399+0000","last_undegraded":"2023-11-08T16:29:04.735399+0000","last_fullsized":"2023-11-08T16:29:04.735399+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:45:23.080055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.737829+0000","last_change":"2023-11-08T16:29:02.741267+0000","last_active":"2023-11-08T16:29:04.737829+0000","last_peered":"2023-11-08T16:29:04.737829+0000","last_clean":"2023-11-08T16:29:04.737829+0000","last_became_active":"2023-11-08T16:29:02.740048+0000","last_became_peered":"2023-11-08T16:29:02.740048+0000","last_unstale":"2023-11-08T16:29:04.737829+0000","last_undegraded":"2023-11-08T16:29:04.737829+0000","last_fullsized":"2023-11-08T16:29:04.737829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:10.935421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538501+0000","last_change":"2023-11-08T16:29:03.755015+0000","last_active":"2023-11-08T16:29:07.538501+0000","last_peered":"2023-11-08T16:29:07.538501+0000","last_clean":"2023-11-08T16:29:07.538501+0000","last_became_active":"2023-11-08T16:29:03.753403+0000","last_became_peered":"2023-11-08T16:29:03.753403+0000","last_unstale":"2023-11-08T16:29:07.538501+0000","last_undegraded":"2023-11-08T16:29:07.538501+0000","last_fullsized":"2023-11-08T16:29:07.538501+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:33.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.604895+0000","last_change":"2023-11-08T16:29:02.749431+0000","last_active":"2023-11-08T16:29:06.604895+0000","last_peered":"2023-11-08T16:29:06.604895+0000","last_clean":"2023-11-08T16:29:06.604895+0000","last_became_active":"2023-11-08T16:29:02.748783+0000","last_became_peered":"2023-11-08T16:29:02.748783+0000","last_unstale":"2023-11-08T16:29:06.604895+0000","last_undegraded":"2023-11-08T16:29:06.604895+0000","last_fullsized":"2023-11-08T16:29:06.604895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:45.459685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739258+0000","last_change":"2023-11-08T16:29:03.759712+0000","last_active":"2023-11-08T16:29:04.739258+0000","last_peered":"2023-11-08T16:29:04.739258+0000","last_clean":"2023-11-08T16:29:04.739258+0000","last_became_active":"2023-11-08T16:29:03.759328+0000","last_became_peered":"2023-11-08T16:29:03.759328+0000","last_unstale":"2023-11-08T16:29:04.739258+0000","last_undegraded":"2023-11-08T16:29:04.739258+0000","last_fullsized":"2023-11-08T16:29:04.739258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:01:42.790782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739532+0000","last_change":"2023-11-08T16:29:02.749359+0000","last_active":"2023-11-08T16:29:04.739532+0000","last_peered":"2023-11-08T16:29:04.739532+0000","last_clean":"2023-11-08T16:29:04.739532+0000","last_became_active":"2023-11-08T16:29:02.748992+0000","last_became_peered":"2023-11-08T16:29:02.748992+0000","last_unstale":"2023-11-08T16:29:04.739532+0000","last_undegraded":"2023-11-08T16:29:04.739532+0000","last_fullsized":"2023-11-08T16:29:04.739532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:29.529005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737622+0000","last_change":"2023-11-08T16:29:03.766416+0000","last_active":"2023-11-08T16:29:06.737622+0000","last_peered":"2023-11-08T16:29:06.737622+0000","last_clean":"2023-11-08T16:29:06.737622+0000","last_became_active":"2023-11-08T16:29:03.766072+0000","last_became_peered":"2023-11-08T16:29:03.766072+0000","last_unstale":"2023-11-08T16:29:06.737622+0000","last_undegraded":"2023-11-08T16:29:06.737622+0000","last_fullsized":"2023-11-08T16:29:06.737622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:03:40.780496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.605209+0000","last_change":"2023-11-08T16:29:02.747464+0000","last_active":"2023-11-08T16:29:06.605209+0000","last_peered":"2023-11-08T16:29:06.605209+0000","last_clean":"2023-11-08T16:29:06.605209+0000","last_became_active":"2023-11-08T16:29:02.747178+0000","last_became_peered":"2023-11-08T16:29:02.747178+0000","last_unstale":"2023-11-08T16:29:06.605209+0000","last_undegraded":"2023-11-08T16:29:06.605209+0000","last_fullsized":"2023-11-08T16:29:06.605209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:13.053337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.703654+0000","last_change":"2023-11-08T16:29:03.768953+0000","last_active":"2023-11-08T16:29:07.703654+0000","last_peered":"2023-11-08T16:29:07.703654+0000","last_clean":"2023-11-08T16:29:07.703654+0000","last_became_active":"2023-11-08T16:29:03.768671+0000","last_became_peered":"2023-11-08T16:29:03.768671+0000","last_unstale":"2023-11-08T16:29:07.703654+0000","last_undegraded":"2023-11-08T16:29:07.703654+0000","last_fullsized":"2023-11-08T16:29:07.703654+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:46.473609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.604768+0000","last_change":"2023-11-08T16:29:02.750178+0000","last_active":"2023-11-08T16:29:06.604768+0000","last_peered":"2023-11-08T16:29:06.604768+0000","last_clean":"2023-11-08T16:29:06.604768+0000","last_became_active":"2023-11-08T16:29:02.749883+0000","last_became_peered":"2023-11-08T16:29:02.749883+0000","last_unstale":"2023-11-08T16:29:06.604768+0000","last_undegraded":"2023-11-08T16:29:06.604768+0000","last_fullsized":"2023-11-08T16:29:06.604768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:50.401270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739830+0000","last_change":"2023-11-08T16:29:03.761268+0000","last_active":"2023-11-08T16:29:04.739830+0000","last_peered":"2023-11-08T16:29:04.739830+0000","last_clean":"2023-11-08T16:29:04.739830+0000","last_became_active":"2023-11-08T16:29:03.760923+0000","last_became_peered":"2023-11-08T16:29:03.760923+0000","last_unstale":"2023-11-08T16:29:04.739830+0000","last_undegraded":"2023-11-08T16:29:04.739830+0000","last_fullsized":"2023-11-08T16:29:04.739830+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:18:29.767162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.600133+0000","last_change":"2023-11-08T16:29:02.741262+0000","last_active":"2023-11-08T16:29:06.600133+0000","last_peered":"2023-11-08T16:29:06.600133+0000","last_clean":"2023-11-08T16:29:06.600133+0000","last_became_active":"2023-11-08T16:29:02.740954+0000","last_became_peered":"2023-11-08T16:29:02.740954+0000","last_unstale":"2023-11-08T16:29:06.600133+0000","last_undegraded":"2023-11-08T16:29:06.600133+0000","last_fullsized":"2023-11-08T16:29:06.600133+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:13.053337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737160+0000","last_change":"2023-11-08T16:29:03.772717+0000","last_active":"2023-11-08T16:29:06.737160+0000","last_peered":"2023-11-08T16:29:06.737160+0000","last_clean":"2023-11-08T16:29:06.737160+0000","last_became_active":"2023-11-08T16:29:03.772441+0000","last_became_peered":"2023-11-08T16:29:03.772441+0000","last_unstale":"2023-11-08T16:29:06.737160+0000","last_undegraded":"2023-11-08T16:29:06.737160+0000","last_fullsized":"2023-11-08T16:29:06.737160+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:30.542928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738807+0000","last_change":"2023-11-08T16:29:02.744749+0000","last_active":"2023-11-08T16:29:04.738807+0000","last_peered":"2023-11-08T16:29:04.738807+0000","last_clean":"2023-11-08T16:29:04.738807+0000","last_became_active":"2023-11-08T16:29:02.744383+0000","last_became_peered":"2023-11-08T16:29:02.744383+0000","last_unstale":"2023-11-08T16:29:04.738807+0000","last_undegraded":"2023-11-08T16:29:04.738807+0000","last_fullsized":"2023-11-08T16:29:04.738807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:32.723904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740184+0000","last_change":"2023-11-08T16:29:03.771356+0000","last_active":"2023-11-08T16:29:04.740184+0000","last_peered":"2023-11-08T16:29:04.740184+0000","last_clean":"2023-11-08T16:29:04.740184+0000","last_became_active":"2023-11-08T16:29:03.771082+0000","last_became_peered":"2023-11-08T16:29:03.771082+0000","last_unstale":"2023-11-08T16:29:04.740184+0000","last_undegraded":"2023-11-08T16:29:04.740184+0000","last_fullsized":"2023-11-08T16:29:04.740184+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:11:11.384242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.604672+0000","last_change":"2023-11-08T16:29:02.752309+0000","last_active":"2023-11-08T16:29:06.604672+0000","last_peered":"2023-11-08T16:29:06.604672+0000","last_clean":"2023-11-08T16:29:06.604672+0000","last_became_active":"2023-11-08T16:29:02.751973+0000","last_became_peered":"2023-11-08T16:29:02.751973+0000","last_unstale":"2023-11-08T16:29:06.604672+0000","last_undegraded":"2023-11-08T16:29:06.604672+0000","last_fullsized":"2023-11-08T16:29:06.604672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.544337+0000","last_change":"2023-11-08T16:29:03.759186+0000","last_active":"2023-11-08T16:29:07.544337+0000","last_peered":"2023-11-08T16:29:07.544337+0000","last_clean":"2023-11-08T16:29:07.544337+0000","last_became_active":"2023-11-08T16:29:03.758790+0000","last_became_peered":"2023-11-08T16:29:03.758790+0000","last_unstale":"2023-11-08T16:29:07.544337+0000","last_undegraded":"2023-11-08T16:29:07.544337+0000","last_fullsized":"2023-11-08T16:29:07.544337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:30.542928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.400909+0000","last_change":"2023-11-08T16:29:02.736184+0000","last_active":"2023-11-08T16:29:05.400909+0000","last_peered":"2023-11-08T16:29:05.400909+0000","last_clean":"2023-11-08T16:29:05.400909+0000","last_became_active":"2023-11-08T16:29:02.735423+0000","last_became_peered":"2023-11-08T16:29:02.735423+0000","last_unstale":"2023-11-08T16:29:05.400909+0000","last_undegraded":"2023-11-08T16:29:05.400909+0000","last_fullsized":"2023-11-08T16:29:05.400909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:46:55.176632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738431+0000","last_change":"2023-11-08T16:29:03.773617+0000","last_active":"2023-11-08T16:29:04.738431+0000","last_peered":"2023-11-08T16:29:04.738431+0000","last_clean":"2023-11-08T16:29:04.738431+0000","last_became_active":"2023-11-08T16:29:03.772381+0000","last_became_peered":"2023-11-08T16:29:03.772381+0000","last_unstale":"2023-11-08T16:29:04.738431+0000","last_undegraded":"2023-11-08T16:29:04.738431+0000","last_fullsized":"2023-11-08T16:29:04.738431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:10.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.597523+0000","last_change":"2023-11-08T16:29:02.750733+0000","last_active":"2023-11-08T16:29:06.597523+0000","last_peered":"2023-11-08T16:29:06.597523+0000","last_clean":"2023-11-08T16:29:06.597523+0000","last_became_active":"2023-11-08T16:29:02.750390+0000","last_became_peered":"2023-11-08T16:29:02.750390+0000","last_unstale":"2023-11-08T16:29:06.597523+0000","last_undegraded":"2023-11-08T16:29:06.597523+0000","last_fullsized":"2023-11-08T16:29:06.597523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739867+0000","last_change":"2023-11-08T16:29:03.772892+0000","last_active":"2023-11-08T16:29:04.739867+0000","last_peered":"2023-11-08T16:29:04.739867+0000","last_clean":"2023-11-08T16:29:04.739867+0000","last_became_active":"2023-11-08T16:29:03.772403+0000","last_became_peered":"2023-11-08T16:29:03.772403+0000","last_unstale":"2023-11-08T16:29:04.739867+0000","last_undegraded":"2023-11-08T16:29:04.739867+0000","last_fullsized":"2023-11-08T16:29:04.739867+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:58.120335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401406+0000","last_change":"2023-11-08T16:29:02.737476+0000","last_active":"2023-11-08T16:29:05.401406+0000","last_peered":"2023-11-08T16:29:05.401406+0000","last_clean":"2023-11-08T16:29:05.401406+0000","last_became_active":"2023-11-08T16:29:02.737136+0000","last_became_peered":"2023-11-08T16:29:02.737136+0000","last_unstale":"2023-11-08T16:29:05.401406+0000","last_undegraded":"2023-11-08T16:29:05.401406+0000","last_fullsized":"2023-11-08T16:29:05.401406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:57:55.731186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.741221+0000","last_change":"2023-11-08T16:29:03.768059+0000","last_active":"2023-11-08T16:29:04.741221+0000","last_peered":"2023-11-08T16:29:04.741221+0000","last_clean":"2023-11-08T16:29:04.741221+0000","last_became_active":"2023-11-08T16:29:03.767846+0000","last_became_peered":"2023-11-08T16:29:03.767846+0000","last_unstale":"2023-11-08T16:29:04.741221+0000","last_undegraded":"2023-11-08T16:29:04.741221+0000","last_fullsized":"2023-11-08T16:29:04.741221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:03:32.436141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.605274+0000","last_change":"2023-11-08T16:29:02.740588+0000","last_active":"2023-11-08T16:29:06.605274+0000","last_peered":"2023-11-08T16:29:06.605274+0000","last_clean":"2023-11-08T16:29:06.605274+0000","last_became_active":"2023-11-08T16:29:02.740216+0000","last_became_peered":"2023-11-08T16:29:02.740216+0000","last_unstale":"2023-11-08T16:29:06.605274+0000","last_undegraded":"2023-11-08T16:29:06.605274+0000","last_fullsized":"2023-11-08T16:29:06.605274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740514+0000","last_change":"2023-11-08T16:29:03.755779+0000","last_active":"2023-11-08T16:29:04.740514+0000","last_peered":"2023-11-08T16:29:04.740514+0000","last_clean":"2023-11-08T16:29:04.740514+0000","last_became_active":"2023-11-08T16:29:03.755430+0000","last_became_peered":"2023-11-08T16:29:03.755430+0000","last_unstale":"2023-11-08T16:29:04.740514+0000","last_undegraded":"2023-11-08T16:29:04.740514+0000","last_fullsized":"2023-11-08T16:29:04.740514+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:20:02.456685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401088+0000","last_change":"2023-11-08T16:29:02.744228+0000","last_active":"2023-11-08T16:29:05.401088+0000","last_peered":"2023-11-08T16:29:05.401088+0000","last_clean":"2023-11-08T16:29:05.401088+0000","last_became_active":"2023-11-08T16:29:02.743952+0000","last_became_peered":"2023-11-08T16:29:02.743952+0000","last_unstale":"2023-11-08T16:29:05.401088+0000","last_undegraded":"2023-11-08T16:29:05.401088+0000","last_fullsized":"2023-11-08T16:29:05.401088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:07.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740168+0000","last_change":"2023-11-08T16:29:03.759959+0000","last_active":"2023-11-08T16:29:04.740168+0000","last_peered":"2023-11-08T16:29:04.740168+0000","last_clean":"2023-11-08T16:29:04.740168+0000","last_became_active":"2023-11-08T16:29:03.759633+0000","last_became_peered":"2023-11-08T16:29:03.759633+0000","last_unstale":"2023-11-08T16:29:04.740168+0000","last_undegraded":"2023-11-08T16:29:04.740168+0000","last_fullsized":"2023-11-08T16:29:04.740168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:18.395303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":165,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:03.735559+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:29:03.735559+0000","last_peered":"2023-11-08T16:29:03.735559+0000","last_clean":"2023-11-08T16:29:03.735559+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:29:03.735559+0000","last_undegraded":"2023-11-08T16:29:03.735559+0000","last_fullsized":"2023-11-08T16:29:03.735559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739097+0000","last_change":"2023-11-08T16:29:02.746555+0000","last_active":"2023-11-08T16:29:04.739097+0000","last_peered":"2023-11-08T16:29:04.739097+0000","last_clean":"2023-11-08T16:29:04.739097+0000","last_became_active":"2023-11-08T16:29:02.745687+0000","last_became_peered":"2023-11-08T16:29:02.745687+0000","last_unstale":"2023-11-08T16:29:04.739097+0000","last_undegraded":"2023-11-08T16:29:04.739097+0000","last_fullsized":"2023-11-08T16:29:04.739097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:48:45.459685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736447+0000","last_change":"2023-11-08T16:29:03.767888+0000","last_active":"2023-11-08T16:29:04.736447+0000","last_peered":"2023-11-08T16:29:04.736447+0000","last_clean":"2023-11-08T16:29:04.736447+0000","last_became_active":"2023-11-08T16:29:03.767484+0000","last_became_peered":"2023-11-08T16:29:03.767484+0000","last_unstale":"2023-11-08T16:29:04.736447+0000","last_undegraded":"2023-11-08T16:29:04.736447+0000","last_fullsized":"2023-11-08T16:29:04.736447+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:10.441326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738882+0000","last_change":"2023-11-08T16:29:02.746067+0000","last_active":"2023-11-08T16:29:04.738882+0000","last_peered":"2023-11-08T16:29:04.738882+0000","last_clean":"2023-11-08T16:29:04.738882+0000","last_became_active":"2023-11-08T16:29:02.745767+0000","last_became_peered":"2023-11-08T16:29:02.745767+0000","last_unstale":"2023-11-08T16:29:04.738882+0000","last_undegraded":"2023-11-08T16:29:04.738882+0000","last_fullsized":"2023-11-08T16:29:04.738882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:10.935421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.736892+0000","last_change":"2023-11-08T16:29:03.773037+0000","last_active":"2023-11-08T16:29:06.736892+0000","last_peered":"2023-11-08T16:29:06.736892+0000","last_clean":"2023-11-08T16:29:06.736892+0000","last_became_active":"2023-11-08T16:29:03.771599+0000","last_became_peered":"2023-11-08T16:29:03.771599+0000","last_unstale":"2023-11-08T16:29:06.736892+0000","last_undegraded":"2023-11-08T16:29:06.736892+0000","last_fullsized":"2023-11-08T16:29:06.736892+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:51.415194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:05.401068+0000","last_change":"2023-11-08T16:29:02.737753+0000","last_active":"2023-11-08T16:29:05.401068+0000","last_peered":"2023-11-08T16:29:05.401068+0000","last_clean":"2023-11-08T16:29:05.401068+0000","last_became_active":"2023-11-08T16:29:02.737374+0000","last_became_peered":"2023-11-08T16:29:02.737374+0000","last_unstale":"2023-11-08T16:29:05.401068+0000","last_undegraded":"2023-11-08T16:29:05.401068+0000","last_fullsized":"2023-11-08T16:29:05.401068+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:46:10.935421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538502+0000","last_change":"2023-11-08T16:29:03.759721+0000","last_active":"2023-11-08T16:29:07.538502+0000","last_peered":"2023-11-08T16:29:07.538502+0000","last_clean":"2023-11-08T16:29:07.538502+0000","last_became_active":"2023-11-08T16:29:03.758129+0000","last_became_peered":"2023-11-08T16:29:03.758129+0000","last_unstale":"2023-11-08T16:29:07.538502+0000","last_undegraded":"2023-11-08T16:29:07.538502+0000","last_fullsized":"2023-11-08T16:29:07.538502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:30.542928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738972+0000","last_change":"2023-11-08T16:29:02.742034+0000","last_active":"2023-11-08T16:29:04.738972+0000","last_peered":"2023-11-08T16:29:04.738972+0000","last_clean":"2023-11-08T16:29:04.738972+0000","last_became_active":"2023-11-08T16:29:02.741705+0000","last_became_peered":"2023-11-08T16:29:02.741705+0000","last_unstale":"2023-11-08T16:29:04.738972+0000","last_undegraded":"2023-11-08T16:29:04.738972+0000","last_fullsized":"2023-11-08T16:29:04.738972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:03.856474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.703155+0000","last_change":"2023-11-08T16:29:03.777239+0000","last_active":"2023-11-08T16:29:07.703155+0000","last_peered":"2023-11-08T16:29:07.703155+0000","last_clean":"2023-11-08T16:29:07.703155+0000","last_became_active":"2023-11-08T16:29:03.776975+0000","last_became_peered":"2023-11-08T16:29:03.776975+0000","last_unstale":"2023-11-08T16:29:07.703155+0000","last_undegraded":"2023-11-08T16:29:07.703155+0000","last_fullsized":"2023-11-08T16:29:07.703155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:46:56.190555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.605164+0000","last_change":"2023-11-08T16:29:02.733716+0000","last_active":"2023-11-08T16:29:06.605164+0000","last_peered":"2023-11-08T16:29:06.605164+0000","last_clean":"2023-11-08T16:29:06.605164+0000","last_became_active":"2023-11-08T16:29:02.733435+0000","last_became_peered":"2023-11-08T16:29:02.733435+0000","last_unstale":"2023-11-08T16:29:06.605164+0000","last_undegraded":"2023-11-08T16:29:06.605164+0000","last_fullsized":"2023-11-08T16:29:06.605164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739982+0000","last_change":"2023-11-08T16:29:03.764171+0000","last_active":"2023-11-08T16:29:04.739982+0000","last_peered":"2023-11-08T16:29:04.739982+0000","last_clean":"2023-11-08T16:29:04.739982+0000","last_became_active":"2023-11-08T16:29:03.762549+0000","last_became_peered":"2023-11-08T16:29:03.762549+0000","last_unstale":"2023-11-08T16:29:04.739982+0000","last_undegraded":"2023-11-08T16:29:04.739982+0000","last_fullsized":"2023-11-08T16:29:04.739982+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:11:11.384242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740565+0000","last_change":"2023-11-08T16:29:02.744381+0000","last_active":"2023-11-08T16:29:04.740565+0000","last_peered":"2023-11-08T16:29:04.740565+0000","last_clean":"2023-11-08T16:29:04.740565+0000","last_became_active":"2023-11-08T16:29:02.744077+0000","last_became_peered":"2023-11-08T16:29:02.744077+0000","last_unstale":"2023-11-08T16:29:04.740565+0000","last_undegraded":"2023-11-08T16:29:04.740565+0000","last_fullsized":"2023-11-08T16:29:04.740565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:04:43.416193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538664+0000","last_change":"2023-11-08T16:29:03.753962+0000","last_active":"2023-11-08T16:29:07.538664+0000","last_peered":"2023-11-08T16:29:07.538664+0000","last_clean":"2023-11-08T16:29:07.538664+0000","last_became_active":"2023-11-08T16:29:03.753317+0000","last_became_peered":"2023-11-08T16:29:03.753317+0000","last_unstale":"2023-11-08T16:29:07.538664+0000","last_undegraded":"2023-11-08T16:29:07.538664+0000","last_fullsized":"2023-11-08T16:29:07.538664+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:04.870398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.538904+0000","last_change":"2023-11-08T16:29:02.745315+0000","last_active":"2023-11-08T16:29:07.538904+0000","last_peered":"2023-11-08T16:29:07.538904+0000","last_clean":"2023-11-08T16:29:07.538904+0000","last_became_active":"2023-11-08T16:29:02.744940+0000","last_became_peered":"2023-11-08T16:29:02.744940+0000","last_unstale":"2023-11-08T16:29:07.538904+0000","last_undegraded":"2023-11-08T16:29:07.538904+0000","last_fullsized":"2023-11-08T16:29:07.538904+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:45:22.066131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735964+0000","last_change":"2023-11-08T16:29:03.763070+0000","last_active":"2023-11-08T16:29:04.735964+0000","last_peered":"2023-11-08T16:29:04.735964+0000","last_clean":"2023-11-08T16:29:04.735964+0000","last_became_active":"2023-11-08T16:29:03.762709+0000","last_became_peered":"2023-11-08T16:29:03.762709+0000","last_unstale":"2023-11-08T16:29:04.735964+0000","last_undegraded":"2023-11-08T16:29:04.735964+0000","last_fullsized":"2023-11-08T16:29:04.735964+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:33.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.604918+0000","last_change":"2023-11-08T16:29:02.749634+0000","last_active":"2023-11-08T16:29:06.604918+0000","last_peered":"2023-11-08T16:29:06.604918+0000","last_clean":"2023-11-08T16:29:06.604918+0000","last_became_active":"2023-11-08T16:29:02.749138+0000","last_became_peered":"2023-11-08T16:29:02.749138+0000","last_unstale":"2023-11-08T16:29:06.604918+0000","last_undegraded":"2023-11-08T16:29:06.604918+0000","last_fullsized":"2023-11-08T16:29:06.604918+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737617+0000","last_change":"2023-11-08T16:29:03.773063+0000","last_active":"2023-11-08T16:29:06.737617+0000","last_peered":"2023-11-08T16:29:06.737617+0000","last_clean":"2023-11-08T16:29:06.737617+0000","last_became_active":"2023-11-08T16:29:03.772775+0000","last_became_peered":"2023-11-08T16:29:03.772775+0000","last_unstale":"2023-11-08T16:29:06.737617+0000","last_undegraded":"2023-11-08T16:29:06.737617+0000","last_fullsized":"2023-11-08T16:29:06.737617+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:06.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738667+0000","last_change":"2023-11-08T16:29:02.745343+0000","last_active":"2023-11-08T16:29:04.738667+0000","last_peered":"2023-11-08T16:29:04.738667+0000","last_clean":"2023-11-08T16:29:04.738667+0000","last_became_active":"2023-11-08T16:29:02.744884+0000","last_became_peered":"2023-11-08T16:29:02.744884+0000","last_unstale":"2023-11-08T16:29:04.738667+0000","last_undegraded":"2023-11-08T16:29:04.738667+0000","last_fullsized":"2023-11-08T16:29:04.738667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:42:57.613339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.703174+0000","last_change":"2023-11-08T16:29:03.767447+0000","last_active":"2023-11-08T16:29:07.703174+0000","last_peered":"2023-11-08T16:29:07.703174+0000","last_clean":"2023-11-08T16:29:07.703174+0000","last_became_active":"2023-11-08T16:29:03.767116+0000","last_became_peered":"2023-11-08T16:29:03.767116+0000","last_unstale":"2023-11-08T16:29:07.703174+0000","last_undegraded":"2023-11-08T16:29:07.703174+0000","last_fullsized":"2023-11-08T16:29:07.703174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:06:05.442030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.605062+0000","last_change":"2023-11-08T16:29:02.746234+0000","last_active":"2023-11-08T16:29:06.605062+0000","last_peered":"2023-11-08T16:29:06.605062+0000","last_clean":"2023-11-08T16:29:06.605062+0000","last_became_active":"2023-11-08T16:29:02.745941+0000","last_became_peered":"2023-11-08T16:29:02.745941+0000","last_unstale":"2023-11-08T16:29:06.605062+0000","last_undegraded":"2023-11-08T16:29:06.605062+0000","last_fullsized":"2023-11-08T16:29:06.605062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:35:55.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.735534+0000","last_change":"2023-11-08T16:29:03.754330+0000","last_active":"2023-11-08T16:29:04.735534+0000","last_peered":"2023-11-08T16:29:04.735534+0000","last_clean":"2023-11-08T16:29:04.735534+0000","last_became_active":"2023-11-08T16:29:03.751108+0000","last_became_peered":"2023-11-08T16:29:03.751108+0000","last_unstale":"2023-11-08T16:29:04.735534+0000","last_undegraded":"2023-11-08T16:29:04.735534+0000","last_fullsized":"2023-11-08T16:29:04.735534+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:15:30.542928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.605087+0000","last_change":"2023-11-08T16:29:02.737421+0000","last_active":"2023-11-08T16:29:06.605087+0000","last_peered":"2023-11-08T16:29:06.605087+0000","last_clean":"2023-11-08T16:29:06.605087+0000","last_became_active":"2023-11-08T16:29:02.737099+0000","last_became_peered":"2023-11-08T16:29:02.737099+0000","last_unstale":"2023-11-08T16:29:06.605087+0000","last_undegraded":"2023-11-08T16:29:06.605087+0000","last_fullsized":"2023-11-08T16:29:06.605087+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:29:07.459233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736361+0000","last_change":"2023-11-08T16:29:03.774533+0000","last_active":"2023-11-08T16:29:04.736361+0000","last_peered":"2023-11-08T16:29:04.736361+0000","last_clean":"2023-11-08T16:29:04.736361+0000","last_became_active":"2023-11-08T16:29:03.774212+0000","last_became_peered":"2023-11-08T16:29:03.774212+0000","last_unstale":"2023-11-08T16:29:04.736361+0000","last_undegraded":"2023-11-08T16:29:04.736361+0000","last_fullsized":"2023-11-08T16:29:04.736361+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:27.108912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740429+0000","last_change":"2023-11-08T16:29:02.754015+0000","last_active":"2023-11-08T16:29:04.740429+0000","last_peered":"2023-11-08T16:29:04.740429+0000","last_clean":"2023-11-08T16:29:04.740429+0000","last_became_active":"2023-11-08T16:29:02.753688+0000","last_became_peered":"2023-11-08T16:29:02.753688+0000","last_unstale":"2023-11-08T16:29:04.740429+0000","last_undegraded":"2023-11-08T16:29:04.740429+0000","last_fullsized":"2023-11-08T16:29:04.740429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:53:08.847077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:07.544487+0000","last_change":"2023-11-08T16:29:03.771420+0000","last_active":"2023-11-08T16:29:07.544487+0000","last_peered":"2023-11-08T16:29:07.544487+0000","last_clean":"2023-11-08T16:29:07.544487+0000","last_became_active":"2023-11-08T16:29:03.771137+0000","last_became_peered":"2023-11-08T16:29:03.771137+0000","last_unstale":"2023-11-08T16:29:07.544487+0000","last_undegraded":"2023-11-08T16:29:07.544487+0000","last_fullsized":"2023-11-08T16:29:07.544487+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:25:33.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.739672+0000","last_change":"2023-11-08T16:29:03.774156+0000","last_active":"2023-11-08T16:29:04.739672+0000","last_peered":"2023-11-08T16:29:04.739672+0000","last_clean":"2023-11-08T16:29:04.739672+0000","last_became_active":"2023-11-08T16:29:03.773842+0000","last_became_peered":"2023-11-08T16:29:03.773842+0000","last_unstale":"2023-11-08T16:29:04.739672+0000","last_undegraded":"2023-11-08T16:29:04.739672+0000","last_fullsized":"2023-11-08T16:29:04.739672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:11:18.395303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738809+0000","last_change":"2023-11-08T16:29:02.747528+0000","last_active":"2023-11-08T16:29:04.738809+0000","last_peered":"2023-11-08T16:29:04.738809+0000","last_clean":"2023-11-08T16:29:04.738809+0000","last_became_active":"2023-11-08T16:29:02.747113+0000","last_became_peered":"2023-11-08T16:29:02.747113+0000","last_unstale":"2023-11-08T16:29:04.738809+0000","last_undegraded":"2023-11-08T16:29:04.738809+0000","last_fullsized":"2023-11-08T16:29:04.738809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:46:55.176632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.738937+0000","last_change":"2023-11-08T16:29:03.778220+0000","last_active":"2023-11-08T16:29:04.738937+0000","last_peered":"2023-11-08T16:29:04.738937+0000","last_clean":"2023-11-08T16:29:04.738937+0000","last_became_active":"2023-11-08T16:29:03.777800+0000","last_became_peered":"2023-11-08T16:29:03.777800+0000","last_unstale":"2023-11-08T16:29:04.738937+0000","last_undegraded":"2023-11-08T16:29:04.738937+0000","last_fullsized":"2023-11-08T16:29:04.738937+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:18:29.767162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.740343+0000","last_change":"2023-11-08T16:29:02.749398+0000","last_active":"2023-11-08T16:29:04.740343+0000","last_peered":"2023-11-08T16:29:04.740343+0000","last_clean":"2023-11-08T16:29:04.740343+0000","last_became_active":"2023-11-08T16:29:02.749084+0000","last_became_peered":"2023-11-08T16:29:02.749084+0000","last_unstale":"2023-11-08T16:29:04.740343+0000","last_undegraded":"2023-11-08T16:29:04.740343+0000","last_fullsized":"2023-11-08T16:29:04.740343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:03.856474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.737570+0000","last_change":"2023-11-08T16:29:03.776424+0000","last_active":"2023-11-08T16:29:06.737570+0000","last_peered":"2023-11-08T16:29:06.737570+0000","last_clean":"2023-11-08T16:29:06.737570+0000","last_became_active":"2023-11-08T16:29:03.774967+0000","last_became_peered":"2023-11-08T16:29:03.774967+0000","last_unstale":"2023-11-08T16:29:06.737570+0000","last_undegraded":"2023-11-08T16:29:06.737570+0000","last_fullsized":"2023-11-08T16:29:06.737570+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:23:04.870398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736286+0000","last_change":"2023-11-08T16:29:02.733433+0000","last_active":"2023-11-08T16:29:04.736286+0000","last_peered":"2023-11-08T16:29:04.736286+0000","last_clean":"2023-11-08T16:29:04.736286+0000","last_became_active":"2023-11-08T16:29:02.733126+0000","last_became_peered":"2023-11-08T16:29:02.733126+0000","last_unstale":"2023-11-08T16:29:04.736286+0000","last_undegraded":"2023-11-08T16:29:04.736286+0000","last_fullsized":"2023-11-08T16:29:04.736286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:35:55.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-08T16:29:04.736155+0000","last_change":"2023-11-08T16:29:03.770442+0000","last_active":"2023-11-08T16:29:04.736155+0000","last_peered":"2023-11-08T16:29:04.736155+0000","last_clean":"2023-11-08T16:29:04.736155+0000","last_became_active":"2023-11-08T16:29:03.770158+0000","last_became_peered":"2023-11-08T16:29:03.770158+0000","last_unstale":"2023-11-08T16:29:04.736155+0000","last_undegraded":"2023-11-08T16:29:04.736155+0000","last_fullsized":"2023-11-08T16:29:04.736155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:02.700092+0000","last_clean_scrub_stamp":"2023-11-08T16:29:02.700092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:36:06.389870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-11-08T16:29:06.600398+0000","last_change":"2023-11-08T16:29:02.744925+0000","last_active":"2023-11-08T16:29:06.600398+0000","last_peered":"2023-11-08T16:29:06.600398+0000","last_clean":"2023-11-08T16:29:06.600398+0000","last_became_active":"2023-11-08T16:29:02.744574+0000","last_became_peered":"2023-11-08T16:29:02.744574+0000","last_unstale":"2023-11-08T16:29:06.600398+0000","last_undegraded":"2023-11-08T16:29:06.600398+0000","last_fullsized":"2023-11-08T16:29:06.600398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:29:01.686169+0000","last_clean_scrub_stamp":"2023-11-08T16:29:01.686169+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:35:55.091068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":815104,"data_stored":666531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2249999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0009999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.083}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.532}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.206}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4359999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":221184,"data_stored":75671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2809999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.107}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3120000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":75707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92900000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.653}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.569}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6140000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9410000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":229376,"data_stored":77193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.087}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6399999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.774}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.599}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":75581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.5369999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4980000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3050000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9409999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8959999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.536}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.962}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":75741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79100000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":823296,"data_stored":666587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0229999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2250000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5609999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:29:13.972 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-08T16:29:13.972 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-08T16:29:13.972 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-08T16:29:13.973 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-08T16:29:14.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.219+0000 7f1f127e2640 1 Processor -- start 2023-11-08T16:29:14.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.219+0000 7f1f127e2640 1 -- start start 2023-11-08T16:29:14.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.220+0000 7f1f127e2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0c15d3b0 0x7f1f0c06d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.220+0000 7f1f127e2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c06cc50 con 0x7f1f0c0cf720 2023-11-08T16:29:14.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.220+0000 7f1f127e2640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c0c90f0 con 0x7f1f0c0ca130 2023-11-08T16:29:14.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.220+0000 7f1f127e2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0adb70 con 0x7f1f0c15d3b0 2023-11-08T16:29:14.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f03fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f1f0c0cf720 0x7f1f0c0c9a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49170/0 (socket says 172.21.15.81:49170) 2023-11-08T16:29:14.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f03fff640 1 -- 172.21.15.81:0/78820299 learned_addr learned my addr 172.21.15.81:0/78820299 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:14.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f0bfff640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0c15d3b0 0x7f1f0c06d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832306980 0 0) 0x7f1f0c06cc50 con 0x7f1f0c0cf720 2023-11-08T16:29:14.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ef8003520 con 0x7f1f0c0cf720 2023-11-08T16:29:14.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4084540090 0 0) 0x7f1ef8003520 con 0x7f1f0c0cf720 2023-11-08T16:29:14.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0c15d3b0 msgr2=0x7f1f0c06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:14.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f0b7fe640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0c15d3b0 0x7f1f0c06d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 >> v1:172.21.15.93:6789/0 conn(0x7f1f0c0ca130 legacy=0x7f1f0c070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0c9360 con 0x7f1f0c0cf720 2023-11-08T16:29:14.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f1ef4001670 con 0x7f1f0c0cf720 2023-11-08T16:29:14.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 >> v1:172.21.15.81:6789/0 conn(0x7f1f0c0cf720 legacy=0x7f1f0c0c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.221+0000 7f1f0b7fe640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0c15d3b0 0x7f1ef80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f0b7fe640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ef8003f70 0x7f1ef80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f0b7fe640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef8004d00 0x7f1ef8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0adb70 con 0x7f1ef8004d00 2023-11-08T16:29:14.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef800dcd0 con 0x7f1ef8003f70 2023-11-08T16:29:14.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef800de90 con 0x7f1f0c15d3b0 2023-11-08T16:29:14.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f10d7f640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef8004d00 0x7f1ef8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f0bfff640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0c15d3b0 0x7f1ef80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f10d7f640 1 -- 172.21.15.81:0/78820299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0c15d3b0 msgr2=0x7f1ef80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:14.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f10d7f640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0c15d3b0 0x7f1ef80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f10d7f640 1 -- 172.21.15.81:0/78820299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ef8003f70 msgr2=0x7f1ef80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:14.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f10d7f640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ef8003f70 0x7f1ef80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f10d7f640 1 -- 172.21.15.81:0/78820299 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef8003520 con 0x7f1ef8004d00 2023-11-08T16:29:14.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.222+0000 7f1f0bfff640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f0c15d3b0 0x7f1ef80039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:14.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.223+0000 7f1f10d7f640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef8004d00 0x7f1ef8005180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0ce590 tx=0x7f1ef80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b123f54a9b03196 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:14.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.224+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04003b70 con 0x7f1ef8004d00 2023-11-08T16:29:14.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.224+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f04003cd0 con 0x7f1ef8004d00 2023-11-08T16:29:14.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.224+0000 7f1f0b7fe640 1 -- 172.21.15.81:0/78820299 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04005310 con 0x7f1ef8004d00 2023-11-08T16:29:14.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.224+0000 7f1f127e2640 1 -- 172.21.15.81:0/78820299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef8004d00 msgr2=0x7f1ef8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.224+0000 7f1f127e2640 1 --2- 172.21.15.81:0/78820299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ef8004d00 0x7f1ef8005180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0ce590 tx=0x7f1ef80057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.224+0000 7f1f10d7f640 1 -- 172.21.15.81:0/78820299 reap_dead start 2023-11-08T16:29:14.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.224+0000 7f1f127e2640 1 -- 172.21.15.81:0/78820299 shutdown_connections 2023-11-08T16:29:14.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.224+0000 7f1f127e2640 1 -- 172.21.15.81:0/78820299 >> 172.21.15.81:0/78820299 conn(0x7f1f0c0ceaa0 msgr2=0x7f1f0c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:14.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.225+0000 7f1f127e2640 1 -- 172.21.15.81:0/78820299 shutdown_connections 2023-11-08T16:29:14.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.225+0000 7f1f127e2640 1 -- 172.21.15.81:0/78820299 wait complete. 2023-11-08T16:29:14.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.227+0000 7f1f127e2640 1 Processor -- start 2023-11-08T16:29:14.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.227+0000 7f1f127e2640 1 -- start start 2023-11-08T16:29:14.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.228+0000 7f1f127e2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1ef8004d00 0x7f1f0c0b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.228+0000 7f1f127e2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f0c0b1620 0x7f1f0c0b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.228+0000 7f1f127e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c159020 0x7f1f0c1594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.228+0000 7f1f127e2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0c6b60 con 0x7f1f0c159020 2023-11-08T16:29:14.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.228+0000 7f1f127e2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0c5f50 con 0x7f1f0c0b1620 2023-11-08T16:29:14.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.228+0000 7f1f127e2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0c6230 con 0x7f1ef8004d00 2023-11-08T16:29:14.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.228+0000 7f1f10d7f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c159020 0x7f1f0c1594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.228+0000 7f1f10d7f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c159020 0x7f1f0c1594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35690/0 (socket says 172.21.15.81:35690) 2023-11-08T16:29:14.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.228+0000 7f1f10d7f640 1 -- 172.21.15.81:0/3529125761 learned_addr learned my addr 172.21.15.81:0/3529125761 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:14.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f0bfff640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1ef8004d00 0x7f1f0c0b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f10d7f640 1 -- 172.21.15.81:0/3529125761 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1ef8004d00 msgr2=0x7f1f0c0b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f10d7f640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1ef8004d00 0x7f1f0c0b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f10d7f640 1 -- 172.21.15.81:0/3529125761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f0c0b1620 msgr2=0x7f1f0c0b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f10d7f640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f0c0b1620 0x7f1f0c0b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f10d7f640 1 -- 172.21.15.81:0/3529125761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c0c6510 con 0x7f1f0c159020 2023-11-08T16:29:14.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f10d7f640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c159020 0x7f1f0c1594a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f0402c020 tx=0x7f1f040014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:14.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f097fa640 1 -- 172.21.15.81:0/3529125761 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0403e070 con 0x7f1f0c159020 2023-11-08T16:29:14.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f097fa640 1 -- 172.21.15.81:0/3529125761 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f04003e40 con 0x7f1f0c159020 2023-11-08T16:29:14.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f097fa640 1 -- 172.21.15.81:0/3529125761 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f04006d50 con 0x7f1f0c159020 2023-11-08T16:29:14.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f127e2640 1 -- 172.21.15.81:0/3529125761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c0c67f0 con 0x7f1f0c159020 2023-11-08T16:29:14.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.229+0000 7f1f127e2640 1 -- 172.21.15.81:0/3529125761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c0c5b70 con 0x7f1f0c159020 2023-11-08T16:29:14.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.231+0000 7f1f127e2640 1 -- 172.21.15.81:0/3529125761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c0c6d10 con 0x7f1f0c159020 2023-11-08T16:29:14.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.231+0000 7f1f097fa640 1 -- 172.21.15.81:0/3529125761 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f04007030 con 0x7f1f0c159020 2023-11-08T16:29:14.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.233+0000 7f1f097fa640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1eec06aea0 0x7f1eec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.233+0000 7f1f097fa640 1 -- 172.21.15.81:0/3529125761 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f040aa7b0 con 0x7f1f0c159020 2023-11-08T16:29:14.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.233+0000 7f1f0bfff640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1eec06aea0 0x7f1eec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.234+0000 7f1f0bfff640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1eec06aea0 0x7f1eec06d360 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009a90 tx=0x7f1efc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:14.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.237+0000 7f1f097fa640 1 -- 172.21.15.81:0/3529125761 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f0c0c6d10 con 0x7f1f0c159020 2023-11-08T16:29:14.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.458+0000 7f1f127e2640 1 -- 172.21.15.81:0/3529125761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f1f0c0743f0 con 0x7f1f0c159020 2023-11-08T16:29:14.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.458+0000 7f1f097fa640 1 -- 172.21.15.81:0/3529125761 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f1f04048080 con 0x7f1f0c159020 2023-11-08T16:29:14.460 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:14.461 INFO:teuthology.orchestra.run.smithi081.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-08T16:29:14.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 -- 172.21.15.81:0/3529125761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1eec06aea0 msgr2=0x7f1eec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1eec06aea0 0x7f1eec06d360 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009a90 tx=0x7f1efc008040 comp rx=0 tx=0).stop 2023-11-08T16:29:14.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 -- 172.21.15.81:0/3529125761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c159020 msgr2=0x7f1f0c1594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c159020 0x7f1f0c1594a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1f0402c020 tx=0x7f1f040014a0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 -- 172.21.15.81:0/3529125761 shutdown_connections 2023-11-08T16:29:14.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1eec06aea0 0x7f1eec06d360 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f0c159020 0x7f1f0c1594a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f0c0b1620 0x7f1f0c0b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 --2- 172.21.15.81:0/3529125761 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1ef8004d00 0x7f1f0c0b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 -- 172.21.15.81:0/3529125761 >> 172.21.15.81:0/3529125761 conn(0x7f1f0c0ceaa0 msgr2=0x7f1f0c0ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:14.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 -- 172.21.15.81:0/3529125761 shutdown_connections 2023-11-08T16:29:14.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.461+0000 7f1f127e2640 1 -- 172.21.15.81:0/3529125761 wait complete. 2023-11-08T16:29:14.481 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-08T16:29:14.482 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:14.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.685+0000 7fbcb2490640 1 Processor -- start 2023-11-08T16:29:14.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.685+0000 7fbcb2490640 1 -- start start 2023-11-08T16:29:14.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.687+0000 7fbcb2490640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 0x7fbcac0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.687+0000 7fbcb2490640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbcac15a680 0x7fbcac15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.687+0000 7fbcb2490640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcac06b630 con 0x7fbcac15b020 2023-11-08T16:29:14.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.687+0000 7fbcb2490640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcac0b1860 con 0x7fbcac15a680 2023-11-08T16:29:14.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.687+0000 7fbcb2490640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0ca9c0 con 0x7fbcac0ade50 2023-11-08T16:29:14.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.687+0000 7fbcabfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 0x7fbcac0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.688+0000 7fbcabfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 0x7fbcac0ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54710/0 (socket says 172.21.15.81:54710) 2023-11-08T16:29:14.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.688+0000 7fbcabfff640 1 -- 172.21.15.81:0/2851772565 learned_addr learned my addr 172.21.15.81:0/2851772565 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:14.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.688+0000 7fbca3fff640 1 --2- 172.21.15.81:0/2851772565 >> v2:172.21.15.93:3300/0 conn(0x7fbcac15a680 0x7fbcac15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.688+0000 7fbcabfff640 1 -- 172.21.15.81:0/2851772565 >> v2:172.21.15.93:3300/0 conn(0x7fbcac15a680 msgr2=0x7fbcac15aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:14.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.688+0000 7fbcabfff640 1 --2- 172.21.15.81:0/2851772565 >> v2:172.21.15.93:3300/0 conn(0x7fbcac15a680 0x7fbcac15aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.688+0000 7fbcabfff640 1 -- 172.21.15.81:0/2851772565 >> v1:172.21.15.93:6789/0 conn(0x7fbcac15b020 legacy=0x7fbcac0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.688+0000 7fbcabfff640 1 -- 172.21.15.81:0/2851772565 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcac0cabb0 con 0x7fbcac0ade50 2023-11-08T16:29:14.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.689+0000 7fbcabfff640 1 --2- 172.21.15.81:0/2851772565 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 0x7fbcac0ca480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009870 tx=0x7fbc9c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=581a40dee6cdd59 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:14.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.690+0000 7fbcab7fe640 1 -- 172.21.15.81:0/2851772565 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c030840 con 0x7fbcac0ade50 2023-11-08T16:29:14.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.690+0000 7fbcab7fe640 1 -- 172.21.15.81:0/2851772565 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9c0309a0 con 0x7fbcac0ade50 2023-11-08T16:29:14.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.690+0000 7fbcab7fe640 1 -- 172.21.15.81:0/2851772565 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c030d30 con 0x7fbcac0ade50 2023-11-08T16:29:14.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.690+0000 7fbcb2490640 1 -- 172.21.15.81:0/2851772565 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 msgr2=0x7fbcac0ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.690+0000 7fbcb2490640 1 --2- 172.21.15.81:0/2851772565 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 0x7fbcac0ca480 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c009870 tx=0x7fbc9c02b980 comp rx=0 tx=0).stop 2023-11-08T16:29:14.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.690+0000 7fbcb2490640 1 -- 172.21.15.81:0/2851772565 shutdown_connections 2023-11-08T16:29:14.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.690+0000 7fbcb2490640 1 --2- 172.21.15.81:0/2851772565 >> v2:172.21.15.93:3300/0 conn(0x7fbcac15a680 0x7fbcac15aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.691+0000 7fbcb2490640 1 --2- 172.21.15.81:0/2851772565 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 0x7fbcac0ca480 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.691+0000 7fbcb2490640 1 -- 172.21.15.81:0/2851772565 >> 172.21.15.81:0/2851772565 conn(0x7fbcac0cb490 msgr2=0x7fbcac0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:14.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.691+0000 7fbcb2490640 1 -- 172.21.15.81:0/2851772565 shutdown_connections 2023-11-08T16:29:14.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.691+0000 7fbcb2490640 1 -- 172.21.15.81:0/2851772565 wait complete. 2023-11-08T16:29:14.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.692+0000 7fbcb2490640 1 Processor -- start 2023-11-08T16:29:14.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.692+0000 7fbcb2490640 1 -- start start 2023-11-08T16:29:14.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.693+0000 7fbcb2490640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 0x7fbcac0ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.693+0000 7fbcb2490640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac15a680 0x7fbcac1f63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.693+0000 7fbcb2490640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcac15b020 0x7fbcac0c2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.693+0000 7fbcb2490640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac0c4270 con 0x7fbcac15b020 2023-11-08T16:29:14.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.693+0000 7fbcb2490640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac1f36a0 con 0x7fbcac15a680 2023-11-08T16:29:14.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.693+0000 7fbcb2490640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcac1f2e00 con 0x7fbcac0ade50 2023-11-08T16:29:14.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbcb0a2d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcac15b020 0x7fbcac0c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbcb0a2d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcac15b020 0x7fbcac0c2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35704/0 (socket says 172.21.15.81:35704) 2023-11-08T16:29:14.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbcb0a2d640 1 -- 172.21.15.81:0/3156159948 learned_addr learned my addr 172.21.15.81:0/3156159948 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:14.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbcabfff640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 0x7fbcac0ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbca3fff640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac15a680 0x7fbcac1f63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbcb0a2d640 1 -- 172.21.15.81:0/3156159948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 msgr2=0x7fbcac0ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbcb0a2d640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 0x7fbcac0ce230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbcb0a2d640 1 -- 172.21.15.81:0/3156159948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac15a680 msgr2=0x7fbcac1f63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbcb0a2d640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac15a680 0x7fbcac1f63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbcb0a2d640 1 -- 172.21.15.81:0/3156159948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c0094d0 con 0x7fbcac15b020 2023-11-08T16:29:14.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbcabfff640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 0x7fbcac0ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.694+0000 7fbcb0a2d640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcac15b020 0x7fbcac0c2540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbca400c6b0 tx=0x7fbca400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.695+0000 7fbca97fa640 1 -- 172.21.15.81:0/3156159948 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4003b70 con 0x7fbcac15b020 2023-11-08T16:29:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.695+0000 7fbca97fa640 1 -- 172.21.15.81:0/3156159948 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca4003cf0 con 0x7fbcac15b020 2023-11-08T16:29:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.695+0000 7fbca97fa640 1 -- 172.21.15.81:0/3156159948 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca4003ea0 con 0x7fbcac15b020 2023-11-08T16:29:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.695+0000 7fbcb2490640 1 -- 172.21.15.81:0/3156159948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcac1f30e0 con 0x7fbcac15b020 2023-11-08T16:29:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.695+0000 7fbcb2490640 1 -- 172.21.15.81:0/3156159948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcac1f5eb0 con 0x7fbcac15b020 2023-11-08T16:29:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.696+0000 7fbcb2490640 1 -- 172.21.15.81:0/3156159948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcac04ece0 con 0x7fbcac15b020 2023-11-08T16:29:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.697+0000 7fbca97fa640 1 -- 172.21.15.81:0/3156159948 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbca4005310 con 0x7fbcac15b020 2023-11-08T16:29:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.698+0000 7fbca97fa640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc8c06ae70 0x7fbc8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.698+0000 7fbca97fa640 1 -- 172.21.15.81:0/3156159948 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbca40893d0 con 0x7fbcac15b020 2023-11-08T16:29:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.698+0000 7fbcabfff640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc8c06ae70 0x7fbc8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.699+0000 7fbcabfff640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc8c06ae70 0x7fbc8c06d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c036000 tx=0x7fbc9c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.703+0000 7fbca97fa640 1 -- 172.21.15.81:0/3156159948 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbca4054470 con 0x7fbcac15b020 2023-11-08T16:29:14.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.883+0000 7fbcb2490640 1 -- 172.21.15.81:0/3156159948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcac0cc1d0 con 0x7fbcac15b020 2023-11-08T16:29:14.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.885+0000 7fbca97fa640 1 -- 172.21.15.81:0/3156159948 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fbca4057a90 con 0x7fbcac15b020 2023-11-08T16:29:14.888 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:14.888 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:04.727022+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:14.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 -- 172.21.15.81:0/3156159948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc8c06ae70 msgr2=0x7fbc8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc8c06ae70 0x7fbc8c06d330 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c036000 tx=0x7fbc9c008030 comp rx=0 tx=0).stop 2023-11-08T16:29:14.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 -- 172.21.15.81:0/3156159948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcac15b020 msgr2=0x7fbcac0c2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:14.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcac15b020 0x7fbcac0c2540 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fbca400c6b0 tx=0x7fbca400cb70 comp rx=0 tx=0).stop 2023-11-08T16:29:14.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 -- 172.21.15.81:0/3156159948 shutdown_connections 2023-11-08T16:29:14.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc8c06ae70 0x7fbc8c06d330 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcac15b020 0x7fbcac0c2540 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcac15a680 0x7fbcac1f63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 --2- 172.21.15.81:0/3156159948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcac0ade50 0x7fbcac0ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:14.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 -- 172.21.15.81:0/3156159948 >> 172.21.15.81:0/3156159948 conn(0x7fbcac0cb490 msgr2=0x7fbcac071140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:14.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 -- 172.21.15.81:0/3156159948 shutdown_connections 2023-11-08T16:29:14.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:14.888+0000 7fbcb2490640 1 -- 172.21.15.81:0/3156159948 wait complete. 2023-11-08T16:29:14.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:14.908 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:15.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.111+0000 7ff27ba4b640 1 Processor -- start 2023-11-08T16:29:15.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.112+0000 7ff27ba4b640 1 -- start start 2023-11-08T16:29:15.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.113+0000 7ff27ba4b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff27415bfc0 0x7ff2740caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.113+0000 7ff27ba4b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff27415f280 0x7ff2741516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.113+0000 7ff27ba4b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27406fa30 con 0x7ff2740ce0d0 2023-11-08T16:29:15.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.113+0000 7ff27ba4b640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2740cda10 con 0x7ff27415f280 2023-11-08T16:29:15.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.113+0000 7ff27ba4b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740ce4b0 con 0x7ff27415bfc0 2023-11-08T16:29:15.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.113+0000 7ff279fe8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff27415f280 0x7ff2741516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.113+0000 7ff279fe8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff27415f280 0x7ff2741516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35718/0 (socket says 172.21.15.81:35718) 2023-11-08T16:29:15.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.113+0000 7ff279fe8640 1 -- 172.21.15.81:0/3369737668 learned_addr learned my addr 172.21.15.81:0/3369737668 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:15.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.114+0000 7ff279fe8640 1 -- 172.21.15.81:0/3369737668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff27415bfc0 msgr2=0x7ff2740caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:15.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.114+0000 7ff278fe6640 1 --2- 172.21.15.81:0/3369737668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff27415bfc0 0x7ff2740caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.114+0000 7ff279fe8640 1 --2- 172.21.15.81:0/3369737668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff27415bfc0 0x7ff2740caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.114+0000 7ff279fe8640 1 -- 172.21.15.81:0/3369737668 >> v1:172.21.15.81:6789/0 conn(0x7ff2740ce0d0 legacy=0x7ff27415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.114+0000 7ff26bfff640 1 -- 172.21.15.81:0/3369737668 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867135590 0 0) 0x7ff27406fa30 con 0x7ff2740ce0d0 2023-11-08T16:29:15.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.114+0000 7ff279fe8640 1 -- 172.21.15.81:0/3369737668 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27406fca0 con 0x7ff27415f280 2023-11-08T16:29:15.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.114+0000 7ff278fe6640 1 --2- 172.21.15.81:0/3369737668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff27415bfc0 0x7ff2740caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:15.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.115+0000 7ff279fe8640 1 --2- 172.21.15.81:0/3369737668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff27415f280 0x7ff2741516e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009e50 tx=0x7ff26c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8801665007c4da2d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:15.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.115+0000 7ff26bfff640 1 -- 172.21.15.81:0/3369737668 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c02caf0 con 0x7ff27415f280 2023-11-08T16:29:15.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.115+0000 7ff26bfff640 1 -- 172.21.15.81:0/3369737668 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c02cc50 con 0x7ff27415f280 2023-11-08T16:29:15.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.115+0000 7ff26bfff640 1 -- 172.21.15.81:0/3369737668 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c0343f0 con 0x7ff27415f280 2023-11-08T16:29:15.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.116+0000 7ff27ba4b640 1 -- 172.21.15.81:0/3369737668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff27415f280 msgr2=0x7ff2741516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.116+0000 7ff27ba4b640 1 --2- 172.21.15.81:0/3369737668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff27415f280 0x7ff2741516e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009e50 tx=0x7ff26c02bf50 comp rx=0 tx=0).stop 2023-11-08T16:29:15.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.116+0000 7ff27ba4b640 1 -- 172.21.15.81:0/3369737668 shutdown_connections 2023-11-08T16:29:15.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.116+0000 7ff27ba4b640 1 --2- 172.21.15.81:0/3369737668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff27415f280 0x7ff2741516e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.116+0000 7ff27ba4b640 1 --2- 172.21.15.81:0/3369737668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff27415bfc0 0x7ff2740caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.116+0000 7ff27ba4b640 1 -- 172.21.15.81:0/3369737668 >> 172.21.15.81:0/3369737668 conn(0x7ff2740cb580 msgr2=0x7ff2740cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:15.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.116+0000 7ff27ba4b640 1 -- 172.21.15.81:0/3369737668 shutdown_connections 2023-11-08T16:29:15.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.116+0000 7ff27ba4b640 1 -- 172.21.15.81:0/3369737668 wait complete. 2023-11-08T16:29:15.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.118+0000 7ff27ba4b640 1 Processor -- start 2023-11-08T16:29:15.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.118+0000 7ff27ba4b640 1 -- start start 2023-11-08T16:29:15.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff27ba4b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2740ce0d0 0x7ff2740c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff27ba4b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff27415bfc0 0x7ff2741543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff27ba4b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff27415f280 0x7ff2741f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff27ba4b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27415a7b0 con 0x7ff27415f280 2023-11-08T16:29:15.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff27ba4b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27415aa90 con 0x7ff27415bfc0 2023-11-08T16:29:15.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff27ba4b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27415ad70 con 0x7ff2740ce0d0 2023-11-08T16:29:15.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff279fe8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff27415f280 0x7ff2741f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff279fe8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff27415f280 0x7ff2741f2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35724/0 (socket says 172.21.15.81:35724) 2023-11-08T16:29:15.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff279fe8640 1 -- 172.21.15.81:0/1783848663 learned_addr learned my addr 172.21.15.81:0/1783848663 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:15.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff279fe8640 1 -- 172.21.15.81:0/1783848663 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2740ce0d0 msgr2=0x7ff2740c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff279fe8640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2740ce0d0 0x7ff2740c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff279fe8640 1 -- 172.21.15.81:0/1783848663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff27415bfc0 msgr2=0x7ff2741543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff279fe8640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff27415bfc0 0x7ff2741543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.119+0000 7ff279fe8640 1 -- 172.21.15.81:0/1783848663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff26c009b30 con 0x7ff27415f280 2023-11-08T16:29:15.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.120+0000 7ff279fe8640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff27415f280 0x7ff2741f2dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009e50 tx=0x7ff26c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:15.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.120+0000 7ff269ffb640 1 -- 172.21.15.81:0/1783848663 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c009510 con 0x7ff27415f280 2023-11-08T16:29:15.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.120+0000 7ff269ffb640 1 -- 172.21.15.81:0/1783848663 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26c03e450 con 0x7ff27415f280 2023-11-08T16:29:15.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.120+0000 7ff269ffb640 1 -- 172.21.15.81:0/1783848663 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26c03e800 con 0x7ff27415f280 2023-11-08T16:29:15.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.120+0000 7ff27ba4b640 1 -- 172.21.15.81:0/1783848663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2740c36e0 con 0x7ff27415f280 2023-11-08T16:29:15.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.120+0000 7ff27ba4b640 1 -- 172.21.15.81:0/1783848663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2740c5da0 con 0x7ff27415f280 2023-11-08T16:29:15.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.121+0000 7ff27ba4b640 1 -- 172.21.15.81:0/1783848663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2740c3ba0 con 0x7ff27415f280 2023-11-08T16:29:15.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.122+0000 7ff269ffb640 1 -- 172.21.15.81:0/1783848663 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff26c03e960 con 0x7ff27415f280 2023-11-08T16:29:15.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.123+0000 7ff269ffb640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff25806ae70 0x7ff25806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.123+0000 7ff269ffb640 1 -- 172.21.15.81:0/1783848663 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff26c0ac5a0 con 0x7ff27415f280 2023-11-08T16:29:15.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.123+0000 7ff2797e7640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff25806ae70 0x7ff25806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.124+0000 7ff2797e7640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff25806ae70 0x7ff25806d330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff270001560 tx=0x7ff270007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:15.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.127+0000 7ff269ffb640 1 -- 172.21.15.81:0/1783848663 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff26c0775d0 con 0x7ff27415f280 2023-11-08T16:29:15.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.315+0000 7ff27ba4b640 1 -- 172.21.15.81:0/1783848663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff27404ece0 con 0x7ff27415f280 2023-11-08T16:29:15.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.316+0000 7ff269ffb640 1 -- 172.21.15.81:0/1783848663 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff26c07abf0 con 0x7ff27415f280 2023-11-08T16:29:15.318 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:15.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.318+0000 7ff27ba4b640 1 -- 172.21.15.81:0/1783848663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff25806ae70 msgr2=0x7ff25806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.318+0000 7ff27ba4b640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff25806ae70 0x7ff25806d330 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff270001560 tx=0x7ff270007930 comp rx=0 tx=0).stop 2023-11-08T16:29:15.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.318+0000 7ff27ba4b640 1 -- 172.21.15.81:0/1783848663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff27415f280 msgr2=0x7ff2741f2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.318+0000 7ff27ba4b640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff27415f280 0x7ff2741f2dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009e50 tx=0x7ff26c004a40 comp rx=0 tx=0).stop 2023-11-08T16:29:15.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.319+0000 7ff27ba4b640 1 -- 172.21.15.81:0/1783848663 shutdown_connections 2023-11-08T16:29:15.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.319+0000 7ff27ba4b640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff25806ae70 0x7ff25806d330 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.319+0000 7ff27ba4b640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff27415f280 0x7ff2741f2dc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.319+0000 7ff27ba4b640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff27415bfc0 0x7ff2741543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.319+0000 7ff27ba4b640 1 --2- 172.21.15.81:0/1783848663 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2740ce0d0 0x7ff2740c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.319+0000 7ff27ba4b640 1 -- 172.21.15.81:0/1783848663 >> 172.21.15.81:0/1783848663 conn(0x7ff2740cb580 msgr2=0x7ff2740cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:15.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.319+0000 7ff27ba4b640 1 -- 172.21.15.81:0/1783848663 shutdown_connections 2023-11-08T16:29:15.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.319+0000 7ff27ba4b640 1 -- 172.21.15.81:0/1783848663 wait complete. 2023-11-08T16:29:15.337 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:15.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.547+0000 7f58e1689640 1 Processor -- start 2023-11-08T16:29:15.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.547+0000 7f58e1689640 1 -- start start 2023-11-08T16:29:15.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.548+0000 7f58e1689640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f58dc15d000 0x7f58dc0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.548+0000 7f58e1689640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f58dc0ce1b0 0x7f58dc0d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.548+0000 7f58e1689640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc06c100 con 0x7f58dc0b8730 2023-11-08T16:29:15.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.548+0000 7f58e1689640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc06d110 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.548+0000 7f58e1689640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc06c350 con 0x7f58dc15d000 2023-11-08T16:29:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.549+0000 7f58db7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f58dc0ce1b0 0x7f58dc0d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.549+0000 7f58db7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f58dc0ce1b0 0x7f58dc0d12c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35738/0 (socket says 172.21.15.81:35738) 2023-11-08T16:29:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.549+0000 7f58db7fe640 1 -- 172.21.15.81:0/3484918394 learned_addr learned my addr 172.21.15.81:0/3484918394 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:15.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.549+0000 7f58daffd640 1 --2- 172.21.15.81:0/3484918394 >> v2:172.21.15.93:3300/0 conn(0x7f58dc15d000 0x7f58dc0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.549+0000 7f58db7fe640 1 -- 172.21.15.81:0/3484918394 >> v2:172.21.15.93:3300/0 conn(0x7f58dc15d000 msgr2=0x7f58dc0b2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:15.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.549+0000 7f58db7fe640 1 --2- 172.21.15.81:0/3484918394 >> v2:172.21.15.93:3300/0 conn(0x7f58dc15d000 0x7f58dc0b2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.549+0000 7f58db7fe640 1 -- 172.21.15.81:0/3484918394 >> v1:172.21.15.93:6789/0 conn(0x7f58dc0b8730 legacy=0x7f58dc0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.549+0000 7f58db7fe640 1 -- 172.21.15.81:0/3484918394 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc06c630 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.549+0000 7f58daffd640 1 --2- 172.21.15.81:0/3484918394 >> v2:172.21.15.93:3300/0 conn(0x7f58dc15d000 0x7f58dc0b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:15.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.551+0000 7f58db7fe640 1 --2- 172.21.15.81:0/3484918394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58dc0ce1b0 0x7f58dc0d12c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58dc06d3a0 tx=0x7f58d402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b04a8e6d1c7758d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:15.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.551+0000 7f58d9ffb640 1 -- 172.21.15.81:0/3484918394 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4030860 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.551+0000 7f58d9ffb640 1 -- 172.21.15.81:0/3484918394 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d40309c0 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.551+0000 7f58d9ffb640 1 -- 172.21.15.81:0/3484918394 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4030d50 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.552+0000 7f58e1689640 1 -- 172.21.15.81:0/3484918394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58dc0ce1b0 msgr2=0x7f58dc0d12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.552+0000 7f58e1689640 1 --2- 172.21.15.81:0/3484918394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58dc0ce1b0 0x7f58dc0d12c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58dc06d3a0 tx=0x7f58d402b9a0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.552+0000 7f58e1689640 1 -- 172.21.15.81:0/3484918394 shutdown_connections 2023-11-08T16:29:15.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.552+0000 7f58e1689640 1 --2- 172.21.15.81:0/3484918394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58dc0ce1b0 0x7f58dc0d12c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.552+0000 7f58e1689640 1 --2- 172.21.15.81:0/3484918394 >> v2:172.21.15.93:3300/0 conn(0x7f58dc15d000 0x7f58dc0b2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.552+0000 7f58e1689640 1 -- 172.21.15.81:0/3484918394 >> 172.21.15.81:0/3484918394 conn(0x7f58dc0d2e10 msgr2=0x7f58dc0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:15.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.552+0000 7f58e1689640 1 -- 172.21.15.81:0/3484918394 shutdown_connections 2023-11-08T16:29:15.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.552+0000 7f58e1689640 1 -- 172.21.15.81:0/3484918394 wait complete. 2023-11-08T16:29:15.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.554+0000 7f58e1689640 1 Processor -- start 2023-11-08T16:29:15.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.554+0000 7f58e1689640 1 -- start start 2023-11-08T16:29:15.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.555+0000 7f58e1689640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58dc0b8730 0x7f58dc155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.555+0000 7f58e1689640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58dc0ce1b0 0x7f58dc153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.555+0000 7f58e1689640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58dc15d000 0x7f58dc1f26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.555+0000 7f58e1689640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc152aa0 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.555+0000 7f58e1689640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc154770 con 0x7f58dc15d000 2023-11-08T16:29:15.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.555+0000 7f58e1689640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc154a50 con 0x7f58dc0b8730 2023-11-08T16:29:15.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.555+0000 7f58da7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58dc0ce1b0 0x7f58dc153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.555+0000 7f58da7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58dc0ce1b0 0x7f58dc153f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35748/0 (socket says 172.21.15.81:35748) 2023-11-08T16:29:15.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.555+0000 7f58da7fc640 1 -- 172.21.15.81:0/3457683190 learned_addr learned my addr 172.21.15.81:0/3457683190 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:15.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.555+0000 7f58db7fe640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58dc15d000 0x7f58dc1f26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.555+0000 7f58daffd640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58dc0b8730 0x7f58dc155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.556+0000 7f58da7fc640 1 -- 172.21.15.81:0/3457683190 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58dc0b8730 msgr2=0x7f58dc155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.556+0000 7f58da7fc640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58dc0b8730 0x7f58dc155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.556+0000 7f58da7fc640 1 -- 172.21.15.81:0/3457683190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58dc15d000 msgr2=0x7f58dc1f26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.556+0000 7f58da7fc640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58dc15d000 0x7f58dc1f26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.556+0000 7f58da7fc640 1 -- 172.21.15.81:0/3457683190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d40094d0 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.556+0000 7f58da7fc640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58dc0ce1b0 0x7f58dc153f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f58c400c450 tx=0x7f58c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:15.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.556+0000 7f58d37fe640 1 -- 172.21.15.81:0/3457683190 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c40052d0 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.556+0000 7f58d37fe640 1 -- 172.21.15.81:0/3457683190 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c4005430 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.556+0000 7f58d37fe640 1 -- 172.21.15.81:0/3457683190 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c4012870 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.556+0000 7f58e1689640 1 -- 172.21.15.81:0/3457683190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc154d30 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.556+0000 7f58e1689640 1 -- 172.21.15.81:0/3457683190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc157cc0 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.558+0000 7f58e1689640 1 -- 172.21.15.81:0/3457683190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc158210 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.558+0000 7f58d37fe640 1 -- 172.21.15.81:0/3457683190 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58c40129d0 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.559+0000 7f58d37fe640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58bc06ae70 0x7f58bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.559+0000 7f58d37fe640 1 -- 172.21.15.81:0/3457683190 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58c40890c0 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.560+0000 7f58daffd640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58bc06ae70 0x7f58bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.560+0000 7f58daffd640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58bc06ae70 0x7f58bc06d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58cc003fd0 tx=0x7f58cc0039f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:15.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.564+0000 7f58d37fe640 1 -- 172.21.15.81:0/3457683190 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58dc158210 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.752+0000 7f58e1689640 1 -- 172.21.15.81:0/3457683190 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58dc158210 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.753+0000 7f58d37fe640 1 -- 172.21.15.81:0/3457683190 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58dc158210 con 0x7f58dc0ce1b0 2023-11-08T16:29:15.755 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.755+0000 7f58e1689640 1 -- 172.21.15.81:0/3457683190 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58bc06ae70 msgr2=0x7f58bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.755+0000 7f58e1689640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58bc06ae70 0x7f58bc06d330 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f58cc003fd0 tx=0x7f58cc0039f0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.755+0000 7f58e1689640 1 -- 172.21.15.81:0/3457683190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58dc0ce1b0 msgr2=0x7f58dc153f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.755+0000 7f58e1689640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58dc0ce1b0 0x7f58dc153f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f58c400c450 tx=0x7f58c400c910 comp rx=0 tx=0).stop 2023-11-08T16:29:15.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.756+0000 7f58e1689640 1 -- 172.21.15.81:0/3457683190 shutdown_connections 2023-11-08T16:29:15.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.756+0000 7f58e1689640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58bc06ae70 0x7f58bc06d330 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.756+0000 7f58e1689640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58dc15d000 0x7f58dc1f26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.756+0000 7f58e1689640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58dc0ce1b0 0x7f58dc153f20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.756+0000 7f58e1689640 1 --2- 172.21.15.81:0/3457683190 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58dc0b8730 0x7f58dc155fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.756+0000 7f58e1689640 1 -- 172.21.15.81:0/3457683190 >> 172.21.15.81:0/3457683190 conn(0x7f58dc0d2e10 msgr2=0x7f58dc15e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:15.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.756+0000 7f58e1689640 1 -- 172.21.15.81:0/3457683190 shutdown_connections 2023-11-08T16:29:15.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.756+0000 7f58e1689640 1 -- 172.21.15.81:0/3457683190 wait complete. 2023-11-08T16:29:15.775 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:15.915 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:29:15.924 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:29:15.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.984+0000 7fb0f55c9640 1 Processor -- start 2023-11-08T16:29:15.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.984+0000 7fb0f55c9640 1 -- start start 2023-11-08T16:29:15.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.986+0000 7fb0f55c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0f00cfcc0 0x7fb0f00cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.986+0000 7fb0f55c9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb0f00d39a0 0x7fb0f00b6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.986+0000 7fb0f55c9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0f00cef70 con 0x7fb0f015d4c0 2023-11-08T16:29:15.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.986+0000 7fb0f55c9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0f00cec90 con 0x7fb0f00d39a0 2023-11-08T16:29:15.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.986+0000 7fb0f55c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00b5e70 con 0x7fb0f00cfcc0 2023-11-08T16:29:15.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.986+0000 7fb0ef7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb0f00d39a0 0x7fb0f00b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.986+0000 7fb0ef7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb0f00d39a0 0x7fb0f00b6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35762/0 (socket says 172.21.15.81:35762) 2023-11-08T16:29:15.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.986+0000 7fb0ef7fe640 1 -- 172.21.15.81:0/381338629 learned_addr learned my addr 172.21.15.81:0/381338629 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:15.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.986+0000 7fb0eeffd640 1 --1- 172.21.15.81:0/381338629 >> v1:172.21.15.93:6789/0 conn(0x7fb0f015d4c0 0x7fb0f00cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50338/0 (socket says 172.21.15.81:50338) 2023-11-08T16:29:15.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.986+0000 7fb0e7fff640 1 --2- 172.21.15.81:0/381338629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0f00cfcc0 0x7fb0f00cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.987+0000 7fb0ef7fe640 1 -- 172.21.15.81:0/381338629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0f00cfcc0 msgr2=0x7fb0f00cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:15.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.987+0000 7fb0ef7fe640 1 --2- 172.21.15.81:0/381338629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0f00cfcc0 0x7fb0f00cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.987+0000 7fb0ef7fe640 1 -- 172.21.15.81:0/381338629 >> v1:172.21.15.93:6789/0 conn(0x7fb0f015d4c0 legacy=0x7fb0f00cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.987+0000 7fb0ef7fe640 1 -- 172.21.15.81:0/381338629 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00b6150 con 0x7fb0f00d39a0 2023-11-08T16:29:15.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.988+0000 7fb0ef7fe640 1 --2- 172.21.15.81:0/381338629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f00d39a0 0x7fb0f00b6950 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009e50 tx=0x7fb0e802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcee378bc34c2594 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:15.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.989+0000 7fb0ee7fc640 1 -- 172.21.15.81:0/381338629 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e802cb90 con 0x7fb0f00d39a0 2023-11-08T16:29:15.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.989+0000 7fb0ee7fc640 1 -- 172.21.15.81:0/381338629 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e802ccf0 con 0x7fb0f00d39a0 2023-11-08T16:29:15.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.989+0000 7fb0ee7fc640 1 -- 172.21.15.81:0/381338629 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8034610 con 0x7fb0f00d39a0 2023-11-08T16:29:15.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.989+0000 7fb0f55c9640 1 -- 172.21.15.81:0/381338629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f00d39a0 msgr2=0x7fb0f00b6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:15.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.989+0000 7fb0f55c9640 1 --2- 172.21.15.81:0/381338629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f00d39a0 0x7fb0f00b6950 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009e50 tx=0x7fb0e802bff0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.989+0000 7fb0f55c9640 1 -- 172.21.15.81:0/381338629 shutdown_connections 2023-11-08T16:29:15.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.989+0000 7fb0f55c9640 1 --2- 172.21.15.81:0/381338629 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f00d39a0 0x7fb0f00b6950 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.989+0000 7fb0f55c9640 1 --2- 172.21.15.81:0/381338629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0f00cfcc0 0x7fb0f00cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:15.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.989+0000 7fb0f55c9640 1 -- 172.21.15.81:0/381338629 >> 172.21.15.81:0/381338629 conn(0x7fb0f00d2e80 msgr2=0x7fb0f00d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:15.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.990+0000 7fb0f55c9640 1 -- 172.21.15.81:0/381338629 shutdown_connections 2023-11-08T16:29:15.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.990+0000 7fb0f55c9640 1 -- 172.21.15.81:0/381338629 wait complete. 2023-11-08T16:29:15.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.991+0000 7fb0f55c9640 1 Processor -- start 2023-11-08T16:29:15.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.992+0000 7fb0f55c9640 1 -- start start 2023-11-08T16:29:15.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.992+0000 7fb0f55c9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0f00cfcc0 0x7fb0f015ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.992+0000 7fb0f55c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0f00d39a0 0x7fb0f00cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.992+0000 7fb0f55c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f015d4c0 0x7fb0f01f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:15.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.992+0000 7fb0f55c9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00ca450 con 0x7fb0f015d4c0 2023-11-08T16:29:15.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0f55c9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00ca730 con 0x7fb0f00cfcc0 2023-11-08T16:29:15.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0f55c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0f00caa10 con 0x7fb0f00d39a0 2023-11-08T16:29:15.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0ef7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f015d4c0 0x7fb0f01f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:15.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0ef7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f015d4c0 0x7fb0f01f2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35764/0 (socket says 172.21.15.81:35764) 2023-11-08T16:29:16.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0ef7fe640 1 -- 172.21.15.81:0/2308338810 learned_addr learned my addr 172.21.15.81:0/2308338810 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:16.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0e7fff640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0f00d39a0 0x7fb0f00cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0eeffd640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0f00cfcc0 0x7fb0f015ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0ef7fe640 1 -- 172.21.15.81:0/2308338810 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0f00d39a0 msgr2=0x7fb0f00cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0ef7fe640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0f00d39a0 0x7fb0f00cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0ef7fe640 1 -- 172.21.15.81:0/2308338810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0f00cfcc0 msgr2=0x7fb0f015ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0ef7fe640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0f00cfcc0 0x7fb0f015ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0ef7fe640 1 -- 172.21.15.81:0/2308338810 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e8009b30 con 0x7fb0f015d4c0 2023-11-08T16:29:16.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.993+0000 7fb0eeffd640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0f00cfcc0 0x7fb0f015ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:16.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.994+0000 7fb0ef7fe640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f015d4c0 0x7fb0f01f2560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009e50 tx=0x7fb0e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:16.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.994+0000 7fb0e77fe640 1 -- 172.21.15.81:0/2308338810 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e8009510 con 0x7fb0f015d4c0 2023-11-08T16:29:16.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.994+0000 7fb0e77fe640 1 -- 172.21.15.81:0/2308338810 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e803e450 con 0x7fb0f015d4c0 2023-11-08T16:29:16.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.994+0000 7fb0e77fe640 1 -- 172.21.15.81:0/2308338810 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e803e800 con 0x7fb0f015d4c0 2023-11-08T16:29:16.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.994+0000 7fb0f55c9640 1 -- 172.21.15.81:0/2308338810 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0f00cacf0 con 0x7fb0f015d4c0 2023-11-08T16:29:16.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.994+0000 7fb0f55c9640 1 -- 172.21.15.81:0/2308338810 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0f00cc3d0 con 0x7fb0f015d4c0 2023-11-08T16:29:16.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.995+0000 7fb0f55c9640 1 -- 172.21.15.81:0/2308338810 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0f00d2860 con 0x7fb0f015d4c0 2023-11-08T16:29:16.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.996+0000 7fb0e77fe640 1 -- 172.21.15.81:0/2308338810 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb0e803e960 con 0x7fb0f015d4c0 2023-11-08T16:29:16.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.997+0000 7fb0e77fe640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb0d006ae70 0x7fb0d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.997+0000 7fb0e77fe640 1 -- 172.21.15.81:0/2308338810 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0e80ac170 con 0x7fb0f015d4c0 2023-11-08T16:29:16.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.997+0000 7fb0eeffd640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb0d006ae70 0x7fb0d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:15.998+0000 7fb0eeffd640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb0d006ae70 0x7fb0d006d330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0007fd0 tx=0x7fb0e0007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:16.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.001+0000 7fb0e77fe640 1 -- 172.21.15.81:0/2308338810 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb0e8077180 con 0x7fb0f015d4c0 2023-11-08T16:29:16.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.191+0000 7fb0f55c9640 1 -- 172.21.15.81:0/2308338810 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0f00cb210 con 0x7fb0f015d4c0 2023-11-08T16:29:16.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.192+0000 7fb0e77fe640 1 -- 172.21.15.81:0/2308338810 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0f00cb210 con 0x7fb0f015d4c0 2023-11-08T16:29:16.194 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:16.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 -- 172.21.15.81:0/2308338810 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb0d006ae70 msgr2=0x7fb0d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb0d006ae70 0x7fb0d006d330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0007fd0 tx=0x7fb0e0007d40 comp rx=0 tx=0).stop 2023-11-08T16:29:16.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 -- 172.21.15.81:0/2308338810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f015d4c0 msgr2=0x7fb0f01f2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f015d4c0 0x7fb0f01f2560 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb0e8009e50 tx=0x7fb0e8004a40 comp rx=0 tx=0).stop 2023-11-08T16:29:16.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 -- 172.21.15.81:0/2308338810 shutdown_connections 2023-11-08T16:29:16.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb0d006ae70 0x7fb0d006d330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb0f015d4c0 0x7fb0f01f2560 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb0f00d39a0 0x7fb0f00cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 --2- 172.21.15.81:0/2308338810 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb0f00cfcc0 0x7fb0f015ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 -- 172.21.15.81:0/2308338810 >> 172.21.15.81:0/2308338810 conn(0x7fb0f00d2e80 msgr2=0x7fb0f00b6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:16.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 -- 172.21.15.81:0/2308338810 shutdown_connections 2023-11-08T16:29:16.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.195+0000 7fb0f55c9640 1 -- 172.21.15.81:0/2308338810 wait complete. 2023-11-08T16:29:16.216 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:29:16.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.417+0000 7f93cb64b640 1 Processor -- start 2023-11-08T16:29:16.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.417+0000 7f93cb64b640 1 -- start start 2023-11-08T16:29:16.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93cb64b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f93c40ade50 0x7f93c40ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93cb64b640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f93c415a680 0x7f93c415ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93cb64b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c415b050 0x7f93c40c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93cb64b640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40b1860 con 0x7f93c40ade50 2023-11-08T16:29:16.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93cb64b640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c40ca9c0 con 0x7f93c415a680 2023-11-08T16:29:16.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93cb64b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c406b630 con 0x7f93c415b050 2023-11-08T16:29:16.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c93e7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f93c40ade50 0x7f93c40ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c93e7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f93c40ade50 0x7f93c40ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35766/0 (socket says 172.21.15.81:35766) 2023-11-08T16:29:16.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c93e7640 1 -- 172.21.15.81:0/2510466758 learned_addr learned my addr 172.21.15.81:0/2510466758 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:16.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c93e7640 1 -- 172.21.15.81:0/2510466758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c415b050 msgr2=0x7f93c40c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:16.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c9be8640 1 --2- 172.21.15.81:0/2510466758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c415b050 0x7f93c40c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c8be6640 1 --2- 172.21.15.81:0/2510466758 >> v2:172.21.15.93:3300/0 conn(0x7f93c415a680 0x7f93c415ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c93e7640 1 --2- 172.21.15.81:0/2510466758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c415b050 0x7f93c40c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c93e7640 1 -- 172.21.15.81:0/2510466758 >> v2:172.21.15.93:3300/0 conn(0x7f93c415a680 msgr2=0x7f93c415ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c93e7640 1 --2- 172.21.15.81:0/2510466758 >> v2:172.21.15.93:3300/0 conn(0x7f93c415a680 0x7f93c415ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c93e7640 1 -- 172.21.15.81:0/2510466758 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c40cabb0 con 0x7f93c40ade50 2023-11-08T16:29:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c9be8640 1 --2- 172.21.15.81:0/2510466758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c415b050 0x7f93c40c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:16.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.419+0000 7f93c8be6640 1 --2- 172.21.15.81:0/2510466758 >> v2:172.21.15.93:3300/0 conn(0x7f93c415a680 0x7f93c415ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:16.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.420+0000 7f93c93e7640 1 --2- 172.21.15.81:0/2510466758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c40ade50 0x7f93c40ca480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f93c4073d00 tx=0x7f93c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139372189c01da50 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:16.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.421+0000 7f93bbfff640 1 -- 172.21.15.81:0/2510466758 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c0031870 con 0x7f93c40ade50 2023-11-08T16:29:16.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.421+0000 7f93bbfff640 1 -- 172.21.15.81:0/2510466758 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c00319d0 con 0x7f93c40ade50 2023-11-08T16:29:16.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.421+0000 7f93bbfff640 1 -- 172.21.15.81:0/2510466758 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c0031d60 con 0x7f93c40ade50 2023-11-08T16:29:16.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.421+0000 7f93cb64b640 1 -- 172.21.15.81:0/2510466758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c40ade50 msgr2=0x7f93c40ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.421+0000 7f93cb64b640 1 --2- 172.21.15.81:0/2510466758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c40ade50 0x7f93c40ca480 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f93c4073d00 tx=0x7f93c002b910 comp rx=0 tx=0).stop 2023-11-08T16:29:16.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.421+0000 7f93cb64b640 1 -- 172.21.15.81:0/2510466758 shutdown_connections 2023-11-08T16:29:16.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.421+0000 7f93cb64b640 1 --2- 172.21.15.81:0/2510466758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c415b050 0x7f93c40c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.421+0000 7f93cb64b640 1 --2- 172.21.15.81:0/2510466758 >> v2:172.21.15.93:3300/0 conn(0x7f93c415a680 0x7f93c415ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.421+0000 7f93cb64b640 1 --2- 172.21.15.81:0/2510466758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c40ade50 0x7f93c40ca480 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.421+0000 7f93cb64b640 1 -- 172.21.15.81:0/2510466758 >> 172.21.15.81:0/2510466758 conn(0x7f93c40cb490 msgr2=0x7f93c40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.422+0000 7f93cb64b640 1 -- 172.21.15.81:0/2510466758 shutdown_connections 2023-11-08T16:29:16.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.422+0000 7f93cb64b640 1 -- 172.21.15.81:0/2510466758 wait complete. 2023-11-08T16:29:16.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.423+0000 7f93cb64b640 1 Processor -- start 2023-11-08T16:29:16.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.423+0000 7f93cb64b640 1 -- start start 2023-11-08T16:29:16.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.424+0000 7f93cb64b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c40ade50 0x7f93c4157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.424+0000 7f93cb64b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c415a680 0x7f93c414ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.424+0000 7f93cb64b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93c415b050 0x7f93c414ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.424+0000 7f93cb64b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c414f490 con 0x7f93c415a680 2023-11-08T16:29:16.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.424+0000 7f93cb64b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c41f6d60 con 0x7f93c415b050 2023-11-08T16:29:16.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.424+0000 7f93cb64b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c415b6c0 con 0x7f93c40ade50 2023-11-08T16:29:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c8be6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c415a680 0x7f93c414ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c8be6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c415a680 0x7f93c414ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35770/0 (socket says 172.21.15.81:35770) 2023-11-08T16:29:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c8be6640 1 -- 172.21.15.81:0/2730943242 learned_addr learned my addr 172.21.15.81:0/2730943242 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c93e7640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c40ade50 0x7f93c4157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c9be8640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93c415b050 0x7f93c414ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c8be6640 1 -- 172.21.15.81:0/2730943242 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c40ade50 msgr2=0x7f93c4157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c8be6640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c40ade50 0x7f93c4157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c8be6640 1 -- 172.21.15.81:0/2730943242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93c415b050 msgr2=0x7f93c414ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c8be6640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93c415b050 0x7f93c414ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c8be6640 1 -- 172.21.15.81:0/2730943242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c00094d0 con 0x7f93c415a680 2023-11-08T16:29:16.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c93e7640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c40ade50 0x7f93c4157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:16.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c9be8640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93c415b050 0x7f93c414ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:16.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.425+0000 7f93c8be6640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c415a680 0x7f93c414ea10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c7c0 tx=0x7f93b400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:16.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.426+0000 7f93b9ffb640 1 -- 172.21.15.81:0/2730943242 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b4007a30 con 0x7f93c415a680 2023-11-08T16:29:16.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.426+0000 7f93b9ffb640 1 -- 172.21.15.81:0/2730943242 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b4007b90 con 0x7f93c415a680 2023-11-08T16:29:16.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.426+0000 7f93b9ffb640 1 -- 172.21.15.81:0/2730943242 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b400ce70 con 0x7f93c415a680 2023-11-08T16:29:16.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.426+0000 7f93cb64b640 1 -- 172.21.15.81:0/2730943242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c415b9a0 con 0x7f93c415a680 2023-11-08T16:29:16.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.426+0000 7f93cb64b640 1 -- 172.21.15.81:0/2730943242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c415bf60 con 0x7f93c415a680 2023-11-08T16:29:16.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.427+0000 7f93cb64b640 1 -- 172.21.15.81:0/2730943242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c415c180 con 0x7f93c415a680 2023-11-08T16:29:16.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.428+0000 7f93b9ffb640 1 -- 172.21.15.81:0/2730943242 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f93b40113f0 con 0x7f93c415a680 2023-11-08T16:29:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.429+0000 7f93b9ffb640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93a806ae70 0x7f93a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.429+0000 7f93b9ffb640 1 -- 172.21.15.81:0/2730943242 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93b4089670 con 0x7f93c415a680 2023-11-08T16:29:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.429+0000 7f93c93e7640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93a806ae70 0x7f93a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.430+0000 7f93c93e7640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93a806ae70 0x7f93a806d330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f93c0036000 tx=0x7f93c003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:16.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.437+0000 7f93b9ffb640 1 -- 172.21.15.81:0/2730943242 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f93b40547b0 con 0x7f93c415a680 2023-11-08T16:29:16.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.668+0000 7f93cb64b640 1 -- 172.21.15.81:0/2730943242 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f93c415c4e0 con 0x7f93c415a680 2023-11-08T16:29:16.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.669+0000 7f93b9ffb640 1 -- 172.21.15.81:0/2730943242 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f93b4057dd0 con 0x7f93c415a680 2023-11-08T16:29:16.671 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:16.672 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4190352668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4190352668},{"type":"v1","addr":"172.21.15.81:6835","nonce":4190352668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2636431103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2636431103},{"type":"v1","addr":"172.21.15.93:6835","nonce":2636431103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3681499690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3681499690},{"type":"v1","addr":"172.21.15.81:6837","nonce":3681499690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:29:03.181472+0000","modified":"2023-11-08T16:29:06.272041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6834/2087517381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2087517381},{"type":"v1","addr":"172.21.15.93:6834","nonce":2087517381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T16:29:16.672 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 8 2023-11-08T16:29:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.672+0000 7f93cb64b640 1 -- 172.21.15.81:0/2730943242 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93a806ae70 msgr2=0x7f93a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.672+0000 7f93cb64b640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93a806ae70 0x7f93a806d330 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f93c0036000 tx=0x7f93c003c040 comp rx=0 tx=0).stop 2023-11-08T16:29:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.672+0000 7f93cb64b640 1 -- 172.21.15.81:0/2730943242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c415a680 msgr2=0x7f93c414ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.672+0000 7f93cb64b640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c415a680 0x7f93c414ea10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f93b400c7c0 tx=0x7f93b400cc80 comp rx=0 tx=0).stop 2023-11-08T16:29:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.672+0000 7f93cb64b640 1 -- 172.21.15.81:0/2730943242 shutdown_connections 2023-11-08T16:29:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.672+0000 7f93cb64b640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f93a806ae70 0x7f93a806d330 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.672+0000 7f93cb64b640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f93c415b050 0x7f93c414ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.672+0000 7f93cb64b640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f93c415a680 0x7f93c414ea10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.672+0000 7f93cb64b640 1 --2- 172.21.15.81:0/2730943242 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f93c40ade50 0x7f93c4157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.672+0000 7f93cb64b640 1 -- 172.21.15.81:0/2730943242 >> 172.21.15.81:0/2730943242 conn(0x7f93c40cb490 msgr2=0x7f93c4073110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:16.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.672+0000 7f93cb64b640 1 -- 172.21.15.81:0/2730943242 shutdown_connections 2023-11-08T16:29:16.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.673+0000 7f93cb64b640 1 -- 172.21.15.81:0/2730943242 wait complete. 2023-11-08T16:29:16.693 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:16.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.894+0000 7fb8a35b6640 1 Processor -- start 2023-11-08T16:29:16.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.895+0000 7fb8a35b6640 1 -- start start 2023-11-08T16:29:16.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.896+0000 7fb8a35b6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb89c0ade50 0x7fb89c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.896+0000 7fb8a35b6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89c06b630 con 0x7fb89c15a680 2023-11-08T16:29:16.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.896+0000 7fb8a35b6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb89c0cabb0 con 0x7fb89c15ea60 2023-11-08T16:29:16.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.896+0000 7fb8a35b6640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb89c0b1860 con 0x7fb89c0ade50 2023-11-08T16:29:16.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.896+0000 7fb8a1352640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb89c0ade50 0x7fb89c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.896+0000 7fb8a1352640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb89c0ade50 0x7fb89c0ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35778/0 (socket says 172.21.15.81:35778) 2023-11-08T16:29:16.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.896+0000 7fb8a1352640 1 -- 172.21.15.81:0/3961628168 learned_addr learned my addr 172.21.15.81:0/3961628168 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:16.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.897+0000 7fb8a1352640 1 -- 172.21.15.81:0/3961628168 >> v1:172.21.15.93:6789/0 conn(0x7fb89c15ea60 legacy=0x7fb89c15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:16.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.897+0000 7fb8a1352640 1 -- 172.21.15.81:0/3961628168 >> v1:172.21.15.81:6789/0 conn(0x7fb89c15a680 legacy=0x7fb89c15ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.897+0000 7fb8a1352640 1 -- 172.21.15.81:0/3961628168 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89c0cae90 con 0x7fb89c0ade50 2023-11-08T16:29:16.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.898+0000 7fb8a1352640 1 --2- 172.21.15.81:0/3961628168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89c0ade50 0x7fb89c0ca480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb898009870 tx=0x7fb89802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45157ef97831ae15 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:16.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.898+0000 7fb893fff640 1 -- 172.21.15.81:0/3961628168 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898030860 con 0x7fb89c0ade50 2023-11-08T16:29:16.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.898+0000 7fb893fff640 1 -- 172.21.15.81:0/3961628168 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8980309c0 con 0x7fb89c0ade50 2023-11-08T16:29:16.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.898+0000 7fb893fff640 1 -- 172.21.15.81:0/3961628168 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898030d50 con 0x7fb89c0ade50 2023-11-08T16:29:16.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.899+0000 7fb8a35b6640 1 -- 172.21.15.81:0/3961628168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89c0ade50 msgr2=0x7fb89c0ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.899+0000 7fb8a35b6640 1 --2- 172.21.15.81:0/3961628168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89c0ade50 0x7fb89c0ca480 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb898009870 tx=0x7fb89802b9a0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.899+0000 7fb8a35b6640 1 -- 172.21.15.81:0/3961628168 shutdown_connections 2023-11-08T16:29:16.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.899+0000 7fb8a35b6640 1 --2- 172.21.15.81:0/3961628168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89c0ade50 0x7fb89c0ca480 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.899+0000 7fb8a35b6640 1 -- 172.21.15.81:0/3961628168 >> 172.21.15.81:0/3961628168 conn(0x7fb89c0cb490 msgr2=0x7fb89c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:16.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.899+0000 7fb8a35b6640 1 -- 172.21.15.81:0/3961628168 shutdown_connections 2023-11-08T16:29:16.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.899+0000 7fb8a35b6640 1 -- 172.21.15.81:0/3961628168 wait complete. 2023-11-08T16:29:16.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.901+0000 7fb8a35b6640 1 Processor -- start 2023-11-08T16:29:16.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.901+0000 7fb8a35b6640 1 -- start start 2023-11-08T16:29:16.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a35b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb89c0ade50 0x7fb89c1f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a35b6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb89c15a680 0x7fb89c1f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a35b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89c15ea60 0x7fb89c15d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a35b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c1f6a80 con 0x7fb89c15ea60 2023-11-08T16:29:16.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a35b6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c1f2e80 con 0x7fb89c15a680 2023-11-08T16:29:16.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a35b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c15e470 con 0x7fb89c0ade50 2023-11-08T16:29:16.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a1b53640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89c15ea60 0x7fb89c15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a1b53640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89c15ea60 0x7fb89c15d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35780/0 (socket says 172.21.15.81:35780) 2023-11-08T16:29:16.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a1b53640 1 -- 172.21.15.81:0/68168137 learned_addr learned my addr 172.21.15.81:0/68168137 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:16.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a0b51640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb89c15a680 0x7fb89c1f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a1352640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb89c0ade50 0x7fb89c1f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a1b53640 1 -- 172.21.15.81:0/68168137 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb89c0ade50 msgr2=0x7fb89c1f2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a1b53640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb89c0ade50 0x7fb89c1f2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a1b53640 1 -- 172.21.15.81:0/68168137 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb89c15a680 msgr2=0x7fb89c1f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:16.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a1b53640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb89c15a680 0x7fb89c1f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:16.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.902+0000 7fb8a1b53640 1 -- 172.21.15.81:0/68168137 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8980094d0 con 0x7fb89c15ea60 2023-11-08T16:29:16.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.903+0000 7fb8a1b53640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89c15ea60 0x7fb89c15d440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb89400b9a0 tx=0x7fb89400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:16.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.903+0000 7fb891ffb640 1 -- 172.21.15.81:0/68168137 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89400cba0 con 0x7fb89c15ea60 2023-11-08T16:29:16.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.903+0000 7fb891ffb640 1 -- 172.21.15.81:0/68168137 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89400cd00 con 0x7fb89c15ea60 2023-11-08T16:29:16.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.903+0000 7fb891ffb640 1 -- 172.21.15.81:0/68168137 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894014680 con 0x7fb89c15ea60 2023-11-08T16:29:16.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.903+0000 7fb8a35b6640 1 -- 172.21.15.81:0/68168137 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c15deb0 con 0x7fb89c15ea60 2023-11-08T16:29:16.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.903+0000 7fb8a35b6640 1 -- 172.21.15.81:0/68168137 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c1f6710 con 0x7fb89c15ea60 2023-11-08T16:29:16.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.904+0000 7fb8a35b6640 1 -- 172.21.15.81:0/68168137 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89c1f6bf0 con 0x7fb89c15ea60 2023-11-08T16:29:16.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.905+0000 7fb891ffb640 1 -- 172.21.15.81:0/68168137 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb89401d3b0 con 0x7fb89c15ea60 2023-11-08T16:29:16.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.906+0000 7fb891ffb640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb88006ae70 0x7fb88006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:16.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.906+0000 7fb891ffb640 1 -- 172.21.15.81:0/68168137 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb894089400 con 0x7fb89c15ea60 2023-11-08T16:29:16.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.906+0000 7fb8a1352640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb88006ae70 0x7fb88006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:16.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.907+0000 7fb8a1352640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb88006ae70 0x7fb88006d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb898036000 tx=0x7fb89803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:16.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:16.911+0000 7fb891ffb640 1 -- 172.21.15.81:0/68168137 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8940544a0 con 0x7fb89c15ea60 2023-11-08T16:29:17.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.101+0000 7fb8a35b6640 1 -- 172.21.15.81:0/68168137 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb89c04ece0 con 0x7fb89c15ea60 2023-11-08T16:29:17.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.103+0000 7fb891ffb640 1 -- 172.21.15.81:0/68168137 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fb894057ac0 con 0x7fb89c15ea60 2023-11-08T16:29:17.105 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:17.106 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:04.727022+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:17.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.106+0000 7fb8a35b6640 1 -- 172.21.15.81:0/68168137 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb88006ae70 msgr2=0x7fb88006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.106+0000 7fb8a35b6640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb88006ae70 0x7fb88006d330 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb898036000 tx=0x7fb89803b040 comp rx=0 tx=0).stop 2023-11-08T16:29:17.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.106+0000 7fb8a35b6640 1 -- 172.21.15.81:0/68168137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89c15ea60 msgr2=0x7fb89c15d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.106+0000 7fb8a35b6640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89c15ea60 0x7fb89c15d440 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb89400b9a0 tx=0x7fb89400be60 comp rx=0 tx=0).stop 2023-11-08T16:29:17.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.107+0000 7fb8a35b6640 1 -- 172.21.15.81:0/68168137 shutdown_connections 2023-11-08T16:29:17.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.107+0000 7fb8a35b6640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb88006ae70 0x7fb88006d330 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.107+0000 7fb8a35b6640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb89c15ea60 0x7fb89c15d440 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.107+0000 7fb8a35b6640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb89c15a680 0x7fb89c1f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.107+0000 7fb8a35b6640 1 --2- 172.21.15.81:0/68168137 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb89c0ade50 0x7fb89c1f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.107+0000 7fb8a35b6640 1 -- 172.21.15.81:0/68168137 >> 172.21.15.81:0/68168137 conn(0x7fb89c0cb490 msgr2=0x7fb89c0b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:17.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.107+0000 7fb8a35b6640 1 -- 172.21.15.81:0/68168137 shutdown_connections 2023-11-08T16:29:17.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.107+0000 7fb8a35b6640 1 -- 172.21.15.81:0/68168137 wait complete. 2023-11-08T16:29:17.127 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:17.128 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:17.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.349+0000 7f22b89a5640 1 Processor -- start 2023-11-08T16:29:17.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.350+0000 7f22b89a5640 1 -- start start 2023-11-08T16:29:17.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.351+0000 7f22b89a5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f22b4162ed0 0x7f22b41652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.351+0000 7f22b89a5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b40ce960 con 0x7f22b40ccc10 2023-11-08T16:29:17.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.351+0000 7f22b89a5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22b40c9480 con 0x7f22b415a0e0 2023-11-08T16:29:17.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.351+0000 7f22b89a5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22b4071980 con 0x7f22b4162ed0 2023-11-08T16:29:17.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.351+0000 7f22b1d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f22b40ccc10 0x7f22b40ca300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49206/0 (socket says 172.21.15.81:49206) 2023-11-08T16:29:17.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.351+0000 7f22b1d9b640 1 -- 172.21.15.81:0/3670995216 learned_addr learned my addr 172.21.15.81:0/3670995216 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:17.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.352+0000 7f22b2d9d640 1 --2- 172.21.15.81:0/3670995216 >> v2:172.21.15.93:3300/0 conn(0x7f22b4162ed0 0x7f22b41652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.352+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559291974 0 0) 0x7f229c001490 con 0x7f22b40ccc10 2023-11-08T16:29:17.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.352+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2294005730 con 0x7f22b40ccc10 2023-11-08T16:29:17.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.352+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3423081706 0 0) 0x7f2294005730 con 0x7f22b40ccc10 2023-11-08T16:29:17.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.352+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 >> v2:172.21.15.93:3300/0 conn(0x7f22b4162ed0 msgr2=0x7f22b41652c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:17.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.352+0000 7f22b159a640 1 --2- 172.21.15.81:0/3670995216 >> v2:172.21.15.93:3300/0 conn(0x7f22b4162ed0 0x7f22b41652c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.352+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 >> v1:172.21.15.93:6789/0 conn(0x7f22b415a0e0 legacy=0x7f22b40c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.352+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b415a4c0 con 0x7f22b40ccc10 2023-11-08T16:29:17.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.352+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f229c001b80 con 0x7f22b40ccc10 2023-11-08T16:29:17.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.352+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 >> v1:172.21.15.81:6789/0 conn(0x7f22b40ccc10 legacy=0x7f22b40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.352+0000 7f22b159a640 1 --2- 172.21.15.81:0/3670995216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2294005bf0 0x7f2294006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b159a640 1 --2- 172.21.15.81:0/3670995216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22940069d0 0x7f2294006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b159a640 1 --2- 172.21.15.81:0/3670995216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2294007390 0x7f229400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4071980 con 0x7f2294005bf0 2023-11-08T16:29:17.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229400e2a0 con 0x7f22940069d0 2023-11-08T16:29:17.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229400e460 con 0x7f2294007390 2023-11-08T16:29:17.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b259c640 1 --2- 172.21.15.81:0/3670995216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2294005bf0 0x7f2294006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b1d9b640 1 --2- 172.21.15.81:0/3670995216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22940069d0 0x7f2294006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b259c640 1 -- 172.21.15.81:0/3670995216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2294007390 msgr2=0x7f229400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:17.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b259c640 1 --2- 172.21.15.81:0/3670995216 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2294007390 0x7f229400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b259c640 1 -- 172.21.15.81:0/3670995216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22940069d0 msgr2=0x7f2294006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:17.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b259c640 1 --2- 172.21.15.81:0/3670995216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22940069d0 0x7f2294006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b259c640 1 -- 172.21.15.81:0/3670995216 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2294005730 con 0x7f2294005bf0 2023-11-08T16:29:17.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.353+0000 7f22b2d9d640 1 -- 172.21.15.81:0/3670995216 reap_dead start 2023-11-08T16:29:17.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.354+0000 7f22b259c640 1 --2- 172.21.15.81:0/3670995216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2294005bf0 0x7f2294006010 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f22a4001750 tx=0x7f2294007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc4a47536ffc2d8f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:17.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.354+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a40052d0 con 0x7f2294005bf0 2023-11-08T16:29:17.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.354+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22a402e050 con 0x7f2294005bf0 2023-11-08T16:29:17.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.354+0000 7f22b159a640 1 -- 172.21.15.81:0/3670995216 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a4003830 con 0x7f2294005bf0 2023-11-08T16:29:17.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.355+0000 7f22b89a5640 1 -- 172.21.15.81:0/3670995216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2294005bf0 msgr2=0x7f2294006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.355+0000 7f22b89a5640 1 --2- 172.21.15.81:0/3670995216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2294005bf0 0x7f2294006010 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f22a4001750 tx=0x7f2294007a00 comp rx=0 tx=0).stop 2023-11-08T16:29:17.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.355+0000 7f22b89a5640 1 -- 172.21.15.81:0/3670995216 shutdown_connections 2023-11-08T16:29:17.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.355+0000 7f22b89a5640 1 --2- 172.21.15.81:0/3670995216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2294005bf0 0x7f2294006010 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.355+0000 7f22b89a5640 1 -- 172.21.15.81:0/3670995216 >> 172.21.15.81:0/3670995216 conn(0x7f22b40cb580 msgr2=0x7f22b40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:17.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.355+0000 7f22b89a5640 1 -- 172.21.15.81:0/3670995216 shutdown_connections 2023-11-08T16:29:17.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.355+0000 7f22b89a5640 1 -- 172.21.15.81:0/3670995216 wait complete. 2023-11-08T16:29:17.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.357+0000 7f22b89a5640 1 Processor -- start 2023-11-08T16:29:17.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.357+0000 7f22b89a5640 1 -- start start 2023-11-08T16:29:17.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.358+0000 7f22b89a5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2294005bf0 0x7f22b415c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.358+0000 7f22b89a5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22b4152f00 0x7f22b4153380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.358+0000 7f22b89a5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22b40c3400 0x7f22b40c3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.358+0000 7f22b89a5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40c6880 con 0x7f22b4152f00 2023-11-08T16:29:17.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.358+0000 7f22b89a5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40c5c70 con 0x7f22b40c3400 2023-11-08T16:29:17.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.358+0000 7f22b89a5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40c5f50 con 0x7f2294005bf0 2023-11-08T16:29:17.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.358+0000 7f22b1d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22b4152f00 0x7f22b4153380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.358+0000 7f22b1d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22b4152f00 0x7f22b4153380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35798/0 (socket says 172.21.15.81:35798) 2023-11-08T16:29:17.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.358+0000 7f22b1d9b640 1 -- 172.21.15.81:0/2971325139 learned_addr learned my addr 172.21.15.81:0/2971325139 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:17.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22b2d9d640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22b40c3400 0x7f22b40c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22b259c640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2294005bf0 0x7f22b415c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22b1d9b640 1 -- 172.21.15.81:0/2971325139 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2294005bf0 msgr2=0x7f22b415c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22b1d9b640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2294005bf0 0x7f22b415c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22b1d9b640 1 -- 172.21.15.81:0/2971325139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22b40c3400 msgr2=0x7f22b40c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22b1d9b640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22b40c3400 0x7f22b40c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22b1d9b640 1 -- 172.21.15.81:0/2971325139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22b40c6230 con 0x7f22b4152f00 2023-11-08T16:29:17.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22b1d9b640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22b4152f00 0x7f22b4153380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f229c0015f0 tx=0x7f229c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:17.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22aaffd640 1 -- 172.21.15.81:0/2971325139 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00edf0 con 0x7f22b4152f00 2023-11-08T16:29:17.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22aaffd640 1 -- 172.21.15.81:0/2971325139 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c013420 con 0x7f22b4152f00 2023-11-08T16:29:17.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22aaffd640 1 -- 172.21.15.81:0/2971325139 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c0137b0 con 0x7f22b4152f00 2023-11-08T16:29:17.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22b89a5640 1 -- 172.21.15.81:0/2971325139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b40c6510 con 0x7f22b4152f00 2023-11-08T16:29:17.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.359+0000 7f22b89a5640 1 -- 172.21.15.81:0/2971325139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b415c810 con 0x7f22b4152f00 2023-11-08T16:29:17.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.360+0000 7f22b89a5640 1 -- 172.21.15.81:0/2971325139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b4071440 con 0x7f22b4152f00 2023-11-08T16:29:17.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.361+0000 7f22aaffd640 1 -- 172.21.15.81:0/2971325139 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f229c013910 con 0x7f22b4152f00 2023-11-08T16:29:17.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.362+0000 7f22aaffd640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f229806ae70 0x7f229806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.362+0000 7f22aaffd640 1 -- 172.21.15.81:0/2971325139 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f229c089c30 con 0x7f22b4152f00 2023-11-08T16:29:17.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.362+0000 7f22b259c640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f229806ae70 0x7f229806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.363+0000 7f22b259c640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f229806ae70 0x7f229806d330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f22a4033000 tx=0x7f22a4009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:17.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.368+0000 7f22aaffd640 1 -- 172.21.15.81:0/2971325139 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22b4071440 con 0x7f22b4152f00 2023-11-08T16:29:17.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.559+0000 7f22b89a5640 1 -- 172.21.15.81:0/2971325139 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22b40ce960 con 0x7f22b4152f00 2023-11-08T16:29:17.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.559+0000 7f22aaffd640 1 -- 172.21.15.81:0/2971325139 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22b40ce960 con 0x7f22b4152f00 2023-11-08T16:29:17.562 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:17.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.562+0000 7f22b89a5640 1 -- 172.21.15.81:0/2971325139 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f229806ae70 msgr2=0x7f229806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.562+0000 7f22b89a5640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f229806ae70 0x7f229806d330 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f22a4033000 tx=0x7f22a4009030 comp rx=0 tx=0).stop 2023-11-08T16:29:17.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.562+0000 7f22b89a5640 1 -- 172.21.15.81:0/2971325139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22b4152f00 msgr2=0x7f22b4153380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.562+0000 7f22b89a5640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22b4152f00 0x7f22b4153380 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f229c0015f0 tx=0x7f229c00e530 comp rx=0 tx=0).stop 2023-11-08T16:29:17.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.562+0000 7f22b89a5640 1 -- 172.21.15.81:0/2971325139 shutdown_connections 2023-11-08T16:29:17.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.562+0000 7f22b89a5640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f229806ae70 0x7f229806d330 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.562+0000 7f22b89a5640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22b40c3400 0x7f22b40c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.562+0000 7f22b89a5640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22b4152f00 0x7f22b4153380 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.562+0000 7f22b89a5640 1 --2- 172.21.15.81:0/2971325139 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2294005bf0 0x7f22b415c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.562+0000 7f22b89a5640 1 -- 172.21.15.81:0/2971325139 >> 172.21.15.81:0/2971325139 conn(0x7f22b40cb580 msgr2=0x7f22b4159060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:17.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.563+0000 7f22b89a5640 1 -- 172.21.15.81:0/2971325139 shutdown_connections 2023-11-08T16:29:17.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.563+0000 7f22b89a5640 1 -- 172.21.15.81:0/2971325139 wait complete. 2023-11-08T16:29:17.584 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:17.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.785+0000 7fbb850c9640 1 Processor -- start 2023-11-08T16:29:17.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.786+0000 7fbb850c9640 1 -- start start 2023-11-08T16:29:17.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.787+0000 7fbb850c9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbb800b51b0 0x7fbb800ca240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.787+0000 7fbb850c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb8006a7c0 0x7fbb801517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.787+0000 7fbb850c9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8015f720 con 0x7fbb800d0710 2023-11-08T16:29:17.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.787+0000 7fbb850c9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb80071a50 con 0x7fbb800b51b0 2023-11-08T16:29:17.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.787+0000 7fbb850c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8015fbb0 con 0x7fbb8006a7c0 2023-11-08T16:29:17.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.788+0000 7fbb7e59c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbb800d0710 0x7fbb8006a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49222/0 (socket says 172.21.15.81:49222) 2023-11-08T16:29:17.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.788+0000 7fbb7e59c640 1 -- 172.21.15.81:0/919902509 learned_addr learned my addr 172.21.15.81:0/919902509 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:17.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.788+0000 7fbb7ed9d640 1 --2- 172.21.15.81:0/919902509 >> v2:172.21.15.93:3300/0 conn(0x7fbb800b51b0 0x7fbb800ca240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.788+0000 7fbb7f59e640 1 --2- 172.21.15.81:0/919902509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb8006a7c0 0x7fbb801517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.788+0000 7fbb7dd9b640 1 -- 172.21.15.81:0/919902509 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122837632 0 0) 0x7fbb8015f720 con 0x7fbb800d0710 2023-11-08T16:29:17.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.788+0000 7fbb7dd9b640 1 -- 172.21.15.81:0/919902509 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb6c003520 con 0x7fbb800d0710 2023-11-08T16:29:17.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.788+0000 7fbb7dd9b640 1 -- 172.21.15.81:0/919902509 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1259757293 0 0) 0x7fbb6c003520 con 0x7fbb800d0710 2023-11-08T16:29:17.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.788+0000 7fbb7ed9d640 1 -- 172.21.15.81:0/919902509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb8006a7c0 msgr2=0x7fbb801517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:17.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.788+0000 7fbb7ed9d640 1 --2- 172.21.15.81:0/919902509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb8006a7c0 0x7fbb801517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.788+0000 7fbb7ed9d640 1 -- 172.21.15.81:0/919902509 >> v1:172.21.15.81:6789/0 conn(0x7fbb800d0710 legacy=0x7fbb8006a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.788+0000 7fbb7ed9d640 1 -- 172.21.15.81:0/919902509 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8015edf0 con 0x7fbb800b51b0 2023-11-08T16:29:17.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.790+0000 7fbb7ed9d640 1 --2- 172.21.15.81:0/919902509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb800b51b0 0x7fbb800ca240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009dc0 tx=0x7fbb7002bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=955d8a320ac3ed35 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:17.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.790+0000 7fbb7dd9b640 1 -- 172.21.15.81:0/919902509 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb68001670 con 0x7fbb800b51b0 2023-11-08T16:29:17.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.790+0000 7fbb7dd9b640 1 -- 172.21.15.81:0/919902509 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7002c880 con 0x7fbb800b51b0 2023-11-08T16:29:17.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.790+0000 7fbb7dd9b640 1 -- 172.21.15.81:0/919902509 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7002cbf0 con 0x7fbb800b51b0 2023-11-08T16:29:17.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.791+0000 7fbb850c9640 1 -- 172.21.15.81:0/919902509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb800b51b0 msgr2=0x7fbb800ca240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.791+0000 7fbb850c9640 1 --2- 172.21.15.81:0/919902509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb800b51b0 0x7fbb800ca240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009dc0 tx=0x7fbb7002bd50 comp rx=0 tx=0).stop 2023-11-08T16:29:17.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.791+0000 7fbb850c9640 1 -- 172.21.15.81:0/919902509 shutdown_connections 2023-11-08T16:29:17.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.791+0000 7fbb850c9640 1 --2- 172.21.15.81:0/919902509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb8006a7c0 0x7fbb801517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.791+0000 7fbb850c9640 1 --2- 172.21.15.81:0/919902509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb800b51b0 0x7fbb800ca240 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.791+0000 7fbb850c9640 1 -- 172.21.15.81:0/919902509 >> 172.21.15.81:0/919902509 conn(0x7fbb800cfbf0 msgr2=0x7fbb800d0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:17.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.791+0000 7fbb850c9640 1 -- 172.21.15.81:0/919902509 shutdown_connections 2023-11-08T16:29:17.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.791+0000 7fbb850c9640 1 -- 172.21.15.81:0/919902509 wait complete. 2023-11-08T16:29:17.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.793+0000 7fbb850c9640 1 Processor -- start 2023-11-08T16:29:17.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.793+0000 7fbb850c9640 1 -- start start 2023-11-08T16:29:17.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb850c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb8006a7c0 0x7fbb801f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb850c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb800b51b0 0x7fbb800c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb850c9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb800d0710 0x7fbb800c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb850c9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb80154980 con 0x7fbb8006a7c0 2023-11-08T16:29:17.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb850c9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb801f3410 con 0x7fbb800d0710 2023-11-08T16:29:17.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb850c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb801f36f0 con 0x7fbb800b51b0 2023-11-08T16:29:17.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb7ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb8006a7c0 0x7fbb801f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb7ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb8006a7c0 0x7fbb801f2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35804/0 (socket says 172.21.15.81:35804) 2023-11-08T16:29:17.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb7ed9d640 1 -- 172.21.15.81:0/2983511732 learned_addr learned my addr 172.21.15.81:0/2983511732 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:17.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb7ed9d640 1 -- 172.21.15.81:0/2983511732 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb800b51b0 msgr2=0x7fbb800c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb7ed9d640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb800b51b0 0x7fbb800c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb7f59e640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb800d0710 0x7fbb800c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb7ed9d640 1 -- 172.21.15.81:0/2983511732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb800d0710 msgr2=0x7fbb800c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb7ed9d640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb800d0710 0x7fbb800c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.794+0000 7fbb7ed9d640 1 -- 172.21.15.81:0/2983511732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb70009a70 con 0x7fbb8006a7c0 2023-11-08T16:29:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.795+0000 7fbb7ed9d640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb8006a7c0 0x7fbb801f2ac0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbb7002c360 tx=0x7fbb70004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.795+0000 7fbb777fe640 1 -- 172.21.15.81:0/2983511732 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb70009450 con 0x7fbb8006a7c0 2023-11-08T16:29:17.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.795+0000 7fbb777fe640 1 -- 172.21.15.81:0/2983511732 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb70004250 con 0x7fbb8006a7c0 2023-11-08T16:29:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.795+0000 7fbb850c9640 1 -- 172.21.15.81:0/2983511732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb801f3d20 con 0x7fbb8006a7c0 2023-11-08T16:29:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.795+0000 7fbb777fe640 1 -- 172.21.15.81:0/2983511732 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700045c0 con 0x7fbb8006a7c0 2023-11-08T16:29:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.795+0000 7fbb850c9640 1 -- 172.21.15.81:0/2983511732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb801f39d0 con 0x7fbb8006a7c0 2023-11-08T16:29:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.796+0000 7fbb850c9640 1 -- 172.21.15.81:0/2983511732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb801f4240 con 0x7fbb8006a7c0 2023-11-08T16:29:17.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.797+0000 7fbb777fe640 1 -- 172.21.15.81:0/2983511732 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb70004720 con 0x7fbb8006a7c0 2023-11-08T16:29:17.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.798+0000 7fbb777fe640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb6006ae70 0x7fbb6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:17.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.798+0000 7fbb777fe640 1 -- 172.21.15.81:0/2983511732 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb700abf20 con 0x7fbb8006a7c0 2023-11-08T16:29:17.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.798+0000 7fbb7e59c640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb6006ae70 0x7fbb6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:17.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.799+0000 7fbb7e59c640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb6006ae70 0x7fbb6006d330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb680011e0 tx=0x7fbb68001a00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:17.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.802+0000 7fbb777fe640 1 -- 172.21.15.81:0/2983511732 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb801f4240 con 0x7fbb8006a7c0 2023-11-08T16:29:17.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.992+0000 7fbb850c9640 1 -- 172.21.15.81:0/2983511732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb801f4240 con 0x7fbb8006a7c0 2023-11-08T16:29:17.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.993+0000 7fbb777fe640 1 -- 172.21.15.81:0/2983511732 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb7004d020 con 0x7fbb8006a7c0 2023-11-08T16:29:17.995 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:17.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.995+0000 7fbb850c9640 1 -- 172.21.15.81:0/2983511732 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb6006ae70 msgr2=0x7fbb6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.995+0000 7fbb850c9640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb6006ae70 0x7fbb6006d330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbb680011e0 tx=0x7fbb68001a00 comp rx=0 tx=0).stop 2023-11-08T16:29:17.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.995+0000 7fbb850c9640 1 -- 172.21.15.81:0/2983511732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb8006a7c0 msgr2=0x7fbb801f2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:17.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.995+0000 7fbb850c9640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb8006a7c0 0x7fbb801f2ac0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbb7002c360 tx=0x7fbb70004a40 comp rx=0 tx=0).stop 2023-11-08T16:29:17.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.996+0000 7fbb850c9640 1 -- 172.21.15.81:0/2983511732 shutdown_connections 2023-11-08T16:29:17.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.996+0000 7fbb850c9640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb6006ae70 0x7fbb6006d330 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.996+0000 7fbb850c9640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb800d0710 0x7fbb800c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.996+0000 7fbb850c9640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb800b51b0 0x7fbb800c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.996+0000 7fbb850c9640 1 --2- 172.21.15.81:0/2983511732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb8006a7c0 0x7fbb801f2ac0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:17.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.996+0000 7fbb850c9640 1 -- 172.21.15.81:0/2983511732 >> 172.21.15.81:0/2983511732 conn(0x7fbb800cfbf0 msgr2=0x7fbb800b43b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:17.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.996+0000 7fbb850c9640 1 -- 172.21.15.81:0/2983511732 shutdown_connections 2023-11-08T16:29:18.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:17.996+0000 7fbb850c9640 1 -- 172.21.15.81:0/2983511732 wait complete. 2023-11-08T16:29:18.015 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:18.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.233+0000 7f2e0b59e640 1 Processor -- start 2023-11-08T16:29:18.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.233+0000 7f2e0b59e640 1 -- start start 2023-11-08T16:29:18.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.234+0000 7f2e0b59e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e0c0c97b0 0x7f2e0c06fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:18.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.235+0000 7f2e0b59e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0c15e580 con 0x7f2e0c070360 2023-11-08T16:29:18.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.235+0000 7f2e0b59e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0c0c9cb0 con 0x7f2e0c15e1a0 2023-11-08T16:29:18.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.235+0000 7f2e0b59e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c0ca830 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.235+0000 7f2e0a59c640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f2e0c15e1a0 0x7f2e0c0caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50370/0 (socket says 172.21.15.81:50370) 2023-11-08T16:29:18.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.235+0000 7f2e0a59c640 1 -- 172.21.15.81:0/1011994458 learned_addr learned my addr 172.21.15.81:0/1011994458 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:18.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.235+0000 7f2e03fff640 1 --2- 172.21.15.81:0/1011994458 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e0c0c97b0 0x7f2e0c06fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:18.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.236+0000 7f2e09d9b640 1 -- 172.21.15.81:0/1011994458 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524265675 0 0) 0x7f2e0c15e580 con 0x7f2e0c070360 2023-11-08T16:29:18.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.236+0000 7f2e09d9b640 1 -- 172.21.15.81:0/1011994458 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2df8003520 con 0x7f2e0c070360 2023-11-08T16:29:18.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.236+0000 7f2e09d9b640 1 -- 172.21.15.81:0/1011994458 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155849745 0 0) 0x7f2e0c0c9cb0 con 0x7f2e0c15e1a0 2023-11-08T16:29:18.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.236+0000 7f2e09d9b640 1 -- 172.21.15.81:0/1011994458 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e0c15e580 con 0x7f2e0c15e1a0 2023-11-08T16:29:18.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.236+0000 7f2e03fff640 1 -- 172.21.15.81:0/1011994458 >> v1:172.21.15.93:6789/0 conn(0x7f2e0c15e1a0 legacy=0x7f2e0c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.236+0000 7f2e03fff640 1 -- 172.21.15.81:0/1011994458 >> v1:172.21.15.81:6789/0 conn(0x7f2e0c070360 legacy=0x7f2e0c06bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.236+0000 7f2e03fff640 1 -- 172.21.15.81:0/1011994458 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0c0c9f90 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.237+0000 7f2e03fff640 1 --2- 172.21.15.81:0/1011994458 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e0c0c97b0 0x7f2e0c06fe20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2df40097c0 tx=0x7f2df402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9222f725a29602c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:18.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.238+0000 7f2e09d9b640 1 -- 172.21.15.81:0/1011994458 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df4031800 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.238+0000 7f2e09d9b640 1 -- 172.21.15.81:0/1011994458 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df4031960 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.238+0000 7f2e09d9b640 1 -- 172.21.15.81:0/1011994458 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df4031cf0 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.238+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1011994458 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e0c0c97b0 msgr2=0x7f2e0c06fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.238+0000 7f2e0b59e640 1 --2- 172.21.15.81:0/1011994458 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e0c0c97b0 0x7f2e0c06fe20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2df40097c0 tx=0x7f2df402b8a0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.239+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1011994458 shutdown_connections 2023-11-08T16:29:18.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.239+0000 7f2e0b59e640 1 --2- 172.21.15.81:0/1011994458 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e0c0c97b0 0x7f2e0c06fe20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.239+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1011994458 >> 172.21.15.81:0/1011994458 conn(0x7f2e0c0ce8d0 msgr2=0x7f2e0c0cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:18.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.239+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1011994458 shutdown_connections 2023-11-08T16:29:18.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.239+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1011994458 wait complete. 2023-11-08T16:29:18.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.241+0000 7f2e0b59e640 1 Processor -- start 2023-11-08T16:29:18.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.241+0000 7f2e0b59e640 1 -- start start 2023-11-08T16:29:18.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.242+0000 7f2e0b59e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e0c070360 0x7f2e0c159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:18.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.242+0000 7f2e0b59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e0c0c97b0 0x7f2e0c1536d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:18.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.242+0000 7f2e0b59e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e0c15e1a0 0x7f2e0c155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:18.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.242+0000 7f2e0b59e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c156670 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.242+0000 7f2e0b59e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c155ba0 con 0x7f2e0c070360 2023-11-08T16:29:18.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.242+0000 7f2e0b59e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0c0c6f60 con 0x7f2e0c15e1a0 2023-11-08T16:29:18.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.243+0000 7f2e03fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e0c0c97b0 0x7f2e0c1536d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:18.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.243+0000 7f2e03fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e0c0c97b0 0x7f2e0c1536d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35814/0 (socket says 172.21.15.81:35814) 2023-11-08T16:29:18.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.243+0000 7f2e03fff640 1 -- 172.21.15.81:0/1543292165 learned_addr learned my addr 172.21.15.81:0/1543292165 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:18.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.243+0000 7f2e0a59c640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e0c070360 0x7f2e0c159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:18.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.243+0000 7f2e0ad9d640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e0c15e1a0 0x7f2e0c155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:18.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.243+0000 7f2e03fff640 1 -- 172.21.15.81:0/1543292165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e0c15e1a0 msgr2=0x7f2e0c155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.243+0000 7f2e03fff640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e0c15e1a0 0x7f2e0c155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.243+0000 7f2e03fff640 1 -- 172.21.15.81:0/1543292165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e0c070360 msgr2=0x7f2e0c159550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.243+0000 7f2e03fff640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e0c070360 0x7f2e0c159550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.243+0000 7f2e03fff640 1 -- 172.21.15.81:0/1543292165 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df40093a0 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.243+0000 7f2e0a59c640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e0c070360 0x7f2e0c159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:18.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.244+0000 7f2e03fff640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e0c0c97b0 0x7f2e0c1536d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2df402beb0 tx=0x7f2df40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:18.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.244+0000 7f2e02ffd640 1 -- 172.21.15.81:0/1543292165 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df403bc70 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.244+0000 7f2e02ffd640 1 -- 172.21.15.81:0/1543292165 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df403bdd0 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.244+0000 7f2e02ffd640 1 -- 172.21.15.81:0/1543292165 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df4045630 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.244+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1543292165 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0c0c6c30 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.244+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1543292165 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e0c1560b0 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.245+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1543292165 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e0c0c5690 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.246+0000 7f2e02ffd640 1 -- 172.21.15.81:0/1543292165 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2df404e450 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.247+0000 7f2e02ffd640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dec06aea0 0x7f2dec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:18.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.248+0000 7f2e02ffd640 1 -- 172.21.15.81:0/1543292165 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2df402e070 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.248+0000 7f2e0a59c640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dec06aea0 0x7f2dec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:18.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.248+0000 7f2e0a59c640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dec06aea0 0x7f2dec06d360 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0011e0 tx=0x7f2dfc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:18.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.251+0000 7f2e02ffd640 1 -- 172.21.15.81:0/1543292165 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2df4076890 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.439+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1543292165 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2e0c0c70d0 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.439+0000 7f2e02ffd640 1 -- 172.21.15.81:0/1543292165 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2df4035070 con 0x7f2e0c0c97b0 2023-11-08T16:29:18.441 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:18.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1543292165 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dec06aea0 msgr2=0x7f2dec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dec06aea0 0x7f2dec06d360 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc0011e0 tx=0x7f2dfc009040 comp rx=0 tx=0).stop 2023-11-08T16:29:18.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1543292165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e0c0c97b0 msgr2=0x7f2e0c1536d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e0c0c97b0 0x7f2e0c1536d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2df402beb0 tx=0x7f2df40046c0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1543292165 shutdown_connections 2023-11-08T16:29:18.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dec06aea0 0x7f2dec06d360 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e0c15e1a0 0x7f2e0c155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e0c0c97b0 0x7f2e0c1536d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 --2- 172.21.15.81:0/1543292165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e0c070360 0x7f2e0c159550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1543292165 >> 172.21.15.81:0/1543292165 conn(0x7f2e0c0ce8d0 msgr2=0x7f2e0c0c4eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:18.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1543292165 shutdown_connections 2023-11-08T16:29:18.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.442+0000 7f2e0b59e640 1 -- 172.21.15.81:0/1543292165 wait complete. 2023-11-08T16:29:18.464 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:29:18.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.672+0000 7f6425bcc640 1 Processor -- start 2023-11-08T16:29:18.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.672+0000 7f6425bcc640 1 -- start start 2023-11-08T16:29:18.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.674+0000 7f6425bcc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64200adcd0 0x7f64200ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:18.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.674+0000 7f6425bcc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6420070050 0x7f64200704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:18.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.674+0000 7f6425bcc640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64200c9a70 con 0x7f64200cc5e0 2023-11-08T16:29:18.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.674+0000 7f6425bcc640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64200cc3e0 con 0x7f6420070050 2023-11-08T16:29:18.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.674+0000 7f6425bcc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f642006d070 con 0x7f64200adcd0 2023-11-08T16:29:18.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.674+0000 7f641ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6420070050 0x7f64200704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:18.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.674+0000 7f641ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6420070050 0x7f64200704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35818/0 (socket says 172.21.15.81:35818) 2023-11-08T16:29:18.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.674+0000 7f641ffff640 1 -- 172.21.15.81:0/2881692040 learned_addr learned my addr 172.21.15.81:0/2881692040 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:18.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.674+0000 7f641f7fe640 1 --2- 172.21.15.81:0/2881692040 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64200adcd0 0x7f64200ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:18.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.674+0000 7f641ffff640 1 -- 172.21.15.81:0/2881692040 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64200adcd0 msgr2=0x7f64200ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:18.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.674+0000 7f641ffff640 1 --2- 172.21.15.81:0/2881692040 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64200adcd0 0x7f64200ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.675+0000 7f641ffff640 1 -- 172.21.15.81:0/2881692040 >> v1:172.21.15.81:6789/0 conn(0x7f64200cc5e0 legacy=0x7f64200cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.675+0000 7f641ffff640 1 -- 172.21.15.81:0/2881692040 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f642006d350 con 0x7f6420070050 2023-11-08T16:29:18.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.676+0000 7f641ffff640 1 --2- 172.21.15.81:0/2881692040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6420070050 0x7f64200704e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6418009e50 tx=0x7f641802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec57b63d33c34e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:18.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.676+0000 7f641e7fc640 1 -- 172.21.15.81:0/2881692040 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641802cb00 con 0x7f6420070050 2023-11-08T16:29:18.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.676+0000 7f641e7fc640 1 -- 172.21.15.81:0/2881692040 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641802cc60 con 0x7f6420070050 2023-11-08T16:29:18.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.676+0000 7f641e7fc640 1 -- 172.21.15.81:0/2881692040 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64180343f0 con 0x7f6420070050 2023-11-08T16:29:18.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.676+0000 7f6425bcc640 1 -- 172.21.15.81:0/2881692040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6420070050 msgr2=0x7f64200704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.676+0000 7f6425bcc640 1 --2- 172.21.15.81:0/2881692040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6420070050 0x7f64200704e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6418009e50 tx=0x7f641802bf60 comp rx=0 tx=0).stop 2023-11-08T16:29:18.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.677+0000 7f6425bcc640 1 -- 172.21.15.81:0/2881692040 shutdown_connections 2023-11-08T16:29:18.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.677+0000 7f6425bcc640 1 --2- 172.21.15.81:0/2881692040 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6420070050 0x7f64200704e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.677+0000 7f6425bcc640 1 --2- 172.21.15.81:0/2881692040 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64200adcd0 0x7f64200ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.677+0000 7f6425bcc640 1 -- 172.21.15.81:0/2881692040 >> 172.21.15.81:0/2881692040 conn(0x7f64200cb490 msgr2=0x7f64200cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:18.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.677+0000 7f6425bcc640 1 -- 172.21.15.81:0/2881692040 shutdown_connections 2023-11-08T16:29:18.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.677+0000 7f6425bcc640 1 -- 172.21.15.81:0/2881692040 wait complete. 2023-11-08T16:29:18.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.678+0000 7f6425bcc640 1 Processor -- start 2023-11-08T16:29:18.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.679+0000 7f6425bcc640 1 -- start start 2023-11-08T16:29:18.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.679+0000 7f6425bcc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6420070050 0x7f6420146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:18.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.679+0000 7f6425bcc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64200adcd0 0x7f6420154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:18.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f6425bcc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64200cc5e0 0x7f6420155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:18.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f6425bcc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64200b2f20 con 0x7f64200cc5e0 2023-11-08T16:29:18.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f6425bcc640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6420147490 con 0x7f6420070050 2023-11-08T16:29:18.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f6425bcc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64201551b0 con 0x7f64200adcd0 2023-11-08T16:29:18.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f641ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64200cc5e0 0x7f6420155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:18.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f641ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64200cc5e0 0x7f6420155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35834/0 (socket says 172.21.15.81:35834) 2023-11-08T16:29:18.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f641ffff640 1 -- 172.21.15.81:0/36470181 learned_addr learned my addr 172.21.15.81:0/36470181 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:18.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f641ffff640 1 -- 172.21.15.81:0/36470181 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64200adcd0 msgr2=0x7f6420154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f641ffff640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64200adcd0 0x7f6420154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f641ffff640 1 -- 172.21.15.81:0/36470181 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6420070050 msgr2=0x7f6420146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f641ffff640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6420070050 0x7f6420146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.680+0000 7f641ffff640 1 -- 172.21.15.81:0/36470181 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6418009b30 con 0x7f64200cc5e0 2023-11-08T16:29:18.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.681+0000 7f641ffff640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64200cc5e0 0x7f6420155940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6418036040 tx=0x7f6418004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:18.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.681+0000 7f6407fff640 1 -- 172.21.15.81:0/36470181 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6418009510 con 0x7f64200cc5e0 2023-11-08T16:29:18.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.681+0000 7f6407fff640 1 -- 172.21.15.81:0/36470181 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641803e450 con 0x7f64200cc5e0 2023-11-08T16:29:18.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.681+0000 7f6407fff640 1 -- 172.21.15.81:0/36470181 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641803e7e0 con 0x7f64200cc5e0 2023-11-08T16:29:18.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.681+0000 7f6425bcc640 1 -- 172.21.15.81:0/36470181 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64200b22a0 con 0x7f64200cc5e0 2023-11-08T16:29:18.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.681+0000 7f6425bcc640 1 -- 172.21.15.81:0/36470181 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64200b2860 con 0x7f64200cc5e0 2023-11-08T16:29:18.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.682+0000 7f6425bcc640 1 -- 172.21.15.81:0/36470181 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6420072170 con 0x7f64200cc5e0 2023-11-08T16:29:18.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.683+0000 7f6407fff640 1 -- 172.21.15.81:0/36470181 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f641803e940 con 0x7f64200cc5e0 2023-11-08T16:29:18.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.684+0000 7f6407fff640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f640c06ae70 0x7f640c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:18.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.685+0000 7f6407fff640 1 -- 172.21.15.81:0/36470181 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64180ac7a0 con 0x7f64200cc5e0 2023-11-08T16:29:18.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.685+0000 7f641f7fe640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f640c06ae70 0x7f640c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:18.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.685+0000 7f641f7fe640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f640c06ae70 0x7f640c06d330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6410006fd0 tx=0x7f6410008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:18.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.689+0000 7f6407fff640 1 -- 172.21.15.81:0/36470181 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f64180777b0 con 0x7f64200cc5e0 2023-11-08T16:29:18.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.919+0000 7f6425bcc640 1 -- 172.21.15.81:0/36470181 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64200b3060 con 0x7f64200cc5e0 2023-11-08T16:29:18.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.920+0000 7f6407fff640 1 -- 172.21.15.81:0/36470181 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f64200b3060 con 0x7f64200cc5e0 2023-11-08T16:29:18.922 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:18.922 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4190352668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4190352668},{"type":"v1","addr":"172.21.15.81:6835","nonce":4190352668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2636431103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2636431103},{"type":"v1","addr":"172.21.15.93:6835","nonce":2636431103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3681499690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3681499690},{"type":"v1","addr":"172.21.15.81:6837","nonce":3681499690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:29:03.181472+0000","modified":"2023-11-08T16:29:06.272041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6834/2087517381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2087517381},{"type":"v1","addr":"172.21.15.93:6834","nonce":2087517381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T16:29:18.922 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 8 2023-11-08T16:29:18.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 -- 172.21.15.81:0/36470181 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f640c06ae70 msgr2=0x7f640c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f640c06ae70 0x7f640c06d330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6410006fd0 tx=0x7f6410008040 comp rx=0 tx=0).stop 2023-11-08T16:29:18.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 -- 172.21.15.81:0/36470181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64200cc5e0 msgr2=0x7f6420155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:18.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64200cc5e0 0x7f6420155940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6418036040 tx=0x7f6418004a40 comp rx=0 tx=0).stop 2023-11-08T16:29:18.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 -- 172.21.15.81:0/36470181 shutdown_connections 2023-11-08T16:29:18.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f640c06ae70 0x7f640c06d330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f64200cc5e0 0x7f6420155940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f64200adcd0 0x7f6420154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 --2- 172.21.15.81:0/36470181 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6420070050 0x7f6420146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:18.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 -- 172.21.15.81:0/36470181 >> 172.21.15.81:0/36470181 conn(0x7f64200cb490 msgr2=0x7f64200cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:18.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 -- 172.21.15.81:0/36470181 shutdown_connections 2023-11-08T16:29:18.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:18.923+0000 7f6425bcc640 1 -- 172.21.15.81:0/36470181 wait complete. 2023-11-08T16:29:18.942 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:19.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.150+0000 7f10a33b6640 1 Processor -- start 2023-11-08T16:29:19.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.150+0000 7f10a33b6640 1 -- start start 2023-11-08T16:29:19.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.151+0000 7f10a33b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f109c161710 0x7f109c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.151+0000 7f10a33b6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c06b980 con 0x7f109c15d2b0 2023-11-08T16:29:19.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.151+0000 7f10a33b6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c06bc60 con 0x7f109c0ccd30 2023-11-08T16:29:19.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.151+0000 7f10a33b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0cd1c0 con 0x7f109c161710 2023-11-08T16:29:19.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.152+0000 7f10a1152640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f109c15d2b0 0x7f109c0c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49254/0 (socket says 172.21.15.81:49254) 2023-11-08T16:29:19.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.152+0000 7f10a1152640 1 -- 172.21.15.81:0/1292443536 learned_addr learned my addr 172.21.15.81:0/1292443536 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:19.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.152+0000 7f10a0951640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f109c161710 0x7f109c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.152+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538798492 0 0) 0x7f109c06b980 con 0x7f109c15d2b0 2023-11-08T16:29:19.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.152+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1088003520 con 0x7f109c15d2b0 2023-11-08T16:29:19.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4019498558 0 0) 0x7f1088003520 con 0x7f109c15d2b0 2023-11-08T16:29:19.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f109c161710 msgr2=0x7f109c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:19.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f109c161710 0x7f109c0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 >> v1:172.21.15.93:6789/0 conn(0x7f109c0ccd30 legacy=0x7f109c06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c06bf40 con 0x7f109c15d2b0 2023-11-08T16:29:19.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f1098001670 con 0x7f109c15d2b0 2023-11-08T16:29:19.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 >> v1:172.21.15.81:6789/0 conn(0x7f109c15d2b0 legacy=0x7f109c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1088003a80 0x7f1088003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10880047c0 0x7f1088004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f109c161710 0x7f1088005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0cd1c0 con 0x7f1088003a80 2023-11-08T16:29:19.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108800ba60 con 0x7f10880047c0 2023-11-08T16:29:19.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108800bc20 con 0x7f109c161710 2023-11-08T16:29:19.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f10a1152640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1088003a80 0x7f1088003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f10a0951640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10880047c0 0x7f1088004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f10a1152640 1 -- 172.21.15.81:0/1292443536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f109c161710 msgr2=0x7f1088005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:19.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f10a1152640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f109c161710 0x7f1088005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f10a1152640 1 -- 172.21.15.81:0/1292443536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10880047c0 msgr2=0x7f1088004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:19.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f10a1152640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10880047c0 0x7f1088004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f10a1152640 1 -- 172.21.15.81:0/1292443536 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1088003520 con 0x7f1088003a80 2023-11-08T16:29:19.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.153+0000 7f10a0951640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10880047c0 0x7f1088004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:19.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.154+0000 7f10a1152640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1088003a80 0x7f1088003e60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f109800c160 tx=0x7f10880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cef92f1ed727e0c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:19.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.155+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098005ad0 con 0x7f1088003a80 2023-11-08T16:29:19.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.155+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098005c50 con 0x7f1088003a80 2023-11-08T16:29:19.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.155+0000 7f1093fff640 1 -- 172.21.15.81:0/1292443536 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109802c810 con 0x7f1088003a80 2023-11-08T16:29:19.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.155+0000 7f10a33b6640 1 -- 172.21.15.81:0/1292443536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1088003a80 msgr2=0x7f1088003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.155+0000 7f10a33b6640 1 --2- 172.21.15.81:0/1292443536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1088003a80 0x7f1088003e60 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f109800c160 tx=0x7f10880057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.155+0000 7f10a1953640 1 -- 172.21.15.81:0/1292443536 reap_dead start 2023-11-08T16:29:19.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.155+0000 7f10a33b6640 1 -- 172.21.15.81:0/1292443536 shutdown_connections 2023-11-08T16:29:19.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.155+0000 7f10a33b6640 1 -- 172.21.15.81:0/1292443536 >> 172.21.15.81:0/1292443536 conn(0x7f109c0cb490 msgr2=0x7f109c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:19.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.155+0000 7f10a33b6640 1 -- 172.21.15.81:0/1292443536 shutdown_connections 2023-11-08T16:29:19.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.155+0000 7f10a33b6640 1 -- 172.21.15.81:0/1292443536 wait complete. 2023-11-08T16:29:19.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.157+0000 7f10a33b6640 1 Processor -- start 2023-11-08T16:29:19.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.157+0000 7f10a33b6640 1 -- start start 2023-11-08T16:29:19.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.158+0000 7f10a33b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1088003a80 0x7f109c0b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.158+0000 7f10a33b6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f109c0c4dc0 0x7f109c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.158+0000 7f10a33b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c0af380 0x7f109c0af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.158+0000 7f10a33b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c158ed0 con 0x7f109c0af380 2023-11-08T16:29:19.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.158+0000 7f10a33b6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0afd40 con 0x7f109c0c4dc0 2023-11-08T16:29:19.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.158+0000 7f10a33b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c153d30 con 0x7f1088003a80 2023-11-08T16:29:19.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.159+0000 7f10a1953640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c0af380 0x7f109c0af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.159+0000 7f10a1953640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c0af380 0x7f109c0af800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35852/0 (socket says 172.21.15.81:35852) 2023-11-08T16:29:19.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.159+0000 7f10a1953640 1 -- 172.21.15.81:0/4041810777 learned_addr learned my addr 172.21.15.81:0/4041810777 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:19.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.159+0000 7f10a1152640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1088003a80 0x7f109c0b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.159+0000 7f10a0951640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f109c0c4dc0 0x7f109c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.159+0000 7f10a1953640 1 -- 172.21.15.81:0/4041810777 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1088003a80 msgr2=0x7f109c0b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.159+0000 7f10a1953640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1088003a80 0x7f109c0b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.159+0000 7f10a1953640 1 -- 172.21.15.81:0/4041810777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f109c0c4dc0 msgr2=0x7f109c1537f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.159+0000 7f10a1953640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f109c0c4dc0 0x7f109c1537f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.159+0000 7f10a1953640 1 -- 172.21.15.81:0/4041810777 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c0b3ab0 con 0x7f109c0af380 2023-11-08T16:29:19.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.159+0000 7f10a1953640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c0af380 0x7f109c0af800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f109c067770 tx=0x7f109400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:19.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.160+0000 7f1091ffb640 1 -- 172.21.15.81:0/4041810777 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094005d40 con 0x7f109c0af380 2023-11-08T16:29:19.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.160+0000 7f1091ffb640 1 -- 172.21.15.81:0/4041810777 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109400a610 con 0x7f109c0af380 2023-11-08T16:29:19.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.160+0000 7f1091ffb640 1 -- 172.21.15.81:0/4041810777 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094005ea0 con 0x7f109c0af380 2023-11-08T16:29:19.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.160+0000 7f10a33b6640 1 -- 172.21.15.81:0/4041810777 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c0b3d90 con 0x7f109c0af380 2023-11-08T16:29:19.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.160+0000 7f10a33b6640 1 -- 172.21.15.81:0/4041810777 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c1590e0 con 0x7f109c0af380 2023-11-08T16:29:19.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.161+0000 7f10a33b6640 1 -- 172.21.15.81:0/4041810777 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c04ece0 con 0x7f109c0af380 2023-11-08T16:29:19.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.162+0000 7f1091ffb640 1 -- 172.21.15.81:0/4041810777 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f109400aa00 con 0x7f109c0af380 2023-11-08T16:29:19.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.163+0000 7f1091ffb640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108006ae70 0x7f108006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.163+0000 7f1091ffb640 1 -- 172.21.15.81:0/4041810777 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10940885c0 con 0x7f109c0af380 2023-11-08T16:29:19.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.163+0000 7f10a1152640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108006ae70 0x7f108006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.164+0000 7f10a1152640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108006ae70 0x7f108006d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1098034000 tx=0x7f1098009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:19.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.167+0000 7f1091ffb640 1 -- 172.21.15.81:0/4041810777 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1094053670 con 0x7f109c0af380 2023-11-08T16:29:19.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.357+0000 7f10a33b6640 1 -- 172.21.15.81:0/4041810777 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f109c0b4990 con 0x7f109c0af380 2023-11-08T16:29:19.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.359+0000 7f1091ffb640 1 -- 172.21.15.81:0/4041810777 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f1094056c90 con 0x7f109c0af380 2023-11-08T16:29:19.362 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:19.362 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:04.727022+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:19.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 -- 172.21.15.81:0/4041810777 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108006ae70 msgr2=0x7f108006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108006ae70 0x7f108006d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1098034000 tx=0x7f1098009030 comp rx=0 tx=0).stop 2023-11-08T16:29:19.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 -- 172.21.15.81:0/4041810777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c0af380 msgr2=0x7f109c0af800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c0af380 0x7f109c0af800 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f109c067770 tx=0x7f109400cb90 comp rx=0 tx=0).stop 2023-11-08T16:29:19.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 -- 172.21.15.81:0/4041810777 shutdown_connections 2023-11-08T16:29:19.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108006ae70 0x7f108006d330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f109c0af380 0x7f109c0af800 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f109c0c4dc0 0x7f109c1537f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 --2- 172.21.15.81:0/4041810777 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1088003a80 0x7f109c0b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 -- 172.21.15.81:0/4041810777 >> 172.21.15.81:0/4041810777 conn(0x7f109c0cb490 msgr2=0x7f109c15bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:19.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 -- 172.21.15.81:0/4041810777 shutdown_connections 2023-11-08T16:29:19.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.362+0000 7f10a33b6640 1 -- 172.21.15.81:0/4041810777 wait complete. 2023-11-08T16:29:19.384 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:29:19.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.585+0000 7f3c5a800640 1 Processor -- start 2023-11-08T16:29:19.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.585+0000 7f3c5a800640 1 -- start start 2023-11-08T16:29:19.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.586+0000 7f3c5a800640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3c540ca480 0x7f3c5415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.587+0000 7f3c5a800640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c5406b630 con 0x7f3c5415ea60 2023-11-08T16:29:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.587+0000 7f3c5a800640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c540ca910 con 0x7f3c540ade50 2023-11-08T16:29:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.587+0000 7f3c5a800640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c540b1860 con 0x7f3c540ca480 2023-11-08T16:29:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.587+0000 7f3c58d9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3c5415ea60 0x7f3c5415eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49268/0 (socket says 172.21.15.81:49268) 2023-11-08T16:29:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.587+0000 7f3c58d9d640 1 -- 172.21.15.81:0/2812069738 learned_addr learned my addr 172.21.15.81:0/2812069738 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.587+0000 7f3c4bfff640 1 --2- 172.21.15.81:0/2812069738 >> v2:172.21.15.93:3300/0 conn(0x7f3c540ca480 0x7f3c5415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.587+0000 7f3c537fe640 1 -- 172.21.15.81:0/2812069738 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942739226 0 0) 0x7f3c5406b630 con 0x7f3c5415ea60 2023-11-08T16:29:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.588+0000 7f3c537fe640 1 -- 172.21.15.81:0/2812069738 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c40003520 con 0x7f3c5415ea60 2023-11-08T16:29:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.588+0000 7f3c4bfff640 1 -- 172.21.15.81:0/2812069738 >> v1:172.21.15.93:6789/0 conn(0x7f3c540ade50 legacy=0x7f3c5415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.588+0000 7f3c4bfff640 1 -- 172.21.15.81:0/2812069738 >> v1:172.21.15.81:6789/0 conn(0x7f3c5415ea60 legacy=0x7f3c5415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.588+0000 7f3c537fe640 1 -- 172.21.15.81:0/2812069738 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2471627239 0 0) 0x7f3c40003520 con 0x7f3c5415ea60 2023-11-08T16:29:19.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.588+0000 7f3c4bfff640 1 -- 172.21.15.81:0/2812069738 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c540cabb0 con 0x7f3c540ca480 2023-11-08T16:29:19.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.589+0000 7f3c4bfff640 1 --2- 172.21.15.81:0/2812069738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c540ca480 0x7f3c5415ad90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0097a0 tx=0x7f3c3c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9646f401bf94791 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:19.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.590+0000 7f3c537fe640 1 -- 172.21.15.81:0/2812069738 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c02f050 con 0x7f3c540ca480 2023-11-08T16:29:19.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.590+0000 7f3c537fe640 1 -- 172.21.15.81:0/2812069738 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3c031500 con 0x7f3c540ca480 2023-11-08T16:29:19.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.590+0000 7f3c537fe640 1 -- 172.21.15.81:0/2812069738 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0019f0 con 0x7f3c540ca480 2023-11-08T16:29:19.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.591+0000 7f3c5a800640 1 -- 172.21.15.81:0/2812069738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c540ca480 msgr2=0x7f3c5415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.591+0000 7f3c5a800640 1 --2- 172.21.15.81:0/2812069738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c540ca480 0x7f3c5415ad90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0097a0 tx=0x7f3c3c02b820 comp rx=0 tx=0).stop 2023-11-08T16:29:19.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.591+0000 7f3c5a800640 1 -- 172.21.15.81:0/2812069738 shutdown_connections 2023-11-08T16:29:19.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.591+0000 7f3c5a800640 1 --2- 172.21.15.81:0/2812069738 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c540ca480 0x7f3c5415ad90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.591+0000 7f3c5a800640 1 -- 172.21.15.81:0/2812069738 >> 172.21.15.81:0/2812069738 conn(0x7f3c540cb490 msgr2=0x7f3c540cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:19.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.591+0000 7f3c5a800640 1 -- 172.21.15.81:0/2812069738 shutdown_connections 2023-11-08T16:29:19.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.591+0000 7f3c5a800640 1 -- 172.21.15.81:0/2812069738 wait complete. 2023-11-08T16:29:19.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.593+0000 7f3c5a800640 1 Processor -- start 2023-11-08T16:29:19.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.593+0000 7f3c5a800640 1 -- start start 2023-11-08T16:29:19.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c5a800640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c540ade50 0x7f3c54152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c5a800640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c540ca480 0x7f3c54152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c5a800640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c5415ea60 0x7f3c5414f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c5a800640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c54166af0 con 0x7f3c5415ea60 2023-11-08T16:29:19.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c5a800640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5414df70 con 0x7f3c540ca480 2023-11-08T16:29:19.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c5a800640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5414e250 con 0x7f3c540ade50 2023-11-08T16:29:19.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c58d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c5415ea60 0x7f3c5414f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c58d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c5415ea60 0x7f3c5414f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35858/0 (socket says 172.21.15.81:35858) 2023-11-08T16:29:19.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c58d9d640 1 -- 172.21.15.81:0/2418755479 learned_addr learned my addr 172.21.15.81:0/2418755479 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:19.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c4bfff640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c540ca480 0x7f3c54152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c58d9d640 1 -- 172.21.15.81:0/2418755479 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c540ade50 msgr2=0x7f3c54152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c53fff640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c540ade50 0x7f3c54152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c58d9d640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c540ade50 0x7f3c54152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.594+0000 7f3c58d9d640 1 -- 172.21.15.81:0/2418755479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c540ca480 msgr2=0x7f3c54152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.595+0000 7f3c58d9d640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c540ca480 0x7f3c54152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.595+0000 7f3c58d9d640 1 -- 172.21.15.81:0/2418755479 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c009400 con 0x7f3c5415ea60 2023-11-08T16:29:19.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.595+0000 7f3c4bfff640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c540ca480 0x7f3c54152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:19.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.595+0000 7f3c53fff640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c540ade50 0x7f3c54152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:19.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.595+0000 7f3c58d9d640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c5415ea60 0x7f3c5414f010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00cc70 tx=0x7f3c4c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:19.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.595+0000 7f3c517fa640 1 -- 172.21.15.81:0/2418755479 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c00ce80 con 0x7f3c5415ea60 2023-11-08T16:29:19.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.595+0000 7f3c517fa640 1 -- 172.21.15.81:0/2418755479 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4c012400 con 0x7f3c5415ea60 2023-11-08T16:29:19.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.595+0000 7f3c5a800640 1 -- 172.21.15.81:0/2418755479 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c5414e530 con 0x7f3c5415ea60 2023-11-08T16:29:19.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.595+0000 7f3c517fa640 1 -- 172.21.15.81:0/2418755479 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c0127b0 con 0x7f3c5415ea60 2023-11-08T16:29:19.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.595+0000 7f3c5a800640 1 -- 172.21.15.81:0/2418755479 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c5414eaf0 con 0x7f3c5415ea60 2023-11-08T16:29:19.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.596+0000 7f3c5a800640 1 -- 172.21.15.81:0/2418755479 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5414ece0 con 0x7f3c5415ea60 2023-11-08T16:29:19.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.597+0000 7f3c517fa640 1 -- 172.21.15.81:0/2418755479 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c4c012910 con 0x7f3c5415ea60 2023-11-08T16:29:19.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.598+0000 7f3c517fa640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c3406aea0 0x7f3c3406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:19.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.598+0000 7f3c517fa640 1 -- 172.21.15.81:0/2418755479 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c4c089dd0 con 0x7f3c5415ea60 2023-11-08T16:29:19.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.598+0000 7f3c53fff640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c3406aea0 0x7f3c3406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:19.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.599+0000 7f3c53fff640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c3406aea0 0x7f3c3406d360 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c44000f60 tx=0x7f3c44009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:19.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.602+0000 7f3c517fa640 1 -- 172.21.15.81:0/2418755479 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c4c054de0 con 0x7f3c5415ea60 2023-11-08T16:29:19.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.831+0000 7f3c5a800640 1 -- 172.21.15.81:0/2418755479 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c540b32e0 con 0x7f3c5415ea60 2023-11-08T16:29:19.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.832+0000 7f3c517fa640 1 -- 172.21.15.81:0/2418755479 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f3c4c058400 con 0x7f3c5415ea60 2023-11-08T16:29:19.834 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:19.834 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4190352668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4190352668},{"type":"v1","addr":"172.21.15.81:6835","nonce":4190352668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2636431103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2636431103},{"type":"v1","addr":"172.21.15.93:6835","nonce":2636431103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3681499690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3681499690},{"type":"v1","addr":"172.21.15.81:6837","nonce":3681499690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:29:03.181472+0000","modified":"2023-11-08T16:29:06.272041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6834/2087517381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2087517381},{"type":"v1","addr":"172.21.15.93:6834","nonce":2087517381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T16:29:19.835 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 8 2023-11-08T16:29:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 -- 172.21.15.81:0/2418755479 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c3406aea0 msgr2=0x7f3c3406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c3406aea0 0x7f3c3406d360 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3c44000f60 tx=0x7f3c44009040 comp rx=0 tx=0).stop 2023-11-08T16:29:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 -- 172.21.15.81:0/2418755479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c5415ea60 msgr2=0x7f3c5414f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c5415ea60 0x7f3c5414f010 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3c4c00cc70 tx=0x7f3c4c00a430 comp rx=0 tx=0).stop 2023-11-08T16:29:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 -- 172.21.15.81:0/2418755479 shutdown_connections 2023-11-08T16:29:19.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c3406aea0 0x7f3c3406d360 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c5415ea60 0x7f3c5414f010 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c540ca480 0x7f3c54152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 --2- 172.21.15.81:0/2418755479 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c540ade50 0x7f3c54152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:19.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 -- 172.21.15.81:0/2418755479 >> 172.21.15.81:0/2418755479 conn(0x7f3c540cb490 msgr2=0x7f3c540b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:19.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 -- 172.21.15.81:0/2418755479 shutdown_connections 2023-11-08T16:29:19.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:19.835+0000 7f3c5a800640 1 -- 172.21.15.81:0/2418755479 wait complete. 2023-11-08T16:29:20.255 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:29:03.181472+0000', 'modified': '2023-11-08T16:29:06.272041+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4270}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4270': {'gid': 4270, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.93:6834/2087517381', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6832', 'nonce': 2087517381}, {'type': 'v1', 'addr': '172.21.15.93:6834', 'nonce': 2087517381}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:29:20.256 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:29:20.256 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T16:29:20.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.482+0000 7f29f2905640 1 Processor -- start 2023-11-08T16:29:20.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.483+0000 7f29f2905640 1 -- start start 2023-11-08T16:29:20.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.484+0000 7f29f2905640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f29ec159530 0x7f29ec0c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.484+0000 7f29f2905640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f29ec0ce020 0x7f29ec0c9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.484+0000 7f29f2905640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29ec159910 con 0x7f29ec0c56b0 2023-11-08T16:29:20.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.484+0000 7f29f2905640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29ec06d440 con 0x7f29ec159530 2023-11-08T16:29:20.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.484+0000 7f29f2905640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec06ce80 con 0x7f29ec0ce020 2023-11-08T16:29:20.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.484+0000 7f29f0ea2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f29ec0ce020 0x7f29ec0c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.484+0000 7f29f0ea2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f29ec0ce020 0x7f29ec0c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54916/0 (socket says 172.21.15.81:54916) 2023-11-08T16:29:20.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.484+0000 7f29f0ea2640 1 -- 172.21.15.81:0/3470234595 learned_addr learned my addr 172.21.15.81:0/3470234595 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:20.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.484+0000 7f29eb7fe640 1 --1- 172.21.15.81:0/3470234595 >> v1:172.21.15.93:6789/0 conn(0x7f29ec0c56b0 0x7f29ec15d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50402/0 (socket says 172.21.15.81:50402) 2023-11-08T16:29:20.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.485+0000 7f29ebfff640 1 --2- 172.21.15.81:0/3470234595 >> v2:172.21.15.93:3300/0 conn(0x7f29ec159530 0x7f29ec0c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.485+0000 7f29f0ea2640 1 -- 172.21.15.81:0/3470234595 >> v2:172.21.15.93:3300/0 conn(0x7f29ec159530 msgr2=0x7f29ec0c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:20.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.485+0000 7f29f0ea2640 1 --2- 172.21.15.81:0/3470234595 >> v2:172.21.15.93:3300/0 conn(0x7f29ec159530 0x7f29ec0c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.485+0000 7f29f0ea2640 1 -- 172.21.15.81:0/3470234595 >> v1:172.21.15.93:6789/0 conn(0x7f29ec0c56b0 legacy=0x7f29ec15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:20.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.485+0000 7f29f0ea2640 1 -- 172.21.15.81:0/3470234595 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29ec06d160 con 0x7f29ec0ce020 2023-11-08T16:29:20.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.485+0000 7f29eaffd640 1 -- 172.21.15.81:0/3470234595 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406942100 0 0) 0x7f29ec159910 con 0x7f29ec0c56b0 2023-11-08T16:29:20.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.486+0000 7f29f0ea2640 1 --2- 172.21.15.81:0/3470234595 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f29ec0ce020 0x7f29ec0c9630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009e50 tx=0x7f29e402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20fa6da72e80cc66 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:20.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.487+0000 7f29eaffd640 1 -- 172.21.15.81:0/3470234595 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e402cb00 con 0x7f29ec0ce020 2023-11-08T16:29:20.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.487+0000 7f29eaffd640 1 -- 172.21.15.81:0/3470234595 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e402cc60 con 0x7f29ec0ce020 2023-11-08T16:29:20.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.487+0000 7f29eaffd640 1 -- 172.21.15.81:0/3470234595 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e40343f0 con 0x7f29ec0ce020 2023-11-08T16:29:20.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.488+0000 7f29f2905640 1 -- 172.21.15.81:0/3470234595 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f29ec0ce020 msgr2=0x7f29ec0c9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:20.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.488+0000 7f29f2905640 1 --2- 172.21.15.81:0/3470234595 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f29ec0ce020 0x7f29ec0c9630 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009e50 tx=0x7f29e402bf60 comp rx=0 tx=0).stop 2023-11-08T16:29:20.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.488+0000 7f29f2905640 1 -- 172.21.15.81:0/3470234595 shutdown_connections 2023-11-08T16:29:20.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.488+0000 7f29f2905640 1 --2- 172.21.15.81:0/3470234595 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f29ec0ce020 0x7f29ec0c9630 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.488+0000 7f29f2905640 1 --2- 172.21.15.81:0/3470234595 >> v2:172.21.15.93:3300/0 conn(0x7f29ec159530 0x7f29ec0c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.488+0000 7f29f2905640 1 -- 172.21.15.81:0/3470234595 >> 172.21.15.81:0/3470234595 conn(0x7f29ec0ceaa0 msgr2=0x7f29ec0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:20.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.488+0000 7f29f2905640 1 -- 172.21.15.81:0/3470234595 shutdown_connections 2023-11-08T16:29:20.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.488+0000 7f29f2905640 1 -- 172.21.15.81:0/3470234595 wait complete. 2023-11-08T16:29:20.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.490+0000 7f29f2905640 1 Processor -- start 2023-11-08T16:29:20.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.490+0000 7f29f2905640 1 -- start start 2023-11-08T16:29:20.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29f2905640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29ec0c56b0 0x7f29ec14f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29f2905640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f29ec0ce020 0x7f29ec0c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29f2905640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29ec159530 0x7f29ec148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29f2905640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec14f610 con 0x7f29ec0c56b0 2023-11-08T16:29:20.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29f2905640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec14fe80 con 0x7f29ec159530 2023-11-08T16:29:20.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29f2905640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29ec150160 con 0x7f29ec0ce020 2023-11-08T16:29:20.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29ebfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29ec0c56b0 0x7f29ec14f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29ebfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29ec0c56b0 0x7f29ec14f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35864/0 (socket says 172.21.15.81:35864) 2023-11-08T16:29:20.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29ebfff640 1 -- 172.21.15.81:0/4253112155 learned_addr learned my addr 172.21.15.81:0/4253112155 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:20.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29ebfff640 1 -- 172.21.15.81:0/4253112155 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f29ec0ce020 msgr2=0x7f29ec0c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:20.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29ebfff640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f29ec0ce020 0x7f29ec0c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29ebfff640 1 -- 172.21.15.81:0/4253112155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29ec159530 msgr2=0x7f29ec148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:20.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29f0ea2640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29ec159530 0x7f29ec148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29ebfff640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29ec159530 0x7f29ec148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.491+0000 7f29ebfff640 1 -- 172.21.15.81:0/4253112155 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e4009b30 con 0x7f29ec0c56b0 2023-11-08T16:29:20.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.492+0000 7f29ebfff640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29ec0c56b0 0x7f29ec14f0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f29d400e900 tx=0x7f29d400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:20.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.492+0000 7f29e8ff9640 1 -- 172.21.15.81:0/4253112155 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4009660 con 0x7f29ec0c56b0 2023-11-08T16:29:20.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.492+0000 7f29e8ff9640 1 -- 172.21.15.81:0/4253112155 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d400ca00 con 0x7f29ec0c56b0 2023-11-08T16:29:20.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.492+0000 7f29f2905640 1 -- 172.21.15.81:0/4253112155 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29ec0c3080 con 0x7f29ec0c56b0 2023-11-08T16:29:20.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.492+0000 7f29e8ff9640 1 -- 172.21.15.81:0/4253112155 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d400cd70 con 0x7f29ec0c56b0 2023-11-08T16:29:20.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.492+0000 7f29f2905640 1 -- 172.21.15.81:0/4253112155 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec0c3640 con 0x7f29ec0c56b0 2023-11-08T16:29:20.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.494+0000 7f29e8ff9640 1 -- 172.21.15.81:0/4253112155 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f29d4013470 con 0x7f29ec0c56b0 2023-11-08T16:29:20.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.495+0000 7f29e8ff9640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f29cc06aea0 0x7f29cc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.495+0000 7f29e8ff9640 1 -- 172.21.15.81:0/4253112155 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29d4089c40 con 0x7f29ec0c56b0 2023-11-08T16:29:20.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.495+0000 7f29eb7fe640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f29cc06aea0 0x7f29cc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.496+0000 7f29eb7fe640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f29cc06aea0 0x7f29cc06d360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29dc007fd0 tx=0x7f29dc007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.496+0000 7f29f5572740 1 Processor -- start 2023-11-08T16:29:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.496+0000 7f29f5572740 1 -- start start 2023-11-08T16:29:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.497+0000 7f29f5572740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445ca8600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.497+0000 7f29f5572740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c61f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.497+0000 7f29f5572740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c624b0 0x55d445c8cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.497+0000 7f29f5572740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445bcf7e0 con 0x55d445caaa40 2023-11-08T16:29:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.497+0000 7f29f5572740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445bcced0 con 0x55d445c8a280 2023-11-08T16:29:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.497+0000 7f29f5572740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d445bc87c0 con 0x55d445c624b0 2023-11-08T16:29:20.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.497+0000 7f29ebfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445ca8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.497+0000 7f29ebfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445ca8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35870/0 (socket says 172.21.15.81:35870) 2023-11-08T16:29:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29ebfff640 1 -- 172.21.15.81:0/687249183 learned_addr learned my addr 172.21.15.81:0/687249183 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29eb7fe640 1 --2- 172.21.15.81:0/687249183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c61f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29f0ea2640 1 --2- 172.21.15.81:0/687249183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c624b0 0x55d445c8cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29ebfff640 1 -- 172.21.15.81:0/687249183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c624b0 msgr2=0x55d445c8cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29ebfff640 1 --2- 172.21.15.81:0/687249183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c624b0 0x55d445c8cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29ebfff640 1 -- 172.21.15.81:0/687249183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 msgr2=0x55d445c61f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29ebfff640 1 --2- 172.21.15.81:0/687249183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c61f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29ebfff640 1 -- 172.21.15.81:0/687249183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d445b9bb10 con 0x55d445caaa40 2023-11-08T16:29:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29ebfff640 1 --2- 172.21.15.81:0/687249183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445ca8600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f29d40606e0 tx=0x7f29d4059010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b675eb21956ae488 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:20.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29e1ffb640 1 -- 172.21.15.81:0/687249183 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4060c10 con 0x55d445caaa40 2023-11-08T16:29:20.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29e1ffb640 1 -- 172.21.15.81:0/687249183 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d4070180 con 0x55d445caaa40 2023-11-08T16:29:20.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.498+0000 7f29e1ffb640 1 -- 172.21.15.81:0/687249183 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4055020 con 0x55d445caaa40 2023-11-08T16:29:20.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.499+0000 7f29f5572740 1 -- 172.21.15.81:0/687249183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 msgr2=0x55d445ca8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:20.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.499+0000 7f29f5572740 1 --2- 172.21.15.81:0/687249183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445ca8600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f29d40606e0 tx=0x7f29d4059010 comp rx=0 tx=0).stop 2023-11-08T16:29:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.499+0000 7f29f5572740 1 -- 172.21.15.81:0/687249183 shutdown_connections 2023-11-08T16:29:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.499+0000 7f29f5572740 1 --2- 172.21.15.81:0/687249183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c624b0 0x55d445c8cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.499+0000 7f29f5572740 1 --2- 172.21.15.81:0/687249183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c61f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.499+0000 7f29f5572740 1 --2- 172.21.15.81:0/687249183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445ca8600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.499+0000 7f29f5572740 1 -- 172.21.15.81:0/687249183 >> 172.21.15.81:0/687249183 conn(0x55d445c5f0d0 msgr2=0x55d445ca7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.499+0000 7f29f5572740 1 -- 172.21.15.81:0/687249183 shutdown_connections 2023-11-08T16:29:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.499+0000 7f29f5572740 1 -- 172.21.15.81:0/687249183 wait complete. 2023-11-08T16:29:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.500+0000 7f29f5572740 1 -- start start 2023-11-08T16:29:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.500+0000 7f29f5572740 1 Processor -- start 2023-11-08T16:29:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.501+0000 7f29f5572740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c85740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.501+0000 7f29f5572740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445caaa40 0x55d445ca5220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.501+0000 7f29f5572740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445ca5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.501+0000 7f29f5572740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445b65c40 con 0x55d445c86310 2023-11-08T16:29:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.501+0000 7f29f5572740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445b658e0 con 0x55d445c8a280 2023-11-08T16:29:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.501+0000 7f29f5572740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d44592bf20 con 0x55d445caaa40 2023-11-08T16:29:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.501+0000 7f29f5572740 10 client.0 _ll_register_callbacks cb 0x55d445b87cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:29:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.501+0000 7f29f0ea2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445ca5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.501+0000 7f29f0ea2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445ca5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35884/0 (socket says 172.21.15.81:35884) 2023-11-08T16:29:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.501+0000 7f29f0ea2640 1 -- 172.21.15.81:0/3946597582 learned_addr learned my addr 172.21.15.81:0/3946597582 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29eb7fe640 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445caaa40 0x55d445ca5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29ebfff640 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c85740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29f0ea2640 1 -- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445caaa40 msgr2=0x55d445ca5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29f0ea2640 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445caaa40 0x55d445ca5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29f0ea2640 1 -- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 msgr2=0x55d445c85740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29f0ea2640 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c85740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29f0ea2640 1 -- 172.21.15.81:0/3946597582 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29ec0c3640 con 0x55d445c86310 2023-11-08T16:29:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29f0ea2640 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445ca5760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f29e4036040 tx=0x7f29e4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fef8f5611b04a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29caffd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:29:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29caffd640 1 -- 172.21.15.81:0/3946597582 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4034b30 con 0x55d445c86310 2023-11-08T16:29:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29caffd640 1 -- 172.21.15.81:0/3946597582 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29e4034c90 con 0x55d445c86310 2023-11-08T16:29:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29f5572740 1 -- 172.21.15.81:0/3946597582 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d445b9cc80 con 0x55d445c86310 2023-11-08T16:29:20.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.502+0000 7f29caffd640 1 -- 172.21.15.81:0/3946597582 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29e4009510 con 0x55d445c86310 2023-11-08T16:29:20.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.503+0000 7f29f5572740 10 client.4578 fetch_fsmap learned FSMap version 8 2023-11-08T16:29:20.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.503+0000 7f29f5572740 1 -- 172.21.15.81:0/3946597582 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55d445bc6e40 con 0x55d445c86310 2023-11-08T16:29:20.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.503+0000 7f29caffd640 1 -- 172.21.15.81:0/3946597582 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d445b9cc80 con 0x55d445c86310 2023-11-08T16:29:20.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.503+0000 7f29caffd640 1 -- 172.21.15.81:0/3946597582 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29e403c040 con 0x55d445c86310 2023-11-08T16:29:20.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.503+0000 7f29f5572740 10 client.4578 fetch_fsmap finished waiting for FSMap version 8 2023-11-08T16:29:20.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.503+0000 7f29f5572740 10 client.4578 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T16:29:20.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.503+0000 7f29f5572740 20 client.4578 populate_metadata read hostname 'smithi081' 2023-11-08T16:29:20.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.503+0000 7f29f5572740 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:6832/2087517381,v1:172.21.15.93:6834/2087517381] conn(0x55d445ca2c20 0x55d445ca3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.503+0000 7f29f5572740 4 client.4578 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:29:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.503+0000 7f29f5572740 1 -- 172.21.15.81:0/3946597582 --> [v2:172.21.15.93:6832/2087517381,v1:172.21.15.93:6834/2087517381] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d445bc17a0 con 0x55d445ca2c20 2023-11-08T16:29:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.504+0000 7f29ebfff640 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:6832/2087517381,v1:172.21.15.93:6834/2087517381] conn(0x55d445ca2c20 0x55d445ca3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.504+0000 7f29caffd640 10 client.4578 ms_handle_connect on v2:172.21.15.93:6832/2087517381 2023-11-08T16:29:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.504+0000 7f29ebfff640 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:6832/2087517381,v1:172.21.15.93:6834/2087517381] conn(0x55d445ca2c20 0x55d445ca3000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5c3b3144f37d7e server_cookie=5e2753ead750ea1b in_seq=0 out_seq=0 2023-11-08T16:29:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.506+0000 7f29caffd640 1 -- 172.21.15.81:0/3946597582 <== mds.0 v2:172.21.15.93:6832/2087517381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d445bc17a0 con 0x55d445ca2c20 2023-11-08T16:29:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29caffd640 10 client.4578 handle_command_reply: tid=0 2023-11-08T16:29:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 1 client.4578 shutdown 2023-11-08T16:29:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 1 -- 172.21.15.81:0/3946597582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 msgr2=0x55d445ca5760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445ca5760 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f29e4036040 tx=0x7f29e4004ab0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 1 -- 172.21.15.81:0/3946597582 shutdown_connections 2023-11-08T16:29:20.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:6832/2087517381,v1:172.21.15.93:6834/2087517381] conn(0x55d445ca2c20 0x55d445ca3000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445ca5760 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445caaa40 0x55d445ca5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 1 --2- 172.21.15.81:0/3946597582 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c85740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29caffd640 0 client.4578 ms_handle_reset on v2:172.21.15.93:6832/2087517381 2023-11-08T16:29:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 1 -- 172.21.15.81:0/3946597582 >> 172.21.15.81:0/3946597582 conn(0x55d445c624b0 msgr2=0x55d445c5be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 1 -- 172.21.15.81:0/3946597582 shutdown_connections 2023-11-08T16:29:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 1 -- 172.21.15.81:0/3946597582 wait complete. 2023-11-08T16:29:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.507+0000 7f29f5572740 20 client.4578 trim_cache size 0 max 16384 2023-11-08T16:29:20.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.524+0000 7f29f5572740 1 Processor -- start 2023-11-08T16:29:20.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.524+0000 7f29f5572740 1 -- start start 2023-11-08T16:29:20.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.525+0000 7f29f5572740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c86310 0x55d445d12640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.525+0000 7f29f5572740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c62e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.525+0000 7f29f5572740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445c90790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.525+0000 7f29f5572740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445c588e0 con 0x55d445caaa40 2023-11-08T16:29:20.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.525+0000 7f29f5572740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445c58b20 con 0x55d445c8a280 2023-11-08T16:29:20.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.525+0000 7f29f5572740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d445c586d0 con 0x55d445c86310 2023-11-08T16:29:20.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.525+0000 7f29f0ea2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445c90790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.525+0000 7f29f0ea2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445c90790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35898/0 (socket says 172.21.15.81:35898) 2023-11-08T16:29:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.525+0000 7f29f0ea2640 1 -- 172.21.15.81:0/1023075155 learned_addr learned my addr 172.21.15.81:0/1023075155 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.525+0000 7f29ebfff640 1 --2- 172.21.15.81:0/1023075155 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c86310 0x55d445d12640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.526+0000 7f29eb7fe640 1 --2- 172.21.15.81:0/1023075155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c62e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.526+0000 7f29f0ea2640 1 -- 172.21.15.81:0/1023075155 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c86310 msgr2=0x55d445d12640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.526+0000 7f29f0ea2640 1 --2- 172.21.15.81:0/1023075155 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c86310 0x55d445d12640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.526+0000 7f29f0ea2640 1 -- 172.21.15.81:0/1023075155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 msgr2=0x55d445c62e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.526+0000 7f29f0ea2640 1 --2- 172.21.15.81:0/1023075155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c62e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.526+0000 7f29f0ea2640 1 -- 172.21.15.81:0/1023075155 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d445b64b90 con 0x55d445caaa40 2023-11-08T16:29:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.526+0000 7f29eb7fe640 1 --2- 172.21.15.81:0/1023075155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c62e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:20.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.526+0000 7f29f0ea2640 1 --2- 172.21.15.81:0/1023075155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445c90790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f29e402cc30 tx=0x7f29e403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce765ab03ecb363 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:20.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.526+0000 7f29e27fc640 1 -- 172.21.15.81:0/1023075155 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4034b30 con 0x55d445caaa40 2023-11-08T16:29:20.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.526+0000 7f29e27fc640 1 -- 172.21.15.81:0/1023075155 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e4034c90 con 0x55d445caaa40 2023-11-08T16:29:20.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.526+0000 7f29e27fc640 1 -- 172.21.15.81:0/1023075155 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4003bd0 con 0x55d445caaa40 2023-11-08T16:29:20.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.527+0000 7f29f5572740 1 -- 172.21.15.81:0/1023075155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 msgr2=0x55d445c90790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:20.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.527+0000 7f29f5572740 1 --2- 172.21.15.81:0/1023075155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445c90790 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f29e402cc30 tx=0x7f29e403e000 comp rx=0 tx=0).stop 2023-11-08T16:29:20.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.527+0000 7f29f5572740 1 -- 172.21.15.81:0/1023075155 shutdown_connections 2023-11-08T16:29:20.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.527+0000 7f29f5572740 1 --2- 172.21.15.81:0/1023075155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445caaa40 0x55d445c90790 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.527+0000 7f29f5572740 1 --2- 172.21.15.81:0/1023075155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445c8a280 0x55d445c62e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.527+0000 7f29f5572740 1 --2- 172.21.15.81:0/1023075155 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c86310 0x55d445d12640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.527+0000 7f29f5572740 1 -- 172.21.15.81:0/1023075155 >> 172.21.15.81:0/1023075155 conn(0x55d445c624b0 msgr2=0x55d445c881b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:20.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.527+0000 7f29f5572740 1 -- 172.21.15.81:0/1023075155 shutdown_connections 2023-11-08T16:29:20.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.527+0000 7f29f5572740 1 -- 172.21.15.81:0/1023075155 wait complete. 2023-11-08T16:29:20.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.528+0000 7f29f5572740 1 -- start start 2023-11-08T16:29:20.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.528+0000 7f29f5572740 1 Processor -- start 2023-11-08T16:29:20.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29f5572740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445cbe930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29f5572740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c8a280 0x55d445c90790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29f5572740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445caaa40 0x55d445c90cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29f5572740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445c9ea00 con 0x55d445c86310 2023-11-08T16:29:20.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29f5572740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d445cbf180 con 0x55d445caaa40 2023-11-08T16:29:20.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29f5572740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d445cbf3d0 con 0x55d445c8a280 2023-11-08T16:29:20.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29f5572740 10 client.0 _ll_register_callbacks cb 0x55d445c617b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:29:20.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29ebfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445cbe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29ebfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445cbe930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35900/0 (socket says 172.21.15.81:35900) 2023-11-08T16:29:20.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29ebfff640 1 -- 172.21.15.81:0/1218601072 learned_addr learned my addr 172.21.15.81:0/1218601072 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:20.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29ebfff640 1 -- 172.21.15.81:0/1218601072 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c8a280 msgr2=0x55d445c90790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:20.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29ebfff640 1 --2- 172.21.15.81:0/1218601072 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c8a280 0x55d445c90790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29ebfff640 1 -- 172.21.15.81:0/1218601072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445caaa40 msgr2=0x55d445c90cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:20.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29ebfff640 1 --2- 172.21.15.81:0/1218601072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445caaa40 0x55d445c90cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.529+0000 7f29ebfff640 1 -- 172.21.15.81:0/1218601072 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d445bc6e40 con 0x55d445c86310 2023-11-08T16:29:20.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29ebfff640 1 --2- 172.21.15.81:0/1218601072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445cbe930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f29d4058ad0 tx=0x7f29d4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de430e540fbad667 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:20.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29caffd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:29:20.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29caffd640 1 -- 172.21.15.81:0/1218601072 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d4060c10 con 0x55d445c86310 2023-11-08T16:29:20.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29caffd640 1 -- 172.21.15.81:0/1218601072 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29d4055020 con 0x55d445c86310 2023-11-08T16:29:20.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29f5572740 1 -- 172.21.15.81:0/1218601072 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29d4070180 con 0x55d445c86310 2023-11-08T16:29:20.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29caffd640 1 -- 172.21.15.81:0/1218601072 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29d40551e0 con 0x55d445c86310 2023-11-08T16:29:20.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29f5572740 10 client.4584 fetch_fsmap learned FSMap version 8 2023-11-08T16:29:20.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29f5572740 1 -- 172.21.15.81:0/1218601072 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x55d445b8a180 con 0x55d445c86310 2023-11-08T16:29:20.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29caffd640 1 -- 172.21.15.81:0/1218601072 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d4080020 con 0x55d445c86310 2023-11-08T16:29:20.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29caffd640 1 -- 172.21.15.81:0/1218601072 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29d4011070 con 0x55d445c86310 2023-11-08T16:29:20.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29f5572740 10 client.4584 fetch_fsmap finished waiting for FSMap version 8 2023-11-08T16:29:20.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29f5572740 10 client.4584 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T16:29:20.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.530+0000 7f29f5572740 20 client.4584 populate_metadata read hostname 'smithi081' 2023-11-08T16:29:20.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.531+0000 7f29f5572740 1 --2- 172.21.15.81:0/1218601072 >> [v2:172.21.15.93:6832/2087517381,v1:172.21.15.93:6834/2087517381] conn(0x55d445c9d9d0 0x55d445c9ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:20.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.531+0000 7f29f5572740 4 client.4584 mds_command: new command op to 4270 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:29:20.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.531+0000 7f29f5572740 1 -- 172.21.15.81:0/1218601072 --> [v2:172.21.15.93:6832/2087517381,v1:172.21.15.93:6834/2087517381] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d445bc14c0 con 0x55d445c9d9d0 2023-11-08T16:29:20.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.531+0000 7f29eb7fe640 1 --2- 172.21.15.81:0/1218601072 >> [v2:172.21.15.93:6832/2087517381,v1:172.21.15.93:6834/2087517381] conn(0x55d445c9d9d0 0x55d445c9ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:20.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.532+0000 7f29eb7fe640 1 --2- 172.21.15.81:0/1218601072 >> [v2:172.21.15.93:6832/2087517381,v1:172.21.15.93:6834/2087517381] conn(0x55d445c9d9d0 0x55d445c9ddb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6b5340f8a07c395 server_cookie=6148fa32f721d621 in_seq=0 out_seq=0 2023-11-08T16:29:20.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.532+0000 7f29caffd640 10 client.4584 ms_handle_connect on v2:172.21.15.93:6832/2087517381 2023-11-08T16:29:20.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.532+0000 7f29caffd640 1 -- 172.21.15.81:0/1218601072 <== mds.0 v2:172.21.15.93:6832/2087517381 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55d445bc14c0 con 0x55d445c9d9d0 2023-11-08T16:29:20.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.532+0000 7f29caffd640 10 client.4584 handle_command_reply: tid=0 2023-11-08T16:29:20.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.532+0000 7f29f5572740 1 client.4584 shutdown 2023-11-08T16:29:20.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f5572740 1 -- 172.21.15.81:0/1218601072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 msgr2=0x55d445cbe930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:20.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f5572740 1 --2- 172.21.15.81:0/1218601072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445cbe930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f29d4058ad0 tx=0x7f29d4000bc0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f5572740 1 -- 172.21.15.81:0/1218601072 shutdown_connections 2023-11-08T16:29:20.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f5572740 1 --2- 172.21.15.81:0/1218601072 >> [v2:172.21.15.93:6832/2087517381,v1:172.21.15.93:6834/2087517381] conn(0x55d445c9d9d0 0x55d445c9ddb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f5572740 1 --2- 172.21.15.81:0/1218601072 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d445caaa40 0x55d445c90cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29caffd640 0 client.4584 ms_handle_reset on v2:172.21.15.93:6832/2087517381 2023-11-08T16:29:20.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f5572740 1 --2- 172.21.15.81:0/1218601072 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d445c8a280 0x55d445c90790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f5572740 1 --2- 172.21.15.81:0/1218601072 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d445c86310 0x55d445cbe930 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f5572740 1 -- 172.21.15.81:0/1218601072 >> 172.21.15.81:0/1218601072 conn(0x55d445c624b0 msgr2=0x55d445c62d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:20.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f5572740 1 -- 172.21.15.81:0/1218601072 shutdown_connections 2023-11-08T16:29:20.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f5572740 1 -- 172.21.15.81:0/1218601072 wait complete. 2023-11-08T16:29:20.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f5572740 20 client.4584 trim_cache size 0 max 16384 2023-11-08T16:29:20.548 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:29:20.548 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:29:20.548 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:29:20.548 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 4270, 2023-11-08T16:29:20.549 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:29:20.549 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:29:20.549 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:29:20.549 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 16.754258696000001, 2023-11-08T16:29:20.549 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 8, 2023-11-08T16:29:20.550 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 21, 2023-11-08T16:29:20.550 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 21, 2023-11-08T16:29:20.550 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 23.004273245 2023-11-08T16:29:20.550 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:29:20.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f2905640 1 -- 172.21.15.81:0/4253112155 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f29cc06aea0 msgr2=0x7f29cc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:20.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f2905640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f29cc06aea0 0x7f29cc06d360 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29dc007fd0 tx=0x7f29dc007960 comp rx=0 tx=0).stop 2023-11-08T16:29:20.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f2905640 1 -- 172.21.15.81:0/4253112155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29ec0c56b0 msgr2=0x7f29ec14f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:20.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.533+0000 7f29f2905640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29ec0c56b0 0x7f29ec14f0d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f29d400e900 tx=0x7f29d400edc0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.534+0000 7f29f2905640 1 -- 172.21.15.81:0/4253112155 shutdown_connections 2023-11-08T16:29:20.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.534+0000 7f29f2905640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f29cc06aea0 0x7f29cc06d360 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.534+0000 7f29f2905640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29ec159530 0x7f29ec148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.534+0000 7f29f2905640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f29ec0ce020 0x7f29ec0c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.534+0000 7f29f2905640 1 --2- 172.21.15.81:0/4253112155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f29ec0c56b0 0x7f29ec14f0d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:20.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.534+0000 7f29f2905640 1 -- 172.21.15.81:0/4253112155 >> 172.21.15.81:0/4253112155 conn(0x7f29ec0ceaa0 msgr2=0x7f29ec0730a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:20.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.534+0000 7f29f2905640 1 -- 172.21.15.81:0/4253112155 shutdown_connections 2023-11-08T16:29:20.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:20.534+0000 7f29f2905640 1 -- 172.21.15.81:0/4253112155 wait complete. 2023-11-08T16:29:20.555 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-08T16:29:20.580 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-08T16:29:20.580 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-08T16:29:20.580 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-08T16:29:20.581 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T16:29:20.581 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:29:20.582 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-08T16:29:20.582 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T16:29:20.582 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-08T16:29:20.583 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-08T16:29:20.583 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T16:29:20.583 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-08T16:29:20.583 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-08T16:29:20.583 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-08T16:29:20.584 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-08T16:29:20.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:20.584 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-11-08T16:29:20.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:20.646 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link delete ceph-brx 2023-11-08T16:29:20.748 INFO:teuthology.orchestra.run.smithi093.stderr:Cannot find device "ceph-brx" 2023-11-08T16:29:20.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:29:20.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:20.752 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:29:20.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:20.814 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link delete ceph-brx 2023-11-08T16:29:20.916 INFO:teuthology.orchestra.run.smithi081.stderr:Cannot find device "ceph-brx" 2023-11-08T16:29:20.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:29:20.922 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-08T16:29:20.922 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:29:21.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.133+0000 7f1396b39640 1 Processor -- start 2023-11-08T16:29:21.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.133+0000 7f1396b39640 1 -- start start 2023-11-08T16:29:21.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.135+0000 7f1396b39640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f13900cc5e0 0x7f13900cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:21.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.135+0000 7f1396b39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900ccfb0 0x7f1390070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:21.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.135+0000 7f1396b39640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13900c9a70 con 0x7f13900adcd0 2023-11-08T16:29:21.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.135+0000 7f1396b39640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13900cc3e0 con 0x7f13900cc5e0 2023-11-08T16:29:21.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.135+0000 7f1396b39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f139006d070 con 0x7f13900ccfb0 2023-11-08T16:29:21.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.135+0000 7f13948d5640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f13900adcd0 0x7f13900ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45418/0 (socket says 172.21.15.81:45418) 2023-11-08T16:29:21.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.135+0000 7f13948d5640 1 -- 172.21.15.81:0/1698080394 learned_addr learned my addr 172.21.15.81:0/1698080394 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:21.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.135+0000 7f13950d6640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900ccfb0 0x7f1390070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:21.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.135+0000 7f138ffff640 1 --2- 172.21.15.81:0/1698080394 >> v2:172.21.15.93:3300/0 conn(0x7f13900cc5e0 0x7f13900cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:21.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.135+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 699649836 0 0) 0x7f13900c9a70 con 0x7f13900adcd0 2023-11-08T16:29:21.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.135+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1380003520 con 0x7f13900adcd0 2023-11-08T16:29:21.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 104902714 0 0) 0x7f1380003520 con 0x7f13900adcd0 2023-11-08T16:29:21.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900ccfb0 msgr2=0x7f1390070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:21.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900ccfb0 0x7f1390070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:21.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 >> v2:172.21.15.93:3300/0 conn(0x7f13900cc5e0 msgr2=0x7f13900cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:21.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 --2- 172.21.15.81:0/1698080394 >> v2:172.21.15.93:3300/0 conn(0x7f13900cc5e0 0x7f13900cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:21.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f139006d350 con 0x7f13900adcd0 2023-11-08T16:29:21.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f1384001670 con 0x7f13900adcd0 2023-11-08T16:29:21.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 >> v1:172.21.15.81:6789/0 conn(0x7f13900adcd0 legacy=0x7f13900ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:21.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900ccfb0 0x7f1380003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:21.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1380004020 0x7f13800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:21.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1380004d00 0x7f1380005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:21.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13900cc3e0 con 0x7f1380004020 2023-11-08T16:29:21.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139006d070 con 0x7f1380004d00 2023-11-08T16:29:21.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138000ba60 con 0x7f13900ccfb0 2023-11-08T16:29:21.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.136+0000 7f138ffff640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1380004020 0x7f13800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:21.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.137+0000 7f13948d5640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900ccfb0 0x7f1380003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:21.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.137+0000 7f138ffff640 1 -- 172.21.15.81:0/1698080394 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900ccfb0 msgr2=0x7f1380003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:21.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.137+0000 7f138ffff640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900ccfb0 0x7f1380003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:21.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.137+0000 7f138ffff640 1 -- 172.21.15.81:0/1698080394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1380004d00 msgr2=0x7f1380005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:21.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.137+0000 7f138ffff640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1380004d00 0x7f1380005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:21.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.137+0000 7f138ffff640 1 -- 172.21.15.81:0/1698080394 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1380003520 con 0x7f1380004020 2023-11-08T16:29:21.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.137+0000 7f13948d5640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900ccfb0 0x7f1380003a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:21.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.138+0000 7f138ffff640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1380004020 0x7f13800047c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f137c009650 tx=0x7f13800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc95d4746f20461 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:21.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.138+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c02e840 con 0x7f1380004020 2023-11-08T16:29:21.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.138+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c02e9a0 con 0x7f1380004020 2023-11-08T16:29:21.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.138+0000 7f138f7fe640 1 -- 172.21.15.81:0/1698080394 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c02ed30 con 0x7f1380004020 2023-11-08T16:29:21.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.139+0000 7f1396b39640 1 -- 172.21.15.81:0/1698080394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1380004020 msgr2=0x7f13800047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:21.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.139+0000 7f1396b39640 1 --2- 172.21.15.81:0/1698080394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1380004020 0x7f13800047c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f137c009650 tx=0x7f13800057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:21.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.139+0000 7f13950d6640 1 -- 172.21.15.81:0/1698080394 reap_dead start 2023-11-08T16:29:21.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.139+0000 7f1396b39640 1 -- 172.21.15.81:0/1698080394 shutdown_connections 2023-11-08T16:29:21.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.139+0000 7f1396b39640 1 -- 172.21.15.81:0/1698080394 >> 172.21.15.81:0/1698080394 conn(0x7f13900cb490 msgr2=0x7f13900cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:21.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.139+0000 7f1396b39640 1 -- 172.21.15.81:0/1698080394 shutdown_connections 2023-11-08T16:29:21.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.139+0000 7f1396b39640 1 -- 172.21.15.81:0/1698080394 wait complete. 2023-11-08T16:29:21.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.141+0000 7f1396b39640 1 Processor -- start 2023-11-08T16:29:21.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.141+0000 7f1396b39640 1 -- start start 2023-11-08T16:29:21.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f1396b39640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1380004020 0x7f139014ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:21.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f1396b39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900b2c20 0x7f13900b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:21.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f1396b39640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13900c5d90 0x7f13900c6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:21.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f1396b39640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13900b35c0 con 0x7f13900c5d90 2023-11-08T16:29:21.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f1396b39640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139014f360 con 0x7f1380004020 2023-11-08T16:29:21.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f1396b39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13900c6b60 con 0x7f13900b2c20 2023-11-08T16:29:21.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f13950d6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13900c5d90 0x7f13900c6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:21.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f13950d6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13900c5d90 0x7f13900c6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43890/0 (socket says 172.21.15.81:43890) 2023-11-08T16:29:21.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f13950d6640 1 -- 172.21.15.81:0/3051555094 learned_addr learned my addr 172.21.15.81:0/3051555094 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:21.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f138ffff640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900b2c20 0x7f13900b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:21.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f13950d6640 1 -- 172.21.15.81:0/3051555094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900b2c20 msgr2=0x7f13900b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:21.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f13950d6640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900b2c20 0x7f13900b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:21.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f13950d6640 1 -- 172.21.15.81:0/3051555094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1380004020 msgr2=0x7f139014ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:21.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f13948d5640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1380004020 0x7f139014ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:21.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f13950d6640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1380004020 0x7f139014ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:21.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f13950d6640 1 -- 172.21.15.81:0/3051555094 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13900c6e40 con 0x7f13900c5d90 2023-11-08T16:29:21.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.142+0000 7f138ffff640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900b2c20 0x7f13900b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:21.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.143+0000 7f13950d6640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13900c5d90 0x7f13900c6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f139006d1b0 tx=0x7f138800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:21.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.143+0000 7f138d7fa640 1 -- 172.21.15.81:0/3051555094 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1388010040 con 0x7f13900c5d90 2023-11-08T16:29:21.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.143+0000 7f138d7fa640 1 -- 172.21.15.81:0/3051555094 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138800bae0 con 0x7f13900c5d90 2023-11-08T16:29:21.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.143+0000 7f138d7fa640 1 -- 172.21.15.81:0/3051555094 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138800be70 con 0x7f13900c5d90 2023-11-08T16:29:21.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.143+0000 7f1396b39640 1 -- 172.21.15.81:0/3051555094 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13900c7120 con 0x7f13900c5d90 2023-11-08T16:29:21.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.143+0000 7f1396b39640 1 -- 172.21.15.81:0/3051555094 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13901e5670 con 0x7f13900c5d90 2023-11-08T16:29:21.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.144+0000 7f1396b39640 1 -- 172.21.15.81:0/3051555094 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1390072170 con 0x7f13900c5d90 2023-11-08T16:29:21.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.145+0000 7f138d7fa640 1 -- 172.21.15.81:0/3051555094 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f138801e410 con 0x7f13900c5d90 2023-11-08T16:29:21.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.146+0000 7f138d7fa640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 0x7f137406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:21.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.146+0000 7f138d7fa640 1 -- 172.21.15.81:0/3051555094 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1388089960 con 0x7f13900c5d90 2023-11-08T16:29:21.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.146+0000 7f13948d5640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 0x7f137406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:21.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.147+0000 7f13948d5640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 0x7f137406d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1384001eb0 tx=0x7f1384009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:21.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.150+0000 7f138d7fa640 1 -- 172.21.15.81:0/3051555094 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f13880548e0 con 0x7f13900c5d90 2023-11-08T16:29:21.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.378+0000 7f1396b39640 1 -- 172.21.15.81:0/3051555094 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f13901e54d0 con 0x7f13900c5d90 2023-11-08T16:29:21.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.379+0000 7f138d7fa640 1 -- 172.21.15.81:0/3051555094 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f13901e54d0 con 0x7f13900c5d90 2023-11-08T16:29:21.381 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T16:29:21.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 -- 172.21.15.81:0/3051555094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 msgr2=0x7f137406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:21.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 0x7f137406d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f1384001eb0 tx=0x7f1384009040 comp rx=0 tx=0).stop 2023-11-08T16:29:21.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 -- 172.21.15.81:0/3051555094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13900c5d90 msgr2=0x7f13900c6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:21.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13900c5d90 0x7f13900c6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f139006d1b0 tx=0x7f138800b480 comp rx=0 tx=0).stop 2023-11-08T16:29:21.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 -- 172.21.15.81:0/3051555094 shutdown_connections 2023-11-08T16:29:21.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 0x7f137406d330 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:21.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f13900c5d90 0x7f13900c6210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:21.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13900b2c20 0x7f13900b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:21.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 --2- 172.21.15.81:0/3051555094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1380004020 0x7f139014ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:21.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 -- 172.21.15.81:0/3051555094 >> 172.21.15.81:0/3051555094 conn(0x7f13900cb490 msgr2=0x7f139015ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:21.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 -- 172.21.15.81:0/3051555094 shutdown_connections 2023-11-08T16:29:21.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:21.382+0000 7f1396b39640 1 -- 172.21.15.81:0/3051555094 wait complete. 2023-11-08T16:29:21.402 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:29:21.403 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:29:21.403 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:29:21.403 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:29:21.404 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:29:21.404 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:29:21.404 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:29:21.404 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:29:21.405 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:29:21.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:21.405 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:29:21.458 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:29:21.458 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:29:21.458 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:29:21.458 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:29:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:21.459 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:21.459 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:29:21.460 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:21.460 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:21.460 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:29:21.460 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:21.460 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:21.461 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:29:21.461 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:21.461 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:29:21.461 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42348sec preferred_lft 42348sec 2023-11-08T16:29:21.461 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:29:21.462 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:21.463 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-08T16:29:21.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:21.464 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:29:21.464 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:29:21.464 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add name ceph-brx type bridge 2023-11-08T16:29:21.465 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip addr flush dev ceph-brx 2023-11-08T16:29:21.465 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set ceph-brx up 2023-11-08T16:29:21.465 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-08T16:29:21.465 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:29:21.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:21.951 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:29:22.034 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:29:22.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:22.038 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:29:22.101 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:29:22.101 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:29:22.101 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:29:22.102 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:29:22.102 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:29:22.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:22.103 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:29:22.104 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:29:22.104 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:29:22.104 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:29:22.105 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:29:22.105 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:29:22.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:22.870 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:29:22.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:22.927 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list-id 2023-11-08T16:29:22.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:22.981 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:29:22.981 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:29:22.982 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:22.982 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-08T16:29:22.983 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:29:23.125 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-08T16:29:23.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:23.126 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:29:23.126 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:29:23.126 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-08T16:29:23.127 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-08T16:29:23.127 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-08T16:29:23.127 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-08T16:29:23.128 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-08T16:29:23.128 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:29:23.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:23.406 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:29:23.407 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:29:23.407 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:29:23.407 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set dev brx.0 master ceph-brx 2023-11-08T16:29:23.408 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:29:23.570 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:29:23.571 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:29:23.571 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:29:23.619 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:29:23.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:29:23.620 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:29:23.670 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:29:23.765 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:29:23.825 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:29:23.825 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:29:23.825 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:29:23.825 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:29:23.826 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:29:23.826 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:29:23.826 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:29:23.826 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:29:23.826 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:29:23.827 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:29:23.827 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:29:23.827 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:29:23.827 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:29:23.827 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:29:23.828 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:29:23.828 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:29:23.828 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:29:23.828 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:29:23.828 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:29:23.829 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:29:23.829 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:29:23.829 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:29:23.829 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:29:23.829 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:29:23.830 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:29:23.830 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:29:23.830 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:29:23.830 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:29:23.830 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:29:23.831 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:23.831 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:23.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:23.832 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:29:23.885 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:29:23.885 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:29:23.890 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:29:23.963 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:29:24.005 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:29:24.006 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:29:24.006 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:29:24.006 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:29:24.006 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:29:24.006 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:29:24.007 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:29:24.007 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:29:24.007 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:29:24.007 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:29:24.007 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:29:24.008 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:29:24.008 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:29:24.008 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:29:24.008 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:29:24.008 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:29:24.009 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:29:24.009 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:29:24.009 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:29:24.009 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:29:24.009 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:29:24.010 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:29:24.010 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:29:24.010 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:29:24.010 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:29:24.010 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:29:24.011 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:29:24.011 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:29:24.011 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:29:24.011 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:24.011 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:24.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:24.013 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:29:24.133 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:29:24.131+0000 7f04cb645480 -1 init, newargv = 0x55ed562280e0 newargc=15 2023-11-08T16:29:24.133 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[95433]: starting ceph client 2023-11-08T16:29:24.146 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[95433]: starting fuse 2023-11-08T16:29:25.204 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:29:25.310 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:29:25.366 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:29:25.366 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:29:25.367 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:29:25.367 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:29:25.367 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:29:25.367 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:29:25.367 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:29:25.368 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:29:25.368 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:29:25.368 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:29:25.368 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:29:25.369 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:29:25.369 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:29:25.369 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:29:25.369 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:29:25.369 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:29:25.370 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:29:25.370 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:29:25.370 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:29:25.370 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:29:25.370 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:29:25.371 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:29:25.371 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:29:25.371 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:29:25.371 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:29:25.371 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:29:25.372 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:29:25.372 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:29:25.372 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:29:25.372 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:25.372 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:25.373 INFO:teuthology.orchestra.run.smithi081.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-08T16:29:25.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:25.373 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:29:25.429 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:29:25.430 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:29:25.430 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:29:25.430 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:29:25.431 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:29:25.431 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:29:25.432 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:29:25.432 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:29:25.432 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:29:25.433 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:29:25.433 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:29:25.433 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:29:25.433 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:29:25.434 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:29:25.434 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:29:25.435 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:29:25.435 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:29:25.435 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:29:25.435 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:29:25.436 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:29:25.436 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:29:25.436 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:29:25.437 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:29:25.437 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:29:25.437 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:29:25.437 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:29:25.438 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:29:25.438 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:29:25.438 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:29:25.616 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.95433.asok 2023-11-08T16:29:25.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:29:25.635 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.95433.asok 2023-11-08T16:29:25.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:25.636 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.95433.asok status 2023-11-08T16:29:25.878 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:29:25.878 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:29:25.879 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:29:25.879 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:29:25.879 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:29:25.879 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:29:25.880 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:29:25.880 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "95433", 2023-11-08T16:29:25.880 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:29:25.880 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:29:25.881 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:29:25.881 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:29:25.881 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 4599, 2023-11-08T16:29:25.881 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:29:25.881 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:29:25.882 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:29:25.882 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 4599 2023-11-08T16:29:25.882 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:29:25.882 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:29:25.882 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:29:25.883 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:29:25.883 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2761055888 2023-11-08T16:29:25.883 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:29:25.883 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:29:25.883 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:29:25.883 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:29:25.884 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:29:25.884 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2761055888 2023-11-08T16:29:25.884 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:29:25.884 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.4599 192.168.0.1:0/2761055888", 2023-11-08T16:29:25.884 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/2761055888", 2023-11-08T16:29:25.885 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:29:25.885 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 8, 2023-11-08T16:29:25.885 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 21, 2023-11-08T16:29:25.885 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:29:25.885 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:29:25.886 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:29:25.886 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:29:25.891 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:29:26.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.101+0000 7ff7673a5640 1 Processor -- start 2023-11-08T16:29:26.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.101+0000 7ff7673a5640 1 -- start start 2023-11-08T16:29:26.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.103+0000 7ff7673a5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff76015d490 0x7ff7600d39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:26.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.103+0000 7ff7673a5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7600ce890 con 0x7ff7600d1bf0 2023-11-08T16:29:26.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.103+0000 7ff7673a5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7600ceb70 con 0x7ff7600d1080 2023-11-08T16:29:26.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.103+0000 7ff7673a5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7600cee50 con 0x7ff76015d490 2023-11-08T16:29:26.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.103+0000 7ff766ba4640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff7600d1bf0 0x7ff7600b5e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45432/0 (socket says 172.21.15.81:45432) 2023-11-08T16:29:26.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.103+0000 7ff766ba4640 1 -- 172.21.15.81:0/3464870219 learned_addr learned my addr 172.21.15.81:0/3464870219 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:26.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.103+0000 7ff7663a3640 1 --2- 172.21.15.81:0/3464870219 >> v2:172.21.15.93:3300/0 conn(0x7ff76015d490 0x7ff7600d39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:26.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3282962289 0 0) 0x7ff7600ce890 con 0x7ff7600d1bf0 2023-11-08T16:29:26.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff74c003520 con 0x7ff7600d1bf0 2023-11-08T16:29:26.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3823944112 0 0) 0x7ff74c003520 con 0x7ff7600d1bf0 2023-11-08T16:29:26.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 >> v2:172.21.15.93:3300/0 conn(0x7ff76015d490 msgr2=0x7ff7600d39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:26.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 --2- 172.21.15.81:0/3464870219 >> v2:172.21.15.93:3300/0 conn(0x7ff76015d490 0x7ff7600d39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 >> v1:172.21.15.93:6789/0 conn(0x7ff7600d1080 legacy=0x7ff7600d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:26.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7600cf130 con 0x7ff7600d1bf0 2023-11-08T16:29:26.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ff7580019f0 con 0x7ff7600d1bf0 2023-11-08T16:29:26.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 >> v1:172.21.15.81:6789/0 conn(0x7ff7600d1bf0 legacy=0x7ff7600b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:26.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff74c003aa0 0x7ff74c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:26.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff74c0047c0 0x7ff74c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:26.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff74c0052a0 0x7ff74c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:26.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7600cee50 con 0x7ff74c0052a0 2023-11-08T16:29:26.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c00c150 con 0x7ff74c003aa0 2023-11-08T16:29:26.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c00c310 con 0x7ff74c0047c0 2023-11-08T16:29:26.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.104+0000 7ff766ba4640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff74c0052a0 0x7ff74c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:26.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.105+0000 7ff7663a3640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff74c003aa0 0x7ff74c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:26.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.105+0000 7ff765ba2640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff74c0047c0 0x7ff74c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:26.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.105+0000 7ff766ba4640 1 -- 172.21.15.81:0/3464870219 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff74c0047c0 msgr2=0x7ff74c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:26.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.105+0000 7ff766ba4640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff74c0047c0 0x7ff74c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.105+0000 7ff766ba4640 1 -- 172.21.15.81:0/3464870219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff74c003aa0 msgr2=0x7ff74c003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:26.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.105+0000 7ff766ba4640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff74c003aa0 0x7ff74c003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.105+0000 7ff766ba4640 1 -- 172.21.15.81:0/3464870219 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c003520 con 0x7ff74c0052a0 2023-11-08T16:29:26.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.105+0000 7ff7663a3640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff74c003aa0 0x7ff74c003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:26.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.106+0000 7ff766ba4640 1 -- 172.21.15.81:0/3464870219 reap_dead start 2023-11-08T16:29:26.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.106+0000 7ff766ba4640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff74c0052a0 0x7ff74c00b980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff758002230 tx=0x7ff74c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24351d31a54e3f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:26.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.106+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758034630 con 0x7ff74c0052a0 2023-11-08T16:29:26.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.106+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75802f070 con 0x7ff74c0052a0 2023-11-08T16:29:26.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.106+0000 7ff7653a1640 1 -- 172.21.15.81:0/3464870219 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7580349d0 con 0x7ff74c0052a0 2023-11-08T16:29:26.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.107+0000 7ff7673a5640 1 -- 172.21.15.81:0/3464870219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff74c0052a0 msgr2=0x7ff74c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:26.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.107+0000 7ff7673a5640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff74c0052a0 0x7ff74c00b980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff758002230 tx=0x7ff74c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.107+0000 7ff7673a5640 1 -- 172.21.15.81:0/3464870219 shutdown_connections 2023-11-08T16:29:26.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.107+0000 7ff7673a5640 1 --2- 172.21.15.81:0/3464870219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff74c0052a0 0x7ff74c00b980 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.107+0000 7ff7673a5640 1 -- 172.21.15.81:0/3464870219 >> 172.21.15.81:0/3464870219 conn(0x7ff7600d2e50 msgr2=0x7ff7600d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:26.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.107+0000 7ff7673a5640 1 -- 172.21.15.81:0/3464870219 shutdown_connections 2023-11-08T16:29:26.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.107+0000 7ff7673a5640 1 -- 172.21.15.81:0/3464870219 wait complete. 2023-11-08T16:29:26.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.109+0000 7ff7673a5640 1 Processor -- start 2023-11-08T16:29:26.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.109+0000 7ff7673a5640 1 -- start start 2023-11-08T16:29:26.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.110+0000 7ff7673a5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff74c0052a0 0x7ff76015e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:26.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.110+0000 7ff7673a5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7601f60b0 0x7ff7600cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:26.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.110+0000 7ff7673a5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760159280 0x7ff76015b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:26.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.110+0000 7ff7673a5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76015ee10 con 0x7ff760159280 2023-11-08T16:29:26.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.110+0000 7ff7673a5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601f6540 con 0x7ff7601f60b0 2023-11-08T16:29:26.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.110+0000 7ff7673a5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7601f6820 con 0x7ff74c0052a0 2023-11-08T16:29:26.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.110+0000 7ff766ba4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760159280 0x7ff76015b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:26.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.110+0000 7ff766ba4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760159280 0x7ff76015b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43906/0 (socket says 172.21.15.81:43906) 2023-11-08T16:29:26.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.110+0000 7ff766ba4640 1 -- 172.21.15.81:0/4085014451 learned_addr learned my addr 172.21.15.81:0/4085014451 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:26.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff765ba2640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7601f60b0 0x7ff7600cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:26.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff7663a3640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff74c0052a0 0x7ff76015e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:26.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff766ba4640 1 -- 172.21.15.81:0/4085014451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff74c0052a0 msgr2=0x7ff76015e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:26.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff766ba4640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff74c0052a0 0x7ff76015e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff766ba4640 1 -- 172.21.15.81:0/4085014451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7601f60b0 msgr2=0x7ff7600cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:26.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff766ba4640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7601f60b0 0x7ff7600cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff766ba4640 1 -- 172.21.15.81:0/4085014451 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7601f6b00 con 0x7ff760159280 2023-11-08T16:29:26.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff7663a3640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff74c0052a0 0x7ff76015e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:26.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff766ba4640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760159280 0x7ff76015b930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff758002230 tx=0x7ff758001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:26.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff756ffd640 1 -- 172.21.15.81:0/4085014451 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758034840 con 0x7ff760159280 2023-11-08T16:29:26.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff756ffd640 1 -- 172.21.15.81:0/4085014451 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7580349a0 con 0x7ff760159280 2023-11-08T16:29:26.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.111+0000 7ff7673a5640 1 -- 172.21.15.81:0/4085014451 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7601f6de0 con 0x7ff760159280 2023-11-08T16:29:26.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.112+0000 7ff756ffd640 1 -- 172.21.15.81:0/4085014451 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff758034d50 con 0x7ff760159280 2023-11-08T16:29:26.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.112+0000 7ff7673a5640 1 -- 172.21.15.81:0/4085014451 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff760159730 con 0x7ff760159280 2023-11-08T16:29:26.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.113+0000 7ff7673a5640 1 -- 172.21.15.81:0/4085014451 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7601f7020 con 0x7ff760159280 2023-11-08T16:29:26.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.114+0000 7ff756ffd640 1 -- 172.21.15.81:0/4085014451 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff758031030 con 0x7ff760159280 2023-11-08T16:29:26.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.115+0000 7ff756ffd640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74406ae70 0x7ff74406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:26.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.115+0000 7ff756ffd640 1 -- 172.21.15.81:0/4085014451 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7580ab2c0 con 0x7ff760159280 2023-11-08T16:29:26.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.115+0000 7ff7663a3640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74406ae70 0x7ff74406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:26.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.116+0000 7ff7663a3640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74406ae70 0x7ff74406d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009a50 tx=0x7ff75c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:26.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.119+0000 7ff756ffd640 1 -- 172.21.15.81:0/4085014451 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff758076410 con 0x7ff760159280 2023-11-08T16:29:26.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.352+0000 7ff7673a5640 1 -- 172.21.15.81:0/4085014451 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff760159950 con 0x7ff760159280 2023-11-08T16:29:26.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.353+0000 7ff756ffd640 1 -- 172.21.15.81:0/4085014451 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff758079a30 con 0x7ff760159280 2023-11-08T16:29:26.355 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T16:29:26.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.356+0000 7ff7673a5640 1 -- 172.21.15.81:0/4085014451 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74406ae70 msgr2=0x7ff74406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:26.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.356+0000 7ff7673a5640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74406ae70 0x7ff74406d330 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff75c009a50 tx=0x7ff75c008040 comp rx=0 tx=0).stop 2023-11-08T16:29:26.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.356+0000 7ff7673a5640 1 -- 172.21.15.81:0/4085014451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760159280 msgr2=0x7ff76015b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:26.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.356+0000 7ff7673a5640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760159280 0x7ff76015b930 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff758002230 tx=0x7ff758001cb0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.356+0000 7ff7673a5640 1 -- 172.21.15.81:0/4085014451 shutdown_connections 2023-11-08T16:29:26.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.356+0000 7ff7673a5640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74406ae70 0x7ff74406d330 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.356+0000 7ff7673a5640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760159280 0x7ff76015b930 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.356+0000 7ff7673a5640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7601f60b0 0x7ff7600cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.356+0000 7ff7673a5640 1 --2- 172.21.15.81:0/4085014451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff74c0052a0 0x7ff76015e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:26.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.356+0000 7ff7673a5640 1 -- 172.21.15.81:0/4085014451 >> 172.21.15.81:0/4085014451 conn(0x7ff7600d2e50 msgr2=0x7ff7600b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:26.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.357+0000 7ff7673a5640 1 -- 172.21.15.81:0/4085014451 shutdown_connections 2023-11-08T16:29:26.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:26.357+0000 7ff7673a5640 1 -- 172.21.15.81:0/4085014451 wait complete. 2023-11-08T16:29:26.376 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:29:26.377 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:29:26.377 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:29:26.377 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-08T16:29:26.378 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-08T16:29:26.378 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:29:26.378 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:29:26.378 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:29:26.379 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-08T16:29:26.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:26.379 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:29:26.426 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:29:26.426 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:29:26.426 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:29:26.427 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:26.427 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:29:26.427 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:26.427 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:26.427 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:26.428 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:29:26.428 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:26.428 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:26.428 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:29:26.428 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:26.428 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:26.429 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:29:26.429 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:26.429 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:29:26.429 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42343sec preferred_lft 42343sec 2023-11-08T16:29:26.429 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:29:26.430 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:26.430 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:29:26.430 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:26.430 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:29:26.430 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:26.431 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:29:26.431 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:26.431 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:29:26.431 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:26.431 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:29:26.432 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:26.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:26.432 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:29:26.553 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:29:26.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:26.557 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:29:26.609 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:29:26.609 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:29:26.610 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:29:26.610 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:29:26.610 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:29:26.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:26.611 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:29:26.611 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:29:26.611 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:29:26.612 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:29:26.612 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:29:26.612 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:29:26.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:26.786 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:29:26.825 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:29:26.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:26.826 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list-id 2023-11-08T16:29:26.913 INFO:teuthology.orchestra.run.smithi081.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-08T16:29:26.914 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:26.915 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:29:26.915 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:29:26.915 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:29:26.915 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-08T16:29:26.916 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:29:27.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:27.069 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-08T16:29:27.144 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:29:27.144 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:29:27.144 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:29:27.144 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:27.145 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:29:27.145 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:27.145 INFO:teuthology.orchestra.run.smithi081.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:29:27.145 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether f2:aa:c0:e0:da:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T16:29:27.145 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T16:29:27.146 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:27.146 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::f0aa:c0ff:fee0:da45/64 scope link 2023-11-08T16:29:27.146 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:27.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:27.148 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-08T16:29:27.232 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:29:27.233 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:29:27.233 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:29:27.233 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:27.233 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:29:27.233 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:27.234 INFO:teuthology.orchestra.run.smithi081.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:29:27.234 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether f2:aa:c0:e0:da:45 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T16:29:27.234 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T16:29:27.234 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:27.235 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::f0aa:c0ff:fee0:da45/64 scope link 2023-11-08T16:29:27.235 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:27.236 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-08T16:29:27.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:27.237 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:29:27.238 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:29:27.238 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-08T16:29:27.238 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-08T16:29:27.239 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-08T16:29:27.239 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-08T16:29:27.239 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-08T16:29:27.239 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:29:27.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:27.475 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:29:27.476 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:29:27.476 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.1 up 2023-11-08T16:29:27.476 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set dev brx.1 master ceph-brx 2023-11-08T16:29:27.477 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:29:27.622 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-08T16:29:27.622 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:29:27.623 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-08T16:29:27.663 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-08T16:29:27.664 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:29:27.664 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-08T16:29:27.756 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:29:27.840 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:29:28.007 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:29:28.007 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:29:28.008 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:29:28.008 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:29:28.008 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:29:28.008 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:29:28.008 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:29:28.009 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:29:28.009 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:29:28.009 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:29:28.009 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:29:28.009 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:29:28.010 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:29:28.010 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:29:28.010 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:29:28.010 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:29:28.010 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:29:28.011 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:29:28.011 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:29:28.011 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:29:28.011 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:29:28.011 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:29:28.012 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:29:28.012 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:29:28.012 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:29:28.012 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:29:28.012 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:29:28.013 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:29:28.013 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:29:28.013 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:28.013 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:28.013 INFO:teuthology.orchestra.run.smithi081.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-08T16:29:28.014 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:29:28.014 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:29:28.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:28.015 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:29:28.071 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:29:28.071 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2023-11-08T16:29:28.072 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-11-08T16:29:28.079 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:29:28.162 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:29:28.204 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:29:28.204 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:29:28.204 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:29:28.204 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:29:28.205 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:29:28.205 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:29:28.205 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:29:28.205 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:29:28.205 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:29:28.206 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:29:28.206 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:29:28.206 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:29:28.206 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:29:28.207 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:29:28.207 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:29:28.207 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:29:28.207 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:29:28.207 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:29:28.207 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:29:28.208 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:29:28.208 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:29:28.208 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:29:28.208 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:29:28.208 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:29:28.209 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:29:28.209 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:29:28.209 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:29:28.209 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:29:28.209 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:29:28.210 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:28.210 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:29:28.210 INFO:teuthology.orchestra.run.smithi081.stdout:/home/ubuntu/cephtest/mnt.0 2023-11-08T16:29:28.210 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:29:28.210 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:29:28.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:28.212 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:29:28.262 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi081.stderr:2023-11-08T16:29:28.261+0000 7f9c97843480 -1 init, newargv = 0x563bcf466e90 newargc=15 2023-11-08T16:29:28.263 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi081.stderr:ceph-fuse[95991]: starting ceph client 2023-11-08T16:29:28.272 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi081.stderr:ceph-fuse[95991]: starting fuse 2023-11-08T16:29:28.294 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:29:28.295 INFO:teuthology.orchestra.run.smithi081.stdout:42 2023-11-08T16:29:28.295 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 42] 2023-11-08T16:29:28.295 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:29:28.296 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:29:28.296 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:29:28.296 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:29:28.297 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:29:28.297 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:29:28.297 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:29:28.297 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-08T16:29:28.298 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:29:28.298 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-08T16:29:28.298 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:29:28.299 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:29:28.299 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:29:28.299 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:29:28.300 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:29:28.300 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:29:28.300 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:29:28.300 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:29:28.301 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:29:28.301 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:29:28.301 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:29:28.301 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:29:28.302 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:29:28.302 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:29:28.303 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:29:28.303 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.2")) 2023-11-08T16:29:28.303 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:29:28.495 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.2.95991.asok 2023-11-08T16:29:28.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:29:28.511 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.95991.asok 2023-11-08T16:29:28.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:28.512 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.95991.asok status 2023-11-08T16:29:28.750 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:29:28.751 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:29:28.751 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:29:28.751 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:29:28.751 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "2", 2023-11-08T16:29:28.752 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:29:28.752 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-08T16:29:28.752 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "95991", 2023-11-08T16:29:28.752 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:29:28.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:29:28.753 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:29:28.753 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:29:28.753 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 4611, 2023-11-08T16:29:28.753 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:29:28.753 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:29:28.754 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:29:28.754 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 4611 2023-11-08T16:29:28.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:29:28.754 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:29:28.754 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:29:28.755 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.2:0", 2023-11-08T16:29:28.755 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1390461209 2023-11-08T16:29:28.755 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:29:28.755 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:29:28.755 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:29:28.756 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:29:28.756 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.2:0", 2023-11-08T16:29:28.756 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1390461209 2023-11-08T16:29:28.756 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:29:28.756 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.4611 192.168.0.2:0/1390461209", 2023-11-08T16:29:28.757 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.2:0/1390461209", 2023-11-08T16:29:28.757 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:29:28.757 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 8, 2023-11-08T16:29:28.757 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 21, 2023-11-08T16:29:28.757 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:29:28.758 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:29:28.758 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:29:28.758 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:29:28.765 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:29:28.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.966+0000 7f0eb9f54640 1 Processor -- start 2023-11-08T16:29:28.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.967+0000 7f0eb9f54640 1 -- start start 2023-11-08T16:29:28.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.968+0000 7f0eb9f54640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0eb40cff50 0x7f0eb4161230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:28.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.968+0000 7f0eb9f54640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f0eb4161770 0x7f0eb40cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:28.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.968+0000 7f0eb9f54640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb415da90 con 0x7f0eb415cfa0 2023-11-08T16:29:28.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.968+0000 7f0eb9f54640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb415dce0 con 0x7f0eb40cff50 2023-11-08T16:29:28.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.968+0000 7f0eb9f54640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb415dfc0 con 0x7f0eb4161770 2023-11-08T16:29:28.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.969+0000 7f0eb3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0eb40cff50 0x7f0eb4161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:28.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.969+0000 7f0eb3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0eb40cff50 0x7f0eb4161230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43920/0 (socket says 172.21.15.81:43920) 2023-11-08T16:29:28.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.969+0000 7f0eb3fff640 1 -- 172.21.15.81:0/4004919935 learned_addr learned my addr 172.21.15.81:0/4004919935 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:28.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.969+0000 7f0eb9753640 1 --2- 172.21.15.81:0/4004919935 >> v2:172.21.15.93:3300/0 conn(0x7f0eb4161770 0x7f0eb40cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:28.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.969+0000 7f0eb3fff640 1 -- 172.21.15.81:0/4004919935 >> v2:172.21.15.93:3300/0 conn(0x7f0eb4161770 msgr2=0x7f0eb40cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:28.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.969+0000 7f0eb3fff640 1 --2- 172.21.15.81:0/4004919935 >> v2:172.21.15.93:3300/0 conn(0x7f0eb4161770 0x7f0eb40cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:28.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.969+0000 7f0eb3fff640 1 -- 172.21.15.81:0/4004919935 >> v1:172.21.15.93:6789/0 conn(0x7f0eb415cfa0 legacy=0x7f0eb415d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:28.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.969+0000 7f0eb3fff640 1 -- 172.21.15.81:0/4004919935 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb415e2a0 con 0x7f0eb40cff50 2023-11-08T16:29:28.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.970+0000 7f0eb3fff640 1 --2- 172.21.15.81:0/4004919935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40cff50 0x7f0eb4161230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4073d50 tx=0x7f0ea002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e7e611d7c3a8bf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:28.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.971+0000 7f0eb37fe640 1 -- 172.21.15.81:0/4004919935 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0031870 con 0x7f0eb40cff50 2023-11-08T16:29:28.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.971+0000 7f0eb37fe640 1 -- 172.21.15.81:0/4004919935 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea00319d0 con 0x7f0eb40cff50 2023-11-08T16:29:28.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.971+0000 7f0eb37fe640 1 -- 172.21.15.81:0/4004919935 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0031d60 con 0x7f0eb40cff50 2023-11-08T16:29:28.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.971+0000 7f0eb9f54640 1 -- 172.21.15.81:0/4004919935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40cff50 msgr2=0x7f0eb4161230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:28.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.971+0000 7f0eb9f54640 1 --2- 172.21.15.81:0/4004919935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40cff50 0x7f0eb4161230 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4073d50 tx=0x7f0ea002b910 comp rx=0 tx=0).stop 2023-11-08T16:29:28.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.971+0000 7f0eb9f54640 1 -- 172.21.15.81:0/4004919935 shutdown_connections 2023-11-08T16:29:28.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.971+0000 7f0eb9f54640 1 --2- 172.21.15.81:0/4004919935 >> v2:172.21.15.93:3300/0 conn(0x7f0eb4161770 0x7f0eb40cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:28.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.971+0000 7f0eb9f54640 1 --2- 172.21.15.81:0/4004919935 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40cff50 0x7f0eb4161230 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:28.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.971+0000 7f0eb9f54640 1 -- 172.21.15.81:0/4004919935 >> 172.21.15.81:0/4004919935 conn(0x7f0eb406a620 msgr2=0x7f0eb406aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:28.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.971+0000 7f0eb9f54640 1 -- 172.21.15.81:0/4004919935 shutdown_connections 2023-11-08T16:29:28.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.972+0000 7f0eb9f54640 1 -- 172.21.15.81:0/4004919935 wait complete. 2023-11-08T16:29:28.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.973+0000 7f0eb9f54640 1 Processor -- start 2023-11-08T16:29:28.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.973+0000 7f0eb9f54640 1 -- start start 2023-11-08T16:29:28.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.974+0000 7f0eb9f54640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40cff50 0x7f0eb41589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:28.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.974+0000 7f0eb9f54640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb415cfa0 0x7f0eb415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:28.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.974+0000 7f0eb9f54640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0eb4161770 0x7f0eb415b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:28.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.974+0000 7f0eb9f54640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb414e0e0 con 0x7f0eb415cfa0 2023-11-08T16:29:28.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.974+0000 7f0eb9f54640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4158f80 con 0x7f0eb4161770 2023-11-08T16:29:28.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.974+0000 7f0eb9f54640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb4159260 con 0x7f0eb40cff50 2023-11-08T16:29:28.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.975+0000 7f0eb3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb415cfa0 0x7f0eb415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:28.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.975+0000 7f0eb3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb415cfa0 0x7f0eb415a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43930/0 (socket says 172.21.15.81:43930) 2023-11-08T16:29:28.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.975+0000 7f0eb3fff640 1 -- 172.21.15.81:0/1634882263 learned_addr learned my addr 172.21.15.81:0/1634882263 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:28.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.975+0000 7f0eb8f52640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40cff50 0x7f0eb41589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:28.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.975+0000 7f0eb3fff640 1 -- 172.21.15.81:0/1634882263 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40cff50 msgr2=0x7f0eb41589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:28.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.975+0000 7f0eb3fff640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40cff50 0x7f0eb41589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:28.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.975+0000 7f0eb3fff640 1 -- 172.21.15.81:0/1634882263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0eb4161770 msgr2=0x7f0eb415b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:28.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.975+0000 7f0eb3fff640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0eb4161770 0x7f0eb415b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:28.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.975+0000 7f0eb3fff640 1 -- 172.21.15.81:0/1634882263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea00094d0 con 0x7f0eb415cfa0 2023-11-08T16:29:28.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.975+0000 7f0eb3fff640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb415cfa0 0x7f0eb415a020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ea002b950 tx=0x7f0ea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:28.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.975+0000 7f0eb17fa640 1 -- 172.21.15.81:0/1634882263 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0031860 con 0x7f0eb415cfa0 2023-11-08T16:29:28.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.976+0000 7f0eb17fa640 1 -- 172.21.15.81:0/1634882263 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea00443f0 con 0x7f0eb415cfa0 2023-11-08T16:29:28.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.976+0000 7f0eb17fa640 1 -- 172.21.15.81:0/1634882263 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00447a0 con 0x7f0eb415cfa0 2023-11-08T16:29:28.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.976+0000 7f0eb9f54640 1 -- 172.21.15.81:0/1634882263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb4159540 con 0x7f0eb415cfa0 2023-11-08T16:29:28.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.976+0000 7f0eb9f54640 1 -- 172.21.15.81:0/1634882263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb4159b00 con 0x7f0eb415cfa0 2023-11-08T16:29:28.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.977+0000 7f0eb9f54640 1 -- 172.21.15.81:0/1634882263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb4159cf0 con 0x7f0eb415cfa0 2023-11-08T16:29:28.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.978+0000 7f0eb17fa640 1 -- 172.21.15.81:0/1634882263 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0ea004d450 con 0x7f0eb415cfa0 2023-11-08T16:29:28.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.979+0000 7f0eb17fa640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 0x7f0e9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:28.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.979+0000 7f0eb17fa640 1 -- 172.21.15.81:0/1634882263 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ea002d040 con 0x7f0eb415cfa0 2023-11-08T16:29:28.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.979+0000 7f0eb8f52640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 0x7f0e9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:28.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.980+0000 7f0eb8f52640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 0x7f0e9806d330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8003d90 tx=0x7f0ea8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:28.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:28.984+0000 7f0eb17fa640 1 -- 172.21.15.81:0/1634882263 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0ea00b2080 con 0x7f0eb415cfa0 2023-11-08T16:29:29.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.211+0000 7f0eb9f54640 1 -- 172.21.15.81:0/1634882263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f0eb4067800 con 0x7f0eb415cfa0 2023-11-08T16:29:29.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.212+0000 7f0eb17fa640 1 -- 172.21.15.81:0/1634882263 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f0ea0035070 con 0x7f0eb415cfa0 2023-11-08T16:29:29.214 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T16:29:29.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 -- 172.21.15.81:0/1634882263 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 msgr2=0x7f0e9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:29.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 0x7f0e9806d330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0ea8003d90 tx=0x7f0ea8009040 comp rx=0 tx=0).stop 2023-11-08T16:29:29.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 -- 172.21.15.81:0/1634882263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb415cfa0 msgr2=0x7f0eb415a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:29.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb415cfa0 0x7f0eb415a020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ea002b950 tx=0x7f0ea00046c0 comp rx=0 tx=0).stop 2023-11-08T16:29:29.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 -- 172.21.15.81:0/1634882263 shutdown_connections 2023-11-08T16:29:29.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 0x7f0e9806d330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:29.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0eb4161770 0x7f0eb415b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:29.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb415cfa0 0x7f0eb415a020 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:29.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 --2- 172.21.15.81:0/1634882263 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40cff50 0x7f0eb41589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:29.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 -- 172.21.15.81:0/1634882263 >> 172.21.15.81:0/1634882263 conn(0x7f0eb406a620 msgr2=0x7f0eb40d2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:29.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 -- 172.21.15.81:0/1634882263 shutdown_connections 2023-11-08T16:29:29.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:29.215+0000 7f0eb9f54640 1 -- 172.21.15.81:0/1634882263 wait complete. 2023-11-08T16:29:29.235 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:29:29.235 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:29:29.236 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi093.front.sepia.ceph.com 2023-11-08T16:29:29.236 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-08T16:29:29.236 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-08T16:29:29.236 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:29:29.237 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:29:29.237 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:29:29.237 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-08T16:29:29.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:29.238 DEBUG:teuthology.orchestra.run.smithi093:> ip addr 2023-11-08T16:29:29.299 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:29:29.300 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:29:29.300 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:29:29.300 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:29.300 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2023-11-08T16:29:29.300 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:29.301 INFO:teuthology.orchestra.run.smithi093.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:29.301 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1c brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:29.301 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f0 2023-11-08T16:29:29.301 INFO:teuthology.orchestra.run.smithi093.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:29.301 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1d brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:29.302 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f1 2023-11-08T16:29:29.302 INFO:teuthology.orchestra.run.smithi093.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:29.302 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:34 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:29.302 INFO:teuthology.orchestra.run.smithi093.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:29:29.302 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:35 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:29.303 INFO:teuthology.orchestra.run.smithi093.stdout: inet 172.21.15.93/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:29:29.303 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft 42382sec preferred_lft 42382sec 2023-11-08T16:29:29.303 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::ec4:7aff:fe8f:d635/64 scope link 2023-11-08T16:29:29.303 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:29.304 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-08T16:29:29.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:29.305 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-08T16:29:29.305 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-08T16:29:29.306 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add name ceph-brx type bridge 2023-11-08T16:29:29.306 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr flush dev ceph-brx 2023-11-08T16:29:29.306 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set ceph-brx up 2023-11-08T16:29:29.307 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-08T16:29:29.307 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-08T16:29:29.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:29.771 DEBUG:teuthology.orchestra.run.smithi093:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:29:29.853 INFO:teuthology.orchestra.run.smithi093.stdout:1 2023-11-08T16:29:29.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:29.856 DEBUG:teuthology.orchestra.run.smithi093:> route 2023-11-08T16:29:29.905 INFO:teuthology.orchestra.run.smithi093.stdout:Kernel IP routing table 2023-11-08T16:29:29.905 INFO:teuthology.orchestra.run.smithi093.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:29:29.905 INFO:teuthology.orchestra.run.smithi093.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:29:29.905 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:29:29.906 INFO:teuthology.orchestra.run.smithi093.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:29:29.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:29.906 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-08T16:29:29.907 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-08T16:29:29.907 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:29:29.907 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:29:29.908 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:29:29.908 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-08T16:29:30.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:30.724 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-11-08T16:29:30.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:30.781 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list-id 2023-11-08T16:29:30.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:30.835 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-08T16:29:30.835 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-08T16:29:30.835 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:30.835 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-08T16:29:30.836 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-08T16:29:30.980 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-08T16:29:30.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:30.981 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-08T16:29:30.981 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-08T16:29:30.981 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-08T16:29:30.982 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-08T16:29:30.982 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-08T16:29:30.982 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-08T16:29:30.983 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-08T16:29:30.983 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-08T16:29:31.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:31.271 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-08T16:29:31.271 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-08T16:29:31.271 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set brx.0 up 2023-11-08T16:29:31.272 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set dev brx.0 master ceph-brx 2023-11-08T16:29:31.272 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-08T16:29:31.438 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-08T16:29:31.439 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:29:31.439 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-08T16:29:31.488 INFO:teuthology.orchestra.run.smithi093.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-08T16:29:31.488 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:29:31.489 DEBUG:teuthology.orchestra.run.smithi093:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-08T16:29:31.584 DEBUG:teuthology.orchestra.run.smithi093:> sudo modprobe fuse 2023-11-08T16:29:31.673 DEBUG:teuthology.orchestra.run.smithi093:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:29:31.724 INFO:teuthology.orchestra.run.smithi093.stdout:/proc 2023-11-08T16:29:31.725 INFO:teuthology.orchestra.run.smithi093.stdout:/sys 2023-11-08T16:29:31.725 INFO:teuthology.orchestra.run.smithi093.stdout:/dev 2023-11-08T16:29:31.725 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/security 2023-11-08T16:29:31.725 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/shm 2023-11-08T16:29:31.726 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/pts 2023-11-08T16:29:31.726 INFO:teuthology.orchestra.run.smithi093.stdout:/run 2023-11-08T16:29:31.726 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/cgroup 2023-11-08T16:29:31.726 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/pstore 2023-11-08T16:29:31.726 INFO:teuthology.orchestra.run.smithi093.stdout:/ 2023-11-08T16:29:31.727 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/selinux 2023-11-08T16:29:31.727 INFO:teuthology.orchestra.run.smithi093.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:29:31.727 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/mqueue 2023-11-08T16:29:31.727 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/hugepages 2023-11-08T16:29:31.727 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/debug 2023-11-08T16:29:31.728 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/tracing 2023-11-08T16:29:31.728 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/fuse/connections 2023-11-08T16:29:31.728 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/config 2023-11-08T16:29:31.728 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:29:31.728 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:29:31.729 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:29:31.729 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:29:31.729 INFO:teuthology.orchestra.run.smithi093.stdout:/run/user/1000 2023-11-08T16:29:31.729 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph 2023-11-08T16:29:31.730 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-4 2023-11-08T16:29:31.730 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-5 2023-11-08T16:29:31.730 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-6 2023-11-08T16:29:31.730 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-7 2023-11-08T16:29:31.730 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns 2023-11-08T16:29:31.731 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:31.731 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:31.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:31.732 DEBUG:teuthology.orchestra.run.smithi093:> ls /sys/fs/fuse/connections 2023-11-08T16:29:31.781 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:29:31.781 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-11-08T16:29:31.829 DEBUG:teuthology.orchestra.run.smithi093:> sudo modprobe fuse 2023-11-08T16:29:31.911 DEBUG:teuthology.orchestra.run.smithi093:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:29:31.959 INFO:teuthology.orchestra.run.smithi093.stdout:/proc 2023-11-08T16:29:31.959 INFO:teuthology.orchestra.run.smithi093.stdout:/sys 2023-11-08T16:29:31.960 INFO:teuthology.orchestra.run.smithi093.stdout:/dev 2023-11-08T16:29:31.960 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/security 2023-11-08T16:29:31.960 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/shm 2023-11-08T16:29:31.960 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/pts 2023-11-08T16:29:31.960 INFO:teuthology.orchestra.run.smithi093.stdout:/run 2023-11-08T16:29:31.961 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/cgroup 2023-11-08T16:29:31.961 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/pstore 2023-11-08T16:29:31.961 INFO:teuthology.orchestra.run.smithi093.stdout:/ 2023-11-08T16:29:31.961 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/selinux 2023-11-08T16:29:31.961 INFO:teuthology.orchestra.run.smithi093.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:29:31.962 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/mqueue 2023-11-08T16:29:31.962 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/hugepages 2023-11-08T16:29:31.962 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/debug 2023-11-08T16:29:31.962 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/tracing 2023-11-08T16:29:31.962 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/fuse/connections 2023-11-08T16:29:31.963 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/config 2023-11-08T16:29:31.963 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:29:31.963 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:29:31.963 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:29:31.963 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:29:31.964 INFO:teuthology.orchestra.run.smithi093.stdout:/run/user/1000 2023-11-08T16:29:31.964 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph 2023-11-08T16:29:31.964 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-4 2023-11-08T16:29:31.964 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-5 2023-11-08T16:29:31.964 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-6 2023-11-08T16:29:31.965 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-7 2023-11-08T16:29:31.965 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns 2023-11-08T16:29:31.965 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:31.965 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:31.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:31.966 DEBUG:teuthology.orchestra.run.smithi093:> ls /sys/fs/fuse/connections 2023-11-08T16:29:32.060 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi093.stderr:2023-11-08T16:29:32.057+0000 7f9bbec41480 -1 init, newargv = 0x564ca1b69060 newargc=15 2023-11-08T16:29:32.060 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi093.stderr:ceph-fuse[93294]: starting ceph client 2023-11-08T16:29:32.073 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi093.stderr:ceph-fuse[93294]: starting fuse 2023-11-08T16:29:33.112 DEBUG:teuthology.orchestra.run.smithi093:> sudo modprobe fuse 2023-11-08T16:29:33.217 DEBUG:teuthology.orchestra.run.smithi093:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:29:33.275 INFO:teuthology.orchestra.run.smithi093.stdout:/proc 2023-11-08T16:29:33.276 INFO:teuthology.orchestra.run.smithi093.stdout:/sys 2023-11-08T16:29:33.276 INFO:teuthology.orchestra.run.smithi093.stdout:/dev 2023-11-08T16:29:33.276 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/security 2023-11-08T16:29:33.277 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/shm 2023-11-08T16:29:33.277 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/pts 2023-11-08T16:29:33.277 INFO:teuthology.orchestra.run.smithi093.stdout:/run 2023-11-08T16:29:33.277 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/cgroup 2023-11-08T16:29:33.278 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/pstore 2023-11-08T16:29:33.278 INFO:teuthology.orchestra.run.smithi093.stdout:/ 2023-11-08T16:29:33.278 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/selinux 2023-11-08T16:29:33.278 INFO:teuthology.orchestra.run.smithi093.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:29:33.279 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/mqueue 2023-11-08T16:29:33.279 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/hugepages 2023-11-08T16:29:33.279 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/debug 2023-11-08T16:29:33.279 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/tracing 2023-11-08T16:29:33.279 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/fuse/connections 2023-11-08T16:29:33.280 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/config 2023-11-08T16:29:33.280 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:29:33.280 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:29:33.280 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:29:33.280 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:29:33.281 INFO:teuthology.orchestra.run.smithi093.stdout:/run/user/1000 2023-11-08T16:29:33.281 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph 2023-11-08T16:29:33.281 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-4 2023-11-08T16:29:33.281 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-5 2023-11-08T16:29:33.282 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-6 2023-11-08T16:29:33.282 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-7 2023-11-08T16:29:33.282 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns 2023-11-08T16:29:33.283 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:33.283 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:33.283 INFO:teuthology.orchestra.run.smithi093.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-08T16:29:33.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:33.285 DEBUG:teuthology.orchestra.run.smithi093:> ls /sys/fs/fuse/connections 2023-11-08T16:29:33.339 INFO:teuthology.orchestra.run.smithi093.stdout:39 2023-11-08T16:29:33.339 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39] 2023-11-08T16:29:33.340 DEBUG:teuthology.orchestra.run.smithi093:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:29:33.340 DEBUG:teuthology.orchestra.run.smithi093:> import glob 2023-11-08T16:29:33.340 DEBUG:teuthology.orchestra.run.smithi093:> import re 2023-11-08T16:29:33.341 DEBUG:teuthology.orchestra.run.smithi093:> import os 2023-11-08T16:29:33.341 DEBUG:teuthology.orchestra.run.smithi093:> import subprocess 2023-11-08T16:29:33.342 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-08T16:29:33.342 DEBUG:teuthology.orchestra.run.smithi093:> def _find_admin_socket(client_name): 2023-11-08T16:29:33.342 DEBUG:teuthology.orchestra.run.smithi093:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-08T16:29:33.343 DEBUG:teuthology.orchestra.run.smithi093:> files = glob.glob(asok_path) 2023-11-08T16:29:33.343 DEBUG:teuthology.orchestra.run.smithi093:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-08T16:29:33.343 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-08T16:29:33.344 DEBUG:teuthology.orchestra.run.smithi093:> # Given a non-glob path, it better be there 2023-11-08T16:29:33.344 DEBUG:teuthology.orchestra.run.smithi093:> if "*" not in asok_path: 2023-11-08T16:29:33.344 DEBUG:teuthology.orchestra.run.smithi093:> assert(len(files) == 1) 2023-11-08T16:29:33.345 DEBUG:teuthology.orchestra.run.smithi093:> return files[0] 2023-11-08T16:29:33.345 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-08T16:29:33.346 DEBUG:teuthology.orchestra.run.smithi093:> for f in files: 2023-11-08T16:29:33.346 DEBUG:teuthology.orchestra.run.smithi093:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:29:33.346 DEBUG:teuthology.orchestra.run.smithi093:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:29:33.347 DEBUG:teuthology.orchestra.run.smithi093:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:29:33.347 DEBUG:teuthology.orchestra.run.smithi093:> contents = proc_f.read() 2023-11-08T16:29:33.347 DEBUG:teuthology.orchestra.run.smithi093:> if mountpoint in contents: 2023-11-08T16:29:33.347 DEBUG:teuthology.orchestra.run.smithi093:> return f 2023-11-08T16:29:33.348 DEBUG:teuthology.orchestra.run.smithi093:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:29:33.348 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-08T16:29:33.348 DEBUG:teuthology.orchestra.run.smithi093:> print(_find_admin_socket("client.1")) 2023-11-08T16:29:33.349 DEBUG:teuthology.orchestra.run.smithi093:> ' 2023-11-08T16:29:33.522 INFO:teuthology.orchestra.run.smithi093.stdout:/var/run/ceph/ceph-client.1.93294.asok 2023-11-08T16:29:33.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:29:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:29:33.540 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.93294.asok 2023-11-08T16:29:33.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:33.541 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.93294.asok status 2023-11-08T16:29:33.786 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-08T16:29:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "metadata": { 2023-11-08T16:29:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:29:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:29:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "entity_id": "1", 2023-11-08T16:29:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": "smithi093", 2023-11-08T16:29:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-08T16:29:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "pid": "93294", 2023-11-08T16:29:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "root": "/" 2023-11-08T16:29:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-08T16:29:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "dentry_count": 0, 2023-11-08T16:29:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "dentry_pinned_count": 0, 2023-11-08T16:29:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 4439, 2023-11-08T16:29:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "inst": { 2023-11-08T16:29:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "name": { 2023-11-08T16:29:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "client", 2023-11-08T16:29:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "num": 4439 2023-11-08T16:29:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-08T16:29:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": { 2023-11-08T16:29:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v1", 2023-11-08T16:29:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:29:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 2888025181 2023-11-08T16:29:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-08T16:29:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-08T16:29:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": { 2023-11-08T16:29:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v1", 2023-11-08T16:29:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:29:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 2888025181 2023-11-08T16:29:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-08T16:29:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "inst_str": "client.4439 192.168.0.1:0/2888025181", 2023-11-08T16:29:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "addr_str": "192.168.0.1:0/2888025181", 2023-11-08T16:29:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "inode_count": 1, 2023-11-08T16:29:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_epoch": 8, 2023-11-08T16:29:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_epoch": 21, 2023-11-08T16:29:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:29:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "blocklisted": false, 2023-11-08T16:29:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs" 2023-11-08T16:29:33.794 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-08T16:29:33.803 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:29:34.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.025+0000 7f21d42a2640 1 Processor -- start 2023-11-08T16:29:34.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.025+0000 7f21d42a2640 1 -- start start 2023-11-08T16:29:34.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.026+0000 7f21d42a2640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f21cc071060 0x7f21cc164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:34.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.026+0000 7f21d42a2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21cc0c9a70 con 0x7f21cc0adcd0 2023-11-08T16:29:34.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.026+0000 7f21d42a2640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21cc0cc3e0 con 0x7f21cc0cc5e0 2023-11-08T16:29:34.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.026+0000 7f21d42a2640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21cc06d070 con 0x7f21cc071060 2023-11-08T16:29:34.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.027+0000 7f21d203e640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f21cc0adcd0 0x7f21cc0ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:37988/0 (socket says 172.21.15.81:37988) 2023-11-08T16:29:34.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.027+0000 7f21d203e640 1 -- 172.21.15.81:0/1423760430 learned_addr learned my addr 172.21.15.81:0/1423760430 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:34.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.027+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858427357 0 0) 0x7f21cc0c9a70 con 0x7f21cc0adcd0 2023-11-08T16:29:34.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.027+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21b8005730 con 0x7f21cc0adcd0 2023-11-08T16:29:34.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.027+0000 7f21d283f640 1 --2- 172.21.15.81:0/1423760430 >> v2:172.21.15.93:3300/0 conn(0x7f21cc071060 0x7f21cc164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:34.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1961512839 0 0) 0x7f21b8005730 con 0x7f21cc0adcd0 2023-11-08T16:29:34.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 >> v2:172.21.15.93:3300/0 conn(0x7f21cc071060 msgr2=0x7f21cc164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:34.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 --2- 172.21.15.81:0/1423760430 >> v2:172.21.15.93:3300/0 conn(0x7f21cc071060 0x7f21cc164eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:34.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 >> v1:172.21.15.93:6789/0 conn(0x7f21cc0cc5e0 legacy=0x7f21cc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:34.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc06d350 con 0x7f21cc0adcd0 2023-11-08T16:29:34.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f21c8001670 con 0x7f21cc0adcd0 2023-11-08T16:29:34.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 >> v1:172.21.15.81:6789/0 conn(0x7f21cc0adcd0 legacy=0x7f21cc0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:34.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 --2- 172.21.15.81:0/1423760430 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f21b8005cb0 0x7f21b80060d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:34.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 --2- 172.21.15.81:0/1423760430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21b80069d0 0x7f21b8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:34.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 --2- 172.21.15.81:0/1423760430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21b8007590 0x7f21b800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:34.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc06d070 con 0x7f21b80069d0 2023-11-08T16:29:34.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21b800e400 con 0x7f21b8007590 2023-11-08T16:29:34.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21b800e5c0 con 0x7f21b8005cb0 2023-11-08T16:29:34.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d183d640 1 --2- 172.21.15.81:0/1423760430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21b80069d0 0x7f21b8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:34.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d283f640 1 --2- 172.21.15.81:0/1423760430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21b8007590 0x7f21b800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:34.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d183d640 1 -- 172.21.15.81:0/1423760430 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f21b8005cb0 msgr2=0x7f21b80060d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:34.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d183d640 1 --2- 172.21.15.81:0/1423760430 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f21b8005cb0 0x7f21b80060d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:34.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d183d640 1 -- 172.21.15.81:0/1423760430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21b8007590 msgr2=0x7f21b800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:34.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d183d640 1 --2- 172.21.15.81:0/1423760430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21b8007590 0x7f21b800db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:34.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d183d640 1 -- 172.21.15.81:0/1423760430 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b8005730 con 0x7f21b80069d0 2023-11-08T16:29:34.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.028+0000 7f21d283f640 1 -- 172.21.15.81:0/1423760430 reap_dead start 2023-11-08T16:29:34.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.029+0000 7f21d183d640 1 --2- 172.21.15.81:0/1423760430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21b80069d0 0x7f21b8006e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21bc001750 tx=0x7f21b8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7487edb93df263e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:34.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.030+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc0052d0 con 0x7f21b80069d0 2023-11-08T16:29:34.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.030+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21bc02e050 con 0x7f21b80069d0 2023-11-08T16:29:34.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.030+0000 7f21d103c640 1 -- 172.21.15.81:0/1423760430 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21bc003830 con 0x7f21b80069d0 2023-11-08T16:29:34.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.030+0000 7f21d42a2640 1 -- 172.21.15.81:0/1423760430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21b80069d0 msgr2=0x7f21b8006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:34.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.030+0000 7f21d42a2640 1 --2- 172.21.15.81:0/1423760430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21b80069d0 0x7f21b8006e50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f21bc001750 tx=0x7f21b8007a00 comp rx=0 tx=0).stop 2023-11-08T16:29:34.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.030+0000 7f21d42a2640 1 -- 172.21.15.81:0/1423760430 shutdown_connections 2023-11-08T16:29:34.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.030+0000 7f21d42a2640 1 --2- 172.21.15.81:0/1423760430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21b80069d0 0x7f21b8006e50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:34.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.030+0000 7f21d42a2640 1 -- 172.21.15.81:0/1423760430 >> 172.21.15.81:0/1423760430 conn(0x7f21cc0cb490 msgr2=0x7f21cc0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:34.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.030+0000 7f21d42a2640 1 -- 172.21.15.81:0/1423760430 shutdown_connections 2023-11-08T16:29:34.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.030+0000 7f21d42a2640 1 -- 172.21.15.81:0/1423760430 wait complete. 2023-11-08T16:29:34.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.032+0000 7f21d42a2640 1 Processor -- start 2023-11-08T16:29:34.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.032+0000 7f21d42a2640 1 -- start start 2023-11-08T16:29:34.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.033+0000 7f21d42a2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f21b80069d0 0x7f21cc14bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:34.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.033+0000 7f21d42a2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21cc0af670 0x7f21cc0afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:34.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.033+0000 7f21d42a2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21cc1e7f00 0x7f21cc1e8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:34.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.033+0000 7f21d42a2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc0b0030 con 0x7f21cc1e7f00 2023-11-08T16:29:34.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.033+0000 7f21d42a2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc1e97f0 con 0x7f21cc0af670 2023-11-08T16:29:34.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.033+0000 7f21d42a2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21cc1e8c70 con 0x7f21b80069d0 2023-11-08T16:29:34.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d283f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21cc1e7f00 0x7f21cc1e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:34.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d283f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21cc1e7f00 0x7f21cc1e8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45228/0 (socket says 172.21.15.81:45228) 2023-11-08T16:29:34.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d283f640 1 -- 172.21.15.81:0/3518096099 learned_addr learned my addr 172.21.15.81:0/3518096099 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:34.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d203e640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f21b80069d0 0x7f21cc14bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:34.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d183d640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21cc0af670 0x7f21cc0afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:34.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d283f640 1 -- 172.21.15.81:0/3518096099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f21b80069d0 msgr2=0x7f21cc14bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:34.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d283f640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f21b80069d0 0x7f21cc14bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:34.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d283f640 1 -- 172.21.15.81:0/3518096099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21cc0af670 msgr2=0x7f21cc0afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:34.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d283f640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21cc0af670 0x7f21cc0afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:34.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d283f640 1 -- 172.21.15.81:0/3518096099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21cc1e8f50 con 0x7f21cc1e7f00 2023-11-08T16:29:34.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d183d640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21cc0af670 0x7f21cc0afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:34.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.034+0000 7f21d283f640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21cc1e7f00 0x7f21cc1e8360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f21cc06be10 tx=0x7f21c4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:34.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.035+0000 7f21c2ffd640 1 -- 172.21.15.81:0/3518096099 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4007d50 con 0x7f21cc1e7f00 2023-11-08T16:29:34.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.035+0000 7f21c2ffd640 1 -- 172.21.15.81:0/3518096099 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c4014400 con 0x7f21cc1e7f00 2023-11-08T16:29:34.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.035+0000 7f21c2ffd640 1 -- 172.21.15.81:0/3518096099 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40147b0 con 0x7f21cc1e7f00 2023-11-08T16:29:34.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.035+0000 7f21d42a2640 1 -- 172.21.15.81:0/3518096099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc1e9230 con 0x7f21cc1e7f00 2023-11-08T16:29:34.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.035+0000 7f21d42a2640 1 -- 172.21.15.81:0/3518096099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21cc14c290 con 0x7f21cc1e7f00 2023-11-08T16:29:34.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.036+0000 7f21d42a2640 1 -- 172.21.15.81:0/3518096099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21cc072170 con 0x7f21cc1e7f00 2023-11-08T16:29:34.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.037+0000 7f21c2ffd640 1 -- 172.21.15.81:0/3518096099 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f21c401d3b0 con 0x7f21cc1e7f00 2023-11-08T16:29:34.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.038+0000 7f21c2ffd640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f21b006ae70 0x7f21b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:34.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.038+0000 7f21c2ffd640 1 -- 172.21.15.81:0/3518096099 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21c4089600 con 0x7f21cc1e7f00 2023-11-08T16:29:34.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.038+0000 7f21d203e640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f21b006ae70 0x7f21b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:34.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.039+0000 7f21d203e640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f21b006ae70 0x7f21b006d330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f21cc063cf0 tx=0x7f21c8001600 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:34.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.043+0000 7f21c2ffd640 1 -- 172.21.15.81:0/3518096099 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21c4054610 con 0x7f21cc1e7f00 2023-11-08T16:29:34.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.269+0000 7f21d42a2640 1 -- 172.21.15.81:0/3518096099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f21cc04ece0 con 0x7f21cc1e7f00 2023-11-08T16:29:34.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.270+0000 7f21c2ffd640 1 -- 172.21.15.81:0/3518096099 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f21c4057c30 con 0x7f21cc1e7f00 2023-11-08T16:29:34.272 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T16:29:34.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 -- 172.21.15.81:0/3518096099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f21b006ae70 msgr2=0x7f21b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:34.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f21b006ae70 0x7f21b006d330 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f21cc063cf0 tx=0x7f21c8001600 comp rx=0 tx=0).stop 2023-11-08T16:29:34.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 -- 172.21.15.81:0/3518096099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21cc1e7f00 msgr2=0x7f21cc1e8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:34.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21cc1e7f00 0x7f21cc1e8360 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f21cc06be10 tx=0x7f21c4009da0 comp rx=0 tx=0).stop 2023-11-08T16:29:34.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 -- 172.21.15.81:0/3518096099 shutdown_connections 2023-11-08T16:29:34.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f21b006ae70 0x7f21b006d330 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:34.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21cc1e7f00 0x7f21cc1e8360 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:34.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21cc0af670 0x7f21cc0afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:34.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 --2- 172.21.15.81:0/3518096099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f21b80069d0 0x7f21cc14bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:34.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 -- 172.21.15.81:0/3518096099 >> 172.21.15.81:0/3518096099 conn(0x7f21cc0cb490 msgr2=0x7f21cc0c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:34.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 -- 172.21.15.81:0/3518096099 shutdown_connections 2023-11-08T16:29:34.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:34.274+0000 7f21d42a2640 1 -- 172.21.15.81:0/3518096099 wait complete. 2023-11-08T16:29:34.296 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:29:34.296 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:29:34.296 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi093.front.sepia.ceph.com 2023-11-08T16:29:34.297 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-08T16:29:34.297 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-08T16:29:34.297 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:29:34.297 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:29:34.297 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:29:34.298 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-08T16:29:34.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:34.299 DEBUG:teuthology.orchestra.run.smithi093:> ip addr 2023-11-08T16:29:34.359 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:29:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:29:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:29:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2023-11-08T16:29:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:34.361 INFO:teuthology.orchestra.run.smithi093.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1c brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f0 2023-11-08T16:29:34.361 INFO:teuthology.orchestra.run.smithi093.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1d brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f1 2023-11-08T16:29:34.362 INFO:teuthology.orchestra.run.smithi093.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:29:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:34 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:34.362 INFO:teuthology.orchestra.run.smithi093.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:29:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:35 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: inet 172.21.15.93/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:29:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft 42377sec preferred_lft 42377sec 2023-11-08T16:29:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::ec4:7aff:fe8f:d635/64 scope link 2023-11-08T16:29:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:34.363 INFO:teuthology.orchestra.run.smithi093.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:29:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether a6:e0:ed:b4:8e:ed brd ff:ff:ff:ff:ff:ff 2023-11-08T16:29:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:29:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::cc2a:7eff:fecc:6b34/64 scope link 2023-11-08T16:29:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:34.365 INFO:teuthology.orchestra.run.smithi093.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:29:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether a6:e0:ed:b4:8e:ed brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::a4e0:edff:feb4:8eed/64 scope link 2023-11-08T16:29:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:34.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:34.367 DEBUG:teuthology.orchestra.run.smithi093:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:29:34.460 INFO:teuthology.orchestra.run.smithi093.stdout:1 2023-11-08T16:29:34.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:34.465 DEBUG:teuthology.orchestra.run.smithi093:> route 2023-11-08T16:29:34.518 INFO:teuthology.orchestra.run.smithi093.stdout:Kernel IP routing table 2023-11-08T16:29:34.518 INFO:teuthology.orchestra.run.smithi093.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:29:34.519 INFO:teuthology.orchestra.run.smithi093.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:29:34.519 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:29:34.519 INFO:teuthology.orchestra.run.smithi093.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:29:34.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:34.521 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-08T16:29:34.521 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-08T16:29:34.521 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:29:34.522 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:29:34.522 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:29:34.522 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-08T16:29:34.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:34.704 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-11-08T16:29:34.757 INFO:teuthology.orchestra.run.smithi093.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-08T16:29:34.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:34.758 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list-id 2023-11-08T16:29:34.806 INFO:teuthology.orchestra.run.smithi093.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-08T16:29:34.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:34.807 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-08T16:29:34.807 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-08T16:29:34.807 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T16:29:34.808 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-08T16:29:34.808 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-08T16:29:34.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:34.939 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-08T16:29:35.029 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:29:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:29:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:29:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2023-11-08T16:29:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:35.030 INFO:teuthology.orchestra.run.smithi093.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:29:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 22:af:91:06:02:e8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T16:29:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T16:29:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::20af:91ff:fe06:2e8/64 scope link 2023-11-08T16:29:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:35.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:35.033 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-08T16:29:35.127 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:29:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:29:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:29:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2023-11-08T16:29:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:35.129 INFO:teuthology.orchestra.run.smithi093.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:29:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 22:af:91:06:02:e8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-08T16:29:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-08T16:29:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::20af:91ff:fe06:2e8/64 scope link 2023-11-08T16:29:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:29:35.131 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-08T16:29:35.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:35.132 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-08T16:29:35.132 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-08T16:29:35.133 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-08T16:29:35.133 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-08T16:29:35.133 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-08T16:29:35.134 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-08T16:29:35.134 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-08T16:29:35.134 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-08T16:29:35.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:35.397 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-11-08T16:29:35.398 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-11-08T16:29:35.398 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set brx.1 up 2023-11-08T16:29:35.398 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set dev brx.1 master ceph-brx 2023-11-08T16:29:35.398 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-11-08T16:29:35.559 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-08T16:29:35.559 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:29:35.560 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-08T16:29:35.605 INFO:teuthology.orchestra.run.smithi093.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-08T16:29:35.606 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:29:35.606 DEBUG:teuthology.orchestra.run.smithi093:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-08T16:29:35.703 DEBUG:teuthology.orchestra.run.smithi093:> sudo modprobe fuse 2023-11-08T16:29:35.805 DEBUG:teuthology.orchestra.run.smithi093:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:29:35.857 INFO:teuthology.orchestra.run.smithi093.stdout:/proc 2023-11-08T16:29:35.858 INFO:teuthology.orchestra.run.smithi093.stdout:/sys 2023-11-08T16:29:35.858 INFO:teuthology.orchestra.run.smithi093.stdout:/dev 2023-11-08T16:29:35.858 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/security 2023-11-08T16:29:35.858 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/shm 2023-11-08T16:29:35.859 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/pts 2023-11-08T16:29:35.859 INFO:teuthology.orchestra.run.smithi093.stdout:/run 2023-11-08T16:29:35.859 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/cgroup 2023-11-08T16:29:35.859 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/pstore 2023-11-08T16:29:35.860 INFO:teuthology.orchestra.run.smithi093.stdout:/ 2023-11-08T16:29:35.860 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/selinux 2023-11-08T16:29:35.860 INFO:teuthology.orchestra.run.smithi093.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:29:35.860 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/mqueue 2023-11-08T16:29:35.860 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/hugepages 2023-11-08T16:29:35.861 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/debug 2023-11-08T16:29:35.861 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/tracing 2023-11-08T16:29:35.861 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/fuse/connections 2023-11-08T16:29:35.861 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/config 2023-11-08T16:29:35.861 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:29:35.862 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:29:35.862 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:29:35.862 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:29:35.862 INFO:teuthology.orchestra.run.smithi093.stdout:/run/user/1000 2023-11-08T16:29:35.862 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph 2023-11-08T16:29:35.863 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-4 2023-11-08T16:29:35.863 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-5 2023-11-08T16:29:35.863 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-6 2023-11-08T16:29:35.863 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-7 2023-11-08T16:29:35.864 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns 2023-11-08T16:29:35.864 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:35.864 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:35.864 INFO:teuthology.orchestra.run.smithi093.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-08T16:29:35.864 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T16:29:35.865 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T16:29:35.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:35.866 DEBUG:teuthology.orchestra.run.smithi093:> ls /sys/fs/fuse/connections 2023-11-08T16:29:35.915 INFO:teuthology.orchestra.run.smithi093.stdout:39 2023-11-08T16:29:35.916 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [39] 2023-11-08T16:29:35.916 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-11-08T16:29:35.922 DEBUG:teuthology.orchestra.run.smithi093:> sudo modprobe fuse 2023-11-08T16:29:35.997 DEBUG:teuthology.orchestra.run.smithi093:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:29:36.038 INFO:teuthology.orchestra.run.smithi093.stdout:/proc 2023-11-08T16:29:36.038 INFO:teuthology.orchestra.run.smithi093.stdout:/sys 2023-11-08T16:29:36.038 INFO:teuthology.orchestra.run.smithi093.stdout:/dev 2023-11-08T16:29:36.039 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/security 2023-11-08T16:29:36.039 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/shm 2023-11-08T16:29:36.039 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/pts 2023-11-08T16:29:36.039 INFO:teuthology.orchestra.run.smithi093.stdout:/run 2023-11-08T16:29:36.039 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/cgroup 2023-11-08T16:29:36.040 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/pstore 2023-11-08T16:29:36.040 INFO:teuthology.orchestra.run.smithi093.stdout:/ 2023-11-08T16:29:36.040 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/selinux 2023-11-08T16:29:36.040 INFO:teuthology.orchestra.run.smithi093.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:29:36.040 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/mqueue 2023-11-08T16:29:36.041 INFO:teuthology.orchestra.run.smithi093.stdout:/dev/hugepages 2023-11-08T16:29:36.041 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/debug 2023-11-08T16:29:36.041 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/tracing 2023-11-08T16:29:36.041 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/fs/fuse/connections 2023-11-08T16:29:36.041 INFO:teuthology.orchestra.run.smithi093.stdout:/sys/kernel/config 2023-11-08T16:29:36.042 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:29:36.042 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:29:36.042 INFO:teuthology.orchestra.run.smithi093.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:29:36.042 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:29:36.043 INFO:teuthology.orchestra.run.smithi093.stdout:/run/user/1000 2023-11-08T16:29:36.043 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph 2023-11-08T16:29:36.043 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-4 2023-11-08T16:29:36.044 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-5 2023-11-08T16:29:36.044 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-6 2023-11-08T16:29:36.044 INFO:teuthology.orchestra.run.smithi093.stdout:/var/lib/ceph/osd/ceph-7 2023-11-08T16:29:36.044 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns 2023-11-08T16:29:36.044 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:36.045 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:29:36.045 INFO:teuthology.orchestra.run.smithi093.stdout:/home/ubuntu/cephtest/mnt.1 2023-11-08T16:29:36.045 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T16:29:36.045 INFO:teuthology.orchestra.run.smithi093.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T16:29:36.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:36.047 DEBUG:teuthology.orchestra.run.smithi093:> ls /sys/fs/fuse/connections 2023-11-08T16:29:36.107 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi093.stderr:2023-11-08T16:29:36.105+0000 7f492a439480 -1 init, newargv = 0x555d5f368270 newargc=15 2023-11-08T16:29:36.108 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi093.stderr:ceph-fuse[93798]: starting ceph client 2023-11-08T16:29:36.116 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi093.stderr:ceph-fuse[93798]: starting fuse 2023-11-08T16:29:36.134 INFO:teuthology.orchestra.run.smithi093.stdout:39 2023-11-08T16:29:36.134 INFO:teuthology.orchestra.run.smithi093.stdout:42 2023-11-08T16:29:36.134 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [39, 42] 2023-11-08T16:29:36.135 DEBUG:teuthology.orchestra.run.smithi093:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:29:36.135 DEBUG:teuthology.orchestra.run.smithi093:> import glob 2023-11-08T16:29:36.136 DEBUG:teuthology.orchestra.run.smithi093:> import re 2023-11-08T16:29:36.136 DEBUG:teuthology.orchestra.run.smithi093:> import os 2023-11-08T16:29:36.136 DEBUG:teuthology.orchestra.run.smithi093:> import subprocess 2023-11-08T16:29:36.137 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-08T16:29:36.137 DEBUG:teuthology.orchestra.run.smithi093:> def _find_admin_socket(client_name): 2023-11-08T16:29:36.137 DEBUG:teuthology.orchestra.run.smithi093:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-08T16:29:36.137 DEBUG:teuthology.orchestra.run.smithi093:> files = glob.glob(asok_path) 2023-11-08T16:29:36.138 DEBUG:teuthology.orchestra.run.smithi093:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-08T16:29:36.138 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-08T16:29:36.138 DEBUG:teuthology.orchestra.run.smithi093:> # Given a non-glob path, it better be there 2023-11-08T16:29:36.139 DEBUG:teuthology.orchestra.run.smithi093:> if "*" not in asok_path: 2023-11-08T16:29:36.139 DEBUG:teuthology.orchestra.run.smithi093:> assert(len(files) == 1) 2023-11-08T16:29:36.139 DEBUG:teuthology.orchestra.run.smithi093:> return files[0] 2023-11-08T16:29:36.140 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-08T16:29:36.140 DEBUG:teuthology.orchestra.run.smithi093:> for f in files: 2023-11-08T16:29:36.140 DEBUG:teuthology.orchestra.run.smithi093:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:29:36.140 DEBUG:teuthology.orchestra.run.smithi093:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:29:36.141 DEBUG:teuthology.orchestra.run.smithi093:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:29:36.141 DEBUG:teuthology.orchestra.run.smithi093:> contents = proc_f.read() 2023-11-08T16:29:36.141 DEBUG:teuthology.orchestra.run.smithi093:> if mountpoint in contents: 2023-11-08T16:29:36.142 DEBUG:teuthology.orchestra.run.smithi093:> return f 2023-11-08T16:29:36.142 DEBUG:teuthology.orchestra.run.smithi093:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:29:36.142 DEBUG:teuthology.orchestra.run.smithi093:> 2023-11-08T16:29:36.143 DEBUG:teuthology.orchestra.run.smithi093:> print(_find_admin_socket("client.3")) 2023-11-08T16:29:36.143 DEBUG:teuthology.orchestra.run.smithi093:> ' 2023-11-08T16:29:36.340 INFO:teuthology.orchestra.run.smithi093.stdout:/var/run/ceph/ceph-client.3.93798.asok 2023-11-08T16:29:36.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-11-08T16:29:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:29:36.357 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.93798.asok 2023-11-08T16:29:36.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:36.358 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.93798.asok status 2023-11-08T16:29:36.597 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-11-08T16:29:36.597 INFO:teuthology.orchestra.run.smithi093.stdout: "metadata": { 2023-11-08T16:29:36.598 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:29:36.598 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:29:36.598 INFO:teuthology.orchestra.run.smithi093.stdout: "entity_id": "3", 2023-11-08T16:29:36.598 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": "smithi093", 2023-11-08T16:29:36.598 INFO:teuthology.orchestra.run.smithi093.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-08T16:29:36.599 INFO:teuthology.orchestra.run.smithi093.stdout: "pid": "93798", 2023-11-08T16:29:36.599 INFO:teuthology.orchestra.run.smithi093.stdout: "root": "/" 2023-11-08T16:29:36.599 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-08T16:29:36.599 INFO:teuthology.orchestra.run.smithi093.stdout: "dentry_count": 0, 2023-11-08T16:29:36.599 INFO:teuthology.orchestra.run.smithi093.stdout: "dentry_pinned_count": 0, 2023-11-08T16:29:36.600 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 4546, 2023-11-08T16:29:36.600 INFO:teuthology.orchestra.run.smithi093.stdout: "inst": { 2023-11-08T16:29:36.600 INFO:teuthology.orchestra.run.smithi093.stdout: "name": { 2023-11-08T16:29:36.600 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "client", 2023-11-08T16:29:36.600 INFO:teuthology.orchestra.run.smithi093.stdout: "num": 4546 2023-11-08T16:29:36.601 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-08T16:29:36.601 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": { 2023-11-08T16:29:36.601 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v1", 2023-11-08T16:29:36.601 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "192.168.0.2:0", 2023-11-08T16:29:36.601 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 1514322501 2023-11-08T16:29:36.602 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-11-08T16:29:36.602 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-08T16:29:36.602 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": { 2023-11-08T16:29:36.602 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v1", 2023-11-08T16:29:36.602 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "192.168.0.2:0", 2023-11-08T16:29:36.603 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 1514322501 2023-11-08T16:29:36.603 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-11-08T16:29:36.603 INFO:teuthology.orchestra.run.smithi093.stdout: "inst_str": "client.4546 192.168.0.2:0/1514322501", 2023-11-08T16:29:36.603 INFO:teuthology.orchestra.run.smithi093.stdout: "addr_str": "192.168.0.2:0/1514322501", 2023-11-08T16:29:36.603 INFO:teuthology.orchestra.run.smithi093.stdout: "inode_count": 1, 2023-11-08T16:29:36.604 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_epoch": 8, 2023-11-08T16:29:36.604 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_epoch": 21, 2023-11-08T16:29:36.604 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:29:36.604 INFO:teuthology.orchestra.run.smithi093.stdout: "blocklisted": false, 2023-11-08T16:29:36.604 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs" 2023-11-08T16:29:36.605 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-11-08T16:29:36.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:36.614 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:29:36.614 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:29:36.676 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:29:36.676 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:29:36.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:36.677 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:29:36.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:36.779 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:29:36.779 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-08T16:29:36.837 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:29:36.838 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-08T16:29:36.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:36.838 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-08T16:29:36.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:36.942 DEBUG:teuthology.orchestra.run.smithi093:> stat --file-system '--printf=%T 2023-11-08T16:29:36.942 DEBUG:teuthology.orchestra.run.smithi093:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-08T16:29:37.002 INFO:teuthology.orchestra.run.smithi093.stdout:fuseblk 2023-11-08T16:29:37.003 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-08T16:29:37.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:37.004 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-08T16:29:37.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:37.101 DEBUG:teuthology.orchestra.run.smithi093:> stat --file-system '--printf=%T 2023-11-08T16:29:37.101 DEBUG:teuthology.orchestra.run.smithi093:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-08T16:29:37.159 INFO:teuthology.orchestra.run.smithi093.stdout:fuseblk 2023-11-08T16:29:37.159 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-08T16:29:37.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:37.160 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-08T16:29:37.256 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-08T16:29:37.272 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:37.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.479+0000 7f57a646f640 1 Processor -- start 2023-11-08T16:29:37.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.479+0000 7f57a646f640 1 -- start start 2023-11-08T16:29:37.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.481+0000 7f57a646f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57a00ca480 0x7f57a015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.481+0000 7f57a646f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f57a015ea60 0x7f57a015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.481+0000 7f57a646f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57a006b630 con 0x7f57a00ade50 2023-11-08T16:29:37.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.481+0000 7f57a646f640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57a00b1860 con 0x7f57a015ea60 2023-11-08T16:29:37.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.481+0000 7f57a646f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00ca8e0 con 0x7f57a00ca480 2023-11-08T16:29:37.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.481+0000 7f57a4a0c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f57a015ea60 0x7f57a015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:37.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.481+0000 7f57a4a0c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f57a015ea60 0x7f57a015eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45236/0 (socket says 172.21.15.81:45236) 2023-11-08T16:29:37.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.481+0000 7f57a4a0c640 1 -- 172.21.15.81:0/3729021391 learned_addr learned my addr 172.21.15.81:0/3729021391 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:37.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.481+0000 7f5797fff640 1 --2- 172.21.15.81:0/3729021391 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57a00ca480 0x7f57a015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:37.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.482+0000 7f57a4a0c640 1 -- 172.21.15.81:0/3729021391 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57a00ca480 msgr2=0x7f57a015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:37.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.482+0000 7f57a4a0c640 1 --2- 172.21.15.81:0/3729021391 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57a00ca480 0x7f57a015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.482+0000 7f57a4a0c640 1 -- 172.21.15.81:0/3729021391 >> v1:172.21.15.93:6789/0 conn(0x7f57a00ade50 legacy=0x7f57a015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:37.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.482+0000 7f57a4a0c640 1 -- 172.21.15.81:0/3729021391 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a00cabb0 con 0x7f57a015ea60 2023-11-08T16:29:37.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.482+0000 7f5797fff640 1 --2- 172.21.15.81:0/3729021391 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57a00ca480 0x7f57a015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:37.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.483+0000 7f57a4a0c640 1 --2- 172.21.15.81:0/3729021391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57a015ea60 0x7f57a015eef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5798009e50 tx=0x7f579802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610dd860a3bb9931 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:37.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.483+0000 7f579f7fe640 1 -- 172.21.15.81:0/3729021391 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579802ca80 con 0x7f57a015ea60 2023-11-08T16:29:37.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.483+0000 7f579f7fe640 1 -- 172.21.15.81:0/3729021391 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579802cbe0 con 0x7f57a015ea60 2023-11-08T16:29:37.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.483+0000 7f579f7fe640 1 -- 172.21.15.81:0/3729021391 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980343f0 con 0x7f57a015ea60 2023-11-08T16:29:37.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.484+0000 7f57a646f640 1 -- 172.21.15.81:0/3729021391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57a015ea60 msgr2=0x7f57a015eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:37.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.484+0000 7f57a646f640 1 --2- 172.21.15.81:0/3729021391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57a015ea60 0x7f57a015eef0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5798009e50 tx=0x7f579802bee0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.484+0000 7f57a646f640 1 -- 172.21.15.81:0/3729021391 shutdown_connections 2023-11-08T16:29:37.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.484+0000 7f57a646f640 1 --2- 172.21.15.81:0/3729021391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57a015ea60 0x7f57a015eef0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.484+0000 7f57a646f640 1 --2- 172.21.15.81:0/3729021391 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57a00ca480 0x7f57a015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.484+0000 7f57a646f640 1 -- 172.21.15.81:0/3729021391 >> 172.21.15.81:0/3729021391 conn(0x7f57a00cb490 msgr2=0x7f57a00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:37.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.484+0000 7f57a646f640 1 -- 172.21.15.81:0/3729021391 shutdown_connections 2023-11-08T16:29:37.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.484+0000 7f57a646f640 1 -- 172.21.15.81:0/3729021391 wait complete. 2023-11-08T16:29:37.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.485+0000 7f57a646f640 1 Processor -- start 2023-11-08T16:29:37.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.486+0000 7f57a646f640 1 -- start start 2023-11-08T16:29:37.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f57a646f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57a00ade50 0x7f57a01f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f57a646f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57a00ca480 0x7f57a00cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f57a646f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57a015ea60 0x7f57a00aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f57a646f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00cd080 con 0x7f57a00ca480 2023-11-08T16:29:37.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f57a646f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00cd360 con 0x7f57a015ea60 2023-11-08T16:29:37.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f57a646f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57a00cd640 con 0x7f57a00ade50 2023-11-08T16:29:37.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f5797fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57a00ca480 0x7f57a00cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:37.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f5797fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57a00ca480 0x7f57a00cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45252/0 (socket says 172.21.15.81:45252) 2023-11-08T16:29:37.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f5797fff640 1 -- 172.21.15.81:0/1280554584 learned_addr learned my addr 172.21.15.81:0/1280554584 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:37.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f5797fff640 1 -- 172.21.15.81:0/1280554584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57a00ade50 msgr2=0x7f57a01f56a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:37.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f5797fff640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57a00ade50 0x7f57a01f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f5797fff640 1 -- 172.21.15.81:0/1280554584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57a015ea60 msgr2=0x7f57a00aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:37.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f5797fff640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57a015ea60 0x7f57a00aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f5797fff640 1 -- 172.21.15.81:0/1280554584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5798009b30 con 0x7f57a00ca480 2023-11-08T16:29:37.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.487+0000 7f5797fff640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57a00ca480 0x7f57a00cc5e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f578800c7f0 tx=0x7f578800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:37.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.488+0000 7f579d7fa640 1 -- 172.21.15.81:0/1280554584 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5788007a30 con 0x7f57a00ca480 2023-11-08T16:29:37.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.488+0000 7f579d7fa640 1 -- 172.21.15.81:0/1280554584 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5788007b90 con 0x7f57a00ca480 2023-11-08T16:29:37.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.488+0000 7f57a646f640 1 -- 172.21.15.81:0/1280554584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57a00cd920 con 0x7f57a00ca480 2023-11-08T16:29:37.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.488+0000 7f579d7fa640 1 -- 172.21.15.81:0/1280554584 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578800ce40 con 0x7f57a00ca480 2023-11-08T16:29:37.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.488+0000 7f57a646f640 1 -- 172.21.15.81:0/1280554584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57a00ad1d0 con 0x7f57a00ca480 2023-11-08T16:29:37.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.489+0000 7f57a646f640 1 -- 172.21.15.81:0/1280554584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57a00cde40 con 0x7f57a00ca480 2023-11-08T16:29:37.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.490+0000 7f579d7fa640 1 -- 172.21.15.81:0/1280554584 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57880113f0 con 0x7f57a00ca480 2023-11-08T16:29:37.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.491+0000 7f579d7fa640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f578006ae70 0x7f578006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.491+0000 7f579d7fa640 1 -- 172.21.15.81:0/1280554584 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57880897c0 con 0x7f57a00ca480 2023-11-08T16:29:37.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.491+0000 7f579ffff640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f578006ae70 0x7f578006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:37.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.492+0000 7f579ffff640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f578006ae70 0x7f578006d330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5790003cc0 tx=0x7f57900079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:37.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.495+0000 7f579d7fa640 1 -- 172.21.15.81:0/1280554584 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57a00cde40 con 0x7f57a00ca480 2023-11-08T16:29:37.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.686+0000 7f57a646f640 1 -- 172.21.15.81:0/1280554584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57a00cde40 con 0x7f57a00ca480 2023-11-08T16:29:37.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.688+0000 7f579d7fa640 1 -- 172.21.15.81:0/1280554584 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f57a00cde40 con 0x7f57a00ca480 2023-11-08T16:29:37.690 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:37.690 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:04.727022+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:37.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.690+0000 7f57a646f640 1 -- 172.21.15.81:0/1280554584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f578006ae70 msgr2=0x7f578006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:37.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.690+0000 7f57a646f640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f578006ae70 0x7f578006d330 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5790003cc0 tx=0x7f57900079b0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.690+0000 7f57a646f640 1 -- 172.21.15.81:0/1280554584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57a00ca480 msgr2=0x7f57a00cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:37.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.690+0000 7f57a646f640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57a00ca480 0x7f57a00cc5e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f578800c7f0 tx=0x7f578800cbf0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.690+0000 7f57a646f640 1 -- 172.21.15.81:0/1280554584 shutdown_connections 2023-11-08T16:29:37.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.690+0000 7f57a646f640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f578006ae70 0x7f578006d330 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.690+0000 7f57a646f640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57a015ea60 0x7f57a00aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.690+0000 7f57a646f640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57a00ca480 0x7f57a00cc5e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.690+0000 7f57a646f640 1 --2- 172.21.15.81:0/1280554584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57a00ade50 0x7f57a01f56a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.690+0000 7f57a646f640 1 -- 172.21.15.81:0/1280554584 >> 172.21.15.81:0/1280554584 conn(0x7f57a00cb490 msgr2=0x7f57a015db00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:37.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.691+0000 7f57a646f640 1 -- 172.21.15.81:0/1280554584 shutdown_connections 2023-11-08T16:29:37.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.691+0000 7f57a646f640 1 -- 172.21.15.81:0/1280554584 wait complete. 2023-11-08T16:29:37.711 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:37.712 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:37.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.931+0000 7fcb8e16c640 1 Processor -- start 2023-11-08T16:29:37.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.931+0000 7fcb8e16c640 1 -- start start 2023-11-08T16:29:37.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.932+0000 7fcb8e16c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb8815d4c0 0x7fcb880cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.932+0000 7fcb8e16c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcb880d39a0 0x7fcb880cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.932+0000 7fcb8e16c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcb880cfaf0 0x7fcb880d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.932+0000 7fcb8e16c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb880cff80 con 0x7fcb880d39a0 2023-11-08T16:29:37.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.932+0000 7fcb8e16c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb880cef70 con 0x7fcb880cfaf0 2023-11-08T16:29:37.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.932+0000 7fcb8e16c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880cec90 con 0x7fcb8815d4c0 2023-11-08T16:29:37.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.933+0000 7fcb86ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcb880d39a0 0x7fcb880cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:37.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.933+0000 7fcb86ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcb880d39a0 0x7fcb880cf5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45258/0 (socket says 172.21.15.81:45258) 2023-11-08T16:29:37.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.933+0000 7fcb86ffd640 1 -- 172.21.15.81:0/64150250 learned_addr learned my addr 172.21.15.81:0/64150250 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:37.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.933+0000 7fcb87fff640 1 --2- 172.21.15.81:0/64150250 >> v2:172.21.15.93:3300/0 conn(0x7fcb880cfaf0 0x7fcb880d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:37.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.933+0000 7fcb877fe640 1 --2- 172.21.15.81:0/64150250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb8815d4c0 0x7fcb880cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:37.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.933+0000 7fcb86ffd640 1 -- 172.21.15.81:0/64150250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb8815d4c0 msgr2=0x7fcb880cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:37.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.933+0000 7fcb86ffd640 1 --2- 172.21.15.81:0/64150250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb8815d4c0 0x7fcb880cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.933+0000 7fcb86ffd640 1 -- 172.21.15.81:0/64150250 >> v2:172.21.15.93:3300/0 conn(0x7fcb880cfaf0 msgr2=0x7fcb880d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:37.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.933+0000 7fcb86ffd640 1 --2- 172.21.15.81:0/64150250 >> v2:172.21.15.93:3300/0 conn(0x7fcb880cfaf0 0x7fcb880d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.933+0000 7fcb86ffd640 1 -- 172.21.15.81:0/64150250 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb880b5e70 con 0x7fcb880d39a0 2023-11-08T16:29:37.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.933+0000 7fcb877fe640 1 --2- 172.21.15.81:0/64150250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb8815d4c0 0x7fcb880cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:37.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.935+0000 7fcb86ffd640 1 --2- 172.21.15.81:0/64150250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb880d39a0 0x7fcb880cf5b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb880cf350 tx=0x7fcb7c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c535f0159b9a6d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:37.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.936+0000 7fcb867fc640 1 -- 172.21.15.81:0/64150250 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c030860 con 0x7fcb880d39a0 2023-11-08T16:29:37.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.936+0000 7fcb867fc640 1 -- 172.21.15.81:0/64150250 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c0309c0 con 0x7fcb880d39a0 2023-11-08T16:29:37.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.936+0000 7fcb867fc640 1 -- 172.21.15.81:0/64150250 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c030d50 con 0x7fcb880d39a0 2023-11-08T16:29:37.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.937+0000 7fcb8e16c640 1 -- 172.21.15.81:0/64150250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb880d39a0 msgr2=0x7fcb880cf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:37.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.937+0000 7fcb8e16c640 1 --2- 172.21.15.81:0/64150250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb880d39a0 0x7fcb880cf5b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb880cf350 tx=0x7fcb7c02b9a0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.937+0000 7fcb8e16c640 1 -- 172.21.15.81:0/64150250 shutdown_connections 2023-11-08T16:29:37.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.937+0000 7fcb8e16c640 1 --2- 172.21.15.81:0/64150250 >> v2:172.21.15.93:3300/0 conn(0x7fcb880cfaf0 0x7fcb880d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.937+0000 7fcb8e16c640 1 --2- 172.21.15.81:0/64150250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb880d39a0 0x7fcb880cf5b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.937+0000 7fcb8e16c640 1 --2- 172.21.15.81:0/64150250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb8815d4c0 0x7fcb880cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.937+0000 7fcb8e16c640 1 -- 172.21.15.81:0/64150250 >> 172.21.15.81:0/64150250 conn(0x7fcb880d2e80 msgr2=0x7fcb880d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:37.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.937+0000 7fcb8e16c640 1 -- 172.21.15.81:0/64150250 shutdown_connections 2023-11-08T16:29:37.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.937+0000 7fcb8e16c640 1 -- 172.21.15.81:0/64150250 wait complete. 2023-11-08T16:29:37.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.939+0000 7fcb8e16c640 1 Processor -- start 2023-11-08T16:29:37.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.939+0000 7fcb8e16c640 1 -- start start 2023-11-08T16:29:37.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.940+0000 7fcb8e16c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb880cfaf0 0x7fcb880c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.940+0000 7fcb8e16c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb880d39a0 0x7fcb8806be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.940+0000 7fcb8e16c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb8815d4c0 0x7fcb8815bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.940+0000 7fcb8e16c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880c78d0 con 0x7fcb880d39a0 2023-11-08T16:29:37.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.940+0000 7fcb8e16c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb880c6660 con 0x7fcb8815d4c0 2023-11-08T16:29:37.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.940+0000 7fcb8e16c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb881f3cc0 con 0x7fcb880cfaf0 2023-11-08T16:29:37.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.940+0000 7fcb86ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb880d39a0 0x7fcb8806be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:37.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.940+0000 7fcb86ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb880d39a0 0x7fcb8806be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45272/0 (socket says 172.21.15.81:45272) 2023-11-08T16:29:37.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.940+0000 7fcb86ffd640 1 -- 172.21.15.81:0/3215368731 learned_addr learned my addr 172.21.15.81:0/3215368731 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:37.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb87fff640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb8815d4c0 0x7fcb8815bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:37.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb877fe640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb880cfaf0 0x7fcb880c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:37.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb86ffd640 1 -- 172.21.15.81:0/3215368731 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb880cfaf0 msgr2=0x7fcb880c7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:37.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb86ffd640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb880cfaf0 0x7fcb880c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb86ffd640 1 -- 172.21.15.81:0/3215368731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb8815d4c0 msgr2=0x7fcb8815bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:37.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb86ffd640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb8815d4c0 0x7fcb8815bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:37.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb86ffd640 1 -- 172.21.15.81:0/3215368731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c0094d0 con 0x7fcb880d39a0 2023-11-08T16:29:37.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb87fff640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb8815d4c0 0x7fcb8815bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:37.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb86ffd640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb880d39a0 0x7fcb8806be40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c000c00 tx=0x7fcb7c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:37.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb6ffff640 1 -- 172.21.15.81:0/3215368731 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c039d60 con 0x7fcb880d39a0 2023-11-08T16:29:37.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb6ffff640 1 -- 172.21.15.81:0/3215368731 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c043400 con 0x7fcb880d39a0 2023-11-08T16:29:37.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb6ffff640 1 -- 172.21.15.81:0/3215368731 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c0437b0 con 0x7fcb880d39a0 2023-11-08T16:29:37.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.941+0000 7fcb8e16c640 1 -- 172.21.15.81:0/3215368731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb880c68b0 con 0x7fcb880d39a0 2023-11-08T16:29:37.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.942+0000 7fcb8e16c640 1 -- 172.21.15.81:0/3215368731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb880c6e70 con 0x7fcb880d39a0 2023-11-08T16:29:37.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.943+0000 7fcb8e16c640 1 -- 172.21.15.81:0/3215368731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8804e490 con 0x7fcb880d39a0 2023-11-08T16:29:37.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.944+0000 7fcb6ffff640 1 -- 172.21.15.81:0/3215368731 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb7c043910 con 0x7fcb880d39a0 2023-11-08T16:29:37.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.945+0000 7fcb6ffff640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb7406ae70 0x7fcb7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:37.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.945+0000 7fcb6ffff640 1 -- 172.21.15.81:0/3215368731 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb7c03e070 con 0x7fcb880d39a0 2023-11-08T16:29:37.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.945+0000 7fcb877fe640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb7406ae70 0x7fcb7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:37.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.946+0000 7fcb877fe640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb7406ae70 0x7fcb7406d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcb78008ee0 tx=0x7fcb78008430 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:37.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:37.949+0000 7fcb6ffff640 1 -- 172.21.15.81:0/3215368731 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb7c0779b0 con 0x7fcb880d39a0 2023-11-08T16:29:38.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.139+0000 7fcb8e16c640 1 -- 172.21.15.81:0/3215368731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb8808d520 con 0x7fcb880d39a0 2023-11-08T16:29:38.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.140+0000 7fcb6ffff640 1 -- 172.21.15.81:0/3215368731 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb7c07afd0 con 0x7fcb880d39a0 2023-11-08T16:29:38.142 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:38.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.142+0000 7fcb8e16c640 1 -- 172.21.15.81:0/3215368731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb7406ae70 msgr2=0x7fcb7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:38.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.142+0000 7fcb8e16c640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb7406ae70 0x7fcb7406d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcb78008ee0 tx=0x7fcb78008430 comp rx=0 tx=0).stop 2023-11-08T16:29:38.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.143+0000 7fcb8e16c640 1 -- 172.21.15.81:0/3215368731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb880d39a0 msgr2=0x7fcb8806be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:38.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.143+0000 7fcb8e16c640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb880d39a0 0x7fcb8806be40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c000c00 tx=0x7fcb7c004730 comp rx=0 tx=0).stop 2023-11-08T16:29:38.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.143+0000 7fcb8e16c640 1 -- 172.21.15.81:0/3215368731 shutdown_connections 2023-11-08T16:29:38.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.143+0000 7fcb8e16c640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb7406ae70 0x7fcb7406d330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.143+0000 7fcb8e16c640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb8815d4c0 0x7fcb8815bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.143+0000 7fcb8e16c640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb880d39a0 0x7fcb8806be40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.143+0000 7fcb8e16c640 1 --2- 172.21.15.81:0/3215368731 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb880cfaf0 0x7fcb880c7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.143+0000 7fcb8e16c640 1 -- 172.21.15.81:0/3215368731 >> 172.21.15.81:0/3215368731 conn(0x7fcb880d2e80 msgr2=0x7fcb881e41b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:38.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.143+0000 7fcb8e16c640 1 -- 172.21.15.81:0/3215368731 shutdown_connections 2023-11-08T16:29:38.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.143+0000 7fcb8e16c640 1 -- 172.21.15.81:0/3215368731 wait complete. 2023-11-08T16:29:38.162 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:38.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.379+0000 7f3b2759e640 1 Processor -- start 2023-11-08T16:29:38.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.379+0000 7f3b2759e640 1 -- start start 2023-11-08T16:29:38.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.381+0000 7f3b2759e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b2815a990 0x7f3b2815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.381+0000 7f3b2759e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3b280d0aa0 0x7f3b280d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.381+0000 7f3b2759e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b280d1430 con 0x7f3b280d1640 2023-11-08T16:29:38.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.381+0000 7f3b2759e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b280b3530 con 0x7f3b2815a990 2023-11-08T16:29:38.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.381+0000 7f3b2759e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b280cd250 con 0x7f3b280d0aa0 2023-11-08T16:29:38.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.381+0000 7f3b2659c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b2815a990 0x7f3b2815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:38.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.381+0000 7f3b2659c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b2815a990 0x7f3b2815ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45282/0 (socket says 172.21.15.81:45282) 2023-11-08T16:29:38.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.381+0000 7f3b2659c640 1 -- 172.21.15.81:0/1493519170 learned_addr learned my addr 172.21.15.81:0/1493519170 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:38.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.382+0000 7f3b26d9d640 1 --2- 172.21.15.81:0/1493519170 >> v2:172.21.15.93:3300/0 conn(0x7f3b280d0aa0 0x7f3b280d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:38.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.382+0000 7f3b2659c640 1 -- 172.21.15.81:0/1493519170 >> v2:172.21.15.93:3300/0 conn(0x7f3b280d0aa0 msgr2=0x7f3b280d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:38.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.382+0000 7f3b2659c640 1 --2- 172.21.15.81:0/1493519170 >> v2:172.21.15.93:3300/0 conn(0x7f3b280d0aa0 0x7f3b280d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.382+0000 7f3b2659c640 1 -- 172.21.15.81:0/1493519170 >> v1:172.21.15.93:6789/0 conn(0x7f3b280d1640 legacy=0x7f3b280b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:38.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.382+0000 7f3b2659c640 1 -- 172.21.15.81:0/1493519170 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b280cd530 con 0x7f3b2815a990 2023-11-08T16:29:38.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.383+0000 7f3b2659c640 1 --2- 172.21.15.81:0/1493519170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b2815a990 0x7f3b2815ad70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3b28070480 tx=0x7f3b1802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91abc4f6817373b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:38.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.383+0000 7f3b25d9b640 1 -- 172.21.15.81:0/1493519170 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18031870 con 0x7f3b2815a990 2023-11-08T16:29:38.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.383+0000 7f3b25d9b640 1 -- 172.21.15.81:0/1493519170 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b180319d0 con 0x7f3b2815a990 2023-11-08T16:29:38.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.383+0000 7f3b25d9b640 1 -- 172.21.15.81:0/1493519170 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b18031d60 con 0x7f3b2815a990 2023-11-08T16:29:38.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.384+0000 7f3b2759e640 1 -- 172.21.15.81:0/1493519170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b2815a990 msgr2=0x7f3b2815ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:38.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.384+0000 7f3b2759e640 1 --2- 172.21.15.81:0/1493519170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b2815a990 0x7f3b2815ad70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3b28070480 tx=0x7f3b1802b910 comp rx=0 tx=0).stop 2023-11-08T16:29:38.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.384+0000 7f3b2759e640 1 -- 172.21.15.81:0/1493519170 shutdown_connections 2023-11-08T16:29:38.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.384+0000 7f3b2759e640 1 --2- 172.21.15.81:0/1493519170 >> v2:172.21.15.93:3300/0 conn(0x7f3b280d0aa0 0x7f3b280d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.384+0000 7f3b2759e640 1 --2- 172.21.15.81:0/1493519170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b2815a990 0x7f3b2815ad70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.384+0000 7f3b2759e640 1 -- 172.21.15.81:0/1493519170 >> 172.21.15.81:0/1493519170 conn(0x7f3b2806d9c0 msgr2=0x7f3b2806dda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:38.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.384+0000 7f3b2759e640 1 -- 172.21.15.81:0/1493519170 shutdown_connections 2023-11-08T16:29:38.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.384+0000 7f3b2759e640 1 -- 172.21.15.81:0/1493519170 wait complete. 2023-11-08T16:29:38.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.386+0000 7f3b2759e640 1 Processor -- start 2023-11-08T16:29:38.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.386+0000 7f3b2759e640 1 -- start start 2023-11-08T16:29:38.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.387+0000 7f3b2759e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b280d0aa0 0x7f3b28156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.387+0000 7f3b2759e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280d1640 0x7f3b2814ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.387+0000 7f3b2759e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b2815a990 0x7f3b28158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.387+0000 7f3b2759e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b280c57b0 con 0x7f3b280d1640 2023-11-08T16:29:38.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.387+0000 7f3b2759e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b28156f80 con 0x7f3b280d0aa0 2023-11-08T16:29:38.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.387+0000 7f3b2759e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2814e310 con 0x7f3b2815a990 2023-11-08T16:29:38.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.387+0000 7f3b1dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280d1640 0x7f3b2814ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:38.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.387+0000 7f3b1dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280d1640 0x7f3b2814ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45290/0 (socket says 172.21.15.81:45290) 2023-11-08T16:29:38.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.387+0000 7f3b1dd9b640 1 -- 172.21.15.81:0/2879024081 learned_addr learned my addr 172.21.15.81:0/2879024081 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:38.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.388+0000 7f3b1dd9b640 1 -- 172.21.15.81:0/2879024081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b2815a990 msgr2=0x7f3b28158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:38.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.388+0000 7f3b1dd9b640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b2815a990 0x7f3b28158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.388+0000 7f3b1dd9b640 1 -- 172.21.15.81:0/2879024081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b280d0aa0 msgr2=0x7f3b28156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:38.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.388+0000 7f3b1dd9b640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b280d0aa0 0x7f3b28156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.388+0000 7f3b1dd9b640 1 -- 172.21.15.81:0/2879024081 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b180094d0 con 0x7f3b280d1640 2023-11-08T16:29:38.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.388+0000 7f3b1dd9b640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280d1640 0x7f3b2814ddd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c400 tx=0x7f3b1000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:38.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.388+0000 7f3b1f7fe640 1 -- 172.21.15.81:0/2879024081 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b100035f0 con 0x7f3b280d1640 2023-11-08T16:29:38.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.388+0000 7f3b1f7fe640 1 -- 172.21.15.81:0/2879024081 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b10003750 con 0x7f3b280d1640 2023-11-08T16:29:38.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.388+0000 7f3b1f7fe640 1 -- 172.21.15.81:0/2879024081 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10003b40 con 0x7f3b280d1640 2023-11-08T16:29:38.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.388+0000 7f3b2759e640 1 -- 172.21.15.81:0/2879024081 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b281572e0 con 0x7f3b280d1640 2023-11-08T16:29:38.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.388+0000 7f3b2759e640 1 -- 172.21.15.81:0/2879024081 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b281578a0 con 0x7f3b280d1640 2023-11-08T16:29:38.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.389+0000 7f3b2759e640 1 -- 172.21.15.81:0/2879024081 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2804ece0 con 0x7f3b280d1640 2023-11-08T16:29:38.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.390+0000 7f3b1f7fe640 1 -- 172.21.15.81:0/2879024081 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3b10003ca0 con 0x7f3b280d1640 2023-11-08T16:29:38.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.391+0000 7f3b1f7fe640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0806ae70 0x7f3b0806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.391+0000 7f3b1f7fe640 1 -- 172.21.15.81:0/2879024081 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b100887c0 con 0x7f3b280d1640 2023-11-08T16:29:38.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.392+0000 7f3b2659c640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0806ae70 0x7f3b0806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:38.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.392+0000 7f3b2659c640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0806ae70 0x7f3b0806d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b18036000 tx=0x7f3b18030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:38.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.396+0000 7f3b1f7fe640 1 -- 172.21.15.81:0/2879024081 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b10053920 con 0x7f3b280d1640 2023-11-08T16:29:38.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.589+0000 7f3b2759e640 1 -- 172.21.15.81:0/2879024081 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b280b29f0 con 0x7f3b280d1640 2023-11-08T16:29:38.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.589+0000 7f3b1f7fe640 1 -- 172.21.15.81:0/2879024081 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b10056f40 con 0x7f3b280d1640 2023-11-08T16:29:38.591 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:38.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.592+0000 7f3b2759e640 1 -- 172.21.15.81:0/2879024081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0806ae70 msgr2=0x7f3b0806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:38.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.592+0000 7f3b2759e640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0806ae70 0x7f3b0806d330 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f3b18036000 tx=0x7f3b18030000 comp rx=0 tx=0).stop 2023-11-08T16:29:38.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.592+0000 7f3b2759e640 1 -- 172.21.15.81:0/2879024081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280d1640 msgr2=0x7f3b2814ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:38.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.592+0000 7f3b2759e640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280d1640 0x7f3b2814ddd0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f3b1000c400 tx=0x7f3b1000c8c0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.592+0000 7f3b2759e640 1 -- 172.21.15.81:0/2879024081 shutdown_connections 2023-11-08T16:29:38.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.592+0000 7f3b2759e640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0806ae70 0x7f3b0806d330 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.592+0000 7f3b2759e640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b2815a990 0x7f3b28158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.592+0000 7f3b2759e640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b280d1640 0x7f3b2814ddd0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.592+0000 7f3b2759e640 1 --2- 172.21.15.81:0/2879024081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b280d0aa0 0x7f3b28156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.592+0000 7f3b2759e640 1 -- 172.21.15.81:0/2879024081 >> 172.21.15.81:0/2879024081 conn(0x7f3b2806d9c0 msgr2=0x7f3b2806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:38.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.592+0000 7f3b2759e640 1 -- 172.21.15.81:0/2879024081 shutdown_connections 2023-11-08T16:29:38.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.593+0000 7f3b2759e640 1 -- 172.21.15.81:0/2879024081 wait complete. 2023-11-08T16:29:38.613 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:38.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.823+0000 7fb20f407640 1 Processor -- start 2023-11-08T16:29:38.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.823+0000 7fb20f407640 1 -- start start 2023-11-08T16:29:38.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.824+0000 7fb20f407640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb2080ce390 0x7fb20815e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.825+0000 7fb20f407640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb20806be20 0x7fb2080ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.825+0000 7fb20f407640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20815e8d0 con 0x7fb2080c9200 2023-11-08T16:29:38.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.825+0000 7fb20f407640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2080ca2f0 con 0x7fb2080ce390 2023-11-08T16:29:38.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.825+0000 7fb20f407640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080ce770 con 0x7fb20806be20 2023-11-08T16:29:38.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.825+0000 7fb20d9a4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb20806be20 0x7fb2080ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:38.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.825+0000 7fb20d1a3640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb2080ce390 0x7fb20815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:38.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.826+0000 7fb20d9a4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb20806be20 0x7fb2080ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:58390/0 (socket says 172.21.15.81:58390) 2023-11-08T16:29:38.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.826+0000 7fb20d9a4640 1 -- 172.21.15.81:0/2645276201 learned_addr learned my addr 172.21.15.81:0/2645276201 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:38.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.826+0000 7fb20d9a4640 1 -- 172.21.15.81:0/2645276201 >> v2:172.21.15.93:3300/0 conn(0x7fb2080ce390 msgr2=0x7fb20815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:38.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.826+0000 7fb20d9a4640 1 --2- 172.21.15.81:0/2645276201 >> v2:172.21.15.93:3300/0 conn(0x7fb2080ce390 0x7fb20815e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.826+0000 7fb20d9a4640 1 -- 172.21.15.81:0/2645276201 >> v1:172.21.15.93:6789/0 conn(0x7fb2080c9200 legacy=0x7fb20806b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:38.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.826+0000 7fb20d9a4640 1 -- 172.21.15.81:0/2645276201 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2080c96e0 con 0x7fb20806be20 2023-11-08T16:29:38.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.828+0000 7fb20d9a4640 1 --2- 172.21.15.81:0/2645276201 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb20806be20 0x7fb2080ac8b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb200009e50 tx=0x7fb20002bfe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3b638d931c7877f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:38.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.828+0000 7fb1fffff640 1 -- 172.21.15.81:0/2645276201 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20002cb80 con 0x7fb20806be20 2023-11-08T16:29:38.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.828+0000 7fb1fffff640 1 -- 172.21.15.81:0/2645276201 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20002cce0 con 0x7fb20806be20 2023-11-08T16:29:38.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.828+0000 7fb1fffff640 1 -- 172.21.15.81:0/2645276201 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb200034610 con 0x7fb20806be20 2023-11-08T16:29:38.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.829+0000 7fb20f407640 1 -- 172.21.15.81:0/2645276201 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb20806be20 msgr2=0x7fb2080ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:38.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.829+0000 7fb20f407640 1 --2- 172.21.15.81:0/2645276201 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb20806be20 0x7fb2080ac8b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb200009e50 tx=0x7fb20002bfe0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.829+0000 7fb20f407640 1 -- 172.21.15.81:0/2645276201 shutdown_connections 2023-11-08T16:29:38.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.829+0000 7fb20f407640 1 --2- 172.21.15.81:0/2645276201 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb20806be20 0x7fb2080ac8b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.829+0000 7fb20f407640 1 --2- 172.21.15.81:0/2645276201 >> v2:172.21.15.93:3300/0 conn(0x7fb2080ce390 0x7fb20815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.829+0000 7fb20f407640 1 -- 172.21.15.81:0/2645276201 >> 172.21.15.81:0/2645276201 conn(0x7fb2080cb580 msgr2=0x7fb2080cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:38.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.829+0000 7fb20f407640 1 -- 172.21.15.81:0/2645276201 shutdown_connections 2023-11-08T16:29:38.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.829+0000 7fb20f407640 1 -- 172.21.15.81:0/2645276201 wait complete. 2023-11-08T16:29:38.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.831+0000 7fb20f407640 1 Processor -- start 2023-11-08T16:29:38.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.831+0000 7fb20f407640 1 -- start start 2023-11-08T16:29:38.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.832+0000 7fb20f407640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb20806be20 0x7fb2080c5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.832+0000 7fb20f407640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2080c9200 0x7fb2081e69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.832+0000 7fb20f407640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2080ce390 0x7fb2081e6f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.832+0000 7fb20f407640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080c3a70 con 0x7fb2080c9200 2023-11-08T16:29:38.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.832+0000 7fb20f407640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2081e7440 con 0x7fb2080ce390 2023-11-08T16:29:38.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.832+0000 7fb20f407640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2080c2df0 con 0x7fb20806be20 2023-11-08T16:29:38.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.832+0000 7fb20c9a2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2080c9200 0x7fb2081e69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:38.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.832+0000 7fb20c9a2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2080c9200 0x7fb2081e69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45296/0 (socket says 172.21.15.81:45296) 2023-11-08T16:29:38.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.832+0000 7fb20c9a2640 1 -- 172.21.15.81:0/1326772409 learned_addr learned my addr 172.21.15.81:0/1326772409 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:38.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb20c9a2640 1 -- 172.21.15.81:0/1326772409 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb20806be20 msgr2=0x7fb2080c5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:38.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb20c9a2640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb20806be20 0x7fb2080c5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb20c9a2640 1 -- 172.21.15.81:0/1326772409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2080ce390 msgr2=0x7fb2081e6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:38.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb20d9a4640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2080ce390 0x7fb2081e6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:38.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb20c9a2640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2080ce390 0x7fb2081e6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:38.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb20c9a2640 1 -- 172.21.15.81:0/1326772409 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb200009b30 con 0x7fb2080c9200 2023-11-08T16:29:38.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb20c9a2640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2080c9200 0x7fb2081e69c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800c440 tx=0x7fb1f8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:38.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb1fdffb640 1 -- 172.21.15.81:0/1326772409 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f80052d0 con 0x7fb2080c9200 2023-11-08T16:29:38.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb1fdffb640 1 -- 172.21.15.81:0/1326772409 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f8005430 con 0x7fb2080c9200 2023-11-08T16:29:38.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb1fdffb640 1 -- 172.21.15.81:0/1326772409 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8007260 con 0x7fb2080c9200 2023-11-08T16:29:38.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb20f407640 1 -- 172.21.15.81:0/1326772409 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2080c30d0 con 0x7fb2080c9200 2023-11-08T16:29:38.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.833+0000 7fb20f407640 1 -- 172.21.15.81:0/1326772409 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2080c3690 con 0x7fb2080c9200 2023-11-08T16:29:38.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.834+0000 7fb20f407640 1 -- 172.21.15.81:0/1326772409 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2080c3bb0 con 0x7fb2080c9200 2023-11-08T16:29:38.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.835+0000 7fb1fdffb640 1 -- 172.21.15.81:0/1326772409 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb1f80073c0 con 0x7fb2080c9200 2023-11-08T16:29:38.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.836+0000 7fb1fdffb640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb1ec06ae70 0x7fb1ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:38.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.836+0000 7fb1fdffb640 1 -- 172.21.15.81:0/1326772409 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1f8088850 con 0x7fb2080c9200 2023-11-08T16:29:38.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.837+0000 7fb20d1a3640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb1ec06ae70 0x7fb1ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:38.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.837+0000 7fb20d1a3640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb1ec06ae70 0x7fb1ec06d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb204006a90 tx=0x7fb204006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:38.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:38.840+0000 7fb1fdffb640 1 -- 172.21.15.81:0/1326772409 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2080c3bb0 con 0x7fb2080c9200 2023-11-08T16:29:39.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.031+0000 7fb20f407640 1 -- 172.21.15.81:0/1326772409 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb20815e8d0 con 0x7fb2080c9200 2023-11-08T16:29:39.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.032+0000 7fb1fdffb640 1 -- 172.21.15.81:0/1326772409 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb20815e8d0 con 0x7fb2080c9200 2023-11-08T16:29:39.034 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:39.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 -- 172.21.15.81:0/1326772409 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb1ec06ae70 msgr2=0x7fb1ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb1ec06ae70 0x7fb1ec06d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb204006a90 tx=0x7fb204006a20 comp rx=0 tx=0).stop 2023-11-08T16:29:39.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 -- 172.21.15.81:0/1326772409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2080c9200 msgr2=0x7fb2081e69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2080c9200 0x7fb2081e69c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800c440 tx=0x7fb1f8001e70 comp rx=0 tx=0).stop 2023-11-08T16:29:39.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 -- 172.21.15.81:0/1326772409 shutdown_connections 2023-11-08T16:29:39.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb1ec06ae70 0x7fb1ec06d330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2080ce390 0x7fb2081e6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2080c9200 0x7fb2081e69c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 --2- 172.21.15.81:0/1326772409 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb20806be20 0x7fb2080c5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 -- 172.21.15.81:0/1326772409 >> 172.21.15.81:0/1326772409 conn(0x7fb2080cb580 msgr2=0x7fb2080ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:39.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 -- 172.21.15.81:0/1326772409 shutdown_connections 2023-11-08T16:29:39.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.035+0000 7fb20f407640 1 -- 172.21.15.81:0/1326772409 wait complete. 2023-11-08T16:29:39.056 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:39.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.259+0000 7f37b2539640 1 Processor -- start 2023-11-08T16:29:39.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.259+0000 7f37b2539640 1 -- start start 2023-11-08T16:29:39.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.261+0000 7f37b2539640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac161710 0x7f37ac0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.261+0000 7f37b2539640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac06b980 con 0x7f37ac15d2b0 2023-11-08T16:29:39.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.261+0000 7f37b2539640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37ac06bc60 con 0x7f37ac0ccd30 2023-11-08T16:29:39.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.261+0000 7f37b2539640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0cd1c0 con 0x7f37ac161710 2023-11-08T16:29:39.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.261+0000 7f37abfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f37ac15d2b0 0x7f37ac0c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38000/0 (socket says 172.21.15.81:38000) 2023-11-08T16:29:39.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.261+0000 7f37abfff640 1 -- 172.21.15.81:0/1445341643 learned_addr learned my addr 172.21.15.81:0/1445341643 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:39.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37a3fff640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac161710 0x7f37ac0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:39.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3035411563 0 0) 0x7f37ac06b980 con 0x7f37ac15d2b0 2023-11-08T16:29:39.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3798003520 con 0x7f37ac15d2b0 2023-11-08T16:29:39.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1209676795 0 0) 0x7f3798003520 con 0x7f37ac15d2b0 2023-11-08T16:29:39.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac161710 msgr2=0x7f37ac0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:39.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac161710 0x7f37ac0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 >> v1:172.21.15.93:6789/0 conn(0x7f37ac0ccd30 legacy=0x7f37ac06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac06bf40 con 0x7f37ac15d2b0 2023-11-08T16:29:39.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f379c001670 con 0x7f37ac15d2b0 2023-11-08T16:29:39.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 >> v1:172.21.15.81:6789/0 conn(0x7f37ac15d2b0 legacy=0x7f37ac0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac161710 0x7f3798003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3798004050 0x7f37980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3798004d50 0x7f37980051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac0cd1c0 con 0x7f3798004050 2023-11-08T16:29:39.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379800bbc0 con 0x7f3798004d50 2023-11-08T16:29:39.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.262+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379800bd80 con 0x7f37ac161710 2023-11-08T16:29:39.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.263+0000 7f37a3fff640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3798004050 0x7f37980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:39.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.263+0000 7f37abfff640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac161710 0x7f3798003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:39.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.263+0000 7f37b0ad6640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3798004d50 0x7f37980051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:39.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.263+0000 7f37a3fff640 1 -- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac161710 msgr2=0x7f3798003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:39.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.263+0000 7f37a3fff640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac161710 0x7f3798003a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.263+0000 7f37a3fff640 1 -- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3798004d50 msgr2=0x7f37980051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:39.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.263+0000 7f37a3fff640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3798004d50 0x7f37980051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.263+0000 7f37a3fff640 1 -- 172.21.15.81:0/1445341643 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798003520 con 0x7f3798004050 2023-11-08T16:29:39.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.263+0000 7f37b0ad6640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3798004d50 0x7f37980051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:39.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.264+0000 7f37a3fff640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3798004050 0x7f37980047c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3794009ac0 tx=0x7f37980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b896275eb18fc8fe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:39.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.265+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379402c6b0 con 0x7f3798004050 2023-11-08T16:29:39.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.265+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379402c810 con 0x7f3798004050 2023-11-08T16:29:39.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.265+0000 7f37ab7fe640 1 -- 172.21.15.81:0/1445341643 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379402cba0 con 0x7f3798004050 2023-11-08T16:29:39.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.265+0000 7f37b2539640 1 -- 172.21.15.81:0/1445341643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3798004050 msgr2=0x7f37980047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.265+0000 7f37b2539640 1 --2- 172.21.15.81:0/1445341643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3798004050 0x7f37980047c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3794009ac0 tx=0x7f37980057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.265+0000 7f37b0ad6640 1 -- 172.21.15.81:0/1445341643 reap_dead start 2023-11-08T16:29:39.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.265+0000 7f37b2539640 1 -- 172.21.15.81:0/1445341643 shutdown_connections 2023-11-08T16:29:39.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.265+0000 7f37b2539640 1 -- 172.21.15.81:0/1445341643 >> 172.21.15.81:0/1445341643 conn(0x7f37ac0cb490 msgr2=0x7f37ac0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:39.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.265+0000 7f37b2539640 1 -- 172.21.15.81:0/1445341643 shutdown_connections 2023-11-08T16:29:39.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.265+0000 7f37b2539640 1 -- 172.21.15.81:0/1445341643 wait complete. 2023-11-08T16:29:39.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.267+0000 7f37b2539640 1 Processor -- start 2023-11-08T16:29:39.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.267+0000 7f37b2539640 1 -- start start 2023-11-08T16:29:39.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.268+0000 7f37b2539640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3798004050 0x7f37ac1f2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.268+0000 7f37b2539640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac1f35f0 0x7f37ac071560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.268+0000 7f37b2539640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37ac15db60 0x7f37ac15e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.268+0000 7f37b2539640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac15e550 con 0x7f37ac15db60 2023-11-08T16:29:39.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.268+0000 7f37b2539640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac160030 con 0x7f3798004050 2023-11-08T16:29:39.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.268+0000 7f37b2539640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37ac15f420 con 0x7f37ac1f35f0 2023-11-08T16:29:39.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.268+0000 7f37b0ad6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37ac15db60 0x7f37ac15e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:39.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.269+0000 7f37b0ad6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37ac15db60 0x7f37ac15e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45318/0 (socket says 172.21.15.81:45318) 2023-11-08T16:29:39.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.269+0000 7f37b0ad6640 1 -- 172.21.15.81:0/1184716130 learned_addr learned my addr 172.21.15.81:0/1184716130 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:39.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.269+0000 7f37b0ad6640 1 -- 172.21.15.81:0/1184716130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac1f35f0 msgr2=0x7f37ac071560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.269+0000 7f37b0ad6640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac1f35f0 0x7f37ac071560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.269+0000 7f37b0ad6640 1 -- 172.21.15.81:0/1184716130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3798004050 msgr2=0x7f37ac1f2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.269+0000 7f37b0ad6640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3798004050 0x7f37ac1f2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.269+0000 7f37b0ad6640 1 -- 172.21.15.81:0/1184716130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37ac15f700 con 0x7f37ac15db60 2023-11-08T16:29:39.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.269+0000 7f37b0ad6640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37ac15db60 0x7f37ac15e010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000f80 tx=0x7f37a400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:39.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.269+0000 7f37a97fa640 1 -- 172.21.15.81:0/1184716130 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4017070 con 0x7f37ac15db60 2023-11-08T16:29:39.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.270+0000 7f37a97fa640 1 -- 172.21.15.81:0/1184716130 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a4006b50 con 0x7f37ac15db60 2023-11-08T16:29:39.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.270+0000 7f37a97fa640 1 -- 172.21.15.81:0/1184716130 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a4003970 con 0x7f37ac15db60 2023-11-08T16:29:39.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.270+0000 7f37b2539640 1 -- 172.21.15.81:0/1184716130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37ac15f9e0 con 0x7f37ac15db60 2023-11-08T16:29:39.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.270+0000 7f37b2539640 1 -- 172.21.15.81:0/1184716130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37ac071aa0 con 0x7f37ac15db60 2023-11-08T16:29:39.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.271+0000 7f37b2539640 1 -- 172.21.15.81:0/1184716130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37ac04ece0 con 0x7f37ac15db60 2023-11-08T16:29:39.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.272+0000 7f37a97fa640 1 -- 172.21.15.81:0/1184716130 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37a4003ad0 con 0x7f37ac15db60 2023-11-08T16:29:39.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.273+0000 7f37a97fa640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f378c06aea0 0x7f378c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.273+0000 7f37a97fa640 1 -- 172.21.15.81:0/1184716130 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37a4088e90 con 0x7f37ac15db60 2023-11-08T16:29:39.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.273+0000 7f37abfff640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f378c06aea0 0x7f378c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:39.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.273+0000 7f37abfff640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f378c06aea0 0x7f378c06d360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f379c001050 tx=0x7f379c000f40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:39.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.277+0000 7f37a97fa640 1 -- 172.21.15.81:0/1184716130 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37a4053f20 con 0x7f37ac15db60 2023-11-08T16:29:39.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.467+0000 7f37b2539640 1 -- 172.21.15.81:0/1184716130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f37ac067300 con 0x7f37ac15db60 2023-11-08T16:29:39.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.469+0000 7f37a97fa640 1 -- 172.21.15.81:0/1184716130 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f37a4057540 con 0x7f37ac15db60 2023-11-08T16:29:39.472 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:39.472 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:04.727022+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:39.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 -- 172.21.15.81:0/1184716130 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f378c06aea0 msgr2=0x7f378c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f378c06aea0 0x7f378c06d360 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f379c001050 tx=0x7f379c000f40 comp rx=0 tx=0).stop 2023-11-08T16:29:39.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 -- 172.21.15.81:0/1184716130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37ac15db60 msgr2=0x7f37ac15e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37ac15db60 0x7f37ac15e010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f37a4000f80 tx=0x7f37a400dc00 comp rx=0 tx=0).stop 2023-11-08T16:29:39.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 -- 172.21.15.81:0/1184716130 shutdown_connections 2023-11-08T16:29:39.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f378c06aea0 0x7f378c06d360 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37ac15db60 0x7f37ac15e010 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37ac1f35f0 0x7f37ac071560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 --2- 172.21.15.81:0/1184716130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3798004050 0x7f37ac1f2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 -- 172.21.15.81:0/1184716130 >> 172.21.15.81:0/1184716130 conn(0x7f37ac0cb490 msgr2=0x7f37ac0af5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:39.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 -- 172.21.15.81:0/1184716130 shutdown_connections 2023-11-08T16:29:39.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.472+0000 7f37b2539640 1 -- 172.21.15.81:0/1184716130 wait complete. 2023-11-08T16:29:39.494 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:39.494 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:39.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.714+0000 7f63c7925640 1 Processor -- start 2023-11-08T16:29:39.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.714+0000 7f63c7925640 1 -- start start 2023-11-08T16:29:39.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.715+0000 7f63c7925640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f63c00ce0d0 0x7f63c00caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c7925640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63c015bfc0 0x7f63c015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c7925640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c006fa30 con 0x7f63c015f0b0 2023-11-08T16:29:39.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c7925640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c00cda10 con 0x7f63c015bfc0 2023-11-08T16:29:39.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c7925640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63c00ce4b0 con 0x7f63c00ce0d0 2023-11-08T16:29:39.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c4ec0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63c015bfc0 0x7f63c015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:39.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c4ec0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63c015bfc0 0x7f63c015eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45332/0 (socket says 172.21.15.81:45332) 2023-11-08T16:29:39.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c4ec0640 1 -- 172.21.15.81:0/1387171208 learned_addr learned my addr 172.21.15.81:0/1387171208 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:39.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c56c1640 1 --2- 172.21.15.81:0/1387171208 >> v2:172.21.15.93:3300/0 conn(0x7f63c00ce0d0 0x7f63c00caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:39.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c4ec0640 1 -- 172.21.15.81:0/1387171208 >> v2:172.21.15.93:3300/0 conn(0x7f63c00ce0d0 msgr2=0x7f63c00caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:39.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c4ec0640 1 --2- 172.21.15.81:0/1387171208 >> v2:172.21.15.93:3300/0 conn(0x7f63c00ce0d0 0x7f63c00caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c4ec0640 1 -- 172.21.15.81:0/1387171208 >> v1:172.21.15.93:6789/0 conn(0x7f63c015f0b0 legacy=0x7f63c01516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:39.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.716+0000 7f63c4ec0640 1 -- 172.21.15.81:0/1387171208 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c015f490 con 0x7f63c015bfc0 2023-11-08T16:29:39.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.718+0000 7f63c4ec0640 1 --2- 172.21.15.81:0/1387171208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c015bfc0 0x7f63c015eb70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f63c006f3a0 tx=0x7f63b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da94507894105d07 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:39.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.718+0000 7f63b7fff640 1 -- 172.21.15.81:0/1387171208 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0031870 con 0x7f63c015bfc0 2023-11-08T16:29:39.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.718+0000 7f63b7fff640 1 -- 172.21.15.81:0/1387171208 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b00319d0 con 0x7f63c015bfc0 2023-11-08T16:29:39.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.718+0000 7f63b7fff640 1 -- 172.21.15.81:0/1387171208 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0031d60 con 0x7f63c015bfc0 2023-11-08T16:29:39.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.718+0000 7f63c7925640 1 -- 172.21.15.81:0/1387171208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c015bfc0 msgr2=0x7f63c015eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.718+0000 7f63c7925640 1 --2- 172.21.15.81:0/1387171208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c015bfc0 0x7f63c015eb70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f63c006f3a0 tx=0x7f63b002b910 comp rx=0 tx=0).stop 2023-11-08T16:29:39.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.719+0000 7f63c7925640 1 -- 172.21.15.81:0/1387171208 shutdown_connections 2023-11-08T16:29:39.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.719+0000 7f63c7925640 1 --2- 172.21.15.81:0/1387171208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c015bfc0 0x7f63c015eb70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.719+0000 7f63c7925640 1 --2- 172.21.15.81:0/1387171208 >> v2:172.21.15.93:3300/0 conn(0x7f63c00ce0d0 0x7f63c00caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.719+0000 7f63c7925640 1 -- 172.21.15.81:0/1387171208 >> 172.21.15.81:0/1387171208 conn(0x7f63c00cb580 msgr2=0x7f63c00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:39.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.719+0000 7f63c7925640 1 -- 172.21.15.81:0/1387171208 shutdown_connections 2023-11-08T16:29:39.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.719+0000 7f63c7925640 1 -- 172.21.15.81:0/1387171208 wait complete. 2023-11-08T16:29:39.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.720+0000 7f63c7925640 1 Processor -- start 2023-11-08T16:29:39.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.721+0000 7f63c7925640 1 -- start start 2023-11-08T16:29:39.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.721+0000 7f63c7925640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63c00ce0d0 0x7f63c00b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.721+0000 7f63c7925640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c015bfc0 0x7f63c0155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c7925640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63c015f0b0 0x7f63c00c3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c7925640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c01555d0 con 0x7f63c015bfc0 2023-11-08T16:29:39.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c7925640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00c6ca0 con 0x7f63c00ce0d0 2023-11-08T16:29:39.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c7925640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00c3ed0 con 0x7f63c015f0b0 2023-11-08T16:29:39.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c4ec0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c015bfc0 0x7f63c0155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:39.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c4ec0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c015bfc0 0x7f63c0155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45340/0 (socket says 172.21.15.81:45340) 2023-11-08T16:29:39.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c4ec0640 1 -- 172.21.15.81:0/2096350352 learned_addr learned my addr 172.21.15.81:0/2096350352 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:39.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c56c1640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63c00ce0d0 0x7f63c00b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:39.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c4ec0640 1 -- 172.21.15.81:0/2096350352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63c015f0b0 msgr2=0x7f63c00c3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c4ec0640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63c015f0b0 0x7f63c00c3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c4ec0640 1 -- 172.21.15.81:0/2096350352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63c00ce0d0 msgr2=0x7f63c00b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c4ec0640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63c00ce0d0 0x7f63c00b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.722+0000 7f63c4ec0640 1 -- 172.21.15.81:0/2096350352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b00094d0 con 0x7f63c015bfc0 2023-11-08T16:29:39.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.723+0000 7f63c4ec0640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c015bfc0 0x7f63c0155090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f63b0000c00 tx=0x7f63b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:39.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.723+0000 7f63b5ffb640 1 -- 172.21.15.81:0/2096350352 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b003abe0 con 0x7f63c015bfc0 2023-11-08T16:29:39.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.723+0000 7f63b5ffb640 1 -- 172.21.15.81:0/2096350352 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b003ad40 con 0x7f63c015bfc0 2023-11-08T16:29:39.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.723+0000 7f63b5ffb640 1 -- 172.21.15.81:0/2096350352 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0044610 con 0x7f63c015bfc0 2023-11-08T16:29:39.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.723+0000 7f63c7925640 1 -- 172.21.15.81:0/2096350352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c00c41b0 con 0x7f63c015bfc0 2023-11-08T16:29:39.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.723+0000 7f63c7925640 1 -- 172.21.15.81:0/2096350352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c00c6f40 con 0x7f63c015bfc0 2023-11-08T16:29:39.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.724+0000 7f63c7925640 1 -- 172.21.15.81:0/2096350352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c00c46d0 con 0x7f63c015bfc0 2023-11-08T16:29:39.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.725+0000 7f63b5ffb640 1 -- 172.21.15.81:0/2096350352 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f63b004d450 con 0x7f63c015bfc0 2023-11-08T16:29:39.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.726+0000 7f63b5ffb640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63a406aea0 0x7f63a406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:39.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.726+0000 7f63b5ffb640 1 -- 172.21.15.81:0/2096350352 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63b002d040 con 0x7f63c015bfc0 2023-11-08T16:29:39.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.726+0000 7f63c56c1640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63a406aea0 0x7f63a406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:39.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.727+0000 7f63c56c1640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63a406aea0 0x7f63a406d360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63bc006fd0 tx=0x7f63bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:39.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.730+0000 7f63b5ffb640 1 -- 172.21.15.81:0/2096350352 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63c00c46d0 con 0x7f63c015bfc0 2023-11-08T16:29:39.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.921+0000 7f63c7925640 1 -- 172.21.15.81:0/2096350352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63c00c46d0 con 0x7f63c015bfc0 2023-11-08T16:29:39.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.922+0000 7f63b5ffb640 1 -- 172.21.15.81:0/2096350352 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f63c00c46d0 con 0x7f63c015bfc0 2023-11-08T16:29:39.924 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:39.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 -- 172.21.15.81:0/2096350352 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63a406aea0 msgr2=0x7f63a406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63a406aea0 0x7f63a406d360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f63bc006fd0 tx=0x7f63bc008040 comp rx=0 tx=0).stop 2023-11-08T16:29:39.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 -- 172.21.15.81:0/2096350352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c015bfc0 msgr2=0x7f63c0155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:39.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c015bfc0 0x7f63c0155090 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f63b0000c00 tx=0x7f63b00046c0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 -- 172.21.15.81:0/2096350352 shutdown_connections 2023-11-08T16:29:39.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63a406aea0 0x7f63a406d360 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63c015f0b0 0x7f63c00c3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63c015bfc0 0x7f63c0155090 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 --2- 172.21.15.81:0/2096350352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63c00ce0d0 0x7f63c00b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:39.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 -- 172.21.15.81:0/2096350352 >> 172.21.15.81:0/2096350352 conn(0x7f63c00cb580 msgr2=0x7f63c00afa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:39.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 -- 172.21.15.81:0/2096350352 shutdown_connections 2023-11-08T16:29:39.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:39.924+0000 7f63c7925640 1 -- 172.21.15.81:0/2096350352 wait complete. 2023-11-08T16:29:39.946 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:40.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.158+0000 7f2f844c2640 1 Processor -- start 2023-11-08T16:29:40.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.159+0000 7f2f844c2640 1 -- start start 2023-11-08T16:29:40.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.160+0000 7f2f844c2640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2f7c161730 0x7f2f7c0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.160+0000 7f2f844c2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2f7c06d250 0x7f2f7c0c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.160+0000 7f2f844c2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2f7c0c1e50 0x7f2f7c06f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.160+0000 7f2f844c2640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7c0c22e0 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.160+0000 7f2f844c2640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7c06cae0 con 0x7f2f7c161730 2023-11-08T16:29:40.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.160+0000 7f2f844c2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c0cdb00 con 0x7f2f7c06d250 2023-11-08T16:29:40.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.160+0000 7f2f82a5f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2f7c0c1e50 0x7f2f7c06f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.160+0000 7f2f82a5f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2f7c0c1e50 0x7f2f7c06f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45344/0 (socket says 172.21.15.81:45344) 2023-11-08T16:29:40.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.161+0000 7f2f82a5f640 1 -- 172.21.15.81:0/1668406517 learned_addr learned my addr 172.21.15.81:0/1668406517 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:40.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.161+0000 7f2f82a5f640 1 -- 172.21.15.81:0/1668406517 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2f7c06d250 msgr2=0x7f2f7c0c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:40.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.161+0000 7f2f82a5f640 1 --2- 172.21.15.81:0/1668406517 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2f7c06d250 0x7f2f7c0c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.161+0000 7f2f82a5f640 1 -- 172.21.15.81:0/1668406517 >> v2:172.21.15.93:3300/0 conn(0x7f2f7c161730 msgr2=0x7f2f7c0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:40.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.161+0000 7f2f8225e640 1 --2- 172.21.15.81:0/1668406517 >> v2:172.21.15.93:3300/0 conn(0x7f2f7c161730 0x7f2f7c0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.161+0000 7f2f82a5f640 1 --2- 172.21.15.81:0/1668406517 >> v2:172.21.15.93:3300/0 conn(0x7f2f7c161730 0x7f2f7c0cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.161+0000 7f2f82a5f640 1 -- 172.21.15.81:0/1668406517 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c161b10 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.162+0000 7f2f82a5f640 1 --2- 172.21.15.81:0/1668406517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f7c0c1e50 0x7f2f7c06f990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009e50 tx=0x7f2f7402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc37bd2b6decbf54 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:40.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f8125c640 1 -- 172.21.15.81:0/1668406517 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7402ca70 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f8125c640 1 -- 172.21.15.81:0/1668406517 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f7402cbd0 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f8125c640 1 -- 172.21.15.81:0/1668406517 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740343f0 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f844c2640 1 -- 172.21.15.81:0/1668406517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f7c0c1e50 msgr2=0x7f2f7c06f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f844c2640 1 --2- 172.21.15.81:0/1668406517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f7c0c1e50 0x7f2f7c06f990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009e50 tx=0x7f2f7402bed0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f844c2640 1 -- 172.21.15.81:0/1668406517 shutdown_connections 2023-11-08T16:29:40.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f844c2640 1 --2- 172.21.15.81:0/1668406517 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2f7c06d250 0x7f2f7c0c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f844c2640 1 --2- 172.21.15.81:0/1668406517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f7c0c1e50 0x7f2f7c06f990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f844c2640 1 --2- 172.21.15.81:0/1668406517 >> v2:172.21.15.93:3300/0 conn(0x7f2f7c161730 0x7f2f7c0cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f844c2640 1 -- 172.21.15.81:0/1668406517 >> 172.21.15.81:0/1668406517 conn(0x7f2f7c0cb580 msgr2=0x7f2f7c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:40.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f844c2640 1 -- 172.21.15.81:0/1668406517 shutdown_connections 2023-11-08T16:29:40.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.163+0000 7f2f844c2640 1 -- 172.21.15.81:0/1668406517 wait complete. 2023-11-08T16:29:40.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.165+0000 7f2f844c2640 1 Processor -- start 2023-11-08T16:29:40.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.165+0000 7f2f844c2640 1 -- start start 2023-11-08T16:29:40.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.166+0000 7f2f844c2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2f7c06d250 0x7f2f7c153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.166+0000 7f2f844c2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f7c0c1e50 0x7f2f7c0b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.166+0000 7f2f844c2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f7c161730 0x7f2f7c161cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.166+0000 7f2f844c2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c163620 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.166+0000 7f2f844c2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c163be0 con 0x7f2f7c161730 2023-11-08T16:29:40.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.166+0000 7f2f844c2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c0b42e0 con 0x7f2f7c06d250 2023-11-08T16:29:40.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.166+0000 7f2f81a5d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f7c0c1e50 0x7f2f7c0b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.166+0000 7f2f81a5d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f7c0c1e50 0x7f2f7c0b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45346/0 (socket says 172.21.15.81:45346) 2023-11-08T16:29:40.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.167+0000 7f2f81a5d640 1 -- 172.21.15.81:0/2832246474 learned_addr learned my addr 172.21.15.81:0/2832246474 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:40.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.167+0000 7f2f8225e640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2f7c06d250 0x7f2f7c153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.167+0000 7f2f82a5f640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f7c161730 0x7f2f7c161cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.167+0000 7f2f81a5d640 1 -- 172.21.15.81:0/2832246474 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2f7c06d250 msgr2=0x7f2f7c153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.167+0000 7f2f81a5d640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2f7c06d250 0x7f2f7c153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.167+0000 7f2f81a5d640 1 -- 172.21.15.81:0/2832246474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f7c161730 msgr2=0x7f2f7c161cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.167+0000 7f2f81a5d640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f7c161730 0x7f2f7c161cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.167+0000 7f2f81a5d640 1 -- 172.21.15.81:0/2832246474 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f74009b30 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.167+0000 7f2f8225e640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2f7c06d250 0x7f2f7c153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:40.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.167+0000 7f2f81a5d640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f7c0c1e50 0x7f2f7c0b1630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000b500 tx=0x7f2f7000b9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:40.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.167+0000 7f2f6effd640 1 -- 172.21.15.81:0/2832246474 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70010870 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.168+0000 7f2f6effd640 1 -- 172.21.15.81:0/2832246474 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f700109d0 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.168+0000 7f2f6effd640 1 -- 172.21.15.81:0/2832246474 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70010d80 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.168+0000 7f2f844c2640 1 -- 172.21.15.81:0/2832246474 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c1628d0 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.168+0000 7f2f844c2640 1 -- 172.21.15.81:0/2832246474 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c163870 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.169+0000 7f2f844c2640 1 -- 172.21.15.81:0/2832246474 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7c04ece0 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.170+0000 7f2f6effd640 1 -- 172.21.15.81:0/2832246474 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f7000f3c0 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.171+0000 7f2f6effd640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f6006ae70 0x7f2f6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.171+0000 7f2f6effd640 1 -- 172.21.15.81:0/2832246474 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f70089690 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.171+0000 7f2f8225e640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f6006ae70 0x7f2f6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.172+0000 7f2f8225e640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f6006ae70 0x7f2f6006d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800b820 tx=0x7f2f78008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:40.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.175+0000 7f2f6effd640 1 -- 172.21.15.81:0/2832246474 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2f700546b0 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.363+0000 7f2f844c2640 1 -- 172.21.15.81:0/2832246474 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f7c0c1c30 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.364+0000 7f2f6effd640 1 -- 172.21.15.81:0/2832246474 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f70057cd0 con 0x7f2f7c0c1e50 2023-11-08T16:29:40.366 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:40.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.367+0000 7f2f844c2640 1 -- 172.21.15.81:0/2832246474 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f6006ae70 msgr2=0x7f2f6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.367+0000 7f2f844c2640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f6006ae70 0x7f2f6006d330 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2f7800b820 tx=0x7f2f78008040 comp rx=0 tx=0).stop 2023-11-08T16:29:40.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.367+0000 7f2f844c2640 1 -- 172.21.15.81:0/2832246474 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f7c0c1e50 msgr2=0x7f2f7c0b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.367+0000 7f2f844c2640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f7c0c1e50 0x7f2f7c0b1630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2f7000b500 tx=0x7f2f7000b9c0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.367+0000 7f2f844c2640 1 -- 172.21.15.81:0/2832246474 shutdown_connections 2023-11-08T16:29:40.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.367+0000 7f2f844c2640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f6006ae70 0x7f2f6006d330 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.367+0000 7f2f844c2640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2f7c161730 0x7f2f7c161cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.367+0000 7f2f844c2640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2f7c0c1e50 0x7f2f7c0b1630 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.367+0000 7f2f844c2640 1 --2- 172.21.15.81:0/2832246474 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2f7c06d250 0x7f2f7c153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.367+0000 7f2f844c2640 1 -- 172.21.15.81:0/2832246474 >> 172.21.15.81:0/2832246474 conn(0x7f2f7c0cb580 msgr2=0x7f2f7c06f970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:40.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.368+0000 7f2f844c2640 1 -- 172.21.15.81:0/2832246474 shutdown_connections 2023-11-08T16:29:40.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.368+0000 7f2f844c2640 1 -- 172.21.15.81:0/2832246474 wait complete. 2023-11-08T16:29:40.387 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:40.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.605+0000 7f16a2b47640 1 Processor -- start 2023-11-08T16:29:40.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.606+0000 7f16a2b47640 1 -- start start 2023-11-08T16:29:40.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.607+0000 7f16a2b47640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f169c0cfd90 0x7f169c0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.607+0000 7f16a2b47640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169c0ce0b0 con 0x7f169c15cf30 2023-11-08T16:29:40.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.607+0000 7f16a2b47640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f169c0ce390 con 0x7f169c15d310 2023-11-08T16:29:40.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.607+0000 7f16a2b47640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c0d1be0 con 0x7f169c0cfd90 2023-11-08T16:29:40.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.608+0000 7f16a1b45640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f169c15cf30 0x7f169c0cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38004/0 (socket says 172.21.15.81:38004) 2023-11-08T16:29:40.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.608+0000 7f16a1b45640 1 -- 172.21.15.81:0/3953593130 learned_addr learned my addr 172.21.15.81:0/3953593130 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:40.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.608+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4226643759 0 0) 0x7f169c0ce0b0 con 0x7f169c15cf30 2023-11-08T16:29:40.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.608+0000 7f16a2346640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f169c0cfd90 0x7f169c0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.608+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1688003520 con 0x7f169c15cf30 2023-11-08T16:29:40.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.608+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 355756458 0 0) 0x7f1688003520 con 0x7f169c15cf30 2023-11-08T16:29:40.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.608+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f169c0cfd90 msgr2=0x7f169c0d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:40.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.608+0000 7f16a0b43640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f169c0cfd90 0x7f169c0d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.608+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 >> v1:172.21.15.93:6789/0 conn(0x7f169c15d310 legacy=0x7f169c0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.608+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c0d1ec0 con 0x7f169c15cf30 2023-11-08T16:29:40.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.608+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f1698001670 con 0x7f169c15cf30 2023-11-08T16:29:40.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 >> v1:172.21.15.81:6789/0 conn(0x7f169c15cf30 legacy=0x7f169c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a0b43640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f16880039a0 0x7f1688003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a0b43640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 0x7f1688004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a0b43640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1688005180 0x7f168800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c0d1be0 con 0x7f16880047c0 2023-11-08T16:29:40.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f168800c080 con 0x7f1688005180 2023-11-08T16:29:40.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f168800c240 con 0x7f16880039a0 2023-11-08T16:29:40.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a2346640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f169c0cfd90 0x7f169c0d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:40.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a1344640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 0x7f1688004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a1b45640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f16880039a0 0x7f1688003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a2346640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1688005180 0x7f168800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a1344640 1 -- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f16880039a0 msgr2=0x7f1688003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:40.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a1344640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f16880039a0 0x7f1688003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.609+0000 7f16a1344640 1 -- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1688005180 msgr2=0x7f168800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:40.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.610+0000 7f16a1344640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1688005180 0x7f168800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.610+0000 7f16a1344640 1 -- 172.21.15.81:0/3953593130 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1688003520 con 0x7f16880047c0 2023-11-08T16:29:40.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.610+0000 7f16a2346640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1688005180 0x7f168800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:40.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.611+0000 7f16a1344640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 0x7f1688004c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f169c0cd2b0 tx=0x7f16880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be46d4c53f5d61f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:40.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.611+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c0052d0 con 0x7f16880047c0 2023-11-08T16:29:40.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.611+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c02e050 con 0x7f16880047c0 2023-11-08T16:29:40.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.611+0000 7f16a0b43640 1 -- 172.21.15.81:0/3953593130 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c003830 con 0x7f16880047c0 2023-11-08T16:29:40.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.612+0000 7f16a2b47640 1 -- 172.21.15.81:0/3953593130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 msgr2=0x7f1688004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.612+0000 7f16a2b47640 1 --2- 172.21.15.81:0/3953593130 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 0x7f1688004c40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f169c0cd2b0 tx=0x7f16880057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.612+0000 7f16a2346640 1 -- 172.21.15.81:0/3953593130 reap_dead start 2023-11-08T16:29:40.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.612+0000 7f16a2b47640 1 -- 172.21.15.81:0/3953593130 shutdown_connections 2023-11-08T16:29:40.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.612+0000 7f16a2b47640 1 -- 172.21.15.81:0/3953593130 >> 172.21.15.81:0/3953593130 conn(0x7f169c06c040 msgr2=0x7f169c06c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:40.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.612+0000 7f16a2b47640 1 -- 172.21.15.81:0/3953593130 shutdown_connections 2023-11-08T16:29:40.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.612+0000 7f16a2b47640 1 -- 172.21.15.81:0/3953593130 wait complete. 2023-11-08T16:29:40.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.614+0000 7f16a2b47640 1 Processor -- start 2023-11-08T16:29:40.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.614+0000 7f16a2b47640 1 -- start start 2023-11-08T16:29:40.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a2b47640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 0x7f169c1f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a2b47640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f169c1f3470 0x7f169c0ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a2b47640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f169c0ac650 0x7f169c0cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a2b47640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c0acad0 con 0x7f16880047c0 2023-11-08T16:29:40.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a2b47640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f169c0cbda0 con 0x7f169c0ac650 2023-11-08T16:29:40.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a2b47640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f169c0cc080 con 0x7f169c1f3470 2023-11-08T16:29:40.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a1b45640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 0x7f169c1f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a1b45640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 0x7f169c1f2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45362/0 (socket says 172.21.15.81:45362) 2023-11-08T16:29:40.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a1b45640 1 -- 172.21.15.81:0/1572032626 learned_addr learned my addr 172.21.15.81:0/1572032626 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:40.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a1b45640 1 -- 172.21.15.81:0/1572032626 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f169c1f3470 msgr2=0x7f169c0ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a1b45640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f169c1f3470 0x7f169c0ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a1b45640 1 -- 172.21.15.81:0/1572032626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f169c0ac650 msgr2=0x7f169c0cb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a1b45640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f169c0ac650 0x7f169c0cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.615+0000 7f16a1b45640 1 -- 172.21.15.81:0/1572032626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c0cc360 con 0x7f16880047c0 2023-11-08T16:29:40.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.616+0000 7f16a1b45640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 0x7f169c1f2f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f169800d7a0 tx=0x7f169800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:40.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.616+0000 7f16927fc640 1 -- 172.21.15.81:0/1572032626 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16980074e0 con 0x7f16880047c0 2023-11-08T16:29:40.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.616+0000 7f16927fc640 1 -- 172.21.15.81:0/1572032626 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1698007640 con 0x7f16880047c0 2023-11-08T16:29:40.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.616+0000 7f16927fc640 1 -- 172.21.15.81:0/1572032626 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698004a20 con 0x7f16880047c0 2023-11-08T16:29:40.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.616+0000 7f16a2b47640 1 -- 172.21.15.81:0/1572032626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f169c0cc640 con 0x7f16880047c0 2023-11-08T16:29:40.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.616+0000 7f16a2b47640 1 -- 172.21.15.81:0/1572032626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f169c0cb5b0 con 0x7f16880047c0 2023-11-08T16:29:40.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.617+0000 7f16a2b47640 1 -- 172.21.15.81:0/1572032626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f169c1f3c20 con 0x7f16880047c0 2023-11-08T16:29:40.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.618+0000 7f16927fc640 1 -- 172.21.15.81:0/1572032626 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1698004b80 con 0x7f16880047c0 2023-11-08T16:29:40.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.619+0000 7f16927fc640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f168006aea0 0x7f168006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:40.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.619+0000 7f16927fc640 1 -- 172.21.15.81:0/1572032626 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1698088eb0 con 0x7f16880047c0 2023-11-08T16:29:40.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.620+0000 7f16a1344640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f168006aea0 0x7f168006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:40.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.620+0000 7f16a1344640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f168006aea0 0x7f168006d360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f168c033000 tx=0x7f168c02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:40.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.624+0000 7f16927fc640 1 -- 172.21.15.81:0/1572032626 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f169c1f3c20 con 0x7f16880047c0 2023-11-08T16:29:40.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.814+0000 7f16a2b47640 1 -- 172.21.15.81:0/1572032626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f169c1f3c20 con 0x7f16880047c0 2023-11-08T16:29:40.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.816+0000 7f16927fc640 1 -- 172.21.15.81:0/1572032626 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f169c1f3c20 con 0x7f16880047c0 2023-11-08T16:29:40.818 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:40.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.818+0000 7f16a2b47640 1 -- 172.21.15.81:0/1572032626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f168006aea0 msgr2=0x7f168006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.818+0000 7f16a2b47640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f168006aea0 0x7f168006d360 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f168c033000 tx=0x7f168c02f000 comp rx=0 tx=0).stop 2023-11-08T16:29:40.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.818+0000 7f16a2b47640 1 -- 172.21.15.81:0/1572032626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 msgr2=0x7f169c1f2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:40.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.818+0000 7f16a2b47640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 0x7f169c1f2f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f169800d7a0 tx=0x7f169800dc60 comp rx=0 tx=0).stop 2023-11-08T16:29:40.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.819+0000 7f16a2b47640 1 -- 172.21.15.81:0/1572032626 shutdown_connections 2023-11-08T16:29:40.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.819+0000 7f16a2b47640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f168006aea0 0x7f168006d360 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.819+0000 7f16a2b47640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f169c0ac650 0x7f169c0cb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.819+0000 7f16a2b47640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f169c1f3470 0x7f169c0ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.819+0000 7f16a2b47640 1 --2- 172.21.15.81:0/1572032626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16880047c0 0x7f169c1f2f30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:40.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.819+0000 7f16a2b47640 1 -- 172.21.15.81:0/1572032626 >> 172.21.15.81:0/1572032626 conn(0x7f169c06c040 msgr2=0x7f169c15e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:40.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.819+0000 7f16a2b47640 1 -- 172.21.15.81:0/1572032626 shutdown_connections 2023-11-08T16:29:40.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:40.819+0000 7f16a2b47640 1 -- 172.21.15.81:0/1572032626 wait complete. 2023-11-08T16:29:40.838 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:41.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.028+0000 7f6ee359e640 1 Processor -- start 2023-11-08T16:29:41.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.028+0000 7f6ee359e640 1 -- start start 2023-11-08T16:29:41.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.030+0000 7f6ee359e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6ee415ab70 0x7f6ee41613e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.030+0000 7f6ee359e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ee40cf170 0x7f6ee40acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.030+0000 7f6ee359e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee406b860 con 0x7f6ee40ced10 2023-11-08T16:29:41.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.030+0000 7f6ee359e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee415e450 con 0x7f6ee40cf170 2023-11-08T16:29:41.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.030+0000 7f6ee359e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee4161920 con 0x7f6ee415ab70 2023-11-08T16:29:41.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.030+0000 7f6ee2d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ee40cf170 0x7f6ee40acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.030+0000 7f6ee2d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ee40cf170 0x7f6ee40acfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55614/0 (socket says 172.21.15.81:55614) 2023-11-08T16:29:41.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.030+0000 7f6ee2d9d640 1 -- 172.21.15.81:0/908549560 learned_addr learned my addr 172.21.15.81:0/908549560 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:41.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.030+0000 7f6edbfff640 1 --1- 172.21.15.81:0/908549560 >> v1:172.21.15.81:6789/0 conn(0x7f6ee40ced10 0x7f6ee40ac8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35920/0 (socket says 172.21.15.81:35920) 2023-11-08T16:29:41.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.031+0000 7f6ee259c640 1 --2- 172.21.15.81:0/908549560 >> v2:172.21.15.93:3300/0 conn(0x7f6ee415ab70 0x7f6ee41613e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.031+0000 7f6ee2d9d640 1 -- 172.21.15.81:0/908549560 >> v2:172.21.15.93:3300/0 conn(0x7f6ee415ab70 msgr2=0x7f6ee41613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:41.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.031+0000 7f6ee2d9d640 1 --2- 172.21.15.81:0/908549560 >> v2:172.21.15.93:3300/0 conn(0x7f6ee415ab70 0x7f6ee41613e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.031+0000 7f6ee2d9d640 1 -- 172.21.15.81:0/908549560 >> v1:172.21.15.81:6789/0 conn(0x7f6ee40ced10 legacy=0x7f6ee40ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.031+0000 7f6ee2d9d640 1 -- 172.21.15.81:0/908549560 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee415d840 con 0x7f6ee40cf170 2023-11-08T16:29:41.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.032+0000 7f6ee2d9d640 1 --2- 172.21.15.81:0/908549560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ee40cf170 0x7f6ee40acfb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009e50 tx=0x7f6edc02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1808b071e4cdb142 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:41.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.032+0000 7f6ee1d9b640 1 -- 172.21.15.81:0/908549560 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc02cb10 con 0x7f6ee40cf170 2023-11-08T16:29:41.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.032+0000 7f6ee1d9b640 1 -- 172.21.15.81:0/908549560 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc02cc70 con 0x7f6ee40cf170 2023-11-08T16:29:41.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.032+0000 7f6ee1d9b640 1 -- 172.21.15.81:0/908549560 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0343f0 con 0x7f6ee40cf170 2023-11-08T16:29:41.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.033+0000 7f6ee359e640 1 -- 172.21.15.81:0/908549560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ee40cf170 msgr2=0x7f6ee40acfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.033+0000 7f6ee359e640 1 --2- 172.21.15.81:0/908549560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ee40cf170 0x7f6ee40acfb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009e50 tx=0x7f6edc02bf70 comp rx=0 tx=0).stop 2023-11-08T16:29:41.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.033+0000 7f6ee359e640 1 -- 172.21.15.81:0/908549560 shutdown_connections 2023-11-08T16:29:41.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.033+0000 7f6ee359e640 1 --2- 172.21.15.81:0/908549560 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ee40cf170 0x7f6ee40acfb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.033+0000 7f6ee359e640 1 --2- 172.21.15.81:0/908549560 >> v2:172.21.15.93:3300/0 conn(0x7f6ee415ab70 0x7f6ee41613e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.033+0000 7f6ee359e640 1 -- 172.21.15.81:0/908549560 >> 172.21.15.81:0/908549560 conn(0x7f6ee406a4a0 msgr2=0x7f6ee406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:41.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.033+0000 7f6ee359e640 1 -- 172.21.15.81:0/908549560 shutdown_connections 2023-11-08T16:29:41.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.033+0000 7f6ee359e640 1 -- 172.21.15.81:0/908549560 wait complete. 2023-11-08T16:29:41.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.035+0000 7f6ee359e640 1 Processor -- start 2023-11-08T16:29:41.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.035+0000 7f6ee359e640 1 -- start start 2023-11-08T16:29:41.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee359e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ee40ced10 0x7f6ee4154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee359e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40cf170 0x7f6ee41f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ee415ab70 0x7f6ee4154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee359e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee415ccd0 con 0x7f6ee415ab70 2023-11-08T16:29:41.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee359e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee41f3730 con 0x7f6ee40cf170 2023-11-08T16:29:41.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee359e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee41f3a10 con 0x7f6ee40ced10 2023-11-08T16:29:41.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ee415ab70 0x7f6ee4154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ee415ab70 0x7f6ee4154f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55618/0 (socket says 172.21.15.81:55618) 2023-11-08T16:29:41.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee2d9d640 1 -- 172.21.15.81:0/2605988108 learned_addr learned my addr 172.21.15.81:0/2605988108 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:41.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee2d9d640 1 -- 172.21.15.81:0/2605988108 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ee40ced10 msgr2=0x7f6ee4154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee2d9d640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ee40ced10 0x7f6ee4154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee2d9d640 1 -- 172.21.15.81:0/2605988108 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40cf170 msgr2=0x7f6ee41f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee2d9d640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40cf170 0x7f6ee41f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.036+0000 7f6ee2d9d640 1 -- 172.21.15.81:0/2605988108 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc009b30 con 0x7f6ee415ab70 2023-11-08T16:29:41.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.037+0000 7f6ee2d9d640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ee415ab70 0x7f6ee4154f60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009e50 tx=0x7f6edc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:41.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.037+0000 7f6edaffd640 1 -- 172.21.15.81:0/2605988108 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc009510 con 0x7f6ee415ab70 2023-11-08T16:29:41.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.037+0000 7f6edaffd640 1 -- 172.21.15.81:0/2605988108 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc03e450 con 0x7f6ee415ab70 2023-11-08T16:29:41.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.037+0000 7f6ee359e640 1 -- 172.21.15.81:0/2605988108 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee41566a0 con 0x7f6ee415ab70 2023-11-08T16:29:41.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.037+0000 7f6edaffd640 1 -- 172.21.15.81:0/2605988108 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc03e800 con 0x7f6ee415ab70 2023-11-08T16:29:41.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.037+0000 7f6ee359e640 1 -- 172.21.15.81:0/2605988108 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee4156c60 con 0x7f6ee415ab70 2023-11-08T16:29:41.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.038+0000 7f6ee359e640 1 -- 172.21.15.81:0/2605988108 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee4156e50 con 0x7f6ee415ab70 2023-11-08T16:29:41.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.039+0000 7f6edaffd640 1 -- 172.21.15.81:0/2605988108 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6edc03e960 con 0x7f6ee415ab70 2023-11-08T16:29:41.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.040+0000 7f6edaffd640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ec406aea0 0x7f6ec406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.040+0000 7f6edaffd640 1 -- 172.21.15.81:0/2605988108 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6edc0ac3c0 con 0x7f6ee415ab70 2023-11-08T16:29:41.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.040+0000 7f6ee259c640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ec406aea0 0x7f6ec406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.041+0000 7f6ee259c640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ec406aea0 0x7f6ec406d360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4006fd0 tx=0x7f6ed4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:41.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.044+0000 7f6edaffd640 1 -- 172.21.15.81:0/2605988108 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6edc077460 con 0x7f6ee415ab70 2023-11-08T16:29:41.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.233+0000 7f6ee359e640 1 -- 172.21.15.81:0/2605988108 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ee4067800 con 0x7f6ee415ab70 2023-11-08T16:29:41.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.236+0000 7f6edaffd640 1 -- 172.21.15.81:0/2605988108 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f6edc07aa80 con 0x7f6ee415ab70 2023-11-08T16:29:41.239 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:41.239 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":21,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:04.727022+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:41.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 -- 172.21.15.81:0/2605988108 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ec406aea0 msgr2=0x7f6ec406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ec406aea0 0x7f6ec406d360 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4006fd0 tx=0x7f6ed4008040 comp rx=0 tx=0).stop 2023-11-08T16:29:41.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 -- 172.21.15.81:0/2605988108 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ee415ab70 msgr2=0x7f6ee4154f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ee415ab70 0x7f6ee4154f60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009e50 tx=0x7f6edc004a40 comp rx=0 tx=0).stop 2023-11-08T16:29:41.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 -- 172.21.15.81:0/2605988108 shutdown_connections 2023-11-08T16:29:41.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ec406aea0 0x7f6ec406d360 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ee415ab70 0x7f6ee4154f60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ee40cf170 0x7f6ee41f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 --2- 172.21.15.81:0/2605988108 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ee40ced10 0x7f6ee4154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 -- 172.21.15.81:0/2605988108 >> 172.21.15.81:0/2605988108 conn(0x7f6ee406a4a0 msgr2=0x7f6ee40cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:41.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 -- 172.21.15.81:0/2605988108 shutdown_connections 2023-11-08T16:29:41.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.240+0000 7f6ee359e640 1 -- 172.21.15.81:0/2605988108 wait complete. 2023-11-08T16:29:41.260 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:41.261 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:41.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.456+0000 7fa19fa6f640 1 Processor -- start 2023-11-08T16:29:41.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.456+0000 7fa19fa6f640 1 -- start start 2023-11-08T16:29:41.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.458+0000 7fa19fa6f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa19815f280 0x7fa1980caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.458+0000 7fa19fa6f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1980cda10 con 0x7fa19815bfc0 2023-11-08T16:29:41.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.458+0000 7fa19fa6f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa19806fa30 con 0x7fa1980ce0d0 2023-11-08T16:29:41.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.458+0000 7fa19fa6f640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1980ce4b0 con 0x7fa19815f280 2023-11-08T16:29:41.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.458+0000 7fa19e00c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa19815f280 0x7fa1980caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.458+0000 7fa19e00c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa19815f280 0x7fa1980caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55628/0 (socket says 172.21.15.81:55628) 2023-11-08T16:29:41.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.458+0000 7fa19e00c640 1 -- 172.21.15.81:0/3504532747 learned_addr learned my addr 172.21.15.81:0/3504532747 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:41.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.458+0000 7fa19d00a640 1 --1- 172.21.15.81:0/3504532747 >> v1:172.21.15.81:6789/0 conn(0x7fa19815bfc0 0x7fa1981516e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35926/0 (socket says 172.21.15.81:35926) 2023-11-08T16:29:41.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.459+0000 7fa19e00c640 1 -- 172.21.15.81:0/3504532747 >> v1:172.21.15.93:6789/0 conn(0x7fa1980ce0d0 legacy=0x7fa19815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:41.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.459+0000 7fa19e00c640 1 -- 172.21.15.81:0/3504532747 >> v1:172.21.15.81:6789/0 conn(0x7fa19815bfc0 legacy=0x7fa1981516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.459+0000 7fa19e00c640 1 -- 172.21.15.81:0/3504532747 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19806fca0 con 0x7fa19815f280 2023-11-08T16:29:41.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.459+0000 7fa19c809640 1 -- 172.21.15.81:0/3504532747 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677207516 0 0) 0x7fa1980cda10 con 0x7fa19815bfc0 2023-11-08T16:29:41.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.460+0000 7fa19e00c640 1 --2- 172.21.15.81:0/3504532747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa19815f280 0x7fa1980caf20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa190009e50 tx=0x7fa19002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b742b4561b92ec07 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:41.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.460+0000 7fa19c809640 1 -- 172.21.15.81:0/3504532747 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19002cb20 con 0x7fa19815f280 2023-11-08T16:29:41.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.460+0000 7fa19c809640 1 -- 172.21.15.81:0/3504532747 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19002cc80 con 0x7fa19815f280 2023-11-08T16:29:41.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.460+0000 7fa19c809640 1 -- 172.21.15.81:0/3504532747 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1900343f0 con 0x7fa19815f280 2023-11-08T16:29:41.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.461+0000 7fa19fa6f640 1 -- 172.21.15.81:0/3504532747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa19815f280 msgr2=0x7fa1980caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.461+0000 7fa19fa6f640 1 --2- 172.21.15.81:0/3504532747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa19815f280 0x7fa1980caf20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa190009e50 tx=0x7fa19002bf80 comp rx=0 tx=0).stop 2023-11-08T16:29:41.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.461+0000 7fa19fa6f640 1 -- 172.21.15.81:0/3504532747 shutdown_connections 2023-11-08T16:29:41.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.461+0000 7fa19fa6f640 1 --2- 172.21.15.81:0/3504532747 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa19815f280 0x7fa1980caf20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.461+0000 7fa19fa6f640 1 -- 172.21.15.81:0/3504532747 >> 172.21.15.81:0/3504532747 conn(0x7fa1980cb580 msgr2=0x7fa1980cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:41.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.461+0000 7fa19fa6f640 1 -- 172.21.15.81:0/3504532747 shutdown_connections 2023-11-08T16:29:41.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.461+0000 7fa19fa6f640 1 -- 172.21.15.81:0/3504532747 wait complete. 2023-11-08T16:29:41.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.463+0000 7fa19fa6f640 1 Processor -- start 2023-11-08T16:29:41.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.463+0000 7fa19fa6f640 1 -- start start 2023-11-08T16:29:41.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.464+0000 7fa19fa6f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1980ce0d0 0x7fa198154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.464+0000 7fa19fa6f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa19815bfc0 0x7fa1980c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.464+0000 7fa19fa6f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa19815f280 0x7fa1980c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.464+0000 7fa19fa6f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa19815d990 con 0x7fa1980ce0d0 2023-11-08T16:29:41.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.464+0000 7fa19fa6f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1981549d0 con 0x7fa19815f280 2023-11-08T16:29:41.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.464+0000 7fa19fa6f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa19806ba80 con 0x7fa19815bfc0 2023-11-08T16:29:41.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.465+0000 7fa19d80b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1980ce0d0 0x7fa198154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.465+0000 7fa19d80b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1980ce0d0 0x7fa198154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55630/0 (socket says 172.21.15.81:55630) 2023-11-08T16:29:41.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.465+0000 7fa19d80b640 1 -- 172.21.15.81:0/227418594 learned_addr learned my addr 172.21.15.81:0/227418594 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:41.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.465+0000 7fa19e00c640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa19815f280 0x7fa1980c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.465+0000 7fa19d80b640 1 -- 172.21.15.81:0/227418594 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa19815bfc0 msgr2=0x7fa1980c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.465+0000 7fa19d80b640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa19815bfc0 0x7fa1980c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.465+0000 7fa19d80b640 1 -- 172.21.15.81:0/227418594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa19815f280 msgr2=0x7fa1980c4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.465+0000 7fa19d80b640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa19815f280 0x7fa1980c4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.465+0000 7fa19d80b640 1 -- 172.21.15.81:0/227418594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa190009b30 con 0x7fa1980ce0d0 2023-11-08T16:29:41.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.465+0000 7fa19d80b640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1980ce0d0 0x7fa198154490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa19400b3f0 tx=0x7fa19400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:41.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.465+0000 7fa18e7fc640 1 -- 172.21.15.81:0/227418594 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194011830 con 0x7fa1980ce0d0 2023-11-08T16:29:41.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.466+0000 7fa18e7fc640 1 -- 172.21.15.81:0/227418594 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa194011990 con 0x7fa1980ce0d0 2023-11-08T16:29:41.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.466+0000 7fa18e7fc640 1 -- 172.21.15.81:0/227418594 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa194011d40 con 0x7fa1980ce0d0 2023-11-08T16:29:41.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.466+0000 7fa19fa6f640 1 -- 172.21.15.81:0/227418594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa19806bd60 con 0x7fa1980ce0d0 2023-11-08T16:29:41.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.466+0000 7fa19fa6f640 1 -- 172.21.15.81:0/227418594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19806c320 con 0x7fa1980ce0d0 2023-11-08T16:29:41.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.467+0000 7fa19fa6f640 1 -- 172.21.15.81:0/227418594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19804ece0 con 0x7fa1980ce0d0 2023-11-08T16:29:41.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.468+0000 7fa18e7fc640 1 -- 172.21.15.81:0/227418594 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa19401a400 con 0x7fa1980ce0d0 2023-11-08T16:29:41.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.468+0000 7fa18e7fc640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa17c06ae70 0x7fa17c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.468+0000 7fa18e7fc640 1 -- 172.21.15.81:0/227418594 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1940894c0 con 0x7fa1980ce0d0 2023-11-08T16:29:41.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.469+0000 7fa19d00a640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa17c06ae70 0x7fa17c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.469+0000 7fa19d00a640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa17c06ae70 0x7fa17c06d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa188007fd0 tx=0x7fa188007920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:41.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.473+0000 7fa18e7fc640 1 -- 172.21.15.81:0/227418594 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa194054570 con 0x7fa1980ce0d0 2023-11-08T16:29:41.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.659+0000 7fa19fa6f640 1 -- 172.21.15.81:0/227418594 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1981557b0 con 0x7fa1980ce0d0 2023-11-08T16:29:41.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.660+0000 7fa18e7fc640 1 -- 172.21.15.81:0/227418594 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa194057b90 con 0x7fa1980ce0d0 2023-11-08T16:29:41.662 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:41.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 -- 172.21.15.81:0/227418594 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa17c06ae70 msgr2=0x7fa17c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa17c06ae70 0x7fa17c06d330 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fa188007fd0 tx=0x7fa188007920 comp rx=0 tx=0).stop 2023-11-08T16:29:41.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 -- 172.21.15.81:0/227418594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1980ce0d0 msgr2=0x7fa198154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1980ce0d0 0x7fa198154490 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa19400b3f0 tx=0x7fa19400b8b0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 -- 172.21.15.81:0/227418594 shutdown_connections 2023-11-08T16:29:41.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa17c06ae70 0x7fa17c06d330 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa19815f280 0x7fa1980c4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa19815bfc0 0x7fa1980c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 --2- 172.21.15.81:0/227418594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1980ce0d0 0x7fa198154490 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 -- 172.21.15.81:0/227418594 >> 172.21.15.81:0/227418594 conn(0x7fa1980cb580 msgr2=0x7fa1980b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:41.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 -- 172.21.15.81:0/227418594 shutdown_connections 2023-11-08T16:29:41.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.662+0000 7fa19fa6f640 1 -- 172.21.15.81:0/227418594 wait complete. 2023-11-08T16:29:41.682 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:41.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.892+0000 7ff9c0234640 1 Processor -- start 2023-11-08T16:29:41.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.892+0000 7ff9c0234640 1 -- start start 2023-11-08T16:29:41.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.894+0000 7ff9c0234640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9b80ccc10 0x7ff9b80ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.894+0000 7ff9c0234640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b80ce960 con 0x7ff9b815a0e0 2023-11-08T16:29:41.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.894+0000 7ff9c0234640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b80c9480 con 0x7ff9b806b710 2023-11-08T16:29:41.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.894+0000 7ff9c0234640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8071980 con 0x7ff9b80ccc10 2023-11-08T16:29:41.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.894+0000 7ff9bdfd0640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff9b815a0e0 0x7ff9b80c9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35938/0 (socket says 172.21.15.81:35938) 2023-11-08T16:29:41.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.894+0000 7ff9bdfd0640 1 -- 172.21.15.81:0/3280816547 learned_addr learned my addr 172.21.15.81:0/3280816547 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:41.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bd7cf640 1 --2- 172.21.15.81:0/3280816547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9b80ccc10 0x7ff9b80ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021032821 0 0) 0x7ff9b4001490 con 0x7ff9b815a0e0 2023-11-08T16:29:41.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9a4003520 con 0x7ff9b815a0e0 2023-11-08T16:29:41.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3554748452 0 0) 0x7ff9a4003520 con 0x7ff9b815a0e0 2023-11-08T16:29:41.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9b80ccc10 msgr2=0x7ff9b80ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:41.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 --2- 172.21.15.81:0/3280816547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9b80ccc10 0x7ff9b80ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 >> v1:172.21.15.93:6789/0 conn(0x7ff9b806b710 legacy=0x7ff9b806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b815a4c0 con 0x7ff9b815a0e0 2023-11-08T16:29:41.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ff9b4001b80 con 0x7ff9b815a0e0 2023-11-08T16:29:41.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 >> v1:172.21.15.81:6789/0 conn(0x7ff9b815a0e0 legacy=0x7ff9b80c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 --2- 172.21.15.81:0/3280816547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9a4003a10 0x7ff9a4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 --2- 172.21.15.81:0/3280816547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 0x7ff9a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 --2- 172.21.15.81:0/3280816547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9b80ccc10 0x7ff9a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8071980 con 0x7ff9a40047c0 2023-11-08T16:29:41.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a400bb10 con 0x7ff9a4003a10 2023-11-08T16:29:41.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.895+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a400dee0 con 0x7ff9b80ccc10 2023-11-08T16:29:41.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.896+0000 7ff9bd7cf640 1 --2- 172.21.15.81:0/3280816547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 0x7ff9a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.896+0000 7ff9bdfd0640 1 --2- 172.21.15.81:0/3280816547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9a4003a10 0x7ff9a4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.896+0000 7ff9bd7cf640 1 -- 172.21.15.81:0/3280816547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9b80ccc10 msgr2=0x7ff9a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:41.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.896+0000 7ff9bd7cf640 1 --2- 172.21.15.81:0/3280816547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9b80ccc10 0x7ff9a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.896+0000 7ff9bd7cf640 1 -- 172.21.15.81:0/3280816547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9a4003a10 msgr2=0x7ff9a4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:41.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.896+0000 7ff9bd7cf640 1 --2- 172.21.15.81:0/3280816547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9a4003a10 0x7ff9a4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.896+0000 7ff9bd7cf640 1 -- 172.21.15.81:0/3280816547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a4003520 con 0x7ff9a40047c0 2023-11-08T16:29:41.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.897+0000 7ff9bd7cf640 1 --2- 172.21.15.81:0/3280816547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 0x7ff9a4004c40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009ac0 tx=0x7ff9a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed474e4d93b2106d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:41.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.897+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a802c710 con 0x7ff9a40047c0 2023-11-08T16:29:41.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.897+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a802c870 con 0x7ff9a40047c0 2023-11-08T16:29:41.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.897+0000 7ff9bcfce640 1 -- 172.21.15.81:0/3280816547 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a802cc00 con 0x7ff9a40047c0 2023-11-08T16:29:41.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.898+0000 7ff9c0234640 1 -- 172.21.15.81:0/3280816547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 msgr2=0x7ff9a4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.898+0000 7ff9c0234640 1 --2- 172.21.15.81:0/3280816547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 0x7ff9a4004c40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8009ac0 tx=0x7ff9a40057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.898+0000 7ff9be7d1640 1 -- 172.21.15.81:0/3280816547 reap_dead start 2023-11-08T16:29:41.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.898+0000 7ff9c0234640 1 -- 172.21.15.81:0/3280816547 shutdown_connections 2023-11-08T16:29:41.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.898+0000 7ff9c0234640 1 -- 172.21.15.81:0/3280816547 >> 172.21.15.81:0/3280816547 conn(0x7ff9b80cb580 msgr2=0x7ff9b80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:41.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.898+0000 7ff9c0234640 1 -- 172.21.15.81:0/3280816547 shutdown_connections 2023-11-08T16:29:41.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.898+0000 7ff9c0234640 1 -- 172.21.15.81:0/3280816547 wait complete. 2023-11-08T16:29:41.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.900+0000 7ff9c0234640 1 Processor -- start 2023-11-08T16:29:41.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.900+0000 7ff9c0234640 1 -- start start 2023-11-08T16:29:41.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9c0234640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 0x7ff9b81f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9c0234640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9b80ac8b0 0x7ff9b80acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9c0234640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9b81f6920 0x7ff9b81f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9c0234640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8143140 con 0x7ff9a40047c0 2023-11-08T16:29:41.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9c0234640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b8156560 con 0x7ff9b81f6920 2023-11-08T16:29:41.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9c0234640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b80af280 con 0x7ff9b80ac8b0 2023-11-08T16:29:41.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9bdfd0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 0x7ff9b81f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9bdfd0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 0x7ff9b81f2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55656/0 (socket says 172.21.15.81:55656) 2023-11-08T16:29:41.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9bdfd0640 1 -- 172.21.15.81:0/3275126122 learned_addr learned my addr 172.21.15.81:0/3275126122 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:41.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9bdfd0640 1 -- 172.21.15.81:0/3275126122 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9b80ac8b0 msgr2=0x7ff9b80acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9bdfd0640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9b80ac8b0 0x7ff9b80acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9bdfd0640 1 -- 172.21.15.81:0/3275126122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9b81f6920 msgr2=0x7ff9b81f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:41.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.901+0000 7ff9bdfd0640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9b81f6920 0x7ff9b81f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:41.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.902+0000 7ff9bdfd0640 1 -- 172.21.15.81:0/3275126122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b80af560 con 0x7ff9a40047c0 2023-11-08T16:29:41.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.902+0000 7ff9bdfd0640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 0x7ff9b81f2bc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40015f0 tx=0x7ff9b400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:41.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.902+0000 7ff9ae7fc640 1 -- 172.21.15.81:0/3275126122 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b400ee20 con 0x7ff9a40047c0 2023-11-08T16:29:41.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.902+0000 7ff9ae7fc640 1 -- 172.21.15.81:0/3275126122 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b4013420 con 0x7ff9a40047c0 2023-11-08T16:29:41.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.902+0000 7ff9c0234640 1 -- 172.21.15.81:0/3275126122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b80af840 con 0x7ff9a40047c0 2023-11-08T16:29:41.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.902+0000 7ff9ae7fc640 1 -- 172.21.15.81:0/3275126122 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40137d0 con 0x7ff9a40047c0 2023-11-08T16:29:41.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.902+0000 7ff9c0234640 1 -- 172.21.15.81:0/3275126122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b815b9e0 con 0x7ff9a40047c0 2023-11-08T16:29:41.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.903+0000 7ff9c0234640 1 -- 172.21.15.81:0/3275126122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b804ece0 con 0x7ff9a40047c0 2023-11-08T16:29:41.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.904+0000 7ff9ae7fc640 1 -- 172.21.15.81:0/3275126122 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9b4013930 con 0x7ff9a40047c0 2023-11-08T16:29:41.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.905+0000 7ff9ae7fc640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff99c06ae70 0x7ff99c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:41.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.905+0000 7ff9ae7fc640 1 -- 172.21.15.81:0/3275126122 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9b4089d10 con 0x7ff9a40047c0 2023-11-08T16:29:41.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.905+0000 7ff9bd7cf640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff99c06ae70 0x7ff99c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:41.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.906+0000 7ff9bd7cf640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff99c06ae70 0x7ff99c06d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8036000 tx=0x7ff9a802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:41.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:41.910+0000 7ff9ae7fc640 1 -- 172.21.15.81:0/3275126122 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9b408f020 con 0x7ff9a40047c0 2023-11-08T16:29:42.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.099+0000 7ff9c0234640 1 -- 172.21.15.81:0/3275126122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff9b80ce960 con 0x7ff9a40047c0 2023-11-08T16:29:42.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.100+0000 7ff9ae7fc640 1 -- 172.21.15.81:0/3275126122 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff9b80ce960 con 0x7ff9a40047c0 2023-11-08T16:29:42.102 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:42.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.102+0000 7ff9c0234640 1 -- 172.21.15.81:0/3275126122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff99c06ae70 msgr2=0x7ff99c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:42.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.102+0000 7ff9c0234640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff99c06ae70 0x7ff99c06d330 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff9a8036000 tx=0x7ff9a802e000 comp rx=0 tx=0).stop 2023-11-08T16:29:42.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.103+0000 7ff9c0234640 1 -- 172.21.15.81:0/3275126122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 msgr2=0x7ff9b81f2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:42.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.103+0000 7ff9c0234640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 0x7ff9b81f2bc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff9b40015f0 tx=0x7ff9b400e530 comp rx=0 tx=0).stop 2023-11-08T16:29:42.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.103+0000 7ff9c0234640 1 -- 172.21.15.81:0/3275126122 shutdown_connections 2023-11-08T16:29:42.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.103+0000 7ff9c0234640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff99c06ae70 0x7ff99c06d330 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.103+0000 7ff9c0234640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9b81f6920 0x7ff9b81f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.103+0000 7ff9c0234640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff9b80ac8b0 0x7ff9b80acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.103+0000 7ff9c0234640 1 --2- 172.21.15.81:0/3275126122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9a40047c0 0x7ff9b81f2bc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.103+0000 7ff9c0234640 1 -- 172.21.15.81:0/3275126122 >> 172.21.15.81:0/3275126122 conn(0x7ff9b80cb580 msgr2=0x7ff9b80cbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:42.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.103+0000 7ff9c0234640 1 -- 172.21.15.81:0/3275126122 shutdown_connections 2023-11-08T16:29:42.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.103+0000 7ff9c0234640 1 -- 172.21.15.81:0/3275126122 wait complete. 2023-11-08T16:29:42.124 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:42.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.338+0000 7fd9328b6640 1 Processor -- start 2023-11-08T16:29:42.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.338+0000 7fd9328b6640 1 -- start start 2023-11-08T16:29:42.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.339+0000 7fd9328b6640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd92c15e1a0 0x7fd92c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.339+0000 7fd9328b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c070530 0x7fd92c06bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.339+0000 7fd9328b6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92c15e580 con 0x7fd92c0c97b0 2023-11-08T16:29:42.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.339+0000 7fd9328b6640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92c0ca830 con 0x7fd92c15e1a0 2023-11-08T16:29:42.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.339+0000 7fd9328b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c0c9cb0 con 0x7fd92c070530 2023-11-08T16:29:42.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.340+0000 7fd9320b5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c070530 0x7fd92c06bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.340+0000 7fd9318b4640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd92c15e1a0 0x7fd92c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.340+0000 7fd9318b4640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd92c15e1a0 0x7fd92c0caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:33796/0 (socket says 172.21.15.81:33796) 2023-11-08T16:29:42.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.340+0000 7fd9320b5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c070530 0x7fd92c06bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:43526/0 (socket says 172.21.15.81:43526) 2023-11-08T16:29:42.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.340+0000 7fd9318b4640 1 -- 172.21.15.81:0/118848340 learned_addr learned my addr 172.21.15.81:0/118848340 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:42.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.340+0000 7fd9310b3640 1 --1- 172.21.15.81:0/118848340 >> v1:172.21.15.93:6789/0 conn(0x7fd92c0c97b0 0x7fd92c06fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:33694/0 (socket says 172.21.15.81:33694) 2023-11-08T16:29:42.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.341+0000 7fd9318b4640 1 -- 172.21.15.81:0/118848340 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c070530 msgr2=0x7fd92c06bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:42.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.341+0000 7fd9318b4640 1 --2- 172.21.15.81:0/118848340 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c070530 0x7fd92c06bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.341+0000 7fd9318b4640 1 -- 172.21.15.81:0/118848340 >> v1:172.21.15.93:6789/0 conn(0x7fd92c0c97b0 legacy=0x7fd92c06fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:42.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.341+0000 7fd9318b4640 1 -- 172.21.15.81:0/118848340 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c0c9f90 con 0x7fd92c15e1a0 2023-11-08T16:29:42.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.342+0000 7fd9318b4640 1 --2- 172.21.15.81:0/118848340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd92c15e1a0 0x7fd92c0caa30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd928009870 tx=0x7fd92802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbcd233002512dd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:42.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.343+0000 7fd9308b2640 1 -- 172.21.15.81:0/118848340 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928031640 con 0x7fd92c15e1a0 2023-11-08T16:29:42.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.343+0000 7fd9308b2640 1 -- 172.21.15.81:0/118848340 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9280317a0 con 0x7fd92c15e1a0 2023-11-08T16:29:42.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.343+0000 7fd9308b2640 1 -- 172.21.15.81:0/118848340 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928031b30 con 0x7fd92c15e1a0 2023-11-08T16:29:42.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.343+0000 7fd9328b6640 1 -- 172.21.15.81:0/118848340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd92c15e1a0 msgr2=0x7fd92c0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:42.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.343+0000 7fd9328b6640 1 --2- 172.21.15.81:0/118848340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd92c15e1a0 0x7fd92c0caa30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd928009870 tx=0x7fd92802b8f0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.344+0000 7fd9328b6640 1 -- 172.21.15.81:0/118848340 shutdown_connections 2023-11-08T16:29:42.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.344+0000 7fd9328b6640 1 --2- 172.21.15.81:0/118848340 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c070530 0x7fd92c06bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.344+0000 7fd9328b6640 1 --2- 172.21.15.81:0/118848340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd92c15e1a0 0x7fd92c0caa30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.344+0000 7fd9328b6640 1 -- 172.21.15.81:0/118848340 >> 172.21.15.81:0/118848340 conn(0x7fd92c0ce8d0 msgr2=0x7fd92c0cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:42.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.344+0000 7fd9328b6640 1 -- 172.21.15.81:0/118848340 shutdown_connections 2023-11-08T16:29:42.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.344+0000 7fd9328b6640 1 -- 172.21.15.81:0/118848340 wait complete. 2023-11-08T16:29:42.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.345+0000 7fd9328b6640 1 Processor -- start 2023-11-08T16:29:42.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.346+0000 7fd9328b6640 1 -- start start 2023-11-08T16:29:42.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.346+0000 7fd9328b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd92c070530 0x7fd92c0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.346+0000 7fd9328b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c0c97b0 0x7fd92c0cd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.346+0000 7fd9328b6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd92c15e1a0 0x7fd92c14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9328b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c0b2be0 con 0x7fd92c070530 2023-11-08T16:29:42.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9328b6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c0cdd50 con 0x7fd92c15e1a0 2023-11-08T16:29:42.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9328b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c150350 con 0x7fd92c0c97b0 2023-11-08T16:29:42.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9318b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd92c070530 0x7fd92c0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9318b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd92c070530 0x7fd92c0b44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55670/0 (socket says 172.21.15.81:55670) 2023-11-08T16:29:42.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9318b4640 1 -- 172.21.15.81:0/1512715232 learned_addr learned my addr 172.21.15.81:0/1512715232 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:42.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9320b5640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd92c15e1a0 0x7fd92c14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9310b3640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c0c97b0 0x7fd92c0cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9318b4640 1 -- 172.21.15.81:0/1512715232 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c0c97b0 msgr2=0x7fd92c0cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:42.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9318b4640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c0c97b0 0x7fd92c0cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9318b4640 1 -- 172.21.15.81:0/1512715232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd92c15e1a0 msgr2=0x7fd92c14fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:42.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9318b4640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd92c15e1a0 0x7fd92c14fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9318b4640 1 -- 172.21.15.81:0/1512715232 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9280094d0 con 0x7fd92c070530 2023-11-08T16:29:42.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9320b5640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd92c15e1a0 0x7fd92c14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:42.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.347+0000 7fd9310b3640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c0c97b0 0x7fd92c0cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:42.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.348+0000 7fd9318b4640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd92c070530 0x7fd92c0b44e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd92802b930 tx=0x7fd9280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:42.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.348+0000 7fd9227fc640 1 -- 172.21.15.81:0/1512715232 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92803a930 con 0x7fd92c070530 2023-11-08T16:29:42.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.348+0000 7fd9227fc640 1 -- 172.21.15.81:0/1512715232 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92803aa90 con 0x7fd92c070530 2023-11-08T16:29:42.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.348+0000 7fd9227fc640 1 -- 172.21.15.81:0/1512715232 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92803ae40 con 0x7fd92c070530 2023-11-08T16:29:42.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.348+0000 7fd9328b6640 1 -- 172.21.15.81:0/1512715232 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92c0cdf80 con 0x7fd92c070530 2023-11-08T16:29:42.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.348+0000 7fd9328b6640 1 -- 172.21.15.81:0/1512715232 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c0b2800 con 0x7fd92c070530 2023-11-08T16:29:42.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.349+0000 7fd9328b6640 1 -- 172.21.15.81:0/1512715232 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92c04ece0 con 0x7fd92c070530 2023-11-08T16:29:42.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.350+0000 7fd9227fc640 1 -- 172.21.15.81:0/1512715232 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd92802f050 con 0x7fd92c070530 2023-11-08T16:29:42.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.351+0000 7fd9227fc640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 0x7fd91006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.351+0000 7fd9227fc640 1 -- 172.21.15.81:0/1512715232 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9280ab500 con 0x7fd92c070530 2023-11-08T16:29:42.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.351+0000 7fd9310b3640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 0x7fd91006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.352+0000 7fd9310b3640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 0x7fd91006d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd92c15f470 tx=0x7fd91c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:42.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.356+0000 7fd9227fc640 1 -- 172.21.15.81:0/1512715232 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd928076520 con 0x7fd92c070530 2023-11-08T16:29:42.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.543+0000 7fd9328b6640 1 -- 172.21.15.81:0/1512715232 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd92c06fd60 con 0x7fd92c070530 2023-11-08T16:29:42.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.544+0000 7fd9227fc640 1 -- 172.21.15.81:0/1512715232 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd928035020 con 0x7fd92c070530 2023-11-08T16:29:42.546 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:42.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 -- 172.21.15.81:0/1512715232 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 msgr2=0x7fd91006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:42.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 0x7fd91006d330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd92c15f470 tx=0x7fd91c009040 comp rx=0 tx=0).stop 2023-11-08T16:29:42.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 -- 172.21.15.81:0/1512715232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd92c070530 msgr2=0x7fd92c0b44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:42.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd92c070530 0x7fd92c0b44e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd92802b930 tx=0x7fd9280046c0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 -- 172.21.15.81:0/1512715232 shutdown_connections 2023-11-08T16:29:42.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 0x7fd91006d330 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd92c15e1a0 0x7fd92c14fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd92c0c97b0 0x7fd92c0cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 --2- 172.21.15.81:0/1512715232 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd92c070530 0x7fd92c0b44e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 -- 172.21.15.81:0/1512715232 >> 172.21.15.81:0/1512715232 conn(0x7fd92c0ce8d0 msgr2=0x7fd92c0ad220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:42.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 -- 172.21.15.81:0/1512715232 shutdown_connections 2023-11-08T16:29:42.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.546+0000 7fd9328b6640 1 -- 172.21.15.81:0/1512715232 wait complete. 2023-11-08T16:29:42.569 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:42.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.781+0000 7f31709a9640 1 Processor -- start 2023-11-08T16:29:42.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.781+0000 7f31709a9640 1 -- start start 2023-11-08T16:29:42.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.782+0000 7f31709a9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f316c0ca480 0x7f316c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.782+0000 7f31709a9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f316c15ea60 0x7f316c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.782+0000 7f31709a9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316c06b630 con 0x7f316c0ade50 2023-11-08T16:29:42.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.782+0000 7f31709a9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316c0b1860 con 0x7f316c0ca480 2023-11-08T16:29:42.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.782+0000 7f31709a9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316c0ca910 con 0x7f316c15ea60 2023-11-08T16:29:42.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.783+0000 7f3161d9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f316c0ca480 0x7f316c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.783+0000 7f3161d9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f316c0ca480 0x7f316c15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55674/0 (socket says 172.21.15.81:55674) 2023-11-08T16:29:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.783+0000 7f3161d9b640 1 -- 172.21.15.81:0/2420548096 learned_addr learned my addr 172.21.15.81:0/2420548096 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.783+0000 7f316ad9d640 1 --2- 172.21.15.81:0/2420548096 >> v2:172.21.15.93:3300/0 conn(0x7f316c15ea60 0x7f316c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.783+0000 7f3161d9b640 1 -- 172.21.15.81:0/2420548096 >> v2:172.21.15.93:3300/0 conn(0x7f316c15ea60 msgr2=0x7f316c15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:42.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.783+0000 7f3161d9b640 1 --2- 172.21.15.81:0/2420548096 >> v2:172.21.15.93:3300/0 conn(0x7f316c15ea60 0x7f316c15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.783+0000 7f3161d9b640 1 -- 172.21.15.81:0/2420548096 >> v1:172.21.15.93:6789/0 conn(0x7f316c0ade50 legacy=0x7f316c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:42.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.783+0000 7f3161d9b640 1 -- 172.21.15.81:0/2420548096 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c0cabb0 con 0x7f316c0ca480 2023-11-08T16:29:42.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.784+0000 7f3161d9b640 1 --2- 172.21.15.81:0/2420548096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f316c0ca480 0x7f316c15ad90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f316c0ca220 tx=0x7f315402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=696c06a3c0a14a24 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:42.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.784+0000 7f3169d9b640 1 -- 172.21.15.81:0/2420548096 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154031870 con 0x7f316c0ca480 2023-11-08T16:29:42.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.785+0000 7f3169d9b640 1 -- 172.21.15.81:0/2420548096 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31540319d0 con 0x7f316c0ca480 2023-11-08T16:29:42.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.785+0000 7f3169d9b640 1 -- 172.21.15.81:0/2420548096 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3154031d60 con 0x7f316c0ca480 2023-11-08T16:29:42.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.785+0000 7f31709a9640 1 -- 172.21.15.81:0/2420548096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f316c0ca480 msgr2=0x7f316c15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:42.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.785+0000 7f31709a9640 1 --2- 172.21.15.81:0/2420548096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f316c0ca480 0x7f316c15ad90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f316c0ca220 tx=0x7f315402b910 comp rx=0 tx=0).stop 2023-11-08T16:29:42.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.785+0000 7f31709a9640 1 -- 172.21.15.81:0/2420548096 shutdown_connections 2023-11-08T16:29:42.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.785+0000 7f31709a9640 1 --2- 172.21.15.81:0/2420548096 >> v2:172.21.15.93:3300/0 conn(0x7f316c15ea60 0x7f316c15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.785+0000 7f31709a9640 1 --2- 172.21.15.81:0/2420548096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f316c0ca480 0x7f316c15ad90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.785+0000 7f31709a9640 1 -- 172.21.15.81:0/2420548096 >> 172.21.15.81:0/2420548096 conn(0x7f316c0cb490 msgr2=0x7f316c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:42.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.785+0000 7f31709a9640 1 -- 172.21.15.81:0/2420548096 shutdown_connections 2023-11-08T16:29:42.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.785+0000 7f31709a9640 1 -- 172.21.15.81:0/2420548096 wait complete. 2023-11-08T16:29:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.787+0000 7f31709a9640 1 Processor -- start 2023-11-08T16:29:42.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.787+0000 7f31709a9640 1 -- start start 2023-11-08T16:29:42.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.788+0000 7f31709a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f316c0ade50 0x7f316c1f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.788+0000 7f31709a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f316c0ca480 0x7f316c155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.788+0000 7f31709a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f316c15ea60 0x7f316c1f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.788+0000 7f31709a9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c1f0a20 con 0x7f316c0ca480 2023-11-08T16:29:42.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.788+0000 7f31709a9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c0b3790 con 0x7f316c15ea60 2023-11-08T16:29:42.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.788+0000 7f31709a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316c0cc970 con 0x7f316c0ade50 2023-11-08T16:29:42.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.788+0000 7f3161d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f316c0ca480 0x7f316c155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.788+0000 7f3161d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f316c0ca480 0x7f316c155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55682/0 (socket says 172.21.15.81:55682) 2023-11-08T16:29:42.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.788+0000 7f3161d9b640 1 -- 172.21.15.81:0/2284217591 learned_addr learned my addr 172.21.15.81:0/2284217591 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:42.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.789+0000 7f316ad9d640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f316c15ea60 0x7f316c1f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.789+0000 7f3161d9b640 1 -- 172.21.15.81:0/2284217591 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f316c0ade50 msgr2=0x7f316c1f04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:42.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.789+0000 7f316a59c640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f316c0ade50 0x7f316c1f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.789+0000 7f3161d9b640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f316c0ade50 0x7f316c1f04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.789+0000 7f3161d9b640 1 -- 172.21.15.81:0/2284217591 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f316c15ea60 msgr2=0x7f316c1f3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:42.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.789+0000 7f3161d9b640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f316c15ea60 0x7f316c1f3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:42.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.789+0000 7f3161d9b640 1 -- 172.21.15.81:0/2284217591 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31540094d0 con 0x7f316c0ca480 2023-11-08T16:29:42.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.789+0000 7f316ad9d640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f316c15ea60 0x7f316c1f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:42.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.789+0000 7f3161d9b640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f316c0ca480 0x7f316c155c10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f315402b950 tx=0x7f31540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:42.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.789+0000 7f31637fe640 1 -- 172.21.15.81:0/2284217591 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315403ad30 con 0x7f316c0ca480 2023-11-08T16:29:42.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.790+0000 7f31637fe640 1 -- 172.21.15.81:0/2284217591 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31540443f0 con 0x7f316c0ca480 2023-11-08T16:29:42.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.790+0000 7f31637fe640 1 -- 172.21.15.81:0/2284217591 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31540447a0 con 0x7f316c0ca480 2023-11-08T16:29:42.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.790+0000 7f31709a9640 1 -- 172.21.15.81:0/2284217591 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c1f3050 con 0x7f316c0ca480 2023-11-08T16:29:42.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.790+0000 7f31709a9640 1 -- 172.21.15.81:0/2284217591 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c156900 con 0x7f316c0ca480 2023-11-08T16:29:42.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.791+0000 7f31709a9640 1 -- 172.21.15.81:0/2284217591 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c04ece0 con 0x7f316c0ca480 2023-11-08T16:29:42.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.792+0000 7f31637fe640 1 -- 172.21.15.81:0/2284217591 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f315404d450 con 0x7f316c0ca480 2023-11-08T16:29:42.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.792+0000 7f31637fe640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f314c06ae70 0x7f314c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:42.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.793+0000 7f31637fe640 1 -- 172.21.15.81:0/2284217591 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f315402d040 con 0x7f316c0ca480 2023-11-08T16:29:42.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.793+0000 7f316a59c640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f314c06ae70 0x7f314c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:42.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.793+0000 7f316a59c640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f314c06ae70 0x7f314c06d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f316c15b3d0 tx=0x7f315c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:42.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.797+0000 7f31637fe640 1 -- 172.21.15.81:0/2284217591 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3154076cd0 con 0x7f316c0ca480 2023-11-08T16:29:42.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:42.996+0000 7f31709a9640 1 -- 172.21.15.81:0/2284217591 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f316c0cc1d0 con 0x7f316c0ca480 2023-11-08T16:29:43.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.000+0000 7f31637fe640 1 -- 172.21.15.81:0/2284217591 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f3154035070 con 0x7f316c0ca480 2023-11-08T16:29:43.002 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:43.002 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":22,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:42.816559+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:43.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.002+0000 7f31709a9640 1 -- 172.21.15.81:0/2284217591 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f314c06ae70 msgr2=0x7f314c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.002+0000 7f31709a9640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f314c06ae70 0x7f314c06d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f316c15b3d0 tx=0x7f315c009040 comp rx=0 tx=0).stop 2023-11-08T16:29:43.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.002+0000 7f31709a9640 1 -- 172.21.15.81:0/2284217591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f316c0ca480 msgr2=0x7f316c155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.002+0000 7f31709a9640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f316c0ca480 0x7f316c155c10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f315402b950 tx=0x7f31540046c0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.003+0000 7f31709a9640 1 -- 172.21.15.81:0/2284217591 shutdown_connections 2023-11-08T16:29:43.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.003+0000 7f31709a9640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f314c06ae70 0x7f314c06d330 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.003+0000 7f31709a9640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f316c15ea60 0x7f316c1f3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.003+0000 7f31709a9640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f316c0ca480 0x7f316c155c10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.003+0000 7f31709a9640 1 --2- 172.21.15.81:0/2284217591 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f316c0ade50 0x7f316c1f04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.003+0000 7f31709a9640 1 -- 172.21.15.81:0/2284217591 >> 172.21.15.81:0/2284217591 conn(0x7f316c0cb490 msgr2=0x7f316c15cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:43.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.003+0000 7f31709a9640 1 -- 172.21.15.81:0/2284217591 shutdown_connections 2023-11-08T16:29:43.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.003+0000 7f31709a9640 1 -- 172.21.15.81:0/2284217591 wait complete. 2023-11-08T16:29:43.024 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:43.025 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:43.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.243+0000 7f3916bb8640 1 Processor -- start 2023-11-08T16:29:43.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.243+0000 7f3916bb8640 1 -- start start 2023-11-08T16:29:43.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.244+0000 7f3916bb8640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39100c97b0 0x7f391006fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:43.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.244+0000 7f3916bb8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3910070360 0x7f391006bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:43.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f3916bb8640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f391015e580 con 0x7f391015e1a0 2023-11-08T16:29:43.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f3916bb8640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39100ca830 con 0x7f3910070360 2023-11-08T16:29:43.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f3916bb8640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39100707d0 con 0x7f39100c97b0 2023-11-08T16:29:43.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f39163b7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3910070360 0x7f391006bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:43.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f39163b7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3910070360 0x7f391006bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55696/0 (socket says 172.21.15.81:55696) 2023-11-08T16:29:43.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f39163b7640 1 -- 172.21.15.81:0/29930984 learned_addr learned my addr 172.21.15.81:0/29930984 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:43.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f39153b5640 1 --2- 172.21.15.81:0/29930984 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39100c97b0 0x7f391006fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:43.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f39163b7640 1 -- 172.21.15.81:0/29930984 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39100c97b0 msgr2=0x7f391006fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:43.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f39163b7640 1 --2- 172.21.15.81:0/29930984 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39100c97b0 0x7f391006fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f39163b7640 1 -- 172.21.15.81:0/29930984 >> v1:172.21.15.93:6789/0 conn(0x7f391015e1a0 legacy=0x7f39100caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:43.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f39163b7640 1 -- 172.21.15.81:0/29930984 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39100c9cb0 con 0x7f3910070360 2023-11-08T16:29:43.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.245+0000 7f39153b5640 1 --2- 172.21.15.81:0/29930984 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39100c97b0 0x7f391006fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:43.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.247+0000 7f39163b7640 1 --2- 172.21.15.81:0/29930984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3910070360 0x7f391006bb10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3908009e80 tx=0x7f390802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4dcbcb9bc658912 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:43.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.247+0000 7f3914bb4640 1 -- 172.21.15.81:0/29930984 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390802c9c0 con 0x7f3910070360 2023-11-08T16:29:43.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.247+0000 7f3914bb4640 1 -- 172.21.15.81:0/29930984 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f390802cb20 con 0x7f3910070360 2023-11-08T16:29:43.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.247+0000 7f3914bb4640 1 -- 172.21.15.81:0/29930984 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39080343f0 con 0x7f3910070360 2023-11-08T16:29:43.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.247+0000 7f3916bb8640 1 -- 172.21.15.81:0/29930984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3910070360 msgr2=0x7f391006bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.248+0000 7f3916bb8640 1 --2- 172.21.15.81:0/29930984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3910070360 0x7f391006bb10 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3908009e80 tx=0x7f390802be20 comp rx=0 tx=0).stop 2023-11-08T16:29:43.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.248+0000 7f3916bb8640 1 -- 172.21.15.81:0/29930984 shutdown_connections 2023-11-08T16:29:43.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.248+0000 7f3916bb8640 1 --2- 172.21.15.81:0/29930984 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3910070360 0x7f391006bb10 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.248+0000 7f3916bb8640 1 --2- 172.21.15.81:0/29930984 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39100c97b0 0x7f391006fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.248+0000 7f3916bb8640 1 -- 172.21.15.81:0/29930984 >> 172.21.15.81:0/29930984 conn(0x7f39100ce8d0 msgr2=0x7f39100cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:43.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.248+0000 7f3916bb8640 1 -- 172.21.15.81:0/29930984 shutdown_connections 2023-11-08T16:29:43.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.248+0000 7f3916bb8640 1 -- 172.21.15.81:0/29930984 wait complete. 2023-11-08T16:29:43.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.250+0000 7f3916bb8640 1 Processor -- start 2023-11-08T16:29:43.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.250+0000 7f3916bb8640 1 -- start start 2023-11-08T16:29:43.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3916bb8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3910070360 0x7f39100cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:43.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3916bb8640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39100c97b0 0x7f39100ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:43.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3916bb8640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f391015e1a0 0x7f39100acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:43.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3916bb8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39100c78a0 con 0x7f3910070360 2023-11-08T16:29:43.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3916bb8640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39100c6a10 con 0x7f39100c97b0 2023-11-08T16:29:43.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3916bb8640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39100c6c90 con 0x7f391015e1a0 2023-11-08T16:29:43.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3915bb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3910070360 0x7f39100cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:43.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3915bb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3910070360 0x7f39100cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55698/0 (socket says 172.21.15.81:55698) 2023-11-08T16:29:43.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3915bb6640 1 -- 172.21.15.81:0/494908823 learned_addr learned my addr 172.21.15.81:0/494908823 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:43.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f39153b5640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39100c97b0 0x7f39100ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:43.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3915bb6640 1 -- 172.21.15.81:0/494908823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f391015e1a0 msgr2=0x7f39100acde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f39163b7640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f391015e1a0 0x7f39100acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:43.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3915bb6640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f391015e1a0 0x7f39100acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3915bb6640 1 -- 172.21.15.81:0/494908823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39100c97b0 msgr2=0x7f39100ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3915bb6640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39100c97b0 0x7f39100ac8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f3915bb6640 1 -- 172.21.15.81:0/494908823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3908009b30 con 0x7f3910070360 2023-11-08T16:29:43.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.251+0000 7f39153b5640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39100c97b0 0x7f39100ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:43.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.252+0000 7f3915bb6640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3910070360 0x7f39100cdd60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f390400b3f0 tx=0x7f390400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:43.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.252+0000 7f39027fc640 1 -- 172.21.15.81:0/494908823 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904011830 con 0x7f3910070360 2023-11-08T16:29:43.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.252+0000 7f39027fc640 1 -- 172.21.15.81:0/494908823 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3904011990 con 0x7f3910070360 2023-11-08T16:29:43.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.252+0000 7f39027fc640 1 -- 172.21.15.81:0/494908823 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3904011d40 con 0x7f3910070360 2023-11-08T16:29:43.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.252+0000 7f3916bb8640 1 -- 172.21.15.81:0/494908823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39100c6f70 con 0x7f3910070360 2023-11-08T16:29:43.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.252+0000 7f3916bb8640 1 -- 172.21.15.81:0/494908823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39100c7530 con 0x7f3910070360 2023-11-08T16:29:43.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.253+0000 7f3916bb8640 1 -- 172.21.15.81:0/494908823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f391004ece0 con 0x7f3910070360 2023-11-08T16:29:43.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.254+0000 7f39027fc640 1 -- 172.21.15.81:0/494908823 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f390401a400 con 0x7f3910070360 2023-11-08T16:29:43.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.255+0000 7f39027fc640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38f406ae70 0x7f38f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:43.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.255+0000 7f39027fc640 1 -- 172.21.15.81:0/494908823 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3904089490 con 0x7f3910070360 2023-11-08T16:29:43.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.256+0000 7f39153b5640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38f406ae70 0x7f38f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:43.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.256+0000 7f39153b5640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38f406ae70 0x7f38f406d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f391014d5b0 tx=0x7f390c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:43.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.260+0000 7f39027fc640 1 -- 172.21.15.81:0/494908823 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3904054540 con 0x7f3910070360 2023-11-08T16:29:43.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.448+0000 7f3916bb8640 1 -- 172.21.15.81:0/494908823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f391006d550 con 0x7f3910070360 2023-11-08T16:29:43.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.449+0000 7f39027fc640 1 -- 172.21.15.81:0/494908823 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3904057b60 con 0x7f3910070360 2023-11-08T16:29:43.451 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:43.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.451+0000 7f3916bb8640 1 -- 172.21.15.81:0/494908823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38f406ae70 msgr2=0x7f38f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.451+0000 7f3916bb8640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38f406ae70 0x7f38f406d330 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f391014d5b0 tx=0x7f390c008040 comp rx=0 tx=0).stop 2023-11-08T16:29:43.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.451+0000 7f3916bb8640 1 -- 172.21.15.81:0/494908823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3910070360 msgr2=0x7f39100cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.451+0000 7f3916bb8640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3910070360 0x7f39100cdd60 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f390400b3f0 tx=0x7f390400b8b0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.452+0000 7f3916bb8640 1 -- 172.21.15.81:0/494908823 shutdown_connections 2023-11-08T16:29:43.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.452+0000 7f3916bb8640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38f406ae70 0x7f38f406d330 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.452+0000 7f3916bb8640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f391015e1a0 0x7f39100acde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.452+0000 7f3916bb8640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39100c97b0 0x7f39100ac8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.452+0000 7f3916bb8640 1 --2- 172.21.15.81:0/494908823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3910070360 0x7f39100cdd60 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.452+0000 7f3916bb8640 1 -- 172.21.15.81:0/494908823 >> 172.21.15.81:0/494908823 conn(0x7f39100ce8d0 msgr2=0x7f391015e750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:43.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.452+0000 7f3916bb8640 1 -- 172.21.15.81:0/494908823 shutdown_connections 2023-11-08T16:29:43.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.452+0000 7f3916bb8640 1 -- 172.21.15.81:0/494908823 wait complete. 2023-11-08T16:29:43.472 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:43.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.693+0000 7f95605f5640 1 Processor -- start 2023-11-08T16:29:43.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.694+0000 7f95605f5640 1 -- start start 2023-11-08T16:29:43.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.695+0000 7f95605f5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f95580ce0d0 0x7f95580caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:43.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.695+0000 7f95605f5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95580cda10 con 0x7f955815f280 2023-11-08T16:29:43.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.695+0000 7f95605f5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f955806fa30 con 0x7f955815bfc0 2023-11-08T16:29:43.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.695+0000 7f95605f5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95580ce4b0 con 0x7f95580ce0d0 2023-11-08T16:29:43.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.696+0000 7f955eb92640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f955815f280 0x7f95581516e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35954/0 (socket says 172.21.15.81:35954) 2023-11-08T16:29:43.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.696+0000 7f955eb92640 1 -- 172.21.15.81:0/3246334948 learned_addr learned my addr 172.21.15.81:0/3246334948 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:43.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.696+0000 7f955e391640 1 --2- 172.21.15.81:0/3246334948 >> v2:172.21.15.93:3300/0 conn(0x7f95580ce0d0 0x7f95580caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:43.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.696+0000 7f955d38f640 1 -- 172.21.15.81:0/3246334948 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17975062 0 0) 0x7f95580cda10 con 0x7f955815f280 2023-11-08T16:29:43.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.696+0000 7f955d38f640 1 -- 172.21.15.81:0/3246334948 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9540003520 con 0x7f955815f280 2023-11-08T16:29:43.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.696+0000 7f955e391640 1 -- 172.21.15.81:0/3246334948 >> v1:172.21.15.93:6789/0 conn(0x7f955815bfc0 legacy=0x7f955815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.696+0000 7f955e391640 1 -- 172.21.15.81:0/3246334948 >> v1:172.21.15.81:6789/0 conn(0x7f955815f280 legacy=0x7f95581516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.696+0000 7f955d38f640 1 -- 172.21.15.81:0/3246334948 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3548432588 0 0) 0x7f9540003520 con 0x7f955815f280 2023-11-08T16:29:43.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.697+0000 7f955e391640 1 -- 172.21.15.81:0/3246334948 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f955806fca0 con 0x7f95580ce0d0 2023-11-08T16:29:43.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.698+0000 7f955e391640 1 --2- 172.21.15.81:0/3246334948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95580ce0d0 0x7f95580caf20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9548009d90 tx=0x7f954802be20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c15cbd9a59d876b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:43.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.698+0000 7f955d38f640 1 -- 172.21.15.81:0/3246334948 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9550001670 con 0x7f95580ce0d0 2023-11-08T16:29:43.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.698+0000 7f955d38f640 1 -- 172.21.15.81:0/3246334948 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f954802c950 con 0x7f95580ce0d0 2023-11-08T16:29:43.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.698+0000 7f955d38f640 1 -- 172.21.15.81:0/3246334948 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f954802ccc0 con 0x7f95580ce0d0 2023-11-08T16:29:43.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.699+0000 7f95605f5640 1 -- 172.21.15.81:0/3246334948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95580ce0d0 msgr2=0x7f95580caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.699+0000 7f95605f5640 1 --2- 172.21.15.81:0/3246334948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95580ce0d0 0x7f95580caf20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9548009d90 tx=0x7f954802be20 comp rx=0 tx=0).stop 2023-11-08T16:29:43.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.699+0000 7f95605f5640 1 -- 172.21.15.81:0/3246334948 shutdown_connections 2023-11-08T16:29:43.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.699+0000 7f95605f5640 1 --2- 172.21.15.81:0/3246334948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95580ce0d0 0x7f95580caf20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.699+0000 7f95605f5640 1 -- 172.21.15.81:0/3246334948 >> 172.21.15.81:0/3246334948 conn(0x7f95580cb580 msgr2=0x7f95580cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:43.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.699+0000 7f95605f5640 1 -- 172.21.15.81:0/3246334948 shutdown_connections 2023-11-08T16:29:43.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.699+0000 7f95605f5640 1 -- 172.21.15.81:0/3246334948 wait complete. 2023-11-08T16:29:43.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.701+0000 7f95605f5640 1 Processor -- start 2023-11-08T16:29:43.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.701+0000 7f95605f5640 1 -- start start 2023-11-08T16:29:43.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.702+0000 7f95605f5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95580ce0d0 0x7f95580c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:43.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.702+0000 7f95605f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f955815bfc0 0x7f9558153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:43.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.702+0000 7f95605f5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f955815f280 0x7f95581544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:43.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.702+0000 7f95605f5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95580c4bc0 con 0x7f95580ce0d0 2023-11-08T16:29:43.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.702+0000 7f95605f5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95580c5990 con 0x7f955815f280 2023-11-08T16:29:43.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.702+0000 7f95605f5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95580c8e40 con 0x7f955815bfc0 2023-11-08T16:29:43.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.702+0000 7f955e391640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95580ce0d0 0x7f95580c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:43.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.702+0000 7f955e391640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95580ce0d0 0x7f95580c8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55704/0 (socket says 172.21.15.81:55704) 2023-11-08T16:29:43.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.702+0000 7f955e391640 1 -- 172.21.15.81:0/3775409245 learned_addr learned my addr 172.21.15.81:0/3775409245 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:43.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f955e391640 1 -- 172.21.15.81:0/3775409245 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f955815bfc0 msgr2=0x7f9558153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f955eb92640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f955815f280 0x7f95581544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:43.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f955db90640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f955815bfc0 0x7f9558153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:43.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f955e391640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f955815bfc0 0x7f9558153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f955e391640 1 -- 172.21.15.81:0/3775409245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f955815f280 msgr2=0x7f95581544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f955e391640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f955815f280 0x7f95581544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f955e391640 1 -- 172.21.15.81:0/3775409245 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9548009a70 con 0x7f95580ce0d0 2023-11-08T16:29:43.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f955e391640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95580ce0d0 0x7f95580c8900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9548009eb0 tx=0x7f9548004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:43.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f9556ffd640 1 -- 172.21.15.81:0/3775409245 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548004250 con 0x7f95580ce0d0 2023-11-08T16:29:43.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f9556ffd640 1 -- 172.21.15.81:0/3775409245 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95480043b0 con 0x7f95580ce0d0 2023-11-08T16:29:43.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f9556ffd640 1 -- 172.21.15.81:0/3775409245 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9548004760 con 0x7f95580ce0d0 2023-11-08T16:29:43.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f95605f5640 1 -- 172.21.15.81:0/3775409245 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95580c4e10 con 0x7f95580ce0d0 2023-11-08T16:29:43.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.703+0000 7f95605f5640 1 -- 172.21.15.81:0/3775409245 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95580c53d0 con 0x7f95580ce0d0 2023-11-08T16:29:43.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.705+0000 7f95605f5640 1 -- 172.21.15.81:0/3775409245 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95580b25d0 con 0x7f95580ce0d0 2023-11-08T16:29:43.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.705+0000 7f9556ffd640 1 -- 172.21.15.81:0/3775409245 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9548034440 con 0x7f95580ce0d0 2023-11-08T16:29:43.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.706+0000 7f9556ffd640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f954406aea0 0x7f954406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:43.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.706+0000 7f9556ffd640 1 -- 172.21.15.81:0/3775409245 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f95480abf60 con 0x7f95580ce0d0 2023-11-08T16:29:43.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.707+0000 7f955db90640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f954406aea0 0x7f954406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:43.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.707+0000 7f955db90640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f954406aea0 0x7f954406d360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95580b4990 tx=0x7f954c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:43.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.711+0000 7f9556ffd640 1 -- 172.21.15.81:0/3775409245 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9548077130 con 0x7f95580ce0d0 2023-11-08T16:29:43.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.907+0000 7f95605f5640 1 -- 172.21.15.81:0/3775409245 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f955804ece0 con 0x7f95580ce0d0 2023-11-08T16:29:43.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.908+0000 7f9556ffd640 1 -- 172.21.15.81:0/3775409245 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9548043020 con 0x7f95580ce0d0 2023-11-08T16:29:43.910 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:43.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.910+0000 7f95605f5640 1 -- 172.21.15.81:0/3775409245 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f954406aea0 msgr2=0x7f954406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.910+0000 7f95605f5640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f954406aea0 0x7f954406d360 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f95580b4990 tx=0x7f954c009040 comp rx=0 tx=0).stop 2023-11-08T16:29:43.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.910+0000 7f95605f5640 1 -- 172.21.15.81:0/3775409245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95580ce0d0 msgr2=0x7f95580c8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:43.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.910+0000 7f95605f5640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95580ce0d0 0x7f95580c8900 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9548009eb0 tx=0x7f9548004a40 comp rx=0 tx=0).stop 2023-11-08T16:29:43.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.910+0000 7f95605f5640 1 -- 172.21.15.81:0/3775409245 shutdown_connections 2023-11-08T16:29:43.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.911+0000 7f95605f5640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f954406aea0 0x7f954406d360 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.911+0000 7f95605f5640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f955815f280 0x7f95581544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.911+0000 7f95605f5640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f955815bfc0 0x7f9558153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.911+0000 7f95605f5640 1 --2- 172.21.15.81:0/3775409245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95580ce0d0 0x7f95580c8900 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:43.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.911+0000 7f95605f5640 1 -- 172.21.15.81:0/3775409245 >> 172.21.15.81:0/3775409245 conn(0x7f95580cb580 msgr2=0x7f95580ca480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:43.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.911+0000 7f95605f5640 1 -- 172.21.15.81:0/3775409245 shutdown_connections 2023-11-08T16:29:43.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:43.911+0000 7f95605f5640 1 -- 172.21.15.81:0/3775409245 wait complete. 2023-11-08T16:29:43.930 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:44.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.136+0000 7fb4214e7640 1 Processor -- start 2023-11-08T16:29:44.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.136+0000 7fb4214e7640 1 -- start start 2023-11-08T16:29:44.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.137+0000 7fb4214e7640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb41c0cd8d0 0x7fb41c06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.137+0000 7fb4214e7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb41c0cc9f0 0x7fb41c15c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.137+0000 7fb4214e7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb41c06cf50 con 0x7fb41c0cdcb0 2023-11-08T16:29:44.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.137+0000 7fb4214e7640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb41c06d230 con 0x7fb41c0cd8d0 2023-11-08T16:29:44.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.137+0000 7fb4214e7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0d0be0 con 0x7fb41c0cc9f0 2023-11-08T16:29:44.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.138+0000 7fb41a7fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb41c0cdcb0 0x7fb41c0cc2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35958/0 (socket says 172.21.15.81:35958) 2023-11-08T16:29:44.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.138+0000 7fb41a7fc640 1 -- 172.21.15.81:0/3314151249 learned_addr learned my addr 172.21.15.81:0/3314151249 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:44.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.138+0000 7fb41b7fe640 1 --2- 172.21.15.81:0/3314151249 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb41c0cc9f0 0x7fb41c15c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.138+0000 7fb41affd640 1 --2- 172.21.15.81:0/3314151249 >> v2:172.21.15.93:3300/0 conn(0x7fb41c0cd8d0 0x7fb41c06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.138+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366858268 0 0) 0x7fb41c06cf50 con 0x7fb41c0cdcb0 2023-11-08T16:29:44.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.138+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb408003520 con 0x7fb41c0cdcb0 2023-11-08T16:29:44.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.138+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3331160467 0 0) 0x7fb408003520 con 0x7fb41c0cdcb0 2023-11-08T16:29:44.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.138+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb41c0cc9f0 msgr2=0x7fb41c15c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:44.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.138+0000 7fb419ffb640 1 --2- 172.21.15.81:0/3314151249 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb41c0cc9f0 0x7fb41c15c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.138+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 >> v2:172.21.15.93:3300/0 conn(0x7fb41c0cd8d0 msgr2=0x7fb41c06e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:44.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.138+0000 7fb419ffb640 1 --2- 172.21.15.81:0/3314151249 >> v2:172.21.15.93:3300/0 conn(0x7fb41c0cd8d0 0x7fb41c06e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c0d0df0 con 0x7fb41c0cdcb0 2023-11-08T16:29:44.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fb404001690 con 0x7fb41c0cdcb0 2023-11-08T16:29:44.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 >> v1:172.21.15.81:6789/0 conn(0x7fb41c0cdcb0 legacy=0x7fb41c0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb419ffb640 1 --2- 172.21.15.81:0/3314151249 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb41c0cc9f0 0x7fb4080039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb419ffb640 1 --2- 172.21.15.81:0/3314151249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb408003fb0 0x7fb4080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb419ffb640 1 --2- 172.21.15.81:0/3314151249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb408004d00 0x7fb408005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c06d230 con 0x7fb408003fb0 2023-11-08T16:29:44.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0d0be0 con 0x7fb408004d00 2023-11-08T16:29:44.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40800ba00 con 0x7fb41c0cc9f0 2023-11-08T16:29:44.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb41a7fc640 1 --2- 172.21.15.81:0/3314151249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb408003fb0 0x7fb4080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb41a7fc640 1 -- 172.21.15.81:0/3314151249 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb41c0cc9f0 msgr2=0x7fb4080039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:44.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb41a7fc640 1 --2- 172.21.15.81:0/3314151249 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb41c0cc9f0 0x7fb4080039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb41a7fc640 1 -- 172.21.15.81:0/3314151249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb408004d00 msgr2=0x7fb408005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:44.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb41a7fc640 1 --2- 172.21.15.81:0/3314151249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb408004d00 0x7fb408005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.139+0000 7fb41a7fc640 1 -- 172.21.15.81:0/3314151249 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb408003520 con 0x7fb408003fb0 2023-11-08T16:29:44.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.140+0000 7fb41a7fc640 1 --2- 172.21.15.81:0/3314151249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb408003fb0 0x7fb4080047c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb40400c160 tx=0x7fb4080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72439ec321cb6ad1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:44.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.141+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404005ad0 con 0x7fb408003fb0 2023-11-08T16:29:44.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.141+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb404005c50 con 0x7fb408003fb0 2023-11-08T16:29:44.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.141+0000 7fb419ffb640 1 -- 172.21.15.81:0/3314151249 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40402c810 con 0x7fb408003fb0 2023-11-08T16:29:44.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.141+0000 7fb4214e7640 1 -- 172.21.15.81:0/3314151249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb408003fb0 msgr2=0x7fb4080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.141+0000 7fb4214e7640 1 --2- 172.21.15.81:0/3314151249 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb408003fb0 0x7fb4080047c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb40400c160 tx=0x7fb4080057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.141+0000 7fb41b7fe640 1 -- 172.21.15.81:0/3314151249 reap_dead start 2023-11-08T16:29:44.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.141+0000 7fb4214e7640 1 -- 172.21.15.81:0/3314151249 shutdown_connections 2023-11-08T16:29:44.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.141+0000 7fb4214e7640 1 -- 172.21.15.81:0/3314151249 >> 172.21.15.81:0/3314151249 conn(0x7fb41c065df0 msgr2=0x7fb41c0661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:44.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.141+0000 7fb4214e7640 1 -- 172.21.15.81:0/3314151249 shutdown_connections 2023-11-08T16:29:44.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.141+0000 7fb4214e7640 1 -- 172.21.15.81:0/3314151249 wait complete. 2023-11-08T16:29:44.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.143+0000 7fb4214e7640 1 Processor -- start 2023-11-08T16:29:44.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.143+0000 7fb4214e7640 1 -- start start 2023-11-08T16:29:44.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.144+0000 7fb4214e7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb408003fb0 0x7fb41c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.144+0000 7fb4214e7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb41c14e250 0x7fb41c0c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.144+0000 7fb4214e7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb41c1f1af0 0x7fb41c1f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.144+0000 7fb4214e7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0c7a10 con 0x7fb41c14e250 2023-11-08T16:29:44.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.144+0000 7fb4214e7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c1f24b0 con 0x7fb41c1f1af0 2023-11-08T16:29:44.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.144+0000 7fb4214e7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb41c0c6d90 con 0x7fb408003fb0 2023-11-08T16:29:44.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.145+0000 7fb41a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb41c14e250 0x7fb41c0c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.145+0000 7fb41a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb41c14e250 0x7fb41c0c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55716/0 (socket says 172.21.15.81:55716) 2023-11-08T16:29:44.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.145+0000 7fb41a7fc640 1 -- 172.21.15.81:0/79243923 learned_addr learned my addr 172.21.15.81:0/79243923 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:44.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.145+0000 7fb41b7fe640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb41c1f1af0 0x7fb41c1f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.145+0000 7fb41affd640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb408003fb0 0x7fb41c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.145+0000 7fb41a7fc640 1 -- 172.21.15.81:0/79243923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb408003fb0 msgr2=0x7fb41c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.145+0000 7fb41a7fc640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb408003fb0 0x7fb41c0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.145+0000 7fb41a7fc640 1 -- 172.21.15.81:0/79243923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb41c1f1af0 msgr2=0x7fb41c1f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.145+0000 7fb41a7fc640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb41c1f1af0 0x7fb41c1f1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.145+0000 7fb41a7fc640 1 -- 172.21.15.81:0/79243923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c0c7070 con 0x7fb41c14e250 2023-11-08T16:29:44.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.145+0000 7fb41a7fc640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb41c14e250 0x7fb41c0c4a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb40402c3b0 tx=0x7fb404001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:44.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.146+0000 7fb4137fe640 1 -- 172.21.15.81:0/79243923 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb40403c040 con 0x7fb41c14e250 2023-11-08T16:29:44.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.146+0000 7fb4137fe640 1 -- 172.21.15.81:0/79243923 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb40402cc40 con 0x7fb41c14e250 2023-11-08T16:29:44.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.146+0000 7fb4137fe640 1 -- 172.21.15.81:0/79243923 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb404007070 con 0x7fb41c14e250 2023-11-08T16:29:44.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.146+0000 7fb4214e7640 1 -- 172.21.15.81:0/79243923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb41c0c7350 con 0x7fb41c14e250 2023-11-08T16:29:44.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.146+0000 7fb4214e7640 1 -- 172.21.15.81:0/79243923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb41c14e6d0 con 0x7fb41c14e250 2023-11-08T16:29:44.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.147+0000 7fb4214e7640 1 -- 172.21.15.81:0/79243923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb41c04ece0 con 0x7fb41c14e250 2023-11-08T16:29:44.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.148+0000 7fb4137fe640 1 -- 172.21.15.81:0/79243923 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb404032070 con 0x7fb41c14e250 2023-11-08T16:29:44.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.149+0000 7fb4137fe640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb3fc06ae70 0x7fb3fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.149+0000 7fb4137fe640 1 -- 172.21.15.81:0/79243923 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb4040aa3c0 con 0x7fb41c14e250 2023-11-08T16:29:44.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.149+0000 7fb41affd640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb3fc06ae70 0x7fb3fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.150+0000 7fb41affd640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb3fc06ae70 0x7fb3fc06d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0073f0 tx=0x7fb40c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:44.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.153+0000 7fb4137fe640 1 -- 172.21.15.81:0/79243923 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb404075470 con 0x7fb41c14e250 2023-11-08T16:29:44.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.327+0000 7fb4214e7640 1 -- 172.21.15.81:0/79243923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb41c06de70 con 0x7fb41c14e250 2023-11-08T16:29:44.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.327+0000 7fb4137fe640 1 -- 172.21.15.81:0/79243923 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb404078a90 con 0x7fb41c14e250 2023-11-08T16:29:44.330 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:44.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 -- 172.21.15.81:0/79243923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb3fc06ae70 msgr2=0x7fb3fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb3fc06ae70 0x7fb3fc06d330 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0073f0 tx=0x7fb40c00b040 comp rx=0 tx=0).stop 2023-11-08T16:29:44.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 -- 172.21.15.81:0/79243923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb41c14e250 msgr2=0x7fb41c0c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb41c14e250 0x7fb41c0c4a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb40402c3b0 tx=0x7fb404001950 comp rx=0 tx=0).stop 2023-11-08T16:29:44.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 -- 172.21.15.81:0/79243923 shutdown_connections 2023-11-08T16:29:44.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb3fc06ae70 0x7fb3fc06d330 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb41c1f1af0 0x7fb41c1f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb41c14e250 0x7fb41c0c4a20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 --2- 172.21.15.81:0/79243923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb408003fb0 0x7fb41c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 -- 172.21.15.81:0/79243923 >> 172.21.15.81:0/79243923 conn(0x7fb41c065df0 msgr2=0x7fb41c0ce830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:44.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 -- 172.21.15.81:0/79243923 shutdown_connections 2023-11-08T16:29:44.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.330+0000 7fb4214e7640 1 -- 172.21.15.81:0/79243923 wait complete. 2023-11-08T16:29:44.362 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_few_files (tasks.cephfs.test_data_scan.TestDataScan) 2023-11-08T16:29:44.364 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.580+0000 7f95ecc64640 1 Processor -- start 2023-11-08T16:29:44.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.581+0000 7f95ecc64640 1 -- start start 2023-11-08T16:29:44.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.582+0000 7f95ecc64640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95e80ca480 0x7f95e815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.582+0000 7f95ecc64640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f95e815ea60 0x7f95e815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.582+0000 7f95ecc64640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95e806b630 con 0x7f95e80ade50 2023-11-08T16:29:44.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.582+0000 7f95ecc64640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95e80b1860 con 0x7f95e815ea60 2023-11-08T16:29:44.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.582+0000 7f95ecc64640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80ca8e0 con 0x7f95e80ca480 2023-11-08T16:29:44.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.582+0000 7f95e6d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f95e815ea60 0x7f95e815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.582+0000 7f95e6d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f95e815ea60 0x7f95e815eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55718/0 (socket says 172.21.15.81:55718) 2023-11-08T16:29:44.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.582+0000 7f95e659c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f95e80ade50 0x7f95e815a680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35968/0 (socket says 172.21.15.81:35968) 2023-11-08T16:29:44.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.582+0000 7f95e6d9d640 1 -- 172.21.15.81:0/2496910916 learned_addr learned my addr 172.21.15.81:0/2496910916 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:44.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.583+0000 7f95dffff640 1 --2- 172.21.15.81:0/2496910916 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95e80ca480 0x7f95e815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.583+0000 7f95e6d9d640 1 -- 172.21.15.81:0/2496910916 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95e80ca480 msgr2=0x7f95e815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:44.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.583+0000 7f95e6d9d640 1 --2- 172.21.15.81:0/2496910916 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95e80ca480 0x7f95e815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.583+0000 7f95e6d9d640 1 -- 172.21.15.81:0/2496910916 >> v1:172.21.15.81:6789/0 conn(0x7f95e80ade50 legacy=0x7f95e815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.583+0000 7f95e5d9b640 1 -- 172.21.15.81:0/2496910916 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552290553 0 0) 0x7f95e806b630 con 0x7f95e80ade50 2023-11-08T16:29:44.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.583+0000 7f95e6d9d640 1 -- 172.21.15.81:0/2496910916 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e80cabb0 con 0x7f95e815ea60 2023-11-08T16:29:44.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.584+0000 7f95e6d9d640 1 --2- 172.21.15.81:0/2496910916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95e815ea60 0x7f95e815eef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009e50 tx=0x7f95e002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e88aefb651b0e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:44.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.584+0000 7f95e5d9b640 1 -- 172.21.15.81:0/2496910916 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e002cb00 con 0x7f95e815ea60 2023-11-08T16:29:44.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.584+0000 7f95e5d9b640 1 -- 172.21.15.81:0/2496910916 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e002cc60 con 0x7f95e815ea60 2023-11-08T16:29:44.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.584+0000 7f95e5d9b640 1 -- 172.21.15.81:0/2496910916 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e00343f0 con 0x7f95e815ea60 2023-11-08T16:29:44.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.585+0000 7f95ecc64640 1 -- 172.21.15.81:0/2496910916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95e815ea60 msgr2=0x7f95e815eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.585+0000 7f95ecc64640 1 --2- 172.21.15.81:0/2496910916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95e815ea60 0x7f95e815eef0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009e50 tx=0x7f95e002bf60 comp rx=0 tx=0).stop 2023-11-08T16:29:44.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.585+0000 7f95ecc64640 1 -- 172.21.15.81:0/2496910916 shutdown_connections 2023-11-08T16:29:44.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.585+0000 7f95ecc64640 1 --2- 172.21.15.81:0/2496910916 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95e815ea60 0x7f95e815eef0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.585+0000 7f95ecc64640 1 --2- 172.21.15.81:0/2496910916 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95e80ca480 0x7f95e815ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.585+0000 7f95ecc64640 1 -- 172.21.15.81:0/2496910916 >> 172.21.15.81:0/2496910916 conn(0x7f95e80cb490 msgr2=0x7f95e80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:44.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.585+0000 7f95ecc64640 1 -- 172.21.15.81:0/2496910916 shutdown_connections 2023-11-08T16:29:44.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.585+0000 7f95ecc64640 1 -- 172.21.15.81:0/2496910916 wait complete. 2023-11-08T16:29:44.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.586+0000 7f95ecc64640 1 Processor -- start 2023-11-08T16:29:44.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.587+0000 7f95ecc64640 1 -- start start 2023-11-08T16:29:44.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.587+0000 7f95ecc64640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95e80ade50 0x7f95e81495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.587+0000 7f95ecc64640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95e80ca480 0x7f95e80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.587+0000 7f95ecc64640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95e815ea60 0x7f95e8148150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95ecc64640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e8147cc0 con 0x7f95e80ca480 2023-11-08T16:29:44.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95ecc64640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e8149b20 con 0x7f95e815ea60 2023-11-08T16:29:44.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95ecc64640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95e80c55e0 con 0x7f95e80ade50 2023-11-08T16:29:44.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95dffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95e80ca480 0x7f95e80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95dffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95e80ca480 0x7f95e80c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55724/0 (socket says 172.21.15.81:55724) 2023-11-08T16:29:44.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95dffff640 1 -- 172.21.15.81:0/3431681094 learned_addr learned my addr 172.21.15.81:0/3431681094 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:44.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95dffff640 1 -- 172.21.15.81:0/3431681094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95e80ade50 msgr2=0x7f95e81495e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95e659c640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95e80ade50 0x7f95e81495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95dffff640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95e80ade50 0x7f95e81495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95dffff640 1 -- 172.21.15.81:0/3431681094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95e815ea60 msgr2=0x7f95e8148150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95dffff640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95e815ea60 0x7f95e8148150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95dffff640 1 -- 172.21.15.81:0/3431681094 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e0009b30 con 0x7f95e80ca480 2023-11-08T16:29:44.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.588+0000 7f95dffff640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95e80ca480 0x7f95e80c50a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f95d000d830 tx=0x7f95d000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:44.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.589+0000 7f95deffd640 1 -- 172.21.15.81:0/3431681094 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000b6c0 con 0x7f95e80ca480 2023-11-08T16:29:44.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.589+0000 7f95deffd640 1 -- 172.21.15.81:0/3431681094 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95d000b820 con 0x7f95e80ca480 2023-11-08T16:29:44.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.589+0000 7f95deffd640 1 -- 172.21.15.81:0/3431681094 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95d000bbb0 con 0x7f95e80ca480 2023-11-08T16:29:44.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.589+0000 7f95ecc64640 1 -- 172.21.15.81:0/3431681094 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e81470b0 con 0x7f95e80ca480 2023-11-08T16:29:44.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.589+0000 7f95ecc64640 1 -- 172.21.15.81:0/3431681094 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e8147670 con 0x7f95e80ca480 2023-11-08T16:29:44.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.590+0000 7f95ecc64640 1 -- 172.21.15.81:0/3431681094 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e8147e70 con 0x7f95e80ca480 2023-11-08T16:29:44.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.591+0000 7f95deffd640 1 -- 172.21.15.81:0/3431681094 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f95d001e460 con 0x7f95e80ca480 2023-11-08T16:29:44.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.592+0000 7f95deffd640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95c806ae70 0x7f95c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:44.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.592+0000 7f95deffd640 1 -- 172.21.15.81:0/3431681094 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f95d0089ec0 con 0x7f95e80ca480 2023-11-08T16:29:44.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.592+0000 7f95e659c640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95c806ae70 0x7f95c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:44.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.593+0000 7f95e659c640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95c806ae70 0x7f95c806d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f95d8007fd0 tx=0x7f95d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:44.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.596+0000 7f95deffd640 1 -- 172.21.15.81:0/3431681094 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95e8147e70 con 0x7f95e80ca480 2023-11-08T16:29:44.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.785+0000 7f95ecc64640 1 -- 172.21.15.81:0/3431681094 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95e8147e70 con 0x7f95e80ca480 2023-11-08T16:29:44.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.788+0000 7f95deffd640 1 -- 172.21.15.81:0/3431681094 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f95e8147e70 con 0x7f95e80ca480 2023-11-08T16:29:44.790 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:44.791 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:44.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 -- 172.21.15.81:0/3431681094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95c806ae70 msgr2=0x7f95c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95c806ae70 0x7f95c806d330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f95d8007fd0 tx=0x7f95d8009040 comp rx=0 tx=0).stop 2023-11-08T16:29:44.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 -- 172.21.15.81:0/3431681094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95e80ca480 msgr2=0x7f95e80c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:44.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95e80ca480 0x7f95e80c50a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f95d000d830 tx=0x7f95d000dcf0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 -- 172.21.15.81:0/3431681094 shutdown_connections 2023-11-08T16:29:44.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95c806ae70 0x7f95c806d330 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95e815ea60 0x7f95e8148150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95e80ca480 0x7f95e80c50a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 --2- 172.21.15.81:0/3431681094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95e80ade50 0x7f95e81495e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:44.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 -- 172.21.15.81:0/3431681094 >> 172.21.15.81:0/3431681094 conn(0x7f95e80cb490 msgr2=0x7f95e80ae230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:44.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 -- 172.21.15.81:0/3431681094 shutdown_connections 2023-11-08T16:29:44.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:44.791+0000 7f95ecc64640 1 -- 172.21.15.81:0/3431681094 wait complete. 2023-11-08T16:29:44.813 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:44.814 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:45.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.010+0000 7f2cf09ce640 1 Processor -- start 2023-11-08T16:29:45.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.010+0000 7f2cf09ce640 1 -- start start 2023-11-08T16:29:45.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.011+0000 7f2cf09ce640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cec0ce390 0x7f2cec15e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.011+0000 7f2cf09ce640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2cec06be20 0x7f2cec0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.011+0000 7f2cf09ce640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cec15e8d0 con 0x7f2cec0c9200 2023-11-08T16:29:45.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.011+0000 7f2cf09ce640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cec0ca2f0 con 0x7f2cec06be20 2023-11-08T16:29:45.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.011+0000 7f2cf09ce640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0ce770 con 0x7f2cec0ce390 2023-11-08T16:29:45.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.012+0000 7f2ce3fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2cec0c9200 0x7f2cec06b710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35976/0 (socket says 172.21.15.81:35976) 2023-11-08T16:29:45.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.012+0000 7f2ce3fff640 1 -- 172.21.15.81:0/1181146298 learned_addr learned my addr 172.21.15.81:0/1181146298 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:45.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.012+0000 7f2cead9d640 1 --2- 172.21.15.81:0/1181146298 >> v2:172.21.15.81:3300/0 conn(0x7f2cec06be20 0x7f2cec0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.012+0000 7f2cea59c640 1 --2- 172.21.15.81:0/1181146298 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cec0ce390 0x7f2cec15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.012+0000 7f2cead9d640 1 -- 172.21.15.81:0/1181146298 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cec0ce390 msgr2=0x7f2cec15e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:45.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.012+0000 7f2cead9d640 1 --2- 172.21.15.81:0/1181146298 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cec0ce390 0x7f2cec15e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.012+0000 7f2cead9d640 1 -- 172.21.15.81:0/1181146298 >> v1:172.21.15.81:6789/0 conn(0x7f2cec0c9200 legacy=0x7f2cec06b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.012+0000 7f2cead9d640 1 -- 172.21.15.81:0/1181146298 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0c96e0 con 0x7f2cec06be20 2023-11-08T16:29:45.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.012+0000 7f2ce9d9b640 1 -- 172.21.15.81:0/1181146298 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737637494 0 0) 0x7f2cd4001110 con 0x7f2cec0c9200 2023-11-08T16:29:45.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.013+0000 7f2cead9d640 1 --2- 172.21.15.81:0/1181146298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cec06be20 0x7f2cec0ac8b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009d90 tx=0x7f2ce402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552779fad0d61180 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:45.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.014+0000 7f2ce9d9b640 1 -- 172.21.15.81:0/1181146298 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce402cce0 con 0x7f2cec06be20 2023-11-08T16:29:45.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.014+0000 7f2ce9d9b640 1 -- 172.21.15.81:0/1181146298 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ce402ce40 con 0x7f2cec06be20 2023-11-08T16:29:45.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.014+0000 7f2ce9d9b640 1 -- 172.21.15.81:0/1181146298 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4034660 con 0x7f2cec06be20 2023-11-08T16:29:45.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.014+0000 7f2cf09ce640 1 -- 172.21.15.81:0/1181146298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cec06be20 msgr2=0x7f2cec0ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.014+0000 7f2cf09ce640 1 --2- 172.21.15.81:0/1181146298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cec06be20 0x7f2cec0ac8b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2ce4009d90 tx=0x7f2ce402bf30 comp rx=0 tx=0).stop 2023-11-08T16:29:45.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.014+0000 7f2cf09ce640 1 -- 172.21.15.81:0/1181146298 shutdown_connections 2023-11-08T16:29:45.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.014+0000 7f2cf09ce640 1 --2- 172.21.15.81:0/1181146298 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cec06be20 0x7f2cec0ac8b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.014+0000 7f2cf09ce640 1 --2- 172.21.15.81:0/1181146298 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cec0ce390 0x7f2cec15e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.014+0000 7f2cf09ce640 1 -- 172.21.15.81:0/1181146298 >> 172.21.15.81:0/1181146298 conn(0x7f2cec0cb580 msgr2=0x7f2cec0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:45.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.014+0000 7f2cf09ce640 1 -- 172.21.15.81:0/1181146298 shutdown_connections 2023-11-08T16:29:45.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.014+0000 7f2cf09ce640 1 -- 172.21.15.81:0/1181146298 wait complete. 2023-11-08T16:29:45.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.016+0000 7f2cf09ce640 1 Processor -- start 2023-11-08T16:29:45.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.016+0000 7f2cf09ce640 1 -- start start 2023-11-08T16:29:45.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.017+0000 7f2cf09ce640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cec06be20 0x7f2cec1f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.017+0000 7f2cf09ce640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cec0c9200 0x7f2cec159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.017+0000 7f2cf09ce640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2cec0ce390 0x7f2cec0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.017+0000 7f2cf09ce640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0b2210 con 0x7f2cec0c9200 2023-11-08T16:29:45.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.017+0000 7f2cf09ce640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec15a2d0 con 0x7f2cec0ce390 2023-11-08T16:29:45.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.017+0000 7f2cf09ce640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec0b2460 con 0x7f2cec06be20 2023-11-08T16:29:45.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.017+0000 7f2ce3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cec0c9200 0x7f2cec159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.017+0000 7f2ce3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cec0c9200 0x7f2cec159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55734/0 (socket says 172.21.15.81:55734) 2023-11-08T16:29:45.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.017+0000 7f2ce3fff640 1 -- 172.21.15.81:0/3181037948 learned_addr learned my addr 172.21.15.81:0/3181037948 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:45.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2cea59c640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cec06be20 0x7f2cec1f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2ce3fff640 1 -- 172.21.15.81:0/3181037948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cec06be20 msgr2=0x7f2cec1f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2ce3fff640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cec06be20 0x7f2cec1f33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2ce3fff640 1 -- 172.21.15.81:0/3181037948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2cec0ce390 msgr2=0x7f2cec0c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2ce3fff640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2cec0ce390 0x7f2cec0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2ce3fff640 1 -- 172.21.15.81:0/3181037948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce4009a70 con 0x7f2cec0c9200 2023-11-08T16:29:45.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2ce3fff640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cec0c9200 0x7f2cec159d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400c780 tx=0x7f2cd400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:45.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2ce2ffd640 1 -- 172.21.15.81:0/3181037948 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40072d0 con 0x7f2cec0c9200 2023-11-08T16:29:45.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2ce2ffd640 1 -- 172.21.15.81:0/3181037948 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd4007450 con 0x7f2cec0c9200 2023-11-08T16:29:45.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2cf09ce640 1 -- 172.21.15.81:0/3181037948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cec0b2740 con 0x7f2cec0c9200 2023-11-08T16:29:45.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2ce2ffd640 1 -- 172.21.15.81:0/3181037948 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd40047e0 con 0x7f2cec0c9200 2023-11-08T16:29:45.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.018+0000 7f2cf09ce640 1 -- 172.21.15.81:0/3181037948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cec0b2d00 con 0x7f2cec0c9200 2023-11-08T16:29:45.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.019+0000 7f2cf09ce640 1 -- 172.21.15.81:0/3181037948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cec0b2f20 con 0x7f2cec0c9200 2023-11-08T16:29:45.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.020+0000 7f2ce2ffd640 1 -- 172.21.15.81:0/3181037948 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2cd4004960 con 0x7f2cec0c9200 2023-11-08T16:29:45.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.021+0000 7f2ce2ffd640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ccc06ae70 0x7f2ccc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.021+0000 7f2ce2ffd640 1 -- 172.21.15.81:0/3181037948 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2cd40889f0 con 0x7f2cec0c9200 2023-11-08T16:29:45.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.021+0000 7f2cea59c640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ccc06ae70 0x7f2ccc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.022+0000 7f2cea59c640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ccc06ae70 0x7f2ccc06d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc006fd0 tx=0x7f2cdc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:45.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.026+0000 7f2ce2ffd640 1 -- 172.21.15.81:0/3181037948 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2cd4053aa0 con 0x7f2cec0c9200 2023-11-08T16:29:45.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.200+0000 7f2cf09ce640 1 -- 172.21.15.81:0/3181037948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cec15e8d0 con 0x7f2cec0c9200 2023-11-08T16:29:45.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.201+0000 7f2ce2ffd640 1 -- 172.21.15.81:0/3181037948 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cec15e8d0 con 0x7f2cec0c9200 2023-11-08T16:29:45.203 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:45.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 -- 172.21.15.81:0/3181037948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ccc06ae70 msgr2=0x7f2ccc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ccc06ae70 0x7f2ccc06d330 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc006fd0 tx=0x7f2cdc008040 comp rx=0 tx=0).stop 2023-11-08T16:29:45.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 -- 172.21.15.81:0/3181037948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cec0c9200 msgr2=0x7f2cec159d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cec0c9200 0x7f2cec159d90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400c780 tx=0x7f2cd400cc40 comp rx=0 tx=0).stop 2023-11-08T16:29:45.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 -- 172.21.15.81:0/3181037948 shutdown_connections 2023-11-08T16:29:45.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ccc06ae70 0x7f2ccc06d330 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2cec0ce390 0x7f2cec0c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2cec0c9200 0x7f2cec159d90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 --2- 172.21.15.81:0/3181037948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2cec06be20 0x7f2cec1f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 -- 172.21.15.81:0/3181037948 >> 172.21.15.81:0/3181037948 conn(0x7f2cec0cb580 msgr2=0x7f2cec0af370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:45.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 -- 172.21.15.81:0/3181037948 shutdown_connections 2023-11-08T16:29:45.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.204+0000 7f2cf09ce640 1 -- 172.21.15.81:0/3181037948 wait complete. 2023-11-08T16:29:45.223 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:45.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.430+0000 7f7e631a5640 1 Processor -- start 2023-11-08T16:29:45.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.430+0000 7f7e631a5640 1 -- start start 2023-11-08T16:29:45.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e631a5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7e5c15e1a0 0x7f7e5c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e631a5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7e5c0c97b0 0x7f7e5c06fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e631a5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e5c15e580 con 0x7f7e5c070360 2023-11-08T16:29:45.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e631a5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5c0ca830 con 0x7f7e5c15e1a0 2023-11-08T16:29:45.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e631a5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e5c070770 con 0x7f7e5c0c97b0 2023-11-08T16:29:45.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e621a3640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7e5c15e1a0 0x7f7e5c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e629a4640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f7e5c070360 0x7f7e5c15e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:35984/0 (socket says 172.21.15.81:35984) 2023-11-08T16:29:45.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e629a4640 1 -- 172.21.15.81:0/3686857588 learned_addr learned my addr 172.21.15.81:0/3686857588 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:45.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e619a2640 1 --2- 172.21.15.81:0/3686857588 >> v2:172.21.15.93:3300/0 conn(0x7f7e5c0c97b0 0x7f7e5c06fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e621a3640 1 -- 172.21.15.81:0/3686857588 >> v2:172.21.15.93:3300/0 conn(0x7f7e5c0c97b0 msgr2=0x7f7e5c06fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:45.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e621a3640 1 --2- 172.21.15.81:0/3686857588 >> v2:172.21.15.93:3300/0 conn(0x7f7e5c0c97b0 0x7f7e5c06fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e621a3640 1 -- 172.21.15.81:0/3686857588 >> v1:172.21.15.81:6789/0 conn(0x7f7e5c070360 legacy=0x7f7e5c15e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e621a3640 1 -- 172.21.15.81:0/3686857588 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e5c0c9cb0 con 0x7f7e5c15e1a0 2023-11-08T16:29:45.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.432+0000 7f7e619a2640 1 --2- 172.21.15.81:0/3686857588 >> v2:172.21.15.93:3300/0 conn(0x7f7e5c0c97b0 0x7f7e5c06fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:45.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.433+0000 7f7e621a3640 1 --2- 172.21.15.81:0/3686857588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e5c15e1a0 0x7f7e5c0caa30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c06fa60 tx=0x7f7e5802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c3bd682882aabe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:45.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.434+0000 7f7e611a1640 1 -- 172.21.15.81:0/3686857588 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5802e070 con 0x7f7e5c15e1a0 2023-11-08T16:29:45.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.434+0000 7f7e611a1640 1 -- 172.21.15.81:0/3686857588 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e580315a0 con 0x7f7e5c15e1a0 2023-11-08T16:29:45.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.434+0000 7f7e611a1640 1 -- 172.21.15.81:0/3686857588 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58031930 con 0x7f7e5c15e1a0 2023-11-08T16:29:45.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.434+0000 7f7e631a5640 1 -- 172.21.15.81:0/3686857588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e5c15e1a0 msgr2=0x7f7e5c0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.434+0000 7f7e631a5640 1 --2- 172.21.15.81:0/3686857588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e5c15e1a0 0x7f7e5c0caa30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c06fa60 tx=0x7f7e5802b840 comp rx=0 tx=0).stop 2023-11-08T16:29:45.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.434+0000 7f7e631a5640 1 -- 172.21.15.81:0/3686857588 shutdown_connections 2023-11-08T16:29:45.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.434+0000 7f7e631a5640 1 --2- 172.21.15.81:0/3686857588 >> v2:172.21.15.93:3300/0 conn(0x7f7e5c0c97b0 0x7f7e5c06fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.434+0000 7f7e631a5640 1 --2- 172.21.15.81:0/3686857588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e5c15e1a0 0x7f7e5c0caa30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.434+0000 7f7e631a5640 1 -- 172.21.15.81:0/3686857588 >> 172.21.15.81:0/3686857588 conn(0x7f7e5c0ce8d0 msgr2=0x7f7e5c0cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:45.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.435+0000 7f7e631a5640 1 -- 172.21.15.81:0/3686857588 shutdown_connections 2023-11-08T16:29:45.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.435+0000 7f7e631a5640 1 -- 172.21.15.81:0/3686857588 wait complete. 2023-11-08T16:29:45.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.436+0000 7f7e631a5640 1 Processor -- start 2023-11-08T16:29:45.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.436+0000 7f7e631a5640 1 -- start start 2023-11-08T16:29:45.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.437+0000 7f7e631a5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e5c070360 0x7f7e5c0cd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.437+0000 7f7e631a5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e5c0c97b0 0x7f7e5c0abd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.437+0000 7f7e631a5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c15e1a0 0x7f7e5c0aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.437+0000 7f7e631a5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0aa9e0 con 0x7f7e5c070360 2023-11-08T16:29:45.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.437+0000 7f7e631a5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0aac80 con 0x7f7e5c0c97b0 2023-11-08T16:29:45.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.437+0000 7f7e631a5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e5c0aaf60 con 0x7f7e5c15e1a0 2023-11-08T16:29:45.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.438+0000 7f7e621a3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e5c070360 0x7f7e5c0cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.438+0000 7f7e621a3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e5c070360 0x7f7e5c0cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55750/0 (socket says 172.21.15.81:55750) 2023-11-08T16:29:45.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.438+0000 7f7e621a3640 1 -- 172.21.15.81:0/1931328724 learned_addr learned my addr 172.21.15.81:0/1931328724 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:45.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.438+0000 7f7e619a2640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e5c0c97b0 0x7f7e5c0abd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.438+0000 7f7e621a3640 1 -- 172.21.15.81:0/1931328724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c15e1a0 msgr2=0x7f7e5c0aa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.438+0000 7f7e621a3640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c15e1a0 0x7f7e5c0aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.438+0000 7f7e621a3640 1 -- 172.21.15.81:0/1931328724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e5c0c97b0 msgr2=0x7f7e5c0abd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.438+0000 7f7e621a3640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e5c0c97b0 0x7f7e5c0abd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.438+0000 7f7e621a3640 1 -- 172.21.15.81:0/1931328724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e58009400 con 0x7f7e5c070360 2023-11-08T16:29:45.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.438+0000 7f7e621a3640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e5c070360 0x7f7e5c0cd110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7e58000c00 tx=0x7f7e580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:45.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.439+0000 7f7e52ffd640 1 -- 172.21.15.81:0/1931328724 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58032900 con 0x7f7e5c070360 2023-11-08T16:29:45.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.439+0000 7f7e52ffd640 1 -- 172.21.15.81:0/1931328724 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e58032a60 con 0x7f7e5c070360 2023-11-08T16:29:45.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.439+0000 7f7e52ffd640 1 -- 172.21.15.81:0/1931328724 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e58032dd0 con 0x7f7e5c070360 2023-11-08T16:29:45.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.439+0000 7f7e631a5640 1 -- 172.21.15.81:0/1931328724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e5c0ab240 con 0x7f7e5c070360 2023-11-08T16:29:45.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.439+0000 7f7e631a5640 1 -- 172.21.15.81:0/1931328724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e5c0ab800 con 0x7f7e5c070360 2023-11-08T16:29:45.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.440+0000 7f7e631a5640 1 -- 172.21.15.81:0/1931328724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e5c04ece0 con 0x7f7e5c070360 2023-11-08T16:29:45.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.441+0000 7f7e52ffd640 1 -- 172.21.15.81:0/1931328724 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e5804d3f0 con 0x7f7e5c070360 2023-11-08T16:29:45.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.441+0000 7f7e52ffd640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e4006aea0 0x7f7e4006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.442+0000 7f7e52ffd640 1 -- 172.21.15.81:0/1931328724 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7e580ab270 con 0x7f7e5c070360 2023-11-08T16:29:45.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.442+0000 7f7e619a2640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e4006aea0 0x7f7e4006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.442+0000 7f7e619a2640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e4006aea0 0x7f7e4006d360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c006fd0 tx=0x7f7e4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:45.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.446+0000 7f7e52ffd640 1 -- 172.21.15.81:0/1931328724 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e58076320 con 0x7f7e5c070360 2023-11-08T16:29:45.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.639+0000 7f7e631a5640 1 -- 172.21.15.81:0/1931328724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e5c0ae2a0 con 0x7f7e5c070360 2023-11-08T16:29:45.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.639+0000 7f7e52ffd640 1 -- 172.21.15.81:0/1931328724 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e58079940 con 0x7f7e5c070360 2023-11-08T16:29:45.642 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:45.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.642+0000 7f7e631a5640 1 -- 172.21.15.81:0/1931328724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e4006aea0 msgr2=0x7f7e4006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.642+0000 7f7e631a5640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e4006aea0 0x7f7e4006d360 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c006fd0 tx=0x7f7e4c008040 comp rx=0 tx=0).stop 2023-11-08T16:29:45.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.642+0000 7f7e631a5640 1 -- 172.21.15.81:0/1931328724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e5c070360 msgr2=0x7f7e5c0cd110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.642+0000 7f7e631a5640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e5c070360 0x7f7e5c0cd110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f7e58000c00 tx=0x7f7e580046c0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.642+0000 7f7e631a5640 1 -- 172.21.15.81:0/1931328724 shutdown_connections 2023-11-08T16:29:45.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.642+0000 7f7e631a5640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e4006aea0 0x7f7e4006d360 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.642+0000 7f7e631a5640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e5c15e1a0 0x7f7e5c0aa4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.642+0000 7f7e631a5640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e5c0c97b0 0x7f7e5c0abd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.642+0000 7f7e631a5640 1 --2- 172.21.15.81:0/1931328724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e5c070360 0x7f7e5c0cd110 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.642+0000 7f7e631a5640 1 -- 172.21.15.81:0/1931328724 >> 172.21.15.81:0/1931328724 conn(0x7f7e5c0ce8d0 msgr2=0x7f7e5c0c48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:45.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.643+0000 7f7e631a5640 1 -- 172.21.15.81:0/1931328724 shutdown_connections 2023-11-08T16:29:45.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.643+0000 7f7e631a5640 1 -- 172.21.15.81:0/1931328724 wait complete. 2023-11-08T16:29:45.664 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:45.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.879+0000 7f08a1161640 1 Processor -- start 2023-11-08T16:29:45.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.879+0000 7f08a1161640 1 -- start start 2023-11-08T16:29:45.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.881+0000 7f08a1161640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f089c0ce390 0x7f089c15e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.881+0000 7f08a1161640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f089c06be20 0x7f089c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.881+0000 7f08a1161640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f089c15e8d0 con 0x7f089c0c9200 2023-11-08T16:29:45.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.881+0000 7f08a1161640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089c0ca2f0 con 0x7f089c06be20 2023-11-08T16:29:45.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.881+0000 7f08a1161640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f089c0ce770 con 0x7f089c0ce390 2023-11-08T16:29:45.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.881+0000 7f089b59e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f089c06be20 0x7f089c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.881+0000 7f089b59e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f089c06be20 0x7f089c0ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55752/0 (socket says 172.21.15.81:55752) 2023-11-08T16:29:45.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.881+0000 7f089b59e640 1 -- 172.21.15.81:0/3159797351 learned_addr learned my addr 172.21.15.81:0/3159797351 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:45.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.881+0000 7f089b59e640 1 -- 172.21.15.81:0/3159797351 >> v2:172.21.15.93:3300/0 conn(0x7f089c0ce390 msgr2=0x7f089c15e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:45.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.881+0000 7f089ad9d640 1 --2- 172.21.15.81:0/3159797351 >> v2:172.21.15.93:3300/0 conn(0x7f089c0ce390 0x7f089c15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.882+0000 7f089b59e640 1 --2- 172.21.15.81:0/3159797351 >> v2:172.21.15.93:3300/0 conn(0x7f089c0ce390 0x7f089c15e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.882+0000 7f089b59e640 1 -- 172.21.15.81:0/3159797351 >> v1:172.21.15.81:6789/0 conn(0x7f089c0c9200 legacy=0x7f089c06b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.882+0000 7f0899d9b640 1 -- 172.21.15.81:0/3159797351 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126633451 0 0) 0x7f0884001110 con 0x7f089c0c9200 2023-11-08T16:29:45.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.882+0000 7f089b59e640 1 -- 172.21.15.81:0/3159797351 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f089c0c96e0 con 0x7f089c06be20 2023-11-08T16:29:45.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.882+0000 7f089ad9d640 1 --2- 172.21.15.81:0/3159797351 >> v2:172.21.15.93:3300/0 conn(0x7f089c0ce390 0x7f089c15e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:45.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.883+0000 7f089b59e640 1 --2- 172.21.15.81:0/3159797351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f089c06be20 0x7f089c0ac8b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0894009e50 tx=0x7f089402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d4b3eadf90170 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:45.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.883+0000 7f0899d9b640 1 -- 172.21.15.81:0/3159797351 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089402cb90 con 0x7f089c06be20 2023-11-08T16:29:45.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.883+0000 7f0899d9b640 1 -- 172.21.15.81:0/3159797351 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089402ccf0 con 0x7f089c06be20 2023-11-08T16:29:45.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.883+0000 7f0899d9b640 1 -- 172.21.15.81:0/3159797351 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0894034610 con 0x7f089c06be20 2023-11-08T16:29:45.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.884+0000 7f08a1161640 1 -- 172.21.15.81:0/3159797351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f089c06be20 msgr2=0x7f089c0ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.884+0000 7f08a1161640 1 --2- 172.21.15.81:0/3159797351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f089c06be20 0x7f089c0ac8b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0894009e50 tx=0x7f089402bff0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.884+0000 7f08a1161640 1 -- 172.21.15.81:0/3159797351 shutdown_connections 2023-11-08T16:29:45.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.884+0000 7f08a1161640 1 --2- 172.21.15.81:0/3159797351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f089c06be20 0x7f089c0ac8b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.884+0000 7f08a1161640 1 --2- 172.21.15.81:0/3159797351 >> v2:172.21.15.93:3300/0 conn(0x7f089c0ce390 0x7f089c15e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.884+0000 7f08a1161640 1 -- 172.21.15.81:0/3159797351 >> 172.21.15.81:0/3159797351 conn(0x7f089c0cb580 msgr2=0x7f089c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:45.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.884+0000 7f08a1161640 1 -- 172.21.15.81:0/3159797351 shutdown_connections 2023-11-08T16:29:45.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.884+0000 7f08a1161640 1 -- 172.21.15.81:0/3159797351 wait complete. 2023-11-08T16:29:45.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.885+0000 7f08a1161640 1 Processor -- start 2023-11-08T16:29:45.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.886+0000 7f08a1161640 1 -- start start 2023-11-08T16:29:45.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.886+0000 7f08a1161640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f089c06be20 0x7f089c154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.886+0000 7f08a1161640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f089c0c9200 0x7f089c0c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.886+0000 7f08a1161640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f089c0ce390 0x7f089c159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.886+0000 7f08a1161640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c1553a0 con 0x7f089c0c9200 2023-11-08T16:29:45.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.886+0000 7f08a1161640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089c156fb0 con 0x7f089c0ce390 2023-11-08T16:29:45.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.886+0000 7f08a1161640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089c1563a0 con 0x7f089c06be20 2023-11-08T16:29:45.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.887+0000 7f089a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f089c0c9200 0x7f089c0c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.887+0000 7f089a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f089c0c9200 0x7f089c0c8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55754/0 (socket says 172.21.15.81:55754) 2023-11-08T16:29:45.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.887+0000 7f089a59c640 1 -- 172.21.15.81:0/2591049695 learned_addr learned my addr 172.21.15.81:0/2591049695 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:45.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.887+0000 7f089a59c640 1 -- 172.21.15.81:0/2591049695 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f089c06be20 msgr2=0x7f089c154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.887+0000 7f089a59c640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f089c06be20 0x7f089c154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.887+0000 7f089a59c640 1 -- 172.21.15.81:0/2591049695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f089c0ce390 msgr2=0x7f089c159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:45.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.887+0000 7f089a59c640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f089c0ce390 0x7f089c159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:45.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.887+0000 7f089a59c640 1 -- 172.21.15.81:0/2591049695 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0894009b30 con 0x7f089c0c9200 2023-11-08T16:29:45.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.887+0000 7f089a59c640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f089c0c9200 0x7f089c0c8310 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f088400c780 tx=0x7f088400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:45.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.888+0000 7f08937fe640 1 -- 172.21.15.81:0/2591049695 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884005270 con 0x7f089c0c9200 2023-11-08T16:29:45.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.888+0000 7f08937fe640 1 -- 172.21.15.81:0/2591049695 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0884005410 con 0x7f089c0c9200 2023-11-08T16:29:45.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.888+0000 7f08937fe640 1 -- 172.21.15.81:0/2591049695 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088400a880 con 0x7f089c0c9200 2023-11-08T16:29:45.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.888+0000 7f08a1161640 1 -- 172.21.15.81:0/2591049695 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f089c156680 con 0x7f089c0c9200 2023-11-08T16:29:45.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.888+0000 7f08a1161640 1 -- 172.21.15.81:0/2591049695 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f089c156c40 con 0x7f089c0c9200 2023-11-08T16:29:45.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.889+0000 7f08a1161640 1 -- 172.21.15.81:0/2591049695 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f089c04ece0 con 0x7f089c0c9200 2023-11-08T16:29:45.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.890+0000 7f08937fe640 1 -- 172.21.15.81:0/2591049695 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f088400a9e0 con 0x7f089c0c9200 2023-11-08T16:29:45.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.891+0000 7f08937fe640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f087c06ae70 0x7f087c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:45.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.891+0000 7f08937fe640 1 -- 172.21.15.81:0/2591049695 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f08840889a0 con 0x7f089c0c9200 2023-11-08T16:29:45.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.891+0000 7f089ad9d640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f087c06ae70 0x7f087c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:45.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.892+0000 7f089ad9d640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f087c06ae70 0x7f087c06d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f088c006fd0 tx=0x7f088c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:45.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:45.895+0000 7f08937fe640 1 -- 172.21.15.81:0/2591049695 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0884053a60 con 0x7f089c0c9200 2023-11-08T16:29:46.063 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:29:46.073 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:29:46.074 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:46.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.069+0000 7f08a1161640 1 -- 172.21.15.81:0/2591049695 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f089c15e8d0 con 0x7f089c0c9200 2023-11-08T16:29:46.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.070+0000 7f08937fe640 1 -- 172.21.15.81:0/2591049695 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f089c15e8d0 con 0x7f089c0c9200 2023-11-08T16:29:46.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 -- 172.21.15.81:0/2591049695 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f087c06ae70 msgr2=0x7f087c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:46.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f087c06ae70 0x7f087c06d330 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f088c006fd0 tx=0x7f088c008040 comp rx=0 tx=0).stop 2023-11-08T16:29:46.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 -- 172.21.15.81:0/2591049695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f089c0c9200 msgr2=0x7f089c0c8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:46.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f089c0c9200 0x7f089c0c8310 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f088400c780 tx=0x7f088400cc40 comp rx=0 tx=0).stop 2023-11-08T16:29:46.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 -- 172.21.15.81:0/2591049695 shutdown_connections 2023-11-08T16:29:46.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f087c06ae70 0x7f087c06d330 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f089c0ce390 0x7f089c159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f089c0c9200 0x7f089c0c8310 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 --2- 172.21.15.81:0/2591049695 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f089c06be20 0x7f089c154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 -- 172.21.15.81:0/2591049695 >> 172.21.15.81:0/2591049695 conn(0x7f089c0cb580 msgr2=0x7f089c0ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 -- 172.21.15.81:0/2591049695 shutdown_connections 2023-11-08T16:29:46.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.073+0000 7f08a1161640 1 -- 172.21.15.81:0/2591049695 wait complete. 2023-11-08T16:29:46.092 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files' 2023-11-08T16:29:46.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.332+0000 7ff0c3589640 1 Processor -- start 2023-11-08T16:29:46.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.333+0000 7ff0c3589640 1 -- start start 2023-11-08T16:29:46.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.334+0000 7ff0c3589640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff0bc0adcd0 0x7ff0bc0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:46.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.334+0000 7ff0c3589640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc0c9a70 con 0x7ff0bc070050 2023-11-08T16:29:46.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.334+0000 7ff0c3589640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0bc0cc3e0 con 0x7ff0bc0cc5e0 2023-11-08T16:29:46.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.334+0000 7ff0c3589640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0bc06d070 con 0x7ff0bc0adcd0 2023-11-08T16:29:46.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.334+0000 7ff0c1b26640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff0bc070050 0x7ff0bc0704e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:36000/0 (socket says 172.21.15.81:36000) 2023-11-08T16:29:46.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.334+0000 7ff0c1b26640 1 -- 172.21.15.81:0/3008488023 learned_addr learned my addr 172.21.15.81:0/3008488023 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:46.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0c1325640 1 --2- 172.21.15.81:0/3008488023 >> v2:172.21.15.93:3300/0 conn(0x7ff0bc0adcd0 0x7ff0bc0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:46.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267878188 0 0) 0x7ff0bc0c9a70 con 0x7ff0bc070050 2023-11-08T16:29:46.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0a8005730 con 0x7ff0bc070050 2023-11-08T16:29:46.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 876434357 0 0) 0x7ff0a8005730 con 0x7ff0bc070050 2023-11-08T16:29:46.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 >> v2:172.21.15.93:3300/0 conn(0x7ff0bc0adcd0 msgr2=0x7ff0bc0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:46.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 --2- 172.21.15.81:0/3008488023 >> v2:172.21.15.93:3300/0 conn(0x7ff0bc0adcd0 0x7ff0bc0ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 >> v1:172.21.15.93:6789/0 conn(0x7ff0bc0cc5e0 legacy=0x7ff0bc0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:46.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc06d350 con 0x7ff0bc070050 2023-11-08T16:29:46.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ff0b40019f0 con 0x7ff0bc070050 2023-11-08T16:29:46.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 >> v1:172.21.15.81:6789/0 conn(0x7ff0bc070050 legacy=0x7ff0bc0704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:46.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 --2- 172.21.15.81:0/3008488023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a8005c60 0x7ff0a8006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:46.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 --2- 172.21.15.81:0/3008488023 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a80069d0 0x7ff0a8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:46.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 --2- 172.21.15.81:0/3008488023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a80073e0 0x7ff0a800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:46.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc06d070 con 0x7ff0a80073e0 2023-11-08T16:29:46.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a800e310 con 0x7ff0a8005c60 2023-11-08T16:29:46.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.335+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a800e4d0 con 0x7ff0a80069d0 2023-11-08T16:29:46.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.336+0000 7ff0c1b26640 1 --2- 172.21.15.81:0/3008488023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a80073e0 0x7ff0a800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:46.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.336+0000 7ff0c1325640 1 --2- 172.21.15.81:0/3008488023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a8005c60 0x7ff0a8006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:46.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.336+0000 7ff0c0b24640 1 --2- 172.21.15.81:0/3008488023 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a80069d0 0x7ff0a8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:46.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.336+0000 7ff0c1b26640 1 -- 172.21.15.81:0/3008488023 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a80069d0 msgr2=0x7ff0a8006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:46.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.336+0000 7ff0c1b26640 1 --2- 172.21.15.81:0/3008488023 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a80069d0 0x7ff0a8006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.336+0000 7ff0c1b26640 1 -- 172.21.15.81:0/3008488023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a8005c60 msgr2=0x7ff0a8006080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:46.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.336+0000 7ff0c1b26640 1 --2- 172.21.15.81:0/3008488023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a8005c60 0x7ff0a8006080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.336+0000 7ff0c1b26640 1 -- 172.21.15.81:0/3008488023 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a8005730 con 0x7ff0a80073e0 2023-11-08T16:29:46.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.337+0000 7ff0c1b26640 1 -- 172.21.15.81:0/3008488023 reap_dead start 2023-11-08T16:29:46.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.337+0000 7ff0c1b26640 1 --2- 172.21.15.81:0/3008488023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a80073e0 0x7ff0a800db90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400a800 tx=0x7ff0a8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11d08138d891300 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:46.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.337+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b40072d0 con 0x7ff0a80073e0 2023-11-08T16:29:46.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.337+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b402f070 con 0x7ff0a80073e0 2023-11-08T16:29:46.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.337+0000 7ff0b3fff640 1 -- 172.21.15.81:0/3008488023 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4007670 con 0x7ff0a80073e0 2023-11-08T16:29:46.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.338+0000 7ff0c3589640 1 -- 172.21.15.81:0/3008488023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a80073e0 msgr2=0x7ff0a800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:46.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.338+0000 7ff0c3589640 1 --2- 172.21.15.81:0/3008488023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a80073e0 0x7ff0a800db90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff0b400a800 tx=0x7ff0a8007a00 comp rx=0 tx=0).stop 2023-11-08T16:29:46.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.338+0000 7ff0c3589640 1 -- 172.21.15.81:0/3008488023 shutdown_connections 2023-11-08T16:29:46.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.338+0000 7ff0c3589640 1 --2- 172.21.15.81:0/3008488023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0a80073e0 0x7ff0a800db90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.338+0000 7ff0c3589640 1 -- 172.21.15.81:0/3008488023 >> 172.21.15.81:0/3008488023 conn(0x7ff0bc0cb490 msgr2=0x7ff0bc0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:46.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.338+0000 7ff0c3589640 1 -- 172.21.15.81:0/3008488023 shutdown_connections 2023-11-08T16:29:46.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.338+0000 7ff0c3589640 1 -- 172.21.15.81:0/3008488023 wait complete. 2023-11-08T16:29:46.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.340+0000 7ff0c3589640 1 Processor -- start 2023-11-08T16:29:46.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.340+0000 7ff0c3589640 1 -- start start 2023-11-08T16:29:46.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.341+0000 7ff0c3589640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a80073e0 0x7ff0bc158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:46.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.341+0000 7ff0c3589640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0bc158f90 0x7ff0bc157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:46.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.341+0000 7ff0c3589640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0bc157660 0x7ff0bc0b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:46.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.341+0000 7ff0c3589640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc1ee530 con 0x7ff0bc158f90 2023-11-08T16:29:46.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.341+0000 7ff0c3589640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc15bbc0 con 0x7ff0a80073e0 2023-11-08T16:29:46.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.341+0000 7ff0c3589640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0bc15bea0 con 0x7ff0bc157660 2023-11-08T16:29:46.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.341+0000 7ff0c0b24640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0bc158f90 0x7ff0bc157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:46.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.341+0000 7ff0c0b24640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0bc158f90 0x7ff0bc157120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55774/0 (socket says 172.21.15.81:55774) 2023-11-08T16:29:46.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.341+0000 7ff0c0b24640 1 -- 172.21.15.81:0/3446139396 learned_addr learned my addr 172.21.15.81:0/3446139396 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:46.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0c0b24640 1 -- 172.21.15.81:0/3446139396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0bc157660 msgr2=0x7ff0bc0b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:46.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0c1b26640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0bc157660 0x7ff0bc0b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:46.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0c1325640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a80073e0 0x7ff0bc158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:46.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0c0b24640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0bc157660 0x7ff0bc0b2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0c0b24640 1 -- 172.21.15.81:0/3446139396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a80073e0 msgr2=0x7ff0bc158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:46.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0c0b24640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a80073e0 0x7ff0bc158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0c0b24640 1 -- 172.21.15.81:0/3446139396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc15c180 con 0x7ff0bc158f90 2023-11-08T16:29:46.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0c0b24640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0bc158f90 0x7ff0bc157120 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00d4c0 tx=0x7ff0ac00d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:46.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0b1ffb640 1 -- 172.21.15.81:0/3446139396 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac005480 con 0x7ff0bc158f90 2023-11-08T16:29:46.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0b1ffb640 1 -- 172.21.15.81:0/3446139396 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac003610 con 0x7ff0bc158f90 2023-11-08T16:29:46.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0b1ffb640 1 -- 172.21.15.81:0/3446139396 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac0039e0 con 0x7ff0bc158f90 2023-11-08T16:29:46.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0c3589640 1 -- 172.21.15.81:0/3446139396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0bc156c00 con 0x7ff0bc158f90 2023-11-08T16:29:46.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.342+0000 7ff0c3589640 1 -- 172.21.15.81:0/3446139396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0bc159410 con 0x7ff0bc158f90 2023-11-08T16:29:46.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.344+0000 7ff0c3589640 1 -- 172.21.15.81:0/3446139396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0bc072170 con 0x7ff0bc158f90 2023-11-08T16:29:46.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.344+0000 7ff0b1ffb640 1 -- 172.21.15.81:0/3446139396 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff0ac003b40 con 0x7ff0bc158f90 2023-11-08T16:29:46.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.345+0000 7ff0b1ffb640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0a006ae70 0x7ff0a006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:46.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.345+0000 7ff0b1ffb640 1 -- 172.21.15.81:0/3446139396 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff0ac089560 con 0x7ff0bc158f90 2023-11-08T16:29:46.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.346+0000 7ff0c1325640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0a006ae70 0x7ff0a006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:46.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.346+0000 7ff0c1325640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0a006ae70 0x7ff0a006d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80098d0 tx=0x7ff0b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:46.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.350+0000 7ff0b1ffb640 1 -- 172.21.15.81:0/3446139396 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0ac054580 con 0x7ff0bc158f90 2023-11-08T16:29:46.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.609+0000 7ff0c3589640 1 -- 172.21.15.81:0/3446139396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]} v 0) v1 -- 0x7ff0bc04ece0 con 0x7ff0bc158f90 2023-11-08T16:29:46.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.862+0000 7ff0b1ffb640 1 -- 172.21.15.81:0/3446139396 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]}]=0 v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff0ac057ba0 con 0x7ff0bc158f90 2023-11-08T16:29:46.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.865+0000 7ff0c3589640 1 -- 172.21.15.81:0/3446139396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0a006ae70 msgr2=0x7ff0a006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:46.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.865+0000 7ff0c3589640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0a006ae70 0x7ff0a006d330 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80098d0 tx=0x7ff0b8008040 comp rx=0 tx=0).stop 2023-11-08T16:29:46.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.865+0000 7ff0c3589640 1 -- 172.21.15.81:0/3446139396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0bc158f90 msgr2=0x7ff0bc157120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:46.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.865+0000 7ff0c3589640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0bc158f90 0x7ff0bc157120 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00d4c0 tx=0x7ff0ac00d980 comp rx=0 tx=0).stop 2023-11-08T16:29:46.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.865+0000 7ff0c3589640 1 -- 172.21.15.81:0/3446139396 shutdown_connections 2023-11-08T16:29:46.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.865+0000 7ff0c3589640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0a006ae70 0x7ff0a006d330 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.865+0000 7ff0c3589640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0bc157660 0x7ff0bc0b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.865+0000 7ff0c3589640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0bc158f90 0x7ff0bc157120 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.865+0000 7ff0c3589640 1 --2- 172.21.15.81:0/3446139396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a80073e0 0x7ff0bc158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:46.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.866+0000 7ff0c3589640 1 -- 172.21.15.81:0/3446139396 >> 172.21.15.81:0/3446139396 conn(0x7ff0bc0cb490 msgr2=0x7ff0bc0af6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:46.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.866+0000 7ff0c3589640 1 -- 172.21.15.81:0/3446139396 shutdown_connections 2023-11-08T16:29:46.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:46.866+0000 7ff0c3589640 1 -- 172.21.15.81:0/3446139396 wait complete. 2023-11-08T16:29:46.888 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T16:29:47.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.100+0000 7fdad359e640 1 Processor -- start 2023-11-08T16:29:47.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.101+0000 7fdad359e640 1 -- start start 2023-11-08T16:29:47.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.102+0000 7fdad359e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdad415d310 0x7fdad40d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.102+0000 7fdad359e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdad40cfd90 0x7fdad40cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.102+0000 7fdad359e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdad40ce0b0 con 0x7fdad415cf30 2023-11-08T16:29:47.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.102+0000 7fdad359e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdad40cdb70 con 0x7fdad40cfd90 2023-11-08T16:29:47.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.102+0000 7fdad359e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40ce390 con 0x7fdad415d310 2023-11-08T16:29:47.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.103+0000 7fdad2d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdad40cfd90 0x7fdad40cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.103+0000 7fdad1d9b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdad415d310 0x7fdad40d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.103+0000 7fdad2d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdad40cfd90 0x7fdad40cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55778/0 (socket says 172.21.15.81:55778) 2023-11-08T16:29:47.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.103+0000 7fdad2d9d640 1 -- 172.21.15.81:0/365150627 learned_addr learned my addr 172.21.15.81:0/365150627 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:47.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.103+0000 7fdad259c640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fdad415cf30 0x7fdad40cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:33742/0 (socket says 172.21.15.81:33742) 2023-11-08T16:29:47.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.103+0000 7fdad2d9d640 1 -- 172.21.15.81:0/365150627 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdad415d310 msgr2=0x7fdad40d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:47.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.103+0000 7fdad2d9d640 1 --2- 172.21.15.81:0/365150627 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdad415d310 0x7fdad40d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.103+0000 7fdad2d9d640 1 -- 172.21.15.81:0/365150627 >> v1:172.21.15.93:6789/0 conn(0x7fdad415cf30 legacy=0x7fdad40cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:47.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.103+0000 7fdad2d9d640 1 -- 172.21.15.81:0/365150627 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad40d1be0 con 0x7fdad40cfd90 2023-11-08T16:29:47.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.104+0000 7fdad2d9d640 1 --2- 172.21.15.81:0/365150627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdad40cfd90 0x7fdad40cd630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009870 tx=0x7fdacc02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244450b9710eea9f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:47.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.105+0000 7fdad159a640 1 -- 172.21.15.81:0/365150627 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc030840 con 0x7fdad40cfd90 2023-11-08T16:29:47.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.105+0000 7fdad159a640 1 -- 172.21.15.81:0/365150627 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc0309a0 con 0x7fdad40cfd90 2023-11-08T16:29:47.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.105+0000 7fdad159a640 1 -- 172.21.15.81:0/365150627 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc030d30 con 0x7fdad40cfd90 2023-11-08T16:29:47.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.105+0000 7fdad359e640 1 -- 172.21.15.81:0/365150627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdad40cfd90 msgr2=0x7fdad40cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:47.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.105+0000 7fdad359e640 1 --2- 172.21.15.81:0/365150627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdad40cfd90 0x7fdad40cd630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009870 tx=0x7fdacc02b980 comp rx=0 tx=0).stop 2023-11-08T16:29:47.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.105+0000 7fdad359e640 1 -- 172.21.15.81:0/365150627 shutdown_connections 2023-11-08T16:29:47.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.105+0000 7fdad359e640 1 --2- 172.21.15.81:0/365150627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdad40cfd90 0x7fdad40cd630 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.105+0000 7fdad359e640 1 --2- 172.21.15.81:0/365150627 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdad415d310 0x7fdad40d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.105+0000 7fdad359e640 1 -- 172.21.15.81:0/365150627 >> 172.21.15.81:0/365150627 conn(0x7fdad406c040 msgr2=0x7fdad406c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:47.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.105+0000 7fdad359e640 1 -- 172.21.15.81:0/365150627 shutdown_connections 2023-11-08T16:29:47.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.106+0000 7fdad359e640 1 -- 172.21.15.81:0/365150627 wait complete. 2023-11-08T16:29:47.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.107+0000 7fdad359e640 1 Processor -- start 2023-11-08T16:29:47.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.107+0000 7fdad359e640 1 -- start start 2023-11-08T16:29:47.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.108+0000 7fdad359e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdad40cfd90 0x7fdad41f1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.108+0000 7fdad359e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdad415cf30 0x7fdad40c9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.108+0000 7fdad359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdad40ca2c0 0x7fdad414af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.108+0000 7fdad359e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad40c9110 con 0x7fdad40ca2c0 2023-11-08T16:29:47.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.108+0000 7fdad359e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad4148ca0 con 0x7fdad40cfd90 2023-11-08T16:29:47.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.108+0000 7fdad359e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdad414be00 con 0x7fdad415cf30 2023-11-08T16:29:47.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdad40ca2c0 0x7fdad414af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdad40ca2c0 0x7fdad414af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55780/0 (socket says 172.21.15.81:55780) 2023-11-08T16:29:47.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad2d9d640 1 -- 172.21.15.81:0/3549452795 learned_addr learned my addr 172.21.15.81:0/3549452795 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:47.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad1d9b640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdad415cf30 0x7fdad40c9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad259c640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdad40cfd90 0x7fdad41f1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad2d9d640 1 -- 172.21.15.81:0/3549452795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdad415cf30 msgr2=0x7fdad40c9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:47.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad2d9d640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdad415cf30 0x7fdad40c9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad2d9d640 1 -- 172.21.15.81:0/3549452795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdad40cfd90 msgr2=0x7fdad41f1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:47.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad2d9d640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdad40cfd90 0x7fdad41f1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad2d9d640 1 -- 172.21.15.81:0/3549452795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc0094d0 con 0x7fdad40ca2c0 2023-11-08T16:29:47.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad259c640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdad40cfd90 0x7fdad41f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:47.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad1d9b640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdad415cf30 0x7fdad40c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:47.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.109+0000 7fdad2d9d640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdad40ca2c0 0x7fdad414af80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009870 tx=0x7fdacc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:47.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.110+0000 7fdacaffd640 1 -- 172.21.15.81:0/3549452795 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc039c90 con 0x7fdad40ca2c0 2023-11-08T16:29:47.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.110+0000 7fdacaffd640 1 -- 172.21.15.81:0/3549452795 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdacc039e10 con 0x7fdad40ca2c0 2023-11-08T16:29:47.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.110+0000 7fdacaffd640 1 -- 172.21.15.81:0/3549452795 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdacc043640 con 0x7fdad40ca2c0 2023-11-08T16:29:47.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.110+0000 7fdad359e640 1 -- 172.21.15.81:0/3549452795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad40c8240 con 0x7fdad40ca2c0 2023-11-08T16:29:47.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.110+0000 7fdad359e640 1 -- 172.21.15.81:0/3549452795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad40c8770 con 0x7fdad40ca2c0 2023-11-08T16:29:47.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.111+0000 7fdad359e640 1 -- 172.21.15.81:0/3549452795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad404ece0 con 0x7fdad40ca2c0 2023-11-08T16:29:47.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.112+0000 7fdacaffd640 1 -- 172.21.15.81:0/3549452795 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdacc0437a0 con 0x7fdad40ca2c0 2023-11-08T16:29:47.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.114+0000 7fdacaffd640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdab806ae70 0x7fdab806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.114+0000 7fdacaffd640 1 -- 172.21.15.81:0/3549452795 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdacc03e070 con 0x7fdad40ca2c0 2023-11-08T16:29:47.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.114+0000 7fdad259c640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdab806ae70 0x7fdab806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.115+0000 7fdad259c640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdab806ae70 0x7fdab806d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdac4007fd0 tx=0x7fdac4007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:47.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.119+0000 7fdacaffd640 1 -- 172.21.15.81:0/3549452795 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdacc077710 con 0x7fdad40ca2c0 2023-11-08T16:29:47.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.309+0000 7fdad359e640 1 -- 172.21.15.81:0/3549452795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdad4068510 con 0x7fdad40ca2c0 2023-11-08T16:29:47.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.314+0000 7fdacaffd640 1 -- 172.21.15.81:0/3549452795 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdacc0340c0 con 0x7fdad40ca2c0 2023-11-08T16:29:47.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 -- 172.21.15.81:0/3549452795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdab806ae70 msgr2=0x7fdab806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:47.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdab806ae70 0x7fdab806d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdac4007fd0 tx=0x7fdac4007d40 comp rx=0 tx=0).stop 2023-11-08T16:29:47.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 -- 172.21.15.81:0/3549452795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdad40ca2c0 msgr2=0x7fdad414af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:47.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdad40ca2c0 0x7fdad414af80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009870 tx=0x7fdacc004730 comp rx=0 tx=0).stop 2023-11-08T16:29:47.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 -- 172.21.15.81:0/3549452795 shutdown_connections 2023-11-08T16:29:47.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdab806ae70 0x7fdab806d330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdad40ca2c0 0x7fdad414af80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdad415cf30 0x7fdad40c9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 --2- 172.21.15.81:0/3549452795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdad40cfd90 0x7fdad41f1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 -- 172.21.15.81:0/3549452795 >> 172.21.15.81:0/3549452795 conn(0x7fdad406c040 msgr2=0x7fdad40b7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:47.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 -- 172.21.15.81:0/3549452795 shutdown_connections 2023-11-08T16:29:47.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.317+0000 7fdad359e640 1 -- 172.21.15.81:0/3549452795 wait complete. 2023-11-08T16:29:47.337 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T16:29:47.338 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:29:47.445 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:29:47.556 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T16:29:47.557 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:29:47.557 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:29:47.617 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:29:47.617 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:29:47.670 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:47.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.869+0000 7f31bde1a640 1 Processor -- start 2023-11-08T16:29:47.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.869+0000 7f31bde1a640 1 -- start start 2023-11-08T16:29:47.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.871+0000 7f31bde1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31b815d490 0x7f31b80d39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.871+0000 7f31bde1a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f31b80d1080 0x7f31b80d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.871+0000 7f31bde1a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b80d1eb0 con 0x7f31b80d1a20 2023-11-08T16:29:47.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.871+0000 7f31bde1a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b80ce890 con 0x7f31b80d1080 2023-11-08T16:29:47.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.871+0000 7f31bde1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b80ceb70 con 0x7f31b815d490 2023-11-08T16:29:47.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.871+0000 7f31affff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f31b80d1080 0x7f31b80d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.871+0000 7f31affff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f31b80d1080 0x7f31b80d14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55790/0 (socket says 172.21.15.81:55790) 2023-11-08T16:29:47.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.871+0000 7f31affff640 1 -- 172.21.15.81:0/499690123 learned_addr learned my addr 172.21.15.81:0/499690123 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:47.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.872+0000 7f31bce18640 1 --2- 172.21.15.81:0/499690123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31b815d490 0x7f31b80d39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.872+0000 7f31affff640 1 -- 172.21.15.81:0/499690123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31b815d490 msgr2=0x7f31b80d39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:47.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.872+0000 7f31affff640 1 --2- 172.21.15.81:0/499690123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31b815d490 0x7f31b80d39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.872+0000 7f31affff640 1 -- 172.21.15.81:0/499690123 >> v1:172.21.15.93:6789/0 conn(0x7f31b80d1a20 legacy=0x7f31b80cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:47.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.872+0000 7f31affff640 1 -- 172.21.15.81:0/499690123 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b80cee50 con 0x7f31b80d1080 2023-11-08T16:29:47.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.873+0000 7f31affff640 1 --2- 172.21.15.81:0/499690123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b80d1080 0x7f31b80d14e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f31a80098a0 tx=0x7f31a802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a383d099bc5cd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:47.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.873+0000 7f31af7fe640 1 -- 172.21.15.81:0/499690123 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8031640 con 0x7f31b80d1080 2023-11-08T16:29:47.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.873+0000 7f31af7fe640 1 -- 172.21.15.81:0/499690123 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a80317a0 con 0x7f31b80d1080 2023-11-08T16:29:47.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.873+0000 7f31af7fe640 1 -- 172.21.15.81:0/499690123 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a8031b30 con 0x7f31b80d1080 2023-11-08T16:29:47.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.874+0000 7f31bde1a640 1 -- 172.21.15.81:0/499690123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b80d1080 msgr2=0x7f31b80d14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:47.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.874+0000 7f31bde1a640 1 --2- 172.21.15.81:0/499690123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b80d1080 0x7f31b80d14e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f31a80098a0 tx=0x7f31a802b8f0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.874+0000 7f31bde1a640 1 -- 172.21.15.81:0/499690123 shutdown_connections 2023-11-08T16:29:47.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.874+0000 7f31bde1a640 1 --2- 172.21.15.81:0/499690123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b80d1080 0x7f31b80d14e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.874+0000 7f31bde1a640 1 --2- 172.21.15.81:0/499690123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31b815d490 0x7f31b80d39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.874+0000 7f31bde1a640 1 -- 172.21.15.81:0/499690123 >> 172.21.15.81:0/499690123 conn(0x7f31b80d2e50 msgr2=0x7f31b80d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:47.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.874+0000 7f31bde1a640 1 -- 172.21.15.81:0/499690123 shutdown_connections 2023-11-08T16:29:47.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.874+0000 7f31bde1a640 1 -- 172.21.15.81:0/499690123 wait complete. 2023-11-08T16:29:47.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.876+0000 7f31bde1a640 1 Processor -- start 2023-11-08T16:29:47.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.876+0000 7f31bde1a640 1 -- start start 2023-11-08T16:29:47.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.877+0000 7f31bde1a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31b80d1080 0x7f31b8158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.877+0000 7f31bde1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31b80d1a20 0x7f31b815f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.877+0000 7f31bde1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b815d490 0x7f31b8144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.877+0000 7f31bde1a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8159140 con 0x7f31b815d490 2023-11-08T16:29:47.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.877+0000 7f31bde1a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8160040 con 0x7f31b80d1080 2023-11-08T16:29:47.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.877+0000 7f31bde1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8160320 con 0x7f31b80d1a20 2023-11-08T16:29:47.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.877+0000 7f31bd619640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b815d490 0x7f31b8144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.877+0000 7f31bd619640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b815d490 0x7f31b8144d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55806/0 (socket says 172.21.15.81:55806) 2023-11-08T16:29:47.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.877+0000 7f31bd619640 1 -- 172.21.15.81:0/702919861 learned_addr learned my addr 172.21.15.81:0/702919861 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:47.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.877+0000 7f31bce18640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31b80d1080 0x7f31b8158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.877+0000 7f31affff640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31b80d1a20 0x7f31b815f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.878+0000 7f31bd619640 1 -- 172.21.15.81:0/702919861 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31b80d1a20 msgr2=0x7f31b815f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:47.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.878+0000 7f31bd619640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31b80d1a20 0x7f31b815f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.878+0000 7f31bd619640 1 -- 172.21.15.81:0/702919861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31b80d1080 msgr2=0x7f31b8158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:47.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.878+0000 7f31bd619640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31b80d1080 0x7f31b8158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:47.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.878+0000 7f31bd619640 1 -- 172.21.15.81:0/702919861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a80094d0 con 0x7f31b815d490 2023-11-08T16:29:47.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.878+0000 7f31bd619640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b815d490 0x7f31b8144d50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f31b000c780 tx=0x7f31b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:47.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.878+0000 7f31ad7fa640 1 -- 172.21.15.81:0/702919861 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b0004b60 con 0x7f31b815d490 2023-11-08T16:29:47.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.878+0000 7f31ad7fa640 1 -- 172.21.15.81:0/702919861 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b000ce90 con 0x7f31b815d490 2023-11-08T16:29:47.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.878+0000 7f31ad7fa640 1 -- 172.21.15.81:0/702919861 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b000a850 con 0x7f31b815d490 2023-11-08T16:29:47.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.878+0000 7f31bde1a640 1 -- 172.21.15.81:0/702919861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b8160600 con 0x7f31b815d490 2023-11-08T16:29:47.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.878+0000 7f31bde1a640 1 -- 172.21.15.81:0/702919861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b815df40 con 0x7f31b815d490 2023-11-08T16:29:47.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.880+0000 7f31bde1a640 1 -- 172.21.15.81:0/702919861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b8160b20 con 0x7f31b815d490 2023-11-08T16:29:47.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.881+0000 7f31ad7fa640 1 -- 172.21.15.81:0/702919861 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f31b000a9b0 con 0x7f31b815d490 2023-11-08T16:29:47.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.881+0000 7f31ad7fa640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f31a406ae70 0x7f31a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:47.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.882+0000 7f31ad7fa640 1 -- 172.21.15.81:0/702919861 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f31b0089b40 con 0x7f31b815d490 2023-11-08T16:29:47.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.882+0000 7f31bce18640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f31a406ae70 0x7f31a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:47.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.882+0000 7f31bce18640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f31a406ae70 0x7f31a406d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006fd0 tx=0x7f31b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:47.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:47.886+0000 7f31ad7fa640 1 -- 172.21.15.81:0/702919861 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f31b8160b20 con 0x7f31b815d490 2023-11-08T16:29:48.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.077+0000 7f31bde1a640 1 -- 172.21.15.81:0/702919861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31b8160b20 con 0x7f31b815d490 2023-11-08T16:29:48.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.080+0000 7f31ad7fa640 1 -- 172.21.15.81:0/702919861 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f31b8160b20 con 0x7f31b815d490 2023-11-08T16:29:48.082 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:48.083 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:48.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.082+0000 7f31bde1a640 1 -- 172.21.15.81:0/702919861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f31a406ae70 msgr2=0x7f31a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.082+0000 7f31bde1a640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f31a406ae70 0x7f31a406d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f31b4006fd0 tx=0x7f31b4008040 comp rx=0 tx=0).stop 2023-11-08T16:29:48.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.082+0000 7f31bde1a640 1 -- 172.21.15.81:0/702919861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b815d490 msgr2=0x7f31b8144d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.082+0000 7f31bde1a640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b815d490 0x7f31b8144d50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f31b000c780 tx=0x7f31b000cc40 comp rx=0 tx=0).stop 2023-11-08T16:29:48.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.083+0000 7f31bde1a640 1 -- 172.21.15.81:0/702919861 shutdown_connections 2023-11-08T16:29:48.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.083+0000 7f31bde1a640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f31a406ae70 0x7f31a406d330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.083+0000 7f31bde1a640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f31b815d490 0x7f31b8144d50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.083+0000 7f31bde1a640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f31b80d1a20 0x7f31b815f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.083+0000 7f31bde1a640 1 --2- 172.21.15.81:0/702919861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f31b80d1080 0x7f31b8158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.083+0000 7f31bde1a640 1 -- 172.21.15.81:0/702919861 >> 172.21.15.81:0/702919861 conn(0x7f31b80d2e50 msgr2=0x7f31b80cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:48.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.083+0000 7f31bde1a640 1 -- 172.21.15.81:0/702919861 shutdown_connections 2023-11-08T16:29:48.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.083+0000 7f31bde1a640 1 -- 172.21.15.81:0/702919861 wait complete. 2023-11-08T16:29:48.104 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:48.105 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:48.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.324+0000 7ff912058640 1 Processor -- start 2023-11-08T16:29:48.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.324+0000 7ff912058640 1 -- start start 2023-11-08T16:29:48.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.326+0000 7ff912058640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff90c0ccc10 0x7ff90c0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.326+0000 7ff912058640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c0ce960 con 0x7ff90c15a0e0 2023-11-08T16:29:48.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.326+0000 7ff912058640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90c0c9480 con 0x7ff90c06b710 2023-11-08T16:29:48.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.326+0000 7ff912058640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90c071980 con 0x7ff90c0ccc10 2023-11-08T16:29:48.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.326+0000 7ff90b7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff90c15a0e0 0x7ff90c0c9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:36006/0 (socket says 172.21.15.81:36006) 2023-11-08T16:29:48.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.326+0000 7ff90b7fe640 1 -- 172.21.15.81:0/2305809169 learned_addr learned my addr 172.21.15.81:0/2305809169 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:48.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.326+0000 7ff903fff640 1 --2- 172.21.15.81:0/2305809169 >> v2:172.21.15.81:3300/0 conn(0x7ff90c0ccc10 0x7ff90c0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.327+0000 7ff903fff640 1 -- 172.21.15.81:0/2305809169 >> v1:172.21.15.93:6789/0 conn(0x7ff90c06b710 legacy=0x7ff90c06bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:48.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.327+0000 7ff903fff640 1 -- 172.21.15.81:0/2305809169 >> v1:172.21.15.81:6789/0 conn(0x7ff90c15a0e0 legacy=0x7ff90c0c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.327+0000 7ff903fff640 1 -- 172.21.15.81:0/2305809169 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c15a4c0 con 0x7ff90c0ccc10 2023-11-08T16:29:48.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.328+0000 7ff903fff640 1 --2- 172.21.15.81:0/2305809169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff90c0ccc10 0x7ff90c0ca300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40097a0 tx=0x7ff8f402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f3c06efdf12587 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:48.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.328+0000 7ff90affd640 1 -- 172.21.15.81:0/2305809169 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4030610 con 0x7ff90c0ccc10 2023-11-08T16:29:48.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.328+0000 7ff90affd640 1 -- 172.21.15.81:0/2305809169 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f4030770 con 0x7ff90c0ccc10 2023-11-08T16:29:48.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.328+0000 7ff90affd640 1 -- 172.21.15.81:0/2305809169 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4030b00 con 0x7ff90c0ccc10 2023-11-08T16:29:48.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.329+0000 7ff912058640 1 -- 172.21.15.81:0/2305809169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff90c0ccc10 msgr2=0x7ff90c0ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.329+0000 7ff912058640 1 --2- 172.21.15.81:0/2305809169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff90c0ccc10 0x7ff90c0ca300 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40097a0 tx=0x7ff8f402b960 comp rx=0 tx=0).stop 2023-11-08T16:29:48.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.329+0000 7ff912058640 1 -- 172.21.15.81:0/2305809169 shutdown_connections 2023-11-08T16:29:48.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.329+0000 7ff912058640 1 --2- 172.21.15.81:0/2305809169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff90c0ccc10 0x7ff90c0ca300 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.329+0000 7ff912058640 1 -- 172.21.15.81:0/2305809169 >> 172.21.15.81:0/2305809169 conn(0x7ff90c0cb580 msgr2=0x7ff90c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:48.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.329+0000 7ff912058640 1 -- 172.21.15.81:0/2305809169 shutdown_connections 2023-11-08T16:29:48.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.329+0000 7ff912058640 1 -- 172.21.15.81:0/2305809169 wait complete. 2023-11-08T16:29:48.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.331+0000 7ff912058640 1 Processor -- start 2023-11-08T16:29:48.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.331+0000 7ff912058640 1 -- start start 2023-11-08T16:29:48.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.332+0000 7ff912058640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff90c06b710 0x7ff90c150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.332+0000 7ff912058640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff90c0ccc10 0x7ff90c1e2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.332+0000 7ff912058640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff90c15a0e0 0x7ff90c1e2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.332+0000 7ff912058640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c151120 con 0x7ff90c0ccc10 2023-11-08T16:29:48.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.332+0000 7ff912058640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c0c3b30 con 0x7ff90c06b710 2023-11-08T16:29:48.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.332+0000 7ff912058640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c0c3e10 con 0x7ff90c15a0e0 2023-11-08T16:29:48.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.332+0000 7ff903fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff90c0ccc10 0x7ff90c1e2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.332+0000 7ff903fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff90c0ccc10 0x7ff90c1e2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55824/0 (socket says 172.21.15.81:55824) 2023-11-08T16:29:48.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.332+0000 7ff903fff640 1 -- 172.21.15.81:0/3759918051 learned_addr learned my addr 172.21.15.81:0/3759918051 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:48.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.333+0000 7ff90b7fe640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff90c06b710 0x7ff90c150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.333+0000 7ff903fff640 1 -- 172.21.15.81:0/3759918051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff90c15a0e0 msgr2=0x7ff90c1e2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.333+0000 7ff903fff640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff90c15a0e0 0x7ff90c1e2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.333+0000 7ff903fff640 1 -- 172.21.15.81:0/3759918051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff90c06b710 msgr2=0x7ff90c150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.333+0000 7ff903fff640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff90c06b710 0x7ff90c150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.333+0000 7ff903fff640 1 -- 172.21.15.81:0/3759918051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f4009400 con 0x7ff90c0ccc10 2023-11-08T16:29:48.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.333+0000 7ff903fff640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff90c0ccc10 0x7ff90c1e2850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40098c0 tx=0x7ff8f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:48.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.333+0000 7ff908ff9640 1 -- 172.21.15.81:0/3759918051 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f403e040 con 0x7ff90c0ccc10 2023-11-08T16:29:48.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.333+0000 7ff908ff9640 1 -- 172.21.15.81:0/3759918051 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f4039cb0 con 0x7ff90c0ccc10 2023-11-08T16:29:48.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.333+0000 7ff912058640 1 -- 172.21.15.81:0/3759918051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90c0c40f0 con 0x7ff90c0ccc10 2023-11-08T16:29:48.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.333+0000 7ff908ff9640 1 -- 172.21.15.81:0/3759918051 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f4044630 con 0x7ff90c0ccc10 2023-11-08T16:29:48.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.334+0000 7ff912058640 1 -- 172.21.15.81:0/3759918051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff90c0c46b0 con 0x7ff90c0ccc10 2023-11-08T16:29:48.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.335+0000 7ff912058640 1 -- 172.21.15.81:0/3759918051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff90c06d0a0 con 0x7ff90c0ccc10 2023-11-08T16:29:48.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.336+0000 7ff908ff9640 1 -- 172.21.15.81:0/3759918051 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff8f404d3b0 con 0x7ff90c0ccc10 2023-11-08T16:29:48.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.337+0000 7ff908ff9640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8ec06aea0 0x7ff8ec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.337+0000 7ff908ff9640 1 -- 172.21.15.81:0/3759918051 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff8f40abc90 con 0x7ff90c0ccc10 2023-11-08T16:29:48.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.337+0000 7ff90b7fe640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8ec06aea0 0x7ff8ec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.338+0000 7ff90b7fe640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8ec06aea0 0x7ff8ec06d360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc005880 tx=0x7ff8fc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:48.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.341+0000 7ff908ff9640 1 -- 172.21.15.81:0/3759918051 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8f4076ca0 con 0x7ff90c0ccc10 2023-11-08T16:29:48.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.530+0000 7ff912058640 1 -- 172.21.15.81:0/3759918051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff90c0c48a0 con 0x7ff90c0ccc10 2023-11-08T16:29:48.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.531+0000 7ff908ff9640 1 -- 172.21.15.81:0/3759918051 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff8f407a2c0 con 0x7ff90c0ccc10 2023-11-08T16:29:48.534 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:48.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.533+0000 7ff912058640 1 -- 172.21.15.81:0/3759918051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8ec06aea0 msgr2=0x7ff8ec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.533+0000 7ff912058640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8ec06aea0 0x7ff8ec06d360 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc005880 tx=0x7ff8fc009040 comp rx=0 tx=0).stop 2023-11-08T16:29:48.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.533+0000 7ff912058640 1 -- 172.21.15.81:0/3759918051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff90c0ccc10 msgr2=0x7ff90c1e2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.533+0000 7ff912058640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff90c0ccc10 0x7ff90c1e2850 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40098c0 tx=0x7ff8f4004730 comp rx=0 tx=0).stop 2023-11-08T16:29:48.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.534+0000 7ff912058640 1 -- 172.21.15.81:0/3759918051 shutdown_connections 2023-11-08T16:29:48.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.534+0000 7ff912058640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8ec06aea0 0x7ff8ec06d360 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.534+0000 7ff912058640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff90c15a0e0 0x7ff90c1e2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.534+0000 7ff912058640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff90c0ccc10 0x7ff90c1e2850 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.534+0000 7ff912058640 1 --2- 172.21.15.81:0/3759918051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff90c06b710 0x7ff90c150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.534+0000 7ff912058640 1 -- 172.21.15.81:0/3759918051 >> 172.21.15.81:0/3759918051 conn(0x7ff90c0cb580 msgr2=0x7ff90c0ca4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:48.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.534+0000 7ff912058640 1 -- 172.21.15.81:0/3759918051 shutdown_connections 2023-11-08T16:29:48.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.534+0000 7ff912058640 1 -- 172.21.15.81:0/3759918051 wait complete. 2023-11-08T16:29:48.553 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:48.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.765+0000 7f2e29bfb640 1 Processor -- start 2023-11-08T16:29:48.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.766+0000 7f2e29bfb640 1 -- start start 2023-11-08T16:29:48.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.767+0000 7f2e29bfb640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e240ce390 0x7f2e2415e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.767+0000 7f2e29bfb640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2e240c9200 0x7f2e2406b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.767+0000 7f2e29bfb640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2415e8d0 con 0x7f2e2406bc50 2023-11-08T16:29:48.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.767+0000 7f2e29bfb640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e2406c100 con 0x7f2e240c9200 2023-11-08T16:29:48.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.767+0000 7f2e29bfb640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240ca2f0 con 0x7f2e240ce390 2023-11-08T16:29:48.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.767+0000 7f2e23fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2e2406bc50 0x7f2e24160e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:36018/0 (socket says 172.21.15.81:36018) 2023-11-08T16:29:48.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e23fff640 1 -- 172.21.15.81:0/444906501 learned_addr learned my addr 172.21.15.81:0/444906501 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:48.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e237fe640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e240ce390 0x7f2e2415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e1affd640 1 --2- 172.21.15.81:0/444906501 >> v2:172.21.15.93:3300/0 conn(0x7f2e240c9200 0x7f2e2406b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3279590523 0 0) 0x7f2e1c001490 con 0x7f2e2406bc50 2023-11-08T16:29:48.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e10003520 con 0x7f2e2406bc50 2023-11-08T16:29:48.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 998614869 0 0) 0x7f2e10003520 con 0x7f2e2406bc50 2023-11-08T16:29:48.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e240ce390 msgr2=0x7f2e2415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:48.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e22ffd640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e240ce390 0x7f2e2415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 >> v2:172.21.15.93:3300/0 conn(0x7f2e240c9200 msgr2=0x7f2e2406b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e22ffd640 1 --2- 172.21.15.81:0/444906501 >> v2:172.21.15.93:3300/0 conn(0x7f2e240c9200 0x7f2e2406b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e240ad8c0 con 0x7f2e2406bc50 2023-11-08T16:29:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.768+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f2e1c001b80 con 0x7f2e2406bc50 2023-11-08T16:29:48.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 >> v1:172.21.15.81:6789/0 conn(0x7f2e2406bc50 legacy=0x7f2e24160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e22ffd640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e240ce390 0x7f2e100039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e22ffd640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e10003fb0 0x7f2e100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e22ffd640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e10004d00 0x7f2e10005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2406c100 con 0x7f2e10004d00 2023-11-08T16:29:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240ca2f0 con 0x7f2e10003fb0 2023-11-08T16:29:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1000ba90 con 0x7f2e240ce390 2023-11-08T16:29:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e23fff640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e10004d00 0x7f2e10005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e237fe640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e240ce390 0x7f2e100039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e1affd640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e10003fb0 0x7f2e100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e23fff640 1 -- 172.21.15.81:0/444906501 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e240ce390 msgr2=0x7f2e100039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e23fff640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e240ce390 0x7f2e100039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e23fff640 1 -- 172.21.15.81:0/444906501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e10003fb0 msgr2=0x7f2e100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e23fff640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e10003fb0 0x7f2e100047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.769+0000 7f2e23fff640 1 -- 172.21.15.81:0/444906501 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e10003520 con 0x7f2e10004d00 2023-11-08T16:29:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.770+0000 7f2e23fff640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e10004d00 0x7f2e10005180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00a780 tx=0x7f2e100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62be4e3ced0d4084 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:48.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.771+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c034630 con 0x7f2e10004d00 2023-11-08T16:29:48.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.771+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1c02f070 con 0x7f2e10004d00 2023-11-08T16:29:48.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.771+0000 7f2e22ffd640 1 -- 172.21.15.81:0/444906501 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1c034a30 con 0x7f2e10004d00 2023-11-08T16:29:48.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.771+0000 7f2e29bfb640 1 -- 172.21.15.81:0/444906501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e10004d00 msgr2=0x7f2e10005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.771+0000 7f2e29bfb640 1 --2- 172.21.15.81:0/444906501 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e10004d00 0x7f2e10005180 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2e1c00a780 tx=0x7f2e100057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.771+0000 7f2e23fff640 1 -- 172.21.15.81:0/444906501 reap_dead start 2023-11-08T16:29:48.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.771+0000 7f2e29bfb640 1 -- 172.21.15.81:0/444906501 shutdown_connections 2023-11-08T16:29:48.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.771+0000 7f2e29bfb640 1 -- 172.21.15.81:0/444906501 >> 172.21.15.81:0/444906501 conn(0x7f2e240cb580 msgr2=0x7f2e240cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:48.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.771+0000 7f2e29bfb640 1 -- 172.21.15.81:0/444906501 shutdown_connections 2023-11-08T16:29:48.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.772+0000 7f2e29bfb640 1 -- 172.21.15.81:0/444906501 wait complete. 2023-11-08T16:29:48.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.773+0000 7f2e29bfb640 1 Processor -- start 2023-11-08T16:29:48.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.773+0000 7f2e29bfb640 1 -- start start 2023-11-08T16:29:48.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.774+0000 7f2e29bfb640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e10004d00 0x7f2e2415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.774+0000 7f2e29bfb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240b1580 0x7f2e240b39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.774+0000 7f2e29bfb640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e240b3f30 0x7f2e241f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.774+0000 7f2e29bfb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240c4f40 con 0x7f2e240b1580 2023-11-08T16:29:48.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.774+0000 7f2e29bfb640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240c4070 con 0x7f2e240b3f30 2023-11-08T16:29:48.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.774+0000 7f2e29bfb640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e240c42c0 con 0x7f2e10004d00 2023-11-08T16:29:48.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240b1580 0x7f2e240b39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e1affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240b1580 0x7f2e240b39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55840/0 (socket says 172.21.15.81:55840) 2023-11-08T16:29:48.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e1affd640 1 -- 172.21.15.81:0/599501321 learned_addr learned my addr 172.21.15.81:0/599501321 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:48.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e237fe640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e10004d00 0x7f2e2415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e23fff640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e240b3f30 0x7f2e241f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e1affd640 1 -- 172.21.15.81:0/599501321 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e10004d00 msgr2=0x7f2e2415cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e1affd640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e10004d00 0x7f2e2415cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e1affd640 1 -- 172.21.15.81:0/599501321 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e240b3f30 msgr2=0x7f2e241f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e1affd640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e240b3f30 0x7f2e241f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e1affd640 1 -- 172.21.15.81:0/599501321 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e240c45a0 con 0x7f2e240b1580 2023-11-08T16:29:48.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e23fff640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e240b3f30 0x7f2e241f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:48.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.775+0000 7f2e1affd640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240b1580 0x7f2e240b39f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0108e0 tx=0x7f2e0c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:48.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.776+0000 7f2e20ff9640 1 -- 172.21.15.81:0/599501321 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c00eb60 con 0x7f2e240b1580 2023-11-08T16:29:48.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.776+0000 7f2e20ff9640 1 -- 172.21.15.81:0/599501321 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e0c00ecc0 con 0x7f2e240b1580 2023-11-08T16:29:48.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.776+0000 7f2e20ff9640 1 -- 172.21.15.81:0/599501321 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e0c01c400 con 0x7f2e240b1580 2023-11-08T16:29:48.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.776+0000 7f2e29bfb640 1 -- 172.21.15.81:0/599501321 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e240c4880 con 0x7f2e240b1580 2023-11-08T16:29:48.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.776+0000 7f2e29bfb640 1 -- 172.21.15.81:0/599501321 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e240b43e0 con 0x7f2e240b1580 2023-11-08T16:29:48.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.777+0000 7f2e29bfb640 1 -- 172.21.15.81:0/599501321 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e2404ece0 con 0x7f2e240b1580 2023-11-08T16:29:48.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.778+0000 7f2e20ff9640 1 -- 172.21.15.81:0/599501321 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e0c01c560 con 0x7f2e240b1580 2023-11-08T16:29:48.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.779+0000 7f2e20ff9640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0406ae70 0x7f2e0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:48.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.779+0000 7f2e20ff9640 1 -- 172.21.15.81:0/599501321 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2e0c089e70 con 0x7f2e240b1580 2023-11-08T16:29:48.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.779+0000 7f2e237fe640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0406ae70 0x7f2e0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:48.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.780+0000 7f2e237fe640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0406ae70 0x7f2e0406d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2e240c6660 tx=0x7f2e14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:48.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.783+0000 7f2e20ff9640 1 -- 172.21.15.81:0/599501321 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e0c054ea0 con 0x7f2e240b1580 2023-11-08T16:29:48.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.974+0000 7f2e29bfb640 1 -- 172.21.15.81:0/599501321 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e241e7820 con 0x7f2e240b1580 2023-11-08T16:29:48.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.975+0000 7f2e20ff9640 1 -- 172.21.15.81:0/599501321 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2e0c0584c0 con 0x7f2e240b1580 2023-11-08T16:29:48.978 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:48.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.977+0000 7f2e29bfb640 1 -- 172.21.15.81:0/599501321 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0406ae70 msgr2=0x7f2e0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.977+0000 7f2e29bfb640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0406ae70 0x7f2e0406d330 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2e240c6660 tx=0x7f2e14008040 comp rx=0 tx=0).stop 2023-11-08T16:29:48.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.977+0000 7f2e29bfb640 1 -- 172.21.15.81:0/599501321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240b1580 msgr2=0x7f2e240b39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:48.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.977+0000 7f2e29bfb640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240b1580 0x7f2e240b39f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2e0c0108e0 tx=0x7f2e0c010da0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.977+0000 7f2e29bfb640 1 -- 172.21.15.81:0/599501321 shutdown_connections 2023-11-08T16:29:48.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.977+0000 7f2e29bfb640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0406ae70 0x7f2e0406d330 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.977+0000 7f2e29bfb640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e240b3f30 0x7f2e241f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.977+0000 7f2e29bfb640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e240b1580 0x7f2e240b39f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.977+0000 7f2e29bfb640 1 --2- 172.21.15.81:0/599501321 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e10004d00 0x7f2e2415cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:48.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.977+0000 7f2e29bfb640 1 -- 172.21.15.81:0/599501321 >> 172.21.15.81:0/599501321 conn(0x7f2e240cb580 msgr2=0x7f2e240ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:48.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.978+0000 7f2e29bfb640 1 -- 172.21.15.81:0/599501321 shutdown_connections 2023-11-08T16:29:48.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:48.978+0000 7f2e29bfb640 1 -- 172.21.15.81:0/599501321 wait complete. 2023-11-08T16:29:48.999 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:49.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.219+0000 7fa126a72640 1 Processor -- start 2023-11-08T16:29:49.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.219+0000 7fa126a72640 1 -- start start 2023-11-08T16:29:49.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.220+0000 7fa126a72640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa12015e1a0 0x7fa1200caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.220+0000 7fa126a72640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1200c97b0 0x7fa12006fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.220+0000 7fa126a72640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12015e580 con 0x7fa120070360 2023-11-08T16:29:49.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.221+0000 7fa126a72640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1200ca830 con 0x7fa12015e1a0 2023-11-08T16:29:49.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.221+0000 7fa126a72640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200707d0 con 0x7fa1200c97b0 2023-11-08T16:29:49.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.221+0000 7fa126271640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa120070360 0x7fa12015e7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:36024/0 (socket says 172.21.15.81:36024) 2023-11-08T16:29:49.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.221+0000 7fa126271640 1 -- 172.21.15.81:0/1359243804 learned_addr learned my addr 172.21.15.81:0/1359243804 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:49.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.221+0000 7fa12526f640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1200c97b0 0x7fa12006fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.221+0000 7fa125a70640 1 --2- 172.21.15.81:0/1359243804 >> v2:172.21.15.93:3300/0 conn(0x7fa12015e1a0 0x7fa1200caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.221+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782145329 0 0) 0x7fa12015e580 con 0x7fa120070360 2023-11-08T16:29:49.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.221+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa10c003520 con 0x7fa120070360 2023-11-08T16:29:49.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2638796886 0 0) 0x7fa10c003520 con 0x7fa120070360 2023-11-08T16:29:49.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1200c97b0 msgr2=0x7fa12006fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:49.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1200c97b0 0x7fa12006fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 >> v2:172.21.15.93:3300/0 conn(0x7fa12015e1a0 msgr2=0x7fa1200caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:49.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 --2- 172.21.15.81:0/1359243804 >> v2:172.21.15.93:3300/0 conn(0x7fa12015e1a0 0x7fa1200caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1200c9cb0 con 0x7fa120070360 2023-11-08T16:29:49.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fa1180019f0 con 0x7fa120070360 2023-11-08T16:29:49.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 >> v1:172.21.15.81:6789/0 conn(0x7fa120070360 legacy=0x7fa12015e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:49.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1200c97b0 0x7fa10c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa10c003f20 0x7fa10c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10c004d00 0x7fa10c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200ca830 con 0x7fa10c004d00 2023-11-08T16:29:49.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200707d0 con 0x7fa10c003f20 2023-11-08T16:29:49.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10c00ba00 con 0x7fa1200c97b0 2023-11-08T16:29:49.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.222+0000 7fa126271640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10c004d00 0x7fa10c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.223+0000 7fa12526f640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa10c003f20 0x7fa10c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.223+0000 7fa125a70640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1200c97b0 0x7fa10c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.223+0000 7fa126271640 1 -- 172.21.15.81:0/1359243804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1200c97b0 msgr2=0x7fa10c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:49.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.223+0000 7fa126271640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1200c97b0 0x7fa10c0039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.223+0000 7fa126271640 1 -- 172.21.15.81:0/1359243804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa10c003f20 msgr2=0x7fa10c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:49.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.223+0000 7fa126271640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa10c003f20 0x7fa10c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.223+0000 7fa126271640 1 -- 172.21.15.81:0/1359243804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10c003520 con 0x7fa10c004d00 2023-11-08T16:29:49.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.224+0000 7fa126271640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10c004d00 0x7fa10c005180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa11800a820 tx=0x7fa10c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed4f08260a2bbb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:49.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.224+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180072d0 con 0x7fa10c004d00 2023-11-08T16:29:49.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.224+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11802f070 con 0x7fa10c004d00 2023-11-08T16:29:49.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.224+0000 7fa124a6e640 1 -- 172.21.15.81:0/1359243804 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1180076b0 con 0x7fa10c004d00 2023-11-08T16:29:49.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.224+0000 7fa126a72640 1 -- 172.21.15.81:0/1359243804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10c004d00 msgr2=0x7fa10c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:49.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.224+0000 7fa126a72640 1 --2- 172.21.15.81:0/1359243804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa10c004d00 0x7fa10c005180 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa11800a820 tx=0x7fa10c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.224+0000 7fa126271640 1 -- 172.21.15.81:0/1359243804 reap_dead start 2023-11-08T16:29:49.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.225+0000 7fa126a72640 1 -- 172.21.15.81:0/1359243804 shutdown_connections 2023-11-08T16:29:49.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.225+0000 7fa126a72640 1 -- 172.21.15.81:0/1359243804 >> 172.21.15.81:0/1359243804 conn(0x7fa1200ce8d0 msgr2=0x7fa1200cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:49.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.225+0000 7fa126a72640 1 -- 172.21.15.81:0/1359243804 shutdown_connections 2023-11-08T16:29:49.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.225+0000 7fa126a72640 1 -- 172.21.15.81:0/1359243804 wait complete. 2023-11-08T16:29:49.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.226+0000 7fa126a72640 1 Processor -- start 2023-11-08T16:29:49.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.227+0000 7fa126a72640 1 -- start start 2023-11-08T16:29:49.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.227+0000 7fa126a72640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa120149270 0x7fa120153960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.227+0000 7fa126a72640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa120149650 0x7fa1200cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa126a72640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa120156830 0x7fa1200cce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa126a72640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa120154ce0 con 0x7fa120149650 2023-11-08T16:29:49.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa126a72640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1201554e0 con 0x7fa120149270 2023-11-08T16:29:49.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa126a72640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa120154060 con 0x7fa120156830 2023-11-08T16:29:49.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa12526f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa120149650 0x7fa1200cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa12526f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa120149650 0x7fa1200cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55846/0 (socket says 172.21.15.81:55846) 2023-11-08T16:29:49.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa12526f640 1 -- 172.21.15.81:0/2568834172 learned_addr learned my addr 172.21.15.81:0/2568834172 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:49.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa12526f640 1 -- 172.21.15.81:0/2568834172 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa120156830 msgr2=0x7fa1200cce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:49.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa12526f640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa120156830 0x7fa1200cce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa12526f640 1 -- 172.21.15.81:0/2568834172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa120149270 msgr2=0x7fa120153960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:49.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa12526f640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa120149270 0x7fa120153960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa12526f640 1 -- 172.21.15.81:0/2568834172 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa120154340 con 0x7fa120149650 2023-11-08T16:29:49.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.228+0000 7fa12526f640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa120149650 0x7fa1200cc8d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa110011570 tx=0x7fa110011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:49.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.229+0000 7fa1167fc640 1 -- 172.21.15.81:0/2568834172 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110015870 con 0x7fa120149650 2023-11-08T16:29:49.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.229+0000 7fa1167fc640 1 -- 172.21.15.81:0/2568834172 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1100159d0 con 0x7fa120149650 2023-11-08T16:29:49.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.229+0000 7fa126a72640 1 -- 172.21.15.81:0/2568834172 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa120154620 con 0x7fa120149650 2023-11-08T16:29:49.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.229+0000 7fa1167fc640 1 -- 172.21.15.81:0/2568834172 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa110015d40 con 0x7fa120149650 2023-11-08T16:29:49.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.229+0000 7fa126a72640 1 -- 172.21.15.81:0/2568834172 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa120151bb0 con 0x7fa120149650 2023-11-08T16:29:49.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.230+0000 7fa126a72640 1 -- 172.21.15.81:0/2568834172 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12004ece0 con 0x7fa120149650 2023-11-08T16:29:49.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.231+0000 7fa1167fc640 1 -- 172.21.15.81:0/2568834172 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa11001d400 con 0x7fa120149650 2023-11-08T16:29:49.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.232+0000 7fa1167fc640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa10406aea0 0x7fa10406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.232+0000 7fa1167fc640 1 -- 172.21.15.81:0/2568834172 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa110089d60 con 0x7fa120149650 2023-11-08T16:29:49.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.232+0000 7fa125a70640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa10406aea0 0x7fa10406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.233+0000 7fa125a70640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa10406aea0 0x7fa10406d360 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0099d0 tx=0x7fa11c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:49.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.236+0000 7fa1167fc640 1 -- 172.21.15.81:0/2568834172 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa110054e10 con 0x7fa120149650 2023-11-08T16:29:49.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.429+0000 7fa126a72640 1 -- 172.21.15.81:0/2568834172 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa1200cf480 con 0x7fa120149650 2023-11-08T16:29:49.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.430+0000 7fa1167fc640 1 -- 172.21.15.81:0/2568834172 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa110058430 con 0x7fa120149650 2023-11-08T16:29:49.433 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.432+0000 7fa126a72640 1 -- 172.21.15.81:0/2568834172 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa10406aea0 msgr2=0x7fa10406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.432+0000 7fa126a72640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa10406aea0 0x7fa10406d360 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0099d0 tx=0x7fa11c008040 comp rx=0 tx=0).stop 2023-11-08T16:29:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.432+0000 7fa126a72640 1 -- 172.21.15.81:0/2568834172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa120149650 msgr2=0x7fa1200cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.432+0000 7fa126a72640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa120149650 0x7fa1200cc8d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa110011570 tx=0x7fa110011a30 comp rx=0 tx=0).stop 2023-11-08T16:29:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.432+0000 7fa126a72640 1 -- 172.21.15.81:0/2568834172 shutdown_connections 2023-11-08T16:29:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.432+0000 7fa126a72640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa10406aea0 0x7fa10406d360 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.432+0000 7fa126a72640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa120156830 0x7fa1200cce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.432+0000 7fa126a72640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa120149650 0x7fa1200cc8d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.432+0000 7fa126a72640 1 --2- 172.21.15.81:0/2568834172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa120149270 0x7fa120153960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.432+0000 7fa126a72640 1 -- 172.21.15.81:0/2568834172 >> 172.21.15.81:0/2568834172 conn(0x7fa1200ce8d0 msgr2=0x7fa12006c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:49.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.433+0000 7fa126a72640 1 -- 172.21.15.81:0/2568834172 shutdown_connections 2023-11-08T16:29:49.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.433+0000 7fa126a72640 1 -- 172.21.15.81:0/2568834172 wait complete. 2023-11-08T16:29:49.453 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:29:49.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.667+0000 7fec1aeec640 1 Processor -- start 2023-11-08T16:29:49.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.668+0000 7fec1aeec640 1 -- start start 2023-11-08T16:29:49.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.669+0000 7fec1aeec640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fec140cc8e0 0x7fec14164500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.669+0000 7fec1aeec640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fec140cd2a0 0x7fec14164a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.669+0000 7fec1aeec640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec1406fa00 con 0x7fec140c90f0 2023-11-08T16:29:49.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.669+0000 7fec1aeec640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec1406b620 con 0x7fec140cd2a0 2023-11-08T16:29:49.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.669+0000 7fec1aeec640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec140caec0 con 0x7fec140cc8e0 2023-11-08T16:29:49.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.669+0000 7fec13fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fec140cd2a0 0x7fec14164a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.670+0000 7fec13fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fec140cd2a0 0x7fec14164a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55850/0 (socket says 172.21.15.81:55850) 2023-11-08T16:29:49.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.670+0000 7fec13fff640 1 -- 172.21.15.81:0/2057504949 learned_addr learned my addr 172.21.15.81:0/2057504949 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:49.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.670+0000 7fec18c88640 1 --2- 172.21.15.81:0/2057504949 >> v2:172.21.15.93:3300/0 conn(0x7fec140cc8e0 0x7fec14164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.670+0000 7fec13fff640 1 -- 172.21.15.81:0/2057504949 >> v2:172.21.15.93:3300/0 conn(0x7fec140cc8e0 msgr2=0x7fec14164500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:49.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.670+0000 7fec13fff640 1 --2- 172.21.15.81:0/2057504949 >> v2:172.21.15.93:3300/0 conn(0x7fec140cc8e0 0x7fec14164500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.670+0000 7fec13fff640 1 -- 172.21.15.81:0/2057504949 >> v1:172.21.15.93:6789/0 conn(0x7fec140c90f0 legacy=0x7fec141650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:49.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.670+0000 7fec13fff640 1 -- 172.21.15.81:0/2057504949 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec140cb1a0 con 0x7fec140cd2a0 2023-11-08T16:29:49.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.671+0000 7fec13fff640 1 --2- 172.21.15.81:0/2057504949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec140cd2a0 0x7fec14164a40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fec140cb480 tx=0x7febfc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d39f835bcc5d1a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:49.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.671+0000 7fec137fe640 1 -- 172.21.15.81:0/2057504949 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc0317e0 con 0x7fec140cd2a0 2023-11-08T16:29:49.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.671+0000 7fec137fe640 1 -- 172.21.15.81:0/2057504949 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc031940 con 0x7fec140cd2a0 2023-11-08T16:29:49.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.671+0000 7fec137fe640 1 -- 172.21.15.81:0/2057504949 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc031cd0 con 0x7fec140cd2a0 2023-11-08T16:29:49.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.672+0000 7fec1aeec640 1 -- 172.21.15.81:0/2057504949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec140cd2a0 msgr2=0x7fec14164a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:49.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.672+0000 7fec1aeec640 1 --2- 172.21.15.81:0/2057504949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec140cd2a0 0x7fec14164a40 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fec140cb480 tx=0x7febfc02b880 comp rx=0 tx=0).stop 2023-11-08T16:29:49.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.672+0000 7fec1aeec640 1 -- 172.21.15.81:0/2057504949 shutdown_connections 2023-11-08T16:29:49.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.672+0000 7fec1aeec640 1 --2- 172.21.15.81:0/2057504949 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec140cd2a0 0x7fec14164a40 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.672+0000 7fec1aeec640 1 --2- 172.21.15.81:0/2057504949 >> v2:172.21.15.93:3300/0 conn(0x7fec140cc8e0 0x7fec14164500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.672+0000 7fec1aeec640 1 -- 172.21.15.81:0/2057504949 >> 172.21.15.81:0/2057504949 conn(0x7fec140ceaa0 msgr2=0x7fec140cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:49.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.672+0000 7fec1aeec640 1 -- 172.21.15.81:0/2057504949 shutdown_connections 2023-11-08T16:29:49.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.672+0000 7fec1aeec640 1 -- 172.21.15.81:0/2057504949 wait complete. 2023-11-08T16:29:49.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.674+0000 7fec1aeec640 1 Processor -- start 2023-11-08T16:29:49.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.674+0000 7fec1aeec640 1 -- start start 2023-11-08T16:29:49.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.675+0000 7fec1aeec640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec140c90f0 0x7fec14150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.675+0000 7fec1aeec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec140cc8e0 0x7fec141f65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.675+0000 7fec1aeec640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fec140cd2a0 0x7fec140c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.675+0000 7fec1aeec640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1414e460 con 0x7fec140cc8e0 2023-11-08T16:29:49.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.675+0000 7fec1aeec640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1414d7e0 con 0x7fec140c90f0 2023-11-08T16:29:49.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.675+0000 7fec1aeec640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec1414dac0 con 0x7fec140cd2a0 2023-11-08T16:29:49.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.675+0000 7fec13fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec140cc8e0 0x7fec141f65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.675+0000 7fec13fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec140cc8e0 0x7fec141f65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55862/0 (socket says 172.21.15.81:55862) 2023-11-08T16:29:49.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.676+0000 7fec13fff640 1 -- 172.21.15.81:0/3474162868 learned_addr learned my addr 172.21.15.81:0/3474162868 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:49.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.676+0000 7fec18c88640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec140c90f0 0x7fec14150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.676+0000 7fec19489640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fec140cd2a0 0x7fec140c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.676+0000 7fec13fff640 1 -- 172.21.15.81:0/3474162868 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fec140cd2a0 msgr2=0x7fec140c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:49.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.676+0000 7fec13fff640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fec140cd2a0 0x7fec140c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.676+0000 7fec13fff640 1 -- 172.21.15.81:0/3474162868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec140c90f0 msgr2=0x7fec14150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:49.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.676+0000 7fec13fff640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec140c90f0 0x7fec14150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.676+0000 7fec13fff640 1 -- 172.21.15.81:0/3474162868 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febfc0094d0 con 0x7fec140cc8e0 2023-11-08T16:29:49.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.676+0000 7fec13fff640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec140cc8e0 0x7fec141f65e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febfc02b8c0 tx=0x7febfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:49.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.676+0000 7fec117fa640 1 -- 172.21.15.81:0/3474162868 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc032880 con 0x7fec140cc8e0 2023-11-08T16:29:49.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.677+0000 7fec117fa640 1 -- 172.21.15.81:0/3474162868 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febfc0329e0 con 0x7fec140cc8e0 2023-11-08T16:29:49.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.677+0000 7fec1aeec640 1 -- 172.21.15.81:0/3474162868 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1414dda0 con 0x7fec140cc8e0 2023-11-08T16:29:49.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.677+0000 7fec117fa640 1 -- 172.21.15.81:0/3474162868 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febfc032d90 con 0x7fec140cc8e0 2023-11-08T16:29:49.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.677+0000 7fec1aeec640 1 -- 172.21.15.81:0/3474162868 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec141f6b20 con 0x7fec140cc8e0 2023-11-08T16:29:49.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.678+0000 7fec1aeec640 1 -- 172.21.15.81:0/3474162868 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1414e5a0 con 0x7fec140cc8e0 2023-11-08T16:29:49.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.679+0000 7fec117fa640 1 -- 172.21.15.81:0/3474162868 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7febfc04e450 con 0x7fec140cc8e0 2023-11-08T16:29:49.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.680+0000 7fec117fa640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febf406ae70 0x7febf406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:49.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.680+0000 7fec117fa640 1 -- 172.21.15.81:0/3474162868 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7febfc0ab710 con 0x7fec140cc8e0 2023-11-08T16:29:49.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.680+0000 7fec18c88640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febf406ae70 0x7febf406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:49.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.681+0000 7fec18c88640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febf406ae70 0x7febf406d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec04009ab0 tx=0x7fec04003990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:49.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.684+0000 7fec117fa640 1 -- 172.21.15.81:0/3474162868 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec1414e5a0 con 0x7fec140cc8e0 2023-11-08T16:29:49.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.887+0000 7fec1aeec640 1 -- 172.21.15.81:0/3474162868 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fec1414e5a0 con 0x7fec140cc8e0 2023-11-08T16:29:49.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.888+0000 7fec117fa640 1 -- 172.21.15.81:0/3474162868 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fec1414e5a0 con 0x7fec140cc8e0 2023-11-08T16:29:49.891 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:29:49.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.890+0000 7fec1aeec640 1 -- 172.21.15.81:0/3474162868 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febf406ae70 msgr2=0x7febf406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:49.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.890+0000 7fec1aeec640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febf406ae70 0x7febf406d330 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec04009ab0 tx=0x7fec04003990 comp rx=0 tx=0).stop 2023-11-08T16:29:49.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.890+0000 7fec1aeec640 1 -- 172.21.15.81:0/3474162868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec140cc8e0 msgr2=0x7fec141f65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:49.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.890+0000 7fec1aeec640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec140cc8e0 0x7fec141f65e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7febfc02b8c0 tx=0x7febfc0046c0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.891+0000 7fec1aeec640 1 -- 172.21.15.81:0/3474162868 shutdown_connections 2023-11-08T16:29:49.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.891+0000 7fec1aeec640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febf406ae70 0x7febf406d330 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.891+0000 7fec1aeec640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fec140cd2a0 0x7fec140c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.891+0000 7fec1aeec640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fec140cc8e0 0x7fec141f65e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.891+0000 7fec1aeec640 1 --2- 172.21.15.81:0/3474162868 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fec140c90f0 0x7fec14150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:49.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.891+0000 7fec1aeec640 1 -- 172.21.15.81:0/3474162868 >> 172.21.15.81:0/3474162868 conn(0x7fec140ceaa0 msgr2=0x7fec141637a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:49.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.891+0000 7fec1aeec640 1 -- 172.21.15.81:0/3474162868 shutdown_connections 2023-11-08T16:29:49.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:49.891+0000 7fec1aeec640 1 -- 172.21.15.81:0/3474162868 wait complete. 2023-11-08T16:29:49.913 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:29:49.913 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:29:49.967 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:50.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.175+0000 7f743d7e2640 1 Processor -- start 2023-11-08T16:29:50.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.176+0000 7f743d7e2640 1 -- start start 2023-11-08T16:29:50.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.177+0000 7f743d7e2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74380ca480 0x7f743815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.177+0000 7f743d7e2640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f743815ea60 0x7f743815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.177+0000 7f743d7e2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743806b630 con 0x7f74380ade50 2023-11-08T16:29:50.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.178+0000 7f743d7e2640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74380b1860 con 0x7f743815ea60 2023-11-08T16:29:50.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.178+0000 7f743d7e2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74380ca8e0 con 0x7f74380ca480 2023-11-08T16:29:50.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.178+0000 7f7436ffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f74380ade50 0x7f743815a680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:36038/0 (socket says 172.21.15.81:36038) 2023-11-08T16:29:50.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.178+0000 7f7436ffd640 1 -- 172.21.15.81:0/941125400 learned_addr learned my addr 172.21.15.81:0/941125400 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:50.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.178+0000 7f742e7fc640 1 --2- 172.21.15.81:0/941125400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74380ca480 0x7f743815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.178+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284850448 0 0) 0x7f743806b630 con 0x7f74380ade50 2023-11-08T16:29:50.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.178+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7424003520 con 0x7f74380ade50 2023-11-08T16:29:50.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.178+0000 7f74377fe640 1 --2- 172.21.15.81:0/941125400 >> v2:172.21.15.93:3300/0 conn(0x7f743815ea60 0x7f743815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1268271881 0 0) 0x7f7424003520 con 0x7f74380ade50 2023-11-08T16:29:50.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74380ca480 msgr2=0x7f743815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:50.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 --2- 172.21.15.81:0/941125400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74380ca480 0x7f743815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 >> v2:172.21.15.93:3300/0 conn(0x7f743815ea60 msgr2=0x7f743815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:50.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 --2- 172.21.15.81:0/941125400 >> v2:172.21.15.93:3300/0 conn(0x7f743815ea60 0x7f743815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74380cabb0 con 0x7f74380ade50 2023-11-08T16:29:50.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f7428001670 con 0x7f74380ade50 2023-11-08T16:29:50.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 >> v1:172.21.15.81:6789/0 conn(0x7f74380ade50 legacy=0x7f743815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:50.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 --2- 172.21.15.81:0/941125400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 0x7f7424003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 --2- 172.21.15.81:0/941125400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74380ca480 0x7f74240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 --2- 172.21.15.81:0/941125400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7424004d00 0x7f7424005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74380ca8e0 con 0x7f7424003a50 2023-11-08T16:29:50.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742400ba30 con 0x7f7424004d00 2023-11-08T16:29:50.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742400bba0 con 0x7f74380ca480 2023-11-08T16:29:50.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f7436ffd640 1 --2- 172.21.15.81:0/941125400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 0x7f7424003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f7436ffd640 1 -- 172.21.15.81:0/941125400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74380ca480 msgr2=0x7f74240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:50.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f7436ffd640 1 --2- 172.21.15.81:0/941125400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74380ca480 0x7f74240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f7436ffd640 1 -- 172.21.15.81:0/941125400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7424004d00 msgr2=0x7f7424005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:50.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f7436ffd640 1 --2- 172.21.15.81:0/941125400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7424004d00 0x7f7424005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.179+0000 7f7436ffd640 1 -- 172.21.15.81:0/941125400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7424003520 con 0x7f7424003a50 2023-11-08T16:29:50.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.180+0000 7f7436ffd640 1 --2- 172.21.15.81:0/941125400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 0x7f7424003e70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f742800c290 tx=0x7f74240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b322f7d12cee5a5a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:50.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.181+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428005ad0 con 0x7f7424003a50 2023-11-08T16:29:50.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.181+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7428005c30 con 0x7f7424003a50 2023-11-08T16:29:50.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.181+0000 7f74367fc640 1 -- 172.21.15.81:0/941125400 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742802c8c0 con 0x7f7424003a50 2023-11-08T16:29:50.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.181+0000 7f743d7e2640 1 -- 172.21.15.81:0/941125400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 msgr2=0x7f7424003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:50.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.181+0000 7f743d7e2640 1 --2- 172.21.15.81:0/941125400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 0x7f7424003e70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f742800c290 tx=0x7f74240057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.181+0000 7f74377fe640 1 -- 172.21.15.81:0/941125400 reap_dead start 2023-11-08T16:29:50.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.181+0000 7f743d7e2640 1 -- 172.21.15.81:0/941125400 shutdown_connections 2023-11-08T16:29:50.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.181+0000 7f743d7e2640 1 -- 172.21.15.81:0/941125400 >> 172.21.15.81:0/941125400 conn(0x7f74380cb490 msgr2=0x7f74380cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:50.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.182+0000 7f743d7e2640 1 -- 172.21.15.81:0/941125400 shutdown_connections 2023-11-08T16:29:50.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.182+0000 7f743d7e2640 1 -- 172.21.15.81:0/941125400 wait complete. 2023-11-08T16:29:50.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.183+0000 7f743d7e2640 1 Processor -- start 2023-11-08T16:29:50.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.184+0000 7f743d7e2640 1 -- start start 2023-11-08T16:29:50.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.184+0000 7f743d7e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 0x7f743806c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.184+0000 7f743d7e2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7438153de0 0x7f7438159a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.184+0000 7f743d7e2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7438154260 0x7f7438159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f743d7e2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74381587d0 con 0x7f7424003a50 2023-11-08T16:29:50.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f743d7e2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7438153340 con 0x7f7438154260 2023-11-08T16:29:50.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f743d7e2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74381584f0 con 0x7f7438153de0 2023-11-08T16:29:50.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f7436ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 0x7f743806c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f7436ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 0x7f743806c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55886/0 (socket says 172.21.15.81:55886) 2023-11-08T16:29:50.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f7436ffd640 1 -- 172.21.15.81:0/3622916718 learned_addr learned my addr 172.21.15.81:0/3622916718 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:50.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f742e7fc640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7438153de0 0x7f7438159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f74377fe640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7438154260 0x7f7438159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f7436ffd640 1 -- 172.21.15.81:0/3622916718 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7438153de0 msgr2=0x7f7438159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:50.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f7436ffd640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7438153de0 0x7f7438159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f7436ffd640 1 -- 172.21.15.81:0/3622916718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7438154260 msgr2=0x7f7438159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:50.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f7436ffd640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7438154260 0x7f7438159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.185+0000 7f7436ffd640 1 -- 172.21.15.81:0/3622916718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74381535e0 con 0x7f7424003a50 2023-11-08T16:29:50.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.186+0000 7f7436ffd640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 0x7f743806c7d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7428000c00 tx=0x7f7428000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:50.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.186+0000 7f742ffff640 1 -- 172.21.15.81:0/3622916718 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7428039070 con 0x7f7424003a50 2023-11-08T16:29:50.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.186+0000 7f742ffff640 1 -- 172.21.15.81:0/3622916718 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7428001660 con 0x7f7424003a50 2023-11-08T16:29:50.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.186+0000 7f742ffff640 1 -- 172.21.15.81:0/3622916718 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742803c040 con 0x7f7424003a50 2023-11-08T16:29:50.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.186+0000 7f743d7e2640 1 -- 172.21.15.81:0/3622916718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74381538c0 con 0x7f7424003a50 2023-11-08T16:29:50.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.186+0000 7f743d7e2640 1 -- 172.21.15.81:0/3622916718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743806cef0 con 0x7f7424003a50 2023-11-08T16:29:50.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.187+0000 7f743d7e2640 1 -- 172.21.15.81:0/3622916718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74381531a0 con 0x7f7424003a50 2023-11-08T16:29:50.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.188+0000 7f742ffff640 1 -- 172.21.15.81:0/3622916718 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7428032070 con 0x7f7424003a50 2023-11-08T16:29:50.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.189+0000 7f742ffff640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f741806aea0 0x7f741806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.189+0000 7f742ffff640 1 -- 172.21.15.81:0/3622916718 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f74280aab60 con 0x7f7424003a50 2023-11-08T16:29:50.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.189+0000 7f742e7fc640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f741806aea0 0x7f741806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.190+0000 7f742e7fc640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f741806aea0 0x7f741806d360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f74380afd00 tx=0x7f7420008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:50.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.194+0000 7f742ffff640 1 -- 172.21.15.81:0/3622916718 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74381531a0 con 0x7f7424003a50 2023-11-08T16:29:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.384+0000 7f743d7e2640 1 -- 172.21.15.81:0/3622916718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f743806cef0 con 0x7f7424003a50 2023-11-08T16:29:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.386+0000 7f742ffff640 1 -- 172.21.15.81:0/3622916718 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f743806cef0 con 0x7f7424003a50 2023-11-08T16:29:50.391 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:50.391 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:50.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 -- 172.21.15.81:0/3622916718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f741806aea0 msgr2=0x7f741806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:50.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f741806aea0 0x7f741806d360 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f74380afd00 tx=0x7f7420008040 comp rx=0 tx=0).stop 2023-11-08T16:29:50.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 -- 172.21.15.81:0/3622916718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 msgr2=0x7f743806c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 0x7f743806c7d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7428000c00 tx=0x7f7428000fe0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 -- 172.21.15.81:0/3622916718 shutdown_connections 2023-11-08T16:29:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f741806aea0 0x7f741806d360 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7438154260 0x7f7438159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7438153de0 0x7f7438159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 --2- 172.21.15.81:0/3622916718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7424003a50 0x7f743806c7d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 -- 172.21.15.81:0/3622916718 >> 172.21.15.81:0/3622916718 conn(0x7f74380cb490 msgr2=0x7f74380b36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 -- 172.21.15.81:0/3622916718 shutdown_connections 2023-11-08T16:29:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.389+0000 7f743d7e2640 1 -- 172.21.15.81:0/3622916718 wait complete. 2023-11-08T16:29:50.411 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:50.412 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:50.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.626+0000 7f4fd2da9640 1 Processor -- start 2023-11-08T16:29:50.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.626+0000 7f4fd2da9640 1 -- start start 2023-11-08T16:29:50.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.627+0000 7f4fd2da9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4fcc15a0e0 0x7f4fcc0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.627+0000 7f4fd2da9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f4fcc0c9bf0 0x7f4fcc0ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.627+0000 7f4fd2da9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4fcc0ca580 0x7f4fcc15dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.627+0000 7f4fd2da9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fcc071980 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.627+0000 7f4fd2da9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fcc0ce960 con 0x7f4fcc0c9bf0 2023-11-08T16:29:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.627+0000 7f4fd2da9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0c9480 con 0x7f4fcc0ca580 2023-11-08T16:29:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.628+0000 7f4fd0b45640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4fcc15a0e0 0x7f4fcc0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.628+0000 7f4fd0b45640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4fcc15a0e0 0x7f4fcc0ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55888/0 (socket says 172.21.15.81:55888) 2023-11-08T16:29:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.628+0000 7f4fd0b45640 1 -- 172.21.15.81:0/3714827133 learned_addr learned my addr 172.21.15.81:0/3714827133 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.628+0000 7f4fd0b45640 1 -- 172.21.15.81:0/3714827133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4fcc0ca580 msgr2=0x7f4fcc15dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.628+0000 7f4fd0b45640 1 --2- 172.21.15.81:0/3714827133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4fcc0ca580 0x7f4fcc15dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.628+0000 7f4fd0b45640 1 -- 172.21.15.81:0/3714827133 >> v2:172.21.15.93:3300/0 conn(0x7f4fcc0c9bf0 msgr2=0x7f4fcc0ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.628+0000 7f4fc3fff640 1 --2- 172.21.15.81:0/3714827133 >> v2:172.21.15.93:3300/0 conn(0x7f4fcc0c9bf0 0x7f4fcc0ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.628+0000 7f4fd0b45640 1 --2- 172.21.15.81:0/3714827133 >> v2:172.21.15.93:3300/0 conn(0x7f4fcc0c9bf0 0x7f4fcc0ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.628+0000 7f4fd0b45640 1 -- 172.21.15.81:0/3714827133 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc15a4c0 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.628+0000 7f4fc3fff640 1 --2- 172.21.15.81:0/3714827133 >> v2:172.21.15.93:3300/0 conn(0x7f4fcc0c9bf0 0x7f4fcc0ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.629+0000 7f4fd0b45640 1 --2- 172.21.15.81:0/3714827133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fcc15a0e0 0x7f4fcc0ccc10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc06b4b0 tx=0x7f4fc802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c630001ad9f0f2f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.629+0000 7f4fc37fe640 1 -- 172.21.15.81:0/3714827133 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8030860 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.630+0000 7f4fc37fe640 1 -- 172.21.15.81:0/3714827133 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc80309c0 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.630+0000 7f4fc37fe640 1 -- 172.21.15.81:0/3714827133 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8030d50 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.630+0000 7f4fd2da9640 1 -- 172.21.15.81:0/3714827133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fcc15a0e0 msgr2=0x7f4fcc0ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.630+0000 7f4fd2da9640 1 --2- 172.21.15.81:0/3714827133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fcc15a0e0 0x7f4fcc0ccc10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc06b4b0 tx=0x7f4fc802b9a0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.630+0000 7f4fd2da9640 1 -- 172.21.15.81:0/3714827133 shutdown_connections 2023-11-08T16:29:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.630+0000 7f4fd2da9640 1 --2- 172.21.15.81:0/3714827133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4fcc0ca580 0x7f4fcc15dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.630+0000 7f4fd2da9640 1 --2- 172.21.15.81:0/3714827133 >> v2:172.21.15.93:3300/0 conn(0x7f4fcc0c9bf0 0x7f4fcc0ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.630+0000 7f4fd2da9640 1 --2- 172.21.15.81:0/3714827133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fcc15a0e0 0x7f4fcc0ccc10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.630+0000 7f4fd2da9640 1 -- 172.21.15.81:0/3714827133 >> 172.21.15.81:0/3714827133 conn(0x7f4fcc0cb580 msgr2=0x7f4fcc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.630+0000 7f4fd2da9640 1 -- 172.21.15.81:0/3714827133 shutdown_connections 2023-11-08T16:29:50.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.630+0000 7f4fd2da9640 1 -- 172.21.15.81:0/3714827133 wait complete. 2023-11-08T16:29:50.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.632+0000 7f4fd2da9640 1 Processor -- start 2023-11-08T16:29:50.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.632+0000 7f4fd2da9640 1 -- start start 2023-11-08T16:29:50.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.633+0000 7f4fd2da9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4fcc0c9bf0 0x7f4fcc0c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.633+0000 7f4fd2da9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fcc0ca580 0x7f4fcc1f6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.633+0000 7f4fd2da9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fcc15a0e0 0x7f4fcc0c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.633+0000 7f4fd2da9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc1f6530 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.633+0000 7f4fd2da9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0c4580 con 0x7f4fcc0ca580 2023-11-08T16:29:50.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.633+0000 7f4fd2da9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0c4860 con 0x7f4fcc0c9bf0 2023-11-08T16:29:50.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.633+0000 7f4fd1346640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fcc15a0e0 0x7f4fcc0c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.633+0000 7f4fd1346640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fcc15a0e0 0x7f4fcc0c3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55904/0 (socket says 172.21.15.81:55904) 2023-11-08T16:29:50.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.633+0000 7f4fd1346640 1 -- 172.21.15.81:0/420229520 learned_addr learned my addr 172.21.15.81:0/420229520 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:50.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fd0b45640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4fcc0c9bf0 0x7f4fcc0c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fc3fff640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fcc0ca580 0x7f4fcc1f6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fd1346640 1 -- 172.21.15.81:0/420229520 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4fcc0c9bf0 msgr2=0x7f4fcc0c8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fd1346640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4fcc0c9bf0 0x7f4fcc0c8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fd1346640 1 -- 172.21.15.81:0/420229520 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fcc0ca580 msgr2=0x7f4fcc1f6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fd1346640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fcc0ca580 0x7f4fcc1f6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fd1346640 1 -- 172.21.15.81:0/420229520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc80094d0 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fd1346640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fcc15a0e0 0x7f4fcc0c3d30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400b970 tx=0x7f4fc400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fc17fa640 1 -- 172.21.15.81:0/420229520 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400cb40 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fc17fa640 1 -- 172.21.15.81:0/420229520 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc400ccc0 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fc17fa640 1 -- 172.21.15.81:0/420229520 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4014680 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.634+0000 7f4fd2da9640 1 -- 172.21.15.81:0/420229520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc1f6780 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.635+0000 7f4fd2da9640 1 -- 172.21.15.81:0/420229520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc0c4270 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.636+0000 7f4fd2da9640 1 -- 172.21.15.81:0/420229520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc04ece0 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.637+0000 7f4fc17fa640 1 -- 172.21.15.81:0/420229520 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4fc401d3b0 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.638+0000 7f4fc17fa640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4fb806ae70 0x7f4fb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.638+0000 7f4fc17fa640 1 -- 172.21.15.81:0/420229520 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4fc4089630 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.638+0000 7f4fd0b45640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4fb806ae70 0x7f4fb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:50.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.639+0000 7f4fd0b45640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4fb806ae70 0x7f4fb806d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8036000 tx=0x7f4fc8008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:50.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.642+0000 7f4fc17fa640 1 -- 172.21.15.81:0/420229520 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4fc4054640 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.834+0000 7f4fd2da9640 1 -- 172.21.15.81:0/420229520 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4fcc071450 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.835+0000 7f4fc17fa640 1 -- 172.21.15.81:0/420229520 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4fc4057c60 con 0x7f4fcc15a0e0 2023-11-08T16:29:50.838 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:50.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.837+0000 7f4fd2da9640 1 -- 172.21.15.81:0/420229520 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4fb806ae70 msgr2=0x7f4fb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:50.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.837+0000 7f4fd2da9640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4fb806ae70 0x7f4fb806d330 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8036000 tx=0x7f4fc8008030 comp rx=0 tx=0).stop 2023-11-08T16:29:50.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.837+0000 7f4fd2da9640 1 -- 172.21.15.81:0/420229520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fcc15a0e0 msgr2=0x7f4fcc0c3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:50.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.837+0000 7f4fd2da9640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fcc15a0e0 0x7f4fcc0c3d30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400b970 tx=0x7f4fc400be30 comp rx=0 tx=0).stop 2023-11-08T16:29:50.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.838+0000 7f4fd2da9640 1 -- 172.21.15.81:0/420229520 shutdown_connections 2023-11-08T16:29:50.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.838+0000 7f4fd2da9640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4fb806ae70 0x7f4fb806d330 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.838+0000 7f4fd2da9640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4fcc15a0e0 0x7f4fcc0c3d30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.838+0000 7f4fd2da9640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4fcc0ca580 0x7f4fcc1f6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.838+0000 7f4fd2da9640 1 --2- 172.21.15.81:0/420229520 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4fcc0c9bf0 0x7f4fcc0c8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.838+0000 7f4fd2da9640 1 -- 172.21.15.81:0/420229520 >> 172.21.15.81:0/420229520 conn(0x7f4fcc0cb580 msgr2=0x7f4fcc1591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.838+0000 7f4fd2da9640 1 -- 172.21.15.81:0/420229520 shutdown_connections 2023-11-08T16:29:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:50.838+0000 7f4fd2da9640 1 -- 172.21.15.81:0/420229520 wait complete. 2023-11-08T16:29:50.861 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:51.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.072+0000 7ff98651d640 1 Processor -- start 2023-11-08T16:29:51.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.072+0000 7ff98651d640 1 -- start start 2023-11-08T16:29:51.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.073+0000 7ff98651d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff980159920 0x7ff98015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff98651d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff98006d330 0x7ff9800cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff98651d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff9800cd7d0 0x7ff98015d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff98651d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9800ce5e0 con 0x7ff9800cd7d0 2023-11-08T16:29:51.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff98651d640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9800cdc60 con 0x7ff980159920 2023-11-08T16:29:51.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff98651d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98006fc70 con 0x7ff98006d330 2023-11-08T16:29:51.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff984aba640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff9800cd7d0 0x7ff98015d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff984aba640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff9800cd7d0 0x7ff98015d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46816/0 (socket says 172.21.15.81:46816) 2023-11-08T16:29:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff984aba640 1 -- 172.21.15.81:0/105739987 learned_addr learned my addr 172.21.15.81:0/105739987 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff97ffff640 1 --2- 172.21.15.81:0/105739987 >> v2:172.21.15.93:3300/0 conn(0x7ff980159920 0x7ff98015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff977fff640 1 --2- 172.21.15.81:0/105739987 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff98006d330 0x7ff9800cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff984aba640 1 -- 172.21.15.81:0/105739987 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff98006d330 msgr2=0x7ff9800cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff984aba640 1 --2- 172.21.15.81:0/105739987 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff98006d330 0x7ff9800cd290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff984aba640 1 -- 172.21.15.81:0/105739987 >> v2:172.21.15.93:3300/0 conn(0x7ff980159920 msgr2=0x7ff98015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:51.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff984aba640 1 --2- 172.21.15.81:0/105739987 >> v2:172.21.15.93:3300/0 conn(0x7ff980159920 0x7ff98015a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff984aba640 1 -- 172.21.15.81:0/105739987 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98015ae00 con 0x7ff9800cd7d0 2023-11-08T16:29:51.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff977fff640 1 --2- 172.21.15.81:0/105739987 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff98006d330 0x7ff9800cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:51.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.074+0000 7ff97ffff640 1 --2- 172.21.15.81:0/105739987 >> v2:172.21.15.93:3300/0 conn(0x7ff980159920 0x7ff98015a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:51.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.075+0000 7ff984aba640 1 --2- 172.21.15.81:0/105739987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9800cd7d0 0x7ff98015d3b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff98006b0e0 tx=0x7ff97802e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a525e562242f6e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:51.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.076+0000 7ff97f7fe640 1 -- 172.21.15.81:0/105739987 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97802ea70 con 0x7ff9800cd7d0 2023-11-08T16:29:51.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.076+0000 7ff97f7fe640 1 -- 172.21.15.81:0/105739987 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97802ebd0 con 0x7ff9800cd7d0 2023-11-08T16:29:51.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.076+0000 7ff97f7fe640 1 -- 172.21.15.81:0/105739987 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff978009600 con 0x7ff9800cd7d0 2023-11-08T16:29:51.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.076+0000 7ff98651d640 1 -- 172.21.15.81:0/105739987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9800cd7d0 msgr2=0x7ff98015d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.076+0000 7ff98651d640 1 --2- 172.21.15.81:0/105739987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9800cd7d0 0x7ff98015d3b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff98006b0e0 tx=0x7ff97802e0d0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.077+0000 7ff98651d640 1 -- 172.21.15.81:0/105739987 shutdown_connections 2023-11-08T16:29:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.077+0000 7ff98651d640 1 --2- 172.21.15.81:0/105739987 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff98006d330 0x7ff9800cd290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.077+0000 7ff98651d640 1 --2- 172.21.15.81:0/105739987 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9800cd7d0 0x7ff98015d3b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.077+0000 7ff98651d640 1 --2- 172.21.15.81:0/105739987 >> v2:172.21.15.93:3300/0 conn(0x7ff980159920 0x7ff98015a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.077+0000 7ff98651d640 1 -- 172.21.15.81:0/105739987 >> 172.21.15.81:0/105739987 conn(0x7ff9800cb580 msgr2=0x7ff9800cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.077+0000 7ff98651d640 1 -- 172.21.15.81:0/105739987 shutdown_connections 2023-11-08T16:29:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.077+0000 7ff98651d640 1 -- 172.21.15.81:0/105739987 wait complete. 2023-11-08T16:29:51.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.078+0000 7ff98651d640 1 Processor -- start 2023-11-08T16:29:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.078+0000 7ff98651d640 1 -- start start 2023-11-08T16:29:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.079+0000 7ff98651d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff98006d330 0x7ff980155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.079+0000 7ff98651d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9800cd7d0 0x7ff9800aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.079+0000 7ff98651d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff980159920 0x7ff9800af470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.079+0000 7ff98651d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9801556a0 con 0x7ff98006d330 2023-11-08T16:29:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.079+0000 7ff98651d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800ad960 con 0x7ff9800cd7d0 2023-11-08T16:29:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.079+0000 7ff98651d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800adc40 con 0x7ff980159920 2023-11-08T16:29:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.080+0000 7ff97ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff98006d330 0x7ff980155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.080+0000 7ff97ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff98006d330 0x7ff980155160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46820/0 (socket says 172.21.15.81:46820) 2023-11-08T16:29:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.080+0000 7ff97ffff640 1 -- 172.21.15.81:0/2508275969 learned_addr learned my addr 172.21.15.81:0/2508275969 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:51.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.080+0000 7ff977fff640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9800cd7d0 0x7ff9800aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.080+0000 7ff984aba640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff980159920 0x7ff9800af470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.080+0000 7ff97ffff640 1 -- 172.21.15.81:0/2508275969 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff980159920 msgr2=0x7ff9800af470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.080+0000 7ff97ffff640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff980159920 0x7ff9800af470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.080+0000 7ff97ffff640 1 -- 172.21.15.81:0/2508275969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9800cd7d0 msgr2=0x7ff9800aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.080+0000 7ff97ffff640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9800cd7d0 0x7ff9800aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.080+0000 7ff97ffff640 1 -- 172.21.15.81:0/2508275969 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff97800bd40 con 0x7ff98006d330 2023-11-08T16:29:51.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.081+0000 7ff97ffff640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff98006d330 0x7ff980155160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c7f0 tx=0x7ff97000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:51.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.081+0000 7ff97d7fa640 1 -- 172.21.15.81:0/2508275969 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970007820 con 0x7ff98006d330 2023-11-08T16:29:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.081+0000 7ff97d7fa640 1 -- 172.21.15.81:0/2508275969 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff970007980 con 0x7ff98006d330 2023-11-08T16:29:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.081+0000 7ff97d7fa640 1 -- 172.21.15.81:0/2508275969 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970007cf0 con 0x7ff98006d330 2023-11-08T16:29:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.081+0000 7ff98651d640 1 -- 172.21.15.81:0/2508275969 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9800adf20 con 0x7ff98006d330 2023-11-08T16:29:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.081+0000 7ff98651d640 1 -- 172.21.15.81:0/2508275969 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9800c6370 con 0x7ff98006d330 2023-11-08T16:29:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.082+0000 7ff98651d640 1 -- 172.21.15.81:0/2508275969 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9800c99d0 con 0x7ff98006d330 2023-11-08T16:29:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.083+0000 7ff97d7fa640 1 -- 172.21.15.81:0/2508275969 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff9700113f0 con 0x7ff98006d330 2023-11-08T16:29:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.084+0000 7ff97d7fa640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff96006aea0 0x7ff96006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.084+0000 7ff97d7fa640 1 -- 172.21.15.81:0/2508275969 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff970089990 con 0x7ff98006d330 2023-11-08T16:29:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.084+0000 7ff977fff640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff96006aea0 0x7ff96006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.085+0000 7ff977fff640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff96006aea0 0x7ff96006d360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff968006fd0 tx=0x7ff968008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:51.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.088+0000 7ff97d7fa640 1 -- 172.21.15.81:0/2508275969 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff9700549a0 con 0x7ff98006d330 2023-11-08T16:29:51.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.277+0000 7ff98651d640 1 -- 172.21.15.81:0/2508275969 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff98004ece0 con 0x7ff98006d330 2023-11-08T16:29:51.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.278+0000 7ff97d7fa640 1 -- 172.21.15.81:0/2508275969 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff97001f020 con 0x7ff98006d330 2023-11-08T16:29:51.281 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 -- 172.21.15.81:0/2508275969 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff96006aea0 msgr2=0x7ff96006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff96006aea0 0x7ff96006d360 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff968006fd0 tx=0x7ff968008040 comp rx=0 tx=0).stop 2023-11-08T16:29:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 -- 172.21.15.81:0/2508275969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff98006d330 msgr2=0x7ff980155160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff98006d330 0x7ff980155160 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff97000c7f0 tx=0x7ff97000ccb0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 -- 172.21.15.81:0/2508275969 shutdown_connections 2023-11-08T16:29:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff96006aea0 0x7ff96006d360 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff980159920 0x7ff9800af470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9800cd7d0 0x7ff9800aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 --2- 172.21.15.81:0/2508275969 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff98006d330 0x7ff980155160 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 -- 172.21.15.81:0/2508275969 >> 172.21.15.81:0/2508275969 conn(0x7ff9800cb580 msgr2=0x7ff9800b30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 -- 172.21.15.81:0/2508275969 shutdown_connections 2023-11-08T16:29:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.281+0000 7ff98651d640 1 -- 172.21.15.81:0/2508275969 wait complete. 2023-11-08T16:29:51.302 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:51.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.515+0000 7f92a8da5640 1 Processor -- start 2023-11-08T16:29:51.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.515+0000 7f92a8da5640 1 -- start start 2023-11-08T16:29:51.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.517+0000 7f92a8da5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f92a40ce0d0 0x7f92a40caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.517+0000 7f92a8da5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f92a415f280 0x7f92a41516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.517+0000 7f92a8da5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a406fa30 con 0x7f92a415bfc0 2023-11-08T16:29:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.517+0000 7f92a8da5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a40cda10 con 0x7f92a415f280 2023-11-08T16:29:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.517+0000 7f92a8da5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a40ce4b0 con 0x7f92a40ce0d0 2023-11-08T16:29:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.517+0000 7f92a2d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f92a415f280 0x7f92a41516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.517+0000 7f92a2d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f92a415f280 0x7f92a41516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46830/0 (socket says 172.21.15.81:46830) 2023-11-08T16:29:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.517+0000 7f92a2d9d640 1 -- 172.21.15.81:0/596161074 learned_addr learned my addr 172.21.15.81:0/596161074 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.518+0000 7f92a259c640 1 --2- 172.21.15.81:0/596161074 >> v2:172.21.15.93:3300/0 conn(0x7f92a40ce0d0 0x7f92a40caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.518+0000 7f92a2d9d640 1 -- 172.21.15.81:0/596161074 >> v2:172.21.15.93:3300/0 conn(0x7f92a40ce0d0 msgr2=0x7f92a40caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.518+0000 7f92a2d9d640 1 --2- 172.21.15.81:0/596161074 >> v2:172.21.15.93:3300/0 conn(0x7f92a40ce0d0 0x7f92a40caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.518+0000 7f92a2d9d640 1 -- 172.21.15.81:0/596161074 >> v1:172.21.15.93:6789/0 conn(0x7f92a415bfc0 legacy=0x7f92a415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:51.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.518+0000 7f92a2d9d640 1 -- 172.21.15.81:0/596161074 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a406fca0 con 0x7f92a415f280 2023-11-08T16:29:51.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.519+0000 7f92a2d9d640 1 --2- 172.21.15.81:0/596161074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92a415f280 0x7f92a41516e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f92a406f3a0 tx=0x7f929c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19e3fe052711bc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:51.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.519+0000 7f92a159a640 1 -- 172.21.15.81:0/596161074 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c02ca80 con 0x7f92a415f280 2023-11-08T16:29:51.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.519+0000 7f92a159a640 1 -- 172.21.15.81:0/596161074 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c02cbe0 con 0x7f92a415f280 2023-11-08T16:29:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.519+0000 7f92a159a640 1 -- 172.21.15.81:0/596161074 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c0343f0 con 0x7f92a415f280 2023-11-08T16:29:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.520+0000 7f92a8da5640 1 -- 172.21.15.81:0/596161074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92a415f280 msgr2=0x7f92a41516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.520+0000 7f92a8da5640 1 --2- 172.21.15.81:0/596161074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92a415f280 0x7f92a41516e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f92a406f3a0 tx=0x7f929c02bee0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.520+0000 7f92a8da5640 1 -- 172.21.15.81:0/596161074 shutdown_connections 2023-11-08T16:29:51.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.520+0000 7f92a8da5640 1 --2- 172.21.15.81:0/596161074 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92a415f280 0x7f92a41516e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.520+0000 7f92a8da5640 1 --2- 172.21.15.81:0/596161074 >> v2:172.21.15.93:3300/0 conn(0x7f92a40ce0d0 0x7f92a40caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.520+0000 7f92a8da5640 1 -- 172.21.15.81:0/596161074 >> 172.21.15.81:0/596161074 conn(0x7f92a40cb580 msgr2=0x7f92a40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:51.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.520+0000 7f92a8da5640 1 -- 172.21.15.81:0/596161074 shutdown_connections 2023-11-08T16:29:51.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.520+0000 7f92a8da5640 1 -- 172.21.15.81:0/596161074 wait complete. 2023-11-08T16:29:51.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.522+0000 7f92a8da5640 1 Processor -- start 2023-11-08T16:29:51.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.522+0000 7f92a8da5640 1 -- start start 2023-11-08T16:29:51.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.523+0000 7f92a8da5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f92a40ce0d0 0x7f92a40c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.523+0000 7f92a8da5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f92a415bfc0 0x7f92a4155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.523+0000 7f92a8da5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92a415f280 0x7f92a4153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.523+0000 7f92a8da5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40c3b70 con 0x7f92a415f280 2023-11-08T16:29:51.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.523+0000 7f92a8da5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40c2c40 con 0x7f92a415bfc0 2023-11-08T16:29:51.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.523+0000 7f92a8da5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40c2f20 con 0x7f92a40ce0d0 2023-11-08T16:29:51.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.523+0000 7f92a2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92a415f280 0x7f92a4153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.523+0000 7f92a2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92a415f280 0x7f92a4153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46846/0 (socket says 172.21.15.81:46846) 2023-11-08T16:29:51.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.523+0000 7f92a2d9d640 1 -- 172.21.15.81:0/3130815773 learned_addr learned my addr 172.21.15.81:0/3130815773 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:51.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f92a259c640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f92a40ce0d0 0x7f92a40c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f92a1d9b640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f92a415bfc0 0x7f92a4155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f92a2d9d640 1 -- 172.21.15.81:0/3130815773 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f92a40ce0d0 msgr2=0x7f92a40c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f92a2d9d640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f92a40ce0d0 0x7f92a40c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f92a2d9d640 1 -- 172.21.15.81:0/3130815773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f92a415bfc0 msgr2=0x7f92a4155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f92a2d9d640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f92a415bfc0 0x7f92a4155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f92a2d9d640 1 -- 172.21.15.81:0/3130815773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c009b30 con 0x7f92a415f280 2023-11-08T16:29:51.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f92a259c640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f92a40ce0d0 0x7f92a40c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:51.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f92a2d9d640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92a415f280 0x7f92a4153c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f929c036040 tx=0x7f929c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:51.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f9292ffd640 1 -- 172.21.15.81:0/3130815773 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c009510 con 0x7f92a415f280 2023-11-08T16:29:51.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f9292ffd640 1 -- 172.21.15.81:0/3130815773 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f929c03e450 con 0x7f92a415f280 2023-11-08T16:29:51.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f9292ffd640 1 -- 172.21.15.81:0/3130815773 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f929c03e7e0 con 0x7f92a415f280 2023-11-08T16:29:51.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.524+0000 7f92a8da5640 1 -- 172.21.15.81:0/3130815773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a40c3200 con 0x7f92a415f280 2023-11-08T16:29:51.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.525+0000 7f92a8da5640 1 -- 172.21.15.81:0/3130815773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a4154180 con 0x7f92a415f280 2023-11-08T16:29:51.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.526+0000 7f92a8da5640 1 -- 172.21.15.81:0/3130815773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a40c3cb0 con 0x7f92a415f280 2023-11-08T16:29:51.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.527+0000 7f9292ffd640 1 -- 172.21.15.81:0/3130815773 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f929c02cd50 con 0x7f92a415f280 2023-11-08T16:29:51.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.528+0000 7f9292ffd640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f928406ae70 0x7f928406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.528+0000 7f9292ffd640 1 -- 172.21.15.81:0/3130815773 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f929c0ac1b0 con 0x7f92a415f280 2023-11-08T16:29:51.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.528+0000 7f92a259c640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f928406ae70 0x7f928406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.529+0000 7f92a259c640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f928406ae70 0x7f928406d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9294006fd0 tx=0x7f9294008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:51.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.532+0000 7f9292ffd640 1 -- 172.21.15.81:0/3130815773 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f92a40c3cb0 con 0x7f92a415f280 2023-11-08T16:29:51.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.722+0000 7f92a8da5640 1 -- 172.21.15.81:0/3130815773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f92a40c3cb0 con 0x7f92a415f280 2023-11-08T16:29:51.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.723+0000 7f9292ffd640 1 -- 172.21.15.81:0/3130815773 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f92a40c3cb0 con 0x7f92a415f280 2023-11-08T16:29:51.726 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:51.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 -- 172.21.15.81:0/3130815773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f928406ae70 msgr2=0x7f928406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f928406ae70 0x7f928406d330 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9294006fd0 tx=0x7f9294008040 comp rx=0 tx=0).stop 2023-11-08T16:29:51.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 -- 172.21.15.81:0/3130815773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92a415f280 msgr2=0x7f92a4153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92a415f280 0x7f92a4153c40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f929c036040 tx=0x7f929c004a40 comp rx=0 tx=0).stop 2023-11-08T16:29:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 -- 172.21.15.81:0/3130815773 shutdown_connections 2023-11-08T16:29:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f928406ae70 0x7f928406d330 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f92a415f280 0x7f92a4153c40 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f92a415bfc0 0x7f92a4155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 --2- 172.21.15.81:0/3130815773 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f92a40ce0d0 0x7f92a40c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 -- 172.21.15.81:0/3130815773 >> 172.21.15.81:0/3130815773 conn(0x7f92a40cb580 msgr2=0x7f92a40cb960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:51.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 -- 172.21.15.81:0/3130815773 shutdown_connections 2023-11-08T16:29:51.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.726+0000 7f92a8da5640 1 -- 172.21.15.81:0/3130815773 wait complete. 2023-11-08T16:29:51.748 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:29:51.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.965+0000 7fb4857e7640 1 Processor -- start 2023-11-08T16:29:51.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.965+0000 7fb4857e7640 1 -- start start 2023-11-08T16:29:51.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.967+0000 7fb4857e7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4800cf720 0x7fb48006d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.967+0000 7fb4857e7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48006cc50 con 0x7fb4800ca130 2023-11-08T16:29:51.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.967+0000 7fb4857e7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4800c90f0 con 0x7fb48015d3b0 2023-11-08T16:29:51.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.967+0000 7fb4857e7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800adb70 con 0x7fb4800cf720 2023-11-08T16:29:51.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.967+0000 7fb47f7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb4800ca130 0x7fb480070c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52686/0 (socket says 172.21.15.81:52686) 2023-11-08T16:29:51.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.967+0000 7fb47f7fe640 1 -- 172.21.15.81:0/391363753 learned_addr learned my addr 172.21.15.81:0/391363753 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:51.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.967+0000 7fb47e7fc640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4800cf720 0x7fb48006d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524498788 0 0) 0x7fb48006cc50 con 0x7fb4800ca130 2023-11-08T16:29:51.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb468003520 con 0x7fb4800ca130 2023-11-08T16:29:51.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 103308991 0 0) 0x7fb468003520 con 0x7fb4800ca130 2023-11-08T16:29:51.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4800cf720 msgr2=0x7fb48006d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:51.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4800cf720 0x7fb48006d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 >> v1:172.21.15.93:6789/0 conn(0x7fb48015d3b0 legacy=0x7fb4800c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4800c9360 con 0x7fb4800ca130 2023-11-08T16:29:51.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fb4780019f0 con 0x7fb4800ca130 2023-11-08T16:29:51.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 >> v1:172.21.15.81:6789/0 conn(0x7fb4800ca130 legacy=0x7fb480070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4680039a0 0x7fb468003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4800cf720 0x7fb4680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb468004d00 0x7fb468005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800adb70 con 0x7fb4680039a0 2023-11-08T16:29:51.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb46800ba30 con 0x7fb468004d00 2023-11-08T16:29:51.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.968+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb46800de00 con 0x7fb4800cf720 2023-11-08T16:29:51.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.969+0000 7fb47effd640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4680039a0 0x7fb468003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.969+0000 7fb47e7fc640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4800cf720 0x7fb4680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.969+0000 7fb47effd640 1 -- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4800cf720 msgr2=0x7fb4680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:51.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.969+0000 7fb47effd640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4800cf720 0x7fb4680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.969+0000 7fb47f7fe640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb468004d00 0x7fb468005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.969+0000 7fb47effd640 1 -- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb468004d00 msgr2=0x7fb468005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:51.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.969+0000 7fb47effd640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb468004d00 0x7fb468005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.969+0000 7fb47effd640 1 -- 172.21.15.81:0/391363753 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb468003520 con 0x7fb4680039a0 2023-11-08T16:29:51.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.969+0000 7fb47e7fc640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4800cf720 0x7fb4680047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:51.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.970+0000 7fb47effd640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4680039a0 0x7fb468003dc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb470007fd0 tx=0x7fb4680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e79490adf701f17 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:51.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.970+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4700052d0 con 0x7fb4680039a0 2023-11-08T16:29:51.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.970+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47002e050 con 0x7fb4680039a0 2023-11-08T16:29:51.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.970+0000 7fb47dffb640 1 -- 172.21.15.81:0/391363753 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb470003640 con 0x7fb4680039a0 2023-11-08T16:29:51.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.971+0000 7fb4857e7640 1 -- 172.21.15.81:0/391363753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4680039a0 msgr2=0x7fb468003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.971+0000 7fb4857e7640 1 --2- 172.21.15.81:0/391363753 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4680039a0 0x7fb468003dc0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb470007fd0 tx=0x7fb4680057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.971+0000 7fb47f7fe640 1 -- 172.21.15.81:0/391363753 reap_dead start 2023-11-08T16:29:51.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.971+0000 7fb4857e7640 1 -- 172.21.15.81:0/391363753 shutdown_connections 2023-11-08T16:29:51.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.971+0000 7fb4857e7640 1 -- 172.21.15.81:0/391363753 >> 172.21.15.81:0/391363753 conn(0x7fb4800ceaa0 msgr2=0x7fb4800cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:51.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.971+0000 7fb4857e7640 1 -- 172.21.15.81:0/391363753 shutdown_connections 2023-11-08T16:29:51.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.971+0000 7fb4857e7640 1 -- 172.21.15.81:0/391363753 wait complete. 2023-11-08T16:29:51.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.973+0000 7fb4857e7640 1 Processor -- start 2023-11-08T16:29:51.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.973+0000 7fb4857e7640 1 -- start start 2023-11-08T16:29:51.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.974+0000 7fb4857e7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4680039a0 0x7fb4801f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.974+0000 7fb4857e7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4800c7250 0x7fb4800c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.974+0000 7fb4857e7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4800cd120 0x7fb480150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.974+0000 7fb4857e7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800c7900 con 0x7fb4800c7250 2023-11-08T16:29:51.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.974+0000 7fb4857e7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4800c2b10 con 0x7fb4800cd120 2023-11-08T16:29:51.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.974+0000 7fb4857e7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4801f6410 con 0x7fb4680039a0 2023-11-08T16:29:51.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.974+0000 7fb47e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4800c7250 0x7fb4800c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.974+0000 7fb47e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4800c7250 0x7fb4800c25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46858/0 (socket says 172.21.15.81:46858) 2023-11-08T16:29:51.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.974+0000 7fb47e7fc640 1 -- 172.21.15.81:0/316518626 learned_addr learned my addr 172.21.15.81:0/316518626 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:51.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.974+0000 7fb47f7fe640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4800cd120 0x7fb480150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb47e7fc640 1 -- 172.21.15.81:0/316518626 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4680039a0 msgr2=0x7fb4801f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb47effd640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4680039a0 0x7fb4801f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb47e7fc640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4680039a0 0x7fb4801f5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb47e7fc640 1 -- 172.21.15.81:0/316518626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4800cd120 msgr2=0x7fb480150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:51.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb47e7fc640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4800cd120 0x7fb480150ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:51.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb47e7fc640 1 -- 172.21.15.81:0/316518626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4801f66f0 con 0x7fb4800c7250 2023-11-08T16:29:51.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb47f7fe640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4800cd120 0x7fb480150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:51.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb47e7fc640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4800c7250 0x7fb4800c25d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb4740108e0 tx=0x7fb474010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:51.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb4677fe640 1 -- 172.21.15.81:0/316518626 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47400ec20 con 0x7fb4800c7250 2023-11-08T16:29:51.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb4677fe640 1 -- 172.21.15.81:0/316518626 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47400ed80 con 0x7fb4800c7250 2023-11-08T16:29:51.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb4857e7640 1 -- 172.21.15.81:0/316518626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4801f69d0 con 0x7fb4800c7250 2023-11-08T16:29:51.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb4677fe640 1 -- 172.21.15.81:0/316518626 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47401c600 con 0x7fb4800c7250 2023-11-08T16:29:51.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.975+0000 7fb4857e7640 1 -- 172.21.15.81:0/316518626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4801f6cb0 con 0x7fb4800c7250 2023-11-08T16:29:51.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.976+0000 7fb4857e7640 1 -- 172.21.15.81:0/316518626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4800743f0 con 0x7fb4800c7250 2023-11-08T16:29:51.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.977+0000 7fb4677fe640 1 -- 172.21.15.81:0/316518626 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb47401c760 con 0x7fb4800c7250 2023-11-08T16:29:51.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.978+0000 7fb4677fe640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 0x7fb46c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:51.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.978+0000 7fb4677fe640 1 -- 172.21.15.81:0/316518626 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb474089ce0 con 0x7fb4800c7250 2023-11-08T16:29:51.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.979+0000 7fb47effd640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 0x7fb46c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:51.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.979+0000 7fb47effd640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 0x7fb46c06d330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb470033000 tx=0x7fb470009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:51.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:51.983+0000 7fb4677fe640 1 -- 172.21.15.81:0/316518626 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb474054da0 con 0x7fb4800c7250 2023-11-08T16:29:52.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.187+0000 7fb4857e7640 1 -- 172.21.15.81:0/316518626 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb4800c7a40 con 0x7fb4800c7250 2023-11-08T16:29:52.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.187+0000 7fb4677fe640 1 -- 172.21.15.81:0/316518626 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb4800c7a40 con 0x7fb4800c7250 2023-11-08T16:29:52.190 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:29:52.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 -- 172.21.15.81:0/316518626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 msgr2=0x7fb46c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:52.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 0x7fb46c06d330 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb470033000 tx=0x7fb470009030 comp rx=0 tx=0).stop 2023-11-08T16:29:52.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 -- 172.21.15.81:0/316518626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4800c7250 msgr2=0x7fb4800c25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:52.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4800c7250 0x7fb4800c25d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb4740108e0 tx=0x7fb474010da0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 -- 172.21.15.81:0/316518626 shutdown_connections 2023-11-08T16:29:52.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 0x7fb46c06d330 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4800cd120 0x7fb480150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4800c7250 0x7fb4800c25d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 --2- 172.21.15.81:0/316518626 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4680039a0 0x7fb4801f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 -- 172.21.15.81:0/316518626 >> 172.21.15.81:0/316518626 conn(0x7fb4800ceaa0 msgr2=0x7fb48006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:52.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 -- 172.21.15.81:0/316518626 shutdown_connections 2023-11-08T16:29:52.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.190+0000 7fb4857e7640 1 -- 172.21.15.81:0/316518626 wait complete. 2023-11-08T16:29:52.211 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:29:52.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:52.212 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:29:52.310 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:29:52.311 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:29:52.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:52.312 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:29:52.370 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:29:52.371 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:29:52.432 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:29:52.433 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:29:52.487 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:52.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.708+0000 7f3e8e94c640 1 Processor -- start 2023-11-08T16:29:52.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.708+0000 7f3e8e94c640 1 -- start start 2023-11-08T16:29:52.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.709+0000 7f3e8e94c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3e880cc5e0 0x7f3e880cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:52.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.709+0000 7f3e8e94c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e880ccfb0 0x7f3e88070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:52.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.709+0000 7f3e8e94c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e880c9a70 con 0x7f3e880adcd0 2023-11-08T16:29:52.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.709+0000 7f3e8e94c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e880cc3e0 con 0x7f3e880cc5e0 2023-11-08T16:29:52.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.709+0000 7f3e8e94c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8806d070 con 0x7f3e880ccfb0 2023-11-08T16:29:52.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.709+0000 7f3e87fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3e880adcd0 0x7f3e880ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52696/0 (socket says 172.21.15.81:52696) 2023-11-08T16:29:52.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.709+0000 7f3e87fff640 1 -- 172.21.15.81:0/3488020304 learned_addr learned my addr 172.21.15.81:0/3488020304 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:52.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e8cee9640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e880ccfb0 0x7f3e88070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:52.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e7ffff640 1 --2- 172.21.15.81:0/3488020304 >> v2:172.21.15.93:3300/0 conn(0x7f3e880cc5e0 0x7f3e880cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:52.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841511938 0 0) 0x7f3e880c9a70 con 0x7f3e880adcd0 2023-11-08T16:29:52.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e74003520 con 0x7f3e880adcd0 2023-11-08T16:29:52.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14305524 0 0) 0x7f3e74003520 con 0x7f3e880adcd0 2023-11-08T16:29:52.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e880ccfb0 msgr2=0x7f3e88070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:52.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e880ccfb0 0x7f3e88070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 >> v2:172.21.15.93:3300/0 conn(0x7f3e880cc5e0 msgr2=0x7f3e880cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:52.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 --2- 172.21.15.81:0/3488020304 >> v2:172.21.15.93:3300/0 conn(0x7f3e880cc5e0 0x7f3e880cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8806d350 con 0x7f3e880adcd0 2023-11-08T16:29:52.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f3e78001670 con 0x7f3e880adcd0 2023-11-08T16:29:52.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e8cee9640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e880ccfb0 0x7f3e88070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:52.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 >> v1:172.21.15.81:6789/0 conn(0x7f3e880adcd0 legacy=0x7f3e880ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:52.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e74003a50 0x7f3e74003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:52.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e740047c0 0x7f3e74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:52.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e880ccfb0 0x7f3e740051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:52.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880cc3e0 con 0x7f3e740047c0 2023-11-08T16:29:52.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8806d070 con 0x7f3e74003a50 2023-11-08T16:29:52.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7400bb40 con 0x7f3e880ccfb0 2023-11-08T16:29:52.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.710+0000 7f3e7ffff640 1 --2- 172.21.15.81:0/3488020304 >> v2:172.21.15.93:3300/0 conn(0x7f3e880cc5e0 0x7f3e880cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:52.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.711+0000 7f3e7ffff640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e740047c0 0x7f3e74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:52.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.711+0000 7f3e87fff640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e74003a50 0x7f3e74003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:52.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.711+0000 7f3e8cee9640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e880ccfb0 0x7f3e740051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:52.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.711+0000 7f3e7ffff640 1 -- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e880ccfb0 msgr2=0x7f3e740051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:52.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.711+0000 7f3e7ffff640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e880ccfb0 0x7f3e740051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.711+0000 7f3e7ffff640 1 -- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e74003a50 msgr2=0x7f3e74003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:52.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.711+0000 7f3e7ffff640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e74003a50 0x7f3e74003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.711+0000 7f3e7ffff640 1 -- 172.21.15.81:0/3488020304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e74003520 con 0x7f3e740047c0 2023-11-08T16:29:52.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.712+0000 7f3e7ffff640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e740047c0 0x7f3e74004c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3e700098a0 tx=0x7f3e740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42c2ab90ea3ba11 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:52.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.712+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70037600 con 0x7f3e740047c0 2023-11-08T16:29:52.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.713+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e70037760 con 0x7f3e740047c0 2023-11-08T16:29:52.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.713+0000 7f3e877fe640 1 -- 172.21.15.81:0/3488020304 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e70037ad0 con 0x7f3e740047c0 2023-11-08T16:29:52.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.713+0000 7f3e8e94c640 1 -- 172.21.15.81:0/3488020304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e740047c0 msgr2=0x7f3e74004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:52.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.713+0000 7f3e8e94c640 1 --2- 172.21.15.81:0/3488020304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e740047c0 0x7f3e74004c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3e700098a0 tx=0x7f3e740057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.713+0000 7f3e8cee9640 1 -- 172.21.15.81:0/3488020304 reap_dead start 2023-11-08T16:29:52.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.713+0000 7f3e8e94c640 1 -- 172.21.15.81:0/3488020304 shutdown_connections 2023-11-08T16:29:52.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.713+0000 7f3e8e94c640 1 -- 172.21.15.81:0/3488020304 >> 172.21.15.81:0/3488020304 conn(0x7f3e880cb490 msgr2=0x7f3e880cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:52.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.713+0000 7f3e8e94c640 1 -- 172.21.15.81:0/3488020304 shutdown_connections 2023-11-08T16:29:52.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.713+0000 7f3e8e94c640 1 -- 172.21.15.81:0/3488020304 wait complete. 2023-11-08T16:29:52.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.715+0000 7f3e8e94c640 1 Processor -- start 2023-11-08T16:29:52.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.715+0000 7f3e8e94c640 1 -- start start 2023-11-08T16:29:52.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.716+0000 7f3e8e94c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e740047c0 0x7f3e881592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:52.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.716+0000 7f3e8e94c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e8815b030 0x7f3e880b3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:52.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.716+0000 7f3e8e94c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e880b3590 0x7f3e88070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:52.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.716+0000 7f3e8e94c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8815a590 con 0x7f3e880b3590 2023-11-08T16:29:52.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.716+0000 7f3e8e94c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e88159800 con 0x7f3e8815b030 2023-11-08T16:29:52.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.716+0000 7f3e8e94c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e880b3aa0 con 0x7f3e740047c0 2023-11-08T16:29:52.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.716+0000 7f3e8cee9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e880b3590 0x7f3e88070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:52.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e8cee9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e880b3590 0x7f3e88070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46878/0 (socket says 172.21.15.81:46878) 2023-11-08T16:29:52.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e8cee9640 1 -- 172.21.15.81:0/1201461013 learned_addr learned my addr 172.21.15.81:0/1201461013 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:52.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e87fff640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e740047c0 0x7f3e881592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:52.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e7ffff640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e8815b030 0x7f3e880b3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:52.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e8cee9640 1 -- 172.21.15.81:0/1201461013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e740047c0 msgr2=0x7f3e881592c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:52.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e8cee9640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e740047c0 0x7f3e881592c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e8cee9640 1 -- 172.21.15.81:0/1201461013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e8815b030 msgr2=0x7f3e880b3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:52.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e8cee9640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e8815b030 0x7f3e880b3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e8cee9640 1 -- 172.21.15.81:0/1201461013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8815a830 con 0x7f3e880b3590 2023-11-08T16:29:52.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e87fff640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e740047c0 0x7f3e881592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:52.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e7ffff640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e8815b030 0x7f3e880b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:52.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e8cee9640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e880b3590 0x7f3e88070010 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3e88071a70 tx=0x7f3e80004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:52.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.717+0000 7f3e857fa640 1 -- 172.21.15.81:0/1201461013 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80007c70 con 0x7f3e880b3590 2023-11-08T16:29:52.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.718+0000 7f3e857fa640 1 -- 172.21.15.81:0/1201461013 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8000f050 con 0x7f3e880b3590 2023-11-08T16:29:52.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.718+0000 7f3e8e94c640 1 -- 172.21.15.81:0/1201461013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8815ab10 con 0x7f3e880b3590 2023-11-08T16:29:52.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.718+0000 7f3e857fa640 1 -- 172.21.15.81:0/1201461013 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80014410 con 0x7f3e880b3590 2023-11-08T16:29:52.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.718+0000 7f3e8e94c640 1 -- 172.21.15.81:0/1201461013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8815a130 con 0x7f3e880b3590 2023-11-08T16:29:52.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.719+0000 7f3e8e94c640 1 -- 172.21.15.81:0/1201461013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e88072170 con 0x7f3e880b3590 2023-11-08T16:29:52.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.720+0000 7f3e857fa640 1 -- 172.21.15.81:0/1201461013 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3e80014590 con 0x7f3e880b3590 2023-11-08T16:29:52.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.721+0000 7f3e857fa640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e6806ae70 0x7f3e6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:52.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.721+0000 7f3e857fa640 1 -- 172.21.15.81:0/1201461013 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3e80089b40 con 0x7f3e880b3590 2023-11-08T16:29:52.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.721+0000 7f3e87fff640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e6806ae70 0x7f3e6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:52.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.722+0000 7f3e87fff640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e6806ae70 0x7f3e6806d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e780058b0 tx=0x7f3e78000f70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:52.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.726+0000 7f3e857fa640 1 -- 172.21.15.81:0/1201461013 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3e80054ac0 con 0x7f3e880b3590 2023-11-08T16:29:52.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.919+0000 7f3e8e94c640 1 -- 172.21.15.81:0/1201461013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e8815a320 con 0x7f3e880b3590 2023-11-08T16:29:52.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.922+0000 7f3e857fa640 1 -- 172.21.15.81:0/1201461013 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f3e80081e60 con 0x7f3e880b3590 2023-11-08T16:29:52.925 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:52.925 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:52.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 -- 172.21.15.81:0/1201461013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e6806ae70 msgr2=0x7f3e6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:52.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e6806ae70 0x7f3e6806d330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3e780058b0 tx=0x7f3e78000f70 comp rx=0 tx=0).stop 2023-11-08T16:29:52.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 -- 172.21.15.81:0/1201461013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e880b3590 msgr2=0x7f3e88070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:52.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e880b3590 0x7f3e88070010 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f3e88071a70 tx=0x7f3e80004700 comp rx=0 tx=0).stop 2023-11-08T16:29:52.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 -- 172.21.15.81:0/1201461013 shutdown_connections 2023-11-08T16:29:52.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3e6806ae70 0x7f3e6806d330 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3e880b3590 0x7f3e88070010 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3e8815b030 0x7f3e880b3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 --2- 172.21.15.81:0/1201461013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3e740047c0 0x7f3e881592c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:52.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 -- 172.21.15.81:0/1201461013 >> 172.21.15.81:0/1201461013 conn(0x7f3e880cb490 msgr2=0x7f3e8815cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:52.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 -- 172.21.15.81:0/1201461013 shutdown_connections 2023-11-08T16:29:52.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:52.925+0000 7f3e8e94c640 1 -- 172.21.15.81:0/1201461013 wait complete. 2023-11-08T16:29:52.946 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:52.947 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:53.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.156+0000 7f6c3ddb6640 1 Processor -- start 2023-11-08T16:29:53.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.157+0000 7f6c3ddb6640 1 -- start start 2023-11-08T16:29:53.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.158+0000 7f6c3ddb6640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6c380ce0d0 0x7f6c380caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.158+0000 7f6c3ddb6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3815bfc0 0x7f6c3815eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.158+0000 7f6c3ddb6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6c3815f0b0 0x7f6c381516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.158+0000 7f6c3ddb6640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3815f540 con 0x7f6c3815f0b0 2023-11-08T16:29:53.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.158+0000 7f6c3ddb6640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c3806fa30 con 0x7f6c380ce0d0 2023-11-08T16:29:53.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.158+0000 7f6c3ddb6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380ce4b0 con 0x7f6c3815bfc0 2023-11-08T16:29:53.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.159+0000 7f6c37fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6c3815f0b0 0x7f6c381516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.159+0000 7f6c37fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6c3815f0b0 0x7f6c381516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46880/0 (socket says 172.21.15.81:46880) 2023-11-08T16:29:53.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.159+0000 7f6c37fff640 1 -- 172.21.15.81:0/3758590123 learned_addr learned my addr 172.21.15.81:0/3758590123 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:53.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.159+0000 7f6c377fe640 1 --2- 172.21.15.81:0/3758590123 >> v2:172.21.15.93:3300/0 conn(0x7f6c380ce0d0 0x7f6c380caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.159+0000 7f6c36ffd640 1 --2- 172.21.15.81:0/3758590123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3815bfc0 0x7f6c3815eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.159+0000 7f6c37fff640 1 -- 172.21.15.81:0/3758590123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3815bfc0 msgr2=0x7f6c3815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:53.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.159+0000 7f6c37fff640 1 --2- 172.21.15.81:0/3758590123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3815bfc0 0x7f6c3815eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.159+0000 7f6c37fff640 1 -- 172.21.15.81:0/3758590123 >> v2:172.21.15.93:3300/0 conn(0x7f6c380ce0d0 msgr2=0x7f6c380caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:53.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.159+0000 7f6c37fff640 1 --2- 172.21.15.81:0/3758590123 >> v2:172.21.15.93:3300/0 conn(0x7f6c380ce0d0 0x7f6c380caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.159+0000 7f6c37fff640 1 -- 172.21.15.81:0/3758590123 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3806fca0 con 0x7f6c3815f0b0 2023-11-08T16:29:53.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.159+0000 7f6c377fe640 1 --2- 172.21.15.81:0/3758590123 >> v2:172.21.15.93:3300/0 conn(0x7f6c380ce0d0 0x7f6c380caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:29:53.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.160+0000 7f6c37fff640 1 --2- 172.21.15.81:0/3758590123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c3815f0b0 0x7f6c381516e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6c3806f3a0 tx=0x7f6c3002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62086d76a674f48e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:53.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.161+0000 7f6c367fc640 1 -- 172.21.15.81:0/3758590123 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3002ca80 con 0x7f6c3815f0b0 2023-11-08T16:29:53.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.161+0000 7f6c367fc640 1 -- 172.21.15.81:0/3758590123 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3002cbe0 con 0x7f6c3815f0b0 2023-11-08T16:29:53.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.161+0000 7f6c367fc640 1 -- 172.21.15.81:0/3758590123 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c300343f0 con 0x7f6c3815f0b0 2023-11-08T16:29:53.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.161+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/3758590123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c3815f0b0 msgr2=0x7f6c381516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.161+0000 7f6c3ddb6640 1 --2- 172.21.15.81:0/3758590123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c3815f0b0 0x7f6c381516e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f6c3806f3a0 tx=0x7f6c3002bee0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.162+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/3758590123 shutdown_connections 2023-11-08T16:29:53.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.162+0000 7f6c3ddb6640 1 --2- 172.21.15.81:0/3758590123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3815bfc0 0x7f6c3815eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.162+0000 7f6c3ddb6640 1 --2- 172.21.15.81:0/3758590123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c3815f0b0 0x7f6c381516e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.162+0000 7f6c3ddb6640 1 --2- 172.21.15.81:0/3758590123 >> v2:172.21.15.93:3300/0 conn(0x7f6c380ce0d0 0x7f6c380caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.162+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/3758590123 >> 172.21.15.81:0/3758590123 conn(0x7f6c380cb580 msgr2=0x7f6c380cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:53.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.162+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/3758590123 shutdown_connections 2023-11-08T16:29:53.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.162+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/3758590123 wait complete. 2023-11-08T16:29:53.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.163+0000 7f6c3ddb6640 1 Processor -- start 2023-11-08T16:29:53.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.163+0000 7f6c3ddb6640 1 -- start start 2023-11-08T16:29:53.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.164+0000 7f6c3ddb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c380ce0d0 0x7f6c381f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.164+0000 7f6c3ddb6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3815bfc0 0x7f6c3806be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.164+0000 7f6c3ddb6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c3815f0b0 0x7f6c380c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.164+0000 7f6c3ddb6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3806c830 con 0x7f6c380ce0d0 2023-11-08T16:29:53.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.164+0000 7f6c3ddb6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3806cb10 con 0x7f6c3815f0b0 2023-11-08T16:29:53.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.164+0000 7f6c3ddb6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3806cdf0 con 0x7f6c3815bfc0 2023-11-08T16:29:53.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.165+0000 7f6c377fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c380ce0d0 0x7f6c381f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.165+0000 7f6c377fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c380ce0d0 0x7f6c381f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46888/0 (socket says 172.21.15.81:46888) 2023-11-08T16:29:53.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.165+0000 7f6c377fe640 1 -- 172.21.15.81:0/286125175 learned_addr learned my addr 172.21.15.81:0/286125175 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:53.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.165+0000 7f6c377fe640 1 -- 172.21.15.81:0/286125175 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3815bfc0 msgr2=0x7f6c3806be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.165+0000 7f6c377fe640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3815bfc0 0x7f6c3806be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.165+0000 7f6c377fe640 1 -- 172.21.15.81:0/286125175 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c3815f0b0 msgr2=0x7f6c380c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.165+0000 7f6c377fe640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c3815f0b0 0x7f6c380c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.165+0000 7f6c377fe640 1 -- 172.21.15.81:0/286125175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30009b30 con 0x7f6c380ce0d0 2023-11-08T16:29:53.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.165+0000 7f6c377fe640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c380ce0d0 0x7f6c381f20b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000d800 tx=0x7f6c2000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:53.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.166+0000 7f6c2ffff640 1 -- 172.21.15.81:0/286125175 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20004030 con 0x7f6c380ce0d0 2023-11-08T16:29:53.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.166+0000 7f6c2ffff640 1 -- 172.21.15.81:0/286125175 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c20004190 con 0x7f6c380ce0d0 2023-11-08T16:29:53.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.166+0000 7f6c2ffff640 1 -- 172.21.15.81:0/286125175 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20004520 con 0x7f6c380ce0d0 2023-11-08T16:29:53.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.166+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/286125175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c3806d0d0 con 0x7f6c380ce0d0 2023-11-08T16:29:53.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.166+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/286125175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c381f25f0 con 0x7f6c380ce0d0 2023-11-08T16:29:53.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.167+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/286125175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3804ece0 con 0x7f6c380ce0d0 2023-11-08T16:29:53.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.168+0000 7f6c2ffff640 1 -- 172.21.15.81:0/286125175 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c200096a0 con 0x7f6c380ce0d0 2023-11-08T16:29:53.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.169+0000 7f6c2ffff640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1806ae70 0x7f6c1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.169+0000 7f6c2ffff640 1 -- 172.21.15.81:0/286125175 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6c200899c0 con 0x7f6c380ce0d0 2023-11-08T16:29:53.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.169+0000 7f6c36ffd640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1806ae70 0x7f6c1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.170+0000 7f6c36ffd640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1806ae70 0x7f6c1806d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c380b4970 tx=0x7f6c28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:53.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.173+0000 7f6c2ffff640 1 -- 172.21.15.81:0/286125175 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c20054940 con 0x7f6c380ce0d0 2023-11-08T16:29:53.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.360+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/286125175 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c38071640 con 0x7f6c380ce0d0 2023-11-08T16:29:53.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.361+0000 7f6c2ffff640 1 -- 172.21.15.81:0/286125175 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c200180a0 con 0x7f6c380ce0d0 2023-11-08T16:29:53.364 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:53.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/286125175 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1806ae70 msgr2=0x7f6c1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1806ae70 0x7f6c1806d330 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6c380b4970 tx=0x7f6c28008040 comp rx=0 tx=0).stop 2023-11-08T16:29:53.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/286125175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c380ce0d0 msgr2=0x7f6c381f20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c380ce0d0 0x7f6c381f20b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6c2000d800 tx=0x7f6c2000dcc0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/286125175 shutdown_connections 2023-11-08T16:29:53.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1806ae70 0x7f6c1806d330 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c3815f0b0 0x7f6c380c4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3815bfc0 0x7f6c3806be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 --2- 172.21.15.81:0/286125175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c380ce0d0 0x7f6c381f20b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/286125175 >> 172.21.15.81:0/286125175 conn(0x7f6c380cb580 msgr2=0x7f6c3815aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:53.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/286125175 shutdown_connections 2023-11-08T16:29:53.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.364+0000 7f6c3ddb6640 1 -- 172.21.15.81:0/286125175 wait complete. 2023-11-08T16:29:53.386 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:53.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.599+0000 7f35060ca640 1 Processor -- start 2023-11-08T16:29:53.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.600+0000 7f35060ca640 1 -- start start 2023-11-08T16:29:53.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.601+0000 7f35060ca640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f350015cf30 0x7f35000d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.601+0000 7f35060ca640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35000ce0b0 con 0x7f350015d310 2023-11-08T16:29:53.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.601+0000 7f35060ca640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35000ce390 con 0x7f35000cfd90 2023-11-08T16:29:53.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.601+0000 7f35060ca640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35000d1be0 con 0x7f350015cf30 2023-11-08T16:29:53.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.601+0000 7f35048c7640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f350015d310 0x7f35000cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52702/0 (socket says 172.21.15.81:52702) 2023-11-08T16:29:53.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.601+0000 7f35048c7640 1 -- 172.21.15.81:0/146388885 learned_addr learned my addr 172.21.15.81:0/146388885 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:53.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f35050c8640 1 --2- 172.21.15.81:0/146388885 >> v2:172.21.15.93:3300/0 conn(0x7f350015cf30 0x7f35000d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350042835 0 0) 0x7f35000ce0b0 con 0x7f350015d310 2023-11-08T16:29:53.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34f0003520 con 0x7f350015d310 2023-11-08T16:29:53.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3856439776 0 0) 0x7f34f0003520 con 0x7f350015d310 2023-11-08T16:29:53.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 >> v2:172.21.15.93:3300/0 conn(0x7f350015cf30 msgr2=0x7f35000d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:53.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 --2- 172.21.15.81:0/146388885 >> v2:172.21.15.93:3300/0 conn(0x7f350015cf30 0x7f35000d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 >> v1:172.21.15.93:6789/0 conn(0x7f35000cfd90 legacy=0x7f35000b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35000d1ec0 con 0x7f350015d310 2023-11-08T16:29:53.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f34ec001670 con 0x7f350015d310 2023-11-08T16:29:53.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 >> v1:172.21.15.81:6789/0 conn(0x7f350015d310 legacy=0x7f35000cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 --2- 172.21.15.81:0/146388885 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34f00039c0 0x7f34f0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 --2- 172.21.15.81:0/146388885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34f00047c0 0x7f34f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.602+0000 7f34fffff640 1 --2- 172.21.15.81:0/146388885 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34f0005180 0x7f34f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.603+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35000d1be0 con 0x7f34f00047c0 2023-11-08T16:29:53.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.603+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f000c080 con 0x7f34f0005180 2023-11-08T16:29:53.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.603+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f000c240 con 0x7f34f00039c0 2023-11-08T16:29:53.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.603+0000 7f35048c7640 1 --2- 172.21.15.81:0/146388885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34f00047c0 0x7f34f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.603+0000 7f35048c7640 1 -- 172.21.15.81:0/146388885 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34f00039c0 msgr2=0x7f34f0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:53.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.603+0000 7f35048c7640 1 --2- 172.21.15.81:0/146388885 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34f00039c0 0x7f34f0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.603+0000 7f35048c7640 1 -- 172.21.15.81:0/146388885 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34f0005180 msgr2=0x7f34f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:53.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.603+0000 7f35058c9640 1 --2- 172.21.15.81:0/146388885 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34f0005180 0x7f34f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.603+0000 7f35048c7640 1 --2- 172.21.15.81:0/146388885 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34f0005180 0x7f34f000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.603+0000 7f35048c7640 1 -- 172.21.15.81:0/146388885 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f0003520 con 0x7f34f00047c0 2023-11-08T16:29:53.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.603+0000 7f35058c9640 1 -- 172.21.15.81:0/146388885 reap_dead start 2023-11-08T16:29:53.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.604+0000 7f35048c7640 1 --2- 172.21.15.81:0/146388885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34f00047c0 0x7f34f0004c40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f35000b2100 tx=0x7f34f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e0dde3ac8bace89 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:53.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.604+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec005ad0 con 0x7f34f00047c0 2023-11-08T16:29:53.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.604+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec005c30 con 0x7f34f00047c0 2023-11-08T16:29:53.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.604+0000 7f34fffff640 1 -- 172.21.15.81:0/146388885 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec02c9d0 con 0x7f34f00047c0 2023-11-08T16:29:53.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.605+0000 7f35060ca640 1 -- 172.21.15.81:0/146388885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34f00047c0 msgr2=0x7f34f0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.605+0000 7f35060ca640 1 --2- 172.21.15.81:0/146388885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34f00047c0 0x7f34f0004c40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f35000b2100 tx=0x7f34f00057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.605+0000 7f35060ca640 1 -- 172.21.15.81:0/146388885 shutdown_connections 2023-11-08T16:29:53.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.605+0000 7f35060ca640 1 --2- 172.21.15.81:0/146388885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34f00047c0 0x7f34f0004c40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.605+0000 7f35060ca640 1 -- 172.21.15.81:0/146388885 >> 172.21.15.81:0/146388885 conn(0x7f350006c040 msgr2=0x7f350006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:53.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.605+0000 7f35060ca640 1 -- 172.21.15.81:0/146388885 shutdown_connections 2023-11-08T16:29:53.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.605+0000 7f35060ca640 1 -- 172.21.15.81:0/146388885 wait complete. 2023-11-08T16:29:53.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.607+0000 7f35060ca640 1 Processor -- start 2023-11-08T16:29:53.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.607+0000 7f35060ca640 1 -- start start 2023-11-08T16:29:53.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.608+0000 7f35060ca640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34f00047c0 0x7f35000b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.608+0000 7f35060ca640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35000b2e20 0x7f35000c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.608+0000 7f35060ca640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f35001f5fd0 0x7f35000cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.608+0000 7f35060ca640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35000b3620 con 0x7f35000b2e20 2023-11-08T16:29:53.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.608+0000 7f35060ca640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35000cadd0 con 0x7f34f00047c0 2023-11-08T16:29:53.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.608+0000 7f35060ca640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35000cafe0 con 0x7f35001f5fd0 2023-11-08T16:29:53.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.608+0000 7f35048c7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35000b2e20 0x7f35000c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.608+0000 7f35048c7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35000b2e20 0x7f35000c7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46906/0 (socket says 172.21.15.81:46906) 2023-11-08T16:29:53.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.608+0000 7f35048c7640 1 -- 172.21.15.81:0/2324790077 learned_addr learned my addr 172.21.15.81:0/2324790077 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:53.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f35058c9640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f35001f5fd0 0x7f35000cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f35048c7640 1 -- 172.21.15.81:0/2324790077 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f35001f5fd0 msgr2=0x7f35000cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f35050c8640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34f00047c0 0x7f35000b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f35048c7640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f35001f5fd0 0x7f35000cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f35048c7640 1 -- 172.21.15.81:0/2324790077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34f00047c0 msgr2=0x7f35000b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f35048c7640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34f00047c0 0x7f35000b2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f35048c7640 1 -- 172.21.15.81:0/2324790077 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35000cb2c0 con 0x7f35000b2e20 2023-11-08T16:29:53.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f35048c7640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35000b2e20 0x7f35000c7430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f34ec02c330 tx=0x7f34ec02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:53.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f34fdffb640 1 -- 172.21.15.81:0/2324790077 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec03c040 con 0x7f35000b2e20 2023-11-08T16:29:53.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f34fdffb640 1 -- 172.21.15.81:0/2324790077 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ec02cc80 con 0x7f35000b2e20 2023-11-08T16:29:53.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f34fdffb640 1 -- 172.21.15.81:0/2324790077 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ec007070 con 0x7f35000b2e20 2023-11-08T16:29:53.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.609+0000 7f35060ca640 1 -- 172.21.15.81:0/2324790077 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35000cb5a0 con 0x7f35000b2e20 2023-11-08T16:29:53.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.610+0000 7f35060ca640 1 -- 172.21.15.81:0/2324790077 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35000b3280 con 0x7f35000b2e20 2023-11-08T16:29:53.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.611+0000 7f35060ca640 1 -- 172.21.15.81:0/2324790077 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350006cd70 con 0x7f35000b2e20 2023-11-08T16:29:53.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.612+0000 7f34fdffb640 1 -- 172.21.15.81:0/2324790077 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f34ec032070 con 0x7f35000b2e20 2023-11-08T16:29:53.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.613+0000 7f34fdffb640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f34e406ae70 0x7f34e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:53.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.613+0000 7f34fdffb640 1 -- 172.21.15.81:0/2324790077 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f34ec0aaba0 con 0x7f35000b2e20 2023-11-08T16:29:53.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.613+0000 7f35050c8640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f34e406ae70 0x7f34e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:53.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.614+0000 7f35050c8640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f34e406ae70 0x7f34e406d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34f40073f0 tx=0x7f34f400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:53.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.617+0000 7f34fdffb640 1 -- 172.21.15.81:0/2324790077 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f34ec075b20 con 0x7f35000b2e20 2023-11-08T16:29:53.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.805+0000 7f35060ca640 1 -- 172.21.15.81:0/2324790077 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f350004ece0 con 0x7f35000b2e20 2023-11-08T16:29:53.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.806+0000 7f34fdffb640 1 -- 172.21.15.81:0/2324790077 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f34ec079140 con 0x7f35000b2e20 2023-11-08T16:29:53.809 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:53.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 -- 172.21.15.81:0/2324790077 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f34e406ae70 msgr2=0x7f34e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f34e406ae70 0x7f34e406d330 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f34f40073f0 tx=0x7f34f400b040 comp rx=0 tx=0).stop 2023-11-08T16:29:53.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 -- 172.21.15.81:0/2324790077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35000b2e20 msgr2=0x7f35000c7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:53.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35000b2e20 0x7f35000c7430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f34ec02c330 tx=0x7f34ec02e070 comp rx=0 tx=0).stop 2023-11-08T16:29:53.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 -- 172.21.15.81:0/2324790077 shutdown_connections 2023-11-08T16:29:53.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f34e406ae70 0x7f34e406d330 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f35001f5fd0 0x7f35000cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f35000b2e20 0x7f35000c7430 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 --2- 172.21.15.81:0/2324790077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34f00047c0 0x7f35000b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:53.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 -- 172.21.15.81:0/2324790077 >> 172.21.15.81:0/2324790077 conn(0x7f350006c040 msgr2=0x7f350006c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:53.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 -- 172.21.15.81:0/2324790077 shutdown_connections 2023-11-08T16:29:53.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:53.809+0000 7f35060ca640 1 -- 172.21.15.81:0/2324790077 wait complete. 2023-11-08T16:29:53.830 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:54.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.038+0000 7f811f4a9640 1 Processor -- start 2023-11-08T16:29:54.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.039+0000 7f811f4a9640 1 -- start start 2023-11-08T16:29:54.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.040+0000 7f811f4a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f811815d4c0 0x7f81180b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.040+0000 7f811f4a9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f811806c830 0x7f811806cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.040+0000 7f811f4a9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f811806d1c0 con 0x7f81180b0010 2023-11-08T16:29:54.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.040+0000 7f811f4a9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81180cefa0 con 0x7f811806c830 2023-11-08T16:29:54.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.040+0000 7f811f4a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81180d0890 con 0x7f811815d4c0 2023-11-08T16:29:54.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.041+0000 7f811da46640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f811806c830 0x7f811806cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:54.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.041+0000 7f811d245640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f811815d4c0 0x7f81180b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:54.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.041+0000 7f811da46640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f811806c830 0x7f811806cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:57686/0 (socket says 172.21.15.81:57686) 2023-11-08T16:29:54.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.041+0000 7f811da46640 1 -- 172.21.15.81:0/2006401613 learned_addr learned my addr 172.21.15.81:0/2006401613 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:54.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.042+0000 7f811da46640 1 -- 172.21.15.81:0/2006401613 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f811815d4c0 msgr2=0x7f81180b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:54.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.042+0000 7f811da46640 1 --2- 172.21.15.81:0/2006401613 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f811815d4c0 0x7f81180b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.042+0000 7f811da46640 1 -- 172.21.15.81:0/2006401613 >> v1:172.21.15.93:6789/0 conn(0x7f81180b0010 legacy=0x7f81180cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.042+0000 7f811da46640 1 -- 172.21.15.81:0/2006401613 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81180ce9e0 con 0x7f811806c830 2023-11-08T16:29:54.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.043+0000 7f811da46640 1 --2- 172.21.15.81:0/2006401613 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f811806c830 0x7f811806cc80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8110009e50 tx=0x7f811002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=baa2c801bf69a336 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:54.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.044+0000 7f810bfff640 1 -- 172.21.15.81:0/2006401613 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811002cb00 con 0x7f811806c830 2023-11-08T16:29:54.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.044+0000 7f810bfff640 1 -- 172.21.15.81:0/2006401613 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811002cc60 con 0x7f811806c830 2023-11-08T16:29:54.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.044+0000 7f810bfff640 1 -- 172.21.15.81:0/2006401613 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81100343f0 con 0x7f811806c830 2023-11-08T16:29:54.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.044+0000 7f811f4a9640 1 -- 172.21.15.81:0/2006401613 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f811806c830 msgr2=0x7f811806cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.044+0000 7f811f4a9640 1 --2- 172.21.15.81:0/2006401613 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f811806c830 0x7f811806cc80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8110009e50 tx=0x7f811002bf60 comp rx=0 tx=0).stop 2023-11-08T16:29:54.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.044+0000 7f811f4a9640 1 -- 172.21.15.81:0/2006401613 shutdown_connections 2023-11-08T16:29:54.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.044+0000 7f811f4a9640 1 --2- 172.21.15.81:0/2006401613 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f811806c830 0x7f811806cc80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.044+0000 7f811f4a9640 1 --2- 172.21.15.81:0/2006401613 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f811815d4c0 0x7f81180b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.044+0000 7f811f4a9640 1 -- 172.21.15.81:0/2006401613 >> 172.21.15.81:0/2006401613 conn(0x7f811806dbe0 msgr2=0x7f811806dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:54.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.045+0000 7f811f4a9640 1 -- 172.21.15.81:0/2006401613 shutdown_connections 2023-11-08T16:29:54.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.045+0000 7f811f4a9640 1 -- 172.21.15.81:0/2006401613 wait complete. 2023-11-08T16:29:54.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.046+0000 7f811f4a9640 1 Processor -- start 2023-11-08T16:29:54.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.046+0000 7f811f4a9640 1 -- start start 2023-11-08T16:29:54.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.047+0000 7f811f4a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f811806c830 0x7f811814d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.047+0000 7f811f4a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81180b0010 0x7f811815fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.047+0000 7f811f4a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f811815d4c0 0x7f8118160340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.047+0000 7f811f4a9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811814db00 con 0x7f81180b0010 2023-11-08T16:29:54.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.047+0000 7f811f4a9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81180cab50 con 0x7f811815d4c0 2023-11-08T16:29:54.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.047+0000 7f811f4a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811814dd50 con 0x7f811806c830 2023-11-08T16:29:54.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.048+0000 7f811ca44640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81180b0010 0x7f811815fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:54.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.048+0000 7f811ca44640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81180b0010 0x7f811815fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46912/0 (socket says 172.21.15.81:46912) 2023-11-08T16:29:54.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.048+0000 7f811ca44640 1 -- 172.21.15.81:0/3623763015 learned_addr learned my addr 172.21.15.81:0/3623763015 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:54.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.048+0000 7f811d245640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f811806c830 0x7f811814d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:54.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.048+0000 7f811da46640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f811815d4c0 0x7f8118160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:54.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.048+0000 7f811ca44640 1 -- 172.21.15.81:0/3623763015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f811806c830 msgr2=0x7f811814d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.048+0000 7f811ca44640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f811806c830 0x7f811814d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.048+0000 7f811ca44640 1 -- 172.21.15.81:0/3623763015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f811815d4c0 msgr2=0x7f8118160340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.048+0000 7f811ca44640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f811815d4c0 0x7f8118160340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.048+0000 7f811ca44640 1 -- 172.21.15.81:0/3623763015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8110009b30 con 0x7f81180b0010 2023-11-08T16:29:54.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.049+0000 7f811ca44640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81180b0010 0x7f811815fe00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f810c00c400 tx=0x7f810c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:54.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.049+0000 7f8109ffb640 1 -- 172.21.15.81:0/3623763015 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c0052d0 con 0x7f81180b0010 2023-11-08T16:29:54.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.049+0000 7f8109ffb640 1 -- 172.21.15.81:0/3623763015 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c005430 con 0x7f81180b0010 2023-11-08T16:29:54.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.049+0000 7f8109ffb640 1 -- 172.21.15.81:0/3623763015 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c003830 con 0x7f81180b0010 2023-11-08T16:29:54.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.049+0000 7f811f4a9640 1 -- 172.21.15.81:0/3623763015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811814e030 con 0x7f81180b0010 2023-11-08T16:29:54.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.049+0000 7f811f4a9640 1 -- 172.21.15.81:0/3623763015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f811814d770 con 0x7f81180b0010 2023-11-08T16:29:54.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.050+0000 7f811f4a9640 1 -- 172.21.15.81:0/3623763015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811814e550 con 0x7f81180b0010 2023-11-08T16:29:54.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.051+0000 7f8109ffb640 1 -- 172.21.15.81:0/3623763015 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f810c003990 con 0x7f81180b0010 2023-11-08T16:29:54.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.052+0000 7f8109ffb640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f80fc06ae70 0x7f80fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.052+0000 7f8109ffb640 1 -- 172.21.15.81:0/3623763015 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f810c088a90 con 0x7f81180b0010 2023-11-08T16:29:54.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.052+0000 7f811d245640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f80fc06ae70 0x7f80fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:54.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.053+0000 7f811d245640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f80fc06ae70 0x7f80fc06d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8114009970 tx=0x7f8114008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:54.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.057+0000 7f8109ffb640 1 -- 172.21.15.81:0/3623763015 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f811814e550 con 0x7f81180b0010 2023-11-08T16:29:54.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.247+0000 7f811f4a9640 1 -- 172.21.15.81:0/3623763015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f811806d1c0 con 0x7f81180b0010 2023-11-08T16:29:54.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.248+0000 7f8109ffb640 1 -- 172.21.15.81:0/3623763015 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f811806d1c0 con 0x7f81180b0010 2023-11-08T16:29:54.251 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:54.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 -- 172.21.15.81:0/3623763015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f80fc06ae70 msgr2=0x7f80fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f80fc06ae70 0x7f80fc06d330 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8114009970 tx=0x7f8114008040 comp rx=0 tx=0).stop 2023-11-08T16:29:54.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 -- 172.21.15.81:0/3623763015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81180b0010 msgr2=0x7f811815fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81180b0010 0x7f811815fe00 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f810c00c400 tx=0x7f810c00c8c0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 -- 172.21.15.81:0/3623763015 shutdown_connections 2023-11-08T16:29:54.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f80fc06ae70 0x7f80fc06d330 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f811815d4c0 0x7f8118160340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81180b0010 0x7f811815fe00 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 --2- 172.21.15.81:0/3623763015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f811806c830 0x7f811814d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 -- 172.21.15.81:0/3623763015 >> 172.21.15.81:0/3623763015 conn(0x7f811806dbe0 msgr2=0x7f811806b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:54.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 -- 172.21.15.81:0/3623763015 shutdown_connections 2023-11-08T16:29:54.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.251+0000 7f811f4a9640 1 -- 172.21.15.81:0/3623763015 wait complete. 2023-11-08T16:29:54.274 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:29:54.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.487+0000 7febc06e0640 1 Processor -- start 2023-11-08T16:29:54.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.487+0000 7febc06e0640 1 -- start start 2023-11-08T16:29:54.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.488+0000 7febc06e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb80cf720 0x7febb806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.488+0000 7febc06e0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febb80ca130 0x7febb8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.488+0000 7febc06e0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb806cc50 con 0x7febb815d3b0 2023-11-08T16:29:54.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.488+0000 7febc06e0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb80c90f0 con 0x7febb80ca130 2023-11-08T16:29:54.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.488+0000 7febc06e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb80adb70 con 0x7febb80cf720 2023-11-08T16:29:54.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.489+0000 7febbe47c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7febb815d3b0 0x7febb80c9a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52706/0 (socket says 172.21.15.81:52706) 2023-11-08T16:29:54.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.489+0000 7febbe47c640 1 -- 172.21.15.81:0/394843503 learned_addr learned my addr 172.21.15.81:0/394843503 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:54.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.489+0000 7febbec7d640 1 --2- 172.21.15.81:0/394843503 >> v2:172.21.15.81:3300/0 conn(0x7febb80ca130 0x7febb8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:54.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.489+0000 7febbdc7b640 1 --2- 172.21.15.81:0/394843503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb80cf720 0x7febb806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:54.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.489+0000 7febbec7d640 1 -- 172.21.15.81:0/394843503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb80cf720 msgr2=0x7febb806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:54.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.489+0000 7febbec7d640 1 --2- 172.21.15.81:0/394843503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb80cf720 0x7febb806d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.489+0000 7febbec7d640 1 -- 172.21.15.81:0/394843503 >> v1:172.21.15.81:6789/0 conn(0x7febb815d3b0 legacy=0x7febb80c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.489+0000 7febbec7d640 1 -- 172.21.15.81:0/394843503 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb80c9360 con 0x7febb80ca130 2023-11-08T16:29:54.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.490+0000 7febbec7d640 1 --2- 172.21.15.81:0/394843503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb80ca130 0x7febb8070c60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7febb0009d90 tx=0x7febb002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64440fbbcfbc347 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:54.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.490+0000 7febbd47a640 1 -- 172.21.15.81:0/394843503 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb002cc60 con 0x7febb80ca130 2023-11-08T16:29:54.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.491+0000 7febbd47a640 1 -- 172.21.15.81:0/394843503 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb002cdc0 con 0x7febb80ca130 2023-11-08T16:29:54.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.491+0000 7febbd47a640 1 -- 172.21.15.81:0/394843503 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0034660 con 0x7febb80ca130 2023-11-08T16:29:54.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.491+0000 7febc06e0640 1 -- 172.21.15.81:0/394843503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb80ca130 msgr2=0x7febb8070c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.491+0000 7febc06e0640 1 --2- 172.21.15.81:0/394843503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb80ca130 0x7febb8070c60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7febb0009d90 tx=0x7febb002beb0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.491+0000 7febc06e0640 1 -- 172.21.15.81:0/394843503 shutdown_connections 2023-11-08T16:29:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.491+0000 7febc06e0640 1 --2- 172.21.15.81:0/394843503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb80ca130 0x7febb8070c60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.491+0000 7febc06e0640 1 --2- 172.21.15.81:0/394843503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb80cf720 0x7febb806d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.491+0000 7febc06e0640 1 -- 172.21.15.81:0/394843503 >> 172.21.15.81:0/394843503 conn(0x7febb80ceaa0 msgr2=0x7febb80cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:54.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.492+0000 7febc06e0640 1 -- 172.21.15.81:0/394843503 shutdown_connections 2023-11-08T16:29:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.492+0000 7febc06e0640 1 -- 172.21.15.81:0/394843503 wait complete. 2023-11-08T16:29:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.493+0000 7febc06e0640 1 Processor -- start 2023-11-08T16:29:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.493+0000 7febc06e0640 1 -- start start 2023-11-08T16:29:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.494+0000 7febc06e0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb80ca130 0x7febb8157050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.494+0000 7febc06e0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febb80cf720 0x7febb815c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.494+0000 7febc06e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb815d3b0 0x7febb80c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.494+0000 7febc06e0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb815c0e0 con 0x7febb80ca130 2023-11-08T16:29:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.494+0000 7febc06e0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb815ac50 con 0x7febb80cf720 2023-11-08T16:29:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.494+0000 7febc06e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb815b460 con 0x7febb815d3b0 2023-11-08T16:29:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.495+0000 7febbe47c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb80ca130 0x7febb8157050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.495+0000 7febbe47c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb80ca130 0x7febb8157050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46942/0 (socket says 172.21.15.81:46942) 2023-11-08T16:29:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.495+0000 7febbe47c640 1 -- 172.21.15.81:0/3222262009 learned_addr learned my addr 172.21.15.81:0/3222262009 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.495+0000 7febbe47c640 1 -- 172.21.15.81:0/3222262009 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb815d3b0 msgr2=0x7febb80c4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.495+0000 7febbe47c640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb815d3b0 0x7febb80c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.495+0000 7febbe47c640 1 -- 172.21.15.81:0/3222262009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febb80cf720 msgr2=0x7febb815c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.495+0000 7febbe47c640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febb80cf720 0x7febb815c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.495+0000 7febbe47c640 1 -- 172.21.15.81:0/3222262009 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb0009a70 con 0x7febb80ca130 2023-11-08T16:29:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.495+0000 7febbe47c640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb80ca130 0x7febb8157050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7febb400c460 tx=0x7febb400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.496+0000 7febaeffd640 1 -- 172.21.15.81:0/3222262009 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb4007480 con 0x7febb80ca130 2023-11-08T16:29:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.496+0000 7febaeffd640 1 -- 172.21.15.81:0/3222262009 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb40075e0 con 0x7febb80ca130 2023-11-08T16:29:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.496+0000 7febaeffd640 1 -- 172.21.15.81:0/3222262009 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb40059c0 con 0x7febb80ca130 2023-11-08T16:29:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.496+0000 7febc06e0640 1 -- 172.21.15.81:0/3222262009 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb815b740 con 0x7febb80ca130 2023-11-08T16:29:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.496+0000 7febc06e0640 1 -- 172.21.15.81:0/3222262009 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb815bd00 con 0x7febb80ca130 2023-11-08T16:29:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.497+0000 7febc06e0640 1 -- 172.21.15.81:0/3222262009 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febb80743f0 con 0x7febb80ca130 2023-11-08T16:29:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.498+0000 7febaeffd640 1 -- 172.21.15.81:0/3222262009 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7febb4005b20 con 0x7febb80ca130 2023-11-08T16:29:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.499+0000 7febaeffd640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06ae70 0x7feb9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.500+0000 7febaeffd640 1 -- 172.21.15.81:0/3222262009 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7febb4088b70 con 0x7febb80ca130 2023-11-08T16:29:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.500+0000 7febbdc7b640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06ae70 0x7feb9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.500+0000 7febbdc7b640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06ae70 0x7feb9c06d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7febb81f6210 tx=0x7feba8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:54.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.504+0000 7febaeffd640 1 -- 172.21.15.81:0/3222262009 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7febb4053b10 con 0x7febb80ca130 2023-11-08T16:29:54.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.712+0000 7febc06e0640 1 -- 172.21.15.81:0/3222262009 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7febb8071c50 con 0x7febb80ca130 2023-11-08T16:29:54.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.712+0000 7febaeffd640 1 -- 172.21.15.81:0/3222262009 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7febb4017030 con 0x7febb80ca130 2023-11-08T16:29:54.715 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:29:54.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 -- 172.21.15.81:0/3222262009 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06ae70 msgr2=0x7feb9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06ae70 0x7feb9c06d330 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7febb81f6210 tx=0x7feba8008040 comp rx=0 tx=0).stop 2023-11-08T16:29:54.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 -- 172.21.15.81:0/3222262009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb80ca130 msgr2=0x7febb8157050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:54.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb80ca130 0x7febb8157050 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7febb400c460 tx=0x7febb400c920 comp rx=0 tx=0).stop 2023-11-08T16:29:54.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 -- 172.21.15.81:0/3222262009 shutdown_connections 2023-11-08T16:29:54.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06ae70 0x7feb9c06d330 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb815d3b0 0x7febb80c4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febb80cf720 0x7febb815c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 --2- 172.21.15.81:0/3222262009 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb80ca130 0x7febb8157050 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:54.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 -- 172.21.15.81:0/3222262009 >> 172.21.15.81:0/3222262009 conn(0x7febb80ceaa0 msgr2=0x7febb806bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:54.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 -- 172.21.15.81:0/3222262009 shutdown_connections 2023-11-08T16:29:54.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:54.715+0000 7febc06e0640 1 -- 172.21.15.81:0/3222262009 wait complete. 2023-11-08T16:29:54.736 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:29:54.736 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:29:54.799 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:55.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.020+0000 7ff183e0e640 1 Processor -- start 2023-11-08T16:29:55.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.020+0000 7ff183e0e640 1 -- start start 2023-11-08T16:29:55.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.021+0000 7ff183e0e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff17c15d2b0 0x7ff17c161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.021+0000 7ff183e0e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff17c0ccf00 0x7ff17c06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.021+0000 7ff183e0e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17c161c50 con 0x7ff17c0c9060 2023-11-08T16:29:55.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.021+0000 7ff183e0e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff17c15d690 con 0x7ff17c0ccf00 2023-11-08T16:29:55.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.021+0000 7ff183e0e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c06b980 con 0x7ff17c15d2b0 2023-11-08T16:29:55.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff1813a9640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff17c0c9060 0x7ff17c0cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52716/0 (socket says 172.21.15.81:52716) 2023-11-08T16:29:55.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff1813a9640 1 -- 172.21.15.81:0/2422852092 learned_addr learned my addr 172.21.15.81:0/2422852092 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:55.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff1823ab640 1 --2- 172.21.15.81:0/2422852092 >> v2:172.21.15.93:3300/0 conn(0x7ff17c0ccf00 0x7ff17c06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3788547344 0 0) 0x7ff168001110 con 0x7ff17c0c9060 2023-11-08T16:29:55.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff181baa640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff17c15d2b0 0x7ff17c161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff16c003520 con 0x7ff17c0c9060 2023-11-08T16:29:55.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 374527942 0 0) 0x7ff16c003520 con 0x7ff17c0c9060 2023-11-08T16:29:55.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff17c15d2b0 msgr2=0x7ff17c161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:55.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff180ba8640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff17c15d2b0 0x7ff17c161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 >> v2:172.21.15.93:3300/0 conn(0x7ff17c0ccf00 msgr2=0x7ff17c06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:55.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff180ba8640 1 --2- 172.21.15.81:0/2422852092 >> v2:172.21.15.93:3300/0 conn(0x7ff17c0ccf00 0x7ff17c06ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c06bc60 con 0x7ff17c0c9060 2023-11-08T16:29:55.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.022+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ff168001800 con 0x7ff17c0c9060 2023-11-08T16:29:55.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 >> v1:172.21.15.81:6789/0 conn(0x7ff17c0c9060 legacy=0x7ff17c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff180ba8640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff16c003a50 0x7ff16c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff180ba8640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff16c0047c0 0x7ff16c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff180ba8640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff17c15d2b0 0x7ff16c0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c15d690 con 0x7ff16c003a50 2023-11-08T16:29:55.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c06b980 con 0x7ff16c0047c0 2023-11-08T16:29:55.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c00bb60 con 0x7ff17c15d2b0 2023-11-08T16:29:55.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff181baa640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff16c003a50 0x7ff16c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff1813a9640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff16c0047c0 0x7ff16c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff1823ab640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff17c15d2b0 0x7ff16c0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff181baa640 1 -- 172.21.15.81:0/2422852092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff17c15d2b0 msgr2=0x7ff16c0051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:55.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff181baa640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff17c15d2b0 0x7ff16c0051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff181baa640 1 -- 172.21.15.81:0/2422852092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff16c0047c0 msgr2=0x7ff16c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:55.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff181baa640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff16c0047c0 0x7ff16c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.023+0000 7ff181baa640 1 -- 172.21.15.81:0/2422852092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c003520 con 0x7ff16c003a50 2023-11-08T16:29:55.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.024+0000 7ff181baa640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff16c003a50 0x7ff16c003e70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff170007390 tx=0x7ff16c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6723d8baa845c773 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:55.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.025+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17002ccb0 con 0x7ff16c003a50 2023-11-08T16:29:55.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.025+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17002ce10 con 0x7ff16c003a50 2023-11-08T16:29:55.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.025+0000 7ff180ba8640 1 -- 172.21.15.81:0/2422852092 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170034660 con 0x7ff16c003a50 2023-11-08T16:29:55.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.025+0000 7ff183e0e640 1 -- 172.21.15.81:0/2422852092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff16c003a50 msgr2=0x7ff16c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.025+0000 7ff183e0e640 1 --2- 172.21.15.81:0/2422852092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff16c003a50 0x7ff16c003e70 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff170007390 tx=0x7ff16c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.025+0000 7ff1823ab640 1 -- 172.21.15.81:0/2422852092 reap_dead start 2023-11-08T16:29:55.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.025+0000 7ff183e0e640 1 -- 172.21.15.81:0/2422852092 shutdown_connections 2023-11-08T16:29:55.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.025+0000 7ff183e0e640 1 -- 172.21.15.81:0/2422852092 >> 172.21.15.81:0/2422852092 conn(0x7ff17c0cb490 msgr2=0x7ff17c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:55.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.025+0000 7ff183e0e640 1 -- 172.21.15.81:0/2422852092 shutdown_connections 2023-11-08T16:29:55.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.025+0000 7ff183e0e640 1 -- 172.21.15.81:0/2422852092 wait complete. 2023-11-08T16:29:55.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.027+0000 7ff183e0e640 1 Processor -- start 2023-11-08T16:29:55.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.027+0000 7ff183e0e640 1 -- start start 2023-11-08T16:29:55.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.028+0000 7ff183e0e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff16c003a50 0x7ff17c156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.028+0000 7ff183e0e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff17c0c5090 0x7ff17c0b2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.028+0000 7ff183e0e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff17c15f4e0 0x7ff17c160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.028+0000 7ff183e0e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c15dbb0 con 0x7ff17c0c5090 2023-11-08T16:29:55.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.028+0000 7ff183e0e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c15fe30 con 0x7ff17c15f4e0 2023-11-08T16:29:55.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.028+0000 7ff183e0e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17c160110 con 0x7ff16c003a50 2023-11-08T16:29:55.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.028+0000 7ff1813a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff17c0c5090 0x7ff17c0b2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.028+0000 7ff1813a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff17c0c5090 0x7ff17c0b2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46958/0 (socket says 172.21.15.81:46958) 2023-11-08T16:29:55.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.028+0000 7ff1813a9640 1 -- 172.21.15.81:0/3978632765 learned_addr learned my addr 172.21.15.81:0/3978632765 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:55.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff181baa640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff16c003a50 0x7ff17c156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff1823ab640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff17c15f4e0 0x7ff17c160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff1813a9640 1 -- 172.21.15.81:0/3978632765 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff16c003a50 msgr2=0x7ff17c156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff1813a9640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff16c003a50 0x7ff17c156f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff1813a9640 1 -- 172.21.15.81:0/3978632765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff17c15f4e0 msgr2=0x7ff17c160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff1813a9640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff17c15f4e0 0x7ff17c160e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff1813a9640 1 -- 172.21.15.81:0/3978632765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c1603f0 con 0x7ff17c0c5090 2023-11-08T16:29:55.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff1813a9640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff17c0c5090 0x7ff17c0b2800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff16800d6c0 tx=0x7ff16800db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:55.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff17a7fc640 1 -- 172.21.15.81:0/3978632765 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1680072d0 con 0x7ff17c0c5090 2023-11-08T16:29:55.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff17a7fc640 1 -- 172.21.15.81:0/3978632765 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff168007430 con 0x7ff17c0c5090 2023-11-08T16:29:55.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff17a7fc640 1 -- 172.21.15.81:0/3978632765 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1680047e0 con 0x7ff17c0c5090 2023-11-08T16:29:55.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.029+0000 7ff183e0e640 1 -- 172.21.15.81:0/3978632765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17c15d7d0 con 0x7ff17c0c5090 2023-11-08T16:29:55.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.030+0000 7ff183e0e640 1 -- 172.21.15.81:0/3978632765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff17c0b2d40 con 0x7ff17c0c5090 2023-11-08T16:29:55.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.031+0000 7ff183e0e640 1 -- 172.21.15.81:0/3978632765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17c15dcf0 con 0x7ff17c0c5090 2023-11-08T16:29:55.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.032+0000 7ff17a7fc640 1 -- 172.21.15.81:0/3978632765 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff168004940 con 0x7ff17c0c5090 2023-11-08T16:29:55.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.032+0000 7ff17a7fc640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff16006ae70 0x7ff16006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.033+0000 7ff17a7fc640 1 -- 172.21.15.81:0/3978632765 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff168088be0 con 0x7ff17c0c5090 2023-11-08T16:29:55.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.033+0000 7ff181baa640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff16006ae70 0x7ff16006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.033+0000 7ff181baa640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff16006ae70 0x7ff16006d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff170036000 tx=0x7ff170030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:55.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.037+0000 7ff17a7fc640 1 -- 172.21.15.81:0/3978632765 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff17c15dcf0 con 0x7ff17c0c5090 2023-11-08T16:29:55.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.226+0000 7ff183e0e640 1 -- 172.21.15.81:0/3978632765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff17c161c50 con 0x7ff17c0c5090 2023-11-08T16:29:55.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.229+0000 7ff17a7fc640 1 -- 172.21.15.81:0/3978632765 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7ff17c161c50 con 0x7ff17c0c5090 2023-11-08T16:29:55.233 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:55.233 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:55.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 -- 172.21.15.81:0/3978632765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff16006ae70 msgr2=0x7ff16006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff16006ae70 0x7ff16006d330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff170036000 tx=0x7ff170030000 comp rx=0 tx=0).stop 2023-11-08T16:29:55.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 -- 172.21.15.81:0/3978632765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff17c0c5090 msgr2=0x7ff17c0b2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff17c0c5090 0x7ff17c0b2800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff16800d6c0 tx=0x7ff16800db80 comp rx=0 tx=0).stop 2023-11-08T16:29:55.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 -- 172.21.15.81:0/3978632765 shutdown_connections 2023-11-08T16:29:55.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff16006ae70 0x7ff16006d330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff17c15f4e0 0x7ff17c160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff17c0c5090 0x7ff17c0b2800 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 --2- 172.21.15.81:0/3978632765 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff16c003a50 0x7ff17c156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 -- 172.21.15.81:0/3978632765 >> 172.21.15.81:0/3978632765 conn(0x7ff17c0cb490 msgr2=0x7ff17c15c680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:55.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 -- 172.21.15.81:0/3978632765 shutdown_connections 2023-11-08T16:29:55.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.232+0000 7ff183e0e640 1 -- 172.21.15.81:0/3978632765 wait complete. 2023-11-08T16:29:55.253 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:55.254 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:55.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.454+0000 7ffaab789640 1 Processor -- start 2023-11-08T16:29:55.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.455+0000 7ffaab789640 1 -- start start 2023-11-08T16:29:55.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.456+0000 7ffaab789640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ffaa40ccc10 0x7ffaa40ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.456+0000 7ffaab789640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ffaa406b710 0x7ffaa406bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.456+0000 7ffaab789640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa40ce960 con 0x7ffaa415a0e0 2023-11-08T16:29:55.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.456+0000 7ffaab789640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa4071980 con 0x7ffaa40ccc10 2023-11-08T16:29:55.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.456+0000 7ffaab789640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaa40c9480 con 0x7ffaa406b710 2023-11-08T16:29:55.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.456+0000 7ffaa9525640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ffaa415a0e0 0x7ffaa40c9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52728/0 (socket says 172.21.15.81:52728) 2023-11-08T16:29:55.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.456+0000 7ffaa9525640 1 -- 172.21.15.81:0/4127008689 learned_addr learned my addr 172.21.15.81:0/4127008689 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:55.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.456+0000 7ffaa8d24640 1 --2- 172.21.15.81:0/4127008689 >> v2:172.21.15.81:3300/0 conn(0x7ffaa40ccc10 0x7ffaa40ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.456+0000 7ffaa9d26640 1 --2- 172.21.15.81:0/4127008689 >> v2:172.21.15.93:3300/0 conn(0x7ffaa406b710 0x7ffaa406bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.457+0000 7ffaa3fff640 1 -- 172.21.15.81:0/4127008689 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106668068 0 0) 0x7ffa98001490 con 0x7ffaa415a0e0 2023-11-08T16:29:55.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.457+0000 7ffaa3fff640 1 -- 172.21.15.81:0/4127008689 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa94003520 con 0x7ffaa415a0e0 2023-11-08T16:29:55.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.457+0000 7ffaa8d24640 1 -- 172.21.15.81:0/4127008689 >> v2:172.21.15.93:3300/0 conn(0x7ffaa406b710 msgr2=0x7ffaa406bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:55.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.457+0000 7ffaa8d24640 1 --2- 172.21.15.81:0/4127008689 >> v2:172.21.15.93:3300/0 conn(0x7ffaa406b710 0x7ffaa406bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.457+0000 7ffaa8d24640 1 -- 172.21.15.81:0/4127008689 >> v1:172.21.15.81:6789/0 conn(0x7ffaa415a0e0 legacy=0x7ffaa40c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.457+0000 7ffaa8d24640 1 -- 172.21.15.81:0/4127008689 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa415a4c0 con 0x7ffaa40ccc10 2023-11-08T16:29:55.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.458+0000 7ffaa8d24640 1 --2- 172.21.15.81:0/4127008689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffaa40ccc10 0x7ffaa40ca300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffaa406fa20 tx=0x7ffa9002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f0ceeccf812ddf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:55.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.458+0000 7ffaa3fff640 1 -- 172.21.15.81:0/4127008689 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90030610 con 0x7ffaa40ccc10 2023-11-08T16:29:55.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.458+0000 7ffaa3fff640 1 -- 172.21.15.81:0/4127008689 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa90030770 con 0x7ffaa40ccc10 2023-11-08T16:29:55.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.458+0000 7ffaa3fff640 1 -- 172.21.15.81:0/4127008689 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa90030b00 con 0x7ffaa40ccc10 2023-11-08T16:29:55.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.459+0000 7ffaab789640 1 -- 172.21.15.81:0/4127008689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffaa40ccc10 msgr2=0x7ffaa40ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.459+0000 7ffaab789640 1 --2- 172.21.15.81:0/4127008689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffaa40ccc10 0x7ffaa40ca300 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffaa406fa20 tx=0x7ffa9002b960 comp rx=0 tx=0).stop 2023-11-08T16:29:55.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.459+0000 7ffaab789640 1 -- 172.21.15.81:0/4127008689 shutdown_connections 2023-11-08T16:29:55.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.459+0000 7ffaab789640 1 --2- 172.21.15.81:0/4127008689 >> v2:172.21.15.93:3300/0 conn(0x7ffaa406b710 0x7ffaa406bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.459+0000 7ffaab789640 1 --2- 172.21.15.81:0/4127008689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffaa40ccc10 0x7ffaa40ca300 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.459+0000 7ffaab789640 1 -- 172.21.15.81:0/4127008689 >> 172.21.15.81:0/4127008689 conn(0x7ffaa40cb580 msgr2=0x7ffaa40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:55.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.459+0000 7ffaab789640 1 -- 172.21.15.81:0/4127008689 shutdown_connections 2023-11-08T16:29:55.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.459+0000 7ffaab789640 1 -- 172.21.15.81:0/4127008689 wait complete. 2023-11-08T16:29:55.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.461+0000 7ffaab789640 1 Processor -- start 2023-11-08T16:29:55.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.461+0000 7ffaab789640 1 -- start start 2023-11-08T16:29:55.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaab789640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffaa406b710 0x7ffaa41f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaab789640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffaa415a0e0 0x7ffaa40c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaab789640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffaa40b2c50 0x7ffaa40b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaab789640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa41f3fc0 con 0x7ffaa406b710 2023-11-08T16:29:55.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaab789640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa41f34f0 con 0x7ffaa40b2c50 2023-11-08T16:29:55.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaab789640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa40c4ff0 con 0x7ffaa415a0e0 2023-11-08T16:29:55.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaa9525640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffaa406b710 0x7ffaa41f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaa9525640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffaa406b710 0x7ffaa41f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46982/0 (socket says 172.21.15.81:46982) 2023-11-08T16:29:55.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaa9525640 1 -- 172.21.15.81:0/2380923947 learned_addr learned my addr 172.21.15.81:0/2380923947 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:55.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaa9525640 1 -- 172.21.15.81:0/2380923947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffaa415a0e0 msgr2=0x7ffaa40c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaa9525640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffaa415a0e0 0x7ffaa40c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaa9525640 1 -- 172.21.15.81:0/2380923947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffaa40b2c50 msgr2=0x7ffaa40b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaa9525640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffaa40b2c50 0x7ffaa40b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.462+0000 7ffaa9525640 1 -- 172.21.15.81:0/2380923947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa90009400 con 0x7ffaa406b710 2023-11-08T16:29:55.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.463+0000 7ffaa9525640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffaa406b710 0x7ffaa41f2fb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800cbb0 tx=0x7ffa9800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:55.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.463+0000 7ffaa1ffb640 1 -- 172.21.15.81:0/2380923947 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9800ad40 con 0x7ffaa406b710 2023-11-08T16:29:55.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.463+0000 7ffaa1ffb640 1 -- 172.21.15.81:0/2380923947 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa98012400 con 0x7ffaa406b710 2023-11-08T16:29:55.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.463+0000 7ffaa1ffb640 1 -- 172.21.15.81:0/2380923947 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9800aea0 con 0x7ffaa406b710 2023-11-08T16:29:55.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.463+0000 7ffaab789640 1 -- 172.21.15.81:0/2380923947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa40ad610 con 0x7ffaa406b710 2023-11-08T16:29:55.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.463+0000 7ffaab789640 1 -- 172.21.15.81:0/2380923947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa41f37d0 con 0x7ffaa406b710 2023-11-08T16:29:55.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.464+0000 7ffaab789640 1 -- 172.21.15.81:0/2380923947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa404ece0 con 0x7ffaa406b710 2023-11-08T16:29:55.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.465+0000 7ffaa1ffb640 1 -- 172.21.15.81:0/2380923947 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa98007070 con 0x7ffaa406b710 2023-11-08T16:29:55.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.466+0000 7ffaa1ffb640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffa8806aea0 0x7ffa8806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.466+0000 7ffaa1ffb640 1 -- 172.21.15.81:0/2380923947 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ffa980894b0 con 0x7ffaa406b710 2023-11-08T16:29:55.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.466+0000 7ffaa8d24640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffa8806aea0 0x7ffa8806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.467+0000 7ffaa8d24640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffa8806aea0 0x7ffa8806d360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffa90036000 tx=0x7ffa9003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:55.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.470+0000 7ffaa1ffb640 1 -- 172.21.15.81:0/2380923947 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffa980544d0 con 0x7ffaa406b710 2023-11-08T16:29:55.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.658+0000 7ffaab789640 1 -- 172.21.15.81:0/2380923947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffaa40ce960 con 0x7ffaa406b710 2023-11-08T16:29:55.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.659+0000 7ffaa1ffb640 1 -- 172.21.15.81:0/2380923947 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa98017020 con 0x7ffaa406b710 2023-11-08T16:29:55.662 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 -- 172.21.15.81:0/2380923947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffa8806aea0 msgr2=0x7ffa8806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffa8806aea0 0x7ffa8806d360 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffa90036000 tx=0x7ffa9003b040 comp rx=0 tx=0).stop 2023-11-08T16:29:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 -- 172.21.15.81:0/2380923947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffaa406b710 msgr2=0x7ffaa41f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffaa406b710 0x7ffaa41f2fb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800cbb0 tx=0x7ffa9800a430 comp rx=0 tx=0).stop 2023-11-08T16:29:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 -- 172.21.15.81:0/2380923947 shutdown_connections 2023-11-08T16:29:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffa8806aea0 0x7ffa8806d360 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffaa40b2c50 0x7ffaa40b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffaa415a0e0 0x7ffaa40c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 --2- 172.21.15.81:0/2380923947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffaa406b710 0x7ffaa41f2fb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 -- 172.21.15.81:0/2380923947 >> 172.21.15.81:0/2380923947 conn(0x7ffaa40cb580 msgr2=0x7ffaa40c9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:55.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 -- 172.21.15.81:0/2380923947 shutdown_connections 2023-11-08T16:29:55.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.662+0000 7ffaab789640 1 -- 172.21.15.81:0/2380923947 wait complete. 2023-11-08T16:29:55.684 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:55.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.899+0000 7ff594664640 1 Processor -- start 2023-11-08T16:29:55.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.900+0000 7ff594664640 1 -- start start 2023-11-08T16:29:55.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.901+0000 7ff594664640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff58c161730 0x7ff58c0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.901+0000 7ff594664640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff58c06d250 0x7ff58c0c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.901+0000 7ff594664640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c06cae0 con 0x7ff58c0c1e50 2023-11-08T16:29:55.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.901+0000 7ff594664640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58c0c22e0 con 0x7ff58c161730 2023-11-08T16:29:55.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.901+0000 7ff594664640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0cdb00 con 0x7ff58c06d250 2023-11-08T16:29:55.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.902+0000 7ff592400640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff58c161730 0x7ff58c0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.902+0000 7ff592400640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff58c161730 0x7ff58c0cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:57770/0 (socket says 172.21.15.81:57770) 2023-11-08T16:29:55.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.902+0000 7ff592400640 1 -- 172.21.15.81:0/1853754602 learned_addr learned my addr 172.21.15.81:0/1853754602 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:55.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.902+0000 7ff591bff640 1 --2- 172.21.15.81:0/1853754602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff58c06d250 0x7ff58c0c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.903+0000 7ff592400640 1 -- 172.21.15.81:0/1853754602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff58c06d250 msgr2=0x7ff58c0c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:55.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.903+0000 7ff592400640 1 --2- 172.21.15.81:0/1853754602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff58c06d250 0x7ff58c0c1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.903+0000 7ff592400640 1 -- 172.21.15.81:0/1853754602 >> v1:172.21.15.93:6789/0 conn(0x7ff58c0c1e50 legacy=0x7ff58c06f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.903+0000 7ff592400640 1 -- 172.21.15.81:0/1853754602 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c161b10 con 0x7ff58c161730 2023-11-08T16:29:55.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.904+0000 7ff592400640 1 --2- 172.21.15.81:0/1853754602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff58c161730 0x7ff58c0cdf90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5880098a0 tx=0x7ff58802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85e302c25e0a66f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:55.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.905+0000 7ff5913fe640 1 -- 172.21.15.81:0/1853754602 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588030840 con 0x7ff58c161730 2023-11-08T16:29:55.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.905+0000 7ff5913fe640 1 -- 172.21.15.81:0/1853754602 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5880309a0 con 0x7ff58c161730 2023-11-08T16:29:55.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.905+0000 7ff5913fe640 1 -- 172.21.15.81:0/1853754602 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff588030d30 con 0x7ff58c161730 2023-11-08T16:29:55.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.905+0000 7ff594664640 1 -- 172.21.15.81:0/1853754602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff58c161730 msgr2=0x7ff58c0cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.905+0000 7ff594664640 1 --2- 172.21.15.81:0/1853754602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff58c161730 0x7ff58c0cdf90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff5880098a0 tx=0x7ff58802b980 comp rx=0 tx=0).stop 2023-11-08T16:29:55.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.906+0000 7ff594664640 1 -- 172.21.15.81:0/1853754602 shutdown_connections 2023-11-08T16:29:55.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.906+0000 7ff594664640 1 --2- 172.21.15.81:0/1853754602 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff58c06d250 0x7ff58c0c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.906+0000 7ff594664640 1 --2- 172.21.15.81:0/1853754602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff58c161730 0x7ff58c0cdf90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.906+0000 7ff594664640 1 -- 172.21.15.81:0/1853754602 >> 172.21.15.81:0/1853754602 conn(0x7ff58c0cb580 msgr2=0x7ff58c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:55.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.906+0000 7ff594664640 1 -- 172.21.15.81:0/1853754602 shutdown_connections 2023-11-08T16:29:55.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.906+0000 7ff594664640 1 -- 172.21.15.81:0/1853754602 wait complete. 2023-11-08T16:29:55.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.908+0000 7ff594664640 1 Processor -- start 2023-11-08T16:29:55.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.908+0000 7ff594664640 1 -- start start 2023-11-08T16:29:55.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff594664640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff58c06d250 0x7ff58c159f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff594664640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff58c0c1e50 0x7ff58c15fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff594664640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff58c161730 0x7ff58c0c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff594664640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c1e9e90 con 0x7ff58c0c1e50 2023-11-08T16:29:55.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff594664640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0c34d0 con 0x7ff58c06d250 2023-11-08T16:29:55.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff594664640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0c37b0 con 0x7ff58c161730 2023-11-08T16:29:55.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff591bff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff58c0c1e50 0x7ff58c15fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff591bff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff58c0c1e50 0x7ff58c15fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46984/0 (socket says 172.21.15.81:46984) 2023-11-08T16:29:55.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff591bff640 1 -- 172.21.15.81:0/3203758696 learned_addr learned my addr 172.21.15.81:0/3203758696 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:55.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff592400640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff58c06d250 0x7ff58c159f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff591bff640 1 -- 172.21.15.81:0/3203758696 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff58c161730 msgr2=0x7ff58c0c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff591bff640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff58c161730 0x7ff58c0c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff591bff640 1 -- 172.21.15.81:0/3203758696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff58c06d250 msgr2=0x7ff58c159f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:55.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff591bff640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff58c06d250 0x7ff58c159f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:55.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.910+0000 7ff591bff640 1 -- 172.21.15.81:0/3203758696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5880094d0 con 0x7ff58c0c1e50 2023-11-08T16:29:55.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.911+0000 7ff591bff640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff58c0c1e50 0x7ff58c15fb60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00e9b0 tx=0x7ff57c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:55.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.911+0000 7ff582ffd640 1 -- 172.21.15.81:0/3203758696 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c009670 con 0x7ff58c0c1e50 2023-11-08T16:29:55.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.911+0000 7ff594664640 1 -- 172.21.15.81:0/3203758696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c1ea070 con 0x7ff58c0c1e50 2023-11-08T16:29:55.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.911+0000 7ff594664640 1 -- 172.21.15.81:0/3203758696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c15a460 con 0x7ff58c0c1e50 2023-11-08T16:29:55.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.912+0000 7ff594664640 1 -- 172.21.15.81:0/3203758696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c04ece0 con 0x7ff58c0c1e50 2023-11-08T16:29:55.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.915+0000 7ff582ffd640 1 -- 172.21.15.81:0/3203758696 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff57c00c9a0 con 0x7ff58c0c1e50 2023-11-08T16:29:55.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.915+0000 7ff582ffd640 1 -- 172.21.15.81:0/3203758696 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff57c00cd10 con 0x7ff58c0c1e50 2023-11-08T16:29:55.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.917+0000 7ff582ffd640 1 -- 172.21.15.81:0/3203758696 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff57c013470 con 0x7ff58c0c1e50 2023-11-08T16:29:55.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.918+0000 7ff582ffd640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff57006ae70 0x7ff57006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:55.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.918+0000 7ff582ffd640 1 -- 172.21.15.81:0/3203758696 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff57c08a060 con 0x7ff58c0c1e50 2023-11-08T16:29:55.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.919+0000 7ff592400640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff57006ae70 0x7ff57006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:55.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.919+0000 7ff592400640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff57006ae70 0x7ff57006d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff588036000 tx=0x7ff58803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:55.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:55.921+0000 7ff582ffd640 1 -- 172.21.15.81:0/3203758696 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff57c055110 con 0x7ff58c0c1e50 2023-11-08T16:29:56.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.116+0000 7ff594664640 1 -- 172.21.15.81:0/3203758696 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff58c0c3970 con 0x7ff58c0c1e50 2023-11-08T16:29:56.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.117+0000 7ff582ffd640 1 -- 172.21.15.81:0/3203758696 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff57c058730 con 0x7ff58c0c1e50 2023-11-08T16:29:56.120 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:56.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 -- 172.21.15.81:0/3203758696 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff57006ae70 msgr2=0x7ff57006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff57006ae70 0x7ff57006d330 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff588036000 tx=0x7ff58803b040 comp rx=0 tx=0).stop 2023-11-08T16:29:56.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 -- 172.21.15.81:0/3203758696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff58c0c1e50 msgr2=0x7ff58c15fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff58c0c1e50 0x7ff58c15fb60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff57c00e9b0 tx=0x7ff57c00ee70 comp rx=0 tx=0).stop 2023-11-08T16:29:56.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 -- 172.21.15.81:0/3203758696 shutdown_connections 2023-11-08T16:29:56.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff57006ae70 0x7ff57006d330 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff58c161730 0x7ff58c0c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff58c0c1e50 0x7ff58c15fb60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 --2- 172.21.15.81:0/3203758696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff58c06d250 0x7ff58c159f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 -- 172.21.15.81:0/3203758696 >> 172.21.15.81:0/3203758696 conn(0x7ff58c0cb580 msgr2=0x7ff58c15d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:56.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 -- 172.21.15.81:0/3203758696 shutdown_connections 2023-11-08T16:29:56.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.120+0000 7ff594664640 1 -- 172.21.15.81:0/3203758696 wait complete. 2023-11-08T16:29:56.141 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:56.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.343+0000 7fedc6978640 1 Processor -- start 2023-11-08T16:29:56.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.343+0000 7fedc6978640 1 -- start start 2023-11-08T16:29:56.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.345+0000 7fedc6978640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fedc015d4c0 0x7fedc00b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.345+0000 7fedc6978640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc006c830 0x7fedc006cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.345+0000 7fedc6978640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc006d1d0 con 0x7fedc00b0010 2023-11-08T16:29:56.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.345+0000 7fedc6978640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc00cefa0 con 0x7fedc015d4c0 2023-11-08T16:29:56.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.345+0000 7fedc6978640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00d0890 con 0x7fedc006c830 2023-11-08T16:29:56.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.345+0000 7fedbffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fedc015d4c0 0x7fedc00b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.345+0000 7fedbffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fedc015d4c0 0x7fedc00b3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46988/0 (socket says 172.21.15.81:46988) 2023-11-08T16:29:56.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.345+0000 7fedbffff640 1 -- 172.21.15.81:0/2754213149 learned_addr learned my addr 172.21.15.81:0/2754213149 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:56.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.346+0000 7fedc4f15640 1 --2- 172.21.15.81:0/2754213149 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc006c830 0x7fedc006cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.346+0000 7fedbffff640 1 -- 172.21.15.81:0/2754213149 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc006c830 msgr2=0x7fedc006cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:56.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.346+0000 7fedbffff640 1 --2- 172.21.15.81:0/2754213149 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc006c830 0x7fedc006cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.346+0000 7fedbffff640 1 -- 172.21.15.81:0/2754213149 >> v1:172.21.15.93:6789/0 conn(0x7fedc00b0010 legacy=0x7fedc00cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:56.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.346+0000 7fedbffff640 1 -- 172.21.15.81:0/2754213149 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00ce9e0 con 0x7fedc015d4c0 2023-11-08T16:29:56.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.347+0000 7fedbffff640 1 --2- 172.21.15.81:0/2754213149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc015d4c0 0x7fedc00b3170 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009870 tx=0x7fedb002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70eeefc469c49863 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:56.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.347+0000 7fedbeffd640 1 -- 172.21.15.81:0/2754213149 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0031640 con 0x7fedc015d4c0 2023-11-08T16:29:56.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.347+0000 7fedbeffd640 1 -- 172.21.15.81:0/2754213149 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb00317a0 con 0x7fedc015d4c0 2023-11-08T16:29:56.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.347+0000 7fedbeffd640 1 -- 172.21.15.81:0/2754213149 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0031b30 con 0x7fedc015d4c0 2023-11-08T16:29:56.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.348+0000 7fedc6978640 1 -- 172.21.15.81:0/2754213149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc015d4c0 msgr2=0x7fedc00b3170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.348+0000 7fedc6978640 1 --2- 172.21.15.81:0/2754213149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc015d4c0 0x7fedc00b3170 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009870 tx=0x7fedb002b8f0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.348+0000 7fedc6978640 1 -- 172.21.15.81:0/2754213149 shutdown_connections 2023-11-08T16:29:56.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.348+0000 7fedc6978640 1 --2- 172.21.15.81:0/2754213149 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc006c830 0x7fedc006cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.348+0000 7fedc6978640 1 --2- 172.21.15.81:0/2754213149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc015d4c0 0x7fedc00b3170 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.348+0000 7fedc6978640 1 -- 172.21.15.81:0/2754213149 >> 172.21.15.81:0/2754213149 conn(0x7fedc006dbe0 msgr2=0x7fedc006dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:56.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.348+0000 7fedc6978640 1 -- 172.21.15.81:0/2754213149 shutdown_connections 2023-11-08T16:29:56.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.348+0000 7fedc6978640 1 -- 172.21.15.81:0/2754213149 wait complete. 2023-11-08T16:29:56.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.350+0000 7fedc6978640 1 Processor -- start 2023-11-08T16:29:56.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.350+0000 7fedc6978640 1 -- start start 2023-11-08T16:29:56.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.351+0000 7fedc6978640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc006c830 0x7fedc0149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.351+0000 7fedc6978640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc00b0010 0x7fedc00b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.351+0000 7fedc6978640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedc015d4c0 0x7fedc00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.351+0000 7fedc6978640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00b4ff0 con 0x7fedc006c830 2023-11-08T16:29:56.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.351+0000 7fedc6978640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc0159300 con 0x7fedc015d4c0 2023-11-08T16:29:56.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.351+0000 7fedc6978640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00b43e0 con 0x7fedc00b0010 2023-11-08T16:29:56.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.351+0000 7fedbffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc006c830 0x7fedc0149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.351+0000 7fedbffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc006c830 0x7fedc0149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47000/0 (socket says 172.21.15.81:47000) 2023-11-08T16:29:56.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.351+0000 7fedbffff640 1 -- 172.21.15.81:0/1667903727 learned_addr learned my addr 172.21.15.81:0/1667903727 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:56.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.351+0000 7fedbffff640 1 -- 172.21.15.81:0/1667903727 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc00b0010 msgr2=0x7fedc00b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.351+0000 7fedbf7fe640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc00b0010 0x7fedc00b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.352+0000 7fedc4f15640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedc015d4c0 0x7fedc00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.352+0000 7fedbffff640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc00b0010 0x7fedc00b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.352+0000 7fedbffff640 1 -- 172.21.15.81:0/1667903727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedc015d4c0 msgr2=0x7fedc00b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.352+0000 7fedbffff640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedc015d4c0 0x7fedc00b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.352+0000 7fedbffff640 1 -- 172.21.15.81:0/1667903727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb00094d0 con 0x7fedc006c830 2023-11-08T16:29:56.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.352+0000 7fedbffff640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc006c830 0x7fedc0149210 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009870 tx=0x7fedb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:56.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.352+0000 7fedbcff9640 1 -- 172.21.15.81:0/1667903727 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb003ac60 con 0x7fedc006c830 2023-11-08T16:29:56.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.352+0000 7fedbcff9640 1 -- 172.21.15.81:0/1667903727 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb003ade0 con 0x7fedc006c830 2023-11-08T16:29:56.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.352+0000 7fedc6978640 1 -- 172.21.15.81:0/1667903727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc00b46c0 con 0x7fedc006c830 2023-11-08T16:29:56.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.352+0000 7fedbcff9640 1 -- 172.21.15.81:0/1667903727 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0044630 con 0x7fedc006c830 2023-11-08T16:29:56.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.352+0000 7fedc6978640 1 -- 172.21.15.81:0/1667903727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc00b4c80 con 0x7fedc006c830 2023-11-08T16:29:56.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.354+0000 7fedc6978640 1 -- 172.21.15.81:0/1667903727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc00d1dc0 con 0x7fedc006c830 2023-11-08T16:29:56.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.354+0000 7fedbcff9640 1 -- 172.21.15.81:0/1667903727 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fedb002f050 con 0x7fedc006c830 2023-11-08T16:29:56.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.355+0000 7fedbcff9640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fedac06ae70 0x7fedac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.355+0000 7fedbcff9640 1 -- 172.21.15.81:0/1667903727 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fedb00ab7c0 con 0x7fedc006c830 2023-11-08T16:29:56.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.356+0000 7fedbf7fe640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fedac06ae70 0x7fedac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.356+0000 7fedbf7fe640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fedac06ae70 0x7fedac06d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fedc01f6a60 tx=0x7fedb4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:56.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.360+0000 7fedbcff9640 1 -- 172.21.15.81:0/1667903727 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fedb00767e0 con 0x7fedc006c830 2023-11-08T16:29:56.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.551+0000 7fedc6978640 1 -- 172.21.15.81:0/1667903727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fedc004ece0 con 0x7fedc006c830 2023-11-08T16:29:56.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.552+0000 7fedbcff9640 1 -- 172.21.15.81:0/1667903727 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fedb0079e00 con 0x7fedc006c830 2023-11-08T16:29:56.555 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:56.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 -- 172.21.15.81:0/1667903727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fedac06ae70 msgr2=0x7fedac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fedac06ae70 0x7fedac06d330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fedc01f6a60 tx=0x7fedb4009040 comp rx=0 tx=0).stop 2023-11-08T16:29:56.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 -- 172.21.15.81:0/1667903727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc006c830 msgr2=0x7fedc0149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc006c830 0x7fedc0149210 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fedb0009870 tx=0x7fedb00046c0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 -- 172.21.15.81:0/1667903727 shutdown_connections 2023-11-08T16:29:56.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fedac06ae70 0x7fedac06d330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fedc015d4c0 0x7fedc00b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fedc00b0010 0x7fedc00b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 --2- 172.21.15.81:0/1667903727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fedc006c830 0x7fedc0149210 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 -- 172.21.15.81:0/1667903727 >> 172.21.15.81:0/1667903727 conn(0x7fedc006dbe0 msgr2=0x7fedc015a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:56.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 -- 172.21.15.81:0/1667903727 shutdown_connections 2023-11-08T16:29:56.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.554+0000 7fedc6978640 1 -- 172.21.15.81:0/1667903727 wait complete. 2023-11-08T16:29:56.575 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:29:56.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.766+0000 7f5f39116640 1 Processor -- start 2023-11-08T16:29:56.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.766+0000 7f5f39116640 1 -- start start 2023-11-08T16:29:56.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.767+0000 7f5f39116640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f3415d310 0x7f5f340d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.767+0000 7f5f39116640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5f340d1300 0x7f5f340cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.767+0000 7f5f39116640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f340d0470 con 0x7f5f340d0000 2023-11-08T16:29:56.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.767+0000 7f5f39116640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f340b6480 con 0x7f5f340d1300 2023-11-08T16:29:56.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.767+0000 7f5f39116640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f340b6100 con 0x7f5f3415d310 2023-11-08T16:29:56.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.768+0000 7f5f3359e640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5f340d0000 0x7f5f340b4b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52732/0 (socket says 172.21.15.81:52732) 2023-11-08T16:29:56.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.768+0000 7f5f3359e640 1 -- 172.21.15.81:0/1115263006 learned_addr learned my addr 172.21.15.81:0/1115263006 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:56.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.768+0000 7f5f2bfff640 1 --2- 172.21.15.81:0/1115263006 >> v2:172.21.15.93:3300/0 conn(0x7f5f340d1300 0x7f5f340cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.768+0000 7f5f32d9d640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f3415d310 0x7f5f340d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.768+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704616077 0 0) 0x7f5f340d0470 con 0x7f5f340d0000 2023-11-08T16:29:56.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.768+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f20003520 con 0x7f5f340d0000 2023-11-08T16:29:56.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1823768591 0 0) 0x7f5f20003520 con 0x7f5f340d0000 2023-11-08T16:29:56.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f3415d310 msgr2=0x7f5f340d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:56.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f3415d310 0x7f5f340d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 >> v2:172.21.15.93:3300/0 conn(0x7f5f340d1300 msgr2=0x7f5f340cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:56.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 --2- 172.21.15.81:0/1115263006 >> v2:172.21.15.93:3300/0 conn(0x7f5f340d1300 0x7f5f340cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f340d1b50 con 0x7f5f340d0000 2023-11-08T16:29:56.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f5f2c0019f0 con 0x7f5f340d0000 2023-11-08T16:29:56.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 >> v1:172.21.15.81:6789/0 conn(0x7f5f340d0000 legacy=0x7f5f340b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 0x7f5f20003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f200047c0 0x7f5f20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f3415d310 0x7f5f20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f340b6480 con 0x7f5f20003970 2023-11-08T16:29:56.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f340b6100 con 0x7f5f200047c0 2023-11-08T16:29:56.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2000b980 con 0x7f5f3415d310 2023-11-08T16:29:56.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f32d9d640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 0x7f5f20003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.769+0000 7f5f2bfff640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f200047c0 0x7f5f20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.770+0000 7f5f3359e640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f3415d310 0x7f5f20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.770+0000 7f5f32d9d640 1 -- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f3415d310 msgr2=0x7f5f20005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:56.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.770+0000 7f5f32d9d640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f3415d310 0x7f5f20005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.770+0000 7f5f32d9d640 1 -- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f200047c0 msgr2=0x7f5f20004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:56.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.770+0000 7f5f32d9d640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f200047c0 0x7f5f20004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.770+0000 7f5f32d9d640 1 -- 172.21.15.81:0/1115263006 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f20003520 con 0x7f5f20003970 2023-11-08T16:29:56.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.770+0000 7f5f3359e640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f3415d310 0x7f5f20005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:56.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.771+0000 7f5f32d9d640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 0x7f5f20003d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009ac0 tx=0x7f5f200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23d6225eefa9501 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:56.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.771+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2402c6b0 con 0x7f5f20003970 2023-11-08T16:29:56.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.771+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2402c810 con 0x7f5f20003970 2023-11-08T16:29:56.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.771+0000 7f5f3259c640 1 -- 172.21.15.81:0/1115263006 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2402cba0 con 0x7f5f20003970 2023-11-08T16:29:56.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.772+0000 7f5f39116640 1 -- 172.21.15.81:0/1115263006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 msgr2=0x7f5f20003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.772+0000 7f5f39116640 1 --2- 172.21.15.81:0/1115263006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 0x7f5f20003d90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009ac0 tx=0x7f5f200057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.772+0000 7f5f3359e640 1 -- 172.21.15.81:0/1115263006 reap_dead start 2023-11-08T16:29:56.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.772+0000 7f5f39116640 1 -- 172.21.15.81:0/1115263006 shutdown_connections 2023-11-08T16:29:56.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.772+0000 7f5f39116640 1 -- 172.21.15.81:0/1115263006 >> 172.21.15.81:0/1115263006 conn(0x7f5f3406aac0 msgr2=0x7f5f3406aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:56.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.772+0000 7f5f39116640 1 -- 172.21.15.81:0/1115263006 shutdown_connections 2023-11-08T16:29:56.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.772+0000 7f5f39116640 1 -- 172.21.15.81:0/1115263006 wait complete. 2023-11-08T16:29:56.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.774+0000 7f5f39116640 1 Processor -- start 2023-11-08T16:29:56.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.774+0000 7f5f39116640 1 -- start start 2023-11-08T16:29:56.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f39116640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 0x7f5f34158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f39116640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f340c9630 0x7f5f340cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f39116640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f340cb1c0 0x7f5f341560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f39116640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f34158a00 con 0x7f5f20003970 2023-11-08T16:29:56.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f39116640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f340cb660 con 0x7f5f340cb1c0 2023-11-08T16:29:56.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f39116640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f341580d0 con 0x7f5f340c9630 2023-11-08T16:29:56.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f32d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 0x7f5f34158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f32d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 0x7f5f34158e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47020/0 (socket says 172.21.15.81:47020) 2023-11-08T16:29:56.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f32d9d640 1 -- 172.21.15.81:0/2631152710 learned_addr learned my addr 172.21.15.81:0/2631152710 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:56.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f32d9d640 1 -- 172.21.15.81:0/2631152710 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f340c9630 msgr2=0x7f5f340cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f32d9d640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f340c9630 0x7f5f340cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f32d9d640 1 -- 172.21.15.81:0/2631152710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f340cb1c0 msgr2=0x7f5f341560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f32d9d640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f340cb1c0 0x7f5f341560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.775+0000 7f5f32d9d640 1 -- 172.21.15.81:0/2631152710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f341583b0 con 0x7f5f20003970 2023-11-08T16:29:56.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.776+0000 7f5f32d9d640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 0x7f5f34158e90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402c250 tx=0x7f5f24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:56.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.776+0000 7f5f2b7fe640 1 -- 172.21.15.81:0/2631152710 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24003810 con 0x7f5f20003970 2023-11-08T16:29:56.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.776+0000 7f5f2b7fe640 1 -- 172.21.15.81:0/2631152710 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f24003970 con 0x7f5f20003970 2023-11-08T16:29:56.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.776+0000 7f5f2b7fe640 1 -- 172.21.15.81:0/2631152710 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24003ce0 con 0x7f5f20003970 2023-11-08T16:29:56.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.776+0000 7f5f39116640 1 -- 172.21.15.81:0/2631152710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f34158690 con 0x7f5f20003970 2023-11-08T16:29:56.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.776+0000 7f5f39116640 1 -- 172.21.15.81:0/2631152710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f3415a480 con 0x7f5f20003970 2023-11-08T16:29:56.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.777+0000 7f5f39116640 1 -- 172.21.15.81:0/2631152710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f340743f0 con 0x7f5f20003970 2023-11-08T16:29:56.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.778+0000 7f5f2b7fe640 1 -- 172.21.15.81:0/2631152710 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f24034440 con 0x7f5f20003970 2023-11-08T16:29:56.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.780+0000 7f5f2b7fe640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5f1406ae70 0x7f5f1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:56.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.780+0000 7f5f2b7fe640 1 -- 172.21.15.81:0/2631152710 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5f240ab9f0 con 0x7f5f20003970 2023-11-08T16:29:56.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.780+0000 7f5f2bfff640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5f1406ae70 0x7f5f1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:56.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.781+0000 7f5f2bfff640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5f1406ae70 0x7f5f1406d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5f341f5ad0 tx=0x7f5f1c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:56.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.784+0000 7f5f2b7fe640 1 -- 172.21.15.81:0/2631152710 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f24076a20 con 0x7f5f20003970 2023-11-08T16:29:56.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.991+0000 7f5f39116640 1 -- 172.21.15.81:0/2631152710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5f340d2dd0 con 0x7f5f20003970 2023-11-08T16:29:56.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.992+0000 7f5f2b7fe640 1 -- 172.21.15.81:0/2631152710 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5f2404c080 con 0x7f5f20003970 2023-11-08T16:29:56.995 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:29:56.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.994+0000 7f5f39116640 1 -- 172.21.15.81:0/2631152710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5f1406ae70 msgr2=0x7f5f1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.994+0000 7f5f39116640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5f1406ae70 0x7f5f1406d330 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5f341f5ad0 tx=0x7f5f1c008040 comp rx=0 tx=0).stop 2023-11-08T16:29:56.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.994+0000 7f5f39116640 1 -- 172.21.15.81:0/2631152710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 msgr2=0x7f5f34158e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:56.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.994+0000 7f5f39116640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 0x7f5f34158e90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5f2402c250 tx=0x7f5f24004730 comp rx=0 tx=0).stop 2023-11-08T16:29:56.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.995+0000 7f5f39116640 1 -- 172.21.15.81:0/2631152710 shutdown_connections 2023-11-08T16:29:56.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.995+0000 7f5f39116640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5f1406ae70 0x7f5f1406d330 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.995+0000 7f5f39116640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f340cb1c0 0x7f5f341560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.995+0000 7f5f39116640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f340c9630 0x7f5f340cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.995+0000 7f5f39116640 1 --2- 172.21.15.81:0/2631152710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f20003970 0x7f5f34158e90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:56.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.995+0000 7f5f39116640 1 -- 172.21.15.81:0/2631152710 >> 172.21.15.81:0/2631152710 conn(0x7f5f3406aac0 msgr2=0x7f5f34155920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:57.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.995+0000 7f5f39116640 1 -- 172.21.15.81:0/2631152710 shutdown_connections 2023-11-08T16:29:57.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:56.995+0000 7f5f39116640 1 -- 172.21.15.81:0/2631152710 wait complete. 2023-11-08T16:29:57.016 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi093.front.sepia.ceph.com... 2023-11-08T16:29:57.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:57.017 DEBUG:teuthology.orchestra.run.smithi093:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-08T16:29:57.117 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:29:57.118 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:29:57.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:29:57.119 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-08T16:29:57.185 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:29:57.186 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:29:57.239 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:29:57.239 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:29:57.289 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:57.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.524+0000 7f051c716640 1 Processor -- start 2023-11-08T16:29:57.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.524+0000 7f051c716640 1 -- start start 2023-11-08T16:29:57.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.525+0000 7f051c716640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0514160700 0x7f05140cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.525+0000 7f051c716640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f05140cf050 0x7f05140cda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.525+0000 7f051c716640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05140cb3a0 con 0x7f05140d17d0 2023-11-08T16:29:57.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.525+0000 7f051c716640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05140cb680 con 0x7f05140cf050 2023-11-08T16:29:57.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.525+0000 7f051c716640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05140cb960 con 0x7f0514160700 2023-11-08T16:29:57.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.525+0000 7f051acb3640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f05140d17d0 0x7f051406b460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52746/0 (socket says 172.21.15.81:52746) 2023-11-08T16:29:57.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.525+0000 7f051acb3640 1 -- 172.21.15.81:0/1463291013 learned_addr learned my addr 172.21.15.81:0/1463291013 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:57.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.525+0000 7f0519cb1640 1 --2- 172.21.15.81:0/1463291013 >> v2:172.21.15.81:3300/0 conn(0x7f05140cf050 0x7f05140cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:57.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.526+0000 7f051a4b2640 1 --2- 172.21.15.81:0/1463291013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0514160700 0x7f05140cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:57.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.526+0000 7f05194b0640 1 -- 172.21.15.81:0/1463291013 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2059843036 0 0) 0x7f05140cb3a0 con 0x7f05140d17d0 2023-11-08T16:29:57.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.526+0000 7f05194b0640 1 -- 172.21.15.81:0/1463291013 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0500003520 con 0x7f05140d17d0 2023-11-08T16:29:57.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.526+0000 7f0519cb1640 1 -- 172.21.15.81:0/1463291013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0514160700 msgr2=0x7f05140cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:57.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.526+0000 7f0519cb1640 1 --2- 172.21.15.81:0/1463291013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0514160700 0x7f05140cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.526+0000 7f0519cb1640 1 -- 172.21.15.81:0/1463291013 >> v1:172.21.15.81:6789/0 conn(0x7f05140d17d0 legacy=0x7f051406b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:57.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.526+0000 7f0519cb1640 1 -- 172.21.15.81:0/1463291013 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140cffc0 con 0x7f05140cf050 2023-11-08T16:29:57.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.527+0000 7f0519cb1640 1 --2- 172.21.15.81:0/1463291013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05140cf050 0x7f05140cda70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f05040097a0 tx=0x7f050402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9eccd7f4bc2895 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:57.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.528+0000 7f05194b0640 1 -- 172.21.15.81:0/1463291013 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504031780 con 0x7f05140cf050 2023-11-08T16:29:57.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.528+0000 7f05194b0640 1 -- 172.21.15.81:0/1463291013 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05040318e0 con 0x7f05140cf050 2023-11-08T16:29:57.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.528+0000 7f05194b0640 1 -- 172.21.15.81:0/1463291013 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504031c70 con 0x7f05140cf050 2023-11-08T16:29:57.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.528+0000 7f051c716640 1 -- 172.21.15.81:0/1463291013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05140cf050 msgr2=0x7f05140cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:57.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.528+0000 7f051c716640 1 --2- 172.21.15.81:0/1463291013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05140cf050 0x7f05140cda70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f05040097a0 tx=0x7f050402b820 comp rx=0 tx=0).stop 2023-11-08T16:29:57.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.528+0000 7f051c716640 1 -- 172.21.15.81:0/1463291013 shutdown_connections 2023-11-08T16:29:57.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.528+0000 7f051c716640 1 --2- 172.21.15.81:0/1463291013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05140cf050 0x7f05140cda70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.528+0000 7f051c716640 1 --2- 172.21.15.81:0/1463291013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0514160700 0x7f05140cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.528+0000 7f051c716640 1 -- 172.21.15.81:0/1463291013 >> 172.21.15.81:0/1463291013 conn(0x7f05140d0cb0 msgr2=0x7f05140d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:57.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.528+0000 7f051c716640 1 -- 172.21.15.81:0/1463291013 shutdown_connections 2023-11-08T16:29:57.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.528+0000 7f051c716640 1 -- 172.21.15.81:0/1463291013 wait complete. 2023-11-08T16:29:57.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.530+0000 7f051c716640 1 Processor -- start 2023-11-08T16:29:57.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.530+0000 7f051c716640 1 -- start start 2023-11-08T16:29:57.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.531+0000 7f051c716640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05140cf050 0x7f05140cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.531+0000 7f051c716640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05140d17d0 0x7f05140c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.531+0000 7f051c716640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0514160700 0x7f05141f2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.531+0000 7f051c716640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140c82c0 con 0x7f05140d17d0 2023-11-08T16:29:57.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.531+0000 7f051c716640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140c8570 con 0x7f0514160700 2023-11-08T16:29:57.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.531+0000 7f051c716640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05140c8850 con 0x7f05140cf050 2023-11-08T16:29:57.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.531+0000 7f0519cb1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05140d17d0 0x7f05140c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:57.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.531+0000 7f0519cb1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05140d17d0 0x7f05140c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47038/0 (socket says 172.21.15.81:47038) 2023-11-08T16:29:57.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.531+0000 7f0519cb1640 1 -- 172.21.15.81:0/2166557006 learned_addr learned my addr 172.21.15.81:0/2166557006 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:57.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.532+0000 7f0519cb1640 1 -- 172.21.15.81:0/2166557006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05140cf050 msgr2=0x7f05140cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:57.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.532+0000 7f0519cb1640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05140cf050 0x7f05140cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.532+0000 7f0519cb1640 1 -- 172.21.15.81:0/2166557006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0514160700 msgr2=0x7f05141f2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:57.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.532+0000 7f0519cb1640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0514160700 0x7f05141f2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.532+0000 7f0519cb1640 1 -- 172.21.15.81:0/2166557006 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0504009400 con 0x7f05140d17d0 2023-11-08T16:29:57.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.532+0000 7f0519cb1640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05140d17d0 0x7f05140c7d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0504034040 tx=0x7f05040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:57.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.532+0000 7f050affd640 1 -- 172.21.15.81:0/2166557006 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504032820 con 0x7f05140d17d0 2023-11-08T16:29:57.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.532+0000 7f050affd640 1 -- 172.21.15.81:0/2166557006 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0504032980 con 0x7f05140d17d0 2023-11-08T16:29:57.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.532+0000 7f051c716640 1 -- 172.21.15.81:0/2166557006 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05140c8b30 con 0x7f05140d17d0 2023-11-08T16:29:57.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.532+0000 7f050affd640 1 -- 172.21.15.81:0/2166557006 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0504032d30 con 0x7f05140d17d0 2023-11-08T16:29:57.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.532+0000 7f051c716640 1 -- 172.21.15.81:0/2166557006 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05141f2610 con 0x7f05140d17d0 2023-11-08T16:29:57.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.534+0000 7f051c716640 1 -- 172.21.15.81:0/2166557006 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05141f2be0 con 0x7f05140d17d0 2023-11-08T16:29:57.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.535+0000 7f050affd640 1 -- 172.21.15.81:0/2166557006 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f050404e440 con 0x7f05140d17d0 2023-11-08T16:29:57.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.536+0000 7f050affd640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04f806ae70 0x7f04f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.536+0000 7f050affd640 1 -- 172.21.15.81:0/2166557006 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f05040ab780 con 0x7f05140d17d0 2023-11-08T16:29:57.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.536+0000 7f051a4b2640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04f806ae70 0x7f04f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:57.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.537+0000 7f051a4b2640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04f806ae70 0x7f04f806d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0510006fd0 tx=0x7f0510008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:57.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.544+0000 7f050affd640 1 -- 172.21.15.81:0/2166557006 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f05141f2be0 con 0x7f05140d17d0 2023-11-08T16:29:57.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.735+0000 7f051c716640 1 -- 172.21.15.81:0/2166557006 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05141f2be0 con 0x7f05140d17d0 2023-11-08T16:29:57.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.738+0000 7f050affd640 1 -- 172.21.15.81:0/2166557006 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f05141f2be0 con 0x7f05140d17d0 2023-11-08T16:29:57.741 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:57.742 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:57.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.740+0000 7f051c716640 1 -- 172.21.15.81:0/2166557006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04f806ae70 msgr2=0x7f04f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:57.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.741+0000 7f051c716640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04f806ae70 0x7f04f806d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0510006fd0 tx=0x7f0510008040 comp rx=0 tx=0).stop 2023-11-08T16:29:57.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.741+0000 7f051c716640 1 -- 172.21.15.81:0/2166557006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05140d17d0 msgr2=0x7f05140c7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:57.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.741+0000 7f051c716640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05140d17d0 0x7f05140c7d80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0504034040 tx=0x7f05040046c0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.741+0000 7f051c716640 1 -- 172.21.15.81:0/2166557006 shutdown_connections 2023-11-08T16:29:57.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.741+0000 7f051c716640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04f806ae70 0x7f04f806d330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.741+0000 7f051c716640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0514160700 0x7f05141f2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.741+0000 7f051c716640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05140d17d0 0x7f05140c7d80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.741+0000 7f051c716640 1 --2- 172.21.15.81:0/2166557006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05140cf050 0x7f05140cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.741+0000 7f051c716640 1 -- 172.21.15.81:0/2166557006 >> 172.21.15.81:0/2166557006 conn(0x7f05140d0cb0 msgr2=0x7f05140d1090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:57.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.741+0000 7f051c716640 1 -- 172.21.15.81:0/2166557006 shutdown_connections 2023-11-08T16:29:57.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.741+0000 7f051c716640 1 -- 172.21.15.81:0/2166557006 wait complete. 2023-11-08T16:29:57.763 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:29:57.764 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:29:57.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.974+0000 7f554770f640 1 Processor -- start 2023-11-08T16:29:57.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.974+0000 7f554770f640 1 -- start start 2023-11-08T16:29:57.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f554770f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f55400ce0d0 0x7f55400caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f554770f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f554015bfc0 0x7f554015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f554770f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554006fa30 con 0x7f554015f0b0 2023-11-08T16:29:57.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f554770f640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55400cda10 con 0x7f55400ce0d0 2023-11-08T16:29:57.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f554770f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55400ce4b0 con 0x7f554015bfc0 2023-11-08T16:29:57.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f55454ab640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f55400ce0d0 0x7f55400caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:57.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f55454ab640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f55400ce0d0 0x7f55400caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47044/0 (socket says 172.21.15.81:47044) 2023-11-08T16:29:57.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f55454ab640 1 -- 172.21.15.81:0/3310676238 learned_addr learned my addr 172.21.15.81:0/3310676238 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:57.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f5544caa640 1 --2- 172.21.15.81:0/3310676238 >> v2:172.21.15.93:3300/0 conn(0x7f554015bfc0 0x7f554015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:57.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f55454ab640 1 -- 172.21.15.81:0/3310676238 >> v2:172.21.15.93:3300/0 conn(0x7f554015bfc0 msgr2=0x7f554015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:57.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f55454ab640 1 --2- 172.21.15.81:0/3310676238 >> v2:172.21.15.93:3300/0 conn(0x7f554015bfc0 0x7f554015eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f55454ab640 1 -- 172.21.15.81:0/3310676238 >> v1:172.21.15.93:6789/0 conn(0x7f554015f0b0 legacy=0x7f55401516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:57.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.976+0000 7f55454ab640 1 -- 172.21.15.81:0/3310676238 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554015f490 con 0x7f55400ce0d0 2023-11-08T16:29:57.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.978+0000 7f55454ab640 1 --2- 172.21.15.81:0/3310676238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55400ce0d0 0x7f55400caf20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f55400ca780 tx=0x7f553c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e98649573a7ded server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:57.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.978+0000 7f5537fff640 1 -- 172.21.15.81:0/3310676238 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c031870 con 0x7f55400ce0d0 2023-11-08T16:29:57.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.978+0000 7f5537fff640 1 -- 172.21.15.81:0/3310676238 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f553c0319d0 con 0x7f55400ce0d0 2023-11-08T16:29:57.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.978+0000 7f5537fff640 1 -- 172.21.15.81:0/3310676238 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c031d60 con 0x7f55400ce0d0 2023-11-08T16:29:57.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.978+0000 7f554770f640 1 -- 172.21.15.81:0/3310676238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55400ce0d0 msgr2=0x7f55400caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:57.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.978+0000 7f554770f640 1 --2- 172.21.15.81:0/3310676238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55400ce0d0 0x7f55400caf20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f55400ca780 tx=0x7f553c02b910 comp rx=0 tx=0).stop 2023-11-08T16:29:57.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.978+0000 7f554770f640 1 -- 172.21.15.81:0/3310676238 shutdown_connections 2023-11-08T16:29:57.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.978+0000 7f554770f640 1 --2- 172.21.15.81:0/3310676238 >> v2:172.21.15.93:3300/0 conn(0x7f554015bfc0 0x7f554015eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.978+0000 7f554770f640 1 --2- 172.21.15.81:0/3310676238 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55400ce0d0 0x7f55400caf20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.978+0000 7f554770f640 1 -- 172.21.15.81:0/3310676238 >> 172.21.15.81:0/3310676238 conn(0x7f55400cb580 msgr2=0x7f55400cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:57.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.979+0000 7f554770f640 1 -- 172.21.15.81:0/3310676238 shutdown_connections 2023-11-08T16:29:57.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.979+0000 7f554770f640 1 -- 172.21.15.81:0/3310676238 wait complete. 2023-11-08T16:29:57.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.980+0000 7f554770f640 1 Processor -- start 2023-11-08T16:29:57.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.980+0000 7f554770f640 1 -- start start 2023-11-08T16:29:57.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.981+0000 7f554770f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55400ce0d0 0x7f55400c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.981+0000 7f554770f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f554015bfc0 0x7f554015dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.981+0000 7f554770f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f554015f0b0 0x7f5540152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.981+0000 7f554770f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55400c37e0 con 0x7f554015bfc0 2023-11-08T16:29:57.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.981+0000 7f554770f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55400c2b60 con 0x7f55400ce0d0 2023-11-08T16:29:57.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.981+0000 7f554770f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55400c2e40 con 0x7f554015f0b0 2023-11-08T16:29:57.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.981+0000 7f5544caa640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f554015bfc0 0x7f554015dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:57.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.982+0000 7f5544caa640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f554015bfc0 0x7f554015dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47060/0 (socket says 172.21.15.81:47060) 2023-11-08T16:29:57.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.982+0000 7f5544caa640 1 -- 172.21.15.81:0/2117891030 learned_addr learned my addr 172.21.15.81:0/2117891030 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:57.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.982+0000 7f5544caa640 1 -- 172.21.15.81:0/2117891030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f554015f0b0 msgr2=0x7f5540152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:57.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.982+0000 7f5544caa640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f554015f0b0 0x7f5540152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.982+0000 7f5544caa640 1 -- 172.21.15.81:0/2117891030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55400ce0d0 msgr2=0x7f55400c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:57.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.982+0000 7f5544caa640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55400ce0d0 0x7f55400c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:57.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.982+0000 7f5544caa640 1 -- 172.21.15.81:0/2117891030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553c0094d0 con 0x7f554015bfc0 2023-11-08T16:29:57.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.982+0000 7f5544caa640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f554015bfc0 0x7f554015dc40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f553000c7f0 tx=0x7f553000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:57.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.982+0000 7f5535ffb640 1 -- 172.21.15.81:0/2117891030 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530004220 con 0x7f554015bfc0 2023-11-08T16:29:57.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.982+0000 7f5535ffb640 1 -- 172.21.15.81:0/2117891030 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55300043a0 con 0x7f554015bfc0 2023-11-08T16:29:57.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.982+0000 7f5535ffb640 1 -- 172.21.15.81:0/2117891030 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5530007860 con 0x7f554015bfc0 2023-11-08T16:29:57.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.983+0000 7f554770f640 1 -- 172.21.15.81:0/2117891030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55400c3120 con 0x7f554015bfc0 2023-11-08T16:29:57.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.983+0000 7f554770f640 1 -- 172.21.15.81:0/2117891030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55400c52e0 con 0x7f554015bfc0 2023-11-08T16:29:57.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.984+0000 7f554770f640 1 -- 172.21.15.81:0/2117891030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55400c3920 con 0x7f554015bfc0 2023-11-08T16:29:57.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.985+0000 7f5535ffb640 1 -- 172.21.15.81:0/2117891030 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5530007bd0 con 0x7f554015bfc0 2023-11-08T16:29:57.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.985+0000 7f5535ffb640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f552406ae70 0x7f552406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:57.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.986+0000 7f5535ffb640 1 -- 172.21.15.81:0/2117891030 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5530089690 con 0x7f554015bfc0 2023-11-08T16:29:57.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.986+0000 7f55454ab640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f552406ae70 0x7f552406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:57.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.986+0000 7f55454ab640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f552406ae70 0x7f552406d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f553c036000 tx=0x7f553c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:57.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:57.990+0000 7f5535ffb640 1 -- 172.21.15.81:0/2117891030 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55300546a0 con 0x7f554015bfc0 2023-11-08T16:29:58.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.164+0000 7f554770f640 1 -- 172.21.15.81:0/2117891030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f554004ece0 con 0x7f554015bfc0 2023-11-08T16:29:58.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.165+0000 7f5535ffb640 1 -- 172.21.15.81:0/2117891030 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5530057cc0 con 0x7f554015bfc0 2023-11-08T16:29:58.168 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:29:58.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 -- 172.21.15.81:0/2117891030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f552406ae70 msgr2=0x7f552406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f552406ae70 0x7f552406d330 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f553c036000 tx=0x7f553c030000 comp rx=0 tx=0).stop 2023-11-08T16:29:58.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 -- 172.21.15.81:0/2117891030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f554015bfc0 msgr2=0x7f554015dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f554015bfc0 0x7f554015dc40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f553000c7f0 tx=0x7f553000ccb0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 -- 172.21.15.81:0/2117891030 shutdown_connections 2023-11-08T16:29:58.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f552406ae70 0x7f552406d330 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f554015f0b0 0x7f5540152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f554015bfc0 0x7f554015dc40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 --2- 172.21.15.81:0/2117891030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55400ce0d0 0x7f55400c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 -- 172.21.15.81:0/2117891030 >> 172.21.15.81:0/2117891030 conn(0x7f55400cb580 msgr2=0x7f55400cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:58.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 -- 172.21.15.81:0/2117891030 shutdown_connections 2023-11-08T16:29:58.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.168+0000 7f554770f640 1 -- 172.21.15.81:0/2117891030 wait complete. 2023-11-08T16:29:58.190 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:29:58.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.394+0000 7fcff52b4640 1 Processor -- start 2023-11-08T16:29:58.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.395+0000 7fcff52b4640 1 -- start start 2023-11-08T16:29:58.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.396+0000 7fcff52b4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcff00cfd90 0x7fcff00d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.396+0000 7fcff52b4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff00ce0b0 con 0x7fcff015cf30 2023-11-08T16:29:58.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.396+0000 7fcff52b4640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff00ce390 con 0x7fcff015d310 2023-11-08T16:29:58.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.396+0000 7fcff52b4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff00d1be0 con 0x7fcff00cfd90 2023-11-08T16:29:58.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.397+0000 7fcfeffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcff015cf30 0x7fcff00cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52752/0 (socket says 172.21.15.81:52752) 2023-11-08T16:29:58.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.397+0000 7fcfeffff640 1 -- 172.21.15.81:0/1282170106 learned_addr learned my addr 172.21.15.81:0/1282170106 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:58.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.397+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409572156 0 0) 0x7fcff00ce0b0 con 0x7fcff015cf30 2023-11-08T16:29:58.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.397+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfdc003520 con 0x7fcff015cf30 2023-11-08T16:29:58.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.397+0000 7fcff4ab3640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcff00cfd90 0x7fcff00d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:58.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.397+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3553289776 0 0) 0x7fcfdc003520 con 0x7fcff015cf30 2023-11-08T16:29:58.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcff00cfd90 msgr2=0x7fcff00d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:58.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcff00cfd90 0x7fcff00d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 >> v1:172.21.15.93:6789/0 conn(0x7fcff015d310 legacy=0x7fcff00b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:58.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff00d1ec0 con 0x7fcff015cf30 2023-11-08T16:29:58.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fcfe00019f0 con 0x7fcff015cf30 2023-11-08T16:29:58.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 >> v1:172.21.15.81:6789/0 conn(0x7fcff015cf30 legacy=0x7fcff00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfdc0039a0 0x7fcfdc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfdc0047c0 0x7fcfdc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcff00cfd90 0x7fcfdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff00d1be0 con 0x7fcfdc0047c0 2023-11-08T16:29:58.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc00bae0 con 0x7fcfdc0039a0 2023-11-08T16:29:58.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc00bca0 con 0x7fcff00cfd90 2023-11-08T16:29:58.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfef7fe640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfdc0047c0 0x7fcfdc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:58.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfef7fe640 1 -- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcff00cfd90 msgr2=0x7fcfdc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:58.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcff4ab3640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcff00cfd90 0x7fcfdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:58.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfeffff640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfdc0039a0 0x7fcfdc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:58.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfef7fe640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcff00cfd90 0x7fcfdc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.398+0000 7fcfef7fe640 1 -- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfdc0039a0 msgr2=0x7fcfdc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:58.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.399+0000 7fcfef7fe640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfdc0039a0 0x7fcfdc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.399+0000 7fcfef7fe640 1 -- 172.21.15.81:0/1282170106 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc003520 con 0x7fcfdc0047c0 2023-11-08T16:29:58.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.399+0000 7fcfeffff640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcfdc0039a0 0x7fcfdc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:58.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.400+0000 7fcfef7fe640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfdc0047c0 0x7fcfdc004c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcff00cd2b0 tx=0x7fcfdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d042dbb66a4f02 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:58.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.400+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80052d0 con 0x7fcfdc0047c0 2023-11-08T16:29:58.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.400+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd802e050 con 0x7fcfdc0047c0 2023-11-08T16:29:58.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.400+0000 7fcfeeffd640 1 -- 172.21.15.81:0/1282170106 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80035f0 con 0x7fcfdc0047c0 2023-11-08T16:29:58.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.400+0000 7fcff52b4640 1 -- 172.21.15.81:0/1282170106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfdc0047c0 msgr2=0x7fcfdc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.400+0000 7fcff52b4640 1 --2- 172.21.15.81:0/1282170106 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcfdc0047c0 0x7fcfdc004c40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcff00cd2b0 tx=0x7fcfdc0057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.400+0000 7fcff4ab3640 1 -- 172.21.15.81:0/1282170106 reap_dead start 2023-11-08T16:29:58.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.400+0000 7fcff52b4640 1 -- 172.21.15.81:0/1282170106 shutdown_connections 2023-11-08T16:29:58.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.400+0000 7fcff52b4640 1 -- 172.21.15.81:0/1282170106 >> 172.21.15.81:0/1282170106 conn(0x7fcff006c040 msgr2=0x7fcff006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:58.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.401+0000 7fcff52b4640 1 -- 172.21.15.81:0/1282170106 shutdown_connections 2023-11-08T16:29:58.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.401+0000 7fcff52b4640 1 -- 172.21.15.81:0/1282170106 wait complete. 2023-11-08T16:29:58.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.402+0000 7fcff52b4640 1 Processor -- start 2023-11-08T16:29:58.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.402+0000 7fcff52b4640 1 -- start start 2023-11-08T16:29:58.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.403+0000 7fcff52b4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcfdc0047c0 0x7fcff0144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.403+0000 7fcff52b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcff01474d0 0x7fcff0147930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.403+0000 7fcff52b4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcff0147e70 0x7fcff00b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.403+0000 7fcff52b4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff01e5860 con 0x7fcff01474d0 2023-11-08T16:29:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.403+0000 7fcff52b4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0145440 con 0x7fcff0147e70 2023-11-08T16:29:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.403+0000 7fcff52b4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff0145160 con 0x7fcfdc0047c0 2023-11-08T16:29:58.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.404+0000 7fcfef7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcff01474d0 0x7fcff0147930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:58.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.404+0000 7fcfef7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcff01474d0 0x7fcff0147930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47068/0 (socket says 172.21.15.81:47068) 2023-11-08T16:29:58.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.404+0000 7fcfef7fe640 1 -- 172.21.15.81:0/1115269817 learned_addr learned my addr 172.21.15.81:0/1115269817 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:58.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.404+0000 7fcff4ab3640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcff0147e70 0x7fcff00b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:58.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.404+0000 7fcfeffff640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcfdc0047c0 0x7fcff0144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.404+0000 7fcfef7fe640 1 -- 172.21.15.81:0/1115269817 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcfdc0047c0 msgr2=0x7fcff0144c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.404+0000 7fcfef7fe640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcfdc0047c0 0x7fcff0144c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.404+0000 7fcfef7fe640 1 -- 172.21.15.81:0/1115269817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcff0147e70 msgr2=0x7fcff00b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.404+0000 7fcfef7fe640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcff0147e70 0x7fcff00b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.404+0000 7fcfef7fe640 1 -- 172.21.15.81:0/1115269817 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff01f26f0 con 0x7fcff01474d0 2023-11-08T16:29:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.405+0000 7fcfef7fe640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcff01474d0 0x7fcff0147930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcfd802bbb0 tx=0x7fcfd8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.405+0000 7fcfecff9640 1 -- 172.21.15.81:0/1115269817 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd80076c0 con 0x7fcff01474d0 2023-11-08T16:29:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.405+0000 7fcfecff9640 1 -- 172.21.15.81:0/1115269817 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd8007a30 con 0x7fcff01474d0 2023-11-08T16:29:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.405+0000 7fcfecff9640 1 -- 172.21.15.81:0/1115269817 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8006be0 con 0x7fcff01474d0 2023-11-08T16:29:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.405+0000 7fcff52b4640 1 -- 172.21.15.81:0/1115269817 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff00b4990 con 0x7fcff01474d0 2023-11-08T16:29:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.405+0000 7fcff52b4640 1 -- 172.21.15.81:0/1115269817 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff0145650 con 0x7fcff01474d0 2023-11-08T16:29:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.406+0000 7fcff52b4640 1 -- 172.21.15.81:0/1115269817 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff004ece0 con 0x7fcff01474d0 2023-11-08T16:29:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.407+0000 7fcfecff9640 1 -- 172.21.15.81:0/1115269817 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcfd8006d40 con 0x7fcff01474d0 2023-11-08T16:29:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.408+0000 7fcfecff9640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcfd006ae70 0x7fcfd006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.408+0000 7fcfecff9640 1 -- 172.21.15.81:0/1115269817 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fcfd80ab920 con 0x7fcff01474d0 2023-11-08T16:29:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.408+0000 7fcfeffff640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcfd006ae70 0x7fcfd006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.409+0000 7fcfeffff640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcfd006ae70 0x7fcfd006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00013d0 tx=0x7fcfe0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.412+0000 7fcfecff9640 1 -- 172.21.15.81:0/1115269817 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcfd8076810 con 0x7fcff01474d0 2023-11-08T16:29:58.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.600+0000 7fcff52b4640 1 -- 172.21.15.81:0/1115269817 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcff00b4eb0 con 0x7fcff01474d0 2023-11-08T16:29:58.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.601+0000 7fcfecff9640 1 -- 172.21.15.81:0/1115269817 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcff00b4eb0 con 0x7fcff01474d0 2023-11-08T16:29:58.604 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:29:58.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 -- 172.21.15.81:0/1115269817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcfd006ae70 msgr2=0x7fcfd006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcfd006ae70 0x7fcfd006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcfe00013d0 tx=0x7fcfe0009040 comp rx=0 tx=0).stop 2023-11-08T16:29:58.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 -- 172.21.15.81:0/1115269817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcff01474d0 msgr2=0x7fcff0147930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcff01474d0 0x7fcff0147930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcfd802bbb0 tx=0x7fcfd8001120 comp rx=0 tx=0).stop 2023-11-08T16:29:58.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 -- 172.21.15.81:0/1115269817 shutdown_connections 2023-11-08T16:29:58.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcfd006ae70 0x7fcfd006d330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcff0147e70 0x7fcff00b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcff01474d0 0x7fcff0147930 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 --2- 172.21.15.81:0/1115269817 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcfdc0047c0 0x7fcff0144c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 -- 172.21.15.81:0/1115269817 >> 172.21.15.81:0/1115269817 conn(0x7fcff006c040 msgr2=0x7fcff006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:58.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 -- 172.21.15.81:0/1115269817 shutdown_connections 2023-11-08T16:29:58.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.604+0000 7fcff52b4640 1 -- 172.21.15.81:0/1115269817 wait complete. 2023-11-08T16:29:58.625 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:29:58.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.834+0000 7fce85f9e640 1 Processor -- start 2023-11-08T16:29:58.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.835+0000 7fce85f9e640 1 -- start start 2023-11-08T16:29:58.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.836+0000 7fce85f9e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fce800cfb20 0x7fce800b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.836+0000 7fce85f9e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce800d38c0 con 0x7fce800b3560 2023-11-08T16:29:58.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.836+0000 7fce85f9e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce800ce4c0 con 0x7fce800cf740 2023-11-08T16:29:58.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.836+0000 7fce85f9e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce800c7900 con 0x7fce800cfb20 2023-11-08T16:29:58.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.836+0000 7fce7ffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fce800b3560 0x7fce800d0b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52768/0 (socket says 172.21.15.81:52768) 2023-11-08T16:29:58.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.836+0000 7fce7ffff640 1 -- 172.21.15.81:0/2951601727 learned_addr learned my addr 172.21.15.81:0/2951601727 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:58.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.837+0000 7fce8579d640 1 --2- 172.21.15.81:0/2951601727 >> v2:172.21.15.81:3300/0 conn(0x7fce800cfb20 0x7fce800b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:58.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.837+0000 7fce7f7fe640 1 -- 172.21.15.81:0/2951601727 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321439339 0 0) 0x7fce800d38c0 con 0x7fce800b3560 2023-11-08T16:29:58.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.837+0000 7fce7f7fe640 1 -- 172.21.15.81:0/2951601727 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce70003520 con 0x7fce800b3560 2023-11-08T16:29:58.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.837+0000 7fce8579d640 1 -- 172.21.15.81:0/2951601727 >> v1:172.21.15.93:6789/0 conn(0x7fce800cf740 legacy=0x7fce800b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:58.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.837+0000 7fce8579d640 1 -- 172.21.15.81:0/2951601727 >> v1:172.21.15.81:6789/0 conn(0x7fce800b3560 legacy=0x7fce800d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.837+0000 7fce8579d640 1 -- 172.21.15.81:0/2951601727 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce800ce7a0 con 0x7fce800cfb20 2023-11-08T16:29:58.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.839+0000 7fce8579d640 1 --2- 172.21.15.81:0/2951601727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce800cfb20 0x7fce800b3e00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fce78009d90 tx=0x7fce7802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570456915f8eda6a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:58.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.839+0000 7fce7f7fe640 1 -- 172.21.15.81:0/2951601727 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7802cc60 con 0x7fce800cfb20 2023-11-08T16:29:58.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.839+0000 7fce7f7fe640 1 -- 172.21.15.81:0/2951601727 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7802cdc0 con 0x7fce800cfb20 2023-11-08T16:29:58.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.839+0000 7fce7f7fe640 1 -- 172.21.15.81:0/2951601727 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce78034660 con 0x7fce800cfb20 2023-11-08T16:29:58.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.839+0000 7fce85f9e640 1 -- 172.21.15.81:0/2951601727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce800cfb20 msgr2=0x7fce800b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.839+0000 7fce85f9e640 1 --2- 172.21.15.81:0/2951601727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce800cfb20 0x7fce800b3e00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fce78009d90 tx=0x7fce7802beb0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.839+0000 7fce85f9e640 1 -- 172.21.15.81:0/2951601727 shutdown_connections 2023-11-08T16:29:58.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.839+0000 7fce85f9e640 1 --2- 172.21.15.81:0/2951601727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce800cfb20 0x7fce800b3e00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.840+0000 7fce85f9e640 1 -- 172.21.15.81:0/2951601727 >> 172.21.15.81:0/2951601727 conn(0x7fce800d2da0 msgr2=0x7fce800d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:58.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.840+0000 7fce85f9e640 1 -- 172.21.15.81:0/2951601727 shutdown_connections 2023-11-08T16:29:58.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.840+0000 7fce85f9e640 1 -- 172.21.15.81:0/2951601727 wait complete. 2023-11-08T16:29:58.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.841+0000 7fce85f9e640 1 Processor -- start 2023-11-08T16:29:58.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.841+0000 7fce85f9e640 1 -- start start 2023-11-08T16:29:58.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.842+0000 7fce85f9e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce800b3560 0x7fce801f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.842+0000 7fce85f9e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fce800cf740 0x7fce80155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.842+0000 7fce85f9e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce800cfb20 0x7fce8015e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.842+0000 7fce85f9e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8015a0c0 con 0x7fce800b3560 2023-11-08T16:29:58.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.842+0000 7fce85f9e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce801f3760 con 0x7fce800cfb20 2023-11-08T16:29:58.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.842+0000 7fce85f9e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8015a2d0 con 0x7fce800cf740 2023-11-08T16:29:58.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.843+0000 7fce84f9c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce800b3560 0x7fce801f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:58.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.843+0000 7fce84f9c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce800b3560 0x7fce801f3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47096/0 (socket says 172.21.15.81:47096) 2023-11-08T16:29:58.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.843+0000 7fce84f9c640 1 -- 172.21.15.81:0/2979890344 learned_addr learned my addr 172.21.15.81:0/2979890344 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:58.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.843+0000 7fce84f9c640 1 -- 172.21.15.81:0/2979890344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fce800cf740 msgr2=0x7fce80155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.843+0000 7fce84f9c640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fce800cf740 0x7fce80155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.843+0000 7fce84f9c640 1 -- 172.21.15.81:0/2979890344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce800cfb20 msgr2=0x7fce8015e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:58.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.843+0000 7fce84f9c640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce800cfb20 0x7fce8015e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:58.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.843+0000 7fce84f9c640 1 -- 172.21.15.81:0/2979890344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce78009a70 con 0x7fce800b3560 2023-11-08T16:29:58.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.843+0000 7fce84f9c640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce800b3560 0x7fce801f3220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce7400c400 tx=0x7fce7400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:58.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.844+0000 7fce7d7fa640 1 -- 172.21.15.81:0/2979890344 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce740052d0 con 0x7fce800b3560 2023-11-08T16:29:58.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.844+0000 7fce7d7fa640 1 -- 172.21.15.81:0/2979890344 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce74005430 con 0x7fce800b3560 2023-11-08T16:29:58.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.844+0000 7fce7d7fa640 1 -- 172.21.15.81:0/2979890344 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74007260 con 0x7fce800b3560 2023-11-08T16:29:58.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.844+0000 7fce85f9e640 1 -- 172.21.15.81:0/2979890344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8015a5b0 con 0x7fce800b3560 2023-11-08T16:29:58.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.844+0000 7fce85f9e640 1 -- 172.21.15.81:0/2979890344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8015ab70 con 0x7fce800b3560 2023-11-08T16:29:58.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.845+0000 7fce85f9e640 1 -- 172.21.15.81:0/2979890344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce80067cb0 con 0x7fce800b3560 2023-11-08T16:29:58.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.846+0000 7fce7d7fa640 1 -- 172.21.15.81:0/2979890344 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce740073c0 con 0x7fce800b3560 2023-11-08T16:29:58.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.847+0000 7fce7d7fa640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce6406aea0 0x7fce6406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:58.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.847+0000 7fce7d7fa640 1 -- 172.21.15.81:0/2979890344 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fce74088ae0 con 0x7fce800b3560 2023-11-08T16:29:58.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.847+0000 7fce7ffff640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce6406aea0 0x7fce6406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:58.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.848+0000 7fce7ffff640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce6406aea0 0x7fce6406d360 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fce80156af0 tx=0x7fce6c007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:58.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:58.851+0000 7fce7d7fa640 1 -- 172.21.15.81:0/2979890344 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce74053a60 con 0x7fce800b3560 2023-11-08T16:29:59.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.029+0000 7fce85f9e640 1 -- 172.21.15.81:0/2979890344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce8004ece0 con 0x7fce800b3560 2023-11-08T16:29:59.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.029+0000 7fce7d7fa640 1 -- 172.21.15.81:0/2979890344 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce74057080 con 0x7fce800b3560 2023-11-08T16:29:59.032 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:29:59.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 -- 172.21.15.81:0/2979890344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce6406aea0 msgr2=0x7fce6406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce6406aea0 0x7fce6406d360 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fce80156af0 tx=0x7fce6c007930 comp rx=0 tx=0).stop 2023-11-08T16:29:59.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 -- 172.21.15.81:0/2979890344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce800b3560 msgr2=0x7fce801f3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce800b3560 0x7fce801f3220 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fce7400c400 tx=0x7fce7400c8c0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 -- 172.21.15.81:0/2979890344 shutdown_connections 2023-11-08T16:29:59.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce6406aea0 0x7fce6406d360 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce800cfb20 0x7fce8015e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fce800cf740 0x7fce80155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 --2- 172.21.15.81:0/2979890344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce800b3560 0x7fce801f3220 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 -- 172.21.15.81:0/2979890344 >> 172.21.15.81:0/2979890344 conn(0x7fce800d2da0 msgr2=0x7fce80160410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:59.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 -- 172.21.15.81:0/2979890344 shutdown_connections 2023-11-08T16:29:59.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.032+0000 7fce85f9e640 1 -- 172.21.15.81:0/2979890344 wait complete. 2023-11-08T16:29:59.053 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:29:59.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.260+0000 7fcaabae7640 1 Processor -- start 2023-11-08T16:29:59.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.261+0000 7fcaabae7640 1 -- start start 2023-11-08T16:29:59.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.262+0000 7fcaabae7640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcaa40c90f0 0x7fcaa40c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.262+0000 7fcaabae7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa406cc60 con 0x7fcaa415aeb0 2023-11-08T16:29:59.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.262+0000 7fcaabae7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa40cabb0 con 0x7fcaa40b18f0 2023-11-08T16:29:59.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.262+0000 7fcaabae7640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40cb3e0 con 0x7fcaa40c90f0 2023-11-08T16:29:59.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.262+0000 7fcaa9883640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcaa415aeb0 0x7fcaa40ca4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52776/0 (socket says 172.21.15.81:52776) 2023-11-08T16:29:59.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.262+0000 7fcaa9883640 1 -- 172.21.15.81:0/1669742126 learned_addr learned my addr 172.21.15.81:0/1669742126 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:29:59.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1746662388 0 0) 0x7fcaa406cc60 con 0x7fcaa415aeb0 2023-11-08T16:29:59.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa9082640 1 --2- 172.21.15.81:0/1669742126 >> v2:172.21.15.93:3300/0 conn(0x7fcaa40c90f0 0x7fcaa40c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:59.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca8c003520 con 0x7fcaa415aeb0 2023-11-08T16:29:59.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1502948894 0 0) 0x7fca8c003520 con 0x7fcaa415aeb0 2023-11-08T16:29:59.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 >> v2:172.21.15.93:3300/0 conn(0x7fcaa40c90f0 msgr2=0x7fcaa40c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:59.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 --2- 172.21.15.81:0/1669742126 >> v2:172.21.15.93:3300/0 conn(0x7fcaa40c90f0 0x7fcaa40c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 >> v1:172.21.15.93:6789/0 conn(0x7fcaa40b18f0 legacy=0x7fcaa40b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:59.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40cae20 con 0x7fcaa415aeb0 2023-11-08T16:29:59.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fca94001b70 con 0x7fcaa415aeb0 2023-11-08T16:29:59.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 >> v1:172.21.15.81:6789/0 conn(0x7fcaa415aeb0 legacy=0x7fcaa40ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 --2- 172.21.15.81:0/1669742126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca8c003970 0x7fca8c003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 --2- 172.21.15.81:0/1669742126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 --2- 172.21.15.81:0/1669742126 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca8c005180 0x7fca8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40cb3e0 con 0x7fca8c0047c0 2023-11-08T16:29:59.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c00c0b0 con 0x7fca8c003970 2023-11-08T16:29:59.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.263+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8c00c270 con 0x7fca8c005180 2023-11-08T16:29:59.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.264+0000 7fcaa9082640 1 --2- 172.21.15.81:0/1669742126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:59.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.264+0000 7fcaa9883640 1 --2- 172.21.15.81:0/1669742126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca8c003970 0x7fca8c003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:59.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.264+0000 7fcaa9082640 1 -- 172.21.15.81:0/1669742126 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca8c005180 msgr2=0x7fca8c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:59.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.264+0000 7fcaa9082640 1 --2- 172.21.15.81:0/1669742126 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca8c005180 0x7fca8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.264+0000 7fcaa9082640 1 -- 172.21.15.81:0/1669742126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca8c003970 msgr2=0x7fca8c003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:59.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.264+0000 7fcaa9082640 1 --2- 172.21.15.81:0/1669742126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca8c003970 0x7fca8c003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.264+0000 7fcaa9082640 1 -- 172.21.15.81:0/1669742126 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca8c003520 con 0x7fca8c0047c0 2023-11-08T16:29:59.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.264+0000 7fcaaa084640 1 -- 172.21.15.81:0/1669742126 reap_dead start 2023-11-08T16:29:59.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.264+0000 7fcaa9883640 1 --2- 172.21.15.81:0/1669742126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca8c003970 0x7fca8c003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:59.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.265+0000 7fcaa9082640 1 --2- 172.21.15.81:0/1669742126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fca9800ba00 tx=0x7fca8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38e3eb87139f453 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:59.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.265+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98032620 con 0x7fca8c0047c0 2023-11-08T16:29:59.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.265+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca98032780 con 0x7fca8c0047c0 2023-11-08T16:29:59.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.265+0000 7fcaa8881640 1 -- 172.21.15.81:0/1669742126 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca98032b10 con 0x7fca8c0047c0 2023-11-08T16:29:59.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.266+0000 7fcaabae7640 1 -- 172.21.15.81:0/1669742126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca8c0047c0 msgr2=0x7fca8c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.266+0000 7fcaabae7640 1 --2- 172.21.15.81:0/1669742126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fca9800ba00 tx=0x7fca8c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.266+0000 7fcaabae7640 1 -- 172.21.15.81:0/1669742126 shutdown_connections 2023-11-08T16:29:59.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.266+0000 7fcaabae7640 1 --2- 172.21.15.81:0/1669742126 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca8c0047c0 0x7fca8c004c40 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.266+0000 7fcaabae7640 1 -- 172.21.15.81:0/1669742126 >> 172.21.15.81:0/1669742126 conn(0x7fcaa40ceaa0 msgr2=0x7fcaa40cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:59.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.266+0000 7fcaabae7640 1 -- 172.21.15.81:0/1669742126 shutdown_connections 2023-11-08T16:29:59.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.266+0000 7fcaabae7640 1 -- 172.21.15.81:0/1669742126 wait complete. 2023-11-08T16:29:59.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.268+0000 7fcaabae7640 1 Processor -- start 2023-11-08T16:29:59.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.268+0000 7fcaabae7640 1 -- start start 2023-11-08T16:29:59.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.269+0000 7fcaabae7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca8c0047c0 0x7fcaa41f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.269+0000 7fcaabae7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcaa40c25d0 0x7fcaa415d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.269+0000 7fcaabae7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcaa415dc50 0x7fcaa40af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.269+0000 7fcaabae7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa415e0d0 con 0x7fcaa40c25d0 2023-11-08T16:29:59.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.269+0000 7fcaabae7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40c2d70 con 0x7fca8c0047c0 2023-11-08T16:29:59.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.269+0000 7fcaabae7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40c3050 con 0x7fcaa415dc50 2023-11-08T16:29:59.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.269+0000 7fcaa9082640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcaa40c25d0 0x7fcaa415d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:59.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.269+0000 7fcaa9082640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcaa40c25d0 0x7fcaa415d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47116/0 (socket says 172.21.15.81:47116) 2023-11-08T16:29:59.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.269+0000 7fcaa9082640 1 -- 172.21.15.81:0/1716489186 learned_addr learned my addr 172.21.15.81:0/1716489186 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:59.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.270+0000 7fcaa9082640 1 -- 172.21.15.81:0/1716489186 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcaa415dc50 msgr2=0x7fcaa40af690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.270+0000 7fcaa9082640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcaa415dc50 0x7fcaa40af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.270+0000 7fcaa9082640 1 -- 172.21.15.81:0/1716489186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca8c0047c0 msgr2=0x7fcaa41f48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.270+0000 7fcaa9082640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca8c0047c0 0x7fcaa41f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.270+0000 7fcaa9082640 1 -- 172.21.15.81:0/1716489186 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40c3330 con 0x7fcaa40c25d0 2023-11-08T16:29:59.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.270+0000 7fcaa9082640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcaa40c25d0 0x7fcaa415d710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fca98035040 tx=0x7fca98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:59.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.270+0000 7fcaa27fc640 1 -- 172.21.15.81:0/1716489186 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9803b620 con 0x7fcaa40c25d0 2023-11-08T16:29:59.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.270+0000 7fcaa27fc640 1 -- 172.21.15.81:0/1716489186 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9803b7a0 con 0x7fcaa40c25d0 2023-11-08T16:29:59.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.270+0000 7fcaa27fc640 1 -- 172.21.15.81:0/1716489186 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9803bb50 con 0x7fcaa40c25d0 2023-11-08T16:29:59.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.270+0000 7fcaabae7640 1 -- 172.21.15.81:0/1716489186 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa40c3610 con 0x7fcaa40c25d0 2023-11-08T16:29:59.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.270+0000 7fcaabae7640 1 -- 172.21.15.81:0/1716489186 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa40c2a50 con 0x7fcaa40c25d0 2023-11-08T16:29:59.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.271+0000 7fcaabae7640 1 -- 172.21.15.81:0/1716489186 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa40743f0 con 0x7fcaa40c25d0 2023-11-08T16:29:59.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.272+0000 7fcaa27fc640 1 -- 172.21.15.81:0/1716489186 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca9803bcb0 con 0x7fcaa40c25d0 2023-11-08T16:29:59.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.273+0000 7fcaa27fc640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca9006ae70 0x7fca9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.273+0000 7fcaa27fc640 1 -- 172.21.15.81:0/1716489186 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fca980ab760 con 0x7fcaa40c25d0 2023-11-08T16:29:59.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.274+0000 7fcaa9883640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca9006ae70 0x7fca9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:59.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.274+0000 7fcaa9883640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca9006ae70 0x7fca9006d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fca94001270 tx=0x7fca94009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:59.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.278+0000 7fcaa27fc640 1 -- 172.21.15.81:0/1716489186 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca98076810 con 0x7fcaa40c25d0 2023-11-08T16:29:59.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.480+0000 7fcaabae7640 1 -- 172.21.15.81:0/1716489186 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcaa4159570 con 0x7fcaa40c25d0 2023-11-08T16:29:59.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.481+0000 7fcaa27fc640 1 -- 172.21.15.81:0/1716489186 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7fca98079e30 con 0x7fcaa40c25d0 2023-11-08T16:29:59.484 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:29:59.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 -- 172.21.15.81:0/1716489186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca9006ae70 msgr2=0x7fca9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca9006ae70 0x7fca9006d330 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fca94001270 tx=0x7fca94009040 comp rx=0 tx=0).stop 2023-11-08T16:29:59.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 -- 172.21.15.81:0/1716489186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcaa40c25d0 msgr2=0x7fcaa415d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcaa40c25d0 0x7fcaa415d710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fca98035040 tx=0x7fca98004730 comp rx=0 tx=0).stop 2023-11-08T16:29:59.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 -- 172.21.15.81:0/1716489186 shutdown_connections 2023-11-08T16:29:59.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca9006ae70 0x7fca9006d330 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcaa415dc50 0x7fcaa40af690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcaa40c25d0 0x7fcaa415d710 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 --2- 172.21.15.81:0/1716489186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca8c0047c0 0x7fcaa41f48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 -- 172.21.15.81:0/1716489186 >> 172.21.15.81:0/1716489186 conn(0x7fcaa40ceaa0 msgr2=0x7fcaa40cba70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:59.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 -- 172.21.15.81:0/1716489186 shutdown_connections 2023-11-08T16:29:59.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.484+0000 7fcaabae7640 1 -- 172.21.15.81:0/1716489186 wait complete. 2023-11-08T16:29:59.507 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:29:59.507 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:29:59.565 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:29:59.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.779+0000 7f879ad25640 1 Processor -- start 2023-11-08T16:29:59.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.779+0000 7f879ad25640 1 -- start start 2023-11-08T16:29:59.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.781+0000 7f879ad25640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f879406e1b0 0x7f87940abd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.781+0000 7f879ad25640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87940ac2a0 0x7f87940cd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.781+0000 7f879ad25640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87940ae180 con 0x7f87940cda00 2023-11-08T16:29:59.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.781+0000 7f879ad25640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87940ac720 con 0x7f879406e1b0 2023-11-08T16:29:59.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.781+0000 7f879ad25640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87940cde90 con 0x7f87940ac2a0 2023-11-08T16:29:59.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.781+0000 7f8798ac1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f879406e1b0 0x7f87940abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:59.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.781+0000 7f8798ac1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f879406e1b0 0x7f87940abd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47128/0 (socket says 172.21.15.81:47128) 2023-11-08T16:29:59.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.781+0000 7f8798ac1640 1 -- 172.21.15.81:0/3500264523 learned_addr learned my addr 172.21.15.81:0/3500264523 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:59.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.781+0000 7f8793fff640 1 --2- 172.21.15.81:0/3500264523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87940ac2a0 0x7f87940cd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:59.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.782+0000 7f8798ac1640 1 -- 172.21.15.81:0/3500264523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87940ac2a0 msgr2=0x7f87940cd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:59.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.782+0000 7f8798ac1640 1 --2- 172.21.15.81:0/3500264523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87940ac2a0 0x7f87940cd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.782+0000 7f8798ac1640 1 -- 172.21.15.81:0/3500264523 >> v1:172.21.15.93:6789/0 conn(0x7f87940cda00 legacy=0x7f879415e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:29:59.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.782+0000 7f8798ac1640 1 -- 172.21.15.81:0/3500264523 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87940705d0 con 0x7f879406e1b0 2023-11-08T16:29:59.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.782+0000 7f8793fff640 1 --2- 172.21.15.81:0/3500264523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87940ac2a0 0x7f87940cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:59.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.783+0000 7f8798ac1640 1 --2- 172.21.15.81:0/3500264523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f879406e1b0 0x7f87940abd60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8788009870 tx=0x7f878802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bc47458ae5d59f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:59.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.783+0000 7f87937fe640 1 -- 172.21.15.81:0/3500264523 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788031640 con 0x7f879406e1b0 2023-11-08T16:29:59.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.783+0000 7f87937fe640 1 -- 172.21.15.81:0/3500264523 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87880317a0 con 0x7f879406e1b0 2023-11-08T16:29:59.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.783+0000 7f87937fe640 1 -- 172.21.15.81:0/3500264523 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788031b30 con 0x7f879406e1b0 2023-11-08T16:29:59.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.784+0000 7f879ad25640 1 -- 172.21.15.81:0/3500264523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f879406e1b0 msgr2=0x7f87940abd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.784+0000 7f879ad25640 1 --2- 172.21.15.81:0/3500264523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f879406e1b0 0x7f87940abd60 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8788009870 tx=0x7f878802b8f0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.784+0000 7f879ad25640 1 -- 172.21.15.81:0/3500264523 shutdown_connections 2023-11-08T16:29:59.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.784+0000 7f879ad25640 1 --2- 172.21.15.81:0/3500264523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87940ac2a0 0x7f87940cd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.784+0000 7f879ad25640 1 --2- 172.21.15.81:0/3500264523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f879406e1b0 0x7f87940abd60 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.784+0000 7f879ad25640 1 -- 172.21.15.81:0/3500264523 >> 172.21.15.81:0/3500264523 conn(0x7f87940ceee0 msgr2=0x7f87940cf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:29:59.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.784+0000 7f879ad25640 1 -- 172.21.15.81:0/3500264523 shutdown_connections 2023-11-08T16:29:59.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.784+0000 7f879ad25640 1 -- 172.21.15.81:0/3500264523 wait complete. 2023-11-08T16:29:59.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.786+0000 7f879ad25640 1 Processor -- start 2023-11-08T16:29:59.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.786+0000 7f879ad25640 1 -- start start 2023-11-08T16:29:59.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.787+0000 7f879ad25640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f879406e1b0 0x7f87940c5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.787+0000 7f879ad25640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87940ac2a0 0x7f87941587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.787+0000 7f879ad25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87940cda00 0x7f87940c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.787+0000 7f879ad25640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879409c650 con 0x7f87940cda00 2023-11-08T16:29:59.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.787+0000 7f879ad25640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87940b0510 con 0x7f879406e1b0 2023-11-08T16:29:59.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.787+0000 7f879ad25640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87940b07f0 con 0x7f87940ac2a0 2023-11-08T16:29:59.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.787+0000 7f87992c2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87940cda00 0x7f87940c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:59.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.787+0000 7f87992c2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87940cda00 0x7f87940c79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47136/0 (socket says 172.21.15.81:47136) 2023-11-08T16:29:59.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.787+0000 7f87992c2640 1 -- 172.21.15.81:0/1292775343 learned_addr learned my addr 172.21.15.81:0/1292775343 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:29:59.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f8793fff640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87940ac2a0 0x7f87941587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:59.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f8798ac1640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f879406e1b0 0x7f87940c5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:59.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f87992c2640 1 -- 172.21.15.81:0/1292775343 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87940ac2a0 msgr2=0x7f87941587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f87992c2640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87940ac2a0 0x7f87941587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f87992c2640 1 -- 172.21.15.81:0/1292775343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f879406e1b0 msgr2=0x7f87940c5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f87992c2640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f879406e1b0 0x7f87940c5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f87992c2640 1 -- 172.21.15.81:0/1292775343 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87880094d0 con 0x7f87940cda00 2023-11-08T16:29:59.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f8798ac1640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f879406e1b0 0x7f87940c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:29:59.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f87992c2640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87940cda00 0x7f87940c79c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f878c00c780 tx=0x7f878c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:59.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f87917fa640 1 -- 172.21.15.81:0/1292775343 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c004b60 con 0x7f87940cda00 2023-11-08T16:29:59.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f87917fa640 1 -- 172.21.15.81:0/1292775343 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c00ce90 con 0x7f87940cda00 2023-11-08T16:29:59.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f879ad25640 1 -- 172.21.15.81:0/1292775343 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87940b0ad0 con 0x7f87940cda00 2023-11-08T16:29:59.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f87917fa640 1 -- 172.21.15.81:0/1292775343 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c00a850 con 0x7f87940cda00 2023-11-08T16:29:59.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.788+0000 7f879ad25640 1 -- 172.21.15.81:0/1292775343 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8794159800 con 0x7f87940cda00 2023-11-08T16:29:59.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.790+0000 7f879ad25640 1 -- 172.21.15.81:0/1292775343 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879404ece0 con 0x7f87940cda00 2023-11-08T16:29:59.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.791+0000 7f87917fa640 1 -- 172.21.15.81:0/1292775343 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f878c00a9b0 con 0x7f87940cda00 2023-11-08T16:29:59.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.792+0000 7f87917fa640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f878006ae70 0x7f878006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:29:59.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.792+0000 7f87917fa640 1 -- 172.21.15.81:0/1292775343 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f878c0898b0 con 0x7f87940cda00 2023-11-08T16:29:59.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.792+0000 7f8798ac1640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f878006ae70 0x7f878006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:29:59.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.793+0000 7f8798ac1640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f878006ae70 0x7f878006d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8788036000 tx=0x7f878803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:29:59.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.796+0000 7f87917fa640 1 -- 172.21.15.81:0/1292775343 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f878c054830 con 0x7f87940cda00 2023-11-08T16:29:59.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.988+0000 7f879ad25640 1 -- 172.21.15.81:0/1292775343 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87940cea60 con 0x7f87940cda00 2023-11-08T16:29:59.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.991+0000 7f87917fa640 1 -- 172.21.15.81:0/1292775343 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f878c057e50 con 0x7f87940cda00 2023-11-08T16:29:59.995 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:29:59.995 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:29:59.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.994+0000 7f879ad25640 1 -- 172.21.15.81:0/1292775343 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f878006ae70 msgr2=0x7f878006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.994+0000 7f879ad25640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f878006ae70 0x7f878006d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8788036000 tx=0x7f878803c040 comp rx=0 tx=0).stop 2023-11-08T16:29:59.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.994+0000 7f879ad25640 1 -- 172.21.15.81:0/1292775343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87940cda00 msgr2=0x7f87940c79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:29:59.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.994+0000 7f879ad25640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87940cda00 0x7f87940c79c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f878c00c780 tx=0x7f878c00cc40 comp rx=0 tx=0).stop 2023-11-08T16:29:59.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.994+0000 7f879ad25640 1 -- 172.21.15.81:0/1292775343 shutdown_connections 2023-11-08T16:29:59.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.994+0000 7f879ad25640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f878006ae70 0x7f878006d330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:29:59.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.994+0000 7f879ad25640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87940cda00 0x7f87940c79c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.994+0000 7f879ad25640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87940ac2a0 0x7f87941587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.994+0000 7f879ad25640 1 --2- 172.21.15.81:0/1292775343 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f879406e1b0 0x7f87940c5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.994+0000 7f879ad25640 1 -- 172.21.15.81:0/1292775343 >> 172.21.15.81:0/1292775343 conn(0x7f87940ceee0 msgr2=0x7f879406ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:00.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.995+0000 7f879ad25640 1 -- 172.21.15.81:0/1292775343 shutdown_connections 2023-11-08T16:30:00.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:29:59.995+0000 7f879ad25640 1 -- 172.21.15.81:0/1292775343 wait complete. 2023-11-08T16:30:00.017 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:30:00.018 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:30:00.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.239+0000 7fa21ea9e640 1 Processor -- start 2023-11-08T16:30:00.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.239+0000 7fa21ea9e640 1 -- start start 2023-11-08T16:30:00.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.240+0000 7fa21ea9e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa2181576a0 0x7fa21815a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.240+0000 7fa21ea9e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2180b11f0 0x7fa218157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.240+0000 7fa21ea9e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa218157a80 con 0x7fa21815b250 2023-11-08T16:30:00.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.240+0000 7fa21ea9e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa21815c360 con 0x7fa2181576a0 2023-11-08T16:30:00.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.240+0000 7fa21ea9e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa21815b6e0 con 0x7fa2180b11f0 2023-11-08T16:30:00.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.240+0000 7fa21da9c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa2181576a0 0x7fa21815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.240+0000 7fa21da9c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa2181576a0 0x7fa21815a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47146/0 (socket says 172.21.15.81:47146) 2023-11-08T16:30:00.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.240+0000 7fa21da9c640 1 -- 172.21.15.81:0/3063492682 learned_addr learned my addr 172.21.15.81:0/3063492682 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:00.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.240+0000 7fa21d29b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa21815b250 0x7fa218070c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52786/0 (socket says 172.21.15.81:52786) 2023-11-08T16:30:00.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.241+0000 7fa21e29d640 1 --2- 172.21.15.81:0/3063492682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2180b11f0 0x7fa218157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.241+0000 7fa21da9c640 1 -- 172.21.15.81:0/3063492682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2180b11f0 msgr2=0x7fa218157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:00.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.241+0000 7fa21da9c640 1 --2- 172.21.15.81:0/3063492682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2180b11f0 0x7fa218157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.241+0000 7fa21da9c640 1 -- 172.21.15.81:0/3063492682 >> v1:172.21.15.81:6789/0 conn(0x7fa21815b250 legacy=0x7fa218070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.241+0000 7fa21da9c640 1 -- 172.21.15.81:0/3063492682 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa21815b9c0 con 0x7fa2181576a0 2023-11-08T16:30:00.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.241+0000 7fa21ca9a640 1 -- 172.21.15.81:0/3063492682 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 181705580 0 0) 0x7fa218157a80 con 0x7fa21815b250 2023-11-08T16:30:00.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.241+0000 7fa21e29d640 1 --2- 172.21.15.81:0/3063492682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2180b11f0 0x7fa218157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:00.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.242+0000 7fa21da9c640 1 --2- 172.21.15.81:0/3063492682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2181576a0 0x7fa21815a010 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa214009870 tx=0x7fa21402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e195cfece21a9e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:00.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.243+0000 7fa21ca9a640 1 -- 172.21.15.81:0/3063492682 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214031640 con 0x7fa2181576a0 2023-11-08T16:30:00.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.243+0000 7fa21ca9a640 1 -- 172.21.15.81:0/3063492682 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2140317a0 con 0x7fa2181576a0 2023-11-08T16:30:00.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.243+0000 7fa21ca9a640 1 -- 172.21.15.81:0/3063492682 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa214031b30 con 0x7fa2181576a0 2023-11-08T16:30:00.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.243+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3063492682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2181576a0 msgr2=0x7fa21815a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.243+0000 7fa21ea9e640 1 --2- 172.21.15.81:0/3063492682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2181576a0 0x7fa21815a010 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa214009870 tx=0x7fa21402b8f0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.243+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3063492682 shutdown_connections 2023-11-08T16:30:00.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.243+0000 7fa21ea9e640 1 --2- 172.21.15.81:0/3063492682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2180b11f0 0x7fa218157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.243+0000 7fa21ea9e640 1 --2- 172.21.15.81:0/3063492682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2181576a0 0x7fa21815a010 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.243+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3063492682 >> 172.21.15.81:0/3063492682 conn(0x7fa2180ce8d0 msgr2=0x7fa2180cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:00.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.243+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3063492682 shutdown_connections 2023-11-08T16:30:00.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.244+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3063492682 wait complete. 2023-11-08T16:30:00.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.245+0000 7fa21ea9e640 1 Processor -- start 2023-11-08T16:30:00.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.245+0000 7fa21ea9e640 1 -- start start 2023-11-08T16:30:00.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.246+0000 7fa21ea9e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2180b11f0 0x7fa2180c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.246+0000 7fa21ea9e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2181576a0 0x7fa21814aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.246+0000 7fa21ea9e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2180c7120 0x7fa2180c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.246+0000 7fa21ea9e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa21814b5e0 con 0x7fa2181576a0 2023-11-08T16:30:00.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.246+0000 7fa21ea9e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2180c5e50 con 0x7fa2180b11f0 2023-11-08T16:30:00.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.246+0000 7fa21ea9e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2180c6080 con 0x7fa2180c7120 2023-11-08T16:30:00.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.246+0000 7fa21d29b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2181576a0 0x7fa21814aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.247+0000 7fa21d29b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2181576a0 0x7fa21814aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47150/0 (socket says 172.21.15.81:47150) 2023-11-08T16:30:00.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.247+0000 7fa21d29b640 1 -- 172.21.15.81:0/3250271963 learned_addr learned my addr 172.21.15.81:0/3250271963 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:00.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.247+0000 7fa21e29d640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2180c7120 0x7fa2180c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.247+0000 7fa21da9c640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2180b11f0 0x7fa2180c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.247+0000 7fa21d29b640 1 -- 172.21.15.81:0/3250271963 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2180c7120 msgr2=0x7fa2180c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.247+0000 7fa21d29b640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2180c7120 0x7fa2180c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.247+0000 7fa21d29b640 1 -- 172.21.15.81:0/3250271963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2180b11f0 msgr2=0x7fa2180c5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.247+0000 7fa21d29b640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2180b11f0 0x7fa2180c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.247+0000 7fa21d29b640 1 -- 172.21.15.81:0/3250271963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2140094d0 con 0x7fa2181576a0 2023-11-08T16:30:00.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.247+0000 7fa21d29b640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2181576a0 0x7fa21814aa90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa20800c460 tx=0x7fa20800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:00.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.247+0000 7fa20e7fc640 1 -- 172.21.15.81:0/3250271963 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2080074e0 con 0x7fa2181576a0 2023-11-08T16:30:00.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.248+0000 7fa20e7fc640 1 -- 172.21.15.81:0/3250271963 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa208007640 con 0x7fa2181576a0 2023-11-08T16:30:00.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.248+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3250271963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2180c6360 con 0x7fa2181576a0 2023-11-08T16:30:00.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.248+0000 7fa20e7fc640 1 -- 172.21.15.81:0/3250271963 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa208004a20 con 0x7fa2181576a0 2023-11-08T16:30:00.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.248+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3250271963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2180c6920 con 0x7fa2181576a0 2023-11-08T16:30:00.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.249+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3250271963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa21804ece0 con 0x7fa2181576a0 2023-11-08T16:30:00.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.250+0000 7fa20e7fc640 1 -- 172.21.15.81:0/3250271963 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa208004b80 con 0x7fa2181576a0 2023-11-08T16:30:00.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.251+0000 7fa20e7fc640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1fc06ae70 0x7fa1fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.251+0000 7fa20e7fc640 1 -- 172.21.15.81:0/3250271963 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa208088bc0 con 0x7fa2181576a0 2023-11-08T16:30:00.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.251+0000 7fa21da9c640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1fc06ae70 0x7fa1fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.252+0000 7fa21da9c640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1fc06ae70 0x7fa1fc06d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa214036000 tx=0x7fa21403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:00.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.255+0000 7fa20e7fc640 1 -- 172.21.15.81:0/3250271963 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa208053c70 con 0x7fa2181576a0 2023-11-08T16:30:00.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.452+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3250271963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa218150da0 con 0x7fa2181576a0 2023-11-08T16:30:00.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.453+0000 7fa20e7fc640 1 -- 172.21.15.81:0/3250271963 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa208018020 con 0x7fa2181576a0 2023-11-08T16:30:00.456 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:30:00.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.455+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3250271963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1fc06ae70 msgr2=0x7fa1fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.455+0000 7fa21ea9e640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1fc06ae70 0x7fa1fc06d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa214036000 tx=0x7fa21403c040 comp rx=0 tx=0).stop 2023-11-08T16:30:00.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.455+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3250271963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2181576a0 msgr2=0x7fa21814aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.455+0000 7fa21ea9e640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2181576a0 0x7fa21814aa90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa20800c460 tx=0x7fa20800c920 comp rx=0 tx=0).stop 2023-11-08T16:30:00.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.456+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3250271963 shutdown_connections 2023-11-08T16:30:00.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.456+0000 7fa21ea9e640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1fc06ae70 0x7fa1fc06d330 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.456+0000 7fa21ea9e640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2180c7120 0x7fa2180c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.456+0000 7fa21ea9e640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa2181576a0 0x7fa21814aa90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.456+0000 7fa21ea9e640 1 --2- 172.21.15.81:0/3250271963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2180b11f0 0x7fa2180c5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.456+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3250271963 >> 172.21.15.81:0/3250271963 conn(0x7fa2180ce8d0 msgr2=0x7fa21815a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:00.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.456+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3250271963 shutdown_connections 2023-11-08T16:30:00.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.456+0000 7fa21ea9e640 1 -- 172.21.15.81:0/3250271963 wait complete. 2023-11-08T16:30:00.477 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:30:00.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.691+0000 7f54fdfc0640 1 Processor -- start 2023-11-08T16:30:00.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.691+0000 7f54fdfc0640 1 -- start start 2023-11-08T16:30:00.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.693+0000 7f54fdfc0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f815f280 0x7f54f80caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.693+0000 7f54fdfc0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f80cda10 con 0x7f54f80ce0d0 2023-11-08T16:30:00.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.693+0000 7f54fdfc0640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54f806fa30 con 0x7f54f815bfc0 2023-11-08T16:30:00.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.693+0000 7f54fdfc0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f80ce4b0 con 0x7f54f815f280 2023-11-08T16:30:00.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.693+0000 7f54f77fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f54f80ce0d0 0x7f54f815eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:52800/0 (socket says 172.21.15.81:52800) 2023-11-08T16:30:00.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.693+0000 7f54f77fe640 1 -- 172.21.15.81:0/1419226360 learned_addr learned my addr 172.21.15.81:0/1419226360 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:00.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.693+0000 7f54f7fff640 1 --2- 172.21.15.81:0/1419226360 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f815f280 0x7f54f80caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.693+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 478895487 0 0) 0x7f54f80cda10 con 0x7f54f80ce0d0 2023-11-08T16:30:00.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.693+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54e0003520 con 0x7f54f80ce0d0 2023-11-08T16:30:00.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 89639151 0 0) 0x7f54e0003520 con 0x7f54f80ce0d0 2023-11-08T16:30:00.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f815f280 msgr2=0x7f54f80caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:00.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 --2- 172.21.15.81:0/1419226360 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f815f280 0x7f54f80caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 >> v1:172.21.15.93:6789/0 conn(0x7f54f815bfc0 legacy=0x7f54f81516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f806fca0 con 0x7f54f80ce0d0 2023-11-08T16:30:00.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f54e8001670 con 0x7f54f80ce0d0 2023-11-08T16:30:00.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 >> v1:172.21.15.81:6789/0 conn(0x7f54f80ce0d0 legacy=0x7f54f815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 --2- 172.21.15.81:0/1419226360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e0003a10 0x7f54e0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 --2- 172.21.15.81:0/1419226360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e00047c0 0x7f54e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 --2- 172.21.15.81:0/1419226360 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f815f280 0x7f54e00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f80ce4b0 con 0x7f54e00047c0 2023-11-08T16:30:00.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e000bb70 con 0x7f54e0003a10 2023-11-08T16:30:00.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e000bd30 con 0x7f54f815f280 2023-11-08T16:30:00.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.694+0000 7f54f6ffd640 1 --2- 172.21.15.81:0/1419226360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e00047c0 0x7f54e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.695+0000 7f54f77fe640 1 --2- 172.21.15.81:0/1419226360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e0003a10 0x7f54e0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.695+0000 7f54f6ffd640 1 -- 172.21.15.81:0/1419226360 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f815f280 msgr2=0x7f54e00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:00.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.695+0000 7f54f6ffd640 1 --2- 172.21.15.81:0/1419226360 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f815f280 0x7f54e00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.695+0000 7f54f6ffd640 1 -- 172.21.15.81:0/1419226360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e0003a10 msgr2=0x7f54e0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:00.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.695+0000 7f54f6ffd640 1 --2- 172.21.15.81:0/1419226360 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e0003a10 0x7f54e0003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.695+0000 7f54f6ffd640 1 -- 172.21.15.81:0/1419226360 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e0003520 con 0x7f54e00047c0 2023-11-08T16:30:00.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.696+0000 7f54f6ffd640 1 --2- 172.21.15.81:0/1419226360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e00047c0 0x7f54e0004c40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54ec001750 tx=0x7f54e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d76369a421d9a81 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:00.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.696+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0052d0 con 0x7f54e00047c0 2023-11-08T16:30:00.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.696+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54ec02e050 con 0x7f54e00047c0 2023-11-08T16:30:00.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.696+0000 7f54f67fc640 1 -- 172.21.15.81:0/1419226360 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54ec0035f0 con 0x7f54e00047c0 2023-11-08T16:30:00.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.696+0000 7f54fdfc0640 1 -- 172.21.15.81:0/1419226360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e00047c0 msgr2=0x7f54e0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.696+0000 7f54fdfc0640 1 --2- 172.21.15.81:0/1419226360 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e00047c0 0x7f54e0004c40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54ec001750 tx=0x7f54e00057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.697+0000 7f54f7fff640 1 -- 172.21.15.81:0/1419226360 reap_dead start 2023-11-08T16:30:00.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.697+0000 7f54fdfc0640 1 -- 172.21.15.81:0/1419226360 shutdown_connections 2023-11-08T16:30:00.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.697+0000 7f54fdfc0640 1 -- 172.21.15.81:0/1419226360 >> 172.21.15.81:0/1419226360 conn(0x7f54f80cb580 msgr2=0x7f54f80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:00.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.697+0000 7f54fdfc0640 1 -- 172.21.15.81:0/1419226360 shutdown_connections 2023-11-08T16:30:00.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.697+0000 7f54fdfc0640 1 -- 172.21.15.81:0/1419226360 wait complete. 2023-11-08T16:30:00.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.699+0000 7f54fdfc0640 1 Processor -- start 2023-11-08T16:30:00.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.699+0000 7f54fdfc0640 1 -- start start 2023-11-08T16:30:00.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54fdfc0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e00047c0 0x7f54f81553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54fdfc0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f80c3680 0x7f54f80a8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54fdfc0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f80a8c80 0x7f54f80b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54fdfc0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f81558f0 con 0x7f54f80a8c80 2023-11-08T16:30:00.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54fdfc0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54f80a9660 con 0x7f54e00047c0 2023-11-08T16:30:00.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54fdfc0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54f80a9940 con 0x7f54f80c3680 2023-11-08T16:30:00.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54f7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f80a8c80 0x7f54f80b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54f7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f80a8c80 0x7f54f80b3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47162/0 (socket says 172.21.15.81:47162) 2023-11-08T16:30:00.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54f7fff640 1 -- 172.21.15.81:0/3658937782 learned_addr learned my addr 172.21.15.81:0/3658937782 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:00.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54f7fff640 1 -- 172.21.15.81:0/3658937782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f80c3680 msgr2=0x7f54f80a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54f6ffd640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f80c3680 0x7f54f80a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54f77fe640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e00047c0 0x7f54f81553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54f7fff640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f80c3680 0x7f54f80a8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54f7fff640 1 -- 172.21.15.81:0/3658937782 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e00047c0 msgr2=0x7f54f81553b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54f7fff640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e00047c0 0x7f54f81553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.700+0000 7f54f7fff640 1 -- 172.21.15.81:0/3658937782 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80a9c20 con 0x7f54f80a8c80 2023-11-08T16:30:00.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.701+0000 7f54f7fff640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f80a8c80 0x7f54f80b3650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f54f806b3e0 tx=0x7f54f000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:00.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.701+0000 7f54dffff640 1 -- 172.21.15.81:0/3658937782 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0010040 con 0x7f54f80a8c80 2023-11-08T16:30:00.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.701+0000 7f54dffff640 1 -- 172.21.15.81:0/3658937782 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f000bce0 con 0x7f54f80a8c80 2023-11-08T16:30:00.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.701+0000 7f54dffff640 1 -- 172.21.15.81:0/3658937782 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f0014640 con 0x7f54f80a8c80 2023-11-08T16:30:00.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.701+0000 7f54fdfc0640 1 -- 172.21.15.81:0/3658937782 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54f81590c0 con 0x7f54f80a8c80 2023-11-08T16:30:00.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.701+0000 7f54fdfc0640 1 -- 172.21.15.81:0/3658937782 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54f80c3b00 con 0x7f54f80a8c80 2023-11-08T16:30:00.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.702+0000 7f54fdfc0640 1 -- 172.21.15.81:0/3658937782 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54f81595e0 con 0x7f54f80a8c80 2023-11-08T16:30:00.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.703+0000 7f54dffff640 1 -- 172.21.15.81:0/3658937782 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f54f00147a0 con 0x7f54f80a8c80 2023-11-08T16:30:00.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.704+0000 7f54dffff640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54e406ae70 0x7f54e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:00.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.704+0000 7f54dffff640 1 -- 172.21.15.81:0/3658937782 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f54f0089eb0 con 0x7f54f80a8c80 2023-11-08T16:30:00.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.704+0000 7f54f77fe640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54e406ae70 0x7f54e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:00.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.705+0000 7f54f77fe640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54e406ae70 0x7f54e406d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f54e8001050 tx=0x7f54e8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:00.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.709+0000 7f54dffff640 1 -- 172.21.15.81:0/3658937782 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f54f81595e0 con 0x7f54f80a8c80 2023-11-08T16:30:00.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.897+0000 7f54fdfc0640 1 -- 172.21.15.81:0/3658937782 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f54f81595e0 con 0x7f54f80a8c80 2023-11-08T16:30:00.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.898+0000 7f54dffff640 1 -- 172.21.15.81:0/3658937782 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f54f81595e0 con 0x7f54f80a8c80 2023-11-08T16:30:00.901 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:30:00.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.900+0000 7f54fdfc0640 1 -- 172.21.15.81:0/3658937782 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54e406ae70 msgr2=0x7f54e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.900+0000 7f54fdfc0640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54e406ae70 0x7f54e406d330 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f54e8001050 tx=0x7f54e8009040 comp rx=0 tx=0).stop 2023-11-08T16:30:00.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.900+0000 7f54fdfc0640 1 -- 172.21.15.81:0/3658937782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f80a8c80 msgr2=0x7f54f80b3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:00.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.900+0000 7f54fdfc0640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f80a8c80 0x7f54f80b3650 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f54f806b3e0 tx=0x7f54f000b480 comp rx=0 tx=0).stop 2023-11-08T16:30:00.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.901+0000 7f54fdfc0640 1 -- 172.21.15.81:0/3658937782 shutdown_connections 2023-11-08T16:30:00.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.901+0000 7f54fdfc0640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54e406ae70 0x7f54e406d330 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.901+0000 7f54fdfc0640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54f80a8c80 0x7f54f80b3650 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.901+0000 7f54fdfc0640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54f80c3680 0x7f54f80a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.901+0000 7f54fdfc0640 1 --2- 172.21.15.81:0/3658937782 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e00047c0 0x7f54f81553b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:00.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.901+0000 7f54fdfc0640 1 -- 172.21.15.81:0/3658937782 >> 172.21.15.81:0/3658937782 conn(0x7f54f80cb580 msgr2=0x7f54f815a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:00.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.901+0000 7f54fdfc0640 1 -- 172.21.15.81:0/3658937782 shutdown_connections 2023-11-08T16:30:00.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:00.901+0000 7f54fdfc0640 1 -- 172.21.15.81:0/3658937782 wait complete. 2023-11-08T16:30:00.920 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:30:01.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.128+0000 7f89878a9640 1 Processor -- start 2023-11-08T16:30:01.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.128+0000 7f89878a9640 1 -- start start 2023-11-08T16:30:01.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.129+0000 7f89878a9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f898015d4c0 0x7f89800cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.129+0000 7f89878a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800cfcc0 0x7f89800b6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.129+0000 7f89878a9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89800cef70 con 0x7f89800d39a0 2023-11-08T16:30:01.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.129+0000 7f89878a9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89800cec90 con 0x7f898015d4c0 2023-11-08T16:30:01.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.129+0000 7f89878a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89800b5e70 con 0x7f89800cfcc0 2023-11-08T16:30:01.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.130+0000 7f8985645640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f898015d4c0 0x7f89800cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.130+0000 7f8985645640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f898015d4c0 0x7f89800cdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58866/0 (socket says 172.21.15.81:58866) 2023-11-08T16:30:01.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.130+0000 7f8985645640 1 -- 172.21.15.81:0/1907852016 learned_addr learned my addr 172.21.15.81:0/1907852016 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:01.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.130+0000 7f8985e46640 1 --2- 172.21.15.81:0/1907852016 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800cfcc0 0x7f89800b6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.130+0000 7f8985645640 1 -- 172.21.15.81:0/1907852016 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800cfcc0 msgr2=0x7f89800b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:01.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.130+0000 7f8985645640 1 --2- 172.21.15.81:0/1907852016 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800cfcc0 0x7f89800b6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.130+0000 7f8985645640 1 -- 172.21.15.81:0/1907852016 >> v1:172.21.15.81:6789/0 conn(0x7f89800d39a0 legacy=0x7f89800cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.130+0000 7f8985645640 1 -- 172.21.15.81:0/1907852016 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89800b6150 con 0x7f898015d4c0 2023-11-08T16:30:01.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.130+0000 7f8985e46640 1 --2- 172.21.15.81:0/1907852016 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800cfcc0 0x7f89800b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:01.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.131+0000 7f8985645640 1 --2- 172.21.15.81:0/1907852016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898015d4c0 0x7f89800cdb40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f897c009870 tx=0x7f897c02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57143fea8890d934 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:01.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.131+0000 7f8977fff640 1 -- 172.21.15.81:0/1907852016 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c02f830 con 0x7f898015d4c0 2023-11-08T16:30:01.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.132+0000 7f8977fff640 1 -- 172.21.15.81:0/1907852016 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897c02f990 con 0x7f898015d4c0 2023-11-08T16:30:01.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.132+0000 7f8977fff640 1 -- 172.21.15.81:0/1907852016 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897c02fd20 con 0x7f898015d4c0 2023-11-08T16:30:01.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.132+0000 7f89878a9640 1 -- 172.21.15.81:0/1907852016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898015d4c0 msgr2=0x7f89800cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.132+0000 7f89878a9640 1 --2- 172.21.15.81:0/1907852016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898015d4c0 0x7f89800cdb40 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f897c009870 tx=0x7f897c02ba10 comp rx=0 tx=0).stop 2023-11-08T16:30:01.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.132+0000 7f89878a9640 1 -- 172.21.15.81:0/1907852016 shutdown_connections 2023-11-08T16:30:01.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.132+0000 7f89878a9640 1 --2- 172.21.15.81:0/1907852016 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800cfcc0 0x7f89800b6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.132+0000 7f89878a9640 1 --2- 172.21.15.81:0/1907852016 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898015d4c0 0x7f89800cdb40 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.132+0000 7f89878a9640 1 -- 172.21.15.81:0/1907852016 >> 172.21.15.81:0/1907852016 conn(0x7f89800d2e80 msgr2=0x7f89800d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:01.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.132+0000 7f89878a9640 1 -- 172.21.15.81:0/1907852016 shutdown_connections 2023-11-08T16:30:01.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.132+0000 7f89878a9640 1 -- 172.21.15.81:0/1907852016 wait complete. 2023-11-08T16:30:01.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.134+0000 7f89878a9640 1 Processor -- start 2023-11-08T16:30:01.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.134+0000 7f89878a9640 1 -- start start 2023-11-08T16:30:01.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.135+0000 7f89878a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89800cfcc0 0x7f8980158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.135+0000 7f89878a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800d39a0 0x7f89800cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.135+0000 7f89878a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898015d4c0 0x7f89800c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.135+0000 7f89878a9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800cce10 con 0x7f898015d4c0 2023-11-08T16:30:01.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.135+0000 7f89878a9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89800cb140 con 0x7f89800cfcc0 2023-11-08T16:30:01.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.135+0000 7f89878a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89800cb420 con 0x7f89800d39a0 2023-11-08T16:30:01.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.135+0000 7f8985e46640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898015d4c0 0x7f89800c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.135+0000 7f8985e46640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898015d4c0 0x7f89800c8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58880/0 (socket says 172.21.15.81:58880) 2023-11-08T16:30:01.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.135+0000 7f8985e46640 1 -- 172.21.15.81:0/1490504976 learned_addr learned my addr 172.21.15.81:0/1490504976 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:01.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8984e44640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800d39a0 0x7f89800cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8985645640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89800cfcc0 0x7f8980158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8985e46640 1 -- 172.21.15.81:0/1490504976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800d39a0 msgr2=0x7f89800cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8985e46640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800d39a0 0x7f89800cc4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8985e46640 1 -- 172.21.15.81:0/1490504976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89800cfcc0 msgr2=0x7f8980158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8985e46640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89800cfcc0 0x7f8980158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8985e46640 1 -- 172.21.15.81:0/1490504976 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f897c0094d0 con 0x7f898015d4c0 2023-11-08T16:30:01.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8984e44640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800d39a0 0x7f89800cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:01.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8985645640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89800cfcc0 0x7f8980158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:01.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8985e46640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898015d4c0 0x7f89800c8660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f897800cef0 tx=0x7f897800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:01.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8975ffb640 1 -- 172.21.15.81:0/1490504976 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897800d840 con 0x7f898015d4c0 2023-11-08T16:30:01.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8975ffb640 1 -- 172.21.15.81:0/1490504976 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f897800d9c0 con 0x7f898015d4c0 2023-11-08T16:30:01.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f8975ffb640 1 -- 172.21.15.81:0/1490504976 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f897800dd70 con 0x7f898015d4c0 2023-11-08T16:30:01.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.136+0000 7f89878a9640 1 -- 172.21.15.81:0/1490504976 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89800cb700 con 0x7f898015d4c0 2023-11-08T16:30:01.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.137+0000 7f89878a9640 1 -- 172.21.15.81:0/1490504976 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8980158f70 con 0x7f898015d4c0 2023-11-08T16:30:01.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.138+0000 7f89878a9640 1 -- 172.21.15.81:0/1490504976 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89800d2860 con 0x7f898015d4c0 2023-11-08T16:30:01.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.139+0000 7f8975ffb640 1 -- 172.21.15.81:0/1490504976 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f897801d360 con 0x7f898015d4c0 2023-11-08T16:30:01.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.139+0000 7f8975ffb640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896406aea0 0x7f896406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.140+0000 7f8975ffb640 1 -- 172.21.15.81:0/1490504976 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f897808a140 con 0x7f898015d4c0 2023-11-08T16:30:01.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.140+0000 7f8985645640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896406aea0 0x7f896406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.140+0000 7f8985645640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896406aea0 0x7f896406d360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f897c036000 tx=0x7f897c039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:01.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.144+0000 7f8975ffb640 1 -- 172.21.15.81:0/1490504976 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8978055220 con 0x7f898015d4c0 2023-11-08T16:30:01.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.332+0000 7f89878a9640 1 -- 172.21.15.81:0/1490504976 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89800cbc20 con 0x7f898015d4c0 2023-11-08T16:30:01.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.333+0000 7f8975ffb640 1 -- 172.21.15.81:0/1490504976 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89800cbc20 con 0x7f898015d4c0 2023-11-08T16:30:01.336 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:30:01.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 -- 172.21.15.81:0/1490504976 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896406aea0 msgr2=0x7f896406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896406aea0 0x7f896406d360 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f897c036000 tx=0x7f897c039040 comp rx=0 tx=0).stop 2023-11-08T16:30:01.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 -- 172.21.15.81:0/1490504976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898015d4c0 msgr2=0x7f89800c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898015d4c0 0x7f89800c8660 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f897800cef0 tx=0x7f897800a710 comp rx=0 tx=0).stop 2023-11-08T16:30:01.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 -- 172.21.15.81:0/1490504976 shutdown_connections 2023-11-08T16:30:01.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896406aea0 0x7f896406d360 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898015d4c0 0x7f89800c8660 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89800d39a0 0x7f89800cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 --2- 172.21.15.81:0/1490504976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89800cfcc0 0x7f8980158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 -- 172.21.15.81:0/1490504976 >> 172.21.15.81:0/1490504976 conn(0x7f89800d2e80 msgr2=0x7f89800b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:01.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 -- 172.21.15.81:0/1490504976 shutdown_connections 2023-11-08T16:30:01.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.336+0000 7f89878a9640 1 -- 172.21.15.81:0/1490504976 wait complete. 2023-11-08T16:30:01.357 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:30:01.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.568+0000 7f6f8759e640 1 Processor -- start 2023-11-08T16:30:01.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.568+0000 7f6f8759e640 1 -- start start 2023-11-08T16:30:01.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.569+0000 7f6f8759e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f88159af0 0x7f6f88070a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.569+0000 7f6f8759e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f88070580 con 0x7f6f880cd110 2023-11-08T16:30:01.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.569+0000 7f6f8759e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f880c9460 con 0x7f6f88070f50 2023-11-08T16:30:01.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.569+0000 7f6f8759e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8815e2a0 con 0x7f6f88159af0 2023-11-08T16:30:01.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.570+0000 7f6f86d9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f6f880cd110 0x7f6f880c99c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60166/0 (socket says 172.21.15.81:60166) 2023-11-08T16:30:01.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.570+0000 7f6f86d9d640 1 -- 172.21.15.81:0/1761225363 learned_addr learned my addr 172.21.15.81:0/1761225363 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:01.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.570+0000 7f6f8659c640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f88159af0 0x7f6f88070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.570+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164134432 0 0) 0x7f6f88070580 con 0x7f6f880cd110 2023-11-08T16:30:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.570+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f74003520 con 0x7f6f880cd110 2023-11-08T16:30:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.570+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 343184608 0 0) 0x7f6f74003520 con 0x7f6f880cd110 2023-11-08T16:30:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f88159af0 msgr2=0x7f6f88070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f88159af0 0x7f6f88070a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 >> v1:172.21.15.93:6789/0 conn(0x7f6f88070f50 legacy=0x7f6f8815d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f8815e580 con 0x7f6f880cd110 2023-11-08T16:30:01.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f6f800019f0 con 0x7f6f880cd110 2023-11-08T16:30:01.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 >> v1:172.21.15.81:6789/0 conn(0x7f6f880cd110 legacy=0x7f6f880c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f740039a0 0x7f6f74003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f740047c0 0x7f6f74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f88159af0 0x7f6f74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f8815e2a0 con 0x7f6f740039a0 2023-11-08T16:30:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7400ba60 con 0x7f6f740047c0 2023-11-08T16:30:01.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7400de30 con 0x7f6f88159af0 2023-11-08T16:30:01.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f8659c640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f740039a0 0x7f6f74003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f8659c640 1 -- 172.21.15.81:0/1761225363 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f88159af0 msgr2=0x7f6f74005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:01.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f8659c640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f88159af0 0x7f6f74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f8659c640 1 -- 172.21.15.81:0/1761225363 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f740047c0 msgr2=0x7f6f74004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:01.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f7ffff640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f740047c0 0x7f6f74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f8659c640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f740047c0 0x7f6f74004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.571+0000 7f6f8659c640 1 -- 172.21.15.81:0/1761225363 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f74003520 con 0x7f6f740039a0 2023-11-08T16:30:01.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.572+0000 7f6f7ffff640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f740047c0 0x7f6f74004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:01.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.573+0000 7f6f8659c640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f740039a0 0x7f6f74003dc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009a90 tx=0x7f6f740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82219d9ad15fef2f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:01.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.573+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7802c6b0 con 0x7f6f740039a0 2023-11-08T16:30:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.573+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f7802c810 con 0x7f6f740039a0 2023-11-08T16:30:01.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.573+0000 7f6f85d9b640 1 -- 172.21.15.81:0/1761225363 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7802cba0 con 0x7f6f740039a0 2023-11-08T16:30:01.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.573+0000 7f6f8759e640 1 -- 172.21.15.81:0/1761225363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f740039a0 msgr2=0x7f6f74003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.573+0000 7f6f8759e640 1 --2- 172.21.15.81:0/1761225363 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f740039a0 0x7f6f74003dc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6f78009a90 tx=0x7f6f740057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.573+0000 7f6f86d9d640 1 -- 172.21.15.81:0/1761225363 reap_dead start 2023-11-08T16:30:01.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.573+0000 7f6f8759e640 1 -- 172.21.15.81:0/1761225363 shutdown_connections 2023-11-08T16:30:01.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.573+0000 7f6f8759e640 1 -- 172.21.15.81:0/1761225363 >> 172.21.15.81:0/1761225363 conn(0x7f6f880b04d0 msgr2=0x7f6f880b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:01.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.574+0000 7f6f8759e640 1 -- 172.21.15.81:0/1761225363 shutdown_connections 2023-11-08T16:30:01.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.574+0000 7f6f8759e640 1 -- 172.21.15.81:0/1761225363 wait complete. 2023-11-08T16:30:01.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.575+0000 7f6f8759e640 1 Processor -- start 2023-11-08T16:30:01.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.575+0000 7f6f8759e640 1 -- start start 2023-11-08T16:30:01.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.576+0000 7f6f8759e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f740039a0 0x7f6f88154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.576+0000 7f6f8759e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f881eefc0 0x7f6f880c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.576+0000 7f6f8759e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f8815cb50 0x7f6f8815bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.576+0000 7f6f8759e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880b15c0 con 0x7f6f8815cb50 2023-11-08T16:30:01.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.576+0000 7f6f8759e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f881550e0 con 0x7f6f881eefc0 2023-11-08T16:30:01.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.576+0000 7f6f8759e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f880b1810 con 0x7f6f740039a0 2023-11-08T16:30:01.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.577+0000 7f6f86d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f8815cb50 0x7f6f8815bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.577+0000 7f6f86d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f8815cb50 0x7f6f8815bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58904/0 (socket says 172.21.15.81:58904) 2023-11-08T16:30:01.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.577+0000 7f6f86d9d640 1 -- 172.21.15.81:0/1389451123 learned_addr learned my addr 172.21.15.81:0/1389451123 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:01.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.577+0000 7f6f8659c640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f740039a0 0x7f6f88154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.577+0000 7f6f7ffff640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f881eefc0 0x7f6f880c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.577+0000 7f6f86d9d640 1 -- 172.21.15.81:0/1389451123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f740039a0 msgr2=0x7f6f88154ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.577+0000 7f6f86d9d640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f740039a0 0x7f6f88154ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.577+0000 7f6f86d9d640 1 -- 172.21.15.81:0/1389451123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f881eefc0 msgr2=0x7f6f880c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.577+0000 7f6f86d9d640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f881eefc0 0x7f6f880c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.577+0000 7f6f86d9d640 1 -- 172.21.15.81:0/1389451123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f880b1af0 con 0x7f6f8815cb50 2023-11-08T16:30:01.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.578+0000 7f6f86d9d640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f8815cb50 0x7f6f8815bdc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6f80001370 tx=0x7f6f8000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:01.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.578+0000 7f6f7effd640 1 -- 172.21.15.81:0/1389451123 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8000a5b0 con 0x7f6f8815cb50 2023-11-08T16:30:01.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.578+0000 7f6f7effd640 1 -- 172.21.15.81:0/1389451123 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f8000a710 con 0x7f6f8815cb50 2023-11-08T16:30:01.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.578+0000 7f6f7effd640 1 -- 172.21.15.81:0/1389451123 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f8000aac0 con 0x7f6f8815cb50 2023-11-08T16:30:01.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.578+0000 7f6f8759e640 1 -- 172.21.15.81:0/1389451123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f880b1dd0 con 0x7f6f8815cb50 2023-11-08T16:30:01.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.578+0000 7f6f8759e640 1 -- 172.21.15.81:0/1389451123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f880c4fb0 con 0x7f6f8815cb50 2023-11-08T16:30:01.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.579+0000 7f6f8759e640 1 -- 172.21.15.81:0/1389451123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f8804ece0 con 0x7f6f8815cb50 2023-11-08T16:30:01.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.580+0000 7f6f7effd640 1 -- 172.21.15.81:0/1389451123 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6f8000ac20 con 0x7f6f8815cb50 2023-11-08T16:30:01.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.581+0000 7f6f7effd640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6f6806aea0 0x7f6f6806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:01.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.581+0000 7f6f7effd640 1 -- 172.21.15.81:0/1389451123 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6f80088f20 con 0x7f6f8815cb50 2023-11-08T16:30:01.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.581+0000 7f6f8659c640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6f6806aea0 0x7f6f6806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:01.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.582+0000 7f6f8659c640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6f6806aea0 0x7f6f6806d360 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6f78036000 tx=0x7f6f78030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:01.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.585+0000 7f6f7effd640 1 -- 172.21.15.81:0/1389451123 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6f80053ea0 con 0x7f6f8815cb50 2023-11-08T16:30:01.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.785+0000 7f6f8759e640 1 -- 172.21.15.81:0/1389451123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6f880ccad0 con 0x7f6f8815cb50 2023-11-08T16:30:01.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.787+0000 7f6f7effd640 1 -- 172.21.15.81:0/1389451123 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6f8001a070 con 0x7f6f8815cb50 2023-11-08T16:30:01.790 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:30:01.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.789+0000 7f6f8759e640 1 -- 172.21.15.81:0/1389451123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6f6806aea0 msgr2=0x7f6f6806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.789+0000 7f6f8759e640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6f6806aea0 0x7f6f6806d360 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6f78036000 tx=0x7f6f78030000 comp rx=0 tx=0).stop 2023-11-08T16:30:01.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.789+0000 7f6f8759e640 1 -- 172.21.15.81:0/1389451123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f8815cb50 msgr2=0x7f6f8815bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:01.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.789+0000 7f6f8759e640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f8815cb50 0x7f6f8815bdc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6f80001370 tx=0x7f6f8000ccd0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.789+0000 7f6f8759e640 1 -- 172.21.15.81:0/1389451123 shutdown_connections 2023-11-08T16:30:01.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.789+0000 7f6f8759e640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6f6806aea0 0x7f6f6806d360 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.789+0000 7f6f8759e640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6f8815cb50 0x7f6f8815bdc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.789+0000 7f6f8759e640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6f881eefc0 0x7f6f880c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.789+0000 7f6f8759e640 1 --2- 172.21.15.81:0/1389451123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6f740039a0 0x7f6f88154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:01.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.789+0000 7f6f8759e640 1 -- 172.21.15.81:0/1389451123 >> 172.21.15.81:0/1389451123 conn(0x7f6f880b04d0 msgr2=0x7f6f880ce2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:01.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.790+0000 7f6f8759e640 1 -- 172.21.15.81:0/1389451123 shutdown_connections 2023-11-08T16:30:01.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:01.790+0000 7f6f8759e640 1 -- 172.21.15.81:0/1389451123 wait complete. 2023-11-08T16:30:01.810 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:30:01.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:01.811 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-08T16:30:01.908 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:30:01.909 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:30:01.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:01.910 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-08T16:30:01.967 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:30:01.968 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:30:02.030 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:30:02.030 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:30:02.079 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:30:02.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.299+0000 7ff97359e640 1 Processor -- start 2023-11-08T16:30:02.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.299+0000 7ff97359e640 1 -- start start 2023-11-08T16:30:02.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97359e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff97415cca0 0x7ff9740707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97359e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9740716d0 con 0x7ff97415d080 2023-11-08T16:30:02.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97359e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9740c6780 con 0x7ff9740c62d0 2023-11-08T16:30:02.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97359e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff974070e30 con 0x7ff97415cca0 2023-11-08T16:30:02.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff971d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff97415d080 0x7ff9740ac8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60180/0 (socket says 172.21.15.81:60180) 2023-11-08T16:30:02.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff971d9b640 1 -- 172.21.15.81:0/3031464860 learned_addr learned my addr 172.21.15.81:0/3031464860 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:02.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97259c640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff97415cca0 0x7ff9740707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455221854 0 0) 0x7ff9740716d0 con 0x7ff97415d080 2023-11-08T16:30:02.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff960003520 con 0x7ff97415d080 2023-11-08T16:30:02.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1218572967 0 0) 0x7ff960003520 con 0x7ff97415d080 2023-11-08T16:30:02.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff97415cca0 msgr2=0x7ff9740707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:02.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff97415cca0 0x7ff9740707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 >> v1:172.21.15.93:6789/0 conn(0x7ff9740c62d0 legacy=0x7ff97406a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff974071110 con 0x7ff97415d080 2023-11-08T16:30:02.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ff95c001670 con 0x7ff97415d080 2023-11-08T16:30:02.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 >> v1:172.21.15.81:6789/0 conn(0x7ff97415d080 legacy=0x7ff9740ac8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff960003a80 0x7ff960003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9600047c0 0x7ff960004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9600051f0 0x7ff96000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff974070e30 con 0x7ff9600047c0 2023-11-08T16:30:02.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96000c100 con 0x7ff9600051f0 2023-11-08T16:30:02.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.301+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96000c2c0 con 0x7ff960003a80 2023-11-08T16:30:02.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.302+0000 7ff971d9b640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9600047c0 0x7ff960004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.302+0000 7ff97259c640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff960003a80 0x7ff960003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.302+0000 7ff972d9d640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9600051f0 0x7ff96000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.302+0000 7ff971d9b640 1 -- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff960003a80 msgr2=0x7ff960003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:02.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.302+0000 7ff971d9b640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff960003a80 0x7ff960003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.302+0000 7ff971d9b640 1 -- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9600051f0 msgr2=0x7ff96000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:02.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.302+0000 7ff971d9b640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9600051f0 0x7ff96000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.302+0000 7ff971d9b640 1 -- 172.21.15.81:0/3031464860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff960003520 con 0x7ff9600047c0 2023-11-08T16:30:02.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.302+0000 7ff972d9d640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9600051f0 0x7ff96000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:02.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.303+0000 7ff971d9b640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9600047c0 0x7ff960004c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff95c007fd0 tx=0x7ff9600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d459a763461e2e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:02.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.304+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c005ad0 con 0x7ff9600047c0 2023-11-08T16:30:02.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.304+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c005c30 con 0x7ff9600047c0 2023-11-08T16:30:02.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.304+0000 7ff97159a640 1 -- 172.21.15.81:0/3031464860 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c02cab0 con 0x7ff9600047c0 2023-11-08T16:30:02.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.304+0000 7ff97359e640 1 -- 172.21.15.81:0/3031464860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9600047c0 msgr2=0x7ff960004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.304+0000 7ff97359e640 1 --2- 172.21.15.81:0/3031464860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff9600047c0 0x7ff960004c40 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff95c007fd0 tx=0x7ff9600057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.304+0000 7ff972d9d640 1 -- 172.21.15.81:0/3031464860 reap_dead start 2023-11-08T16:30:02.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.304+0000 7ff97359e640 1 -- 172.21.15.81:0/3031464860 shutdown_connections 2023-11-08T16:30:02.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.304+0000 7ff97359e640 1 -- 172.21.15.81:0/3031464860 >> 172.21.15.81:0/3031464860 conn(0x7ff9740cc410 msgr2=0x7ff9740cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:02.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.304+0000 7ff97359e640 1 -- 172.21.15.81:0/3031464860 shutdown_connections 2023-11-08T16:30:02.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.304+0000 7ff97359e640 1 -- 172.21.15.81:0/3031464860 wait complete. 2023-11-08T16:30:02.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.306+0000 7ff97359e640 1 Processor -- start 2023-11-08T16:30:02.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.306+0000 7ff97359e640 1 -- start start 2023-11-08T16:30:02.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.307+0000 7ff97359e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9600047c0 0x7ff9740b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.307+0000 7ff97359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974159560 0x7ff9740afe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.307+0000 7ff97359e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff97414c050 0x7ff97414c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.307+0000 7ff97359e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9741f2770 con 0x7ff974159560 2023-11-08T16:30:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.307+0000 7ff97359e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff97414ca10 con 0x7ff9600047c0 2023-11-08T16:30:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.307+0000 7ff97359e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740c6af0 con 0x7ff97414c050 2023-11-08T16:30:02.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff971d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974159560 0x7ff9740afe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff971d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974159560 0x7ff9740afe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58926/0 (socket says 172.21.15.81:58926) 2023-11-08T16:30:02.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff971d9b640 1 -- 172.21.15.81:0/1680963801 learned_addr learned my addr 172.21.15.81:0/1680963801 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:02.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff97259c640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9600047c0 0x7ff9740b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff972d9d640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff97414c050 0x7ff97414c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff971d9b640 1 -- 172.21.15.81:0/1680963801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff97414c050 msgr2=0x7ff97414c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff971d9b640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff97414c050 0x7ff97414c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff971d9b640 1 -- 172.21.15.81:0/1680963801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9600047c0 msgr2=0x7ff9740b50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff971d9b640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9600047c0 0x7ff9740b50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff971d9b640 1 -- 172.21.15.81:0/1680963801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9740c8d70 con 0x7ff974159560 2023-11-08T16:30:02.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff972d9d640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff97414c050 0x7ff97414c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:02.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.308+0000 7ff971d9b640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974159560 0x7ff9740afe70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff95c02c410 tx=0x7ff95c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:02.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.309+0000 7ff96affd640 1 -- 172.21.15.81:0/1680963801 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c03c040 con 0x7ff974159560 2023-11-08T16:30:02.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.309+0000 7ff96affd640 1 -- 172.21.15.81:0/1680963801 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c02cc80 con 0x7ff974159560 2023-11-08T16:30:02.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.309+0000 7ff96affd640 1 -- 172.21.15.81:0/1680963801 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c007070 con 0x7ff974159560 2023-11-08T16:30:02.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.309+0000 7ff97359e640 1 -- 172.21.15.81:0/1680963801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9740c9050 con 0x7ff974159560 2023-11-08T16:30:02.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.309+0000 7ff97359e640 1 -- 172.21.15.81:0/1680963801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9740c9610 con 0x7ff974159560 2023-11-08T16:30:02.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.310+0000 7ff97359e640 1 -- 172.21.15.81:0/1680963801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9740c6c60 con 0x7ff974159560 2023-11-08T16:30:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.311+0000 7ff96affd640 1 -- 172.21.15.81:0/1680963801 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff95c032070 con 0x7ff974159560 2023-11-08T16:30:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.312+0000 7ff96affd640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff95406aec0 0x7ff95406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.312+0000 7ff96affd640 1 -- 172.21.15.81:0/1680963801 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff95c0aab10 con 0x7ff974159560 2023-11-08T16:30:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.312+0000 7ff97259c640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff95406aec0 0x7ff95406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.313+0000 7ff97259c640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff95406aec0 0x7ff95406d380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff9740aedf0 tx=0x7ff96400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:02.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.316+0000 7ff96affd640 1 -- 172.21.15.81:0/1680963801 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff95c075b10 con 0x7ff974159560 2023-11-08T16:30:02.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.492+0000 7ff97359e640 1 -- 172.21.15.81:0/1680963801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9740c9800 con 0x7ff974159560 2023-11-08T16:30:02.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.494+0000 7ff96affd640 1 -- 172.21.15.81:0/1680963801 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7ff95c079130 con 0x7ff974159560 2023-11-08T16:30:02.497 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:02.498 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:30:02.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 -- 172.21.15.81:0/1680963801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff95406aec0 msgr2=0x7ff95406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff95406aec0 0x7ff95406d380 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff9740aedf0 tx=0x7ff96400b040 comp rx=0 tx=0).stop 2023-11-08T16:30:02.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 -- 172.21.15.81:0/1680963801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974159560 msgr2=0x7ff9740afe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974159560 0x7ff9740afe70 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff95c02c410 tx=0x7ff95c02e070 comp rx=0 tx=0).stop 2023-11-08T16:30:02.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 -- 172.21.15.81:0/1680963801 shutdown_connections 2023-11-08T16:30:02.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff95406aec0 0x7ff95406d380 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff97414c050 0x7ff97414c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff974159560 0x7ff9740afe70 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 --2- 172.21.15.81:0/1680963801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff9600047c0 0x7ff9740b50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 -- 172.21.15.81:0/1680963801 >> 172.21.15.81:0/1680963801 conn(0x7ff9740cc410 msgr2=0x7ff9740b0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:02.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 -- 172.21.15.81:0/1680963801 shutdown_connections 2023-11-08T16:30:02.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.497+0000 7ff97359e640 1 -- 172.21.15.81:0/1680963801 wait complete. 2023-11-08T16:30:02.516 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:30:02.517 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:30:02.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.721+0000 7fe4174e0640 1 Processor -- start 2023-11-08T16:30:02.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.721+0000 7fe4174e0640 1 -- start start 2023-11-08T16:30:02.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.722+0000 7fe4174e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe41015a0e0 0x7fe4100ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.722+0000 7fe4174e0640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe4100c9bf0 0x7fe4100ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.722+0000 7fe4174e0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe4100ca550 0x7fe41015dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.722+0000 7fe4174e0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe410071980 con 0x7fe4100ca550 2023-11-08T16:30:02.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.722+0000 7fe4174e0640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4100ce960 con 0x7fe4100c9bf0 2023-11-08T16:30:02.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.722+0000 7fe4174e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100c9480 con 0x7fe41015a0e0 2023-11-08T16:30:02.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.723+0000 7fe415a7d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe4100ca550 0x7fe41015dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.723+0000 7fe415a7d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe4100ca550 0x7fe41015dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58934/0 (socket says 172.21.15.81:58934) 2023-11-08T16:30:02.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.723+0000 7fe415a7d640 1 -- 172.21.15.81:0/2025487914 learned_addr learned my addr 172.21.15.81:0/2025487914 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:02.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.723+0000 7fe41527c640 1 --2- 172.21.15.81:0/2025487914 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe41015a0e0 0x7fe4100ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.723+0000 7fe414a7b640 1 --2- 172.21.15.81:0/2025487914 >> v2:172.21.15.93:3300/0 conn(0x7fe4100c9bf0 0x7fe4100ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.723+0000 7fe415a7d640 1 -- 172.21.15.81:0/2025487914 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe41015a0e0 msgr2=0x7fe4100ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:02.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.723+0000 7fe415a7d640 1 --2- 172.21.15.81:0/2025487914 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe41015a0e0 0x7fe4100ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.723+0000 7fe415a7d640 1 -- 172.21.15.81:0/2025487914 >> v2:172.21.15.93:3300/0 conn(0x7fe4100c9bf0 msgr2=0x7fe4100ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:02.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.723+0000 7fe415a7d640 1 --2- 172.21.15.81:0/2025487914 >> v2:172.21.15.93:3300/0 conn(0x7fe4100c9bf0 0x7fe4100ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.723+0000 7fe415a7d640 1 -- 172.21.15.81:0/2025487914 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41015a4c0 con 0x7fe4100ca550 2023-11-08T16:30:02.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.723+0000 7fe414a7b640 1 --2- 172.21.15.81:0/2025487914 >> v2:172.21.15.93:3300/0 conn(0x7fe4100c9bf0 0x7fe4100ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:02.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.724+0000 7fe415a7d640 1 --2- 172.21.15.81:0/2025487914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4100ca550 0x7fe41015dbb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe4100cc680 tx=0x7fe40802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f747907f19a4917e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:02.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.724+0000 7fe407fff640 1 -- 172.21.15.81:0/2025487914 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40802cb00 con 0x7fe4100ca550 2023-11-08T16:30:02.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.725+0000 7fe407fff640 1 -- 172.21.15.81:0/2025487914 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40802cc60 con 0x7fe4100ca550 2023-11-08T16:30:02.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.725+0000 7fe407fff640 1 -- 172.21.15.81:0/2025487914 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4080343f0 con 0x7fe4100ca550 2023-11-08T16:30:02.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.725+0000 7fe4174e0640 1 -- 172.21.15.81:0/2025487914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4100ca550 msgr2=0x7fe41015dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.725+0000 7fe4174e0640 1 --2- 172.21.15.81:0/2025487914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4100ca550 0x7fe41015dbb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe4100cc680 tx=0x7fe40802bf60 comp rx=0 tx=0).stop 2023-11-08T16:30:02.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.725+0000 7fe4174e0640 1 -- 172.21.15.81:0/2025487914 shutdown_connections 2023-11-08T16:30:02.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.725+0000 7fe4174e0640 1 --2- 172.21.15.81:0/2025487914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4100ca550 0x7fe41015dbb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.725+0000 7fe4174e0640 1 --2- 172.21.15.81:0/2025487914 >> v2:172.21.15.93:3300/0 conn(0x7fe4100c9bf0 0x7fe4100ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.725+0000 7fe4174e0640 1 --2- 172.21.15.81:0/2025487914 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe41015a0e0 0x7fe4100ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.725+0000 7fe4174e0640 1 -- 172.21.15.81:0/2025487914 >> 172.21.15.81:0/2025487914 conn(0x7fe4100cb580 msgr2=0x7fe4100cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:02.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.725+0000 7fe4174e0640 1 -- 172.21.15.81:0/2025487914 shutdown_connections 2023-11-08T16:30:02.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.725+0000 7fe4174e0640 1 -- 172.21.15.81:0/2025487914 wait complete. 2023-11-08T16:30:02.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.727+0000 7fe4174e0640 1 Processor -- start 2023-11-08T16:30:02.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.727+0000 7fe4174e0640 1 -- start start 2023-11-08T16:30:02.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.728+0000 7fe4174e0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4100c9bf0 0x7fe4101554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.728+0000 7fe4174e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4100ca550 0x7fe4101565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.728+0000 7fe4174e0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe41015a0e0 0x7fe4100c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.728+0000 7fe4174e0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4100a9300 con 0x7fe4100c9bf0 2023-11-08T16:30:02.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.728+0000 7fe4174e0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe410156fd0 con 0x7fe41015a0e0 2023-11-08T16:30:02.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.728+0000 7fe4174e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe410155a20 con 0x7fe4100ca550 2023-11-08T16:30:02.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.728+0000 7fe41527c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4100c9bf0 0x7fe4101554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.728+0000 7fe41527c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4100c9bf0 0x7fe4101554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58938/0 (socket says 172.21.15.81:58938) 2023-11-08T16:30:02.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.728+0000 7fe41527c640 1 -- 172.21.15.81:0/652969326 learned_addr learned my addr 172.21.15.81:0/652969326 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:02.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe414a7b640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4100ca550 0x7fe4101565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe41527c640 1 -- 172.21.15.81:0/652969326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4100ca550 msgr2=0x7fe4101565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe41527c640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4100ca550 0x7fe4101565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe415a7d640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe41015a0e0 0x7fe4100c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe41527c640 1 -- 172.21.15.81:0/652969326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe41015a0e0 msgr2=0x7fe4100c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe41527c640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe41015a0e0 0x7fe4100c3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe41527c640 1 -- 172.21.15.81:0/652969326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe408009b30 con 0x7fe4100c9bf0 2023-11-08T16:30:02.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe41527c640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4100c9bf0 0x7fe4101554e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00d800 tx=0x7fe40c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:02.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe405ffb640 1 -- 172.21.15.81:0/652969326 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00b660 con 0x7fe4100c9bf0 2023-11-08T16:30:02.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe405ffb640 1 -- 172.21.15.81:0/652969326 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c00b7c0 con 0x7fe4100c9bf0 2023-11-08T16:30:02.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe4174e0640 1 -- 172.21.15.81:0/652969326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4100c6b40 con 0x7fe4100c9bf0 2023-11-08T16:30:02.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe405ffb640 1 -- 172.21.15.81:0/652969326 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c00bb50 con 0x7fe4100c9bf0 2023-11-08T16:30:02.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.729+0000 7fe4174e0640 1 -- 172.21.15.81:0/652969326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4100c7100 con 0x7fe4100c9bf0 2023-11-08T16:30:02.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.730+0000 7fe4174e0640 1 -- 172.21.15.81:0/652969326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41004ece0 con 0x7fe4100c9bf0 2023-11-08T16:30:02.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.731+0000 7fe405ffb640 1 -- 172.21.15.81:0/652969326 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe40c00bcb0 con 0x7fe4100c9bf0 2023-11-08T16:30:02.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.732+0000 7fe405ffb640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe3f406ae70 0x7fe3f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:02.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.732+0000 7fe405ffb640 1 -- 172.21.15.81:0/652969326 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe40c089bf0 con 0x7fe4100c9bf0 2023-11-08T16:30:02.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.732+0000 7fe414a7b640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe3f406ae70 0x7fe3f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:02.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.733+0000 7fe414a7b640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe3f406ae70 0x7fe3f406d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe41006c8b0 tx=0x7fe400008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:02.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.736+0000 7fe405ffb640 1 -- 172.21.15.81:0/652969326 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe40c054c00 con 0x7fe4100c9bf0 2023-11-08T16:30:02.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.912+0000 7fe4174e0640 1 -- 172.21.15.81:0/652969326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe410157140 con 0x7fe4100c9bf0 2023-11-08T16:30:02.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.914+0000 7fe405ffb640 1 -- 172.21.15.81:0/652969326 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe40c058220 con 0x7fe4100c9bf0 2023-11-08T16:30:02.916 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:30:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.915+0000 7fe4174e0640 1 -- 172.21.15.81:0/652969326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe3f406ae70 msgr2=0x7fe3f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.915+0000 7fe4174e0640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe3f406ae70 0x7fe3f406d330 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe41006c8b0 tx=0x7fe400008040 comp rx=0 tx=0).stop 2023-11-08T16:30:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.915+0000 7fe4174e0640 1 -- 172.21.15.81:0/652969326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4100c9bf0 msgr2=0x7fe4101554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.915+0000 7fe4174e0640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4100c9bf0 0x7fe4101554e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00d800 tx=0x7fe40c00dcc0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.916+0000 7fe4174e0640 1 -- 172.21.15.81:0/652969326 shutdown_connections 2023-11-08T16:30:02.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.916+0000 7fe4174e0640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe3f406ae70 0x7fe3f406d330 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.916+0000 7fe4174e0640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe41015a0e0 0x7fe4100c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.916+0000 7fe4174e0640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4100ca550 0x7fe4101565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.916+0000 7fe4174e0640 1 --2- 172.21.15.81:0/652969326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4100c9bf0 0x7fe4101554e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:02.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.916+0000 7fe4174e0640 1 -- 172.21.15.81:0/652969326 >> 172.21.15.81:0/652969326 conn(0x7fe4100cb580 msgr2=0x7fe41006bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:02.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.916+0000 7fe4174e0640 1 -- 172.21.15.81:0/652969326 shutdown_connections 2023-11-08T16:30:02.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:02.916+0000 7fe4174e0640 1 -- 172.21.15.81:0/652969326 wait complete. 2023-11-08T16:30:02.937 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:30:03.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.139+0000 7f446b839640 1 Processor -- start 2023-11-08T16:30:03.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.139+0000 7f446b839640 1 -- start start 2023-11-08T16:30:03.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.141+0000 7f446b839640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44640ce0d0 0x7f44640caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.141+0000 7f446b839640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f446415bfc0 0x7f446415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.141+0000 7f446b839640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f446415f0b0 0x7f44641516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.141+0000 7f446b839640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446415f500 con 0x7f446415bfc0 2023-11-08T16:30:03.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.141+0000 7f446b839640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f446406fa30 con 0x7f446415f0b0 2023-11-08T16:30:03.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.141+0000 7f446b839640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44640ce4b0 con 0x7f44640ce0d0 2023-11-08T16:30:03.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.141+0000 7f4468dd4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f446415bfc0 0x7f446415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.141+0000 7f4468dd4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f446415bfc0 0x7f446415eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58954/0 (socket says 172.21.15.81:58954) 2023-11-08T16:30:03.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.141+0000 7f4468dd4640 1 -- 172.21.15.81:0/3422360735 learned_addr learned my addr 172.21.15.81:0/3422360735 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:03.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.142+0000 7f4469dd6640 1 --2- 172.21.15.81:0/3422360735 >> v2:172.21.15.93:3300/0 conn(0x7f446415f0b0 0x7f44641516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.142+0000 7f44695d5640 1 --2- 172.21.15.81:0/3422360735 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44640ce0d0 0x7f44640caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.142+0000 7f4468dd4640 1 -- 172.21.15.81:0/3422360735 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44640ce0d0 msgr2=0x7f44640caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:03.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.142+0000 7f4468dd4640 1 --2- 172.21.15.81:0/3422360735 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44640ce0d0 0x7f44640caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.142+0000 7f4468dd4640 1 -- 172.21.15.81:0/3422360735 >> v2:172.21.15.93:3300/0 conn(0x7f446415f0b0 msgr2=0x7f44641516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:03.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.142+0000 7f4468dd4640 1 --2- 172.21.15.81:0/3422360735 >> v2:172.21.15.93:3300/0 conn(0x7f446415f0b0 0x7f44641516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.142+0000 7f4468dd4640 1 -- 172.21.15.81:0/3422360735 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f446406fca0 con 0x7f446415bfc0 2023-11-08T16:30:03.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.142+0000 7f44695d5640 1 --2- 172.21.15.81:0/3422360735 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44640ce0d0 0x7f44640caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:03.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.143+0000 7f4468dd4640 1 --2- 172.21.15.81:0/3422360735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f446415bfc0 0x7f446415eb70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f44640ae5b0 tx=0x7f445402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5d6b872d0b3e9e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:03.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.143+0000 7f445bfff640 1 -- 172.21.15.81:0/3422360735 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454031870 con 0x7f446415bfc0 2023-11-08T16:30:03.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.144+0000 7f445bfff640 1 -- 172.21.15.81:0/3422360735 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44540319d0 con 0x7f446415bfc0 2023-11-08T16:30:03.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.144+0000 7f445bfff640 1 -- 172.21.15.81:0/3422360735 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4454031d60 con 0x7f446415bfc0 2023-11-08T16:30:03.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.144+0000 7f446b839640 1 -- 172.21.15.81:0/3422360735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f446415bfc0 msgr2=0x7f446415eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.144+0000 7f446b839640 1 --2- 172.21.15.81:0/3422360735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f446415bfc0 0x7f446415eb70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f44640ae5b0 tx=0x7f445402b910 comp rx=0 tx=0).stop 2023-11-08T16:30:03.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.144+0000 7f446b839640 1 -- 172.21.15.81:0/3422360735 shutdown_connections 2023-11-08T16:30:03.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.144+0000 7f446b839640 1 --2- 172.21.15.81:0/3422360735 >> v2:172.21.15.93:3300/0 conn(0x7f446415f0b0 0x7f44641516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.144+0000 7f446b839640 1 --2- 172.21.15.81:0/3422360735 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f446415bfc0 0x7f446415eb70 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.144+0000 7f446b839640 1 --2- 172.21.15.81:0/3422360735 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44640ce0d0 0x7f44640caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.144+0000 7f446b839640 1 -- 172.21.15.81:0/3422360735 >> 172.21.15.81:0/3422360735 conn(0x7f44640cb580 msgr2=0x7f44640cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:03.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.144+0000 7f446b839640 1 -- 172.21.15.81:0/3422360735 shutdown_connections 2023-11-08T16:30:03.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.144+0000 7f446b839640 1 -- 172.21.15.81:0/3422360735 wait complete. 2023-11-08T16:30:03.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.146+0000 7f446b839640 1 Processor -- start 2023-11-08T16:30:03.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.146+0000 7f446b839640 1 -- start start 2023-11-08T16:30:03.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.147+0000 7f446b839640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44640ce0d0 0x7f4464155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.147+0000 7f446b839640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f446415bfc0 0x7f446406cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.147+0000 7f446b839640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f446415f0b0 0x7f446406d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.147+0000 7f446b839640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44640c4f90 con 0x7f446415bfc0 2023-11-08T16:30:03.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.147+0000 7f446b839640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44641f2520 con 0x7f44640ce0d0 2023-11-08T16:30:03.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.147+0000 7f446b839640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44640c32b0 con 0x7f446415f0b0 2023-11-08T16:30:03.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.147+0000 7f4468dd4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f446415bfc0 0x7f446406cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.148+0000 7f4468dd4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f446415bfc0 0x7f446406cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58970/0 (socket says 172.21.15.81:58970) 2023-11-08T16:30:03.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.148+0000 7f4468dd4640 1 -- 172.21.15.81:0/3324451739 learned_addr learned my addr 172.21.15.81:0/3324451739 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:03.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.148+0000 7f44695d5640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44640ce0d0 0x7f4464155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.148+0000 7f4469dd6640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f446415f0b0 0x7f446406d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.148+0000 7f4468dd4640 1 -- 172.21.15.81:0/3324451739 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f446415f0b0 msgr2=0x7f446406d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.148+0000 7f4468dd4640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f446415f0b0 0x7f446406d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.148+0000 7f4468dd4640 1 -- 172.21.15.81:0/3324451739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44640ce0d0 msgr2=0x7f4464155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.148+0000 7f4468dd4640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44640ce0d0 0x7f4464155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.148+0000 7f4468dd4640 1 -- 172.21.15.81:0/3324451739 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44540094d0 con 0x7f446415bfc0 2023-11-08T16:30:03.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.148+0000 7f4468dd4640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f446415bfc0 0x7f446406cb30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f445402b950 tx=0x7f44540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:03.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.149+0000 7f4459ffb640 1 -- 172.21.15.81:0/3324451739 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f445403ad30 con 0x7f446415bfc0 2023-11-08T16:30:03.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.149+0000 7f4459ffb640 1 -- 172.21.15.81:0/3324451739 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44540443f0 con 0x7f446415bfc0 2023-11-08T16:30:03.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.149+0000 7f4459ffb640 1 -- 172.21.15.81:0/3324451739 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44540447a0 con 0x7f446415bfc0 2023-11-08T16:30:03.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.149+0000 7f446b839640 1 -- 172.21.15.81:0/3324451739 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44640c3590 con 0x7f446415bfc0 2023-11-08T16:30:03.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.149+0000 7f446b839640 1 -- 172.21.15.81:0/3324451739 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44640c5f50 con 0x7f446415bfc0 2023-11-08T16:30:03.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.150+0000 7f446b839640 1 -- 172.21.15.81:0/3324451739 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f446404ece0 con 0x7f446415bfc0 2023-11-08T16:30:03.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.151+0000 7f4459ffb640 1 -- 172.21.15.81:0/3324451739 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f445404d450 con 0x7f446415bfc0 2023-11-08T16:30:03.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.152+0000 7f4459ffb640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f444806ae70 0x7f444806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.152+0000 7f4459ffb640 1 -- 172.21.15.81:0/3324451739 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f445402d040 con 0x7f446415bfc0 2023-11-08T16:30:03.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.152+0000 7f44695d5640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f444806ae70 0x7f444806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.152+0000 7f44695d5640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f444806ae70 0x7f444806d330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f446000a980 tx=0x7f4460008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:03.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.157+0000 7f4459ffb640 1 -- 172.21.15.81:0/3324451739 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44540ade60 con 0x7f446415bfc0 2023-11-08T16:30:03.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.349+0000 7f446b839640 1 -- 172.21.15.81:0/3324451739 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f44640ce6b0 con 0x7f446415bfc0 2023-11-08T16:30:03.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.349+0000 7f4459ffb640 1 -- 172.21.15.81:0/3324451739 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4454035020 con 0x7f446415bfc0 2023-11-08T16:30:03.351 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:30:03.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 -- 172.21.15.81:0/3324451739 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f444806ae70 msgr2=0x7f444806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f444806ae70 0x7f444806d330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f446000a980 tx=0x7f4460008040 comp rx=0 tx=0).stop 2023-11-08T16:30:03.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 -- 172.21.15.81:0/3324451739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f446415bfc0 msgr2=0x7f446406cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f446415bfc0 0x7f446406cb30 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f445402b950 tx=0x7f44540046c0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 -- 172.21.15.81:0/3324451739 shutdown_connections 2023-11-08T16:30:03.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f444806ae70 0x7f444806d330 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f446415f0b0 0x7f446406d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f446415bfc0 0x7f446406cb30 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 --2- 172.21.15.81:0/3324451739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44640ce0d0 0x7f4464155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 -- 172.21.15.81:0/3324451739 >> 172.21.15.81:0/3324451739 conn(0x7f44640cb580 msgr2=0x7f446406fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:03.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 -- 172.21.15.81:0/3324451739 shutdown_connections 2023-11-08T16:30:03.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.352+0000 7f446b839640 1 -- 172.21.15.81:0/3324451739 wait complete. 2023-11-08T16:30:03.372 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:30:03.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.559+0000 7fe9b616e640 1 Processor -- start 2023-11-08T16:30:03.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.559+0000 7fe9b616e640 1 -- start start 2023-11-08T16:30:03.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.561+0000 7fe9b616e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe9b0159370 0x7fe9b015a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.561+0000 7fe9b616e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe9b0159750 0x7fe9b00cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.561+0000 7fe9b616e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b00cce40 con 0x7fe9b00cc950 2023-11-08T16:30:03.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.561+0000 7fe9b616e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b00cd120 con 0x7fe9b0159750 2023-11-08T16:30:03.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.561+0000 7fe9b616e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b0159bf0 con 0x7fe9b0159370 2023-11-08T16:30:03.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.561+0000 7fe9b496b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe9b0159750 0x7fe9b00cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.561+0000 7fe9b596d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe9b00cc950 0x7fe9b006bfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60186/0 (socket says 172.21.15.81:60186) 2023-11-08T16:30:03.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.561+0000 7fe9b596d640 1 -- 172.21.15.81:0/556641579 learned_addr learned my addr 172.21.15.81:0/556641579 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:03.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.561+0000 7fe9b516c640 1 --2- 172.21.15.81:0/556641579 >> v2:172.21.15.93:3300/0 conn(0x7fe9b0159370 0x7fe9b015a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.562+0000 7fe9affff640 1 -- 172.21.15.81:0/556641579 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323388624 0 0) 0x7fe9b00cce40 con 0x7fe9b00cc950 2023-11-08T16:30:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.562+0000 7fe9affff640 1 -- 172.21.15.81:0/556641579 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe998003520 con 0x7fe9b00cc950 2023-11-08T16:30:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.562+0000 7fe9b496b640 1 -- 172.21.15.81:0/556641579 >> v2:172.21.15.93:3300/0 conn(0x7fe9b0159370 msgr2=0x7fe9b015a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:03.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.562+0000 7fe9b496b640 1 --2- 172.21.15.81:0/556641579 >> v2:172.21.15.93:3300/0 conn(0x7fe9b0159370 0x7fe9b015a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.562+0000 7fe9b496b640 1 -- 172.21.15.81:0/556641579 >> v1:172.21.15.81:6789/0 conn(0x7fe9b00cc950 legacy=0x7fe9b006bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.562+0000 7fe9b496b640 1 -- 172.21.15.81:0/556641579 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b0159ed0 con 0x7fe9b0159750 2023-11-08T16:30:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.563+0000 7fe9b496b640 1 --2- 172.21.15.81:0/556641579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b0159750 0x7fe9b00cc410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00cf8c0 tx=0x7fe9a402cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729ea1f1cebb63c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.563+0000 7fe9affff640 1 -- 172.21.15.81:0/556641579 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a402dab0 con 0x7fe9b0159750 2023-11-08T16:30:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.563+0000 7fe9affff640 1 -- 172.21.15.81:0/556641579 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a402dc10 con 0x7fe9b0159750 2023-11-08T16:30:03.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.563+0000 7fe9affff640 1 -- 172.21.15.81:0/556641579 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4035440 con 0x7fe9b0159750 2023-11-08T16:30:03.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.564+0000 7fe9b616e640 1 -- 172.21.15.81:0/556641579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b0159750 msgr2=0x7fe9b00cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.564+0000 7fe9b616e640 1 --2- 172.21.15.81:0/556641579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b0159750 0x7fe9b00cc410 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00cf8c0 tx=0x7fe9a402cd00 comp rx=0 tx=0).stop 2023-11-08T16:30:03.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.564+0000 7fe9b616e640 1 -- 172.21.15.81:0/556641579 shutdown_connections 2023-11-08T16:30:03.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.564+0000 7fe9b616e640 1 --2- 172.21.15.81:0/556641579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b0159750 0x7fe9b00cc410 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.564+0000 7fe9b616e640 1 --2- 172.21.15.81:0/556641579 >> v2:172.21.15.93:3300/0 conn(0x7fe9b0159370 0x7fe9b015a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.564+0000 7fe9b616e640 1 -- 172.21.15.81:0/556641579 >> 172.21.15.81:0/556641579 conn(0x7fe9b00ce8d0 msgr2=0x7fe9b00cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:03.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.564+0000 7fe9b616e640 1 -- 172.21.15.81:0/556641579 shutdown_connections 2023-11-08T16:30:03.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.564+0000 7fe9b616e640 1 -- 172.21.15.81:0/556641579 wait complete. 2023-11-08T16:30:03.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.565+0000 7fe9b616e640 1 Processor -- start 2023-11-08T16:30:03.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.566+0000 7fe9b616e640 1 -- start start 2023-11-08T16:30:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.566+0000 7fe9b616e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe9b00cc950 0x7fe9b00c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b616e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9b0159370 0x7fe9b014a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b616e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b0159750 0x7fe9b00c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b616e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b0156570 con 0x7fe9b0159750 2023-11-08T16:30:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b616e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00b3bc0 con 0x7fe9b0159370 2023-11-08T16:30:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b616e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b00b3ea0 con 0x7fe9b00cc950 2023-11-08T16:30:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b596d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b0159750 0x7fe9b00c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b596d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b0159750 0x7fe9b00c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58996/0 (socket says 172.21.15.81:58996) 2023-11-08T16:30:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b596d640 1 -- 172.21.15.81:0/3966400457 learned_addr learned my addr 172.21.15.81:0/3966400457 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b596d640 1 -- 172.21.15.81:0/3966400457 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe9b00cc950 msgr2=0x7fe9b00c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b596d640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe9b00cc950 0x7fe9b00c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b596d640 1 -- 172.21.15.81:0/3966400457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9b0159370 msgr2=0x7fe9b014a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b596d640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9b0159370 0x7fe9b014a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.567+0000 7fe9b596d640 1 -- 172.21.15.81:0/3966400457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a400a540 con 0x7fe9b0159750 2023-11-08T16:30:03.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.568+0000 7fe9b596d640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b0159750 0x7fe9b00c3790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800c430 tx=0x7fe9a800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:03.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.568+0000 7fe9adffb640 1 -- 172.21.15.81:0/3966400457 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8005ce0 con 0x7fe9b0159750 2023-11-08T16:30:03.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.568+0000 7fe9adffb640 1 -- 172.21.15.81:0/3966400457 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8005e40 con 0x7fe9b0159750 2023-11-08T16:30:03.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.568+0000 7fe9adffb640 1 -- 172.21.15.81:0/3966400457 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80072f0 con 0x7fe9b0159750 2023-11-08T16:30:03.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.568+0000 7fe9b616e640 1 -- 172.21.15.81:0/3966400457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b00b4180 con 0x7fe9b0159750 2023-11-08T16:30:03.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.568+0000 7fe9b616e640 1 -- 172.21.15.81:0/3966400457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b00c4f60 con 0x7fe9b0159750 2023-11-08T16:30:03.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.569+0000 7fe9b616e640 1 -- 172.21.15.81:0/3966400457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b00af8a0 con 0x7fe9b0159750 2023-11-08T16:30:03.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.570+0000 7fe9adffb640 1 -- 172.21.15.81:0/3966400457 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe9a8007450 con 0x7fe9b0159750 2023-11-08T16:30:03.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.571+0000 7fe9adffb640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe99c06ae70 0x7fe99c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.571+0000 7fe9adffb640 1 -- 172.21.15.81:0/3966400457 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe9a8088d80 con 0x7fe9b0159750 2023-11-08T16:30:03.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.571+0000 7fe9b516c640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe99c06ae70 0x7fe99c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.572+0000 7fe9b516c640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe99c06ae70 0x7fe99c06d330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00070b0 tx=0x7fe9a0007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:03.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.575+0000 7fe9adffb640 1 -- 172.21.15.81:0/3966400457 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe9b00af8a0 con 0x7fe9b0159750 2023-11-08T16:30:03.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.764+0000 7fe9b616e640 1 -- 172.21.15.81:0/3966400457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe9b00af8a0 con 0x7fe9b0159750 2023-11-08T16:30:03.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.765+0000 7fe9adffb640 1 -- 172.21.15.81:0/3966400457 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe9b00af8a0 con 0x7fe9b0159750 2023-11-08T16:30:03.767 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:30:03.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 -- 172.21.15.81:0/3966400457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe99c06ae70 msgr2=0x7fe99c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe99c06ae70 0x7fe99c06d330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe9a00070b0 tx=0x7fe9a0007040 comp rx=0 tx=0).stop 2023-11-08T16:30:03.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 -- 172.21.15.81:0/3966400457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b0159750 msgr2=0x7fe9b00c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b0159750 0x7fe9b00c3790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800c430 tx=0x7fe9a800c8f0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 -- 172.21.15.81:0/3966400457 shutdown_connections 2023-11-08T16:30:03.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe99c06ae70 0x7fe99c06d330 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9b0159750 0x7fe9b00c3790 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9b0159370 0x7fe9b014a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 --2- 172.21.15.81:0/3966400457 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe9b00cc950 0x7fe9b00c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 -- 172.21.15.81:0/3966400457 >> 172.21.15.81:0/3966400457 conn(0x7fe9b00ce8d0 msgr2=0x7fe9b01580f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:03.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 -- 172.21.15.81:0/3966400457 shutdown_connections 2023-11-08T16:30:03.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.768+0000 7fe9b616e640 1 -- 172.21.15.81:0/3966400457 wait complete. 2023-11-08T16:30:03.787 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:30:03.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.989+0000 7f28aaee5640 1 Processor -- start 2023-11-08T16:30:03.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.989+0000 7f28aaee5640 1 -- start start 2023-11-08T16:30:03.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.990+0000 7f28aaee5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f28a40d2a10 0x7f28a40b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.990+0000 7f28aaee5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a40d2120 con 0x7f28a40d25b0 2023-11-08T16:30:03.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.990+0000 7f28aaee5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28a40d1b40 con 0x7f28a415d140 2023-11-08T16:30:03.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.990+0000 7f28aaee5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28a40b2600 con 0x7f28a40d2a10 2023-11-08T16:30:03.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.991+0000 7f28a3fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f28a40d25b0 0x7f28a40cf850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60192/0 (socket says 172.21.15.81:60192) 2023-11-08T16:30:03.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.991+0000 7f28a3fff640 1 -- 172.21.15.81:0/1362157000 learned_addr learned my addr 172.21.15.81:0/1362157000 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:03.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.991+0000 7f28a9482640 1 --2- 172.21.15.81:0/1362157000 >> v2:172.21.15.93:3300/0 conn(0x7f28a40d2a10 0x7f28a40b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:03.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.991+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580106491 0 0) 0x7f28a40d2120 con 0x7f28a40d25b0 2023-11-08T16:30:03.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.991+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2890003520 con 0x7f28a40d25b0 2023-11-08T16:30:03.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.991+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 164399894 0 0) 0x7f2890003520 con 0x7f28a40d25b0 2023-11-08T16:30:03.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.991+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 >> v2:172.21.15.93:3300/0 conn(0x7f28a40d2a10 msgr2=0x7f28a40b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:03.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.991+0000 7f28a37fe640 1 --2- 172.21.15.81:0/1362157000 >> v2:172.21.15.93:3300/0 conn(0x7f28a40d2a10 0x7f28a40b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:03.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.991+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 >> v1:172.21.15.93:6789/0 conn(0x7f28a415d140 legacy=0x7f28a40cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.991+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40ce350 con 0x7f28a40d25b0 2023-11-08T16:30:03.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.991+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f288c001670 con 0x7f28a40d25b0 2023-11-08T16:30:03.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 >> v1:172.21.15.81:6789/0 conn(0x7f28a40d25b0 legacy=0x7f28a40cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:03.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a37fe640 1 --2- 172.21.15.81:0/1362157000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28900039c0 0x7f2890003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a37fe640 1 --2- 172.21.15.81:0/1362157000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28900047c0 0x7f2890004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a37fe640 1 --2- 172.21.15.81:0/1362157000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2890005180 0x7f289000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:03.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a40b2600 con 0x7f2890005180 2023-11-08T16:30:03.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289000c060 con 0x7f28900039c0 2023-11-08T16:30:04.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289000c220 con 0x7f28900047c0 2023-11-08T16:30:04.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a9482640 1 --2- 172.21.15.81:0/1362157000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2890005180 0x7f289000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a3fff640 1 --2- 172.21.15.81:0/1362157000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28900047c0 0x7f2890004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a9482640 1 -- 172.21.15.81:0/1362157000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28900047c0 msgr2=0x7f2890004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:04.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a9482640 1 --2- 172.21.15.81:0/1362157000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28900047c0 0x7f2890004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a9482640 1 -- 172.21.15.81:0/1362157000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28900039c0 msgr2=0x7f2890003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:04.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a9482640 1 --2- 172.21.15.81:0/1362157000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28900039c0 0x7f2890003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.992+0000 7f28a9482640 1 -- 172.21.15.81:0/1362157000 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2890003520 con 0x7f2890005180 2023-11-08T16:30:04.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.993+0000 7f28a9482640 1 -- 172.21.15.81:0/1362157000 reap_dead start 2023-11-08T16:30:04.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.994+0000 7f28a9482640 1 --2- 172.21.15.81:0/1362157000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2890005180 0x7f289000b980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f289c009f90 tx=0x7f28900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e813a45122cd9f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:04.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.994+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c02cc20 con 0x7f2890005180 2023-11-08T16:30:04.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.994+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f289c00f040 con 0x7f2890005180 2023-11-08T16:30:04.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.994+0000 7f28a37fe640 1 -- 172.21.15.81:0/1362157000 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f289c003990 con 0x7f2890005180 2023-11-08T16:30:04.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.994+0000 7f28aaee5640 1 -- 172.21.15.81:0/1362157000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2890005180 msgr2=0x7f289000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.994+0000 7f28aaee5640 1 --2- 172.21.15.81:0/1362157000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2890005180 0x7f289000b980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f289c009f90 tx=0x7f28900057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.994+0000 7f28aaee5640 1 -- 172.21.15.81:0/1362157000 shutdown_connections 2023-11-08T16:30:04.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.994+0000 7f28aaee5640 1 --2- 172.21.15.81:0/1362157000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2890005180 0x7f289000b980 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.994+0000 7f28aaee5640 1 -- 172.21.15.81:0/1362157000 >> 172.21.15.81:0/1362157000 conn(0x7f28a406a950 msgr2=0x7f28a406ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:04.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.995+0000 7f28aaee5640 1 -- 172.21.15.81:0/1362157000 shutdown_connections 2023-11-08T16:30:04.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.995+0000 7f28aaee5640 1 -- 172.21.15.81:0/1362157000 wait complete. 2023-11-08T16:30:04.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.996+0000 7f28aaee5640 1 Processor -- start 2023-11-08T16:30:04.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.996+0000 7f28aaee5640 1 -- start start 2023-11-08T16:30:04.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.997+0000 7f28aaee5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2890005180 0x7f28a41f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.997+0000 7f28aaee5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28a40b3310 0x7f28a4152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.997+0000 7f28aaee5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28a40cbcd0 0x7f28a40cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.997+0000 7f28aaee5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a40cb840 con 0x7f28a40b3310 2023-11-08T16:30:04.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.997+0000 7f28aaee5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28a40b3ab0 con 0x7f2890005180 2023-11-08T16:30:04.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.997+0000 7f28aaee5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28a41534c0 con 0x7f28a40cbcd0 2023-11-08T16:30:04.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.998+0000 7f28a3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28a40b3310 0x7f28a4152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.998+0000 7f28a3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28a40b3310 0x7f28a4152f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59024/0 (socket says 172.21.15.81:59024) 2023-11-08T16:30:04.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.998+0000 7f28a3fff640 1 -- 172.21.15.81:0/2907712693 learned_addr learned my addr 172.21.15.81:0/2907712693 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:04.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.998+0000 7f28a9482640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28a40cbcd0 0x7f28a40cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.998+0000 7f28a3fff640 1 -- 172.21.15.81:0/2907712693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28a40cbcd0 msgr2=0x7f28a40cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.998+0000 7f28a3fff640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28a40cbcd0 0x7f28a40cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.998+0000 7f28a3fff640 1 -- 172.21.15.81:0/2907712693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2890005180 msgr2=0x7f28a41f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.998+0000 7f28a3fff640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2890005180 0x7f28a41f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.998+0000 7f28a3fff640 1 -- 172.21.15.81:0/2907712693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28a40b37d0 con 0x7f28a40b3310 2023-11-08T16:30:04.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.998+0000 7f28a3fff640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28a40b3310 0x7f28a4152f80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f288c00d7d0 tx=0x7f288c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:04.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.999+0000 7f28a17fa640 1 -- 172.21.15.81:0/2907712693 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c0074e0 con 0x7f28a40b3310 2023-11-08T16:30:04.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.999+0000 7f28a17fa640 1 -- 172.21.15.81:0/2907712693 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c007640 con 0x7f28a40b3310 2023-11-08T16:30:04.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.999+0000 7f28a17fa640 1 -- 172.21.15.81:0/2907712693 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c0049e0 con 0x7f28a40b3310 2023-11-08T16:30:04.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.999+0000 7f28aaee5640 1 -- 172.21.15.81:0/2907712693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28a40cb1f0 con 0x7f28a40b3310 2023-11-08T16:30:04.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:03.999+0000 7f28aaee5640 1 -- 172.21.15.81:0/2907712693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28a4159d20 con 0x7f28a40b3310 2023-11-08T16:30:04.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.000+0000 7f28aaee5640 1 -- 172.21.15.81:0/2907712693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28a40cb050 con 0x7f28a40b3310 2023-11-08T16:30:04.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.001+0000 7f28a17fa640 1 -- 172.21.15.81:0/2907712693 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f288c004b40 con 0x7f28a40b3310 2023-11-08T16:30:04.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.002+0000 7f28a17fa640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f288406ae70 0x7f288406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.002+0000 7f28a17fa640 1 -- 172.21.15.81:0/2907712693 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f288c088f00 con 0x7f28a40b3310 2023-11-08T16:30:04.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.002+0000 7f28a8c81640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f288406ae70 0x7f288406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.003+0000 7f28a8c81640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f288406ae70 0x7f288406d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2894007fd0 tx=0x7f28940079a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:04.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.006+0000 7f28a17fa640 1 -- 172.21.15.81:0/2907712693 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28a40cb050 con 0x7f28a40b3310 2023-11-08T16:30:04.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.194+0000 7f28aaee5640 1 -- 172.21.15.81:0/2907712693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f28a40cb050 con 0x7f28a40b3310 2023-11-08T16:30:04.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.195+0000 7f28a17fa640 1 -- 172.21.15.81:0/2907712693 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f28a40cb050 con 0x7f28a40b3310 2023-11-08T16:30:04.197 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:30:04.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 -- 172.21.15.81:0/2907712693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f288406ae70 msgr2=0x7f288406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f288406ae70 0x7f288406d330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2894007fd0 tx=0x7f28940079a0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 -- 172.21.15.81:0/2907712693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28a40b3310 msgr2=0x7f28a4152f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28a40b3310 0x7f28a4152f80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f288c00d7d0 tx=0x7f288c00dc90 comp rx=0 tx=0).stop 2023-11-08T16:30:04.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 -- 172.21.15.81:0/2907712693 shutdown_connections 2023-11-08T16:30:04.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f288406ae70 0x7f288406d330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28a40cbcd0 0x7f28a40cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28a40b3310 0x7f28a4152f80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 --2- 172.21.15.81:0/2907712693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2890005180 0x7f28a41f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 -- 172.21.15.81:0/2907712693 >> 172.21.15.81:0/2907712693 conn(0x7f28a406a950 msgr2=0x7f28a40d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:04.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 -- 172.21.15.81:0/2907712693 shutdown_connections 2023-11-08T16:30:04.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.197+0000 7f28aaee5640 1 -- 172.21.15.81:0/2907712693 wait complete. 2023-11-08T16:30:04.217 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:30:04.218 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:30:04.277 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:30:04.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.483+0000 7f1eac24f640 1 Processor -- start 2023-11-08T16:30:04.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.483+0000 7f1eac24f640 1 -- start start 2023-11-08T16:30:04.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.484+0000 7f1eac24f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1ea40ade50 0x7f1ea40ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.484+0000 7f1eac24f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1ea415ea60 0x7f1ea415eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.484+0000 7f1eac24f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea406b630 con 0x7f1ea415a680 2023-11-08T16:30:04.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.484+0000 7f1eac24f640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea40b1860 con 0x7f1ea415ea60 2023-11-08T16:30:04.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.484+0000 7f1eac24f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea40ca9c0 con 0x7f1ea40ade50 2023-11-08T16:30:04.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.485+0000 7f1eaa7ec640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1ea415ea60 0x7f1ea415eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.485+0000 7f1eaa7ec640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1ea415ea60 0x7f1ea415eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59028/0 (socket says 172.21.15.81:59028) 2023-11-08T16:30:04.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.485+0000 7f1eaa7ec640 1 -- 172.21.15.81:0/488637483 learned_addr learned my addr 172.21.15.81:0/488637483 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:04.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.485+0000 7f1eaa7ec640 1 -- 172.21.15.81:0/488637483 >> v2:172.21.15.93:3300/0 conn(0x7f1ea40ade50 msgr2=0x7f1ea40ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:04.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.485+0000 7f1ea9feb640 1 --2- 172.21.15.81:0/488637483 >> v2:172.21.15.93:3300/0 conn(0x7f1ea40ade50 0x7f1ea40ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.485+0000 7f1eaa7ec640 1 --2- 172.21.15.81:0/488637483 >> v2:172.21.15.93:3300/0 conn(0x7f1ea40ade50 0x7f1ea40ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.485+0000 7f1eaa7ec640 1 -- 172.21.15.81:0/488637483 >> v1:172.21.15.93:6789/0 conn(0x7f1ea415a680 legacy=0x7f1ea415ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:04.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.485+0000 7f1eaa7ec640 1 -- 172.21.15.81:0/488637483 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea40cabb0 con 0x7f1ea415ea60 2023-11-08T16:30:04.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.485+0000 7f1ea9feb640 1 --2- 172.21.15.81:0/488637483 >> v2:172.21.15.93:3300/0 conn(0x7f1ea40ade50 0x7f1ea40ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:04.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.486+0000 7f1eaa7ec640 1 --2- 172.21.15.81:0/488637483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea415ea60 0x7f1ea415eef0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4067770 tx=0x7f1e9c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4504fe5781999930 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:04.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.486+0000 7f1ea8fe9640 1 -- 172.21.15.81:0/488637483 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c02ca80 con 0x7f1ea415ea60 2023-11-08T16:30:04.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.487+0000 7f1ea8fe9640 1 -- 172.21.15.81:0/488637483 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9c02cbe0 con 0x7f1ea415ea60 2023-11-08T16:30:04.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.487+0000 7f1ea8fe9640 1 -- 172.21.15.81:0/488637483 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c0343f0 con 0x7f1ea415ea60 2023-11-08T16:30:04.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.487+0000 7f1eac24f640 1 -- 172.21.15.81:0/488637483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea415ea60 msgr2=0x7f1ea415eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.487+0000 7f1eac24f640 1 --2- 172.21.15.81:0/488637483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea415ea60 0x7f1ea415eef0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4067770 tx=0x7f1e9c02bee0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.487+0000 7f1eac24f640 1 -- 172.21.15.81:0/488637483 shutdown_connections 2023-11-08T16:30:04.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.487+0000 7f1eac24f640 1 --2- 172.21.15.81:0/488637483 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea415ea60 0x7f1ea415eef0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.487+0000 7f1eac24f640 1 --2- 172.21.15.81:0/488637483 >> v2:172.21.15.93:3300/0 conn(0x7f1ea40ade50 0x7f1ea40ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.487+0000 7f1eac24f640 1 -- 172.21.15.81:0/488637483 >> 172.21.15.81:0/488637483 conn(0x7f1ea40cb490 msgr2=0x7f1ea40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:04.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.487+0000 7f1eac24f640 1 -- 172.21.15.81:0/488637483 shutdown_connections 2023-11-08T16:30:04.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.487+0000 7f1eac24f640 1 -- 172.21.15.81:0/488637483 wait complete. 2023-11-08T16:30:04.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.489+0000 7f1eac24f640 1 Processor -- start 2023-11-08T16:30:04.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.489+0000 7f1eac24f640 1 -- start start 2023-11-08T16:30:04.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.490+0000 7f1eac24f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ea40ade50 0x7f1ea40a9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.490+0000 7f1eac24f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea415a680 0x7f1ea4148390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.490+0000 7f1eac24f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1ea415ea60 0x7f1ea415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.490+0000 7f1eac24f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea414acd0 con 0x7f1ea415a680 2023-11-08T16:30:04.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.490+0000 7f1eac24f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea40aa0c0 con 0x7f1ea40ade50 2023-11-08T16:30:04.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.490+0000 7f1eac24f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea41f6880 con 0x7f1ea415ea60 2023-11-08T16:30:04.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.490+0000 7f1ea97ea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea415a680 0x7f1ea4148390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.490+0000 7f1ea97ea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea415a680 0x7f1ea4148390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59034/0 (socket says 172.21.15.81:59034) 2023-11-08T16:30:04.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.490+0000 7f1ea97ea640 1 -- 172.21.15.81:0/2577647304 learned_addr learned my addr 172.21.15.81:0/2577647304 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:04.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1eaa7ec640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1ea415ea60 0x7f1ea415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1ea97ea640 1 -- 172.21.15.81:0/2577647304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1ea415ea60 msgr2=0x7f1ea415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1ea97ea640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1ea415ea60 0x7f1ea415cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1ea97ea640 1 -- 172.21.15.81:0/2577647304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ea40ade50 msgr2=0x7f1ea40a9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1ea9feb640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ea40ade50 0x7f1ea40a9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1ea97ea640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ea40ade50 0x7f1ea40a9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1ea97ea640 1 -- 172.21.15.81:0/2577647304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e9c009b30 con 0x7f1ea415a680 2023-11-08T16:30:04.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1ea9feb640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ea40ade50 0x7f1ea40a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:04.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1ea97ea640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea415a680 0x7f1ea4148390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400c460 tx=0x7f1e9400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:04.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1e9a7fc640 1 -- 172.21.15.81:0/2577647304 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94012830 con 0x7f1ea415a680 2023-11-08T16:30:04.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1e9a7fc640 1 -- 172.21.15.81:0/2577647304 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e940129d0 con 0x7f1ea415a680 2023-11-08T16:30:04.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1eac24f640 1 -- 172.21.15.81:0/2577647304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea41f6b60 con 0x7f1ea415a680 2023-11-08T16:30:04.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.491+0000 7f1e9a7fc640 1 -- 172.21.15.81:0/2577647304 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e94012d80 con 0x7f1ea415a680 2023-11-08T16:30:04.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.492+0000 7f1eac24f640 1 -- 172.21.15.81:0/2577647304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea41f7120 con 0x7f1ea415a680 2023-11-08T16:30:04.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.492+0000 7f1eac24f640 1 -- 172.21.15.81:0/2577647304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea41f7340 con 0x7f1ea415a680 2023-11-08T16:30:04.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.494+0000 7f1e9a7fc640 1 -- 172.21.15.81:0/2577647304 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1e9401a3b0 con 0x7f1ea415a680 2023-11-08T16:30:04.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.495+0000 7f1e9a7fc640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1e8806ae70 0x7f1e8806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.495+0000 7f1e9a7fc640 1 -- 172.21.15.81:0/2577647304 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1e940893d0 con 0x7f1ea415a680 2023-11-08T16:30:04.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.495+0000 7f1ea9feb640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1e8806ae70 0x7f1e8806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.495+0000 7f1ea9feb640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1e8806ae70 0x7f1e8806d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40ad580 tx=0x7f1ea0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:04.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.499+0000 7f1e9a7fc640 1 -- 172.21.15.81:0/2577647304 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1e94054350 con 0x7f1ea415a680 2023-11-08T16:30:04.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.684+0000 7f1eac24f640 1 -- 172.21.15.81:0/2577647304 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ea414ae10 con 0x7f1ea415a680 2023-11-08T16:30:04.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.686+0000 7f1e9a7fc640 1 -- 172.21.15.81:0/2577647304 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f1e94057970 con 0x7f1ea415a680 2023-11-08T16:30:04.689 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:04.689 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:30:04.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 -- 172.21.15.81:0/2577647304 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1e8806ae70 msgr2=0x7f1e8806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1e8806ae70 0x7f1e8806d330 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40ad580 tx=0x7f1ea0008040 comp rx=0 tx=0).stop 2023-11-08T16:30:04.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 -- 172.21.15.81:0/2577647304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea415a680 msgr2=0x7f1ea4148390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea415a680 0x7f1ea4148390 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1e9400c460 tx=0x7f1e9400c920 comp rx=0 tx=0).stop 2023-11-08T16:30:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 -- 172.21.15.81:0/2577647304 shutdown_connections 2023-11-08T16:30:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1e8806ae70 0x7f1e8806d330 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1ea415ea60 0x7f1ea415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1ea415a680 0x7f1ea4148390 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 --2- 172.21.15.81:0/2577647304 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ea40ade50 0x7f1ea40a9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 -- 172.21.15.81:0/2577647304 >> 172.21.15.81:0/2577647304 conn(0x7f1ea40cb490 msgr2=0x7f1ea4158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 -- 172.21.15.81:0/2577647304 shutdown_connections 2023-11-08T16:30:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.689+0000 7f1eac24f640 1 -- 172.21.15.81:0/2577647304 wait complete. 2023-11-08T16:30:04.709 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:30:04.709 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:30:04.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.922+0000 7f6c5e178640 1 Processor -- start 2023-11-08T16:30:04.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.922+0000 7f6c5e178640 1 -- start start 2023-11-08T16:30:04.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.923+0000 7f6c5e178640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6c580ce390 0x7f6c5815e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.923+0000 7f6c5e178640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c5806be20 0x7f6c580ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.923+0000 7f6c5e178640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5815e8d0 con 0x7f6c580c9200 2023-11-08T16:30:04.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.923+0000 7f6c5e178640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c580ca2f0 con 0x7f6c580ce390 2023-11-08T16:30:04.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.923+0000 7f6c5e178640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580ce770 con 0x7f6c5806be20 2023-11-08T16:30:04.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.924+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c5806be20 0x7f6c580ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.924+0000 7f6c57fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c5806be20 0x7f6c580ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54902/0 (socket says 172.21.15.81:54902) 2023-11-08T16:30:04.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.924+0000 7f6c57fff640 1 -- 172.21.15.81:0/614891242 learned_addr learned my addr 172.21.15.81:0/614891242 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:04.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.924+0000 7f6c577fe640 1 --2- 172.21.15.81:0/614891242 >> v2:172.21.15.93:3300/0 conn(0x7f6c580ce390 0x7f6c5815e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.925+0000 7f6c57fff640 1 -- 172.21.15.81:0/614891242 >> v2:172.21.15.93:3300/0 conn(0x7f6c580ce390 msgr2=0x7f6c5815e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:04.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.925+0000 7f6c57fff640 1 --2- 172.21.15.81:0/614891242 >> v2:172.21.15.93:3300/0 conn(0x7f6c580ce390 0x7f6c5815e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.925+0000 7f6c57fff640 1 -- 172.21.15.81:0/614891242 >> v1:172.21.15.93:6789/0 conn(0x7f6c580c9200 legacy=0x7f6c5806b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.925+0000 7f6c57fff640 1 -- 172.21.15.81:0/614891242 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580c96e0 con 0x7f6c5806be20 2023-11-08T16:30:04.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.925+0000 7f6c56ffd640 1 -- 172.21.15.81:0/614891242 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3234940388 0 0) 0x7f6c40001110 con 0x7f6c580c9200 2023-11-08T16:30:04.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.926+0000 7f6c57fff640 1 --2- 172.21.15.81:0/614891242 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c5806be20 0x7f6c580ac8b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009e50 tx=0x7f6c5002bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b13155b17725c60 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:04.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.927+0000 7f6c56ffd640 1 -- 172.21.15.81:0/614891242 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c5002cb90 con 0x7f6c5806be20 2023-11-08T16:30:04.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.927+0000 7f6c56ffd640 1 -- 172.21.15.81:0/614891242 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c5002ccf0 con 0x7f6c5806be20 2023-11-08T16:30:04.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.927+0000 7f6c56ffd640 1 -- 172.21.15.81:0/614891242 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c50034610 con 0x7f6c5806be20 2023-11-08T16:30:04.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.927+0000 7f6c5e178640 1 -- 172.21.15.81:0/614891242 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c5806be20 msgr2=0x7f6c580ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.927+0000 7f6c5e178640 1 --2- 172.21.15.81:0/614891242 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c5806be20 0x7f6c580ac8b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c50009e50 tx=0x7f6c5002bff0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.928+0000 7f6c5e178640 1 -- 172.21.15.81:0/614891242 shutdown_connections 2023-11-08T16:30:04.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.928+0000 7f6c5e178640 1 --2- 172.21.15.81:0/614891242 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c5806be20 0x7f6c580ac8b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.928+0000 7f6c5e178640 1 --2- 172.21.15.81:0/614891242 >> v2:172.21.15.93:3300/0 conn(0x7f6c580ce390 0x7f6c5815e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.928+0000 7f6c5e178640 1 -- 172.21.15.81:0/614891242 >> 172.21.15.81:0/614891242 conn(0x7f6c580cb580 msgr2=0x7f6c580cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:04.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.928+0000 7f6c5e178640 1 -- 172.21.15.81:0/614891242 shutdown_connections 2023-11-08T16:30:04.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.928+0000 7f6c5e178640 1 -- 172.21.15.81:0/614891242 wait complete. 2023-11-08T16:30:04.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.929+0000 7f6c5e178640 1 Processor -- start 2023-11-08T16:30:04.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.930+0000 7f6c5e178640 1 -- start start 2023-11-08T16:30:04.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.930+0000 7f6c5e178640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c5806be20 0x7f6c581e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.930+0000 7f6c5e178640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c580c9200 0x7f6c580c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.930+0000 7f6c5e178640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c580ce390 0x7f6c581e4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.930+0000 7f6c5e178640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c580c8590 con 0x7f6c580c9200 2023-11-08T16:30:04.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.930+0000 7f6c5e178640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581e46d0 con 0x7f6c5806be20 2023-11-08T16:30:04.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.930+0000 7f6c5e178640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c581e5510 con 0x7f6c580ce390 2023-11-08T16:30:04.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.931+0000 7f6c4ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c580c9200 0x7f6c580c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.931+0000 7f6c4ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c580c9200 0x7f6c580c8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59038/0 (socket says 172.21.15.81:59038) 2023-11-08T16:30:04.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.931+0000 7f6c4ffff640 1 -- 172.21.15.81:0/3205510652 learned_addr learned my addr 172.21.15.81:0/3205510652 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:04.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.931+0000 7f6c57fff640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c580ce390 0x7f6c581e4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.931+0000 7f6c577fe640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c5806be20 0x7f6c581e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.931+0000 7f6c4ffff640 1 -- 172.21.15.81:0/3205510652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c580ce390 msgr2=0x7f6c581e4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.931+0000 7f6c4ffff640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c580ce390 0x7f6c581e4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.931+0000 7f6c4ffff640 1 -- 172.21.15.81:0/3205510652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c5806be20 msgr2=0x7f6c581e59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:04.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.931+0000 7f6c4ffff640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c5806be20 0x7f6c581e59a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:04.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.931+0000 7f6c4ffff640 1 -- 172.21.15.81:0/3205510652 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c50009b30 con 0x7f6c580c9200 2023-11-08T16:30:04.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.932+0000 7f6c4ffff640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c580c9200 0x7f6c580c8050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000c780 tx=0x7f6c4000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:04.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.932+0000 7f6c54ff9640 1 -- 172.21.15.81:0/3205510652 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40005270 con 0x7f6c580c9200 2023-11-08T16:30:04.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.932+0000 7f6c54ff9640 1 -- 172.21.15.81:0/3205510652 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c40005410 con 0x7f6c580c9200 2023-11-08T16:30:04.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.932+0000 7f6c54ff9640 1 -- 172.21.15.81:0/3205510652 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000a880 con 0x7f6c580c9200 2023-11-08T16:30:04.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.932+0000 7f6c5e178640 1 -- 172.21.15.81:0/3205510652 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c581e4900 con 0x7f6c580c9200 2023-11-08T16:30:04.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.932+0000 7f6c5e178640 1 -- 172.21.15.81:0/3205510652 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c581e4ec0 con 0x7f6c580c9200 2023-11-08T16:30:04.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.933+0000 7f6c5e178640 1 -- 172.21.15.81:0/3205510652 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5804ece0 con 0x7f6c580c9200 2023-11-08T16:30:04.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.934+0000 7f6c54ff9640 1 -- 172.21.15.81:0/3205510652 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c4000a9e0 con 0x7f6c580c9200 2023-11-08T16:30:04.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.935+0000 7f6c54ff9640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c3806ae70 0x7f6c3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:04.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.935+0000 7f6c54ff9640 1 -- 172.21.15.81:0/3205510652 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6c40088a20 con 0x7f6c580c9200 2023-11-08T16:30:04.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.935+0000 7f6c577fe640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c3806ae70 0x7f6c3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:04.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.936+0000 7f6c577fe640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c3806ae70 0x7f6c3806d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c580b4880 tx=0x7f6c48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:04.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:04.941+0000 7f6c54ff9640 1 -- 172.21.15.81:0/3205510652 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c40053a50 con 0x7f6c580c9200 2023-11-08T16:30:05.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.133+0000 7f6c5e178640 1 -- 172.21.15.81:0/3205510652 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c5815e8d0 con 0x7f6c580c9200 2023-11-08T16:30:05.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.134+0000 7f6c54ff9640 1 -- 172.21.15.81:0/3205510652 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c5815e8d0 con 0x7f6c580c9200 2023-11-08T16:30:05.136 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:30:05.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 -- 172.21.15.81:0/3205510652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c3806ae70 msgr2=0x7f6c3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c3806ae70 0x7f6c3806d330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f6c580b4880 tx=0x7f6c48008040 comp rx=0 tx=0).stop 2023-11-08T16:30:05.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 -- 172.21.15.81:0/3205510652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c580c9200 msgr2=0x7f6c580c8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c580c9200 0x7f6c580c8050 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000c780 tx=0x7f6c4000cc40 comp rx=0 tx=0).stop 2023-11-08T16:30:05.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 -- 172.21.15.81:0/3205510652 shutdown_connections 2023-11-08T16:30:05.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c3806ae70 0x7f6c3806d330 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c580ce390 0x7f6c581e4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c580c9200 0x7f6c580c8050 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 --2- 172.21.15.81:0/3205510652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c5806be20 0x7f6c581e59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 -- 172.21.15.81:0/3205510652 >> 172.21.15.81:0/3205510652 conn(0x7f6c580cb580 msgr2=0x7f6c580ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:05.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 -- 172.21.15.81:0/3205510652 shutdown_connections 2023-11-08T16:30:05.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.137+0000 7f6c5e178640 1 -- 172.21.15.81:0/3205510652 wait complete. 2023-11-08T16:30:05.158 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:30:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.371+0000 7f5695464640 1 Processor -- start 2023-11-08T16:30:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.371+0000 7f5695464640 1 -- start start 2023-11-08T16:30:05.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.373+0000 7f5695464640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f56900ce390 0x7f569015e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.373+0000 7f5695464640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f569015e8d0 con 0x7f569006be20 2023-11-08T16:30:05.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.373+0000 7f5695464640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56900ca2f0 con 0x7f56900c9200 2023-11-08T16:30:05.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.373+0000 7f5695464640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56900ce770 con 0x7f56900ce390 2023-11-08T16:30:05.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.373+0000 7f568f7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f569006be20 0x7f56900ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60198/0 (socket says 172.21.15.81:60198) 2023-11-08T16:30:05.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.373+0000 7f568f7fe640 1 -- 172.21.15.81:0/2766070598 learned_addr learned my addr 172.21.15.81:0/2766070598 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:05.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568effd640 1 --2- 172.21.15.81:0/2766070598 >> v2:172.21.15.93:3300/0 conn(0x7f56900ce390 0x7f569015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1928937944 0 0) 0x7f5688001490 con 0x7f569006be20 2023-11-08T16:30:05.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5678003520 con 0x7f569006be20 2023-11-08T16:30:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 414066926 0 0) 0x7f5678003520 con 0x7f569006be20 2023-11-08T16:30:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 >> v2:172.21.15.93:3300/0 conn(0x7f56900ce390 msgr2=0x7f569015e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 --2- 172.21.15.81:0/2766070598 >> v2:172.21.15.93:3300/0 conn(0x7f56900ce390 0x7f569015e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 >> v1:172.21.15.93:6789/0 conn(0x7f56900c9200 legacy=0x7f569006b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56900c96e0 con 0x7f569006be20 2023-11-08T16:30:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f5688001b80 con 0x7f569006be20 2023-11-08T16:30:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 >> v1:172.21.15.81:6789/0 conn(0x7f569006be20 legacy=0x7f56900ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 --2- 172.21.15.81:0/2766070598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5678003a30 0x7f5678003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 --2- 172.21.15.81:0/2766070598 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56780047c0 0x7f5678004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 --2- 172.21.15.81:0/2766070598 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56780051d0 0x7f567800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56900ce770 con 0x7f5678003a30 2023-11-08T16:30:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f567800c0e0 con 0x7f56780051d0 2023-11-08T16:30:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.374+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f567800c2a0 con 0x7f56780047c0 2023-11-08T16:30:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.375+0000 7f568effd640 1 --2- 172.21.15.81:0/2766070598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5678003a30 0x7f5678003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.375+0000 7f568effd640 1 -- 172.21.15.81:0/2766070598 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56780047c0 msgr2=0x7f5678004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.375+0000 7f568effd640 1 --2- 172.21.15.81:0/2766070598 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56780047c0 0x7f5678004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.375+0000 7f568effd640 1 -- 172.21.15.81:0/2766070598 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56780051d0 msgr2=0x7f567800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.375+0000 7f568effd640 1 --2- 172.21.15.81:0/2766070598 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56780051d0 0x7f567800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.375+0000 7f568effd640 1 -- 172.21.15.81:0/2766070598 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5678003520 con 0x7f5678003a30 2023-11-08T16:30:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.375+0000 7f568f7fe640 1 -- 172.21.15.81:0/2766070598 reap_dead start 2023-11-08T16:30:05.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.376+0000 7f568effd640 1 --2- 172.21.15.81:0/2766070598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5678003a30 0x7f5678003e50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5680009ac0 tx=0x7f56780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88338a59beb5ab9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:05.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.376+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568002c710 con 0x7f5678003a30 2023-11-08T16:30:05.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.376+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f568002c870 con 0x7f5678003a30 2023-11-08T16:30:05.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.376+0000 7f568dffb640 1 -- 172.21.15.81:0/2766070598 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f568002cc00 con 0x7f5678003a30 2023-11-08T16:30:05.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.377+0000 7f5695464640 1 -- 172.21.15.81:0/2766070598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5678003a30 msgr2=0x7f5678003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.377+0000 7f5695464640 1 --2- 172.21.15.81:0/2766070598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5678003a30 0x7f5678003e50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5680009ac0 tx=0x7f56780057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.377+0000 7f5695464640 1 -- 172.21.15.81:0/2766070598 shutdown_connections 2023-11-08T16:30:05.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.377+0000 7f5695464640 1 --2- 172.21.15.81:0/2766070598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5678003a30 0x7f5678003e50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.377+0000 7f5695464640 1 -- 172.21.15.81:0/2766070598 >> 172.21.15.81:0/2766070598 conn(0x7f56900cb580 msgr2=0x7f56900cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:05.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.377+0000 7f5695464640 1 -- 172.21.15.81:0/2766070598 shutdown_connections 2023-11-08T16:30:05.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.377+0000 7f5695464640 1 -- 172.21.15.81:0/2766070598 wait complete. 2023-11-08T16:30:05.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.379+0000 7f5695464640 1 Processor -- start 2023-11-08T16:30:05.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.379+0000 7f5695464640 1 -- start start 2023-11-08T16:30:05.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.380+0000 7f5695464640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5678003a30 0x7f5690155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.380+0000 7f5695464640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56900c8100 0x7f56900b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.380+0000 7f5695464640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56900c8580 0x7f56900b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.380+0000 7f5695464640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56900c7600 con 0x7f56900c8100 2023-11-08T16:30:05.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.380+0000 7f5695464640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56900c6980 con 0x7f56900c8580 2023-11-08T16:30:05.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.380+0000 7f5695464640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56900c6c60 con 0x7f5678003a30 2023-11-08T16:30:05.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.380+0000 7f568e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56900c8100 0x7f56900b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.380+0000 7f568e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56900c8100 0x7f56900b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59062/0 (socket says 172.21.15.81:59062) 2023-11-08T16:30:05.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.380+0000 7f568e7fc640 1 -- 172.21.15.81:0/224067785 learned_addr learned my addr 172.21.15.81:0/224067785 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:05.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f568effd640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5678003a30 0x7f5690155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f568f7fe640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56900c8580 0x7f56900b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f568e7fc640 1 -- 172.21.15.81:0/224067785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5678003a30 msgr2=0x7f5690155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f568e7fc640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5678003a30 0x7f5690155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f568e7fc640 1 -- 172.21.15.81:0/224067785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56900c8580 msgr2=0x7f56900b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f568e7fc640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56900c8580 0x7f56900b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f568e7fc640 1 -- 172.21.15.81:0/224067785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56900c6f40 con 0x7f56900c8100 2023-11-08T16:30:05.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f568effd640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5678003a30 0x7f5690155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:30:05.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f568e7fc640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56900c8100 0x7f56900b1580 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f568400c430 tx=0x7f568400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:05.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f56777fe640 1 -- 172.21.15.81:0/224067785 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56840052d0 con 0x7f56900c8100 2023-11-08T16:30:05.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f56777fe640 1 -- 172.21.15.81:0/224067785 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5684005450 con 0x7f56900c8100 2023-11-08T16:30:05.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f56777fe640 1 -- 172.21.15.81:0/224067785 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5684003830 con 0x7f56900c8100 2023-11-08T16:30:05.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.381+0000 7f5695464640 1 -- 172.21.15.81:0/224067785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56900c7220 con 0x7f56900c8100 2023-11-08T16:30:05.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.382+0000 7f5695464640 1 -- 172.21.15.81:0/224067785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56900b2e10 con 0x7f56900c8100 2023-11-08T16:30:05.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.383+0000 7f5695464640 1 -- 172.21.15.81:0/224067785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f569004ece0 con 0x7f56900c8100 2023-11-08T16:30:05.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.384+0000 7f56777fe640 1 -- 172.21.15.81:0/224067785 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5684003990 con 0x7f56900c8100 2023-11-08T16:30:05.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.384+0000 7f56777fe640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f567c06ae70 0x7f567c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.385+0000 7f56777fe640 1 -- 172.21.15.81:0/224067785 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f56840888d0 con 0x7f56900c8100 2023-11-08T16:30:05.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.385+0000 7f568effd640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f567c06ae70 0x7f567c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.385+0000 7f568effd640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f567c06ae70 0x7f567c06d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5680036000 tx=0x7f5680030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:05.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.389+0000 7f56777fe640 1 -- 172.21.15.81:0/224067785 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f56840539a0 con 0x7f56900c8100 2023-11-08T16:30:05.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.575+0000 7f5695464640 1 -- 172.21.15.81:0/224067785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5690157060 con 0x7f56900c8100 2023-11-08T16:30:05.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.576+0000 7f56777fe640 1 -- 172.21.15.81:0/224067785 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5684017020 con 0x7f56900c8100 2023-11-08T16:30:05.578 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:30:05.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 -- 172.21.15.81:0/224067785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f567c06ae70 msgr2=0x7f567c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f567c06ae70 0x7f567c06d330 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5680036000 tx=0x7f5680030000 comp rx=0 tx=0).stop 2023-11-08T16:30:05.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 -- 172.21.15.81:0/224067785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56900c8100 msgr2=0x7f56900b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56900c8100 0x7f56900b1580 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f568400c430 tx=0x7f568400c8f0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 -- 172.21.15.81:0/224067785 shutdown_connections 2023-11-08T16:30:05.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f567c06ae70 0x7f567c06d330 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56900c8580 0x7f56900b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56900c8100 0x7f56900b1580 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 --2- 172.21.15.81:0/224067785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5678003a30 0x7f5690155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 -- 172.21.15.81:0/224067785 >> 172.21.15.81:0/224067785 conn(0x7f56900cb580 msgr2=0x7f5690155b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:05.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 -- 172.21.15.81:0/224067785 shutdown_connections 2023-11-08T16:30:05.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.579+0000 7f5695464640 1 -- 172.21.15.81:0/224067785 wait complete. 2023-11-08T16:30:05.600 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:30:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.794+0000 7f034014c640 1 Processor -- start 2023-11-08T16:30:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.794+0000 7f034014c640 1 -- start start 2023-11-08T16:30:05.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.796+0000 7f034014c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f03380ccc10 0x7f03380ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.796+0000 7f034014c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03380ce960 con 0x7f033806b710 2023-11-08T16:30:05.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.796+0000 7f034014c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03380c9480 con 0x7f033815a0e0 2023-11-08T16:30:05.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.796+0000 7f034014c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0338071980 con 0x7f03380ccc10 2023-11-08T16:30:05.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.796+0000 7f033e6e9640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f033806b710 0x7f033806bb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60212/0 (socket says 172.21.15.81:60212) 2023-11-08T16:30:05.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.796+0000 7f033e6e9640 1 -- 172.21.15.81:0/1127542387 learned_addr learned my addr 172.21.15.81:0/1127542387 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:05.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.796+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1314666995 0 0) 0x7f0330001490 con 0x7f033806b710 2023-11-08T16:30:05.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.796+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0324003520 con 0x7f033806b710 2023-11-08T16:30:05.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033d6e7640 1 --2- 172.21.15.81:0/1127542387 >> v2:172.21.15.93:3300/0 conn(0x7f03380ccc10 0x7f03380ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3681982598 0 0) 0x7f0324003520 con 0x7f033806b710 2023-11-08T16:30:05.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 >> v2:172.21.15.93:3300/0 conn(0x7f03380ccc10 msgr2=0x7f03380ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:05.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 --2- 172.21.15.81:0/1127542387 >> v2:172.21.15.93:3300/0 conn(0x7f03380ccc10 0x7f03380ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 >> v1:172.21.15.93:6789/0 conn(0x7f033815a0e0 legacy=0x7f03380c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033815a4c0 con 0x7f033806b710 2023-11-08T16:30:05.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f0330001b80 con 0x7f033806b710 2023-11-08T16:30:05.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 >> v1:172.21.15.81:6789/0 conn(0x7f033806b710 legacy=0x7f033806bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 --2- 172.21.15.81:0/1127542387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03240039e0 0x7f0324003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 --2- 172.21.15.81:0/1127542387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03240047c0 0x7f0324004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 --2- 172.21.15.81:0/1127542387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03240051f0 0x7f032400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0338071980 con 0x7f03240051f0 2023-11-08T16:30:05.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032400c140 con 0x7f03240039e0 2023-11-08T16:30:05.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032400e510 con 0x7f03240047c0 2023-11-08T16:30:05.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.797+0000 7f033e6e9640 1 --2- 172.21.15.81:0/1127542387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03240051f0 0x7f032400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.798+0000 7f033dee8640 1 --2- 172.21.15.81:0/1127542387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03240039e0 0x7f0324003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.798+0000 7f033d6e7640 1 --2- 172.21.15.81:0/1127542387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03240047c0 0x7f0324004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.798+0000 7f033e6e9640 1 -- 172.21.15.81:0/1127542387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03240047c0 msgr2=0x7f0324004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:05.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.798+0000 7f033e6e9640 1 --2- 172.21.15.81:0/1127542387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03240047c0 0x7f0324004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.798+0000 7f033e6e9640 1 -- 172.21.15.81:0/1127542387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03240039e0 msgr2=0x7f0324003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:05.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.798+0000 7f033e6e9640 1 --2- 172.21.15.81:0/1127542387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03240039e0 0x7f0324003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.798+0000 7f033e6e9640 1 -- 172.21.15.81:0/1127542387 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0324003520 con 0x7f03240051f0 2023-11-08T16:30:05.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.799+0000 7f033e6e9640 1 -- 172.21.15.81:0/1127542387 reap_dead start 2023-11-08T16:30:05.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.799+0000 7f033e6e9640 1 --2- 172.21.15.81:0/1127542387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03240051f0 0x7f032400b980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f033000a830 tx=0x7f03240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dad14df95b6aa790 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:05.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.799+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0330034630 con 0x7f03240051f0 2023-11-08T16:30:05.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.799+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f033002f070 con 0x7f03240051f0 2023-11-08T16:30:05.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.799+0000 7f033cee6640 1 -- 172.21.15.81:0/1127542387 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03300349f0 con 0x7f03240051f0 2023-11-08T16:30:05.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.800+0000 7f034014c640 1 -- 172.21.15.81:0/1127542387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03240051f0 msgr2=0x7f032400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.800+0000 7f034014c640 1 --2- 172.21.15.81:0/1127542387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03240051f0 0x7f032400b980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f033000a830 tx=0x7f03240057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.800+0000 7f034014c640 1 -- 172.21.15.81:0/1127542387 shutdown_connections 2023-11-08T16:30:05.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.800+0000 7f034014c640 1 --2- 172.21.15.81:0/1127542387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03240051f0 0x7f032400b980 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.800+0000 7f034014c640 1 -- 172.21.15.81:0/1127542387 >> 172.21.15.81:0/1127542387 conn(0x7f03380cb580 msgr2=0x7f03380cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:05.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.800+0000 7f034014c640 1 -- 172.21.15.81:0/1127542387 shutdown_connections 2023-11-08T16:30:05.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.800+0000 7f034014c640 1 -- 172.21.15.81:0/1127542387 wait complete. 2023-11-08T16:30:05.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.802+0000 7f034014c640 1 Processor -- start 2023-11-08T16:30:05.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.802+0000 7f034014c640 1 -- start start 2023-11-08T16:30:05.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.803+0000 7f034014c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03240051f0 0x7f033814e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.803+0000 7f034014c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03380b0240 0x7f03380c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.803+0000 7f034014c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f033815bbf0 0x7f033815c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.803+0000 7f034014c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033815d360 con 0x7f033815bbf0 2023-11-08T16:30:05.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.803+0000 7f034014c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03380c55f0 con 0x7f03240051f0 2023-11-08T16:30:05.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.803+0000 7f034014c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033814e6f0 con 0x7f03380b0240 2023-11-08T16:30:05.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.803+0000 7f033e6e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f033815bbf0 0x7f033815c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.803+0000 7f033e6e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f033815bbf0 0x7f033815c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59092/0 (socket says 172.21.15.81:59092) 2023-11-08T16:30:05.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.803+0000 7f033e6e9640 1 -- 172.21.15.81:0/84218584 learned_addr learned my addr 172.21.15.81:0/84218584 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:05.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.803+0000 7f033d6e7640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03380b0240 0x7f03380c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f033e6e9640 1 -- 172.21.15.81:0/84218584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03380b0240 msgr2=0x7f03380c4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f033e6e9640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03380b0240 0x7f03380c4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f033e6e9640 1 -- 172.21.15.81:0/84218584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03240051f0 msgr2=0x7f033814e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:05.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f033dee8640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03240051f0 0x7f033814e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f033e6e9640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03240051f0 0x7f033814e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:05.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f033e6e9640 1 -- 172.21.15.81:0/84218584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033815c9c0 con 0x7f033815bbf0 2023-11-08T16:30:05.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f033d6e7640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03380b0240 0x7f03380c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:05.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f033e6e9640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f033815bbf0 0x7f033815c080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f033002cc40 tx=0x7f0330001f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:05.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f032e7fc640 1 -- 172.21.15.81:0/84218584 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0330004d60 con 0x7f033815bbf0 2023-11-08T16:30:05.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f032e7fc640 1 -- 172.21.15.81:0/84218584 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0330001250 con 0x7f033815bbf0 2023-11-08T16:30:05.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f034014c640 1 -- 172.21.15.81:0/84218584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033815cca0 con 0x7f033815bbf0 2023-11-08T16:30:05.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f032e7fc640 1 -- 172.21.15.81:0/84218584 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f033003f690 con 0x7f033815bbf0 2023-11-08T16:30:05.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.804+0000 7f034014c640 1 -- 172.21.15.81:0/84218584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033815c5c0 con 0x7f033815bbf0 2023-11-08T16:30:05.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.805+0000 7f034014c640 1 -- 172.21.15.81:0/84218584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f033815d4a0 con 0x7f033815bbf0 2023-11-08T16:30:05.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.806+0000 7f032e7fc640 1 -- 172.21.15.81:0/84218584 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0330031030 con 0x7f033815bbf0 2023-11-08T16:30:05.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.807+0000 7f032e7fc640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031c06aea0 0x7f031c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:05.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.807+0000 7f032e7fc640 1 -- 172.21.15.81:0/84218584 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f03300aade0 con 0x7f033815bbf0 2023-11-08T16:30:05.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.807+0000 7f033dee8640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031c06aea0 0x7f031c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:05.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.808+0000 7f033dee8640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031c06aea0 0x7f031c06d360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0334007fd0 tx=0x7f0334009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:05.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:05.812+0000 7f032e7fc640 1 -- 172.21.15.81:0/84218584 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f033815d4a0 con 0x7f033815bbf0 2023-11-08T16:30:06.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.000+0000 7f034014c640 1 -- 172.21.15.81:0/84218584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03380ce960 con 0x7f033815bbf0 2023-11-08T16:30:06.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.000+0000 7f032e7fc640 1 -- 172.21.15.81:0/84218584 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03380ce960 con 0x7f033815bbf0 2023-11-08T16:30:06.002 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:30:06.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.003+0000 7f034014c640 1 -- 172.21.15.81:0/84218584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031c06aea0 msgr2=0x7f031c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.003+0000 7f034014c640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031c06aea0 0x7f031c06d360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0334007fd0 tx=0x7f0334009040 comp rx=0 tx=0).stop 2023-11-08T16:30:06.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.003+0000 7f034014c640 1 -- 172.21.15.81:0/84218584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f033815bbf0 msgr2=0x7f033815c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.003+0000 7f034014c640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f033815bbf0 0x7f033815c080 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f033002cc40 tx=0x7f0330001f10 comp rx=0 tx=0).stop 2023-11-08T16:30:06.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.004+0000 7f034014c640 1 -- 172.21.15.81:0/84218584 shutdown_connections 2023-11-08T16:30:06.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.004+0000 7f034014c640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031c06aea0 0x7f031c06d360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.004+0000 7f034014c640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f033815bbf0 0x7f033815c080 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.004+0000 7f034014c640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03380b0240 0x7f03380c4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.004+0000 7f034014c640 1 --2- 172.21.15.81:0/84218584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03240051f0 0x7f033814e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.004+0000 7f034014c640 1 -- 172.21.15.81:0/84218584 >> 172.21.15.81:0/84218584 conn(0x7f03380cb580 msgr2=0x7f03380c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:06.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.004+0000 7f034014c640 1 -- 172.21.15.81:0/84218584 shutdown_connections 2023-11-08T16:30:06.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.004+0000 7f034014c640 1 -- 172.21.15.81:0/84218584 wait complete. 2023-11-08T16:30:06.024 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:30:06.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.232+0000 7f5cb33e9640 1 Processor -- start 2023-11-08T16:30:06.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.232+0000 7f5cb33e9640 1 -- start start 2023-11-08T16:30:06.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.234+0000 7f5cb33e9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5cac15cfe0 0x7f5cac0b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.234+0000 7f5cb33e9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 0x7f5cac0b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.234+0000 7f5cb33e9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac0b5210 con 0x7f5cac160d40 2023-11-08T16:30:06.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.234+0000 7f5cb33e9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac1618c0 con 0x7f5cac15cfe0 2023-11-08T16:30:06.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.234+0000 7f5cb33e9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0b6540 con 0x7f5cac0b4870 2023-11-08T16:30:06.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.235+0000 7f5cb1be6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 0x7f5cac0b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.235+0000 7f5cb1be6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 0x7f5cac0b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54962/0 (socket says 172.21.15.81:54962) 2023-11-08T16:30:06.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.235+0000 7f5cb1be6640 1 -- 172.21.15.81:0/2771977194 learned_addr learned my addr 172.21.15.81:0/2771977194 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:06.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.235+0000 7f5cb23e7640 1 --2- 172.21.15.81:0/2771977194 >> v2:172.21.15.93:3300/0 conn(0x7f5cac15cfe0 0x7f5cac0b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.235+0000 7f5cb1be6640 1 -- 172.21.15.81:0/2771977194 >> v2:172.21.15.93:3300/0 conn(0x7f5cac15cfe0 msgr2=0x7f5cac0b6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:06.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.235+0000 7f5cb1be6640 1 --2- 172.21.15.81:0/2771977194 >> v2:172.21.15.93:3300/0 conn(0x7f5cac15cfe0 0x7f5cac0b6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.235+0000 7f5cb1be6640 1 -- 172.21.15.81:0/2771977194 >> v1:172.21.15.93:6789/0 conn(0x7f5cac160d40 legacy=0x7f5cac1611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.236+0000 7f5cb1be6640 1 -- 172.21.15.81:0/2771977194 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac161ab0 con 0x7f5cac0b4870 2023-11-08T16:30:06.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.237+0000 7f5cb1be6640 1 --2- 172.21.15.81:0/2771977194 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 0x7f5cac0b4cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009870 tx=0x7f5c9c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f271fd0e69755c84 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:06.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.237+0000 7f5cb13e5640 1 -- 172.21.15.81:0/2771977194 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c030840 con 0x7f5cac0b4870 2023-11-08T16:30:06.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.238+0000 7f5cb13e5640 1 -- 172.21.15.81:0/2771977194 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c9c0309a0 con 0x7f5cac0b4870 2023-11-08T16:30:06.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.238+0000 7f5cb13e5640 1 -- 172.21.15.81:0/2771977194 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c030d30 con 0x7f5cac0b4870 2023-11-08T16:30:06.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.238+0000 7f5cb33e9640 1 -- 172.21.15.81:0/2771977194 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 msgr2=0x7f5cac0b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.238+0000 7f5cb33e9640 1 --2- 172.21.15.81:0/2771977194 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 0x7f5cac0b4cd0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009870 tx=0x7f5c9c02b980 comp rx=0 tx=0).stop 2023-11-08T16:30:06.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.238+0000 7f5cb33e9640 1 -- 172.21.15.81:0/2771977194 shutdown_connections 2023-11-08T16:30:06.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.238+0000 7f5cb33e9640 1 --2- 172.21.15.81:0/2771977194 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 0x7f5cac0b4cd0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.238+0000 7f5cb33e9640 1 --2- 172.21.15.81:0/2771977194 >> v2:172.21.15.93:3300/0 conn(0x7f5cac15cfe0 0x7f5cac0b6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.238+0000 7f5cb33e9640 1 -- 172.21.15.81:0/2771977194 >> 172.21.15.81:0/2771977194 conn(0x7f5cac0cf870 msgr2=0x7f5cac0cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:06.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.238+0000 7f5cb33e9640 1 -- 172.21.15.81:0/2771977194 shutdown_connections 2023-11-08T16:30:06.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.238+0000 7f5cb33e9640 1 -- 172.21.15.81:0/2771977194 wait complete. 2023-11-08T16:30:06.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.240+0000 7f5cb33e9640 1 Processor -- start 2023-11-08T16:30:06.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.240+0000 7f5cb33e9640 1 -- start start 2023-11-08T16:30:06.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.241+0000 7f5cb33e9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 0x7f5cac156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.241+0000 7f5cb33e9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cac15cfe0 0x7f5cac0ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.241+0000 7f5cb33e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cac160d40 0x7f5cac0c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.241+0000 7f5cb33e9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac155790 con 0x7f5cac160d40 2023-11-08T16:30:06.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.241+0000 7f5cb33e9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac154b80 con 0x7f5cac15cfe0 2023-11-08T16:30:06.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.241+0000 7f5cb33e9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac154e60 con 0x7f5cac0b4870 2023-11-08T16:30:06.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.241+0000 7f5cb2be8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cac160d40 0x7f5cac0c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.242+0000 7f5cb2be8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cac160d40 0x7f5cac0c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59098/0 (socket says 172.21.15.81:59098) 2023-11-08T16:30:06.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.242+0000 7f5cb2be8640 1 -- 172.21.15.81:0/1122786410 learned_addr learned my addr 172.21.15.81:0/1122786410 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:06.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.242+0000 7f5cb23e7640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 0x7f5cac156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.242+0000 7f5cb1be6640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cac15cfe0 0x7f5cac0ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.242+0000 7f5cb2be8640 1 -- 172.21.15.81:0/1122786410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 msgr2=0x7f5cac156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.242+0000 7f5cb2be8640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 0x7f5cac156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.242+0000 7f5cb2be8640 1 -- 172.21.15.81:0/1122786410 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cac15cfe0 msgr2=0x7f5cac0ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.242+0000 7f5cb2be8640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cac15cfe0 0x7f5cac0ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.242+0000 7f5cb2be8640 1 -- 172.21.15.81:0/1122786410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c0094d0 con 0x7f5cac160d40 2023-11-08T16:30:06.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.242+0000 7f5cb23e7640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 0x7f5cac156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:30:06.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.242+0000 7f5cb2be8640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cac160d40 0x7f5cac0c8760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c7a0 tx=0x7f5ca400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:06.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.243+0000 7f5ca2ffd640 1 -- 172.21.15.81:0/1122786410 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4003b70 con 0x7f5cac160d40 2023-11-08T16:30:06.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.243+0000 7f5ca2ffd640 1 -- 172.21.15.81:0/1122786410 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca4003cf0 con 0x7f5cac160d40 2023-11-08T16:30:06.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.243+0000 7f5ca2ffd640 1 -- 172.21.15.81:0/1122786410 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4003ea0 con 0x7f5cac160d40 2023-11-08T16:30:06.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.243+0000 7f5cb33e9640 1 -- 172.21.15.81:0/1122786410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac155140 con 0x7f5cac160d40 2023-11-08T16:30:06.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.243+0000 7f5cb33e9640 1 -- 172.21.15.81:0/1122786410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac15c6f0 con 0x7f5cac160d40 2023-11-08T16:30:06.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.244+0000 7f5cb33e9640 1 -- 172.21.15.81:0/1122786410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac155900 con 0x7f5cac160d40 2023-11-08T16:30:06.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.245+0000 7f5ca2ffd640 1 -- 172.21.15.81:0/1122786410 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ca400aa90 con 0x7f5cac160d40 2023-11-08T16:30:06.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.246+0000 7f5ca2ffd640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c9006ae70 0x7f5c9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.246+0000 7f5ca2ffd640 1 -- 172.21.15.81:0/1122786410 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5ca4089500 con 0x7f5cac160d40 2023-11-08T16:30:06.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.246+0000 7f5cb23e7640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c9006ae70 0x7f5c9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.247+0000 7f5cb23e7640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c9006ae70 0x7f5c9006d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009b90 tx=0x7f5ca8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:06.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.250+0000 7f5ca2ffd640 1 -- 172.21.15.81:0/1122786410 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ca4054520 con 0x7f5cac160d40 2023-11-08T16:30:06.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.450+0000 7f5cb33e9640 1 -- 172.21.15.81:0/1122786410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5cac04ece0 con 0x7f5cac160d40 2023-11-08T16:30:06.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.450+0000 7f5ca2ffd640 1 -- 172.21.15.81:0/1122786410 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5ca4057b40 con 0x7f5cac160d40 2023-11-08T16:30:06.452 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:30:06.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 -- 172.21.15.81:0/1122786410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c9006ae70 msgr2=0x7f5c9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c9006ae70 0x7f5c9006d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5ca8009b90 tx=0x7f5ca8008040 comp rx=0 tx=0).stop 2023-11-08T16:30:06.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 -- 172.21.15.81:0/1122786410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cac160d40 msgr2=0x7f5cac0c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cac160d40 0x7f5cac0c8760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c7a0 tx=0x7f5ca400cc60 comp rx=0 tx=0).stop 2023-11-08T16:30:06.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 -- 172.21.15.81:0/1122786410 shutdown_connections 2023-11-08T16:30:06.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c9006ae70 0x7f5c9006d330 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cac160d40 0x7f5cac0c8760 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cac15cfe0 0x7f5cac0ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 --2- 172.21.15.81:0/1122786410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cac0b4870 0x7f5cac156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 -- 172.21.15.81:0/1122786410 >> 172.21.15.81:0/1122786410 conn(0x7f5cac0cf870 msgr2=0x7f5cac0cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:06.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 -- 172.21.15.81:0/1122786410 shutdown_connections 2023-11-08T16:30:06.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.453+0000 7f5cb33e9640 1 -- 172.21.15.81:0/1122786410 wait complete. 2023-11-08T16:30:06.474 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi093.front.sepia.ceph.com... 2023-11-08T16:30:06.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:06.475 DEBUG:teuthology.orchestra.run.smithi093:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-08T16:30:06.580 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:30:06.580 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:30:06.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:06.581 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-08T16:30:06.630 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T16:30:06.630 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:30:06.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.853+0000 7f801c8b4640 1 Processor -- start 2023-11-08T16:30:06.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.854+0000 7f801c8b4640 1 -- start start 2023-11-08T16:30:06.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.855+0000 7f801c8b4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80180d1700 0x7f80180cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.855+0000 7f801c8b4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80180ccc70 con 0x7f80180cda40 2023-11-08T16:30:06.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.855+0000 7f801c8b4640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f801806de20 con 0x7f801815aff0 2023-11-08T16:30:06.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.855+0000 7f801c8b4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80180cc990 con 0x7f80180d1700 2023-11-08T16:30:06.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f8015d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f80180cda40 0x7f801806e2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60222/0 (socket says 172.21.15.81:60222) 2023-11-08T16:30:06.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f8015d9b640 1 -- 172.21.15.81:0/1481506096 learned_addr learned my addr 172.21.15.81:0/1481506096 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:06.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f8016d9d640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80180d1700 0x7f80180cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4123500820 0 0) 0x7f80180ccc70 con 0x7f80180cda40 2023-11-08T16:30:06.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8004003520 con 0x7f80180cda40 2023-11-08T16:30:06.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1092865335 0 0) 0x7f8004003520 con 0x7f80180cda40 2023-11-08T16:30:06.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80180d1700 msgr2=0x7f80180cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:06.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f801559a640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80180d1700 0x7f80180cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 >> v1:172.21.15.93:6789/0 conn(0x7f801815aff0 legacy=0x7f80180cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801806d7d0 con 0x7f80180cda40 2023-11-08T16:30:06.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f8000001670 con 0x7f80180cda40 2023-11-08T16:30:06.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 >> v1:172.21.15.81:6789/0 conn(0x7f80180cda40 legacy=0x7f801806e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.856+0000 7f801559a640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8004003a80 0x7f8004003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f801559a640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80180d1700 0x7f80040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f801559a640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8004004d00 0x7f8004005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80180cc990 con 0x7f8004004d00 2023-11-08T16:30:06.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800400bb90 con 0x7f8004003a80 2023-11-08T16:30:06.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800400bd50 con 0x7f80180d1700 2023-11-08T16:30:06.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f8016d9d640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8004004d00 0x7f8004005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f801659c640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8004003a80 0x7f8004003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f8015d9b640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80180d1700 0x7f80040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f8016d9d640 1 -- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80180d1700 msgr2=0x7f80040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:06.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f8016d9d640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80180d1700 0x7f80040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f8016d9d640 1 -- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8004003a80 msgr2=0x7f8004003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:06.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f8016d9d640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8004003a80 0x7f8004003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f8016d9d640 1 -- 172.21.15.81:0/1481506096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8004003520 con 0x7f8004004d00 2023-11-08T16:30:06.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.857+0000 7f8015d9b640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80180d1700 0x7f80040047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:06.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.858+0000 7f8016d9d640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8004004d00 0x7f8004005180 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8010009a90 tx=0x7f80040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f22b5eae3208a94 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:06.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.859+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801002c680 con 0x7f8004004d00 2023-11-08T16:30:06.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.859+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f801002c7e0 con 0x7f8004004d00 2023-11-08T16:30:06.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.859+0000 7f801559a640 1 -- 172.21.15.81:0/1481506096 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801002cb70 con 0x7f8004004d00 2023-11-08T16:30:06.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.859+0000 7f801c8b4640 1 -- 172.21.15.81:0/1481506096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8004004d00 msgr2=0x7f8004005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.859+0000 7f801c8b4640 1 --2- 172.21.15.81:0/1481506096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8004004d00 0x7f8004005180 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8010009a90 tx=0x7f80040057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.859+0000 7f8016d9d640 1 -- 172.21.15.81:0/1481506096 reap_dead start 2023-11-08T16:30:06.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.859+0000 7f801c8b4640 1 -- 172.21.15.81:0/1481506096 shutdown_connections 2023-11-08T16:30:06.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.859+0000 7f801c8b4640 1 -- 172.21.15.81:0/1481506096 >> 172.21.15.81:0/1481506096 conn(0x7f80180d0be0 msgr2=0x7f80180d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:06.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.859+0000 7f801c8b4640 1 -- 172.21.15.81:0/1481506096 shutdown_connections 2023-11-08T16:30:06.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.859+0000 7f801c8b4640 1 -- 172.21.15.81:0/1481506096 wait complete. 2023-11-08T16:30:06.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.861+0000 7f801c8b4640 1 Processor -- start 2023-11-08T16:30:06.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.861+0000 7f801c8b4640 1 -- start start 2023-11-08T16:30:06.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.862+0000 7f801c8b4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8004004d00 0x7f8018152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.862+0000 7f801c8b4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f801815c9f0 0x7f801815ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.862+0000 7f801c8b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8018158af0 0x7f8018158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.862+0000 7f801c8b4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80181594b0 con 0x7f8018158af0 2023-11-08T16:30:06.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.862+0000 7f801c8b4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f801815b950 con 0x7f801815c9f0 2023-11-08T16:30:06.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.862+0000 7f801c8b4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f801815bc30 con 0x7f8004004d00 2023-11-08T16:30:06.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.862+0000 7f8016d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8018158af0 0x7f8018158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.863+0000 7f8016d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8018158af0 0x7f8018158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59126/0 (socket says 172.21.15.81:59126) 2023-11-08T16:30:06.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.863+0000 7f8016d9d640 1 -- 172.21.15.81:0/2628432411 learned_addr learned my addr 172.21.15.81:0/2628432411 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:06.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.863+0000 7f801659c640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8004004d00 0x7f8018152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.863+0000 7f8015d9b640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f801815c9f0 0x7f801815ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.863+0000 7f8016d9d640 1 -- 172.21.15.81:0/2628432411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8004004d00 msgr2=0x7f8018152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.863+0000 7f8016d9d640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8004004d00 0x7f8018152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.863+0000 7f8016d9d640 1 -- 172.21.15.81:0/2628432411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f801815c9f0 msgr2=0x7f801815ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:06.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.863+0000 7f8016d9d640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f801815c9f0 0x7f801815ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:06.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.863+0000 7f8016d9d640 1 -- 172.21.15.81:0/2628432411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801815bf10 con 0x7f8018158af0 2023-11-08T16:30:06.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.863+0000 7f8016d9d640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8018158af0 0x7f8018158f70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8010037040 tx=0x7f8010004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:06.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.864+0000 7f800effd640 1 -- 172.21.15.81:0/2628432411 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f801002cd60 con 0x7f8018158af0 2023-11-08T16:30:06.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.864+0000 7f800effd640 1 -- 172.21.15.81:0/2628432411 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8010003610 con 0x7f8018158af0 2023-11-08T16:30:06.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.864+0000 7f800effd640 1 -- 172.21.15.81:0/2628432411 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80100039c0 con 0x7f8018158af0 2023-11-08T16:30:06.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.864+0000 7f801c8b4640 1 -- 172.21.15.81:0/2628432411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f801815c1f0 con 0x7f8018158af0 2023-11-08T16:30:06.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.864+0000 7f801c8b4640 1 -- 172.21.15.81:0/2628432411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8018153220 con 0x7f8018158af0 2023-11-08T16:30:06.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.865+0000 7f801c8b4640 1 -- 172.21.15.81:0/2628432411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f801804ece0 con 0x7f8018158af0 2023-11-08T16:30:06.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.866+0000 7f800effd640 1 -- 172.21.15.81:0/2628432411 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8010003b20 con 0x7f8018158af0 2023-11-08T16:30:06.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.867+0000 7f800effd640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ff806ae70 0x7f7ff806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:06.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.867+0000 7f800effd640 1 -- 172.21.15.81:0/2628432411 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f80100aa550 con 0x7f8018158af0 2023-11-08T16:30:06.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.867+0000 7f801659c640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ff806ae70 0x7f7ff806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:06.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.868+0000 7f801659c640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ff806ae70 0x7f7ff806d330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8008005210 tx=0x7f8008009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:06.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:06.872+0000 7f800effd640 1 -- 172.21.15.81:0/2628432411 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80100754d0 con 0x7f8018158af0 2023-11-08T16:30:07.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.099+0000 7f801c8b4640 1 -- 172.21.15.81:0/2628432411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f801806cc60 con 0x7f8018158af0 2023-11-08T16:30:07.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.101+0000 7f800effd640 1 -- 172.21.15.81:0/2628432411 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f8010078af0 con 0x7f8018158af0 2023-11-08T16:30:07.103 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:07.103 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4190352668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4190352668},{"type":"v1","addr":"172.21.15.81:6835","nonce":4190352668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2636431103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2636431103},{"type":"v1","addr":"172.21.15.93:6835","nonce":2636431103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3681499690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3681499690},{"type":"v1","addr":"172.21.15.81:6837","nonce":3681499690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:29:03.181472+0000","modified":"2023-11-08T16:29:06.272041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6834/2087517381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2087517381},{"type":"v1","addr":"172.21.15.93:6834","nonce":2087517381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T16:30:07.103 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 8 2023-11-08T16:30:07.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.103+0000 7f801c8b4640 1 -- 172.21.15.81:0/2628432411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ff806ae70 msgr2=0x7f7ff806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.103+0000 7f801c8b4640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ff806ae70 0x7f7ff806d330 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8008005210 tx=0x7f8008009040 comp rx=0 tx=0).stop 2023-11-08T16:30:07.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.103+0000 7f801c8b4640 1 -- 172.21.15.81:0/2628432411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8018158af0 msgr2=0x7f8018158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.103+0000 7f801c8b4640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8018158af0 0x7f8018158f70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8010037040 tx=0x7f8010004730 comp rx=0 tx=0).stop 2023-11-08T16:30:07.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.103+0000 7f801c8b4640 1 -- 172.21.15.81:0/2628432411 shutdown_connections 2023-11-08T16:30:07.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.103+0000 7f801c8b4640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ff806ae70 0x7f7ff806d330 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.103+0000 7f801c8b4640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8018158af0 0x7f8018158f70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.103+0000 7f801c8b4640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f801815c9f0 0x7f801815ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.103+0000 7f801c8b4640 1 --2- 172.21.15.81:0/2628432411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8004004d00 0x7f8018152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.103+0000 7f801c8b4640 1 -- 172.21.15.81:0/2628432411 >> 172.21.15.81:0/2628432411 conn(0x7f80180d0be0 msgr2=0x7f80180cc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:07.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.104+0000 7f801c8b4640 1 -- 172.21.15.81:0/2628432411 shutdown_connections 2023-11-08T16:30:07.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.104+0000 7f801c8b4640 1 -- 172.21.15.81:0/2628432411 wait complete. 2023-11-08T16:30:07.123 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:30:07.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.329+0000 7f70160fc640 1 Processor -- start 2023-11-08T16:30:07.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.330+0000 7f70160fc640 1 -- start start 2023-11-08T16:30:07.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.331+0000 7f70160fc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f70100b0070 0x7f701006d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.331+0000 7f70160fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f70100cd5c0 0x7f70100cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.331+0000 7f70160fc640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70100cdf60 con 0x7f70100cffd0 2023-11-08T16:30:07.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.331+0000 7f70160fc640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70100cbee0 con 0x7f70100cd5c0 2023-11-08T16:30:07.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.331+0000 7f70160fc640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70100b0450 con 0x7f70100b0070 2023-11-08T16:30:07.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.331+0000 7f7007fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f70100cd5c0 0x7f70100cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:07.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.332+0000 7f7007fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f70100cd5c0 0x7f70100cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59142/0 (socket says 172.21.15.81:59142) 2023-11-08T16:30:07.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.332+0000 7f7007fff640 1 -- 172.21.15.81:0/963015147 learned_addr learned my addr 172.21.15.81:0/963015147 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:07.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.332+0000 7f7007fff640 1 -- 172.21.15.81:0/963015147 >> v2:172.21.15.93:3300/0 conn(0x7f70100b0070 msgr2=0x7f701006d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:07.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.332+0000 7f700f7fe640 1 --2- 172.21.15.81:0/963015147 >> v2:172.21.15.93:3300/0 conn(0x7f70100b0070 0x7f701006d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:07.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.332+0000 7f7007fff640 1 --2- 172.21.15.81:0/963015147 >> v2:172.21.15.93:3300/0 conn(0x7f70100b0070 0x7f701006d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.332+0000 7f7007fff640 1 -- 172.21.15.81:0/963015147 >> v1:172.21.15.93:6789/0 conn(0x7f70100cffd0 legacy=0x7f70100d0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:07.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.332+0000 7f7007fff640 1 -- 172.21.15.81:0/963015147 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70100cb2d0 con 0x7f70100cd5c0 2023-11-08T16:30:07.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.333+0000 7f7007fff640 1 --2- 172.21.15.81:0/963015147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70100cd5c0 0x7f70100cda20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7010071020 tx=0x7f6ff802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f520b65099e911 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:07.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.333+0000 7f700effd640 1 -- 172.21.15.81:0/963015147 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8031870 con 0x7f70100cd5c0 2023-11-08T16:30:07.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.333+0000 7f700effd640 1 -- 172.21.15.81:0/963015147 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ff80319d0 con 0x7f70100cd5c0 2023-11-08T16:30:07.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.333+0000 7f700effd640 1 -- 172.21.15.81:0/963015147 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8031d60 con 0x7f70100cd5c0 2023-11-08T16:30:07.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.334+0000 7f70160fc640 1 -- 172.21.15.81:0/963015147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70100cd5c0 msgr2=0x7f70100cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.334+0000 7f70160fc640 1 --2- 172.21.15.81:0/963015147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70100cd5c0 0x7f70100cda20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f7010071020 tx=0x7f6ff802b910 comp rx=0 tx=0).stop 2023-11-08T16:30:07.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.334+0000 7f70160fc640 1 -- 172.21.15.81:0/963015147 shutdown_connections 2023-11-08T16:30:07.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.334+0000 7f70160fc640 1 --2- 172.21.15.81:0/963015147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70100cd5c0 0x7f70100cda20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.334+0000 7f70160fc640 1 --2- 172.21.15.81:0/963015147 >> v2:172.21.15.93:3300/0 conn(0x7f70100b0070 0x7f701006d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.334+0000 7f70160fc640 1 -- 172.21.15.81:0/963015147 >> 172.21.15.81:0/963015147 conn(0x7f70100d0be0 msgr2=0x7f70100d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:07.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.334+0000 7f70160fc640 1 -- 172.21.15.81:0/963015147 shutdown_connections 2023-11-08T16:30:07.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.334+0000 7f70160fc640 1 -- 172.21.15.81:0/963015147 wait complete. 2023-11-08T16:30:07.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.336+0000 7f70160fc640 1 Processor -- start 2023-11-08T16:30:07.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.336+0000 7f70160fc640 1 -- start start 2023-11-08T16:30:07.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.337+0000 7f70160fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70100b0070 0x7f7010160790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.337+0000 7f70160fc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f70100cd5c0 0x7f701015c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.337+0000 7f70160fc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70100cffd0 0x7f70101f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.337+0000 7f70160fc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f701015c750 con 0x7f70100b0070 2023-11-08T16:30:07.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.337+0000 7f70160fc640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70100b4e10 con 0x7f70100cffd0 2023-11-08T16:30:07.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.337+0000 7f70160fc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70101f3380 con 0x7f70100cd5c0 2023-11-08T16:30:07.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.337+0000 7f700f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70100b0070 0x7f7010160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:07.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.337+0000 7f700f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70100b0070 0x7f7010160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59150/0 (socket says 172.21.15.81:59150) 2023-11-08T16:30:07.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.337+0000 7f700f7fe640 1 -- 172.21.15.81:0/3590650577 learned_addr learned my addr 172.21.15.81:0/3590650577 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:07.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.338+0000 7f700f7fe640 1 -- 172.21.15.81:0/3590650577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f70100cd5c0 msgr2=0x7f701015c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.338+0000 7f700f7fe640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f70100cd5c0 0x7f701015c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.338+0000 7f700f7fe640 1 -- 172.21.15.81:0/3590650577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70100cffd0 msgr2=0x7f70101f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.338+0000 7f700f7fe640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70100cffd0 0x7f70101f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.338+0000 7f700f7fe640 1 -- 172.21.15.81:0/3590650577 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff80094d0 con 0x7f70100b0070 2023-11-08T16:30:07.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.338+0000 7f700f7fe640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70100b0070 0x7f7010160790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f700000c7f0 tx=0x7f700000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:07.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.338+0000 7f700cff9640 1 -- 172.21.15.81:0/3590650577 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000007820 con 0x7f70100b0070 2023-11-08T16:30:07.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.338+0000 7f700cff9640 1 -- 172.21.15.81:0/3590650577 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7000007980 con 0x7f70100b0070 2023-11-08T16:30:07.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.338+0000 7f70160fc640 1 -- 172.21.15.81:0/3590650577 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70101f3620 con 0x7f70100b0070 2023-11-08T16:30:07.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.338+0000 7f700cff9640 1 -- 172.21.15.81:0/3590650577 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000007d30 con 0x7f70100b0070 2023-11-08T16:30:07.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.338+0000 7f70160fc640 1 -- 172.21.15.81:0/3590650577 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70101f3be0 con 0x7f70100b0070 2023-11-08T16:30:07.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.339+0000 7f70160fc640 1 -- 172.21.15.81:0/3590650577 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70100b4c00 con 0x7f70100b0070 2023-11-08T16:30:07.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.340+0000 7f700cff9640 1 -- 172.21.15.81:0/3590650577 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7000004020 con 0x7f70100b0070 2023-11-08T16:30:07.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.341+0000 7f700cff9640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ff006ae70 0x7f6ff006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.341+0000 7f700cff9640 1 -- 172.21.15.81:0/3590650577 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7000089aa0 con 0x7f70100b0070 2023-11-08T16:30:07.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.341+0000 7f7007fff640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ff006ae70 0x7f6ff006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:07.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.342+0000 7f7007fff640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ff006ae70 0x7f6ff006d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8036000 tx=0x7f6ff8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:07.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.345+0000 7f700cff9640 1 -- 172.21.15.81:0/3590650577 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7000054a20 con 0x7f70100b0070 2023-11-08T16:30:07.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.535+0000 7f70160fc640 1 -- 172.21.15.81:0/3590650577 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70101f3dd0 con 0x7f70100b0070 2023-11-08T16:30:07.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.538+0000 7f700cff9640 1 -- 172.21.15.81:0/3590650577 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f700001f020 con 0x7f70100b0070 2023-11-08T16:30:07.541 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:07.541 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:30:07.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.541+0000 7f70160fc640 1 -- 172.21.15.81:0/3590650577 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ff006ae70 msgr2=0x7f6ff006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.541+0000 7f70160fc640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ff006ae70 0x7f6ff006d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6ff8036000 tx=0x7f6ff8030000 comp rx=0 tx=0).stop 2023-11-08T16:30:07.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.541+0000 7f70160fc640 1 -- 172.21.15.81:0/3590650577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70100b0070 msgr2=0x7f7010160790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.541+0000 7f70160fc640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70100b0070 0x7f7010160790 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f700000c7f0 tx=0x7f700000ccb0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.541+0000 7f70160fc640 1 -- 172.21.15.81:0/3590650577 shutdown_connections 2023-11-08T16:30:07.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.541+0000 7f70160fc640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ff006ae70 0x7f6ff006d330 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.541+0000 7f70160fc640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70100cffd0 0x7f70101f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.541+0000 7f70160fc640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f70100cd5c0 0x7f701015c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.541+0000 7f70160fc640 1 --2- 172.21.15.81:0/3590650577 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70100b0070 0x7f7010160790 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.541+0000 7f70160fc640 1 -- 172.21.15.81:0/3590650577 >> 172.21.15.81:0/3590650577 conn(0x7f70100d0be0 msgr2=0x7f70100cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:07.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.541+0000 7f70160fc640 1 -- 172.21.15.81:0/3590650577 shutdown_connections 2023-11-08T16:30:07.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.542+0000 7f70160fc640 1 -- 172.21.15.81:0/3590650577 wait complete. 2023-11-08T16:30:07.562 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:29:00.903464+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:29:02.199149+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:30:07.563 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:30:07.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.770+0000 7fcb5ffff640 1 Processor -- start 2023-11-08T16:30:07.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.771+0000 7fcb5ffff640 1 -- start start 2023-11-08T16:30:07.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.772+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb600c97b0 0x7fcb6006fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.772+0000 7fcb5ffff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcb60070360 0x7fcb6006bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.772+0000 7fcb5ffff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6015e580 con 0x7fcb6015e1a0 2023-11-08T16:30:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.772+0000 7fcb5ffff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb600ca830 con 0x7fcb60070360 2023-11-08T16:30:07.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.772+0000 7fcb5ffff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600707d0 con 0x7fcb600c97b0 2023-11-08T16:30:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.773+0000 7fcb567fc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb600c97b0 0x7fcb6006fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:07.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.773+0000 7fcb5f7fe640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcb60070360 0x7fcb6006bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:07.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.773+0000 7fcb567fc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb600c97b0 0x7fcb6006fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:55008/0 (socket says 172.21.15.81:55008) 2023-11-08T16:30:07.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.773+0000 7fcb567fc640 1 -- 172.21.15.81:0/3026751668 learned_addr learned my addr 172.21.15.81:0/3026751668 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:07.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.774+0000 7fcb567fc640 1 -- 172.21.15.81:0/3026751668 >> v2:172.21.15.93:3300/0 conn(0x7fcb60070360 msgr2=0x7fcb6006bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:07.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.774+0000 7fcb567fc640 1 --2- 172.21.15.81:0/3026751668 >> v2:172.21.15.93:3300/0 conn(0x7fcb60070360 0x7fcb6006bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.774+0000 7fcb567fc640 1 -- 172.21.15.81:0/3026751668 >> v1:172.21.15.93:6789/0 conn(0x7fcb6015e1a0 legacy=0x7fcb600caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:07.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.774+0000 7fcb567fc640 1 -- 172.21.15.81:0/3026751668 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb600c9cb0 con 0x7fcb600c97b0 2023-11-08T16:30:07.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.775+0000 7fcb567fc640 1 --2- 172.21.15.81:0/3026751668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb600c97b0 0x7fcb6006fe20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009850 tx=0x7fcb4802b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ea4a82cdda86790 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:07.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.776+0000 7fcb5e7fc640 1 -- 172.21.15.81:0/3026751668 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4802e070 con 0x7fcb600c97b0 2023-11-08T16:30:07.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.776+0000 7fcb5e7fc640 1 -- 172.21.15.81:0/3026751668 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb48004030 con 0x7fcb600c97b0 2023-11-08T16:30:07.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.776+0000 7fcb5e7fc640 1 -- 172.21.15.81:0/3026751668 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb480043e0 con 0x7fcb600c97b0 2023-11-08T16:30:07.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.777+0000 7fcb5ffff640 1 -- 172.21.15.81:0/3026751668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb600c97b0 msgr2=0x7fcb6006fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.777+0000 7fcb5ffff640 1 --2- 172.21.15.81:0/3026751668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb600c97b0 0x7fcb6006fe20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcb48009850 tx=0x7fcb4802b840 comp rx=0 tx=0).stop 2023-11-08T16:30:07.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.777+0000 7fcb5ffff640 1 -- 172.21.15.81:0/3026751668 shutdown_connections 2023-11-08T16:30:07.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.777+0000 7fcb5ffff640 1 --2- 172.21.15.81:0/3026751668 >> v2:172.21.15.93:3300/0 conn(0x7fcb60070360 0x7fcb6006bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.777+0000 7fcb5ffff640 1 --2- 172.21.15.81:0/3026751668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb600c97b0 0x7fcb6006fe20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.777+0000 7fcb5ffff640 1 -- 172.21.15.81:0/3026751668 >> 172.21.15.81:0/3026751668 conn(0x7fcb600ce8d0 msgr2=0x7fcb600cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:07.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.777+0000 7fcb5ffff640 1 -- 172.21.15.81:0/3026751668 shutdown_connections 2023-11-08T16:30:07.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.777+0000 7fcb5ffff640 1 -- 172.21.15.81:0/3026751668 wait complete. 2023-11-08T16:30:07.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.779+0000 7fcb5ffff640 1 Processor -- start 2023-11-08T16:30:07.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.779+0000 7fcb5ffff640 1 -- start start 2023-11-08T16:30:07.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb60070360 0x7fcb6014cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb600c97b0 0x7fcb600c6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb6015e1a0 0x7fcb600c3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5ffff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb601562e0 con 0x7fcb6015e1a0 2023-11-08T16:30:07.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5ffff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60156560 con 0x7fcb600c97b0 2023-11-08T16:30:07.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5ffff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600c8100 con 0x7fcb60070360 2023-11-08T16:30:07.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb6015e1a0 0x7fcb600c3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:07.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb6015e1a0 0x7fcb600c3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59154/0 (socket says 172.21.15.81:59154) 2023-11-08T16:30:07.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5f7fe640 1 -- 172.21.15.81:0/603599584 learned_addr learned my addr 172.21.15.81:0/603599584 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:07.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5effd640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb60070360 0x7fcb6014cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:07.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb567fc640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb600c97b0 0x7fcb600c6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:07.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5f7fe640 1 -- 172.21.15.81:0/603599584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb60070360 msgr2=0x7fcb6014cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5f7fe640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb60070360 0x7fcb6014cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5f7fe640 1 -- 172.21.15.81:0/603599584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb600c97b0 msgr2=0x7fcb600c6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5f7fe640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb600c97b0 0x7fcb600c6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.780+0000 7fcb5f7fe640 1 -- 172.21.15.81:0/603599584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb48009460 con 0x7fcb6015e1a0 2023-11-08T16:30:07.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.781+0000 7fcb5f7fe640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb6015e1a0 0x7fcb600c3790 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800e930 tx=0x7fcb5800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:07.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.781+0000 7fcb5cff9640 1 -- 172.21.15.81:0/603599584 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58009690 con 0x7fcb6015e1a0 2023-11-08T16:30:07.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.781+0000 7fcb5cff9640 1 -- 172.21.15.81:0/603599584 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5800c980 con 0x7fcb6015e1a0 2023-11-08T16:30:07.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.781+0000 7fcb5cff9640 1 -- 172.21.15.81:0/603599584 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5800cd30 con 0x7fcb6015e1a0 2023-11-08T16:30:07.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.781+0000 7fcb5ffff640 1 -- 172.21.15.81:0/603599584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb600c83e0 con 0x7fcb6015e1a0 2023-11-08T16:30:07.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.781+0000 7fcb5ffff640 1 -- 172.21.15.81:0/603599584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb600c6f40 con 0x7fcb6015e1a0 2023-11-08T16:30:07.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.782+0000 7fcb5ffff640 1 -- 172.21.15.81:0/603599584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb60156780 con 0x7fcb6015e1a0 2023-11-08T16:30:07.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.783+0000 7fcb5cff9640 1 -- 172.21.15.81:0/603599584 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcb58013470 con 0x7fcb6015e1a0 2023-11-08T16:30:07.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.784+0000 7fcb5cff9640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb4006ae70 0x7fcb4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:07.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.784+0000 7fcb5cff9640 1 -- 172.21.15.81:0/603599584 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fcb580898e0 con 0x7fcb6015e1a0 2023-11-08T16:30:07.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.784+0000 7fcb5effd640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb4006ae70 0x7fcb4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:07.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.785+0000 7fcb5effd640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb4006ae70 0x7fcb4006d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009f20 tx=0x7fcb500094d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:07.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.789+0000 7fcb5cff9640 1 -- 172.21.15.81:0/603599584 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcb60156780 con 0x7fcb6015e1a0 2023-11-08T16:30:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.981+0000 7fcb5ffff640 1 -- 172.21.15.81:0/603599584 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb600c7190 con 0x7fcb6015e1a0 2023-11-08T16:30:07.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.982+0000 7fcb5cff9640 1 -- 172.21.15.81:0/603599584 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb580170a0 con 0x7fcb6015e1a0 2023-11-08T16:30:07.984 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:30:07.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.984+0000 7fcb5ffff640 1 -- 172.21.15.81:0/603599584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb4006ae70 msgr2=0x7fcb4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.984+0000 7fcb5ffff640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb4006ae70 0x7fcb4006d330 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009f20 tx=0x7fcb500094d0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.985+0000 7fcb5ffff640 1 -- 172.21.15.81:0/603599584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb6015e1a0 msgr2=0x7fcb600c3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:07.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.985+0000 7fcb5ffff640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb6015e1a0 0x7fcb600c3790 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcb5800e930 tx=0x7fcb5800edf0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.985+0000 7fcb5ffff640 1 -- 172.21.15.81:0/603599584 shutdown_connections 2023-11-08T16:30:07.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.985+0000 7fcb5ffff640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcb4006ae70 0x7fcb4006d330 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.985+0000 7fcb5ffff640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcb6015e1a0 0x7fcb600c3790 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.985+0000 7fcb5ffff640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcb600c97b0 0x7fcb600c6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.985+0000 7fcb5ffff640 1 --2- 172.21.15.81:0/603599584 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcb60070360 0x7fcb6014cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:07.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.985+0000 7fcb5ffff640 1 -- 172.21.15.81:0/603599584 >> 172.21.15.81:0/603599584 conn(0x7fcb600ce8d0 msgr2=0x7fcb6006ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:07.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.985+0000 7fcb5ffff640 1 -- 172.21.15.81:0/603599584 shutdown_connections 2023-11-08T16:30:07.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:07.985+0000 7fcb5ffff640 1 -- 172.21.15.81:0/603599584 wait complete. 2023-11-08T16:30:08.006 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:30:08.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.212+0000 7f52763d7640 1 Processor -- start 2023-11-08T16:30:08.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.212+0000 7f52763d7640 1 -- start start 2023-11-08T16:30:08.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.214+0000 7f52763d7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f527015b140 0x7f52700b6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.214+0000 7f52763d7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52700cc7f0 con 0x7f527015ad60 2023-11-08T16:30:08.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.214+0000 7f52763d7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52700cca90 con 0x7f527006e240 2023-11-08T16:30:08.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.214+0000 7f52763d7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52700ccd70 con 0x7f527015b140 2023-11-08T16:30:08.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.214+0000 7f52753d5640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f527015ad60 0x7f52700b39a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60226/0 (socket says 172.21.15.81:60226) 2023-11-08T16:30:08.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.214+0000 7f52753d5640 1 -- 172.21.15.81:0/4241422092 learned_addr learned my addr 172.21.15.81:0/4241422092 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:08.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.214+0000 7f5274bd4640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f527015b140 0x7f52700b6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.214+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893796062 0 0) 0x7f52700cc7f0 con 0x7f527015ad60 2023-11-08T16:30:08.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.214+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f525c003520 con 0x7f527015ad60 2023-11-08T16:30:08.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1958066881 0 0) 0x7f525c003520 con 0x7f527015ad60 2023-11-08T16:30:08.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f527015b140 msgr2=0x7f52700b6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:08.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f527015b140 0x7f52700b6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 >> v1:172.21.15.93:6789/0 conn(0x7f527006e240 legacy=0x7f52700d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52700cd050 con 0x7f527015ad60 2023-11-08T16:30:08.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f526c001670 con 0x7f527015ad60 2023-11-08T16:30:08.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 >> v1:172.21.15.81:6789/0 conn(0x7f527015ad60 legacy=0x7f52700b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f525c0039a0 0x7f525c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f525c0047c0 0x7f525c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f525c005180 0x7f525c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52700ccd70 con 0x7f525c005180 2023-11-08T16:30:08.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525c00bf70 con 0x7f525c0047c0 2023-11-08T16:30:08.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525c00c130 con 0x7f525c0039a0 2023-11-08T16:30:08.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.215+0000 7f5275bd6640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f525c005180 0x7f525c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.216+0000 7f52753d5640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f525c0039a0 0x7f525c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.216+0000 7f5274bd4640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f525c0047c0 0x7f525c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.216+0000 7f5275bd6640 1 -- 172.21.15.81:0/4241422092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f525c0039a0 msgr2=0x7f525c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:08.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.216+0000 7f5275bd6640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f525c0039a0 0x7f525c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.216+0000 7f5275bd6640 1 -- 172.21.15.81:0/4241422092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f525c0047c0 msgr2=0x7f525c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:08.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.216+0000 7f5275bd6640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f525c0047c0 0x7f525c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.216+0000 7f5275bd6640 1 -- 172.21.15.81:0/4241422092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f525c003520 con 0x7f525c005180 2023-11-08T16:30:08.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.217+0000 7f5275bd6640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f525c005180 0x7f525c00b980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5268001e20 tx=0x7f525c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a891c8395427ca server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.217+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268003b70 con 0x7f525c005180 2023-11-08T16:30:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.217+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5268003cd0 con 0x7f525c005180 2023-11-08T16:30:08.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.217+0000 7f5267fff640 1 -- 172.21.15.81:0/4241422092 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268005310 con 0x7f525c005180 2023-11-08T16:30:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.218+0000 7f52763d7640 1 -- 172.21.15.81:0/4241422092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f525c005180 msgr2=0x7f525c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.218+0000 7f52763d7640 1 --2- 172.21.15.81:0/4241422092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f525c005180 0x7f525c00b980 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f5268001e20 tx=0x7f525c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.218+0000 7f5275bd6640 1 -- 172.21.15.81:0/4241422092 reap_dead start 2023-11-08T16:30:08.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.218+0000 7f52763d7640 1 -- 172.21.15.81:0/4241422092 shutdown_connections 2023-11-08T16:30:08.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.218+0000 7f52763d7640 1 -- 172.21.15.81:0/4241422092 >> 172.21.15.81:0/4241422092 conn(0x7f5270065df0 msgr2=0x7f5270066200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:08.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.218+0000 7f52763d7640 1 -- 172.21.15.81:0/4241422092 shutdown_connections 2023-11-08T16:30:08.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.218+0000 7f52763d7640 1 -- 172.21.15.81:0/4241422092 wait complete. 2023-11-08T16:30:08.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.220+0000 7f52763d7640 1 Processor -- start 2023-11-08T16:30:08.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.220+0000 7f52763d7640 1 -- start start 2023-11-08T16:30:08.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f52763d7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f525c005180 0x7f52700b60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f52763d7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5270151810 0x7f52701582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f52763d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52701508b0 0x7f5270150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f52763d7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52701f6790 con 0x7f52701508b0 2023-11-08T16:30:08.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f52763d7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52701f5b80 con 0x7f5270151810 2023-11-08T16:30:08.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f52763d7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52701f5e60 con 0x7f525c005180 2023-11-08T16:30:08.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f5275bd6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52701508b0 0x7f5270150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f5275bd6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52701508b0 0x7f5270150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59174/0 (socket says 172.21.15.81:59174) 2023-11-08T16:30:08.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f5275bd6640 1 -- 172.21.15.81:0/2317421169 learned_addr learned my addr 172.21.15.81:0/2317421169 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:08.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f52753d5640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f525c005180 0x7f52700b60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f5274bd4640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5270151810 0x7f52701582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f5275bd6640 1 -- 172.21.15.81:0/2317421169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f525c005180 msgr2=0x7f52700b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f5275bd6640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f525c005180 0x7f52700b60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f5275bd6640 1 -- 172.21.15.81:0/2317421169 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5270151810 msgr2=0x7f52701582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f5275bd6640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5270151810 0x7f52701582c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f5275bd6640 1 -- 172.21.15.81:0/2317421169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52701f6140 con 0x7f52701508b0 2023-11-08T16:30:08.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.221+0000 7f5274bd4640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5270151810 0x7f52701582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:08.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.222+0000 7f5275bd6640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52701508b0 0x7f5270150d40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5268001f40 tx=0x7f52680014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:08.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.222+0000 7f5265ffb640 1 -- 172.21.15.81:0/2317421169 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526803e070 con 0x7f52701508b0 2023-11-08T16:30:08.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.222+0000 7f5265ffb640 1 -- 172.21.15.81:0/2317421169 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5268003e40 con 0x7f52701508b0 2023-11-08T16:30:08.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.222+0000 7f5265ffb640 1 -- 172.21.15.81:0/2317421169 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5268006d90 con 0x7f52701508b0 2023-11-08T16:30:08.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.222+0000 7f52763d7640 1 -- 172.21.15.81:0/2317421169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52701f6420 con 0x7f52701508b0 2023-11-08T16:30:08.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.222+0000 7f52763d7640 1 -- 172.21.15.81:0/2317421169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5270158800 con 0x7f52701508b0 2023-11-08T16:30:08.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.223+0000 7f52763d7640 1 -- 172.21.15.81:0/2317421169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f527004ece0 con 0x7f52701508b0 2023-11-08T16:30:08.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.224+0000 7f5265ffb640 1 -- 172.21.15.81:0/2317421169 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5268007030 con 0x7f52701508b0 2023-11-08T16:30:08.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.225+0000 7f5265ffb640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f525406ae70 0x7f525406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.225+0000 7f5265ffb640 1 -- 172.21.15.81:0/2317421169 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f52680aa920 con 0x7f52701508b0 2023-11-08T16:30:08.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.226+0000 7f52753d5640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f525406ae70 0x7f525406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.226+0000 7f52753d5640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f525406ae70 0x7f525406d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f526c001eb0 tx=0x7f526c000f50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:08.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.230+0000 7f5265ffb640 1 -- 172.21.15.81:0/2317421169 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f52680758a0 con 0x7f52701508b0 2023-11-08T16:30:08.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.419+0000 7f52763d7640 1 -- 172.21.15.81:0/2317421169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52700b6930 con 0x7f52701508b0 2023-11-08T16:30:08.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.420+0000 7f5265ffb640 1 -- 172.21.15.81:0/2317421169 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52680480a0 con 0x7f52701508b0 2023-11-08T16:30:08.422 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:30:08.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.423+0000 7f52763d7640 1 -- 172.21.15.81:0/2317421169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f525406ae70 msgr2=0x7f525406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.423+0000 7f52763d7640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f525406ae70 0x7f525406d330 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f526c001eb0 tx=0x7f526c000f50 comp rx=0 tx=0).stop 2023-11-08T16:30:08.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.423+0000 7f52763d7640 1 -- 172.21.15.81:0/2317421169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52701508b0 msgr2=0x7f5270150d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.423+0000 7f52763d7640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52701508b0 0x7f5270150d40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5268001f40 tx=0x7f52680014a0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.423+0000 7f52763d7640 1 -- 172.21.15.81:0/2317421169 shutdown_connections 2023-11-08T16:30:08.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.423+0000 7f52763d7640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f525406ae70 0x7f525406d330 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.423+0000 7f52763d7640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f52701508b0 0x7f5270150d40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.424+0000 7f52763d7640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5270151810 0x7f52701582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.424+0000 7f52763d7640 1 --2- 172.21.15.81:0/2317421169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f525c005180 0x7f52700b60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.424+0000 7f52763d7640 1 -- 172.21.15.81:0/2317421169 >> 172.21.15.81:0/2317421169 conn(0x7f5270065df0 msgr2=0x7f52700cfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:08.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.424+0000 7f52763d7640 1 -- 172.21.15.81:0/2317421169 shutdown_connections 2023-11-08T16:30:08.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.424+0000 7f52763d7640 1 -- 172.21.15.81:0/2317421169 wait complete. 2023-11-08T16:30:08.445 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:30:08.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.658+0000 7fb9a2175640 1 Processor -- start 2023-11-08T16:30:08.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.658+0000 7fb9a2175640 1 -- start start 2023-11-08T16:30:08.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.660+0000 7fb9a2175640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb99c15b250 0x7fb99c15a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.660+0000 7fb9a2175640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb99c0b11f0 0x7fb99c157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.660+0000 7fb9a2175640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb99c157a80 con 0x7fb99c1576a0 2023-11-08T16:30:08.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.660+0000 7fb9a2175640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c15c360 con 0x7fb99c0b11f0 2023-11-08T16:30:08.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.660+0000 7fb9a2175640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c15b6e0 con 0x7fb99c15b250 2023-11-08T16:30:08.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.660+0000 7fb9a1173640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb99c1576a0 0x7fb99c070c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60234/0 (socket says 172.21.15.81:60234) 2023-11-08T16:30:08.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.660+0000 7fb9a1173640 1 -- 172.21.15.81:0/3194816772 learned_addr learned my addr 172.21.15.81:0/3194816772 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:08.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.660+0000 7fb9a0972640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb99c15b250 0x7fb99c15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.660+0000 7fb9a1974640 1 --2- 172.21.15.81:0/3194816772 >> v2:172.21.15.93:3300/0 conn(0x7fb99c0b11f0 0x7fb99c157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896658141 0 0) 0x7fb99c157a80 con 0x7fb99c1576a0 2023-11-08T16:30:08.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb988003520 con 0x7fb99c1576a0 2023-11-08T16:30:08.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3948148760 0 0) 0x7fb988003520 con 0x7fb99c1576a0 2023-11-08T16:30:08.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb99c15b250 msgr2=0x7fb99c15a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:08.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb99c15b250 0x7fb99c15a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 >> v2:172.21.15.93:3300/0 conn(0x7fb99c0b11f0 msgr2=0x7fb99c157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:08.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 --2- 172.21.15.81:0/3194816772 >> v2:172.21.15.93:3300/0 conn(0x7fb99c0b11f0 0x7fb99c157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c15b9c0 con 0x7fb99c1576a0 2023-11-08T16:30:08.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fb998001670 con 0x7fb99c1576a0 2023-11-08T16:30:08.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 >> v1:172.21.15.81:6789/0 conn(0x7fb99c1576a0 legacy=0x7fb99c070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 0x7fb988003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9880047c0 0x7fb988004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb988005180 0x7fb98800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c15c360 con 0x7fb988003970 2023-11-08T16:30:08.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c15b6e0 con 0x7fb9880047c0 2023-11-08T16:30:08.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb98800e0d0 con 0x7fb988005180 2023-11-08T16:30:08.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.661+0000 7fb9a1173640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 0x7fb988003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.662+0000 7fb9a0972640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9880047c0 0x7fb988004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.662+0000 7fb9a1173640 1 -- 172.21.15.81:0/3194816772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb988005180 msgr2=0x7fb98800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:08.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.662+0000 7fb9a1173640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb988005180 0x7fb98800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.662+0000 7fb9a1173640 1 -- 172.21.15.81:0/3194816772 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9880047c0 msgr2=0x7fb988004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:08.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.662+0000 7fb9a1173640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9880047c0 0x7fb988004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.662+0000 7fb9a1173640 1 -- 172.21.15.81:0/3194816772 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb988003520 con 0x7fb988003970 2023-11-08T16:30:08.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.662+0000 7fb9a0972640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9880047c0 0x7fb988004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:08.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.663+0000 7fb9a1173640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 0x7fb988003d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c190 tx=0x7fb9880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36995049bc67ceb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:08.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.663+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998005ad0 con 0x7fb988003970 2023-11-08T16:30:08.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.663+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb998005c30 con 0x7fb988003970 2023-11-08T16:30:08.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.663+0000 7fb993fff640 1 -- 172.21.15.81:0/3194816772 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99802c810 con 0x7fb988003970 2023-11-08T16:30:08.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.664+0000 7fb9a2175640 1 -- 172.21.15.81:0/3194816772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 msgr2=0x7fb988003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.664+0000 7fb9a2175640 1 --2- 172.21.15.81:0/3194816772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 0x7fb988003d90 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fb99800c190 tx=0x7fb9880057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.664+0000 7fb9a1974640 1 -- 172.21.15.81:0/3194816772 reap_dead start 2023-11-08T16:30:08.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.664+0000 7fb9a2175640 1 -- 172.21.15.81:0/3194816772 shutdown_connections 2023-11-08T16:30:08.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.664+0000 7fb9a2175640 1 -- 172.21.15.81:0/3194816772 >> 172.21.15.81:0/3194816772 conn(0x7fb99c0ce8d0 msgr2=0x7fb99c0cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:08.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.664+0000 7fb9a2175640 1 -- 172.21.15.81:0/3194816772 shutdown_connections 2023-11-08T16:30:08.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.664+0000 7fb9a2175640 1 -- 172.21.15.81:0/3194816772 wait complete. 2023-11-08T16:30:08.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.666+0000 7fb9a2175640 1 Processor -- start 2023-11-08T16:30:08.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.666+0000 7fb9a2175640 1 -- start start 2023-11-08T16:30:08.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a2175640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 0x7fb99c0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a2175640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb99c1f68b0 0x7fb99c1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a2175640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb99c0c98d0 0x7fb99c0c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a2175640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c153910 con 0x7fb988003970 2023-11-08T16:30:08.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a2175640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0c59b0 con 0x7fb99c1f68b0 2023-11-08T16:30:08.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a2175640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c0c5c90 con 0x7fb99c0c98d0 2023-11-08T16:30:08.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a1173640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 0x7fb99c0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a1173640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 0x7fb99c0b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59200/0 (socket says 172.21.15.81:59200) 2023-11-08T16:30:08.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a1173640 1 -- 172.21.15.81:0/106295011 learned_addr learned my addr 172.21.15.81:0/106295011 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:08.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a1974640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb99c0c98d0 0x7fb99c0c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a0972640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb99c1f68b0 0x7fb99c1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a1173640 1 -- 172.21.15.81:0/106295011 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb99c0c98d0 msgr2=0x7fb99c0c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a1173640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb99c0c98d0 0x7fb99c0c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a1173640 1 -- 172.21.15.81:0/106295011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb99c1f68b0 msgr2=0x7fb99c1533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a1173640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb99c1f68b0 0x7fb99c1533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.667+0000 7fb9a1173640 1 -- 172.21.15.81:0/106295011 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c0c5f70 con 0x7fb988003970 2023-11-08T16:30:08.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.668+0000 7fb9a1974640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb99c0c98d0 0x7fb99c0c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:08.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.668+0000 7fb9a0972640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb99c1f68b0 0x7fb99c1533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:08.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.668+0000 7fb9a1173640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 0x7fb99c0b18b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb99802c3b0 tx=0x7fb998001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:08.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.668+0000 7fb991ffb640 1 -- 172.21.15.81:0/106295011 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998039070 con 0x7fb988003970 2023-11-08T16:30:08.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.668+0000 7fb991ffb640 1 -- 172.21.15.81:0/106295011 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99802cc20 con 0x7fb988003970 2023-11-08T16:30:08.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.668+0000 7fb991ffb640 1 -- 172.21.15.81:0/106295011 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99803c040 con 0x7fb988003970 2023-11-08T16:30:08.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.668+0000 7fb9a2175640 1 -- 172.21.15.81:0/106295011 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c0c6250 con 0x7fb988003970 2023-11-08T16:30:08.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.668+0000 7fb9a2175640 1 -- 172.21.15.81:0/106295011 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c15d8a0 con 0x7fb988003970 2023-11-08T16:30:08.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.669+0000 7fb9a2175640 1 -- 172.21.15.81:0/106295011 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99c04ece0 con 0x7fb988003970 2023-11-08T16:30:08.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.670+0000 7fb991ffb640 1 -- 172.21.15.81:0/106295011 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb998032070 con 0x7fb988003970 2023-11-08T16:30:08.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.671+0000 7fb991ffb640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb98006ae70 0x7fb98006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:08.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.671+0000 7fb991ffb640 1 -- 172.21.15.81:0/106295011 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb9980aa9c0 con 0x7fb988003970 2023-11-08T16:30:08.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.671+0000 7fb9a0972640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb98006ae70 0x7fb98006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:08.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.672+0000 7fb9a0972640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb98006ae70 0x7fb98006d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb99c156d40 tx=0x7fb98c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:08.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.675+0000 7fb991ffb640 1 -- 172.21.15.81:0/106295011 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb9980759d0 con 0x7fb988003970 2023-11-08T16:30:08.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.865+0000 7fb9a2175640 1 -- 172.21.15.81:0/106295011 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb99c06ba40 con 0x7fb988003970 2023-11-08T16:30:08.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.866+0000 7fb991ffb640 1 -- 172.21.15.81:0/106295011 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb998078ff0 con 0x7fb988003970 2023-11-08T16:30:08.868 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:30:08.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 -- 172.21.15.81:0/106295011 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb98006ae70 msgr2=0x7fb98006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb98006ae70 0x7fb98006d330 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb99c156d40 tx=0x7fb98c008040 comp rx=0 tx=0).stop 2023-11-08T16:30:08.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 -- 172.21.15.81:0/106295011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 msgr2=0x7fb99c0b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:08.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 0x7fb99c0b18b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fb99802c3b0 tx=0x7fb998001930 comp rx=0 tx=0).stop 2023-11-08T16:30:08.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 -- 172.21.15.81:0/106295011 shutdown_connections 2023-11-08T16:30:08.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb98006ae70 0x7fb98006d330 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb99c0c98d0 0x7fb99c0c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb99c1f68b0 0x7fb99c1533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 --2- 172.21.15.81:0/106295011 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb988003970 0x7fb99c0b18b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:08.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 -- 172.21.15.81:0/106295011 >> 172.21.15.81:0/106295011 conn(0x7fb99c0ce8d0 msgr2=0x7fb99c15cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:08.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 -- 172.21.15.81:0/106295011 shutdown_connections 2023-11-08T16:30:08.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:08.869+0000 7fb9a2175640 1 -- 172.21.15.81:0/106295011 wait complete. 2023-11-08T16:30:08.890 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:30:09.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.103+0000 7fd934fb0640 1 Processor -- start 2023-11-08T16:30:09.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.103+0000 7fd934fb0640 1 -- start start 2023-11-08T16:30:09.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.104+0000 7fd934fb0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300adcd0 0x7fd9300ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.105+0000 7fd934fb0640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd9300cc5e0 0x7fd9300cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.105+0000 7fd934fb0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9300c9a70 con 0x7fd9300ccf80 2023-11-08T16:30:09.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.105+0000 7fd934fb0640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9300cc3e0 con 0x7fd9300cc5e0 2023-11-08T16:30:09.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.105+0000 7fd934fb0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd93006d070 con 0x7fd9300adcd0 2023-11-08T16:30:09.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.105+0000 7fd92ed9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd9300ccf80 0x7fd93015db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60240/0 (socket says 172.21.15.81:60240) 2023-11-08T16:30:09.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.105+0000 7fd92ed9d640 1 -- 172.21.15.81:0/4058652938 learned_addr learned my addr 172.21.15.81:0/4058652938 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:09.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.105+0000 7fd92e59c640 1 --2- 172.21.15.81:0/4058652938 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300adcd0 0x7fd9300ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.105+0000 7fd927fff640 1 --2- 172.21.15.81:0/4058652938 >> v2:172.21.15.93:3300/0 conn(0x7fd9300cc5e0 0x7fd9300cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.105+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887411092 0 0) 0x7fd9300c9a70 con 0x7fd9300ccf80 2023-11-08T16:30:09.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd91c003520 con 0x7fd9300ccf80 2023-11-08T16:30:09.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4029136664 0 0) 0x7fd91c003520 con 0x7fd9300ccf80 2023-11-08T16:30:09.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300adcd0 msgr2=0x7fd9300ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:09.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 --2- 172.21.15.81:0/4058652938 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300adcd0 0x7fd9300ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 >> v2:172.21.15.93:3300/0 conn(0x7fd9300cc5e0 msgr2=0x7fd9300cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:09.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 --2- 172.21.15.81:0/4058652938 >> v2:172.21.15.93:3300/0 conn(0x7fd9300cc5e0 0x7fd9300cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93006d350 con 0x7fd9300ccf80 2023-11-08T16:30:09.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fd9280019f0 con 0x7fd9300ccf80 2023-11-08T16:30:09.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 >> v1:172.21.15.81:6789/0 conn(0x7fd9300ccf80 legacy=0x7fd93015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:09.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 --2- 172.21.15.81:0/4058652938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd91c003a50 0x7fd91c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 --2- 172.21.15.81:0/4058652938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd91c0047c0 0x7fd91c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 --2- 172.21.15.81:0/4058652938 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300adcd0 0x7fd91c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300cc3e0 con 0x7fd91c003a50 2023-11-08T16:30:09.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93006d070 con 0x7fd91c0047c0 2023-11-08T16:30:09.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c00ba00 con 0x7fd9300adcd0 2023-11-08T16:30:09.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.106+0000 7fd92e59c640 1 --2- 172.21.15.81:0/4058652938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd91c003a50 0x7fd91c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.107+0000 7fd92e59c640 1 -- 172.21.15.81:0/4058652938 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300adcd0 msgr2=0x7fd91c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:09.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.107+0000 7fd92e59c640 1 --2- 172.21.15.81:0/4058652938 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300adcd0 0x7fd91c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.107+0000 7fd92e59c640 1 -- 172.21.15.81:0/4058652938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd91c0047c0 msgr2=0x7fd91c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:09.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.107+0000 7fd92e59c640 1 --2- 172.21.15.81:0/4058652938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd91c0047c0 0x7fd91c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.107+0000 7fd92e59c640 1 -- 172.21.15.81:0/4058652938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c003520 con 0x7fd91c003a50 2023-11-08T16:30:09.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.108+0000 7fd92e59c640 1 --2- 172.21.15.81:0/4058652938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd91c003a50 0x7fd91c003e70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd920009ac0 tx=0x7fd91c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f1f4aa1223dafc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:09.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.108+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92002c710 con 0x7fd91c003a50 2023-11-08T16:30:09.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.108+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd92002c870 con 0x7fd91c003a50 2023-11-08T16:30:09.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.108+0000 7fd92dd9b640 1 -- 172.21.15.81:0/4058652938 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92002cc00 con 0x7fd91c003a50 2023-11-08T16:30:09.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.109+0000 7fd934fb0640 1 -- 172.21.15.81:0/4058652938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd91c003a50 msgr2=0x7fd91c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:09.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.109+0000 7fd934fb0640 1 --2- 172.21.15.81:0/4058652938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd91c003a50 0x7fd91c003e70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd920009ac0 tx=0x7fd91c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.109+0000 7fd92ed9d640 1 -- 172.21.15.81:0/4058652938 reap_dead start 2023-11-08T16:30:09.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.109+0000 7fd934fb0640 1 -- 172.21.15.81:0/4058652938 shutdown_connections 2023-11-08T16:30:09.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.109+0000 7fd934fb0640 1 -- 172.21.15.81:0/4058652938 >> 172.21.15.81:0/4058652938 conn(0x7fd9300cb490 msgr2=0x7fd9300cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:09.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.109+0000 7fd934fb0640 1 -- 172.21.15.81:0/4058652938 shutdown_connections 2023-11-08T16:30:09.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.109+0000 7fd934fb0640 1 -- 172.21.15.81:0/4058652938 wait complete. 2023-11-08T16:30:09.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.111+0000 7fd934fb0640 1 Processor -- start 2023-11-08T16:30:09.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.111+0000 7fd934fb0640 1 -- start start 2023-11-08T16:30:09.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.112+0000 7fd934fb0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd91c003a50 0x7fd930151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.112+0000 7fd934fb0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300c5b50 0x7fd9300c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.112+0000 7fd934fb0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9300b3bb0 0x7fd93015a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.112+0000 7fd934fb0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93014d990 con 0x7fd9300b3bb0 2023-11-08T16:30:09.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.112+0000 7fd934fb0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd930152420 con 0x7fd91c003a50 2023-11-08T16:30:09.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.112+0000 7fd934fb0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300c8970 con 0x7fd9300c5b50 2023-11-08T16:30:09.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.112+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9300b3bb0 0x7fd93015a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.112+0000 7fd92ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9300b3bb0 0x7fd93015a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59220/0 (socket says 172.21.15.81:59220) 2023-11-08T16:30:09.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.112+0000 7fd92ed9d640 1 -- 172.21.15.81:0/135920039 learned_addr learned my addr 172.21.15.81:0/135920039 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:09.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.112+0000 7fd927fff640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300c5b50 0x7fd9300c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.112+0000 7fd92e59c640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd91c003a50 0x7fd930151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.113+0000 7fd92ed9d640 1 -- 172.21.15.81:0/135920039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300c5b50 msgr2=0x7fd9300c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:09.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.113+0000 7fd92ed9d640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300c5b50 0x7fd9300c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.113+0000 7fd92ed9d640 1 -- 172.21.15.81:0/135920039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd91c003a50 msgr2=0x7fd930151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:09.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.113+0000 7fd92ed9d640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd91c003a50 0x7fd930151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.113+0000 7fd92ed9d640 1 -- 172.21.15.81:0/135920039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd930151120 con 0x7fd9300b3bb0 2023-11-08T16:30:09.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.113+0000 7fd92ed9d640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9300b3bb0 0x7fd93015a5d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd930071a70 tx=0x7fd92800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:09.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.113+0000 7fd926ffd640 1 -- 172.21.15.81:0/135920039 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92800cec0 con 0x7fd9300b3bb0 2023-11-08T16:30:09.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.113+0000 7fd926ffd640 1 -- 172.21.15.81:0/135920039 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd928007110 con 0x7fd9300b3bb0 2023-11-08T16:30:09.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.113+0000 7fd926ffd640 1 -- 172.21.15.81:0/135920039 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9280074c0 con 0x7fd9300b3bb0 2023-11-08T16:30:09.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.113+0000 7fd934fb0640 1 -- 172.21.15.81:0/135920039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd930151400 con 0x7fd9300b3bb0 2023-11-08T16:30:09.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.113+0000 7fd934fb0640 1 -- 172.21.15.81:0/135920039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9301519c0 con 0x7fd9300b3bb0 2023-11-08T16:30:09.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.114+0000 7fd934fb0640 1 -- 172.21.15.81:0/135920039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd930072170 con 0x7fd9300b3bb0 2023-11-08T16:30:09.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.116+0000 7fd926ffd640 1 -- 172.21.15.81:0/135920039 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd928007620 con 0x7fd9300b3bb0 2023-11-08T16:30:09.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.116+0000 7fd926ffd640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 0x7fd91006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.117+0000 7fd926ffd640 1 -- 172.21.15.81:0/135920039 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd928088f00 con 0x7fd9300b3bb0 2023-11-08T16:30:09.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.117+0000 7fd92e59c640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 0x7fd91006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.117+0000 7fd92e59c640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 0x7fd91006d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd920009ac0 tx=0x7fd920030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:09.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.121+0000 7fd926ffd640 1 -- 172.21.15.81:0/135920039 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd9280579a0 con 0x7fd9300b3bb0 2023-11-08T16:30:09.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.345+0000 7fd934fb0640 1 -- 172.21.15.81:0/135920039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd930151bb0 con 0x7fd9300b3bb0 2023-11-08T16:30:09.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.346+0000 7fd926ffd640 1 -- 172.21.15.81:0/135920039 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7fd92801a070 con 0x7fd9300b3bb0 2023-11-08T16:30:09.348 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:09.349 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4190352668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4190352668},{"type":"v1","addr":"172.21.15.81:6835","nonce":4190352668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2636431103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2636431103},{"type":"v1","addr":"172.21.15.93:6835","nonce":2636431103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3681499690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3681499690},{"type":"v1","addr":"172.21.15.81:6837","nonce":3681499690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:29:03.181472+0000","modified":"2023-11-08T16:29:06.272041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6834/2087517381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2087517381},{"type":"v1","addr":"172.21.15.93:6834","nonce":2087517381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T16:30:09.349 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 8 2023-11-08T16:30:09.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 -- 172.21.15.81:0/135920039 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 msgr2=0x7fd91006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:09.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 0x7fd91006d330 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd920009ac0 tx=0x7fd920030000 comp rx=0 tx=0).stop 2023-11-08T16:30:09.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 -- 172.21.15.81:0/135920039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9300b3bb0 msgr2=0x7fd93015a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:09.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9300b3bb0 0x7fd93015a5d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd930071a70 tx=0x7fd92800ccd0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 -- 172.21.15.81:0/135920039 shutdown_connections 2023-11-08T16:30:09.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd91006ae70 0x7fd91006d330 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd9300b3bb0 0x7fd93015a5d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd9300c5b50 0x7fd9300c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 --2- 172.21.15.81:0/135920039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd91c003a50 0x7fd930151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 -- 172.21.15.81:0/135920039 >> 172.21.15.81:0/135920039 conn(0x7fd9300cb490 msgr2=0x7fd930070050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:09.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 -- 172.21.15.81:0/135920039 shutdown_connections 2023-11-08T16:30:09.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.349+0000 7fd934fb0640 1 -- 172.21.15.81:0/135920039 wait complete. 2023-11-08T16:30:09.368 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:30:09.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.574+0000 7ff5c2c97640 1 Processor -- start 2023-11-08T16:30:09.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.574+0000 7ff5c2c97640 1 -- start start 2023-11-08T16:30:09.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.575+0000 7ff5c2c97640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff5bc06e7c0 0x7ff5bc0afb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.575+0000 7ff5c2c97640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff5bc0ce870 0x7ff5bc06d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.575+0000 7ff5c2c97640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5bc0b2720 0x7ff5bc0b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.575+0000 7ff5c2c97640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc0b30f0 con 0x7ff5bc0ce870 2023-11-08T16:30:09.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.575+0000 7ff5c2c97640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc06de30 con 0x7ff5bc06e7c0 2023-11-08T16:30:09.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.575+0000 7ff5c2c97640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc0cecf0 con 0x7ff5bc0b2720 2023-11-08T16:30:09.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.576+0000 7ff5bbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff5bc0ce870 0x7ff5bc06d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.576+0000 7ff5bbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff5bc0ce870 0x7ff5bc06d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59236/0 (socket says 172.21.15.81:59236) 2023-11-08T16:30:09.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.576+0000 7ff5bbfff640 1 -- 172.21.15.81:0/4151956782 learned_addr learned my addr 172.21.15.81:0/4151956782 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:09.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.576+0000 7ff5c0a33640 1 --2- 172.21.15.81:0/4151956782 >> v2:172.21.15.93:3300/0 conn(0x7ff5bc06e7c0 0x7ff5bc0afb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.576+0000 7ff5c1234640 1 --2- 172.21.15.81:0/4151956782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5bc0b2720 0x7ff5bc0b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.576+0000 7ff5bbfff640 1 -- 172.21.15.81:0/4151956782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5bc0b2720 msgr2=0x7ff5bc0b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:09.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.576+0000 7ff5bbfff640 1 --2- 172.21.15.81:0/4151956782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5bc0b2720 0x7ff5bc0b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.576+0000 7ff5bbfff640 1 -- 172.21.15.81:0/4151956782 >> v2:172.21.15.93:3300/0 conn(0x7ff5bc06e7c0 msgr2=0x7ff5bc0afb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:09.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.576+0000 7ff5bbfff640 1 --2- 172.21.15.81:0/4151956782 >> v2:172.21.15.93:3300/0 conn(0x7ff5bc06e7c0 0x7ff5bc0afb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.576+0000 7ff5bbfff640 1 -- 172.21.15.81:0/4151956782 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc06c3e0 con 0x7ff5bc0ce870 2023-11-08T16:30:09.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.578+0000 7ff5bbfff640 1 --2- 172.21.15.81:0/4151956782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5bc0ce870 0x7ff5bc06d8f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0d1d60 tx=0x7ff5ac02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27282fa161fdd300 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:09.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.578+0000 7ff5bb7fe640 1 -- 172.21.15.81:0/4151956782 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac02c9e0 con 0x7ff5bc0ce870 2023-11-08T16:30:09.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.578+0000 7ff5bb7fe640 1 -- 172.21.15.81:0/4151956782 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ac02cb40 con 0x7ff5bc0ce870 2023-11-08T16:30:09.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.578+0000 7ff5bb7fe640 1 -- 172.21.15.81:0/4151956782 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac0343f0 con 0x7ff5bc0ce870 2023-11-08T16:30:09.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.579+0000 7ff5c2c97640 1 -- 172.21.15.81:0/4151956782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5bc0ce870 msgr2=0x7ff5bc06d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:09.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.579+0000 7ff5c2c97640 1 --2- 172.21.15.81:0/4151956782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5bc0ce870 0x7ff5bc06d8f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0d1d60 tx=0x7ff5ac02be40 comp rx=0 tx=0).stop 2023-11-08T16:30:09.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.579+0000 7ff5c2c97640 1 -- 172.21.15.81:0/4151956782 shutdown_connections 2023-11-08T16:30:09.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.579+0000 7ff5c2c97640 1 --2- 172.21.15.81:0/4151956782 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5bc0b2720 0x7ff5bc0b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.579+0000 7ff5c2c97640 1 --2- 172.21.15.81:0/4151956782 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5bc0ce870 0x7ff5bc06d8f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.579+0000 7ff5c2c97640 1 --2- 172.21.15.81:0/4151956782 >> v2:172.21.15.93:3300/0 conn(0x7ff5bc06e7c0 0x7ff5bc0afb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.579+0000 7ff5c2c97640 1 -- 172.21.15.81:0/4151956782 >> 172.21.15.81:0/4151956782 conn(0x7ff5bc0cd720 msgr2=0x7ff5bc0cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:09.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.579+0000 7ff5c2c97640 1 -- 172.21.15.81:0/4151956782 shutdown_connections 2023-11-08T16:30:09.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.579+0000 7ff5c2c97640 1 -- 172.21.15.81:0/4151956782 wait complete. 2023-11-08T16:30:09.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.581+0000 7ff5c2c97640 1 Processor -- start 2023-11-08T16:30:09.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.581+0000 7ff5c2c97640 1 -- start start 2023-11-08T16:30:09.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.582+0000 7ff5c2c97640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5bc06e7c0 0x7ff5bc0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.582+0000 7ff5c2c97640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5bc0b2720 0x7ff5bc1f25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.582+0000 7ff5c2c97640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5bc0ce870 0x7ff5bc0c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.582+0000 7ff5c2c97640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc0b5430 con 0x7ff5bc0ce870 2023-11-08T16:30:09.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.582+0000 7ff5c2c97640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc0b46d0 con 0x7ff5bc0b2720 2023-11-08T16:30:09.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.582+0000 7ff5c2c97640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc0b4d70 con 0x7ff5bc06e7c0 2023-11-08T16:30:09.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.582+0000 7ff5c1234640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5bc0ce870 0x7ff5bc0c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.582+0000 7ff5c1234640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5bc0ce870 0x7ff5bc0c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59246/0 (socket says 172.21.15.81:59246) 2023-11-08T16:30:09.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.582+0000 7ff5c1234640 1 -- 172.21.15.81:0/2874884102 learned_addr learned my addr 172.21.15.81:0/2874884102 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:09.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5c0a33640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5bc06e7c0 0x7ff5bc0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5bbfff640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5bc0b2720 0x7ff5bc1f25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5c1234640 1 -- 172.21.15.81:0/2874884102 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5bc06e7c0 msgr2=0x7ff5bc0af5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:09.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5c1234640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5bc06e7c0 0x7ff5bc0af5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5c1234640 1 -- 172.21.15.81:0/2874884102 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5bc0b2720 msgr2=0x7ff5bc1f25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:09.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5c1234640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5bc0b2720 0x7ff5bc1f25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5c1234640 1 -- 172.21.15.81:0/2874884102 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac009b30 con 0x7ff5bc0ce870 2023-11-08T16:30:09.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5c1234640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5bc0ce870 0x7ff5bc0c8dc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400d5d0 tx=0x7ff5b400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:09.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5b97fa640 1 -- 172.21.15.81:0/2874884102 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b40125f0 con 0x7ff5bc0ce870 2023-11-08T16:30:09.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5b97fa640 1 -- 172.21.15.81:0/2874884102 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b4012770 con 0x7ff5bc0ce870 2023-11-08T16:30:09.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5c2c97640 1 -- 172.21.15.81:0/2874884102 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc0b5050 con 0x7ff5bc0ce870 2023-11-08T16:30:09.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5b97fa640 1 -- 172.21.15.81:0/2874884102 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4012b20 con 0x7ff5bc0ce870 2023-11-08T16:30:09.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.583+0000 7ff5c2c97640 1 -- 172.21.15.81:0/2874884102 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc0b4420 con 0x7ff5bc0ce870 2023-11-08T16:30:09.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.585+0000 7ff5c2c97640 1 -- 172.21.15.81:0/2874884102 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5bc0b5570 con 0x7ff5bc0ce870 2023-11-08T16:30:09.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.585+0000 7ff5b97fa640 1 -- 172.21.15.81:0/2874884102 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5b4012c80 con 0x7ff5bc0ce870 2023-11-08T16:30:09.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.586+0000 7ff5b97fa640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff59c06ae70 0x7ff59c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:09.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.586+0000 7ff5b97fa640 1 -- 172.21.15.81:0/2874884102 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff5b4089c10 con 0x7ff5bc0ce870 2023-11-08T16:30:09.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.587+0000 7ff5c0a33640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff59c06ae70 0x7ff59c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:09.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.587+0000 7ff5c0a33640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff59c06ae70 0x7ff59c06d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4006fd0 tx=0x7ff5a4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:09.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.591+0000 7ff5b97fa640 1 -- 172.21.15.81:0/2874884102 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff5bc0b5570 con 0x7ff5bc0ce870 2023-11-08T16:30:09.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.780+0000 7ff5c2c97640 1 -- 172.21.15.81:0/2874884102 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5bc0b5570 con 0x7ff5bc0ce870 2023-11-08T16:30:09.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.782+0000 7ff5b97fa640 1 -- 172.21.15.81:0/2874884102 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7ff5bc0b5570 con 0x7ff5bc0ce870 2023-11-08T16:30:09.785 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:09.785 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:30:09.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 -- 172.21.15.81:0/2874884102 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff59c06ae70 msgr2=0x7ff59c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:09.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff59c06ae70 0x7ff59c06d330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4006fd0 tx=0x7ff5a4008040 comp rx=0 tx=0).stop 2023-11-08T16:30:09.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 -- 172.21.15.81:0/2874884102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5bc0ce870 msgr2=0x7ff5bc0c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:09.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5bc0ce870 0x7ff5bc0c8dc0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400d5d0 tx=0x7ff5b400da90 comp rx=0 tx=0).stop 2023-11-08T16:30:09.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 -- 172.21.15.81:0/2874884102 shutdown_connections 2023-11-08T16:30:09.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff59c06ae70 0x7ff59c06d330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff5bc0ce870 0x7ff5bc0c8dc0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff5bc0b2720 0x7ff5bc1f25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 --2- 172.21.15.81:0/2874884102 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff5bc06e7c0 0x7ff5bc0af5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:09.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 -- 172.21.15.81:0/2874884102 >> 172.21.15.81:0/2874884102 conn(0x7ff5bc0cd720 msgr2=0x7ff5bc06b170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:09.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 -- 172.21.15.81:0/2874884102 shutdown_connections 2023-11-08T16:30:09.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:09.786+0000 7ff5c2c97640 1 -- 172.21.15.81:0/2874884102 wait complete. 2023-11-08T16:30:09.807 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T16:30:09.808 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:30:10.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.021+0000 7f2a8d225640 1 Processor -- start 2023-11-08T16:30:10.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.021+0000 7f2a8d225640 1 -- start start 2023-11-08T16:30:10.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.023+0000 7f2a8d225640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2a880cf170 0x7f2a8815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.023+0000 7f2a8d225640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a88070470 con 0x7f2a8815aac0 2023-11-08T16:30:10.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.023+0000 7f2a8d225640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8806d580 con 0x7f2a880ced10 2023-11-08T16:30:10.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.023+0000 7f2a8d225640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8806f650 con 0x7f2a880cf170 2023-11-08T16:30:10.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.023+0000 7f2a87fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2a8815aac0 0x7f2a880711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60254/0 (socket says 172.21.15.81:60254) 2023-11-08T16:30:10.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.023+0000 7f2a87fff640 1 -- 172.21.15.81:0/900366517 learned_addr learned my addr 172.21.15.81:0/900366517 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:10.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.023+0000 7f2a8ca24640 1 --2- 172.21.15.81:0/900366517 >> v2:172.21.15.81:3300/0 conn(0x7f2a880cf170 0x7f2a8815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.023+0000 7f2a86ffd640 1 -- 172.21.15.81:0/900366517 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256020542 0 0) 0x7f2a88070470 con 0x7f2a8815aac0 2023-11-08T16:30:10.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.024+0000 7f2a8ca24640 1 -- 172.21.15.81:0/900366517 >> v1:172.21.15.93:6789/0 conn(0x7f2a880ced10 legacy=0x7f2a8815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:10.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.024+0000 7f2a8ca24640 1 -- 172.21.15.81:0/900366517 >> v1:172.21.15.81:6789/0 conn(0x7f2a8815aac0 legacy=0x7f2a880711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.024+0000 7f2a8ca24640 1 -- 172.21.15.81:0/900366517 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8806ca00 con 0x7f2a880cf170 2023-11-08T16:30:10.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.025+0000 7f2a8ca24640 1 --2- 172.21.15.81:0/900366517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a880cf170 0x7f2a8815b5b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009d90 tx=0x7f2a8002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7573ecebc5c100f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:10.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.025+0000 7f2a86ffd640 1 -- 172.21.15.81:0/900366517 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8002cc60 con 0x7f2a880cf170 2023-11-08T16:30:10.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.025+0000 7f2a86ffd640 1 -- 172.21.15.81:0/900366517 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8002cdc0 con 0x7f2a880cf170 2023-11-08T16:30:10.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.025+0000 7f2a86ffd640 1 -- 172.21.15.81:0/900366517 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80034660 con 0x7f2a880cf170 2023-11-08T16:30:10.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.026+0000 7f2a8d225640 1 -- 172.21.15.81:0/900366517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a880cf170 msgr2=0x7f2a8815b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.026+0000 7f2a8d225640 1 --2- 172.21.15.81:0/900366517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a880cf170 0x7f2a8815b5b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2a80009d90 tx=0x7f2a8002beb0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.026+0000 7f2a8d225640 1 -- 172.21.15.81:0/900366517 shutdown_connections 2023-11-08T16:30:10.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.026+0000 7f2a8d225640 1 --2- 172.21.15.81:0/900366517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a880cf170 0x7f2a8815b5b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.026+0000 7f2a8d225640 1 -- 172.21.15.81:0/900366517 >> 172.21.15.81:0/900366517 conn(0x7f2a8806a4a0 msgr2=0x7f2a8806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:10.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.026+0000 7f2a8d225640 1 -- 172.21.15.81:0/900366517 shutdown_connections 2023-11-08T16:30:10.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.026+0000 7f2a8d225640 1 -- 172.21.15.81:0/900366517 wait complete. 2023-11-08T16:30:10.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.027+0000 7f2a8d225640 1 Processor -- start 2023-11-08T16:30:10.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.028+0000 7f2a8d225640 1 -- start start 2023-11-08T16:30:10.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a8d225640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a880ced10 0x7f2a88155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a8d225640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a880cf170 0x7f2a881e53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a8d225640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a8815aac0 0x7f2a880b0a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a8d225640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a88156400 con 0x7f2a880cf170 2023-11-08T16:30:10.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a8d225640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880afe90 con 0x7f2a880ced10 2023-11-08T16:30:10.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a8d225640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a880af280 con 0x7f2a8815aac0 2023-11-08T16:30:10.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a877fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a880cf170 0x7f2a881e53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a877fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a880cf170 0x7f2a881e53c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59266/0 (socket says 172.21.15.81:59266) 2023-11-08T16:30:10.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a877fe640 1 -- 172.21.15.81:0/3565239936 learned_addr learned my addr 172.21.15.81:0/3565239936 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:10.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a877fe640 1 -- 172.21.15.81:0/3565239936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a8815aac0 msgr2=0x7f2a880b0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a877fe640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a8815aac0 0x7f2a880b0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a877fe640 1 -- 172.21.15.81:0/3565239936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a880ced10 msgr2=0x7f2a88155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a877fe640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a880ced10 0x7f2a88155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.029+0000 7f2a877fe640 1 -- 172.21.15.81:0/3565239936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a80009a70 con 0x7f2a880cf170 2023-11-08T16:30:10.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.030+0000 7f2a877fe640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a880cf170 0x7f2a881e53c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c430 tx=0x7f2a7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:10.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.030+0000 7f2a84ff9640 1 -- 172.21.15.81:0/3565239936 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c011790 con 0x7f2a880cf170 2023-11-08T16:30:10.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.030+0000 7f2a84ff9640 1 -- 172.21.15.81:0/3565239936 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c0118f0 con 0x7f2a880cf170 2023-11-08T16:30:10.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.030+0000 7f2a84ff9640 1 -- 172.21.15.81:0/3565239936 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c011ce0 con 0x7f2a880cf170 2023-11-08T16:30:10.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.030+0000 7f2a8d225640 1 -- 172.21.15.81:0/3565239936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a880af560 con 0x7f2a880cf170 2023-11-08T16:30:10.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.030+0000 7f2a8d225640 1 -- 172.21.15.81:0/3565239936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a880afb20 con 0x7f2a880cf170 2023-11-08T16:30:10.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.031+0000 7f2a8d225640 1 -- 172.21.15.81:0/3565239936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a88067800 con 0x7f2a880cf170 2023-11-08T16:30:10.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.032+0000 7f2a84ff9640 1 -- 172.21.15.81:0/3565239936 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a7c01c460 con 0x7f2a880cf170 2023-11-08T16:30:10.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.033+0000 7f2a84ff9640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a7406ae70 0x7f2a7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.033+0000 7f2a84ff9640 1 -- 172.21.15.81:0/3565239936 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2a7c0896e0 con 0x7f2a880cf170 2023-11-08T16:30:10.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.033+0000 7f2a87fff640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a7406ae70 0x7f2a7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.034+0000 7f2a87fff640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a7406ae70 0x7f2a7406d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2a78007fd0 tx=0x7f2a78007960 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:10.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.037+0000 7f2a84ff9640 1 -- 172.21.15.81:0/3565239936 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2a7c054660 con 0x7f2a880cf170 2023-11-08T16:30:10.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.258+0000 7f2a8d225640 1 -- 172.21.15.81:0/3565239936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a881e4e80 con 0x7f2a880cf170 2023-11-08T16:30:10.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.259+0000 7f2a84ff9640 1 -- 172.21.15.81:0/3565239936 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f2a7c057c80 con 0x7f2a880cf170 2023-11-08T16:30:10.261 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:10.261 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4190352668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4190352668},{"type":"v1","addr":"172.21.15.81:6835","nonce":4190352668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2636431103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2636431103},{"type":"v1","addr":"172.21.15.93:6835","nonce":2636431103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3681499690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3681499690},{"type":"v1","addr":"172.21.15.81:6837","nonce":3681499690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:29:03.181472+0000","modified":"2023-11-08T16:29:06.272041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6834/2087517381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2087517381},{"type":"v1","addr":"172.21.15.93:6834","nonce":2087517381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T16:30:10.262 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 8 2023-11-08T16:30:10.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 -- 172.21.15.81:0/3565239936 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a7406ae70 msgr2=0x7f2a7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a7406ae70 0x7f2a7406d330 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2a78007fd0 tx=0x7f2a78007960 comp rx=0 tx=0).stop 2023-11-08T16:30:10.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 -- 172.21.15.81:0/3565239936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a880cf170 msgr2=0x7f2a881e53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a880cf170 0x7f2a881e53c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c430 tx=0x7f2a7c00c8f0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 -- 172.21.15.81:0/3565239936 shutdown_connections 2023-11-08T16:30:10.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2a7406ae70 0x7f2a7406d330 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2a8815aac0 0x7f2a880b0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2a880cf170 0x7f2a881e53c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 --2- 172.21.15.81:0/3565239936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a880ced10 0x7f2a88155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 -- 172.21.15.81:0/3565239936 >> 172.21.15.81:0/3565239936 conn(0x7f2a8806a4a0 msgr2=0x7f2a880b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:10.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 -- 172.21.15.81:0/3565239936 shutdown_connections 2023-11-08T16:30:10.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.261+0000 7f2a8d225640 1 -- 172.21.15.81:0/3565239936 wait complete. 2023-11-08T16:30:10.280 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:30:10.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.488+0000 7f9aa26af640 1 Processor -- start 2023-11-08T16:30:10.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.488+0000 7f9aa26af640 1 -- start start 2023-11-08T16:30:10.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.490+0000 7f9aa26af640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a9c0ade50 0x7f9a9c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.490+0000 7f9aa26af640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9a9c15a680 0x7f9a9c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.490+0000 7f9aa26af640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9c06b630 con 0x7f9a9c15b020 2023-11-08T16:30:10.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.490+0000 7f9aa26af640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9c0b1860 con 0x7f9a9c15a680 2023-11-08T16:30:10.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.490+0000 7f9aa26af640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c0ca9c0 con 0x7f9a9c0ade50 2023-11-08T16:30:10.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.490+0000 7f9a93fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9a9c15a680 0x7f9a9c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.490+0000 7f9aa0c4c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f9a9c15b020 0x7f9a9c0c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60270/0 (socket says 172.21.15.81:60270) 2023-11-08T16:30:10.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.490+0000 7f9aa0c4c640 1 -- 172.21.15.81:0/461915686 learned_addr learned my addr 172.21.15.81:0/461915686 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:10.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.490+0000 7f9a9bfff640 1 --2- 172.21.15.81:0/461915686 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a9c0ade50 0x7f9a9c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.491+0000 7f9a93fff640 1 -- 172.21.15.81:0/461915686 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a9c0ade50 msgr2=0x7f9a9c0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:10.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.491+0000 7f9a93fff640 1 --2- 172.21.15.81:0/461915686 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a9c0ade50 0x7f9a9c0ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.491+0000 7f9a93fff640 1 -- 172.21.15.81:0/461915686 >> v1:172.21.15.81:6789/0 conn(0x7f9a9c15b020 legacy=0x7f9a9c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.491+0000 7f9a93fff640 1 -- 172.21.15.81:0/461915686 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c0cabb0 con 0x7f9a9c15a680 2023-11-08T16:30:10.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.491+0000 7f9a9b7fe640 1 -- 172.21.15.81:0/461915686 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3594555320 0 0) 0x7f9a9c06b630 con 0x7f9a9c15b020 2023-11-08T16:30:10.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.492+0000 7f9a93fff640 1 --2- 172.21.15.81:0/461915686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a9c15a680 0x7f9a9c15aae0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9a840097a0 tx=0x7f9a8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb06cdefeb5d1a97 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:10.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.492+0000 7f9a9b7fe640 1 -- 172.21.15.81:0/461915686 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84031830 con 0x7f9a9c15a680 2023-11-08T16:30:10.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.492+0000 7f9a9b7fe640 1 -- 172.21.15.81:0/461915686 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a84031990 con 0x7f9a9c15a680 2023-11-08T16:30:10.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.492+0000 7f9a9b7fe640 1 -- 172.21.15.81:0/461915686 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a84031d20 con 0x7f9a9c15a680 2023-11-08T16:30:10.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.493+0000 7f9aa26af640 1 -- 172.21.15.81:0/461915686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a9c15a680 msgr2=0x7f9a9c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.493+0000 7f9aa26af640 1 --2- 172.21.15.81:0/461915686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a9c15a680 0x7f9a9c15aae0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9a840097a0 tx=0x7f9a8402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.493+0000 7f9aa26af640 1 -- 172.21.15.81:0/461915686 shutdown_connections 2023-11-08T16:30:10.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.493+0000 7f9aa26af640 1 --2- 172.21.15.81:0/461915686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a9c15a680 0x7f9a9c15aae0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.493+0000 7f9aa26af640 1 --2- 172.21.15.81:0/461915686 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a9c0ade50 0x7f9a9c0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.493+0000 7f9aa26af640 1 -- 172.21.15.81:0/461915686 >> 172.21.15.81:0/461915686 conn(0x7f9a9c0cb490 msgr2=0x7f9a9c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:10.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.493+0000 7f9aa26af640 1 -- 172.21.15.81:0/461915686 shutdown_connections 2023-11-08T16:30:10.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.493+0000 7f9aa26af640 1 -- 172.21.15.81:0/461915686 wait complete. 2023-11-08T16:30:10.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.495+0000 7f9aa26af640 1 Processor -- start 2023-11-08T16:30:10.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.495+0000 7f9aa26af640 1 -- start start 2023-11-08T16:30:10.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9aa26af640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a9c0ade50 0x7f9a9c156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9aa26af640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a9c15a680 0x7f9a9c15b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9aa26af640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9a9c15b020 0x7f9a9c15ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9aa26af640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c148780 con 0x7f9a9c0ade50 2023-11-08T16:30:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9aa26af640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c14e750 con 0x7f9a9c15b020 2023-11-08T16:30:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9aa26af640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c15c0b0 con 0x7f9a9c15a680 2023-11-08T16:30:10.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9a9bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a9c0ade50 0x7f9a9c156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9a9bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a9c0ade50 0x7f9a9c156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59278/0 (socket says 172.21.15.81:59278) 2023-11-08T16:30:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9a9bfff640 1 -- 172.21.15.81:0/4180123963 learned_addr learned my addr 172.21.15.81:0/4180123963 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9aa0c4c640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9a9c15b020 0x7f9a9c15ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9a93fff640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a9c15a680 0x7f9a9c15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9a9bfff640 1 -- 172.21.15.81:0/4180123963 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a9c15a680 msgr2=0x7f9a9c15b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9a9bfff640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a9c15a680 0x7f9a9c15b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9a9bfff640 1 -- 172.21.15.81:0/4180123963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9a9c15b020 msgr2=0x7f9a9c15ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9a9bfff640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9a9c15b020 0x7f9a9c15ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9a9bfff640 1 -- 172.21.15.81:0/4180123963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a84009400 con 0x7f9a9c0ade50 2023-11-08T16:30:10.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.496+0000 7f9aa0c4c640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9a9c15b020 0x7f9a9c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.497+0000 7f9a9bfff640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a9c0ade50 0x7f9a9c156200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00d800 tx=0x7f9a8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.497+0000 7f9a997fa640 1 -- 172.21.15.81:0/4180123963 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00b6c0 con 0x7f9a9c0ade50 2023-11-08T16:30:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.497+0000 7f9a997fa640 1 -- 172.21.15.81:0/4180123963 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c00b820 con 0x7f9a9c0ade50 2023-11-08T16:30:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.497+0000 7f9a997fa640 1 -- 172.21.15.81:0/4180123963 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c00bbb0 con 0x7f9a9c0ade50 2023-11-08T16:30:10.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.497+0000 7f9aa26af640 1 -- 172.21.15.81:0/4180123963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9c15c390 con 0x7f9a9c0ade50 2023-11-08T16:30:10.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.497+0000 7f9aa26af640 1 -- 172.21.15.81:0/4180123963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c15c950 con 0x7f9a9c0ade50 2023-11-08T16:30:10.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.498+0000 7f9aa26af640 1 -- 172.21.15.81:0/4180123963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9c04ece0 con 0x7f9a9c0ade50 2023-11-08T16:30:10.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.499+0000 7f9a997fa640 1 -- 172.21.15.81:0/4180123963 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9a8c00bd10 con 0x7f9a9c0ade50 2023-11-08T16:30:10.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.500+0000 7f9a997fa640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9a7c06ae70 0x7f9a7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.500+0000 7f9a997fa640 1 -- 172.21.15.81:0/4180123963 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9a8c089ce0 con 0x7f9a9c0ade50 2023-11-08T16:30:10.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.500+0000 7f9a93fff640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9a7c06ae70 0x7f9a7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.501+0000 7f9a93fff640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9a7c06ae70 0x7f9a7c06d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9a84036000 tx=0x7f9a84008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:10.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.504+0000 7f9a997fa640 1 -- 172.21.15.81:0/4180123963 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9a8c054c60 con 0x7f9a9c0ade50 2023-11-08T16:30:10.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.730+0000 7f9aa26af640 1 -- 172.21.15.81:0/4180123963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a9c06b8a0 con 0x7f9a9c0ade50 2023-11-08T16:30:10.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.731+0000 7f9a997fa640 1 -- 172.21.15.81:0/4180123963 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4907 (secure 0 0 0) 0x7f9a8c058280 con 0x7f9a9c0ade50 2023-11-08T16:30:10.733 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:10.733 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4190352668","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4190352668},{"type":"v1","addr":"172.21.15.81:6835","nonce":4190352668}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2636431103","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2636431103},{"type":"v1","addr":"172.21.15.93:6835","nonce":2636431103}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4264,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3681499690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3681499690},{"type":"v1","addr":"172.21.15.81:6837","nonce":3681499690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:29:03.181472+0000","modified":"2023-11-08T16:29:06.272041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4270},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4270":{"gid":4270,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6834/2087517381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2087517381},{"type":"v1","addr":"172.21.15.93:6834","nonce":2087517381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-08T16:30:10.734 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 8 2023-11-08T16:30:10.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 -- 172.21.15.81:0/4180123963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9a7c06ae70 msgr2=0x7f9a7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9a7c06ae70 0x7f9a7c06d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9a84036000 tx=0x7f9a84008030 comp rx=0 tx=0).stop 2023-11-08T16:30:10.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 -- 172.21.15.81:0/4180123963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a9c0ade50 msgr2=0x7f9a9c156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a9c0ade50 0x7f9a9c156200 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00d800 tx=0x7f9a8c00dcc0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 -- 172.21.15.81:0/4180123963 shutdown_connections 2023-11-08T16:30:10.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9a7c06ae70 0x7f9a7c06d330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9a9c15b020 0x7f9a9c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9a9c15a680 0x7f9a9c15b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 --2- 172.21.15.81:0/4180123963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9a9c0ade50 0x7f9a9c156200 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 -- 172.21.15.81:0/4180123963 >> 172.21.15.81:0/4180123963 conn(0x7f9a9c0cb490 msgr2=0x7f9a9c0b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:10.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 -- 172.21.15.81:0/4180123963 shutdown_connections 2023-11-08T16:30:10.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.734+0000 7f9aa26af640 1 -- 172.21.15.81:0/4180123963 wait complete. 2023-11-08T16:30:10.754 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:30:10.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.961+0000 7f53b2139640 1 Processor -- start 2023-11-08T16:30:10.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.961+0000 7f53b2139640 1 -- start start 2023-11-08T16:30:10.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.962+0000 7f53b2139640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f53ac06bdf0 0x7f53ac06c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.962+0000 7f53b2139640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ac06b630 con 0x7f53ac0ca480 2023-11-08T16:30:10.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.962+0000 7f53b2139640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ac0ca910 con 0x7f53ac0ade50 2023-11-08T16:30:10.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.962+0000 7f53b2139640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ac0b1860 con 0x7f53ac06bdf0 2023-11-08T16:30:10.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.963+0000 7f53a3fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f53ac0ca480 0x7f53ac15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:37684/0 (socket says 172.21.15.81:37684) 2023-11-08T16:30:10.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.963+0000 7f53a3fff640 1 -- 172.21.15.81:0/1200767210 learned_addr learned my addr 172.21.15.81:0/1200767210 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:10.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.963+0000 7f53abfff640 1 --2- 172.21.15.81:0/1200767210 >> v2:172.21.15.93:3300/0 conn(0x7f53ac06bdf0 0x7f53ac06c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.963+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3867581297 0 0) 0x7f53ac06b630 con 0x7f53ac0ca480 2023-11-08T16:30:10.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.963+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5398003520 con 0x7f53ac0ca480 2023-11-08T16:30:10.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2814018886 0 0) 0x7f5398003520 con 0x7f53ac0ca480 2023-11-08T16:30:10.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 >> v2:172.21.15.93:3300/0 conn(0x7f53ac06bdf0 msgr2=0x7f53ac06c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:10.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 --2- 172.21.15.81:0/1200767210 >> v2:172.21.15.93:3300/0 conn(0x7f53ac06bdf0 0x7f53ac06c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 >> v1:172.21.15.93:6789/0 conn(0x7f53ac0ade50 legacy=0x7f53ac15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac0cabb0 con 0x7f53ac0ca480 2023-11-08T16:30:10.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f5394001670 con 0x7f53ac0ca480 2023-11-08T16:30:10.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53abfff640 1 --2- 172.21.15.81:0/1200767210 >> v2:172.21.15.93:3300/0 conn(0x7f53ac06bdf0 0x7f53ac06c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:30:10.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 >> v1:172.21.15.81:6789/0 conn(0x7f53ac0ca480 legacy=0x7f53ac15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 --2- 172.21.15.81:0/1200767210 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5398003a50 0x7f5398003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 --2- 172.21.15.81:0/1200767210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53980047c0 0x7f5398004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 --2- 172.21.15.81:0/1200767210 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f53980051d0 0x7f539800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539800c0d0 con 0x7f53980047c0 2023-11-08T16:30:10.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f539800c240 con 0x7f5398003a50 2023-11-08T16:30:10.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539800c400 con 0x7f53980051d0 2023-11-08T16:30:10.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53a3fff640 1 --2- 172.21.15.81:0/1200767210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53980047c0 0x7f5398004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53a3fff640 1 -- 172.21.15.81:0/1200767210 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f53980051d0 msgr2=0x7f539800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:10.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53a3fff640 1 --2- 172.21.15.81:0/1200767210 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f53980051d0 0x7f539800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53a3fff640 1 -- 172.21.15.81:0/1200767210 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5398003a50 msgr2=0x7f5398003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:10.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53a3fff640 1 --2- 172.21.15.81:0/1200767210 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5398003a50 0x7f5398003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.964+0000 7f53a3fff640 1 -- 172.21.15.81:0/1200767210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5398003520 con 0x7f53980047c0 2023-11-08T16:30:10.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.965+0000 7f53abfff640 1 -- 172.21.15.81:0/1200767210 reap_dead start 2023-11-08T16:30:10.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.966+0000 7f53a3fff640 1 --2- 172.21.15.81:0/1200767210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53980047c0 0x7f5398004c40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f539400c210 tx=0x7f53980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4c5354a6b505b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:10.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.966+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394005ad0 con 0x7f53980047c0 2023-11-08T16:30:10.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.966+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5394005c30 con 0x7f53980047c0 2023-11-08T16:30:10.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.966+0000 7f53aaffd640 1 -- 172.21.15.81:0/1200767210 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539402ca70 con 0x7f53980047c0 2023-11-08T16:30:10.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.966+0000 7f53b2139640 1 -- 172.21.15.81:0/1200767210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53980047c0 msgr2=0x7f5398004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.966+0000 7f53b2139640 1 --2- 172.21.15.81:0/1200767210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53980047c0 0x7f5398004c40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f539400c210 tx=0x7f53980057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.967+0000 7f53b2139640 1 -- 172.21.15.81:0/1200767210 shutdown_connections 2023-11-08T16:30:10.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.967+0000 7f53b2139640 1 --2- 172.21.15.81:0/1200767210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53980047c0 0x7f5398004c40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.967+0000 7f53b2139640 1 -- 172.21.15.81:0/1200767210 >> 172.21.15.81:0/1200767210 conn(0x7f53ac0cb490 msgr2=0x7f53ac0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:10.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.967+0000 7f53b2139640 1 -- 172.21.15.81:0/1200767210 shutdown_connections 2023-11-08T16:30:10.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.967+0000 7f53b2139640 1 -- 172.21.15.81:0/1200767210 wait complete. 2023-11-08T16:30:10.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.968+0000 7f53b2139640 1 Processor -- start 2023-11-08T16:30:10.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.968+0000 7f53b2139640 1 -- start start 2023-11-08T16:30:10.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.969+0000 7f53b2139640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f53980047c0 0x7f53ac0cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.969+0000 7f53b2139640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53ac1574a0 0x7f53ac15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.969+0000 7f53b2139640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53ac15ae70 0x7f53ac1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.969+0000 7f53b2139640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac1f3730 con 0x7f53ac1574a0 2023-11-08T16:30:10.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.969+0000 7f53b2139640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac0cdf10 con 0x7f53ac15ae70 2023-11-08T16:30:10.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.969+0000 7f53b2139640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ac0cc930 con 0x7f53980047c0 2023-11-08T16:30:10.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53a3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53ac1574a0 0x7f53ac15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53a3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53ac1574a0 0x7f53ac15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51184/0 (socket says 172.21.15.81:51184) 2023-11-08T16:30:10.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53a3fff640 1 -- 172.21.15.81:0/176245018 learned_addr learned my addr 172.21.15.81:0/176245018 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:10.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53abfff640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53ac15ae70 0x7f53ac1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53ab7fe640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f53980047c0 0x7f53ac0cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53a3fff640 1 -- 172.21.15.81:0/176245018 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f53980047c0 msgr2=0x7f53ac0cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53a3fff640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f53980047c0 0x7f53ac0cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53a3fff640 1 -- 172.21.15.81:0/176245018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53ac15ae70 msgr2=0x7f53ac1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:10.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53a3fff640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53ac15ae70 0x7f53ac1588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:10.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53a3fff640 1 -- 172.21.15.81:0/176245018 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ac0ccc10 con 0x7f53ac1574a0 2023-11-08T16:30:10.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53abfff640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53ac15ae70 0x7f53ac1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:10.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.970+0000 7f53a3fff640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53ac1574a0 0x7f53ac15a930 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f539402c3d0 tx=0x7f539402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:10.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.971+0000 7f53a8ff9640 1 -- 172.21.15.81:0/176245018 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539403c040 con 0x7f53ac1574a0 2023-11-08T16:30:10.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.971+0000 7f53a8ff9640 1 -- 172.21.15.81:0/176245018 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539402cd80 con 0x7f53ac1574a0 2023-11-08T16:30:10.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.971+0000 7f53b2139640 1 -- 172.21.15.81:0/176245018 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ac0ccef0 con 0x7f53ac1574a0 2023-11-08T16:30:10.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.971+0000 7f53a8ff9640 1 -- 172.21.15.81:0/176245018 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394007290 con 0x7f53ac1574a0 2023-11-08T16:30:10.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.971+0000 7f53b2139640 1 -- 172.21.15.81:0/176245018 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ac0cd4b0 con 0x7f53ac1574a0 2023-11-08T16:30:10.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.972+0000 7f53b2139640 1 -- 172.21.15.81:0/176245018 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53ac04ece0 con 0x7f53ac1574a0 2023-11-08T16:30:10.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.973+0000 7f53a8ff9640 1 -- 172.21.15.81:0/176245018 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5394032070 con 0x7f53ac1574a0 2023-11-08T16:30:10.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.974+0000 7f53a8ff9640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538c06ae70 0x7f538c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:10.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.974+0000 7f53a8ff9640 1 -- 172.21.15.81:0/176245018 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f53940aadc0 con 0x7f53ac1574a0 2023-11-08T16:30:10.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.974+0000 7f53ab7fe640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538c06ae70 0x7f538c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:10.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.974+0000 7f53ab7fe640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538c06ae70 0x7f538c06d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f539c000f60 tx=0x7f539c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:10.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:10.978+0000 7f53a8ff9640 1 -- 172.21.15.81:0/176245018 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5394075d40 con 0x7f53ac1574a0 2023-11-08T16:30:11.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.169+0000 7f53b2139640 1 -- 172.21.15.81:0/176245018 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53ac0cc1d0 con 0x7f53ac1574a0 2023-11-08T16:30:11.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.172+0000 7f53a8ff9640 1 -- 172.21.15.81:0/176245018 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16742 (secure 0 0 0) 0x7f5394079360 con 0x7f53ac1574a0 2023-11-08T16:30:11.174 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:11.175 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":23,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:29:43.825045+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:29:00.903464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-08T16:29:02.199149+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.17","mappings":[{"from":5,"to":1}]},{"pgid":"3.28","mappings":[{"from":3,"to":1}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3a","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:30:11.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.174+0000 7f53b2139640 1 -- 172.21.15.81:0/176245018 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538c06ae70 msgr2=0x7f538c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.175+0000 7f53b2139640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538c06ae70 0x7f538c06d330 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f539c000f60 tx=0x7f539c009040 comp rx=0 tx=0).stop 2023-11-08T16:30:11.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.175+0000 7f53b2139640 1 -- 172.21.15.81:0/176245018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53ac1574a0 msgr2=0x7f53ac15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.175+0000 7f53b2139640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53ac1574a0 0x7f53ac15a930 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f539402c3d0 tx=0x7f539402e070 comp rx=0 tx=0).stop 2023-11-08T16:30:11.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.175+0000 7f53b2139640 1 -- 172.21.15.81:0/176245018 shutdown_connections 2023-11-08T16:30:11.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.175+0000 7f53b2139640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538c06ae70 0x7f538c06d330 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.175+0000 7f53b2139640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53ac15ae70 0x7f53ac1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.175+0000 7f53b2139640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f53ac1574a0 0x7f53ac15a930 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.175+0000 7f53b2139640 1 --2- 172.21.15.81:0/176245018 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f53980047c0 0x7f53ac0cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.175+0000 7f53b2139640 1 -- 172.21.15.81:0/176245018 >> 172.21.15.81:0/176245018 conn(0x7f53ac0cb490 msgr2=0x7f53ac071180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:11.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.175+0000 7f53b2139640 1 -- 172.21.15.81:0/176245018 shutdown_connections 2023-11-08T16:30:11.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.175+0000 7f53b2139640 1 -- 172.21.15.81:0/176245018 wait complete. 2023-11-08T16:30:11.194 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:30:11.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.396+0000 7fd254ba5640 1 Processor -- start 2023-11-08T16:30:11.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.397+0000 7fd254ba5640 1 -- start start 2023-11-08T16:30:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.398+0000 7fd254ba5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd25006f7c0 0x7fd2500cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.398+0000 7fd254ba5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25006cd70 con 0x7fd2500ad940 2023-11-08T16:30:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.398+0000 7fd254ba5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2500ca500 con 0x7fd25015ac20 2023-11-08T16:30:11.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.398+0000 7fd254ba5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2500ae130 con 0x7fd25006f7c0 2023-11-08T16:30:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24ffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd2500ad940 0x7fd25006b390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:37700/0 (socket says 172.21.15.81:37700) 2023-11-08T16:30:11.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24ffff640 1 -- 172.21.15.81:0/3361218330 learned_addr learned my addr 172.21.15.81:0/3361218330 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:11.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd247fff640 1 --2- 172.21.15.81:0/3361218330 >> v2:172.21.15.93:3300/0 conn(0x7fd25006f7c0 0x7fd2500cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944652832 0 0) 0x7fd25006cd70 con 0x7fd2500ad940 2023-11-08T16:30:11.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd23c003520 con 0x7fd2500ad940 2023-11-08T16:30:11.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 472748717 0 0) 0x7fd23c003520 con 0x7fd2500ad940 2023-11-08T16:30:11.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 >> v2:172.21.15.93:3300/0 conn(0x7fd25006f7c0 msgr2=0x7fd2500cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:11.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24effd640 1 --2- 172.21.15.81:0/3361218330 >> v2:172.21.15.93:3300/0 conn(0x7fd25006f7c0 0x7fd2500cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 >> v1:172.21.15.93:6789/0 conn(0x7fd25015ac20 legacy=0x7fd250070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25006d030 con 0x7fd2500ad940 2023-11-08T16:30:11.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fd2480019f0 con 0x7fd2500ad940 2023-11-08T16:30:11.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 >> v1:172.21.15.81:6789/0 conn(0x7fd2500ad940 legacy=0x7fd25006b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.399+0000 7fd24effd640 1 --2- 172.21.15.81:0/3361218330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd23c0039a0 0x7fd23c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24effd640 1 --2- 172.21.15.81:0/3361218330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd23c0047c0 0x7fd23c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24effd640 1 --2- 172.21.15.81:0/3361218330 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd23c0051d0 0x7fd23c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2500ae130 con 0x7fd23c0039a0 2023-11-08T16:30:11.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c00c100 con 0x7fd23c0047c0 2023-11-08T16:30:11.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c00c2c0 con 0x7fd23c0051d0 2023-11-08T16:30:11.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24f7fe640 1 --2- 172.21.15.81:0/3361218330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd23c0039a0 0x7fd23c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24f7fe640 1 -- 172.21.15.81:0/3361218330 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd23c0051d0 msgr2=0x7fd23c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:11.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24f7fe640 1 --2- 172.21.15.81:0/3361218330 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd23c0051d0 0x7fd23c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24f7fe640 1 -- 172.21.15.81:0/3361218330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd23c0047c0 msgr2=0x7fd23c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:11.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24f7fe640 1 --2- 172.21.15.81:0/3361218330 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd23c0047c0 0x7fd23c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24f7fe640 1 -- 172.21.15.81:0/3361218330 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c003520 con 0x7fd23c0039a0 2023-11-08T16:30:11.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.400+0000 7fd24ffff640 1 -- 172.21.15.81:0/3361218330 reap_dead start 2023-11-08T16:30:11.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.401+0000 7fd24f7fe640 1 --2- 172.21.15.81:0/3361218330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd23c0039a0 0x7fd23c003dc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd240001750 tx=0x7fd23c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd142d889b2565ce server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:11.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.402+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2400052d0 con 0x7fd23c0039a0 2023-11-08T16:30:11.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.402+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24002e050 con 0x7fd23c0039a0 2023-11-08T16:30:11.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.402+0000 7fd24effd640 1 -- 172.21.15.81:0/3361218330 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd240003830 con 0x7fd23c0039a0 2023-11-08T16:30:11.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.402+0000 7fd254ba5640 1 -- 172.21.15.81:0/3361218330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd23c0039a0 msgr2=0x7fd23c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.402+0000 7fd254ba5640 1 --2- 172.21.15.81:0/3361218330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd23c0039a0 0x7fd23c003dc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd240001750 tx=0x7fd23c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.402+0000 7fd254ba5640 1 -- 172.21.15.81:0/3361218330 shutdown_connections 2023-11-08T16:30:11.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.402+0000 7fd254ba5640 1 --2- 172.21.15.81:0/3361218330 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd23c0039a0 0x7fd23c003dc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.402+0000 7fd254ba5640 1 -- 172.21.15.81:0/3361218330 >> 172.21.15.81:0/3361218330 conn(0x7fd2500b04d0 msgr2=0x7fd2500b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:11.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.402+0000 7fd254ba5640 1 -- 172.21.15.81:0/3361218330 shutdown_connections 2023-11-08T16:30:11.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.402+0000 7fd254ba5640 1 -- 172.21.15.81:0/3361218330 wait complete. 2023-11-08T16:30:11.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.404+0000 7fd254ba5640 1 Processor -- start 2023-11-08T16:30:11.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.404+0000 7fd254ba5640 1 -- start start 2023-11-08T16:30:11.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.405+0000 7fd254ba5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd23c0039a0 0x7fd25015cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.405+0000 7fd254ba5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2501590d0 0x7fd25015d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.405+0000 7fd254ba5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd250151b50 0x7fd250151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.405+0000 7fd254ba5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd250152520 con 0x7fd2501590d0 2023-11-08T16:30:11.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.405+0000 7fd254ba5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd250152bd0 con 0x7fd23c0039a0 2023-11-08T16:30:11.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.405+0000 7fd254ba5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd250157da0 con 0x7fd250151b50 2023-11-08T16:30:11.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.405+0000 7fd247fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2501590d0 0x7fd25015d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.406+0000 7fd247fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2501590d0 0x7fd25015d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51202/0 (socket says 172.21.15.81:51202) 2023-11-08T16:30:11.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.406+0000 7fd247fff640 1 -- 172.21.15.81:0/2020106853 learned_addr learned my addr 172.21.15.81:0/2020106853 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:11.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.406+0000 7fd247fff640 1 -- 172.21.15.81:0/2020106853 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd250151b50 msgr2=0x7fd250151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.406+0000 7fd247fff640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd250151b50 0x7fd250151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.406+0000 7fd247fff640 1 -- 172.21.15.81:0/2020106853 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd23c0039a0 msgr2=0x7fd25015cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.406+0000 7fd24f7fe640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd23c0039a0 0x7fd25015cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.406+0000 7fd247fff640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd23c0039a0 0x7fd25015cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.406+0000 7fd247fff640 1 -- 172.21.15.81:0/2020106853 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd250158080 con 0x7fd2501590d0 2023-11-08T16:30:11.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.406+0000 7fd247fff640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2501590d0 0x7fd25015d920 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd23800e770 tx=0x7fd23800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:11.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.406+0000 7fd24cff9640 1 -- 172.21.15.81:0/2020106853 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23800c730 con 0x7fd2501590d0 2023-11-08T16:30:11.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.407+0000 7fd24cff9640 1 -- 172.21.15.81:0/2020106853 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23800c890 con 0x7fd2501590d0 2023-11-08T16:30:11.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.407+0000 7fd254ba5640 1 -- 172.21.15.81:0/2020106853 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd250158360 con 0x7fd2501590d0 2023-11-08T16:30:11.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.407+0000 7fd24cff9640 1 -- 172.21.15.81:0/2020106853 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23800cc40 con 0x7fd2501590d0 2023-11-08T16:30:11.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.407+0000 7fd254ba5640 1 -- 172.21.15.81:0/2020106853 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25015d130 con 0x7fd2501590d0 2023-11-08T16:30:11.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.408+0000 7fd254ba5640 1 -- 172.21.15.81:0/2020106853 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25004ece0 con 0x7fd2501590d0 2023-11-08T16:30:11.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.409+0000 7fd24cff9640 1 -- 172.21.15.81:0/2020106853 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd238013470 con 0x7fd2501590d0 2023-11-08T16:30:11.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.410+0000 7fd24cff9640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd23006aec0 0x7fd23006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.410+0000 7fd24cff9640 1 -- 172.21.15.81:0/2020106853 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd238089550 con 0x7fd2501590d0 2023-11-08T16:30:11.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.410+0000 7fd24f7fe640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd23006aec0 0x7fd23006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.411+0000 7fd24f7fe640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd23006aec0 0x7fd23006d380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd240033000 tx=0x7fd240009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:11.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.415+0000 7fd24cff9640 1 -- 172.21.15.81:0/2020106853 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd238054650 con 0x7fd2501590d0 2023-11-08T16:30:11.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.626+0000 7fd254ba5640 1 -- 172.21.15.81:0/2020106853 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd250152d10 con 0x7fd2501590d0 2023-11-08T16:30:11.629 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:30:11.627+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:30:11.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.631+0000 7fd24cff9640 1 -- 172.21.15.81:0/2020106853 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd238057c70 con 0x7fd2501590d0 2023-11-08T16:30:11.632 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:30:11.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.633+0000 7fd254ba5640 1 -- 172.21.15.81:0/2020106853 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd23006aec0 msgr2=0x7fd23006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.633+0000 7fd254ba5640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd23006aec0 0x7fd23006d380 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd240033000 tx=0x7fd240009030 comp rx=0 tx=0).stop 2023-11-08T16:30:11.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.633+0000 7fd254ba5640 1 -- 172.21.15.81:0/2020106853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2501590d0 msgr2=0x7fd25015d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.633+0000 7fd254ba5640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2501590d0 0x7fd25015d920 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fd23800e770 tx=0x7fd23800ec30 comp rx=0 tx=0).stop 2023-11-08T16:30:11.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.634+0000 7fd254ba5640 1 -- 172.21.15.81:0/2020106853 shutdown_connections 2023-11-08T16:30:11.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.634+0000 7fd254ba5640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd23006aec0 0x7fd23006d380 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.634+0000 7fd254ba5640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd250151b50 0x7fd250151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.634+0000 7fd254ba5640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd2501590d0 0x7fd25015d920 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.634+0000 7fd254ba5640 1 --2- 172.21.15.81:0/2020106853 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd23c0039a0 0x7fd25015cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.634+0000 7fd254ba5640 1 -- 172.21.15.81:0/2020106853 >> 172.21.15.81:0/2020106853 conn(0x7fd2500b04d0 msgr2=0x7fd250070f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:11.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.634+0000 7fd254ba5640 1 -- 172.21.15.81:0/2020106853 shutdown_connections 2023-11-08T16:30:11.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.634+0000 7fd254ba5640 1 -- 172.21.15.81:0/2020106853 wait complete. 2023-11-08T16:30:11.654 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T16:30:11.664 INFO:tasks.ceph.mds.d.smithi093.stderr: -3675> 2023-11-08T16:29:08.691+0000 7fc8b50ea640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:30:11.751 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:30:11.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.864+0000 7f74b2500640 1 Processor -- start 2023-11-08T16:30:11.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.864+0000 7f74b2500640 1 -- start start 2023-11-08T16:30:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.866+0000 7f74b2500640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac15e550 0x7f74ac0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.866+0000 7f74b2500640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ac0b0e70 con 0x7f74ac15e170 2023-11-08T16:30:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.866+0000 7f74b2500640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74ac071460 con 0x7f74ac0b0a20 2023-11-08T16:30:11.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.866+0000 7f74b2500640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac0703c0 con 0x7f74ac15e550 2023-11-08T16:30:11.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.866+0000 7f74abfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f74ac15e170 0x7f74ac15ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:37712/0 (socket says 172.21.15.81:37712) 2023-11-08T16:30:11.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.866+0000 7f74abfff640 1 -- 172.21.15.81:0/722533559 learned_addr learned my addr 172.21.15.81:0/722533559 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:11.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74ab7fe640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac15e550 0x7f74ac0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307927491 0 0) 0x7f749c001110 con 0x7f74ac15e170 2023-11-08T16:30:11.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7494003520 con 0x7f74ac15e170 2023-11-08T16:30:11.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1669169779 0 0) 0x7f7494003520 con 0x7f74ac15e170 2023-11-08T16:30:11.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac15e550 msgr2=0x7f74ac0b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:11.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac15e550 0x7f74ac0b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 >> v1:172.21.15.93:6789/0 conn(0x7f74ac0b0a20 legacy=0x7f74ac0ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:11.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac0706a0 con 0x7f74ac15e170 2023-11-08T16:30:11.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f749c001800 con 0x7f74ac15e170 2023-11-08T16:30:11.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 >> v1:172.21.15.81:6789/0 conn(0x7f74ac15e170 legacy=0x7f74ac15ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7494003a10 0x7f7494003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac15e550 0x7f74940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7494004d50 0x7f74940051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac0703c0 con 0x7f7494003a10 2023-11-08T16:30:11.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749400bb60 con 0x7f7494004d50 2023-11-08T16:30:11.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749400bd20 con 0x7f74ac15e550 2023-11-08T16:30:11.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.867+0000 7f74abfff640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7494003a10 0x7f7494003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.868+0000 7f74ab7fe640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac15e550 0x7f74940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.868+0000 7f74b0a9d640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7494004d50 0x7f74940051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.868+0000 7f74abfff640 1 -- 172.21.15.81:0/722533559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac15e550 msgr2=0x7f74940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:11.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.868+0000 7f74abfff640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac15e550 0x7f74940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.868+0000 7f74abfff640 1 -- 172.21.15.81:0/722533559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7494004d50 msgr2=0x7f74940051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:11.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.868+0000 7f74abfff640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7494004d50 0x7f74940051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.868+0000 7f74abfff640 1 -- 172.21.15.81:0/722533559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7494003520 con 0x7f7494003a10 2023-11-08T16:30:11.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.869+0000 7f74abfff640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7494003a10 0x7f7494003e10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c0e0 tx=0x7f74940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9af45f0fd7e11a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:11.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.869+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c005ad0 con 0x7f7494003a10 2023-11-08T16:30:11.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.869+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c005c30 con 0x7f7494003a10 2023-11-08T16:30:11.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.869+0000 7f74aaffd640 1 -- 172.21.15.81:0/722533559 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c02c7c0 con 0x7f7494003a10 2023-11-08T16:30:11.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.870+0000 7f74b2500640 1 -- 172.21.15.81:0/722533559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7494003a10 msgr2=0x7f7494003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.870+0000 7f74b2500640 1 --2- 172.21.15.81:0/722533559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7494003a10 0x7f7494003e10 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f749c00c0e0 tx=0x7f74940057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.870+0000 7f74b0a9d640 1 -- 172.21.15.81:0/722533559 reap_dead start 2023-11-08T16:30:11.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.870+0000 7f74b2500640 1 -- 172.21.15.81:0/722533559 shutdown_connections 2023-11-08T16:30:11.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.870+0000 7f74b2500640 1 -- 172.21.15.81:0/722533559 >> 172.21.15.81:0/722533559 conn(0x7f74ac0cb4e0 msgr2=0x7f74ac0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:11.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.870+0000 7f74b2500640 1 -- 172.21.15.81:0/722533559 shutdown_connections 2023-11-08T16:30:11.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.870+0000 7f74b2500640 1 -- 172.21.15.81:0/722533559 wait complete. 2023-11-08T16:30:11.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.872+0000 7f74b2500640 1 Processor -- start 2023-11-08T16:30:11.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.872+0000 7f74b2500640 1 -- start start 2023-11-08T16:30:11.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.873+0000 7f74b2500640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7494003a10 0x7f74ac147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.873+0000 7f74b2500640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac06b2e0 0x7f74ac141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.873+0000 7f74b2500640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74ac168de0 0x7f74ac167c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.873+0000 7f74b2500640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac158ca0 con 0x7f74ac168de0 2023-11-08T16:30:11.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.873+0000 7f74b2500640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac158f80 con 0x7f7494003a10 2023-11-08T16:30:11.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.873+0000 7f74b2500640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74ac0c3c50 con 0x7f74ac06b2e0 2023-11-08T16:30:11.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.873+0000 7f74b0a9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74ac168de0 0x7f74ac167c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.873+0000 7f74b0a9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74ac168de0 0x7f74ac167c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51216/0 (socket says 172.21.15.81:51216) 2023-11-08T16:30:11.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.873+0000 7f74b0a9d640 1 -- 172.21.15.81:0/2290172211 learned_addr learned my addr 172.21.15.81:0/2290172211 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:11.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.873+0000 7f74abfff640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7494003a10 0x7f74ac147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.873+0000 7f74ab7fe640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac06b2e0 0x7f74ac141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74b0a9d640 1 -- 172.21.15.81:0/2290172211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac06b2e0 msgr2=0x7f74ac141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74b0a9d640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac06b2e0 0x7f74ac141d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74b0a9d640 1 -- 172.21.15.81:0/2290172211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7494003a10 msgr2=0x7f74ac147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:11.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74b0a9d640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7494003a10 0x7f74ac147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:11.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74b0a9d640 1 -- 172.21.15.81:0/2290172211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74ac0c3f30 con 0x7f74ac168de0 2023-11-08T16:30:11.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74abfff640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7494003a10 0x7f74ac147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:11.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74b0a9d640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74ac168de0 0x7f74ac167c40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0ccc30 tx=0x7f74a400ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:11.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74a8ff9640 1 -- 172.21.15.81:0/2290172211 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a4003df0 con 0x7f74ac168de0 2023-11-08T16:30:11.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74a8ff9640 1 -- 172.21.15.81:0/2290172211 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a4013420 con 0x7f74ac168de0 2023-11-08T16:30:11.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74a8ff9640 1 -- 172.21.15.81:0/2290172211 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a40137d0 con 0x7f74ac168de0 2023-11-08T16:30:11.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74b2500640 1 -- 172.21.15.81:0/2290172211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74ac159260 con 0x7f74ac168de0 2023-11-08T16:30:11.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.874+0000 7f74b2500640 1 -- 172.21.15.81:0/2290172211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74ac142450 con 0x7f74ac168de0 2023-11-08T16:30:11.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.875+0000 7f74b2500640 1 -- 172.21.15.81:0/2290172211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74ac0c4150 con 0x7f74ac168de0 2023-11-08T16:30:11.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.876+0000 7f74a8ff9640 1 -- 172.21.15.81:0/2290172211 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f74a4013930 con 0x7f74ac168de0 2023-11-08T16:30:11.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.878+0000 7f74a8ff9640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f749806ae70 0x7f749806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:11.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.878+0000 7f74a8ff9640 1 -- 172.21.15.81:0/2290172211 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f74a40890f0 con 0x7f74ac168de0 2023-11-08T16:30:11.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.878+0000 7f74abfff640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f749806ae70 0x7f749806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:11.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.879+0000 7f74abfff640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f749806ae70 0x7f749806d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f749c034000 tx=0x7f749c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:11.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:11.882+0000 7f74a8ff9640 1 -- 172.21.15.81:0/2290172211 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74a4054010 con 0x7f74ac168de0 2023-11-08T16:30:12.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.098+0000 7f74b2500640 1 -- 172.21.15.81:0/2290172211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f74ac142450 con 0x7f74ac168de0 2023-11-08T16:30:12.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.634+0000 7f74a8ff9640 1 -- 172.21.15.81:0/2290172211 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f74ac142450 con 0x7f74ac168de0 2023-11-08T16:30:12.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.637+0000 7f74b2500640 1 -- 172.21.15.81:0/2290172211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f749806ae70 msgr2=0x7f749806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:12.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.637+0000 7f74b2500640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f749806ae70 0x7f749806d330 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f749c034000 tx=0x7f749c009030 comp rx=0 tx=0).stop 2023-11-08T16:30:12.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.637+0000 7f74b2500640 1 -- 172.21.15.81:0/2290172211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74ac168de0 msgr2=0x7f74ac167c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:12.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.637+0000 7f74b2500640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74ac168de0 0x7f74ac167c40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0ccc30 tx=0x7f74a400ea00 comp rx=0 tx=0).stop 2023-11-08T16:30:12.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.638+0000 7f74b2500640 1 -- 172.21.15.81:0/2290172211 shutdown_connections 2023-11-08T16:30:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.638+0000 7f74b2500640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f749806ae70 0x7f749806d330 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.638+0000 7f74b2500640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74ac168de0 0x7f74ac167c40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.638+0000 7f74b2500640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74ac06b2e0 0x7f74ac141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.638+0000 7f74b2500640 1 --2- 172.21.15.81:0/2290172211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7494003a10 0x7f74ac147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:12.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.638+0000 7f74b2500640 1 -- 172.21.15.81:0/2290172211 >> 172.21.15.81:0/2290172211 conn(0x7f74ac0cb4e0 msgr2=0x7f74ac14bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:12.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.638+0000 7f74b2500640 1 -- 172.21.15.81:0/2290172211 shutdown_connections 2023-11-08T16:30:12.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.638+0000 7f74b2500640 1 -- 172.21.15.81:0/2290172211 wait complete. 2023-11-08T16:30:12.659 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T16:30:12.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.876+0000 7f27b656e640 1 Processor -- start 2023-11-08T16:30:12.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.877+0000 7f27b656e640 1 -- start start 2023-11-08T16:30:12.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.878+0000 7f27b656e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f27b00b0bf0 0x7f27b00ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:12.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.878+0000 7f27b656e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b0071460 con 0x7f27b015e170 2023-11-08T16:30:12.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.878+0000 7f27b656e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b00703c0 con 0x7f27b015e550 2023-11-08T16:30:12.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.878+0000 7f27b656e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b00706a0 con 0x7f27b00b0bf0 2023-11-08T16:30:12.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.878+0000 7f27affff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f27b015e170 0x7f27b015ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:37714/0 (socket says 172.21.15.81:37714) 2023-11-08T16:30:12.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.878+0000 7f27affff640 1 -- 172.21.15.81:0/3806851263 learned_addr learned my addr 172.21.15.81:0/3806851263 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:12.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27b4b0b640 1 --2- 172.21.15.81:0/3806851263 >> v2:172.21.15.93:3300/0 conn(0x7f27b00b0bf0 0x7f27b00ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3850197556 0 0) 0x7f27b0071460 con 0x7f27b015e170 2023-11-08T16:30:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f279c005730 con 0x7f27b015e170 2023-11-08T16:30:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2853615843 0 0) 0x7f279c005730 con 0x7f27b015e170 2023-11-08T16:30:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 >> v2:172.21.15.93:3300/0 conn(0x7f27b00b0bf0 msgr2=0x7f27b00ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 --2- 172.21.15.81:0/3806851263 >> v2:172.21.15.93:3300/0 conn(0x7f27b00b0bf0 0x7f27b00ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 >> v1:172.21.15.93:6789/0 conn(0x7f27b015e550 legacy=0x7f27b00b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0070980 con 0x7f27b015e170 2023-11-08T16:30:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f27a0001670 con 0x7f27b015e170 2023-11-08T16:30:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 >> v1:172.21.15.81:6789/0 conn(0x7f27b015e170 legacy=0x7f27b015ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:12.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 --2- 172.21.15.81:0/3806851263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279c005b80 0x7f279c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:12.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 --2- 172.21.15.81:0/3806851263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f279c0069d0 0x7f279c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:12.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 --2- 172.21.15.81:0/3806851263 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f279c007390 0x7f279c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:12.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00706a0 con 0x7f279c005b80 2023-11-08T16:30:12.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279c00e230 con 0x7f279c0069d0 2023-11-08T16:30:12.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.879+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c00e3f0 con 0x7f279c007390 2023-11-08T16:30:12.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.880+0000 7f27affff640 1 --2- 172.21.15.81:0/3806851263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279c005b80 0x7f279c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:12.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.880+0000 7f27a7fff640 1 --2- 172.21.15.81:0/3806851263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f279c0069d0 0x7f279c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:12.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.880+0000 7f27affff640 1 -- 172.21.15.81:0/3806851263 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f279c007390 msgr2=0x7f279c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:12.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.880+0000 7f27affff640 1 --2- 172.21.15.81:0/3806851263 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f279c007390 0x7f279c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:12.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.880+0000 7f27affff640 1 -- 172.21.15.81:0/3806851263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f279c0069d0 msgr2=0x7f279c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:12.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.880+0000 7f27affff640 1 --2- 172.21.15.81:0/3806851263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f279c0069d0 0x7f279c006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:12.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.880+0000 7f27affff640 1 -- 172.21.15.81:0/3806851263 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c005730 con 0x7f279c005b80 2023-11-08T16:30:12.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.880+0000 7f27b4b0b640 1 -- 172.21.15.81:0/3806851263 reap_dead start 2023-11-08T16:30:12.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.881+0000 7f27affff640 1 --2- 172.21.15.81:0/3806851263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279c005b80 0x7f279c005fa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f27a000c240 tx=0x7f279c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e53b0d937b1e323 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:12.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.881+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a0005ad0 con 0x7f279c005b80 2023-11-08T16:30:12.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.881+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27a0005c30 con 0x7f279c005b80 2023-11-08T16:30:12.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.881+0000 7f27af7fe640 1 -- 172.21.15.81:0/3806851263 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27a002cab0 con 0x7f279c005b80 2023-11-08T16:30:12.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.882+0000 7f27b656e640 1 -- 172.21.15.81:0/3806851263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279c005b80 msgr2=0x7f279c005fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:12.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.882+0000 7f27b656e640 1 --2- 172.21.15.81:0/3806851263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279c005b80 0x7f279c005fa0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f27a000c240 tx=0x7f279c007a00 comp rx=0 tx=0).stop 2023-11-08T16:30:12.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.882+0000 7f27b656e640 1 -- 172.21.15.81:0/3806851263 shutdown_connections 2023-11-08T16:30:12.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.882+0000 7f27b656e640 1 --2- 172.21.15.81:0/3806851263 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279c005b80 0x7f279c005fa0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:12.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.882+0000 7f27b656e640 1 -- 172.21.15.81:0/3806851263 >> 172.21.15.81:0/3806851263 conn(0x7f27b00cb4e0 msgr2=0x7f27b00cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:12.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.882+0000 7f27b656e640 1 -- 172.21.15.81:0/3806851263 shutdown_connections 2023-11-08T16:30:12.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.882+0000 7f27b656e640 1 -- 172.21.15.81:0/3806851263 wait complete. 2023-11-08T16:30:12.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.884+0000 7f27b656e640 1 Processor -- start 2023-11-08T16:30:12.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.884+0000 7f27b656e640 1 -- start start 2023-11-08T16:30:12.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.885+0000 7f27b656e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f279c005b80 0x7f27b01f28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:12.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.885+0000 7f27b656e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27b014f440 0x7f27b014ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:12.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.885+0000 7f27b656e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27b015cc50 0x7f27b015d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:12.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.885+0000 7f27b656e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b01f2e30 con 0x7f27b014f440 2023-11-08T16:30:12.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.885+0000 7f27b656e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b015d610 con 0x7f279c005b80 2023-11-08T16:30:12.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.885+0000 7f27b656e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00c3f80 con 0x7f27b015cc50 2023-11-08T16:30:12.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.885+0000 7f27a7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27b014f440 0x7f27b014ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:12.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.885+0000 7f27a7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27b014f440 0x7f27b014ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51242/0 (socket says 172.21.15.81:51242) 2023-11-08T16:30:12.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.885+0000 7f27a7fff640 1 -- 172.21.15.81:0/784689133 learned_addr learned my addr 172.21.15.81:0/784689133 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:12.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27b4b0b640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27b015cc50 0x7f27b015d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:12.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27affff640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f279c005b80 0x7f27b01f28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:12.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27a7fff640 1 -- 172.21.15.81:0/784689133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27b015cc50 msgr2=0x7f27b015d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:12.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27a7fff640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27b015cc50 0x7f27b015d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:12.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27a7fff640 1 -- 172.21.15.81:0/784689133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f279c005b80 msgr2=0x7f27b01f28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:12.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27a7fff640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f279c005b80 0x7f27b01f28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:12.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27a7fff640 1 -- 172.21.15.81:0/784689133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b014f180 con 0x7f27b014f440 2023-11-08T16:30:12.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27a7fff640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27b014f440 0x7f27b014ec40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f279800d4d0 tx=0x7f279800d990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:12.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27ad7fa640 1 -- 172.21.15.81:0/784689133 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798005480 con 0x7f27b014f440 2023-11-08T16:30:12.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27ad7fa640 1 -- 172.21.15.81:0/784689133 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2798003610 con 0x7f27b014f440 2023-11-08T16:30:12.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27b656e640 1 -- 172.21.15.81:0/784689133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b00c41d0 con 0x7f27b014f440 2023-11-08T16:30:12.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27ad7fa640 1 -- 172.21.15.81:0/784689133 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2798003980 con 0x7f27b014f440 2023-11-08T16:30:12.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.886+0000 7f27b656e640 1 -- 172.21.15.81:0/784689133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b00c4d10 con 0x7f27b014f440 2023-11-08T16:30:12.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.887+0000 7f27b656e640 1 -- 172.21.15.81:0/784689133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b00743f0 con 0x7f27b014f440 2023-11-08T16:30:12.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.888+0000 7f27ad7fa640 1 -- 172.21.15.81:0/784689133 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2798003ae0 con 0x7f27b014f440 2023-11-08T16:30:12.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.890+0000 7f27ad7fa640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f279006aea0 0x7f279006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:12.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.890+0000 7f27ad7fa640 1 -- 172.21.15.81:0/784689133 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f2798089490 con 0x7f27b014f440 2023-11-08T16:30:12.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.890+0000 7f27affff640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f279006aea0 0x7f279006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:12.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.891+0000 7f27affff640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f279006aea0 0x7f279006d360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27a0034000 tx=0x7f27a0009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:12.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:12.894+0000 7f27ad7fa640 1 -- 172.21.15.81:0/784689133 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2798054500 con 0x7f27b014f440 2023-11-08T16:30:13.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:13.072+0000 7f27b656e640 1 -- 172.21.15.81:0/784689133 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f27b00c9950 con 0x7f27b014f440 2023-11-08T16:30:14.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.031+0000 7f27ad7fa640 1 -- 172.21.15.81:0/784689133 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2798057b20 con 0x7f27b014f440 2023-11-08T16:30:14.033 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' removed 2023-11-08T16:30:14.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.035+0000 7f27b656e640 1 -- 172.21.15.81:0/784689133 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f279006aea0 msgr2=0x7f279006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:14.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.035+0000 7f27b656e640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f279006aea0 0x7f279006d360 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f27a0034000 tx=0x7f27a0009030 comp rx=0 tx=0).stop 2023-11-08T16:30:14.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.035+0000 7f27b656e640 1 -- 172.21.15.81:0/784689133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27b014f440 msgr2=0x7f27b014ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:14.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.035+0000 7f27b656e640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27b014f440 0x7f27b014ec40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f279800d4d0 tx=0x7f279800d990 comp rx=0 tx=0).stop 2023-11-08T16:30:14.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.035+0000 7f27b656e640 1 -- 172.21.15.81:0/784689133 shutdown_connections 2023-11-08T16:30:14.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.035+0000 7f27b656e640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f279006aea0 0x7f279006d360 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:14.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.035+0000 7f27b656e640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27b015cc50 0x7f27b015d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:14.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.035+0000 7f27b656e640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27b014f440 0x7f27b014ec40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:14.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.035+0000 7f27b656e640 1 --2- 172.21.15.81:0/784689133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f279c005b80 0x7f27b01f28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:14.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.035+0000 7f27b656e640 1 -- 172.21.15.81:0/784689133 >> 172.21.15.81:0/784689133 conn(0x7f27b00cb4e0 msgr2=0x7f27b01518e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:14.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.035+0000 7f27b656e640 1 -- 172.21.15.81:0/784689133 shutdown_connections 2023-11-08T16:30:14.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.036+0000 7f27b656e640 1 -- 172.21.15.81:0/784689133 wait complete. 2023-11-08T16:30:14.058 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T16:30:14.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.274+0000 7f4956768640 1 Processor -- start 2023-11-08T16:30:14.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.274+0000 7f4956768640 1 -- start start 2023-11-08T16:30:14.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.276+0000 7f4956768640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f495015afd0 0x7f495015f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:14.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.276+0000 7f4956768640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49500b09f0 con 0x7f4950071890 2023-11-08T16:30:14.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.276+0000 7f4956768640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49500ca360 con 0x7f49500714b0 2023-11-08T16:30:14.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.276+0000 7f4956768640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f495006b250 con 0x7f495015afd0 2023-11-08T16:30:14.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.276+0000 7f4954d05640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f495015afd0 0x7f495015f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:14.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.276+0000 7f49477fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4950071890 0x7f495015e9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:37730/0 (socket says 172.21.15.81:37730) 2023-11-08T16:30:14.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.276+0000 7f49477fe640 1 -- 172.21.15.81:0/3676975944 learned_addr learned my addr 172.21.15.81:0/3676975944 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:14.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.276+0000 7f4954d05640 1 -- 172.21.15.81:0/3676975944 >> v1:172.21.15.93:6789/0 conn(0x7f49500714b0 legacy=0x7f495015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:14.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.277+0000 7f4954d05640 1 -- 172.21.15.81:0/3676975944 >> v1:172.21.15.81:6789/0 conn(0x7f4950071890 legacy=0x7f495015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:14.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.277+0000 7f4954d05640 1 -- 172.21.15.81:0/3676975944 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49500ca640 con 0x7f495015afd0 2023-11-08T16:30:14.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.277+0000 7f494f7fe640 1 -- 172.21.15.81:0/3676975944 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592250221 0 0) 0x7f49500b09f0 con 0x7f4950071890 2023-11-08T16:30:14.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.278+0000 7f4954d05640 1 --2- 172.21.15.81:0/3676975944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f495015afd0 0x7f495015f0e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f49480097a0 tx=0x7f494802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5241cbf174c579 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:14.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.278+0000 7f494f7fe640 1 -- 172.21.15.81:0/3676975944 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948031830 con 0x7f495015afd0 2023-11-08T16:30:14.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.278+0000 7f494f7fe640 1 -- 172.21.15.81:0/3676975944 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4948031990 con 0x7f495015afd0 2023-11-08T16:30:14.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.278+0000 7f494f7fe640 1 -- 172.21.15.81:0/3676975944 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948031d20 con 0x7f495015afd0 2023-11-08T16:30:14.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.278+0000 7f4956768640 1 -- 172.21.15.81:0/3676975944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f495015afd0 msgr2=0x7f495015f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:14.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.278+0000 7f4956768640 1 --2- 172.21.15.81:0/3676975944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f495015afd0 0x7f495015f0e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f49480097a0 tx=0x7f494802b8d0 comp rx=0 tx=0).stop 2023-11-08T16:30:14.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.279+0000 7f4956768640 1 -- 172.21.15.81:0/3676975944 shutdown_connections 2023-11-08T16:30:14.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.279+0000 7f4956768640 1 --2- 172.21.15.81:0/3676975944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f495015afd0 0x7f495015f0e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:14.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.279+0000 7f4956768640 1 -- 172.21.15.81:0/3676975944 >> 172.21.15.81:0/3676975944 conn(0x7f49500cb4e0 msgr2=0x7f49500cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:14.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.279+0000 7f4956768640 1 -- 172.21.15.81:0/3676975944 shutdown_connections 2023-11-08T16:30:14.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.279+0000 7f4956768640 1 -- 172.21.15.81:0/3676975944 wait complete. 2023-11-08T16:30:14.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.280+0000 7f4956768640 1 Processor -- start 2023-11-08T16:30:14.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.280+0000 7f4956768640 1 -- start start 2023-11-08T16:30:14.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.281+0000 7f4956768640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49500714b0 0x7f49501527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:14.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.281+0000 7f4956768640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4950071890 0x7f4950147f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:14.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.281+0000 7f4956768640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f495015afd0 0x7f4950153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:14.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.281+0000 7f4956768640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49500c6c20 con 0x7f49500714b0 2023-11-08T16:30:14.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.281+0000 7f4956768640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49500c6010 con 0x7f495015afd0 2023-11-08T16:30:14.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.281+0000 7f4956768640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49500c62f0 con 0x7f4950071890 2023-11-08T16:30:14.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.282+0000 7f494ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49500714b0 0x7f49501527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:14.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.282+0000 7f494ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49500714b0 0x7f49501527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51252/0 (socket says 172.21.15.81:51252) 2023-11-08T16:30:14.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.282+0000 7f494ffff640 1 -- 172.21.15.81:0/3441751616 learned_addr learned my addr 172.21.15.81:0/3441751616 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:14.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.282+0000 7f494ffff640 1 -- 172.21.15.81:0/3441751616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4950071890 msgr2=0x7f4950147f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:14.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.282+0000 7f4954d05640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f495015afd0 0x7f4950153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:14.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.282+0000 7f494ffff640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4950071890 0x7f4950147f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:14.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.282+0000 7f494ffff640 1 -- 172.21.15.81:0/3441751616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f495015afd0 msgr2=0x7f4950153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:14.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.282+0000 7f494ffff640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f495015afd0 0x7f4950153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:14.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.282+0000 7f494ffff640 1 -- 172.21.15.81:0/3441751616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4948009400 con 0x7f49500714b0 2023-11-08T16:30:14.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.282+0000 7f494ffff640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49500714b0 0x7f49501527e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f494000c7c0 tx=0x7f494000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:14.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.283+0000 7f494d7fa640 1 -- 172.21.15.81:0/3441751616 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494000aa50 con 0x7f49500714b0 2023-11-08T16:30:14.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.283+0000 7f494d7fa640 1 -- 172.21.15.81:0/3441751616 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494000abb0 con 0x7f49500714b0 2023-11-08T16:30:14.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.283+0000 7f494d7fa640 1 -- 172.21.15.81:0/3441751616 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4940012420 con 0x7f49500714b0 2023-11-08T16:30:14.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.283+0000 7f4956768640 1 -- 172.21.15.81:0/3441751616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49500c65d0 con 0x7f49500714b0 2023-11-08T16:30:14.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.283+0000 7f4956768640 1 -- 172.21.15.81:0/3441751616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4950152460 con 0x7f49500714b0 2023-11-08T16:30:14.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.284+0000 7f4956768640 1 -- 172.21.15.81:0/3441751616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49500b0e60 con 0x7f49500714b0 2023-11-08T16:30:14.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.285+0000 7f494d7fa640 1 -- 172.21.15.81:0/3441751616 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4940012580 con 0x7f49500714b0 2023-11-08T16:30:14.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.286+0000 7f494d7fa640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f493006ae70 0x7f493006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:14.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.286+0000 7f494d7fa640 1 -- 172.21.15.81:0/3441751616 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f4940089820 con 0x7f49500714b0 2023-11-08T16:30:14.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.286+0000 7f49477fe640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f493006ae70 0x7f493006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:14.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.287+0000 7f49477fe640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f493006ae70 0x7f493006d330 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49380011e0 tx=0x7f4938007940 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:14.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.290+0000 7f494d7fa640 1 -- 172.21.15.81:0/3441751616 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f49500b0e60 con 0x7f49500714b0 2023-11-08T16:30:14.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:14.475+0000 7f4956768640 1 -- 172.21.15.81:0/3441751616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f49500b0e60 con 0x7f49500714b0 2023-11-08T16:30:15.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.041+0000 7f494d7fa640 1 -- 172.21.15.81:0/3441751616 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f49500b0e60 con 0x7f49500714b0 2023-11-08T16:30:15.044 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' removed 2023-11-08T16:30:15.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 -- 172.21.15.81:0/3441751616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f493006ae70 msgr2=0x7f493006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:15.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f493006ae70 0x7f493006d330 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49380011e0 tx=0x7f4938007940 comp rx=0 tx=0).stop 2023-11-08T16:30:15.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 -- 172.21.15.81:0/3441751616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49500714b0 msgr2=0x7f49501527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:15.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49500714b0 0x7f49501527e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f494000c7c0 tx=0x7f494000cc80 comp rx=0 tx=0).stop 2023-11-08T16:30:15.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 -- 172.21.15.81:0/3441751616 shutdown_connections 2023-11-08T16:30:15.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f493006ae70 0x7f493006d330 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:15.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f495015afd0 0x7f4950153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:15.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4950071890 0x7f4950147f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:15.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 --2- 172.21.15.81:0/3441751616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49500714b0 0x7f49501527e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:15.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 -- 172.21.15.81:0/3441751616 >> 172.21.15.81:0/3441751616 conn(0x7f49500cb4e0 msgr2=0x7f4950159780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:15.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 -- 172.21.15.81:0/3441751616 shutdown_connections 2023-11-08T16:30:15.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:15.045+0000 7f4956768640 1 -- 172.21.15.81:0/3441751616 wait complete. 2023-11-08T16:30:15.069 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T16:30:15.069 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T16:30:15.069 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T16:30:15.070 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:30:15.070 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T16:30:15.070 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T16:30:15.071 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T16:30:15.071 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:30:15.072 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T16:30:15.072 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T16:30:15.072 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T16:30:15.072 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:30:15.073 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T16:30:15.073 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T16:30:15.073 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T16:30:15.074 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:30:15.089 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T16:30:15.089 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T16:30:15.094 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T16:30:15.095 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T16:30:15.096 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T16:30:15.096 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T16:30:15.141 INFO:tasks.ceph.mds.a:Started 2023-11-08T16:30:15.142 DEBUG:teuthology.parallel:result is None 2023-11-08T16:30:15.147 INFO:tasks.ceph.mds.d:Started 2023-11-08T16:30:15.148 DEBUG:teuthology.parallel:result is None 2023-11-08T16:30:15.157 INFO:tasks.ceph.mds.b:Started 2023-11-08T16:30:15.157 DEBUG:teuthology.parallel:result is None 2023-11-08T16:30:15.304 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:30:15.307 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2023-11-08T16:30:15.323 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:30:16.207 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:30:16.217 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:30:17.484 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-08T16:30:21.092 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T16:30:21.092 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T16:30:21.103 INFO:tasks.ceph.mds.c:Started 2023-11-08T16:30:21.103 DEBUG:teuthology.parallel:result is None 2023-11-08T16:30:21.104 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T16:30:21.270 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:30:21.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.294+0000 7f2dd016c640 1 Processor -- start 2023-11-08T16:30:21.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.294+0000 7f2dd016c640 1 -- start start 2023-11-08T16:30:21.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.295+0000 7f2dd016c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2dc80cf720 0x7f2dc806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:21.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.295+0000 7f2dd016c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc80ca130 0x7f2dc8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:21.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.295+0000 7f2dd016c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc806cc50 con 0x7f2dc815d3b0 2023-11-08T16:30:21.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.295+0000 7f2dd016c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc80c90f0 con 0x7f2dc80cf720 2023-11-08T16:30:21.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.295+0000 7f2dd016c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80adb70 con 0x7f2dc80ca130 2023-11-08T16:30:21.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dcdf08640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2dc815d3b0 0x7f2dc80c9a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44092/0 (socket says 172.21.15.81:44092) 2023-11-08T16:30:21.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dcdf08640 1 -- 172.21.15.81:0/1818465680 learned_addr learned my addr 172.21.15.81:0/1818465680 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:21.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645941228 0 0) 0x7f2dc806cc50 con 0x7f2dc815d3b0 2023-11-08T16:30:21.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dce709640 1 --2- 172.21.15.81:0/1818465680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc80ca130 0x7f2dc8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:21.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dcd707640 1 --2- 172.21.15.81:0/1818465680 >> v2:172.21.15.93:3300/0 conn(0x7f2dc80cf720 0x7f2dc806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:21.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2db4003520 con 0x7f2dc815d3b0 2023-11-08T16:30:21.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3336132504 0 0) 0x7f2db4003520 con 0x7f2dc815d3b0 2023-11-08T16:30:21.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc80ca130 msgr2=0x7f2dc8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:21.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dccf06640 1 --2- 172.21.15.81:0/1818465680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc80ca130 0x7f2dc8070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:21.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 >> v2:172.21.15.93:3300/0 conn(0x7f2dc80cf720 msgr2=0x7f2dc806d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:21.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dccf06640 1 --2- 172.21.15.81:0/1818465680 >> v2:172.21.15.93:3300/0 conn(0x7f2dc80cf720 0x7f2dc806d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:21.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc80c9360 con 0x7f2dc815d3b0 2023-11-08T16:30:21.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f2dc40017f0 con 0x7f2dc815d3b0 2023-11-08T16:30:21.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 >> v1:172.21.15.81:6789/0 conn(0x7f2dc815d3b0 legacy=0x7f2dc80c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:21.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.296+0000 7f2dccf06640 1 --2- 172.21.15.81:0/1818465680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db4003970 0x7f2db4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:21.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dccf06640 1 --2- 172.21.15.81:0/1818465680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc80ca130 0x7f2db40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:21.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dccf06640 1 --2- 172.21.15.81:0/1818465680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2db4004d00 0x7f2db4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:21.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80c90f0 con 0x7f2db4003970 2023-11-08T16:30:21.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc80adb70 con 0x7f2db4004d00 2023-11-08T16:30:21.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2db400ba30 con 0x7f2dc80ca130 2023-11-08T16:30:21.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dcdf08640 1 --2- 172.21.15.81:0/1818465680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db4003970 0x7f2db4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:21.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dcd707640 1 --2- 172.21.15.81:0/1818465680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc80ca130 0x7f2db40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:21.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dcdf08640 1 -- 172.21.15.81:0/1818465680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc80ca130 msgr2=0x7f2db40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:21.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dcdf08640 1 --2- 172.21.15.81:0/1818465680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc80ca130 0x7f2db40047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:21.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dcdf08640 1 -- 172.21.15.81:0/1818465680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2db4004d00 msgr2=0x7f2db4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:21.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dcdf08640 1 --2- 172.21.15.81:0/1818465680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2db4004d00 0x7f2db4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:21.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.297+0000 7f2dcdf08640 1 -- 172.21.15.81:0/1818465680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db4003520 con 0x7f2db4003970 2023-11-08T16:30:21.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.298+0000 7f2dcdf08640 1 --2- 172.21.15.81:0/1818465680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db4003970 0x7f2db4003d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400c110 tx=0x7f2db40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=675541f1ff659831 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:21.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.298+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4007270 con 0x7f2db4003970 2023-11-08T16:30:21.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.298+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc40073f0 con 0x7f2db4003970 2023-11-08T16:30:21.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.298+0000 7f2dccf06640 1 -- 172.21.15.81:0/1818465680 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc402c830 con 0x7f2db4003970 2023-11-08T16:30:21.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.299+0000 7f2dd016c640 1 -- 172.21.15.81:0/1818465680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db4003970 msgr2=0x7f2db4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:21.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.299+0000 7f2dd016c640 1 --2- 172.21.15.81:0/1818465680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2db4003970 0x7f2db4003d90 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400c110 tx=0x7f2db40057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:21.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.299+0000 7f2dce709640 1 -- 172.21.15.81:0/1818465680 reap_dead start 2023-11-08T16:30:21.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.299+0000 7f2dd016c640 1 -- 172.21.15.81:0/1818465680 shutdown_connections 2023-11-08T16:30:21.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.299+0000 7f2dd016c640 1 -- 172.21.15.81:0/1818465680 >> 172.21.15.81:0/1818465680 conn(0x7f2dc80ceaa0 msgr2=0x7f2dc80cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:21.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.299+0000 7f2dd016c640 1 -- 172.21.15.81:0/1818465680 shutdown_connections 2023-11-08T16:30:21.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.299+0000 7f2dd016c640 1 -- 172.21.15.81:0/1818465680 wait complete. 2023-11-08T16:30:21.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.301+0000 7f2dd016c640 1 Processor -- start 2023-11-08T16:30:21.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.301+0000 7f2dd016c640 1 -- start start 2023-11-08T16:30:21.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.302+0000 7f2dd016c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2db4003970 0x7f2dc8154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.302+0000 7f2dd016c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc814f520 0x7f2dc80c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.302+0000 7f2dd016c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc815c2a0 0x7f2dc815c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.302+0000 7f2dd016c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc815cc90 con 0x7f2dc815c2a0 2023-11-08T16:30:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.302+0000 7f2dd016c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc8154ed0 con 0x7f2db4003970 2023-11-08T16:30:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.302+0000 7f2dd016c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc8155760 con 0x7f2dc814f520 2023-11-08T16:30:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.302+0000 7f2dce709640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc815c2a0 0x7f2dc815c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.302+0000 7f2dce709640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc815c2a0 0x7f2dc815c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46652/0 (socket says 172.21.15.81:46652) 2023-11-08T16:30:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.302+0000 7f2dce709640 1 -- 172.21.15.81:0/1605830152 learned_addr learned my addr 172.21.15.81:0/1605830152 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.303+0000 7f2dcdf08640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2db4003970 0x7f2dc8154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.303+0000 7f2dcd707640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc814f520 0x7f2dc80c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.303+0000 7f2dce709640 1 -- 172.21.15.81:0/1605830152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc814f520 msgr2=0x7f2dc80c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.303+0000 7f2dce709640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc814f520 0x7f2dc80c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.303+0000 7f2dce709640 1 -- 172.21.15.81:0/1605830152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2db4003970 msgr2=0x7f2dc8154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.303+0000 7f2dce709640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2db4003970 0x7f2dc8154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.303+0000 7f2dce709640 1 -- 172.21.15.81:0/1605830152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc8155a40 con 0x7f2dc815c2a0 2023-11-08T16:30:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.303+0000 7f2dce709640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc815c2a0 0x7f2dc815c750 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2dc806bb10 tx=0x7f2dc000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.303+0000 7f2dbe7fc640 1 -- 172.21.15.81:0/1605830152 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0018070 con 0x7f2dc815c2a0 2023-11-08T16:30:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.303+0000 7f2dbe7fc640 1 -- 172.21.15.81:0/1605830152 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc000eb60 con 0x7f2dc815c2a0 2023-11-08T16:30:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.304+0000 7f2dd016c640 1 -- 172.21.15.81:0/1605830152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc8153eb0 con 0x7f2dc815c2a0 2023-11-08T16:30:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.304+0000 7f2dbe7fc640 1 -- 172.21.15.81:0/1605830152 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0003990 con 0x7f2dc815c2a0 2023-11-08T16:30:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.304+0000 7f2dd016c640 1 -- 172.21.15.81:0/1605830152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc8154470 con 0x7f2dc815c2a0 2023-11-08T16:30:21.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.305+0000 7f2dd016c640 1 -- 172.21.15.81:0/1605830152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc80743f0 con 0x7f2dc815c2a0 2023-11-08T16:30:21.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.306+0000 7f2dbe7fc640 1 -- 172.21.15.81:0/1605830152 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2dc0014420 con 0x7f2dc815c2a0 2023-11-08T16:30:21.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.307+0000 7f2dbe7fc640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dac06aea0 0x7f2dac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:21.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.307+0000 7f2dbe7fc640 1 -- 172.21.15.81:0/1605830152 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2dc00891b0 con 0x7f2dc815c2a0 2023-11-08T16:30:21.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.307+0000 7f2dcdf08640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dac06aea0 0x7f2dac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:21.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.308+0000 7f2dcdf08640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dac06aea0 0x7f2dac06d360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4034000 tx=0x7f2dc4007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:21.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.312+0000 7f2dbe7fc640 1 -- 172.21.15.81:0/1605830152 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2dc0054550 con 0x7f2dc815c2a0 2023-11-08T16:30:21.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:21.514+0000 7f2dd016c640 1 -- 172.21.15.81:0/1605830152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2dc80cd7c0 con 0x7f2dc815c2a0 2023-11-08T16:30:22.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.632+0000 7f2dbe7fc640 1 -- 172.21.15.81:0/1605830152 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2dc00097a0 con 0x7f2dc815c2a0 2023-11-08T16:30:22.635 INFO:teuthology.orchestra.run.smithi081.stderr: removed all blocklist entries 2023-11-08T16:30:22.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.636+0000 7f2dd016c640 1 -- 172.21.15.81:0/1605830152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dac06aea0 msgr2=0x7f2dac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:22.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.636+0000 7f2dd016c640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dac06aea0 0x7f2dac06d360 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4034000 tx=0x7f2dc4007000 comp rx=0 tx=0).stop 2023-11-08T16:30:22.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.636+0000 7f2dd016c640 1 -- 172.21.15.81:0/1605830152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc815c2a0 msgr2=0x7f2dc815c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:22.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.636+0000 7f2dd016c640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc815c2a0 0x7f2dc815c750 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2dc806bb10 tx=0x7f2dc000e4b0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.637+0000 7f2dd016c640 1 -- 172.21.15.81:0/1605830152 shutdown_connections 2023-11-08T16:30:22.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.637+0000 7f2dd016c640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2dac06aea0 0x7f2dac06d360 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.637+0000 7f2dd016c640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2dc815c2a0 0x7f2dc815c750 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.637+0000 7f2dd016c640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2dc814f520 0x7f2dc80c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.637+0000 7f2dd016c640 1 --2- 172.21.15.81:0/1605830152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2db4003970 0x7f2dc8154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.637+0000 7f2dd016c640 1 -- 172.21.15.81:0/1605830152 >> 172.21.15.81:0/1605830152 conn(0x7f2dc80ceaa0 msgr2=0x7f2dc806c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:22.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.637+0000 7f2dd016c640 1 -- 172.21.15.81:0/1605830152 shutdown_connections 2023-11-08T16:30:22.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.637+0000 7f2dd016c640 1 -- 172.21.15.81:0/1605830152 wait complete. 2023-11-08T16:30:22.659 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T16:30:22.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.871+0000 7f87dea76640 1 Processor -- start 2023-11-08T16:30:22.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.872+0000 7f87dea76640 1 -- start start 2023-11-08T16:30:22.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.873+0000 7f87dea76640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f87d80d01d0 0x7f87d8167f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:22.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.873+0000 7f87dea76640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d80b6100 con 0x7f87d815d310 2023-11-08T16:30:22.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.873+0000 7f87dea76640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d80d1b50 con 0x7f87d80d2b70 2023-11-08T16:30:22.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.873+0000 7f87dea76640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87d80b6480 con 0x7f87d80d01d0 2023-11-08T16:30:22.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.873+0000 7f87dc812640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f87d815d310 0x7f87d80d1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44096/0 (socket says 172.21.15.81:44096) 2023-11-08T16:30:22.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.873+0000 7f87dc812640 1 -- 172.21.15.81:0/1594812886 learned_addr learned my addr 172.21.15.81:0/1594812886 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:22.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 563218308 0 0) 0x7f87d80b6100 con 0x7f87d815d310 2023-11-08T16:30:22.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87c4003520 con 0x7f87d815d310 2023-11-08T16:30:22.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87dd013640 1 --2- 172.21.15.81:0/1594812886 >> v2:172.21.15.93:3300/0 conn(0x7f87d80d01d0 0x7f87d8167f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:22.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 855276335 0 0) 0x7f87c4003520 con 0x7f87d815d310 2023-11-08T16:30:22.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 >> v2:172.21.15.93:3300/0 conn(0x7f87d80d01d0 msgr2=0x7f87d8167f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:22.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 --2- 172.21.15.81:0/1594812886 >> v2:172.21.15.93:3300/0 conn(0x7f87d80d01d0 0x7f87d8167f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 >> v1:172.21.15.93:6789/0 conn(0x7f87d80d2b70 legacy=0x7f87d80cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:22.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d80d1e30 con 0x7f87d815d310 2023-11-08T16:30:22.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f87c80019f0 con 0x7f87d815d310 2023-11-08T16:30:22.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 >> v1:172.21.15.81:6789/0 conn(0x7f87d815d310 legacy=0x7f87d80d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:22.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c4003970 0x7f87c4003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:22.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87c40047c0 0x7f87c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:22.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c4005180 0x7f87c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:22.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80b6480 con 0x7f87c4003970 2023-11-08T16:30:22.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c400c090 con 0x7f87c40047c0 2023-11-08T16:30:22.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.874+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c400c250 con 0x7f87c4005180 2023-11-08T16:30:22.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.875+0000 7f87dc812640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c4003970 0x7f87c4003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:22.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.875+0000 7f87d7fff640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87c40047c0 0x7f87c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:22.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.875+0000 7f87dd013640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c4005180 0x7f87c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:22.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.875+0000 7f87dc812640 1 -- 172.21.15.81:0/1594812886 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c4005180 msgr2=0x7f87c400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:22.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.875+0000 7f87dc812640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c4005180 0x7f87c400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.875+0000 7f87dc812640 1 -- 172.21.15.81:0/1594812886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87c40047c0 msgr2=0x7f87c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:22.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.875+0000 7f87dc812640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87c40047c0 0x7f87c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.875+0000 7f87dc812640 1 -- 172.21.15.81:0/1594812886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c4003520 con 0x7f87c4003970 2023-11-08T16:30:22.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.875+0000 7f87d7fff640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87c40047c0 0x7f87c4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:30:22.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.875+0000 7f87dd013640 1 -- 172.21.15.81:0/1594812886 reap_dead start 2023-11-08T16:30:22.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.877+0000 7f87dc812640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c4003970 0x7f87c4003d70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f87c800a830 tx=0x7f87c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551983f427e75e6b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:22.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.877+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8034630 con 0x7f87c4003970 2023-11-08T16:30:22.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.877+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c802f070 con 0x7f87c4003970 2023-11-08T16:30:22.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.877+0000 7f87d77fe640 1 -- 172.21.15.81:0/1594812886 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c80349d0 con 0x7f87c4003970 2023-11-08T16:30:22.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.877+0000 7f87dea76640 1 -- 172.21.15.81:0/1594812886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c4003970 msgr2=0x7f87c4003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:22.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.877+0000 7f87dea76640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c4003970 0x7f87c4003d70 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f87c800a830 tx=0x7f87c40057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.877+0000 7f87dea76640 1 -- 172.21.15.81:0/1594812886 shutdown_connections 2023-11-08T16:30:22.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.878+0000 7f87dea76640 1 --2- 172.21.15.81:0/1594812886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87c4003970 0x7f87c4003d70 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.878+0000 7f87dea76640 1 -- 172.21.15.81:0/1594812886 >> 172.21.15.81:0/1594812886 conn(0x7f87d806aac0 msgr2=0x7f87d806aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:22.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.878+0000 7f87dea76640 1 -- 172.21.15.81:0/1594812886 shutdown_connections 2023-11-08T16:30:22.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.878+0000 7f87dea76640 1 -- 172.21.15.81:0/1594812886 wait complete. 2023-11-08T16:30:22.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.879+0000 7f87dea76640 1 Processor -- start 2023-11-08T16:30:22.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.880+0000 7f87dea76640 1 -- start start 2023-11-08T16:30:22.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87dea76640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c4003970 0x7f87d815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:22.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87dea76640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87d80ca280 0x7f87d80ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:22.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87dea76640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87d8152d20 0x7f87d81531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:22.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87dea76640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d8153710 con 0x7f87d80ca280 2023-11-08T16:30:22.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87dea76640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d80cac40 con 0x7f87d8152d20 2023-11-08T16:30:22.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87dea76640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d815ea30 con 0x7f87c4003970 2023-11-08T16:30:22.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87d7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87d80ca280 0x7f87d80ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:22.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87d7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87d80ca280 0x7f87d80ca700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46670/0 (socket says 172.21.15.81:46670) 2023-11-08T16:30:22.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87d7fff640 1 -- 172.21.15.81:0/1295167050 learned_addr learned my addr 172.21.15.81:0/1295167050 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:22.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87dc812640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c4003970 0x7f87d815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:22.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87d7fff640 1 -- 172.21.15.81:0/1295167050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c4003970 msgr2=0x7f87d815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:22.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87d7fff640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c4003970 0x7f87d815f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87d7fff640 1 -- 172.21.15.81:0/1295167050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87d8152d20 msgr2=0x7f87d81531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:22.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87d7fff640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87d8152d20 0x7f87d81531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:22.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.881+0000 7f87d7fff640 1 -- 172.21.15.81:0/1295167050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8153900 con 0x7f87d80ca280 2023-11-08T16:30:22.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.882+0000 7f87d7fff640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87d80ca280 0x7f87d80ca700 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f87c000e440 tx=0x7f87c000e840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:22.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.882+0000 7f87d57fa640 1 -- 172.21.15.81:0/1295167050 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c0018070 con 0x7f87d80ca280 2023-11-08T16:30:22.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.882+0000 7f87d57fa640 1 -- 172.21.15.81:0/1295167050 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c0007020 con 0x7f87d80ca280 2023-11-08T16:30:22.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.882+0000 7f87d57fa640 1 -- 172.21.15.81:0/1295167050 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c00071b0 con 0x7f87d80ca280 2023-11-08T16:30:22.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.882+0000 7f87dea76640 1 -- 172.21.15.81:0/1295167050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d8153be0 con 0x7f87d80ca280 2023-11-08T16:30:22.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.882+0000 7f87dea76640 1 -- 172.21.15.81:0/1295167050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d81541a0 con 0x7f87d80ca280 2023-11-08T16:30:22.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.883+0000 7f87dea76640 1 -- 172.21.15.81:0/1295167050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d80743f0 con 0x7f87d80ca280 2023-11-08T16:30:22.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.884+0000 7f87d57fa640 1 -- 172.21.15.81:0/1295167050 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f87c0007310 con 0x7f87d80ca280 2023-11-08T16:30:22.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.885+0000 7f87d57fa640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87b806ae70 0x7f87b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:22.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.885+0000 7f87d57fa640 1 -- 172.21.15.81:0/1295167050 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f87c0088d40 con 0x7f87d80ca280 2023-11-08T16:30:22.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.885+0000 7f87dc812640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87b806ae70 0x7f87b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:22.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.886+0000 7f87dc812640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87b806ae70 0x7f87b806d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f87c8034000 tx=0x7f87c8036040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:22.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:22.890+0000 7f87d57fa640 1 -- 172.21.15.81:0/1295167050 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87c0054130 con 0x7f87d80ca280 2023-11-08T16:30:23.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.146+0000 7f87dea76640 1 -- 172.21.15.81:0/1295167050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f87d80cde50 con 0x7f87d80ca280 2023-11-08T16:30:23.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.147+0000 7f87d57fa640 1 -- 172.21.15.81:0/1295167050 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f87c0057750 con 0x7f87d80ca280 2023-11-08T16:30:23.150 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:23.150 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:30:23.150 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_dump": [ 2023-11-08T16:30:23.151 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.151 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.a", 2023-11-08T16:30:23.151 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlL0bmNxAAlrzRL4depZdd+lBB5HN09g==", 2023-11-08T16:30:23.153 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.153 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:30:23.153 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:30:23.155 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:30:23.155 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.155 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.157 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.157 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.157 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.b", 2023-11-08T16:30:23.157 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tl2cFWHhAAvwSSL9e5996l9MBwNQzs6A==", 2023-11-08T16:30:23.158 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.158 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:30:23.158 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:30:23.158 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:30:23.158 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.159 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.159 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.159 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.159 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.c", 2023-11-08T16:30:23.159 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tlwo23DBAAl1sBjOJMIQRb1fH3o9XoiQ==", 2023-11-08T16:30:23.160 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.160 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:30:23.160 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:30:23.160 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:30:23.160 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.161 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.161 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.161 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.161 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.d", 2023-11-08T16:30:23.161 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tl5DdQLxAA1aU7e9epGtKnZ6hkHZTnsQ==", 2023-11-08T16:30:23.162 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.162 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:30:23.162 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:30:23.162 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:30:23.162 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.163 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.163 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.163 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.163 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.0", 2023-11-08T16:30:23.163 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQALt0tl3kkwKRAAgDwD1eQYk7qjvV+/gyol/w==", 2023-11-08T16:30:23.164 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.164 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:30:23.164 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:30:23.164 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.164 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.165 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.165 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.165 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.1", 2023-11-08T16:30:23.165 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAPt0tlpIh+BRAAUukXDBbd9Vev1TzIrv9sRg==", 2023-11-08T16:30:23.165 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.165 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:30:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:30:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.166 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.2", 2023-11-08T16:30:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQASt0tlmzJfDhAASO7zeg/9ddmLsoOTe+MIug==", 2023-11-08T16:30:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:30:23.167 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:30:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.168 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.3", 2023-11-08T16:30:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAVt0tlGoQSGRAAQsrZhrx4kpYUm3KEb6ZUGQ==", 2023-11-08T16:30:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:30:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:30:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.169 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.4", 2023-11-08T16:30:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAct0tlOS/kIxAAxC2I47B2WeGTFZxvdnmkMg==", 2023-11-08T16:30:23.170 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:30:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:30:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.171 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.5", 2023-11-08T16:30:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAgt0tlnAgEBRAADIRSyu/bxiykCc2uiIt0kA==", 2023-11-08T16:30:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:30:23.172 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:30:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.173 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.6", 2023-11-08T16:30:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAjt0tlrdfZHBAAHGdvBBzVwXMmy8GmsZLNbg==", 2023-11-08T16:30:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:30:23.174 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:30:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.7", 2023-11-08T16:30:23.175 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAmt0tl4nN9NRAAzV9abuBfhybYCsSFnqHEYA==", 2023-11-08T16:30:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:30:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:30:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.176 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.0", 2023-11-08T16:30:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ==", 2023-11-08T16:30:23.177 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:30:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:30:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:30:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:30:23.178 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.1", 2023-11-08T16:30:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlgn/qGBAA+WHDN1CJ1v62EbX1GEFBWA==", 2023-11-08T16:30:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.179 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:30:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:30:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:30:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:30:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.2", 2023-11-08T16:30:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlbQmgDRAAL/b9NwCBVT+uGxuv3VAWRw==", 2023-11-08T16:30:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.181 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:30:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:30:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:30:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:30:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.182 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.3", 2023-11-08T16:30:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlah+ZIxAAg53qCmUjPtG+l++2Byn1NA==", 2023-11-08T16:30:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:30:23.183 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:30:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:30:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:30:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.184 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.admin", 2023-11-08T16:30:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAEt0tlNjJ+KRAATvWBm8Guj09sQQB2U1Afzw==", 2023-11-08T16:30:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:30:23.185 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:30:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:30:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.186 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-mds", 2023-11-08T16:30:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQA1t0tlbjUzARAA19fzd9eXPB9S3gRI5NKEcA==", 2023-11-08T16:30:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-mds" 2023-11-08T16:30:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.187 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-mgr", 2023-11-08T16:30:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQA1t0tltlAzARAATHJqioyA2hnaXk2yFILzaQ==", 2023-11-08T16:30:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.188 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-08T16:30:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-osd", 2023-11-08T16:30:23.189 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQA1t0tlVGgzARAARoAIydVBNv9u+LnnHSEtQg==", 2023-11-08T16:30:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-osd" 2023-11-08T16:30:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.190 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-rbd", 2023-11-08T16:30:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQA1t0tl8n8zARAAIxOSTpyYWV1rMqyprmZDwA==", 2023-11-08T16:30:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-08T16:30:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.191 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-08T16:30:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQA1t0tlqJczARAAFUTkhFS8ZdhAvUy7eLNC5g==", 2023-11-08T16:30:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.192 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-08T16:30:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.bootstrap-rgw", 2023-11-08T16:30:23.193 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQA1t0tlrLAzARAAvv3ShGgry7vLFmaRIsbHtg==", 2023-11-08T16:30:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-08T16:30:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.194 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.x", 2023-11-08T16:30:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlhZFjLRAAANBB8roHZH4z4pE4C2Aq/Q==", 2023-11-08T16:30:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:30:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2023-11-08T16:30:23.195 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.y", 2023-11-08T16:30:23.196 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlNpDaIBAAtgLFNQitjQnLny6SRY7hHw==", 2023-11-08T16:30:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:30:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:30:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2023-11-08T16:30:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:30:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.197 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:23.198 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2023-11-08T16:30:23.198 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:30:23.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 -- 172.21.15.81:0/1295167050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87b806ae70 msgr2=0x7f87b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:23.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87b806ae70 0x7f87b806d330 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f87c8034000 tx=0x7f87c8036040 comp rx=0 tx=0).stop 2023-11-08T16:30:23.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 -- 172.21.15.81:0/1295167050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87d80ca280 msgr2=0x7f87d80ca700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:23.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87d80ca280 0x7f87d80ca700 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f87c000e440 tx=0x7f87c000e840 comp rx=0 tx=0).stop 2023-11-08T16:30:23.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 -- 172.21.15.81:0/1295167050 shutdown_connections 2023-11-08T16:30:23.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87b806ae70 0x7f87b806d330 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87d8152d20 0x7f87d81531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87d80ca280 0x7f87d80ca700 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 --2- 172.21.15.81:0/1295167050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87c4003970 0x7f87d815f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 -- 172.21.15.81:0/1295167050 >> 172.21.15.81:0/1295167050 conn(0x7f87d806aac0 msgr2=0x7f87d8152310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:23.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 -- 172.21.15.81:0/1295167050 shutdown_connections 2023-11-08T16:30:23.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.150+0000 7f87dea76640 1 -- 172.21.15.81:0/1295167050 wait complete. 2023-11-08T16:30:23.202 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-08T16:30:23.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.416+0000 7f6dbc294640 1 Processor -- start 2023-11-08T16:30:23.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.417+0000 7f6dbc294640 1 -- start start 2023-11-08T16:30:23.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.418+0000 7f6dbc294640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6db40d2b70 0x7f6db40cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.418+0000 7f6dbc294640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 0x7f6db40b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.418+0000 7f6dbc294640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db40b6100 con 0x7f6db415d310 2023-11-08T16:30:23.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.418+0000 7f6dbc294640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db40b6480 con 0x7f6db40d2b70 2023-11-08T16:30:23.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.418+0000 7f6dbc294640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40d0450 con 0x7f6db40d0000 2023-11-08T16:30:23.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.418+0000 7f6dba831640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 0x7f6db40b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:23.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.419+0000 7f6db982f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6db40d2b70 0x7f6db40cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:23.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.419+0000 7f6dba831640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 0x7f6db40b68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:53286/0 (socket says 172.21.15.81:53286) 2023-11-08T16:30:23.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.419+0000 7f6dba831640 1 -- 172.21.15.81:0/2753240595 learned_addr learned my addr 172.21.15.81:0/2753240595 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:23.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.419+0000 7f6dba831640 1 -- 172.21.15.81:0/2753240595 >> v2:172.21.15.93:3300/0 conn(0x7f6db40d2b70 msgr2=0x7f6db40cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:23.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.419+0000 7f6dba831640 1 --2- 172.21.15.81:0/2753240595 >> v2:172.21.15.93:3300/0 conn(0x7f6db40d2b70 0x7f6db40cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.419+0000 7f6dba831640 1 -- 172.21.15.81:0/2753240595 >> v1:172.21.15.93:6789/0 conn(0x7f6db415d310 legacy=0x7f6db40d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:23.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.419+0000 7f6dba831640 1 -- 172.21.15.81:0/2753240595 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db40d1b50 con 0x7f6db40d0000 2023-11-08T16:30:23.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.421+0000 7f6dba831640 1 --2- 172.21.15.81:0/2753240595 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 0x7f6db40b68a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6db408c570 tx=0x7f6dac02bf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e672e97c6ab5f202 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:23.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.421+0000 7f6db902e640 1 -- 172.21.15.81:0/2753240595 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac02cac0 con 0x7f6db40d0000 2023-11-08T16:30:23.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.421+0000 7f6db902e640 1 -- 172.21.15.81:0/2753240595 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac02cc20 con 0x7f6db40d0000 2023-11-08T16:30:23.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.421+0000 7f6db902e640 1 -- 172.21.15.81:0/2753240595 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0343f0 con 0x7f6db40d0000 2023-11-08T16:30:23.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.422+0000 7f6dbc294640 1 -- 172.21.15.81:0/2753240595 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 msgr2=0x7f6db40b68a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:23.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.422+0000 7f6dbc294640 1 --2- 172.21.15.81:0/2753240595 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 0x7f6db40b68a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6db408c570 tx=0x7f6dac02bf20 comp rx=0 tx=0).stop 2023-11-08T16:30:23.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.422+0000 7f6dbc294640 1 -- 172.21.15.81:0/2753240595 shutdown_connections 2023-11-08T16:30:23.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.422+0000 7f6dbc294640 1 --2- 172.21.15.81:0/2753240595 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 0x7f6db40b68a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.422+0000 7f6dbc294640 1 --2- 172.21.15.81:0/2753240595 >> v2:172.21.15.93:3300/0 conn(0x7f6db40d2b70 0x7f6db40cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.422+0000 7f6dbc294640 1 -- 172.21.15.81:0/2753240595 >> 172.21.15.81:0/2753240595 conn(0x7f6db406aac0 msgr2=0x7f6db406aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:23.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.422+0000 7f6dbc294640 1 -- 172.21.15.81:0/2753240595 shutdown_connections 2023-11-08T16:30:23.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.422+0000 7f6dbc294640 1 -- 172.21.15.81:0/2753240595 wait complete. 2023-11-08T16:30:23.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.424+0000 7f6dbc294640 1 Processor -- start 2023-11-08T16:30:23.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.424+0000 7f6dbc294640 1 -- start start 2023-11-08T16:30:23.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.425+0000 7f6dbc294640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 0x7f6db415c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.425+0000 7f6dbc294640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6db40d2b70 0x7f6db40aa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.425+0000 7f6dbc294640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6db415d310 0x7f6db40ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.425+0000 7f6dbc294640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40abea0 con 0x7f6db40d2b70 2023-11-08T16:30:23.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.425+0000 7f6dbc294640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40ac180 con 0x7f6db415d310 2023-11-08T16:30:23.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.425+0000 7f6dbc294640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40a9ac0 con 0x7f6db40d0000 2023-11-08T16:30:23.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6db982f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6db40d2b70 0x7f6db40aa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:23.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6db982f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6db40d2b70 0x7f6db40aa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46674/0 (socket says 172.21.15.81:46674) 2023-11-08T16:30:23.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6db982f640 1 -- 172.21.15.81:0/2587527212 learned_addr learned my addr 172.21.15.81:0/2587527212 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:23.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6dba030640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 0x7f6db415c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:23.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6dba831640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6db415d310 0x7f6db40ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:23.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6db982f640 1 -- 172.21.15.81:0/2587527212 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 msgr2=0x7f6db415c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:23.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6db982f640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 0x7f6db415c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6db982f640 1 -- 172.21.15.81:0/2587527212 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6db415d310 msgr2=0x7f6db40ab3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:23.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6db982f640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6db415d310 0x7f6db40ab3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6db982f640 1 -- 172.21.15.81:0/2587527212 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dac009b30 con 0x7f6db40d2b70 2023-11-08T16:30:23.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6dba831640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6db415d310 0x7f6db40ab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.426+0000 7f6db982f640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6db40d2b70 0x7f6db40aa5a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6da400e870 tx=0x7f6da400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.427+0000 7f6daaffd640 1 -- 172.21.15.81:0/2587527212 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da40094f0 con 0x7f6db40d2b70 2023-11-08T16:30:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.427+0000 7f6daaffd640 1 -- 172.21.15.81:0/2587527212 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da400c940 con 0x7f6db40d2b70 2023-11-08T16:30:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.427+0000 7f6daaffd640 1 -- 172.21.15.81:0/2587527212 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da400ccf0 con 0x7f6db40d2b70 2023-11-08T16:30:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.427+0000 7f6dbc294640 1 -- 172.21.15.81:0/2587527212 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db40a9da0 con 0x7f6db40d2b70 2023-11-08T16:30:23.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.427+0000 7f6dbc294640 1 -- 172.21.15.81:0/2587527212 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db415c9d0 con 0x7f6db40d2b70 2023-11-08T16:30:23.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.428+0000 7f6dbc294640 1 -- 172.21.15.81:0/2587527212 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db40ac3a0 con 0x7f6db40d2b70 2023-11-08T16:30:23.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.429+0000 7f6daaffd640 1 -- 172.21.15.81:0/2587527212 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6da4013470 con 0x7f6db40d2b70 2023-11-08T16:30:23.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.429+0000 7f6daaffd640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d9806ae70 0x7f6d9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.430+0000 7f6daaffd640 1 -- 172.21.15.81:0/2587527212 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6da4089480 con 0x7f6db40d2b70 2023-11-08T16:30:23.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.430+0000 7f6dba030640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d9806ae70 0x7f6d9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:23.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.430+0000 7f6dba030640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d9806ae70 0x7f6d9806d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6db41e7c90 tx=0x7f6db0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:23.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.434+0000 7f6daaffd640 1 -- 172.21.15.81:0/2587527212 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6da40547e0 con 0x7f6db40d2b70 2023-11-08T16:30:23.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.694+0000 7f6dbc294640 1 -- 172.21.15.81:0/2587527212 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f6db40743f0 con 0x7f6db40d2b70 2023-11-08T16:30:23.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.699+0000 7f6daaffd640 1 -- 172.21.15.81:0/2587527212 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6da4057e00 con 0x7f6db40d2b70 2023-11-08T16:30:23.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.701+0000 7f6dbc294640 1 -- 172.21.15.81:0/2587527212 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d9806ae70 msgr2=0x7f6d9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:23.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.701+0000 7f6dbc294640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d9806ae70 0x7f6d9806d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6db41e7c90 tx=0x7f6db0009040 comp rx=0 tx=0).stop 2023-11-08T16:30:23.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.702+0000 7f6dbc294640 1 -- 172.21.15.81:0/2587527212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6db40d2b70 msgr2=0x7f6db40aa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:23.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.702+0000 7f6dbc294640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6db40d2b70 0x7f6db40aa5a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6da400e870 tx=0x7f6da400ed30 comp rx=0 tx=0).stop 2023-11-08T16:30:23.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.702+0000 7f6dbc294640 1 -- 172.21.15.81:0/2587527212 shutdown_connections 2023-11-08T16:30:23.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.702+0000 7f6dbc294640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d9806ae70 0x7f6d9806d330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.702+0000 7f6dbc294640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6db415d310 0x7f6db40ab3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.702+0000 7f6dbc294640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6db40d2b70 0x7f6db40aa5a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.702+0000 7f6dbc294640 1 --2- 172.21.15.81:0/2587527212 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6db40d0000 0x7f6db415c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.702+0000 7f6dbc294640 1 -- 172.21.15.81:0/2587527212 >> 172.21.15.81:0/2587527212 conn(0x7f6db406aac0 msgr2=0x7f6db4160270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:23.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.702+0000 7f6dbc294640 1 -- 172.21.15.81:0/2587527212 shutdown_connections 2023-11-08T16:30:23.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.702+0000 7f6dbc294640 1 -- 172.21.15.81:0/2587527212 wait complete. 2023-11-08T16:30:23.721 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-08T16:30:23.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.935+0000 7f6bdc2c9640 1 Processor -- start 2023-11-08T16:30:23.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.935+0000 7f6bdc2c9640 1 -- start start 2023-11-08T16:30:23.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bdc2c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bd40c90f0 0x7f6bd40c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bdc2c9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6bd40b18f0 0x7f6bd40b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bdc2c9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd406cc60 con 0x7f6bd415aeb0 2023-11-08T16:30:23.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bdc2c9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd40cb3e0 con 0x7f6bd40b18f0 2023-11-08T16:30:23.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bdc2c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd40cabb0 con 0x7f6bd40c90f0 2023-11-08T16:30:23.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bda866640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6bd40b18f0 0x7f6bd40b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:23.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bda866640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6bd40b18f0 0x7f6bd40b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46684/0 (socket says 172.21.15.81:46684) 2023-11-08T16:30:23.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bda866640 1 -- 172.21.15.81:0/2071038125 learned_addr learned my addr 172.21.15.81:0/2071038125 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:23.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bd9864640 1 --2- 172.21.15.81:0/2071038125 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bd40c90f0 0x7f6bd40c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:23.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bda866640 1 -- 172.21.15.81:0/2071038125 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bd40c90f0 msgr2=0x7f6bd40c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:23.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bda866640 1 --2- 172.21.15.81:0/2071038125 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bd40c90f0 0x7f6bd40c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bda866640 1 -- 172.21.15.81:0/2071038125 >> v1:172.21.15.93:6789/0 conn(0x7f6bd415aeb0 legacy=0x7f6bd40ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:23.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.937+0000 7f6bda866640 1 -- 172.21.15.81:0/2071038125 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd40c9a90 con 0x7f6bd40b18f0 2023-11-08T16:30:23.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.939+0000 7f6bda866640 1 --2- 172.21.15.81:0/2071038125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd40b18f0 0x7f6bd40b1d40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009e50 tx=0x7f6bcc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57b1e31e60a5a2de server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:23.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.939+0000 7f6bd9063640 1 -- 172.21.15.81:0/2071038125 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc02ca70 con 0x7f6bd40b18f0 2023-11-08T16:30:23.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.939+0000 7f6bd9063640 1 -- 172.21.15.81:0/2071038125 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bcc02cbd0 con 0x7f6bd40b18f0 2023-11-08T16:30:23.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.939+0000 7f6bd9063640 1 -- 172.21.15.81:0/2071038125 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc0343f0 con 0x7f6bd40b18f0 2023-11-08T16:30:23.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.939+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2071038125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd40b18f0 msgr2=0x7f6bd40b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:23.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.939+0000 7f6bdc2c9640 1 --2- 172.21.15.81:0/2071038125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd40b18f0 0x7f6bd40b1d40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc009e50 tx=0x7f6bcc02bed0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.939+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2071038125 shutdown_connections 2023-11-08T16:30:23.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.939+0000 7f6bdc2c9640 1 --2- 172.21.15.81:0/2071038125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd40b18f0 0x7f6bd40b1d40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.939+0000 7f6bdc2c9640 1 --2- 172.21.15.81:0/2071038125 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bd40c90f0 0x7f6bd40c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.939+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2071038125 >> 172.21.15.81:0/2071038125 conn(0x7f6bd40ceaa0 msgr2=0x7f6bd40cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.940+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2071038125 shutdown_connections 2023-11-08T16:30:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.940+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2071038125 wait complete. 2023-11-08T16:30:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.941+0000 7f6bdc2c9640 1 Processor -- start 2023-11-08T16:30:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.942+0000 7f6bdc2c9640 1 -- start start 2023-11-08T16:30:23.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.942+0000 7f6bdc2c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd40b18f0 0x7f6bd414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.942+0000 7f6bdc2c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bd40c90f0 0x7f6bd40c47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.942+0000 7f6bdc2c9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bd415aeb0 0x7f6bd40c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.942+0000 7f6bdc2c9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd40c4380 con 0x7f6bd40b18f0 2023-11-08T16:30:23.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.943+0000 7f6bdc2c9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd40c4030 con 0x7f6bd415aeb0 2023-11-08T16:30:23.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.943+0000 7f6bdc2c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd415c4f0 con 0x7f6bd40c90f0 2023-11-08T16:30:23.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.943+0000 7f6bda065640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd40b18f0 0x7f6bd414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:23.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.943+0000 7f6bda065640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd40b18f0 0x7f6bd414fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46700/0 (socket says 172.21.15.81:46700) 2023-11-08T16:30:23.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.943+0000 7f6bda065640 1 -- 172.21.15.81:0/2448878663 learned_addr learned my addr 172.21.15.81:0/2448878663 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:23.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.943+0000 7f6bda065640 1 -- 172.21.15.81:0/2448878663 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bd40c90f0 msgr2=0x7f6bd40c47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:23.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.943+0000 7f6bda065640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bd40c90f0 0x7f6bd40c47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.943+0000 7f6bda065640 1 -- 172.21.15.81:0/2448878663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bd415aeb0 msgr2=0x7f6bd40c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:23.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.943+0000 7f6bda065640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bd415aeb0 0x7f6bd40c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:23.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.943+0000 7f6bda065640 1 -- 172.21.15.81:0/2448878663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc009b30 con 0x7f6bd40b18f0 2023-11-08T16:30:23.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.943+0000 7f6bda065640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd40b18f0 0x7f6bd414fd00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000b3e0 tx=0x7f6bd000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:23.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.944+0000 7f6bcaffd640 1 -- 172.21.15.81:0/2448878663 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0011830 con 0x7f6bd40b18f0 2023-11-08T16:30:23.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.944+0000 7f6bcaffd640 1 -- 172.21.15.81:0/2448878663 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd00119b0 con 0x7f6bd40b18f0 2023-11-08T16:30:23.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.944+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2448878663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd415c7d0 con 0x7f6bd40b18f0 2023-11-08T16:30:23.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.944+0000 7f6bcaffd640 1 -- 172.21.15.81:0/2448878663 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0011d60 con 0x7f6bd40b18f0 2023-11-08T16:30:23.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.944+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2448878663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd4150550 con 0x7f6bd40b18f0 2023-11-08T16:30:23.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.945+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2448878663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd40743f0 con 0x7f6bd40b18f0 2023-11-08T16:30:23.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.946+0000 7f6bcaffd640 1 -- 172.21.15.81:0/2448878663 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6bd001a400 con 0x7f6bd40b18f0 2023-11-08T16:30:23.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.947+0000 7f6bcaffd640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6bb806ae70 0x7f6bb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:23.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.947+0000 7f6bcaffd640 1 -- 172.21.15.81:0/2448878663 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6bd00893e0 con 0x7f6bd40b18f0 2023-11-08T16:30:23.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.947+0000 7f6bd9864640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6bb806ae70 0x7f6bb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:23.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.948+0000 7f6bd9864640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6bb806ae70 0x7f6bb806d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6bd41f3d90 tx=0x7f6bc4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:23.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:23.951+0000 7f6bcaffd640 1 -- 172.21.15.81:0/2448878663 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6bd00547d0 con 0x7f6bd40b18f0 2023-11-08T16:30:24.205 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-08T16:30:24.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.207+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2448878663 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f6bd406c730 con 0x7f6bd40b18f0 2023-11-08T16:30:24.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.212+0000 7f6bcaffd640 1 -- 172.21.15.81:0/2448878663 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6bd00150a0 con 0x7f6bd40b18f0 2023-11-08T16:30:24.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.214+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2448878663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6bb806ae70 msgr2=0x7f6bb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:24.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.214+0000 7f6bdc2c9640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6bb806ae70 0x7f6bb806d330 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6bd41f3d90 tx=0x7f6bc4008040 comp rx=0 tx=0).stop 2023-11-08T16:30:24.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.214+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2448878663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd40b18f0 msgr2=0x7f6bd414fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:24.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.214+0000 7f6bdc2c9640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd40b18f0 0x7f6bd414fd00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000b3e0 tx=0x7f6bd000b8a0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.214+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2448878663 shutdown_connections 2023-11-08T16:30:24.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.214+0000 7f6bdc2c9640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6bb806ae70 0x7f6bb806d330 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.214+0000 7f6bdc2c9640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bd415aeb0 0x7f6bd40c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.214+0000 7f6bdc2c9640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bd40c90f0 0x7f6bd40c47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.214+0000 7f6bdc2c9640 1 --2- 172.21.15.81:0/2448878663 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bd40b18f0 0x7f6bd414fd00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.214+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2448878663 >> 172.21.15.81:0/2448878663 conn(0x7f6bd40ceaa0 msgr2=0x7f6bd40b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:24.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.214+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2448878663 shutdown_connections 2023-11-08T16:30:24.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.215+0000 7f6bdc2c9640 1 -- 172.21.15.81:0/2448878663 wait complete. 2023-11-08T16:30:24.233 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-08T16:30:24.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.441+0000 7f63fb59e640 1 Processor -- start 2023-11-08T16:30:24.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.441+0000 7f63fb59e640 1 -- start start 2023-11-08T16:30:24.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.443+0000 7f63fb59e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63fc15d750 0x7f63fc15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.443+0000 7f63fb59e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f63fc0cd110 0x7f63fc0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.443+0000 7f63fb59e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63fc070580 con 0x7f63fc159af0 2023-11-08T16:30:24.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.443+0000 7f63fb59e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63fc0cdac0 con 0x7f63fc15d750 2023-11-08T16:30:24.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.443+0000 7f63fb59e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63fc0c9460 con 0x7f63fc0cd110 2023-11-08T16:30:24.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.443+0000 7f63f3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63fc15d750 0x7f63fc15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.443+0000 7f63f3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63fc15d750 0x7f63fc15dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46710/0 (socket says 172.21.15.81:46710) 2023-11-08T16:30:24.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.443+0000 7f63f3fff640 1 -- 172.21.15.81:0/3306457451 learned_addr learned my addr 172.21.15.81:0/3306457451 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:24.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.443+0000 7f63fad9d640 1 --2- 172.21.15.81:0/3306457451 >> v2:172.21.15.93:3300/0 conn(0x7f63fc0cd110 0x7f63fc0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.444+0000 7f63f3fff640 1 -- 172.21.15.81:0/3306457451 >> v2:172.21.15.93:3300/0 conn(0x7f63fc0cd110 msgr2=0x7f63fc0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:24.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.444+0000 7f63f3fff640 1 --2- 172.21.15.81:0/3306457451 >> v2:172.21.15.93:3300/0 conn(0x7f63fc0cd110 0x7f63fc0cd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.444+0000 7f63f3fff640 1 -- 172.21.15.81:0/3306457451 >> v1:172.21.15.93:6789/0 conn(0x7f63fc159af0 legacy=0x7f63fc070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:24.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.444+0000 7f63f3fff640 1 -- 172.21.15.81:0/3306457451 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63fc15e2a0 con 0x7f63fc15d750 2023-11-08T16:30:24.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.445+0000 7f63f3fff640 1 --2- 172.21.15.81:0/3306457451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63fc15d750 0x7f63fc15dbe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f63fc06f750 tx=0x7f63e402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e54c94bc784c6cb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:24.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.445+0000 7f63f9d9b640 1 -- 172.21.15.81:0/3306457451 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e402ee90 con 0x7f63fc15d750 2023-11-08T16:30:24.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.445+0000 7f63f9d9b640 1 -- 172.21.15.81:0/3306457451 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e4036440 con 0x7f63fc15d750 2023-11-08T16:30:24.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.445+0000 7f63f9d9b640 1 -- 172.21.15.81:0/3306457451 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e40367d0 con 0x7f63fc15d750 2023-11-08T16:30:24.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.446+0000 7f63fb59e640 1 -- 172.21.15.81:0/3306457451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63fc15d750 msgr2=0x7f63fc15dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:24.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.446+0000 7f63fb59e640 1 --2- 172.21.15.81:0/3306457451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63fc15d750 0x7f63fc15dbe0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f63fc06f750 tx=0x7f63e402e0e0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.446+0000 7f63fb59e640 1 -- 172.21.15.81:0/3306457451 shutdown_connections 2023-11-08T16:30:24.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.446+0000 7f63fb59e640 1 --2- 172.21.15.81:0/3306457451 >> v2:172.21.15.93:3300/0 conn(0x7f63fc0cd110 0x7f63fc0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.446+0000 7f63fb59e640 1 --2- 172.21.15.81:0/3306457451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63fc15d750 0x7f63fc15dbe0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.446+0000 7f63fb59e640 1 -- 172.21.15.81:0/3306457451 >> 172.21.15.81:0/3306457451 conn(0x7f63fc0b04d0 msgr2=0x7f63fc0b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:24.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.446+0000 7f63fb59e640 1 -- 172.21.15.81:0/3306457451 shutdown_connections 2023-11-08T16:30:24.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.446+0000 7f63fb59e640 1 -- 172.21.15.81:0/3306457451 wait complete. 2023-11-08T16:30:24.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.448+0000 7f63fb59e640 1 Processor -- start 2023-11-08T16:30:24.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.448+0000 7f63fb59e640 1 -- start start 2023-11-08T16:30:24.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.449+0000 7f63fb59e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63fc0cd110 0x7f63fc159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.449+0000 7f63fb59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63fc159af0 0x7f63fc1f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.449+0000 7f63fb59e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63fc15d750 0x7f63fc1f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.449+0000 7f63fb59e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc0c7920 con 0x7f63fc159af0 2023-11-08T16:30:24.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.449+0000 7f63fb59e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc0c6da0 con 0x7f63fc0cd110 2023-11-08T16:30:24.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.449+0000 7f63fb59e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63fc0c7080 con 0x7f63fc15d750 2023-11-08T16:30:24.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.449+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63fc159af0 0x7f63fc1f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.449+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63fc159af0 0x7f63fc1f6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46720/0 (socket says 172.21.15.81:46720) 2023-11-08T16:30:24.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.449+0000 7f63f3fff640 1 -- 172.21.15.81:0/2786974292 learned_addr learned my addr 172.21.15.81:0/2786974292 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:24.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.450+0000 7f63f3fff640 1 -- 172.21.15.81:0/2786974292 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63fc15d750 msgr2=0x7f63fc1f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:24.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.450+0000 7f63f3fff640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63fc15d750 0x7f63fc1f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.450+0000 7f63f3fff640 1 -- 172.21.15.81:0/2786974292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63fc0cd110 msgr2=0x7f63fc159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:24.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.450+0000 7f63f3fff640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63fc0cd110 0x7f63fc159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.450+0000 7f63f3fff640 1 -- 172.21.15.81:0/2786974292 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e400b770 con 0x7f63fc159af0 2023-11-08T16:30:24.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.450+0000 7f63f3fff640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63fc159af0 0x7f63fc1f6650 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f63e402e120 tx=0x7f63e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.450+0000 7f63f2ffd640 1 -- 172.21.15.81:0/2786974292 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e40348d0 con 0x7f63fc159af0 2023-11-08T16:30:24.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.450+0000 7f63f2ffd640 1 -- 172.21.15.81:0/2786974292 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e4034a30 con 0x7f63fc159af0 2023-11-08T16:30:24.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.450+0000 7f63f2ffd640 1 -- 172.21.15.81:0/2786974292 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e4034de0 con 0x7f63fc159af0 2023-11-08T16:30:24.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.450+0000 7f63fb59e640 1 -- 172.21.15.81:0/2786974292 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63fc0c7360 con 0x7f63fc159af0 2023-11-08T16:30:24.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.450+0000 7f63fb59e640 1 -- 172.21.15.81:0/2786974292 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63fc0c69c0 con 0x7f63fc159af0 2023-11-08T16:30:24.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.451+0000 7f63fb59e640 1 -- 172.21.15.81:0/2786974292 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63fc0c6b50 con 0x7f63fc159af0 2023-11-08T16:30:24.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.452+0000 7f63f2ffd640 1 -- 172.21.15.81:0/2786974292 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f63e40403e0 con 0x7f63fc159af0 2023-11-08T16:30:24.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.453+0000 7f63f2ffd640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63dc06ae70 0x7f63dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.453+0000 7f63f2ffd640 1 -- 172.21.15.81:0/2786974292 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f63e40ab610 con 0x7f63fc159af0 2023-11-08T16:30:24.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.453+0000 7f63fa59c640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63dc06ae70 0x7f63dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.454+0000 7f63fa59c640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63dc06ae70 0x7f63dc06d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63fc157d00 tx=0x7f63ec0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:24.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.457+0000 7f63f2ffd640 1 -- 172.21.15.81:0/2786974292 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63e4076a00 con 0x7f63fc159af0 2023-11-08T16:30:24.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.714+0000 7f63fb59e640 1 -- 172.21.15.81:0/2786974292 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f63fc06fe30 con 0x7f63fc159af0 2023-11-08T16:30:24.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.719+0000 7f63f2ffd640 1 -- 172.21.15.81:0/2786974292 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f63e40460a0 con 0x7f63fc159af0 2023-11-08T16:30:24.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 -- 172.21.15.81:0/2786974292 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63dc06ae70 msgr2=0x7f63dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:24.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63dc06ae70 0x7f63dc06d330 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f63fc157d00 tx=0x7f63ec0079b0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 -- 172.21.15.81:0/2786974292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63fc159af0 msgr2=0x7f63fc1f6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:24.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63fc159af0 0x7f63fc1f6650 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f63e402e120 tx=0x7f63e40046c0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 -- 172.21.15.81:0/2786974292 shutdown_connections 2023-11-08T16:30:24.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63dc06ae70 0x7f63dc06d330 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63fc15d750 0x7f63fc1f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63fc159af0 0x7f63fc1f6650 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 --2- 172.21.15.81:0/2786974292 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63fc0cd110 0x7f63fc159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 -- 172.21.15.81:0/2786974292 >> 172.21.15.81:0/2786974292 conn(0x7f63fc0b04d0 msgr2=0x7f63fc0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:24.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 -- 172.21.15.81:0/2786974292 shutdown_connections 2023-11-08T16:30:24.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.722+0000 7f63fb59e640 1 -- 172.21.15.81:0/2786974292 wait complete. 2023-11-08T16:30:24.742 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-08T16:30:24.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.942+0000 7f7498e60640 1 Processor -- start 2023-11-08T16:30:24.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.943+0000 7f7498e60640 1 -- start start 2023-11-08T16:30:24.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.944+0000 7f7498e60640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f749406f7c0 0x7f74940cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.944+0000 7f7498e60640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f749406cd70 con 0x7f74940ad940 2023-11-08T16:30:24.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.944+0000 7f7498e60640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74940ca500 con 0x7f749415ac20 2023-11-08T16:30:24.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.944+0000 7f7498e60640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74940ae130 con 0x7f749406f7c0 2023-11-08T16:30:24.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.944+0000 7f7493fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f74940ad940 0x7f749406b390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44100/0 (socket says 172.21.15.81:44100) 2023-11-08T16:30:24.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.944+0000 7f7493fff640 1 -- 172.21.15.81:0/1937659893 learned_addr learned my addr 172.21.15.81:0/1937659893 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:24.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.944+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 698490054 0 0) 0x7f749406cd70 con 0x7f74940ad940 2023-11-08T16:30:24.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.944+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7480003520 con 0x7f74940ad940 2023-11-08T16:30:24.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f748bfff640 1 --2- 172.21.15.81:0/1937659893 >> v2:172.21.15.93:3300/0 conn(0x7f749406f7c0 0x7f74940cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 486961252 0 0) 0x7f7480003520 con 0x7f74940ad940 2023-11-08T16:30:24.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 >> v2:172.21.15.93:3300/0 conn(0x7f749406f7c0 msgr2=0x7f74940cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:24.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 --2- 172.21.15.81:0/1937659893 >> v2:172.21.15.93:3300/0 conn(0x7f749406f7c0 0x7f74940cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 >> v1:172.21.15.93:6789/0 conn(0x7f749415ac20 legacy=0x7f7494070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:24.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f749406d030 con 0x7f74940ad940 2023-11-08T16:30:24.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f748c0019f0 con 0x7f74940ad940 2023-11-08T16:30:24.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 >> v1:172.21.15.81:6789/0 conn(0x7f74940ad940 legacy=0x7f749406b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:24.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 --2- 172.21.15.81:0/1937659893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800039a0 0x7f7480003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 --2- 172.21.15.81:0/1937659893 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800047c0 0x7f7480004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 --2- 172.21.15.81:0/1937659893 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74800052a0 0x7f748000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74940ae130 con 0x7f74800039a0 2023-11-08T16:30:24.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748000c170 con 0x7f74800052a0 2023-11-08T16:30:24.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748000c330 con 0x7f74800047c0 2023-11-08T16:30:24.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.945+0000 7f74937fe640 1 --2- 172.21.15.81:0/1937659893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800039a0 0x7f7480003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.946+0000 7f74937fe640 1 -- 172.21.15.81:0/1937659893 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800047c0 msgr2=0x7f7480004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:24.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.946+0000 7f748bfff640 1 --2- 172.21.15.81:0/1937659893 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800047c0 0x7f7480004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.946+0000 7f7493fff640 1 --2- 172.21.15.81:0/1937659893 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74800052a0 0x7f748000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.946+0000 7f74937fe640 1 --2- 172.21.15.81:0/1937659893 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800047c0 0x7f7480004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.946+0000 7f74937fe640 1 -- 172.21.15.81:0/1937659893 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74800052a0 msgr2=0x7f748000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:24.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.946+0000 7f74937fe640 1 --2- 172.21.15.81:0/1937659893 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74800052a0 0x7f748000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.946+0000 7f7493fff640 1 -- 172.21.15.81:0/1937659893 reap_dead start 2023-11-08T16:30:24.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.946+0000 7f74937fe640 1 -- 172.21.15.81:0/1937659893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7480003520 con 0x7f74800039a0 2023-11-08T16:30:24.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.947+0000 7f74937fe640 1 --2- 172.21.15.81:0/1937659893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800039a0 0x7f7480003da0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7484001650 tx=0x7f74800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b8bde86e96a2c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:24.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.947+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484005270 con 0x7f74800039a0 2023-11-08T16:30:24.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.947+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748402e070 con 0x7f74800039a0 2023-11-08T16:30:24.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.947+0000 7f7492ffd640 1 -- 172.21.15.81:0/1937659893 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7484003610 con 0x7f74800039a0 2023-11-08T16:30:24.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.947+0000 7f7498e60640 1 -- 172.21.15.81:0/1937659893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800039a0 msgr2=0x7f7480003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:24.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.947+0000 7f7498e60640 1 --2- 172.21.15.81:0/1937659893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800039a0 0x7f7480003da0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7484001650 tx=0x7f74800057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.947+0000 7f7498e60640 1 -- 172.21.15.81:0/1937659893 shutdown_connections 2023-11-08T16:30:24.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.947+0000 7f7498e60640 1 --2- 172.21.15.81:0/1937659893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800039a0 0x7f7480003da0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.947+0000 7f7498e60640 1 -- 172.21.15.81:0/1937659893 >> 172.21.15.81:0/1937659893 conn(0x7f74940b04d0 msgr2=0x7f74940b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:24.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.948+0000 7f7498e60640 1 -- 172.21.15.81:0/1937659893 shutdown_connections 2023-11-08T16:30:24.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.948+0000 7f7498e60640 1 -- 172.21.15.81:0/1937659893 wait complete. 2023-11-08T16:30:24.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.949+0000 7f7498e60640 1 Processor -- start 2023-11-08T16:30:24.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.949+0000 7f7498e60640 1 -- start start 2023-11-08T16:30:24.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.950+0000 7f7498e60640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800039a0 0x7f74940ab400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.950+0000 7f7498e60640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74940a8ec0 0x7f74941f1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.950+0000 7f7498e60640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f749415b390 0x7f749415b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.950+0000 7f7498e60640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7494152910 con 0x7f749415b390 2023-11-08T16:30:24.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.950+0000 7f7498e60640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74940aba90 con 0x7f74940a8ec0 2023-11-08T16:30:24.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.950+0000 7f7498e60640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74940abd70 con 0x7f74800039a0 2023-11-08T16:30:24.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.950+0000 7f7493fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f749415b390 0x7f749415b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7493fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f749415b390 0x7f749415b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46744/0 (socket says 172.21.15.81:46744) 2023-11-08T16:30:24.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7493fff640 1 -- 172.21.15.81:0/1585099682 learned_addr learned my addr 172.21.15.81:0/1585099682 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:24.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f748bfff640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74940a8ec0 0x7f74941f1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f74937fe640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800039a0 0x7f74940ab400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7493fff640 1 -- 172.21.15.81:0/1585099682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800039a0 msgr2=0x7f74940ab400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:24.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7493fff640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800039a0 0x7f74940ab400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7493fff640 1 -- 172.21.15.81:0/1585099682 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74940a8ec0 msgr2=0x7f74941f1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:24.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7493fff640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74940a8ec0 0x7f74941f1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:24.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7493fff640 1 -- 172.21.15.81:0/1585099682 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74940ac050 con 0x7f749415b390 2023-11-08T16:30:24.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7493fff640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f749415b390 0x7f749415b810 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f748c007c40 tx=0x7f748c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:24.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7490ff9640 1 -- 172.21.15.81:0/1585099682 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c00de90 con 0x7f749415b390 2023-11-08T16:30:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7490ff9640 1 -- 172.21.15.81:0/1585099682 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748c00bb80 con 0x7f749415b390 2023-11-08T16:30:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7490ff9640 1 -- 172.21.15.81:0/1585099682 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748c007070 con 0x7f749415b390 2023-11-08T16:30:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.951+0000 7f7498e60640 1 -- 172.21.15.81:0/1585099682 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74940ac330 con 0x7f749415b390 2023-11-08T16:30:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.952+0000 7f7498e60640 1 -- 172.21.15.81:0/1585099682 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74941f23e0 con 0x7f749415b390 2023-11-08T16:30:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.953+0000 7f7498e60640 1 -- 172.21.15.81:0/1585099682 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74940ac850 con 0x7f749415b390 2023-11-08T16:30:24.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.953+0000 7f7490ff9640 1 -- 172.21.15.81:0/1585099682 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f748c0071d0 con 0x7f749415b390 2023-11-08T16:30:24.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.954+0000 7f7490ff9640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f747406ae70 0x7f747406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:24.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.954+0000 7f7490ff9640 1 -- 172.21.15.81:0/1585099682 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f748c0895c0 con 0x7f749415b390 2023-11-08T16:30:24.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.955+0000 7f74937fe640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f747406ae70 0x7f747406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:24.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.955+0000 7f74937fe640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f747406ae70 0x7f747406d330 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7484033000 tx=0x7f7484009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:24.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:24.959+0000 7f7490ff9640 1 -- 172.21.15.81:0/1585099682 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f74940ac850 con 0x7f749415b390 2023-11-08T16:30:25.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.196+0000 7f7498e60640 1 -- 172.21.15.81:0/1585099682 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f74940ac850 con 0x7f749415b390 2023-11-08T16:30:25.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.200+0000 7f7490ff9640 1 -- 172.21.15.81:0/1585099682 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f74940ac850 con 0x7f749415b390 2023-11-08T16:30:25.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 -- 172.21.15.81:0/1585099682 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f747406ae70 msgr2=0x7f747406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f747406ae70 0x7f747406d330 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7484033000 tx=0x7f7484009030 comp rx=0 tx=0).stop 2023-11-08T16:30:25.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 -- 172.21.15.81:0/1585099682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f749415b390 msgr2=0x7f749415b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f749415b390 0x7f749415b810 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f748c007c40 tx=0x7f748c00b4a0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 -- 172.21.15.81:0/1585099682 shutdown_connections 2023-11-08T16:30:25.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f747406ae70 0x7f747406d330 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f749415b390 0x7f749415b810 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f74940a8ec0 0x7f74941f1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 --2- 172.21.15.81:0/1585099682 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800039a0 0x7f74940ab400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 -- 172.21.15.81:0/1585099682 >> 172.21.15.81:0/1585099682 conn(0x7f74940b04d0 msgr2=0x7f74940b4120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:25.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 -- 172.21.15.81:0/1585099682 shutdown_connections 2023-11-08T16:30:25.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.202+0000 7f7498e60640 1 -- 172.21.15.81:0/1585099682 wait complete. 2023-11-08T16:30:25.221 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-08T16:30:25.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.425+0000 7fd3d8101640 1 Processor -- start 2023-11-08T16:30:25.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.425+0000 7fd3d8101640 1 -- start start 2023-11-08T16:30:25.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.426+0000 7fd3d8101640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d00c9ae0 0x7fd3d006f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.426+0000 7fd3d8101640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d00c9ec0 con 0x7fd3d006cce0 2023-11-08T16:30:25.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.427+0000 7fd3d8101640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d00cc710 con 0x7fd3d006bec0 2023-11-08T16:30:25.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.427+0000 7fd3d8101640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d00cc9f0 con 0x7fd3d00c9ae0 2023-11-08T16:30:25.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.427+0000 7fd3d669e640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd3d006cce0 0x7fd3d006d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44108/0 (socket says 172.21.15.81:44108) 2023-11-08T16:30:25.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.427+0000 7fd3d669e640 1 -- 172.21.15.81:0/833813640 learned_addr learned my addr 172.21.15.81:0/833813640 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:25.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.427+0000 7fd3d5e9d640 1 --2- 172.21.15.81:0/833813640 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d00c9ae0 0x7fd3d006f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.427+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816182240 0 0) 0x7fd3d00c9ec0 con 0x7fd3d006cce0 2023-11-08T16:30:25.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.427+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3bc003520 con 0x7fd3d006cce0 2023-11-08T16:30:25.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 42529691 0 0) 0x7fd3bc003520 con 0x7fd3d006cce0 2023-11-08T16:30:25.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d00c9ae0 msgr2=0x7fd3d006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:25.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 --2- 172.21.15.81:0/833813640 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d00c9ae0 0x7fd3d006f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 >> v1:172.21.15.93:6789/0 conn(0x7fd3d006bec0 legacy=0x7fd3d006c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d00cccd0 con 0x7fd3d006cce0 2023-11-08T16:30:25.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fd3c80019f0 con 0x7fd3d006cce0 2023-11-08T16:30:25.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 >> v1:172.21.15.81:6789/0 conn(0x7fd3d006cce0 legacy=0x7fd3d006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 --2- 172.21.15.81:0/833813640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 0x7fd3bc003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 --2- 172.21.15.81:0/833813640 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3bc0047c0 0x7fd3bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 --2- 172.21.15.81:0/833813640 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d00c9ae0 0x7fd3bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d00cc9f0 con 0x7fd3bc003a80 2023-11-08T16:30:25.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc0054e0 con 0x7fd3bc0047c0 2023-11-08T16:30:25.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3bc005650 con 0x7fd3d00c9ae0 2023-11-08T16:30:25.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d5e9d640 1 --2- 172.21.15.81:0/833813640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 0x7fd3bc003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d5e9d640 1 -- 172.21.15.81:0/833813640 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d00c9ae0 msgr2=0x7fd3bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:25.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.428+0000 7fd3d5e9d640 1 --2- 172.21.15.81:0/833813640 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d00c9ae0 0x7fd3bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.429+0000 7fd3d5e9d640 1 -- 172.21.15.81:0/833813640 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3bc0047c0 msgr2=0x7fd3bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:25.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.429+0000 7fd3d569c640 1 --2- 172.21.15.81:0/833813640 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3bc0047c0 0x7fd3bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.429+0000 7fd3d5e9d640 1 --2- 172.21.15.81:0/833813640 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3bc0047c0 0x7fd3bc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.429+0000 7fd3d5e9d640 1 -- 172.21.15.81:0/833813640 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3bc003520 con 0x7fd3bc003a80 2023-11-08T16:30:25.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.430+0000 7fd3d5e9d640 1 --2- 172.21.15.81:0/833813640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 0x7fd3bc003ea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009af0 tx=0x7fd3bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bb06453c4566786 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:25.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.430+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc02c710 con 0x7fd3bc003a80 2023-11-08T16:30:25.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.430+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc02c870 con 0x7fd3bc003a80 2023-11-08T16:30:25.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.430+0000 7fd3d4e9b640 1 -- 172.21.15.81:0/833813640 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc02cc00 con 0x7fd3bc003a80 2023-11-08T16:30:25.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.430+0000 7fd3d8101640 1 -- 172.21.15.81:0/833813640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 msgr2=0x7fd3bc003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.430+0000 7fd3d8101640 1 --2- 172.21.15.81:0/833813640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 0x7fd3bc003ea0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc009af0 tx=0x7fd3bc0057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.430+0000 7fd3d669e640 1 -- 172.21.15.81:0/833813640 reap_dead start 2023-11-08T16:30:25.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.431+0000 7fd3d8101640 1 -- 172.21.15.81:0/833813640 shutdown_connections 2023-11-08T16:30:25.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.431+0000 7fd3d8101640 1 -- 172.21.15.81:0/833813640 >> 172.21.15.81:0/833813640 conn(0x7fd3d00cb490 msgr2=0x7fd3d00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:25.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.431+0000 7fd3d8101640 1 -- 172.21.15.81:0/833813640 shutdown_connections 2023-11-08T16:30:25.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.431+0000 7fd3d8101640 1 -- 172.21.15.81:0/833813640 wait complete. 2023-11-08T16:30:25.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.432+0000 7fd3d8101640 1 Processor -- start 2023-11-08T16:30:25.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.432+0000 7fd3d8101640 1 -- start start 2023-11-08T16:30:25.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.433+0000 7fd3d8101640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 0x7fd3d00afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.433+0000 7fd3d8101640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d013d7f0 0x7fd3d015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.433+0000 7fd3d8101640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d014ed50 0x7fd3d014f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.433+0000 7fd3d8101640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d013b980 con 0x7fd3bc003a80 2023-11-08T16:30:25.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.433+0000 7fd3d8101640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d014f710 con 0x7fd3d013d7f0 2023-11-08T16:30:25.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.433+0000 7fd3d8101640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d013e7b0 con 0x7fd3d014ed50 2023-11-08T16:30:25.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d5e9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 0x7fd3d00afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d5e9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 0x7fd3d00afe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46766/0 (socket says 172.21.15.81:46766) 2023-11-08T16:30:25.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d5e9d640 1 -- 172.21.15.81:0/2893107420 learned_addr learned my addr 172.21.15.81:0/2893107420 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:25.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d669e640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d014ed50 0x7fd3d014f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d5e9d640 1 -- 172.21.15.81:0/2893107420 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d014ed50 msgr2=0x7fd3d014f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d569c640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d013d7f0 0x7fd3d015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d5e9d640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d014ed50 0x7fd3d014f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d5e9d640 1 -- 172.21.15.81:0/2893107420 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d013d7f0 msgr2=0x7fd3d015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d5e9d640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d013d7f0 0x7fd3d015d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d5e9d640 1 -- 172.21.15.81:0/2893107420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d013dc70 con 0x7fd3bc003a80 2023-11-08T16:30:25.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d669e640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d014ed50 0x7fd3d014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:25.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.434+0000 7fd3d5e9d640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 0x7fd3d00afe40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc02c2b0 tx=0x7fd3cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:25.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.435+0000 7fd3c67fc640 1 -- 172.21.15.81:0/2893107420 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc003810 con 0x7fd3bc003a80 2023-11-08T16:30:25.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.435+0000 7fd3c67fc640 1 -- 172.21.15.81:0/2893107420 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc02de90 con 0x7fd3bc003a80 2023-11-08T16:30:25.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.435+0000 7fd3c67fc640 1 -- 172.21.15.81:0/2893107420 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc003b70 con 0x7fd3bc003a80 2023-11-08T16:30:25.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.435+0000 7fd3d8101640 1 -- 172.21.15.81:0/2893107420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d013df10 con 0x7fd3bc003a80 2023-11-08T16:30:25.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.435+0000 7fd3d8101640 1 -- 172.21.15.81:0/2893107420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d013e4d0 con 0x7fd3bc003a80 2023-11-08T16:30:25.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.436+0000 7fd3d8101640 1 -- 172.21.15.81:0/2893107420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d004ece0 con 0x7fd3bc003a80 2023-11-08T16:30:25.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.437+0000 7fd3c67fc640 1 -- 172.21.15.81:0/2893107420 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd3cc003cd0 con 0x7fd3bc003a80 2023-11-08T16:30:25.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.438+0000 7fd3c67fc640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd3b406ae70 0x7fd3b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.438+0000 7fd3c67fc640 1 -- 172.21.15.81:0/2893107420 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd3cc0ab050 con 0x7fd3bc003a80 2023-11-08T16:30:25.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.438+0000 7fd3d569c640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd3b406ae70 0x7fd3b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.439+0000 7fd3d569c640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd3b406ae70 0x7fd3b406d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd3d013b690 tx=0x7fd3c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:25.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.442+0000 7fd3c67fc640 1 -- 172.21.15.81:0/2893107420 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd3cc0765e0 con 0x7fd3bc003a80 2023-11-08T16:30:25.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.684+0000 7fd3d8101640 1 -- 172.21.15.81:0/2893107420 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fd3d00c5850 con 0x7fd3bc003a80 2023-11-08T16:30:25.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.689+0000 7fd3c67fc640 1 -- 172.21.15.81:0/2893107420 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd3cc079c00 con 0x7fd3bc003a80 2023-11-08T16:30:25.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.692+0000 7fd3d8101640 1 -- 172.21.15.81:0/2893107420 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd3b406ae70 msgr2=0x7fd3b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.692+0000 7fd3d8101640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd3b406ae70 0x7fd3b406d330 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd3d013b690 tx=0x7fd3c0009040 comp rx=0 tx=0).stop 2023-11-08T16:30:25.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.692+0000 7fd3d8101640 1 -- 172.21.15.81:0/2893107420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 msgr2=0x7fd3d00afe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.692+0000 7fd3d8101640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 0x7fd3d00afe40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc02c2b0 tx=0x7fd3cc004730 comp rx=0 tx=0).stop 2023-11-08T16:30:25.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.693+0000 7fd3d8101640 1 -- 172.21.15.81:0/2893107420 shutdown_connections 2023-11-08T16:30:25.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.693+0000 7fd3d8101640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd3b406ae70 0x7fd3b406d330 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.693+0000 7fd3d8101640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd3d014ed50 0x7fd3d014f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.693+0000 7fd3d8101640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd3d013d7f0 0x7fd3d015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.693+0000 7fd3d8101640 1 --2- 172.21.15.81:0/2893107420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd3bc003a80 0x7fd3d00afe40 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.693+0000 7fd3d8101640 1 -- 172.21.15.81:0/2893107420 >> 172.21.15.81:0/2893107420 conn(0x7fd3d00cb490 msgr2=0x7fd3d00ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.693+0000 7fd3d8101640 1 -- 172.21.15.81:0/2893107420 shutdown_connections 2023-11-08T16:30:25.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.693+0000 7fd3d8101640 1 -- 172.21.15.81:0/2893107420 wait complete. 2023-11-08T16:30:25.712 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-08T16:30:25.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.919+0000 7f7c84d61640 1 Processor -- start 2023-11-08T16:30:25.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.920+0000 7f7c84d61640 1 -- start start 2023-11-08T16:30:25.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.921+0000 7f7c84d61640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c8015d310 0x7f7c800d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.921+0000 7f7c84d61640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7c800d01d0 0x7f7c800b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.921+0000 7f7c84d61640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c800b6100 con 0x7f7c800d1300 2023-11-08T16:30:25.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.921+0000 7f7c84d61640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c800b6480 con 0x7f7c800d01d0 2023-11-08T16:30:25.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.921+0000 7f7c84d61640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800d1b50 con 0x7f7c8015d310 2023-11-08T16:30:25.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.922+0000 7f7c7e59c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c8015d310 0x7f7c800d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.922+0000 7f7c7ed9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7c800d01d0 0x7f7c800b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.922+0000 7f7c7e59c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c8015d310 0x7f7c800d2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:53358/0 (socket says 172.21.15.81:53358) 2023-11-08T16:30:25.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.922+0000 7f7c7e59c640 1 -- 172.21.15.81:0/937947585 learned_addr learned my addr 172.21.15.81:0/937947585 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:25.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.922+0000 7f7c7e59c640 1 -- 172.21.15.81:0/937947585 >> v2:172.21.15.93:3300/0 conn(0x7f7c800d01d0 msgr2=0x7f7c800b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:25.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.922+0000 7f7c7e59c640 1 --2- 172.21.15.81:0/937947585 >> v2:172.21.15.93:3300/0 conn(0x7f7c800d01d0 0x7f7c800b68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.922+0000 7f7c7e59c640 1 -- 172.21.15.81:0/937947585 >> v1:172.21.15.93:6789/0 conn(0x7f7c800d1300 legacy=0x7f7c800cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.922+0000 7f7c7e59c640 1 -- 172.21.15.81:0/937947585 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c800d1e30 con 0x7f7c8015d310 2023-11-08T16:30:25.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.924+0000 7f7c7e59c640 1 --2- 172.21.15.81:0/937947585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c8015d310 0x7f7c800d2b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009e50 tx=0x7f7c7002bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e8de1488f4beb82 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:25.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.924+0000 7f7c7d59a640 1 -- 172.21.15.81:0/937947585 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7002cb90 con 0x7f7c8015d310 2023-11-08T16:30:25.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.924+0000 7f7c7d59a640 1 -- 172.21.15.81:0/937947585 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c7002ccf0 con 0x7f7c8015d310 2023-11-08T16:30:25.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.924+0000 7f7c7d59a640 1 -- 172.21.15.81:0/937947585 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70034610 con 0x7f7c8015d310 2023-11-08T16:30:25.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.925+0000 7f7c84d61640 1 -- 172.21.15.81:0/937947585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c8015d310 msgr2=0x7f7c800d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.925+0000 7f7c84d61640 1 --2- 172.21.15.81:0/937947585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c8015d310 0x7f7c800d2b70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009e50 tx=0x7f7c7002bff0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.925+0000 7f7c84d61640 1 -- 172.21.15.81:0/937947585 shutdown_connections 2023-11-08T16:30:25.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.925+0000 7f7c84d61640 1 --2- 172.21.15.81:0/937947585 >> v2:172.21.15.93:3300/0 conn(0x7f7c800d01d0 0x7f7c800b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.925+0000 7f7c84d61640 1 --2- 172.21.15.81:0/937947585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c8015d310 0x7f7c800d2b70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.925+0000 7f7c84d61640 1 -- 172.21.15.81:0/937947585 >> 172.21.15.81:0/937947585 conn(0x7f7c8006aac0 msgr2=0x7f7c8006aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:25.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.925+0000 7f7c84d61640 1 -- 172.21.15.81:0/937947585 shutdown_connections 2023-11-08T16:30:25.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.925+0000 7f7c84d61640 1 -- 172.21.15.81:0/937947585 wait complete. 2023-11-08T16:30:25.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.926+0000 7f7c84d61640 1 Processor -- start 2023-11-08T16:30:25.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.927+0000 7f7c84d61640 1 -- start start 2023-11-08T16:30:25.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c84d61640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c800d01d0 0x7f7c8015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c84d61640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c800d1300 0x7f7c800cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c84d61640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c8015d310 0x7f7c800cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c84d61640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c801f5f80 con 0x7f7c800d1300 2023-11-08T16:30:25.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c84d61640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c801f6de0 con 0x7f7c8015d310 2023-11-08T16:30:25.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c84d61640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8015fd20 con 0x7f7c800d01d0 2023-11-08T16:30:25.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c7dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c800d1300 0x7f7c800cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c7dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c800d1300 0x7f7c800cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46778/0 (socket says 172.21.15.81:46778) 2023-11-08T16:30:25.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c7dd9b640 1 -- 172.21.15.81:0/3223936344 learned_addr learned my addr 172.21.15.81:0/3223936344 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:25.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c7e59c640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c800d01d0 0x7f7c8015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c7dd9b640 1 -- 172.21.15.81:0/3223936344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c800d01d0 msgr2=0x7f7c8015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c7dd9b640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c800d01d0 0x7f7c8015f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c7dd9b640 1 -- 172.21.15.81:0/3223936344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c8015d310 msgr2=0x7f7c800cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:25.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c7dd9b640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c8015d310 0x7f7c800cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:25.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c7dd9b640 1 -- 172.21.15.81:0/3223936344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c70009b30 con 0x7f7c800d1300 2023-11-08T16:30:25.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.928+0000 7f7c7e59c640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c800d01d0 0x7f7c8015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:25.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.929+0000 7f7c7dd9b640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c800d1300 0x7f7c800cb5f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800c480 tx=0x7f7c6800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:25.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.929+0000 7f7c76ffd640 1 -- 172.21.15.81:0/3223936344 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c680052d0 con 0x7f7c800d1300 2023-11-08T16:30:25.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.929+0000 7f7c76ffd640 1 -- 172.21.15.81:0/3223936344 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c68005430 con 0x7f7c800d1300 2023-11-08T16:30:25.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.929+0000 7f7c76ffd640 1 -- 172.21.15.81:0/3223936344 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68012870 con 0x7f7c800d1300 2023-11-08T16:30:25.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.929+0000 7f7c84d61640 1 -- 172.21.15.81:0/3223936344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c801f61d0 con 0x7f7c800d1300 2023-11-08T16:30:25.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.929+0000 7f7c84d61640 1 -- 172.21.15.81:0/3223936344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c801f6790 con 0x7f7c800d1300 2023-11-08T16:30:25.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.930+0000 7f7c84d61640 1 -- 172.21.15.81:0/3223936344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c800743f0 con 0x7f7c800d1300 2023-11-08T16:30:25.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.931+0000 7f7c76ffd640 1 -- 172.21.15.81:0/3223936344 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c680129d0 con 0x7f7c800d1300 2023-11-08T16:30:25.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.932+0000 7f7c76ffd640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c6006ae70 0x7f7c6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:25.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.932+0000 7f7c76ffd640 1 -- 172.21.15.81:0/3223936344 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7c68088e90 con 0x7f7c800d1300 2023-11-08T16:30:25.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.932+0000 7f7c7e59c640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c6006ae70 0x7f7c6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:25.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.933+0000 7f7c7e59c640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c6006ae70 0x7f7c6006d330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c70035000 tx=0x7f7c70030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:25.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:25.936+0000 7f7c76ffd640 1 -- 172.21.15.81:0/3223936344 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c68054280 con 0x7f7c800d1300 2023-11-08T16:30:26.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.180+0000 7f7c84d61640 1 -- 172.21.15.81:0/3223936344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f7c800b6a40 con 0x7f7c800d1300 2023-11-08T16:30:26.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.184+0000 7f7c76ffd640 1 -- 172.21.15.81:0/3223936344 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7c6808a0a0 con 0x7f7c800d1300 2023-11-08T16:30:26.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 -- 172.21.15.81:0/3223936344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c6006ae70 msgr2=0x7f7c6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c6006ae70 0x7f7c6006d330 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7c70035000 tx=0x7f7c70030000 comp rx=0 tx=0).stop 2023-11-08T16:30:26.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 -- 172.21.15.81:0/3223936344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c800d1300 msgr2=0x7f7c800cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c800d1300 0x7f7c800cb5f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f7c6800c480 tx=0x7f7c6800c940 comp rx=0 tx=0).stop 2023-11-08T16:30:26.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 -- 172.21.15.81:0/3223936344 shutdown_connections 2023-11-08T16:30:26.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c6006ae70 0x7f7c6006d330 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c8015d310 0x7f7c800cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c800d1300 0x7f7c800cb5f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 --2- 172.21.15.81:0/3223936344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c800d01d0 0x7f7c8015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 -- 172.21.15.81:0/3223936344 >> 172.21.15.81:0/3223936344 conn(0x7f7c8006aac0 msgr2=0x7f7c800caa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:26.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 -- 172.21.15.81:0/3223936344 shutdown_connections 2023-11-08T16:30:26.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.187+0000 7f7c84d61640 1 -- 172.21.15.81:0/3223936344 wait complete. 2023-11-08T16:30:26.207 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:30:26.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.410+0000 7f0791425640 1 Processor -- start 2023-11-08T16:30:26.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.410+0000 7f0791425640 1 -- start start 2023-11-08T16:30:26.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.411+0000 7f0791425640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f078c0ced10 0x7f078c0cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.411+0000 7f0791425640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f078c15aea0 0x7f078c15b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.411+0000 7f0791425640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078c0cf6b0 con 0x7f078c15aac0 2023-11-08T16:30:26.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.411+0000 7f0791425640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078c06f650 con 0x7f078c15aea0 2023-11-08T16:30:26.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.411+0000 7f0791425640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f078c070470 con 0x7f078c0ced10 2023-11-08T16:30:26.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.412+0000 7f078bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f078c15aac0 0x7f078c0711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44122/0 (socket says 172.21.15.81:44122) 2023-11-08T16:30:26.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.412+0000 7f078bfff640 1 -- 172.21.15.81:0/258238587 learned_addr learned my addr 172.21.15.81:0/258238587 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:26.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.412+0000 7f0790c24640 1 --2- 172.21.15.81:0/258238587 >> v2:172.21.15.81:3300/0 conn(0x7f078c15aea0 0x7f078c15b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:26.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.412+0000 7f0783fff640 1 --2- 172.21.15.81:0/258238587 >> v2:172.21.15.93:3300/0 conn(0x7f078c0ced10 0x7f078c0cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:26.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.412+0000 7f078b7fe640 1 -- 172.21.15.81:0/258238587 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955115553 0 0) 0x7f078c0cf6b0 con 0x7f078c15aac0 2023-11-08T16:30:26.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.412+0000 7f078b7fe640 1 -- 172.21.15.81:0/258238587 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0778003520 con 0x7f078c15aac0 2023-11-08T16:30:26.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.412+0000 7f0790c24640 1 -- 172.21.15.81:0/258238587 >> v2:172.21.15.93:3300/0 conn(0x7f078c0ced10 msgr2=0x7f078c0cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:26.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.412+0000 7f0790c24640 1 --2- 172.21.15.81:0/258238587 >> v2:172.21.15.93:3300/0 conn(0x7f078c0ced10 0x7f078c0cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.412+0000 7f0790c24640 1 -- 172.21.15.81:0/258238587 >> v1:172.21.15.81:6789/0 conn(0x7f078c15aac0 legacy=0x7f078c0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.412+0000 7f0790c24640 1 -- 172.21.15.81:0/258238587 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f078c06d580 con 0x7f078c15aea0 2023-11-08T16:30:26.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.413+0000 7f0790c24640 1 --2- 172.21.15.81:0/258238587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f078c15aea0 0x7f078c15b2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f078c15e790 tx=0x7f078402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e843f4d1ed6ca6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:26.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.413+0000 7f078b7fe640 1 -- 172.21.15.81:0/258238587 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078402cc70 con 0x7f078c15aea0 2023-11-08T16:30:26.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.413+0000 7f078b7fe640 1 -- 172.21.15.81:0/258238587 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078402cdd0 con 0x7f078c15aea0 2023-11-08T16:30:26.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.414+0000 7f078b7fe640 1 -- 172.21.15.81:0/258238587 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0784034660 con 0x7f078c15aea0 2023-11-08T16:30:26.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.414+0000 7f0791425640 1 -- 172.21.15.81:0/258238587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f078c15aea0 msgr2=0x7f078c15b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.414+0000 7f0791425640 1 --2- 172.21.15.81:0/258238587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f078c15aea0 0x7f078c15b2f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f078c15e790 tx=0x7f078402bec0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.414+0000 7f0791425640 1 -- 172.21.15.81:0/258238587 shutdown_connections 2023-11-08T16:30:26.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.414+0000 7f0791425640 1 --2- 172.21.15.81:0/258238587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f078c15aea0 0x7f078c15b2f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.414+0000 7f0791425640 1 --2- 172.21.15.81:0/258238587 >> v2:172.21.15.93:3300/0 conn(0x7f078c0ced10 0x7f078c0cf170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.414+0000 7f0791425640 1 -- 172.21.15.81:0/258238587 >> 172.21.15.81:0/258238587 conn(0x7f078c06a4a0 msgr2=0x7f078c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:26.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.414+0000 7f0791425640 1 -- 172.21.15.81:0/258238587 shutdown_connections 2023-11-08T16:30:26.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.414+0000 7f0791425640 1 -- 172.21.15.81:0/258238587 wait complete. 2023-11-08T16:30:26.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.416+0000 7f0791425640 1 Processor -- start 2023-11-08T16:30:26.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.416+0000 7f0791425640 1 -- start start 2023-11-08T16:30:26.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f0791425640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f078c0ced10 0x7f078c154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f0791425640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f078c15aac0 0x7f078c1484e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f0791425640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f078c15aea0 0x7f078c0a8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f0791425640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c151c70 con 0x7f078c0ced10 2023-11-08T16:30:26.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f0791425640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078c1535d0 con 0x7f078c15aac0 2023-11-08T16:30:26.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f0791425640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f078c1538b0 con 0x7f078c15aea0 2023-11-08T16:30:26.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f078bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f078c0ced10 0x7f078c154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:26.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f078bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f078c0ced10 0x7f078c154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46808/0 (socket says 172.21.15.81:46808) 2023-11-08T16:30:26.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f078bfff640 1 -- 172.21.15.81:0/857372728 learned_addr learned my addr 172.21.15.81:0/857372728 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:26.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f078bfff640 1 -- 172.21.15.81:0/857372728 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f078c15aea0 msgr2=0x7f078c0a8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f078bfff640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f078c15aea0 0x7f078c0a8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f078bfff640 1 -- 172.21.15.81:0/857372728 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f078c15aac0 msgr2=0x7f078c1484e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f078bfff640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f078c15aac0 0x7f078c1484e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.417+0000 7f078bfff640 1 -- 172.21.15.81:0/857372728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784009a70 con 0x7f078c0ced10 2023-11-08T16:30:26.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.418+0000 7f078bfff640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f078c0ced10 0x7f078c154670 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f077c00c500 tx=0x7f077c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:26.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.418+0000 7f07897fa640 1 -- 172.21.15.81:0/857372728 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c005ce0 con 0x7f078c0ced10 2023-11-08T16:30:26.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.418+0000 7f07897fa640 1 -- 172.21.15.81:0/857372728 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077c005e40 con 0x7f078c0ced10 2023-11-08T16:30:26.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.418+0000 7f07897fa640 1 -- 172.21.15.81:0/857372728 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077c007330 con 0x7f078c0ced10 2023-11-08T16:30:26.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.418+0000 7f0791425640 1 -- 172.21.15.81:0/857372728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f078c153b90 con 0x7f078c0ced10 2023-11-08T16:30:26.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.418+0000 7f0791425640 1 -- 172.21.15.81:0/857372728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078c154150 con 0x7f078c0ced10 2023-11-08T16:30:26.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.419+0000 7f0791425640 1 -- 172.21.15.81:0/857372728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078c154340 con 0x7f078c0ced10 2023-11-08T16:30:26.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.420+0000 7f07897fa640 1 -- 172.21.15.81:0/857372728 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f077c007550 con 0x7f078c0ced10 2023-11-08T16:30:26.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.421+0000 7f07897fa640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f076c06ae70 0x7f076c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.421+0000 7f07897fa640 1 -- 172.21.15.81:0/857372728 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f077c088970 con 0x7f078c0ced10 2023-11-08T16:30:26.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.421+0000 7f0783fff640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f076c06ae70 0x7f076c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:26.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.422+0000 7f0783fff640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f076c06ae70 0x7f076c06d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f078c154c40 tx=0x7f0774008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:26.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.425+0000 7f07897fa640 1 -- 172.21.15.81:0/857372728 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f077c053e20 con 0x7f078c0ced10 2023-11-08T16:30:26.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.614+0000 7f0791425640 1 -- 172.21.15.81:0/857372728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f078c067800 con 0x7f078c0ced10 2023-11-08T16:30:26.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.616+0000 7f07897fa640 1 -- 172.21.15.81:0/857372728 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12797 (secure 0 0 0) 0x7f077c017020 con 0x7f078c0ced10 2023-11-08T16:30:26.618 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:26.618 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":31,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:30:22.629674+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:30:26.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.618+0000 7f0791425640 1 -- 172.21.15.81:0/857372728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f076c06ae70 msgr2=0x7f076c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.618+0000 7f0791425640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f076c06ae70 0x7f076c06d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f078c154c40 tx=0x7f0774008040 comp rx=0 tx=0).stop 2023-11-08T16:30:26.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.618+0000 7f0791425640 1 -- 172.21.15.81:0/857372728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f078c0ced10 msgr2=0x7f078c154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.618+0000 7f0791425640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f078c0ced10 0x7f078c154670 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f077c00c500 tx=0x7f077c00c9c0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.619+0000 7f0791425640 1 -- 172.21.15.81:0/857372728 shutdown_connections 2023-11-08T16:30:26.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.619+0000 7f0791425640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f076c06ae70 0x7f076c06d330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.619+0000 7f0791425640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f078c15aea0 0x7f078c0a8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.619+0000 7f0791425640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f078c15aac0 0x7f078c1484e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.619+0000 7f0791425640 1 --2- 172.21.15.81:0/857372728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f078c0ced10 0x7f078c154670 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.619+0000 7f0791425640 1 -- 172.21.15.81:0/857372728 >> 172.21.15.81:0/857372728 conn(0x7f078c06a4a0 msgr2=0x7f078c0cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:26.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.619+0000 7f0791425640 1 -- 172.21.15.81:0/857372728 shutdown_connections 2023-11-08T16:30:26.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.619+0000 7f0791425640 1 -- 172.21.15.81:0/857372728 wait complete. 2023-11-08T16:30:26.640 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:30:26.641 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:30:26.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.854+0000 7f2e26b1d640 1 Processor -- start 2023-11-08T16:30:26.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.855+0000 7f2e26b1d640 1 -- start start 2023-11-08T16:30:26.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.856+0000 7f2e26b1d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e2015bfc0 0x7f2e200caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.856+0000 7f2e26b1d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2e2015f280 0x7f2e201516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.856+0000 7f2e26b1d640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e2006fa30 con 0x7f2e200ce0d0 2023-11-08T16:30:26.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.856+0000 7f2e26b1d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e200cda10 con 0x7f2e2015f280 2023-11-08T16:30:26.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.856+0000 7f2e26b1d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e200ce4b0 con 0x7f2e2015bfc0 2023-11-08T16:30:26.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.856+0000 7f2e250ba640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2e2015f280 0x7f2e201516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:26.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.856+0000 7f2e248b9640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2e200ce0d0 0x7f2e2015eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44134/0 (socket says 172.21.15.81:44134) 2023-11-08T16:30:26.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.856+0000 7f2e248b9640 1 -- 172.21.15.81:0/315178026 learned_addr learned my addr 172.21.15.81:0/315178026 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:26.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.857+0000 7f2e1ffff640 1 --2- 172.21.15.81:0/315178026 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e2015bfc0 0x7f2e200caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:26.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.857+0000 7f2e1f7fe640 1 -- 172.21.15.81:0/315178026 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2379087787 0 0) 0x7f2e2006fa30 con 0x7f2e200ce0d0 2023-11-08T16:30:26.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.857+0000 7f2e1f7fe640 1 -- 172.21.15.81:0/315178026 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e0c003520 con 0x7f2e200ce0d0 2023-11-08T16:30:26.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.857+0000 7f2e250ba640 1 -- 172.21.15.81:0/315178026 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e2015bfc0 msgr2=0x7f2e200caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:26.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.857+0000 7f2e250ba640 1 --2- 172.21.15.81:0/315178026 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e2015bfc0 0x7f2e200caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.857+0000 7f2e250ba640 1 -- 172.21.15.81:0/315178026 >> v1:172.21.15.81:6789/0 conn(0x7f2e200ce0d0 legacy=0x7f2e2015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.857+0000 7f2e250ba640 1 -- 172.21.15.81:0/315178026 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e2006fca0 con 0x7f2e2015f280 2023-11-08T16:30:26.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.858+0000 7f2e250ba640 1 --2- 172.21.15.81:0/315178026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2015f280 0x7f2e201516e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009d90 tx=0x7f2e1802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935b7702665910eb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:26.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.858+0000 7f2e1f7fe640 1 -- 172.21.15.81:0/315178026 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e1802cc60 con 0x7f2e2015f280 2023-11-08T16:30:26.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.858+0000 7f2e1f7fe640 1 -- 172.21.15.81:0/315178026 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1802cdc0 con 0x7f2e2015f280 2023-11-08T16:30:26.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.858+0000 7f2e1f7fe640 1 -- 172.21.15.81:0/315178026 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18034660 con 0x7f2e2015f280 2023-11-08T16:30:26.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.859+0000 7f2e26b1d640 1 -- 172.21.15.81:0/315178026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2015f280 msgr2=0x7f2e201516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.859+0000 7f2e26b1d640 1 --2- 172.21.15.81:0/315178026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2015f280 0x7f2e201516e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009d90 tx=0x7f2e1802beb0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.859+0000 7f2e26b1d640 1 -- 172.21.15.81:0/315178026 shutdown_connections 2023-11-08T16:30:26.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.859+0000 7f2e26b1d640 1 --2- 172.21.15.81:0/315178026 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2015f280 0x7f2e201516e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.859+0000 7f2e26b1d640 1 --2- 172.21.15.81:0/315178026 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e2015bfc0 0x7f2e200caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.859+0000 7f2e26b1d640 1 -- 172.21.15.81:0/315178026 >> 172.21.15.81:0/315178026 conn(0x7f2e200cb580 msgr2=0x7f2e200cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:26.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.859+0000 7f2e26b1d640 1 -- 172.21.15.81:0/315178026 shutdown_connections 2023-11-08T16:30:26.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.859+0000 7f2e26b1d640 1 -- 172.21.15.81:0/315178026 wait complete. 2023-11-08T16:30:26.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.861+0000 7f2e26b1d640 1 Processor -- start 2023-11-08T16:30:26.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.861+0000 7f2e26b1d640 1 -- start start 2023-11-08T16:30:26.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e26b1d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e200ce0d0 0x7f2e201544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e26b1d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e2015bfc0 0x7f2e200c6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e26b1d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2015f280 0x7f2e201f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e26b1d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e201f64f0 con 0x7f2e2015f280 2023-11-08T16:30:26.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e26b1d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e201f67d0 con 0x7f2e200ce0d0 2023-11-08T16:30:26.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e26b1d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e201f6ab0 con 0x7f2e2015bfc0 2023-11-08T16:30:26.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e250ba640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2015f280 0x7f2e201f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:26.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e250ba640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2015f280 0x7f2e201f2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46832/0 (socket says 172.21.15.81:46832) 2023-11-08T16:30:26.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e250ba640 1 -- 172.21.15.81:0/1443696586 learned_addr learned my addr 172.21.15.81:0/1443696586 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:26.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e250ba640 1 -- 172.21.15.81:0/1443696586 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e2015bfc0 msgr2=0x7f2e200c6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e250ba640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e2015bfc0 0x7f2e200c6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e250ba640 1 -- 172.21.15.81:0/1443696586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e200ce0d0 msgr2=0x7f2e201544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:26.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e250ba640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e200ce0d0 0x7f2e201544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:26.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.862+0000 7f2e250ba640 1 -- 172.21.15.81:0/1443696586 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18009a70 con 0x7f2e2015f280 2023-11-08T16:30:26.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.863+0000 7f2e250ba640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2015f280 0x7f2e201f2fc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009d90 tx=0x7f2e18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:26.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.863+0000 7f2e1d7fa640 1 -- 172.21.15.81:0/1443696586 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18040650 con 0x7f2e2015f280 2023-11-08T16:30:26.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.863+0000 7f2e1d7fa640 1 -- 172.21.15.81:0/1443696586 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e180407b0 con 0x7f2e2015f280 2023-11-08T16:30:26.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.863+0000 7f2e1d7fa640 1 -- 172.21.15.81:0/1443696586 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18040b60 con 0x7f2e2015f280 2023-11-08T16:30:26.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.863+0000 7f2e26b1d640 1 -- 172.21.15.81:0/1443696586 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e201f6d90 con 0x7f2e2015f280 2023-11-08T16:30:26.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.863+0000 7f2e26b1d640 1 -- 172.21.15.81:0/1443696586 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e201549f0 con 0x7f2e2015f280 2023-11-08T16:30:26.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.864+0000 7f2e26b1d640 1 -- 172.21.15.81:0/1443696586 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e201f2ce0 con 0x7f2e2015f280 2023-11-08T16:30:26.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.865+0000 7f2e1d7fa640 1 -- 172.21.15.81:0/1443696586 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2e18040cc0 con 0x7f2e2015f280 2023-11-08T16:30:26.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.866+0000 7f2e1d7fa640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0006ae70 0x7f2e0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:26.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.866+0000 7f2e1d7fa640 1 -- 172.21.15.81:0/1443696586 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2e180ac0e0 con 0x7f2e2015f280 2023-11-08T16:30:26.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.866+0000 7f2e248b9640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0006ae70 0x7f2e0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.867+0000 7f2e248b9640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0006ae70 0x7f2e0006d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2e10001570 tx=0x7f2e10007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:26.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:26.871+0000 7f2e1d7fa640 1 -- 172.21.15.81:0/1443696586 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2e201f2ce0 con 0x7f2e2015f280 2023-11-08T16:30:27.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.045+0000 7f2e26b1d640 1 -- 172.21.15.81:0/1443696586 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2e201f2ce0 con 0x7f2e2015f280 2023-11-08T16:30:27.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.046+0000 7f2e1d7fa640 1 -- 172.21.15.81:0/1443696586 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2e201f2ce0 con 0x7f2e2015f280 2023-11-08T16:30:27.048 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:30:27.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.048+0000 7f2e26b1d640 1 -- 172.21.15.81:0/1443696586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0006ae70 msgr2=0x7f2e0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.048+0000 7f2e26b1d640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0006ae70 0x7f2e0006d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2e10001570 tx=0x7f2e10007930 comp rx=0 tx=0).stop 2023-11-08T16:30:27.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.048+0000 7f2e26b1d640 1 -- 172.21.15.81:0/1443696586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2015f280 msgr2=0x7f2e201f2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.048+0000 7f2e26b1d640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2015f280 0x7f2e201f2fc0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f2e18009d90 tx=0x7f2e18004a40 comp rx=0 tx=0).stop 2023-11-08T16:30:27.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.049+0000 7f2e26b1d640 1 -- 172.21.15.81:0/1443696586 shutdown_connections 2023-11-08T16:30:27.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.049+0000 7f2e26b1d640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2e0006ae70 0x7f2e0006d330 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.049+0000 7f2e26b1d640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2e2015f280 0x7f2e201f2fc0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.049+0000 7f2e26b1d640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2e2015bfc0 0x7f2e200c6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.049+0000 7f2e26b1d640 1 --2- 172.21.15.81:0/1443696586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e200ce0d0 0x7f2e201544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.049+0000 7f2e26b1d640 1 -- 172.21.15.81:0/1443696586 >> 172.21.15.81:0/1443696586 conn(0x7f2e200cb580 msgr2=0x7f2e2006bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:27.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.049+0000 7f2e26b1d640 1 -- 172.21.15.81:0/1443696586 shutdown_connections 2023-11-08T16:30:27.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.049+0000 7f2e26b1d640 1 -- 172.21.15.81:0/1443696586 wait complete. 2023-11-08T16:30:27.069 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T16:30:27.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.275+0000 7f099ad78640 1 Processor -- start 2023-11-08T16:30:27.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.275+0000 7f099ad78640 1 -- start start 2023-11-08T16:30:27.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.276+0000 7f099ad78640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f09940d2b70 0x7f09940cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.276+0000 7f099ad78640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f09940d0000 0x7f09940b68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.276+0000 7f099ad78640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09940b6100 con 0x7f099415d310 2023-11-08T16:30:27.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.276+0000 7f099ad78640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09940b6480 con 0x7f09940d2b70 2023-11-08T16:30:27.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.276+0000 7f099ad78640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09940d1b50 con 0x7f09940d0000 2023-11-08T16:30:27.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.277+0000 7f098bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f09940d2b70 0x7f09940cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.277+0000 7f098bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f09940d2b70 0x7f09940cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46838/0 (socket says 172.21.15.81:46838) 2023-11-08T16:30:27.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.277+0000 7f098bfff640 1 -- 172.21.15.81:0/370996168 learned_addr learned my addr 172.21.15.81:0/370996168 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:27.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.277+0000 7f0999315640 1 --2- 172.21.15.81:0/370996168 >> v2:172.21.15.93:3300/0 conn(0x7f09940d0000 0x7f09940b68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.277+0000 7f098bfff640 1 -- 172.21.15.81:0/370996168 >> v2:172.21.15.93:3300/0 conn(0x7f09940d0000 msgr2=0x7f09940b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:27.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.277+0000 7f098bfff640 1 --2- 172.21.15.81:0/370996168 >> v2:172.21.15.93:3300/0 conn(0x7f09940d0000 0x7f09940b68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.277+0000 7f098bfff640 1 -- 172.21.15.81:0/370996168 >> v1:172.21.15.93:6789/0 conn(0x7f099415d310 legacy=0x7f09940d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.277+0000 7f098bfff640 1 -- 172.21.15.81:0/370996168 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09940d1e30 con 0x7f09940d2b70 2023-11-08T16:30:27.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.278+0000 7f098bfff640 1 --2- 172.21.15.81:0/370996168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09940d2b70 0x7f09940cfac0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f099408b1c0 tx=0x7f098402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a792c22089d7ad1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:27.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.279+0000 7f098b7fe640 1 -- 172.21.15.81:0/370996168 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098402f850 con 0x7f09940d2b70 2023-11-08T16:30:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.279+0000 7f098b7fe640 1 -- 172.21.15.81:0/370996168 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098402f9b0 con 0x7f09940d2b70 2023-11-08T16:30:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.279+0000 7f098b7fe640 1 -- 172.21.15.81:0/370996168 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098402fd40 con 0x7f09940d2b70 2023-11-08T16:30:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.279+0000 7f099ad78640 1 -- 172.21.15.81:0/370996168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09940d2b70 msgr2=0x7f09940cfac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.279+0000 7f099ad78640 1 --2- 172.21.15.81:0/370996168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09940d2b70 0x7f09940cfac0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f099408b1c0 tx=0x7f098402ba30 comp rx=0 tx=0).stop 2023-11-08T16:30:27.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.279+0000 7f099ad78640 1 -- 172.21.15.81:0/370996168 shutdown_connections 2023-11-08T16:30:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.279+0000 7f099ad78640 1 --2- 172.21.15.81:0/370996168 >> v2:172.21.15.93:3300/0 conn(0x7f09940d0000 0x7f09940b68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.279+0000 7f099ad78640 1 --2- 172.21.15.81:0/370996168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09940d2b70 0x7f09940cfac0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.279+0000 7f099ad78640 1 -- 172.21.15.81:0/370996168 >> 172.21.15.81:0/370996168 conn(0x7f099406aac0 msgr2=0x7f099406aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.279+0000 7f099ad78640 1 -- 172.21.15.81:0/370996168 shutdown_connections 2023-11-08T16:30:27.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.279+0000 7f099ad78640 1 -- 172.21.15.81:0/370996168 wait complete. 2023-11-08T16:30:27.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.281+0000 7f099ad78640 1 Processor -- start 2023-11-08T16:30:27.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.281+0000 7f099ad78640 1 -- start start 2023-11-08T16:30:27.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.282+0000 7f099ad78640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09940d0000 0x7f09940c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.282+0000 7f099ad78640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09940d2b70 0x7f0994158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.282+0000 7f099ad78640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f099415d310 0x7f09940c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.282+0000 7f099ad78640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09940c6160 con 0x7f099415d310 2023-11-08T16:30:27.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.282+0000 7f099ad78640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0994157b50 con 0x7f09940d0000 2023-11-08T16:30:27.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.282+0000 7f099ad78640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09941589b0 con 0x7f09940d2b70 2023-11-08T16:30:27.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.282+0000 7f0999315640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f099415d310 0x7f09940c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.282+0000 7f0999315640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f099415d310 0x7f09940c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46852/0 (socket says 172.21.15.81:46852) 2023-11-08T16:30:27.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.282+0000 7f0999315640 1 -- 172.21.15.81:0/642435534 learned_addr learned my addr 172.21.15.81:0/642435534 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:27.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f0998b14640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09940d0000 0x7f09940c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f098bfff640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09940d2b70 0x7f0994158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f0999315640 1 -- 172.21.15.81:0/642435534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09940d2b70 msgr2=0x7f0994158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f0999315640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09940d2b70 0x7f0994158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f0999315640 1 -- 172.21.15.81:0/642435534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09940d0000 msgr2=0x7f09940c71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f0999315640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09940d0000 0x7f09940c71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f0999315640 1 -- 172.21.15.81:0/642435534 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840094d0 con 0x7f099415d310 2023-11-08T16:30:27.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f098bfff640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09940d2b70 0x7f0994158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:27.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f0999315640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f099415d310 0x7f09940c9590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f098c007e90 tx=0x7f098c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:27.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f09897fa640 1 -- 172.21.15.81:0/642435534 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c014420 con 0x7f099415d310 2023-11-08T16:30:27.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f09897fa640 1 -- 172.21.15.81:0/642435534 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f098c014580 con 0x7f099415d310 2023-11-08T16:30:27.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f09897fa640 1 -- 172.21.15.81:0/642435534 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f098c014930 con 0x7f099415d310 2023-11-08T16:30:27.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.283+0000 7f099ad78640 1 -- 172.21.15.81:0/642435534 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0994157da0 con 0x7f099415d310 2023-11-08T16:30:27.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.284+0000 7f099ad78640 1 -- 172.21.15.81:0/642435534 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0994158360 con 0x7f099415d310 2023-11-08T16:30:27.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.285+0000 7f099ad78640 1 -- 172.21.15.81:0/642435534 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09940743f0 con 0x7f099415d310 2023-11-08T16:30:27.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.285+0000 7f09897fa640 1 -- 172.21.15.81:0/642435534 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f098c014a90 con 0x7f099415d310 2023-11-08T16:30:27.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.286+0000 7f09897fa640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f098006ae70 0x7f098006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.286+0000 7f09897fa640 1 -- 172.21.15.81:0/642435534 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f098c089c40 con 0x7f099415d310 2023-11-08T16:30:27.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.287+0000 7f0998b14640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f098006ae70 0x7f098006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.287+0000 7f0998b14640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f098006ae70 0x7f098006d330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f099415def0 tx=0x7f0990009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:27.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.290+0000 7f09897fa640 1 -- 172.21.15.81:0/642435534 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f098c054fa0 con 0x7f099415d310 2023-11-08T16:30:27.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.481+0000 7f099ad78640 1 -- 172.21.15.81:0/642435534 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0994158af0 con 0x7f099415d310 2023-11-08T16:30:27.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.482+0000 7f09897fa640 1 -- 172.21.15.81:0/642435534 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7f098c0585c0 con 0x7f099415d310 2023-11-08T16:30:27.484 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:27.485 INFO:teuthology.orchestra.run.smithi081.stdout:[ 2023-11-08T16:30:27.485 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:30:27.485 INFO:teuthology.orchestra.run.smithi081.stdout: "poolnum": 1, 2023-11-08T16:30:27.485 INFO:teuthology.orchestra.run.smithi081.stdout: "poolname": ".mgr" 2023-11-08T16:30:27.485 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:27.486 INFO:teuthology.orchestra.run.smithi081.stdout:] 2023-11-08T16:30:27.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 -- 172.21.15.81:0/642435534 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f098006ae70 msgr2=0x7f098006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f098006ae70 0x7f098006d330 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f099415def0 tx=0x7f0990009040 comp rx=0 tx=0).stop 2023-11-08T16:30:27.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 -- 172.21.15.81:0/642435534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f099415d310 msgr2=0x7f09940c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f099415d310 0x7f09940c9590 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f098c007e90 tx=0x7f098c00e4b0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 -- 172.21.15.81:0/642435534 shutdown_connections 2023-11-08T16:30:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f098006ae70 0x7f098006d330 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f099415d310 0x7f09940c9590 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09940d2b70 0x7f0994158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 --2- 172.21.15.81:0/642435534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09940d0000 0x7f09940c71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 -- 172.21.15.81:0/642435534 >> 172.21.15.81:0/642435534 conn(0x7f099406aac0 msgr2=0x7f099415b620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:27.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 -- 172.21.15.81:0/642435534 shutdown_connections 2023-11-08T16:30:27.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.485+0000 7f099ad78640 1 -- 172.21.15.81:0/642435534 wait complete. 2023-11-08T16:30:27.506 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T16:30:27.507 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T16:30:27.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.722+0000 7f193ca0e640 1 Processor -- start 2023-11-08T16:30:27.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.722+0000 7f193ca0e640 1 -- start start 2023-11-08T16:30:27.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.724+0000 7f193ca0e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f19380714b0 0x7f193815a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.724+0000 7f193ca0e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1938071890 0x7f193815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.724+0000 7f193ca0e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f193815b340 con 0x7f19380adf00 2023-11-08T16:30:27.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.724+0000 7f193ca0e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f193806b250 con 0x7f19380714b0 2023-11-08T16:30:27.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.724+0000 7f193ca0e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19380b09f0 con 0x7f1938071890 2023-11-08T16:30:27.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.724+0000 7f1936d9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f19380adf00 0x7f19380c90b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44142/0 (socket says 172.21.15.81:44142) 2023-11-08T16:30:27.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.724+0000 7f1936d9d640 1 -- 172.21.15.81:0/2801744962 learned_addr learned my addr 172.21.15.81:0/2801744962 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:27.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f192ffff640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1938071890 0x7f193815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185054905 0 0) 0x7f193815b340 con 0x7f19380adf00 2023-11-08T16:30:27.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1924003520 con 0x7f19380adf00 2023-11-08T16:30:27.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f193659c640 1 --2- 172.21.15.81:0/2801744962 >> v2:172.21.15.93:3300/0 conn(0x7f19380714b0 0x7f193815a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 903696884 0 0) 0x7f1924003520 con 0x7f19380adf00 2023-11-08T16:30:27.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1938071890 msgr2=0x7f193815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:27.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1938071890 0x7f193815ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 >> v2:172.21.15.93:3300/0 conn(0x7f19380714b0 msgr2=0x7f193815a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:27.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 --2- 172.21.15.81:0/2801744962 >> v2:172.21.15.93:3300/0 conn(0x7f19380714b0 0x7f193815a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19380ca360 con 0x7f19380adf00 2023-11-08T16:30:27.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f1930001a10 con 0x7f19380adf00 2023-11-08T16:30:27.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 >> v1:172.21.15.81:6789/0 conn(0x7f19380adf00 legacy=0x7f19380c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1924003970 0x7f1924003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240047c0 0x7f1924004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1938071890 0x7f1924005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19380b09f0 con 0x7f19240047c0 2023-11-08T16:30:27.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f192400b9b0 con 0x7f1924003970 2023-11-08T16:30:27.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f192400bb70 con 0x7f1938071890 2023-11-08T16:30:27.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.725+0000 7f192ffff640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240047c0 0x7f1924004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.726+0000 7f193659c640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1924003970 0x7f1924003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.726+0000 7f192ffff640 1 -- 172.21.15.81:0/2801744962 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1938071890 msgr2=0x7f1924005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:27.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.726+0000 7f1936d9d640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1938071890 0x7f1924005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.726+0000 7f192ffff640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1938071890 0x7f1924005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.726+0000 7f192ffff640 1 -- 172.21.15.81:0/2801744962 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1924003970 msgr2=0x7f1924003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:27.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.726+0000 7f192ffff640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1924003970 0x7f1924003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.726+0000 7f192ffff640 1 -- 172.21.15.81:0/2801744962 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1924003520 con 0x7f19240047c0 2023-11-08T16:30:27.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.727+0000 7f192ffff640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240047c0 0x7f1924004c40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f192000ba00 tx=0x7f19240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc6c938b86e6dde2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:27.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.727+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920032620 con 0x7f19240047c0 2023-11-08T16:30:27.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.727+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1920032780 con 0x7f19240047c0 2023-11-08T16:30:27.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.727+0000 7f1935d9b640 1 -- 172.21.15.81:0/2801744962 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920032b10 con 0x7f19240047c0 2023-11-08T16:30:27.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.728+0000 7f193ca0e640 1 -- 172.21.15.81:0/2801744962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240047c0 msgr2=0x7f1924004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.728+0000 7f193ca0e640 1 --2- 172.21.15.81:0/2801744962 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240047c0 0x7f1924004c40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f192000ba00 tx=0x7f19240057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.728+0000 7f1936d9d640 1 -- 172.21.15.81:0/2801744962 reap_dead start 2023-11-08T16:30:27.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.728+0000 7f193ca0e640 1 -- 172.21.15.81:0/2801744962 shutdown_connections 2023-11-08T16:30:27.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.728+0000 7f193ca0e640 1 -- 172.21.15.81:0/2801744962 >> 172.21.15.81:0/2801744962 conn(0x7f19380cb4e0 msgr2=0x7f19380cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:27.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.728+0000 7f193ca0e640 1 -- 172.21.15.81:0/2801744962 shutdown_connections 2023-11-08T16:30:27.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.728+0000 7f193ca0e640 1 -- 172.21.15.81:0/2801744962 wait complete. 2023-11-08T16:30:27.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.730+0000 7f193ca0e640 1 Processor -- start 2023-11-08T16:30:27.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.730+0000 7f193ca0e640 1 -- start start 2023-11-08T16:30:27.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.731+0000 7f193ca0e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19240047c0 0x7f19380c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.731+0000 7f193ca0e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1938156f80 0x7f193806b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.731+0000 7f193ca0e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f193806b840 0x7f193815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.731+0000 7f193ca0e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19380a8e20 con 0x7f1938156f80 2023-11-08T16:30:27.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.731+0000 7f193ca0e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193806bca0 con 0x7f19240047c0 2023-11-08T16:30:27.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.731+0000 7f193ca0e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19380c7f70 con 0x7f193806b840 2023-11-08T16:30:27.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.731+0000 7f192ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1938156f80 0x7f193806b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.731+0000 7f192ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1938156f80 0x7f193806b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46870/0 (socket says 172.21.15.81:46870) 2023-11-08T16:30:27.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.731+0000 7f192ffff640 1 -- 172.21.15.81:0/3121378572 learned_addr learned my addr 172.21.15.81:0/3121378572 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:27.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.731+0000 7f1936d9d640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f193806b840 0x7f193815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.731+0000 7f193659c640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19240047c0 0x7f19380c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f192ffff640 1 -- 172.21.15.81:0/3121378572 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f193806b840 msgr2=0x7f193815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f192ffff640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f193806b840 0x7f193815a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f192ffff640 1 -- 172.21.15.81:0/3121378572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19240047c0 msgr2=0x7f19380c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:27.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f192ffff640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19240047c0 0x7f19380c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:27.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f192ffff640 1 -- 172.21.15.81:0/3121378572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19380b3c10 con 0x7f1938156f80 2023-11-08T16:30:27.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f193659c640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19240047c0 0x7f19380c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:27.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f192ffff640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1938156f80 0x7f193806b300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f192000bb20 tx=0x7f1920004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:27.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f192effd640 1 -- 172.21.15.81:0/3121378572 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920004020 con 0x7f1938156f80 2023-11-08T16:30:27.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f192effd640 1 -- 172.21.15.81:0/3121378572 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1920004180 con 0x7f1938156f80 2023-11-08T16:30:27.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f192effd640 1 -- 172.21.15.81:0/3121378572 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920003610 con 0x7f1938156f80 2023-11-08T16:30:27.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f193ca0e640 1 -- 172.21.15.81:0/3121378572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19380b3ef0 con 0x7f1938156f80 2023-11-08T16:30:27.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.732+0000 7f193ca0e640 1 -- 172.21.15.81:0/3121378572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19380b44b0 con 0x7f1938156f80 2023-11-08T16:30:27.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.733+0000 7f193ca0e640 1 -- 172.21.15.81:0/3121378572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19380743f0 con 0x7f1938156f80 2023-11-08T16:30:27.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.734+0000 7f192effd640 1 -- 172.21.15.81:0/3121378572 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1920003770 con 0x7f1938156f80 2023-11-08T16:30:27.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.735+0000 7f192effd640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191806ae70 0x7f191806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:27.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.735+0000 7f192effd640 1 -- 172.21.15.81:0/3121378572 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f19200ab530 con 0x7f1938156f80 2023-11-08T16:30:27.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.736+0000 7f193659c640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191806ae70 0x7f191806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:27.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.736+0000 7f193659c640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191806ae70 0x7f191806d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1928006fd0 tx=0x7f1928008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:27.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.739+0000 7f192effd640 1 -- 172.21.15.81:0/3121378572 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1920076930 con 0x7f1938156f80 2023-11-08T16:30:27.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:27.928+0000 7f193ca0e640 1 -- 172.21.15.81:0/3121378572 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f19380743f0 con 0x7f1938156f80 2023-11-08T16:30:28.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.672+0000 7f192effd640 1 -- 172.21.15.81:0/3121378572 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1920006c30 con 0x7f1938156f80 2023-11-08T16:30:28.678 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' created 2023-11-08T16:30:28.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.676+0000 7f193ca0e640 1 -- 172.21.15.81:0/3121378572 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191806ae70 msgr2=0x7f191806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:28.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.676+0000 7f193ca0e640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191806ae70 0x7f191806d330 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1928006fd0 tx=0x7f1928008040 comp rx=0 tx=0).stop 2023-11-08T16:30:28.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.676+0000 7f193ca0e640 1 -- 172.21.15.81:0/3121378572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1938156f80 msgr2=0x7f193806b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:28.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.676+0000 7f193ca0e640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1938156f80 0x7f193806b300 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f192000bb20 tx=0x7f1920004730 comp rx=0 tx=0).stop 2023-11-08T16:30:28.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.677+0000 7f193ca0e640 1 -- 172.21.15.81:0/3121378572 shutdown_connections 2023-11-08T16:30:28.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.677+0000 7f193ca0e640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191806ae70 0x7f191806d330 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1928006fd0 tx=0x7f1928008040 comp rx=0 tx=0).stop 2023-11-08T16:30:28.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.677+0000 7f193ca0e640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f193806b840 0x7f193815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:28.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.677+0000 7f193ca0e640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1938156f80 0x7f193806b300 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:28.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.677+0000 7f193ca0e640 1 --2- 172.21.15.81:0/3121378572 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19240047c0 0x7f19380c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:28.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.677+0000 7f193ca0e640 1 -- 172.21.15.81:0/3121378572 >> 172.21.15.81:0/3121378572 conn(0x7f19380cb4e0 msgr2=0x7f19380c98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:28.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.678+0000 7f193ca0e640 1 -- 172.21.15.81:0/3121378572 shutdown_connections 2023-11-08T16:30:28.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.678+0000 7f193ca0e640 1 -- 172.21.15.81:0/3121378572 wait complete. 2023-11-08T16:30:28.703 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T16:30:28.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.907+0000 7f6037044640 1 Processor -- start 2023-11-08T16:30:28.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.907+0000 7f6037044640 1 -- start start 2023-11-08T16:30:28.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.909+0000 7f6037044640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f60300b0770 0x7f603006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:28.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.909+0000 7f6037044640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f60300ae000 0x7f60300cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:28.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.909+0000 7f6037044640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603006fe50 con 0x7f60300b0b50 2023-11-08T16:30:28.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.909+0000 7f6037044640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60300cb3c0 con 0x7f60300ae000 2023-11-08T16:30:28.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.909+0000 7f6037044640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60300ca580 con 0x7f60300b0770 2023-11-08T16:30:28.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.909+0000 7f6027fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f60300ae000 0x7f60300cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:28.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.909+0000 7f60355e1640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f60300b0b50 0x7f60300b3b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44144/0 (socket says 172.21.15.81:44144) 2023-11-08T16:30:28.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.909+0000 7f60355e1640 1 -- 172.21.15.81:0/3691124186 learned_addr learned my addr 172.21.15.81:0/3691124186 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:28.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.909+0000 7f6034de0640 1 --2- 172.21.15.81:0/3691124186 >> v2:172.21.15.93:3300/0 conn(0x7f60300b0770 0x7f603006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:28.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.910+0000 7f6027fff640 1 -- 172.21.15.81:0/3691124186 >> v2:172.21.15.93:3300/0 conn(0x7f60300b0770 msgr2=0x7f603006f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:28.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.910+0000 7f6027fff640 1 --2- 172.21.15.81:0/3691124186 >> v2:172.21.15.93:3300/0 conn(0x7f60300b0770 0x7f603006f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:28.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.910+0000 7f6027fff640 1 -- 172.21.15.81:0/3691124186 >> v1:172.21.15.81:6789/0 conn(0x7f60300b0b50 legacy=0x7f60300b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:28.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.910+0000 7f6027fff640 1 -- 172.21.15.81:0/3691124186 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60300ccec0 con 0x7f60300ae000 2023-11-08T16:30:28.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.911+0000 7f6027fff640 1 --2- 172.21.15.81:0/3691124186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60300ae000 0x7f60300cc980 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f60300cc290 tx=0x7f602002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47a91db24115fb8e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:28.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.911+0000 7f60277fe640 1 -- 172.21.15.81:0/3691124186 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020031830 con 0x7f60300ae000 2023-11-08T16:30:28.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.911+0000 7f60277fe640 1 -- 172.21.15.81:0/3691124186 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6020031990 con 0x7f60300ae000 2023-11-08T16:30:28.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.911+0000 7f60277fe640 1 -- 172.21.15.81:0/3691124186 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6020031d20 con 0x7f60300ae000 2023-11-08T16:30:28.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.912+0000 7f6037044640 1 -- 172.21.15.81:0/3691124186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60300ae000 msgr2=0x7f60300cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:28.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.912+0000 7f6037044640 1 --2- 172.21.15.81:0/3691124186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60300ae000 0x7f60300cc980 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f60300cc290 tx=0x7f602002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:30:28.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.912+0000 7f6037044640 1 -- 172.21.15.81:0/3691124186 shutdown_connections 2023-11-08T16:30:28.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.912+0000 7f6037044640 1 --2- 172.21.15.81:0/3691124186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60300ae000 0x7f60300cc980 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:28.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.912+0000 7f6037044640 1 --2- 172.21.15.81:0/3691124186 >> v2:172.21.15.93:3300/0 conn(0x7f60300b0770 0x7f603006f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:28.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.912+0000 7f6037044640 1 -- 172.21.15.81:0/3691124186 >> 172.21.15.81:0/3691124186 conn(0x7f60300cb500 msgr2=0x7f60300cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:28.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.912+0000 7f6037044640 1 -- 172.21.15.81:0/3691124186 shutdown_connections 2023-11-08T16:30:28.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.912+0000 7f6037044640 1 -- 172.21.15.81:0/3691124186 wait complete. 2023-11-08T16:30:28.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.914+0000 7f6037044640 1 Processor -- start 2023-11-08T16:30:28.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.914+0000 7f6037044640 1 -- start start 2023-11-08T16:30:28.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6037044640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60300ae000 0x7f60300b35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:28.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6037044640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f60300b0770 0x7f60300af570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:28.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6037044640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60300b0b50 0x7f60301509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:28.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6037044640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603006ce50 con 0x7f60300ae000 2023-11-08T16:30:28.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6037044640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603006d270 con 0x7f60300b0b50 2023-11-08T16:30:28.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6037044640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603006c4b0 con 0x7f60300b0770 2023-11-08T16:30:28.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6034de0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60300ae000 0x7f60300b35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:28.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6034de0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60300ae000 0x7f60300b35a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46894/0 (socket says 172.21.15.81:46894) 2023-11-08T16:30:28.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6034de0640 1 -- 172.21.15.81:0/3821591421 learned_addr learned my addr 172.21.15.81:0/3821591421 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:28.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6034de0640 1 -- 172.21.15.81:0/3821591421 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f60300b0770 msgr2=0x7f60300af570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:28.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6034de0640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f60300b0770 0x7f60300af570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:28.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6034de0640 1 -- 172.21.15.81:0/3821591421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60300b0b50 msgr2=0x7f60301509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:28.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6034de0640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60300b0b50 0x7f60301509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:28.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.915+0000 7f6034de0640 1 -- 172.21.15.81:0/3821591421 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6020009400 con 0x7f60300ae000 2023-11-08T16:30:28.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.916+0000 7f6034de0640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60300ae000 0x7f60300b35a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f602c007ef0 tx=0x7f602c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:28.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.916+0000 7f60257fa640 1 -- 172.21.15.81:0/3821591421 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c014620 con 0x7f60300ae000 2023-11-08T16:30:28.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.916+0000 7f60257fa640 1 -- 172.21.15.81:0/3821591421 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c014780 con 0x7f60300ae000 2023-11-08T16:30:28.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.916+0000 7f60257fa640 1 -- 172.21.15.81:0/3821591421 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c014b30 con 0x7f60300ae000 2023-11-08T16:30:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.916+0000 7f6037044640 1 -- 172.21.15.81:0/3821591421 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603006c790 con 0x7f60300ae000 2023-11-08T16:30:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.916+0000 7f6037044640 1 -- 172.21.15.81:0/3821591421 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60300b3ae0 con 0x7f60300ae000 2023-11-08T16:30:28.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.917+0000 7f6037044640 1 -- 172.21.15.81:0/3821591421 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60300743f0 con 0x7f60300ae000 2023-11-08T16:30:28.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.919+0000 7f60257fa640 1 -- 172.21.15.81:0/3821591421 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f602c014c90 con 0x7f60300ae000 2023-11-08T16:30:28.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.920+0000 7f60257fa640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f601c06ae70 0x7f601c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:28.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.920+0000 7f60257fa640 1 -- 172.21.15.81:0/3821591421 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f602c08ac00 con 0x7f60300ae000 2023-11-08T16:30:28.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.920+0000 7f6027fff640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f601c06ae70 0x7f601c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:28.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.921+0000 7f6027fff640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f601c06ae70 0x7f601c06d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6020036000 tx=0x7f6020030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:28.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:28.924+0000 7f60257fa640 1 -- 172.21.15.81:0/3821591421 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f602c055f70 con 0x7f60300ae000 2023-11-08T16:30:29.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.117+0000 7f6037044640 1 -- 172.21.15.81:0/3821591421 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f60300cde50 con 0x7f60300ae000 2023-11-08T16:30:29.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.678+0000 7f60257fa640 1 -- 172.21.15.81:0/3821591421 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7f602c059590 con 0x7f60300ae000 2023-11-08T16:30:29.681 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' created 2023-11-08T16:30:29.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.682+0000 7f6037044640 1 -- 172.21.15.81:0/3821591421 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f601c06ae70 msgr2=0x7f601c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:29.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.682+0000 7f6037044640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f601c06ae70 0x7f601c06d330 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6020036000 tx=0x7f6020030000 comp rx=0 tx=0).stop 2023-11-08T16:30:29.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.682+0000 7f6037044640 1 -- 172.21.15.81:0/3821591421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60300ae000 msgr2=0x7f60300b35a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:29.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.682+0000 7f6037044640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60300ae000 0x7f60300b35a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f602c007ef0 tx=0x7f602c00e4b0 comp rx=0 tx=0).stop 2023-11-08T16:30:29.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.682+0000 7f6037044640 1 -- 172.21.15.81:0/3821591421 shutdown_connections 2023-11-08T16:30:29.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.682+0000 7f6037044640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f601c06ae70 0x7f601c06d330 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:29.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.682+0000 7f6037044640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60300b0b50 0x7f60301509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:29.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.682+0000 7f6037044640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f60300b0770 0x7f60300af570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:29.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.682+0000 7f6037044640 1 --2- 172.21.15.81:0/3821591421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60300ae000 0x7f60300b35a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:29.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.682+0000 7f6037044640 1 -- 172.21.15.81:0/3821591421 >> 172.21.15.81:0/3821591421 conn(0x7f60300cb500 msgr2=0x7f60300cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:29.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.683+0000 7f6037044640 1 -- 172.21.15.81:0/3821591421 shutdown_connections 2023-11-08T16:30:29.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.683+0000 7f6037044640 1 -- 172.21.15.81:0/3821591421 wait complete. 2023-11-08T16:30:29.715 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T16:30:29.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.900+0000 7f8f50797640 1 Processor -- start 2023-11-08T16:30:29.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.901+0000 7f8f50797640 1 -- start start 2023-11-08T16:30:29.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.902+0000 7f8f50797640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8f4815a0e0 0x7f8f480ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:29.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.902+0000 7f8f50797640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8f480c9bf0 0x7f8f480ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:29.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.902+0000 7f8f50797640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f480ce960 con 0x7f8f480ca580 2023-11-08T16:30:29.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.902+0000 7f8f50797640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f48071980 con 0x7f8f480c9bf0 2023-11-08T16:30:29.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.902+0000 7f8f50797640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f480c9480 con 0x7f8f4815a0e0 2023-11-08T16:30:29.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.902+0000 7f8f4dd32640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8f480c9bf0 0x7f8f480ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:29.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.902+0000 7f8f4dd32640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8f480c9bf0 0x7f8f480ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46904/0 (socket says 172.21.15.81:46904) 2023-11-08T16:30:29.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.902+0000 7f8f4dd32640 1 -- 172.21.15.81:0/1787732322 learned_addr learned my addr 172.21.15.81:0/1787732322 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:29.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.903+0000 7f8f4e533640 1 --2- 172.21.15.81:0/1787732322 >> v2:172.21.15.93:3300/0 conn(0x7f8f4815a0e0 0x7f8f480ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:29.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.903+0000 7f8f4dd32640 1 -- 172.21.15.81:0/1787732322 >> v2:172.21.15.93:3300/0 conn(0x7f8f4815a0e0 msgr2=0x7f8f480ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:29.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.903+0000 7f8f4dd32640 1 --2- 172.21.15.81:0/1787732322 >> v2:172.21.15.93:3300/0 conn(0x7f8f4815a0e0 0x7f8f480ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:29.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.903+0000 7f8f4dd32640 1 -- 172.21.15.81:0/1787732322 >> v1:172.21.15.93:6789/0 conn(0x7f8f480ca580 legacy=0x7f8f4815dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:29.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.903+0000 7f8f4dd32640 1 -- 172.21.15.81:0/1787732322 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4815a4c0 con 0x7f8f480c9bf0 2023-11-08T16:30:29.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.904+0000 7f8f4dd32640 1 --2- 172.21.15.81:0/1787732322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f480c9bf0 0x7f8f480ca040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8f48067640 tx=0x7f8f3802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf9f3088f9dd880f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:29.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.904+0000 7f8f4d531640 1 -- 172.21.15.81:0/1787732322 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38030860 con 0x7f8f480c9bf0 2023-11-08T16:30:29.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.904+0000 7f8f4d531640 1 -- 172.21.15.81:0/1787732322 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f380309c0 con 0x7f8f480c9bf0 2023-11-08T16:30:29.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.904+0000 7f8f4d531640 1 -- 172.21.15.81:0/1787732322 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f38030d50 con 0x7f8f480c9bf0 2023-11-08T16:30:29.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.905+0000 7f8f50797640 1 -- 172.21.15.81:0/1787732322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f480c9bf0 msgr2=0x7f8f480ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:29.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.905+0000 7f8f50797640 1 --2- 172.21.15.81:0/1787732322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f480c9bf0 0x7f8f480ca040 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8f48067640 tx=0x7f8f3802b9a0 comp rx=0 tx=0).stop 2023-11-08T16:30:29.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.905+0000 7f8f50797640 1 -- 172.21.15.81:0/1787732322 shutdown_connections 2023-11-08T16:30:29.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.905+0000 7f8f50797640 1 --2- 172.21.15.81:0/1787732322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f480c9bf0 0x7f8f480ca040 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:29.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.905+0000 7f8f50797640 1 --2- 172.21.15.81:0/1787732322 >> v2:172.21.15.93:3300/0 conn(0x7f8f4815a0e0 0x7f8f480ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:29.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.905+0000 7f8f50797640 1 -- 172.21.15.81:0/1787732322 >> 172.21.15.81:0/1787732322 conn(0x7f8f480cb580 msgr2=0x7f8f480cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:29.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.905+0000 7f8f50797640 1 -- 172.21.15.81:0/1787732322 shutdown_connections 2023-11-08T16:30:29.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.905+0000 7f8f50797640 1 -- 172.21.15.81:0/1787732322 wait complete. 2023-11-08T16:30:29.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.907+0000 7f8f50797640 1 Processor -- start 2023-11-08T16:30:29.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.907+0000 7f8f50797640 1 -- start start 2023-11-08T16:30:29.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.908+0000 7f8f50797640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f480c9bf0 0x7f8f480a8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:29.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.908+0000 7f8f50797640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f480ca580 0x7f8f480afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:29.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.908+0000 7f8f50797640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f4815a0e0 0x7f8f480aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:29.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.908+0000 7f8f50797640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f48149200 con 0x7f8f4815a0e0 2023-11-08T16:30:29.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.908+0000 7f8f50797640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480b0490 con 0x7f8f480ca580 2023-11-08T16:30:29.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.908+0000 7f8f50797640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f480c8590 con 0x7f8f480c9bf0 2023-11-08T16:30:29.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.908+0000 7f8f4ed34640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f4815a0e0 0x7f8f480aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:29.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.908+0000 7f8f4ed34640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f4815a0e0 0x7f8f480aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46908/0 (socket says 172.21.15.81:46908) 2023-11-08T16:30:29.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.908+0000 7f8f4ed34640 1 -- 172.21.15.81:0/1828853648 learned_addr learned my addr 172.21.15.81:0/1828853648 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:29.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.909+0000 7f8f4ed34640 1 -- 172.21.15.81:0/1828853648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f480c9bf0 msgr2=0x7f8f480a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:29.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.909+0000 7f8f4ed34640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f480c9bf0 0x7f8f480a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:29.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.909+0000 7f8f4ed34640 1 -- 172.21.15.81:0/1828853648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f480ca580 msgr2=0x7f8f480afa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:29.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.909+0000 7f8f4ed34640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f480ca580 0x7f8f480afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:29.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.909+0000 7f8f4ed34640 1 -- 172.21.15.81:0/1828853648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f380094d0 con 0x7f8f4815a0e0 2023-11-08T16:30:29.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.909+0000 7f8f4ed34640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f4815a0e0 0x7f8f480aff50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000c7b0 tx=0x7f8f4000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:29.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.909+0000 7f8f3effd640 1 -- 172.21.15.81:0/1828853648 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f40004b60 con 0x7f8f4815a0e0 2023-11-08T16:30:29.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.909+0000 7f8f3effd640 1 -- 172.21.15.81:0/1828853648 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f40004cc0 con 0x7f8f4815a0e0 2023-11-08T16:30:29.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.909+0000 7f8f3effd640 1 -- 172.21.15.81:0/1828853648 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f4000aa90 con 0x7f8f4815a0e0 2023-11-08T16:30:29.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.909+0000 7f8f50797640 1 -- 172.21.15.81:0/1828853648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f48148860 con 0x7f8f4815a0e0 2023-11-08T16:30:29.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.909+0000 7f8f50797640 1 -- 172.21.15.81:0/1828853648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f48148e20 con 0x7f8f4815a0e0 2023-11-08T16:30:29.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.910+0000 7f8f50797640 1 -- 172.21.15.81:0/1828853648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f4804ece0 con 0x7f8f4815a0e0 2023-11-08T16:30:29.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.912+0000 7f8f3effd640 1 -- 172.21.15.81:0/1828853648 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f4000abf0 con 0x7f8f4815a0e0 2023-11-08T16:30:29.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.913+0000 7f8f3effd640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f2c06aea0 0x7f8f2c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:29.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.913+0000 7f8f3effd640 1 -- 172.21.15.81:0/1828853648 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f8f40089ec0 con 0x7f8f4815a0e0 2023-11-08T16:30:29.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.913+0000 7f8f4e533640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f2c06aea0 0x7f8f2c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:29.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.914+0000 7f8f4e533640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f2c06aea0 0x7f8f2c06d360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8f44006fd0 tx=0x7f8f44008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:29.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:29.916+0000 7f8f3effd640 1 -- 172.21.15.81:0/1828853648 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f40054fb0 con 0x7f8f4815a0e0 2023-11-08T16:30:30.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.139+0000 7f8f50797640 1 -- 172.21.15.81:0/1828853648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8f480cc3b0 con 0x7f8f4815a0e0 2023-11-08T16:30:30.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.700+0000 7f8f3effd640 1 -- 172.21.15.81:0/1828853648 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8f400585d0 con 0x7f8f4815a0e0 2023-11-08T16:30:30.702 INFO:teuthology.orchestra.run.smithi081.stderr:new fs with metadata pool 4 and data pool 5 2023-11-08T16:30:30.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.703+0000 7f8f50797640 1 -- 172.21.15.81:0/1828853648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f2c06aea0 msgr2=0x7f8f2c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:30.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.703+0000 7f8f50797640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f2c06aea0 0x7f8f2c06d360 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8f44006fd0 tx=0x7f8f44008040 comp rx=0 tx=0).stop 2023-11-08T16:30:30.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.703+0000 7f8f50797640 1 -- 172.21.15.81:0/1828853648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f4815a0e0 msgr2=0x7f8f480aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.703+0000 7f8f50797640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f4815a0e0 0x7f8f480aff50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000c7b0 tx=0x7f8f4000cc70 comp rx=0 tx=0).stop 2023-11-08T16:30:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.704+0000 7f8f50797640 1 -- 172.21.15.81:0/1828853648 shutdown_connections 2023-11-08T16:30:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.704+0000 7f8f50797640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f2c06aea0 0x7f8f2c06d360 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.704+0000 7f8f50797640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f4815a0e0 0x7f8f480aff50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.704+0000 7f8f50797640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f480ca580 0x7f8f480afa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.704+0000 7f8f50797640 1 --2- 172.21.15.81:0/1828853648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f480c9bf0 0x7f8f480a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.704+0000 7f8f50797640 1 -- 172.21.15.81:0/1828853648 >> 172.21.15.81:0/1828853648 conn(0x7f8f480cb580 msgr2=0x7f8f4806c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.704+0000 7f8f50797640 1 -- 172.21.15.81:0/1828853648 shutdown_connections 2023-11-08T16:30:30.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.704+0000 7f8f50797640 1 -- 172.21.15.81:0/1828853648 wait complete. 2023-11-08T16:30:30.740 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:30:30.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.917+0000 7fdda750f640 1 Processor -- start 2023-11-08T16:30:30.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.917+0000 7fdda750f640 1 -- start start 2023-11-08T16:30:30.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.918+0000 7fdda750f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda015d2b0 0x7fdda0161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:30.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.918+0000 7fdda750f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fdda00ccf00 0x7fdda006ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:30.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.918+0000 7fdda750f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda0161c50 con 0x7fdda00c9060 2023-11-08T16:30:30.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.918+0000 7fdda750f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda015d690 con 0x7fdda00ccf00 2023-11-08T16:30:30.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.918+0000 7fdda750f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda006b980 con 0x7fdda015d2b0 2023-11-08T16:30:30.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.919+0000 7fdda52ab640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda015d2b0 0x7fdda0161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:30.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.919+0000 7fdda5aac640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fdda00ccf00 0x7fdda006ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:30.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.919+0000 7fdda52ab640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda015d2b0 0x7fdda0161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:53382/0 (socket says 172.21.15.81:53382) 2023-11-08T16:30:30.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.919+0000 7fdda52ab640 1 -- 172.21.15.81:0/690793524 learned_addr learned my addr 172.21.15.81:0/690793524 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:30.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.920+0000 7fdda52ab640 1 -- 172.21.15.81:0/690793524 >> v2:172.21.15.93:3300/0 conn(0x7fdda00ccf00 msgr2=0x7fdda006ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:30.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.920+0000 7fdda52ab640 1 --2- 172.21.15.81:0/690793524 >> v2:172.21.15.93:3300/0 conn(0x7fdda00ccf00 0x7fdda006ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:30.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.920+0000 7fdda52ab640 1 -- 172.21.15.81:0/690793524 >> v1:172.21.15.93:6789/0 conn(0x7fdda00c9060 legacy=0x7fdda00cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:30.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.920+0000 7fdda52ab640 1 -- 172.21.15.81:0/690793524 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda006bc60 con 0x7fdda015d2b0 2023-11-08T16:30:30.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.921+0000 7fdda52ab640 1 --2- 172.21.15.81:0/690793524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda015d2b0 0x7fdda0161710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009870 tx=0x7fdd9c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c9526f1960d841d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:30.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.922+0000 7fdd97fff640 1 -- 172.21.15.81:0/690793524 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c030840 con 0x7fdda015d2b0 2023-11-08T16:30:30.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.922+0000 7fdd97fff640 1 -- 172.21.15.81:0/690793524 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9c0309a0 con 0x7fdda015d2b0 2023-11-08T16:30:30.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.922+0000 7fdd97fff640 1 -- 172.21.15.81:0/690793524 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c030d30 con 0x7fdda015d2b0 2023-11-08T16:30:30.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.923+0000 7fdda750f640 1 -- 172.21.15.81:0/690793524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda015d2b0 msgr2=0x7fdda0161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:30.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.923+0000 7fdda750f640 1 --2- 172.21.15.81:0/690793524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda015d2b0 0x7fdda0161710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c009870 tx=0x7fdd9c02b980 comp rx=0 tx=0).stop 2023-11-08T16:30:30.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.923+0000 7fdda750f640 1 -- 172.21.15.81:0/690793524 shutdown_connections 2023-11-08T16:30:30.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.923+0000 7fdda750f640 1 --2- 172.21.15.81:0/690793524 >> v2:172.21.15.93:3300/0 conn(0x7fdda00ccf00 0x7fdda006ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:30.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.923+0000 7fdda750f640 1 --2- 172.21.15.81:0/690793524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda015d2b0 0x7fdda0161710 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:30.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.923+0000 7fdda750f640 1 -- 172.21.15.81:0/690793524 >> 172.21.15.81:0/690793524 conn(0x7fdda00cb490 msgr2=0x7fdda00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:30.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.923+0000 7fdda750f640 1 -- 172.21.15.81:0/690793524 shutdown_connections 2023-11-08T16:30:30.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.923+0000 7fdda750f640 1 -- 172.21.15.81:0/690793524 wait complete. 2023-11-08T16:30:30.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.925+0000 7fdda750f640 1 Processor -- start 2023-11-08T16:30:30.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.925+0000 7fdda750f640 1 -- start start 2023-11-08T16:30:30.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.926+0000 7fdda750f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda00c9060 0x7fdda01f0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:30.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.926+0000 7fdda750f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdda00ccf00 0x7fdda01f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:30.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.926+0000 7fdda750f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdda015d2b0 0x7fdda00c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:30.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.926+0000 7fdda750f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda01f72a0 con 0x7fdda015d2b0 2023-11-08T16:30:30.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.926+0000 7fdda750f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda01f3170 con 0x7fdda00ccf00 2023-11-08T16:30:30.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.926+0000 7fdda750f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda01f3450 con 0x7fdda00c9060 2023-11-08T16:30:30.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.926+0000 7fdda5aac640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdda015d2b0 0x7fdda00c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:30.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.926+0000 7fdda5aac640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdda015d2b0 0x7fdda00c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40656/0 (socket says 172.21.15.81:40656) 2023-11-08T16:30:30.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.926+0000 7fdda5aac640 1 -- 172.21.15.81:0/2873682177 learned_addr learned my addr 172.21.15.81:0/2873682177 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:30.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdda52ab640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda00c9060 0x7fdda01f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:30.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdda4aaa640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdda00ccf00 0x7fdda01f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:30.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdda5aac640 1 -- 172.21.15.81:0/2873682177 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda00c9060 msgr2=0x7fdda01f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:30.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdda5aac640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda00c9060 0x7fdda01f0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:30.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdda5aac640 1 -- 172.21.15.81:0/2873682177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdda00ccf00 msgr2=0x7fdda01f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:30.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdda5aac640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdda00ccf00 0x7fdda01f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:30.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdda5aac640 1 -- 172.21.15.81:0/2873682177 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c0094d0 con 0x7fdda015d2b0 2023-11-08T16:30:30.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdda4aaa640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdda00ccf00 0x7fdda01f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:30.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdda5aac640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdda015d2b0 0x7fdda00c77a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800efc0 tx=0x7fdd9800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:30.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdd95ffb640 1 -- 172.21.15.81:0/2873682177 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98014480 con 0x7fdda015d2b0 2023-11-08T16:30:30.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdd95ffb640 1 -- 172.21.15.81:0/2873682177 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9800f040 con 0x7fdda015d2b0 2023-11-08T16:30:30.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdda750f640 1 -- 172.21.15.81:0/2873682177 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda01f3730 con 0x7fdda015d2b0 2023-11-08T16:30:30.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdd95ffb640 1 -- 172.21.15.81:0/2873682177 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd98014820 con 0x7fdda015d2b0 2023-11-08T16:30:30.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.927+0000 7fdda750f640 1 -- 172.21.15.81:0/2873682177 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda01f3cf0 con 0x7fdda015d2b0 2023-11-08T16:30:30.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.928+0000 7fdda750f640 1 -- 172.21.15.81:0/2873682177 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda00b1610 con 0x7fdda015d2b0 2023-11-08T16:30:30.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.929+0000 7fdd95ffb640 1 -- 172.21.15.81:0/2873682177 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd98014980 con 0x7fdda015d2b0 2023-11-08T16:30:30.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.930+0000 7fdd95ffb640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd8406ae70 0x7fdd8406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:30.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.930+0000 7fdd95ffb640 1 -- 172.21.15.81:0/2873682177 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd9808a1f0 con 0x7fdda015d2b0 2023-11-08T16:30:30.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.930+0000 7fdda52ab640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd8406ae70 0x7fdd8406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:30.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.931+0000 7fdda52ab640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd8406ae70 0x7fdd8406d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c036000 tx=0x7fdd9c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:30.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:30.934+0000 7fdd95ffb640 1 -- 172.21.15.81:0/2873682177 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdda00b1610 con 0x7fdda015d2b0 2023-11-08T16:30:31.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.123+0000 7fdda750f640 1 -- 172.21.15.81:0/2873682177 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdda00b1610 con 0x7fdda015d2b0 2023-11-08T16:30:31.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.126+0000 7fdd95ffb640 1 -- 172.21.15.81:0/2873682177 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16522 (secure 0 0 0) 0x7fdda00b1610 con 0x7fdda015d2b0 2023-11-08T16:30:31.129 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:31.129 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":34,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:30:30.141238+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:30:27.977481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T16:30:29.171902+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:30:31.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 -- 172.21.15.81:0/2873682177 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd8406ae70 msgr2=0x7fdd8406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:31.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd8406ae70 0x7fdd8406d330 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c036000 tx=0x7fdd9c03b040 comp rx=0 tx=0).stop 2023-11-08T16:30:31.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 -- 172.21.15.81:0/2873682177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdda015d2b0 msgr2=0x7fdda00c77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:31.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdda015d2b0 0x7fdda00c77a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdd9800efc0 tx=0x7fdd9800c490 comp rx=0 tx=0).stop 2023-11-08T16:30:31.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 -- 172.21.15.81:0/2873682177 shutdown_connections 2023-11-08T16:30:31.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdd8406ae70 0x7fdd8406d330 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdda015d2b0 0x7fdda00c77a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdda00ccf00 0x7fdda01f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 --2- 172.21.15.81:0/2873682177 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdda00c9060 0x7fdda01f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 -- 172.21.15.81:0/2873682177 >> 172.21.15.81:0/2873682177 conn(0x7fdda00cb490 msgr2=0x7fdda00c9460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:31.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 -- 172.21.15.81:0/2873682177 shutdown_connections 2023-11-08T16:30:31.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.129+0000 7fdda750f640 1 -- 172.21.15.81:0/2873682177 wait complete. 2023-11-08T16:30:31.150 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:30:31.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.356+0000 7fb88cb18640 1 Processor -- start 2023-11-08T16:30:31.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.356+0000 7fb88cb18640 1 -- start start 2023-11-08T16:30:31.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.358+0000 7fb88cb18640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb8880ade50 0x7fb8880ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.358+0000 7fb88cb18640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb88815a680 0x7fb88815ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.358+0000 7fb88cb18640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88806b630 con 0x7fb88815b050 2023-11-08T16:30:31.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.358+0000 7fb88cb18640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8880b1860 con 0x7fb88815a680 2023-11-08T16:30:31.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.358+0000 7fb88cb18640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8880ca9c0 con 0x7fb8880ade50 2023-11-08T16:30:31.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.358+0000 7fb87dd9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb88815a680 0x7fb88815ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:31.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.358+0000 7fb886d9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb88815b050 0x7fb8880c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53654/0 (socket says 172.21.15.81:53654) 2023-11-08T16:30:31.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.358+0000 7fb886d9d640 1 -- 172.21.15.81:0/3670082175 learned_addr learned my addr 172.21.15.81:0/3670082175 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:31.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.359+0000 7fb88659c640 1 --2- 172.21.15.81:0/3670082175 >> v2:172.21.15.93:3300/0 conn(0x7fb8880ade50 0x7fb8880ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:31.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.359+0000 7fb87dd9b640 1 -- 172.21.15.81:0/3670082175 >> v2:172.21.15.93:3300/0 conn(0x7fb8880ade50 msgr2=0x7fb8880ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:31.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.359+0000 7fb87dd9b640 1 --2- 172.21.15.81:0/3670082175 >> v2:172.21.15.93:3300/0 conn(0x7fb8880ade50 0x7fb8880ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.359+0000 7fb87dd9b640 1 -- 172.21.15.81:0/3670082175 >> v1:172.21.15.81:6789/0 conn(0x7fb88815b050 legacy=0x7fb8880c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:31.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.359+0000 7fb87dd9b640 1 -- 172.21.15.81:0/3670082175 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8880cabb0 con 0x7fb88815a680 2023-11-08T16:30:31.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.359+0000 7fb88659c640 1 --2- 172.21.15.81:0/3670082175 >> v2:172.21.15.93:3300/0 conn(0x7fb8880ade50 0x7fb8880ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:31.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.360+0000 7fb87dd9b640 1 --2- 172.21.15.81:0/3670082175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88815a680 0x7fb88815ab10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb888067770 tx=0x7fb87002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481a71a85fcf64c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:31.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.360+0000 7fb885d9b640 1 -- 172.21.15.81:0/3670082175 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870031830 con 0x7fb88815a680 2023-11-08T16:30:31.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.360+0000 7fb885d9b640 1 -- 172.21.15.81:0/3670082175 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb870031990 con 0x7fb88815a680 2023-11-08T16:30:31.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.360+0000 7fb885d9b640 1 -- 172.21.15.81:0/3670082175 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb870031d20 con 0x7fb88815a680 2023-11-08T16:30:31.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.361+0000 7fb88cb18640 1 -- 172.21.15.81:0/3670082175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88815a680 msgr2=0x7fb88815ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:31.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.361+0000 7fb88cb18640 1 --2- 172.21.15.81:0/3670082175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88815a680 0x7fb88815ab10 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb888067770 tx=0x7fb87002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.361+0000 7fb88cb18640 1 -- 172.21.15.81:0/3670082175 shutdown_connections 2023-11-08T16:30:31.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.361+0000 7fb88cb18640 1 --2- 172.21.15.81:0/3670082175 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88815a680 0x7fb88815ab10 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.361+0000 7fb88cb18640 1 --2- 172.21.15.81:0/3670082175 >> v2:172.21.15.93:3300/0 conn(0x7fb8880ade50 0x7fb8880ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.361+0000 7fb88cb18640 1 -- 172.21.15.81:0/3670082175 >> 172.21.15.81:0/3670082175 conn(0x7fb8880cb490 msgr2=0x7fb8880cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:31.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.361+0000 7fb88cb18640 1 -- 172.21.15.81:0/3670082175 shutdown_connections 2023-11-08T16:30:31.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.361+0000 7fb88cb18640 1 -- 172.21.15.81:0/3670082175 wait complete. 2023-11-08T16:30:31.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.362+0000 7fb88cb18640 1 Processor -- start 2023-11-08T16:30:31.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.363+0000 7fb88cb18640 1 -- start start 2023-11-08T16:30:31.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.363+0000 7fb88cb18640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8880ade50 0x7fb888151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb88cb18640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb88815a680 0x7fb88806bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb88cb18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88815b050 0x7fb88806c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb88cb18640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8881522f0 con 0x7fb88815b050 2023-11-08T16:30:31.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb88cb18640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c73f0 con 0x7fb8880ade50 2023-11-08T16:30:31.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb88cb18640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c76d0 con 0x7fb88815a680 2023-11-08T16:30:31.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb886d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88815b050 0x7fb88806c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:31.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb886d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88815b050 0x7fb88806c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40676/0 (socket says 172.21.15.81:40676) 2023-11-08T16:30:31.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb886d9d640 1 -- 172.21.15.81:0/2268402779 learned_addr learned my addr 172.21.15.81:0/2268402779 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:31.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb88659c640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8880ade50 0x7fb888151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:31.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb886d9d640 1 -- 172.21.15.81:0/2268402779 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb88815a680 msgr2=0x7fb88806bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:31.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb886d9d640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb88815a680 0x7fb88806bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb886d9d640 1 -- 172.21.15.81:0/2268402779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8880ade50 msgr2=0x7fb888151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:31.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb886d9d640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8880ade50 0x7fb888151db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.364+0000 7fb886d9d640 1 -- 172.21.15.81:0/2268402779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb870009400 con 0x7fb88815b050 2023-11-08T16:30:31.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.365+0000 7fb886d9d640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88815b050 0x7fb88806c230 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c370 tx=0x7fb88000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:31.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.365+0000 7fb87f7fe640 1 -- 172.21.15.81:0/2268402779 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880005ce0 con 0x7fb88815b050 2023-11-08T16:30:31.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.365+0000 7fb87f7fe640 1 -- 172.21.15.81:0/2268402779 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb880005e40 con 0x7fb88815b050 2023-11-08T16:30:31.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.365+0000 7fb87f7fe640 1 -- 172.21.15.81:0/2268402779 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8800072d0 con 0x7fb88815b050 2023-11-08T16:30:31.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.365+0000 7fb88cb18640 1 -- 172.21.15.81:0/2268402779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8880c79b0 con 0x7fb88815b050 2023-11-08T16:30:31.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.365+0000 7fb88cb18640 1 -- 172.21.15.81:0/2268402779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8880c9530 con 0x7fb88815b050 2023-11-08T16:30:31.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.366+0000 7fb88cb18640 1 -- 172.21.15.81:0/2268402779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8880c81b0 con 0x7fb88815b050 2023-11-08T16:30:31.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.367+0000 7fb87f7fe640 1 -- 172.21.15.81:0/2268402779 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb8800074f0 con 0x7fb88815b050 2023-11-08T16:30:31.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.368+0000 7fb87f7fe640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb86806ae70 0x7fb86806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.368+0000 7fb87f7fe640 1 -- 172.21.15.81:0/2268402779 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb880088ac0 con 0x7fb88815b050 2023-11-08T16:30:31.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.368+0000 7fb88659c640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb86806ae70 0x7fb86806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:31.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.369+0000 7fb88659c640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb86806ae70 0x7fb86806d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb878008f80 tx=0x7fb8780084d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:31.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.372+0000 7fb87f7fe640 1 -- 172.21.15.81:0/2268402779 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8880c81b0 con 0x7fb88815b050 2023-11-08T16:30:31.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.563+0000 7fb88cb18640 1 -- 172.21.15.81:0/2268402779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8880c81b0 con 0x7fb88815b050 2023-11-08T16:30:31.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.566+0000 7fb87f7fe640 1 -- 172.21.15.81:0/2268402779 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16522 (secure 0 0 0) 0x7fb8880c81b0 con 0x7fb88815b050 2023-11-08T16:30:31.568 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:31.568 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":34,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:30:30.141238+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:30:27.977481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T16:30:29.171902+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:30:31.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 -- 172.21.15.81:0/2268402779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb86806ae70 msgr2=0x7fb86806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:31.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb86806ae70 0x7fb86806d330 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb878008f80 tx=0x7fb8780084d0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 -- 172.21.15.81:0/2268402779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88815b050 msgr2=0x7fb88806c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:31.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88815b050 0x7fb88806c230 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb88000c370 tx=0x7fb88000c830 comp rx=0 tx=0).stop 2023-11-08T16:30:31.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 -- 172.21.15.81:0/2268402779 shutdown_connections 2023-11-08T16:30:31.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb86806ae70 0x7fb86806d330 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb88815b050 0x7fb88806c230 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb88815a680 0x7fb88806bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 --2- 172.21.15.81:0/2268402779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8880ade50 0x7fb888151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 -- 172.21.15.81:0/2268402779 >> 172.21.15.81:0/2268402779 conn(0x7fb8880cb490 msgr2=0x7fb8880b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:31.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 -- 172.21.15.81:0/2268402779 shutdown_connections 2023-11-08T16:30:31.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.568+0000 7fb88cb18640 1 -- 172.21.15.81:0/2268402779 wait complete. 2023-11-08T16:30:31.588 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T16:30:31.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.803+0000 7f6cabc15640 1 Processor -- start 2023-11-08T16:30:31.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.803+0000 7f6cabc15640 1 -- start start 2023-11-08T16:30:31.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.804+0000 7f6cabc15640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ca415a0e0 0x7f6ca40ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.805+0000 7f6cabc15640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ca40ca720 0x7f6ca406b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.805+0000 7f6cabc15640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca40ce960 con 0x7f6ca40c9bf0 2023-11-08T16:30:31.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.805+0000 7f6cabc15640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca4071980 con 0x7f6ca40ca720 2023-11-08T16:30:31.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.805+0000 7f6cabc15640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40c9480 con 0x7f6ca415a0e0 2023-11-08T16:30:31.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.805+0000 7f6caa1b2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ca40ca720 0x7f6ca406b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:31.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.805+0000 7f6caa1b2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ca40ca720 0x7f6ca406b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40686/0 (socket says 172.21.15.81:40686) 2023-11-08T16:30:31.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.805+0000 7f6caa1b2640 1 -- 172.21.15.81:0/1075949208 learned_addr learned my addr 172.21.15.81:0/1075949208 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:31.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.805+0000 7f6caa1b2640 1 -- 172.21.15.81:0/1075949208 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ca415a0e0 msgr2=0x7f6ca40ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:31.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.805+0000 7f6caa1b2640 1 --2- 172.21.15.81:0/1075949208 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ca415a0e0 0x7f6ca40ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.805+0000 7f6caa1b2640 1 -- 172.21.15.81:0/1075949208 >> v1:172.21.15.93:6789/0 conn(0x7f6ca40c9bf0 legacy=0x7f6ca40ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:31.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.805+0000 7f6caa1b2640 1 -- 172.21.15.81:0/1075949208 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca415a4c0 con 0x7f6ca40ca720 2023-11-08T16:30:31.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.806+0000 7f6caa1b2640 1 --2- 172.21.15.81:0/1075949208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ca40ca720 0x7f6ca406b710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009870 tx=0x7f6c9c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9a2c217d27b04a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:31.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.807+0000 7f6ca89af640 1 -- 172.21.15.81:0/1075949208 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c030840 con 0x7f6ca40ca720 2023-11-08T16:30:31.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.807+0000 7f6ca89af640 1 -- 172.21.15.81:0/1075949208 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c0309a0 con 0x7f6ca40ca720 2023-11-08T16:30:31.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.807+0000 7f6ca89af640 1 -- 172.21.15.81:0/1075949208 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c030d30 con 0x7f6ca40ca720 2023-11-08T16:30:31.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.807+0000 7f6cabc15640 1 -- 172.21.15.81:0/1075949208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ca40ca720 msgr2=0x7f6ca406b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:31.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.807+0000 7f6cabc15640 1 --2- 172.21.15.81:0/1075949208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ca40ca720 0x7f6ca406b710 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c009870 tx=0x7f6c9c02b980 comp rx=0 tx=0).stop 2023-11-08T16:30:31.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.808+0000 7f6cabc15640 1 -- 172.21.15.81:0/1075949208 shutdown_connections 2023-11-08T16:30:31.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.808+0000 7f6cabc15640 1 --2- 172.21.15.81:0/1075949208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ca40ca720 0x7f6ca406b710 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.808+0000 7f6cabc15640 1 --2- 172.21.15.81:0/1075949208 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ca415a0e0 0x7f6ca40ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.808+0000 7f6cabc15640 1 -- 172.21.15.81:0/1075949208 >> 172.21.15.81:0/1075949208 conn(0x7f6ca40cb580 msgr2=0x7f6ca40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:31.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.808+0000 7f6cabc15640 1 -- 172.21.15.81:0/1075949208 shutdown_connections 2023-11-08T16:30:31.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.808+0000 7f6cabc15640 1 -- 172.21.15.81:0/1075949208 wait complete. 2023-11-08T16:30:31.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.810+0000 7f6cabc15640 1 Processor -- start 2023-11-08T16:30:31.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.810+0000 7f6cabc15640 1 -- start start 2023-11-08T16:30:31.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.811+0000 7f6cabc15640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ca40c9bf0 0x7f6ca4153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.811+0000 7f6cabc15640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ca40ca720 0x7f6ca40c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.811+0000 7f6cabc15640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ca415a0e0 0x7f6ca406c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.811+0000 7f6cabc15640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca40c3850 con 0x7f6ca415a0e0 2023-11-08T16:30:31.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.811+0000 7f6cabc15640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41570d0 con 0x7f6ca40c9bf0 2023-11-08T16:30:31.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.811+0000 7f6cabc15640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca41573b0 con 0x7f6ca40ca720 2023-11-08T16:30:31.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.812+0000 7f6caa1b2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ca415a0e0 0x7f6ca406c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:31.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.812+0000 7f6caa1b2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ca415a0e0 0x7f6ca406c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40694/0 (socket says 172.21.15.81:40694) 2023-11-08T16:30:31.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.812+0000 7f6caa1b2640 1 -- 172.21.15.81:0/3050022927 learned_addr learned my addr 172.21.15.81:0/3050022927 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:31.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.812+0000 7f6ca91b0640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ca40ca720 0x7f6ca40c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:31.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.812+0000 7f6ca99b1640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ca40c9bf0 0x7f6ca4153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:31.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.812+0000 7f6caa1b2640 1 -- 172.21.15.81:0/3050022927 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ca40ca720 msgr2=0x7f6ca40c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:31.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.812+0000 7f6caa1b2640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ca40ca720 0x7f6ca40c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.812+0000 7f6caa1b2640 1 -- 172.21.15.81:0/3050022927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ca40c9bf0 msgr2=0x7f6ca4153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:31.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.812+0000 7f6caa1b2640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ca40c9bf0 0x7f6ca4153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:31.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.812+0000 7f6caa1b2640 1 -- 172.21.15.81:0/3050022927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c0094d0 con 0x7f6ca415a0e0 2023-11-08T16:30:31.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.813+0000 7f6caa1b2640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ca415a0e0 0x7f6ca406c320 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c02b9c0 tx=0x7f6c9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:31.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.813+0000 7f6c9a7fc640 1 -- 172.21.15.81:0/3050022927 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c043400 con 0x7f6ca415a0e0 2023-11-08T16:30:31.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.813+0000 7f6c9a7fc640 1 -- 172.21.15.81:0/3050022927 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9c043580 con 0x7f6ca415a0e0 2023-11-08T16:30:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.813+0000 7f6c9a7fc640 1 -- 172.21.15.81:0/3050022927 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c043930 con 0x7f6ca415a0e0 2023-11-08T16:30:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.813+0000 7f6cabc15640 1 -- 172.21.15.81:0/3050022927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca4157690 con 0x7f6ca415a0e0 2023-11-08T16:30:31.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.813+0000 7f6cabc15640 1 -- 172.21.15.81:0/3050022927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca4153740 con 0x7f6ca415a0e0 2023-11-08T16:30:31.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.814+0000 7f6cabc15640 1 -- 172.21.15.81:0/3050022927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ca4157bb0 con 0x7f6ca415a0e0 2023-11-08T16:30:31.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.815+0000 7f6c9a7fc640 1 -- 172.21.15.81:0/3050022927 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c9c043a90 con 0x7f6ca415a0e0 2023-11-08T16:30:31.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.816+0000 7f6c9a7fc640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c8806ae70 0x7f6c8806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:31.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.816+0000 7f6c9a7fc640 1 -- 172.21.15.81:0/3050022927 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c9c03e070 con 0x7f6ca415a0e0 2023-11-08T16:30:31.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.816+0000 7f6ca99b1640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c8806ae70 0x7f6c8806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:31.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.817+0000 7f6ca99b1640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c8806ae70 0x7f6c8806d330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009f20 tx=0x7f6ca00094f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:31.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:31.821+0000 7f6c9a7fc640 1 -- 172.21.15.81:0/3050022927 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6ca4157bb0 con 0x7f6ca415a0e0 2023-11-08T16:30:32.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.046+0000 7f6cabc15640 1 -- 172.21.15.81:0/3050022927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6ca40ce960 con 0x7f6ca415a0e0 2023-11-08T16:30:32.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.812+0000 7f6c9a7fc640 1 -- 172.21.15.81:0/3050022927 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6ca40ce960 con 0x7f6ca415a0e0 2023-11-08T16:30:32.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.816+0000 7f6cabc15640 1 -- 172.21.15.81:0/3050022927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c8806ae70 msgr2=0x7f6c8806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:32.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.816+0000 7f6cabc15640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c8806ae70 0x7f6c8806d330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009f20 tx=0x7f6ca00094f0 comp rx=0 tx=0).stop 2023-11-08T16:30:32.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.816+0000 7f6cabc15640 1 -- 172.21.15.81:0/3050022927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ca415a0e0 msgr2=0x7f6ca406c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:32.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.816+0000 7f6cabc15640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ca415a0e0 0x7f6ca406c320 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c02b9c0 tx=0x7f6c9c004730 comp rx=0 tx=0).stop 2023-11-08T16:30:32.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.817+0000 7f6cabc15640 1 -- 172.21.15.81:0/3050022927 shutdown_connections 2023-11-08T16:30:32.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.817+0000 7f6cabc15640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c8806ae70 0x7f6c8806d330 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:32.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.817+0000 7f6cabc15640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ca415a0e0 0x7f6ca406c320 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:32.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.817+0000 7f6cabc15640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ca40ca720 0x7f6ca40c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:32.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.817+0000 7f6cabc15640 1 --2- 172.21.15.81:0/3050022927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ca40c9bf0 0x7f6ca4153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:32.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.817+0000 7f6cabc15640 1 -- 172.21.15.81:0/3050022927 >> 172.21.15.81:0/3050022927 conn(0x7f6ca40cb580 msgr2=0x7f6ca415c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:32.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.817+0000 7f6cabc15640 1 -- 172.21.15.81:0/3050022927 shutdown_connections 2023-11-08T16:30:32.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:32.817+0000 7f6cabc15640 1 -- 172.21.15.81:0/3050022927 wait complete. 2023-11-08T16:30:32.838 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T16:30:32.839 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T16:30:33.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.049+0000 7fe71486e640 1 Processor -- start 2023-11-08T16:30:33.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.049+0000 7fe71486e640 1 -- start start 2023-11-08T16:30:33.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.051+0000 7fe71486e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe71015e1a0 0x7fe7100caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:33.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.051+0000 7fe71486e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe71015e580 con 0x7fe710070530 2023-11-08T16:30:33.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.051+0000 7fe71486e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7100c9cb0 con 0x7fe7100c97b0 2023-11-08T16:30:33.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.051+0000 7fe71486e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100ca830 con 0x7fe71015e1a0 2023-11-08T16:30:33.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.051+0000 7fe70f59e640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe710070530 0x7fe71006bb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53662/0 (socket says 172.21.15.81:53662) 2023-11-08T16:30:33.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.051+0000 7fe70f59e640 1 -- 172.21.15.81:0/4012636748 learned_addr learned my addr 172.21.15.81:0/4012636748 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:33.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.052+0000 7fe70ed9d640 1 --2- 172.21.15.81:0/4012636748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe71015e1a0 0x7fe7100caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:33.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.052+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2708909696 0 0) 0x7fe71015e580 con 0x7fe710070530 2023-11-08T16:30:33.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.052+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6fc003520 con 0x7fe710070530 2023-11-08T16:30:33.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.052+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 927175572 0 0) 0x7fe6fc003520 con 0x7fe710070530 2023-11-08T16:30:33.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.052+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe71015e1a0 msgr2=0x7fe7100caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:33.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.052+0000 7fe70dd9b640 1 --2- 172.21.15.81:0/4012636748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe71015e1a0 0x7fe7100caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:33.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.052+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 >> v1:172.21.15.93:6789/0 conn(0x7fe7100c97b0 legacy=0x7fe71006fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:33.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.052+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7100c9f90 con 0x7fe710070530 2023-11-08T16:30:33.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.052+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fe7080019f0 con 0x7fe710070530 2023-11-08T16:30:33.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.052+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 >> v1:172.21.15.81:6789/0 conn(0x7fe710070530 legacy=0x7fe71006bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:33.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.052+0000 7fe70dd9b640 1 --2- 172.21.15.81:0/4012636748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc0039a0 0x7fe6fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:33.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70dd9b640 1 --2- 172.21.15.81:0/4012636748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe71015e1a0 0x7fe6fc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:33.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70dd9b640 1 --2- 172.21.15.81:0/4012636748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6fc004d00 0x7fe6fc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:33.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100ca830 con 0x7fe6fc0039a0 2023-11-08T16:30:33.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc00bb10 con 0x7fe6fc004d00 2023-11-08T16:30:33.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6fc00bcd0 con 0x7fe71015e1a0 2023-11-08T16:30:33.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70ed9d640 1 --2- 172.21.15.81:0/4012636748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc0039a0 0x7fe6fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:33.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70ed9d640 1 -- 172.21.15.81:0/4012636748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe71015e1a0 msgr2=0x7fe6fc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:33.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70ed9d640 1 --2- 172.21.15.81:0/4012636748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe71015e1a0 0x7fe6fc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:33.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70ed9d640 1 -- 172.21.15.81:0/4012636748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6fc004d00 msgr2=0x7fe6fc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:33.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70f59e640 1 --2- 172.21.15.81:0/4012636748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6fc004d00 0x7fe6fc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:33.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70ed9d640 1 --2- 172.21.15.81:0/4012636748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6fc004d00 0x7fe6fc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:33.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.053+0000 7fe70ed9d640 1 -- 172.21.15.81:0/4012636748 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6fc003520 con 0x7fe6fc0039a0 2023-11-08T16:30:33.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.054+0000 7fe70ed9d640 1 --2- 172.21.15.81:0/4012636748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc0039a0 0x7fe6fc003dc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe700009ac0 tx=0x7fe6fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f022b2852b6f723 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:33.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.055+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70002c710 con 0x7fe6fc0039a0 2023-11-08T16:30:33.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.055+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70002c870 con 0x7fe6fc0039a0 2023-11-08T16:30:33.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.055+0000 7fe70dd9b640 1 -- 172.21.15.81:0/4012636748 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70002cc00 con 0x7fe6fc0039a0 2023-11-08T16:30:33.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.055+0000 7fe71486e640 1 -- 172.21.15.81:0/4012636748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc0039a0 msgr2=0x7fe6fc003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:33.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.055+0000 7fe71486e640 1 --2- 172.21.15.81:0/4012636748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6fc0039a0 0x7fe6fc003dc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe700009ac0 tx=0x7fe6fc0057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:33.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.055+0000 7fe70f59e640 1 -- 172.21.15.81:0/4012636748 reap_dead start 2023-11-08T16:30:33.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.055+0000 7fe71486e640 1 -- 172.21.15.81:0/4012636748 shutdown_connections 2023-11-08T16:30:33.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.055+0000 7fe71486e640 1 -- 172.21.15.81:0/4012636748 >> 172.21.15.81:0/4012636748 conn(0x7fe7100ce8d0 msgr2=0x7fe7100cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:33.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.055+0000 7fe71486e640 1 -- 172.21.15.81:0/4012636748 shutdown_connections 2023-11-08T16:30:33.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.055+0000 7fe71486e640 1 -- 172.21.15.81:0/4012636748 wait complete. 2023-11-08T16:30:33.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.057+0000 7fe71486e640 1 Processor -- start 2023-11-08T16:30:33.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.057+0000 7fe71486e640 1 -- start start 2023-11-08T16:30:33.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.058+0000 7fe71486e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6fc0039a0 0x7fe7100c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:33.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.058+0000 7fe71486e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe71015ca40 0x7fe71014b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:33.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.058+0000 7fe71486e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7101569f0 0x7fe710156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:33.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.058+0000 7fe71486e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7101573b0 con 0x7fe71015ca40 2023-11-08T16:30:33.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.058+0000 7fe71486e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71014bea0 con 0x7fe7101569f0 2023-11-08T16:30:33.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.058+0000 7fe71486e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7100c6b70 con 0x7fe6fc0039a0 2023-11-08T16:30:33.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.058+0000 7fe70e59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe71015ca40 0x7fe71014b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:33.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.058+0000 7fe70e59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe71015ca40 0x7fe71014b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40722/0 (socket says 172.21.15.81:40722) 2023-11-08T16:30:33.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.058+0000 7fe70e59c640 1 -- 172.21.15.81:0/1052461375 learned_addr learned my addr 172.21.15.81:0/1052461375 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:33.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.059+0000 7fe70ed9d640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6fc0039a0 0x7fe7100c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:33.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.059+0000 7fe70f59e640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7101569f0 0x7fe710156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:33.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.059+0000 7fe70e59c640 1 -- 172.21.15.81:0/1052461375 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6fc0039a0 msgr2=0x7fe7100c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:33.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.059+0000 7fe70e59c640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6fc0039a0 0x7fe7100c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:33.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.059+0000 7fe70e59c640 1 -- 172.21.15.81:0/1052461375 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7101569f0 msgr2=0x7fe710156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:33.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.059+0000 7fe70e59c640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7101569f0 0x7fe710156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:33.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.059+0000 7fe70e59c640 1 -- 172.21.15.81:0/1052461375 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7100c6e50 con 0x7fe71015ca40 2023-11-08T16:30:33.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.059+0000 7fe70e59c640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe71015ca40 0x7fe71014b5e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8001f00 tx=0x7fe6f800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:33.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.059+0000 7fe7077fe640 1 -- 172.21.15.81:0/1052461375 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80052d0 con 0x7fe71015ca40 2023-11-08T16:30:33.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.060+0000 7fe7077fe640 1 -- 172.21.15.81:0/1052461375 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8005430 con 0x7fe71015ca40 2023-11-08T16:30:33.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.060+0000 7fe71486e640 1 -- 172.21.15.81:0/1052461375 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7100c7130 con 0x7fe71015ca40 2023-11-08T16:30:33.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.060+0000 7fe7077fe640 1 -- 172.21.15.81:0/1052461375 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8003880 con 0x7fe71015ca40 2023-11-08T16:30:33.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.060+0000 7fe71486e640 1 -- 172.21.15.81:0/1052461375 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71014bb20 con 0x7fe71015ca40 2023-11-08T16:30:33.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.061+0000 7fe71486e640 1 -- 172.21.15.81:0/1052461375 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71004ece0 con 0x7fe71015ca40 2023-11-08T16:30:33.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.062+0000 7fe7077fe640 1 -- 172.21.15.81:0/1052461375 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6f80039e0 con 0x7fe71015ca40 2023-11-08T16:30:33.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.063+0000 7fe7077fe640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6f006ae70 0x7fe6f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:33.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.063+0000 7fe7077fe640 1 -- 172.21.15.81:0/1052461375 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6f8089150 con 0x7fe71015ca40 2023-11-08T16:30:33.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.063+0000 7fe70ed9d640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6f006ae70 0x7fe6f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:33.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.064+0000 7fe70ed9d640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6f006ae70 0x7fe6f006d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe700036000 tx=0x7fe700030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:33.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.068+0000 7fe7077fe640 1 -- 172.21.15.81:0/1052461375 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6f80541f0 con 0x7fe71015ca40 2023-11-08T16:30:33.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.293+0000 7fe71486e640 1 -- 172.21.15.81:0/1052461375 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe7100cf480 con 0x7fe71015ca40 2023-11-08T16:30:33.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.816+0000 7fe7077fe640 1 -- 172.21.15.81:0/1052461375 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe6f8057810 con 0x7fe71015ca40 2023-11-08T16:30:33.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.819+0000 7fe71486e640 1 -- 172.21.15.81:0/1052461375 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6f006ae70 msgr2=0x7fe6f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:33.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.819+0000 7fe71486e640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6f006ae70 0x7fe6f006d330 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe700036000 tx=0x7fe700030000 comp rx=0 tx=0).stop 2023-11-08T16:30:33.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.819+0000 7fe71486e640 1 -- 172.21.15.81:0/1052461375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe71015ca40 msgr2=0x7fe71014b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:33.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.819+0000 7fe71486e640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe71015ca40 0x7fe71014b5e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8001f00 tx=0x7fe6f800c960 comp rx=0 tx=0).stop 2023-11-08T16:30:33.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.820+0000 7fe71486e640 1 -- 172.21.15.81:0/1052461375 shutdown_connections 2023-11-08T16:30:33.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.820+0000 7fe71486e640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6f006ae70 0x7fe6f006d330 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:33.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.820+0000 7fe71486e640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7101569f0 0x7fe710156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:33.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.820+0000 7fe71486e640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe71015ca40 0x7fe71014b5e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:33.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.820+0000 7fe71486e640 1 --2- 172.21.15.81:0/1052461375 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6fc0039a0 0x7fe7100c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:33.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.820+0000 7fe71486e640 1 -- 172.21.15.81:0/1052461375 >> 172.21.15.81:0/1052461375 conn(0x7fe7100ce8d0 msgr2=0x7fe71006ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:33.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.820+0000 7fe71486e640 1 -- 172.21.15.81:0/1052461375 shutdown_connections 2023-11-08T16:30:33.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:33.820+0000 7fe71486e640 1 -- 172.21.15.81:0/1052461375 wait complete. 2023-11-08T16:30:33.841 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T16:30:33.842 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:30:34.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.044+0000 7f553d434640 1 Processor -- start 2023-11-08T16:30:34.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.044+0000 7f553d434640 1 -- start start 2023-11-08T16:30:34.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.046+0000 7f553d434640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f55380adcd0 0x7f55380ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.046+0000 7f553d434640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55380c9a70 con 0x7f55380cc5e0 2023-11-08T16:30:34.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.046+0000 7f553d434640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55380cc3e0 con 0x7f5538070050 2023-11-08T16:30:34.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.046+0000 7f553d434640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f553806d070 con 0x7f55380adcd0 2023-11-08T16:30:34.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.046+0000 7f552e7fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f55380cc5e0 0x7f55380cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53664/0 (socket says 172.21.15.81:53664) 2023-11-08T16:30:34.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.046+0000 7f552e7fc640 1 -- 172.21.15.81:0/1571267147 learned_addr learned my addr 172.21.15.81:0/1571267147 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:34.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f5536ffd640 1 --2- 172.21.15.81:0/1571267147 >> v2:172.21.15.93:3300/0 conn(0x7f55380adcd0 0x7f55380ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506168382 0 0) 0x7f55380c9a70 con 0x7f55380cc5e0 2023-11-08T16:30:34.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5524003520 con 0x7f55380cc5e0 2023-11-08T16:30:34.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 503184451 0 0) 0x7f5524003520 con 0x7f55380cc5e0 2023-11-08T16:30:34.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 >> v2:172.21.15.93:3300/0 conn(0x7f55380adcd0 msgr2=0x7f55380ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:34.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f55367fc640 1 --2- 172.21.15.81:0/1571267147 >> v2:172.21.15.93:3300/0 conn(0x7f55380adcd0 0x7f55380ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 >> v1:172.21.15.93:6789/0 conn(0x7f5538070050 legacy=0x7f55380704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f553806d350 con 0x7f55380cc5e0 2023-11-08T16:30:34.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f5520001670 con 0x7f55380cc5e0 2023-11-08T16:30:34.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f5536ffd640 1 --2- 172.21.15.81:0/1571267147 >> v2:172.21.15.93:3300/0 conn(0x7f55380adcd0 0x7f55380ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:30:34.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 >> v1:172.21.15.81:6789/0 conn(0x7f55380cc5e0 legacy=0x7f55380cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f55367fc640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5524003aa0 0x7f5524003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.047+0000 7f55367fc640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55240047c0 0x7f5524004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f55367fc640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5524005380 0x7f552400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f553806d070 con 0x7f55240047c0 2023-11-08T16:30:34.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552400c1f0 con 0x7f5524005380 2023-11-08T16:30:34.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552400e5c0 con 0x7f5524003aa0 2023-11-08T16:30:34.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f552e7fc640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55240047c0 0x7f5524004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f55377fe640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5524005380 0x7f552400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f5536ffd640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5524003aa0 0x7f5524003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f552e7fc640 1 -- 172.21.15.81:0/1571267147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5524003aa0 msgr2=0x7f5524003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:34.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f552e7fc640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5524003aa0 0x7f5524003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f552e7fc640 1 -- 172.21.15.81:0/1571267147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5524005380 msgr2=0x7f552400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:34.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f552e7fc640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5524005380 0x7f552400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f552e7fc640 1 -- 172.21.15.81:0/1571267147 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5524003520 con 0x7f55240047c0 2023-11-08T16:30:34.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f55377fe640 1 -- 172.21.15.81:0/1571267147 reap_dead start 2023-11-08T16:30:34.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.048+0000 7f5536ffd640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5524003aa0 0x7f5524003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:34.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.049+0000 7f552e7fc640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55240047c0 0x7f5524004c40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f552000c0e0 tx=0x7f55240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5553fe60a5744784 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:34.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.050+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5520005ad0 con 0x7f55240047c0 2023-11-08T16:30:34.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.050+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5520005c30 con 0x7f55240047c0 2023-11-08T16:30:34.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.050+0000 7f55367fc640 1 -- 172.21.15.81:0/1571267147 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552002ca30 con 0x7f55240047c0 2023-11-08T16:30:34.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.050+0000 7f553d434640 1 -- 172.21.15.81:0/1571267147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55240047c0 msgr2=0x7f5524004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.050+0000 7f553d434640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55240047c0 0x7f5524004c40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f552000c0e0 tx=0x7f55240057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.050+0000 7f553d434640 1 -- 172.21.15.81:0/1571267147 shutdown_connections 2023-11-08T16:30:34.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.050+0000 7f553d434640 1 --2- 172.21.15.81:0/1571267147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55240047c0 0x7f5524004c40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.050+0000 7f553d434640 1 -- 172.21.15.81:0/1571267147 >> 172.21.15.81:0/1571267147 conn(0x7f55380cb490 msgr2=0x7f55380cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:34.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.050+0000 7f553d434640 1 -- 172.21.15.81:0/1571267147 shutdown_connections 2023-11-08T16:30:34.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.050+0000 7f553d434640 1 -- 172.21.15.81:0/1571267147 wait complete. 2023-11-08T16:30:34.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.052+0000 7f553d434640 1 Processor -- start 2023-11-08T16:30:34.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.052+0000 7f553d434640 1 -- start start 2023-11-08T16:30:34.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.053+0000 7f553d434640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55240047c0 0x7f55380b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.053+0000 7f553d434640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f553815afe0 0x7f55380c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.053+0000 7f553d434640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55380c9580 0x7f55381eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.053+0000 7f553d434640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380b2e70 con 0x7f553815afe0 2023-11-08T16:30:34.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.053+0000 7f553d434640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55380c99e0 con 0x7f55240047c0 2023-11-08T16:30:34.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.053+0000 7f553d434640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553815b460 con 0x7f55380c9580 2023-11-08T16:30:34.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.053+0000 7f552e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f553815afe0 0x7f55380c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.054+0000 7f552e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f553815afe0 0x7f55380c9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40754/0 (socket says 172.21.15.81:40754) 2023-11-08T16:30:34.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.054+0000 7f552e7fc640 1 -- 172.21.15.81:0/2855667289 learned_addr learned my addr 172.21.15.81:0/2855667289 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:34.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.054+0000 7f5536ffd640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55240047c0 0x7f55380b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.054+0000 7f55377fe640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55380c9580 0x7f55381eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.054+0000 7f552e7fc640 1 -- 172.21.15.81:0/2855667289 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55380c9580 msgr2=0x7f55381eeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.054+0000 7f552e7fc640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55380c9580 0x7f55381eeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.054+0000 7f552e7fc640 1 -- 172.21.15.81:0/2855667289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55240047c0 msgr2=0x7f55380b3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.054+0000 7f552e7fc640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55240047c0 0x7f55380b3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.054+0000 7f552e7fc640 1 -- 172.21.15.81:0/2855667289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55381ef880 con 0x7f553815afe0 2023-11-08T16:30:34.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.054+0000 7f552e7fc640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f553815afe0 0x7f55380c9040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f552000c0e0 tx=0x7f552002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:34.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.055+0000 7f552ffff640 1 -- 172.21.15.81:0/2855667289 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552003c040 con 0x7f553815afe0 2023-11-08T16:30:34.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.055+0000 7f552ffff640 1 -- 172.21.15.81:0/2855667289 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552002cce0 con 0x7f553815afe0 2023-11-08T16:30:34.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.055+0000 7f552ffff640 1 -- 172.21.15.81:0/2855667289 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55200072b0 con 0x7f553815afe0 2023-11-08T16:30:34.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.055+0000 7f553d434640 1 -- 172.21.15.81:0/2855667289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55380b27b0 con 0x7f553815afe0 2023-11-08T16:30:34.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.055+0000 7f553d434640 1 -- 172.21.15.81:0/2855667289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55381ef060 con 0x7f553815afe0 2023-11-08T16:30:34.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.056+0000 7f553d434640 1 -- 172.21.15.81:0/2855667289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55380b30e0 con 0x7f553815afe0 2023-11-08T16:30:34.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.057+0000 7f552ffff640 1 -- 172.21.15.81:0/2855667289 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5520032070 con 0x7f553815afe0 2023-11-08T16:30:34.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.057+0000 7f552ffff640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f551806aea0 0x7f551806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.058+0000 7f552ffff640 1 -- 172.21.15.81:0/2855667289 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55200aaa20 con 0x7f553815afe0 2023-11-08T16:30:34.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.058+0000 7f5536ffd640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f551806aea0 0x7f551806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.058+0000 7f5536ffd640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f551806aea0 0x7f551806d360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5528006fd0 tx=0x7f5528008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:34.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.062+0000 7f552ffff640 1 -- 172.21.15.81:0/2855667289 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5520075ac0 con 0x7f553815afe0 2023-11-08T16:30:34.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.290+0000 7f553d434640 1 -- 172.21.15.81:0/2855667289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5538072170 con 0x7f553815afe0 2023-11-08T16:30:34.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.291+0000 7f552ffff640 1 -- 172.21.15.81:0/2855667289 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4907 (secure 0 0 0) 0x7f55200790e0 con 0x7f553815afe0 2023-11-08T16:30:34.293 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:34.293 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/1453041994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":1453041994},{"type":"v1","addr":"172.21.15.93:6835","nonce":1453041994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/757624701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":757624701},{"type":"v1","addr":"172.21.15.93:6833","nonce":757624701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1735232404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1735232404},{"type":"v1","addr":"172.21.15.81:6835","nonce":1735232404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:30:30.141674+0000","modified":"2023-11-08T16:30:33.294977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5139":{"gid":5139,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/773130363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":773130363},{"type":"v1","addr":"172.21.15.81:6837","nonce":773130363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:30:34.294 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 18 2023-11-08T16:30:34.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.293+0000 7f553d434640 1 -- 172.21.15.81:0/2855667289 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f551806aea0 msgr2=0x7f551806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.293+0000 7f553d434640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f551806aea0 0x7f551806d360 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f5528006fd0 tx=0x7f5528008040 comp rx=0 tx=0).stop 2023-11-08T16:30:34.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.293+0000 7f553d434640 1 -- 172.21.15.81:0/2855667289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f553815afe0 msgr2=0x7f55380c9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.293+0000 7f553d434640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f553815afe0 0x7f55380c9040 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f552000c0e0 tx=0x7f552002e070 comp rx=0 tx=0).stop 2023-11-08T16:30:34.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.294+0000 7f553d434640 1 -- 172.21.15.81:0/2855667289 shutdown_connections 2023-11-08T16:30:34.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.294+0000 7f553d434640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f551806aea0 0x7f551806d360 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.294+0000 7f553d434640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55380c9580 0x7f55381eeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.294+0000 7f553d434640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f553815afe0 0x7f55380c9040 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.294+0000 7f553d434640 1 --2- 172.21.15.81:0/2855667289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55240047c0 0x7f55380b3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.294+0000 7f553d434640 1 -- 172.21.15.81:0/2855667289 >> 172.21.15.81:0/2855667289 conn(0x7f55380cb490 msgr2=0x7f55380af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:34.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.294+0000 7f553d434640 1 -- 172.21.15.81:0/2855667289 shutdown_connections 2023-11-08T16:30:34.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.294+0000 7f553d434640 1 -- 172.21.15.81:0/2855667289 wait complete. 2023-11-08T16:30:34.312 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:30:34.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.522+0000 7fe43abb8640 1 Processor -- start 2023-11-08T16:30:34.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.522+0000 7fe43abb8640 1 -- start start 2023-11-08T16:30:34.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.524+0000 7fe43abb8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe4340ced10 0x7fe4340cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.524+0000 7fe43abb8640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe43406bcd0 0x7fe43406c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.524+0000 7fe43abb8640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe43406b860 con 0x7fe43415ab70 2023-11-08T16:30:34.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.524+0000 7fe43abb8640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe43415e450 con 0x7fe4340ced10 2023-11-08T16:30:34.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.524+0000 7fe43abb8640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43415d840 con 0x7fe43406bcd0 2023-11-08T16:30:34.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.524+0000 7fe4393b5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe4340ced10 0x7fe4340cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.524+0000 7fe4393b5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe4340ced10 0x7fe4340cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40760/0 (socket says 172.21.15.81:40760) 2023-11-08T16:30:34.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.524+0000 7fe4393b5640 1 -- 172.21.15.81:0/3514502146 learned_addr learned my addr 172.21.15.81:0/3514502146 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:34.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.525+0000 7fe43a3b7640 1 --2- 172.21.15.81:0/3514502146 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe43406bcd0 0x7fe43406c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.525+0000 7fe4393b5640 1 -- 172.21.15.81:0/3514502146 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe43406bcd0 msgr2=0x7fe43406c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:34.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.525+0000 7fe4393b5640 1 --2- 172.21.15.81:0/3514502146 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe43406bcd0 0x7fe43406c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.525+0000 7fe4393b5640 1 -- 172.21.15.81:0/3514502146 >> v1:172.21.15.93:6789/0 conn(0x7fe43415ab70 legacy=0x7fe4341613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:34.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.525+0000 7fe4393b5640 1 -- 172.21.15.81:0/3514502146 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4340cf6b0 con 0x7fe4340ced10 2023-11-08T16:30:34.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.526+0000 7fe4393b5640 1 --2- 172.21.15.81:0/3514502146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4340ced10 0x7fe4340cf170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe424009870 tx=0x7fe42402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72d633f0ced781 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:34.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.526+0000 7fe438bb4640 1 -- 172.21.15.81:0/3514502146 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424031640 con 0x7fe4340ced10 2023-11-08T16:30:34.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.526+0000 7fe438bb4640 1 -- 172.21.15.81:0/3514502146 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4240317a0 con 0x7fe4340ced10 2023-11-08T16:30:34.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.526+0000 7fe438bb4640 1 -- 172.21.15.81:0/3514502146 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe424031b30 con 0x7fe4340ced10 2023-11-08T16:30:34.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.527+0000 7fe43abb8640 1 -- 172.21.15.81:0/3514502146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4340ced10 msgr2=0x7fe4340cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.527+0000 7fe43abb8640 1 --2- 172.21.15.81:0/3514502146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4340ced10 0x7fe4340cf170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe424009870 tx=0x7fe42402b8f0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.527+0000 7fe43abb8640 1 -- 172.21.15.81:0/3514502146 shutdown_connections 2023-11-08T16:30:34.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.527+0000 7fe43abb8640 1 --2- 172.21.15.81:0/3514502146 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe43406bcd0 0x7fe43406c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.527+0000 7fe43abb8640 1 --2- 172.21.15.81:0/3514502146 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4340ced10 0x7fe4340cf170 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.527+0000 7fe43abb8640 1 -- 172.21.15.81:0/3514502146 >> 172.21.15.81:0/3514502146 conn(0x7fe43406a4a0 msgr2=0x7fe43406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:34.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.527+0000 7fe43abb8640 1 -- 172.21.15.81:0/3514502146 shutdown_connections 2023-11-08T16:30:34.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.527+0000 7fe43abb8640 1 -- 172.21.15.81:0/3514502146 wait complete. 2023-11-08T16:30:34.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.529+0000 7fe43abb8640 1 Processor -- start 2023-11-08T16:30:34.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.529+0000 7fe43abb8640 1 -- start start 2023-11-08T16:30:34.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.530+0000 7fe43abb8640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe43406bcd0 0x7fe434156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.530+0000 7fe43abb8640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4340ced10 0x7fe434148350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.530+0000 7fe43abb8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe43415ab70 0x7fe43415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.530+0000 7fe43abb8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe434159c40 con 0x7fe43415ab70 2023-11-08T16:30:34.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.530+0000 7fe43abb8640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe434156e60 con 0x7fe4340ced10 2023-11-08T16:30:34.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.530+0000 7fe43abb8640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe434148890 con 0x7fe43406bcd0 2023-11-08T16:30:34.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.530+0000 7fe43a3b7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe43415ab70 0x7fe43415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.530+0000 7fe43a3b7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe43415ab70 0x7fe43415c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40772/0 (socket says 172.21.15.81:40772) 2023-11-08T16:30:34.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.530+0000 7fe43a3b7640 1 -- 172.21.15.81:0/1776854789 learned_addr learned my addr 172.21.15.81:0/1776854789 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:34.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.530+0000 7fe4393b5640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4340ced10 0x7fe434148350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe439bb6640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe43406bcd0 0x7fe434156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe43a3b7640 1 -- 172.21.15.81:0/1776854789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe43406bcd0 msgr2=0x7fe434156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe43a3b7640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe43406bcd0 0x7fe434156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe43a3b7640 1 -- 172.21.15.81:0/1776854789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4340ced10 msgr2=0x7fe434148350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe43a3b7640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4340ced10 0x7fe434148350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe43a3b7640 1 -- 172.21.15.81:0/1776854789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4240094d0 con 0x7fe43415ab70 2023-11-08T16:30:34.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe439bb6640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe43406bcd0 0x7fe434156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:34.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe4393b5640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4340ced10 0x7fe434148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:34.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe43a3b7640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe43415ab70 0x7fe43415c090 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe42c007e60 tx=0x7fe42c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:34.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe42a7fc640 1 -- 172.21.15.81:0/1776854789 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c014420 con 0x7fe43415ab70 2023-11-08T16:30:34.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe42a7fc640 1 -- 172.21.15.81:0/1776854789 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c0145a0 con 0x7fe43415ab70 2023-11-08T16:30:34.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe42a7fc640 1 -- 172.21.15.81:0/1776854789 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c014950 con 0x7fe43415ab70 2023-11-08T16:30:34.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe43abb8640 1 -- 172.21.15.81:0/1776854789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43415bc00 con 0x7fe43415ab70 2023-11-08T16:30:34.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.531+0000 7fe43abb8640 1 -- 172.21.15.81:0/1776854789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43415b2d0 con 0x7fe43415ab70 2023-11-08T16:30:34.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.532+0000 7fe43abb8640 1 -- 172.21.15.81:0/1776854789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43415bd90 con 0x7fe43415ab70 2023-11-08T16:30:34.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.533+0000 7fe42a7fc640 1 -- 172.21.15.81:0/1776854789 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe42c014ab0 con 0x7fe43415ab70 2023-11-08T16:30:34.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.534+0000 7fe42a7fc640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe41806ae70 0x7fe41806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.534+0000 7fe42a7fc640 1 -- 172.21.15.81:0/1776854789 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe42c08a110 con 0x7fe43415ab70 2023-11-08T16:30:34.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.535+0000 7fe439bb6640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe41806ae70 0x7fe41806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.535+0000 7fe439bb6640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe41806ae70 0x7fe41806d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe430003eb0 tx=0x7fe430009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:34.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.539+0000 7fe42a7fc640 1 -- 172.21.15.81:0/1776854789 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe42c055090 con 0x7fe43415ab70 2023-11-08T16:30:34.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.732+0000 7fe43abb8640 1 -- 172.21.15.81:0/1776854789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe434067800 con 0x7fe43415ab70 2023-11-08T16:30:34.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.735+0000 7fe42a7fc640 1 -- 172.21.15.81:0/1776854789 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16509 (secure 0 0 0) 0x7fe42c0586b0 con 0x7fe43415ab70 2023-11-08T16:30:34.737 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:34.737 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":35,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:30:31.716901+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:30:27.977481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T16:30:29.171902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:30:34.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.737+0000 7fe43abb8640 1 -- 172.21.15.81:0/1776854789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe41806ae70 msgr2=0x7fe41806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.737+0000 7fe43abb8640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe41806ae70 0x7fe41806d330 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe430003eb0 tx=0x7fe430009040 comp rx=0 tx=0).stop 2023-11-08T16:30:34.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.737+0000 7fe43abb8640 1 -- 172.21.15.81:0/1776854789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe43415ab70 msgr2=0x7fe43415c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.737+0000 7fe43abb8640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe43415ab70 0x7fe43415c090 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe42c007e60 tx=0x7fe42c00e4b0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.737+0000 7fe43abb8640 1 -- 172.21.15.81:0/1776854789 shutdown_connections 2023-11-08T16:30:34.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.737+0000 7fe43abb8640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe41806ae70 0x7fe41806d330 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.737+0000 7fe43abb8640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe43415ab70 0x7fe43415c090 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.737+0000 7fe43abb8640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4340ced10 0x7fe434148350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.737+0000 7fe43abb8640 1 --2- 172.21.15.81:0/1776854789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe43406bcd0 0x7fe434156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.737+0000 7fe43abb8640 1 -- 172.21.15.81:0/1776854789 >> 172.21.15.81:0/1776854789 conn(0x7fe43406a4a0 msgr2=0x7fe4340712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:34.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.738+0000 7fe43abb8640 1 -- 172.21.15.81:0/1776854789 shutdown_connections 2023-11-08T16:30:34.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.738+0000 7fe43abb8640 1 -- 172.21.15.81:0/1776854789 wait complete. 2023-11-08T16:30:34.759 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T16:30:34.761 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:30:34.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.972+0000 7fa7f80e5640 1 Processor -- start 2023-11-08T16:30:34.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.973+0000 7fa7f80e5640 1 -- start start 2023-11-08T16:30:34.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.974+0000 7fa7f80e5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa7f00adcd0 0x7fa7f00ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.974+0000 7fa7f80e5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa7f00cc5e0 0x7fa7f00cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.974+0000 7fa7f80e5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7f00ccfb0 0x7fa7f015db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.974+0000 7fa7f80e5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f00c9a70 con 0x7fa7f00adcd0 2023-11-08T16:30:34.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.974+0000 7fa7f80e5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7f00cc3e0 con 0x7fa7f00cc5e0 2023-11-08T16:30:34.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.974+0000 7fa7f80e5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f006d070 con 0x7fa7f00ccfb0 2023-11-08T16:30:34.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.974+0000 7fa7f5e81640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa7f00adcd0 0x7fa7f00ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.974+0000 7fa7f5e81640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa7f00adcd0 0x7fa7f00ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40780/0 (socket says 172.21.15.81:40780) 2023-11-08T16:30:34.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.974+0000 7fa7f5e81640 1 -- 172.21.15.81:0/2616443246 learned_addr learned my addr 172.21.15.81:0/2616443246 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:34.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.975+0000 7fa7f5680640 1 --2- 172.21.15.81:0/2616443246 >> v2:172.21.15.93:3300/0 conn(0x7fa7f00cc5e0 0x7fa7f00cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.975+0000 7fa7f5e81640 1 -- 172.21.15.81:0/2616443246 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7f00ccfb0 msgr2=0x7fa7f015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:34.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.975+0000 7fa7f5e81640 1 --2- 172.21.15.81:0/2616443246 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7f00ccfb0 0x7fa7f015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.975+0000 7fa7f5e81640 1 -- 172.21.15.81:0/2616443246 >> v2:172.21.15.93:3300/0 conn(0x7fa7f00cc5e0 msgr2=0x7fa7f00cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:34.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.975+0000 7fa7f5e81640 1 --2- 172.21.15.81:0/2616443246 >> v2:172.21.15.93:3300/0 conn(0x7fa7f00cc5e0 0x7fa7f00cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.975+0000 7fa7f5e81640 1 -- 172.21.15.81:0/2616443246 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7f006d350 con 0x7fa7f00adcd0 2023-11-08T16:30:34.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.975+0000 7fa7f5680640 1 --2- 172.21.15.81:0/2616443246 >> v2:172.21.15.93:3300/0 conn(0x7fa7f00cc5e0 0x7fa7f00cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:34.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.976+0000 7fa7f5e81640 1 --2- 172.21.15.81:0/2616443246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7f00adcd0 0x7fa7f00ca1e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00cc0d0 tx=0x7fa7e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f55561939ba51aa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:34.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.976+0000 7fa7f4e7f640 1 -- 172.21.15.81:0/2616443246 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0031870 con 0x7fa7f00adcd0 2023-11-08T16:30:34.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.976+0000 7fa7f4e7f640 1 -- 172.21.15.81:0/2616443246 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7e00319d0 con 0x7fa7f00adcd0 2023-11-08T16:30:34.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.976+0000 7fa7f4e7f640 1 -- 172.21.15.81:0/2616443246 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0031d60 con 0x7fa7f00adcd0 2023-11-08T16:30:34.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.977+0000 7fa7f80e5640 1 -- 172.21.15.81:0/2616443246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7f00adcd0 msgr2=0x7fa7f00ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.977+0000 7fa7f80e5640 1 --2- 172.21.15.81:0/2616443246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7f00adcd0 0x7fa7f00ca1e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa7f00cc0d0 tx=0x7fa7e002b910 comp rx=0 tx=0).stop 2023-11-08T16:30:34.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.977+0000 7fa7f80e5640 1 -- 172.21.15.81:0/2616443246 shutdown_connections 2023-11-08T16:30:34.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.977+0000 7fa7f80e5640 1 --2- 172.21.15.81:0/2616443246 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7f00ccfb0 0x7fa7f015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.977+0000 7fa7f80e5640 1 --2- 172.21.15.81:0/2616443246 >> v2:172.21.15.93:3300/0 conn(0x7fa7f00cc5e0 0x7fa7f00cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.977+0000 7fa7f80e5640 1 --2- 172.21.15.81:0/2616443246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7f00adcd0 0x7fa7f00ca1e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.977+0000 7fa7f80e5640 1 -- 172.21.15.81:0/2616443246 >> 172.21.15.81:0/2616443246 conn(0x7fa7f00cb490 msgr2=0x7fa7f00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:34.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.977+0000 7fa7f80e5640 1 -- 172.21.15.81:0/2616443246 shutdown_connections 2023-11-08T16:30:34.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.977+0000 7fa7f80e5640 1 -- 172.21.15.81:0/2616443246 wait complete. 2023-11-08T16:30:34.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.979+0000 7fa7f80e5640 1 Processor -- start 2023-11-08T16:30:34.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.979+0000 7fa7f80e5640 1 -- start start 2023-11-08T16:30:34.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.980+0000 7fa7f80e5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7f00adcd0 0x7fa7f013ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.980+0000 7fa7f80e5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7f00cc5e0 0x7fa7f00b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.980+0000 7fa7f80e5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7f00ccfb0 0x7fa7f015ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.980+0000 7fa7f80e5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f013d290 con 0x7fa7f00cc5e0 2023-11-08T16:30:34.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.980+0000 7fa7f80e5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f013d570 con 0x7fa7f00ccfb0 2023-11-08T16:30:34.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.980+0000 7fa7f80e5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f013d850 con 0x7fa7f00adcd0 2023-11-08T16:30:34.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.980+0000 7fa7f5680640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7f00cc5e0 0x7fa7f00b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.980+0000 7fa7f5680640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7f00cc5e0 0x7fa7f00b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40792/0 (socket says 172.21.15.81:40792) 2023-11-08T16:30:34.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.980+0000 7fa7f5680640 1 -- 172.21.15.81:0/1737665237 learned_addr learned my addr 172.21.15.81:0/1737665237 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:34.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7f5e81640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7f00adcd0 0x7fa7f013ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7f6682640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7f00ccfb0 0x7fa7f015ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7f5680640 1 -- 172.21.15.81:0/1737665237 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7f00adcd0 msgr2=0x7fa7f013ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7f5680640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7f00adcd0 0x7fa7f013ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7f5680640 1 -- 172.21.15.81:0/1737665237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7f00ccfb0 msgr2=0x7fa7f015ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:34.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7f5680640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7f00ccfb0 0x7fa7f015ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:34.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7f5680640 1 -- 172.21.15.81:0/1737665237 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00094d0 con 0x7fa7f00cc5e0 2023-11-08T16:30:34.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7f6682640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7f00ccfb0 0x7fa7f015ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:34.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7f5680640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7f00cc5e0 0x7fa7f00b3830 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00c7f0 tx=0x7fa7ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:34.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7e67fc640 1 -- 172.21.15.81:0/1737665237 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec007820 con 0x7fa7f00cc5e0 2023-11-08T16:30:34.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7e67fc640 1 -- 172.21.15.81:0/1737665237 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ec007980 con 0x7fa7f00cc5e0 2023-11-08T16:30:34.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7e67fc640 1 -- 172.21.15.81:0/1737665237 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ec007cf0 con 0x7fa7f00cc5e0 2023-11-08T16:30:34.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7f80e5640 1 -- 172.21.15.81:0/1737665237 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f013db30 con 0x7fa7f00cc5e0 2023-11-08T16:30:34.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.981+0000 7fa7f80e5640 1 -- 172.21.15.81:0/1737665237 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f00b3d70 con 0x7fa7f00cc5e0 2023-11-08T16:30:34.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.983+0000 7fa7f80e5640 1 -- 172.21.15.81:0/1737665237 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f0072170 con 0x7fa7f00cc5e0 2023-11-08T16:30:34.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.983+0000 7fa7e67fc640 1 -- 172.21.15.81:0/1737665237 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7ec0113f0 con 0x7fa7f00cc5e0 2023-11-08T16:30:34.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.984+0000 7fa7e67fc640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7dc06ae70 0x7fa7dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:34.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.984+0000 7fa7e67fc640 1 -- 172.21.15.81:0/1737665237 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7ec0899c0 con 0x7fa7f00cc5e0 2023-11-08T16:30:34.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.985+0000 7fa7f5e81640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7dc06ae70 0x7fa7dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:34.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.985+0000 7fa7f5e81640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7dc06ae70 0x7fa7dc06d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0036000 tx=0x7fa7e003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:34.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:34.989+0000 7fa7e67fc640 1 -- 172.21.15.81:0/1737665237 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7ec0549d0 con 0x7fa7f00cc5e0 2023-11-08T16:30:35.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.175+0000 7fa7f80e5640 1 -- 172.21.15.81:0/1737665237 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7f00ad780 con 0x7fa7dc06ae70 2023-11-08T16:30:35.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.200+0000 7fa7e67fc640 1 -- 172.21.15.81:0/1737665237 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286473 (secure 0 0 0) 0x7fa7f00ad780 con 0x7fa7dc06ae70 2023-11-08T16:30:35.208 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:35.210 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:30:35.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 -- 172.21.15.81:0/1737665237 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7dc06ae70 msgr2=0x7fa7dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:35.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7dc06ae70 0x7fa7dc06d330 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0036000 tx=0x7fa7e003c040 comp rx=0 tx=0).stop 2023-11-08T16:30:35.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 -- 172.21.15.81:0/1737665237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7f00cc5e0 msgr2=0x7fa7f00b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:35.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7f00cc5e0 0x7fa7f00b3830 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec00c7f0 tx=0x7fa7ec00ccb0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 -- 172.21.15.81:0/1737665237 shutdown_connections 2023-11-08T16:30:35.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7dc06ae70 0x7fa7dc06d330 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7f00ccfb0 0x7fa7f015ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7f00cc5e0 0x7fa7f00b3830 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 --2- 172.21.15.81:0/1737665237 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7f00adcd0 0x7fa7f013ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 -- 172.21.15.81:0/1737665237 >> 172.21.15.81:0/1737665237 conn(0x7fa7f00cb490 msgr2=0x7fa7f00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:35.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 -- 172.21.15.81:0/1737665237 shutdown_connections 2023-11-08T16:30:35.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.206+0000 7fa7f80e5640 1 -- 172.21.15.81:0/1737665237 wait complete. 2023-11-08T16:30:35.228 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-11-08T16:30:34.574988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220440,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754280,"statfs":{"total":773094113280,"available":772868382720,"internally_reserved":0,"allocated":4333568,"data_stored":2389076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.900592"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.738655+0000","last_change":"2023-11-08T16:30:30.734047+0000","last_active":"2023-11-08T16:30:30.738655+0000","last_peered":"2023-11-08T16:30:30.738655+0000","last_clean":"2023-11-08T16:30:30.738655+0000","last_became_active":"2023-11-08T16:30:30.733209+0000","last_became_peered":"2023-11-08T16:30:30.733209+0000","last_unstale":"2023-11-08T16:30:30.738655+0000","last_undegraded":"2023-11-08T16:30:30.738655+0000","last_fullsized":"2023-11-08T16:30:30.738655+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:09:34.574098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751952+0000","last_change":"2023-11-08T16:30:30.744347+0000","last_active":"2023-11-08T16:30:30.751952+0000","last_peered":"2023-11-08T16:30:30.751952+0000","last_clean":"2023-11-08T16:30:30.751952+0000","last_became_active":"2023-11-08T16:30:30.743988+0000","last_became_peered":"2023-11-08T16:30:30.743988+0000","last_unstale":"2023-11-08T16:30:30.751952+0000","last_undegraded":"2023-11-08T16:30:30.751952+0000","last_fullsized":"2023-11-08T16:30:30.751952+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:41:10.086203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.734167+0000","last_change":"2023-11-08T16:30:30.728777+0000","last_active":"2023-11-08T16:30:30.734167+0000","last_peered":"2023-11-08T16:30:30.734167+0000","last_clean":"2023-11-08T16:30:30.734167+0000","last_became_active":"2023-11-08T16:30:30.728478+0000","last_became_peered":"2023-11-08T16:30:30.728478+0000","last_unstale":"2023-11-08T16:30:30.734167+0000","last_undegraded":"2023-11-08T16:30:30.734167+0000","last_fullsized":"2023-11-08T16:30:30.734167+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:43:05.965214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751008+0000","last_change":"2023-11-08T16:30:30.743548+0000","last_active":"2023-11-08T16:30:30.751008+0000","last_peered":"2023-11-08T16:30:30.751008+0000","last_clean":"2023-11-08T16:30:30.751008+0000","last_became_active":"2023-11-08T16:30:30.743257+0000","last_became_peered":"2023-11-08T16:30:30.743257+0000","last_unstale":"2023-11-08T16:30:30.751008+0000","last_undegraded":"2023-11-08T16:30:30.751008+0000","last_fullsized":"2023-11-08T16:30:30.751008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:49.297882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751594+0000","last_change":"2023-11-08T16:30:30.748616+0000","last_active":"2023-11-08T16:30:30.751594+0000","last_peered":"2023-11-08T16:30:30.751594+0000","last_clean":"2023-11-08T16:30:30.751594+0000","last_became_active":"2023-11-08T16:30:30.747263+0000","last_became_peered":"2023-11-08T16:30:30.747263+0000","last_unstale":"2023-11-08T16:30:30.751594+0000","last_undegraded":"2023-11-08T16:30:30.751594+0000","last_fullsized":"2023-11-08T16:30:30.751594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.503135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.753543+0000","last_change":"2023-11-08T16:30:30.744435+0000","last_active":"2023-11-08T16:30:30.753543+0000","last_peered":"2023-11-08T16:30:30.753543+0000","last_clean":"2023-11-08T16:30:30.753543+0000","last_became_active":"2023-11-08T16:30:30.743961+0000","last_became_peered":"2023-11-08T16:30:30.743961+0000","last_unstale":"2023-11-08T16:30:30.753543+0000","last_undegraded":"2023-11-08T16:30:30.753543+0000","last_fullsized":"2023-11-08T16:30:30.753543+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:36.347998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.736330+0000","last_change":"2023-11-08T16:30:30.732447+0000","last_active":"2023-11-08T16:30:30.736330+0000","last_peered":"2023-11-08T16:30:30.736330+0000","last_clean":"2023-11-08T16:30:30.736330+0000","last_became_active":"2023-11-08T16:30:30.732168+0000","last_became_peered":"2023-11-08T16:30:30.732168+0000","last_unstale":"2023-11-08T16:30:30.736330+0000","last_undegraded":"2023-11-08T16:30:30.736330+0000","last_fullsized":"2023-11-08T16:30:30.736330+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:17:55.087658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.733699+0000","last_change":"2023-11-08T16:30:30.728079+0000","last_active":"2023-11-08T16:30:30.733699+0000","last_peered":"2023-11-08T16:30:30.733699+0000","last_clean":"2023-11-08T16:30:30.733699+0000","last_became_active":"2023-11-08T16:30:30.727782+0000","last_became_peered":"2023-11-08T16:30:30.727782+0000","last_unstale":"2023-11-08T16:30:30.733699+0000","last_undegraded":"2023-11-08T16:30:30.733699+0000","last_fullsized":"2023-11-08T16:30:30.733699+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:50:36.916601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.757155+0000","last_change":"2023-11-08T16:30:30.751749+0000","last_active":"2023-11-08T16:30:30.757155+0000","last_peered":"2023-11-08T16:30:30.757155+0000","last_clean":"2023-11-08T16:30:30.757155+0000","last_became_active":"2023-11-08T16:30:30.751450+0000","last_became_peered":"2023-11-08T16:30:30.751450+0000","last_unstale":"2023-11-08T16:30:30.757155+0000","last_undegraded":"2023-11-08T16:30:30.757155+0000","last_fullsized":"2023-11-08T16:30:30.757155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:01:01.465958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.753833+0000","last_change":"2023-11-08T16:30:30.749015+0000","last_active":"2023-11-08T16:30:30.753833+0000","last_peered":"2023-11-08T16:30:30.753833+0000","last_clean":"2023-11-08T16:30:30.753833+0000","last_became_active":"2023-11-08T16:30:30.746444+0000","last_became_peered":"2023-11-08T16:30:30.746444+0000","last_unstale":"2023-11-08T16:30:30.753833+0000","last_undegraded":"2023-11-08T16:30:30.753833+0000","last_fullsized":"2023-11-08T16:30:30.753833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:29:58.478672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745398+0000","last_change":"2023-11-08T16:30:30.739840+0000","last_active":"2023-11-08T16:30:30.745398+0000","last_peered":"2023-11-08T16:30:30.745398+0000","last_clean":"2023-11-08T16:30:30.745398+0000","last_became_active":"2023-11-08T16:30:30.739556+0000","last_became_peered":"2023-11-08T16:30:30.739556+0000","last_unstale":"2023-11-08T16:30:30.745398+0000","last_undegraded":"2023-11-08T16:30:30.745398+0000","last_fullsized":"2023-11-08T16:30:30.745398+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:17.680411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.754292+0000","last_change":"2023-11-08T16:30:30.749585+0000","last_active":"2023-11-08T16:30:30.754292+0000","last_peered":"2023-11-08T16:30:30.754292+0000","last_clean":"2023-11-08T16:30:30.754292+0000","last_became_active":"2023-11-08T16:30:30.749041+0000","last_became_peered":"2023-11-08T16:30:30.749041+0000","last_unstale":"2023-11-08T16:30:30.754292+0000","last_undegraded":"2023-11-08T16:30:30.754292+0000","last_fullsized":"2023-11-08T16:30:30.754292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:17.680411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.731987+0000","last_change":"2023-11-08T16:30:30.728270+0000","last_active":"2023-11-08T16:30:30.731987+0000","last_peered":"2023-11-08T16:30:30.731987+0000","last_clean":"2023-11-08T16:30:30.731987+0000","last_became_active":"2023-11-08T16:30:30.728009+0000","last_became_peered":"2023-11-08T16:30:30.728009+0000","last_unstale":"2023-11-08T16:30:30.731987+0000","last_undegraded":"2023-11-08T16:30:30.731987+0000","last_fullsized":"2023-11-08T16:30:30.731987+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:36:25.618863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.748925+0000","last_change":"2023-11-08T16:30:30.735428+0000","last_active":"2023-11-08T16:30:30.748925+0000","last_peered":"2023-11-08T16:30:30.748925+0000","last_clean":"2023-11-08T16:30:30.748925+0000","last_became_active":"2023-11-08T16:30:30.735150+0000","last_became_peered":"2023-11-08T16:30:30.735150+0000","last_unstale":"2023-11-08T16:30:30.748925+0000","last_undegraded":"2023-11-08T16:30:30.748925+0000","last_fullsized":"2023-11-08T16:30:30.748925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:40:52.036835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.739584+0000","last_change":"2023-11-08T16:30:30.733711+0000","last_active":"2023-11-08T16:30:30.739584+0000","last_peered":"2023-11-08T16:30:30.739584+0000","last_clean":"2023-11-08T16:30:30.739584+0000","last_became_active":"2023-11-08T16:30:30.733177+0000","last_became_peered":"2023-11-08T16:30:30.733177+0000","last_unstale":"2023-11-08T16:30:30.739584+0000","last_undegraded":"2023-11-08T16:30:30.739584+0000","last_fullsized":"2023-11-08T16:30:30.739584+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:36.347998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.754117+0000","last_change":"2023-11-08T16:30:30.750658+0000","last_active":"2023-11-08T16:30:30.754117+0000","last_peered":"2023-11-08T16:30:30.754117+0000","last_clean":"2023-11-08T16:30:30.754117+0000","last_became_active":"2023-11-08T16:30:30.750447+0000","last_became_peered":"2023-11-08T16:30:30.750447+0000","last_unstale":"2023-11-08T16:30:30.754117+0000","last_undegraded":"2023-11-08T16:30:30.754117+0000","last_fullsized":"2023-11-08T16:30:30.754117+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:43:05.965214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.744573+0000","last_change":"2023-11-08T16:30:30.737545+0000","last_active":"2023-11-08T16:30:30.744573+0000","last_peered":"2023-11-08T16:30:30.744573+0000","last_clean":"2023-11-08T16:30:30.744573+0000","last_became_active":"2023-11-08T16:30:30.737221+0000","last_became_peered":"2023-11-08T16:30:30.737221+0000","last_unstale":"2023-11-08T16:30:30.744573+0000","last_undegraded":"2023-11-08T16:30:30.744573+0000","last_fullsized":"2023-11-08T16:30:30.744573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:30:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751940+0000","last_change":"2023-11-08T16:30:30.743535+0000","last_active":"2023-11-08T16:30:30.751940+0000","last_peered":"2023-11-08T16:30:30.751940+0000","last_clean":"2023-11-08T16:30:30.751940+0000","last_became_active":"2023-11-08T16:30:30.743284+0000","last_became_peered":"2023-11-08T16:30:30.743284+0000","last_unstale":"2023-11-08T16:30:30.751940+0000","last_undegraded":"2023-11-08T16:30:30.751940+0000","last_fullsized":"2023-11-08T16:30:30.751940+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:17:55.087658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.755716+0000","last_change":"2023-11-08T16:30:30.751153+0000","last_active":"2023-11-08T16:30:30.755716+0000","last_peered":"2023-11-08T16:30:30.755716+0000","last_clean":"2023-11-08T16:30:30.755716+0000","last_became_active":"2023-11-08T16:30:30.750642+0000","last_became_peered":"2023-11-08T16:30:30.750642+0000","last_unstale":"2023-11-08T16:30:30.755716+0000","last_undegraded":"2023-11-08T16:30:30.755716+0000","last_fullsized":"2023-11-08T16:30:30.755716+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:41.681026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.738171+0000","last_change":"2023-11-08T16:30:30.733893+0000","last_active":"2023-11-08T16:30:30.738171+0000","last_peered":"2023-11-08T16:30:30.738171+0000","last_clean":"2023-11-08T16:30:30.738171+0000","last_became_active":"2023-11-08T16:30:30.733611+0000","last_became_peered":"2023-11-08T16:30:30.733611+0000","last_unstale":"2023-11-08T16:30:30.738171+0000","last_undegraded":"2023-11-08T16:30:30.738171+0000","last_fullsized":"2023-11-08T16:30:30.738171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:26:23.385574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746714+0000","last_change":"2023-11-08T16:30:30.740830+0000","last_active":"2023-11-08T16:30:30.746714+0000","last_peered":"2023-11-08T16:30:30.746714+0000","last_clean":"2023-11-08T16:30:30.746714+0000","last_became_active":"2023-11-08T16:30:30.740242+0000","last_became_peered":"2023-11-08T16:30:30.740242+0000","last_unstale":"2023-11-08T16:30:30.746714+0000","last_undegraded":"2023-11-08T16:30:30.746714+0000","last_fullsized":"2023-11-08T16:30:30.746714+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:26:23.385574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.757202+0000","last_change":"2023-11-08T16:30:30.754302+0000","last_active":"2023-11-08T16:30:30.757202+0000","last_peered":"2023-11-08T16:30:30.757202+0000","last_clean":"2023-11-08T16:30:30.757202+0000","last_became_active":"2023-11-08T16:30:30.754014+0000","last_became_peered":"2023-11-08T16:30:30.754014+0000","last_unstale":"2023-11-08T16:30:30.757202+0000","last_undegraded":"2023-11-08T16:30:30.757202+0000","last_fullsized":"2023-11-08T16:30:30.757202+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:47.363218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.743081+0000","last_change":"2023-11-08T16:30:30.735655+0000","last_active":"2023-11-08T16:30:30.743081+0000","last_peered":"2023-11-08T16:30:30.743081+0000","last_clean":"2023-11-08T16:30:30.743081+0000","last_became_active":"2023-11-08T16:30:30.735401+0000","last_became_peered":"2023-11-08T16:30:30.735401+0000","last_unstale":"2023-11-08T16:30:30.743081+0000","last_undegraded":"2023-11-08T16:30:30.743081+0000","last_fullsized":"2023-11-08T16:30:30.743081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:29:58.478672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.730678+0000","last_change":"2023-11-08T16:30:30.722152+0000","last_active":"2023-11-08T16:30:30.730678+0000","last_peered":"2023-11-08T16:30:30.730678+0000","last_clean":"2023-11-08T16:30:30.730678+0000","last_became_active":"2023-11-08T16:30:30.721663+0000","last_became_peered":"2023-11-08T16:30:30.721663+0000","last_unstale":"2023-11-08T16:30:30.730678+0000","last_undegraded":"2023-11-08T16:30:30.730678+0000","last_fullsized":"2023-11-08T16:30:30.730678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:16.017807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750291+0000","last_change":"2023-11-08T16:30:30.736433+0000","last_active":"2023-11-08T16:30:30.750291+0000","last_peered":"2023-11-08T16:30:30.750291+0000","last_clean":"2023-11-08T16:30:30.750291+0000","last_became_active":"2023-11-08T16:30:30.734973+0000","last_became_peered":"2023-11-08T16:30:30.734973+0000","last_unstale":"2023-11-08T16:30:30.750291+0000","last_undegraded":"2023-11-08T16:30:30.750291+0000","last_fullsized":"2023-11-08T16:30:30.750291+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:43.258248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.754116+0000","last_change":"2023-11-08T16:30:30.748464+0000","last_active":"2023-11-08T16:30:30.754116+0000","last_peered":"2023-11-08T16:30:30.754116+0000","last_clean":"2023-11-08T16:30:30.754116+0000","last_became_active":"2023-11-08T16:30:30.748172+0000","last_became_peered":"2023-11-08T16:30:30.748172+0000","last_unstale":"2023-11-08T16:30:30.754116+0000","last_undegraded":"2023-11-08T16:30:30.754116+0000","last_fullsized":"2023-11-08T16:30:30.754116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:29.269838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750542+0000","last_change":"2023-11-08T16:30:30.741546+0000","last_active":"2023-11-08T16:30:30.750542+0000","last_peered":"2023-11-08T16:30:30.750542+0000","last_clean":"2023-11-08T16:30:30.750542+0000","last_became_active":"2023-11-08T16:30:30.740966+0000","last_became_peered":"2023-11-08T16:30:30.740966+0000","last_unstale":"2023-11-08T16:30:30.750542+0000","last_undegraded":"2023-11-08T16:30:30.750542+0000","last_fullsized":"2023-11-08T16:30:30.750542+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:41.681026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.737820+0000","last_change":"2023-11-08T16:30:30.730928+0000","last_active":"2023-11-08T16:30:30.737820+0000","last_peered":"2023-11-08T16:30:30.737820+0000","last_clean":"2023-11-08T16:30:30.737820+0000","last_became_active":"2023-11-08T16:30:30.730624+0000","last_became_peered":"2023-11-08T16:30:30.730624+0000","last_unstale":"2023-11-08T16:30:30.737820+0000","last_undegraded":"2023-11-08T16:30:30.737820+0000","last_fullsized":"2023-11-08T16:30:30.737820+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:17:55.087658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.756704+0000","last_change":"2023-11-08T16:30:30.751206+0000","last_active":"2023-11-08T16:30:30.756704+0000","last_peered":"2023-11-08T16:30:30.756704+0000","last_clean":"2023-11-08T16:30:30.756704+0000","last_became_active":"2023-11-08T16:30:30.750976+0000","last_became_peered":"2023-11-08T16:30:30.750976+0000","last_unstale":"2023-11-08T16:30:30.756704+0000","last_undegraded":"2023-11-08T16:30:30.756704+0000","last_fullsized":"2023-11-08T16:30:30.756704+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:22:53.461699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.743725+0000","last_change":"2023-11-08T16:30:30.735107+0000","last_active":"2023-11-08T16:30:30.743725+0000","last_peered":"2023-11-08T16:30:30.743725+0000","last_clean":"2023-11-08T16:30:30.743725+0000","last_became_active":"2023-11-08T16:30:30.734807+0000","last_became_peered":"2023-11-08T16:30:30.734807+0000","last_unstale":"2023-11-08T16:30:30.743725+0000","last_undegraded":"2023-11-08T16:30:30.743725+0000","last_fullsized":"2023-11-08T16:30:30.743725+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:29.269838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.730583+0000","last_change":"2023-11-08T16:30:30.719326+0000","last_active":"2023-11-08T16:30:30.730583+0000","last_peered":"2023-11-08T16:30:30.730583+0000","last_clean":"2023-11-08T16:30:30.730583+0000","last_became_active":"2023-11-08T16:30:30.718978+0000","last_became_peered":"2023-11-08T16:30:30.718978+0000","last_unstale":"2023-11-08T16:30:30.730583+0000","last_undegraded":"2023-11-08T16:30:30.730583+0000","last_fullsized":"2023-11-08T16:30:30.730583+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:49.297882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.743772+0000","last_change":"2023-11-08T16:30:30.735879+0000","last_active":"2023-11-08T16:30:30.743772+0000","last_peered":"2023-11-08T16:30:30.743772+0000","last_clean":"2023-11-08T16:30:30.743772+0000","last_became_active":"2023-11-08T16:30:30.734679+0000","last_became_peered":"2023-11-08T16:30:30.734679+0000","last_unstale":"2023-11-08T16:30:30.743772+0000","last_undegraded":"2023-11-08T16:30:30.743772+0000","last_fullsized":"2023-11-08T16:30:30.743772+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:16.017807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.743977+0000","last_change":"2023-11-08T16:30:30.736452+0000","last_active":"2023-11-08T16:30:30.743977+0000","last_peered":"2023-11-08T16:30:30.743977+0000","last_clean":"2023-11-08T16:30:30.743977+0000","last_became_active":"2023-11-08T16:30:30.736021+0000","last_became_peered":"2023-11-08T16:30:30.736021+0000","last_unstale":"2023-11-08T16:30:30.743977+0000","last_undegraded":"2023-11-08T16:30:30.743977+0000","last_fullsized":"2023-11-08T16:30:30.743977+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:16.017807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.740003+0000","last_change":"2023-11-08T16:30:29.713222+0000","last_active":"2023-11-08T16:30:30.740003+0000","last_peered":"2023-11-08T16:30:30.740003+0000","last_clean":"2023-11-08T16:30:30.740003+0000","last_became_active":"2023-11-08T16:30:29.712473+0000","last_became_peered":"2023-11-08T16:30:29.712473+0000","last_unstale":"2023-11-08T16:30:30.740003+0000","last_undegraded":"2023-11-08T16:30:30.740003+0000","last_fullsized":"2023-11-08T16:30:30.740003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:42.245668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746520+0000","last_change":"2023-11-08T16:30:30.744333+0000","last_active":"2023-11-08T16:30:30.746520+0000","last_peered":"2023-11-08T16:30:30.746520+0000","last_clean":"2023-11-08T16:30:30.746520+0000","last_became_active":"2023-11-08T16:30:30.744066+0000","last_became_peered":"2023-11-08T16:30:30.744066+0000","last_unstale":"2023-11-08T16:30:30.746520+0000","last_undegraded":"2023-11-08T16:30:30.746520+0000","last_fullsized":"2023-11-08T16:30:30.746520+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:29:58.478672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.736278+0000","last_change":"2023-11-08T16:30:29.714369+0000","last_active":"2023-11-08T16:30:30.736278+0000","last_peered":"2023-11-08T16:30:30.736278+0000","last_clean":"2023-11-08T16:30:30.736278+0000","last_became_active":"2023-11-08T16:30:29.714093+0000","last_became_peered":"2023-11-08T16:30:29.714093+0000","last_unstale":"2023-11-08T16:30:30.736278+0000","last_undegraded":"2023-11-08T16:30:30.736278+0000","last_fullsized":"2023-11-08T16:30:30.736278+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:40:51.024256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.757207+0000","last_change":"2023-11-08T16:30:30.754902+0000","last_active":"2023-11-08T16:30:30.757207+0000","last_peered":"2023-11-08T16:30:30.757207+0000","last_clean":"2023-11-08T16:30:30.757207+0000","last_became_active":"2023-11-08T16:30:30.754671+0000","last_became_peered":"2023-11-08T16:30:30.754671+0000","last_unstale":"2023-11-08T16:30:30.757207+0000","last_undegraded":"2023-11-08T16:30:30.757207+0000","last_fullsized":"2023-11-08T16:30:30.757207+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:43.673461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751655+0000","last_change":"2023-11-08T16:30:29.714179+0000","last_active":"2023-11-08T16:30:30.751655+0000","last_peered":"2023-11-08T16:30:30.751655+0000","last_clean":"2023-11-08T16:30:30.751655+0000","last_became_active":"2023-11-08T16:30:29.713139+0000","last_became_peered":"2023-11-08T16:30:29.713139+0000","last_unstale":"2023-11-08T16:30:30.751655+0000","last_undegraded":"2023-11-08T16:30:30.751655+0000","last_fullsized":"2023-11-08T16:30:30.751655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:36:40.585696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750524+0000","last_change":"2023-11-08T16:30:30.741379+0000","last_active":"2023-11-08T16:30:30.750524+0000","last_peered":"2023-11-08T16:30:30.750524+0000","last_clean":"2023-11-08T16:30:30.750524+0000","last_became_active":"2023-11-08T16:30:30.740659+0000","last_became_peered":"2023-11-08T16:30:30.740659+0000","last_unstale":"2023-11-08T16:30:30.750524+0000","last_undegraded":"2023-11-08T16:30:30.750524+0000","last_fullsized":"2023-11-08T16:30:30.750524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:09:34.574098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.749677+0000","last_change":"2023-11-08T16:30:29.714540+0000","last_active":"2023-11-08T16:30:30.749677+0000","last_peered":"2023-11-08T16:30:30.749677+0000","last_clean":"2023-11-08T16:30:30.749677+0000","last_became_active":"2023-11-08T16:30:29.714162+0000","last_became_peered":"2023-11-08T16:30:29.714162+0000","last_unstale":"2023-11-08T16:30:30.749677+0000","last_undegraded":"2023-11-08T16:30:30.749677+0000","last_fullsized":"2023-11-08T16:30:30.749677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:42.660881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745451+0000","last_change":"2023-11-08T16:30:30.741940+0000","last_active":"2023-11-08T16:30:30.745451+0000","last_peered":"2023-11-08T16:30:30.745451+0000","last_clean":"2023-11-08T16:30:30.745451+0000","last_became_active":"2023-11-08T16:30:30.741527+0000","last_became_peered":"2023-11-08T16:30:30.741527+0000","last_unstale":"2023-11-08T16:30:30.745451+0000","last_undegraded":"2023-11-08T16:30:30.745451+0000","last_fullsized":"2023-11-08T16:30:30.745451+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:41:10.086203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.752039+0000","last_change":"2023-11-08T16:30:29.719718+0000","last_active":"2023-11-08T16:30:30.752039+0000","last_peered":"2023-11-08T16:30:30.752039+0000","last_clean":"2023-11-08T16:30:30.752039+0000","last_became_active":"2023-11-08T16:30:29.719420+0000","last_became_peered":"2023-11-08T16:30:29.719420+0000","last_unstale":"2023-11-08T16:30:30.752039+0000","last_undegraded":"2023-11-08T16:30:30.752039+0000","last_fullsized":"2023-11-08T16:30:30.752039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:41.770901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.733806+0000","last_change":"2023-11-08T16:30:30.725830+0000","last_active":"2023-11-08T16:30:30.733806+0000","last_peered":"2023-11-08T16:30:30.733806+0000","last_clean":"2023-11-08T16:30:30.733806+0000","last_became_active":"2023-11-08T16:30:30.725538+0000","last_became_peered":"2023-11-08T16:30:30.725538+0000","last_unstale":"2023-11-08T16:30:30.733806+0000","last_undegraded":"2023-11-08T16:30:30.733806+0000","last_fullsized":"2023-11-08T16:30:30.733806+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:40:52.036835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.753845+0000","last_change":"2023-11-08T16:30:29.719564+0000","last_active":"2023-11-08T16:30:30.753845+0000","last_peered":"2023-11-08T16:30:30.753845+0000","last_clean":"2023-11-08T16:30:30.753845+0000","last_became_active":"2023-11-08T16:30:29.719271+0000","last_became_peered":"2023-11-08T16:30:29.719271+0000","last_unstale":"2023-11-08T16:30:30.753845+0000","last_undegraded":"2023-11-08T16:30:30.753845+0000","last_fullsized":"2023-11-08T16:30:30.753845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:08.194612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746774+0000","last_change":"2023-11-08T16:30:30.741069+0000","last_active":"2023-11-08T16:30:30.746774+0000","last_peered":"2023-11-08T16:30:30.746774+0000","last_clean":"2023-11-08T16:30:30.746774+0000","last_became_active":"2023-11-08T16:30:30.740761+0000","last_became_peered":"2023-11-08T16:30:30.740761+0000","last_unstale":"2023-11-08T16:30:30.746774+0000","last_undegraded":"2023-11-08T16:30:30.746774+0000","last_fullsized":"2023-11-08T16:30:30.746774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:52.347858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.727682+0000","last_change":"2023-11-08T16:30:29.707635+0000","last_active":"2023-11-08T16:30:30.727682+0000","last_peered":"2023-11-08T16:30:30.727682+0000","last_clean":"2023-11-08T16:30:30.727682+0000","last_became_active":"2023-11-08T16:30:29.707266+0000","last_became_peered":"2023-11-08T16:30:29.707266+0000","last_unstale":"2023-11-08T16:30:30.727682+0000","last_undegraded":"2023-11-08T16:30:30.727682+0000","last_fullsized":"2023-11-08T16:30:30.727682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:30:44.639668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.747823+0000","last_change":"2023-11-08T16:30:30.732414+0000","last_active":"2023-11-08T16:30:30.747823+0000","last_peered":"2023-11-08T16:30:30.747823+0000","last_clean":"2023-11-08T16:30:30.747823+0000","last_became_active":"2023-11-08T16:30:30.731922+0000","last_became_peered":"2023-11-08T16:30:30.731922+0000","last_unstale":"2023-11-08T16:30:30.747823+0000","last_undegraded":"2023-11-08T16:30:30.747823+0000","last_fullsized":"2023-11-08T16:30:30.747823+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:09.207192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.736172+0000","last_change":"2023-11-08T16:30:29.719181+0000","last_active":"2023-11-08T16:30:30.736172+0000","last_peered":"2023-11-08T16:30:30.736172+0000","last_clean":"2023-11-08T16:30:30.736172+0000","last_became_active":"2023-11-08T16:30:29.718815+0000","last_became_peered":"2023-11-08T16:30:29.718815+0000","last_unstale":"2023-11-08T16:30:30.736172+0000","last_undegraded":"2023-11-08T16:30:30.736172+0000","last_fullsized":"2023-11-08T16:30:30.736172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:29:57.466092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746899+0000","last_change":"2023-11-08T16:30:30.745451+0000","last_active":"2023-11-08T16:30:30.746899+0000","last_peered":"2023-11-08T16:30:30.746899+0000","last_clean":"2023-11-08T16:30:30.746899+0000","last_became_active":"2023-11-08T16:30:30.745038+0000","last_became_peered":"2023-11-08T16:30:30.745038+0000","last_unstale":"2023-11-08T16:30:30.746899+0000","last_undegraded":"2023-11-08T16:30:30.746899+0000","last_fullsized":"2023-11-08T16:30:30.746899+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:41.681026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.740306+0000","last_change":"2023-11-08T16:30:29.713680+0000","last_active":"2023-11-08T16:30:30.740306+0000","last_peered":"2023-11-08T16:30:30.740306+0000","last_clean":"2023-11-08T16:30:30.740306+0000","last_became_active":"2023-11-08T16:30:29.713405+0000","last_became_peered":"2023-11-08T16:30:29.713405+0000","last_unstale":"2023-11-08T16:30:30.740306+0000","last_undegraded":"2023-11-08T16:30:30.740306+0000","last_fullsized":"2023-11-08T16:30:30.740306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:24:13.302756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.749125+0000","last_change":"2023-11-08T16:30:30.735536+0000","last_active":"2023-11-08T16:30:30.749125+0000","last_peered":"2023-11-08T16:30:30.749125+0000","last_clean":"2023-11-08T16:30:30.749125+0000","last_became_active":"2023-11-08T16:30:30.735222+0000","last_became_peered":"2023-11-08T16:30:30.735222+0000","last_unstale":"2023-11-08T16:30:30.749125+0000","last_undegraded":"2023-11-08T16:30:30.749125+0000","last_fullsized":"2023-11-08T16:30:30.749125+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:16.017807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.722358+0000","last_change":"2023-11-08T16:30:29.717064+0000","last_active":"2023-11-08T16:30:30.722358+0000","last_peered":"2023-11-08T16:30:30.722358+0000","last_clean":"2023-11-08T16:30:30.722358+0000","last_became_active":"2023-11-08T16:30:29.716703+0000","last_became_peered":"2023-11-08T16:30:29.716703+0000","last_unstale":"2023-11-08T16:30:30.722358+0000","last_undegraded":"2023-11-08T16:30:30.722358+0000","last_fullsized":"2023-11-08T16:30:30.722358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:35.335418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750362+0000","last_change":"2023-11-08T16:30:30.739848+0000","last_active":"2023-11-08T16:30:30.750362+0000","last_peered":"2023-11-08T16:30:30.750362+0000","last_clean":"2023-11-08T16:30:30.750362+0000","last_became_active":"2023-11-08T16:30:30.737365+0000","last_became_peered":"2023-11-08T16:30:30.737365+0000","last_unstale":"2023-11-08T16:30:30.750362+0000","last_undegraded":"2023-11-08T16:30:30.750362+0000","last_fullsized":"2023-11-08T16:30:30.750362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:20:58.235245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.739869+0000","last_change":"2023-11-08T16:30:29.712482+0000","last_active":"2023-11-08T16:30:30.739869+0000","last_peered":"2023-11-08T16:30:30.739869+0000","last_clean":"2023-11-08T16:30:30.739869+0000","last_became_active":"2023-11-08T16:30:29.712185+0000","last_became_peered":"2023-11-08T16:30:29.712185+0000","last_unstale":"2023-11-08T16:30:30.739869+0000","last_undegraded":"2023-11-08T16:30:30.739869+0000","last_fullsized":"2023-11-08T16:30:30.739869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:26:35.052911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.739753+0000","last_change":"2023-11-08T16:30:30.732888+0000","last_active":"2023-11-08T16:30:30.739753+0000","last_peered":"2023-11-08T16:30:30.739753+0000","last_clean":"2023-11-08T16:30:30.739753+0000","last_became_active":"2023-11-08T16:30:30.732430+0000","last_became_peered":"2023-11-08T16:30:30.732430+0000","last_unstale":"2023-11-08T16:30:30.739753+0000","last_undegraded":"2023-11-08T16:30:30.739753+0000","last_fullsized":"2023-11-08T16:30:30.739753+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:24:51.640333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.761484+0000","last_change":"2023-11-08T16:30:29.712622+0000","last_active":"2023-11-08T16:30:30.761484+0000","last_peered":"2023-11-08T16:30:30.761484+0000","last_clean":"2023-11-08T16:30:30.761484+0000","last_became_active":"2023-11-08T16:30:29.712332+0000","last_became_peered":"2023-11-08T16:30:29.712332+0000","last_unstale":"2023-11-08T16:30:30.761484+0000","last_undegraded":"2023-11-08T16:30:30.761484+0000","last_fullsized":"2023-11-08T16:30:30.761484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:26:35.052911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.749130+0000","last_change":"2023-11-08T16:30:30.732641+0000","last_active":"2023-11-08T16:30:30.749130+0000","last_peered":"2023-11-08T16:30:30.749130+0000","last_clean":"2023-11-08T16:30:30.749130+0000","last_became_active":"2023-11-08T16:30:30.732349+0000","last_became_peered":"2023-11-08T16:30:30.732349+0000","last_unstale":"2023-11-08T16:30:30.749130+0000","last_undegraded":"2023-11-08T16:30:30.749130+0000","last_fullsized":"2023-11-08T16:30:30.749130+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:46.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745683+0000","last_change":"2023-11-08T16:30:29.717572+0000","last_active":"2023-11-08T16:30:30.745683+0000","last_peered":"2023-11-08T16:30:30.745683+0000","last_clean":"2023-11-08T16:30:30.745683+0000","last_became_active":"2023-11-08T16:30:29.716652+0000","last_became_peered":"2023-11-08T16:30:29.716652+0000","last_unstale":"2023-11-08T16:30:30.745683+0000","last_undegraded":"2023-11-08T16:30:30.745683+0000","last_fullsized":"2023-11-08T16:30:30.745683+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:48.285303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745848+0000","last_change":"2023-11-08T16:30:30.726248+0000","last_active":"2023-11-08T16:30:30.745848+0000","last_peered":"2023-11-08T16:30:30.745848+0000","last_clean":"2023-11-08T16:30:30.745848+0000","last_became_active":"2023-11-08T16:30:30.725970+0000","last_became_peered":"2023-11-08T16:30:30.725970+0000","last_unstale":"2023-11-08T16:30:30.745848+0000","last_undegraded":"2023-11-08T16:30:30.745848+0000","last_fullsized":"2023-11-08T16:30:30.745848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:46.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.713057+0000","last_change":"2023-11-08T16:30:29.709073+0000","last_active":"2023-11-08T16:30:30.713057+0000","last_peered":"2023-11-08T16:30:30.713057+0000","last_clean":"2023-11-08T16:30:30.713057+0000","last_became_active":"2023-11-08T16:30:29.708592+0000","last_became_peered":"2023-11-08T16:30:29.708592+0000","last_unstale":"2023-11-08T16:30:30.713057+0000","last_undegraded":"2023-11-08T16:30:30.713057+0000","last_fullsized":"2023-11-08T16:30:30.713057+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:20:57.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745336+0000","last_change":"2023-11-08T16:30:30.740631+0000","last_active":"2023-11-08T16:30:30.745336+0000","last_peered":"2023-11-08T16:30:30.745336+0000","last_clean":"2023-11-08T16:30:30.745336+0000","last_became_active":"2023-11-08T16:30:30.740342+0000","last_became_peered":"2023-11-08T16:30:30.740342+0000","last_unstale":"2023-11-08T16:30:30.745336+0000","last_undegraded":"2023-11-08T16:30:30.745336+0000","last_fullsized":"2023-11-08T16:30:30.745336+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:36.347998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.727866+0000","last_change":"2023-11-08T16:30:29.713952+0000","last_active":"2023-11-08T16:30:30.727866+0000","last_peered":"2023-11-08T16:30:30.727866+0000","last_clean":"2023-11-08T16:30:30.727866+0000","last_became_active":"2023-11-08T16:30:29.713684+0000","last_became_peered":"2023-11-08T16:30:29.713684+0000","last_unstale":"2023-11-08T16:30:30.727866+0000","last_undegraded":"2023-11-08T16:30:30.727866+0000","last_fullsized":"2023-11-08T16:30:30.727866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:24:50.627753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745002+0000","last_change":"2023-11-08T16:30:30.738280+0000","last_active":"2023-11-08T16:30:30.745002+0000","last_peered":"2023-11-08T16:30:30.745002+0000","last_clean":"2023-11-08T16:30:30.745002+0000","last_became_active":"2023-11-08T16:30:30.737897+0000","last_became_peered":"2023-11-08T16:30:30.737897+0000","last_unstale":"2023-11-08T16:30:30.745002+0000","last_undegraded":"2023-11-08T16:30:30.745002+0000","last_fullsized":"2023-11-08T16:30:30.745002+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:46.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.752276+0000","last_change":"2023-11-08T16:30:29.708852+0000","last_active":"2023-11-08T16:30:30.752276+0000","last_peered":"2023-11-08T16:30:30.752276+0000","last_clean":"2023-11-08T16:30:30.752276+0000","last_became_active":"2023-11-08T16:30:29.708573+0000","last_became_peered":"2023-11-08T16:30:29.708573+0000","last_unstale":"2023-11-08T16:30:30.752276+0000","last_undegraded":"2023-11-08T16:30:30.752276+0000","last_fullsized":"2023-11-08T16:30:30.752276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:16.667831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746932+0000","last_change":"2023-11-08T16:30:30.742885+0000","last_active":"2023-11-08T16:30:30.746932+0000","last_peered":"2023-11-08T16:30:30.746932+0000","last_clean":"2023-11-08T16:30:30.746932+0000","last_became_active":"2023-11-08T16:30:30.742017+0000","last_became_peered":"2023-11-08T16:30:30.742017+0000","last_unstale":"2023-11-08T16:30:30.746932+0000","last_undegraded":"2023-11-08T16:30:30.746932+0000","last_fullsized":"2023-11-08T16:30:30.746932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:40:43.098228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.752264+0000","last_change":"2023-11-08T16:30:29.709425+0000","last_active":"2023-11-08T16:30:30.752264+0000","last_peered":"2023-11-08T16:30:30.752264+0000","last_clean":"2023-11-08T16:30:30.752264+0000","last_became_active":"2023-11-08T16:30:29.709148+0000","last_became_peered":"2023-11-08T16:30:29.709148+0000","last_unstale":"2023-11-08T16:30:30.752264+0000","last_undegraded":"2023-11-08T16:30:30.752264+0000","last_fullsized":"2023-11-08T16:30:30.752264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:40:51.024256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746175+0000","last_change":"2023-11-08T16:30:30.744346+0000","last_active":"2023-11-08T16:30:30.746175+0000","last_peered":"2023-11-08T16:30:30.746175+0000","last_clean":"2023-11-08T16:30:30.746175+0000","last_became_active":"2023-11-08T16:30:30.744025+0000","last_became_peered":"2023-11-08T16:30:30.744025+0000","last_unstale":"2023-11-08T16:30:30.746175+0000","last_undegraded":"2023-11-08T16:30:30.746175+0000","last_fullsized":"2023-11-08T16:30:30.746175+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:30:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746878+0000","last_change":"2023-11-08T16:30:29.719132+0000","last_active":"2023-11-08T16:30:30.746878+0000","last_peered":"2023-11-08T16:30:30.746878+0000","last_clean":"2023-11-08T16:30:30.746878+0000","last_became_active":"2023-11-08T16:30:29.718811+0000","last_became_peered":"2023-11-08T16:30:29.718811+0000","last_unstale":"2023-11-08T16:30:30.746878+0000","last_undegraded":"2023-11-08T16:30:30.746878+0000","last_fullsized":"2023-11-08T16:30:30.746878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:42.245668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750568+0000","last_change":"2023-11-08T16:30:30.741287+0000","last_active":"2023-11-08T16:30:30.750568+0000","last_peered":"2023-11-08T16:30:30.750568+0000","last_clean":"2023-11-08T16:30:30.750568+0000","last_became_active":"2023-11-08T16:30:30.740650+0000","last_became_peered":"2023-11-08T16:30:30.740650+0000","last_unstale":"2023-11-08T16:30:30.750568+0000","last_undegraded":"2023-11-08T16:30:30.750568+0000","last_fullsized":"2023-11-08T16:30:30.750568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:42.783481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.749112+0000","last_change":"2023-11-08T16:30:29.718853+0000","last_active":"2023-11-08T16:30:30.749112+0000","last_peered":"2023-11-08T16:30:30.749112+0000","last_clean":"2023-11-08T16:30:30.749112+0000","last_became_active":"2023-11-08T16:30:29.718494+0000","last_became_peered":"2023-11-08T16:30:29.718494+0000","last_unstale":"2023-11-08T16:30:30.749112+0000","last_undegraded":"2023-11-08T16:30:30.749112+0000","last_fullsized":"2023-11-08T16:30:30.749112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:45.269178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.734316+0000","last_change":"2023-11-08T16:30:30.732075+0000","last_active":"2023-11-08T16:30:30.734316+0000","last_peered":"2023-11-08T16:30:30.734316+0000","last_clean":"2023-11-08T16:30:30.734316+0000","last_became_active":"2023-11-08T16:30:30.731663+0000","last_became_peered":"2023-11-08T16:30:30.731663+0000","last_unstale":"2023-11-08T16:30:30.734316+0000","last_undegraded":"2023-11-08T16:30:30.734316+0000","last_fullsized":"2023-11-08T16:30:30.734316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:22:53.461699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.739676+0000","last_change":"2023-11-08T16:30:29.719508+0000","last_active":"2023-11-08T16:30:30.739676+0000","last_peered":"2023-11-08T16:30:30.739676+0000","last_clean":"2023-11-08T16:30:30.739676+0000","last_became_active":"2023-11-08T16:30:29.719157+0000","last_became_peered":"2023-11-08T16:30:29.719157+0000","last_unstale":"2023-11-08T16:30:30.739676+0000","last_undegraded":"2023-11-08T16:30:30.739676+0000","last_fullsized":"2023-11-08T16:30:30.739676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:20:57.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.748422+0000","last_change":"2023-11-08T16:30:30.741880+0000","last_active":"2023-11-08T16:30:30.748422+0000","last_peered":"2023-11-08T16:30:30.748422+0000","last_clean":"2023-11-08T16:30:30.748422+0000","last_became_active":"2023-11-08T16:30:30.741610+0000","last_became_peered":"2023-11-08T16:30:30.741610+0000","last_unstale":"2023-11-08T16:30:30.748422+0000","last_undegraded":"2023-11-08T16:30:30.748422+0000","last_fullsized":"2023-11-08T16:30:30.748422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:20:05.238900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.738974+0000","last_change":"2023-11-08T16:30:29.713858+0000","last_active":"2023-11-08T16:30:30.738974+0000","last_peered":"2023-11-08T16:30:30.738974+0000","last_clean":"2023-11-08T16:30:30.738974+0000","last_became_active":"2023-11-08T16:30:29.713552+0000","last_became_peered":"2023-11-08T16:30:29.713552+0000","last_unstale":"2023-11-08T16:30:30.738974+0000","last_undegraded":"2023-11-08T16:30:30.738974+0000","last_fullsized":"2023-11-08T16:30:30.738974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:20:57.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.733329+0000","last_change":"2023-11-08T16:30:30.729904+0000","last_active":"2023-11-08T16:30:30.733329+0000","last_peered":"2023-11-08T16:30:30.733329+0000","last_clean":"2023-11-08T16:30:30.733329+0000","last_became_active":"2023-11-08T16:30:30.729628+0000","last_became_peered":"2023-11-08T16:30:30.729628+0000","last_unstale":"2023-11-08T16:30:30.733329+0000","last_undegraded":"2023-11-08T16:30:30.733329+0000","last_fullsized":"2023-11-08T16:30:30.733329+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:13:59.773111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.709658+0000","last_change":"2023-11-08T16:30:29.718001+0000","last_active":"2023-11-08T16:30:30.709658+0000","last_peered":"2023-11-08T16:30:30.709658+0000","last_clean":"2023-11-08T16:30:30.709658+0000","last_became_active":"2023-11-08T16:30:29.717708+0000","last_became_peered":"2023-11-08T16:30:29.717708+0000","last_unstale":"2023-11-08T16:30:30.709658+0000","last_undegraded":"2023-11-08T16:30:30.709658+0000","last_fullsized":"2023-11-08T16:30:30.709658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:20:57.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.728035+0000","last_change":"2023-11-08T16:30:30.726011+0000","last_active":"2023-11-08T16:30:30.728035+0000","last_peered":"2023-11-08T16:30:30.728035+0000","last_clean":"2023-11-08T16:30:30.728035+0000","last_became_active":"2023-11-08T16:30:30.724356+0000","last_became_peered":"2023-11-08T16:30:30.724356+0000","last_unstale":"2023-11-08T16:30:30.728035+0000","last_undegraded":"2023-11-08T16:30:30.728035+0000","last_fullsized":"2023-11-08T16:30:30.728035+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:12:03.632756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.735745+0000","last_change":"2023-11-08T16:30:29.717115+0000","last_active":"2023-11-08T16:30:30.735745+0000","last_peered":"2023-11-08T16:30:30.735745+0000","last_clean":"2023-11-08T16:30:30.735745+0000","last_became_active":"2023-11-08T16:30:29.716797+0000","last_became_peered":"2023-11-08T16:30:29.716797+0000","last_unstale":"2023-11-08T16:30:30.735745+0000","last_undegraded":"2023-11-08T16:30:30.735745+0000","last_fullsized":"2023-11-08T16:30:30.735745+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:09:33.561519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.737651+0000","last_change":"2023-11-08T16:30:30.734244+0000","last_active":"2023-11-08T16:30:30.737651+0000","last_peered":"2023-11-08T16:30:30.737651+0000","last_clean":"2023-11-08T16:30:30.737651+0000","last_became_active":"2023-11-08T16:30:30.733917+0000","last_became_peered":"2023-11-08T16:30:30.733917+0000","last_unstale":"2023-11-08T16:30:30.737651+0000","last_undegraded":"2023-11-08T16:30:30.737651+0000","last_fullsized":"2023-11-08T16:30:30.737651+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:41.681026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.712351+0000","last_change":"2023-11-08T16:30:29.712429+0000","last_active":"2023-11-08T16:30:30.712351+0000","last_peered":"2023-11-08T16:30:30.712351+0000","last_clean":"2023-11-08T16:30:30.712351+0000","last_became_active":"2023-11-08T16:30:29.712021+0000","last_became_peered":"2023-11-08T16:30:29.712021+0000","last_unstale":"2023-11-08T16:30:30.712351+0000","last_undegraded":"2023-11-08T16:30:30.712351+0000","last_fullsized":"2023-11-08T16:30:30.712351+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:28.257258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.734864+0000","last_change":"2023-11-08T16:30:30.733372+0000","last_active":"2023-11-08T16:30:30.734864+0000","last_peered":"2023-11-08T16:30:30.734864+0000","last_clean":"2023-11-08T16:30:30.734864+0000","last_became_active":"2023-11-08T16:30:30.733085+0000","last_became_peered":"2023-11-08T16:30:30.733085+0000","last_unstale":"2023-11-08T16:30:30.734864+0000","last_undegraded":"2023-11-08T16:30:30.734864+0000","last_fullsized":"2023-11-08T16:30:30.734864+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:52.347858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.735699+0000","last_change":"2023-11-08T16:30:29.705156+0000","last_active":"2023-11-08T16:30:30.735699+0000","last_peered":"2023-11-08T16:30:30.735699+0000","last_clean":"2023-11-08T16:30:30.735699+0000","last_became_active":"2023-11-08T16:30:29.704878+0000","last_became_peered":"2023-11-08T16:30:29.704878+0000","last_unstale":"2023-11-08T16:30:30.735699+0000","last_undegraded":"2023-11-08T16:30:30.735699+0000","last_fullsized":"2023-11-08T16:30:30.735699+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:35.335418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.739511+0000","last_change":"2023-11-08T16:30:30.732855+0000","last_active":"2023-11-08T16:30:30.739511+0000","last_peered":"2023-11-08T16:30:30.739511+0000","last_clean":"2023-11-08T16:30:30.739511+0000","last_became_active":"2023-11-08T16:30:30.732495+0000","last_became_peered":"2023-11-08T16:30:30.732495+0000","last_unstale":"2023-11-08T16:30:30.739511+0000","last_undegraded":"2023-11-08T16:30:30.739511+0000","last_fullsized":"2023-11-08T16:30:30.739511+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:41:10.086203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.722462+0000","last_change":"2023-11-08T16:30:29.714099+0000","last_active":"2023-11-08T16:30:30.722462+0000","last_peered":"2023-11-08T16:30:30.722462+0000","last_clean":"2023-11-08T16:30:30.722462+0000","last_became_active":"2023-11-08T16:30:29.713778+0000","last_became_peered":"2023-11-08T16:30:29.713778+0000","last_unstale":"2023-11-08T16:30:30.722462+0000","last_undegraded":"2023-11-08T16:30:30.722462+0000","last_fullsized":"2023-11-08T16:30:30.722462+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:40.668446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.743830+0000","last_change":"2023-11-08T16:30:30.735989+0000","last_active":"2023-11-08T16:30:30.743830+0000","last_peered":"2023-11-08T16:30:30.743830+0000","last_clean":"2023-11-08T16:30:30.743830+0000","last_became_active":"2023-11-08T16:30:30.735690+0000","last_became_peered":"2023-11-08T16:30:30.735690+0000","last_unstale":"2023-11-08T16:30:30.743830+0000","last_undegraded":"2023-11-08T16:30:30.743830+0000","last_fullsized":"2023-11-08T16:30:30.743830+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:46.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.713127+0000","last_change":"2023-11-08T16:30:29.706362+0000","last_active":"2023-11-08T16:30:30.713127+0000","last_peered":"2023-11-08T16:30:30.713127+0000","last_clean":"2023-11-08T16:30:30.713127+0000","last_became_active":"2023-11-08T16:30:29.706081+0000","last_became_peered":"2023-11-08T16:30:29.706081+0000","last_unstale":"2023-11-08T16:30:30.713127+0000","last_undegraded":"2023-11-08T16:30:30.713127+0000","last_fullsized":"2023-11-08T16:30:30.713127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:50:35.904021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.733520+0000","last_change":"2023-11-08T16:30:30.728611+0000","last_active":"2023-11-08T16:30:30.733520+0000","last_peered":"2023-11-08T16:30:30.733520+0000","last_clean":"2023-11-08T16:30:30.733520+0000","last_became_active":"2023-11-08T16:30:30.728332+0000","last_became_peered":"2023-11-08T16:30:30.728332+0000","last_unstale":"2023-11-08T16:30:30.733520+0000","last_undegraded":"2023-11-08T16:30:30.733520+0000","last_fullsized":"2023-11-08T16:30:30.733520+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:42.783481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.721817+0000","last_change":"2023-11-08T16:30:29.709784+0000","last_active":"2023-11-08T16:30:30.721817+0000","last_peered":"2023-11-08T16:30:30.721817+0000","last_clean":"2023-11-08T16:30:30.721817+0000","last_became_active":"2023-11-08T16:30:29.709427+0000","last_became_peered":"2023-11-08T16:30:29.709427+0000","last_unstale":"2023-11-08T16:30:30.721817+0000","last_undegraded":"2023-11-08T16:30:30.721817+0000","last_fullsized":"2023-11-08T16:30:30.721817+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:50:35.904021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745659+0000","last_change":"2023-11-08T16:30:30.738578+0000","last_active":"2023-11-08T16:30:30.745659+0000","last_peered":"2023-11-08T16:30:30.745659+0000","last_clean":"2023-11-08T16:30:30.745659+0000","last_became_active":"2023-11-08T16:30:30.738305+0000","last_became_peered":"2023-11-08T16:30:30.738305+0000","last_unstale":"2023-11-08T16:30:30.745659+0000","last_undegraded":"2023-11-08T16:30:30.745659+0000","last_fullsized":"2023-11-08T16:30:30.745659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:43.258248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.713123+0000","last_change":"2023-11-08T16:30:29.708860+0000","last_active":"2023-11-08T16:30:30.713123+0000","last_peered":"2023-11-08T16:30:30.713123+0000","last_clean":"2023-11-08T16:30:30.713123+0000","last_became_active":"2023-11-08T16:30:29.707687+0000","last_became_peered":"2023-11-08T16:30:29.707687+0000","last_unstale":"2023-11-08T16:30:30.713123+0000","last_undegraded":"2023-11-08T16:30:30.713123+0000","last_fullsized":"2023-11-08T16:30:30.713123+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:15.005227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750200+0000","last_change":"2023-11-08T16:30:30.739635+0000","last_active":"2023-11-08T16:30:30.750200+0000","last_peered":"2023-11-08T16:30:30.750200+0000","last_clean":"2023-11-08T16:30:30.750200+0000","last_became_active":"2023-11-08T16:30:30.737160+0000","last_became_peered":"2023-11-08T16:30:30.737160+0000","last_unstale":"2023-11-08T16:30:30.750200+0000","last_undegraded":"2023-11-08T16:30:30.750200+0000","last_fullsized":"2023-11-08T16:30:30.750200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:50:36.916601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.735337+0000","last_change":"2023-11-08T16:30:29.717678+0000","last_active":"2023-11-08T16:30:30.735337+0000","last_peered":"2023-11-08T16:30:30.735337+0000","last_clean":"2023-11-08T16:30:30.735337+0000","last_became_active":"2023-11-08T16:30:29.717280+0000","last_became_peered":"2023-11-08T16:30:29.717280+0000","last_unstale":"2023-11-08T16:30:30.735337+0000","last_undegraded":"2023-11-08T16:30:30.735337+0000","last_fullsized":"2023-11-08T16:30:30.735337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:06.494468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751934+0000","last_change":"2023-11-08T16:30:30.746551+0000","last_active":"2023-11-08T16:30:30.751934+0000","last_peered":"2023-11-08T16:30:30.751934+0000","last_clean":"2023-11-08T16:30:30.751934+0000","last_became_active":"2023-11-08T16:30:30.746253+0000","last_became_peered":"2023-11-08T16:30:30.746253+0000","last_unstale":"2023-11-08T16:30:30.751934+0000","last_undegraded":"2023-11-08T16:30:30.751934+0000","last_fullsized":"2023-11-08T16:30:30.751934+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.503135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.727216+0000","last_change":"2023-11-08T16:30:29.711664+0000","last_active":"2023-11-08T16:30:30.727216+0000","last_peered":"2023-11-08T16:30:30.727216+0000","last_clean":"2023-11-08T16:30:30.727216+0000","last_became_active":"2023-11-08T16:30:29.710855+0000","last_became_peered":"2023-11-08T16:30:29.710855+0000","last_unstale":"2023-11-08T16:30:30.727216+0000","last_undegraded":"2023-11-08T16:30:30.727216+0000","last_fullsized":"2023-11-08T16:30:30.727216+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:43:04.952635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.737334+0000","last_change":"2023-11-08T16:30:30.733579+0000","last_active":"2023-11-08T16:30:30.737334+0000","last_peered":"2023-11-08T16:30:30.737334+0000","last_clean":"2023-11-08T16:30:30.737334+0000","last_became_active":"2023-11-08T16:30:30.730239+0000","last_became_peered":"2023-11-08T16:30:30.730239+0000","last_unstale":"2023-11-08T16:30:30.737334+0000","last_undegraded":"2023-11-08T16:30:30.737334+0000","last_fullsized":"2023-11-08T16:30:30.737334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:40:52.036835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.707768+0000","last_change":"2023-11-08T16:30:29.720980+0000","last_active":"2023-11-08T16:30:30.707768+0000","last_peered":"2023-11-08T16:30:30.707768+0000","last_clean":"2023-11-08T16:30:30.707768+0000","last_became_active":"2023-11-08T16:30:29.720697+0000","last_became_peered":"2023-11-08T16:30:29.720697+0000","last_unstale":"2023-11-08T16:30:30.707768+0000","last_undegraded":"2023-11-08T16:30:30.707768+0000","last_fullsized":"2023-11-08T16:30:30.707768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:08.194612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T16:30:29.706461+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:30:29.706461+0000","last_peered":"2023-11-08T16:30:29.706461+0000","last_clean":"2023-11-08T16:30:29.706461+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:30:29.706461+0000","last_undegraded":"2023-11-08T16:30:29.706461+0000","last_fullsized":"2023-11-08T16:30:29.706461+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":740898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":4.2729999999999997,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":0.81499999999999995}]},{"osd":1,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":1.131},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":4.2249999999999996,"5min":4.2249999999999996,"15min":4.2249999999999996},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.278}]},{"osd":3,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":4.181,"5min":4.181,"15min":4.181},"last":1.405}]},{"osd":4,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.635}]},{"osd":5,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":1.4970000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":1.026,"5min":1.026,"15min":1.026},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":1.7},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.5649999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.77600000000000002}]},{"osd":1,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":1.472}]},{"osd":2,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.71699999999999997}]},{"osd":3,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.3600000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.619}]},{"osd":5,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":1.577}]},{"osd":7,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"min":{"1min":1.119,"5min":1.119,"15min":1.119},"max":{"1min":3.8959999999999999,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":1.524},{"interface":"front","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.2470000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.367},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.46}]},{"osd":1,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":3.8690000000000002,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.4330000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.534},{"interface":"front","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.3879999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.262}]},{"osd":4,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.9790000000000001,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":1.04},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"max":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"last":1.2070000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.5860000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":150544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.95699999999999996}]},{"osd":1,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.90000000000000002}]},{"osd":2,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.173},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.008}]},{"osd":3,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.4259999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.123},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.0629999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.3320000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"min":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.631}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":152074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.35799999999999998}]},{"osd":1,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":0.57399999999999995}]},{"osd":4,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.62},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.0469999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":1.018},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.762,"5min":2.762,"15min":2.762},"last":0.748}]},{"osd":7,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.8700000000000001,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.1120000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.5369999999999999,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":4.4980000000000002,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0209999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.5899999999999999,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":1.0669999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":4.9409999999999998,"5min":4.9409999999999998,"15min":4.9409999999999998},"last":1.119},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.4119999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.298},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.88400000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.536,"5min":3.536,"15min":3.536},"last":1.286},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":4.7549999999999999,"5min":4.7549999999999999,"15min":4.7549999999999999},"last":1.341}]},{"osd":7,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.546,"15min":1.546},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.635,"5min":1.635,"15min":1.635},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":3.4950000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.236}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":152074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":1.236}]},{"osd":2,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.0339999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":1.139}]},{"osd":3,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.3179999999999996,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.9390000000000001,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":0.90800000000000003}]},{"osd":4,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":3.1659999999999999,"5min":3.1659999999999999,"15min":3.1659999999999999},"last":1.099},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.734,"5min":2.734,"15min":2.734},"last":1.345}]},{"osd":5,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.913,"5min":1.913,"15min":1.913},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":8.4730000000000008,"5min":8.4730000000000008,"15min":8.4730000000000008},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":8.8040000000000003,"5min":8.8040000000000003,"15min":8.8040000000000003},"last":1.1839999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":3.984,"5min":3.984,"15min":3.984},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.538,"5min":1.538,"15min":1.538},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.319}]},{"osd":7,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":1.712,"15min":1.712},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":4.3609999999999998,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":1.657},{"interface":"front","average":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.3840000000000003,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":1.6850000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":740808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.66800000000000004}]},{"osd":2,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.93500000000000005}]},{"osd":3,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.7679999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":1.109}]},{"osd":4,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.431},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.867,"5min":2.867,"15min":2.867},"last":1.28}]},{"osd":5,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.7280000000000002,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.466},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.252}]},{"osd":6,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.2}]},{"osd":7,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.345},{"interface":"front","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":1.3180000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:30:35.234 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:30:35.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.485+0000 7f45c8988640 1 Processor -- start 2023-11-08T16:30:35.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.486+0000 7f45c8988640 1 -- start start 2023-11-08T16:30:35.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.487+0000 7f45c8988640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45c40ced10 0x7f45c40cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:35.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.487+0000 7f45c8988640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c406b860 con 0x7f45c406bcd0 2023-11-08T16:30:35.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.487+0000 7f45c8988640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c40cf6b0 con 0x7f45c415ab70 2023-11-08T16:30:35.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.487+0000 7f45c8988640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c415e450 con 0x7f45c40ced10 2023-11-08T16:30:35.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.488+0000 7f45c259c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45c40ced10 0x7f45c40cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:35.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.488+0000 7f45c359e640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f45c406bcd0 0x7f45c406c160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53674/0 (socket says 172.21.15.81:53674) 2023-11-08T16:30:35.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.488+0000 7f45c359e640 1 -- 172.21.15.81:0/1177201563 learned_addr learned my addr 172.21.15.81:0/1177201563 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:35.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.489+0000 7f45c1d9b640 1 -- 172.21.15.81:0/1177201563 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1330973598 0 0) 0x7f45c406b860 con 0x7f45c406bcd0 2023-11-08T16:30:35.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.489+0000 7f45c1d9b640 1 -- 172.21.15.81:0/1177201563 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45ac003520 con 0x7f45c406bcd0 2023-11-08T16:30:35.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.489+0000 7f45c259c640 1 -- 172.21.15.81:0/1177201563 >> v1:172.21.15.93:6789/0 conn(0x7f45c415ab70 legacy=0x7f45c41613e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:35.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.489+0000 7f45c259c640 1 -- 172.21.15.81:0/1177201563 >> v1:172.21.15.81:6789/0 conn(0x7f45c406bcd0 legacy=0x7f45c406c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:35.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.489+0000 7f45c259c640 1 -- 172.21.15.81:0/1177201563 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c415d840 con 0x7f45c40ced10 2023-11-08T16:30:35.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.490+0000 7f45c259c640 1 --2- 172.21.15.81:0/1177201563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45c40ced10 0x7f45c40cf170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009800 tx=0x7f45b802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90e56ae94808c42b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:35.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.490+0000 7f45c1d9b640 1 -- 172.21.15.81:0/1177201563 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8031890 con 0x7f45c40ced10 2023-11-08T16:30:35.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.491+0000 7f45c1d9b640 1 -- 172.21.15.81:0/1177201563 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b80319f0 con 0x7f45c40ced10 2023-11-08T16:30:35.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.491+0000 7f45c1d9b640 1 -- 172.21.15.81:0/1177201563 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8031d60 con 0x7f45c40ced10 2023-11-08T16:30:35.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.491+0000 7f45c8988640 1 -- 172.21.15.81:0/1177201563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45c40ced10 msgr2=0x7f45c40cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:35.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.491+0000 7f45c8988640 1 --2- 172.21.15.81:0/1177201563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45c40ced10 0x7f45c40cf170 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009800 tx=0x7f45b802b930 comp rx=0 tx=0).stop 2023-11-08T16:30:35.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.491+0000 7f45c8988640 1 -- 172.21.15.81:0/1177201563 shutdown_connections 2023-11-08T16:30:35.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.491+0000 7f45c8988640 1 --2- 172.21.15.81:0/1177201563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45c40ced10 0x7f45c40cf170 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.491+0000 7f45c8988640 1 -- 172.21.15.81:0/1177201563 >> 172.21.15.81:0/1177201563 conn(0x7f45c406a4a0 msgr2=0x7f45c406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:35.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.491+0000 7f45c8988640 1 -- 172.21.15.81:0/1177201563 shutdown_connections 2023-11-08T16:30:35.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.491+0000 7f45c8988640 1 -- 172.21.15.81:0/1177201563 wait complete. 2023-11-08T16:30:35.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.493+0000 7f45c8988640 1 Processor -- start 2023-11-08T16:30:35.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.493+0000 7f45c8988640 1 -- start start 2023-11-08T16:30:35.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.494+0000 7f45c8988640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45c406bcd0 0x7f45c41f2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:35.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.494+0000 7f45c8988640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c40ced10 0x7f45c4141dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:35.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.494+0000 7f45c8988640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45c415ab70 0x7f45c4156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:35.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.494+0000 7f45c8988640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41f26b0 con 0x7f45c40ced10 2023-11-08T16:30:35.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.494+0000 7f45c8988640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c4157180 con 0x7f45c406bcd0 2023-11-08T16:30:35.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.494+0000 7f45c8988640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c415b550 con 0x7f45c415ab70 2023-11-08T16:30:35.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.494+0000 7f45c259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c40ced10 0x7f45c4141dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:35.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.494+0000 7f45c259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c40ced10 0x7f45c4141dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40808/0 (socket says 172.21.15.81:40808) 2023-11-08T16:30:35.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.494+0000 7f45c259c640 1 -- 172.21.15.81:0/1722051129 learned_addr learned my addr 172.21.15.81:0/1722051129 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:35.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.495+0000 7f45c2d9d640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45c406bcd0 0x7f45c41f2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:35.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.495+0000 7f45c359e640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45c415ab70 0x7f45c4156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:35.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.495+0000 7f45c259c640 1 -- 172.21.15.81:0/1722051129 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45c415ab70 msgr2=0x7f45c4156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:35.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.495+0000 7f45c259c640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45c415ab70 0x7f45c4156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.495+0000 7f45c259c640 1 -- 172.21.15.81:0/1722051129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45c406bcd0 msgr2=0x7f45c41f2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:35.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.495+0000 7f45c259c640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45c406bcd0 0x7f45c41f2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.495+0000 7f45c259c640 1 -- 172.21.15.81:0/1722051129 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b8009400 con 0x7f45c40ced10 2023-11-08T16:30:35.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.495+0000 7f45c259c640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c40ced10 0x7f45c4141dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009720 tx=0x7f45b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:35.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.496+0000 7f45ab7fe640 1 -- 172.21.15.81:0/1722051129 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8003810 con 0x7f45c40ced10 2023-11-08T16:30:35.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.496+0000 7f45ab7fe640 1 -- 172.21.15.81:0/1722051129 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b8003970 con 0x7f45c40ced10 2023-11-08T16:30:35.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.496+0000 7f45ab7fe640 1 -- 172.21.15.81:0/1722051129 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8003ce0 con 0x7f45c40ced10 2023-11-08T16:30:35.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.496+0000 7f45c8988640 1 -- 172.21.15.81:0/1722051129 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c415b830 con 0x7f45c40ced10 2023-11-08T16:30:35.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.496+0000 7f45c8988640 1 -- 172.21.15.81:0/1722051129 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c4142300 con 0x7f45c40ced10 2023-11-08T16:30:35.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.497+0000 7f45c8988640 1 -- 172.21.15.81:0/1722051129 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c4067800 con 0x7f45c40ced10 2023-11-08T16:30:35.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.498+0000 7f45ab7fe640 1 -- 172.21.15.81:0/1722051129 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f45b803a450 con 0x7f45c40ced10 2023-11-08T16:30:35.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.499+0000 7f45ab7fe640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45b006aea0 0x7f45b006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:35.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.499+0000 7f45ab7fe640 1 -- 172.21.15.81:0/1722051129 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45b80aba70 con 0x7f45c40ced10 2023-11-08T16:30:35.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.499+0000 7f45c2d9d640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45b006aea0 0x7f45b006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:35.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.500+0000 7f45c2d9d640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45b006aea0 0x7f45b006d360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f45b4007fd0 tx=0x7f45b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:35.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.503+0000 7f45ab7fe640 1 -- 172.21.15.81:0/1722051129 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f45b8076b30 con 0x7f45c40ced10 2023-11-08T16:30:35.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.691+0000 7f45c8988640 1 -- 172.21.15.81:0/1722051129 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45c415de00 con 0x7f45b006aea0 2023-11-08T16:30:35.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.701+0000 7f45ab7fe640 1 -- 172.21.15.81:0/1722051129 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286473 (secure 0 0 0) 0x7f45b40050d0 con 0x7f45b006aea0 2023-11-08T16:30:35.706 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:35.708 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:30:35.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.705+0000 7f45c8988640 1 -- 172.21.15.81:0/1722051129 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45b006aea0 msgr2=0x7f45b006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:35.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.705+0000 7f45c8988640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45b006aea0 0x7f45b006d360 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f45b4007fd0 tx=0x7f45b4009040 comp rx=0 tx=0).stop 2023-11-08T16:30:35.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.706+0000 7f45c8988640 1 -- 172.21.15.81:0/1722051129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c40ced10 msgr2=0x7f45c4141dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:35.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.706+0000 7f45c8988640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c40ced10 0x7f45c4141dc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009720 tx=0x7f45b80046c0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.706+0000 7f45c8988640 1 -- 172.21.15.81:0/1722051129 shutdown_connections 2023-11-08T16:30:35.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.706+0000 7f45c8988640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45b006aea0 0x7f45b006d360 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.706+0000 7f45c8988640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f45c415ab70 0x7f45c4156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.706+0000 7f45c8988640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f45c40ced10 0x7f45c4141dc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.706+0000 7f45c8988640 1 --2- 172.21.15.81:0/1722051129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f45c406bcd0 0x7f45c41f2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.706+0000 7f45c8988640 1 -- 172.21.15.81:0/1722051129 >> 172.21.15.81:0/1722051129 conn(0x7f45c406a4a0 msgr2=0x7f45c40707d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.706+0000 7f45c8988640 1 -- 172.21.15.81:0/1722051129 shutdown_connections 2023-11-08T16:30:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.706+0000 7f45c8988640 1 -- 172.21.15.81:0/1722051129 wait complete. 2023-11-08T16:30:35.725 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2023-11-08T16:30:34.574988+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":75,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220440,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754280,"statfs":{"total":773094113280,"available":772868382720,"internally_reserved":0,"allocated":4333568,"data_stored":2389076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.900592"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.738655+0000","last_change":"2023-11-08T16:30:30.734047+0000","last_active":"2023-11-08T16:30:30.738655+0000","last_peered":"2023-11-08T16:30:30.738655+0000","last_clean":"2023-11-08T16:30:30.738655+0000","last_became_active":"2023-11-08T16:30:30.733209+0000","last_became_peered":"2023-11-08T16:30:30.733209+0000","last_unstale":"2023-11-08T16:30:30.738655+0000","last_undegraded":"2023-11-08T16:30:30.738655+0000","last_fullsized":"2023-11-08T16:30:30.738655+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:09:34.574098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751952+0000","last_change":"2023-11-08T16:30:30.744347+0000","last_active":"2023-11-08T16:30:30.751952+0000","last_peered":"2023-11-08T16:30:30.751952+0000","last_clean":"2023-11-08T16:30:30.751952+0000","last_became_active":"2023-11-08T16:30:30.743988+0000","last_became_peered":"2023-11-08T16:30:30.743988+0000","last_unstale":"2023-11-08T16:30:30.751952+0000","last_undegraded":"2023-11-08T16:30:30.751952+0000","last_fullsized":"2023-11-08T16:30:30.751952+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:41:10.086203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.734167+0000","last_change":"2023-11-08T16:30:30.728777+0000","last_active":"2023-11-08T16:30:30.734167+0000","last_peered":"2023-11-08T16:30:30.734167+0000","last_clean":"2023-11-08T16:30:30.734167+0000","last_became_active":"2023-11-08T16:30:30.728478+0000","last_became_peered":"2023-11-08T16:30:30.728478+0000","last_unstale":"2023-11-08T16:30:30.734167+0000","last_undegraded":"2023-11-08T16:30:30.734167+0000","last_fullsized":"2023-11-08T16:30:30.734167+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:43:05.965214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751008+0000","last_change":"2023-11-08T16:30:30.743548+0000","last_active":"2023-11-08T16:30:30.751008+0000","last_peered":"2023-11-08T16:30:30.751008+0000","last_clean":"2023-11-08T16:30:30.751008+0000","last_became_active":"2023-11-08T16:30:30.743257+0000","last_became_peered":"2023-11-08T16:30:30.743257+0000","last_unstale":"2023-11-08T16:30:30.751008+0000","last_undegraded":"2023-11-08T16:30:30.751008+0000","last_fullsized":"2023-11-08T16:30:30.751008+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:49.297882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751594+0000","last_change":"2023-11-08T16:30:30.748616+0000","last_active":"2023-11-08T16:30:30.751594+0000","last_peered":"2023-11-08T16:30:30.751594+0000","last_clean":"2023-11-08T16:30:30.751594+0000","last_became_active":"2023-11-08T16:30:30.747263+0000","last_became_peered":"2023-11-08T16:30:30.747263+0000","last_unstale":"2023-11-08T16:30:30.751594+0000","last_undegraded":"2023-11-08T16:30:30.751594+0000","last_fullsized":"2023-11-08T16:30:30.751594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.503135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.753543+0000","last_change":"2023-11-08T16:30:30.744435+0000","last_active":"2023-11-08T16:30:30.753543+0000","last_peered":"2023-11-08T16:30:30.753543+0000","last_clean":"2023-11-08T16:30:30.753543+0000","last_became_active":"2023-11-08T16:30:30.743961+0000","last_became_peered":"2023-11-08T16:30:30.743961+0000","last_unstale":"2023-11-08T16:30:30.753543+0000","last_undegraded":"2023-11-08T16:30:30.753543+0000","last_fullsized":"2023-11-08T16:30:30.753543+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:36.347998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.736330+0000","last_change":"2023-11-08T16:30:30.732447+0000","last_active":"2023-11-08T16:30:30.736330+0000","last_peered":"2023-11-08T16:30:30.736330+0000","last_clean":"2023-11-08T16:30:30.736330+0000","last_became_active":"2023-11-08T16:30:30.732168+0000","last_became_peered":"2023-11-08T16:30:30.732168+0000","last_unstale":"2023-11-08T16:30:30.736330+0000","last_undegraded":"2023-11-08T16:30:30.736330+0000","last_fullsized":"2023-11-08T16:30:30.736330+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:17:55.087658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.733699+0000","last_change":"2023-11-08T16:30:30.728079+0000","last_active":"2023-11-08T16:30:30.733699+0000","last_peered":"2023-11-08T16:30:30.733699+0000","last_clean":"2023-11-08T16:30:30.733699+0000","last_became_active":"2023-11-08T16:30:30.727782+0000","last_became_peered":"2023-11-08T16:30:30.727782+0000","last_unstale":"2023-11-08T16:30:30.733699+0000","last_undegraded":"2023-11-08T16:30:30.733699+0000","last_fullsized":"2023-11-08T16:30:30.733699+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:50:36.916601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.757155+0000","last_change":"2023-11-08T16:30:30.751749+0000","last_active":"2023-11-08T16:30:30.757155+0000","last_peered":"2023-11-08T16:30:30.757155+0000","last_clean":"2023-11-08T16:30:30.757155+0000","last_became_active":"2023-11-08T16:30:30.751450+0000","last_became_peered":"2023-11-08T16:30:30.751450+0000","last_unstale":"2023-11-08T16:30:30.757155+0000","last_undegraded":"2023-11-08T16:30:30.757155+0000","last_fullsized":"2023-11-08T16:30:30.757155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:01:01.465958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.753833+0000","last_change":"2023-11-08T16:30:30.749015+0000","last_active":"2023-11-08T16:30:30.753833+0000","last_peered":"2023-11-08T16:30:30.753833+0000","last_clean":"2023-11-08T16:30:30.753833+0000","last_became_active":"2023-11-08T16:30:30.746444+0000","last_became_peered":"2023-11-08T16:30:30.746444+0000","last_unstale":"2023-11-08T16:30:30.753833+0000","last_undegraded":"2023-11-08T16:30:30.753833+0000","last_fullsized":"2023-11-08T16:30:30.753833+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:29:58.478672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745398+0000","last_change":"2023-11-08T16:30:30.739840+0000","last_active":"2023-11-08T16:30:30.745398+0000","last_peered":"2023-11-08T16:30:30.745398+0000","last_clean":"2023-11-08T16:30:30.745398+0000","last_became_active":"2023-11-08T16:30:30.739556+0000","last_became_peered":"2023-11-08T16:30:30.739556+0000","last_unstale":"2023-11-08T16:30:30.745398+0000","last_undegraded":"2023-11-08T16:30:30.745398+0000","last_fullsized":"2023-11-08T16:30:30.745398+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:17.680411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.754292+0000","last_change":"2023-11-08T16:30:30.749585+0000","last_active":"2023-11-08T16:30:30.754292+0000","last_peered":"2023-11-08T16:30:30.754292+0000","last_clean":"2023-11-08T16:30:30.754292+0000","last_became_active":"2023-11-08T16:30:30.749041+0000","last_became_peered":"2023-11-08T16:30:30.749041+0000","last_unstale":"2023-11-08T16:30:30.754292+0000","last_undegraded":"2023-11-08T16:30:30.754292+0000","last_fullsized":"2023-11-08T16:30:30.754292+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:17.680411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.731987+0000","last_change":"2023-11-08T16:30:30.728270+0000","last_active":"2023-11-08T16:30:30.731987+0000","last_peered":"2023-11-08T16:30:30.731987+0000","last_clean":"2023-11-08T16:30:30.731987+0000","last_became_active":"2023-11-08T16:30:30.728009+0000","last_became_peered":"2023-11-08T16:30:30.728009+0000","last_unstale":"2023-11-08T16:30:30.731987+0000","last_undegraded":"2023-11-08T16:30:30.731987+0000","last_fullsized":"2023-11-08T16:30:30.731987+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:36:25.618863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.748925+0000","last_change":"2023-11-08T16:30:30.735428+0000","last_active":"2023-11-08T16:30:30.748925+0000","last_peered":"2023-11-08T16:30:30.748925+0000","last_clean":"2023-11-08T16:30:30.748925+0000","last_became_active":"2023-11-08T16:30:30.735150+0000","last_became_peered":"2023-11-08T16:30:30.735150+0000","last_unstale":"2023-11-08T16:30:30.748925+0000","last_undegraded":"2023-11-08T16:30:30.748925+0000","last_fullsized":"2023-11-08T16:30:30.748925+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:40:52.036835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.739584+0000","last_change":"2023-11-08T16:30:30.733711+0000","last_active":"2023-11-08T16:30:30.739584+0000","last_peered":"2023-11-08T16:30:30.739584+0000","last_clean":"2023-11-08T16:30:30.739584+0000","last_became_active":"2023-11-08T16:30:30.733177+0000","last_became_peered":"2023-11-08T16:30:30.733177+0000","last_unstale":"2023-11-08T16:30:30.739584+0000","last_undegraded":"2023-11-08T16:30:30.739584+0000","last_fullsized":"2023-11-08T16:30:30.739584+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:36.347998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.754117+0000","last_change":"2023-11-08T16:30:30.750658+0000","last_active":"2023-11-08T16:30:30.754117+0000","last_peered":"2023-11-08T16:30:30.754117+0000","last_clean":"2023-11-08T16:30:30.754117+0000","last_became_active":"2023-11-08T16:30:30.750447+0000","last_became_peered":"2023-11-08T16:30:30.750447+0000","last_unstale":"2023-11-08T16:30:30.754117+0000","last_undegraded":"2023-11-08T16:30:30.754117+0000","last_fullsized":"2023-11-08T16:30:30.754117+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:43:05.965214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.744573+0000","last_change":"2023-11-08T16:30:30.737545+0000","last_active":"2023-11-08T16:30:30.744573+0000","last_peered":"2023-11-08T16:30:30.744573+0000","last_clean":"2023-11-08T16:30:30.744573+0000","last_became_active":"2023-11-08T16:30:30.737221+0000","last_became_peered":"2023-11-08T16:30:30.737221+0000","last_unstale":"2023-11-08T16:30:30.744573+0000","last_undegraded":"2023-11-08T16:30:30.744573+0000","last_fullsized":"2023-11-08T16:30:30.744573+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:30:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751940+0000","last_change":"2023-11-08T16:30:30.743535+0000","last_active":"2023-11-08T16:30:30.751940+0000","last_peered":"2023-11-08T16:30:30.751940+0000","last_clean":"2023-11-08T16:30:30.751940+0000","last_became_active":"2023-11-08T16:30:30.743284+0000","last_became_peered":"2023-11-08T16:30:30.743284+0000","last_unstale":"2023-11-08T16:30:30.751940+0000","last_undegraded":"2023-11-08T16:30:30.751940+0000","last_fullsized":"2023-11-08T16:30:30.751940+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:17:55.087658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.755716+0000","last_change":"2023-11-08T16:30:30.751153+0000","last_active":"2023-11-08T16:30:30.755716+0000","last_peered":"2023-11-08T16:30:30.755716+0000","last_clean":"2023-11-08T16:30:30.755716+0000","last_became_active":"2023-11-08T16:30:30.750642+0000","last_became_peered":"2023-11-08T16:30:30.750642+0000","last_unstale":"2023-11-08T16:30:30.755716+0000","last_undegraded":"2023-11-08T16:30:30.755716+0000","last_fullsized":"2023-11-08T16:30:30.755716+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:41.681026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.738171+0000","last_change":"2023-11-08T16:30:30.733893+0000","last_active":"2023-11-08T16:30:30.738171+0000","last_peered":"2023-11-08T16:30:30.738171+0000","last_clean":"2023-11-08T16:30:30.738171+0000","last_became_active":"2023-11-08T16:30:30.733611+0000","last_became_peered":"2023-11-08T16:30:30.733611+0000","last_unstale":"2023-11-08T16:30:30.738171+0000","last_undegraded":"2023-11-08T16:30:30.738171+0000","last_fullsized":"2023-11-08T16:30:30.738171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:26:23.385574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746714+0000","last_change":"2023-11-08T16:30:30.740830+0000","last_active":"2023-11-08T16:30:30.746714+0000","last_peered":"2023-11-08T16:30:30.746714+0000","last_clean":"2023-11-08T16:30:30.746714+0000","last_became_active":"2023-11-08T16:30:30.740242+0000","last_became_peered":"2023-11-08T16:30:30.740242+0000","last_unstale":"2023-11-08T16:30:30.746714+0000","last_undegraded":"2023-11-08T16:30:30.746714+0000","last_fullsized":"2023-11-08T16:30:30.746714+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:26:23.385574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.757202+0000","last_change":"2023-11-08T16:30:30.754302+0000","last_active":"2023-11-08T16:30:30.757202+0000","last_peered":"2023-11-08T16:30:30.757202+0000","last_clean":"2023-11-08T16:30:30.757202+0000","last_became_active":"2023-11-08T16:30:30.754014+0000","last_became_peered":"2023-11-08T16:30:30.754014+0000","last_unstale":"2023-11-08T16:30:30.757202+0000","last_undegraded":"2023-11-08T16:30:30.757202+0000","last_fullsized":"2023-11-08T16:30:30.757202+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:47.363218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.743081+0000","last_change":"2023-11-08T16:30:30.735655+0000","last_active":"2023-11-08T16:30:30.743081+0000","last_peered":"2023-11-08T16:30:30.743081+0000","last_clean":"2023-11-08T16:30:30.743081+0000","last_became_active":"2023-11-08T16:30:30.735401+0000","last_became_peered":"2023-11-08T16:30:30.735401+0000","last_unstale":"2023-11-08T16:30:30.743081+0000","last_undegraded":"2023-11-08T16:30:30.743081+0000","last_fullsized":"2023-11-08T16:30:30.743081+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:29:58.478672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.730678+0000","last_change":"2023-11-08T16:30:30.722152+0000","last_active":"2023-11-08T16:30:30.730678+0000","last_peered":"2023-11-08T16:30:30.730678+0000","last_clean":"2023-11-08T16:30:30.730678+0000","last_became_active":"2023-11-08T16:30:30.721663+0000","last_became_peered":"2023-11-08T16:30:30.721663+0000","last_unstale":"2023-11-08T16:30:30.730678+0000","last_undegraded":"2023-11-08T16:30:30.730678+0000","last_fullsized":"2023-11-08T16:30:30.730678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:16.017807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750291+0000","last_change":"2023-11-08T16:30:30.736433+0000","last_active":"2023-11-08T16:30:30.750291+0000","last_peered":"2023-11-08T16:30:30.750291+0000","last_clean":"2023-11-08T16:30:30.750291+0000","last_became_active":"2023-11-08T16:30:30.734973+0000","last_became_peered":"2023-11-08T16:30:30.734973+0000","last_unstale":"2023-11-08T16:30:30.750291+0000","last_undegraded":"2023-11-08T16:30:30.750291+0000","last_fullsized":"2023-11-08T16:30:30.750291+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:43.258248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.754116+0000","last_change":"2023-11-08T16:30:30.748464+0000","last_active":"2023-11-08T16:30:30.754116+0000","last_peered":"2023-11-08T16:30:30.754116+0000","last_clean":"2023-11-08T16:30:30.754116+0000","last_became_active":"2023-11-08T16:30:30.748172+0000","last_became_peered":"2023-11-08T16:30:30.748172+0000","last_unstale":"2023-11-08T16:30:30.754116+0000","last_undegraded":"2023-11-08T16:30:30.754116+0000","last_fullsized":"2023-11-08T16:30:30.754116+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:29.269838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750542+0000","last_change":"2023-11-08T16:30:30.741546+0000","last_active":"2023-11-08T16:30:30.750542+0000","last_peered":"2023-11-08T16:30:30.750542+0000","last_clean":"2023-11-08T16:30:30.750542+0000","last_became_active":"2023-11-08T16:30:30.740966+0000","last_became_peered":"2023-11-08T16:30:30.740966+0000","last_unstale":"2023-11-08T16:30:30.750542+0000","last_undegraded":"2023-11-08T16:30:30.750542+0000","last_fullsized":"2023-11-08T16:30:30.750542+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:41.681026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.737820+0000","last_change":"2023-11-08T16:30:30.730928+0000","last_active":"2023-11-08T16:30:30.737820+0000","last_peered":"2023-11-08T16:30:30.737820+0000","last_clean":"2023-11-08T16:30:30.737820+0000","last_became_active":"2023-11-08T16:30:30.730624+0000","last_became_peered":"2023-11-08T16:30:30.730624+0000","last_unstale":"2023-11-08T16:30:30.737820+0000","last_undegraded":"2023-11-08T16:30:30.737820+0000","last_fullsized":"2023-11-08T16:30:30.737820+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:17:55.087658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.756704+0000","last_change":"2023-11-08T16:30:30.751206+0000","last_active":"2023-11-08T16:30:30.756704+0000","last_peered":"2023-11-08T16:30:30.756704+0000","last_clean":"2023-11-08T16:30:30.756704+0000","last_became_active":"2023-11-08T16:30:30.750976+0000","last_became_peered":"2023-11-08T16:30:30.750976+0000","last_unstale":"2023-11-08T16:30:30.756704+0000","last_undegraded":"2023-11-08T16:30:30.756704+0000","last_fullsized":"2023-11-08T16:30:30.756704+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:22:53.461699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.743725+0000","last_change":"2023-11-08T16:30:30.735107+0000","last_active":"2023-11-08T16:30:30.743725+0000","last_peered":"2023-11-08T16:30:30.743725+0000","last_clean":"2023-11-08T16:30:30.743725+0000","last_became_active":"2023-11-08T16:30:30.734807+0000","last_became_peered":"2023-11-08T16:30:30.734807+0000","last_unstale":"2023-11-08T16:30:30.743725+0000","last_undegraded":"2023-11-08T16:30:30.743725+0000","last_fullsized":"2023-11-08T16:30:30.743725+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:29.269838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.730583+0000","last_change":"2023-11-08T16:30:30.719326+0000","last_active":"2023-11-08T16:30:30.730583+0000","last_peered":"2023-11-08T16:30:30.730583+0000","last_clean":"2023-11-08T16:30:30.730583+0000","last_became_active":"2023-11-08T16:30:30.718978+0000","last_became_peered":"2023-11-08T16:30:30.718978+0000","last_unstale":"2023-11-08T16:30:30.730583+0000","last_undegraded":"2023-11-08T16:30:30.730583+0000","last_fullsized":"2023-11-08T16:30:30.730583+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:49.297882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.743772+0000","last_change":"2023-11-08T16:30:30.735879+0000","last_active":"2023-11-08T16:30:30.743772+0000","last_peered":"2023-11-08T16:30:30.743772+0000","last_clean":"2023-11-08T16:30:30.743772+0000","last_became_active":"2023-11-08T16:30:30.734679+0000","last_became_peered":"2023-11-08T16:30:30.734679+0000","last_unstale":"2023-11-08T16:30:30.743772+0000","last_undegraded":"2023-11-08T16:30:30.743772+0000","last_fullsized":"2023-11-08T16:30:30.743772+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:16.017807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.743977+0000","last_change":"2023-11-08T16:30:30.736452+0000","last_active":"2023-11-08T16:30:30.743977+0000","last_peered":"2023-11-08T16:30:30.743977+0000","last_clean":"2023-11-08T16:30:30.743977+0000","last_became_active":"2023-11-08T16:30:30.736021+0000","last_became_peered":"2023-11-08T16:30:30.736021+0000","last_unstale":"2023-11-08T16:30:30.743977+0000","last_undegraded":"2023-11-08T16:30:30.743977+0000","last_fullsized":"2023-11-08T16:30:30.743977+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:16.017807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.740003+0000","last_change":"2023-11-08T16:30:29.713222+0000","last_active":"2023-11-08T16:30:30.740003+0000","last_peered":"2023-11-08T16:30:30.740003+0000","last_clean":"2023-11-08T16:30:30.740003+0000","last_became_active":"2023-11-08T16:30:29.712473+0000","last_became_peered":"2023-11-08T16:30:29.712473+0000","last_unstale":"2023-11-08T16:30:30.740003+0000","last_undegraded":"2023-11-08T16:30:30.740003+0000","last_fullsized":"2023-11-08T16:30:30.740003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:42.245668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746520+0000","last_change":"2023-11-08T16:30:30.744333+0000","last_active":"2023-11-08T16:30:30.746520+0000","last_peered":"2023-11-08T16:30:30.746520+0000","last_clean":"2023-11-08T16:30:30.746520+0000","last_became_active":"2023-11-08T16:30:30.744066+0000","last_became_peered":"2023-11-08T16:30:30.744066+0000","last_unstale":"2023-11-08T16:30:30.746520+0000","last_undegraded":"2023-11-08T16:30:30.746520+0000","last_fullsized":"2023-11-08T16:30:30.746520+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:29:58.478672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.736278+0000","last_change":"2023-11-08T16:30:29.714369+0000","last_active":"2023-11-08T16:30:30.736278+0000","last_peered":"2023-11-08T16:30:30.736278+0000","last_clean":"2023-11-08T16:30:30.736278+0000","last_became_active":"2023-11-08T16:30:29.714093+0000","last_became_peered":"2023-11-08T16:30:29.714093+0000","last_unstale":"2023-11-08T16:30:30.736278+0000","last_undegraded":"2023-11-08T16:30:30.736278+0000","last_fullsized":"2023-11-08T16:30:30.736278+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:40:51.024256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.757207+0000","last_change":"2023-11-08T16:30:30.754902+0000","last_active":"2023-11-08T16:30:30.757207+0000","last_peered":"2023-11-08T16:30:30.757207+0000","last_clean":"2023-11-08T16:30:30.757207+0000","last_became_active":"2023-11-08T16:30:30.754671+0000","last_became_peered":"2023-11-08T16:30:30.754671+0000","last_unstale":"2023-11-08T16:30:30.757207+0000","last_undegraded":"2023-11-08T16:30:30.757207+0000","last_fullsized":"2023-11-08T16:30:30.757207+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:43.673461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751655+0000","last_change":"2023-11-08T16:30:29.714179+0000","last_active":"2023-11-08T16:30:30.751655+0000","last_peered":"2023-11-08T16:30:30.751655+0000","last_clean":"2023-11-08T16:30:30.751655+0000","last_became_active":"2023-11-08T16:30:29.713139+0000","last_became_peered":"2023-11-08T16:30:29.713139+0000","last_unstale":"2023-11-08T16:30:30.751655+0000","last_undegraded":"2023-11-08T16:30:30.751655+0000","last_fullsized":"2023-11-08T16:30:30.751655+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:36:40.585696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750524+0000","last_change":"2023-11-08T16:30:30.741379+0000","last_active":"2023-11-08T16:30:30.750524+0000","last_peered":"2023-11-08T16:30:30.750524+0000","last_clean":"2023-11-08T16:30:30.750524+0000","last_became_active":"2023-11-08T16:30:30.740659+0000","last_became_peered":"2023-11-08T16:30:30.740659+0000","last_unstale":"2023-11-08T16:30:30.750524+0000","last_undegraded":"2023-11-08T16:30:30.750524+0000","last_fullsized":"2023-11-08T16:30:30.750524+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:09:34.574098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.749677+0000","last_change":"2023-11-08T16:30:29.714540+0000","last_active":"2023-11-08T16:30:30.749677+0000","last_peered":"2023-11-08T16:30:30.749677+0000","last_clean":"2023-11-08T16:30:30.749677+0000","last_became_active":"2023-11-08T16:30:29.714162+0000","last_became_peered":"2023-11-08T16:30:29.714162+0000","last_unstale":"2023-11-08T16:30:30.749677+0000","last_undegraded":"2023-11-08T16:30:30.749677+0000","last_fullsized":"2023-11-08T16:30:30.749677+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:43:42.660881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745451+0000","last_change":"2023-11-08T16:30:30.741940+0000","last_active":"2023-11-08T16:30:30.745451+0000","last_peered":"2023-11-08T16:30:30.745451+0000","last_clean":"2023-11-08T16:30:30.745451+0000","last_became_active":"2023-11-08T16:30:30.741527+0000","last_became_peered":"2023-11-08T16:30:30.741527+0000","last_unstale":"2023-11-08T16:30:30.745451+0000","last_undegraded":"2023-11-08T16:30:30.745451+0000","last_fullsized":"2023-11-08T16:30:30.745451+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:41:10.086203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.752039+0000","last_change":"2023-11-08T16:30:29.719718+0000","last_active":"2023-11-08T16:30:30.752039+0000","last_peered":"2023-11-08T16:30:30.752039+0000","last_clean":"2023-11-08T16:30:30.752039+0000","last_became_active":"2023-11-08T16:30:29.719420+0000","last_became_peered":"2023-11-08T16:30:29.719420+0000","last_unstale":"2023-11-08T16:30:30.752039+0000","last_undegraded":"2023-11-08T16:30:30.752039+0000","last_fullsized":"2023-11-08T16:30:30.752039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:41.770901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.733806+0000","last_change":"2023-11-08T16:30:30.725830+0000","last_active":"2023-11-08T16:30:30.733806+0000","last_peered":"2023-11-08T16:30:30.733806+0000","last_clean":"2023-11-08T16:30:30.733806+0000","last_became_active":"2023-11-08T16:30:30.725538+0000","last_became_peered":"2023-11-08T16:30:30.725538+0000","last_unstale":"2023-11-08T16:30:30.733806+0000","last_undegraded":"2023-11-08T16:30:30.733806+0000","last_fullsized":"2023-11-08T16:30:30.733806+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:40:52.036835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.753845+0000","last_change":"2023-11-08T16:30:29.719564+0000","last_active":"2023-11-08T16:30:30.753845+0000","last_peered":"2023-11-08T16:30:30.753845+0000","last_clean":"2023-11-08T16:30:30.753845+0000","last_became_active":"2023-11-08T16:30:29.719271+0000","last_became_peered":"2023-11-08T16:30:29.719271+0000","last_unstale":"2023-11-08T16:30:30.753845+0000","last_undegraded":"2023-11-08T16:30:30.753845+0000","last_fullsized":"2023-11-08T16:30:30.753845+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:08.194612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746774+0000","last_change":"2023-11-08T16:30:30.741069+0000","last_active":"2023-11-08T16:30:30.746774+0000","last_peered":"2023-11-08T16:30:30.746774+0000","last_clean":"2023-11-08T16:30:30.746774+0000","last_became_active":"2023-11-08T16:30:30.740761+0000","last_became_peered":"2023-11-08T16:30:30.740761+0000","last_unstale":"2023-11-08T16:30:30.746774+0000","last_undegraded":"2023-11-08T16:30:30.746774+0000","last_fullsized":"2023-11-08T16:30:30.746774+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:52.347858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.727682+0000","last_change":"2023-11-08T16:30:29.707635+0000","last_active":"2023-11-08T16:30:30.727682+0000","last_peered":"2023-11-08T16:30:30.727682+0000","last_clean":"2023-11-08T16:30:30.727682+0000","last_became_active":"2023-11-08T16:30:29.707266+0000","last_became_peered":"2023-11-08T16:30:29.707266+0000","last_unstale":"2023-11-08T16:30:30.727682+0000","last_undegraded":"2023-11-08T16:30:30.727682+0000","last_fullsized":"2023-11-08T16:30:30.727682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:30:44.639668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.747823+0000","last_change":"2023-11-08T16:30:30.732414+0000","last_active":"2023-11-08T16:30:30.747823+0000","last_peered":"2023-11-08T16:30:30.747823+0000","last_clean":"2023-11-08T16:30:30.747823+0000","last_became_active":"2023-11-08T16:30:30.731922+0000","last_became_peered":"2023-11-08T16:30:30.731922+0000","last_unstale":"2023-11-08T16:30:30.747823+0000","last_undegraded":"2023-11-08T16:30:30.747823+0000","last_fullsized":"2023-11-08T16:30:30.747823+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:09.207192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.736172+0000","last_change":"2023-11-08T16:30:29.719181+0000","last_active":"2023-11-08T16:30:30.736172+0000","last_peered":"2023-11-08T16:30:30.736172+0000","last_clean":"2023-11-08T16:30:30.736172+0000","last_became_active":"2023-11-08T16:30:29.718815+0000","last_became_peered":"2023-11-08T16:30:29.718815+0000","last_unstale":"2023-11-08T16:30:30.736172+0000","last_undegraded":"2023-11-08T16:30:30.736172+0000","last_fullsized":"2023-11-08T16:30:30.736172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:29:57.466092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746899+0000","last_change":"2023-11-08T16:30:30.745451+0000","last_active":"2023-11-08T16:30:30.746899+0000","last_peered":"2023-11-08T16:30:30.746899+0000","last_clean":"2023-11-08T16:30:30.746899+0000","last_became_active":"2023-11-08T16:30:30.745038+0000","last_became_peered":"2023-11-08T16:30:30.745038+0000","last_unstale":"2023-11-08T16:30:30.746899+0000","last_undegraded":"2023-11-08T16:30:30.746899+0000","last_fullsized":"2023-11-08T16:30:30.746899+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:41.681026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.740306+0000","last_change":"2023-11-08T16:30:29.713680+0000","last_active":"2023-11-08T16:30:30.740306+0000","last_peered":"2023-11-08T16:30:30.740306+0000","last_clean":"2023-11-08T16:30:30.740306+0000","last_became_active":"2023-11-08T16:30:29.713405+0000","last_became_peered":"2023-11-08T16:30:29.713405+0000","last_unstale":"2023-11-08T16:30:30.740306+0000","last_undegraded":"2023-11-08T16:30:30.740306+0000","last_fullsized":"2023-11-08T16:30:30.740306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:24:13.302756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.749125+0000","last_change":"2023-11-08T16:30:30.735536+0000","last_active":"2023-11-08T16:30:30.749125+0000","last_peered":"2023-11-08T16:30:30.749125+0000","last_clean":"2023-11-08T16:30:30.749125+0000","last_became_active":"2023-11-08T16:30:30.735222+0000","last_became_peered":"2023-11-08T16:30:30.735222+0000","last_unstale":"2023-11-08T16:30:30.749125+0000","last_undegraded":"2023-11-08T16:30:30.749125+0000","last_fullsized":"2023-11-08T16:30:30.749125+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:16.017807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.722358+0000","last_change":"2023-11-08T16:30:29.717064+0000","last_active":"2023-11-08T16:30:30.722358+0000","last_peered":"2023-11-08T16:30:30.722358+0000","last_clean":"2023-11-08T16:30:30.722358+0000","last_became_active":"2023-11-08T16:30:29.716703+0000","last_became_peered":"2023-11-08T16:30:29.716703+0000","last_unstale":"2023-11-08T16:30:30.722358+0000","last_undegraded":"2023-11-08T16:30:30.722358+0000","last_fullsized":"2023-11-08T16:30:30.722358+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:35.335418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750362+0000","last_change":"2023-11-08T16:30:30.739848+0000","last_active":"2023-11-08T16:30:30.750362+0000","last_peered":"2023-11-08T16:30:30.750362+0000","last_clean":"2023-11-08T16:30:30.750362+0000","last_became_active":"2023-11-08T16:30:30.737365+0000","last_became_peered":"2023-11-08T16:30:30.737365+0000","last_unstale":"2023-11-08T16:30:30.750362+0000","last_undegraded":"2023-11-08T16:30:30.750362+0000","last_fullsized":"2023-11-08T16:30:30.750362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:20:58.235245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.739869+0000","last_change":"2023-11-08T16:30:29.712482+0000","last_active":"2023-11-08T16:30:30.739869+0000","last_peered":"2023-11-08T16:30:30.739869+0000","last_clean":"2023-11-08T16:30:30.739869+0000","last_became_active":"2023-11-08T16:30:29.712185+0000","last_became_peered":"2023-11-08T16:30:29.712185+0000","last_unstale":"2023-11-08T16:30:30.739869+0000","last_undegraded":"2023-11-08T16:30:30.739869+0000","last_fullsized":"2023-11-08T16:30:30.739869+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:26:35.052911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.739753+0000","last_change":"2023-11-08T16:30:30.732888+0000","last_active":"2023-11-08T16:30:30.739753+0000","last_peered":"2023-11-08T16:30:30.739753+0000","last_clean":"2023-11-08T16:30:30.739753+0000","last_became_active":"2023-11-08T16:30:30.732430+0000","last_became_peered":"2023-11-08T16:30:30.732430+0000","last_unstale":"2023-11-08T16:30:30.739753+0000","last_undegraded":"2023-11-08T16:30:30.739753+0000","last_fullsized":"2023-11-08T16:30:30.739753+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:24:51.640333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.761484+0000","last_change":"2023-11-08T16:30:29.712622+0000","last_active":"2023-11-08T16:30:30.761484+0000","last_peered":"2023-11-08T16:30:30.761484+0000","last_clean":"2023-11-08T16:30:30.761484+0000","last_became_active":"2023-11-08T16:30:29.712332+0000","last_became_peered":"2023-11-08T16:30:29.712332+0000","last_unstale":"2023-11-08T16:30:30.761484+0000","last_undegraded":"2023-11-08T16:30:30.761484+0000","last_fullsized":"2023-11-08T16:30:30.761484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:26:35.052911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.749130+0000","last_change":"2023-11-08T16:30:30.732641+0000","last_active":"2023-11-08T16:30:30.749130+0000","last_peered":"2023-11-08T16:30:30.749130+0000","last_clean":"2023-11-08T16:30:30.749130+0000","last_became_active":"2023-11-08T16:30:30.732349+0000","last_became_peered":"2023-11-08T16:30:30.732349+0000","last_unstale":"2023-11-08T16:30:30.749130+0000","last_undegraded":"2023-11-08T16:30:30.749130+0000","last_fullsized":"2023-11-08T16:30:30.749130+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:46.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745683+0000","last_change":"2023-11-08T16:30:29.717572+0000","last_active":"2023-11-08T16:30:30.745683+0000","last_peered":"2023-11-08T16:30:30.745683+0000","last_clean":"2023-11-08T16:30:30.745683+0000","last_became_active":"2023-11-08T16:30:29.716652+0000","last_became_peered":"2023-11-08T16:30:29.716652+0000","last_unstale":"2023-11-08T16:30:30.745683+0000","last_undegraded":"2023-11-08T16:30:30.745683+0000","last_fullsized":"2023-11-08T16:30:30.745683+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:48.285303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745848+0000","last_change":"2023-11-08T16:30:30.726248+0000","last_active":"2023-11-08T16:30:30.745848+0000","last_peered":"2023-11-08T16:30:30.745848+0000","last_clean":"2023-11-08T16:30:30.745848+0000","last_became_active":"2023-11-08T16:30:30.725970+0000","last_became_peered":"2023-11-08T16:30:30.725970+0000","last_unstale":"2023-11-08T16:30:30.745848+0000","last_undegraded":"2023-11-08T16:30:30.745848+0000","last_fullsized":"2023-11-08T16:30:30.745848+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:46.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.713057+0000","last_change":"2023-11-08T16:30:29.709073+0000","last_active":"2023-11-08T16:30:30.713057+0000","last_peered":"2023-11-08T16:30:30.713057+0000","last_clean":"2023-11-08T16:30:30.713057+0000","last_became_active":"2023-11-08T16:30:29.708592+0000","last_became_peered":"2023-11-08T16:30:29.708592+0000","last_unstale":"2023-11-08T16:30:30.713057+0000","last_undegraded":"2023-11-08T16:30:30.713057+0000","last_fullsized":"2023-11-08T16:30:30.713057+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:20:57.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745336+0000","last_change":"2023-11-08T16:30:30.740631+0000","last_active":"2023-11-08T16:30:30.745336+0000","last_peered":"2023-11-08T16:30:30.745336+0000","last_clean":"2023-11-08T16:30:30.745336+0000","last_became_active":"2023-11-08T16:30:30.740342+0000","last_became_peered":"2023-11-08T16:30:30.740342+0000","last_unstale":"2023-11-08T16:30:30.745336+0000","last_undegraded":"2023-11-08T16:30:30.745336+0000","last_fullsized":"2023-11-08T16:30:30.745336+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:36.347998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.727866+0000","last_change":"2023-11-08T16:30:29.713952+0000","last_active":"2023-11-08T16:30:30.727866+0000","last_peered":"2023-11-08T16:30:30.727866+0000","last_clean":"2023-11-08T16:30:30.727866+0000","last_became_active":"2023-11-08T16:30:29.713684+0000","last_became_peered":"2023-11-08T16:30:29.713684+0000","last_unstale":"2023-11-08T16:30:30.727866+0000","last_undegraded":"2023-11-08T16:30:30.727866+0000","last_fullsized":"2023-11-08T16:30:30.727866+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:24:50.627753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745002+0000","last_change":"2023-11-08T16:30:30.738280+0000","last_active":"2023-11-08T16:30:30.745002+0000","last_peered":"2023-11-08T16:30:30.745002+0000","last_clean":"2023-11-08T16:30:30.745002+0000","last_became_active":"2023-11-08T16:30:30.737897+0000","last_became_peered":"2023-11-08T16:30:30.737897+0000","last_unstale":"2023-11-08T16:30:30.745002+0000","last_undegraded":"2023-11-08T16:30:30.745002+0000","last_fullsized":"2023-11-08T16:30:30.745002+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:46.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.752276+0000","last_change":"2023-11-08T16:30:29.708852+0000","last_active":"2023-11-08T16:30:30.752276+0000","last_peered":"2023-11-08T16:30:30.752276+0000","last_clean":"2023-11-08T16:30:30.752276+0000","last_became_active":"2023-11-08T16:30:29.708573+0000","last_became_peered":"2023-11-08T16:30:29.708573+0000","last_unstale":"2023-11-08T16:30:30.752276+0000","last_undegraded":"2023-11-08T16:30:30.752276+0000","last_fullsized":"2023-11-08T16:30:30.752276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:16.667831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746932+0000","last_change":"2023-11-08T16:30:30.742885+0000","last_active":"2023-11-08T16:30:30.746932+0000","last_peered":"2023-11-08T16:30:30.746932+0000","last_clean":"2023-11-08T16:30:30.746932+0000","last_became_active":"2023-11-08T16:30:30.742017+0000","last_became_peered":"2023-11-08T16:30:30.742017+0000","last_unstale":"2023-11-08T16:30:30.746932+0000","last_undegraded":"2023-11-08T16:30:30.746932+0000","last_fullsized":"2023-11-08T16:30:30.746932+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:40:43.098228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.752264+0000","last_change":"2023-11-08T16:30:29.709425+0000","last_active":"2023-11-08T16:30:30.752264+0000","last_peered":"2023-11-08T16:30:30.752264+0000","last_clean":"2023-11-08T16:30:30.752264+0000","last_became_active":"2023-11-08T16:30:29.709148+0000","last_became_peered":"2023-11-08T16:30:29.709148+0000","last_unstale":"2023-11-08T16:30:30.752264+0000","last_undegraded":"2023-11-08T16:30:30.752264+0000","last_fullsized":"2023-11-08T16:30:30.752264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:40:51.024256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746175+0000","last_change":"2023-11-08T16:30:30.744346+0000","last_active":"2023-11-08T16:30:30.746175+0000","last_peered":"2023-11-08T16:30:30.746175+0000","last_clean":"2023-11-08T16:30:30.746175+0000","last_became_active":"2023-11-08T16:30:30.744025+0000","last_became_peered":"2023-11-08T16:30:30.744025+0000","last_unstale":"2023-11-08T16:30:30.746175+0000","last_undegraded":"2023-11-08T16:30:30.746175+0000","last_fullsized":"2023-11-08T16:30:30.746175+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:30:45.652248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.746878+0000","last_change":"2023-11-08T16:30:29.719132+0000","last_active":"2023-11-08T16:30:30.746878+0000","last_peered":"2023-11-08T16:30:30.746878+0000","last_clean":"2023-11-08T16:30:30.746878+0000","last_became_active":"2023-11-08T16:30:29.718811+0000","last_became_peered":"2023-11-08T16:30:29.718811+0000","last_unstale":"2023-11-08T16:30:30.746878+0000","last_undegraded":"2023-11-08T16:30:30.746878+0000","last_fullsized":"2023-11-08T16:30:30.746878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:42.245668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750568+0000","last_change":"2023-11-08T16:30:30.741287+0000","last_active":"2023-11-08T16:30:30.750568+0000","last_peered":"2023-11-08T16:30:30.750568+0000","last_clean":"2023-11-08T16:30:30.750568+0000","last_became_active":"2023-11-08T16:30:30.740650+0000","last_became_peered":"2023-11-08T16:30:30.740650+0000","last_unstale":"2023-11-08T16:30:30.750568+0000","last_undegraded":"2023-11-08T16:30:30.750568+0000","last_fullsized":"2023-11-08T16:30:30.750568+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:42.783481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.749112+0000","last_change":"2023-11-08T16:30:29.718853+0000","last_active":"2023-11-08T16:30:30.749112+0000","last_peered":"2023-11-08T16:30:30.749112+0000","last_clean":"2023-11-08T16:30:30.749112+0000","last_became_active":"2023-11-08T16:30:29.718494+0000","last_became_peered":"2023-11-08T16:30:29.718494+0000","last_unstale":"2023-11-08T16:30:30.749112+0000","last_undegraded":"2023-11-08T16:30:30.749112+0000","last_fullsized":"2023-11-08T16:30:30.749112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:45.269178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.734316+0000","last_change":"2023-11-08T16:30:30.732075+0000","last_active":"2023-11-08T16:30:30.734316+0000","last_peered":"2023-11-08T16:30:30.734316+0000","last_clean":"2023-11-08T16:30:30.734316+0000","last_became_active":"2023-11-08T16:30:30.731663+0000","last_became_peered":"2023-11-08T16:30:30.731663+0000","last_unstale":"2023-11-08T16:30:30.734316+0000","last_undegraded":"2023-11-08T16:30:30.734316+0000","last_fullsized":"2023-11-08T16:30:30.734316+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:22:53.461699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.739676+0000","last_change":"2023-11-08T16:30:29.719508+0000","last_active":"2023-11-08T16:30:30.739676+0000","last_peered":"2023-11-08T16:30:30.739676+0000","last_clean":"2023-11-08T16:30:30.739676+0000","last_became_active":"2023-11-08T16:30:29.719157+0000","last_became_peered":"2023-11-08T16:30:29.719157+0000","last_unstale":"2023-11-08T16:30:30.739676+0000","last_undegraded":"2023-11-08T16:30:30.739676+0000","last_fullsized":"2023-11-08T16:30:30.739676+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:20:57.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.748422+0000","last_change":"2023-11-08T16:30:30.741880+0000","last_active":"2023-11-08T16:30:30.748422+0000","last_peered":"2023-11-08T16:30:30.748422+0000","last_clean":"2023-11-08T16:30:30.748422+0000","last_became_active":"2023-11-08T16:30:30.741610+0000","last_became_peered":"2023-11-08T16:30:30.741610+0000","last_unstale":"2023-11-08T16:30:30.748422+0000","last_undegraded":"2023-11-08T16:30:30.748422+0000","last_fullsized":"2023-11-08T16:30:30.748422+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:20:05.238900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.738974+0000","last_change":"2023-11-08T16:30:29.713858+0000","last_active":"2023-11-08T16:30:30.738974+0000","last_peered":"2023-11-08T16:30:30.738974+0000","last_clean":"2023-11-08T16:30:30.738974+0000","last_became_active":"2023-11-08T16:30:29.713552+0000","last_became_peered":"2023-11-08T16:30:29.713552+0000","last_unstale":"2023-11-08T16:30:30.738974+0000","last_undegraded":"2023-11-08T16:30:30.738974+0000","last_fullsized":"2023-11-08T16:30:30.738974+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:20:57.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.733329+0000","last_change":"2023-11-08T16:30:30.729904+0000","last_active":"2023-11-08T16:30:30.733329+0000","last_peered":"2023-11-08T16:30:30.733329+0000","last_clean":"2023-11-08T16:30:30.733329+0000","last_became_active":"2023-11-08T16:30:30.729628+0000","last_became_peered":"2023-11-08T16:30:30.729628+0000","last_unstale":"2023-11-08T16:30:30.733329+0000","last_undegraded":"2023-11-08T16:30:30.733329+0000","last_fullsized":"2023-11-08T16:30:30.733329+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:13:59.773111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.709658+0000","last_change":"2023-11-08T16:30:29.718001+0000","last_active":"2023-11-08T16:30:30.709658+0000","last_peered":"2023-11-08T16:30:30.709658+0000","last_clean":"2023-11-08T16:30:30.709658+0000","last_became_active":"2023-11-08T16:30:29.717708+0000","last_became_peered":"2023-11-08T16:30:29.717708+0000","last_unstale":"2023-11-08T16:30:30.709658+0000","last_undegraded":"2023-11-08T16:30:30.709658+0000","last_fullsized":"2023-11-08T16:30:30.709658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:20:57.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.728035+0000","last_change":"2023-11-08T16:30:30.726011+0000","last_active":"2023-11-08T16:30:30.728035+0000","last_peered":"2023-11-08T16:30:30.728035+0000","last_clean":"2023-11-08T16:30:30.728035+0000","last_became_active":"2023-11-08T16:30:30.724356+0000","last_became_peered":"2023-11-08T16:30:30.724356+0000","last_unstale":"2023-11-08T16:30:30.728035+0000","last_undegraded":"2023-11-08T16:30:30.728035+0000","last_fullsized":"2023-11-08T16:30:30.728035+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:12:03.632756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.735745+0000","last_change":"2023-11-08T16:30:29.717115+0000","last_active":"2023-11-08T16:30:30.735745+0000","last_peered":"2023-11-08T16:30:30.735745+0000","last_clean":"2023-11-08T16:30:30.735745+0000","last_became_active":"2023-11-08T16:30:29.716797+0000","last_became_peered":"2023-11-08T16:30:29.716797+0000","last_unstale":"2023-11-08T16:30:30.735745+0000","last_undegraded":"2023-11-08T16:30:30.735745+0000","last_fullsized":"2023-11-08T16:30:30.735745+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:09:33.561519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.737651+0000","last_change":"2023-11-08T16:30:30.734244+0000","last_active":"2023-11-08T16:30:30.737651+0000","last_peered":"2023-11-08T16:30:30.737651+0000","last_clean":"2023-11-08T16:30:30.737651+0000","last_became_active":"2023-11-08T16:30:30.733917+0000","last_became_peered":"2023-11-08T16:30:30.733917+0000","last_unstale":"2023-11-08T16:30:30.737651+0000","last_undegraded":"2023-11-08T16:30:30.737651+0000","last_fullsized":"2023-11-08T16:30:30.737651+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:41.681026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.712351+0000","last_change":"2023-11-08T16:30:29.712429+0000","last_active":"2023-11-08T16:30:30.712351+0000","last_peered":"2023-11-08T16:30:30.712351+0000","last_clean":"2023-11-08T16:30:30.712351+0000","last_became_active":"2023-11-08T16:30:29.712021+0000","last_became_peered":"2023-11-08T16:30:29.712021+0000","last_unstale":"2023-11-08T16:30:30.712351+0000","last_undegraded":"2023-11-08T16:30:30.712351+0000","last_fullsized":"2023-11-08T16:30:30.712351+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:28.257258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.734864+0000","last_change":"2023-11-08T16:30:30.733372+0000","last_active":"2023-11-08T16:30:30.734864+0000","last_peered":"2023-11-08T16:30:30.734864+0000","last_clean":"2023-11-08T16:30:30.734864+0000","last_became_active":"2023-11-08T16:30:30.733085+0000","last_became_peered":"2023-11-08T16:30:30.733085+0000","last_unstale":"2023-11-08T16:30:30.734864+0000","last_undegraded":"2023-11-08T16:30:30.734864+0000","last_fullsized":"2023-11-08T16:30:30.734864+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:29:52.347858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.735699+0000","last_change":"2023-11-08T16:30:29.705156+0000","last_active":"2023-11-08T16:30:30.735699+0000","last_peered":"2023-11-08T16:30:30.735699+0000","last_clean":"2023-11-08T16:30:30.735699+0000","last_became_active":"2023-11-08T16:30:29.704878+0000","last_became_peered":"2023-11-08T16:30:29.704878+0000","last_unstale":"2023-11-08T16:30:30.735699+0000","last_undegraded":"2023-11-08T16:30:30.735699+0000","last_fullsized":"2023-11-08T16:30:30.735699+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:35.335418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.739511+0000","last_change":"2023-11-08T16:30:30.732855+0000","last_active":"2023-11-08T16:30:30.739511+0000","last_peered":"2023-11-08T16:30:30.739511+0000","last_clean":"2023-11-08T16:30:30.739511+0000","last_became_active":"2023-11-08T16:30:30.732495+0000","last_became_peered":"2023-11-08T16:30:30.732495+0000","last_unstale":"2023-11-08T16:30:30.739511+0000","last_undegraded":"2023-11-08T16:30:30.739511+0000","last_fullsized":"2023-11-08T16:30:30.739511+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:41:10.086203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.722462+0000","last_change":"2023-11-08T16:30:29.714099+0000","last_active":"2023-11-08T16:30:30.722462+0000","last_peered":"2023-11-08T16:30:30.722462+0000","last_clean":"2023-11-08T16:30:30.722462+0000","last_became_active":"2023-11-08T16:30:29.713778+0000","last_became_peered":"2023-11-08T16:30:29.713778+0000","last_unstale":"2023-11-08T16:30:30.722462+0000","last_undegraded":"2023-11-08T16:30:30.722462+0000","last_fullsized":"2023-11-08T16:30:30.722462+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:15:40.668446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.743830+0000","last_change":"2023-11-08T16:30:30.735989+0000","last_active":"2023-11-08T16:30:30.743830+0000","last_peered":"2023-11-08T16:30:30.743830+0000","last_clean":"2023-11-08T16:30:30.743830+0000","last_became_active":"2023-11-08T16:30:30.735690+0000","last_became_peered":"2023-11-08T16:30:30.735690+0000","last_unstale":"2023-11-08T16:30:30.743830+0000","last_undegraded":"2023-11-08T16:30:30.743830+0000","last_fullsized":"2023-11-08T16:30:30.743830+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:46.281758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.713127+0000","last_change":"2023-11-08T16:30:29.706362+0000","last_active":"2023-11-08T16:30:30.713127+0000","last_peered":"2023-11-08T16:30:30.713127+0000","last_clean":"2023-11-08T16:30:30.713127+0000","last_became_active":"2023-11-08T16:30:29.706081+0000","last_became_peered":"2023-11-08T16:30:29.706081+0000","last_unstale":"2023-11-08T16:30:30.713127+0000","last_undegraded":"2023-11-08T16:30:30.713127+0000","last_fullsized":"2023-11-08T16:30:30.713127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:50:35.904021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.733520+0000","last_change":"2023-11-08T16:30:30.728611+0000","last_active":"2023-11-08T16:30:30.733520+0000","last_peered":"2023-11-08T16:30:30.733520+0000","last_clean":"2023-11-08T16:30:30.733520+0000","last_became_active":"2023-11-08T16:30:30.728332+0000","last_became_peered":"2023-11-08T16:30:30.728332+0000","last_unstale":"2023-11-08T16:30:30.733520+0000","last_undegraded":"2023-11-08T16:30:30.733520+0000","last_fullsized":"2023-11-08T16:30:30.733520+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:47:42.783481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.721817+0000","last_change":"2023-11-08T16:30:29.709784+0000","last_active":"2023-11-08T16:30:30.721817+0000","last_peered":"2023-11-08T16:30:30.721817+0000","last_clean":"2023-11-08T16:30:30.721817+0000","last_became_active":"2023-11-08T16:30:29.709427+0000","last_became_peered":"2023-11-08T16:30:29.709427+0000","last_unstale":"2023-11-08T16:30:30.721817+0000","last_undegraded":"2023-11-08T16:30:30.721817+0000","last_fullsized":"2023-11-08T16:30:30.721817+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:50:35.904021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.745659+0000","last_change":"2023-11-08T16:30:30.738578+0000","last_active":"2023-11-08T16:30:30.745659+0000","last_peered":"2023-11-08T16:30:30.745659+0000","last_clean":"2023-11-08T16:30:30.745659+0000","last_became_active":"2023-11-08T16:30:30.738305+0000","last_became_peered":"2023-11-08T16:30:30.738305+0000","last_unstale":"2023-11-08T16:30:30.745659+0000","last_undegraded":"2023-11-08T16:30:30.745659+0000","last_fullsized":"2023-11-08T16:30:30.745659+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:43.258248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.713123+0000","last_change":"2023-11-08T16:30:29.708860+0000","last_active":"2023-11-08T16:30:30.713123+0000","last_peered":"2023-11-08T16:30:30.713123+0000","last_clean":"2023-11-08T16:30:30.713123+0000","last_became_active":"2023-11-08T16:30:29.707687+0000","last_became_peered":"2023-11-08T16:30:29.707687+0000","last_unstale":"2023-11-08T16:30:30.713123+0000","last_undegraded":"2023-11-08T16:30:30.713123+0000","last_fullsized":"2023-11-08T16:30:30.713123+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:27:15.005227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.750200+0000","last_change":"2023-11-08T16:30:30.739635+0000","last_active":"2023-11-08T16:30:30.750200+0000","last_peered":"2023-11-08T16:30:30.750200+0000","last_clean":"2023-11-08T16:30:30.750200+0000","last_became_active":"2023-11-08T16:30:30.737160+0000","last_became_peered":"2023-11-08T16:30:30.737160+0000","last_unstale":"2023-11-08T16:30:30.750200+0000","last_undegraded":"2023-11-08T16:30:30.750200+0000","last_fullsized":"2023-11-08T16:30:30.750200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:50:36.916601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.735337+0000","last_change":"2023-11-08T16:30:29.717678+0000","last_active":"2023-11-08T16:30:30.735337+0000","last_peered":"2023-11-08T16:30:30.735337+0000","last_clean":"2023-11-08T16:30:30.735337+0000","last_became_active":"2023-11-08T16:30:29.717280+0000","last_became_peered":"2023-11-08T16:30:29.717280+0000","last_unstale":"2023-11-08T16:30:30.735337+0000","last_undegraded":"2023-11-08T16:30:30.735337+0000","last_fullsized":"2023-11-08T16:30:30.735337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:20:06.494468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.751934+0000","last_change":"2023-11-08T16:30:30.746551+0000","last_active":"2023-11-08T16:30:30.751934+0000","last_peered":"2023-11-08T16:30:30.751934+0000","last_clean":"2023-11-08T16:30:30.751934+0000","last_became_active":"2023-11-08T16:30:30.746253+0000","last_became_peered":"2023-11-08T16:30:30.746253+0000","last_unstale":"2023-11-08T16:30:30.751934+0000","last_undegraded":"2023-11-08T16:30:30.751934+0000","last_fullsized":"2023-11-08T16:30:30.751934+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:49:41.503135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.727216+0000","last_change":"2023-11-08T16:30:29.711664+0000","last_active":"2023-11-08T16:30:30.727216+0000","last_peered":"2023-11-08T16:30:30.727216+0000","last_clean":"2023-11-08T16:30:30.727216+0000","last_became_active":"2023-11-08T16:30:29.710855+0000","last_became_peered":"2023-11-08T16:30:29.710855+0000","last_unstale":"2023-11-08T16:30:30.727216+0000","last_undegraded":"2023-11-08T16:30:30.727216+0000","last_fullsized":"2023-11-08T16:30:30.727216+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:43:04.952635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.737334+0000","last_change":"2023-11-08T16:30:30.733579+0000","last_active":"2023-11-08T16:30:30.737334+0000","last_peered":"2023-11-08T16:30:30.737334+0000","last_clean":"2023-11-08T16:30:30.737334+0000","last_became_active":"2023-11-08T16:30:30.730239+0000","last_became_peered":"2023-11-08T16:30:30.730239+0000","last_unstale":"2023-11-08T16:30:30.737334+0000","last_undegraded":"2023-11-08T16:30:30.737334+0000","last_fullsized":"2023-11-08T16:30:30.737334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:29.672835+0000","last_clean_scrub_stamp":"2023-11-08T16:30:29.672835+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:40:52.036835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-11-08T16:30:30.707768+0000","last_change":"2023-11-08T16:30:29.720980+0000","last_active":"2023-11-08T16:30:30.707768+0000","last_peered":"2023-11-08T16:30:30.707768+0000","last_clean":"2023-11-08T16:30:30.707768+0000","last_became_active":"2023-11-08T16:30:29.720697+0000","last_became_peered":"2023-11-08T16:30:29.720697+0000","last_unstale":"2023-11-08T16:30:30.707768+0000","last_undegraded":"2023-11-08T16:30:30.707768+0000","last_fullsized":"2023-11-08T16:30:30.707768+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:30:28.660255+0000","last_clean_scrub_stamp":"2023-11-08T16:30:28.660255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:08.194612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":178,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-08T16:30:29.706461+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:30:29.706461+0000","last_peered":"2023-11-08T16:30:29.706461+0000","last_clean":"2023-11-08T16:30:29.706461+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:30:29.706461+0000","last_undegraded":"2023-11-08T16:30:29.706461+0000","last_fullsized":"2023-11-08T16:30:29.706461+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343860,"statfs":{"total":96636764160,"available":96608112640,"internally_reserved":0,"allocated":983040,"data_stored":740898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":4.2729999999999997,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":0.81499999999999995}]},{"osd":1,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.125,"15min":1.125},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":1.131},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2090000000000001,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":0.70099999999999996}]},{"osd":2,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":4.2249999999999996,"5min":4.2249999999999996,"15min":4.2249999999999996},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.992,"5min":1.992,"15min":1.992},"last":1.278}]},{"osd":3,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.544,"5min":1.544,"15min":1.544},"min":{"1min":0.88,"5min":0.88,"15min":0.88},"max":{"1min":4.181,"5min":4.181,"15min":4.181},"last":1.405}]},{"osd":4,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.629,"5min":1.629,"15min":1.629},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":2.3730000000000002,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.635}]},{"osd":5,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":2.589,"5min":2.589,"15min":2.589},"last":1.4970000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.534,"5min":1.534,"15min":1.534},"min":{"1min":1.026,"5min":1.026,"15min":1.026},"max":{"1min":2.496,"5min":2.496,"15min":2.496},"last":1.7},{"interface":"front","average":{"1min":1.478,"5min":1.478,"15min":1.478},"min":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.5649999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":1.083,"5min":1.083,"15min":1.083},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.77600000000000002}]},{"osd":1,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":1.472}]},{"osd":2,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.71699999999999997}]},{"osd":3,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.8220000000000001,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":1.95,"5min":1.95,"15min":1.95},"last":1.3600000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.395,"15min":1.395},"min":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"max":{"1min":2.0790000000000002,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.46,"5min":1.46,"15min":1.46},"min":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.619}]},{"osd":5,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.5820000000000001,"15min":1.5820000000000001},"min":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":1.577}]},{"osd":7,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"min":{"1min":1.119,"5min":1.119,"15min":1.119},"max":{"1min":3.8959999999999999,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":1.524},{"interface":"front","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":1.173,"5min":1.173,"15min":1.173},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.2470000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":151112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":2.4609999999999999,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.367},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.3639999999999999,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.46}]},{"osd":1,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":3.8690000000000002,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":3.2349999999999999,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.4330000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.28,"15min":1.28},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.534},{"interface":"front","average":{"1min":1.24,"5min":1.24,"15min":1.24},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.3879999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.355,"15min":1.355},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":2.8330000000000002,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.3360000000000001,"15min":1.3360000000000001},"min":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.262}]},{"osd":4,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.4139999999999999,"15min":1.4139999999999999},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":3.9790000000000001,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":1.04},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":3.0209999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":0.94699999999999995}]},{"osd":6,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"min":{"1min":0.752,"5min":0.752,"15min":0.752},"max":{"1min":2.5550000000000002,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"min":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"max":{"1min":4.1189999999999998,"5min":4.1189999999999998,"15min":4.1189999999999998},"last":1.2070000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:29:54 2023","interfaces":[{"interface":"back","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":1.054,"5min":1.054,"15min":1.054},"max":{"1min":3.1160000000000001,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"min":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":2.7050000000000001,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.5860000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":150544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.95699999999999996}]},{"osd":1,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.153,"5min":1.153,"15min":1.153},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.90000000000000002}]},{"osd":2,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.173},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"max":{"1min":1.7829999999999999,"5min":1.7829999999999999,"15min":1.7829999999999999},"last":1.008}]},{"osd":3,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.653,"5min":1.653,"15min":1.653},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"max":{"1min":1.728,"5min":1.728,"15min":1.728},"last":1.4259999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.9870000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.123},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.0629999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.5609999999999999,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.5600000000000001},{"interface":"front","average":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"min":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.3320000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.51,"15min":1.51},"min":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":1.9410000000000001},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"min":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.631}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":152074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.585,"5min":1.585,"15min":1.585},"last":0.35799999999999998}]},{"osd":1,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.829,"5min":1.829,"15min":1.829},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.175,"5min":1.175,"15min":1.175},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.4660000000000002,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":0.57399999999999995}]},{"osd":4,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.62},{"interface":"front","average":{"1min":1.27,"5min":1.27,"15min":1.27},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.1829999999999998,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.0469999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.371,"5min":1.371,"15min":1.371},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.593,"5min":2.593,"15min":2.593},"last":0.77200000000000002}]},{"osd":6,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.454,"15min":1.454},"min":{"1min":0.997,"5min":0.997,"15min":0.997},"max":{"1min":2.9569999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":1.018},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":2.762,"5min":2.762,"15min":2.762},"last":0.748}]},{"osd":7,"last update":"Wed Nov 8 16:29:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":2.8700000000000001,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"max":{"1min":2.633,"5min":2.633,"15min":2.633},"last":1.1120000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":150510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":4.5369999999999999,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":4.4980000000000002,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.089,"5min":1.089,"15min":1.089},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6879999999999999,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0209999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":3.3050000000000002,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.383,"5min":1.383,"15min":1.383},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":4.5899999999999999,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":1.0669999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.54,"15min":1.54},"min":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"max":{"1min":4.9409999999999998,"5min":4.9409999999999998,"15min":4.9409999999999998},"last":1.119},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"min":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.4119999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":2.8530000000000002,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.298},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.88400000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":3.536,"5min":3.536,"15min":3.536},"last":1.286},{"interface":"front","average":{"1min":1.627,"5min":1.627,"15min":1.627},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":4.7549999999999999,"5min":4.7549999999999999,"15min":4.7549999999999999},"last":1.341}]},{"osd":7,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.546,"15min":1.546},"min":{"1min":1.093,"5min":1.093,"15min":1.093},"max":{"1min":2.9700000000000002,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.635,"5min":1.635,"15min":1.635},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":3.4950000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.236}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":152074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":4.0309999999999997,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":1.236}]},{"osd":2,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.169,"15min":1.169},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":3.0339999999999998,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.179,"15min":1.179},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.6840000000000002,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":1.139}]},{"osd":3,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":4.3179999999999996,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":3.9390000000000001,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":0.90800000000000003}]},{"osd":4,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.4199999999999999,"15min":1.4199999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":3.1659999999999999,"5min":3.1659999999999999,"15min":3.1659999999999999},"last":1.099},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":2.734,"5min":2.734,"15min":2.734},"last":1.345}]},{"osd":5,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.913,"5min":1.913,"15min":1.913},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"max":{"1min":8.4730000000000008,"5min":8.4730000000000008,"15min":8.4730000000000008},"last":1.5880000000000001},{"interface":"front","average":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":8.8040000000000003,"5min":8.8040000000000003,"15min":8.8040000000000003},"last":1.1839999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.5669999999999999,"15min":1.5669999999999999},"min":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"max":{"1min":3.984,"5min":3.984,"15min":3.984},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.538,"5min":1.538,"15min":1.538},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":3.0899999999999999,"5min":3.0899999999999999,"15min":3.0899999999999999},"last":1.319}]},{"osd":7,"last update":"Wed Nov 8 16:29:51 2023","interfaces":[{"interface":"back","average":{"1min":1.712,"5min":1.712,"15min":1.712},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":4.3609999999999998,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":1.657},{"interface":"front","average":{"1min":1.7749999999999999,"5min":1.7749999999999999,"15min":1.7749999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.3840000000000003,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":1.6850000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":978944,"data_stored":740808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.66800000000000004}]},{"osd":2,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.762,"5min":1.762,"15min":1.762},"last":0.63600000000000001},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.879,"5min":1.879,"15min":1.879},"last":0.93500000000000005}]},{"osd":3,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.204,"5min":1.204,"15min":1.204},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.7679999999999998,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":1.109}]},{"osd":4,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.431},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.867,"5min":2.867,"15min":2.867},"last":1.28}]},{"osd":5,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"min":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":2.7280000000000002,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.466},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.252}]},{"osd":6,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.6739999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.3899999999999999},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"max":{"1min":2.8260000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.2}]},{"osd":7,"last update":"Wed Nov 8 16:29:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4219999999999999,"5min":1.4219999999999999,"15min":1.4219999999999999},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":2.625,"5min":2.625,"15min":2.625},"last":1.345},{"interface":"front","average":{"1min":1.385,"5min":1.385,"15min":1.385},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.431,"5min":2.431,"15min":2.431},"last":1.3180000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:30:35.732 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T16:30:35.732 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T16:30:35.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.982+0000 7f5c9ffff640 1 Processor -- start 2023-11-08T16:30:35.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.982+0000 7f5c9ffff640 1 -- start start 2023-11-08T16:30:35.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.984+0000 7f5c9ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5ca00c97b0 0x7f5ca006fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:35.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.984+0000 7f5c9ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ca0070360 0x7f5ca006bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:35.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.984+0000 7f5c9ffff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ca0070790 con 0x7f5ca015e1a0 2023-11-08T16:30:35.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.984+0000 7f5c9ffff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ca00ca830 con 0x7f5ca00c97b0 2023-11-08T16:30:35.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.984+0000 7f5c9ffff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca015e580 con 0x7f5ca0070360 2023-11-08T16:30:35.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.984+0000 7f5c9effd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5ca015e1a0 0x7f5ca00caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53688/0 (socket says 172.21.15.81:53688) 2023-11-08T16:30:35.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.984+0000 7f5c9effd640 1 -- 172.21.15.81:0/2512501587 learned_addr learned my addr 172.21.15.81:0/2512501587 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:35.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.984+0000 7f5c9e7fc640 1 --2- 172.21.15.81:0/2512501587 >> v2:172.21.15.81:3300/0 conn(0x7f5ca00c97b0 0x7f5ca006fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:35.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.984+0000 7f5c9f7fe640 1 --2- 172.21.15.81:0/2512501587 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ca0070360 0x7f5ca006bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:35.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.984+0000 7f5c9dffb640 1 -- 172.21.15.81:0/2512501587 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626544503 0 0) 0x7f5ca0070790 con 0x7f5ca015e1a0 2023-11-08T16:30:35.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.985+0000 7f5c9dffb640 1 -- 172.21.15.81:0/2512501587 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c88003520 con 0x7f5ca015e1a0 2023-11-08T16:30:35.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.985+0000 7f5c9e7fc640 1 -- 172.21.15.81:0/2512501587 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ca0070360 msgr2=0x7f5ca006bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:35.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.985+0000 7f5c9e7fc640 1 --2- 172.21.15.81:0/2512501587 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ca0070360 0x7f5ca006bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.985+0000 7f5c9e7fc640 1 -- 172.21.15.81:0/2512501587 >> v1:172.21.15.81:6789/0 conn(0x7f5ca015e1a0 legacy=0x7f5ca00caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:35.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.985+0000 7f5c9e7fc640 1 -- 172.21.15.81:0/2512501587 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ca00c9cb0 con 0x7f5ca00c97b0 2023-11-08T16:30:35.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.986+0000 7f5c9e7fc640 1 --2- 172.21.15.81:0/2512501587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ca00c97b0 0x7f5ca006fe20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009790 tx=0x7f5c9402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad466cb21ffd4061 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:35.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.986+0000 7f5c9dffb640 1 -- 172.21.15.81:0/2512501587 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94031780 con 0x7f5ca00c97b0 2023-11-08T16:30:35.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.986+0000 7f5c9dffb640 1 -- 172.21.15.81:0/2512501587 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c940318e0 con 0x7f5ca00c97b0 2023-11-08T16:30:35.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.986+0000 7f5c9dffb640 1 -- 172.21.15.81:0/2512501587 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c94031c70 con 0x7f5ca00c97b0 2023-11-08T16:30:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.987+0000 7f5c9ffff640 1 -- 172.21.15.81:0/2512501587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ca00c97b0 msgr2=0x7f5ca006fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.987+0000 7f5c9ffff640 1 --2- 172.21.15.81:0/2512501587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ca00c97b0 0x7f5ca006fe20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5c94009790 tx=0x7f5c9402b820 comp rx=0 tx=0).stop 2023-11-08T16:30:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.987+0000 7f5c9ffff640 1 -- 172.21.15.81:0/2512501587 shutdown_connections 2023-11-08T16:30:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.987+0000 7f5c9ffff640 1 --2- 172.21.15.81:0/2512501587 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ca0070360 0x7f5ca006bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.987+0000 7f5c9ffff640 1 --2- 172.21.15.81:0/2512501587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ca00c97b0 0x7f5ca006fe20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.987+0000 7f5c9ffff640 1 -- 172.21.15.81:0/2512501587 >> 172.21.15.81:0/2512501587 conn(0x7f5ca00ce8d0 msgr2=0x7f5ca00cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:35.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.987+0000 7f5c9ffff640 1 -- 172.21.15.81:0/2512501587 shutdown_connections 2023-11-08T16:30:35.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.987+0000 7f5c9ffff640 1 -- 172.21.15.81:0/2512501587 wait complete. 2023-11-08T16:30:35.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.989+0000 7f5c9ffff640 1 Processor -- start 2023-11-08T16:30:35.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.989+0000 7f5c9ffff640 1 -- start start 2023-11-08T16:30:35.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.990+0000 7f5c9ffff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ca0070360 0x7f5ca00c7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:35.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.990+0000 7f5c9ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ca00c97b0 0x7f5ca014b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.990+0000 7f5c9ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ca015e1a0 0x7f5ca00c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.990+0000 7f5c9ffff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00c6f50 con 0x7f5ca015e1a0 2023-11-08T16:30:36.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.990+0000 7f5c9ffff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00c62d0 con 0x7f5ca0070360 2023-11-08T16:30:36.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.990+0000 7f5c9ffff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ca00c65b0 con 0x7f5ca00c97b0 2023-11-08T16:30:36.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.990+0000 7f5c9f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ca015e1a0 0x7f5ca00c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:36.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.990+0000 7f5c9f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ca015e1a0 0x7f5ca00c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40828/0 (socket says 172.21.15.81:40828) 2023-11-08T16:30:36.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.990+0000 7f5c9f7fe640 1 -- 172.21.15.81:0/3542181224 learned_addr learned my addr 172.21.15.81:0/3542181224 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:36.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.990+0000 7f5c9e7fc640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ca00c97b0 0x7f5ca014b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:36.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c9f7fe640 1 -- 172.21.15.81:0/3542181224 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ca00c97b0 msgr2=0x7f5ca014b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:36.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c9effd640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ca0070360 0x7f5ca00c7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:36.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c9f7fe640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ca00c97b0 0x7f5ca014b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c9f7fe640 1 -- 172.21.15.81:0/3542181224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ca0070360 msgr2=0x7f5ca00c7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:36.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c9f7fe640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ca0070360 0x7f5ca00c7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c9f7fe640 1 -- 172.21.15.81:0/3542181224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c940093a0 con 0x7f5ca015e1a0 2023-11-08T16:30:36.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c9e7fc640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ca00c97b0 0x7f5ca014b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:36.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c9effd640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ca0070360 0x7f5ca00c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:36.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c9f7fe640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ca015e1a0 0x7f5ca00c5910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5c98007e60 tx=0x7f5c9800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:36.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c877fe640 1 -- 172.21.15.81:0/3542181224 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98014420 con 0x7f5ca015e1a0 2023-11-08T16:30:36.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c877fe640 1 -- 172.21.15.81:0/3542181224 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c980145a0 con 0x7f5ca015e1a0 2023-11-08T16:30:36.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c877fe640 1 -- 172.21.15.81:0/3542181224 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98014950 con 0x7f5ca015e1a0 2023-11-08T16:30:36.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c9ffff640 1 -- 172.21.15.81:0/3542181224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ca00c6890 con 0x7f5ca015e1a0 2023-11-08T16:30:36.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.991+0000 7f5c9ffff640 1 -- 172.21.15.81:0/3542181224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ca014b970 con 0x7f5ca015e1a0 2023-11-08T16:30:36.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.993+0000 7f5c9ffff640 1 -- 172.21.15.81:0/3542181224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ca00c7090 con 0x7f5ca015e1a0 2023-11-08T16:30:36.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.993+0000 7f5c877fe640 1 -- 172.21.15.81:0/3542181224 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c98014ab0 con 0x7f5ca015e1a0 2023-11-08T16:30:36.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.995+0000 7f5c877fe640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c8c06ae70 0x7f5c8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.995+0000 7f5c877fe640 1 -- 172.21.15.81:0/3542181224 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c98089ed0 con 0x7f5ca015e1a0 2023-11-08T16:30:36.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.995+0000 7f5c9effd640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c8c06ae70 0x7f5c8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:36.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.996+0000 7f5c9effd640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c8c06ae70 0x7f5c8c06d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c900011e0 tx=0x7f5c90009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:36.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:35.999+0000 7f5c877fe640 1 -- 172.21.15.81:0/3542181224 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ca00c7090 con 0x7f5ca015e1a0 2023-11-08T16:30:36.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.256+0000 7f5c9ffff640 1 -- 172.21.15.81:0/3542181224 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5ca00c7090 con 0x7f5ca015e1a0 2023-11-08T16:30:36.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.261+0000 7f5c877fe640 1 -- 172.21.15.81:0/3542181224 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+147 (secure 0 0 0) 0x7f5ca00c7090 con 0x7f5ca015e1a0 2023-11-08T16:30:36.263 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2023-11-08T16:30:36.263 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ== 2023-11-08T16:30:36.263 INFO:teuthology.orchestra.run.smithi081.stdout: caps mds = "allow" 2023-11-08T16:30:36.264 INFO:teuthology.orchestra.run.smithi081.stdout: caps mon = "allow r" 2023-11-08T16:30:36.264 INFO:teuthology.orchestra.run.smithi081.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T16:30:36.264 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2023-11-08T16:30:36.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.263+0000 7f5c9ffff640 1 -- 172.21.15.81:0/3542181224 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c8c06ae70 msgr2=0x7f5c8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:36.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.263+0000 7f5c9ffff640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c8c06ae70 0x7f5c8c06d330 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5c900011e0 tx=0x7f5c90009040 comp rx=0 tx=0).stop 2023-11-08T16:30:36.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.263+0000 7f5c9ffff640 1 -- 172.21.15.81:0/3542181224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ca015e1a0 msgr2=0x7f5ca00c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:36.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.263+0000 7f5c9ffff640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ca015e1a0 0x7f5ca00c5910 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5c98007e60 tx=0x7f5c9800e4b0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.263+0000 7f5c9ffff640 1 -- 172.21.15.81:0/3542181224 shutdown_connections 2023-11-08T16:30:36.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.263+0000 7f5c9ffff640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c8c06ae70 0x7f5c8c06d330 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.263+0000 7f5c9ffff640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ca015e1a0 0x7f5ca00c5910 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.263+0000 7f5c9ffff640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ca00c97b0 0x7f5ca014b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.263+0000 7f5c9ffff640 1 --2- 172.21.15.81:0/3542181224 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ca0070360 0x7f5ca00c7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.263+0000 7f5c9ffff640 1 -- 172.21.15.81:0/3542181224 >> 172.21.15.81:0/3542181224 conn(0x7f5ca00ce8d0 msgr2=0x7f5ca015e750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:36.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.264+0000 7f5c9ffff640 1 -- 172.21.15.81:0/3542181224 shutdown_connections 2023-11-08T16:30:36.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.264+0000 7f5c9ffff640 1 -- 172.21.15.81:0/3542181224 wait complete. 2023-11-08T16:30:36.285 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:30:36.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.486+0000 7feee14ff640 1 Processor -- start 2023-11-08T16:30:36.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.486+0000 7feee14ff640 1 -- start start 2023-11-08T16:30:36.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.487+0000 7feee14ff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7feedc15aac0 0x7feedc0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.487+0000 7feee14ff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc0716e0 0x7feedc0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.488+0000 7feee14ff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feedc070470 con 0x7feedc0cf250 2023-11-08T16:30:36.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.488+0000 7feee14ff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feedc06f650 con 0x7feedc15aac0 2023-11-08T16:30:36.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.488+0000 7feee14ff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0cf6c0 con 0x7feedc0716e0 2023-11-08T16:30:36.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.488+0000 7feee0cfe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7feedc0cf250 0x7feedc06b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53700/0 (socket says 172.21.15.81:53700) 2023-11-08T16:30:36.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.488+0000 7feee0cfe640 1 -- 172.21.15.81:0/46570005 learned_addr learned my addr 172.21.15.81:0/46570005 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:36.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.488+0000 7feedbfff640 1 --2- 172.21.15.81:0/46570005 >> v2:172.21.15.93:3300/0 conn(0x7feedc15aac0 0x7feedc0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:36.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.488+0000 7feed3fff640 1 --2- 172.21.15.81:0/46570005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc0716e0 0x7feedc0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:36.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.488+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640733172 0 0) 0x7feedc070470 con 0x7feedc0cf250 2023-11-08T16:30:36.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.488+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feec8003520 con 0x7feedc0cf250 2023-11-08T16:30:36.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1738951356 0 0) 0x7feec8003520 con 0x7feedc0cf250 2023-11-08T16:30:36.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc0716e0 msgr2=0x7feedc0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:36.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 --2- 172.21.15.81:0/46570005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc0716e0 0x7feedc0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 >> v2:172.21.15.93:3300/0 conn(0x7feedc15aac0 msgr2=0x7feedc0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:36.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 --2- 172.21.15.81:0/46570005 >> v2:172.21.15.93:3300/0 conn(0x7feedc15aac0 0x7feedc0711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc06d580 con 0x7feedc0cf250 2023-11-08T16:30:36.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7feed40019f0 con 0x7feedc0cf250 2023-11-08T16:30:36.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 >> v1:172.21.15.81:6789/0 conn(0x7feedc0cf250 legacy=0x7feedc06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:36.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 --2- 172.21.15.81:0/46570005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 0x7feec8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 --2- 172.21.15.81:0/46570005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc0716e0 0x7feec80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 --2- 172.21.15.81:0/46570005 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feec8004d00 0x7feec8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc06f650 con 0x7feec8003a50 2023-11-08T16:30:36.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0cf6c0 con 0x7feec8004d00 2023-11-08T16:30:36.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec800b9b0 con 0x7feedc0716e0 2023-11-08T16:30:36.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedbfff640 1 --2- 172.21.15.81:0/46570005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 0x7feec8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:36.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedbfff640 1 -- 172.21.15.81:0/46570005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc0716e0 msgr2=0x7feec80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:36.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedbfff640 1 --2- 172.21.15.81:0/46570005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc0716e0 0x7feec80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedbfff640 1 -- 172.21.15.81:0/46570005 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feec8004d00 msgr2=0x7feec8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:36.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedbfff640 1 --2- 172.21.15.81:0/46570005 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feec8004d00 0x7feec8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.489+0000 7feedbfff640 1 -- 172.21.15.81:0/46570005 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec8003520 con 0x7feec8003a50 2023-11-08T16:30:36.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.490+0000 7feedbfff640 1 --2- 172.21.15.81:0/46570005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 0x7feec8003e70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feecc009ac0 tx=0x7feec80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90230d2d9a18514 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:36.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.491+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc02c710 con 0x7feec8003a50 2023-11-08T16:30:36.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.491+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc02c870 con 0x7feec8003a50 2023-11-08T16:30:36.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.491+0000 7feedb7fe640 1 -- 172.21.15.81:0/46570005 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc02cc00 con 0x7feec8003a50 2023-11-08T16:30:36.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.491+0000 7feee14ff640 1 -- 172.21.15.81:0/46570005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 msgr2=0x7feec8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:36.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.491+0000 7feee14ff640 1 --2- 172.21.15.81:0/46570005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 0x7feec8003e70 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7feecc009ac0 tx=0x7feec80057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.491+0000 7feee0cfe640 1 -- 172.21.15.81:0/46570005 reap_dead start 2023-11-08T16:30:36.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.491+0000 7feee14ff640 1 -- 172.21.15.81:0/46570005 shutdown_connections 2023-11-08T16:30:36.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.491+0000 7feee14ff640 1 -- 172.21.15.81:0/46570005 >> 172.21.15.81:0/46570005 conn(0x7feedc06a4a0 msgr2=0x7feedc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:36.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.491+0000 7feee14ff640 1 -- 172.21.15.81:0/46570005 shutdown_connections 2023-11-08T16:30:36.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.491+0000 7feee14ff640 1 -- 172.21.15.81:0/46570005 wait complete. 2023-11-08T16:30:36.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.493+0000 7feee14ff640 1 Processor -- start 2023-11-08T16:30:36.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.493+0000 7feee14ff640 1 -- start start 2023-11-08T16:30:36.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.494+0000 7feee14ff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 0x7feedc155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.494+0000 7feee14ff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc156060 0x7feedc0af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.494+0000 7feee14ff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc156590 0x7feedc1e5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.494+0000 7feee14ff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0b0e80 con 0x7feec8003a50 2023-11-08T16:30:36.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.494+0000 7feee14ff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0b1160 con 0x7feedc156060 2023-11-08T16:30:36.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.494+0000 7feee14ff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feedc0af060 con 0x7feedc156590 2023-11-08T16:30:36.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feedbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 0x7feedc155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:36.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feedbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 0x7feedc155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40850/0 (socket says 172.21.15.81:40850) 2023-11-08T16:30:36.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feedbfff640 1 -- 172.21.15.81:0/3982771921 learned_addr learned my addr 172.21.15.81:0/3982771921 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:36.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feed3fff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc156060 0x7feedc0af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:36.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feee0cfe640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc156590 0x7feedc1e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:36.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feedbfff640 1 -- 172.21.15.81:0/3982771921 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc156590 msgr2=0x7feedc1e5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:36.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feedbfff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc156590 0x7feedc1e5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feedbfff640 1 -- 172.21.15.81:0/3982771921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc156060 msgr2=0x7feedc0af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:36.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feedbfff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc156060 0x7feedc0af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feedbfff640 1 -- 172.21.15.81:0/3982771921 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feedc0af340 con 0x7feec8003a50 2023-11-08T16:30:36.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feed3fff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc156060 0x7feedc0af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:36.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feedbfff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 0x7feedc155ae0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feecc037040 tx=0x7feecc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:36.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.495+0000 7feed97fa640 1 -- 172.21.15.81:0/3982771921 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc003810 con 0x7feec8003a50 2023-11-08T16:30:36.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.496+0000 7feed97fa640 1 -- 172.21.15.81:0/3982771921 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feecc003970 con 0x7feec8003a50 2023-11-08T16:30:36.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.496+0000 7feed97fa640 1 -- 172.21.15.81:0/3982771921 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feecc003d00 con 0x7feec8003a50 2023-11-08T16:30:36.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.496+0000 7feee14ff640 1 -- 172.21.15.81:0/3982771921 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feedc0afda0 con 0x7feec8003a50 2023-11-08T16:30:36.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.496+0000 7feee14ff640 1 -- 172.21.15.81:0/3982771921 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feedc1e5970 con 0x7feec8003a50 2023-11-08T16:30:36.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.497+0000 7feee14ff640 1 -- 172.21.15.81:0/3982771921 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feedc067800 con 0x7feec8003a50 2023-11-08T16:30:36.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.498+0000 7feed97fa640 1 -- 172.21.15.81:0/3982771921 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feecc040450 con 0x7feec8003a50 2023-11-08T16:30:36.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.499+0000 7feed97fa640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feebc06aea0 0x7feebc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:36.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.499+0000 7feed97fa640 1 -- 172.21.15.81:0/3982771921 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feecc0ab4d0 con 0x7feec8003a50 2023-11-08T16:30:36.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.499+0000 7feed3fff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feebc06aea0 0x7feebc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:36.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.499+0000 7feed3fff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feebc06aea0 0x7feebc06d360 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feedc159c20 tx=0x7feec4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:36.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.503+0000 7feed97fa640 1 -- 172.21.15.81:0/3982771921 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feecc076570 con 0x7feec8003a50 2023-11-08T16:30:36.647 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:30:36.646+0000 7f42849a8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:30:36.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.720+0000 7feee14ff640 1 -- 172.21.15.81:0/3982771921 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feedc159c50 con 0x7feec8003a50 2023-11-08T16:30:36.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.722+0000 7feed97fa640 1 -- 172.21.15.81:0/3982771921 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+4907 (secure 0 0 0) 0x7feecc003e60 con 0x7feec8003a50 2023-11-08T16:30:36.724 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:36.724 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":18,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/1453041994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":1453041994},{"type":"v1","addr":"172.21.15.93:6835","nonce":1453041994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/757624701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":757624701},{"type":"v1","addr":"172.21.15.93:6833","nonce":757624701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1735232404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1735232404},{"type":"v1","addr":"172.21.15.81:6835","nonce":1735232404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:30:30.141674+0000","modified":"2023-11-08T16:30:33.294977+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5139":{"gid":5139,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/773130363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":773130363},{"type":"v1","addr":"172.21.15.81:6837","nonce":773130363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:30:36.725 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 18 2023-11-08T16:30:36.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.724+0000 7feee14ff640 1 -- 172.21.15.81:0/3982771921 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feebc06aea0 msgr2=0x7feebc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:36.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.724+0000 7feee14ff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feebc06aea0 0x7feebc06d360 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feedc159c20 tx=0x7feec4008040 comp rx=0 tx=0).stop 2023-11-08T16:30:36.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.724+0000 7feee14ff640 1 -- 172.21.15.81:0/3982771921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 msgr2=0x7feedc155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:36.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.724+0000 7feee14ff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 0x7feedc155ae0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feecc037040 tx=0x7feecc004730 comp rx=0 tx=0).stop 2023-11-08T16:30:36.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.724+0000 7feee14ff640 1 -- 172.21.15.81:0/3982771921 shutdown_connections 2023-11-08T16:30:36.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.724+0000 7feee14ff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feebc06aea0 0x7feebc06d360 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.724+0000 7feee14ff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feedc156590 0x7feedc1e5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.724+0000 7feee14ff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feedc156060 0x7feedc0af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.724+0000 7feee14ff640 1 --2- 172.21.15.81:0/3982771921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feec8003a50 0x7feedc155ae0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:36.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.725+0000 7feee14ff640 1 -- 172.21.15.81:0/3982771921 >> 172.21.15.81:0/3982771921 conn(0x7feedc06a4a0 msgr2=0x7feedc15ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:36.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.725+0000 7feee14ff640 1 -- 172.21.15.81:0/3982771921 shutdown_connections 2023-11-08T16:30:36.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:36.725+0000 7feee14ff640 1 -- 172.21.15.81:0/3982771921 wait complete. 2023-11-08T16:30:37.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 18, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:30:30.141674+0000', 'modified': '2023-11-08T16:30:33.294977+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5139}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5139': {'gid': 5139, 'name': 'c', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6837/773130363', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 773130363}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 773130363}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:30:37.148 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:30:37.149 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T16:30:37.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.366+0000 7fe03204b640 1 Processor -- start 2023-11-08T16:30:37.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.366+0000 7fe03204b640 1 -- start start 2023-11-08T16:30:37.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.367+0000 7fe03204b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe02c0c90f0 0x7fe02c0c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.367+0000 7fe03204b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe02c0b18f0 0x7fe02c0b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.367+0000 7fe03204b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe02c06cc60 con 0x7fe02c15aeb0 2023-11-08T16:30:37.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.367+0000 7fe03204b640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe02c0cb3e0 con 0x7fe02c0b18f0 2023-11-08T16:30:37.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.368+0000 7fe03204b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c0cabb0 con 0x7fe02c0c90f0 2023-11-08T16:30:37.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.368+0000 7fe02bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe02c0b18f0 0x7fe02c0b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.368+0000 7fe02b7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe02c15aeb0 0x7fe02c0ca4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53708/0 (socket says 172.21.15.81:53708) 2023-11-08T16:30:37.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.368+0000 7fe02bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe02c0b18f0 0x7fe02c0b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40864/0 (socket says 172.21.15.81:40864) 2023-11-08T16:30:37.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.368+0000 7fe02b7fe640 1 -- 172.21.15.81:0/3001074227 learned_addr learned my addr 172.21.15.81:0/3001074227 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:37.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.368+0000 7fe02affd640 1 --2- 172.21.15.81:0/3001074227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe02c0c90f0 0x7fe02c0c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.368+0000 7fe02bfff640 1 -- 172.21.15.81:0/3001074227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe02c0c90f0 msgr2=0x7fe02c0c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.368+0000 7fe02bfff640 1 --2- 172.21.15.81:0/3001074227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe02c0c90f0 0x7fe02c0c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.368+0000 7fe02bfff640 1 -- 172.21.15.81:0/3001074227 >> v1:172.21.15.81:6789/0 conn(0x7fe02c15aeb0 legacy=0x7fe02c0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.368+0000 7fe02bfff640 1 -- 172.21.15.81:0/3001074227 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe02c0c9a90 con 0x7fe02c0b18f0 2023-11-08T16:30:37.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.369+0000 7fe02bfff640 1 --2- 172.21.15.81:0/3001074227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0b18f0 0x7fe02c0b1d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe0240097a0 tx=0x7fe02402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b0f98af2de1469 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:37.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.370+0000 7fe02a7fc640 1 -- 172.21.15.81:0/3001074227 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02402d040 con 0x7fe02c0b18f0 2023-11-08T16:30:37.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.370+0000 7fe02a7fc640 1 -- 172.21.15.81:0/3001074227 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe024031810 con 0x7fe02c0b18f0 2023-11-08T16:30:37.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.370+0000 7fe02a7fc640 1 -- 172.21.15.81:0/3001074227 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024031ba0 con 0x7fe02c0b18f0 2023-11-08T16:30:37.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.370+0000 7fe03204b640 1 -- 172.21.15.81:0/3001074227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0b18f0 msgr2=0x7fe02c0b1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.370+0000 7fe03204b640 1 --2- 172.21.15.81:0/3001074227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0b18f0 0x7fe02c0b1d40 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe0240097a0 tx=0x7fe02402b8b0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.370+0000 7fe03204b640 1 -- 172.21.15.81:0/3001074227 shutdown_connections 2023-11-08T16:30:37.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.370+0000 7fe03204b640 1 --2- 172.21.15.81:0/3001074227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c0b18f0 0x7fe02c0b1d40 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.370+0000 7fe03204b640 1 --2- 172.21.15.81:0/3001074227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe02c0c90f0 0x7fe02c0c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.370+0000 7fe03204b640 1 -- 172.21.15.81:0/3001074227 >> 172.21.15.81:0/3001074227 conn(0x7fe02c0ceaa0 msgr2=0x7fe02c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:37.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.371+0000 7fe03204b640 1 -- 172.21.15.81:0/3001074227 shutdown_connections 2023-11-08T16:30:37.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.371+0000 7fe03204b640 1 -- 172.21.15.81:0/3001074227 wait complete. 2023-11-08T16:30:37.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.372+0000 7fe03204b640 1 Processor -- start 2023-11-08T16:30:37.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.372+0000 7fe03204b640 1 -- start start 2023-11-08T16:30:37.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.373+0000 7fe03204b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe02c0b18f0 0x7fe02c152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.373+0000 7fe03204b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe02c0c90f0 0x7fe02c15d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.373+0000 7fe03204b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c15aeb0 0x7fe02c15d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.373+0000 7fe03204b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c15c2f0 con 0x7fe02c15aeb0 2023-11-08T16:30:37.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.373+0000 7fe03204b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c15c5d0 con 0x7fe02c0b18f0 2023-11-08T16:30:37.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.373+0000 7fe03204b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe02c15c8b0 con 0x7fe02c0c90f0 2023-11-08T16:30:37.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.374+0000 7fe02bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c15aeb0 0x7fe02c15d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.374+0000 7fe02bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c15aeb0 0x7fe02c15d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40866/0 (socket says 172.21.15.81:40866) 2023-11-08T16:30:37.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.374+0000 7fe02bfff640 1 -- 172.21.15.81:0/2579394156 learned_addr learned my addr 172.21.15.81:0/2579394156 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:37.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.374+0000 7fe02bfff640 1 -- 172.21.15.81:0/2579394156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe02c0c90f0 msgr2=0x7fe02c15d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.374+0000 7fe02bfff640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe02c0c90f0 0x7fe02c15d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.374+0000 7fe02bfff640 1 -- 172.21.15.81:0/2579394156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe02c0b18f0 msgr2=0x7fe02c152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.374+0000 7fe02bfff640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe02c0b18f0 0x7fe02c152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.374+0000 7fe02bfff640 1 -- 172.21.15.81:0/2579394156 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe024009400 con 0x7fe02c15aeb0 2023-11-08T16:30:37.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.374+0000 7fe02bfff640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c15aeb0 0x7fe02c15d6a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe02402b8f0 tx=0x7fe0240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:37.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.375+0000 7fe023fff640 1 -- 172.21.15.81:0/2579394156 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02403ace0 con 0x7fe02c15aeb0 2023-11-08T16:30:37.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.375+0000 7fe023fff640 1 -- 172.21.15.81:0/2579394156 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02403ae60 con 0x7fe02c15aeb0 2023-11-08T16:30:37.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.375+0000 7fe03204b640 1 -- 172.21.15.81:0/2579394156 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe02c15cb90 con 0x7fe02c15aeb0 2023-11-08T16:30:37.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.375+0000 7fe023fff640 1 -- 172.21.15.81:0/2579394156 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024044630 con 0x7fe02c15aeb0 2023-11-08T16:30:37.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.375+0000 7fe03204b640 1 -- 172.21.15.81:0/2579394156 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c15b780 con 0x7fe02c15aeb0 2023-11-08T16:30:37.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.377+0000 7fe023fff640 1 -- 172.21.15.81:0/2579394156 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe024044790 con 0x7fe02c15aeb0 2023-11-08T16:30:37.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.378+0000 7fe023fff640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe00c06ae70 0x7fe00c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.378+0000 7fe023fff640 1 -- 172.21.15.81:0/2579394156 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0240aba10 con 0x7fe02c15aeb0 2023-11-08T16:30:37.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.378+0000 7fe02b7fe640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe00c06ae70 0x7fe00c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.379+0000 7fe02b7fe640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe00c06ae70 0x7fe00c06d330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe01c001c30 tx=0x7fe01c007c30 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:37.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.379+0000 7fe034bd6740 1 Processor -- start 2023-11-08T16:30:37.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.379+0000 7fe034bd6740 1 -- start start 2023-11-08T16:30:37.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d97ec0 0x559899d6adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d982a0 0x559899d9ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d6b300 0x559899d9b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe034bd6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559899cdd7e0 con 0x559899d6b300 2023-11-08T16:30:37.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe034bd6740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559899cdaed0 con 0x559899d982a0 2023-11-08T16:30:37.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe034bd6740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x559899cd67c0 con 0x559899d97ec0 2023-11-08T16:30:37.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe02bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d6b300 0x559899d9b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe02bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d6b300 0x559899d9b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40868/0 (socket says 172.21.15.81:40868) 2023-11-08T16:30:37.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe02bfff640 1 -- 172.21.15.81:0/2079950393 learned_addr learned my addr 172.21.15.81:0/2079950393 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:37.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe02b7fe640 1 --2- 172.21.15.81:0/2079950393 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d97ec0 0x559899d6adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe02bfff640 1 -- 172.21.15.81:0/2079950393 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d97ec0 msgr2=0x559899d6adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe02bfff640 1 --2- 172.21.15.81:0/2079950393 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d97ec0 0x559899d6adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe02bfff640 1 -- 172.21.15.81:0/2079950393 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d982a0 msgr2=0x559899d9ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe02bfff640 1 --2- 172.21.15.81:0/2079950393 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d982a0 0x559899d9ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.380+0000 7fe02bfff640 1 -- 172.21.15.81:0/2079950393 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559899ca9b10 con 0x559899d6b300 2023-11-08T16:30:37.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.381+0000 7fe02bfff640 1 --2- 172.21.15.81:0/2079950393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d6b300 0x559899d9b190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe024076be0 tx=0x7fe02407ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7a87498ca9d196 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:37.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.381+0000 7fe0217fa640 1 -- 172.21.15.81:0/2079950393 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02407e830 con 0x559899d6b300 2023-11-08T16:30:37.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.381+0000 7fe0217fa640 1 -- 172.21.15.81:0/2079950393 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0240829e0 con 0x559899d6b300 2023-11-08T16:30:37.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.381+0000 7fe0217fa640 1 -- 172.21.15.81:0/2079950393 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024076df0 con 0x559899d6b300 2023-11-08T16:30:37.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.381+0000 7fe034bd6740 1 -- 172.21.15.81:0/2079950393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d6b300 msgr2=0x559899d9b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.381+0000 7fe034bd6740 1 --2- 172.21.15.81:0/2079950393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d6b300 0x559899d9b190 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe024076be0 tx=0x7fe02407ade0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.382+0000 7fe034bd6740 1 -- 172.21.15.81:0/2079950393 shutdown_connections 2023-11-08T16:30:37.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.382+0000 7fe034bd6740 1 --2- 172.21.15.81:0/2079950393 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d6b300 0x559899d9b190 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.382+0000 7fe034bd6740 1 --2- 172.21.15.81:0/2079950393 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d982a0 0x559899d9ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.382+0000 7fe034bd6740 1 --2- 172.21.15.81:0/2079950393 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d97ec0 0x559899d6adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.382+0000 7fe034bd6740 1 -- 172.21.15.81:0/2079950393 >> 172.21.15.81:0/2079950393 conn(0x559899d6d0d0 msgr2=0x559899db5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:37.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.382+0000 7fe034bd6740 1 -- 172.21.15.81:0/2079950393 shutdown_connections 2023-11-08T16:30:37.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.382+0000 7fe034bd6740 1 -- 172.21.15.81:0/2079950393 wait complete. 2023-11-08T16:30:37.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.382+0000 7fe034bd6740 1 -- start start 2023-11-08T16:30:37.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.383+0000 7fe034bd6740 1 Processor -- start 2023-11-08T16:30:37.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d97ec0 0x559899d9dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d982a0 0x559899d9e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe034bd6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559899c73c40 con 0x559899d901c0 2023-11-08T16:30:37.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe034bd6740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559899c738e0 con 0x559899d97ec0 2023-11-08T16:30:37.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe034bd6740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x559899a39f20 con 0x559899d982a0 2023-11-08T16:30:37.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe034bd6740 10 client.0 _ll_register_callbacks cb 0x559899c95cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:30:37.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe02bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe02bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d992b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40882/0 (socket says 172.21.15.81:40882) 2023-11-08T16:30:37.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe02bfff640 1 -- 172.21.15.81:0/2783992995 learned_addr learned my addr 172.21.15.81:0/2783992995 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:37.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe02bfff640 1 -- 172.21.15.81:0/2783992995 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d982a0 msgr2=0x559899d9e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe02bfff640 1 --2- 172.21.15.81:0/2783992995 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d982a0 0x559899d9e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe02bfff640 1 -- 172.21.15.81:0/2783992995 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d97ec0 msgr2=0x559899d9dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe02bfff640 1 --2- 172.21.15.81:0/2783992995 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d97ec0 0x559899d9dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.384+0000 7fe02bfff640 1 -- 172.21.15.81:0/2783992995 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c15b780 con 0x559899d901c0 2023-11-08T16:30:37.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe02bfff640 1 --2- 172.21.15.81:0/2783992995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d992b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe02407a620 tx=0x7fe024031f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e1047002525a8f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:37.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe00a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:30:37.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe00a7fc640 1 -- 172.21.15.81:0/2783992995 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024092040 con 0x559899d901c0 2023-11-08T16:30:37.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe00a7fc640 1 -- 172.21.15.81:0/2783992995 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe024082550 con 0x559899d901c0 2023-11-08T16:30:37.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe034bd6740 1 -- 172.21.15.81:0/2783992995 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559899caac80 con 0x559899d901c0 2023-11-08T16:30:37.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe00a7fc640 1 -- 172.21.15.81:0/2783992995 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe024082710 con 0x559899d901c0 2023-11-08T16:30:37.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe034bd6740 10 client.5304 fetch_fsmap learned FSMap version 18 2023-11-08T16:30:37.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe034bd6740 1 -- 172.21.15.81:0/2783992995 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x559899cd4e40 con 0x559899d901c0 2023-11-08T16:30:37.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe00a7fc640 1 -- 172.21.15.81:0/2783992995 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559899caac80 con 0x559899d901c0 2023-11-08T16:30:37.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe00a7fc640 1 -- 172.21.15.81:0/2783992995 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe0240764d0 con 0x559899d901c0 2023-11-08T16:30:37.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe034bd6740 10 client.5304 fetch_fsmap finished waiting for FSMap version 18 2023-11-08T16:30:37.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe034bd6740 10 client.5304 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:30:37.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe034bd6740 20 client.5304 populate_metadata read hostname 'smithi081' 2023-11-08T16:30:37.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe034bd6740 1 --2- 172.21.15.81:0/2783992995 >> [v2:172.21.15.81:6836/773130363,v1:172.21.15.81:6837/773130363] conn(0x559899d9a170 0x559899dc5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe034bd6740 4 client.5304 mds_command: new command op to 5139 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:30:37.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.385+0000 7fe034bd6740 1 -- 172.21.15.81:0/2783992995 --> [v2:172.21.15.81:6836/773130363,v1:172.21.15.81:6837/773130363] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559899ccf7a0 con 0x559899d9a170 2023-11-08T16:30:37.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.386+0000 7fe02b7fe640 1 --2- 172.21.15.81:0/2783992995 >> [v2:172.21.15.81:6836/773130363,v1:172.21.15.81:6837/773130363] conn(0x559899d9a170 0x559899dc5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.386+0000 7fe02b7fe640 1 --2- 172.21.15.81:0/2783992995 >> [v2:172.21.15.81:6836/773130363,v1:172.21.15.81:6837/773130363] conn(0x559899d9a170 0x559899dc5580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f48a2b1b64185ca server_cookie=80f89c0d7fba543d in_seq=0 out_seq=0 2023-11-08T16:30:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.386+0000 7fe00a7fc640 10 client.5304 ms_handle_connect on v2:172.21.15.81:6836/773130363 2023-11-08T16:30:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe00a7fc640 1 -- 172.21.15.81:0/2783992995 <== mds.0 v2:172.21.15.81:6836/773130363 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559899ccf7a0 con 0x559899d9a170 2023-11-08T16:30:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe00a7fc640 10 client.5304 handle_command_reply: tid=0 2023-11-08T16:30:37.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe034bd6740 1 client.5304 shutdown 2023-11-08T16:30:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe034bd6740 1 -- 172.21.15.81:0/2783992995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 msgr2=0x559899d992b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe034bd6740 1 --2- 172.21.15.81:0/2783992995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d992b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe02407a620 tx=0x7fe024031f60 comp rx=0 tx=0).stop 2023-11-08T16:30:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe034bd6740 1 -- 172.21.15.81:0/2783992995 shutdown_connections 2023-11-08T16:30:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe034bd6740 1 --2- 172.21.15.81:0/2783992995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d992b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe034bd6740 1 --2- 172.21.15.81:0/2783992995 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d982a0 0x559899d9e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe034bd6740 1 --2- 172.21.15.81:0/2783992995 >> [v2:172.21.15.81:6836/773130363,v1:172.21.15.81:6837/773130363] conn(0x559899d9a170 0x559899dc5580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe034bd6740 1 --2- 172.21.15.81:0/2783992995 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d97ec0 0x559899d9dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe034bd6740 1 -- 172.21.15.81:0/2783992995 >> 172.21.15.81:0/2783992995 conn(0x559899d6b300 msgr2=0x559899d6ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe00a7fc640 0 client.5304 ms_handle_reset on v2:172.21.15.81:6836/773130363 2023-11-08T16:30:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe034bd6740 1 -- 172.21.15.81:0/2783992995 shutdown_connections 2023-11-08T16:30:37.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.387+0000 7fe034bd6740 1 -- 172.21.15.81:0/2783992995 wait complete. 2023-11-08T16:30:37.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.388+0000 7fe034bd6740 20 client.5304 trim_cache size 0 max 16384 2023-11-08T16:30:37.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.404+0000 7fe034bd6740 1 Processor -- start 2023-11-08T16:30:37.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.404+0000 7fe034bd6740 1 -- start start 2023-11-08T16:30:37.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.405+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d93820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.405+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d97ec0 0x559899d6bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.405+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d982a0 0x559899db9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.405+0000 7fe034bd6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559899db6b40 con 0x559899d901c0 2023-11-08T16:30:37.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.405+0000 7fe034bd6740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559899d6e660 con 0x559899d982a0 2023-11-08T16:30:37.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.405+0000 7fe034bd6740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x559899d6ccd0 con 0x559899d97ec0 2023-11-08T16:30:37.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.405+0000 7fe02b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d93820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.406+0000 7fe02b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d93820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40892/0 (socket says 172.21.15.81:40892) 2023-11-08T16:30:37.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.406+0000 7fe02b7fe640 1 -- 172.21.15.81:0/997462958 learned_addr learned my addr 172.21.15.81:0/997462958 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:37.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.406+0000 7fe02bfff640 1 --2- 172.21.15.81:0/997462958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d982a0 0x559899db9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.406+0000 7fe02b7fe640 1 -- 172.21.15.81:0/997462958 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d97ec0 msgr2=0x559899d6bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.406+0000 7fe02b7fe640 1 --2- 172.21.15.81:0/997462958 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d97ec0 0x559899d6bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.406+0000 7fe02b7fe640 1 -- 172.21.15.81:0/997462958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d982a0 msgr2=0x559899db9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.406+0000 7fe02b7fe640 1 --2- 172.21.15.81:0/997462958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d982a0 0x559899db9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.406+0000 7fe02b7fe640 1 -- 172.21.15.81:0/997462958 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559899c72b90 con 0x559899d901c0 2023-11-08T16:30:37.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.406+0000 7fe02b7fe640 1 --2- 172.21.15.81:0/997462958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d93820 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe01c001380 tx=0x7fe01c001580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7d0f49f0faadd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:37.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe021ffb640 1 -- 172.21.15.81:0/997462958 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c00cdb0 con 0x559899d901c0 2023-11-08T16:30:37.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe021ffb640 1 -- 172.21.15.81:0/997462958 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe01c0203c0 con 0x559899d901c0 2023-11-08T16:30:37.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe021ffb640 1 -- 172.21.15.81:0/997462958 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c020730 con 0x559899d901c0 2023-11-08T16:30:37.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe034bd6740 1 -- 172.21.15.81:0/997462958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 msgr2=0x559899d93820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe034bd6740 1 --2- 172.21.15.81:0/997462958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d93820 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe01c001380 tx=0x7fe01c001580 comp rx=0 tx=0).stop 2023-11-08T16:30:37.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe034bd6740 1 -- 172.21.15.81:0/997462958 shutdown_connections 2023-11-08T16:30:37.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe034bd6740 1 --2- 172.21.15.81:0/997462958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d982a0 0x559899db9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe034bd6740 1 --2- 172.21.15.81:0/997462958 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d97ec0 0x559899d6bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe034bd6740 1 --2- 172.21.15.81:0/997462958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899d93820 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe034bd6740 1 -- 172.21.15.81:0/997462958 >> 172.21.15.81:0/997462958 conn(0x559899d6b300 msgr2=0x559899d66560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:37.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe034bd6740 1 -- 172.21.15.81:0/997462958 shutdown_connections 2023-11-08T16:30:37.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.407+0000 7fe034bd6740 1 -- 172.21.15.81:0/997462958 wait complete. 2023-11-08T16:30:37.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.408+0000 7fe034bd6740 1 -- start start 2023-11-08T16:30:37.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.408+0000 7fe034bd6740 1 Processor -- start 2023-11-08T16:30:37.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.409+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899db5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.409+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d97ec0 0x559899e060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.409+0000 7fe034bd6740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d982a0 0x559899db9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.409+0000 7fe034bd6740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x559899db6160 con 0x559899d901c0 2023-11-08T16:30:37.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.409+0000 7fe034bd6740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559899d97160 con 0x559899d97ec0 2023-11-08T16:30:37.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.409+0000 7fe034bd6740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x559899db5c30 con 0x559899d982a0 2023-11-08T16:30:37.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.409+0000 7fe034bd6740 10 client.0 _ll_register_callbacks cb 0x7fe02407a2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:30:37.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.409+0000 7fe02b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899db5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.409+0000 7fe02b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899db5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40902/0 (socket says 172.21.15.81:40902) 2023-11-08T16:30:37.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.409+0000 7fe02b7fe640 1 -- 172.21.15.81:0/1053093619 learned_addr learned my addr 172.21.15.81:0/1053093619 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:37.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe02affd640 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d97ec0 0x559899e060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe02bfff640 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d982a0 0x559899db9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe02b7fe640 1 -- 172.21.15.81:0/1053093619 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d982a0 msgr2=0x559899db9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe02b7fe640 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d982a0 0x559899db9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe02b7fe640 1 -- 172.21.15.81:0/1053093619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d97ec0 msgr2=0x559899e060f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe02b7fe640 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d97ec0 0x559899e060f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe02b7fe640 1 -- 172.21.15.81:0/1053093619 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe02c06cc60 con 0x559899d901c0 2023-11-08T16:30:37.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe02bfff640 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d982a0 0x559899db9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:37.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe02b7fe640 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899db5630 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe01c022040 tx=0x7fe02404f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88aecedef6a17e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:37.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe00a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:30:37.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe00a7fc640 1 -- 172.21.15.81:0/1053093619 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01c020730 con 0x559899d901c0 2023-11-08T16:30:37.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe00a7fc640 1 -- 172.21.15.81:0/1053093619 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe01c020890 con 0x559899d901c0 2023-11-08T16:30:37.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.410+0000 7fe034bd6740 1 -- 172.21.15.81:0/1053093619 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe0240829e0 con 0x559899d901c0 2023-11-08T16:30:37.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe00a7fc640 1 -- 172.21.15.81:0/1053093619 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=18) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe01c0070d0 con 0x559899d901c0 2023-11-08T16:30:37.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe034bd6740 10 client.5310 fetch_fsmap learned FSMap version 18 2023-11-08T16:30:37.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe034bd6740 1 -- 172.21.15.81:0/1053093619 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=18,monmap=2+}) v3 -- 0x559899c98180 con 0x559899d901c0 2023-11-08T16:30:37.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe00a7fc640 1 -- 172.21.15.81:0/1053093619 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0240829e0 con 0x559899d901c0 2023-11-08T16:30:37.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe00a7fc640 1 -- 172.21.15.81:0/1053093619 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 18) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe01c01f040 con 0x559899d901c0 2023-11-08T16:30:37.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe034bd6740 10 client.5310 fetch_fsmap finished waiting for FSMap version 18 2023-11-08T16:30:37.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe034bd6740 10 client.5310 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:30:37.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe034bd6740 20 client.5310 populate_metadata read hostname 'smithi081' 2023-11-08T16:30:37.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe034bd6740 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.81:6836/773130363,v1:172.21.15.81:6837/773130363] conn(0x559899d96240 0x559899d96620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe034bd6740 4 client.5310 mds_command: new command op to 5139 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:30:37.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe034bd6740 1 -- 172.21.15.81:0/1053093619 --> [v2:172.21.15.81:6836/773130363,v1:172.21.15.81:6837/773130363] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559899ccf4c0 con 0x559899d96240 2023-11-08T16:30:37.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.411+0000 7fe02affd640 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.81:6836/773130363,v1:172.21.15.81:6837/773130363] conn(0x559899d96240 0x559899d96620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe00a7fc640 10 client.5310 ms_handle_connect on v2:172.21.15.81:6836/773130363 2023-11-08T16:30:37.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe02affd640 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.81:6836/773130363,v1:172.21.15.81:6837/773130363] conn(0x559899d96240 0x559899d96620 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57545b05e8f7c1e9 server_cookie=6d323d0b45c3fe6e in_seq=0 out_seq=0 2023-11-08T16:30:37.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe00a7fc640 1 -- 172.21.15.81:0/1053093619 <== mds.0 v2:172.21.15.81:6836/773130363 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x559899ccf4c0 con 0x559899d96240 2023-11-08T16:30:37.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe00a7fc640 10 client.5310 handle_command_reply: tid=0 2023-11-08T16:30:37.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe034bd6740 1 client.5310 shutdown 2023-11-08T16:30:37.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe034bd6740 1 -- 172.21.15.81:0/1053093619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 msgr2=0x559899db5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe034bd6740 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899db5630 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe01c022040 tx=0x7fe02404f000 comp rx=0 tx=0).stop 2023-11-08T16:30:37.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe034bd6740 1 -- 172.21.15.81:0/1053093619 shutdown_connections 2023-11-08T16:30:37.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe034bd6740 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x559899d982a0 0x559899db9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe034bd6740 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.81:6836/773130363,v1:172.21.15.81:6837/773130363] conn(0x559899d96240 0x559899d96620 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe034bd6740 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559899d97ec0 0x559899e060f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe034bd6740 1 --2- 172.21.15.81:0/1053093619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x559899d901c0 0x559899db5630 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe00a7fc640 0 client.5310 ms_handle_reset on v2:172.21.15.81:6836/773130363 2023-11-08T16:30:37.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe034bd6740 1 -- 172.21.15.81:0/1053093619 >> 172.21.15.81:0/1053093619 conn(0x559899d6b300 msgr2=0x559899d93610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:37.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe034bd6740 1 -- 172.21.15.81:0/1053093619 shutdown_connections 2023-11-08T16:30:37.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.412+0000 7fe034bd6740 1 -- 172.21.15.81:0/1053093619 wait complete. 2023-11-08T16:30:37.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe034bd6740 20 client.5310 trim_cache size 0 max 16384 2023-11-08T16:30:37.432 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:30:37.432 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:30:37.432 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:30:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 5139, 2023-11-08T16:30:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:30:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:30:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:30:37.433 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 6.7103910229999997, 2023-11-08T16:30:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 18, 2023-11-08T16:30:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 35, 2023-11-08T16:30:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 35, 2023-11-08T16:30:37.434 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 16.139456524 2023-11-08T16:30:37.435 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:30:37.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 -- 172.21.15.81:0/2579394156 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe00c06ae70 msgr2=0x7fe00c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe00c06ae70 0x7fe00c06d330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fe01c001c30 tx=0x7fe01c007c30 comp rx=0 tx=0).stop 2023-11-08T16:30:37.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 -- 172.21.15.81:0/2579394156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c15aeb0 msgr2=0x7fe02c15d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c15aeb0 0x7fe02c15d6a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe02402b8f0 tx=0x7fe0240046c0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 -- 172.21.15.81:0/2579394156 shutdown_connections 2023-11-08T16:30:37.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe00c06ae70 0x7fe00c06d330 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe02c15aeb0 0x7fe02c15d6a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe02c0c90f0 0x7fe02c15d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 --2- 172.21.15.81:0/2579394156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe02c0b18f0 0x7fe02c152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 -- 172.21.15.81:0/2579394156 >> 172.21.15.81:0/2579394156 conn(0x7fe02c0ceaa0 msgr2=0x7fe02c06b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:37.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 -- 172.21.15.81:0/2579394156 shutdown_connections 2023-11-08T16:30:37.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.413+0000 7fe03204b640 1 -- 172.21.15.81:0/2579394156 wait complete. 2023-11-08T16:30:37.440 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:30:37.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.642+0000 7ffb61975640 1 Processor -- start 2023-11-08T16:30:37.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.642+0000 7ffb61975640 1 -- start start 2023-11-08T16:30:37.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.644+0000 7ffb61975640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15aac0 0x7ffb5c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.644+0000 7ffb61975640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ffb5c0cf420 0x7ffb5c15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.644+0000 7ffb61975640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5c070470 con 0x7ffb5c0716e0 2023-11-08T16:30:37.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.644+0000 7ffb61975640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb5c06f650 con 0x7ffb5c0cf420 2023-11-08T16:30:37.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.644+0000 7ffb61975640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c06d580 con 0x7ffb5c15aac0 2023-11-08T16:30:37.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.644+0000 7ffb5bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ffb5c0716e0 0x7ffb5c0ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53718/0 (socket says 172.21.15.81:53718) 2023-11-08T16:30:37.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.644+0000 7ffb5bfff640 1 -- 172.21.15.81:0/3078598882 learned_addr learned my addr 172.21.15.81:0/3078598882 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:37.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.644+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1830028163 0 0) 0x7ffb5c070470 con 0x7ffb5c0716e0 2023-11-08T16:30:37.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.644+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb4c003520 con 0x7ffb5c0716e0 2023-11-08T16:30:37.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.644+0000 7ffb61174640 1 --2- 172.21.15.81:0/3078598882 >> v2:172.21.15.93:3300/0 conn(0x7ffb5c0cf420 0x7ffb5c15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.644+0000 7ffb60973640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15aac0 0x7ffb5c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 278312159 0 0) 0x7ffb4c003520 con 0x7ffb5c0716e0 2023-11-08T16:30:37.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15aac0 msgr2=0x7ffb5c0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15aac0 0x7ffb5c0711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 >> v2:172.21.15.93:3300/0 conn(0x7ffb5c0cf420 msgr2=0x7ffb5c15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 --2- 172.21.15.81:0/3078598882 >> v2:172.21.15.93:3300/0 conn(0x7ffb5c0cf420 0x7ffb5c15aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c06ca00 con 0x7ffb5c0716e0 2023-11-08T16:30:37.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ffb48001670 con 0x7ffb5c0716e0 2023-11-08T16:30:37.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 >> v1:172.21.15.81:6789/0 conn(0x7ffb5c0716e0 legacy=0x7ffb5c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 0x7ffb4c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15aac0 0x7ffb4c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb4c004d00 0x7ffb4c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c06f650 con 0x7ffb4c003a50 2023-11-08T16:30:37.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c06d580 con 0x7ffb4c004d00 2023-11-08T16:30:37.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4c00bb60 con 0x7ffb5c15aac0 2023-11-08T16:30:37.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb60973640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 0x7ffb4c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.645+0000 7ffb5bfff640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15aac0 0x7ffb4c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.646+0000 7ffb61174640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb4c004d00 0x7ffb4c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.646+0000 7ffb60973640 1 -- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15aac0 msgr2=0x7ffb4c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.646+0000 7ffb60973640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15aac0 0x7ffb4c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.646+0000 7ffb60973640 1 -- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb4c004d00 msgr2=0x7ffb4c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.646+0000 7ffb60973640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb4c004d00 0x7ffb4c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.646+0000 7ffb60973640 1 -- 172.21.15.81:0/3078598882 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c003520 con 0x7ffb4c003a50 2023-11-08T16:30:37.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.646+0000 7ffb5bfff640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15aac0 0x7ffb4c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:30:37.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.647+0000 7ffb60973640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 0x7ffb4c003e70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000ba60 tx=0x7ffb4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cce0ddc24a7dc2b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:37.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.647+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50032630 con 0x7ffb4c003a50 2023-11-08T16:30:37.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.647+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb50032790 con 0x7ffb4c003a50 2023-11-08T16:30:37.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.647+0000 7ffb5b7fe640 1 -- 172.21.15.81:0/3078598882 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50032b20 con 0x7ffb4c003a50 2023-11-08T16:30:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.648+0000 7ffb61975640 1 -- 172.21.15.81:0/3078598882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 msgr2=0x7ffb4c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.648+0000 7ffb61975640 1 --2- 172.21.15.81:0/3078598882 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 0x7ffb4c003e70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000ba60 tx=0x7ffb4c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.648+0000 7ffb61174640 1 -- 172.21.15.81:0/3078598882 reap_dead start 2023-11-08T16:30:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.648+0000 7ffb61975640 1 -- 172.21.15.81:0/3078598882 shutdown_connections 2023-11-08T16:30:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.648+0000 7ffb61975640 1 -- 172.21.15.81:0/3078598882 >> 172.21.15.81:0/3078598882 conn(0x7ffb5c06a4a0 msgr2=0x7ffb5c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:37.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.648+0000 7ffb61975640 1 -- 172.21.15.81:0/3078598882 shutdown_connections 2023-11-08T16:30:37.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.648+0000 7ffb61975640 1 -- 172.21.15.81:0/3078598882 wait complete. 2023-11-08T16:30:37.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.650+0000 7ffb61975640 1 Processor -- start 2023-11-08T16:30:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.650+0000 7ffb61975640 1 -- start start 2023-11-08T16:30:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.651+0000 7ffb61975640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 0x7ffb5c1423f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.651+0000 7ffb61975640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15b8b0 0x7ffb5c151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.651+0000 7ffb61975640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb5c152310 0x7ffb5c1f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.651+0000 7ffb61975640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c142930 con 0x7ffb4c003a50 2023-11-08T16:30:37.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.651+0000 7ffb61975640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c152790 con 0x7ffb5c152310 2023-11-08T16:30:37.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.651+0000 7ffb61975640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c15c070 con 0x7ffb5c15b8b0 2023-11-08T16:30:37.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.651+0000 7ffb60973640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 0x7ffb5c1423f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.651+0000 7ffb60973640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 0x7ffb5c1423f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40922/0 (socket says 172.21.15.81:40922) 2023-11-08T16:30:37.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.651+0000 7ffb60973640 1 -- 172.21.15.81:0/1833181870 learned_addr learned my addr 172.21.15.81:0/1833181870 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.652+0000 7ffb60973640 1 -- 172.21.15.81:0/1833181870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15b8b0 msgr2=0x7ffb5c151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.652+0000 7ffb60973640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15b8b0 0x7ffb5c151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.652+0000 7ffb60973640 1 -- 172.21.15.81:0/1833181870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb5c152310 msgr2=0x7ffb5c1f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.652+0000 7ffb60973640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb5c152310 0x7ffb5c1f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.652+0000 7ffb60973640 1 -- 172.21.15.81:0/1833181870 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c15c350 con 0x7ffb4c003a50 2023-11-08T16:30:37.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.652+0000 7ffb60973640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 0x7ffb5c1423f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ffb5002beb0 tx=0x7ffb50004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:37.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.652+0000 7ffb597fa640 1 -- 172.21.15.81:0/1833181870 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50004220 con 0x7ffb4c003a50 2023-11-08T16:30:37.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.652+0000 7ffb597fa640 1 -- 172.21.15.81:0/1833181870 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb50004380 con 0x7ffb4c003a50 2023-11-08T16:30:37.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.652+0000 7ffb61975640 1 -- 172.21.15.81:0/1833181870 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5c15c630 con 0x7ffb4c003a50 2023-11-08T16:30:37.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.652+0000 7ffb597fa640 1 -- 172.21.15.81:0/1833181870 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5003ba10 con 0x7ffb4c003a50 2023-11-08T16:30:37.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.652+0000 7ffb61975640 1 -- 172.21.15.81:0/1833181870 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5c15bd10 con 0x7ffb4c003a50 2023-11-08T16:30:37.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.653+0000 7ffb61975640 1 -- 172.21.15.81:0/1833181870 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5c15cb50 con 0x7ffb4c003a50 2023-11-08T16:30:37.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.654+0000 7ffb597fa640 1 -- 172.21.15.81:0/1833181870 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffb5003bb70 con 0x7ffb4c003a50 2023-11-08T16:30:37.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.655+0000 7ffb597fa640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffb4006aea0 0x7ffb4006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:37.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.655+0000 7ffb597fa640 1 -- 172.21.15.81:0/1833181870 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb500abed0 con 0x7ffb4c003a50 2023-11-08T16:30:37.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.655+0000 7ffb5bfff640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffb4006aea0 0x7ffb4006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:37.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.656+0000 7ffb5bfff640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffb4006aea0 0x7ffb4006d360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb48000c00 tx=0x7ffb48001b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:37.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.660+0000 7ffb597fa640 1 -- 172.21.15.81:0/1833181870 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffb5c15cb50 con 0x7ffb4c003a50 2023-11-08T16:30:37.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.889+0000 7ffb61975640 1 -- 172.21.15.81:0/1833181870 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffb5c15cb50 con 0x7ffb4c003a50 2023-11-08T16:30:37.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.890+0000 7ffb597fa640 1 -- 172.21.15.81:0/1833181870 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v18) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffb5c15cb50 con 0x7ffb4c003a50 2023-11-08T16:30:37.892 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T16:30:37.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.893+0000 7ffb61975640 1 -- 172.21.15.81:0/1833181870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffb4006aea0 msgr2=0x7ffb4006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.893+0000 7ffb61975640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffb4006aea0 0x7ffb4006d360 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ffb48000c00 tx=0x7ffb48001b70 comp rx=0 tx=0).stop 2023-11-08T16:30:37.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.893+0000 7ffb61975640 1 -- 172.21.15.81:0/1833181870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 msgr2=0x7ffb5c1423f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:37.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.893+0000 7ffb61975640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 0x7ffb5c1423f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ffb5002beb0 tx=0x7ffb50004730 comp rx=0 tx=0).stop 2023-11-08T16:30:37.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.893+0000 7ffb61975640 1 -- 172.21.15.81:0/1833181870 shutdown_connections 2023-11-08T16:30:37.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.893+0000 7ffb61975640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffb4006aea0 0x7ffb4006d360 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.893+0000 7ffb61975640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffb5c152310 0x7ffb5c1f0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.893+0000 7ffb61975640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffb5c15b8b0 0x7ffb5c151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.893+0000 7ffb61975640 1 --2- 172.21.15.81:0/1833181870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffb4c003a50 0x7ffb5c1423f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:37.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.893+0000 7ffb61975640 1 -- 172.21.15.81:0/1833181870 >> 172.21.15.81:0/1833181870 conn(0x7ffb5c06a4a0 msgr2=0x7ffb5c0af1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:37.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.894+0000 7ffb61975640 1 -- 172.21.15.81:0/1833181870 shutdown_connections 2023-11-08T16:30:37.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:37.894+0000 7ffb61975640 1 -- 172.21.15.81:0/1833181870 wait complete. 2023-11-08T16:30:37.913 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:30:37.913 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:30:37.914 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:30:37.914 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:30:37.914 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:30:37.915 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:30:37.915 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:30:37.915 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:30:37.916 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:30:37.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:37.916 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:30:37.967 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:30:37.967 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:30:37.967 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:30:37.967 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:30:37.968 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:30:37.968 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:30:37.968 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:30:37.968 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:30:37.968 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:30:37.969 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:30:37.969 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:30:37.969 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:30:37.969 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:30:37.969 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:30:37.970 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:30:37.970 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:30:37.970 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:30:37.970 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42272sec preferred_lft 42272sec 2023-11-08T16:30:37.970 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:30:37.971 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:30:37.971 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:30:37.971 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff 2023-11-08T16:30:37.971 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:30:37.972 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:30:37.972 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:30:37.972 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:30:37.972 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:30:37.972 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:30:37.973 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:30:37.973 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:30:37.973 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:30:37.973 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:30:37.973 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::2cac:17ff:fe64:67da/64 scope link 2023-11-08T16:30:37.973 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:30:37.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:37.974 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:30:38.100 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:30:38.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:38.105 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:30:38.158 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:30:38.159 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:30:38.159 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:30:38.159 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:30:38.159 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:30:38.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:38.161 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:30:38.161 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:30:38.162 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:30:38.162 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:30:38.162 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:30:38.163 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:30:38.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:38.348 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:30:38.399 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:30:38.399 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:30:38.400 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:30:38.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:38.401 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:30:38.500 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:30:38.501 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:30:38.501 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:30:38.555 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:30:38.555 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:30:38.556 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:30:38.608 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:30:38.704 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:30:38.761 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:30:38.762 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:30:38.762 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:30:38.762 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:30:38.762 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:30:38.762 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:30:38.763 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:30:38.763 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:30:38.763 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:30:38.763 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:30:38.763 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:30:38.764 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:30:38.764 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:30:38.764 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:30:38.764 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:30:38.764 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:30:38.765 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:30:38.765 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:30:38.765 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:30:38.765 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:30:38.765 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:30:38.765 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:30:38.766 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:30:38.766 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:30:38.766 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:30:38.766 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:30:38.766 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:30:38.767 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:30:38.767 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:30:38.767 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:30:38.767 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:30:38.767 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:30:38.768 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:30:38.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:38.769 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:30:38.819 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:30:38.819 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:30:38.826 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:30:38.899 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:30:38.943 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:30:38.943 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:30:38.944 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:30:38.944 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:30:38.944 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:30:38.944 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:30:38.944 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:30:38.945 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:30:38.945 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:30:38.946 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:30:38.946 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:30:38.946 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:30:38.947 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:30:38.947 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:30:38.947 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:30:38.947 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:30:38.947 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:30:38.948 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:30:38.948 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:30:38.948 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:30:38.948 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:30:38.948 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:30:38.949 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:30:38.949 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:30:38.949 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:30:38.949 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:30:38.949 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:30:38.950 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:30:38.950 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:30:38.950 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:30:38.950 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:30:38.950 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:30:38.951 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:30:38.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:38.952 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:30:39.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:30:39.012+0000 7f4dc3234480 -1 init, newargv = 0x55c6cfcd5b60 newargc=15 2023-11-08T16:30:39.014 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[101122]: starting ceph client 2023-11-08T16:30:39.021 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[101122]: starting fuse 2023-11-08T16:30:39.041 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:30:39.042 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:30:39.042 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:30:39.043 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:30:39.043 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:30:39.043 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:30:39.044 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:30:39.044 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:30:39.044 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:30:39.045 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:30:39.045 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:30:39.045 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:30:39.046 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:30:39.046 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:30:39.047 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:30:39.047 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:30:39.047 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:30:39.048 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:30:39.048 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:30:39.048 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:30:39.049 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:30:39.049 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:30:39.049 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:30:39.050 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:30:39.050 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:30:39.051 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:30:39.051 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:30:39.051 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:30:39.052 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:30:39.247 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.101122.asok 2023-11-08T16:30:39.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:30:39.264 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.101122.asok 2023-11-08T16:30:39.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:39.265 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.101122.asok status 2023-11-08T16:30:39.503 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:30:39.503 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:30:39.503 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:30:39.503 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:30:39.504 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:30:39.504 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:30:39.504 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:30:39.504 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "101122", 2023-11-08T16:30:39.504 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:30:39.505 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:39.505 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:30:39.505 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:30:39.505 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 5322, 2023-11-08T16:30:39.505 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:30:39.506 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:30:39.506 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:30:39.506 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 5322 2023-11-08T16:30:39.506 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:39.506 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:30:39.507 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:30:39.507 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:30:39.507 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2593250366 2023-11-08T16:30:39.507 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:30:39.507 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:39.508 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:30:39.508 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:30:39.508 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:30:39.508 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2593250366 2023-11-08T16:30:39.508 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:30:39.509 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.5322 192.168.0.1:0/2593250366", 2023-11-08T16:30:39.509 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/2593250366", 2023-11-08T16:30:39.509 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:30:39.509 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 18, 2023-11-08T16:30:39.510 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 35, 2023-11-08T16:30:39.510 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:30:39.510 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:30:39.511 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:30:39.511 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:30:39.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:39.518 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:30:39.518 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:30:39.572 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:30:39.573 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:30:39.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:39.574 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:30:39.662 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-11-08T16:30:39.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.882+0000 7f3090e64640 1 Processor -- start 2023-11-08T16:30:39.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.882+0000 7f3090e64640 1 -- start start 2023-11-08T16:30:39.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.884+0000 7f3090e64640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f308c071890 0x7f308c15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:39.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.884+0000 7f3090e64640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f308c15afd0 0x7f308c15ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:39.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.884+0000 7f3090e64640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f308c0b09f0 con 0x7f308c0714b0 2023-11-08T16:30:39.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.884+0000 7f3090e64640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308c06b250 con 0x7f308c071890 2023-11-08T16:30:39.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.884+0000 7f3090e64640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0ca360 con 0x7f308c15afd0 2023-11-08T16:30:39.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.884+0000 7f3089d9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f308c071890 0x7f308c15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:39.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.884+0000 7f308a59c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f308c0714b0 0x7f308c15a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53732/0 (socket says 172.21.15.81:53732) 2023-11-08T16:30:39.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.884+0000 7f308a59c640 1 -- 172.21.15.81:0/1267230720 learned_addr learned my addr 172.21.15.81:0/1267230720 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:39.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.885+0000 7f308ad9d640 1 --2- 172.21.15.81:0/1267230720 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f308c15afd0 0x7f308c15ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:39.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.885+0000 7f3089d9b640 1 -- 172.21.15.81:0/1267230720 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f308c15afd0 msgr2=0x7f308c15ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:39.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.885+0000 7f3089d9b640 1 --2- 172.21.15.81:0/1267230720 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f308c15afd0 0x7f308c15ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:39.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.885+0000 7f3089d9b640 1 -- 172.21.15.81:0/1267230720 >> v1:172.21.15.81:6789/0 conn(0x7f308c0714b0 legacy=0x7f308c15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:39.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.885+0000 7f3089d9b640 1 -- 172.21.15.81:0/1267230720 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308c0ca640 con 0x7f308c071890 2023-11-08T16:30:39.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.885+0000 7f308959a640 1 -- 172.21.15.81:0/1267230720 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765863613 0 0) 0x7f308c0b09f0 con 0x7f308c0714b0 2023-11-08T16:30:39.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.886+0000 7f3089d9b640 1 --2- 172.21.15.81:0/1267230720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308c071890 0x7f308c15e9d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f30740097a0 tx=0x7f307402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d39940ac99a887 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:39.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.886+0000 7f308959a640 1 -- 172.21.15.81:0/1267230720 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307402d040 con 0x7f308c071890 2023-11-08T16:30:39.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.886+0000 7f308959a640 1 -- 172.21.15.81:0/1267230720 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3074031810 con 0x7f308c071890 2023-11-08T16:30:39.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.886+0000 7f308959a640 1 -- 172.21.15.81:0/1267230720 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3074031ba0 con 0x7f308c071890 2023-11-08T16:30:39.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.887+0000 7f3090e64640 1 -- 172.21.15.81:0/1267230720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308c071890 msgr2=0x7f308c15e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:39.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.887+0000 7f3090e64640 1 --2- 172.21.15.81:0/1267230720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308c071890 0x7f308c15e9d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f30740097a0 tx=0x7f307402b8b0 comp rx=0 tx=0).stop 2023-11-08T16:30:39.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.887+0000 7f3090e64640 1 -- 172.21.15.81:0/1267230720 shutdown_connections 2023-11-08T16:30:39.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.887+0000 7f3090e64640 1 --2- 172.21.15.81:0/1267230720 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f308c15afd0 0x7f308c15ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:39.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.887+0000 7f3090e64640 1 --2- 172.21.15.81:0/1267230720 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308c071890 0x7f308c15e9d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:39.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.887+0000 7f3090e64640 1 -- 172.21.15.81:0/1267230720 >> 172.21.15.81:0/1267230720 conn(0x7f308c0cb4e0 msgr2=0x7f308c0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:39.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.887+0000 7f3090e64640 1 -- 172.21.15.81:0/1267230720 shutdown_connections 2023-11-08T16:30:39.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.887+0000 7f3090e64640 1 -- 172.21.15.81:0/1267230720 wait complete. 2023-11-08T16:30:39.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.889+0000 7f3090e64640 1 Processor -- start 2023-11-08T16:30:39.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.889+0000 7f3090e64640 1 -- start start 2023-11-08T16:30:39.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.890+0000 7f3090e64640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f308c0714b0 0x7f308c0b26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:39.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.890+0000 7f3090e64640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f308c071890 0x7f308c0c8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:39.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.890+0000 7f3090e64640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308c15afd0 0x7f308c152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:39.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.890+0000 7f3090e64640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0b3970 con 0x7f308c15afd0 2023-11-08T16:30:39.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.890+0000 7f3090e64640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0c8940 con 0x7f308c0714b0 2023-11-08T16:30:39.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.890+0000 7f3090e64640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308c0b2d60 con 0x7f308c071890 2023-11-08T16:30:39.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.890+0000 7f308ad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308c15afd0 0x7f308c152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:39.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.890+0000 7f308ad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308c15afd0 0x7f308c152ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40944/0 (socket says 172.21.15.81:40944) 2023-11-08T16:30:39.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.890+0000 7f308ad9d640 1 -- 172.21.15.81:0/363752374 learned_addr learned my addr 172.21.15.81:0/363752374 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:39.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f308a59c640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f308c0714b0 0x7f308c0b26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:39.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f3089d9b640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f308c071890 0x7f308c0c8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:39.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f308ad9d640 1 -- 172.21.15.81:0/363752374 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f308c071890 msgr2=0x7f308c0c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:39.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f308ad9d640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f308c071890 0x7f308c0c8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:39.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f308ad9d640 1 -- 172.21.15.81:0/363752374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f308c0714b0 msgr2=0x7f308c0b26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:39.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f308ad9d640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f308c0714b0 0x7f308c0b26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:39.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f308ad9d640 1 -- 172.21.15.81:0/363752374 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3074009400 con 0x7f308c15afd0 2023-11-08T16:30:39.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f308a59c640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f308c0714b0 0x7f308c0b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:39.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f308ad9d640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308c15afd0 0x7f308c152ed0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f308400c550 tx=0x7f308400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:39.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f3082ffd640 1 -- 172.21.15.81:0/363752374 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084004220 con 0x7f308c15afd0 2023-11-08T16:30:39.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f3082ffd640 1 -- 172.21.15.81:0/363752374 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3084004380 con 0x7f308c15afd0 2023-11-08T16:30:39.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f3082ffd640 1 -- 172.21.15.81:0/363752374 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3084003830 con 0x7f308c15afd0 2023-11-08T16:30:39.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f3090e64640 1 -- 172.21.15.81:0/363752374 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f308c0b3040 con 0x7f308c15afd0 2023-11-08T16:30:39.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.891+0000 7f3090e64640 1 -- 172.21.15.81:0/363752374 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f308c0b3600 con 0x7f308c15afd0 2023-11-08T16:30:39.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.892+0000 7f3090e64640 1 -- 172.21.15.81:0/363752374 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f308c0670f0 con 0x7f308c15afd0 2023-11-08T16:30:39.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.893+0000 7f3082ffd640 1 -- 172.21.15.81:0/363752374 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3084003990 con 0x7f308c15afd0 2023-11-08T16:30:39.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.895+0000 7f3082ffd640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f306c06ae70 0x7f306c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:39.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.895+0000 7f3082ffd640 1 -- 172.21.15.81:0/363752374 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30840892e0 con 0x7f308c15afd0 2023-11-08T16:30:39.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.895+0000 7f308a59c640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f306c06ae70 0x7f306c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:39.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.896+0000 7f308a59c640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f306c06ae70 0x7f306c06d330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f307c0012e0 tx=0x7f307c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:39.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:39.900+0000 7f3082ffd640 1 -- 172.21.15.81:0/363752374 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3084054390 con 0x7f308c15afd0 2023-11-08T16:30:40.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.090+0000 7f3090e64640 1 -- 172.21.15.81:0/363752374 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f308c0743f0 con 0x7f308c15afd0 2023-11-08T16:30:40.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.288+0000 7f3082ffd640 1 -- 172.21.15.81:0/363752374 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v36) v1 ==== 142+0+0 (secure 0 0 0) 0x7f30840579b0 con 0x7f308c15afd0 2023-11-08T16:30:40.291 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data_extra' created 2023-11-08T16:30:40.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.293+0000 7f3090e64640 1 -- 172.21.15.81:0/363752374 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f306c06ae70 msgr2=0x7f306c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:40.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.293+0000 7f3090e64640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f306c06ae70 0x7f306c06d330 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f307c0012e0 tx=0x7f307c009040 comp rx=0 tx=0).stop 2023-11-08T16:30:40.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.293+0000 7f3090e64640 1 -- 172.21.15.81:0/363752374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308c15afd0 msgr2=0x7f308c152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:40.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.293+0000 7f3090e64640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308c15afd0 0x7f308c152ed0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f308400c550 tx=0x7f308400ca10 comp rx=0 tx=0).stop 2023-11-08T16:30:40.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.293+0000 7f3090e64640 1 -- 172.21.15.81:0/363752374 shutdown_connections 2023-11-08T16:30:40.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.293+0000 7f3090e64640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f306c06ae70 0x7f306c06d330 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:40.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.293+0000 7f3090e64640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f308c15afd0 0x7f308c152ed0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:40.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.293+0000 7f3090e64640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f308c071890 0x7f308c0c8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:40.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.293+0000 7f3090e64640 1 --2- 172.21.15.81:0/363752374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f308c0714b0 0x7f308c0b26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:40.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.293+0000 7f3090e64640 1 -- 172.21.15.81:0/363752374 >> 172.21.15.81:0/363752374 conn(0x7f308c0cb4e0 msgr2=0x7f308c158990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:40.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.294+0000 7f3090e64640 1 -- 172.21.15.81:0/363752374 shutdown_connections 2023-11-08T16:30:40.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.294+0000 7f3090e64640 1 -- 172.21.15.81:0/363752374 wait complete. 2023-11-08T16:30:40.325 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-11-08T16:30:40.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.528+0000 7f87f1213640 1 Processor -- start 2023-11-08T16:30:40.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.528+0000 7f87f1213640 1 -- start start 2023-11-08T16:30:40.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.529+0000 7f87f1213640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f87ec15d310 0x7f87ec0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:40.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.529+0000 7f87f1213640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec0ce0b0 con 0x7f87ec0cfd90 2023-11-08T16:30:40.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.529+0000 7f87f1213640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ec0ce390 con 0x7f87ec15cf30 2023-11-08T16:30:40.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.529+0000 7f87f1213640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ec0d1be0 con 0x7f87ec15d310 2023-11-08T16:30:40.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.530+0000 7f87e3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f87ec15d310 0x7f87ec0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:40.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.530+0000 7f87e3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f87ec15d310 0x7f87ec0d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40960/0 (socket says 172.21.15.81:40960) 2023-11-08T16:30:40.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.530+0000 7f87e3fff640 1 -- 172.21.15.81:0/3755527017 learned_addr learned my addr 172.21.15.81:0/3755527017 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:40.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.530+0000 7f87f0a12640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f87ec0cfd90 0x7f87ec0b5f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53736/0 (socket says 172.21.15.81:53736) 2023-11-08T16:30:40.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.530+0000 7f87e3fff640 1 -- 172.21.15.81:0/3755527017 >> v1:172.21.15.93:6789/0 conn(0x7f87ec15cf30 legacy=0x7f87ec0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:40.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.530+0000 7f87e3fff640 1 -- 172.21.15.81:0/3755527017 >> v1:172.21.15.81:6789/0 conn(0x7f87ec0cfd90 legacy=0x7f87ec0b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:40.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.530+0000 7f87e3fff640 1 -- 172.21.15.81:0/3755527017 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ec0d1ec0 con 0x7f87ec15d310 2023-11-08T16:30:40.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.531+0000 7f87e3fff640 1 --2- 172.21.15.81:0/3755527017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87ec15d310 0x7f87ec0d26c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f87d40098a0 tx=0x7f87d402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d887cccc95114735 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:40.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.532+0000 7f87eb7fe640 1 -- 172.21.15.81:0/3755527017 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4030840 con 0x7f87ec15d310 2023-11-08T16:30:40.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.532+0000 7f87eb7fe640 1 -- 172.21.15.81:0/3755527017 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d40309a0 con 0x7f87ec15d310 2023-11-08T16:30:40.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.532+0000 7f87eb7fe640 1 -- 172.21.15.81:0/3755527017 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4030d30 con 0x7f87ec15d310 2023-11-08T16:30:40.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.532+0000 7f87f1213640 1 -- 172.21.15.81:0/3755527017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87ec15d310 msgr2=0x7f87ec0d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:40.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.532+0000 7f87f1213640 1 --2- 172.21.15.81:0/3755527017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87ec15d310 0x7f87ec0d26c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f87d40098a0 tx=0x7f87d402b980 comp rx=0 tx=0).stop 2023-11-08T16:30:40.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.532+0000 7f87f1213640 1 -- 172.21.15.81:0/3755527017 shutdown_connections 2023-11-08T16:30:40.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.532+0000 7f87f1213640 1 --2- 172.21.15.81:0/3755527017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87ec15d310 0x7f87ec0d26c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:40.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.532+0000 7f87f1213640 1 -- 172.21.15.81:0/3755527017 >> 172.21.15.81:0/3755527017 conn(0x7f87ec06c040 msgr2=0x7f87ec06c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:40.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.533+0000 7f87f1213640 1 -- 172.21.15.81:0/3755527017 shutdown_connections 2023-11-08T16:30:40.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.533+0000 7f87f1213640 1 -- 172.21.15.81:0/3755527017 wait complete. 2023-11-08T16:30:40.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.534+0000 7f87f1213640 1 Processor -- start 2023-11-08T16:30:40.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.534+0000 7f87f1213640 1 -- start start 2023-11-08T16:30:40.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.535+0000 7f87f1213640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87ec0cfd90 0x7f87ec0caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:40.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.535+0000 7f87f1213640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87ec15cf30 0x7f87ec0b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:40.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.535+0000 7f87f1213640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87ec0cafb0 0x7f87ec0b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:40.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.535+0000 7f87f1213640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0ca650 con 0x7f87ec0cfd90 2023-11-08T16:30:40.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.535+0000 7f87f1213640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0b4920 con 0x7f87ec15cf30 2023-11-08T16:30:40.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.535+0000 7f87f1213640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ec0b4b60 con 0x7f87ec0cafb0 2023-11-08T16:30:40.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.536+0000 7f87ebfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87ec0cfd90 0x7f87ec0caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:40.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.536+0000 7f87ebfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87ec0cfd90 0x7f87ec0caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40976/0 (socket says 172.21.15.81:40976) 2023-11-08T16:30:40.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.536+0000 7f87ebfff640 1 -- 172.21.15.81:0/3061497923 learned_addr learned my addr 172.21.15.81:0/3061497923 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:40.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.536+0000 7f87e3fff640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87ec15cf30 0x7f87ec0b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:40.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.536+0000 7f87f0a12640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87ec0cafb0 0x7f87ec0b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:40.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.536+0000 7f87ebfff640 1 -- 172.21.15.81:0/3061497923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87ec0cafb0 msgr2=0x7f87ec0b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:40.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.536+0000 7f87ebfff640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87ec0cafb0 0x7f87ec0b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:40.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.536+0000 7f87ebfff640 1 -- 172.21.15.81:0/3061497923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87ec15cf30 msgr2=0x7f87ec0b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:40.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.536+0000 7f87ebfff640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87ec15cf30 0x7f87ec0b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:40.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.536+0000 7f87ebfff640 1 -- 172.21.15.81:0/3061497923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d40094d0 con 0x7f87ec0cfd90 2023-11-08T16:30:40.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.536+0000 7f87ebfff640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87ec0cfd90 0x7f87ec0caa70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00c430 tx=0x7f87dc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:40.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.537+0000 7f87e97fa640 1 -- 172.21.15.81:0/3061497923 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc011790 con 0x7f87ec0cfd90 2023-11-08T16:30:40.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.537+0000 7f87e97fa640 1 -- 172.21.15.81:0/3061497923 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc0118f0 con 0x7f87ec0cfd90 2023-11-08T16:30:40.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.537+0000 7f87e97fa640 1 -- 172.21.15.81:0/3061497923 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc011ce0 con 0x7f87ec0cfd90 2023-11-08T16:30:40.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.537+0000 7f87f1213640 1 -- 172.21.15.81:0/3061497923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ec0b4e40 con 0x7f87ec0cfd90 2023-11-08T16:30:40.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.537+0000 7f87f1213640 1 -- 172.21.15.81:0/3061497923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ec0ca270 con 0x7f87ec0cfd90 2023-11-08T16:30:40.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.538+0000 7f87f1213640 1 -- 172.21.15.81:0/3061497923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ec0ca790 con 0x7f87ec0cfd90 2023-11-08T16:30:40.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.539+0000 7f87e97fa640 1 -- 172.21.15.81:0/3061497923 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f87dc01c460 con 0x7f87ec0cfd90 2023-11-08T16:30:40.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.540+0000 7f87e97fa640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87cc06ae70 0x7f87cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:40.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.540+0000 7f87e97fa640 1 -- 172.21.15.81:0/3061497923 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f87dc089670 con 0x7f87ec0cfd90 2023-11-08T16:30:40.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.540+0000 7f87e3fff640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87cc06ae70 0x7f87cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:40.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.541+0000 7f87e3fff640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87cc06ae70 0x7f87cc06d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f87d4036000 tx=0x7f87d403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:40.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.545+0000 7f87e97fa640 1 -- 172.21.15.81:0/3061497923 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f87ec0ca790 con 0x7f87ec0cfd90 2023-11-08T16:30:40.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:40.772+0000 7f87f1213640 1 -- 172.21.15.81:0/3061497923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f87ec0ca790 con 0x7f87ec0cfd90 2023-11-08T16:30:41.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.310+0000 7f87e97fa640 1 -- 172.21.15.81:0/3061497923 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 6 to fsmap v19) v1 ==== 140+0+0 (secure 0 0 0) 0x7f87ec0ca790 con 0x7f87ec0cfd90 2023-11-08T16:30:41.317 INFO:teuthology.orchestra.run.smithi081.stderr:added data pool 6 to fsmap 2023-11-08T16:30:41.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.318+0000 7f87f1213640 1 -- 172.21.15.81:0/3061497923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87cc06ae70 msgr2=0x7f87cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:41.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.318+0000 7f87f1213640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87cc06ae70 0x7f87cc06d330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f87d4036000 tx=0x7f87d403b040 comp rx=0 tx=0).stop 2023-11-08T16:30:41.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.318+0000 7f87f1213640 1 -- 172.21.15.81:0/3061497923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87ec0cfd90 msgr2=0x7f87ec0caa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:41.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.319+0000 7f87f1213640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87ec0cfd90 0x7f87ec0caa70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f87dc00c430 tx=0x7f87dc00c8f0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.319+0000 7f87f1213640 1 -- 172.21.15.81:0/3061497923 shutdown_connections 2023-11-08T16:30:41.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.319+0000 7f87f1213640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f87cc06ae70 0x7f87cc06d330 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.319+0000 7f87f1213640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f87ec0cafb0 0x7f87ec0b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.319+0000 7f87f1213640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87ec15cf30 0x7f87ec0b3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.319+0000 7f87f1213640 1 --2- 172.21.15.81:0/3061497923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f87ec0cfd90 0x7f87ec0caa70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.319+0000 7f87f1213640 1 -- 172.21.15.81:0/3061497923 >> 172.21.15.81:0/3061497923 conn(0x7f87ec06c040 msgr2=0x7f87ec15ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.319+0000 7f87f1213640 1 -- 172.21.15.81:0/3061497923 shutdown_connections 2023-11-08T16:30:41.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.319+0000 7f87f1213640 1 -- 172.21.15.81:0/3061497923 wait complete. 2023-11-08T16:30:41.345 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:30:41.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.528+0000 7f5a2c81a640 1 Processor -- start 2023-11-08T16:30:41.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.529+0000 7f5a2c81a640 1 -- start start 2023-11-08T16:30:41.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.530+0000 7f5a2c81a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5a280b0070 0x7f5a2806d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:41.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.530+0000 7f5a2c81a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a280cd5c0 0x7f5a280cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:41.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.530+0000 7f5a2c81a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a280cdf60 con 0x7f5a280cffd0 2023-11-08T16:30:41.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.530+0000 7f5a2c81a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a280cbee0 con 0x7f5a280b0070 2023-11-08T16:30:41.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.530+0000 7f5a2c81a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a280b0450 con 0x7f5a280cd5c0 2023-11-08T16:30:41.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.530+0000 7f5a26d9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5a280cffd0 0x7f5a280d0460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51076/0 (socket says 172.21.15.81:51076) 2023-11-08T16:30:41.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.530+0000 7f5a26d9d640 1 -- 172.21.15.81:0/2725972956 learned_addr learned my addr 172.21.15.81:0/2725972956 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:30:41.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.530+0000 7f5a2659c640 1 --2- 172.21.15.81:0/2725972956 >> v2:172.21.15.93:3300/0 conn(0x7f5a280b0070 0x7f5a2806d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:41.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.530+0000 7f5a25d9b640 1 --2- 172.21.15.81:0/2725972956 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a280cd5c0 0x7f5a280cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:41.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.530+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202196304 0 0) 0x7f5a280cdf60 con 0x7f5a280cffd0 2023-11-08T16:30:41.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a14003520 con 0x7f5a280cffd0 2023-11-08T16:30:41.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3107264838 0 0) 0x7f5a14003520 con 0x7f5a280cffd0 2023-11-08T16:30:41.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a280cd5c0 msgr2=0x7f5a280cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:41.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 --2- 172.21.15.81:0/2725972956 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a280cd5c0 0x7f5a280cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 >> v2:172.21.15.93:3300/0 conn(0x7f5a280b0070 msgr2=0x7f5a2806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:41.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 --2- 172.21.15.81:0/2725972956 >> v2:172.21.15.93:3300/0 conn(0x7f5a280b0070 0x7f5a2806d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a280cb2d0 con 0x7f5a280cffd0 2023-11-08T16:30:41.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f5a20001670 con 0x7f5a280cffd0 2023-11-08T16:30:41.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 >> v1:172.21.15.81:6789/0 conn(0x7f5a280cffd0 legacy=0x7f5a280d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:41.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 --2- 172.21.15.81:0/2725972956 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a280cd5c0 0x7f5a14003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:41.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 --2- 172.21.15.81:0/2725972956 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a14003fd0 0x7f5a140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:41.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 --2- 172.21.15.81:0/2725972956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a14004d00 0x7f5a14005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:41.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a280cbee0 con 0x7f5a14004d00 2023-11-08T16:30:41.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a280b0450 con 0x7f5a14003fd0 2023-11-08T16:30:41.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1400ba00 con 0x7f5a280cd5c0 2023-11-08T16:30:41.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.531+0000 7f5a26d9d640 1 --2- 172.21.15.81:0/2725972956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a14004d00 0x7f5a14005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:41.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.532+0000 7f5a26d9d640 1 -- 172.21.15.81:0/2725972956 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a280cd5c0 msgr2=0x7f5a14003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:41.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.532+0000 7f5a26d9d640 1 --2- 172.21.15.81:0/2725972956 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a280cd5c0 0x7f5a14003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.532+0000 7f5a26d9d640 1 -- 172.21.15.81:0/2725972956 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a14003fd0 msgr2=0x7f5a140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:41.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.532+0000 7f5a26d9d640 1 --2- 172.21.15.81:0/2725972956 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a14003fd0 0x7f5a140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.532+0000 7f5a26d9d640 1 -- 172.21.15.81:0/2725972956 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a14003520 con 0x7f5a14004d00 2023-11-08T16:30:41.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.533+0000 7f5a26d9d640 1 --2- 172.21.15.81:0/2725972956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a14004d00 0x7f5a14005180 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c290 tx=0x7f5a140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0ab2133e3a954d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:41.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.533+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a20005ad0 con 0x7f5a14004d00 2023-11-08T16:30:41.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.533+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a20005c30 con 0x7f5a14004d00 2023-11-08T16:30:41.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.533+0000 7f5a2559a640 1 -- 172.21.15.81:0/2725972956 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2002c8d0 con 0x7f5a14004d00 2023-11-08T16:30:41.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.534+0000 7f5a2c81a640 1 -- 172.21.15.81:0/2725972956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a14004d00 msgr2=0x7f5a14005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:41.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.534+0000 7f5a2c81a640 1 --2- 172.21.15.81:0/2725972956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a14004d00 0x7f5a14005180 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c290 tx=0x7f5a140057f0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.534+0000 7f5a26d9d640 1 -- 172.21.15.81:0/2725972956 reap_dead start 2023-11-08T16:30:41.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.534+0000 7f5a2c81a640 1 -- 172.21.15.81:0/2725972956 shutdown_connections 2023-11-08T16:30:41.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.534+0000 7f5a2c81a640 1 -- 172.21.15.81:0/2725972956 >> 172.21.15.81:0/2725972956 conn(0x7f5a280d0be0 msgr2=0x7f5a280d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:41.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.534+0000 7f5a2c81a640 1 -- 172.21.15.81:0/2725972956 shutdown_connections 2023-11-08T16:30:41.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.534+0000 7f5a2c81a640 1 -- 172.21.15.81:0/2725972956 wait complete. 2023-11-08T16:30:41.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.536+0000 7f5a2c81a640 1 Processor -- start 2023-11-08T16:30:41.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.536+0000 7f5a2c81a640 1 -- start start 2023-11-08T16:30:41.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a2c81a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a14004d00 0x7f5a281f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:41.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a2c81a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a280b47d0 0x7f5a280b4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:41.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a2c81a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a28157f10 0x7f5a281583c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:41.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a2c81a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281573b0 con 0x7f5a28157f10 2023-11-08T16:30:41.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a2c81a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a280b5190 con 0x7f5a280b47d0 2023-11-08T16:30:41.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a2c81a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a281f60f0 con 0x7f5a14004d00 2023-11-08T16:30:41.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a26d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a28157f10 0x7f5a281583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:41.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a26d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a28157f10 0x7f5a281583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40908/0 (socket says 172.21.15.81:40908) 2023-11-08T16:30:41.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a26d9d640 1 -- 172.21.15.81:0/289593819 learned_addr learned my addr 172.21.15.81:0/289593819 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:41.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a2659c640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a14004d00 0x7f5a281f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:41.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a26d9d640 1 -- 172.21.15.81:0/289593819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a14004d00 msgr2=0x7f5a281f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:41.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.537+0000 7f5a26d9d640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a14004d00 0x7f5a281f5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.538+0000 7f5a26d9d640 1 -- 172.21.15.81:0/289593819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a280b47d0 msgr2=0x7f5a280b4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:41.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.538+0000 7f5a26d9d640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a280b47d0 0x7f5a280b4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.538+0000 7f5a26d9d640 1 -- 172.21.15.81:0/289593819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a281597a0 con 0x7f5a28157f10 2023-11-08T16:30:41.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.538+0000 7f5a26d9d640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a28157f10 0x7f5a281583c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c290 tx=0x7f5a20001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:41.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.538+0000 7f5a1effd640 1 -- 172.21.15.81:0/289593819 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2003c040 con 0x7f5a28157f10 2023-11-08T16:30:41.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.538+0000 7f5a1effd640 1 -- 172.21.15.81:0/289593819 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a20001c80 con 0x7f5a28157f10 2023-11-08T16:30:41.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.538+0000 7f5a1effd640 1 -- 172.21.15.81:0/289593819 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a200074a0 con 0x7f5a28157f10 2023-11-08T16:30:41.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.538+0000 7f5a2c81a640 1 -- 172.21.15.81:0/289593819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a28158900 con 0x7f5a28157f10 2023-11-08T16:30:41.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.538+0000 7f5a2c81a640 1 -- 172.21.15.81:0/289593819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a28158e70 con 0x7f5a28157f10 2023-11-08T16:30:41.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.539+0000 7f5a2c81a640 1 -- 172.21.15.81:0/289593819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a280cf370 con 0x7f5a28157f10 2023-11-08T16:30:41.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.540+0000 7f5a1effd640 1 -- 172.21.15.81:0/289593819 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5a20032070 con 0x7f5a28157f10 2023-11-08T16:30:41.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.541+0000 7f5a1effd640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5a0806aea0 0x7f5a0806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:41.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.541+0000 7f5a1effd640 1 -- 172.21.15.81:0/289593819 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f5a200aab40 con 0x7f5a28157f10 2023-11-08T16:30:41.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.542+0000 7f5a2659c640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5a0806aea0 0x7f5a0806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:41.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.542+0000 7f5a2659c640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5a0806aea0 0x7f5a0806d360 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a18004b30 tx=0x7f5a1800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:41.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.546+0000 7f5a1effd640 1 -- 172.21.15.81:0/289593819 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5a200794d0 con 0x7f5a28157f10 2023-11-08T16:30:41.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.770+0000 7f5a2c81a640 1 -- 172.21.15.81:0/289593819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a280cb860 con 0x7f5a28157f10 2023-11-08T16:30:41.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.771+0000 7f5a1effd640 1 -- 172.21.15.81:0/289593819 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4909 (secure 0 0 0) 0x7f5a20047020 con 0x7f5a28157f10 2023-11-08T16:30:41.773 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:41.773 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/1453041994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":1453041994},{"type":"v1","addr":"172.21.15.93:6835","nonce":1453041994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/757624701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":757624701},{"type":"v1","addr":"172.21.15.93:6833","nonce":757624701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1735232404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1735232404},{"type":"v1","addr":"172.21.15.81:6835","nonce":1735232404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:30:30.141674+0000","modified":"2023-11-08T16:30:40.774892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5139":{"gid":5139,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/773130363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":773130363},{"type":"v1","addr":"172.21.15.81:6837","nonce":773130363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:30:41.774 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 19 2023-11-08T16:30:41.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.773+0000 7f5a2c81a640 1 -- 172.21.15.81:0/289593819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5a0806aea0 msgr2=0x7f5a0806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:41.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.773+0000 7f5a2c81a640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5a0806aea0 0x7f5a0806d360 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5a18004b30 tx=0x7f5a1800b040 comp rx=0 tx=0).stop 2023-11-08T16:30:41.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.773+0000 7f5a2c81a640 1 -- 172.21.15.81:0/289593819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a28157f10 msgr2=0x7f5a281583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:41.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.773+0000 7f5a2c81a640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a28157f10 0x7f5a281583c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000c290 tx=0x7f5a20001a00 comp rx=0 tx=0).stop 2023-11-08T16:30:41.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.774+0000 7f5a2c81a640 1 -- 172.21.15.81:0/289593819 shutdown_connections 2023-11-08T16:30:41.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.774+0000 7f5a2c81a640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5a0806aea0 0x7f5a0806d360 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.774+0000 7f5a2c81a640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5a28157f10 0x7f5a281583c0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.774+0000 7f5a2c81a640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5a280b47d0 0x7f5a280b4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.774+0000 7f5a2c81a640 1 --2- 172.21.15.81:0/289593819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5a14004d00 0x7f5a281f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:41.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.774+0000 7f5a2c81a640 1 -- 172.21.15.81:0/289593819 >> 172.21.15.81:0/289593819 conn(0x7f5a280d0be0 msgr2=0x7f5a280aef90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:41.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.774+0000 7f5a2c81a640 1 -- 172.21.15.81:0/289593819 shutdown_connections 2023-11-08T16:30:41.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:41.774+0000 7f5a2c81a640 1 -- 172.21.15.81:0/289593819 wait complete. 2023-11-08T16:30:41.793 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:30:42.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.003+0000 7f63d2b63640 1 Processor -- start 2023-11-08T16:30:42.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.003+0000 7f63d2b63640 1 -- start start 2023-11-08T16:30:42.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.004+0000 7f63d2b63640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f63cc150b80 0x7f63cc15a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:42.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.004+0000 7f63d2b63640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63cc0ced10 0x7f63cc0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:42.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.004+0000 7f63d2b63640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63cc0b2fe0 con 0x7f63cc070760 2023-11-08T16:30:42.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.004+0000 7f63d2b63640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc0cf6c0 con 0x7f63cc0ced10 2023-11-08T16:30:42.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.004+0000 7f63d2b63640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63cc0b32c0 con 0x7f63cc150b80 2023-11-08T16:30:42.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.004+0000 7f63d1360640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63cc0ced10 0x7f63cc0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:42.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.004+0000 7f63d1360640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63cc0ced10 0x7f63cc0cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40910/0 (socket says 172.21.15.81:40910) 2023-11-08T16:30:42.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.005+0000 7f63d1360640 1 -- 172.21.15.81:0/1768130831 learned_addr learned my addr 172.21.15.81:0/1768130831 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:42.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.005+0000 7f63d2362640 1 --1- 172.21.15.81:0/1768130831 >> v1:172.21.15.81:6789/0 conn(0x7f63cc070760 0x7f63cc070b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51084/0 (socket says 172.21.15.81:51084) 2023-11-08T16:30:42.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.005+0000 7f63d1b61640 1 --2- 172.21.15.81:0/1768130831 >> v2:172.21.15.93:3300/0 conn(0x7f63cc150b80 0x7f63cc15a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:42.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.005+0000 7f63d1360640 1 -- 172.21.15.81:0/1768130831 >> v2:172.21.15.93:3300/0 conn(0x7f63cc150b80 msgr2=0x7f63cc15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:30:42.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.005+0000 7f63d1360640 1 --2- 172.21.15.81:0/1768130831 >> v2:172.21.15.93:3300/0 conn(0x7f63cc150b80 0x7f63cc15a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:42.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.005+0000 7f63d1360640 1 -- 172.21.15.81:0/1768130831 >> v1:172.21.15.81:6789/0 conn(0x7f63cc070760 legacy=0x7f63cc070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:42.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.005+0000 7f63d1360640 1 -- 172.21.15.81:0/1768130831 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63cc0b35a0 con 0x7f63cc0ced10 2023-11-08T16:30:42.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.005+0000 7f63d0b5f640 1 -- 172.21.15.81:0/1768130831 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2833671058 0 0) 0x7f63cc0b2fe0 con 0x7f63cc070760 2023-11-08T16:30:42.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.005+0000 7f63d1b61640 1 --2- 172.21.15.81:0/1768130831 >> v2:172.21.15.93:3300/0 conn(0x7f63cc150b80 0x7f63cc15a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:30:42.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.006+0000 7f63d1360640 1 --2- 172.21.15.81:0/1768130831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc0ced10 0x7f63cc0cf180 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0cf8d0 tx=0x7f63bc02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82f63695d94310e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:42.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.006+0000 7f63d0b5f640 1 -- 172.21.15.81:0/1768130831 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc030870 con 0x7f63cc0ced10 2023-11-08T16:30:42.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.006+0000 7f63d0b5f640 1 -- 172.21.15.81:0/1768130831 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63bc0309d0 con 0x7f63cc0ced10 2023-11-08T16:30:42.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.006+0000 7f63d0b5f640 1 -- 172.21.15.81:0/1768130831 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63bc030d60 con 0x7f63cc0ced10 2023-11-08T16:30:42.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.007+0000 7f63d2b63640 1 -- 172.21.15.81:0/1768130831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc0ced10 msgr2=0x7f63cc0cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:42.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.007+0000 7f63d2b63640 1 --2- 172.21.15.81:0/1768130831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc0ced10 0x7f63cc0cf180 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f63cc0cf8d0 tx=0x7f63bc02b9b0 comp rx=0 tx=0).stop 2023-11-08T16:30:42.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.007+0000 7f63d2b63640 1 -- 172.21.15.81:0/1768130831 shutdown_connections 2023-11-08T16:30:42.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.007+0000 7f63d2b63640 1 --2- 172.21.15.81:0/1768130831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc0ced10 0x7f63cc0cf180 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:42.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.007+0000 7f63d2b63640 1 --2- 172.21.15.81:0/1768130831 >> v2:172.21.15.93:3300/0 conn(0x7f63cc150b80 0x7f63cc15a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:42.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.007+0000 7f63d2b63640 1 -- 172.21.15.81:0/1768130831 >> 172.21.15.81:0/1768130831 conn(0x7f63cc06a4a0 msgr2=0x7f63cc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:42.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.007+0000 7f63d2b63640 1 -- 172.21.15.81:0/1768130831 shutdown_connections 2023-11-08T16:30:42.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.007+0000 7f63d2b63640 1 -- 172.21.15.81:0/1768130831 wait complete. 2023-11-08T16:30:42.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.009+0000 7f63d2b63640 1 Processor -- start 2023-11-08T16:30:42.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.009+0000 7f63d2b63640 1 -- start start 2023-11-08T16:30:42.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.010+0000 7f63d2b63640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc070760 0x7f63cc148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:42.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.010+0000 7f63d2b63640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63cc0ced10 0x7f63cc0cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:42.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.010+0000 7f63d2b63640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63cc150b80 0x7f63cc0af530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:42.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.010+0000 7f63d2b63640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc149290 con 0x7f63cc070760 2023-11-08T16:30:42.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.010+0000 7f63d2b63640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc152660 con 0x7f63cc0ced10 2023-11-08T16:30:42.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.010+0000 7f63d2b63640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63cc0cccc0 con 0x7f63cc150b80 2023-11-08T16:30:42.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.010+0000 7f63d1b61640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc070760 0x7f63cc148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:42.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.010+0000 7f63d1b61640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc070760 0x7f63cc148d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40916/0 (socket says 172.21.15.81:40916) 2023-11-08T16:30:42.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.010+0000 7f63d1b61640 1 -- 172.21.15.81:0/4238941341 learned_addr learned my addr 172.21.15.81:0/4238941341 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:30:42.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63d1360640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63cc0ced10 0x7f63cc0cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:42.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63d2362640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63cc150b80 0x7f63cc0af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:42.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63d1b61640 1 -- 172.21.15.81:0/4238941341 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63cc150b80 msgr2=0x7f63cc0af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:42.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63d1b61640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63cc150b80 0x7f63cc0af530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:42.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63d1b61640 1 -- 172.21.15.81:0/4238941341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63cc0ced10 msgr2=0x7f63cc0cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:42.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63d1b61640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63cc0ced10 0x7f63cc0cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:42.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63d1b61640 1 -- 172.21.15.81:0/4238941341 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63bc0094d0 con 0x7f63cc070760 2023-11-08T16:30:42.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63d1b61640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc070760 0x7f63cc148d50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c7f0 tx=0x7f63c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:42.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63c27fc640 1 -- 172.21.15.81:0/4238941341 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8007820 con 0x7f63cc070760 2023-11-08T16:30:42.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63c27fc640 1 -- 172.21.15.81:0/4238941341 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c8007980 con 0x7f63cc070760 2023-11-08T16:30:42.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63c27fc640 1 -- 172.21.15.81:0/4238941341 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8007cf0 con 0x7f63cc070760 2023-11-08T16:30:42.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.011+0000 7f63d2b63640 1 -- 172.21.15.81:0/4238941341 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63cc0cbcb0 con 0x7f63cc070760 2023-11-08T16:30:42.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.012+0000 7f63d2b63640 1 -- 172.21.15.81:0/4238941341 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63cc0cc1e0 con 0x7f63cc070760 2023-11-08T16:30:42.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.013+0000 7f63d2b63640 1 -- 172.21.15.81:0/4238941341 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63cc067800 con 0x7f63cc070760 2023-11-08T16:30:42.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.014+0000 7f63c27fc640 1 -- 172.21.15.81:0/4238941341 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f63c80113f0 con 0x7f63cc070760 2023-11-08T16:30:42.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.015+0000 7f63c27fc640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63b006ae70 0x7f63b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:30:42.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.015+0000 7f63c27fc640 1 -- 172.21.15.81:0/4238941341 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f63c8089b60 con 0x7f63cc070760 2023-11-08T16:30:42.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.016+0000 7f63d1360640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63b006ae70 0x7f63b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:30:42.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.016+0000 7f63d1360640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63b006ae70 0x7f63b006d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f63bc036000 tx=0x7f63bc03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:30:42.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.020+0000 7f63c27fc640 1 -- 172.21.15.81:0/4238941341 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63c80549d0 con 0x7f63cc070760 2023-11-08T16:30:42.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.209+0000 7f63d2b63640 1 -- 172.21.15.81:0/4238941341 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63cc0b2710 con 0x7f63cc070760 2023-11-08T16:30:42.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.211+0000 7f63c27fc640 1 -- 172.21.15.81:0/4238941341 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18310 (secure 0 0 0) 0x7f63c801f090 con 0x7f63cc070760 2023-11-08T16:30:42.214 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:30:42.214 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":37,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:30:40.774313+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:30:27.977481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T16:30:29.171902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:30:40.141171+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:30:42.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 -- 172.21.15.81:0/4238941341 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63b006ae70 msgr2=0x7f63b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:42.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63b006ae70 0x7f63b006d330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f63bc036000 tx=0x7f63bc03b040 comp rx=0 tx=0).stop 2023-11-08T16:30:42.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 -- 172.21.15.81:0/4238941341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc070760 msgr2=0x7f63cc148d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:30:42.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc070760 0x7f63cc148d50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f63c800c7f0 tx=0x7f63c800ccb0 comp rx=0 tx=0).stop 2023-11-08T16:30:42.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 -- 172.21.15.81:0/4238941341 shutdown_connections 2023-11-08T16:30:42.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f63b006ae70 0x7f63b006d330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:42.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63cc150b80 0x7f63cc0af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:42.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63cc0ced10 0x7f63cc0cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:42.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 --2- 172.21.15.81:0/4238941341 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63cc070760 0x7f63cc148d50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:30:42.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 -- 172.21.15.81:0/4238941341 >> 172.21.15.81:0/4238941341 conn(0x7f63cc06a4a0 msgr2=0x7f63cc150350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:30:42.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 -- 172.21.15.81:0/4238941341 shutdown_connections 2023-11-08T16:30:42.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:42.214+0000 7f63d2b63640 1 -- 172.21.15.81:0/4238941341 wait complete. 2023-11-08T16:30:42.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:42.234 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-11-08T16:30:42.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:30:42.307 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-11-08T16:30:42.374 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/0 2023-11-08T16:30:42.375 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:30:42.375 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:30:42.376 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-11-08T16:30:42.376 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:30:42.376 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:30:42.377 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:30:42.377 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:30:42.377 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:30:46.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:30:46.193 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/1 2023-11-08T16:30:46.194 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:30:46.195 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:30:46.196 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-11-08T16:30:46.197 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:30:46.197 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:30:46.197 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:30:46.198 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:30:46.198 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:30:46.346 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:30:46.352 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:30:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:30:49.986 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/2 2023-11-08T16:30:49.987 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:30:49.987 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:30:49.988 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-11-08T16:30:49.988 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:30:49.988 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:30:49.989 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:30:49.989 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:30:49.989 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:30:53.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:30:53.768 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/3 2023-11-08T16:30:53.768 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:30:53.769 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:30:53.769 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-11-08T16:30:53.769 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:30:53.770 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:30:53.770 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:30:53.771 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:30:53.771 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:30:57.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:30:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:30:57.562 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/4 2023-11-08T16:30:57.563 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:30:57.563 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:30:57.564 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-11-08T16:30:57.564 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:30:57.565 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:30:57.565 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:30:57.565 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:30:57.566 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:31:01.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:31:01.391 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:31:01.391 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:31:01.487 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:31:01.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.698+0000 7f39dfc6f640 1 Processor -- start 2023-11-08T16:31:01.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.699+0000 7f39dfc6f640 1 -- start start 2023-11-08T16:31:01.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.700+0000 7f39dfc6f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f39d80cf1a0 0x7f39d8065df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:01.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.700+0000 7f39dfc6f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d80b4210 con 0x7f39d815d4c0 2023-11-08T16:31:01.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.700+0000 7f39dfc6f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39d80b44f0 con 0x7f39d80b4fd0 2023-11-08T16:31:01.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.700+0000 7f39dfc6f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39d80664f0 con 0x7f39d80cf1a0 2023-11-08T16:31:01.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.700+0000 7f39dd20a640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f39d815d4c0 0x7f39d80cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45300/0 (socket says 172.21.15.81:45300) 2023-11-08T16:31:01.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.700+0000 7f39dd20a640 1 -- 172.21.15.81:0/3392410030 learned_addr learned my addr 172.21.15.81:0/3392410030 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:01.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dda0b640 1 --2- 172.21.15.81:0/3392410030 >> v2:172.21.15.93:3300/0 conn(0x7f39d80cf1a0 0x7f39d8065df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:01.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1850532298 0 0) 0x7f39d80b4210 con 0x7f39d815d4c0 2023-11-08T16:31:01.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39c8003520 con 0x7f39d815d4c0 2023-11-08T16:31:01.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2754820340 0 0) 0x7f39c8003520 con 0x7f39d815d4c0 2023-11-08T16:31:01.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 >> v2:172.21.15.93:3300/0 conn(0x7f39d80cf1a0 msgr2=0x7f39d8065df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:01.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 --2- 172.21.15.81:0/3392410030 >> v2:172.21.15.93:3300/0 conn(0x7f39d80cf1a0 0x7f39d8065df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:01.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 >> v1:172.21.15.93:6789/0 conn(0x7f39d80b4fd0 legacy=0x7f39d80af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:01.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d80b47d0 con 0x7f39d815d4c0 2023-11-08T16:31:01.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f39c4001670 con 0x7f39d815d4c0 2023-11-08T16:31:01.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 >> v1:172.21.15.81:6789/0 conn(0x7f39d815d4c0 legacy=0x7f39d80cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:01.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39c8003aa0 0x7f39c8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:01.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39c80047c0 0x7f39c8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:01.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39c8005280 0x7f39c800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:01.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80664f0 con 0x7f39c8003aa0 2023-11-08T16:31:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39c800c1c0 con 0x7f39c80047c0 2023-11-08T16:31:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.701+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39c800c380 con 0x7f39c8005280 2023-11-08T16:31:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.702+0000 7f39dda0b640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39c8003aa0 0x7f39c8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.702+0000 7f39dd20a640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39c80047c0 0x7f39c8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.702+0000 7f39de20c640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39c8005280 0x7f39c800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:01.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.702+0000 7f39dda0b640 1 -- 172.21.15.81:0/3392410030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39c8005280 msgr2=0x7f39c800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:01.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.702+0000 7f39dda0b640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39c8005280 0x7f39c800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:01.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.702+0000 7f39dda0b640 1 -- 172.21.15.81:0/3392410030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39c80047c0 msgr2=0x7f39c8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:01.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.702+0000 7f39dda0b640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39c80047c0 0x7f39c8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:01.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.702+0000 7f39dda0b640 1 -- 172.21.15.81:0/3392410030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39c8003520 con 0x7f39c8003aa0 2023-11-08T16:31:01.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.702+0000 7f39de20c640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39c8005280 0x7f39c800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:01.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.702+0000 7f39dd20a640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39c80047c0 0x7f39c8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:01.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.702+0000 7f39de20c640 1 -- 172.21.15.81:0/3392410030 reap_dead start 2023-11-08T16:31:01.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.703+0000 7f39dda0b640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39c8003aa0 0x7f39c8003ea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00e170 tx=0x7f39c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca649804fae1e08a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:01.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.703+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc02cc30 con 0x7f39c8003aa0 2023-11-08T16:31:01.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.703+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39cc00f040 con 0x7f39c8003aa0 2023-11-08T16:31:01.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.703+0000 7f39dca09640 1 -- 172.21.15.81:0/3392410030 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc034440 con 0x7f39c8003aa0 2023-11-08T16:31:01.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.704+0000 7f39dfc6f640 1 -- 172.21.15.81:0/3392410030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39c8003aa0 msgr2=0x7f39c8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:01.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.704+0000 7f39dfc6f640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39c8003aa0 0x7f39c8003ea0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f39cc00e170 tx=0x7f39c80057f0 comp rx=0 tx=0).stop 2023-11-08T16:31:01.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.704+0000 7f39dfc6f640 1 -- 172.21.15.81:0/3392410030 shutdown_connections 2023-11-08T16:31:01.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.704+0000 7f39dfc6f640 1 --2- 172.21.15.81:0/3392410030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39c8003aa0 0x7f39c8003ea0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:01.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.704+0000 7f39dfc6f640 1 -- 172.21.15.81:0/3392410030 >> 172.21.15.81:0/3392410030 conn(0x7f39d8071540 msgr2=0x7f39d8071950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:01.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.704+0000 7f39dfc6f640 1 -- 172.21.15.81:0/3392410030 shutdown_connections 2023-11-08T16:31:01.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.704+0000 7f39dfc6f640 1 -- 172.21.15.81:0/3392410030 wait complete. 2023-11-08T16:31:01.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.706+0000 7f39dfc6f640 1 Processor -- start 2023-11-08T16:31:01.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.706+0000 7f39dfc6f640 1 -- start start 2023-11-08T16:31:01.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.707+0000 7f39dfc6f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39c8003aa0 0x7f39d8156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:01.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.707+0000 7f39dfc6f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39d80ae1b0 0x7f39d80c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:01.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.707+0000 7f39dfc6f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39d80c4270 0x7f39d80c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:01.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.707+0000 7f39dfc6f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d8156c50 con 0x7f39d80c4270 2023-11-08T16:31:01.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.707+0000 7f39dfc6f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80c6610 con 0x7f39c8003aa0 2023-11-08T16:31:01.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.707+0000 7f39dfc6f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39d80c68f0 con 0x7f39d80ae1b0 2023-11-08T16:31:01.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.707+0000 7f39de20c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39d80c4270 0x7f39d80c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:01.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.708+0000 7f39de20c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39d80c4270 0x7f39d80c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42378/0 (socket says 172.21.15.81:42378) 2023-11-08T16:31:01.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.708+0000 7f39de20c640 1 -- 172.21.15.81:0/1848823547 learned_addr learned my addr 172.21.15.81:0/1848823547 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:01.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.708+0000 7f39dda0b640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39c8003aa0 0x7f39d8156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:01.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.708+0000 7f39dd20a640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39d80ae1b0 0x7f39d80c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:01.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.708+0000 7f39de20c640 1 -- 172.21.15.81:0/1848823547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39d80ae1b0 msgr2=0x7f39d80c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:01.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.708+0000 7f39de20c640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39d80ae1b0 0x7f39d80c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:01.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.708+0000 7f39de20c640 1 -- 172.21.15.81:0/1848823547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39c8003aa0 msgr2=0x7f39d8156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:01.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.708+0000 7f39de20c640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39c8003aa0 0x7f39d8156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:01.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.708+0000 7f39de20c640 1 -- 172.21.15.81:0/1848823547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39d80c6bd0 con 0x7f39d80c4270 2023-11-08T16:31:01.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.708+0000 7f39de20c640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39d80c4270 0x7f39d80c73d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f39d8160d60 tx=0x7f39d000d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:01.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.709+0000 7f39d67fc640 1 -- 172.21.15.81:0/1848823547 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d0017070 con 0x7f39d80c4270 2023-11-08T16:31:01.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.709+0000 7f39d67fc640 1 -- 172.21.15.81:0/1848823547 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d0007020 con 0x7f39d80c4270 2023-11-08T16:31:01.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.709+0000 7f39d67fc640 1 -- 172.21.15.81:0/1848823547 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d00073d0 con 0x7f39d80c4270 2023-11-08T16:31:01.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.709+0000 7f39dfc6f640 1 -- 172.21.15.81:0/1848823547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d80c6eb0 con 0x7f39d80c4270 2023-11-08T16:31:01.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.709+0000 7f39dfc6f640 1 -- 172.21.15.81:0/1848823547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d80c7910 con 0x7f39d80c4270 2023-11-08T16:31:01.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.710+0000 7f39dfc6f640 1 -- 172.21.15.81:0/1848823547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d80c70f0 con 0x7f39d80c4270 2023-11-08T16:31:01.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.711+0000 7f39d67fc640 1 -- 172.21.15.81:0/1848823547 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f39d0007530 con 0x7f39d80c4270 2023-11-08T16:31:01.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.712+0000 7f39d67fc640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f39bc06ae70 0x7f39bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:01.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.712+0000 7f39d67fc640 1 -- 172.21.15.81:0/1848823547 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f39d0088d90 con 0x7f39d80c4270 2023-11-08T16:31:01.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.712+0000 7f39dda0b640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f39bc06ae70 0x7f39bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:01.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.713+0000 7f39dda0b640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f39bc06ae70 0x7f39bc06d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f39cc036000 tx=0x7f39cc02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:01.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.717+0000 7f39d67fc640 1 -- 172.21.15.81:0/1848823547 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f39d80c70f0 con 0x7f39d80c4270 2023-11-08T16:31:01.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.909+0000 7f39dfc6f640 1 -- 172.21.15.81:0/1848823547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f39d80c70a0 con 0x7f39d80c4270 2023-11-08T16:31:01.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.912+0000 7f39d67fc640 1 -- 172.21.15.81:0/1848823547 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18297 (secure 0 0 0) 0x7f39d0057750 con 0x7f39d80c4270 2023-11-08T16:31:01.914 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:01.914 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":38,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:30:42.310793+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:30:27.977481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T16:30:29.171902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:30:40.141171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:31:01.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.914+0000 7f39dfc6f640 1 -- 172.21.15.81:0/1848823547 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f39bc06ae70 msgr2=0x7f39bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:01.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.914+0000 7f39dfc6f640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f39bc06ae70 0x7f39bc06d330 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f39cc036000 tx=0x7f39cc02e000 comp rx=0 tx=0).stop 2023-11-08T16:31:01.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.914+0000 7f39dfc6f640 1 -- 172.21.15.81:0/1848823547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39d80c4270 msgr2=0x7f39d80c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:01.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.914+0000 7f39dfc6f640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39d80c4270 0x7f39d80c73d0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f39d8160d60 tx=0x7f39d000d890 comp rx=0 tx=0).stop 2023-11-08T16:31:01.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.915+0000 7f39dfc6f640 1 -- 172.21.15.81:0/1848823547 shutdown_connections 2023-11-08T16:31:01.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.915+0000 7f39dfc6f640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f39bc06ae70 0x7f39bc06d330 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:01.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.915+0000 7f39dfc6f640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f39d80c4270 0x7f39d80c73d0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:01.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.915+0000 7f39dfc6f640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f39d80ae1b0 0x7f39d80c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:01.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.915+0000 7f39dfc6f640 1 --2- 172.21.15.81:0/1848823547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f39c8003aa0 0x7f39d8156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:01.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.915+0000 7f39dfc6f640 1 -- 172.21.15.81:0/1848823547 >> 172.21.15.81:0/1848823547 conn(0x7f39d8071540 msgr2=0x7f39d80ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:01.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.915+0000 7f39dfc6f640 1 -- 172.21.15.81:0/1848823547 shutdown_connections 2023-11-08T16:31:01.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:01.915+0000 7f39dfc6f640 1 -- 172.21.15.81:0/1848823547 wait complete. 2023-11-08T16:31:01.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:30:27.977481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:30:29.171902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:30:40.141171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:31:01.935 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:31:02.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.140+0000 7f26ecd0c640 1 Processor -- start 2023-11-08T16:31:02.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.140+0000 7f26ecd0c640 1 -- start start 2023-11-08T16:31:02.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.141+0000 7f26ecd0c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26e80ccc10 0x7f26e80ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.141+0000 7f26ecd0c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f26e806b710 0x7f26e806bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.141+0000 7f26ecd0c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e80ce960 con 0x7f26e815a0e0 2023-11-08T16:31:02.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.141+0000 7f26ecd0c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e8071980 con 0x7f26e806b710 2023-11-08T16:31:02.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.141+0000 7f26ecd0c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80c9480 con 0x7f26e80ccc10 2023-11-08T16:31:02.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.142+0000 7f26e6d9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f26e806b710 0x7f26e806bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.142+0000 7f26e5d9b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26e80ccc10 0x7f26e80ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.142+0000 7f26e6d9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f26e806b710 0x7f26e806bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:39756/0 (socket says 172.21.15.81:39756) 2023-11-08T16:31:02.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.142+0000 7f26e5d9b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26e80ccc10 0x7f26e80ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:54274/0 (socket says 172.21.15.81:54274) 2023-11-08T16:31:02.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.142+0000 7f26e6d9d640 1 -- 172.21.15.81:0/316897867 learned_addr learned my addr 172.21.15.81:0/316897867 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:02.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.143+0000 7f26e6d9d640 1 -- 172.21.15.81:0/316897867 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26e80ccc10 msgr2=0x7f26e80ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:02.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.143+0000 7f26e6d9d640 1 --2- 172.21.15.81:0/316897867 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26e80ccc10 0x7f26e80ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.143+0000 7f26e6d9d640 1 -- 172.21.15.81:0/316897867 >> v1:172.21.15.93:6789/0 conn(0x7f26e815a0e0 legacy=0x7f26e80c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.143+0000 7f26e6d9d640 1 -- 172.21.15.81:0/316897867 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e815a4c0 con 0x7f26e806b710 2023-11-08T16:31:02.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.144+0000 7f26e6d9d640 1 --2- 172.21.15.81:0/316897867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26e806b710 0x7f26e806bb80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009e50 tx=0x7f26e002bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7e18665cefa796a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:02.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.145+0000 7f26e559a640 1 -- 172.21.15.81:0/316897867 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e002cb90 con 0x7f26e806b710 2023-11-08T16:31:02.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.145+0000 7f26e559a640 1 -- 172.21.15.81:0/316897867 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e002ccf0 con 0x7f26e806b710 2023-11-08T16:31:02.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.145+0000 7f26e559a640 1 -- 172.21.15.81:0/316897867 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0034610 con 0x7f26e806b710 2023-11-08T16:31:02.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.145+0000 7f26ecd0c640 1 -- 172.21.15.81:0/316897867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26e806b710 msgr2=0x7f26e806bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.145+0000 7f26ecd0c640 1 --2- 172.21.15.81:0/316897867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26e806b710 0x7f26e806bb80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009e50 tx=0x7f26e002bff0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.145+0000 7f26ecd0c640 1 -- 172.21.15.81:0/316897867 shutdown_connections 2023-11-08T16:31:02.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.145+0000 7f26ecd0c640 1 --2- 172.21.15.81:0/316897867 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26e806b710 0x7f26e806bb80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.145+0000 7f26ecd0c640 1 --2- 172.21.15.81:0/316897867 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26e80ccc10 0x7f26e80ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.145+0000 7f26ecd0c640 1 -- 172.21.15.81:0/316897867 >> 172.21.15.81:0/316897867 conn(0x7f26e80cb580 msgr2=0x7f26e80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:02.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.145+0000 7f26ecd0c640 1 -- 172.21.15.81:0/316897867 shutdown_connections 2023-11-08T16:31:02.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.146+0000 7f26ecd0c640 1 -- 172.21.15.81:0/316897867 wait complete. 2023-11-08T16:31:02.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.147+0000 7f26ecd0c640 1 Processor -- start 2023-11-08T16:31:02.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.147+0000 7f26ecd0c640 1 -- start start 2023-11-08T16:31:02.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.148+0000 7f26ecd0c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26e806b710 0x7f26e80c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.148+0000 7f26ecd0c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26e80ccc10 0x7f26e8156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.148+0000 7f26ecd0c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26e815a0e0 0x7f26e80c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.148+0000 7f26ecd0c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80b2750 con 0x7f26e80ccc10 2023-11-08T16:31:02.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.148+0000 7f26ecd0c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80c6110 con 0x7f26e806b710 2023-11-08T16:31:02.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.148+0000 7f26ecd0c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e80c55a0 con 0x7f26e815a0e0 2023-11-08T16:31:02.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.148+0000 7f26e5d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26e80ccc10 0x7f26e8156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.149+0000 7f26e5d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26e80ccc10 0x7f26e8156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42382/0 (socket says 172.21.15.81:42382) 2023-11-08T16:31:02.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.149+0000 7f26e5d9b640 1 -- 172.21.15.81:0/160169209 learned_addr learned my addr 172.21.15.81:0/160169209 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:02.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.149+0000 7f26e659c640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26e806b710 0x7f26e80c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.149+0000 7f26e6d9d640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26e815a0e0 0x7f26e80c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.149+0000 7f26e5d9b640 1 -- 172.21.15.81:0/160169209 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26e815a0e0 msgr2=0x7f26e80c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.149+0000 7f26e5d9b640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26e815a0e0 0x7f26e80c42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.149+0000 7f26e5d9b640 1 -- 172.21.15.81:0/160169209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26e806b710 msgr2=0x7f26e80c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.149+0000 7f26e5d9b640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26e806b710 0x7f26e80c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.149+0000 7f26e5d9b640 1 -- 172.21.15.81:0/160169209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e0009b30 con 0x7f26e80ccc10 2023-11-08T16:31:02.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.149+0000 7f26e5d9b640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26e80ccc10 0x7f26e8156ed0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00e9b0 tx=0x7f26dc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:02.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.150+0000 7f26d6ffd640 1 -- 172.21.15.81:0/160169209 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc009670 con 0x7f26e80ccc10 2023-11-08T16:31:02.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.150+0000 7f26d6ffd640 1 -- 172.21.15.81:0/160169209 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc00c9a0 con 0x7f26e80ccc10 2023-11-08T16:31:02.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.150+0000 7f26d6ffd640 1 -- 172.21.15.81:0/160169209 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc00cd50 con 0x7f26e80ccc10 2023-11-08T16:31:02.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.150+0000 7f26ecd0c640 1 -- 172.21.15.81:0/160169209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e80c5880 con 0x7f26e80ccc10 2023-11-08T16:31:02.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.150+0000 7f26ecd0c640 1 -- 172.21.15.81:0/160169209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e80b23e0 con 0x7f26e80ccc10 2023-11-08T16:31:02.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.151+0000 7f26ecd0c640 1 -- 172.21.15.81:0/160169209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e8071160 con 0x7f26e80ccc10 2023-11-08T16:31:02.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.152+0000 7f26d6ffd640 1 -- 172.21.15.81:0/160169209 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f26dc013470 con 0x7f26e80ccc10 2023-11-08T16:31:02.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.153+0000 7f26d6ffd640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f26c806ae70 0x7f26c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.153+0000 7f26d6ffd640 1 -- 172.21.15.81:0/160169209 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f26dc089a10 con 0x7f26e80ccc10 2023-11-08T16:31:02.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.153+0000 7f26e659c640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f26c806ae70 0x7f26c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.154+0000 7f26e659c640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f26c806ae70 0x7f26c806d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f26d8000f60 tx=0x7f26d8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:02.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.157+0000 7f26d6ffd640 1 -- 172.21.15.81:0/160169209 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26e8071160 con 0x7f26e80ccc10 2023-11-08T16:31:02.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.346+0000 7f26ecd0c640 1 -- 172.21.15.81:0/160169209 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26e8071160 con 0x7f26e80ccc10 2023-11-08T16:31:02.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.347+0000 7f26d6ffd640 1 -- 172.21.15.81:0/160169209 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26e8071160 con 0x7f26e80ccc10 2023-11-08T16:31:02.349 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:31:02.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.349+0000 7f26ecd0c640 1 -- 172.21.15.81:0/160169209 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f26c806ae70 msgr2=0x7f26c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.349+0000 7f26ecd0c640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f26c806ae70 0x7f26c806d330 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f26d8000f60 tx=0x7f26d8009040 comp rx=0 tx=0).stop 2023-11-08T16:31:02.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.349+0000 7f26ecd0c640 1 -- 172.21.15.81:0/160169209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26e80ccc10 msgr2=0x7f26e8156ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.349+0000 7f26ecd0c640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26e80ccc10 0x7f26e8156ed0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f26dc00e9b0 tx=0x7f26dc00ee70 comp rx=0 tx=0).stop 2023-11-08T16:31:02.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.349+0000 7f26ecd0c640 1 -- 172.21.15.81:0/160169209 shutdown_connections 2023-11-08T16:31:02.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.349+0000 7f26ecd0c640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f26c806ae70 0x7f26c806d330 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.349+0000 7f26ecd0c640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26e815a0e0 0x7f26e80c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.349+0000 7f26ecd0c640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26e80ccc10 0x7f26e8156ed0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.349+0000 7f26ecd0c640 1 --2- 172.21.15.81:0/160169209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26e806b710 0x7f26e80c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.349+0000 7f26ecd0c640 1 -- 172.21.15.81:0/160169209 >> 172.21.15.81:0/160169209 conn(0x7f26e80cb580 msgr2=0x7f26e80cd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:02.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.350+0000 7f26ecd0c640 1 -- 172.21.15.81:0/160169209 shutdown_connections 2023-11-08T16:31:02.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.350+0000 7f26ecd0c640 1 -- 172.21.15.81:0/160169209 wait complete. 2023-11-08T16:31:02.370 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:31:02.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.586+0000 7f888cfc0640 1 Processor -- start 2023-11-08T16:31:02.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.586+0000 7f888cfc0640 1 -- start start 2023-11-08T16:31:02.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.588+0000 7f888cfc0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88880ce0d0 0x7f88880caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.588+0000 7f888cfc0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88880cda10 con 0x7f888815f280 2023-11-08T16:31:02.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.588+0000 7f888cfc0640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f888806fa30 con 0x7f888815bfc0 2023-11-08T16:31:02.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.588+0000 7f888cfc0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88880ce4b0 con 0x7f88880ce0d0 2023-11-08T16:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.588+0000 7f8886d9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f888815f280 0x7f88881516e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45310/0 (socket says 172.21.15.81:45310) 2023-11-08T16:31:02.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.588+0000 7f8886d9d640 1 -- 172.21.15.81:0/1472783772 learned_addr learned my addr 172.21.15.81:0/1472783772 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:02.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4051939151 0 0) 0x7f88880cda10 con 0x7f888815f280 2023-11-08T16:31:02.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8874003520 con 0x7f888815f280 2023-11-08T16:31:02.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f888659c640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88880ce0d0 0x7f88880caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2816879379 0 0) 0x7f8874003520 con 0x7f888815f280 2023-11-08T16:31:02.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88880ce0d0 msgr2=0x7f88880caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:02.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88880ce0d0 0x7f88880caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 >> v1:172.21.15.93:6789/0 conn(0x7f888815bfc0 legacy=0x7f888815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888806fca0 con 0x7f888815f280 2023-11-08T16:31:02.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f88800019f0 con 0x7f888815f280 2023-11-08T16:31:02.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f888659c640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88880ce0d0 0x7f88880caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:02.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 >> v1:172.21.15.81:6789/0 conn(0x7f888815f280 legacy=0x7f88881516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88880ce0d0 0x7f8874003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8874004130 0x7f88740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8874004e00 0x7f8874005280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88880ce4b0 con 0x7f8874004e00 2023-11-08T16:31:02.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f887400bc80 con 0x7f8874004130 2023-11-08T16:31:02.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.589+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887400be40 con 0x7f88880ce0d0 2023-11-08T16:31:02.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.590+0000 7f8886d9d640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8874004e00 0x7f8874005280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.590+0000 7f888659c640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88880ce0d0 0x7f8874003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.590+0000 7f887ffff640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8874004130 0x7f88740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.590+0000 7f8886d9d640 1 -- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88880ce0d0 msgr2=0x7f8874003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:02.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.590+0000 7f8886d9d640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88880ce0d0 0x7f8874003a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.590+0000 7f8886d9d640 1 -- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8874004130 msgr2=0x7f88740047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:02.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.590+0000 7f8886d9d640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8874004130 0x7f88740047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.590+0000 7f8886d9d640 1 -- 172.21.15.81:0/1472783772 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8874003520 con 0x7f8874004e00 2023-11-08T16:31:02.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.590+0000 7f887ffff640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8874004130 0x7f88740047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:31:02.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.591+0000 7f8886d9d640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8874004e00 0x7f8874005280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f88880ca7a0 tx=0x7f88740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641e9d311b4b321b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:02.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.591+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880034630 con 0x7f8874004e00 2023-11-08T16:31:02.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.591+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888002f070 con 0x7f8874004e00 2023-11-08T16:31:02.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.592+0000 7f8885d9b640 1 -- 172.21.15.81:0/1472783772 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880034a50 con 0x7f8874004e00 2023-11-08T16:31:02.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.592+0000 7f888cfc0640 1 -- 172.21.15.81:0/1472783772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8874004e00 msgr2=0x7f8874005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.592+0000 7f888cfc0640 1 --2- 172.21.15.81:0/1472783772 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8874004e00 0x7f8874005280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f88880ca7a0 tx=0x7f88740057f0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.592+0000 7f8886d9d640 1 -- 172.21.15.81:0/1472783772 reap_dead start 2023-11-08T16:31:02.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.592+0000 7f888cfc0640 1 -- 172.21.15.81:0/1472783772 shutdown_connections 2023-11-08T16:31:02.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.592+0000 7f888cfc0640 1 -- 172.21.15.81:0/1472783772 >> 172.21.15.81:0/1472783772 conn(0x7f88880cb580 msgr2=0x7f88880cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:02.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.592+0000 7f888cfc0640 1 -- 172.21.15.81:0/1472783772 shutdown_connections 2023-11-08T16:31:02.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.592+0000 7f888cfc0640 1 -- 172.21.15.81:0/1472783772 wait complete. 2023-11-08T16:31:02.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.594+0000 7f888cfc0640 1 Processor -- start 2023-11-08T16:31:02.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.594+0000 7f888cfc0640 1 -- start start 2023-11-08T16:31:02.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.595+0000 7f888cfc0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8874004e00 0x7f8888149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.595+0000 7f888cfc0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f888815a790 0x7f8888158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.595+0000 7f888cfc0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88881591e0 0x7f88880c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.595+0000 7f888cfc0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888814a520 con 0x7f88881591e0 2023-11-08T16:31:02.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.595+0000 7f888cfc0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88881596f0 con 0x7f8874004e00 2023-11-08T16:31:02.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.595+0000 7f888cfc0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88881599d0 con 0x7f888815a790 2023-11-08T16:31:02.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.595+0000 7f8886d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88881591e0 0x7f88880c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.595+0000 7f8886d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88881591e0 0x7f88880c4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42406/0 (socket says 172.21.15.81:42406) 2023-11-08T16:31:02.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.595+0000 7f8886d9d640 1 -- 172.21.15.81:0/545499111 learned_addr learned my addr 172.21.15.81:0/545499111 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:02.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.596+0000 7f888659c640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8874004e00 0x7f8888149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.596+0000 7f887ffff640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f888815a790 0x7f8888158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.596+0000 7f8886d9d640 1 -- 172.21.15.81:0/545499111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f888815a790 msgr2=0x7f8888158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.596+0000 7f8886d9d640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f888815a790 0x7f8888158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.596+0000 7f8886d9d640 1 -- 172.21.15.81:0/545499111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8874004e00 msgr2=0x7f8888149fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.596+0000 7f8886d9d640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8874004e00 0x7f8888149fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.596+0000 7f8886d9d640 1 -- 172.21.15.81:0/545499111 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8888159cb0 con 0x7f88881591e0 2023-11-08T16:31:02.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.596+0000 7f888659c640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8874004e00 0x7f8888149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:02.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.596+0000 7f887ffff640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f888815a790 0x7f8888158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:02.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.596+0000 7f8886d9d640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88881591e0 0x7f88880c4800 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f888000a850 tx=0x7f8880001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:02.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.596+0000 7f887effd640 1 -- 172.21.15.81:0/545499111 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880034840 con 0x7f88881591e0 2023-11-08T16:31:02.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.597+0000 7f887effd640 1 -- 172.21.15.81:0/545499111 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88800349a0 con 0x7f88881591e0 2023-11-08T16:31:02.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.597+0000 7f887effd640 1 -- 172.21.15.81:0/545499111 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880034d10 con 0x7f88881591e0 2023-11-08T16:31:02.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.597+0000 7f888cfc0640 1 -- 172.21.15.81:0/545499111 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8888159f90 con 0x7f88881591e0 2023-11-08T16:31:02.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.597+0000 7f888cfc0640 1 -- 172.21.15.81:0/545499111 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f888815ac10 con 0x7f88881591e0 2023-11-08T16:31:02.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.598+0000 7f888cfc0640 1 -- 172.21.15.81:0/545499111 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888815a4b0 con 0x7f88881591e0 2023-11-08T16:31:02.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.599+0000 7f887effd640 1 -- 172.21.15.81:0/545499111 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8880031030 con 0x7f88881591e0 2023-11-08T16:31:02.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.600+0000 7f887effd640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f886806ae70 0x7f886806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:02.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.600+0000 7f887effd640 1 -- 172.21.15.81:0/545499111 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f88800abbf0 con 0x7f88881591e0 2023-11-08T16:31:02.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.600+0000 7f888659c640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f886806ae70 0x7f886806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:02.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.601+0000 7f888659c640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f886806ae70 0x7f886806d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f887800a9b0 tx=0x7f8878008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:02.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.604+0000 7f887effd640 1 -- 172.21.15.81:0/545499111 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f888815a4b0 con 0x7f88881591e0 2023-11-08T16:31:02.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.797+0000 7f888cfc0640 1 -- 172.21.15.81:0/545499111 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f888815a4b0 con 0x7f88881591e0 2023-11-08T16:31:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.798+0000 7f887effd640 1 -- 172.21.15.81:0/545499111 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f888815a4b0 con 0x7f88881591e0 2023-11-08T16:31:02.800 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:31:02.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 -- 172.21.15.81:0/545499111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f886806ae70 msgr2=0x7f886806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f886806ae70 0x7f886806d330 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f887800a9b0 tx=0x7f8878008040 comp rx=0 tx=0).stop 2023-11-08T16:31:02.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 -- 172.21.15.81:0/545499111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88881591e0 msgr2=0x7f88880c4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:02.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88881591e0 0x7f88880c4800 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f888000a850 tx=0x7f8880001cb0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 -- 172.21.15.81:0/545499111 shutdown_connections 2023-11-08T16:31:02.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f886806ae70 0x7f886806d330 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88881591e0 0x7f88880c4800 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f888815a790 0x7f8888158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 --2- 172.21.15.81:0/545499111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8874004e00 0x7f8888149fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:02.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 -- 172.21.15.81:0/545499111 >> 172.21.15.81:0/545499111 conn(0x7f88880cb580 msgr2=0x7f88880b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:02.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 -- 172.21.15.81:0/545499111 shutdown_connections 2023-11-08T16:31:02.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:02.801+0000 7f888cfc0640 1 -- 172.21.15.81:0/545499111 wait complete. 2023-11-08T16:31:02.820 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:31:03.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.017+0000 7f239ffff640 1 Processor -- start 2023-11-08T16:31:03.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.018+0000 7f239ffff640 1 -- start start 2023-11-08T16:31:03.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.019+0000 7f239ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f23a015cf30 0x7f23a00d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.019+0000 7f239ffff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f23a00cfd90 0x7f23a00cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.019+0000 7f239ffff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23a00ce0b0 con 0x7f23a015d310 2023-11-08T16:31:03.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.019+0000 7f239ffff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a00cdb70 con 0x7f23a015cf30 2023-11-08T16:31:03.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.019+0000 7f239ffff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23a00ce390 con 0x7f23a00cfd90 2023-11-08T16:31:03.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.020+0000 7f239effd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f23a015cf30 0x7f23a00d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.020+0000 7f2397fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f23a015d310 0x7f23a00cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45312/0 (socket says 172.21.15.81:45312) 2023-11-08T16:31:03.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.020+0000 7f2397fff640 1 -- 172.21.15.81:0/2792251123 learned_addr learned my addr 172.21.15.81:0/2792251123 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:03.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.020+0000 7f239effd640 1 -- 172.21.15.81:0/2792251123 >> v2:172.21.15.93:3300/0 conn(0x7f23a00cfd90 msgr2=0x7f23a00cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:03.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.020+0000 7f239effd640 1 --2- 172.21.15.81:0/2792251123 >> v2:172.21.15.93:3300/0 conn(0x7f23a00cfd90 0x7f23a00cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.020+0000 7f239effd640 1 -- 172.21.15.81:0/2792251123 >> v1:172.21.15.81:6789/0 conn(0x7f23a015d310 legacy=0x7f23a00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.020+0000 7f239effd640 1 -- 172.21.15.81:0/2792251123 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a00d1be0 con 0x7f23a015cf30 2023-11-08T16:31:03.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.020+0000 7f239e7fc640 1 -- 172.21.15.81:0/2792251123 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121958780 0 0) 0x7f23a00ce0b0 con 0x7f23a015d310 2023-11-08T16:31:03.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.021+0000 7f239effd640 1 --2- 172.21.15.81:0/2792251123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23a015cf30 0x7f23a00d26c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f23900097a0 tx=0x7f239002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137e35e3380f53e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:03.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.022+0000 7f239e7fc640 1 -- 172.21.15.81:0/2792251123 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390031830 con 0x7f23a015cf30 2023-11-08T16:31:03.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.022+0000 7f239e7fc640 1 -- 172.21.15.81:0/2792251123 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2390031990 con 0x7f23a015cf30 2023-11-08T16:31:03.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.022+0000 7f239e7fc640 1 -- 172.21.15.81:0/2792251123 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2390031d20 con 0x7f23a015cf30 2023-11-08T16:31:03.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.022+0000 7f239ffff640 1 -- 172.21.15.81:0/2792251123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23a015cf30 msgr2=0x7f23a00d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.022+0000 7f239ffff640 1 --2- 172.21.15.81:0/2792251123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23a015cf30 0x7f23a00d26c0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f23900097a0 tx=0x7f239002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.022+0000 7f239ffff640 1 -- 172.21.15.81:0/2792251123 shutdown_connections 2023-11-08T16:31:03.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.022+0000 7f239ffff640 1 --2- 172.21.15.81:0/2792251123 >> v2:172.21.15.93:3300/0 conn(0x7f23a00cfd90 0x7f23a00cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.022+0000 7f239ffff640 1 --2- 172.21.15.81:0/2792251123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23a015cf30 0x7f23a00d26c0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.022+0000 7f239ffff640 1 -- 172.21.15.81:0/2792251123 >> 172.21.15.81:0/2792251123 conn(0x7f23a006c040 msgr2=0x7f23a006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:03.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.022+0000 7f239ffff640 1 -- 172.21.15.81:0/2792251123 shutdown_connections 2023-11-08T16:31:03.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.022+0000 7f239ffff640 1 -- 172.21.15.81:0/2792251123 wait complete. 2023-11-08T16:31:03.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.024+0000 7f239ffff640 1 Processor -- start 2023-11-08T16:31:03.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.024+0000 7f239ffff640 1 -- start start 2023-11-08T16:31:03.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.025+0000 7f239ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a00cfd90 0x7f23a01f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.025+0000 7f239ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23a015cf30 0x7f23a01f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.025+0000 7f239ffff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23a00aa630 0x7f23a015a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.025+0000 7f239ffff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a01f6c30 con 0x7f23a015cf30 2023-11-08T16:31:03.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.025+0000 7f239ffff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23a01f6670 con 0x7f23a00aa630 2023-11-08T16:31:03.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.025+0000 7f239ffff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23a01f6950 con 0x7f23a00cfd90 2023-11-08T16:31:03.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.026+0000 7f2397fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23a015cf30 0x7f23a01f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.026+0000 7f2397fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23a015cf30 0x7f23a01f3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42434/0 (socket says 172.21.15.81:42434) 2023-11-08T16:31:03.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.026+0000 7f2397fff640 1 -- 172.21.15.81:0/3790595749 learned_addr learned my addr 172.21.15.81:0/3790595749 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:03.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.026+0000 7f239f7fe640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23a00aa630 0x7f23a015a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.026+0000 7f239effd640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a00cfd90 0x7f23a01f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.026+0000 7f2397fff640 1 -- 172.21.15.81:0/3790595749 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a00cfd90 msgr2=0x7f23a01f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.026+0000 7f2397fff640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a00cfd90 0x7f23a01f5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.026+0000 7f2397fff640 1 -- 172.21.15.81:0/3790595749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23a00aa630 msgr2=0x7f23a015a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.026+0000 7f2397fff640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23a00aa630 0x7f23a015a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.026+0000 7f2397fff640 1 -- 172.21.15.81:0/3790595749 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2390009400 con 0x7f23a015cf30 2023-11-08T16:31:03.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.026+0000 7f2397fff640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23a015cf30 0x7f23a01f3100 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f238800c470 tx=0x7f238800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:03.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.027+0000 7f23977fe640 1 -- 172.21.15.81:0/3790595749 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23880074e0 con 0x7f23a015cf30 2023-11-08T16:31:03.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.027+0000 7f23977fe640 1 -- 172.21.15.81:0/3790595749 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2388007640 con 0x7f23a015cf30 2023-11-08T16:31:03.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.027+0000 7f23977fe640 1 -- 172.21.15.81:0/3790595749 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388004a20 con 0x7f23a015cf30 2023-11-08T16:31:03.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.027+0000 7f239ffff640 1 -- 172.21.15.81:0/3790595749 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23a01f2900 con 0x7f23a015cf30 2023-11-08T16:31:03.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.027+0000 7f239ffff640 1 -- 172.21.15.81:0/3790595749 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23a01f6320 con 0x7f23a015cf30 2023-11-08T16:31:03.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.028+0000 7f239ffff640 1 -- 172.21.15.81:0/3790595749 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23a01f2e20 con 0x7f23a015cf30 2023-11-08T16:31:03.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.029+0000 7f23977fe640 1 -- 172.21.15.81:0/3790595749 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2388004b80 con 0x7f23a015cf30 2023-11-08T16:31:03.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.030+0000 7f23977fe640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f238006ae70 0x7f238006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.030+0000 7f23977fe640 1 -- 172.21.15.81:0/3790595749 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f2388088a30 con 0x7f23a015cf30 2023-11-08T16:31:03.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.030+0000 7f239effd640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f238006ae70 0x7f238006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.031+0000 7f239effd640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f238006ae70 0x7f238006d330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2390036000 tx=0x7f2390008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:03.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.034+0000 7f23977fe640 1 -- 172.21.15.81:0/3790595749 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f23a01f2e20 con 0x7f23a015cf30 2023-11-08T16:31:03.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.225+0000 7f239ffff640 1 -- 172.21.15.81:0/3790595749 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23a01f2e20 con 0x7f23a015cf30 2023-11-08T16:31:03.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.226+0000 7f23977fe640 1 -- 172.21.15.81:0/3790595749 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7f23a01f2e20 con 0x7f23a015cf30 2023-11-08T16:31:03.228 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:31:03.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.228+0000 7f239ffff640 1 -- 172.21.15.81:0/3790595749 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f238006ae70 msgr2=0x7f238006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.228+0000 7f239ffff640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f238006ae70 0x7f238006d330 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2390036000 tx=0x7f2390008030 comp rx=0 tx=0).stop 2023-11-08T16:31:03.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.228+0000 7f239ffff640 1 -- 172.21.15.81:0/3790595749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23a015cf30 msgr2=0x7f23a01f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.228+0000 7f239ffff640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23a015cf30 0x7f23a01f3100 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f238800c470 tx=0x7f238800c930 comp rx=0 tx=0).stop 2023-11-08T16:31:03.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.228+0000 7f239ffff640 1 -- 172.21.15.81:0/3790595749 shutdown_connections 2023-11-08T16:31:03.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.228+0000 7f239ffff640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f238006ae70 0x7f238006d330 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.229+0000 7f239ffff640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23a00aa630 0x7f23a015a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.229+0000 7f239ffff640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23a015cf30 0x7f23a01f3100 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.229+0000 7f239ffff640 1 --2- 172.21.15.81:0/3790595749 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23a00cfd90 0x7f23a01f5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.229+0000 7f239ffff640 1 -- 172.21.15.81:0/3790595749 >> 172.21.15.81:0/3790595749 conn(0x7f23a006c040 msgr2=0x7f23a006c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.229+0000 7f239ffff640 1 -- 172.21.15.81:0/3790595749 shutdown_connections 2023-11-08T16:31:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.229+0000 7f239ffff640 1 -- 172.21.15.81:0/3790595749 wait complete. 2023-11-08T16:31:03.248 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:31:03.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.454+0000 7fcdaca1a640 1 Processor -- start 2023-11-08T16:31:03.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.454+0000 7fcdaca1a640 1 -- start start 2023-11-08T16:31:03.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.455+0000 7fcdaca1a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcda815b250 0x7fcda815a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.455+0000 7fcdaca1a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda8157a80 con 0x7fcda80b11f0 2023-11-08T16:31:03.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.455+0000 7fcdaca1a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcda815b6e0 con 0x7fcda81576a0 2023-11-08T16:31:03.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.455+0000 7fcdaca1a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcda815c360 con 0x7fcda815b250 2023-11-08T16:31:03.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.456+0000 7fcda7fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcda80b11f0 0x7fcda815c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45316/0 (socket says 172.21.15.81:45316) 2023-11-08T16:31:03.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.456+0000 7fcda7fff640 1 -- 172.21.15.81:0/2048600256 learned_addr learned my addr 172.21.15.81:0/2048600256 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:03.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.456+0000 7fcd9ffff640 1 --2- 172.21.15.81:0/2048600256 >> v2:172.21.15.93:3300/0 conn(0x7fcda815b250 0x7fcda815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.456+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524075286 0 0) 0x7fcda8157a80 con 0x7fcda80b11f0 2023-11-08T16:31:03.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.456+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd94003520 con 0x7fcda80b11f0 2023-11-08T16:31:03.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731415460 0 0) 0x7fcda815b6e0 con 0x7fcda81576a0 2023-11-08T16:31:03.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcda8157a80 con 0x7fcda81576a0 2023-11-08T16:31:03.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3987396012 0 0) 0x7fcd94003520 con 0x7fcda80b11f0 2023-11-08T16:31:03.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 >> v2:172.21.15.93:3300/0 conn(0x7fcda815b250 msgr2=0x7fcda815a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:03.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 --2- 172.21.15.81:0/2048600256 >> v2:172.21.15.93:3300/0 conn(0x7fcda815b250 0x7fcda815a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 >> v1:172.21.15.93:6789/0 conn(0x7fcda81576a0 legacy=0x7fcda8070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda815b9c0 con 0x7fcda80b11f0 2023-11-08T16:31:03.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fcda00019f0 con 0x7fcda80b11f0 2023-11-08T16:31:03.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 >> v1:172.21.15.81:6789/0 conn(0x7fcda80b11f0 legacy=0x7fcda815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 0x7fcd94003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd940047c0 0x7fcd94004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd940051d0 0x7fcd9400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda815c360 con 0x7fcd94003b50 2023-11-08T16:31:03.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9400c100 con 0x7fcd940047c0 2023-11-08T16:31:03.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9400c2c0 con 0x7fcd940051d0 2023-11-08T16:31:03.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.457+0000 7fcda77fe640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 0x7fcd94003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.458+0000 7fcd9ffff640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd940047c0 0x7fcd94004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.458+0000 7fcda7fff640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd940051d0 0x7fcd9400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.458+0000 7fcda77fe640 1 -- 172.21.15.81:0/2048600256 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd940051d0 msgr2=0x7fcd9400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:03.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.458+0000 7fcda77fe640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd940051d0 0x7fcd9400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.458+0000 7fcda77fe640 1 -- 172.21.15.81:0/2048600256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd940047c0 msgr2=0x7fcd94004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:03.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.458+0000 7fcda77fe640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd940047c0 0x7fcd94004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.458+0000 7fcda77fe640 1 -- 172.21.15.81:0/2048600256 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd94003520 con 0x7fcd94003b50 2023-11-08T16:31:03.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.458+0000 7fcda7fff640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd940051d0 0x7fcd9400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:03.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.458+0000 7fcda7fff640 1 -- 172.21.15.81:0/2048600256 reap_dead start 2023-11-08T16:31:03.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.459+0000 7fcda77fe640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 0x7fcd94003f50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcd980018d0 tx=0x7fcd940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7b934be21b371c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:03.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.459+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd980052d0 con 0x7fcd94003b50 2023-11-08T16:31:03.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.459+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd98005430 con 0x7fcd94003b50 2023-11-08T16:31:03.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.459+0000 7fcda6ffd640 1 -- 172.21.15.81:0/2048600256 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98003810 con 0x7fcd94003b50 2023-11-08T16:31:03.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.460+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2048600256 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 msgr2=0x7fcd94003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.460+0000 7fcdaca1a640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 0x7fcd94003f50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fcd980018d0 tx=0x7fcd940057f0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.460+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2048600256 shutdown_connections 2023-11-08T16:31:03.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.460+0000 7fcdaca1a640 1 --2- 172.21.15.81:0/2048600256 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 0x7fcd94003f50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.460+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2048600256 >> 172.21.15.81:0/2048600256 conn(0x7fcda80ce8d0 msgr2=0x7fcda80cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:03.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.460+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2048600256 shutdown_connections 2023-11-08T16:31:03.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.460+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2048600256 wait complete. 2023-11-08T16:31:03.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.462+0000 7fcdaca1a640 1 Processor -- start 2023-11-08T16:31:03.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.462+0000 7fcdaca1a640 1 -- start start 2023-11-08T16:31:03.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcdaca1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 0x7fcda8140550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcdaca1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcda8140a90 0x7fcda80c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcdaca1a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcda80c44d0 0x7fcda81593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcdaca1a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda815cf80 con 0x7fcd94003b50 2023-11-08T16:31:03.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcdaca1a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80c4960 con 0x7fcda80c44d0 2023-11-08T16:31:03.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcdaca1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda80c8d40 con 0x7fcda8140a90 2023-11-08T16:31:03.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcda77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 0x7fcda8140550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcda77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 0x7fcda8140550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42446/0 (socket says 172.21.15.81:42446) 2023-11-08T16:31:03.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcda77fe640 1 -- 172.21.15.81:0/2613254788 learned_addr learned my addr 172.21.15.81:0/2613254788 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:03.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcda77fe640 1 -- 172.21.15.81:0/2613254788 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcda8140a90 msgr2=0x7fcda80c3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcda77fe640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcda8140a90 0x7fcda80c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcda77fe640 1 -- 172.21.15.81:0/2613254788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcda80c44d0 msgr2=0x7fcda81593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcda77fe640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcda80c44d0 0x7fcda81593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.463+0000 7fcda77fe640 1 -- 172.21.15.81:0/2613254788 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda8140f40 con 0x7fcd94003b50 2023-11-08T16:31:03.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.464+0000 7fcda77fe640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 0x7fcda8140550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcd98035040 tx=0x7fcd980012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:03.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.464+0000 7fcda4ff9640 1 -- 172.21.15.81:0/2613254788 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd98006d90 con 0x7fcd94003b50 2023-11-08T16:31:03.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.464+0000 7fcda4ff9640 1 -- 172.21.15.81:0/2613254788 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd98007040 con 0x7fcd94003b50 2023-11-08T16:31:03.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.464+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2613254788 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcda8141220 con 0x7fcd94003b50 2023-11-08T16:31:03.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.464+0000 7fcda4ff9640 1 -- 172.21.15.81:0/2613254788 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd980073f0 con 0x7fcd94003b50 2023-11-08T16:31:03.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.464+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2613254788 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda81417e0 con 0x7fcd94003b50 2023-11-08T16:31:03.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.465+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2613254788 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcda804ece0 con 0x7fcd94003b50 2023-11-08T16:31:03.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.466+0000 7fcda4ff9640 1 -- 172.21.15.81:0/2613254788 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd98007550 con 0x7fcd94003b50 2023-11-08T16:31:03.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.467+0000 7fcda4ff9640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd8806aea0 0x7fcd8806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.467+0000 7fcda4ff9640 1 -- 172.21.15.81:0/2613254788 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7fcd980ab0c0 con 0x7fcd94003b50 2023-11-08T16:31:03.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.467+0000 7fcd9ffff640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd8806aea0 0x7fcd8806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.468+0000 7fcd9ffff640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd8806aea0 0x7fcd8806d360 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcda8140e70 tx=0x7fcd90008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:03.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.471+0000 7fcda4ff9640 1 -- 172.21.15.81:0/2613254788 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd98076100 con 0x7fcd94003b50 2023-11-08T16:31:03.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.659+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2613254788 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fcda806ba40 con 0x7fcd94003b50 2023-11-08T16:31:03.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.660+0000 7fcda4ff9640 1 -- 172.21.15.81:0/2613254788 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v38) v1 ==== 106+0+11 (secure 0 0 0) 0x7fcd98041020 con 0x7fcd94003b50 2023-11-08T16:31:03.662 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:31:03.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2613254788 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd8806aea0 msgr2=0x7fcd8806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd8806aea0 0x7fcd8806d360 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fcda8140e70 tx=0x7fcd90008040 comp rx=0 tx=0).stop 2023-11-08T16:31:03.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2613254788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 msgr2=0x7fcda8140550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 0x7fcda8140550 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcd98035040 tx=0x7fcd980012d0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2613254788 shutdown_connections 2023-11-08T16:31:03.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd8806aea0 0x7fcd8806d360 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcda80c44d0 0x7fcda81593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcda8140a90 0x7fcda80c3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 --2- 172.21.15.81:0/2613254788 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd94003b50 0x7fcda8140550 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2613254788 >> 172.21.15.81:0/2613254788 conn(0x7fcda80ce8d0 msgr2=0x7fcda80c3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:03.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2613254788 shutdown_connections 2023-11-08T16:31:03.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.663+0000 7fcdaca1a640 1 -- 172.21.15.81:0/2613254788 wait complete. 2023-11-08T16:31:03.684 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:31:03.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.896+0000 7f805f70f640 1 Processor -- start 2023-11-08T16:31:03.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.896+0000 7f805f70f640 1 -- start start 2023-11-08T16:31:03.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.897+0000 7f805f70f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580d2a10 0x7f80580b6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.897+0000 7f805f70f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80580d2120 con 0x7f805815d140 2023-11-08T16:31:03.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.897+0000 7f805f70f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80580d1b40 con 0x7f80580d25b0 2023-11-08T16:31:03.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.897+0000 7f805f70f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80580b2600 con 0x7f80580d2a10 2023-11-08T16:31:03.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f805d4ab640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f805815d140 0x7f80580cd790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45324/0 (socket says 172.21.15.81:45324) 2023-11-08T16:31:03.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f805d4ab640 1 -- 172.21.15.81:0/3104662541 learned_addr learned my addr 172.21.15.81:0/3104662541 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:03.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926487418 0 0) 0x7f80580d2120 con 0x7f805815d140 2023-11-08T16:31:03.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f805dcac640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580d2a10 0x7f80580b6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8044003520 con 0x7f805815d140 2023-11-08T16:31:03.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1274091847 0 0) 0x7f8044003520 con 0x7f805815d140 2023-11-08T16:31:03.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580d2a10 msgr2=0x7f80580b6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:03.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f804ffff640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580d2a10 0x7f80580b6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 >> v1:172.21.15.93:6789/0 conn(0x7f80580d25b0 legacy=0x7f80580cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80580ce350 con 0x7f805815d140 2023-11-08T16:31:03.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f80540017f0 con 0x7f805815d140 2023-11-08T16:31:03.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 >> v1:172.21.15.81:6789/0 conn(0x7f805815d140 legacy=0x7f80580cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f804ffff640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80440039a0 0x7f8044003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.898+0000 7f804ffff640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580d2a10 0x7f80440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f804ffff640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8044004d00 0x7f8044005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80580b2600 con 0x7f8044004d00 2023-11-08T16:31:03.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804400bc20 con 0x7f80440039a0 2023-11-08T16:31:03.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804400bde0 con 0x7f80580d2a10 2023-11-08T16:31:03.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f805dcac640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8044004d00 0x7f8044005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f805d4ab640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80440039a0 0x7f8044003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f805ccaa640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580d2a10 0x7f80440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f805dcac640 1 -- 172.21.15.81:0/3104662541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580d2a10 msgr2=0x7f80440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:03.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f805dcac640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580d2a10 0x7f80440047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f805dcac640 1 -- 172.21.15.81:0/3104662541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80440039a0 msgr2=0x7f8044003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:03.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f805dcac640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f80440039a0 0x7f8044003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.899+0000 7f805dcac640 1 -- 172.21.15.81:0/3104662541 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8044003520 con 0x7f8044004d00 2023-11-08T16:31:03.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.900+0000 7f805dcac640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8044004d00 0x7f8044005180 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f805808c3f0 tx=0x7f80440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7e0d7e5d585301 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:03.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.901+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805002cc70 con 0x7f8044004d00 2023-11-08T16:31:03.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.901+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f805002cdd0 con 0x7f8044004d00 2023-11-08T16:31:03.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.901+0000 7f804ffff640 1 -- 172.21.15.81:0/3104662541 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050034660 con 0x7f8044004d00 2023-11-08T16:31:03.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.901+0000 7f805f70f640 1 -- 172.21.15.81:0/3104662541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8044004d00 msgr2=0x7f8044005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.901+0000 7f805f70f640 1 --2- 172.21.15.81:0/3104662541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8044004d00 0x7f8044005180 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f805808c3f0 tx=0x7f80440057f0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.901+0000 7f805dcac640 1 -- 172.21.15.81:0/3104662541 reap_dead start 2023-11-08T16:31:03.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.901+0000 7f805f70f640 1 -- 172.21.15.81:0/3104662541 shutdown_connections 2023-11-08T16:31:03.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.901+0000 7f805f70f640 1 -- 172.21.15.81:0/3104662541 >> 172.21.15.81:0/3104662541 conn(0x7f805806a950 msgr2=0x7f805806ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:03.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.901+0000 7f805f70f640 1 -- 172.21.15.81:0/3104662541 shutdown_connections 2023-11-08T16:31:03.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.901+0000 7f805f70f640 1 -- 172.21.15.81:0/3104662541 wait complete. 2023-11-08T16:31:03.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.903+0000 7f805f70f640 1 Processor -- start 2023-11-08T16:31:03.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.903+0000 7f805f70f640 1 -- start start 2023-11-08T16:31:03.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.904+0000 7f805f70f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8044004d00 0x7f80581f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.904+0000 7f805f70f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805814b6e0 0x7f80580b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.904+0000 7f805f70f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580b4100 0x7f80580cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.904+0000 7f805f70f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80580b4580 con 0x7f805814b6e0 2023-11-08T16:31:03.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.904+0000 7f805f70f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80581f6e10 con 0x7f8044004d00 2023-11-08T16:31:03.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.904+0000 7f805f70f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805814e310 con 0x7f80580b4100 2023-11-08T16:31:03.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.905+0000 7f805ccaa640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805814b6e0 0x7f80580b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.905+0000 7f805ccaa640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805814b6e0 0x7f80580b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42470/0 (socket says 172.21.15.81:42470) 2023-11-08T16:31:03.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.905+0000 7f805ccaa640 1 -- 172.21.15.81:0/1094079358 learned_addr learned my addr 172.21.15.81:0/1094079358 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:03.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.905+0000 7f805d4ab640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8044004d00 0x7f80581f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.905+0000 7f805dcac640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580b4100 0x7f80580cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.905+0000 7f805ccaa640 1 -- 172.21.15.81:0/1094079358 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580b4100 msgr2=0x7f80580cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.905+0000 7f805ccaa640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580b4100 0x7f80580cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.905+0000 7f805ccaa640 1 -- 172.21.15.81:0/1094079358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8044004d00 msgr2=0x7f80581f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:03.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.905+0000 7f805ccaa640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8044004d00 0x7f80581f68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:03.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.905+0000 7f805ccaa640 1 -- 172.21.15.81:0/1094079358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805814e5f0 con 0x7f805814b6e0 2023-11-08T16:31:03.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.905+0000 7f805ccaa640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805814b6e0 0x7f80580b3bc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f804800d570 tx=0x7f804800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:03.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.906+0000 7f804dffb640 1 -- 172.21.15.81:0/1094079358 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048005480 con 0x7f805814b6e0 2023-11-08T16:31:03.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.906+0000 7f804dffb640 1 -- 172.21.15.81:0/1094079358 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8048003610 con 0x7f805814b6e0 2023-11-08T16:31:03.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.906+0000 7f804dffb640 1 -- 172.21.15.81:0/1094079358 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8048003980 con 0x7f805814b6e0 2023-11-08T16:31:03.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.906+0000 7f805f70f640 1 -- 172.21.15.81:0/1094079358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805814e8d0 con 0x7f805814b6e0 2023-11-08T16:31:03.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.906+0000 7f805f70f640 1 -- 172.21.15.81:0/1094079358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805814ee90 con 0x7f805814b6e0 2023-11-08T16:31:03.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.907+0000 7f805f70f640 1 -- 172.21.15.81:0/1094079358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8058067540 con 0x7f805814b6e0 2023-11-08T16:31:03.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.908+0000 7f804dffb640 1 -- 172.21.15.81:0/1094079358 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8048003ae0 con 0x7f805814b6e0 2023-11-08T16:31:03.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.909+0000 7f804dffb640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f803c06ae70 0x7f803c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:03.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.909+0000 7f804dffb640 1 -- 172.21.15.81:0/1094079358 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f8048089780 con 0x7f805814b6e0 2023-11-08T16:31:03.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.909+0000 7f805d4ab640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f803c06ae70 0x7f803c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:03.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.910+0000 7f805d4ab640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f803c06ae70 0x7f803c06d330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8054001d40 tx=0x7f8054009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:03.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:03.913+0000 7f804dffb640 1 -- 172.21.15.81:0/1094079358 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f80480545f0 con 0x7f805814b6e0 2023-11-08T16:31:04.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.113+0000 7f805f70f640 1 -- 172.21.15.81:0/1094079358 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f80580cfff0 con 0x7f805814b6e0 2023-11-08T16:31:04.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.114+0000 7f804dffb640 1 -- 172.21.15.81:0/1094079358 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8048001230 con 0x7f805814b6e0 2023-11-08T16:31:04.115 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:31:04.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.116+0000 7f805f70f640 1 -- 172.21.15.81:0/1094079358 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f803c06ae70 msgr2=0x7f803c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.116+0000 7f805f70f640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f803c06ae70 0x7f803c06d330 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8054001d40 tx=0x7f8054009040 comp rx=0 tx=0).stop 2023-11-08T16:31:04.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.116+0000 7f805f70f640 1 -- 172.21.15.81:0/1094079358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805814b6e0 msgr2=0x7f80580b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.116+0000 7f805f70f640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805814b6e0 0x7f80580b3bc0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f804800d570 tx=0x7f804800da30 comp rx=0 tx=0).stop 2023-11-08T16:31:04.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.116+0000 7f805f70f640 1 -- 172.21.15.81:0/1094079358 shutdown_connections 2023-11-08T16:31:04.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.116+0000 7f805f70f640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f803c06ae70 0x7f803c06d330 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.116+0000 7f805f70f640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f80580b4100 0x7f80580cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.116+0000 7f805f70f640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805814b6e0 0x7f80580b3bc0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.116+0000 7f805f70f640 1 --2- 172.21.15.81:0/1094079358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8044004d00 0x7f80581f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.116+0000 7f805f70f640 1 -- 172.21.15.81:0/1094079358 >> 172.21.15.81:0/1094079358 conn(0x7f805806a950 msgr2=0x7f80580b4e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:04.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.117+0000 7f805f70f640 1 -- 172.21.15.81:0/1094079358 shutdown_connections 2023-11-08T16:31:04.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.117+0000 7f805f70f640 1 -- 172.21.15.81:0/1094079358 wait complete. 2023-11-08T16:31:04.134 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:31:04.135 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:31:04.186 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:31:04.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.438+0000 7f28f784f640 1 Processor -- start 2023-11-08T16:31:04.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.438+0000 7f28f784f640 1 -- start start 2023-11-08T16:31:04.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.440+0000 7f28f784f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f28f00d1af0 0x7f28f00d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.440+0000 7f28f784f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f28f00b2c30 0x7f28f00b30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.440+0000 7f28f784f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28f00b2450 con 0x7f28f015d1a0 2023-11-08T16:31:04.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.440+0000 7f28f784f640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f00ce800 con 0x7f28f00b2c30 2023-11-08T16:31:04.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.440+0000 7f28f784f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28f00ceae0 con 0x7f28f00d1af0 2023-11-08T16:31:04.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.440+0000 7f28f5dec640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f28f00b2c30 0x7f28f00b30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:04.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.440+0000 7f28f5dec640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f28f00b2c30 0x7f28f00b30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42474/0 (socket says 172.21.15.81:42474) 2023-11-08T16:31:04.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.440+0000 7f28f5dec640 1 -- 172.21.15.81:0/3783910872 learned_addr learned my addr 172.21.15.81:0/3783910872 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:04.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.440+0000 7f28f55eb640 1 --1- 172.21.15.81:0/3783910872 >> v1:172.21.15.81:6789/0 conn(0x7f28f015d1a0 0x7f28f00d13e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45332/0 (socket says 172.21.15.81:45332) 2023-11-08T16:31:04.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.441+0000 7f28f4dea640 1 --2- 172.21.15.81:0/3783910872 >> v2:172.21.15.93:3300/0 conn(0x7f28f00d1af0 0x7f28f00d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:04.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.441+0000 7f28f5dec640 1 -- 172.21.15.81:0/3783910872 >> v2:172.21.15.93:3300/0 conn(0x7f28f00d1af0 msgr2=0x7f28f00d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:04.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.441+0000 7f28f5dec640 1 --2- 172.21.15.81:0/3783910872 >> v2:172.21.15.93:3300/0 conn(0x7f28f00d1af0 0x7f28f00d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.441+0000 7f28f5dec640 1 -- 172.21.15.81:0/3783910872 >> v1:172.21.15.81:6789/0 conn(0x7f28f015d1a0 legacy=0x7f28f00d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.441+0000 7f28f5dec640 1 -- 172.21.15.81:0/3783910872 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f00b2710 con 0x7f28f00b2c30 2023-11-08T16:31:04.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.441+0000 7f28e7fff640 1 -- 172.21.15.81:0/3783910872 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1386207130 0 0) 0x7f28f00b2450 con 0x7f28f015d1a0 2023-11-08T16:31:04.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.441+0000 7f28f4dea640 1 --2- 172.21.15.81:0/3783910872 >> v2:172.21.15.93:3300/0 conn(0x7f28f00d1af0 0x7f28f00d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:04.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.442+0000 7f28f5dec640 1 --2- 172.21.15.81:0/3783910872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28f00b2c30 0x7f28f00b30c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f28f00b2640 tx=0x7f28e802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c3235f238762f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:04.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.442+0000 7f28e7fff640 1 -- 172.21.15.81:0/3783910872 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e802cb00 con 0x7f28f00b2c30 2023-11-08T16:31:04.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.442+0000 7f28e7fff640 1 -- 172.21.15.81:0/3783910872 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e802cc60 con 0x7f28f00b2c30 2023-11-08T16:31:04.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.442+0000 7f28e7fff640 1 -- 172.21.15.81:0/3783910872 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e80343f0 con 0x7f28f00b2c30 2023-11-08T16:31:04.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.443+0000 7f28f784f640 1 -- 172.21.15.81:0/3783910872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28f00b2c30 msgr2=0x7f28f00b30c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.443+0000 7f28f784f640 1 --2- 172.21.15.81:0/3783910872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28f00b2c30 0x7f28f00b30c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f28f00b2640 tx=0x7f28e802bf60 comp rx=0 tx=0).stop 2023-11-08T16:31:04.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.443+0000 7f28f784f640 1 -- 172.21.15.81:0/3783910872 shutdown_connections 2023-11-08T16:31:04.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.443+0000 7f28f784f640 1 --2- 172.21.15.81:0/3783910872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28f00b2c30 0x7f28f00b30c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.443+0000 7f28f784f640 1 --2- 172.21.15.81:0/3783910872 >> v2:172.21.15.93:3300/0 conn(0x7f28f00d1af0 0x7f28f00d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.443+0000 7f28f784f640 1 -- 172.21.15.81:0/3783910872 >> 172.21.15.81:0/3783910872 conn(0x7f28f00d2e00 msgr2=0x7f28f00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:04.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.443+0000 7f28f784f640 1 -- 172.21.15.81:0/3783910872 shutdown_connections 2023-11-08T16:31:04.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.443+0000 7f28f784f640 1 -- 172.21.15.81:0/3783910872 wait complete. 2023-11-08T16:31:04.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.445+0000 7f28f784f640 1 Processor -- start 2023-11-08T16:31:04.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.445+0000 7f28f784f640 1 -- start start 2023-11-08T16:31:04.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f784f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28f00b2c30 0x7f28f00b7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f784f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28f00d1af0 0x7f28f00c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f784f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28f015d1a0 0x7f28f00ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f784f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00ca950 con 0x7f28f00d1af0 2023-11-08T16:31:04.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f784f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00cab90 con 0x7f28f00b2c30 2023-11-08T16:31:04.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f784f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28f00cae70 con 0x7f28f015d1a0 2023-11-08T16:31:04.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f4dea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28f00d1af0 0x7f28f00c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:04.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f4dea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28f00d1af0 0x7f28f00c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42480/0 (socket says 172.21.15.81:42480) 2023-11-08T16:31:04.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f4dea640 1 -- 172.21.15.81:0/924644051 learned_addr learned my addr 172.21.15.81:0/924644051 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:04.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f4dea640 1 -- 172.21.15.81:0/924644051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28f015d1a0 msgr2=0x7f28f00ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f4dea640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28f015d1a0 0x7f28f00ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f4dea640 1 -- 172.21.15.81:0/924644051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28f00b2c30 msgr2=0x7f28f00b7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f4dea640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28f00b2c30 0x7f28f00b7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.446+0000 7f28f4dea640 1 -- 172.21.15.81:0/924644051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e8009b30 con 0x7f28f00d1af0 2023-11-08T16:31:04.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.447+0000 7f28f4dea640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28f00d1af0 0x7f28f00c9ed0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f28e000c7f0 tx=0x7f28e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:04.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.447+0000 7f28e5ffb640 1 -- 172.21.15.81:0/924644051 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0007820 con 0x7f28f00d1af0 2023-11-08T16:31:04.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.447+0000 7f28e5ffb640 1 -- 172.21.15.81:0/924644051 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e0007980 con 0x7f28f00d1af0 2023-11-08T16:31:04.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.447+0000 7f28e5ffb640 1 -- 172.21.15.81:0/924644051 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0007d10 con 0x7f28f00d1af0 2023-11-08T16:31:04.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.447+0000 7f28f784f640 1 -- 172.21.15.81:0/924644051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f00cb150 con 0x7f28f00d1af0 2023-11-08T16:31:04.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.447+0000 7f28f784f640 1 -- 172.21.15.81:0/924644051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f00cb710 con 0x7f28f00d1af0 2023-11-08T16:31:04.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.448+0000 7f28f784f640 1 -- 172.21.15.81:0/924644051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f00cb930 con 0x7f28f00d1af0 2023-11-08T16:31:04.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.449+0000 7f28e5ffb640 1 -- 172.21.15.81:0/924644051 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28e00113f0 con 0x7f28f00d1af0 2023-11-08T16:31:04.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.450+0000 7f28e5ffb640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28d406ae70 0x7f28d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.450+0000 7f28e5ffb640 1 -- 172.21.15.81:0/924644051 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f28e0089b00 con 0x7f28f00d1af0 2023-11-08T16:31:04.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.451+0000 7f28f55eb640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28d406ae70 0x7f28d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:04.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.451+0000 7f28f55eb640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28d406ae70 0x7f28d406d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0012a0 tx=0x7f28ec007840 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:04.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.455+0000 7f28e5ffb640 1 -- 172.21.15.81:0/924644051 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28e00549f0 con 0x7f28f00d1af0 2023-11-08T16:31:04.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.647+0000 7f28f784f640 1 -- 172.21.15.81:0/924644051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28f004ece0 con 0x7f28f00d1af0 2023-11-08T16:31:04.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.650+0000 7f28e5ffb640 1 -- 172.21.15.81:0/924644051 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+18297 (secure 0 0 0) 0x7f28e001f090 con 0x7f28f00d1af0 2023-11-08T16:31:04.652 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:04.653 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":38,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:30:42.310793+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:30:27.977481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T16:30:29.171902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:30:40.141171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:31:04.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 -- 172.21.15.81:0/924644051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28d406ae70 msgr2=0x7f28d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28d406ae70 0x7f28d406d330 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0012a0 tx=0x7f28ec007840 comp rx=0 tx=0).stop 2023-11-08T16:31:04.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 -- 172.21.15.81:0/924644051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28f00d1af0 msgr2=0x7f28f00c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28f00d1af0 0x7f28f00c9ed0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f28e000c7f0 tx=0x7f28e000ccb0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 -- 172.21.15.81:0/924644051 shutdown_connections 2023-11-08T16:31:04.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28d406ae70 0x7f28d406d330 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28f015d1a0 0x7f28f00ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28f00d1af0 0x7f28f00c9ed0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 --2- 172.21.15.81:0/924644051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28f00b2c30 0x7f28f00b7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 -- 172.21.15.81:0/924644051 >> 172.21.15.81:0/924644051 conn(0x7f28f00d2e00 msgr2=0x7f28f00b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:04.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 -- 172.21.15.81:0/924644051 shutdown_connections 2023-11-08T16:31:04.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.653+0000 7f28f784f640 1 -- 172.21.15.81:0/924644051 wait complete. 2023-11-08T16:31:04.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:30:27.977481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:30:29.171902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:30:40.141171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:31:04.675 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:31:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.888+0000 7fd80ea90640 1 Processor -- start 2023-11-08T16:31:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.888+0000 7fd80ea90640 1 -- start start 2023-11-08T16:31:04.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.890+0000 7fd80ea90640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd8080ce0d0 0x7fd8080caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.890+0000 7fd80ea90640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd80815bfc0 0x7fd80815eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.890+0000 7fd80ea90640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd80806fa30 con 0x7fd80815f0b0 2023-11-08T16:31:04.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.890+0000 7fd80ea90640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080cda10 con 0x7fd80815bfc0 2023-11-08T16:31:04.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.890+0000 7fd80ea90640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080ce4b0 con 0x7fd8080ce0d0 2023-11-08T16:31:04.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.890+0000 7fd807fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd80815bfc0 0x7fd80815eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:04.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.890+0000 7fd807fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd80815bfc0 0x7fd80815eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42488/0 (socket says 172.21.15.81:42488) 2023-11-08T16:31:04.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.890+0000 7fd807fff640 1 -- 172.21.15.81:0/354751194 learned_addr learned my addr 172.21.15.81:0/354751194 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.891+0000 7fd80c82c640 1 --2- 172.21.15.81:0/354751194 >> v2:172.21.15.93:3300/0 conn(0x7fd8080ce0d0 0x7fd8080caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.891+0000 7fd807fff640 1 -- 172.21.15.81:0/354751194 >> v2:172.21.15.93:3300/0 conn(0x7fd8080ce0d0 msgr2=0x7fd8080caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.891+0000 7fd807fff640 1 --2- 172.21.15.81:0/354751194 >> v2:172.21.15.93:3300/0 conn(0x7fd8080ce0d0 0x7fd8080caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.891+0000 7fd807fff640 1 -- 172.21.15.81:0/354751194 >> v1:172.21.15.81:6789/0 conn(0x7fd80815f0b0 legacy=0x7fd8081516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.891+0000 7fd807fff640 1 -- 172.21.15.81:0/354751194 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80815f490 con 0x7fd80815bfc0 2023-11-08T16:31:04.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.891+0000 7fd80c82c640 1 --2- 172.21.15.81:0/354751194 >> v2:172.21.15.93:3300/0 conn(0x7fd8080ce0d0 0x7fd8080caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:04.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.892+0000 7fd807fff640 1 --2- 172.21.15.81:0/354751194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80815bfc0 0x7fd80815eb70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd80806f3a0 tx=0x7fd7f402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6432cc3ef55f464 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:04.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.892+0000 7fd8077fe640 1 -- 172.21.15.81:0/354751194 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4030870 con 0x7fd80815bfc0 2023-11-08T16:31:04.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.892+0000 7fd8077fe640 1 -- 172.21.15.81:0/354751194 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f40309d0 con 0x7fd80815bfc0 2023-11-08T16:31:04.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.892+0000 7fd8077fe640 1 -- 172.21.15.81:0/354751194 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4030d60 con 0x7fd80815bfc0 2023-11-08T16:31:04.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.893+0000 7fd80ea90640 1 -- 172.21.15.81:0/354751194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80815bfc0 msgr2=0x7fd80815eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.893+0000 7fd80ea90640 1 --2- 172.21.15.81:0/354751194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80815bfc0 0x7fd80815eb70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd80806f3a0 tx=0x7fd7f402b9b0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.893+0000 7fd80ea90640 1 -- 172.21.15.81:0/354751194 shutdown_connections 2023-11-08T16:31:04.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.893+0000 7fd80ea90640 1 --2- 172.21.15.81:0/354751194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80815bfc0 0x7fd80815eb70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.893+0000 7fd80ea90640 1 --2- 172.21.15.81:0/354751194 >> v2:172.21.15.93:3300/0 conn(0x7fd8080ce0d0 0x7fd8080caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.893+0000 7fd80ea90640 1 -- 172.21.15.81:0/354751194 >> 172.21.15.81:0/354751194 conn(0x7fd8080cb580 msgr2=0x7fd8080cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:04.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.893+0000 7fd80ea90640 1 -- 172.21.15.81:0/354751194 shutdown_connections 2023-11-08T16:31:04.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.893+0000 7fd80ea90640 1 -- 172.21.15.81:0/354751194 wait complete. 2023-11-08T16:31:04.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.895+0000 7fd80ea90640 1 Processor -- start 2023-11-08T16:31:04.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.895+0000 7fd80ea90640 1 -- start start 2023-11-08T16:31:04.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.896+0000 7fd80ea90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8080ce0d0 0x7fd808154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.896+0000 7fd80ea90640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd80815bfc0 0x7fd8080c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.896+0000 7fd80ea90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80815f0b0 0x7fd808153530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.896+0000 7fd80ea90640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8080b41b0 con 0x7fd80815f0b0 2023-11-08T16:31:04.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.896+0000 7fd80ea90640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd808153a70 con 0x7fd80815bfc0 2023-11-08T16:31:04.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.896+0000 7fd80ea90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd808153d80 con 0x7fd8080ce0d0 2023-11-08T16:31:04.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.896+0000 7fd80d02d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80815f0b0 0x7fd808153530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:04.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.896+0000 7fd80d02d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80815f0b0 0x7fd808153530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42494/0 (socket says 172.21.15.81:42494) 2023-11-08T16:31:04.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.896+0000 7fd80d02d640 1 -- 172.21.15.81:0/1948547012 learned_addr learned my addr 172.21.15.81:0/1948547012 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:04.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd80c82c640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8080ce0d0 0x7fd808154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:04.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd807fff640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd80815bfc0 0x7fd8080c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:04.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd80d02d640 1 -- 172.21.15.81:0/1948547012 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8080ce0d0 msgr2=0x7fd808154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd80d02d640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8080ce0d0 0x7fd808154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd80d02d640 1 -- 172.21.15.81:0/1948547012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd80815bfc0 msgr2=0x7fd8080c8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:04.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd80d02d640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd80815bfc0 0x7fd8080c8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:04.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd80d02d640 1 -- 172.21.15.81:0/1948547012 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f40094d0 con 0x7fd80815f0b0 2023-11-08T16:31:04.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd807fff640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd80815bfc0 0x7fd8080c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:04.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd80c82c640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8080ce0d0 0x7fd808154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:04.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd80d02d640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80815f0b0 0x7fd808153530 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd80000c6b0 tx=0x7fd80000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:04.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd8057fa640 1 -- 172.21.15.81:0/1948547012 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800003b70 con 0x7fd80815f0b0 2023-11-08T16:31:04.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd8057fa640 1 -- 172.21.15.81:0/1948547012 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd800003cf0 con 0x7fd80815f0b0 2023-11-08T16:31:04.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd8057fa640 1 -- 172.21.15.81:0/1948547012 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd800003ea0 con 0x7fd80815f0b0 2023-11-08T16:31:04.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.897+0000 7fd80ea90640 1 -- 172.21.15.81:0/1948547012 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd808154060 con 0x7fd80815f0b0 2023-11-08T16:31:04.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.898+0000 7fd80ea90640 1 -- 172.21.15.81:0/1948547012 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd808154620 con 0x7fd80815f0b0 2023-11-08T16:31:04.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.899+0000 7fd80ea90640 1 -- 172.21.15.81:0/1948547012 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd808154810 con 0x7fd80815f0b0 2023-11-08T16:31:04.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.900+0000 7fd8057fa640 1 -- 172.21.15.81:0/1948547012 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd800005310 con 0x7fd80815f0b0 2023-11-08T16:31:04.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.901+0000 7fd8057fa640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd7ec06ae70 0x7fd7ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:04.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.901+0000 7fd8057fa640 1 -- 172.21.15.81:0/1948547012 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7fd800089550 con 0x7fd80815f0b0 2023-11-08T16:31:04.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.902+0000 7fd80c82c640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd7ec06ae70 0x7fd7ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:04.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.902+0000 7fd80c82c640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd7ec06ae70 0x7fd7ec06d330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc006fd0 tx=0x7fd7fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:04.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:04.906+0000 7fd8057fa640 1 -- 172.21.15.81:0/1948547012 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd800054450 con 0x7fd80815f0b0 2023-11-08T16:31:05.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.097+0000 7fd80ea90640 1 -- 172.21.15.81:0/1948547012 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd80804ece0 con 0x7fd80815f0b0 2023-11-08T16:31:05.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.098+0000 7fd8057fa640 1 -- 172.21.15.81:0/1948547012 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd800057a70 con 0x7fd80815f0b0 2023-11-08T16:31:05.100 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:31:05.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 -- 172.21.15.81:0/1948547012 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd7ec06ae70 msgr2=0x7fd7ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd7ec06ae70 0x7fd7ec06d330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc006fd0 tx=0x7fd7fc008040 comp rx=0 tx=0).stop 2023-11-08T16:31:05.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 -- 172.21.15.81:0/1948547012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80815f0b0 msgr2=0x7fd808153530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80815f0b0 0x7fd808153530 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fd80000c6b0 tx=0x7fd80000cb70 comp rx=0 tx=0).stop 2023-11-08T16:31:05.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 -- 172.21.15.81:0/1948547012 shutdown_connections 2023-11-08T16:31:05.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd7ec06ae70 0x7fd7ec06d330 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd80815f0b0 0x7fd808153530 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd80815bfc0 0x7fd8080c8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 --2- 172.21.15.81:0/1948547012 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8080ce0d0 0x7fd808154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 -- 172.21.15.81:0/1948547012 >> 172.21.15.81:0/1948547012 conn(0x7fd8080cb580 msgr2=0x7fd8080aeca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:05.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 -- 172.21.15.81:0/1948547012 shutdown_connections 2023-11-08T16:31:05.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.101+0000 7fd80ea90640 1 -- 172.21.15.81:0/1948547012 wait complete. 2023-11-08T16:31:05.122 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:31:05.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.326+0000 7fee759f5640 1 Processor -- start 2023-11-08T16:31:05.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.326+0000 7fee759f5640 1 -- start start 2023-11-08T16:31:05.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.328+0000 7fee759f5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fee7015d4c0 0x7fee700cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:05.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.328+0000 7fee759f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee700d39a0 0x7fee700cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:05.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.328+0000 7fee759f5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee700cef70 con 0x7fee700cfaf0 2023-11-08T16:31:05.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.328+0000 7fee759f5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee700cff80 con 0x7fee7015d4c0 2023-11-08T16:31:05.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.328+0000 7fee759f5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee700cec90 con 0x7fee700d39a0 2023-11-08T16:31:05.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.328+0000 7fee6f7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fee700cfaf0 0x7fee700d0d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45338/0 (socket says 172.21.15.81:45338) 2023-11-08T16:31:05.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.328+0000 7fee6f7fe640 1 -- 172.21.15.81:0/905930639 learned_addr learned my addr 172.21.15.81:0/905930639 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:05.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.328+0000 7fee6effd640 1 --2- 172.21.15.81:0/905930639 >> v2:172.21.15.93:3300/0 conn(0x7fee7015d4c0 0x7fee700cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:05.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.329+0000 7fee6e7fc640 1 --2- 172.21.15.81:0/905930639 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee700d39a0 0x7fee700cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:05.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.329+0000 7fee6dffb640 1 -- 172.21.15.81:0/905930639 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4288295610 0 0) 0x7fee68001490 con 0x7fee700cfaf0 2023-11-08T16:31:05.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.329+0000 7fee6dffb640 1 -- 172.21.15.81:0/905930639 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee58003520 con 0x7fee700cfaf0 2023-11-08T16:31:05.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.329+0000 7fee6effd640 1 -- 172.21.15.81:0/905930639 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee700d39a0 msgr2=0x7fee700cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:05.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.329+0000 7fee6effd640 1 --2- 172.21.15.81:0/905930639 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee700d39a0 0x7fee700cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.329+0000 7fee6effd640 1 -- 172.21.15.81:0/905930639 >> v1:172.21.15.81:6789/0 conn(0x7fee700cfaf0 legacy=0x7fee700d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.329+0000 7fee6effd640 1 -- 172.21.15.81:0/905930639 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee700b5e70 con 0x7fee7015d4c0 2023-11-08T16:31:05.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.331+0000 7fee6effd640 1 --2- 172.21.15.81:0/905930639 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee7015d4c0 0x7fee700cdb40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee600097a0 tx=0x7fee6002b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c4f556f87a781e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:05.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.331+0000 7fee6dffb640 1 -- 172.21.15.81:0/905930639 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60030800 con 0x7fee7015d4c0 2023-11-08T16:31:05.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.331+0000 7fee6dffb640 1 -- 172.21.15.81:0/905930639 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee60030960 con 0x7fee7015d4c0 2023-11-08T16:31:05.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.331+0000 7fee6dffb640 1 -- 172.21.15.81:0/905930639 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee60030cf0 con 0x7fee7015d4c0 2023-11-08T16:31:05.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.332+0000 7fee759f5640 1 -- 172.21.15.81:0/905930639 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee7015d4c0 msgr2=0x7fee700cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.332+0000 7fee759f5640 1 --2- 172.21.15.81:0/905930639 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee7015d4c0 0x7fee700cdb40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fee600097a0 tx=0x7fee6002b940 comp rx=0 tx=0).stop 2023-11-08T16:31:05.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.332+0000 7fee759f5640 1 -- 172.21.15.81:0/905930639 shutdown_connections 2023-11-08T16:31:05.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.332+0000 7fee759f5640 1 --2- 172.21.15.81:0/905930639 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee700d39a0 0x7fee700cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.332+0000 7fee759f5640 1 --2- 172.21.15.81:0/905930639 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee7015d4c0 0x7fee700cdb40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.332+0000 7fee759f5640 1 -- 172.21.15.81:0/905930639 >> 172.21.15.81:0/905930639 conn(0x7fee700d2e80 msgr2=0x7fee700d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:05.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.332+0000 7fee759f5640 1 -- 172.21.15.81:0/905930639 shutdown_connections 2023-11-08T16:31:05.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.332+0000 7fee759f5640 1 -- 172.21.15.81:0/905930639 wait complete. 2023-11-08T16:31:05.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.333+0000 7fee759f5640 1 Processor -- start 2023-11-08T16:31:05.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.334+0000 7fee759f5640 1 -- start start 2023-11-08T16:31:05.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.334+0000 7fee759f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee700cfaf0 0x7fee70156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:05.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee759f5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee700d39a0 0x7fee70151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:05.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee759f5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee7015d4c0 0x7fee7015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:05.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee759f5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee70156950 con 0x7fee700d39a0 2023-11-08T16:31:05.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee759f5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee700c97a0 con 0x7fee7015d4c0 2023-11-08T16:31:05.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee759f5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee701f3bd0 con 0x7fee700cfaf0 2023-11-08T16:31:05.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee700d39a0 0x7fee70151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:05.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee6e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee700d39a0 0x7fee70151060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42508/0 (socket says 172.21.15.81:42508) 2023-11-08T16:31:05.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee6e7fc640 1 -- 172.21.15.81:0/3939162331 learned_addr learned my addr 172.21.15.81:0/3939162331 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:05.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee6e7fc640 1 -- 172.21.15.81:0/3939162331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee700cfaf0 msgr2=0x7fee70156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee6effd640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee700cfaf0 0x7fee70156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:05.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee6e7fc640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee700cfaf0 0x7fee70156de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee6e7fc640 1 -- 172.21.15.81:0/3939162331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee7015d4c0 msgr2=0x7fee7015a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee6e7fc640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee7015d4c0 0x7fee7015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.335+0000 7fee6e7fc640 1 -- 172.21.15.81:0/3939162331 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee60009400 con 0x7fee700d39a0 2023-11-08T16:31:05.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.336+0000 7fee6e7fc640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee700d39a0 0x7fee70151060 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fee6400e9b0 tx=0x7fee6400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:05.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.336+0000 7fee57fff640 1 -- 172.21.15.81:0/3939162331 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee64009670 con 0x7fee700d39a0 2023-11-08T16:31:05.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.336+0000 7fee57fff640 1 -- 172.21.15.81:0/3939162331 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6400c9a0 con 0x7fee700d39a0 2023-11-08T16:31:05.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.336+0000 7fee57fff640 1 -- 172.21.15.81:0/3939162331 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6400cd50 con 0x7fee700d39a0 2023-11-08T16:31:05.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.336+0000 7fee759f5640 1 -- 172.21.15.81:0/3939162331 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee70156300 con 0x7fee700d39a0 2023-11-08T16:31:05.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.336+0000 7fee759f5640 1 -- 172.21.15.81:0/3939162331 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee701f3eb0 con 0x7fee700d39a0 2023-11-08T16:31:05.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.337+0000 7fee759f5640 1 -- 172.21.15.81:0/3939162331 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee700d2860 con 0x7fee700d39a0 2023-11-08T16:31:05.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.338+0000 7fee57fff640 1 -- 172.21.15.81:0/3939162331 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee64013470 con 0x7fee700d39a0 2023-11-08T16:31:05.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.339+0000 7fee57fff640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee5c06ae70 0x7fee5c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:05.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.340+0000 7fee57fff640 1 -- 172.21.15.81:0/3939162331 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7fee640899c0 con 0x7fee700d39a0 2023-11-08T16:31:05.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.340+0000 7fee6effd640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee5c06ae70 0x7fee5c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:05.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.340+0000 7fee6effd640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee5c06ae70 0x7fee5c06d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fee60036000 tx=0x7fee6003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:05.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.344+0000 7fee57fff640 1 -- 172.21.15.81:0/3939162331 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee640548d0 con 0x7fee700d39a0 2023-11-08T16:31:05.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.538+0000 7fee759f5640 1 -- 172.21.15.81:0/3939162331 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee7004ece0 con 0x7fee700d39a0 2023-11-08T16:31:05.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.539+0000 7fee57fff640 1 -- 172.21.15.81:0/3939162331 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee64017020 con 0x7fee700d39a0 2023-11-08T16:31:05.541 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:31:05.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 -- 172.21.15.81:0/3939162331 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee5c06ae70 msgr2=0x7fee5c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee5c06ae70 0x7fee5c06d330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fee60036000 tx=0x7fee6003b040 comp rx=0 tx=0).stop 2023-11-08T16:31:05.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 -- 172.21.15.81:0/3939162331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee700d39a0 msgr2=0x7fee70151060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee700d39a0 0x7fee70151060 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fee6400e9b0 tx=0x7fee6400ee70 comp rx=0 tx=0).stop 2023-11-08T16:31:05.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 -- 172.21.15.81:0/3939162331 shutdown_connections 2023-11-08T16:31:05.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee5c06ae70 0x7fee5c06d330 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee7015d4c0 0x7fee7015a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee700d39a0 0x7fee70151060 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 --2- 172.21.15.81:0/3939162331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee700cfaf0 0x7fee70156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 -- 172.21.15.81:0/3939162331 >> 172.21.15.81:0/3939162331 conn(0x7fee700d2e80 msgr2=0x7fee700b6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:05.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 -- 172.21.15.81:0/3939162331 shutdown_connections 2023-11-08T16:31:05.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.542+0000 7fee759f5640 1 -- 172.21.15.81:0/3939162331 wait complete. 2023-11-08T16:31:05.563 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:31:05.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.779+0000 7f30ce41a640 1 Processor -- start 2023-11-08T16:31:05.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.779+0000 7f30ce41a640 1 -- start start 2023-11-08T16:31:05.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.780+0000 7f30ce41a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f30c815d4c0 0x7f30c80cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:05.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.780+0000 7f30ce41a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c80cef70 con 0x7f30c80cfcc0 2023-11-08T16:31:05.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.780+0000 7f30ce41a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c80cec90 con 0x7f30c80d39a0 2023-11-08T16:31:05.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.780+0000 7f30ce41a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c80b5e70 con 0x7f30c815d4c0 2023-11-08T16:31:05.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.780+0000 7f30cc9b7640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f30c80cfcc0 0x7f30c80b6950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45352/0 (socket says 172.21.15.81:45352) 2023-11-08T16:31:05.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.780+0000 7f30cc9b7640 1 -- 172.21.15.81:0/184854584 learned_addr learned my addr 172.21.15.81:0/184854584 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:05.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.780+0000 7f30c7fff640 1 --2- 172.21.15.81:0/184854584 >> v2:172.21.15.81:3300/0 conn(0x7f30c815d4c0 0x7f30c80cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:05.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.781+0000 7f30c6ffd640 1 -- 172.21.15.81:0/184854584 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3679376659 0 0) 0x7f30c0001490 con 0x7f30c80cfcc0 2023-11-08T16:31:05.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.781+0000 7f30c6ffd640 1 -- 172.21.15.81:0/184854584 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30b0003520 con 0x7f30c80cfcc0 2023-11-08T16:31:05.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.781+0000 7f30c7fff640 1 -- 172.21.15.81:0/184854584 >> v1:172.21.15.93:6789/0 conn(0x7f30c80d39a0 legacy=0x7f30c80cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.781+0000 7f30c7fff640 1 -- 172.21.15.81:0/184854584 >> v1:172.21.15.81:6789/0 conn(0x7f30c80cfcc0 legacy=0x7f30c80b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.781+0000 7f30c7fff640 1 -- 172.21.15.81:0/184854584 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c80b6150 con 0x7f30c815d4c0 2023-11-08T16:31:05.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.782+0000 7f30c7fff640 1 --2- 172.21.15.81:0/184854584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30c815d4c0 0x7f30c80cdb40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f30b80097a0 tx=0x7f30b802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dead66e019c8134c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:05.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.782+0000 7f30c6ffd640 1 -- 172.21.15.81:0/184854584 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b802f810 con 0x7f30c815d4c0 2023-11-08T16:31:05.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.782+0000 7f30c6ffd640 1 -- 172.21.15.81:0/184854584 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b802f970 con 0x7f30c815d4c0 2023-11-08T16:31:05.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.782+0000 7f30c6ffd640 1 -- 172.21.15.81:0/184854584 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b802fd00 con 0x7f30c815d4c0 2023-11-08T16:31:05.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.783+0000 7f30ce41a640 1 -- 172.21.15.81:0/184854584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30c815d4c0 msgr2=0x7f30c80cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.783+0000 7f30ce41a640 1 --2- 172.21.15.81:0/184854584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30c815d4c0 0x7f30c80cdb40 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f30b80097a0 tx=0x7f30b802b9f0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.783+0000 7f30ce41a640 1 -- 172.21.15.81:0/184854584 shutdown_connections 2023-11-08T16:31:05.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.783+0000 7f30ce41a640 1 --2- 172.21.15.81:0/184854584 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30c815d4c0 0x7f30c80cdb40 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.783+0000 7f30ce41a640 1 -- 172.21.15.81:0/184854584 >> 172.21.15.81:0/184854584 conn(0x7f30c80d2e80 msgr2=0x7f30c80d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:05.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.783+0000 7f30ce41a640 1 -- 172.21.15.81:0/184854584 shutdown_connections 2023-11-08T16:31:05.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.783+0000 7f30ce41a640 1 -- 172.21.15.81:0/184854584 wait complete. 2023-11-08T16:31:05.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.785+0000 7f30ce41a640 1 Processor -- start 2023-11-08T16:31:05.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.785+0000 7f30ce41a640 1 -- start start 2023-11-08T16:31:05.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.786+0000 7f30ce41a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30c80cfcc0 0x7f30c80b6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:05.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.786+0000 7f30ce41a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f30c80d39a0 0x7f30c81f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:05.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.786+0000 7f30ce41a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30c815d4c0 0x7f30c81f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:05.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.786+0000 7f30ce41a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c80ca6b0 con 0x7f30c80cfcc0 2023-11-08T16:31:05.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.786+0000 7f30ce41a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c81f7400 con 0x7f30c815d4c0 2023-11-08T16:31:05.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.786+0000 7f30ce41a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c81f76e0 con 0x7f30c80d39a0 2023-11-08T16:31:05.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.786+0000 7f30c7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30c80cfcc0 0x7f30c80b6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:05.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.786+0000 7f30c7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30c80cfcc0 0x7f30c80b6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42524/0 (socket says 172.21.15.81:42524) 2023-11-08T16:31:05.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.786+0000 7f30c7fff640 1 -- 172.21.15.81:0/3607084579 learned_addr learned my addr 172.21.15.81:0/3607084579 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:05.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.787+0000 7f30c7fff640 1 -- 172.21.15.81:0/3607084579 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f30c80d39a0 msgr2=0x7f30c81f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.787+0000 7f30c7fff640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f30c80d39a0 0x7f30c81f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.787+0000 7f30c7fff640 1 -- 172.21.15.81:0/3607084579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30c815d4c0 msgr2=0x7f30c81f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.787+0000 7f30c7fff640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30c815d4c0 0x7f30c81f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.787+0000 7f30c7fff640 1 -- 172.21.15.81:0/3607084579 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b8009400 con 0x7f30c80cfcc0 2023-11-08T16:31:05.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.787+0000 7f30c7fff640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30c80cfcc0 0x7f30c80b6ac0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f30b80098c0 tx=0x7f30b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.787+0000 7f30c4ff9640 1 -- 172.21.15.81:0/3607084579 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b803c040 con 0x7f30c80cfcc0 2023-11-08T16:31:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.787+0000 7f30c4ff9640 1 -- 172.21.15.81:0/3607084579 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b80378f0 con 0x7f30c80cfcc0 2023-11-08T16:31:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.787+0000 7f30c4ff9640 1 -- 172.21.15.81:0/3607084579 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b8037ca0 con 0x7f30c80cfcc0 2023-11-08T16:31:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.787+0000 7f30ce41a640 1 -- 172.21.15.81:0/3607084579 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c80c9ff0 con 0x7f30c80cfcc0 2023-11-08T16:31:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.788+0000 7f30ce41a640 1 -- 172.21.15.81:0/3607084579 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c8158e90 con 0x7f30c80cfcc0 2023-11-08T16:31:05.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.789+0000 7f30ce41a640 1 -- 172.21.15.81:0/3607084579 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c80d2860 con 0x7f30c80cfcc0 2023-11-08T16:31:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.789+0000 7f30c4ff9640 1 -- 172.21.15.81:0/3607084579 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f30b802d3a0 con 0x7f30c80cfcc0 2023-11-08T16:31:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.791+0000 7f30c4ff9640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30b406ae70 0x7f30b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.791+0000 7f30c4ff9640 1 -- 172.21.15.81:0/3607084579 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f30b80ab420 con 0x7f30c80cfcc0 2023-11-08T16:31:05.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.791+0000 7f30c77fe640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30b406ae70 0x7f30b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:05.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.792+0000 7f30c77fe640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30b406ae70 0x7f30b406d330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30bc000f60 tx=0x7f30bc0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:05.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.795+0000 7f30c4ff9640 1 -- 172.21.15.81:0/3607084579 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30b8076380 con 0x7f30c80cfcc0 2023-11-08T16:31:05.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.986+0000 7f30ce41a640 1 -- 172.21.15.81:0/3607084579 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f30c804ece0 con 0x7f30c80cfcc0 2023-11-08T16:31:05.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.987+0000 7f30c4ff9640 1 -- 172.21.15.81:0/3607084579 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7f30b80799a0 con 0x7f30c80cfcc0 2023-11-08T16:31:05.989 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:31:05.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 -- 172.21.15.81:0/3607084579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30b406ae70 msgr2=0x7f30b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30b406ae70 0x7f30b406d330 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f30bc000f60 tx=0x7f30bc0079b0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 -- 172.21.15.81:0/3607084579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30c80cfcc0 msgr2=0x7f30c80b6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:05.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30c80cfcc0 0x7f30c80b6ac0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f30b80098c0 tx=0x7f30b8004730 comp rx=0 tx=0).stop 2023-11-08T16:31:05.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 -- 172.21.15.81:0/3607084579 shutdown_connections 2023-11-08T16:31:05.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30b406ae70 0x7f30b406d330 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30c815d4c0 0x7f30c81f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f30c80d39a0 0x7f30c81f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 --2- 172.21.15.81:0/3607084579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30c80cfcc0 0x7f30c80b6ac0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:05.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 -- 172.21.15.81:0/3607084579 >> 172.21.15.81:0/3607084579 conn(0x7f30c80d2e80 msgr2=0x7f30c806bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:05.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 -- 172.21.15.81:0/3607084579 shutdown_connections 2023-11-08T16:31:05.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:05.990+0000 7f30ce41a640 1 -- 172.21.15.81:0/3607084579 wait complete. 2023-11-08T16:31:06.011 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:31:06.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.227+0000 7ff68772c640 1 Processor -- start 2023-11-08T16:31:06.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.227+0000 7ff68772c640 1 -- start start 2023-11-08T16:31:06.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.228+0000 7ff68772c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff6800ce0d0 0x7ff6800caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff68772c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff68015bfc0 0x7ff68015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff68772c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68006fa30 con 0x7ff68015f0b0 2023-11-08T16:31:06.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff68772c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6800cda10 con 0x7ff68015bfc0 2023-11-08T16:31:06.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff68772c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6800ce4b0 con 0x7ff6800ce0d0 2023-11-08T16:31:06.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff684cc7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff68015bfc0 0x7ff68015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:06.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff685cc9640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff68015f0b0 0x7ff6801516e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45356/0 (socket says 172.21.15.81:45356) 2023-11-08T16:31:06.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff685cc9640 1 -- 172.21.15.81:0/4079679270 learned_addr learned my addr 172.21.15.81:0/4079679270 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:06.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff6854c8640 1 --2- 172.21.15.81:0/4079679270 >> v2:172.21.15.93:3300/0 conn(0x7ff6800ce0d0 0x7ff6800caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:06.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff684cc7640 1 -- 172.21.15.81:0/4079679270 >> v2:172.21.15.93:3300/0 conn(0x7ff6800ce0d0 msgr2=0x7ff6800caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:06.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff684cc7640 1 --2- 172.21.15.81:0/4079679270 >> v2:172.21.15.93:3300/0 conn(0x7ff6800ce0d0 0x7ff6800caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff684cc7640 1 -- 172.21.15.81:0/4079679270 >> v1:172.21.15.81:6789/0 conn(0x7ff68015f0b0 legacy=0x7ff6801516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:06.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff684cc7640 1 -- 172.21.15.81:0/4079679270 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68015f490 con 0x7ff68015bfc0 2023-11-08T16:31:06.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.229+0000 7ff677fff640 1 -- 172.21.15.81:0/4079679270 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296915504 0 0) 0x7ff68006fa30 con 0x7ff68015f0b0 2023-11-08T16:31:06.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.230+0000 7ff684cc7640 1 --2- 172.21.15.81:0/4079679270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff68015bfc0 0x7ff68015eb70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff68006f3a0 tx=0x7ff67002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcf5c9306525672 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:06.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.231+0000 7ff677fff640 1 -- 172.21.15.81:0/4079679270 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670031830 con 0x7ff68015bfc0 2023-11-08T16:31:06.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.231+0000 7ff677fff640 1 -- 172.21.15.81:0/4079679270 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff670031990 con 0x7ff68015bfc0 2023-11-08T16:31:06.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.231+0000 7ff677fff640 1 -- 172.21.15.81:0/4079679270 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670031d20 con 0x7ff68015bfc0 2023-11-08T16:31:06.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.231+0000 7ff68772c640 1 -- 172.21.15.81:0/4079679270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff68015bfc0 msgr2=0x7ff68015eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:06.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.231+0000 7ff68772c640 1 --2- 172.21.15.81:0/4079679270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff68015bfc0 0x7ff68015eb70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7ff68006f3a0 tx=0x7ff67002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.231+0000 7ff68772c640 1 -- 172.21.15.81:0/4079679270 shutdown_connections 2023-11-08T16:31:06.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.231+0000 7ff68772c640 1 --2- 172.21.15.81:0/4079679270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff68015bfc0 0x7ff68015eb70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.231+0000 7ff68772c640 1 --2- 172.21.15.81:0/4079679270 >> v2:172.21.15.93:3300/0 conn(0x7ff6800ce0d0 0x7ff6800caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.231+0000 7ff68772c640 1 -- 172.21.15.81:0/4079679270 >> 172.21.15.81:0/4079679270 conn(0x7ff6800cb580 msgr2=0x7ff6800cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:06.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.232+0000 7ff68772c640 1 -- 172.21.15.81:0/4079679270 shutdown_connections 2023-11-08T16:31:06.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.232+0000 7ff68772c640 1 -- 172.21.15.81:0/4079679270 wait complete. 2023-11-08T16:31:06.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.233+0000 7ff68772c640 1 Processor -- start 2023-11-08T16:31:06.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.233+0000 7ff68772c640 1 -- start start 2023-11-08T16:31:06.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.234+0000 7ff68772c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6800ce0d0 0x7ff6801f0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.234+0000 7ff68772c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff68015bfc0 0x7ff6801f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.234+0000 7ff68772c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68015f0b0 0x7ff6801f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.234+0000 7ff68772c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6801f0660 con 0x7ff6800ce0d0 2023-11-08T16:31:06.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.234+0000 7ff68772c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6800b35a0 con 0x7ff68015f0b0 2023-11-08T16:31:06.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.234+0000 7ff68772c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6800c83d0 con 0x7ff68015bfc0 2023-11-08T16:31:06.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.235+0000 7ff6854c8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6800ce0d0 0x7ff6801f0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:06.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.235+0000 7ff6854c8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6800ce0d0 0x7ff6801f0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42546/0 (socket says 172.21.15.81:42546) 2023-11-08T16:31:06.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.235+0000 7ff6854c8640 1 -- 172.21.15.81:0/3757102765 learned_addr learned my addr 172.21.15.81:0/3757102765 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:06.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.235+0000 7ff6854c8640 1 -- 172.21.15.81:0/3757102765 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff68015bfc0 msgr2=0x7ff6801f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:06.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.235+0000 7ff6854c8640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff68015bfc0 0x7ff6801f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.235+0000 7ff6854c8640 1 -- 172.21.15.81:0/3757102765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68015f0b0 msgr2=0x7ff6801f4250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:06.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.235+0000 7ff6854c8640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68015f0b0 0x7ff6801f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.235+0000 7ff6854c8640 1 -- 172.21.15.81:0/3757102765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff670009400 con 0x7ff6800ce0d0 2023-11-08T16:31:06.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.235+0000 7ff6854c8640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6800ce0d0 0x7ff6801f0120 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c7f0 tx=0x7ff67c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:06.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.235+0000 7ff675ffb640 1 -- 172.21.15.81:0/3757102765 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c004220 con 0x7ff6800ce0d0 2023-11-08T16:31:06.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.236+0000 7ff675ffb640 1 -- 172.21.15.81:0/3757102765 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67c0043a0 con 0x7ff6800ce0d0 2023-11-08T16:31:06.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.236+0000 7ff68772c640 1 -- 172.21.15.81:0/3757102765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6800c86b0 con 0x7ff6800ce0d0 2023-11-08T16:31:06.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.236+0000 7ff675ffb640 1 -- 172.21.15.81:0/3757102765 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67c007860 con 0x7ff6800ce0d0 2023-11-08T16:31:06.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.236+0000 7ff68772c640 1 -- 172.21.15.81:0/3757102765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6800b17d0 con 0x7ff6800ce0d0 2023-11-08T16:31:06.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.237+0000 7ff68772c640 1 -- 172.21.15.81:0/3757102765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6800b19c0 con 0x7ff6800ce0d0 2023-11-08T16:31:06.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.238+0000 7ff675ffb640 1 -- 172.21.15.81:0/3757102765 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff67c007bd0 con 0x7ff6800ce0d0 2023-11-08T16:31:06.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.239+0000 7ff675ffb640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff66406ae70 0x7ff66406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.239+0000 7ff675ffb640 1 -- 172.21.15.81:0/3757102765 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7ff67c089810 con 0x7ff6800ce0d0 2023-11-08T16:31:06.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.240+0000 7ff684cc7640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff66406ae70 0x7ff66406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:06.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.240+0000 7ff684cc7640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff66406ae70 0x7ff66406d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff670036000 tx=0x7ff670030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:06.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.244+0000 7ff675ffb640 1 -- 172.21.15.81:0/3757102765 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff67c054680 con 0x7ff6800ce0d0 2023-11-08T16:31:06.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.433+0000 7ff68772c640 1 -- 172.21.15.81:0/3757102765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7ff68004ece0 con 0x7ff6800ce0d0 2023-11-08T16:31:06.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.434+0000 7ff675ffb640 1 -- 172.21.15.81:0/3757102765 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v38) v1 ==== 106+0+11 (secure 0 0 0) 0x7ff67c01e020 con 0x7ff6800ce0d0 2023-11-08T16:31:06.436 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:31:06.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 -- 172.21.15.81:0/3757102765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff66406ae70 msgr2=0x7ff66406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:06.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff66406ae70 0x7ff66406d330 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff670036000 tx=0x7ff670030000 comp rx=0 tx=0).stop 2023-11-08T16:31:06.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 -- 172.21.15.81:0/3757102765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6800ce0d0 msgr2=0x7ff6801f0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:06.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6800ce0d0 0x7ff6801f0120 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00c7f0 tx=0x7ff67c00ccb0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 -- 172.21.15.81:0/3757102765 shutdown_connections 2023-11-08T16:31:06.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff66406ae70 0x7ff66406d330 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff68015f0b0 0x7ff6801f4250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff68015bfc0 0x7ff6801f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 --2- 172.21.15.81:0/3757102765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6800ce0d0 0x7ff6801f0120 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 -- 172.21.15.81:0/3757102765 >> 172.21.15.81:0/3757102765 conn(0x7ff6800cb580 msgr2=0x7ff68015e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:06.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 -- 172.21.15.81:0/3757102765 shutdown_connections 2023-11-08T16:31:06.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.437+0000 7ff68772c640 1 -- 172.21.15.81:0/3757102765 wait complete. 2023-11-08T16:31:06.456 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:31:06.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.666+0000 7f9b3b303640 1 Processor -- start 2023-11-08T16:31:06.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.666+0000 7f9b3b303640 1 -- start start 2023-11-08T16:31:06.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.667+0000 7f9b3b303640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b3415ac20 0x7f9b340cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.667+0000 7f9b3b303640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9b3406f7c0 0x7f9b34070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.667+0000 7f9b3b303640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9b340ad940 0x7f9b34071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.667+0000 7f9b3b303640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b340ae130 con 0x7f9b340ad940 2023-11-08T16:31:06.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.667+0000 7f9b3b303640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b340cb370 con 0x7f9b3406f7c0 2023-11-08T16:31:06.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.667+0000 7f9b3b303640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b340719b0 con 0x7f9b3415ac20 2023-11-08T16:31:06.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.668+0000 7f9b3ab02640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9b340ad940 0x7f9b34071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:06.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.668+0000 7f9b3ab02640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9b340ad940 0x7f9b34071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42550/0 (socket says 172.21.15.81:42550) 2023-11-08T16:31:06.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.668+0000 7f9b3ab02640 1 -- 172.21.15.81:0/4166811967 learned_addr learned my addr 172.21.15.81:0/4166811967 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:06.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.668+0000 7f9b3a301640 1 --2- 172.21.15.81:0/4166811967 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b3415ac20 0x7f9b340cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:06.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.668+0000 7f9b3ab02640 1 -- 172.21.15.81:0/4166811967 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b3415ac20 msgr2=0x7f9b340cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:06.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.668+0000 7f9b3ab02640 1 --2- 172.21.15.81:0/4166811967 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b3415ac20 0x7f9b340cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.668+0000 7f9b3ab02640 1 -- 172.21.15.81:0/4166811967 >> v2:172.21.15.93:3300/0 conn(0x7f9b3406f7c0 msgr2=0x7f9b34070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:06.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.668+0000 7f9b39b00640 1 --2- 172.21.15.81:0/4166811967 >> v2:172.21.15.93:3300/0 conn(0x7f9b3406f7c0 0x7f9b34070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:06.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.668+0000 7f9b3ab02640 1 --2- 172.21.15.81:0/4166811967 >> v2:172.21.15.93:3300/0 conn(0x7f9b3406f7c0 0x7f9b34070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.668+0000 7f9b3ab02640 1 -- 172.21.15.81:0/4166811967 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3406cd70 con 0x7f9b340ad940 2023-11-08T16:31:06.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.669+0000 7f9b3ab02640 1 --2- 172.21.15.81:0/4166811967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b340ad940 0x7f9b34071470 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9b3406eb40 tx=0x7f9b2c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a220b5dd9a9b5d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:06.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b392ff640 1 -- 172.21.15.81:0/4166811967 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c02c9e0 con 0x7f9b340ad940 2023-11-08T16:31:06.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b392ff640 1 -- 172.21.15.81:0/4166811967 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b2c02cb40 con 0x7f9b340ad940 2023-11-08T16:31:06.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b392ff640 1 -- 172.21.15.81:0/4166811967 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c0343f0 con 0x7f9b340ad940 2023-11-08T16:31:06.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b3b303640 1 -- 172.21.15.81:0/4166811967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b340ad940 msgr2=0x7f9b34071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:06.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b3b303640 1 --2- 172.21.15.81:0/4166811967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b340ad940 0x7f9b34071470 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9b3406eb40 tx=0x7f9b2c02be40 comp rx=0 tx=0).stop 2023-11-08T16:31:06.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b3b303640 1 -- 172.21.15.81:0/4166811967 shutdown_connections 2023-11-08T16:31:06.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b3b303640 1 --2- 172.21.15.81:0/4166811967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b340ad940 0x7f9b34071470 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b3b303640 1 --2- 172.21.15.81:0/4166811967 >> v2:172.21.15.93:3300/0 conn(0x7f9b3406f7c0 0x7f9b34070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b3b303640 1 --2- 172.21.15.81:0/4166811967 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b3415ac20 0x7f9b340cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b3b303640 1 -- 172.21.15.81:0/4166811967 >> 172.21.15.81:0/4166811967 conn(0x7f9b340b04d0 msgr2=0x7f9b340b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:06.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b3b303640 1 -- 172.21.15.81:0/4166811967 shutdown_connections 2023-11-08T16:31:06.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.670+0000 7f9b3b303640 1 -- 172.21.15.81:0/4166811967 wait complete. 2023-11-08T16:31:06.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.672+0000 7f9b3b303640 1 Processor -- start 2023-11-08T16:31:06.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.672+0000 7f9b3b303640 1 -- start start 2023-11-08T16:31:06.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.673+0000 7f9b3b303640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b3406f7c0 0x7f9b3414cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.673+0000 7f9b3b303640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b340ad940 0x7f9b340c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.673+0000 7f9b3b303640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b3415ac20 0x7f9b340c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.673+0000 7f9b3b303640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b340c9a90 con 0x7f9b3406f7c0 2023-11-08T16:31:06.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.673+0000 7f9b3b303640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b340c8f10 con 0x7f9b340ad940 2023-11-08T16:31:06.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.673+0000 7f9b3b303640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b340c91f0 con 0x7f9b3415ac20 2023-11-08T16:31:06.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.673+0000 7f9b3a301640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b3406f7c0 0x7f9b3414cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:06.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.673+0000 7f9b3a301640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b3406f7c0 0x7f9b3414cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42562/0 (socket says 172.21.15.81:42562) 2023-11-08T16:31:06.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b3a301640 1 -- 172.21.15.81:0/1158560163 learned_addr learned my addr 172.21.15.81:0/1158560163 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:06.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b39b00640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b340ad940 0x7f9b340c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:06.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b3a301640 1 -- 172.21.15.81:0/1158560163 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b3415ac20 msgr2=0x7f9b340c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:06.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b3a301640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b3415ac20 0x7f9b340c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b3a301640 1 -- 172.21.15.81:0/1158560163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b340ad940 msgr2=0x7f9b340c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:06.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b3a301640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b340ad940 0x7f9b340c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b3a301640 1 -- 172.21.15.81:0/1158560163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b2c009b30 con 0x7f9b3406f7c0 2023-11-08T16:31:06.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b39b00640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b340ad940 0x7f9b340c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:06.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b3a301640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b3406f7c0 0x7f9b3414cee0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000c7f0 tx=0x7f9b3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:06.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b2affd640 1 -- 172.21.15.81:0/1158560163 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30004220 con 0x7f9b3406f7c0 2023-11-08T16:31:06.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b2affd640 1 -- 172.21.15.81:0/1158560163 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b300043a0 con 0x7f9b3406f7c0 2023-11-08T16:31:06.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b2affd640 1 -- 172.21.15.81:0/1158560163 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30007860 con 0x7f9b3406f7c0 2023-11-08T16:31:06.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.674+0000 7f9b3b303640 1 -- 172.21.15.81:0/1158560163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b340c94d0 con 0x7f9b3406f7c0 2023-11-08T16:31:06.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.675+0000 7f9b3b303640 1 -- 172.21.15.81:0/1158560163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b341e0990 con 0x7f9b3406f7c0 2023-11-08T16:31:06.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.676+0000 7f9b3b303640 1 -- 172.21.15.81:0/1158560163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3404ece0 con 0x7f9b3406f7c0 2023-11-08T16:31:06.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.677+0000 7f9b2affd640 1 -- 172.21.15.81:0/1158560163 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9b30007bd0 con 0x7f9b3406f7c0 2023-11-08T16:31:06.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.677+0000 7f9b2affd640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b1806ae70 0x7f9b1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:06.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.678+0000 7f9b2affd640 1 -- 172.21.15.81:0/1158560163 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f9b300896a0 con 0x7f9b3406f7c0 2023-11-08T16:31:06.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.678+0000 7f9b39b00640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b1806ae70 0x7f9b1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:06.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.678+0000 7f9b39b00640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b1806ae70 0x7f9b1806d330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9b3406cb60 tx=0x7f9b24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:06.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.682+0000 7f9b2affd640 1 -- 172.21.15.81:0/1158560163 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b30054640 con 0x7f9b3406f7c0 2023-11-08T16:31:06.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.885+0000 7f9b3b303640 1 -- 172.21.15.81:0/1158560163 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9b3406c640 con 0x7f9b3406f7c0 2023-11-08T16:31:06.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.885+0000 7f9b2affd640 1 -- 172.21.15.81:0/1158560163 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9b30057c60 con 0x7f9b3406f7c0 2023-11-08T16:31:06.887 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:31:06.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 -- 172.21.15.81:0/1158560163 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b1806ae70 msgr2=0x7f9b1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:06.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b1806ae70 0x7f9b1806d330 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9b3406cb60 tx=0x7f9b24008040 comp rx=0 tx=0).stop 2023-11-08T16:31:06.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 -- 172.21.15.81:0/1158560163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b3406f7c0 msgr2=0x7f9b3414cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:06.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b3406f7c0 0x7f9b3414cee0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9b3000c7f0 tx=0x7f9b3000ccb0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 -- 172.21.15.81:0/1158560163 shutdown_connections 2023-11-08T16:31:06.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b1806ae70 0x7f9b1806d330 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b3415ac20 0x7f9b340c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b340ad940 0x7f9b340c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 --2- 172.21.15.81:0/1158560163 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b3406f7c0 0x7f9b3414cee0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:06.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 -- 172.21.15.81:0/1158560163 >> 172.21.15.81:0/1158560163 conn(0x7f9b340b04d0 msgr2=0x7f9b340af960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:06.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 -- 172.21.15.81:0/1158560163 shutdown_connections 2023-11-08T16:31:06.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:06.888+0000 7f9b3b303640 1 -- 172.21.15.81:0/1158560163 wait complete. 2023-11-08T16:31:06.908 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:31:06.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:06.909 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:31:06.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[101122]: fuse finished with error 0 and tester_r 0 2023-11-08T16:31:07.005 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:31:13.107 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:31:13.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:13.108 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:31:13.170 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:31:13.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.391+0000 7fcd5b8e0640 1 Processor -- start 2023-11-08T16:31:13.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.391+0000 7fcd5b8e0640 1 -- start start 2023-11-08T16:31:13.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.392+0000 7fcd5b8e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540c9ae0 0x7fcd5406f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:13.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.392+0000 7fcd5b8e0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd540c9ec0 con 0x7fcd5406cce0 2023-11-08T16:31:13.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.392+0000 7fcd5b8e0640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd540cc710 con 0x7fcd5406bec0 2023-11-08T16:31:13.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.392+0000 7fcd5b8e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540cc9f0 con 0x7fcd540c9ae0 2023-11-08T16:31:13.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.393+0000 7fcd59e7d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcd5406cce0 0x7fcd5406d170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40520/0 (socket says 172.21.15.81:40520) 2023-11-08T16:31:13.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.393+0000 7fcd59e7d640 1 -- 172.21.15.81:0/1123714183 learned_addr learned my addr 172.21.15.81:0/1123714183 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:13.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.393+0000 7fcd5967c640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540c9ae0 0x7fcd5406f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:13.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.393+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421475714 0 0) 0x7fcd540c9ec0 con 0x7fcd5406cce0 2023-11-08T16:31:13.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.393+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd40003520 con 0x7fcd5406cce0 2023-11-08T16:31:13.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.393+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1394307811 0 0) 0x7fcd40003520 con 0x7fcd5406cce0 2023-11-08T16:31:13.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.393+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540c9ae0 msgr2=0x7fcd5406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:13.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.393+0000 7fcd4bfff640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540c9ae0 0x7fcd5406f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:13.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.393+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 >> v1:172.21.15.93:6789/0 conn(0x7fcd5406bec0 legacy=0x7fcd5406c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:13.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.393+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd540cccd0 con 0x7fcd5406cce0 2023-11-08T16:31:13.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fcd4c0019f0 con 0x7fcd5406cce0 2023-11-08T16:31:13.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 >> v1:172.21.15.81:6789/0 conn(0x7fcd5406cce0 legacy=0x7fcd5406d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:13.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd4bfff640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd40003a80 0x7fcd40003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:13.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd4bfff640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540c9ae0 0x7fcd400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:13.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd4bfff640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd40004d00 0x7fcd40005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:13.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540cc9f0 con 0x7fcd40004d00 2023-11-08T16:31:13.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000bb90 con 0x7fcd40003a80 2023-11-08T16:31:13.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd4000df60 con 0x7fcd540c9ae0 2023-11-08T16:31:13.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd59e7d640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd40004d00 0x7fcd40005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:13.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd5967c640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd40003a80 0x7fcd40003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:13.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd58e7b640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540c9ae0 0x7fcd400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:13.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd59e7d640 1 -- 172.21.15.81:0/1123714183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540c9ae0 msgr2=0x7fcd400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:13.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd59e7d640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540c9ae0 0x7fcd400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:13.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd59e7d640 1 -- 172.21.15.81:0/1123714183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd40003a80 msgr2=0x7fcd40003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:13.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd59e7d640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd40003a80 0x7fcd40003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:13.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.394+0000 7fcd59e7d640 1 -- 172.21.15.81:0/1123714183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd40003520 con 0x7fcd40004d00 2023-11-08T16:31:13.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.395+0000 7fcd59e7d640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd40004d00 0x7fcd40005180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00a800 tx=0x7fcd400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293f2ebc412e63f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:13.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.396+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0072d0 con 0x7fcd40004d00 2023-11-08T16:31:13.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.396+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd4c02f070 con 0x7fcd40004d00 2023-11-08T16:31:13.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.396+0000 7fcd4bfff640 1 -- 172.21.15.81:0/1123714183 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd4c0076b0 con 0x7fcd40004d00 2023-11-08T16:31:13.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.396+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/1123714183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd40004d00 msgr2=0x7fcd40005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:13.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.396+0000 7fcd5b8e0640 1 --2- 172.21.15.81:0/1123714183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd40004d00 0x7fcd40005180 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fcd4c00a800 tx=0x7fcd400057f0 comp rx=0 tx=0).stop 2023-11-08T16:31:13.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.396+0000 7fcd59e7d640 1 -- 172.21.15.81:0/1123714183 reap_dead start 2023-11-08T16:31:13.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.396+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/1123714183 shutdown_connections 2023-11-08T16:31:13.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.396+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/1123714183 >> 172.21.15.81:0/1123714183 conn(0x7fcd540cb490 msgr2=0x7fcd540cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:13.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.396+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/1123714183 shutdown_connections 2023-11-08T16:31:13.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.397+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/1123714183 wait complete. 2023-11-08T16:31:13.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.398+0000 7fcd5b8e0640 1 Processor -- start 2023-11-08T16:31:13.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.399+0000 7fcd5b8e0640 1 -- start start 2023-11-08T16:31:13.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.399+0000 7fcd5b8e0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd40004d00 0x7fcd540b2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:13.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.399+0000 7fcd5b8e0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd541f2910 0x7fcd541584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:13.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd5b8e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540b1ec0 0x7fcd5415bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:13.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd5b8e0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540b1aa0 con 0x7fcd541f2910 2023-11-08T16:31:13.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd5b8e0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540b34c0 con 0x7fcd40004d00 2023-11-08T16:31:13.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd5b8e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd540b2f00 con 0x7fcd540b1ec0 2023-11-08T16:31:13.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd58e7b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd541f2910 0x7fcd541584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:13.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd58e7b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd541f2910 0x7fcd541584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52956/0 (socket says 172.21.15.81:52956) 2023-11-08T16:31:13.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd58e7b640 1 -- 172.21.15.81:0/73073641 learned_addr learned my addr 172.21.15.81:0/73073641 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:13.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd5967c640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd40004d00 0x7fcd540b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:13.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd58e7b640 1 -- 172.21.15.81:0/73073641 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540b1ec0 msgr2=0x7fcd5415bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:13.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd58e7b640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540b1ec0 0x7fcd5415bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:13.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd58e7b640 1 -- 172.21.15.81:0/73073641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd40004d00 msgr2=0x7fcd540b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:13.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd58e7b640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd40004d00 0x7fcd540b2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:13.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd58e7b640 1 -- 172.21.15.81:0/73073641 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd540b31e0 con 0x7fcd541f2910 2023-11-08T16:31:13.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.400+0000 7fcd58e7b640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd541f2910 0x7fcd541584a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400e550 tx=0x7fcd4400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:13.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.401+0000 7fcd49ffb640 1 -- 172.21.15.81:0/73073641 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd440052e0 con 0x7fcd541f2910 2023-11-08T16:31:13.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.401+0000 7fcd49ffb640 1 -- 172.21.15.81:0/73073641 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd44005440 con 0x7fcd541f2910 2023-11-08T16:31:13.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.401+0000 7fcd49ffb640 1 -- 172.21.15.81:0/73073641 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd44003810 con 0x7fcd541f2910 2023-11-08T16:31:13.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.401+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/73073641 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd540b16c0 con 0x7fcd541f2910 2023-11-08T16:31:13.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.401+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/73073641 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd540b2350 con 0x7fcd541f2910 2023-11-08T16:31:13.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.402+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/73073641 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd540b1520 con 0x7fcd541f2910 2023-11-08T16:31:13.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.403+0000 7fcd49ffb640 1 -- 172.21.15.81:0/73073641 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd44003970 con 0x7fcd541f2910 2023-11-08T16:31:13.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.404+0000 7fcd49ffb640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd3806aea0 0x7fcd3806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:13.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.404+0000 7fcd49ffb640 1 -- 172.21.15.81:0/73073641 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7fcd440894e0 con 0x7fcd541f2910 2023-11-08T16:31:13.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.404+0000 7fcd5967c640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd3806aea0 0x7fcd3806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:13.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.405+0000 7fcd5967c640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd3806aea0 0x7fcd3806d360 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009950 tx=0x7fcd50008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:13.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.409+0000 7fcd49ffb640 1 -- 172.21.15.81:0/73073641 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd540b1520 con 0x7fcd541f2910 2023-11-08T16:31:13.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.634+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/73073641 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd540b1520 con 0x7fcd541f2910 2023-11-08T16:31:13.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.635+0000 7fcd49ffb640 1 -- 172.21.15.81:0/73073641 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4909 (secure 0 0 0) 0x7fcd540b1520 con 0x7fcd541f2910 2023-11-08T16:31:13.638 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:13.638 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/1453041994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":1453041994},{"type":"v1","addr":"172.21.15.93:6835","nonce":1453041994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/757624701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":757624701},{"type":"v1","addr":"172.21.15.93:6833","nonce":757624701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1735232404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1735232404},{"type":"v1","addr":"172.21.15.81:6835","nonce":1735232404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:30:30.141674+0000","modified":"2023-11-08T16:30:40.774892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5139},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5139":{"gid":5139,"name":"c","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/773130363","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":773130363},{"type":"v1","addr":"172.21.15.81:6837","nonce":773130363}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:31:13.638 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 19 2023-11-08T16:31:13.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/73073641 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd3806aea0 msgr2=0x7fcd3806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:13.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd3806aea0 0x7fcd3806d360 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcd50009950 tx=0x7fcd50008040 comp rx=0 tx=0).stop 2023-11-08T16:31:13.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/73073641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd541f2910 msgr2=0x7fcd541584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:13.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd541f2910 0x7fcd541584a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fcd4400e550 tx=0x7fcd4400ea10 comp rx=0 tx=0).stop 2023-11-08T16:31:13.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/73073641 shutdown_connections 2023-11-08T16:31:13.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd3806aea0 0x7fcd3806d360 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:13.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd540b1ec0 0x7fcd5415bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:13.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd541f2910 0x7fcd541584a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:13.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 --2- 172.21.15.81:0/73073641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd40004d00 0x7fcd540b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:13.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/73073641 >> 172.21.15.81:0/73073641 conn(0x7fcd540cb490 msgr2=0x7fcd5406a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:13.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/73073641 shutdown_connections 2023-11-08T16:31:13.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:13.638+0000 7fcd5b8e0640 1 -- 172.21.15.81:0/73073641 wait complete. 2023-11-08T16:31:13.658 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2023-11-08T16:31:13.898 INFO:teuthology.orchestra.run.smithi081.stdout:{"message":"","return_code":0} 2023-11-08T16:31:13.898 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-08T16:31:13.899 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:31:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.105+0000 7f84837e7640 1 Processor -- start 2023-11-08T16:31:14.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.106+0000 7f84837e7640 1 -- start start 2023-11-08T16:31:14.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.107+0000 7f84837e7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f847c15d310 0x7f847c0d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.107+0000 7f84837e7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c0b6100 con 0x7f847c0d01d0 2023-11-08T16:31:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.107+0000 7f84837e7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f847c0d1b50 con 0x7f847c0d1300 2023-11-08T16:31:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.107+0000 7f84837e7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f847c0b6480 con 0x7f847c15d310 2023-11-08T16:31:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.107+0000 7f8481583640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f847c15d310 0x7f847c0d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:14.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.107+0000 7f8481d84640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f847c0d01d0 0x7f847c0b68a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40524/0 (socket says 172.21.15.81:40524) 2023-11-08T16:31:14.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.107+0000 7f8481d84640 1 -- 172.21.15.81:0/2665070613 learned_addr learned my addr 172.21.15.81:0/2665070613 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:14.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.108+0000 7f8473fff640 1 -- 172.21.15.81:0/2665070613 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188192034 0 0) 0x7f847c0b6100 con 0x7f847c0d01d0 2023-11-08T16:31:14.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.108+0000 7f8473fff640 1 -- 172.21.15.81:0/2665070613 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8468003520 con 0x7f847c0d01d0 2023-11-08T16:31:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.108+0000 7f8481583640 1 -- 172.21.15.81:0/2665070613 >> v1:172.21.15.93:6789/0 conn(0x7f847c0d1300 legacy=0x7f847c0cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.108+0000 7f8481583640 1 -- 172.21.15.81:0/2665070613 >> v1:172.21.15.81:6789/0 conn(0x7f847c0d01d0 legacy=0x7f847c0b68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.108+0000 7f8481583640 1 -- 172.21.15.81:0/2665070613 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f847c0d1e30 con 0x7f847c15d310 2023-11-08T16:31:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.109+0000 7f8481583640 1 --2- 172.21.15.81:0/2665070613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f847c15d310 0x7f847c0d2b70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f84780097a0 tx=0x7f847802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284980db422ffced server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:14.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.109+0000 7f8473fff640 1 -- 172.21.15.81:0/2665070613 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478030610 con 0x7f847c15d310 2023-11-08T16:31:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.109+0000 7f8473fff640 1 -- 172.21.15.81:0/2665070613 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8478030770 con 0x7f847c15d310 2023-11-08T16:31:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.109+0000 7f8473fff640 1 -- 172.21.15.81:0/2665070613 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478030b00 con 0x7f847c15d310 2023-11-08T16:31:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.109+0000 7f84837e7640 1 -- 172.21.15.81:0/2665070613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f847c15d310 msgr2=0x7f847c0d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.110+0000 7f84837e7640 1 --2- 172.21.15.81:0/2665070613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f847c15d310 0x7f847c0d2b70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f84780097a0 tx=0x7f847802b960 comp rx=0 tx=0).stop 2023-11-08T16:31:14.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.110+0000 7f84837e7640 1 -- 172.21.15.81:0/2665070613 shutdown_connections 2023-11-08T16:31:14.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.110+0000 7f84837e7640 1 --2- 172.21.15.81:0/2665070613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f847c15d310 0x7f847c0d2b70 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.110+0000 7f84837e7640 1 -- 172.21.15.81:0/2665070613 >> 172.21.15.81:0/2665070613 conn(0x7f847c06aac0 msgr2=0x7f847c06aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:14.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.110+0000 7f84837e7640 1 -- 172.21.15.81:0/2665070613 shutdown_connections 2023-11-08T16:31:14.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.110+0000 7f84837e7640 1 -- 172.21.15.81:0/2665070613 wait complete. 2023-11-08T16:31:14.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.112+0000 7f84837e7640 1 Processor -- start 2023-11-08T16:31:14.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.112+0000 7f84837e7640 1 -- start start 2023-11-08T16:31:14.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f84837e7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f847c0d01d0 0x7f847c0c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f84837e7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f847c0d1300 0x7f847c0c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f84837e7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f847c15d310 0x7f847c1f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f84837e7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c15eac0 con 0x7f847c15d310 2023-11-08T16:31:14.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f84837e7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847c15ecd0 con 0x7f847c0d1300 2023-11-08T16:31:14.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f84837e7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847c15efb0 con 0x7f847c0d01d0 2023-11-08T16:31:14.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f8481d84640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f847c15d310 0x7f847c1f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:14.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f8481d84640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f847c15d310 0x7f847c1f2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52978/0 (socket says 172.21.15.81:52978) 2023-11-08T16:31:14.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f8481d84640 1 -- 172.21.15.81:0/1877544691 learned_addr learned my addr 172.21.15.81:0/1877544691 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:14.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f8481583640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f847c0d01d0 0x7f847c0c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:14.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f8480d82640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f847c0d1300 0x7f847c0c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:14.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f8481d84640 1 -- 172.21.15.81:0/1877544691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f847c0d01d0 msgr2=0x7f847c0c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:14.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f8481d84640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f847c0d01d0 0x7f847c0c7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f8481d84640 1 -- 172.21.15.81:0/1877544691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f847c0d1300 msgr2=0x7f847c0c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:14.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f8481d84640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f847c0d1300 0x7f847c0c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.113+0000 7f8481d84640 1 -- 172.21.15.81:0/1877544691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8478009400 con 0x7f847c15d310 2023-11-08T16:31:14.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.114+0000 7f8480d82640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f847c0d1300 0x7f847c0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:14.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.114+0000 7f8481d84640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f847c15d310 0x7f847c1f2e50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f847400caa0 tx=0x7f847400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:14.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.114+0000 7f8471ffb640 1 -- 172.21.15.81:0/1877544691 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847400ac00 con 0x7f847c15d310 2023-11-08T16:31:14.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.114+0000 7f8471ffb640 1 -- 172.21.15.81:0/1877544691 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847400ad60 con 0x7f847c15d310 2023-11-08T16:31:14.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.114+0000 7f8471ffb640 1 -- 172.21.15.81:0/1877544691 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8474012620 con 0x7f847c15d310 2023-11-08T16:31:14.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.114+0000 7f84837e7640 1 -- 172.21.15.81:0/1877544691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847c15f290 con 0x7f847c15d310 2023-11-08T16:31:14.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.114+0000 7f84837e7640 1 -- 172.21.15.81:0/1877544691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847c0c9ce0 con 0x7f847c15d310 2023-11-08T16:31:14.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.115+0000 7f84837e7640 1 -- 172.21.15.81:0/1877544691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847c0d1770 con 0x7f847c15d310 2023-11-08T16:31:14.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.116+0000 7f8471ffb640 1 -- 172.21.15.81:0/1877544691 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8474007070 con 0x7f847c15d310 2023-11-08T16:31:14.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.117+0000 7f8471ffb640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f846006aea0 0x7f846006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.117+0000 7f8471ffb640 1 -- 172.21.15.81:0/1877544691 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f8474089800 con 0x7f847c15d310 2023-11-08T16:31:14.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.117+0000 7f8481583640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f846006aea0 0x7f846006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:14.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.118+0000 7f8481583640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f846006aea0 0x7f846006d360 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8478036000 tx=0x7f847803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:14.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.121+0000 7f8471ffb640 1 -- 172.21.15.81:0/1877544691 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f847c0d1770 con 0x7f847c15d310 2023-11-08T16:31:14.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.351+0000 7f84837e7640 1 -- 172.21.15.81:0/1877544691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f847c15f7b0 con 0x7f847c15d310 2023-11-08T16:31:14.354 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:31:14.352+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:31:14.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.356+0000 7f8471ffb640 1 -- 172.21.15.81:0/1877544691 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) v1 ==== 161+0+0 (secure 0 0 0) 0x7f847c15f7b0 con 0x7f847c15d310 2023-11-08T16:31:14.358 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:31:14.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.359+0000 7f84837e7640 1 -- 172.21.15.81:0/1877544691 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f846006aea0 msgr2=0x7f846006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:14.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.359+0000 7f84837e7640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f846006aea0 0x7f846006d360 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8478036000 tx=0x7f847803b040 comp rx=0 tx=0).stop 2023-11-08T16:31:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.359+0000 7f84837e7640 1 -- 172.21.15.81:0/1877544691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f847c15d310 msgr2=0x7f847c1f2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.359+0000 7f84837e7640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f847c15d310 0x7f847c1f2e50 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f847400caa0 tx=0x7f847400cf60 comp rx=0 tx=0).stop 2023-11-08T16:31:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.360+0000 7f84837e7640 1 -- 172.21.15.81:0/1877544691 shutdown_connections 2023-11-08T16:31:14.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.360+0000 7f84837e7640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f846006aea0 0x7f846006d360 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.360+0000 7f84837e7640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f847c15d310 0x7f847c1f2e50 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.360+0000 7f84837e7640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f847c0d1300 0x7f847c0c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.360+0000 7f84837e7640 1 --2- 172.21.15.81:0/1877544691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f847c0d01d0 0x7f847c0c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.360+0000 7f84837e7640 1 -- 172.21.15.81:0/1877544691 >> 172.21.15.81:0/1877544691 conn(0x7f847c06aac0 msgr2=0x7f847c0cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:14.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.360+0000 7f84837e7640 1 -- 172.21.15.81:0/1877544691 shutdown_connections 2023-11-08T16:31:14.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.360+0000 7f84837e7640 1 -- 172.21.15.81:0/1877544691 wait complete. 2023-11-08T16:31:14.382 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:31:14.388 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:31:14.393 INFO:tasks.ceph.mds.c.smithi081.stderr: -6995> 2023-11-08T16:30:36.646+0000 7f42849a8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:31:14.474 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:31:14.531 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2023-11-08T16:31:14.532 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2023-11-08T16:31:14.532 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:31:14.532 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2023-11-08T16:31:14.532 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.533 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2023-11-08T16:31:14.533 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.533 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2023-11-08T16:31:14.533 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.533 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2023-11-08T16:31:14.534 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2023-11-08T16:31:14.534 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.534 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2023-11-08T16:31:14.534 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2023-11-08T16:31:14.534 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.535 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:31:14.535 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2023-11-08T16:31:14.535 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.535 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:31:14.535 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:31:14.536 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.536 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2023-11-08T16:31:14.536 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.536 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2023-11-08T16:31:14.536 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2023-11-08T16:31:14.537 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:31:14.537 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:31:14.537 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2023-11-08T16:31:14.537 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:31:14.537 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2023-11-08T16:31:14.538 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:31:14.538 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:31:14.538 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2023-11-08T16:31:14.538 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.538 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:31:14.539 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.539 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2023-11-08T16:31:14.539 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.539 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2023-11-08T16:31:14.539 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.540 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2023-11-08T16:31:14.540 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2023-11-08T16:31:14.540 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2023-11-08T16:31:14.540 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:31:14.540 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:31:14.540 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.541 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2023-11-08T16:31:14.541 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2023-11-08T16:31:14.541 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.541 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2023-11-08T16:31:14.541 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2023-11-08T16:31:14.542 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:31:14.542 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.542 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2023-11-08T16:31:14.542 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2023-11-08T16:31:14.542 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.543 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2023-11-08T16:31:14.543 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2023-11-08T16:31:14.543 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2023-11-08T16:31:14.543 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2023-11-08T16:31:14.543 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.543 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2023-11-08T16:31:14.544 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2023-11-08T16:31:14.544 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.544 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:31:14.544 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.544 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2023-11-08T16:31:14.545 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.545 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2023-11-08T16:31:14.545 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.545 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:31:14.545 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2023-11-08T16:31:14.546 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.546 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2023-11-08T16:31:14.546 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2023-11-08T16:31:14.546 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2023-11-08T16:31:14.546 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2023-11-08T16:31:14.546 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2023-11-08T16:31:14.547 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2023-11-08T16:31:14.547 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2023-11-08T16:31:14.547 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2023-11-08T16:31:14.547 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2023-11-08T16:31:14.547 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:31:14.548 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2023-11-08T16:31:14.548 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2023-11-08T16:31:14.548 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:31:14.548 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2023-11-08T16:31:14.548 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:31:14.549 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:31:14.549 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2023-11-08T16:31:14.549 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:31:14.549 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.549 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2023-11-08T16:31:14.549 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:31:14.550 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2023-11-08T16:31:14.550 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:31:14.550 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:31:14.550 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:31:14.550 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:31:14.551 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2023-11-08T16:31:14.551 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.551 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2023-11-08T16:31:14.551 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:31:14.551 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:31:14.552 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2023-11-08T16:31:14.552 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:31:14.552 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2023-11-08T16:31:14.552 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2023-11-08T16:31:14.552 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:31:14.553 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2023-11-08T16:31:14.553 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2023-11-08T16:31:14.553 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:31:14.553 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.553 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:31:14.553 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:31:14.554 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:31:14.554 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.554 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:31:14.554 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:31:14.554 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:31:14.555 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.555 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2023-11-08T16:31:14.555 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:31:14.555 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.555 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:31:14.556 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2023-11-08T16:31:14.556 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.556 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2023-11-08T16:31:14.556 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2023-11-08T16:31:14.556 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2023-11-08T16:31:14.556 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2023-11-08T16:31:14.557 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:31:14.557 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:31:14.557 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2023-11-08T16:31:14.557 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2023-11-08T16:31:14.557 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:31:14.558 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2023-11-08T16:31:14.558 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:31:14.558 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:31:14.558 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2023-11-08T16:31:14.558 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2023-11-08T16:31:14.559 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2023-11-08T16:31:14.559 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:31:14.559 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:31:14.559 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:31:14.559 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:31:14.559 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2023-11-08T16:31:14.560 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2023-11-08T16:31:14.560 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2023-11-08T16:31:14.560 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:31:14.560 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2023-11-08T16:31:14.560 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = True 2023-11-08T16:31:14.561 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2023-11-08T16:31:14.561 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = False 2023-11-08T16:31:14.561 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:31:14.561 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:31:14.561 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:31:14.562 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-08T16:31:14.562 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2023-11-08T16:31:14.562 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:31:14.563 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2023-11-08T16:31:14.563 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.563 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2023-11-08T16:31:14.563 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.563 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2023-11-08T16:31:14.564 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.564 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2023-11-08T16:31:14.564 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2023-11-08T16:31:14.564 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.564 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2023-11-08T16:31:14.565 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2023-11-08T16:31:14.565 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.565 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:31:14.565 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2023-11-08T16:31:14.565 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.566 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:31:14.566 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:31:14.566 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.566 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2023-11-08T16:31:14.566 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.567 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2023-11-08T16:31:14.567 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2023-11-08T16:31:14.567 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:31:14.567 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:31:14.567 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2023-11-08T16:31:14.568 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:31:14.568 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2023-11-08T16:31:14.568 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:31:14.568 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:31:14.568 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2023-11-08T16:31:14.568 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.569 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:31:14.569 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.569 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2023-11-08T16:31:14.569 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.569 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2023-11-08T16:31:14.570 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.570 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2023-11-08T16:31:14.570 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2023-11-08T16:31:14.570 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2023-11-08T16:31:14.570 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:31:14.571 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:31:14.571 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.571 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2023-11-08T16:31:14.571 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2023-11-08T16:31:14.571 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.571 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2023-11-08T16:31:14.572 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2023-11-08T16:31:14.572 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:31:14.572 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.572 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2023-11-08T16:31:14.572 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2023-11-08T16:31:14.573 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.573 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2023-11-08T16:31:14.573 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2023-11-08T16:31:14.573 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2023-11-08T16:31:14.573 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2023-11-08T16:31:14.574 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.574 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2023-11-08T16:31:14.574 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2023-11-08T16:31:14.574 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.574 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:31:14.575 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.575 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2023-11-08T16:31:14.575 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.575 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2023-11-08T16:31:14.575 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.576 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:31:14.576 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2023-11-08T16:31:14.576 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.576 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2023-11-08T16:31:14.576 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2023-11-08T16:31:14.577 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2023-11-08T16:31:14.577 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2023-11-08T16:31:14.577 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2023-11-08T16:31:14.577 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2023-11-08T16:31:14.577 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2023-11-08T16:31:14.577 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2023-11-08T16:31:14.578 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2023-11-08T16:31:14.578 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:31:14.578 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2023-11-08T16:31:14.578 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2023-11-08T16:31:14.578 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:31:14.579 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2023-11-08T16:31:14.579 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:31:14.579 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:31:14.579 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2023-11-08T16:31:14.579 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:31:14.580 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.580 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2023-11-08T16:31:14.580 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:31:14.580 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2023-11-08T16:31:14.580 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:31:14.581 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:31:14.581 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:31:14.581 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:31:14.581 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2023-11-08T16:31:14.581 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.582 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2023-11-08T16:31:14.582 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:31:14.582 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:31:14.582 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2023-11-08T16:31:14.582 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:31:14.582 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2023-11-08T16:31:14.583 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2023-11-08T16:31:14.583 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:31:14.583 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2023-11-08T16:31:14.583 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2023-11-08T16:31:14.583 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:31:14.584 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.584 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:31:14.584 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:31:14.584 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:31:14.584 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.585 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:31:14.585 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:31:14.585 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:31:14.585 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.585 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2023-11-08T16:31:14.586 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:31:14.586 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.586 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:31:14.586 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2023-11-08T16:31:14.586 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.587 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2023-11-08T16:31:14.587 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2023-11-08T16:31:14.587 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2023-11-08T16:31:14.587 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2023-11-08T16:31:14.587 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:31:14.588 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:31:14.588 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2023-11-08T16:31:14.588 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2023-11-08T16:31:14.588 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:31:14.588 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2023-11-08T16:31:14.589 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:31:14.589 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:31:14.589 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2023-11-08T16:31:14.589 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2023-11-08T16:31:14.589 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2023-11-08T16:31:14.589 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:31:14.590 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:31:14.590 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:31:14.590 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:31:14.590 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2023-11-08T16:31:14.590 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2023-11-08T16:31:14.591 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2023-11-08T16:31:14.591 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:31:14.591 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2023-11-08T16:31:14.591 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = True 2023-11-08T16:31:14.591 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2023-11-08T16:31:14.592 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = False 2023-11-08T16:31:14.592 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:31:14.592 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:31:14.592 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:31:14.597 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:31:14.643 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:31:14.806 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2023-11-08T16:31:14.806 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2023-11-08T16:31:14.807 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:31:14.807 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2023-11-08T16:31:14.807 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.807 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2023-11-08T16:31:14.808 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.808 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2023-11-08T16:31:14.808 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.808 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2023-11-08T16:31:14.808 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2023-11-08T16:31:14.808 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.809 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2023-11-08T16:31:14.809 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2023-11-08T16:31:14.809 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.809 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:31:14.809 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2023-11-08T16:31:14.810 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.810 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:31:14.810 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:31:14.810 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.810 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2023-11-08T16:31:14.811 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.811 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2023-11-08T16:31:14.811 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2023-11-08T16:31:14.811 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:31:14.811 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:31:14.812 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2023-11-08T16:31:14.812 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:31:14.812 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2023-11-08T16:31:14.812 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:31:14.812 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:31:14.813 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2023-11-08T16:31:14.813 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.813 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:31:14.813 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.813 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2023-11-08T16:31:14.814 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.814 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2023-11-08T16:31:14.814 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.814 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2023-11-08T16:31:14.814 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2023-11-08T16:31:14.815 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2023-11-08T16:31:14.815 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:31:14.815 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:31:14.815 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.816 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2023-11-08T16:31:14.816 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2023-11-08T16:31:14.816 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.816 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2023-11-08T16:31:14.817 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2023-11-08T16:31:14.817 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:31:14.817 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.817 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2023-11-08T16:31:14.817 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2023-11-08T16:31:14.818 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.818 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2023-11-08T16:31:14.818 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2023-11-08T16:31:14.818 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2023-11-08T16:31:14.818 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2023-11-08T16:31:14.819 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.819 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2023-11-08T16:31:14.819 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2023-11-08T16:31:14.819 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.820 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:31:14.820 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.820 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2023-11-08T16:31:14.820 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.820 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2023-11-08T16:31:14.821 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.821 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:31:14.821 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2023-11-08T16:31:14.821 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.821 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2023-11-08T16:31:14.822 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2023-11-08T16:31:14.822 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2023-11-08T16:31:14.822 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2023-11-08T16:31:14.822 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2023-11-08T16:31:14.822 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2023-11-08T16:31:14.823 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2023-11-08T16:31:14.823 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2023-11-08T16:31:14.823 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2023-11-08T16:31:14.823 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:31:14.823 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2023-11-08T16:31:14.824 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2023-11-08T16:31:14.824 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:31:14.824 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2023-11-08T16:31:14.824 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:31:14.824 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:31:14.825 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2023-11-08T16:31:14.825 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:31:14.825 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.825 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2023-11-08T16:31:14.825 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:31:14.825 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2023-11-08T16:31:14.826 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:31:14.826 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:31:14.826 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:31:14.826 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:31:14.826 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2023-11-08T16:31:14.827 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.827 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2023-11-08T16:31:14.827 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:31:14.827 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:31:14.827 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2023-11-08T16:31:14.828 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:31:14.828 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2023-11-08T16:31:14.828 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2023-11-08T16:31:14.828 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:31:14.828 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2023-11-08T16:31:14.828 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2023-11-08T16:31:14.829 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:31:14.829 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.829 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:31:14.829 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:31:14.829 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:31:14.830 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.830 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:31:14.830 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:31:14.830 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:31:14.830 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.831 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2023-11-08T16:31:14.831 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:31:14.831 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.831 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:31:14.831 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2023-11-08T16:31:14.831 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:14.832 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2023-11-08T16:31:14.832 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2023-11-08T16:31:14.832 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2023-11-08T16:31:14.832 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2023-11-08T16:31:14.832 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:31:14.833 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:31:14.833 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2023-11-08T16:31:14.833 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2023-11-08T16:31:14.833 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:31:14.833 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2023-11-08T16:31:14.834 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:31:14.834 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:31:14.834 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2023-11-08T16:31:14.834 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2023-11-08T16:31:14.834 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2023-11-08T16:31:14.834 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:31:14.835 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:31:14.835 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:31:14.835 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:31:14.835 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2023-11-08T16:31:14.835 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2023-11-08T16:31:14.836 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2023-11-08T16:31:14.836 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:31:14.836 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2023-11-08T16:31:14.836 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = False 2023-11-08T16:31:14.836 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2023-11-08T16:31:14.837 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = False 2023-11-08T16:31:14.837 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:31:14.837 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:31:14.837 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:31:14.838 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-08T16:31:14.838 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2023-11-08T16:31:14.838 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:31:14.838 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2023-11-08T16:31:14.839 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.839 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2023-11-08T16:31:14.839 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.839 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2023-11-08T16:31:14.839 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.840 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2023-11-08T16:31:14.840 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2023-11-08T16:31:14.840 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.840 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2023-11-08T16:31:14.840 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2023-11-08T16:31:14.841 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.841 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:31:14.841 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2023-11-08T16:31:14.841 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.841 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:31:14.842 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:31:14.842 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.842 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2023-11-08T16:31:14.842 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.842 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2023-11-08T16:31:14.843 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2023-11-08T16:31:14.843 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:31:14.843 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:31:14.843 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2023-11-08T16:31:14.843 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:31:14.844 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2023-11-08T16:31:14.844 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:31:14.844 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:31:14.844 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2023-11-08T16:31:14.844 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.845 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:31:14.845 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.845 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2023-11-08T16:31:14.845 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.845 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2023-11-08T16:31:14.845 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.846 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2023-11-08T16:31:14.846 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2023-11-08T16:31:14.846 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2023-11-08T16:31:14.846 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:31:14.846 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:31:14.847 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.847 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2023-11-08T16:31:14.847 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2023-11-08T16:31:14.847 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.847 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2023-11-08T16:31:14.847 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2023-11-08T16:31:14.848 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:31:14.848 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.848 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2023-11-08T16:31:14.848 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2023-11-08T16:31:14.848 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.849 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2023-11-08T16:31:14.849 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2023-11-08T16:31:14.849 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2023-11-08T16:31:14.849 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2023-11-08T16:31:14.849 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.850 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2023-11-08T16:31:14.850 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2023-11-08T16:31:14.850 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.850 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:31:14.850 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.851 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2023-11-08T16:31:14.851 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.851 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2023-11-08T16:31:14.851 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.851 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:31:14.851 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2023-11-08T16:31:14.852 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.852 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2023-11-08T16:31:14.852 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2023-11-08T16:31:14.852 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2023-11-08T16:31:14.852 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2023-11-08T16:31:14.853 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2023-11-08T16:31:14.853 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2023-11-08T16:31:14.853 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2023-11-08T16:31:14.853 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2023-11-08T16:31:14.853 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2023-11-08T16:31:14.853 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:31:14.854 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2023-11-08T16:31:14.854 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2023-11-08T16:31:14.854 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:31:14.854 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2023-11-08T16:31:14.854 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:31:14.855 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:31:14.855 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2023-11-08T16:31:14.855 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:31:14.855 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.855 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2023-11-08T16:31:14.856 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:31:14.856 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2023-11-08T16:31:14.856 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:31:14.856 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:31:14.856 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:31:14.856 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:31:14.857 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2023-11-08T16:31:14.857 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.857 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2023-11-08T16:31:14.857 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:31:14.857 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:31:14.858 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2023-11-08T16:31:14.858 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:31:14.858 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2023-11-08T16:31:14.858 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2023-11-08T16:31:14.858 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:31:14.859 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2023-11-08T16:31:14.859 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2023-11-08T16:31:14.859 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:31:14.859 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.859 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:31:14.859 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:31:14.860 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:31:14.860 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.860 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:31:14.860 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:31:14.860 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:31:14.861 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.861 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2023-11-08T16:31:14.861 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:31:14.861 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.861 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:31:14.861 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2023-11-08T16:31:14.862 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:31:14.862 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2023-11-08T16:31:14.862 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2023-11-08T16:31:14.862 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2023-11-08T16:31:14.862 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2023-11-08T16:31:14.863 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:31:14.863 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:31:14.863 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2023-11-08T16:31:14.863 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2023-11-08T16:31:14.863 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:31:14.864 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2023-11-08T16:31:14.864 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:31:14.864 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:31:14.864 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2023-11-08T16:31:14.864 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2023-11-08T16:31:14.864 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2023-11-08T16:31:14.865 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:31:14.865 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:31:14.865 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:31:14.865 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:31:14.865 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2023-11-08T16:31:14.866 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2023-11-08T16:31:14.866 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2023-11-08T16:31:14.866 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:31:14.866 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2023-11-08T16:31:14.866 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = False 2023-11-08T16:31:14.867 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2023-11-08T16:31:14.867 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = False 2023-11-08T16:31:14.867 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:31:14.867 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:31:14.867 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:31:14.869 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T16:31:14.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.978+0000 7f808f00ef40 1 Processor -- start 2023-11-08T16:31:14.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.978+0000 7f808f00ef40 1 -- start start 2023-11-08T16:31:14.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.979+0000 7f808f00ef40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5575f747e1d0 0x5575f75520b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.979+0000 7f808f00ef40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575f7553580 0x5575f7558a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.979+0000 7f808f00ef40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f74c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.979+0000 7f808f00ef40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575f742d340 con 0x5575f7553960 2023-11-08T16:31:14.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.979+0000 7f808f00ef40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575f73c7a70 con 0x5575f7553580 2023-11-08T16:31:14.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.979+0000 7f808f00ef40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575f73c7710 con 0x5575f747e1d0 2023-11-08T16:31:14.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808c761640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f74c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:14.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808c761640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f74c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:52990/0 (socket says 172.21.15.81:52990) 2023-11-08T16:31:14.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808c761640 1 -- 172.21.15.81:0/3887572792 learned_addr learned my addr 172.21.15.81:0/3887572792 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:14.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808bf60640 1 --2- 172.21.15.81:0/3887572792 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5575f747e1d0 0x5575f75520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:14.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808b75f640 1 --2- 172.21.15.81:0/3887572792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575f7553580 0x5575f7558a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:14.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808c761640 1 -- 172.21.15.81:0/3887572792 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5575f747e1d0 msgr2=0x5575f75520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:14.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808c761640 1 --2- 172.21.15.81:0/3887572792 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5575f747e1d0 0x5575f75520b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808c761640 1 -- 172.21.15.81:0/3887572792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575f7553580 msgr2=0x5575f7558a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:14.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808c761640 1 --2- 172.21.15.81:0/3887572792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575f7553580 0x5575f7558a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808c761640 1 -- 172.21.15.81:0/3887572792 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c005de0 con 0x5575f7553960 2023-11-08T16:31:14.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808b75f640 1 --2- 172.21.15.81:0/3887572792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575f7553580 0x5575f7558a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:14.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.980+0000 7f808c761640 1 --2- 172.21.15.81:0/3887572792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f74c90b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8084002880 tx=0x7f8084002d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187b984e52aa8cf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:14.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808a75d640 1 -- 172.21.15.81:0/3887572792 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808400d960 con 0x5575f7553960 2023-11-08T16:31:14.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808a75d640 1 -- 172.21.15.81:0/3887572792 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808400dac0 con 0x5575f7553960 2023-11-08T16:31:14.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808a75d640 1 -- 172.21.15.81:0/3887572792 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808400de50 con 0x5575f7553960 2023-11-08T16:31:14.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808f00ef40 1 -- 172.21.15.81:0/3887572792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 msgr2=0x5575f74c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:14.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808f00ef40 1 --2- 172.21.15.81:0/3887572792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f74c90b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f8084002880 tx=0x7f8084002d40 comp rx=0 tx=0).stop 2023-11-08T16:31:14.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808f00ef40 1 -- 172.21.15.81:0/3887572792 shutdown_connections 2023-11-08T16:31:14.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808f00ef40 1 --2- 172.21.15.81:0/3887572792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f74c90b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808f00ef40 1 --2- 172.21.15.81:0/3887572792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575f7553580 0x5575f7558a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808f00ef40 1 --2- 172.21.15.81:0/3887572792 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5575f747e1d0 0x5575f75520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808f00ef40 1 -- 172.21.15.81:0/3887572792 >> 172.21.15.81:0/3887572792 conn(0x5575f74c8c60 msgr2=0x5575f747c710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:14.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808f00ef40 1 -- 172.21.15.81:0/3887572792 shutdown_connections 2023-11-08T16:31:14.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.981+0000 7f808f00ef40 1 -- 172.21.15.81:0/3887572792 wait complete. 2023-11-08T16:31:14.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.982+0000 7f808f00ef40 1 Processor -- start 2023-11-08T16:31:14.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.982+0000 7f808f00ef40 1 -- start start 2023-11-08T16:31:14.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.983+0000 7f808f00ef40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575f747e1d0 0x5575f7541920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.983+0000 7f808f00ef40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5575f7553580 0x5575f7558340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.983+0000 7f808f00ef40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f7558880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.983+0000 7f808f00ef40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575f73bb520 con 0x5575f7553960 2023-11-08T16:31:14.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.983+0000 7f808f00ef40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575f74c5a30 con 0x5575f747e1d0 2023-11-08T16:31:14.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.983+0000 7f808f00ef40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5575f7544c40 con 0x5575f7553580 2023-11-08T16:31:14.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.984+0000 7f808c761640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f7558880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:14.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.984+0000 7f808c761640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f7558880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53000/0 (socket says 172.21.15.81:53000) 2023-11-08T16:31:14.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.984+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 learned_addr learned my addr 172.21.15.81:0/1646191478 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:14.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.984+0000 7f808bf60640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575f747e1d0 0x5575f7541920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:14.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.984+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5575f7553580 msgr2=0x5575f7558340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:14.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.984+0000 7f808c761640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5575f7553580 0x5575f7558340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.984+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575f747e1d0 msgr2=0x5575f7541920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:14.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.984+0000 7f808c761640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575f747e1d0 0x5575f7541920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:14.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.984+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80840024d0 con 0x5575f7553960 2023-11-08T16:31:14.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.984+0000 7f808bf60640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575f747e1d0 0x5575f7541920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:14.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.985+0000 7f808c761640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f7558880 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f808400d5a0 tx=0x7f80840033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:14.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.985+0000 7f808975b640 1 -- 172.21.15.81:0/1646191478 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808401b830 con 0x5575f7553960 2023-11-08T16:31:14.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.985+0000 7f808975b640 1 -- 172.21.15.81:0/1646191478 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f808401f070 con 0x5575f7553960 2023-11-08T16:31:14.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.985+0000 7f808975b640 1 -- 172.21.15.81:0/1646191478 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f808401bbf0 con 0x5575f7553960 2023-11-08T16:31:14.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.985+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5575f73d3680 con 0x5575f7553960 2023-11-08T16:31:14.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.985+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575f73c6830 con 0x5575f7553960 2023-11-08T16:31:14.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.987+0000 7f808975b640 1 -- 172.21.15.81:0/1646191478 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8084003040 con 0x5575f7553960 2023-11-08T16:31:14.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.988+0000 7f808975b640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8078069850 0x7f807806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:14.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.988+0000 7f808975b640 1 -- 172.21.15.81:0/1646191478 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6199+0+0 (secure 0 0 0) 0x7f8084021070 con 0x5575f7553960 2023-11-08T16:31:14.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.988+0000 7f808bf60640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8078069850 0x7f807806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:14.998 INFO:teuthology.orchestra.run.smithi081.stdout:Warning: using slow linear search 2023-11-08T16:31:15.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.988+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5575f7553f20 0x5575f75fb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.988+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 4.0 4:00000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x5575f7541e60 con 0x5575f7553f20 2023-11-08T16:31:15.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.989+0000 7f808b75f640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5575f7553f20 0x5575f75fb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.989+0000 7f808bf60640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8078069850 0x7f807806bd10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f807c054000 tx=0x7f807c04e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:15.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.989+0000 7f808b75f640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5575f7553f20 0x5575f75fb0d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:15.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.992+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.992+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:2 4.10 4:08000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x5575f7553f20 2023-11-08T16:31:15.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.993+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(2 [pgnls start_epoch 39 out=177b] v38'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.993+0000 7f808b75f640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f8080013040 0x7f808000e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.993+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:3 4.8 4:10000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080013040 2023-11-08T16:31:15.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.993+0000 7f808bf60640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f8080013040 0x7f808000e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.994+0000 7f808bf60640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f8080013040 0x7f808000e310 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:15.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.995+0000 7f808bf60640 1 -- 172.21.15.81:0/1646191478 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(3 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f807c052870 con 0x7f8080013040 2023-11-08T16:31:15.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.995+0000 7f808bf60640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f807c069920 0x7f807c05d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.995+0000 7f808bf60640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:4 4.18 4:18000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f807c069920 2023-11-08T16:31:15.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.996+0000 7f808b75f640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f807c069920 0x7f807c05d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:14.997+0000 7f808b75f640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f807c069920 0x7f807c05d3f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:15.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.004+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(4 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.004+0000 7f808b75f640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f8080011990 0x7f8080015430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.004+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:5 4.4 4:20000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080011990 2023-11-08T16:31:15.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.005+0000 7f808c761640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f8080011990 0x7f8080015430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.005+0000 7f808c761640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f8080011990 0x7f8080015430 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:15.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.008+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(5 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8084077040 con 0x7f8080011990 2023-11-08T16:31:15.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.008+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:6 4.14 4:28000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x5575f7553f20 2023-11-08T16:31:15.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.009+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(6 [pgnls start_epoch 39 out=149b] v38'35 uv35 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.009+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:7 4.c 4:30000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f807c069920 2023-11-08T16:31:15.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.010+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(7 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.010+0000 7f808b75f640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f80800212b0 0x7f8080021730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.010+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:8 4.1c 4:38000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f80800212b0 2023-11-08T16:31:15.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.010+0000 7f808bf60640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f80800212b0 0x7f8080021730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.011+0000 7f808bf60640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f80800212b0 0x7f8080021730 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:15.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.014+0000 7f808bf60640 1 -- 172.21.15.81:0/1646191478 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(8 [pgnls start_epoch 39 out=73b] v34'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f807c052870 con 0x7f80800212b0 2023-11-08T16:31:15.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.014+0000 7f808bf60640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:9 4.2 4:40000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080011990 2023-11-08T16:31:15.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.014+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(9 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8084077040 con 0x7f8080011990 2023-11-08T16:31:15.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.014+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:10 4.12 4:48000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080011990 2023-11-08T16:31:15.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.014+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(10 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8084077040 con 0x7f8080011990 2023-11-08T16:31:15.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.014+0000 7f808c761640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f808401b040 0x7f8084057c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.014+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:11 4.a 4:50000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f808401b040 2023-11-08T16:31:15.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.015+0000 7f808b75f640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f808401b040 0x7f8084057c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.016+0000 7f808b75f640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f808401b040 0x7f8084057c20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:15.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.017+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(11 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x7f808401b040 2023-11-08T16:31:15.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.017+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:12 4.1a 4:58000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f807c069920 2023-11-08T16:31:15.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.018+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(12 [pgnls start_epoch 39 out=49b] v34'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.018+0000 7f808b75f640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7f8080029440 0x7f80800298a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.018+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:13 4.6 4:60000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080029440 2023-11-08T16:31:15.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.019+0000 7f808c761640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7f8080029440 0x7f80800298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.019+0000 7f808c761640 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7f8080029440 0x7f80800298a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:15.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.024+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(13 [pgnls start_epoch 39 out=77b] v38'8 uv8 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f8084077040 con 0x7f8080029440 2023-11-08T16:31:15.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.024+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:14 4.16 4:68000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080029440 2023-11-08T16:31:15.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.025+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(14 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8084077040 con 0x7f8080029440 2023-11-08T16:31:15.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.025+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:15 4.e 4:70000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f807c069920 2023-11-08T16:31:15.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.026+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(15 [pgnls start_epoch 39 out=73b] v34'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.026+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:16 4.1e 4:78000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080029440 2023-11-08T16:31:15.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.026+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(16 [pgnls start_epoch 39 out=49b] v34'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8084077040 con 0x7f8080029440 2023-11-08T16:31:15.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.026+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:17 4.1 4:80000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f807c069920 2023-11-08T16:31:15.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.027+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(17 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.027+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:18 4.11 4:88000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x5575f7553f20 2023-11-08T16:31:15.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.028+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(18 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.028+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:19 4.9 4:90000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f807c069920 2023-11-08T16:31:15.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.029+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(19 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.029+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:20 4.19 4:98000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x5575f7553f20 2023-11-08T16:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.029+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(20 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.029+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:21 4.5 4:a0000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f808401b040 2023-11-08T16:31:15.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.030+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(21 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x7f808401b040 2023-11-08T16:31:15.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.030+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:22 4.15 4:a8000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080013040 2023-11-08T16:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.032+0000 7f808bf60640 1 -- 172.21.15.81:0/1646191478 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(22 [pgnls start_epoch 39 out=97b] v34'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f807c052870 con 0x7f8080013040 2023-11-08T16:31:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.032+0000 7f808bf60640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:23 4.d 4:b0000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f807c069920 2023-11-08T16:31:15.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.033+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(23 [pgnls start_epoch 39 out=74b] v34'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.033+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:24 4.1d 4:b8000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x5575f7553f20 2023-11-08T16:31:15.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.034+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(24 [pgnls start_epoch 39 out=73b] v34'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.034+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:25 4.3 4:c0000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080029440 2023-11-08T16:31:15.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.034+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(25 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f8084077040 con 0x7f8080029440 2023-11-08T16:31:15.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.034+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:26 4.13 4:c8000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f807c069920 2023-11-08T16:31:15.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.035+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(26 [pgnls start_epoch 39 out=98b] v38'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.036+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:27 4.b 4:d0000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080029440 2023-11-08T16:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.036+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(27 [pgnls start_epoch 39 out=76b] v38'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f8084077040 con 0x7f8080029440 2023-11-08T16:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.036+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:28 4.1b 4:d8000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f807c069920 2023-11-08T16:31:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.037+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(28 [pgnls start_epoch 39 out=49b] v34'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.037+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:29 4.7 4:e0000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080011990 2023-11-08T16:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.038+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(29 [pgnls start_epoch 39 out=73b] v34'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f8084077040 con 0x7f8080011990 2023-11-08T16:31:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.038+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:30 4.17 4:e8000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x5575f7553f20 2023-11-08T16:31:15.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.038+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(30 [pgnls start_epoch 39 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.038+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:31 4.f 4:f0000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f8080011990 2023-11-08T16:31:15.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.039+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(31 [pgnls start_epoch 39 out=73b] v34'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f8084077040 con 0x7f8080011990 2023-11-08T16:31:15.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.039+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:32 4.1f 4:f8000000:%01:::head [pgnls start_epoch 39 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e39) v8 -- 0x7f808000fa70 con 0x7f808401b040 2023-11-08T16:31:15.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.040+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(32 [pgnls start_epoch 39 out=101b] v38'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f808000f380 con 0x7f808401b040 2023-11-08T16:31:15.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:33 4.10 4:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f747c4f0 con 0x5575f7553f20 2023-11-08T16:31:15.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:34 4.10 4:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f74c5e80 con 0x5575f7553f20 2023-11-08T16:31:15.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:35 4.10 4:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75fb610 con 0x5575f7553f20 2023-11-08T16:31:15.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:36 4.10 4:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75fbe10 con 0x5575f7553f20 2023-11-08T16:31:15.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:37 4.10 4:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75fc610 con 0x5575f7553f20 2023-11-08T16:31:15.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:38 4.14 4:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75fce10 con 0x5575f7553f20 2023-11-08T16:31:15.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:39 4.14 4:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75fd610 con 0x5575f7553f20 2023-11-08T16:31:15.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:40 4.14 4:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75fde10 con 0x5575f7553f20 2023-11-08T16:31:15.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:41 4.14 4:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75fe610 con 0x5575f7553f20 2023-11-08T16:31:15.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:42 4.1c 4:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75fee10 con 0x7f80800212b0 2023-11-08T16:31:15.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:43 4.6 4:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75ff610 con 0x7f8080029440 2023-11-08T16:31:15.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:44 4.e 4:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75ffe10 con 0x7f807c069920 2023-11-08T16:31:15.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:45 4.15 4:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f7600610 con 0x7f8080013040 2023-11-08T16:31:15.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:46 4.15 4:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f7600e10 con 0x7f8080013040 2023-11-08T16:31:15.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:47 4.d 4:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75fde10 con 0x7f807c069920 2023-11-08T16:31:15.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.041+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:48 4.1d 4:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f7601d00 con 0x5575f7553f20 2023-11-08T16:31:15.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.047+0000 7f808bf60640 1 -- 172.21.15.81:0/1646191478 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(42 607.00000000 [delete] v39'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f807c052870 con 0x7f80800212b0 2023-11-08T16:31:15.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.047+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:49 4.13 4:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75ffe10 con 0x7f807c069920 2023-11-08T16:31:15.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.047+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(47 mds_snaptable [delete] v39'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.047+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:50 4.13 4:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75fde10 con 0x7f807c069920 2023-11-08T16:31:15.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(38 1.00000000.inode [delete] v39'36 uv36 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(33 601.00000000 [delete] v39'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(48 605.00000000 [delete] v39'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(34 602.00000000 [delete] v39'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(44 608.00000000 [delete] v39'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:51 4.b 4:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75fde10 con 0x7f8080029440 2023-11-08T16:31:15.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:52 4.7 4:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75ff970 con 0x7f8080011990 2023-11-08T16:31:15.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:53 4.f 4:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f75ffd40 con 0x7f8080011990 2023-11-08T16:31:15.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:54 4.1f 4:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f7601d00 con 0x7f808401b040 2023-11-08T16:31:15.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:55 4.1f 4:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e39) v8 -- 0x5575f7602540 con 0x7f808401b040 2023-11-08T16:31:15.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v39'9 uv9 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f8084077040 con 0x7f8080029440 2023-11-08T16:31:15.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.048+0000 7f808bf60640 1 -- 172.21.15.81:0/1646191478 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(45 604.00000000 [delete] v39'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f807c052870 con 0x7f8080013040 2023-11-08T16:31:15.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.049+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(35 10000000000.00000000 [delete] v39'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.049+0000 7f808bf60640 1 -- 172.21.15.81:0/1646191478 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(46 500.00000000 [delete] v39'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f807c052870 con 0x7f8080013040 2023-11-08T16:31:15.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.049+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(39 200.00000000 [delete] v39'37 uv37 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.051+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(36 600.00000000 [delete] v39'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.051+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(40 200.00000001 [delete] v39'38 uv38 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.051+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(37 603.00000000 [delete] v39'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.051+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(41 606.00000000 [delete] v39'39 uv39 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f808000f380 con 0x5575f7553f20 2023-11-08T16:31:15.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.052+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(49 mds0_inotable [delete] v39'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.4 v2:172.21.15.93:6800/3259756279 13 ==== osd_op_reply(50 100.00000000 [delete] v39'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f808000f380 con 0x7f807c069920 2023-11-08T16:31:15.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(52 609.00000000 [delete] v39'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8084077040 con 0x7f8080011990 2023-11-08T16:31:15.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(51 mds0_sessionmap [delete] v39'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f8084077040 con 0x7f8080029440 2023-11-08T16:31:15.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(53 400.00000000 [delete] v39'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f8084077040 con 0x7f8080011990 2023-11-08T16:31:15.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(54 100.00000000.inode [delete] v39'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f808000f380 con 0x7f808401b040 2023-11-08T16:31:15.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808b75f640 1 -- 172.21.15.81:0/1646191478 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(55 1.00000000 [delete] v39'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f808000f380 con 0x7f808401b040 2023-11-08T16:31:15.057 INFO:teuthology.orchestra.run.smithi081.stdout:Removed 23 objects 2023-11-08T16:31:15.058 INFO:teuthology.orchestra.run.smithi081.stdout:successfully purged pool cephfs_metadata 2023-11-08T16:31:15.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7f8080029440 msgr2=0x7f80800298a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7f8080029440 0x7f80800298a0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f8080011990 msgr2=0x7f8080015430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f8080011990 0x7f8080015430 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f80800212b0 msgr2=0x7f8080021730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f80800212b0 0x7f8080021730 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5575f7553f20 msgr2=0x5575f75fb0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5575f7553f20 0x5575f75fb0d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f807c069920 msgr2=0x7f807c05d3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f807c069920 0x7f807c05d3f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f8080013040 msgr2=0x7f808000e310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f8080013040 0x7f808000e310 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f808401b040 msgr2=0x7f8084057c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f808401b040 0x7f8084057c20 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 reap_dead start 2023-11-08T16:31:15.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.054+0000 7f808c761640 1 -- 172.21.15.81:0/1646191478 reap_dead start 2023-11-08T16:31:15.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.055+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8078069850 msgr2=0x7f807806bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.055+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8078069850 0x7f807806bd10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f807c054000 tx=0x7f807c04e000 comp rx=0 tx=0).stop 2023-11-08T16:31:15.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.055+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 msgr2=0x5575f7558880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.055+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f7558880 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f808400d5a0 tx=0x7f80840033a0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.055+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 shutdown_connections 2023-11-08T16:31:15.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.055+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8078069850 0x7f807806bd10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.055+0000 7f808f00ef40 1 --2- 172.21.15.81:0/1646191478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5575f7553960 0x5575f7558880 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.055+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 >> 172.21.15.81:0/1646191478 conn(0x5575f74c8c60 msgr2=0x5575f747dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:15.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.055+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 shutdown_connections 2023-11-08T16:31:15.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.055+0000 7f808f00ef40 1 -- 172.21.15.81:0/1646191478 wait complete. 2023-11-08T16:31:15.065 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-11-08T16:31:15.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.267+0000 7fe6fa597640 1 Processor -- start 2023-11-08T16:31:15.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.267+0000 7fe6fa597640 1 -- start start 2023-11-08T16:31:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.268+0000 7fe6fa597640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe6f40cab40 0x7fe6f40cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.268+0000 7fe6fa597640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe6f406b250 0x7fe6f40b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.269+0000 7fe6fa597640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f4160b40 con 0x7fe6f40b0a20 2023-11-08T16:31:15.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.269+0000 7fe6fa597640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f40caf20 con 0x7fe6f40cab40 2023-11-08T16:31:15.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.269+0000 7fe6fa597640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f40b0e30 con 0x7fe6f406b250 2023-11-08T16:31:15.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.269+0000 7fe6f3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe6f40cab40 0x7fe6f40cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.269+0000 7fe6f3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe6f40cab40 0x7fe6f40cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53006/0 (socket says 172.21.15.81:53006) 2023-11-08T16:31:15.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.269+0000 7fe6f3fff640 1 -- 172.21.15.81:0/2828169053 learned_addr learned my addr 172.21.15.81:0/2828169053 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:15.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.269+0000 7fe6f37fe640 1 --2- 172.21.15.81:0/2828169053 >> v2:172.21.15.93:3300/0 conn(0x7fe6f406b250 0x7fe6f40b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.269+0000 7fe6f3fff640 1 -- 172.21.15.81:0/2828169053 >> v2:172.21.15.93:3300/0 conn(0x7fe6f406b250 msgr2=0x7fe6f40b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:15.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.269+0000 7fe6f3fff640 1 --2- 172.21.15.81:0/2828169053 >> v2:172.21.15.93:3300/0 conn(0x7fe6f406b250 0x7fe6f40b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.269+0000 7fe6f3fff640 1 -- 172.21.15.81:0/2828169053 >> v1:172.21.15.93:6789/0 conn(0x7fe6f40b0a20 legacy=0x7fe6f4160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:15.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.269+0000 7fe6f3fff640 1 -- 172.21.15.81:0/2828169053 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f415db40 con 0x7fe6f40cab40 2023-11-08T16:31:15.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.270+0000 7fe6f3fff640 1 --2- 172.21.15.81:0/2828169053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6f40cab40 0x7fe6f40cd040 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe6f406c7d0 tx=0x7fe6e402b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66564cefbe8cf4d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:15.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.271+0000 7fe6f2ffd640 1 -- 172.21.15.81:0/2828169053 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4031750 con 0x7fe6f40cab40 2023-11-08T16:31:15.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.271+0000 7fe6f2ffd640 1 -- 172.21.15.81:0/2828169053 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e40318b0 con 0x7fe6f40cab40 2023-11-08T16:31:15.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.271+0000 7fe6f2ffd640 1 -- 172.21.15.81:0/2828169053 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4031c40 con 0x7fe6f40cab40 2023-11-08T16:31:15.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.271+0000 7fe6fa597640 1 -- 172.21.15.81:0/2828169053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6f40cab40 msgr2=0x7fe6f40cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.271+0000 7fe6fa597640 1 --2- 172.21.15.81:0/2828169053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6f40cab40 0x7fe6f40cd040 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe6f406c7d0 tx=0x7fe6e402b7f0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.271+0000 7fe6fa597640 1 -- 172.21.15.81:0/2828169053 shutdown_connections 2023-11-08T16:31:15.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.271+0000 7fe6fa597640 1 --2- 172.21.15.81:0/2828169053 >> v2:172.21.15.93:3300/0 conn(0x7fe6f406b250 0x7fe6f40b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.271+0000 7fe6fa597640 1 --2- 172.21.15.81:0/2828169053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6f40cab40 0x7fe6f40cd040 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.271+0000 7fe6fa597640 1 -- 172.21.15.81:0/2828169053 >> 172.21.15.81:0/2828169053 conn(0x7fe6f40cb4e0 msgr2=0x7fe6f40cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:15.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.272+0000 7fe6fa597640 1 -- 172.21.15.81:0/2828169053 shutdown_connections 2023-11-08T16:31:15.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.272+0000 7fe6fa597640 1 -- 172.21.15.81:0/2828169053 wait complete. 2023-11-08T16:31:15.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.274+0000 7fe6fa597640 1 Processor -- start 2023-11-08T16:31:15.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.274+0000 7fe6fa597640 1 -- start start 2023-11-08T16:31:15.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.275+0000 7fe6fa597640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6f406b250 0x7fe6f415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.275+0000 7fe6fa597640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6f40b0a20 0x7fe6f415be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.275+0000 7fe6fa597640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6f40cab40 0x7fe6f415c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.275+0000 7fe6fa597640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40b16e0 con 0x7fe6f40b0a20 2023-11-08T16:31:15.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.275+0000 7fe6fa597640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40c88f0 con 0x7fe6f40cab40 2023-11-08T16:31:15.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.275+0000 7fe6fa597640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f40c2610 con 0x7fe6f406b250 2023-11-08T16:31:15.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.275+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6f40b0a20 0x7fe6f415be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.275+0000 7fe6f37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6f40b0a20 0x7fe6f415be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53018/0 (socket says 172.21.15.81:53018) 2023-11-08T16:31:15.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.275+0000 7fe6f37fe640 1 -- 172.21.15.81:0/531010515 learned_addr learned my addr 172.21.15.81:0/531010515 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:15.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6f8b34640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6f40cab40 0x7fe6f415c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6f3fff640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6f406b250 0x7fe6f415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6f37fe640 1 -- 172.21.15.81:0/531010515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6f406b250 msgr2=0x7fe6f415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6f37fe640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6f406b250 0x7fe6f415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6f37fe640 1 -- 172.21.15.81:0/531010515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6f40cab40 msgr2=0x7fe6f415c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:15.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6f37fe640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6f40cab40 0x7fe6f415c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:15.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6f37fe640 1 -- 172.21.15.81:0/531010515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e40094d0 con 0x7fe6f40b0a20 2023-11-08T16:31:15.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6f37fe640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6f40b0a20 0x7fe6f415be10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800d800 tx=0x7fe6e800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:15.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6f0ff9640 1 -- 172.21.15.81:0/531010515 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e800b6c0 con 0x7fe6f40b0a20 2023-11-08T16:31:15.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6f0ff9640 1 -- 172.21.15.81:0/531010515 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e800b820 con 0x7fe6f40b0a20 2023-11-08T16:31:15.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6f0ff9640 1 -- 172.21.15.81:0/531010515 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e800bbb0 con 0x7fe6f40b0a20 2023-11-08T16:31:15.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.276+0000 7fe6fa597640 1 -- 172.21.15.81:0/531010515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f40c28f0 con 0x7fe6f40b0a20 2023-11-08T16:31:15.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.277+0000 7fe6fa597640 1 -- 172.21.15.81:0/531010515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f40c2eb0 con 0x7fe6f40b0a20 2023-11-08T16:31:15.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.278+0000 7fe6fa597640 1 -- 172.21.15.81:0/531010515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f40c3040 con 0x7fe6f40b0a20 2023-11-08T16:31:15.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.279+0000 7fe6f0ff9640 1 -- 172.21.15.81:0/531010515 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6e800bd10 con 0x7fe6f40b0a20 2023-11-08T16:31:15.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.280+0000 7fe6f0ff9640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6e006ae70 0x7fe6e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:15.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.280+0000 7fe6f0ff9640 1 -- 172.21.15.81:0/531010515 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6199+0+0 (secure 0 0 0) 0x7fe6e8089da0 con 0x7fe6f40b0a20 2023-11-08T16:31:15.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.280+0000 7fe6f3fff640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6e006ae70 0x7fe6e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:15.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.280+0000 7fe6f3fff640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6e006ae70 0x7fe6e006d330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4035000 tx=0x7fe6e4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:15.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.284+0000 7fe6f0ff9640 1 -- 172.21.15.81:0/531010515 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6e8054c60 con 0x7fe6f40b0a20 2023-11-08T16:31:15.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:15.513+0000 7fe6fa597640 1 -- 172.21.15.81:0/531010515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe6f404ece0 con 0x7fe6f40b0a20 2023-11-08T16:31:16.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.372+0000 7fe6f0ff9640 1 -- 172.21.15.81:0/531010515 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 107+0+0 (secure 0 0 0) 0x7fe6e8058280 con 0x7fe6f40b0a20 2023-11-08T16:31:16.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.375+0000 7fe6fa597640 1 -- 172.21.15.81:0/531010515 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6e006ae70 msgr2=0x7fe6e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:16.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.375+0000 7fe6fa597640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6e006ae70 0x7fe6e006d330 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe6e4035000 tx=0x7fe6e4008030 comp rx=0 tx=0).stop 2023-11-08T16:31:16.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.375+0000 7fe6fa597640 1 -- 172.21.15.81:0/531010515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6f40b0a20 msgr2=0x7fe6f415be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:16.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.375+0000 7fe6fa597640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6f40b0a20 0x7fe6f415be10 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe6e800d800 tx=0x7fe6e800dcc0 comp rx=0 tx=0).stop 2023-11-08T16:31:16.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.376+0000 7fe6fa597640 1 -- 172.21.15.81:0/531010515 shutdown_connections 2023-11-08T16:31:16.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.376+0000 7fe6fa597640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe6e006ae70 0x7fe6e006d330 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:16.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.376+0000 7fe6fa597640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6f40cab40 0x7fe6f415c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:16.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.376+0000 7fe6fa597640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6f40b0a20 0x7fe6f415be10 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:16.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.376+0000 7fe6fa597640 1 --2- 172.21.15.81:0/531010515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6f406b250 0x7fe6f415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:16.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.376+0000 7fe6fa597640 1 -- 172.21.15.81:0/531010515 >> 172.21.15.81:0/531010515 conn(0x7fe6f40cb4e0 msgr2=0x7fe6f406b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:16.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.376+0000 7fe6fa597640 1 -- 172.21.15.81:0/531010515 shutdown_connections 2023-11-08T16:31:16.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.376+0000 7fe6fa597640 1 -- 172.21.15.81:0/531010515 wait complete. 2023-11-08T16:31:16.385 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:16.384+0000 7f3a69727640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-11-08T16:31:16.386 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:16.384+0000 7f3a69727640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-11-08T16:31:16.386 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:31:16.385+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:31:16.386 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:31:16.385+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-08T16:31:16.396 INFO:tasks.ceph.mds.c.smithi081.stderr: -31> 2023-11-08T16:31:16.384+0000 7f3a69727640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-11-08T16:31:16.396 INFO:tasks.ceph.mds.c.smithi081.stderr: -28> 2023-11-08T16:31:16.384+0000 7f3a69727640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-11-08T16:31:16.401 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-08T16:31:16.446 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:31:16.494 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:31:16.502 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:31:16.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.590+0000 7f75c586b640 1 Processor -- start 2023-11-08T16:31:16.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.591+0000 7f75c586b640 1 -- start start 2023-11-08T16:31:16.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.592+0000 7f75c586b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f75c00ce0d0 0x7f75c00caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:16.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.592+0000 7f75c586b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75c015bfc0 0x7f75c015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:16.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.592+0000 7f75c586b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75c006fa30 con 0x7f75c015f0b0 2023-11-08T16:31:16.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.592+0000 7f75c586b640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75c015f540 con 0x7f75c00ce0d0 2023-11-08T16:31:16.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.592+0000 7f75c586b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c00cda10 con 0x7f75c015bfc0 2023-11-08T16:31:16.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.592+0000 7f75beffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f75c00ce0d0 0x7f75c00caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:16.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.592+0000 7f75bf7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f75c015f0b0 0x7f75c01516e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40532/0 (socket says 172.21.15.81:40532) 2023-11-08T16:31:16.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.592+0000 7f75bf7fe640 1 -- 172.21.15.81:0/2015858598 learned_addr learned my addr 172.21.15.81:0/2015858598 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:16.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.593+0000 7f75be7fc640 1 --2- 172.21.15.81:0/2015858598 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75c015bfc0 0x7f75c015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:16.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.593+0000 7f75beffd640 1 -- 172.21.15.81:0/2015858598 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75c015bfc0 msgr2=0x7f75c015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:16.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.593+0000 7f75beffd640 1 --2- 172.21.15.81:0/2015858598 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75c015bfc0 0x7f75c015eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:16.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.593+0000 7f75beffd640 1 -- 172.21.15.81:0/2015858598 >> v1:172.21.15.81:6789/0 conn(0x7f75c015f0b0 legacy=0x7f75c01516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:16.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.593+0000 7f75beffd640 1 -- 172.21.15.81:0/2015858598 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c00ce4b0 con 0x7f75c00ce0d0 2023-11-08T16:31:16.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.593+0000 7f75bdffb640 1 -- 172.21.15.81:0/2015858598 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936857666 0 0) 0x7f75c006fa30 con 0x7f75c015f0b0 2023-11-08T16:31:16.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.594+0000 7f75beffd640 1 --2- 172.21.15.81:0/2015858598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75c00ce0d0 0x7f75c00caf20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f75b00097a0 tx=0x7f75b002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb6cc5e6b6ddd29 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:16.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.594+0000 7f75bdffb640 1 -- 172.21.15.81:0/2015858598 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b002d040 con 0x7f75c00ce0d0 2023-11-08T16:31:16.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.594+0000 7f75bdffb640 1 -- 172.21.15.81:0/2015858598 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b0031810 con 0x7f75c00ce0d0 2023-11-08T16:31:16.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.594+0000 7f75bdffb640 1 -- 172.21.15.81:0/2015858598 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b0031ba0 con 0x7f75c00ce0d0 2023-11-08T16:31:16.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.595+0000 7f75c586b640 1 -- 172.21.15.81:0/2015858598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75c00ce0d0 msgr2=0x7f75c00caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:16.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.595+0000 7f75c586b640 1 --2- 172.21.15.81:0/2015858598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75c00ce0d0 0x7f75c00caf20 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f75b00097a0 tx=0x7f75b002b8b0 comp rx=0 tx=0).stop 2023-11-08T16:31:16.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.595+0000 7f75c586b640 1 -- 172.21.15.81:0/2015858598 shutdown_connections 2023-11-08T16:31:16.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.595+0000 7f75c586b640 1 --2- 172.21.15.81:0/2015858598 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75c015bfc0 0x7f75c015eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:16.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.595+0000 7f75c586b640 1 --2- 172.21.15.81:0/2015858598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75c00ce0d0 0x7f75c00caf20 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:16.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.595+0000 7f75c586b640 1 -- 172.21.15.81:0/2015858598 >> 172.21.15.81:0/2015858598 conn(0x7f75c00cb580 msgr2=0x7f75c00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:16.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.595+0000 7f75c586b640 1 -- 172.21.15.81:0/2015858598 shutdown_connections 2023-11-08T16:31:16.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.595+0000 7f75c586b640 1 -- 172.21.15.81:0/2015858598 wait complete. 2023-11-08T16:31:16.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.597+0000 7f75c586b640 1 Processor -- start 2023-11-08T16:31:16.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.597+0000 7f75c586b640 1 -- start start 2023-11-08T16:31:16.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.598+0000 7f75c586b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75c00ce0d0 0x7f75c015a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:16.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.598+0000 7f75c586b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75c015bfc0 0x7f75c00b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:16.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.598+0000 7f75c586b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f75c015f0b0 0x7f75c014ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:16.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.598+0000 7f75c586b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c006c040 con 0x7f75c015bfc0 2023-11-08T16:31:16.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.598+0000 7f75c586b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c00b2140 con 0x7f75c015f0b0 2023-11-08T16:31:16.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.598+0000 7f75c586b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c006b710 con 0x7f75c00ce0d0 2023-11-08T16:31:16.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.598+0000 7f75be7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75c015bfc0 0x7f75c00b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:16.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.598+0000 7f75be7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75c015bfc0 0x7f75c00b1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53044/0 (socket says 172.21.15.81:53044) 2023-11-08T16:31:16.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.598+0000 7f75be7fc640 1 -- 172.21.15.81:0/2426139893 learned_addr learned my addr 172.21.15.81:0/2426139893 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:16.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.598+0000 7f75beffd640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75c00ce0d0 0x7f75c015a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:16.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.598+0000 7f75bf7fe640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f75c015f0b0 0x7f75c014ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:16.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75be7fc640 1 -- 172.21.15.81:0/2426139893 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75c00ce0d0 msgr2=0x7f75c015a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:16.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75be7fc640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75c00ce0d0 0x7f75c015a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:16.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75be7fc640 1 -- 172.21.15.81:0/2426139893 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f75c015f0b0 msgr2=0x7f75c014ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:16.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75be7fc640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f75c015f0b0 0x7f75c014ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:16.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75be7fc640 1 -- 172.21.15.81:0/2426139893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b0009400 con 0x7f75c015bfc0 2023-11-08T16:31:16.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75beffd640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75c00ce0d0 0x7f75c015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:31:16.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75be7fc640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75c015bfc0 0x7f75c00b1640 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f75b400d800 tx=0x7f75b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:16.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75a7fff640 1 -- 172.21.15.81:0/2426139893 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b400b6c0 con 0x7f75c015bfc0 2023-11-08T16:31:16.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75a7fff640 1 -- 172.21.15.81:0/2426139893 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b400b820 con 0x7f75c015bfc0 2023-11-08T16:31:16.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75a7fff640 1 -- 172.21.15.81:0/2426139893 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b400bbb0 con 0x7f75c015bfc0 2023-11-08T16:31:16.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75c586b640 1 -- 172.21.15.81:0/2426139893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75c006b9f0 con 0x7f75c015bfc0 2023-11-08T16:31:16.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.599+0000 7f75c586b640 1 -- 172.21.15.81:0/2426139893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75c00b4480 con 0x7f75c015bfc0 2023-11-08T16:31:16.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.600+0000 7f75c586b640 1 -- 172.21.15.81:0/2426139893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75c004ece0 con 0x7f75c015bfc0 2023-11-08T16:31:16.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.601+0000 7f75a7fff640 1 -- 172.21.15.81:0/2426139893 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f75b400bd10 con 0x7f75c015bfc0 2023-11-08T16:31:16.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.602+0000 7f75a7fff640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75ac06ae70 0x7f75ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:16.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.602+0000 7f75a7fff640 1 -- 172.21.15.81:0/2426139893 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f75b4089dc0 con 0x7f75c015bfc0 2023-11-08T16:31:16.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.603+0000 7f75beffd640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75ac06ae70 0x7f75ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:16.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.603+0000 7f75beffd640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75ac06ae70 0x7f75ac06d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f75b0036000 tx=0x7f75b003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:16.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.607+0000 7f75a7fff640 1 -- 172.21.15.81:0/2426139893 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f75b4054c20 con 0x7f75c015bfc0 2023-11-08T16:31:16.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:16.830+0000 7f75c586b640 1 -- 172.21.15.81:0/2426139893 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f75c015afc0 con 0x7f75c015bfc0 2023-11-08T16:31:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.400+0000 7f75a7fff640 1 -- 172.21.15.81:0/2426139893 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v25) v1 ==== 162+0+0 (secure 0 0 0) 0x7f75b4058240 con 0x7f75c015bfc0 2023-11-08T16:31:17.402 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-08T16:31:17.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.403+0000 7f75c586b640 1 -- 172.21.15.81:0/2426139893 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75ac06ae70 msgr2=0x7f75ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:17.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.403+0000 7f75c586b640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75ac06ae70 0x7f75ac06d330 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f75b0036000 tx=0x7f75b003c040 comp rx=0 tx=0).stop 2023-11-08T16:31:17.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.403+0000 7f75c586b640 1 -- 172.21.15.81:0/2426139893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75c015bfc0 msgr2=0x7f75c00b1640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:17.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.403+0000 7f75c586b640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75c015bfc0 0x7f75c00b1640 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f75b400d800 tx=0x7f75b400dcc0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.404+0000 7f75c586b640 1 -- 172.21.15.81:0/2426139893 shutdown_connections 2023-11-08T16:31:17.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.404+0000 7f75c586b640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75ac06ae70 0x7f75ac06d330 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.404+0000 7f75c586b640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f75c015f0b0 0x7f75c014ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.404+0000 7f75c586b640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75c015bfc0 0x7f75c00b1640 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.404+0000 7f75c586b640 1 --2- 172.21.15.81:0/2426139893 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75c00ce0d0 0x7f75c015a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.404+0000 7f75c586b640 1 -- 172.21.15.81:0/2426139893 >> 172.21.15.81:0/2426139893 conn(0x7f75c00cb580 msgr2=0x7f75c00c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:17.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.404+0000 7f75c586b640 1 -- 172.21.15.81:0/2426139893 shutdown_connections 2023-11-08T16:31:17.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.404+0000 7f75c586b640 1 -- 172.21.15.81:0/2426139893 wait complete. 2023-11-08T16:31:17.426 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:31:17.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.684+0000 7fc6f612c640 1 Processor -- start 2023-11-08T16:31:17.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.684+0000 7fc6f612c640 1 -- start start 2023-11-08T16:31:17.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.686+0000 7fc6f612c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc6f00ca480 0x7fc6f015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:17.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.686+0000 7fc6f612c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc6f015ea60 0x7fc6f015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:17.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.686+0000 7fc6f612c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f006b630 con 0x7fc6f00ade50 2023-11-08T16:31:17.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.686+0000 7fc6f612c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f00b1860 con 0x7fc6f015ea60 2023-11-08T16:31:17.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.686+0000 7fc6f612c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f00ca910 con 0x7fc6f00ca480 2023-11-08T16:31:17.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.686+0000 7fc6ef7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc6f00ade50 0x7fc6f015a680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40544/0 (socket says 172.21.15.81:40544) 2023-11-08T16:31:17.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.686+0000 7fc6ef7fe640 1 -- 172.21.15.81:0/3373635544 learned_addr learned my addr 172.21.15.81:0/3373635544 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:17.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.686+0000 7fc6effff640 1 --2- 172.21.15.81:0/3373635544 >> v2:172.21.15.81:3300/0 conn(0x7fc6f015ea60 0x7fc6f015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:17.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.686+0000 7fc6eeffd640 1 -- 172.21.15.81:0/3373635544 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1360804863 0 0) 0x7fc6f006b630 con 0x7fc6f00ade50 2023-11-08T16:31:17.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.686+0000 7fc6eeffd640 1 -- 172.21.15.81:0/3373635544 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6dc003520 con 0x7fc6f00ade50 2023-11-08T16:31:17.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.686+0000 7fc6e7fff640 1 --2- 172.21.15.81:0/3373635544 >> v2:172.21.15.93:3300/0 conn(0x7fc6f00ca480 0x7fc6f015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:17.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.687+0000 7fc6effff640 1 -- 172.21.15.81:0/3373635544 >> v2:172.21.15.93:3300/0 conn(0x7fc6f00ca480 msgr2=0x7fc6f015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:17.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.687+0000 7fc6effff640 1 --2- 172.21.15.81:0/3373635544 >> v2:172.21.15.93:3300/0 conn(0x7fc6f00ca480 0x7fc6f015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.687+0000 7fc6effff640 1 -- 172.21.15.81:0/3373635544 >> v1:172.21.15.81:6789/0 conn(0x7fc6f00ade50 legacy=0x7fc6f015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:17.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.687+0000 7fc6effff640 1 -- 172.21.15.81:0/3373635544 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f00cabb0 con 0x7fc6f015ea60 2023-11-08T16:31:17.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.688+0000 7fc6effff640 1 --2- 172.21.15.81:0/3373635544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6f015ea60 0x7fc6f015eef0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0073d00 tx=0x7fc6e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2d8852efed9b47 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:17.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.688+0000 7fc6eeffd640 1 -- 172.21.15.81:0/3373635544 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e802cc80 con 0x7fc6f015ea60 2023-11-08T16:31:17.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.688+0000 7fc6eeffd640 1 -- 172.21.15.81:0/3373635544 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e802cde0 con 0x7fc6f015ea60 2023-11-08T16:31:17.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.688+0000 7fc6eeffd640 1 -- 172.21.15.81:0/3373635544 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8034660 con 0x7fc6f015ea60 2023-11-08T16:31:17.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.689+0000 7fc6f612c640 1 -- 172.21.15.81:0/3373635544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6f015ea60 msgr2=0x7fc6f015eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:17.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.689+0000 7fc6f612c640 1 --2- 172.21.15.81:0/3373635544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6f015ea60 0x7fc6f015eef0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0073d00 tx=0x7fc6e802bed0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.689+0000 7fc6f612c640 1 -- 172.21.15.81:0/3373635544 shutdown_connections 2023-11-08T16:31:17.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.689+0000 7fc6f612c640 1 --2- 172.21.15.81:0/3373635544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6f015ea60 0x7fc6f015eef0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.689+0000 7fc6f612c640 1 --2- 172.21.15.81:0/3373635544 >> v2:172.21.15.93:3300/0 conn(0x7fc6f00ca480 0x7fc6f015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.689+0000 7fc6f612c640 1 -- 172.21.15.81:0/3373635544 >> 172.21.15.81:0/3373635544 conn(0x7fc6f00cb490 msgr2=0x7fc6f00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:17.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.689+0000 7fc6f612c640 1 -- 172.21.15.81:0/3373635544 shutdown_connections 2023-11-08T16:31:17.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.689+0000 7fc6f612c640 1 -- 172.21.15.81:0/3373635544 wait complete. 2023-11-08T16:31:17.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.690+0000 7fc6f612c640 1 Processor -- start 2023-11-08T16:31:17.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.691+0000 7fc6f612c640 1 -- start start 2023-11-08T16:31:17.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.691+0000 7fc6f612c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc6f00ade50 0x7fc6f0156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:17.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.691+0000 7fc6f612c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f00ca480 0x7fc6f01531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:17.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.691+0000 7fc6f612c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6f015ea60 0x7fc6f00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:17.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6f612c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0155f80 con 0x7fc6f015ea60 2023-11-08T16:31:17.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6f612c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f01568c0 con 0x7fc6f00ca480 2023-11-08T16:31:17.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6f612c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00ccae0 con 0x7fc6f00ade50 2023-11-08T16:31:17.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6effff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6f015ea60 0x7fc6f00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:17.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6effff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6f015ea60 0x7fc6f00ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53050/0 (socket says 172.21.15.81:53050) 2023-11-08T16:31:17.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6effff640 1 -- 172.21.15.81:0/459731050 learned_addr learned my addr 172.21.15.81:0/459731050 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:17.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6effff640 1 -- 172.21.15.81:0/459731050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc6f00ade50 msgr2=0x7fc6f0156380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:17.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6effff640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc6f00ade50 0x7fc6f0156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6effff640 1 -- 172.21.15.81:0/459731050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f00ca480 msgr2=0x7fc6f01531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:17.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6effff640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f00ca480 0x7fc6f01531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6effff640 1 -- 172.21.15.81:0/459731050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8009a70 con 0x7fc6f015ea60 2023-11-08T16:31:17.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.692+0000 7fc6effff640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6f015ea60 0x7fc6f00ac8b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009e80 tx=0x7fc6e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:17.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.693+0000 7fc6ecff9640 1 -- 172.21.15.81:0/459731050 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8040650 con 0x7fc6f015ea60 2023-11-08T16:31:17.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.693+0000 7fc6ecff9640 1 -- 172.21.15.81:0/459731050 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e80407b0 con 0x7fc6f015ea60 2023-11-08T16:31:17.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.693+0000 7fc6f612c640 1 -- 172.21.15.81:0/459731050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6f00ccdc0 con 0x7fc6f015ea60 2023-11-08T16:31:17.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.693+0000 7fc6ecff9640 1 -- 172.21.15.81:0/459731050 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8040b60 con 0x7fc6f015ea60 2023-11-08T16:31:17.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.693+0000 7fc6f612c640 1 -- 172.21.15.81:0/459731050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f00cd380 con 0x7fc6f015ea60 2023-11-08T16:31:17.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.694+0000 7fc6f612c640 1 -- 172.21.15.81:0/459731050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f004ece0 con 0x7fc6f015ea60 2023-11-08T16:31:17.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.695+0000 7fc6ecff9640 1 -- 172.21.15.81:0/459731050 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc6e8040cc0 con 0x7fc6f015ea60 2023-11-08T16:31:17.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.695+0000 7fc6ecff9640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc6d006ae70 0x7fc6d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:17.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.696+0000 7fc6ecff9640 1 -- 172.21.15.81:0/459731050 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fc6e80ac5f0 con 0x7fc6f015ea60 2023-11-08T16:31:17.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.696+0000 7fc6ef7fe640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc6d006ae70 0x7fc6d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:17.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.696+0000 7fc6ef7fe640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc6d006ae70 0x7fc6d006d330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0157970 tx=0x7fc6e0007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:17.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.700+0000 7fc6ecff9640 1 -- 172.21.15.81:0/459731050 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc6e80773c0 con 0x7fc6f015ea60 2023-11-08T16:31:17.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.929+0000 7fc6f612c640 1 -- 172.21.15.81:0/459731050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6f00cc1d0 con 0x7fc6f015ea60 2023-11-08T16:31:17.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.930+0000 7fc6ecff9640 1 -- 172.21.15.81:0/459731050 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7fc6e8009450 con 0x7fc6f015ea60 2023-11-08T16:31:17.932 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:17.932 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/1453041994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":1453041994},{"type":"v1","addr":"172.21.15.93:6835","nonce":1453041994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/757624701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":757624701},{"type":"v1","addr":"172.21.15.93:6833","nonce":757624701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1735232404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1735232404},{"type":"v1","addr":"172.21.15.81:6835","nonce":1735232404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1759199271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1759199271},{"type":"v1","addr":"172.21.15.81:6837","nonce":1759199271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:16.832167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:31:17.933 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 25 2023-11-08T16:31:17.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 -- 172.21.15.81:0/459731050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc6d006ae70 msgr2=0x7fc6d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:17.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc6d006ae70 0x7fc6d006d330 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc6f0157970 tx=0x7fc6e0007930 comp rx=0 tx=0).stop 2023-11-08T16:31:17.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 -- 172.21.15.81:0/459731050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6f015ea60 msgr2=0x7fc6f00ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:17.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6f015ea60 0x7fc6f00ac8b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009e80 tx=0x7fc6e8004a40 comp rx=0 tx=0).stop 2023-11-08T16:31:17.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 -- 172.21.15.81:0/459731050 shutdown_connections 2023-11-08T16:31:17.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc6d006ae70 0x7fc6d006d330 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6f015ea60 0x7fc6f00ac8b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc6f00ca480 0x7fc6f01531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 --2- 172.21.15.81:0/459731050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc6f00ade50 0x7fc6f0156380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:17.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 -- 172.21.15.81:0/459731050 >> 172.21.15.81:0/459731050 conn(0x7fc6f00cb490 msgr2=0x7fc6f0153af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:17.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 -- 172.21.15.81:0/459731050 shutdown_connections 2023-11-08T16:31:17.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:17.933+0000 7fc6f612c640 1 -- 172.21.15.81:0/459731050 wait complete. 2023-11-08T16:31:17.952 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T16:31:17.952 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:31:18.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.163+0000 7f6bc7c32640 1 Processor -- start 2023-11-08T16:31:18.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.163+0000 7f6bc7c32640 1 -- start start 2023-11-08T16:31:18.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.165+0000 7f6bc7c32640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc00ade50 0x7f6bc00ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.165+0000 7f6bc7c32640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6bc015a680 0x7f6bc015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.165+0000 7f6bc7c32640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc006b630 con 0x7f6bc015b020 2023-11-08T16:31:18.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.165+0000 7f6bc7c32640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc00b1860 con 0x7f6bc015a680 2023-11-08T16:31:18.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.165+0000 7f6bc7c32640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc00ca9c0 con 0x7f6bc00ade50 2023-11-08T16:31:18.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.165+0000 7f6bc61cf640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f6bc015b020 0x7f6bc00c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40548/0 (socket says 172.21.15.81:40548) 2023-11-08T16:31:18.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.165+0000 7f6bc61cf640 1 -- 172.21.15.81:0/2436445460 learned_addr learned my addr 172.21.15.81:0/2436445460 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:18.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc59ce640 1 --2- 172.21.15.81:0/2436445460 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc00ade50 0x7f6bc00ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc51cd640 1 --2- 172.21.15.81:0/2436445460 >> v2:172.21.15.93:3300/0 conn(0x7f6bc015a680 0x7f6bc015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1088767281 0 0) 0x7f6bc006b630 con 0x7f6bc015b020 2023-11-08T16:31:18.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bac003520 con 0x7f6bc015b020 2023-11-08T16:31:18.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1912519516 0 0) 0x7f6bac003520 con 0x7f6bc015b020 2023-11-08T16:31:18.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc00ade50 msgr2=0x7f6bc00ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:18.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 --2- 172.21.15.81:0/2436445460 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc00ade50 0x7f6bc00ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 >> v2:172.21.15.93:3300/0 conn(0x7f6bc015a680 msgr2=0x7f6bc015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:18.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 --2- 172.21.15.81:0/2436445460 >> v2:172.21.15.93:3300/0 conn(0x7f6bc015a680 0x7f6bc015aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00cabb0 con 0x7f6bc015b020 2023-11-08T16:31:18.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f6bb80019f0 con 0x7f6bc015b020 2023-11-08T16:31:18.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 >> v1:172.21.15.81:6789/0 conn(0x7f6bc015b020 legacy=0x7f6bc00c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 --2- 172.21.15.81:0/2436445460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bac003a50 0x7f6bac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.166+0000 7f6bc49cc640 1 --2- 172.21.15.81:0/2436445460 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc00ade50 0x7f6bac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.167+0000 7f6bc49cc640 1 --2- 172.21.15.81:0/2436445460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bac004d00 0x7f6bac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.167+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac00b980 con 0x7f6bac003a50 2023-11-08T16:31:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.167+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac00bac0 con 0x7f6bac004d00 2023-11-08T16:31:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.167+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac00bc30 con 0x7f6bc00ade50 2023-11-08T16:31:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.167+0000 7f6bc59ce640 1 --2- 172.21.15.81:0/2436445460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bac003a50 0x7f6bac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.167+0000 7f6bc59ce640 1 -- 172.21.15.81:0/2436445460 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc00ade50 msgr2=0x7f6bac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.167+0000 7f6bc59ce640 1 --2- 172.21.15.81:0/2436445460 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc00ade50 0x7f6bac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.167+0000 7f6bc59ce640 1 -- 172.21.15.81:0/2436445460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bac004d00 msgr2=0x7f6bac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.167+0000 7f6bc61cf640 1 --2- 172.21.15.81:0/2436445460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bac004d00 0x7f6bac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.167+0000 7f6bc59ce640 1 --2- 172.21.15.81:0/2436445460 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bac004d00 0x7f6bac005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.167+0000 7f6bc59ce640 1 -- 172.21.15.81:0/2436445460 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac003520 con 0x7f6bac003a50 2023-11-08T16:31:18.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.168+0000 7f6bc59ce640 1 --2- 172.21.15.81:0/2436445460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bac003a50 0x7f6bac003e70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009ac0 tx=0x7f6bac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec9526b0b615468 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:18.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.169+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc02c710 con 0x7f6bac003a50 2023-11-08T16:31:18.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.169+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bbc02c870 con 0x7f6bac003a50 2023-11-08T16:31:18.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.169+0000 7f6bc49cc640 1 -- 172.21.15.81:0/2436445460 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc02cc00 con 0x7f6bac003a50 2023-11-08T16:31:18.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.169+0000 7f6bc7c32640 1 -- 172.21.15.81:0/2436445460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bac003a50 msgr2=0x7f6bac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.169+0000 7f6bc7c32640 1 --2- 172.21.15.81:0/2436445460 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bac003a50 0x7f6bac003e70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009ac0 tx=0x7f6bac0057f0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.169+0000 7f6bc61cf640 1 -- 172.21.15.81:0/2436445460 reap_dead start 2023-11-08T16:31:18.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.169+0000 7f6bc7c32640 1 -- 172.21.15.81:0/2436445460 shutdown_connections 2023-11-08T16:31:18.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.169+0000 7f6bc7c32640 1 -- 172.21.15.81:0/2436445460 >> 172.21.15.81:0/2436445460 conn(0x7f6bc00cb490 msgr2=0x7f6bc00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:18.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.169+0000 7f6bc7c32640 1 -- 172.21.15.81:0/2436445460 shutdown_connections 2023-11-08T16:31:18.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.169+0000 7f6bc7c32640 1 -- 172.21.15.81:0/2436445460 wait complete. 2023-11-08T16:31:18.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.171+0000 7f6bc7c32640 1 Processor -- start 2023-11-08T16:31:18.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.171+0000 7f6bc7c32640 1 -- start start 2023-11-08T16:31:18.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.172+0000 7f6bc7c32640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bac003a50 0x7f6bc015c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.172+0000 7f6bc7c32640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc014fd30 0x7f6bc0148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.172+0000 7f6bc7c32640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc0152b70 0x7f6bc00b3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.172+0000 7f6bc7c32640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc01537a0 con 0x7f6bc0152b70 2023-11-08T16:31:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.172+0000 7f6bc7c32640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc0149260 con 0x7f6bac003a50 2023-11-08T16:31:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.172+0000 7f6bc7c32640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc01534c0 con 0x7f6bc014fd30 2023-11-08T16:31:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.173+0000 7f6bc61cf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc0152b70 0x7f6bc00b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.173+0000 7f6bc61cf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc0152b70 0x7f6bc00b3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53066/0 (socket says 172.21.15.81:53066) 2023-11-08T16:31:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.173+0000 7f6bc61cf640 1 -- 172.21.15.81:0/1030461683 learned_addr learned my addr 172.21.15.81:0/1030461683 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.173+0000 7f6bc59ce640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bac003a50 0x7f6bc015c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.173+0000 7f6bc51cd640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc014fd30 0x7f6bc0148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.173+0000 7f6bc61cf640 1 -- 172.21.15.81:0/1030461683 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc014fd30 msgr2=0x7f6bc0148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.173+0000 7f6bc61cf640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc014fd30 0x7f6bc0148d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.173+0000 7f6bc61cf640 1 -- 172.21.15.81:0/1030461683 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bac003a50 msgr2=0x7f6bc015c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.173+0000 7f6bc61cf640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bac003a50 0x7f6bc015c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.173+0000 7f6bc61cf640 1 -- 172.21.15.81:0/1030461683 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc015b700 con 0x7f6bc0152b70 2023-11-08T16:31:18.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.173+0000 7f6bc61cf640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc0152b70 0x7f6bc00b3440 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6bc015b400 tx=0x7f6bb800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:18.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.174+0000 7f6bb67fc640 1 -- 172.21.15.81:0/1030461683 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8004d60 con 0x7f6bc0152b70 2023-11-08T16:31:18.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.174+0000 7f6bb67fc640 1 -- 172.21.15.81:0/1030461683 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb800bb20 con 0x7f6bc0152b70 2023-11-08T16:31:18.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.174+0000 7f6bb67fc640 1 -- 172.21.15.81:0/1030461683 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb800bed0 con 0x7f6bc0152b70 2023-11-08T16:31:18.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.174+0000 7f6bc7c32640 1 -- 172.21.15.81:0/1030461683 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc015b9e0 con 0x7f6bc0152b70 2023-11-08T16:31:18.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.174+0000 7f6bc7c32640 1 -- 172.21.15.81:0/1030461683 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc0153000 con 0x7f6bc0152b70 2023-11-08T16:31:18.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.175+0000 7f6bc7c32640 1 -- 172.21.15.81:0/1030461683 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc015b560 con 0x7f6bc0152b70 2023-11-08T16:31:18.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.176+0000 7f6bb67fc640 1 -- 172.21.15.81:0/1030461683 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6bb8007070 con 0x7f6bc0152b70 2023-11-08T16:31:18.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.177+0000 7f6bb67fc640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ba406ae70 0x7f6ba406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.177+0000 7f6bb67fc640 1 -- 172.21.15.81:0/1030461683 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f6bb8089a50 con 0x7f6bc0152b70 2023-11-08T16:31:18.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.177+0000 7f6bc59ce640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ba406ae70 0x7f6ba406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.178+0000 7f6bc59ce640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ba406ae70 0x7f6ba406d330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009ac0 tx=0x7f6bbc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:18.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.181+0000 7f6bb67fc640 1 -- 172.21.15.81:0/1030461683 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6bc015b560 con 0x7f6bc0152b70 2023-11-08T16:31:18.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.414+0000 7f6bc7c32640 1 -- 172.21.15.81:0/1030461683 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bc015bf00 con 0x7f6bc0152b70 2023-11-08T16:31:18.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.416+0000 7f6bb67fc640 1 -- 172.21.15.81:0/1030461683 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f6bc015bf00 con 0x7f6bc0152b70 2023-11-08T16:31:18.418 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:18.418 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/1453041994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":1453041994},{"type":"v1","addr":"172.21.15.93:6835","nonce":1453041994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/757624701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":757624701},{"type":"v1","addr":"172.21.15.93:6833","nonce":757624701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1735232404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1735232404},{"type":"v1","addr":"172.21.15.81:6835","nonce":1735232404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1759199271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1759199271},{"type":"v1","addr":"172.21.15.81:6837","nonce":1759199271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:16.832167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:31:18.418 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 25 2023-11-08T16:31:18.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 -- 172.21.15.81:0/1030461683 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ba406ae70 msgr2=0x7f6ba406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ba406ae70 0x7f6ba406d330 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009ac0 tx=0x7f6bbc030000 comp rx=0 tx=0).stop 2023-11-08T16:31:18.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 -- 172.21.15.81:0/1030461683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc0152b70 msgr2=0x7f6bc00b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc0152b70 0x7f6bc00b3440 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6bc015b400 tx=0x7f6bb800b4a0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 -- 172.21.15.81:0/1030461683 shutdown_connections 2023-11-08T16:31:18.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6ba406ae70 0x7f6ba406d330 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6bc0152b70 0x7f6bc00b3440 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6bc014fd30 0x7f6bc0148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 --2- 172.21.15.81:0/1030461683 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6bac003a50 0x7f6bc015c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 -- 172.21.15.81:0/1030461683 >> 172.21.15.81:0/1030461683 conn(0x7f6bc00cb490 msgr2=0x7f6bc00b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:18.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 -- 172.21.15.81:0/1030461683 shutdown_connections 2023-11-08T16:31:18.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.418+0000 7f6bc7c32640 1 -- 172.21.15.81:0/1030461683 wait complete. 2023-11-08T16:31:18.438 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:31:18.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.638+0000 7f19845b6640 1 Processor -- start 2023-11-08T16:31:18.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.638+0000 7f19845b6640 1 -- start start 2023-11-08T16:31:18.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.640+0000 7f19845b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f197c161710 0x7f197c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.640+0000 7f19845b6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c06b980 con 0x7f197c15d2b0 2023-11-08T16:31:18.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.640+0000 7f19845b6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f197c06bc60 con 0x7f197c0ccd30 2023-11-08T16:31:18.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.640+0000 7f19845b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c0cd1c0 con 0x7f197c161710 2023-11-08T16:31:18.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.640+0000 7f1982352640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f197c15d2b0 0x7f197c0c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40562/0 (socket says 172.21.15.81:40562) 2023-11-08T16:31:18.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.640+0000 7f1982352640 1 -- 172.21.15.81:0/260502722 learned_addr learned my addr 172.21.15.81:0/260502722 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:18.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981b51640 1 --2- 172.21.15.81:0/260502722 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f197c161710 0x7f197c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857004629 0 0) 0x7f197c06b980 con 0x7f197c15d2b0 2023-11-08T16:31:18.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1968003520 con 0x7f197c15d2b0 2023-11-08T16:31:18.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3664303022 0 0) 0x7f1968003520 con 0x7f197c15d2b0 2023-11-08T16:31:18.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f197c161710 msgr2=0x7f197c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:18.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 --2- 172.21.15.81:0/260502722 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f197c161710 0x7f197c0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 >> v1:172.21.15.93:6789/0 conn(0x7f197c0ccd30 legacy=0x7f197c06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c06bf40 con 0x7f197c15d2b0 2023-11-08T16:31:18.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f1978001670 con 0x7f197c15d2b0 2023-11-08T16:31:18.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 >> v1:172.21.15.81:6789/0 conn(0x7f197c15d2b0 legacy=0x7f197c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 --2- 172.21.15.81:0/260502722 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1968003a80 0x7f1968003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 --2- 172.21.15.81:0/260502722 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f197c161710 0x7f19680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 --2- 172.21.15.81:0/260502722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1968004d50 0x7f19680051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c0cd1c0 con 0x7f1968004d50 2023-11-08T16:31:18.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196800bb40 con 0x7f1968003a80 2023-11-08T16:31:18.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.641+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196800bd00 con 0x7f197c161710 2023-11-08T16:31:18.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.642+0000 7f1982b53640 1 --2- 172.21.15.81:0/260502722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1968004d50 0x7f19680051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.642+0000 7f1981b51640 1 --2- 172.21.15.81:0/260502722 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f197c161710 0x7f19680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.642+0000 7f1982b53640 1 -- 172.21.15.81:0/260502722 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f197c161710 msgr2=0x7f19680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:18.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.642+0000 7f1982b53640 1 --2- 172.21.15.81:0/260502722 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f197c161710 0x7f19680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.642+0000 7f1982b53640 1 -- 172.21.15.81:0/260502722 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1968003a80 msgr2=0x7f1968003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:18.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.642+0000 7f1982b53640 1 --2- 172.21.15.81:0/260502722 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1968003a80 0x7f1968003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.642+0000 7f1982b53640 1 -- 172.21.15.81:0/260502722 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1968003520 con 0x7f1968004d50 2023-11-08T16:31:18.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.643+0000 7f1982b53640 1 --2- 172.21.15.81:0/260502722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1968004d50 0x7f19680051d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1974001e50 tx=0x7f19680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602c87c216592ef9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:18.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.643+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974003b70 con 0x7f1968004d50 2023-11-08T16:31:18.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.643+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1974003cd0 con 0x7f1968004d50 2023-11-08T16:31:18.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.643+0000 7f1981350640 1 -- 172.21.15.81:0/260502722 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1974005310 con 0x7f1968004d50 2023-11-08T16:31:18.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.644+0000 7f19845b6640 1 -- 172.21.15.81:0/260502722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1968004d50 msgr2=0x7f19680051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.644+0000 7f19845b6640 1 --2- 172.21.15.81:0/260502722 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1968004d50 0x7f19680051d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1974001e50 tx=0x7f19680057f0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.644+0000 7f1982b53640 1 -- 172.21.15.81:0/260502722 reap_dead start 2023-11-08T16:31:18.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.644+0000 7f19845b6640 1 -- 172.21.15.81:0/260502722 shutdown_connections 2023-11-08T16:31:18.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.644+0000 7f19845b6640 1 -- 172.21.15.81:0/260502722 >> 172.21.15.81:0/260502722 conn(0x7f197c0cb490 msgr2=0x7f197c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:18.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.644+0000 7f19845b6640 1 -- 172.21.15.81:0/260502722 shutdown_connections 2023-11-08T16:31:18.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.644+0000 7f19845b6640 1 -- 172.21.15.81:0/260502722 wait complete. 2023-11-08T16:31:18.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.646+0000 7f19845b6640 1 Processor -- start 2023-11-08T16:31:18.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.646+0000 7f19845b6640 1 -- start start 2023-11-08T16:31:18.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f19845b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1968004d50 0x7f197c0b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f19845b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f197c15bb90 0x7f197c0c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f19845b6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f197c15bff0 0x7f197c0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f19845b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c15a8a0 con 0x7f197c15bb90 2023-11-08T16:31:18.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f19845b6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c15aaf0 con 0x7f197c15bff0 2023-11-08T16:31:18.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f19845b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c15add0 con 0x7f1968004d50 2023-11-08T16:31:18.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f1981b51640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f197c15bb90 0x7f197c0c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f1981b51640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f197c15bb90 0x7f197c0c8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53082/0 (socket says 172.21.15.81:53082) 2023-11-08T16:31:18.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f1981b51640 1 -- 172.21.15.81:0/787319013 learned_addr learned my addr 172.21.15.81:0/787319013 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:18.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f1981b51640 1 -- 172.21.15.81:0/787319013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1968004d50 msgr2=0x7f197c0b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f1981b51640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1968004d50 0x7f197c0b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f1981b51640 1 -- 172.21.15.81:0/787319013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f197c15bff0 msgr2=0x7f197c0c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f1981b51640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f197c15bff0 0x7f197c0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.647+0000 7f1981b51640 1 -- 172.21.15.81:0/787319013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c15b0b0 con 0x7f197c15bb90 2023-11-08T16:31:18.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.648+0000 7f1981b51640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f197c15bb90 0x7f197c0c8500 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f19700115f0 tx=0x7f1970011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:18.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.648+0000 7f196effd640 1 -- 172.21.15.81:0/787319013 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970015870 con 0x7f197c15bb90 2023-11-08T16:31:18.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.648+0000 7f196effd640 1 -- 172.21.15.81:0/787319013 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19700159d0 con 0x7f197c15bb90 2023-11-08T16:31:18.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.648+0000 7f196effd640 1 -- 172.21.15.81:0/787319013 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970015d40 con 0x7f197c15bb90 2023-11-08T16:31:18.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.648+0000 7f19845b6640 1 -- 172.21.15.81:0/787319013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f197c15b390 con 0x7f197c15bb90 2023-11-08T16:31:18.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.648+0000 7f19845b6640 1 -- 172.21.15.81:0/787319013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197c0b4700 con 0x7f197c15bb90 2023-11-08T16:31:18.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.649+0000 7f19845b6640 1 -- 172.21.15.81:0/787319013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f197c04ece0 con 0x7f197c15bb90 2023-11-08T16:31:18.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.651+0000 7f196effd640 1 -- 172.21.15.81:0/787319013 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f197000f460 con 0x7f197c15bb90 2023-11-08T16:31:18.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.652+0000 7f196effd640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f196006ae70 0x7f196006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:18.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.652+0000 7f196effd640 1 -- 172.21.15.81:0/787319013 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f197008a0a0 con 0x7f197c15bb90 2023-11-08T16:31:18.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.652+0000 7f1982352640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f196006ae70 0x7f196006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:18.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.653+0000 7f1982352640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f196006ae70 0x7f196006d330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1978001eb0 tx=0x7f1978001500 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:18.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.656+0000 7f196effd640 1 -- 172.21.15.81:0/787319013 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1970054f00 con 0x7f197c15bb90 2023-11-08T16:31:18.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.888+0000 7f19845b6640 1 -- 172.21.15.81:0/787319013 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f197c0b4990 con 0x7f197c15bb90 2023-11-08T16:31:18.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.890+0000 7f196effd640 1 -- 172.21.15.81:0/787319013 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f1970058520 con 0x7f197c15bb90 2023-11-08T16:31:18.892 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:18.892 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/1453041994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":1453041994},{"type":"v1","addr":"172.21.15.93:6835","nonce":1453041994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/757624701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":757624701},{"type":"v1","addr":"172.21.15.93:6833","nonce":757624701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1735232404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1735232404},{"type":"v1","addr":"172.21.15.81:6835","nonce":1735232404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1759199271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1759199271},{"type":"v1","addr":"172.21.15.81:6837","nonce":1759199271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:16.832167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:31:18.892 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 25 2023-11-08T16:31:18.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 -- 172.21.15.81:0/787319013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f196006ae70 msgr2=0x7f196006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f196006ae70 0x7f196006d330 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1978001eb0 tx=0x7f1978001500 comp rx=0 tx=0).stop 2023-11-08T16:31:18.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 -- 172.21.15.81:0/787319013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f197c15bb90 msgr2=0x7f197c0c8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:18.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f197c15bb90 0x7f197c0c8500 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f19700115f0 tx=0x7f1970011ab0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 -- 172.21.15.81:0/787319013 shutdown_connections 2023-11-08T16:31:18.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f196006ae70 0x7f196006d330 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f197c15bff0 0x7f197c0c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f197c15bb90 0x7f197c0c8500 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 --2- 172.21.15.81:0/787319013 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1968004d50 0x7f197c0b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:18.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 -- 172.21.15.81:0/787319013 >> 172.21.15.81:0/787319013 conn(0x7f197c0cb490 msgr2=0x7f197c0cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:18.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 -- 172.21.15.81:0/787319013 shutdown_connections 2023-11-08T16:31:18.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:18.892+0000 7f19845b6640 1 -- 172.21.15.81:0/787319013 wait complete. 2023-11-08T16:31:18.912 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:31:19.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.118+0000 7fc66ffff640 1 Processor -- start 2023-11-08T16:31:19.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.118+0000 7fc66ffff640 1 -- start start 2023-11-08T16:31:19.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.120+0000 7fc66ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc6700cf170 0x7fc67015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.120+0000 7fc66ffff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc670070470 con 0x7fc67015aac0 2023-11-08T16:31:19.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.120+0000 7fc66ffff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc67006d580 con 0x7fc6700ced10 2023-11-08T16:31:19.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.120+0000 7fc66ffff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc67006f650 con 0x7fc6700cf170 2023-11-08T16:31:19.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.120+0000 7fc66f7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc6700cf170 0x7fc67015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.120+0000 7fc66effd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc67015aac0 0x7fc6700711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40566/0 (socket says 172.21.15.81:40566) 2023-11-08T16:31:19.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.120+0000 7fc66effd640 1 -- 172.21.15.81:0/4229946907 learned_addr learned my addr 172.21.15.81:0/4229946907 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:19.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.121+0000 7fc66f7fe640 1 -- 172.21.15.81:0/4229946907 >> v1:172.21.15.93:6789/0 conn(0x7fc6700ced10 legacy=0x7fc67015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:19.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.121+0000 7fc66f7fe640 1 -- 172.21.15.81:0/4229946907 >> v1:172.21.15.81:6789/0 conn(0x7fc67015aac0 legacy=0x7fc6700711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.121+0000 7fc66f7fe640 1 -- 172.21.15.81:0/4229946907 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc67006ca00 con 0x7fc6700cf170 2023-11-08T16:31:19.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.121+0000 7fc66dffb640 1 -- 172.21.15.81:0/4229946907 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332118830 0 0) 0x7fc670070470 con 0x7fc67015aac0 2023-11-08T16:31:19.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.122+0000 7fc66f7fe640 1 --2- 172.21.15.81:0/4229946907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6700cf170 0x7fc67015b5b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc668009d90 tx=0x7fc66802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d0b43d829050e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:19.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.122+0000 7fc66dffb640 1 -- 172.21.15.81:0/4229946907 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc66802cc60 con 0x7fc6700cf170 2023-11-08T16:31:19.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.122+0000 7fc66dffb640 1 -- 172.21.15.81:0/4229946907 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc66802cdc0 con 0x7fc6700cf170 2023-11-08T16:31:19.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.122+0000 7fc66dffb640 1 -- 172.21.15.81:0/4229946907 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668034660 con 0x7fc6700cf170 2023-11-08T16:31:19.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.122+0000 7fc66ffff640 1 -- 172.21.15.81:0/4229946907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6700cf170 msgr2=0x7fc67015b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.122+0000 7fc66ffff640 1 --2- 172.21.15.81:0/4229946907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6700cf170 0x7fc67015b5b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc668009d90 tx=0x7fc66802beb0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.123+0000 7fc66ffff640 1 -- 172.21.15.81:0/4229946907 shutdown_connections 2023-11-08T16:31:19.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.123+0000 7fc66ffff640 1 --2- 172.21.15.81:0/4229946907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6700cf170 0x7fc67015b5b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.123+0000 7fc66ffff640 1 -- 172.21.15.81:0/4229946907 >> 172.21.15.81:0/4229946907 conn(0x7fc67006a4a0 msgr2=0x7fc67006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:19.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.123+0000 7fc66ffff640 1 -- 172.21.15.81:0/4229946907 shutdown_connections 2023-11-08T16:31:19.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.123+0000 7fc66ffff640 1 -- 172.21.15.81:0/4229946907 wait complete. 2023-11-08T16:31:19.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.124+0000 7fc66ffff640 1 Processor -- start 2023-11-08T16:31:19.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.124+0000 7fc66ffff640 1 -- start start 2023-11-08T16:31:19.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.125+0000 7fc66ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6700ced10 0x7fc6701f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.125+0000 7fc66ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc6700cf170 0x7fc6700a86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.125+0000 7fc66ffff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc67015aac0 0x7fc67014ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.125+0000 7fc66ffff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6701f48a0 con 0x7fc6700ced10 2023-11-08T16:31:19.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.125+0000 7fc66ffff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6700a8bf0 con 0x7fc67015aac0 2023-11-08T16:31:19.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.125+0000 7fc66ffff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc67014f3b0 con 0x7fc6700cf170 2023-11-08T16:31:19.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.126+0000 7fc66effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6700ced10 0x7fc6701f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.126+0000 7fc66effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6700ced10 0x7fc6701f3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53106/0 (socket says 172.21.15.81:53106) 2023-11-08T16:31:19.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.126+0000 7fc66effd640 1 -- 172.21.15.81:0/1112963643 learned_addr learned my addr 172.21.15.81:0/1112963643 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:19.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.126+0000 7fc66effd640 1 -- 172.21.15.81:0/1112963643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc6700cf170 msgr2=0x7fc6700a86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.126+0000 7fc66effd640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc6700cf170 0x7fc6700a86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.126+0000 7fc66effd640 1 -- 172.21.15.81:0/1112963643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc67015aac0 msgr2=0x7fc67014ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.126+0000 7fc66effd640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc67015aac0 0x7fc67014ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.126+0000 7fc66effd640 1 -- 172.21.15.81:0/1112963643 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc668009a70 con 0x7fc6700ced10 2023-11-08T16:31:19.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.126+0000 7fc66effd640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6700ced10 0x7fc6701f3900 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c460 tx=0x7fc66000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:19.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.126+0000 7fc657fff640 1 -- 172.21.15.81:0/1112963643 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6600074e0 con 0x7fc6700ced10 2023-11-08T16:31:19.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.127+0000 7fc657fff640 1 -- 172.21.15.81:0/1112963643 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc660007660 con 0x7fc6700ced10 2023-11-08T16:31:19.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.127+0000 7fc66ffff640 1 -- 172.21.15.81:0/1112963643 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6701f4000 con 0x7fc6700ced10 2023-11-08T16:31:19.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.127+0000 7fc657fff640 1 -- 172.21.15.81:0/1112963643 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc660004a20 con 0x7fc6700ced10 2023-11-08T16:31:19.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.127+0000 7fc66ffff640 1 -- 172.21.15.81:0/1112963643 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6701f45c0 con 0x7fc6700ced10 2023-11-08T16:31:19.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.128+0000 7fc66ffff640 1 -- 172.21.15.81:0/1112963643 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc670067800 con 0x7fc6700ced10 2023-11-08T16:31:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.129+0000 7fc657fff640 1 -- 172.21.15.81:0/1112963643 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc660004b80 con 0x7fc6700ced10 2023-11-08T16:31:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.130+0000 7fc657fff640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc65c06ae70 0x7fc65c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.130+0000 7fc657fff640 1 -- 172.21.15.81:0/1112963643 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fc660088d80 con 0x7fc6700ced10 2023-11-08T16:31:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.130+0000 7fc66e7fc640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc65c06ae70 0x7fc65c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.131+0000 7fc66e7fc640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc65c06ae70 0x7fc65c06d330 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc670159960 tx=0x7fc6640079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.134+0000 7fc657fff640 1 -- 172.21.15.81:0/1112963643 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc660053b50 con 0x7fc6700ced10 2023-11-08T16:31:19.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.361+0000 7fc66ffff640 1 -- 172.21.15.81:0/1112963643 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc670159460 con 0x7fc6700ced10 2023-11-08T16:31:19.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.362+0000 7fc657fff640 1 -- 172.21.15.81:0/1112963643 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7fc660018020 con 0x7fc6700ced10 2023-11-08T16:31:19.364 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:19.364 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/1453041994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":1453041994},{"type":"v1","addr":"172.21.15.93:6835","nonce":1453041994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/757624701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":757624701},{"type":"v1","addr":"172.21.15.93:6833","nonce":757624701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1735232404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1735232404},{"type":"v1","addr":"172.21.15.81:6835","nonce":1735232404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1759199271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1759199271},{"type":"v1","addr":"172.21.15.81:6837","nonce":1759199271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:16.832167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:31:19.365 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 25 2023-11-08T16:31:19.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.364+0000 7fc66ffff640 1 -- 172.21.15.81:0/1112963643 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc65c06ae70 msgr2=0x7fc65c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.364+0000 7fc66ffff640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc65c06ae70 0x7fc65c06d330 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc670159960 tx=0x7fc6640079b0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.364+0000 7fc66ffff640 1 -- 172.21.15.81:0/1112963643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6700ced10 msgr2=0x7fc6701f3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.364+0000 7fc66ffff640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6700ced10 0x7fc6701f3900 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc66000c460 tx=0x7fc66000c920 comp rx=0 tx=0).stop 2023-11-08T16:31:19.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.365+0000 7fc66ffff640 1 -- 172.21.15.81:0/1112963643 shutdown_connections 2023-11-08T16:31:19.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.365+0000 7fc66ffff640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc65c06ae70 0x7fc65c06d330 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.365+0000 7fc66ffff640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc67015aac0 0x7fc67014ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.365+0000 7fc66ffff640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc6700cf170 0x7fc6700a86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.365+0000 7fc66ffff640 1 --2- 172.21.15.81:0/1112963643 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc6700ced10 0x7fc6701f3900 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.365+0000 7fc66ffff640 1 -- 172.21.15.81:0/1112963643 >> 172.21.15.81:0/1112963643 conn(0x7fc67006a4a0 msgr2=0x7fc67015ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:19.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.365+0000 7fc66ffff640 1 -- 172.21.15.81:0/1112963643 shutdown_connections 2023-11-08T16:31:19.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.365+0000 7fc66ffff640 1 -- 172.21.15.81:0/1112963643 wait complete. 2023-11-08T16:31:19.386 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:31:19.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.585+0000 7f1811724640 1 Processor -- start 2023-11-08T16:31:19.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.585+0000 7f1811724640 1 -- start start 2023-11-08T16:31:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f1811724640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f180c0ca480 0x7f180c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f1811724640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f180c15ea60 0x7f180c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f1811724640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c06b630 con 0x7f180c0ade50 2023-11-08T16:31:19.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f1811724640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180c0b1860 con 0x7f180c0ca480 2023-11-08T16:31:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f1811724640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0ca910 con 0x7f180c15ea60 2023-11-08T16:31:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f180a7fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f180c0ca480 0x7f180c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f180a7fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f180c0ca480 0x7f180c15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53110/0 (socket says 172.21.15.81:53110) 2023-11-08T16:31:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f180a7fc640 1 -- 172.21.15.81:0/3628729001 learned_addr learned my addr 172.21.15.81:0/3628729001 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:19.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f180affd640 1 --1- 172.21.15.81:0/3628729001 >> v1:172.21.15.81:6789/0 conn(0x7f180c0ade50 0x7f180c15a680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40568/0 (socket says 172.21.15.81:40568) 2023-11-08T16:31:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f180b7fe640 1 --2- 172.21.15.81:0/3628729001 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f180c15ea60 0x7f180c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f180a7fc640 1 -- 172.21.15.81:0/3628729001 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f180c15ea60 msgr2=0x7f180c15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f180a7fc640 1 --2- 172.21.15.81:0/3628729001 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f180c15ea60 0x7f180c15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f180a7fc640 1 -- 172.21.15.81:0/3628729001 >> v1:172.21.15.81:6789/0 conn(0x7f180c0ade50 legacy=0x7f180c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.587+0000 7f180a7fc640 1 -- 172.21.15.81:0/3628729001 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0cabb0 con 0x7f180c0ca480 2023-11-08T16:31:19.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.588+0000 7f180b7fe640 1 --2- 172.21.15.81:0/3628729001 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f180c15ea60 0x7f180c15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:19.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.589+0000 7f180a7fc640 1 --2- 172.21.15.81:0/3628729001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f180c0ca480 0x7f180c15ad90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1800009870 tx=0x7f180002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9987c582db90d3e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:19.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.589+0000 7f1809ffb640 1 -- 172.21.15.81:0/3628729001 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800030840 con 0x7f180c0ca480 2023-11-08T16:31:19.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.589+0000 7f1809ffb640 1 -- 172.21.15.81:0/3628729001 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18000309a0 con 0x7f180c0ca480 2023-11-08T16:31:19.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.589+0000 7f1809ffb640 1 -- 172.21.15.81:0/3628729001 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800030d30 con 0x7f180c0ca480 2023-11-08T16:31:19.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.589+0000 7f1811724640 1 -- 172.21.15.81:0/3628729001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f180c0ca480 msgr2=0x7f180c15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.589+0000 7f1811724640 1 --2- 172.21.15.81:0/3628729001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f180c0ca480 0x7f180c15ad90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1800009870 tx=0x7f180002b980 comp rx=0 tx=0).stop 2023-11-08T16:31:19.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.590+0000 7f1811724640 1 -- 172.21.15.81:0/3628729001 shutdown_connections 2023-11-08T16:31:19.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.590+0000 7f1811724640 1 --2- 172.21.15.81:0/3628729001 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f180c15ea60 0x7f180c15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.590+0000 7f1811724640 1 --2- 172.21.15.81:0/3628729001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f180c0ca480 0x7f180c15ad90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.590+0000 7f1811724640 1 -- 172.21.15.81:0/3628729001 >> 172.21.15.81:0/3628729001 conn(0x7f180c0cb490 msgr2=0x7f180c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:19.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.590+0000 7f1811724640 1 -- 172.21.15.81:0/3628729001 shutdown_connections 2023-11-08T16:31:19.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.590+0000 7f1811724640 1 -- 172.21.15.81:0/3628729001 wait complete. 2023-11-08T16:31:19.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.591+0000 7f1811724640 1 Processor -- start 2023-11-08T16:31:19.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.591+0000 7f1811724640 1 -- start start 2023-11-08T16:31:19.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.592+0000 7f1811724640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f180c0ade50 0x7f180c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.592+0000 7f1811724640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f180c0ca480 0x7f180c15a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.592+0000 7f1811724640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f180c15ea60 0x7f180c15a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.592+0000 7f1811724640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c153910 con 0x7f180c0ade50 2023-11-08T16:31:19.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.592+0000 7f1811724640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c153100 con 0x7f180c0ca480 2023-11-08T16:31:19.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.592+0000 7f1811724640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c153350 con 0x7f180c15ea60 2023-11-08T16:31:19.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.593+0000 7f180affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f180c0ade50 0x7f180c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.593+0000 7f180affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f180c0ade50 0x7f180c151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53112/0 (socket says 172.21.15.81:53112) 2023-11-08T16:31:19.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.593+0000 7f180affd640 1 -- 172.21.15.81:0/3634079392 learned_addr learned my addr 172.21.15.81:0/3634079392 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:19.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.593+0000 7f180affd640 1 -- 172.21.15.81:0/3634079392 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f180c15ea60 msgr2=0x7f180c15a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.593+0000 7f180affd640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f180c15ea60 0x7f180c15a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.593+0000 7f180affd640 1 -- 172.21.15.81:0/3634079392 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f180c0ca480 msgr2=0x7f180c15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.593+0000 7f180affd640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f180c0ca480 0x7f180c15a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.593+0000 7f180affd640 1 -- 172.21.15.81:0/3634079392 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18000094d0 con 0x7f180c0ade50 2023-11-08T16:31:19.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.593+0000 7f180affd640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f180c0ade50 0x7f180c151c20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00c4d0 tx=0x7f17fc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:19.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.594+0000 7f17f37fe640 1 -- 172.21.15.81:0/3634079392 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc007480 con 0x7f180c0ade50 2023-11-08T16:31:19.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.594+0000 7f17f37fe640 1 -- 172.21.15.81:0/3634079392 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc0075e0 con 0x7f180c0ade50 2023-11-08T16:31:19.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.594+0000 7f17f37fe640 1 -- 172.21.15.81:0/3634079392 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0049e0 con 0x7f180c0ade50 2023-11-08T16:31:19.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.594+0000 7f1811724640 1 -- 172.21.15.81:0/3634079392 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c153630 con 0x7f180c0ade50 2023-11-08T16:31:19.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.594+0000 7f1811724640 1 -- 172.21.15.81:0/3634079392 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c152160 con 0x7f180c0ade50 2023-11-08T16:31:19.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.595+0000 7f1811724640 1 -- 172.21.15.81:0/3634079392 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c157330 con 0x7f180c0ade50 2023-11-08T16:31:19.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.596+0000 7f17f37fe640 1 -- 172.21.15.81:0/3634079392 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17fc004b40 con 0x7f180c0ade50 2023-11-08T16:31:19.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.597+0000 7f17f37fe640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17f806aea0 0x7f17f806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.597+0000 7f17f37fe640 1 -- 172.21.15.81:0/3634079392 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f17fc088990 con 0x7f180c0ade50 2023-11-08T16:31:19.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.597+0000 7f180a7fc640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17f806aea0 0x7f17f806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.598+0000 7f180a7fc640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17f806aea0 0x7f17f806d360 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1800036000 tx=0x7f180003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:19.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.601+0000 7f17f37fe640 1 -- 172.21.15.81:0/3634079392 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f180c157330 con 0x7f180c0ade50 2023-11-08T16:31:19.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.829+0000 7f1811724640 1 -- 172.21.15.81:0/3634079392 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f180c157330 con 0x7f180c0ade50 2023-11-08T16:31:19.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.831+0000 7f17f37fe640 1 -- 172.21.15.81:0/3634079392 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4904 (secure 0 0 0) 0x7f180c157330 con 0x7f180c0ade50 2023-11-08T16:31:19.833 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:19.833 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/1453041994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":1453041994},{"type":"v1","addr":"172.21.15.93:6835","nonce":1453041994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/757624701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":757624701},{"type":"v1","addr":"172.21.15.93:6833","nonce":757624701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1735232404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1735232404},{"type":"v1","addr":"172.21.15.81:6835","nonce":1735232404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1759199271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1759199271},{"type":"v1","addr":"172.21.15.81:6837","nonce":1759199271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:16.832167+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:31:19.833 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 25 2023-11-08T16:31:19.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 -- 172.21.15.81:0/3634079392 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17f806aea0 msgr2=0x7f17f806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17f806aea0 0x7f17f806d360 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1800036000 tx=0x7f180003b040 comp rx=0 tx=0).stop 2023-11-08T16:31:19.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 -- 172.21.15.81:0/3634079392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f180c0ade50 msgr2=0x7f180c151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f180c0ade50 0x7f180c151c20 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00c4d0 tx=0x7f17fc00c990 comp rx=0 tx=0).stop 2023-11-08T16:31:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 -- 172.21.15.81:0/3634079392 shutdown_connections 2023-11-08T16:31:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17f806aea0 0x7f17f806d360 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f180c15ea60 0x7f180c15a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f180c0ca480 0x7f180c15a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 --2- 172.21.15.81:0/3634079392 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f180c0ade50 0x7f180c151c20 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 -- 172.21.15.81:0/3634079392 >> 172.21.15.81:0/3634079392 conn(0x7f180c0cb490 msgr2=0x7f180c15ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:19.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 -- 172.21.15.81:0/3634079392 shutdown_connections 2023-11-08T16:31:19.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.834+0000 7f1811724640 1 -- 172.21.15.81:0/3634079392 wait complete. 2023-11-08T16:31:19.858 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-11-08T16:31:19.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.947+0000 7f61dd60c100 1 -- start start 2023-11-08T16:31:19.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.947+0000 7f61dd60c100 1 Processor -- start 2023-11-08T16:31:19.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.948+0000 7f61dd60c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2330bac00 0x55e233126580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.948+0000 7f61dd60c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2330bb800 0x55e233126b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.948+0000 7f61dd60c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2330bbc00 0x55e233127080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.948+0000 7f61dd60c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e233105180 con 0x55e2330bb800 2023-11-08T16:31:19.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.948+0000 7f61dd60c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e233189180 con 0x55e2330bac00 2023-11-08T16:31:19.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.948+0000 7f61dd60c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e233188dc0 con 0x55e2330bbc00 2023-11-08T16:31:19.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.949+0000 7f61daa6e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2330bb800 0x55e233126b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.949+0000 7f61daa6e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2330bb800 0x55e233126b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53122/0 (socket says 172.21.15.81:53122) 2023-11-08T16:31:19.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.949+0000 7f61daa6e640 1 -- 172.21.15.81:0/1959550616 learned_addr learned my addr 172.21.15.81:0/1959550616 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:19.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.949+0000 7f61daa6e640 1 -- 172.21.15.81:0/1959550616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2330bbc00 msgr2=0x55e233127080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:19.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.949+0000 7f61daa6e640 1 --2- 172.21.15.81:0/1959550616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2330bbc00 0x55e233127080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.949+0000 7f61daa6e640 1 -- 172.21.15.81:0/1959550616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2330bac00 msgr2=0x55e233126580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:19.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.949+0000 7f61db26f640 1 --2- 172.21.15.81:0/1959550616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2330bac00 0x55e233126580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.949+0000 7f61daa6e640 1 --2- 172.21.15.81:0/1959550616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2330bac00 0x55e233126580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.949+0000 7f61daa6e640 1 -- 172.21.15.81:0/1959550616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2330e09c0 con 0x55e2330bb800 2023-11-08T16:31:19.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.949+0000 7f61daa6e640 1 --2- 172.21.15.81:0/1959550616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2330bb800 0x55e233126b00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55e233198d80 tx=0x55e23339e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ece189afe67a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:19.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.950+0000 7f61d8a6a640 1 -- 172.21.15.81:0/1959550616 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e2331031e0 con 0x55e2330bb800 2023-11-08T16:31:19.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.950+0000 7f61d8a6a640 1 -- 172.21.15.81:0/1959550616 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e233116180 con 0x55e2330bb800 2023-11-08T16:31:19.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.950+0000 7f61d8a6a640 1 -- 172.21.15.81:0/1959550616 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e233103080 con 0x55e2330bb800 2023-11-08T16:31:19.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.950+0000 7f61dd60c100 1 -- 172.21.15.81:0/1959550616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2330e0b60 con 0x55e2330bb800 2023-11-08T16:31:19.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.950+0000 7f61dd60c100 1 -- 172.21.15.81:0/1959550616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55e2330e0820 con 0x55e2330bb800 2023-11-08T16:31:19.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.950+0000 7f61dd60c100 4 waiting for MDS map... 2023-11-08T16:31:19.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.950+0000 7f61d8a6a640 1 -- 172.21.15.81:0/1959550616 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x55e2330e6380 con 0x55e2330bb800 2023-11-08T16:31:19.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.950+0000 7f61d8a6a640 1 -- 172.21.15.81:0/1959550616 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x55e23310cf00 con 0x55e2330bb800 2023-11-08T16:31:19.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.950+0000 7f61dd60c100 4 Got MDS map 25 2023-11-08T16:31:19.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.950+0000 7f61dd60c100 10 main: main 2023-11-08T16:31:19.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.950+0000 7f61dd60c100 4 main: connecting to RADOS... 2023-11-08T16:31:19.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.951+0000 7f61dd60c100 1 Processor -- start 2023-11-08T16:31:19.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.951+0000 7f61dd60c100 1 -- start start 2023-11-08T16:31:19.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.952+0000 7f61dd60c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2333ad000 0x55e233127600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.952+0000 7f61dd60c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad400 0x55e233127b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.952+0000 7f61dd60c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2333ad800 0x55e233128100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.952+0000 7f61dd60c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e233104140 con 0x55e2333ad400 2023-11-08T16:31:19.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.952+0000 7f61dd60c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e233104b40 con 0x55e2333ad800 2023-11-08T16:31:19.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.952+0000 7f61dd60c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e233189180 con 0x55e2333ad000 2023-11-08T16:31:19.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.952+0000 7f61db26f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad400 0x55e233127b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.952+0000 7f61db26f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad400 0x55e233127b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53134/0 (socket says 172.21.15.81:53134) 2023-11-08T16:31:19.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.952+0000 7f61db26f640 1 -- 172.21.15.81:0/2252592641 learned_addr learned my addr 172.21.15.81:0/2252592641 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:19.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.953+0000 7f61db26f640 1 -- 172.21.15.81:0/2252592641 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2333ad000 msgr2=0x55e233127600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:19.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.953+0000 7f61db26f640 1 --2- 172.21.15.81:0/2252592641 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2333ad000 0x55e233127600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.953+0000 7f61daa6e640 1 --2- 172.21.15.81:0/2252592641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2333ad800 0x55e233128100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.953+0000 7f61db26f640 1 -- 172.21.15.81:0/2252592641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2333ad800 msgr2=0x55e233128100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:19.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.953+0000 7f61db26f640 1 --2- 172.21.15.81:0/2252592641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2333ad800 0x55e233128100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.953+0000 7f61db26f640 1 -- 172.21.15.81:0/2252592641 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2330e04e0 con 0x55e2333ad400 2023-11-08T16:31:19.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.953+0000 7f61db26f640 1 --2- 172.21.15.81:0/2252592641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad400 0x55e233127b80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55e233198420 tx=0x55e23339e240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b159a0cea134ea4c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:19.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.953+0000 7f61d6a66640 1 -- 172.21.15.81:0/2252592641 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e233102f20 con 0x55e2333ad400 2023-11-08T16:31:19.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.953+0000 7f61d6a66640 1 -- 172.21.15.81:0/2252592641 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e233e8af00 con 0x55e2333ad400 2023-11-08T16:31:19.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.953+0000 7f61d6a66640 1 -- 172.21.15.81:0/2252592641 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e2331031e0 con 0x55e2333ad400 2023-11-08T16:31:19.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.954+0000 7f61dd60c100 1 -- 172.21.15.81:0/2252592641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad400 msgr2=0x55e233127b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.954+0000 7f61dd60c100 1 --2- 172.21.15.81:0/2252592641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad400 0x55e233127b80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x55e233198420 tx=0x55e23339e240 comp rx=0 tx=0).stop 2023-11-08T16:31:19.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.954+0000 7f61dd60c100 1 -- 172.21.15.81:0/2252592641 shutdown_connections 2023-11-08T16:31:19.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.954+0000 7f61dd60c100 1 --2- 172.21.15.81:0/2252592641 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2333ad800 0x55e233128100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.954+0000 7f61dd60c100 1 --2- 172.21.15.81:0/2252592641 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad400 0x55e233127b80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.954+0000 7f61dd60c100 1 --2- 172.21.15.81:0/2252592641 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2333ad000 0x55e233127600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.954+0000 7f61dd60c100 1 -- 172.21.15.81:0/2252592641 >> 172.21.15.81:0/2252592641 conn(0x55e2333acc00 msgr2=0x55e233e42800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:19.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.954+0000 7f61dd60c100 1 -- 172.21.15.81:0/2252592641 shutdown_connections 2023-11-08T16:31:19.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.954+0000 7f61dd60c100 1 -- 172.21.15.81:0/2252592641 wait complete. 2023-11-08T16:31:19.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.955+0000 7f61dd60c100 1 Processor -- start 2023-11-08T16:31:19.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.955+0000 7f61dd60c100 1 -- start start 2023-11-08T16:31:19.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.955+0000 7f61dd60c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad000 0x55e233129180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.955+0000 7f61dd60c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2333ad400 0x55e233128c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.955+0000 7f61dd60c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2333ad800 0x55e233128680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.955+0000 7f61dd60c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e233188c80 con 0x55e2333ad000 2023-11-08T16:31:19.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.955+0000 7f61dd60c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e233188b40 con 0x55e2333ad800 2023-11-08T16:31:19.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.955+0000 7f61dd60c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e233188a00 con 0x55e2333ad400 2023-11-08T16:31:19.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61dba70640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad000 0x55e233129180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61dba70640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad000 0x55e233129180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53142/0 (socket says 172.21.15.81:53142) 2023-11-08T16:31:19.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61dba70640 1 -- 172.21.15.81:0/2169280286 learned_addr learned my addr 172.21.15.81:0/2169280286 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:19.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61daa6e640 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2333ad800 0x55e233128680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61db26f640 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2333ad400 0x55e233128c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61dba70640 1 -- 172.21.15.81:0/2169280286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2333ad400 msgr2=0x55e233128c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61dba70640 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2333ad400 0x55e233128c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61dba70640 1 -- 172.21.15.81:0/2169280286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2333ad800 msgr2=0x55e233128680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61dba70640 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2333ad800 0x55e233128680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61dba70640 1 -- 172.21.15.81:0/2169280286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e2330e0d00 con 0x55e2333ad000 2023-11-08T16:31:19.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61db26f640 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2333ad400 0x55e233128c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:19.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.956+0000 7f61dba70640 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad000 0x55e233129180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x55e233e82f60 tx=0x55e2333c4d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:19.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.957+0000 7f61d5263640 1 -- 172.21.15.81:0/2169280286 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e233102f20 con 0x55e2333ad000 2023-11-08T16:31:19.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.957+0000 7f61d5263640 1 -- 172.21.15.81:0/2169280286 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e233116480 con 0x55e2333ad000 2023-11-08T16:31:19.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.957+0000 7f61d5263640 1 -- 172.21.15.81:0/2169280286 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e2331031e0 con 0x55e2333ad000 2023-11-08T16:31:19.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.957+0000 7f61dd60c100 1 -- 172.21.15.81:0/2169280286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e2330e04e0 con 0x55e2333ad000 2023-11-08T16:31:19.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.957+0000 7f61dd60c100 1 -- 172.21.15.81:0/2169280286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e2330e09c0 con 0x55e2333ad000 2023-11-08T16:31:19.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.957+0000 7f61dd60c100 4 main: resolving pool 4 2023-11-08T16:31:19.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.959+0000 7f61d5263640 1 -- 172.21.15.81:0/2169280286 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55e233118840 con 0x55e2333ad000 2023-11-08T16:31:19.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.959+0000 7f61d5263640 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55e233428800 0x55e233129700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.959+0000 7f61d5263640 1 -- 172.21.15.81:0/2169280286 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x55e2330e6540 con 0x55e2333ad000 2023-11-08T16:31:19.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.959+0000 7f61dd60c100 4 main: creating IoCtx.. 2023-11-08T16:31:19.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.959+0000 7f61dd60c100 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55e233428c00 0x55e233128100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:19.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.959+0000 7f61dd60c100 1 -- 172.21.15.81:0/2169280286 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:1 4.b 4:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55e233115880 con 0x55e233428c00 2023-11-08T16:31:19.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.959+0000 7f61db26f640 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55e233428800 0x55e233129700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.960+0000 7f61daa6e640 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55e233428c00 0x55e233128100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:19.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.960+0000 7f61db26f640 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55e233428800 0x55e233129700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55e233198420 tx=0x55e23339e120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:19.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.960+0000 7f61daa6e640 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55e233428c00 0x55e233128100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:19.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61daa6e640 1 -- 172.21.15.81:0/2169280286 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v40'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55e233f5eb40 con 0x55e233428c00 2023-11-08T16:31:19.978 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:31:19.978 INFO:teuthology.orchestra.run.smithi081.stdout: "0": { 2023-11-08T16:31:19.978 INFO:teuthology.orchestra.run.smithi081.stdout: "data": {}, 2023-11-08T16:31:19.979 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:31:19.979 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:31:19.979 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:31:19.979 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:19.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61dd60c100 1 -- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55e233428c00 msgr2=0x55e233128100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61dd60c100 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55e233428c00 0x55e233128100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61dd60c100 1 -- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55e233428800 msgr2=0x55e233129700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61dd60c100 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55e233428800 0x55e233129700 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x55e233198420 tx=0x55e23339e120 comp rx=0 tx=0).stop 2023-11-08T16:31:19.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61dd60c100 1 -- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad000 msgr2=0x55e233129180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61dd60c100 1 --2- 172.21.15.81:0/2169280286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2333ad000 0x55e233129180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x55e233e82f60 tx=0x55e2333c4d20 comp rx=0 tx=0).stop 2023-11-08T16:31:19.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61db26f640 1 -- 172.21.15.81:0/2169280286 reap_dead start 2023-11-08T16:31:19.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61dd60c100 1 -- 172.21.15.81:0/2169280286 shutdown_connections 2023-11-08T16:31:19.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61dd60c100 1 -- 172.21.15.81:0/2169280286 >> 172.21.15.81:0/2169280286 conn(0x55e2333acc00 msgr2=0x55e233e42800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:19.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61dd60c100 1 -- 172.21.15.81:0/2169280286 shutdown_connections 2023-11-08T16:31:19.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.962+0000 7f61dd60c100 1 -- 172.21.15.81:0/2169280286 wait complete. 2023-11-08T16:31:19.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.963+0000 7f61dd60c100 1 -- 172.21.15.81:0/1959550616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2330bb800 msgr2=0x55e233126b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:19.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.963+0000 7f61dd60c100 1 --2- 172.21.15.81:0/1959550616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2330bb800 0x55e233126b00 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55e233198d80 tx=0x55e23339e420 comp rx=0 tx=0).stop 2023-11-08T16:31:19.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.963+0000 7f61dd60c100 1 -- 172.21.15.81:0/1959550616 shutdown_connections 2023-11-08T16:31:19.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.963+0000 7f61dd60c100 1 --2- 172.21.15.81:0/1959550616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e2330bbc00 0x55e233127080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.963+0000 7f61dd60c100 1 --2- 172.21.15.81:0/1959550616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e2330bb800 0x55e233126b00 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.963+0000 7f61dd60c100 1 --2- 172.21.15.81:0/1959550616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e2330bac00 0x55e233126580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:19.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.963+0000 7f61dd60c100 1 -- 172.21.15.81:0/1959550616 >> 172.21.15.81:0/1959550616 conn(0x55e2330bb000 msgr2=0x55e233e43000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:19.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.963+0000 7f61dd60c100 1 -- 172.21.15.81:0/1959550616 shutdown_connections 2023-11-08T16:31:19.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:19.963+0000 7f61dd60c100 1 -- 172.21.15.81:0/1959550616 wait complete. 2023-11-08T16:31:19.986 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.127461, result: { "0": { "data": {}, "result": 0 } } 2023-11-08T16:31:19.986 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-11-08T16:31:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.077+0000 7fe28b83b100 1 -- start start 2023-11-08T16:31:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.078+0000 7fe28b83b100 1 Processor -- start 2023-11-08T16:31:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.078+0000 7fe28b83b100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562183534c00 0x5621835a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.078+0000 7fe28b83b100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562183535800 0x5621835a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.078+0000 7fe28b83b100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562183535c00 0x5621835a1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe28b83b100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56218357f180 con 0x562183535c00 2023-11-08T16:31:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe28b83b100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56218357e140 con 0x562183535800 2023-11-08T16:31:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe28b83b100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56218357e000 con 0x562183534c00 2023-11-08T16:31:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe28a088640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562183535c00 0x5621835a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe28a088640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562183535c00 0x5621835a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53166/0 (socket says 172.21.15.81:53166) 2023-11-08T16:31:20.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe28a088640 1 -- 172.21.15.81:0/3392931923 learned_addr learned my addr 172.21.15.81:0/3392931923 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe289086640 1 --2- 172.21.15.81:0/3392931923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562183535800 0x5621835a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe289887640 1 --2- 172.21.15.81:0/3392931923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562183534c00 0x5621835a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe28a088640 1 -- 172.21.15.81:0/3392931923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562183534c00 msgr2=0x5621835a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe28a088640 1 --2- 172.21.15.81:0/3392931923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562183534c00 0x5621835a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe28a088640 1 -- 172.21.15.81:0/3392931923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562183535800 msgr2=0x5621835a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe28a088640 1 --2- 172.21.15.81:0/3392931923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562183535800 0x5621835a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.079+0000 7fe28a088640 1 -- 172.21.15.81:0/3392931923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56218355a9c0 con 0x562183535c00 2023-11-08T16:31:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.080+0000 7fe28a088640 1 --2- 172.21.15.81:0/3392931923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562183535c00 0x5621835a1080 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x56218435abd0 tx=0x5621843ead20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8a7c5eb1acf171 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.080+0000 7fe287082640 1 -- 172.21.15.81:0/3392931923 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56218357d1e0 con 0x562183535c00 2023-11-08T16:31:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.080+0000 7fe287082640 1 -- 172.21.15.81:0/3392931923 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562183590480 con 0x562183535c00 2023-11-08T16:31:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.080+0000 7fe287082640 1 -- 172.21.15.81:0/3392931923 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56218357d080 con 0x562183535c00 2023-11-08T16:31:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.080+0000 7fe28b83b100 1 -- 172.21.15.81:0/3392931923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56218355ab60 con 0x562183535c00 2023-11-08T16:31:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.080+0000 7fe28b83b100 1 -- 172.21.15.81:0/3392931923 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56218355a820 con 0x562183535c00 2023-11-08T16:31:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.080+0000 7fe28b83b100 4 waiting for MDS map... 2023-11-08T16:31:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.080+0000 7fe287082640 1 -- 172.21.15.81:0/3392931923 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x562183560380 con 0x562183535c00 2023-11-08T16:31:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.080+0000 7fe287082640 1 -- 172.21.15.81:0/3392931923 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x562183586f00 con 0x562183535c00 2023-11-08T16:31:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.080+0000 7fe28b83b100 4 Got MDS map 25 2023-11-08T16:31:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.080+0000 7fe28b83b100 10 main: main 2023-11-08T16:31:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.081+0000 7fe28b83b100 4 main: connecting to RADOS... 2023-11-08T16:31:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.081+0000 7fe28b83b100 1 Processor -- start 2023-11-08T16:31:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.081+0000 7fe28b83b100 1 -- start start 2023-11-08T16:31:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.082+0000 7fe28b83b100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5621843f7000 0x5621835a1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.082+0000 7fe28b83b100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7400 0x5621835a1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.082+0000 7fe28b83b100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5621843f7800 0x5621835a2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.082+0000 7fe28b83b100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56218357ea00 con 0x5621843f7400 2023-11-08T16:31:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.082+0000 7fe28b83b100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562183603180 con 0x5621843f7800 2023-11-08T16:31:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.082+0000 7fe28b83b100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562183602dc0 con 0x5621843f7000 2023-11-08T16:31:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.082+0000 7fe289887640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7400 0x5621835a1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.082+0000 7fe289887640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7400 0x5621835a1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53180/0 (socket says 172.21.15.81:53180) 2023-11-08T16:31:20.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.082+0000 7fe289887640 1 -- 172.21.15.81:0/3382268320 learned_addr learned my addr 172.21.15.81:0/3382268320 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe289086640 1 --2- 172.21.15.81:0/3382268320 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5621843f7800 0x5621835a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe289086640 1 --2- 172.21.15.81:0/3382268320 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5621843f7000 0x5621835a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe289887640 1 -- 172.21.15.81:0/3382268320 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5621843f7000 msgr2=0x5621835a1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe289887640 1 --2- 172.21.15.81:0/3382268320 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5621843f7000 0x5621835a1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe289887640 1 -- 172.21.15.81:0/3382268320 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5621843f7800 msgr2=0x5621835a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe289887640 1 --2- 172.21.15.81:0/3382268320 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5621843f7800 0x5621835a2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe289887640 1 -- 172.21.15.81:0/3382268320 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56218355a4e0 con 0x5621843f7400 2023-11-08T16:31:20.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe289086640 1 --2- 172.21.15.81:0/3382268320 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5621843f7800 0x5621835a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:20.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe289887640 1 --2- 172.21.15.81:0/3382268320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7400 0x5621835a1b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x56218445e090 tx=0x56218440cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c1416ba259425e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe28507e640 1 -- 172.21.15.81:0/3382268320 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56218357d1e0 con 0x5621843f7400 2023-11-08T16:31:20.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe28507e640 1 -- 172.21.15.81:0/3382268320 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56218430cc00 con 0x5621843f7400 2023-11-08T16:31:20.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.083+0000 7fe28507e640 1 -- 172.21.15.81:0/3382268320 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56218357d080 con 0x5621843f7400 2023-11-08T16:31:20.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.084+0000 7fe28b83b100 1 -- 172.21.15.81:0/3382268320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7400 msgr2=0x5621835a1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.084+0000 7fe28b83b100 1 --2- 172.21.15.81:0/3382268320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7400 0x5621835a1b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x56218445e090 tx=0x56218440cc00 comp rx=0 tx=0).stop 2023-11-08T16:31:20.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.084+0000 7fe28b83b100 1 -- 172.21.15.81:0/3382268320 shutdown_connections 2023-11-08T16:31:20.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.084+0000 7fe28b83b100 1 --2- 172.21.15.81:0/3382268320 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5621843f7800 0x5621835a2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.084+0000 7fe28b83b100 1 --2- 172.21.15.81:0/3382268320 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7400 0x5621835a1b80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.084+0000 7fe28b83b100 1 --2- 172.21.15.81:0/3382268320 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5621843f7000 0x5621835a1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.084+0000 7fe28b83b100 1 -- 172.21.15.81:0/3382268320 >> 172.21.15.81:0/3382268320 conn(0x5621843f6c00 msgr2=0x5621842af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.084+0000 7fe28b83b100 1 -- 172.21.15.81:0/3382268320 shutdown_connections 2023-11-08T16:31:20.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.084+0000 7fe28b83b100 1 -- 172.21.15.81:0/3382268320 wait complete. 2023-11-08T16:31:20.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.085+0000 7fe28b83b100 1 Processor -- start 2023-11-08T16:31:20.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.085+0000 7fe28b83b100 1 -- start start 2023-11-08T16:31:20.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe28b83b100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7000 0x5621835a1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe28b83b100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5621843f7800 0x5621835a2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe28b83b100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5621843f7c00 0x5621835a2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe28b83b100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562183602c80 con 0x5621843f7000 2023-11-08T16:31:20.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe28b83b100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56218357e140 con 0x5621843f7800 2023-11-08T16:31:20.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe28b83b100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56218357eb40 con 0x5621843f7c00 2023-11-08T16:31:20.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe289086640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7000 0x5621835a1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe289086640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7000 0x5621835a1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53182/0 (socket says 172.21.15.81:53182) 2023-11-08T16:31:20.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe289086640 1 -- 172.21.15.81:0/1275537845 learned_addr learned my addr 172.21.15.81:0/1275537845 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe289086640 1 -- 172.21.15.81:0/1275537845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5621843f7c00 msgr2=0x5621835a2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe289086640 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5621843f7c00 0x5621835a2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe289086640 1 -- 172.21.15.81:0/1275537845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5621843f7800 msgr2=0x5621835a2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe289887640 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5621843f7800 0x5621835a2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe289086640 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5621843f7800 0x5621835a2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.086+0000 7fe289086640 1 -- 172.21.15.81:0/1275537845 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56218355a9c0 con 0x5621843f7000 2023-11-08T16:31:20.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.087+0000 7fe289086640 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7000 0x5621835a1600 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x56218445ed20 tx=0x5621842c57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.087+0000 7fe28387b640 1 -- 172.21.15.81:0/1275537845 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56218357cf20 con 0x5621843f7000 2023-11-08T16:31:20.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.087+0000 7fe28387b640 1 -- 172.21.15.81:0/1275537845 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562183590180 con 0x5621843f7000 2023-11-08T16:31:20.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.087+0000 7fe28b83b100 1 -- 172.21.15.81:0/1275537845 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56218355ab60 con 0x5621843f7000 2023-11-08T16:31:20.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.087+0000 7fe28387b640 1 -- 172.21.15.81:0/1275537845 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56218357d1e0 con 0x5621843f7000 2023-11-08T16:31:20.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.087+0000 7fe28b83b100 1 -- 172.21.15.81:0/1275537845 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56218355a820 con 0x5621843f7000 2023-11-08T16:31:20.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.087+0000 7fe28b83b100 4 main: resolving pool 4 2023-11-08T16:31:20.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.088+0000 7fe28387b640 1 -- 172.21.15.81:0/1275537845 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x562183593080 con 0x5621843f7000 2023-11-08T16:31:20.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.089+0000 7fe28387b640 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56218388a000 0x5621835a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.089+0000 7fe28387b640 1 -- 172.21.15.81:0/1275537845 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x562183560540 con 0x5621843f7000 2023-11-08T16:31:20.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.089+0000 7fe28b83b100 4 main: creating IoCtx.. 2023-11-08T16:31:20.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.089+0000 7fe28b83b100 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5621843f7400 0x5621835a2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.089+0000 7fe28b83b100 1 -- 172.21.15.81:0/1275537845 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:1 4.d 4:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x56218358f880 con 0x5621843f7400 2023-11-08T16:31:20.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.089+0000 7fe289887640 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56218388a000 0x5621835a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.090+0000 7fe289887640 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56218388a000 0x5621835a3700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5621835c45d0 tx=0x5621842c4120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.090+0000 7fe289086640 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5621843f7400 0x5621835a2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.091+0000 7fe289086640 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5621843f7400 0x5621835a2c00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe289086640 1 -- 172.21.15.81:0/1275537845 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v40'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x56218428cb40 con 0x5621843f7400 2023-11-08T16:31:20.110 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:31:20.110 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:31:20.111 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:31:20.111 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:20.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe28b83b100 1 -- 172.21.15.81:0/1275537845 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5621843f7400 msgr2=0x5621835a2c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe28b83b100 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5621843f7400 0x5621835a2c00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe28b83b100 1 -- 172.21.15.81:0/1275537845 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56218388a000 msgr2=0x5621835a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe28b83b100 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56218388a000 0x5621835a3700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x5621835c45d0 tx=0x5621842c4120 comp rx=0 tx=0).stop 2023-11-08T16:31:20.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe28b83b100 1 -- 172.21.15.81:0/1275537845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7000 msgr2=0x5621835a1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe28b83b100 1 --2- 172.21.15.81:0/1275537845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5621843f7000 0x5621835a1600 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x56218445ed20 tx=0x5621842c57a0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe289887640 1 -- 172.21.15.81:0/1275537845 reap_dead start 2023-11-08T16:31:20.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe28b83b100 1 -- 172.21.15.81:0/1275537845 shutdown_connections 2023-11-08T16:31:20.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe28b83b100 1 -- 172.21.15.81:0/1275537845 >> 172.21.15.81:0/1275537845 conn(0x5621843f6c00 msgr2=0x5621842af000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe28b83b100 1 -- 172.21.15.81:0/1275537845 shutdown_connections 2023-11-08T16:31:20.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.094+0000 7fe28b83b100 1 -- 172.21.15.81:0/1275537845 wait complete. 2023-11-08T16:31:20.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.095+0000 7fe28b83b100 1 -- 172.21.15.81:0/3392931923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562183535c00 msgr2=0x5621835a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.095+0000 7fe28b83b100 1 --2- 172.21.15.81:0/3392931923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562183535c00 0x5621835a1080 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x56218435abd0 tx=0x5621843ead20 comp rx=0 tx=0).stop 2023-11-08T16:31:20.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.095+0000 7fe28b83b100 1 -- 172.21.15.81:0/3392931923 shutdown_connections 2023-11-08T16:31:20.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.095+0000 7fe28b83b100 1 --2- 172.21.15.81:0/3392931923 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562183535c00 0x5621835a1080 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.095+0000 7fe28b83b100 1 --2- 172.21.15.81:0/3392931923 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562183535800 0x5621835a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.095+0000 7fe28b83b100 1 --2- 172.21.15.81:0/3392931923 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562183534c00 0x5621835a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.095+0000 7fe28b83b100 1 -- 172.21.15.81:0/3392931923 >> 172.21.15.81:0/3392931923 conn(0x562183535000 msgr2=0x5621842ae800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.095+0000 7fe28b83b100 1 -- 172.21.15.81:0/3392931923 shutdown_connections 2023-11-08T16:31:20.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.095+0000 7fe28b83b100 1 -- 172.21.15.81:0/3392931923 wait complete. 2023-11-08T16:31:20.118 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.131295, result: { "result": 0 } 2023-11-08T16:31:20.118 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-11-08T16:31:20.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.214+0000 7fa7e70b4100 1 -- start start 2023-11-08T16:31:20.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.214+0000 7fa7e70b4100 1 Processor -- start 2023-11-08T16:31:20.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e70b4100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa8d74c00 0x557fa8de0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e70b4100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa8d75800 0x557fa8de0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e70b4100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557fa8d75c00 0x557fa8de1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e70b4100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fa8dbf180 con 0x557fa8d75800 2023-11-08T16:31:20.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e70b4100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fa8dbe140 con 0x557fa8d74c00 2023-11-08T16:31:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e70b4100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fa8dbe000 con 0x557fa8d75c00 2023-11-08T16:31:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e5059640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa8d75800 0x557fa8de0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e5059640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa8d75800 0x557fa8de0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53202/0 (socket says 172.21.15.81:53202) 2023-11-08T16:31:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e5059640 1 -- 172.21.15.81:0/2494238201 learned_addr learned my addr 172.21.15.81:0/2494238201 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e5059640 1 -- 172.21.15.81:0/2494238201 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557fa8d75c00 msgr2=0x557fa8de1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e5059640 1 --2- 172.21.15.81:0/2494238201 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557fa8d75c00 0x557fa8de1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e5059640 1 -- 172.21.15.81:0/2494238201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa8d74c00 msgr2=0x557fa8de0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e585a640 1 --2- 172.21.15.81:0/2494238201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa8d74c00 0x557fa8de0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e5059640 1 --2- 172.21.15.81:0/2494238201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa8d74c00 0x557fa8de0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.215+0000 7fa7e5059640 1 -- 172.21.15.81:0/2494238201 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fa8d9a9c0 con 0x557fa8d75800 2023-11-08T16:31:20.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.216+0000 7fa7e5059640 1 --2- 172.21.15.81:0/2494238201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa8d75800 0x557fa8de0b00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x557fa8e52c90 tx=0x557fa9052180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6aba28921de144 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.216+0000 7fa7e3055640 1 -- 172.21.15.81:0/2494238201 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557fa8dbd1e0 con 0x557fa8d75800 2023-11-08T16:31:20.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.216+0000 7fa7e3055640 1 -- 172.21.15.81:0/2494238201 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557fa8dd0480 con 0x557fa8d75800 2023-11-08T16:31:20.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.216+0000 7fa7e3055640 1 -- 172.21.15.81:0/2494238201 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557fa8dbd080 con 0x557fa8d75800 2023-11-08T16:31:20.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.216+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2494238201 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fa8d9ab60 con 0x557fa8d75800 2023-11-08T16:31:20.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.216+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2494238201 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557fa8d9a820 con 0x557fa8d75800 2023-11-08T16:31:20.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.216+0000 7fa7e70b4100 4 waiting for MDS map... 2023-11-08T16:31:20.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.216+0000 7fa7e3055640 1 -- 172.21.15.81:0/2494238201 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x557fa8da0380 con 0x557fa8d75800 2023-11-08T16:31:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.217+0000 7fa7e3055640 1 -- 172.21.15.81:0/2494238201 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x557fa8dc6f00 con 0x557fa8d75800 2023-11-08T16:31:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.217+0000 7fa7e70b4100 4 Got MDS map 25 2023-11-08T16:31:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.217+0000 7fa7e70b4100 10 main: main 2023-11-08T16:31:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.217+0000 7fa7e70b4100 4 main: connecting to RADOS... 2023-11-08T16:31:20.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.217+0000 7fa7e70b4100 1 Processor -- start 2023-11-08T16:31:20.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.218+0000 7fa7e70b4100 1 -- start start 2023-11-08T16:31:20.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.218+0000 7fa7e70b4100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 0x557fa8de1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.218+0000 7fa7e70b4100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa9061400 0x557fa8de1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.218+0000 7fa7e70b4100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557fa9061800 0x557fa8de2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.218+0000 7fa7e70b4100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fa8dbea00 con 0x557fa9061000 2023-11-08T16:31:20.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.218+0000 7fa7e70b4100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fa8e43180 con 0x557fa9061400 2023-11-08T16:31:20.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.218+0000 7fa7e70b4100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fa8e42dc0 con 0x557fa9061800 2023-11-08T16:31:20.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.219+0000 7fa7e605b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 0x557fa8de1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.219+0000 7fa7e605b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 0x557fa8de1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53214/0 (socket says 172.21.15.81:53214) 2023-11-08T16:31:20.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.219+0000 7fa7e605b640 1 -- 172.21.15.81:0/2643443028 learned_addr learned my addr 172.21.15.81:0/2643443028 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.219+0000 7fa7e605b640 1 -- 172.21.15.81:0/2643443028 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557fa9061800 msgr2=0x557fa8de2100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T16:31:20.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.219+0000 7fa7e605b640 1 --2- 172.21.15.81:0/2643443028 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557fa9061800 0x557fa8de2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.219+0000 7fa7e605b640 1 -- 172.21.15.81:0/2643443028 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa9061400 msgr2=0x557fa8de1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.219+0000 7fa7e605b640 1 --2- 172.21.15.81:0/2643443028 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa9061400 0x557fa8de1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.219+0000 7fa7e605b640 1 -- 172.21.15.81:0/2643443028 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fa8d9a4e0 con 0x557fa9061000 2023-11-08T16:31:20.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.219+0000 7fa7e605b640 1 --2- 172.21.15.81:0/2643443028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 0x557fa8de1600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x557fa90cc1b0 tx=0x557fa9056d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1323dabee1dc24 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.219+0000 7fa7e1051640 1 -- 172.21.15.81:0/2643443028 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557fa8dbcf20 con 0x557fa9061000 2023-11-08T16:31:20.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.220+0000 7fa7e1051640 1 -- 172.21.15.81:0/2643443028 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557fa9b44c00 con 0x557fa9061000 2023-11-08T16:31:20.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.220+0000 7fa7e1051640 1 -- 172.21.15.81:0/2643443028 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557fa8dbd1e0 con 0x557fa9061000 2023-11-08T16:31:20.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.220+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2643443028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 msgr2=0x557fa8de1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.220+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2643443028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 0x557fa8de1600 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x557fa90cc1b0 tx=0x557fa9056d20 comp rx=0 tx=0).stop 2023-11-08T16:31:20.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.220+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2643443028 shutdown_connections 2023-11-08T16:31:20.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.220+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2643443028 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557fa9061800 0x557fa8de2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.220+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2643443028 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa9061400 0x557fa8de1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.220+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2643443028 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 0x557fa8de1600 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.220+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2643443028 >> 172.21.15.81:0/2643443028 conn(0x557fa9060c00 msgr2=0x557fa9aff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.220+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2643443028 shutdown_connections 2023-11-08T16:31:20.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.220+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2643443028 wait complete. 2023-11-08T16:31:20.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.221+0000 7fa7e70b4100 1 Processor -- start 2023-11-08T16:31:20.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.221+0000 7fa7e70b4100 1 -- start start 2023-11-08T16:31:20.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e70b4100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 0x557fa8de3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e70b4100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557fa9061400 0x557fa8de2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e70b4100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa9061800 0x557fa8de2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e70b4100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fa8e42c80 con 0x557fa9061000 2023-11-08T16:31:20.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e70b4100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fa8dbe140 con 0x557fa9061800 2023-11-08T16:31:20.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e70b4100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fa8dbeb40 con 0x557fa9061400 2023-11-08T16:31:20.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e605b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 0x557fa8de3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e605b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 0x557fa8de3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53230/0 (socket says 172.21.15.81:53230) 2023-11-08T16:31:20.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e605b640 1 -- 172.21.15.81:0/2883044703 learned_addr learned my addr 172.21.15.81:0/2883044703 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e605b640 1 -- 172.21.15.81:0/2883044703 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557fa9061400 msgr2=0x557fa8de2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e605b640 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557fa9061400 0x557fa8de2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e605b640 1 -- 172.21.15.81:0/2883044703 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa9061800 msgr2=0x557fa8de2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e605b640 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa9061800 0x557fa8de2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e605b640 1 -- 172.21.15.81:0/2883044703 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fa8d9ad00 con 0x557fa9061000 2023-11-08T16:31:20.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.222+0000 7fa7e605b640 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 0x557fa8de3180 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x557fa90cc4e0 tx=0x557fa9056c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.223+0000 7fa7df84e640 1 -- 172.21.15.81:0/2883044703 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557fa8dbd080 con 0x557fa9061000 2023-11-08T16:31:20.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.223+0000 7fa7df84e640 1 -- 172.21.15.81:0/2883044703 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557fa9b45500 con 0x557fa9061000 2023-11-08T16:31:20.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.223+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2883044703 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557fa8d9a4e0 con 0x557fa9061000 2023-11-08T16:31:20.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.223+0000 7fa7df84e640 1 -- 172.21.15.81:0/2883044703 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557fa8dbc420 con 0x557fa9061000 2023-11-08T16:31:20.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.223+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2883044703 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fa8d9a9c0 con 0x557fa9061000 2023-11-08T16:31:20.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.223+0000 7fa7e70b4100 4 main: resolving pool 4 2023-11-08T16:31:20.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.225+0000 7fa7df84e640 1 -- 172.21.15.81:0/2883044703 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x557fa8dd3080 con 0x557fa9061000 2023-11-08T16:31:20.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.225+0000 7fa7df84e640 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x557fa9061c00 0x557fa8de3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.225+0000 7fa7df84e640 1 -- 172.21.15.81:0/2883044703 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x557fa8da0540 con 0x557fa9061000 2023-11-08T16:31:20.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.225+0000 7fa7e70b4100 4 main: creating IoCtx.. 2023-11-08T16:31:20.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.226+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x557fa91d0000 0x557fa8de2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.226+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2883044703 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:1 4.13 4:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x557fa8dcf880 con 0x557fa91d0000 2023-11-08T16:31:20.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.226+0000 7fa7e585a640 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x557fa9061c00 0x557fa8de3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.226+0000 7fa7e585a640 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x557fa9061c00 0x557fa8de3700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x557fa9ace0c0 tx=0x557fa9057e00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.226+0000 7fa7e5059640 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x557fa91d0000 0x557fa8de2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.227+0000 7fa7e5059640 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x557fa91d0000 0x557fa8de2100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.231+0000 7fa7e5059640 1 -- 172.21.15.81:0/2883044703 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v40'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x557fa9c14b40 con 0x557fa91d0000 2023-11-08T16:31:20.244 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:31:20.244 INFO:teuthology.orchestra.run.smithi081.stdout: "0": { 2023-11-08T16:31:20.244 INFO:teuthology.orchestra.run.smithi081.stdout: "data": {}, 2023-11-08T16:31:20.244 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:31:20.245 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:31:20.245 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:31:20.245 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:20.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.231+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2883044703 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x557fa91d0000 msgr2=0x557fa8de2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.231+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x557fa91d0000 0x557fa8de2100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2883044703 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x557fa9061c00 msgr2=0x557fa8de3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x557fa9061c00 0x557fa8de3700 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x557fa9ace0c0 tx=0x557fa9057e00 comp rx=0 tx=0).stop 2023-11-08T16:31:20.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2883044703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 msgr2=0x557fa8de3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2883044703 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa9061000 0x557fa8de3180 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x557fa90cc4e0 tx=0x557fa9056c00 comp rx=0 tx=0).stop 2023-11-08T16:31:20.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e585a640 1 -- 172.21.15.81:0/2883044703 reap_dead start 2023-11-08T16:31:20.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2883044703 shutdown_connections 2023-11-08T16:31:20.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2883044703 >> 172.21.15.81:0/2883044703 conn(0x557fa9060c00 msgr2=0x557fa9aff000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2883044703 shutdown_connections 2023-11-08T16:31:20.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2883044703 wait complete. 2023-11-08T16:31:20.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2494238201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa8d75800 msgr2=0x557fa8de0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2494238201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa8d75800 0x557fa8de0b00 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x557fa8e52c90 tx=0x557fa9052180 comp rx=0 tx=0).stop 2023-11-08T16:31:20.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2494238201 shutdown_connections 2023-11-08T16:31:20.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2494238201 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557fa8d75c00 0x557fa8de1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2494238201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557fa8d75800 0x557fa8de0b00 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 --2- 172.21.15.81:0/2494238201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557fa8d74c00 0x557fa8de0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2494238201 >> 172.21.15.81:0/2494238201 conn(0x557fa8d75000 msgr2=0x557fa9afe800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2494238201 shutdown_connections 2023-11-08T16:31:20.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.232+0000 7fa7e70b4100 1 -- 172.21.15.81:0/2494238201 wait complete. 2023-11-08T16:31:20.252 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.133874, result: { "0": { "data": {}, "result": 0 } } 2023-11-08T16:31:20.252 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-11-08T16:31:20.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.353+0000 7fc118045100 1 -- start start 2023-11-08T16:31:20.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.353+0000 7fc118045100 1 Processor -- start 2023-11-08T16:31:20.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.354+0000 7fc118045100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea3c2c00 0x5581ea42e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.354+0000 7fc118045100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea3c3800 0x5581ea42eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.354+0000 7fc118045100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581ea3c3c00 0x5581ea42f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.354+0000 7fc118045100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ea40d180 con 0x5581ea3c2c00 2023-11-08T16:31:20.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.354+0000 7fc118045100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ea40c140 con 0x5581ea3c3c00 2023-11-08T16:31:20.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.354+0000 7fc118045100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581ea40c000 con 0x5581ea3c3800 2023-11-08T16:31:20.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.354+0000 7fc1160b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea3c2c00 0x5581ea42e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.355+0000 7fc1160b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea3c2c00 0x5581ea42e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53250/0 (socket says 172.21.15.81:53250) 2023-11-08T16:31:20.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.355+0000 7fc1160b5640 1 -- 172.21.15.81:0/3148840905 learned_addr learned my addr 172.21.15.81:0/3148840905 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.355+0000 7fc1158b4640 1 --2- 172.21.15.81:0/3148840905 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea3c3800 0x5581ea42eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.355+0000 7fc1168b6640 1 --2- 172.21.15.81:0/3148840905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581ea3c3c00 0x5581ea42f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.355+0000 7fc1160b5640 1 -- 172.21.15.81:0/3148840905 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea3c3800 msgr2=0x5581ea42eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.355+0000 7fc1160b5640 1 --2- 172.21.15.81:0/3148840905 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea3c3800 0x5581ea42eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.355+0000 7fc1160b5640 1 -- 172.21.15.81:0/3148840905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581ea3c3c00 msgr2=0x5581ea42f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.355+0000 7fc1160b5640 1 --2- 172.21.15.81:0/3148840905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581ea3c3c00 0x5581ea42f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.355+0000 7fc1160b5640 1 -- 172.21.15.81:0/3148840905 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581ea3e89c0 con 0x5581ea3c2c00 2023-11-08T16:31:20.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.355+0000 7fc1160b5640 1 --2- 172.21.15.81:0/3148840905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea3c2c00 0x5581ea42e580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5581eb1cca50 tx=0x5581eb1520c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf73b9abaf79d35b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.355+0000 7fc1138b0640 1 -- 172.21.15.81:0/3148840905 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5581ea40af20 con 0x5581ea3c2c00 2023-11-08T16:31:20.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.356+0000 7fc1138b0640 1 -- 172.21.15.81:0/3148840905 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581eb194180 con 0x5581ea3c2c00 2023-11-08T16:31:20.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.356+0000 7fc1138b0640 1 -- 172.21.15.81:0/3148840905 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5581ea40b1e0 con 0x5581ea3c2c00 2023-11-08T16:31:20.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.356+0000 7fc118045100 1 -- 172.21.15.81:0/3148840905 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581ea3e8d00 con 0x5581ea3c2c00 2023-11-08T16:31:20.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.356+0000 7fc118045100 1 -- 172.21.15.81:0/3148840905 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5581ea3e89c0 con 0x5581ea3c2c00 2023-11-08T16:31:20.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.356+0000 7fc118045100 4 waiting for MDS map... 2023-11-08T16:31:20.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.356+0000 7fc1138b0640 1 -- 172.21.15.81:0/3148840905 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x5581ea3ee380 con 0x5581ea3c2c00 2023-11-08T16:31:20.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.356+0000 7fc1138b0640 1 -- 172.21.15.81:0/3148840905 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x5581ea415200 con 0x5581ea3c2c00 2023-11-08T16:31:20.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.356+0000 7fc118045100 4 Got MDS map 25 2023-11-08T16:31:20.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.356+0000 7fc118045100 10 main: JournalTool::main 2023-11-08T16:31:20.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.356+0000 7fc118045100 4 main: JournalTool: connecting to RADOS... 2023-11-08T16:31:20.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.357+0000 7fc118045100 1 Processor -- start 2023-11-08T16:31:20.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.357+0000 7fc118045100 1 -- start start 2023-11-08T16:31:20.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.358+0000 7fc118045100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581eb19dc00 0x5581ea42f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.358+0000 7fc118045100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710000 0x5581ea42fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.358+0000 7fc118045100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea710400 0x5581ea430100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.358+0000 7fc118045100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ea40ca00 con 0x5581ea710000 2023-11-08T16:31:20.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.358+0000 7fc118045100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ea491180 con 0x5581eb19dc00 2023-11-08T16:31:20.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.358+0000 7fc118045100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581ea490dc0 con 0x5581ea710400 2023-11-08T16:31:20.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.358+0000 7fc1160b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710000 0x5581ea42fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.358+0000 7fc1160b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710000 0x5581ea42fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53262/0 (socket says 172.21.15.81:53262) 2023-11-08T16:31:20.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.358+0000 7fc1160b5640 1 -- 172.21.15.81:0/3212981613 learned_addr learned my addr 172.21.15.81:0/3212981613 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc1160b5640 1 -- 172.21.15.81:0/3212981613 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea710400 msgr2=0x5581ea430100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc1160b5640 1 --2- 172.21.15.81:0/3212981613 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea710400 0x5581ea430100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc1160b5640 1 -- 172.21.15.81:0/3212981613 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581eb19dc00 msgr2=0x5581ea42f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc1160b5640 1 --2- 172.21.15.81:0/3212981613 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581eb19dc00 0x5581ea42f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc1160b5640 1 -- 172.21.15.81:0/3212981613 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581ea3e8b60 con 0x5581ea710000 2023-11-08T16:31:20.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc1160b5640 1 --2- 172.21.15.81:0/3212981613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710000 0x5581ea42fb80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x5581eb1cd4d0 tx=0x5581eb145ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f418cb7d3f4fd74c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc1118ac640 1 -- 172.21.15.81:0/3212981613 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5581ea40b080 con 0x5581ea710000 2023-11-08T16:31:20.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc1118ac640 1 -- 172.21.15.81:0/3212981613 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581eb194c00 con 0x5581ea710000 2023-11-08T16:31:20.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc1118ac640 1 -- 172.21.15.81:0/3212981613 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5581ea40adc0 con 0x5581ea710000 2023-11-08T16:31:20.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc118045100 1 -- 172.21.15.81:0/3212981613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710000 msgr2=0x5581ea42fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc118045100 1 --2- 172.21.15.81:0/3212981613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710000 0x5581ea42fb80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x5581eb1cd4d0 tx=0x5581eb145ce0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.359+0000 7fc118045100 1 -- 172.21.15.81:0/3212981613 shutdown_connections 2023-11-08T16:31:20.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.360+0000 7fc118045100 1 --2- 172.21.15.81:0/3212981613 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea710400 0x5581ea430100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.360+0000 7fc118045100 1 --2- 172.21.15.81:0/3212981613 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710000 0x5581ea42fb80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.360+0000 7fc118045100 1 --2- 172.21.15.81:0/3212981613 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581eb19dc00 0x5581ea42f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.360+0000 7fc118045100 1 -- 172.21.15.81:0/3212981613 >> 172.21.15.81:0/3212981613 conn(0x5581eb19d800 msgr2=0x5581eb14d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.360+0000 7fc118045100 1 -- 172.21.15.81:0/3212981613 shutdown_connections 2023-11-08T16:31:20.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.360+0000 7fc118045100 1 -- 172.21.15.81:0/3212981613 wait complete. 2023-11-08T16:31:20.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.360+0000 7fc118045100 1 Processor -- start 2023-11-08T16:31:20.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.360+0000 7fc118045100 1 -- start start 2023-11-08T16:31:20.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.361+0000 7fc118045100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581eb19d800 0x5581ea431180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.361+0000 7fc118045100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea711000 0x5581ea430c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.361+0000 7fc118045100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710c00 0x5581ea430680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.361+0000 7fc118045100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ea40c140 con 0x5581ea710c00 2023-11-08T16:31:20.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.361+0000 7fc118045100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ea40d180 con 0x5581eb19d800 2023-11-08T16:31:20.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.361+0000 7fc118045100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581ea40cb40 con 0x5581ea711000 2023-11-08T16:31:20.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.361+0000 7fc1158b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710c00 0x5581ea430680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.361+0000 7fc1158b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710c00 0x5581ea430680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53264/0 (socket says 172.21.15.81:53264) 2023-11-08T16:31:20.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.362+0000 7fc1158b4640 1 -- 172.21.15.81:0/2769887997 learned_addr learned my addr 172.21.15.81:0/2769887997 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.362+0000 7fc1168b6640 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581eb19d800 0x5581ea431180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.362+0000 7fc1160b5640 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea711000 0x5581ea430c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.362+0000 7fc1158b4640 1 -- 172.21.15.81:0/2769887997 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea711000 msgr2=0x5581ea430c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.362+0000 7fc1158b4640 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea711000 0x5581ea430c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.362+0000 7fc1158b4640 1 -- 172.21.15.81:0/2769887997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581eb19d800 msgr2=0x5581ea431180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.362+0000 7fc1158b4640 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581eb19d800 0x5581ea431180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.362+0000 7fc1158b4640 1 -- 172.21.15.81:0/2769887997 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581ea3e8d00 con 0x5581ea710c00 2023-11-08T16:31:20.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.362+0000 7fc1168b6640 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581eb19d800 0x5581ea431180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:20.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.362+0000 7fc1158b4640 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710c00 0x5581ea430680 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x5581ea6bd470 tx=0x5581ea6c0cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.362+0000 7fc1100a9640 1 -- 172.21.15.81:0/2769887997 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5581ea40b080 con 0x5581ea710c00 2023-11-08T16:31:20.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.363+0000 7fc1100a9640 1 -- 172.21.15.81:0/2769887997 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581eb194480 con 0x5581ea710c00 2023-11-08T16:31:20.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.363+0000 7fc1100a9640 1 -- 172.21.15.81:0/2769887997 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5581ea40adc0 con 0x5581ea710c00 2023-11-08T16:31:20.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.363+0000 7fc118045100 1 -- 172.21.15.81:0/2769887997 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5581ea3e89c0 con 0x5581ea710c00 2023-11-08T16:31:20.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.363+0000 7fc118045100 1 -- 172.21.15.81:0/2769887997 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581ea3e8b60 con 0x5581ea710c00 2023-11-08T16:31:20.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.363+0000 7fc118045100 4 main: JournalTool: resolving pool 4 2023-11-08T16:31:20.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.364+0000 7fc1100a9640 1 -- 172.21.15.81:0/2769887997 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x5581ea452840 con 0x5581ea710c00 2023-11-08T16:31:20.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.365+0000 7fc1100a9640 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5581ea728c00 0x5581ea431700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.365+0000 7fc1100a9640 1 -- 172.21.15.81:0/2769887997 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x5581ea3ee540 con 0x5581ea710c00 2023-11-08T16:31:20.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.365+0000 7fc118045100 4 main: JournalTool: creating IoCtx.. 2023-11-08T16:31:20.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.365+0000 7fc118045100 4 main: Executing for rank 0 2023-11-08T16:31:20.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.365+0000 7fc1168b6640 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5581ea728c00 0x5581ea431700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.366+0000 7fc1168b6640 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5581ea728c00 0x5581ea431700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x5581ea4b67e0 tx=0x5581ea825800 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.367+0000 7fc118045100 1 -- start start 2023-11-08T16:31:20.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.367+0000 7fc118045100 1 Processor -- start 2023-11-08T16:31:20.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc118045100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710400 0x5581ea42f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc118045100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea710000 0x5581ea430100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc118045100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581ea729c00 0x5581ea42fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc118045100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ea490c80 con 0x5581ea710400 2023-11-08T16:31:20.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc118045100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5581ea490b40 con 0x5581ea729c00 2023-11-08T16:31:20.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc118045100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5581ea490a00 con 0x5581ea710000 2023-11-08T16:31:20.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc1168b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710400 0x5581ea42f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc1168b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710400 0x5581ea42f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53274/0 (socket says 172.21.15.81:53274) 2023-11-08T16:31:20.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc1168b6640 1 -- 172.21.15.81:0/323248398 learned_addr learned my addr 172.21.15.81:0/323248398 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc1168b6640 1 -- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea710000 msgr2=0x5581ea430100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc1168b6640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea710000 0x5581ea430100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc1168b6640 1 -- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581ea729c00 msgr2=0x5581ea42fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc1168b6640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581ea729c00 0x5581ea42fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.368+0000 7fc1168b6640 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5581ea754ea0 con 0x5581ea710400 2023-11-08T16:31:20.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.369+0000 7fc1168b6640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710400 0x5581ea42f600 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x5581ea7c7290 tx=0x5581eb152180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0a84a308ed8252 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.369+0000 7fc10d0a3640 1 -- 172.21.15.81:0/323248398 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5581ea40a420 con 0x5581ea710400 2023-11-08T16:31:20.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.369+0000 7fc10d0a3640 1 -- 172.21.15.81:0/323248398 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5581eb195b00 con 0x5581ea710400 2023-11-08T16:31:20.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.369+0000 7fc10d0a3640 1 -- 172.21.15.81:0/323248398 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5581ea40a2c0 con 0x5581ea710400 2023-11-08T16:31:20.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.369+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5581ea754d00 con 0x5581ea710400 2023-11-08T16:31:20.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.369+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5581ea754b60 con 0x5581ea710400 2023-11-08T16:31:20.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.369+0000 7fc118045100 4 waiting for MDS map... 2023-11-08T16:31:20.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.369+0000 7fc10d0a3640 1 -- 172.21.15.81:0/323248398 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x5581ea3eea80 con 0x5581ea710400 2023-11-08T16:31:20.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.369+0000 7fc10d0a3640 1 -- 172.21.15.81:0/323248398 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x5581ea86d200 con 0x5581ea710400 2023-11-08T16:31:20.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.369+0000 7fc118045100 4 Got MDS map 25 2023-11-08T16:31:20.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.370+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5581ea888c00 0x5581ea89a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.370+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:1 4.f 4:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5581ea89e000 con 0x5581ea888c00 2023-11-08T16:31:20.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.370+0000 7fc1160b5640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5581ea888c00 0x5581ea89a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.370+0000 7fc1160b5640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5581ea888c00 0x5581ea89a000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=64d663bddb047cf4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.372+0000 7fc1160b5640 1 -- 172.21.15.81:0/323248398 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v40'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5581ea4a2b40 con 0x5581ea888c00 2023-11-08T16:31:20.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.372+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5581ea889000 0x5581ea89a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.372+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:2 4.14 4:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5581ea89e380 con 0x5581ea889000 2023-11-08T16:31:20.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.372+0000 7fc1158b4640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5581ea889000 0x5581ea89a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.373+0000 7fc1158b4640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5581ea889000 0x5581ea89a580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=e301aa33f8978a05 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc1158b4640 1 -- 172.21.15.81:0/323248398 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v40'40 uv40 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5581ea4a2d80 con 0x5581ea889000 2023-11-08T16:31:20.396 INFO:teuthology.orchestra.run.smithi081.stdout:writing EResetJournal entry 2023-11-08T16:31:20.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 4.14 4:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5581ea89e700 con 0x5581ea889000 2023-11-08T16:31:20.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5581ea889400 0x5581ea89ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:4 4.1b 4:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5581ea89ea80 con 0x5581ea889400 2023-11-08T16:31:20.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5581ea889800 0x5581ea89b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 4.b 4:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5581ea89ee00 con 0x5581ea889800 2023-11-08T16:31:20.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5581ea889c00 0x5581ea89b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:6 4.1f 4:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5581ea89f180 con 0x5581ea889c00 2023-11-08T16:31:20.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:7 4.1a 4:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5581ea89f500 con 0x5581ea889400 2023-11-08T16:31:20.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:8 4.1e 4:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5581ea89e380 con 0x5581ea889800 2023-11-08T16:31:20.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc1160b5640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5581ea889800 0x5581ea89b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc1168b6640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5581ea889400 0x5581ea89ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.375+0000 7fc1158b4640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5581ea889c00 0x5581ea89b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.376+0000 7fc1160b5640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5581ea889800 0x5581ea89b080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=df310b371cee19f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.376+0000 7fc1168b6640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5581ea889400 0x5581ea89ab00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=2956e367697c601f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.376+0000 7fc1158b4640 1 -- 172.21.15.81:0/323248398 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(3 200.00000001 [delete] v40'41 uv40 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5581ea4a2d80 con 0x5581ea889000 2023-11-08T16:31:20.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.377+0000 7fc1158b4640 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5581ea889c00 0x5581ea89b600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=75696b52e2663fac server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.377+0000 7fc1160b5640 1 -- 172.21.15.81:0/323248398 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(5 200.00000003 [delete] v40'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5581ea4a2b40 con 0x5581ea889800 2023-11-08T16:31:20.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.379+0000 7fc1160b5640 1 -- 172.21.15.81:0/323248398 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(8 200.00000006 [delete] v40'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5581ea4a2b40 con 0x5581ea889800 2023-11-08T16:31:20.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.379+0000 7fc1168b6640 1 -- 172.21.15.81:0/323248398 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(4 200.00000002 [delete] v40'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5581ea4a38c0 con 0x5581ea889400 2023-11-08T16:31:20.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.379+0000 7fc10b8a0640 1 -- 172.21.15.81:0/323248398 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 4.14 4:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5581ea89e000 con 0x5581ea889000 2023-11-08T16:31:20.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.380+0000 7fc1158b4640 1 -- 172.21.15.81:0/323248398 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(6 200.00000004 [delete] v40'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5581ea4a2d80 con 0x5581ea889c00 2023-11-08T16:31:20.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.382+0000 7fc1168b6640 1 -- 172.21.15.81:0/323248398 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(7 200.00000005 [delete] v40'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5581ea4a38c0 con 0x5581ea889400 2023-11-08T16:31:20.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc1158b4640 1 -- 172.21.15.81:0/323248398 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v40'42 uv42 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5581ea4a2d80 con 0x5581ea889000 2023-11-08T16:31:20.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc118045100 4 Successfully wrote new journal pointer and header for rank 2:0 2023-11-08T16:31:20.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5581ea889800 msgr2=0x5581ea89b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5581ea889800 0x5581ea89b080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5581ea888c00 msgr2=0x5581ea89a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5581ea888c00 0x5581ea89a000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5581ea889000 msgr2=0x5581ea89a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5581ea889000 0x5581ea89a580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5581ea889400 msgr2=0x5581ea89ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5581ea889400 0x5581ea89ab00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5581ea889c00 msgr2=0x5581ea89b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.383+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5581ea889c00 0x5581ea89b600 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc1160b5640 1 -- 172.21.15.81:0/323248398 reap_dead start 2023-11-08T16:31:20.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710400 msgr2=0x5581ea42f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710400 0x5581ea42f600 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x5581ea7c7290 tx=0x5581eb152180 comp rx=0 tx=0).stop 2023-11-08T16:31:20.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 shutdown_connections 2023-11-08T16:31:20.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 --2- 172.21.15.81:0/323248398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710400 0x5581ea42f600 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 >> 172.21.15.81:0/323248398 conn(0x5581ea710800 msgr2=0x5581eb14d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 shutdown_connections 2023-11-08T16:31:20.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 -- 172.21.15.81:0/323248398 wait complete. 2023-11-08T16:31:20.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 -- 172.21.15.81:0/2769887997 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5581ea728c00 msgr2=0x5581ea431700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5581ea728c00 0x5581ea431700 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x5581ea4b67e0 tx=0x5581ea825800 comp rx=0 tx=0).stop 2023-11-08T16:31:20.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 -- 172.21.15.81:0/2769887997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710c00 msgr2=0x5581ea430680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710c00 0x5581ea430680 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x5581ea6bd470 tx=0x5581ea6c0cc0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 -- 172.21.15.81:0/2769887997 shutdown_connections 2023-11-08T16:31:20.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5581ea728c00 0x5581ea431700 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea710c00 0x5581ea430680 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea711000 0x5581ea430c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 --2- 172.21.15.81:0/2769887997 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581eb19d800 0x5581ea431180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.384+0000 7fc118045100 1 -- 172.21.15.81:0/2769887997 >> 172.21.15.81:0/2769887997 conn(0x5581eb19dc00 msgr2=0x5581eb14d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.385+0000 7fc118045100 1 -- 172.21.15.81:0/2769887997 shutdown_connections 2023-11-08T16:31:20.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.385+0000 7fc118045100 1 -- 172.21.15.81:0/2769887997 wait complete. 2023-11-08T16:31:20.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.385+0000 7fc118045100 1 -- 172.21.15.81:0/3148840905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea3c2c00 msgr2=0x5581ea42e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.385+0000 7fc118045100 1 --2- 172.21.15.81:0/3148840905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea3c2c00 0x5581ea42e580 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x5581eb1cca50 tx=0x5581eb1520c0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.385+0000 7fc118045100 1 -- 172.21.15.81:0/3148840905 shutdown_connections 2023-11-08T16:31:20.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.385+0000 7fc118045100 1 --2- 172.21.15.81:0/3148840905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5581ea3c3c00 0x5581ea42f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.385+0000 7fc118045100 1 --2- 172.21.15.81:0/3148840905 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5581ea3c3800 0x5581ea42eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.385+0000 7fc118045100 1 --2- 172.21.15.81:0/3148840905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5581ea3c2c00 0x5581ea42e580 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.385+0000 7fc118045100 1 -- 172.21.15.81:0/3148840905 >> 172.21.15.81:0/3148840905 conn(0x5581ea3c3000 msgr2=0x5581eb14c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.385+0000 7fc118045100 1 -- 172.21.15.81:0/3148840905 shutdown_connections 2023-11-08T16:31:20.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.385+0000 7fc118045100 1 -- 172.21.15.81:0/3148840905 wait complete. 2023-11-08T16:31:20.414 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.161794, result: writing EResetJournal entry 2023-11-08T16:31:20.415 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-11-08T16:31:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.513+0000 7f4de863a5c0 1 -- start start 2023-11-08T16:31:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.513+0000 7f4de863a5c0 1 Processor -- start 2023-11-08T16:31:20.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.514+0000 7f4de863a5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1705f5c00 0x55d17065e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.514+0000 7f4de863a5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1713c8800 0x55d17065eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.514+0000 7f4de863a5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1713c9c00 0x55d17065fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.514+0000 7f4de863a5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17063d180 con 0x55d1713c8800 2023-11-08T16:31:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.514+0000 7f4de863a5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17063cb40 con 0x55d1705f5c00 2023-11-08T16:31:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.514+0000 7f4de863a5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d17063c140 con 0x55d1713c9c00 2023-11-08T16:31:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.515+0000 7f4de5e67640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1713c8800 0x55d17065eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.515+0000 7f4de5e67640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1713c8800 0x55d17065eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53290/0 (socket says 172.21.15.81:53290) 2023-11-08T16:31:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.515+0000 7f4de5e67640 1 -- 172.21.15.81:0/1175640436 learned_addr learned my addr 172.21.15.81:0/1175640436 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.515+0000 7f4de6668640 1 --2- 172.21.15.81:0/1175640436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1705f5c00 0x55d17065e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.515+0000 7f4de6e69640 1 --2- 172.21.15.81:0/1175640436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1713c9c00 0x55d17065fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.515+0000 7f4de5e67640 1 -- 172.21.15.81:0/1175640436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1713c9c00 msgr2=0x55d17065fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.515+0000 7f4de5e67640 1 --2- 172.21.15.81:0/1175640436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1713c9c00 0x55d17065fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.515+0000 7f4de5e67640 1 -- 172.21.15.81:0/1175640436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1705f5c00 msgr2=0x55d17065e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.515+0000 7f4de5e67640 1 --2- 172.21.15.81:0/1175640436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1705f5c00 0x55d17065e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.515+0000 7f4de5e67640 1 -- 172.21.15.81:0/1175640436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d17061a4e0 con 0x55d1713c8800 2023-11-08T16:31:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.515+0000 7f4de5e67640 1 --2- 172.21.15.81:0/1175640436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1713c8800 0x55d17065eb00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x55d17140b290 tx=0x55d1713ca480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ccc4dd3bee4159a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.516+0000 7f4de3e63640 1 -- 172.21.15.81:0/1175640436 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d17063af20 con 0x55d1713c8800 2023-11-08T16:31:20.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.516+0000 7f4de3e63640 1 -- 172.21.15.81:0/1175640436 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d1714c6000 con 0x55d1713c8800 2023-11-08T16:31:20.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.516+0000 7f4de3e63640 1 -- 172.21.15.81:0/1175640436 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d17063b1e0 con 0x55d1713c8800 2023-11-08T16:31:20.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.516+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1175640436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d17061a1a0 con 0x55d1713c8800 2023-11-08T16:31:20.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.516+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1175640436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55d1714c8000 con 0x55d1713c8800 2023-11-08T16:31:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.516+0000 7f4de863a5c0 4 waiting for MDS map... 2023-11-08T16:31:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.516+0000 7f4de3e63640 1 -- 172.21.15.81:0/1175640436 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x55d1706b4c40 con 0x55d1713c8800 2023-11-08T16:31:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.516+0000 7f4de3e63640 1 -- 172.21.15.81:0/1175640436 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x55d170644f00 con 0x55d1713c8800 2023-11-08T16:31:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.516+0000 7f4de863a5c0 4 Got MDS map 25 2023-11-08T16:31:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.516+0000 7f4de863a5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:31:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.516+0000 7f4de863a5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:31:20.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.517+0000 7f4de863a5c0 1 Processor -- start 2023-11-08T16:31:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.517+0000 7f4de863a5c0 1 -- start start 2023-11-08T16:31:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.518+0000 7f4de863a5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1713c9800 0x55d17065f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.518+0000 7f4de863a5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1714c4c00 0x55d17065f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.518+0000 7f4de863a5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5000 0x55d170660100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.518+0000 7f4de863a5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17063ca00 con 0x55d1714c5000 2023-11-08T16:31:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.518+0000 7f4de863a5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17063c000 con 0x55d1713c9800 2023-11-08T16:31:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.518+0000 7f4de863a5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d1706c3180 con 0x55d1714c4c00 2023-11-08T16:31:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.518+0000 7f4de5e67640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5000 0x55d170660100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.518+0000 7f4de5e67640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5000 0x55d170660100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53296/0 (socket says 172.21.15.81:53296) 2023-11-08T16:31:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.518+0000 7f4de5e67640 1 -- 172.21.15.81:0/1875260291 learned_addr learned my addr 172.21.15.81:0/1875260291 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.519+0000 7f4de5e67640 1 -- 172.21.15.81:0/1875260291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1714c4c00 msgr2=0x55d17065f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.519+0000 7f4de5e67640 1 --2- 172.21.15.81:0/1875260291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1714c4c00 0x55d17065f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.519+0000 7f4de5e67640 1 -- 172.21.15.81:0/1875260291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1713c9800 msgr2=0x55d17065f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.519+0000 7f4de5e67640 1 --2- 172.21.15.81:0/1875260291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1713c9800 0x55d17065f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.519+0000 7f4de5e67640 1 -- 172.21.15.81:0/1875260291 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1714c81a0 con 0x55d1714c5000 2023-11-08T16:31:20.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.519+0000 7f4de5e67640 1 --2- 172.21.15.81:0/1875260291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5000 0x55d170660100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55d1714e05d0 tx=0x55d1713ca1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc42137abb2bb6e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.519+0000 7f4de1e5f640 1 -- 172.21.15.81:0/1875260291 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d17063af20 con 0x55d1714c5000 2023-11-08T16:31:20.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.519+0000 7f4de1e5f640 1 -- 172.21.15.81:0/1875260291 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d1714c6c00 con 0x55d1714c5000 2023-11-08T16:31:20.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.519+0000 7f4de1e5f640 1 -- 172.21.15.81:0/1875260291 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d17063b1e0 con 0x55d1714c5000 2023-11-08T16:31:20.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.520+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1875260291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5000 msgr2=0x55d170660100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.520+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/1875260291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5000 0x55d170660100 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x55d1714e05d0 tx=0x55d1713ca1e0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.520+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1875260291 shutdown_connections 2023-11-08T16:31:20.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.520+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/1875260291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5000 0x55d170660100 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.520+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/1875260291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1714c4c00 0x55d17065f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.520+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/1875260291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1713c9800 0x55d17065f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.520+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1875260291 >> 172.21.15.81:0/1875260291 conn(0x55d1713c9400 msgr2=0x55d171370800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.520+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1875260291 shutdown_connections 2023-11-08T16:31:20.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.520+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1875260291 wait complete. 2023-11-08T16:31:20.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.521+0000 7f4de863a5c0 1 Processor -- start 2023-11-08T16:31:20.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.521+0000 7f4de863a5c0 1 -- start start 2023-11-08T16:31:20.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de863a5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1713c9800 0x55d17065f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de863a5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1714c4c00 0x55d17065f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de863a5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5400 0x55d170660680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de863a5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1706c2dc0 con 0x55d1714c5400 2023-11-08T16:31:20.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de863a5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1706c2c80 con 0x55d1714c4c00 2023-11-08T16:31:20.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de863a5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d17063cb40 con 0x55d1713c9800 2023-11-08T16:31:20.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de5e67640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5400 0x55d170660680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de5e67640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5400 0x55d170660680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53306/0 (socket says 172.21.15.81:53306) 2023-11-08T16:31:20.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de5e67640 1 -- 172.21.15.81:0/2343698184 learned_addr learned my addr 172.21.15.81:0/2343698184 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de6668640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1714c4c00 0x55d17065f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de6e69640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1713c9800 0x55d17065f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de5e67640 1 -- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1713c9800 msgr2=0x55d17065f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de5e67640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1713c9800 0x55d17065f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de5e67640 1 -- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1714c4c00 msgr2=0x55d17065f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de5e67640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1714c4c00 0x55d17065f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.522+0000 7f4de5e67640 1 -- 172.21.15.81:0/2343698184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1714c8340 con 0x55d1714c5400 2023-11-08T16:31:20.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.523+0000 7f4de5e67640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5400 0x55d170660680 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55d1714e1800 tx=0x55d1714e2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.523+0000 7f4de065c640 1 -- 172.21.15.81:0/2343698184 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d17063b080 con 0x55d1714c5400 2023-11-08T16:31:20.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.523+0000 7f4de065c640 1 -- 172.21.15.81:0/2343698184 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d1714c6a80 con 0x55d1714c5400 2023-11-08T16:31:20.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.523+0000 7f4de065c640 1 -- 172.21.15.81:0/2343698184 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55d17063af20 con 0x55d1714c5400 2023-11-08T16:31:20.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.523+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55d1714c84e0 con 0x55d1714c5400 2023-11-08T16:31:20.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.523+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1714c8680 con 0x55d1714c5400 2023-11-08T16:31:20.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.523+0000 7f4de863a5c0 4 datascan.init: resolving metadata pool 4 2023-11-08T16:31:20.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.524+0000 7f4de065c640 1 -- 172.21.15.81:0/2343698184 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55d170651340 con 0x55d1714c5400 2023-11-08T16:31:20.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.525+0000 7f4de065c640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55d170958000 0x55d170661700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.525+0000 7f4de065c640 1 -- 172.21.15.81:0/2343698184 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x55d1706b4a80 con 0x55d1714c5400 2023-11-08T16:31:20.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.525+0000 7f4de863a5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:31:20.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.525+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55d1714c5800 0x55d170660c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.525+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 4.14 4:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55d17064d880 con 0x55d1714c5800 2023-11-08T16:31:20.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.525+0000 7f4de6e69640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55d170958000 0x55d170661700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.526+0000 7f4de6668640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55d1714c5800 0x55d170660c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.526+0000 7f4de6e69640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55d170958000 0x55d170661700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55d171372a50 tx=0x55d1713ca4e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.526+0000 7f4de6668640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55d1714c5800 0x55d170660c00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.526+0000 7f4de6668640 1 -- 172.21.15.81:0/2343698184 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x55d1714b0b40 con 0x55d1714c5800 2023-11-08T16:31:20.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.526+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:2 4.14 4:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55d17064dc00 con 0x55d1714c5800 2023-11-08T16:31:20.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.528+0000 7f4de6668640 1 -- 172.21.15.81:0/2343698184 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v40'43 uv43 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x55d1714b0b40 con 0x55d1714c5800 2023-11-08T16:31:20.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.528+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55d1714c5c00 0x55d170661180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.528+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:3 4.1f 4:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55d17064d880 con 0x55d1714c5c00 2023-11-08T16:31:20.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.529+0000 7f4de6e69640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55d1714c5c00 0x55d170661180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.530+0000 7f4de6e69640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55d1714c5c00 0x55d170661180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.530+0000 7f4de6e69640 1 -- 172.21.15.81:0/2343698184 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x55d1714b1200 con 0x55d1714c5c00 2023-11-08T16:31:20.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.530+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:4 4.1f 4:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55d17064dc00 con 0x55d1714c5c00 2023-11-08T16:31:20.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.534+0000 7f4de6e69640 1 -- 172.21.15.81:0/2343698184 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v40'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x55d1714b1200 con 0x55d1714c5c00 2023-11-08T16:31:20.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.535+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55d170958400 0x55d170966000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.535+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:5 4.13 4:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55d17064d880 con 0x55d170958400 2023-11-08T16:31:20.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.535+0000 7f4de6668640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55d170958400 0x55d170966000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.537+0000 7f4de6668640 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55d170958400 0x55d170966000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.537+0000 7f4de6668640 1 -- 172.21.15.81:0/2343698184 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55d1714b0b40 con 0x55d170958400 2023-11-08T16:31:20.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.537+0000 7f4de863a5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:31:20.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.538+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:6 4.13 4:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55d17064dc00 con 0x55d170958400 2023-11-08T16:31:20.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de6668640 1 -- 172.21.15.81:0/2343698184 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v40'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x55d1714b0b40 con 0x55d170958400 2023-11-08T16:31:20.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-11-08T16:31:20.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55d1714c5800 msgr2=0x55d170660c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55d1714c5800 0x55d170660c00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55d170958400 msgr2=0x55d170966000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55d170958400 0x55d170966000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55d1714c5c00 msgr2=0x55d170661180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55d1714c5c00 0x55d170661180 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de6668640 1 -- 172.21.15.81:0/2343698184 reap_dead start 2023-11-08T16:31:20.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55d170958000 msgr2=0x55d170661700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55d170958000 0x55d170661700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x55d171372a50 tx=0x55d1713ca4e0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5400 msgr2=0x55d170660680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5400 0x55d170660680 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x55d1714e1800 tx=0x55d1714e2c00 comp rx=0 tx=0).stop 2023-11-08T16:31:20.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 shutdown_connections 2023-11-08T16:31:20.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55d170958000 0x55d170661700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/2343698184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1714c5400 0x55d170660680 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 >> 172.21.15.81:0/2343698184 conn(0x55d1713c9400 msgr2=0x55d171370800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 shutdown_connections 2023-11-08T16:31:20.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.542+0000 7f4de863a5c0 1 -- 172.21.15.81:0/2343698184 wait complete. 2023-11-08T16:31:20.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.543+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1175640436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1713c8800 msgr2=0x55d17065eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.543+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/1175640436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1713c8800 0x55d17065eb00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x55d17140b290 tx=0x55d1713ca480 comp rx=0 tx=0).stop 2023-11-08T16:31:20.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.543+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1175640436 shutdown_connections 2023-11-08T16:31:20.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.543+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/1175640436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d1713c9c00 0x55d17065fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.543+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/1175640436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d1713c8800 0x55d17065eb00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.543+0000 7f4de863a5c0 1 --2- 172.21.15.81:0/1175640436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d1705f5c00 0x55d17065e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.543+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1175640436 >> 172.21.15.81:0/1175640436 conn(0x55d1705f5000 msgr2=0x55d171371000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.543+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1175640436 shutdown_connections 2023-11-08T16:31:20.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.543+0000 7f4de863a5c0 1 -- 172.21.15.81:0/1175640436 wait complete. 2023-11-08T16:31:20.555 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.140159, result: 2023-11-08T16:31:20.555 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-11-08T16:31:20.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.641+0000 7f19a300c5c0 1 -- start start 2023-11-08T16:31:20.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.641+0000 7f19a300c5c0 1 Processor -- start 2023-11-08T16:31:20.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.642+0000 7f19a300c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed78288400 0x55ed78213600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.642+0000 7f19a300c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed78288800 0x55ed78212b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.642+0000 7f19a300c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ed78289c00 0x55ed78213b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.642+0000 7f19a300c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed781f1180 con 0x55ed78288400 2023-11-08T16:31:20.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.642+0000 7f19a300c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed781f0140 con 0x55ed78288800 2023-11-08T16:31:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.642+0000 7f19a300c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed781f0b40 con 0x55ed78289c00 2023-11-08T16:31:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.642+0000 7f19a0b5a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed78288400 0x55ed78213600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.642+0000 7f19a0b5a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed78288400 0x55ed78213600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53336/0 (socket says 172.21.15.81:53336) 2023-11-08T16:31:20.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.642+0000 7f19a0b5a640 1 -- 172.21.15.81:0/2859282266 learned_addr learned my addr 172.21.15.81:0/2859282266 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f19a0b5a640 1 -- 172.21.15.81:0/2859282266 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ed78289c00 msgr2=0x55ed78213b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f19a0b5a640 1 --2- 172.21.15.81:0/2859282266 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ed78289c00 0x55ed78213b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f19a0b5a640 1 -- 172.21.15.81:0/2859282266 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed78288800 msgr2=0x55ed78212b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f19a0b5a640 1 --2- 172.21.15.81:0/2859282266 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed78288800 0x55ed78212b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f19a0b5a640 1 -- 172.21.15.81:0/2859282266 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed781ce4e0 con 0x55ed78288400 2023-11-08T16:31:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f19a0b5a640 1 --2- 172.21.15.81:0/2859282266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed78288400 0x55ed78213600 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55ed78fbd320 tx=0x55ed7905ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f954b35a6a4d212 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f199e355640 1 -- 172.21.15.81:0/2859282266 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ed781ef1e0 con 0x55ed78288400 2023-11-08T16:31:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f199e355640 1 -- 172.21.15.81:0/2859282266 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ed79068000 con 0x55ed78288400 2023-11-08T16:31:20.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f199e355640 1 -- 172.21.15.81:0/2859282266 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ed781eef20 con 0x55ed78288400 2023-11-08T16:31:20.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f19a300c5c0 1 -- 172.21.15.81:0/2859282266 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed781ce1a0 con 0x55ed78288400 2023-11-08T16:31:20.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f19a300c5c0 1 -- 172.21.15.81:0/2859282266 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ed7906a000 con 0x55ed78288400 2023-11-08T16:31:20.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f19a300c5c0 4 waiting for MDS map... 2023-11-08T16:31:20.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.643+0000 7f199e355640 1 -- 172.21.15.81:0/2859282266 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x55ed78268c40 con 0x55ed78288400 2023-11-08T16:31:20.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.644+0000 7f199e355640 1 -- 172.21.15.81:0/2859282266 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x55ed781f8f00 con 0x55ed78288400 2023-11-08T16:31:20.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.644+0000 7f19a300c5c0 4 Got MDS map 25 2023-11-08T16:31:20.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.644+0000 7f19a300c5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:31:20.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.644+0000 7f19a300c5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:31:20.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.644+0000 7f19a300c5c0 1 Processor -- start 2023-11-08T16:31:20.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.645+0000 7f19a300c5c0 1 -- start start 2023-11-08T16:31:20.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.645+0000 7f19a300c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ed78289800 0x55ed78213080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.645+0000 7f19a300c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed79066c00 0x55ed78212580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.645+0000 7f19a300c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067000 0x55ed78214100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.645+0000 7f19a300c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed781f0a00 con 0x55ed79067000 2023-11-08T16:31:20.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.645+0000 7f19a300c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed781f0000 con 0x55ed79066c00 2023-11-08T16:31:20.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.645+0000 7f19a300c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed78276dc0 con 0x55ed78289800 2023-11-08T16:31:20.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f19a0359640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067000 0x55ed78214100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f19a0359640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067000 0x55ed78214100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53352/0 (socket says 172.21.15.81:53352) 2023-11-08T16:31:20.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f19a0359640 1 -- 172.21.15.81:0/1198805794 learned_addr learned my addr 172.21.15.81:0/1198805794 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f19a0359640 1 -- 172.21.15.81:0/1198805794 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ed78289800 msgr2=0x55ed78213080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f19a0359640 1 --2- 172.21.15.81:0/1198805794 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ed78289800 0x55ed78213080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f19a0359640 1 -- 172.21.15.81:0/1198805794 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed79066c00 msgr2=0x55ed78212580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f19a0359640 1 --2- 172.21.15.81:0/1198805794 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed79066c00 0x55ed78212580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f19a0359640 1 -- 172.21.15.81:0/1198805794 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed7906a1a0 con 0x55ed79067000 2023-11-08T16:31:20.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f19a0359640 1 --2- 172.21.15.81:0/1198805794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067000 0x55ed78214100 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55ed79082750 tx=0x55ed79084cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8112d99c74d00b08 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f199c351640 1 -- 172.21.15.81:0/1198805794 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ed781ef1e0 con 0x55ed79067000 2023-11-08T16:31:20.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f199c351640 1 -- 172.21.15.81:0/1198805794 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ed79068000 con 0x55ed79067000 2023-11-08T16:31:20.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.646+0000 7f199c351640 1 -- 172.21.15.81:0/1198805794 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ed781eef20 con 0x55ed79067000 2023-11-08T16:31:20.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.647+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1198805794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067000 msgr2=0x55ed78214100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.647+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1198805794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067000 0x55ed78214100 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x55ed79082750 tx=0x55ed79084cc0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.647+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1198805794 shutdown_connections 2023-11-08T16:31:20.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.647+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1198805794 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067000 0x55ed78214100 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.647+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1198805794 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed79066c00 0x55ed78212580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.647+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1198805794 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ed78289800 0x55ed78213080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.647+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1198805794 >> 172.21.15.81:0/1198805794 conn(0x55ed78289400 msgr2=0x55ed78f25800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.647+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1198805794 shutdown_connections 2023-11-08T16:31:20.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.647+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1198805794 wait complete. 2023-11-08T16:31:20.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.648+0000 7f19a300c5c0 1 Processor -- start 2023-11-08T16:31:20.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.648+0000 7f19a300c5c0 1 -- start start 2023-11-08T16:31:20.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a300c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ed78289400 0x55ed78215180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a300c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed79067c00 0x55ed78214c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a300c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067800 0x55ed78214680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a300c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed78277180 con 0x55ed79067800 2023-11-08T16:31:20.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a300c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed78276c80 con 0x55ed79067c00 2023-11-08T16:31:20.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a300c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ed78276b40 con 0x55ed78289400 2023-11-08T16:31:20.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a0359640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067800 0x55ed78214680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a0359640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067800 0x55ed78214680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53358/0 (socket says 172.21.15.81:53358) 2023-11-08T16:31:20.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 learned_addr learned my addr 172.21.15.81:0/1960618411 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ed78289400 msgr2=0x55ed78215180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a0359640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ed78289400 0x55ed78215180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed79067c00 msgr2=0x55ed78214c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a0359640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed79067c00 0x55ed78214c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed7906a340 con 0x55ed79067800 2023-11-08T16:31:20.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.649+0000 7f19a0359640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067800 0x55ed78214680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55ed79083a10 tx=0x55ed79084c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.650+0000 7f199ab4e640 1 -- 172.21.15.81:0/1960618411 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ed781ef1e0 con 0x55ed79067800 2023-11-08T16:31:20.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.650+0000 7f199ab4e640 1 -- 172.21.15.81:0/1960618411 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ed79069800 con 0x55ed79067800 2023-11-08T16:31:20.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.650+0000 7f199ab4e640 1 -- 172.21.15.81:0/1960618411 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ed781eef20 con 0x55ed79067800 2023-11-08T16:31:20.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.650+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ed7906a4e0 con 0x55ed79067800 2023-11-08T16:31:20.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.650+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed7906a1a0 con 0x55ed79067800 2023-11-08T16:31:20.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.650+0000 7f19a300c5c0 4 datascan.init: resolving metadata pool 4 2023-11-08T16:31:20.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.651+0000 7f199ab4e640 1 -- 172.21.15.81:0/1960618411 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55ed78204840 con 0x55ed79067800 2023-11-08T16:31:20.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.652+0000 7f199ab4e640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55ed790dcc00 0x55ed78215700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.652+0000 7f199ab4e640 1 -- 172.21.15.81:0/1960618411 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x55ed78268a80 con 0x55ed79067800 2023-11-08T16:31:20.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.652+0000 7f19a300c5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:31:20.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.652+0000 7f19a300c5c0 4 datascan.main: data pool 'cephfs_data' has ID 5 2023-11-08T16:31:20.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.652+0000 7f19a300c5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:31:20.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.652+0000 7f19a300c5c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 6 2023-11-08T16:31:20.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.652+0000 7f19a300c5c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-11-08T16:31:20.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.652+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55ed79067400 0x55ed78214100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.652+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 5.0 5:00000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78201880 con 0x55ed79067400 2023-11-08T16:31:20.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.652+0000 7f19a135b640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55ed790dcc00 0x55ed78215700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.653+0000 7f19a0b5a640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55ed79067400 0x55ed78214100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.653+0000 7f19a135b640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55ed790dcc00 0x55ed78215700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ed7829a7b0 tx=0x55ed78f38060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.653+0000 7f19a0b5a640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55ed79067400 0x55ed78214100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.653+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.654+0000 7f199bb50640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55ed790dd000 0x55ed78514000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.654+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:2 5.20 5:04000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e000 con 0x55ed790dd000 2023-11-08T16:31:20.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.654+0000 7f19a135b640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55ed790dd000 0x55ed78514000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.655+0000 7f19a135b640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55ed790dd000 0x55ed78514000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.656+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(2 [pgnls start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.656+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:3 5.10 5:08000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e380 con 0x55ed790dd000 2023-11-08T16:31:20.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.657+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(3 [pgnls start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.657+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:4 5.30 5:0c000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e000 con 0x55ed790dd000 2023-11-08T16:31:20.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.657+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(4 [pgnls start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.658+0000 7f199c351640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55ed790dd400 0x55ed78514580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.658+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:5 5.8 5:10000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e700 con 0x55ed790dd400 2023-11-08T16:31:20.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.658+0000 7f19a0b5a640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55ed790dd400 0x55ed78514580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.658+0000 7f19a0b5a640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55ed790dd400 0x55ed78514580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.659+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(5 [pgnls start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed790dd400 2023-11-08T16:31:20.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.659+0000 7f199bb50640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55ed790dd800 0x55ed78514b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.659+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:6 5.28 5:14000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e380 con 0x55ed790dd800 2023-11-08T16:31:20.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.659+0000 7f19a0359640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55ed790dd800 0x55ed78514b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.660+0000 7f19a0359640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55ed790dd800 0x55ed78514b00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.661+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(6 [pgnls start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.661+0000 7f199c351640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55ed7852a000 0x55ed78515600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.661+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:7 5.18 5:18000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851ea80 con 0x55ed7852a000 2023-11-08T16:31:20.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.662+0000 7f19a135b640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55ed7852a000 0x55ed78515600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.662+0000 7f19a135b640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55ed7852a000 0x55ed78515600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.663+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(7 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.663+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:8 5.38 5:1c000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851ee00 con 0x55ed790dd000 2023-11-08T16:31:20.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.664+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(8 [pgnls start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.664+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:9 5.4 5:20000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e000 con 0x55ed790dd000 2023-11-08T16:31:20.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.665+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(9 [pgnls start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.665+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:10 5.24 5:24000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851f180 con 0x55ed790dd800 2023-11-08T16:31:20.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.666+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(10 [pgnls start_epoch 40 out=49b] v38'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.666+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851ea80 con 0x55ed79067400 2023-11-08T16:31:20.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.666+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(11 [pgnls start_epoch 40 out=49b] v38'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.666+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:12 5.34 5:2c000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851f500 con 0x55ed790dd800 2023-11-08T16:31:20.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.667+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(12 [pgnls start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.667+0000 7f199c351640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55ed7852a400 0x55ed78515b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.667+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:13 5.c 5:30000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851ee00 con 0x55ed7852a400 2023-11-08T16:31:20.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.667+0000 7f19a0b5a640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55ed7852a400 0x55ed78515b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.668+0000 7f19a0b5a640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55ed7852a400 0x55ed78515b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.668+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(13 [pgnls start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.668+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:14 5.2c 5:34000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e700 con 0x55ed790dd400 2023-11-08T16:31:20.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.668+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(14 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed790dd400 2023-11-08T16:31:20.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.668+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:15 5.1c 5:38000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851f180 con 0x55ed7852a000 2023-11-08T16:31:20.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.669+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(15 [pgnls start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.669+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:16 5.3c 5:3c000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851ea80 con 0x55ed7852a400 2023-11-08T16:31:20.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.670+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(16 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.670+0000 7f199c351640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55ed7852a800 0x55ed78516100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.670+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:17 5.2 5:40000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851f880 con 0x55ed7852a800 2023-11-08T16:31:20.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.670+0000 7f19a0359640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55ed7852a800 0x55ed78516100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.671+0000 7f19a0359640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55ed7852a800 0x55ed78516100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.672+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(17 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed7852a800 2023-11-08T16:31:20.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.672+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:18 5.22 5:44000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e380 con 0x55ed790dd000 2023-11-08T16:31:20.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.673+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(18 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.673+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:19 5.12 5:48000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851fc00 con 0x55ed7852a400 2023-11-08T16:31:20.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.673+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(19 [pgnls start_epoch 40 out=49b] v38'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.674+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:20 5.32 5:4c000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78201880 con 0x55ed79067400 2023-11-08T16:31:20.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.674+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(20 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.674+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:21 5.a 5:50000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851ee00 con 0x55ed790dd400 2023-11-08T16:31:20.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.674+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(21 [pgnls start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed790dd400 2023-11-08T16:31:20.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.674+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:22 5.2a 5:54000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540000 con 0x55ed7852a000 2023-11-08T16:31:20.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.675+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(22 [pgnls start_epoch 40 out=81b] v38'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.675+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:23 5.2a 5:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78201c00 con 0x55ed7852a000 2023-11-08T16:31:20.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.676+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(23 100000001f6.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.676+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628278.0 2023-11-08T16:31:20.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.676+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:24 5.2a 5:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540000 con 0x55ed7852a000 2023-11-08T16:31:20.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.679+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(24 100000001f6.00000000 [call] v40'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.679+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:25 5.1a 5:58000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e000 con 0x55ed790dd000 2023-11-08T16:31:20.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.680+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(25 [pgnls start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.680+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:26 5.3a 5:5c000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540380 con 0x55ed79067400 2023-11-08T16:31:20.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.680+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(26 [pgnls start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.680+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:27 5.6 5:60000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e700 con 0x55ed790dd400 2023-11-08T16:31:20.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.681+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(27 [pgnls start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed790dd400 2023-11-08T16:31:20.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.681+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:28 5.26 5:64000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540700 con 0x55ed79067400 2023-11-08T16:31:20.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.681+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(28 [pgnls start_epoch 40 out=49b] v38'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.681+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:29 5.16 5:68000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851ea80 con 0x55ed790dd800 2023-11-08T16:31:20.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.682+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(29 [pgnls start_epoch 40 out=49b] v38'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.682+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:30 5.36 5:6c000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540a80 con 0x55ed7852a400 2023-11-08T16:31:20.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.683+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(30 [pgnls start_epoch 40 out=101b] v38'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.683+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:31 5.36 5:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851f180 con 0x55ed7852a400 2023-11-08T16:31:20.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.683+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(31 100000001f9.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.683+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628281.0 2023-11-08T16:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.683+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:32 5.36 5:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e380 con 0x55ed7852a400 2023-11-08T16:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.685+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(32 100000001f9.00000000 [call] v40'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.685+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:33 5.36 5:6fe38dc2::::head [pgnls start_epoch 40 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851f500 con 0x55ed7852a400 2023-11-08T16:31:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.685+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(33 [pgnls start_epoch 40 out=81b] v40'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.685+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:34 5.36 5:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851f880 con 0x55ed7852a400 2023-11-08T16:31:20.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.686+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(34 100000001f7.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.686+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628279.0 2023-11-08T16:31:20.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.686+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:35 5.36 5:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851ea80 con 0x55ed7852a400 2023-11-08T16:31:20.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.688+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(35 100000001f7.00000000 [call] v40'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.688+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:36 5.e 5:70000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851ee00 con 0x55ed7852a000 2023-11-08T16:31:20.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.688+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(36 [pgnls start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.689+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:37 5.2e 5:74000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851fc00 con 0x55ed79067400 2023-11-08T16:31:20.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.689+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(37 [pgnls start_epoch 40 out=49b] v38'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.689+0000 7f199bb50640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55ed790ddc00 0x55ed78515080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.689+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:38 5.1e 5:78000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540e00 con 0x55ed790ddc00 2023-11-08T16:31:20.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.689+0000 7f19a135b640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55ed790ddc00 0x55ed78515080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.690+0000 7f19a135b640 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55ed790ddc00 0x55ed78515080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.690+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(38 [pgnls start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.690+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:39 5.3e 5:7c000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e000 con 0x55ed79067400 2023-11-08T16:31:20.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.691+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(39 [pgnls start_epoch 40 out=49b] v38'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.691+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:40 5.1 5:80000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851f880 con 0x55ed7852a000 2023-11-08T16:31:20.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.692+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(40 [pgnls start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.692+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:41 5.21 5:84000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851ee00 con 0x55ed7852a800 2023-11-08T16:31:20.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.693+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(41 [pgnls start_epoch 40 out=49b] v38'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed7852a800 2023-11-08T16:31:20.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.693+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:42 5.11 5:88000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851ea80 con 0x55ed7852a800 2023-11-08T16:31:20.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.693+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(42 [pgnls start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed7852a800 2023-11-08T16:31:20.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.693+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:43 5.31 5:8c000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851f180 con 0x55ed790dd800 2023-11-08T16:31:20.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.694+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(43 [pgnls start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.695+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:44 5.9 5:90000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e700 con 0x55ed790dd000 2023-11-08T16:31:20.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.696+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(44 [pgnls start_epoch 40 out=49b] v38'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.696+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:45 5.29 5:94000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851e380 con 0x55ed790ddc00 2023-11-08T16:31:20.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.696+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(45 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.696+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:46 5.19 5:98000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540000 con 0x55ed7852a400 2023-11-08T16:31:20.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.697+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(46 [pgnls start_epoch 40 out=81b] v38'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.697+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:47 5.19 5:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78201880 con 0x55ed7852a400 2023-11-08T16:31:20.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.697+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(47 100000001f8.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.697+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628280.0 2023-11-08T16:31:20.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.697+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:48 5.19 5:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541880 con 0x55ed7852a400 2023-11-08T16:31:20.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.700+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 12 ==== osd_op_reply(48 100000001f8.00000000 [call] v40'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.700+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:49 5.39 5:9c000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541c00 con 0x55ed790dd000 2023-11-08T16:31:20.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.702+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 9 ==== osd_op_reply(49 [pgnls start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.702+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:50 5.5 5:a0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540380 con 0x55ed790ddc00 2023-11-08T16:31:20.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.702+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(50 [pgnls start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.702+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:51 5.25 5:a4000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7851f500 con 0x55ed7852a400 2023-11-08T16:31:20.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.703+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 13 ==== osd_op_reply(51 [pgnls start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.703+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:52 5.15 5:a8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540700 con 0x55ed790dd800 2023-11-08T16:31:20.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.704+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(52 [pgnls start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.704+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:53 5.35 5:ac000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78201c00 con 0x55ed790dd000 2023-11-08T16:31:20.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.705+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 10 ==== osd_op_reply(53 [pgnls start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.705+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:54 5.d 5:b0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540a80 con 0x55ed790dd400 2023-11-08T16:31:20.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.706+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.2 v2:172.21.15.81:6824/3759311428 5 ==== osd_op_reply(54 [pgnls start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed790dd400 2023-11-08T16:31:20.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.706+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:55 5.2d 5:b4000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540000 con 0x55ed79067400 2023-11-08T16:31:20.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.706+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(55 [pgnls start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.707+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:56 5.1d 5:b8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540e00 con 0x55ed7852a400 2023-11-08T16:31:20.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.707+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 14 ==== osd_op_reply(56 [pgnls start_epoch 40 out=49b] v38'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.707+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:57 5.3d 5:bc000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541880 con 0x55ed790dd800 2023-11-08T16:31:20.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.708+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(57 [pgnls start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.708+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:58 5.3 5:c0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541180 con 0x55ed790ddc00 2023-11-08T16:31:20.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.709+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(58 [pgnls start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.709+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:59 5.23 5:c4000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540a80 con 0x55ed7852a000 2023-11-08T16:31:20.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.710+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(59 [pgnls start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.710+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:60 5.13 5:c8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541500 con 0x55ed79067400 2023-11-08T16:31:20.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.711+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(60 [pgnls start_epoch 40 out=49b] v38'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.711+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:61 5.33 5:cc000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541c00 con 0x55ed790dd400 2023-11-08T16:31:20.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.712+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.2 v2:172.21.15.81:6824/3759311428 6 ==== osd_op_reply(61 [pgnls start_epoch 40 out=49b] v38'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed790dd400 2023-11-08T16:31:20.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.712+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:62 5.b 5:d0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540700 con 0x55ed790dd400 2023-11-08T16:31:20.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.712+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.2 v2:172.21.15.81:6824/3759311428 7 ==== osd_op_reply(62 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed790dd400 2023-11-08T16:31:20.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.712+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:63 5.2b 5:d4000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540380 con 0x55ed7852a000 2023-11-08T16:31:20.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.713+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(63 [pgnls start_epoch 40 out=49b] v38'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.714+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:64 5.1b 5:d8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78558000 con 0x55ed790dd800 2023-11-08T16:31:20.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.715+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(64 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.715+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:65 5.3b 5:dc000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78201880 con 0x55ed790dd400 2023-11-08T16:31:20.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.715+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.2 v2:172.21.15.81:6824/3759311428 8 ==== osd_op_reply(65 [pgnls start_epoch 40 out=49b] v38'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed790dd400 2023-11-08T16:31:20.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.715+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:66 5.7 5:e0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78558380 con 0x55ed790dd800 2023-11-08T16:31:20.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.717+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 9 ==== osd_op_reply(66 [pgnls start_epoch 40 out=81b] v38'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.717+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:67 5.7 5:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855a000 con 0x55ed790dd800 2023-11-08T16:31:20.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.718+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 10 ==== osd_op_reply(67 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.718+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511627777.0 2023-11-08T16:31:20.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.718+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:68 5.7 5:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855a380 con 0x55ed790dd800 2023-11-08T16:31:20.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.722+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 11 ==== osd_op_reply(68 10000000001.00000000 [call] v40'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.722+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:69 5.27 5:e4000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855a700 con 0x55ed7852a400 2023-11-08T16:31:20.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.722+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 15 ==== osd_op_reply(69 [pgnls start_epoch 40 out=49b] v38'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.723+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:70 5.17 5:e8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78558700 con 0x55ed79067400 2023-11-08T16:31:20.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.723+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(70 [pgnls start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.723+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:71 5.37 5:ec000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78201c00 con 0x55ed7852a000 2023-11-08T16:31:20.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.724+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(71 [pgnls start_epoch 40 out=49b] v38'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.724+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:72 5.f 5:f0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78558a80 con 0x55ed790dd800 2023-11-08T16:31:20.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.725+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 12 ==== osd_op_reply(72 [pgnls start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.726+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:73 5.2f 5:f4000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78558700 con 0x55ed790dd000 2023-11-08T16:31:20.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.727+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 11 ==== osd_op_reply(73 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.727+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:74 5.1f 5:f8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78558e00 con 0x55ed7852a800 2023-11-08T16:31:20.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.728+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(74 [pgnls start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed7852a800 2023-11-08T16:31:20.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.728+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:75 5.3f 5:fc000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540000 con 0x55ed7852a800 2023-11-08T16:31:20.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.729+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(75 [pgnls start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed7852a800 2023-11-08T16:31:20.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.729+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:76 6.0 6:00000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855aa80 con 0x55ed790ddc00 2023-11-08T16:31:20.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.730+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(76 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.730+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:77 6.10 6:08000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540e00 con 0x55ed790ddc00 2023-11-08T16:31:20.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.730+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(77 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.730+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:78 6.8 6:10000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78559180 con 0x55ed790dd000 2023-11-08T16:31:20.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.732+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 12 ==== osd_op_reply(78 [pgnls start_epoch 40 out=81b] v38'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.732+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:79 6.8 6:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855ae00 con 0x55ed790dd000 2023-11-08T16:31:20.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.733+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 13 ==== osd_op_reply(79 100000001f9.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.733+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628281.1 2023-11-08T16:31:20.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.733+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:80 5.36 5:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855b180 con 0x55ed7852a400 2023-11-08T16:31:20.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.737+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 16 ==== osd_op_reply(80 100000001f9.00000000 [call,setxattr (8)] v40'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.737+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:81 6.18 6:18000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855b500 con 0x55ed790ddc00 2023-11-08T16:31:20.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.737+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(81 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.737+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:82 6.4 6:20000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78559500 con 0x55ed7852a400 2023-11-08T16:31:20.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.738+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 17 ==== osd_op_reply(82 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.738+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:83 6.14 6:28000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541500 con 0x55ed790dd400 2023-11-08T16:31:20.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.739+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.2 v2:172.21.15.81:6824/3759311428 9 ==== osd_op_reply(83 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed790dd400 2023-11-08T16:31:20.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.739+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:84 6.c 6:30000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78559880 con 0x55ed79067400 2023-11-08T16:31:20.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.740+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(84 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.740+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:85 6.1c 6:38000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541c00 con 0x55ed790dd000 2023-11-08T16:31:20.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.741+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 14 ==== osd_op_reply(85 [pgnls start_epoch 40 out=81b] v38'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.741+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:86 6.1c 6:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78558700 con 0x55ed790dd000 2023-11-08T16:31:20.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.742+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 15 ==== osd_op_reply(86 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.742+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628278.1 2023-11-08T16:31:20.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.742+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:87 5.2a 5:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78558000 con 0x55ed7852a000 2023-11-08T16:31:20.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.746+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(87 100000001f6.00000000 [call,setxattr (8)] v40'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.746+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:88 6.2 6:40000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78558380 con 0x55ed7852a000 2023-11-08T16:31:20.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.747+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(88 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.747+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:89 6.12 6:48000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540700 con 0x55ed790dd000 2023-11-08T16:31:20.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.748+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 16 ==== osd_op_reply(89 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.748+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:90 6.a 6:50000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78559c00 con 0x55ed790dd800 2023-11-08T16:31:20.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.749+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 13 ==== osd_op_reply(90 [pgnls start_epoch 40 out=81b] v38'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.749+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:91 6.a 6:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541180 con 0x55ed790dd800 2023-11-08T16:31:20.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.750+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 14 ==== osd_op_reply(91 100000001f6.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.751+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628278.0 2023-11-08T16:31:20.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.751+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:92 5.2a 5:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540a80 con 0x55ed7852a000 2023-11-08T16:31:20.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.754+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 13 ==== osd_op_reply(92 100000001f6.00000000 [call,setxattr (8)] v40'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.755+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:93 6.1a 6:58000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540380 con 0x55ed7852a000 2023-11-08T16:31:20.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.756+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 14 ==== osd_op_reply(93 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.756+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:94 6.6 6:60000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855ae00 con 0x55ed79067400 2023-11-08T16:31:20.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.756+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(94 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.756+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:95 6.16 6:68000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541880 con 0x55ed790ddc00 2023-11-08T16:31:20.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.757+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(95 [pgnls start_epoch 40 out=101b] v38'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.757+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:96 6.16 6:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540e00 con 0x55ed790ddc00 2023-11-08T16:31:20.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.758+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(96 100000001f9.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.758+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628281.0 2023-11-08T16:31:20.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.758+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:97 5.36 5:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78201c00 con 0x55ed7852a400 2023-11-08T16:31:20.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.762+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 18 ==== osd_op_reply(97 100000001f9.00000000 [call,setxattr (8)] v40'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.762+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:98 6.16 6:6fe38dc2::::head [pgnls start_epoch 40 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855a000 con 0x55ed790ddc00 2023-11-08T16:31:20.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.763+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(98 [pgnls start_epoch 40 out=81b] v38'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.763+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:99 6.16 6:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855a380 con 0x55ed790ddc00 2023-11-08T16:31:20.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.764+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 11 ==== osd_op_reply(99 100000001f7.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.764+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628279.0 2023-11-08T16:31:20.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.764+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:100 5.36 5:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855b880 con 0x55ed7852a400 2023-11-08T16:31:20.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.767+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 19 ==== osd_op_reply(100 100000001f7.00000000 [call,setxattr (8)] v40'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.767+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:101 6.e 6:70000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855bc00 con 0x55ed7852a000 2023-11-08T16:31:20.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.768+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 15 ==== osd_op_reply(101 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.768+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:102 6.1e 6:78000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855b500 con 0x55ed7852a000 2023-11-08T16:31:20.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.769+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 16 ==== osd_op_reply(102 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.769+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:103 6.1 6:80000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78201880 con 0x55ed7852a400 2023-11-08T16:31:20.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.770+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 20 ==== osd_op_reply(103 [pgnls start_epoch 40 out=81b] v38'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.770+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:104 6.1 6:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78566000 con 0x55ed7852a400 2023-11-08T16:31:20.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.771+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 21 ==== osd_op_reply(104 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.771+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628279.1 2023-11-08T16:31:20.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.771+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:105 5.36 5:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78566380 con 0x55ed7852a400 2023-11-08T16:31:20.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.775+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 22 ==== osd_op_reply(105 100000001f7.00000000 [call,setxattr (8)] v40'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.775+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:106 6.11 6:88000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78566700 con 0x55ed79067400 2023-11-08T16:31:20.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.776+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(106 [pgnls start_epoch 40 out=81b] v38'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.776+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:107 6.11 6:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78566a80 con 0x55ed79067400 2023-11-08T16:31:20.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.777+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(107 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.777+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628280.1 2023-11-08T16:31:20.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.777+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:108 5.19 5:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78566e00 con 0x55ed7852a400 2023-11-08T16:31:20.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.781+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 23 ==== osd_op_reply(108 100000001f8.00000000 [call,setxattr (8)] v40'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.781+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:109 6.9 6:90000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78567180 con 0x55ed790ddc00 2023-11-08T16:31:20.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.782+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.0 v2:172.21.15.81:6808/1666604866 12 ==== osd_op_reply(109 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790ddc00 2023-11-08T16:31:20.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.782+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:110 6.19 6:98000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855aa80 con 0x55ed790dd800 2023-11-08T16:31:20.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.783+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 15 ==== osd_op_reply(110 [pgnls start_epoch 40 out=81b] v38'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.783+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:111 6.19 6:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78567500 con 0x55ed790dd800 2023-11-08T16:31:20.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.784+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 16 ==== osd_op_reply(111 100000001f8.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.784+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511628280.0 2023-11-08T16:31:20.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.784+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:112 5.19 5:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78567880 con 0x55ed7852a400 2023-11-08T16:31:20.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.788+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 24 ==== osd_op_reply(112 100000001f8.00000000 [call,setxattr (8)] v40'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.788+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:113 6.5 6:a0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78567c00 con 0x55ed790dd000 2023-11-08T16:31:20.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.789+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 17 ==== osd_op_reply(113 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.789+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:114 6.15 6:a8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855a700 con 0x55ed790dd000 2023-11-08T16:31:20.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.790+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 18 ==== osd_op_reply(114 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.790+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:115 6.d 6:b0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78558a80 con 0x55ed790dd800 2023-11-08T16:31:20.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.791+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 17 ==== osd_op_reply(115 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.791+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:116 6.1d 6:b8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541c00 con 0x55ed7852a400 2023-11-08T16:31:20.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.792+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.1 v2:172.21.15.81:6800/2060798739 25 ==== osd_op_reply(116 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed7852a400 2023-11-08T16:31:20.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.792+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:117 6.3 6:c0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78558e00 con 0x55ed790dd000 2023-11-08T16:31:20.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.793+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.7 v2:172.21.15.93:6818/433691658 19 ==== osd_op_reply(117 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed790dd000 2023-11-08T16:31:20.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.793+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:118 6.13 6:c8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540700 con 0x55ed79067400 2023-11-08T16:31:20.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.794+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(118 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.794+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:119 6.b 6:d0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540000 con 0x55ed79067400 2023-11-08T16:31:20.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.795+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(119 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.795+0000 7f199bb50640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:120 6.1b 6:d8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78559180 con 0x55ed79067400 2023-11-08T16:31:20.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.796+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(120 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.796+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:121 6.7 6:e0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78541180 con 0x55ed790dd800 2023-11-08T16:31:20.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.798+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 18 ==== osd_op_reply(121 [pgnls start_epoch 40 out=81b] v38'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.798+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:122 6.7 6:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855aa80 con 0x55ed790dd800 2023-11-08T16:31:20.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.799+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 19 ==== osd_op_reply(122 10000000001.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.799+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511627777.0 2023-11-08T16:31:20.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.799+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:123 5.7 5:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855a000 con 0x55ed790dd800 2023-11-08T16:31:20.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.804+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 20 ==== osd_op_reply(123 10000000001.00000000 [call,setxattr (8)] v40'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.804+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:124 6.17 6:e8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855a380 con 0x55ed7852a000 2023-11-08T16:31:20.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.805+0000 7f19a135b640 1 -- 172.21.15.81:0/1960618411 <== osd.4 v2:172.21.15.93:6800/3259756279 17 ==== osd_op_reply(124 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02d80 con 0x55ed7852a000 2023-11-08T16:31:20.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.805+0000 7f199c351640 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:125 6.f 6:f0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed78540a80 con 0x55ed790dd400 2023-11-08T16:31:20.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.806+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.2 v2:172.21.15.81:6824/3759311428 10 ==== osd_op_reply(125 [pgnls start_epoch 40 out=81b] v38'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed790dd400 2023-11-08T16:31:20.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.806+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:126 6.f 6:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855bc00 con 0x55ed790dd400 2023-11-08T16:31:20.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.808+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.2 v2:172.21.15.81:6824/3759311428 11 ==== osd_op_reply(126 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed790dd400 2023-11-08T16:31:20.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.808+0000 7f19a300c5c0 10 datascan.operator(): handling object 1099511627777.1 2023-11-08T16:31:20.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.808+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:127 5.7 5:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855b500 con 0x55ed790dd800 2023-11-08T16:31:20.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.812+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 <== osd.5 v2:172.21.15.93:6802/2406579932 21 ==== osd_op_reply(127 10000000001.00000000 [call,setxattr (8)] v40'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55ed78f02fc0 con 0x55ed790dd800 2023-11-08T16:31:20.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.812+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:128 6.1f 6:f8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x55ed7855b180 con 0x55ed79067400 2023-11-08T16:31:20.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a0b5a640 1 -- 172.21.15.81:0/1960618411 <== osd.3 v2:172.21.15.81:6816/2756229107 18 ==== osd_op_reply(128 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55ed78f02b40 con 0x55ed79067400 2023-11-08T16:31:20.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55ed790ddc00 msgr2=0x55ed78515080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55ed790ddc00 0x55ed78515080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55ed7852a400 msgr2=0x55ed78515b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55ed7852a400 0x55ed78515b80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55ed790dd400 msgr2=0x55ed78514580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55ed790dd400 0x55ed78514580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55ed79067400 msgr2=0x55ed78214100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55ed79067400 0x55ed78214100 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 reap_dead start 2023-11-08T16:31:20.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55ed7852a000 msgr2=0x55ed78515600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.813+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55ed7852a000 0x55ed78515600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.814+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55ed790dd800 msgr2=0x55ed78514b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.814+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55ed790dd800 0x55ed78514b00 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.814+0000 7f19a0359640 1 -- 172.21.15.81:0/1960618411 reap_dead start 2023-11-08T16:31:20.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.814+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55ed7852a800 msgr2=0x55ed78516100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.814+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55ed7852a800 0x55ed78516100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.814+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55ed790dd000 msgr2=0x55ed78514000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.814+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55ed790dd000 0x55ed78514000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.814+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55ed790dcc00 msgr2=0x55ed78215700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.814+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55ed790dcc00 0x55ed78215700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x55ed7829a7b0 tx=0x55ed78f38060 comp rx=0 tx=0).stop 2023-11-08T16:31:20.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.814+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067800 msgr2=0x55ed78214680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.814+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067800 0x55ed78214680 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x55ed79083a10 tx=0x55ed79084c00 comp rx=0 tx=0).stop 2023-11-08T16:31:20.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.815+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 shutdown_connections 2023-11-08T16:31:20.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.815+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55ed7852a800 0x55ed78516100 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.815+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55ed790dd000 0x55ed78514000 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.815+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55ed790dcc00 0x55ed78215700 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.815+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/1960618411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed79067800 0x55ed78214680 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.815+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 >> 172.21.15.81:0/1960618411 conn(0x55ed78289800 msgr2=0x55ed78f25800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.815+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 shutdown_connections 2023-11-08T16:31:20.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.815+0000 7f19a300c5c0 1 -- 172.21.15.81:0/1960618411 wait complete. 2023-11-08T16:31:20.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.816+0000 7f19a300c5c0 1 -- 172.21.15.81:0/2859282266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed78288400 msgr2=0x55ed78213600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.816+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/2859282266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed78288400 0x55ed78213600 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x55ed78fbd320 tx=0x55ed7905ac00 comp rx=0 tx=0).stop 2023-11-08T16:31:20.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.816+0000 7f19a300c5c0 1 -- 172.21.15.81:0/2859282266 shutdown_connections 2023-11-08T16:31:20.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.816+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/2859282266 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ed78289c00 0x55ed78213b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.816+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/2859282266 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ed78288800 0x55ed78212b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.816+0000 7f19a300c5c0 1 --2- 172.21.15.81:0/2859282266 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ed78288400 0x55ed78213600 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.816+0000 7f19a300c5c0 1 -- 172.21.15.81:0/2859282266 >> 172.21.15.81:0/2859282266 conn(0x55ed781a9000 msgr2=0x55ed78f24000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.817+0000 7f19a300c5c0 1 -- 172.21.15.81:0/2859282266 shutdown_connections 2023-11-08T16:31:20.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.817+0000 7f19a300c5c0 1 -- 172.21.15.81:0/2859282266 wait complete. 2023-11-08T16:31:20.831 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.276240, result: 2023-11-08T16:31:20.832 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-11-08T16:31:20.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.932+0000 7fe9b504b5c0 1 -- start start 2023-11-08T16:31:20.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.932+0000 7fe9b504b5c0 1 Processor -- start 2023-11-08T16:31:20.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.933+0000 7fe9b504b5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561542c1c000 0x561542b0d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.933+0000 7fe9b504b5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561542aa3c00 0x561542b0cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.933+0000 7fe9b504b5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561542c1c800 0x561542b0db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.933+0000 7fe9b504b5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561542aeb180 con 0x561542c1c000 2023-11-08T16:31:20.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.933+0000 7fe9b504b5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561542aea140 con 0x561542c1c800 2023-11-08T16:31:20.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.933+0000 7fe9b504b5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x561542aeab40 con 0x561542aa3c00 2023-11-08T16:31:20.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b30bc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561542c1c000 0x561542b0d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b30bc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561542c1c000 0x561542b0d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45998/0 (socket says 172.21.15.81:45998) 2023-11-08T16:31:20.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b30bc640 1 -- 172.21.15.81:0/4068859666 learned_addr learned my addr 172.21.15.81:0/4068859666 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/4068859666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561542aa3c00 0x561542b0cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b38bd640 1 --2- 172.21.15.81:0/4068859666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561542c1c800 0x561542b0db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b30bc640 1 -- 172.21.15.81:0/4068859666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561542aa3c00 msgr2=0x561542b0cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b30bc640 1 --2- 172.21.15.81:0/4068859666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561542aa3c00 0x561542b0cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b30bc640 1 -- 172.21.15.81:0/4068859666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561542c1c800 msgr2=0x561542b0db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b30bc640 1 --2- 172.21.15.81:0/4068859666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561542c1c800 0x561542b0db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b30bc640 1 -- 172.21.15.81:0/4068859666 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561542ac84e0 con 0x561542c1c000 2023-11-08T16:31:20.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/4068859666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561542aa3c00 0x561542b0cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:20.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.934+0000 7fe9b30bc640 1 --2- 172.21.15.81:0/4068859666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561542c1c000 0x561542b0d080 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x5615438b2cf0 tx=0x561543956120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc1fefd0b91f9ea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.935+0000 7fe9b08b7640 1 -- 172.21.15.81:0/4068859666 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561542ae8dc0 con 0x561542c1c000 2023-11-08T16:31:20.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.935+0000 7fe9b08b7640 1 -- 172.21.15.81:0/4068859666 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561543960000 con 0x561542c1c000 2023-11-08T16:31:20.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.935+0000 7fe9b08b7640 1 -- 172.21.15.81:0/4068859666 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561542ae8f20 con 0x561542c1c000 2023-11-08T16:31:20.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.935+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4068859666 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561542ac8820 con 0x561542c1c000 2023-11-08T16:31:20.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.935+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4068859666 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x561542ac8680 con 0x561542c1c000 2023-11-08T16:31:20.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.935+0000 7fe9b504b5c0 4 waiting for MDS map... 2023-11-08T16:31:20.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.935+0000 7fe9b08b7640 1 -- 172.21.15.81:0/4068859666 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x561542b62c40 con 0x561542c1c000 2023-11-08T16:31:20.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.935+0000 7fe9b08b7640 1 -- 172.21.15.81:0/4068859666 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x561542af2f00 con 0x561542c1c000 2023-11-08T16:31:20.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.935+0000 7fe9b504b5c0 4 Got MDS map 25 2023-11-08T16:31:20.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.935+0000 7fe9b504b5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:31:20.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.935+0000 7fe9b504b5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:31:20.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.936+0000 7fe9b504b5c0 1 Processor -- start 2023-11-08T16:31:20.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.936+0000 7fe9b504b5c0 1 -- start start 2023-11-08T16:31:20.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.937+0000 7fe9b504b5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615439c8000 0x561542b0d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.937+0000 7fe9b504b5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5615439c8400 0x561542b0c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.937+0000 7fe9b504b5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.937+0000 7fe9b504b5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561542aeaa00 con 0x5615439c8800 2023-11-08T16:31:20.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.937+0000 7fe9b504b5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561542aea000 con 0x5615439c8000 2023-11-08T16:31:20.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.937+0000 7fe9b504b5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x561542b71180 con 0x5615439c8400 2023-11-08T16:31:20.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.937+0000 7fe9b28bb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.937+0000 7fe9b28bb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46006/0 (socket says 172.21.15.81:46006) 2023-11-08T16:31:20.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.937+0000 7fe9b28bb640 1 -- 172.21.15.81:0/4019840279 learned_addr learned my addr 172.21.15.81:0/4019840279 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.938+0000 7fe9b38bd640 1 --2- 172.21.15.81:0/4019840279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615439c8000 0x561542b0d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.938+0000 7fe9b28bb640 1 -- 172.21.15.81:0/4019840279 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5615439c8400 msgr2=0x561542b0c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.938+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/4019840279 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5615439c8400 0x561542b0c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.938+0000 7fe9b28bb640 1 -- 172.21.15.81:0/4019840279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615439c8000 msgr2=0x561542b0d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:20.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.938+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/4019840279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615439c8000 0x561542b0d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.938+0000 7fe9b28bb640 1 -- 172.21.15.81:0/4019840279 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561542ac84e0 con 0x5615439c8800 2023-11-08T16:31:20.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.938+0000 7fe9b38bd640 1 --2- 172.21.15.81:0/4019840279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615439c8000 0x561542b0d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:20.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.938+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/4019840279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e100 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5615439786f0 tx=0x56154397ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa06e1124b2d5973 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.938+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/4019840279 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561542ae8dc0 con 0x5615439c8800 2023-11-08T16:31:20.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.938+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/4019840279 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561543960d80 con 0x5615439c8800 2023-11-08T16:31:20.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.938+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/4019840279 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561542ae8f20 con 0x5615439c8800 2023-11-08T16:31:20.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.939+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4019840279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 msgr2=0x561542b0e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.939+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/4019840279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e100 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5615439786f0 tx=0x56154397ac00 comp rx=0 tx=0).stop 2023-11-08T16:31:20.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.939+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4019840279 shutdown_connections 2023-11-08T16:31:20.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.939+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/4019840279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e100 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.939+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/4019840279 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5615439c8400 0x561542b0c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.939+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/4019840279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615439c8000 0x561542b0d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.939+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4019840279 >> 172.21.15.81:0/4019840279 conn(0x561542c1c400 msgr2=0x56154381d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:20.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.939+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4019840279 shutdown_connections 2023-11-08T16:31:20.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.939+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4019840279 wait complete. 2023-11-08T16:31:20.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.940+0000 7fe9b504b5c0 1 Processor -- start 2023-11-08T16:31:20.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.940+0000 7fe9b504b5c0 1 -- start start 2023-11-08T16:31:20.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.940+0000 7fe9b504b5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615439c8000 0x561542b0f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.940+0000 7fe9b504b5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5615439c8400 0x561542b0ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.940+0000 7fe9b504b5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b504b5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561542b70c80 con 0x5615439c8800 2023-11-08T16:31:20.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b504b5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561542b70b40 con 0x5615439c8000 2023-11-08T16:31:20.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b504b5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x561542b70a00 con 0x5615439c8400 2023-11-08T16:31:20.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b28bb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b28bb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46020/0 (socket says 172.21.15.81:46020) 2023-11-08T16:31:20.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 learned_addr learned my addr 172.21.15.81:0/1878422677 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:20.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5615439c8400 msgr2=0x561542b0ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5615439c8400 0x561542b0ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615439c8000 msgr2=0x561542b0f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:20.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5615439c8000 0x561542b0f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:20.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561542ac81a0 con 0x5615439c8800 2023-11-08T16:31:20.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.941+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e680 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x5615439799b0 tx=0x561543832180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.942+0000 7fe9ad0b0640 1 -- 172.21.15.81:0/1878422677 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561542ae8420 con 0x5615439c8800 2023-11-08T16:31:20.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.942+0000 7fe9ad0b0640 1 -- 172.21.15.81:0/1878422677 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561543961500 con 0x5615439c8800 2023-11-08T16:31:20.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.942+0000 7fe9ad0b0640 1 -- 172.21.15.81:0/1878422677 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561542ae8dc0 con 0x5615439c8800 2023-11-08T16:31:20.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.942+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561542ac84e0 con 0x5615439c8800 2023-11-08T16:31:20.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.942+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561542ac81a0 con 0x5615439c8800 2023-11-08T16:31:20.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.942+0000 7fe9b504b5c0 4 datascan.init: resolving metadata pool 4 2023-11-08T16:31:20.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.943+0000 7fe9ad0b0640 1 -- 172.21.15.81:0/1878422677 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x561542afe840 con 0x5615439c8800 2023-11-08T16:31:20.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.944+0000 7fe9ad0b0640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x561542dfe000 0x561542b0f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.944+0000 7fe9ad0b0640 1 -- 172.21.15.81:0/1878422677 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x561542b62a80 con 0x5615439c8800 2023-11-08T16:31:20.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.944+0000 7fe9b504b5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:31:20.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.944+0000 7fe9b504b5c0 4 datascan.main: data pool 'cephfs_data' has ID 5 2023-11-08T16:31:20.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.944+0000 7fe9b504b5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:31:20.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.944+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5615439c8c00 0x561542b0e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.944+0000 7fe9b38bd640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x561542dfe000 0x561542b0f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.944+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 4.14 4:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afb880 con 0x5615439c8c00 2023-11-08T16:31:20.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.945+0000 7fe9b30bc640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5615439c8c00 0x561542b0e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.945+0000 7fe9b38bd640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x561542dfe000 0x561542b0f700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x561542b94870 tx=0x56154397bce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.945+0000 7fe9b30bc640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5615439c8c00 0x561542b0e100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.945+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv43 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:20.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.945+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5615439c9000 0x561542b0c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.945+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:2 4.1f 4:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afbc00 con 0x5615439c9000 2023-11-08T16:31:20.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.946+0000 7fe9b38bd640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5615439c9000 0x561542b0c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.947+0000 7fe9b38bd640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5615439c9000 0x561542b0c580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.948+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:20.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.948+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 5.0 5:00000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14000 con 0x5615439c8c00 2023-11-08T16:31:20.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.948+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:20.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.948+0000 7fe9ae0b2640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x561542dfe400 0x561542e1e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.949+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:4 5.20 5:04000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14e00 con 0x561542dfe400 2023-11-08T16:31:20.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.949+0000 7fe9b30bc640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x561542dfe400 0x561542e1e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.950+0000 7fe9b30bc640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x561542dfe400 0x561542e1e000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.951+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.951+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:5 5.10 5:08000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afb880 con 0x561542dfe400 2023-11-08T16:31:20.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.951+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.951+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:6 5.30 5:0c000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14000 con 0x561542dfe400 2023-11-08T16:31:20.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.952+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.952+0000 7fe9ae8b3640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x561542dfe800 0x561542e1e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.952+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:7 5.8 5:10000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15180 con 0x561542dfe800 2023-11-08T16:31:20.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.952+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x561542dfe800 0x561542e1e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.953+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x561542dfe800 0x561542e1e580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.953+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dfe800 2023-11-08T16:31:20.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.953+0000 7fe9ae0b2640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x561542dfec00 0x561542e1eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.953+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:8 5.28 5:14000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14e00 con 0x561542dfec00 2023-11-08T16:31:20.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.954+0000 7fe9b38bd640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x561542dfec00 0x561542e1eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.955+0000 7fe9b38bd640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x561542dfec00 0x561542e1eb00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.956+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:20.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.956+0000 7fe9ae8b3640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x561542dff400 0x561542e1f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.956+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15500 con 0x561542dff400 2023-11-08T16:31:20.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.956+0000 7fe9b30bc640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x561542dff400 0x561542e1f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.957+0000 7fe9b30bc640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x561542dff400 0x561542e1f600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.958+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:20.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.958+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:10 5.38 5:1c000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14e00 con 0x561542dfe400 2023-11-08T16:31:20.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.959+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(10 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.959+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:11 5.4 5:20000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14000 con 0x561542dfe400 2023-11-08T16:31:20.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.960+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(11 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.960+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:12 5.24 5:24000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15880 con 0x561542dfec00 2023-11-08T16:31:20.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.961+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(12 [pgnls-filter start_epoch 40 out=49b] v38'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:20.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.961+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:13 5.14 5:28000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15500 con 0x5615439c8c00 2023-11-08T16:31:20.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.961+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 40 out=49b] v38'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:20.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.961+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:14 5.34 5:2c000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afbc00 con 0x561542dfec00 2023-11-08T16:31:20.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.962+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:20.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.962+0000 7fe9ae8b3640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x561542dff800 0x561542e1fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:20.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.963+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:15 5.c 5:30000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afb880 con 0x561542dff800 2023-11-08T16:31:20.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.963+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x561542dff800 0x561542e1fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:20.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.963+0000 7fe9b28bb640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x561542dff800 0x561542e1fb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:20.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.964+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:20.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.964+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:16 5.2c 5:34000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15180 con 0x561542dfe800 2023-11-08T16:31:20.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.964+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(16 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dfe800 2023-11-08T16:31:20.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.964+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:17 5.1c 5:38000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a000 con 0x561542dff400 2023-11-08T16:31:20.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.965+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:20.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.965+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:18 5.3c 5:3c000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14e00 con 0x561542dff800 2023-11-08T16:31:20.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.966+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:20.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.966+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:19 5.2 5:40000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a380 con 0x5615439c9000 2023-11-08T16:31:20.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.967+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:20.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.967+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:20 5.22 5:44000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15c00 con 0x561542dfe400 2023-11-08T16:31:20.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.968+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(20 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.968+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:21 5.12 5:48000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a700 con 0x561542dff800 2023-11-08T16:31:20.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.968+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 40 out=49b] v38'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:20.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.968+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:22 5.32 5:4c000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afb880 con 0x5615439c8c00 2023-11-08T16:31:20.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.969+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(22 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:20.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.969+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:23 5.a 5:50000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3aa80 con 0x561542dfe800 2023-11-08T16:31:20.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.969+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dfe800 2023-11-08T16:31:20.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.969+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:24 5.2a 5:54000000::::head [pgnls-filter start_epoch 40 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14000 con 0x561542dff400 2023-11-08T16:31:20.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.971+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(24 [pgnls-filter start_epoch 40 out=81b] v40'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:20.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.971+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:25 5.0 5:00000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14380 con 0x5615439c8c00 2023-11-08T16:31:20.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.971+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:20.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.971+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:26 5.20 5:04000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15500 con 0x561542dfe400 2023-11-08T16:31:20.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.972+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(26 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.973+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:27 5.10 5:08000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a000 con 0x561542dfe400 2023-11-08T16:31:20.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.974+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(27 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.974+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:28 5.30 5:0c000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15c00 con 0x561542dfe400 2023-11-08T16:31:20.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.975+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 9 ==== osd_op_reply(28 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.975+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:29 5.8 5:10000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a700 con 0x561542dfe800 2023-11-08T16:31:20.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.975+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dfe800 2023-11-08T16:31:20.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.975+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:30 5.28 5:14000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15180 con 0x561542dfec00 2023-11-08T16:31:20.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.976+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:20.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.977+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:31 5.18 5:18000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afb880 con 0x561542dff400 2023-11-08T16:31:20.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.978+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:20.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.978+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:32 5.38 5:1c000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14e00 con 0x561542dfe400 2023-11-08T16:31:20.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.979+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 10 ==== osd_op_reply(32 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.979+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:33 5.4 5:20000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afbc00 con 0x561542dfe400 2023-11-08T16:31:20.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.980+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 11 ==== osd_op_reply(33 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.980+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:34 5.24 5:24000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a000 con 0x561542dfec00 2023-11-08T16:31:20.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.981+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(34 [pgnls-filter start_epoch 40 out=49b] v38'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:20.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.981+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:35 5.14 5:28000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15880 con 0x5615439c8c00 2023-11-08T16:31:20.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.982+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(35 [pgnls-filter start_epoch 40 out=49b] v38'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:20.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.982+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:36 5.34 5:2c000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afb880 con 0x561542dfec00 2023-11-08T16:31:20.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.983+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(36 [pgnls-filter start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:20.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.983+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:37 5.c 5:30000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a380 con 0x561542dff800 2023-11-08T16:31:20.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.984+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(37 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:20.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.984+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:38 5.2c 5:34000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14000 con 0x561542dfe800 2023-11-08T16:31:20.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.984+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.2 v2:172.21.15.81:6824/3759311428 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dfe800 2023-11-08T16:31:20.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.985+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:39 5.1c 5:38000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3ae00 con 0x561542dff400 2023-11-08T16:31:20.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.986+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:20.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.986+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:40 5.3c 5:3c000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14380 con 0x561542dff800 2023-11-08T16:31:20.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.986+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(40 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:20.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.987+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:41 5.2 5:40000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3b180 con 0x5615439c9000 2023-11-08T16:31:20.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.988+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(41 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:20.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.988+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:42 5.22 5:44000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15500 con 0x561542dfe400 2023-11-08T16:31:20.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.989+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 12 ==== osd_op_reply(42 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:20.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.989+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:43 5.12 5:48000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3b500 con 0x561542dff800 2023-11-08T16:31:20.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.990+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(43 [pgnls-filter start_epoch 40 out=49b] v38'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:20.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.990+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:44 5.32 5:4c000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15c00 con 0x5615439c8c00 2023-11-08T16:31:20.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.990+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(44 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:20.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.990+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:45 5.a 5:50000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3b880 con 0x561542dfe800 2023-11-08T16:31:20.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.991+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.2 v2:172.21.15.81:6824/3759311428 6 ==== osd_op_reply(45 [pgnls-filter start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dfe800 2023-11-08T16:31:20.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.991+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:46 5.2a 5:54000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15180 con 0x561542dff400 2023-11-08T16:31:20.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.992+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(46 [pgnls-filter start_epoch 40 out=81b] v40'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:20.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.992+0000 7fe9b504b5c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-11-08T16:31:20.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.992+0000 7fe9b504b5c0 10 datascan.operator(): handling object 100000001f6.0 2023-11-08T16:31:20.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.992+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:47 5.2a 5:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14700 con 0x561542dff400 2023-11-08T16:31:20.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.993+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(47 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:20.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.993+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:48 6.a 6:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14e00 con 0x561542dfec00 2023-11-08T16:31:20.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.994+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(48 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:20.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.994+0000 7fe9b504b5c0 20 datascan.operator(): loaded layout from xattr: pi: 6 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:31:20.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.994+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-11-08T16:31:20.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.994+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-11-08T16:31:20.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.994+0000 7fe9b504b5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-11-08T16:31:20.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.994+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:49 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15880 con 0x5615439c8c00 2023-11-08T16:31:20.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.995+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(49 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:20.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.995+0000 7fe9b504b5c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-11-08T16:31:20.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.995+0000 7fe9b504b5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/1, will insert to root 2023-11-08T16:31:20.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.995+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:50 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15500 con 0x5615439c8c00 2023-11-08T16:31:20.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.996+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(50 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:20.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.996+0000 7fe9b504b5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:31:20.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.996+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:51 4.10 4:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3ae00 con 0x5615439c8c00 2023-11-08T16:31:21.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.999+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(51 10000000000.00000000 [create,omap-set-header] v40'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.999+0000 7fe9b504b5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-11-08T16:31:21.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:20.999+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:52 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afbc00 con 0x5615439c8c00 2023-11-08T16:31:21.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.000+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.000+0000 7fe9b504b5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-11-08T16:31:21.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.000+0000 7fe9b504b5c0 20 datascan.read_dentry: 1_head not found in result 2023-11-08T16:31:21.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.000+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/1 with size=6291456 bytes 2023-11-08T16:31:21.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.000+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:53 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15c00 con 0x5615439c8c00 2023-11-08T16:31:21.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.004+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(53 10000000000.00000000 [omap-set-vals] v40'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.004+0000 7fe9b504b5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-11-08T16:31:21.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.004+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-11-08T16:31:21.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.004+0000 7fe9b504b5c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-11-08T16:31:21.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.004+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:54 4.1f 4:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15180 con 0x5615439c9000 2023-11-08T16:31:21.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.005+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(54 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:21.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.005+0000 7fe9b504b5c0 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-11-08T16:31:21.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.005+0000 7fe9b504b5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-11-08T16:31:21.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.005+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:55 4.1f 4:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14000 con 0x5615439c9000 2023-11-08T16:31:21.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.006+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(55 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:21.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.006+0000 7fe9b504b5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:31:21.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.006+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:56 4.1f 4:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8000 con 0x5615439c9000 2023-11-08T16:31:21.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.010+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(56 1.00000000 [create,omap-set-header] v40'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:21.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.011+0000 7fe9b504b5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-11-08T16:31:21.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.011+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:57 4.1f 4:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8380 con 0x5615439c9000 2023-11-08T16:31:21.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.012+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(57 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:21.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.012+0000 7fe9b504b5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:31:21.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.012+0000 7fe9b504b5c0 20 datascan.read_dentry: subdir_head not found in result 2023-11-08T16:31:21.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.012+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-11-08T16:31:21.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.012+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:58 4.1f 4:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8700 con 0x5615439c9000 2023-11-08T16:31:21.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.017+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(58 1.00000000 [omap-set-vals] v40'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:21.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.017+0000 7fe9b504b5c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-11-08T16:31:21.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.017+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:59 5.1a 5:58000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8a80 con 0x561542dfe400 2023-11-08T16:31:21.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.019+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 13 ==== osd_op_reply(59 [pgnls-filter start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:21.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.019+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:60 5.3a 5:5c000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a000 con 0x5615439c8c00 2023-11-08T16:31:21.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.020+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(60 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.020+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:61 5.6 5:60000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3bc00 con 0x561542dfe800 2023-11-08T16:31:21.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.021+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.2 v2:172.21.15.81:6824/3759311428 7 ==== osd_op_reply(61 [pgnls-filter start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dfe800 2023-11-08T16:31:21.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.021+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:62 5.26 5:64000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3b180 con 0x5615439c8c00 2023-11-08T16:31:21.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.022+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(62 [pgnls-filter start_epoch 40 out=49b] v38'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.022+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:63 5.16 5:68000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14a80 con 0x561542dfec00 2023-11-08T16:31:21.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.023+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(63 [pgnls-filter start_epoch 40 out=49b] v38'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:21.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.023+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:64 5.36 5:6c000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3aa80 con 0x561542dff800 2023-11-08T16:31:21.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.024+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(64 [pgnls-filter start_epoch 40 out=101b] v40'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:21.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.024+0000 7fe9b504b5c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-11-08T16:31:21.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.024+0000 7fe9b504b5c0 10 datascan.operator(): handling object 100000001f9.0 2023-11-08T16:31:21.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.024+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:65 5.36 5:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8e00 con 0x561542dff800 2023-11-08T16:31:21.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.026+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(65 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:21.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.026+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561542e4a400 0x561542b0d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.026+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:66 6.16 6:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b9500 con 0x561542e4a400 2023-11-08T16:31:21.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.027+0000 7fe9b38bd640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561542e4a400 0x561542b0d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.028+0000 7fe9b38bd640 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561542e4a400 0x561542b0d600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.029+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(66 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x561543950d80 con 0x561542e4a400 2023-11-08T16:31:21.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.029+0000 7fe9b504b5c0 20 datascan.operator(): loaded layout from xattr: pi: 6 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.029+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-11-08T16:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.029+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-11-08T16:31:21.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.029+0000 7fe9b504b5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-11-08T16:31:21.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.029+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:67 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14380 con 0x5615439c8c00 2023-11-08T16:31:21.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.030+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(67 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.030+0000 7fe9b504b5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:31:21.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.030+0000 7fe9b504b5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:31:21.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.030+0000 7fe9b504b5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/4, will insert to root 2023-11-08T16:31:21.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.030+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:68 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14e00 con 0x5615439c8c00 2023-11-08T16:31:21.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.031+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(68 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.031+0000 7fe9b504b5c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-11-08T16:31:21.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.031+0000 7fe9b504b5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:31:21.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.031+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:69 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15180 con 0x5615439c8c00 2023-11-08T16:31:21.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.032+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(69 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.032+0000 7fe9b504b5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-11-08T16:31:21.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.032+0000 7fe9b504b5c0 20 datascan.read_dentry: 4_head not found in result 2023-11-08T16:31:21.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.032+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/4 with size=6291456 bytes 2023-11-08T16:31:21.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.032+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:70 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14000 con 0x5615439c8c00 2023-11-08T16:31:21.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.036+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 18 ==== osd_op_reply(70 10000000000.00000000 [omap-set-vals] v40'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.036+0000 7fe9b504b5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-11-08T16:31:21.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.036+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:71 5.36 5:6fe38dc2::::head [pgnls-filter start_epoch 40 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3b500 con 0x561542dff800 2023-11-08T16:31:21.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.037+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(71 [pgnls-filter start_epoch 40 out=81b] v40'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:21.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.038+0000 7fe9b504b5c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-11-08T16:31:21.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.038+0000 7fe9b504b5c0 10 datascan.operator(): handling object 100000001f7.0 2023-11-08T16:31:21.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.038+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:72 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3b880 con 0x561542dff800 2023-11-08T16:31:21.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.039+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(72 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:21.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.039+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:73 6.16 6:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3aa80 con 0x561542e4a400 2023-11-08T16:31:21.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.040+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(73 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x561543950d80 con 0x561542e4a400 2023-11-08T16:31:21.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.040+0000 7fe9b504b5c0 20 datascan.operator(): loaded layout from xattr: pi: 6 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:31:21.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.040+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-11-08T16:31:21.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.040+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-11-08T16:31:21.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.040+0000 7fe9b504b5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-11-08T16:31:21.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.040+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:74 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8a80 con 0x5615439c8c00 2023-11-08T16:31:21.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.041+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 19 ==== osd_op_reply(74 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.041+0000 7fe9b504b5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:31:21.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.041+0000 7fe9b504b5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:31:21.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.041+0000 7fe9b504b5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/2, will insert to root 2023-11-08T16:31:21.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.041+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:75 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8000 con 0x5615439c8c00 2023-11-08T16:31:21.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.042+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 20 ==== osd_op_reply(75 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.042+0000 7fe9b504b5c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-11-08T16:31:21.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.042+0000 7fe9b504b5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:31:21.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.042+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:76 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8380 con 0x5615439c8c00 2023-11-08T16:31:21.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.043+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 21 ==== osd_op_reply(76 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.043+0000 7fe9b504b5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-11-08T16:31:21.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.043+0000 7fe9b504b5c0 20 datascan.read_dentry: 2_head not found in result 2023-11-08T16:31:21.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.043+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/2 with size=6291456 bytes 2023-11-08T16:31:21.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.043+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:77 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b9880 con 0x5615439c8c00 2023-11-08T16:31:21.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.047+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 22 ==== osd_op_reply(77 10000000000.00000000 [omap-set-vals] v40'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.047+0000 7fe9b504b5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-11-08T16:31:21.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.047+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:78 5.e 5:70000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b9c00 con 0x561542dff400 2023-11-08T16:31:21.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.048+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(78 [pgnls-filter start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:21.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.048+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:79 5.2e 5:74000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14700 con 0x5615439c8c00 2023-11-08T16:31:21.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.049+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 23 ==== osd_op_reply(79 [pgnls-filter start_epoch 40 out=49b] v38'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.049+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:80 5.1e 5:78000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a700 con 0x561542e4a400 2023-11-08T16:31:21.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.051+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(80 [pgnls-filter start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542e4a400 2023-11-08T16:31:21.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.051+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:81 5.3e 5:7c000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15880 con 0x5615439c8c00 2023-11-08T16:31:21.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.052+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 24 ==== osd_op_reply(81 [pgnls-filter start_epoch 40 out=49b] v38'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.052+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:82 5.1 5:80000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a380 con 0x561542dff400 2023-11-08T16:31:21.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.053+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(82 [pgnls-filter start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:21.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.053+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:83 5.21 5:84000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15500 con 0x5615439c9000 2023-11-08T16:31:21.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.054+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 40 out=49b] v38'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:21.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.054+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:84 5.11 5:88000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afb880 con 0x5615439c9000 2023-11-08T16:31:21.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.055+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(84 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:21.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.056+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:85 5.31 5:8c000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15c00 con 0x561542dfec00 2023-11-08T16:31:21.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.057+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 9 ==== osd_op_reply(85 [pgnls-filter start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:21.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.057+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:86 5.9 5:90000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14a80 con 0x561542dfe400 2023-11-08T16:31:21.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.058+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 14 ==== osd_op_reply(86 [pgnls-filter start_epoch 40 out=49b] v38'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:21.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.058+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:87 5.29 5:94000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542afbc00 con 0x561542e4a400 2023-11-08T16:31:21.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.059+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(87 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542e4a400 2023-11-08T16:31:21.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.059+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:88 5.19 5:98000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15500 con 0x561542dff800 2023-11-08T16:31:21.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.060+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(88 [pgnls-filter start_epoch 40 out=81b] v40'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:21.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.060+0000 7fe9b504b5c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-11-08T16:31:21.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.060+0000 7fe9b504b5c0 10 datascan.operator(): handling object 100000001f8.0 2023-11-08T16:31:21.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.060+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:89 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b9180 con 0x561542dff800 2023-11-08T16:31:21.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.061+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 12 ==== osd_op_reply(89 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:21.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.061+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:90 6.19 6:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8700 con 0x561542dfec00 2023-11-08T16:31:21.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.063+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 10 ==== osd_op_reply(90 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:21.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.063+0000 7fe9b504b5c0 20 datascan.operator(): loaded layout from xattr: pi: 6 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:31:21.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.063+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-11-08T16:31:21.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.063+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-11-08T16:31:21.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.063+0000 7fe9b504b5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-11-08T16:31:21.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.063+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:91 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b9500 con 0x5615439c8c00 2023-11-08T16:31:21.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.064+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 25 ==== osd_op_reply(91 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.064+0000 7fe9b504b5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:31:21.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.064+0000 7fe9b504b5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:31:21.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.064+0000 7fe9b504b5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/3, will insert to root 2023-11-08T16:31:21.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.064+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:92 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15500 con 0x5615439c8c00 2023-11-08T16:31:21.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.065+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 26 ==== osd_op_reply(92 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.065+0000 7fe9b504b5c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-11-08T16:31:21.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.065+0000 7fe9b504b5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:31:21.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.065+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:93 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15180 con 0x5615439c8c00 2023-11-08T16:31:21.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.066+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 27 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.066+0000 7fe9b504b5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-11-08T16:31:21.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.066+0000 7fe9b504b5c0 20 datascan.read_dentry: 3_head not found in result 2023-11-08T16:31:21.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.066+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/3 with size=6291456 bytes 2023-11-08T16:31:21.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.066+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:94 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14000 con 0x5615439c8c00 2023-11-08T16:31:21.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.069+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 28 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v40'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.069+0000 7fe9b504b5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-11-08T16:31:21.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.069+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:95 5.39 5:9c000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3b500 con 0x561542dfe400 2023-11-08T16:31:21.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.070+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 15 ==== osd_op_reply(95 [pgnls-filter start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:21.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.070+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:96 5.5 5:a0000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14380 con 0x561542e4a400 2023-11-08T16:31:21.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.071+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(96 [pgnls-filter start_epoch 40 out=49b] v38'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542e4a400 2023-11-08T16:31:21.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.071+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:97 5.25 5:a4000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3ae00 con 0x561542dff800 2023-11-08T16:31:21.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.072+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 13 ==== osd_op_reply(97 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:21.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.072+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:98 5.15 5:a8000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14e00 con 0x561542dfec00 2023-11-08T16:31:21.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.073+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 11 ==== osd_op_reply(98 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:21.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.074+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:99 5.35 5:ac000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b9880 con 0x561542dfe400 2023-11-08T16:31:21.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.075+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 16 ==== osd_op_reply(99 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:21.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.075+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:100 5.d 5:b0000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3aa80 con 0x561542dfe800 2023-11-08T16:31:21.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.076+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.2 v2:172.21.15.81:6824/3759311428 8 ==== osd_op_reply(100 [pgnls-filter start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dfe800 2023-11-08T16:31:21.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.076+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:101 5.2d 5:b4000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14700 con 0x5615439c8c00 2023-11-08T16:31:21.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.077+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 29 ==== osd_op_reply(101 [pgnls-filter start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.077+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:102 5.1d 5:b8000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a700 con 0x561542dff800 2023-11-08T16:31:21.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.078+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 14 ==== osd_op_reply(102 [pgnls-filter start_epoch 40 out=49b] v38'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:21.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.078+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:103 5.3d 5:bc000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15880 con 0x561542dfec00 2023-11-08T16:31:21.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.079+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 12 ==== osd_op_reply(103 [pgnls-filter start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:21.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.079+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:104 5.3 5:c0000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a000 con 0x561542e4a400 2023-11-08T16:31:21.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.080+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(104 [pgnls-filter start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542e4a400 2023-11-08T16:31:21.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.080+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:105 5.23 5:c4000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14a80 con 0x561542dff400 2023-11-08T16:31:21.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.081+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(105 [pgnls-filter start_epoch 40 out=49b] v38'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:21.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.082+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:106 5.13 5:c8000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3b180 con 0x5615439c8c00 2023-11-08T16:31:21.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.082+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 30 ==== osd_op_reply(106 [pgnls-filter start_epoch 40 out=49b] v38'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.083+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:107 5.33 5:cc000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15500 con 0x561542dfe800 2023-11-08T16:31:21.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.084+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.2 v2:172.21.15.81:6824/3759311428 9 ==== osd_op_reply(107 [pgnls-filter start_epoch 40 out=49b] v38'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dfe800 2023-11-08T16:31:21.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.084+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:108 5.b 5:d0000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8700 con 0x561542dfe800 2023-11-08T16:31:21.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.085+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.2 v2:172.21.15.81:6824/3759311428 10 ==== osd_op_reply(108 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dfe800 2023-11-08T16:31:21.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.085+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:109 5.2b 5:d4000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15180 con 0x561542dff400 2023-11-08T16:31:21.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.086+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(109 [pgnls-filter start_epoch 40 out=49b] v38'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:21.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.086+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:110 5.1b 5:d8000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b9c00 con 0x561542dfec00 2023-11-08T16:31:21.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.087+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 13 ==== osd_op_reply(110 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:21.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.087+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:111 5.3b 5:dc000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14000 con 0x561542dfe800 2023-11-08T16:31:21.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.088+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.2 v2:172.21.15.81:6824/3759311428 11 ==== osd_op_reply(111 [pgnls-filter start_epoch 40 out=49b] v38'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dfe800 2023-11-08T16:31:21.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.088+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:112 5.7 5:e0000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b9500 con 0x561542dfec00 2023-11-08T16:31:21.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.089+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 14 ==== osd_op_reply(112 [pgnls-filter start_epoch 40 out=81b] v40'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:21.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.090+0000 7fe9b504b5c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-11-08T16:31:21.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.090+0000 7fe9b504b5c0 10 datascan.operator(): handling object 10000000001.0 2023-11-08T16:31:21.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.090+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:113 5.7 5:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3bc00 con 0x561542dfec00 2023-11-08T16:31:21.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.091+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 15 ==== osd_op_reply(113 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:21.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.091+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:114 6.7 6:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8e00 con 0x561542dfec00 2023-11-08T16:31:21.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.092+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 16 ==== osd_op_reply(114 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:21.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.092+0000 7fe9b504b5c0 20 datascan.operator(): loaded layout from xattr: pi: 6 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:31:21.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.092+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-11-08T16:31:21.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.092+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-11-08T16:31:21.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.092+0000 7fe9b504b5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-11-08T16:31:21.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.092+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:115 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b9180 con 0x5615439c8c00 2023-11-08T16:31:21.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.093+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 31 ==== osd_op_reply(115 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.093+0000 7fe9b504b5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:31:21.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.093+0000 7fe9b504b5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:31:21.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.093+0000 7fe9b504b5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/0, will insert to root 2023-11-08T16:31:21.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.093+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:116 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8700 con 0x5615439c8c00 2023-11-08T16:31:21.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.094+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 32 ==== osd_op_reply(116 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.094+0000 7fe9b504b5c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-11-08T16:31:21.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.094+0000 7fe9b504b5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:31:21.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.094+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:117 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8a80 con 0x5615439c8c00 2023-11-08T16:31:21.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.095+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 33 ==== osd_op_reply(117 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.095+0000 7fe9b504b5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-11-08T16:31:21.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.095+0000 7fe9b504b5c0 20 datascan.read_dentry: 0_head not found in result 2023-11-08T16:31:21.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.095+0000 7fe9b504b5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/0 with size=6291456 bytes 2023-11-08T16:31:21.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.095+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:118 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8000 con 0x5615439c8c00 2023-11-08T16:31:21.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.099+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 34 ==== osd_op_reply(118 10000000000.00000000 [omap-set-vals] v40'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.099+0000 7fe9b504b5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-11-08T16:31:21.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.099+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:119 5.27 5:e4000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x5615438b8380 con 0x561542dff800 2023-11-08T16:31:21.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.100+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 <== osd.1 v2:172.21.15.81:6800/2060798739 15 ==== osd_op_reply(119 [pgnls-filter start_epoch 40 out=49b] v38'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950fc0 con 0x561542dff800 2023-11-08T16:31:21.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.100+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:120 5.17 5:e8000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e15c00 con 0x5615439c8c00 2023-11-08T16:31:21.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.101+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.3 v2:172.21.15.81:6816/2756229107 35 ==== osd_op_reply(120 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x5615439c8c00 2023-11-08T16:31:21.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.101+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:121 5.37 5:ec000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e3a380 con 0x561542dff400 2023-11-08T16:31:21.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.102+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(121 [pgnls-filter start_epoch 40 out=49b] v38'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dff400 2023-11-08T16:31:21.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.102+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:122 5.f 5:f0000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14380 con 0x561542dfec00 2023-11-08T16:31:21.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.103+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.5 v2:172.21.15.93:6802/2406579932 17 ==== osd_op_reply(122 [pgnls-filter start_epoch 40 out=49b] v38'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x561542dfec00 2023-11-08T16:31:21.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.103+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:123 5.2f 5:f4000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14700 con 0x561542dfe400 2023-11-08T16:31:21.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.104+0000 7fe9b30bc640 1 -- 172.21.15.81:0/1878422677 <== osd.7 v2:172.21.15.93:6818/433691658 17 ==== osd_op_reply(123 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950b40 con 0x561542dfe400 2023-11-08T16:31:21.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.105+0000 7fe9ae0b2640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:124 5.1f 5:f8000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14e00 con 0x5615439c9000 2023-11-08T16:31:21.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.106+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 11 ==== osd_op_reply(124 [pgnls-filter start_epoch 40 out=49b] v38'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:21.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.106+0000 7fe9ae8b3640 1 -- 172.21.15.81:0/1878422677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:125 5.3f 5:fc000000::::head [pgnls-filter start_epoch 40 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x561542e14a80 con 0x5615439c9000 2023-11-08T16:31:21.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b38bd640 1 -- 172.21.15.81:0/1878422677 <== osd.6 v2:172.21.15.93:6816/607960134 12 ==== osd_op_reply(125 [pgnls-filter start_epoch 40 out=49b] v38'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x561543950d80 con 0x5615439c9000 2023-11-08T16:31:21.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561542e4a400 msgr2=0x561542b0d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561542e4a400 0x561542b0d600 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x561542dff800 msgr2=0x561542e1fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x561542dff800 0x561542e1fb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x561542dfe800 msgr2=0x561542e1e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x561542dfe800 0x561542e1e580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5615439c8c00 msgr2=0x561542b0e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5615439c8c00 0x561542b0e100 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x561542dff400 msgr2=0x561542e1f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x561542dff400 0x561542e1f600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x561542dfec00 msgr2=0x561542e1eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x561542dfec00 0x561542e1eb00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5615439c9000 msgr2=0x561542b0c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5615439c9000 0x561542b0c580 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x561542dfe400 msgr2=0x561542e1e000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x561542dfe400 0x561542e1e000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.107+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 reap_dead start 2023-11-08T16:31:21.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.108+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 reap_dead start 2023-11-08T16:31:21.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.108+0000 7fe9b28bb640 1 -- 172.21.15.81:0/1878422677 reap_dead start 2023-11-08T16:31:21.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.108+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x561542dfe000 msgr2=0x561542b0f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.108+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x561542dfe000 0x561542b0f700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x561542b94870 tx=0x56154397bce0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.108+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 msgr2=0x561542b0e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.108+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e680 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x5615439799b0 tx=0x561543832180 comp rx=0 tx=0).stop 2023-11-08T16:31:21.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.108+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 shutdown_connections 2023-11-08T16:31:21.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.108+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x561542dfe000 0x561542b0f700 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.108+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/1878422677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5615439c8800 0x561542b0e680 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.108+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 >> 172.21.15.81:0/1878422677 conn(0x561542c1c400 msgr2=0x56154381d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:21.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.108+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 shutdown_connections 2023-11-08T16:31:21.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.109+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/1878422677 wait complete. 2023-11-08T16:31:21.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.109+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4068859666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561542c1c000 msgr2=0x561542b0d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.110+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/4068859666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561542c1c000 0x561542b0d080 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x5615438b2cf0 tx=0x561543956120 comp rx=0 tx=0).stop 2023-11-08T16:31:21.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.110+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4068859666 shutdown_connections 2023-11-08T16:31:21.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.110+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/4068859666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561542c1c800 0x561542b0db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.110+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/4068859666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561542aa3c00 0x561542b0cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.110+0000 7fe9b504b5c0 1 --2- 172.21.15.81:0/4068859666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561542c1c000 0x561542b0d080 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.110+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4068859666 >> 172.21.15.81:0/4068859666 conn(0x561542aa3000 msgr2=0x56154381e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:21.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.110+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4068859666 shutdown_connections 2023-11-08T16:31:21.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.110+0000 7fe9b504b5c0 1 -- 172.21.15.81:0/4068859666 wait complete. 2023-11-08T16:31:21.124 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.291728, result: 2023-11-08T16:31:21.124 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-11-08T16:31:21.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.230+0000 7f27dfa0d5c0 1 -- start start 2023-11-08T16:31:21.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.230+0000 7f27dfa0d5c0 1 Processor -- start 2023-11-08T16:31:21.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dfa0d5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69bbec00 0x556c69c28580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dfa0d5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69bbf800 0x556c69c28b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dfa0d5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69bbfc00 0x556c69c29080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dfa0d5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c69c07180 con 0x556c69bbfc00 2023-11-08T16:31:21.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dfa0d5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c69c8d180 con 0x556c69bbec00 2023-11-08T16:31:21.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dfa0d5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c69c8cdc0 con 0x556c69bbf800 2023-11-08T16:31:21.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dddcb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69bbfc00 0x556c69c29080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dddcb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69bbfc00 0x556c69c29080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46044/0 (socket says 172.21.15.81:46044) 2023-11-08T16:31:21.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dddcb640 1 -- 172.21.15.81:0/563855180 learned_addr learned my addr 172.21.15.81:0/563855180 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:21.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dddcb640 1 -- 172.21.15.81:0/563855180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69bbf800 msgr2=0x556c69c28b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:21.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dddcb640 1 --2- 172.21.15.81:0/563855180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69bbf800 0x556c69c28b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dddcb640 1 -- 172.21.15.81:0/563855180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69bbec00 msgr2=0x556c69c28580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:21.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dddcb640 1 --2- 172.21.15.81:0/563855180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69bbec00 0x556c69c28580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.231+0000 7f27dddcb640 1 -- 172.21.15.81:0/563855180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c69be49c0 con 0x556c69bbfc00 2023-11-08T16:31:21.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.232+0000 7f27dddcb640 1 --2- 172.21.15.81:0/563855180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69bbfc00 0x556c69c29080 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x556c6a9c5440 tx=0x556c69ea2120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de5209be3eca801 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.232+0000 7f27dadc5640 1 -- 172.21.15.81:0/563855180 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c69c051e0 con 0x556c69bbfc00 2023-11-08T16:31:21.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.232+0000 7f27dadc5640 1 -- 172.21.15.81:0/563855180 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c69c18180 con 0x556c69bbfc00 2023-11-08T16:31:21.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.232+0000 7f27dadc5640 1 -- 172.21.15.81:0/563855180 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c69c05080 con 0x556c69bbfc00 2023-11-08T16:31:21.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.232+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/563855180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c69be4b60 con 0x556c69bbfc00 2023-11-08T16:31:21.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.232+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/563855180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x556c69be4820 con 0x556c69bbfc00 2023-11-08T16:31:21.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.232+0000 7f27dfa0d5c0 4 waiting for MDS map... 2023-11-08T16:31:21.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.232+0000 7f27dadc5640 1 -- 172.21.15.81:0/563855180 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x556c69c7ec40 con 0x556c69bbfc00 2023-11-08T16:31:21.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.232+0000 7f27dadc5640 1 -- 172.21.15.81:0/563855180 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 25) v1 ==== 3306+0+0 (secure 0 0 0) 0x556c69c0ef00 con 0x556c69bbfc00 2023-11-08T16:31:21.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.232+0000 7f27dfa0d5c0 4 Got MDS map 25 2023-11-08T16:31:21.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.233+0000 7f27dfa0d5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:31:21.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.233+0000 7f27dfa0d5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:31:21.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.233+0000 7f27dfa0d5c0 1 Processor -- start 2023-11-08T16:31:21.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.233+0000 7f27dfa0d5c0 1 -- start start 2023-11-08T16:31:21.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.234+0000 7f27dfa0d5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69eb1000 0x556c69c29600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.234+0000 7f27dfa0d5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 0x556c69c29b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.234+0000 7f27dfa0d5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69eb1800 0x556c69c2a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.234+0000 7f27dfa0d5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c69c06140 con 0x556c69eb1400 2023-11-08T16:31:21.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.234+0000 7f27dfa0d5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c69c06b40 con 0x556c69eb1000 2023-11-08T16:31:21.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.234+0000 7f27dfa0d5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c69c8d180 con 0x556c69eb1800 2023-11-08T16:31:21.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.234+0000 7f27dd5ca640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 0x556c69c29b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.235+0000 7f27dd5ca640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 0x556c69c29b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46048/0 (socket says 172.21.15.81:46048) 2023-11-08T16:31:21.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.235+0000 7f27dd5ca640 1 -- 172.21.15.81:0/284260841 learned_addr learned my addr 172.21.15.81:0/284260841 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:21.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.235+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/284260841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69eb1000 0x556c69c29600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.235+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/284260841 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69eb1800 0x556c69c2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.235+0000 7f27dd5ca640 1 -- 172.21.15.81:0/284260841 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69eb1800 msgr2=0x556c69c2a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:21.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.235+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/284260841 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69eb1800 0x556c69c2a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.235+0000 7f27dd5ca640 1 -- 172.21.15.81:0/284260841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69eb1000 msgr2=0x556c69c29600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:21.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.235+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/284260841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69eb1000 0x556c69c29600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.235+0000 7f27dd5ca640 1 -- 172.21.15.81:0/284260841 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c69be44e0 con 0x556c69eb1400 2023-11-08T16:31:21.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.235+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/284260841 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69eb1800 0x556c69c2a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:21.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.235+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/284260841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 0x556c69c29b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x556c69ec4690 tx=0x556c69ed3020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7debfd1428ec2eb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27d8dc1640 1 -- 172.21.15.81:0/284260841 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c69c051e0 con 0x556c69eb1400 2023-11-08T16:31:21.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27d8dc1640 1 -- 172.21.15.81:0/284260841 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c69c18180 con 0x556c69eb1400 2023-11-08T16:31:21.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27d8dc1640 1 -- 172.21.15.81:0/284260841 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c69c05080 con 0x556c69eb1400 2023-11-08T16:31:21.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/284260841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 msgr2=0x556c69c29b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/284260841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 0x556c69c29b80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x556c69ec4690 tx=0x556c69ed3020 comp rx=0 tx=0).stop 2023-11-08T16:31:21.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/284260841 shutdown_connections 2023-11-08T16:31:21.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/284260841 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69eb1800 0x556c69c2a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/284260841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 0x556c69c29b80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/284260841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69eb1000 0x556c69c29600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/284260841 >> 172.21.15.81:0/284260841 conn(0x556c69eb0c00 msgr2=0x556c6a938800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:21.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/284260841 shutdown_connections 2023-11-08T16:31:21.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.236+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/284260841 wait complete. 2023-11-08T16:31:21.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.237+0000 7f27dfa0d5c0 1 Processor -- start 2023-11-08T16:31:21.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.237+0000 7f27dfa0d5c0 1 -- start start 2023-11-08T16:31:21.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.238+0000 7f27dfa0d5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69eb1000 0x556c69c2b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.238+0000 7f27dfa0d5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 0x556c69c2ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.238+0000 7f27dfa0d5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69eb1800 0x556c69c2a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.238+0000 7f27dfa0d5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c69c8cc80 con 0x556c69eb1400 2023-11-08T16:31:21.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.238+0000 7f27dfa0d5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c69c8cb40 con 0x556c69eb1800 2023-11-08T16:31:21.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.238+0000 7f27dfa0d5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x556c69c8ca00 con 0x556c69eb1000 2023-11-08T16:31:21.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.238+0000 7f27dd5ca640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 0x556c69c2ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.238+0000 7f27dd5ca640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 0x556c69c2ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46058/0 (socket says 172.21.15.81:46058) 2023-11-08T16:31:21.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.238+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 learned_addr learned my addr 172.21.15.81:0/1230958031 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:21.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69eb1000 0x556c69c2b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69eb1000 msgr2=0x556c69c2b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69eb1000 0x556c69c2b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69eb1800 0x556c69c2a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69eb1800 msgr2=0x556c69c2a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69eb1800 0x556c69c2a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c69be49c0 con 0x556c69eb1400 2023-11-08T16:31:21.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 0x556c69c2ac00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x556c69ec5350 tx=0x556c69ed2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27d75be640 1 -- 172.21.15.81:0/1230958031 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c69c04420 con 0x556c69eb1400 2023-11-08T16:31:21.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27d75be640 1 -- 172.21.15.81:0/1230958031 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556c6a992c00 con 0x556c69eb1400 2023-11-08T16:31:21.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27d75be640 1 -- 172.21.15.81:0/1230958031 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x556c69c051e0 con 0x556c69eb1400 2023-11-08T16:31:21.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556c69be4b60 con 0x556c69eb1400 2023-11-08T16:31:21.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c69be4820 con 0x556c69eb1400 2023-11-08T16:31:21.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.239+0000 7f27dfa0d5c0 4 datascan.init: resolving metadata pool 4 2023-11-08T16:31:21.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.241+0000 7f27d75be640 1 -- 172.21.15.81:0/1230958031 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x556c69c1a840 con 0x556c69eb1400 2023-11-08T16:31:21.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.242+0000 7f27d75be640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x556c69f2e800 0x556c69c2b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.242+0000 7f27d75be640 1 -- 172.21.15.81:0/1230958031 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x556c69c7ea80 con 0x556c69eb1400 2023-11-08T16:31:21.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.242+0000 7f27dfa0d5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:31:21.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.242+0000 7f27dfa0d5c0 4 datascan.main: resolving metadata pool 4 2023-11-08T16:31:21.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.242+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x556c69f2ec00 0x556c69c2a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.242+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 4.0 4:00000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69f2ec00 2023-11-08T16:31:21.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.242+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x556c69f2e800 0x556c69c2b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.242+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x556c69f2ec00 0x556c69c2a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.243+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x556c69f2e800 0x556c69c2b700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x556c6a9c4930 tx=0x556c69ed2f00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.243+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x556c69f2ec00 0x556c69c2a100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.243+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.243+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:2 4.10 4:08000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69f2ec00 2023-11-08T16:31:21.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.244+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(2 [pgnls start_epoch 40 out=81b] v40'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.244+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x556c6a995000 0x556c6a038000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.244+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:3 4.8 4:10000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a995000 2023-11-08T16:31:21.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.244+0000 7f27dddcb640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x556c6a995000 0x556c6a038000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.245+0000 7f27dddcb640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x556c6a995000 0x556c6a038000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.246+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(3 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38d80 con 0x556c6a995000 2023-11-08T16:31:21.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.246+0000 7f27dddcb640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x556c69eb0400 0x556c6a038580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.246+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:4 4.18 4:18000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69eb0400 2023-11-08T16:31:21.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.247+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x556c69eb0400 0x556c6a038580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.248+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x556c69eb0400 0x556c6a038580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.249+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(4 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.249+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x556c6a048000 0x556c6a038b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.249+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:5 4.4 4:20000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a048000 2023-11-08T16:31:21.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.249+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x556c6a048000 0x556c6a038b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.249+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x556c6a048000 0x556c6a038b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.250+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(5 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048000 2023-11-08T16:31:21.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.250+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:6 4.14 4:28000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69f2ec00 2023-11-08T16:31:21.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.250+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(6 [pgnls start_epoch 40 out=125b] v40'43 uv43 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.250+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:7 4.c 4:30000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69eb0400 2023-11-08T16:31:21.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.251+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(7 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.251+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x556c6a048400 0x556c6a039080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.251+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:8 4.1c 4:38000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a048400 2023-11-08T16:31:21.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.251+0000 7f27dddcb640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x556c6a048400 0x556c6a039080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.252+0000 7f27dddcb640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x556c6a048400 0x556c6a039080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.252+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(8 [pgnls start_epoch 40 out=49b] v39'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38d80 con 0x556c6a048400 2023-11-08T16:31:21.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.252+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:9 4.2 4:40000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a048000 2023-11-08T16:31:21.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.253+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(9 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048000 2023-11-08T16:31:21.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.253+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:10 4.12 4:48000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a048000 2023-11-08T16:31:21.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.253+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(10 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048000 2023-11-08T16:31:21.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.253+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x556c6a995400 0x556c6a039b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.253+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:11 4.a 4:50000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a995400 2023-11-08T16:31:21.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.254+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x556c6a995400 0x556c6a039b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.255+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x556c6a995400 0x556c6a039b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.256+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(11 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c6a995400 2023-11-08T16:31:21.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.256+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:12 4.1a 4:58000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69eb0400 2023-11-08T16:31:21.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.257+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(12 [pgnls start_epoch 40 out=49b] v40'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.257+0000 7f27dd5ca640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x556c6a048800 0x556c6a039600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.257+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:13 4.6 4:60000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a048800 2023-11-08T16:31:21.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.257+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x556c6a048800 0x556c6a039600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.258+0000 7f27dcdc9640 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x556c6a048800 0x556c6a039600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.259+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(13 [pgnls start_epoch 40 out=49b] v39'9 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048800 2023-11-08T16:31:21.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.259+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:14 4.16 4:68000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a048800 2023-11-08T16:31:21.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.259+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(14 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048800 2023-11-08T16:31:21.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.259+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:15 4.e 4:70000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69eb0400 2023-11-08T16:31:21.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.260+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(15 [pgnls start_epoch 40 out=49b] v39'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.260+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:16 4.1e 4:78000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a048800 2023-11-08T16:31:21.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.260+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(16 [pgnls start_epoch 40 out=49b] v40'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048800 2023-11-08T16:31:21.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.260+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:17 4.1 4:80000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69eb0400 2023-11-08T16:31:21.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.261+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(17 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.261+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:18 4.11 4:88000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69f2ec00 2023-11-08T16:31:21.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.262+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(18 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.262+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:19 4.9 4:90000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69eb0400 2023-11-08T16:31:21.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.263+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(19 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.263+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:20 4.19 4:98000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69f2ec00 2023-11-08T16:31:21.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.263+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(20 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.263+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:21 4.5 4:a0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a995400 2023-11-08T16:31:21.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.264+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(21 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c6a995400 2023-11-08T16:31:21.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.265+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:22 4.15 4:a8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a995000 2023-11-08T16:31:21.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.265+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(22 [pgnls start_epoch 40 out=73b] v39'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x556c69d38d80 con 0x556c6a995000 2023-11-08T16:31:21.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.266+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:23 4.d 4:b0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69eb0400 2023-11-08T16:31:21.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.267+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(23 [pgnls start_epoch 40 out=74b] v40'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.267+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:24 4.1d 4:b8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69f2ec00 2023-11-08T16:31:21.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.267+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(24 [pgnls start_epoch 40 out=49b] v39'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.267+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:25 4.3 4:c0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a048800 2023-11-08T16:31:21.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.268+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(25 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048800 2023-11-08T16:31:21.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.268+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:26 4.13 4:c8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69eb0400 2023-11-08T16:31:21.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.269+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(26 [pgnls start_epoch 40 out=98b] v40'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.269+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:27 4.b 4:d0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a048800 2023-11-08T16:31:21.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.270+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(27 [pgnls start_epoch 40 out=76b] v40'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048800 2023-11-08T16:31:21.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.270+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:28 4.1b 4:d8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69eb0400 2023-11-08T16:31:21.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.271+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(28 [pgnls start_epoch 40 out=49b] v40'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.271+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:29 4.7 4:e0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a048000 2023-11-08T16:31:21.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.271+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(29 [pgnls start_epoch 40 out=49b] v39'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048000 2023-11-08T16:31:21.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.271+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:30 4.17 4:e8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69f2ec00 2023-11-08T16:31:21.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.272+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(30 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.272+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:31 4.f 4:f0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a048000 2023-11-08T16:31:21.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.272+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(31 [pgnls start_epoch 40 out=73b] v40'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048000 2023-11-08T16:31:21.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.272+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:32 4.1f 4:f8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c6a995400 2023-11-08T16:31:21.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.273+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(32 [pgnls start_epoch 40 out=101b] v40'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x556c69d38fc0 con 0x556c6a995400 2023-11-08T16:31:21.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.273+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-11-08T16:31:21.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.273+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:33 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69f2ec00 2023-11-08T16:31:21.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=2495b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+2495 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-11-08T16:31:21.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-08T16:31:21.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-11-08T16:31:21.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-08T16:31:21.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-11-08T16:31:21.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-08T16:31:21.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-11-08T16:31:21.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-08T16:31:21.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-11-08T16:31:21.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-08T16:31:21.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-11-08T16:31:21.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-08T16:31:21.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-11-08T16:31:21.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.274+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:34 4.13 4:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05a000 con 0x556c69eb0400 2023-11-08T16:31:21.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.275+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.275+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-11-08T16:31:21.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.275+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-08T16:31:21.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.275+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-11-08T16:31:21.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.275+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-08T16:31:21.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.275+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-11-08T16:31:21.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.275+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-08T16:31:21.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.275+0000 7f27dfa0d5c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-11-08T16:31:21.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.275+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:35 4.1f 4:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05a380 con 0x556c6a995400 2023-11-08T16:31:21.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.277+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x556c69d38fc0 con 0x556c6a995400 2023-11-08T16:31:21.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.277+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.277+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:36 4.0 4:00000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05a700 con 0x556c69f2ec00 2023-11-08T16:31:21.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.277+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(36 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.277+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:37 4.10 4:08000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69f2ec00 2023-11-08T16:31:21.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.278+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(37 [pgnls start_epoch 40 out=81b] v40'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.278+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:38 4.8 4:10000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a995000 2023-11-08T16:31:21.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.279+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(38 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38d80 con 0x556c6a995000 2023-11-08T16:31:21.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.279+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:39 4.18 4:18000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69eb0400 2023-11-08T16:31:21.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.280+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(39 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.280+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:40 4.4 4:20000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a048000 2023-11-08T16:31:21.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.281+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(40 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048000 2023-11-08T16:31:21.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.281+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:41 4.14 4:28000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69f2ec00 2023-11-08T16:31:21.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.281+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(41 [pgnls start_epoch 40 out=125b] v40'43 uv43 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.282+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:42 4.c 4:30000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69eb0400 2023-11-08T16:31:21.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.283+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(42 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.283+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:43 4.1c 4:38000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a048400 2023-11-08T16:31:21.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.283+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(43 [pgnls start_epoch 40 out=49b] v39'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38d80 con 0x556c6a048400 2023-11-08T16:31:21.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.283+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:44 4.2 4:40000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a048000 2023-11-08T16:31:21.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.284+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(44 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048000 2023-11-08T16:31:21.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.284+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:45 4.12 4:48000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a048000 2023-11-08T16:31:21.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.285+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(45 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048000 2023-11-08T16:31:21.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.285+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:46 4.a 4:50000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a995400 2023-11-08T16:31:21.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.286+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(46 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c6a995400 2023-11-08T16:31:21.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.286+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:47 4.1a 4:58000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69eb0400 2023-11-08T16:31:21.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.287+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 13 ==== osd_op_reply(47 [pgnls start_epoch 40 out=49b] v40'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.287+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:48 4.6 4:60000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a048800 2023-11-08T16:31:21.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.287+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(48 [pgnls start_epoch 40 out=49b] v39'9 uv9 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048800 2023-11-08T16:31:21.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.288+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:49 4.16 4:68000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a048800 2023-11-08T16:31:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.288+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(49 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048800 2023-11-08T16:31:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.288+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:50 4.e 4:70000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69eb0400 2023-11-08T16:31:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.289+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 14 ==== osd_op_reply(50 [pgnls start_epoch 40 out=49b] v39'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.289+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:51 4.1e 4:78000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a048800 2023-11-08T16:31:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.290+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(51 [pgnls start_epoch 40 out=49b] v40'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048800 2023-11-08T16:31:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.290+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:52 4.1 4:80000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69eb0400 2023-11-08T16:31:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.291+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 15 ==== osd_op_reply(52 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.291+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:53 4.11 4:88000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69f2ec00 2023-11-08T16:31:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.292+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(53 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.292+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:54 4.9 4:90000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69eb0400 2023-11-08T16:31:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.293+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 16 ==== osd_op_reply(54 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.293+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:55 4.19 4:98000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69f2ec00 2023-11-08T16:31:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.294+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(55 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.294+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:56 4.5 4:a0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a995400 2023-11-08T16:31:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.295+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(56 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c6a995400 2023-11-08T16:31:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.295+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:57 4.15 4:a8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a995000 2023-11-08T16:31:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.296+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(57 [pgnls start_epoch 40 out=73b] v39'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x556c69d38d80 con 0x556c6a995000 2023-11-08T16:31:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.296+0000 7f27dddcb640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:58 4.d 4:b0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69eb0400 2023-11-08T16:31:21.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.298+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 17 ==== osd_op_reply(58 [pgnls start_epoch 40 out=74b] v40'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.298+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:59 4.1d 4:b8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69f2ec00 2023-11-08T16:31:21.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.298+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(59 [pgnls start_epoch 40 out=49b] v39'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.298+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:60 4.3 4:c0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a048800 2023-11-08T16:31:21.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.299+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(60 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048800 2023-11-08T16:31:21.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.299+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:61 4.13 4:c8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69eb0400 2023-11-08T16:31:21.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.301+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 18 ==== osd_op_reply(61 [pgnls start_epoch 40 out=98b] v40'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.301+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:62 4.b 4:d0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a048800 2023-11-08T16:31:21.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.301+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(62 [pgnls start_epoch 40 out=76b] v40'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048800 2023-11-08T16:31:21.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.302+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:63 4.1b 4:d8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69eb0400 2023-11-08T16:31:21.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.303+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 19 ==== osd_op_reply(63 [pgnls start_epoch 40 out=49b] v40'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.303+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:64 4.7 4:e0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a048000 2023-11-08T16:31:21.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.304+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(64 [pgnls start_epoch 40 out=49b] v39'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048000 2023-11-08T16:31:21.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.304+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:65 4.17 4:e8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69f2ec00 2023-11-08T16:31:21.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.304+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(65 [pgnls start_epoch 40 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.304+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:66 4.f 4:f0000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a048000 2023-11-08T16:31:21.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.305+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(66 [pgnls start_epoch 40 out=73b] v40'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x556c69d38b40 con 0x556c6a048000 2023-11-08T16:31:21.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.305+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:67 4.1f 4:f8000000::::head [pgnls start_epoch 40 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c6a995400 2023-11-08T16:31:21.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.306+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(67 [pgnls start_epoch 40 out=101b] v40'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x556c69d38fc0 con 0x556c6a995400 2023-11-08T16:31:21.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.307+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-11-08T16:31:21.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.307+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:68 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05aa80 con 0x556c69f2ec00 2023-11-08T16:31:21.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=2495b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+2495 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-11-08T16:31:21.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-11-08T16:31:21.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-11-08T16:31:21.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-11-08T16:31:21.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-11-08T16:31:21.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-11-08T16:31:21.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-08T16:31:21.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-11-08T16:31:21.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-08T16:31:21.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-11-08T16:31:21.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-08T16:31:21.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-11-08T16:31:21.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-08T16:31:21.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-11-08T16:31:21.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-08T16:31:21.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-11-08T16:31:21.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-08T16:31:21.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-11-08T16:31:21.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.308+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:69 4.13 4:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05a700 con 0x556c69eb0400 2023-11-08T16:31:21.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.309+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 20 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.309+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-11-08T16:31:21.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.309+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-08T16:31:21.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.309+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-11-08T16:31:21.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.309+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-08T16:31:21.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.309+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-11-08T16:31:21.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.309+0000 7f27dfa0d5c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-08T16:31:21.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.309+0000 7f27dfa0d5c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-11-08T16:31:21.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.309+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:70 4.1f 4:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05a000 con 0x556c6a995400 2023-11-08T16:31:21.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.311+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x556c69d38fc0 con 0x556c6a995400 2023-11-08T16:31:21.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.311+0000 7f27dfa0d5c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:31:21.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.311+0000 7f27dfa0d5c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-11-08T16:31:21.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.311+0000 7f27dfa0d5c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-11-08T16:31:21.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.311+0000 7f27dfa0d5c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-11-08T16:31:21.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.311+0000 7f27dfa0d5c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-11-08T16:31:21.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.311+0000 7f27dfa0d5c0 10 datascan.scan_links: processing 6 injected_inos 2023-11-08T16:31:21.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.311+0000 7f27dfa0d5c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-11-08T16:31:21.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.311+0000 7f27dfa0d5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-11-08T16:31:21.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.311+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:71 4.1f 4:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05ae00 con 0x556c6a995400 2023-11-08T16:31:21.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.312+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x556c69d38fc0 con 0x556c6a995400 2023-11-08T16:31:21.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.312+0000 7f27dfa0d5c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:31:21.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.312+0000 7f27dfa0d5c0 20 datascan.scan_links: first is now 2 2023-11-08T16:31:21.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.312+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:72 4.1f 4:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05b180 con 0x556c6a995400 2023-11-08T16:31:21.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.316+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v40'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x556c69d38fc0 con 0x556c6a995400 2023-11-08T16:31:21.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.316+0000 7f27dfa0d5c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-11-08T16:31:21.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.316+0000 7f27dfa0d5c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-11-08T16:31:21.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.316+0000 7f27dfa0d5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:0 2023-11-08T16:31:21.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.316+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:73 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05b500 con 0x556c69f2ec00 2023-11-08T16:31:21.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.317+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.317+0000 7f27dfa0d5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-11-08T16:31:21.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.317+0000 7f27dfa0d5c0 20 datascan.scan_links: first is now 2 2023-11-08T16:31:21.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.317+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:74 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05a380 con 0x556c69f2ec00 2023-11-08T16:31:21.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.321+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 18 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v40'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.321+0000 7f27dfa0d5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-11-08T16:31:21.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.321+0000 7f27dfa0d5c0 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-11-08T16:31:21.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.321+0000 7f27dfa0d5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:1 2023-11-08T16:31:21.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.321+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:75 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05a700 con 0x556c69f2ec00 2023-11-08T16:31:21.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.322+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 19 ==== osd_op_reply(75 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.322+0000 7f27dfa0d5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-11-08T16:31:21.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.322+0000 7f27dfa0d5c0 20 datascan.scan_links: first is now 2 2023-11-08T16:31:21.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.322+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:76 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05b880 con 0x556c69f2ec00 2023-11-08T16:31:21.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.325+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 20 ==== osd_op_reply(76 10000000000.00000000 [omap-set-vals] v40'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.325+0000 7f27dfa0d5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-11-08T16:31:21.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.325+0000 7f27dfa0d5c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-11-08T16:31:21.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.325+0000 7f27dfa0d5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:2 2023-11-08T16:31:21.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.325+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:77 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05bc00 con 0x556c69f2ec00 2023-11-08T16:31:21.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.327+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 21 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.327+0000 7f27dfa0d5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-11-08T16:31:21.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.327+0000 7f27dfa0d5c0 20 datascan.scan_links: first is now 2 2023-11-08T16:31:21.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.327+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:78 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17c00 con 0x556c69f2ec00 2023-11-08T16:31:21.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.331+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 22 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v40'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.331+0000 7f27dfa0d5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-11-08T16:31:21.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.331+0000 7f27dfa0d5c0 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-11-08T16:31:21.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.331+0000 7f27dfa0d5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:3 2023-11-08T16:31:21.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.331+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:79 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05aa80 con 0x556c69f2ec00 2023-11-08T16:31:21.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.332+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 23 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.332+0000 7f27dfa0d5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-11-08T16:31:21.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.332+0000 7f27dfa0d5c0 20 datascan.scan_links: first is now 2 2023-11-08T16:31:21.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.332+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:80 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05b500 con 0x556c69f2ec00 2023-11-08T16:31:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.336+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 24 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v40'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.336+0000 7f27dfa0d5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-11-08T16:31:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.336+0000 7f27dfa0d5c0 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-11-08T16:31:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.336+0000 7f27dfa0d5c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:4 2023-11-08T16:31:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.336+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:81 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05a380 con 0x556c69f2ec00 2023-11-08T16:31:21.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.337+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.337+0000 7f27dfa0d5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-11-08T16:31:21.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.337+0000 7f27dfa0d5c0 20 datascan.scan_links: first is now 2 2023-11-08T16:31:21.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.337+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:82 4.10 4:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c69c17880 con 0x556c69f2ec00 2023-11-08T16:31:21.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.341+0000 7f27dcdc9640 1 -- 172.21.15.81:0/1230958031 <== osd.3 v2:172.21.15.81:6816/2756229107 26 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v40'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x556c69d38b40 con 0x556c69f2ec00 2023-11-08T16:31:21.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.341+0000 7f27dfa0d5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-11-08T16:31:21.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.341+0000 7f27dfa0d5c0 10 datascan.scan_links: updating inotable 2023-11-08T16:31:21.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.341+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:83 4.13 4:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05a700 con 0x556c69eb0400 2023-11-08T16:31:21.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.342+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 21 ==== osd_op_reply(83 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.342+0000 7f27dfa0d5c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-11-08T16:31:21.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.342+0000 7f27dfa0d5c0 10 mds.0.inotable: skip_inos now [0x100000001fa~0xfffffffe06] 2023-11-08T16:31:21.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.342+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:84 4.13 4:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05b880 con 0x556c69eb0400 2023-11-08T16:31:21.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.347+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 22 ==== osd_op_reply(84 mds0_inotable [writefull 0~34] v40'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.347+0000 7f27dfa0d5c0 10 datascan.scan_links: updating snaptable 2023-11-08T16:31:21.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.347+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:85 4.d 4:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40) v8 -- 0x556c6a05bc00 con 0x556c69eb0400 2023-11-08T16:31:21.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.348+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 <== osd.4 v2:172.21.15.93:6800/3259756279 23 ==== osd_op_reply(85 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x556c69d38fc0 con 0x556c69eb0400 2023-11-08T16:31:21.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.348+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x556c6a048800 msgr2=0x556c6a039600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.348+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x556c6a048800 0x556c6a039600 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.348+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x556c6a048000 msgr2=0x556c6a038b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.348+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x556c6a048000 0x556c6a038b00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.348+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x556c6a048400 msgr2=0x556c6a039080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.348+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x556c6a048400 0x556c6a039080 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.348+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x556c69f2ec00 msgr2=0x556c69c2a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.348+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x556c69f2ec00 0x556c69c2a100 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.348+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 reap_dead start 2023-11-08T16:31:21.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x556c69eb0400 msgr2=0x556c6a038580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x556c69eb0400 0x556c6a038580 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x556c6a995000 msgr2=0x556c6a038000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x556c6a995000 0x556c6a038000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x556c6a995400 msgr2=0x556c6a039b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x556c6a995400 0x556c6a039b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x556c69f2e800 msgr2=0x556c69c2b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x556c69f2e800 0x556c69c2b700 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x556c6a9c4930 tx=0x556c69ed2f00 comp rx=0 tx=0).stop 2023-11-08T16:31:21.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 msgr2=0x556c69c2ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/1230958031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69eb1400 0x556c69c2ac00 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x556c69ec5350 tx=0x556c69ed2c00 comp rx=0 tx=0).stop 2023-11-08T16:31:21.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 reap_dead start 2023-11-08T16:31:21.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dd5ca640 1 -- 172.21.15.81:0/1230958031 reap_dead start 2023-11-08T16:31:21.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 shutdown_connections 2023-11-08T16:31:21.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.349+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 >> 172.21.15.81:0/1230958031 conn(0x556c69eb0c00 msgr2=0x556c6a938800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:21.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.350+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 shutdown_connections 2023-11-08T16:31:21.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.350+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/1230958031 wait complete. 2023-11-08T16:31:21.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.351+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/563855180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69bbfc00 msgr2=0x556c69c29080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.351+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/563855180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69bbfc00 0x556c69c29080 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x556c6a9c5440 tx=0x556c69ea2120 comp rx=0 tx=0).stop 2023-11-08T16:31:21.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.351+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/563855180 shutdown_connections 2023-11-08T16:31:21.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.351+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/563855180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556c69bbfc00 0x556c69c29080 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.351+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/563855180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556c69bbf800 0x556c69c28b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.351+0000 7f27dfa0d5c0 1 --2- 172.21.15.81:0/563855180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556c69bbec00 0x556c69c28580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.351+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/563855180 >> 172.21.15.81:0/563855180 conn(0x556c69bbf000 msgr2=0x556c6a939000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:21.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.351+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/563855180 shutdown_connections 2023-11-08T16:31:21.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.351+0000 7f27dfa0d5c0 1 -- 172.21.15.81:0/563855180 wait complete. 2023-11-08T16:31:21.363 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.238724, result: 2023-11-08T16:31:21.363 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-08T16:31:21.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.581+0000 7f7903fff640 1 Processor -- start 2023-11-08T16:31:21.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.582+0000 7f7903fff640 1 -- start start 2023-11-08T16:31:21.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.583+0000 7f7903fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f790406f7c0 0x7f79040cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.583+0000 7f7903fff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f79040ad940 0x7f790406b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.583+0000 7f7903fff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f790406cd70 con 0x7f790415ac20 2023-11-08T16:31:21.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.583+0000 7f7903fff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79040ae130 con 0x7f79040ad940 2023-11-08T16:31:21.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.583+0000 7f7903fff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79040cb370 con 0x7f790406f7c0 2023-11-08T16:31:21.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.583+0000 7f7902ffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f790415ac20 0x7f7904070f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54486/0 (socket says 172.21.15.81:54486) 2023-11-08T16:31:21.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.583+0000 7f7902ffd640 1 -- 172.21.15.81:0/89646784 learned_addr learned my addr 172.21.15.81:0/89646784 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:21.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4192487101 0 0) 0x7f790406cd70 con 0x7f790415ac20 2023-11-08T16:31:21.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78f0003520 con 0x7f790415ac20 2023-11-08T16:31:21.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79037fe640 1 --2- 172.21.15.81:0/89646784 >> v2:172.21.15.93:3300/0 conn(0x7f79040ad940 0x7f790406b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f78fa7fc640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f790406f7c0 0x7f79040cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3103765498 0 0) 0x7f78f0003520 con 0x7f790415ac20 2023-11-08T16:31:21.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f790406f7c0 msgr2=0x7f79040cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:21.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f790406f7c0 0x7f79040cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 >> v2:172.21.15.93:3300/0 conn(0x7f79040ad940 msgr2=0x7f790406b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:21.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 --2- 172.21.15.81:0/89646784 >> v2:172.21.15.93:3300/0 conn(0x7f79040ad940 0x7f790406b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79040ca500 con 0x7f790415ac20 2023-11-08T16:31:21.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f78f4001670 con 0x7f790415ac20 2023-11-08T16:31:21.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 >> v1:172.21.15.81:6789/0 conn(0x7f790415ac20 legacy=0x7f7904070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f790406f7c0 0x7f78f0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 0x7f78f00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78f0004d00 0x7f78f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79040ae130 con 0x7f78f0003ff0 2023-11-08T16:31:21.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79040cb370 con 0x7f78f0004d00 2023-11-08T16:31:21.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.584+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78f000bb90 con 0x7f790406f7c0 2023-11-08T16:31:21.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.585+0000 7f78fa7fc640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 0x7f78f00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.585+0000 7f7902ffd640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f790406f7c0 0x7f78f0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.585+0000 7f78fa7fc640 1 -- 172.21.15.81:0/89646784 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f790406f7c0 msgr2=0x7f78f0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:21.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.585+0000 7f79037fe640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78f0004d00 0x7f78f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.585+0000 7f78fa7fc640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f790406f7c0 0x7f78f0003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.585+0000 7f78fa7fc640 1 -- 172.21.15.81:0/89646784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78f0004d00 msgr2=0x7f78f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:21.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.585+0000 7f78fa7fc640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78f0004d00 0x7f78f0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.585+0000 7f78fa7fc640 1 -- 172.21.15.81:0/89646784 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78f0003520 con 0x7f78f0003ff0 2023-11-08T16:31:21.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.586+0000 7f78fa7fc640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 0x7f78f00047c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00ba30 tx=0x7f78f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61e28a11385f021 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.586+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec032630 con 0x7f78f0003ff0 2023-11-08T16:31:21.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.586+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec032790 con 0x7f78f0003ff0 2023-11-08T16:31:21.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.586+0000 7f79027fc640 1 -- 172.21.15.81:0/89646784 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec032b20 con 0x7f78f0003ff0 2023-11-08T16:31:21.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.587+0000 7f7903fff640 1 -- 172.21.15.81:0/89646784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 msgr2=0x7f78f00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.587+0000 7f7903fff640 1 --2- 172.21.15.81:0/89646784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 0x7f78f00047c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00ba30 tx=0x7f78f00057f0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.587+0000 7f79037fe640 1 -- 172.21.15.81:0/89646784 reap_dead start 2023-11-08T16:31:21.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.587+0000 7f7903fff640 1 -- 172.21.15.81:0/89646784 shutdown_connections 2023-11-08T16:31:21.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.587+0000 7f7903fff640 1 -- 172.21.15.81:0/89646784 >> 172.21.15.81:0/89646784 conn(0x7f79040b04d0 msgr2=0x7f79040b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:21.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.587+0000 7f7903fff640 1 -- 172.21.15.81:0/89646784 shutdown_connections 2023-11-08T16:31:21.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.587+0000 7f7903fff640 1 -- 172.21.15.81:0/89646784 wait complete. 2023-11-08T16:31:21.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.589+0000 7f7903fff640 1 Processor -- start 2023-11-08T16:31:21.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.589+0000 7f7903fff640 1 -- start start 2023-11-08T16:31:21.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.590+0000 7f7903fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 0x7f7904156070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.590+0000 7f7903fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7904151790 0x7f7904152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.590+0000 7f7903fff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f790415c050 0x7f790415c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.590+0000 7f7903fff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79041536c0 con 0x7f78f0003ff0 2023-11-08T16:31:21.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.590+0000 7f7903fff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79041539a0 con 0x7f790415c050 2023-11-08T16:31:21.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.590+0000 7f7903fff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7904153c80 con 0x7f7904151790 2023-11-08T16:31:21.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.590+0000 7f7902ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 0x7f7904156070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.590+0000 7f7902ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 0x7f7904156070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46086/0 (socket says 172.21.15.81:46086) 2023-11-08T16:31:21.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.590+0000 7f7902ffd640 1 -- 172.21.15.81:0/660610322 learned_addr learned my addr 172.21.15.81:0/660610322 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:21.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.591+0000 7f7902ffd640 1 -- 172.21.15.81:0/660610322 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7904151790 msgr2=0x7f7904152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.591+0000 7f7902ffd640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7904151790 0x7f7904152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.591+0000 7f7902ffd640 1 -- 172.21.15.81:0/660610322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f790415c050 msgr2=0x7f790415c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:21.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.591+0000 7f7902ffd640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f790415c050 0x7f790415c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:21.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.591+0000 7f7902ffd640 1 -- 172.21.15.81:0/660610322 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7904153f60 con 0x7f78f0003ff0 2023-11-08T16:31:21.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.591+0000 7f7902ffd640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 0x7f7904156070 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f78f400d810 tx=0x7f78f400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.591+0000 7f78fbfff640 1 -- 172.21.15.81:0/660610322 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f40074e0 con 0x7f78f0003ff0 2023-11-08T16:31:21.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.591+0000 7f78fbfff640 1 -- 172.21.15.81:0/660610322 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f4007640 con 0x7f78f0003ff0 2023-11-08T16:31:21.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.591+0000 7f78fbfff640 1 -- 172.21.15.81:0/660610322 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4004a20 con 0x7f78f0003ff0 2023-11-08T16:31:21.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.591+0000 7f7903fff640 1 -- 172.21.15.81:0/660610322 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7904154240 con 0x7f78f0003ff0 2023-11-08T16:31:21.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.591+0000 7f7903fff640 1 -- 172.21.15.81:0/660610322 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7904151c10 con 0x7f78f0003ff0 2023-11-08T16:31:21.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.592+0000 7f7903fff640 1 -- 172.21.15.81:0/660610322 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7904154430 con 0x7f78f0003ff0 2023-11-08T16:31:21.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.593+0000 7f78fbfff640 1 -- 172.21.15.81:0/660610322 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78f4004b80 con 0x7f78f0003ff0 2023-11-08T16:31:21.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.594+0000 7f78fbfff640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78e406aea0 0x7f78e406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:21.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.594+0000 7f78fbfff640 1 -- 172.21.15.81:0/660610322 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x7f78f40892b0 con 0x7f78f0003ff0 2023-11-08T16:31:21.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.595+0000 7f78fa7fc640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78e406aea0 0x7f78e406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:21.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.595+0000 7f78fa7fc640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78e406aea0 0x7f78e406d360 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00ba00 tx=0x7f78ec032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:21.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.599+0000 7f78fbfff640 1 -- 172.21.15.81:0/660610322 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78f4054080 con 0x7f78f0003ff0 2023-11-08T16:31:21.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:21.860+0000 7f7903fff640 1 -- 172.21.15.81:0/660610322 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f790404ece0 con 0x7f78f0003ff0 2023-11-08T16:31:22.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.833+0000 7f78fbfff640 1 -- 172.21.15.81:0/660610322 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 2:0 v26) v1 ==== 101+0+0 (secure 0 0 0) 0x7f78f40576a0 con 0x7f78f0003ff0 2023-11-08T16:31:22.835 INFO:teuthology.orchestra.run.smithi081.stderr:repaired: restoring rank 2:0 2023-11-08T16:31:22.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 -- 172.21.15.81:0/660610322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78e406aea0 msgr2=0x7f78e406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:22.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78e406aea0 0x7f78e406d360 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f78ec00ba00 tx=0x7f78ec032000 comp rx=0 tx=0).stop 2023-11-08T16:31:22.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 -- 172.21.15.81:0/660610322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 msgr2=0x7f7904156070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:22.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 0x7f7904156070 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f78f400d810 tx=0x7f78f400dcd0 comp rx=0 tx=0).stop 2023-11-08T16:31:22.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 -- 172.21.15.81:0/660610322 shutdown_connections 2023-11-08T16:31:22.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78e406aea0 0x7f78e406d360 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:22.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f790415c050 0x7f790415c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:22.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7904151790 0x7f7904152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:22.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 --2- 172.21.15.81:0/660610322 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78f0003ff0 0x7f7904156070 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:22.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 -- 172.21.15.81:0/660610322 >> 172.21.15.81:0/660610322 conn(0x7f79040b04d0 msgr2=0x7f79040cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:22.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 -- 172.21.15.81:0/660610322 shutdown_connections 2023-11-08T16:31:22.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:22.836+0000 7f7903fff640 1 -- 172.21.15.81:0/660610322 wait complete. 2023-11-08T16:31:22.849 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:22.847+0000 7f61b6f71640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-08T16:31:22.861 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T16:31:22.862 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T16:31:22.862 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T16:31:22.862 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:31:22.862 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T16:31:22.863 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T16:31:22.863 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T16:31:22.863 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:31:22.864 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T16:31:22.864 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T16:31:22.864 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T16:31:22.864 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:31:22.865 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T16:31:22.865 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T16:31:22.865 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T16:31:22.866 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:31:22.879 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T16:31:22.880 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T16:31:22.881 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T16:31:22.882 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T16:31:22.884 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T16:31:22.884 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T16:31:22.893 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T16:31:22.893 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T16:31:22.927 INFO:tasks.ceph.mds.a:Started 2023-11-08T16:31:22.928 DEBUG:teuthology.parallel:result is None 2023-11-08T16:31:22.935 INFO:tasks.ceph.mds.c:Started 2023-11-08T16:31:22.936 DEBUG:teuthology.parallel:result is None 2023-11-08T16:31:22.941 INFO:tasks.ceph.mds.d:Started 2023-11-08T16:31:22.941 DEBUG:teuthology.parallel:result is None 2023-11-08T16:31:22.949 INFO:tasks.ceph.mds.b:Started 2023-11-08T16:31:22.949 DEBUG:teuthology.parallel:result is None 2023-11-08T16:31:22.950 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:31:23.086 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:31:23.093 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:31:23.108 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2023-11-08T16:31:23.108 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:31:23.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.139+0000 7fad33ab4640 1 Processor -- start 2023-11-08T16:31:23.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.139+0000 7fad33ab4640 1 -- start start 2023-11-08T16:31:23.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.140+0000 7fad33ab4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fad2c0cc5e0 0x7fad2c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:23.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.140+0000 7fad33ab4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2c0c9a70 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.140+0000 7fad33ab4640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad2c0cc3e0 con 0x7fad2c0adcd0 2023-11-08T16:31:23.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.140+0000 7fad33ab4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad2c06d070 con 0x7fad2c0cc5e0 2023-11-08T16:31:23.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.140+0000 7fad32051640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fad2c0ccfb0 0x7fad2c070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54518/0 (socket says 172.21.15.81:54518) 2023-11-08T16:31:23.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.140+0000 7fad32051640 1 -- 172.21.15.81:0/1524417926 learned_addr learned my addr 172.21.15.81:0/1524417926 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:23.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.140+0000 7fad3104f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fad2c0cc5e0 0x7fad2c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:23.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.141+0000 7fad3104f640 1 -- 172.21.15.81:0/1524417926 >> v1:172.21.15.93:6789/0 conn(0x7fad2c0adcd0 legacy=0x7fad2c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:23.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.141+0000 7fad3104f640 1 -- 172.21.15.81:0/1524417926 >> v1:172.21.15.81:6789/0 conn(0x7fad2c0ccfb0 legacy=0x7fad2c070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:23.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.141+0000 7fad3104f640 1 -- 172.21.15.81:0/1524417926 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad2c06d350 con 0x7fad2c0cc5e0 2023-11-08T16:31:23.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.141+0000 7fad3084e640 1 -- 172.21.15.81:0/1524417926 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732644188 0 0) 0x7fad2c0cc3e0 con 0x7fad2c0adcd0 2023-11-08T16:31:23.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.142+0000 7fad3104f640 1 --2- 172.21.15.81:0/1524417926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad2c0cc5e0 0x7fad2c0cca70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0097a0 tx=0x7fad1c02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615861775d6dbd7e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:23.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.142+0000 7fad3084e640 1 -- 172.21.15.81:0/1524417926 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c030710 con 0x7fad2c0cc5e0 2023-11-08T16:31:23.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.142+0000 7fad3084e640 1 -- 172.21.15.81:0/1524417926 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad1c02e050 con 0x7fad2c0cc5e0 2023-11-08T16:31:23.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.142+0000 7fad3084e640 1 -- 172.21.15.81:0/1524417926 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1c030a90 con 0x7fad2c0cc5e0 2023-11-08T16:31:23.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.143+0000 7fad33ab4640 1 -- 172.21.15.81:0/1524417926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad2c0cc5e0 msgr2=0x7fad2c0cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:23.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.143+0000 7fad33ab4640 1 --2- 172.21.15.81:0/1524417926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad2c0cc5e0 0x7fad2c0cca70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fad1c0097a0 tx=0x7fad1c02b790 comp rx=0 tx=0).stop 2023-11-08T16:31:23.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.143+0000 7fad33ab4640 1 -- 172.21.15.81:0/1524417926 shutdown_connections 2023-11-08T16:31:23.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.143+0000 7fad33ab4640 1 --2- 172.21.15.81:0/1524417926 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad2c0cc5e0 0x7fad2c0cca70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:23.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.143+0000 7fad33ab4640 1 -- 172.21.15.81:0/1524417926 >> 172.21.15.81:0/1524417926 conn(0x7fad2c0cb490 msgr2=0x7fad2c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:23.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.143+0000 7fad33ab4640 1 -- 172.21.15.81:0/1524417926 shutdown_connections 2023-11-08T16:31:23.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.143+0000 7fad33ab4640 1 -- 172.21.15.81:0/1524417926 wait complete. 2023-11-08T16:31:23.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.144+0000 7fad33ab4640 1 Processor -- start 2023-11-08T16:31:23.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.145+0000 7fad33ab4640 1 -- start start 2023-11-08T16:31:23.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.145+0000 7fad33ab4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad2c0adcd0 0x7fad2c0c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:23.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad33ab4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad2c0cc5e0 0x7fad2c155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:23.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad33ab4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad2c0ccfb0 0x7fad2c0c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:23.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad33ab4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c071750 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad33ab4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c0c8250 con 0x7fad2c0adcd0 2023-11-08T16:31:23.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad33ab4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad2c070ad0 con 0x7fad2c0cc5e0 2023-11-08T16:31:23.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad32051640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad2c0ccfb0 0x7fad2c0c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:23.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad32051640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad2c0ccfb0 0x7fad2c0c7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46124/0 (socket says 172.21.15.81:46124) 2023-11-08T16:31:23.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad32051640 1 -- 172.21.15.81:0/3439157239 learned_addr learned my addr 172.21.15.81:0/3439157239 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:23.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad3104f640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad2c0cc5e0 0x7fad2c155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:23.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad31850640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad2c0adcd0 0x7fad2c0c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:23.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad32051640 1 -- 172.21.15.81:0/3439157239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad2c0cc5e0 msgr2=0x7fad2c155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:23.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad32051640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad2c0cc5e0 0x7fad2c155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:23.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad32051640 1 -- 172.21.15.81:0/3439157239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad2c0adcd0 msgr2=0x7fad2c0c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:23.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad32051640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad2c0adcd0 0x7fad2c0c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:23.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.146+0000 7fad32051640 1 -- 172.21.15.81:0/3439157239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c009400 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.147+0000 7fad32051640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad2c0ccfb0 0x7fad2c0c7d10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fad2400c8c0 tx=0x7fad2400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:23.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.147+0000 7fad227fc640 1 -- 172.21.15.81:0/3439157239 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400a640 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.147+0000 7fad227fc640 1 -- 172.21.15.81:0/3439157239 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2400a7a0 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.147+0000 7fad33ab4640 1 -- 172.21.15.81:0/3439157239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad2c070db0 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.147+0000 7fad227fc640 1 -- 172.21.15.81:0/3439157239 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400ab50 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.147+0000 7fad33ab4640 1 -- 172.21.15.81:0/3439157239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad2c071370 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.148+0000 7fad33ab4640 1 -- 172.21.15.81:0/3439157239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad2c072170 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.149+0000 7fad227fc640 1 -- 172.21.15.81:0/3439157239 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad2400acb0 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.150+0000 7fad227fc640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad1006ae70 0x7fad1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:23.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.150+0000 7fad227fc640 1 -- 172.21.15.81:0/3439157239 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6285+0+0 (secure 0 0 0) 0x7fad24088e30 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.150+0000 7fad31850640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad1006ae70 0x7fad1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:23.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.151+0000 7fad31850640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad1006ae70 0x7fad1006d330 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fad28000f60 tx=0x7fad28009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:23.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.155+0000 7fad227fc640 1 -- 172.21.15.81:0/3439157239 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fad24053d40 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.375+0000 7fad33ab4640 1 -- 172.21.15.81:0/3439157239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad2c04ece0 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.376+0000 7fad227fc640 1 -- 172.21.15.81:0/3439157239 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fad24018070 con 0x7fad2c0ccfb0 2023-11-08T16:31:23.378 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:23.378 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":27,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4625,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/1453041994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":1453041994},{"type":"v1","addr":"172.21.15.93:6835","nonce":1453041994}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4780,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/757624701","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":757624701},{"type":"v1","addr":"172.21.15.93:6833","nonce":757624701}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1735232404","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1735232404},{"type":"v1","addr":"172.21.15.81:6835","nonce":1735232404}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":27,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:22.837099+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":40,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5475":{"gid":5475,"name":"c","rank":0,"incarnation":27,"state":"up:replay","state_seq":1,"addr":"172.21.15.81:6837/1759199271","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1759199271},{"type":"v1","addr":"172.21.15.81:6837","nonce":1759199271}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:31:23.378 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 27 2023-11-08T16:31:23.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.378+0000 7fad33ab4640 1 -- 172.21.15.81:0/3439157239 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad1006ae70 msgr2=0x7fad1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:23.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.378+0000 7fad33ab4640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad1006ae70 0x7fad1006d330 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fad28000f60 tx=0x7fad28009040 comp rx=0 tx=0).stop 2023-11-08T16:31:23.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.378+0000 7fad33ab4640 1 -- 172.21.15.81:0/3439157239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad2c0ccfb0 msgr2=0x7fad2c0c7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:23.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.378+0000 7fad33ab4640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad2c0ccfb0 0x7fad2c0c7d10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fad2400c8c0 tx=0x7fad2400cd80 comp rx=0 tx=0).stop 2023-11-08T16:31:23.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.379+0000 7fad33ab4640 1 -- 172.21.15.81:0/3439157239 shutdown_connections 2023-11-08T16:31:23.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.379+0000 7fad33ab4640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad1006ae70 0x7fad1006d330 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:23.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.379+0000 7fad33ab4640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fad2c0ccfb0 0x7fad2c0c7d10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:23.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.379+0000 7fad33ab4640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fad2c0cc5e0 0x7fad2c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:23.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.379+0000 7fad33ab4640 1 --2- 172.21.15.81:0/3439157239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fad2c0adcd0 0x7fad2c0c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:23.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.379+0000 7fad33ab4640 1 -- 172.21.15.81:0/3439157239 >> 172.21.15.81:0/3439157239 conn(0x7fad2c0cb490 msgr2=0x7fad2c15a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:23.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.379+0000 7fad33ab4640 1 -- 172.21.15.81:0/3439157239 shutdown_connections 2023-11-08T16:31:23.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:23.379+0000 7fad33ab4640 1 -- 172.21.15.81:0/3439157239 wait complete. 2023-11-08T16:31:23.800 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:31:15.514873+0000', 'modified': '2023-11-08T16:31:22.837099+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 40, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5475': {'gid': 5475, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.81:6837/1759199271', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 1759199271}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 1759199271}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5, 6], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:31:23.801 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5475:up:replay 2023-11-08T16:31:23.836 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:31:23.834+0000 7f5cc09c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:31:23.866 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:23.864+0000 7fbf103d6640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-08T16:31:24.802 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:31:25.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.019+0000 7f1437267640 1 Processor -- start 2023-11-08T16:31:25.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.019+0000 7f1437267640 1 -- start start 2023-11-08T16:31:25.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.021+0000 7f1437267640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143015d490 0x7f14300d39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:25.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.021+0000 7f1437267640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f14300d1bf0 0x7f14300b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:25.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.021+0000 7f1437267640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14300ce890 con 0x7f14300d1080 2023-11-08T16:31:25.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.021+0000 7f1437267640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14300ceb70 con 0x7f14300d1bf0 2023-11-08T16:31:25.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.021+0000 7f1437267640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14300cee50 con 0x7f143015d490 2023-11-08T16:31:25.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.021+0000 7f1436a66640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f14300d1bf0 0x7f14300b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:25.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.021+0000 7f1436a66640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f14300d1bf0 0x7f14300b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46136/0 (socket says 172.21.15.81:46136) 2023-11-08T16:31:25.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.021+0000 7f1436a66640 1 -- 172.21.15.81:0/2078709543 learned_addr learned my addr 172.21.15.81:0/2078709543 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:25.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.021+0000 7f1436265640 1 --2- 172.21.15.81:0/2078709543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143015d490 0x7f14300d39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:25.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.022+0000 7f1436a66640 1 -- 172.21.15.81:0/2078709543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143015d490 msgr2=0x7f14300d39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:25.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.022+0000 7f1436a66640 1 --2- 172.21.15.81:0/2078709543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143015d490 0x7f14300d39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:25.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.022+0000 7f1436a66640 1 -- 172.21.15.81:0/2078709543 >> v1:172.21.15.93:6789/0 conn(0x7f14300d1080 legacy=0x7f14300d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:25.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.022+0000 7f1436a66640 1 -- 172.21.15.81:0/2078709543 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14300cf130 con 0x7f14300d1bf0 2023-11-08T16:31:25.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.023+0000 7f1436a66640 1 --2- 172.21.15.81:0/2078709543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14300d1bf0 0x7f14300b5e50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f14280098a0 tx=0x7f142802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1beb48e77c259 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:25.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.024+0000 7f1435263640 1 -- 172.21.15.81:0/2078709543 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428031640 con 0x7f14300d1bf0 2023-11-08T16:31:25.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.024+0000 7f1435263640 1 -- 172.21.15.81:0/2078709543 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14280317a0 con 0x7f14300d1bf0 2023-11-08T16:31:25.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.024+0000 7f1435263640 1 -- 172.21.15.81:0/2078709543 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428031b30 con 0x7f14300d1bf0 2023-11-08T16:31:25.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.024+0000 7f1437267640 1 -- 172.21.15.81:0/2078709543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14300d1bf0 msgr2=0x7f14300b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:25.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.024+0000 7f1437267640 1 --2- 172.21.15.81:0/2078709543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14300d1bf0 0x7f14300b5e50 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f14280098a0 tx=0x7f142802b8f0 comp rx=0 tx=0).stop 2023-11-08T16:31:25.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.025+0000 7f1437267640 1 -- 172.21.15.81:0/2078709543 shutdown_connections 2023-11-08T16:31:25.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.025+0000 7f1437267640 1 --2- 172.21.15.81:0/2078709543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14300d1bf0 0x7f14300b5e50 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:25.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.025+0000 7f1437267640 1 --2- 172.21.15.81:0/2078709543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143015d490 0x7f14300d39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:25.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.025+0000 7f1437267640 1 -- 172.21.15.81:0/2078709543 >> 172.21.15.81:0/2078709543 conn(0x7f14300d2e50 msgr2=0x7f14300d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:25.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.025+0000 7f1437267640 1 -- 172.21.15.81:0/2078709543 shutdown_connections 2023-11-08T16:31:25.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.025+0000 7f1437267640 1 -- 172.21.15.81:0/2078709543 wait complete. 2023-11-08T16:31:25.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.027+0000 7f1437267640 1 Processor -- start 2023-11-08T16:31:25.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.027+0000 7f1437267640 1 -- start start 2023-11-08T16:31:25.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1437267640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14300d1080 0x7f14300ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:25.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1437267640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14300d1bf0 0x7f14300caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:25.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1437267640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143015d490 0x7f143014dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:25.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1437267640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14300cba40 con 0x7f14300d1080 2023-11-08T16:31:25.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1437267640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143014f240 con 0x7f14300d1bf0 2023-11-08T16:31:25.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1437267640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14300cb760 con 0x7f143015d490 2023-11-08T16:31:25.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1436265640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14300d1080 0x7f14300ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:25.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1436265640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14300d1080 0x7f14300ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46142/0 (socket says 172.21.15.81:46142) 2023-11-08T16:31:25.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1436265640 1 -- 172.21.15.81:0/1549601250 learned_addr learned my addr 172.21.15.81:0/1549601250 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:25.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1436265640 1 -- 172.21.15.81:0/1549601250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143015d490 msgr2=0x7f143014dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:25.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1436265640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143015d490 0x7f143014dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:25.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1436265640 1 -- 172.21.15.81:0/1549601250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14300d1bf0 msgr2=0x7f14300caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:25.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1436265640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14300d1bf0 0x7f14300caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:25.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.028+0000 7f1436265640 1 -- 172.21.15.81:0/1549601250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14280094d0 con 0x7f14300d1080 2023-11-08T16:31:25.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.029+0000 7f1436265640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14300d1080 0x7f14300ca9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f142c007f20 tx=0x7f142c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:25.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.029+0000 7f1426ffd640 1 -- 172.21.15.81:0/1549601250 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c014620 con 0x7f14300d1080 2023-11-08T16:31:25.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.029+0000 7f1426ffd640 1 -- 172.21.15.81:0/1549601250 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c014780 con 0x7f14300d1080 2023-11-08T16:31:25.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.029+0000 7f1426ffd640 1 -- 172.21.15.81:0/1549601250 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c014b10 con 0x7f14300d1080 2023-11-08T16:31:25.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.029+0000 7f1437267640 1 -- 172.21.15.81:0/1549601250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143014bef0 con 0x7f14300d1080 2023-11-08T16:31:25.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.029+0000 7f1437267640 1 -- 172.21.15.81:0/1549601250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143014c4b0 con 0x7f14300d1080 2023-11-08T16:31:25.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.030+0000 7f1437267640 1 -- 172.21.15.81:0/1549601250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143014c670 con 0x7f14300d1080 2023-11-08T16:31:25.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.031+0000 7f1426ffd640 1 -- 172.21.15.81:0/1549601250 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f142c014c70 con 0x7f14300d1080 2023-11-08T16:31:25.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.032+0000 7f1426ffd640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141406ae70 0x7f141406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:25.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.032+0000 7f1426ffd640 1 -- 172.21.15.81:0/1549601250 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f142c08a2f0 con 0x7f14300d1080 2023-11-08T16:31:25.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.033+0000 7f1435a64640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141406ae70 0x7f141406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:25.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.033+0000 7f1435a64640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141406ae70 0x7f141406d330 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1420006fd0 tx=0x7f1420006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:25.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.037+0000 7f1426ffd640 1 -- 172.21.15.81:0/1549601250 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f142c0550f0 con 0x7f14300d1080 2023-11-08T16:31:25.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.267+0000 7f1437267640 1 -- 172.21.15.81:0/1549601250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f143014bd50 con 0x7f14300d1080 2023-11-08T16:31:25.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.268+0000 7f1426ffd640 1 -- 172.21.15.81:0/1549601250 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f143014bd50 con 0x7f14300d1080 2023-11-08T16:31:25.270 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:25.271 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":30,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3654712142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3654712142},{"type":"v1","addr":"172.21.15.93:6834","nonce":3654712142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5014,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/66363251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":66363251},{"type":"v1","addr":"172.21.15.93:6835","nonce":66363251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4031527046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4031527046},{"type":"v1","addr":"172.21.15.81:6835","nonce":4031527046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:23.885171+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":29,"state":"up:reconnect","state_seq":2,"addr":"172.21.15.81:6837/2942618483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2942618483},{"type":"v1","addr":"172.21.15.81:6837","nonce":2942618483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:31:25.271 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 30 2023-11-08T16:31:25.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 -- 172.21.15.81:0/1549601250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141406ae70 msgr2=0x7f141406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:25.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141406ae70 0x7f141406d330 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1420006fd0 tx=0x7f1420006bc0 comp rx=0 tx=0).stop 2023-11-08T16:31:25.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 -- 172.21.15.81:0/1549601250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14300d1080 msgr2=0x7f14300ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:25.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14300d1080 0x7f14300ca9c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f142c007f20 tx=0x7f142c00e570 comp rx=0 tx=0).stop 2023-11-08T16:31:25.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 -- 172.21.15.81:0/1549601250 shutdown_connections 2023-11-08T16:31:25.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141406ae70 0x7f141406d330 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:25.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143015d490 0x7f143014dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:25.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14300d1bf0 0x7f14300caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:25.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 --2- 172.21.15.81:0/1549601250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14300d1080 0x7f14300ca9c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:25.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 -- 172.21.15.81:0/1549601250 >> 172.21.15.81:0/1549601250 conn(0x7f14300d2e50 msgr2=0x7f14300d14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:25.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 -- 172.21.15.81:0/1549601250 shutdown_connections 2023-11-08T16:31:25.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:25.271+0000 7f1437267640 1 -- 172.21.15.81:0/1549601250 wait complete. 2023-11-08T16:31:25.694 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:31:15.514873+0000', 'modified': '2023-11-08T16:31:23.885171+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 42, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5655}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5655': {'gid': 5655, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:reconnect', 'state_seq': 2, 'addr': '172.21.15.81:6837/2942618483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 2942618483}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 2942618483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5, 6], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:31:25.695 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5655:up:reconnect 2023-11-08T16:31:26.697 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:31:26.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.914+0000 7f783527e640 1 Processor -- start 2023-11-08T16:31:26.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.914+0000 7f783527e640 1 -- start start 2023-11-08T16:31:26.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.915+0000 7f783527e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f783015d490 0x7f78300d39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:26.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.915+0000 7f783527e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f78300d1080 0x7f78300d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:26.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.915+0000 7f783527e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78300d1eb0 con 0x7f78300d1a20 2023-11-08T16:31:26.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.915+0000 7f783527e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78300ce890 con 0x7f78300d1080 2023-11-08T16:31:26.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.915+0000 7f783527e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78300ceb70 con 0x7f783015d490 2023-11-08T16:31:26.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.916+0000 7f7834a7d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f78300d1a20 0x7f78300cf820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54528/0 (socket says 172.21.15.81:54528) 2023-11-08T16:31:26.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.916+0000 7f7834a7d640 1 -- 172.21.15.81:0/3516269674 learned_addr learned my addr 172.21.15.81:0/3516269674 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:26.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.916+0000 7f7827fff640 1 --2- 172.21.15.81:0/3516269674 >> v2:172.21.15.93:3300/0 conn(0x7f78300d1080 0x7f78300d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:26.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.916+0000 7f782ffff640 1 --2- 172.21.15.81:0/3516269674 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f783015d490 0x7f78300d39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:26.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.916+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2125028266 0 0) 0x7f78300d1eb0 con 0x7f78300d1a20 2023-11-08T16:31:26.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.916+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f781c003520 con 0x7f78300d1a20 2023-11-08T16:31:26.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.916+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 925645145 0 0) 0x7f781c003520 con 0x7f78300d1a20 2023-11-08T16:31:26.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.916+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f783015d490 msgr2=0x7f78300d39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:26.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 --2- 172.21.15.81:0/3516269674 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f783015d490 0x7f78300d39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:26.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 >> v2:172.21.15.93:3300/0 conn(0x7f78300d1080 msgr2=0x7f78300d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:26.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 --2- 172.21.15.81:0/3516269674 >> v2:172.21.15.93:3300/0 conn(0x7f78300d1080 0x7f78300d14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:26.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78300cee50 con 0x7f78300d1a20 2023-11-08T16:31:26.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f78280019f0 con 0x7f78300d1a20 2023-11-08T16:31:26.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 >> v1:172.21.15.81:6789/0 conn(0x7f78300d1a20 legacy=0x7f78300cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:26.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 --2- 172.21.15.81:0/3516269674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f781c003a50 0x7f781c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:26.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 --2- 172.21.15.81:0/3516269674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f781c0047c0 0x7f781c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:26.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 --2- 172.21.15.81:0/3516269674 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f783015d490 0x7f781c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:26.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78300ce890 con 0x7f781c0047c0 2023-11-08T16:31:26.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78300ceb70 con 0x7f781c003a50 2023-11-08T16:31:26.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781c00b9b0 con 0x7f783015d490 2023-11-08T16:31:26.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f7827fff640 1 --2- 172.21.15.81:0/3516269674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f781c0047c0 0x7f781c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:26.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f7827fff640 1 -- 172.21.15.81:0/3516269674 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f783015d490 msgr2=0x7f781c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:26.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f7827fff640 1 --2- 172.21.15.81:0/3516269674 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f783015d490 0x7f781c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:26.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f7827fff640 1 -- 172.21.15.81:0/3516269674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f781c003a50 msgr2=0x7f781c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:26.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f7827fff640 1 --2- 172.21.15.81:0/3516269674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f781c003a50 0x7f781c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:26.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.917+0000 7f7827fff640 1 -- 172.21.15.81:0/3516269674 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c003520 con 0x7f781c0047c0 2023-11-08T16:31:26.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.918+0000 7f7827fff640 1 --2- 172.21.15.81:0/3516269674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f781c0047c0 0x7f781c004c40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7818009ac0 tx=0x7f781c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a55e3dd906e6a28b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:26.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.919+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781802c710 con 0x7f781c0047c0 2023-11-08T16:31:26.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.919+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781802c870 con 0x7f781c0047c0 2023-11-08T16:31:26.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.919+0000 7f782f7fe640 1 -- 172.21.15.81:0/3516269674 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781802cc00 con 0x7f781c0047c0 2023-11-08T16:31:26.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.919+0000 7f783527e640 1 -- 172.21.15.81:0/3516269674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f781c0047c0 msgr2=0x7f781c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:26.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.919+0000 7f783527e640 1 --2- 172.21.15.81:0/3516269674 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f781c0047c0 0x7f781c004c40 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7818009ac0 tx=0x7f781c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:31:26.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.919+0000 7f7834a7d640 1 -- 172.21.15.81:0/3516269674 reap_dead start 2023-11-08T16:31:26.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.919+0000 7f783527e640 1 -- 172.21.15.81:0/3516269674 shutdown_connections 2023-11-08T16:31:26.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.919+0000 7f783527e640 1 -- 172.21.15.81:0/3516269674 >> 172.21.15.81:0/3516269674 conn(0x7f78300d2e50 msgr2=0x7f78300d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:26.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.920+0000 7f783527e640 1 -- 172.21.15.81:0/3516269674 shutdown_connections 2023-11-08T16:31:26.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.920+0000 7f783527e640 1 -- 172.21.15.81:0/3516269674 wait complete. 2023-11-08T16:31:26.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.921+0000 7f783527e640 1 Processor -- start 2023-11-08T16:31:26.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.922+0000 7f783527e640 1 -- start start 2023-11-08T16:31:26.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f783527e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f781c0047c0 0x7f78300c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:26.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f783527e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78300cbba0 0x7f78301582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:26.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f783527e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7830158a40 0x7f7830158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:26.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f783527e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78301e7d10 con 0x7f78300cbba0 2023-11-08T16:31:26.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f783527e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78300ca460 con 0x7f7830158a40 2023-11-08T16:31:26.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f783527e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7830159400 con 0x7f781c0047c0 2023-11-08T16:31:26.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f7827fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78300cbba0 0x7f78301582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:26.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f7827fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78300cbba0 0x7f78301582d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46160/0 (socket says 172.21.15.81:46160) 2023-11-08T16:31:26.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f7827fff640 1 -- 172.21.15.81:0/2430495751 learned_addr learned my addr 172.21.15.81:0/2430495751 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:26.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f7827fff640 1 -- 172.21.15.81:0/2430495751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f781c0047c0 msgr2=0x7f78300c9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:26.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f7834a7d640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7830158a40 0x7f7830158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:26.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f782ffff640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f781c0047c0 0x7f78300c9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:26.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f7827fff640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f781c0047c0 0x7f78300c9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:26.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f7827fff640 1 -- 172.21.15.81:0/2430495751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7830158a40 msgr2=0x7f7830158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:26.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f7827fff640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7830158a40 0x7f7830158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:26.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.923+0000 7f7827fff640 1 -- 172.21.15.81:0/2430495751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7830158810 con 0x7f78300cbba0 2023-11-08T16:31:26.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.924+0000 7f7827fff640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78300cbba0 0x7f78301582d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f781802c2b0 tx=0x7f7818004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:26.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.924+0000 7f782d7fa640 1 -- 172.21.15.81:0/2430495751 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818003810 con 0x7f78300cbba0 2023-11-08T16:31:26.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.924+0000 7f782d7fa640 1 -- 172.21.15.81:0/2430495751 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781802de90 con 0x7f78300cbba0 2023-11-08T16:31:26.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.924+0000 7f783527e640 1 -- 172.21.15.81:0/2430495751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78300ca670 con 0x7f78300cbba0 2023-11-08T16:31:26.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.924+0000 7f782d7fa640 1 -- 172.21.15.81:0/2430495751 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818003b70 con 0x7f78300cbba0 2023-11-08T16:31:26.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.924+0000 7f783527e640 1 -- 172.21.15.81:0/2430495751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78300cac30 con 0x7f78300cbba0 2023-11-08T16:31:26.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.925+0000 7f783527e640 1 -- 172.21.15.81:0/2430495751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783015eef0 con 0x7f78300cbba0 2023-11-08T16:31:26.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.926+0000 7f782d7fa640 1 -- 172.21.15.81:0/2430495751 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7818003cd0 con 0x7f78300cbba0 2023-11-08T16:31:26.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.927+0000 7f782d7fa640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781006ae70 0x7f781006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:26.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.927+0000 7f782d7fa640 1 -- 172.21.15.81:0/2430495751 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f78180ab8a0 con 0x7f78300cbba0 2023-11-08T16:31:26.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.927+0000 7f782ffff640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781006ae70 0x7f781006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:26.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.928+0000 7f782ffff640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781006ae70 0x7f781006d330 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7820009890 tx=0x7f7820008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:26.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:26.932+0000 7f782d7fa640 1 -- 172.21.15.81:0/2430495751 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7818076610 con 0x7f78300cbba0 2023-11-08T16:31:27.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.157+0000 7f783527e640 1 -- 172.21.15.81:0/2430495751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7830067800 con 0x7f78300cbba0 2023-11-08T16:31:27.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.158+0000 7f782d7fa640 1 -- 172.21.15.81:0/2430495751 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f7818079c30 con 0x7f78300cbba0 2023-11-08T16:31:27.160 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:27.161 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3654712142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3654712142},{"type":"v1","addr":"172.21.15.93:6834","nonce":3654712142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5014,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/66363251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":66363251},{"type":"v1","addr":"172.21.15.93:6835","nonce":66363251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4031527046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4031527046},{"type":"v1","addr":"172.21.15.81:6835","nonce":4031527046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:25.862327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/2942618483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2942618483},{"type":"v1","addr":"172.21.15.81:6837","nonce":2942618483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:31:27.161 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 32 2023-11-08T16:31:27.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 -- 172.21.15.81:0/2430495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781006ae70 msgr2=0x7f781006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:27.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781006ae70 0x7f781006d330 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7820009890 tx=0x7f7820008040 comp rx=0 tx=0).stop 2023-11-08T16:31:27.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 -- 172.21.15.81:0/2430495751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78300cbba0 msgr2=0x7f78301582d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:27.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78300cbba0 0x7f78301582d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f781802c2b0 tx=0x7f7818004730 comp rx=0 tx=0).stop 2023-11-08T16:31:27.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 -- 172.21.15.81:0/2430495751 shutdown_connections 2023-11-08T16:31:27.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781006ae70 0x7f781006d330 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7830158a40 0x7f7830158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78300cbba0 0x7f78301582d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 --2- 172.21.15.81:0/2430495751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f781c0047c0 0x7f78300c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 -- 172.21.15.81:0/2430495751 >> 172.21.15.81:0/2430495751 conn(0x7f78300d2e50 msgr2=0x7f78300b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:27.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 -- 172.21.15.81:0/2430495751 shutdown_connections 2023-11-08T16:31:27.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.161+0000 7f783527e640 1 -- 172.21.15.81:0/2430495751 wait complete. 2023-11-08T16:31:27.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:31:15.514873+0000', 'modified': '2023-11-08T16:31:25.862327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 42, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5655}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5655': {'gid': 5655, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6837/2942618483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 2942618483}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 2942618483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5, 6], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:31:27.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:31:27.584 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T16:31:27.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.802+0000 7fbc341c9640 1 Processor -- start 2023-11-08T16:31:27.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.802+0000 7fbc341c9640 1 -- start start 2023-11-08T16:31:27.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.804+0000 7fbc341c9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbc2c0c90f0 0x7fbc2c0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.804+0000 7fbc341c9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbc2c0b18f0 0x7fbc2c0b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.804+0000 7fbc341c9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc2c06cc60 con 0x7fbc2c15aeb0 2023-11-08T16:31:27.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.804+0000 7fbc341c9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c0cb3e0 con 0x7fbc2c0b18f0 2023-11-08T16:31:27.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.804+0000 7fbc341c9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc2c0c9ac0 con 0x7fbc2c0c90f0 2023-11-08T16:31:27.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.804+0000 7fbc32766640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbc2c0b18f0 0x7fbc2c0b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.804+0000 7fbc32766640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbc2c0b18f0 0x7fbc2c0b1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46174/0 (socket says 172.21.15.81:46174) 2023-11-08T16:31:27.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.804+0000 7fbc32766640 1 -- 172.21.15.81:0/2685545754 learned_addr learned my addr 172.21.15.81:0/2685545754 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:27.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.805+0000 7fbc31764640 1 --2- 172.21.15.81:0/2685545754 >> v2:172.21.15.93:3300/0 conn(0x7fbc2c0c90f0 0x7fbc2c0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.805+0000 7fbc32766640 1 -- 172.21.15.81:0/2685545754 >> v2:172.21.15.93:3300/0 conn(0x7fbc2c0c90f0 msgr2=0x7fbc2c0c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.805+0000 7fbc32766640 1 --2- 172.21.15.81:0/2685545754 >> v2:172.21.15.93:3300/0 conn(0x7fbc2c0c90f0 0x7fbc2c0c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.805+0000 7fbc32766640 1 -- 172.21.15.81:0/2685545754 >> v1:172.21.15.93:6789/0 conn(0x7fbc2c15aeb0 legacy=0x7fbc2c0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.805+0000 7fbc32766640 1 -- 172.21.15.81:0/2685545754 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc2c0cabb0 con 0x7fbc2c0b18f0 2023-11-08T16:31:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.806+0000 7fbc32766640 1 --2- 172.21.15.81:0/2685545754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc2c0b18f0 0x7fbc2c0b1cf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c071e30 tx=0x7fbc2402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54fda3b65c18d27 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.806+0000 7fbc30f63640 1 -- 172.21.15.81:0/2685545754 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2402ca80 con 0x7fbc2c0b18f0 2023-11-08T16:31:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.806+0000 7fbc30f63640 1 -- 172.21.15.81:0/2685545754 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2402cbe0 con 0x7fbc2c0b18f0 2023-11-08T16:31:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.806+0000 7fbc30f63640 1 -- 172.21.15.81:0/2685545754 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc240343f0 con 0x7fbc2c0b18f0 2023-11-08T16:31:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.807+0000 7fbc341c9640 1 -- 172.21.15.81:0/2685545754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc2c0b18f0 msgr2=0x7fbc2c0b1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:27.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.807+0000 7fbc341c9640 1 --2- 172.21.15.81:0/2685545754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc2c0b18f0 0x7fbc2c0b1cf0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c071e30 tx=0x7fbc2402bee0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.807+0000 7fbc341c9640 1 -- 172.21.15.81:0/2685545754 shutdown_connections 2023-11-08T16:31:27.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.807+0000 7fbc341c9640 1 --2- 172.21.15.81:0/2685545754 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc2c0b18f0 0x7fbc2c0b1cf0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.807+0000 7fbc341c9640 1 --2- 172.21.15.81:0/2685545754 >> v2:172.21.15.93:3300/0 conn(0x7fbc2c0c90f0 0x7fbc2c0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.807+0000 7fbc341c9640 1 -- 172.21.15.81:0/2685545754 >> 172.21.15.81:0/2685545754 conn(0x7fbc2c0ceaa0 msgr2=0x7fbc2c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:27.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.807+0000 7fbc341c9640 1 -- 172.21.15.81:0/2685545754 shutdown_connections 2023-11-08T16:31:27.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.807+0000 7fbc341c9640 1 -- 172.21.15.81:0/2685545754 wait complete. 2023-11-08T16:31:27.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.809+0000 7fbc341c9640 1 Processor -- start 2023-11-08T16:31:27.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.809+0000 7fbc341c9640 1 -- start start 2023-11-08T16:31:27.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc341c9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc2c0b18f0 0x7fbc2c14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc341c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc2c0c90f0 0x7fbc2c1f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc341c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbc2c15aeb0 0x7fbc2c0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc341c9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c155b20 con 0x7fbc2c0c90f0 2023-11-08T16:31:27.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc341c9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0cc840 con 0x7fbc2c0b18f0 2023-11-08T16:31:27.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc341c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c0ccb20 con 0x7fbc2c15aeb0 2023-11-08T16:31:27.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc31764640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc2c0c90f0 0x7fbc2c1f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc31764640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc2c0c90f0 0x7fbc2c1f3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46186/0 (socket says 172.21.15.81:46186) 2023-11-08T16:31:27.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc31764640 1 -- 172.21.15.81:0/1499485089 learned_addr learned my addr 172.21.15.81:0/1499485089 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:27.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc32766640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbc2c15aeb0 0x7fbc2c0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc31764640 1 -- 172.21.15.81:0/1499485089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbc2c15aeb0 msgr2=0x7fbc2c0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:27.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc31f65640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc2c0b18f0 0x7fbc2c14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc31764640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbc2c15aeb0 0x7fbc2c0cd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc31764640 1 -- 172.21.15.81:0/1499485089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc2c0b18f0 msgr2=0x7fbc2c14f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:27.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc31764640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc2c0b18f0 0x7fbc2c14f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.810+0000 7fbc31764640 1 -- 172.21.15.81:0/1499485089 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc24009b30 con 0x7fbc2c0c90f0 2023-11-08T16:31:27.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.811+0000 7fbc31764640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc2c0c90f0 0x7fbc2c1f3070 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d800 tx=0x7fbc1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:27.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.811+0000 7fbc227fc640 1 -- 172.21.15.81:0/1499485089 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00b6c0 con 0x7fbc2c0c90f0 2023-11-08T16:31:27.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.811+0000 7fbc227fc640 1 -- 172.21.15.81:0/1499485089 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc1c00b820 con 0x7fbc2c0c90f0 2023-11-08T16:31:27.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.811+0000 7fbc341c9640 1 -- 172.21.15.81:0/1499485089 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c0cce00 con 0x7fbc2c0c90f0 2023-11-08T16:31:27.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.811+0000 7fbc227fc640 1 -- 172.21.15.81:0/1499485089 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00bbb0 con 0x7fbc2c0c90f0 2023-11-08T16:31:27.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.811+0000 7fbc341c9640 1 -- 172.21.15.81:0/1499485089 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c14fe10 con 0x7fbc2c0c90f0 2023-11-08T16:31:27.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.814+0000 7fbc227fc640 1 -- 172.21.15.81:0/1499485089 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbc1c00bd10 con 0x7fbc2c0c90f0 2023-11-08T16:31:27.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.815+0000 7fbc227fc640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc1006ae70 0x7fbc1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.815+0000 7fbc227fc640 1 -- 172.21.15.81:0/1499485089 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7fbc1c089f00 con 0x7fbc2c0c90f0 2023-11-08T16:31:27.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.815+0000 7fbc31f65640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc1006ae70 0x7fbc1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.816+0000 7fbc36e39740 1 Processor -- start 2023-11-08T16:31:27.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.816+0000 7fbc36e39740 1 -- start start 2023-11-08T16:31:27.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.816+0000 7fbc31f65640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc1006ae70 0x7fbc1006d330 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800a710 tx=0x7fbc28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:27.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.817+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa270ec0 0x5601fa243dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.817+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa2712a0 0x5601fa273c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.817+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa244300 0x5601fa274190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.817+0000 7fbc36e39740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fa1b67e0 con 0x5601fa270ec0 2023-11-08T16:31:27.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.817+0000 7fbc36e39740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fa1b3ed0 con 0x5601fa244300 2023-11-08T16:31:27.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.817+0000 7fbc36e39740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601fa1af7c0 con 0x5601fa2712a0 2023-11-08T16:31:27.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.817+0000 7fbc31f65640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa270ec0 0x5601fa243dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.817+0000 7fbc31f65640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa270ec0 0x5601fa243dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46198/0 (socket says 172.21.15.81:46198) 2023-11-08T16:31:27.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.817+0000 7fbc31f65640 1 -- 172.21.15.81:0/3460737010 learned_addr learned my addr 172.21.15.81:0/3460737010 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:27.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.817+0000 7fbc31f65640 1 -- 172.21.15.81:0/3460737010 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa2712a0 msgr2=0x5601fa273c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:27.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.817+0000 7fbc31f65640 1 --2- 172.21.15.81:0/3460737010 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa2712a0 0x5601fa273c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.818+0000 7fbc31f65640 1 -- 172.21.15.81:0/3460737010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa244300 msgr2=0x5601fa274190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:27.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.818+0000 7fbc31f65640 1 --2- 172.21.15.81:0/3460737010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa244300 0x5601fa274190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.818+0000 7fbc31f65640 1 -- 172.21.15.81:0/3460737010 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601fa182b10 con 0x5601fa270ec0 2023-11-08T16:31:27.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.818+0000 7fbc31f65640 1 --2- 172.21.15.81:0/3460737010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa270ec0 0x5601fa243dc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800c930 tx=0x7fbc2800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a01028b88987b26 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:27.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.818+0000 7fbc0f7fe640 1 -- 172.21.15.81:0/3460737010 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc28013b80 con 0x5601fa270ec0 2023-11-08T16:31:27.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.818+0000 7fbc0f7fe640 1 -- 172.21.15.81:0/3460737010 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc28013ce0 con 0x5601fa270ec0 2023-11-08T16:31:27.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.818+0000 7fbc0f7fe640 1 -- 172.21.15.81:0/3460737010 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2801d630 con 0x5601fa270ec0 2023-11-08T16:31:27.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.819+0000 7fbc36e39740 1 -- 172.21.15.81:0/3460737010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa270ec0 msgr2=0x5601fa243dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:27.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.819+0000 7fbc36e39740 1 --2- 172.21.15.81:0/3460737010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa270ec0 0x5601fa243dc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800c930 tx=0x7fbc2800f000 comp rx=0 tx=0).stop 2023-11-08T16:31:27.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.819+0000 7fbc36e39740 1 -- 172.21.15.81:0/3460737010 shutdown_connections 2023-11-08T16:31:27.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.819+0000 7fbc36e39740 1 --2- 172.21.15.81:0/3460737010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa244300 0x5601fa274190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.819+0000 7fbc36e39740 1 --2- 172.21.15.81:0/3460737010 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa2712a0 0x5601fa273c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.819+0000 7fbc36e39740 1 --2- 172.21.15.81:0/3460737010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa270ec0 0x5601fa243dc0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.819+0000 7fbc36e39740 1 -- 172.21.15.81:0/3460737010 >> 172.21.15.81:0/3460737010 conn(0x5601fa2460d0 msgr2=0x5601fa28ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:27.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.819+0000 7fbc36e39740 1 -- 172.21.15.81:0/3460737010 shutdown_connections 2023-11-08T16:31:27.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.819+0000 7fbc36e39740 1 -- 172.21.15.81:0/3460737010 wait complete. 2023-11-08T16:31:27.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.820+0000 7fbc36e39740 1 -- start start 2023-11-08T16:31:27.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.820+0000 7fbc36e39740 1 Processor -- start 2023-11-08T16:31:27.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.821+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa270ec0 0x5601fa25b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.821+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa2712a0 0x5601fa29f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.821+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.821+0000 7fbc36e39740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fa14cc40 con 0x5601fa25a980 2023-11-08T16:31:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.821+0000 7fbc36e39740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fa14c8e0 con 0x5601fa2712a0 2023-11-08T16:31:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.821+0000 7fbc36e39740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601f9f12f20 con 0x5601fa270ec0 2023-11-08T16:31:27.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.821+0000 7fbc36e39740 10 client.0 _ll_register_callbacks cb 0x5601fa16ecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:31:27.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.821+0000 7fbc32766640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.821+0000 7fbc32766640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2909e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46200/0 (socket says 172.21.15.81:46200) 2023-11-08T16:31:27.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.821+0000 7fbc32766640 1 -- 172.21.15.81:0/4069162446 learned_addr learned my addr 172.21.15.81:0/4069162446 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:27.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc31764640 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa2712a0 0x5601fa29f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc31f65640 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa270ec0 0x5601fa25b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc32766640 1 -- 172.21.15.81:0/4069162446 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa270ec0 msgr2=0x5601fa25b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:27.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc32766640 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa270ec0 0x5601fa25b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc32766640 1 -- 172.21.15.81:0/4069162446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa2712a0 msgr2=0x5601fa29f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:27.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc32766640 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa2712a0 0x5601fa29f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc32766640 1 -- 172.21.15.81:0/4069162446 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2800c5b0 con 0x5601fa25a980 2023-11-08T16:31:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc31764640 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa2712a0 0x5601fa29f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc32766640 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2909e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009f40 tx=0x7fbc24004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbbb4669392d76d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:31:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc0cff9640 1 -- 172.21.15.81:0/4069162446 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24034b30 con 0x5601fa25a980 2023-11-08T16:31:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc0cff9640 1 -- 172.21.15.81:0/4069162446 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7fbc24034c90 con 0x5601fa25a980 2023-11-08T16:31:27.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.822+0000 7fbc36e39740 1 -- 172.21.15.81:0/4069162446 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5601fa183c80 con 0x5601fa25a980 2023-11-08T16:31:27.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc0cff9640 1 -- 172.21.15.81:0/4069162446 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc24034e50 con 0x5601fa25a980 2023-11-08T16:31:27.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc36e39740 10 client.5700 fetch_fsmap learned FSMap version 32 2023-11-08T16:31:27.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc36e39740 1 -- 172.21.15.81:0/4069162446 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5601fa1ade40 con 0x5601fa25a980 2023-11-08T16:31:27.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc0cff9640 1 -- 172.21.15.81:0/4069162446 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5601fa183c80 con 0x5601fa25a980 2023-11-08T16:31:27.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc0cff9640 1 -- 172.21.15.81:0/4069162446 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fbc2403c040 con 0x5601fa25a980 2023-11-08T16:31:27.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc36e39740 10 client.5700 fetch_fsmap finished waiting for FSMap version 32 2023-11-08T16:31:27.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc36e39740 10 client.5700 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:31:27.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc36e39740 20 client.5700 populate_metadata read hostname 'smithi081' 2023-11-08T16:31:27.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc36e39740 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5601fa25d5e0 0x5601fa25d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc36e39740 4 client.5700 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:31:27.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc36e39740 1 -- 172.21.15.81:0/4069162446 --> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5601fa1a87a0 con 0x5601fa25d5e0 2023-11-08T16:31:27.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.823+0000 7fbc31f65640 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5601fa25d5e0 0x5601fa25d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.824+0000 7fbc31f65640 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5601fa25d5e0 0x5601fa25d9c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4b4b1f49374385 server_cookie=8092b61f3d9cd163 in_seq=0 out_seq=0 2023-11-08T16:31:27.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.824+0000 7fbc0cff9640 10 client.5700 ms_handle_connect on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:27.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc0cff9640 1 -- 172.21.15.81:0/4069162446 <== mds.0 v2:172.21.15.81:6836/2942618483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5601fa1a87a0 con 0x5601fa25d5e0 2023-11-08T16:31:27.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc0cff9640 10 client.5700 handle_command_reply: tid=0 2023-11-08T16:31:27.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 1 client.5700 shutdown 2023-11-08T16:31:27.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 1 -- 172.21.15.81:0/4069162446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 msgr2=0x5601fa2909e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:27.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2909e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009f40 tx=0x7fbc24004a40 comp rx=0 tx=0).stop 2023-11-08T16:31:27.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 1 -- 172.21.15.81:0/4069162446 shutdown_connections 2023-11-08T16:31:27.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5601fa25d5e0 0x5601fa25d9c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2909e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa2712a0 0x5601fa29f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 1 --2- 172.21.15.81:0/4069162446 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa270ec0 0x5601fa25b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 1 -- 172.21.15.81:0/4069162446 >> 172.21.15.81:0/4069162446 conn(0x5601fa244300 msgr2=0x5601fa292160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:27.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc0cff9640 0 client.5700 ms_handle_reset on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:27.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 1 -- 172.21.15.81:0/4069162446 shutdown_connections 2023-11-08T16:31:27.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 1 -- 172.21.15.81:0/4069162446 wait complete. 2023-11-08T16:31:27.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.825+0000 7fbc36e39740 20 client.5700 trim_cache size 0 max 16384 2023-11-08T16:31:27.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.842+0000 7fbc36e39740 1 Processor -- start 2023-11-08T16:31:27.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.842+0000 7fbc36e39740 1 -- start start 2023-11-08T16:31:27.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.843+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa25a980 0x5601fa270150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.843+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa270ec0 0x5601fa292350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.843+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa2712a0 0x5601fa2a0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.843+0000 7fbc36e39740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fa2483d0 con 0x5601fa2712a0 2023-11-08T16:31:27.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.843+0000 7fbc36e39740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fa285690 con 0x5601fa25a980 2023-11-08T16:31:27.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.843+0000 7fbc36e39740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601fa23fb90 con 0x5601fa270ec0 2023-11-08T16:31:27.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.844+0000 7fbc32766640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa2712a0 0x5601fa2a0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.844+0000 7fbc32766640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa2712a0 0x5601fa2a0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46202/0 (socket says 172.21.15.81:46202) 2023-11-08T16:31:27.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.844+0000 7fbc32766640 1 -- 172.21.15.81:0/2029923210 learned_addr learned my addr 172.21.15.81:0/2029923210 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:27.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.844+0000 7fbc32766640 1 -- 172.21.15.81:0/2029923210 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa270ec0 msgr2=0x5601fa292350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:27.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.844+0000 7fbc32766640 1 --2- 172.21.15.81:0/2029923210 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa270ec0 0x5601fa292350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.844+0000 7fbc32766640 1 -- 172.21.15.81:0/2029923210 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa25a980 msgr2=0x5601fa270150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:27.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.844+0000 7fbc32766640 1 --2- 172.21.15.81:0/2029923210 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa25a980 0x5601fa270150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.844+0000 7fbc32766640 1 -- 172.21.15.81:0/2029923210 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5601fa14bb90 con 0x5601fa2712a0 2023-11-08T16:31:27.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.844+0000 7fbc32766640 1 --2- 172.21.15.81:0/2029923210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa2712a0 0x5601fa2a0840 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbc2402bf20 tx=0x7fbc24040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37fe1f68b9888c16 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:27.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.844+0000 7fbc0ffff640 1 -- 172.21.15.81:0/2029923210 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24058070 con 0x5601fa2712a0 2023-11-08T16:31:27.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.845+0000 7fbc0ffff640 1 -- 172.21.15.81:0/2029923210 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc240496d0 con 0x5601fa2712a0 2023-11-08T16:31:27.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.845+0000 7fbc0ffff640 1 -- 172.21.15.81:0/2029923210 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc24049ac0 con 0x5601fa2712a0 2023-11-08T16:31:27.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.845+0000 7fbc36e39740 1 -- 172.21.15.81:0/2029923210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa2712a0 msgr2=0x5601fa2a0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:27.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.845+0000 7fbc36e39740 1 --2- 172.21.15.81:0/2029923210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa2712a0 0x5601fa2a0840 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fbc2402bf20 tx=0x7fbc24040000 comp rx=0 tx=0).stop 2023-11-08T16:31:27.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.845+0000 7fbc36e39740 1 -- 172.21.15.81:0/2029923210 shutdown_connections 2023-11-08T16:31:27.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.845+0000 7fbc36e39740 1 --2- 172.21.15.81:0/2029923210 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa2712a0 0x5601fa2a0840 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.845+0000 7fbc36e39740 1 --2- 172.21.15.81:0/2029923210 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa270ec0 0x5601fa292350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.845+0000 7fbc36e39740 1 --2- 172.21.15.81:0/2029923210 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa25a980 0x5601fa270150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.845+0000 7fbc36e39740 1 -- 172.21.15.81:0/2029923210 >> 172.21.15.81:0/2029923210 conn(0x5601fa244300 msgr2=0x5601fa26ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:27.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.845+0000 7fbc36e39740 1 -- 172.21.15.81:0/2029923210 shutdown_connections 2023-11-08T16:31:27.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.845+0000 7fbc36e39740 1 -- 172.21.15.81:0/2029923210 wait complete. 2023-11-08T16:31:27.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.846+0000 7fbc36e39740 1 -- start start 2023-11-08T16:31:27.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.846+0000 7fbc36e39740 1 Processor -- start 2023-11-08T16:31:27.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.847+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2fd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.847+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa270ec0 0x5601fa277160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.847+0000 7fbc36e39740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa2712a0 0x5601fa2f9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.847+0000 7fbc36e39740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fa2f9db0 con 0x5601fa25a980 2023-11-08T16:31:27.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.847+0000 7fbc36e39740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5601fa285500 con 0x5601fa270ec0 2023-11-08T16:31:27.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.847+0000 7fbc36e39740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5601fa2fd5d0 con 0x5601fa2712a0 2023-11-08T16:31:27.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.847+0000 7fbc36e39740 10 client.0 _ll_register_callbacks cb 0x7fbc24009d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:31:27.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.847+0000 7fbc31f65640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2fd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.847+0000 7fbc31f65640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2fd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46212/0 (socket says 172.21.15.81:46212) 2023-11-08T16:31:27.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.847+0000 7fbc31f65640 1 -- 172.21.15.81:0/3618527481 learned_addr learned my addr 172.21.15.81:0/3618527481 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:27.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc31764640 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa270ec0 0x5601fa277160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc32766640 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa2712a0 0x5601fa2f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc31f65640 1 -- 172.21.15.81:0/3618527481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa2712a0 msgr2=0x5601fa2f9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:27.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc31f65640 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa2712a0 0x5601fa2f9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc31f65640 1 -- 172.21.15.81:0/3618527481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa270ec0 msgr2=0x5601fa277160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:27.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc31f65640 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa270ec0 0x5601fa277160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc31f65640 1 -- 172.21.15.81:0/3618527481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5601fa1ade40 con 0x5601fa25a980 2023-11-08T16:31:27.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc31764640 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa270ec0 0x5601fa277160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:27.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc31f65640 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2fd090 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbc2801f040 tx=0x7fbc28004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981d9d41200a6103 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:27.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:31:27.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc0cff9640 1 -- 172.21.15.81:0/3618527481 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2802c040 con 0x5601fa25a980 2023-11-08T16:31:27.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc0cff9640 1 -- 172.21.15.81:0/3618527481 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7fbc2802b040 con 0x5601fa25a980 2023-11-08T16:31:27.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.848+0000 7fbc36e39740 1 -- 172.21.15.81:0/3618527481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbc28013ce0 con 0x5601fa25a980 2023-11-08T16:31:27.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc0cff9640 1 -- 172.21.15.81:0/3618527481 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc28013b10 con 0x5601fa25a980 2023-11-08T16:31:27.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc36e39740 10 client.5706 fetch_fsmap learned FSMap version 32 2023-11-08T16:31:27.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc36e39740 1 -- 172.21.15.81:0/3618527481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5601fa171180 con 0x5601fa25a980 2023-11-08T16:31:27.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc0cff9640 1 -- 172.21.15.81:0/3618527481 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc28013ce0 con 0x5601fa25a980 2023-11-08T16:31:27.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc0cff9640 1 -- 172.21.15.81:0/3618527481 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fbc28031630 con 0x5601fa25a980 2023-11-08T16:31:27.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc36e39740 10 client.5706 fetch_fsmap finished waiting for FSMap version 32 2023-11-08T16:31:27.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc36e39740 10 client.5706 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:31:27.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc36e39740 20 client.5706 populate_metadata read hostname 'smithi081' 2023-11-08T16:31:27.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc36e39740 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5601fa248ca0 0x5601fa249080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:27.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc36e39740 4 client.5706 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:31:27.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc36e39740 1 -- 172.21.15.81:0/3618527481 --> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5601fa1a84c0 con 0x5601fa248ca0 2023-11-08T16:31:27.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.849+0000 7fbc31764640 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5601fa248ca0 0x5601fa249080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:27.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc31764640 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5601fa248ca0 0x5601fa249080 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0877ef39d24b346 server_cookie=128b9937cbc246dd in_seq=0 out_seq=0 2023-11-08T16:31:27.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc0cff9640 10 client.5706 ms_handle_connect on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:27.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc0cff9640 1 -- 172.21.15.81:0/3618527481 <== mds.0 v2:172.21.15.81:6836/2942618483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5601fa1a84c0 con 0x5601fa248ca0 2023-11-08T16:31:27.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc0cff9640 10 client.5706 handle_command_reply: tid=0 2023-11-08T16:31:27.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc36e39740 1 client.5706 shutdown 2023-11-08T16:31:27.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc36e39740 1 -- 172.21.15.81:0/3618527481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 msgr2=0x5601fa2fd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:27.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc36e39740 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2fd090 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fbc2801f040 tx=0x7fbc28004690 comp rx=0 tx=0).stop 2023-11-08T16:31:27.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc36e39740 1 -- 172.21.15.81:0/3618527481 shutdown_connections 2023-11-08T16:31:27.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc36e39740 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5601fa248ca0 0x5601fa249080 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc36e39740 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5601fa2712a0 0x5601fa2f9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc36e39740 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5601fa270ec0 0x5601fa277160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc36e39740 1 --2- 172.21.15.81:0/3618527481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5601fa25a980 0x5601fa2fd090 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc36e39740 1 -- 172.21.15.81:0/3618527481 >> 172.21.15.81:0/3618527481 conn(0x5601fa244300 msgr2=0x5601fa25cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:27.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc0cff9640 0 client.5706 ms_handle_reset on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:27.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc36e39740 1 -- 172.21.15.81:0/3618527481 shutdown_connections 2023-11-08T16:31:27.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.850+0000 7fbc36e39740 1 -- 172.21.15.81:0/3618527481 wait complete. 2023-11-08T16:31:27.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc36e39740 20 client.5706 trim_cache size 0 max 16384 2023-11-08T16:31:27.865 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:31:27.865 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:31:27.866 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:31:27.866 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 5655, 2023-11-08T16:31:27.866 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:31:27.866 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:31:27.867 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:31:27.867 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 3.9934638730000001, 2023-11-08T16:31:27.867 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 32, 2023-11-08T16:31:27.867 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 44, 2023-11-08T16:31:27.867 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 44, 2023-11-08T16:31:27.867 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 4.7518300160000004 2023-11-08T16:31:27.868 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:31:27.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 -- 172.21.15.81:0/1499485089 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc1006ae70 msgr2=0x7fbc1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:27.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc1006ae70 0x7fbc1006d330 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbc2800a710 tx=0x7fbc28008040 comp rx=0 tx=0).stop 2023-11-08T16:31:27.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 -- 172.21.15.81:0/1499485089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc2c0c90f0 msgr2=0x7fbc2c1f3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:27.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc2c0c90f0 0x7fbc2c1f3070 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d800 tx=0x7fbc1c00dcc0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 -- 172.21.15.81:0/1499485089 shutdown_connections 2023-11-08T16:31:27.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc1006ae70 0x7fbc1006d330 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbc2c15aeb0 0x7fbc2c0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbc2c0c90f0 0x7fbc2c1f3070 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 --2- 172.21.15.81:0/1499485089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbc2c0b18f0 0x7fbc2c14f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:27.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 -- 172.21.15.81:0/1499485089 >> 172.21.15.81:0/1499485089 conn(0x7fbc2c0ceaa0 msgr2=0x7fbc2c15a0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:27.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 -- 172.21.15.81:0/1499485089 shutdown_connections 2023-11-08T16:31:27.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:27.851+0000 7fbc341c9640 1 -- 172.21.15.81:0/1499485089 wait complete. 2023-11-08T16:31:27.872 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:31:28.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.080+0000 7efc05390640 1 Processor -- start 2023-11-08T16:31:28.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.080+0000 7efc05390640 1 -- start start 2023-11-08T16:31:28.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.082+0000 7efc05390640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc000adb40 0x7efc000c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.082+0000 7efc05390640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efc00165940 0x7efc0006b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.082+0000 7efc05390640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc000ce3f0 con 0x7efc000adf20 2023-11-08T16:31:28.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.082+0000 7efc05390640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc000cb340 con 0x7efc00165940 2023-11-08T16:31:28.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.082+0000 7efc05390640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc000cac80 con 0x7efc000adb40 2023-11-08T16:31:28.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.083+0000 7efbff7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efc00165940 0x7efc0006b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:28.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.083+0000 7efbf67fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7efc000adf20 0x7efc00165230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54544/0 (socket says 172.21.15.81:54544) 2023-11-08T16:31:28.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.083+0000 7efbff7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efc00165940 0x7efc0006b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46228/0 (socket says 172.21.15.81:46228) 2023-11-08T16:31:28.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.083+0000 7efbf67fc640 1 -- 172.21.15.81:0/3804376687 learned_addr learned my addr 172.21.15.81:0/3804376687 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:28.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.083+0000 7efbfeffd640 1 --2- 172.21.15.81:0/3804376687 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc000adb40 0x7efc000c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:28.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.083+0000 7efbff7fe640 1 -- 172.21.15.81:0/3804376687 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc000adb40 msgr2=0x7efc000c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:28.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.083+0000 7efbff7fe640 1 --2- 172.21.15.81:0/3804376687 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc000adb40 0x7efc000c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.083+0000 7efbff7fe640 1 -- 172.21.15.81:0/3804376687 >> v1:172.21.15.81:6789/0 conn(0x7efc000adf20 legacy=0x7efc00165230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.083+0000 7efbff7fe640 1 -- 172.21.15.81:0/3804376687 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc000caf60 con 0x7efc00165940 2023-11-08T16:31:28.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.084+0000 7efbff7fe640 1 --2- 172.21.15.81:0/3804376687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc00165940 0x7efc0006b910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009d90 tx=0x7efbf802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993bf9697bf97db8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:28.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.085+0000 7efbfe7fc640 1 -- 172.21.15.81:0/3804376687 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf802cbd0 con 0x7efc00165940 2023-11-08T16:31:28.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.085+0000 7efbfe7fc640 1 -- 172.21.15.81:0/3804376687 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf802cd30 con 0x7efc00165940 2023-11-08T16:31:28.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.085+0000 7efbfe7fc640 1 -- 172.21.15.81:0/3804376687 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf802cea0 con 0x7efc00165940 2023-11-08T16:31:28.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.085+0000 7efc05390640 1 -- 172.21.15.81:0/3804376687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc00165940 msgr2=0x7efc0006b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.085+0000 7efc05390640 1 --2- 172.21.15.81:0/3804376687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc00165940 0x7efc0006b910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efbf8009d90 tx=0x7efbf802be20 comp rx=0 tx=0).stop 2023-11-08T16:31:28.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.085+0000 7efc05390640 1 -- 172.21.15.81:0/3804376687 shutdown_connections 2023-11-08T16:31:28.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.085+0000 7efc05390640 1 --2- 172.21.15.81:0/3804376687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc00165940 0x7efc0006b910 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.085+0000 7efc05390640 1 --2- 172.21.15.81:0/3804376687 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc000adb40 0x7efc000c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.085+0000 7efc05390640 1 -- 172.21.15.81:0/3804376687 >> 172.21.15.81:0/3804376687 conn(0x7efc000cb490 msgr2=0x7efc000cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:28.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.085+0000 7efc05390640 1 -- 172.21.15.81:0/3804376687 shutdown_connections 2023-11-08T16:31:28.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.085+0000 7efc05390640 1 -- 172.21.15.81:0/3804376687 wait complete. 2023-11-08T16:31:28.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.087+0000 7efc05390640 1 Processor -- start 2023-11-08T16:31:28.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.087+0000 7efc05390640 1 -- start start 2023-11-08T16:31:28.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.088+0000 7efc05390640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc000adb40 0x7efc00161020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.088+0000 7efc05390640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc000adf20 0x7efc001633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.088+0000 7efc05390640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc00165940 0x7efc000c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.088+0000 7efc05390640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc00161560 con 0x7efc000adb40 2023-11-08T16:31:28.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.088+0000 7efc05390640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc000c3630 con 0x7efc000adf20 2023-11-08T16:31:28.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.088+0000 7efc05390640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc000c3910 con 0x7efc00165940 2023-11-08T16:31:28.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.089+0000 7efbfeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc000adb40 0x7efc00161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:28.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.089+0000 7efbfeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc000adb40 0x7efc00161020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46230/0 (socket says 172.21.15.81:46230) 2023-11-08T16:31:28.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.089+0000 7efbfeffd640 1 -- 172.21.15.81:0/91634901 learned_addr learned my addr 172.21.15.81:0/91634901 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:28.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.089+0000 7efbfeffd640 1 -- 172.21.15.81:0/91634901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc00165940 msgr2=0x7efc000c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.089+0000 7efbfeffd640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc00165940 0x7efc000c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.089+0000 7efbfeffd640 1 -- 172.21.15.81:0/91634901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc000adf20 msgr2=0x7efc001633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.089+0000 7efbf67fc640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc000adf20 0x7efc001633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:28.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.089+0000 7efbfeffd640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc000adf20 0x7efc001633f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.089+0000 7efbfeffd640 1 -- 172.21.15.81:0/91634901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf8009a70 con 0x7efc000adb40 2023-11-08T16:31:28.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.089+0000 7efbf67fc640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc000adf20 0x7efc001633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:28.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.089+0000 7efbfeffd640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc000adb40 0x7efc00161020 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efbf000d570 tx=0x7efbf000da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:28.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.090+0000 7efbf7fff640 1 -- 172.21.15.81:0/91634901 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf00125f0 con 0x7efc000adb40 2023-11-08T16:31:28.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.090+0000 7efbf7fff640 1 -- 172.21.15.81:0/91634901 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf0012750 con 0x7efc000adb40 2023-11-08T16:31:28.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.090+0000 7efbf7fff640 1 -- 172.21.15.81:0/91634901 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf0012b00 con 0x7efc000adb40 2023-11-08T16:31:28.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.090+0000 7efc05390640 1 -- 172.21.15.81:0/91634901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc000c3bf0 con 0x7efc000adb40 2023-11-08T16:31:28.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.090+0000 7efc05390640 1 -- 172.21.15.81:0/91634901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc000c41b0 con 0x7efc000adb40 2023-11-08T16:31:28.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.091+0000 7efc05390640 1 -- 172.21.15.81:0/91634901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc000c4340 con 0x7efc000adb40 2023-11-08T16:31:28.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.092+0000 7efbf7fff640 1 -- 172.21.15.81:0/91634901 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efbf0012c60 con 0x7efc000adb40 2023-11-08T16:31:28.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.093+0000 7efbf7fff640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efbe006ae70 0x7efbe006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.093+0000 7efbf7fff640 1 -- 172.21.15.81:0/91634901 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7efbf0089d00 con 0x7efc000adb40 2023-11-08T16:31:28.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.093+0000 7efbf67fc640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efbe006ae70 0x7efbe006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:28.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.094+0000 7efbf67fc640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efbe006ae70 0x7efbe006d330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efc000cd0c0 tx=0x7efbe8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:28.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.097+0000 7efbf7fff640 1 -- 172.21.15.81:0/91634901 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efbf0054b00 con 0x7efc000adb40 2023-11-08T16:31:28.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.326+0000 7efc05390640 1 -- 172.21.15.81:0/91634901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efc0004ece0 con 0x7efc000adb40 2023-11-08T16:31:28.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.327+0000 7efbf7fff640 1 -- 172.21.15.81:0/91634901 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4910 (secure 0 0 0) 0x7efbf0058120 con 0x7efc000adb40 2023-11-08T16:31:28.329 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:31:28.330 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3654712142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3654712142},{"type":"v1","addr":"172.21.15.93:6834","nonce":3654712142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5014,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/66363251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":66363251},{"type":"v1","addr":"172.21.15.93:6835","nonce":66363251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4031527046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4031527046},{"type":"v1","addr":"172.21.15.81:6835","nonce":4031527046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:25.862327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/2942618483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2942618483},{"type":"v1","addr":"172.21.15.81:6837","nonce":2942618483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:31:28.330 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 32 2023-11-08T16:31:28.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.330+0000 7efc05390640 1 -- 172.21.15.81:0/91634901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efbe006ae70 msgr2=0x7efbe006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.330+0000 7efc05390640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efbe006ae70 0x7efbe006d330 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efc000cd0c0 tx=0x7efbe8009040 comp rx=0 tx=0).stop 2023-11-08T16:31:28.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.330+0000 7efc05390640 1 -- 172.21.15.81:0/91634901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc000adb40 msgr2=0x7efc00161020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.330+0000 7efc05390640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc000adb40 0x7efc00161020 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efbf000d570 tx=0x7efbf000da30 comp rx=0 tx=0).stop 2023-11-08T16:31:28.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.330+0000 7efc05390640 1 -- 172.21.15.81:0/91634901 shutdown_connections 2023-11-08T16:31:28.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.330+0000 7efc05390640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efbe006ae70 0x7efbe006d330 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.330+0000 7efc05390640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc00165940 0x7efc000c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.330+0000 7efc05390640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc000adf20 0x7efc001633f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.330+0000 7efc05390640 1 --2- 172.21.15.81:0/91634901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc000adb40 0x7efc00161020 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.330+0000 7efc05390640 1 -- 172.21.15.81:0/91634901 >> 172.21.15.81:0/91634901 conn(0x7efc000cb490 msgr2=0x7efc000cc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:28.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.331+0000 7efc05390640 1 -- 172.21.15.81:0/91634901 shutdown_connections 2023-11-08T16:31:28.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.331+0000 7efc05390640 1 -- 172.21.15.81:0/91634901 wait complete. 2023-11-08T16:31:28.352 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 32, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-08T16:31:15.514873+0000", "damaged": [], "data_pools": [ 5, 6 ], "enabled": true, "epoch": 32, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_5655": { "addr": "172.21.15.81:6837/2942618483", "addrs": { "addrvec": [ { "addr": "172.21.15.81:6836", "nonce": 2942618483, "type": "v2" }, { "addr": "172.21.15.81:6837", "nonce": 2942618483, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5655, "incarnation": 29, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 42, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 4, "modified": "2023-11-08T16:31:25.862327+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5655 } } } ], "standbys": [ { "addr": "172.21.15.93:6834/3654712142", "addrs": { "addrvec": [ { "addr": "172.21.15.93:6832", "nonce": 3654712142, "type": "v2" }, { "addr": "172.21.15.93:6834", "nonce": 3654712142, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 29, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 4787, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.93:6835/66363251", "addrs": { "addrvec": [ { "addr": "172.21.15.93:6833", "nonce": 66363251, "type": "v2" }, { "addr": "172.21.15.93:6835", "nonce": 66363251, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 30, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5014, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.81:6835/4031527046", "addrs": { "addrvec": [ { "addr": "172.21.15.81:6834", "nonce": 4031527046, "type": "v2" }, { "addr": "172.21.15.81:6835", "nonce": 4031527046, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 28, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5652, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-08T16:31:28.352 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:31:28.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.555+0000 7f6ec0300640 1 Processor -- start 2023-11-08T16:31:28.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.555+0000 7f6ec0300640 1 -- start start 2023-11-08T16:31:28.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.557+0000 7f6ec0300640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6eb80cc5e0 0x7f6eb80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.557+0000 7f6ec0300640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6eb80ccfb0 0x7f6eb8070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.557+0000 7f6ec0300640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb80c9a70 con 0x7f6eb80adcd0 2023-11-08T16:31:28.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.557+0000 7f6ec0300640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb80cc3e0 con 0x7f6eb80ccfb0 2023-11-08T16:31:28.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.557+0000 7f6ec0300640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb806d070 con 0x7f6eb80cc5e0 2023-11-08T16:31:28.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.557+0000 7f6ebe89d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6eb80ccfb0 0x7f6eb8070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:28.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.557+0000 7f6ebe09c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f6eb80adcd0 0x7f6eb80ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54558/0 (socket says 172.21.15.81:54558) 2023-11-08T16:31:28.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.557+0000 7f6ebe89d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6eb80ccfb0 0x7f6eb8070050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46236/0 (socket says 172.21.15.81:46236) 2023-11-08T16:31:28.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.557+0000 7f6ebe09c640 1 -- 172.21.15.81:0/2732079665 learned_addr learned my addr 172.21.15.81:0/2732079665 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:28.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.557+0000 7f6ebd89b640 1 --2- 172.21.15.81:0/2732079665 >> v2:172.21.15.93:3300/0 conn(0x7f6eb80cc5e0 0x7f6eb80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:28.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.558+0000 7f6ebe89d640 1 -- 172.21.15.81:0/2732079665 >> v2:172.21.15.93:3300/0 conn(0x7f6eb80cc5e0 msgr2=0x7f6eb80cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:28.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.558+0000 7f6ebe89d640 1 --2- 172.21.15.81:0/2732079665 >> v2:172.21.15.93:3300/0 conn(0x7f6eb80cc5e0 0x7f6eb80cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.558+0000 7f6ebe89d640 1 -- 172.21.15.81:0/2732079665 >> v1:172.21.15.81:6789/0 conn(0x7f6eb80adcd0 legacy=0x7f6eb80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.558+0000 7f6ebe89d640 1 -- 172.21.15.81:0/2732079665 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb806d350 con 0x7f6eb80ccfb0 2023-11-08T16:31:28.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.558+0000 7f6ebd09a640 1 -- 172.21.15.81:0/2732079665 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967967489 0 0) 0x7f6eb80c9a70 con 0x7f6eb80adcd0 2023-11-08T16:31:28.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.559+0000 7f6ebe89d640 1 --2- 172.21.15.81:0/2732079665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80ccfb0 0x7f6eb8070050 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80cc0d0 tx=0x7f6eb002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f62a58c253acfe5d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:28.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.559+0000 7f6ebd09a640 1 -- 172.21.15.81:0/2732079665 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb002ea80 con 0x7f6eb80ccfb0 2023-11-08T16:31:28.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.559+0000 7f6ebd09a640 1 -- 172.21.15.81:0/2732079665 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb002ebe0 con 0x7f6eb80ccfb0 2023-11-08T16:31:28.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.559+0000 7f6ebd09a640 1 -- 172.21.15.81:0/2732079665 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0009540 con 0x7f6eb80ccfb0 2023-11-08T16:31:28.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.560+0000 7f6ec0300640 1 -- 172.21.15.81:0/2732079665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80ccfb0 msgr2=0x7f6eb8070050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.560+0000 7f6ec0300640 1 --2- 172.21.15.81:0/2732079665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80ccfb0 0x7f6eb8070050 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80cc0d0 tx=0x7f6eb002e0e0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.560+0000 7f6ec0300640 1 -- 172.21.15.81:0/2732079665 shutdown_connections 2023-11-08T16:31:28.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.560+0000 7f6ec0300640 1 --2- 172.21.15.81:0/2732079665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80ccfb0 0x7f6eb8070050 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.560+0000 7f6ec0300640 1 --2- 172.21.15.81:0/2732079665 >> v2:172.21.15.93:3300/0 conn(0x7f6eb80cc5e0 0x7f6eb80cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.560+0000 7f6ec0300640 1 -- 172.21.15.81:0/2732079665 >> 172.21.15.81:0/2732079665 conn(0x7f6eb80cb490 msgr2=0x7f6eb80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:28.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.560+0000 7f6ec0300640 1 -- 172.21.15.81:0/2732079665 shutdown_connections 2023-11-08T16:31:28.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.560+0000 7f6ec0300640 1 -- 172.21.15.81:0/2732079665 wait complete. 2023-11-08T16:31:28.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.562+0000 7f6ec0300640 1 Processor -- start 2023-11-08T16:31:28.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.562+0000 7f6ec0300640 1 -- start start 2023-11-08T16:31:28.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ec0300640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80adcd0 0x7f6eb81f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ec0300640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb80cc5e0 0x7f6eb80acc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ec0300640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6eb80ccfb0 0x7f6eb815be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ec0300640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb81f2bd0 con 0x7f6eb80adcd0 2023-11-08T16:31:28.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ec0300640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb814b8a0 con 0x7f6eb80cc5e0 2023-11-08T16:31:28.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ec0300640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb814bb80 con 0x7f6eb80ccfb0 2023-11-08T16:31:28.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ebe09c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80adcd0 0x7f6eb81f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:28.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ebe09c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80adcd0 0x7f6eb81f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46252/0 (socket says 172.21.15.81:46252) 2023-11-08T16:31:28.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ebe09c640 1 -- 172.21.15.81:0/1399246963 learned_addr learned my addr 172.21.15.81:0/1399246963 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:28.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ebe09c640 1 -- 172.21.15.81:0/1399246963 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6eb80ccfb0 msgr2=0x7f6eb815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ebe09c640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6eb80ccfb0 0x7f6eb815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ebe09c640 1 -- 172.21.15.81:0/1399246963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb80cc5e0 msgr2=0x7f6eb80acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.563+0000 7f6ebd89b640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb80cc5e0 0x7f6eb80acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:28.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.564+0000 7f6ebe09c640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb80cc5e0 0x7f6eb80acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.564+0000 7f6ebe09c640 1 -- 172.21.15.81:0/1399246963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb000bc80 con 0x7f6eb80adcd0 2023-11-08T16:31:28.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.564+0000 7f6ebd89b640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb80cc5e0 0x7f6eb80acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:28.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.564+0000 7f6ebe09c640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80adcd0 0x7f6eb81f2690 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400c430 tx=0x7f6eb400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:28.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.564+0000 7f6eaeffd640 1 -- 172.21.15.81:0/1399246963 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb40074e0 con 0x7f6eb80adcd0 2023-11-08T16:31:28.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.564+0000 7f6eaeffd640 1 -- 172.21.15.81:0/1399246963 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4007660 con 0x7f6eb80adcd0 2023-11-08T16:31:28.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.564+0000 7f6ec0300640 1 -- 172.21.15.81:0/1399246963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb814be60 con 0x7f6eb80adcd0 2023-11-08T16:31:28.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.564+0000 7f6eaeffd640 1 -- 172.21.15.81:0/1399246963 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4004a20 con 0x7f6eb80adcd0 2023-11-08T16:31:28.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.564+0000 7f6ec0300640 1 -- 172.21.15.81:0/1399246963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb814c420 con 0x7f6eb80adcd0 2023-11-08T16:31:28.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.566+0000 7f6ec0300640 1 -- 172.21.15.81:0/1399246963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb8072170 con 0x7f6eb80adcd0 2023-11-08T16:31:28.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.566+0000 7f6eaeffd640 1 -- 172.21.15.81:0/1399246963 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6eb4004b80 con 0x7f6eb80adcd0 2023-11-08T16:31:28.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.568+0000 7f6eaeffd640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06ae70 0x7f6e9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:28.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.568+0000 7f6eaeffd640 1 -- 172.21.15.81:0/1399246963 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f6eb40896f0 con 0x7f6eb80adcd0 2023-11-08T16:31:28.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.568+0000 7f6ebd89b640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06ae70 0x7f6e9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:28.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.569+0000 7f6ebd89b640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06ae70 0x7f6e9c06d330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8006fd0 tx=0x7f6ea8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:28.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.572+0000 7f6eaeffd640 1 -- 172.21.15.81:0/1399246963 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6eb40544f0 con 0x7f6eb80adcd0 2023-11-08T16:31:28.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.803+0000 7f6ec0300640 1 -- 172.21.15.81:0/1399246963 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6eb814c640 con 0x7f6eb80adcd0 2023-11-08T16:31:28.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.804+0000 7f6eaeffd640 1 -- 172.21.15.81:0/1399246963 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) v1 ==== 53+0+91 (secure 0 0 0) 0x7f6eb4057b10 con 0x7f6eb80adcd0 2023-11-08T16:31:28.806 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-11-08T16:31:28.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 -- 172.21.15.81:0/1399246963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06ae70 msgr2=0x7f6e9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06ae70 0x7f6e9c06d330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6ea8006fd0 tx=0x7f6ea8008040 comp rx=0 tx=0).stop 2023-11-08T16:31:28.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 -- 172.21.15.81:0/1399246963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80adcd0 msgr2=0x7f6eb81f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:28.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80adcd0 0x7f6eb81f2690 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f6eb400c430 tx=0x7f6eb400c8f0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 -- 172.21.15.81:0/1399246963 shutdown_connections 2023-11-08T16:31:28.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06ae70 0x7f6e9c06d330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6eb80ccfb0 0x7f6eb815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb80cc5e0 0x7f6eb80acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 --2- 172.21.15.81:0/1399246963 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80adcd0 0x7f6eb81f2690 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:28.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 -- 172.21.15.81:0/1399246963 >> 172.21.15.81:0/1399246963 conn(0x7f6eb80cb490 msgr2=0x7f6eb81e4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:28.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 -- 172.21.15.81:0/1399246963 shutdown_connections 2023-11-08T16:31:28.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:28.807+0000 7f6ec0300640 1 -- 172.21.15.81:0/1399246963 wait complete. 2023-11-08T16:31:28.826 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:31:28.826 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:31:28.826 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:31:28.827 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:31:28.827 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:31:28.827 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:31:28.828 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:31:28.829 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:31:28.829 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:31:28.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:28.831 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:31:28.862 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:28.860+0000 7fbf13bdd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:31:28.884 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:31:28.884 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:31:28.884 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:31:28.885 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:31:28.885 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:31:28.885 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:31:28.885 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:31:28.886 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:31:28.886 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:31:28.886 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:31:28.886 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:31:28.886 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:31:28.887 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:31:28.887 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:31:28.887 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:31:28.887 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:31:28.888 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:31:28.888 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42221sec preferred_lft 42221sec 2023-11-08T16:31:28.888 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:31:28.888 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:31:28.888 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:31:28.889 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff 2023-11-08T16:31:28.889 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:31:28.889 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:31:28.890 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:31:28.890 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:31:28.891 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:31:28.891 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:31:28.891 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:31:28.892 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:31:28.892 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:31:28.893 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:31:28.893 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::2cac:17ff:fe64:67da/64 scope link 2023-11-08T16:31:28.893 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:31:28.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:28.895 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:31:29.016 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:31:29.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:29.021 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:31:29.073 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:31:29.074 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:31:29.074 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:31:29.074 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:31:29.075 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:31:29.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:29.075 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:31:29.076 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:31:29.076 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:31:29.076 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:31:29.077 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:31:29.077 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:31:29.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:29.259 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:31:29.313 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:31:29.314 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:31:29.315 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:31:29.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:29.316 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:31:29.409 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:31:29.409 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:31:29.410 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:31:29.463 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:31:29.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:31:29.464 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:31:29.518 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:31:29.608 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:31:29.665 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:31:29.666 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:31:29.666 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:31:29.666 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:31:29.667 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:31:29.667 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:31:29.667 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:31:29.667 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:31:29.668 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:31:29.668 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:31:29.668 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:31:29.668 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:31:29.668 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:31:29.668 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:31:29.669 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:31:29.669 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:31:29.669 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:31:29.669 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:31:29.670 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:31:29.670 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:31:29.670 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:31:29.670 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:31:29.670 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:31:29.671 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:31:29.671 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:31:29.671 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:31:29.671 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:31:29.671 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:31:29.672 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:31:29.672 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:31:29.672 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:31:29.672 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:31:29.672 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:31:29.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:29.674 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:31:29.726 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:31:29.727 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:31:29.732 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:31:29.809 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:31:29.850 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:31:29.850 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:31:29.850 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:31:29.851 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:31:29.851 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:31:29.851 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:31:29.851 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:31:29.851 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:31:29.852 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:31:29.852 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:31:29.852 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:31:29.852 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:31:29.852 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:31:29.853 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:31:29.853 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:31:29.853 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:31:29.853 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:31:29.853 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:31:29.854 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:31:29.854 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:31:29.854 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:31:29.854 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:31:29.854 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:31:29.855 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:31:29.855 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:31:29.855 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:31:29.855 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:31:29.855 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:31:29.856 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:31:29.856 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:31:29.856 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:31:29.856 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:31:29.856 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:31:29.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:29.858 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:31:29.911 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:31:29.910+0000 7f5b21864480 -1 init, newargv = 0x560ae871b2d0 newargc=15 2023-11-08T16:31:29.912 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[103691]: starting ceph client 2023-11-08T16:31:29.919 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[103691]: starting fuse 2023-11-08T16:31:29.943 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:31:29.943 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:31:29.944 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:31:29.944 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:31:29.945 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:31:29.945 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:31:29.945 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:31:29.946 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:31:29.946 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:31:29.947 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:31:29.947 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:31:29.947 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:31:29.948 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:31:29.948 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:31:29.948 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:31:29.949 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:31:29.949 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:31:29.950 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:31:29.950 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:31:29.950 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:31:29.951 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:31:29.951 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:31:29.951 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:31:29.952 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:31:29.952 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:31:29.953 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:31:29.953 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:31:29.953 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:31:29.954 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:31:30.141 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.103691.asok 2023-11-08T16:31:30.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:31:30.158 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.103691.asok 2023-11-08T16:31:30.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:30.159 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.103691.asok status 2023-11-08T16:31:30.397 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:31:30.398 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:31:30.398 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:31:30.398 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:31:30.399 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:31:30.399 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:31:30.399 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:31:30.399 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "103691", 2023-11-08T16:31:30.399 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:31:30.400 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:31:30.400 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:31:30.400 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:31:30.400 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 5730, 2023-11-08T16:31:30.400 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:31:30.401 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:31:30.401 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:31:30.401 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 5730 2023-11-08T16:31:30.401 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:31:30.401 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:31:30.401 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:31:30.402 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:31:30.402 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 4179384039 2023-11-08T16:31:30.402 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:31:30.402 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:31:30.402 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:31:30.403 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:31:30.403 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:31:30.403 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 4179384039 2023-11-08T16:31:30.403 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:31:30.403 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.5730 192.168.0.1:0/4179384039", 2023-11-08T16:31:30.404 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/4179384039", 2023-11-08T16:31:30.404 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:31:30.404 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 32, 2023-11-08T16:31:30.404 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 44, 2023-11-08T16:31:30.404 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:31:30.405 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:31:30.405 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:31:30.405 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:31:30.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:30.413 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:31:30.413 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:31:30.468 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:31:30.468 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:31:30.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:31:30.469 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:31:30.561 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub start / repair,recursive 2023-11-08T16:31:30.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.774+0000 7f27ce04b640 1 Processor -- start 2023-11-08T16:31:30.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.774+0000 7f27ce04b640 1 -- start start 2023-11-08T16:31:30.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.776+0000 7f27ce04b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27c80add40 0x7f27c806cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.776+0000 7f27ce04b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f27c80ce5d0 0x7f27c80cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.776+0000 7f27ce04b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27c8165f90 con 0x7f27c80caa80 2023-11-08T16:31:30.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.776+0000 7f27ce04b640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27c80cb280 con 0x7f27c80ce5d0 2023-11-08T16:31:30.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.776+0000 7f27ce04b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c806d400 con 0x7f27c80add40 2023-11-08T16:31:30.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.776+0000 7f27c7fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f27c80ce5d0 0x7f27c80cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.776+0000 7f27c7fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f27c80ce5d0 0x7f27c80cc630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46262/0 (socket says 172.21.15.81:46262) 2023-11-08T16:31:30.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.776+0000 7f27bffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f27c80caa80 0x7f27c80cdec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54566/0 (socket says 172.21.15.81:54566) 2023-11-08T16:31:30.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.776+0000 7f27c7fff640 1 -- 172.21.15.81:0/3861222870 learned_addr learned my addr 172.21.15.81:0/3861222870 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:30.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.777+0000 7f27c77fe640 1 --2- 172.21.15.81:0/3861222870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27c80add40 0x7f27c806cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.777+0000 7f27c6ffd640 1 -- 172.21.15.81:0/3861222870 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1161977985 0 0) 0x7f27c8165f90 con 0x7f27c80caa80 2023-11-08T16:31:30.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.777+0000 7f27c7fff640 1 -- 172.21.15.81:0/3861222870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27c80add40 msgr2=0x7f27c806cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:30.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.777+0000 7f27c7fff640 1 --2- 172.21.15.81:0/3861222870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27c80add40 0x7f27c806cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.777+0000 7f27c7fff640 1 -- 172.21.15.81:0/3861222870 >> v1:172.21.15.81:6789/0 conn(0x7f27c80caa80 legacy=0x7f27c80cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:30.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.777+0000 7f27c7fff640 1 -- 172.21.15.81:0/3861222870 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c80ae120 con 0x7f27c80ce5d0 2023-11-08T16:31:30.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.778+0000 7f27c7fff640 1 --2- 172.21.15.81:0/3861222870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c80ce5d0 0x7f27c80cc630 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009e50 tx=0x7f27c002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335c8716b1e047fc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:30.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.779+0000 7f27c6ffd640 1 -- 172.21.15.81:0/3861222870 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c002c9e0 con 0x7f27c80ce5d0 2023-11-08T16:31:30.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.779+0000 7f27c6ffd640 1 -- 172.21.15.81:0/3861222870 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c002cb40 con 0x7f27c80ce5d0 2023-11-08T16:31:30.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.779+0000 7f27c6ffd640 1 -- 172.21.15.81:0/3861222870 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c00343f0 con 0x7f27c80ce5d0 2023-11-08T16:31:30.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.779+0000 7f27ce04b640 1 -- 172.21.15.81:0/3861222870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c80ce5d0 msgr2=0x7f27c80cc630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:30.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.779+0000 7f27ce04b640 1 --2- 172.21.15.81:0/3861222870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c80ce5d0 0x7f27c80cc630 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f27c0009e50 tx=0x7f27c002be40 comp rx=0 tx=0).stop 2023-11-08T16:31:30.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.779+0000 7f27ce04b640 1 -- 172.21.15.81:0/3861222870 shutdown_connections 2023-11-08T16:31:30.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.779+0000 7f27ce04b640 1 --2- 172.21.15.81:0/3861222870 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c80ce5d0 0x7f27c80cc630 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.779+0000 7f27ce04b640 1 --2- 172.21.15.81:0/3861222870 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27c80add40 0x7f27c806cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.779+0000 7f27ce04b640 1 -- 172.21.15.81:0/3861222870 >> 172.21.15.81:0/3861222870 conn(0x7f27c80cb4e0 msgr2=0x7f27c80cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:30.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.780+0000 7f27ce04b640 1 -- 172.21.15.81:0/3861222870 shutdown_connections 2023-11-08T16:31:30.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.780+0000 7f27ce04b640 1 -- 172.21.15.81:0/3861222870 wait complete. 2023-11-08T16:31:30.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.781+0000 7f27ce04b640 1 Processor -- start 2023-11-08T16:31:30.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.781+0000 7f27ce04b640 1 -- start start 2023-11-08T16:31:30.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.782+0000 7f27ce04b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27c80add40 0x7f27c81f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.782+0000 7f27ce04b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c80caa80 0x7f27c80b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.782+0000 7f27ce04b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27c80ce5d0 0x7f27c80b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.782+0000 7f27ce04b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c80c4bc0 con 0x7f27c80caa80 2023-11-08T16:31:30.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.782+0000 7f27ce04b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27c81f7dd0 con 0x7f27c80add40 2023-11-08T16:31:30.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.782+0000 7f27ce04b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27c80c4220 con 0x7f27c80ce5d0 2023-11-08T16:31:30.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.783+0000 7f27bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c80caa80 0x7f27c80b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.783+0000 7f27bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c80caa80 0x7f27c80b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46278/0 (socket says 172.21.15.81:46278) 2023-11-08T16:31:30.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.783+0000 7f27bffff640 1 -- 172.21.15.81:0/2618961464 learned_addr learned my addr 172.21.15.81:0/2618961464 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:30.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.783+0000 7f27bffff640 1 -- 172.21.15.81:0/2618961464 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27c80ce5d0 msgr2=0x7f27c80b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:30.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.783+0000 7f27c7fff640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27c80ce5d0 0x7f27c80b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.783+0000 7f27bffff640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27c80ce5d0 0x7f27c80b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.783+0000 7f27bffff640 1 -- 172.21.15.81:0/2618961464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27c80add40 msgr2=0x7f27c81f68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:30.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.783+0000 7f27c77fe640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27c80add40 0x7f27c81f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.783+0000 7f27bffff640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27c80add40 0x7f27c81f68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.783+0000 7f27bffff640 1 -- 172.21.15.81:0/2618961464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c0009b30 con 0x7f27c80caa80 2023-11-08T16:31:30.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.783+0000 7f27bffff640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c80caa80 0x7f27c80b2520 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f27b000c4d0 tx=0x7f27b000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:30.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.784+0000 7f27c4ff9640 1 -- 172.21.15.81:0/2618961464 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b00074e0 con 0x7f27c80caa80 2023-11-08T16:31:30.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.784+0000 7f27c4ff9640 1 -- 172.21.15.81:0/2618961464 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b0007660 con 0x7f27c80caa80 2023-11-08T16:31:30.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.784+0000 7f27c4ff9640 1 -- 172.21.15.81:0/2618961464 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0004a20 con 0x7f27c80caa80 2023-11-08T16:31:30.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.784+0000 7f27ce04b640 1 -- 172.21.15.81:0/2618961464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27c80c4500 con 0x7f27c80caa80 2023-11-08T16:31:30.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.784+0000 7f27ce04b640 1 -- 172.21.15.81:0/2618961464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c80c4fe0 con 0x7f27c80caa80 2023-11-08T16:31:30.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.786+0000 7f27c4ff9640 1 -- 172.21.15.81:0/2618961464 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27b0004b80 con 0x7f27c80caa80 2023-11-08T16:31:30.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.787+0000 7f27c4ff9640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f27a806ae70 0x7f27a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.787+0000 7f27c4ff9640 1 -- 172.21.15.81:0/2618961464 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f27b0088b40 con 0x7f27c80caa80 2023-11-08T16:31:30.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.787+0000 7f27c77fe640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f27a806ae70 0x7f27a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.788+0000 7f27c77fe640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f27a806ae70 0x7f27a806d330 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f27b8006fd0 tx=0x7f27b8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:30.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.788+0000 7f27d0bdb740 1 Processor -- start 2023-11-08T16:31:30.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.788+0000 7f27d0bdb740 1 -- start start 2023-11-08T16:31:30.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de39b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de39be00 0x5602de3cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602de30e5f0 con 0x5602de3c8b10 2023-11-08T16:31:30.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602de30bce0 con 0x5602de3c8730 2023-11-08T16:31:30.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602de3075d0 con 0x5602de39be00 2023-11-08T16:31:30.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46290/0 (socket says 172.21.15.81:46290) 2023-11-08T16:31:30.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27bffff640 1 -- 172.21.15.81:0/37321996 learned_addr learned my addr 172.21.15.81:0/37321996 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:30.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27bffff640 1 -- 172.21.15.81:0/37321996 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de39be00 msgr2=0x5602de3cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:30.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27bffff640 1 --2- 172.21.15.81:0/37321996 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de39be00 0x5602de3cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27bffff640 1 -- 172.21.15.81:0/37321996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 msgr2=0x5602de39b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:30.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27bffff640 1 --2- 172.21.15.81:0/37321996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de39b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.789+0000 7f27bffff640 1 -- 172.21.15.81:0/37321996 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602de2e1a60 con 0x5602de3c8b10 2023-11-08T16:31:30.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.790+0000 7f27bffff640 1 --2- 172.21.15.81:0/37321996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3cb4c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f27b0053a70 tx=0x7f27b0057c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c414c815507230a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:30.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.790+0000 7f27bd7fa640 1 -- 172.21.15.81:0/37321996 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b005b6c0 con 0x5602de3c8b10 2023-11-08T16:31:30.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.790+0000 7f27bd7fa640 1 -- 172.21.15.81:0/37321996 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b005f870 con 0x5602de3c8b10 2023-11-08T16:31:30.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.790+0000 7f27bd7fa640 1 -- 172.21.15.81:0/37321996 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0053c80 con 0x5602de3c8b10 2023-11-08T16:31:30.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.791+0000 7f27d0bdb740 1 -- 172.21.15.81:0/37321996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 msgr2=0x5602de3cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:30.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.791+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/37321996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3cb4c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f27b0053a70 tx=0x7f27b0057c70 comp rx=0 tx=0).stop 2023-11-08T16:31:30.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.791+0000 7f27d0bdb740 1 -- 172.21.15.81:0/37321996 shutdown_connections 2023-11-08T16:31:30.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.791+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/37321996 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de39be00 0x5602de3cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.791+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/37321996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3cb4c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.791+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/37321996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de39b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.791+0000 7f27d0bdb740 1 -- 172.21.15.81:0/37321996 >> 172.21.15.81:0/37321996 conn(0x5602de39dbd0 msgr2=0x5602de3e64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:30.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.791+0000 7f27d0bdb740 1 -- 172.21.15.81:0/37321996 shutdown_connections 2023-11-08T16:31:30.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.791+0000 7f27d0bdb740 1 -- 172.21.15.81:0/37321996 wait complete. 2023-11-08T16:31:30.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.791+0000 7f27d0bdb740 1 -- start start 2023-11-08T16:31:30.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.792+0000 7f27d0bdb740 1 Processor -- start 2023-11-08T16:31:30.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de3ce9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3bcc20 0x5602de3ddcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602de2a4590 con 0x5602de3c8b10 2023-11-08T16:31:30.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602de2a4230 con 0x5602de3c8730 2023-11-08T16:31:30.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602de06b0c0 con 0x5602de3bcc20 2023-11-08T16:31:30.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27d0bdb740 10 client.0 _ll_register_callbacks cb 0x5602de2c6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:31:30.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27bffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46306/0 (socket says 172.21.15.81:46306) 2023-11-08T16:31:30.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27bffff640 1 -- 172.21.15.81:0/2746437901 learned_addr learned my addr 172.21.15.81:0/2746437901 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:30.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27c7fff640 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3bcc20 0x5602de3ddcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27c77fe640 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de3ce9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27bffff640 1 -- 172.21.15.81:0/2746437901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3bcc20 msgr2=0x5602de3ddcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:30.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27bffff640 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3bcc20 0x5602de3ddcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27bffff640 1 -- 172.21.15.81:0/2746437901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 msgr2=0x5602de3ce9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:30.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27bffff640 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de3ce9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.793+0000 7f27bffff640 1 -- 172.21.15.81:0/2746437901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27c80c4fe0 con 0x5602de3c8b10 2023-11-08T16:31:30.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.794+0000 7f27bffff640 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0430 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f27b0057730 tx=0x7f27b0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=851eb0e5af66e6ee server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:30.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.794+0000 7f27a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:31:30.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.794+0000 7f27a67fc640 1 -- 172.21.15.81:0/2746437901 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b0071070 con 0x5602de3c8b10 2023-11-08T16:31:30.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.794+0000 7f27a67fc640 1 -- 172.21.15.81:0/2746437901 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f27b0088040 con 0x5602de3c8b10 2023-11-08T16:31:30.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.794+0000 7f27d0bdb740 1 -- 172.21.15.81:0/2746437901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5602de32d910 con 0x5602de3c8b10 2023-11-08T16:31:30.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.794+0000 7f27a67fc640 1 -- 172.21.15.81:0/2746437901 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27b0085070 con 0x5602de3c8b10 2023-11-08T16:31:30.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.794+0000 7f27d0bdb740 10 client.5745 fetch_fsmap learned FSMap version 32 2023-11-08T16:31:30.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.794+0000 7f27d0bdb740 1 -- 172.21.15.81:0/2746437901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5602de2a2810 con 0x5602de3c8b10 2023-11-08T16:31:30.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.794+0000 7f27a67fc640 1 -- 172.21.15.81:0/2746437901 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5602de32d910 con 0x5602de3c8b10 2023-11-08T16:31:30.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.795+0000 7f27a67fc640 1 -- 172.21.15.81:0/2746437901 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f27b005f3e0 con 0x5602de3c8b10 2023-11-08T16:31:30.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.795+0000 7f27d0bdb740 10 client.5745 fetch_fsmap finished waiting for FSMap version 32 2023-11-08T16:31:30.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.795+0000 7f27d0bdb740 10 client.5745 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-08T16:31:30.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.795+0000 7f27d0bdb740 20 client.5745 populate_metadata read hostname 'smithi081' 2023-11-08T16:31:30.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.795+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5602de3f6280 0x5602de3f6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.795+0000 7f27d0bdb740 4 client.5745 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:31:30.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.795+0000 7f27d0bdb740 1 -- 172.21.15.81:0/2746437901 --> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5602de2da450 con 0x5602de3f6280 2023-11-08T16:31:30.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.795+0000 7f27c77fe640 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5602de3f6280 0x5602de3f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.795+0000 7f27a67fc640 10 client.5745 ms_handle_connect on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:30.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.795+0000 7f27c77fe640 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5602de3f6280 0x5602de3f6660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa15ae0542539819 server_cookie=54c4ad3aa5805ff5 in_seq=0 out_seq=0 2023-11-08T16:31:30.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27a67fc640 1 -- 172.21.15.81:0/2746437901 <== mds.0 v2:172.21.15.81:6836/2942618483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5602de2da450 con 0x5602de3f6280 2023-11-08T16:31:30.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27a67fc640 10 client.5745 handle_command_reply: tid=0 2023-11-08T16:31:30.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27d0bdb740 1 client.5745 shutdown 2023-11-08T16:31:30.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27d0bdb740 1 -- 172.21.15.81:0/2746437901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 msgr2=0x5602de3e0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:30.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0430 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f27b0057730 tx=0x7f27b0000ed0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27d0bdb740 1 -- 172.21.15.81:0/2746437901 shutdown_connections 2023-11-08T16:31:30.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5602de3f6280 0x5602de3f6660 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3bcc20 0x5602de3ddcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0430 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/2746437901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de3ce9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27a67fc640 0 client.5745 ms_handle_reset on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:30.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.796+0000 7f27d0bdb740 1 -- 172.21.15.81:0/2746437901 >> 172.21.15.81:0/2746437901 conn(0x5602de39be00 msgr2=0x5602de3e99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:30.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.797+0000 7f27d0bdb740 1 -- 172.21.15.81:0/2746437901 shutdown_connections 2023-11-08T16:31:30.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.797+0000 7f27d0bdb740 1 -- 172.21.15.81:0/2746437901 wait complete. 2023-11-08T16:31:30.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.797+0000 7f27d0bdb740 20 client.5745 trim_cache size 0 max 16384 2023-11-08T16:31:30.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.813+0000 7f27d0bdb740 1 Processor -- start 2023-11-08T16:31:30.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.813+0000 7f27d0bdb740 1 -- start start 2023-11-08T16:31:30.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.814+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3bcc20 0x5602de3ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.814+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de39f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.814+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3c8b10 0x5602de3e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.814+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602de39b250 con 0x5602de3bcc20 2023-11-08T16:31:30.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.814+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602de39a9b0 con 0x5602de3c8730 2023-11-08T16:31:30.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.814+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602de3dcfe0 con 0x5602de3c8b10 2023-11-08T16:31:30.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27c77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3bcc20 0x5602de3ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27c77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3bcc20 0x5602de3ce8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46320/0 (socket says 172.21.15.81:46320) 2023-11-08T16:31:30.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27c77fe640 1 -- 172.21.15.81:0/1208056466 learned_addr learned my addr 172.21.15.81:0/1208056466 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:30.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27bffff640 1 --2- 172.21.15.81:0/1208056466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de39f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27c77fe640 1 -- 172.21.15.81:0/1208056466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3c8b10 msgr2=0x5602de3e0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:30.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27c7fff640 1 --2- 172.21.15.81:0/1208056466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3c8b10 0x5602de3e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27c77fe640 1 --2- 172.21.15.81:0/1208056466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3c8b10 0x5602de3e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27c77fe640 1 -- 172.21.15.81:0/1208056466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 msgr2=0x5602de39f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:30.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27c77fe640 1 --2- 172.21.15.81:0/1208056466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de39f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27c77fe640 1 -- 172.21.15.81:0/1208056466 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602de336c60 con 0x5602de3bcc20 2023-11-08T16:31:30.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27bffff640 1 --2- 172.21.15.81:0/1208056466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de39f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:30.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.815+0000 7f27c77fe640 1 --2- 172.21.15.81:0/1208056466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3bcc20 0x5602de3ce8f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x5602de3f6c60 tx=0x7f27b8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc977acea7a48d8a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:30.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27bdffb640 1 -- 172.21.15.81:0/1208056466 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b8024460 con 0x5602de3bcc20 2023-11-08T16:31:30.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27bdffb640 1 -- 172.21.15.81:0/1208056466 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b801b070 con 0x5602de3bcc20 2023-11-08T16:31:30.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27bdffb640 1 -- 172.21.15.81:0/1208056466 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b80247c0 con 0x5602de3bcc20 2023-11-08T16:31:30.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1208056466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3bcc20 msgr2=0x5602de3ce8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:30.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/1208056466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3bcc20 0x5602de3ce8f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x5602de3f6c60 tx=0x7f27b8011000 comp rx=0 tx=0).stop 2023-11-08T16:31:30.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1208056466 shutdown_connections 2023-11-08T16:31:30.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/1208056466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3c8b10 0x5602de3e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/1208056466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3c8730 0x5602de39f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/1208056466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3bcc20 0x5602de3ce8f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1208056466 >> 172.21.15.81:0/1208056466 conn(0x5602de39be00 msgr2=0x5602de3a1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:30.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1208056466 shutdown_connections 2023-11-08T16:31:30.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.816+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1208056466 wait complete. 2023-11-08T16:31:30.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.817+0000 7f27d0bdb740 1 -- start start 2023-11-08T16:31:30.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.817+0000 7f27d0bdb740 1 Processor -- start 2023-11-08T16:31:30.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.818+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3bcc20 0x5602de3a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.818+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3c8730 0x5602de44a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.818+0000 7f27d0bdb740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.818+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602de450920 con 0x5602de3c8b10 2023-11-08T16:31:30.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.818+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602de44acb0 con 0x5602de3bcc20 2023-11-08T16:31:30.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.818+0000 7f27d0bdb740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602de3e7570 con 0x5602de3c8730 2023-11-08T16:31:30.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.818+0000 7f27d0bdb740 10 client.0 _ll_register_callbacks cb 0x7f27b0056f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:31:30.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.818+0000 7f27c7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27c7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46326/0 (socket says 172.21.15.81:46326) 2023-11-08T16:31:30.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27c7fff640 1 -- 172.21.15.81:0/1429515405 learned_addr learned my addr 172.21.15.81:0/1429515405 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:30.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27c77fe640 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3bcc20 0x5602de3a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27bffff640 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3c8730 0x5602de44a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27c7fff640 1 -- 172.21.15.81:0/1429515405 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3c8730 msgr2=0x5602de44a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:30.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27c7fff640 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3c8730 0x5602de44a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27c7fff640 1 -- 172.21.15.81:0/1429515405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3bcc20 msgr2=0x5602de3a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:30.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27c7fff640 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3bcc20 0x5602de3a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27c7fff640 1 -- 172.21.15.81:0/1429515405 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b800e8c0 con 0x5602de3c8b10 2023-11-08T16:31:30.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27c77fe640 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3bcc20 0x5602de3a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:30.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27c7fff640 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0090 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f27c0036040 tx=0x7f27c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96716f1c88a81999 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:30.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.819+0000 7f27a67fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:31:30.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27a67fc640 1 -- 172.21.15.81:0/1429515405 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c0034b30 con 0x5602de3c8b10 2023-11-08T16:31:30.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27a67fc640 1 -- 172.21.15.81:0/1429515405 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f27c0034c90 con 0x5602de3c8b10 2023-11-08T16:31:30.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1429515405 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f27b005f870 con 0x5602de3c8b10 2023-11-08T16:31:30.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27a67fc640 1 -- 172.21.15.81:0/1429515405 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27c0034e70 con 0x5602de3c8b10 2023-11-08T16:31:30.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27d0bdb740 10 client.5751 fetch_fsmap learned FSMap version 32 2023-11-08T16:31:30.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1429515405 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x5602de2a3580 con 0x5602de3c8b10 2023-11-08T16:31:30.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27a67fc640 1 -- 172.21.15.81:0/1429515405 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b005f870 con 0x5602de3c8b10 2023-11-08T16:31:30.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27a67fc640 1 -- 172.21.15.81:0/1429515405 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f27c0048410 con 0x5602de3c8b10 2023-11-08T16:31:30.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27d0bdb740 10 client.5751 fetch_fsmap finished waiting for FSMap version 32 2023-11-08T16:31:30.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27d0bdb740 10 client.5751 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-08T16:31:30.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27d0bdb740 20 client.5751 populate_metadata read hostname 'smithi081' 2023-11-08T16:31:30.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5602de3f4be0 0x5602de3f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:30.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27d0bdb740 4 client.5751 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-08T16:31:30.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1429515405 --> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x5602de3005b0 con 0x5602de3f4be0 2023-11-08T16:31:30.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.820+0000 7f27c77fe640 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5602de3f4be0 0x5602de3f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:30.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.821+0000 7f27c77fe640 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5602de3f4be0 0x5602de3f4ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eab4e837eb238b13 server_cookie=56823ddda74fdb9 in_seq=0 out_seq=0 2023-11-08T16:31:30.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.821+0000 7f27a67fc640 10 client.5751 ms_handle_connect on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:30.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27a67fc640 1 -- 172.21.15.81:0/1429515405 <== mds.0 v2:172.21.15.81:6836/2942618483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5602de3005b0 con 0x5602de3f4be0 2023-11-08T16:31:30.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27a67fc640 10 client.5751 handle_command_reply: tid=0 2023-11-08T16:31:30.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27d0bdb740 1 client.5751 shutdown 2023-11-08T16:31:30.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1429515405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 msgr2=0x5602de3e0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:30.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0090 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f27c0036040 tx=0x7f27c0004a40 comp rx=0 tx=0).stop 2023-11-08T16:31:30.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1429515405 shutdown_connections 2023-11-08T16:31:30.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x5602de3f4be0 0x5602de3f4ff0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5602de3c8b10 0x5602de3e0090 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5602de3c8730 0x5602de44a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27d0bdb740 1 --2- 172.21.15.81:0/1429515405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5602de3bcc20 0x5602de3a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27a67fc640 0 client.5751 ms_handle_reset on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:30.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.822+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1429515405 >> 172.21.15.81:0/1429515405 conn(0x5602de39be00 msgr2=0x5602de3cd980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:30.838 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:30.822+0000 7fbf103d6640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-11-08T16:31:30.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.823+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1429515405 shutdown_connections 2023-11-08T16:31:30.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.823+0000 7f27d0bdb740 1 -- 172.21.15.81:0/1429515405 wait complete. 2023-11-08T16:31:30.839 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:30.823+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55c9f2dafb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(4)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:31:30.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.824+0000 7f27d0bdb740 20 client.5751 trim_cache size 0 max 16384 2023-11-08T16:31:30.840 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:31:30.840 INFO:teuthology.orchestra.run.smithi081.stdout: "return_code": 0, 2023-11-08T16:31:30.840 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_tag": "a6af52f2-e4be-4f7f-8ec9-6ba39af032ac", 2023-11-08T16:31:30.841 INFO:teuthology.orchestra.run.smithi081.stdout: "mode": "asynchronous" 2023-11-08T16:31:30.841 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:31:30.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 -- 172.21.15.81:0/2618961464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f27a806ae70 msgr2=0x7f27a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:30.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f27a806ae70 0x7f27a806d330 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f27b8006fd0 tx=0x7f27b8008040 comp rx=0 tx=0).stop 2023-11-08T16:31:30.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 -- 172.21.15.81:0/2618961464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c80caa80 msgr2=0x7f27c80b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:30.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c80caa80 0x7f27c80b2520 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f27b000c4d0 tx=0x7f27b000c990 comp rx=0 tx=0).stop 2023-11-08T16:31:30.842 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:30.825+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55c9f2e52b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x100000001f7:[<0x10000000000/2 v32>,<0x1/subdir v44>]//[5]","memoryvalue":"(6)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:31:30.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 -- 172.21.15.81:0/2618961464 shutdown_connections 2023-11-08T16:31:30.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f27a806ae70 0x7f27a806d330 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f27c80ce5d0 0x7f27c80b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f27c80caa80 0x7f27c80b2520 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.844 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:30.825+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55c9f2e52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x100000001f9:[<0x10000000000/4 v40>,<0x1/subdir v44>]//[5]","memoryvalue":"(6)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:31:30.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 --2- 172.21.15.81:0/2618961464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27c80add40 0x7f27c81f68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:30.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 -- 172.21.15.81:0/2618961464 >> 172.21.15.81:0/2618961464 conn(0x7f27c80cb4e0 msgr2=0x7f27c806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:30.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 -- 172.21.15.81:0/2618961464 shutdown_connections 2023-11-08T16:31:30.845 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:30.825+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55c9f2e52580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x100000001f8:[<0x10000000000/3 v38>,<0x1/subdir v44>]//[5]","memoryvalue":"(6)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:31:30.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:30.825+0000 7f27ce04b640 1 -- 172.21.15.81:0/2618961464 wait complete. 2023-11-08T16:31:30.846 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:30.825+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55c9f2e53080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x100000001f6:[<0x10000000000/1 v24>,<0x1/subdir v44>]//[5]","memoryvalue":"(6)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:31:30.847 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:30.826+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55c9f2e53600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x10000000001:[<0x10000000000/0 v16>,<0x1/subdir v44>]//[5]","memoryvalue":"(6)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:31:30.847 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:31:30.829+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x55c9f2da46c0 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-11-08T16:31:30.555816+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={5730=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55c9f2daf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(4)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-11-08T16:31:30.555816+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:31:30.848 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-11-08T16:31:31.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.049+0000 7f4ab5739640 1 Processor -- start 2023-11-08T16:31:31.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.049+0000 7f4ab5739640 1 -- start start 2023-11-08T16:31:31.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.051+0000 7f4ab5739640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab015e170 0x7f4ab015ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.051+0000 7f4ab5739640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f4ab00b0bf0 0x7f4ab00ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.051+0000 7f4ab5739640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab0071460 con 0x7f4ab015e550 2023-11-08T16:31:31.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.051+0000 7f4ab5739640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ab00703c0 con 0x7f4ab00b0bf0 2023-11-08T16:31:31.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.051+0000 7f4ab5739640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00706a0 con 0x7f4ab015e170 2023-11-08T16:31:31.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.052+0000 7f4aa67fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4ab015e550 0x7f4ab00b04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58252/0 (socket says 172.21.15.81:58252) 2023-11-08T16:31:31.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.052+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1223270305 learned_addr learned my addr 172.21.15.81:0/1223270305 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:31:31.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.052+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab015e170 0x7f4ab015ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.052+0000 7f4aaf7fe640 1 --2- 172.21.15.81:0/1223270305 >> v2:172.21.15.93:3300/0 conn(0x7f4ab00b0bf0 0x7f4ab00ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.052+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480226834 0 0) 0x7f4ab0071460 con 0x7f4ab015e550 2023-11-08T16:31:31.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.052+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a9c005730 con 0x7f4ab015e550 2023-11-08T16:31:31.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.052+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3388618223 0 0) 0x7f4a9c005730 con 0x7f4ab015e550 2023-11-08T16:31:31.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.052+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab015e170 msgr2=0x7f4ab015ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.052+0000 7f4aae7fc640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab015e170 0x7f4ab015ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.052+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 >> v2:172.21.15.93:3300/0 conn(0x7f4ab00b0bf0 msgr2=0x7f4ab00ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.052+0000 7f4aae7fc640 1 --2- 172.21.15.81:0/1223270305 >> v2:172.21.15.93:3300/0 conn(0x7f4ab00b0bf0 0x7f4ab00ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab0070980 con 0x7f4ab015e550 2023-11-08T16:31:31.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f4a98001670 con 0x7f4ab015e550 2023-11-08T16:31:31.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 >> v1:172.21.15.81:6789/0 conn(0x7f4ab015e550 legacy=0x7f4ab00b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:31.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aae7fc640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab015e170 0x7f4a9c005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aae7fc640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 0x7f4a9c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aae7fc640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a9c006f10 0x7f4a9c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00703c0 con 0x7f4a9c006170 2023-11-08T16:31:31.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00706a0 con 0x7f4a9c006f10 2023-11-08T16:31:31.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c00dbc0 con 0x7f4ab015e170 2023-11-08T16:31:31.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 0x7f4a9c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab015e170 0x7f4a9c005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aaf7fe640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a9c006f10 0x7f4a9c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1223270305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab015e170 msgr2=0x7f4a9c005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab015e170 0x7f4a9c005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1223270305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a9c006f10 msgr2=0x7f4a9c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4a9c006f10 0x7f4a9c007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.053+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1223270305 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9c005730 con 0x7f4a9c006170 2023-11-08T16:31:31.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.055+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 0x7f4a9c0069d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800c210 tx=0x7f4a9c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d387e125b1343a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:31.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.055+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98005ad0 con 0x7f4a9c006170 2023-11-08T16:31:31.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.055+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a98005c30 con 0x7f4a9c006170 2023-11-08T16:31:31.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.055+0000 7f4aae7fc640 1 -- 172.21.15.81:0/1223270305 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9802caf0 con 0x7f4a9c006170 2023-11-08T16:31:31.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.055+0000 7f4ab5739640 1 -- 172.21.15.81:0/1223270305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 msgr2=0x7f4a9c0069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:31.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.055+0000 7f4ab5739640 1 --2- 172.21.15.81:0/1223270305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 0x7f4a9c0069d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4a9800c210 tx=0x7f4a9c007a00 comp rx=0 tx=0).stop 2023-11-08T16:31:31.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.055+0000 7f4aaf7fe640 1 -- 172.21.15.81:0/1223270305 reap_dead start 2023-11-08T16:31:31.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.055+0000 7f4ab5739640 1 -- 172.21.15.81:0/1223270305 shutdown_connections 2023-11-08T16:31:31.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.055+0000 7f4ab5739640 1 -- 172.21.15.81:0/1223270305 >> 172.21.15.81:0/1223270305 conn(0x7f4ab00cb4e0 msgr2=0x7f4ab00cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:31.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.056+0000 7f4ab5739640 1 -- 172.21.15.81:0/1223270305 shutdown_connections 2023-11-08T16:31:31.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.056+0000 7f4ab5739640 1 -- 172.21.15.81:0/1223270305 wait complete. 2023-11-08T16:31:31.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.057+0000 7f4ab5739640 1 Processor -- start 2023-11-08T16:31:31.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.058+0000 7f4ab5739640 1 -- start start 2023-11-08T16:31:31.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.058+0000 7f4ab5739640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 0x7f4ab014f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.058+0000 7f4ab5739640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab01f3b40 0x7f4ab00c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4ab5739640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ab00c53e0 0x7f4ab00b1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4ab5739640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00b4130 con 0x7f4a9c006170 2023-11-08T16:31:31.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4ab5739640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00b3ca0 con 0x7f4ab00c53e0 2023-11-08T16:31:31.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4ab5739640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00b3090 con 0x7f4ab01f3b40 2023-11-08T16:31:31.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4aaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 0x7f4ab014f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4aaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 0x7f4ab014f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55514/0 (socket says 172.21.15.81:55514) 2023-11-08T16:31:31.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4aaeffd640 1 -- 172.21.15.81:0/588026114 learned_addr learned my addr 172.21.15.81:0/588026114 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:31.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4aaeffd640 1 -- 172.21.15.81:0/588026114 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab01f3b40 msgr2=0x7f4ab00c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:31.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab01f3b40 0x7f4ab00c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4aaeffd640 1 -- 172.21.15.81:0/588026114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ab00c53e0 msgr2=0x7f4ab00b1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:31.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ab00c53e0 0x7f4ab00b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.059+0000 7f4aaeffd640 1 -- 172.21.15.81:0/588026114 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00b3370 con 0x7f4a9c006170 2023-11-08T16:31:31.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.060+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 0x7f4ab014f890 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0011630 tx=0x7f4aa0011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:31.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.060+0000 7f4aa7fff640 1 -- 172.21.15.81:0/588026114 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0015870 con 0x7f4a9c006170 2023-11-08T16:31:31.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.060+0000 7f4aa7fff640 1 -- 172.21.15.81:0/588026114 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa00159d0 con 0x7f4a9c006170 2023-11-08T16:31:31.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.060+0000 7f4aa7fff640 1 -- 172.21.15.81:0/588026114 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0015d40 con 0x7f4a9c006170 2023-11-08T16:31:31.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.060+0000 7f4ab5739640 1 -- 172.21.15.81:0/588026114 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab00b3650 con 0x7f4a9c006170 2023-11-08T16:31:31.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.060+0000 7f4ab5739640 1 -- 172.21.15.81:0/588026114 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab00b1ba0 con 0x7f4a9c006170 2023-11-08T16:31:31.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.063+0000 7f4aa7fff640 1 -- 172.21.15.81:0/588026114 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4aa000f490 con 0x7f4a9c006170 2023-11-08T16:31:31.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.064+0000 7f4aa7fff640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4a9006aea0 0x7f4a9006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.064+0000 7f4aa7fff640 1 -- 172.21.15.81:0/588026114 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f4aa0089c60 con 0x7f4a9c006170 2023-11-08T16:31:31.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.064+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4a9006aea0 0x7f4a9006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.064+0000 7f4ab83a5740 1 Processor -- start 2023-11-08T16:31:31.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.065+0000 7f4ab83a5740 1 -- start start 2023-11-08T16:31:31.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.065+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4a9006aea0 0x7f4a9006d360 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4a98034000 tx=0x7f4a9802e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:31.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.065+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809dcb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.065+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8650 0x55e809dfb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.065+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809dcb710 0x55e809dfb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.065+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e809d3db60 con 0x55e809df8270 2023-11-08T16:31:31.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.065+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e809d3b250 con 0x55e809dcb710 2023-11-08T16:31:31.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.065+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e809d36b40 con 0x55e809df8650 2023-11-08T16:31:31.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809dcb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809dcb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55524/0 (socket says 172.21.15.81:55524) 2023-11-08T16:31:31.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aaeffd640 1 -- 172.21.15.81:0/586993541 learned_addr learned my addr 172.21.15.81:0/586993541 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:31.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/586993541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8650 0x55e809dfb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aaeffd640 1 -- 172.21.15.81:0/586993541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8650 msgr2=0x55e809dfb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/586993541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8650 0x55e809dfb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aaeffd640 1 -- 172.21.15.81:0/586993541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809dcb710 msgr2=0x55e809dfb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/586993541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809dcb710 0x55e809dfb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aaeffd640 1 -- 172.21.15.81:0/586993541 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e809d351c0 con 0x55e809df8270 2023-11-08T16:31:31.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/586993541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8650 0x55e809dfb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:31:31.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/586993541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809dcb1d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0087e50 tx=0x7f4aa0058f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb0616602d41a02 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:31.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.066+0000 7f4aa4ff9640 1 -- 172.21.15.81:0/586993541 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0015ea0 con 0x55e809df8270 2023-11-08T16:31:31.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.067+0000 7f4aa4ff9640 1 -- 172.21.15.81:0/586993541 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa0060a20 con 0x55e809df8270 2023-11-08T16:31:31.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.067+0000 7f4aa4ff9640 1 -- 172.21.15.81:0/586993541 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0056ac0 con 0x55e809df8270 2023-11-08T16:31:31.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.067+0000 7f4ab83a5740 1 -- 172.21.15.81:0/586993541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 msgr2=0x55e809dcb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:31.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.067+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/586993541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809dcb1d0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0087e50 tx=0x7f4aa0058f60 comp rx=0 tx=0).stop 2023-11-08T16:31:31.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.067+0000 7f4ab83a5740 1 -- 172.21.15.81:0/586993541 shutdown_connections 2023-11-08T16:31:31.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.067+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/586993541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809dcb710 0x55e809dfb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.067+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/586993541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8650 0x55e809dfb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.067+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/586993541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809dcb1d0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.067+0000 7f4ab83a5740 1 -- 172.21.15.81:0/586993541 >> 172.21.15.81:0/586993541 conn(0x55e809dcd480 msgr2=0x55e809e16000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:31.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.067+0000 7f4ab83a5740 1 -- 172.21.15.81:0/586993541 shutdown_connections 2023-11-08T16:31:31.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.067+0000 7f4ab83a5740 1 -- 172.21.15.81:0/586993541 wait complete. 2023-11-08T16:31:31.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.068+0000 7f4ab83a5740 1 -- start start 2023-11-08T16:31:31.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.068+0000 7f4ab83a5740 1 Processor -- start 2023-11-08T16:31:31.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.069+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8270 0x55e809de5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.069+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809de5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.069+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809e0cb20 0x55e809dfdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.069+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e809cd3dd0 con 0x55e809df8650 2023-11-08T16:31:31.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.069+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e809cd3a70 con 0x55e809e0cb20 2023-11-08T16:31:31.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.069+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e809a2ff90 con 0x55e809df8270 2023-11-08T16:31:31.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.069+0000 7f4ab83a5740 10 client.0 _ll_register_callbacks cb 0x55e809cf5e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:31:31.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4aa67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809de5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4aa67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809de5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55532/0 (socket says 172.21.15.81:55532) 2023-11-08T16:31:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1923462105 learned_addr learned my addr 172.21.15.81:0/1923462105 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4aaf7fe640 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809e0cb20 0x55e809dfdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8270 0x55e809de5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1923462105 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8270 msgr2=0x55e809de5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8270 0x55e809de5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1923462105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809e0cb20 msgr2=0x55e809dfdf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809e0cb20 0x55e809dfdf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1923462105 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab00b1ba0 con 0x55e809df8650 2023-11-08T16:31:31.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809de5eb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4a9802cb30 tx=0x7f4a9802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d57ee80fa785aa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:31.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.070+0000 7f4a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:31:31.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.071+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1923462105 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a98007270 con 0x55e809df8650 2023-11-08T16:31:31.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.071+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1923462105 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f4a9803c040 con 0x55e809df8650 2023-11-08T16:31:31.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.071+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1923462105 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e809d5d400 con 0x55e809df8650 2023-11-08T16:31:31.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.071+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1923462105 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a980073d0 con 0x55e809df8650 2023-11-08T16:31:31.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.071+0000 7f4ab83a5740 10 client.5763 fetch_fsmap learned FSMap version 32 2023-11-08T16:31:31.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.071+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1923462105 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e809cd2d20 con 0x55e809df8650 2023-11-08T16:31:31.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.071+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1923462105 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e809d5d400 con 0x55e809df8650 2023-11-08T16:31:31.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.071+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1923462105 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f4a98007530 con 0x55e809df8650 2023-11-08T16:31:31.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.071+0000 7f4ab83a5740 10 client.5763 fetch_fsmap finished waiting for FSMap version 32 2023-11-08T16:31:31.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.071+0000 7f4ab83a5740 10 client.5763 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-08T16:31:31.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.071+0000 7f4ab83a5740 20 client.5763 populate_metadata read hostname 'smithi081' 2023-11-08T16:31:31.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.072+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x55e809dfe440 0x55e809e336a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.072+0000 7f4ab83a5740 4 client.5763 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:31:31.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.072+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1923462105 --> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e809d2fb20 con 0x55e809dfe440 2023-11-08T16:31:31.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.072+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x55e809dfe440 0x55e809e336a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.072+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x55e809dfe440 0x55e809e336a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aeb4219ead7fb7 server_cookie=35276c4389cc8fd5 in_seq=0 out_seq=0 2023-11-08T16:31:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.072+0000 7f4a8dffb640 10 client.5763 ms_handle_connect on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1923462105 <== mds.0 v2:172.21.15.81:6836/2942618483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e809d2fb20 con 0x55e809dfe440 2023-11-08T16:31:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4a8dffb640 10 client.5763 handle_command_reply: tid=0 2023-11-08T16:31:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4ab83a5740 1 client.5763 shutdown 2023-11-08T16:31:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1923462105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 msgr2=0x55e809de5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809de5eb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4a9802cb30 tx=0x7f4a9802e070 comp rx=0 tx=0).stop 2023-11-08T16:31:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1923462105 shutdown_connections 2023-11-08T16:31:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x55e809dfe440 0x55e809e336a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809e0cb20 0x55e809dfdf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809de5eb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1923462105 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8270 0x55e809de5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4a8dffb640 0 client.5763 ms_handle_reset on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1923462105 >> 172.21.15.81:0/1923462105 conn(0x55e809dcb710 msgr2=0x55e809dcf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1923462105 shutdown_connections 2023-11-08T16:31:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.073+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1923462105 wait complete. 2023-11-08T16:31:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.074+0000 7f4ab83a5740 20 client.5763 trim_cache size 0 max 16384 2023-11-08T16:31:31.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.090+0000 7f4ab83a5740 1 Processor -- start 2023-11-08T16:31:31.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.090+0000 7f4ab83a5740 1 -- start start 2023-11-08T16:31:31.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.090+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809df5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.090+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8650 0x55e809dcbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.090+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809e0cb20 0x55e809df33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.090+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e809e16df0 con 0x55e809df8270 2023-11-08T16:31:31.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.090+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e809dca410 con 0x55e809e0cb20 2023-11-08T16:31:31.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.090+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e809e16a70 con 0x55e809df8650 2023-11-08T16:31:31.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.091+0000 7f4aaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809df5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.091+0000 7f4aaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809df5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55534/0 (socket says 172.21.15.81:55534) 2023-11-08T16:31:31.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.091+0000 7f4aaeffd640 1 -- 172.21.15.81:0/1331812211 learned_addr learned my addr 172.21.15.81:0/1331812211 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:31.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.091+0000 7f4aaeffd640 1 -- 172.21.15.81:0/1331812211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8650 msgr2=0x55e809dcbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.091+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/1331812211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8650 0x55e809dcbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.091+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/1331812211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8650 0x55e809dcbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.091+0000 7f4aaeffd640 1 -- 172.21.15.81:0/1331812211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809e0cb20 msgr2=0x55e809df33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.091+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/1331812211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809e0cb20 0x55e809df33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.091+0000 7f4aaeffd640 1 -- 172.21.15.81:0/1331812211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e809d66750 con 0x55e809df8270 2023-11-08T16:31:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.092+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/1331812211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809df5820 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0058420 tx=0x7f4aa001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785c4f4607417ff0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.092+0000 7f4aa57fa640 1 -- 172.21.15.81:0/1331812211 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0087040 con 0x55e809df8270 2023-11-08T16:31:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.092+0000 7f4aa57fa640 1 -- 172.21.15.81:0/1331812211 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa0054a60 con 0x55e809df8270 2023-11-08T16:31:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.092+0000 7f4aa57fa640 1 -- 172.21.15.81:0/1331812211 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0015ea0 con 0x55e809df8270 2023-11-08T16:31:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.092+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1331812211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 msgr2=0x55e809df5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.092+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1331812211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809df5820 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0058420 tx=0x7f4aa001f000 comp rx=0 tx=0).stop 2023-11-08T16:31:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.092+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1331812211 shutdown_connections 2023-11-08T16:31:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.092+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1331812211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809e0cb20 0x55e809df33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.092+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1331812211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809df8650 0x55e809dcbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.092+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1331812211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8270 0x55e809df5820 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.092+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1331812211 >> 172.21.15.81:0/1331812211 conn(0x55e809dcb710 msgr2=0x55e809dca580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:31.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.093+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1331812211 shutdown_connections 2023-11-08T16:31:31.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.093+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1331812211 wait complete. 2023-11-08T16:31:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.093+0000 7f4ab83a5740 1 -- start start 2023-11-08T16:31:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.093+0000 7f4ab83a5740 1 Processor -- start 2023-11-08T16:31:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.094+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809df8270 0x55e809e251f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.094+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809dcee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.094+0000 7f4ab83a5740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809e0cb20 0x55e809dcf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.094+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e809e25c00 con 0x55e809df8650 2023-11-08T16:31:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.094+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e809e80a10 con 0x55e809df8270 2023-11-08T16:31:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.094+0000 7f4ab83a5740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e809e80cf0 con 0x55e809e0cb20 2023-11-08T16:31:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.094+0000 7f4ab83a5740 10 client.0 _ll_register_callbacks cb 0x7f4a9c0058c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:31:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4aa67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809dcee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4aa67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809dcee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55542/0 (socket says 172.21.15.81:55542) 2023-11-08T16:31:31.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1735083681 learned_addr learned my addr 172.21.15.81:0/1735083681 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:31:31.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4aaf7fe640 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809e0cb20 0x55e809dcf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809df8270 0x55e809e251f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1735083681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809e0cb20 msgr2=0x55e809dcf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809e0cb20 0x55e809dcf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1735083681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809df8270 msgr2=0x55e809e251f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:31:31.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809df8270 0x55e809e251f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4aa67fc640 1 -- 172.21.15.81:0/1735083681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e809d351c0 con 0x55e809df8650 2023-11-08T16:31:31.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4aa67fc640 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809dcee10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4a9802c470 tx=0x7f4a98001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd502028d9a1a9f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:31:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4a8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:31:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.095+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1735083681 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a9805d070 con 0x55e809df8650 2023-11-08T16:31:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1735083681 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f4a9803c040 con 0x55e809df8650 2023-11-08T16:31:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1735083681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4aa0060a20 con 0x55e809df8650 2023-11-08T16:31:31.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1735083681 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4a980405c0 con 0x55e809df8650 2023-11-08T16:31:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4ab83a5740 10 client.5769 fetch_fsmap learned FSMap version 32 2023-11-08T16:31:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1735083681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x55e809cf8310 con 0x55e809df8650 2023-11-08T16:31:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1735083681 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa0060a20 con 0x55e809df8650 2023-11-08T16:31:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1735083681 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f4a98060020 con 0x55e809df8650 2023-11-08T16:31:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4ab83a5740 10 client.5769 fetch_fsmap finished waiting for FSMap version 32 2023-11-08T16:31:31.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4ab83a5740 10 client.5769 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-08T16:31:31.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4ab83a5740 20 client.5769 populate_metadata read hostname 'smithi081' 2023-11-08T16:31:31.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x55e809e25730 0x55e809e2f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:31:31.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4ab83a5740 4 client.5769 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-08T16:31:31.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.096+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1735083681 --> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e809d2f840 con 0x55e809e25730 2023-11-08T16:31:31.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.097+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x55e809e25730 0x55e809e2f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:31:31.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.097+0000 7f4a8dffb640 10 client.5769 ms_handle_connect on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:31.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.097+0000 7f4aaeffd640 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x55e809e25730 0x55e809e2f360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43495c6a7d8081e7 server_cookie=84a2367d58c7c92 in_seq=0 out_seq=0 2023-11-08T16:31:31.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.097+0000 7f4a8dffb640 1 -- 172.21.15.81:0/1735083681 <== mds.0 v2:172.21.15.81:6836/2942618483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55e809d2f840 con 0x55e809e25730 2023-11-08T16:31:31.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.097+0000 7f4a8dffb640 10 client.5769 handle_command_reply: tid=0 2023-11-08T16:31:31.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.097+0000 7f4ab83a5740 1 client.5769 shutdown 2023-11-08T16:31:31.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.097+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1735083681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 msgr2=0x55e809dcee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:31.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.097+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809dcee10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4a9802c470 tx=0x7f4a98001a70 comp rx=0 tx=0).stop 2023-11-08T16:31:31.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1735083681 shutdown_connections 2023-11-08T16:31:31.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x55e809e25730 0x55e809e2f360 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e809e0cb20 0x55e809dcf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e809df8650 0x55e809dcee10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab83a5740 1 --2- 172.21.15.81:0/1735083681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e809df8270 0x55e809e251f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4a8dffb640 0 client.5769 ms_handle_reset on v2:172.21.15.81:6836/2942618483 2023-11-08T16:31:31.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1735083681 >> 172.21.15.81:0/1735083681 conn(0x55e809dcb710 msgr2=0x55e809dfc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:31.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1735083681 shutdown_connections 2023-11-08T16:31:31.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab83a5740 1 -- 172.21.15.81:0/1735083681 wait complete. 2023-11-08T16:31:31.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab83a5740 20 client.5769 trim_cache size 0 max 16384 2023-11-08T16:31:31.116 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:31:31.116 INFO:teuthology.orchestra.run.smithi081.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-08T16:31:31.117 INFO:teuthology.orchestra.run.smithi081.stdout: "scrubs": { 2023-11-08T16:31:31.117 INFO:teuthology.orchestra.run.smithi081.stdout: "a6af52f2-e4be-4f7f-8ec9-6ba39af032ac": { 2023-11-08T16:31:31.117 INFO:teuthology.orchestra.run.smithi081.stdout: "path": "/", 2023-11-08T16:31:31.117 INFO:teuthology.orchestra.run.smithi081.stdout: "tag": "a6af52f2-e4be-4f7f-8ec9-6ba39af032ac", 2023-11-08T16:31:31.117 INFO:teuthology.orchestra.run.smithi081.stdout: "options": "recursive,repair" 2023-11-08T16:31:31.118 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:31:31.118 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:31:31.118 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:31:31.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab5739640 1 -- 172.21.15.81:0/588026114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4a9006aea0 msgr2=0x7f4a9006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:31.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab5739640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4a9006aea0 0x7f4a9006d360 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4a98034000 tx=0x7f4a9802e000 comp rx=0 tx=0).stop 2023-11-08T16:31:31.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab5739640 1 -- 172.21.15.81:0/588026114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 msgr2=0x7f4ab014f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:31:31.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab5739640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 0x7f4ab014f890 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0011630 tx=0x7f4aa0011af0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab5739640 1 -- 172.21.15.81:0/588026114 shutdown_connections 2023-11-08T16:31:31.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab5739640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4a9006aea0 0x7f4a9006d360 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab5739640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ab00c53e0 0x7f4ab00b1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab5739640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ab01f3b40 0x7f4ab00c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab5739640 1 --2- 172.21.15.81:0/588026114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4a9c006170 0x7f4ab014f890 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:31:31.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.098+0000 7f4ab5739640 1 -- 172.21.15.81:0/588026114 >> 172.21.15.81:0/588026114 conn(0x7f4ab00cb4e0 msgr2=0x7f4ab015c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:31:31.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.099+0000 7f4ab5739640 1 -- 172.21.15.81:0/588026114 shutdown_connections 2023-11-08T16:31:31.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:31:31.099+0000 7f4ab5739640 1 -- 172.21.15.81:0/588026114 wait complete. 2023-11-08T16:31:31.122 INFO:tasks.cephfs.filesystem:scrub status for tag:a6af52f2-e4be-4f7f-8ec9-6ba39af032ac - {'path': '/', 'tag': 'a6af52f2-e4be-4f7f-8ec9-6ba39af032ac', 'options': 'recursive,repair'} 2023-11-08T16:31:46.633 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:31:46.643 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:32:01.123 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 scrub status 2023-11-08T16:32:01.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.347+0000 7f7afe207640 1 Processor -- start 2023-11-08T16:32:01.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.348+0000 7f7afe207640 1 -- start start 2023-11-08T16:32:01.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.349+0000 7f7afe207640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7af815e170 0x7f7af815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.349+0000 7f7afe207640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7af815e550 0x7f7af80b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.349+0000 7f7afe207640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7af80b0e70 con 0x7f7af80b0a20 2023-11-08T16:32:01.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.349+0000 7f7afe207640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7af8071460 con 0x7f7af815e550 2023-11-08T16:32:01.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.349+0000 7f7afe207640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af80703c0 con 0x7f7af815e170 2023-11-08T16:32:01.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.349+0000 7f7aeffff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7af815e550 0x7f7af80b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.349+0000 7f7af77fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7af815e170 0x7f7af815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.350+0000 7f7af77fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7af815e170 0x7f7af815ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:43798/0 (socket says 172.21.15.81:43798) 2023-11-08T16:32:01.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.350+0000 7f7af77fe640 1 -- 172.21.15.81:0/978429416 learned_addr learned my addr 172.21.15.81:0/978429416 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:01.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.350+0000 7f7aeffff640 1 -- 172.21.15.81:0/978429416 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7af815e170 msgr2=0x7f7af815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:01.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.350+0000 7f7aeffff640 1 --2- 172.21.15.81:0/978429416 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7af815e170 0x7f7af815ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.350+0000 7f7aeffff640 1 -- 172.21.15.81:0/978429416 >> v1:172.21.15.93:6789/0 conn(0x7f7af80b0a20 legacy=0x7f7af80ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:01.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.350+0000 7f7aeffff640 1 -- 172.21.15.81:0/978429416 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af80706a0 con 0x7f7af815e550 2023-11-08T16:32:01.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.352+0000 7f7aeffff640 1 --2- 172.21.15.81:0/978429416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7af815e550 0x7f7af80b04e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00097a0 tx=0x7f7ae002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30d77a5f61e8f381 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:01.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.352+0000 7f7af6ffd640 1 -- 172.21.15.81:0/978429416 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae002d040 con 0x7f7af815e550 2023-11-08T16:32:01.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.352+0000 7f7af6ffd640 1 -- 172.21.15.81:0/978429416 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae0031810 con 0x7f7af815e550 2023-11-08T16:32:01.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.352+0000 7f7af6ffd640 1 -- 172.21.15.81:0/978429416 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0031ba0 con 0x7f7af815e550 2023-11-08T16:32:01.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.353+0000 7f7afe207640 1 -- 172.21.15.81:0/978429416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7af815e550 msgr2=0x7f7af80b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:01.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.353+0000 7f7afe207640 1 --2- 172.21.15.81:0/978429416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7af815e550 0x7f7af80b04e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7ae00097a0 tx=0x7f7ae002b8b0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.353+0000 7f7afe207640 1 -- 172.21.15.81:0/978429416 shutdown_connections 2023-11-08T16:32:01.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.353+0000 7f7afe207640 1 --2- 172.21.15.81:0/978429416 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7af815e550 0x7f7af80b04e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.353+0000 7f7afe207640 1 --2- 172.21.15.81:0/978429416 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7af815e170 0x7f7af815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.353+0000 7f7afe207640 1 -- 172.21.15.81:0/978429416 >> 172.21.15.81:0/978429416 conn(0x7f7af80cb4e0 msgr2=0x7f7af80cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:01.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.353+0000 7f7afe207640 1 -- 172.21.15.81:0/978429416 shutdown_connections 2023-11-08T16:32:01.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.353+0000 7f7afe207640 1 -- 172.21.15.81:0/978429416 wait complete. 2023-11-08T16:32:01.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.355+0000 7f7afe207640 1 Processor -- start 2023-11-08T16:32:01.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.355+0000 7f7afe207640 1 -- start start 2023-11-08T16:32:01.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.356+0000 7f7afe207640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7af80b0a20 0x7f7af815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.356+0000 7f7afe207640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af815e170 0x7f7af814e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.356+0000 7f7afe207640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7af815e550 0x7f7af8152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.356+0000 7f7afe207640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af814ed10 con 0x7f7af815e170 2023-11-08T16:32:01.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.356+0000 7f7afe207640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af8152b50 con 0x7f7af815e550 2023-11-08T16:32:01.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.356+0000 7f7afe207640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7af815ae20 con 0x7f7af80b0a20 2023-11-08T16:32:01.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.356+0000 7f7aeffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af815e170 0x7f7af814e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.356+0000 7f7aeffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af815e170 0x7f7af814e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46896/0 (socket says 172.21.15.81:46896) 2023-11-08T16:32:01.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.356+0000 7f7aeffff640 1 -- 172.21.15.81:0/1263034489 learned_addr learned my addr 172.21.15.81:0/1263034489 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:01.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7af77fe640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7af80b0a20 0x7f7af815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7af7fff640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7af815e550 0x7f7af8152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7aeffff640 1 -- 172.21.15.81:0/1263034489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7af80b0a20 msgr2=0x7f7af815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:01.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7aeffff640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7af80b0a20 0x7f7af815bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7aeffff640 1 -- 172.21.15.81:0/1263034489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7af815e550 msgr2=0x7f7af8152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:01.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7aeffff640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7af815e550 0x7f7af8152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7aeffff640 1 -- 172.21.15.81:0/1263034489 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae0009400 con 0x7f7af815e170 2023-11-08T16:32:01.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7aeffff640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af815e170 0x7f7af814e7d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0000c00 tx=0x7f7ae00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:01.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7af4ff9640 1 -- 172.21.15.81:0/1263034489 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0006e10 con 0x7f7af815e170 2023-11-08T16:32:01.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7af4ff9640 1 -- 172.21.15.81:0/1263034489 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae003ad60 con 0x7f7af815e170 2023-11-08T16:32:01.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7afe207640 1 -- 172.21.15.81:0/1263034489 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af815b100 con 0x7f7af815e170 2023-11-08T16:32:01.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.357+0000 7f7af4ff9640 1 -- 172.21.15.81:0/1263034489 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0044630 con 0x7f7af815e170 2023-11-08T16:32:01.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.358+0000 7f7afe207640 1 -- 172.21.15.81:0/1263034489 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af815b6c0 con 0x7f7af815e170 2023-11-08T16:32:01.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.360+0000 7f7af4ff9640 1 -- 172.21.15.81:0/1263034489 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ae004d450 con 0x7f7af815e170 2023-11-08T16:32:01.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.361+0000 7f7af4ff9640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ad806ae70 0x7f7ad806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.361+0000 7f7af4ff9640 1 -- 172.21.15.81:0/1263034489 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f7ae00abaa0 con 0x7f7af815e170 2023-11-08T16:32:01.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.361+0000 7f7af77fe640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ad806ae70 0x7f7ad806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.362+0000 7f7af77fe640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ad806ae70 0x7f7ad806d330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009b60 tx=0x7f7ae8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:01.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.362+0000 7f7b00d97740 1 Processor -- start 2023-11-08T16:32:01.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.362+0000 7f7b00d97740 1 -- start start 2023-11-08T16:32:01.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24270 0x56152eef71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24650 0x56152ef27000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152eef7710 0x56152ef27540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7b00d97740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152ee69b60 con 0x56152ef24270 2023-11-08T16:32:01.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7b00d97740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152ee67250 con 0x56152eef7710 2023-11-08T16:32:01.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7b00d97740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56152ee62b40 con 0x56152ef24650 2023-11-08T16:32:01.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7af77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24270 0x56152eef71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7af77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24270 0x56152eef71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46902/0 (socket says 172.21.15.81:46902) 2023-11-08T16:32:01.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7af77fe640 1 -- 172.21.15.81:0/1445262708 learned_addr learned my addr 172.21.15.81:0/1445262708 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:01.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7af77fe640 1 -- 172.21.15.81:0/1445262708 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24650 msgr2=0x56152ef27000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:01.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7af77fe640 1 --2- 172.21.15.81:0/1445262708 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24650 0x56152ef27000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7af77fe640 1 -- 172.21.15.81:0/1445262708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152eef7710 msgr2=0x56152ef27540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:01.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7af77fe640 1 --2- 172.21.15.81:0/1445262708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152eef7710 0x56152ef27540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.363+0000 7f7af77fe640 1 -- 172.21.15.81:0/1445262708 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56152ee611c0 con 0x56152ef24270 2023-11-08T16:32:01.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.364+0000 7f7af77fe640 1 --2- 172.21.15.81:0/1445262708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24270 0x56152eef71d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8011dc0 tx=0x7f7ae8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87d5d667b0c4427 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:01.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.364+0000 7f7aed7fa640 1 -- 172.21.15.81:0/1445262708 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8011df0 con 0x56152ef24270 2023-11-08T16:32:01.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.364+0000 7f7aed7fa640 1 -- 172.21.15.81:0/1445262708 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae8021440 con 0x56152ef24270 2023-11-08T16:32:01.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.364+0000 7f7aed7fa640 1 -- 172.21.15.81:0/1445262708 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae80217b0 con 0x56152ef24270 2023-11-08T16:32:01.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.364+0000 7f7b00d97740 1 -- 172.21.15.81:0/1445262708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24270 msgr2=0x56152eef71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:01.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.364+0000 7f7b00d97740 1 --2- 172.21.15.81:0/1445262708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24270 0x56152eef71d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8011dc0 tx=0x7f7ae8014000 comp rx=0 tx=0).stop 2023-11-08T16:32:01.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.365+0000 7f7b00d97740 1 -- 172.21.15.81:0/1445262708 shutdown_connections 2023-11-08T16:32:01.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.365+0000 7f7b00d97740 1 --2- 172.21.15.81:0/1445262708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152eef7710 0x56152ef27540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.365+0000 7f7b00d97740 1 --2- 172.21.15.81:0/1445262708 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24650 0x56152ef27000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.365+0000 7f7b00d97740 1 --2- 172.21.15.81:0/1445262708 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24270 0x56152eef71d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.365+0000 7f7b00d97740 1 -- 172.21.15.81:0/1445262708 >> 172.21.15.81:0/1445262708 conn(0x56152eef9480 msgr2=0x56152ef42000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:01.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.365+0000 7f7b00d97740 1 -- 172.21.15.81:0/1445262708 shutdown_connections 2023-11-08T16:32:01.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.365+0000 7f7b00d97740 1 -- 172.21.15.81:0/1445262708 wait complete. 2023-11-08T16:32:01.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.365+0000 7f7b00d97740 1 -- start start 2023-11-08T16:32:01.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.366+0000 7f7b00d97740 1 Processor -- start 2023-11-08T16:32:01.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef24270 0x56152ef28d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef29290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef42bd0 0x56152ef3b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7b00d97740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152edffdd0 con 0x56152ef24650 2023-11-08T16:32:01.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7b00d97740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152edffa70 con 0x56152ef24270 2023-11-08T16:32:01.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7b00d97740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56152eb5bf90 con 0x56152ef42bd0 2023-11-08T16:32:01.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7b00d97740 10 client.0 _ll_register_callbacks cb 0x56152ee21e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:32:01.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7aeffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef29290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7aeffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef29290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46914/0 (socket says 172.21.15.81:46914) 2023-11-08T16:32:01.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7aeffff640 1 -- 172.21.15.81:0/892388953 learned_addr learned my addr 172.21.15.81:0/892388953 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:01.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7aeffff640 1 -- 172.21.15.81:0/892388953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef42bd0 msgr2=0x56152ef3b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:01.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7aeffff640 1 --2- 172.21.15.81:0/892388953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef42bd0 0x56152ef3b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7aeffff640 1 -- 172.21.15.81:0/892388953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef24270 msgr2=0x56152ef28d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:01.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7aeffff640 1 --2- 172.21.15.81:0/892388953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef24270 0x56152ef28d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.367+0000 7f7aeffff640 1 -- 172.21.15.81:0/892388953 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae8011b00 con 0x56152ef24650 2023-11-08T16:32:01.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.368+0000 7f7aeffff640 1 --2- 172.21.15.81:0/892388953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef29290 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7ae007a4f0 tx=0x7f7ae007ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64aa704bd0bcc7d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:01.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.368+0000 7f7ad67fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:32:01.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.368+0000 7f7ad67fc640 1 -- 172.21.15.81:0/892388953 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae007a9c0 con 0x56152ef24650 2023-11-08T16:32:01.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.368+0000 7f7ad67fc640 1 -- 172.21.15.81:0/892388953 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f7ae0079a50 con 0x56152ef24650 2023-11-08T16:32:01.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.368+0000 7f7b00d97740 1 -- 172.21.15.81:0/892388953 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56152ee89400 con 0x56152ef24650 2023-11-08T16:32:01.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.368+0000 7f7ad67fc640 1 -- 172.21.15.81:0/892388953 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae0079c10 con 0x56152ef24650 2023-11-08T16:32:01.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.368+0000 7f7b00d97740 10 client.5778 fetch_fsmap learned FSMap version 32 2023-11-08T16:32:01.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.368+0000 7f7b00d97740 1 -- 172.21.15.81:0/892388953 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x56152edfed20 con 0x56152ef24650 2023-11-08T16:32:01.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.368+0000 7f7ad67fc640 1 -- 172.21.15.81:0/892388953 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56152ee89400 con 0x56152ef24650 2023-11-08T16:32:01.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.368+0000 7f7ad67fc640 1 -- 172.21.15.81:0/892388953 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f7ae0098070 con 0x56152ef24650 2023-11-08T16:32:01.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.369+0000 7f7b00d97740 10 client.5778 fetch_fsmap finished waiting for FSMap version 32 2023-11-08T16:32:01.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.369+0000 7f7b00d97740 10 client.5778 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-08T16:32:01.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.369+0000 7f7b00d97740 20 client.5778 populate_metadata read hostname 'smithi081' 2023-11-08T16:32:01.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.369+0000 7f7b00d97740 1 --2- 172.21.15.81:0/892388953 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x56152ef125f0 0x56152ef12a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.369+0000 7f7b00d97740 4 client.5778 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:32:01.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.369+0000 7f7b00d97740 1 -- 172.21.15.81:0/892388953 --> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56152ee5bb20 con 0x56152ef125f0 2023-11-08T16:32:01.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.369+0000 7f7af77fe640 1 --2- 172.21.15.81:0/892388953 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x56152ef125f0 0x56152ef12a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.370+0000 7f7ad67fc640 10 client.5778 ms_handle_connect on v2:172.21.15.81:6836/2942618483 2023-11-08T16:32:01.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.370+0000 7f7af77fe640 1 --2- 172.21.15.81:0/892388953 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x56152ef125f0 0x56152ef12a00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60ff5964b48ff04a server_cookie=e047eddc23860dc9 in_seq=0 out_seq=0 2023-11-08T16:32:01.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.370+0000 7f7ad67fc640 1 -- 172.21.15.81:0/892388953 <== mds.0 v2:172.21.15.81:6836/2942618483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56152ee5bb20 con 0x56152ef125f0 2023-11-08T16:32:01.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.370+0000 7f7ad67fc640 10 client.5778 handle_command_reply: tid=0 2023-11-08T16:32:01.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.370+0000 7f7b00d97740 1 client.5778 shutdown 2023-11-08T16:32:01.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.370+0000 7f7b00d97740 1 -- 172.21.15.81:0/892388953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 msgr2=0x56152ef29290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:01.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.370+0000 7f7b00d97740 1 --2- 172.21.15.81:0/892388953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef29290 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f7ae007a4f0 tx=0x7f7ae007ada0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.371+0000 7f7b00d97740 1 -- 172.21.15.81:0/892388953 shutdown_connections 2023-11-08T16:32:01.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.371+0000 7f7b00d97740 1 --2- 172.21.15.81:0/892388953 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x56152ef125f0 0x56152ef12a00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.371+0000 7f7b00d97740 1 --2- 172.21.15.81:0/892388953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef42bd0 0x56152ef3b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.371+0000 7f7b00d97740 1 --2- 172.21.15.81:0/892388953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef29290 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.371+0000 7f7b00d97740 1 --2- 172.21.15.81:0/892388953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef24270 0x56152ef28d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.371+0000 7f7ad67fc640 0 client.5778 ms_handle_reset on v2:172.21.15.81:6836/2942618483 2023-11-08T16:32:01.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.371+0000 7f7b00d97740 1 -- 172.21.15.81:0/892388953 >> 172.21.15.81:0/892388953 conn(0x56152eef7710 msgr2=0x56152eefb220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:01.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.371+0000 7f7b00d97740 1 -- 172.21.15.81:0/892388953 shutdown_connections 2023-11-08T16:32:01.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.371+0000 7f7b00d97740 1 -- 172.21.15.81:0/892388953 wait complete. 2023-11-08T16:32:01.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.371+0000 7f7b00d97740 20 client.5778 trim_cache size 0 max 16384 2023-11-08T16:32:01.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.388+0000 7f7b00d97740 1 Processor -- start 2023-11-08T16:32:01.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.388+0000 7f7b00d97740 1 -- start start 2023-11-08T16:32:01.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.388+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24270 0x56152eef84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef44e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef42bd0 0x56152efa6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7b00d97740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152ef45d50 con 0x56152ef24650 2023-11-08T16:32:01.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7b00d97740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152ef51a60 con 0x56152ef42bd0 2023-11-08T16:32:01.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7b00d97740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56152ef51630 con 0x56152ef24270 2023-11-08T16:32:01.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7aeffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef44e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7aeffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef44e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46928/0 (socket says 172.21.15.81:46928) 2023-11-08T16:32:01.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7aeffff640 1 -- 172.21.15.81:0/3434359020 learned_addr learned my addr 172.21.15.81:0/3434359020 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:01.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7aeffff640 1 -- 172.21.15.81:0/3434359020 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24270 msgr2=0x56152eef84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:01.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7aeffff640 1 --2- 172.21.15.81:0/3434359020 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24270 0x56152eef84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7aeffff640 1 -- 172.21.15.81:0/3434359020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef42bd0 msgr2=0x56152efa6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:01.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7aeffff640 1 --2- 172.21.15.81:0/3434359020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef42bd0 0x56152efa6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7aeffff640 1 -- 172.21.15.81:0/3434359020 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56152ee92750 con 0x56152ef24650 2023-11-08T16:32:01.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.389+0000 7f7aeffff640 1 --2- 172.21.15.81:0/3434359020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef44e80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0076f80 tx=0x7f7ae0035000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a8d58966156df8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:01.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7aedffb640 1 -- 172.21.15.81:0/3434359020 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae007a0a0 con 0x56152ef24650 2023-11-08T16:32:01.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7aedffb640 1 -- 172.21.15.81:0/3434359020 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae0079a50 con 0x56152ef24650 2023-11-08T16:32:01.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7aedffb640 1 -- 172.21.15.81:0/3434359020 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0082640 con 0x56152ef24650 2023-11-08T16:32:01.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7b00d97740 1 -- 172.21.15.81:0/3434359020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 msgr2=0x56152ef44e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:01.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7b00d97740 1 --2- 172.21.15.81:0/3434359020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef44e80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0076f80 tx=0x7f7ae0035000 comp rx=0 tx=0).stop 2023-11-08T16:32:01.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7b00d97740 1 -- 172.21.15.81:0/3434359020 shutdown_connections 2023-11-08T16:32:01.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7b00d97740 1 --2- 172.21.15.81:0/3434359020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef42bd0 0x56152efa6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7b00d97740 1 --2- 172.21.15.81:0/3434359020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152ef44e80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7b00d97740 1 --2- 172.21.15.81:0/3434359020 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24270 0x56152eef84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7b00d97740 1 -- 172.21.15.81:0/3434359020 >> 172.21.15.81:0/3434359020 conn(0x56152eef7710 msgr2=0x56152ef45510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:01.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7b00d97740 1 -- 172.21.15.81:0/3434359020 shutdown_connections 2023-11-08T16:32:01.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.390+0000 7f7b00d97740 1 -- 172.21.15.81:0/3434359020 wait complete. 2023-11-08T16:32:01.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.391+0000 7f7b00d97740 1 -- start start 2023-11-08T16:32:01.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.391+0000 7f7b00d97740 1 Processor -- start 2023-11-08T16:32:01.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.392+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24270 0x56152ef13600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.392+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152eefad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.392+0000 7f7b00d97740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef42bd0 0x56152eefb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.392+0000 7f7b00d97740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152ef43250 con 0x56152ef24650 2023-11-08T16:32:01.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.392+0000 7f7b00d97740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152ef54180 con 0x56152ef42bd0 2023-11-08T16:32:01.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.392+0000 7f7b00d97740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56152ef54460 con 0x56152ef24270 2023-11-08T16:32:01.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.392+0000 7f7b00d97740 10 client.0 _ll_register_callbacks cb 0x7f7ae0009110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:32:01.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.392+0000 7f7aeffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152eefad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7aeffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152eefad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:46930/0 (socket says 172.21.15.81:46930) 2023-11-08T16:32:01.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7aeffff640 1 -- 172.21.15.81:0/1756850533 learned_addr learned my addr 172.21.15.81:0/1756850533 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:01.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7aeffff640 1 -- 172.21.15.81:0/1756850533 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24270 msgr2=0x56152ef13600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:01.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7aeffff640 1 --2- 172.21.15.81:0/1756850533 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24270 0x56152ef13600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7aeffff640 1 -- 172.21.15.81:0/1756850533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef42bd0 msgr2=0x56152eefb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:01.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7aeffff640 1 --2- 172.21.15.81:0/1756850533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef42bd0 0x56152eefb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7aeffff640 1 -- 172.21.15.81:0/1756850533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56152edfed20 con 0x56152ef24650 2023-11-08T16:32:01.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7aeffff640 1 --2- 172.21.15.81:0/1756850533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152eefad70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7ae002b8f0 tx=0x7f7ae002bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fb28579641dabf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:01.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7ad67fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:32:01.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7ad67fc640 1 -- 172.21.15.81:0/1756850533 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae0079bc0 con 0x56152ef24650 2023-11-08T16:32:01.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7ad67fc640 1 -- 172.21.15.81:0/1756850533 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f7ae007a9d0 con 0x56152ef24650 2023-11-08T16:32:01.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7b00d97740 1 -- 172.21.15.81:0/1756850533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ae8021440 con 0x56152ef24650 2023-11-08T16:32:01.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.393+0000 7f7ad67fc640 1 -- 172.21.15.81:0/1756850533 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae007e170 con 0x56152ef24650 2023-11-08T16:32:01.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.394+0000 7f7b00d97740 10 client.5784 fetch_fsmap learned FSMap version 32 2023-11-08T16:32:01.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.394+0000 7f7b00d97740 1 -- 172.21.15.81:0/1756850533 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) v3 -- 0x56152ee24310 con 0x56152ef24650 2023-11-08T16:32:01.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.394+0000 7f7ad67fc640 1 -- 172.21.15.81:0/1756850533 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae8021440 con 0x56152ef24650 2023-11-08T16:32:01.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.394+0000 7f7ad67fc640 1 -- 172.21.15.81:0/1756850533 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 32) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f7ae0098070 con 0x56152ef24650 2023-11-08T16:32:01.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.394+0000 7f7b00d97740 10 client.5784 fetch_fsmap finished waiting for FSMap version 32 2023-11-08T16:32:01.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.394+0000 7f7b00d97740 10 client.5784 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2023-11-08T16:32:01.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.394+0000 7f7b00d97740 20 client.5784 populate_metadata read hostname 'smithi081' 2023-11-08T16:32:01.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.394+0000 7f7b00d97740 1 --2- 172.21.15.81:0/1756850533 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x56152ef93bb0 0x56152ef37990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:01.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.394+0000 7f7b00d97740 4 client.5784 mds_command: new command op to 5655 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-08T16:32:01.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.394+0000 7f7b00d97740 1 -- 172.21.15.81:0/1756850533 --> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56152ee5b840 con 0x56152ef93bb0 2023-11-08T16:32:01.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.394+0000 7f7af77fe640 1 --2- 172.21.15.81:0/1756850533 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x56152ef93bb0 0x56152ef37990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:01.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7af77fe640 1 --2- 172.21.15.81:0/1756850533 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x56152ef93bb0 0x56152ef37990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a6faf654b0c0dcc server_cookie=7539d273f969f205 in_seq=0 out_seq=0 2023-11-08T16:32:01.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7ad67fc640 10 client.5784 ms_handle_connect on v2:172.21.15.81:6836/2942618483 2023-11-08T16:32:01.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7ad67fc640 1 -- 172.21.15.81:0/1756850533 <== mds.0 v2:172.21.15.81:6836/2942618483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56152ee5b840 con 0x56152ef93bb0 2023-11-08T16:32:01.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7ad67fc640 10 client.5784 handle_command_reply: tid=0 2023-11-08T16:32:01.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 1 client.5784 shutdown 2023-11-08T16:32:01.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 1 -- 172.21.15.81:0/1756850533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 msgr2=0x56152eefad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:01.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 1 --2- 172.21.15.81:0/1756850533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152eefad70 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f7ae002b8f0 tx=0x7f7ae002bdd0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 1 -- 172.21.15.81:0/1756850533 shutdown_connections 2023-11-08T16:32:01.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 1 --2- 172.21.15.81:0/1756850533 >> [v2:172.21.15.81:6836/2942618483,v1:172.21.15.81:6837/2942618483] conn(0x56152ef93bb0 0x56152ef37990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 1 --2- 172.21.15.81:0/1756850533 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152ef42bd0 0x56152eefb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 1 --2- 172.21.15.81:0/1756850533 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152ef24650 0x56152eefad70 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 1 --2- 172.21.15.81:0/1756850533 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152ef24270 0x56152ef13600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7ad67fc640 0 client.5784 ms_handle_reset on v2:172.21.15.81:6836/2942618483 2023-11-08T16:32:01.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 1 -- 172.21.15.81:0/1756850533 >> 172.21.15.81:0/1756850533 conn(0x56152eef7710 msgr2=0x56152eef6270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:01.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 1 -- 172.21.15.81:0/1756850533 shutdown_connections 2023-11-08T16:32:01.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 1 -- 172.21.15.81:0/1756850533 wait complete. 2023-11-08T16:32:01.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.395+0000 7f7b00d97740 20 client.5784 trim_cache size 0 max 16384 2023-11-08T16:32:01.410 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:32:01.410 INFO:teuthology.orchestra.run.smithi081.stdout: "status": "no active scrubs running", 2023-11-08T16:32:01.410 INFO:teuthology.orchestra.run.smithi081.stdout: "scrubs": {} 2023-11-08T16:32:01.411 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:32:01.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 -- 172.21.15.81:0/1263034489 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ad806ae70 msgr2=0x7f7ad806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:01.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ad806ae70 0x7f7ad806d330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8009b60 tx=0x7f7ae8008040 comp rx=0 tx=0).stop 2023-11-08T16:32:01.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 -- 172.21.15.81:0/1263034489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af815e170 msgr2=0x7f7af814e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:01.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af815e170 0x7f7af814e7d0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0000c00 tx=0x7f7ae00046c0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 -- 172.21.15.81:0/1263034489 shutdown_connections 2023-11-08T16:32:01.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7ad806ae70 0x7f7ad806d330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7af815e550 0x7f7af8152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7af815e170 0x7f7af814e7d0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 --2- 172.21.15.81:0/1263034489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7af80b0a20 0x7f7af815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:01.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 -- 172.21.15.81:0/1263034489 >> 172.21.15.81:0/1263034489 conn(0x7f7af80cb4e0 msgr2=0x7f7af80cf160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:01.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 -- 172.21.15.81:0/1263034489 shutdown_connections 2023-11-08T16:32:01.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:01.396+0000 7f7afe207640 1 -- 172.21.15.81:0/1263034489 wait complete. 2023-11-08T16:32:01.416 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-08T16:32:01.416 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/0 2023-11-08T16:32:01.417 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:32:01.417 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:32:01.417 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-11-08T16:32:01.418 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:32:01.418 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:32:01.418 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:32:01.419 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:32:01.419 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:32:01.419 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:32:01.420 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:32:01.420 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:32:01.420 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:32:01.421 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:32:01.421 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:32:05.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:32:05.129 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/1 2023-11-08T16:32:05.130 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:32:05.130 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:32:05.130 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-11-08T16:32:05.131 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:32:05.131 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:32:05.131 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:32:05.132 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:32:05.132 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:32:05.133 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:32:05.133 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:32:05.134 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:32:05.134 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:32:05.134 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:32:05.135 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:32:08.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:32:08.799 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/2 2023-11-08T16:32:08.800 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:32:08.800 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:32:08.801 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-11-08T16:32:08.801 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:32:08.801 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:32:08.802 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:32:08.802 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:32:08.803 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:32:08.803 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:32:08.803 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:32:08.804 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:32:08.804 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:32:08.805 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:32:08.805 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:32:12.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:32:12.471 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/3 2023-11-08T16:32:12.472 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:32:12.472 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:32:12.472 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-11-08T16:32:12.473 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:32:12.473 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:32:12.473 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:32:12.474 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:32:12.474 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:32:12.475 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:32:12.475 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:32:12.476 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:32:12.476 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:32:12.476 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:32:12.477 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:32:16.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:32:16.154 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/4 2023-11-08T16:32:16.155 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:32:16.155 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:32:16.155 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-11-08T16:32:16.156 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:32:16.156 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:32:16.156 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:32:16.157 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:32:16.157 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:32:16.158 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:32:16.158 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:32:16.158 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:32:16.159 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:32:16.159 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:32:16.159 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:32:16.780 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:32:16.789 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:32:19.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:32:19.794 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T16:32:19.795 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:32:19.902 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:32:20.018 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T16:32:20.019 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:32:20.019 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:32:20.081 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:32:20.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.288+0000 7fac9201a640 1 Processor -- start 2023-11-08T16:32:20.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.288+0000 7fac9201a640 1 -- start start 2023-11-08T16:32:20.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.290+0000 7fac9201a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac8c15aac0 0x7fac8c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:20.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.290+0000 7fac9201a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fac8c0716e0 0x7fac8c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:20.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.290+0000 7fac9201a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac8c070470 con 0x7fac8c0cf250 2023-11-08T16:32:20.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.290+0000 7fac9201a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac8c06f650 con 0x7fac8c0716e0 2023-11-08T16:32:20.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.290+0000 7fac9201a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c0cf6c0 con 0x7fac8c15aac0 2023-11-08T16:32:20.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.290+0000 7fac90817640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fac8c0716e0 0x7fac8c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:20.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.290+0000 7fac91819640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fac8c0cf250 0x7fac8c06b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46732/0 (socket says 172.21.15.81:46732) 2023-11-08T16:32:20.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.290+0000 7fac91819640 1 -- 172.21.15.81:0/24369903 learned_addr learned my addr 172.21.15.81:0/24369903 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:20.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.290+0000 7fac91018640 1 --2- 172.21.15.81:0/24369903 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac8c15aac0 0x7fac8c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:20.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.291+0000 7fac90817640 1 -- 172.21.15.81:0/24369903 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac8c15aac0 msgr2=0x7fac8c0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:20.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.291+0000 7fac90817640 1 --2- 172.21.15.81:0/24369903 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac8c15aac0 0x7fac8c0711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.291+0000 7fac90817640 1 -- 172.21.15.81:0/24369903 >> v1:172.21.15.81:6789/0 conn(0x7fac8c0cf250 legacy=0x7fac8c06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.291+0000 7fac90817640 1 -- 172.21.15.81:0/24369903 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8c06d580 con 0x7fac8c0716e0 2023-11-08T16:32:20.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.291+0000 7fac83fff640 1 -- 172.21.15.81:0/24369903 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784288560 0 0) 0x7fac8c070470 con 0x7fac8c0cf250 2023-11-08T16:32:20.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.292+0000 7fac90817640 1 --2- 172.21.15.81:0/24369903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac8c0716e0 0x7fac8c0ced10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0097a0 tx=0x7fac7c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ac434e843cab23a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:20.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.292+0000 7fac83fff640 1 -- 172.21.15.81:0/24369903 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c031830 con 0x7fac8c0716e0 2023-11-08T16:32:20.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.292+0000 7fac83fff640 1 -- 172.21.15.81:0/24369903 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c031990 con 0x7fac8c0716e0 2023-11-08T16:32:20.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.292+0000 7fac83fff640 1 -- 172.21.15.81:0/24369903 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c031d20 con 0x7fac8c0716e0 2023-11-08T16:32:20.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.292+0000 7fac9201a640 1 -- 172.21.15.81:0/24369903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac8c0716e0 msgr2=0x7fac8c0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.292+0000 7fac9201a640 1 --2- 172.21.15.81:0/24369903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac8c0716e0 0x7fac8c0ced10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0097a0 tx=0x7fac7c02b8d0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.293+0000 7fac9201a640 1 -- 172.21.15.81:0/24369903 shutdown_connections 2023-11-08T16:32:20.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.293+0000 7fac9201a640 1 --2- 172.21.15.81:0/24369903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac8c0716e0 0x7fac8c0ced10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.293+0000 7fac9201a640 1 --2- 172.21.15.81:0/24369903 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac8c15aac0 0x7fac8c0711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.293+0000 7fac9201a640 1 -- 172.21.15.81:0/24369903 >> 172.21.15.81:0/24369903 conn(0x7fac8c06a4a0 msgr2=0x7fac8c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:20.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.293+0000 7fac9201a640 1 -- 172.21.15.81:0/24369903 shutdown_connections 2023-11-08T16:32:20.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.293+0000 7fac9201a640 1 -- 172.21.15.81:0/24369903 wait complete. 2023-11-08T16:32:20.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.294+0000 7fac9201a640 1 Processor -- start 2023-11-08T16:32:20.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.294+0000 7fac9201a640 1 -- start start 2023-11-08T16:32:20.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.295+0000 7fac9201a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac8c0716e0 0x7fac8c13fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:20.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.295+0000 7fac9201a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac8c0cf250 0x7fac8c1425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:20.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.295+0000 7fac9201a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac8c15aac0 0x7fac8c142b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:20.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.295+0000 7fac9201a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c15c830 con 0x7fac8c15aac0 2023-11-08T16:32:20.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.295+0000 7fac9201a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c140220 con 0x7fac8c0cf250 2023-11-08T16:32:20.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.295+0000 7fac9201a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c15c270 con 0x7fac8c0716e0 2023-11-08T16:32:20.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.296+0000 7fac91819640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac8c15aac0 0x7fac8c142b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:20.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.296+0000 7fac91819640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac8c15aac0 0x7fac8c142b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56008/0 (socket says 172.21.15.81:56008) 2023-11-08T16:32:20.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.296+0000 7fac91819640 1 -- 172.21.15.81:0/3500720684 learned_addr learned my addr 172.21.15.81:0/3500720684 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:20.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.296+0000 7fac91018640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac8c0716e0 0x7fac8c13fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:20.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.296+0000 7fac90817640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac8c0cf250 0x7fac8c1425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:20.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.296+0000 7fac91819640 1 -- 172.21.15.81:0/3500720684 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac8c0716e0 msgr2=0x7fac8c13fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.296+0000 7fac91819640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac8c0716e0 0x7fac8c13fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.296+0000 7fac91819640 1 -- 172.21.15.81:0/3500720684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac8c0cf250 msgr2=0x7fac8c1425c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.296+0000 7fac91819640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac8c0cf250 0x7fac8c1425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.296+0000 7fac91819640 1 -- 172.21.15.81:0/3500720684 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac7c009400 con 0x7fac8c15aac0 2023-11-08T16:32:20.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.296+0000 7fac91819640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac8c15aac0 0x7fac8c142b00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fac8400c7e0 tx=0x7fac8400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:20.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.297+0000 7fac81ffb640 1 -- 172.21.15.81:0/3500720684 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400a640 con 0x7fac8c15aac0 2023-11-08T16:32:20.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.297+0000 7fac81ffb640 1 -- 172.21.15.81:0/3500720684 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8400a7c0 con 0x7fac8c15aac0 2023-11-08T16:32:20.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.297+0000 7fac81ffb640 1 -- 172.21.15.81:0/3500720684 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8400ab70 con 0x7fac8c15aac0 2023-11-08T16:32:20.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.297+0000 7fac9201a640 1 -- 172.21.15.81:0/3500720684 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8c15c550 con 0x7fac8c15aac0 2023-11-08T16:32:20.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.297+0000 7fac9201a640 1 -- 172.21.15.81:0/3500720684 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8c140940 con 0x7fac8c15aac0 2023-11-08T16:32:20.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.298+0000 7fac9201a640 1 -- 172.21.15.81:0/3500720684 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac8c0b30f0 con 0x7fac8c15aac0 2023-11-08T16:32:20.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.299+0000 7fac81ffb640 1 -- 172.21.15.81:0/3500720684 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fac8400acd0 con 0x7fac8c15aac0 2023-11-08T16:32:20.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.300+0000 7fac81ffb640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac7006ae70 0x7fac7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:20.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.300+0000 7fac81ffb640 1 -- 172.21.15.81:0/3500720684 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7fac84088ea0 con 0x7fac8c15aac0 2023-11-08T16:32:20.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.300+0000 7fac91018640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac7006ae70 0x7fac7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:20.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.301+0000 7fac91018640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac7006ae70 0x7fac7006d330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fac88006fd0 tx=0x7fac88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:20.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.305+0000 7fac81ffb640 1 -- 172.21.15.81:0/3500720684 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fac84053ca0 con 0x7fac8c15aac0 2023-11-08T16:32:20.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.480+0000 7fac9201a640 1 -- 172.21.15.81:0/3500720684 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac8c06d190 con 0x7fac8c15aac0 2023-11-08T16:32:20.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.483+0000 7fac81ffb640 1 -- 172.21.15.81:0/3500720684 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18684 (secure 0 0 0) 0x7fac84018070 con 0x7fac8c15aac0 2023-11-08T16:32:20.486 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:20.486 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":44,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:31:24.833641+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:30:27.977481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T16:30:29.171902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:30:40.141171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/1759199271":"2023-11-09T16:31:23.834479+0000","172.21.15.81:6836/1759199271":"2023-11-09T16:31:23.834479+0000","172.21.15.81:6837/3931731986":"2023-11-09T16:31:16.385652+0000","172.21.15.81:6836/3931731986":"2023-11-09T16:31:16.385652+0000","172.21.15.81:6837/773130363":"2023-11-09T16:31:14.352977+0000","172.21.15.81:6836/773130363":"2023-11-09T16:31:14.352977+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:32:20.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 -- 172.21.15.81:0/3500720684 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac7006ae70 msgr2=0x7fac7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac7006ae70 0x7fac7006d330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fac88006fd0 tx=0x7fac88008040 comp rx=0 tx=0).stop 2023-11-08T16:32:20.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 -- 172.21.15.81:0/3500720684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac8c15aac0 msgr2=0x7fac8c142b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac8c15aac0 0x7fac8c142b00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fac8400c7e0 tx=0x7fac8400cca0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 -- 172.21.15.81:0/3500720684 shutdown_connections 2023-11-08T16:32:20.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac7006ae70 0x7fac7006d330 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac8c15aac0 0x7fac8c142b00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac8c0cf250 0x7fac8c1425c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 --2- 172.21.15.81:0/3500720684 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac8c0716e0 0x7fac8c13fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 -- 172.21.15.81:0/3500720684 >> 172.21.15.81:0/3500720684 conn(0x7fac8c06a4a0 msgr2=0x7fac8c0adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:20.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 -- 172.21.15.81:0/3500720684 shutdown_connections 2023-11-08T16:32:20.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.486+0000 7fac9201a640 1 -- 172.21.15.81:0/3500720684 wait complete. 2023-11-08T16:32:20.504 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:30:27.977481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:30:29.171902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:30:40.141171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:32:20.505 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:32:20.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.718+0000 7fd7463a9640 1 Processor -- start 2023-11-08T16:32:20.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.718+0000 7fd7463a9640 1 -- start start 2023-11-08T16:32:20.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.720+0000 7fd7463a9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd7400ccc10 0x7fd7400ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:20.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.720+0000 7fd7463a9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7400ce960 con 0x7fd74006b710 2023-11-08T16:32:20.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.720+0000 7fd7463a9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7400c9480 con 0x7fd74015a0e0 2023-11-08T16:32:20.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.720+0000 7fd7463a9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd740071980 con 0x7fd7400ccc10 2023-11-08T16:32:20.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.720+0000 7fd737fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd7400ccc10 0x7fd7400ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:20.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.720+0000 7fd744946640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd74006b710 0x7fd74006bb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46738/0 (socket says 172.21.15.81:46738) 2023-11-08T16:32:20.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.720+0000 7fd744946640 1 -- 172.21.15.81:0/565147504 learned_addr learned my addr 172.21.15.81:0/565147504 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:20.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.721+0000 7fd737fff640 1 -- 172.21.15.81:0/565147504 >> v1:172.21.15.93:6789/0 conn(0x7fd74015a0e0 legacy=0x7fd7400c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:20.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.721+0000 7fd737fff640 1 -- 172.21.15.81:0/565147504 >> v1:172.21.15.81:6789/0 conn(0x7fd74006b710 legacy=0x7fd74006bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.721+0000 7fd737fff640 1 -- 172.21.15.81:0/565147504 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74015a4c0 con 0x7fd7400ccc10 2023-11-08T16:32:20.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.721+0000 7fd73f7fe640 1 -- 172.21.15.81:0/565147504 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195700050 0 0) 0x7fd738001490 con 0x7fd74006b710 2023-11-08T16:32:20.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.722+0000 7fd737fff640 1 --2- 172.21.15.81:0/565147504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7400ccc10 0x7fd7400ca300 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd7280097a0 tx=0x7fd72802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838777445732d9d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:20.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.722+0000 7fd73f7fe640 1 -- 172.21.15.81:0/565147504 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728030610 con 0x7fd7400ccc10 2023-11-08T16:32:20.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.722+0000 7fd73f7fe640 1 -- 172.21.15.81:0/565147504 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd728030770 con 0x7fd7400ccc10 2023-11-08T16:32:20.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.722+0000 7fd73f7fe640 1 -- 172.21.15.81:0/565147504 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd728030b00 con 0x7fd7400ccc10 2023-11-08T16:32:20.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.723+0000 7fd7463a9640 1 -- 172.21.15.81:0/565147504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7400ccc10 msgr2=0x7fd7400ca300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.723+0000 7fd7463a9640 1 --2- 172.21.15.81:0/565147504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7400ccc10 0x7fd7400ca300 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd7280097a0 tx=0x7fd72802b960 comp rx=0 tx=0).stop 2023-11-08T16:32:20.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.723+0000 7fd7463a9640 1 -- 172.21.15.81:0/565147504 shutdown_connections 2023-11-08T16:32:20.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.723+0000 7fd7463a9640 1 --2- 172.21.15.81:0/565147504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7400ccc10 0x7fd7400ca300 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.723+0000 7fd7463a9640 1 -- 172.21.15.81:0/565147504 >> 172.21.15.81:0/565147504 conn(0x7fd7400cb580 msgr2=0x7fd7400cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:20.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.723+0000 7fd7463a9640 1 -- 172.21.15.81:0/565147504 shutdown_connections 2023-11-08T16:32:20.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.723+0000 7fd7463a9640 1 -- 172.21.15.81:0/565147504 wait complete. 2023-11-08T16:32:20.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.725+0000 7fd7463a9640 1 Processor -- start 2023-11-08T16:32:20.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.725+0000 7fd7463a9640 1 -- start start 2023-11-08T16:32:20.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd7463a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd74006b710 0x7fd74015c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:20.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd7463a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7400ccc10 0x7fd7401528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:20.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd7463a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd74015a0e0 0x7fd740152de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:20.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd7463a9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400c5c40 con 0x7fd7400ccc10 2023-11-08T16:32:20.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd7463a9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd740150680 con 0x7fd74015a0e0 2023-11-08T16:32:20.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd7463a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd740153480 con 0x7fd74006b710 2023-11-08T16:32:20.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd737fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7400ccc10 0x7fd7401528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:20.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd737fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7400ccc10 0x7fd7401528a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56018/0 (socket says 172.21.15.81:56018) 2023-11-08T16:32:20.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd737fff640 1 -- 172.21.15.81:0/2821808324 learned_addr learned my addr 172.21.15.81:0/2821808324 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:20.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd737fff640 1 -- 172.21.15.81:0/2821808324 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd74006b710 msgr2=0x7fd74015c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd737fff640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd74006b710 0x7fd74015c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd737fff640 1 -- 172.21.15.81:0/2821808324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd74015a0e0 msgr2=0x7fd740152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.726+0000 7fd737fff640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd74015a0e0 0x7fd740152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.727+0000 7fd737fff640 1 -- 172.21.15.81:0/2821808324 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd728009400 con 0x7fd7400ccc10 2023-11-08T16:32:20.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.727+0000 7fd737fff640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7400ccc10 0x7fd7401528a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd7280098c0 tx=0x7fd728004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:20.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.727+0000 7fd73d7fa640 1 -- 172.21.15.81:0/2821808324 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd72803e040 con 0x7fd7400ccc10 2023-11-08T16:32:20.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.727+0000 7fd73d7fa640 1 -- 172.21.15.81:0/2821808324 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd728039c90 con 0x7fd7400ccc10 2023-11-08T16:32:20.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.727+0000 7fd73d7fa640 1 -- 172.21.15.81:0/2821808324 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7280443f0 con 0x7fd7400ccc10 2023-11-08T16:32:20.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.727+0000 7fd7463a9640 1 -- 172.21.15.81:0/2821808324 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd740153760 con 0x7fd7400ccc10 2023-11-08T16:32:20.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.727+0000 7fd7463a9640 1 -- 172.21.15.81:0/2821808324 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd740153d20 con 0x7fd7400ccc10 2023-11-08T16:32:20.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.728+0000 7fd7463a9640 1 -- 172.21.15.81:0/2821808324 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74004ece0 con 0x7fd7400ccc10 2023-11-08T16:32:20.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.729+0000 7fd73d7fa640 1 -- 172.21.15.81:0/2821808324 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd728043390 con 0x7fd7400ccc10 2023-11-08T16:32:20.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.730+0000 7fd73d7fa640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd72006aea0 0x7fd72006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:20.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.730+0000 7fd73d7fa640 1 -- 172.21.15.81:0/2821808324 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7fd7280abd40 con 0x7fd7400ccc10 2023-11-08T16:32:20.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.730+0000 7fd73ffff640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd72006aea0 0x7fd72006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:20.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.731+0000 7fd73ffff640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd72006aea0 0x7fd72006d360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd730006fd0 tx=0x7fd730006bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:20.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.735+0000 7fd73d7fa640 1 -- 172.21.15.81:0/2821808324 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd728076bd0 con 0x7fd7400ccc10 2023-11-08T16:32:20.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.922+0000 7fd7463a9640 1 -- 172.21.15.81:0/2821808324 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd74015d6e0 con 0x7fd7400ccc10 2023-11-08T16:32:20.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.923+0000 7fd73d7fa640 1 -- 172.21.15.81:0/2821808324 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7280340a0 con 0x7fd7400ccc10 2023-11-08T16:32:20.925 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:32:20.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 -- 172.21.15.81:0/2821808324 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd72006aea0 msgr2=0x7fd72006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd72006aea0 0x7fd72006d360 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd730006fd0 tx=0x7fd730006bc0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 -- 172.21.15.81:0/2821808324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7400ccc10 msgr2=0x7fd7401528a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:20.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7400ccc10 0x7fd7401528a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd7280098c0 tx=0x7fd728004730 comp rx=0 tx=0).stop 2023-11-08T16:32:20.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 -- 172.21.15.81:0/2821808324 shutdown_connections 2023-11-08T16:32:20.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd72006aea0 0x7fd72006d360 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd74015a0e0 0x7fd740152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd7400ccc10 0x7fd7401528a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 --2- 172.21.15.81:0/2821808324 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd74006b710 0x7fd74015c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:20.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 -- 172.21.15.81:0/2821808324 >> 172.21.15.81:0/2821808324 conn(0x7fd7400cb580 msgr2=0x7fd740158510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:20.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 -- 172.21.15.81:0/2821808324 shutdown_connections 2023-11-08T16:32:20.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:20.926+0000 7fd7463a9640 1 -- 172.21.15.81:0/2821808324 wait complete. 2023-11-08T16:32:20.945 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:32:21.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.160+0000 7fd860ce1640 1 Processor -- start 2023-11-08T16:32:21.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.160+0000 7fd860ce1640 1 -- start start 2023-11-08T16:32:21.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.161+0000 7fd860ce1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd85c15ad70 0x7fd85c15f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.161+0000 7fd860ce1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd85c06e5a0 0x7fd85c0cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.161+0000 7fd860ce1640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd85c0cf620 con 0x7fd85c0cffb0 2023-11-08T16:32:21.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.162+0000 7fd860ce1640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd85c0ccd90 con 0x7fd85c06e5a0 2023-11-08T16:32:21.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.162+0000 7fd860ce1640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0cd070 con 0x7fd85c15ad70 2023-11-08T16:32:21.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.162+0000 7fd859d9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd85c06e5a0 0x7fd85c0cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:21.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.162+0000 7fd859d9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd85c06e5a0 0x7fd85c0cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33998/0 (socket says 172.21.15.81:33998) 2023-11-08T16:32:21.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.162+0000 7fd859d9b640 1 -- 172.21.15.81:0/2941828590 learned_addr learned my addr 172.21.15.81:0/2941828590 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:21.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.162+0000 7fd85a59c640 1 --2- 172.21.15.81:0/2941828590 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd85c15ad70 0x7fd85c15f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:21.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.163+0000 7fd859d9b640 1 -- 172.21.15.81:0/2941828590 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd85c15ad70 msgr2=0x7fd85c15f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:21.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.163+0000 7fd859d9b640 1 --2- 172.21.15.81:0/2941828590 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd85c15ad70 0x7fd85c15f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.163+0000 7fd859d9b640 1 -- 172.21.15.81:0/2941828590 >> v1:172.21.15.93:6789/0 conn(0x7fd85c0cffb0 legacy=0x7fd85c0ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.163+0000 7fd859d9b640 1 -- 172.21.15.81:0/2941828590 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd85c0cd350 con 0x7fd85c06e5a0 2023-11-08T16:32:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.164+0000 7fd859d9b640 1 --2- 172.21.15.81:0/2941828590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85c06e5a0 0x7fd85c0cfa70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd844009870 tx=0x7fd84402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7def92dd7f6995f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.164+0000 7fd85959a640 1 -- 172.21.15.81:0/2941828590 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844031640 con 0x7fd85c06e5a0 2023-11-08T16:32:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.164+0000 7fd85959a640 1 -- 172.21.15.81:0/2941828590 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8440317a0 con 0x7fd85c06e5a0 2023-11-08T16:32:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.164+0000 7fd85959a640 1 -- 172.21.15.81:0/2941828590 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd844031b30 con 0x7fd85c06e5a0 2023-11-08T16:32:21.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.165+0000 7fd860ce1640 1 -- 172.21.15.81:0/2941828590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85c06e5a0 msgr2=0x7fd85c0cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.165+0000 7fd860ce1640 1 --2- 172.21.15.81:0/2941828590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85c06e5a0 0x7fd85c0cfa70 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd844009870 tx=0x7fd84402b8f0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.165+0000 7fd860ce1640 1 -- 172.21.15.81:0/2941828590 shutdown_connections 2023-11-08T16:32:21.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.165+0000 7fd860ce1640 1 --2- 172.21.15.81:0/2941828590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85c06e5a0 0x7fd85c0cfa70 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.165+0000 7fd860ce1640 1 --2- 172.21.15.81:0/2941828590 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd85c15ad70 0x7fd85c15f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.165+0000 7fd860ce1640 1 -- 172.21.15.81:0/2941828590 >> 172.21.15.81:0/2941828590 conn(0x7fd85c065df0 msgr2=0x7fd85c0661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:21.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.165+0000 7fd860ce1640 1 -- 172.21.15.81:0/2941828590 shutdown_connections 2023-11-08T16:32:21.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.165+0000 7fd860ce1640 1 -- 172.21.15.81:0/2941828590 wait complete. 2023-11-08T16:32:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.167+0000 7fd860ce1640 1 Processor -- start 2023-11-08T16:32:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.167+0000 7fd860ce1640 1 -- start start 2023-11-08T16:32:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd860ce1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85c06e5a0 0x7fd85c148ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd860ce1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd85c0cffb0 0x7fd85c0c9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd860ce1640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd85c15ad70 0x7fd85c15ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd860ce1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0c8b00 con 0x7fd85c06e5a0 2023-11-08T16:32:21.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd860ce1640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0a85b0 con 0x7fd85c15ad70 2023-11-08T16:32:21.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd860ce1640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85c0c7e80 con 0x7fd85c0cffb0 2023-11-08T16:32:21.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd85a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85c06e5a0 0x7fd85c148ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:21.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd85a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85c06e5a0 0x7fd85c148ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34008/0 (socket says 172.21.15.81:34008) 2023-11-08T16:32:21.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd85a59c640 1 -- 172.21.15.81:0/119997901 learned_addr learned my addr 172.21.15.81:0/119997901 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:21.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd85a59c640 1 -- 172.21.15.81:0/119997901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd85c0cffb0 msgr2=0x7fd85c0c9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd85a59c640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd85c0cffb0 0x7fd85c0c9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd85a59c640 1 -- 172.21.15.81:0/119997901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd85c15ad70 msgr2=0x7fd85c15ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd85a59c640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd85c15ad70 0x7fd85c15ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.168+0000 7fd85a59c640 1 -- 172.21.15.81:0/119997901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8440094d0 con 0x7fd85c06e5a0 2023-11-08T16:32:21.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.169+0000 7fd85a59c640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85c06e5a0 0x7fd85c148ff0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00efc0 tx=0x7fd84c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:21.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.169+0000 7fd852ffd640 1 -- 172.21.15.81:0/119997901 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c014480 con 0x7fd85c06e5a0 2023-11-08T16:32:21.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.169+0000 7fd852ffd640 1 -- 172.21.15.81:0/119997901 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c00f040 con 0x7fd85c06e5a0 2023-11-08T16:32:21.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.169+0000 7fd860ce1640 1 -- 172.21.15.81:0/119997901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85c0c8160 con 0x7fd85c06e5a0 2023-11-08T16:32:21.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.169+0000 7fd852ffd640 1 -- 172.21.15.81:0/119997901 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c014800 con 0x7fd85c06e5a0 2023-11-08T16:32:21.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.169+0000 7fd860ce1640 1 -- 172.21.15.81:0/119997901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85c0c8720 con 0x7fd85c06e5a0 2023-11-08T16:32:21.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.170+0000 7fd860ce1640 1 -- 172.21.15.81:0/119997901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85c04ece0 con 0x7fd85c06e5a0 2023-11-08T16:32:21.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.171+0000 7fd852ffd640 1 -- 172.21.15.81:0/119997901 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd84c014960 con 0x7fd85c06e5a0 2023-11-08T16:32:21.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.172+0000 7fd852ffd640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83c06ae70 0x7fd83c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.172+0000 7fd852ffd640 1 -- 172.21.15.81:0/119997901 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7fd84c08a420 con 0x7fd85c06e5a0 2023-11-08T16:32:21.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.172+0000 7fd859d9b640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83c06ae70 0x7fd83c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:21.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.173+0000 7fd859d9b640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83c06ae70 0x7fd83c06d330 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd844036000 tx=0x7fd844030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:21.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.177+0000 7fd852ffd640 1 -- 172.21.15.81:0/119997901 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd84c055220 con 0x7fd85c06e5a0 2023-11-08T16:32:21.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.365+0000 7fd860ce1640 1 -- 172.21.15.81:0/119997901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd85c06de70 con 0x7fd85c06e5a0 2023-11-08T16:32:21.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.366+0000 7fd852ffd640 1 -- 172.21.15.81:0/119997901 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd84c058840 con 0x7fd85c06e5a0 2023-11-08T16:32:21.367 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:32:21.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.368+0000 7fd860ce1640 1 -- 172.21.15.81:0/119997901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83c06ae70 msgr2=0x7fd83c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.368+0000 7fd860ce1640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83c06ae70 0x7fd83c06d330 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd844036000 tx=0x7fd844030000 comp rx=0 tx=0).stop 2023-11-08T16:32:21.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.368+0000 7fd860ce1640 1 -- 172.21.15.81:0/119997901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85c06e5a0 msgr2=0x7fd85c148ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.368+0000 7fd860ce1640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85c06e5a0 0x7fd85c148ff0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fd84c00efc0 tx=0x7fd84c00c490 comp rx=0 tx=0).stop 2023-11-08T16:32:21.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.368+0000 7fd860ce1640 1 -- 172.21.15.81:0/119997901 shutdown_connections 2023-11-08T16:32:21.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.368+0000 7fd860ce1640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83c06ae70 0x7fd83c06d330 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.368+0000 7fd860ce1640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd85c15ad70 0x7fd85c15ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.368+0000 7fd860ce1640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd85c0cffb0 0x7fd85c0c9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.368+0000 7fd860ce1640 1 --2- 172.21.15.81:0/119997901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd85c06e5a0 0x7fd85c148ff0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.368+0000 7fd860ce1640 1 -- 172.21.15.81:0/119997901 >> 172.21.15.81:0/119997901 conn(0x7fd85c065df0 msgr2=0x7fd85c15a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:21.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.369+0000 7fd860ce1640 1 -- 172.21.15.81:0/119997901 shutdown_connections 2023-11-08T16:32:21.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.369+0000 7fd860ce1640 1 -- 172.21.15.81:0/119997901 wait complete. 2023-11-08T16:32:21.388 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:32:21.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.595+0000 7fd10fa1a640 1 Processor -- start 2023-11-08T16:32:21.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.595+0000 7fd10fa1a640 1 -- start start 2023-11-08T16:32:21.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.597+0000 7fd10fa1a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd108162ed0 0x7fd1081652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.597+0000 7fd10fa1a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1080ce960 con 0x7fd10815a0e0 2023-11-08T16:32:21.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.597+0000 7fd10fa1a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1080c9480 con 0x7fd1080ccc10 2023-11-08T16:32:21.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.597+0000 7fd10fa1a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd108071980 con 0x7fd108162ed0 2023-11-08T16:32:21.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.597+0000 7fd10d7b6640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd10815a0e0 0x7fd1080c9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50266/0 (socket says 172.21.15.81:50266) 2023-11-08T16:32:21.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.597+0000 7fd10d7b6640 1 -- 172.21.15.81:0/1412187844 learned_addr learned my addr 172.21.15.81:0/1412187844 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:21.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.597+0000 7fd10dfb7640 1 --2- 172.21.15.81:0/1412187844 >> v2:172.21.15.93:3300/0 conn(0x7fd108162ed0 0x7fd1081652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:21.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.597+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108053844 0 0) 0x7fd104001490 con 0x7fd10815a0e0 2023-11-08T16:32:21.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.597+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0f4003520 con 0x7fd10815a0e0 2023-11-08T16:32:21.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 372791743 0 0) 0x7fd0f4003520 con 0x7fd10815a0e0 2023-11-08T16:32:21.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 >> v2:172.21.15.93:3300/0 conn(0x7fd108162ed0 msgr2=0x7fd1081652c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:21.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 --2- 172.21.15.81:0/1412187844 >> v2:172.21.15.93:3300/0 conn(0x7fd108162ed0 0x7fd1081652c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 >> v1:172.21.15.93:6789/0 conn(0x7fd1080ccc10 legacy=0x7fd1080ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd10815a4c0 con 0x7fd10815a0e0 2023-11-08T16:32:21.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fd104001b80 con 0x7fd10815a0e0 2023-11-08T16:32:21.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 >> v1:172.21.15.81:6789/0 conn(0x7fd10815a0e0 legacy=0x7fd1080c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 --2- 172.21.15.81:0/1412187844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f40039e0 0x7fd0f4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 --2- 172.21.15.81:0/1412187844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0f40047c0 0x7fd0f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 --2- 172.21.15.81:0/1412187844 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0f4005180 0x7fd0f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd108071980 con 0x7fd0f40039e0 2023-11-08T16:32:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f400c090 con 0x7fd0f40047c0 2023-11-08T16:32:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0f400c250 con 0x7fd0f4005180 2023-11-08T16:32:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.598+0000 7fd10d7b6640 1 --2- 172.21.15.81:0/1412187844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f40039e0 0x7fd0f4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:21.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.599+0000 7fd10d7b6640 1 -- 172.21.15.81:0/1412187844 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0f4005180 msgr2=0x7fd0f400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:21.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.599+0000 7fd10d7b6640 1 --2- 172.21.15.81:0/1412187844 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0f4005180 0x7fd0f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.599+0000 7fd10d7b6640 1 -- 172.21.15.81:0/1412187844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0f40047c0 msgr2=0x7fd0f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:21.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.599+0000 7fd10d7b6640 1 --2- 172.21.15.81:0/1412187844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0f40047c0 0x7fd0f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.599+0000 7fd10d7b6640 1 -- 172.21.15.81:0/1412187844 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f4003520 con 0x7fd0f40039e0 2023-11-08T16:32:21.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.599+0000 7fd10dfb7640 1 -- 172.21.15.81:0/1412187844 reap_dead start 2023-11-08T16:32:21.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.600+0000 7fd10d7b6640 1 --2- 172.21.15.81:0/1412187844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f40039e0 0x7fd0f4003e00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd10400a830 tx=0x7fd0f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a666c7fe320f814c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:21.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.600+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104034630 con 0x7fd0f40039e0 2023-11-08T16:32:21.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.600+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10402f070 con 0x7fd0f40039e0 2023-11-08T16:32:21.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.600+0000 7fd0fffff640 1 -- 172.21.15.81:0/1412187844 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd104034a30 con 0x7fd0f40039e0 2023-11-08T16:32:21.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.600+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1412187844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f40039e0 msgr2=0x7fd0f4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.601+0000 7fd10fa1a640 1 --2- 172.21.15.81:0/1412187844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f40039e0 0x7fd0f4003e00 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fd10400a830 tx=0x7fd0f40057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.601+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1412187844 shutdown_connections 2023-11-08T16:32:21.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.601+0000 7fd10fa1a640 1 --2- 172.21.15.81:0/1412187844 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0f40039e0 0x7fd0f4003e00 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.601+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1412187844 >> 172.21.15.81:0/1412187844 conn(0x7fd1080cb580 msgr2=0x7fd1080cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:21.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.601+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1412187844 shutdown_connections 2023-11-08T16:32:21.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.601+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1412187844 wait complete. 2023-11-08T16:32:21.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.602+0000 7fd10fa1a640 1 Processor -- start 2023-11-08T16:32:21.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.603+0000 7fd10fa1a640 1 -- start start 2023-11-08T16:32:21.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10fa1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0f40039e0 0x7fd108150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10fa1a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1081f2930 0x7fd10815a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10fa1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10815abf0 0x7fd108158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10fa1a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd10815b080 con 0x7fd10815abf0 2023-11-08T16:32:21.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10fa1a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd108158f90 con 0x7fd1081f2930 2023-11-08T16:32:21.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10fa1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd108150120 con 0x7fd0f40039e0 2023-11-08T16:32:21.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10dfb7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10815abf0 0x7fd108158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:21.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10dfb7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10815abf0 0x7fd108158160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34032/0 (socket says 172.21.15.81:34032) 2023-11-08T16:32:21.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10dfb7640 1 -- 172.21.15.81:0/1189428436 learned_addr learned my addr 172.21.15.81:0/1189428436 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:21.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10d7b6640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0f40039e0 0x7fd108150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:21.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10dfb7640 1 -- 172.21.15.81:0/1189428436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0f40039e0 msgr2=0x7fd108150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10dfb7640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0f40039e0 0x7fd108150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10dfb7640 1 -- 172.21.15.81:0/1189428436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1081f2930 msgr2=0x7fd10815a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10cfb5640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1081f2930 0x7fd10815a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:21.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10dfb7640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1081f2930 0x7fd10815a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10dfb7640 1 -- 172.21.15.81:0/1189428436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd108150400 con 0x7fd10815abf0 2023-11-08T16:32:21.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.604+0000 7fd10cfb5640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1081f2930 0x7fd10815a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:21.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.605+0000 7fd10dfb7640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10815abf0 0x7fd108158160 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd10815db10 tx=0x7fd10000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:21.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.605+0000 7fd0fdffb640 1 -- 172.21.15.81:0/1189428436 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10000c680 con 0x7fd10815abf0 2023-11-08T16:32:21.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.605+0000 7fd0fdffb640 1 -- 172.21.15.81:0/1189428436 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10000c7e0 con 0x7fd10815abf0 2023-11-08T16:32:21.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.605+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1189428436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1081506e0 con 0x7fd10815abf0 2023-11-08T16:32:21.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.605+0000 7fd0fdffb640 1 -- 172.21.15.81:0/1189428436 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10000cb90 con 0x7fd10815abf0 2023-11-08T16:32:21.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.605+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1189428436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1081586a0 con 0x7fd10815abf0 2023-11-08T16:32:21.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.606+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1189428436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd108070110 con 0x7fd10815abf0 2023-11-08T16:32:21.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.607+0000 7fd0fdffb640 1 -- 172.21.15.81:0/1189428436 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd10000ccf0 con 0x7fd10815abf0 2023-11-08T16:32:21.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.609+0000 7fd0fdffb640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0ec06aea0 0x7fd0ec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:21.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.609+0000 7fd0fdffb640 1 -- 172.21.15.81:0/1189428436 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7fd1000892f0 con 0x7fd10815abf0 2023-11-08T16:32:21.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.609+0000 7fd10d7b6640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0ec06aea0 0x7fd0ec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:21.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.609+0000 7fd10d7b6640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0ec06aea0 0x7fd0ec06d360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd104034000 tx=0x7fd10402f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:21.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.613+0000 7fd0fdffb640 1 -- 172.21.15.81:0/1189428436 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1000540f0 con 0x7fd10815abf0 2023-11-08T16:32:21.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.800+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1189428436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd108071150 con 0x7fd10815abf0 2023-11-08T16:32:21.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.801+0000 7fd0fdffb640 1 -- 172.21.15.81:0/1189428436 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd100057710 con 0x7fd10815abf0 2023-11-08T16:32:21.804 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:32:21.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.803+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1189428436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0ec06aea0 msgr2=0x7fd0ec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.803+0000 7fd10fa1a640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0ec06aea0 0x7fd0ec06d360 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd104034000 tx=0x7fd10402f000 comp rx=0 tx=0).stop 2023-11-08T16:32:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.803+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1189428436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10815abf0 msgr2=0x7fd108158160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.803+0000 7fd10fa1a640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10815abf0 0x7fd108158160 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd10815db10 tx=0x7fd10000ec00 comp rx=0 tx=0).stop 2023-11-08T16:32:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.804+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1189428436 shutdown_connections 2023-11-08T16:32:21.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.804+0000 7fd10fa1a640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0ec06aea0 0x7fd0ec06d360 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.804+0000 7fd10fa1a640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd10815abf0 0x7fd108158160 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.804+0000 7fd10fa1a640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1081f2930 0x7fd10815a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.804+0000 7fd10fa1a640 1 --2- 172.21.15.81:0/1189428436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0f40039e0 0x7fd108150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:21.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.804+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1189428436 >> 172.21.15.81:0/1189428436 conn(0x7fd1080cb580 msgr2=0x7fd108161f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:21.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.804+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1189428436 shutdown_connections 2023-11-08T16:32:21.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:21.804+0000 7fd10fa1a640 1 -- 172.21.15.81:0/1189428436 wait complete. 2023-11-08T16:32:21.823 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:32:22.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.029+0000 7f4453e9b640 1 Processor -- start 2023-11-08T16:32:22.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.029+0000 7f4453e9b640 1 -- start start 2023-11-08T16:32:22.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.030+0000 7f4453e9b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f444c0cd9a0 0x7f444c15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.030+0000 7f4453e9b640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f444c06d330 0x7f444c06be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.030+0000 7f4453e9b640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f444c15ae00 con 0x7f444c159920 2023-11-08T16:32:22.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.030+0000 7f4453e9b640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f444c0ce5e0 con 0x7f444c06d330 2023-11-08T16:32:22.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.030+0000 7f4453e9b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c06fc70 con 0x7f444c0cd9a0 2023-11-08T16:32:22.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.031+0000 7f4452438640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f444c06d330 0x7f444c06be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.031+0000 7f4451436640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f444c0cd9a0 0x7f444c15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.031+0000 7f4451436640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f444c0cd9a0 0x7f444c15a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:49170/0 (socket says 172.21.15.81:49170) 2023-11-08T16:32:22.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.031+0000 7f4451436640 1 -- 172.21.15.81:0/3111388629 learned_addr learned my addr 172.21.15.81:0/3111388629 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:22.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.032+0000 7f4452438640 1 -- 172.21.15.81:0/3111388629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f444c0cd9a0 msgr2=0x7f444c15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:22.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.032+0000 7f4452438640 1 --2- 172.21.15.81:0/3111388629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f444c0cd9a0 0x7f444c15a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.032+0000 7f4452438640 1 -- 172.21.15.81:0/3111388629 >> v1:172.21.15.93:6789/0 conn(0x7f444c159920 legacy=0x7f444c0cd290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.032+0000 7f4452438640 1 -- 172.21.15.81:0/3111388629 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f444c159d00 con 0x7f444c06d330 2023-11-08T16:32:22.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.033+0000 7f4452438640 1 --2- 172.21.15.81:0/3111388629 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f444c06d330 0x7f444c06be80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4444009d90 tx=0x7f444402bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e221ffe4c3d2db0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:22.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.034+0000 7f4450c35640 1 -- 172.21.15.81:0/3111388629 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f444402cc50 con 0x7f444c06d330 2023-11-08T16:32:22.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.034+0000 7f4450c35640 1 -- 172.21.15.81:0/3111388629 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f444402cdb0 con 0x7f444c06d330 2023-11-08T16:32:22.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.034+0000 7f4450c35640 1 -- 172.21.15.81:0/3111388629 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4444034660 con 0x7f444c06d330 2023-11-08T16:32:22.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.035+0000 7f4453e9b640 1 -- 172.21.15.81:0/3111388629 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f444c06d330 msgr2=0x7f444c06be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.035+0000 7f4453e9b640 1 --2- 172.21.15.81:0/3111388629 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f444c06d330 0x7f444c06be80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4444009d90 tx=0x7f444402bea0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.035+0000 7f4453e9b640 1 -- 172.21.15.81:0/3111388629 shutdown_connections 2023-11-08T16:32:22.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.035+0000 7f4453e9b640 1 --2- 172.21.15.81:0/3111388629 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f444c06d330 0x7f444c06be80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.035+0000 7f4453e9b640 1 --2- 172.21.15.81:0/3111388629 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f444c0cd9a0 0x7f444c15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.035+0000 7f4453e9b640 1 -- 172.21.15.81:0/3111388629 >> 172.21.15.81:0/3111388629 conn(0x7f444c0cb580 msgr2=0x7f444c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:22.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.035+0000 7f4453e9b640 1 -- 172.21.15.81:0/3111388629 shutdown_connections 2023-11-08T16:32:22.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.035+0000 7f4453e9b640 1 -- 172.21.15.81:0/3111388629 wait complete. 2023-11-08T16:32:22.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.037+0000 7f4453e9b640 1 Processor -- start 2023-11-08T16:32:22.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.037+0000 7f4453e9b640 1 -- start start 2023-11-08T16:32:22.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4453e9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f444c06d330 0x7f444c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4453e9b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f444c0cd9a0 0x7f444c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4453e9b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f444c159920 0x7f444c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4453e9b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c1f4080 con 0x7f444c06d330 2023-11-08T16:32:22.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4453e9b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f444c157860 con 0x7f444c159920 2023-11-08T16:32:22.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4453e9b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f444c1f3da0 con 0x7f444c0cd9a0 2023-11-08T16:32:22.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4451c37640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f444c06d330 0x7f444c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4451c37640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f444c06d330 0x7f444c157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34048/0 (socket says 172.21.15.81:34048) 2023-11-08T16:32:22.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4451c37640 1 -- 172.21.15.81:0/1003060448 learned_addr learned my addr 172.21.15.81:0/1003060448 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:22.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4451436640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f444c0cd9a0 0x7f444c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4452438640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f444c159920 0x7f444c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4451c37640 1 -- 172.21.15.81:0/1003060448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f444c0cd9a0 msgr2=0x7f444c151fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4451c37640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f444c0cd9a0 0x7f444c151fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.038+0000 7f4451c37640 1 -- 172.21.15.81:0/1003060448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f444c159920 msgr2=0x7f444c153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.039+0000 7f4451c37640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f444c159920 0x7f444c153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.039+0000 7f4451c37640 1 -- 172.21.15.81:0/1003060448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4444009a70 con 0x7f444c06d330 2023-11-08T16:32:22.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.039+0000 7f4452438640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f444c159920 0x7f444c153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:22.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.039+0000 7f4451c37640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f444c06d330 0x7f444c157cf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f444800c400 tx=0x7f444800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:22.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.039+0000 7f44427fc640 1 -- 172.21.15.81:0/1003060448 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44480052d0 con 0x7f444c06d330 2023-11-08T16:32:22.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.039+0000 7f44427fc640 1 -- 172.21.15.81:0/1003060448 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4448005430 con 0x7f444c06d330 2023-11-08T16:32:22.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.039+0000 7f4453e9b640 1 -- 172.21.15.81:0/1003060448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f444c157210 con 0x7f444c06d330 2023-11-08T16:32:22.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.039+0000 7f44427fc640 1 -- 172.21.15.81:0/1003060448 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4448003830 con 0x7f444c06d330 2023-11-08T16:32:22.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.039+0000 7f4453e9b640 1 -- 172.21.15.81:0/1003060448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f444c152510 con 0x7f444c06d330 2023-11-08T16:32:22.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.040+0000 7f4453e9b640 1 -- 172.21.15.81:0/1003060448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f444c0c99d0 con 0x7f444c06d330 2023-11-08T16:32:22.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.041+0000 7f44427fc640 1 -- 172.21.15.81:0/1003060448 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4448003990 con 0x7f444c06d330 2023-11-08T16:32:22.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.042+0000 7f44427fc640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f443006ae70 0x7f443006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.042+0000 7f44427fc640 1 -- 172.21.15.81:0/1003060448 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f4448088cb0 con 0x7f444c06d330 2023-11-08T16:32:22.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.042+0000 7f4451436640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f443006ae70 0x7f443006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.043+0000 7f4451436640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f443006ae70 0x7f443006d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f444c154850 tx=0x7f443c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:22.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.047+0000 7f44427fc640 1 -- 172.21.15.81:0/1003060448 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4448053ac0 con 0x7f444c06d330 2023-11-08T16:32:22.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.237+0000 7f4453e9b640 1 -- 172.21.15.81:0/1003060448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f444c157070 con 0x7f444c06d330 2023-11-08T16:32:22.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.237+0000 7f44427fc640 1 -- 172.21.15.81:0/1003060448 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v44) v1 ==== 106+0+11 (secure 0 0 0) 0x7f444c157070 con 0x7f444c06d330 2023-11-08T16:32:22.239 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:32:22.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 -- 172.21.15.81:0/1003060448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f443006ae70 msgr2=0x7f443006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f443006ae70 0x7f443006d330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f444c154850 tx=0x7f443c008040 comp rx=0 tx=0).stop 2023-11-08T16:32:22.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 -- 172.21.15.81:0/1003060448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f444c06d330 msgr2=0x7f444c157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f444c06d330 0x7f444c157cf0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f444800c400 tx=0x7f444800c8c0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 -- 172.21.15.81:0/1003060448 shutdown_connections 2023-11-08T16:32:22.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f443006ae70 0x7f443006d330 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f444c159920 0x7f444c153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f444c0cd9a0 0x7f444c151fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 --2- 172.21.15.81:0/1003060448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f444c06d330 0x7f444c157cf0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 -- 172.21.15.81:0/1003060448 >> 172.21.15.81:0/1003060448 conn(0x7f444c0cb580 msgr2=0x7f444c06a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:22.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 -- 172.21.15.81:0/1003060448 shutdown_connections 2023-11-08T16:32:22.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.240+0000 7f4453e9b640 1 -- 172.21.15.81:0/1003060448 wait complete. 2023-11-08T16:32:22.260 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:32:22.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.468+0000 7efff4210640 1 Processor -- start 2023-11-08T16:32:22.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.468+0000 7efff4210640 1 -- start start 2023-11-08T16:32:22.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.470+0000 7efff4210640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7effec15d3b0 0x7effec06d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.470+0000 7efff4210640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effec06cc50 con 0x7effec0cf720 2023-11-08T16:32:22.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.470+0000 7efff4210640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effec0c90f0 con 0x7effec0ca130 2023-11-08T16:32:22.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.470+0000 7efff4210640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effec0adb70 con 0x7effec15d3b0 2023-11-08T16:32:22.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.470+0000 7efff17ab640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7effec0cf720 0x7effec0c9a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50282/0 (socket says 172.21.15.81:50282) 2023-11-08T16:32:22.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.470+0000 7efff17ab640 1 -- 172.21.15.81:0/2605384597 learned_addr learned my addr 172.21.15.81:0/2605384597 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.470+0000 7efff1fac640 1 --2- 172.21.15.81:0/2605384597 >> v2:172.21.15.93:3300/0 conn(0x7effec15d3b0 0x7effec06d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990336230 0 0) 0x7effec06cc50 con 0x7effec0cf720 2023-11-08T16:32:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effd8005730 con 0x7effec0cf720 2023-11-08T16:32:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 597284993 0 0) 0x7effd8005730 con 0x7effec0cf720 2023-11-08T16:32:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 >> v2:172.21.15.93:3300/0 conn(0x7effec15d3b0 msgr2=0x7effec06d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:22.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 --2- 172.21.15.81:0/2605384597 >> v2:172.21.15.93:3300/0 conn(0x7effec15d3b0 0x7effec06d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 >> v1:172.21.15.93:6789/0 conn(0x7effec0ca130 legacy=0x7effec070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec0c9360 con 0x7effec0cf720 2023-11-08T16:32:22.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7effdc001670 con 0x7effec0cf720 2023-11-08T16:32:22.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 >> v1:172.21.15.81:6789/0 conn(0x7effec0cf720 legacy=0x7effec0c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 --2- 172.21.15.81:0/2605384597 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7effd8005b80 0x7effd8005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 --2- 172.21.15.81:0/2605384597 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7effd80069d0 0x7effd8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 --2- 172.21.15.81:0/2605384597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effd8007390 0x7effd800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec0adb70 con 0x7effd8007390 2023-11-08T16:32:22.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd800e230 con 0x7effd8005b80 2023-11-08T16:32:22.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd800e3f0 con 0x7effd80069d0 2023-11-08T16:32:22.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.471+0000 7efff27ad640 1 --2- 172.21.15.81:0/2605384597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effd8007390 0x7effd800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.472+0000 7efff1fac640 1 --2- 172.21.15.81:0/2605384597 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7effd8005b80 0x7effd8005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.472+0000 7efff17ab640 1 --2- 172.21.15.81:0/2605384597 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7effd80069d0 0x7effd8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.472+0000 7efff27ad640 1 -- 172.21.15.81:0/2605384597 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7effd80069d0 msgr2=0x7effd8006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:22.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.472+0000 7efff27ad640 1 --2- 172.21.15.81:0/2605384597 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7effd80069d0 0x7effd8006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.472+0000 7efff27ad640 1 -- 172.21.15.81:0/2605384597 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7effd8005b80 msgr2=0x7effd8005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:22.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.472+0000 7efff27ad640 1 --2- 172.21.15.81:0/2605384597 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7effd8005b80 0x7effd8005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.472+0000 7efff27ad640 1 -- 172.21.15.81:0/2605384597 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd8005730 con 0x7effd8007390 2023-11-08T16:32:22.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.473+0000 7efff27ad640 1 -- 172.21.15.81:0/2605384597 reap_dead start 2023-11-08T16:32:22.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.473+0000 7efff27ad640 1 --2- 172.21.15.81:0/2605384597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effd8007390 0x7effd800db90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7effe4001e50 tx=0x7effd8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bad27379b2062d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:22.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.473+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4003b70 con 0x7effd8007390 2023-11-08T16:32:22.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.473+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe4003d10 con 0x7effd8007390 2023-11-08T16:32:22.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.473+0000 7efff0faa640 1 -- 172.21.15.81:0/2605384597 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4005310 con 0x7effd8007390 2023-11-08T16:32:22.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.474+0000 7efff4210640 1 -- 172.21.15.81:0/2605384597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effd8007390 msgr2=0x7effd800db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.474+0000 7efff4210640 1 --2- 172.21.15.81:0/2605384597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effd8007390 0x7effd800db90 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7effe4001e50 tx=0x7effd8007a00 comp rx=0 tx=0).stop 2023-11-08T16:32:22.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.474+0000 7efff4210640 1 -- 172.21.15.81:0/2605384597 shutdown_connections 2023-11-08T16:32:22.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.474+0000 7efff4210640 1 --2- 172.21.15.81:0/2605384597 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effd8007390 0x7effd800db90 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.474+0000 7efff4210640 1 -- 172.21.15.81:0/2605384597 >> 172.21.15.81:0/2605384597 conn(0x7effec0ceaa0 msgr2=0x7effec0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:22.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.474+0000 7efff4210640 1 -- 172.21.15.81:0/2605384597 shutdown_connections 2023-11-08T16:32:22.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.474+0000 7efff4210640 1 -- 172.21.15.81:0/2605384597 wait complete. 2023-11-08T16:32:22.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.476+0000 7efff4210640 1 Processor -- start 2023-11-08T16:32:22.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.476+0000 7efff4210640 1 -- start start 2023-11-08T16:32:22.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff4210640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7effd8007390 0x7effec0b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff4210640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7effec148080 0x7effec149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff4210640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effec148500 0x7effec14a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff4210640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec144970 con 0x7effec148500 2023-11-08T16:32:22.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff4210640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effec0b40a0 con 0x7effd8007390 2023-11-08T16:32:22.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff4210640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effec148ab0 con 0x7effec148080 2023-11-08T16:32:22.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff27ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effec148500 0x7effec14a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff27ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effec148500 0x7effec14a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34060/0 (socket says 172.21.15.81:34060) 2023-11-08T16:32:22.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff27ad640 1 -- 172.21.15.81:0/1650291030 learned_addr learned my addr 172.21.15.81:0/1650291030 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:22.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff27ad640 1 -- 172.21.15.81:0/1650291030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7effec148080 msgr2=0x7effec149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff17ab640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7effec148080 0x7effec149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff1fac640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7effd8007390 0x7effec0b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff27ad640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7effec148080 0x7effec149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff27ad640 1 -- 172.21.15.81:0/1650291030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7effd8007390 msgr2=0x7effec0b3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff27ad640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7effd8007390 0x7effec0b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.477+0000 7efff27ad640 1 -- 172.21.15.81:0/1650291030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effec148d90 con 0x7effec148500 2023-11-08T16:32:22.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.478+0000 7efff27ad640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effec148500 0x7effec14a180 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7effe4001f70 tx=0x7effe40014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:22.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.478+0000 7effe27fc640 1 -- 172.21.15.81:0/1650291030 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4006d50 con 0x7effec148500 2023-11-08T16:32:22.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.478+0000 7effe27fc640 1 -- 172.21.15.81:0/1650291030 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe4007030 con 0x7effec148500 2023-11-08T16:32:22.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.478+0000 7effe27fc640 1 -- 172.21.15.81:0/1650291030 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe4007420 con 0x7effec148500 2023-11-08T16:32:22.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.478+0000 7efff4210640 1 -- 172.21.15.81:0/1650291030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effec149070 con 0x7effec148500 2023-11-08T16:32:22.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.478+0000 7efff4210640 1 -- 172.21.15.81:0/1650291030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effec0c5c10 con 0x7effec148500 2023-11-08T16:32:22.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.479+0000 7efff4210640 1 -- 172.21.15.81:0/1650291030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effec0c5e30 con 0x7effec148500 2023-11-08T16:32:22.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.480+0000 7effe27fc640 1 -- 172.21.15.81:0/1650291030 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7effe4007580 con 0x7effec148500 2023-11-08T16:32:22.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.481+0000 7effe27fc640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7effd006aea0 0x7effd006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:22.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.481+0000 7effe27fc640 1 -- 172.21.15.81:0/1650291030 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7effe40ab630 con 0x7effec148500 2023-11-08T16:32:22.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.481+0000 7efff1fac640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7effd006aea0 0x7effd006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:22.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.482+0000 7efff1fac640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7effd006aea0 0x7effd006d360 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7effe80098c0 tx=0x7effe8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:22.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.485+0000 7effe27fc640 1 -- 172.21.15.81:0/1650291030 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7effe4076430 con 0x7effec148500 2023-11-08T16:32:22.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.686+0000 7efff4210640 1 -- 172.21.15.81:0/1650291030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7effec0743f0 con 0x7effec148500 2023-11-08T16:32:22.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.687+0000 7effe27fc640 1 -- 172.21.15.81:0/1650291030 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v44) v1 ==== 80+0+364 (secure 0 0 0) 0x7effe4079a50 con 0x7effec148500 2023-11-08T16:32:22.689 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/1759199271 2023-11-09T16:31:23.834479+0000 2023-11-08T16:32:22.689 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6836/1759199271 2023-11-09T16:31:23.834479+0000 2023-11-08T16:32:22.690 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/3931731986 2023-11-09T16:31:16.385652+0000 2023-11-08T16:32:22.690 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6836/3931731986 2023-11-09T16:31:16.385652+0000 2023-11-08T16:32:22.690 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/773130363 2023-11-09T16:31:14.352977+0000 2023-11-08T16:32:22.690 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6836/773130363 2023-11-09T16:31:14.352977+0000 2023-11-08T16:32:22.691 INFO:teuthology.orchestra.run.smithi081.stderr:listed 6 entries 2023-11-08T16:32:22.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 -- 172.21.15.81:0/1650291030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7effd006aea0 msgr2=0x7effd006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7effd006aea0 0x7effd006d360 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7effe80098c0 tx=0x7effe8008040 comp rx=0 tx=0).stop 2023-11-08T16:32:22.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 -- 172.21.15.81:0/1650291030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effec148500 msgr2=0x7effec14a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:22.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effec148500 0x7effec14a180 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7effe4001f70 tx=0x7effe40014a0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 -- 172.21.15.81:0/1650291030 shutdown_connections 2023-11-08T16:32:22.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7effd006aea0 0x7effd006d360 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7effec148500 0x7effec14a180 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7effec148080 0x7effec149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 --2- 172.21.15.81:0/1650291030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7effd8007390 0x7effec0b3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:22.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 -- 172.21.15.81:0/1650291030 >> 172.21.15.81:0/1650291030 conn(0x7effec0ceaa0 msgr2=0x7effec0712e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:22.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 -- 172.21.15.81:0/1650291030 shutdown_connections 2023-11-08T16:32:22.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:22.690+0000 7efff4210640 1 -- 172.21.15.81:0/1650291030 wait complete. 2023-11-08T16:32:22.712 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:32:22.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:22.713 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:32:22.803 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[103691]: fuse finished with error 0 and tester_r 0 2023-11-08T16:32:22.815 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:32:22.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:22.816 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:32:22.985 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:32:22.985 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:32:23.046 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:32:23.046 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:32:23.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:23.047 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-08T16:32:23.105 INFO:teuthology.orchestra.run.smithi093.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-08T16:32:23.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:32:23.107 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:32:23.107 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:32:23.168 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:32:23.169 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:32:23.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:23.170 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-08T16:32:23.223 INFO:teuthology.orchestra.run.smithi081.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-08T16:32:23.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:32:23.224 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:32:23.225 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:32:23.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:32:23.286 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:32:23.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:23.287 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-08T16:32:23.341 INFO:teuthology.orchestra.run.smithi093.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-08T16:32:23.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:32:23.343 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:32:23.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.561+0000 7f37f944b640 1 Processor -- start 2023-11-08T16:32:23.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.561+0000 7f37f944b640 1 -- start start 2023-11-08T16:32:23.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.562+0000 7f37f944b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f40ade50 0x7f37f40ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:23.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.562+0000 7f37f944b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f406b630 con 0x7f37f415a680 2023-11-08T16:32:23.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.562+0000 7f37f944b640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37f40cabb0 con 0x7f37f415ea60 2023-11-08T16:32:23.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.562+0000 7f37f944b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f40b1860 con 0x7f37f40ade50 2023-11-08T16:32:23.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.563+0000 7f37ea7fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f37f415a680 0x7f37f415aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50286/0 (socket says 172.21.15.81:50286) 2023-11-08T16:32:23.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.563+0000 7f37ea7fc640 1 -- 172.21.15.81:0/3003281515 learned_addr learned my addr 172.21.15.81:0/3003281515 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:23.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.563+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320619390 0 0) 0x7f37f406b630 con 0x7f37f415a680 2023-11-08T16:32:23.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.563+0000 7f37f2ffd640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f40ade50 0x7f37f40ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:23.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.563+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37e0003520 con 0x7f37f415a680 2023-11-08T16:32:23.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.563+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2407181148 0 0) 0x7f37e0003520 con 0x7f37f415a680 2023-11-08T16:32:23.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.563+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f40ade50 msgr2=0x7f37f40ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:23.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.563+0000 7f37f27fc640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f40ade50 0x7f37f40ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:23.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.563+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 >> v1:172.21.15.93:6789/0 conn(0x7f37f415ea60 legacy=0x7f37f415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:23.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.563+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f40cae90 con 0x7f37f415a680 2023-11-08T16:32:23.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f37dc001670 con 0x7f37f415a680 2023-11-08T16:32:23.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 >> v1:172.21.15.81:6789/0 conn(0x7f37f415a680 legacy=0x7f37f415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:23.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f27fc640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37e0003a80 0x7f37e0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:23.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f27fc640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f40ade50 0x7f37e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:23.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f27fc640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37e0004dd0 0x7f37e0005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:23.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e000bbb0 con 0x7f37e0004dd0 2023-11-08T16:32:23.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e000bd20 con 0x7f37e0003a80 2023-11-08T16:32:23.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e000bee0 con 0x7f37f40ade50 2023-11-08T16:32:23.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f37fe640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37e0004dd0 0x7f37e0005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:23.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f2ffd640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37e0003a80 0x7f37e0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:23.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37ea7fc640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f40ade50 0x7f37e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:23.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f37fe640 1 -- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f40ade50 msgr2=0x7f37e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:23.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f37fe640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f40ade50 0x7f37e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:23.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f37fe640 1 -- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37e0003a80 msgr2=0x7f37e0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:23.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f37fe640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37e0003a80 0x7f37e0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:23.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f37fe640 1 -- 172.21.15.81:0/3003281515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e0003520 con 0x7f37e0004dd0 2023-11-08T16:32:23.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37f2ffd640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37e0003a80 0x7f37e0003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:23.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.564+0000 7f37ea7fc640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f40ade50 0x7f37e00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:23.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.565+0000 7f37f37fe640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37e0004dd0 0x7f37e0005250 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f37ec001e50 tx=0x7f37e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b29fdf6f71fa59 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:23.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.566+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec003b70 con 0x7f37e0004dd0 2023-11-08T16:32:23.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.566+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec003cd0 con 0x7f37e0004dd0 2023-11-08T16:32:23.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.566+0000 7f37f27fc640 1 -- 172.21.15.81:0/3003281515 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec005310 con 0x7f37e0004dd0 2023-11-08T16:32:23.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.566+0000 7f37f944b640 1 -- 172.21.15.81:0/3003281515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37e0004dd0 msgr2=0x7f37e0005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:23.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.566+0000 7f37f944b640 1 --2- 172.21.15.81:0/3003281515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37e0004dd0 0x7f37e0005250 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f37ec001e50 tx=0x7f37e00057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:23.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.566+0000 7f37f37fe640 1 -- 172.21.15.81:0/3003281515 reap_dead start 2023-11-08T16:32:23.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.566+0000 7f37f944b640 1 -- 172.21.15.81:0/3003281515 shutdown_connections 2023-11-08T16:32:23.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.566+0000 7f37f944b640 1 -- 172.21.15.81:0/3003281515 >> 172.21.15.81:0/3003281515 conn(0x7f37f40cb490 msgr2=0x7f37f40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:23.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.566+0000 7f37f944b640 1 -- 172.21.15.81:0/3003281515 shutdown_connections 2023-11-08T16:32:23.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.566+0000 7f37f944b640 1 -- 172.21.15.81:0/3003281515 wait complete. 2023-11-08T16:32:23.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.568+0000 7f37f944b640 1 Processor -- start 2023-11-08T16:32:23.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.568+0000 7f37f944b640 1 -- start start 2023-11-08T16:32:23.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.569+0000 7f37f944b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37e0004dd0 0x7f37f414e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:23.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.569+0000 7f37f944b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f4152820 0x7f37f414f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:23.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.569+0000 7f37f944b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37f415d1b0 0x7f37f415d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:23.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.569+0000 7f37f944b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f415db70 con 0x7f37f415d1b0 2023-11-08T16:32:23.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.569+0000 7f37f944b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f4150250 con 0x7f37e0004dd0 2023-11-08T16:32:23.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.569+0000 7f37f944b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f414e800 con 0x7f37f4152820 2023-11-08T16:32:23.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.570+0000 7f37f37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37f415d1b0 0x7f37f415d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:23.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.570+0000 7f37f37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37f415d1b0 0x7f37f415d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34082/0 (socket says 172.21.15.81:34082) 2023-11-08T16:32:23.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.570+0000 7f37f37fe640 1 -- 172.21.15.81:0/966115718 learned_addr learned my addr 172.21.15.81:0/966115718 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:23.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.570+0000 7f37f2ffd640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37e0004dd0 0x7f37f414e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:23.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.570+0000 7f37ea7fc640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f4152820 0x7f37f414f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:23.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.570+0000 7f37f37fe640 1 -- 172.21.15.81:0/966115718 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f4152820 msgr2=0x7f37f414f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:23.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.570+0000 7f37f37fe640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f4152820 0x7f37f414f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:23.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.570+0000 7f37f37fe640 1 -- 172.21.15.81:0/966115718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37e0004dd0 msgr2=0x7f37f414e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:23.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.570+0000 7f37f37fe640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37e0004dd0 0x7f37f414e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:23.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.570+0000 7f37f37fe640 1 -- 172.21.15.81:0/966115718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f414f9b0 con 0x7f37f415d1b0 2023-11-08T16:32:23.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.570+0000 7f37f37fe640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37f415d1b0 0x7f37f415d630 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f37ec02c060 tx=0x7f37ec0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:23.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.571+0000 7f37ebfff640 1 -- 172.21.15.81:0/966115718 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec006d50 con 0x7f37f415d1b0 2023-11-08T16:32:23.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.571+0000 7f37ebfff640 1 -- 172.21.15.81:0/966115718 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37ec003e40 con 0x7f37f415d1b0 2023-11-08T16:32:23.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.571+0000 7f37f944b640 1 -- 172.21.15.81:0/966115718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37f414fc90 con 0x7f37f415d1b0 2023-11-08T16:32:23.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.571+0000 7f37ebfff640 1 -- 172.21.15.81:0/966115718 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37ec007230 con 0x7f37f415d1b0 2023-11-08T16:32:23.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.571+0000 7f37f944b640 1 -- 172.21.15.81:0/966115718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37f415ac10 con 0x7f37f415d1b0 2023-11-08T16:32:23.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.572+0000 7f37f944b640 1 -- 172.21.15.81:0/966115718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37f404ece0 con 0x7f37f415d1b0 2023-11-08T16:32:23.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.573+0000 7f37ebfff640 1 -- 172.21.15.81:0/966115718 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37ec007390 con 0x7f37f415d1b0 2023-11-08T16:32:23.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.574+0000 7f37ebfff640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f37d406ae70 0x7f37d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:23.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.574+0000 7f37ebfff640 1 -- 172.21.15.81:0/966115718 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f37ec0aaf60 con 0x7f37f415d1b0 2023-11-08T16:32:23.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.574+0000 7f37f2ffd640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f37d406ae70 0x7f37d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:23.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.575+0000 7f37f2ffd640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f37d406ae70 0x7f37d406d330 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f37f41f44e0 tx=0x7f37e4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:23.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.578+0000 7f37ebfff640 1 -- 172.21.15.81:0/966115718 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37ec075d60 con 0x7f37f415d1b0 2023-11-08T16:32:23.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.806+0000 7f37f944b640 1 -- 172.21.15.81:0/966115718 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37f40cc1d0 con 0x7f37f415d1b0 2023-11-08T16:32:23.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.808+0000 7f37ebfff640 1 -- 172.21.15.81:0/966115718 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f37ec048070 con 0x7f37f415d1b0 2023-11-08T16:32:23.810 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:23.810 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3654712142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3654712142},{"type":"v1","addr":"172.21.15.93:6834","nonce":3654712142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5014,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/66363251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":66363251},{"type":"v1","addr":"172.21.15.93:6835","nonce":66363251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4031527046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4031527046},{"type":"v1","addr":"172.21.15.81:6835","nonce":4031527046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:25.862327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/2942618483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2942618483},{"type":"v1","addr":"172.21.15.81:6837","nonce":2942618483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:32:23.810 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 32 2023-11-08T16:32:23.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 -- 172.21.15.81:0/966115718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f37d406ae70 msgr2=0x7f37d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:23.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f37d406ae70 0x7f37d406d330 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f37f41f44e0 tx=0x7f37e4008040 comp rx=0 tx=0).stop 2023-11-08T16:32:23.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 -- 172.21.15.81:0/966115718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37f415d1b0 msgr2=0x7f37f415d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:23.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37f415d1b0 0x7f37f415d630 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f37ec02c060 tx=0x7f37ec0014a0 comp rx=0 tx=0).stop 2023-11-08T16:32:23.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 -- 172.21.15.81:0/966115718 shutdown_connections 2023-11-08T16:32:23.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f37d406ae70 0x7f37d406d330 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:23.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37f415d1b0 0x7f37f415d630 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:23.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37f4152820 0x7f37f414f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:23.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 --2- 172.21.15.81:0/966115718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37e0004dd0 0x7f37f414e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:23.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 -- 172.21.15.81:0/966115718 >> 172.21.15.81:0/966115718 conn(0x7f37f40cb490 msgr2=0x7f37f41593a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:23.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 -- 172.21.15.81:0/966115718 shutdown_connections 2023-11-08T16:32:23.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:23.810+0000 7f37f944b640 1 -- 172.21.15.81:0/966115718 wait complete. 2023-11-08T16:32:23.832 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:32:24.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.042+0000 7f7836a26640 1 Processor -- start 2023-11-08T16:32:24.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.042+0000 7f7836a26640 1 -- start start 2023-11-08T16:32:24.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.044+0000 7f7836a26640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f783015d2b0 0x7f7830161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.044+0000 7f7836a26640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78300c9060 0x7f78300cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.044+0000 7f7836a26640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7830161c50 con 0x7f78300c94e0 2023-11-08T16:32:24.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.044+0000 7f7836a26640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f783015d690 con 0x7f783015d2b0 2023-11-08T16:32:24.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.044+0000 7f7836a26640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783006b980 con 0x7f78300c9060 2023-11-08T16:32:24.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.044+0000 7f7834fc3640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f78300c94e0 0x7f783006ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50292/0 (socket says 172.21.15.81:50292) 2023-11-08T16:32:24.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.044+0000 7f7834fc3640 1 -- 172.21.15.81:0/2764704952 learned_addr learned my addr 172.21.15.81:0/2764704952 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:24.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782ffff640 1 --2- 172.21.15.81:0/2764704952 >> v2:172.21.15.93:3300/0 conn(0x7f783015d2b0 0x7f7830161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782f7fe640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78300c9060 0x7f78300cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910773899 0 0) 0x7f7828001490 con 0x7f78300c94e0 2023-11-08T16:32:24.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7818003520 con 0x7f78300c94e0 2023-11-08T16:32:24.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 444768881 0 0) 0x7f7818003520 con 0x7f78300c94e0 2023-11-08T16:32:24.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78300c9060 msgr2=0x7f78300cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:24.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78300c9060 0x7f78300cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 >> v2:172.21.15.93:3300/0 conn(0x7f783015d2b0 msgr2=0x7f7830161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 --2- 172.21.15.81:0/2764704952 >> v2:172.21.15.93:3300/0 conn(0x7f783015d2b0 0x7f7830161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783006bc60 con 0x7f78300c94e0 2023-11-08T16:32:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f7828001b80 con 0x7f78300c94e0 2023-11-08T16:32:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 >> v1:172.21.15.81:6789/0 conn(0x7f78300c94e0 legacy=0x7f783006ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7818003a50 0x7f7818003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.045+0000 7f782effd640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78180047c0 0x7f7818004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f782effd640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78300c9060 0x7f7818005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783015d690 con 0x7f7818003a50 2023-11-08T16:32:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783006b980 con 0x7f78180047c0 2023-11-08T16:32:24.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f781800b9b0 con 0x7f78300c9060 2023-11-08T16:32:24.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f782ffff640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7818003a50 0x7f7818003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f7834fc3640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78300c9060 0x7f7818005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f782f7fe640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78180047c0 0x7f7818004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f782ffff640 1 -- 172.21.15.81:0/2764704952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78300c9060 msgr2=0x7f7818005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:24.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f782ffff640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78300c9060 0x7f7818005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f782ffff640 1 -- 172.21.15.81:0/2764704952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78180047c0 msgr2=0x7f7818004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:24.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f782ffff640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78180047c0 0x7f7818004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.046+0000 7f782ffff640 1 -- 172.21.15.81:0/2764704952 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7818003520 con 0x7f7818003a50 2023-11-08T16:32:24.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.047+0000 7f782ffff640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7818003a50 0x7f7818003e70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7820009ac0 tx=0x7f78180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad6cb646c2bf834 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:24.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.047+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782002c710 con 0x7f7818003a50 2023-11-08T16:32:24.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.048+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782002c870 con 0x7f7818003a50 2023-11-08T16:32:24.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.048+0000 7f782effd640 1 -- 172.21.15.81:0/2764704952 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782002cc00 con 0x7f7818003a50 2023-11-08T16:32:24.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.048+0000 7f7836a26640 1 -- 172.21.15.81:0/2764704952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7818003a50 msgr2=0x7f7818003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.048+0000 7f7836a26640 1 --2- 172.21.15.81:0/2764704952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7818003a50 0x7f7818003e70 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7820009ac0 tx=0x7f78180057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.048+0000 7f7834fc3640 1 -- 172.21.15.81:0/2764704952 reap_dead start 2023-11-08T16:32:24.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.048+0000 7f7836a26640 1 -- 172.21.15.81:0/2764704952 shutdown_connections 2023-11-08T16:32:24.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.048+0000 7f7836a26640 1 -- 172.21.15.81:0/2764704952 >> 172.21.15.81:0/2764704952 conn(0x7f78300cb490 msgr2=0x7f78300cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:24.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.048+0000 7f7836a26640 1 -- 172.21.15.81:0/2764704952 shutdown_connections 2023-11-08T16:32:24.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.048+0000 7f7836a26640 1 -- 172.21.15.81:0/2764704952 wait complete. 2023-11-08T16:32:24.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.050+0000 7f7836a26640 1 Processor -- start 2023-11-08T16:32:24.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.050+0000 7f7836a26640 1 -- start start 2023-11-08T16:32:24.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.051+0000 7f7836a26640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7818003a50 0x7f7830149590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.051+0000 7f7836a26640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78301f65f0 0x7f78300c49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.051+0000 7f7836a26640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78301f6a50 0x7f78300c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.051+0000 7f7836a26640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7830146900 con 0x7f78301f6a50 2023-11-08T16:32:24.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.051+0000 7f7836a26640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78300c8870 con 0x7f7818003a50 2023-11-08T16:32:24.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.051+0000 7f7836a26640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7830149ad0 con 0x7f78301f65f0 2023-11-08T16:32:24.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.052+0000 7f7834fc3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78301f6a50 0x7f78300c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.052+0000 7f7834fc3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78301f6a50 0x7f78300c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34108/0 (socket says 172.21.15.81:34108) 2023-11-08T16:32:24.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.052+0000 7f7834fc3640 1 -- 172.21.15.81:0/589797829 learned_addr learned my addr 172.21.15.81:0/589797829 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:24.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.052+0000 7f782ffff640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7818003a50 0x7f7830149590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.052+0000 7f782f7fe640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78301f65f0 0x7f78300c49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.052+0000 7f7834fc3640 1 -- 172.21.15.81:0/589797829 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78301f65f0 msgr2=0x7f78300c49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.052+0000 7f7834fc3640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78301f65f0 0x7f78300c49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.052+0000 7f7834fc3640 1 -- 172.21.15.81:0/589797829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7818003a50 msgr2=0x7f7830149590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.052+0000 7f7834fc3640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7818003a50 0x7f7830149590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.052+0000 7f7834fc3640 1 -- 172.21.15.81:0/589797829 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78300c7bf0 con 0x7f78301f6a50 2023-11-08T16:32:24.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.052+0000 7f7834fc3640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78301f6a50 0x7f78300c4f20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f78280012e0 tx=0x7f782800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:24.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.053+0000 7f782cff9640 1 -- 172.21.15.81:0/589797829 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828001b20 con 0x7f78301f6a50 2023-11-08T16:32:24.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.053+0000 7f782cff9640 1 -- 172.21.15.81:0/589797829 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782800bb80 con 0x7f78301f6a50 2023-11-08T16:32:24.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.053+0000 7f782cff9640 1 -- 172.21.15.81:0/589797829 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828007110 con 0x7f78301f6a50 2023-11-08T16:32:24.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.053+0000 7f7836a26640 1 -- 172.21.15.81:0/589797829 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78300c7ed0 con 0x7f78301f6a50 2023-11-08T16:32:24.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.053+0000 7f7836a26640 1 -- 172.21.15.81:0/589797829 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78300c8490 con 0x7f78301f6a50 2023-11-08T16:32:24.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.054+0000 7f7836a26640 1 -- 172.21.15.81:0/589797829 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78300c89b0 con 0x7f78301f6a50 2023-11-08T16:32:24.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.055+0000 7f782cff9640 1 -- 172.21.15.81:0/589797829 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7828007270 con 0x7f78301f6a50 2023-11-08T16:32:24.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.056+0000 7f782cff9640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781c06ae70 0x7f781c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.056+0000 7f782cff9640 1 -- 172.21.15.81:0/589797829 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f7828089bc0 con 0x7f78301f6a50 2023-11-08T16:32:24.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.056+0000 7f782ffff640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781c06ae70 0x7f781c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.057+0000 7f782ffff640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781c06ae70 0x7f781c06d330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7820036000 tx=0x7f7820030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:24.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.060+0000 7f782cff9640 1 -- 172.21.15.81:0/589797829 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78300c89b0 con 0x7f78301f6a50 2023-11-08T16:32:24.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.250+0000 7f7836a26640 1 -- 172.21.15.81:0/589797829 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7830161c50 con 0x7f78301f6a50 2023-11-08T16:32:24.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.253+0000 7f782cff9640 1 -- 172.21.15.81:0/589797829 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18684 (secure 0 0 0) 0x7f7830161c50 con 0x7f78301f6a50 2023-11-08T16:32:24.255 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:24.256 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":44,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:31:24.833641+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:30:27.977481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T16:30:29.171902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:30:40.141171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/1759199271":"2023-11-09T16:31:23.834479+0000","172.21.15.81:6836/1759199271":"2023-11-09T16:31:23.834479+0000","172.21.15.81:6837/3931731986":"2023-11-09T16:31:16.385652+0000","172.21.15.81:6836/3931731986":"2023-11-09T16:31:16.385652+0000","172.21.15.81:6837/773130363":"2023-11-09T16:31:14.352977+0000","172.21.15.81:6836/773130363":"2023-11-09T16:31:14.352977+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:32:24.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 -- 172.21.15.81:0/589797829 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781c06ae70 msgr2=0x7f781c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781c06ae70 0x7f781c06d330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7820036000 tx=0x7f7820030000 comp rx=0 tx=0).stop 2023-11-08T16:32:24.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 -- 172.21.15.81:0/589797829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78301f6a50 msgr2=0x7f78300c4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78301f6a50 0x7f78300c4f20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f78280012e0 tx=0x7f782800b4a0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 -- 172.21.15.81:0/589797829 shutdown_connections 2023-11-08T16:32:24.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f781c06ae70 0x7f781c06d330 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78301f6a50 0x7f78300c4f20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78301f65f0 0x7f78300c49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 --2- 172.21.15.81:0/589797829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7818003a50 0x7f7830149590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 -- 172.21.15.81:0/589797829 >> 172.21.15.81:0/589797829 conn(0x7f78300cb490 msgr2=0x7f78300af530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:24.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 -- 172.21.15.81:0/589797829 shutdown_connections 2023-11-08T16:32:24.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.256+0000 7f7836a26640 1 -- 172.21.15.81:0/589797829 wait complete. 2023-11-08T16:32:24.277 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:30:27.977481+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:30:29.171902+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:30:40.141171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:32:24.278 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:32:24.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.496+0000 7f8498b41640 1 Processor -- start 2023-11-08T16:32:24.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.496+0000 7f8498b41640 1 -- start start 2023-11-08T16:32:24.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.498+0000 7f8498b41640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f849415a0e0 0x7f84940ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.498+0000 7f8498b41640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84940c9bf0 0x7f84940ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.498+0000 7f8498b41640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f84940ca550 0x7f849415dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.498+0000 7f8498b41640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8494071980 con 0x7f84940ca550 2023-11-08T16:32:24.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.498+0000 7f8498b41640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84940ce960 con 0x7f849415a0e0 2023-11-08T16:32:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.498+0000 7f8498b41640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84940c9480 con 0x7f84940c9bf0 2023-11-08T16:32:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.498+0000 7f8492d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f84940ca550 0x7f849415dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.498+0000 7f8492d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f84940ca550 0x7f849415dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34112/0 (socket says 172.21.15.81:34112) 2023-11-08T16:32:24.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.498+0000 7f8492d9d640 1 -- 172.21.15.81:0/3172576264 learned_addr learned my addr 172.21.15.81:0/3172576264 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:24.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.498+0000 7f849259c640 1 --2- 172.21.15.81:0/3172576264 >> v2:172.21.15.93:3300/0 conn(0x7f849415a0e0 0x7f84940ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.498+0000 7f848bfff640 1 --2- 172.21.15.81:0/3172576264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84940c9bf0 0x7f84940ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.499+0000 7f8492d9d640 1 -- 172.21.15.81:0/3172576264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84940c9bf0 msgr2=0x7f84940ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:24.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.499+0000 7f8492d9d640 1 --2- 172.21.15.81:0/3172576264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84940c9bf0 0x7f84940ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.499+0000 7f8492d9d640 1 -- 172.21.15.81:0/3172576264 >> v2:172.21.15.93:3300/0 conn(0x7f849415a0e0 msgr2=0x7f84940ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:24.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.499+0000 7f8492d9d640 1 --2- 172.21.15.81:0/3172576264 >> v2:172.21.15.93:3300/0 conn(0x7f849415a0e0 0x7f84940ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.499+0000 7f8492d9d640 1 -- 172.21.15.81:0/3172576264 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849415a4c0 con 0x7f84940ca550 2023-11-08T16:32:24.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.500+0000 7f8492d9d640 1 --2- 172.21.15.81:0/3172576264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84940ca550 0x7f849415dbb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8494067640 tx=0x7f848c02e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a571de2e179e7a31 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:24.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.500+0000 7f8491d9b640 1 -- 172.21.15.81:0/3172576264 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c00b0d0 con 0x7f84940ca550 2023-11-08T16:32:24.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.500+0000 7f8491d9b640 1 -- 172.21.15.81:0/3172576264 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848c00b230 con 0x7f84940ca550 2023-11-08T16:32:24.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.500+0000 7f8491d9b640 1 -- 172.21.15.81:0/3172576264 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848c009600 con 0x7f84940ca550 2023-11-08T16:32:24.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.501+0000 7f8498b41640 1 -- 172.21.15.81:0/3172576264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84940ca550 msgr2=0x7f849415dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.501+0000 7f8498b41640 1 --2- 172.21.15.81:0/3172576264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84940ca550 0x7f849415dbb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8494067640 tx=0x7f848c02e180 comp rx=0 tx=0).stop 2023-11-08T16:32:24.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.501+0000 7f8498b41640 1 -- 172.21.15.81:0/3172576264 shutdown_connections 2023-11-08T16:32:24.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.501+0000 7f8498b41640 1 --2- 172.21.15.81:0/3172576264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84940c9bf0 0x7f84940ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.501+0000 7f8498b41640 1 --2- 172.21.15.81:0/3172576264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84940ca550 0x7f849415dbb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.501+0000 7f8498b41640 1 --2- 172.21.15.81:0/3172576264 >> v2:172.21.15.93:3300/0 conn(0x7f849415a0e0 0x7f84940ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.501+0000 7f8498b41640 1 -- 172.21.15.81:0/3172576264 >> 172.21.15.81:0/3172576264 conn(0x7f84940cb580 msgr2=0x7f84940cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:24.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.501+0000 7f8498b41640 1 -- 172.21.15.81:0/3172576264 shutdown_connections 2023-11-08T16:32:24.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.501+0000 7f8498b41640 1 -- 172.21.15.81:0/3172576264 wait complete. 2023-11-08T16:32:24.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.503+0000 7f8498b41640 1 Processor -- start 2023-11-08T16:32:24.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.503+0000 7f8498b41640 1 -- start start 2023-11-08T16:32:24.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.504+0000 7f8498b41640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84940c9bf0 0x7f84941f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.504+0000 7f8498b41640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84940ca550 0x7f84941f0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.504+0000 7f8498b41640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f849415a0e0 0x7f849415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.504+0000 7f8498b41640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84941f06c0 con 0x7f84940c9bf0 2023-11-08T16:32:24.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.504+0000 7f8498b41640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84941f3030 con 0x7f849415a0e0 2023-11-08T16:32:24.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.504+0000 7f8498b41640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84941f3310 con 0x7f84940ca550 2023-11-08T16:32:24.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.505+0000 7f849259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84940c9bf0 0x7f84941f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.505+0000 7f849259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84940c9bf0 0x7f84941f3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34120/0 (socket says 172.21.15.81:34120) 2023-11-08T16:32:24.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.505+0000 7f849259c640 1 -- 172.21.15.81:0/173454250 learned_addr learned my addr 172.21.15.81:0/173454250 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:24.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.505+0000 7f848bfff640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84940ca550 0x7f84941f0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.505+0000 7f8492d9d640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f849415a0e0 0x7f849415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.505+0000 7f849259c640 1 -- 172.21.15.81:0/173454250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84940ca550 msgr2=0x7f84941f0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.505+0000 7f849259c640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84940ca550 0x7f84941f0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.505+0000 7f849259c640 1 -- 172.21.15.81:0/173454250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f849415a0e0 msgr2=0x7f849415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.505+0000 7f849259c640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f849415a0e0 0x7f849415a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.505+0000 7f849259c640 1 -- 172.21.15.81:0/173454250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c00bd40 con 0x7f84940c9bf0 2023-11-08T16:32:24.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.505+0000 7f849259c640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84940c9bf0 0x7f84941f3e10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f848400d800 tx=0x7f848400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:24.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.506+0000 7f848affd640 1 -- 172.21.15.81:0/173454250 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400b6c0 con 0x7f84940c9bf0 2023-11-08T16:32:24.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.506+0000 7f848affd640 1 -- 172.21.15.81:0/173454250 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f848400b820 con 0x7f84940c9bf0 2023-11-08T16:32:24.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.506+0000 7f848affd640 1 -- 172.21.15.81:0/173454250 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400bbb0 con 0x7f84940c9bf0 2023-11-08T16:32:24.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.506+0000 7f8498b41640 1 -- 172.21.15.81:0/173454250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84941f35f0 con 0x7f84940c9bf0 2023-11-08T16:32:24.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.506+0000 7f8498b41640 1 -- 172.21.15.81:0/173454250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84941f3bb0 con 0x7f84940c9bf0 2023-11-08T16:32:24.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.507+0000 7f8498b41640 1 -- 172.21.15.81:0/173454250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f849404ece0 con 0x7f84940c9bf0 2023-11-08T16:32:24.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.508+0000 7f848affd640 1 -- 172.21.15.81:0/173454250 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f848400bd10 con 0x7f84940c9bf0 2023-11-08T16:32:24.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.509+0000 7f848affd640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847406aec0 0x7f847406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.509+0000 7f848affd640 1 -- 172.21.15.81:0/173454250 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f8484089e30 con 0x7f84940c9bf0 2023-11-08T16:32:24.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.509+0000 7f848bfff640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847406aec0 0x7f847406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.510+0000 7f848bfff640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847406aec0 0x7f847406d380 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f847c006fd0 tx=0x7f847c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:24.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.514+0000 7f848affd640 1 -- 172.21.15.81:0/173454250 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8484054c20 con 0x7f84940c9bf0 2023-11-08T16:32:24.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.706+0000 7f8498b41640 1 -- 172.21.15.81:0/173454250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f849406d030 con 0x7f84940c9bf0 2023-11-08T16:32:24.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.711+0000 7f848affd640 1 -- 172.21.15.81:0/173454250 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8484058240 con 0x7f84940c9bf0 2023-11-08T16:32:24.713 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:32:24.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 -- 172.21.15.81:0/173454250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847406aec0 msgr2=0x7f847406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847406aec0 0x7f847406d380 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f847c006fd0 tx=0x7f847c008040 comp rx=0 tx=0).stop 2023-11-08T16:32:24.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 -- 172.21.15.81:0/173454250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84940c9bf0 msgr2=0x7f84941f3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84940c9bf0 0x7f84941f3e10 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f848400d800 tx=0x7f848400dcc0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 -- 172.21.15.81:0/173454250 shutdown_connections 2023-11-08T16:32:24.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847406aec0 0x7f847406d380 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f849415a0e0 0x7f849415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84940ca550 0x7f84941f0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 --2- 172.21.15.81:0/173454250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84940c9bf0 0x7f84941f3e10 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 -- 172.21.15.81:0/173454250 >> 172.21.15.81:0/173454250 conn(0x7f84940cb580 msgr2=0x7f849406bfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 -- 172.21.15.81:0/173454250 shutdown_connections 2023-11-08T16:32:24.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.714+0000 7f8498b41640 1 -- 172.21.15.81:0/173454250 wait complete. 2023-11-08T16:32:24.735 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:32:24.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.947+0000 7f3c1b290640 1 Processor -- start 2023-11-08T16:32:24.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.947+0000 7f3c1b290640 1 -- start start 2023-11-08T16:32:24.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.948+0000 7f3c1b290640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c140ce390 0x7f3c1415e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.949+0000 7f3c1b290640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3c1406be20 0x7f3c140ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.949+0000 7f3c1b290640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c1415e8d0 con 0x7f3c140c9200 2023-11-08T16:32:24.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.949+0000 7f3c1b290640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c140ca2f0 con 0x7f3c1406be20 2023-11-08T16:32:24.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.949+0000 7f3c1b290640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140ce770 con 0x7f3c140ce390 2023-11-08T16:32:24.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.949+0000 7f3c1982d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3c1406be20 0x7f3c140ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.949+0000 7f3c1982d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3c1406be20 0x7f3c140ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34124/0 (socket says 172.21.15.81:34124) 2023-11-08T16:32:24.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.949+0000 7f3c1982d640 1 -- 172.21.15.81:0/2936944625 learned_addr learned my addr 172.21.15.81:0/2936944625 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:24.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.949+0000 7f3c1902c640 1 --2- 172.21.15.81:0/2936944625 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c140ce390 0x7f3c1415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.950+0000 7f3c1982d640 1 -- 172.21.15.81:0/2936944625 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c140ce390 msgr2=0x7f3c1415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:24.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.950+0000 7f3c1982d640 1 --2- 172.21.15.81:0/2936944625 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c140ce390 0x7f3c1415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.950+0000 7f3c1982d640 1 -- 172.21.15.81:0/2936944625 >> v1:172.21.15.93:6789/0 conn(0x7f3c140c9200 legacy=0x7f3c1406b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:24.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.950+0000 7f3c1982d640 1 -- 172.21.15.81:0/2936944625 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c140c96e0 con 0x7f3c1406be20 2023-11-08T16:32:24.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.950+0000 7f3c1902c640 1 --2- 172.21.15.81:0/2936944625 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c140ce390 0x7f3c1415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:24.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.951+0000 7f3c1982d640 1 --2- 172.21.15.81:0/2936944625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c1406be20 0x7f3c140ac8b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009e50 tx=0x7f3c0c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=451e3bf2736d84b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:24.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.951+0000 7f3c0bfff640 1 -- 172.21.15.81:0/2936944625 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c02cb00 con 0x7f3c1406be20 2023-11-08T16:32:24.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.951+0000 7f3c0bfff640 1 -- 172.21.15.81:0/2936944625 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c02cc60 con 0x7f3c1406be20 2023-11-08T16:32:24.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.951+0000 7f3c0bfff640 1 -- 172.21.15.81:0/2936944625 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0343f0 con 0x7f3c1406be20 2023-11-08T16:32:24.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.952+0000 7f3c1b290640 1 -- 172.21.15.81:0/2936944625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c1406be20 msgr2=0x7f3c140ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.952+0000 7f3c1b290640 1 --2- 172.21.15.81:0/2936944625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c1406be20 0x7f3c140ac8b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009e50 tx=0x7f3c0c02bf60 comp rx=0 tx=0).stop 2023-11-08T16:32:24.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.952+0000 7f3c1b290640 1 -- 172.21.15.81:0/2936944625 shutdown_connections 2023-11-08T16:32:24.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.952+0000 7f3c1b290640 1 --2- 172.21.15.81:0/2936944625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c1406be20 0x7f3c140ac8b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.952+0000 7f3c1b290640 1 --2- 172.21.15.81:0/2936944625 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c140ce390 0x7f3c1415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.952+0000 7f3c1b290640 1 -- 172.21.15.81:0/2936944625 >> 172.21.15.81:0/2936944625 conn(0x7f3c140cb580 msgr2=0x7f3c140cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:24.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.952+0000 7f3c1b290640 1 -- 172.21.15.81:0/2936944625 shutdown_connections 2023-11-08T16:32:24.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.952+0000 7f3c1b290640 1 -- 172.21.15.81:0/2936944625 wait complete. 2023-11-08T16:32:24.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.954+0000 7f3c1b290640 1 Processor -- start 2023-11-08T16:32:24.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.954+0000 7f3c1b290640 1 -- start start 2023-11-08T16:32:24.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.955+0000 7f3c1b290640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c1406be20 0x7f3c140c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.955+0000 7f3c1b290640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c140c9200 0x7f3c140c2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.955+0000 7f3c1b290640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c140ce390 0x7f3c140c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.955+0000 7f3c1b290640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140c2b50 con 0x7f3c140ce390 2023-11-08T16:32:24.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.955+0000 7f3c1b290640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140c85e0 con 0x7f3c140c9200 2023-11-08T16:32:24.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.955+0000 7f3c1b290640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c140c2d40 con 0x7f3c1406be20 2023-11-08T16:32:24.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.955+0000 7f3c1982d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c140ce390 0x7f3c140c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.955+0000 7f3c1982d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c140ce390 0x7f3c140c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34138/0 (socket says 172.21.15.81:34138) 2023-11-08T16:32:24.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.955+0000 7f3c1982d640 1 -- 172.21.15.81:0/2496880008 learned_addr learned my addr 172.21.15.81:0/2496880008 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:24.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.956+0000 7f3c1882b640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c140c9200 0x7f3c140c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.956+0000 7f3c1902c640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c1406be20 0x7f3c140c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.956+0000 7f3c1982d640 1 -- 172.21.15.81:0/2496880008 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c1406be20 msgr2=0x7f3c140c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.956+0000 7f3c1982d640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c1406be20 0x7f3c140c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.956+0000 7f3c1982d640 1 -- 172.21.15.81:0/2496880008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c140c9200 msgr2=0x7f3c140c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.956+0000 7f3c1982d640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c140c9200 0x7f3c140c2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.956+0000 7f3c1982d640 1 -- 172.21.15.81:0/2496880008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0c009b30 con 0x7f3c140ce390 2023-11-08T16:32:24.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.956+0000 7f3c1982d640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c140ce390 0x7f3c140c80a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c036040 tx=0x7f3c0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:24.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.957+0000 7f3c09ffb640 1 -- 172.21.15.81:0/2496880008 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c03e450 con 0x7f3c140ce390 2023-11-08T16:32:24.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.957+0000 7f3c09ffb640 1 -- 172.21.15.81:0/2496880008 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c03e5d0 con 0x7f3c140ce390 2023-11-08T16:32:24.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.957+0000 7f3c1b290640 1 -- 172.21.15.81:0/2496880008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c140c3020 con 0x7f3c140ce390 2023-11-08T16:32:24.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.957+0000 7f3c09ffb640 1 -- 172.21.15.81:0/2496880008 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c03e980 con 0x7f3c140ce390 2023-11-08T16:32:24.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.957+0000 7f3c1b290640 1 -- 172.21.15.81:0/2496880008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c140c35e0 con 0x7f3c140ce390 2023-11-08T16:32:24.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.958+0000 7f3c1b290640 1 -- 172.21.15.81:0/2496880008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1404ece0 con 0x7f3c140ce390 2023-11-08T16:32:24.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.959+0000 7f3c09ffb640 1 -- 172.21.15.81:0/2496880008 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c0c03eae0 con 0x7f3c140ce390 2023-11-08T16:32:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.960+0000 7f3c09ffb640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3bf806aea0 0x7f3bf806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.960+0000 7f3c09ffb640 1 -- 172.21.15.81:0/2496880008 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f3c0c0abf60 con 0x7f3c140ce390 2023-11-08T16:32:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.961+0000 7f3c1902c640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3bf806aea0 0x7f3bf806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.961+0000 7f3c1902c640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3bf806aea0 0x7f3bf806d360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000a950 tx=0x7f3c10008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:24.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:24.965+0000 7f3c09ffb640 1 -- 172.21.15.81:0/2496880008 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c0c076c40 con 0x7f3c140ce390 2023-11-08T16:32:25.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.156+0000 7f3c1b290640 1 -- 172.21.15.81:0/2496880008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3c1415e8d0 con 0x7f3c140ce390 2023-11-08T16:32:25.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.157+0000 7f3c09ffb640 1 -- 172.21.15.81:0/2496880008 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v44) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3c1415e8d0 con 0x7f3c140ce390 2023-11-08T16:32:25.159 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:32:25.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.159+0000 7f3c1b290640 1 -- 172.21.15.81:0/2496880008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3bf806aea0 msgr2=0x7f3bf806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.159+0000 7f3c1b290640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3bf806aea0 0x7f3bf806d360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3c1000a950 tx=0x7f3c10008040 comp rx=0 tx=0).stop 2023-11-08T16:32:25.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.159+0000 7f3c1b290640 1 -- 172.21.15.81:0/2496880008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c140ce390 msgr2=0x7f3c140c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.159+0000 7f3c1b290640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c140ce390 0x7f3c140c80a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c036040 tx=0x7f3c0c004a40 comp rx=0 tx=0).stop 2023-11-08T16:32:25.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.160+0000 7f3c1b290640 1 -- 172.21.15.81:0/2496880008 shutdown_connections 2023-11-08T16:32:25.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.160+0000 7f3c1b290640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3bf806aea0 0x7f3bf806d360 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.160+0000 7f3c1b290640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c140ce390 0x7f3c140c80a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.160+0000 7f3c1b290640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c140c9200 0x7f3c140c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.160+0000 7f3c1b290640 1 --2- 172.21.15.81:0/2496880008 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c1406be20 0x7f3c140c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.160+0000 7f3c1b290640 1 -- 172.21.15.81:0/2496880008 >> 172.21.15.81:0/2496880008 conn(0x7f3c140cb580 msgr2=0x7f3c140ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:25.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.160+0000 7f3c1b290640 1 -- 172.21.15.81:0/2496880008 shutdown_connections 2023-11-08T16:32:25.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.160+0000 7f3c1b290640 1 -- 172.21.15.81:0/2496880008 wait complete. 2023-11-08T16:32:25.181 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:32:25.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.398+0000 7fb8f8789640 1 Processor -- start 2023-11-08T16:32:25.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.398+0000 7fb8f8789640 1 -- start start 2023-11-08T16:32:25.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.399+0000 7fb8f8789640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00ccc10 0x7fb8f00ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.399+0000 7fb8f8789640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f00ce960 con 0x7fb8f006b710 2023-11-08T16:32:25.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.399+0000 7fb8f8789640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8f00c9480 con 0x7fb8f015a0e0 2023-11-08T16:32:25.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.399+0000 7fb8f8789640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f0071980 con 0x7fb8f00ccc10 2023-11-08T16:32:25.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.400+0000 7fb8f6d26640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb8f006b710 0x7fb8f006bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50294/0 (socket says 172.21.15.81:50294) 2023-11-08T16:32:25.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.400+0000 7fb8f6d26640 1 -- 172.21.15.81:0/3001337276 learned_addr learned my addr 172.21.15.81:0/3001337276 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:25.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.400+0000 7fb8f5d24640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00ccc10 0x7fb8f00ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:25.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.400+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561509909 0 0) 0x7fb8e8001490 con 0x7fb8f006b710 2023-11-08T16:32:25.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.400+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8dc003520 con 0x7fb8f006b710 2023-11-08T16:32:25.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.400+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2189508181 0 0) 0x7fb8dc003520 con 0x7fb8f006b710 2023-11-08T16:32:25.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.400+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00ccc10 msgr2=0x7fb8f00ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:25.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.400+0000 7fb8f5523640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00ccc10 0x7fb8f00ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 >> v1:172.21.15.93:6789/0 conn(0x7fb8f015a0e0 legacy=0x7fb8f00c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f015a4c0 con 0x7fb8f006b710 2023-11-08T16:32:25.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fb8e8001b80 con 0x7fb8f006b710 2023-11-08T16:32:25.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 >> v1:172.21.15.81:6789/0 conn(0x7fb8f006b710 legacy=0x7fb8f006bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f5523640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 0x7fb8dc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f5523640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00ccc10 0x7fb8dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f5523640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8dc004d00 0x7fb8dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f0071980 con 0x7fb8dc003a10 2023-11-08T16:32:25.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc00bb20 con 0x7fb8dc004d00 2023-11-08T16:32:25.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc00def0 con 0x7fb8f00ccc10 2023-11-08T16:32:25.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f6525640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 0x7fb8dc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:25.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f5d24640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00ccc10 0x7fb8dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:25.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f6525640 1 -- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00ccc10 msgr2=0x7fb8dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:25.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f6525640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00ccc10 0x7fb8dc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f6525640 1 -- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8dc004d00 msgr2=0x7fb8dc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:25.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f6d26640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8dc004d00 0x7fb8dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:25.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f6525640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8dc004d00 0x7fb8dc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f6525640 1 -- 172.21.15.81:0/3001337276 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc003520 con 0x7fb8dc003a10 2023-11-08T16:32:25.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.401+0000 7fb8f6d26640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8dc004d00 0x7fb8dc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:25.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.403+0000 7fb8f6525640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 0x7fb8dc003e30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec001720 tx=0x7fb8dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353d8fba6458b552 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:25.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.403+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0052d0 con 0x7fb8dc003a10 2023-11-08T16:32:25.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.403+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec02e050 con 0x7fb8dc003a10 2023-11-08T16:32:25.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.403+0000 7fb8f5523640 1 -- 172.21.15.81:0/3001337276 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0035f0 con 0x7fb8dc003a10 2023-11-08T16:32:25.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.403+0000 7fb8f8789640 1 -- 172.21.15.81:0/3001337276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 msgr2=0x7fb8dc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.403+0000 7fb8f8789640 1 --2- 172.21.15.81:0/3001337276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 0x7fb8dc003e30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec001720 tx=0x7fb8dc0057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.403+0000 7fb8f6d26640 1 -- 172.21.15.81:0/3001337276 reap_dead start 2023-11-08T16:32:25.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.403+0000 7fb8f8789640 1 -- 172.21.15.81:0/3001337276 shutdown_connections 2023-11-08T16:32:25.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.403+0000 7fb8f8789640 1 -- 172.21.15.81:0/3001337276 >> 172.21.15.81:0/3001337276 conn(0x7fb8f00cb580 msgr2=0x7fb8f00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:25.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.404+0000 7fb8f8789640 1 -- 172.21.15.81:0/3001337276 shutdown_connections 2023-11-08T16:32:25.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.404+0000 7fb8f8789640 1 -- 172.21.15.81:0/3001337276 wait complete. 2023-11-08T16:32:25.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.405+0000 7fb8f8789640 1 Processor -- start 2023-11-08T16:32:25.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.405+0000 7fb8f8789640 1 -- start start 2023-11-08T16:32:25.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.406+0000 7fb8f8789640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 0x7fb8f0155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.406+0000 7fb8f8789640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8f00ac8b0 0x7fb8f0155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.406+0000 7fb8f8789640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00c50e0 0x7fb8f00c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.406+0000 7fb8f8789640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00c78a0 con 0x7fb8dc003a10 2023-11-08T16:32:25.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.406+0000 7fb8f8789640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f0155b90 con 0x7fb8f00ac8b0 2023-11-08T16:32:25.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.406+0000 7fb8f8789640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00acd30 con 0x7fb8f00c50e0 2023-11-08T16:32:25.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f6525640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 0x7fb8f0155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:25.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f6525640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 0x7fb8f0155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34154/0 (socket says 172.21.15.81:34154) 2023-11-08T16:32:25.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f6525640 1 -- 172.21.15.81:0/2392003726 learned_addr learned my addr 172.21.15.81:0/2392003726 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:25.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f5d24640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8f00ac8b0 0x7fb8f0155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:25.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f6d26640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00c50e0 0x7fb8f00c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:25.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f6525640 1 -- 172.21.15.81:0/2392003726 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00c50e0 msgr2=0x7fb8f00c2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f6525640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00c50e0 0x7fb8f00c2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f6525640 1 -- 172.21.15.81:0/2392003726 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8f00ac8b0 msgr2=0x7fb8f0155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f6525640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8f00ac8b0 0x7fb8f0155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f6525640 1 -- 172.21.15.81:0/2392003726 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f00c59c0 con 0x7fb8dc003a10 2023-11-08T16:32:25.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f6d26640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00c50e0 0x7fb8f00c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:25.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.407+0000 7fb8f6525640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 0x7fb8f0155d00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec005040 tx=0x7fb8ec001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:25.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.408+0000 7fb8e6ffd640 1 -- 172.21.15.81:0/2392003726 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec007a30 con 0x7fb8dc003a10 2023-11-08T16:32:25.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.408+0000 7fb8e6ffd640 1 -- 172.21.15.81:0/2392003726 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8ec0069e0 con 0x7fb8dc003a10 2023-11-08T16:32:25.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.408+0000 7fb8e6ffd640 1 -- 172.21.15.81:0/2392003726 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec006d90 con 0x7fb8dc003a10 2023-11-08T16:32:25.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.408+0000 7fb8f8789640 1 -- 172.21.15.81:0/2392003726 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f00c6f40 con 0x7fb8dc003a10 2023-11-08T16:32:25.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.408+0000 7fb8f8789640 1 -- 172.21.15.81:0/2392003726 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f00c74c0 con 0x7fb8dc003a10 2023-11-08T16:32:25.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.409+0000 7fb8f8789640 1 -- 172.21.15.81:0/2392003726 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f00c7b10 con 0x7fb8dc003a10 2023-11-08T16:32:25.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.411+0000 7fb8e6ffd640 1 -- 172.21.15.81:0/2392003726 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb8ec0035f0 con 0x7fb8dc003a10 2023-11-08T16:32:25.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.412+0000 7fb8e6ffd640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb8d406ae70 0x7fb8d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.412+0000 7fb8e6ffd640 1 -- 172.21.15.81:0/2392003726 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7fb8ec0ab820 con 0x7fb8dc003a10 2023-11-08T16:32:25.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.412+0000 7fb8f5d24640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb8d406ae70 0x7fb8d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:25.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.413+0000 7fb8f5d24640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb8d406ae70 0x7fb8d406d330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00c6900 tx=0x7fb8e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:25.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.417+0000 7fb8e6ffd640 1 -- 172.21.15.81:0/2392003726 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb8ec076620 con 0x7fb8dc003a10 2023-11-08T16:32:25.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.605+0000 7fb8f8789640 1 -- 172.21.15.81:0/2392003726 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8f004ece0 con 0x7fb8dc003a10 2023-11-08T16:32:25.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.606+0000 7fb8e6ffd640 1 -- 172.21.15.81:0/2392003726 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v44) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb8ec080070 con 0x7fb8dc003a10 2023-11-08T16:32:25.608 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:32:25.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.608+0000 7fb8f8789640 1 -- 172.21.15.81:0/2392003726 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb8d406ae70 msgr2=0x7fb8d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.609+0000 7fb8f8789640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb8d406ae70 0x7fb8d406d330 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00c6900 tx=0x7fb8e0008040 comp rx=0 tx=0).stop 2023-11-08T16:32:25.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.609+0000 7fb8f8789640 1 -- 172.21.15.81:0/2392003726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 msgr2=0x7fb8f0155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.609+0000 7fb8f8789640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 0x7fb8f0155d00 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec005040 tx=0x7fb8ec001120 comp rx=0 tx=0).stop 2023-11-08T16:32:25.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.609+0000 7fb8f8789640 1 -- 172.21.15.81:0/2392003726 shutdown_connections 2023-11-08T16:32:25.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.609+0000 7fb8f8789640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb8d406ae70 0x7fb8d406d330 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.609+0000 7fb8f8789640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb8f00c50e0 0x7fb8f00c2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.609+0000 7fb8f8789640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb8f00ac8b0 0x7fb8f0155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.609+0000 7fb8f8789640 1 --2- 172.21.15.81:0/2392003726 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb8dc003a10 0x7fb8f0155d00 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.609+0000 7fb8f8789640 1 -- 172.21.15.81:0/2392003726 >> 172.21.15.81:0/2392003726 conn(0x7fb8f00cb580 msgr2=0x7fb8f006bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:25.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.609+0000 7fb8f8789640 1 -- 172.21.15.81:0/2392003726 shutdown_connections 2023-11-08T16:32:25.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.609+0000 7fb8f8789640 1 -- 172.21.15.81:0/2392003726 wait complete. 2023-11-08T16:32:25.628 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:32:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.835+0000 7f0768f39640 1 Processor -- start 2023-11-08T16:32:25.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.835+0000 7f0768f39640 1 -- start start 2023-11-08T16:32:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.836+0000 7f0768f39640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f07640ce0d0 0x7f07640caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.837+0000 7f0768f39640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07640cda10 con 0x7f076415bfc0 2023-11-08T16:32:25.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.837+0000 7f0768f39640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f076406fa30 con 0x7f076415f280 2023-11-08T16:32:25.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.837+0000 7f0768f39640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07640ce4b0 con 0x7f07640ce0d0 2023-11-08T16:32:25.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.837+0000 7f076259c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f07640ce0d0 0x7f07640caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:25.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.837+0000 7f0761d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f076415bfc0 0x7f076415eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50298/0 (socket says 172.21.15.81:50298) 2023-11-08T16:32:25.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.837+0000 7f0761d9b640 1 -- 172.21.15.81:0/1660223978 learned_addr learned my addr 172.21.15.81:0/1660223978 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:25.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.837+0000 7f076259c640 1 -- 172.21.15.81:0/1660223978 >> v1:172.21.15.93:6789/0 conn(0x7f076415f280 legacy=0x7f07641516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:25.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.837+0000 7f076259c640 1 -- 172.21.15.81:0/1660223978 >> v1:172.21.15.81:6789/0 conn(0x7f076415bfc0 legacy=0x7f076415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.838+0000 7f076259c640 1 -- 172.21.15.81:0/1660223978 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076406fca0 con 0x7f07640ce0d0 2023-11-08T16:32:25.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.839+0000 7f076259c640 1 --2- 172.21.15.81:0/1660223978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07640ce0d0 0x7f07640caf20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f07540097a0 tx=0x7f075402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0083dc3fa009cf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:25.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.839+0000 7f076159a640 1 -- 172.21.15.81:0/1660223978 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754031830 con 0x7f07640ce0d0 2023-11-08T16:32:25.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.839+0000 7f076159a640 1 -- 172.21.15.81:0/1660223978 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0754031990 con 0x7f07640ce0d0 2023-11-08T16:32:25.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.839+0000 7f076159a640 1 -- 172.21.15.81:0/1660223978 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0754031d20 con 0x7f07640ce0d0 2023-11-08T16:32:25.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.840+0000 7f0768f39640 1 -- 172.21.15.81:0/1660223978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07640ce0d0 msgr2=0x7f07640caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.840+0000 7f0768f39640 1 --2- 172.21.15.81:0/1660223978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07640ce0d0 0x7f07640caf20 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f07540097a0 tx=0x7f075402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.840+0000 7f0768f39640 1 -- 172.21.15.81:0/1660223978 shutdown_connections 2023-11-08T16:32:25.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.840+0000 7f0768f39640 1 --2- 172.21.15.81:0/1660223978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07640ce0d0 0x7f07640caf20 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.840+0000 7f0768f39640 1 -- 172.21.15.81:0/1660223978 >> 172.21.15.81:0/1660223978 conn(0x7f07640cb580 msgr2=0x7f07640cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:25.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.840+0000 7f0768f39640 1 -- 172.21.15.81:0/1660223978 shutdown_connections 2023-11-08T16:32:25.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.840+0000 7f0768f39640 1 -- 172.21.15.81:0/1660223978 wait complete. 2023-11-08T16:32:25.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.842+0000 7f0768f39640 1 Processor -- start 2023-11-08T16:32:25.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.842+0000 7f0768f39640 1 -- start start 2023-11-08T16:32:25.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f0768f39640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07640ce0d0 0x7f07640c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f0768f39640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f076415bfc0 0x7f07640ae9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f0768f39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f076415f280 0x7f07640c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f0768f39640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07640c7e70 con 0x7f07640ce0d0 2023-11-08T16:32:25.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f0768f39640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07640c6c20 con 0x7f076415bfc0 2023-11-08T16:32:25.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f0768f39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07641f7bc0 con 0x7f076415f280 2023-11-08T16:32:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f076259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07640ce0d0 0x7f07640c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f076259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07640ce0d0 0x7f07640c7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34178/0 (socket says 172.21.15.81:34178) 2023-11-08T16:32:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f076259c640 1 -- 172.21.15.81:0/122416095 learned_addr learned my addr 172.21.15.81:0/122416095 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:25.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f076259c640 1 -- 172.21.15.81:0/122416095 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f076415f280 msgr2=0x7f07640c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f076259c640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f076415f280 0x7f07640c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f076259c640 1 -- 172.21.15.81:0/122416095 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f076415bfc0 msgr2=0x7f07640ae9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:25.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f076259c640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f076415bfc0 0x7f07640ae9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:25.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.843+0000 7f076259c640 1 -- 172.21.15.81:0/122416095 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0754009400 con 0x7f07640ce0d0 2023-11-08T16:32:25.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.844+0000 7f076259c640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07640ce0d0 0x7f07640c7930 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f07540098c0 tx=0x7f07540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:25.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.844+0000 7f0752ffd640 1 -- 172.21.15.81:0/122416095 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075403ba70 con 0x7f07640ce0d0 2023-11-08T16:32:25.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.844+0000 7f0752ffd640 1 -- 172.21.15.81:0/122416095 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075403bbf0 con 0x7f07640ce0d0 2023-11-08T16:32:25.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.844+0000 7f0752ffd640 1 -- 172.21.15.81:0/122416095 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07540453f0 con 0x7f07640ce0d0 2023-11-08T16:32:25.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.844+0000 7f0768f39640 1 -- 172.21.15.81:0/122416095 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07640c6ed0 con 0x7f07640ce0d0 2023-11-08T16:32:25.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.844+0000 7f0768f39640 1 -- 172.21.15.81:0/122416095 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07640c7430 con 0x7f07640ce0d0 2023-11-08T16:32:25.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.845+0000 7f0768f39640 1 -- 172.21.15.81:0/122416095 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076404ece0 con 0x7f07640ce0d0 2023-11-08T16:32:25.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.846+0000 7f0752ffd640 1 -- 172.21.15.81:0/122416095 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f075404e450 con 0x7f07640ce0d0 2023-11-08T16:32:25.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.847+0000 7f0752ffd640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f074406ae70 0x7f074406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:25.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.847+0000 7f0752ffd640 1 -- 172.21.15.81:0/122416095 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f07540abac0 con 0x7f07640ce0d0 2023-11-08T16:32:25.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.847+0000 7f0761d9b640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f074406ae70 0x7f074406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:25.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.848+0000 7f0761d9b640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f074406ae70 0x7f074406d330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f07640af6f0 tx=0x7f0758007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:25.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:25.851+0000 7f0752ffd640 1 -- 172.21.15.81:0/122416095 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0754076960 con 0x7f07640ce0d0 2023-11-08T16:32:26.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.041+0000 7f0768f39640 1 -- 172.21.15.81:0/122416095 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f07640cb1d0 con 0x7f07640ce0d0 2023-11-08T16:32:26.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.042+0000 7f0752ffd640 1 -- 172.21.15.81:0/122416095 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v44) v1 ==== 106+0+11 (secure 0 0 0) 0x7f07540abe60 con 0x7f07640ce0d0 2023-11-08T16:32:26.044 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:32:26.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.044+0000 7f0768f39640 1 -- 172.21.15.81:0/122416095 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f074406ae70 msgr2=0x7f074406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.044+0000 7f0768f39640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f074406ae70 0x7f074406d330 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f07640af6f0 tx=0x7f0758007a40 comp rx=0 tx=0).stop 2023-11-08T16:32:26.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.045+0000 7f0768f39640 1 -- 172.21.15.81:0/122416095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07640ce0d0 msgr2=0x7f07640c7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.045+0000 7f0768f39640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07640ce0d0 0x7f07640c7930 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f07540098c0 tx=0x7f07540046c0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.045+0000 7f0768f39640 1 -- 172.21.15.81:0/122416095 shutdown_connections 2023-11-08T16:32:26.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.045+0000 7f0768f39640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f074406ae70 0x7f074406d330 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.045+0000 7f0768f39640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f076415f280 0x7f07640c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.045+0000 7f0768f39640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f076415bfc0 0x7f07640ae9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.045+0000 7f0768f39640 1 --2- 172.21.15.81:0/122416095 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07640ce0d0 0x7f07640c7930 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.045+0000 7f0768f39640 1 -- 172.21.15.81:0/122416095 >> 172.21.15.81:0/122416095 conn(0x7f07640cb580 msgr2=0x7f076415b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:26.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.045+0000 7f0768f39640 1 -- 172.21.15.81:0/122416095 shutdown_connections 2023-11-08T16:32:26.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.045+0000 7f0768f39640 1 -- 172.21.15.81:0/122416095 wait complete. 2023-11-08T16:32:26.065 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:32:26.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.273+0000 7fcaa1bd7640 1 Processor -- start 2023-11-08T16:32:26.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.273+0000 7fcaa1bd7640 1 -- start start 2023-11-08T16:32:26.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.275+0000 7fcaa1bd7640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fca9c15d2b0 0x7fca9c161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.275+0000 7fcaa1bd7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9c161c50 con 0x7fca9c0c9060 2023-11-08T16:32:26.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.275+0000 7fcaa1bd7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca9c06b980 con 0x7fca9c0ccf00 2023-11-08T16:32:26.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.275+0000 7fcaa1bd7640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca9c15d690 con 0x7fca9c15d2b0 2023-11-08T16:32:26.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.275+0000 7fca92ffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fca9c0c9060 0x7fca9c0cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50314/0 (socket says 172.21.15.81:50314) 2023-11-08T16:32:26.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.275+0000 7fca92ffd640 1 -- 172.21.15.81:0/1125968732 learned_addr learned my addr 172.21.15.81:0/1125968732 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:26.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.275+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2226205995 0 0) 0x7fca84001110 con 0x7fca9c0c9060 2023-11-08T16:32:26.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.275+0000 7fca9b7fe640 1 --2- 172.21.15.81:0/1125968732 >> v2:172.21.15.93:3300/0 conn(0x7fca9c15d2b0 0x7fca9c161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.275+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca88003520 con 0x7fca9c0c9060 2023-11-08T16:32:26.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 246224722 0 0) 0x7fca88003520 con 0x7fca9c0c9060 2023-11-08T16:32:26.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 >> v2:172.21.15.93:3300/0 conn(0x7fca9c15d2b0 msgr2=0x7fca9c161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:26.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 --2- 172.21.15.81:0/1125968732 >> v2:172.21.15.93:3300/0 conn(0x7fca9c15d2b0 0x7fca9c161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 >> v1:172.21.15.93:6789/0 conn(0x7fca9c0ccf00 legacy=0x7fca9c06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c06bc60 con 0x7fca9c0c9060 2023-11-08T16:32:26.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fca84001800 con 0x7fca9c0c9060 2023-11-08T16:32:26.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 >> v1:172.21.15.81:6789/0 conn(0x7fca9c0c9060 legacy=0x7fca9c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 --2- 172.21.15.81:0/1125968732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca88003a50 0x7fca88003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 --2- 172.21.15.81:0/1125968732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca880047c0 0x7fca88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 --2- 172.21.15.81:0/1125968732 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca880051d0 0x7fca8800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c15d690 con 0x7fca88003a50 2023-11-08T16:32:26.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca8800c100 con 0x7fca880047c0 2023-11-08T16:32:26.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca8800c2c0 con 0x7fca880051d0 2023-11-08T16:32:26.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.276+0000 7fca9b7fe640 1 --2- 172.21.15.81:0/1125968732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca88003a50 0x7fca88003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.277+0000 7fca92ffd640 1 --2- 172.21.15.81:0/1125968732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca880047c0 0x7fca88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.277+0000 7fca9bfff640 1 --2- 172.21.15.81:0/1125968732 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca880051d0 0x7fca8800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.277+0000 7fca9b7fe640 1 -- 172.21.15.81:0/1125968732 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca880051d0 msgr2=0x7fca8800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:26.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.277+0000 7fca9b7fe640 1 --2- 172.21.15.81:0/1125968732 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca880051d0 0x7fca8800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.277+0000 7fca9b7fe640 1 -- 172.21.15.81:0/1125968732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca880047c0 msgr2=0x7fca88004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:26.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.277+0000 7fca9b7fe640 1 --2- 172.21.15.81:0/1125968732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca880047c0 0x7fca88004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.277+0000 7fca9b7fe640 1 -- 172.21.15.81:0/1125968732 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca88003520 con 0x7fca88003a50 2023-11-08T16:32:26.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.277+0000 7fca9bfff640 1 -- 172.21.15.81:0/1125968732 reap_dead start 2023-11-08T16:32:26.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.278+0000 7fca9b7fe640 1 --2- 172.21.15.81:0/1125968732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca88003a50 0x7fca88003e70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009a90 tx=0x7fca880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57231e6ca8b9f27d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:26.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.278+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c02c680 con 0x7fca88003a50 2023-11-08T16:32:26.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.278+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8c02c7e0 con 0x7fca88003a50 2023-11-08T16:32:26.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.278+0000 7fca9affd640 1 -- 172.21.15.81:0/1125968732 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca8c02cb70 con 0x7fca88003a50 2023-11-08T16:32:26.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.279+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/1125968732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca88003a50 msgr2=0x7fca88003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.279+0000 7fcaa1bd7640 1 --2- 172.21.15.81:0/1125968732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca88003a50 0x7fca88003e70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fca8c009a90 tx=0x7fca880057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.279+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/1125968732 shutdown_connections 2023-11-08T16:32:26.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.279+0000 7fcaa1bd7640 1 --2- 172.21.15.81:0/1125968732 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca88003a50 0x7fca88003e70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.279+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/1125968732 >> 172.21.15.81:0/1125968732 conn(0x7fca9c0cb490 msgr2=0x7fca9c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:26.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.279+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/1125968732 shutdown_connections 2023-11-08T16:32:26.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.279+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/1125968732 wait complete. 2023-11-08T16:32:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.281+0000 7fcaa1bd7640 1 Processor -- start 2023-11-08T16:32:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.281+0000 7fcaa1bd7640 1 -- start start 2023-11-08T16:32:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.282+0000 7fcaa1bd7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca88003a50 0x7fca9c1f6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.282+0000 7fcaa1bd7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca9c0ca8b0 0x7fca9c14a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.282+0000 7fcaa1bd7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca9c14aac0 0x7fca9c156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.282+0000 7fcaa1bd7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c0b16b0 con 0x7fca9c14aac0 2023-11-08T16:32:26.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.282+0000 7fcaa1bd7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c1f6910 con 0x7fca9c0ca8b0 2023-11-08T16:32:26.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.282+0000 7fcaa1bd7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9c1f6bf0 con 0x7fca88003a50 2023-11-08T16:32:26.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.282+0000 7fca9bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca9c14aac0 0x7fca9c156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.282+0000 7fca9bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca9c14aac0 0x7fca9c156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34186/0 (socket says 172.21.15.81:34186) 2023-11-08T16:32:26.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.282+0000 7fca9bfff640 1 -- 172.21.15.81:0/2031344544 learned_addr learned my addr 172.21.15.81:0/2031344544 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:26.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca92ffd640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca9c0ca8b0 0x7fca9c14a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca9b7fe640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca88003a50 0x7fca9c1f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca9bfff640 1 -- 172.21.15.81:0/2031344544 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca88003a50 msgr2=0x7fca9c1f6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca9bfff640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca88003a50 0x7fca9c1f6330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca9bfff640 1 -- 172.21.15.81:0/2031344544 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca9c0ca8b0 msgr2=0x7fca9c14a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca9bfff640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca9c0ca8b0 0x7fca9c14a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca9bfff640 1 -- 172.21.15.81:0/2031344544 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca9c1f6ed0 con 0x7fca9c14aac0 2023-11-08T16:32:26.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca92ffd640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca9c0ca8b0 0x7fca9c14a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:26.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca9bfff640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca9c14aac0 0x7fca9c156560 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0ce0f0 tx=0x7fca9400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:26.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca98ff9640 1 -- 172.21.15.81:0/2031344544 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94003b70 con 0x7fca9c14aac0 2023-11-08T16:32:26.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca98ff9640 1 -- 172.21.15.81:0/2031344544 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca94003cf0 con 0x7fca9c14aac0 2023-11-08T16:32:26.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/2031344544 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca9c1f71b0 con 0x7fca9c14aac0 2023-11-08T16:32:26.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fca98ff9640 1 -- 172.21.15.81:0/2031344544 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94003ea0 con 0x7fca9c14aac0 2023-11-08T16:32:26.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.283+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/2031344544 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca9c156aa0 con 0x7fca9c14aac0 2023-11-08T16:32:26.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.284+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/2031344544 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca9c0c5930 con 0x7fca9c14aac0 2023-11-08T16:32:26.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.285+0000 7fca98ff9640 1 -- 172.21.15.81:0/2031344544 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fca940052b0 con 0x7fca9c14aac0 2023-11-08T16:32:26.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.286+0000 7fca98ff9640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca7c06ae70 0x7fca7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.286+0000 7fca98ff9640 1 -- 172.21.15.81:0/2031344544 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7fca940891b0 con 0x7fca9c14aac0 2023-11-08T16:32:26.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.287+0000 7fca9b7fe640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca7c06ae70 0x7fca7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.287+0000 7fca9b7fe640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca7c06ae70 0x7fca7c06d330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fca8c036000 tx=0x7fca8c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:26.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.290+0000 7fca98ff9640 1 -- 172.21.15.81:0/2031344544 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fca94053fb0 con 0x7fca9c14aac0 2023-11-08T16:32:26.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.517+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/2031344544 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fca9c06d550 con 0x7fca9c14aac0 2023-11-08T16:32:26.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.518+0000 7fca98ff9640 1 -- 172.21.15.81:0/2031344544 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4910 (secure 0 0 0) 0x7fca940575d0 con 0x7fca9c14aac0 2023-11-08T16:32:26.521 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:26.521 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3654712142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3654712142},{"type":"v1","addr":"172.21.15.93:6834","nonce":3654712142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5014,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/66363251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":66363251},{"type":"v1","addr":"172.21.15.93:6835","nonce":66363251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4031527046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4031527046},{"type":"v1","addr":"172.21.15.81:6835","nonce":4031527046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:25.862327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/2942618483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2942618483},{"type":"v1","addr":"172.21.15.81:6837","nonce":2942618483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:32:26.521 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 32 2023-11-08T16:32:26.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.521+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/2031344544 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca7c06ae70 msgr2=0x7fca7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.521+0000 7fcaa1bd7640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca7c06ae70 0x7fca7c06d330 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fca8c036000 tx=0x7fca8c030000 comp rx=0 tx=0).stop 2023-11-08T16:32:26.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.521+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/2031344544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca9c14aac0 msgr2=0x7fca9c156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.521+0000 7fcaa1bd7640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca9c14aac0 0x7fca9c156560 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fca9c0ce0f0 tx=0x7fca9400dc00 comp rx=0 tx=0).stop 2023-11-08T16:32:26.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.521+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/2031344544 shutdown_connections 2023-11-08T16:32:26.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.521+0000 7fcaa1bd7640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fca7c06ae70 0x7fca7c06d330 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.521+0000 7fcaa1bd7640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fca9c14aac0 0x7fca9c156560 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.521+0000 7fcaa1bd7640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fca9c0ca8b0 0x7fca9c14a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.521+0000 7fcaa1bd7640 1 --2- 172.21.15.81:0/2031344544 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fca88003a50 0x7fca9c1f6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.521+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/2031344544 >> 172.21.15.81:0/2031344544 conn(0x7fca9c0cb490 msgr2=0x7fca9c0ae540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:26.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.522+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/2031344544 shutdown_connections 2023-11-08T16:32:26.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.522+0000 7fcaa1bd7640 1 -- 172.21.15.81:0/2031344544 wait complete. 2023-11-08T16:32:26.542 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:32:26.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.759+0000 7f33e359e640 1 Processor -- start 2023-11-08T16:32:26.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.759+0000 7f33e359e640 1 -- start start 2023-11-08T16:32:26.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.760+0000 7f33e359e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415d490 0x7f33e40d39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.761+0000 7f33e359e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e40ce890 con 0x7f33e40d1080 2023-11-08T16:32:26.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.761+0000 7f33e359e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e40ceb70 con 0x7f33e40d1bf0 2023-11-08T16:32:26.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.761+0000 7f33e359e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40cee50 con 0x7f33e415d490 2023-11-08T16:32:26.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.761+0000 7f33e1d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f33e40d1080 0x7f33e40d14e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50324/0 (socket says 172.21.15.81:50324) 2023-11-08T16:32:26.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.761+0000 7f33e1d9b640 1 -- 172.21.15.81:0/3698092509 learned_addr learned my addr 172.21.15.81:0/3698092509 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:26.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.761+0000 7f33e259c640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415d490 0x7f33e40d39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.761+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235373833 0 0) 0x7f33e40ce890 con 0x7f33e40d1080 2023-11-08T16:32:26.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.761+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33d0003520 con 0x7f33e40d1080 2023-11-08T16:32:26.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1215432275 0 0) 0x7f33d0003520 con 0x7f33e40d1080 2023-11-08T16:32:26.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415d490 msgr2=0x7f33e40d39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:26.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415d490 0x7f33e40d39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 >> v1:172.21.15.93:6789/0 conn(0x7f33e40d1bf0 legacy=0x7f33e40b5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e40cf130 con 0x7f33e40d1080 2023-11-08T16:32:26.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f33cc001670 con 0x7f33e40d1080 2023-11-08T16:32:26.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 >> v1:172.21.15.81:6789/0 conn(0x7f33e40d1080 legacy=0x7f33e40d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415d490 0x7f33d0003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d0004000 0x7f33d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33d0004d00 0x7f33d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40cee50 con 0x7f33d0004000 2023-11-08T16:32:26.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d000bb70 con 0x7f33d0004d00 2023-11-08T16:32:26.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d000bd30 con 0x7f33e415d490 2023-11-08T16:32:26.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e1d9b640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d0004000 0x7f33d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e2d9d640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33d0004d00 0x7f33d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e259c640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415d490 0x7f33d0003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.762+0000 7f33e1d9b640 1 -- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415d490 msgr2=0x7f33d0003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:26.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.763+0000 7f33e1d9b640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415d490 0x7f33d0003a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.763+0000 7f33e1d9b640 1 -- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33d0004d00 msgr2=0x7f33d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:26.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.763+0000 7f33e1d9b640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33d0004d00 0x7f33d0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.763+0000 7f33e1d9b640 1 -- 172.21.15.81:0/3698092509 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d0003520 con 0x7f33d0004000 2023-11-08T16:32:26.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.763+0000 7f33e2d9d640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33d0004d00 0x7f33d0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:26.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.764+0000 7f33e1d9b640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d0004000 0x7f33d00047c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00c240 tx=0x7f33d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdda1699fa9e2672 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:26.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.764+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc005ad0 con 0x7f33d0004000 2023-11-08T16:32:26.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.764+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc005c30 con 0x7f33d0004000 2023-11-08T16:32:26.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.764+0000 7f33e159a640 1 -- 172.21.15.81:0/3698092509 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc02caf0 con 0x7f33d0004000 2023-11-08T16:32:26.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.764+0000 7f33e359e640 1 -- 172.21.15.81:0/3698092509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d0004000 msgr2=0x7f33d00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.764+0000 7f33e359e640 1 --2- 172.21.15.81:0/3698092509 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33d0004000 0x7f33d00047c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f33cc00c240 tx=0x7f33d00057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.764+0000 7f33e2d9d640 1 -- 172.21.15.81:0/3698092509 reap_dead start 2023-11-08T16:32:26.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.765+0000 7f33e359e640 1 -- 172.21.15.81:0/3698092509 shutdown_connections 2023-11-08T16:32:26.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.765+0000 7f33e359e640 1 -- 172.21.15.81:0/3698092509 >> 172.21.15.81:0/3698092509 conn(0x7f33e40d2e50 msgr2=0x7f33e40d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:26.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.765+0000 7f33e359e640 1 -- 172.21.15.81:0/3698092509 shutdown_connections 2023-11-08T16:32:26.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.765+0000 7f33e359e640 1 -- 172.21.15.81:0/3698092509 wait complete. 2023-11-08T16:32:26.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.766+0000 7f33e359e640 1 Processor -- start 2023-11-08T16:32:26.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.767+0000 7f33e359e640 1 -- start start 2023-11-08T16:32:26.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.767+0000 7f33e359e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33d0004000 0x7f33e41f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.767+0000 7f33e359e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415f580 0x7f33e41555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.767+0000 7f33e359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33e40cb200 0x7f33e415a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.767+0000 7f33e359e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e41f6570 con 0x7f33e40cb200 2023-11-08T16:32:26.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.767+0000 7f33e359e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e4155ea0 con 0x7f33d0004000 2023-11-08T16:32:26.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.767+0000 7f33e359e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40cb970 con 0x7f33e415f580 2023-11-08T16:32:26.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.768+0000 7f33e2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33e40cb200 0x7f33e415a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.768+0000 7f33e2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33e40cb200 0x7f33e415a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34212/0 (socket says 172.21.15.81:34212) 2023-11-08T16:32:26.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.768+0000 7f33e2d9d640 1 -- 172.21.15.81:0/936096686 learned_addr learned my addr 172.21.15.81:0/936096686 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:26.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.768+0000 7f33e259c640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33d0004000 0x7f33e41f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.768+0000 7f33e1d9b640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415f580 0x7f33e41555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.768+0000 7f33e2d9d640 1 -- 172.21.15.81:0/936096686 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415f580 msgr2=0x7f33e41555f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.768+0000 7f33e2d9d640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415f580 0x7f33e41555f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.768+0000 7f33e2d9d640 1 -- 172.21.15.81:0/936096686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33d0004000 msgr2=0x7f33e41f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.768+0000 7f33e2d9d640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33d0004000 0x7f33e41f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.768+0000 7f33e2d9d640 1 -- 172.21.15.81:0/936096686 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e41f5eb0 con 0x7f33e40cb200 2023-11-08T16:32:26.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.769+0000 7f33e2d9d640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33e40cb200 0x7f33e415a670 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f33e408bab0 tx=0x7f33d400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:26.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.769+0000 7f33daffd640 1 -- 172.21.15.81:0/936096686 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d40052d0 con 0x7f33e40cb200 2023-11-08T16:32:26.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.769+0000 7f33daffd640 1 -- 172.21.15.81:0/936096686 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d4005450 con 0x7f33e40cb200 2023-11-08T16:32:26.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.769+0000 7f33e359e640 1 -- 172.21.15.81:0/936096686 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e41f6190 con 0x7f33e40cb200 2023-11-08T16:32:26.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.769+0000 7f33daffd640 1 -- 172.21.15.81:0/936096686 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d4007480 con 0x7f33e40cb200 2023-11-08T16:32:26.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.769+0000 7f33e359e640 1 -- 172.21.15.81:0/936096686 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e40cbc50 con 0x7f33e40cb200 2023-11-08T16:32:26.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.770+0000 7f33e359e640 1 -- 172.21.15.81:0/936096686 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e4067800 con 0x7f33e40cb200 2023-11-08T16:32:26.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.771+0000 7f33daffd640 1 -- 172.21.15.81:0/936096686 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33d40067d0 con 0x7f33e40cb200 2023-11-08T16:32:26.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.773+0000 7f33daffd640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33c406ae70 0x7f33c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:26.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.773+0000 7f33daffd640 1 -- 172.21.15.81:0/936096686 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f33d40887a0 con 0x7f33e40cb200 2023-11-08T16:32:26.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.773+0000 7f33e259c640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33c406ae70 0x7f33c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:26.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.774+0000 7f33e259c640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33c406ae70 0x7f33c406d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0074a0 tx=0x7f33dc00d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:26.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.777+0000 7f33daffd640 1 -- 172.21.15.81:0/936096686 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33d4053640 con 0x7f33e40cb200 2023-11-08T16:32:26.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.970+0000 7f33e359e640 1 -- 172.21.15.81:0/936096686 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33e40b8280 con 0x7f33e40cb200 2023-11-08T16:32:26.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.972+0000 7f33daffd640 1 -- 172.21.15.81:0/936096686 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18684 (secure 0 0 0) 0x7f33d4056c60 con 0x7f33e40cb200 2023-11-08T16:32:26.974 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:26.974 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":44,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:31:24.833641+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:30:27.977481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T16:30:29.171902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:30:40.141171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/1759199271":"2023-11-09T16:31:23.834479+0000","172.21.15.81:6836/1759199271":"2023-11-09T16:31:23.834479+0000","172.21.15.81:6837/3931731986":"2023-11-09T16:31:16.385652+0000","172.21.15.81:6836/3931731986":"2023-11-09T16:31:16.385652+0000","172.21.15.81:6837/773130363":"2023-11-09T16:31:14.352977+0000","172.21.15.81:6836/773130363":"2023-11-09T16:31:14.352977+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:32:26.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.974+0000 7f33e359e640 1 -- 172.21.15.81:0/936096686 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33c406ae70 msgr2=0x7f33c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.974+0000 7f33e359e640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33c406ae70 0x7f33c406d330 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0074a0 tx=0x7f33dc00d040 comp rx=0 tx=0).stop 2023-11-08T16:32:26.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.974+0000 7f33e359e640 1 -- 172.21.15.81:0/936096686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33e40cb200 msgr2=0x7f33e415a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:26.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.974+0000 7f33e359e640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33e40cb200 0x7f33e415a670 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f33e408bab0 tx=0x7f33d400d890 comp rx=0 tx=0).stop 2023-11-08T16:32:26.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.975+0000 7f33e359e640 1 -- 172.21.15.81:0/936096686 shutdown_connections 2023-11-08T16:32:26.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.975+0000 7f33e359e640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33c406ae70 0x7f33c406d330 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.975+0000 7f33e359e640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33e40cb200 0x7f33e415a670 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.975+0000 7f33e359e640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33e415f580 0x7f33e41555f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.975+0000 7f33e359e640 1 --2- 172.21.15.81:0/936096686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33d0004000 0x7f33e41f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:26.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.975+0000 7f33e359e640 1 -- 172.21.15.81:0/936096686 >> 172.21.15.81:0/936096686 conn(0x7f33e40d2e50 msgr2=0x7f33e406be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:26.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.975+0000 7f33e359e640 1 -- 172.21.15.81:0/936096686 shutdown_connections 2023-11-08T16:32:26.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:26.975+0000 7f33e359e640 1 -- 172.21.15.81:0/936096686 wait complete. 2023-11-08T16:32:26.994 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T16:32:26.995 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:32:27.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.205+0000 7fa2374c9640 1 Processor -- start 2023-11-08T16:32:27.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.205+0000 7fa2374c9640 1 -- start start 2023-11-08T16:32:27.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.206+0000 7fa2374c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2300c9ae0 0x7fa23006f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.206+0000 7fa2374c9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa23006bec0 0x7fa23006c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.206+0000 7fa2374c9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2300c9ec0 con 0x7fa23006cce0 2023-11-08T16:32:27.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.206+0000 7fa2374c9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23006c820 con 0x7fa23006bec0 2023-11-08T16:32:27.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.206+0000 7fa2374c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2300cc710 con 0x7fa2300c9ae0 2023-11-08T16:32:27.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.207+0000 7fa235265640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2300c9ae0 0x7fa23006f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:27.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.207+0000 7fa234a64640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa23006bec0 0x7fa23006c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:27.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.207+0000 7fa235265640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2300c9ae0 0x7fa23006f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:49400/0 (socket says 172.21.15.81:49400) 2023-11-08T16:32:27.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.207+0000 7fa234a64640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa23006bec0 0x7fa23006c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:52100/0 (socket says 172.21.15.81:52100) 2023-11-08T16:32:27.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.207+0000 7fa235265640 1 -- 172.21.15.81:0/1080848089 learned_addr learned my addr 172.21.15.81:0/1080848089 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:27.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.208+0000 7fa234a64640 1 -- 172.21.15.81:0/1080848089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2300c9ae0 msgr2=0x7fa23006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:27.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.208+0000 7fa234a64640 1 --2- 172.21.15.81:0/1080848089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2300c9ae0 0x7fa23006f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.208+0000 7fa234a64640 1 -- 172.21.15.81:0/1080848089 >> v1:172.21.15.93:6789/0 conn(0x7fa23006cce0 legacy=0x7fa23006d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.208+0000 7fa234a64640 1 -- 172.21.15.81:0/1080848089 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2300cc9f0 con 0x7fa23006bec0 2023-11-08T16:32:27.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.208+0000 7fa235265640 1 --2- 172.21.15.81:0/1080848089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2300c9ae0 0x7fa23006f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:32:27.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.210+0000 7fa234a64640 1 --2- 172.21.15.81:0/1080848089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa23006bec0 0x7fa23006c2e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa2200097a0 tx=0x7fa22002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7db9af345c564be2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:27.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.210+0000 7fa227fff640 1 -- 172.21.15.81:0/1080848089 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220031780 con 0x7fa23006bec0 2023-11-08T16:32:27.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.210+0000 7fa227fff640 1 -- 172.21.15.81:0/1080848089 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2200318e0 con 0x7fa23006bec0 2023-11-08T16:32:27.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.210+0000 7fa227fff640 1 -- 172.21.15.81:0/1080848089 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220031c70 con 0x7fa23006bec0 2023-11-08T16:32:27.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.211+0000 7fa2374c9640 1 -- 172.21.15.81:0/1080848089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa23006bec0 msgr2=0x7fa23006c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.211+0000 7fa2374c9640 1 --2- 172.21.15.81:0/1080848089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa23006bec0 0x7fa23006c2e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa2200097a0 tx=0x7fa22002b820 comp rx=0 tx=0).stop 2023-11-08T16:32:27.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.211+0000 7fa2374c9640 1 -- 172.21.15.81:0/1080848089 shutdown_connections 2023-11-08T16:32:27.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.211+0000 7fa2374c9640 1 --2- 172.21.15.81:0/1080848089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa23006bec0 0x7fa23006c2e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.211+0000 7fa2374c9640 1 --2- 172.21.15.81:0/1080848089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2300c9ae0 0x7fa23006f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.211+0000 7fa2374c9640 1 -- 172.21.15.81:0/1080848089 >> 172.21.15.81:0/1080848089 conn(0x7fa2300cb490 msgr2=0x7fa2300cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:27.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.211+0000 7fa2374c9640 1 -- 172.21.15.81:0/1080848089 shutdown_connections 2023-11-08T16:32:27.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.211+0000 7fa2374c9640 1 -- 172.21.15.81:0/1080848089 wait complete. 2023-11-08T16:32:27.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.213+0000 7fa2374c9640 1 Processor -- start 2023-11-08T16:32:27.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.213+0000 7fa2374c9640 1 -- start start 2023-11-08T16:32:27.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa2374c9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa23006bec0 0x7fa2300b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa2374c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa23006cce0 0x7fa2301579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa2374c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2300c9ae0 0x7fa2300b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa2374c9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2301e5240 con 0x7fa23006cce0 2023-11-08T16:32:27.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa2374c9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2301e5520 con 0x7fa23006bec0 2023-11-08T16:32:27.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa2374c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2300c5ae0 con 0x7fa2300c9ae0 2023-11-08T16:32:27.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa234a64640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa23006cce0 0x7fa2301579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:27.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa234a64640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa23006cce0 0x7fa2301579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34222/0 (socket says 172.21.15.81:34222) 2023-11-08T16:32:27.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa234a64640 1 -- 172.21.15.81:0/4293032448 learned_addr learned my addr 172.21.15.81:0/4293032448 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:27.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa235265640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa23006bec0 0x7fa2300b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:27.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa234a64640 1 -- 172.21.15.81:0/4293032448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2300c9ae0 msgr2=0x7fa2300b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa234a64640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2300c9ae0 0x7fa2300b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa234a64640 1 -- 172.21.15.81:0/4293032448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa23006bec0 msgr2=0x7fa2300b4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa234a64640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa23006bec0 0x7fa2300b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.214+0000 7fa234a64640 1 -- 172.21.15.81:0/4293032448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c00a670 con 0x7fa23006cce0 2023-11-08T16:32:27.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.215+0000 7fa234a64640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa23006cce0 0x7fa2301579b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa2200097a0 tx=0x7fa2200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:27.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.215+0000 7fa225ffb640 1 -- 172.21.15.81:0/4293032448 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220032820 con 0x7fa23006cce0 2023-11-08T16:32:27.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.215+0000 7fa225ffb640 1 -- 172.21.15.81:0/4293032448 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa220032980 con 0x7fa23006cce0 2023-11-08T16:32:27.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.215+0000 7fa225ffb640 1 -- 172.21.15.81:0/4293032448 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa220032d30 con 0x7fa23006cce0 2023-11-08T16:32:27.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.215+0000 7fa2374c9640 1 -- 172.21.15.81:0/4293032448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa220009400 con 0x7fa23006cce0 2023-11-08T16:32:27.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.215+0000 7fa2374c9640 1 -- 172.21.15.81:0/4293032448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2300c60a0 con 0x7fa23006cce0 2023-11-08T16:32:27.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.216+0000 7fa2374c9640 1 -- 172.21.15.81:0/4293032448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa230071bd0 con 0x7fa23006cce0 2023-11-08T16:32:27.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.217+0000 7fa225ffb640 1 -- 172.21.15.81:0/4293032448 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa22004d3f0 con 0x7fa23006cce0 2023-11-08T16:32:27.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.218+0000 7fa225ffb640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa21406ae70 0x7fa21406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.218+0000 7fa225ffb640 1 -- 172.21.15.81:0/4293032448 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7fa2200ab590 con 0x7fa23006cce0 2023-11-08T16:32:27.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.218+0000 7fa235265640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa21406ae70 0x7fa21406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:27.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.219+0000 7fa235265640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa21406ae70 0x7fa21406d330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa2301f48b0 tx=0x7fa22c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:27.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.222+0000 7fa225ffb640 1 -- 172.21.15.81:0/4293032448 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa220076390 con 0x7fa23006cce0 2023-11-08T16:32:27.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.447+0000 7fa2374c9640 1 -- 172.21.15.81:0/4293032448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2300c6290 con 0x7fa23006cce0 2023-11-08T16:32:27.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.448+0000 7fa225ffb640 1 -- 172.21.15.81:0/4293032448 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4910 (secure 0 0 0) 0x7fa2200799b0 con 0x7fa23006cce0 2023-11-08T16:32:27.450 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:27.451 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3654712142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3654712142},{"type":"v1","addr":"172.21.15.93:6834","nonce":3654712142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5014,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/66363251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":66363251},{"type":"v1","addr":"172.21.15.93:6835","nonce":66363251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4031527046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4031527046},{"type":"v1","addr":"172.21.15.81:6835","nonce":4031527046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:25.862327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/2942618483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2942618483},{"type":"v1","addr":"172.21.15.81:6837","nonce":2942618483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:32:27.451 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 32 2023-11-08T16:32:27.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 -- 172.21.15.81:0/4293032448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa21406ae70 msgr2=0x7fa21406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa21406ae70 0x7fa21406d330 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa2301f48b0 tx=0x7fa22c008040 comp rx=0 tx=0).stop 2023-11-08T16:32:27.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 -- 172.21.15.81:0/4293032448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa23006cce0 msgr2=0x7fa2301579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa23006cce0 0x7fa2301579b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fa2200097a0 tx=0x7fa2200046c0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 -- 172.21.15.81:0/4293032448 shutdown_connections 2023-11-08T16:32:27.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa21406ae70 0x7fa21406d330 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa2300c9ae0 0x7fa2300b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa23006cce0 0x7fa2301579b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 --2- 172.21.15.81:0/4293032448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa23006bec0 0x7fa2300b4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 -- 172.21.15.81:0/4293032448 >> 172.21.15.81:0/4293032448 conn(0x7fa2300cb490 msgr2=0x7fa230158cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:27.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 -- 172.21.15.81:0/4293032448 shutdown_connections 2023-11-08T16:32:27.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.451+0000 7fa2374c9640 1 -- 172.21.15.81:0/4293032448 wait complete. 2023-11-08T16:32:27.472 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:32:27.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.679+0000 7f05c0eec640 1 Processor -- start 2023-11-08T16:32:27.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.679+0000 7f05c0eec640 1 -- start start 2023-11-08T16:32:27.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.681+0000 7f05c0eec640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f05bc06bdf0 0x7f05bc06c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.681+0000 7f05c0eec640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc06b630 con 0x7f05bc0ca480 2023-11-08T16:32:27.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.681+0000 7f05c0eec640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc0ca910 con 0x7f05bc0ade50 2023-11-08T16:32:27.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.681+0000 7f05c0eec640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05bc0b1860 con 0x7f05bc06bdf0 2023-11-08T16:32:27.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.681+0000 7f05b3fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f05bc0ca480 0x7f05bc15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50330/0 (socket says 172.21.15.81:50330) 2023-11-08T16:32:27.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.681+0000 7f05b3fff640 1 -- 172.21.15.81:0/2142788793 learned_addr learned my addr 172.21.15.81:0/2142788793 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:27.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05bad9d640 1 --2- 172.21.15.81:0/2142788793 >> v2:172.21.15.93:3300/0 conn(0x7f05bc06bdf0 0x7f05bc06c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:27.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3751729447 0 0) 0x7f05bc06b630 con 0x7f05bc0ca480 2023-11-08T16:32:27.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a8003520 con 0x7f05bc0ca480 2023-11-08T16:32:27.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3609236341 0 0) 0x7f05a8003520 con 0x7f05bc0ca480 2023-11-08T16:32:27.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 >> v2:172.21.15.93:3300/0 conn(0x7f05bc06bdf0 msgr2=0x7f05bc06c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:27.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 --2- 172.21.15.81:0/2142788793 >> v2:172.21.15.93:3300/0 conn(0x7f05bc06bdf0 0x7f05bc06c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 >> v1:172.21.15.93:6789/0 conn(0x7f05bc0ade50 legacy=0x7f05bc15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc0cabb0 con 0x7f05bc0ca480 2023-11-08T16:32:27.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f05a4001670 con 0x7f05bc0ca480 2023-11-08T16:32:27.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 >> v1:172.21.15.81:6789/0 conn(0x7f05bc0ca480 legacy=0x7f05bc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05a8003a50 0x7f05a8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05a80047c0 0x7f05a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 0x7f05a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a800c0d0 con 0x7f05a80051d0 2023-11-08T16:32:27.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a800c240 con 0x7f05a80047c0 2023-11-08T16:32:27.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.682+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a800c400 con 0x7f05a8003a50 2023-11-08T16:32:27.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.683+0000 7f05bad9d640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 0x7f05a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:27.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.683+0000 7f05b3fff640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05a80047c0 0x7f05a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:27.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.683+0000 7f05ba59c640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05a8003a50 0x7f05a8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:27.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.683+0000 7f05bad9d640 1 -- 172.21.15.81:0/2142788793 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05a8003a50 msgr2=0x7f05a8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:27.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.683+0000 7f05bad9d640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05a8003a50 0x7f05a8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.683+0000 7f05bad9d640 1 -- 172.21.15.81:0/2142788793 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05a80047c0 msgr2=0x7f05a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:27.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.683+0000 7f05bad9d640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05a80047c0 0x7f05a8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.683+0000 7f05bad9d640 1 -- 172.21.15.81:0/2142788793 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a8003520 con 0x7f05a80051d0 2023-11-08T16:32:27.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.683+0000 7f05b3fff640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05a80047c0 0x7f05a8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:27.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.683+0000 7f05ba59c640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05a8003a50 0x7f05a8003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:27.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.684+0000 7f05bad9d640 1 -- 172.21.15.81:0/2142788793 reap_dead start 2023-11-08T16:32:27.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.684+0000 7f05bad9d640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 0x7f05a800b980 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009f90 tx=0x7f05a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb72a557d9c3a77f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:27.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.684+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b402cc20 con 0x7f05a80051d0 2023-11-08T16:32:27.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.684+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b400f040 con 0x7f05a80051d0 2023-11-08T16:32:27.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.684+0000 7f05b9d9b640 1 -- 172.21.15.81:0/2142788793 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b4003990 con 0x7f05a80051d0 2023-11-08T16:32:27.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.685+0000 7f05c0eec640 1 -- 172.21.15.81:0/2142788793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 msgr2=0x7f05a800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.685+0000 7f05c0eec640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 0x7f05a800b980 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f05b4009f90 tx=0x7f05a80057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.685+0000 7f05c0eec640 1 -- 172.21.15.81:0/2142788793 shutdown_connections 2023-11-08T16:32:27.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.685+0000 7f05c0eec640 1 --2- 172.21.15.81:0/2142788793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 0x7f05a800b980 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.685+0000 7f05c0eec640 1 -- 172.21.15.81:0/2142788793 >> 172.21.15.81:0/2142788793 conn(0x7f05bc0cb490 msgr2=0x7f05bc0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:27.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.685+0000 7f05c0eec640 1 -- 172.21.15.81:0/2142788793 shutdown_connections 2023-11-08T16:32:27.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.685+0000 7f05c0eec640 1 -- 172.21.15.81:0/2142788793 wait complete. 2023-11-08T16:32:27.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.687+0000 7f05c0eec640 1 Processor -- start 2023-11-08T16:32:27.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.687+0000 7f05c0eec640 1 -- start start 2023-11-08T16:32:27.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05c0eec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 0x7f05bc1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05c0eec640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05bc1f2400 0x7f05bc0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05c0eec640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05bc0acdf0 0x7f05bc142600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05c0eec640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0ad2a0 con 0x7f05a80051d0 2023-11-08T16:32:27.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05c0eec640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc15be50 con 0x7f05bc0acdf0 2023-11-08T16:32:27.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05c0eec640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc1566e0 con 0x7f05bc1f2400 2023-11-08T16:32:27.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05ba59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 0x7f05bc1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:27.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05ba59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 0x7f05bc1571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34226/0 (socket says 172.21.15.81:34226) 2023-11-08T16:32:27.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05ba59c640 1 -- 172.21.15.81:0/1615330442 learned_addr learned my addr 172.21.15.81:0/1615330442 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:27.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05ba59c640 1 -- 172.21.15.81:0/1615330442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05bc1f2400 msgr2=0x7f05bc0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05ba59c640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05bc1f2400 0x7f05bc0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05ba59c640 1 -- 172.21.15.81:0/1615330442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05bc0acdf0 msgr2=0x7f05bc142600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05ba59c640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05bc0acdf0 0x7f05bc142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.688+0000 7f05ba59c640 1 -- 172.21.15.81:0/1615330442 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc1569c0 con 0x7f05a80051d0 2023-11-08T16:32:27.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.689+0000 7f05ba59c640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 0x7f05bc1571c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000f60 tx=0x7f05ac00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:27.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.689+0000 7f05b2ffd640 1 -- 172.21.15.81:0/1615330442 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac005480 con 0x7f05a80051d0 2023-11-08T16:32:27.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.689+0000 7f05b2ffd640 1 -- 172.21.15.81:0/1615330442 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac003610 con 0x7f05a80051d0 2023-11-08T16:32:27.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.689+0000 7f05b2ffd640 1 -- 172.21.15.81:0/1615330442 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0039c0 con 0x7f05a80051d0 2023-11-08T16:32:27.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.689+0000 7f05c0eec640 1 -- 172.21.15.81:0/1615330442 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc156ca0 con 0x7f05a80051d0 2023-11-08T16:32:27.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.689+0000 7f05c0eec640 1 -- 172.21.15.81:0/1615330442 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc156300 con 0x7f05a80051d0 2023-11-08T16:32:27.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.690+0000 7f05c0eec640 1 -- 172.21.15.81:0/1615330442 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc156520 con 0x7f05a80051d0 2023-11-08T16:32:27.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.691+0000 7f05b2ffd640 1 -- 172.21.15.81:0/1615330442 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f05ac003b20 con 0x7f05a80051d0 2023-11-08T16:32:27.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.692+0000 7f05b2ffd640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f059c06aea0 0x7f059c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:27.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.692+0000 7f05b2ffd640 1 -- 172.21.15.81:0/1615330442 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f05ac089780 con 0x7f05a80051d0 2023-11-08T16:32:27.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.692+0000 7f05b3fff640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f059c06aea0 0x7f059c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:27.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.693+0000 7f05b3fff640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f059c06aea0 0x7f059c06d360 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f05a40017a0 tx=0x7f05a4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:27.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.696+0000 7f05b2ffd640 1 -- 172.21.15.81:0/1615330442 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f05ac054640 con 0x7f05a80051d0 2023-11-08T16:32:27.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.927+0000 7f05c0eec640 1 -- 172.21.15.81:0/1615330442 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05bc04ece0 con 0x7f05a80051d0 2023-11-08T16:32:27.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.928+0000 7f05b2ffd640 1 -- 172.21.15.81:0/1615330442 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f05ac001230 con 0x7f05a80051d0 2023-11-08T16:32:27.930 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:27.931 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":32,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3654712142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3654712142},{"type":"v1","addr":"172.21.15.93:6834","nonce":3654712142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5014,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/66363251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":66363251},{"type":"v1","addr":"172.21.15.93:6835","nonce":66363251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4031527046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4031527046},{"type":"v1","addr":"172.21.15.81:6835","nonce":4031527046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:31:15.514873+0000","modified":"2023-11-08T16:31:25.862327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":42,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5655},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5655":{"gid":5655,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/2942618483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2942618483},{"type":"v1","addr":"172.21.15.81:6837","nonce":2942618483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5,6],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-08T16:32:27.931 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 32 2023-11-08T16:32:27.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 -- 172.21.15.81:0/1615330442 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f059c06aea0 msgr2=0x7f059c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f059c06aea0 0x7f059c06d360 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f05a40017a0 tx=0x7f05a4009040 comp rx=0 tx=0).stop 2023-11-08T16:32:27.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 -- 172.21.15.81:0/1615330442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 msgr2=0x7f05bc1571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:27.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 0x7f05bc1571c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f05ac000f60 tx=0x7f05ac00d9f0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 -- 172.21.15.81:0/1615330442 shutdown_connections 2023-11-08T16:32:27.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f059c06aea0 0x7f059c06d360 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f05bc0acdf0 0x7f05bc142600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f05bc1f2400 0x7f05bc0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 --2- 172.21.15.81:0/1615330442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f05a80051d0 0x7f05bc1571c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:27.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 -- 172.21.15.81:0/1615330442 >> 172.21.15.81:0/1615330442 conn(0x7f05bc0cb490 msgr2=0x7f05bc0b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:27.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 -- 172.21.15.81:0/1615330442 shutdown_connections 2023-11-08T16:32:27.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:27.931+0000 7f05c0eec640 1 -- 172.21.15.81:0/1615330442 wait complete. 2023-11-08T16:32:27.951 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.158+0000 7f08fbf05640 1 Processor -- start 2023-11-08T16:32:28.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.158+0000 7f08fbf05640 1 -- start start 2023-11-08T16:32:28.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.159+0000 7f08fbf05640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f08f40adcd0 0x7f08f40ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08fbf05640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f08f40cc5e0 0x7f08f40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08fbf05640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f40ccfb0 0x7f08f415db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08fbf05640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f40c9a70 con 0x7f08f40adcd0 2023-11-08T16:32:28.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08fbf05640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f40cc3e0 con 0x7f08f40cc5e0 2023-11-08T16:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08fbf05640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f406d070 con 0x7f08f40ccfb0 2023-11-08T16:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08f9ca1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f08f40adcd0 0x7f08f40ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08f9ca1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f08f40adcd0 0x7f08f40ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34234/0 (socket says 172.21.15.81:34234) 2023-11-08T16:32:28.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08f9ca1640 1 -- 172.21.15.81:0/4289467823 learned_addr learned my addr 172.21.15.81:0/4289467823 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:28.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08f9ca1640 1 -- 172.21.15.81:0/4289467823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f40ccfb0 msgr2=0x7f08f415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:28.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08f94a0640 1 --2- 172.21.15.81:0/4289467823 >> v2:172.21.15.93:3300/0 conn(0x7f08f40cc5e0 0x7f08f40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08fa4a2640 1 --2- 172.21.15.81:0/4289467823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f40ccfb0 0x7f08f415db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08f9ca1640 1 --2- 172.21.15.81:0/4289467823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f40ccfb0 0x7f08f415db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08f9ca1640 1 -- 172.21.15.81:0/4289467823 >> v2:172.21.15.93:3300/0 conn(0x7f08f40cc5e0 msgr2=0x7f08f40cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:28.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08f9ca1640 1 --2- 172.21.15.81:0/4289467823 >> v2:172.21.15.93:3300/0 conn(0x7f08f40cc5e0 0x7f08f40cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.160+0000 7f08f9ca1640 1 -- 172.21.15.81:0/4289467823 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f406d350 con 0x7f08f40adcd0 2023-11-08T16:32:28.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.161+0000 7f08f94a0640 1 --2- 172.21.15.81:0/4289467823 >> v2:172.21.15.93:3300/0 conn(0x7f08f40cc5e0 0x7f08f40cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:28.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.161+0000 7f08f9ca1640 1 --2- 172.21.15.81:0/4289467823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f40adcd0 0x7f08f40ca1e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f08f40cc0d0 tx=0x7f08f002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b917f6fec47917e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:28.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.162+0000 7f08f8c9f640 1 -- 172.21.15.81:0/4289467823 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f002ca80 con 0x7f08f40adcd0 2023-11-08T16:32:28.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.162+0000 7f08f8c9f640 1 -- 172.21.15.81:0/4289467823 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f002cbe0 con 0x7f08f40adcd0 2023-11-08T16:32:28.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.162+0000 7f08f8c9f640 1 -- 172.21.15.81:0/4289467823 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f00343f0 con 0x7f08f40adcd0 2023-11-08T16:32:28.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.162+0000 7f08fbf05640 1 -- 172.21.15.81:0/4289467823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f40adcd0 msgr2=0x7f08f40ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:28.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.162+0000 7f08fbf05640 1 --2- 172.21.15.81:0/4289467823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f40adcd0 0x7f08f40ca1e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f08f40cc0d0 tx=0x7f08f002bee0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.163+0000 7f08fbf05640 1 -- 172.21.15.81:0/4289467823 shutdown_connections 2023-11-08T16:32:28.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.163+0000 7f08fbf05640 1 --2- 172.21.15.81:0/4289467823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f40ccfb0 0x7f08f415db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.163+0000 7f08fbf05640 1 --2- 172.21.15.81:0/4289467823 >> v2:172.21.15.93:3300/0 conn(0x7f08f40cc5e0 0x7f08f40cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.163+0000 7f08fbf05640 1 --2- 172.21.15.81:0/4289467823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f40adcd0 0x7f08f40ca1e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.163+0000 7f08fbf05640 1 -- 172.21.15.81:0/4289467823 >> 172.21.15.81:0/4289467823 conn(0x7f08f40cb490 msgr2=0x7f08f40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:28.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.163+0000 7f08fbf05640 1 -- 172.21.15.81:0/4289467823 shutdown_connections 2023-11-08T16:32:28.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.163+0000 7f08fbf05640 1 -- 172.21.15.81:0/4289467823 wait complete. 2023-11-08T16:32:28.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.164+0000 7f08fbf05640 1 Processor -- start 2023-11-08T16:32:28.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.164+0000 7f08fbf05640 1 -- start start 2023-11-08T16:32:28.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.165+0000 7f08fbf05640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08f40adcd0 0x7f08f4156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.165+0000 7f08fbf05640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f40cc5e0 0x7f08f41590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.165+0000 7f08fbf05640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f40ccfb0 0x7f08f415b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.165+0000 7f08fbf05640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4159a30 con 0x7f08f40cc5e0 2023-11-08T16:32:28.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.165+0000 7f08fbf05640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4159d10 con 0x7f08f40adcd0 2023-11-08T16:32:28.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.165+0000 7f08fbf05640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f4159ff0 con 0x7f08f40ccfb0 2023-11-08T16:32:28.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.166+0000 7f08f94a0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f40cc5e0 0x7f08f41590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.166+0000 7f08f94a0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f40cc5e0 0x7f08f41590c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34248/0 (socket says 172.21.15.81:34248) 2023-11-08T16:32:28.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.166+0000 7f08f94a0640 1 -- 172.21.15.81:0/2526142556 learned_addr learned my addr 172.21.15.81:0/2526142556 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:28.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.166+0000 7f08fa4a2640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f40ccfb0 0x7f08f415b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.166+0000 7f08f94a0640 1 -- 172.21.15.81:0/2526142556 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f40ccfb0 msgr2=0x7f08f415b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:28.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.166+0000 7f08f9ca1640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08f40adcd0 0x7f08f4156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.166+0000 7f08f94a0640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f40ccfb0 0x7f08f415b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.166+0000 7f08f94a0640 1 -- 172.21.15.81:0/2526142556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08f40adcd0 msgr2=0x7f08f4156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:28.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.166+0000 7f08f94a0640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08f40adcd0 0x7f08f4156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.166+0000 7f08f94a0640 1 -- 172.21.15.81:0/2526142556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f0009b30 con 0x7f08f40cc5e0 2023-11-08T16:32:28.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.166+0000 7f08f94a0640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f40cc5e0 0x7f08f41590c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f08e400c7f0 tx=0x7f08e400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:28.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.167+0000 7f08ea7fc640 1 -- 172.21.15.81:0/2526142556 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4007820 con 0x7f08f40cc5e0 2023-11-08T16:32:28.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.167+0000 7f08ea7fc640 1 -- 172.21.15.81:0/2526142556 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e4007980 con 0x7f08f40cc5e0 2023-11-08T16:32:28.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.167+0000 7f08ea7fc640 1 -- 172.21.15.81:0/2526142556 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4007d30 con 0x7f08f40cc5e0 2023-11-08T16:32:28.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.167+0000 7f08fbf05640 1 -- 172.21.15.81:0/2526142556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f415a2d0 con 0x7f08f40cc5e0 2023-11-08T16:32:28.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.167+0000 7f08fbf05640 1 -- 172.21.15.81:0/2526142556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f4156dd0 con 0x7f08f40cc5e0 2023-11-08T16:32:28.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.168+0000 7f08fbf05640 1 -- 172.21.15.81:0/2526142556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f4072170 con 0x7f08f40cc5e0 2023-11-08T16:32:28.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.169+0000 7f08ea7fc640 1 -- 172.21.15.81:0/2526142556 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f08e40113f0 con 0x7f08f40cc5e0 2023-11-08T16:32:28.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.170+0000 7f08ea7fc640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08d806ae70 0x7f08d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.170+0000 7f08ea7fc640 1 -- 172.21.15.81:0/2526142556 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7f08e4089d00 con 0x7f08f40cc5e0 2023-11-08T16:32:28.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.170+0000 7f08f9ca1640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08d806ae70 0x7f08d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.171+0000 7f08f9ca1640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08d806ae70 0x7f08d806d330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f08f0035000 tx=0x7f08f0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:28.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.174+0000 7f08ea7fc640 1 -- 172.21.15.81:0/2526142556 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08e40549f0 con 0x7f08f40cc5e0 2023-11-08T16:32:28.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.364+0000 7f08fbf05640 1 -- 172.21.15.81:0/2526142556 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08f415a7f0 con 0x7f08f40cc5e0 2023-11-08T16:32:28.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.366+0000 7f08ea7fc640 1 -- 172.21.15.81:0/2526142556 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+18684 (secure 0 0 0) 0x7f08f415a7f0 con 0x7f08f40cc5e0 2023-11-08T16:32:28.368 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:28.368 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":44,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:31:24.833641+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:30:27.977481+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-08T16:30:29.171902+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:30:40.141171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/1759199271":"2023-11-09T16:31:23.834479+0000","172.21.15.81:6836/1759199271":"2023-11-09T16:31:23.834479+0000","172.21.15.81:6837/3931731986":"2023-11-09T16:31:16.385652+0000","172.21.15.81:6836/3931731986":"2023-11-09T16:31:16.385652+0000","172.21.15.81:6837/773130363":"2023-11-09T16:31:14.352977+0000","172.21.15.81:6836/773130363":"2023-11-09T16:31:14.352977+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:32:28.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.368+0000 7f08fbf05640 1 -- 172.21.15.81:0/2526142556 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08d806ae70 msgr2=0x7f08d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:28.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.368+0000 7f08fbf05640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08d806ae70 0x7f08d806d330 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f08f0035000 tx=0x7f08f0030000 comp rx=0 tx=0).stop 2023-11-08T16:32:28.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.368+0000 7f08fbf05640 1 -- 172.21.15.81:0/2526142556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f40cc5e0 msgr2=0x7f08f41590c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:28.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.368+0000 7f08fbf05640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f40cc5e0 0x7f08f41590c0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f08e400c7f0 tx=0x7f08e400ccb0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.369+0000 7f08fbf05640 1 -- 172.21.15.81:0/2526142556 shutdown_connections 2023-11-08T16:32:28.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.369+0000 7f08fbf05640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08d806ae70 0x7f08d806d330 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.369+0000 7f08fbf05640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f40ccfb0 0x7f08f415b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.369+0000 7f08fbf05640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f40cc5e0 0x7f08f41590c0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.369+0000 7f08fbf05640 1 --2- 172.21.15.81:0/2526142556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08f40adcd0 0x7f08f4156890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.369+0000 7f08fbf05640 1 -- 172.21.15.81:0/2526142556 >> 172.21.15.81:0/2526142556 conn(0x7f08f40cb490 msgr2=0x7f08f40ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:28.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.369+0000 7f08fbf05640 1 -- 172.21.15.81:0/2526142556 shutdown_connections 2023-11-08T16:32:28.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.369+0000 7f08fbf05640 1 -- 172.21.15.81:0/2526142556 wait complete. 2023-11-08T16:32:28.389 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:32:28.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.596+0000 7fdb49df5640 1 Processor -- start 2023-11-08T16:32:28.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.596+0000 7fdb49df5640 1 -- start start 2023-11-08T16:32:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.598+0000 7fdb49df5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdb440cf720 0x7fdb4406d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.598+0000 7fdb49df5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb4406cc50 con 0x7fdb4415d3b0 2023-11-08T16:32:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.598+0000 7fdb49df5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb440c90f0 con 0x7fdb440ca130 2023-11-08T16:32:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.598+0000 7fdb49df5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb440adb70 con 0x7fdb440cf720 2023-11-08T16:32:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.598+0000 7fdb3bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdb440cf720 0x7fdb4406d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.598+0000 7fdb437fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fdb4415d3b0 0x7fdb440c9a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50344/0 (socket says 172.21.15.81:50344) 2023-11-08T16:32:28.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.598+0000 7fdb437fe640 1 -- 172.21.15.81:0/2510661983 learned_addr learned my addr 172.21.15.81:0/2510661983 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:28.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.598+0000 7fdb3bfff640 1 -- 172.21.15.81:0/2510661983 >> v1:172.21.15.93:6789/0 conn(0x7fdb440ca130 legacy=0x7fdb44070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:28.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.599+0000 7fdb3bfff640 1 -- 172.21.15.81:0/2510661983 >> v1:172.21.15.81:6789/0 conn(0x7fdb4415d3b0 legacy=0x7fdb440c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.599+0000 7fdb42ffd640 1 -- 172.21.15.81:0/2510661983 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080452823 0 0) 0x7fdb4406cc50 con 0x7fdb4415d3b0 2023-11-08T16:32:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.599+0000 7fdb3bfff640 1 -- 172.21.15.81:0/2510661983 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440c9360 con 0x7fdb440cf720 2023-11-08T16:32:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.600+0000 7fdb3bfff640 1 --2- 172.21.15.81:0/2510661983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb440cf720 0x7fdb4406d0e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0097a0 tx=0x7fdb2c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f0249c822a6cd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.600+0000 7fdb42ffd640 1 -- 172.21.15.81:0/2510661983 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c031830 con 0x7fdb440cf720 2023-11-08T16:32:28.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.600+0000 7fdb42ffd640 1 -- 172.21.15.81:0/2510661983 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c031990 con 0x7fdb440cf720 2023-11-08T16:32:28.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.600+0000 7fdb42ffd640 1 -- 172.21.15.81:0/2510661983 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c031d20 con 0x7fdb440cf720 2023-11-08T16:32:28.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.601+0000 7fdb49df5640 1 -- 172.21.15.81:0/2510661983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb440cf720 msgr2=0x7fdb4406d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:28.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.601+0000 7fdb49df5640 1 --2- 172.21.15.81:0/2510661983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb440cf720 0x7fdb4406d0e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0097a0 tx=0x7fdb2c02b8d0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.601+0000 7fdb49df5640 1 -- 172.21.15.81:0/2510661983 shutdown_connections 2023-11-08T16:32:28.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.601+0000 7fdb49df5640 1 --2- 172.21.15.81:0/2510661983 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb440cf720 0x7fdb4406d0e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.601+0000 7fdb49df5640 1 -- 172.21.15.81:0/2510661983 >> 172.21.15.81:0/2510661983 conn(0x7fdb440ceaa0 msgr2=0x7fdb440cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:28.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.601+0000 7fdb49df5640 1 -- 172.21.15.81:0/2510661983 shutdown_connections 2023-11-08T16:32:28.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.601+0000 7fdb49df5640 1 -- 172.21.15.81:0/2510661983 wait complete. 2023-11-08T16:32:28.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.603+0000 7fdb49df5640 1 Processor -- start 2023-11-08T16:32:28.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.603+0000 7fdb49df5640 1 -- start start 2023-11-08T16:32:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb49df5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb440ca130 0x7fdb440c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb49df5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb440cf720 0x7fdb44148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb49df5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4415d3b0 0x7fdb440c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb49df5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440c7820 con 0x7fdb4415d3b0 2023-11-08T16:32:28.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb49df5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440c6bc0 con 0x7fdb440cf720 2023-11-08T16:32:28.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb49df5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440c6ea0 con 0x7fdb440ca130 2023-11-08T16:32:28.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb43fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4415d3b0 0x7fdb440c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb43fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4415d3b0 0x7fdb440c4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34256/0 (socket says 172.21.15.81:34256) 2023-11-08T16:32:28.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb43fff640 1 -- 172.21.15.81:0/1137215944 learned_addr learned my addr 172.21.15.81:0/1137215944 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:28.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb3bfff640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb440cf720 0x7fdb44148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb43fff640 1 -- 172.21.15.81:0/1137215944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb440ca130 msgr2=0x7fdb440c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb437fe640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb440ca130 0x7fdb440c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb43fff640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb440ca130 0x7fdb440c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb43fff640 1 -- 172.21.15.81:0/1137215944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb440cf720 msgr2=0x7fdb44148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb43fff640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb440cf720 0x7fdb44148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb43fff640 1 -- 172.21.15.81:0/1137215944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c009400 con 0x7fdb4415d3b0 2023-11-08T16:32:28.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.604+0000 7fdb437fe640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb440ca130 0x7fdb440c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:28.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.605+0000 7fdb3bfff640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb440cf720 0x7fdb44148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:28.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.605+0000 7fdb43fff640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4415d3b0 0x7fdb440c4ef0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c780 tx=0x7fdb3c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:28.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.605+0000 7fdb40ff9640 1 -- 172.21.15.81:0/1137215944 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c004b60 con 0x7fdb4415d3b0 2023-11-08T16:32:28.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.605+0000 7fdb40ff9640 1 -- 172.21.15.81:0/1137215944 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb3c00ce90 con 0x7fdb4415d3b0 2023-11-08T16:32:28.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.605+0000 7fdb40ff9640 1 -- 172.21.15.81:0/1137215944 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb3c00a850 con 0x7fdb4415d3b0 2023-11-08T16:32:28.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.605+0000 7fdb49df5640 1 -- 172.21.15.81:0/1137215944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb440c7180 con 0x7fdb4415d3b0 2023-11-08T16:32:28.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.605+0000 7fdb49df5640 1 -- 172.21.15.81:0/1137215944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb440c5430 con 0x7fdb4415d3b0 2023-11-08T16:32:28.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.606+0000 7fdb49df5640 1 -- 172.21.15.81:0/1137215944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb440cc630 con 0x7fdb4415d3b0 2023-11-08T16:32:28.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.607+0000 7fdb40ff9640 1 -- 172.21.15.81:0/1137215944 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb3c00a9b0 con 0x7fdb4415d3b0 2023-11-08T16:32:28.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.608+0000 7fdb40ff9640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb2406ae70 0x7fdb2406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:28.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.608+0000 7fdb40ff9640 1 -- 172.21.15.81:0/1137215944 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6371+0+0 (secure 0 0 0) 0x7fdb3c089a70 con 0x7fdb4415d3b0 2023-11-08T16:32:28.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.608+0000 7fdb437fe640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb2406ae70 0x7fdb2406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:28.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.609+0000 7fdb437fe640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb2406ae70 0x7fdb2406d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdb340016b0 tx=0x7fdb34009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:28.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.612+0000 7fdb40ff9640 1 -- 172.21.15.81:0/1137215944 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb3c054880 con 0x7fdb4415d3b0 2023-11-08T16:32:28.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.841+0000 7fdb49df5640 1 -- 172.21.15.81:0/1137215944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdb440743f0 con 0x7fdb4415d3b0 2023-11-08T16:32:28.844 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:32:28.843+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:32:28.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.847+0000 7fdb40ff9640 1 -- 172.21.15.81:0/1137215944 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdb3c057ea0 con 0x7fdb4415d3b0 2023-11-08T16:32:28.849 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:32:28.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 -- 172.21.15.81:0/1137215944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb2406ae70 msgr2=0x7fdb2406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:28.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb2406ae70 0x7fdb2406d330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdb340016b0 tx=0x7fdb34009040 comp rx=0 tx=0).stop 2023-11-08T16:32:28.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 -- 172.21.15.81:0/1137215944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4415d3b0 msgr2=0x7fdb440c4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:28.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4415d3b0 0x7fdb440c4ef0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c00c780 tx=0x7fdb3c00cc40 comp rx=0 tx=0).stop 2023-11-08T16:32:28.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 -- 172.21.15.81:0/1137215944 shutdown_connections 2023-11-08T16:32:28.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb2406ae70 0x7fdb2406d330 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb4415d3b0 0x7fdb440c4ef0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb440cf720 0x7fdb44148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 --2- 172.21.15.81:0/1137215944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb440ca130 0x7fdb440c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:28.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 -- 172.21.15.81:0/1137215944 >> 172.21.15.81:0/1137215944 conn(0x7fdb440ceaa0 msgr2=0x7fdb44158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:28.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 -- 172.21.15.81:0/1137215944 shutdown_connections 2023-11-08T16:32:28.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:28.851+0000 7fdb49df5640 1 -- 172.21.15.81:0/1137215944 wait complete. 2023-11-08T16:32:28.863 INFO:tasks.ceph.mds.c.smithi081.stderr: -5851> 2023-11-08T16:31:23.864+0000 7fbf103d6640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-08T16:32:28.869 INFO:tasks.ceph.mds.c.smithi081.stderr: -4364> 2023-11-08T16:31:28.860+0000 7fbf13bdd640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:32:28.871 INFO:tasks.ceph.mds.c.smithi081.stderr: -3999> 2023-11-08T16:31:30.822+0000 7fbf103d6640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-11-08T16:32:28.871 INFO:tasks.ceph.mds.c.smithi081.stderr: -3842> 2023-11-08T16:31:30.823+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55c9f2dafb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(4)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:32:28.872 INFO:tasks.ceph.mds.c.smithi081.stderr: -3682> 2023-11-08T16:31:30.825+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55c9f2e52b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x100000001f7:[<0x10000000000/2 v32>,<0x1/subdir v44>]//[5]","memoryvalue":"(6)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:32:28.872 INFO:tasks.ceph.mds.c.smithi081.stderr: -3659> 2023-11-08T16:31:30.825+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55c9f2e52000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x100000001f9:[<0x10000000000/4 v40>,<0x1/subdir v44>]//[5]","memoryvalue":"(6)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:32:28.873 INFO:tasks.ceph.mds.c.smithi081.stderr: -3641> 2023-11-08T16:31:30.825+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55c9f2e52580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x100000001f8:[<0x10000000000/3 v38>,<0x1/subdir v44>]//[5]","memoryvalue":"(6)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:32:28.873 INFO:tasks.ceph.mds.c.smithi081.stderr: -3625> 2023-11-08T16:31:30.825+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55c9f2e53080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x100000001f6:[<0x10000000000/1 v24>,<0x1/subdir v44>]//[5]","memoryvalue":"(6)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:32:28.874 INFO:tasks.ceph.mds.c.smithi081.stderr: -3607> 2023-11-08T16:31:30.826+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55c9f2e53600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x10000000001:[<0x10000000000/0 v16>,<0x1/subdir v44>]//[5]","memoryvalue":"(6)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:32:28.874 INFO:tasks.ceph.mds.c.smithi081.stderr: -3327> 2023-11-08T16:31:30.829+0000 7fbf103d6640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x55c9f2da46c0 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-11-08T16:31:30.555816+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={5730=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55c9f2daf600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(4)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-11-08T16:31:30.555816+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:32:28.879 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T16:32:28.933 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:32:29.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.069+0000 7fb46b3b6640 1 Processor -- start 2023-11-08T16:32:29.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.069+0000 7fb46b3b6640 1 -- start start 2023-11-08T16:32:29.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.070+0000 7fb46b3b6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb46415e550 0x7fb4640b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:29.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.070+0000 7fb46b3b6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb464071460 con 0x7fb46415e170 2023-11-08T16:32:29.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.070+0000 7fb46b3b6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4640b0e00 con 0x7fb4640b0a20 2023-11-08T16:32:29.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.070+0000 7fb46b3b6640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4640703c0 con 0x7fb46415e550 2023-11-08T16:32:29.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.070+0000 7fb469152640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb46415e170 0x7fb46415ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50354/0 (socket says 172.21.15.81:50354) 2023-11-08T16:32:29.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.070+0000 7fb469152640 1 -- 172.21.15.81:0/279765465 learned_addr learned my addr 172.21.15.81:0/279765465 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:29.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.070+0000 7fb468951640 1 --2- 172.21.15.81:0/279765465 >> v2:172.21.15.81:3300/0 conn(0x7fb46415e550 0x7fb4640b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:29.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.071+0000 7fb45bfff640 1 -- 172.21.15.81:0/279765465 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3021754503 0 0) 0x7fb464071460 con 0x7fb46415e170 2023-11-08T16:32:29.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.071+0000 7fb45bfff640 1 -- 172.21.15.81:0/279765465 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb450003520 con 0x7fb46415e170 2023-11-08T16:32:29.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.071+0000 7fb468951640 1 -- 172.21.15.81:0/279765465 >> v1:172.21.15.93:6789/0 conn(0x7fb4640b0a20 legacy=0x7fb4640ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:29.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.071+0000 7fb468951640 1 -- 172.21.15.81:0/279765465 >> v1:172.21.15.81:6789/0 conn(0x7fb46415e170 legacy=0x7fb46415ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:29.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.071+0000 7fb468951640 1 -- 172.21.15.81:0/279765465 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4640706a0 con 0x7fb46415e550 2023-11-08T16:32:29.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.072+0000 7fb468951640 1 --2- 172.21.15.81:0/279765465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb46415e550 0x7fb4640b04e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb4540097a0 tx=0x7fb45402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee4046b742c7dc6b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:29.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.072+0000 7fb45bfff640 1 -- 172.21.15.81:0/279765465 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454031830 con 0x7fb46415e550 2023-11-08T16:32:29.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.072+0000 7fb45bfff640 1 -- 172.21.15.81:0/279765465 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb454031990 con 0x7fb46415e550 2023-11-08T16:32:29.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.072+0000 7fb45bfff640 1 -- 172.21.15.81:0/279765465 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454031d20 con 0x7fb46415e550 2023-11-08T16:32:29.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.073+0000 7fb46b3b6640 1 -- 172.21.15.81:0/279765465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb46415e550 msgr2=0x7fb4640b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:29.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.073+0000 7fb46b3b6640 1 --2- 172.21.15.81:0/279765465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb46415e550 0x7fb4640b04e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb4540097a0 tx=0x7fb45402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:32:29.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.073+0000 7fb46b3b6640 1 -- 172.21.15.81:0/279765465 shutdown_connections 2023-11-08T16:32:29.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.073+0000 7fb46b3b6640 1 --2- 172.21.15.81:0/279765465 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb46415e550 0x7fb4640b04e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:29.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.073+0000 7fb46b3b6640 1 -- 172.21.15.81:0/279765465 >> 172.21.15.81:0/279765465 conn(0x7fb4640cb4e0 msgr2=0x7fb4640cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:29.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.073+0000 7fb46b3b6640 1 -- 172.21.15.81:0/279765465 shutdown_connections 2023-11-08T16:32:29.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.073+0000 7fb46b3b6640 1 -- 172.21.15.81:0/279765465 wait complete. 2023-11-08T16:32:29.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.075+0000 7fb46b3b6640 1 Processor -- start 2023-11-08T16:32:29.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.075+0000 7fb46b3b6640 1 -- start start 2023-11-08T16:32:29.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.076+0000 7fb46b3b6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4640b0a20 0x7fb46414e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:29.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.076+0000 7fb46b3b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46415e170 0x7fb4640c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:29.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.076+0000 7fb46b3b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb46415e550 0x7fb4640c16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:29.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.076+0000 7fb46b3b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640c4ee0 con 0x7fb46415e550 2023-11-08T16:32:29.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.076+0000 7fb46b3b6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4640c4510 con 0x7fb4640b0a20 2023-11-08T16:32:29.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.076+0000 7fb46b3b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4641519c0 con 0x7fb46415e170 2023-11-08T16:32:29.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.076+0000 7fb469953640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb46415e550 0x7fb4640c16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:29.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.076+0000 7fb469953640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb46415e550 0x7fb4640c16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34270/0 (socket says 172.21.15.81:34270) 2023-11-08T16:32:29.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.076+0000 7fb469953640 1 -- 172.21.15.81:0/3360302899 learned_addr learned my addr 172.21.15.81:0/3360302899 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:29.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb469152640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4640b0a20 0x7fb46414e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:29.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb468951640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46415e170 0x7fb4640c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:29.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb469953640 1 -- 172.21.15.81:0/3360302899 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46415e170 msgr2=0x7fb4640c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:29.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb469953640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46415e170 0x7fb4640c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:29.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb469953640 1 -- 172.21.15.81:0/3360302899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4640b0a20 msgr2=0x7fb46414e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:29.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb469953640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4640b0a20 0x7fb46414e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:29.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb469953640 1 -- 172.21.15.81:0/3360302899 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb454009400 con 0x7fb46415e550 2023-11-08T16:32:29.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb469152640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4640b0a20 0x7fb46414e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:32:29.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb469953640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb46415e550 0x7fb4640c16d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00c780 tx=0x7fb45c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:29.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb459ffb640 1 -- 172.21.15.81:0/3360302899 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c004b60 con 0x7fb46415e550 2023-11-08T16:32:29.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb459ffb640 1 -- 172.21.15.81:0/3360302899 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45c00ce90 con 0x7fb46415e550 2023-11-08T16:32:29.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb46b3b6640 1 -- 172.21.15.81:0/3360302899 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4640c3be0 con 0x7fb46415e550 2023-11-08T16:32:29.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.077+0000 7fb459ffb640 1 -- 172.21.15.81:0/3360302899 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45c00a850 con 0x7fb46415e550 2023-11-08T16:32:29.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.078+0000 7fb46b3b6640 1 -- 172.21.15.81:0/3360302899 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4640c41a0 con 0x7fb46415e550 2023-11-08T16:32:29.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.079+0000 7fb46b3b6640 1 -- 172.21.15.81:0/3360302899 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4640c4680 con 0x7fb46415e550 2023-11-08T16:32:29.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.080+0000 7fb459ffb640 1 -- 172.21.15.81:0/3360302899 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb45c00a9b0 con 0x7fb46415e550 2023-11-08T16:32:29.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.080+0000 7fb459ffb640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb44806aea0 0x7fb44806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:29.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.081+0000 7fb459ffb640 1 -- 172.21.15.81:0/3360302899 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fb45c089ac0 con 0x7fb46415e550 2023-11-08T16:32:29.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.081+0000 7fb469152640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb44806aea0 0x7fb44806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:29.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.081+0000 7fb469152640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb44806aea0 0x7fb44806d360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb4600011e0 tx=0x7fb460009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:29.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.085+0000 7fb459ffb640 1 -- 172.21.15.81:0/3360302899 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb45c0547e0 con 0x7fb46415e550 2023-11-08T16:32:29.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.306+0000 7fb46b3b6640 1 -- 172.21.15.81:0/3360302899 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb46404ece0 con 0x7fb46415e550 2023-11-08T16:32:29.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.850+0000 7fb459ffb640 1 -- 172.21.15.81:0/3360302899 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb45c057e00 con 0x7fb46415e550 2023-11-08T16:32:29.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.854+0000 7fb46b3b6640 1 -- 172.21.15.81:0/3360302899 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb44806aea0 msgr2=0x7fb44806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:29.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.854+0000 7fb46b3b6640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb44806aea0 0x7fb44806d360 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb4600011e0 tx=0x7fb460009040 comp rx=0 tx=0).stop 2023-11-08T16:32:29.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.854+0000 7fb46b3b6640 1 -- 172.21.15.81:0/3360302899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb46415e550 msgr2=0x7fb4640c16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:29.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.854+0000 7fb46b3b6640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb46415e550 0x7fb4640c16d0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb45c00c780 tx=0x7fb45c00cc40 comp rx=0 tx=0).stop 2023-11-08T16:32:29.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.855+0000 7fb46b3b6640 1 -- 172.21.15.81:0/3360302899 shutdown_connections 2023-11-08T16:32:29.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.855+0000 7fb46b3b6640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb44806aea0 0x7fb44806d360 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:29.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.855+0000 7fb46b3b6640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb46415e550 0x7fb4640c16d0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:29.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.855+0000 7fb46b3b6640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb46415e170 0x7fb4640c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:29.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.855+0000 7fb46b3b6640 1 --2- 172.21.15.81:0/3360302899 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4640b0a20 0x7fb46414e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:29.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.855+0000 7fb46b3b6640 1 -- 172.21.15.81:0/3360302899 >> 172.21.15.81:0/3360302899 conn(0x7fb4640cb4e0 msgr2=0x7fb46406b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:29.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.855+0000 7fb46b3b6640 1 -- 172.21.15.81:0/3360302899 shutdown_connections 2023-11-08T16:32:29.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:29.855+0000 7fb46b3b6640 1 -- 172.21.15.81:0/3360302899 wait complete. 2023-11-08T16:32:29.878 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T16:32:30.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.082+0000 7f01dbfff640 1 Processor -- start 2023-11-08T16:32:30.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.082+0000 7f01dbfff640 1 -- start start 2023-11-08T16:32:30.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.084+0000 7f01dbfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01dc15d140 0x7f01dc0cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:30.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.084+0000 7f01dbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f01dc0ce4c0 0x7f01dc0cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:30.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.084+0000 7f01dbfff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f01dc0d38c0 0x7f01dc0d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:30.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.084+0000 7f01dbfff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01dc0ce9c0 con 0x7f01dc0ce4c0 2023-11-08T16:32:30.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.084+0000 7f01dbfff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01dc0ceca0 con 0x7f01dc0d38c0 2023-11-08T16:32:30.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.084+0000 7f01dbfff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc0b26e0 con 0x7f01dc15d140 2023-11-08T16:32:30.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.084+0000 7f01d27fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f01dc0ce4c0 0x7f01dc0cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:30.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.085+0000 7f01d27fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f01dc0ce4c0 0x7f01dc0cfa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34278/0 (socket says 172.21.15.81:34278) 2023-11-08T16:32:30.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.085+0000 7f01d27fc640 1 -- 172.21.15.81:0/222816421 learned_addr learned my addr 172.21.15.81:0/222816421 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:30.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.085+0000 7f01daffd640 1 --2- 172.21.15.81:0/222816421 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01dc15d140 0x7f01dc0cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:30.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.085+0000 7f01db7fe640 1 --2- 172.21.15.81:0/222816421 >> v2:172.21.15.93:3300/0 conn(0x7f01dc0d38c0 0x7f01dc0d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:30.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.085+0000 7f01d27fc640 1 -- 172.21.15.81:0/222816421 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01dc15d140 msgr2=0x7f01dc0cf4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:30.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.085+0000 7f01d27fc640 1 --2- 172.21.15.81:0/222816421 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01dc15d140 0x7f01dc0cf4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.085+0000 7f01d27fc640 1 -- 172.21.15.81:0/222816421 >> v2:172.21.15.93:3300/0 conn(0x7f01dc0d38c0 msgr2=0x7f01dc0d0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:30.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.085+0000 7f01d27fc640 1 --2- 172.21.15.81:0/222816421 >> v2:172.21.15.93:3300/0 conn(0x7f01dc0d38c0 0x7f01dc0d0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.085+0000 7f01d27fc640 1 -- 172.21.15.81:0/222816421 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc0b29c0 con 0x7f01dc0ce4c0 2023-11-08T16:32:30.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.085+0000 7f01daffd640 1 --2- 172.21.15.81:0/222816421 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01dc15d140 0x7f01dc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:30.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.085+0000 7f01db7fe640 1 --2- 172.21.15.81:0/222816421 >> v2:172.21.15.93:3300/0 conn(0x7f01dc0d38c0 0x7f01dc0d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:30.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.086+0000 7f01d27fc640 1 --2- 172.21.15.81:0/222816421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01dc0ce4c0 0x7f01dc0cfa10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0d0880 tx=0x7f01c402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d4eeea71d2db88 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:30.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.086+0000 7f01da7fc640 1 -- 172.21.15.81:0/222816421 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4031880 con 0x7f01dc0ce4c0 2023-11-08T16:32:30.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.086+0000 7f01da7fc640 1 -- 172.21.15.81:0/222816421 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c40319e0 con 0x7f01dc0ce4c0 2023-11-08T16:32:30.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.086+0000 7f01da7fc640 1 -- 172.21.15.81:0/222816421 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4031d70 con 0x7f01dc0ce4c0 2023-11-08T16:32:30.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.087+0000 7f01dbfff640 1 -- 172.21.15.81:0/222816421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01dc0ce4c0 msgr2=0x7f01dc0cfa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:30.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.087+0000 7f01dbfff640 1 --2- 172.21.15.81:0/222816421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01dc0ce4c0 0x7f01dc0cfa10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0d0880 tx=0x7f01c402b920 comp rx=0 tx=0).stop 2023-11-08T16:32:30.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.087+0000 7f01dbfff640 1 -- 172.21.15.81:0/222816421 shutdown_connections 2023-11-08T16:32:30.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.087+0000 7f01dbfff640 1 --2- 172.21.15.81:0/222816421 >> v2:172.21.15.93:3300/0 conn(0x7f01dc0d38c0 0x7f01dc0d0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.087+0000 7f01dbfff640 1 --2- 172.21.15.81:0/222816421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01dc0ce4c0 0x7f01dc0cfa10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.087+0000 7f01dbfff640 1 --2- 172.21.15.81:0/222816421 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01dc15d140 0x7f01dc0cf4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.087+0000 7f01dbfff640 1 -- 172.21.15.81:0/222816421 >> 172.21.15.81:0/222816421 conn(0x7f01dc0d2da0 msgr2=0x7f01dc0d31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:30.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.087+0000 7f01dbfff640 1 -- 172.21.15.81:0/222816421 shutdown_connections 2023-11-08T16:32:30.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.087+0000 7f01dbfff640 1 -- 172.21.15.81:0/222816421 wait complete. 2023-11-08T16:32:30.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.089+0000 7f01dbfff640 1 Processor -- start 2023-11-08T16:32:30.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.089+0000 7f01dbfff640 1 -- start start 2023-11-08T16:32:30.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.090+0000 7f01dbfff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc0ce4c0 0x7f01dc152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:30.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.090+0000 7f01dbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01dc0d38c0 0x7f01dc154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:30.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.090+0000 7f01dbfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01dc15d140 0x7f01dc155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:30.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.090+0000 7f01dbfff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc152640 con 0x7f01dc0d38c0 2023-11-08T16:32:30.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.090+0000 7f01dbfff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc159710 con 0x7f01dc0ce4c0 2023-11-08T16:32:30.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.090+0000 7f01dbfff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc156300 con 0x7f01dc15d140 2023-11-08T16:32:30.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.090+0000 7f01d27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01dc0d38c0 0x7f01dc154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:30.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.090+0000 7f01d27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01dc0d38c0 0x7f01dc154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34290/0 (socket says 172.21.15.81:34290) 2023-11-08T16:32:30.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.090+0000 7f01d27fc640 1 -- 172.21.15.81:0/2448695647 learned_addr learned my addr 172.21.15.81:0/2448695647 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:30.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01db7fe640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01dc15d140 0x7f01dc155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:30.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01d27fc640 1 -- 172.21.15.81:0/2448695647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01dc15d140 msgr2=0x7f01dc155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:30.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01d27fc640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01dc15d140 0x7f01dc155450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01d27fc640 1 -- 172.21.15.81:0/2448695647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc0ce4c0 msgr2=0x7f01dc152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:30.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01daffd640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc0ce4c0 0x7f01dc152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:30.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01d27fc640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc0ce4c0 0x7f01dc152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01d27fc640 1 -- 172.21.15.81:0/2448695647 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c40094d0 con 0x7f01dc0d38c0 2023-11-08T16:32:30.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01d27fc640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01dc0d38c0 0x7f01dc154f10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f01c4000c00 tx=0x7f01c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:30.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01d8ff9640 1 -- 172.21.15.81:0/2448695647 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4031850 con 0x7f01dc0d38c0 2023-11-08T16:32:30.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01d8ff9640 1 -- 172.21.15.81:0/2448695647 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c40443f0 con 0x7f01dc0d38c0 2023-11-08T16:32:30.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01dbfff640 1 -- 172.21.15.81:0/2448695647 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01dc158890 con 0x7f01dc0d38c0 2023-11-08T16:32:30.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.091+0000 7f01d8ff9640 1 -- 172.21.15.81:0/2448695647 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c40447a0 con 0x7f01dc0d38c0 2023-11-08T16:32:30.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.092+0000 7f01dbfff640 1 -- 172.21.15.81:0/2448695647 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01dc158e50 con 0x7f01dc0d38c0 2023-11-08T16:32:30.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.092+0000 7f01dbfff640 1 -- 172.21.15.81:0/2448695647 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01dc067cb0 con 0x7f01dc0d38c0 2023-11-08T16:32:30.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.095+0000 7f01d8ff9640 1 -- 172.21.15.81:0/2448695647 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f01c40303d0 con 0x7f01dc0d38c0 2023-11-08T16:32:30.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.096+0000 7f01d8ff9640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f01bc06aea0 0x7f01bc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:30.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.096+0000 7f01d8ff9640 1 -- 172.21.15.81:0/2448695647 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6457+0+0 (secure 0 0 0) 0x7f01c40ac550 con 0x7f01dc0d38c0 2023-11-08T16:32:30.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.096+0000 7f01daffd640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f01bc06aea0 0x7f01bc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:30.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.097+0000 7f01daffd640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f01bc06aea0 0x7f01bc06d360 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00b840 tx=0x7f01cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:30.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.099+0000 7f01d8ff9640 1 -- 172.21.15.81:0/2448695647 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f01c4077510 con 0x7f01dc0d38c0 2023-11-08T16:32:30.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.295+0000 7f01dbfff640 1 -- 172.21.15.81:0/2448695647 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01dc159880 con 0x7f01dc0d38c0 2023-11-08T16:32:30.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.853+0000 7f01d8ff9640 1 -- 172.21.15.81:0/2448695647 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v46) v1 ==== 181+0+0 (secure 0 0 0) 0x7f01dc159880 con 0x7f01dc0d38c0 2023-11-08T16:32:30.856 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' removed 2023-11-08T16:32:30.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.858+0000 7f01dbfff640 1 -- 172.21.15.81:0/2448695647 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f01bc06aea0 msgr2=0x7f01bc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:30.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.858+0000 7f01dbfff640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f01bc06aea0 0x7f01bc06d360 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00b840 tx=0x7f01cc008040 comp rx=0 tx=0).stop 2023-11-08T16:32:30.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.858+0000 7f01dbfff640 1 -- 172.21.15.81:0/2448695647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01dc0d38c0 msgr2=0x7f01dc154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:30.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.858+0000 7f01dbfff640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01dc0d38c0 0x7f01dc154f10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f01c4000c00 tx=0x7f01c40046c0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.858+0000 7f01dbfff640 1 -- 172.21.15.81:0/2448695647 shutdown_connections 2023-11-08T16:32:30.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.859+0000 7f01dbfff640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f01bc06aea0 0x7f01bc06d360 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.859+0000 7f01dbfff640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f01dc15d140 0x7f01dc155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.859+0000 7f01dbfff640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f01dc0d38c0 0x7f01dc154f10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.859+0000 7f01dbfff640 1 --2- 172.21.15.81:0/2448695647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f01dc0ce4c0 0x7f01dc152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:30.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.859+0000 7f01dbfff640 1 -- 172.21.15.81:0/2448695647 >> 172.21.15.81:0/2448695647 conn(0x7f01dc0d2da0 msgr2=0x7f01dc06bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:30.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.859+0000 7f01dbfff640 1 -- 172.21.15.81:0/2448695647 shutdown_connections 2023-11-08T16:32:30.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:30.859+0000 7f01dbfff640 1 -- 172.21.15.81:0/2448695647 wait complete. 2023-11-08T16:32:30.885 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T16:32:31.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.082+0000 7fcd8bfff640 1 Processor -- start 2023-11-08T16:32:31.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.082+0000 7fcd8bfff640 1 -- start start 2023-11-08T16:32:31.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.084+0000 7fcd8bfff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcd8c1576c0 0x7fcd8c15dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:31.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.084+0000 7fcd8bfff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c15d530 con 0x7fcd8c0b42d0 2023-11-08T16:32:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.084+0000 7fcd8bfff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd8c0b47b0 con 0x7fcd8c15e250 2023-11-08T16:32:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.084+0000 7fcd8bfff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8c15d8f0 con 0x7fcd8c1576c0 2023-11-08T16:32:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.084+0000 7fcd8b7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcd8c0b42d0 0x7fcd8c06a0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40986/0 (socket says 172.21.15.81:40986) 2023-11-08T16:32:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.084+0000 7fcd8b7fe640 1 -- 172.21.15.81:0/3168473852 learned_addr learned my addr 172.21.15.81:0/3168473852 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:31.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.084+0000 7fcd8affd640 1 --2- 172.21.15.81:0/3168473852 >> v2:172.21.15.93:3300/0 conn(0x7fcd8c1576c0 0x7fcd8c15dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:31.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.084+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126729450 0 0) 0x7fcd8c15d530 con 0x7fcd8c0b42d0 2023-11-08T16:32:31.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.084+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd78003520 con 0x7fcd8c0b42d0 2023-11-08T16:32:31.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2414226912 0 0) 0x7fcd78003520 con 0x7fcd8c0b42d0 2023-11-08T16:32:31.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 >> v2:172.21.15.93:3300/0 conn(0x7fcd8c1576c0 msgr2=0x7fcd8c15dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:31.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 --2- 172.21.15.81:0/3168473852 >> v2:172.21.15.93:3300/0 conn(0x7fcd8c1576c0 0x7fcd8c15dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 >> v1:172.21.15.93:6789/0 conn(0x7fcd8c15e250 legacy=0x7fcd8c071330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c0b4a90 con 0x7fcd8c0b42d0 2023-11-08T16:32:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fcd84001a10 con 0x7fcd8c0b42d0 2023-11-08T16:32:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 >> v1:172.21.15.81:6789/0 conn(0x7fcd8c0b42d0 legacy=0x7fcd8c06a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd78003970 0x7fcd78003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd780047c0 0x7fcd78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd78005180 0x7fcd7800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c15d8f0 con 0x7fcd78005180 2023-11-08T16:32:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7800c0b0 con 0x7fcd780047c0 2023-11-08T16:32:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7800c270 con 0x7fcd78003970 2023-11-08T16:32:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.085+0000 7fcd8b7fe640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd78005180 0x7fcd7800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.086+0000 7fcd8affd640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd78003970 0x7fcd78003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.086+0000 7fcd8b7fe640 1 -- 172.21.15.81:0/3168473852 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd78003970 msgr2=0x7fcd78003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.086+0000 7fcd8b7fe640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd78003970 0x7fcd78003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.086+0000 7fcd8b7fe640 1 -- 172.21.15.81:0/3168473852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd780047c0 msgr2=0x7fcd78004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.086+0000 7fcd827fc640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd780047c0 0x7fcd78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.086+0000 7fcd8b7fe640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd780047c0 0x7fcd78004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.086+0000 7fcd8b7fe640 1 -- 172.21.15.81:0/3168473852 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd78003520 con 0x7fcd78005180 2023-11-08T16:32:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.086+0000 7fcd8affd640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd78003970 0x7fcd78003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.087+0000 7fcd8b7fe640 1 -- 172.21.15.81:0/3168473852 reap_dead start 2023-11-08T16:32:31.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.087+0000 7fcd8b7fe640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd78005180 0x7fcd7800b980 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0b46e0 tx=0x7fcd780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6a15a433c0f418 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:31.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.087+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd84034630 con 0x7fcd78005180 2023-11-08T16:32:31.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.087+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8402f070 con 0x7fcd78005180 2023-11-08T16:32:31.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.087+0000 7fcd8a7fc640 1 -- 172.21.15.81:0/3168473852 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840349f0 con 0x7fcd78005180 2023-11-08T16:32:31.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.087+0000 7fcd8bfff640 1 -- 172.21.15.81:0/3168473852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd78005180 msgr2=0x7fcd7800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:31.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.087+0000 7fcd8bfff640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd78005180 0x7fcd7800b980 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c0b46e0 tx=0x7fcd780057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.087+0000 7fcd8bfff640 1 -- 172.21.15.81:0/3168473852 shutdown_connections 2023-11-08T16:32:31.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.087+0000 7fcd8bfff640 1 --2- 172.21.15.81:0/3168473852 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd78005180 0x7fcd7800b980 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.087+0000 7fcd8bfff640 1 -- 172.21.15.81:0/3168473852 >> 172.21.15.81:0/3168473852 conn(0x7fcd8c06bb70 msgr2=0x7fcd8c0b16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:31.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.088+0000 7fcd8bfff640 1 -- 172.21.15.81:0/3168473852 shutdown_connections 2023-11-08T16:32:31.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.088+0000 7fcd8bfff640 1 -- 172.21.15.81:0/3168473852 wait complete. 2023-11-08T16:32:31.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.089+0000 7fcd8bfff640 1 Processor -- start 2023-11-08T16:32:31.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.089+0000 7fcd8bfff640 1 -- start start 2023-11-08T16:32:31.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.090+0000 7fcd8bfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd78005180 0x7fcd8c1f21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:31.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.090+0000 7fcd8bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd8c06d610 0x7fcd8c06da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:31.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.090+0000 7fcd8bfff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd8c15e640 0x7fcd8c15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:31.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.090+0000 7fcd8bfff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c06dfd0 con 0x7fcd8c06d610 2023-11-08T16:32:31.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.090+0000 7fcd8bfff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c15f000 con 0x7fcd8c15e640 2023-11-08T16:32:31.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.090+0000 7fcd8bfff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8c1604e0 con 0x7fcd78005180 2023-11-08T16:32:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.091+0000 7fcd827fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd8c06d610 0x7fcd8c06da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.091+0000 7fcd827fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd8c06d610 0x7fcd8c06da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33614/0 (socket says 172.21.15.81:33614) 2023-11-08T16:32:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.091+0000 7fcd827fc640 1 -- 172.21.15.81:0/1015584093 learned_addr learned my addr 172.21.15.81:0/1015584093 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:31.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.091+0000 7fcd8affd640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd78005180 0x7fcd8c1f21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.091+0000 7fcd827fc640 1 -- 172.21.15.81:0/1015584093 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd78005180 msgr2=0x7fcd8c1f21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.091+0000 7fcd8b7fe640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd8c15e640 0x7fcd8c15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.091+0000 7fcd827fc640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd78005180 0x7fcd8c1f21d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.091+0000 7fcd827fc640 1 -- 172.21.15.81:0/1015584093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd8c15e640 msgr2=0x7fcd8c15eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:31.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.091+0000 7fcd827fc640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd8c15e640 0x7fcd8c15eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.091+0000 7fcd827fc640 1 -- 172.21.15.81:0/1015584093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8c1f2710 con 0x7fcd8c06d610 2023-11-08T16:32:31.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.091+0000 7fcd827fc640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd8c06d610 0x7fcd8c06da90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400d4f0 tx=0x7fcd7400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:31.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.092+0000 7fcd83fff640 1 -- 172.21.15.81:0/1015584093 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74005230 con 0x7fcd8c06d610 2023-11-08T16:32:31.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.092+0000 7fcd83fff640 1 -- 172.21.15.81:0/1015584093 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd74005390 con 0x7fcd8c06d610 2023-11-08T16:32:31.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.092+0000 7fcd83fff640 1 -- 172.21.15.81:0/1015584093 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd74012840 con 0x7fcd8c06d610 2023-11-08T16:32:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.092+0000 7fcd8bfff640 1 -- 172.21.15.81:0/1015584093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8c15f250 con 0x7fcd8c06d610 2023-11-08T16:32:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.092+0000 7fcd8bfff640 1 -- 172.21.15.81:0/1015584093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd8c0af560 con 0x7fcd8c06d610 2023-11-08T16:32:31.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.093+0000 7fcd8bfff640 1 -- 172.21.15.81:0/1015584093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8c04ece0 con 0x7fcd8c06d610 2023-11-08T16:32:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.094+0000 7fcd83fff640 1 -- 172.21.15.81:0/1015584093 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd740129a0 con 0x7fcd8c06d610 2023-11-08T16:32:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.095+0000 7fcd83fff640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6c06aea0 0x7fcd6c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.095+0000 7fcd83fff640 1 -- 172.21.15.81:0/1015584093 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6019+0+0 (secure 0 0 0) 0x7fcd74089540 con 0x7fcd8c06d610 2023-11-08T16:32:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.095+0000 7fcd8affd640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6c06aea0 0x7fcd6c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:31.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.096+0000 7fcd8affd640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6c06aea0 0x7fcd6c06d360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009aa0 tx=0x7fcd7c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:31.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.099+0000 7fcd83fff640 1 -- 172.21.15.81:0/1015584093 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd74054540 con 0x7fcd8c06d610 2023-11-08T16:32:31.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.291+0000 7fcd8bfff640 1 -- 172.21.15.81:0/1015584093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcd8c0b3bd0 con 0x7fcd8c06d610 2023-11-08T16:32:31.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.869+0000 7fcd83fff640 1 -- 172.21.15.81:0/1015584093 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v47) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcd74057b60 con 0x7fcd8c06d610 2023-11-08T16:32:31.873 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' removed 2023-11-08T16:32:31.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.873+0000 7fcd8bfff640 1 -- 172.21.15.81:0/1015584093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6c06aea0 msgr2=0x7fcd6c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:31.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.874+0000 7fcd8bfff640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6c06aea0 0x7fcd6c06d360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009aa0 tx=0x7fcd7c008040 comp rx=0 tx=0).stop 2023-11-08T16:32:31.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.874+0000 7fcd8bfff640 1 -- 172.21.15.81:0/1015584093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd8c06d610 msgr2=0x7fcd8c06da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:31.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.874+0000 7fcd8bfff640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd8c06d610 0x7fcd8c06da90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fcd7400d4f0 tx=0x7fcd7400d9b0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.876+0000 7fcd8bfff640 1 -- 172.21.15.81:0/1015584093 shutdown_connections 2023-11-08T16:32:31.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.876+0000 7fcd8bfff640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6c06aea0 0x7fcd6c06d360 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.876+0000 7fcd8bfff640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd8c15e640 0x7fcd8c15eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.876+0000 7fcd8bfff640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd8c06d610 0x7fcd8c06da90 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.876+0000 7fcd8bfff640 1 --2- 172.21.15.81:0/1015584093 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd78005180 0x7fcd8c1f21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:31.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.876+0000 7fcd8bfff640 1 -- 172.21.15.81:0/1015584093 >> 172.21.15.81:0/1015584093 conn(0x7fcd8c06bb70 msgr2=0x7fcd8c0b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:31.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.877+0000 7fcd8bfff640 1 -- 172.21.15.81:0/1015584093 shutdown_connections 2023-11-08T16:32:31.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:31.877+0000 7fcd8bfff640 1 -- 172.21.15.81:0/1015584093 wait complete. 2023-11-08T16:32:31.906 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-11-08T16:32:32.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.112+0000 7fcce4afc640 1 Processor -- start 2023-11-08T16:32:32.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.112+0000 7fcce4afc640 1 -- start start 2023-11-08T16:32:32.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.113+0000 7fcce4afc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 0x7fcce015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:32.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.113+0000 7fcce4afc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcce015afd0 0x7fcce015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:32.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.113+0000 7fcce4afc640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce00b09f0 con 0x7fcce00714b0 2023-11-08T16:32:32.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.113+0000 7fcce4afc640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce006b250 con 0x7fcce015afd0 2023-11-08T16:32:32.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.113+0000 7fcce4afc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00ca360 con 0x7fcce0071890 2023-11-08T16:32:32.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.114+0000 7fccd7fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 0x7fcce015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:32.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.114+0000 7fccded9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcce015afd0 0x7fcce015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:32.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.114+0000 7fccded9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcce015afd0 0x7fcce015ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:46542/0 (socket says 172.21.15.81:46542) 2023-11-08T16:32:32.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.114+0000 7fccde59c640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fcce00714b0 0x7fcce015a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:34496/0 (socket says 172.21.15.81:34496) 2023-11-08T16:32:32.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.114+0000 7fccde59c640 1 -- 172.21.15.81:0/2197034930 learned_addr learned my addr 172.21.15.81:0/2197034930 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:32.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.114+0000 7fccd7fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 0x7fcce015e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:39074/0 (socket says 172.21.15.81:39074) 2023-11-08T16:32:32.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.115+0000 7fccd7fff640 1 -- 172.21.15.81:0/2197034930 >> v2:172.21.15.93:3300/0 conn(0x7fcce015afd0 msgr2=0x7fcce015ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:32.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.115+0000 7fccd7fff640 1 --2- 172.21.15.81:0/2197034930 >> v2:172.21.15.93:3300/0 conn(0x7fcce015afd0 0x7fcce015ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:32.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.115+0000 7fccd7fff640 1 -- 172.21.15.81:0/2197034930 >> v1:172.21.15.93:6789/0 conn(0x7fcce00714b0 legacy=0x7fcce015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:32.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.115+0000 7fccd7fff640 1 -- 172.21.15.81:0/2197034930 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce00ca640 con 0x7fcce0071890 2023-11-08T16:32:32.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.115+0000 7fccded9d640 1 --2- 172.21.15.81:0/2197034930 >> v2:172.21.15.93:3300/0 conn(0x7fcce015afd0 0x7fcce015ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:32:32.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.117+0000 7fccd7fff640 1 --2- 172.21.15.81:0/2197034930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 0x7fcce015e9d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccc80097a0 tx=0x7fccc802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7251254f918e34fc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:32.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.117+0000 7fccddd9b640 1 -- 172.21.15.81:0/2197034930 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8031780 con 0x7fcce0071890 2023-11-08T16:32:32.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.117+0000 7fccddd9b640 1 -- 172.21.15.81:0/2197034930 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc80318e0 con 0x7fcce0071890 2023-11-08T16:32:32.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.117+0000 7fccddd9b640 1 -- 172.21.15.81:0/2197034930 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8031c70 con 0x7fcce0071890 2023-11-08T16:32:32.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.118+0000 7fcce4afc640 1 -- 172.21.15.81:0/2197034930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 msgr2=0x7fcce015e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:32.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.118+0000 7fcce4afc640 1 --2- 172.21.15.81:0/2197034930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 0x7fcce015e9d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fccc80097a0 tx=0x7fccc802b820 comp rx=0 tx=0).stop 2023-11-08T16:32:32.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.118+0000 7fcce4afc640 1 -- 172.21.15.81:0/2197034930 shutdown_connections 2023-11-08T16:32:32.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.118+0000 7fcce4afc640 1 --2- 172.21.15.81:0/2197034930 >> v2:172.21.15.93:3300/0 conn(0x7fcce015afd0 0x7fcce015ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:32.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.118+0000 7fcce4afc640 1 --2- 172.21.15.81:0/2197034930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 0x7fcce015e9d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:32.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.118+0000 7fcce4afc640 1 -- 172.21.15.81:0/2197034930 >> 172.21.15.81:0/2197034930 conn(0x7fcce00cb4e0 msgr2=0x7fcce00cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:32.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.118+0000 7fcce4afc640 1 -- 172.21.15.81:0/2197034930 shutdown_connections 2023-11-08T16:32:32.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.118+0000 7fcce4afc640 1 -- 172.21.15.81:0/2197034930 wait complete. 2023-11-08T16:32:32.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.119+0000 7fcce4afc640 1 Processor -- start 2023-11-08T16:32:32.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.120+0000 7fcce4afc640 1 -- start start 2023-11-08T16:32:32.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fcce4afc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcce00714b0 0x7fcce0148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:32.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fcce4afc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 0x7fcce00c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:32.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fcce4afc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce015afd0 0x7fcce00af640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:32.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fcce4afc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00c64f0 con 0x7fcce015afd0 2023-11-08T16:32:32.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fcce4afc640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00c67d0 con 0x7fcce00714b0 2023-11-08T16:32:32.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fcce4afc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00c6ab0 con 0x7fcce0071890 2023-11-08T16:32:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fccded9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce015afd0 0x7fcce00af640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fccded9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce015afd0 0x7fcce00af640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33622/0 (socket says 172.21.15.81:33622) 2023-11-08T16:32:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fccded9d640 1 -- 172.21.15.81:0/3640837908 learned_addr learned my addr 172.21.15.81:0/3640837908 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fccde59c640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcce00714b0 0x7fcce0148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:32.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fccd7fff640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 0x7fcce00c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:32.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fccded9d640 1 -- 172.21.15.81:0/3640837908 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 msgr2=0x7fcce00c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:32.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fccded9d640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 0x7fcce00c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:32.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fccded9d640 1 -- 172.21.15.81:0/3640837908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcce00714b0 msgr2=0x7fcce0148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:32.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.121+0000 7fccded9d640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcce00714b0 0x7fcce0148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:32.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.122+0000 7fccded9d640 1 -- 172.21.15.81:0/3640837908 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd8009a70 con 0x7fcce015afd0 2023-11-08T16:32:32.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.122+0000 7fccded9d640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce015afd0 0x7fcce00af640 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fccd800efc0 tx=0x7fccd800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:32.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.122+0000 7fccd6ffd640 1 -- 172.21.15.81:0/3640837908 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8009cd0 con 0x7fcce015afd0 2023-11-08T16:32:32.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.122+0000 7fccd6ffd640 1 -- 172.21.15.81:0/3640837908 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd800f040 con 0x7fcce015afd0 2023-11-08T16:32:32.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.122+0000 7fccd6ffd640 1 -- 172.21.15.81:0/3640837908 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8003990 con 0x7fcce015afd0 2023-11-08T16:32:32.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.122+0000 7fcce4afc640 1 -- 172.21.15.81:0/3640837908 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc8009400 con 0x7fcce015afd0 2023-11-08T16:32:32.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.122+0000 7fcce4afc640 1 -- 172.21.15.81:0/3640837908 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce00c7070 con 0x7fcce015afd0 2023-11-08T16:32:32.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.123+0000 7fcce4afc640 1 -- 172.21.15.81:0/3640837908 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce00c7260 con 0x7fcce015afd0 2023-11-08T16:32:32.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.125+0000 7fccd6ffd640 1 -- 172.21.15.81:0/3640837908 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fccd8014480 con 0x7fcce015afd0 2023-11-08T16:32:32.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.126+0000 7fccd6ffd640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fccc006ae70 0x7fccc006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:32.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.126+0000 7fccd6ffd640 1 -- 172.21.15.81:0/3640837908 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5605+0+0 (secure 0 0 0) 0x7fccd808a190 con 0x7fcce015afd0 2023-11-08T16:32:32.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.128+0000 7fccde59c640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fccc006ae70 0x7fccc006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:32.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.129+0000 7fccde59c640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fccc006ae70 0x7fccc006d330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fccd0000f60 tx=0x7fccd0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:32.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.132+0000 7fccd6ffd640 1 -- 172.21.15.81:0/3640837908 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fccd80552b0 con 0x7fcce015afd0 2023-11-08T16:32:32.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.333+0000 7fcce4afc640 1 -- 172.21.15.81:0/3640837908 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcce006fb80 con 0x7fcce015afd0 2023-11-08T16:32:32.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.879+0000 7fccd6ffd640 1 -- 172.21.15.81:0/3640837908 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v48) v1 ==== 187+0+0 (secure 0 0 0) 0x7fccd80588d0 con 0x7fcce015afd0 2023-11-08T16:32:32.883 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data_extra' removed 2023-11-08T16:32:32.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.883+0000 7fcce4afc640 1 -- 172.21.15.81:0/3640837908 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fccc006ae70 msgr2=0x7fccc006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:32.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.883+0000 7fcce4afc640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fccc006ae70 0x7fccc006d330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fccd0000f60 tx=0x7fccd0009040 comp rx=0 tx=0).stop 2023-11-08T16:32:32.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.883+0000 7fcce4afc640 1 -- 172.21.15.81:0/3640837908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce015afd0 msgr2=0x7fcce00af640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:32.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.883+0000 7fcce4afc640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce015afd0 0x7fcce00af640 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fccd800efc0 tx=0x7fccd800c490 comp rx=0 tx=0).stop 2023-11-08T16:32:32.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.883+0000 7fcce4afc640 1 -- 172.21.15.81:0/3640837908 shutdown_connections 2023-11-08T16:32:32.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.883+0000 7fcce4afc640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fccc006ae70 0x7fccc006d330 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:32.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.883+0000 7fcce4afc640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcce015afd0 0x7fcce00af640 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:32.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.883+0000 7fcce4afc640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcce0071890 0x7fcce00c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:32.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.883+0000 7fcce4afc640 1 --2- 172.21.15.81:0/3640837908 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcce00714b0 0x7fcce0148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:32.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.883+0000 7fcce4afc640 1 -- 172.21.15.81:0/3640837908 >> 172.21.15.81:0/3640837908 conn(0x7fcce00cb4e0 msgr2=0x7fcce00ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:32.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.884+0000 7fcce4afc640 1 -- 172.21.15.81:0/3640837908 shutdown_connections 2023-11-08T16:32:32.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:32.884+0000 7fcce4afc640 1 -- 172.21.15.81:0/3640837908 wait complete. 2023-11-08T16:32:32.909 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T16:32:33.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.103+0000 7fe7d7733640 1 Processor -- start 2023-11-08T16:32:33.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.103+0000 7fe7d7733640 1 -- start start 2023-11-08T16:32:33.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.104+0000 7fe7d7733640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe7d0071890 0x7fe7d015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d7733640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe7d015afd0 0x7fe7d015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d7733640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d00b09f0 con 0x7fe7d00714b0 2023-11-08T16:32:33.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d7733640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d006b250 con 0x7fe7d015afd0 2023-11-08T16:32:33.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d7733640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d00ca360 con 0x7fe7d0071890 2023-11-08T16:32:33.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d54cf640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe7d00714b0 0x7fe7d015a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40998/0 (socket says 172.21.15.81:40998) 2023-11-08T16:32:33.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d54cf640 1 -- 172.21.15.81:0/1710158648 learned_addr learned my addr 172.21.15.81:0/1710158648 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:33.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d5cd0640 1 --2- 172.21.15.81:0/1710158648 >> v2:172.21.15.81:3300/0 conn(0x7fe7d015afd0 0x7fe7d015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:33.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7c7fff640 1 -- 172.21.15.81:0/1710158648 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705381453 0 0) 0x7fe7d00b09f0 con 0x7fe7d00714b0 2023-11-08T16:32:33.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7c7fff640 1 -- 172.21.15.81:0/1710158648 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7bc003520 con 0x7fe7d00714b0 2023-11-08T16:32:33.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d4cce640 1 --2- 172.21.15.81:0/1710158648 >> v2:172.21.15.93:3300/0 conn(0x7fe7d0071890 0x7fe7d015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:33.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d5cd0640 1 -- 172.21.15.81:0/1710158648 >> v2:172.21.15.93:3300/0 conn(0x7fe7d0071890 msgr2=0x7fe7d015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:33.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d5cd0640 1 --2- 172.21.15.81:0/1710158648 >> v2:172.21.15.93:3300/0 conn(0x7fe7d0071890 0x7fe7d015e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d5cd0640 1 -- 172.21.15.81:0/1710158648 >> v1:172.21.15.81:6789/0 conn(0x7fe7d00714b0 legacy=0x7fe7d015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:33.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.105+0000 7fe7d5cd0640 1 -- 172.21.15.81:0/1710158648 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d00ca640 con 0x7fe7d015afd0 2023-11-08T16:32:33.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.106+0000 7fe7d5cd0640 1 --2- 172.21.15.81:0/1710158648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7d015afd0 0x7fe7d015ef10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe7d015e930 tx=0x7fe7c802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd9791c4dd6bc834 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:33.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.107+0000 7fe7c7fff640 1 -- 172.21.15.81:0/1710158648 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c802cc70 con 0x7fe7d015afd0 2023-11-08T16:32:33.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.107+0000 7fe7c7fff640 1 -- 172.21.15.81:0/1710158648 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c802cdd0 con 0x7fe7d015afd0 2023-11-08T16:32:33.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.107+0000 7fe7c7fff640 1 -- 172.21.15.81:0/1710158648 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8034660 con 0x7fe7d015afd0 2023-11-08T16:32:33.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.107+0000 7fe7d7733640 1 -- 172.21.15.81:0/1710158648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7d015afd0 msgr2=0x7fe7d015ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:33.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.107+0000 7fe7d7733640 1 --2- 172.21.15.81:0/1710158648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7d015afd0 0x7fe7d015ef10 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe7d015e930 tx=0x7fe7c802bec0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.107+0000 7fe7d7733640 1 -- 172.21.15.81:0/1710158648 shutdown_connections 2023-11-08T16:32:33.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.107+0000 7fe7d7733640 1 --2- 172.21.15.81:0/1710158648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7d015afd0 0x7fe7d015ef10 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.107+0000 7fe7d7733640 1 --2- 172.21.15.81:0/1710158648 >> v2:172.21.15.93:3300/0 conn(0x7fe7d0071890 0x7fe7d015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.107+0000 7fe7d7733640 1 -- 172.21.15.81:0/1710158648 >> 172.21.15.81:0/1710158648 conn(0x7fe7d00cb4e0 msgr2=0x7fe7d00cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:33.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.107+0000 7fe7d7733640 1 -- 172.21.15.81:0/1710158648 shutdown_connections 2023-11-08T16:32:33.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.108+0000 7fe7d7733640 1 -- 172.21.15.81:0/1710158648 wait complete. 2023-11-08T16:32:33.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.109+0000 7fe7d7733640 1 Processor -- start 2023-11-08T16:32:33.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.109+0000 7fe7d7733640 1 -- start start 2023-11-08T16:32:33.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.110+0000 7fe7d7733640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7d00714b0 0x7fe7d01efd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.110+0000 7fe7d7733640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7d0071890 0x7fe7d01f2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.110+0000 7fe7d7733640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7d015afd0 0x7fe7d00cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.110+0000 7fe7d7733640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d00ccab0 con 0x7fe7d00714b0 2023-11-08T16:32:33.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.110+0000 7fe7d7733640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d01f3100 con 0x7fe7d015afd0 2023-11-08T16:32:33.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.110+0000 7fe7d7733640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d01f33e0 con 0x7fe7d0071890 2023-11-08T16:32:33.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.111+0000 7fe7d54cf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7d00714b0 0x7fe7d01efd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:33.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.111+0000 7fe7d54cf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7d00714b0 0x7fe7d01efd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33630/0 (socket says 172.21.15.81:33630) 2023-11-08T16:32:33.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.111+0000 7fe7d54cf640 1 -- 172.21.15.81:0/4160588858 learned_addr learned my addr 172.21.15.81:0/4160588858 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:33.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.111+0000 7fe7d5cd0640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7d015afd0 0x7fe7d00cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:33.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.111+0000 7fe7d4cce640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7d0071890 0x7fe7d01f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:33.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.111+0000 7fe7d54cf640 1 -- 172.21.15.81:0/4160588858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7d0071890 msgr2=0x7fe7d01f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:33.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.111+0000 7fe7d54cf640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7d0071890 0x7fe7d01f2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.111+0000 7fe7d54cf640 1 -- 172.21.15.81:0/4160588858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7d015afd0 msgr2=0x7fe7d00cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:33.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.111+0000 7fe7d54cf640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7d015afd0 0x7fe7d00cced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.111+0000 7fe7d54cf640 1 -- 172.21.15.81:0/4160588858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c8009a70 con 0x7fe7d00714b0 2023-11-08T16:32:33.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.111+0000 7fe7d54cf640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7d00714b0 0x7fe7d01efd60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00c500 tx=0x7fe7cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:33.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.112+0000 7fe7c5ffb640 1 -- 172.21.15.81:0/4160588858 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc005ce0 con 0x7fe7d00714b0 2023-11-08T16:32:33.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.112+0000 7fe7c5ffb640 1 -- 172.21.15.81:0/4160588858 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7cc005e40 con 0x7fe7d00714b0 2023-11-08T16:32:33.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.112+0000 7fe7c5ffb640 1 -- 172.21.15.81:0/4160588858 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc007330 con 0x7fe7d00714b0 2023-11-08T16:32:33.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.112+0000 7fe7d7733640 1 -- 172.21.15.81:0/4160588858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d01f36c0 con 0x7fe7d00714b0 2023-11-08T16:32:33.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.112+0000 7fe7d7733640 1 -- 172.21.15.81:0/4160588858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d01f2c70 con 0x7fe7d00714b0 2023-11-08T16:32:33.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.113+0000 7fe7d7733640 1 -- 172.21.15.81:0/4160588858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d01f2ec0 con 0x7fe7d00714b0 2023-11-08T16:32:33.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.114+0000 7fe7c5ffb640 1 -- 172.21.15.81:0/4160588858 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe7cc007550 con 0x7fe7d00714b0 2023-11-08T16:32:33.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.115+0000 7fe7c5ffb640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe7b406ae70 0x7fe7b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.115+0000 7fe7c5ffb640 1 -- 172.21.15.81:0/4160588858 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5201+0+0 (secure 0 0 0) 0x7fe7cc088ad0 con 0x7fe7d00714b0 2023-11-08T16:32:33.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.115+0000 7fe7d4cce640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe7b406ae70 0x7fe7b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:33.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.116+0000 7fe7d4cce640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe7b406ae70 0x7fe7b406d330 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0006fd0 tx=0x7fe7c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:33.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.120+0000 7fe7c5ffb640 1 -- 172.21.15.81:0/4160588858 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe7d01f2ec0 con 0x7fe7d00714b0 2023-11-08T16:32:33.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.317+0000 7fe7d7733640 1 -- 172.21.15.81:0/4160588858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe7d01f2ec0 con 0x7fe7d00714b0 2023-11-08T16:32:33.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.324+0000 7fe7c5ffb640 1 -- 172.21.15.81:0/4160588858 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe7d01f2ec0 con 0x7fe7d00714b0 2023-11-08T16:32:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 -- 172.21.15.81:0/4160588858 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe7b406ae70 msgr2=0x7fe7b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe7b406ae70 0x7fe7b406d330 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0006fd0 tx=0x7fe7c0008040 comp rx=0 tx=0).stop 2023-11-08T16:32:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 -- 172.21.15.81:0/4160588858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7d00714b0 msgr2=0x7fe7d01efd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7d00714b0 0x7fe7d01efd60 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc00c500 tx=0x7fe7cc00c9c0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 -- 172.21.15.81:0/4160588858 shutdown_connections 2023-11-08T16:32:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe7b406ae70 0x7fe7b406d330 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7d015afd0 0x7fe7d00cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7d0071890 0x7fe7d01f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 --2- 172.21.15.81:0/4160588858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7d00714b0 0x7fe7d01efd60 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 -- 172.21.15.81:0/4160588858 >> 172.21.15.81:0/4160588858 conn(0x7fe7d00cb4e0 msgr2=0x7fe7d0159d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:33.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 -- 172.21.15.81:0/4160588858 shutdown_connections 2023-11-08T16:32:33.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.327+0000 7fe7d7733640 1 -- 172.21.15.81:0/4160588858 wait complete. 2023-11-08T16:32:33.348 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files' 2023-11-08T16:32:33.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.556+0000 7efcee251640 1 Processor -- start 2023-11-08T16:32:33.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.556+0000 7efcee251640 1 -- start start 2023-11-08T16:32:33.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.557+0000 7efcee251640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efce80ca480 0x7efce815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.557+0000 7efcee251640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efce815ea60 0x7efce815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.557+0000 7efcee251640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce806b630 con 0x7efce80ade50 2023-11-08T16:32:33.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.557+0000 7efcee251640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce80b1860 con 0x7efce80ca480 2023-11-08T16:32:33.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.557+0000 7efcee251640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce80ca910 con 0x7efce815ea60 2023-11-08T16:32:33.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.558+0000 7efcdffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efce80ca480 0x7efce815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:33.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.558+0000 7efcdffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efce80ca480 0x7efce815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33632/0 (socket says 172.21.15.81:33632) 2023-11-08T16:32:33.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.558+0000 7efcdffff640 1 -- 172.21.15.81:0/4282034559 learned_addr learned my addr 172.21.15.81:0/4282034559 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:33.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.558+0000 7efcdffff640 1 -- 172.21.15.81:0/4282034559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efce815ea60 msgr2=0x7efce815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:33.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.558+0000 7efcdffff640 1 --2- 172.21.15.81:0/4282034559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efce815ea60 0x7efce815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.558+0000 7efcdffff640 1 -- 172.21.15.81:0/4282034559 >> v1:172.21.15.93:6789/0 conn(0x7efce80ade50 legacy=0x7efce815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:33.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.558+0000 7efcdffff640 1 -- 172.21.15.81:0/4282034559 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce80cabb0 con 0x7efce80ca480 2023-11-08T16:32:33.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.559+0000 7efcdffff640 1 --2- 172.21.15.81:0/4282034559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efce80ca480 0x7efce815ad90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9754b7041857d82 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:33.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.560+0000 7efce6ffd640 1 -- 172.21.15.81:0/4282034559 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0031640 con 0x7efce80ca480 2023-11-08T16:32:33.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.560+0000 7efce6ffd640 1 -- 172.21.15.81:0/4282034559 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd00317a0 con 0x7efce80ca480 2023-11-08T16:32:33.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.560+0000 7efce6ffd640 1 -- 172.21.15.81:0/4282034559 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd0031b30 con 0x7efce80ca480 2023-11-08T16:32:33.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.560+0000 7efcee251640 1 -- 172.21.15.81:0/4282034559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efce80ca480 msgr2=0x7efce815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.560+0000 7efcee251640 1 --2- 172.21.15.81:0/4282034559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efce80ca480 0x7efce815ad90 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009870 tx=0x7efcd002b8f0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.561+0000 7efcee251640 1 -- 172.21.15.81:0/4282034559 shutdown_connections 2023-11-08T16:32:33.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.561+0000 7efcee251640 1 --2- 172.21.15.81:0/4282034559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efce815ea60 0x7efce815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.561+0000 7efcee251640 1 --2- 172.21.15.81:0/4282034559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efce80ca480 0x7efce815ad90 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.561+0000 7efcee251640 1 -- 172.21.15.81:0/4282034559 >> 172.21.15.81:0/4282034559 conn(0x7efce80cb490 msgr2=0x7efce80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:33.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.561+0000 7efcee251640 1 -- 172.21.15.81:0/4282034559 shutdown_connections 2023-11-08T16:32:33.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.561+0000 7efcee251640 1 -- 172.21.15.81:0/4282034559 wait complete. 2023-11-08T16:32:33.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.563+0000 7efcee251640 1 Processor -- start 2023-11-08T16:32:33.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.563+0000 7efcee251640 1 -- start start 2023-11-08T16:32:33.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.564+0000 7efcee251640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efce80ade50 0x7efce81f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.564+0000 7efcee251640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efce80ca480 0x7efce8151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.564+0000 7efcee251640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efce815ea60 0x7efce81570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.564+0000 7efcee251640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce81525c0 con 0x7efce815ea60 2023-11-08T16:32:33.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.564+0000 7efcee251640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce80cd300 con 0x7efce80ca480 2023-11-08T16:32:33.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.564+0000 7efcee251640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce81f39b0 con 0x7efce80ade50 2023-11-08T16:32:33.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.564+0000 7efce7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efce815ea60 0x7efce81570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:33.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.564+0000 7efce7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efce815ea60 0x7efce81570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33640/0 (socket says 172.21.15.81:33640) 2023-11-08T16:32:33.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.564+0000 7efce7fff640 1 -- 172.21.15.81:0/2774126836 learned_addr learned my addr 172.21.15.81:0/2774126836 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:33.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efcdffff640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efce80ca480 0x7efce8151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:33.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efce77fe640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efce80ade50 0x7efce81f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:33.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efce7fff640 1 -- 172.21.15.81:0/2774126836 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efce80ade50 msgr2=0x7efce81f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:33.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efce7fff640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efce80ade50 0x7efce81f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efce7fff640 1 -- 172.21.15.81:0/2774126836 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efce80ca480 msgr2=0x7efce8151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efce7fff640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efce80ca480 0x7efce8151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efce7fff640 1 -- 172.21.15.81:0/2774126836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd00094d0 con 0x7efce815ea60 2023-11-08T16:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efcdffff640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efce80ca480 0x7efce8151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:33.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efce7fff640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efce815ea60 0x7efce81570d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efce000b970 tx=0x7efce000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:33.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efce4ff9640 1 -- 172.21.15.81:0/2774126836 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce000cb40 con 0x7efce815ea60 2023-11-08T16:32:33.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efce4ff9640 1 -- 172.21.15.81:0/2774126836 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efce000ccc0 con 0x7efce815ea60 2023-11-08T16:32:33.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efce4ff9640 1 -- 172.21.15.81:0/2774126836 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efce0014680 con 0x7efce815ea60 2023-11-08T16:32:33.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.565+0000 7efcee251640 1 -- 172.21.15.81:0/2774126836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce80cd550 con 0x7efce815ea60 2023-11-08T16:32:33.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.566+0000 7efcee251640 1 -- 172.21.15.81:0/2774126836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce80cdb10 con 0x7efce815ea60 2023-11-08T16:32:33.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.567+0000 7efcee251640 1 -- 172.21.15.81:0/2774126836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce804ece0 con 0x7efce815ea60 2023-11-08T16:32:33.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.568+0000 7efce4ff9640 1 -- 172.21.15.81:0/2774126836 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efce001d3b0 con 0x7efce815ea60 2023-11-08T16:32:33.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.568+0000 7efce4ff9640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcc806ae70 0x7efcc806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:33.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.569+0000 7efce4ff9640 1 -- 172.21.15.81:0/2774126836 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5201+0+0 (secure 0 0 0) 0x7efce0089460 con 0x7efce815ea60 2023-11-08T16:32:33.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.569+0000 7efce77fe640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcc806ae70 0x7efcc806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:33.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.569+0000 7efce77fe640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcc806ae70 0x7efcc806d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efce80b3290 tx=0x7efcd8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:33.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.573+0000 7efce4ff9640 1 -- 172.21.15.81:0/2774126836 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efce0054660 con 0x7efce815ea60 2023-11-08T16:32:33.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.830+0000 7efcee251640 1 -- 172.21.15.81:0/2774126836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]} v 0) v1 -- 0x7efce80cc1d0 con 0x7efce815ea60 2023-11-08T16:32:33.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.882+0000 7efce4ff9640 1 -- 172.21.15.81:0/2774126836 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_few_files"]}]=0 v190) v1 ==== 158+0+0 (secure 0 0 0) 0x7efce0057c80 con 0x7efce815ea60 2023-11-08T16:32:33.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 -- 172.21.15.81:0/2774126836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcc806ae70 msgr2=0x7efcc806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:33.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcc806ae70 0x7efcc806d330 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7efce80b3290 tx=0x7efcd8009040 comp rx=0 tx=0).stop 2023-11-08T16:32:33.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 -- 172.21.15.81:0/2774126836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efce815ea60 msgr2=0x7efce81570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:33.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efce815ea60 0x7efce81570d0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efce000b970 tx=0x7efce000be30 comp rx=0 tx=0).stop 2023-11-08T16:32:33.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 -- 172.21.15.81:0/2774126836 shutdown_connections 2023-11-08T16:32:33.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efcc806ae70 0x7efcc806d330 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efce815ea60 0x7efce81570d0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efce80ca480 0x7efce8151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 --2- 172.21.15.81:0/2774126836 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efce80ade50 0x7efce81f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:33.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 -- 172.21.15.81:0/2774126836 >> 172.21.15.81:0/2774126836 conn(0x7efce80cb490 msgr2=0x7efce8153e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:33.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 -- 172.21.15.81:0/2774126836 shutdown_connections 2023-11-08T16:32:33.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:33.885+0000 7efcee251640 1 -- 172.21.15.81:0/2774126836 wait complete. 2023-11-08T16:32:33.906 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_few_files (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-11-08T16:32:33.907 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_many_files_many_workers (tasks.cephfs.test_data_scan.TestDataScan) 2023-11-08T16:32:33.907 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:32:34.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.121+0000 7f217f989640 1 Processor -- start 2023-11-08T16:32:34.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.121+0000 7f217f989640 1 -- start start 2023-11-08T16:32:34.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.123+0000 7f217f989640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f21780ca480 0x7f217815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.123+0000 7f217f989640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f217815ea60 0x7f217815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.123+0000 7f217f989640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f217806b630 con 0x7f21780ade50 2023-11-08T16:32:34.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.123+0000 7f217f989640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21780b1860 con 0x7f21780ca480 2023-11-08T16:32:34.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.123+0000 7f217f989640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21780ca910 con 0x7f217815ea60 2023-11-08T16:32:34.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.123+0000 7f217cf24640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f21780ca480 0x7f217815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:34.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.123+0000 7f217cf24640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f21780ca480 0x7f217815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33656/0 (socket says 172.21.15.81:33656) 2023-11-08T16:32:34.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.123+0000 7f217cf24640 1 -- 172.21.15.81:0/1799878676 learned_addr learned my addr 172.21.15.81:0/1799878676 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:34.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.124+0000 7f217df26640 1 --2- 172.21.15.81:0/1799878676 >> v2:172.21.15.93:3300/0 conn(0x7f217815ea60 0x7f217815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:34.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.124+0000 7f217cf24640 1 -- 172.21.15.81:0/1799878676 >> v2:172.21.15.93:3300/0 conn(0x7f217815ea60 msgr2=0x7f217815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:34.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.124+0000 7f217cf24640 1 --2- 172.21.15.81:0/1799878676 >> v2:172.21.15.93:3300/0 conn(0x7f217815ea60 0x7f217815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.124+0000 7f217cf24640 1 -- 172.21.15.81:0/1799878676 >> v1:172.21.15.93:6789/0 conn(0x7f21780ade50 legacy=0x7f217815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:34.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.124+0000 7f217cf24640 1 -- 172.21.15.81:0/1799878676 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21780cabb0 con 0x7f21780ca480 2023-11-08T16:32:34.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.124+0000 7f217df26640 1 --2- 172.21.15.81:0/1799878676 >> v2:172.21.15.93:3300/0 conn(0x7f217815ea60 0x7f217815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:34.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.125+0000 7f217cf24640 1 --2- 172.21.15.81:0/1799878676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21780ca480 0x7f217815ad90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f21780ca220 tx=0x7f216802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec26a1f3080ded9c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:34.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.125+0000 7f216ffff640 1 -- 172.21.15.81:0/1799878676 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168031870 con 0x7f21780ca480 2023-11-08T16:32:34.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.125+0000 7f216ffff640 1 -- 172.21.15.81:0/1799878676 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21680319d0 con 0x7f21780ca480 2023-11-08T16:32:34.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.125+0000 7f216ffff640 1 -- 172.21.15.81:0/1799878676 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168031d60 con 0x7f21780ca480 2023-11-08T16:32:34.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.126+0000 7f217f989640 1 -- 172.21.15.81:0/1799878676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21780ca480 msgr2=0x7f217815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:34.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.126+0000 7f217f989640 1 --2- 172.21.15.81:0/1799878676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21780ca480 0x7f217815ad90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f21780ca220 tx=0x7f216802b910 comp rx=0 tx=0).stop 2023-11-08T16:32:34.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.126+0000 7f217f989640 1 -- 172.21.15.81:0/1799878676 shutdown_connections 2023-11-08T16:32:34.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.126+0000 7f217f989640 1 --2- 172.21.15.81:0/1799878676 >> v2:172.21.15.93:3300/0 conn(0x7f217815ea60 0x7f217815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.126+0000 7f217f989640 1 --2- 172.21.15.81:0/1799878676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21780ca480 0x7f217815ad90 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.126+0000 7f217f989640 1 -- 172.21.15.81:0/1799878676 >> 172.21.15.81:0/1799878676 conn(0x7f21780cb490 msgr2=0x7f21780cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:34.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.126+0000 7f217f989640 1 -- 172.21.15.81:0/1799878676 shutdown_connections 2023-11-08T16:32:34.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.126+0000 7f217f989640 1 -- 172.21.15.81:0/1799878676 wait complete. 2023-11-08T16:32:34.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.128+0000 7f217f989640 1 Processor -- start 2023-11-08T16:32:34.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.129+0000 7f217f989640 1 -- start start 2023-11-08T16:32:34.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217f989640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21780ade50 0x7f21781f5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217f989640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21780ca480 0x7f21780cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217f989640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f217815ea60 0x7f21780aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217f989640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21781542e0 con 0x7f21780ca480 2023-11-08T16:32:34.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217f989640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21781f60f0 con 0x7f21780ade50 2023-11-08T16:32:34.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217f989640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21781539b0 con 0x7f217815ea60 2023-11-08T16:32:34.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217cf24640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21780ca480 0x7f21780cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:34.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217cf24640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21780ca480 0x7f21780cdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33658/0 (socket says 172.21.15.81:33658) 2023-11-08T16:32:34.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217cf24640 1 -- 172.21.15.81:0/1340317246 learned_addr learned my addr 172.21.15.81:0/1340317246 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:34.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217cf24640 1 -- 172.21.15.81:0/1340317246 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f217815ea60 msgr2=0x7f21780aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:34.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217cf24640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f217815ea60 0x7f21780aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217cf24640 1 -- 172.21.15.81:0/1340317246 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21780ade50 msgr2=0x7f21781f5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:34.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217cf24640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21780ade50 0x7f21781f5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.130+0000 7f217cf24640 1 -- 172.21.15.81:0/1340317246 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21680094d0 con 0x7f21780ca480 2023-11-08T16:32:34.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.131+0000 7f217cf24640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21780ca480 0x7f21780cdaf0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f216802b950 tx=0x7f21680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:34.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.131+0000 7f216dffb640 1 -- 172.21.15.81:0/1340317246 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2168031860 con 0x7f21780ca480 2023-11-08T16:32:34.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.131+0000 7f216dffb640 1 -- 172.21.15.81:0/1340317246 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21680443f0 con 0x7f21780ca480 2023-11-08T16:32:34.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.131+0000 7f216dffb640 1 -- 172.21.15.81:0/1340317246 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21680447a0 con 0x7f21780ca480 2023-11-08T16:32:34.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.131+0000 7f217f989640 1 -- 172.21.15.81:0/1340317246 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2178153c90 con 0x7f21780ca480 2023-11-08T16:32:34.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.131+0000 7f217f989640 1 -- 172.21.15.81:0/1340317246 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21780ce030 con 0x7f21780ca480 2023-11-08T16:32:34.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.132+0000 7f217f989640 1 -- 172.21.15.81:0/1340317246 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f217804ece0 con 0x7f21780ca480 2023-11-08T16:32:34.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.133+0000 7f216dffb640 1 -- 172.21.15.81:0/1340317246 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f216804d450 con 0x7f21780ca480 2023-11-08T16:32:34.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.134+0000 7f216dffb640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f215c06ae70 0x7f215c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.134+0000 7f216dffb640 1 -- 172.21.15.81:0/1340317246 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f216802d040 con 0x7f21780ca480 2023-11-08T16:32:34.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.134+0000 7f217d725640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f215c06ae70 0x7f215c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:34.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.135+0000 7f217d725640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f215c06ae70 0x7f215c06d330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2174001560 tx=0x7f2174007990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:34.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.139+0000 7f216dffb640 1 -- 172.21.15.81:0/1340317246 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2168076e90 con 0x7f21780ca480 2023-11-08T16:32:34.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.329+0000 7f217f989640 1 -- 172.21.15.81:0/1340317246 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21780cc1d0 con 0x7f21780ca480 2023-11-08T16:32:34.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.330+0000 7f216dffb640 1 -- 172.21.15.81:0/1340317246 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+13314 (secure 0 0 0) 0x7f2168035070 con 0x7f21780ca480 2023-11-08T16:32:34.333 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:34.334 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":48,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:32:32.871966+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/2942618483":"2023-11-09T16:32:28.843246+0000","172.21.15.81:6836/2942618483":"2023-11-09T16:32:28.843246+0000","172.21.15.81:6837/1759199271":"2023-11-09T16:31:23.834479+0000","172.21.15.81:6836/1759199271":"2023-11-09T16:31:23.834479+0000","172.21.15.81:6837/3931731986":"2023-11-09T16:31:16.385652+0000","172.21.15.81:6836/3931731986":"2023-11-09T16:31:16.385652+0000","172.21.15.81:6837/773130363":"2023-11-09T16:31:14.352977+0000","172.21.15.81:6836/773130363":"2023-11-09T16:31:14.352977+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:32:34.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 -- 172.21.15.81:0/1340317246 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f215c06ae70 msgr2=0x7f215c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:34.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f215c06ae70 0x7f215c06d330 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f2174001560 tx=0x7f2174007990 comp rx=0 tx=0).stop 2023-11-08T16:32:34.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 -- 172.21.15.81:0/1340317246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21780ca480 msgr2=0x7f21780cdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:34.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21780ca480 0x7f21780cdaf0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f216802b950 tx=0x7f21680046c0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 -- 172.21.15.81:0/1340317246 shutdown_connections 2023-11-08T16:32:34.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f215c06ae70 0x7f215c06d330 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f217815ea60 0x7f21780aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f21780ca480 0x7f21780cdaf0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 --2- 172.21.15.81:0/1340317246 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21780ade50 0x7f21781f5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 -- 172.21.15.81:0/1340317246 >> 172.21.15.81:0/1340317246 conn(0x7f21780cb490 msgr2=0x7f2178157300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:34.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 -- 172.21.15.81:0/1340317246 shutdown_connections 2023-11-08T16:32:34.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.333+0000 7f217f989640 1 -- 172.21.15.81:0/1340317246 wait complete. 2023-11-08T16:32:34.356 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:32:34.357 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:32:34.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.570+0000 7f54e90b4640 1 Processor -- start 2023-11-08T16:32:34.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.570+0000 7f54e90b4640 1 -- start start 2023-11-08T16:32:34.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.572+0000 7f54e90b4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 0x7f54e415e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.572+0000 7f54e90b4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f54e40c9200 0x7f54e40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.572+0000 7f54e90b4640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54e415e8d0 con 0x7f54e40ce390 2023-11-08T16:32:34.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.572+0000 7f54e90b4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54e40ca2f0 con 0x7f54e40c9200 2023-11-08T16:32:34.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.572+0000 7f54e90b4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40ce770 con 0x7f54e406be20 2023-11-08T16:32:34.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.572+0000 7f54e359e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f54e40c9200 0x7f54e40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:34.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.572+0000 7f54e359e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f54e40c9200 0x7f54e40ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33670/0 (socket says 172.21.15.81:33670) 2023-11-08T16:32:34.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.572+0000 7f54e359e640 1 -- 172.21.15.81:0/1942179580 learned_addr learned my addr 172.21.15.81:0/1942179580 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:34.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.572+0000 7f54e259c640 1 --2- 172.21.15.81:0/1942179580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 0x7f54e415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:34.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.572+0000 7f54e359e640 1 -- 172.21.15.81:0/1942179580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 msgr2=0x7f54e415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:34.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.573+0000 7f54e359e640 1 --2- 172.21.15.81:0/1942179580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 0x7f54e415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.573+0000 7f54e359e640 1 -- 172.21.15.81:0/1942179580 >> v1:172.21.15.93:6789/0 conn(0x7f54e40ce390 legacy=0x7f54e406b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:34.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.573+0000 7f54e359e640 1 -- 172.21.15.81:0/1942179580 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54e40c96e0 con 0x7f54e40c9200 2023-11-08T16:32:34.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.573+0000 7f54e259c640 1 --2- 172.21.15.81:0/1942179580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 0x7f54e415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:34.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.574+0000 7f54e359e640 1 --2- 172.21.15.81:0/1942179580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e40c9200 0x7f54e40ac8b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009e50 tx=0x7f54dc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d6b22ad73b8873 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:34.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.574+0000 7f54e1d9b640 1 -- 172.21.15.81:0/1942179580 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc02cb00 con 0x7f54e40c9200 2023-11-08T16:32:34.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.574+0000 7f54e1d9b640 1 -- 172.21.15.81:0/1942179580 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54dc02cc60 con 0x7f54e40c9200 2023-11-08T16:32:34.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.574+0000 7f54e1d9b640 1 -- 172.21.15.81:0/1942179580 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54dc0343f0 con 0x7f54e40c9200 2023-11-08T16:32:34.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.575+0000 7f54e90b4640 1 -- 172.21.15.81:0/1942179580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e40c9200 msgr2=0x7f54e40ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:34.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.575+0000 7f54e90b4640 1 --2- 172.21.15.81:0/1942179580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e40c9200 0x7f54e40ac8b0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009e50 tx=0x7f54dc02bf60 comp rx=0 tx=0).stop 2023-11-08T16:32:34.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.575+0000 7f54e90b4640 1 -- 172.21.15.81:0/1942179580 shutdown_connections 2023-11-08T16:32:34.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.575+0000 7f54e90b4640 1 --2- 172.21.15.81:0/1942179580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e40c9200 0x7f54e40ac8b0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.575+0000 7f54e90b4640 1 --2- 172.21.15.81:0/1942179580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 0x7f54e415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.575+0000 7f54e90b4640 1 -- 172.21.15.81:0/1942179580 >> 172.21.15.81:0/1942179580 conn(0x7f54e40cb580 msgr2=0x7f54e40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:34.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.575+0000 7f54e90b4640 1 -- 172.21.15.81:0/1942179580 shutdown_connections 2023-11-08T16:32:34.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.575+0000 7f54e90b4640 1 -- 172.21.15.81:0/1942179580 wait complete. 2023-11-08T16:32:34.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.577+0000 7f54e90b4640 1 Processor -- start 2023-11-08T16:32:34.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.577+0000 7f54e90b4640 1 -- start start 2023-11-08T16:32:34.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.578+0000 7f54e90b4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 0x7f54e40c2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.578+0000 7f54e90b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e40c9200 0x7f54e40b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.578+0000 7f54e90b4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ce390 0x7f54e40c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.578+0000 7f54e90b4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40c5140 con 0x7f54e40c9200 2023-11-08T16:32:34.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.578+0000 7f54e90b4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40b4420 con 0x7f54e40ce390 2023-11-08T16:32:34.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.578+0000 7f54e90b4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54e40c2cf0 con 0x7f54e406be20 2023-11-08T16:32:34.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.578+0000 7f54e259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e40c9200 0x7f54e40b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:34.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.578+0000 7f54e259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e40c9200 0x7f54e40b3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33684/0 (socket says 172.21.15.81:33684) 2023-11-08T16:32:34.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.578+0000 7f54e259c640 1 -- 172.21.15.81:0/2184585857 learned_addr learned my addr 172.21.15.81:0/2184585857 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:34.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.578+0000 7f54e2d9d640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 0x7f54e40c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:34.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54e259c640 1 -- 172.21.15.81:0/2184585857 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 msgr2=0x7f54e40c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:34.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54e259c640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 0x7f54e40c2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54e359e640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ce390 0x7f54e40c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:34.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54e259c640 1 -- 172.21.15.81:0/2184585857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ce390 msgr2=0x7f54e40c7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:34.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54e259c640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ce390 0x7f54e40c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54e259c640 1 -- 172.21.15.81:0/2184585857 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54dc009b30 con 0x7f54e40c9200 2023-11-08T16:32:34.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54e2d9d640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 0x7f54e40c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:34.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54e259c640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e40c9200 0x7f54e40b3ee0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c7f0 tx=0x7f54cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:34.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54db7fe640 1 -- 172.21.15.81:0/2184585857 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc007a30 con 0x7f54e40c9200 2023-11-08T16:32:34.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54db7fe640 1 -- 172.21.15.81:0/2184585857 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc007b90 con 0x7f54e40c9200 2023-11-08T16:32:34.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54e90b4640 1 -- 172.21.15.81:0/2184585857 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54e40c2fd0 con 0x7f54e40c9200 2023-11-08T16:32:34.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54db7fe640 1 -- 172.21.15.81:0/2184585857 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00ce40 con 0x7f54e40c9200 2023-11-08T16:32:34.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.579+0000 7f54e90b4640 1 -- 172.21.15.81:0/2184585857 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54e40c3590 con 0x7f54e40c9200 2023-11-08T16:32:34.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.580+0000 7f54e90b4640 1 -- 172.21.15.81:0/2184585857 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54e404ece0 con 0x7f54e40c9200 2023-11-08T16:32:34.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.581+0000 7f54db7fe640 1 -- 172.21.15.81:0/2184585857 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f54cc0113f0 con 0x7f54e40c9200 2023-11-08T16:32:34.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.582+0000 7f54db7fe640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54c406ae70 0x7f54c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:34.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.582+0000 7f54db7fe640 1 -- 172.21.15.81:0/2184585857 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f54cc089690 con 0x7f54e40c9200 2023-11-08T16:32:34.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.583+0000 7f54e2d9d640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54c406ae70 0x7f54c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:34.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.583+0000 7f54e2d9d640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54c406ae70 0x7f54c406d330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f54d400a830 tx=0x7f54d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:34.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.586+0000 7f54db7fe640 1 -- 172.21.15.81:0/2184585857 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f54cc054890 con 0x7f54e40c9200 2023-11-08T16:32:34.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.776+0000 7f54e90b4640 1 -- 172.21.15.81:0/2184585857 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54e40c42f0 con 0x7f54e40c9200 2023-11-08T16:32:34.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.777+0000 7f54db7fe640 1 -- 172.21.15.81:0/2184585857 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54cc057eb0 con 0x7f54e40c9200 2023-11-08T16:32:34.780 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:32:34.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 -- 172.21.15.81:0/2184585857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54c406ae70 msgr2=0x7f54c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:34.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54c406ae70 0x7f54c406d330 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f54d400a830 tx=0x7f54d4008040 comp rx=0 tx=0).stop 2023-11-08T16:32:34.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 -- 172.21.15.81:0/2184585857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e40c9200 msgr2=0x7f54e40b3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:34.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e40c9200 0x7f54e40b3ee0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00c7f0 tx=0x7f54cc00cbf0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 -- 172.21.15.81:0/2184585857 shutdown_connections 2023-11-08T16:32:34.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f54c406ae70 0x7f54c406d330 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f54e40ce390 0x7f54e40c7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f54e40c9200 0x7f54e40b3ee0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 --2- 172.21.15.81:0/2184585857 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f54e406be20 0x7f54e40c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:34.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 -- 172.21.15.81:0/2184585857 >> 172.21.15.81:0/2184585857 conn(0x7f54e40cb580 msgr2=0x7f54e40af930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:34.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 -- 172.21.15.81:0/2184585857 shutdown_connections 2023-11-08T16:32:34.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:34.779+0000 7f54e90b4640 1 -- 172.21.15.81:0/2184585857 wait complete. 2023-11-08T16:32:34.799 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers' 2023-11-08T16:32:35.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.011+0000 7f79b2e75640 1 Processor -- start 2023-11-08T16:32:35.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.012+0000 7f79b2e75640 1 -- start start 2023-11-08T16:32:35.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.013+0000 7f79b2e75640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f79ac0adf20 0x7f79ac165230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:35.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.013+0000 7f79b2e75640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ac165b80 con 0x7f79ac0adb40 2023-11-08T16:32:35.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.013+0000 7f79b2e75640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ac0ce3f0 con 0x7f79ac165770 2023-11-08T16:32:35.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.013+0000 7f79b2e75640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0cb340 con 0x7f79ac0adf20 2023-11-08T16:32:35.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.013+0000 7f79b0c11640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f79ac0adb40 0x7f79ac0c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:41012/0 (socket says 172.21.15.81:41012) 2023-11-08T16:32:35.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.013+0000 7f79b0c11640 1 -- 172.21.15.81:0/1805684397 learned_addr learned my addr 172.21.15.81:0/1805684397 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:35.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455151186 0 0) 0x7f79ac165b80 con 0x7f79ac0adb40 2023-11-08T16:32:35.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79abfff640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f79ac0adf20 0x7f79ac165230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:35.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7998003520 con 0x7f79ac0adb40 2023-11-08T16:32:35.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3115276939 0 0) 0x7f7998003520 con 0x7f79ac0adb40 2023-11-08T16:32:35.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f79ac0adf20 msgr2=0x7f79ac165230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:35.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f79ac0adf20 0x7f79ac165230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:35.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 >> v1:172.21.15.93:6789/0 conn(0x7f79ac165770 legacy=0x7f79ac06b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:35.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac0cac80 con 0x7f79ac0adb40 2023-11-08T16:32:35.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f799c0019f0 con 0x7f79ac0adb40 2023-11-08T16:32:35.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79abfff640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f79ac0adf20 0x7f79ac165230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:32:35.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 >> v1:172.21.15.81:6789/0 conn(0x7f79ac0adb40 legacy=0x7f79ac0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:35.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7998003a80 0x7f7998003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:35.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f79ac0adf20 0x7f79980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:35.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.014+0000 7f79ab7fe640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7998004e20 0x7f799800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:35.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79980055b0 con 0x7f7998004e20 2023-11-08T16:32:35.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f799800bec0 con 0x7f7998003a80 2023-11-08T16:32:35.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799800c000 con 0x7f79ac0adf20 2023-11-08T16:32:35.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79b1412640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7998004e20 0x7f799800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:35.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79b0c11640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7998003a80 0x7f7998003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:35.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79abfff640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f79ac0adf20 0x7f79980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:35.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79b1412640 1 -- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f79ac0adf20 msgr2=0x7f79980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:35.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79b1412640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f79ac0adf20 0x7f79980047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:35.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79b1412640 1 -- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7998003a80 msgr2=0x7f7998003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:35.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79b1412640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7998003a80 0x7f7998003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:35.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79b1412640 1 -- 172.21.15.81:0/1805684397 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998003520 con 0x7f7998004e20 2023-11-08T16:32:35.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.015+0000 7f79b0c11640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7998003a80 0x7f7998003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:35.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.016+0000 7f79b1412640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7998004e20 0x7f799800b980 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f79a4001780 tx=0x7f79980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b21a8a671345c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:35.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.016+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a40052d0 con 0x7f7998004e20 2023-11-08T16:32:35.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.016+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a402e050 con 0x7f7998004e20 2023-11-08T16:32:35.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.017+0000 7f79ab7fe640 1 -- 172.21.15.81:0/1805684397 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a40035f0 con 0x7f7998004e20 2023-11-08T16:32:35.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.017+0000 7f79b2e75640 1 -- 172.21.15.81:0/1805684397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7998004e20 msgr2=0x7f799800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:35.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.017+0000 7f79b2e75640 1 --2- 172.21.15.81:0/1805684397 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7998004e20 0x7f799800b980 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f79a4001780 tx=0x7f79980057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:35.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.017+0000 7f79b1412640 1 -- 172.21.15.81:0/1805684397 reap_dead start 2023-11-08T16:32:35.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.017+0000 7f79b2e75640 1 -- 172.21.15.81:0/1805684397 shutdown_connections 2023-11-08T16:32:35.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.017+0000 7f79b2e75640 1 -- 172.21.15.81:0/1805684397 >> 172.21.15.81:0/1805684397 conn(0x7f79ac0cb490 msgr2=0x7f79ac0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:35.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.017+0000 7f79b2e75640 1 -- 172.21.15.81:0/1805684397 shutdown_connections 2023-11-08T16:32:35.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.017+0000 7f79b2e75640 1 -- 172.21.15.81:0/1805684397 wait complete. 2023-11-08T16:32:35.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.019+0000 7f79b2e75640 1 Processor -- start 2023-11-08T16:32:35.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.019+0000 7f79b2e75640 1 -- start start 2023-11-08T16:32:35.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.020+0000 7f79b2e75640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7998004e20 0x7f79ac159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:35.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.020+0000 7f79b2e75640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79ac0c34e0 0x7f79ac06c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:35.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.020+0000 7f79b2e75640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79ac06c6e0 0x7f79ac0c3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:35.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.020+0000 7f79b2e75640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac1597b0 con 0x7f79ac06c6e0 2023-11-08T16:32:35.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.020+0000 7f79b2e75640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac06cb60 con 0x7f79ac0c34e0 2023-11-08T16:32:35.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.020+0000 7f79b2e75640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ac0c2440 con 0x7f7998004e20 2023-11-08T16:32:35.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.020+0000 7f79b1412640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79ac06c6e0 0x7f79ac0c3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:35.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.020+0000 7f79b1412640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79ac06c6e0 0x7f79ac0c3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33712/0 (socket says 172.21.15.81:33712) 2023-11-08T16:32:35.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.020+0000 7f79b1412640 1 -- 172.21.15.81:0/1074335061 learned_addr learned my addr 172.21.15.81:0/1074335061 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:35.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79b1412640 1 -- 172.21.15.81:0/1074335061 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7998004e20 msgr2=0x7f79ac159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:35.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79b0c11640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7998004e20 0x7f79ac159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:35.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79b1412640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7998004e20 0x7f79ac159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:35.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79b1412640 1 -- 172.21.15.81:0/1074335061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79ac0c34e0 msgr2=0x7f79ac06c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:35.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79b1412640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79ac0c34e0 0x7f79ac06c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:35.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79b1412640 1 -- 172.21.15.81:0/1074335061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ac0c2720 con 0x7f79ac06c6e0 2023-11-08T16:32:35.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79b1412640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79ac06c6e0 0x7f79ac0c3d60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f79a4005040 tx=0x7f79a4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:35.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79a97fa640 1 -- 172.21.15.81:0/1074335061 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4043070 con 0x7f79ac06c6e0 2023-11-08T16:32:35.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79a97fa640 1 -- 172.21.15.81:0/1074335061 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a4007a30 con 0x7f79ac06c6e0 2023-11-08T16:32:35.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79a97fa640 1 -- 172.21.15.81:0/1074335061 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a40067d0 con 0x7f79ac06c6e0 2023-11-08T16:32:35.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79b2e75640 1 -- 172.21.15.81:0/1074335061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79ac0c2a00 con 0x7f79ac06c6e0 2023-11-08T16:32:35.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.021+0000 7f79b2e75640 1 -- 172.21.15.81:0/1074335061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ac0c2fc0 con 0x7f79ac06c6e0 2023-11-08T16:32:35.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.023+0000 7f79b2e75640 1 -- 172.21.15.81:0/1074335061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ac0c31b0 con 0x7f79ac06c6e0 2023-11-08T16:32:35.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.024+0000 7f79a97fa640 1 -- 172.21.15.81:0/1074335061 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f79a4006950 con 0x7f79ac06c6e0 2023-11-08T16:32:35.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.024+0000 7f79a97fa640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f798c06ae70 0x7f798c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:35.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.024+0000 7f79a97fa640 1 -- 172.21.15.81:0/1074335061 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f79a40aa2d0 con 0x7f79ac06c6e0 2023-11-08T16:32:35.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.025+0000 7f79b0c11640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f798c06ae70 0x7f798c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:35.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.025+0000 7f79b0c11640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f798c06ae70 0x7f798c06d330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f799c001370 tx=0x7f799c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:35.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.029+0000 7f79a97fa640 1 -- 172.21.15.81:0/1074335061 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f79a4075570 con 0x7f79ac06c6e0 2023-11-08T16:32:35.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.285+0000 7f79b2e75640 1 -- 172.21.15.81:0/1074335061 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]} v 0) v1 -- 0x7f79ac04ece0 con 0x7f79ac06c6e0 2023-11-08T16:32:35.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.897+0000 7f79a97fa640 1 -- 172.21.15.81:0/1074335061 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]}]=0 v192) v1 ==== 175+0+0 (secure 0 0 0) 0x7f79a4031e80 con 0x7f79ac06c6e0 2023-11-08T16:32:35.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.900+0000 7f79b2e75640 1 -- 172.21.15.81:0/1074335061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f798c06ae70 msgr2=0x7f798c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:35.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.900+0000 7f79b2e75640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f798c06ae70 0x7f798c06d330 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f799c001370 tx=0x7f799c009040 comp rx=0 tx=0).stop 2023-11-08T16:32:35.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.900+0000 7f79b2e75640 1 -- 172.21.15.81:0/1074335061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79ac06c6e0 msgr2=0x7f79ac0c3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:35.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.900+0000 7f79b2e75640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79ac06c6e0 0x7f79ac0c3d60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f79a4005040 tx=0x7f79a4001120 comp rx=0 tx=0).stop 2023-11-08T16:32:35.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.900+0000 7f79b2e75640 1 -- 172.21.15.81:0/1074335061 shutdown_connections 2023-11-08T16:32:35.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.900+0000 7f79b2e75640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f798c06ae70 0x7f798c06d330 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:35.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.900+0000 7f79b2e75640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79ac06c6e0 0x7f79ac0c3d60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:35.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.900+0000 7f79b2e75640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f79ac0c34e0 0x7f79ac06c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:35.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.900+0000 7f79b2e75640 1 --2- 172.21.15.81:0/1074335061 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7998004e20 0x7f79ac159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:35.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.900+0000 7f79b2e75640 1 -- 172.21.15.81:0/1074335061 >> 172.21.15.81:0/1074335061 conn(0x7f79ac0cb490 msgr2=0x7f79ac160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:35.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.901+0000 7f79b2e75640 1 -- 172.21.15.81:0/1074335061 shutdown_connections 2023-11-08T16:32:35.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:35.901+0000 7f79b2e75640 1 -- 172.21.15.81:0/1074335061 wait complete. 2023-11-08T16:32:35.925 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T16:32:36.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.139+0000 7fe44f656640 1 Processor -- start 2023-11-08T16:32:36.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.139+0000 7fe44f656640 1 -- start start 2023-11-08T16:32:36.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.141+0000 7fe44f656640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe44815d4c0 0x7fe4480cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:36.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.141+0000 7fe44f656640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe4480d39a0 0x7fe4480cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:36.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.141+0000 7fe44f656640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4480cef70 con 0x7fe4480cfaf0 2023-11-08T16:32:36.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.141+0000 7fe44f656640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4480cff80 con 0x7fe4480d39a0 2023-11-08T16:32:36.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.141+0000 7fe44f656640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4480cec90 con 0x7fe44815d4c0 2023-11-08T16:32:36.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.141+0000 7fe44dbf3640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe4480cfaf0 0x7fe4480d0d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:41014/0 (socket says 172.21.15.81:41014) 2023-11-08T16:32:36.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.141+0000 7fe44dbf3640 1 -- 172.21.15.81:0/250522512 learned_addr learned my addr 172.21.15.81:0/250522512 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:36.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.141+0000 7fe44d3f2640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe44815d4c0 0x7fe4480cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:36.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe44cbf1640 1 --2- 172.21.15.81:0/250522512 >> v2:172.21.15.93:3300/0 conn(0x7fe4480d39a0 0x7fe4480cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:36.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3079847489 0 0) 0x7fe440001110 con 0x7fe4480cfaf0 2023-11-08T16:32:36.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe430003520 con 0x7fe4480cfaf0 2023-11-08T16:32:36.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3752319198 0 0) 0x7fe430003520 con 0x7fe4480cfaf0 2023-11-08T16:32:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe44815d4c0 msgr2=0x7fe4480cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe44815d4c0 0x7fe4480cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 >> v2:172.21.15.93:3300/0 conn(0x7fe4480d39a0 msgr2=0x7fe4480cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:36.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 --2- 172.21.15.81:0/250522512 >> v2:172.21.15.93:3300/0 conn(0x7fe4480d39a0 0x7fe4480cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4480b5e70 con 0x7fe4480cfaf0 2023-11-08T16:32:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fe440001800 con 0x7fe4480cfaf0 2023-11-08T16:32:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 >> v1:172.21.15.81:6789/0 conn(0x7fe4480cfaf0 legacy=0x7fe4480d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 0x7fe430003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:36.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe44815d4c0 0x7fe4300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe430004d00 0x7fe430005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4480cff80 con 0x7fe4300039e0 2023-11-08T16:32:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4480cec90 con 0x7fe430004d00 2023-11-08T16:32:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe43000ba90 con 0x7fe44815d4c0 2023-11-08T16:32:36.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe44cbf1640 1 --2- 172.21.15.81:0/250522512 >> v2:172.21.15.93:3300/0 conn(0x7fe4480d39a0 0x7fe4480cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:32:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.142+0000 7fe44d3f2640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 0x7fe430003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.143+0000 7fe44dbf3640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe430004d00 0x7fe430005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.143+0000 7fe44cbf1640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe44815d4c0 0x7fe4300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.143+0000 7fe44d3f2640 1 -- 172.21.15.81:0/250522512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe44815d4c0 msgr2=0x7fe4300047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:36.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.143+0000 7fe44d3f2640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe44815d4c0 0x7fe4300047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:36.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.143+0000 7fe44d3f2640 1 -- 172.21.15.81:0/250522512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe430004d00 msgr2=0x7fe430005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:36.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.143+0000 7fe44d3f2640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe430004d00 0x7fe430005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:36.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.143+0000 7fe44d3f2640 1 -- 172.21.15.81:0/250522512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe430003520 con 0x7fe4300039e0 2023-11-08T16:32:36.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.144+0000 7fe44d3f2640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 0x7fe430003e00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe438009ac0 tx=0x7fe4300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0d0c8c3d329b20 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:36.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.144+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43802c6b0 con 0x7fe4300039e0 2023-11-08T16:32:36.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.144+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43802c810 con 0x7fe4300039e0 2023-11-08T16:32:36.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.144+0000 7fe43ffff640 1 -- 172.21.15.81:0/250522512 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43802cba0 con 0x7fe4300039e0 2023-11-08T16:32:36.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.145+0000 7fe44f656640 1 -- 172.21.15.81:0/250522512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 msgr2=0x7fe430003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:36.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.145+0000 7fe44f656640 1 --2- 172.21.15.81:0/250522512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 0x7fe430003e00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe438009ac0 tx=0x7fe4300057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:36.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.145+0000 7fe44dbf3640 1 -- 172.21.15.81:0/250522512 reap_dead start 2023-11-08T16:32:36.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.145+0000 7fe44f656640 1 -- 172.21.15.81:0/250522512 shutdown_connections 2023-11-08T16:32:36.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.145+0000 7fe44f656640 1 -- 172.21.15.81:0/250522512 >> 172.21.15.81:0/250522512 conn(0x7fe4480d2e80 msgr2=0x7fe4480d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:36.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.145+0000 7fe44f656640 1 -- 172.21.15.81:0/250522512 shutdown_connections 2023-11-08T16:32:36.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.145+0000 7fe44f656640 1 -- 172.21.15.81:0/250522512 wait complete. 2023-11-08T16:32:36.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.146+0000 7fe44f656640 1 Processor -- start 2023-11-08T16:32:36.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.147+0000 7fe44f656640 1 -- start start 2023-11-08T16:32:36.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.147+0000 7fe44f656640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 0x7fe4480b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:36.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.147+0000 7fe44f656640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe44815d8a0 0x7fe4481f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:36.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44f656640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4481f6b00 0x7fe448155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:36.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44f656640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe448159cd0 con 0x7fe4300039e0 2023-11-08T16:32:36.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44f656640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4481f7130 con 0x7fe44815d8a0 2023-11-08T16:32:36.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44f656640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4481f7410 con 0x7fe4481f6b00 2023-11-08T16:32:36.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44d3f2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 0x7fe4480b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:36.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44d3f2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 0x7fe4480b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33726/0 (socket says 172.21.15.81:33726) 2023-11-08T16:32:36.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44d3f2640 1 -- 172.21.15.81:0/644512758 learned_addr learned my addr 172.21.15.81:0/644512758 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:36.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44d3f2640 1 -- 172.21.15.81:0/644512758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4481f6b00 msgr2=0x7fe448155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:36.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44d3f2640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4481f6b00 0x7fe448155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:36.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44cbf1640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe44815d8a0 0x7fe4481f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:36.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44d3f2640 1 -- 172.21.15.81:0/644512758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe44815d8a0 msgr2=0x7fe4481f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:36.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44d3f2640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe44815d8a0 0x7fe4481f65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:36.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44d3f2640 1 -- 172.21.15.81:0/644512758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4480cb600 con 0x7fe4300039e0 2023-11-08T16:32:36.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.148+0000 7fe44d3f2640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 0x7fe4480b4be0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe43802c250 tx=0x7fe438004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:36.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.149+0000 7fe43dffb640 1 -- 172.21.15.81:0/644512758 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438003810 con 0x7fe4300039e0 2023-11-08T16:32:36.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.149+0000 7fe43dffb640 1 -- 172.21.15.81:0/644512758 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe438003970 con 0x7fe4300039e0 2023-11-08T16:32:36.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.149+0000 7fe43dffb640 1 -- 172.21.15.81:0/644512758 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe438003ce0 con 0x7fe4300039e0 2023-11-08T16:32:36.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.149+0000 7fe44f656640 1 -- 172.21.15.81:0/644512758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4480cb8e0 con 0x7fe4300039e0 2023-11-08T16:32:36.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.149+0000 7fe44f656640 1 -- 172.21.15.81:0/644512758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4480b5120 con 0x7fe4300039e0 2023-11-08T16:32:36.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.150+0000 7fe44f656640 1 -- 172.21.15.81:0/644512758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4480d2860 con 0x7fe4300039e0 2023-11-08T16:32:36.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.151+0000 7fe43dffb640 1 -- 172.21.15.81:0/644512758 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe438034440 con 0x7fe4300039e0 2023-11-08T16:32:36.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.152+0000 7fe43dffb640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe43406ae70 0x7fe43406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:36.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.152+0000 7fe43dffb640 1 -- 172.21.15.81:0/644512758 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5201+0+0 (secure 0 0 0) 0x7fe4380ab300 con 0x7fe4300039e0 2023-11-08T16:32:36.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.152+0000 7fe44cbf1640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe43406ae70 0x7fe43406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:36.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.153+0000 7fe44cbf1640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe43406ae70 0x7fe43406d330 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe44815dc80 tx=0x7fe44400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:36.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.156+0000 7fe43dffb640 1 -- 172.21.15.81:0/644512758 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe438076760 con 0x7fe4300039e0 2023-11-08T16:32:36.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.345+0000 7fe44f656640 1 -- 172.21.15.81:0/644512758 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe44804ece0 con 0x7fe4300039e0 2023-11-08T16:32:36.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.352+0000 7fe43dffb640 1 -- 172.21.15.81:0/644512758 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe4380044e0 con 0x7fe4300039e0 2023-11-08T16:32:36.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 -- 172.21.15.81:0/644512758 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe43406ae70 msgr2=0x7fe43406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:36.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe43406ae70 0x7fe43406d330 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe44815dc80 tx=0x7fe44400b040 comp rx=0 tx=0).stop 2023-11-08T16:32:36.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 -- 172.21.15.81:0/644512758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 msgr2=0x7fe4480b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:36.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 0x7fe4480b4be0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe43802c250 tx=0x7fe438004730 comp rx=0 tx=0).stop 2023-11-08T16:32:36.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 -- 172.21.15.81:0/644512758 shutdown_connections 2023-11-08T16:32:36.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe43406ae70 0x7fe43406d330 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:36.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4481f6b00 0x7fe448155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:36.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe44815d8a0 0x7fe4481f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:36.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 --2- 172.21.15.81:0/644512758 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4300039e0 0x7fe4480b4be0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:36.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 -- 172.21.15.81:0/644512758 >> 172.21.15.81:0/644512758 conn(0x7fe4480d2e80 msgr2=0x7fe44806ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:36.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 -- 172.21.15.81:0/644512758 shutdown_connections 2023-11-08T16:32:36.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:36.355+0000 7fe44f656640 1 -- 172.21.15.81:0/644512758 wait complete. 2023-11-08T16:32:36.375 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T16:32:36.376 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:32:36.479 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:32:36.607 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T16:32:36.607 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:32:36.608 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:32:36.667 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:32:36.667 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:32:36.668 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:32:36.728 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:32:36.729 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:32:36.729 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:32:36.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:32:36.790 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:32:36.791 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:32:36.850 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:32:36.851 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T16:32:36.851 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:32:37.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.069+0000 7faec524f640 1 Processor -- start 2023-11-08T16:32:37.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.069+0000 7faec524f640 1 -- start start 2023-11-08T16:32:37.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.071+0000 7faec524f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7faec00ade50 0x7faec00ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.071+0000 7faec524f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec015ea60 0x7faec015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.071+0000 7faec524f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec006b630 con 0x7faec015a680 2023-11-08T16:32:37.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.071+0000 7faec524f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec00b1860 con 0x7faec00ade50 2023-11-08T16:32:37.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.071+0000 7faec524f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec00ca9c0 con 0x7faec015ea60 2023-11-08T16:32:37.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.071+0000 7faebed9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7faec00ade50 0x7faec00ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.071+0000 7faebf59e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec015ea60 0x7faec015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.071+0000 7faebf59e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec015ea60 0x7faec015eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:39198/0 (socket says 172.21.15.81:39198) 2023-11-08T16:32:37.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.071+0000 7faebf59e640 1 -- 172.21.15.81:0/2004663049 learned_addr learned my addr 172.21.15.81:0/2004663049 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:37.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.072+0000 7faebed9d640 1 -- 172.21.15.81:0/2004663049 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec015ea60 msgr2=0x7faec015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:37.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.072+0000 7faebed9d640 1 --2- 172.21.15.81:0/2004663049 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec015ea60 0x7faec015eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.072+0000 7faebed9d640 1 -- 172.21.15.81:0/2004663049 >> v1:172.21.15.93:6789/0 conn(0x7faec015a680 legacy=0x7faec015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:37.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.072+0000 7faebed9d640 1 -- 172.21.15.81:0/2004663049 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec00cabb0 con 0x7faec00ade50 2023-11-08T16:32:37.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.072+0000 7faebf59e640 1 --2- 172.21.15.81:0/2004663049 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec015ea60 0x7faec015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:32:37.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.074+0000 7faebed9d640 1 --2- 172.21.15.81:0/2004663049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faec00ade50 0x7faec00ca480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faeb00097a0 tx=0x7faeb002b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a581ed795438a3e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:37.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.074+0000 7faebe59c640 1 -- 172.21.15.81:0/2004663049 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb002d040 con 0x7faec00ade50 2023-11-08T16:32:37.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.074+0000 7faebe59c640 1 -- 172.21.15.81:0/2004663049 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb0031810 con 0x7faec00ade50 2023-11-08T16:32:37.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.074+0000 7faebe59c640 1 -- 172.21.15.81:0/2004663049 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb0031ba0 con 0x7faec00ade50 2023-11-08T16:32:37.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.075+0000 7faec524f640 1 -- 172.21.15.81:0/2004663049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faec00ade50 msgr2=0x7faec00ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:37.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.075+0000 7faec524f640 1 --2- 172.21.15.81:0/2004663049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faec00ade50 0x7faec00ca480 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faeb00097a0 tx=0x7faeb002b8b0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.075+0000 7faec524f640 1 -- 172.21.15.81:0/2004663049 shutdown_connections 2023-11-08T16:32:37.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.075+0000 7faec524f640 1 --2- 172.21.15.81:0/2004663049 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec015ea60 0x7faec015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.075+0000 7faec524f640 1 --2- 172.21.15.81:0/2004663049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faec00ade50 0x7faec00ca480 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.075+0000 7faec524f640 1 -- 172.21.15.81:0/2004663049 >> 172.21.15.81:0/2004663049 conn(0x7faec00cb490 msgr2=0x7faec00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:37.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.075+0000 7faec524f640 1 -- 172.21.15.81:0/2004663049 shutdown_connections 2023-11-08T16:32:37.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.075+0000 7faec524f640 1 -- 172.21.15.81:0/2004663049 wait complete. 2023-11-08T16:32:37.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.077+0000 7faec524f640 1 Processor -- start 2023-11-08T16:32:37.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.077+0000 7faec524f640 1 -- start start 2023-11-08T16:32:37.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.078+0000 7faec524f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec00ade50 0x7faec01f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.078+0000 7faec524f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec015a680 0x7faec0153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.078+0000 7faec524f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faec015ea60 0x7faec01f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.078+0000 7faec524f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec01f6980 con 0x7faec015a680 2023-11-08T16:32:37.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.078+0000 7faec524f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec01f2c10 con 0x7faec015ea60 2023-11-08T16:32:37.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.078+0000 7faec524f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec01f3ca0 con 0x7faec00ade50 2023-11-08T16:32:37.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.079+0000 7faeb7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec015a680 0x7faec0153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.079+0000 7faeb7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec015a680 0x7faec0153df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33738/0 (socket says 172.21.15.81:33738) 2023-11-08T16:32:37.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.079+0000 7faeb7fff640 1 -- 172.21.15.81:0/974202044 learned_addr learned my addr 172.21.15.81:0/974202044 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:37.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.079+0000 7faebed9d640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec00ade50 0x7faec01f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.079+0000 7faebf59e640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faec015ea60 0x7faec01f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.079+0000 7faeb7fff640 1 -- 172.21.15.81:0/974202044 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec00ade50 msgr2=0x7faec01f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:37.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.079+0000 7faeb7fff640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec00ade50 0x7faec01f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.079+0000 7faeb7fff640 1 -- 172.21.15.81:0/974202044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faec015ea60 msgr2=0x7faec01f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:37.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.079+0000 7faeb7fff640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faec015ea60 0x7faec01f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.079+0000 7faeb7fff640 1 -- 172.21.15.81:0/974202044 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb8009b30 con 0x7faec015a680 2023-11-08T16:32:37.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.079+0000 7faeb7fff640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec015a680 0x7faec0153df0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7faea800c480 tx=0x7faea800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:37.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.080+0000 7faeb77fe640 1 -- 172.21.15.81:0/974202044 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea80052d0 con 0x7faec015a680 2023-11-08T16:32:37.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.080+0000 7faeb77fe640 1 -- 172.21.15.81:0/974202044 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faea8005430 con 0x7faec015a680 2023-11-08T16:32:37.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.080+0000 7faec524f640 1 -- 172.21.15.81:0/974202044 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb0009400 con 0x7faec015a680 2023-11-08T16:32:37.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.080+0000 7faeb77fe640 1 -- 172.21.15.81:0/974202044 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea8012870 con 0x7faec015a680 2023-11-08T16:32:37.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.080+0000 7faec524f640 1 -- 172.21.15.81:0/974202044 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec0155750 con 0x7faec015a680 2023-11-08T16:32:37.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.081+0000 7faec524f640 1 -- 172.21.15.81:0/974202044 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec01f6ac0 con 0x7faec015a680 2023-11-08T16:32:37.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.082+0000 7faeb77fe640 1 -- 172.21.15.81:0/974202044 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faea80129d0 con 0x7faec015a680 2023-11-08T16:32:37.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.083+0000 7faeb77fe640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faea006ae70 0x7faea006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.083+0000 7faeb77fe640 1 -- 172.21.15.81:0/974202044 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5201+0+0 (secure 0 0 0) 0x7faea8088fd0 con 0x7faec015a680 2023-11-08T16:32:37.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.083+0000 7faebed9d640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faea006ae70 0x7faea006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.084+0000 7faebed9d640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faea006ae70 0x7faea006d330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faeb0036000 tx=0x7faeb003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:37.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.087+0000 7faeb77fe640 1 -- 172.21.15.81:0/974202044 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faea8054260 con 0x7faec015a680 2023-11-08T16:32:37.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.317+0000 7faec524f640 1 -- 172.21.15.81:0/974202044 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faec004ece0 con 0x7faec015a680 2023-11-08T16:32:37.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.318+0000 7faeb77fe640 1 -- 172.21.15.81:0/974202044 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3690 (secure 0 0 0) 0x7faea8057880 con 0x7faec015a680 2023-11-08T16:32:37.321 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:37.321 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4787,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3654712142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3654712142},{"type":"v1","addr":"172.21.15.93:6834","nonce":3654712142}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5014,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/66363251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":66363251},{"type":"v1","addr":"172.21.15.93:6835","nonce":66363251}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":30},{"gid":5652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/4031527046","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":4031527046},{"type":"v1","addr":"172.21.15.81:6835","nonce":4031527046}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5901,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/816134959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":816134959},{"type":"v1","addr":"172.21.15.81:6837","nonce":816134959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34}],"filesystems":[]} 2023-11-08T16:32:37.321 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 34 2023-11-08T16:32:37.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.320+0000 7faec524f640 1 -- 172.21.15.81:0/974202044 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faea006ae70 msgr2=0x7faea006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:37.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.320+0000 7faec524f640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faea006ae70 0x7faea006d330 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faeb0036000 tx=0x7faeb003c040 comp rx=0 tx=0).stop 2023-11-08T16:32:37.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.320+0000 7faec524f640 1 -- 172.21.15.81:0/974202044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec015a680 msgr2=0x7faec0153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:37.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.320+0000 7faec524f640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec015a680 0x7faec0153df0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7faea800c480 tx=0x7faea800c940 comp rx=0 tx=0).stop 2023-11-08T16:32:37.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.321+0000 7faec524f640 1 -- 172.21.15.81:0/974202044 shutdown_connections 2023-11-08T16:32:37.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.321+0000 7faec524f640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faea006ae70 0x7faea006d330 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.321+0000 7faec524f640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faec015ea60 0x7faec01f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.321+0000 7faec524f640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faec015a680 0x7faec0153df0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.321+0000 7faec524f640 1 --2- 172.21.15.81:0/974202044 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faec00ade50 0x7faec01f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.321+0000 7faec524f640 1 -- 172.21.15.81:0/974202044 >> 172.21.15.81:0/974202044 conn(0x7faec00cb490 msgr2=0x7faec00afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:37.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.321+0000 7faec524f640 1 -- 172.21.15.81:0/974202044 shutdown_connections 2023-11-08T16:32:37.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.321+0000 7faec524f640 1 -- 172.21.15.81:0/974202044 wait complete. 2023-11-08T16:32:37.342 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T16:32:37.343 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T16:32:37.343 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T16:32:37.343 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:32:37.344 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T16:32:37.344 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T16:32:37.344 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T16:32:37.344 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:32:37.345 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T16:32:37.345 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T16:32:37.345 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T16:32:37.345 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:32:37.346 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T16:32:37.346 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T16:32:37.346 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T16:32:37.347 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:32:37.361 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T16:32:37.362 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T16:32:37.363 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T16:32:37.363 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T16:32:37.367 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T16:32:37.367 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T16:32:37.369 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T16:32:37.369 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T16:32:37.410 INFO:tasks.ceph.mds.a:Started 2023-11-08T16:32:37.411 DEBUG:teuthology.parallel:result is None 2023-11-08T16:32:37.417 INFO:tasks.ceph.mds.d:Started 2023-11-08T16:32:37.418 DEBUG:teuthology.parallel:result is None 2023-11-08T16:32:37.418 INFO:tasks.ceph.mds.c:Started 2023-11-08T16:32:37.419 DEBUG:teuthology.parallel:result is None 2023-11-08T16:32:37.425 INFO:tasks.ceph.mds.b:Started 2023-11-08T16:32:37.425 DEBUG:teuthology.parallel:result is None 2023-11-08T16:32:37.426 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T16:32:37.570 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:32:37.570 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:32:37.577 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2023-11-08T16:32:37.581 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:32:37.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.615+0000 7f7e82a90640 1 Processor -- start 2023-11-08T16:32:37.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.615+0000 7f7e82a90640 1 -- start start 2023-11-08T16:32:37.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.617+0000 7f7e82a90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c15aeb0 0x7f7e7c0ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.617+0000 7f7e82a90640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7e7c0c9550 0x7f7e7c06b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.617+0000 7f7e82a90640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7c06cc60 con 0x7f7e7c0c90f0 2023-11-08T16:32:37.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.617+0000 7f7e82a90640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7c0cb3e0 con 0x7f7e7c0c9550 2023-11-08T16:32:37.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.617+0000 7f7e82a90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c0cae20 con 0x7f7e7c15aeb0 2023-11-08T16:32:37.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.617+0000 7f7e8082c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c15aeb0 0x7f7e7c0ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.617+0000 7f7e8082c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c15aeb0 0x7f7e7c0ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:39260/0 (socket says 172.21.15.81:39260) 2023-11-08T16:32:37.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.617+0000 7f7e8082c640 1 -- 172.21.15.81:0/1253416632 learned_addr learned my addr 172.21.15.81:0/1253416632 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:37.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.617+0000 7f7e8102d640 1 --2- 172.21.15.81:0/1253416632 >> v2:172.21.15.93:3300/0 conn(0x7f7e7c0c9550 0x7f7e7c06b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.618+0000 7f7e8082c640 1 -- 172.21.15.81:0/1253416632 >> v2:172.21.15.93:3300/0 conn(0x7f7e7c0c9550 msgr2=0x7f7e7c06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:37.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.618+0000 7f7e8082c640 1 --2- 172.21.15.81:0/1253416632 >> v2:172.21.15.93:3300/0 conn(0x7f7e7c0c9550 0x7f7e7c06b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.618+0000 7f7e8082c640 1 -- 172.21.15.81:0/1253416632 >> v1:172.21.15.93:6789/0 conn(0x7f7e7c0c90f0 legacy=0x7f7e7c06b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:37.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.618+0000 7f7e8082c640 1 -- 172.21.15.81:0/1253416632 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c0cb100 con 0x7f7e7c15aeb0 2023-11-08T16:32:37.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.619+0000 7f7e8082c640 1 --2- 172.21.15.81:0/1253416632 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c15aeb0 0x7f7e7c0ca4a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009850 tx=0x7f7e7802b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faa767999e6b93c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:37.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.619+0000 7f7e737fe640 1 -- 172.21.15.81:0/1253416632 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78004230 con 0x7f7e7c15aeb0 2023-11-08T16:32:37.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.619+0000 7f7e737fe640 1 -- 172.21.15.81:0/1253416632 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e78004390 con 0x7f7e7c15aeb0 2023-11-08T16:32:37.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.619+0000 7f7e737fe640 1 -- 172.21.15.81:0/1253416632 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e78003810 con 0x7f7e7c15aeb0 2023-11-08T16:32:37.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.620+0000 7f7e82a90640 1 -- 172.21.15.81:0/1253416632 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c15aeb0 msgr2=0x7f7e7c0ca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:37.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.620+0000 7f7e82a90640 1 --2- 172.21.15.81:0/1253416632 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c15aeb0 0x7f7e7c0ca4a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7e78009850 tx=0x7f7e7802b960 comp rx=0 tx=0).stop 2023-11-08T16:32:37.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.620+0000 7f7e82a90640 1 -- 172.21.15.81:0/1253416632 shutdown_connections 2023-11-08T16:32:37.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.620+0000 7f7e82a90640 1 --2- 172.21.15.81:0/1253416632 >> v2:172.21.15.93:3300/0 conn(0x7f7e7c0c9550 0x7f7e7c06b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.620+0000 7f7e82a90640 1 --2- 172.21.15.81:0/1253416632 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c15aeb0 0x7f7e7c0ca4a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.620+0000 7f7e82a90640 1 -- 172.21.15.81:0/1253416632 >> 172.21.15.81:0/1253416632 conn(0x7f7e7c0ceaa0 msgr2=0x7f7e7c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:37.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.620+0000 7f7e82a90640 1 -- 172.21.15.81:0/1253416632 shutdown_connections 2023-11-08T16:32:37.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.620+0000 7f7e82a90640 1 -- 172.21.15.81:0/1253416632 wait complete. 2023-11-08T16:32:37.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.622+0000 7f7e82a90640 1 Processor -- start 2023-11-08T16:32:37.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.622+0000 7f7e82a90640 1 -- start start 2023-11-08T16:32:37.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e82a90640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e7c0c90f0 0x7f7e7c1ed940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e82a90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c0c9550 0x7f7e7c154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e82a90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e7c15aeb0 0x7f7e7c154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e82a90640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c156e00 con 0x7f7e7c15aeb0 2023-11-08T16:32:37.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e82a90640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c1570e0 con 0x7f7e7c0c90f0 2023-11-08T16:32:37.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e82a90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c1573c0 con 0x7f7e7c0c9550 2023-11-08T16:32:37.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e8102d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e7c15aeb0 0x7f7e7c154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e73fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c0c9550 0x7f7e7c154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e73fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c0c9550 0x7f7e7c154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:39274/0 (socket says 172.21.15.81:39274) 2023-11-08T16:32:37.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e73fff640 1 -- 172.21.15.81:0/1207316179 learned_addr learned my addr 172.21.15.81:0/1207316179 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:37.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e8082c640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e7c0c90f0 0x7f7e7c1ed940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e73fff640 1 -- 172.21.15.81:0/1207316179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e7c0c90f0 msgr2=0x7f7e7c1ed940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:37.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e73fff640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e7c0c90f0 0x7f7e7c1ed940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e73fff640 1 -- 172.21.15.81:0/1207316179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e7c15aeb0 msgr2=0x7f7e7c154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:37.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e73fff640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e7c15aeb0 0x7f7e7c154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:37.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e73fff640 1 -- 172.21.15.81:0/1207316179 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e78009460 con 0x7f7e7c0c9550 2023-11-08T16:32:37.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.623+0000 7f7e8102d640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e7c15aeb0 0x7f7e7c154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:32:37.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.624+0000 7f7e73fff640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c0c9550 0x7f7e7c154a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00c400 tx=0x7f7e6c00c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:37.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.624+0000 7f7e717fa640 1 -- 172.21.15.81:0/1207316179 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c0052d0 con 0x7f7e7c0c9550 2023-11-08T16:32:37.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.624+0000 7f7e717fa640 1 -- 172.21.15.81:0/1207316179 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c005430 con 0x7f7e7c0c9550 2023-11-08T16:32:37.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.624+0000 7f7e82a90640 1 -- 172.21.15.81:0/1207316179 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7c1576a0 con 0x7f7e7c0c9550 2023-11-08T16:32:37.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.624+0000 7f7e717fa640 1 -- 172.21.15.81:0/1207316179 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c003830 con 0x7f7e7c0c9550 2023-11-08T16:32:37.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.624+0000 7f7e82a90640 1 -- 172.21.15.81:0/1207316179 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7c1ede80 con 0x7f7e7c0c9550 2023-11-08T16:32:37.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.625+0000 7f7e82a90640 1 -- 172.21.15.81:0/1207316179 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c0ca9b0 con 0x7f7e7c0c9550 2023-11-08T16:32:37.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.626+0000 7f7e717fa640 1 -- 172.21.15.81:0/1207316179 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7e6c003990 con 0x7f7e7c0c9550 2023-11-08T16:32:37.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.627+0000 7f7e717fa640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e6806aea0 0x7f7e6806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:37.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.627+0000 7f7e717fa640 1 -- 172.21.15.81:0/1207316179 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f7e6c088930 con 0x7f7e7c0c9550 2023-11-08T16:32:37.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.627+0000 7f7e8082c640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e6806aea0 0x7f7e6806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:37.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.628+0000 7f7e8082c640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e6806aea0 0x7f7e6806d360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7e78035000 tx=0x7f7e78008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:37.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.636+0000 7f7e717fa640 1 -- 172.21.15.81:0/1207316179 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7e6c053be0 con 0x7f7e7c0c9550 2023-11-08T16:32:37.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:37.828+0000 7f7e82a90640 1 -- 172.21.15.81:0/1207316179 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7e7c157500 con 0x7f7e7c0c9550 2023-11-08T16:32:38.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.950+0000 7f7e717fa640 1 -- 172.21.15.81:0/1207316179 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v53) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7e7c157500 con 0x7f7e7c0c9550 2023-11-08T16:32:38.952 INFO:teuthology.orchestra.run.smithi081.stderr: removed all blocklist entries 2023-11-08T16:32:38.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.952+0000 7f7e82a90640 1 -- 172.21.15.81:0/1207316179 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e6806aea0 msgr2=0x7f7e6806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:38.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.952+0000 7f7e82a90640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e6806aea0 0x7f7e6806d360 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7e78035000 tx=0x7f7e78008030 comp rx=0 tx=0).stop 2023-11-08T16:32:38.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.952+0000 7f7e82a90640 1 -- 172.21.15.81:0/1207316179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c0c9550 msgr2=0x7f7e7c154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:38.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.952+0000 7f7e82a90640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c0c9550 0x7f7e7c154a90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c00c400 tx=0x7f7e6c00c8c0 comp rx=0 tx=0).stop 2023-11-08T16:32:38.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.953+0000 7f7e82a90640 1 -- 172.21.15.81:0/1207316179 shutdown_connections 2023-11-08T16:32:38.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.953+0000 7f7e82a90640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7e6806aea0 0x7f7e6806d360 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:38.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.953+0000 7f7e82a90640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7e7c15aeb0 0x7f7e7c154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:38.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.953+0000 7f7e82a90640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7e7c0c9550 0x7f7e7c154a90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:38.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.953+0000 7f7e82a90640 1 --2- 172.21.15.81:0/1207316179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7e7c0c90f0 0x7f7e7c1ed940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:38.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.953+0000 7f7e82a90640 1 -- 172.21.15.81:0/1207316179 >> 172.21.15.81:0/1207316179 conn(0x7f7e7c0ceaa0 msgr2=0x7f7e7c0b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:38.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.953+0000 7f7e82a90640 1 -- 172.21.15.81:0/1207316179 shutdown_connections 2023-11-08T16:32:38.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:38.953+0000 7f7e82a90640 1 -- 172.21.15.81:0/1207316179 wait complete. 2023-11-08T16:32:38.978 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T16:32:39.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.189+0000 7f713b139640 1 Processor -- start 2023-11-08T16:32:39.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.189+0000 7f713b139640 1 -- start start 2023-11-08T16:32:39.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.190+0000 7f713b139640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f713415aeb0 0x7f71340ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.190+0000 7f713b139640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f713406cc60 con 0x7f71340c90f0 2023-11-08T16:32:39.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.190+0000 7f713b139640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71340cae20 con 0x7f71340c9580 2023-11-08T16:32:39.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.190+0000 7f713b139640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71340cb3e0 con 0x7f713415aeb0 2023-11-08T16:32:39.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.191+0000 7f7133fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f71340c90f0 0x7f713406b2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:41032/0 (socket says 172.21.15.81:41032) 2023-11-08T16:32:39.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.191+0000 7f7133fff640 1 -- 172.21.15.81:0/1070879050 learned_addr learned my addr 172.21.15.81:0/1070879050 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:39.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.191+0000 7f7138ed5640 1 --2- 172.21.15.81:0/1070879050 >> v2:172.21.15.93:3300/0 conn(0x7f713415aeb0 0x7f71340ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:39.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.191+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880115813 0 0) 0x7f713406cc60 con 0x7f71340c90f0 2023-11-08T16:32:39.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.191+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7124003520 con 0x7f71340c90f0 2023-11-08T16:32:39.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2071401050 0 0) 0x7f7124003520 con 0x7f71340c90f0 2023-11-08T16:32:39.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 >> v2:172.21.15.93:3300/0 conn(0x7f713415aeb0 msgr2=0x7f71340ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:39.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 --2- 172.21.15.81:0/1070879050 >> v2:172.21.15.93:3300/0 conn(0x7f713415aeb0 0x7f71340ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 >> v1:172.21.15.93:6789/0 conn(0x7f71340c9580 legacy=0x7f713406b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340cb100 con 0x7f71340c90f0 2023-11-08T16:32:39.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f7120001670 con 0x7f71340c90f0 2023-11-08T16:32:39.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 >> v1:172.21.15.81:6789/0 conn(0x7f71340c90f0 legacy=0x7f713406b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 --2- 172.21.15.81:0/1070879050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71240039c0 0x7f7124003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 --2- 172.21.15.81:0/1070879050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71240047c0 0x7f7124004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 --2- 172.21.15.81:0/1070879050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71240051d0 0x7f712400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340cb3e0 con 0x7f71240051d0 2023-11-08T16:32:39.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712400c120 con 0x7f71240039c0 2023-11-08T16:32:39.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712400c2e0 con 0x7f71240047c0 2023-11-08T16:32:39.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f7138ed5640 1 --2- 172.21.15.81:0/1070879050 >> v2:172.21.15.93:3300/0 conn(0x7f713415aeb0 0x7f71340ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:32:39.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71396d6640 1 --2- 172.21.15.81:0/1070879050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71240051d0 0x7f712400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:39.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71396d6640 1 -- 172.21.15.81:0/1070879050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71240047c0 msgr2=0x7f7124004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:39.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71396d6640 1 --2- 172.21.15.81:0/1070879050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71240047c0 0x7f7124004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71396d6640 1 -- 172.21.15.81:0/1070879050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71240039c0 msgr2=0x7f7124003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:39.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71396d6640 1 --2- 172.21.15.81:0/1070879050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71240039c0 0x7f7124003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.192+0000 7f71396d6640 1 -- 172.21.15.81:0/1070879050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7124003520 con 0x7f71240051d0 2023-11-08T16:32:39.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.193+0000 7f71396d6640 1 -- 172.21.15.81:0/1070879050 reap_dead start 2023-11-08T16:32:39.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.193+0000 7f71396d6640 1 --2- 172.21.15.81:0/1070879050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71240051d0 0x7f712400b980 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f712c001e50 tx=0x7f71240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6577396b584985 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:39.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.194+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c003b70 con 0x7f71240051d0 2023-11-08T16:32:39.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.194+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712c003d10 con 0x7f71240051d0 2023-11-08T16:32:39.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.194+0000 7f71337fe640 1 -- 172.21.15.81:0/1070879050 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712c005310 con 0x7f71240051d0 2023-11-08T16:32:39.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.194+0000 7f713b139640 1 -- 172.21.15.81:0/1070879050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71240051d0 msgr2=0x7f712400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.194+0000 7f713b139640 1 --2- 172.21.15.81:0/1070879050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71240051d0 0x7f712400b980 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f712c001e50 tx=0x7f71240057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.194+0000 7f713b139640 1 -- 172.21.15.81:0/1070879050 shutdown_connections 2023-11-08T16:32:39.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.194+0000 7f713b139640 1 --2- 172.21.15.81:0/1070879050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71240051d0 0x7f712400b980 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.194+0000 7f713b139640 1 -- 172.21.15.81:0/1070879050 >> 172.21.15.81:0/1070879050 conn(0x7f71340ceaa0 msgr2=0x7f71340cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:39.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.195+0000 7f713b139640 1 -- 172.21.15.81:0/1070879050 shutdown_connections 2023-11-08T16:32:39.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.195+0000 7f713b139640 1 -- 172.21.15.81:0/1070879050 wait complete. 2023-11-08T16:32:39.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.196+0000 7f713b139640 1 Processor -- start 2023-11-08T16:32:39.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.197+0000 7f713b139640 1 -- start start 2023-11-08T16:32:39.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.197+0000 7f713b139640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71240051d0 0x7f71340c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.197+0000 7f713b139640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f713415de50 0x7f71341f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f713b139640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71341f6670 0x7f71340cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f713b139640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7134156820 con 0x7f713415de50 2023-11-08T16:32:39.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f713b139640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71341f5950 con 0x7f71341f6670 2023-11-08T16:32:39.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f713b139640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71340c73f0 con 0x7f71240051d0 2023-11-08T16:32:39.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f7133fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f713415de50 0x7f71341f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:39.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f7133fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f713415de50 0x7f71341f6130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33790/0 (socket says 172.21.15.81:33790) 2023-11-08T16:32:39.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f7133fff640 1 -- 172.21.15.81:0/452321526 learned_addr learned my addr 172.21.15.81:0/452321526 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:39.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f7138ed5640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71240051d0 0x7f71340c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:39.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f7133fff640 1 -- 172.21.15.81:0/452321526 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71240051d0 msgr2=0x7f71340c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f7133fff640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71240051d0 0x7f71340c6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f7133fff640 1 -- 172.21.15.81:0/452321526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71341f6670 msgr2=0x7f71340cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f7133fff640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71341f6670 0x7f71340cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f7133fff640 1 -- 172.21.15.81:0/452321526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340c76d0 con 0x7f713415de50 2023-11-08T16:32:39.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.198+0000 7f7133fff640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f713415de50 0x7f71341f6130 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f712000c730 tx=0x7f712000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:39.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.199+0000 7f71317fa640 1 -- 172.21.15.81:0/452321526 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200072d0 con 0x7f713415de50 2023-11-08T16:32:39.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.199+0000 7f71317fa640 1 -- 172.21.15.81:0/452321526 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7120007430 con 0x7f713415de50 2023-11-08T16:32:39.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.199+0000 7f71317fa640 1 -- 172.21.15.81:0/452321526 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71200047e0 con 0x7f713415de50 2023-11-08T16:32:39.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.199+0000 7f713b139640 1 -- 172.21.15.81:0/452321526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71340c79b0 con 0x7f713415de50 2023-11-08T16:32:39.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.199+0000 7f713b139640 1 -- 172.21.15.81:0/452321526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71340c7100 con 0x7f713415de50 2023-11-08T16:32:39.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.200+0000 7f713b139640 1 -- 172.21.15.81:0/452321526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71340743f0 con 0x7f713415de50 2023-11-08T16:32:39.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.201+0000 7f71317fa640 1 -- 172.21.15.81:0/452321526 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7120004b50 con 0x7f713415de50 2023-11-08T16:32:39.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.202+0000 7f71317fa640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f711806aea0 0x7f711806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.202+0000 7f71317fa640 1 -- 172.21.15.81:0/452321526 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7120088a00 con 0x7f713415de50 2023-11-08T16:32:39.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.202+0000 7f7138ed5640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f711806aea0 0x7f711806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:39.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.203+0000 7f7138ed5640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f711806aea0 0x7f711806d360 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f712800d850 tx=0x7f7128008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:39.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.206+0000 7f71317fa640 1 -- 172.21.15.81:0/452321526 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7120053d60 con 0x7f713415de50 2023-11-08T16:32:39.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.465+0000 7f713b139640 1 -- 172.21.15.81:0/452321526 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f71340c8980 con 0x7f713415de50 2023-11-08T16:32:39.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.465+0000 7f71317fa640 1 -- 172.21.15.81:0/452321526 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7120018070 con 0x7f713415de50 2023-11-08T16:32:39.468 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:39.469 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:32:39.469 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_dump": [ 2023-11-08T16:32:39.469 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.469 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.a", 2023-11-08T16:32:39.469 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlL0bmNxAAlrzRL4depZdd+lBB5HN09g==", 2023-11-08T16:32:39.470 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.470 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:32:39.470 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:32:39.470 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:32:39.470 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.471 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.471 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.471 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.471 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.b", 2023-11-08T16:32:39.471 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tl2cFWHhAAvwSSL9e5996l9MBwNQzs6A==", 2023-11-08T16:32:39.472 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.472 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:32:39.472 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:32:39.472 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:32:39.472 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.472 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.473 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.473 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.473 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.c", 2023-11-08T16:32:39.473 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tlwo23DBAAl1sBjOJMIQRb1fH3o9XoiQ==", 2023-11-08T16:32:39.473 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.474 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:32:39.474 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:32:39.474 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:32:39.474 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.474 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.475 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.475 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.475 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.d", 2023-11-08T16:32:39.475 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tl5DdQLxAA1aU7e9epGtKnZ6hkHZTnsQ==", 2023-11-08T16:32:39.475 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.476 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:32:39.476 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:32:39.476 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:32:39.476 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.476 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.477 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.477 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.477 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.0", 2023-11-08T16:32:39.477 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQALt0tl3kkwKRAAgDwD1eQYk7qjvV+/gyol/w==", 2023-11-08T16:32:39.477 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.477 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:32:39.478 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:32:39.478 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.478 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.478 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.478 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.479 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.1", 2023-11-08T16:32:39.479 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAPt0tlpIh+BRAAUukXDBbd9Vev1TzIrv9sRg==", 2023-11-08T16:32:39.479 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.479 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:32:39.479 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:32:39.480 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.480 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.480 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.480 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.480 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.2", 2023-11-08T16:32:39.480 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQASt0tlmzJfDhAASO7zeg/9ddmLsoOTe+MIug==", 2023-11-08T16:32:39.481 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.481 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:32:39.481 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:32:39.481 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.481 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.482 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.482 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.482 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.3", 2023-11-08T16:32:39.482 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAVt0tlGoQSGRAAQsrZhrx4kpYUm3KEb6ZUGQ==", 2023-11-08T16:32:39.482 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.483 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:32:39.483 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:32:39.483 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.483 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.483 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.484 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.484 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.4", 2023-11-08T16:32:39.484 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAct0tlOS/kIxAAxC2I47B2WeGTFZxvdnmkMg==", 2023-11-08T16:32:39.484 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.484 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:32:39.484 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:32:39.485 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.485 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.485 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.485 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.485 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.5", 2023-11-08T16:32:39.486 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAgt0tlnAgEBRAADIRSyu/bxiykCc2uiIt0kA==", 2023-11-08T16:32:39.486 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.486 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:32:39.486 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:32:39.486 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.487 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.487 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.487 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.487 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.6", 2023-11-08T16:32:39.487 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAjt0tlrdfZHBAAHGdvBBzVwXMmy8GmsZLNbg==", 2023-11-08T16:32:39.487 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.488 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:32:39.488 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:32:39.488 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.488 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.488 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.489 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.489 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.7", 2023-11-08T16:32:39.489 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAmt0tl4nN9NRAAzV9abuBfhybYCsSFnqHEYA==", 2023-11-08T16:32:39.489 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.489 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:32:39.490 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:32:39.490 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.490 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.490 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.490 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.490 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.0", 2023-11-08T16:32:39.491 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ==", 2023-11-08T16:32:39.491 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.491 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:32:39.491 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow r", 2023-11-08T16:32:39.491 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T16:32:39.492 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.492 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.492 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.492 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.1", 2023-11-08T16:32:39.492 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlgn/qGBAA+WHDN1CJ1v62EbX1GEFBWA==", 2023-11-08T16:32:39.493 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.493 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:32:39.493 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:32:39.493 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:32:39.493 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:32:39.494 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.494 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.494 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.494 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.2", 2023-11-08T16:32:39.494 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlbQmgDRAAL/b9NwCBVT+uGxuv3VAWRw==", 2023-11-08T16:32:39.495 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.495 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:32:39.495 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:32:39.495 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:32:39.495 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:32:39.496 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.496 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.496 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.496 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.3", 2023-11-08T16:32:39.496 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlah+ZIxAAg53qCmUjPtG+l++2Byn1NA==", 2023-11-08T16:32:39.497 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.497 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:32:39.497 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:32:39.497 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:32:39.497 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:32:39.498 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.498 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.498 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.498 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.admin", 2023-11-08T16:32:39.498 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAEt0tlNjJ+KRAATvWBm8Guj09sQQB2U1Afzw==", 2023-11-08T16:32:39.499 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.499 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:32:39.499 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:32:39.499 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:32:39.499 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.499 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.500 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.500 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.500 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.x", 2023-11-08T16:32:39.500 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlhZFjLRAAANBB8roHZH4z4pE4C2Aq/Q==", 2023-11-08T16:32:39.500 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.501 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:32:39.501 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2023-11-08T16:32:39.501 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.501 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.501 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:39.502 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:39.502 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.y", 2023-11-08T16:32:39.502 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlNpDaIBAAtgLFNQitjQnLny6SRY7hHw==", 2023-11-08T16:32:39.502 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:32:39.502 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:32:39.503 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2023-11-08T16:32:39.503 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:32:39.503 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.503 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:39.503 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2023-11-08T16:32:39.504 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:32:39.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 -- 172.21.15.81:0/452321526 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f711806aea0 msgr2=0x7f711806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f711806aea0 0x7f711806d360 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f712800d850 tx=0x7f7128008040 comp rx=0 tx=0).stop 2023-11-08T16:32:39.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 -- 172.21.15.81:0/452321526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f713415de50 msgr2=0x7f71341f6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f713415de50 0x7f71341f6130 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f712000c730 tx=0x7f712000cbf0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 -- 172.21.15.81:0/452321526 shutdown_connections 2023-11-08T16:32:39.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f711806aea0 0x7f711806d360 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71341f6670 0x7f71340cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f713415de50 0x7f71341f6130 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 --2- 172.21.15.81:0/452321526 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71240051d0 0x7f71340c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 -- 172.21.15.81:0/452321526 >> 172.21.15.81:0/452321526 conn(0x7f71340ceaa0 msgr2=0x7f713415a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:39.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 -- 172.21.15.81:0/452321526 shutdown_connections 2023-11-08T16:32:39.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.468+0000 7f713b139640 1 -- 172.21.15.81:0/452321526 wait complete. 2023-11-08T16:32:39.508 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:32:39.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.720+0000 7f539bc10640 1 Processor -- start 2023-11-08T16:32:39.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.721+0000 7f539bc10640 1 -- start start 2023-11-08T16:32:39.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.722+0000 7f539bc10640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f539415d1f0 0x7f53940cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.722+0000 7f539bc10640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f539415d5d0 0x7f53940ce490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.722+0000 7f539bc10640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53940cc350 con 0x7f53940ce9d0 2023-11-08T16:32:39.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.722+0000 7f539bc10640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53940cad70 con 0x7f539415d1f0 2023-11-08T16:32:39.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.722+0000 7f539bc10640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53940cb050 con 0x7f539415d5d0 2023-11-08T16:32:39.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.722+0000 7f53999ac640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f539415d1f0 0x7f53940cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:39.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.722+0000 7f53999ac640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f539415d1f0 0x7f53940cbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33804/0 (socket says 172.21.15.81:33804) 2023-11-08T16:32:39.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.722+0000 7f53999ac640 1 -- 172.21.15.81:0/809562886 learned_addr learned my addr 172.21.15.81:0/809562886 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:39.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.723+0000 7f53999ac640 1 -- 172.21.15.81:0/809562886 >> v2:172.21.15.93:3300/0 conn(0x7f539415d5d0 msgr2=0x7f53940ce490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:39.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.723+0000 7f53999ac640 1 --2- 172.21.15.81:0/809562886 >> v2:172.21.15.93:3300/0 conn(0x7f539415d5d0 0x7f53940ce490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.723+0000 7f53999ac640 1 -- 172.21.15.81:0/809562886 >> v1:172.21.15.81:6789/0 conn(0x7f53940ce9d0 legacy=0x7f53940709c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.723+0000 7f53999ac640 1 -- 172.21.15.81:0/809562886 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53940cb330 con 0x7f539415d1f0 2023-11-08T16:32:39.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.724+0000 7f53999ac640 1 --2- 172.21.15.81:0/809562886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f539415d1f0 0x7f53940cbe10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5394071d60 tx=0x7f538402e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a575e6ef2c6046 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:39.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.724+0000 7f53989aa640 1 -- 172.21.15.81:0/809562886 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f538402ec90 con 0x7f539415d1f0 2023-11-08T16:32:39.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.724+0000 7f53989aa640 1 -- 172.21.15.81:0/809562886 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538402edf0 con 0x7f539415d1f0 2023-11-08T16:32:39.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.724+0000 7f53989aa640 1 -- 172.21.15.81:0/809562886 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384036610 con 0x7f539415d1f0 2023-11-08T16:32:39.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.725+0000 7f539bc10640 1 -- 172.21.15.81:0/809562886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f539415d1f0 msgr2=0x7f53940cbe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.725+0000 7f539bc10640 1 --2- 172.21.15.81:0/809562886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f539415d1f0 0x7f53940cbe10 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5394071d60 tx=0x7f538402e0f0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.725+0000 7f539bc10640 1 -- 172.21.15.81:0/809562886 shutdown_connections 2023-11-08T16:32:39.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.725+0000 7f539bc10640 1 --2- 172.21.15.81:0/809562886 >> v2:172.21.15.93:3300/0 conn(0x7f539415d5d0 0x7f53940ce490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.725+0000 7f539bc10640 1 --2- 172.21.15.81:0/809562886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f539415d1f0 0x7f53940cbe10 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.725+0000 7f539bc10640 1 -- 172.21.15.81:0/809562886 >> 172.21.15.81:0/809562886 conn(0x7f53940b1600 msgr2=0x7f53940b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:39.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.725+0000 7f539bc10640 1 -- 172.21.15.81:0/809562886 shutdown_connections 2023-11-08T16:32:39.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.725+0000 7f539bc10640 1 -- 172.21.15.81:0/809562886 wait complete. 2023-11-08T16:32:39.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.727+0000 7f539bc10640 1 Processor -- start 2023-11-08T16:32:39.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.727+0000 7f539bc10640 1 -- start start 2023-11-08T16:32:39.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.728+0000 7f539bc10640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53940ce9d0 0x7f53940c3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.728+0000 7f539bc10640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f539415d1f0 0x7f5394158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.728+0000 7f539bc10640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f539415d5d0 0x7f53940afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.728+0000 7f539bc10640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53940ad5d0 con 0x7f539415d1f0 2023-11-08T16:32:39.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.728+0000 7f539bc10640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53940ac950 con 0x7f53940ce9d0 2023-11-08T16:32:39.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.728+0000 7f539bc10640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53940acc30 con 0x7f539415d5d0 2023-11-08T16:32:39.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.728+0000 7f53991ab640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f539415d1f0 0x7f5394158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:39.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.729+0000 7f53991ab640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f539415d1f0 0x7f5394158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33806/0 (socket says 172.21.15.81:33806) 2023-11-08T16:32:39.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.729+0000 7f53991ab640 1 -- 172.21.15.81:0/3821530958 learned_addr learned my addr 172.21.15.81:0/3821530958 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:39.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.729+0000 7f53991ab640 1 -- 172.21.15.81:0/3821530958 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f539415d5d0 msgr2=0x7f53940afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.729+0000 7f53999ac640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53940ce9d0 0x7f53940c3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:39.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.729+0000 7f53991ab640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f539415d5d0 0x7f53940afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.729+0000 7f53991ab640 1 -- 172.21.15.81:0/3821530958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53940ce9d0 msgr2=0x7f53940c3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.729+0000 7f53991ab640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53940ce9d0 0x7f53940c3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.729+0000 7f53991ab640 1 -- 172.21.15.81:0/3821530958 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5384009b30 con 0x7f539415d1f0 2023-11-08T16:32:39.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.729+0000 7f53991ab640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f539415d1f0 0x7f5394158010 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f538800b450 tx=0x7f538800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:39.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.729+0000 7f53927fc640 1 -- 172.21.15.81:0/3821530958 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388011890 con 0x7f539415d1f0 2023-11-08T16:32:39.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.730+0000 7f53927fc640 1 -- 172.21.15.81:0/3821530958 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53880119f0 con 0x7f539415d1f0 2023-11-08T16:32:39.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.730+0000 7f539bc10640 1 -- 172.21.15.81:0/3821530958 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53940acf10 con 0x7f539415d1f0 2023-11-08T16:32:39.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.730+0000 7f53927fc640 1 -- 172.21.15.81:0/3821530958 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388011da0 con 0x7f539415d1f0 2023-11-08T16:32:39.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.730+0000 7f539bc10640 1 -- 172.21.15.81:0/3821530958 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5394158550 con 0x7f539415d1f0 2023-11-08T16:32:39.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.731+0000 7f539bc10640 1 -- 172.21.15.81:0/3821530958 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53940ad710 con 0x7f539415d1f0 2023-11-08T16:32:39.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.732+0000 7f53927fc640 1 -- 172.21.15.81:0/3821530958 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f538801a400 con 0x7f539415d1f0 2023-11-08T16:32:39.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.733+0000 7f53927fc640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538006aea0 0x7f538006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:39.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.733+0000 7f53927fc640 1 -- 172.21.15.81:0/3821530958 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5388089660 con 0x7f539415d1f0 2023-11-08T16:32:39.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.733+0000 7f53999ac640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538006aea0 0x7f538006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:39.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.733+0000 7f53999ac640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538006aea0 0x7f538006d360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5384037000 tx=0x7f5384032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:39.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.737+0000 7f53927fc640 1 -- 172.21.15.81:0/3821530958 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f53940ad710 con 0x7f539415d1f0 2023-11-08T16:32:39.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.916+0000 7f539bc10640 1 -- 172.21.15.81:0/3821530958 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53940ad710 con 0x7f539415d1f0 2023-11-08T16:32:39.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.917+0000 7f53927fc640 1 -- 172.21.15.81:0/3821530958 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+12797 (secure 0 0 0) 0x7f53940ad710 con 0x7f539415d1f0 2023-11-08T16:32:39.920 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:39.920 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":53,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:32:38.938611+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:32:39.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.919+0000 7f539bc10640 1 -- 172.21.15.81:0/3821530958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538006aea0 msgr2=0x7f538006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.919+0000 7f539bc10640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538006aea0 0x7f538006d360 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5384037000 tx=0x7f5384032000 comp rx=0 tx=0).stop 2023-11-08T16:32:39.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.919+0000 7f539bc10640 1 -- 172.21.15.81:0/3821530958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f539415d1f0 msgr2=0x7f5394158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:39.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.919+0000 7f539bc10640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f539415d1f0 0x7f5394158010 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f538800b450 tx=0x7f538800b910 comp rx=0 tx=0).stop 2023-11-08T16:32:39.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.920+0000 7f539bc10640 1 -- 172.21.15.81:0/3821530958 shutdown_connections 2023-11-08T16:32:39.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.920+0000 7f539bc10640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f538006aea0 0x7f538006d360 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.920+0000 7f539bc10640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f539415d5d0 0x7f53940afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.920+0000 7f539bc10640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f539415d1f0 0x7f5394158010 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.920+0000 7f539bc10640 1 --2- 172.21.15.81:0/3821530958 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f53940ce9d0 0x7f53940c3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:39.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.920+0000 7f539bc10640 1 -- 172.21.15.81:0/3821530958 >> 172.21.15.81:0/3821530958 conn(0x7f53940b1600 msgr2=0x7f539415c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:39.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.920+0000 7f539bc10640 1 -- 172.21.15.81:0/3821530958 shutdown_connections 2023-11-08T16:32:39.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:39.920+0000 7f539bc10640 1 -- 172.21.15.81:0/3821530958 wait complete. 2023-11-08T16:32:39.940 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:32:39.941 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:32:40.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.146+0000 7fabd6e00640 1 Processor -- start 2023-11-08T16:32:40.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.147+0000 7fabd6e00640 1 -- start start 2023-11-08T16:32:40.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.148+0000 7fabd6e00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fabd00cfcc0 0x7fabd00cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:40.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.148+0000 7fabd6e00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd00cef70 con 0x7fabd015d4c0 2023-11-08T16:32:40.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.148+0000 7fabd6e00640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd00cec90 con 0x7fabd00d39a0 2023-11-08T16:32:40.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.148+0000 7fabd6e00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd00b5e70 con 0x7fabd00cfcc0 2023-11-08T16:32:40.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.148+0000 7fabd4b9c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fabd015d4c0 0x7fabd00cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:41050/0 (socket says 172.21.15.81:41050) 2023-11-08T16:32:40.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.148+0000 7fabd4b9c640 1 -- 172.21.15.81:0/2140743172 learned_addr learned my addr 172.21.15.81:0/2140743172 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:40.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.148+0000 7fabcffff640 1 --2- 172.21.15.81:0/2140743172 >> v2:172.21.15.81:3300/0 conn(0x7fabd00cfcc0 0x7fabd00cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:40.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.149+0000 7fabcf7fe640 1 -- 172.21.15.81:0/2140743172 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2884502921 0 0) 0x7fabc4001110 con 0x7fabd015d4c0 2023-11-08T16:32:40.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.149+0000 7fabcf7fe640 1 -- 172.21.15.81:0/2140743172 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabc0003520 con 0x7fabd015d4c0 2023-11-08T16:32:40.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.149+0000 7fabcffff640 1 -- 172.21.15.81:0/2140743172 >> v1:172.21.15.93:6789/0 conn(0x7fabd00d39a0 legacy=0x7fabd00b6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:40.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.149+0000 7fabcffff640 1 -- 172.21.15.81:0/2140743172 >> v1:172.21.15.81:6789/0 conn(0x7fabd015d4c0 legacy=0x7fabd00cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.149+0000 7fabcffff640 1 -- 172.21.15.81:0/2140743172 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd00b6150 con 0x7fabd00cfcc0 2023-11-08T16:32:40.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.150+0000 7fabcffff640 1 --2- 172.21.15.81:0/2140743172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd00cfcc0 0x7fabd00cdb40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0097a0 tx=0x7fabbc02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95aba66419c5a9f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:40.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.151+0000 7fabcf7fe640 1 -- 172.21.15.81:0/2140743172 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc030610 con 0x7fabd00cfcc0 2023-11-08T16:32:40.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.151+0000 7fabcf7fe640 1 -- 172.21.15.81:0/2140743172 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc030770 con 0x7fabd00cfcc0 2023-11-08T16:32:40.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.151+0000 7fabcf7fe640 1 -- 172.21.15.81:0/2140743172 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc030b00 con 0x7fabd00cfcc0 2023-11-08T16:32:40.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.151+0000 7fabd6e00640 1 -- 172.21.15.81:0/2140743172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd00cfcc0 msgr2=0x7fabd00cdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.151+0000 7fabd6e00640 1 --2- 172.21.15.81:0/2140743172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd00cfcc0 0x7fabd00cdb40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fabbc0097a0 tx=0x7fabbc02b960 comp rx=0 tx=0).stop 2023-11-08T16:32:40.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.151+0000 7fabd6e00640 1 -- 172.21.15.81:0/2140743172 shutdown_connections 2023-11-08T16:32:40.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.151+0000 7fabd6e00640 1 --2- 172.21.15.81:0/2140743172 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd00cfcc0 0x7fabd00cdb40 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.151+0000 7fabd6e00640 1 -- 172.21.15.81:0/2140743172 >> 172.21.15.81:0/2140743172 conn(0x7fabd00d2e80 msgr2=0x7fabd00d3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:40.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.151+0000 7fabd6e00640 1 -- 172.21.15.81:0/2140743172 shutdown_connections 2023-11-08T16:32:40.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.151+0000 7fabd6e00640 1 -- 172.21.15.81:0/2140743172 wait complete. 2023-11-08T16:32:40.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.153+0000 7fabd6e00640 1 Processor -- start 2023-11-08T16:32:40.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.153+0000 7fabd6e00640 1 -- start start 2023-11-08T16:32:40.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.154+0000 7fabd6e00640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabd00cfcc0 0x7fabd00c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:40.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.154+0000 7fabd6e00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd00d39a0 0x7fabd0157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:40.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.154+0000 7fabd6e00640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd015d4c0 0x7fabd015abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:40.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.154+0000 7fabd6e00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd015f510 con 0x7fabd00d39a0 2023-11-08T16:32:40.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.154+0000 7fabd6e00640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd0151900 con 0x7fabd015d4c0 2023-11-08T16:32:40.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.154+0000 7fabd6e00640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd0156900 con 0x7fabd00cfcc0 2023-11-08T16:32:40.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.154+0000 7fabcffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd00d39a0 0x7fabd0157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:40.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabcffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd00d39a0 0x7fabd0157ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33836/0 (socket says 172.21.15.81:33836) 2023-11-08T16:32:40.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabcffff640 1 -- 172.21.15.81:0/4041129581 learned_addr learned my addr 172.21.15.81:0/4041129581 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:40.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabd4b9c640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabd00cfcc0 0x7fabd00c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:40.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabcffff640 1 -- 172.21.15.81:0/4041129581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabd00cfcc0 msgr2=0x7fabd00c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabcffff640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabd00cfcc0 0x7fabd00c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabcffff640 1 -- 172.21.15.81:0/4041129581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd015d4c0 msgr2=0x7fabd015abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabcffff640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd015d4c0 0x7fabd015abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabcffff640 1 -- 172.21.15.81:0/4041129581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc009400 con 0x7fabd00d39a0 2023-11-08T16:32:40.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabd4b9c640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabd00cfcc0 0x7fabd00c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:40.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabcffff640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd00d39a0 0x7fabd0157ed0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fabbc02bf70 tx=0x7fabbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:40.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabcd7fa640 1 -- 172.21.15.81:0/4041129581 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc03e040 con 0x7fabd00d39a0 2023-11-08T16:32:40.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabcd7fa640 1 -- 172.21.15.81:0/4041129581 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabbc039c90 con 0x7fabd00d39a0 2023-11-08T16:32:40.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabcd7fa640 1 -- 172.21.15.81:0/4041129581 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabbc030e10 con 0x7fabd00d39a0 2023-11-08T16:32:40.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.155+0000 7fabd6e00640 1 -- 172.21.15.81:0/4041129581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd0156be0 con 0x7fabd00d39a0 2023-11-08T16:32:40.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.156+0000 7fabd6e00640 1 -- 172.21.15.81:0/4041129581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd01571a0 con 0x7fabd00d39a0 2023-11-08T16:32:40.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.157+0000 7fabd6e00640 1 -- 172.21.15.81:0/4041129581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd00d2860 con 0x7fabd00d39a0 2023-11-08T16:32:40.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.158+0000 7fabcd7fa640 1 -- 172.21.15.81:0/4041129581 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fabbc0423e0 con 0x7fabd00d39a0 2023-11-08T16:32:40.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.159+0000 7fabcd7fa640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabb406aea0 0x7fabb406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:40.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.159+0000 7fabcd7fa640 1 -- 172.21.15.81:0/4041129581 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fabbc0ab380 con 0x7fabd00d39a0 2023-11-08T16:32:40.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.159+0000 7fabd4b9c640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabb406aea0 0x7fabb406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:40.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.160+0000 7fabd4b9c640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabb406aea0 0x7fabb406d360 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fabc4001370 tx=0x7fabc4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:40.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.163+0000 7fabcd7fa640 1 -- 172.21.15.81:0/4041129581 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabbc076650 con 0x7fabd00d39a0 2023-11-08T16:32:40.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.339+0000 7fabd6e00640 1 -- 172.21.15.81:0/4041129581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fabd004ece0 con 0x7fabd00d39a0 2023-11-08T16:32:40.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.340+0000 7fabcd7fa640 1 -- 172.21.15.81:0/4041129581 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7fabbc079c70 con 0x7fabd00d39a0 2023-11-08T16:32:40.343 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:32:40.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 -- 172.21.15.81:0/4041129581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabb406aea0 msgr2=0x7fabb406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabb406aea0 0x7fabb406d360 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fabc4001370 tx=0x7fabc4009040 comp rx=0 tx=0).stop 2023-11-08T16:32:40.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 -- 172.21.15.81:0/4041129581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd00d39a0 msgr2=0x7fabd0157ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd00d39a0 0x7fabd0157ed0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fabbc02bf70 tx=0x7fabbc004730 comp rx=0 tx=0).stop 2023-11-08T16:32:40.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 -- 172.21.15.81:0/4041129581 shutdown_connections 2023-11-08T16:32:40.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabb406aea0 0x7fabb406d360 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd015d4c0 0x7fabd015abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd00d39a0 0x7fabd0157ed0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 --2- 172.21.15.81:0/4041129581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabd00cfcc0 0x7fabd00c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 -- 172.21.15.81:0/4041129581 >> 172.21.15.81:0/4041129581 conn(0x7fabd00d2e80 msgr2=0x7fabd00b7dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:40.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 -- 172.21.15.81:0/4041129581 shutdown_connections 2023-11-08T16:32:40.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.342+0000 7fabd6e00640 1 -- 172.21.15.81:0/4041129581 wait complete. 2023-11-08T16:32:40.362 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T16:32:40.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.570+0000 7f15d2a39640 1 Processor -- start 2023-11-08T16:32:40.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.570+0000 7f15d2a39640 1 -- start start 2023-11-08T16:32:40.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.572+0000 7f15d2a39640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f15cc15cf70 0x7f15cc0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:40.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.572+0000 7f15d2a39640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f15cc0b4e80 0x7f15cc161470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:40.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.572+0000 7f15d2a39640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc1619b0 con 0x7f15cc0b8610 2023-11-08T16:32:40.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.572+0000 7f15d2a39640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15cc15d350 con 0x7f15cc15cf70 2023-11-08T16:32:40.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.572+0000 7f15d2a39640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15cc0b8aa0 con 0x7f15cc0b4e80 2023-11-08T16:32:40.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.572+0000 7f15d1a37640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f15cc15cf70 0x7f15cc0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:40.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.572+0000 7f15d1a37640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f15cc15cf70 0x7f15cc0cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33850/0 (socket says 172.21.15.81:33850) 2023-11-08T16:32:40.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.572+0000 7f15d1a37640 1 -- 172.21.15.81:0/3440345751 learned_addr learned my addr 172.21.15.81:0/3440345751 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:40.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.572+0000 7f15d1236640 1 --1- 172.21.15.81:0/3440345751 >> v1:172.21.15.81:6789/0 conn(0x7f15cc0b8610 0x7f15cc0b4770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:41052/0 (socket says 172.21.15.81:41052) 2023-11-08T16:32:40.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.573+0000 7f15d1a37640 1 -- 172.21.15.81:0/3440345751 >> v2:172.21.15.93:3300/0 conn(0x7f15cc0b4e80 msgr2=0x7f15cc161470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:40.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.573+0000 7f15d2238640 1 --2- 172.21.15.81:0/3440345751 >> v2:172.21.15.93:3300/0 conn(0x7f15cc0b4e80 0x7f15cc161470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:40.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.573+0000 7f15d1a37640 1 --2- 172.21.15.81:0/3440345751 >> v2:172.21.15.93:3300/0 conn(0x7f15cc0b4e80 0x7f15cc161470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.573+0000 7f15d1a37640 1 -- 172.21.15.81:0/3440345751 >> v1:172.21.15.81:6789/0 conn(0x7f15cc0b8610 legacy=0x7f15cc0b4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.573+0000 7f15d1a37640 1 -- 172.21.15.81:0/3440345751 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc0cd4e0 con 0x7f15cc15cf70 2023-11-08T16:32:40.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.573+0000 7f15d2238640 1 --2- 172.21.15.81:0/3440345751 >> v2:172.21.15.93:3300/0 conn(0x7f15cc0b4e80 0x7f15cc161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:40.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.573+0000 7f15d0a35640 1 -- 172.21.15.81:0/3440345751 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853563863 0 0) 0x7f15bc001110 con 0x7f15cc0b8610 2023-11-08T16:32:40.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.574+0000 7f15d1a37640 1 --2- 172.21.15.81:0/3440345751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15cc15cf70 0x7f15cc0cdfc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f15cc06d3a0 tx=0x7f15c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544b2febd742fb48 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:40.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.574+0000 7f15d0a35640 1 -- 172.21.15.81:0/3440345751 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8031870 con 0x7f15cc15cf70 2023-11-08T16:32:40.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.574+0000 7f15d0a35640 1 -- 172.21.15.81:0/3440345751 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c80319d0 con 0x7f15cc15cf70 2023-11-08T16:32:40.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.574+0000 7f15d0a35640 1 -- 172.21.15.81:0/3440345751 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8031d60 con 0x7f15cc15cf70 2023-11-08T16:32:40.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.575+0000 7f15d2a39640 1 -- 172.21.15.81:0/3440345751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15cc15cf70 msgr2=0x7f15cc0cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.575+0000 7f15d2a39640 1 --2- 172.21.15.81:0/3440345751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15cc15cf70 0x7f15cc0cdfc0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f15cc06d3a0 tx=0x7f15c802b910 comp rx=0 tx=0).stop 2023-11-08T16:32:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.575+0000 7f15d2a39640 1 -- 172.21.15.81:0/3440345751 shutdown_connections 2023-11-08T16:32:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.575+0000 7f15d2a39640 1 --2- 172.21.15.81:0/3440345751 >> v2:172.21.15.93:3300/0 conn(0x7f15cc0b4e80 0x7f15cc161470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.575+0000 7f15d2a39640 1 --2- 172.21.15.81:0/3440345751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15cc15cf70 0x7f15cc0cdfc0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.575+0000 7f15d2a39640 1 -- 172.21.15.81:0/3440345751 >> 172.21.15.81:0/3440345751 conn(0x7f15cc0cf770 msgr2=0x7f15cc0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.575+0000 7f15d2a39640 1 -- 172.21.15.81:0/3440345751 shutdown_connections 2023-11-08T16:32:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.575+0000 7f15d2a39640 1 -- 172.21.15.81:0/3440345751 wait complete. 2023-11-08T16:32:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.576+0000 7f15d2a39640 1 Processor -- start 2023-11-08T16:32:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.577+0000 7f15d2a39640 1 -- start start 2023-11-08T16:32:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2a39640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15cc0b4e80 0x7f15cc146730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:40.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2a39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15cc15cf70 0x7f15cc1f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:40.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2a39640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15cc15bc30 0x7f15cc0cc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:40.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2a39640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc146c70 con 0x7f15cc15bc30 2023-11-08T16:32:40.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2a39640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc15ab90 con 0x7f15cc0b4e80 2023-11-08T16:32:40.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2a39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc15ae70 con 0x7f15cc15cf70 2023-11-08T16:32:40.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2238640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15cc15bc30 0x7f15cc0cc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:40.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2238640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15cc15bc30 0x7f15cc0cc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:33860/0 (socket says 172.21.15.81:33860) 2023-11-08T16:32:40.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2238640 1 -- 172.21.15.81:0/671279174 learned_addr learned my addr 172.21.15.81:0/671279174 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:40.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d1236640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15cc15cf70 0x7f15cc1f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:40.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d1a37640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15cc0b4e80 0x7f15cc146730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:40.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2238640 1 -- 172.21.15.81:0/671279174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15cc15cf70 msgr2=0x7f15cc1f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2238640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15cc15cf70 0x7f15cc1f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2238640 1 -- 172.21.15.81:0/671279174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15cc0b4e80 msgr2=0x7f15cc146730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2238640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15cc0b4e80 0x7f15cc146730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.578+0000 7f15d2238640 1 -- 172.21.15.81:0/671279174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c80094d0 con 0x7f15cc15bc30 2023-11-08T16:32:40.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.579+0000 7f15d2238640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15cc15bc30 0x7f15cc0cc110 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f15c400cef0 tx=0x7f15c400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:40.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.579+0000 7f15c27fc640 1 -- 172.21.15.81:0/671279174 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c400d840 con 0x7f15cc15bc30 2023-11-08T16:32:40.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.579+0000 7f15c27fc640 1 -- 172.21.15.81:0/671279174 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c400d9c0 con 0x7f15cc15bc30 2023-11-08T16:32:40.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.579+0000 7f15d2a39640 1 -- 172.21.15.81:0/671279174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc15b150 con 0x7f15cc15bc30 2023-11-08T16:32:40.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.579+0000 7f15c27fc640 1 -- 172.21.15.81:0/671279174 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c400dd70 con 0x7f15cc15bc30 2023-11-08T16:32:40.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.579+0000 7f15d2a39640 1 -- 172.21.15.81:0/671279174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc15b710 con 0x7f15cc15bc30 2023-11-08T16:32:40.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.580+0000 7f15d2a39640 1 -- 172.21.15.81:0/671279174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc04ece0 con 0x7f15cc15bc30 2023-11-08T16:32:40.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.581+0000 7f15c27fc640 1 -- 172.21.15.81:0/671279174 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15c401e3b0 con 0x7f15cc15bc30 2023-11-08T16:32:40.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.582+0000 7f15c27fc640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15b006ae70 0x7f15b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:40.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.582+0000 7f15c27fc640 1 -- 172.21.15.81:0/671279174 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f15c4089d10 con 0x7f15cc15bc30 2023-11-08T16:32:40.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.582+0000 7f15d1a37640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15b006ae70 0x7f15b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:40.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.583+0000 7f15d1a37640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15b006ae70 0x7f15b006d330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f15c8036000 tx=0x7f15c803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:40.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.586+0000 7f15c27fc640 1 -- 172.21.15.81:0/671279174 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15c4055260 con 0x7f15cc15bc30 2023-11-08T16:32:40.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.766+0000 7f15d2a39640 1 -- 172.21.15.81:0/671279174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f15cc153700 con 0x7f15cc15bc30 2023-11-08T16:32:40.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.767+0000 7f15c27fc640 1 -- 172.21.15.81:0/671279174 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v53) v1 ==== 84+0+65 (secure 0 0 0) 0x7f15c401a090 con 0x7f15cc15bc30 2023-11-08T16:32:40.770 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:40.770 INFO:teuthology.orchestra.run.smithi081.stdout:[ 2023-11-08T16:32:40.770 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:32:40.770 INFO:teuthology.orchestra.run.smithi081.stdout: "poolnum": 1, 2023-11-08T16:32:40.771 INFO:teuthology.orchestra.run.smithi081.stdout: "poolname": ".mgr" 2023-11-08T16:32:40.771 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:40.771 INFO:teuthology.orchestra.run.smithi081.stdout:] 2023-11-08T16:32:40.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.769+0000 7f15d2a39640 1 -- 172.21.15.81:0/671279174 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15b006ae70 msgr2=0x7f15b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.769+0000 7f15d2a39640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15b006ae70 0x7f15b006d330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f15c8036000 tx=0x7f15c803c040 comp rx=0 tx=0).stop 2023-11-08T16:32:40.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.769+0000 7f15d2a39640 1 -- 172.21.15.81:0/671279174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15cc15bc30 msgr2=0x7f15cc0cc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:40.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.769+0000 7f15d2a39640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15cc15bc30 0x7f15cc0cc110 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f15c400cef0 tx=0x7f15c400a710 comp rx=0 tx=0).stop 2023-11-08T16:32:40.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.770+0000 7f15d2a39640 1 -- 172.21.15.81:0/671279174 shutdown_connections 2023-11-08T16:32:40.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.770+0000 7f15d2a39640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15b006ae70 0x7f15b006d330 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.770+0000 7f15d2a39640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f15cc15bc30 0x7f15cc0cc110 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.770+0000 7f15d2a39640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15cc15cf70 0x7f15cc1f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.770+0000 7f15d2a39640 1 --2- 172.21.15.81:0/671279174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15cc0b4e80 0x7f15cc146730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:40.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.770+0000 7f15d2a39640 1 -- 172.21.15.81:0/671279174 >> 172.21.15.81:0/671279174 conn(0x7f15cc0cf770 msgr2=0x7f15cc0b6900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:40.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.770+0000 7f15d2a39640 1 -- 172.21.15.81:0/671279174 shutdown_connections 2023-11-08T16:32:40.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:40.770+0000 7f15d2a39640 1 -- 172.21.15.81:0/671279174 wait complete. 2023-11-08T16:32:40.791 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T16:32:40.791 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T16:32:41.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.003+0000 7fa9039b6640 1 Processor -- start 2023-11-08T16:32:41.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.003+0000 7fa9039b6640 1 -- start start 2023-11-08T16:32:41.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.004+0000 7fa9039b6640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa8fc071890 0x7fa8fc15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:41.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.005+0000 7fa9039b6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa8fc15afd0 0x7fa8fc15ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:41.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.005+0000 7fa9039b6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8fc0b09f0 con 0x7fa8fc0714b0 2023-11-08T16:32:41.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.005+0000 7fa9039b6640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8fc06b250 con 0x7fa8fc15afd0 2023-11-08T16:32:41.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.005+0000 7fa9039b6640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8fc0ca360 con 0x7fa8fc071890 2023-11-08T16:32:41.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.005+0000 7fa901f53640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa8fc15afd0 0x7fa8fc15ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:41.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.005+0000 7fa901f53640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa8fc15afd0 0x7fa8fc15ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43632/0 (socket says 172.21.15.81:43632) 2023-11-08T16:32:41.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.005+0000 7fa901f53640 1 -- 172.21.15.81:0/2526769020 learned_addr learned my addr 172.21.15.81:0/2526769020 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:41.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.005+0000 7fa900f51640 1 --2- 172.21.15.81:0/2526769020 >> v2:172.21.15.93:3300/0 conn(0x7fa8fc071890 0x7fa8fc15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:41.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.006+0000 7fa901f53640 1 -- 172.21.15.81:0/2526769020 >> v2:172.21.15.93:3300/0 conn(0x7fa8fc071890 msgr2=0x7fa8fc15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:41.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.006+0000 7fa901f53640 1 --2- 172.21.15.81:0/2526769020 >> v2:172.21.15.93:3300/0 conn(0x7fa8fc071890 0x7fa8fc15e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:41.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.006+0000 7fa901f53640 1 -- 172.21.15.81:0/2526769020 >> v1:172.21.15.93:6789/0 conn(0x7fa8fc0714b0 legacy=0x7fa8fc15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:41.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.006+0000 7fa901f53640 1 -- 172.21.15.81:0/2526769020 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8fc0ca640 con 0x7fa8fc15afd0 2023-11-08T16:32:41.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.007+0000 7fa901f53640 1 --2- 172.21.15.81:0/2526769020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8fc15afd0 0x7fa8fc15ef10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc15e930 tx=0x7fa8f402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1af5d4c7972d73f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:41.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.007+0000 7fa8f3fff640 1 -- 172.21.15.81:0/2526769020 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f402ca80 con 0x7fa8fc15afd0 2023-11-08T16:32:41.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.007+0000 7fa8f3fff640 1 -- 172.21.15.81:0/2526769020 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f402cbe0 con 0x7fa8fc15afd0 2023-11-08T16:32:41.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.007+0000 7fa8f3fff640 1 -- 172.21.15.81:0/2526769020 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f40343f0 con 0x7fa8fc15afd0 2023-11-08T16:32:41.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.008+0000 7fa9039b6640 1 -- 172.21.15.81:0/2526769020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8fc15afd0 msgr2=0x7fa8fc15ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:41.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.008+0000 7fa9039b6640 1 --2- 172.21.15.81:0/2526769020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8fc15afd0 0x7fa8fc15ef10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc15e930 tx=0x7fa8f402bee0 comp rx=0 tx=0).stop 2023-11-08T16:32:41.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.008+0000 7fa9039b6640 1 -- 172.21.15.81:0/2526769020 shutdown_connections 2023-11-08T16:32:41.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.008+0000 7fa9039b6640 1 --2- 172.21.15.81:0/2526769020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8fc15afd0 0x7fa8fc15ef10 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:41.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.008+0000 7fa9039b6640 1 --2- 172.21.15.81:0/2526769020 >> v2:172.21.15.93:3300/0 conn(0x7fa8fc071890 0x7fa8fc15e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:41.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.008+0000 7fa9039b6640 1 -- 172.21.15.81:0/2526769020 >> 172.21.15.81:0/2526769020 conn(0x7fa8fc0cb4e0 msgr2=0x7fa8fc0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:41.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.008+0000 7fa9039b6640 1 -- 172.21.15.81:0/2526769020 shutdown_connections 2023-11-08T16:32:41.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.008+0000 7fa9039b6640 1 -- 172.21.15.81:0/2526769020 wait complete. 2023-11-08T16:32:41.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.010+0000 7fa9039b6640 1 Processor -- start 2023-11-08T16:32:41.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.010+0000 7fa9039b6640 1 -- start start 2023-11-08T16:32:41.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.011+0000 7fa9039b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8fc0714b0 0x7fa8fc1500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:41.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.011+0000 7fa9039b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8fc071890 0x7fa8fc0c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:41.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.011+0000 7fa9039b6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8fc15afd0 0x7fa8fc1f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:41.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.011+0000 7fa9039b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc1505e0 con 0x7fa8fc0714b0 2023-11-08T16:32:41.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.011+0000 7fa9039b6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc0c6380 con 0x7fa8fc15afd0 2023-11-08T16:32:41.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.011+0000 7fa9039b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8fc0c6660 con 0x7fa8fc071890 2023-11-08T16:32:41.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.011+0000 7fa901752640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8fc0714b0 0x7fa8fc1500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:41.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.011+0000 7fa901752640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8fc0714b0 0x7fa8fc1500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43640/0 (socket says 172.21.15.81:43640) 2023-11-08T16:32:41.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.011+0000 7fa901752640 1 -- 172.21.15.81:0/387172193 learned_addr learned my addr 172.21.15.81:0/387172193 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:41.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.012+0000 7fa901752640 1 -- 172.21.15.81:0/387172193 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8fc071890 msgr2=0x7fa8fc0c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:41.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.012+0000 7fa901752640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8fc071890 0x7fa8fc0c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:41.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.012+0000 7fa901752640 1 -- 172.21.15.81:0/387172193 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8fc15afd0 msgr2=0x7fa8fc1f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:41.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.012+0000 7fa901752640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8fc15afd0 0x7fa8fc1f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:41.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.012+0000 7fa901752640 1 -- 172.21.15.81:0/387172193 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f4009b30 con 0x7fa8fc0714b0 2023-11-08T16:32:41.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.012+0000 7fa901752640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8fc0714b0 0x7fa8fc1500a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c430 tx=0x7fa8f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:41.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.012+0000 7fa8f1ffb640 1 -- 172.21.15.81:0/387172193 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8011790 con 0x7fa8fc0714b0 2023-11-08T16:32:41.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.012+0000 7fa8f1ffb640 1 -- 172.21.15.81:0/387172193 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f80118f0 con 0x7fa8fc0714b0 2023-11-08T16:32:41.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.012+0000 7fa8f1ffb640 1 -- 172.21.15.81:0/387172193 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8011ce0 con 0x7fa8fc0714b0 2023-11-08T16:32:41.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.012+0000 7fa9039b6640 1 -- 172.21.15.81:0/387172193 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8fc0c6940 con 0x7fa8fc0714b0 2023-11-08T16:32:41.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.012+0000 7fa9039b6640 1 -- 172.21.15.81:0/387172193 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8fc0c6f00 con 0x7fa8fc0714b0 2023-11-08T16:32:41.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.013+0000 7fa9039b6640 1 -- 172.21.15.81:0/387172193 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8fc0b0e60 con 0x7fa8fc0714b0 2023-11-08T16:32:41.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.014+0000 7fa8f1ffb640 1 -- 172.21.15.81:0/387172193 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa8f801c460 con 0x7fa8fc0714b0 2023-11-08T16:32:41.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.015+0000 7fa8f1ffb640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa8e006ae70 0x7fa8e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:41.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.015+0000 7fa8f1ffb640 1 -- 172.21.15.81:0/387172193 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa8f8089320 con 0x7fa8fc0714b0 2023-11-08T16:32:41.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.016+0000 7fa900f51640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa8e006ae70 0x7fa8e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:41.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.016+0000 7fa900f51640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa8e006ae70 0x7fa8e006d330 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0b4840 tx=0x7fa8ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:41.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.020+0000 7fa8f1ffb640 1 -- 172.21.15.81:0/387172193 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa8fc0b0e60 con 0x7fa8fc0714b0 2023-11-08T16:32:41.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.212+0000 7fa9039b6640 1 -- 172.21.15.81:0/387172193 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa8fc0b0e60 con 0x7fa8fc0714b0 2023-11-08T16:32:41.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.965+0000 7fa8f1ffb640 1 -- 172.21.15.81:0/387172193 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v54) v1 ==== 138+0+0 (secure 0 0 0) 0x7fa8fc0b0e60 con 0x7fa8fc0714b0 2023-11-08T16:32:41.968 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' created 2023-11-08T16:32:41.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.968+0000 7fa9039b6640 1 -- 172.21.15.81:0/387172193 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa8e006ae70 msgr2=0x7fa8e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:41.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.968+0000 7fa9039b6640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa8e006ae70 0x7fa8e006d330 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc0b4840 tx=0x7fa8ec008040 comp rx=0 tx=0).stop 2023-11-08T16:32:41.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.968+0000 7fa9039b6640 1 -- 172.21.15.81:0/387172193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8fc0714b0 msgr2=0x7fa8fc1500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:41.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.968+0000 7fa9039b6640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8fc0714b0 0x7fa8fc1500a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c430 tx=0x7fa8f800c8f0 comp rx=0 tx=0).stop 2023-11-08T16:32:41.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.968+0000 7fa9039b6640 1 -- 172.21.15.81:0/387172193 shutdown_connections 2023-11-08T16:32:41.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.968+0000 7fa9039b6640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa8e006ae70 0x7fa8e006d330 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:41.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.968+0000 7fa9039b6640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa8fc15afd0 0x7fa8fc1f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:41.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.968+0000 7fa9039b6640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa8fc071890 0x7fa8fc0c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.968+0000 7fa9039b6640 1 --2- 172.21.15.81:0/387172193 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa8fc0714b0 0x7fa8fc1500a0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.968+0000 7fa9039b6640 1 -- 172.21.15.81:0/387172193 >> 172.21.15.81:0/387172193 conn(0x7fa8fc0cb4e0 msgr2=0x7fa8fc158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.968+0000 7fa9039b6640 1 -- 172.21.15.81:0/387172193 shutdown_connections 2023-11-08T16:32:41.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:41.969+0000 7fa9039b6640 1 -- 172.21.15.81:0/387172193 wait complete. 2023-11-08T16:32:41.996 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T16:32:42.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.191+0000 7f4e35610640 1 Processor -- start 2023-11-08T16:32:42.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.192+0000 7f4e35610640 1 -- start start 2023-11-08T16:32:42.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.193+0000 7f4e35610640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e30159f40 0x7f4e3006b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:42.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.193+0000 7f4e35610640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4e300ca880 0x7f4e300cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:42.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.193+0000 7f4e35610640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e300cad00 con 0x7f4e300c9cb0 2023-11-08T16:32:42.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.193+0000 7f4e35610640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e300ccf80 con 0x7f4e300ca880 2023-11-08T16:32:42.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.193+0000 7f4e35610640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300caf60 con 0x7f4e30159f40 2023-11-08T16:32:42.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.194+0000 7f4e2f7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4e300ca880 0x7f4e300cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:42.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.194+0000 7f4e2f7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f4e300ca880 0x7f4e300cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43642/0 (socket says 172.21.15.81:43642) 2023-11-08T16:32:42.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.194+0000 7f4e2f7fe640 1 -- 172.21.15.81:0/4084580648 learned_addr learned my addr 172.21.15.81:0/4084580648 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:42.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.194+0000 7f4e2effd640 1 --2- 172.21.15.81:0/4084580648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e30159f40 0x7f4e3006b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:42.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.194+0000 7f4e2f7fe640 1 -- 172.21.15.81:0/4084580648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e30159f40 msgr2=0x7f4e3006b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:42.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.194+0000 7f4e2f7fe640 1 --2- 172.21.15.81:0/4084580648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e30159f40 0x7f4e3006b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:42.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.194+0000 7f4e2f7fe640 1 -- 172.21.15.81:0/4084580648 >> v1:172.21.15.93:6789/0 conn(0x7f4e300c9cb0 legacy=0x7f4e300ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:42.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.194+0000 7f4e2f7fe640 1 -- 172.21.15.81:0/4084580648 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e30065f90 con 0x7f4e300ca880 2023-11-08T16:32:42.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.195+0000 7f4e2f7fe640 1 --2- 172.21.15.81:0/4084580648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e300ca880 0x7f4e300cd460 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009870 tx=0x7f4e2802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695a51712bca11f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:42.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.196+0000 7f4e2dffb640 1 -- 172.21.15.81:0/4084580648 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28030840 con 0x7f4e300ca880 2023-11-08T16:32:42.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.196+0000 7f4e2dffb640 1 -- 172.21.15.81:0/4084580648 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e280309a0 con 0x7f4e300ca880 2023-11-08T16:32:42.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.196+0000 7f4e2dffb640 1 -- 172.21.15.81:0/4084580648 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e28030d30 con 0x7f4e300ca880 2023-11-08T16:32:42.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.196+0000 7f4e35610640 1 -- 172.21.15.81:0/4084580648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e300ca880 msgr2=0x7f4e300cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:42.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.196+0000 7f4e35610640 1 --2- 172.21.15.81:0/4084580648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e300ca880 0x7f4e300cd460 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009870 tx=0x7f4e2802b980 comp rx=0 tx=0).stop 2023-11-08T16:32:42.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.197+0000 7f4e35610640 1 -- 172.21.15.81:0/4084580648 shutdown_connections 2023-11-08T16:32:42.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.197+0000 7f4e35610640 1 --2- 172.21.15.81:0/4084580648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e300ca880 0x7f4e300cd460 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.197+0000 7f4e35610640 1 --2- 172.21.15.81:0/4084580648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e30159f40 0x7f4e3006b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.197+0000 7f4e35610640 1 -- 172.21.15.81:0/4084580648 >> 172.21.15.81:0/4084580648 conn(0x7f4e300cbe30 msgr2=0x7f4e300cc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.197+0000 7f4e35610640 1 -- 172.21.15.81:0/4084580648 shutdown_connections 2023-11-08T16:32:42.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.197+0000 7f4e35610640 1 -- 172.21.15.81:0/4084580648 wait complete. 2023-11-08T16:32:42.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.198+0000 7f4e35610640 1 Processor -- start 2023-11-08T16:32:42.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.199+0000 7f4e35610640 1 -- start start 2023-11-08T16:32:42.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.199+0000 7f4e35610640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e300c9cb0 0x7f4e301f2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:42.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e35610640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e300ca880 0x7f4e301f68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:42.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e35610640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e30159f40 0x7f4e30153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:42.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e35610640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e301f2e50 con 0x7f4e300c9cb0 2023-11-08T16:32:42.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e35610640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e300b2f20 con 0x7f4e300ca880 2023-11-08T16:32:42.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e35610640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e301f6e20 con 0x7f4e30159f40 2023-11-08T16:32:42.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e300c9cb0 0x7f4e301f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:42.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e300c9cb0 0x7f4e301f2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43652/0 (socket says 172.21.15.81:43652) 2023-11-08T16:32:42.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2effd640 1 -- 172.21.15.81:0/2640455215 learned_addr learned my addr 172.21.15.81:0/2640455215 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:42.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2effd640 1 -- 172.21.15.81:0/2640455215 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e30159f40 msgr2=0x7f4e30153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:42.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2f7fe640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e30159f40 0x7f4e30153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:42.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2e7fc640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e300ca880 0x7f4e301f68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:42.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2effd640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e30159f40 0x7f4e30153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:42.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2effd640 1 -- 172.21.15.81:0/2640455215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e300ca880 msgr2=0x7f4e301f68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:42.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2effd640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e300ca880 0x7f4e301f68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:42.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2effd640 1 -- 172.21.15.81:0/2640455215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e280094d0 con 0x7f4e300c9cb0 2023-11-08T16:32:42.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2f7fe640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e30159f40 0x7f4e30153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:42.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.200+0000 7f4e2e7fc640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e300ca880 0x7f4e301f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:42.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.201+0000 7f4e2effd640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e300c9cb0 0x7f4e301f2910 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4e20007e40 tx=0x7f4e2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:42.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.201+0000 7f4e277fe640 1 -- 172.21.15.81:0/2640455215 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e20014620 con 0x7f4e300c9cb0 2023-11-08T16:32:42.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.201+0000 7f4e277fe640 1 -- 172.21.15.81:0/2640455215 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2000ee90 con 0x7f4e300c9cb0 2023-11-08T16:32:42.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.201+0000 7f4e277fe640 1 -- 172.21.15.81:0/2640455215 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200149c0 con 0x7f4e300c9cb0 2023-11-08T16:32:42.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.201+0000 7f4e35610640 1 -- 172.21.15.81:0/2640455215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e300b2310 con 0x7f4e300c9cb0 2023-11-08T16:32:42.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.201+0000 7f4e35610640 1 -- 172.21.15.81:0/2640455215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e300b28d0 con 0x7f4e300c9cb0 2023-11-08T16:32:42.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.202+0000 7f4e35610640 1 -- 172.21.15.81:0/2640455215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e300b5470 con 0x7f4e300c9cb0 2023-11-08T16:32:42.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.203+0000 7f4e277fe640 1 -- 172.21.15.81:0/2640455215 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4e20014b20 con 0x7f4e300c9cb0 2023-11-08T16:32:42.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.205+0000 7f4e277fe640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4e1006ae70 0x7f4e1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:42.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.205+0000 7f4e277fe640 1 -- 172.21.15.81:0/2640455215 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f4e2008ace0 con 0x7f4e300c9cb0 2023-11-08T16:32:42.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.205+0000 7f4e2e7fc640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4e1006ae70 0x7f4e1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:42.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.206+0000 7f4e2e7fc640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4e1006ae70 0x7f4e1006d330 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4e18007fd0 tx=0x7f4e18009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:42.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.209+0000 7f4e277fe640 1 -- 172.21.15.81:0/2640455215 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4e20055f50 con 0x7f4e300c9cb0 2023-11-08T16:32:42.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.399+0000 7f4e35610640 1 -- 172.21.15.81:0/2640455215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f4e30067540 con 0x7f4e300c9cb0 2023-11-08T16:32:42.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.979+0000 7f4e277fe640 1 -- 172.21.15.81:0/2640455215 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v55) v1 ==== 170+0+0 (secure 0 0 0) 0x7f4e20059570 con 0x7f4e300c9cb0 2023-11-08T16:32:42.983 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' created 2023-11-08T16:32:42.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 -- 172.21.15.81:0/2640455215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4e1006ae70 msgr2=0x7f4e1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:42.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4e1006ae70 0x7f4e1006d330 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4e18007fd0 tx=0x7f4e18009040 comp rx=0 tx=0).stop 2023-11-08T16:32:42.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 -- 172.21.15.81:0/2640455215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e300c9cb0 msgr2=0x7f4e301f2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:42.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e300c9cb0 0x7f4e301f2910 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4e20007e40 tx=0x7f4e2000e4b0 comp rx=0 tx=0).stop 2023-11-08T16:32:42.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 -- 172.21.15.81:0/2640455215 shutdown_connections 2023-11-08T16:32:42.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4e1006ae70 0x7f4e1006d330 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:42.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4e30159f40 0x7f4e30153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:42.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4e300ca880 0x7f4e301f68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:42.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 --2- 172.21.15.81:0/2640455215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4e300c9cb0 0x7f4e301f2910 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:42.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 -- 172.21.15.81:0/2640455215 >> 172.21.15.81:0/2640455215 conn(0x7f4e300cbe30 msgr2=0x7f4e30070f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:42.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 -- 172.21.15.81:0/2640455215 shutdown_connections 2023-11-08T16:32:42.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:42.983+0000 7f4e35610640 1 -- 172.21.15.81:0/2640455215 wait complete. 2023-11-08T16:32:43.021 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T16:32:43.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.203+0000 7f7c46ad4640 1 Processor -- start 2023-11-08T16:32:43.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.203+0000 7f7c46ad4640 1 -- start start 2023-11-08T16:32:43.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.204+0000 7f7c46ad4640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7c40161730 0x7f7c400cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:43.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c46ad4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c4006d250 0x7f7c400c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:43.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c46ad4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7c400c1e50 0x7f7c4006f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:43.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c46ad4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c400c22e0 con 0x7f7c400c1e50 2023-11-08T16:32:43.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c46ad4640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c4006cae0 con 0x7f7c40161730 2023-11-08T16:32:43.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c46ad4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c400cdb00 con 0x7f7c4006d250 2023-11-08T16:32:43.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c45071640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7c400c1e50 0x7f7c4006f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:43.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c45071640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7c400c1e50 0x7f7c4006f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43662/0 (socket says 172.21.15.81:43662) 2023-11-08T16:32:43.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c45071640 1 -- 172.21.15.81:0/3807814420 learned_addr learned my addr 172.21.15.81:0/3807814420 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:43.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c44870640 1 --2- 172.21.15.81:0/3807814420 >> v2:172.21.15.93:3300/0 conn(0x7f7c40161730 0x7f7c400cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:43.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c45071640 1 -- 172.21.15.81:0/3807814420 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c4006d250 msgr2=0x7f7c400c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:43.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c45071640 1 --2- 172.21.15.81:0/3807814420 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c4006d250 0x7f7c400c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c45071640 1 -- 172.21.15.81:0/3807814420 >> v2:172.21.15.93:3300/0 conn(0x7f7c40161730 msgr2=0x7f7c400cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:43.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c45071640 1 --2- 172.21.15.81:0/3807814420 >> v2:172.21.15.93:3300/0 conn(0x7f7c40161730 0x7f7c400cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c45071640 1 -- 172.21.15.81:0/3807814420 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c40161b10 con 0x7f7c400c1e50 2023-11-08T16:32:43.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.205+0000 7f7c44870640 1 --2- 172.21.15.81:0/3807814420 >> v2:172.21.15.93:3300/0 conn(0x7f7c40161730 0x7f7c400cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:43.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.207+0000 7f7c45071640 1 --2- 172.21.15.81:0/3807814420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c400c1e50 0x7f7c4006f990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009e50 tx=0x7f7c3802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f30421cb5ca6237 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:43.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.207+0000 7f7c3f7fe640 1 -- 172.21.15.81:0/3807814420 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3802ca80 con 0x7f7c400c1e50 2023-11-08T16:32:43.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.207+0000 7f7c3f7fe640 1 -- 172.21.15.81:0/3807814420 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3802cbe0 con 0x7f7c400c1e50 2023-11-08T16:32:43.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.207+0000 7f7c3f7fe640 1 -- 172.21.15.81:0/3807814420 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c380343f0 con 0x7f7c400c1e50 2023-11-08T16:32:43.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.208+0000 7f7c46ad4640 1 -- 172.21.15.81:0/3807814420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c400c1e50 msgr2=0x7f7c4006f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:43.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.208+0000 7f7c46ad4640 1 --2- 172.21.15.81:0/3807814420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c400c1e50 0x7f7c4006f990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009e50 tx=0x7f7c3802bee0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.208+0000 7f7c46ad4640 1 -- 172.21.15.81:0/3807814420 shutdown_connections 2023-11-08T16:32:43.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.208+0000 7f7c46ad4640 1 --2- 172.21.15.81:0/3807814420 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c4006d250 0x7f7c400c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.208+0000 7f7c46ad4640 1 --2- 172.21.15.81:0/3807814420 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c400c1e50 0x7f7c4006f990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.208+0000 7f7c46ad4640 1 --2- 172.21.15.81:0/3807814420 >> v2:172.21.15.93:3300/0 conn(0x7f7c40161730 0x7f7c400cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.208+0000 7f7c46ad4640 1 -- 172.21.15.81:0/3807814420 >> 172.21.15.81:0/3807814420 conn(0x7f7c400cb580 msgr2=0x7f7c400cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:43.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.208+0000 7f7c46ad4640 1 -- 172.21.15.81:0/3807814420 shutdown_connections 2023-11-08T16:32:43.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.208+0000 7f7c46ad4640 1 -- 172.21.15.81:0/3807814420 wait complete. 2023-11-08T16:32:43.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.210+0000 7f7c46ad4640 1 Processor -- start 2023-11-08T16:32:43.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.210+0000 7f7c46ad4640 1 -- start start 2023-11-08T16:32:43.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c46ad4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c4006d250 0x7f7c401519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:43.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c46ad4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c400c1e50 0x7f7c400c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:43.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c46ad4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c40161730 0x7f7c400cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:43.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c46ad4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c400c5190 con 0x7f7c40161730 2023-11-08T16:32:43.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c46ad4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c400c5470 con 0x7f7c400c1e50 2023-11-08T16:32:43.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c46ad4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c40161ec0 con 0x7f7c4006d250 2023-11-08T16:32:43.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c45071640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c40161730 0x7f7c400cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:43.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c45071640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c40161730 0x7f7c400cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43670/0 (socket says 172.21.15.81:43670) 2023-11-08T16:32:43.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c45071640 1 -- 172.21.15.81:0/2896770208 learned_addr learned my addr 172.21.15.81:0/2896770208 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:43.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c44870640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c4006d250 0x7f7c401519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:43.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c45071640 1 -- 172.21.15.81:0/2896770208 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c4006d250 msgr2=0x7f7c401519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:43.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c3ffff640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c400c1e50 0x7f7c400c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:43.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c45071640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c4006d250 0x7f7c401519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c45071640 1 -- 172.21.15.81:0/2896770208 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c400c1e50 msgr2=0x7f7c400c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:43.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c45071640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c400c1e50 0x7f7c400c2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c45071640 1 -- 172.21.15.81:0/2896770208 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c38009b30 con 0x7f7c40161730 2023-11-08T16:32:43.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c44870640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c4006d250 0x7f7c401519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:43.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.211+0000 7f7c3ffff640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c400c1e50 0x7f7c400c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:43.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.212+0000 7f7c45071640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c40161730 0x7f7c400cdf90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009e50 tx=0x7f7c38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:43.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.212+0000 7f7c3d7fa640 1 -- 172.21.15.81:0/2896770208 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c38009510 con 0x7f7c40161730 2023-11-08T16:32:43.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.212+0000 7f7c3d7fa640 1 -- 172.21.15.81:0/2896770208 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3803e450 con 0x7f7c40161730 2023-11-08T16:32:43.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.212+0000 7f7c3d7fa640 1 -- 172.21.15.81:0/2896770208 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3803e800 con 0x7f7c40161730 2023-11-08T16:32:43.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.212+0000 7f7c46ad4640 1 -- 172.21.15.81:0/2896770208 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c401621a0 con 0x7f7c40161730 2023-11-08T16:32:43.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.212+0000 7f7c46ad4640 1 -- 172.21.15.81:0/2896770208 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c400c2b70 con 0x7f7c40161730 2023-11-08T16:32:43.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.213+0000 7f7c46ad4640 1 -- 172.21.15.81:0/2896770208 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c40162660 con 0x7f7c40161730 2023-11-08T16:32:43.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.214+0000 7f7c3d7fa640 1 -- 172.21.15.81:0/2896770208 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c3802cd50 con 0x7f7c40161730 2023-11-08T16:32:43.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.215+0000 7f7c3d7fa640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c2406ae70 0x7f7c2406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:43.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.216+0000 7f7c3d7fa640 1 -- 172.21.15.81:0/2896770208 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f7c380ac250 con 0x7f7c40161730 2023-11-08T16:32:43.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.216+0000 7f7c44870640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c2406ae70 0x7f7c2406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:43.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.216+0000 7f7c44870640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c2406ae70 0x7f7c2406d330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7c34006fd0 tx=0x7f7c34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:43.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.220+0000 7f7c3d7fa640 1 -- 172.21.15.81:0/2896770208 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c38077370 con 0x7f7c40161730 2023-11-08T16:32:43.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.446+0000 7f7c46ad4640 1 -- 172.21.15.81:0/2896770208 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f7c4004ece0 con 0x7f7c40161730 2023-11-08T16:32:43.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.984+0000 7f7c3d7fa640 1 -- 172.21.15.81:0/2896770208 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v36) v1 ==== 172+0+0 (secure 0 0 0) 0x7f7c3807a990 con 0x7f7c40161730 2023-11-08T16:32:43.987 INFO:teuthology.orchestra.run.smithi081.stderr:new fs with metadata pool 7 and data pool 8 2023-11-08T16:32:43.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.988+0000 7f7c46ad4640 1 -- 172.21.15.81:0/2896770208 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c2406ae70 msgr2=0x7f7c2406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:43.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.988+0000 7f7c46ad4640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c2406ae70 0x7f7c2406d330 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7c34006fd0 tx=0x7f7c34008040 comp rx=0 tx=0).stop 2023-11-08T16:32:43.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.988+0000 7f7c46ad4640 1 -- 172.21.15.81:0/2896770208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c40161730 msgr2=0x7f7c400cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:43.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.988+0000 7f7c46ad4640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c40161730 0x7f7c400cdf90 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7c38009e50 tx=0x7f7c38004a40 comp rx=0 tx=0).stop 2023-11-08T16:32:43.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.989+0000 7f7c46ad4640 1 -- 172.21.15.81:0/2896770208 shutdown_connections 2023-11-08T16:32:43.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.989+0000 7f7c46ad4640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c2406ae70 0x7f7c2406d330 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.989+0000 7f7c46ad4640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c40161730 0x7f7c400cdf90 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.989+0000 7f7c46ad4640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c400c1e50 0x7f7c400c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.989+0000 7f7c46ad4640 1 --2- 172.21.15.81:0/2896770208 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c4006d250 0x7f7c401519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:43.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.989+0000 7f7c46ad4640 1 -- 172.21.15.81:0/2896770208 >> 172.21.15.81:0/2896770208 conn(0x7f7c400cb580 msgr2=0x7f7c40160140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:43.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.989+0000 7f7c46ad4640 1 -- 172.21.15.81:0/2896770208 shutdown_connections 2023-11-08T16:32:43.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:43.990+0000 7f7c46ad4640 1 -- 172.21.15.81:0/2896770208 wait complete. 2023-11-08T16:32:44.035 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:32:44.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.210+0000 7fc883d1d640 1 Processor -- start 2023-11-08T16:32:44.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.210+0000 7fc883d1d640 1 -- start start 2023-11-08T16:32:44.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.211+0000 7fc883d1d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc87c0cc5e0 0x7fc87c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.212+0000 7fc883d1d640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87c0c9a70 con 0x7fc87c0adcd0 2023-11-08T16:32:44.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.212+0000 7fc883d1d640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc87c0cc3e0 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.212+0000 7fc883d1d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87c06d070 con 0x7fc87c0cc5e0 2023-11-08T16:32:44.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.212+0000 7fc881ab9640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc87c0adcd0 0x7fc87c0ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51874/0 (socket says 172.21.15.81:51874) 2023-11-08T16:32:44.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.212+0000 7fc881ab9640 1 -- 172.21.15.81:0/1855776845 learned_addr learned my addr 172.21.15.81:0/1855776845 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:44.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.212+0000 7fc8812b8640 1 --2- 172.21.15.81:0/1855776845 >> v2:172.21.15.81:3300/0 conn(0x7fc87c0cc5e0 0x7fc87c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:44.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.212+0000 7fc880ab7640 1 -- 172.21.15.81:0/1855776845 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914872210 0 0) 0x7fc87c0c9a70 con 0x7fc87c0adcd0 2023-11-08T16:32:44.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.212+0000 7fc880ab7640 1 -- 172.21.15.81:0/1855776845 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc868003520 con 0x7fc87c0adcd0 2023-11-08T16:32:44.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.212+0000 7fc8812b8640 1 -- 172.21.15.81:0/1855776845 >> v1:172.21.15.93:6789/0 conn(0x7fc87c0ccfb0 legacy=0x7fc87c070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:44.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.212+0000 7fc8812b8640 1 -- 172.21.15.81:0/1855776845 >> v1:172.21.15.81:6789/0 conn(0x7fc87c0adcd0 legacy=0x7fc87c0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.212+0000 7fc8812b8640 1 -- 172.21.15.81:0/1855776845 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c06d350 con 0x7fc87c0cc5e0 2023-11-08T16:32:44.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.213+0000 7fc8812b8640 1 --2- 172.21.15.81:0/1855776845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc87c0cc5e0 0x7fc87c0cca70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc8700097a0 tx=0x7fc87002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4e79aba2f340e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.214+0000 7fc880ab7640 1 -- 172.21.15.81:0/1855776845 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870031830 con 0x7fc87c0cc5e0 2023-11-08T16:32:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.214+0000 7fc880ab7640 1 -- 172.21.15.81:0/1855776845 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc870031990 con 0x7fc87c0cc5e0 2023-11-08T16:32:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.214+0000 7fc880ab7640 1 -- 172.21.15.81:0/1855776845 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870031d20 con 0x7fc87c0cc5e0 2023-11-08T16:32:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.214+0000 7fc883d1d640 1 -- 172.21.15.81:0/1855776845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc87c0cc5e0 msgr2=0x7fc87c0cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.214+0000 7fc883d1d640 1 --2- 172.21.15.81:0/1855776845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc87c0cc5e0 0x7fc87c0cca70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc8700097a0 tx=0x7fc87002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.214+0000 7fc883d1d640 1 -- 172.21.15.81:0/1855776845 shutdown_connections 2023-11-08T16:32:44.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.214+0000 7fc883d1d640 1 --2- 172.21.15.81:0/1855776845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc87c0cc5e0 0x7fc87c0cca70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.214+0000 7fc883d1d640 1 -- 172.21.15.81:0/1855776845 >> 172.21.15.81:0/1855776845 conn(0x7fc87c0cb490 msgr2=0x7fc87c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:44.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.214+0000 7fc883d1d640 1 -- 172.21.15.81:0/1855776845 shutdown_connections 2023-11-08T16:32:44.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.214+0000 7fc883d1d640 1 -- 172.21.15.81:0/1855776845 wait complete. 2023-11-08T16:32:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.216+0000 7fc883d1d640 1 Processor -- start 2023-11-08T16:32:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.216+0000 7fc883d1d640 1 -- start start 2023-11-08T16:32:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.217+0000 7fc883d1d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc87c0adcd0 0x7fc87c1539a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.217+0000 7fc883d1d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc87c0cc5e0 0x7fc87c1533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.217+0000 7fc883d1d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc87c0ccfb0 0x7fc87c1f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.217+0000 7fc883d1d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c1f6f90 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.217+0000 7fc883d1d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c154220 con 0x7fc87c0cc5e0 2023-11-08T16:32:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.217+0000 7fc883d1d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87c157400 con 0x7fc87c0adcd0 2023-11-08T16:32:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.217+0000 7fc8822ba640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc87c0ccfb0 0x7fc87c1f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.217+0000 7fc8822ba640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc87c0ccfb0 0x7fc87c1f6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43692/0 (socket says 172.21.15.81:43692) 2023-11-08T16:32:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.217+0000 7fc8822ba640 1 -- 172.21.15.81:0/713510078 learned_addr learned my addr 172.21.15.81:0/713510078 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc8812b8640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc87c0cc5e0 0x7fc87c1533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc881ab9640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc87c0adcd0 0x7fc87c1539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:44.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc8822ba640 1 -- 172.21.15.81:0/713510078 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc87c0adcd0 msgr2=0x7fc87c1539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc8822ba640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc87c0adcd0 0x7fc87c1539a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc8822ba640 1 -- 172.21.15.81:0/713510078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc87c0cc5e0 msgr2=0x7fc87c1533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc8822ba640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc87c0cc5e0 0x7fc87c1533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc8822ba640 1 -- 172.21.15.81:0/713510078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc870009400 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc881ab9640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc87c0adcd0 0x7fc87c1539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:44.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc8822ba640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc87c0ccfb0 0x7fc87c1f6a50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc87400ba10 tx=0x7fc87400bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:44.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc86e7fc640 1 -- 172.21.15.81:0/713510078 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87400cba0 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc86e7fc640 1 -- 172.21.15.81:0/713510078 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87400cd00 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.218+0000 7fc86e7fc640 1 -- 172.21.15.81:0/713510078 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc874014680 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.219+0000 7fc883d1d640 1 -- 172.21.15.81:0/713510078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87c1576e0 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.219+0000 7fc883d1d640 1 -- 172.21.15.81:0/713510078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87c1556b0 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.220+0000 7fc883d1d640 1 -- 172.21.15.81:0/713510078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87c072170 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.221+0000 7fc86e7fc640 1 -- 172.21.15.81:0/713510078 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc8740147e0 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.222+0000 7fc86e7fc640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc86006ae70 0x7fc86006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.222+0000 7fc86e7fc640 1 -- 172.21.15.81:0/713510078 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc874089480 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.222+0000 7fc881ab9640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc86006ae70 0x7fc86006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.222+0000 7fc881ab9640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc86006ae70 0x7fc86006d330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc878007fd0 tx=0x7fc878009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.226+0000 7fc86e7fc640 1 -- 172.21.15.81:0/713510078 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc8740545c0 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.408+0000 7fc883d1d640 1 -- 172.21.15.81:0/713510078 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc87c155500 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.409+0000 7fc86e7fc640 1 -- 172.21.15.81:0/713510078 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16522 (secure 0 0 0) 0x7fc874057be0 con 0x7fc87c0ccfb0 2023-11-08T16:32:44.413 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:44.413 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":56,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:32:43.448984+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:32:41.268027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T16:32:42.449607+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:32:44.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 -- 172.21.15.81:0/713510078 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc86006ae70 msgr2=0x7fc86006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc86006ae70 0x7fc86006d330 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc878007fd0 tx=0x7fc878009040 comp rx=0 tx=0).stop 2023-11-08T16:32:44.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 -- 172.21.15.81:0/713510078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc87c0ccfb0 msgr2=0x7fc87c1f6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc87c0ccfb0 0x7fc87c1f6a50 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc87400ba10 tx=0x7fc87400bed0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 -- 172.21.15.81:0/713510078 shutdown_connections 2023-11-08T16:32:44.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc86006ae70 0x7fc86006d330 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc87c0ccfb0 0x7fc87c1f6a50 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc87c0cc5e0 0x7fc87c1533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 --2- 172.21.15.81:0/713510078 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc87c0adcd0 0x7fc87c1539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 -- 172.21.15.81:0/713510078 >> 172.21.15.81:0/713510078 conn(0x7fc87c0cb490 msgr2=0x7fc87c06bec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:44.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 -- 172.21.15.81:0/713510078 shutdown_connections 2023-11-08T16:32:44.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.412+0000 7fc883d1d640 1 -- 172.21.15.81:0/713510078 wait complete. 2023-11-08T16:32:44.433 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:32:44.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.629+0000 7f519cdf5640 1 Processor -- start 2023-11-08T16:32:44.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.629+0000 7f519cdf5640 1 -- start start 2023-11-08T16:32:44.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.629+0000 7f519cdf5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51980ade50 0x7f51980ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.629+0000 7f519cdf5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f519815a680 0x7f519815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.629+0000 7f519cdf5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519806b630 con 0x7f519815b020 2023-11-08T16:32:44.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.629+0000 7f519cdf5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51980b1860 con 0x7f519815a680 2023-11-08T16:32:44.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.629+0000 7f519cdf5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51980ca9c0 con 0x7f51980ade50 2023-11-08T16:32:44.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.630+0000 7f5196d9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f519815b020 0x7f51980c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51878/0 (socket says 172.21.15.81:51878) 2023-11-08T16:32:44.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.630+0000 7f5196d9d640 1 -- 172.21.15.81:0/2680780045 learned_addr learned my addr 172.21.15.81:0/2680780045 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:44.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.630+0000 7f519659c640 1 --2- 172.21.15.81:0/2680780045 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51980ade50 0x7f51980ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:44.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.630+0000 7f518dd9b640 1 --2- 172.21.15.81:0/2680780045 >> v2:172.21.15.93:3300/0 conn(0x7f519815a680 0x7f519815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:44.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.630+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1599847737 0 0) 0x7f519806b630 con 0x7f519815b020 2023-11-08T16:32:44.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.630+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5184003520 con 0x7f519815b020 2023-11-08T16:32:44.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1010543683 0 0) 0x7f5184003520 con 0x7f519815b020 2023-11-08T16:32:44.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51980ade50 msgr2=0x7f51980ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:44.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 --2- 172.21.15.81:0/2680780045 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51980ade50 0x7f51980ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 >> v2:172.21.15.93:3300/0 conn(0x7f519815a680 msgr2=0x7f519815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:44.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 --2- 172.21.15.81:0/2680780045 >> v2:172.21.15.93:3300/0 conn(0x7f519815a680 0x7f519815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980cabb0 con 0x7f519815b020 2023-11-08T16:32:44.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f5190001670 con 0x7f519815b020 2023-11-08T16:32:44.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f518dd9b640 1 --2- 172.21.15.81:0/2680780045 >> v2:172.21.15.93:3300/0 conn(0x7f519815a680 0x7f519815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:32:44.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 >> v1:172.21.15.81:6789/0 conn(0x7f519815b020 legacy=0x7f51980c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 --2- 172.21.15.81:0/2680780045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 0x7f5184003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 --2- 172.21.15.81:0/2680780045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51840047c0 0x7f5184004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 --2- 172.21.15.81:0/2680780045 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51980ade50 0x7f51840051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518400bb10 con 0x7f5184003a50 2023-11-08T16:32:44.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518400bc50 con 0x7f51840047c0 2023-11-08T16:32:44.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518400bde0 con 0x7f51980ade50 2023-11-08T16:32:44.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f519659c640 1 --2- 172.21.15.81:0/2680780045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 0x7f5184003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:44.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f519659c640 1 -- 172.21.15.81:0/2680780045 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51980ade50 msgr2=0x7f51840051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:44.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f519659c640 1 --2- 172.21.15.81:0/2680780045 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51980ade50 0x7f51840051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f519659c640 1 -- 172.21.15.81:0/2680780045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51840047c0 msgr2=0x7f5184004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:44.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f518dd9b640 1 --2- 172.21.15.81:0/2680780045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51840047c0 0x7f5184004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:44.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f519659c640 1 --2- 172.21.15.81:0/2680780045 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51840047c0 0x7f5184004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.631+0000 7f519659c640 1 -- 172.21.15.81:0/2680780045 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5184003520 con 0x7f5184003a50 2023-11-08T16:32:44.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.632+0000 7f519659c640 1 --2- 172.21.15.81:0/2680780045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 0x7f5184003e70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5188007390 tx=0x7f51840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9f4b7333aa6786 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:44.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.633+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518802ccf0 con 0x7f5184003a50 2023-11-08T16:32:44.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.633+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518802ce50 con 0x7f5184003a50 2023-11-08T16:32:44.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.633+0000 7f5195d9b640 1 -- 172.21.15.81:0/2680780045 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188034660 con 0x7f5184003a50 2023-11-08T16:32:44.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.633+0000 7f519cdf5640 1 -- 172.21.15.81:0/2680780045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 msgr2=0x7f5184003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.633+0000 7f519cdf5640 1 --2- 172.21.15.81:0/2680780045 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 0x7f5184003e70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5188007390 tx=0x7f51840057f0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.633+0000 7f5196d9d640 1 -- 172.21.15.81:0/2680780045 reap_dead start 2023-11-08T16:32:44.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.633+0000 7f519cdf5640 1 -- 172.21.15.81:0/2680780045 shutdown_connections 2023-11-08T16:32:44.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.633+0000 7f519cdf5640 1 -- 172.21.15.81:0/2680780045 >> 172.21.15.81:0/2680780045 conn(0x7f51980cb490 msgr2=0x7f51980cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:44.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.633+0000 7f519cdf5640 1 -- 172.21.15.81:0/2680780045 shutdown_connections 2023-11-08T16:32:44.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.633+0000 7f519cdf5640 1 -- 172.21.15.81:0/2680780045 wait complete. 2023-11-08T16:32:44.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.635+0000 7f519cdf5640 1 Processor -- start 2023-11-08T16:32:44.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.635+0000 7f519cdf5640 1 -- start start 2023-11-08T16:32:44.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.636+0000 7f519cdf5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 0x7f51980cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.636+0000 7f519cdf5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519814fc70 0x7f51980cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.636+0000 7f519cdf5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51980ce330 0x7f5198150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.636+0000 7f519cdf5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980cd9d0 con 0x7f5184003a50 2023-11-08T16:32:44.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.636+0000 7f519cdf5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51980ccb20 con 0x7f51980ce330 2023-11-08T16:32:44.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.636+0000 7f519cdf5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51980b4200 con 0x7f519814fc70 2023-11-08T16:32:44.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.636+0000 7f519659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 0x7f51980cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:44.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.636+0000 7f519659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 0x7f51980cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43698/0 (socket says 172.21.15.81:43698) 2023-11-08T16:32:44.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.636+0000 7f519659c640 1 -- 172.21.15.81:0/2271535551 learned_addr learned my addr 172.21.15.81:0/2271535551 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:44.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.637+0000 7f519659c640 1 -- 172.21.15.81:0/2271535551 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519814fc70 msgr2=0x7f51980cddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.637+0000 7f519659c640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519814fc70 0x7f51980cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.637+0000 7f519659c640 1 -- 172.21.15.81:0/2271535551 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51980ce330 msgr2=0x7f5198150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.637+0000 7f519659c640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51980ce330 0x7f5198150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.637+0000 7f519659c640 1 -- 172.21.15.81:0/2271535551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980ccd50 con 0x7f5184003a50 2023-11-08T16:32:44.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.637+0000 7f519659c640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 0x7f51980cc5e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5188037040 tx=0x7f5188004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:44.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.637+0000 7f518f7fe640 1 -- 172.21.15.81:0/2271535551 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188003b90 con 0x7f5184003a50 2023-11-08T16:32:44.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.637+0000 7f518f7fe640 1 -- 172.21.15.81:0/2271535551 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5188003cf0 con 0x7f5184003a50 2023-11-08T16:32:44.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.637+0000 7f519cdf5640 1 -- 172.21.15.81:0/2271535551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51980cd030 con 0x7f5184003a50 2023-11-08T16:32:44.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.637+0000 7f518f7fe640 1 -- 172.21.15.81:0/2271535551 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188004260 con 0x7f5184003a50 2023-11-08T16:32:44.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.638+0000 7f519cdf5640 1 -- 172.21.15.81:0/2271535551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51980cd5f0 con 0x7f5184003a50 2023-11-08T16:32:44.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.638+0000 7f519cdf5640 1 -- 172.21.15.81:0/2271535551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51980cb150 con 0x7f5184003a50 2023-11-08T16:32:44.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.639+0000 7f518f7fe640 1 -- 172.21.15.81:0/2271535551 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f518803b040 con 0x7f5184003a50 2023-11-08T16:32:44.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.640+0000 7f518f7fe640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f517806ae70 0x7f517806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:44.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.640+0000 7f518f7fe640 1 -- 172.21.15.81:0/2271535551 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51880ab650 con 0x7f5184003a50 2023-11-08T16:32:44.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.641+0000 7f518dd9b640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f517806ae70 0x7f517806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:44.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.641+0000 7f518dd9b640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f517806ae70 0x7f517806d330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5180006e80 tx=0x7f5180006e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:44.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.645+0000 7f518f7fe640 1 -- 172.21.15.81:0/2271535551 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51980cb150 con 0x7f5184003a50 2023-11-08T16:32:44.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.821+0000 7f519cdf5640 1 -- 172.21.15.81:0/2271535551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51980cb150 con 0x7f5184003a50 2023-11-08T16:32:44.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.823+0000 7f518f7fe640 1 -- 172.21.15.81:0/2271535551 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16522 (secure 0 0 0) 0x7f51980cb150 con 0x7f5184003a50 2023-11-08T16:32:44.826 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:44.826 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":56,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:32:43.448984+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:32:41.268027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T16:32:42.449607+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:32:44.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.825+0000 7f519cdf5640 1 -- 172.21.15.81:0/2271535551 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f517806ae70 msgr2=0x7f517806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.826+0000 7f519cdf5640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f517806ae70 0x7f517806d330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5180006e80 tx=0x7f5180006e10 comp rx=0 tx=0).stop 2023-11-08T16:32:44.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.826+0000 7f519cdf5640 1 -- 172.21.15.81:0/2271535551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 msgr2=0x7f51980cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:44.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.826+0000 7f519cdf5640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 0x7f51980cc5e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5188037040 tx=0x7f5188004ab0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.826+0000 7f519cdf5640 1 -- 172.21.15.81:0/2271535551 shutdown_connections 2023-11-08T16:32:44.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.826+0000 7f519cdf5640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f517806ae70 0x7f517806d330 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.826+0000 7f519cdf5640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51980ce330 0x7f5198150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.826+0000 7f519cdf5640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519814fc70 0x7f51980cddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.826+0000 7f519cdf5640 1 --2- 172.21.15.81:0/2271535551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5184003a50 0x7f51980cc5e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:44.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.826+0000 7f519cdf5640 1 -- 172.21.15.81:0/2271535551 >> 172.21.15.81:0/2271535551 conn(0x7f51980cb490 msgr2=0x7f51980b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:44.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.826+0000 7f519cdf5640 1 -- 172.21.15.81:0/2271535551 shutdown_connections 2023-11-08T16:32:44.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:44.826+0000 7f519cdf5640 1 -- 172.21.15.81:0/2271535551 wait complete. 2023-11-08T16:32:44.847 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T16:32:45.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.061+0000 7febc2533640 1 Processor -- start 2023-11-08T16:32:45.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.061+0000 7febc2533640 1 -- start start 2023-11-08T16:32:45.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.062+0000 7febc2533640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febbc15bfc0 0x7febbc0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:45.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.062+0000 7febc2533640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febbc15f280 0x7febbc1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:45.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.062+0000 7febc2533640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febbc06fa30 con 0x7febbc0ce0d0 2023-11-08T16:32:45.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.062+0000 7febc2533640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febbc0cda10 con 0x7febbc15bfc0 2023-11-08T16:32:45.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.062+0000 7febc2533640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0ce4b0 con 0x7febbc15f280 2023-11-08T16:32:45.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.062+0000 7febbb7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febbc15bfc0 0x7febbc0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:45.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.062+0000 7febbb7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febbc15bfc0 0x7febbc0caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43706/0 (socket says 172.21.15.81:43706) 2023-11-08T16:32:45.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.062+0000 7febbb7fe640 1 -- 172.21.15.81:0/1900361042 learned_addr learned my addr 172.21.15.81:0/1900361042 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:45.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.063+0000 7febc0ad0640 1 --2- 172.21.15.81:0/1900361042 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febbc15f280 0x7febbc1516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:45.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.063+0000 7febbb7fe640 1 -- 172.21.15.81:0/1900361042 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febbc15f280 msgr2=0x7febbc1516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:45.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.063+0000 7febbb7fe640 1 --2- 172.21.15.81:0/1900361042 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febbc15f280 0x7febbc1516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:45.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.063+0000 7febbb7fe640 1 -- 172.21.15.81:0/1900361042 >> v1:172.21.15.93:6789/0 conn(0x7febbc0ce0d0 legacy=0x7febbc15eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:45.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.063+0000 7febbb7fe640 1 -- 172.21.15.81:0/1900361042 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febbc06fca0 con 0x7febbc15bfc0 2023-11-08T16:32:45.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.064+0000 7febbb7fe640 1 --2- 172.21.15.81:0/1900361042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febbc15bfc0 0x7febbc0caf20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7febb0009870 tx=0x7febb002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ca2620b3f5b5c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:45.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.064+0000 7febbaffd640 1 -- 172.21.15.81:0/1900361042 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0031640 con 0x7febbc15bfc0 2023-11-08T16:32:45.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.064+0000 7febbaffd640 1 -- 172.21.15.81:0/1900361042 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febb00317a0 con 0x7febbc15bfc0 2023-11-08T16:32:45.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.064+0000 7febbaffd640 1 -- 172.21.15.81:0/1900361042 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febb0031b30 con 0x7febbc15bfc0 2023-11-08T16:32:45.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.065+0000 7febc2533640 1 -- 172.21.15.81:0/1900361042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febbc15bfc0 msgr2=0x7febbc0caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:45.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.065+0000 7febc2533640 1 --2- 172.21.15.81:0/1900361042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febbc15bfc0 0x7febbc0caf20 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7febb0009870 tx=0x7febb002b8f0 comp rx=0 tx=0).stop 2023-11-08T16:32:45.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.065+0000 7febc2533640 1 -- 172.21.15.81:0/1900361042 shutdown_connections 2023-11-08T16:32:45.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.065+0000 7febc2533640 1 --2- 172.21.15.81:0/1900361042 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febbc15f280 0x7febbc1516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:45.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.065+0000 7febc2533640 1 --2- 172.21.15.81:0/1900361042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febbc15bfc0 0x7febbc0caf20 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:45.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.065+0000 7febc2533640 1 -- 172.21.15.81:0/1900361042 >> 172.21.15.81:0/1900361042 conn(0x7febbc0cb580 msgr2=0x7febbc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:45.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.065+0000 7febc2533640 1 -- 172.21.15.81:0/1900361042 shutdown_connections 2023-11-08T16:32:45.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.065+0000 7febc2533640 1 -- 172.21.15.81:0/1900361042 wait complete. 2023-11-08T16:32:45.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.067+0000 7febc2533640 1 Processor -- start 2023-11-08T16:32:45.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.067+0000 7febc2533640 1 -- start start 2023-11-08T16:32:45.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febc2533640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febbc0ce0d0 0x7febbc0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:45.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febc2533640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febbc15bfc0 0x7febbc0c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:45.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febc2533640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febbc15f280 0x7febbc0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:45.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febc2533640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0b14f0 con 0x7febbc0ce0d0 2023-11-08T16:32:45.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febc2533640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0c88b0 con 0x7febbc15bfc0 2023-11-08T16:32:45.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febc2533640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febbc0b17b0 con 0x7febbc15f280 2023-11-08T16:32:45.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febbbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febbc0ce0d0 0x7febbc0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:45.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febbbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febbc0ce0d0 0x7febbc0b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43718/0 (socket says 172.21.15.81:43718) 2023-11-08T16:32:45.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febbbfff640 1 -- 172.21.15.81:0/4114768574 learned_addr learned my addr 172.21.15.81:0/4114768574 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:45.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febbbfff640 1 -- 172.21.15.81:0/4114768574 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febbc15f280 msgr2=0x7febbc0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:45.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febbbfff640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febbc15f280 0x7febbc0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:45.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febbbfff640 1 -- 172.21.15.81:0/4114768574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febbc15bfc0 msgr2=0x7febbc0c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:45.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febbbfff640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febbc15bfc0 0x7febbc0c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:45.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.068+0000 7febbbfff640 1 -- 172.21.15.81:0/4114768574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb00094d0 con 0x7febbc0ce0d0 2023-11-08T16:32:45.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.069+0000 7febbbfff640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febbc0ce0d0 0x7febbc0b3110 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7febac00b3f0 tx=0x7febac00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:45.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.069+0000 7febb8ff9640 1 -- 172.21.15.81:0/4114768574 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac011830 con 0x7febbc0ce0d0 2023-11-08T16:32:45.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.069+0000 7febb8ff9640 1 -- 172.21.15.81:0/4114768574 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febac011990 con 0x7febbc0ce0d0 2023-11-08T16:32:45.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.069+0000 7febb8ff9640 1 -- 172.21.15.81:0/4114768574 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febac011d40 con 0x7febbc0ce0d0 2023-11-08T16:32:45.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.069+0000 7febc2533640 1 -- 172.21.15.81:0/4114768574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febbc0b1a90 con 0x7febbc0ce0d0 2023-11-08T16:32:45.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.069+0000 7febc2533640 1 -- 172.21.15.81:0/4114768574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febbc1f3a50 con 0x7febbc0ce0d0 2023-11-08T16:32:45.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.070+0000 7febc2533640 1 -- 172.21.15.81:0/4114768574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febbc0b1fb0 con 0x7febbc0ce0d0 2023-11-08T16:32:45.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.071+0000 7febb8ff9640 1 -- 172.21.15.81:0/4114768574 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7febac01a400 con 0x7febbc0ce0d0 2023-11-08T16:32:45.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.073+0000 7febb8ff9640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feba806ae70 0x7feba806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:45.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.073+0000 7febb8ff9640 1 -- 172.21.15.81:0/4114768574 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febac0893c0 con 0x7febbc0ce0d0 2023-11-08T16:32:45.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.073+0000 7febbb7fe640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feba806ae70 0x7feba806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:45.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.074+0000 7febbb7fe640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feba806ae70 0x7feba806d330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7febb0036000 tx=0x7febb0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:45.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.078+0000 7febb8ff9640 1 -- 172.21.15.81:0/4114768574 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7febbc0b1fb0 con 0x7febbc0ce0d0 2023-11-08T16:32:45.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:45.288+0000 7febc2533640 1 -- 172.21.15.81:0/4114768574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7febbc0b1fb0 con 0x7febbc0ce0d0 2023-11-08T16:32:46.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.021+0000 7febb8ff9640 1 -- 172.21.15.81:0/4114768574 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7febac054500 con 0x7febbc0ce0d0 2023-11-08T16:32:46.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.024+0000 7febc2533640 1 -- 172.21.15.81:0/4114768574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feba806ae70 msgr2=0x7feba806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:46.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.024+0000 7febc2533640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feba806ae70 0x7feba806d330 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7febb0036000 tx=0x7febb0030000 comp rx=0 tx=0).stop 2023-11-08T16:32:46.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.025+0000 7febc2533640 1 -- 172.21.15.81:0/4114768574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febbc0ce0d0 msgr2=0x7febbc0b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:46.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.025+0000 7febc2533640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febbc0ce0d0 0x7febbc0b3110 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7febac00b3f0 tx=0x7febac00b8b0 comp rx=0 tx=0).stop 2023-11-08T16:32:46.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.025+0000 7febc2533640 1 -- 172.21.15.81:0/4114768574 shutdown_connections 2023-11-08T16:32:46.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.025+0000 7febc2533640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feba806ae70 0x7feba806d330 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:46.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.025+0000 7febc2533640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febbc15f280 0x7febbc0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:46.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.025+0000 7febc2533640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febbc15bfc0 0x7febbc0c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:46.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.025+0000 7febc2533640 1 --2- 172.21.15.81:0/4114768574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febbc0ce0d0 0x7febbc0b3110 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:46.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.025+0000 7febc2533640 1 -- 172.21.15.81:0/4114768574 >> 172.21.15.81:0/4114768574 conn(0x7febbc0cb580 msgr2=0x7febbc0c9ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:46.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.025+0000 7febc2533640 1 -- 172.21.15.81:0/4114768574 shutdown_connections 2023-11-08T16:32:46.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.025+0000 7febc2533640 1 -- 172.21.15.81:0/4114768574 wait complete. 2023-11-08T16:32:46.046 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T16:32:46.047 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T16:32:46.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.259+0000 7f76858df640 1 Processor -- start 2023-11-08T16:32:46.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.259+0000 7f76858df640 1 -- start start 2023-11-08T16:32:46.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.261+0000 7f76858df640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f768015b250 0x7f768015a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:46.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.261+0000 7f76858df640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7680157a80 con 0x7f76801576a0 2023-11-08T16:32:46.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.261+0000 7f76858df640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f768015b6e0 con 0x7f76800b11f0 2023-11-08T16:32:46.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.261+0000 7f76858df640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f768015c360 con 0x7f768015b250 2023-11-08T16:32:46.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.261+0000 7f76848dd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f76801576a0 0x7f7680070c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51888/0 (socket says 172.21.15.81:51888) 2023-11-08T16:32:46.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.261+0000 7f76848dd640 1 -- 172.21.15.81:0/1287569279 learned_addr learned my addr 172.21.15.81:0/1287569279 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:46.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.261+0000 7f767ffff640 1 --2- 172.21.15.81:0/1287569279 >> v2:172.21.15.81:3300/0 conn(0x7f768015b250 0x7f768015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:46.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.262+0000 7f767f7fe640 1 -- 172.21.15.81:0/1287569279 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402427195 0 0) 0x7f7680157a80 con 0x7f76801576a0 2023-11-08T16:32:46.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.262+0000 7f767f7fe640 1 -- 172.21.15.81:0/1287569279 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7670003520 con 0x7f76801576a0 2023-11-08T16:32:46.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.262+0000 7f767ffff640 1 -- 172.21.15.81:0/1287569279 >> v1:172.21.15.93:6789/0 conn(0x7f76800b11f0 legacy=0x7f768015c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:46.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.262+0000 7f767ffff640 1 -- 172.21.15.81:0/1287569279 >> v1:172.21.15.81:6789/0 conn(0x7f76801576a0 legacy=0x7f7680070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:46.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.262+0000 7f767ffff640 1 -- 172.21.15.81:0/1287569279 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f768015b9c0 con 0x7f768015b250 2023-11-08T16:32:46.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.263+0000 7f767ffff640 1 --2- 172.21.15.81:0/1287569279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f768015b250 0x7f768015a010 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f766c009790 tx=0x7f766c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd7aad3b2c23c85 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:46.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.264+0000 7f767f7fe640 1 -- 172.21.15.81:0/1287569279 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c031780 con 0x7f768015b250 2023-11-08T16:32:46.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.264+0000 7f767f7fe640 1 -- 172.21.15.81:0/1287569279 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766c0318e0 con 0x7f768015b250 2023-11-08T16:32:46.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.264+0000 7f767f7fe640 1 -- 172.21.15.81:0/1287569279 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766c031c70 con 0x7f768015b250 2023-11-08T16:32:46.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.264+0000 7f76858df640 1 -- 172.21.15.81:0/1287569279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f768015b250 msgr2=0x7f768015a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:46.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.264+0000 7f76858df640 1 --2- 172.21.15.81:0/1287569279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f768015b250 0x7f768015a010 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f766c009790 tx=0x7f766c02b820 comp rx=0 tx=0).stop 2023-11-08T16:32:46.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.264+0000 7f76858df640 1 -- 172.21.15.81:0/1287569279 shutdown_connections 2023-11-08T16:32:46.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.264+0000 7f76858df640 1 --2- 172.21.15.81:0/1287569279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f768015b250 0x7f768015a010 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:46.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.264+0000 7f76858df640 1 -- 172.21.15.81:0/1287569279 >> 172.21.15.81:0/1287569279 conn(0x7f76800ce8d0 msgr2=0x7f76800cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:46.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.264+0000 7f76858df640 1 -- 172.21.15.81:0/1287569279 shutdown_connections 2023-11-08T16:32:46.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.264+0000 7f76858df640 1 -- 172.21.15.81:0/1287569279 wait complete. 2023-11-08T16:32:46.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.266+0000 7f76858df640 1 Processor -- start 2023-11-08T16:32:46.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.266+0000 7f76858df640 1 -- start start 2023-11-08T16:32:46.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.267+0000 7f76858df640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76800b11f0 0x7f768014ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:46.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.267+0000 7f76858df640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76801576a0 0x7f76800c4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:46.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.267+0000 7f76858df640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76800c4e50 0x7f7680155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:46.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.267+0000 7f76858df640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76801f2490 con 0x7f76800c4e50 2023-11-08T16:32:46.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.267+0000 7f76858df640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76800c52e0 con 0x7f76800b11f0 2023-11-08T16:32:46.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.267+0000 7f76858df640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76800c5dd0 con 0x7f76801576a0 2023-11-08T16:32:46.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f76850de640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76800c4e50 0x7f7680155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:46.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f76850de640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76800c4e50 0x7f7680155ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43724/0 (socket says 172.21.15.81:43724) 2023-11-08T16:32:46.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f76850de640 1 -- 172.21.15.81:0/3522744347 learned_addr learned my addr 172.21.15.81:0/3522744347 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:46.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f76848dd640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76800b11f0 0x7f768014ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:46.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f767ffff640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76801576a0 0x7f76800c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:46.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f76850de640 1 -- 172.21.15.81:0/3522744347 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76801576a0 msgr2=0x7f76800c4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:46.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f76850de640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76801576a0 0x7f76800c4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:46.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f76850de640 1 -- 172.21.15.81:0/3522744347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76800b11f0 msgr2=0x7f768014ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:46.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f76850de640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76800b11f0 0x7f768014ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:46.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f76850de640 1 -- 172.21.15.81:0/3522744347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f766c0093a0 con 0x7f76800c4e50 2023-11-08T16:32:46.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f767ffff640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76801576a0 0x7f76800c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:46.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.268+0000 7f76850de640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76800c4e50 0x7f7680155ce0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f767800c7b0 tx=0x7f767800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:46.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.269+0000 7f767d7fa640 1 -- 172.21.15.81:0/3522744347 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7678004b60 con 0x7f76800c4e50 2023-11-08T16:32:46.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.269+0000 7f767d7fa640 1 -- 172.21.15.81:0/3522744347 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7678004cc0 con 0x7f76800c4e50 2023-11-08T16:32:46.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.269+0000 7f767d7fa640 1 -- 172.21.15.81:0/3522744347 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f767800aa30 con 0x7f76800c4e50 2023-11-08T16:32:46.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.269+0000 7f76858df640 1 -- 172.21.15.81:0/3522744347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76800c95e0 con 0x7f76800c4e50 2023-11-08T16:32:46.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.269+0000 7f76858df640 1 -- 172.21.15.81:0/3522744347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7680156cd0 con 0x7f76800c4e50 2023-11-08T16:32:46.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.270+0000 7f76858df640 1 -- 172.21.15.81:0/3522744347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f768004ece0 con 0x7f76800c4e50 2023-11-08T16:32:46.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.271+0000 7f767d7fa640 1 -- 172.21.15.81:0/3522744347 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f767800ab90 con 0x7f76800c4e50 2023-11-08T16:32:46.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.272+0000 7f767d7fa640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f766406aea0 0x7f766406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:46.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.272+0000 7f767d7fa640 1 -- 172.21.15.81:0/3522744347 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7678089810 con 0x7f76800c4e50 2023-11-08T16:32:46.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.272+0000 7f76848dd640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f766406aea0 0x7f766406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:46.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.273+0000 7f76848dd640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f766406aea0 0x7f766406d360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f76740011e0 tx=0x7f7674009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:46.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.276+0000 7f767d7fa640 1 -- 172.21.15.81:0/3522744347 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7678054820 con 0x7f76800c4e50 2023-11-08T16:32:46.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:46.503+0000 7f76858df640 1 -- 172.21.15.81:0/3522744347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f76801548a0 con 0x7f76800c4e50 2023-11-08T16:32:46.918 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:32:46.928 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:32:47.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.032+0000 7f767d7fa640 1 -- 172.21.15.81:0/3522744347 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7678057e40 con 0x7f76800c4e50 2023-11-08T16:32:47.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 -- 172.21.15.81:0/3522744347 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f766406aea0 msgr2=0x7f766406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f766406aea0 0x7f766406d360 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f76740011e0 tx=0x7f7674009040 comp rx=0 tx=0).stop 2023-11-08T16:32:47.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 -- 172.21.15.81:0/3522744347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76800c4e50 msgr2=0x7f7680155ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76800c4e50 0x7f7680155ce0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f767800c7b0 tx=0x7f767800cc70 comp rx=0 tx=0).stop 2023-11-08T16:32:47.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 -- 172.21.15.81:0/3522744347 shutdown_connections 2023-11-08T16:32:47.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f766406aea0 0x7f766406d360 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76800c4e50 0x7f7680155ce0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76801576a0 0x7f76800c4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 --2- 172.21.15.81:0/3522744347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76800b11f0 0x7f768014ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 -- 172.21.15.81:0/3522744347 >> 172.21.15.81:0/3522744347 conn(0x7f76800ce8d0 msgr2=0x7f768015cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:47.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 -- 172.21.15.81:0/3522744347 shutdown_connections 2023-11-08T16:32:47.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.035+0000 7f76858df640 1 -- 172.21.15.81:0/3522744347 wait complete. 2023-11-08T16:32:47.057 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T16:32:47.058 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:32:47.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.261+0000 7f0e343f5640 1 Processor -- start 2023-11-08T16:32:47.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.261+0000 7f0e343f5640 1 -- start start 2023-11-08T16:32:47.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.262+0000 7f0e343f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0e2c0ade50 0x7f0e2c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.262+0000 7f0e343f5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0e2c15a680 0x7f0e2c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.262+0000 7f0e343f5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c06b630 con 0x7f0e2c15b020 2023-11-08T16:32:47.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.262+0000 7f0e343f5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c0b1860 con 0x7f0e2c15a680 2023-11-08T16:32:47.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.262+0000 7f0e343f5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0ca9c0 con 0x7f0e2c0ade50 2023-11-08T16:32:47.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.263+0000 7f0e31990640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0e2c15a680 0x7f0e2c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.263+0000 7f0e31990640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0e2c15a680 0x7f0e2c15aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43738/0 (socket says 172.21.15.81:43738) 2023-11-08T16:32:47.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.263+0000 7f0e31990640 1 -- 172.21.15.81:0/3374008331 learned_addr learned my addr 172.21.15.81:0/3374008331 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:47.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.263+0000 7f0e32191640 1 --2- 172.21.15.81:0/3374008331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0e2c0ade50 0x7f0e2c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.263+0000 7f0e31990640 1 -- 172.21.15.81:0/3374008331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0e2c0ade50 msgr2=0x7f0e2c0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:47.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.263+0000 7f0e31990640 1 --2- 172.21.15.81:0/3374008331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0e2c0ade50 0x7f0e2c0ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.263+0000 7f0e31990640 1 -- 172.21.15.81:0/3374008331 >> v1:172.21.15.93:6789/0 conn(0x7f0e2c15b020 legacy=0x7f0e2c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:47.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.263+0000 7f0e31990640 1 -- 172.21.15.81:0/3374008331 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0cabb0 con 0x7f0e2c15a680 2023-11-08T16:32:47.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.264+0000 7f0e31990640 1 --2- 172.21.15.81:0/3374008331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0e2c15a680 0x7f0e2c15aae0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009870 tx=0x7f0e1c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49140873e01f1f17 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:47.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.265+0000 7f0e3118f640 1 -- 172.21.15.81:0/3374008331 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c031640 con 0x7f0e2c15a680 2023-11-08T16:32:47.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.265+0000 7f0e3118f640 1 -- 172.21.15.81:0/3374008331 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c0317a0 con 0x7f0e2c15a680 2023-11-08T16:32:47.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.265+0000 7f0e3118f640 1 -- 172.21.15.81:0/3374008331 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c031b30 con 0x7f0e2c15a680 2023-11-08T16:32:47.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.265+0000 7f0e343f5640 1 -- 172.21.15.81:0/3374008331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0e2c15a680 msgr2=0x7f0e2c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.265+0000 7f0e343f5640 1 --2- 172.21.15.81:0/3374008331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0e2c15a680 0x7f0e2c15aae0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009870 tx=0x7f0e1c02b8f0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.265+0000 7f0e343f5640 1 -- 172.21.15.81:0/3374008331 shutdown_connections 2023-11-08T16:32:47.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.265+0000 7f0e343f5640 1 --2- 172.21.15.81:0/3374008331 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0e2c15a680 0x7f0e2c15aae0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.265+0000 7f0e343f5640 1 --2- 172.21.15.81:0/3374008331 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0e2c0ade50 0x7f0e2c0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.265+0000 7f0e343f5640 1 -- 172.21.15.81:0/3374008331 >> 172.21.15.81:0/3374008331 conn(0x7f0e2c0cb490 msgr2=0x7f0e2c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:47.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.266+0000 7f0e343f5640 1 -- 172.21.15.81:0/3374008331 shutdown_connections 2023-11-08T16:32:47.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.266+0000 7f0e343f5640 1 -- 172.21.15.81:0/3374008331 wait complete. 2023-11-08T16:32:47.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.267+0000 7f0e343f5640 1 Processor -- start 2023-11-08T16:32:47.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.267+0000 7f0e343f5640 1 -- start start 2023-11-08T16:32:47.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.268+0000 7f0e343f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0e2c0ade50 0x7f0e2c0c3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.268+0000 7f0e343f5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0e2c15a680 0x7f0e2c14bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.268+0000 7f0e343f5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e2c15b020 0x7f0e2c0a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.268+0000 7f0e343f5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c14c4c0 con 0x7f0e2c15a680 2023-11-08T16:32:47.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.268+0000 7f0e343f5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0c4360 con 0x7f0e2c15b020 2023-11-08T16:32:47.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.268+0000 7f0e343f5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0c4640 con 0x7f0e2c0ade50 2023-11-08T16:32:47.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.269+0000 7f0e31990640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0e2c15a680 0x7f0e2c14bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.269+0000 7f0e31990640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0e2c15a680 0x7f0e2c14bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43742/0 (socket says 172.21.15.81:43742) 2023-11-08T16:32:47.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.269+0000 7f0e31990640 1 -- 172.21.15.81:0/2685161881 learned_addr learned my addr 172.21.15.81:0/2685161881 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:47.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.269+0000 7f0e32191640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0e2c0ade50 0x7f0e2c0c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.269+0000 7f0e31990640 1 -- 172.21.15.81:0/2685161881 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0e2c0ade50 msgr2=0x7f0e2c0c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.269+0000 7f0e32992640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e2c15b020 0x7f0e2c0a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.269+0000 7f0e31990640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0e2c0ade50 0x7f0e2c0c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.269+0000 7f0e31990640 1 -- 172.21.15.81:0/2685161881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e2c15b020 msgr2=0x7f0e2c0a99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.269+0000 7f0e31990640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e2c15b020 0x7f0e2c0a99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.269+0000 7f0e31990640 1 -- 172.21.15.81:0/2685161881 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c0094d0 con 0x7f0e2c15a680 2023-11-08T16:32:47.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.269+0000 7f0e31990640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0e2c15a680 0x7f0e2c14bf80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009870 tx=0x7f0e1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:47.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.270+0000 7f0e22ffd640 1 -- 172.21.15.81:0/2685161881 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c03ac60 con 0x7f0e2c15a680 2023-11-08T16:32:47.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.270+0000 7f0e22ffd640 1 -- 172.21.15.81:0/2685161881 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c03ade0 con 0x7f0e2c15a680 2023-11-08T16:32:47.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.270+0000 7f0e22ffd640 1 -- 172.21.15.81:0/2685161881 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c044630 con 0x7f0e2c15a680 2023-11-08T16:32:47.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.270+0000 7f0e343f5640 1 -- 172.21.15.81:0/2685161881 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c0c4920 con 0x7f0e2c15a680 2023-11-08T16:32:47.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.270+0000 7f0e343f5640 1 -- 172.21.15.81:0/2685161881 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c0a9f30 con 0x7f0e2c15a680 2023-11-08T16:32:47.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.271+0000 7f0e343f5640 1 -- 172.21.15.81:0/2685161881 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c0c4e40 con 0x7f0e2c15a680 2023-11-08T16:32:47.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.272+0000 7f0e22ffd640 1 -- 172.21.15.81:0/2685161881 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e1c02f050 con 0x7f0e2c15a680 2023-11-08T16:32:47.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.273+0000 7f0e22ffd640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e1006ae70 0x7f0e1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.273+0000 7f0e22ffd640 1 -- 172.21.15.81:0/2685161881 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e1c0ab5c0 con 0x7f0e2c15a680 2023-11-08T16:32:47.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.273+0000 7f0e32191640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e1006ae70 0x7f0e1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.274+0000 7f0e32191640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e1006ae70 0x7f0e1006d330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0e28006fd0 tx=0x7f0e28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:47.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.278+0000 7f0e22ffd640 1 -- 172.21.15.81:0/2685161881 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0e2c0c4e40 con 0x7f0e2c15a680 2023-11-08T16:32:47.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.507+0000 7f0e343f5640 1 -- 172.21.15.81:0/2685161881 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e2c0c4e40 con 0x7f0e2c15a680 2023-11-08T16:32:47.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.509+0000 7f0e22ffd640 1 -- 172.21.15.81:0/2685161881 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f0e2c0c4e40 con 0x7f0e2c15a680 2023-11-08T16:32:47.511 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:47.511 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3492741644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3492741644},{"type":"v1","addr":"172.21.15.93:6833","nonce":3492741644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":4916,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/119699732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":119699732},{"type":"v1","addr":"172.21.15.93:6835","nonce":119699732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5170,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/1473474760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1473474760},{"type":"v1","addr":"172.21.15.81:6836","nonce":1473474760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:32:43.449426+0000","modified":"2023-11-08T16:32:46.504943+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5979},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5979":{"gid":5979,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/1446454901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":1446454901},{"type":"v1","addr":"172.21.15.81:6837","nonce":1446454901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:32:47.511 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 40 2023-11-08T16:32:47.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.511+0000 7f0e343f5640 1 -- 172.21.15.81:0/2685161881 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e1006ae70 msgr2=0x7f0e1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.511+0000 7f0e343f5640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e1006ae70 0x7f0e1006d330 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0e28006fd0 tx=0x7f0e28008040 comp rx=0 tx=0).stop 2023-11-08T16:32:47.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.511+0000 7f0e343f5640 1 -- 172.21.15.81:0/2685161881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0e2c15a680 msgr2=0x7f0e2c14bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.511+0000 7f0e343f5640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0e2c15a680 0x7f0e2c14bf80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c009870 tx=0x7f0e1c0046c0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.511+0000 7f0e343f5640 1 -- 172.21.15.81:0/2685161881 shutdown_connections 2023-11-08T16:32:47.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.511+0000 7f0e343f5640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e1006ae70 0x7f0e1006d330 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.511+0000 7f0e343f5640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0e2c15b020 0x7f0e2c0a99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.511+0000 7f0e343f5640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0e2c15a680 0x7f0e2c14bf80 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.511+0000 7f0e343f5640 1 --2- 172.21.15.81:0/2685161881 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0e2c0ade50 0x7f0e2c0c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.511+0000 7f0e343f5640 1 -- 172.21.15.81:0/2685161881 >> 172.21.15.81:0/2685161881 conn(0x7f0e2c0cb490 msgr2=0x7f0e2c0afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:47.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.512+0000 7f0e343f5640 1 -- 172.21.15.81:0/2685161881 shutdown_connections 2023-11-08T16:32:47.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.512+0000 7f0e343f5640 1 -- 172.21.15.81:0/2685161881 wait complete. 2023-11-08T16:32:47.531 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:32:47.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.743+0000 7ff8fec90640 1 Processor -- start 2023-11-08T16:32:47.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.743+0000 7ff8fec90640 1 -- start start 2023-11-08T16:32:47.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.744+0000 7ff8fec90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8f815acf0 0x7ff8f80ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.744+0000 7ff8fec90640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff8f806cd50 0x7ff8f815de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.744+0000 7ff8fec90640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f80ad100 con 0x7ff8f80b03d0 2023-11-08T16:32:47.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.744+0000 7ff8fec90640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f80ad3e0 con 0x7ff8f806cd50 2023-11-08T16:32:47.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.744+0000 7ff8fec90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80ad6c0 con 0x7ff8f815acf0 2023-11-08T16:32:47.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.745+0000 7ff8f7fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff8f806cd50 0x7ff8f815de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.745+0000 7ff8f7fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff8f806cd50 0x7ff8f815de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43752/0 (socket says 172.21.15.81:43752) 2023-11-08T16:32:47.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.745+0000 7ff8f7fff640 1 -- 172.21.15.81:0/1198805748 learned_addr learned my addr 172.21.15.81:0/1198805748 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:47.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.745+0000 7ff8fca2c640 1 --2- 172.21.15.81:0/1198805748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8f815acf0 0x7ff8f80ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.745+0000 7ff8f7fff640 1 -- 172.21.15.81:0/1198805748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8f815acf0 msgr2=0x7ff8f80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:47.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.745+0000 7ff8f7fff640 1 --2- 172.21.15.81:0/1198805748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8f815acf0 0x7ff8f80ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.745+0000 7ff8f7fff640 1 -- 172.21.15.81:0/1198805748 >> v1:172.21.15.93:6789/0 conn(0x7ff8f80b03d0 legacy=0x7ff8f8160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:47.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.745+0000 7ff8f7fff640 1 -- 172.21.15.81:0/1198805748 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f815d630 con 0x7ff8f806cd50 2023-11-08T16:32:47.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.745+0000 7ff8fca2c640 1 --2- 172.21.15.81:0/1198805748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8f815acf0 0x7ff8f80ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:47.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.746+0000 7ff8f7fff640 1 --2- 172.21.15.81:0/1198805748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8f806cd50 0x7ff8f815de30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009870 tx=0x7ff8e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a8987dd6c64a3a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:47.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.746+0000 7ff8f77fe640 1 -- 172.21.15.81:0/1198805748 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4031640 con 0x7ff8f806cd50 2023-11-08T16:32:47.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.747+0000 7ff8f77fe640 1 -- 172.21.15.81:0/1198805748 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e40317a0 con 0x7ff8f806cd50 2023-11-08T16:32:47.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.747+0000 7ff8f77fe640 1 -- 172.21.15.81:0/1198805748 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4031b30 con 0x7ff8f806cd50 2023-11-08T16:32:47.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.747+0000 7ff8fec90640 1 -- 172.21.15.81:0/1198805748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8f806cd50 msgr2=0x7ff8f815de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.747+0000 7ff8fec90640 1 --2- 172.21.15.81:0/1198805748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8f806cd50 0x7ff8f815de30 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009870 tx=0x7ff8e402b8f0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.747+0000 7ff8fec90640 1 -- 172.21.15.81:0/1198805748 shutdown_connections 2023-11-08T16:32:47.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.747+0000 7ff8fec90640 1 --2- 172.21.15.81:0/1198805748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8f806cd50 0x7ff8f815de30 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.747+0000 7ff8fec90640 1 --2- 172.21.15.81:0/1198805748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8f815acf0 0x7ff8f80ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.747+0000 7ff8fec90640 1 -- 172.21.15.81:0/1198805748 >> 172.21.15.81:0/1198805748 conn(0x7ff8f80cb3d0 msgr2=0x7ff8f80cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:47.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.747+0000 7ff8fec90640 1 -- 172.21.15.81:0/1198805748 shutdown_connections 2023-11-08T16:32:47.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.747+0000 7ff8fec90640 1 -- 172.21.15.81:0/1198805748 wait complete. 2023-11-08T16:32:47.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.749+0000 7ff8fec90640 1 Processor -- start 2023-11-08T16:32:47.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.749+0000 7ff8fec90640 1 -- start start 2023-11-08T16:32:47.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.750+0000 7ff8fec90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8f806cd50 0x7ff8f81f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.750+0000 7ff8fec90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8f80b03d0 0x7ff8f814cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.750+0000 7ff8fec90640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8f815acf0 0x7ff8f80c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.750+0000 7ff8fec90640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f81f6eb0 con 0x7ff8f806cd50 2023-11-08T16:32:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.750+0000 7ff8fec90640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f814d310 con 0x7ff8f815acf0 2023-11-08T16:32:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.750+0000 7ff8fec90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80c7040 con 0x7ff8f80b03d0 2023-11-08T16:32:47.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.750+0000 7ff8fca2c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8f806cd50 0x7ff8f81f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.750+0000 7ff8fca2c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8f806cd50 0x7ff8f81f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43764/0 (socket says 172.21.15.81:43764) 2023-11-08T16:32:47.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.750+0000 7ff8fca2c640 1 -- 172.21.15.81:0/3781469538 learned_addr learned my addr 172.21.15.81:0/3781469538 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:47.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8fd22d640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8f815acf0 0x7ff8f80c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8fca2c640 1 -- 172.21.15.81:0/3781469538 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8f80b03d0 msgr2=0x7ff8f814cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8f7fff640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8f80b03d0 0x7ff8f814cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8fca2c640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8f80b03d0 0x7ff8f814cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8fca2c640 1 -- 172.21.15.81:0/3781469538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8f815acf0 msgr2=0x7ff8f80c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8fca2c640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8f815acf0 0x7ff8f80c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8fca2c640 1 -- 172.21.15.81:0/3781469538 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e40094d0 con 0x7ff8f806cd50 2023-11-08T16:32:47.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8fca2c640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8f806cd50 0x7ff8f81f6970 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00c7f0 tx=0x7ff8ec00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:47.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8f57fa640 1 -- 172.21.15.81:0/3781469538 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec007a30 con 0x7ff8f806cd50 2023-11-08T16:32:47.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8f57fa640 1 -- 172.21.15.81:0/3781469538 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec007b90 con 0x7ff8f806cd50 2023-11-08T16:32:47.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8f57fa640 1 -- 172.21.15.81:0/3781469538 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec00ce40 con 0x7ff8f806cd50 2023-11-08T16:32:47.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.751+0000 7ff8fec90640 1 -- 172.21.15.81:0/3781469538 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f80c7320 con 0x7ff8f806cd50 2023-11-08T16:32:47.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.752+0000 7ff8fec90640 1 -- 172.21.15.81:0/3781469538 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f80c78e0 con 0x7ff8f806cd50 2023-11-08T16:32:47.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.753+0000 7ff8fec90640 1 -- 172.21.15.81:0/3781469538 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f804ece0 con 0x7ff8f806cd50 2023-11-08T16:32:47.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.754+0000 7ff8f57fa640 1 -- 172.21.15.81:0/3781469538 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff8ec0113f0 con 0x7ff8f806cd50 2023-11-08T16:32:47.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.755+0000 7ff8f57fa640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8dc06ae70 0x7ff8dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:47.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.755+0000 7ff8f57fa640 1 -- 172.21.15.81:0/3781469538 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8ec0897b0 con 0x7ff8f806cd50 2023-11-08T16:32:47.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.755+0000 7ff8f7fff640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8dc06ae70 0x7ff8dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:47.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.756+0000 7ff8f7fff640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8dc06ae70 0x7ff8dc06d330 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4036000 tx=0x7ff8e403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:47.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.759+0000 7ff8f57fa640 1 -- 172.21.15.81:0/3781469538 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff8ec054860 con 0x7ff8f806cd50 2023-11-08T16:32:47.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.949+0000 7ff8fec90640 1 -- 172.21.15.81:0/3781469538 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8f80addc0 con 0x7ff8f806cd50 2023-11-08T16:32:47.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.952+0000 7ff8f57fa640 1 -- 172.21.15.81:0/3781469538 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16509 (secure 0 0 0) 0x7ff8ec057e80 con 0x7ff8f806cd50 2023-11-08T16:32:47.954 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:47.954 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":57,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:32:44.999579+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:32:41.268027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T16:32:42.449607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:32:47.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 -- 172.21.15.81:0/3781469538 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8dc06ae70 msgr2=0x7ff8dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8dc06ae70 0x7ff8dc06d330 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4036000 tx=0x7ff8e403c040 comp rx=0 tx=0).stop 2023-11-08T16:32:47.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 -- 172.21.15.81:0/3781469538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8f806cd50 msgr2=0x7ff8f81f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:47.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8f806cd50 0x7ff8f81f6970 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00c7f0 tx=0x7ff8ec00cbf0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 -- 172.21.15.81:0/3781469538 shutdown_connections 2023-11-08T16:32:47.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff8dc06ae70 0x7ff8dc06d330 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff8f815acf0 0x7ff8f80c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff8f80b03d0 0x7ff8f814cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 --2- 172.21.15.81:0/3781469538 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff8f806cd50 0x7ff8f81f6970 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:47.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 -- 172.21.15.81:0/3781469538 >> 172.21.15.81:0/3781469538 conn(0x7ff8f80cb3d0 msgr2=0x7ff8f80cb7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:47.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 -- 172.21.15.81:0/3781469538 shutdown_connections 2023-11-08T16:32:47.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:47.955+0000 7ff8fec90640 1 -- 172.21.15.81:0/3781469538 wait complete. 2023-11-08T16:32:47.974 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T16:32:47.975 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:32:48.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.179+0000 7f7b9c961640 1 Processor -- start 2023-11-08T16:32:48.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.179+0000 7f7b9c961640 1 -- start start 2023-11-08T16:32:48.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.181+0000 7f7b9c961640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b980adcd0 0x7f7b980ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.181+0000 7f7b9c961640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b980cc5e0 0x7f7b980cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.181+0000 7f7b9c961640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b980c9a70 con 0x7f7b980ccf80 2023-11-08T16:32:48.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.181+0000 7f7b9c961640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b980cc3e0 con 0x7f7b980cc5e0 2023-11-08T16:32:48.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.181+0000 7f7b9c961640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9806d070 con 0x7f7b980adcd0 2023-11-08T16:32:48.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.181+0000 7f7b8dd9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b980cc5e0 0x7f7b980cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:48.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.181+0000 7f7b8dd9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b980cc5e0 0x7f7b980cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43774/0 (socket says 172.21.15.81:43774) 2023-11-08T16:32:48.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.181+0000 7f7b8dd9b640 1 -- 172.21.15.81:0/1136999496 learned_addr learned my addr 172.21.15.81:0/1136999496 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:48.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.182+0000 7f7b9659c640 1 --2- 172.21.15.81:0/1136999496 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b980adcd0 0x7f7b980ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:48.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.182+0000 7f7b8dd9b640 1 -- 172.21.15.81:0/1136999496 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b980adcd0 msgr2=0x7f7b980ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:48.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.182+0000 7f7b8dd9b640 1 --2- 172.21.15.81:0/1136999496 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b980adcd0 0x7f7b980ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.182+0000 7f7b8dd9b640 1 -- 172.21.15.81:0/1136999496 >> v1:172.21.15.93:6789/0 conn(0x7f7b980ccf80 legacy=0x7f7b9815db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:48.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.182+0000 7f7b8dd9b640 1 -- 172.21.15.81:0/1136999496 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9806d350 con 0x7f7b980cc5e0 2023-11-08T16:32:48.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.183+0000 7f7b8dd9b640 1 --2- 172.21.15.81:0/1136999496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b980cc5e0 0x7f7b980cca40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009870 tx=0x7f7b8002db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a52e8a1e90bac7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:48.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.183+0000 7f7b95d9b640 1 -- 172.21.15.81:0/1136999496 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8002f8c0 con 0x7f7b980cc5e0 2023-11-08T16:32:48.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.183+0000 7f7b95d9b640 1 -- 172.21.15.81:0/1136999496 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8002fa20 con 0x7f7b980cc5e0 2023-11-08T16:32:48.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.183+0000 7f7b95d9b640 1 -- 172.21.15.81:0/1136999496 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8002fdb0 con 0x7f7b980cc5e0 2023-11-08T16:32:48.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.184+0000 7f7b9c961640 1 -- 172.21.15.81:0/1136999496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b980cc5e0 msgr2=0x7f7b980cca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:48.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.184+0000 7f7b9c961640 1 --2- 172.21.15.81:0/1136999496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b980cc5e0 0x7f7b980cca40 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009870 tx=0x7f7b8002db00 comp rx=0 tx=0).stop 2023-11-08T16:32:48.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.184+0000 7f7b9c961640 1 -- 172.21.15.81:0/1136999496 shutdown_connections 2023-11-08T16:32:48.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.184+0000 7f7b9c961640 1 --2- 172.21.15.81:0/1136999496 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b980cc5e0 0x7f7b980cca40 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.184+0000 7f7b9c961640 1 --2- 172.21.15.81:0/1136999496 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b980adcd0 0x7f7b980ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.184+0000 7f7b9c961640 1 -- 172.21.15.81:0/1136999496 >> 172.21.15.81:0/1136999496 conn(0x7f7b980cb490 msgr2=0x7f7b980cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:48.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.184+0000 7f7b9c961640 1 -- 172.21.15.81:0/1136999496 shutdown_connections 2023-11-08T16:32:48.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.184+0000 7f7b9c961640 1 -- 172.21.15.81:0/1136999496 wait complete. 2023-11-08T16:32:48.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.186+0000 7f7b9c961640 1 Processor -- start 2023-11-08T16:32:48.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.186+0000 7f7b9c961640 1 -- start start 2023-11-08T16:32:48.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.187+0000 7f7b9c961640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b980adcd0 0x7f7b9814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.187+0000 7f7b9c961640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b980cc5e0 0x7f7b9814f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.187+0000 7f7b9c961640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b980ccf80 0x7f7b981f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.187+0000 7f7b9c961640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980c8990 con 0x7f7b980cc5e0 2023-11-08T16:32:48.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.187+0000 7f7b9c961640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b981e6fb0 con 0x7f7b980adcd0 2023-11-08T16:32:48.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.187+0000 7f7b9c961640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980c86b0 con 0x7f7b980ccf80 2023-11-08T16:32:48.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.187+0000 7f7b8dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b980cc5e0 0x7f7b9814f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:48.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.187+0000 7f7b8dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b980cc5e0 0x7f7b9814f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43790/0 (socket says 172.21.15.81:43790) 2023-11-08T16:32:48.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.187+0000 7f7b8dd9b640 1 -- 172.21.15.81:0/3720824191 learned_addr learned my addr 172.21.15.81:0/3720824191 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:48.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.187+0000 7f7b8dd9b640 1 -- 172.21.15.81:0/3720824191 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b980ccf80 msgr2=0x7f7b981f62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:48.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.187+0000 7f7b8dd9b640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b980ccf80 0x7f7b981f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.188+0000 7f7b8dd9b640 1 -- 172.21.15.81:0/3720824191 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b980adcd0 msgr2=0x7f7b9814eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:48.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.188+0000 7f7b8dd9b640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b980adcd0 0x7f7b9814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.188+0000 7f7b8dd9b640 1 -- 172.21.15.81:0/3720824191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b800094d0 con 0x7f7b980cc5e0 2023-11-08T16:32:48.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.188+0000 7f7b8dd9b640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b980cc5e0 0x7f7b9814f0c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7b8002db40 tx=0x7f7b800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:48.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.188+0000 7f7b8f7fe640 1 -- 172.21.15.81:0/3720824191 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80042880 con 0x7f7b980cc5e0 2023-11-08T16:32:48.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.188+0000 7f7b8f7fe640 1 -- 172.21.15.81:0/3720824191 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b80042a00 con 0x7f7b980cc5e0 2023-11-08T16:32:48.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.188+0000 7f7b9c961640 1 -- 172.21.15.81:0/3720824191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b981e6960 con 0x7f7b980cc5e0 2023-11-08T16:32:48.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.188+0000 7f7b8f7fe640 1 -- 172.21.15.81:0/3720824191 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b80042db0 con 0x7f7b980cc5e0 2023-11-08T16:32:48.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.188+0000 7f7b9c961640 1 -- 172.21.15.81:0/3720824191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9814f8c0 con 0x7f7b980cc5e0 2023-11-08T16:32:48.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.189+0000 7f7b9c961640 1 -- 172.21.15.81:0/3720824191 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b98072170 con 0x7f7b980cc5e0 2023-11-08T16:32:48.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.190+0000 7f7b8f7fe640 1 -- 172.21.15.81:0/3720824191 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b80040440 con 0x7f7b980cc5e0 2023-11-08T16:32:48.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.191+0000 7f7b8f7fe640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7806ae70 0x7f7b7806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.191+0000 7f7b8f7fe640 1 -- 172.21.15.81:0/3720824191 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b80033070 con 0x7f7b980cc5e0 2023-11-08T16:32:48.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.191+0000 7f7b9659c640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7806ae70 0x7f7b7806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:48.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.192+0000 7f7b9659c640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7806ae70 0x7f7b7806d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7b880098f0 tx=0x7f7b88008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:48.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.195+0000 7f7b8f7fe640 1 -- 172.21.15.81:0/3720824191 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b800773f0 con 0x7f7b980cc5e0 2023-11-08T16:32:48.371 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:32:48.370+0000 7f2ebf664640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:32:48.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.385+0000 7f7b9c961640 1 -- 172.21.15.81:0/3720824191 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b980c9cc0 con 0x7f7b7806ae70 2023-11-08T16:32:48.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.409+0000 7f7b8f7fe640 1 -- 172.21.15.81:0/3720824191 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+272220 (secure 0 0 0) 0x7f7b88009e70 con 0x7f7b7806ae70 2023-11-08T16:32:48.415 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:48.416 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:32:48.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.414+0000 7f7b9c961640 1 -- 172.21.15.81:0/3720824191 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7806ae70 msgr2=0x7f7b7806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:48.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.414+0000 7f7b9c961640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7806ae70 0x7f7b7806d330 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7b880098f0 tx=0x7f7b88008040 comp rx=0 tx=0).stop 2023-11-08T16:32:48.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.414+0000 7f7b9c961640 1 -- 172.21.15.81:0/3720824191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b980cc5e0 msgr2=0x7f7b9814f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:48.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.414+0000 7f7b9c961640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b980cc5e0 0x7f7b9814f0c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f7b8002db40 tx=0x7f7b800046c0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.415+0000 7f7b9c961640 1 -- 172.21.15.81:0/3720824191 shutdown_connections 2023-11-08T16:32:48.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.415+0000 7f7b9c961640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7806ae70 0x7f7b7806d330 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.415+0000 7f7b9c961640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b980ccf80 0x7f7b981f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.415+0000 7f7b9c961640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b980cc5e0 0x7f7b9814f0c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.415+0000 7f7b9c961640 1 --2- 172.21.15.81:0/3720824191 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b980adcd0 0x7f7b9814eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.415+0000 7f7b9c961640 1 -- 172.21.15.81:0/3720824191 >> 172.21.15.81:0/3720824191 conn(0x7f7b980cb490 msgr2=0x7f7b980afad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:48.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.415+0000 7f7b9c961640 1 -- 172.21.15.81:0/3720824191 shutdown_connections 2023-11-08T16:32:48.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.415+0000 7f7b9c961640 1 -- 172.21.15.81:0/3720824191 wait complete. 2023-11-08T16:32:48.435 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2023-11-08T16:32:46.622660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":226472,"kb_used_data":6088,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754748248,"statfs":{"total":773094113280,"available":772862205952,"internally_reserved":0,"allocated":6234112,"data_stored":3329328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":225496625},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.654750"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:42.967619+0000","last_change":"2023-11-08T16:32:42.967619+0000","last_active":"2023-11-08T16:32:42.967619+0000","last_peered":"2023-11-08T16:32:42.967619+0000","last_clean":"2023-11-08T16:32:42.967619+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:42.967619+0000","last_undegraded":"2023-11-08T16:32:42.967619+0000","last_fullsized":"2023-11-08T16:32:42.967619+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-08T16:32:41.956433+0000","last_change":"2023-11-08T16:32:41.956433+0000","last_active":"2023-11-08T16:32:41.956433+0000","last_peered":"2023-11-08T16:32:41.956433+0000","last_clean":"2023-11-08T16:32:41.956433+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-08T16:32:41.956433+0000","last_undegraded":"2023-11-08T16:32:41.956433+0000","last_fullsized":"2023-11-08T16:32:41.956433+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":198,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-11-08T16:32:37.946210+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:32:37.946210+0000","last_peered":"2023-11-08T16:32:37.946210+0000","last_clean":"2023-11-08T16:32:37.946210+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:32:37.946210+0000","last_undegraded":"2023-11-08T16:32:37.946210+0000","last_fullsized":"2023-11-08T16:32:37.946210+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1224704,"data_stored":858942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.0800000000000001,"15min":1.085},"min":{"1min":0.77700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.619,"5min":1.738,"15min":1.738},"last":0.82199999999999995},{"interface":"front","average":{"1min":1.248,"5min":1.1180000000000001,"15min":1.1240000000000001},"min":{"1min":0.88200000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5740000000000001,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.2290000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.131,"15min":1.127},"min":{"1min":0.873,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.216,"15min":1.232},"min":{"1min":0.93899999999999995,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.923,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.0649999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2629999999999999,"15min":1.294},"min":{"1min":0.70899999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.992,"5min":4.2249999999999996,"15min":4.2249999999999996},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.1879999999999999,"15min":1.1919999999999999},"min":{"1min":0.82799999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6679999999999999,"5min":1.992,"15min":1.992},"last":1.1739999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.1579999999999999,"15min":1.1539999999999999},"min":{"1min":0.96799999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5800000000000001,"5min":1.609,"15min":1.609},"last":1.034},{"interface":"front","average":{"1min":1.385,"5min":1.454,"15min":1.514},"min":{"1min":1.095,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.772,"5min":4.181,"15min":4.181},"last":1.478}]},{"osd":4,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.236,"15min":1.214},"min":{"1min":0.90500000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.7689999999999999,"5min":1.956,"15min":1.956},"last":1.1279999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.343,"15min":1.343},"min":{"1min":0.92000000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9490000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.264}]},{"osd":5,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3400000000000001,"15min":1.3320000000000001},"min":{"1min":1.0389999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7070000000000001,"5min":2.222,"15min":2.222},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4550000000000001,"15min":1.47},"min":{"1min":0.96199999999999997,"5min":0.878,"15min":0.878},"max":{"1min":1.7250000000000001,"5min":2.589,"15min":2.589},"last":1.548}]},{"osd":6,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5189999999999999,"15min":1.5289999999999999},"min":{"1min":1.111,"5min":1.026,"15min":1.026},"max":{"1min":1.873,"5min":2.496,"15min":2.496},"last":1.611},{"interface":"front","average":{"1min":1.448,"5min":1.458,"15min":1.4710000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.865,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.6399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":268574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0669999999999999,"15min":1.0629999999999999},"min":{"1min":0.74099999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.742,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.099,"5min":1.056,"15min":1.0740000000000001},"min":{"1min":0.70599999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.56999999999999995}]},{"osd":1,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.0640000000000001,"15min":1.024},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.879},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.1279999999999999,"15min":1.113},"min":{"1min":0.94499999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.601,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":1.119}]},{"osd":2,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.216,"15min":1.23},"min":{"1min":0.80100000000000005,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1819999999999999,"15min":1.1839999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5920000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.92400000000000004}]},{"osd":3,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.258,"15min":1.298},"min":{"1min":0.94199999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.821,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":1.274},{"interface":"front","average":{"1min":1.355,"5min":1.323,"15min":1.319},"min":{"1min":0.82999999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.7749999999999999,"5min":1.95,"15min":1.95},"last":0.91000000000000003}]},{"osd":4,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.357,"15min":1.3819999999999999},"min":{"1min":0.86599999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.7430000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.4450000000000001,"15min":1.4550000000000001},"min":{"1min":1.026,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.8859999999999999,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.64900000000000002}]},{"osd":5,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.327,"15min":1.3029999999999999},"min":{"1min":0.94299999999999995,"5min":0.877,"15min":0.877},"max":{"1min":1.665,"5min":2.105,"15min":2.105},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.593,"5min":1.5589999999999999,"15min":1.5740000000000001},"min":{"1min":1.347,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.982,"5min":2.202,"15min":2.202},"last":1.325}]},{"osd":7,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.595,"15min":1.6259999999999999},"min":{"1min":1.3680000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0249999999999999,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.522,"15min":1.534},"min":{"1min":1.169,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":1.762,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.0589999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27652,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344188,"statfs":{"total":96636764160,"available":96608448512,"internally_reserved":0,"allocated":630784,"data_stored":268574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.071,"15min":1.0580000000000001},"min":{"1min":0.68700000000000006,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.712,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.115,"15min":1.1319999999999999},"min":{"1min":0.76000000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5720000000000001,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.23}]},{"osd":1,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.228,"15min":1.246},"min":{"1min":0.878,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6839999999999999,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":1.613},{"interface":"front","average":{"1min":1.242,"5min":1.2669999999999999,"15min":1.3129999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.486,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.1419999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.2310000000000001,"15min":1.2629999999999999},"min":{"1min":0.79300000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6519999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.2070000000000001,"15min":1.2290000000000001},"min":{"1min":0.78500000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5880000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.5089999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2909999999999999,"15min":1.333},"min":{"1min":0.85399999999999998,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6519999999999999,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.026},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3129999999999999,"15min":1.3280000000000001},"min":{"1min":0.89300000000000002,"5min":0.876,"15min":0.876},"max":{"1min":1.6539999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.6659999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.361,"15min":1.3959999999999999},"min":{"1min":0.85499999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6279999999999999,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":1.456},{"interface":"front","average":{"1min":1.345,"5min":1.46,"15min":1.514},"min":{"1min":0.92200000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.6779999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.407}]},{"osd":6,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.403,"15min":1.4279999999999999},"min":{"1min":1.02,"5min":0.752,"15min":0.752},"max":{"1min":1.7130000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.714},{"interface":"front","average":{"1min":1.421,"5min":1.498,"15min":1.542},"min":{"1min":1.0620000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":1.887,"5min":4.1189999999999998,"15min":4.1189999999999998},"last":1.5489999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.552,"15min":1.6000000000000001},"min":{"1min":1.0580000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"max":{"1min":2.101,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.5740000000000001},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5620000000000001,"15min":1.5720000000000001},"min":{"1min":1.2190000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":1.976,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":1.774}]}]},{"osd":4,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27636,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344204,"statfs":{"total":96636764160,"available":96608464896,"internally_reserved":0,"allocated":630784,"data_stored":268574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.056,"15min":1.0680000000000001},"min":{"1min":0.84599999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.319,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.0640000000000001,"15min":1.0409999999999999},"min":{"1min":0.72599999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.78200000000000003}]},{"osd":1,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1240000000000001,"15min":1.1200000000000001},"min":{"1min":0.78300000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.262,"5min":1.1619999999999999,"15min":1.1559999999999999},"min":{"1min":0.80500000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.6879999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.198,"15min":1.1799999999999999},"min":{"1min":0.85499999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.2689999999999999,"15min":1.282},"min":{"1min":0.89200000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.423}]},{"osd":3,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.294,"15min":1.2729999999999999},"min":{"1min":0.85899999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.875,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.2929999999999999,"15min":1.2949999999999999},"min":{"1min":1.0389999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.3400000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3500000000000001,"15min":1.371},"min":{"1min":1.0920000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.6970000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.8959999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.417,"15min":1.421},"min":{"1min":0.86899999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.9830000000000001,"5min":2.46,"15min":2.46},"last":1.788}]},{"osd":6,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4330000000000001,"15min":1.4450000000000001},"min":{"1min":0.95199999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.2890000000000001,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.48,"5min":1.5489999999999999,"15min":1.5880000000000001},"min":{"1min":1.0069999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.1549999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":2.0049999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.536,"15min":1.518},"min":{"1min":1.1299999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":2.077,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.968},{"interface":"front","average":{"1min":1.573,"5min":1.607,"15min":1.6259999999999999},"min":{"1min":1.1719999999999999,"5min":1.169,"15min":1.169},"max":{"1min":2.1949999999999998,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.8440000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574897,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27640,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344200,"statfs":{"total":96636764160,"available":96608460800,"internally_reserved":0,"allocated":630784,"data_stored":268574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.82899999999999996,"15min":0.83799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.94799999999999995,"15min":0.98399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3440000000000001,"5min":1.585,"15min":1.585},"last":1.22}]},{"osd":1,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.89200000000000002,"15min":0.91300000000000003},"min":{"1min":0.45900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.2869999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":1.1559999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.98399999999999999,"15min":0.97799999999999998},"min":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5720000000000001,"5min":1.829,"15min":1.829},"last":1.0249999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.98999999999999999,"15min":0.98299999999999998},"min":{"1min":0.47399999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.452,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.325},{"interface":"front","average":{"1min":1.056,"5min":1.137,"15min":1.1619999999999999},"min":{"1min":0.56999999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.375,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.353}]},{"osd":4,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.216,"15min":1.2270000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8120000000000001,"5min":2.25,"15min":2.25},"last":1.5029999999999999},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.286,"15min":1.2749999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.786,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.5389999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3400000000000001,"15min":1.343},"min":{"1min":0.90300000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.7689999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.446},{"interface":"front","average":{"1min":1.264,"5min":1.3540000000000001,"15min":1.365},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.593,"5min":2.593,"15min":2.593},"last":1.478}]},{"osd":6,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.407,"15min":1.4379999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.6539999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.4159999999999999,"15min":1.452},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.774,"5min":2.762,"15min":2.762},"last":1.591}]},{"osd":7,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4710000000000001,"15min":1.4930000000000001},"min":{"1min":0.79900000000000004,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.827,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.395},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4430000000000001,"15min":1.4610000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6899999999999999,"5min":2.633,"15min":2.633},"last":1.631}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27648,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":630784,"data_stored":268574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":1.0049999999999999,"15min":1.107},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.575,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.89000000000000001,"15min":0.89300000000000002},"min":{"1min":0.49099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":1.2849999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1240000000000001,"15min":1.1859999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.7829999999999999,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.089,"5min":1.0720000000000001,"15min":1.083},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6120000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":0.83799999999999997}]},{"osd":3,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.04,"15min":1.1379999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.746,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.59899999999999998},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.226,"15min":1.3300000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6819999999999999,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":1.149}]},{"osd":4,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.4139999999999999,"15min":1.498},"min":{"1min":0.92400000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.627,"5min":4.9409999999999998,"15min":4.9409999999999998},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6240000000000001,"5min":2.46,"15min":2.46},"last":1.5}]},{"osd":5,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3260000000000001,"15min":1.3799999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.706,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.3600000000000001,"15min":1.4339999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.47,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":1.032}]},{"osd":6,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.4099999999999999,"15min":1.4850000000000001},"min":{"1min":0.66800000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.595,"5min":3.536,"15min":3.536},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.512,"15min":1.5880000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.877,"5min":4.7549999999999999,"15min":4.7549999999999999},"last":1.415}]},{"osd":7,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.4590000000000001,"15min":1.5169999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.7070000000000001,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":1.452},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.502,"15min":1.5900000000000001},"min":{"1min":0.80600000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6020000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.4379999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27644,"kb_used_data":616,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344196,"statfs":{"total":96636764160,"available":96608456704,"internally_reserved":0,"allocated":630784,"data_stored":268574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.76900000000000002,"15min":0.78800000000000003},"min":{"1min":0.39300000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.169,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0129999999999999,"15min":1.0760000000000001},"min":{"1min":0.45100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.6459999999999999,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":0.52200000000000002}]},{"osd":2,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.089,"15min":1.1419999999999999},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.593,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.079,"15min":1.145},"min":{"1min":0.47999999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.4850000000000001,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.54000000000000004}]},{"osd":3,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.113,"15min":1.1619999999999999},"min":{"1min":0.432,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.73,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.105,"5min":1.107,"15min":1.1579999999999999},"min":{"1min":0.58099999999999996,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.548,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":0.57599999999999996}]},{"osd":4,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.3169999999999999,"15min":1.385},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.637,"5min":3.1659999999999999,"15min":3.1659999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.3400000000000001,"15min":1.3979999999999999},"min":{"1min":0.78500000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5840000000000001,"5min":2.734,"15min":2.734},"last":1.343}]},{"osd":5,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.6519999999999999,"15min":1.8260000000000001},"min":{"1min":0.93899999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8799999999999999,"5min":8.4730000000000008,"15min":8.4730000000000008},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.304,"5min":1.6120000000000001,"15min":1.7689999999999999},"min":{"1min":0.72199999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8149999999999999,"5min":8.8040000000000003,"15min":8.8040000000000003},"last":1.3660000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4750000000000001,"15min":1.536},"min":{"1min":0.80400000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.0129999999999999,"5min":3.984,"15min":3.984},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.464,"15min":1.5129999999999999},"min":{"1min":0.74099999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.984,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":1.1359999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.534,"15min":1.6519999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.546,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.472,"5min":1.6240000000000001,"15min":1.724},"min":{"1min":1.022,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.958,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":1.04}]}]},{"osd":0,"up_from":13,"seq":55834574896,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32380,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339460,"statfs":{"total":96636764160,"available":96603607040,"internally_reserved":0,"allocated":1224704,"data_stored":858942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.77200000000000002,"15min":0.78400000000000003},"min":{"1min":0.48899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.202,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.96499999999999997,"15min":0.98699999999999999},"min":{"1min":0.59599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3009999999999999,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":0.498}]},{"osd":2,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.92800000000000005,"15min":0.96699999999999997},"min":{"1min":0.55100000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3560000000000001,"5min":1.762,"15min":1.762},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0149999999999999,"15min":1.0469999999999999},"min":{"1min":0.51600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.395,"5min":1.879,"15min":1.879},"last":1.016}]},{"osd":3,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.94499999999999995,"15min":0.96099999999999997},"min":{"1min":0.438,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.399,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.1930000000000001},"min":{"1min":0.60799999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.528,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1519999999999999,"15min":1.1579999999999999},"min":{"1min":0.68700000000000006,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5529999999999999,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2789999999999999,"15min":1.306},"min":{"1min":0.80300000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.6499999999999999,"5min":2.867,"15min":2.867},"last":0.95899999999999996}]},{"osd":5,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.264,"15min":1.256},"min":{"1min":0.78400000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.599,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.86899999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4990000000000001,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.97599999999999998}]},{"osd":6,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.3420000000000001,"15min":1.367},"min":{"1min":0.83299999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5669999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.371,"15min":1.383},"min":{"1min":0.99099999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7230000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.1359999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.4059999999999999,"15min":1.4159999999999999},"min":{"1min":0.94099999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.6679999999999999,"5min":2.625,"15min":2.625},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.345,"15min":1.371},"min":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.6080000000000001,"5min":2.431,"15min":2.431},"last":1.2370000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:32:48.443 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:32:48.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.694+0000 7f2351c61640 1 Processor -- start 2023-11-08T16:32:48.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.694+0000 7f2351c61640 1 -- start start 2023-11-08T16:32:48.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.696+0000 7f2351c61640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f234c0ade50 0x7f234c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.696+0000 7f2351c61640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f234c15ea60 0x7f234c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.696+0000 7f2351c61640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234c06b630 con 0x7f234c15a680 2023-11-08T16:32:48.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.696+0000 7f2351c61640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234c0b1860 con 0x7f234c0ade50 2023-11-08T16:32:48.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.696+0000 7f2351c61640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f234c0ca9c0 con 0x7f234c15ea60 2023-11-08T16:32:48.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.696+0000 7f234b7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f234c0ade50 0x7f234c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:48.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.696+0000 7f234b7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f234c0ade50 0x7f234c0ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43804/0 (socket says 172.21.15.81:43804) 2023-11-08T16:32:48.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.696+0000 7f234b7fe640 1 -- 172.21.15.81:0/3609033282 learned_addr learned my addr 172.21.15.81:0/3609033282 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:48.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.697+0000 7f234bfff640 1 --2- 172.21.15.81:0/3609033282 >> v2:172.21.15.93:3300/0 conn(0x7f234c15ea60 0x7f234c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:48.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.697+0000 7f234b7fe640 1 -- 172.21.15.81:0/3609033282 >> v2:172.21.15.93:3300/0 conn(0x7f234c15ea60 msgr2=0x7f234c15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:48.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.697+0000 7f234b7fe640 1 --2- 172.21.15.81:0/3609033282 >> v2:172.21.15.93:3300/0 conn(0x7f234c15ea60 0x7f234c15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.697+0000 7f234b7fe640 1 -- 172.21.15.81:0/3609033282 >> v1:172.21.15.81:6789/0 conn(0x7f234c15a680 legacy=0x7f234c15ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:48.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.697+0000 7f234b7fe640 1 -- 172.21.15.81:0/3609033282 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f234c0cabb0 con 0x7f234c0ade50 2023-11-08T16:32:48.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.697+0000 7f234bfff640 1 --2- 172.21.15.81:0/3609033282 >> v2:172.21.15.93:3300/0 conn(0x7f234c15ea60 0x7f234c15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:48.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.698+0000 7f234b7fe640 1 --2- 172.21.15.81:0/3609033282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f234c0ade50 0x7f234c0ca480 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f234c0ca220 tx=0x7f233c02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27fa6392af89fe2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:48.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.698+0000 7f234a7fc640 1 -- 172.21.15.81:0/3609033282 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c030870 con 0x7f234c0ade50 2023-11-08T16:32:48.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.698+0000 7f234a7fc640 1 -- 172.21.15.81:0/3609033282 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c0309d0 con 0x7f234c0ade50 2023-11-08T16:32:48.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.698+0000 7f234a7fc640 1 -- 172.21.15.81:0/3609033282 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c030d60 con 0x7f234c0ade50 2023-11-08T16:32:48.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.699+0000 7f2351c61640 1 -- 172.21.15.81:0/3609033282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f234c0ade50 msgr2=0x7f234c0ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:48.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.699+0000 7f2351c61640 1 --2- 172.21.15.81:0/3609033282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f234c0ade50 0x7f234c0ca480 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f234c0ca220 tx=0x7f233c02b9b0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.699+0000 7f2351c61640 1 -- 172.21.15.81:0/3609033282 shutdown_connections 2023-11-08T16:32:48.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.699+0000 7f2351c61640 1 --2- 172.21.15.81:0/3609033282 >> v2:172.21.15.93:3300/0 conn(0x7f234c15ea60 0x7f234c15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.699+0000 7f2351c61640 1 --2- 172.21.15.81:0/3609033282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f234c0ade50 0x7f234c0ca480 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.699+0000 7f2351c61640 1 -- 172.21.15.81:0/3609033282 >> 172.21.15.81:0/3609033282 conn(0x7f234c0cb490 msgr2=0x7f234c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:48.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.699+0000 7f2351c61640 1 -- 172.21.15.81:0/3609033282 shutdown_connections 2023-11-08T16:32:48.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.699+0000 7f2351c61640 1 -- 172.21.15.81:0/3609033282 wait complete. 2023-11-08T16:32:48.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.701+0000 7f2351c61640 1 Processor -- start 2023-11-08T16:32:48.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.701+0000 7f2351c61640 1 -- start start 2023-11-08T16:32:48.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f2351c61640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f234c0ade50 0x7f234c1e5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f2351c61640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f234c15a680 0x7f234c15b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f2351c61640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f234c15ea60 0x7f234c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f2351c61640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c14f760 con 0x7f234c0ade50 2023-11-08T16:32:48.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f2351c61640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234c15c1d0 con 0x7f234c15ea60 2023-11-08T16:32:48.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f2351c61640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f234c0cce00 con 0x7f234c15a680 2023-11-08T16:32:48.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f234b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f234c0ade50 0x7f234c1e5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:48.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f234b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f234c0ade50 0x7f234c1e5780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43814/0 (socket says 172.21.15.81:43814) 2023-11-08T16:32:48.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f234b7fe640 1 -- 172.21.15.81:0/4265992215 learned_addr learned my addr 172.21.15.81:0/4265992215 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:48.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f234b7fe640 1 -- 172.21.15.81:0/4265992215 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f234c15a680 msgr2=0x7f234c15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:48.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f234b7fe640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f234c15a680 0x7f234c15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f234b7fe640 1 -- 172.21.15.81:0/4265992215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f234c15ea60 msgr2=0x7f234c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:48.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f234b7fe640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f234c15ea60 0x7f234c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.702+0000 7f234b7fe640 1 -- 172.21.15.81:0/4265992215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c0094d0 con 0x7f234c0ade50 2023-11-08T16:32:48.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.703+0000 7f234b7fe640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f234c0ade50 0x7f234c1e5780 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f233c02b9f0 tx=0x7f233c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:48.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.703+0000 7f2333fff640 1 -- 172.21.15.81:0/4265992215 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c030860 con 0x7f234c0ade50 2023-11-08T16:32:48.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.703+0000 7f2333fff640 1 -- 172.21.15.81:0/4265992215 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233c043400 con 0x7f234c0ade50 2023-11-08T16:32:48.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.703+0000 7f2333fff640 1 -- 172.21.15.81:0/4265992215 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233c0437b0 con 0x7f234c0ade50 2023-11-08T16:32:48.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.703+0000 7f2351c61640 1 -- 172.21.15.81:0/4265992215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f234c0cd0e0 con 0x7f234c0ade50 2023-11-08T16:32:48.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.703+0000 7f2351c61640 1 -- 172.21.15.81:0/4265992215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f234c15c4b0 con 0x7f234c0ade50 2023-11-08T16:32:48.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.704+0000 7f2351c61640 1 -- 172.21.15.81:0/4265992215 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f234c0cd600 con 0x7f234c0ade50 2023-11-08T16:32:48.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.705+0000 7f2333fff640 1 -- 172.21.15.81:0/4265992215 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f233c043910 con 0x7f234c0ade50 2023-11-08T16:32:48.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.706+0000 7f2333fff640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 0x7f233806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:48.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.706+0000 7f2333fff640 1 -- 172.21.15.81:0/4265992215 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f233c03e070 con 0x7f234c0ade50 2023-11-08T16:32:48.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.706+0000 7f234affd640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 0x7f233806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:48.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.707+0000 7f234affd640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 0x7f233806d330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2340000f60 tx=0x7f23400079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:48.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.710+0000 7f2333fff640 1 -- 172.21.15.81:0/4265992215 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f234c0cd600 con 0x7f234c0ade50 2023-11-08T16:32:48.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.898+0000 7f2351c61640 1 -- 172.21.15.81:0/4265992215 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f234c0cb170 con 0x7f233806ae70 2023-11-08T16:32:48.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.908+0000 7f2333fff640 1 -- 172.21.15.81:0/4265992215 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286803 (secure 0 0 0) 0x7f2340005080 con 0x7f233806ae70 2023-11-08T16:32:48.913 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:48.915 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:32:48.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.913+0000 7f2351c61640 1 -- 172.21.15.81:0/4265992215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 msgr2=0x7f233806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:48.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.913+0000 7f2351c61640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 0x7f233806d330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2340000f60 tx=0x7f23400079b0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.913+0000 7f2351c61640 1 -- 172.21.15.81:0/4265992215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f234c0ade50 msgr2=0x7f234c1e5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:48.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.913+0000 7f2351c61640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f234c0ade50 0x7f234c1e5780 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f233c02b9f0 tx=0x7f233c004730 comp rx=0 tx=0).stop 2023-11-08T16:32:48.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.913+0000 7f2351c61640 1 -- 172.21.15.81:0/4265992215 shutdown_connections 2023-11-08T16:32:48.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.913+0000 7f2351c61640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 0x7f233806d330 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.913+0000 7f2351c61640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f234c15ea60 0x7f234c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.913+0000 7f2351c61640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f234c15a680 0x7f234c15b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.913+0000 7f2351c61640 1 --2- 172.21.15.81:0/4265992215 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f234c0ade50 0x7f234c1e5780 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:48.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.914+0000 7f2351c61640 1 -- 172.21.15.81:0/4265992215 >> 172.21.15.81:0/4265992215 conn(0x7f234c0cb490 msgr2=0x7f234c0ca280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:48.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.914+0000 7f2351c61640 1 -- 172.21.15.81:0/4265992215 shutdown_connections 2023-11-08T16:32:48.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:48.914+0000 7f2351c61640 1 -- 172.21.15.81:0/4265992215 wait complete. 2023-11-08T16:32:48.935 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2023-11-08T16:32:48.623279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":179,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239376,"kb_used_data":6536,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735344,"statfs":{"total":773094113280,"available":772848992256,"internally_reserved":0,"allocated":6692864,"data_stored":3535484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":41,"apply_latency_ms":41,"commit_latency_ns":41000000,"apply_latency_ns":41000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001467"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.434343+0000","last_change":"2023-11-08T16:32:44.020088+0000","last_active":"2023-11-08T16:32:45.434343+0000","last_peered":"2023-11-08T16:32:45.434343+0000","last_clean":"2023-11-08T16:32:45.434343+0000","last_became_active":"2023-11-08T16:32:44.019811+0000","last_became_peered":"2023-11-08T16:32:44.019811+0000","last_unstale":"2023-11-08T16:32:45.434343+0000","last_undegraded":"2023-11-08T16:32:45.434343+0000","last_fullsized":"2023-11-08T16:32:45.434343+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:20:48.176682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.022228+0000","last_change":"2023-11-08T16:32:44.028870+0000","last_active":"2023-11-08T16:32:45.022228+0000","last_peered":"2023-11-08T16:32:45.022228+0000","last_clean":"2023-11-08T16:32:45.022228+0000","last_became_active":"2023-11-08T16:32:44.028584+0000","last_became_peered":"2023-11-08T16:32:44.028584+0000","last_unstale":"2023-11-08T16:32:45.022228+0000","last_undegraded":"2023-11-08T16:32:45.022228+0000","last_fullsized":"2023-11-08T16:32:45.022228+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:14.142137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503106+0000","last_change":"2023-11-08T16:32:44.030404+0000","last_active":"2023-11-08T16:32:45.503106+0000","last_peered":"2023-11-08T16:32:45.503106+0000","last_clean":"2023-11-08T16:32:45.503106+0000","last_became_active":"2023-11-08T16:32:44.029055+0000","last_became_peered":"2023-11-08T16:32:44.029055+0000","last_unstale":"2023-11-08T16:32:45.503106+0000","last_undegraded":"2023-11-08T16:32:45.503106+0000","last_fullsized":"2023-11-08T16:32:45.503106+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:16:02.938516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.020751+0000","last_change":"2023-11-08T16:32:44.034429+0000","last_active":"2023-11-08T16:32:45.020751+0000","last_peered":"2023-11-08T16:32:45.020751+0000","last_clean":"2023-11-08T16:32:45.020751+0000","last_became_active":"2023-11-08T16:32:44.033989+0000","last_became_peered":"2023-11-08T16:32:44.033989+0000","last_unstale":"2023-11-08T16:32:45.020751+0000","last_undegraded":"2023-11-08T16:32:45.020751+0000","last_fullsized":"2023-11-08T16:32:45.020751+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:28:56.638214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.501687+0000","last_change":"2023-11-08T16:32:44.023830+0000","last_active":"2023-11-08T16:32:45.501687+0000","last_peered":"2023-11-08T16:32:45.501687+0000","last_clean":"2023-11-08T16:32:45.501687+0000","last_became_active":"2023-11-08T16:32:44.023564+0000","last_became_peered":"2023-11-08T16:32:44.023564+0000","last_unstale":"2023-11-08T16:32:45.501687+0000","last_undegraded":"2023-11-08T16:32:45.501687+0000","last_fullsized":"2023-11-08T16:32:45.501687+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:37:21.935022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.433789+0000","last_change":"2023-11-08T16:32:44.028794+0000","last_active":"2023-11-08T16:32:45.433789+0000","last_peered":"2023-11-08T16:32:45.433789+0000","last_clean":"2023-11-08T16:32:45.433789+0000","last_became_active":"2023-11-08T16:32:44.022012+0000","last_became_peered":"2023-11-08T16:32:44.022012+0000","last_unstale":"2023-11-08T16:32:45.433789+0000","last_undegraded":"2023-11-08T16:32:45.433789+0000","last_fullsized":"2023-11-08T16:32:45.433789+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:02:40.083916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502214+0000","last_change":"2023-11-08T16:32:44.029231+0000","last_active":"2023-11-08T16:32:45.502214+0000","last_peered":"2023-11-08T16:32:45.502214+0000","last_clean":"2023-11-08T16:32:45.502214+0000","last_became_active":"2023-11-08T16:32:44.028966+0000","last_became_peered":"2023-11-08T16:32:44.028966+0000","last_unstale":"2023-11-08T16:32:45.502214+0000","last_undegraded":"2023-11-08T16:32:45.502214+0000","last_fullsized":"2023-11-08T16:32:45.502214+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:28:56.638214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.015223+0000","last_change":"2023-11-08T16:32:44.020602+0000","last_active":"2023-11-08T16:32:45.015223+0000","last_peered":"2023-11-08T16:32:45.015223+0000","last_clean":"2023-11-08T16:32:45.015223+0000","last_became_active":"2023-11-08T16:32:44.014326+0000","last_became_peered":"2023-11-08T16:32:44.014326+0000","last_unstale":"2023-11-08T16:32:45.015223+0000","last_undegraded":"2023-11-08T16:32:45.015223+0000","last_fullsized":"2023-11-08T16:32:45.015223+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:05:11.019162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.016473+0000","last_change":"2023-11-08T16:32:44.012892+0000","last_active":"2023-11-08T16:32:45.016473+0000","last_peered":"2023-11-08T16:32:45.016473+0000","last_clean":"2023-11-08T16:32:45.016473+0000","last_became_active":"2023-11-08T16:32:44.012575+0000","last_became_peered":"2023-11-08T16:32:44.012575+0000","last_unstale":"2023-11-08T16:32:45.016473+0000","last_undegraded":"2023-11-08T16:32:45.016473+0000","last_fullsized":"2023-11-08T16:32:45.016473+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:03:12.122524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.433904+0000","last_change":"2023-11-08T16:32:44.019476+0000","last_active":"2023-11-08T16:32:45.433904+0000","last_peered":"2023-11-08T16:32:45.433904+0000","last_clean":"2023-11-08T16:32:45.433904+0000","last_became_active":"2023-11-08T16:32:44.019053+0000","last_became_peered":"2023-11-08T16:32:44.019053+0000","last_unstale":"2023-11-08T16:32:45.433904+0000","last_undegraded":"2023-11-08T16:32:45.433904+0000","last_fullsized":"2023-11-08T16:32:45.433904+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:48.647621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.501658+0000","last_change":"2023-11-08T16:32:44.024381+0000","last_active":"2023-11-08T16:32:45.501658+0000","last_peered":"2023-11-08T16:32:45.501658+0000","last_clean":"2023-11-08T16:32:45.501658+0000","last_became_active":"2023-11-08T16:32:44.024116+0000","last_became_peered":"2023-11-08T16:32:44.024116+0000","last_unstale":"2023-11-08T16:32:45.501658+0000","last_undegraded":"2023-11-08T16:32:45.501658+0000","last_fullsized":"2023-11-08T16:32:45.501658+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:37:14.260374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.016178+0000","last_change":"2023-11-08T16:32:44.010292+0000","last_active":"2023-11-08T16:32:45.016178+0000","last_peered":"2023-11-08T16:32:45.016178+0000","last_clean":"2023-11-08T16:32:45.016178+0000","last_became_active":"2023-11-08T16:32:44.010027+0000","last_became_peered":"2023-11-08T16:32:44.010027+0000","last_unstale":"2023-11-08T16:32:45.016178+0000","last_undegraded":"2023-11-08T16:32:45.016178+0000","last_fullsized":"2023-11-08T16:32:45.016178+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:03:52.814585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.433782+0000","last_change":"2023-11-08T16:32:44.016726+0000","last_active":"2023-11-08T16:32:45.433782+0000","last_peered":"2023-11-08T16:32:45.433782+0000","last_clean":"2023-11-08T16:32:45.433782+0000","last_became_active":"2023-11-08T16:32:44.016483+0000","last_became_peered":"2023-11-08T16:32:44.016483+0000","last_unstale":"2023-11-08T16:32:45.433782+0000","last_undegraded":"2023-11-08T16:32:45.433782+0000","last_fullsized":"2023-11-08T16:32:45.433782+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:25:51.770691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502369+0000","last_change":"2023-11-08T16:32:44.014033+0000","last_active":"2023-11-08T16:32:45.502369+0000","last_peered":"2023-11-08T16:32:45.502369+0000","last_clean":"2023-11-08T16:32:45.502369+0000","last_became_active":"2023-11-08T16:32:44.013718+0000","last_became_peered":"2023-11-08T16:32:44.013718+0000","last_unstale":"2023-11-08T16:32:45.502369+0000","last_undegraded":"2023-11-08T16:32:45.502369+0000","last_fullsized":"2023-11-08T16:32:45.502369+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:05.943932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.020619+0000","last_change":"2023-11-08T16:32:44.024973+0000","last_active":"2023-11-08T16:32:45.020619+0000","last_peered":"2023-11-08T16:32:45.020619+0000","last_clean":"2023-11-08T16:32:45.020619+0000","last_became_active":"2023-11-08T16:32:44.022763+0000","last_became_peered":"2023-11-08T16:32:44.022763+0000","last_unstale":"2023-11-08T16:32:45.020619+0000","last_undegraded":"2023-11-08T16:32:45.020619+0000","last_fullsized":"2023-11-08T16:32:45.020619+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:44:56.291376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.433436+0000","last_change":"2023-11-08T16:32:44.017430+0000","last_active":"2023-11-08T16:32:45.433436+0000","last_peered":"2023-11-08T16:32:45.433436+0000","last_clean":"2023-11-08T16:32:45.433436+0000","last_became_active":"2023-11-08T16:32:44.017168+0000","last_became_peered":"2023-11-08T16:32:44.017168+0000","last_unstale":"2023-11-08T16:32:45.433436+0000","last_undegraded":"2023-11-08T16:32:45.433436+0000","last_fullsized":"2023-11-08T16:32:45.433436+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:24:26.724168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.504159+0000","last_change":"2023-11-08T16:32:44.025971+0000","last_active":"2023-11-08T16:32:45.504159+0000","last_peered":"2023-11-08T16:32:45.504159+0000","last_clean":"2023-11-08T16:32:45.504159+0000","last_became_active":"2023-11-08T16:32:44.017278+0000","last_became_peered":"2023-11-08T16:32:44.017278+0000","last_unstale":"2023-11-08T16:32:45.504159+0000","last_undegraded":"2023-11-08T16:32:45.504159+0000","last_fullsized":"2023-11-08T16:32:45.504159+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:00.761321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.022348+0000","last_change":"2023-11-08T16:32:44.029311+0000","last_active":"2023-11-08T16:32:45.022348+0000","last_peered":"2023-11-08T16:32:45.022348+0000","last_clean":"2023-11-08T16:32:45.022348+0000","last_became_active":"2023-11-08T16:32:44.029024+0000","last_became_peered":"2023-11-08T16:32:44.029024+0000","last_unstale":"2023-11-08T16:32:45.022348+0000","last_undegraded":"2023-11-08T16:32:45.022348+0000","last_fullsized":"2023-11-08T16:32:45.022348+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:16:02.938516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503203+0000","last_change":"2023-11-08T16:32:44.018485+0000","last_active":"2023-11-08T16:32:45.503203+0000","last_peered":"2023-11-08T16:32:45.503203+0000","last_clean":"2023-11-08T16:32:45.503203+0000","last_became_active":"2023-11-08T16:32:44.018064+0000","last_became_peered":"2023-11-08T16:32:44.018064+0000","last_unstale":"2023-11-08T16:32:45.503203+0000","last_undegraded":"2023-11-08T16:32:45.503203+0000","last_fullsized":"2023-11-08T16:32:45.503203+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:57:58.519538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503512+0000","last_change":"2023-11-08T16:32:44.017756+0000","last_active":"2023-11-08T16:32:45.503512+0000","last_peered":"2023-11-08T16:32:45.503512+0000","last_clean":"2023-11-08T16:32:45.503512+0000","last_became_active":"2023-11-08T16:32:44.016100+0000","last_became_peered":"2023-11-08T16:32:44.016100+0000","last_unstale":"2023-11-08T16:32:45.503512+0000","last_undegraded":"2023-11-08T16:32:45.503512+0000","last_fullsized":"2023-11-08T16:32:45.503512+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:55:15.779428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503516+0000","last_change":"2023-11-08T16:32:44.024081+0000","last_active":"2023-11-08T16:32:45.503516+0000","last_peered":"2023-11-08T16:32:45.503516+0000","last_clean":"2023-11-08T16:32:45.503516+0000","last_became_active":"2023-11-08T16:32:44.023799+0000","last_became_peered":"2023-11-08T16:32:44.023799+0000","last_unstale":"2023-11-08T16:32:45.503516+0000","last_undegraded":"2023-11-08T16:32:45.503516+0000","last_fullsized":"2023-11-08T16:32:45.503516+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:16:20.382141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.015445+0000","last_change":"2023-11-08T16:32:44.011677+0000","last_active":"2023-11-08T16:32:45.015445+0000","last_peered":"2023-11-08T16:32:45.015445+0000","last_clean":"2023-11-08T16:32:45.015445+0000","last_became_active":"2023-11-08T16:32:44.011426+0000","last_became_peered":"2023-11-08T16:32:44.011426+0000","last_unstale":"2023-11-08T16:32:45.015445+0000","last_undegraded":"2023-11-08T16:32:45.015445+0000","last_fullsized":"2023-11-08T16:32:45.015445+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:17:00.761321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.500806+0000","last_change":"2023-11-08T16:32:44.019340+0000","last_active":"2023-11-08T16:32:45.500806+0000","last_peered":"2023-11-08T16:32:45.500806+0000","last_clean":"2023-11-08T16:32:45.500806+0000","last_became_active":"2023-11-08T16:32:44.019067+0000","last_became_peered":"2023-11-08T16:32:44.019067+0000","last_unstale":"2023-11-08T16:32:45.500806+0000","last_undegraded":"2023-11-08T16:32:45.500806+0000","last_fullsized":"2023-11-08T16:32:45.500806+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:40.409632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.501600+0000","last_change":"2023-11-08T16:32:44.015252+0000","last_active":"2023-11-08T16:32:45.501600+0000","last_peered":"2023-11-08T16:32:45.501600+0000","last_clean":"2023-11-08T16:32:45.501600+0000","last_became_active":"2023-11-08T16:32:44.014990+0000","last_became_peered":"2023-11-08T16:32:44.014990+0000","last_unstale":"2023-11-08T16:32:45.501600+0000","last_undegraded":"2023-11-08T16:32:45.501600+0000","last_fullsized":"2023-11-08T16:32:45.501600+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:44:56.291376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.504322+0000","last_change":"2023-11-08T16:32:44.031270+0000","last_active":"2023-11-08T16:32:45.504322+0000","last_peered":"2023-11-08T16:32:45.504322+0000","last_clean":"2023-11-08T16:32:45.504322+0000","last_became_active":"2023-11-08T16:32:44.030030+0000","last_became_peered":"2023-11-08T16:32:44.030030+0000","last_unstale":"2023-11-08T16:32:45.504322+0000","last_undegraded":"2023-11-08T16:32:45.504322+0000","last_fullsized":"2023-11-08T16:32:45.504322+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:14:06.356903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.015877+0000","last_change":"2023-11-08T16:32:44.013642+0000","last_active":"2023-11-08T16:32:45.015877+0000","last_peered":"2023-11-08T16:32:45.015877+0000","last_clean":"2023-11-08T16:32:45.015877+0000","last_became_active":"2023-11-08T16:32:44.013303+0000","last_became_peered":"2023-11-08T16:32:44.013303+0000","last_unstale":"2023-11-08T16:32:45.015877+0000","last_undegraded":"2023-11-08T16:32:45.015877+0000","last_fullsized":"2023-11-08T16:32:45.015877+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:14:06.356903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.434380+0000","last_change":"2023-11-08T16:32:44.015348+0000","last_active":"2023-11-08T16:32:45.434380+0000","last_peered":"2023-11-08T16:32:45.434380+0000","last_clean":"2023-11-08T16:32:45.434380+0000","last_became_active":"2023-11-08T16:32:44.014992+0000","last_became_peered":"2023-11-08T16:32:44.014992+0000","last_unstale":"2023-11-08T16:32:45.434380+0000","last_undegraded":"2023-11-08T16:32:45.434380+0000","last_fullsized":"2023-11-08T16:32:45.434380+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:35:45.917388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.504031+0000","last_change":"2023-11-08T16:32:44.010230+0000","last_active":"2023-11-08T16:32:45.504031+0000","last_peered":"2023-11-08T16:32:45.504031+0000","last_clean":"2023-11-08T16:32:45.504031+0000","last_became_active":"2023-11-08T16:32:44.009814+0000","last_became_peered":"2023-11-08T16:32:44.009814+0000","last_unstale":"2023-11-08T16:32:45.504031+0000","last_undegraded":"2023-11-08T16:32:45.504031+0000","last_fullsized":"2023-11-08T16:32:45.504031+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:21:51.048285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502475+0000","last_change":"2023-11-08T16:32:44.026657+0000","last_active":"2023-11-08T16:32:45.502475+0000","last_peered":"2023-11-08T16:32:45.502475+0000","last_clean":"2023-11-08T16:32:45.502475+0000","last_became_active":"2023-11-08T16:32:44.026387+0000","last_became_peered":"2023-11-08T16:32:44.026387+0000","last_unstale":"2023-11-08T16:32:45.502475+0000","last_undegraded":"2023-11-08T16:32:45.502475+0000","last_fullsized":"2023-11-08T16:32:45.502475+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:49:29.369557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.022076+0000","last_change":"2023-11-08T16:32:44.032673+0000","last_active":"2023-11-08T16:32:45.022076+0000","last_peered":"2023-11-08T16:32:45.022076+0000","last_clean":"2023-11-08T16:32:45.022076+0000","last_became_active":"2023-11-08T16:32:44.032396+0000","last_became_peered":"2023-11-08T16:32:44.032396+0000","last_unstale":"2023-11-08T16:32:45.022076+0000","last_undegraded":"2023-11-08T16:32:45.022076+0000","last_fullsized":"2023-11-08T16:32:45.022076+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:49:29.369557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502451+0000","last_change":"2023-11-08T16:32:44.015431+0000","last_active":"2023-11-08T16:32:45.502451+0000","last_peered":"2023-11-08T16:32:45.502451+0000","last_clean":"2023-11-08T16:32:45.502451+0000","last_became_active":"2023-11-08T16:32:44.013684+0000","last_became_peered":"2023-11-08T16:32:44.013684+0000","last_unstale":"2023-11-08T16:32:45.502451+0000","last_undegraded":"2023-11-08T16:32:45.502451+0000","last_fullsized":"2023-11-08T16:32:45.502451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:53:30.126707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.020576+0000","last_change":"2023-11-08T16:32:44.013134+0000","last_active":"2023-11-08T16:32:45.020576+0000","last_peered":"2023-11-08T16:32:45.020576+0000","last_clean":"2023-11-08T16:32:45.020576+0000","last_became_active":"2023-11-08T16:32:44.012777+0000","last_became_peered":"2023-11-08T16:32:44.012777+0000","last_unstale":"2023-11-08T16:32:45.020576+0000","last_undegraded":"2023-11-08T16:32:45.020576+0000","last_fullsized":"2023-11-08T16:32:45.020576+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:02:40.083916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.020035+0000","last_change":"2023-11-08T16:32:44.024535+0000","last_active":"2023-11-08T16:32:45.020035+0000","last_peered":"2023-11-08T16:32:45.020035+0000","last_clean":"2023-11-08T16:32:45.020035+0000","last_became_active":"2023-11-08T16:32:44.024261+0000","last_became_peered":"2023-11-08T16:32:44.024261+0000","last_unstale":"2023-11-08T16:32:45.020035+0000","last_undegraded":"2023-11-08T16:32:45.020035+0000","last_fullsized":"2023-11-08T16:32:45.020035+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:35:45.917388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"56'4","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503923+0000","last_change":"2023-11-08T16:32:43.013726+0000","last_active":"2023-11-08T16:32:45.503923+0000","last_peered":"2023-11-08T16:32:45.503923+0000","last_clean":"2023-11-08T16:32:45.503923+0000","last_became_active":"2023-11-08T16:32:43.013433+0000","last_became_peered":"2023-11-08T16:32:43.013433+0000","last_unstale":"2023-11-08T16:32:45.503923+0000","last_undegraded":"2023-11-08T16:32:45.503923+0000","last_fullsized":"2023-11-08T16:32:45.503923+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:13:38.156343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.021081+0000","last_change":"2023-11-08T16:32:44.023859+0000","last_active":"2023-11-08T16:32:45.021081+0000","last_peered":"2023-11-08T16:32:45.021081+0000","last_clean":"2023-11-08T16:32:45.021081+0000","last_became_active":"2023-11-08T16:32:44.023596+0000","last_became_peered":"2023-11-08T16:32:44.023596+0000","last_unstale":"2023-11-08T16:32:45.021081+0000","last_undegraded":"2023-11-08T16:32:45.021081+0000","last_fullsized":"2023-11-08T16:32:45.021081+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:14:21.835500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.434119+0000","last_change":"2023-11-08T16:32:43.000526+0000","last_active":"2023-11-08T16:32:45.434119+0000","last_peered":"2023-11-08T16:32:45.434119+0000","last_clean":"2023-11-08T16:32:45.434119+0000","last_became_active":"2023-11-08T16:32:43.000147+0000","last_became_peered":"2023-11-08T16:32:43.000147+0000","last_unstale":"2023-11-08T16:32:45.434119+0000","last_undegraded":"2023-11-08T16:32:45.434119+0000","last_fullsized":"2023-11-08T16:32:45.434119+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:14:18.365850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502875+0000","last_change":"2023-11-08T16:32:44.024339+0000","last_active":"2023-11-08T16:32:45.502875+0000","last_peered":"2023-11-08T16:32:45.502875+0000","last_clean":"2023-11-08T16:32:45.502875+0000","last_became_active":"2023-11-08T16:32:44.024031+0000","last_became_peered":"2023-11-08T16:32:44.024031+0000","last_unstale":"2023-11-08T16:32:45.502875+0000","last_undegraded":"2023-11-08T16:32:45.502875+0000","last_fullsized":"2023-11-08T16:32:45.502875+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:35:51.165824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.434404+0000","last_change":"2023-11-08T16:32:43.014630+0000","last_active":"2023-11-08T16:32:45.434404+0000","last_peered":"2023-11-08T16:32:45.434404+0000","last_clean":"2023-11-08T16:32:45.434404+0000","last_became_active":"2023-11-08T16:32:43.014266+0000","last_became_peered":"2023-11-08T16:32:43.014266+0000","last_unstale":"2023-11-08T16:32:45.434404+0000","last_undegraded":"2023-11-08T16:32:45.434404+0000","last_fullsized":"2023-11-08T16:32:45.434404+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:03.691049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.015248+0000","last_change":"2023-11-08T16:32:44.007223+0000","last_active":"2023-11-08T16:32:45.015248+0000","last_peered":"2023-11-08T16:32:45.015248+0000","last_clean":"2023-11-08T16:32:45.015248+0000","last_became_active":"2023-11-08T16:32:44.006957+0000","last_became_peered":"2023-11-08T16:32:44.006957+0000","last_unstale":"2023-11-08T16:32:45.015248+0000","last_undegraded":"2023-11-08T16:32:45.015248+0000","last_fullsized":"2023-11-08T16:32:45.015248+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:40:04.861154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.022568+0000","last_change":"2023-11-08T16:32:43.011444+0000","last_active":"2023-11-08T16:32:45.022568+0000","last_peered":"2023-11-08T16:32:45.022568+0000","last_clean":"2023-11-08T16:32:45.022568+0000","last_became_active":"2023-11-08T16:32:43.011114+0000","last_became_peered":"2023-11-08T16:32:43.011114+0000","last_unstale":"2023-11-08T16:32:45.022568+0000","last_undegraded":"2023-11-08T16:32:45.022568+0000","last_fullsized":"2023-11-08T16:32:45.022568+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:26:59.706891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.501902+0000","last_change":"2023-11-08T16:32:44.023209+0000","last_active":"2023-11-08T16:32:45.501902+0000","last_peered":"2023-11-08T16:32:45.501902+0000","last_clean":"2023-11-08T16:32:45.501902+0000","last_became_active":"2023-11-08T16:32:44.022620+0000","last_became_peered":"2023-11-08T16:32:44.022620+0000","last_unstale":"2023-11-08T16:32:45.501902+0000","last_undegraded":"2023-11-08T16:32:45.501902+0000","last_fullsized":"2023-11-08T16:32:45.501902+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:29:04.702235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502473+0000","last_change":"2023-11-08T16:32:43.012056+0000","last_active":"2023-11-08T16:32:45.502473+0000","last_peered":"2023-11-08T16:32:45.502473+0000","last_clean":"2023-11-08T16:32:45.502473+0000","last_became_active":"2023-11-08T16:32:43.011701+0000","last_became_peered":"2023-11-08T16:32:43.011701+0000","last_unstale":"2023-11-08T16:32:45.502473+0000","last_undegraded":"2023-11-08T16:32:45.502473+0000","last_fullsized":"2023-11-08T16:32:45.502473+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:24:25.712982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.021451+0000","last_change":"2023-11-08T16:32:44.032690+0000","last_active":"2023-11-08T16:32:45.021451+0000","last_peered":"2023-11-08T16:32:45.021451+0000","last_clean":"2023-11-08T16:32:45.021451+0000","last_became_active":"2023-11-08T16:32:44.032322+0000","last_became_peered":"2023-11-08T16:32:44.032322+0000","last_unstale":"2023-11-08T16:32:45.021451+0000","last_undegraded":"2023-11-08T16:32:45.021451+0000","last_fullsized":"2023-11-08T16:32:45.021451+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:05.943932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.501281+0000","last_change":"2023-11-08T16:32:43.014975+0000","last_active":"2023-11-08T16:32:45.501281+0000","last_peered":"2023-11-08T16:32:45.501281+0000","last_clean":"2023-11-08T16:32:45.501281+0000","last_became_active":"2023-11-08T16:32:43.014525+0000","last_became_peered":"2023-11-08T16:32:43.014525+0000","last_unstale":"2023-11-08T16:32:45.501281+0000","last_undegraded":"2023-11-08T16:32:45.501281+0000","last_fullsized":"2023-11-08T16:32:45.501281+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:43:24.480528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.016135+0000","last_change":"2023-11-08T16:32:44.005167+0000","last_active":"2023-11-08T16:32:45.016135+0000","last_peered":"2023-11-08T16:32:45.016135+0000","last_clean":"2023-11-08T16:32:45.016135+0000","last_became_active":"2023-11-08T16:32:44.004888+0000","last_became_peered":"2023-11-08T16:32:44.004888+0000","last_unstale":"2023-11-08T16:32:45.016135+0000","last_undegraded":"2023-11-08T16:32:45.016135+0000","last_fullsized":"2023-11-08T16:32:45.016135+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:21:14.493624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502536+0000","last_change":"2023-11-08T16:32:43.013365+0000","last_active":"2023-11-08T16:32:45.502536+0000","last_peered":"2023-11-08T16:32:45.502536+0000","last_clean":"2023-11-08T16:32:45.502536+0000","last_became_active":"2023-11-08T16:32:43.012899+0000","last_became_peered":"2023-11-08T16:32:43.012899+0000","last_unstale":"2023-11-08T16:32:45.502536+0000","last_undegraded":"2023-11-08T16:32:45.502536+0000","last_fullsized":"2023-11-08T16:32:45.502536+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:34:07.002976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.021573+0000","last_change":"2023-11-08T16:32:44.030835+0000","last_active":"2023-11-08T16:32:45.021573+0000","last_peered":"2023-11-08T16:32:45.021573+0000","last_clean":"2023-11-08T16:32:45.021573+0000","last_became_active":"2023-11-08T16:32:44.030524+0000","last_became_peered":"2023-11-08T16:32:44.030524+0000","last_unstale":"2023-11-08T16:32:45.021573+0000","last_undegraded":"2023-11-08T16:32:45.021573+0000","last_fullsized":"2023-11-08T16:32:45.021573+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:53:30.126707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.501672+0000","last_change":"2023-11-08T16:32:43.016667+0000","last_active":"2023-11-08T16:32:45.501672+0000","last_peered":"2023-11-08T16:32:45.501672+0000","last_clean":"2023-11-08T16:32:45.501672+0000","last_became_active":"2023-11-08T16:32:43.016324+0000","last_became_peered":"2023-11-08T16:32:43.016324+0000","last_unstale":"2023-11-08T16:32:45.501672+0000","last_undegraded":"2023-11-08T16:32:45.501672+0000","last_fullsized":"2023-11-08T16:32:45.501672+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:02:39.072731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.501340+0000","last_change":"2023-11-08T16:32:44.026890+0000","last_active":"2023-11-08T16:32:45.501340+0000","last_peered":"2023-11-08T16:32:45.501340+0000","last_clean":"2023-11-08T16:32:45.501340+0000","last_became_active":"2023-11-08T16:32:44.026574+0000","last_became_peered":"2023-11-08T16:32:44.026574+0000","last_unstale":"2023-11-08T16:32:45.501340+0000","last_undegraded":"2023-11-08T16:32:45.501340+0000","last_fullsized":"2023-11-08T16:32:45.501340+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:22:32.594226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.021811+0000","last_change":"2023-11-08T16:32:43.011542+0000","last_active":"2023-11-08T16:32:45.021811+0000","last_peered":"2023-11-08T16:32:45.021811+0000","last_clean":"2023-11-08T16:32:45.021811+0000","last_became_active":"2023-11-08T16:32:43.011246+0000","last_became_peered":"2023-11-08T16:32:43.011246+0000","last_unstale":"2023-11-08T16:32:45.021811+0000","last_undegraded":"2023-11-08T16:32:45.021811+0000","last_fullsized":"2023-11-08T16:32:45.021811+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:02:39.072731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.020752+0000","last_change":"2023-11-08T16:32:44.015210+0000","last_active":"2023-11-08T16:32:45.020752+0000","last_peered":"2023-11-08T16:32:45.020752+0000","last_clean":"2023-11-08T16:32:45.020752+0000","last_became_active":"2023-11-08T16:32:44.014872+0000","last_became_peered":"2023-11-08T16:32:44.014872+0000","last_unstale":"2023-11-08T16:32:45.020752+0000","last_undegraded":"2023-11-08T16:32:45.020752+0000","last_fullsized":"2023-11-08T16:32:45.020752+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:51.976792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502542+0000","last_change":"2023-11-08T16:32:43.020833+0000","last_active":"2023-11-08T16:32:45.502542+0000","last_peered":"2023-11-08T16:32:45.502542+0000","last_clean":"2023-11-08T16:32:45.502542+0000","last_became_active":"2023-11-08T16:32:43.020401+0000","last_became_peered":"2023-11-08T16:32:43.020401+0000","last_unstale":"2023-11-08T16:32:45.502542+0000","last_undegraded":"2023-11-08T16:32:45.502542+0000","last_fullsized":"2023-11-08T16:32:45.502542+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:44:31.698592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502568+0000","last_change":"2023-11-08T16:32:44.021142+0000","last_active":"2023-11-08T16:32:45.502568+0000","last_peered":"2023-11-08T16:32:45.502568+0000","last_clean":"2023-11-08T16:32:45.502568+0000","last_became_active":"2023-11-08T16:32:44.020868+0000","last_became_peered":"2023-11-08T16:32:44.020868+0000","last_unstale":"2023-11-08T16:32:45.502568+0000","last_undegraded":"2023-11-08T16:32:45.502568+0000","last_fullsized":"2023-11-08T16:32:45.502568+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:14.142137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"56'3","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.021784+0000","last_change":"2023-11-08T16:32:43.010767+0000","last_active":"2023-11-08T16:32:45.021784+0000","last_peered":"2023-11-08T16:32:45.021784+0000","last_clean":"2023-11-08T16:32:45.021784+0000","last_became_active":"2023-11-08T16:32:43.009853+0000","last_became_peered":"2023-11-08T16:32:43.009853+0000","last_unstale":"2023-11-08T16:32:45.021784+0000","last_undegraded":"2023-11-08T16:32:45.021784+0000","last_fullsized":"2023-11-08T16:32:45.021784+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:47.636435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.501721+0000","last_change":"2023-11-08T16:32:44.025841+0000","last_active":"2023-11-08T16:32:45.501721+0000","last_peered":"2023-11-08T16:32:45.501721+0000","last_clean":"2023-11-08T16:32:45.501721+0000","last_became_active":"2023-11-08T16:32:44.025526+0000","last_became_peered":"2023-11-08T16:32:44.025526+0000","last_unstale":"2023-11-08T16:32:45.501721+0000","last_undegraded":"2023-11-08T16:32:45.501721+0000","last_fullsized":"2023-11-08T16:32:45.501721+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:29.425322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"56'6","reported_seq":18,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.016006+0000","last_change":"2023-11-08T16:32:43.004600+0000","last_active":"2023-11-08T16:32:45.016006+0000","last_peered":"2023-11-08T16:32:45.016006+0000","last_clean":"2023-11-08T16:32:45.016006+0000","last_became_active":"2023-11-08T16:32:43.004291+0000","last_became_peered":"2023-11-08T16:32:43.004291+0000","last_unstale":"2023-11-08T16:32:45.016006+0000","last_undegraded":"2023-11-08T16:32:45.016006+0000","last_fullsized":"2023-11-08T16:32:45.016006+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:05.772873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.022959+0000","last_change":"2023-11-08T16:32:44.025077+0000","last_active":"2023-11-08T16:32:45.022959+0000","last_peered":"2023-11-08T16:32:45.022959+0000","last_clean":"2023-11-08T16:32:45.022959+0000","last_became_active":"2023-11-08T16:32:44.024804+0000","last_became_peered":"2023-11-08T16:32:44.024804+0000","last_unstale":"2023-11-08T16:32:45.022959+0000","last_undegraded":"2023-11-08T16:32:45.022959+0000","last_fullsized":"2023-11-08T16:32:45.022959+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:44:56.291376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502704+0000","last_change":"2023-11-08T16:32:43.013381+0000","last_active":"2023-11-08T16:32:45.502704+0000","last_peered":"2023-11-08T16:32:45.502704+0000","last_clean":"2023-11-08T16:32:45.502704+0000","last_became_active":"2023-11-08T16:32:43.012073+0000","last_became_peered":"2023-11-08T16:32:43.012073+0000","last_unstale":"2023-11-08T16:32:45.502704+0000","last_undegraded":"2023-11-08T16:32:45.502704+0000","last_fullsized":"2023-11-08T16:32:45.502704+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:42.781714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503407+0000","last_change":"2023-11-08T16:32:44.018315+0000","last_active":"2023-11-08T16:32:45.503407+0000","last_peered":"2023-11-08T16:32:45.503407+0000","last_clean":"2023-11-08T16:32:45.503407+0000","last_became_active":"2023-11-08T16:32:44.017703+0000","last_became_peered":"2023-11-08T16:32:44.017703+0000","last_unstale":"2023-11-08T16:32:45.503407+0000","last_undegraded":"2023-11-08T16:32:45.503407+0000","last_fullsized":"2023-11-08T16:32:45.503407+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:40:04.861154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503280+0000","last_change":"2023-11-08T16:32:43.020883+0000","last_active":"2023-11-08T16:32:45.503280+0000","last_peered":"2023-11-08T16:32:45.503280+0000","last_clean":"2023-11-08T16:32:45.503280+0000","last_became_active":"2023-11-08T16:32:43.020475+0000","last_became_peered":"2023-11-08T16:32:43.020475+0000","last_unstale":"2023-11-08T16:32:45.503280+0000","last_undegraded":"2023-11-08T16:32:45.503280+0000","last_fullsized":"2023-11-08T16:32:45.503280+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:44:23.707997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.016007+0000","last_change":"2023-11-08T16:32:44.023705+0000","last_active":"2023-11-08T16:32:45.016007+0000","last_peered":"2023-11-08T16:32:45.016007+0000","last_clean":"2023-11-08T16:32:45.016007+0000","last_became_active":"2023-11-08T16:32:44.023442+0000","last_became_peered":"2023-11-08T16:32:44.023442+0000","last_unstale":"2023-11-08T16:32:45.016007+0000","last_undegraded":"2023-11-08T16:32:45.016007+0000","last_fullsized":"2023-11-08T16:32:45.016007+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:06:50.387928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.020840+0000","last_change":"2023-11-08T16:32:43.014810+0000","last_active":"2023-11-08T16:32:45.020840+0000","last_peered":"2023-11-08T16:32:45.020840+0000","last_clean":"2023-11-08T16:32:45.020840+0000","last_became_active":"2023-11-08T16:32:43.014522+0000","last_became_peered":"2023-11-08T16:32:43.014522+0000","last_unstale":"2023-11-08T16:32:45.020840+0000","last_undegraded":"2023-11-08T16:32:45.020840+0000","last_fullsized":"2023-11-08T16:32:45.020840+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:14:18.365850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.504181+0000","last_change":"2023-11-08T16:32:44.019444+0000","last_active":"2023-11-08T16:32:45.504181+0000","last_peered":"2023-11-08T16:32:45.504181+0000","last_clean":"2023-11-08T16:32:45.504181+0000","last_became_active":"2023-11-08T16:32:44.017651+0000","last_became_peered":"2023-11-08T16:32:44.017651+0000","last_unstale":"2023-11-08T16:32:45.504181+0000","last_undegraded":"2023-11-08T16:32:45.504181+0000","last_fullsized":"2023-11-08T16:32:45.504181+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:16:35.526220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"56'8","reported_seq":20,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.015323+0000","last_change":"2023-11-08T16:32:43.003941+0000","last_active":"2023-11-08T16:32:45.015323+0000","last_peered":"2023-11-08T16:32:45.015323+0000","last_clean":"2023-11-08T16:32:45.015323+0000","last_became_active":"2023-11-08T16:32:43.003615+0000","last_became_peered":"2023-11-08T16:32:43.003615+0000","last_unstale":"2023-11-08T16:32:45.015323+0000","last_undegraded":"2023-11-08T16:32:45.015323+0000","last_fullsized":"2023-11-08T16:32:45.015323+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:50.965606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.015420+0000","last_change":"2023-11-08T16:32:44.021381+0000","last_active":"2023-11-08T16:32:45.015420+0000","last_peered":"2023-11-08T16:32:45.015420+0000","last_clean":"2023-11-08T16:32:45.015420+0000","last_became_active":"2023-11-08T16:32:44.021108+0000","last_became_peered":"2023-11-08T16:32:44.021108+0000","last_unstale":"2023-11-08T16:32:45.015420+0000","last_undegraded":"2023-11-08T16:32:45.015420+0000","last_fullsized":"2023-11-08T16:32:45.015420+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:41.043633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.504419+0000","last_change":"2023-11-08T16:32:43.022303+0000","last_active":"2023-11-08T16:32:45.504419+0000","last_peered":"2023-11-08T16:32:45.504419+0000","last_clean":"2023-11-08T16:32:45.504419+0000","last_became_active":"2023-11-08T16:32:43.021988+0000","last_became_peered":"2023-11-08T16:32:43.021988+0000","last_unstale":"2023-11-08T16:32:45.504419+0000","last_undegraded":"2023-11-08T16:32:45.504419+0000","last_fullsized":"2023-11-08T16:32:45.504419+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:35:44.906202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.433444+0000","last_change":"2023-11-08T16:32:44.015308+0000","last_active":"2023-11-08T16:32:45.433444+0000","last_peered":"2023-11-08T16:32:45.433444+0000","last_clean":"2023-11-08T16:32:45.433444+0000","last_became_active":"2023-11-08T16:32:44.015025+0000","last_became_peered":"2023-11-08T16:32:44.015025+0000","last_unstale":"2023-11-08T16:32:45.433444+0000","last_undegraded":"2023-11-08T16:32:45.433444+0000","last_fullsized":"2023-11-08T16:32:45.433444+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:43.792899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.501698+0000","last_change":"2023-11-08T16:32:43.012452+0000","last_active":"2023-11-08T16:32:45.501698+0000","last_peered":"2023-11-08T16:32:45.501698+0000","last_clean":"2023-11-08T16:32:45.501698+0000","last_became_active":"2023-11-08T16:32:43.012132+0000","last_became_peered":"2023-11-08T16:32:43.012132+0000","last_unstale":"2023-11-08T16:32:45.501698+0000","last_undegraded":"2023-11-08T16:32:45.501698+0000","last_fullsized":"2023-11-08T16:32:45.501698+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:17:03.100846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.501293+0000","last_change":"2023-11-08T16:32:44.025320+0000","last_active":"2023-11-08T16:32:45.501293+0000","last_peered":"2023-11-08T16:32:45.501293+0000","last_clean":"2023-11-08T16:32:45.501293+0000","last_became_active":"2023-11-08T16:32:44.024927+0000","last_became_peered":"2023-11-08T16:32:44.024927+0000","last_unstale":"2023-11-08T16:32:45.501293+0000","last_undegraded":"2023-11-08T16:32:45.501293+0000","last_fullsized":"2023-11-08T16:32:45.501293+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:38:08.280908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"56'1","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.504001+0000","last_change":"2023-11-08T16:32:43.015641+0000","last_active":"2023-11-08T16:32:45.504001+0000","last_peered":"2023-11-08T16:32:45.504001+0000","last_clean":"2023-11-08T16:32:45.504001+0000","last_became_active":"2023-11-08T16:32:43.015376+0000","last_became_peered":"2023-11-08T16:32:43.015376+0000","last_unstale":"2023-11-08T16:32:45.504001+0000","last_undegraded":"2023-11-08T16:32:45.504001+0000","last_fullsized":"2023-11-08T16:32:45.504001+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:42:01.982997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.433767+0000","last_change":"2023-11-08T16:32:44.016244+0000","last_active":"2023-11-08T16:32:45.433767+0000","last_peered":"2023-11-08T16:32:45.433767+0000","last_clean":"2023-11-08T16:32:45.433767+0000","last_became_active":"2023-11-08T16:32:44.015905+0000","last_became_peered":"2023-11-08T16:32:44.015905+0000","last_unstale":"2023-11-08T16:32:45.433767+0000","last_undegraded":"2023-11-08T16:32:45.433767+0000","last_fullsized":"2023-11-08T16:32:45.433767+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:29:21.693551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503810+0000","last_change":"2023-11-08T16:32:43.016922+0000","last_active":"2023-11-08T16:32:45.503810+0000","last_peered":"2023-11-08T16:32:45.503810+0000","last_clean":"2023-11-08T16:32:45.503810+0000","last_became_active":"2023-11-08T16:32:43.016719+0000","last_became_peered":"2023-11-08T16:32:43.016719+0000","last_unstale":"2023-11-08T16:32:45.503810+0000","last_undegraded":"2023-11-08T16:32:45.503810+0000","last_fullsized":"2023-11-08T16:32:45.503810+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:41:15.791764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.022295+0000","last_change":"2023-11-08T16:32:44.014133+0000","last_active":"2023-11-08T16:32:45.022295+0000","last_peered":"2023-11-08T16:32:45.022295+0000","last_clean":"2023-11-08T16:32:45.022295+0000","last_became_active":"2023-11-08T16:32:44.013837+0000","last_became_peered":"2023-11-08T16:32:44.013837+0000","last_unstale":"2023-11-08T16:32:45.022295+0000","last_undegraded":"2023-11-08T16:32:45.022295+0000","last_fullsized":"2023-11-08T16:32:45.022295+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:20:48.176682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.433983+0000","last_change":"2023-11-08T16:32:43.001503+0000","last_active":"2023-11-08T16:32:45.433983+0000","last_peered":"2023-11-08T16:32:45.433983+0000","last_clean":"2023-11-08T16:32:45.433983+0000","last_became_active":"2023-11-08T16:32:43.001236+0000","last_became_peered":"2023-11-08T16:32:43.001236+0000","last_unstale":"2023-11-08T16:32:45.433983+0000","last_undegraded":"2023-11-08T16:32:45.433983+0000","last_fullsized":"2023-11-08T16:32:45.433983+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:22:28.414136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503896+0000","last_change":"2023-11-08T16:32:44.017095+0000","last_active":"2023-11-08T16:32:45.503896+0000","last_peered":"2023-11-08T16:32:45.503896+0000","last_clean":"2023-11-08T16:32:45.503896+0000","last_became_active":"2023-11-08T16:32:44.016808+0000","last_became_peered":"2023-11-08T16:32:44.016808+0000","last_unstale":"2023-11-08T16:32:45.503896+0000","last_undegraded":"2023-11-08T16:32:45.503896+0000","last_fullsized":"2023-11-08T16:32:45.503896+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:42:27.345040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.015264+0000","last_change":"2023-11-08T16:32:43.003941+0000","last_active":"2023-11-08T16:32:45.015264+0000","last_peered":"2023-11-08T16:32:45.015264+0000","last_clean":"2023-11-08T16:32:45.015264+0000","last_became_active":"2023-11-08T16:32:43.003632+0000","last_became_peered":"2023-11-08T16:32:43.003632+0000","last_unstale":"2023-11-08T16:32:45.015264+0000","last_undegraded":"2023-11-08T16:32:45.015264+0000","last_fullsized":"2023-11-08T16:32:45.015264+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:16:01.927330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503849+0000","last_change":"2023-11-08T16:32:44.019304+0000","last_active":"2023-11-08T16:32:45.503849+0000","last_peered":"2023-11-08T16:32:45.503849+0000","last_clean":"2023-11-08T16:32:45.503849+0000","last_became_active":"2023-11-08T16:32:44.019050+0000","last_became_peered":"2023-11-08T16:32:44.019050+0000","last_unstale":"2023-11-08T16:32:45.503849+0000","last_undegraded":"2023-11-08T16:32:45.503849+0000","last_fullsized":"2023-11-08T16:32:45.503849+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:41.043633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502471+0000","last_change":"2023-11-08T16:32:43.010129+0000","last_active":"2023-11-08T16:32:45.502471+0000","last_peered":"2023-11-08T16:32:45.502471+0000","last_clean":"2023-11-08T16:32:45.502471+0000","last_became_active":"2023-11-08T16:32:43.009826+0000","last_became_peered":"2023-11-08T16:32:43.009826+0000","last_unstale":"2023-11-08T16:32:45.502471+0000","last_undegraded":"2023-11-08T16:32:45.502471+0000","last_fullsized":"2023-11-08T16:32:45.502471+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:47.636435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.434252+0000","last_change":"2023-11-08T16:32:44.025430+0000","last_active":"2023-11-08T16:32:45.434252+0000","last_peered":"2023-11-08T16:32:45.434252+0000","last_clean":"2023-11-08T16:32:45.434252+0000","last_became_active":"2023-11-08T16:32:44.025165+0000","last_became_peered":"2023-11-08T16:32:44.025165+0000","last_unstale":"2023-11-08T16:32:45.434252+0000","last_undegraded":"2023-11-08T16:32:45.434252+0000","last_fullsized":"2023-11-08T16:32:45.434252+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:14:21.835500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.500632+0000","last_change":"2023-11-08T16:32:43.011713+0000","last_active":"2023-11-08T16:32:45.500632+0000","last_peered":"2023-11-08T16:32:45.500632+0000","last_clean":"2023-11-08T16:32:45.500632+0000","last_became_active":"2023-11-08T16:32:43.011376+0000","last_became_peered":"2023-11-08T16:32:43.011376+0000","last_unstale":"2023-11-08T16:32:45.500632+0000","last_undegraded":"2023-11-08T16:32:45.500632+0000","last_fullsized":"2023-11-08T16:32:45.500632+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:39:14.064442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.022688+0000","last_change":"2023-11-08T16:32:44.021797+0000","last_active":"2023-11-08T16:32:45.022688+0000","last_peered":"2023-11-08T16:32:45.022688+0000","last_clean":"2023-11-08T16:32:45.022688+0000","last_became_active":"2023-11-08T16:32:44.021032+0000","last_became_peered":"2023-11-08T16:32:44.021032+0000","last_unstale":"2023-11-08T16:32:45.022688+0000","last_undegraded":"2023-11-08T16:32:45.022688+0000","last_fullsized":"2023-11-08T16:32:45.022688+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:35:45.917388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"56'2","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503924+0000","last_change":"2023-11-08T16:32:43.019241+0000","last_active":"2023-11-08T16:32:45.503924+0000","last_peered":"2023-11-08T16:32:45.503924+0000","last_clean":"2023-11-08T16:32:45.503924+0000","last_became_active":"2023-11-08T16:32:43.018976+0000","last_became_peered":"2023-11-08T16:32:43.018976+0000","last_unstale":"2023-11-08T16:32:45.503924+0000","last_undegraded":"2023-11-08T16:32:45.503924+0000","last_fullsized":"2023-11-08T16:32:45.503924+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:35:39.226180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.433812+0000","last_change":"2023-11-08T16:32:44.012036+0000","last_active":"2023-11-08T16:32:45.433812+0000","last_peered":"2023-11-08T16:32:45.433812+0000","last_clean":"2023-11-08T16:32:45.433812+0000","last_became_active":"2023-11-08T16:32:44.011743+0000","last_became_peered":"2023-11-08T16:32:44.011743+0000","last_unstale":"2023-11-08T16:32:45.433812+0000","last_undegraded":"2023-11-08T16:32:45.433812+0000","last_fullsized":"2023-11-08T16:32:45.433812+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:39:51.976792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.021925+0000","last_change":"2023-11-08T16:32:43.010550+0000","last_active":"2023-11-08T16:32:45.021925+0000","last_peered":"2023-11-08T16:32:45.021925+0000","last_clean":"2023-11-08T16:32:45.021925+0000","last_became_active":"2023-11-08T16:32:43.008948+0000","last_became_peered":"2023-11-08T16:32:43.008948+0000","last_unstale":"2023-11-08T16:32:45.021925+0000","last_undegraded":"2023-11-08T16:32:45.021925+0000","last_fullsized":"2023-11-08T16:32:45.021925+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:14:20.824314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.015534+0000","last_change":"2023-11-08T16:32:44.015388+0000","last_active":"2023-11-08T16:32:45.015534+0000","last_peered":"2023-11-08T16:32:45.015534+0000","last_clean":"2023-11-08T16:32:45.015534+0000","last_became_active":"2023-11-08T16:32:44.015136+0000","last_became_peered":"2023-11-08T16:32:44.015136+0000","last_unstale":"2023-11-08T16:32:45.015534+0000","last_undegraded":"2023-11-08T16:32:45.015534+0000","last_fullsized":"2023-11-08T16:32:45.015534+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:16:35.526220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503069+0000","last_change":"2023-11-08T16:32:43.019715+0000","last_active":"2023-11-08T16:32:45.503069+0000","last_peered":"2023-11-08T16:32:45.503069+0000","last_clean":"2023-11-08T16:32:45.503069+0000","last_became_active":"2023-11-08T16:32:43.018663+0000","last_became_peered":"2023-11-08T16:32:43.018663+0000","last_unstale":"2023-11-08T16:32:45.503069+0000","last_undegraded":"2023-11-08T16:32:45.503069+0000","last_fullsized":"2023-11-08T16:32:45.503069+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:14:20.824314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503185+0000","last_change":"2023-11-08T16:32:44.022565+0000","last_active":"2023-11-08T16:32:45.503185+0000","last_peered":"2023-11-08T16:32:45.503185+0000","last_clean":"2023-11-08T16:32:45.503185+0000","last_became_active":"2023-11-08T16:32:44.022278+0000","last_became_peered":"2023-11-08T16:32:44.022278+0000","last_unstale":"2023-11-08T16:32:45.503185+0000","last_undegraded":"2023-11-08T16:32:45.503185+0000","last_fullsized":"2023-11-08T16:32:45.503185+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:33:06.784059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.015621+0000","last_change":"2023-11-08T16:32:42.999751+0000","last_active":"2023-11-08T16:32:45.015621+0000","last_peered":"2023-11-08T16:32:45.015621+0000","last_clean":"2023-11-08T16:32:45.015621+0000","last_became_active":"2023-11-08T16:32:42.997167+0000","last_became_peered":"2023-11-08T16:32:42.997167+0000","last_unstale":"2023-11-08T16:32:45.015621+0000","last_undegraded":"2023-11-08T16:32:45.015621+0000","last_fullsized":"2023-11-08T16:32:45.015621+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:49:28.358371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.020912+0000","last_change":"2023-11-08T16:32:44.024536+0000","last_active":"2023-11-08T16:32:45.020912+0000","last_peered":"2023-11-08T16:32:45.020912+0000","last_clean":"2023-11-08T16:32:45.020912+0000","last_became_active":"2023-11-08T16:32:44.023634+0000","last_became_peered":"2023-11-08T16:32:44.023634+0000","last_unstale":"2023-11-08T16:32:45.020912+0000","last_undegraded":"2023-11-08T16:32:45.020912+0000","last_fullsized":"2023-11-08T16:32:45.020912+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:32:48.647621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.015504+0000","last_change":"2023-11-08T16:32:43.009878+0000","last_active":"2023-11-08T16:32:45.015504+0000","last_peered":"2023-11-08T16:32:45.015504+0000","last_clean":"2023-11-08T16:32:45.015504+0000","last_became_active":"2023-11-08T16:32:43.009590+0000","last_became_peered":"2023-11-08T16:32:43.009590+0000","last_unstale":"2023-11-08T16:32:45.015504+0000","last_undegraded":"2023-11-08T16:32:45.015504+0000","last_fullsized":"2023-11-08T16:32:45.015504+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:48:13.130951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503359+0000","last_change":"2023-11-08T16:32:44.012030+0000","last_active":"2023-11-08T16:32:45.503359+0000","last_peered":"2023-11-08T16:32:45.503359+0000","last_clean":"2023-11-08T16:32:45.503359+0000","last_became_active":"2023-11-08T16:32:44.011779+0000","last_became_peered":"2023-11-08T16:32:44.011779+0000","last_unstale":"2023-11-08T16:32:45.503359+0000","last_undegraded":"2023-11-08T16:32:45.503359+0000","last_fullsized":"2023-11-08T16:32:45.503359+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:05:11.019162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.501811+0000","last_change":"2023-11-08T16:32:43.010207+0000","last_active":"2023-11-08T16:32:45.501811+0000","last_peered":"2023-11-08T16:32:45.501811+0000","last_clean":"2023-11-08T16:32:45.501811+0000","last_became_active":"2023-11-08T16:32:43.009779+0000","last_became_peered":"2023-11-08T16:32:43.009779+0000","last_unstale":"2023-11-08T16:32:45.501811+0000","last_undegraded":"2023-11-08T16:32:45.501811+0000","last_fullsized":"2023-11-08T16:32:45.501811+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:09:12.621520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.016506+0000","last_change":"2023-11-08T16:32:44.001094+0000","last_active":"2023-11-08T16:32:45.016506+0000","last_peered":"2023-11-08T16:32:45.016506+0000","last_clean":"2023-11-08T16:32:45.016506+0000","last_became_active":"2023-11-08T16:32:44.000835+0000","last_became_peered":"2023-11-08T16:32:44.000835+0000","last_unstale":"2023-11-08T16:32:45.016506+0000","last_undegraded":"2023-11-08T16:32:45.016506+0000","last_fullsized":"2023-11-08T16:32:45.016506+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:02:50.903021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.020840+0000","last_change":"2023-11-08T16:32:43.011493+0000","last_active":"2023-11-08T16:32:45.020840+0000","last_peered":"2023-11-08T16:32:45.020840+0000","last_clean":"2023-11-08T16:32:45.020840+0000","last_became_active":"2023-11-08T16:32:43.011222+0000","last_became_peered":"2023-11-08T16:32:43.011222+0000","last_unstale":"2023-11-08T16:32:45.020840+0000","last_undegraded":"2023-11-08T16:32:45.020840+0000","last_fullsized":"2023-11-08T16:32:45.020840+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:49:42.781714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.502324+0000","last_change":"2023-11-08T16:32:44.013820+0000","last_active":"2023-11-08T16:32:45.502324+0000","last_peered":"2023-11-08T16:32:45.502324+0000","last_clean":"2023-11-08T16:32:45.502324+0000","last_became_active":"2023-11-08T16:32:44.013468+0000","last_became_peered":"2023-11-08T16:32:44.013468+0000","last_unstale":"2023-11-08T16:32:45.502324+0000","last_undegraded":"2023-11-08T16:32:45.502324+0000","last_fullsized":"2023-11-08T16:32:45.502324+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:42.967619+0000","last_clean_scrub_stamp":"2023-11-08T16:32:42.967619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:53:15.698962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-11-08T16:32:45.503682+0000","last_change":"2023-11-08T16:32:43.017975+0000","last_active":"2023-11-08T16:32:45.503682+0000","last_peered":"2023-11-08T16:32:45.503682+0000","last_clean":"2023-11-08T16:32:45.503682+0000","last_became_active":"2023-11-08T16:32:43.017714+0000","last_became_peered":"2023-11-08T16:32:43.017714+0000","last_unstale":"2023-11-08T16:32:45.503682+0000","last_undegraded":"2023-11-08T16:32:45.503682+0000","last_fullsized":"2023-11-08T16:32:45.503682+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":54,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:32:41.956433+0000","last_clean_scrub_stamp":"2023-11-08T16:32:41.956433+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:20:47.165497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":202,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-08T16:32:44.011647+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:32:44.011647+0000","last_peered":"2023-11-08T16:32:44.011647+0000","last_clean":"2023-11-08T16:32:44.011647+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:32:44.011647+0000","last_undegraded":"2023-11-08T16:32:44.011647+0000","last_fullsized":"2023-11-08T16:32:44.011647+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28280,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343560,"statfs":{"total":96636764160,"available":96607805440,"internally_reserved":0,"allocated":1282048,"data_stored":884233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.0800000000000001,"15min":1.085},"min":{"1min":0.77700000000000002,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.619,"5min":1.738,"15min":1.738},"last":1.522},{"interface":"front","average":{"1min":1.248,"5min":1.1180000000000001,"15min":1.1240000000000001},"min":{"1min":0.88200000000000001,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.5740000000000001,"5min":4.2729999999999997,"15min":4.2729999999999997},"last":1.179}]},{"osd":1,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.131,"15min":1.127},"min":{"1min":0.873,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.216,"15min":1.232},"min":{"1min":0.93899999999999995,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.923,"5min":2.2090000000000001,"15min":2.2090000000000001},"last":1.3680000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.2629999999999999,"15min":1.294},"min":{"1min":0.70899999999999996,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.992,"5min":4.2249999999999996,"15min":4.2249999999999996},"last":1.319},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.1879999999999999,"15min":1.1919999999999999},"min":{"1min":0.82799999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6679999999999999,"5min":1.992,"15min":1.992},"last":1.8260000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.1579999999999999,"15min":1.1539999999999999},"min":{"1min":0.96799999999999997,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.5800000000000001,"5min":1.609,"15min":1.609},"last":1.631},{"interface":"front","average":{"1min":1.385,"5min":1.454,"15min":1.514},"min":{"1min":1.095,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.772,"5min":4.181,"15min":4.181},"last":1.2330000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.236,"15min":1.214},"min":{"1min":0.90500000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"max":{"1min":1.7689999999999999,"5min":1.956,"15min":1.956},"last":0.999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.343,"15min":1.343},"min":{"1min":0.92000000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9490000000000001,"5min":2.3730000000000002,"15min":2.3730000000000002},"last":1.474}]},{"osd":5,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.3400000000000001,"15min":1.3320000000000001},"min":{"1min":1.0389999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.7070000000000001,"5min":2.222,"15min":2.222},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.4550000000000001,"15min":1.47},"min":{"1min":0.96199999999999997,"5min":0.878,"15min":0.878},"max":{"1min":1.7250000000000001,"5min":2.589,"15min":2.589},"last":1.155}]},{"osd":6,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5189999999999999,"15min":1.5289999999999999},"min":{"1min":1.111,"5min":1.026,"15min":1.026},"max":{"1min":1.873,"5min":2.496,"15min":2.496},"last":1.419},{"interface":"front","average":{"1min":1.448,"5min":1.458,"15min":1.4710000000000001},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":1.865,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.1060000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27696,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":688128,"data_stored":294393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0669999999999999,"15min":1.0629999999999999},"min":{"1min":0.74099999999999999,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.742,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.74199999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.056,"15min":1.0740000000000001},"min":{"1min":0.70599999999999996,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.661,"5min":1.661,"15min":1.661},"last":0.68799999999999994}]},{"osd":1,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.0640000000000001,"15min":1.024},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.86199999999999999},{"interface":"front","average":{"1min":1.2390000000000001,"5min":1.1279999999999999,"15min":1.113},"min":{"1min":0.94499999999999995,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":1.601,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.82499999999999996}]},{"osd":2,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.216,"15min":1.23},"min":{"1min":0.80100000000000005,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.173},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1819999999999999,"15min":1.1839999999999999},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.5920000000000001,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":0.96899999999999997}]},{"osd":3,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.258,"15min":1.298},"min":{"1min":0.94199999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.821,"5min":1.8220000000000001,"15min":1.8220000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.323,"15min":1.319},"min":{"1min":0.82999999999999996,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.7749999999999999,"5min":1.95,"15min":1.95},"last":1.0249999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.349,"5min":1.357,"15min":1.3819999999999999},"min":{"1min":0.86599999999999999,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.7430000000000001,"5min":2.0790000000000002,"15min":2.0790000000000002},"last":1.0760000000000001},{"interface":"front","average":{"1min":1.458,"5min":1.4450000000000001,"15min":1.4550000000000001},"min":{"1min":1.026,"5min":0.84699999999999998,"15min":0.84699999999999998},"max":{"1min":1.8859999999999999,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.3240000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.327,"15min":1.3029999999999999},"min":{"1min":0.94299999999999995,"5min":0.877,"15min":0.877},"max":{"1min":1.665,"5min":2.105,"15min":2.105},"last":1.21},{"interface":"front","average":{"1min":1.593,"5min":1.5589999999999999,"15min":1.5740000000000001},"min":{"1min":1.347,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.982,"5min":2.202,"15min":2.202},"last":1.276}]},{"osd":7,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.595,"15min":1.6259999999999999},"min":{"1min":1.3680000000000001,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":2.0249999999999999,"5min":3.8959999999999999,"15min":3.8959999999999999},"last":1.246},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.522,"15min":1.534},"min":{"1min":1.169,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":1.762,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.1299999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":679936,"data_stored":293741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.071,"15min":1.0580000000000001},"min":{"1min":0.68700000000000006,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.712,"5min":2.4609999999999999,"15min":2.4609999999999999},"last":1.7969999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.115,"15min":1.1319999999999999},"min":{"1min":0.76000000000000001,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.5720000000000001,"5min":2.3639999999999999,"15min":2.3639999999999999},"last":1.4179999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.228,"15min":1.246},"min":{"1min":0.878,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.6839999999999999,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":1.4870000000000001},{"interface":"front","average":{"1min":1.242,"5min":1.2669999999999999,"15min":1.3129999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.486,"5min":3.2349999999999999,"15min":3.2349999999999999},"last":1.6319999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.2310000000000001,"15min":1.2629999999999999},"min":{"1min":0.79300000000000004,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.6519999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.238},{"interface":"front","average":{"1min":1.1890000000000001,"5min":1.2070000000000001,"15min":1.2290000000000001},"min":{"1min":0.78500000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.5880000000000001,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.089}]},{"osd":3,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2609999999999999,"5min":1.2909999999999999,"15min":1.333},"min":{"1min":0.85399999999999998,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.6519999999999999,"5min":2.8330000000000002,"15min":2.8330000000000002},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3160000000000001,"5min":1.3129999999999999,"15min":1.3280000000000001},"min":{"1min":0.89300000000000002,"5min":0.876,"15min":0.876},"max":{"1min":1.6539999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.575}]},{"osd":4,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.361,"15min":1.3959999999999999},"min":{"1min":0.85499999999999998,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.6279999999999999,"5min":3.9790000000000001,"15min":3.9790000000000001},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.46,"15min":1.514},"min":{"1min":0.92200000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.6779999999999999,"5min":3.0209999999999999,"15min":3.0209999999999999},"last":1.536}]},{"osd":6,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.403,"15min":1.4279999999999999},"min":{"1min":1.02,"5min":0.752,"15min":0.752},"max":{"1min":1.7130000000000001,"5min":2.5550000000000002,"15min":2.5550000000000002},"last":1.292},{"interface":"front","average":{"1min":1.421,"5min":1.498,"15min":1.542},"min":{"1min":1.0620000000000001,"5min":0.96099999999999997,"15min":0.96099999999999997},"max":{"1min":1.887,"5min":4.1189999999999998,"15min":4.1189999999999998},"last":1.944}]},{"osd":7,"last update":"Wed Nov 8 16:32:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.552,"15min":1.6000000000000001},"min":{"1min":1.0580000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"max":{"1min":2.101,"5min":3.1160000000000001,"15min":3.1160000000000001},"last":1.7050000000000001},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5620000000000001,"15min":1.5720000000000001},"min":{"1min":1.2190000000000001,"5min":0.98499999999999999,"15min":0.98499999999999999},"max":{"1min":1.976,"5min":2.7050000000000001,"15min":2.7050000000000001},"last":0.996}]}]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":684032,"data_stored":294323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.056,"15min":1.0680000000000001},"min":{"1min":0.84599999999999997,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.319,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.0640000000000001,"15min":1.0409999999999999},"min":{"1min":0.72599999999999998,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":1.3320000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.1240000000000001,"15min":1.1200000000000001},"min":{"1min":0.78300000000000003,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.996},{"interface":"front","average":{"1min":1.262,"5min":1.1619999999999999,"15min":1.1559999999999999},"min":{"1min":0.80500000000000005,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.93700000000000006}]},{"osd":2,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.198,"15min":1.1799999999999999},"min":{"1min":0.85499999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.746,"5min":1.746,"15min":1.746},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.371,"5min":1.2689999999999999,"15min":1.282},"min":{"1min":0.89200000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":1.8240000000000001},"last":1.0409999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.294,"15min":1.2729999999999999},"min":{"1min":0.85899999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"max":{"1min":1.875,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.2929999999999999,"15min":1.2949999999999999},"min":{"1min":1.0389999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.194}]},{"osd":5,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3500000000000001,"15min":1.371},"min":{"1min":1.0920000000000001,"5min":0.80100000000000005,"15min":0.80100000000000005},"max":{"1min":1.6970000000000001,"5min":1.9870000000000001,"15min":1.9870000000000001},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.417,"15min":1.421},"min":{"1min":0.86899999999999999,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.9830000000000001,"5min":2.46,"15min":2.46},"last":1.4279999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3979999999999999,"5min":1.4330000000000001,"15min":1.4450000000000001},"min":{"1min":0.95199999999999996,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.2890000000000001,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":1.361},{"interface":"front","average":{"1min":1.48,"5min":1.5489999999999999,"15min":1.5880000000000001},"min":{"1min":1.0069999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":2.1549999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.3799999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:31:59 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.536,"15min":1.518},"min":{"1min":1.1299999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"max":{"1min":2.077,"5min":2.1309999999999998,"15min":2.1309999999999998},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.573,"5min":1.607,"15min":1.6259999999999999},"min":{"1min":1.1719999999999999,"5min":1.169,"15min":1.169},"max":{"1min":2.1949999999999998,"5min":2.6859999999999999,"15min":2.6859999999999999},"last":1.3049999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":696320,"data_stored":295375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.82899999999999996,"15min":0.83799999999999997},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":1.079},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.94799999999999995,"15min":0.98399999999999999},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.3440000000000001,"5min":1.585,"15min":1.585},"last":0.879}]},{"osd":1,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.89200000000000002,"15min":0.91300000000000003},"min":{"1min":0.45900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.2869999999999999,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.98399999999999999,"15min":0.97799999999999998},"min":{"1min":0.51300000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.5720000000000001,"5min":1.829,"15min":1.829},"last":1.196}]},{"osd":2,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.98999999999999999,"15min":0.98299999999999998},"min":{"1min":0.47399999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.452,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.056,"5min":1.137,"15min":1.1619999999999999},"min":{"1min":0.56999999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.375,"5min":2.4660000000000002,"15min":2.4660000000000002},"last":1.252}]},{"osd":4,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.216,"15min":1.2270000000000001},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.8120000000000001,"5min":2.25,"15min":2.25},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.286,"15min":1.2749999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.786,"5min":2.1829999999999998,"15min":2.1829999999999998},"last":1.288}]},{"osd":5,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3400000000000001,"15min":1.343},"min":{"1min":0.90300000000000002,"5min":0.83399999999999996,"15min":0.83399999999999996},"max":{"1min":1.7689999999999999,"5min":2.3999999999999999,"15min":2.3999999999999999},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.3540000000000001,"15min":1.365},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.593,"5min":2.593,"15min":2.593},"last":1.3169999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.407,"15min":1.4379999999999999},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.6539999999999999,"5min":2.9569999999999999,"15min":2.9569999999999999},"last":1.363},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.4159999999999999,"15min":1.452},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.774,"5min":2.762,"15min":2.762},"last":1.0449999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.4710000000000001,"15min":1.4930000000000001},"min":{"1min":0.79900000000000004,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.827,"5min":2.8700000000000001,"15min":2.8700000000000001},"last":1.137},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.4430000000000001,"15min":1.4610000000000001},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.6899999999999999,"5min":2.633,"15min":2.633},"last":1.161}]}]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":692224,"data_stored":295353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":1.0049999999999999,"15min":1.107},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.575,"5min":4.5369999999999999,"15min":4.5369999999999999},"last":1.345},{"interface":"front","average":{"1min":0.92100000000000004,"5min":0.89000000000000001,"15min":0.89300000000000002},"min":{"1min":0.49099999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.89400000000000002}]},{"osd":1,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1240000000000001,"15min":1.1859999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.7829999999999999,"5min":4.4980000000000002,"15min":4.4980000000000002},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.089,"5min":1.0720000000000001,"15min":1.083},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6120000000000001,"5min":1.6879999999999999,"15min":1.6879999999999999},"last":1.0609999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":1.04,"15min":1.1379999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.746,"5min":3.3050000000000002,"15min":3.3050000000000002},"last":0.748},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.226,"15min":1.3300000000000001},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.6819999999999999,"5min":4.5899999999999999,"15min":4.5899999999999999},"last":1.095}]},{"osd":4,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.4139999999999999,"15min":1.498},"min":{"1min":0.92400000000000004,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.627,"5min":4.9409999999999998,"15min":4.9409999999999998},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.2969999999999999,"15min":1.333},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.6240000000000001,"5min":2.46,"15min":2.46},"last":1.2470000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.3260000000000001,"15min":1.3799999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.706,"5min":2.8530000000000002,"15min":2.8530000000000002},"last":1.319},{"interface":"front","average":{"1min":1.175,"5min":1.3600000000000001,"15min":1.4339999999999999},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.47,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":1.147}]},{"osd":6,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.4099999999999999,"15min":1.4850000000000001},"min":{"1min":0.66800000000000004,"5min":0.625,"15min":0.625},"max":{"1min":1.595,"5min":3.536,"15min":3.536},"last":1.373},{"interface":"front","average":{"1min":1.306,"5min":1.512,"15min":1.5880000000000001},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.877,"5min":4.7549999999999999,"15min":4.7549999999999999},"last":1.1990000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:31:54 2023","interfaces":[{"interface":"back","average":{"1min":1.331,"5min":1.4590000000000001,"15min":1.5169999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.7070000000000001,"5min":2.9700000000000002,"15min":2.9700000000000002},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.502,"15min":1.5900000000000001},"min":{"1min":0.80600000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.6020000000000001,"5min":3.4950000000000001,"15min":3.4950000000000001},"last":1.2649999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":688128,"data_stored":293901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.76900000000000002,"15min":0.78800000000000003},"min":{"1min":0.39300000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.169,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.96499999999999997,"5min":1.0129999999999999,"15min":1.0760000000000001},"min":{"1min":0.45100000000000001,"5min":0.309,"15min":0.309},"max":{"1min":1.6459999999999999,"5min":4.0309999999999997,"15min":4.0309999999999997},"last":1.1100000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.089,"15min":1.1419999999999999},"min":{"1min":0.496,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.593,"5min":3.0339999999999998,"15min":3.0339999999999998},"last":1.1779999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.079,"15min":1.145},"min":{"1min":0.47999999999999998,"5min":0.443,"15min":0.443},"max":{"1min":1.4850000000000001,"5min":2.6840000000000002,"15min":2.6840000000000002},"last":0.78200000000000003}]},{"osd":3,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":1.113,"15min":1.1619999999999999},"min":{"1min":0.432,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.73,"5min":4.3179999999999996,"15min":4.3179999999999996},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.107,"15min":1.1579999999999999},"min":{"1min":0.58099999999999996,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.548,"5min":3.9390000000000001,"15min":3.9390000000000001},"last":1.024}]},{"osd":4,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.3169999999999999,"15min":1.385},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.637,"5min":3.1659999999999999,"15min":3.1659999999999999},"last":1.339},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.3400000000000001,"15min":1.3979999999999999},"min":{"1min":0.78500000000000003,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5840000000000001,"5min":2.734,"15min":2.734},"last":1.472}]},{"osd":5,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.6519999999999999,"15min":1.8260000000000001},"min":{"1min":0.93899999999999995,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.8799999999999999,"5min":8.4730000000000008,"15min":8.4730000000000008},"last":1.55},{"interface":"front","average":{"1min":1.304,"5min":1.6120000000000001,"15min":1.7689999999999999},"min":{"1min":0.72199999999999998,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.8149999999999999,"5min":8.8040000000000003,"15min":8.8040000000000003},"last":1.4990000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4750000000000001,"15min":1.536},"min":{"1min":0.80400000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":2.0129999999999999,"5min":3.984,"15min":3.984},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.464,"15min":1.5129999999999999},"min":{"1min":0.74099999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.984,"5min":3.2450000000000001,"15min":3.2450000000000001},"last":1.4370000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:31:56 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.534,"15min":1.6519999999999999},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.546,"5min":4.3609999999999998,"15min":4.3609999999999998},"last":1.381},{"interface":"front","average":{"1min":1.472,"5min":1.6240000000000001,"15min":1.724},"min":{"1min":1.022,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.958,"5min":5.3840000000000003,"15min":5.3840000000000003},"last":1.2969999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574897,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":884165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.77200000000000002,"15min":0.78400000000000003},"min":{"1min":0.48899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.202,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.878},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.96499999999999997,"15min":0.98699999999999999},"min":{"1min":0.59599999999999997,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.3009999999999999,"5min":2.3340000000000001,"15min":2.3340000000000001},"last":1.1910000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.92800000000000005,"15min":0.96699999999999997},"min":{"1min":0.55100000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3560000000000001,"5min":1.762,"15min":1.762},"last":1.101},{"interface":"front","average":{"1min":0.91500000000000004,"5min":1.0149999999999999,"15min":1.0469999999999999},"min":{"1min":0.51600000000000001,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.395,"5min":1.879,"15min":1.879},"last":0.97099999999999997}]},{"osd":3,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.94499999999999995,"15min":0.96099999999999997},"min":{"1min":0.438,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.399,"5min":1.6970000000000001,"15min":1.6970000000000001},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.1719999999999999,"15min":1.1930000000000001},"min":{"1min":0.60799999999999998,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.528,"5min":2.7679999999999998,"15min":2.7679999999999998},"last":1.45}]},{"osd":4,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.1519999999999999,"15min":1.1579999999999999},"min":{"1min":0.68700000000000006,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.5529999999999999,"5min":2.2559999999999998,"15min":2.2559999999999998},"last":1.496},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2789999999999999,"15min":1.306},"min":{"1min":0.80300000000000005,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.6499999999999999,"5min":2.867,"15min":2.867},"last":1.3149999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.264,"15min":1.256},"min":{"1min":0.78400000000000003,"5min":0.75600000000000001,"15min":0.75600000000000001},"max":{"1min":1.599,"5min":2.7280000000000002,"15min":2.7280000000000002},"last":1.724},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.2250000000000001,"15min":1.2490000000000001},"min":{"1min":0.86899999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.4990000000000001,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.401}]},{"osd":6,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.3420000000000001,"15min":1.367},"min":{"1min":0.83299999999999996,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":1.5669999999999999,"5min":2.6739999999999999,"15min":2.6739999999999999},"last":1.538},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.371,"15min":1.383},"min":{"1min":0.99099999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.7230000000000001,"5min":2.8260000000000001,"15min":2.8260000000000001},"last":1.6739999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:31:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.4059999999999999,"15min":1.4159999999999999},"min":{"1min":0.94099999999999995,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.6679999999999999,"5min":2.625,"15min":2.625},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.345,"15min":1.371},"min":{"1min":0.71899999999999997,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.6080000000000001,"5min":2.431,"15min":2.431},"last":1.5649999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:32:48.942 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T16:32:48.943 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T16:32:49.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.195+0000 7fe546dee640 1 Processor -- start 2023-11-08T16:32:49.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.195+0000 7fe546dee640 1 -- start start 2023-11-08T16:32:49.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.197+0000 7fe546dee640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5400b0770 0x7fe54006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.197+0000 7fe546dee640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe5400ae000 0x7fe5400cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.197+0000 7fe546dee640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5400b0b50 0x7fe5400b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.197+0000 7fe546dee640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5400cb3c0 con 0x7fe5400b0770 2023-11-08T16:32:49.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.197+0000 7fe546dee640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5400ca580 con 0x7fe5400ae000 2023-11-08T16:32:49.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.197+0000 7fe546dee640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54006fe50 con 0x7fe5400b0b50 2023-11-08T16:32:49.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.197+0000 7fe544b8a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5400b0770 0x7fe54006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:49.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.197+0000 7fe544b8a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe5400b0770 0x7fe54006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43820/0 (socket says 172.21.15.81:43820) 2023-11-08T16:32:49.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.197+0000 7fe544b8a640 1 -- 172.21.15.81:0/1637085808 learned_addr learned my addr 172.21.15.81:0/1637085808 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.198+0000 7fe54538b640 1 --2- 172.21.15.81:0/1637085808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5400b0b50 0x7fe5400b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:49.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.198+0000 7fe53ffff640 1 --2- 172.21.15.81:0/1637085808 >> v2:172.21.15.93:3300/0 conn(0x7fe5400ae000 0x7fe5400cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:49.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.198+0000 7fe544b8a640 1 -- 172.21.15.81:0/1637085808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5400b0b50 msgr2=0x7fe5400b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:49.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.198+0000 7fe544b8a640 1 --2- 172.21.15.81:0/1637085808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5400b0b50 0x7fe5400b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.198+0000 7fe544b8a640 1 -- 172.21.15.81:0/1637085808 >> v2:172.21.15.93:3300/0 conn(0x7fe5400ae000 msgr2=0x7fe5400cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:49.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.198+0000 7fe544b8a640 1 --2- 172.21.15.81:0/1637085808 >> v2:172.21.15.93:3300/0 conn(0x7fe5400ae000 0x7fe5400cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.198+0000 7fe544b8a640 1 -- 172.21.15.81:0/1637085808 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5400ccec0 con 0x7fe5400b0770 2023-11-08T16:32:49.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.198+0000 7fe53ffff640 1 --2- 172.21.15.81:0/1637085808 >> v2:172.21.15.93:3300/0 conn(0x7fe5400ae000 0x7fe5400cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:49.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.199+0000 7fe544b8a640 1 --2- 172.21.15.81:0/1637085808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5400b0770 0x7fe54006f910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe54015e9c0 tx=0x7fe53402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6bbd8581204edc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:49.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.199+0000 7fe53f7fe640 1 -- 172.21.15.81:0/1637085808 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534031860 con 0x7fe5400b0770 2023-11-08T16:32:49.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.199+0000 7fe53f7fe640 1 -- 172.21.15.81:0/1637085808 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5340319c0 con 0x7fe5400b0770 2023-11-08T16:32:49.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.199+0000 7fe53f7fe640 1 -- 172.21.15.81:0/1637085808 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534031d50 con 0x7fe5400b0770 2023-11-08T16:32:49.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.200+0000 7fe546dee640 1 -- 172.21.15.81:0/1637085808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5400b0770 msgr2=0x7fe54006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:49.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.200+0000 7fe546dee640 1 --2- 172.21.15.81:0/1637085808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5400b0770 0x7fe54006f910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe54015e9c0 tx=0x7fe53402b900 comp rx=0 tx=0).stop 2023-11-08T16:32:49.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.200+0000 7fe546dee640 1 -- 172.21.15.81:0/1637085808 shutdown_connections 2023-11-08T16:32:49.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.200+0000 7fe546dee640 1 --2- 172.21.15.81:0/1637085808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5400b0b50 0x7fe5400b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.200+0000 7fe546dee640 1 --2- 172.21.15.81:0/1637085808 >> v2:172.21.15.93:3300/0 conn(0x7fe5400ae000 0x7fe5400cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.200+0000 7fe546dee640 1 --2- 172.21.15.81:0/1637085808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5400b0770 0x7fe54006f910 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.200+0000 7fe546dee640 1 -- 172.21.15.81:0/1637085808 >> 172.21.15.81:0/1637085808 conn(0x7fe5400cb500 msgr2=0x7fe5400cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:49.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.200+0000 7fe546dee640 1 -- 172.21.15.81:0/1637085808 shutdown_connections 2023-11-08T16:32:49.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.200+0000 7fe546dee640 1 -- 172.21.15.81:0/1637085808 wait complete. 2023-11-08T16:32:49.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.202+0000 7fe546dee640 1 Processor -- start 2023-11-08T16:32:49.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.202+0000 7fe546dee640 1 -- start start 2023-11-08T16:32:49.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.203+0000 7fe546dee640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ae000 0x7fe54014e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.203+0000 7fe546dee640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5400b0770 0x7fe5400a8750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.203+0000 7fe546dee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5400b0b50 0x7fe5400c4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.203+0000 7fe546dee640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5401f4b90 con 0x7fe5400b0b50 2023-11-08T16:32:49.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.203+0000 7fe546dee640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54014ead0 con 0x7fe5400ae000 2023-11-08T16:32:49.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.203+0000 7fe546dee640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54014b790 con 0x7fe5400b0770 2023-11-08T16:32:49.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.203+0000 7fe54538b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5400b0b50 0x7fe5400c4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:49.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.203+0000 7fe54538b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5400b0b50 0x7fe5400c4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43822/0 (socket says 172.21.15.81:43822) 2023-11-08T16:32:49.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.203+0000 7fe54538b640 1 -- 172.21.15.81:0/4087833188 learned_addr learned my addr 172.21.15.81:0/4087833188 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:49.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe53ffff640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5400b0770 0x7fe5400a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:49.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe544b8a640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ae000 0x7fe54014e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:49.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe54538b640 1 -- 172.21.15.81:0/4087833188 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5400b0770 msgr2=0x7fe5400a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:49.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe54538b640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5400b0770 0x7fe5400a8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe54538b640 1 -- 172.21.15.81:0/4087833188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ae000 msgr2=0x7fe54014e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:49.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe54538b640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ae000 0x7fe54014e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe54538b640 1 -- 172.21.15.81:0/4087833188 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5340094d0 con 0x7fe5400b0b50 2023-11-08T16:32:49.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe54538b640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5400b0b50 0x7fe5400c4780 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe538009fd0 tx=0x7fe53800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:49.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe53d7fa640 1 -- 172.21.15.81:0/4087833188 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538014420 con 0x7fe5400b0b50 2023-11-08T16:32:49.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe53d7fa640 1 -- 172.21.15.81:0/4087833188 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe538014580 con 0x7fe5400b0b50 2023-11-08T16:32:49.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe53d7fa640 1 -- 172.21.15.81:0/4087833188 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe538014930 con 0x7fe5400b0b50 2023-11-08T16:32:49.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.204+0000 7fe546dee640 1 -- 172.21.15.81:0/4087833188 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5401f3f10 con 0x7fe5400b0b50 2023-11-08T16:32:49.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.205+0000 7fe546dee640 1 -- 172.21.15.81:0/4087833188 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5401f44d0 con 0x7fe5400b0b50 2023-11-08T16:32:49.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.206+0000 7fe546dee640 1 -- 172.21.15.81:0/4087833188 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5401f4cd0 con 0x7fe5400b0b50 2023-11-08T16:32:49.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.207+0000 7fe53d7fa640 1 -- 172.21.15.81:0/4087833188 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe538014a90 con 0x7fe5400b0b50 2023-11-08T16:32:49.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.208+0000 7fe53d7fa640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe52406ae70 0x7fe52406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.208+0000 7fe53d7fa640 1 -- 172.21.15.81:0/4087833188 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe538089f10 con 0x7fe5400b0b50 2023-11-08T16:32:49.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.208+0000 7fe544b8a640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe52406ae70 0x7fe52406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:49.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.209+0000 7fe544b8a640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe52406ae70 0x7fe52406d330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe534036000 tx=0x7fe53403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:49.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.213+0000 7fe53d7fa640 1 -- 172.21.15.81:0/4087833188 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5401f4cd0 con 0x7fe5400b0b50 2023-11-08T16:32:49.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.470+0000 7fe546dee640 1 -- 172.21.15.81:0/4087833188 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe5401f4cd0 con 0x7fe5400b0b50 2023-11-08T16:32:49.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.475+0000 7fe53d7fa640 1 -- 172.21.15.81:0/4087833188 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe5401f4cd0 con 0x7fe5400b0b50 2023-11-08T16:32:49.477 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2023-11-08T16:32:49.477 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ== 2023-11-08T16:32:49.477 INFO:teuthology.orchestra.run.smithi081.stdout: caps mds = "allow" 2023-11-08T16:32:49.477 INFO:teuthology.orchestra.run.smithi081.stdout: caps mon = "allow r" 2023-11-08T16:32:49.478 INFO:teuthology.orchestra.run.smithi081.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T16:32:49.478 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2023-11-08T16:32:49.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.477+0000 7fe546dee640 1 -- 172.21.15.81:0/4087833188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe52406ae70 msgr2=0x7fe52406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:49.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.477+0000 7fe546dee640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe52406ae70 0x7fe52406d330 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe534036000 tx=0x7fe53403c040 comp rx=0 tx=0).stop 2023-11-08T16:32:49.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.477+0000 7fe546dee640 1 -- 172.21.15.81:0/4087833188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5400b0b50 msgr2=0x7fe5400c4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:49.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.477+0000 7fe546dee640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5400b0b50 0x7fe5400c4780 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe538009fd0 tx=0x7fe53800e4b0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.478+0000 7fe546dee640 1 -- 172.21.15.81:0/4087833188 shutdown_connections 2023-11-08T16:32:49.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.478+0000 7fe546dee640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe52406ae70 0x7fe52406d330 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.478+0000 7fe546dee640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5400b0b50 0x7fe5400c4780 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.478+0000 7fe546dee640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5400b0770 0x7fe5400a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.478+0000 7fe546dee640 1 --2- 172.21.15.81:0/4087833188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ae000 0x7fe54014e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.478+0000 7fe546dee640 1 -- 172.21.15.81:0/4087833188 >> 172.21.15.81:0/4087833188 conn(0x7fe5400cb500 msgr2=0x7fe5400af860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:49.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.478+0000 7fe546dee640 1 -- 172.21.15.81:0/4087833188 shutdown_connections 2023-11-08T16:32:49.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.478+0000 7fe546dee640 1 -- 172.21.15.81:0/4087833188 wait complete. 2023-11-08T16:32:49.498 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:32:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.698+0000 7f84cf494640 1 Processor -- start 2023-11-08T16:32:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.698+0000 7f84cf494640 1 -- start start 2023-11-08T16:32:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.700+0000 7f84cf494640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f84c80ade50 0x7f84c80ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.700+0000 7f84cf494640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f84c815ea60 0x7f84c815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.700+0000 7f84cf494640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84c806b630 con 0x7f84c815a680 2023-11-08T16:32:49.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.700+0000 7f84cf494640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c80b1860 con 0x7f84c80ade50 2023-11-08T16:32:49.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.700+0000 7f84cf494640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84c80ca9c0 con 0x7f84c815ea60 2023-11-08T16:32:49.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.700+0000 7f84cd230640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f84c80ade50 0x7f84c80ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:49.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.700+0000 7f84cca2f640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f84c815a680 0x7f84c815ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51894/0 (socket says 172.21.15.81:51894) 2023-11-08T16:32:49.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.700+0000 7f84cca2f640 1 -- 172.21.15.81:0/3079313167 learned_addr learned my addr 172.21.15.81:0/3079313167 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:49.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.701+0000 7f84cda31640 1 --2- 172.21.15.81:0/3079313167 >> v2:172.21.15.93:3300/0 conn(0x7f84c815ea60 0x7f84c815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:49.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.701+0000 7f84cd230640 1 -- 172.21.15.81:0/3079313167 >> v2:172.21.15.93:3300/0 conn(0x7f84c815ea60 msgr2=0x7f84c815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:49.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.701+0000 7f84bffff640 1 -- 172.21.15.81:0/3079313167 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252908745 0 0) 0x7f84c806b630 con 0x7f84c815a680 2023-11-08T16:32:49.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.701+0000 7f84cd230640 1 --2- 172.21.15.81:0/3079313167 >> v2:172.21.15.93:3300/0 conn(0x7f84c815ea60 0x7f84c815eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.701+0000 7f84cd230640 1 -- 172.21.15.81:0/3079313167 >> v1:172.21.15.81:6789/0 conn(0x7f84c815a680 legacy=0x7f84c815ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:49.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.701+0000 7f84cd230640 1 -- 172.21.15.81:0/3079313167 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c80cabb0 con 0x7f84c80ade50 2023-11-08T16:32:49.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.701+0000 7f84cda31640 1 --2- 172.21.15.81:0/3079313167 >> v2:172.21.15.93:3300/0 conn(0x7f84c815ea60 0x7f84c815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:49.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.702+0000 7f84cd230640 1 --2- 172.21.15.81:0/3079313167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84c80ade50 0x7f84c80ca480 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f84c80ca220 tx=0x7f84c402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2611d9242374803 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:49.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.702+0000 7f84bffff640 1 -- 172.21.15.81:0/3079313167 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4031830 con 0x7f84c80ade50 2023-11-08T16:32:49.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.702+0000 7f84bffff640 1 -- 172.21.15.81:0/3079313167 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c4031990 con 0x7f84c80ade50 2023-11-08T16:32:49.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.702+0000 7f84bffff640 1 -- 172.21.15.81:0/3079313167 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4031d20 con 0x7f84c80ade50 2023-11-08T16:32:49.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.703+0000 7f84cf494640 1 -- 172.21.15.81:0/3079313167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84c80ade50 msgr2=0x7f84c80ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:49.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.703+0000 7f84cf494640 1 --2- 172.21.15.81:0/3079313167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84c80ade50 0x7f84c80ca480 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f84c80ca220 tx=0x7f84c402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.703+0000 7f84cf494640 1 -- 172.21.15.81:0/3079313167 shutdown_connections 2023-11-08T16:32:49.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.703+0000 7f84cf494640 1 --2- 172.21.15.81:0/3079313167 >> v2:172.21.15.93:3300/0 conn(0x7f84c815ea60 0x7f84c815eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.703+0000 7f84cf494640 1 --2- 172.21.15.81:0/3079313167 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84c80ade50 0x7f84c80ca480 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.703+0000 7f84cf494640 1 -- 172.21.15.81:0/3079313167 >> 172.21.15.81:0/3079313167 conn(0x7f84c80cb490 msgr2=0x7f84c80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:49.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.703+0000 7f84cf494640 1 -- 172.21.15.81:0/3079313167 shutdown_connections 2023-11-08T16:32:49.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.703+0000 7f84cf494640 1 -- 172.21.15.81:0/3079313167 wait complete. 2023-11-08T16:32:49.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.705+0000 7f84cf494640 1 Processor -- start 2023-11-08T16:32:49.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.705+0000 7f84cf494640 1 -- start start 2023-11-08T16:32:49.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cf494640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84c80ade50 0x7f84c814efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cf494640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84c815a680 0x7f84c81f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cf494640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84c815ea60 0x7f84c81f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cf494640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c8157720 con 0x7f84c815a680 2023-11-08T16:32:49.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cf494640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c814f4f0 con 0x7f84c80ade50 2023-11-08T16:32:49.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cf494640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c81f3780 con 0x7f84c815ea60 2023-11-08T16:32:49.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cca2f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84c815a680 0x7f84c81f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:49.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cca2f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84c815a680 0x7f84c81f2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43840/0 (socket says 172.21.15.81:43840) 2023-11-08T16:32:49.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cca2f640 1 -- 172.21.15.81:0/2960917415 learned_addr learned my addr 172.21.15.81:0/2960917415 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:49.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cca2f640 1 -- 172.21.15.81:0/2960917415 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84c815ea60 msgr2=0x7f84c81f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:49.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cca2f640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84c815ea60 0x7f84c81f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cca2f640 1 -- 172.21.15.81:0/2960917415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84c80ade50 msgr2=0x7f84c814efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:49.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cca2f640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84c80ade50 0x7f84c814efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.706+0000 7f84cca2f640 1 -- 172.21.15.81:0/2960917415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c4009400 con 0x7f84c815a680 2023-11-08T16:32:49.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.707+0000 7f84cca2f640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84c815a680 0x7f84c81f2c60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c500 tx=0x7f84b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:49.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.707+0000 7f84bdffb640 1 -- 172.21.15.81:0/2960917415 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8007560 con 0x7f84c815a680 2023-11-08T16:32:49.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.707+0000 7f84bdffb640 1 -- 172.21.15.81:0/2960917415 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b80047e0 con 0x7f84c815a680 2023-11-08T16:32:49.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.707+0000 7f84bdffb640 1 -- 172.21.15.81:0/2960917415 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8004b90 con 0x7f84c815a680 2023-11-08T16:32:49.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.707+0000 7f84cf494640 1 -- 172.21.15.81:0/2960917415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c81f3a60 con 0x7f84c815a680 2023-11-08T16:32:49.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.707+0000 7f84cf494640 1 -- 172.21.15.81:0/2960917415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c8157c10 con 0x7f84c815a680 2023-11-08T16:32:49.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.708+0000 7f84cf494640 1 -- 172.21.15.81:0/2960917415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84c81f3c50 con 0x7f84c815a680 2023-11-08T16:32:49.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.709+0000 7f84bdffb640 1 -- 172.21.15.81:0/2960917415 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84b8004cf0 con 0x7f84c815a680 2023-11-08T16:32:49.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.710+0000 7f84bdffb640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84ac06ae70 0x7f84ac06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:49.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.710+0000 7f84bdffb640 1 -- 172.21.15.81:0/2960917415 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f84b8088730 con 0x7f84c815a680 2023-11-08T16:32:49.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.710+0000 7f84cd230640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84ac06ae70 0x7f84ac06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:49.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.711+0000 7f84cd230640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84ac06ae70 0x7f84ac06d330 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f84c4036000 tx=0x7f84c4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:49.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.714+0000 7f84bdffb640 1 -- 172.21.15.81:0/2960917415 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84c81f3c50 con 0x7f84c815a680 2023-11-08T16:32:49.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.925+0000 7f84cf494640 1 -- 172.21.15.81:0/2960917415 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84c81f3c50 con 0x7f84c815a680 2023-11-08T16:32:49.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.926+0000 7f84bdffb640 1 -- 172.21.15.81:0/2960917415 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f84c81f3c50 con 0x7f84c815a680 2023-11-08T16:32:49.928 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:49.928 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3492741644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3492741644},{"type":"v1","addr":"172.21.15.93:6833","nonce":3492741644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":4916,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/119699732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":119699732},{"type":"v1","addr":"172.21.15.93:6835","nonce":119699732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5170,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/1473474760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1473474760},{"type":"v1","addr":"172.21.15.81:6836","nonce":1473474760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:32:43.449426+0000","modified":"2023-11-08T16:32:46.504943+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5979},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5979":{"gid":5979,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/1446454901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":1446454901},{"type":"v1","addr":"172.21.15.81:6837","nonce":1446454901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:32:49.929 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 40 2023-11-08T16:32:49.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.928+0000 7f84cf494640 1 -- 172.21.15.81:0/2960917415 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84ac06ae70 msgr2=0x7f84ac06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.928+0000 7f84cf494640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84ac06ae70 0x7f84ac06d330 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f84c4036000 tx=0x7f84c4030000 comp rx=0 tx=0).stop 2023-11-08T16:32:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.928+0000 7f84cf494640 1 -- 172.21.15.81:0/2960917415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84c815a680 msgr2=0x7f84c81f2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.928+0000 7f84cf494640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84c815a680 0x7f84c81f2c60 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f84b800c500 tx=0x7f84b800c9c0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.929+0000 7f84cf494640 1 -- 172.21.15.81:0/2960917415 shutdown_connections 2023-11-08T16:32:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.929+0000 7f84cf494640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84ac06ae70 0x7f84ac06d330 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.929+0000 7f84cf494640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84c815ea60 0x7f84c81f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.929+0000 7f84cf494640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84c815a680 0x7f84c81f2c60 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.929+0000 7f84cf494640 1 --2- 172.21.15.81:0/2960917415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84c80ade50 0x7f84c814efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:49.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.929+0000 7f84cf494640 1 -- 172.21.15.81:0/2960917415 >> 172.21.15.81:0/2960917415 conn(0x7f84c80cb490 msgr2=0x7f84c814fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:49.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.929+0000 7f84cf494640 1 -- 172.21.15.81:0/2960917415 shutdown_connections 2023-11-08T16:32:49.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:49.929+0000 7f84cf494640 1 -- 172.21.15.81:0/2960917415 wait complete. 2023-11-08T16:32:50.350 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:32:43.449426+0000', 'modified': '2023-11-08T16:32:46.504943+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5979}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5979': {'gid': 5979, 'name': 'a', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.81:6837/1446454901', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6835', 'nonce': 1446454901}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 1446454901}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:32:50.350 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:32:50.351 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-08T16:32:50.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.574+0000 7f1d21856640 1 Processor -- start 2023-11-08T16:32:50.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.575+0000 7f1d21856640 1 -- start start 2023-11-08T16:32:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.576+0000 7f1d21856640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1d1c0c90f0 0x7f1d1c0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.576+0000 7f1d21856640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1d1c0b18f0 0x7f1d1c0b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.576+0000 7f1d21856640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d1c06cc60 con 0x7f1d1c15aeb0 2023-11-08T16:32:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.576+0000 7f1d21856640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c0cb3e0 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.576+0000 7f1d21856640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d1c0c9ac0 con 0x7f1d1c0b18f0 2023-11-08T16:32:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.577+0000 7f1d127fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1d1c0c90f0 0x7f1d1c0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.577+0000 7f1d127fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1d1c0c90f0 0x7f1d1c0c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43854/0 (socket says 172.21.15.81:43854) 2023-11-08T16:32:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.577+0000 7f1d127fc640 1 -- 172.21.15.81:0/4088049612 learned_addr learned my addr 172.21.15.81:0/4088049612 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.577+0000 7f1d1b7fe640 1 --2- 172.21.15.81:0/4088049612 >> v2:172.21.15.93:3300/0 conn(0x7f1d1c0b18f0 0x7f1d1c0b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.577+0000 7f1d127fc640 1 -- 172.21.15.81:0/4088049612 >> v2:172.21.15.93:3300/0 conn(0x7f1d1c0b18f0 msgr2=0x7f1d1c0b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.577+0000 7f1d127fc640 1 --2- 172.21.15.81:0/4088049612 >> v2:172.21.15.93:3300/0 conn(0x7f1d1c0b18f0 0x7f1d1c0b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.577+0000 7f1d127fc640 1 -- 172.21.15.81:0/4088049612 >> v1:172.21.15.93:6789/0 conn(0x7f1d1c15aeb0 legacy=0x7f1d1c0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.577+0000 7f1d127fc640 1 -- 172.21.15.81:0/4088049612 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c0cabb0 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.577+0000 7f1d1b7fe640 1 --2- 172.21.15.81:0/4088049612 >> v2:172.21.15.93:3300/0 conn(0x7f1d1c0b18f0 0x7f1d1c0b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:50.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.578+0000 7f1d127fc640 1 --2- 172.21.15.81:0/4088049612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d1c0c90f0 0x7f1d1c0c9580 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0c2570 tx=0x7f1d0402b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0f191136324687 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:50.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.578+0000 7f1d1a7fc640 1 -- 172.21.15.81:0/4088049612 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04031890 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.578+0000 7f1d1a7fc640 1 -- 172.21.15.81:0/4088049612 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d040319f0 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.579+0000 7f1d1a7fc640 1 -- 172.21.15.81:0/4088049612 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04031d80 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.579+0000 7f1d21856640 1 -- 172.21.15.81:0/4088049612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d1c0c90f0 msgr2=0x7f1d1c0c9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.579+0000 7f1d21856640 1 --2- 172.21.15.81:0/4088049612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d1c0c90f0 0x7f1d1c0c9580 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0c2570 tx=0x7f1d0402b930 comp rx=0 tx=0).stop 2023-11-08T16:32:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.579+0000 7f1d21856640 1 -- 172.21.15.81:0/4088049612 shutdown_connections 2023-11-08T16:32:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.579+0000 7f1d21856640 1 --2- 172.21.15.81:0/4088049612 >> v2:172.21.15.93:3300/0 conn(0x7f1d1c0b18f0 0x7f1d1c0b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.579+0000 7f1d21856640 1 --2- 172.21.15.81:0/4088049612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d1c0c90f0 0x7f1d1c0c9580 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.579+0000 7f1d21856640 1 -- 172.21.15.81:0/4088049612 >> 172.21.15.81:0/4088049612 conn(0x7f1d1c0ceaa0 msgr2=0x7f1d1c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:50.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.579+0000 7f1d21856640 1 -- 172.21.15.81:0/4088049612 shutdown_connections 2023-11-08T16:32:50.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.579+0000 7f1d21856640 1 -- 172.21.15.81:0/4088049612 wait complete. 2023-11-08T16:32:50.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.581+0000 7f1d21856640 1 Processor -- start 2023-11-08T16:32:50.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.581+0000 7f1d21856640 1 -- start start 2023-11-08T16:32:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.582+0000 7f1d21856640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1d1c0b18f0 0x7f1d1c14d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.582+0000 7f1d21856640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d1c0c90f0 0x7f1d1c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.582+0000 7f1d21856640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d1c15aeb0 0x7f1d1c0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.582+0000 7f1d21856640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c0ac920 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.582+0000 7f1d21856640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c1557b0 con 0x7f1d1c15aeb0 2023-11-08T16:32:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.582+0000 7f1d21856640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d1c154ba0 con 0x7f1d1c0b18f0 2023-11-08T16:32:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.582+0000 7f1d127fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d1c0c90f0 0x7f1d1c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.582+0000 7f1d127fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d1c0c90f0 0x7f1d1c0cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43858/0 (socket says 172.21.15.81:43858) 2023-11-08T16:32:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.582+0000 7f1d127fc640 1 -- 172.21.15.81:0/337749157 learned_addr learned my addr 172.21.15.81:0/337749157 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.583+0000 7f1d127fc640 1 -- 172.21.15.81:0/337749157 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1d1c0b18f0 msgr2=0x7f1d1c14d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.583+0000 7f1d127fc640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1d1c0b18f0 0x7f1d1c14d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.583+0000 7f1d1b7fe640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d1c15aeb0 0x7f1d1c0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.583+0000 7f1d127fc640 1 -- 172.21.15.81:0/337749157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d1c15aeb0 msgr2=0x7f1d1c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.583+0000 7f1d127fc640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d1c15aeb0 0x7f1d1c0cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.583+0000 7f1d127fc640 1 -- 172.21.15.81:0/337749157 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d040094d0 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.583+0000 7f1d127fc640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d1c0c90f0 0x7f1d1c0cd4f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1d04000c00 tx=0x7f1d040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.583+0000 7f1d18ff9640 1 -- 172.21.15.81:0/337749157 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d04031840 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.583+0000 7f1d18ff9640 1 -- 172.21.15.81:0/337749157 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d040443f0 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.583+0000 7f1d18ff9640 1 -- 172.21.15.81:0/337749157 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d040447a0 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.583+0000 7f1d21856640 1 -- 172.21.15.81:0/337749157 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d1c154e80 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.584+0000 7f1d21856640 1 -- 172.21.15.81:0/337749157 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d1c155440 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.586+0000 7f1d18ff9640 1 -- 172.21.15.81:0/337749157 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d0404d450 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.587+0000 7f1d18ff9640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1cfc06ae70 0x7f1cfc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.587+0000 7f1d18ff9640 1 -- 172.21.15.81:0/337749157 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d040abff0 con 0x7f1d1c0c90f0 2023-11-08T16:32:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.587+0000 7f1d1affd640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1cfc06ae70 0x7f1cfc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.587+0000 7f1d243ed740 1 Processor -- start 2023-11-08T16:32:50.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.588+0000 7f1d1affd640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1cfc06ae70 0x7f1cfc06d330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009690 tx=0x7f1d0c006b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:50.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.588+0000 7f1d243ed740 1 -- start start 2023-11-08T16:32:50.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.588+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411d0ec0 0x562e411a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411d3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411a4300 0x562e411d4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d243ed740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e411167e0 con 0x562e411a4300 2023-11-08T16:32:50.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d243ed740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e41113ed0 con 0x562e411d0ec0 2023-11-08T16:32:50.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d243ed740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e4110f7c0 con 0x562e411d12a0 2023-11-08T16:32:50.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d1b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411a4300 0x562e411d4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d1b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411a4300 0x562e411d4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43874/0 (socket says 172.21.15.81:43874) 2023-11-08T16:32:50.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d1b7fe640 1 -- 172.21.15.81:0/3473322283 learned_addr learned my addr 172.21.15.81:0/3473322283 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:50.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d1affd640 1 --2- 172.21.15.81:0/3473322283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411d0ec0 0x562e411a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d1b7fe640 1 -- 172.21.15.81:0/3473322283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 msgr2=0x562e411d3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d1b7fe640 1 --2- 172.21.15.81:0/3473322283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411d3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d1b7fe640 1 -- 172.21.15.81:0/3473322283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411d0ec0 msgr2=0x562e411a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d1b7fe640 1 --2- 172.21.15.81:0/3473322283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411d0ec0 0x562e411a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.589+0000 7f1d1b7fe640 1 -- 172.21.15.81:0/3473322283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e410e2b10 con 0x562e411a4300 2023-11-08T16:32:50.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.590+0000 7f1d1b7fe640 1 --2- 172.21.15.81:0/3473322283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411a4300 0x562e411d4190 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400d460 tx=0x7f1d1400d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fc3417398efa9c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:50.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.590+0000 7f1d117fa640 1 -- 172.21.15.81:0/3473322283 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400b740 con 0x562e411a4300 2023-11-08T16:32:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.590+0000 7f1d117fa640 1 -- 172.21.15.81:0/3473322283 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1400b8a0 con 0x562e411a4300 2023-11-08T16:32:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.590+0000 7f1d117fa640 1 -- 172.21.15.81:0/3473322283 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400bc30 con 0x562e411a4300 2023-11-08T16:32:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.590+0000 7f1d243ed740 1 -- 172.21.15.81:0/3473322283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411a4300 msgr2=0x562e411d4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.590+0000 7f1d243ed740 1 --2- 172.21.15.81:0/3473322283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411a4300 0x562e411d4190 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400d460 tx=0x7f1d1400d920 comp rx=0 tx=0).stop 2023-11-08T16:32:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.591+0000 7f1d243ed740 1 -- 172.21.15.81:0/3473322283 shutdown_connections 2023-11-08T16:32:50.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.591+0000 7f1d243ed740 1 --2- 172.21.15.81:0/3473322283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411a4300 0x562e411d4190 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.591+0000 7f1d243ed740 1 --2- 172.21.15.81:0/3473322283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411d3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.591+0000 7f1d243ed740 1 --2- 172.21.15.81:0/3473322283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411d0ec0 0x562e411a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.591+0000 7f1d243ed740 1 -- 172.21.15.81:0/3473322283 >> 172.21.15.81:0/3473322283 conn(0x562e411a60d0 msgr2=0x562e411eec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:50.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.591+0000 7f1d243ed740 1 -- 172.21.15.81:0/3473322283 shutdown_connections 2023-11-08T16:32:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.591+0000 7f1d243ed740 1 -- 172.21.15.81:0/3473322283 wait complete. 2023-11-08T16:32:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.591+0000 7f1d243ed740 1 -- start start 2023-11-08T16:32:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.592+0000 7f1d243ed740 1 Processor -- start 2023-11-08T16:32:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.592+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411badc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.592+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411e5770 0x562e411e5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d243ed740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e410acc40 con 0x562e411d0ec0 2023-11-08T16:32:50.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d243ed740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e410ac8e0 con 0x562e411e5770 2023-11-08T16:32:50.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d243ed740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e40e72f20 con 0x562e411d12a0 2023-11-08T16:32:50.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d243ed740 10 client.0 _ll_register_callbacks cb 0x562e410cecf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:32:50.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d1affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411badc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d1affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411badc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43878/0 (socket says 172.21.15.81:43878) 2023-11-08T16:32:50.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d1affd640 1 -- 172.21.15.81:0/1001988842 learned_addr learned my addr 172.21.15.81:0/1001988842 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:50.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d1affd640 1 -- 172.21.15.81:0/1001988842 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 msgr2=0x562e411d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d1affd640 1 --2- 172.21.15.81:0/1001988842 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d1affd640 1 -- 172.21.15.81:0/1001988842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411e5770 msgr2=0x562e411e5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d1affd640 1 --2- 172.21.15.81:0/1001988842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411e5770 0x562e411e5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d1affd640 1 -- 172.21.15.81:0/1001988842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d14008c30 con 0x562e411d0ec0 2023-11-08T16:32:50.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1d1affd640 1 --2- 172.21.15.81:0/1001988842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411badc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c013fc0 tx=0x7f1d0c004770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c596529409b8f5f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:50.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.593+0000 7f1cfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:32:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/1001988842 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c018050 con 0x562e411d0ec0 2023-11-08T16:32:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/1001988842 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d0c01d7d0 con 0x562e411d0ec0 2023-11-08T16:32:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1d243ed740 1 -- 172.21.15.81:0/1001988842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e410e3c80 con 0x562e411d0ec0 2023-11-08T16:32:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/1001988842 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d0c013430 con 0x562e411d0ec0 2023-11-08T16:32:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1d243ed740 10 client.6123 fetch_fsmap learned FSMap version 40 2023-11-08T16:32:50.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1d243ed740 1 -- 172.21.15.81:0/1001988842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x562e4110de40 con 0x562e411d0ec0 2023-11-08T16:32:50.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/1001988842 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562e410e3c80 con 0x562e411d0ec0 2023-11-08T16:32:50.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/1001988842 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1d0c01d990 con 0x562e411d0ec0 2023-11-08T16:32:50.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1d243ed740 10 client.6123 fetch_fsmap finished waiting for FSMap version 40 2023-11-08T16:32:50.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1d243ed740 10 client.6123 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T16:32:50.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1d243ed740 20 client.6123 populate_metadata read hostname 'smithi081' 2023-11-08T16:32:50.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1d243ed740 1 --2- 172.21.15.81:0/1001988842 >> [v2:172.21.15.81:6835/1446454901,v1:172.21.15.81:6837/1446454901] conn(0x562e411bc280 0x562e411bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1d243ed740 4 client.6123 mds_command: new command op to 5979 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:32:50.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.594+0000 7f1d243ed740 1 -- 172.21.15.81:0/1001988842 --> [v2:172.21.15.81:6835/1446454901,v1:172.21.15.81:6837/1446454901] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e411087a0 con 0x562e411bc280 2023-11-08T16:32:50.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.595+0000 7f1d127fc640 1 --2- 172.21.15.81:0/1001988842 >> [v2:172.21.15.81:6835/1446454901,v1:172.21.15.81:6837/1446454901] conn(0x562e411bc280 0x562e411bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.595+0000 7f1d127fc640 1 --2- 172.21.15.81:0/1001988842 >> [v2:172.21.15.81:6835/1446454901,v1:172.21.15.81:6837/1446454901] conn(0x562e411bc280 0x562e411bd5c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=493255eaea85689c server_cookie=161c5c27675b3240 in_seq=0 out_seq=0 2023-11-08T16:32:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.595+0000 7f1cfa7fc640 10 client.6123 ms_handle_connect on v2:172.21.15.81:6835/1446454901 2023-11-08T16:32:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/1001988842 <== mds.0 v2:172.21.15.81:6835/1446454901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e411087a0 con 0x562e411bc280 2023-11-08T16:32:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1cfa7fc640 10 client.6123 handle_command_reply: tid=0 2023-11-08T16:32:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1d243ed740 1 client.6123 shutdown 2023-11-08T16:32:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1d243ed740 1 -- 172.21.15.81:0/1001988842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 msgr2=0x562e411badc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1d243ed740 1 --2- 172.21.15.81:0/1001988842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411badc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c013fc0 tx=0x7f1d0c004770 comp rx=0 tx=0).stop 2023-11-08T16:32:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1d243ed740 1 -- 172.21.15.81:0/1001988842 shutdown_connections 2023-11-08T16:32:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1d243ed740 1 --2- 172.21.15.81:0/1001988842 >> [v2:172.21.15.81:6835/1446454901,v1:172.21.15.81:6837/1446454901] conn(0x562e411bc280 0x562e411bd5c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1d243ed740 1 --2- 172.21.15.81:0/1001988842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411e5770 0x562e411e5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1d243ed740 1 --2- 172.21.15.81:0/1001988842 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411d1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1d243ed740 1 --2- 172.21.15.81:0/1001988842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411badc0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1cfa7fc640 0 client.6123 ms_handle_reset on v2:172.21.15.81:6835/1446454901 2023-11-08T16:32:50.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1d243ed740 1 -- 172.21.15.81:0/1001988842 >> 172.21.15.81:0/1001988842 conn(0x562e411a4300 msgr2=0x562e411a7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:50.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1d243ed740 1 -- 172.21.15.81:0/1001988842 shutdown_connections 2023-11-08T16:32:50.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.596+0000 7f1d243ed740 1 -- 172.21.15.81:0/1001988842 wait complete. 2023-11-08T16:32:50.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.597+0000 7f1d243ed740 20 client.6123 trim_cache size 0 max 16384 2023-11-08T16:32:50.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.613+0000 7f1d243ed740 1 Processor -- start 2023-11-08T16:32:50.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.613+0000 7f1d243ed740 1 -- start start 2023-11-08T16:32:50.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411d7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411d3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411e5770 0x562e41253740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d243ed740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e411ef960 con 0x562e411d0ec0 2023-11-08T16:32:50.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d243ed740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e411cfe30 con 0x562e411e5770 2023-11-08T16:32:50.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d243ed740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e411d2c30 con 0x562e411d12a0 2023-11-08T16:32:50.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d1affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411d7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d1affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411d7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43882/0 (socket says 172.21.15.81:43882) 2023-11-08T16:32:50.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d1affd640 1 -- 172.21.15.81:0/609944633 learned_addr learned my addr 172.21.15.81:0/609944633 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:50.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d1affd640 1 -- 172.21.15.81:0/609944633 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 msgr2=0x562e411d3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d1affd640 1 --2- 172.21.15.81:0/609944633 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411d3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d1affd640 1 -- 172.21.15.81:0/609944633 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411e5770 msgr2=0x562e41253740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d1affd640 1 --2- 172.21.15.81:0/609944633 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411e5770 0x562e41253740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.614+0000 7f1d1affd640 1 -- 172.21.15.81:0/609944633 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e410abb90 con 0x562e411d0ec0 2023-11-08T16:32:50.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.615+0000 7f1d1affd640 1 --2- 172.21.15.81:0/609944633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411d7cb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c0134b0 tx=0x7f1d0c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a04856cf5d3d43 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:50.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.615+0000 7f1d11ffb640 1 -- 172.21.15.81:0/609944633 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c018050 con 0x562e411d0ec0 2023-11-08T16:32:50.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.615+0000 7f1d11ffb640 1 -- 172.21.15.81:0/609944633 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d0c033070 con 0x562e411d0ec0 2023-11-08T16:32:50.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.615+0000 7f1d11ffb640 1 -- 172.21.15.81:0/609944633 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c01d7f0 con 0x562e411d0ec0 2023-11-08T16:32:50.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.615+0000 7f1d243ed740 1 -- 172.21.15.81:0/609944633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 msgr2=0x562e411d7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.615+0000 7f1d243ed740 1 --2- 172.21.15.81:0/609944633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411d7cb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c0134b0 tx=0x7f1d0c01b000 comp rx=0 tx=0).stop 2023-11-08T16:32:50.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.616+0000 7f1d243ed740 1 -- 172.21.15.81:0/609944633 shutdown_connections 2023-11-08T16:32:50.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.616+0000 7f1d243ed740 1 --2- 172.21.15.81:0/609944633 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411e5770 0x562e41253740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.616+0000 7f1d243ed740 1 --2- 172.21.15.81:0/609944633 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411d3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.616+0000 7f1d243ed740 1 --2- 172.21.15.81:0/609944633 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411d0ec0 0x562e411d7cb0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.616+0000 7f1d243ed740 1 -- 172.21.15.81:0/609944633 >> 172.21.15.81:0/609944633 conn(0x562e411a4300 msgr2=0x562e412117b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:50.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.616+0000 7f1d243ed740 1 -- 172.21.15.81:0/609944633 shutdown_connections 2023-11-08T16:32:50.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.616+0000 7f1d243ed740 1 -- 172.21.15.81:0/609944633 wait complete. 2023-11-08T16:32:50.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.616+0000 7f1d243ed740 1 -- start start 2023-11-08T16:32:50.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.616+0000 7f1d243ed740 1 Processor -- start 2023-11-08T16:32:50.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.617+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411d0ec0 0x562e411e4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.617+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.617+0000 7f1d243ed740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411e5770 0x562e4123f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.617+0000 7f1d243ed740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e411e5530 con 0x562e411e5770 2023-11-08T16:32:50.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.617+0000 7f1d243ed740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e41240fe0 con 0x562e411d0ec0 2023-11-08T16:32:50.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.617+0000 7f1d243ed740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e411bb010 con 0x562e411d12a0 2023-11-08T16:32:50.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d243ed740 10 client.0 _ll_register_callbacks cb 0x7f1d0c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:32:50.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d1b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411e5770 0x562e4123f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d1b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411e5770 0x562e4123f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43884/0 (socket says 172.21.15.81:43884) 2023-11-08T16:32:50.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d1b7fe640 1 -- 172.21.15.81:0/249595822 learned_addr learned my addr 172.21.15.81:0/249595822 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:50.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d1affd640 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411d0ec0 0x562e411e4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d127fc640 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d1b7fe640 1 -- 172.21.15.81:0/249595822 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 msgr2=0x562e411fd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d1b7fe640 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411fd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d1b7fe640 1 -- 172.21.15.81:0/249595822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411d0ec0 msgr2=0x562e411e4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d1b7fe640 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411d0ec0 0x562e411e4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d1b7fe640 1 -- 172.21.15.81:0/249595822 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e4110de40 con 0x562e411e5770 2023-11-08T16:32:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.618+0000 7f1d1b7fe640 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411e5770 0x562e4123f150 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400dfd0 tx=0x7f1d14008f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ce8fd9eb54db4f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1cfa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:32:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/249595822 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1400b740 con 0x562e411e5770 2023-11-08T16:32:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/249595822 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d14021860 con 0x562e411e5770 2023-11-08T16:32:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1d243ed740 1 -- 172.21.15.81:0/249595822 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d1400b8a0 con 0x562e411e5770 2023-11-08T16:32:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/249595822 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d14008630 con 0x562e411e5770 2023-11-08T16:32:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1d243ed740 10 client.6129 fetch_fsmap learned FSMap version 40 2023-11-08T16:32:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1d243ed740 1 -- 172.21.15.81:0/249595822 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x562e410d1180 con 0x562e411e5770 2023-11-08T16:32:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/249595822 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1400b8a0 con 0x562e411e5770 2023-11-08T16:32:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/249595822 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1d14008040 con 0x562e411e5770 2023-11-08T16:32:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1d243ed740 10 client.6129 fetch_fsmap finished waiting for FSMap version 40 2023-11-08T16:32:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1d243ed740 10 client.6129 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T16:32:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1d243ed740 20 client.6129 populate_metadata read hostname 'smithi081' 2023-11-08T16:32:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1d243ed740 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.81:6835/1446454901,v1:172.21.15.81:6837/1446454901] conn(0x562e41240020 0x562e41240400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1d243ed740 4 client.6129 mds_command: new command op to 5979 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:32:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.619+0000 7f1d243ed740 1 -- 172.21.15.81:0/249595822 --> [v2:172.21.15.81:6835/1446454901,v1:172.21.15.81:6837/1446454901] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562e411084c0 con 0x562e41240020 2023-11-08T16:32:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.620+0000 7f1d1affd640 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.81:6835/1446454901,v1:172.21.15.81:6837/1446454901] conn(0x562e41240020 0x562e41240400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.620+0000 7f1d1affd640 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.81:6835/1446454901,v1:172.21.15.81:6837/1446454901] conn(0x562e41240020 0x562e41240400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f518db0693f4a6 server_cookie=3d41fd6bf9c2475a in_seq=0 out_seq=0 2023-11-08T16:32:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.620+0000 7f1cfa7fc640 10 client.6129 ms_handle_connect on v2:172.21.15.81:6835/1446454901 2023-11-08T16:32:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.620+0000 7f1cfa7fc640 1 -- 172.21.15.81:0/249595822 <== mds.0 v2:172.21.15.81:6835/1446454901 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x562e411084c0 con 0x562e41240020 2023-11-08T16:32:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.620+0000 7f1cfa7fc640 10 client.6129 handle_command_reply: tid=0 2023-11-08T16:32:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.620+0000 7f1d243ed740 1 client.6129 shutdown 2023-11-08T16:32:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1d243ed740 1 -- 172.21.15.81:0/249595822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411e5770 msgr2=0x562e4123f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1d243ed740 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411e5770 0x562e4123f150 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1d1400dfd0 tx=0x7f1d14008f00 comp rx=0 tx=0).stop 2023-11-08T16:32:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1d243ed740 1 -- 172.21.15.81:0/249595822 shutdown_connections 2023-11-08T16:32:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1d243ed740 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.81:6835/1446454901,v1:172.21.15.81:6837/1446454901] conn(0x562e41240020 0x562e41240400 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1d243ed740 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562e411e5770 0x562e4123f150 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1d243ed740 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562e411d12a0 0x562e411fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1d243ed740 1 --2- 172.21.15.81:0/249595822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562e411d0ec0 0x562e411e4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1cfa7fc640 0 client.6129 ms_handle_reset on v2:172.21.15.81:6835/1446454901 2023-11-08T16:32:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1d243ed740 1 -- 172.21.15.81:0/249595822 >> 172.21.15.81:0/249595822 conn(0x562e411a4300 msgr2=0x562e411a9fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:50.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1d243ed740 1 -- 172.21.15.81:0/249595822 shutdown_connections 2023-11-08T16:32:50.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1d243ed740 1 -- 172.21.15.81:0/249595822 wait complete. 2023-11-08T16:32:50.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.621+0000 7f1d243ed740 20 client.6129 trim_cache size 0 max 16384 2023-11-08T16:32:50.638 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:32:50.639 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:32:50.639 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:32:50.639 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 5979, 2023-11-08T16:32:50.639 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:32:50.640 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:32:50.640 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:32:50.640 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 6.6337756929999996, 2023-11-08T16:32:50.640 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 40, 2023-11-08T16:32:50.640 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 57, 2023-11-08T16:32:50.640 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 57, 2023-11-08T16:32:50.641 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 13.044713646 2023-11-08T16:32:50.641 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:32:50.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 -- 172.21.15.81:0/337749157 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1cfc06ae70 msgr2=0x7f1cfc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1cfc06ae70 0x7f1cfc06d330 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009690 tx=0x7f1d0c006b90 comp rx=0 tx=0).stop 2023-11-08T16:32:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 -- 172.21.15.81:0/337749157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d1c0c90f0 msgr2=0x7f1d1c0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d1c0c90f0 0x7f1d1c0cd4f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f1d04000c00 tx=0x7f1d040046c0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 -- 172.21.15.81:0/337749157 shutdown_connections 2023-11-08T16:32:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1cfc06ae70 0x7f1cfc06d330 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d1c15aeb0 0x7f1d1c0cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d1c0c90f0 0x7f1d1c0cd4f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 --2- 172.21.15.81:0/337749157 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1d1c0b18f0 0x7f1d1c14d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 -- 172.21.15.81:0/337749157 >> 172.21.15.81:0/337749157 conn(0x7f1d1c0ceaa0 msgr2=0x7f1d1c0c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 -- 172.21.15.81:0/337749157 shutdown_connections 2023-11-08T16:32:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.622+0000 7f1d21856640 1 -- 172.21.15.81:0/337749157 wait complete. 2023-11-08T16:32:50.645 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:32:50.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.846+0000 7fbb6498a640 1 Processor -- start 2023-11-08T16:32:50.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.846+0000 7fbb6498a640 1 -- start start 2023-11-08T16:32:50.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.847+0000 7fbb6498a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb600adcd0 0x7fbb600ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.847+0000 7fbb6498a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbb600cc5e0 0x7fbb600cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.847+0000 7fbb6498a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600ccfb0 0x7fbb6015db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.847+0000 7fbb6498a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb600c9a70 con 0x7fbb600adcd0 2023-11-08T16:32:50.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.847+0000 7fbb6498a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb600cc3e0 con 0x7fbb600cc5e0 2023-11-08T16:32:50.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.847+0000 7fbb6498a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6006d070 con 0x7fbb600ccfb0 2023-11-08T16:32:50.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.847+0000 7fbb5e59c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb600adcd0 0x7fbb600ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.848+0000 7fbb5e59c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb600adcd0 0x7fbb600ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43898/0 (socket says 172.21.15.81:43898) 2023-11-08T16:32:50.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.848+0000 7fbb5e59c640 1 -- 172.21.15.81:0/2954802410 learned_addr learned my addr 172.21.15.81:0/2954802410 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:50.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.848+0000 7fbb5e59c640 1 -- 172.21.15.81:0/2954802410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600ccfb0 msgr2=0x7fbb6015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.848+0000 7fbb5e59c640 1 --2- 172.21.15.81:0/2954802410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600ccfb0 0x7fbb6015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.848+0000 7fbb5e59c640 1 -- 172.21.15.81:0/2954802410 >> v2:172.21.15.93:3300/0 conn(0x7fbb600cc5e0 msgr2=0x7fbb600cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:50.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.848+0000 7fbb5e59c640 1 --2- 172.21.15.81:0/2954802410 >> v2:172.21.15.93:3300/0 conn(0x7fbb600cc5e0 0x7fbb600cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.848+0000 7fbb5e59c640 1 -- 172.21.15.81:0/2954802410 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb6006d350 con 0x7fbb600adcd0 2023-11-08T16:32:50.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.849+0000 7fbb5e59c640 1 --2- 172.21.15.81:0/2954802410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600adcd0 0x7fbb600ca1e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbb600cc0d0 tx=0x7fbb5002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f303da1437b952d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:50.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.849+0000 7fbb5d59a640 1 -- 172.21.15.81:0/2954802410 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50031870 con 0x7fbb600adcd0 2023-11-08T16:32:50.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.849+0000 7fbb5d59a640 1 -- 172.21.15.81:0/2954802410 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb500319d0 con 0x7fbb600adcd0 2023-11-08T16:32:50.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.849+0000 7fbb5d59a640 1 -- 172.21.15.81:0/2954802410 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50031d60 con 0x7fbb600adcd0 2023-11-08T16:32:50.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.850+0000 7fbb6498a640 1 -- 172.21.15.81:0/2954802410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600adcd0 msgr2=0x7fbb600ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.850+0000 7fbb6498a640 1 --2- 172.21.15.81:0/2954802410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600adcd0 0x7fbb600ca1e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fbb600cc0d0 tx=0x7fbb5002b910 comp rx=0 tx=0).stop 2023-11-08T16:32:50.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.850+0000 7fbb6498a640 1 -- 172.21.15.81:0/2954802410 shutdown_connections 2023-11-08T16:32:50.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.850+0000 7fbb6498a640 1 --2- 172.21.15.81:0/2954802410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600ccfb0 0x7fbb6015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.850+0000 7fbb6498a640 1 --2- 172.21.15.81:0/2954802410 >> v2:172.21.15.93:3300/0 conn(0x7fbb600cc5e0 0x7fbb600cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.850+0000 7fbb6498a640 1 --2- 172.21.15.81:0/2954802410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600adcd0 0x7fbb600ca1e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.850+0000 7fbb6498a640 1 -- 172.21.15.81:0/2954802410 >> 172.21.15.81:0/2954802410 conn(0x7fbb600cb490 msgr2=0x7fbb600cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:50.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.850+0000 7fbb6498a640 1 -- 172.21.15.81:0/2954802410 shutdown_connections 2023-11-08T16:32:50.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.850+0000 7fbb6498a640 1 -- 172.21.15.81:0/2954802410 wait complete. 2023-11-08T16:32:50.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.852+0000 7fbb6498a640 1 Processor -- start 2023-11-08T16:32:50.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.852+0000 7fbb6498a640 1 -- start start 2023-11-08T16:32:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb6498a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600adcd0 0x7fbb60157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb6498a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb600cc5e0 0x7fbb6015b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb6498a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600ccfb0 0x7fbb60147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb6498a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6015b660 con 0x7fbb600adcd0 2023-11-08T16:32:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb6498a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb60156670 con 0x7fbb600cc5e0 2023-11-08T16:32:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb6498a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb60156950 con 0x7fbb600ccfb0 2023-11-08T16:32:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb5e59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600adcd0 0x7fbb60157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb5e59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600adcd0 0x7fbb60157710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43904/0 (socket says 172.21.15.81:43904) 2023-11-08T16:32:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb5e59c640 1 -- 172.21.15.81:0/2888685728 learned_addr learned my addr 172.21.15.81:0/2888685728 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:50.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb5e59c640 1 -- 172.21.15.81:0/2888685728 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600ccfb0 msgr2=0x7fbb60147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb5e59c640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600ccfb0 0x7fbb60147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb5e59c640 1 -- 172.21.15.81:0/2888685728 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb600cc5e0 msgr2=0x7fbb6015b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:50.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb5e59c640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb600cc5e0 0x7fbb6015b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:50.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.853+0000 7fbb5e59c640 1 -- 172.21.15.81:0/2888685728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500094d0 con 0x7fbb600adcd0 2023-11-08T16:32:50.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.854+0000 7fbb5e59c640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600adcd0 0x7fbb60157710 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbb50000c00 tx=0x7fbb500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:50.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.854+0000 7fbb4effd640 1 -- 172.21.15.81:0/2888685728 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb50031840 con 0x7fbb600adcd0 2023-11-08T16:32:50.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.854+0000 7fbb4effd640 1 -- 172.21.15.81:0/2888685728 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb500443f0 con 0x7fbb600adcd0 2023-11-08T16:32:50.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.854+0000 7fbb6498a640 1 -- 172.21.15.81:0/2888685728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb60156c30 con 0x7fbb600adcd0 2023-11-08T16:32:50.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.854+0000 7fbb4effd640 1 -- 172.21.15.81:0/2888685728 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb500447a0 con 0x7fbb600adcd0 2023-11-08T16:32:50.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.854+0000 7fbb6498a640 1 -- 172.21.15.81:0/2888685728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb601571f0 con 0x7fbb600adcd0 2023-11-08T16:32:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.855+0000 7fbb6498a640 1 -- 172.21.15.81:0/2888685728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb60072170 con 0x7fbb600adcd0 2023-11-08T16:32:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.856+0000 7fbb4effd640 1 -- 172.21.15.81:0/2888685728 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb5004d450 con 0x7fbb600adcd0 2023-11-08T16:32:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.857+0000 7fbb4effd640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb4006ae70 0x7fbb4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.857+0000 7fbb4effd640 1 -- 172.21.15.81:0/2888685728 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb5002d040 con 0x7fbb600adcd0 2023-11-08T16:32:50.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.857+0000 7fbb5dd9b640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb4006ae70 0x7fbb4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:50.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.858+0000 7fbb5dd9b640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb4006ae70 0x7fbb4006d330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb6015b070 tx=0x7fbb54006bf0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:50.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:50.861+0000 7fbb4effd640 1 -- 172.21.15.81:0/2888685728 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb50076e20 con 0x7fbb600adcd0 2023-11-08T16:32:51.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.077+0000 7fbb6498a640 1 -- 172.21.15.81:0/2888685728 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbb601573e0 con 0x7fbb600adcd0 2023-11-08T16:32:51.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.078+0000 7fbb4effd640 1 -- 172.21.15.81:0/2888685728 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbb5007a440 con 0x7fbb600adcd0 2023-11-08T16:32:51.080 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T16:32:51.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.080+0000 7fbb6498a640 1 -- 172.21.15.81:0/2888685728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb4006ae70 msgr2=0x7fbb4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.080+0000 7fbb6498a640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb4006ae70 0x7fbb4006d330 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fbb6015b070 tx=0x7fbb54006bf0 comp rx=0 tx=0).stop 2023-11-08T16:32:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.080+0000 7fbb6498a640 1 -- 172.21.15.81:0/2888685728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600adcd0 msgr2=0x7fbb60157710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.080+0000 7fbb6498a640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600adcd0 0x7fbb60157710 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fbb50000c00 tx=0x7fbb500046c0 comp rx=0 tx=0).stop 2023-11-08T16:32:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.081+0000 7fbb6498a640 1 -- 172.21.15.81:0/2888685728 shutdown_connections 2023-11-08T16:32:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.081+0000 7fbb6498a640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb4006ae70 0x7fbb4006d330 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.081+0000 7fbb6498a640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb600ccfb0 0x7fbb60147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.081+0000 7fbb6498a640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb600cc5e0 0x7fbb6015b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.081+0000 7fbb6498a640 1 --2- 172.21.15.81:0/2888685728 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb600adcd0 0x7fbb60157710 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.081+0000 7fbb6498a640 1 -- 172.21.15.81:0/2888685728 >> 172.21.15.81:0/2888685728 conn(0x7fbb600cb490 msgr2=0x7fbb600c9250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.081+0000 7fbb6498a640 1 -- 172.21.15.81:0/2888685728 shutdown_connections 2023-11-08T16:32:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:51.081+0000 7fbb6498a640 1 -- 172.21.15.81:0/2888685728 wait complete. 2023-11-08T16:32:51.100 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:32:51.100 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:32:51.101 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:32:51.101 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:32:51.101 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:32:51.102 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:32:51.102 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:32:51.103 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:32:51.103 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:32:51.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:51.104 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:32:51.154 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:32:51.154 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:32:51.155 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:32:51.155 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:32:51.155 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:32:51.155 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:32:51.155 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:32:51.156 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:32:51.156 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:32:51.156 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:32:51.156 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:32:51.157 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:32:51.157 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:32:51.157 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:32:51.157 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:32:51.157 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:32:51.158 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:32:51.158 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42138sec preferred_lft 42138sec 2023-11-08T16:32:51.158 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:32:51.158 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:32:51.158 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:32:51.159 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff 2023-11-08T16:32:51.159 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:32:51.159 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:32:51.159 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:32:51.159 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:32:51.160 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:32:51.160 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:32:51.160 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:32:51.160 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:32:51.160 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:32:51.161 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:32:51.161 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::2cac:17ff:fe64:67da/64 scope link 2023-11-08T16:32:51.161 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:32:51.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:51.163 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:32:51.250 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:32:51.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:51.255 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:32:51.305 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:32:51.306 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:32:51.306 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:32:51.306 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:32:51.306 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:32:51.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:51.307 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:32:51.308 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:32:51.308 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:32:51.308 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:32:51.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:32:51.309 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:32:51.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:51.491 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:32:51.545 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:32:51.545 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:32:51.545 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:32:51.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:51.546 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:32:51.635 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:32:51.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:32:51.636 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:32:51.690 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:32:51.691 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:32:51.691 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:32:51.740 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:32:51.888 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:32:51.946 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:32:51.946 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:32:51.946 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:32:51.947 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:32:51.947 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:32:51.947 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:32:51.947 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:32:51.947 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:32:51.948 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:32:51.948 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:32:51.948 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:32:51.948 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:32:51.948 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:32:51.949 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:32:51.949 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:32:51.949 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:32:51.949 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:32:51.949 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:32:51.950 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:32:51.950 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:32:51.950 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:32:51.950 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:32:51.950 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:32:51.951 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:32:51.951 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:32:51.951 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:32:51.951 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:32:51.951 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:32:51.952 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:32:51.952 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:32:51.952 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:32:51.952 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:32:51.952 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:32:51.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:51.954 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:32:52.004 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:32:52.005 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:32:52.010 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:32:52.087 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:32:52.129 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:32:52.129 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:32:52.129 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:32:52.129 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:32:52.130 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:32:52.130 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:32:52.130 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:32:52.130 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:32:52.131 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:32:52.131 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:32:52.131 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:32:52.131 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:32:52.131 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:32:52.131 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:32:52.132 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:32:52.132 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:32:52.132 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:32:52.132 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:32:52.132 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:32:52.133 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:32:52.133 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:32:52.133 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:32:52.133 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:32:52.133 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:32:52.134 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:32:52.134 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:32:52.134 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:32:52.134 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:32:52.135 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:32:52.135 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:32:52.135 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:32:52.135 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:32:52.135 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:32:52.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:52.137 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:32:52.197 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:32:52.196+0000 7f2fdf24b480 -1 init, newargv = 0x5643561071a0 newargc=15 2023-11-08T16:32:52.198 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[106765]: starting ceph client 2023-11-08T16:32:52.205 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[106765]: starting fuse 2023-11-08T16:32:52.226 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:32:52.227 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:32:52.227 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:32:52.228 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:32:52.228 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:32:52.228 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:32:52.229 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:32:52.229 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:32:52.230 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:32:52.230 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:32:52.230 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:32:52.231 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:32:52.231 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:32:52.231 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:32:52.232 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:32:52.232 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:32:52.232 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:32:52.233 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:32:52.233 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:32:52.234 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:32:52.234 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:32:52.234 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:32:52.235 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:32:52.235 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:32:52.235 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:32:52.236 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:32:52.236 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:32:52.236 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:32:52.237 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:32:52.428 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.106765.asok 2023-11-08T16:32:52.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:32:52.444 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106765.asok 2023-11-08T16:32:52.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:52.445 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106765.asok status 2023-11-08T16:32:52.682 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:32:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:32:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:32:52.683 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:32:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:32:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:32:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:32:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "106765", 2023-11-08T16:32:52.684 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:32:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:32:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:32:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 6141, 2023-11-08T16:32:52.685 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:32:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:32:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:32:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 6141 2023-11-08T16:32:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:52.686 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:32:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:32:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:32:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 695798309 2023-11-08T16:32:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:32:52.687 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:32:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:32:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:32:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 695798309 2023-11-08T16:32:52.688 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:32:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.6141 192.168.0.1:0/695798309", 2023-11-08T16:32:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/695798309", 2023-11-08T16:32:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:32:52.689 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 40, 2023-11-08T16:32:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 57, 2023-11-08T16:32:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:32:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:32:52.690 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:32:52.691 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:32:52.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:52.698 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:32:52.699 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:32:52.752 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:32:52.753 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:32:52.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:52.753 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:32:52.847 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-11-08T16:32:53.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.068+0000 7f6e8671d640 1 Processor -- start 2023-11-08T16:32:53.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.068+0000 7f6e8671d640 1 -- start start 2023-11-08T16:32:53.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.069+0000 7f6e8671d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6e80071890 0x7f6e8015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:53.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.069+0000 7f6e8671d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e8015afd0 0x7f6e8015ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:53.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.070+0000 7f6e8671d640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e800b09f0 con 0x7f6e800714b0 2023-11-08T16:32:53.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.070+0000 7f6e8671d640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8006b250 con 0x7f6e80071890 2023-11-08T16:32:53.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.070+0000 7f6e8671d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e800ca360 con 0x7f6e8015afd0 2023-11-08T16:32:53.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.070+0000 7f6e84cba640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e8015afd0 0x7f6e8015ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:53.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.070+0000 7f6e84cba640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e8015afd0 0x7f6e8015ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:36620/0 (socket says 172.21.15.81:36620) 2023-11-08T16:32:53.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.070+0000 7f6e84cba640 1 -- 172.21.15.81:0/1743553048 learned_addr learned my addr 172.21.15.81:0/1743553048 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:53.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.070+0000 7f6e77fff640 1 --2- 172.21.15.81:0/1743553048 >> v2:172.21.15.93:3300/0 conn(0x7f6e80071890 0x7f6e8015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:53.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.071+0000 7f6e84cba640 1 -- 172.21.15.81:0/1743553048 >> v2:172.21.15.93:3300/0 conn(0x7f6e80071890 msgr2=0x7f6e8015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:53.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.071+0000 7f6e84cba640 1 --2- 172.21.15.81:0/1743553048 >> v2:172.21.15.93:3300/0 conn(0x7f6e80071890 0x7f6e8015e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:53.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.071+0000 7f6e84cba640 1 -- 172.21.15.81:0/1743553048 >> v1:172.21.15.93:6789/0 conn(0x7f6e800714b0 legacy=0x7f6e8015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:53.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.071+0000 7f6e84cba640 1 -- 172.21.15.81:0/1743553048 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e800ca640 con 0x7f6e8015afd0 2023-11-08T16:32:53.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.071+0000 7f6e77fff640 1 --2- 172.21.15.81:0/1743553048 >> v2:172.21.15.93:3300/0 conn(0x7f6e80071890 0x7f6e8015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:32:53.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.072+0000 7f6e84cba640 1 --2- 172.21.15.81:0/1743553048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e8015afd0 0x7f6e8015ef10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009e50 tx=0x7f6e7802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c4372feaf585c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:53.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.076+0000 7f6e7f7fe640 1 -- 172.21.15.81:0/1743553048 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7802cb00 con 0x7f6e8015afd0 2023-11-08T16:32:53.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.076+0000 7f6e7f7fe640 1 -- 172.21.15.81:0/1743553048 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7802cc60 con 0x7f6e8015afd0 2023-11-08T16:32:53.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.076+0000 7f6e7f7fe640 1 -- 172.21.15.81:0/1743553048 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e780343f0 con 0x7f6e8015afd0 2023-11-08T16:32:53.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.076+0000 7f6e8671d640 1 -- 172.21.15.81:0/1743553048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e8015afd0 msgr2=0x7f6e8015ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:53.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.076+0000 7f6e8671d640 1 --2- 172.21.15.81:0/1743553048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e8015afd0 0x7f6e8015ef10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009e50 tx=0x7f6e7802bf60 comp rx=0 tx=0).stop 2023-11-08T16:32:53.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.077+0000 7f6e8671d640 1 -- 172.21.15.81:0/1743553048 shutdown_connections 2023-11-08T16:32:53.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.077+0000 7f6e8671d640 1 --2- 172.21.15.81:0/1743553048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e8015afd0 0x7f6e8015ef10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:53.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.077+0000 7f6e8671d640 1 --2- 172.21.15.81:0/1743553048 >> v2:172.21.15.93:3300/0 conn(0x7f6e80071890 0x7f6e8015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:53.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.077+0000 7f6e8671d640 1 -- 172.21.15.81:0/1743553048 >> 172.21.15.81:0/1743553048 conn(0x7f6e800cb4e0 msgr2=0x7f6e800cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:53.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.077+0000 7f6e8671d640 1 -- 172.21.15.81:0/1743553048 shutdown_connections 2023-11-08T16:32:53.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.077+0000 7f6e8671d640 1 -- 172.21.15.81:0/1743553048 wait complete. 2023-11-08T16:32:53.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.079+0000 7f6e8671d640 1 Processor -- start 2023-11-08T16:32:53.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.079+0000 7f6e8671d640 1 -- start start 2023-11-08T16:32:53.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e8671d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e800714b0 0x7f6e800b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:53.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e8671d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e80071890 0x7f6e8014fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:53.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e8671d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e8015afd0 0x7f6e8015bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:53.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e8671d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e800b2b20 con 0x7f6e8015afd0 2023-11-08T16:32:53.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e8671d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e80150190 con 0x7f6e80071890 2023-11-08T16:32:53.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e8671d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e800c5bd0 con 0x7f6e800714b0 2023-11-08T16:32:53.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e84cba640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e8015afd0 0x7f6e8015bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:53.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e84cba640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e8015afd0 0x7f6e8015bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44180/0 (socket says 172.21.15.81:44180) 2023-11-08T16:32:53.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e84cba640 1 -- 172.21.15.81:0/2286082902 learned_addr learned my addr 172.21.15.81:0/2286082902 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:53.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e77fff640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e80071890 0x7f6e8014fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:53.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e7ffff640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e800714b0 0x7f6e800b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:53.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e84cba640 1 -- 172.21.15.81:0/2286082902 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e800714b0 msgr2=0x7f6e800b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:53.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e84cba640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e800714b0 0x7f6e800b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:53.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e84cba640 1 -- 172.21.15.81:0/2286082902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e80071890 msgr2=0x7f6e8014fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:53.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e84cba640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e80071890 0x7f6e8014fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:53.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e84cba640 1 -- 172.21.15.81:0/2286082902 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68009400 con 0x7f6e8015afd0 2023-11-08T16:32:53.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e77fff640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e80071890 0x7f6e8014fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:53.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.080+0000 7f6e7ffff640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e800714b0 0x7f6e800b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:53.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.081+0000 7f6e84cba640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e8015afd0 0x7f6e8015bec0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009e50 tx=0x7f6e78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:53.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.081+0000 7f6e7d7fa640 1 -- 172.21.15.81:0/2286082902 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78040670 con 0x7f6e8015afd0 2023-11-08T16:32:53.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.081+0000 7f6e7d7fa640 1 -- 172.21.15.81:0/2286082902 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e78040810 con 0x7f6e8015afd0 2023-11-08T16:32:53.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.081+0000 7f6e7d7fa640 1 -- 172.21.15.81:0/2286082902 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78040bc0 con 0x7f6e8015afd0 2023-11-08T16:32:53.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.081+0000 7f6e8671d640 1 -- 172.21.15.81:0/2286082902 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e78009b30 con 0x7f6e8015afd0 2023-11-08T16:32:53.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.081+0000 7f6e8671d640 1 -- 172.21.15.81:0/2286082902 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e800c6190 con 0x7f6e8015afd0 2023-11-08T16:32:53.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.082+0000 7f6e8671d640 1 -- 172.21.15.81:0/2286082902 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e800743f0 con 0x7f6e8015afd0 2023-11-08T16:32:53.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.083+0000 7f6e7d7fa640 1 -- 172.21.15.81:0/2286082902 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e78040d20 con 0x7f6e8015afd0 2023-11-08T16:32:53.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.084+0000 7f6e7d7fa640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006ae70 0x7f6e6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:53.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.084+0000 7f6e7d7fa640 1 -- 172.21.15.81:0/2286082902 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e780ac680 con 0x7f6e8015afd0 2023-11-08T16:32:53.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.084+0000 7f6e7ffff640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006ae70 0x7f6e6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:53.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.085+0000 7f6e7ffff640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006ae70 0x7f6e6006d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6e70000f60 tx=0x7f6e70009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:53.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.089+0000 7f6e7d7fa640 1 -- 172.21.15.81:0/2286082902 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e780777b0 con 0x7f6e8015afd0 2023-11-08T16:32:53.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:53.282+0000 7f6e8671d640 1 -- 172.21.15.81:0/2286082902 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f6e80156240 con 0x7f6e8015afd0 2023-11-08T16:32:54.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.088+0000 7f6e7d7fa640 1 -- 172.21.15.81:0/2286082902 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v58) v1 ==== 142+0+0 (secure 0 0 0) 0x7f6e78009510 con 0x7f6e8015afd0 2023-11-08T16:32:54.090 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data_extra' created 2023-11-08T16:32:54.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.092+0000 7f6e8671d640 1 -- 172.21.15.81:0/2286082902 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006ae70 msgr2=0x7f6e6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:54.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.092+0000 7f6e8671d640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006ae70 0x7f6e6006d330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f6e70000f60 tx=0x7f6e70009040 comp rx=0 tx=0).stop 2023-11-08T16:32:54.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.092+0000 7f6e8671d640 1 -- 172.21.15.81:0/2286082902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e8015afd0 msgr2=0x7f6e8015bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:54.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.092+0000 7f6e8671d640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e8015afd0 0x7f6e8015bec0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009e50 tx=0x7f6e78004a40 comp rx=0 tx=0).stop 2023-11-08T16:32:54.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.092+0000 7f6e8671d640 1 -- 172.21.15.81:0/2286082902 shutdown_connections 2023-11-08T16:32:54.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.092+0000 7f6e8671d640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006ae70 0x7f6e6006d330 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:54.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.092+0000 7f6e8671d640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e8015afd0 0x7f6e8015bec0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:54.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.092+0000 7f6e8671d640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e80071890 0x7f6e8014fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:54.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.092+0000 7f6e8671d640 1 --2- 172.21.15.81:0/2286082902 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e800714b0 0x7f6e800b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:54.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.092+0000 7f6e8671d640 1 -- 172.21.15.81:0/2286082902 >> 172.21.15.81:0/2286082902 conn(0x7f6e800cb4e0 msgr2=0x7f6e80158d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:54.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.092+0000 7f6e8671d640 1 -- 172.21.15.81:0/2286082902 shutdown_connections 2023-11-08T16:32:54.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.093+0000 7f6e8671d640 1 -- 172.21.15.81:0/2286082902 wait complete. 2023-11-08T16:32:54.121 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-11-08T16:32:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.314+0000 7f84f4075640 1 Processor -- start 2023-11-08T16:32:54.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.315+0000 7f84f4075640 1 -- start start 2023-11-08T16:32:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.316+0000 7f84f4075640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f84ec15e440 0x7f84ec071180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:54.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.316+0000 7f84f4075640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f84ec0cacc0 0x7f84ec06b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:54.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.316+0000 7f84f4075640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84ec0b18b0 con 0x7f84ec0716c0 2023-11-08T16:32:54.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.316+0000 7f84f4075640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84ec15e820 con 0x7f84ec0cacc0 2023-11-08T16:32:54.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.316+0000 7f84f4075640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84ec0c90b0 con 0x7f84ec15e440 2023-11-08T16:32:54.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.317+0000 7f84f1610640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f84ec0716c0 0x7f84ec0ca5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48728/0 (socket says 172.21.15.81:48728) 2023-11-08T16:32:54.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.317+0000 7f84f1610640 1 -- 172.21.15.81:0/4059107640 learned_addr learned my addr 172.21.15.81:0/4059107640 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:54.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.317+0000 7f84f2612640 1 --2- 172.21.15.81:0/4059107640 >> v2:172.21.15.81:3300/0 conn(0x7f84ec0cacc0 0x7f84ec06b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:54.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.317+0000 7f84f1e11640 1 --2- 172.21.15.81:0/4059107640 >> v2:172.21.15.93:3300/0 conn(0x7f84ec15e440 0x7f84ec071180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:54.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.317+0000 7f84f0e0f640 1 -- 172.21.15.81:0/4059107640 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4273564209 0 0) 0x7f84ec0b18b0 con 0x7f84ec0716c0 2023-11-08T16:32:54.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.317+0000 7f84f0e0f640 1 -- 172.21.15.81:0/4059107640 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84d8003520 con 0x7f84ec0716c0 2023-11-08T16:32:54.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.317+0000 7f84f2612640 1 -- 172.21.15.81:0/4059107640 >> v2:172.21.15.93:3300/0 conn(0x7f84ec15e440 msgr2=0x7f84ec071180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:54.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.317+0000 7f84f2612640 1 --2- 172.21.15.81:0/4059107640 >> v2:172.21.15.93:3300/0 conn(0x7f84ec15e440 0x7f84ec071180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:54.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.317+0000 7f84f2612640 1 -- 172.21.15.81:0/4059107640 >> v1:172.21.15.81:6789/0 conn(0x7f84ec0716c0 legacy=0x7f84ec0ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:54.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.317+0000 7f84f2612640 1 -- 172.21.15.81:0/4059107640 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec0ca0c0 con 0x7f84ec0cacc0 2023-11-08T16:32:54.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.318+0000 7f84f2612640 1 --2- 172.21.15.81:0/4059107640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84ec0cacc0 0x7f84ec06b280 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0cc2c0 tx=0x7f84e402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a25c192e10c48f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:54.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.319+0000 7f84f0e0f640 1 -- 172.21.15.81:0/4059107640 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e402cc70 con 0x7f84ec0cacc0 2023-11-08T16:32:54.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.319+0000 7f84f0e0f640 1 -- 172.21.15.81:0/4059107640 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e402cdd0 con 0x7f84ec0cacc0 2023-11-08T16:32:54.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.319+0000 7f84f0e0f640 1 -- 172.21.15.81:0/4059107640 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4034660 con 0x7f84ec0cacc0 2023-11-08T16:32:54.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.319+0000 7f84f4075640 1 -- 172.21.15.81:0/4059107640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84ec0cacc0 msgr2=0x7f84ec06b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:54.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.319+0000 7f84f4075640 1 --2- 172.21.15.81:0/4059107640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84ec0cacc0 0x7f84ec06b280 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f84ec0cc2c0 tx=0x7f84e402bec0 comp rx=0 tx=0).stop 2023-11-08T16:32:54.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.319+0000 7f84f4075640 1 -- 172.21.15.81:0/4059107640 shutdown_connections 2023-11-08T16:32:54.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.319+0000 7f84f4075640 1 --2- 172.21.15.81:0/4059107640 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84ec0cacc0 0x7f84ec06b280 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:54.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.319+0000 7f84f4075640 1 --2- 172.21.15.81:0/4059107640 >> v2:172.21.15.93:3300/0 conn(0x7f84ec15e440 0x7f84ec071180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:54.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.319+0000 7f84f4075640 1 -- 172.21.15.81:0/4059107640 >> 172.21.15.81:0/4059107640 conn(0x7f84ec0b05b0 msgr2=0x7f84ec0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:54.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.319+0000 7f84f4075640 1 -- 172.21.15.81:0/4059107640 shutdown_connections 2023-11-08T16:32:54.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.319+0000 7f84f4075640 1 -- 172.21.15.81:0/4059107640 wait complete. 2023-11-08T16:32:54.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.321+0000 7f84f4075640 1 Processor -- start 2023-11-08T16:32:54.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.321+0000 7f84f4075640 1 -- start start 2023-11-08T16:32:54.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.322+0000 7f84f4075640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84ec0716c0 0x7f84ec148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:54.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.322+0000 7f84f4075640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84ec0cacc0 0x7f84ec0c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:54.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.322+0000 7f84f4075640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84ec15e440 0x7f84ec0c4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:54.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.322+0000 7f84f4075640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec0b2890 con 0x7f84ec15e440 2023-11-08T16:32:54.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.322+0000 7f84f4075640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec0c54f0 con 0x7f84ec0cacc0 2023-11-08T16:32:54.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.322+0000 7f84f4075640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84ec0cd320 con 0x7f84ec0716c0 2023-11-08T16:32:54.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.323+0000 7f84f2612640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84ec15e440 0x7f84ec0c4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:54.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.323+0000 7f84f2612640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84ec15e440 0x7f84ec0c4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44184/0 (socket says 172.21.15.81:44184) 2023-11-08T16:32:54.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.323+0000 7f84f2612640 1 -- 172.21.15.81:0/2317686929 learned_addr learned my addr 172.21.15.81:0/2317686929 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:54.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.323+0000 7f84f1e11640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84ec0716c0 0x7f84ec148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:54.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.323+0000 7f84f1610640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84ec0cacc0 0x7f84ec0c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:54.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.323+0000 7f84f2612640 1 -- 172.21.15.81:0/2317686929 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84ec0716c0 msgr2=0x7f84ec148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:54.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.323+0000 7f84f2612640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84ec0716c0 0x7f84ec148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:54.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.323+0000 7f84f2612640 1 -- 172.21.15.81:0/2317686929 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84ec0cacc0 msgr2=0x7f84ec0c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:54.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.323+0000 7f84f2612640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84ec0cacc0 0x7f84ec0c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:54.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.323+0000 7f84f2612640 1 -- 172.21.15.81:0/2317686929 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e4009a70 con 0x7f84ec15e440 2023-11-08T16:32:54.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.323+0000 7f84f2612640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84ec15e440 0x7f84ec0c4fb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009e80 tx=0x7f84e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:54.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.324+0000 7f84de7fc640 1 -- 172.21.15.81:0/2317686929 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4040650 con 0x7f84ec15e440 2023-11-08T16:32:54.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.324+0000 7f84de7fc640 1 -- 172.21.15.81:0/2317686929 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e40407b0 con 0x7f84ec15e440 2023-11-08T16:32:54.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.324+0000 7f84de7fc640 1 -- 172.21.15.81:0/2317686929 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4040b60 con 0x7f84ec15e440 2023-11-08T16:32:54.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.324+0000 7f84f4075640 1 -- 172.21.15.81:0/2317686929 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84ec0cd600 con 0x7f84ec15e440 2023-11-08T16:32:54.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.324+0000 7f84f4075640 1 -- 172.21.15.81:0/2317686929 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84ec0cdbc0 con 0x7f84ec15e440 2023-11-08T16:32:54.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.325+0000 7f84f4075640 1 -- 172.21.15.81:0/2317686929 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84ec0b29d0 con 0x7f84ec15e440 2023-11-08T16:32:54.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.326+0000 7f84de7fc640 1 -- 172.21.15.81:0/2317686929 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84e4040cc0 con 0x7f84ec15e440 2023-11-08T16:32:54.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.327+0000 7f84de7fc640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84d006ae70 0x7f84d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:54.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.327+0000 7f84de7fc640 1 -- 172.21.15.81:0/2317686929 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f84e40aca10 con 0x7f84ec15e440 2023-11-08T16:32:54.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.327+0000 7f84f1e11640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84d006ae70 0x7f84d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:54.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.328+0000 7f84f1e11640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84d006ae70 0x7f84d006d330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f84e8006fd0 tx=0x7f84e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:54.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.331+0000 7f84de7fc640 1 -- 172.21.15.81:0/2317686929 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84e40779b0 con 0x7f84ec15e440 2023-11-08T16:32:54.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:54.558+0000 7f84f4075640 1 -- 172.21.15.81:0/2317686929 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f84ec0b29d0 con 0x7f84ec15e440 2023-11-08T16:32:55.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.092+0000 7f84de7fc640 1 -- 172.21.15.81:0/2317686929 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 9 to fsmap v41) v1 ==== 140+0+0 (secure 0 0 0) 0x7f84ec0b29d0 con 0x7f84ec15e440 2023-11-08T16:32:55.095 INFO:teuthology.orchestra.run.smithi081.stderr:added data pool 9 to fsmap 2023-11-08T16:32:55.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.096+0000 7f84f4075640 1 -- 172.21.15.81:0/2317686929 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84d006ae70 msgr2=0x7f84d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.096+0000 7f84f4075640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84d006ae70 0x7f84d006d330 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f84e8006fd0 tx=0x7f84e8008040 comp rx=0 tx=0).stop 2023-11-08T16:32:55.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.096+0000 7f84f4075640 1 -- 172.21.15.81:0/2317686929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84ec15e440 msgr2=0x7f84ec0c4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.096+0000 7f84f4075640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84ec15e440 0x7f84ec0c4fb0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f84e4009e80 tx=0x7f84e4004a40 comp rx=0 tx=0).stop 2023-11-08T16:32:55.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.097+0000 7f84f4075640 1 -- 172.21.15.81:0/2317686929 shutdown_connections 2023-11-08T16:32:55.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.097+0000 7f84f4075640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84d006ae70 0x7f84d006d330 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.097+0000 7f84f4075640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84ec15e440 0x7f84ec0c4fb0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.097+0000 7f84f4075640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84ec0cacc0 0x7f84ec0c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.097+0000 7f84f4075640 1 --2- 172.21.15.81:0/2317686929 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84ec0716c0 0x7f84ec148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.097+0000 7f84f4075640 1 -- 172.21.15.81:0/2317686929 >> 172.21.15.81:0/2317686929 conn(0x7f84ec0b05b0 msgr2=0x7f84ec15b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:55.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.098+0000 7f84f4075640 1 -- 172.21.15.81:0/2317686929 shutdown_connections 2023-11-08T16:32:55.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.098+0000 7f84f4075640 1 -- 172.21.15.81:0/2317686929 wait complete. 2023-11-08T16:32:55.126 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:32:55.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.309+0000 7f9ead2b0640 1 Processor -- start 2023-11-08T16:32:55.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.309+0000 7f9ead2b0640 1 -- start start 2023-11-08T16:32:55.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.310+0000 7f9ead2b0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ea80cc5e0 0x7f9ea80cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.310+0000 7f9ead2b0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea80c9a70 con 0x7f9ea80ccf80 2023-11-08T16:32:55.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.310+0000 7f9ead2b0640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea80cc3e0 con 0x7f9ea80adcd0 2023-11-08T16:32:55.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.310+0000 7f9ead2b0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea806d070 con 0x7f9ea80cc5e0 2023-11-08T16:32:55.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.311+0000 7f9ea77fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f9ea80ccf80 0x7f9ea8070050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48744/0 (socket says 172.21.15.81:48744) 2023-11-08T16:32:55.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.311+0000 7f9ea77fe640 1 -- 172.21.15.81:0/1357797445 learned_addr learned my addr 172.21.15.81:0/1357797445 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:55.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.311+0000 7f9ea67fc640 1 --2- 172.21.15.81:0/1357797445 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ea80cc5e0 0x7f9ea80cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:55.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.311+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24350549 0 0) 0x7f9ea80c9a70 con 0x7f9ea80ccf80 2023-11-08T16:32:55.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.311+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e90005730 con 0x7f9ea80ccf80 2023-11-08T16:32:55.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4097521026 0 0) 0x7f9e90005730 con 0x7f9ea80ccf80 2023-11-08T16:32:55.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ea80cc5e0 msgr2=0x7f9ea80cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:55.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 --2- 172.21.15.81:0/1357797445 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ea80cc5e0 0x7f9ea80cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 >> v1:172.21.15.93:6789/0 conn(0x7f9ea80adcd0 legacy=0x7f9ea80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea806d350 con 0x7f9ea80ccf80 2023-11-08T16:32:55.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f9ea00019f0 con 0x7f9ea80ccf80 2023-11-08T16:32:55.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 >> v1:172.21.15.81:6789/0 conn(0x7f9ea80ccf80 legacy=0x7f9ea8070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 --2- 172.21.15.81:0/1357797445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 0x7f9e900060b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 --2- 172.21.15.81:0/1357797445 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ea80cc5e0 0x7f9e900069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 --2- 172.21.15.81:0/1357797445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e90006f10 0x7f9e90007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea806d070 con 0x7f9e90005c90 2023-11-08T16:32:55.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9000dd20 con 0x7f9e90006f10 2023-11-08T16:32:55.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9000dee0 con 0x7f9ea80cc5e0 2023-11-08T16:32:55.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea6ffd640 1 --2- 172.21.15.81:0/1357797445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 0x7f9e900060b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:55.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea6ffd640 1 -- 172.21.15.81:0/1357797445 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ea80cc5e0 msgr2=0x7f9e900069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:55.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea6ffd640 1 --2- 172.21.15.81:0/1357797445 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ea80cc5e0 0x7f9e900069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea6ffd640 1 -- 172.21.15.81:0/1357797445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e90006f10 msgr2=0x7f9e90007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:55.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.312+0000 7f9ea6ffd640 1 --2- 172.21.15.81:0/1357797445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e90006f10 0x7f9e90007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.313+0000 7f9ea6ffd640 1 -- 172.21.15.81:0/1357797445 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e90005730 con 0x7f9e90005c90 2023-11-08T16:32:55.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.314+0000 7f9ea6ffd640 1 --2- 172.21.15.81:0/1357797445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 0x7f9e900060b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9e98001750 tx=0x7f9e90007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397a747fc703eb42 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:55.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.314+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980052d0 con 0x7f9e90005c90 2023-11-08T16:32:55.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.314+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9802e050 con 0x7f9e90005c90 2023-11-08T16:32:55.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.314+0000 7f9ea5ffb640 1 -- 172.21.15.81:0/1357797445 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980035f0 con 0x7f9e90005c90 2023-11-08T16:32:55.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.314+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1357797445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 msgr2=0x7f9e900060b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.314+0000 7f9ead2b0640 1 --2- 172.21.15.81:0/1357797445 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 0x7f9e900060b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9e98001750 tx=0x7f9e90007a00 comp rx=0 tx=0).stop 2023-11-08T16:32:55.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.314+0000 7f9ea77fe640 1 -- 172.21.15.81:0/1357797445 reap_dead start 2023-11-08T16:32:55.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.314+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1357797445 shutdown_connections 2023-11-08T16:32:55.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.314+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1357797445 >> 172.21.15.81:0/1357797445 conn(0x7f9ea80cb490 msgr2=0x7f9ea80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:55.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.315+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1357797445 shutdown_connections 2023-11-08T16:32:55.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.315+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1357797445 wait complete. 2023-11-08T16:32:55.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.316+0000 7f9ead2b0640 1 Processor -- start 2023-11-08T16:32:55.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.316+0000 7f9ead2b0640 1 -- start start 2023-11-08T16:32:55.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.317+0000 7f9ead2b0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 0x7f9ea80c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.317+0000 7f9ead2b0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ea80c2f50 0x7f9ea80acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.317+0000 7f9ead2b0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ea8151c70 0x7f9ea81520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.317+0000 7f9ead2b0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea8152630 con 0x7f9e90005c90 2023-11-08T16:32:55.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.317+0000 7f9ead2b0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea80c3780 con 0x7f9ea80c2f50 2023-11-08T16:32:55.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.317+0000 7f9ead2b0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea80c3a60 con 0x7f9ea8151c70 2023-11-08T16:32:55.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.318+0000 7f9ea6ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 0x7f9ea80c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:55.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.318+0000 7f9ea6ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 0x7f9ea80c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44210/0 (socket says 172.21.15.81:44210) 2023-11-08T16:32:55.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.318+0000 7f9ea6ffd640 1 -- 172.21.15.81:0/1225650693 learned_addr learned my addr 172.21.15.81:0/1225650693 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:55.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.318+0000 7f9ea6ffd640 1 -- 172.21.15.81:0/1225650693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ea8151c70 msgr2=0x7f9ea81520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.318+0000 7f9ea6ffd640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ea8151c70 0x7f9ea81520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.318+0000 7f9ea67fc640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ea80c2f50 0x7f9ea80acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:55.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.318+0000 7f9ea6ffd640 1 -- 172.21.15.81:0/1225650693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ea80c2f50 msgr2=0x7f9ea80acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.318+0000 7f9ea6ffd640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ea80c2f50 0x7f9ea80acb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.318+0000 7f9ea6ffd640 1 -- 172.21.15.81:0/1225650693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea80c3d40 con 0x7f9e90005c90 2023-11-08T16:32:55.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.318+0000 7f9ea6ffd640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 0x7f9ea80c7810 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9e98008fd0 tx=0x7f9e98001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:55.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.319+0000 7f9e8f7fe640 1 -- 172.21.15.81:0/1225650693 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980076c0 con 0x7f9e90005c90 2023-11-08T16:32:55.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.319+0000 7f9e8f7fe640 1 -- 172.21.15.81:0/1225650693 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98007a30 con 0x7f9e90005c90 2023-11-08T16:32:55.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.319+0000 7f9e8f7fe640 1 -- 172.21.15.81:0/1225650693 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98006be0 con 0x7f9e90005c90 2023-11-08T16:32:55.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.319+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1225650693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea80c4020 con 0x7f9e90005c90 2023-11-08T16:32:55.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.319+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1225650693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea80c33d0 con 0x7f9e90005c90 2023-11-08T16:32:55.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.320+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1225650693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea8072170 con 0x7f9e90005c90 2023-11-08T16:32:55.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.321+0000 7f9e8f7fe640 1 -- 172.21.15.81:0/1225650693 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e98006d40 con 0x7f9e90005c90 2023-11-08T16:32:55.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.322+0000 7f9e8f7fe640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9e9406ae70 0x7f9e9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.322+0000 7f9e8f7fe640 1 -- 172.21.15.81:0/1225650693 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f9e980ab9b0 con 0x7f9e90005c90 2023-11-08T16:32:55.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.322+0000 7f9ea67fc640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9e9406ae70 0x7f9e9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:55.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.323+0000 7f9ea67fc640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9e9406ae70 0x7f9e9406d330 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80c5a10 tx=0x7f9e9c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:55.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.326+0000 7f9e8f7fe640 1 -- 172.21.15.81:0/1225650693 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9e98076820 con 0x7f9e90005c90 2023-11-08T16:32:55.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.539+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1225650693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ea80c3ba0 con 0x7f9e90005c90 2023-11-08T16:32:55.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.540+0000 7f9e8f7fe640 1 -- 172.21.15.81:0/1225650693 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f9ea80c3ba0 con 0x7f9e90005c90 2023-11-08T16:32:55.542 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:55.543 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3492741644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3492741644},{"type":"v1","addr":"172.21.15.93:6833","nonce":3492741644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":4916,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/119699732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":119699732},{"type":"v1","addr":"172.21.15.93:6835","nonce":119699732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5170,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/1473474760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1473474760},{"type":"v1","addr":"172.21.15.81:6836","nonce":1473474760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:32:43.449426+0000","modified":"2023-11-08T16:32:54.560616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5979},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5979":{"gid":5979,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/1446454901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":1446454901},{"type":"v1","addr":"172.21.15.81:6837","nonce":1446454901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:32:55.543 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 41 2023-11-08T16:32:55.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1225650693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9e9406ae70 msgr2=0x7f9e9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9e9406ae70 0x7f9e9406d330 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9ea80c5a10 tx=0x7f9e9c008040 comp rx=0 tx=0).stop 2023-11-08T16:32:55.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1225650693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 msgr2=0x7f9ea80c7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 0x7f9ea80c7810 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f9e98008fd0 tx=0x7f9e98001120 comp rx=0 tx=0).stop 2023-11-08T16:32:55.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1225650693 shutdown_connections 2023-11-08T16:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9e9406ae70 0x7f9e9406d330 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9ea8151c70 0x7f9ea81520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ea80c2f50 0x7f9ea80acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 --2- 172.21.15.81:0/1225650693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9e90005c90 0x7f9ea80c7810 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1225650693 >> 172.21.15.81:0/1225650693 conn(0x7f9ea80cb490 msgr2=0x7f9ea815cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:55.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1225650693 shutdown_connections 2023-11-08T16:32:55.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.543+0000 7f9ead2b0640 1 -- 172.21.15.81:0/1225650693 wait complete. 2023-11-08T16:32:55.562 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:32:55.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.760+0000 7f18d8275640 1 Processor -- start 2023-11-08T16:32:55.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.760+0000 7f18d8275640 1 -- start start 2023-11-08T16:32:55.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.762+0000 7f18d8275640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f18d00ade50 0x7f18d00ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.762+0000 7f18d8275640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f18d015a680 0x7f18d015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.762+0000 7f18d8275640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d006b630 con 0x7f18d015b050 2023-11-08T16:32:55.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.762+0000 7f18d8275640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d00b1860 con 0x7f18d015a680 2023-11-08T16:32:55.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.762+0000 7f18d8275640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d00ca9c0 con 0x7f18d00ade50 2023-11-08T16:32:55.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.762+0000 7f18d5810640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f18d015a680 0x7f18d015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:55.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.762+0000 7f18d6812640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f18d015b050 0x7f18d00c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48758/0 (socket says 172.21.15.81:48758) 2023-11-08T16:32:55.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.762+0000 7f18d5810640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f18d015a680 0x7f18d015ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44226/0 (socket says 172.21.15.81:44226) 2023-11-08T16:32:55.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.762+0000 7f18d6812640 1 -- 172.21.15.81:0/1162691766 learned_addr learned my addr 172.21.15.81:0/1162691766 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:32:55.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.762+0000 7f18d6011640 1 --2- 172.21.15.81:0/1162691766 >> v2:172.21.15.93:3300/0 conn(0x7f18d00ade50 0x7f18d00ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:55.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.763+0000 7f18d5810640 1 -- 172.21.15.81:0/1162691766 >> v2:172.21.15.93:3300/0 conn(0x7f18d00ade50 msgr2=0x7f18d00ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:32:55.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.763+0000 7f18d5810640 1 --2- 172.21.15.81:0/1162691766 >> v2:172.21.15.93:3300/0 conn(0x7f18d00ade50 0x7f18d00ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.763+0000 7f18d5810640 1 -- 172.21.15.81:0/1162691766 >> v1:172.21.15.81:6789/0 conn(0x7f18d015b050 legacy=0x7f18d00c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.763+0000 7f18d5810640 1 -- 172.21.15.81:0/1162691766 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00cabb0 con 0x7f18d015a680 2023-11-08T16:32:55.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.763+0000 7f18d6011640 1 --2- 172.21.15.81:0/1162691766 >> v2:172.21.15.93:3300/0 conn(0x7f18d00ade50 0x7f18d00ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:32:55.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.764+0000 7f18d5810640 1 --2- 172.21.15.81:0/1162691766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d015a680 0x7f18d015ab10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f18d0067770 tx=0x7f18c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fbd949510084a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:55.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.764+0000 7f18d500f640 1 -- 172.21.15.81:0/1162691766 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0031830 con 0x7f18d015a680 2023-11-08T16:32:55.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.764+0000 7f18d500f640 1 -- 172.21.15.81:0/1162691766 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c0031990 con 0x7f18d015a680 2023-11-08T16:32:55.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.764+0000 7f18d500f640 1 -- 172.21.15.81:0/1162691766 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c0031d20 con 0x7f18d015a680 2023-11-08T16:32:55.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.764+0000 7f18d8275640 1 -- 172.21.15.81:0/1162691766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d015a680 msgr2=0x7f18d015ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.764+0000 7f18d8275640 1 --2- 172.21.15.81:0/1162691766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d015a680 0x7f18d015ab10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f18d0067770 tx=0x7f18c002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.765+0000 7f18d8275640 1 -- 172.21.15.81:0/1162691766 shutdown_connections 2023-11-08T16:32:55.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.765+0000 7f18d8275640 1 --2- 172.21.15.81:0/1162691766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d015a680 0x7f18d015ab10 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.765+0000 7f18d8275640 1 --2- 172.21.15.81:0/1162691766 >> v2:172.21.15.93:3300/0 conn(0x7f18d00ade50 0x7f18d00ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.765+0000 7f18d8275640 1 -- 172.21.15.81:0/1162691766 >> 172.21.15.81:0/1162691766 conn(0x7f18d00cb490 msgr2=0x7f18d00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:55.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.765+0000 7f18d8275640 1 -- 172.21.15.81:0/1162691766 shutdown_connections 2023-11-08T16:32:55.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.765+0000 7f18d8275640 1 -- 172.21.15.81:0/1162691766 wait complete. 2023-11-08T16:32:55.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.766+0000 7f18d8275640 1 Processor -- start 2023-11-08T16:32:55.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.767+0000 7f18d8275640 1 -- start start 2023-11-08T16:32:55.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.767+0000 7f18d8275640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d00ade50 0x7f18d0153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.767+0000 7f18d8275640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18d015a680 0x7f18d0151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.767+0000 7f18d8275640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d015b050 0x7f18d006bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.767+0000 7f18d8275640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d01527c0 con 0x7f18d015b050 2023-11-08T16:32:55.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.767+0000 7f18d8275640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d0152aa0 con 0x7f18d00ade50 2023-11-08T16:32:55.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.767+0000 7f18d8275640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d0152d80 con 0x7f18d015a680 2023-11-08T16:32:55.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.768+0000 7f18d6812640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d015b050 0x7f18d006bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:55.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.768+0000 7f18d6812640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d015b050 0x7f18d006bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44228/0 (socket says 172.21.15.81:44228) 2023-11-08T16:32:55.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.768+0000 7f18d6812640 1 -- 172.21.15.81:0/1372398802 learned_addr learned my addr 172.21.15.81:0/1372398802 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:32:55.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.768+0000 7f18d6011640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d00ade50 0x7f18d0153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:55.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.768+0000 7f18d5810640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18d015a680 0x7f18d0151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:55.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.768+0000 7f18d6812640 1 -- 172.21.15.81:0/1372398802 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18d015a680 msgr2=0x7f18d0151ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.768+0000 7f18d6812640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18d015a680 0x7f18d0151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.768+0000 7f18d6812640 1 -- 172.21.15.81:0/1372398802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d00ade50 msgr2=0x7f18d0153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.768+0000 7f18d6812640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d00ade50 0x7f18d0153860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.768+0000 7f18d6812640 1 -- 172.21.15.81:0/1372398802 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0009400 con 0x7f18d015b050 2023-11-08T16:32:55.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.769+0000 7f18d6812640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d015b050 0x7f18d006bf60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f18c800ca80 tx=0x7f18c800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:55.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.769+0000 7f18c6ffd640 1 -- 172.21.15.81:0/1372398802 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c800ac30 con 0x7f18d015b050 2023-11-08T16:32:55.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.769+0000 7f18c6ffd640 1 -- 172.21.15.81:0/1372398802 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c800ad90 con 0x7f18d015b050 2023-11-08T16:32:55.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.769+0000 7f18c6ffd640 1 -- 172.21.15.81:0/1372398802 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c8012600 con 0x7f18d015b050 2023-11-08T16:32:55.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.769+0000 7f18d8275640 1 -- 172.21.15.81:0/1372398802 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d0153060 con 0x7f18d015b050 2023-11-08T16:32:55.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.769+0000 7f18d8275640 1 -- 172.21.15.81:0/1372398802 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d0152400 con 0x7f18d015b050 2023-11-08T16:32:55.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.770+0000 7f18d8275640 1 -- 172.21.15.81:0/1372398802 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d0153580 con 0x7f18d015b050 2023-11-08T16:32:55.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.771+0000 7f18c6ffd640 1 -- 172.21.15.81:0/1372398802 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18c8007070 con 0x7f18d015b050 2023-11-08T16:32:55.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.772+0000 7f18c6ffd640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f18b406ae70 0x7f18b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:32:55.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.772+0000 7f18c6ffd640 1 -- 172.21.15.81:0/1372398802 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f18c8089590 con 0x7f18d015b050 2023-11-08T16:32:55.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.772+0000 7f18d6011640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f18b406ae70 0x7f18b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:32:55.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.773+0000 7f18d6011640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f18b406ae70 0x7f18b406d330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f18cc008850 tx=0x7f18cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:32:55.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.776+0000 7f18c6ffd640 1 -- 172.21.15.81:0/1372398802 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18d0153580 con 0x7f18d015b050 2023-11-08T16:32:55.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.967+0000 7f18d8275640 1 -- 172.21.15.81:0/1372398802 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18d0153580 con 0x7f18d015b050 2023-11-08T16:32:55.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.971+0000 7f18c6ffd640 1 -- 172.21.15.81:0/1372398802 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18310 (secure 0 0 0) 0x7f18d0153580 con 0x7f18d015b050 2023-11-08T16:32:55.973 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:32:55.973 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":59,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:32:54.560319+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:32:41.268027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T16:32:42.449607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:32:53.335342+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:32:55.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.973+0000 7f18d8275640 1 -- 172.21.15.81:0/1372398802 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f18b406ae70 msgr2=0x7f18b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.973+0000 7f18d8275640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f18b406ae70 0x7f18b406d330 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f18cc008850 tx=0x7f18cc008040 comp rx=0 tx=0).stop 2023-11-08T16:32:55.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.973+0000 7f18d8275640 1 -- 172.21.15.81:0/1372398802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d015b050 msgr2=0x7f18d006bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:32:55.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.973+0000 7f18d8275640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d015b050 0x7f18d006bf60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f18c800ca80 tx=0x7f18c800cf40 comp rx=0 tx=0).stop 2023-11-08T16:32:55.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.973+0000 7f18d8275640 1 -- 172.21.15.81:0/1372398802 shutdown_connections 2023-11-08T16:32:55.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.973+0000 7f18d8275640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f18b406ae70 0x7f18b406d330 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.973+0000 7f18d8275640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18d015b050 0x7f18d006bf60 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.973+0000 7f18d8275640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18d015a680 0x7f18d0151ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.973+0000 7f18d8275640 1 --2- 172.21.15.81:0/1372398802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18d00ade50 0x7f18d0153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:32:55.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.973+0000 7f18d8275640 1 -- 172.21.15.81:0/1372398802 >> 172.21.15.81:0/1372398802 conn(0x7f18d00cb490 msgr2=0x7f18d00ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:32:55.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.973+0000 7f18d8275640 1 -- 172.21.15.81:0/1372398802 shutdown_connections 2023-11-08T16:32:55.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:55.974+0000 7f18d8275640 1 -- 172.21.15.81:0/1372398802 wait complete. 2023-11-08T16:32:55.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:55.995 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-11-08T16:32:56.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:32:56.058 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-11-08T16:32:56.113 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/0 2023-11-08T16:32:56.113 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:32:56.114 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:32:56.114 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-11-08T16:32:56.115 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:32:56.115 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:32:56.115 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:32:56.116 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:32:56.116 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:32:59.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:32:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:32:59.866 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/1 2023-11-08T16:32:59.869 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:32:59.869 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:32:59.870 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-11-08T16:32:59.870 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:32:59.871 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:32:59.871 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:32:59.871 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:32:59.872 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:03.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:03.679 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/2 2023-11-08T16:33:03.680 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:03.681 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:03.681 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-11-08T16:33:03.681 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:03.682 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:03.682 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:03.682 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:03.683 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:07.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:07.466 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/3 2023-11-08T16:33:07.467 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:07.467 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:07.467 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-11-08T16:33:07.468 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:07.468 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:07.468 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:07.469 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:07.469 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:11.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:11.280 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/4 2023-11-08T16:33:11.281 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:11.281 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:11.281 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-11-08T16:33:11.282 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:11.282 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:11.283 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:11.283 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:11.283 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:15.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:15.055 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/5 2023-11-08T16:33:15.056 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:15.056 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:15.056 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/5" 2023-11-08T16:33:15.057 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:15.057 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:15.057 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:15.058 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:15.058 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:17.068 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:33:17.078 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:33:18.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:18.902 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/6 2023-11-08T16:33:18.902 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:18.903 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:18.903 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/6" 2023-11-08T16:33:18.903 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:18.904 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:18.904 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:18.905 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:18.905 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:22.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:22.622 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/7 2023-11-08T16:33:22.623 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:22.623 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:22.624 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/7" 2023-11-08T16:33:22.624 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:22.625 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:22.625 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:22.625 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:22.626 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:26.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:26.477 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/8 2023-11-08T16:33:26.478 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:26.478 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:26.479 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/8" 2023-11-08T16:33:26.479 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:26.479 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:26.480 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:26.480 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:26.480 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:30.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:30.233 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/9 2023-11-08T16:33:30.234 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:30.234 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:30.234 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/9" 2023-11-08T16:33:30.235 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:30.235 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:30.235 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:30.236 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:30.236 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:34.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:34.065 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/10 2023-11-08T16:33:34.065 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:34.066 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:34.066 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/10" 2023-11-08T16:33:34.067 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:34.067 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:34.067 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:34.068 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:34.068 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:37.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:37.818 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/11 2023-11-08T16:33:37.819 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:37.819 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:37.819 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/11" 2023-11-08T16:33:37.820 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:37.820 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:37.820 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:37.821 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:37.821 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:41.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:41.651 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/12 2023-11-08T16:33:41.652 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:41.652 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:41.652 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/12" 2023-11-08T16:33:41.653 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:41.653 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:41.653 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:41.654 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:41.654 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:45.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:45.444 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/13 2023-11-08T16:33:45.445 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:45.445 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:45.446 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/13" 2023-11-08T16:33:45.446 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:45.446 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:45.447 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:45.447 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:45.448 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:47.212 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:33:47.221 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:33:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:49.267 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/14 2023-11-08T16:33:49.268 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:49.268 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:49.268 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/14" 2023-11-08T16:33:49.269 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:49.269 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:49.270 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:49.270 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:49.270 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:52.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:52.983 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/15 2023-11-08T16:33:52.984 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:52.984 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:52.985 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/15" 2023-11-08T16:33:52.985 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:52.985 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:52.986 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:52.986 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:52.986 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:33:56.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:33:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:33:56.764 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/16 2023-11-08T16:33:56.765 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:33:56.765 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:33:56.766 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/16" 2023-11-08T16:33:56.766 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:33:56.766 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:33:56.767 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:33:56.767 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:33:56.767 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:34:00.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:34:00.576 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/17 2023-11-08T16:34:00.576 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:34:00.577 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:34:00.577 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/17" 2023-11-08T16:34:00.577 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:34:00.578 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:34:00.578 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:34:00.579 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:34:00.579 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:34:04.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:34:04.422 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/18 2023-11-08T16:34:04.423 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:34:04.423 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:34:04.424 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/18" 2023-11-08T16:34:04.424 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:34:04.424 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:34:04.425 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:34:04.425 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:34:04.425 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:34:08.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:34:08.176 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/19 2023-11-08T16:34:08.176 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:34:08.177 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:34:08.177 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/19" 2023-11-08T16:34:08.177 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:34:08.178 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:34:08.178 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:34:08.178 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:34:08.179 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:34:11.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:34:11.968 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/20 2023-11-08T16:34:11.969 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:34:11.969 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:34:11.969 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/20" 2023-11-08T16:34:11.970 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:34:11.970 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:34:11.971 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:34:11.971 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:34:11.971 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:34:15.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:34:15.749 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/21 2023-11-08T16:34:15.750 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:34:15.750 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:34:15.751 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/21" 2023-11-08T16:34:15.751 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:34:15.751 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:34:15.752 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:34:15.752 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:34:15.752 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:34:17.359 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:34:17.368 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:34:19.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:34:19.579 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/22 2023-11-08T16:34:19.579 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:34:19.580 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:34:19.580 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/22" 2023-11-08T16:34:19.580 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:34:19.581 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:34:19.581 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:34:19.582 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:34:19.582 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:34:23.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:34:23.264 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/23 2023-11-08T16:34:23.265 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:34:23.265 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:34:23.265 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/23" 2023-11-08T16:34:23.266 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:34:23.266 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:34:23.266 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:34:23.267 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:34:23.267 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:34:27.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:34:27.076 INFO:tasks.cephfs.mount:Writing 6291456 bytes to subdir/24 2023-11-08T16:34:27.077 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:34:27.077 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:34:27.077 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/24" 2023-11-08T16:34:27.078 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:34:27.078 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 6291456): 2023-11-08T16:34:27.079 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:34:27.079 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:34:27.079 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:34:30.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:34:30.857 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:34:30.857 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:34:30.951 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:34:31.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.207+0000 7f2c62b65640 1 Processor -- start 2023-11-08T16:34:31.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.207+0000 7f2c62b65640 1 -- start start 2023-11-08T16:34:31.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.208+0000 7f2c62b65640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2c5c0ca480 0x7f2c5c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.208+0000 7f2c62b65640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2c5c15ea60 0x7f2c5c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.208+0000 7f2c62b65640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5c06b630 con 0x7f2c5c0ade50 2023-11-08T16:34:31.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.208+0000 7f2c62b65640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c0b1860 con 0x7f2c5c15ea60 2023-11-08T16:34:31.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.208+0000 7f2c62b65640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c0ca910 con 0x7f2c5c0ca480 2023-11-08T16:34:31.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.208+0000 7f2c60901640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2c5c0ade50 0x7f2c5c15a680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58906/0 (socket says 172.21.15.81:58906) 2023-11-08T16:34:31.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.208+0000 7f2c60901640 1 -- 172.21.15.81:0/912159863 learned_addr learned my addr 172.21.15.81:0/912159863 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:31.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c61102640 1 --2- 172.21.15.81:0/912159863 >> v2:172.21.15.81:3300/0 conn(0x7f2c5c15ea60 0x7f2c5c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55254495 0 0) 0x7f2c5c06b630 con 0x7f2c5c0ade50 2023-11-08T16:34:31.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c44003520 con 0x7f2c5c0ade50 2023-11-08T16:34:31.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c53fff640 1 --2- 172.21.15.81:0/912159863 >> v2:172.21.15.93:3300/0 conn(0x7f2c5c0ca480 0x7f2c5c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 744837214 0 0) 0x7f2c44003520 con 0x7f2c5c0ade50 2023-11-08T16:34:31.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 >> v2:172.21.15.93:3300/0 conn(0x7f2c5c0ca480 msgr2=0x7f2c5c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:31.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 --2- 172.21.15.81:0/912159863 >> v2:172.21.15.93:3300/0 conn(0x7f2c5c0ca480 0x7f2c5c15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 >> v2:172.21.15.81:3300/0 conn(0x7f2c5c15ea60 msgr2=0x7f2c5c15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:31.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 --2- 172.21.15.81:0/912159863 >> v2:172.21.15.81:3300/0 conn(0x7f2c5c15ea60 0x7f2c5c15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c0cabb0 con 0x7f2c5c0ade50 2023-11-08T16:34:31.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f2c58001670 con 0x7f2c5c0ade50 2023-11-08T16:34:31.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 >> v1:172.21.15.81:6789/0 conn(0x7f2c5c0ade50 legacy=0x7f2c5c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 --2- 172.21.15.81:0/912159863 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c44003a50 0x7f2c44003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 --2- 172.21.15.81:0/912159863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 0x7f2c44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 --2- 172.21.15.81:0/912159863 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c440051d0 0x7f2c4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c0ca910 con 0x7f2c440047c0 2023-11-08T16:34:31.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4400c100 con 0x7f2c44003a50 2023-11-08T16:34:31.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.209+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4400c2c0 con 0x7f2c440051d0 2023-11-08T16:34:31.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.210+0000 7f2c53fff640 1 --2- 172.21.15.81:0/912159863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 0x7f2c44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.210+0000 7f2c60901640 1 --2- 172.21.15.81:0/912159863 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c44003a50 0x7f2c44003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.210+0000 7f2c53fff640 1 -- 172.21.15.81:0/912159863 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c440051d0 msgr2=0x7f2c4400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:31.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.210+0000 7f2c53fff640 1 --2- 172.21.15.81:0/912159863 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c440051d0 0x7f2c4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.210+0000 7f2c53fff640 1 -- 172.21.15.81:0/912159863 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c44003a50 msgr2=0x7f2c44003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:31.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.210+0000 7f2c53fff640 1 --2- 172.21.15.81:0/912159863 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c44003a50 0x7f2c44003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.210+0000 7f2c61102640 1 -- 172.21.15.81:0/912159863 reap_dead start 2023-11-08T16:34:31.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.210+0000 7f2c53fff640 1 -- 172.21.15.81:0/912159863 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c44003520 con 0x7f2c440047c0 2023-11-08T16:34:31.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.211+0000 7f2c53fff640 1 --2- 172.21.15.81:0/912159863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 0x7f2c44004c40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00a970 tx=0x7f2c440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b72380121e30c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:31.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.211+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c032670 con 0x7f2c440047c0 2023-11-08T16:34:31.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.211+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c4c0327d0 con 0x7f2c440047c0 2023-11-08T16:34:31.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.211+0000 7f2c537fe640 1 -- 172.21.15.81:0/912159863 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c032bc0 con 0x7f2c440047c0 2023-11-08T16:34:31.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.212+0000 7f2c62b65640 1 -- 172.21.15.81:0/912159863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 msgr2=0x7f2c44004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.212+0000 7f2c62b65640 1 --2- 172.21.15.81:0/912159863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 0x7f2c44004c40 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c00a970 tx=0x7f2c440057f0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.212+0000 7f2c62b65640 1 -- 172.21.15.81:0/912159863 shutdown_connections 2023-11-08T16:34:31.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.212+0000 7f2c62b65640 1 --2- 172.21.15.81:0/912159863 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 0x7f2c44004c40 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.212+0000 7f2c62b65640 1 -- 172.21.15.81:0/912159863 >> 172.21.15.81:0/912159863 conn(0x7f2c5c0cb490 msgr2=0x7f2c5c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:31.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.212+0000 7f2c62b65640 1 -- 172.21.15.81:0/912159863 shutdown_connections 2023-11-08T16:34:31.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.212+0000 7f2c62b65640 1 -- 172.21.15.81:0/912159863 wait complete. 2023-11-08T16:34:31.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.214+0000 7f2c62b65640 1 Processor -- start 2023-11-08T16:34:31.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.214+0000 7f2c62b65640 1 -- start start 2023-11-08T16:34:31.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.215+0000 7f2c62b65640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 0x7f2c5c14c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.215+0000 7f2c62b65640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c150320 0x7f2c5c1507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.215+0000 7f2c62b65640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c1553f0 0x7f2c5c155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.215+0000 7f2c62b65640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c154350 con 0x7f2c440047c0 2023-11-08T16:34:31.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.215+0000 7f2c62b65640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c154630 con 0x7f2c5c150320 2023-11-08T16:34:31.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.215+0000 7f2c62b65640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c154910 con 0x7f2c5c1553f0 2023-11-08T16:34:31.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.215+0000 7f2c60901640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 0x7f2c5c14c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.215+0000 7f2c60901640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 0x7f2c5c14c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50996/0 (socket says 172.21.15.81:50996) 2023-11-08T16:34:31.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.215+0000 7f2c60901640 1 -- 172.21.15.81:0/3954403313 learned_addr learned my addr 172.21.15.81:0/3954403313 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:31.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c53fff640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c150320 0x7f2c5c1507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c61102640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c1553f0 0x7f2c5c155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c60901640 1 -- 172.21.15.81:0/3954403313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c1553f0 msgr2=0x7f2c5c155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c60901640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c1553f0 0x7f2c5c155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c60901640 1 -- 172.21.15.81:0/3954403313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c150320 msgr2=0x7f2c5c1507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c60901640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c150320 0x7f2c5c1507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c60901640 1 -- 172.21.15.81:0/3954403313 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c154bf0 con 0x7f2c440047c0 2023-11-08T16:34:31.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c53fff640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c150320 0x7f2c5c1507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:31.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c60901640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 0x7f2c5c14c5a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2c58001eb0 tx=0x7f2c5800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:31.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c517fa640 1 -- 172.21.15.81:0/3954403313 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c580074e0 con 0x7f2c440047c0 2023-11-08T16:34:31.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c517fa640 1 -- 172.21.15.81:0/3954403313 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c58007640 con 0x7f2c440047c0 2023-11-08T16:34:31.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c62b65640 1 -- 172.21.15.81:0/3954403313 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c154ed0 con 0x7f2c440047c0 2023-11-08T16:34:31.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c517fa640 1 -- 172.21.15.81:0/3954403313 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58004a00 con 0x7f2c440047c0 2023-11-08T16:34:31.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.216+0000 7f2c62b65640 1 -- 172.21.15.81:0/3954403313 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c14cae0 con 0x7f2c440047c0 2023-11-08T16:34:31.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.217+0000 7f2c62b65640 1 -- 172.21.15.81:0/3954403313 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5c04ece0 con 0x7f2c440047c0 2023-11-08T16:34:31.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.218+0000 7f2c517fa640 1 -- 172.21.15.81:0/3954403313 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c58004b60 con 0x7f2c440047c0 2023-11-08T16:34:31.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.219+0000 7f2c517fa640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4806ae70 0x7f2c4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.219+0000 7f2c517fa640 1 -- 172.21.15.81:0/3954403313 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f2c580894b0 con 0x7f2c440047c0 2023-11-08T16:34:31.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.220+0000 7f2c53fff640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4806ae70 0x7f2c4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.220+0000 7f2c53fff640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4806ae70 0x7f2c4806d330 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c036000 tx=0x7f2c4c03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:31.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.224+0000 7f2c517fa640 1 -- 172.21.15.81:0/3954403313 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c58054360 con 0x7f2c440047c0 2023-11-08T16:34:31.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.413+0000 7f2c62b65640 1 -- 172.21.15.81:0/3954403313 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c5c06b8a0 con 0x7f2c440047c0 2023-11-08T16:34:31.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.416+0000 7f2c517fa640 1 -- 172.21.15.81:0/3954403313 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18383 (secure 0 0 0) 0x7f2c58057980 con 0x7f2c440047c0 2023-11-08T16:34:31.420 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:31.420 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":62,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:33:43.881792+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:32:41.268027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T16:32:42.449607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:32:53.335342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.e","mappings":[{"from":0,"to":6}]},{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"9.11","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:34:31.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 -- 172.21.15.81:0/3954403313 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4806ae70 msgr2=0x7f2c4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4806ae70 0x7f2c4806d330 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c036000 tx=0x7f2c4c03d040 comp rx=0 tx=0).stop 2023-11-08T16:34:31.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 -- 172.21.15.81:0/3954403313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 msgr2=0x7f2c5c14c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 0x7f2c5c14c5a0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f2c58001eb0 tx=0x7f2c5800dc00 comp rx=0 tx=0).stop 2023-11-08T16:34:31.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 -- 172.21.15.81:0/3954403313 shutdown_connections 2023-11-08T16:34:31.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c4806ae70 0x7f2c4806d330 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c5c1553f0 0x7f2c5c155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c5c150320 0x7f2c5c1507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 --2- 172.21.15.81:0/3954403313 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c440047c0 0x7f2c5c14c5a0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 -- 172.21.15.81:0/3954403313 >> 172.21.15.81:0/3954403313 conn(0x7f2c5c0cb490 msgr2=0x7f2c5c0afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:31.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 -- 172.21.15.81:0/3954403313 shutdown_connections 2023-11-08T16:34:31.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.419+0000 7f2c62b65640 1 -- 172.21.15.81:0/3954403313 wait complete. 2023-11-08T16:34:31.439 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:32:41.268027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:32:42.449607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:32:53.335342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:34:31.440 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:34:31.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.637+0000 7f11e71b7640 1 Processor -- start 2023-11-08T16:34:31.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.638+0000 7f11e71b7640 1 -- start start 2023-11-08T16:34:31.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.639+0000 7f11e71b7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e00ce0d0 0x7f11e00caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.639+0000 7f11e71b7640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f11e015f280 0x7f11e01516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.639+0000 7f11e71b7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e006fa30 con 0x7f11e015bfc0 2023-11-08T16:34:31.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.639+0000 7f11e71b7640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e00cda10 con 0x7f11e015f280 2023-11-08T16:34:31.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.639+0000 7f11e71b7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00ce4b0 con 0x7f11e00ce0d0 2023-11-08T16:34:31.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.639+0000 7f11dffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f11e015bfc0 0x7f11e015eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58920/0 (socket says 172.21.15.81:58920) 2023-11-08T16:34:31.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.639+0000 7f11dffff640 1 -- 172.21.15.81:0/594967992 learned_addr learned my addr 172.21.15.81:0/594967992 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:31.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11e4f53640 1 --2- 172.21.15.81:0/594967992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e00ce0d0 0x7f11e00caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11e5754640 1 --2- 172.21.15.81:0/594967992 >> v2:172.21.15.93:3300/0 conn(0x7f11e015f280 0x7f11e01516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767203494 0 0) 0x7f11e006fa30 con 0x7f11e015bfc0 2023-11-08T16:34:31.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11d0003520 con 0x7f11e015bfc0 2023-11-08T16:34:31.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2504477031 0 0) 0x7f11d0003520 con 0x7f11e015bfc0 2023-11-08T16:34:31.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e00ce0d0 msgr2=0x7f11e00caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:31.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 --2- 172.21.15.81:0/594967992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e00ce0d0 0x7f11e00caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 >> v2:172.21.15.93:3300/0 conn(0x7f11e015f280 msgr2=0x7f11e01516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:31.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 --2- 172.21.15.81:0/594967992 >> v2:172.21.15.93:3300/0 conn(0x7f11e015f280 0x7f11e01516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e006fca0 con 0x7f11e015bfc0 2023-11-08T16:34:31.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f11cc001670 con 0x7f11e015bfc0 2023-11-08T16:34:31.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 >> v1:172.21.15.81:6789/0 conn(0x7f11e015bfc0 legacy=0x7f11e015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 --2- 172.21.15.81:0/594967992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11d00039e0 0x7f11d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 --2- 172.21.15.81:0/594967992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e00ce0d0 0x7f11d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 --2- 172.21.15.81:0/594967992 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11d0004d00 0x7f11d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00cda10 con 0x7f11d00039e0 2023-11-08T16:34:31.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00ce4b0 con 0x7f11d0004d00 2023-11-08T16:34:31.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.640+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d000b980 con 0x7f11e00ce0d0 2023-11-08T16:34:31.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.641+0000 7f11e4f53640 1 --2- 172.21.15.81:0/594967992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11d00039e0 0x7f11d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.641+0000 7f11dffff640 1 --2- 172.21.15.81:0/594967992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e00ce0d0 0x7f11d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.641+0000 7f11e4f53640 1 -- 172.21.15.81:0/594967992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e00ce0d0 msgr2=0x7f11d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:31.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.641+0000 7f11e4f53640 1 --2- 172.21.15.81:0/594967992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e00ce0d0 0x7f11d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.641+0000 7f11e4f53640 1 -- 172.21.15.81:0/594967992 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11d0004d00 msgr2=0x7f11d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:31.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.641+0000 7f11e4f53640 1 --2- 172.21.15.81:0/594967992 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11d0004d00 0x7f11d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.641+0000 7f11e4f53640 1 -- 172.21.15.81:0/594967992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d0003520 con 0x7f11d00039e0 2023-11-08T16:34:31.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.642+0000 7f11e4f53640 1 --2- 172.21.15.81:0/594967992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11d00039e0 0x7f11d0003e00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009ac0 tx=0x7f11d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941f61afe011b05b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:31.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.642+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d402c6b0 con 0x7f11d00039e0 2023-11-08T16:34:31.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.642+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11d402c810 con 0x7f11d00039e0 2023-11-08T16:34:31.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.642+0000 7f11df7fe640 1 -- 172.21.15.81:0/594967992 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11d402cba0 con 0x7f11d00039e0 2023-11-08T16:34:31.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.643+0000 7f11e71b7640 1 -- 172.21.15.81:0/594967992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11d00039e0 msgr2=0x7f11d0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.643+0000 7f11e71b7640 1 --2- 172.21.15.81:0/594967992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11d00039e0 0x7f11d0003e00 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009ac0 tx=0x7f11d00057f0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.643+0000 7f11e5754640 1 -- 172.21.15.81:0/594967992 reap_dead start 2023-11-08T16:34:31.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.643+0000 7f11e71b7640 1 -- 172.21.15.81:0/594967992 shutdown_connections 2023-11-08T16:34:31.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.643+0000 7f11e71b7640 1 -- 172.21.15.81:0/594967992 >> 172.21.15.81:0/594967992 conn(0x7f11e00cb580 msgr2=0x7f11e00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:31.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.643+0000 7f11e71b7640 1 -- 172.21.15.81:0/594967992 shutdown_connections 2023-11-08T16:34:31.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.643+0000 7f11e71b7640 1 -- 172.21.15.81:0/594967992 wait complete. 2023-11-08T16:34:31.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.645+0000 7f11e71b7640 1 Processor -- start 2023-11-08T16:34:31.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.645+0000 7f11e71b7640 1 -- start start 2023-11-08T16:34:31.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11e71b7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11d00039e0 0x7f11e01f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11e71b7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e006cc10 0x7f11e014d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11e71b7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e014e1b0 0x7f11e015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11e71b7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00c5bd0 con 0x7f11e006cc10 2023-11-08T16:34:31.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11e71b7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e00c5e70 con 0x7f11d00039e0 2023-11-08T16:34:31.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11e71b7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e01f6760 con 0x7f11e014e1b0 2023-11-08T16:34:31.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11dffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e006cc10 0x7f11e014d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11dffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e006cc10 0x7f11e014d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51002/0 (socket says 172.21.15.81:51002) 2023-11-08T16:34:31.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11dffff640 1 -- 172.21.15.81:0/3821836510 learned_addr learned my addr 172.21.15.81:0/3821836510 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:31.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11e4f53640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11d00039e0 0x7f11e01f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11e5754640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e014e1b0 0x7f11e015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.646+0000 7f11dffff640 1 -- 172.21.15.81:0/3821836510 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e014e1b0 msgr2=0x7f11e015a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.647+0000 7f11dffff640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e014e1b0 0x7f11e015a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.647+0000 7f11dffff640 1 -- 172.21.15.81:0/3821836510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11d00039e0 msgr2=0x7f11e01f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.647+0000 7f11dffff640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11d00039e0 0x7f11e01f6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.647+0000 7f11dffff640 1 -- 172.21.15.81:0/3821836510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e01f6a00 con 0x7f11e006cc10 2023-11-08T16:34:31.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.647+0000 7f11e4f53640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11d00039e0 0x7f11e01f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:31.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.647+0000 7f11dffff640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e006cc10 0x7f11e014d8d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00e730 tx=0x7f11cc00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:31.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.647+0000 7f11dd7fa640 1 -- 172.21.15.81:0/3821836510 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00ee40 con 0x7f11e006cc10 2023-11-08T16:34:31.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.647+0000 7f11dd7fa640 1 -- 172.21.15.81:0/3821836510 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11cc00c870 con 0x7f11e006cc10 2023-11-08T16:34:31.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.647+0000 7f11dd7fa640 1 -- 172.21.15.81:0/3821836510 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc00cc20 con 0x7f11e006cc10 2023-11-08T16:34:31.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.647+0000 7f11e71b7640 1 -- 172.21.15.81:0/3821836510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e01f6ce0 con 0x7f11e006cc10 2023-11-08T16:34:31.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.647+0000 7f11e71b7640 1 -- 172.21.15.81:0/3821836510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e014de10 con 0x7f11e006cc10 2023-11-08T16:34:31.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.648+0000 7f11e71b7640 1 -- 172.21.15.81:0/3821836510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e014dfd0 con 0x7f11e006cc10 2023-11-08T16:34:31.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.649+0000 7f11dd7fa640 1 -- 172.21.15.81:0/3821836510 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11cc01d460 con 0x7f11e006cc10 2023-11-08T16:34:31.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.651+0000 7f11dd7fa640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c406ae70 0x7f11c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:31.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.651+0000 7f11dd7fa640 1 -- 172.21.15.81:0/3821836510 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f11cc089790 con 0x7f11e006cc10 2023-11-08T16:34:31.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.651+0000 7f11e4f53640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c406ae70 0x7f11c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:31.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.651+0000 7f11e4f53640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c406ae70 0x7f11c406d330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f11d4036000 tx=0x7f11d4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:31.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.655+0000 7f11dd7fa640 1 -- 172.21.15.81:0/3821836510 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11cc054640 con 0x7f11e006cc10 2023-11-08T16:34:31.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.845+0000 7f11e71b7640 1 -- 172.21.15.81:0/3821836510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11e00ca8d0 con 0x7f11e006cc10 2023-11-08T16:34:31.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.845+0000 7f11dd7fa640 1 -- 172.21.15.81:0/3821836510 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11cc057c60 con 0x7f11e006cc10 2023-11-08T16:34:31.849 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:34:31.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 -- 172.21.15.81:0/3821836510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c406ae70 msgr2=0x7f11c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c406ae70 0x7f11c406d330 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f11d4036000 tx=0x7f11d4030000 comp rx=0 tx=0).stop 2023-11-08T16:34:31.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 -- 172.21.15.81:0/3821836510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e006cc10 msgr2=0x7f11e014d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:31.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e006cc10 0x7f11e014d8d0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f11cc00e730 tx=0x7f11cc00ebf0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 -- 172.21.15.81:0/3821836510 shutdown_connections 2023-11-08T16:34:31.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c406ae70 0x7f11c406d330 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e014e1b0 0x7f11e015a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e006cc10 0x7f11e014d8d0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 --2- 172.21.15.81:0/3821836510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11d00039e0 0x7f11e01f6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:31.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 -- 172.21.15.81:0/3821836510 >> 172.21.15.81:0/3821836510 conn(0x7f11e00cb580 msgr2=0x7f11e00cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:31.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 -- 172.21.15.81:0/3821836510 shutdown_connections 2023-11-08T16:34:31.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:31.848+0000 7f11e71b7640 1 -- 172.21.15.81:0/3821836510 wait complete. 2023-11-08T16:34:31.869 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:34:32.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.085+0000 7f6ddd9b6640 1 Processor -- start 2023-11-08T16:34:32.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.085+0000 7f6ddd9b6640 1 -- start start 2023-11-08T16:34:32.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.087+0000 7f6ddd9b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6dd815f420 0x7f6dd8071090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.087+0000 7f6ddd9b6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6dd806f8b0 0x7f6dd80715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.087+0000 7f6ddd9b6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd8071b10 con 0x7f6dd815eaa0 2023-11-08T16:34:32.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.087+0000 7f6ddd9b6640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd80c9060 con 0x7f6dd806f8b0 2023-11-08T16:34:32.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.087+0000 7f6ddd9b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd80c9340 con 0x7f6dd815f420 2023-11-08T16:34:32.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.087+0000 7f6dcffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6dd806f8b0 0x7f6dd80715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.088+0000 7f6dcffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6dd806f8b0 0x7f6dd80715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51004/0 (socket says 172.21.15.81:51004) 2023-11-08T16:34:32.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.088+0000 7f6dcffff640 1 -- 172.21.15.81:0/500039487 learned_addr learned my addr 172.21.15.81:0/500039487 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:32.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.088+0000 7f6dd6ffd640 1 --2- 172.21.15.81:0/500039487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6dd815f420 0x7f6dd8071090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.088+0000 7f6dcffff640 1 -- 172.21.15.81:0/500039487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6dd815f420 msgr2=0x7f6dd8071090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:32.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.088+0000 7f6dcffff640 1 --2- 172.21.15.81:0/500039487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6dd815f420 0x7f6dd8071090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.088+0000 7f6dcffff640 1 -- 172.21.15.81:0/500039487 >> v1:172.21.15.93:6789/0 conn(0x7f6dd815eaa0 legacy=0x7f6dd815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.088+0000 7f6dcffff640 1 -- 172.21.15.81:0/500039487 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd80c9620 con 0x7f6dd806f8b0 2023-11-08T16:34:32.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.089+0000 7f6dcffff640 1 --2- 172.21.15.81:0/500039487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dd806f8b0 0x7f6dd80715d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009870 tx=0x7f6dc002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369e0f152b69180a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:32.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.089+0000 7f6dd67fc640 1 -- 172.21.15.81:0/500039487 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0030840 con 0x7f6dd806f8b0 2023-11-08T16:34:32.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.089+0000 7f6dd67fc640 1 -- 172.21.15.81:0/500039487 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc00309a0 con 0x7f6dd806f8b0 2023-11-08T16:34:32.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.089+0000 7f6dd67fc640 1 -- 172.21.15.81:0/500039487 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0030d30 con 0x7f6dd806f8b0 2023-11-08T16:34:32.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.090+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/500039487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dd806f8b0 msgr2=0x7f6dd80715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.090+0000 7f6ddd9b6640 1 --2- 172.21.15.81:0/500039487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dd806f8b0 0x7f6dd80715d0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009870 tx=0x7f6dc002b980 comp rx=0 tx=0).stop 2023-11-08T16:34:32.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.090+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/500039487 shutdown_connections 2023-11-08T16:34:32.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.090+0000 7f6ddd9b6640 1 --2- 172.21.15.81:0/500039487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dd806f8b0 0x7f6dd80715d0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.090+0000 7f6ddd9b6640 1 --2- 172.21.15.81:0/500039487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6dd815f420 0x7f6dd8071090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.090+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/500039487 >> 172.21.15.81:0/500039487 conn(0x7f6dd80cb490 msgr2=0x7f6dd80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:32.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.090+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/500039487 shutdown_connections 2023-11-08T16:34:32.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.090+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/500039487 wait complete. 2023-11-08T16:34:32.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.092+0000 7f6ddd9b6640 1 Processor -- start 2023-11-08T16:34:32.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.092+0000 7f6ddd9b6640 1 -- start start 2023-11-08T16:34:32.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6ddd9b6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6dd806f8b0 0x7f6dd80cac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6ddd9b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dd815eaa0 0x7f6dd80c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6ddd9b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6dd815f420 0x7f6dd80acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6ddd9b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd815c570 con 0x7f6dd815eaa0 2023-11-08T16:34:32.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6ddd9b6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd80c49e0 con 0x7f6dd806f8b0 2023-11-08T16:34:32.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6ddd9b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd80c4cc0 con 0x7f6dd815f420 2023-11-08T16:34:32.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6dcffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dd815eaa0 0x7f6dd80c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6dcffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dd815eaa0 0x7f6dd80c2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51010/0 (socket says 172.21.15.81:51010) 2023-11-08T16:34:32.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6dcffff640 1 -- 172.21.15.81:0/1899290875 learned_addr learned my addr 172.21.15.81:0/1899290875 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:32.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6dcffff640 1 -- 172.21.15.81:0/1899290875 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6dd815f420 msgr2=0x7f6dd80acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6dcffff640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6dd815f420 0x7f6dd80acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6dcffff640 1 -- 172.21.15.81:0/1899290875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6dd806f8b0 msgr2=0x7f6dd80cac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6dcffff640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6dd806f8b0 0x7f6dd80cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.093+0000 7f6dcffff640 1 -- 172.21.15.81:0/1899290875 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00094d0 con 0x7f6dd815eaa0 2023-11-08T16:34:32.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.094+0000 7f6dcffff640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dd815eaa0 0x7f6dd80c2f50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009870 tx=0x7f6dc0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:32.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.094+0000 7f6dd4ff9640 1 -- 172.21.15.81:0/1899290875 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0039c90 con 0x7f6dd815eaa0 2023-11-08T16:34:32.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.094+0000 7f6dd4ff9640 1 -- 172.21.15.81:0/1899290875 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc0039df0 con 0x7f6dd815eaa0 2023-11-08T16:34:32.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.094+0000 7f6dd4ff9640 1 -- 172.21.15.81:0/1899290875 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0043640 con 0x7f6dd815eaa0 2023-11-08T16:34:32.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.094+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/1899290875 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd80c4fa0 con 0x7f6dd815eaa0 2023-11-08T16:34:32.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.094+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/1899290875 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd80ad060 con 0x7f6dd815eaa0 2023-11-08T16:34:32.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.095+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/1899290875 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd80ad1f0 con 0x7f6dd815eaa0 2023-11-08T16:34:32.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.096+0000 7f6dd4ff9640 1 -- 172.21.15.81:0/1899290875 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6dc00437a0 con 0x7f6dd815eaa0 2023-11-08T16:34:32.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.097+0000 7f6dd4ff9640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6db806ae70 0x7f6db806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.097+0000 7f6dd4ff9640 1 -- 172.21.15.81:0/1899290875 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f6dc003e070 con 0x7f6dd815eaa0 2023-11-08T16:34:32.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.097+0000 7f6dd6ffd640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6db806ae70 0x7f6db806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.098+0000 7f6dd6ffd640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6db806ae70 0x7f6db806d330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8006fd0 tx=0x7f6dc8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:32.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.101+0000 7f6dd4ff9640 1 -- 172.21.15.81:0/1899290875 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6dc0077790 con 0x7f6dd815eaa0 2023-11-08T16:34:32.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.292+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/1899290875 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6dd80c57a0 con 0x7f6dd815eaa0 2023-11-08T16:34:32.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.293+0000 7f6dd4ff9640 1 -- 172.21.15.81:0/1899290875 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6dd80c57a0 con 0x7f6dd815eaa0 2023-11-08T16:34:32.296 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:34:32.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/1899290875 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6db806ae70 msgr2=0x7f6db806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6db806ae70 0x7f6db806d330 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8006fd0 tx=0x7f6dc8008040 comp rx=0 tx=0).stop 2023-11-08T16:34:32.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/1899290875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dd815eaa0 msgr2=0x7f6dd80c2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dd815eaa0 0x7f6dd80c2f50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009870 tx=0x7f6dc0004730 comp rx=0 tx=0).stop 2023-11-08T16:34:32.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/1899290875 shutdown_connections 2023-11-08T16:34:32.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6db806ae70 0x7f6db806d330 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6dd815f420 0x7f6dd80acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6dd815eaa0 0x7f6dd80c2f50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 --2- 172.21.15.81:0/1899290875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6dd806f8b0 0x7f6dd80cac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/1899290875 >> 172.21.15.81:0/1899290875 conn(0x7f6dd80cb490 msgr2=0x7f6dd815cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:32.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/1899290875 shutdown_connections 2023-11-08T16:34:32.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.296+0000 7f6ddd9b6640 1 -- 172.21.15.81:0/1899290875 wait complete. 2023-11-08T16:34:32.317 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:34:32.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.533+0000 7faafba61640 1 Processor -- start 2023-11-08T16:34:32.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.533+0000 7faafba61640 1 -- start start 2023-11-08T16:34:32.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.534+0000 7faafba61640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaf40ce0d0 0x7faaf40caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.534+0000 7faafba61640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf40cda10 con 0x7faaf415bfc0 2023-11-08T16:34:32.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.534+0000 7faafba61640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaf406fa30 con 0x7faaf415f280 2023-11-08T16:34:32.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.534+0000 7faafba61640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf40ce4b0 con 0x7faaf40ce0d0 2023-11-08T16:34:32.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.534+0000 7faaf8ffc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7faaf415bfc0 0x7faaf415eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58930/0 (socket says 172.21.15.81:58930) 2023-11-08T16:34:32.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.534+0000 7faaf8ffc640 1 -- 172.21.15.81:0/898809975 learned_addr learned my addr 172.21.15.81:0/898809975 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:32.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaf97fd640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaf40ce0d0 0x7faaf40caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2972209407 0 0) 0x7faaf40cda10 con 0x7faaf415bfc0 2023-11-08T16:34:32.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faae0003520 con 0x7faaf415bfc0 2023-11-08T16:34:32.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 456802406 0 0) 0x7faae0003520 con 0x7faaf415bfc0 2023-11-08T16:34:32.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaf40ce0d0 msgr2=0x7faaf40caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:32.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaf40ce0d0 0x7faaf40caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 >> v1:172.21.15.93:6789/0 conn(0x7faaf415f280 legacy=0x7faaf41516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:32.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf406fca0 con 0x7faaf415bfc0 2023-11-08T16:34:32.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7faae4001670 con 0x7faaf415bfc0 2023-11-08T16:34:32.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 >> v1:172.21.15.81:6789/0 conn(0x7faaf415bfc0 legacy=0x7faaf415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae0003a10 0x7faae0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae00047c0 0x7faae0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaf40ce0d0 0x7faae0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf40ce4b0 con 0x7faae00047c0 2023-11-08T16:34:32.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae000ba60 con 0x7faae0003a10 2023-11-08T16:34:32.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.535+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faae000bc20 con 0x7faaf40ce0d0 2023-11-08T16:34:32.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.536+0000 7faaf8ffc640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae00047c0 0x7faae0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.536+0000 7faaf97fd640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae0003a10 0x7faae0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.536+0000 7faaf9ffe640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaf40ce0d0 0x7faae0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.536+0000 7faaf8ffc640 1 -- 172.21.15.81:0/898809975 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaf40ce0d0 msgr2=0x7faae0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:32.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.536+0000 7faaf8ffc640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaf40ce0d0 0x7faae0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.536+0000 7faaf8ffc640 1 -- 172.21.15.81:0/898809975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae0003a10 msgr2=0x7faae0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:32.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.536+0000 7faaf8ffc640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae0003a10 0x7faae0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.536+0000 7faaf8ffc640 1 -- 172.21.15.81:0/898809975 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae0003520 con 0x7faae00047c0 2023-11-08T16:34:32.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.537+0000 7faaf8ffc640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae00047c0 0x7faae0004c40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faae4001eb0 tx=0x7faae00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11274c3611931f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:32.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.537+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4005ad0 con 0x7faae00047c0 2023-11-08T16:34:32.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.537+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae4005c50 con 0x7faae00047c0 2023-11-08T16:34:32.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.537+0000 7faaebfff640 1 -- 172.21.15.81:0/898809975 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae402c850 con 0x7faae00047c0 2023-11-08T16:34:32.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.538+0000 7faafba61640 1 -- 172.21.15.81:0/898809975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae00047c0 msgr2=0x7faae0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.538+0000 7faafba61640 1 --2- 172.21.15.81:0/898809975 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faae00047c0 0x7faae0004c40 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7faae4001eb0 tx=0x7faae00057f0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.538+0000 7faaf9ffe640 1 -- 172.21.15.81:0/898809975 reap_dead start 2023-11-08T16:34:32.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.538+0000 7faafba61640 1 -- 172.21.15.81:0/898809975 shutdown_connections 2023-11-08T16:34:32.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.538+0000 7faafba61640 1 -- 172.21.15.81:0/898809975 >> 172.21.15.81:0/898809975 conn(0x7faaf40cb580 msgr2=0x7faaf40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:32.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.538+0000 7faafba61640 1 -- 172.21.15.81:0/898809975 shutdown_connections 2023-11-08T16:34:32.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.538+0000 7faafba61640 1 -- 172.21.15.81:0/898809975 wait complete. 2023-11-08T16:34:32.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.540+0000 7faafba61640 1 Processor -- start 2023-11-08T16:34:32.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.540+0000 7faafba61640 1 -- start start 2023-11-08T16:34:32.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.541+0000 7faafba61640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faae00047c0 0x7faaf40c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.541+0000 7faafba61640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf40b2540 0x7faaf4148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.541+0000 7faafba61640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faaf40b3d30 0x7faaf40af890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.541+0000 7faafba61640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf40b2c90 con 0x7faaf40b2540 2023-11-08T16:34:32.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.541+0000 7faafba61640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf40b2f70 con 0x7faaf40b3d30 2023-11-08T16:34:32.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.541+0000 7faafba61640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaf40b3250 con 0x7faae00047c0 2023-11-08T16:34:32.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.541+0000 7faaf8ffc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf40b2540 0x7faaf4148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.541+0000 7faaf8ffc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf40b2540 0x7faaf4148e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51032/0 (socket says 172.21.15.81:51032) 2023-11-08T16:34:32.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.541+0000 7faaf8ffc640 1 -- 172.21.15.81:0/2616392787 learned_addr learned my addr 172.21.15.81:0/2616392787 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:32.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faaf97fd640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faae00047c0 0x7faaf40c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faaf9ffe640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faaf40b3d30 0x7faaf40af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faaf8ffc640 1 -- 172.21.15.81:0/2616392787 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faae00047c0 msgr2=0x7faaf40c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faaf8ffc640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faae00047c0 0x7faaf40c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faaf8ffc640 1 -- 172.21.15.81:0/2616392787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faaf40b3d30 msgr2=0x7faaf40af890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faaf8ffc640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faaf40b3d30 0x7faaf40af890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faaf8ffc640 1 -- 172.21.15.81:0/2616392787 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf40b3530 con 0x7faaf40b2540 2023-11-08T16:34:32.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faaf97fd640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faae00047c0 0x7faaf40c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:32.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faaf8ffc640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf40b2540 0x7faaf4148e90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7faae402c3f0 tx=0x7faae4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:32.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faae9ffb640 1 -- 172.21.15.81:0/2616392787 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae4039030 con 0x7faaf40b2540 2023-11-08T16:34:32.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faae9ffb640 1 -- 172.21.15.81:0/2616392787 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faae402cc60 con 0x7faaf40b2540 2023-11-08T16:34:32.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faae9ffb640 1 -- 172.21.15.81:0/2616392787 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae403c040 con 0x7faaf40b2540 2023-11-08T16:34:32.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.542+0000 7faafba61640 1 -- 172.21.15.81:0/2616392787 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaf40b3810 con 0x7faaf40b2540 2023-11-08T16:34:32.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.543+0000 7faafba61640 1 -- 172.21.15.81:0/2616392787 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf41493d0 con 0x7faaf40b2540 2023-11-08T16:34:32.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.544+0000 7faafba61640 1 -- 172.21.15.81:0/2616392787 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaf40b30b0 con 0x7faaf40b2540 2023-11-08T16:34:32.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.545+0000 7faae9ffb640 1 -- 172.21.15.81:0/2616392787 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faae4032070 con 0x7faaf40b2540 2023-11-08T16:34:32.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.546+0000 7faae9ffb640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faad806ae70 0x7faad806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.546+0000 7faae9ffb640 1 -- 172.21.15.81:0/2616392787 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7faae40aad30 con 0x7faaf40b2540 2023-11-08T16:34:32.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.546+0000 7faaf97fd640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faad806ae70 0x7faad806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.547+0000 7faaf97fd640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faad806ae70 0x7faad806d330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009a00 tx=0x7faaf0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:32.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.551+0000 7faae9ffb640 1 -- 172.21.15.81:0/2616392787 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faaf40b30b0 con 0x7faaf40b2540 2023-11-08T16:34:32.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.745+0000 7faafba61640 1 -- 172.21.15.81:0/2616392787 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faaf40b3390 con 0x7faaf40b2540 2023-11-08T16:34:32.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.746+0000 7faae9ffb640 1 -- 172.21.15.81:0/2616392787 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7faaf40b3390 con 0x7faaf40b2540 2023-11-08T16:34:32.748 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:34:32.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.748+0000 7faafba61640 1 -- 172.21.15.81:0/2616392787 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faad806ae70 msgr2=0x7faad806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.748+0000 7faafba61640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faad806ae70 0x7faad806d330 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009a00 tx=0x7faaf0008040 comp rx=0 tx=0).stop 2023-11-08T16:34:32.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.748+0000 7faafba61640 1 -- 172.21.15.81:0/2616392787 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf40b2540 msgr2=0x7faaf4148e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.748+0000 7faafba61640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf40b2540 0x7faaf4148e90 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7faae402c3f0 tx=0x7faae4001a00 comp rx=0 tx=0).stop 2023-11-08T16:34:32.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.748+0000 7faafba61640 1 -- 172.21.15.81:0/2616392787 shutdown_connections 2023-11-08T16:34:32.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.748+0000 7faafba61640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faad806ae70 0x7faad806d330 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.748+0000 7faafba61640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faaf40b3d30 0x7faaf40af890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.748+0000 7faafba61640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaf40b2540 0x7faaf4148e90 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.748+0000 7faafba61640 1 --2- 172.21.15.81:0/2616392787 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faae00047c0 0x7faaf40c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.748+0000 7faafba61640 1 -- 172.21.15.81:0/2616392787 >> 172.21.15.81:0/2616392787 conn(0x7faaf40cb580 msgr2=0x7faaf40cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:32.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.749+0000 7faafba61640 1 -- 172.21.15.81:0/2616392787 shutdown_connections 2023-11-08T16:34:32.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.749+0000 7faafba61640 1 -- 172.21.15.81:0/2616392787 wait complete. 2023-11-08T16:34:32.769 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:34:32.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.987+0000 7f2c7eb90640 1 Processor -- start 2023-11-08T16:34:32.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.987+0000 7f2c7eb90640 1 -- start start 2023-11-08T16:34:32.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.988+0000 7f2c7eb90640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2c780cdcb0 0x7f2c7806e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.988+0000 7f2c7eb90640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c7806cf50 con 0x7f2c780cc9f0 2023-11-08T16:34:32.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.989+0000 7f2c7eb90640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c780d0df0 con 0x7f2c780cd8d0 2023-11-08T16:34:32.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.989+0000 7f2c7eb90640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c7806d230 con 0x7f2c780cdcb0 2023-11-08T16:34:32.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.989+0000 7f2c7d12d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2c780cc9f0 0x7f2c7815c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58936/0 (socket says 172.21.15.81:58936) 2023-11-08T16:34:32.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.989+0000 7f2c7d12d640 1 -- 172.21.15.81:0/1415645442 learned_addr learned my addr 172.21.15.81:0/1415645442 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:32.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.989+0000 7f2c77fff640 1 --2- 172.21.15.81:0/1415645442 >> v2:172.21.15.93:3300/0 conn(0x7f2c780cdcb0 0x7f2c7806e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.989+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006859482 0 0) 0x7f2c7806cf50 con 0x7f2c780cc9f0 2023-11-08T16:34:32.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.989+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c60003520 con 0x7f2c780cc9f0 2023-11-08T16:34:32.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2766186756 0 0) 0x7f2c60003520 con 0x7f2c780cc9f0 2023-11-08T16:34:32.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 >> v2:172.21.15.93:3300/0 conn(0x7f2c780cdcb0 msgr2=0x7f2c7806e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:32.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 --2- 172.21.15.81:0/1415645442 >> v2:172.21.15.93:3300/0 conn(0x7f2c780cdcb0 0x7f2c7806e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 >> v1:172.21.15.93:6789/0 conn(0x7f2c780cd8d0 legacy=0x7f2c780cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:32.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c780d10d0 con 0x7f2c780cc9f0 2023-11-08T16:34:32.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f2c70001670 con 0x7f2c780cc9f0 2023-11-08T16:34:32.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 >> v1:172.21.15.81:6789/0 conn(0x7f2c780cc9f0 legacy=0x7f2c7815c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:32.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 --2- 172.21.15.81:0/1415645442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c600039e0 0x7f2c60003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 --2- 172.21.15.81:0/1415645442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c600047c0 0x7f2c60004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 --2- 172.21.15.81:0/1415645442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c60005180 0x7f2c6000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:32.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7806d230 con 0x7f2c600039e0 2023-11-08T16:34:32.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6000c080 con 0x7f2c600047c0 2023-11-08T16:34:32.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6000c240 con 0x7f2c60005180 2023-11-08T16:34:32.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.990+0000 7f2c7c92c640 1 --2- 172.21.15.81:0/1415645442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c600039e0 0x7f2c60003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.991+0000 7f2c77fff640 1 --2- 172.21.15.81:0/1415645442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c600047c0 0x7f2c60004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:32.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.991+0000 7f2c7c92c640 1 -- 172.21.15.81:0/1415645442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c60005180 msgr2=0x7f2c6000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:32.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.991+0000 7f2c7c92c640 1 --2- 172.21.15.81:0/1415645442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c60005180 0x7f2c6000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.991+0000 7f2c7c92c640 1 -- 172.21.15.81:0/1415645442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c600047c0 msgr2=0x7f2c60004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:32.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.991+0000 7f2c7c92c640 1 --2- 172.21.15.81:0/1415645442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c600047c0 0x7f2c60004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:32.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.991+0000 7f2c7c92c640 1 -- 172.21.15.81:0/1415645442 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c60003520 con 0x7f2c600039e0 2023-11-08T16:34:32.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.991+0000 7f2c77fff640 1 --2- 172.21.15.81:0/1415645442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c600047c0 0x7f2c60004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:32.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.991+0000 7f2c7d12d640 1 -- 172.21.15.81:0/1415645442 reap_dead start 2023-11-08T16:34:33.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.992+0000 7f2c7c92c640 1 --2- 172.21.15.81:0/1415645442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c600039e0 0x7f2c60003de0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2c680044b0 tx=0x7f2c600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a980c8a1e5ff379 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:33.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.992+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68030400 con 0x7f2c600039e0 2023-11-08T16:34:33.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.992+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c68030560 con 0x7f2c600039e0 2023-11-08T16:34:33.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.992+0000 7f2c777fe640 1 -- 172.21.15.81:0/1415645442 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c68030950 con 0x7f2c600039e0 2023-11-08T16:34:33.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.992+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1415645442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c600039e0 msgr2=0x7f2c60003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.992+0000 7f2c7eb90640 1 --2- 172.21.15.81:0/1415645442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c600039e0 0x7f2c60003de0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2c680044b0 tx=0x7f2c600057f0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.993+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1415645442 shutdown_connections 2023-11-08T16:34:33.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.993+0000 7f2c7eb90640 1 --2- 172.21.15.81:0/1415645442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c600039e0 0x7f2c60003de0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.993+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1415645442 >> 172.21.15.81:0/1415645442 conn(0x7f2c78065df0 msgr2=0x7f2c780661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:33.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.993+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1415645442 shutdown_connections 2023-11-08T16:34:33.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.993+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1415645442 wait complete. 2023-11-08T16:34:33.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.994+0000 7f2c7eb90640 1 Processor -- start 2023-11-08T16:34:33.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.995+0000 7f2c7eb90640 1 -- start start 2023-11-08T16:34:33.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.995+0000 7f2c7eb90640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c600039e0 0x7f2c781f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.995+0000 7f2c7eb90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c7814a9f0 0x7f2c7814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.995+0000 7f2c7eb90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c780b4990 0x7f2c780b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.995+0000 7f2c7eb90640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c781f2680 con 0x7f2c7814a9f0 2023-11-08T16:34:33.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.995+0000 7f2c7eb90640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7814b3b0 con 0x7f2c600039e0 2023-11-08T16:34:33.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.995+0000 7f2c7eb90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c780b5730 con 0x7f2c780b4990 2023-11-08T16:34:33.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c77fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c7814a9f0 0x7f2c7814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c77fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c7814a9f0 0x7f2c7814ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51042/0 (socket says 172.21.15.81:51042) 2023-11-08T16:34:33.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c77fff640 1 -- 172.21.15.81:0/1632703913 learned_addr learned my addr 172.21.15.81:0/1632703913 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:33.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c7c92c640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c600039e0 0x7f2c781f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c7d12d640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c780b4990 0x7f2c780b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c77fff640 1 -- 172.21.15.81:0/1632703913 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c780b4990 msgr2=0x7f2c780b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c77fff640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c780b4990 0x7f2c780b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c77fff640 1 -- 172.21.15.81:0/1632703913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c600039e0 msgr2=0x7f2c781f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c77fff640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c600039e0 0x7f2c781f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c77fff640 1 -- 172.21.15.81:0/1632703913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c780b5a10 con 0x7f2c7814a9f0 2023-11-08T16:34:33.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c7d12d640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c780b4990 0x7f2c780b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:33.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.996+0000 7f2c7c92c640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c600039e0 0x7f2c781f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:33.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.997+0000 7f2c77fff640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c7814a9f0 0x7f2c7814ae70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0107c0 tx=0x7f2c6c010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:33.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.997+0000 7f2c757fa640 1 -- 172.21.15.81:0/1632703913 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c00eb60 con 0x7f2c7814a9f0 2023-11-08T16:34:33.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.997+0000 7f2c757fa640 1 -- 172.21.15.81:0/1632703913 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6c010ed0 con 0x7f2c7814a9f0 2023-11-08T16:34:33.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.997+0000 7f2c757fa640 1 -- 172.21.15.81:0/1632703913 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c01c400 con 0x7f2c7814a9f0 2023-11-08T16:34:33.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.997+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1632703913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c780b5cf0 con 0x7f2c7814a9f0 2023-11-08T16:34:33.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.997+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1632703913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c780b5350 con 0x7f2c7814a9f0 2023-11-08T16:34:33.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:32.998+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1632703913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c780b6210 con 0x7f2c7814a9f0 2023-11-08T16:34:33.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.000+0000 7f2c757fa640 1 -- 172.21.15.81:0/1632703913 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c6c01c560 con 0x7f2c7814a9f0 2023-11-08T16:34:33.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.001+0000 7f2c757fa640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406ae70 0x7f2c6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.001+0000 7f2c757fa640 1 -- 172.21.15.81:0/1632703913 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f2c6c089c40 con 0x7f2c7814a9f0 2023-11-08T16:34:33.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.001+0000 7f2c7c92c640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406ae70 0x7f2c6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.002+0000 7f2c7c92c640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406ae70 0x7f2c6406d330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2c68036000 tx=0x7f2c68009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:33.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.006+0000 7f2c757fa640 1 -- 172.21.15.81:0/1632703913 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c780b6210 con 0x7f2c7814a9f0 2023-11-08T16:34:33.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.193+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1632703913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f2c780b6210 con 0x7f2c7814a9f0 2023-11-08T16:34:33.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.194+0000 7f2c757fa640 1 -- 172.21.15.81:0/1632703913 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v62) v1 ==== 106+0+11 (secure 0 0 0) 0x7f2c780b6210 con 0x7f2c7814a9f0 2023-11-08T16:34:33.197 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:34:33.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.196+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1632703913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406ae70 msgr2=0x7f2c6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.196+0000 7f2c7eb90640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406ae70 0x7f2c6406d330 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2c68036000 tx=0x7f2c68009030 comp rx=0 tx=0).stop 2023-11-08T16:34:33.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.196+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1632703913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c7814a9f0 msgr2=0x7f2c7814ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.196+0000 7f2c7eb90640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c7814a9f0 0x7f2c7814ae70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c0107c0 tx=0x7f2c6c010c80 comp rx=0 tx=0).stop 2023-11-08T16:34:33.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.197+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1632703913 shutdown_connections 2023-11-08T16:34:33.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.197+0000 7f2c7eb90640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406ae70 0x7f2c6406d330 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.197+0000 7f2c7eb90640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c780b4990 0x7f2c780b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.197+0000 7f2c7eb90640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c7814a9f0 0x7f2c7814ae70 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.197+0000 7f2c7eb90640 1 --2- 172.21.15.81:0/1632703913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c600039e0 0x7f2c781f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.197+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1632703913 >> 172.21.15.81:0/1632703913 conn(0x7f2c78065df0 msgr2=0x7f2c780cbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:33.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.197+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1632703913 shutdown_connections 2023-11-08T16:34:33.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.197+0000 7f2c7eb90640 1 -- 172.21.15.81:0/1632703913 wait complete. 2023-11-08T16:34:33.220 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:34:33.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.430+0000 7f665f59e640 1 Processor -- start 2023-11-08T16:34:33.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.430+0000 7f665f59e640 1 -- start start 2023-11-08T16:34:33.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665f59e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f666015d750 0x7f666015dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665f59e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6660070580 con 0x7f66600cd110 2023-11-08T16:34:33.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665f59e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66600c9460 con 0x7f6660159af0 2023-11-08T16:34:33.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665f59e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f666015e2a0 con 0x7f666015d750 2023-11-08T16:34:33.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665ed9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f66600cd110 0x7f66600c99c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58938/0 (socket says 172.21.15.81:58938) 2023-11-08T16:34:33.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665ed9d640 1 -- 172.21.15.81:0/2931173977 learned_addr learned my addr 172.21.15.81:0/2931173977 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:33.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f6657fff640 1 --2- 172.21.15.81:0/2931173977 >> v2:172.21.15.93:3300/0 conn(0x7f666015d750 0x7f666015dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532038917 0 0) 0x7f6660070580 con 0x7f66600cd110 2023-11-08T16:34:33.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f664c003520 con 0x7f66600cd110 2023-11-08T16:34:33.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3673861380 0 0) 0x7f664c003520 con 0x7f66600cd110 2023-11-08T16:34:33.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 >> v2:172.21.15.93:3300/0 conn(0x7f666015d750 msgr2=0x7f666015dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:33.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 --2- 172.21.15.81:0/2931173977 >> v2:172.21.15.93:3300/0 conn(0x7f666015d750 0x7f666015dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 >> v1:172.21.15.93:6789/0 conn(0x7f6660159af0 legacy=0x7f6660070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666015e580 con 0x7f66600cd110 2023-11-08T16:34:33.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f66580019f0 con 0x7f66600cd110 2023-11-08T16:34:33.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 >> v1:172.21.15.81:6789/0 conn(0x7f66600cd110 legacy=0x7f66600c99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 --2- 172.21.15.81:0/2931173977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f664c003970 0x7f664c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 --2- 172.21.15.81:0/2931173977 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f664c0047c0 0x7f664c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 --2- 172.21.15.81:0/2931173977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f664c005180 0x7f664c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666015e2a0 con 0x7f664c003970 2023-11-08T16:34:33.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664c00c020 con 0x7f664c005180 2023-11-08T16:34:33.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664c00c1e0 con 0x7f664c0047c0 2023-11-08T16:34:33.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.432+0000 7f665e59c640 1 --2- 172.21.15.81:0/2931173977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f664c003970 0x7f664c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.433+0000 7f665ed9d640 1 --2- 172.21.15.81:0/2931173977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f664c005180 0x7f664c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.433+0000 7f6657fff640 1 --2- 172.21.15.81:0/2931173977 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f664c0047c0 0x7f664c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.433+0000 7f665e59c640 1 -- 172.21.15.81:0/2931173977 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f664c0047c0 msgr2=0x7f664c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:33.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.433+0000 7f665e59c640 1 --2- 172.21.15.81:0/2931173977 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f664c0047c0 0x7f664c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.433+0000 7f665e59c640 1 -- 172.21.15.81:0/2931173977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f664c005180 msgr2=0x7f664c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:33.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.433+0000 7f665e59c640 1 --2- 172.21.15.81:0/2931173977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f664c005180 0x7f664c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.433+0000 7f665e59c640 1 -- 172.21.15.81:0/2931173977 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f664c003520 con 0x7f664c003970 2023-11-08T16:34:33.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.433+0000 7f665ed9d640 1 -- 172.21.15.81:0/2931173977 reap_dead start 2023-11-08T16:34:33.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.434+0000 7f665e59c640 1 --2- 172.21.15.81:0/2931173977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f664c003970 0x7f664c003d90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6650001750 tx=0x7f664c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ac485ba0d468f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:33.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.434+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66500052d0 con 0x7f664c003970 2023-11-08T16:34:33.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.434+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665002e050 con 0x7f664c003970 2023-11-08T16:34:33.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.434+0000 7f665dd9b640 1 -- 172.21.15.81:0/2931173977 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6650003830 con 0x7f664c003970 2023-11-08T16:34:33.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.435+0000 7f665f59e640 1 -- 172.21.15.81:0/2931173977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f664c003970 msgr2=0x7f664c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.435+0000 7f665f59e640 1 --2- 172.21.15.81:0/2931173977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f664c003970 0x7f664c003d90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6650001750 tx=0x7f664c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.435+0000 7f665f59e640 1 -- 172.21.15.81:0/2931173977 shutdown_connections 2023-11-08T16:34:33.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.435+0000 7f665f59e640 1 --2- 172.21.15.81:0/2931173977 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f664c003970 0x7f664c003d90 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.435+0000 7f665f59e640 1 -- 172.21.15.81:0/2931173977 >> 172.21.15.81:0/2931173977 conn(0x7f66600b04d0 msgr2=0x7f66600b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:33.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.435+0000 7f665f59e640 1 -- 172.21.15.81:0/2931173977 shutdown_connections 2023-11-08T16:34:33.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.435+0000 7f665f59e640 1 -- 172.21.15.81:0/2931173977 wait complete. 2023-11-08T16:34:33.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.437+0000 7f665f59e640 1 Processor -- start 2023-11-08T16:34:33.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.437+0000 7f665f59e640 1 -- start start 2023-11-08T16:34:33.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665f59e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f664c003970 0x7f6660154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665f59e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66600c5b40 0x7f66601547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665f59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66600b34e0 0x7f6660158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665f59e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6660154d30 con 0x7f66600b34e0 2023-11-08T16:34:33.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665f59e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66600c2530 con 0x7f66600c5b40 2023-11-08T16:34:33.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665f59e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66600c2810 con 0x7f664c003970 2023-11-08T16:34:33.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66600b34e0 0x7f6660158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66600b34e0 0x7f6660158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51056/0 (socket says 172.21.15.81:51056) 2023-11-08T16:34:33.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665ed9d640 1 -- 172.21.15.81:0/549625554 learned_addr learned my addr 172.21.15.81:0/549625554 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:33.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f6657fff640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66600c5b40 0x7f66601547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665e59c640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f664c003970 0x7f6660154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665ed9d640 1 -- 172.21.15.81:0/549625554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f664c003970 msgr2=0x7f6660154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665ed9d640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f664c003970 0x7f6660154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665ed9d640 1 -- 172.21.15.81:0/549625554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66600c5b40 msgr2=0x7f66601547f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665ed9d640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66600c5b40 0x7f66601547f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.438+0000 7f665ed9d640 1 -- 172.21.15.81:0/549625554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66600c2af0 con 0x7f66600b34e0 2023-11-08T16:34:33.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.439+0000 7f665ed9d640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66600b34e0 0x7f6660158350 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6658000f80 tx=0x7f665800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:33.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.439+0000 7f6656ffd640 1 -- 172.21.15.81:0/549625554 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658004d60 con 0x7f66600b34e0 2023-11-08T16:34:33.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.439+0000 7f6656ffd640 1 -- 172.21.15.81:0/549625554 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665800bb20 con 0x7f66600b34e0 2023-11-08T16:34:33.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.439+0000 7f665f59e640 1 -- 172.21.15.81:0/549625554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66600c2dd0 con 0x7f66600b34e0 2023-11-08T16:34:33.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.439+0000 7f6656ffd640 1 -- 172.21.15.81:0/549625554 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665800bed0 con 0x7f66600b34e0 2023-11-08T16:34:33.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.439+0000 7f665f59e640 1 -- 172.21.15.81:0/549625554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66600b3970 con 0x7f66600b34e0 2023-11-08T16:34:33.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.440+0000 7f665f59e640 1 -- 172.21.15.81:0/549625554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66600c32f0 con 0x7f66600b34e0 2023-11-08T16:34:33.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.441+0000 7f6656ffd640 1 -- 172.21.15.81:0/549625554 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6658007110 con 0x7f66600b34e0 2023-11-08T16:34:33.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.443+0000 7f6656ffd640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f664006ae70 0x7f664006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.443+0000 7f6656ffd640 1 -- 172.21.15.81:0/549625554 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f66580896a0 con 0x7f66600b34e0 2023-11-08T16:34:33.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.443+0000 7f665e59c640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f664006ae70 0x7f664006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.444+0000 7f665e59c640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f664006ae70 0x7f664006d330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6650033000 tx=0x7f6650009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:33.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.447+0000 7f6656ffd640 1 -- 172.21.15.81:0/549625554 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66600c32f0 con 0x7f66600b34e0 2023-11-08T16:34:33.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.650+0000 7f665f59e640 1 -- 172.21.15.81:0/549625554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f66600c32f0 con 0x7f66600b34e0 2023-11-08T16:34:33.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.651+0000 7f6656ffd640 1 -- 172.21.15.81:0/549625554 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7f66600c32f0 con 0x7f66600b34e0 2023-11-08T16:34:33.652 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:34:33.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.653+0000 7f665f59e640 1 -- 172.21.15.81:0/549625554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f664006ae70 msgr2=0x7f664006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.653+0000 7f665f59e640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f664006ae70 0x7f664006d330 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6650033000 tx=0x7f6650009030 comp rx=0 tx=0).stop 2023-11-08T16:34:33.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.653+0000 7f665f59e640 1 -- 172.21.15.81:0/549625554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66600b34e0 msgr2=0x7f6660158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.653+0000 7f665f59e640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66600b34e0 0x7f6660158350 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6658000f80 tx=0x7f665800b4a0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.653+0000 7f665f59e640 1 -- 172.21.15.81:0/549625554 shutdown_connections 2023-11-08T16:34:33.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.653+0000 7f665f59e640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f664006ae70 0x7f664006d330 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.653+0000 7f665f59e640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f66600b34e0 0x7f6660158350 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.653+0000 7f665f59e640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f66600c5b40 0x7f66601547f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.653+0000 7f665f59e640 1 --2- 172.21.15.81:0/549625554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f664c003970 0x7f6660154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.653+0000 7f665f59e640 1 -- 172.21.15.81:0/549625554 >> 172.21.15.81:0/549625554 conn(0x7f66600b04d0 msgr2=0x7f66600cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:33.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.654+0000 7f665f59e640 1 -- 172.21.15.81:0/549625554 shutdown_connections 2023-11-08T16:34:33.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.654+0000 7f665f59e640 1 -- 172.21.15.81:0/549625554 wait complete. 2023-11-08T16:34:33.674 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:34:33.674 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:34:33.726 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:34:33.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.929+0000 7f3622644640 1 Processor -- start 2023-11-08T16:34:33.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.929+0000 7f3622644640 1 -- start start 2023-11-08T16:34:33.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.930+0000 7f3622644640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f361c161710 0x7f361c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.930+0000 7f3622644640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f361c0ccd30 0x7f361c06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.930+0000 7f3622644640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f361c06b980 con 0x7f361c15d2b0 2023-11-08T16:34:33.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.930+0000 7f3622644640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f361c0cd1c0 con 0x7f361c0ccd30 2023-11-08T16:34:33.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.931+0000 7f3622644640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c15d690 con 0x7f361c161710 2023-11-08T16:34:33.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.931+0000 7f361bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f361c15d2b0 0x7f361c0c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58942/0 (socket says 172.21.15.81:58942) 2023-11-08T16:34:33.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.931+0000 7f361bfff640 1 -- 172.21.15.81:0/3185892741 learned_addr learned my addr 172.21.15.81:0/3185892741 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:33.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.931+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523822819 0 0) 0x7f361c06b980 con 0x7f361c15d2b0 2023-11-08T16:34:33.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.931+0000 7f3613fff640 1 --2- 172.21.15.81:0/3185892741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f361c161710 0x7f361c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.931+0000 7f3620be1640 1 --2- 172.21.15.81:0/3185892741 >> v2:172.21.15.93:3300/0 conn(0x7f361c0ccd30 0x7f361c06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.931+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3608003520 con 0x7f361c15d2b0 2023-11-08T16:34:33.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3341951201 0 0) 0x7f3608003520 con 0x7f361c15d2b0 2023-11-08T16:34:33.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f361c161710 msgr2=0x7f361c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:33.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 --2- 172.21.15.81:0/3185892741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f361c161710 0x7f361c0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 >> v2:172.21.15.93:3300/0 conn(0x7f361c0ccd30 msgr2=0x7f361c06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:33.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 --2- 172.21.15.81:0/3185892741 >> v2:172.21.15.93:3300/0 conn(0x7f361c0ccd30 0x7f361c06ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c06bc60 con 0x7f361c15d2b0 2023-11-08T16:34:33.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f360c001670 con 0x7f361c15d2b0 2023-11-08T16:34:33.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f3620be1640 1 --2- 172.21.15.81:0/3185892741 >> v2:172.21.15.93:3300/0 conn(0x7f361c0ccd30 0x7f361c06ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:34:33.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 >> v1:172.21.15.81:6789/0 conn(0x7f361c15d2b0 legacy=0x7f361c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 --2- 172.21.15.81:0/3185892741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608003a50 0x7f3608003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 --2- 172.21.15.81:0/3185892741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f361c161710 0x7f36080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 --2- 172.21.15.81:0/3185892741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3608004d00 0x7f3608005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c0cd1c0 con 0x7f3608003a50 2023-11-08T16:34:33.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c15d690 con 0x7f3608004d00 2023-11-08T16:34:33.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f360800bb10 con 0x7f361c161710 2023-11-08T16:34:33.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361bfff640 1 --2- 172.21.15.81:0/3185892741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608003a50 0x7f3608003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361bfff640 1 -- 172.21.15.81:0/3185892741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f361c161710 msgr2=0x7f36080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:33.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361bfff640 1 --2- 172.21.15.81:0/3185892741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f361c161710 0x7f36080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361bfff640 1 -- 172.21.15.81:0/3185892741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3608004d00 msgr2=0x7f3608005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:33.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361bfff640 1 --2- 172.21.15.81:0/3185892741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3608004d00 0x7f3608005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.932+0000 7f361bfff640 1 -- 172.21.15.81:0/3185892741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3608003520 con 0x7f3608003a50 2023-11-08T16:34:33.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.933+0000 7f361bfff640 1 --2- 172.21.15.81:0/3185892741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608003a50 0x7f3608003e70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c240 tx=0x7f36080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453645edc77a0cbe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:33.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.934+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c005ad0 con 0x7f3608003a50 2023-11-08T16:34:33.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.934+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f360c005c30 con 0x7f3608003a50 2023-11-08T16:34:33.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.934+0000 7f361b7fe640 1 -- 172.21.15.81:0/3185892741 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f360c02c850 con 0x7f3608003a50 2023-11-08T16:34:33.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.934+0000 7f3622644640 1 -- 172.21.15.81:0/3185892741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608003a50 msgr2=0x7f3608003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.934+0000 7f3622644640 1 --2- 172.21.15.81:0/3185892741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3608003a50 0x7f3608003e70 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f360c00c240 tx=0x7f36080057f0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.934+0000 7f3620be1640 1 -- 172.21.15.81:0/3185892741 reap_dead start 2023-11-08T16:34:33.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.934+0000 7f3622644640 1 -- 172.21.15.81:0/3185892741 shutdown_connections 2023-11-08T16:34:33.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.934+0000 7f3622644640 1 -- 172.21.15.81:0/3185892741 >> 172.21.15.81:0/3185892741 conn(0x7f361c0cb490 msgr2=0x7f361c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:33.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.934+0000 7f3622644640 1 -- 172.21.15.81:0/3185892741 shutdown_connections 2023-11-08T16:34:33.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.934+0000 7f3622644640 1 -- 172.21.15.81:0/3185892741 wait complete. 2023-11-08T16:34:33.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.936+0000 7f3622644640 1 Processor -- start 2023-11-08T16:34:33.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.936+0000 7f3622644640 1 -- start start 2023-11-08T16:34:33.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.937+0000 7f3622644640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3608003a50 0x7f361c157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.937+0000 7f3622644640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f361c0b15c0 0x7f361c0b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.937+0000 7f3622644640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f361c0b1a40 0x7f361c0b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.937+0000 7f3622644640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c157570 con 0x7f361c0b1a40 2023-11-08T16:34:33.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.937+0000 7f3622644640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f361c15f980 con 0x7f361c0b15c0 2023-11-08T16:34:33.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.937+0000 7f3622644640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f361c1f3ae0 con 0x7f3608003a50 2023-11-08T16:34:33.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.937+0000 7f3620be1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f361c0b1a40 0x7f361c0b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.937+0000 7f3620be1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f361c0b1a40 0x7f361c0b3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51082/0 (socket says 172.21.15.81:51082) 2023-11-08T16:34:33.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.937+0000 7f3620be1640 1 -- 172.21.15.81:0/3191309580 learned_addr learned my addr 172.21.15.81:0/3191309580 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f361bfff640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3608003a50 0x7f361c157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f3613fff640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f361c0b15c0 0x7f361c0b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f3620be1640 1 -- 172.21.15.81:0/3191309580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3608003a50 msgr2=0x7f361c157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f3620be1640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3608003a50 0x7f361c157030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f3620be1640 1 -- 172.21.15.81:0/3191309580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f361c0b15c0 msgr2=0x7f361c0b38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:33.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f3620be1640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f361c0b15c0 0x7f361c0b38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:33.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f3620be1640 1 -- 172.21.15.81:0/3191309580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f361c15f050 con 0x7f361c0b1a40 2023-11-08T16:34:33.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f3620be1640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f361c0b1a40 0x7f361c0b3e30 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f361c0cc0d0 tx=0x7f361400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:33.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f36197fa640 1 -- 172.21.15.81:0/3191309580 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614007e20 con 0x7f361c0b1a40 2023-11-08T16:34:33.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f36197fa640 1 -- 172.21.15.81:0/3191309580 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f361400f040 con 0x7f361c0b1a40 2023-11-08T16:34:33.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f3622644640 1 -- 172.21.15.81:0/3191309580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f361c15f330 con 0x7f361c0b1a40 2023-11-08T16:34:33.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f36197fa640 1 -- 172.21.15.81:0/3191309580 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614003bd0 con 0x7f361c0b1a40 2023-11-08T16:34:33.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.938+0000 7f3622644640 1 -- 172.21.15.81:0/3191309580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361c0b4460 con 0x7f361c0b1a40 2023-11-08T16:34:33.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.939+0000 7f3622644640 1 -- 172.21.15.81:0/3191309580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f361c04ece0 con 0x7f361c0b1a40 2023-11-08T16:34:33.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.940+0000 7f36197fa640 1 -- 172.21.15.81:0/3191309580 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3614003d30 con 0x7f361c0b1a40 2023-11-08T16:34:33.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.941+0000 7f36197fa640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f35fc06aea0 0x7f35fc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:33.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.941+0000 7f36197fa640 1 -- 172.21.15.81:0/3191309580 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f36140895f0 con 0x7f361c0b1a40 2023-11-08T16:34:33.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.942+0000 7f361bfff640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f35fc06aea0 0x7f35fc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:33.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.942+0000 7f361bfff640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f35fc06aea0 0x7f35fc06d360 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f360c034000 tx=0x7f360c007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:33.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:33.946+0000 7f36197fa640 1 -- 172.21.15.81:0/3191309580 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36140544b0 con 0x7f361c0b1a40 2023-11-08T16:34:34.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.137+0000 7f3622644640 1 -- 172.21.15.81:0/3191309580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f361c15fac0 con 0x7f361c0b1a40 2023-11-08T16:34:34.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.139+0000 7f36197fa640 1 -- 172.21.15.81:0/3191309580 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+18383 (secure 0 0 0) 0x7f3614057ad0 con 0x7f361c0b1a40 2023-11-08T16:34:34.142 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:34.142 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":62,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:33:43.881792+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:32:41.268027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T16:32:42.449607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:32:53.335342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.e","mappings":[{"from":0,"to":6}]},{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"9.11","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:34:34.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.142+0000 7f3622644640 1 -- 172.21.15.81:0/3191309580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f35fc06aea0 msgr2=0x7f35fc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:34.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.142+0000 7f3622644640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f35fc06aea0 0x7f35fc06d360 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f360c034000 tx=0x7f360c007000 comp rx=0 tx=0).stop 2023-11-08T16:34:34.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.142+0000 7f3622644640 1 -- 172.21.15.81:0/3191309580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f361c0b1a40 msgr2=0x7f361c0b3e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:34.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.142+0000 7f3622644640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f361c0b1a40 0x7f361c0b3e30 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f361c0cc0d0 tx=0x7f361400d4a0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.142+0000 7f3622644640 1 -- 172.21.15.81:0/3191309580 shutdown_connections 2023-11-08T16:34:34.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.142+0000 7f3622644640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f35fc06aea0 0x7f35fc06d360 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.142+0000 7f3622644640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f361c0b1a40 0x7f361c0b3e30 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.142+0000 7f3622644640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f361c0b15c0 0x7f361c0b38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.142+0000 7f3622644640 1 --2- 172.21.15.81:0/3191309580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3608003a50 0x7f361c157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.142+0000 7f3622644640 1 -- 172.21.15.81:0/3191309580 >> 172.21.15.81:0/3191309580 conn(0x7f361c0cb490 msgr2=0x7f361c0ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:34.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.143+0000 7f3622644640 1 -- 172.21.15.81:0/3191309580 shutdown_connections 2023-11-08T16:34:34.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.143+0000 7f3622644640 1 -- 172.21.15.81:0/3191309580 wait complete. 2023-11-08T16:34:34.162 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:32:41.268027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:32:42.449607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:32:53.335342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:34:34.163 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:34:34.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.373+0000 7fe4da7c7640 1 Processor -- start 2023-11-08T16:34:34.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.374+0000 7fe4da7c7640 1 -- start start 2023-11-08T16:34:34.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.375+0000 7fe4da7c7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4d415a0e0 0x7fe4d40ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.375+0000 7fe4da7c7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe4d40ca720 0x7fe4d406b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.375+0000 7fe4da7c7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d40ce960 con 0x7fe4d40c9bf0 2023-11-08T16:34:34.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.375+0000 7fe4da7c7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d4071980 con 0x7fe4d40ca720 2023-11-08T16:34:34.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.375+0000 7fe4da7c7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40c9480 con 0x7fe4d415a0e0 2023-11-08T16:34:34.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.376+0000 7fe4cbfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe4d40c9bf0 0x7fe4d40ca010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58954/0 (socket says 172.21.15.81:58954) 2023-11-08T16:34:34.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.376+0000 7fe4cbfff640 1 -- 172.21.15.81:0/2613323566 learned_addr learned my addr 172.21.15.81:0/2613323566 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:34.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.376+0000 7fe4d8d64640 1 --2- 172.21.15.81:0/2613323566 >> v2:172.21.15.81:3300/0 conn(0x7fe4d40ca720 0x7fe4d406b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.376+0000 7fe4d3fff640 1 --2- 172.21.15.81:0/2613323566 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4d415a0e0 0x7fe4d40ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.376+0000 7fe4d37fe640 1 -- 172.21.15.81:0/2613323566 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911817824 0 0) 0x7fe4bc001110 con 0x7fe4d40c9bf0 2023-11-08T16:34:34.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.376+0000 7fe4d8d64640 1 -- 172.21.15.81:0/2613323566 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4d415a0e0 msgr2=0x7fe4d40ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:34.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.376+0000 7fe4d8d64640 1 --2- 172.21.15.81:0/2613323566 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4d415a0e0 0x7fe4d40ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.376+0000 7fe4d8d64640 1 -- 172.21.15.81:0/2613323566 >> v1:172.21.15.81:6789/0 conn(0x7fe4d40c9bf0 legacy=0x7fe4d40ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:34.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.376+0000 7fe4d8d64640 1 -- 172.21.15.81:0/2613323566 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d415a4c0 con 0x7fe4d40ca720 2023-11-08T16:34:34.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.377+0000 7fe4d8d64640 1 --2- 172.21.15.81:0/2613323566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4d40ca720 0x7fe4d406b710 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009d90 tx=0x7fe4cc02e140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08a4d782da28525 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:34.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.378+0000 7fe4d37fe640 1 -- 172.21.15.81:0/2613323566 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc036440 con 0x7fe4d40ca720 2023-11-08T16:34:34.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.378+0000 7fe4d37fe640 1 -- 172.21.15.81:0/2613323566 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc0365a0 con 0x7fe4d40ca720 2023-11-08T16:34:34.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.378+0000 7fe4d37fe640 1 -- 172.21.15.81:0/2613323566 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc036930 con 0x7fe4d40ca720 2023-11-08T16:34:34.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.378+0000 7fe4da7c7640 1 -- 172.21.15.81:0/2613323566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4d40ca720 msgr2=0x7fe4d406b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:34.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.378+0000 7fe4da7c7640 1 --2- 172.21.15.81:0/2613323566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4d40ca720 0x7fe4d406b710 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009d90 tx=0x7fe4cc02e140 comp rx=0 tx=0).stop 2023-11-08T16:34:34.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.378+0000 7fe4da7c7640 1 -- 172.21.15.81:0/2613323566 shutdown_connections 2023-11-08T16:34:34.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.378+0000 7fe4da7c7640 1 --2- 172.21.15.81:0/2613323566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4d40ca720 0x7fe4d406b710 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.378+0000 7fe4da7c7640 1 --2- 172.21.15.81:0/2613323566 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4d415a0e0 0x7fe4d40ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.378+0000 7fe4da7c7640 1 -- 172.21.15.81:0/2613323566 >> 172.21.15.81:0/2613323566 conn(0x7fe4d40cb580 msgr2=0x7fe4d40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:34.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.378+0000 7fe4da7c7640 1 -- 172.21.15.81:0/2613323566 shutdown_connections 2023-11-08T16:34:34.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.378+0000 7fe4da7c7640 1 -- 172.21.15.81:0/2613323566 wait complete. 2023-11-08T16:34:34.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.380+0000 7fe4da7c7640 1 Processor -- start 2023-11-08T16:34:34.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.380+0000 7fe4da7c7640 1 -- start start 2023-11-08T16:34:34.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.381+0000 7fe4da7c7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4d40c9bf0 0x7fe4d40c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.381+0000 7fe4da7c7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4d40ca720 0x7fe4d41ee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.381+0000 7fe4da7c7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4d415a0e0 0x7fe4d40c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.381+0000 7fe4da7c7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40c5460 con 0x7fe4d415a0e0 2023-11-08T16:34:34.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.381+0000 7fe4da7c7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40c4720 con 0x7fe4d40c9bf0 2023-11-08T16:34:34.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.381+0000 7fe4da7c7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d40c6f20 con 0x7fe4d40ca720 2023-11-08T16:34:34.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.381+0000 7fe4d8d64640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4d415a0e0 0x7fe4d40c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.381+0000 7fe4d8d64640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4d415a0e0 0x7fe4d40c4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51096/0 (socket says 172.21.15.81:51096) 2023-11-08T16:34:34.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.381+0000 7fe4d8d64640 1 -- 172.21.15.81:0/846680456 learned_addr learned my addr 172.21.15.81:0/846680456 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:34.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.382+0000 7fe4cbfff640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4d40ca720 0x7fe4d41ee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.382+0000 7fe4d8d64640 1 -- 172.21.15.81:0/846680456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4d40ca720 msgr2=0x7fe4d41ee1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:34.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.382+0000 7fe4d8d64640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4d40ca720 0x7fe4d41ee1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.382+0000 7fe4d8d64640 1 -- 172.21.15.81:0/846680456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4d40c9bf0 msgr2=0x7fe4d40c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:34.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.382+0000 7fe4d8d64640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4d40c9bf0 0x7fe4d40c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.382+0000 7fe4d8d64640 1 -- 172.21.15.81:0/846680456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4cc009a70 con 0x7fe4d415a0e0 2023-11-08T16:34:34.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.382+0000 7fe4cbfff640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4d40ca720 0x7fe4d41ee1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:34.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.382+0000 7fe4d8d64640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4d415a0e0 0x7fe4d40c4f20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009d90 tx=0x7fe4cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:34.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.382+0000 7fe4d17fa640 1 -- 172.21.15.81:0/846680456 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc03f850 con 0x7fe4d415a0e0 2023-11-08T16:34:34.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.383+0000 7fe4d17fa640 1 -- 172.21.15.81:0/846680456 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc03f9b0 con 0x7fe4d415a0e0 2023-11-08T16:34:34.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.383+0000 7fe4d17fa640 1 -- 172.21.15.81:0/846680456 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc03fd60 con 0x7fe4d415a0e0 2023-11-08T16:34:34.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.383+0000 7fe4da7c7640 1 -- 172.21.15.81:0/846680456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d40c7200 con 0x7fe4d415a0e0 2023-11-08T16:34:34.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.383+0000 7fe4da7c7640 1 -- 172.21.15.81:0/846680456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d40c6bd0 con 0x7fe4d415a0e0 2023-11-08T16:34:34.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.384+0000 7fe4da7c7640 1 -- 172.21.15.81:0/846680456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d40c6d60 con 0x7fe4d415a0e0 2023-11-08T16:34:34.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.385+0000 7fe4d17fa640 1 -- 172.21.15.81:0/846680456 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe4cc03d070 con 0x7fe4d415a0e0 2023-11-08T16:34:34.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.385+0000 7fe4d17fa640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4b406ae70 0x7fe4b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.386+0000 7fe4d17fa640 1 -- 172.21.15.81:0/846680456 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7fe4cc0ac120 con 0x7fe4d415a0e0 2023-11-08T16:34:34.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.386+0000 7fe4d3fff640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4b406ae70 0x7fe4b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.386+0000 7fe4d3fff640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4b406ae70 0x7fe4b406d330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4006fd0 tx=0x7fe4c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:34.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.390+0000 7fe4d17fa640 1 -- 172.21.15.81:0/846680456 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4cc076f40 con 0x7fe4d415a0e0 2023-11-08T16:34:34.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.575+0000 7fe4da7c7640 1 -- 172.21.15.81:0/846680456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4d4071420 con 0x7fe4d415a0e0 2023-11-08T16:34:34.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.576+0000 7fe4d17fa640 1 -- 172.21.15.81:0/846680456 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4cc07a560 con 0x7fe4d415a0e0 2023-11-08T16:34:34.578 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:34:34.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.578+0000 7fe4da7c7640 1 -- 172.21.15.81:0/846680456 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4b406ae70 msgr2=0x7fe4b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:34.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.578+0000 7fe4da7c7640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4b406ae70 0x7fe4b406d330 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4006fd0 tx=0x7fe4c4008040 comp rx=0 tx=0).stop 2023-11-08T16:34:34.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.578+0000 7fe4da7c7640 1 -- 172.21.15.81:0/846680456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4d415a0e0 msgr2=0x7fe4d40c4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:34.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.578+0000 7fe4da7c7640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4d415a0e0 0x7fe4d40c4f20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc009d90 tx=0x7fe4cc004a40 comp rx=0 tx=0).stop 2023-11-08T16:34:34.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.578+0000 7fe4da7c7640 1 -- 172.21.15.81:0/846680456 shutdown_connections 2023-11-08T16:34:34.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.578+0000 7fe4da7c7640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4b406ae70 0x7fe4b406d330 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.578+0000 7fe4da7c7640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe4d415a0e0 0x7fe4d40c4f20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.578+0000 7fe4da7c7640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4d40ca720 0x7fe4d41ee1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.578+0000 7fe4da7c7640 1 --2- 172.21.15.81:0/846680456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4d40c9bf0 0x7fe4d40c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.578+0000 7fe4da7c7640 1 -- 172.21.15.81:0/846680456 >> 172.21.15.81:0/846680456 conn(0x7fe4d40cb580 msgr2=0x7fe4d4159310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:34.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.579+0000 7fe4da7c7640 1 -- 172.21.15.81:0/846680456 shutdown_connections 2023-11-08T16:34:34.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.579+0000 7fe4da7c7640 1 -- 172.21.15.81:0/846680456 wait complete. 2023-11-08T16:34:34.598 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:34:34.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.811+0000 7fcee4739640 1 Processor -- start 2023-11-08T16:34:34.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.811+0000 7fcee4739640 1 -- start start 2023-11-08T16:34:34.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.812+0000 7fcee4739640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcedc159920 0x7fcedc15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.812+0000 7fcee4739640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcedc06d330 0x7fcedc0cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.812+0000 7fcee4739640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcedc0cd7d0 0x7fcedc15d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.812+0000 7fcee4739640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc0ce5e0 con 0x7fcedc06d330 2023-11-08T16:34:34.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.812+0000 7fcee4739640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcedc0cdc60 con 0x7fcedc0cd7d0 2023-11-08T16:34:34.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.812+0000 7fcee4739640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc06fc70 con 0x7fcedc159920 2023-11-08T16:34:34.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.813+0000 7fcee1cd4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcedc06d330 0x7fcedc0cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.813+0000 7fcee1cd4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcedc06d330 0x7fcedc0cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51110/0 (socket says 172.21.15.81:51110) 2023-11-08T16:34:34.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.813+0000 7fcee1cd4640 1 -- 172.21.15.81:0/936181924 learned_addr learned my addr 172.21.15.81:0/936181924 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:34.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.813+0000 7fcee24d5640 1 --2- 172.21.15.81:0/936181924 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcedc159920 0x7fcedc15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.813+0000 7fcee2cd6640 1 --2- 172.21.15.81:0/936181924 >> v2:172.21.15.93:3300/0 conn(0x7fcedc0cd7d0 0x7fcedc15d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.813+0000 7fcee1cd4640 1 -- 172.21.15.81:0/936181924 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcedc159920 msgr2=0x7fcedc15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:34.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.813+0000 7fcee1cd4640 1 --2- 172.21.15.81:0/936181924 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcedc159920 0x7fcedc15a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.813+0000 7fcee1cd4640 1 -- 172.21.15.81:0/936181924 >> v2:172.21.15.93:3300/0 conn(0x7fcedc0cd7d0 msgr2=0x7fcedc15d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:34.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.813+0000 7fcee1cd4640 1 --2- 172.21.15.81:0/936181924 >> v2:172.21.15.93:3300/0 conn(0x7fcedc0cd7d0 0x7fcedc15d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.813+0000 7fcee1cd4640 1 -- 172.21.15.81:0/936181924 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcedc15ae00 con 0x7fcedc06d330 2023-11-08T16:34:34.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.814+0000 7fcee1cd4640 1 --2- 172.21.15.81:0/936181924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcedc06d330 0x7fcedc0cd290 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0cb030 tx=0x7fced002e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ebcb3c5f2bced9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:34.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.815+0000 7fcee14d3640 1 -- 172.21.15.81:0/936181924 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced002ee70 con 0x7fcedc06d330 2023-11-08T16:34:34.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.815+0000 7fcee14d3640 1 -- 172.21.15.81:0/936181924 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced0036440 con 0x7fcedc06d330 2023-11-08T16:34:34.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.815+0000 7fcee14d3640 1 -- 172.21.15.81:0/936181924 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced00367d0 con 0x7fcedc06d330 2023-11-08T16:34:34.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.815+0000 7fcee4739640 1 -- 172.21.15.81:0/936181924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcedc06d330 msgr2=0x7fcedc0cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:34.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.815+0000 7fcee4739640 1 --2- 172.21.15.81:0/936181924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcedc06d330 0x7fcedc0cd290 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0cb030 tx=0x7fced002e0c0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.815+0000 7fcee4739640 1 -- 172.21.15.81:0/936181924 shutdown_connections 2023-11-08T16:34:34.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.815+0000 7fcee4739640 1 --2- 172.21.15.81:0/936181924 >> v2:172.21.15.93:3300/0 conn(0x7fcedc0cd7d0 0x7fcedc15d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.815+0000 7fcee4739640 1 --2- 172.21.15.81:0/936181924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcedc06d330 0x7fcedc0cd290 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.815+0000 7fcee4739640 1 --2- 172.21.15.81:0/936181924 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcedc159920 0x7fcedc15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.815+0000 7fcee4739640 1 -- 172.21.15.81:0/936181924 >> 172.21.15.81:0/936181924 conn(0x7fcedc0cb580 msgr2=0x7fcedc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:34.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.815+0000 7fcee4739640 1 -- 172.21.15.81:0/936181924 shutdown_connections 2023-11-08T16:34:34.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.816+0000 7fcee4739640 1 -- 172.21.15.81:0/936181924 wait complete. 2023-11-08T16:34:34.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.817+0000 7fcee4739640 1 Processor -- start 2023-11-08T16:34:34.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.817+0000 7fcee4739640 1 -- start start 2023-11-08T16:34:34.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.818+0000 7fcee4739640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcedc06d330 0x7fcedc0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.818+0000 7fcee4739640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcedc0cd7d0 0x7fcedc157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.818+0000 7fcee4739640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcedc159920 0x7fcedc06a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.818+0000 7fcee4739640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc1584a0 con 0x7fcedc159920 2023-11-08T16:34:34.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.818+0000 7fcee4739640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc14dc20 con 0x7fcedc0cd7d0 2023-11-08T16:34:34.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.818+0000 7fcee4739640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc14df00 con 0x7fcedc06d330 2023-11-08T16:34:34.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.819+0000 7fcee2cd6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcedc159920 0x7fcedc06a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.819+0000 7fcee2cd6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcedc159920 0x7fcedc06a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51122/0 (socket says 172.21.15.81:51122) 2023-11-08T16:34:34.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.819+0000 7fcee2cd6640 1 -- 172.21.15.81:0/1205985563 learned_addr learned my addr 172.21.15.81:0/1205985563 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:34.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.819+0000 7fcee24d5640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcedc06d330 0x7fcedc0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.819+0000 7fcee1cd4640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcedc0cd7d0 0x7fcedc157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.819+0000 7fcee2cd6640 1 -- 172.21.15.81:0/1205985563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcedc06d330 msgr2=0x7fcedc0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:34.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.819+0000 7fcee2cd6640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcedc06d330 0x7fcedc0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.819+0000 7fcee2cd6640 1 -- 172.21.15.81:0/1205985563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcedc0cd7d0 msgr2=0x7fcedc157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:34.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.819+0000 7fcee2cd6640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcedc0cd7d0 0x7fcedc157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:34.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.819+0000 7fcee2cd6640 1 -- 172.21.15.81:0/1205985563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced000b770 con 0x7fcedc159920 2023-11-08T16:34:34.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.819+0000 7fcee2cd6640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcedc159920 0x7fcedc06a320 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fced4007e60 tx=0x7fced400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:34.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.820+0000 7fceceffd640 1 -- 172.21.15.81:0/1205985563 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced4014620 con 0x7fcedc159920 2023-11-08T16:34:34.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.820+0000 7fceceffd640 1 -- 172.21.15.81:0/1205985563 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fced400ee90 con 0x7fcedc159920 2023-11-08T16:34:34.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.820+0000 7fceceffd640 1 -- 172.21.15.81:0/1205985563 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced40149e0 con 0x7fcedc159920 2023-11-08T16:34:34.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.820+0000 7fcee4739640 1 -- 172.21.15.81:0/1205985563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc14e1e0 con 0x7fcedc159920 2023-11-08T16:34:34.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.820+0000 7fcee4739640 1 -- 172.21.15.81:0/1205985563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc14e830 con 0x7fcedc159920 2023-11-08T16:34:34.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.821+0000 7fcee4739640 1 -- 172.21.15.81:0/1205985563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc0c99d0 con 0x7fcedc159920 2023-11-08T16:34:34.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.822+0000 7fceceffd640 1 -- 172.21.15.81:0/1205985563 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fced4014b40 con 0x7fcedc159920 2023-11-08T16:34:34.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.823+0000 7fceceffd640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcec006aea0 0x7fcec006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:34.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.823+0000 7fceceffd640 1 -- 172.21.15.81:0/1205985563 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7fced408a110 con 0x7fcedc159920 2023-11-08T16:34:34.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.823+0000 7fcee24d5640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcec006aea0 0x7fcec006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:34.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.824+0000 7fcee24d5640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcec006aea0 0x7fcec006d360 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fced8006fd0 tx=0x7fced8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:34.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:34.827+0000 7fceceffd640 1 -- 172.21.15.81:0/1205985563 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fced4054fd0 con 0x7fcedc159920 2023-11-08T16:34:35.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.022+0000 7fcee4739640 1 -- 172.21.15.81:0/1205985563 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcedc1f6f00 con 0x7fcedc159920 2023-11-08T16:34:35.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.023+0000 7fceceffd640 1 -- 172.21.15.81:0/1205985563 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcedc1f6f00 con 0x7fcedc159920 2023-11-08T16:34:35.025 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:34:35.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.025+0000 7fcee4739640 1 -- 172.21.15.81:0/1205985563 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcec006aea0 msgr2=0x7fcec006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.025+0000 7fcee4739640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcec006aea0 0x7fcec006d360 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fced8006fd0 tx=0x7fced8008040 comp rx=0 tx=0).stop 2023-11-08T16:34:35.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.025+0000 7fcee4739640 1 -- 172.21.15.81:0/1205985563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcedc159920 msgr2=0x7fcedc06a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.025+0000 7fcee4739640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcedc159920 0x7fcedc06a320 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fced4007e60 tx=0x7fced400e4b0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.026+0000 7fcee4739640 1 -- 172.21.15.81:0/1205985563 shutdown_connections 2023-11-08T16:34:35.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.026+0000 7fcee4739640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcec006aea0 0x7fcec006d360 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.026+0000 7fcee4739640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcedc159920 0x7fcedc06a320 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.026+0000 7fcee4739640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcedc0cd7d0 0x7fcedc157f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.026+0000 7fcee4739640 1 --2- 172.21.15.81:0/1205985563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcedc06d330 0x7fcedc0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.026+0000 7fcee4739640 1 -- 172.21.15.81:0/1205985563 >> 172.21.15.81:0/1205985563 conn(0x7fcedc0cb580 msgr2=0x7fcedc06fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:35.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.026+0000 7fcee4739640 1 -- 172.21.15.81:0/1205985563 shutdown_connections 2023-11-08T16:34:35.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.026+0000 7fcee4739640 1 -- 172.21.15.81:0/1205985563 wait complete. 2023-11-08T16:34:35.045 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:34:35.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.255+0000 7f5cc8244640 1 Processor -- start 2023-11-08T16:34:35.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.255+0000 7f5cc8244640 1 -- start start 2023-11-08T16:34:35.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.257+0000 7f5cc8244640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5cc015a0e0 0x7f5cc00ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.257+0000 7f5cc8244640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5cc00c9bf0 0x7f5cc00ca040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.257+0000 7f5cc8244640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cc00ca580 0x7f5cc015dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.257+0000 7f5cc8244640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc0071980 con 0x7f5cc015a0e0 2023-11-08T16:34:35.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.257+0000 7f5cc8244640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc00ce960 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.257+0000 7f5cc8244640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00c9480 con 0x7f5cc00ca580 2023-11-08T16:34:35.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.257+0000 7f5cc5fe0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5cc015a0e0 0x7f5cc00ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.257+0000 7f5cc5fe0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5cc015a0e0 0x7f5cc00ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51124/0 (socket says 172.21.15.81:51124) 2023-11-08T16:34:35.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.257+0000 7f5cc5fe0640 1 -- 172.21.15.81:0/233932104 learned_addr learned my addr 172.21.15.81:0/233932104 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:35.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.257+0000 7f5cc57df640 1 --2- 172.21.15.81:0/233932104 >> v2:172.21.15.93:3300/0 conn(0x7f5cc00c9bf0 0x7f5cc00ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.257+0000 7f5cc67e1640 1 --2- 172.21.15.81:0/233932104 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cc00ca580 0x7f5cc015dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.258+0000 7f5cc5fe0640 1 -- 172.21.15.81:0/233932104 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cc00ca580 msgr2=0x7f5cc015dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:35.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.258+0000 7f5cc5fe0640 1 --2- 172.21.15.81:0/233932104 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cc00ca580 0x7f5cc015dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.258+0000 7f5cc5fe0640 1 -- 172.21.15.81:0/233932104 >> v2:172.21.15.93:3300/0 conn(0x7f5cc00c9bf0 msgr2=0x7f5cc00ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:35.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.258+0000 7f5cc5fe0640 1 --2- 172.21.15.81:0/233932104 >> v2:172.21.15.93:3300/0 conn(0x7f5cc00c9bf0 0x7f5cc00ca040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.258+0000 7f5cc5fe0640 1 -- 172.21.15.81:0/233932104 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc015a4c0 con 0x7f5cc015a0e0 2023-11-08T16:34:35.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.258+0000 7f5cc57df640 1 --2- 172.21.15.81:0/233932104 >> v2:172.21.15.93:3300/0 conn(0x7f5cc00c9bf0 0x7f5cc00ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:35.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.259+0000 7f5cc5fe0640 1 --2- 172.21.15.81:0/233932104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cc015a0e0 0x7f5cc00ccc10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5cc006b4b0 tx=0x7f5cbc02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7591ab6f878ced server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:35.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.259+0000 7f5cc4fde640 1 -- 172.21.15.81:0/233932104 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc030850 con 0x7f5cc015a0e0 2023-11-08T16:34:35.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.259+0000 7f5cc4fde640 1 -- 172.21.15.81:0/233932104 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc0309b0 con 0x7f5cc015a0e0 2023-11-08T16:34:35.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.259+0000 7f5cc4fde640 1 -- 172.21.15.81:0/233932104 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc030d40 con 0x7f5cc015a0e0 2023-11-08T16:34:35.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.260+0000 7f5cc8244640 1 -- 172.21.15.81:0/233932104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cc015a0e0 msgr2=0x7f5cc00ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.260+0000 7f5cc8244640 1 --2- 172.21.15.81:0/233932104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cc015a0e0 0x7f5cc00ccc10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5cc006b4b0 tx=0x7f5cbc02b990 comp rx=0 tx=0).stop 2023-11-08T16:34:35.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.260+0000 7f5cc8244640 1 -- 172.21.15.81:0/233932104 shutdown_connections 2023-11-08T16:34:35.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.260+0000 7f5cc8244640 1 --2- 172.21.15.81:0/233932104 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cc00ca580 0x7f5cc015dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.260+0000 7f5cc8244640 1 --2- 172.21.15.81:0/233932104 >> v2:172.21.15.93:3300/0 conn(0x7f5cc00c9bf0 0x7f5cc00ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.260+0000 7f5cc8244640 1 --2- 172.21.15.81:0/233932104 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cc015a0e0 0x7f5cc00ccc10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.260+0000 7f5cc8244640 1 -- 172.21.15.81:0/233932104 >> 172.21.15.81:0/233932104 conn(0x7f5cc00cb580 msgr2=0x7f5cc00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:35.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.260+0000 7f5cc8244640 1 -- 172.21.15.81:0/233932104 shutdown_connections 2023-11-08T16:34:35.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.260+0000 7f5cc8244640 1 -- 172.21.15.81:0/233932104 wait complete. 2023-11-08T16:34:35.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.262+0000 7f5cc8244640 1 Processor -- start 2023-11-08T16:34:35.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.262+0000 7f5cc8244640 1 -- start start 2023-11-08T16:34:35.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.263+0000 7f5cc8244640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cc00c9bf0 0x7f5cc014d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.263+0000 7f5cc8244640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cc00ca580 0x7f5cc015c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.263+0000 7f5cc8244640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cc015a0e0 0x7f5cc015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.263+0000 7f5cc8244640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc013f740 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.263+0000 7f5cc8244640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc013eb90 con 0x7f5cc015a0e0 2023-11-08T16:34:35.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.263+0000 7f5cc8244640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc013ee10 con 0x7f5cc00ca580 2023-11-08T16:34:35.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.263+0000 7f5cc5fe0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cc00c9bf0 0x7f5cc014d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.263+0000 7f5cc5fe0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cc00c9bf0 0x7f5cc014d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51136/0 (socket says 172.21.15.81:51136) 2023-11-08T16:34:35.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.263+0000 7f5cc5fe0640 1 -- 172.21.15.81:0/507251779 learned_addr learned my addr 172.21.15.81:0/507251779 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:35.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.263+0000 7f5cc57df640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cc00ca580 0x7f5cc015c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cc67e1640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cc015a0e0 0x7f5cc015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cc5fe0640 1 -- 172.21.15.81:0/507251779 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cc00ca580 msgr2=0x7f5cc015c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cc5fe0640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cc00ca580 0x7f5cc015c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cc5fe0640 1 -- 172.21.15.81:0/507251779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cc015a0e0 msgr2=0x7f5cc015c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cc5fe0640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cc015a0e0 0x7f5cc015c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cc5fe0640 1 -- 172.21.15.81:0/507251779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc0094d0 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cc67e1640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cc015a0e0 0x7f5cc015c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:35.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cc5fe0640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cc00c9bf0 0x7f5cc014d060 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc000c00 tx=0x7f5cbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:35.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cb67fc640 1 -- 172.21.15.81:0/507251779 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc030840 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cb67fc640 1 -- 172.21.15.81:0/507251779 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cbc044480 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cb67fc640 1 -- 172.21.15.81:0/507251779 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc044830 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.264+0000 7f5cc8244640 1 -- 172.21.15.81:0/507251779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc013f0f0 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.265+0000 7f5cc8244640 1 -- 172.21.15.81:0/507251779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc00b3990 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.266+0000 7f5cc8244640 1 -- 172.21.15.81:0/507251779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc004ece0 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.267+0000 7f5cb67fc640 1 -- 172.21.15.81:0/507251779 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5cbc02e070 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.267+0000 7f5cb67fc640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ca406ae70 0x7f5ca406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.268+0000 7f5cb67fc640 1 -- 172.21.15.81:0/507251779 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f5cbc03e070 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.268+0000 7f5cc57df640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ca406ae70 0x7f5ca406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.268+0000 7f5cc57df640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ca406ae70 0x7f5ca406d330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5cc015d180 tx=0x7f5cb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:35.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.272+0000 7f5cb67fc640 1 -- 172.21.15.81:0/507251779 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5cbc077240 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.460+0000 7f5cc8244640 1 -- 172.21.15.81:0/507251779 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5cc00c5440 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.461+0000 7f5cb67fc640 1 -- 172.21.15.81:0/507251779 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5cbc07a860 con 0x7f5cc00c9bf0 2023-11-08T16:34:35.463 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:34:35.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 -- 172.21.15.81:0/507251779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ca406ae70 msgr2=0x7f5ca406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ca406ae70 0x7f5ca406d330 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5cc015d180 tx=0x7f5cb0008040 comp rx=0 tx=0).stop 2023-11-08T16:34:35.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 -- 172.21.15.81:0/507251779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cc00c9bf0 msgr2=0x7f5cc014d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cc00c9bf0 0x7f5cc014d060 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc000c00 tx=0x7f5cbc004730 comp rx=0 tx=0).stop 2023-11-08T16:34:35.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 -- 172.21.15.81:0/507251779 shutdown_connections 2023-11-08T16:34:35.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ca406ae70 0x7f5ca406d330 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5cc015a0e0 0x7f5cc015c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5cc00ca580 0x7f5cc015c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 --2- 172.21.15.81:0/507251779 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5cc00c9bf0 0x7f5cc014d060 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 -- 172.21.15.81:0/507251779 >> 172.21.15.81:0/507251779 conn(0x7f5cc00cb580 msgr2=0x7f5cc0153f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:35.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 -- 172.21.15.81:0/507251779 shutdown_connections 2023-11-08T16:34:35.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.464+0000 7f5cc8244640 1 -- 172.21.15.81:0/507251779 wait complete. 2023-11-08T16:34:35.484 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:34:35.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.695+0000 7fd201bc9640 1 Processor -- start 2023-11-08T16:34:35.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.695+0000 7fd201bc9640 1 -- start start 2023-11-08T16:34:35.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.696+0000 7fd201bc9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd1fc0ce0d0 0x7fd1fc0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.696+0000 7fd201bc9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd1fc15f280 0x7fd1fc1516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.696+0000 7fd201bc9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1fc06fa30 con 0x7fd1fc15bfc0 2023-11-08T16:34:35.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.696+0000 7fd201bc9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1fc0cda10 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.696+0000 7fd201bc9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1fc0ce4b0 con 0x7fd1fc15f280 2023-11-08T16:34:35.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.697+0000 7fd1f3fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd1fc15bfc0 0x7fd1fc15eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:58962/0 (socket says 172.21.15.81:58962) 2023-11-08T16:34:35.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.697+0000 7fd1f3fff640 1 -- 172.21.15.81:0/2054824738 learned_addr learned my addr 172.21.15.81:0/2054824738 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:35.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.697+0000 7fd1fb7fe640 1 --2- 172.21.15.81:0/2054824738 >> v2:172.21.15.81:3300/0 conn(0x7fd1fc0ce0d0 0x7fd1fc0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.697+0000 7fd1fbfff640 1 --2- 172.21.15.81:0/2054824738 >> v2:172.21.15.93:3300/0 conn(0x7fd1fc15f280 0x7fd1fc1516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.697+0000 7fd1faffd640 1 -- 172.21.15.81:0/2054824738 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966374463 0 0) 0x7fd1fc06fa30 con 0x7fd1fc15bfc0 2023-11-08T16:34:35.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.697+0000 7fd1fb7fe640 1 -- 172.21.15.81:0/2054824738 >> v2:172.21.15.93:3300/0 conn(0x7fd1fc15f280 msgr2=0x7fd1fc1516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:35.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.697+0000 7fd1fb7fe640 1 --2- 172.21.15.81:0/2054824738 >> v2:172.21.15.93:3300/0 conn(0x7fd1fc15f280 0x7fd1fc1516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.697+0000 7fd1fb7fe640 1 -- 172.21.15.81:0/2054824738 >> v1:172.21.15.81:6789/0 conn(0x7fd1fc15bfc0 legacy=0x7fd1fc15eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.697+0000 7fd1fb7fe640 1 -- 172.21.15.81:0/2054824738 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc06fca0 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.698+0000 7fd1fb7fe640 1 --2- 172.21.15.81:0/2054824738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1fc0ce0d0 0x7fd1fc0caf20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc15eaa0 tx=0x7fd1ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4790c623453e7f1f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:35.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.699+0000 7fd1faffd640 1 -- 172.21.15.81:0/2054824738 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec031830 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.699+0000 7fd1faffd640 1 -- 172.21.15.81:0/2054824738 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec031990 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.699+0000 7fd1faffd640 1 -- 172.21.15.81:0/2054824738 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec031d20 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.699+0000 7fd201bc9640 1 -- 172.21.15.81:0/2054824738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1fc0ce0d0 msgr2=0x7fd1fc0caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.699+0000 7fd201bc9640 1 --2- 172.21.15.81:0/2054824738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1fc0ce0d0 0x7fd1fc0caf20 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc15eaa0 tx=0x7fd1ec02b8d0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.700+0000 7fd201bc9640 1 -- 172.21.15.81:0/2054824738 shutdown_connections 2023-11-08T16:34:35.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.700+0000 7fd201bc9640 1 --2- 172.21.15.81:0/2054824738 >> v2:172.21.15.93:3300/0 conn(0x7fd1fc15f280 0x7fd1fc1516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.700+0000 7fd201bc9640 1 --2- 172.21.15.81:0/2054824738 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1fc0ce0d0 0x7fd1fc0caf20 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.700+0000 7fd201bc9640 1 -- 172.21.15.81:0/2054824738 >> 172.21.15.81:0/2054824738 conn(0x7fd1fc0cb580 msgr2=0x7fd1fc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:35.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.700+0000 7fd201bc9640 1 -- 172.21.15.81:0/2054824738 shutdown_connections 2023-11-08T16:34:35.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.700+0000 7fd201bc9640 1 -- 172.21.15.81:0/2054824738 wait complete. 2023-11-08T16:34:35.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.701+0000 7fd201bc9640 1 Processor -- start 2023-11-08T16:34:35.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.701+0000 7fd201bc9640 1 -- start start 2023-11-08T16:34:35.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.702+0000 7fd201bc9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1fc0ce0d0 0x7fd1fc0c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.702+0000 7fd201bc9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1fc15bfc0 0x7fd1fc0b1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.702+0000 7fd201bc9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd1fc15f280 0x7fd1fc149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.702+0000 7fd201bc9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc1f1420 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.702+0000 7fd201bc9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc0c5d50 con 0x7fd1fc15bfc0 2023-11-08T16:34:35.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.702+0000 7fd201bc9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1fc0c6030 con 0x7fd1fc15f280 2023-11-08T16:34:35.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.703+0000 7fd1fb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1fc0ce0d0 0x7fd1fc0c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.703+0000 7fd1fb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1fc0ce0d0 0x7fd1fc0c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51148/0 (socket says 172.21.15.81:51148) 2023-11-08T16:34:35.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.703+0000 7fd1fb7fe640 1 -- 172.21.15.81:0/3375206701 learned_addr learned my addr 172.21.15.81:0/3375206701 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:35.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.703+0000 7fd1f3fff640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1fc15bfc0 0x7fd1fc0b1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.703+0000 7fd1fbfff640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd1fc15f280 0x7fd1fc149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.703+0000 7fd1fb7fe640 1 -- 172.21.15.81:0/3375206701 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd1fc15f280 msgr2=0x7fd1fc149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.703+0000 7fd1fb7fe640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd1fc15f280 0x7fd1fc149a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.703+0000 7fd1fb7fe640 1 -- 172.21.15.81:0/3375206701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1fc15bfc0 msgr2=0x7fd1fc0b1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.703+0000 7fd1fb7fe640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1fc15bfc0 0x7fd1fc0b1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.703+0000 7fd1fb7fe640 1 -- 172.21.15.81:0/3375206701 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1ec009400 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.704+0000 7fd1fb7fe640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1fc0ce0d0 0x7fd1fc0c76f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec000c00 tx=0x7fd1ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.704+0000 7fd1f8ff9640 1 -- 172.21.15.81:0/3375206701 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec03aa60 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.704+0000 7fd1f8ff9640 1 -- 172.21.15.81:0/3375206701 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1ec03abe0 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.704+0000 7fd1f8ff9640 1 -- 172.21.15.81:0/3375206701 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0443f0 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.704+0000 7fd201bc9640 1 -- 172.21.15.81:0/3375206701 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1fc0c6310 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.704+0000 7fd201bc9640 1 -- 172.21.15.81:0/3375206701 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1fc0c7e10 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.706+0000 7fd201bc9640 1 -- 172.21.15.81:0/3375206701 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1fc0c5bb0 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.706+0000 7fd1f8ff9640 1 -- 172.21.15.81:0/3375206701 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1ec04d450 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.708+0000 7fd1f8ff9640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd1dc06ae70 0x7fd1dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:35.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.708+0000 7fd1f8ff9640 1 -- 172.21.15.81:0/3375206701 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7fd1ec0ab7e0 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.708+0000 7fd1f3fff640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd1dc06ae70 0x7fd1dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:35.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.709+0000 7fd1f3fff640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd1dc06ae70 0x7fd1dc06d330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0b0640 tx=0x7fd1e4007b50 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:35.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.712+0000 7fd1f8ff9640 1 -- 172.21.15.81:0/3375206701 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1fc0c5bb0 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.899+0000 7fd201bc9640 1 -- 172.21.15.81:0/3375206701 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7fd1fc0c7e10 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.900+0000 7fd1f8ff9640 1 -- 172.21.15.81:0/3375206701 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v62) v1 ==== 106+0+11 (secure 0 0 0) 0x7fd1fc0c7e10 con 0x7fd1fc0ce0d0 2023-11-08T16:34:35.902 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:34:35.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 -- 172.21.15.81:0/3375206701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd1dc06ae70 msgr2=0x7fd1dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd1dc06ae70 0x7fd1dc06d330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0b0640 tx=0x7fd1e4007b50 comp rx=0 tx=0).stop 2023-11-08T16:34:35.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 -- 172.21.15.81:0/3375206701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1fc0ce0d0 msgr2=0x7fd1fc0c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:35.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1fc0ce0d0 0x7fd1fc0c76f0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec000c00 tx=0x7fd1ec0046c0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 -- 172.21.15.81:0/3375206701 shutdown_connections 2023-11-08T16:34:35.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd1dc06ae70 0x7fd1dc06d330 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd1fc15f280 0x7fd1fc149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1fc15bfc0 0x7fd1fc0b1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 --2- 172.21.15.81:0/3375206701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1fc0ce0d0 0x7fd1fc0c76f0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:35.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 -- 172.21.15.81:0/3375206701 >> 172.21.15.81:0/3375206701 conn(0x7fd1fc0cb580 msgr2=0x7fd1fc0aefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:35.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 -- 172.21.15.81:0/3375206701 shutdown_connections 2023-11-08T16:34:35.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:35.903+0000 7fd201bc9640 1 -- 172.21.15.81:0/3375206701 wait complete. 2023-11-08T16:34:35.923 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:34:36.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.125+0000 7f98ee8af640 1 Processor -- start 2023-11-08T16:34:36.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.125+0000 7f98ee8af640 1 -- start start 2023-11-08T16:34:36.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.126+0000 7f98ee8af640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f98e80cf720 0x7f98e806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:36.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.126+0000 7f98ee8af640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f98e80ca130 0x7f98e8070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:36.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.126+0000 7f98ee8af640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e806cc50 con 0x7f98e815d3b0 2023-11-08T16:34:36.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.126+0000 7f98ee8af640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e80c90f0 con 0x7f98e80cf720 2023-11-08T16:34:36.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.126+0000 7f98ee8af640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98e80adb70 con 0x7f98e80ca130 2023-11-08T16:34:36.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.126+0000 7f98dffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f98e80cf720 0x7f98e806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:36.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.126+0000 7f98dffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f98e80cf720 0x7f98e806d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51160/0 (socket says 172.21.15.81:51160) 2023-11-08T16:34:36.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.126+0000 7f98dffff640 1 -- 172.21.15.81:0/2960026942 learned_addr learned my addr 172.21.15.81:0/2960026942 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:36.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.127+0000 7f98ece4c640 1 --2- 172.21.15.81:0/2960026942 >> v2:172.21.15.93:3300/0 conn(0x7f98e80ca130 0x7f98e8070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:36.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.127+0000 7f98dffff640 1 -- 172.21.15.81:0/2960026942 >> v2:172.21.15.93:3300/0 conn(0x7f98e80ca130 msgr2=0x7f98e8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:36.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.127+0000 7f98dffff640 1 --2- 172.21.15.81:0/2960026942 >> v2:172.21.15.93:3300/0 conn(0x7f98e80ca130 0x7f98e8070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:36.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.127+0000 7f98dffff640 1 -- 172.21.15.81:0/2960026942 >> v1:172.21.15.93:6789/0 conn(0x7f98e815d3b0 legacy=0x7f98e80c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:36.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.127+0000 7f98dffff640 1 -- 172.21.15.81:0/2960026942 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e80c9360 con 0x7f98e80cf720 2023-11-08T16:34:36.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.128+0000 7f98dffff640 1 --2- 172.21.15.81:0/2960026942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98e80cf720 0x7f98e806d0e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f98e806baf0 tx=0x7f98d002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca380b6f223461d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:36.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.128+0000 7f98e77fe640 1 -- 172.21.15.81:0/2960026942 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d002ee90 con 0x7f98e80cf720 2023-11-08T16:34:36.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.128+0000 7f98e77fe640 1 -- 172.21.15.81:0/2960026942 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d0036440 con 0x7f98e80cf720 2023-11-08T16:34:36.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.128+0000 7f98e77fe640 1 -- 172.21.15.81:0/2960026942 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d00367d0 con 0x7f98e80cf720 2023-11-08T16:34:36.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.129+0000 7f98ee8af640 1 -- 172.21.15.81:0/2960026942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98e80cf720 msgr2=0x7f98e806d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:36.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.129+0000 7f98ee8af640 1 --2- 172.21.15.81:0/2960026942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98e80cf720 0x7f98e806d0e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f98e806baf0 tx=0x7f98d002e0e0 comp rx=0 tx=0).stop 2023-11-08T16:34:36.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.129+0000 7f98ee8af640 1 -- 172.21.15.81:0/2960026942 shutdown_connections 2023-11-08T16:34:36.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.129+0000 7f98ee8af640 1 --2- 172.21.15.81:0/2960026942 >> v2:172.21.15.93:3300/0 conn(0x7f98e80ca130 0x7f98e8070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:36.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.129+0000 7f98ee8af640 1 --2- 172.21.15.81:0/2960026942 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98e80cf720 0x7f98e806d0e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:36.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.129+0000 7f98ee8af640 1 -- 172.21.15.81:0/2960026942 >> 172.21.15.81:0/2960026942 conn(0x7f98e80ceaa0 msgr2=0x7f98e80cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:36.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.129+0000 7f98ee8af640 1 -- 172.21.15.81:0/2960026942 shutdown_connections 2023-11-08T16:34:36.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.129+0000 7f98ee8af640 1 -- 172.21.15.81:0/2960026942 wait complete. 2023-11-08T16:34:36.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.131+0000 7f98ee8af640 1 Processor -- start 2023-11-08T16:34:36.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.131+0000 7f98ee8af640 1 -- start start 2023-11-08T16:34:36.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98ee8af640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98e80ca130 0x7f98e80b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:36.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98ee8af640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98e80cf720 0x7f98e80c7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:36.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98ee8af640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98e815d3b0 0x7f98e80c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:36.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98ee8af640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80c36a0 con 0x7f98e80cf720 2023-11-08T16:34:36.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98ee8af640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80b45a0 con 0x7f98e80ca130 2023-11-08T16:34:36.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98ee8af640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e80c33c0 con 0x7f98e815d3b0 2023-11-08T16:34:36.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98dffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98e80cf720 0x7f98e80c7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:36.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98dffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98e80cf720 0x7f98e80c7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51170/0 (socket says 172.21.15.81:51170) 2023-11-08T16:34:36.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98dffff640 1 -- 172.21.15.81:0/1368837055 learned_addr learned my addr 172.21.15.81:0/1368837055 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:36.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98dffff640 1 -- 172.21.15.81:0/1368837055 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98e815d3b0 msgr2=0x7f98e80c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:36.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98dffff640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98e815d3b0 0x7f98e80c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:36.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98dffff640 1 -- 172.21.15.81:0/1368837055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98e80ca130 msgr2=0x7f98e80b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:36.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98dffff640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98e80ca130 0x7f98e80b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:36.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.132+0000 7f98dffff640 1 -- 172.21.15.81:0/1368837055 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d000b770 con 0x7f98e80cf720 2023-11-08T16:34:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.133+0000 7f98dffff640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98e80cf720 0x7f98e80c7160 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f98d002e120 tx=0x7f98d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.133+0000 7f98e57fa640 1 -- 172.21.15.81:0/1368837055 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d00348d0 con 0x7f98e80cf720 2023-11-08T16:34:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.133+0000 7f98e57fa640 1 -- 172.21.15.81:0/1368837055 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d0034a30 con 0x7f98e80cf720 2023-11-08T16:34:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.133+0000 7f98e57fa640 1 -- 172.21.15.81:0/1368837055 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0034de0 con 0x7f98e80cf720 2023-11-08T16:34:36.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.133+0000 7f98ee8af640 1 -- 172.21.15.81:0/1368837055 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e80b1860 con 0x7f98e80cf720 2023-11-08T16:34:36.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.133+0000 7f98ee8af640 1 -- 172.21.15.81:0/1368837055 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e80b1e20 con 0x7f98e80cf720 2023-11-08T16:34:36.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.134+0000 7f98ee8af640 1 -- 172.21.15.81:0/1368837055 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e80b16a0 con 0x7f98e80cf720 2023-11-08T16:34:36.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.135+0000 7f98e57fa640 1 -- 172.21.15.81:0/1368837055 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98d003d040 con 0x7f98e80cf720 2023-11-08T16:34:36.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.136+0000 7f98e57fa640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f98c806aea0 0x7f98c806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:36.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.136+0000 7f98e57fa640 1 -- 172.21.15.81:0/1368837055 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f98d00ae600 con 0x7f98e80cf720 2023-11-08T16:34:36.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.136+0000 7f98e7fff640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f98c806aea0 0x7f98c806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:36.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.137+0000 7f98e7fff640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f98c806aea0 0x7f98c806d360 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009690 tx=0x7f98d8006b90 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:36.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.140+0000 7f98e57fa640 1 -- 172.21.15.81:0/1368837055 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f98d0079420 con 0x7f98e80cf720 2023-11-08T16:34:36.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.343+0000 7f98ee8af640 1 -- 172.21.15.81:0/1368837055 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f98e80743f0 con 0x7f98e80cf720 2023-11-08T16:34:36.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.343+0000 7f98e57fa640 1 -- 172.21.15.81:0/1368837055 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) v1 ==== 80+0+0 (secure 0 0 0) 0x7f98d007ca40 con 0x7f98e80cf720 2023-11-08T16:34:36.345 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:34:36.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.346+0000 7f98ee8af640 1 -- 172.21.15.81:0/1368837055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f98c806aea0 msgr2=0x7f98c806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:36.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.346+0000 7f98ee8af640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f98c806aea0 0x7f98c806d360 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009690 tx=0x7f98d8006b90 comp rx=0 tx=0).stop 2023-11-08T16:34:36.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.346+0000 7f98ee8af640 1 -- 172.21.15.81:0/1368837055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98e80cf720 msgr2=0x7f98e80c7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:36.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.346+0000 7f98ee8af640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98e80cf720 0x7f98e80c7160 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f98d002e120 tx=0x7f98d00046c0 comp rx=0 tx=0).stop 2023-11-08T16:34:36.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.347+0000 7f98ee8af640 1 -- 172.21.15.81:0/1368837055 shutdown_connections 2023-11-08T16:34:36.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.347+0000 7f98ee8af640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f98c806aea0 0x7f98c806d360 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:36.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.347+0000 7f98ee8af640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f98e815d3b0 0x7f98e80c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:36.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.347+0000 7f98ee8af640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f98e80cf720 0x7f98e80c7160 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:36.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.347+0000 7f98ee8af640 1 --2- 172.21.15.81:0/1368837055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f98e80ca130 0x7f98e80b4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:36.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.347+0000 7f98ee8af640 1 -- 172.21.15.81:0/1368837055 >> 172.21.15.81:0/1368837055 conn(0x7f98e80ceaa0 msgr2=0x7f98e80ca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:36.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.347+0000 7f98ee8af640 1 -- 172.21.15.81:0/1368837055 shutdown_connections 2023-11-08T16:34:36.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:36.347+0000 7f98ee8af640 1 -- 172.21.15.81:0/1368837055 wait complete. 2023-11-08T16:34:36.366 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:34:36.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:34:36.367 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:34:36.454 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[106765]: fuse finished with error 0 and tester_r 0 2023-11-08T16:34:36.501 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:34:42.604 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:34:42.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:34:42.605 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:34:42.666 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:34:42.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.881+0000 7f71ca74c640 1 Processor -- start 2023-11-08T16:34:42.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.881+0000 7f71ca74c640 1 -- start start 2023-11-08T16:34:42.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.883+0000 7f71ca74c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71c40cc5e0 0x7f71c40cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:42.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.883+0000 7f71ca74c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f71c40ccf80 0x7f71c4070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:42.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.883+0000 7f71ca74c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71c40c9a70 con 0x7f71c40adcd0 2023-11-08T16:34:42.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.883+0000 7f71ca74c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c40cc3e0 con 0x7f71c40ccf80 2023-11-08T16:34:42.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.883+0000 7f71ca74c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c406d070 con 0x7f71c40cc5e0 2023-11-08T16:34:42.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.883+0000 7f71c8ce9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f71c40ccf80 0x7f71c4070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:42.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.883+0000 7f71c8ce9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f71c40ccf80 0x7f71c4070050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47718/0 (socket says 172.21.15.81:47718) 2023-11-08T16:34:42.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.883+0000 7f71c8ce9640 1 -- 172.21.15.81:0/393178137 learned_addr learned my addr 172.21.15.81:0/393178137 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:42.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.883+0000 7f71bb7fe640 1 --2- 172.21.15.81:0/393178137 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71c40cc5e0 0x7f71c40cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:42.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.884+0000 7f71c8ce9640 1 -- 172.21.15.81:0/393178137 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71c40cc5e0 msgr2=0x7f71c40cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:42.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.884+0000 7f71c8ce9640 1 --2- 172.21.15.81:0/393178137 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71c40cc5e0 0x7f71c40cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:42.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.884+0000 7f71c8ce9640 1 -- 172.21.15.81:0/393178137 >> v1:172.21.15.93:6789/0 conn(0x7f71c40adcd0 legacy=0x7f71c40ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:42.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.884+0000 7f71c8ce9640 1 -- 172.21.15.81:0/393178137 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c406d350 con 0x7f71c40ccf80 2023-11-08T16:34:42.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.885+0000 7f71c8ce9640 1 --2- 172.21.15.81:0/393178137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71c40ccf80 0x7f71c4070050 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009870 tx=0x7f71bc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b56254d9bfe20a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:42.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.885+0000 7f71c37fe640 1 -- 172.21.15.81:0/393178137 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc031640 con 0x7f71c40ccf80 2023-11-08T16:34:42.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.885+0000 7f71c37fe640 1 -- 172.21.15.81:0/393178137 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71bc0317a0 con 0x7f71c40ccf80 2023-11-08T16:34:42.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.885+0000 7f71c37fe640 1 -- 172.21.15.81:0/393178137 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71bc031b30 con 0x7f71c40ccf80 2023-11-08T16:34:42.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.886+0000 7f71ca74c640 1 -- 172.21.15.81:0/393178137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71c40ccf80 msgr2=0x7f71c4070050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:42.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.886+0000 7f71ca74c640 1 --2- 172.21.15.81:0/393178137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71c40ccf80 0x7f71c4070050 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009870 tx=0x7f71bc02b8f0 comp rx=0 tx=0).stop 2023-11-08T16:34:42.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.886+0000 7f71ca74c640 1 -- 172.21.15.81:0/393178137 shutdown_connections 2023-11-08T16:34:42.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.886+0000 7f71ca74c640 1 --2- 172.21.15.81:0/393178137 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71c40ccf80 0x7f71c4070050 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:42.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.886+0000 7f71ca74c640 1 --2- 172.21.15.81:0/393178137 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71c40cc5e0 0x7f71c40cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:42.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.886+0000 7f71ca74c640 1 -- 172.21.15.81:0/393178137 >> 172.21.15.81:0/393178137 conn(0x7f71c40cb490 msgr2=0x7f71c40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:42.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.886+0000 7f71ca74c640 1 -- 172.21.15.81:0/393178137 shutdown_connections 2023-11-08T16:34:42.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.886+0000 7f71ca74c640 1 -- 172.21.15.81:0/393178137 wait complete. 2023-11-08T16:34:42.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.888+0000 7f71ca74c640 1 Processor -- start 2023-11-08T16:34:42.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.888+0000 7f71ca74c640 1 -- start start 2023-11-08T16:34:42.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.889+0000 7f71ca74c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71c40adcd0 0x7f71c41571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:42.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.889+0000 7f71ca74c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71c40cc5e0 0x7f71c40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:42.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.889+0000 7f71ca74c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71c40ccf80 0x7f71c40acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:42.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.889+0000 7f71ca74c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c41587e0 con 0x7f71c40adcd0 2023-11-08T16:34:42.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.889+0000 7f71ca74c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4157bd0 con 0x7f71c40cc5e0 2023-11-08T16:34:42.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.889+0000 7f71ca74c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4157eb0 con 0x7f71c40ccf80 2023-11-08T16:34:42.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.889+0000 7f71c3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71c40adcd0 0x7f71c41571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:42.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.889+0000 7f71c3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71c40adcd0 0x7f71c41571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47730/0 (socket says 172.21.15.81:47730) 2023-11-08T16:34:42.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.889+0000 7f71c3fff640 1 -- 172.21.15.81:0/3854344293 learned_addr learned my addr 172.21.15.81:0/3854344293 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:42.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.890+0000 7f71bb7fe640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71c40cc5e0 0x7f71c40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:42.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.890+0000 7f71c3fff640 1 -- 172.21.15.81:0/3854344293 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71c40ccf80 msgr2=0x7f71c40acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:42.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.890+0000 7f71c3fff640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71c40ccf80 0x7f71c40acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:42.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.890+0000 7f71c3fff640 1 -- 172.21.15.81:0/3854344293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71c40cc5e0 msgr2=0x7f71c40ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:42.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.890+0000 7f71c3fff640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71c40cc5e0 0x7f71c40ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:42.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.890+0000 7f71c3fff640 1 -- 172.21.15.81:0/3854344293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc0094d0 con 0x7f71c40adcd0 2023-11-08T16:34:42.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.890+0000 7f71c3fff640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71c40adcd0 0x7f71c41571e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f71b400ba50 tx=0x7f71b400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:42.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.890+0000 7f71c17fa640 1 -- 172.21.15.81:0/3854344293 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b400cbe0 con 0x7f71c40adcd0 2023-11-08T16:34:42.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.890+0000 7f71c17fa640 1 -- 172.21.15.81:0/3854344293 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b400cd80 con 0x7f71c40adcd0 2023-11-08T16:34:42.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.890+0000 7f71c17fa640 1 -- 172.21.15.81:0/3854344293 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4014680 con 0x7f71c40adcd0 2023-11-08T16:34:42.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.890+0000 7f71ca74c640 1 -- 172.21.15.81:0/3854344293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c4158190 con 0x7f71c40adcd0 2023-11-08T16:34:42.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.891+0000 7f71ca74c640 1 -- 172.21.15.81:0/3854344293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c414a5c0 con 0x7f71c40adcd0 2023-11-08T16:34:42.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.892+0000 7f71ca74c640 1 -- 172.21.15.81:0/3854344293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c4072170 con 0x7f71c40adcd0 2023-11-08T16:34:42.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.893+0000 7f71c17fa640 1 -- 172.21.15.81:0/3854344293 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f71b40147e0 con 0x7f71c40adcd0 2023-11-08T16:34:42.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.894+0000 7f71c17fa640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f71a406ae70 0x7f71a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:42.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.894+0000 7f71c17fa640 1 -- 172.21.15.81:0/3854344293 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f71b4089670 con 0x7f71c40adcd0 2023-11-08T16:34:42.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.894+0000 7f71bb7fe640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f71a406ae70 0x7f71a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:42.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.895+0000 7f71bb7fe640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f71a406ae70 0x7f71a406d330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f71ac006fd0 tx=0x7f71ac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:42.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:42.898+0000 7f71c17fa640 1 -- 172.21.15.81:0/3854344293 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f71b40545d0 con 0x7f71c40adcd0 2023-11-08T16:34:43.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.127+0000 7f71ca74c640 1 -- 172.21.15.81:0/3854344293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71c40ccda0 con 0x7f71c40adcd0 2023-11-08T16:34:43.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.128+0000 7f71c17fa640 1 -- 172.21.15.81:0/3854344293 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f71b4057bf0 con 0x7f71c40adcd0 2023-11-08T16:34:43.130 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:43.130 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3492741644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3492741644},{"type":"v1","addr":"172.21.15.93:6833","nonce":3492741644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":4916,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/119699732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":119699732},{"type":"v1","addr":"172.21.15.93:6835","nonce":119699732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5170,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/1473474760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1473474760},{"type":"v1","addr":"172.21.15.81:6836","nonce":1473474760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:32:43.449426+0000","modified":"2023-11-08T16:32:54.560616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5979},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5979":{"gid":5979,"name":"a","rank":0,"incarnation":36,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/1446454901","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":1446454901},{"type":"v1","addr":"172.21.15.81:6837","nonce":1446454901}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:34:43.131 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 41 2023-11-08T16:34:43.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 -- 172.21.15.81:0/3854344293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f71a406ae70 msgr2=0x7f71a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:43.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f71a406ae70 0x7f71a406d330 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f71ac006fd0 tx=0x7f71ac008040 comp rx=0 tx=0).stop 2023-11-08T16:34:43.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 -- 172.21.15.81:0/3854344293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71c40adcd0 msgr2=0x7f71c41571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:43.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71c40adcd0 0x7f71c41571e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f71b400ba50 tx=0x7f71b400bf10 comp rx=0 tx=0).stop 2023-11-08T16:34:43.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 -- 172.21.15.81:0/3854344293 shutdown_connections 2023-11-08T16:34:43.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f71a406ae70 0x7f71a406d330 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f71c40ccf80 0x7f71c40acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f71c40cc5e0 0x7f71c40ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 --2- 172.21.15.81:0/3854344293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f71c40adcd0 0x7f71c41571e0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 -- 172.21.15.81:0/3854344293 >> 172.21.15.81:0/3854344293 conn(0x7f71c40cb490 msgr2=0x7f71c40c7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:43.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 -- 172.21.15.81:0/3854344293 shutdown_connections 2023-11-08T16:34:43.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.131+0000 7f71ca74c640 1 -- 172.21.15.81:0/3854344293 wait complete. 2023-11-08T16:34:43.152 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-08T16:34:43.413 INFO:teuthology.orchestra.run.smithi081.stdout:{"message":"","return_code":0} 2023-11-08T16:34:43.413 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-08T16:34:43.414 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:34:43.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.606+0000 7f8048aca640 1 Processor -- start 2023-11-08T16:34:43.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.606+0000 7f8048aca640 1 -- start start 2023-11-08T16:34:43.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.607+0000 7f8048aca640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f804415ac50 0x7f80440b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:43.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.608+0000 7f8048aca640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f804406c730 0x7f804415e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:43.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.608+0000 7f8048aca640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f80440b0470 0x7f80440b0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:43.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.608+0000 7f8048aca640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440b30f0 con 0x7f80440b0470 2023-11-08T16:34:43.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.608+0000 7f8048aca640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80440b0e40 con 0x7f804415ac50 2023-11-08T16:34:43.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.608+0000 7f8048aca640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804406f4e0 con 0x7f804406c730 2023-11-08T16:34:43.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.608+0000 7f8043fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f80440b0470 0x7f80440b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:43.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.608+0000 7f8043fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f80440b0470 0x7f80440b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47742/0 (socket says 172.21.15.81:47742) 2023-11-08T16:34:43.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.608+0000 7f8043fff640 1 -- 172.21.15.81:0/3691702035 learned_addr learned my addr 172.21.15.81:0/3691702035 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:43.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.608+0000 7f8042ffd640 1 --2- 172.21.15.81:0/3691702035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f804406c730 0x7f804415e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:43.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.608+0000 7f80437fe640 1 --2- 172.21.15.81:0/3691702035 >> v2:172.21.15.93:3300/0 conn(0x7f804415ac50 0x7f80440b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.609+0000 7f8043fff640 1 -- 172.21.15.81:0/3691702035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f804406c730 msgr2=0x7f804415e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.609+0000 7f8043fff640 1 --2- 172.21.15.81:0/3691702035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f804406c730 0x7f804415e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.609+0000 7f8043fff640 1 -- 172.21.15.81:0/3691702035 >> v2:172.21.15.93:3300/0 conn(0x7f804415ac50 msgr2=0x7f80440b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.609+0000 7f8043fff640 1 --2- 172.21.15.81:0/3691702035 >> v2:172.21.15.93:3300/0 conn(0x7f804415ac50 0x7f80440b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.609+0000 7f8043fff640 1 -- 172.21.15.81:0/3691702035 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804415ed80 con 0x7f80440b0470 2023-11-08T16:34:43.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.609+0000 7f80437fe640 1 --2- 172.21.15.81:0/3691702035 >> v2:172.21.15.93:3300/0 conn(0x7f804415ac50 0x7f80440b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.610+0000 7f8043fff640 1 --2- 172.21.15.81:0/3691702035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80440b0470 0x7f80440b0900 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f803c009870 tx=0x7f803c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73aa1b0d7b3cdd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.610+0000 7f80427fc640 1 -- 172.21.15.81:0/3691702035 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c031640 con 0x7f80440b0470 2023-11-08T16:34:43.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.610+0000 7f80427fc640 1 -- 172.21.15.81:0/3691702035 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f803c0317a0 con 0x7f80440b0470 2023-11-08T16:34:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.610+0000 7f80427fc640 1 -- 172.21.15.81:0/3691702035 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803c031b30 con 0x7f80440b0470 2023-11-08T16:34:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.611+0000 7f8048aca640 1 -- 172.21.15.81:0/3691702035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80440b0470 msgr2=0x7f80440b0900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.611+0000 7f8048aca640 1 --2- 172.21.15.81:0/3691702035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80440b0470 0x7f80440b0900 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f803c009870 tx=0x7f803c02b8f0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.611+0000 7f8048aca640 1 -- 172.21.15.81:0/3691702035 shutdown_connections 2023-11-08T16:34:43.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.611+0000 7f8048aca640 1 --2- 172.21.15.81:0/3691702035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f804406c730 0x7f804415e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.611+0000 7f8048aca640 1 --2- 172.21.15.81:0/3691702035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80440b0470 0x7f80440b0900 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.611+0000 7f8048aca640 1 --2- 172.21.15.81:0/3691702035 >> v2:172.21.15.93:3300/0 conn(0x7f804415ac50 0x7f80440b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.611+0000 7f8048aca640 1 -- 172.21.15.81:0/3691702035 >> 172.21.15.81:0/3691702035 conn(0x7f80440ced40 msgr2=0x7f80440cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:43.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.611+0000 7f8048aca640 1 -- 172.21.15.81:0/3691702035 shutdown_connections 2023-11-08T16:34:43.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.611+0000 7f8048aca640 1 -- 172.21.15.81:0/3691702035 wait complete. 2023-11-08T16:34:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.613+0000 7f8048aca640 1 Processor -- start 2023-11-08T16:34:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.613+0000 7f8048aca640 1 -- start start 2023-11-08T16:34:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.614+0000 7f8048aca640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f804406c730 0x7f804415ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.614+0000 7f8048aca640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80440b0470 0x7f80441f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:43.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.614+0000 7f8048aca640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f804415ac50 0x7f8044157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:43.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.614+0000 7f8048aca640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441f29d0 con 0x7f80440b0470 2023-11-08T16:34:43.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.614+0000 7f8048aca640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441533c0 con 0x7f804415ac50 2023-11-08T16:34:43.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.614+0000 7f8048aca640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80441f3460 con 0x7f804406c730 2023-11-08T16:34:43.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.614+0000 7f8042ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80440b0470 0x7f80441f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:43.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.614+0000 7f8042ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80440b0470 0x7f80441f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47752/0 (socket says 172.21.15.81:47752) 2023-11-08T16:34:43.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.614+0000 7f8042ffd640 1 -- 172.21.15.81:0/580153448 learned_addr learned my addr 172.21.15.81:0/580153448 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:43.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f8042ffd640 1 -- 172.21.15.81:0/580153448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f804406c730 msgr2=0x7f804415ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:43.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f80437fe640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f804406c730 0x7f804415ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:43.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f8043fff640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f804415ac50 0x7f8044157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:43.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f8042ffd640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f804406c730 0x7f804415ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f8042ffd640 1 -- 172.21.15.81:0/580153448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f804415ac50 msgr2=0x7f8044157a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:43.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f8042ffd640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f804415ac50 0x7f8044157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f8042ffd640 1 -- 172.21.15.81:0/580153448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c0094d0 con 0x7f80440b0470 2023-11-08T16:34:43.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f8042ffd640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80440b0470 0x7f80441f2490 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f802c00d830 tx=0x7f802c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:43.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f803bfff640 1 -- 172.21.15.81:0/580153448 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c00be10 con 0x7f80440b0470 2023-11-08T16:34:43.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f803bfff640 1 -- 172.21.15.81:0/580153448 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802c0098b0 con 0x7f80440b0470 2023-11-08T16:34:43.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f803bfff640 1 -- 172.21.15.81:0/580153448 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c009a40 con 0x7f80440b0470 2023-11-08T16:34:43.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f8048aca640 1 -- 172.21.15.81:0/580153448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80441f2bc0 con 0x7f80440b0470 2023-11-08T16:34:43.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.615+0000 7f8048aca640 1 -- 172.21.15.81:0/580153448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80441f3180 con 0x7f80440b0470 2023-11-08T16:34:43.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.616+0000 7f8048aca640 1 -- 172.21.15.81:0/580153448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8044067800 con 0x7f80440b0470 2023-11-08T16:34:43.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.617+0000 7f803bfff640 1 -- 172.21.15.81:0/580153448 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f802c009ba0 con 0x7f80440b0470 2023-11-08T16:34:43.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.618+0000 7f803bfff640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f802406ae70 0x7f802406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:43.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.618+0000 7f803bfff640 1 -- 172.21.15.81:0/580153448 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6200+0+0 (secure 0 0 0) 0x7f802c089c20 con 0x7f80440b0470 2023-11-08T16:34:43.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.619+0000 7f80437fe640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f802406ae70 0x7f802406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:43.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.619+0000 7f80437fe640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f802406ae70 0x7f802406d330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8034008f80 tx=0x7f80340084d0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:43.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.623+0000 7f803bfff640 1 -- 172.21.15.81:0/580153448 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f802c054a40 con 0x7f80440b0470 2023-11-08T16:34:43.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.851+0000 7f8048aca640 1 -- 172.21.15.81:0/580153448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f804406bdb0 con 0x7f80440b0470 2023-11-08T16:34:43.854 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:34:43.852+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:34:43.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.856+0000 7f803bfff640 1 -- 172.21.15.81:0/580153448 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) v1 ==== 161+0+0 (secure 0 0 0) 0x7f802c058060 con 0x7f80440b0470 2023-11-08T16:34:43.858 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:34:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 -- 172.21.15.81:0/580153448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f802406ae70 msgr2=0x7f802406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f802406ae70 0x7f802406d330 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8034008f80 tx=0x7f80340084d0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 -- 172.21.15.81:0/580153448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80440b0470 msgr2=0x7f80441f2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80440b0470 0x7f80441f2490 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f802c00d830 tx=0x7f802c00dcf0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 -- 172.21.15.81:0/580153448 shutdown_connections 2023-11-08T16:34:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f802406ae70 0x7f802406d330 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f804415ac50 0x7f8044157a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f80440b0470 0x7f80441f2490 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 --2- 172.21.15.81:0/580153448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f804406c730 0x7f804415ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 -- 172.21.15.81:0/580153448 >> 172.21.15.81:0/580153448 conn(0x7f80440ced40 msgr2=0x7f80440cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 -- 172.21.15.81:0/580153448 shutdown_connections 2023-11-08T16:34:43.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:43.859+0000 7f8048aca640 1 -- 172.21.15.81:0/580153448 wait complete. 2023-11-08T16:34:43.880 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:34:43.885 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:34:43.942 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:34:44.021 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2023-11-08T16:34:44.021 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2023-11-08T16:34:44.022 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:34:44.022 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2023-11-08T16:34:44.022 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.022 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2023-11-08T16:34:44.022 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.023 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2023-11-08T16:34:44.023 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.023 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2023-11-08T16:34:44.023 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2023-11-08T16:34:44.023 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.024 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2023-11-08T16:34:44.024 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2023-11-08T16:34:44.024 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.024 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:34:44.024 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2023-11-08T16:34:44.025 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.025 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:34:44.025 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:34:44.025 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.025 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2023-11-08T16:34:44.025 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.026 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2023-11-08T16:34:44.026 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2023-11-08T16:34:44.026 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:34:44.026 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:34:44.026 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2023-11-08T16:34:44.027 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:34:44.027 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2023-11-08T16:34:44.027 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:34:44.027 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:34:44.027 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2023-11-08T16:34:44.028 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.028 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:34:44.028 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.028 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2023-11-08T16:34:44.028 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.029 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2023-11-08T16:34:44.029 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.029 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2023-11-08T16:34:44.029 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2023-11-08T16:34:44.029 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2023-11-08T16:34:44.030 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:34:44.030 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:34:44.030 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.030 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2023-11-08T16:34:44.030 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2023-11-08T16:34:44.030 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.031 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2023-11-08T16:34:44.031 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2023-11-08T16:34:44.031 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:34:44.031 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.031 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2023-11-08T16:34:44.032 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2023-11-08T16:34:44.032 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.032 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2023-11-08T16:34:44.032 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2023-11-08T16:34:44.032 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2023-11-08T16:34:44.033 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2023-11-08T16:34:44.033 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.033 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2023-11-08T16:34:44.033 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2023-11-08T16:34:44.033 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.034 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:34:44.034 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.034 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2023-11-08T16:34:44.034 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.034 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2023-11-08T16:34:44.035 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.035 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:34:44.035 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2023-11-08T16:34:44.035 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.035 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2023-11-08T16:34:44.036 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2023-11-08T16:34:44.036 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2023-11-08T16:34:44.036 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2023-11-08T16:34:44.036 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2023-11-08T16:34:44.036 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2023-11-08T16:34:44.036 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2023-11-08T16:34:44.037 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2023-11-08T16:34:44.037 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2023-11-08T16:34:44.037 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:34:44.037 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2023-11-08T16:34:44.037 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2023-11-08T16:34:44.038 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:34:44.038 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2023-11-08T16:34:44.038 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:34:44.038 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:34:44.038 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2023-11-08T16:34:44.039 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:34:44.039 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.039 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2023-11-08T16:34:44.039 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:34:44.039 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2023-11-08T16:34:44.039 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:34:44.040 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:34:44.040 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:34:44.040 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:34:44.040 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2023-11-08T16:34:44.040 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.041 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2023-11-08T16:34:44.041 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:34:44.041 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:34:44.041 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2023-11-08T16:34:44.041 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:34:44.042 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2023-11-08T16:34:44.042 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2023-11-08T16:34:44.042 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:34:44.042 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2023-11-08T16:34:44.042 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2023-11-08T16:34:44.043 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:34:44.043 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.043 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:34:44.043 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:34:44.043 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:34:44.043 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.044 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:34:44.044 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:34:44.044 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:34:44.044 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.044 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2023-11-08T16:34:44.045 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:34:44.045 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.045 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:34:44.045 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2023-11-08T16:34:44.045 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.045 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2023-11-08T16:34:44.046 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2023-11-08T16:34:44.046 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2023-11-08T16:34:44.046 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2023-11-08T16:34:44.046 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:34:44.046 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:34:44.047 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2023-11-08T16:34:44.047 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2023-11-08T16:34:44.047 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:34:44.047 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2023-11-08T16:34:44.047 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:34:44.048 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:34:44.048 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2023-11-08T16:34:44.048 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2023-11-08T16:34:44.048 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2023-11-08T16:34:44.048 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:34:44.048 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:34:44.049 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:34:44.049 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:34:44.049 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2023-11-08T16:34:44.049 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2023-11-08T16:34:44.049 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2023-11-08T16:34:44.050 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:34:44.050 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2023-11-08T16:34:44.050 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = False 2023-11-08T16:34:44.050 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2023-11-08T16:34:44.050 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = False 2023-11-08T16:34:44.051 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:34:44.051 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:34:44.051 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:34:44.057 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-08T16:34:44.058 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2023-11-08T16:34:44.058 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:34:44.058 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2023-11-08T16:34:44.058 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.058 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2023-11-08T16:34:44.059 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.059 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2023-11-08T16:34:44.059 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.059 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2023-11-08T16:34:44.059 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2023-11-08T16:34:44.060 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.060 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2023-11-08T16:34:44.060 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2023-11-08T16:34:44.060 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.060 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:34:44.060 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2023-11-08T16:34:44.061 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.061 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:34:44.061 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:34:44.061 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.061 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2023-11-08T16:34:44.062 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.062 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2023-11-08T16:34:44.062 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2023-11-08T16:34:44.062 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:34:44.062 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:34:44.063 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2023-11-08T16:34:44.063 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:34:44.063 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2023-11-08T16:34:44.063 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:34:44.063 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:34:44.063 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2023-11-08T16:34:44.064 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.064 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:34:44.064 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.064 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2023-11-08T16:34:44.064 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.065 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2023-11-08T16:34:44.065 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.065 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2023-11-08T16:34:44.065 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2023-11-08T16:34:44.065 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2023-11-08T16:34:44.065 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:34:44.066 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:34:44.066 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.066 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2023-11-08T16:34:44.066 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2023-11-08T16:34:44.066 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.067 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2023-11-08T16:34:44.067 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2023-11-08T16:34:44.067 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:34:44.067 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.067 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2023-11-08T16:34:44.068 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2023-11-08T16:34:44.068 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.068 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2023-11-08T16:34:44.068 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2023-11-08T16:34:44.068 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2023-11-08T16:34:44.068 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2023-11-08T16:34:44.069 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.069 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2023-11-08T16:34:44.069 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2023-11-08T16:34:44.069 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.069 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:34:44.070 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.070 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2023-11-08T16:34:44.070 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.070 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2023-11-08T16:34:44.070 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.071 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:34:44.071 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2023-11-08T16:34:44.071 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.071 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2023-11-08T16:34:44.071 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2023-11-08T16:34:44.071 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2023-11-08T16:34:44.072 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2023-11-08T16:34:44.072 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2023-11-08T16:34:44.072 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2023-11-08T16:34:44.072 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2023-11-08T16:34:44.072 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2023-11-08T16:34:44.073 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2023-11-08T16:34:44.073 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:34:44.073 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2023-11-08T16:34:44.073 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2023-11-08T16:34:44.073 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:34:44.074 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2023-11-08T16:34:44.074 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:34:44.074 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:34:44.074 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2023-11-08T16:34:44.074 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:34:44.074 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.075 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2023-11-08T16:34:44.075 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:34:44.075 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2023-11-08T16:34:44.075 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:34:44.075 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:34:44.076 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:34:44.076 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:34:44.076 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2023-11-08T16:34:44.076 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.076 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2023-11-08T16:34:44.076 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:34:44.077 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:34:44.077 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2023-11-08T16:34:44.077 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:34:44.077 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2023-11-08T16:34:44.077 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2023-11-08T16:34:44.078 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:34:44.078 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2023-11-08T16:34:44.078 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2023-11-08T16:34:44.078 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:34:44.078 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.079 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:34:44.079 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:34:44.079 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:34:44.079 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.079 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:34:44.079 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:34:44.080 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:34:44.080 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.080 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2023-11-08T16:34:44.080 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:34:44.080 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.081 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:34:44.081 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2023-11-08T16:34:44.081 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.081 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2023-11-08T16:34:44.081 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2023-11-08T16:34:44.081 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2023-11-08T16:34:44.082 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2023-11-08T16:34:44.082 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:34:44.082 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:34:44.082 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2023-11-08T16:34:44.082 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2023-11-08T16:34:44.083 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:34:44.083 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2023-11-08T16:34:44.083 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:34:44.083 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:34:44.083 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2023-11-08T16:34:44.083 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2023-11-08T16:34:44.084 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2023-11-08T16:34:44.084 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:34:44.084 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:34:44.084 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:34:44.084 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:34:44.085 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2023-11-08T16:34:44.085 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2023-11-08T16:34:44.085 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2023-11-08T16:34:44.085 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:34:44.085 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2023-11-08T16:34:44.085 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = False 2023-11-08T16:34:44.086 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2023-11-08T16:34:44.086 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = False 2023-11-08T16:34:44.086 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:34:44.086 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:34:44.086 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:34:44.101 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:34:44.137 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:34:44.295 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2023-11-08T16:34:44.296 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2023-11-08T16:34:44.296 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:34:44.296 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2023-11-08T16:34:44.296 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.297 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2023-11-08T16:34:44.297 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.297 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2023-11-08T16:34:44.297 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.297 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2023-11-08T16:34:44.298 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2023-11-08T16:34:44.298 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.298 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2023-11-08T16:34:44.298 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2023-11-08T16:34:44.298 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.299 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:34:44.299 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2023-11-08T16:34:44.299 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.299 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:34:44.299 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:34:44.299 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.300 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2023-11-08T16:34:44.300 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.300 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2023-11-08T16:34:44.300 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2023-11-08T16:34:44.300 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:34:44.301 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:34:44.301 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2023-11-08T16:34:44.301 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:34:44.301 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2023-11-08T16:34:44.301 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:34:44.302 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:34:44.302 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2023-11-08T16:34:44.302 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.302 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:34:44.302 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.303 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2023-11-08T16:34:44.303 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.303 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2023-11-08T16:34:44.303 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.303 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2023-11-08T16:34:44.304 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2023-11-08T16:34:44.304 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2023-11-08T16:34:44.304 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:34:44.304 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:34:44.304 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.304 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2023-11-08T16:34:44.305 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2023-11-08T16:34:44.305 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.305 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2023-11-08T16:34:44.305 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2023-11-08T16:34:44.305 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:34:44.306 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.306 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2023-11-08T16:34:44.306 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2023-11-08T16:34:44.306 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.306 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2023-11-08T16:34:44.307 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2023-11-08T16:34:44.307 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2023-11-08T16:34:44.307 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2023-11-08T16:34:44.307 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.307 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2023-11-08T16:34:44.308 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2023-11-08T16:34:44.308 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.308 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:34:44.308 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.308 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2023-11-08T16:34:44.308 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.309 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2023-11-08T16:34:44.309 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.309 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:34:44.309 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2023-11-08T16:34:44.309 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.310 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2023-11-08T16:34:44.310 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2023-11-08T16:34:44.310 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2023-11-08T16:34:44.310 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2023-11-08T16:34:44.310 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2023-11-08T16:34:44.311 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2023-11-08T16:34:44.311 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2023-11-08T16:34:44.311 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2023-11-08T16:34:44.311 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2023-11-08T16:34:44.311 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:34:44.311 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2023-11-08T16:34:44.312 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2023-11-08T16:34:44.312 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:34:44.312 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2023-11-08T16:34:44.312 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:34:44.312 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:34:44.313 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2023-11-08T16:34:44.313 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:34:44.313 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.313 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2023-11-08T16:34:44.313 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:34:44.313 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2023-11-08T16:34:44.314 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:34:44.314 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:34:44.314 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:34:44.314 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:34:44.314 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2023-11-08T16:34:44.315 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.315 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2023-11-08T16:34:44.315 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:34:44.315 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:34:44.315 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2023-11-08T16:34:44.315 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:34:44.316 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2023-11-08T16:34:44.316 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2023-11-08T16:34:44.316 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:34:44.316 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2023-11-08T16:34:44.316 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2023-11-08T16:34:44.317 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:34:44.317 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.317 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:34:44.317 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:34:44.317 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:34:44.318 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.318 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:34:44.318 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:34:44.318 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:34:44.318 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.319 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2023-11-08T16:34:44.319 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:34:44.319 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.319 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:34:44.319 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2023-11-08T16:34:44.319 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:44.320 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2023-11-08T16:34:44.320 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2023-11-08T16:34:44.320 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2023-11-08T16:34:44.320 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2023-11-08T16:34:44.320 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:34:44.321 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:34:44.321 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2023-11-08T16:34:44.321 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2023-11-08T16:34:44.321 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:34:44.321 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2023-11-08T16:34:44.322 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:34:44.322 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:34:44.322 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2023-11-08T16:34:44.322 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2023-11-08T16:34:44.322 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2023-11-08T16:34:44.322 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:34:44.323 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:34:44.323 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:34:44.323 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:34:44.323 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2023-11-08T16:34:44.323 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2023-11-08T16:34:44.324 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2023-11-08T16:34:44.324 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:34:44.324 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2023-11-08T16:34:44.324 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = False 2023-11-08T16:34:44.324 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2023-11-08T16:34:44.324 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = False 2023-11-08T16:34:44.325 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:34:44.325 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:34:44.325 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:34:44.326 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-08T16:34:44.326 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2023-11-08T16:34:44.326 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:34:44.326 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2023-11-08T16:34:44.327 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.327 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2023-11-08T16:34:44.327 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.327 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2023-11-08T16:34:44.327 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.328 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2023-11-08T16:34:44.328 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2023-11-08T16:34:44.328 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.328 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2023-11-08T16:34:44.328 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2023-11-08T16:34:44.328 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.329 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:34:44.329 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2023-11-08T16:34:44.329 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.329 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:34:44.329 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:34:44.330 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.330 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2023-11-08T16:34:44.330 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.330 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2023-11-08T16:34:44.330 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2023-11-08T16:34:44.331 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:34:44.331 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:34:44.331 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2023-11-08T16:34:44.331 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:34:44.331 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2023-11-08T16:34:44.331 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:34:44.332 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:34:44.332 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2023-11-08T16:34:44.332 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.332 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:34:44.332 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.333 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2023-11-08T16:34:44.333 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.333 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2023-11-08T16:34:44.333 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.334 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2023-11-08T16:34:44.334 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2023-11-08T16:34:44.334 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2023-11-08T16:34:44.334 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:34:44.334 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:34:44.335 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.335 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2023-11-08T16:34:44.335 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2023-11-08T16:34:44.335 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.335 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2023-11-08T16:34:44.335 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2023-11-08T16:34:44.336 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:34:44.336 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.336 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2023-11-08T16:34:44.336 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2023-11-08T16:34:44.336 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.337 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2023-11-08T16:34:44.337 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2023-11-08T16:34:44.337 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2023-11-08T16:34:44.337 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2023-11-08T16:34:44.337 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.338 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2023-11-08T16:34:44.338 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2023-11-08T16:34:44.338 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.338 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:34:44.338 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.339 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2023-11-08T16:34:44.339 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.339 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2023-11-08T16:34:44.339 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.339 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:34:44.340 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2023-11-08T16:34:44.340 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.340 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2023-11-08T16:34:44.340 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2023-11-08T16:34:44.340 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2023-11-08T16:34:44.340 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2023-11-08T16:34:44.341 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2023-11-08T16:34:44.341 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2023-11-08T16:34:44.341 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2023-11-08T16:34:44.341 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2023-11-08T16:34:44.341 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2023-11-08T16:34:44.342 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:34:44.342 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2023-11-08T16:34:44.342 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2023-11-08T16:34:44.342 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:34:44.342 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2023-11-08T16:34:44.343 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:34:44.343 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:34:44.343 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2023-11-08T16:34:44.343 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:34:44.343 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.343 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2023-11-08T16:34:44.344 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:34:44.344 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2023-11-08T16:34:44.344 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:34:44.344 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:34:44.344 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:34:44.345 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:34:44.345 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2023-11-08T16:34:44.345 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.345 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2023-11-08T16:34:44.345 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:34:44.345 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:34:44.346 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2023-11-08T16:34:44.346 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:34:44.346 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2023-11-08T16:34:44.346 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2023-11-08T16:34:44.346 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:34:44.347 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2023-11-08T16:34:44.347 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2023-11-08T16:34:44.347 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:34:44.347 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.347 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:34:44.348 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:34:44.348 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:34:44.348 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.348 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:34:44.348 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:34:44.348 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:34:44.349 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.349 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2023-11-08T16:34:44.349 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:34:44.349 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.349 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:34:44.350 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2023-11-08T16:34:44.350 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:34:44.350 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2023-11-08T16:34:44.350 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2023-11-08T16:34:44.350 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2023-11-08T16:34:44.350 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2023-11-08T16:34:44.351 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:34:44.351 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:34:44.351 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2023-11-08T16:34:44.351 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2023-11-08T16:34:44.351 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:34:44.352 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2023-11-08T16:34:44.352 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:34:44.352 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:34:44.352 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2023-11-08T16:34:44.352 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2023-11-08T16:34:44.353 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2023-11-08T16:34:44.353 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:34:44.353 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:34:44.353 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:34:44.353 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:34:44.353 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2023-11-08T16:34:44.354 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2023-11-08T16:34:44.354 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2023-11-08T16:34:44.354 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:34:44.354 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2023-11-08T16:34:44.354 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = False 2023-11-08T16:34:44.355 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2023-11-08T16:34:44.355 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = False 2023-11-08T16:34:44.355 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:34:44.355 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:34:44.355 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:34:44.357 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T16:34:44.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.463+0000 7fa1f0c39f40 1 Processor -- start 2023-11-08T16:34:44.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.463+0000 7fa1f0c39f40 1 -- start start 2023-11-08T16:34:44.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.464+0000 7fa1f0c39f40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556f3eb92b70 0x556f3ec198d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.464+0000 7fa1f0c39f40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556f3ec1a580 0x556f3ec17e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.464+0000 7fa1f0c39f40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a960 0x556f3eb42e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.464+0000 7fa1f0c39f40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f3eaf4340 con 0x556f3ec1a960 2023-11-08T16:34:44.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.464+0000 7fa1f0c39f40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f3ea8ea70 con 0x556f3eb92b70 2023-11-08T16:34:44.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.464+0000 7fa1f0c39f40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f3ea8e710 con 0x556f3ec1a580 2023-11-08T16:34:44.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1ee826640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a960 0x556f3eb42e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1ee826640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a960 0x556f3eb42e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47788/0 (socket says 172.21.15.81:47788) 2023-11-08T16:34:44.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1ee826640 1 -- 172.21.15.81:0/2587353311 learned_addr learned my addr 172.21.15.81:0/2587353311 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:44.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1e5824640 1 --2- 172.21.15.81:0/2587353311 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556f3ec1a580 0x556f3ec17e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1ee826640 1 -- 172.21.15.81:0/2587353311 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556f3ec1a580 msgr2=0x556f3ec17e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:44.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1ee826640 1 --2- 172.21.15.81:0/2587353311 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556f3ec1a580 0x556f3ec17e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1ee025640 1 --2- 172.21.15.81:0/2587353311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556f3eb92b70 0x556f3ec198d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1ee826640 1 -- 172.21.15.81:0/2587353311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556f3eb92b70 msgr2=0x556f3ec198d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:44.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1ee826640 1 --2- 172.21.15.81:0/2587353311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556f3eb92b70 0x556f3ec198d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1ee826640 1 -- 172.21.15.81:0/2587353311 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e0005de0 con 0x556f3ec1a960 2023-11-08T16:34:44.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1e5824640 1 --2- 172.21.15.81:0/2587353311 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556f3ec1a580 0x556f3ec17e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:44.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.465+0000 7fa1ee826640 1 --2- 172.21.15.81:0/2587353311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a960 0x556f3eb42e80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800a4b0 tx=0x7fa1e800a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f5e27093f318713 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1ed023640 1 -- 172.21.15.81:0/2587353311 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800e710 con 0x556f3ec1a960 2023-11-08T16:34:44.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1ed023640 1 -- 172.21.15.81:0/2587353311 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e800e870 con 0x556f3ec1a960 2023-11-08T16:34:44.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1ed023640 1 -- 172.21.15.81:0/2587353311 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800ec20 con 0x556f3ec1a960 2023-11-08T16:34:44.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/2587353311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a960 msgr2=0x556f3eb42e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/2587353311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a960 0x556f3eb42e80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa1e800a4b0 tx=0x7fa1e800a970 comp rx=0 tx=0).stop 2023-11-08T16:34:44.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/2587353311 shutdown_connections 2023-11-08T16:34:44.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/2587353311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a960 0x556f3eb42e80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/2587353311 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556f3ec1a580 0x556f3ec17e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/2587353311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556f3eb92b70 0x556f3ec198d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/2587353311 >> 172.21.15.81:0/2587353311 conn(0x556f3eb8fc60 msgr2=0x556f3eb8d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:44.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/2587353311 shutdown_connections 2023-11-08T16:34:44.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.466+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/2587353311 wait complete. 2023-11-08T16:34:44.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.467+0000 7fa1f0c39f40 1 Processor -- start 2023-11-08T16:34:44.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.467+0000 7fa1f0c39f40 1 -- start start 2023-11-08T16:34:44.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.468+0000 7fa1f0c39f40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556f3eb92b70 0x556f3eb434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.468+0000 7fa1f0c39f40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a580 0x556f3eb738b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.468+0000 7fa1f0c39f40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556f3ec1a960 0x556f3ecb8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.468+0000 7fa1f0c39f40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f3ea82520 con 0x556f3ec1a580 2023-11-08T16:34:44.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.468+0000 7fa1f0c39f40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f3ec1ecd0 con 0x556f3ec1a960 2023-11-08T16:34:44.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.468+0000 7fa1f0c39f40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x556f3ec1fb30 con 0x556f3eb92b70 2023-11-08T16:34:44.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.469+0000 7fa1e5824640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a580 0x556f3eb738b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.469+0000 7fa1e5824640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a580 0x556f3eb738b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47800/0 (socket says 172.21.15.81:47800) 2023-11-08T16:34:44.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.469+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 learned_addr learned my addr 172.21.15.81:0/264735660 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:44.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.469+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556f3eb92b70 0x556f3eb434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.469+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556f3eb92b70 msgr2=0x556f3eb434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.469+0000 7fa1e5824640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x556f3eb92b70 0x556f3eb434b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.469+0000 7fa1ee826640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556f3ec1a960 0x556f3ecb8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.469+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556f3ec1a960 msgr2=0x556f3ecb8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.469+0000 7fa1e5824640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556f3ec1a960 0x556f3ecb8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.469+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e800a0e0 con 0x556f3ec1a580 2023-11-08T16:34:44.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.469+0000 7fa1e5824640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a580 0x556f3eb738b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8007dc0 tx=0x7fa1d8006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.470+0000 7fa1e7fff640 1 -- 172.21.15.81:0/264735660 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8010710 con 0x556f3ec1a580 2023-11-08T16:34:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.470+0000 7fa1e7fff640 1 -- 172.21.15.81:0/264735660 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d8010870 con 0x556f3ec1a580 2023-11-08T16:34:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.470+0000 7fa1e7fff640 1 -- 172.21.15.81:0/264735660 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8010c20 con 0x556f3ec1a580 2023-11-08T16:34:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.470+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556f3ea9a680 con 0x556f3ec1a580 2023-11-08T16:34:44.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.470+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f3ea8d830 con 0x556f3ec1a580 2023-11-08T16:34:44.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.472+0000 7fa1e7fff640 1 -- 172.21.15.81:0/264735660 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1d8012400 con 0x556f3ec1a580 2023-11-08T16:34:44.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.473+0000 7fa1e7fff640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1d0069820 0x7fa1d006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.473+0000 7fa1e7fff640 1 -- 172.21.15.81:0/264735660 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6286+0+0 (secure 0 0 0) 0x7fa1d8087bf0 con 0x556f3ec1a580 2023-11-08T16:34:44.485 INFO:teuthology.orchestra.run.smithi081.stdout:Warning: using slow linear search 2023-11-08T16:34:44.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.473+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1d0069820 0x7fa1d006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.473+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x556f3eb73df0 0x556f3ecb93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.473+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:1 7.0 7:00000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x556f3eb73df0 2023-11-08T16:34:44.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.474+0000 7fa1ee826640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x556f3eb73df0 0x556f3ecb93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.474+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1d0069820 0x7fa1d006bce0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0054000 tx=0x7fa1e004e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.475+0000 7fa1ee826640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x556f3eb73df0 0x556f3ecb93a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.476+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(1 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e800e9e0 con 0x556f3eb73df0 2023-11-08T16:34:44.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.476+0000 7fa1ee826640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7fa1e800ec30 0x7fa1e801c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.476+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:2 7.10 7:08000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e800ec30 2023-11-08T16:34:44.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.476+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7fa1e800ec30 0x7fa1e801c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.477+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7fa1e800ec30 0x7fa1e801c400 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.477+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(2 [pgnls start_epoch 63 out=177b] v62'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7fa1e0052920 con 0x7fa1e800ec30 2023-11-08T16:34:44.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.477+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7fa1e005d3f0 0x7fa1e005d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.477+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:3 7.8 7:10000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e005d3f0 2023-11-08T16:34:44.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.477+0000 7fa1e5824640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7fa1e005d3f0 0x7fa1e005d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.478+0000 7fa1e5824640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7fa1e005d3f0 0x7fa1e005d850 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.478+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(3 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1d8085f50 con 0x7fa1e005d3f0 2023-11-08T16:34:44.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.478+0000 7fa1e5824640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fa1d8056970 0x7fa1d805c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.478+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:4 7.18 7:18000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d8056970 2023-11-08T16:34:44.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.479+0000 7fa1ee826640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fa1d8056970 0x7fa1d805c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.479+0000 7fa1ee826640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fa1d8056970 0x7fa1d805c5b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.479+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(4 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8056970 2023-11-08T16:34:44.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.479+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 7.4 7:20000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d8056970 2023-11-08T16:34:44.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.480+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(5 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8056970 2023-11-08T16:34:44.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.480+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:6 7.14 7:28000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e800ec30 2023-11-08T16:34:44.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.480+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(6 [pgnls start_epoch 63 out=149b] v62'96 uv96 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7fa1e0052920 con 0x7fa1e800ec30 2023-11-08T16:34:44.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.480+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7fa1e00688b0 0x7fa1e0006380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.480+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:7 7.c 7:30000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e00688b0 2023-11-08T16:34:44.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.481+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7fa1e00688b0 0x7fa1e0006380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.482+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7fa1e00688b0 0x7fa1e0006380 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.483+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(7 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e00688b0 2023-11-08T16:34:44.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.483+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:8 7.1c 7:38000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x556f3eb73df0 2023-11-08T16:34:44.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.484+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(8 [pgnls start_epoch 63 out=73b] v56'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fa1e800e9e0 con 0x556f3eb73df0 2023-11-08T16:34:44.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.484+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 7.2 7:40000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e800ec30 2023-11-08T16:34:44.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.484+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(9 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.484+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:10 7.12 7:48000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e00688b0 2023-11-08T16:34:44.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.485+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(10 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e00688b0 2023-11-08T16:34:44.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.485+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:11 7.a 7:50000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e800ec30 2023-11-08T16:34:44.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.486+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(11 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.486+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7fa1e0079040 0x7fa1e0079420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.486+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:12 7.1a 7:58000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e0079040 2023-11-08T16:34:44.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.486+0000 7fa1e5824640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7fa1e0079040 0x7fa1e0079420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.487+0000 7fa1e5824640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7fa1e0079040 0x7fa1e0079420 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.488+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(12 [pgnls start_epoch 63 out=49b] v56'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1d8085f50 con 0x7fa1e0079040 2023-11-08T16:34:44.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.488+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:13 7.6 7:60000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d8056970 2023-11-08T16:34:44.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.489+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(13 [pgnls start_epoch 63 out=77b] v62'24 uv24 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8056970 2023-11-08T16:34:44.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.489+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:14 7.16 7:68000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e0079040 2023-11-08T16:34:44.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.490+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(14 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1d8085f50 con 0x7fa1e0079040 2023-11-08T16:34:44.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.490+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:15 7.e 7:70000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e005d3f0 2023-11-08T16:34:44.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.490+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(15 [pgnls start_epoch 63 out=73b] v56'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fa1d8085f50 con 0x7fa1e005d3f0 2023-11-08T16:34:44.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.490+0000 7fa1e5824640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fa1d8055c00 0x7fa1d805a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.490+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:16 7.1e 7:78000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d8055c00 2023-11-08T16:34:44.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.490+0000 7fa1ee826640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fa1d8055c00 0x7fa1d805a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.491+0000 7fa1ee826640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fa1d8055c00 0x7fa1d805a120 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.491+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(16 [pgnls start_epoch 63 out=49b] v56'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8055c00 2023-11-08T16:34:44.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.491+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:17 7.1 7:80000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e0079040 2023-11-08T16:34:44.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.492+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(17 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1d8085f50 con 0x7fa1e0079040 2023-11-08T16:34:44.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.492+0000 7fa1e5824640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fa1d805e580 0x7fa1d8054d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.492+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:18 7.11 7:88000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d805e580 2023-11-08T16:34:44.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.493+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fa1d805e580 0x7fa1d8054d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.494+0000 7fa1ee025640 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fa1d805e580 0x7fa1d8054d70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.495+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(18 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1d805e580 2023-11-08T16:34:44.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.495+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:19 7.9 7:90000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d8056970 2023-11-08T16:34:44.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.495+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(19 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8056970 2023-11-08T16:34:44.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.495+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:20 7.19 7:98000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e00688b0 2023-11-08T16:34:44.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.496+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(20 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e00688b0 2023-11-08T16:34:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.496+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:21 7.5 7:a0000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d805e580 2023-11-08T16:34:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.497+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(21 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1d805e580 2023-11-08T16:34:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.497+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:22 7.15 7:a8000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d805e580 2023-11-08T16:34:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.499+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(22 [pgnls start_epoch 63 out=97b] v56'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1d805e580 2023-11-08T16:34:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.499+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:23 7.d 7:b0000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e00688b0 2023-11-08T16:34:44.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.500+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(23 [pgnls start_epoch 63 out=74b] v56'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e00688b0 2023-11-08T16:34:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.500+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:24 7.1d 7:b8000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d8055c00 2023-11-08T16:34:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.500+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(24 [pgnls start_epoch 63 out=73b] v56'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8055c00 2023-11-08T16:34:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.500+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:25 7.3 7:c0000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e800ec30 2023-11-08T16:34:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.500+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(25 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.501+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:26 7.13 7:c8000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d8056970 2023-11-08T16:34:44.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.501+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(26 [pgnls start_epoch 63 out=98b] v62'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8056970 2023-11-08T16:34:44.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.501+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:27 7.b 7:d0000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d8055c00 2023-11-08T16:34:44.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.501+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(27 [pgnls start_epoch 63 out=76b] v62'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8055c00 2023-11-08T16:34:44.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.501+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:28 7.1b 7:d8000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d8056970 2023-11-08T16:34:44.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.502+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(28 [pgnls start_epoch 63 out=49b] v56'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8056970 2023-11-08T16:34:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.502+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:29 7.7 7:e0000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e00688b0 2023-11-08T16:34:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.503+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(29 [pgnls start_epoch 63 out=73b] v56'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e00688b0 2023-11-08T16:34:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.503+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:30 7.17 7:e8000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d805e580 2023-11-08T16:34:44.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.504+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(30 [pgnls start_epoch 63 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1d805e580 2023-11-08T16:34:44.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.504+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:31 7.f 7:f0000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1d8056970 2023-11-08T16:34:44.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.504+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(31 [pgnls start_epoch 63 out=73b] v56'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8056970 2023-11-08T16:34:44.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.504+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:32 7.1f 7:f8000000:%01:::head [pgnls start_epoch 63 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e63) v8 -- 0x556f3ec084b0 con 0x7fa1e00688b0 2023-11-08T16:34:44.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.505+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(32 [pgnls start_epoch 63 out=101b] v62'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e00688b0 2023-11-08T16:34:44.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.505+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:33 7.10 7:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ec1bb00 con 0x7fa1e800ec30 2023-11-08T16:34:44.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:34 7.10 7:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ec1c610 con 0x7fa1e800ec30 2023-11-08T16:34:44.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:35 7.10 7:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc20c0 con 0x7fa1e800ec30 2023-11-08T16:34:44.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:36 7.10 7:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc28c0 con 0x7fa1e800ec30 2023-11-08T16:34:44.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:37 7.10 7:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc30c0 con 0x7fa1e800ec30 2023-11-08T16:34:44.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:38 7.14 7:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc38c0 con 0x7fa1e800ec30 2023-11-08T16:34:44.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:39 7.14 7:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc40c0 con 0x7fa1e800ec30 2023-11-08T16:34:44.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:40 7.14 7:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc30c0 con 0x7fa1e800ec30 2023-11-08T16:34:44.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:41 7.14 7:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc38c0 con 0x7fa1e800ec30 2023-11-08T16:34:44.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:42 7.1c 7:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc5200 con 0x556f3eb73df0 2023-11-08T16:34:44.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:43 7.6 7:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc5b10 con 0x7fa1d8056970 2023-11-08T16:34:44.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:44 7.e 7:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc6310 con 0x7fa1e005d3f0 2023-11-08T16:34:44.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:45 7.15 7:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc38c0 con 0x7fa1d805e580 2023-11-08T16:34:44.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:46 7.15 7:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc7250 con 0x7fa1d805e580 2023-11-08T16:34:44.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:47 7.d 7:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc7d10 con 0x7fa1e00688b0 2023-11-08T16:34:44.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.506+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:48 7.1d 7:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc87d0 con 0x7fa1d8055c00 2023-11-08T16:34:44.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.511+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(48 605.00000000 [delete] v63'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8055c00 2023-11-08T16:34:44.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.511+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(47 mds_snaptable [delete] v63'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e00688b0 2023-11-08T16:34:44.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.511+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(45 604.00000000 [delete] v63'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1d805e580 2023-11-08T16:34:44.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.511+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v63'25 uv25 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8056970 2023-11-08T16:34:44.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.511+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:49 7.13 7:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc7250 con 0x7fa1d8056970 2023-11-08T16:34:44.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.511+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:50 7.13 7:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc7cd0 con 0x7fa1d8056970 2023-11-08T16:34:44.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.511+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:51 7.b 7:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc7750 con 0x7fa1d8055c00 2023-11-08T16:34:44.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.511+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:52 7.7 7:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc9050 con 0x7fa1e00688b0 2023-11-08T16:34:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.512+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(46 500.00000000 [delete] v63'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1d805e580 2023-11-08T16:34:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.512+0000 7fa1e5824640 1 -- 172.21.15.81:0/264735660 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(44 608.00000000 [delete] v63'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1d8085f50 con 0x7fa1e005d3f0 2023-11-08T16:34:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.512+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:53 7.f 7:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc5e70 con 0x7fa1d8056970 2023-11-08T16:34:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.512+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:54 7.1f 7:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ecc94f0 con 0x7fa1e00688b0 2023-11-08T16:34:44.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.512+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(33 601.00000000 [delete] v63'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.512+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(38 1.00000000.inode [delete] v63'97 uv97 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.513+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(39 200.00000000 [delete] v63'98 uv98 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.513+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(34 602.00000000 [delete] v63'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.513+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:55 7.1f 7:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e63) v8 -- 0x556f3ec1c170 con 0x7fa1e00688b0 2023-11-08T16:34:44.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.513+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(42 607.00000000 [delete] v63'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e800e9e0 con 0x556f3eb73df0 2023-11-08T16:34:44.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.514+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(40 200.00000001 [delete] v63'99 uv99 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.514+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(35 10000000000.00000000 [delete] v63'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.514+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(41 606.00000000 [delete] v63'100 uv100 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.514+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(36 600.00000000 [delete] v63'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.514+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(37 603.00000000 [delete] v63'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e800ec30 2023-11-08T16:34:44.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.514+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(53 400.00000000 [delete] v63'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8056970 2023-11-08T16:34:44.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.515+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(49 mds0_inotable [delete] v63'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8056970 2023-11-08T16:34:44.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.515+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(51 mds0_sessionmap [delete] v63'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8055c00 2023-11-08T16:34:44.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.517+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 <== osd.0 v2:172.21.15.81:6808/1666604866 11 ==== osd_op_reply(50 100.00000000 [delete] v63'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e800e9e0 con 0x7fa1d8056970 2023-11-08T16:34:44.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.517+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(52 609.00000000 [delete] v63'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e00688b0 2023-11-08T16:34:44.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.518+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(54 100.00000000.inode [delete] v63'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e00688b0 2023-11-08T16:34:44.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1ee025640 1 -- 172.21.15.81:0/264735660 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(55 1.00000000 [delete] v63'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fa1e0072040 con 0x7fa1e00688b0 2023-11-08T16:34:44.526 INFO:teuthology.orchestra.run.smithi081.stdout:Removed 23 objects 2023-11-08T16:34:44.527 INFO:teuthology.orchestra.run.smithi081.stdout:successfully purged pool cephfs_metadata 2023-11-08T16:34:44.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fa1d8056970 msgr2=0x7fa1d805c5b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fa1d8056970 0x7fa1d805c5b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fa1d8055c00 msgr2=0x7fa1d805a120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fa1d8055c00 0x7fa1d805a120 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7fa1e005d3f0 msgr2=0x7fa1e005d850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7fa1e005d3f0 0x7fa1e005d850 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7fa1e800ec30 msgr2=0x7fa1e801c400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7fa1e800ec30 0x7fa1e801c400 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7fa1e0079040 msgr2=0x7fa1e0079420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7fa1e0079040 0x7fa1e0079420 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x556f3eb73df0 msgr2=0x556f3ecb93a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x556f3eb73df0 0x556f3ecb93a0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7fa1e00688b0 msgr2=0x7fa1e0006380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7fa1e00688b0 0x7fa1e0006380 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fa1d805e580 msgr2=0x7fa1d8054d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fa1d805e580 0x7fa1d8054d70 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 reap_dead start 2023-11-08T16:34:44.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 reap_dead start 2023-11-08T16:34:44.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.519+0000 7fa1ee826640 1 -- 172.21.15.81:0/264735660 reap_dead start 2023-11-08T16:34:44.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.520+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1d0069820 msgr2=0x7fa1d006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.520+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1d0069820 0x7fa1d006bce0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa1e0054000 tx=0x7fa1e004e000 comp rx=0 tx=0).stop 2023-11-08T16:34:44.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.520+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a580 msgr2=0x556f3eb738b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.520+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a580 0x556f3eb738b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa1d8007dc0 tx=0x7fa1d8006750 comp rx=0 tx=0).stop 2023-11-08T16:34:44.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.520+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 shutdown_connections 2023-11-08T16:34:44.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.520+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1d0069820 0x7fa1d006bce0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.520+0000 7fa1f0c39f40 1 --2- 172.21.15.81:0/264735660 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x556f3ec1a580 0x556f3eb738b0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.520+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 >> 172.21.15.81:0/264735660 conn(0x556f3eb8fc60 msgr2=0x556f3eb8d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:44.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.520+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 shutdown_connections 2023-11-08T16:34:44.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.520+0000 7fa1f0c39f40 1 -- 172.21.15.81:0/264735660 wait complete. 2023-11-08T16:34:44.536 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-11-08T16:34:44.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.744+0000 7f889f17c640 1 Processor -- start 2023-11-08T16:34:44.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.745+0000 7f889f17c640 1 -- start start 2023-11-08T16:34:44.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.746+0000 7f889f17c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f88980b0bf0 0x7f88980ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.746+0000 7f889f17c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8898071460 con 0x7f889815e550 2023-11-08T16:34:44.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.746+0000 7f889f17c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88980703c0 con 0x7f889815e170 2023-11-08T16:34:44.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.746+0000 7f889f17c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88980706a0 con 0x7f88980b0bf0 2023-11-08T16:34:44.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.746+0000 7f8897fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f889815e550 0x7f88980b04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:43074/0 (socket says 172.21.15.81:43074) 2023-11-08T16:34:44.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.746+0000 7f8897fff640 1 -- 172.21.15.81:0/1240524030 learned_addr learned my addr 172.21.15.81:0/1240524030 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:44.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f889d719640 1 --2- 172.21.15.81:0/1240524030 >> v2:172.21.15.93:3300/0 conn(0x7f88980b0bf0 0x7f88980ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1166050255 0 0) 0x7f8898071460 con 0x7f889815e550 2023-11-08T16:34:44.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8880003520 con 0x7f889815e550 2023-11-08T16:34:44.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3229243096 0 0) 0x7f8880003520 con 0x7f889815e550 2023-11-08T16:34:44.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 >> v2:172.21.15.93:3300/0 conn(0x7f88980b0bf0 msgr2=0x7f88980ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:44.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 --2- 172.21.15.81:0/1240524030 >> v2:172.21.15.93:3300/0 conn(0x7f88980b0bf0 0x7f88980ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 >> v1:172.21.15.93:6789/0 conn(0x7f889815e170 legacy=0x7f889815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8898070980 con 0x7f889815e550 2023-11-08T16:34:44.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f888c001670 con 0x7f889815e550 2023-11-08T16:34:44.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 >> v1:172.21.15.81:6789/0 conn(0x7f889815e550 legacy=0x7f88980b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 --2- 172.21.15.81:0/1240524030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88800039e0 0x7f8880003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 --2- 172.21.15.81:0/1240524030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88800047c0 0x7f8880004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 --2- 172.21.15.81:0/1240524030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8880005180 0x7f888000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88980706a0 con 0x7f88800039e0 2023-11-08T16:34:44.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888000c0b0 con 0x7f88800047c0 2023-11-08T16:34:44.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888000c270 con 0x7f8880005180 2023-11-08T16:34:44.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.747+0000 7f889cf18640 1 --2- 172.21.15.81:0/1240524030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88800039e0 0x7f8880003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.748+0000 7f8897fff640 1 --2- 172.21.15.81:0/1240524030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88800047c0 0x7f8880004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.748+0000 7f889cf18640 1 -- 172.21.15.81:0/1240524030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8880005180 msgr2=0x7f888000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:44.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.748+0000 7f889cf18640 1 --2- 172.21.15.81:0/1240524030 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8880005180 0x7f888000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.748+0000 7f889cf18640 1 -- 172.21.15.81:0/1240524030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88800047c0 msgr2=0x7f8880004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:44.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.748+0000 7f889cf18640 1 --2- 172.21.15.81:0/1240524030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88800047c0 0x7f8880004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.748+0000 7f889cf18640 1 -- 172.21.15.81:0/1240524030 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8880003520 con 0x7f88800039e0 2023-11-08T16:34:44.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.748+0000 7f889d719640 1 -- 172.21.15.81:0/1240524030 reap_dead start 2023-11-08T16:34:44.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.749+0000 7f889cf18640 1 --2- 172.21.15.81:0/1240524030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88800039e0 0x7f8880003e00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8888001750 tx=0x7f88800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b93050d1c1559dda server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.749+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88880052d0 con 0x7f88800039e0 2023-11-08T16:34:44.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.749+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888802e050 con 0x7f88800039e0 2023-11-08T16:34:44.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.749+0000 7f88977fe640 1 -- 172.21.15.81:0/1240524030 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8888003830 con 0x7f88800039e0 2023-11-08T16:34:44.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.750+0000 7f889f17c640 1 -- 172.21.15.81:0/1240524030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88800039e0 msgr2=0x7f8880003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.750+0000 7f889f17c640 1 --2- 172.21.15.81:0/1240524030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88800039e0 0x7f8880003e00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f8888001750 tx=0x7f88800057f0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.750+0000 7f889f17c640 1 -- 172.21.15.81:0/1240524030 shutdown_connections 2023-11-08T16:34:44.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.750+0000 7f889f17c640 1 --2- 172.21.15.81:0/1240524030 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88800039e0 0x7f8880003e00 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.750+0000 7f889f17c640 1 -- 172.21.15.81:0/1240524030 >> 172.21.15.81:0/1240524030 conn(0x7f88980cb4e0 msgr2=0x7f88980cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:44.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.750+0000 7f889f17c640 1 -- 172.21.15.81:0/1240524030 shutdown_connections 2023-11-08T16:34:44.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.750+0000 7f889f17c640 1 -- 172.21.15.81:0/1240524030 wait complete. 2023-11-08T16:34:44.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.752+0000 7f889f17c640 1 Processor -- start 2023-11-08T16:34:44.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.752+0000 7f889f17c640 1 -- start start 2023-11-08T16:34:44.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889f17c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88800039e0 0x7f88980b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889f17c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88981f5e00 0x7f88980c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889f17c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88980c3700 0x7f8898154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889f17c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88980c4b60 con 0x7f88980c3700 2023-11-08T16:34:44.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889f17c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88980c3f50 con 0x7f88981f5e00 2023-11-08T16:34:44.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889f17c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88980c4230 con 0x7f88800039e0 2023-11-08T16:34:44.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889d719640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88980c3700 0x7f8898154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889d719640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88980c3700 0x7f8898154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47820/0 (socket says 172.21.15.81:47820) 2023-11-08T16:34:44.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889d719640 1 -- 172.21.15.81:0/1193919053 learned_addr learned my addr 172.21.15.81:0/1193919053 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:44.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889d719640 1 -- 172.21.15.81:0/1193919053 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88800039e0 msgr2=0x7f88980b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889d719640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88800039e0 0x7f88980b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889d719640 1 -- 172.21.15.81:0/1193919053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88981f5e00 msgr2=0x7f88980c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:44.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889d719640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88981f5e00 0x7f88980c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:44.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.753+0000 7f889d719640 1 -- 172.21.15.81:0/1193919053 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88980c4510 con 0x7f88980c3700 2023-11-08T16:34:44.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.754+0000 7f889d719640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88980c3700 0x7f8898154920 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f88980ccc30 tx=0x7f889000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.754+0000 7f88957fa640 1 -- 172.21.15.81:0/1193919053 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890010040 con 0x7f88980c3700 2023-11-08T16:34:44.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.754+0000 7f88957fa640 1 -- 172.21.15.81:0/1193919053 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889000bc80 con 0x7f88980c3700 2023-11-08T16:34:44.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.754+0000 7f88957fa640 1 -- 172.21.15.81:0/1193919053 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890014400 con 0x7f88980c3700 2023-11-08T16:34:44.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.754+0000 7f889f17c640 1 -- 172.21.15.81:0/1193919053 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88980c47f0 con 0x7f88980c3700 2023-11-08T16:34:44.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.754+0000 7f889f17c640 1 -- 172.21.15.81:0/1193919053 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88981f6260 con 0x7f88980c3700 2023-11-08T16:34:44.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.755+0000 7f889f17c640 1 -- 172.21.15.81:0/1193919053 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f889804ece0 con 0x7f88980c3700 2023-11-08T16:34:44.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.756+0000 7f88957fa640 1 -- 172.21.15.81:0/1193919053 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8890014560 con 0x7f88980c3700 2023-11-08T16:34:44.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.757+0000 7f88957fa640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f888406ae70 0x7f888406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:44.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.757+0000 7f88957fa640 1 -- 172.21.15.81:0/1193919053 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6286+0+0 (secure 0 0 0) 0x7f8890089e50 con 0x7f88980c3700 2023-11-08T16:34:44.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.757+0000 7f889cf18640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f888406ae70 0x7f888406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:44.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.758+0000 7f889cf18640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f888406ae70 0x7f888406d330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8888033000 tx=0x7f888802f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:44.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.762+0000 7f88957fa640 1 -- 172.21.15.81:0/1193919053 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8890054b90 con 0x7f88980c3700 2023-11-08T16:34:44.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:44.992+0000 7f889f17c640 1 -- 172.21.15.81:0/1193919053 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f88980ccc60 con 0x7f88980c3700 2023-11-08T16:34:45.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.923+0000 7f88957fa640 1 -- 172.21.15.81:0/1193919053 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 107+0+0 (secure 0 0 0) 0x7f88900581b0 con 0x7f88980c3700 2023-11-08T16:34:45.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 -- 172.21.15.81:0/1193919053 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f888406ae70 msgr2=0x7f888406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:45.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f888406ae70 0x7f888406d330 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8888033000 tx=0x7f888802f000 comp rx=0 tx=0).stop 2023-11-08T16:34:45.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 -- 172.21.15.81:0/1193919053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88980c3700 msgr2=0x7f8898154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:45.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88980c3700 0x7f8898154920 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f88980ccc30 tx=0x7f889000b480 comp rx=0 tx=0).stop 2023-11-08T16:34:45.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 -- 172.21.15.81:0/1193919053 shutdown_connections 2023-11-08T16:34:45.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f888406ae70 0x7f888406d330 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:45.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f88980c3700 0x7f8898154920 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:45.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f88981f5e00 0x7f88980c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:45.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 --2- 172.21.15.81:0/1193919053 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f88800039e0 0x7f88980b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:45.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 -- 172.21.15.81:0/1193919053 >> 172.21.15.81:0/1193919053 conn(0x7f88980cb4e0 msgr2=0x7f889806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:45.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 -- 172.21.15.81:0/1193919053 shutdown_connections 2023-11-08T16:34:45.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:45.927+0000 7f889f17c640 1 -- 172.21.15.81:0/1193919053 wait complete. 2023-11-08T16:34:45.940 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:34:45.939+0000 7fe2c3ee9640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-11-08T16:34:45.940 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:34:45.939+0000 7fe2c3ee9640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-11-08T16:34:45.941 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:34:45.939+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:34:45.941 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:34:45.940+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-08T16:34:45.952 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-08T16:34:45.958 INFO:tasks.ceph.mds.a.smithi081.stderr: -32> 2023-11-08T16:34:45.939+0000 7fe2c3ee9640 -1 mds.0.sessionmap _load_finish got (2) No such file or directory 2023-11-08T16:34:45.958 INFO:tasks.ceph.mds.a.smithi081.stderr: -31> 2023-11-08T16:34:45.939+0000 7fe2c3ee9640 -1 log_channel(cluster) log [ERR] : error reading sessionmap 'mds0_sessionmap' -2 ((2) No such file or directory) 2023-11-08T16:34:46.002 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:34:46.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.149+0000 7f6ed30fc640 1 Processor -- start 2023-11-08T16:34:46.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.149+0000 7f6ed30fc640 1 -- start start 2023-11-08T16:34:46.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.151+0000 7f6ed30fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ecc0701f0 0x7f6ecc0ca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:46.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.151+0000 7f6ed30fc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6ecc06a950 0x7f6ecc06ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:46.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.151+0000 7f6ed30fc640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ecc0cb950 con 0x7f6ecc15b040 2023-11-08T16:34:46.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.151+0000 7f6ed30fc640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ecc0cbc30 con 0x7f6ecc0701f0 2023-11-08T16:34:46.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.151+0000 7f6ed30fc640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ecc06a150 con 0x7f6ecc06a950 2023-11-08T16:34:46.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.151+0000 7f6ecbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ecc0701f0 0x7f6ecc0ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:46.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.152+0000 7f6ecbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6ecc0701f0 0x7f6ecc0ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47844/0 (socket says 172.21.15.81:47844) 2023-11-08T16:34:46.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.152+0000 7f6ecbfff640 1 -- 172.21.15.81:0/118237948 learned_addr learned my addr 172.21.15.81:0/118237948 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:46.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.152+0000 7f6ed1699640 1 --2- 172.21.15.81:0/118237948 >> v2:172.21.15.93:3300/0 conn(0x7f6ecc06a950 0x7f6ecc06ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:46.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.152+0000 7f6ecbfff640 1 -- 172.21.15.81:0/118237948 >> v2:172.21.15.93:3300/0 conn(0x7f6ecc06a950 msgr2=0x7f6ecc06ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:46.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.152+0000 7f6ecbfff640 1 --2- 172.21.15.81:0/118237948 >> v2:172.21.15.93:3300/0 conn(0x7f6ecc06a950 0x7f6ecc06ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:46.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.152+0000 7f6ecbfff640 1 -- 172.21.15.81:0/118237948 >> v1:172.21.15.81:6789/0 conn(0x7f6ecc15b040 legacy=0x7f6ecc0c9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:46.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.152+0000 7f6ecbfff640 1 -- 172.21.15.81:0/118237948 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc06a430 con 0x7f6ecc0701f0 2023-11-08T16:34:46.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.153+0000 7f6ecbfff640 1 --2- 172.21.15.81:0/118237948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc0701f0 0x7f6ecc0ca150 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc068410 tx=0x7f6eb802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c01cc56242c12c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:46.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.153+0000 7f6ecb7fe640 1 -- 172.21.15.81:0/118237948 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8031870 con 0x7f6ecc0701f0 2023-11-08T16:34:46.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.154+0000 7f6ecb7fe640 1 -- 172.21.15.81:0/118237948 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb80319d0 con 0x7f6ecc0701f0 2023-11-08T16:34:46.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.154+0000 7f6ecb7fe640 1 -- 172.21.15.81:0/118237948 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8031d60 con 0x7f6ecc0701f0 2023-11-08T16:34:46.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.154+0000 7f6ed30fc640 1 -- 172.21.15.81:0/118237948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc0701f0 msgr2=0x7f6ecc0ca150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:46.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.154+0000 7f6ed30fc640 1 --2- 172.21.15.81:0/118237948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc0701f0 0x7f6ecc0ca150 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc068410 tx=0x7f6eb802b910 comp rx=0 tx=0).stop 2023-11-08T16:34:46.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.154+0000 7f6ed30fc640 1 -- 172.21.15.81:0/118237948 shutdown_connections 2023-11-08T16:34:46.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.154+0000 7f6ed30fc640 1 --2- 172.21.15.81:0/118237948 >> v2:172.21.15.93:3300/0 conn(0x7f6ecc06a950 0x7f6ecc06ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:46.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.154+0000 7f6ed30fc640 1 --2- 172.21.15.81:0/118237948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc0701f0 0x7f6ecc0ca150 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:46.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.154+0000 7f6ed30fc640 1 -- 172.21.15.81:0/118237948 >> 172.21.15.81:0/118237948 conn(0x7f6ecc0cbe70 msgr2=0x7f6ecc0cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:46.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.155+0000 7f6ed30fc640 1 -- 172.21.15.81:0/118237948 shutdown_connections 2023-11-08T16:34:46.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.155+0000 7f6ed30fc640 1 -- 172.21.15.81:0/118237948 wait complete. 2023-11-08T16:34:46.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.157+0000 7f6ed30fc640 1 Processor -- start 2023-11-08T16:34:46.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.157+0000 7f6ed30fc640 1 -- start start 2023-11-08T16:34:46.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.158+0000 7f6ed30fc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ecc06a950 0x7f6ecc14ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:46.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.158+0000 7f6ed30fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc0701f0 0x7f6ecc157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:46.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.158+0000 7f6ed30fc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ecc15b040 0x7f6ecc157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:46.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.158+0000 7f6ed30fc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0ad5d0 con 0x7f6ecc0701f0 2023-11-08T16:34:46.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.158+0000 7f6ed30fc640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0ad8b0 con 0x7f6ecc06a950 2023-11-08T16:34:46.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.158+0000 7f6ed30fc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0adb90 con 0x7f6ecc15b040 2023-11-08T16:34:46.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.158+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc0701f0 0x7f6ecc157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:46.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.158+0000 7f6ecbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc0701f0 0x7f6ecc157a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47858/0 (socket says 172.21.15.81:47858) 2023-11-08T16:34:46.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.158+0000 7f6ecbfff640 1 -- 172.21.15.81:0/2134108457 learned_addr learned my addr 172.21.15.81:0/2134108457 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:46.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.159+0000 7f6ed1699640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ecc15b040 0x7f6ecc157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:46.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.159+0000 7f6ed0e98640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ecc06a950 0x7f6ecc14ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:46.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.159+0000 7f6ecbfff640 1 -- 172.21.15.81:0/2134108457 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ecc15b040 msgr2=0x7f6ecc157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:46.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.159+0000 7f6ecbfff640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ecc15b040 0x7f6ecc157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:46.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.159+0000 7f6ecbfff640 1 -- 172.21.15.81:0/2134108457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ecc06a950 msgr2=0x7f6ecc14ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:46.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.159+0000 7f6ecbfff640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ecc06a950 0x7f6ecc14ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:46.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.159+0000 7f6ecbfff640 1 -- 172.21.15.81:0/2134108457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb80094d0 con 0x7f6ecc0701f0 2023-11-08T16:34:46.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.159+0000 7f6ed1699640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ecc15b040 0x7f6ecc157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:46.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.159+0000 7f6ed0e98640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ecc06a950 0x7f6ecc14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:46.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.159+0000 7f6ecbfff640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc0701f0 0x7f6ecc157a30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8000c00 tx=0x7f6eb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:46.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.160+0000 7f6ec97fa640 1 -- 172.21.15.81:0/2134108457 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb803aca0 con 0x7f6ecc0701f0 2023-11-08T16:34:46.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.160+0000 7f6ec97fa640 1 -- 172.21.15.81:0/2134108457 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb803ae00 con 0x7f6ecc0701f0 2023-11-08T16:34:46.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.160+0000 7f6ec97fa640 1 -- 172.21.15.81:0/2134108457 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb8044630 con 0x7f6ecc0701f0 2023-11-08T16:34:46.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.160+0000 7f6ed30fc640 1 -- 172.21.15.81:0/2134108457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc0ade70 con 0x7f6ecc0701f0 2023-11-08T16:34:46.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.160+0000 7f6ed30fc640 1 -- 172.21.15.81:0/2134108457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc0ae150 con 0x7f6ecc0701f0 2023-11-08T16:34:46.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.161+0000 7f6ed30fc640 1 -- 172.21.15.81:0/2134108457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc04ece0 con 0x7f6ecc0701f0 2023-11-08T16:34:46.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.162+0000 7f6ec97fa640 1 -- 172.21.15.81:0/2134108457 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6eb804d450 con 0x7f6ecc0701f0 2023-11-08T16:34:46.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.163+0000 7f6ec97fa640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6eb006ae70 0x7f6eb006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:46.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.164+0000 7f6ec97fa640 1 -- 172.21.15.81:0/2134108457 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x7f6eb802d040 con 0x7f6ecc0701f0 2023-11-08T16:34:46.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.164+0000 7f6ed0e98640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6eb006ae70 0x7f6eb006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:46.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.164+0000 7f6ed0e98640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6eb006ae70 0x7f6eb006d330 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0cf1a0 tx=0x7f6ec0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:46.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.168+0000 7f6ec97fa640 1 -- 172.21.15.81:0/2134108457 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6eb8076c80 con 0x7f6ecc0701f0 2023-11-08T16:34:46.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.399+0000 7f6ed30fc640 1 -- 172.21.15.81:0/2134108457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f6ecc0ae9b0 con 0x7f6ecc0701f0 2023-11-08T16:34:46.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.946+0000 7f6ec97fa640 1 -- 172.21.15.81:0/2134108457 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v47) v1 ==== 162+0+0 (secure 0 0 0) 0x7f6eb807a2a0 con 0x7f6ecc0701f0 2023-11-08T16:34:46.948 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-08T16:34:46.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 -- 172.21.15.81:0/2134108457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6eb006ae70 msgr2=0x7f6eb006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:46.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6eb006ae70 0x7f6eb006d330 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc0cf1a0 tx=0x7f6ec0009040 comp rx=0 tx=0).stop 2023-11-08T16:34:46.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 -- 172.21.15.81:0/2134108457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc0701f0 msgr2=0x7f6ecc157a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:46.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc0701f0 0x7f6ecc157a30 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f6eb8000c00 tx=0x7f6eb80046c0 comp rx=0 tx=0).stop 2023-11-08T16:34:46.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 -- 172.21.15.81:0/2134108457 shutdown_connections 2023-11-08T16:34:46.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6eb006ae70 0x7f6eb006d330 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:46.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6ecc15b040 0x7f6ecc157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:46.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6ecc0701f0 0x7f6ecc157a30 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:46.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 --2- 172.21.15.81:0/2134108457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6ecc06a950 0x7f6ecc14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:46.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 -- 172.21.15.81:0/2134108457 >> 172.21.15.81:0/2134108457 conn(0x7f6ecc0cbe70 msgr2=0x7f6ecc06d990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:46.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 -- 172.21.15.81:0/2134108457 shutdown_connections 2023-11-08T16:34:46.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:46.950+0000 7f6ed30fc640 1 -- 172.21.15.81:0/2134108457 wait complete. 2023-11-08T16:34:46.973 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:34:47.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.184+0000 7f5dbbe1a640 1 Processor -- start 2023-11-08T16:34:47.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.184+0000 7f5dbbe1a640 1 -- start start 2023-11-08T16:34:47.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.186+0000 7f5dbbe1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5db406b0c0 0x7f5db40c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.186+0000 7f5dbbe1a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5db40cb980 0x7f5db4164c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.186+0000 7f5dbbe1a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5db40c5030 con 0x7f5db415b6d0 2023-11-08T16:34:47.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.186+0000 7f5dbbe1a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5db40c61f0 con 0x7f5db40cb980 2023-11-08T16:34:47.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.186+0000 7f5dbbe1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40c4420 con 0x7f5db406b0c0 2023-11-08T16:34:47.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.187+0000 7f5dba3b7640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5db40cb980 0x7f5db4164c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.187+0000 7f5dba3b7640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5db40cb980 0x7f5db4164c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:51464/0 (socket says 172.21.15.81:51464) 2023-11-08T16:34:47.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.187+0000 7f5dba3b7640 1 -- 172.21.15.81:0/2469547466 learned_addr learned my addr 172.21.15.81:0/2469547466 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:47.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.187+0000 7f5db93b5640 1 --2- 172.21.15.81:0/2469547466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5db406b0c0 0x7f5db40c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.188+0000 7f5db93b5640 1 -- 172.21.15.81:0/2469547466 >> v2:172.21.15.93:3300/0 conn(0x7f5db40cb980 msgr2=0x7f5db4164c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:47.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.188+0000 7f5db93b5640 1 --2- 172.21.15.81:0/2469547466 >> v2:172.21.15.93:3300/0 conn(0x7f5db40cb980 0x7f5db4164c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.188+0000 7f5db93b5640 1 -- 172.21.15.81:0/2469547466 >> v1:172.21.15.93:6789/0 conn(0x7f5db415b6d0 legacy=0x7f5db415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.188+0000 7f5db93b5640 1 -- 172.21.15.81:0/2469547466 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db40c4700 con 0x7f5db406b0c0 2023-11-08T16:34:47.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.189+0000 7f5db93b5640 1 --2- 172.21.15.81:0/2469547466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5db406b0c0 0x7f5db40c5cb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009d90 tx=0x7f5da802be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5b0173b80cde770 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:47.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.190+0000 7f5db8bb4640 1 -- 172.21.15.81:0/2469547466 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da802cbd0 con 0x7f5db406b0c0 2023-11-08T16:34:47.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.190+0000 7f5db8bb4640 1 -- 172.21.15.81:0/2469547466 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5da802cd30 con 0x7f5db406b0c0 2023-11-08T16:34:47.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.190+0000 7f5db8bb4640 1 -- 172.21.15.81:0/2469547466 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5da802cea0 con 0x7f5db406b0c0 2023-11-08T16:34:47.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.190+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/2469547466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5db406b0c0 msgr2=0x7f5db40c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.191+0000 7f5dbbe1a640 1 --2- 172.21.15.81:0/2469547466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5db406b0c0 0x7f5db40c5cb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009d90 tx=0x7f5da802be20 comp rx=0 tx=0).stop 2023-11-08T16:34:47.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.191+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/2469547466 shutdown_connections 2023-11-08T16:34:47.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.191+0000 7f5dbbe1a640 1 --2- 172.21.15.81:0/2469547466 >> v2:172.21.15.93:3300/0 conn(0x7f5db40cb980 0x7f5db4164c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.191+0000 7f5dbbe1a640 1 --2- 172.21.15.81:0/2469547466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5db406b0c0 0x7f5db40c5cb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.191+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/2469547466 >> 172.21.15.81:0/2469547466 conn(0x7f5db40cbe40 msgr2=0x7f5db40cc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:47.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.191+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/2469547466 shutdown_connections 2023-11-08T16:34:47.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.191+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/2469547466 wait complete. 2023-11-08T16:34:47.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.192+0000 7f5dbbe1a640 1 Processor -- start 2023-11-08T16:34:47.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.193+0000 7f5dbbe1a640 1 -- start start 2023-11-08T16:34:47.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5dbbe1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5db406b0c0 0x7f5db4144ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5dbbe1a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5db40cb980 0x7f5db40b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5dbbe1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5db415b6d0 0x7f5db40b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5dbbe1a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db4153600 con 0x7f5db406b0c0 2023-11-08T16:34:47.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5dbbe1a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db40c6fc0 con 0x7f5db40cb980 2023-11-08T16:34:47.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5dbbe1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5db41529f0 con 0x7f5db415b6d0 2023-11-08T16:34:47.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5db9bb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5db406b0c0 0x7f5db4144ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5db9bb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5db406b0c0 0x7f5db4144ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47860/0 (socket says 172.21.15.81:47860) 2023-11-08T16:34:47.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5db9bb6640 1 -- 172.21.15.81:0/3030527469 learned_addr learned my addr 172.21.15.81:0/3030527469 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:47.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5db93b5640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5db40cb980 0x7f5db40b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5dba3b7640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5db415b6d0 0x7f5db40b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5db9bb6640 1 -- 172.21.15.81:0/3030527469 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5db415b6d0 msgr2=0x7f5db40b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5db9bb6640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5db415b6d0 0x7f5db40b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5db9bb6640 1 -- 172.21.15.81:0/3030527469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5db40cb980 msgr2=0x7f5db40b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5db9bb6640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5db40cb980 0x7f5db40b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5db9bb6640 1 -- 172.21.15.81:0/3030527469 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5da8009a70 con 0x7f5db406b0c0 2023-11-08T16:34:47.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.194+0000 7f5db93b5640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5db40cb980 0x7f5db40b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:47.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.195+0000 7f5db9bb6640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5db406b0c0 0x7f5db4144ba0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5db000c430 tx=0x7f5db000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:47.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.195+0000 7f5da67fc640 1 -- 172.21.15.81:0/3030527469 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db00052d0 con 0x7f5db406b0c0 2023-11-08T16:34:47.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.195+0000 7f5da67fc640 1 -- 172.21.15.81:0/3030527469 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5db0005430 con 0x7f5db406b0c0 2023-11-08T16:34:47.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.195+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/3030527469 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db4152cd0 con 0x7f5db406b0c0 2023-11-08T16:34:47.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.195+0000 7f5da67fc640 1 -- 172.21.15.81:0/3030527469 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5db0003830 con 0x7f5db406b0c0 2023-11-08T16:34:47.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.195+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/3030527469 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db4153290 con 0x7f5db406b0c0 2023-11-08T16:34:47.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.196+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/3030527469 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db4153420 con 0x7f5db406b0c0 2023-11-08T16:34:47.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.197+0000 7f5da67fc640 1 -- 172.21.15.81:0/3030527469 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5db0003990 con 0x7f5db406b0c0 2023-11-08T16:34:47.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.198+0000 7f5da67fc640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5d9806ae70 0x7f5d9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.198+0000 7f5da67fc640 1 -- 172.21.15.81:0/3030527469 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x7f5db0088d80 con 0x7f5db406b0c0 2023-11-08T16:34:47.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.198+0000 7f5db93b5640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5d9806ae70 0x7f5d9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.199+0000 7f5db93b5640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5d9806ae70 0x7f5d9806d330 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5da8036000 tx=0x7f5da8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:47.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.203+0000 7f5da67fc640 1 -- 172.21.15.81:0/3030527469 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5db0053c20 con 0x7f5db406b0c0 2023-11-08T16:34:47.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.428+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/3030527469 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5db406d0c0 con 0x7f5db406b0c0 2023-11-08T16:34:47.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.429+0000 7f5da67fc640 1 -- 172.21.15.81:0/3030527469 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4901 (secure 0 0 0) 0x7f5db0017090 con 0x7f5db406b0c0 2023-11-08T16:34:47.430 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:47.431 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3492741644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3492741644},{"type":"v1","addr":"172.21.15.93:6833","nonce":3492741644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":4916,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/119699732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":119699732},{"type":"v1","addr":"172.21.15.93:6835","nonce":119699732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5170,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/1473474760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1473474760},{"type":"v1","addr":"172.21.15.81:6836","nonce":1473474760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/494395478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":494395478},{"type":"v1","addr":"172.21.15.81:6837","nonce":494395478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:46.400575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:34:47.431 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 47 2023-11-08T16:34:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.431+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/3030527469 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5d9806ae70 msgr2=0x7f5d9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.431+0000 7f5dbbe1a640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5d9806ae70 0x7f5d9806d330 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5da8036000 tx=0x7f5da8030000 comp rx=0 tx=0).stop 2023-11-08T16:34:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.431+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/3030527469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5db406b0c0 msgr2=0x7f5db4144ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.431+0000 7f5dbbe1a640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5db406b0c0 0x7f5db4144ba0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5db000c430 tx=0x7f5db000c8f0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.431+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/3030527469 shutdown_connections 2023-11-08T16:34:47.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.431+0000 7f5dbbe1a640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5d9806ae70 0x7f5d9806d330 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.431+0000 7f5dbbe1a640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5db415b6d0 0x7f5db40b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.431+0000 7f5dbbe1a640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5db40cb980 0x7f5db40b2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.431+0000 7f5dbbe1a640 1 --2- 172.21.15.81:0/3030527469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5db406b0c0 0x7f5db4144ba0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.431+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/3030527469 >> 172.21.15.81:0/3030527469 conn(0x7f5db40cbe40 msgr2=0x7f5db415d440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:47.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.432+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/3030527469 shutdown_connections 2023-11-08T16:34:47.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.432+0000 7f5dbbe1a640 1 -- 172.21.15.81:0/3030527469 wait complete. 2023-11-08T16:34:47.451 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T16:34:47.452 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:34:47.507 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:34:47.513 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:34:47.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.656+0000 7f11200a9640 1 Processor -- start 2023-11-08T16:34:47.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.657+0000 7f11200a9640 1 -- start start 2023-11-08T16:34:47.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.658+0000 7f11200a9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f11180ade50 0x7f11180ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.658+0000 7f11200a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f111815a680 0x7f111815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.658+0000 7f11200a9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f111806b630 con 0x7f111815b020 2023-11-08T16:34:47.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.658+0000 7f11200a9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11180b1860 con 0x7f11180ade50 2023-11-08T16:34:47.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.658+0000 7f11200a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11180ca9c0 con 0x7f111815a680 2023-11-08T16:34:47.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.658+0000 7f111e646640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f111815b020 0x7f11180c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:43094/0 (socket says 172.21.15.81:43094) 2023-11-08T16:34:47.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.658+0000 7f111e646640 1 -- 172.21.15.81:0/1059667953 learned_addr learned my addr 172.21.15.81:0/1059667953 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:47.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111de45640 1 --2- 172.21.15.81:0/1059667953 >> v2:172.21.15.93:3300/0 conn(0x7f11180ade50 0x7f11180ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111d644640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f111815a680 0x7f111815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785772944 0 0) 0x7f111806b630 con 0x7f111815b020 2023-11-08T16:34:47.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1104003520 con 0x7f111815b020 2023-11-08T16:34:47.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3151534738 0 0) 0x7f1104003520 con 0x7f111815b020 2023-11-08T16:34:47.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f111815a680 msgr2=0x7f111815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:47.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111ce43640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f111815a680 0x7f111815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 >> v2:172.21.15.93:3300/0 conn(0x7f11180ade50 msgr2=0x7f11180ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:47.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111ce43640 1 --2- 172.21.15.81:0/1059667953 >> v2:172.21.15.93:3300/0 conn(0x7f11180ade50 0x7f11180ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11180cabb0 con 0x7f111815b020 2023-11-08T16:34:47.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f11100019f0 con 0x7f111815b020 2023-11-08T16:34:47.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 >> v1:172.21.15.81:6789/0 conn(0x7f111815b020 legacy=0x7f11180c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.659+0000 7f111ce43640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f111815a680 0x7f1104003a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111ce43640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1104003fd0 0x7f11040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111ce43640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1104004d00 0x7f1104005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110400b980 con 0x7f1104004d00 2023-11-08T16:34:47.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110400bac0 con 0x7f1104003fd0 2023-11-08T16:34:47.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110400bc80 con 0x7f111815a680 2023-11-08T16:34:47.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111e646640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1104004d00 0x7f1104005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111d644640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1104003fd0 0x7f11040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111de45640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f111815a680 0x7f1104003a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111e646640 1 -- 172.21.15.81:0/1059667953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f111815a680 msgr2=0x7f1104003a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:47.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111e646640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f111815a680 0x7f1104003a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111e646640 1 -- 172.21.15.81:0/1059667953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1104003fd0 msgr2=0x7f11040047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:47.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111e646640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1104003fd0 0x7f11040047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.660+0000 7f111e646640 1 -- 172.21.15.81:0/1059667953 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1104003520 con 0x7f1104004d00 2023-11-08T16:34:47.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.661+0000 7f111e646640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1104004d00 0x7f1104005180 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f111000a860 tx=0x7f11040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9766fc737a6c39 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:47.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.662+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110034630 con 0x7f1104004d00 2023-11-08T16:34:47.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.662+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111002f070 con 0x7f1104004d00 2023-11-08T16:34:47.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.662+0000 7f111ce43640 1 -- 172.21.15.81:0/1059667953 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1110034a30 con 0x7f1104004d00 2023-11-08T16:34:47.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.662+0000 7f11200a9640 1 -- 172.21.15.81:0/1059667953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1104004d00 msgr2=0x7f1104005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.662+0000 7f11200a9640 1 --2- 172.21.15.81:0/1059667953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1104004d00 0x7f1104005180 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f111000a860 tx=0x7f11040057f0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.662+0000 7f111e646640 1 -- 172.21.15.81:0/1059667953 reap_dead start 2023-11-08T16:34:47.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.662+0000 7f11200a9640 1 -- 172.21.15.81:0/1059667953 shutdown_connections 2023-11-08T16:34:47.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.662+0000 7f11200a9640 1 -- 172.21.15.81:0/1059667953 >> 172.21.15.81:0/1059667953 conn(0x7f11180cb490 msgr2=0x7f11180cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:47.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.662+0000 7f11200a9640 1 -- 172.21.15.81:0/1059667953 shutdown_connections 2023-11-08T16:34:47.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.662+0000 7f11200a9640 1 -- 172.21.15.81:0/1059667953 wait complete. 2023-11-08T16:34:47.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.664+0000 7f11200a9640 1 Processor -- start 2023-11-08T16:34:47.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.664+0000 7f11200a9640 1 -- start start 2023-11-08T16:34:47.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.665+0000 7f11200a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11181f0100 0x7f1118156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.665+0000 7f11200a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11181f04e0 0x7f11180aebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.665+0000 7f11200a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11180af110 0x7f111815c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.665+0000 7f11200a9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11180af5c0 con 0x7f11181f0100 2023-11-08T16:34:47.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.665+0000 7f11200a9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1118156400 con 0x7f11180af110 2023-11-08T16:34:47.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.665+0000 7f11200a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111815d3e0 con 0x7f11181f04e0 2023-11-08T16:34:47.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.665+0000 7f111de45640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11181f0100 0x7f1118156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.665+0000 7f111de45640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11181f0100 0x7f1118156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47872/0 (socket says 172.21.15.81:47872) 2023-11-08T16:34:47.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.665+0000 7f111de45640 1 -- 172.21.15.81:0/3200315655 learned_addr learned my addr 172.21.15.81:0/3200315655 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:47.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.666+0000 7f111de45640 1 -- 172.21.15.81:0/3200315655 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11181f04e0 msgr2=0x7f11180aebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.666+0000 7f111de45640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11181f04e0 0x7f11180aebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.666+0000 7f111de45640 1 -- 172.21.15.81:0/3200315655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11180af110 msgr2=0x7f111815c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.666+0000 7f111de45640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11180af110 0x7f111815c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.666+0000 7f111de45640 1 -- 172.21.15.81:0/3200315655 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1118155ad0 con 0x7f11181f0100 2023-11-08T16:34:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.666+0000 7f111de45640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11181f0100 0x7f1118156890 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1114011aa0 tx=0x7f1114011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.666+0000 7f110e7fc640 1 -- 172.21.15.81:0/3200315655 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1114009680 con 0x7f11181f0100 2023-11-08T16:34:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.666+0000 7f110e7fc640 1 -- 172.21.15.81:0/3200315655 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111400fb00 con 0x7f11181f0100 2023-11-08T16:34:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.666+0000 7f11200a9640 1 -- 172.21.15.81:0/3200315655 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1118155db0 con 0x7f11181f0100 2023-11-08T16:34:47.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.666+0000 7f110e7fc640 1 -- 172.21.15.81:0/3200315655 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111400fe90 con 0x7f11181f0100 2023-11-08T16:34:47.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.666+0000 7f11200a9640 1 -- 172.21.15.81:0/3200315655 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111815cb80 con 0x7f11181f0100 2023-11-08T16:34:47.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.668+0000 7f11200a9640 1 -- 172.21.15.81:0/3200315655 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111804ece0 con 0x7f11181f0100 2023-11-08T16:34:47.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.668+0000 7f110e7fc640 1 -- 172.21.15.81:0/3200315655 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1114014440 con 0x7f11181f0100 2023-11-08T16:34:47.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.669+0000 7f110e7fc640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10fc06aea0 0x7f10fc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:47.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.669+0000 7f110e7fc640 1 -- 172.21.15.81:0/3200315655 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x7f111408a1f0 con 0x7f11181f0100 2023-11-08T16:34:47.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.669+0000 7f111d644640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10fc06aea0 0x7f10fc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:47.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.670+0000 7f111d644640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10fc06aea0 0x7f10fc06d360 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1118155120 tx=0x7f1108009320 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:47.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.673+0000 7f110e7fc640 1 -- 172.21.15.81:0/3200315655 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1114054f60 con 0x7f11181f0100 2023-11-08T16:34:47.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.899+0000 7f11200a9640 1 -- 172.21.15.81:0/3200315655 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11180cc1d0 con 0x7f11181f0100 2023-11-08T16:34:47.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.900+0000 7f110e7fc640 1 -- 172.21.15.81:0/3200315655 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4901 (secure 0 0 0) 0x7f1114058580 con 0x7f11181f0100 2023-11-08T16:34:47.902 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:47.902 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3492741644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3492741644},{"type":"v1","addr":"172.21.15.93:6833","nonce":3492741644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":4916,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/119699732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":119699732},{"type":"v1","addr":"172.21.15.93:6835","nonce":119699732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5170,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/1473474760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1473474760},{"type":"v1","addr":"172.21.15.81:6836","nonce":1473474760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/494395478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":494395478},{"type":"v1","addr":"172.21.15.81:6837","nonce":494395478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:46.400575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:34:47.902 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 47 2023-11-08T16:34:47.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 -- 172.21.15.81:0/3200315655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10fc06aea0 msgr2=0x7f10fc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10fc06aea0 0x7f10fc06d360 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1118155120 tx=0x7f1108009320 comp rx=0 tx=0).stop 2023-11-08T16:34:47.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 -- 172.21.15.81:0/3200315655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11181f0100 msgr2=0x7f1118156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:47.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11181f0100 0x7f1118156890 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f1114011aa0 tx=0x7f1114011f60 comp rx=0 tx=0).stop 2023-11-08T16:34:47.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 -- 172.21.15.81:0/3200315655 shutdown_connections 2023-11-08T16:34:47.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f10fc06aea0 0x7f10fc06d360 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11180af110 0x7f111815c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11181f04e0 0x7f11180aebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 --2- 172.21.15.81:0/3200315655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11181f0100 0x7f1118156890 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:47.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 -- 172.21.15.81:0/3200315655 >> 172.21.15.81:0/3200315655 conn(0x7f11180cb490 msgr2=0x7f11180afd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:47.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 -- 172.21.15.81:0/3200315655 shutdown_connections 2023-11-08T16:34:47.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:47.902+0000 7f11200a9640 1 -- 172.21.15.81:0/3200315655 wait complete. 2023-11-08T16:34:47.924 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:34:48.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.179+0000 7fb83350c640 1 Processor -- start 2023-11-08T16:34:48.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.179+0000 7fb83350c640 1 -- start start 2023-11-08T16:34:48.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.180+0000 7fb83350c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb82c06c5d0 0x7fb82c06f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.180+0000 7fb83350c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c06cce0 0x7fb82c06d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.180+0000 7fb83350c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82c0c9ec0 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.180+0000 7fb83350c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82c0cc710 con 0x7fb82c06c5d0 2023-11-08T16:34:48.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.180+0000 7fb83350c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c0cc9f0 con 0x7fb82c06cce0 2023-11-08T16:34:48.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.181+0000 7fb830aa7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb82c06c5d0 0x7fb82c06f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.181+0000 7fb830aa7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb82c06c5d0 0x7fb82c06f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47876/0 (socket says 172.21.15.81:47876) 2023-11-08T16:34:48.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.181+0000 7fb830aa7640 1 -- 172.21.15.81:0/2366964751 learned_addr learned my addr 172.21.15.81:0/2366964751 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:48.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.181+0000 7fb831aa9640 1 --2- 172.21.15.81:0/2366964751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c06cce0 0x7fb82c06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.181+0000 7fb830aa7640 1 -- 172.21.15.81:0/2366964751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c06cce0 msgr2=0x7fb82c06d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:48.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.181+0000 7fb830aa7640 1 --2- 172.21.15.81:0/2366964751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c06cce0 0x7fb82c06d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.181+0000 7fb830aa7640 1 -- 172.21.15.81:0/2366964751 >> v1:172.21.15.93:6789/0 conn(0x7fb82c0c9ae0 legacy=0x7fb82c06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.181+0000 7fb830aa7640 1 -- 172.21.15.81:0/2366964751 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c0cccd0 con 0x7fb82c06c5d0 2023-11-08T16:34:48.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.183+0000 7fb830aa7640 1 --2- 172.21.15.81:0/2366964751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c06c5d0 0x7fb82c06f8b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb81800bb10 tx=0x7fb81802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d597163414077 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:48.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.183+0000 7fb82bfff640 1 -- 172.21.15.81:0/2366964751 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81802ee70 con 0x7fb82c06c5d0 2023-11-08T16:34:48.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.183+0000 7fb82bfff640 1 -- 172.21.15.81:0/2366964751 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818036440 con 0x7fb82c06c5d0 2023-11-08T16:34:48.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.183+0000 7fb82bfff640 1 -- 172.21.15.81:0/2366964751 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180367d0 con 0x7fb82c06c5d0 2023-11-08T16:34:48.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.184+0000 7fb83350c640 1 -- 172.21.15.81:0/2366964751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c06c5d0 msgr2=0x7fb82c06f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.184+0000 7fb83350c640 1 --2- 172.21.15.81:0/2366964751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c06c5d0 0x7fb82c06f8b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fb81800bb10 tx=0x7fb81802e0c0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.184+0000 7fb83350c640 1 -- 172.21.15.81:0/2366964751 shutdown_connections 2023-11-08T16:34:48.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.184+0000 7fb83350c640 1 --2- 172.21.15.81:0/2366964751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c06cce0 0x7fb82c06d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.184+0000 7fb83350c640 1 --2- 172.21.15.81:0/2366964751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c06c5d0 0x7fb82c06f8b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.184+0000 7fb83350c640 1 -- 172.21.15.81:0/2366964751 >> 172.21.15.81:0/2366964751 conn(0x7fb82c0cb490 msgr2=0x7fb82c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:48.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.184+0000 7fb83350c640 1 -- 172.21.15.81:0/2366964751 shutdown_connections 2023-11-08T16:34:48.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.184+0000 7fb83350c640 1 -- 172.21.15.81:0/2366964751 wait complete. 2023-11-08T16:34:48.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.186+0000 7fb83350c640 1 Processor -- start 2023-11-08T16:34:48.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.186+0000 7fb83350c640 1 -- start start 2023-11-08T16:34:48.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb83350c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c06c5d0 0x7fb82c0b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb83350c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c06cce0 0x7fb82c150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb83350c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c9ae0 0x7fb82c0b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb83350c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c159d80 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb83350c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c0b24c0 con 0x7fb82c06cce0 2023-11-08T16:34:48.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb83350c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c0b1f00 con 0x7fb82c06c5d0 2023-11-08T16:34:48.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb831aa9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c9ae0 0x7fb82c0b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb831aa9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c9ae0 0x7fb82c0b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47880/0 (socket says 172.21.15.81:47880) 2023-11-08T16:34:48.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb831aa9640 1 -- 172.21.15.81:0/3801066817 learned_addr learned my addr 172.21.15.81:0/3801066817 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:48.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb830aa7640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c06cce0 0x7fb82c150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb831aa9640 1 -- 172.21.15.81:0/3801066817 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c06c5d0 msgr2=0x7fb82c0b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.187+0000 7fb8312a8640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c06c5d0 0x7fb82c0b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.188+0000 7fb831aa9640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c06c5d0 0x7fb82c0b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.188+0000 7fb831aa9640 1 -- 172.21.15.81:0/3801066817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c06cce0 msgr2=0x7fb82c150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.188+0000 7fb831aa9640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c06cce0 0x7fb82c150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.188+0000 7fb831aa9640 1 -- 172.21.15.81:0/3801066817 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81800b770 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.188+0000 7fb830aa7640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c06cce0 0x7fb82c150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:48.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.188+0000 7fb831aa9640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c9ae0 0x7fb82c0b1520 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb82400d800 tx=0x7fb82400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:48.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.188+0000 7fb829ffb640 1 -- 172.21.15.81:0/3801066817 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82400b6c0 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.188+0000 7fb829ffb640 1 -- 172.21.15.81:0/3801066817 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82400b840 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.188+0000 7fb829ffb640 1 -- 172.21.15.81:0/3801066817 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82400bbf0 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.188+0000 7fb83350c640 1 -- 172.21.15.81:0/3801066817 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82c0b21e0 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.188+0000 7fb83350c640 1 -- 172.21.15.81:0/3801066817 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c0b2b90 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.190+0000 7fb83350c640 1 -- 172.21.15.81:0/3801066817 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82c159800 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.190+0000 7fb829ffb640 1 -- 172.21.15.81:0/3801066817 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb82400bd50 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.191+0000 7fb829ffb640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006ae70 0x7fb81006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.191+0000 7fb829ffb640 1 -- 172.21.15.81:0/3801066817 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x7fb824089d20 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.192+0000 7fb8312a8640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006ae70 0x7fb81006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.192+0000 7fb8312a8640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006ae70 0x7fb81006d330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb820007fd0 tx=0x7fb820009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:48.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.197+0000 7fb829ffb640 1 -- 172.21.15.81:0/3801066817 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb82c159800 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.411+0000 7fb83350c640 1 -- 172.21.15.81:0/3801066817 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb82c159800 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.412+0000 7fb829ffb640 1 -- 172.21.15.81:0/3801066817 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4901 (secure 0 0 0) 0x7fb82c159800 con 0x7fb82c0c9ae0 2023-11-08T16:34:48.414 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:48.414 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3492741644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3492741644},{"type":"v1","addr":"172.21.15.93:6833","nonce":3492741644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":4916,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/119699732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":119699732},{"type":"v1","addr":"172.21.15.93:6835","nonce":119699732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5170,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/1473474760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1473474760},{"type":"v1","addr":"172.21.15.81:6836","nonce":1473474760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/494395478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":494395478},{"type":"v1","addr":"172.21.15.81:6837","nonce":494395478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:46.400575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:34:48.415 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 47 2023-11-08T16:34:48.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 -- 172.21.15.81:0/3801066817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006ae70 msgr2=0x7fb81006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006ae70 0x7fb81006d330 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb820007fd0 tx=0x7fb820009040 comp rx=0 tx=0).stop 2023-11-08T16:34:48.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 -- 172.21.15.81:0/3801066817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c9ae0 msgr2=0x7fb82c0b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c9ae0 0x7fb82c0b1520 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fb82400d800 tx=0x7fb82400dcc0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 -- 172.21.15.81:0/3801066817 shutdown_connections 2023-11-08T16:34:48.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006ae70 0x7fb81006d330 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c9ae0 0x7fb82c0b1520 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c06cce0 0x7fb82c150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 --2- 172.21.15.81:0/3801066817 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c06c5d0 0x7fb82c0b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 -- 172.21.15.81:0/3801066817 >> 172.21.15.81:0/3801066817 conn(0x7fb82c0cb490 msgr2=0x7fb82c070d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:48.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 -- 172.21.15.81:0/3801066817 shutdown_connections 2023-11-08T16:34:48.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.415+0000 7fb83350c640 1 -- 172.21.15.81:0/3801066817 wait complete. 2023-11-08T16:34:48.436 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:34:48.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.637+0000 7f9c79d55640 1 Processor -- start 2023-11-08T16:34:48.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.638+0000 7f9c79d55640 1 -- start start 2023-11-08T16:34:48.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.639+0000 7f9c79d55640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c740adcd0 0x7f9c740ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.639+0000 7f9c79d55640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9c740cc5e0 0x7f9c740cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.639+0000 7f9c79d55640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c740c9a70 con 0x7f9c740ccf80 2023-11-08T16:34:48.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.639+0000 7f9c79d55640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c740cc3e0 con 0x7f9c740cc5e0 2023-11-08T16:34:48.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.639+0000 7f9c79d55640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7406d070 con 0x7f9c740adcd0 2023-11-08T16:34:48.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.640+0000 7f9c6bfff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9c740cc5e0 0x7f9c740cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.640+0000 7f9c737fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c740adcd0 0x7f9c740ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.640+0000 7f9c6bfff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f9c740cc5e0 0x7f9c740cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:51512/0 (socket says 172.21.15.81:51512) 2023-11-08T16:34:48.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.640+0000 7f9c73fff640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f9c740ccf80 0x7f9c7415db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:59324/0 (socket says 172.21.15.81:59324) 2023-11-08T16:34:48.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.640+0000 7f9c6bfff640 1 -- 172.21.15.81:0/1058287283 learned_addr learned my addr 172.21.15.81:0/1058287283 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:48.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.641+0000 7f9c737fe640 1 -- 172.21.15.81:0/1058287283 >> v2:172.21.15.93:3300/0 conn(0x7f9c740cc5e0 msgr2=0x7f9c740cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:48.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.641+0000 7f9c737fe640 1 --2- 172.21.15.81:0/1058287283 >> v2:172.21.15.93:3300/0 conn(0x7f9c740cc5e0 0x7f9c740cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.641+0000 7f9c737fe640 1 -- 172.21.15.81:0/1058287283 >> v1:172.21.15.93:6789/0 conn(0x7f9c740ccf80 legacy=0x7f9c7415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.641+0000 7f9c737fe640 1 -- 172.21.15.81:0/1058287283 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c7406d350 con 0x7f9c740adcd0 2023-11-08T16:34:48.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.641+0000 7f9c6bfff640 1 --2- 172.21.15.81:0/1058287283 >> v2:172.21.15.93:3300/0 conn(0x7f9c740cc5e0 0x7f9c740cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:34:48.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.642+0000 7f9c737fe640 1 --2- 172.21.15.81:0/1058287283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c740adcd0 0x7f9c740ca1e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400ba40 tx=0x7f9c6402e080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9441a43772044229 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:48.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.643+0000 7f9c72ffd640 1 -- 172.21.15.81:0/1058287283 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6402ee30 con 0x7f9c740adcd0 2023-11-08T16:34:48.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.643+0000 7f9c72ffd640 1 -- 172.21.15.81:0/1058287283 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c64036440 con 0x7f9c740adcd0 2023-11-08T16:34:48.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.643+0000 7f9c72ffd640 1 -- 172.21.15.81:0/1058287283 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c640367d0 con 0x7f9c740adcd0 2023-11-08T16:34:48.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.643+0000 7f9c79d55640 1 -- 172.21.15.81:0/1058287283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c740adcd0 msgr2=0x7f9c740ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.643+0000 7f9c79d55640 1 --2- 172.21.15.81:0/1058287283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c740adcd0 0x7f9c740ca1e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400ba40 tx=0x7f9c6402e080 comp rx=0 tx=0).stop 2023-11-08T16:34:48.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.643+0000 7f9c79d55640 1 -- 172.21.15.81:0/1058287283 shutdown_connections 2023-11-08T16:34:48.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.643+0000 7f9c79d55640 1 --2- 172.21.15.81:0/1058287283 >> v2:172.21.15.93:3300/0 conn(0x7f9c740cc5e0 0x7f9c740cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.643+0000 7f9c79d55640 1 --2- 172.21.15.81:0/1058287283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c740adcd0 0x7f9c740ca1e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.643+0000 7f9c79d55640 1 -- 172.21.15.81:0/1058287283 >> 172.21.15.81:0/1058287283 conn(0x7f9c740cb490 msgr2=0x7f9c740cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:48.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.643+0000 7f9c79d55640 1 -- 172.21.15.81:0/1058287283 shutdown_connections 2023-11-08T16:34:48.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.643+0000 7f9c79d55640 1 -- 172.21.15.81:0/1058287283 wait complete. 2023-11-08T16:34:48.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.645+0000 7f9c79d55640 1 Processor -- start 2023-11-08T16:34:48.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.645+0000 7f9c79d55640 1 -- start start 2023-11-08T16:34:48.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.646+0000 7f9c79d55640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c740adcd0 0x7f9c740c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.646+0000 7f9c79d55640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c740cc5e0 0x7f9c740c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.646+0000 7f9c79d55640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c740ccf80 0x7f9c74159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.646+0000 7f9c79d55640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c741e55e0 con 0x7f9c740adcd0 2023-11-08T16:34:48.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.646+0000 7f9c79d55640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740c87f0 con 0x7f9c740cc5e0 2023-11-08T16:34:48.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.646+0000 7f9c79d55640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740c5ec0 con 0x7f9c740ccf80 2023-11-08T16:34:48.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.646+0000 7f9c737fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c740adcd0 0x7f9c740c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.647+0000 7f9c737fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c740adcd0 0x7f9c740c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47890/0 (socket says 172.21.15.81:47890) 2023-11-08T16:34:48.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.647+0000 7f9c737fe640 1 -- 172.21.15.81:0/1911406679 learned_addr learned my addr 172.21.15.81:0/1911406679 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:48.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.647+0000 7f9c6bfff640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c740cc5e0 0x7f9c740c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.647+0000 7f9c73fff640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c740ccf80 0x7f9c74159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.647+0000 7f9c737fe640 1 -- 172.21.15.81:0/1911406679 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c740ccf80 msgr2=0x7f9c74159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.647+0000 7f9c737fe640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c740ccf80 0x7f9c74159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.647+0000 7f9c737fe640 1 -- 172.21.15.81:0/1911406679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c740cc5e0 msgr2=0x7f9c740c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.647+0000 7f9c737fe640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c740cc5e0 0x7f9c740c82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.647+0000 7f9c737fe640 1 -- 172.21.15.81:0/1911406679 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5c0094d0 con 0x7f9c740adcd0 2023-11-08T16:34:48.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.647+0000 7f9c737fe640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c740adcd0 0x7f9c740c7d70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9c64039040 tx=0x7f9c640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:48.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.648+0000 7f9c70ff9640 1 -- 172.21.15.81:0/1911406679 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6403f8f0 con 0x7f9c740adcd0 2023-11-08T16:34:48.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.648+0000 7f9c70ff9640 1 -- 172.21.15.81:0/1911406679 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6403fa50 con 0x7f9c740adcd0 2023-11-08T16:34:48.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.648+0000 7f9c70ff9640 1 -- 172.21.15.81:0/1911406679 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6403fe00 con 0x7f9c740adcd0 2023-11-08T16:34:48.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.648+0000 7f9c79d55640 1 -- 172.21.15.81:0/1911406679 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6400b6a0 con 0x7f9c740adcd0 2023-11-08T16:34:48.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.648+0000 7f9c79d55640 1 -- 172.21.15.81:0/1911406679 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c740c6480 con 0x7f9c740adcd0 2023-11-08T16:34:48.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.649+0000 7f9c79d55640 1 -- 172.21.15.81:0/1911406679 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c74072170 con 0x7f9c740adcd0 2023-11-08T16:34:48.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.650+0000 7f9c70ff9640 1 -- 172.21.15.81:0/1911406679 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c6403d070 con 0x7f9c740adcd0 2023-11-08T16:34:48.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.651+0000 7f9c70ff9640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c5406aea0 0x7f9c5406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:48.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.651+0000 7f9c70ff9640 1 -- 172.21.15.81:0/1911406679 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x7f9c640ac6e0 con 0x7f9c740adcd0 2023-11-08T16:34:48.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.651+0000 7f9c6bfff640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c5406aea0 0x7f9c5406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:48.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.652+0000 7f9c6bfff640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c5406aea0 0x7f9c5406d360 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9c741f6a30 tx=0x7f9c5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:48.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.655+0000 7f9c70ff9640 1 -- 172.21.15.81:0/1911406679 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c640774f0 con 0x7f9c740adcd0 2023-11-08T16:34:48.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.885+0000 7f9c79d55640 1 -- 172.21.15.81:0/1911406679 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c7404ece0 con 0x7f9c740adcd0 2023-11-08T16:34:48.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.886+0000 7f9c70ff9640 1 -- 172.21.15.81:0/1911406679 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4901 (secure 0 0 0) 0x7f9c6407ab10 con 0x7f9c740adcd0 2023-11-08T16:34:48.888 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:48.888 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3492741644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3492741644},{"type":"v1","addr":"172.21.15.93:6833","nonce":3492741644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":4916,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/119699732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":119699732},{"type":"v1","addr":"172.21.15.93:6835","nonce":119699732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5170,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/1473474760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1473474760},{"type":"v1","addr":"172.21.15.81:6836","nonce":1473474760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/494395478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":494395478},{"type":"v1","addr":"172.21.15.81:6837","nonce":494395478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:46.400575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:34:48.888 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 47 2023-11-08T16:34:48.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.888+0000 7f9c79d55640 1 -- 172.21.15.81:0/1911406679 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c5406aea0 msgr2=0x7f9c5406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.888+0000 7f9c79d55640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c5406aea0 0x7f9c5406d360 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9c741f6a30 tx=0x7f9c5c008040 comp rx=0 tx=0).stop 2023-11-08T16:34:48.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.888+0000 7f9c79d55640 1 -- 172.21.15.81:0/1911406679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c740adcd0 msgr2=0x7f9c740c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:48.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.888+0000 7f9c79d55640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c740adcd0 0x7f9c740c7d70 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9c64039040 tx=0x7f9c640046c0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.889+0000 7f9c79d55640 1 -- 172.21.15.81:0/1911406679 shutdown_connections 2023-11-08T16:34:48.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.889+0000 7f9c79d55640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c5406aea0 0x7f9c5406d360 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.889+0000 7f9c79d55640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c740ccf80 0x7f9c74159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.889+0000 7f9c79d55640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c740cc5e0 0x7f9c740c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.889+0000 7f9c79d55640 1 --2- 172.21.15.81:0/1911406679 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c740adcd0 0x7f9c740c7d70 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:48.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.889+0000 7f9c79d55640 1 -- 172.21.15.81:0/1911406679 >> 172.21.15.81:0/1911406679 conn(0x7f9c740cb490 msgr2=0x7f9c7415ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:48.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.889+0000 7f9c79d55640 1 -- 172.21.15.81:0/1911406679 shutdown_connections 2023-11-08T16:34:48.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:48.889+0000 7f9c79d55640 1 -- 172.21.15.81:0/1911406679 wait complete. 2023-11-08T16:34:48.909 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:34:49.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.116+0000 7fd18fd4c640 1 Processor -- start 2023-11-08T16:34:49.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.116+0000 7fd18fd4c640 1 -- start start 2023-11-08T16:34:49.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18fd4c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd18815d2b0 0x7fd188161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18fd4c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd1880c9060 0x7fd1880cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18fd4c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd1880c9510 0x7fd1880ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18fd4c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd18815d690 con 0x7fd1880c9060 2023-11-08T16:34:49.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18fd4c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd188161c50 con 0x7fd18815d2b0 2023-11-08T16:34:49.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18fd4c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd18806b980 con 0x7fd1880c9510 2023-11-08T16:34:49.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18d2e7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd1880c9060 0x7fd1880cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18d2e7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd1880c9060 0x7fd1880cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47906/0 (socket says 172.21.15.81:47906) 2023-11-08T16:34:49.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18d2e7640 1 -- 172.21.15.81:0/3528753823 learned_addr learned my addr 172.21.15.81:0/3528753823 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18d2e7640 1 -- 172.21.15.81:0/3528753823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd1880c9510 msgr2=0x7fd1880ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18d2e7640 1 --2- 172.21.15.81:0/3528753823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd1880c9510 0x7fd1880ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18d2e7640 1 -- 172.21.15.81:0/3528753823 >> v2:172.21.15.93:3300/0 conn(0x7fd18815d2b0 msgr2=0x7fd188161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.118+0000 7fd18d2e7640 1 --2- 172.21.15.81:0/3528753823 >> v2:172.21.15.93:3300/0 conn(0x7fd18815d2b0 0x7fd188161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.119+0000 7fd18d2e7640 1 -- 172.21.15.81:0/3528753823 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd18806bc60 con 0x7fd1880c9060 2023-11-08T16:34:49.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.120+0000 7fd18d2e7640 1 --2- 172.21.15.81:0/3528753823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1880c9060 0x7fd1880cc7f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd18806b0e0 tx=0x7fd17802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3c71f361c9f11f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.120+0000 7fd18cae6640 1 -- 172.21.15.81:0/3528753823 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178031870 con 0x7fd1880c9060 2023-11-08T16:34:49.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.120+0000 7fd18cae6640 1 -- 172.21.15.81:0/3528753823 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1780319d0 con 0x7fd1880c9060 2023-11-08T16:34:49.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.120+0000 7fd18cae6640 1 -- 172.21.15.81:0/3528753823 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd178031d60 con 0x7fd1880c9060 2023-11-08T16:34:49.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.120+0000 7fd18fd4c640 1 -- 172.21.15.81:0/3528753823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1880c9060 msgr2=0x7fd1880cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.120+0000 7fd18fd4c640 1 --2- 172.21.15.81:0/3528753823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1880c9060 0x7fd1880cc7f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd18806b0e0 tx=0x7fd17802b910 comp rx=0 tx=0).stop 2023-11-08T16:34:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.121+0000 7fd18fd4c640 1 -- 172.21.15.81:0/3528753823 shutdown_connections 2023-11-08T16:34:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.121+0000 7fd18fd4c640 1 --2- 172.21.15.81:0/3528753823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd1880c9510 0x7fd1880ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.121+0000 7fd18fd4c640 1 --2- 172.21.15.81:0/3528753823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1880c9060 0x7fd1880cc7f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.121+0000 7fd18fd4c640 1 --2- 172.21.15.81:0/3528753823 >> v2:172.21.15.93:3300/0 conn(0x7fd18815d2b0 0x7fd188161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.121+0000 7fd18fd4c640 1 -- 172.21.15.81:0/3528753823 >> 172.21.15.81:0/3528753823 conn(0x7fd1880cb490 msgr2=0x7fd1880cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.121+0000 7fd18fd4c640 1 -- 172.21.15.81:0/3528753823 shutdown_connections 2023-11-08T16:34:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.121+0000 7fd18fd4c640 1 -- 172.21.15.81:0/3528753823 wait complete. 2023-11-08T16:34:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.122+0000 7fd18fd4c640 1 Processor -- start 2023-11-08T16:34:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.123+0000 7fd18fd4c640 1 -- start start 2023-11-08T16:34:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.123+0000 7fd18fd4c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1880c9060 0x7fd1880c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.123+0000 7fd18fd4c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1880c9510 0x7fd1880b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.123+0000 7fd18fd4c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd18815d2b0 0x7fd1880b2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.123+0000 7fd18fd4c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880c5c80 con 0x7fd1880c9060 2023-11-08T16:34:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.123+0000 7fd18fd4c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880c5f60 con 0x7fd1880c9510 2023-11-08T16:34:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.123+0000 7fd18fd4c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1880c6240 con 0x7fd18815d2b0 2023-11-08T16:34:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.124+0000 7fd18dae8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1880c9060 0x7fd1880c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.124+0000 7fd18dae8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1880c9060 0x7fd1880c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47908/0 (socket says 172.21.15.81:47908) 2023-11-08T16:34:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.124+0000 7fd18dae8640 1 -- 172.21.15.81:0/2856212685 learned_addr learned my addr 172.21.15.81:0/2856212685 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.124+0000 7fd18dae8640 1 -- 172.21.15.81:0/2856212685 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd18815d2b0 msgr2=0x7fd1880b2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.124+0000 7fd18dae8640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd18815d2b0 0x7fd1880b2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.124+0000 7fd18dae8640 1 -- 172.21.15.81:0/2856212685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1880c9510 msgr2=0x7fd1880b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.124+0000 7fd18dae8640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1880c9510 0x7fd1880b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.124+0000 7fd18dae8640 1 -- 172.21.15.81:0/2856212685 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780094d0 con 0x7fd1880c9060 2023-11-08T16:34:49.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.124+0000 7fd18dae8640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1880c9060 0x7fd1880c6d20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b4a0 tx=0x7fd18400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.125+0000 7fd17e7fc640 1 -- 172.21.15.81:0/2856212685 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184010610 con 0x7fd1880c9060 2023-11-08T16:34:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.125+0000 7fd17e7fc640 1 -- 172.21.15.81:0/2856212685 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184010770 con 0x7fd1880c9060 2023-11-08T16:34:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.125+0000 7fd17e7fc640 1 -- 172.21.15.81:0/2856212685 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184010b20 con 0x7fd1880c9060 2023-11-08T16:34:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.125+0000 7fd18fd4c640 1 -- 172.21.15.81:0/2856212685 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1880c6520 con 0x7fd1880c9060 2023-11-08T16:34:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.125+0000 7fd18fd4c640 1 -- 172.21.15.81:0/2856212685 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd188159ed0 con 0x7fd1880c9060 2023-11-08T16:34:49.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.126+0000 7fd18fd4c640 1 -- 172.21.15.81:0/2856212685 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1880c6a40 con 0x7fd1880c9060 2023-11-08T16:34:49.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.127+0000 7fd17e7fc640 1 -- 172.21.15.81:0/2856212685 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd184010c80 con 0x7fd1880c9060 2023-11-08T16:34:49.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.128+0000 7fd17e7fc640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd16c06ae70 0x7fd16c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.128+0000 7fd17e7fc640 1 -- 172.21.15.81:0/2856212685 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x7fd184089570 con 0x7fd1880c9060 2023-11-08T16:34:49.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.128+0000 7fd18d2e7640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd16c06ae70 0x7fd16c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.129+0000 7fd18d2e7640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd16c06ae70 0x7fd16c06d330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd178036000 tx=0x7fd178030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.133+0000 7fd17e7fc640 1 -- 172.21.15.81:0/2856212685 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd1880c6a40 con 0x7fd1880c9060 2023-11-08T16:34:49.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.361+0000 7fd18fd4c640 1 -- 172.21.15.81:0/2856212685 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1880c5a40 con 0x7fd1880c9060 2023-11-08T16:34:49.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.363+0000 7fd17e7fc640 1 -- 172.21.15.81:0/2856212685 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4901 (secure 0 0 0) 0x7fd1880c5a40 con 0x7fd1880c9060 2023-11-08T16:34:49.365 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:49.365 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":47,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3492741644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3492741644},{"type":"v1","addr":"172.21.15.93:6833","nonce":3492741644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":4916,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/119699732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":119699732},{"type":"v1","addr":"172.21.15.93:6835","nonce":119699732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":5170,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/1473474760","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1473474760},{"type":"v1","addr":"172.21.15.81:6836","nonce":1473474760}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6288,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/494395478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":494395478},{"type":"v1","addr":"172.21.15.81:6837","nonce":494395478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:46.400575+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":64,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:34:49.365 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 47 2023-11-08T16:34:49.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.365+0000 7fd18fd4c640 1 -- 172.21.15.81:0/2856212685 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd16c06ae70 msgr2=0x7fd16c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.365+0000 7fd18fd4c640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd16c06ae70 0x7fd16c06d330 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd178036000 tx=0x7fd178030000 comp rx=0 tx=0).stop 2023-11-08T16:34:49.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.365+0000 7fd18fd4c640 1 -- 172.21.15.81:0/2856212685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1880c9060 msgr2=0x7fd1880c6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.365+0000 7fd18fd4c640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1880c9060 0x7fd1880c6d20 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd18400b4a0 tx=0x7fd18400b960 comp rx=0 tx=0).stop 2023-11-08T16:34:49.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.365+0000 7fd18fd4c640 1 -- 172.21.15.81:0/2856212685 shutdown_connections 2023-11-08T16:34:49.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.365+0000 7fd18fd4c640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd16c06ae70 0x7fd16c06d330 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.365+0000 7fd18fd4c640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd18815d2b0 0x7fd1880b2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.365+0000 7fd18fd4c640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1880c9510 0x7fd1880b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.365+0000 7fd18fd4c640 1 --2- 172.21.15.81:0/2856212685 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd1880c9060 0x7fd1880c6d20 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.365+0000 7fd18fd4c640 1 -- 172.21.15.81:0/2856212685 >> 172.21.15.81:0/2856212685 conn(0x7fd1880cb490 msgr2=0x7fd1880c9b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.366+0000 7fd18fd4c640 1 -- 172.21.15.81:0/2856212685 shutdown_connections 2023-11-08T16:34:49.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.366+0000 7fd18fd4c640 1 -- 172.21.15.81:0/2856212685 wait complete. 2023-11-08T16:34:49.387 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-11-08T16:34:49.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.475+0000 7fe207839100 1 -- start start 2023-11-08T16:34:49.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.475+0000 7fe207839100 1 Processor -- start 2023-11-08T16:34:49.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.476+0000 7fe207839100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66f12ac00 0x55c66f196580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.476+0000 7fe207839100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66f12b800 0x55c66f196b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.476+0000 7fe207839100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66f12bc00 0x55c66f197080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.476+0000 7fe207839100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66f175180 con 0x55c66f12b800 2023-11-08T16:34:49.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.476+0000 7fe207839100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66f174140 con 0x55c66f12bc00 2023-11-08T16:34:49.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.476+0000 7fe207839100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c66f174000 con 0x55c66f12ac00 2023-11-08T16:34:49.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.476+0000 7fe204f33640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66f12b800 0x55c66f196b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.477+0000 7fe204f33640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66f12b800 0x55c66f196b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47924/0 (socket says 172.21.15.81:47924) 2023-11-08T16:34:49.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.477+0000 7fe204f33640 1 -- 172.21.15.81:0/3839024928 learned_addr learned my addr 172.21.15.81:0/3839024928 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.477+0000 7fe205f35640 1 --2- 172.21.15.81:0/3839024928 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66f12bc00 0x55c66f197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.477+0000 7fe205734640 1 --2- 172.21.15.81:0/3839024928 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66f12ac00 0x55c66f196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.477+0000 7fe204f33640 1 -- 172.21.15.81:0/3839024928 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66f12ac00 msgr2=0x55c66f196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.477+0000 7fe204f33640 1 --2- 172.21.15.81:0/3839024928 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66f12ac00 0x55c66f196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.477+0000 7fe204f33640 1 -- 172.21.15.81:0/3839024928 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66f12bc00 msgr2=0x55c66f197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.477+0000 7fe204f33640 1 --2- 172.21.15.81:0/3839024928 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66f12bc00 0x55c66f197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.477+0000 7fe204f33640 1 -- 172.21.15.81:0/3839024928 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c66f1509c0 con 0x55c66f12b800 2023-11-08T16:34:49.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.477+0000 7fe205734640 1 --2- 172.21.15.81:0/3839024928 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66f12ac00 0x55c66f196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:49.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.477+0000 7fe204f33640 1 --2- 172.21.15.81:0/3839024928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66f12b800 0x55c66f196b00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x55c66ff50c00 tx=0x55c66fe8c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6ab2c12654b5c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.478+0000 7fe202f2f640 1 -- 172.21.15.81:0/3839024928 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c66f1731e0 con 0x55c66f12b800 2023-11-08T16:34:49.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.478+0000 7fe202f2f640 1 -- 172.21.15.81:0/3839024928 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c66f186480 con 0x55c66f12b800 2023-11-08T16:34:49.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.478+0000 7fe202f2f640 1 -- 172.21.15.81:0/3839024928 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c66f173080 con 0x55c66f12b800 2023-11-08T16:34:49.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.478+0000 7fe207839100 1 -- 172.21.15.81:0/3839024928 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c66f150b60 con 0x55c66f12b800 2023-11-08T16:34:49.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.478+0000 7fe207839100 1 -- 172.21.15.81:0/3839024928 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c66f150820 con 0x55c66f12b800 2023-11-08T16:34:49.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.478+0000 7fe207839100 4 waiting for MDS map... 2023-11-08T16:34:49.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.478+0000 7fe202f2f640 1 -- 172.21.15.81:0/3839024928 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55c66f156380 con 0x55c66f12b800 2023-11-08T16:34:49.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.478+0000 7fe202f2f640 1 -- 172.21.15.81:0/3839024928 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55c66f17cf00 con 0x55c66f12b800 2023-11-08T16:34:49.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.478+0000 7fe207839100 4 Got MDS map 47 2023-11-08T16:34:49.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.478+0000 7fe207839100 10 main: main 2023-11-08T16:34:49.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.479+0000 7fe207839100 4 main: connecting to RADOS... 2023-11-08T16:34:49.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.479+0000 7fe207839100 1 Processor -- start 2023-11-08T16:34:49.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.479+0000 7fe207839100 1 -- start start 2023-11-08T16:34:49.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.480+0000 7fe207839100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66ffe3000 0x55c66f197600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.480+0000 7fe207839100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66ffe3400 0x55c66f197b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.480+0000 7fe207839100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 0x55c66f198100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.480+0000 7fe207839100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66f174a00 con 0x55c66ffe3800 2023-11-08T16:34:49.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.480+0000 7fe207839100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66f1f9180 con 0x55c66ffe3400 2023-11-08T16:34:49.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.480+0000 7fe207839100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c66f1f8dc0 con 0x55c66ffe3000 2023-11-08T16:34:49.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe204f33640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 0x55c66f198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe204f33640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 0x55c66f198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47940/0 (socket says 172.21.15.81:47940) 2023-11-08T16:34:49.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe204f33640 1 -- 172.21.15.81:0/745105922 learned_addr learned my addr 172.21.15.81:0/745105922 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe205734640 1 --2- 172.21.15.81:0/745105922 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66ffe3400 0x55c66f197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe205f35640 1 --2- 172.21.15.81:0/745105922 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66ffe3000 0x55c66f197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe204f33640 1 -- 172.21.15.81:0/745105922 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66ffe3000 msgr2=0x55c66f197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe204f33640 1 --2- 172.21.15.81:0/745105922 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66ffe3000 0x55c66f197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe204f33640 1 -- 172.21.15.81:0/745105922 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66ffe3400 msgr2=0x55c66f197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe204f33640 1 --2- 172.21.15.81:0/745105922 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66ffe3400 0x55c66f197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe204f33640 1 -- 172.21.15.81:0/745105922 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c66f1504e0 con 0x55c66ffe3800 2023-11-08T16:34:49.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe204f33640 1 --2- 172.21.15.81:0/745105922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 0x55c66f198100 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55c67004a0f0 tx=0x55c66fe8c060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fdd2810c170a707 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe200f2b640 1 -- 172.21.15.81:0/745105922 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c66f172dc0 con 0x55c66ffe3800 2023-11-08T16:34:49.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe200f2b640 1 -- 172.21.15.81:0/745105922 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c66ff0ad80 con 0x55c66ffe3800 2023-11-08T16:34:49.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.481+0000 7fe200f2b640 1 -- 172.21.15.81:0/745105922 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c66f1731e0 con 0x55c66ffe3800 2023-11-08T16:34:49.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.482+0000 7fe207839100 1 -- 172.21.15.81:0/745105922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 msgr2=0x55c66f198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.482+0000 7fe207839100 1 --2- 172.21.15.81:0/745105922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 0x55c66f198100 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x55c67004a0f0 tx=0x55c66fe8c060 comp rx=0 tx=0).stop 2023-11-08T16:34:49.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.482+0000 7fe207839100 1 -- 172.21.15.81:0/745105922 shutdown_connections 2023-11-08T16:34:49.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.482+0000 7fe207839100 1 --2- 172.21.15.81:0/745105922 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 0x55c66f198100 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.482+0000 7fe207839100 1 --2- 172.21.15.81:0/745105922 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66ffe3400 0x55c66f197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.482+0000 7fe207839100 1 --2- 172.21.15.81:0/745105922 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66ffe3000 0x55c66f197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.482+0000 7fe207839100 1 -- 172.21.15.81:0/745105922 >> 172.21.15.81:0/745105922 conn(0x55c66ffe2c00 msgr2=0x55c66feb5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.482+0000 7fe207839100 1 -- 172.21.15.81:0/745105922 shutdown_connections 2023-11-08T16:34:49.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.482+0000 7fe207839100 1 -- 172.21.15.81:0/745105922 wait complete. 2023-11-08T16:34:49.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.483+0000 7fe207839100 1 Processor -- start 2023-11-08T16:34:49.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.483+0000 7fe207839100 1 -- start start 2023-11-08T16:34:49.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe207839100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66ffe3000 0x55c66f199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe207839100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66ffe3400 0x55c66f198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe207839100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 0x55c66f198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe207839100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66f1f8c80 con 0x55c66ffe3800 2023-11-08T16:34:49.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe207839100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c66f174140 con 0x55c66ffe3000 2023-11-08T16:34:49.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe207839100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c66f174b40 con 0x55c66ffe3400 2023-11-08T16:34:49.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe204f33640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 0x55c66f198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe204f33640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 0x55c66f198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47942/0 (socket says 172.21.15.81:47942) 2023-11-08T16:34:49.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe204f33640 1 -- 172.21.15.81:0/4285134691 learned_addr learned my addr 172.21.15.81:0/4285134691 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe204f33640 1 -- 172.21.15.81:0/4285134691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66ffe3400 msgr2=0x55c66f198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe204f33640 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66ffe3400 0x55c66f198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe204f33640 1 -- 172.21.15.81:0/4285134691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66ffe3000 msgr2=0x55c66f199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe204f33640 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66ffe3000 0x55c66f199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.484+0000 7fe204f33640 1 -- 172.21.15.81:0/4285134691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c66f1509c0 con 0x55c66ffe3800 2023-11-08T16:34:49.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.485+0000 7fe204f33640 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 0x55c66f198680 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x55c67004adb0 tx=0x55c66fff8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.485+0000 7fe1ff728640 1 -- 172.21.15.81:0/4285134691 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c66f173080 con 0x55c66ffe3800 2023-11-08T16:34:49.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.485+0000 7fe1ff728640 1 -- 172.21.15.81:0/4285134691 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c66ff0ac00 con 0x55c66ffe3800 2023-11-08T16:34:49.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.485+0000 7fe207839100 1 -- 172.21.15.81:0/4285134691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c66f150b60 con 0x55c66ffe3800 2023-11-08T16:34:49.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.485+0000 7fe1ff728640 1 -- 172.21.15.81:0/4285134691 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c66f172420 con 0x55c66ffe3800 2023-11-08T16:34:49.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.485+0000 7fe207839100 1 -- 172.21.15.81:0/4285134691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c66f1501a0 con 0x55c66ffe3800 2023-11-08T16:34:49.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.485+0000 7fe207839100 4 main: resolving pool 7 2023-11-08T16:34:49.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.487+0000 7fe1ff728640 1 -- 172.21.15.81:0/4285134691 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55c66f188840 con 0x55c66ffe3800 2023-11-08T16:34:49.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.487+0000 7fe1ff728640 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55c66f47e000 0x55c66f199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.487+0000 7fe1ff728640 1 -- 172.21.15.81:0/4285134691 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55c66f156540 con 0x55c66ffe3800 2023-11-08T16:34:49.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.487+0000 7fe207839100 4 main: creating IoCtx.. 2023-11-08T16:34:49.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.487+0000 7fe207839100 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55c66f47e400 0x55c66f198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.487+0000 7fe205f35640 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55c66f47e000 0x55c66f199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.487+0000 7fe207839100 1 -- 172.21.15.81:0/4285134691 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:1 7.b 7:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55c66f185880 con 0x55c66f47e400 2023-11-08T16:34:49.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.488+0000 7fe205734640 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55c66f47e400 0x55c66f198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.488+0000 7fe205f35640 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55c66f47e000 0x55c66f199700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55c66f1bb830 tx=0x55c66fe8c240 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.488+0000 7fe205734640 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55c66f47e400 0x55c66f198100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe205734640 1 -- 172.21.15.81:0/4285134691 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v64'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55c66fe86b40 con 0x55c66f47e400 2023-11-08T16:34:49.503 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:34:49.504 INFO:teuthology.orchestra.run.smithi081.stdout: "0": { 2023-11-08T16:34:49.504 INFO:teuthology.orchestra.run.smithi081.stdout: "data": {}, 2023-11-08T16:34:49.504 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:34:49.504 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:34:49.504 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:34:49.505 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:49.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 -- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55c66f47e400 msgr2=0x55c66f198100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55c66f47e400 0x55c66f198100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 -- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55c66f47e000 msgr2=0x55c66f199700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55c66f47e000 0x55c66f199700 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55c66f1bb830 tx=0x55c66fe8c240 comp rx=0 tx=0).stop 2023-11-08T16:34:49.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 -- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 msgr2=0x55c66f198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 --2- 172.21.15.81:0/4285134691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66ffe3800 0x55c66f198680 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x55c67004adb0 tx=0x55c66fff8c00 comp rx=0 tx=0).stop 2023-11-08T16:34:49.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe205734640 1 -- 172.21.15.81:0/4285134691 reap_dead start 2023-11-08T16:34:49.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 -- 172.21.15.81:0/4285134691 shutdown_connections 2023-11-08T16:34:49.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 -- 172.21.15.81:0/4285134691 >> 172.21.15.81:0/4285134691 conn(0x55c66ffe2c00 msgr2=0x55c66feb5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 -- 172.21.15.81:0/4285134691 shutdown_connections 2023-11-08T16:34:49.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 -- 172.21.15.81:0/4285134691 wait complete. 2023-11-08T16:34:49.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 -- 172.21.15.81:0/3839024928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66f12b800 msgr2=0x55c66f196b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.492+0000 7fe207839100 1 --2- 172.21.15.81:0/3839024928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66f12b800 0x55c66f196b00 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x55c66ff50c00 tx=0x55c66fe8c420 comp rx=0 tx=0).stop 2023-11-08T16:34:49.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.493+0000 7fe207839100 1 -- 172.21.15.81:0/3839024928 shutdown_connections 2023-11-08T16:34:49.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.493+0000 7fe207839100 1 --2- 172.21.15.81:0/3839024928 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c66f12bc00 0x55c66f197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.493+0000 7fe207839100 1 --2- 172.21.15.81:0/3839024928 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c66f12b800 0x55c66f196b00 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.493+0000 7fe207839100 1 --2- 172.21.15.81:0/3839024928 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c66f12ac00 0x55c66f196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.493+0000 7fe207839100 1 -- 172.21.15.81:0/3839024928 >> 172.21.15.81:0/3839024928 conn(0x55c66f12b000 msgr2=0x55c66feb4800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.493+0000 7fe207839100 1 -- 172.21.15.81:0/3839024928 shutdown_connections 2023-11-08T16:34:49.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.493+0000 7fe207839100 1 -- 172.21.15.81:0/3839024928 wait complete. 2023-11-08T16:34:49.511 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.123878, result: { "0": { "data": {}, "result": 0 } } 2023-11-08T16:34:49.511 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-11-08T16:34:49.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.610+0000 7f892804c100 1 -- start start 2023-11-08T16:34:49.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.610+0000 7f892804c100 1 Processor -- start 2023-11-08T16:34:49.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.611+0000 7f892804c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c4926e800 0x563c48503b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.611+0000 7f892804c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c4926fc00 0x563c48503080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.611+0000 7f892804c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563c4926f400 0x563c48502b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.611+0000 7f892804c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c484e1180 con 0x563c4926e800 2023-11-08T16:34:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.611+0000 7f892804c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c484e0b40 con 0x563c4926fc00 2023-11-08T16:34:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.611+0000 7f892804c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c484e0000 con 0x563c4926f400 2023-11-08T16:34:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.611+0000 7f8926055640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c4926e800 0x563c48503b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.611+0000 7f8926055640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c4926e800 0x563c48503b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47970/0 (socket says 172.21.15.81:47970) 2023-11-08T16:34:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.611+0000 7f8926055640 1 -- 172.21.15.81:0/2351403427 learned_addr learned my addr 172.21.15.81:0/2351403427 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f8926055640 1 -- 172.21.15.81:0/2351403427 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563c4926f400 msgr2=0x563c48502b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f8926055640 1 --2- 172.21.15.81:0/2351403427 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563c4926f400 0x563c48502b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f8926055640 1 -- 172.21.15.81:0/2351403427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c4926fc00 msgr2=0x563c48503080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f8926055640 1 --2- 172.21.15.81:0/2351403427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c4926fc00 0x563c48503080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f8926055640 1 -- 172.21.15.81:0/2351403427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c484bc4e0 con 0x563c4926e800 2023-11-08T16:34:49.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f8926055640 1 --2- 172.21.15.81:0/2351403427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c4926e800 0x563c48503b80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x563c492b0b70 tx=0x563c492b6d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988acfa1a06252d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f8923850640 1 -- 172.21.15.81:0/2351403427 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c484def20 con 0x563c4926e800 2023-11-08T16:34:49.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f8923850640 1 -- 172.21.15.81:0/2351403427 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563c4935e000 con 0x563c4926e800 2023-11-08T16:34:49.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f8923850640 1 -- 172.21.15.81:0/2351403427 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c484df1e0 con 0x563c4926e800 2023-11-08T16:34:49.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f892804c100 1 -- 172.21.15.81:0/2351403427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c484bc1a0 con 0x563c4926e800 2023-11-08T16:34:49.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f892804c100 1 -- 172.21.15.81:0/2351403427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563c49360000 con 0x563c4926e800 2023-11-08T16:34:49.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f892804c100 4 waiting for MDS map... 2023-11-08T16:34:49.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.612+0000 7f8923850640 1 -- 172.21.15.81:0/2351403427 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x563c484c2380 con 0x563c4926e800 2023-11-08T16:34:49.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.613+0000 7f8923850640 1 -- 172.21.15.81:0/2351403427 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x563c484e8f00 con 0x563c4926e800 2023-11-08T16:34:49.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.613+0000 7f892804c100 4 Got MDS map 47 2023-11-08T16:34:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.613+0000 7f892804c100 10 main: main 2023-11-08T16:34:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.613+0000 7f892804c100 4 main: connecting to RADOS... 2023-11-08T16:34:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.614+0000 7f892804c100 1 Processor -- start 2023-11-08T16:34:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.614+0000 7f892804c100 1 -- start start 2023-11-08T16:34:49.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.614+0000 7f892804c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c49358c00 0x563c48504100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.614+0000 7f892804c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563c49359000 0x563c48504680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.614+0000 7f892804c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 0x563c48504c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.614+0000 7f892804c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c484e0a00 con 0x563c49359400 2023-11-08T16:34:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.615+0000 7f892804c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c484e0140 con 0x563c49358c00 2023-11-08T16:34:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.615+0000 7f892804c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c48564dc0 con 0x563c49359000 2023-11-08T16:34:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.615+0000 7f8925854640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 0x563c48504c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.615+0000 7f8925854640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 0x563c48504c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47976/0 (socket says 172.21.15.81:47976) 2023-11-08T16:34:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.615+0000 7f8925854640 1 -- 172.21.15.81:0/2247167701 learned_addr learned my addr 172.21.15.81:0/2247167701 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.615+0000 7f8925854640 1 -- 172.21.15.81:0/2247167701 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563c49359000 msgr2=0x563c48504680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.615+0000 7f8925854640 1 --2- 172.21.15.81:0/2247167701 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563c49359000 0x563c48504680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.615+0000 7f8925854640 1 -- 172.21.15.81:0/2247167701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c49358c00 msgr2=0x563c48504100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.615+0000 7f8925854640 1 --2- 172.21.15.81:0/2247167701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c49358c00 0x563c48504100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.615+0000 7f8925854640 1 -- 172.21.15.81:0/2247167701 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c493601a0 con 0x563c49359400 2023-11-08T16:34:49.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.615+0000 7f8925854640 1 --2- 172.21.15.81:0/2247167701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 0x563c48504c00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x563c493c60c0 tx=0x563c49378c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbdc4515050884f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892184c640 1 -- 172.21.15.81:0/2247167701 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c484def20 con 0x563c49359400 2023-11-08T16:34:49.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892184c640 1 -- 172.21.15.81:0/2247167701 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563c4935ec00 con 0x563c49359400 2023-11-08T16:34:49.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892184c640 1 -- 172.21.15.81:0/2247167701 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c484df1e0 con 0x563c49359400 2023-11-08T16:34:49.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892804c100 1 -- 172.21.15.81:0/2247167701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 msgr2=0x563c48504c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892804c100 1 --2- 172.21.15.81:0/2247167701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 0x563c48504c00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x563c493c60c0 tx=0x563c49378c00 comp rx=0 tx=0).stop 2023-11-08T16:34:49.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892804c100 1 -- 172.21.15.81:0/2247167701 shutdown_connections 2023-11-08T16:34:49.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892804c100 1 --2- 172.21.15.81:0/2247167701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 0x563c48504c00 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892804c100 1 --2- 172.21.15.81:0/2247167701 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563c49359000 0x563c48504680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892804c100 1 --2- 172.21.15.81:0/2247167701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c49358c00 0x563c48504100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892804c100 1 -- 172.21.15.81:0/2247167701 >> 172.21.15.81:0/2247167701 conn(0x563c4926f800 msgr2=0x563c48575800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892804c100 1 -- 172.21.15.81:0/2247167701 shutdown_connections 2023-11-08T16:34:49.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.616+0000 7f892804c100 1 -- 172.21.15.81:0/2247167701 wait complete. 2023-11-08T16:34:49.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.617+0000 7f892804c100 1 Processor -- start 2023-11-08T16:34:49.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.617+0000 7f892804c100 1 -- start start 2023-11-08T16:34:49.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f892804c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c49358c00 0x563c48504100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f892804c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563c49359000 0x563c48504680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f892804c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 0x563c48504c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f892804c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c48565180 con 0x563c49359400 2023-11-08T16:34:49.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f892804c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563c48564c80 con 0x563c49358c00 2023-11-08T16:34:49.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f892804c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x563c484e0b40 con 0x563c49359000 2023-11-08T16:34:49.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f8925854640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 0x563c48504c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f8925854640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 0x563c48504c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47990/0 (socket says 172.21.15.81:47990) 2023-11-08T16:34:49.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f8925854640 1 -- 172.21.15.81:0/4290867807 learned_addr learned my addr 172.21.15.81:0/4290867807 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f8925854640 1 -- 172.21.15.81:0/4290867807 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563c49359000 msgr2=0x563c48504680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f8925854640 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563c49359000 0x563c48504680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f8925854640 1 -- 172.21.15.81:0/4290867807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c49358c00 msgr2=0x563c48504100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f8925854640 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c49358c00 0x563c48504100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.618+0000 7f8925854640 1 -- 172.21.15.81:0/4290867807 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563c49360340 con 0x563c49359400 2023-11-08T16:34:49.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.619+0000 7f8925854640 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 0x563c48504c00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x563c493c6d80 tx=0x563c49219f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.619+0000 7f8920049640 1 -- 172.21.15.81:0/4290867807 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c484df080 con 0x563c49359400 2023-11-08T16:34:49.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.619+0000 7f8920049640 1 -- 172.21.15.81:0/4290867807 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563c4935ea80 con 0x563c49359400 2023-11-08T16:34:49.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.619+0000 7f8920049640 1 -- 172.21.15.81:0/4290867807 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563c484def20 con 0x563c49359400 2023-11-08T16:34:49.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.619+0000 7f892804c100 1 -- 172.21.15.81:0/4290867807 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563c493604e0 con 0x563c49359400 2023-11-08T16:34:49.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.619+0000 7f892804c100 1 -- 172.21.15.81:0/4290867807 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563c493601a0 con 0x563c49359400 2023-11-08T16:34:49.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.619+0000 7f892804c100 4 main: resolving pool 7 2023-11-08T16:34:49.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.621+0000 7f8920049640 1 -- 172.21.15.81:0/4290867807 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x563c484f4b00 con 0x563c49359400 2023-11-08T16:34:49.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.621+0000 7f8920049640 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x563c493ccc00 0x563c48505700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.621+0000 7f8920049640 1 -- 172.21.15.81:0/4290867807 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x563c484c2540 con 0x563c49359400 2023-11-08T16:34:49.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.621+0000 7f892804c100 4 main: creating IoCtx.. 2023-11-08T16:34:49.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.622+0000 7f892804c100 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x563c49359800 0x563c48505180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.622+0000 7f892804c100 1 -- 172.21.15.81:0/4290867807 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:1 7.d 7:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x563c484f1880 con 0x563c49359800 2023-11-08T16:34:49.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.622+0000 7f8926856640 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x563c493ccc00 0x563c48505700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.622+0000 7f8926856640 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x563c493ccc00 0x563c48505700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x563c48526de0 tx=0x563c492b60c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.622+0000 7f8926055640 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x563c49359800 0x563c48505180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.623+0000 7f8926055640 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x563c49359800 0x563c48505180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.637 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:34:49.637 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:34:49.638 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:34:49.638 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.626+0000 7f8926055640 1 -- 172.21.15.81:0/4290867807 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v64'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x563c4934cb40 con 0x563c49359800 2023-11-08T16:34:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.626+0000 7f892804c100 1 -- 172.21.15.81:0/4290867807 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x563c49359800 msgr2=0x563c48505180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.626+0000 7f892804c100 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x563c49359800 0x563c48505180 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.626+0000 7f892804c100 1 -- 172.21.15.81:0/4290867807 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x563c493ccc00 msgr2=0x563c48505700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.626+0000 7f892804c100 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x563c493ccc00 0x563c48505700 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x563c48526de0 tx=0x563c492b60c0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.626+0000 7f892804c100 1 -- 172.21.15.81:0/4290867807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 msgr2=0x563c48504c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.626+0000 7f892804c100 1 --2- 172.21.15.81:0/4290867807 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c49359400 0x563c48504c00 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x563c493c6d80 tx=0x563c49219f80 comp rx=0 tx=0).stop 2023-11-08T16:34:49.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.626+0000 7f8925854640 1 -- 172.21.15.81:0/4290867807 reap_dead start 2023-11-08T16:34:49.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.626+0000 7f892804c100 1 -- 172.21.15.81:0/4290867807 shutdown_connections 2023-11-08T16:34:49.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.626+0000 7f892804c100 1 -- 172.21.15.81:0/4290867807 >> 172.21.15.81:0/4290867807 conn(0x563c4926f800 msgr2=0x563c48575800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.626+0000 7f892804c100 1 -- 172.21.15.81:0/4290867807 shutdown_connections 2023-11-08T16:34:49.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.627+0000 7f892804c100 1 -- 172.21.15.81:0/4290867807 wait complete. 2023-11-08T16:34:49.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.627+0000 7f892804c100 1 -- 172.21.15.81:0/2351403427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c4926e800 msgr2=0x563c48503b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.627+0000 7f892804c100 1 --2- 172.21.15.81:0/2351403427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c4926e800 0x563c48503b80 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x563c492b0b70 tx=0x563c492b6d20 comp rx=0 tx=0).stop 2023-11-08T16:34:49.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.627+0000 7f892804c100 1 -- 172.21.15.81:0/2351403427 shutdown_connections 2023-11-08T16:34:49.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.627+0000 7f892804c100 1 --2- 172.21.15.81:0/2351403427 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563c4926f400 0x563c48502b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.627+0000 7f892804c100 1 --2- 172.21.15.81:0/2351403427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563c4926fc00 0x563c48503080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.627+0000 7f892804c100 1 --2- 172.21.15.81:0/2351403427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563c4926e800 0x563c48503b80 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.627+0000 7f892804c100 1 -- 172.21.15.81:0/2351403427 >> 172.21.15.81:0/2351403427 conn(0x563c48497000 msgr2=0x563c48575000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.627+0000 7f892804c100 1 -- 172.21.15.81:0/2351403427 shutdown_connections 2023-11-08T16:34:49.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.627+0000 7f892804c100 1 -- 172.21.15.81:0/2351403427 wait complete. 2023-11-08T16:34:49.644 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.133298, result: { "result": 0 } 2023-11-08T16:34:49.645 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-11-08T16:34:49.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.747+0000 7fcd83638100 1 -- start start 2023-11-08T16:34:49.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.747+0000 7fcd83638100 1 Processor -- start 2023-11-08T16:34:49.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.748+0000 7fcd83638100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58c28cc00 0x55b58c2f8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.748+0000 7fcd83638100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58c28d800 0x55b58c2f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.748+0000 7fcd83638100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58c28dc00 0x55b58c2f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.748+0000 7fcd83638100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58c2d7180 con 0x55b58c28d800 2023-11-08T16:34:49.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.748+0000 7fcd83638100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58c2d6000 con 0x55b58c28dc00 2023-11-08T16:34:49.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.748+0000 7fcd83638100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b58c2d6140 con 0x55b58c28cc00 2023-11-08T16:34:49.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.749+0000 7fcd80e67640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58c28d800 0x55b58c2f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.749+0000 7fcd80e67640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58c28d800 0x55b58c2f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47994/0 (socket says 172.21.15.81:47994) 2023-11-08T16:34:49.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.749+0000 7fcd80e67640 1 -- 172.21.15.81:0/829119336 learned_addr learned my addr 172.21.15.81:0/829119336 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.749+0000 7fcd81e69640 1 --2- 172.21.15.81:0/829119336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58c28dc00 0x55b58c2f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.749+0000 7fcd81668640 1 --2- 172.21.15.81:0/829119336 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58c28cc00 0x55b58c2f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.749+0000 7fcd80e67640 1 -- 172.21.15.81:0/829119336 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58c28cc00 msgr2=0x55b58c2f8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.749+0000 7fcd80e67640 1 --2- 172.21.15.81:0/829119336 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58c28cc00 0x55b58c2f8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.749+0000 7fcd80e67640 1 -- 172.21.15.81:0/829119336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58c28dc00 msgr2=0x55b58c2f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.749+0000 7fcd80e67640 1 --2- 172.21.15.81:0/829119336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58c28dc00 0x55b58c2f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.750+0000 7fcd80e67640 1 -- 172.21.15.81:0/829119336 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b58c2b29c0 con 0x55b58c28d800 2023-11-08T16:34:49.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.750+0000 7fcd81e69640 1 --2- 172.21.15.81:0/829119336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58c28dc00 0x55b58c2f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:34:49.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.750+0000 7fcd80e67640 1 --2- 172.21.15.81:0/829119336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58c28d800 0x55b58c2f8b00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x55b58d0aeba0 tx=0x55b58d136120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb834fba45dac9a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.750+0000 7fcd7ee63640 1 -- 172.21.15.81:0/829119336 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b58c2d4f20 con 0x55b58c28d800 2023-11-08T16:34:49.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.750+0000 7fcd7ee63640 1 -- 172.21.15.81:0/829119336 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b58d070300 con 0x55b58c28d800 2023-11-08T16:34:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.750+0000 7fcd7ee63640 1 -- 172.21.15.81:0/829119336 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b58c2d51e0 con 0x55b58c28d800 2023-11-08T16:34:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.750+0000 7fcd83638100 1 -- 172.21.15.81:0/829119336 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b58c2b2d00 con 0x55b58c28d800 2023-11-08T16:34:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.750+0000 7fcd83638100 1 -- 172.21.15.81:0/829119336 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b58c2b29c0 con 0x55b58c28d800 2023-11-08T16:34:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.750+0000 7fcd83638100 4 waiting for MDS map... 2023-11-08T16:34:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.750+0000 7fcd7ee63640 1 -- 172.21.15.81:0/829119336 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55b58c2b8380 con 0x55b58c28d800 2023-11-08T16:34:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.751+0000 7fcd7ee63640 1 -- 172.21.15.81:0/829119336 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55b58c2def00 con 0x55b58c28d800 2023-11-08T16:34:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.751+0000 7fcd83638100 4 Got MDS map 47 2023-11-08T16:34:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.751+0000 7fcd83638100 10 main: main 2023-11-08T16:34:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.751+0000 7fcd83638100 4 main: connecting to RADOS... 2023-11-08T16:34:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.752+0000 7fcd83638100 1 Processor -- start 2023-11-08T16:34:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.752+0000 7fcd83638100 1 -- start start 2023-11-08T16:34:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd83638100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d06bc00 0x55b58c2f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd83638100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58d1a2000 0x55b58c2f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd83638100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58d1a2400 0x55b58c2fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd83638100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58c2d6a00 con 0x55b58d06bc00 2023-11-08T16:34:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd83638100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58c35b180 con 0x55b58d1a2000 2023-11-08T16:34:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd83638100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b58c35adc0 con 0x55b58d1a2400 2023-11-08T16:34:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd81e69640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d06bc00 0x55b58c2f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd81e69640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d06bc00 0x55b58c2f9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48010/0 (socket says 172.21.15.81:48010) 2023-11-08T16:34:49.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd81e69640 1 -- 172.21.15.81:0/3087947275 learned_addr learned my addr 172.21.15.81:0/3087947275 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd81668640 1 --2- 172.21.15.81:0/3087947275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58d1a2000 0x55b58c2f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd80e67640 1 --2- 172.21.15.81:0/3087947275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58d1a2400 0x55b58c2fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd81e69640 1 -- 172.21.15.81:0/3087947275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58d1a2400 msgr2=0x55b58c2fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd81e69640 1 --2- 172.21.15.81:0/3087947275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58d1a2400 0x55b58c2fa100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd81e69640 1 -- 172.21.15.81:0/3087947275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58d1a2000 msgr2=0x55b58c2f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd81e69640 1 --2- 172.21.15.81:0/3087947275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58d1a2000 0x55b58c2f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd81e69640 1 -- 172.21.15.81:0/3087947275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b58c2b2b60 con 0x55b58d06bc00 2023-11-08T16:34:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.753+0000 7fcd81668640 1 --2- 172.21.15.81:0/3087947275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58d1a2000 0x55b58c2f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.754+0000 7fcd81e69640 1 --2- 172.21.15.81:0/3087947275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d06bc00 0x55b58c2f9600 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55b58d1ac0c0 tx=0x55b58d154c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56c0ea2dcbcbd77 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.754+0000 7fcd7ce5f640 1 -- 172.21.15.81:0/3087947275 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b58c2d4f20 con 0x55b58d06bc00 2023-11-08T16:34:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.754+0000 7fcd7ce5f640 1 -- 172.21.15.81:0/3087947275 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b58d070a80 con 0x55b58d06bc00 2023-11-08T16:34:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.754+0000 7fcd7ce5f640 1 -- 172.21.15.81:0/3087947275 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b58c2d51e0 con 0x55b58d06bc00 2023-11-08T16:34:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.754+0000 7fcd83638100 1 -- 172.21.15.81:0/3087947275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d06bc00 msgr2=0x55b58c2f9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.754+0000 7fcd83638100 1 --2- 172.21.15.81:0/3087947275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d06bc00 0x55b58c2f9600 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55b58d1ac0c0 tx=0x55b58d154c00 comp rx=0 tx=0).stop 2023-11-08T16:34:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.755+0000 7fcd83638100 1 -- 172.21.15.81:0/3087947275 shutdown_connections 2023-11-08T16:34:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.755+0000 7fcd83638100 1 --2- 172.21.15.81:0/3087947275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58d1a2400 0x55b58c2fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.755+0000 7fcd83638100 1 --2- 172.21.15.81:0/3087947275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58d1a2000 0x55b58c2f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.755+0000 7fcd83638100 1 --2- 172.21.15.81:0/3087947275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d06bc00 0x55b58c2f9600 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.755+0000 7fcd83638100 1 -- 172.21.15.81:0/3087947275 >> 172.21.15.81:0/3087947275 conn(0x55b58d06b800 msgr2=0x55b58d017000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.755+0000 7fcd83638100 1 -- 172.21.15.81:0/3087947275 shutdown_connections 2023-11-08T16:34:49.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.755+0000 7fcd83638100 1 -- 172.21.15.81:0/3087947275 wait complete. 2023-11-08T16:34:49.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.755+0000 7fcd83638100 1 Processor -- start 2023-11-08T16:34:49.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.756+0000 7fcd83638100 1 -- start start 2023-11-08T16:34:49.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.756+0000 7fcd83638100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58d06b800 0x55b58c2fb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.756+0000 7fcd83638100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d1a3000 0x55b58c2fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.756+0000 7fcd83638100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58d1a2c00 0x55b58c2fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.756+0000 7fcd83638100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58c2d6000 con 0x55b58d1a3000 2023-11-08T16:34:49.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.756+0000 7fcd83638100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b58c2d7180 con 0x55b58d1a2c00 2023-11-08T16:34:49.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.756+0000 7fcd83638100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b58c2d6b40 con 0x55b58d06b800 2023-11-08T16:34:49.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd81668640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d1a3000 0x55b58c2fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd81668640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d1a3000 0x55b58c2fac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48020/0 (socket says 172.21.15.81:48020) 2023-11-08T16:34:49.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd81668640 1 -- 172.21.15.81:0/1674140840 learned_addr learned my addr 172.21.15.81:0/1674140840 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd81e69640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58d06b800 0x55b58c2fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd80e67640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58d1a2c00 0x55b58c2fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd81668640 1 -- 172.21.15.81:0/1674140840 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58d06b800 msgr2=0x55b58c2fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd81668640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58d06b800 0x55b58c2fb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd81668640 1 -- 172.21.15.81:0/1674140840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58d1a2c00 msgr2=0x55b58c2fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd81668640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58d1a2c00 0x55b58c2fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd81668640 1 -- 172.21.15.81:0/1674140840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b58c2b2820 con 0x55b58d1a3000 2023-11-08T16:34:49.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd80e67640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58d1a2c00 0x55b58c2fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:49.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd81e69640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58d06b800 0x55b58c2fb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:49.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.757+0000 7fcd81668640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d1a3000 0x55b58c2fac00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55b58d1acde0 tx=0x55b58d154d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.758+0000 7fcd7b65c640 1 -- 172.21.15.81:0/1674140840 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b58c2d4f20 con 0x55b58d1a3000 2023-11-08T16:34:49.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.758+0000 7fcd7b65c640 1 -- 172.21.15.81:0/1674140840 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b58c2e8180 con 0x55b58d1a3000 2023-11-08T16:34:49.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.758+0000 7fcd83638100 1 -- 172.21.15.81:0/1674140840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b58c2b24e0 con 0x55b58d1a3000 2023-11-08T16:34:49.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.758+0000 7fcd7b65c640 1 -- 172.21.15.81:0/1674140840 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b58c2d51e0 con 0x55b58d1a3000 2023-11-08T16:34:49.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.758+0000 7fcd83638100 1 -- 172.21.15.81:0/1674140840 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b58c2b21a0 con 0x55b58d1a3000 2023-11-08T16:34:49.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.758+0000 7fcd83638100 4 main: resolving pool 7 2023-11-08T16:34:49.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.760+0000 7fcd7b65c640 1 -- 172.21.15.81:0/1674140840 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55b58c2ea840 con 0x55b58d1a3000 2023-11-08T16:34:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.760+0000 7fcd7b65c640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b58d1c0c00 0x55b58c2fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.760+0000 7fcd7b65c640 1 -- 172.21.15.81:0/1674140840 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55b58c2b8540 con 0x55b58d1a3000 2023-11-08T16:34:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.760+0000 7fcd83638100 4 main: creating IoCtx.. 2023-11-08T16:34:49.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.760+0000 7fcd83638100 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b58d1a2800 0x55b58c2fa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.760+0000 7fcd83638100 1 -- 172.21.15.81:0/1674140840 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:1 7.13 7:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55b58c2e7880 con 0x55b58d1a2800 2023-11-08T16:34:49.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.760+0000 7fcd81e69640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b58d1c0c00 0x55b58c2fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.761+0000 7fcd80e67640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b58d1a2800 0x55b58c2fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.761+0000 7fcd81e69640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b58d1c0c00 0x55b58c2fb700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55b58d1ac0c0 tx=0x55b58d01c180 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.761+0000 7fcd80e67640 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b58d1a2800 0x55b58c2fa100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd80e67640 1 -- 172.21.15.81:0/1674140840 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v64'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55b58cfe8b40 con 0x55b58d1a2800 2023-11-08T16:34:49.790 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:34:49.790 INFO:teuthology.orchestra.run.smithi081.stdout: "0": { 2023-11-08T16:34:49.791 INFO:teuthology.orchestra.run.smithi081.stdout: "data": {}, 2023-11-08T16:34:49.791 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:34:49.791 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:34:49.792 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:34:49.792 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd83638100 1 -- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b58d1a2800 msgr2=0x55b58c2fa100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd83638100 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b58d1a2800 0x55b58c2fa100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd83638100 1 -- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b58d1c0c00 msgr2=0x55b58c2fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd83638100 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b58d1c0c00 0x55b58c2fb700 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x55b58d1ac0c0 tx=0x55b58d01c180 comp rx=0 tx=0).stop 2023-11-08T16:34:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd83638100 1 -- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d1a3000 msgr2=0x55b58c2fac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd83638100 1 --2- 172.21.15.81:0/1674140840 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58d1a3000 0x55b58c2fac00 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x55b58d1acde0 tx=0x55b58d154d20 comp rx=0 tx=0).stop 2023-11-08T16:34:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd81668640 1 -- 172.21.15.81:0/1674140840 reap_dead start 2023-11-08T16:34:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd83638100 1 -- 172.21.15.81:0/1674140840 shutdown_connections 2023-11-08T16:34:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd83638100 1 -- 172.21.15.81:0/1674140840 >> 172.21.15.81:0/1674140840 conn(0x55b58d06bc00 msgr2=0x55b58d017000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd83638100 1 -- 172.21.15.81:0/1674140840 shutdown_connections 2023-11-08T16:34:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.765+0000 7fcd83638100 1 -- 172.21.15.81:0/1674140840 wait complete. 2023-11-08T16:34:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.766+0000 7fcd83638100 1 -- 172.21.15.81:0/829119336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58c28d800 msgr2=0x55b58c2f8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.766+0000 7fcd83638100 1 --2- 172.21.15.81:0/829119336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58c28d800 0x55b58c2f8b00 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x55b58d0aeba0 tx=0x55b58d136120 comp rx=0 tx=0).stop 2023-11-08T16:34:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.766+0000 7fcd83638100 1 -- 172.21.15.81:0/829119336 shutdown_connections 2023-11-08T16:34:49.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.766+0000 7fcd83638100 1 --2- 172.21.15.81:0/829119336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b58c28dc00 0x55b58c2f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.766+0000 7fcd83638100 1 --2- 172.21.15.81:0/829119336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b58c28d800 0x55b58c2f8b00 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.766+0000 7fcd83638100 1 --2- 172.21.15.81:0/829119336 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b58c28cc00 0x55b58c2f8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.766+0000 7fcd83638100 1 -- 172.21.15.81:0/829119336 >> 172.21.15.81:0/829119336 conn(0x55b58c28d000 msgr2=0x55b58d016800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.766+0000 7fcd83638100 1 -- 172.21.15.81:0/829119336 shutdown_connections 2023-11-08T16:34:49.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.766+0000 7fcd83638100 1 -- 172.21.15.81:0/829119336 wait complete. 2023-11-08T16:34:49.801 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.155985, result: { "0": { "data": {}, "result": 0 } } 2023-11-08T16:34:49.801 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-11-08T16:34:49.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.911+0000 7ff4e1817100 1 -- start start 2023-11-08T16:34:49.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.911+0000 7ff4e1817100 1 Processor -- start 2023-11-08T16:34:49.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.912+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f390c00 0x56104f3fc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.912+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f391800 0x56104f3fcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.912+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f391c00 0x56104f3fd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.912+0000 7ff4e1817100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104f3db180 con 0x56104f391800 2023-11-08T16:34:49.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.912+0000 7ff4e1817100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104f45f180 con 0x56104f390c00 2023-11-08T16:34:49.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.912+0000 7ff4e1817100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56104f45edc0 con 0x56104f391c00 2023-11-08T16:34:49.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.912+0000 7ff4deb95640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f391800 0x56104f3fcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.912+0000 7ff4deb95640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f391800 0x56104f3fcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48044/0 (socket says 172.21.15.81:48044) 2023-11-08T16:34:49.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.912+0000 7ff4deb95640 1 -- 172.21.15.81:0/1588002428 learned_addr learned my addr 172.21.15.81:0/1588002428 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.912+0000 7ff4dfb97640 1 --2- 172.21.15.81:0/1588002428 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f391c00 0x56104f3fd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.912+0000 7ff4df396640 1 --2- 172.21.15.81:0/1588002428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f390c00 0x56104f3fc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4deb95640 1 -- 172.21.15.81:0/1588002428 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f391c00 msgr2=0x56104f3fd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4deb95640 1 --2- 172.21.15.81:0/1588002428 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f391c00 0x56104f3fd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4deb95640 1 -- 172.21.15.81:0/1588002428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f390c00 msgr2=0x56104f3fc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4deb95640 1 --2- 172.21.15.81:0/1588002428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f390c00 0x56104f3fc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4deb95640 1 -- 172.21.15.81:0/1588002428 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56104f3b69c0 con 0x56104f391800 2023-11-08T16:34:49.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4deb95640 1 --2- 172.21.15.81:0/1588002428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f391800 0x56104f3fcb00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x56105019d1a0 tx=0x56104f676540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a067bee174c6235 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4dcb91640 1 -- 172.21.15.81:0/1588002428 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56104f3d91e0 con 0x56104f391800 2023-11-08T16:34:49.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4dcb91640 1 -- 172.21.15.81:0/1588002428 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56104f3ec480 con 0x56104f391800 2023-11-08T16:34:49.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4dcb91640 1 -- 172.21.15.81:0/1588002428 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56104f3d9080 con 0x56104f391800 2023-11-08T16:34:49.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4e1817100 1 -- 172.21.15.81:0/1588002428 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56104f3b6b60 con 0x56104f391800 2023-11-08T16:34:49.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4e1817100 1 -- 172.21.15.81:0/1588002428 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56104f3b6820 con 0x56104f391800 2023-11-08T16:34:49.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4e1817100 4 waiting for MDS map... 2023-11-08T16:34:49.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.913+0000 7ff4dcb91640 1 -- 172.21.15.81:0/1588002428 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56104f3bc380 con 0x56104f391800 2023-11-08T16:34:49.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.914+0000 7ff4dcb91640 1 -- 172.21.15.81:0/1588002428 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x56104f3e2f00 con 0x56104f391800 2023-11-08T16:34:49.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.914+0000 7ff4e1817100 4 Got MDS map 47 2023-11-08T16:34:49.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.914+0000 7ff4e1817100 10 main: JournalTool::main 2023-11-08T16:34:49.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.914+0000 7ff4e1817100 4 main: JournalTool: connecting to RADOS... 2023-11-08T16:34:49.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.914+0000 7ff4e1817100 1 Processor -- start 2023-11-08T16:34:49.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.915+0000 7ff4e1817100 1 -- start start 2023-11-08T16:34:49.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.915+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f673000 0x56104f3fd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.915+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.915+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f673800 0x56104f3fe100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.915+0000 7ff4e1817100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104f3da140 con 0x56104f673400 2023-11-08T16:34:49.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.915+0000 7ff4e1817100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104f3dab40 con 0x56104f673800 2023-11-08T16:34:49.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.915+0000 7ff4e1817100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56104f45f180 con 0x56104f673000 2023-11-08T16:34:49.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4df396640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4df396640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48048/0 (socket says 172.21.15.81:48048) 2023-11-08T16:34:49.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4df396640 1 -- 172.21.15.81:0/2385072305 learned_addr learned my addr 172.21.15.81:0/2385072305 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4dfb97640 1 --2- 172.21.15.81:0/2385072305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f673000 0x56104f3fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4df396640 1 -- 172.21.15.81:0/2385072305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f673000 msgr2=0x56104f3fd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4deb95640 1 --2- 172.21.15.81:0/2385072305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f673800 0x56104f3fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4df396640 1 --2- 172.21.15.81:0/2385072305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f673000 0x56104f3fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4df396640 1 -- 172.21.15.81:0/2385072305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f673800 msgr2=0x56104f3fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4df396640 1 --2- 172.21.15.81:0/2385072305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f673800 0x56104f3fe100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4df396640 1 -- 172.21.15.81:0/2385072305 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56104f3b64e0 con 0x56104f673400 2023-11-08T16:34:49.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4df396640 1 --2- 172.21.15.81:0/2385072305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fdb80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x56105019ca20 tx=0x56104f6762a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac70a3707c348e17 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.916+0000 7ff4dab8d640 1 -- 172.21.15.81:0/2385072305 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56104f3d8f20 con 0x56104f673400 2023-11-08T16:34:49.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.917+0000 7ff4dab8d640 1 -- 172.21.15.81:0/2385072305 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561050160f00 con 0x56104f673400 2023-11-08T16:34:49.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.917+0000 7ff4dab8d640 1 -- 172.21.15.81:0/2385072305 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56104f3d91e0 con 0x56104f673400 2023-11-08T16:34:49.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.917+0000 7ff4e1817100 1 -- 172.21.15.81:0/2385072305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 msgr2=0x56104f3fdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.917+0000 7ff4e1817100 1 --2- 172.21.15.81:0/2385072305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fdb80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x56105019ca20 tx=0x56104f6762a0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.917+0000 7ff4e1817100 1 -- 172.21.15.81:0/2385072305 shutdown_connections 2023-11-08T16:34:49.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.917+0000 7ff4e1817100 1 --2- 172.21.15.81:0/2385072305 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f673800 0x56104f3fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.917+0000 7ff4e1817100 1 --2- 172.21.15.81:0/2385072305 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fdb80 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.917+0000 7ff4e1817100 1 --2- 172.21.15.81:0/2385072305 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f673000 0x56104f3fd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.917+0000 7ff4e1817100 1 -- 172.21.15.81:0/2385072305 >> 172.21.15.81:0/2385072305 conn(0x56104f672c00 msgr2=0x561050118800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.917+0000 7ff4e1817100 1 -- 172.21.15.81:0/2385072305 shutdown_connections 2023-11-08T16:34:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.917+0000 7ff4e1817100 1 -- 172.21.15.81:0/2385072305 wait complete. 2023-11-08T16:34:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.918+0000 7ff4e1817100 1 Processor -- start 2023-11-08T16:34:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.918+0000 7ff4e1817100 1 -- start start 2023-11-08T16:34:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f673000 0x56104f3fd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f673800 0x56104f3fe100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fe680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4e1817100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104f45ec80 con 0x56104f673400 2023-11-08T16:34:49.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4e1817100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104f45eb40 con 0x56104f673000 2023-11-08T16:34:49.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4e1817100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56104f45ea00 con 0x56104f673800 2023-11-08T16:34:49.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4deb95640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fe680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4deb95640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fe680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48058/0 (socket says 172.21.15.81:48058) 2023-11-08T16:34:49.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4deb95640 1 -- 172.21.15.81:0/357163082 learned_addr learned my addr 172.21.15.81:0/357163082 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4dfb97640 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f673000 0x56104f3fd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4df396640 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f673800 0x56104f3fe100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4deb95640 1 -- 172.21.15.81:0/357163082 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f673800 msgr2=0x56104f3fe100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4deb95640 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f673800 0x56104f3fe100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4deb95640 1 -- 172.21.15.81:0/357163082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f673000 msgr2=0x56104f3fd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4deb95640 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f673000 0x56104f3fd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4deb95640 1 -- 172.21.15.81:0/357163082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56104f3b6d00 con 0x56104f673400 2023-11-08T16:34:49.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.919+0000 7ff4df396640 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f673800 0x56104f3fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:49.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.920+0000 7ff4deb95640 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fe680 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x56104f6eb470 tx=0x56104f676060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.920+0000 7ff4d938a640 1 -- 172.21.15.81:0/357163082 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56104f3d8dc0 con 0x56104f673400 2023-11-08T16:34:49.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.920+0000 7ff4d938a640 1 -- 172.21.15.81:0/357163082 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561050161680 con 0x56104f673400 2023-11-08T16:34:49.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.920+0000 7ff4e1817100 1 -- 172.21.15.81:0/357163082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56104f3b64e0 con 0x56104f673400 2023-11-08T16:34:49.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.920+0000 7ff4d938a640 1 -- 172.21.15.81:0/357163082 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56104f3d8f20 con 0x56104f673400 2023-11-08T16:34:49.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.920+0000 7ff4e1817100 1 -- 172.21.15.81:0/357163082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56104f3b69c0 con 0x56104f673400 2023-11-08T16:34:49.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.920+0000 7ff4e1817100 4 main: JournalTool: resolving pool 7 2023-11-08T16:34:49.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.922+0000 7ff4d938a640 1 -- 172.21.15.81:0/357163082 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x56104f420840 con 0x56104f673400 2023-11-08T16:34:49.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.922+0000 7ff4d938a640 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56104f7e0000 0x56104f3ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.922+0000 7ff4d938a640 1 -- 172.21.15.81:0/357163082 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56104f3bc540 con 0x56104f673400 2023-11-08T16:34:49.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.923+0000 7ff4e1817100 4 main: JournalTool: creating IoCtx.. 2023-11-08T16:34:49.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.923+0000 7ff4e1817100 4 main: Executing for rank 0 2023-11-08T16:34:49.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.923+0000 7ff4dfb97640 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56104f7e0000 0x56104f3ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.923+0000 7ff4dfb97640 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56104f7e0000 0x56104f3ff700 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x56104f484840 tx=0x56104f7f5800 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.924+0000 7ff4e1817100 1 -- start start 2023-11-08T16:34:49.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.924+0000 7ff4e1817100 1 Processor -- start 2023-11-08T16:34:49.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561050163000 0x56104f3fec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561050162c00 0x56104f3ff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4e1817100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561050162800 0x56104f84a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4e1817100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104f45e8c0 con 0x561050162800 2023-11-08T16:34:49.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4e1817100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56104f45ea00 con 0x561050162c00 2023-11-08T16:34:49.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4e1817100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56104f45eb40 con 0x561050163000 2023-11-08T16:34:49.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4deb95640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561050162800 0x56104f84a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4deb95640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561050162800 0x56104f84a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48062/0 (socket says 172.21.15.81:48062) 2023-11-08T16:34:49.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4deb95640 1 -- 172.21.15.81:0/1777927239 learned_addr learned my addr 172.21.15.81:0/1777927239 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:49.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4dfb97640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561050163000 0x56104f3fec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4df396640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561050162c00 0x56104f3ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4deb95640 1 -- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561050163000 msgr2=0x56104f3fec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4deb95640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561050163000 0x56104f3fec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4deb95640 1 -- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561050162c00 msgr2=0x56104f3ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:49.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4deb95640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561050162c00 0x56104f3ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4deb95640 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56104f721040 con 0x561050162800 2023-11-08T16:34:49.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.925+0000 7ff4dfb97640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561050163000 0x56104f3fec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:49.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.926+0000 7ff4deb95640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561050162800 0x56104f84a000 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x56104f727410 tx=0x56104f7f5980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb6768ccf688f5dc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.926+0000 7ff4d6384640 1 -- 172.21.15.81:0/1777927239 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56104f3d91e0 con 0x561050162800 2023-11-08T16:34:49.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.926+0000 7ff4d6384640 1 -- 172.21.15.81:0/1777927239 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561050161800 con 0x561050162800 2023-11-08T16:34:49.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.926+0000 7ff4d6384640 1 -- 172.21.15.81:0/1777927239 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56104f3d82c0 con 0x561050162800 2023-11-08T16:34:49.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.926+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56104f720ea0 con 0x561050162800 2023-11-08T16:34:49.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.926+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56104f720d00 con 0x561050162800 2023-11-08T16:34:49.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.926+0000 7ff4e1817100 4 waiting for MDS map... 2023-11-08T16:34:49.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.926+0000 7ff4d6384640 1 -- 172.21.15.81:0/1777927239 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56104f3bca80 con 0x561050162800 2023-11-08T16:34:49.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.926+0000 7ff4d6384640 1 -- 172.21.15.81:0/1777927239 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x56104f85a000 con 0x561050162800 2023-11-08T16:34:49.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.926+0000 7ff4e1817100 4 Got MDS map 47 2023-11-08T16:34:49.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.927+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56104f86a000 0x56104f84a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.927+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:1 7.f 7:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56104f868380 con 0x56104f86a000 2023-11-08T16:34:49.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.927+0000 7ff4dfb97640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56104f86a000 0x56104f84a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.927+0000 7ff4dfb97640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56104f86a000 0x56104f84a580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=f00aad8759e64807 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.929+0000 7ff4dfb97640 1 -- 172.21.15.81:0/1777927239 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v64'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x56104f666b40 con 0x56104f86a000 2023-11-08T16:34:49.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.929+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56104f86a400 0x56104f84ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.929+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:2 7.14 7:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56104f868700 con 0x56104f86a400 2023-11-08T16:34:49.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.930+0000 7ff4df396640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56104f86a400 0x56104f84ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.930+0000 7ff4df396640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56104f86a400 0x56104f84ab00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=a3e040391fb270c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.935+0000 7ff4df396640 1 -- 172.21.15.81:0/1777927239 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v64'101 uv101 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x56104f666d80 con 0x56104f86a400 2023-11-08T16:34:49.948 INFO:teuthology.orchestra.run.smithi081.stdout:writing EResetJournal entry 2023-11-08T16:34:49.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.935+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 7.14 7:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56104f868a80 con 0x56104f86a400 2023-11-08T16:34:49.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.935+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:4 7.1b 7:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56104f868e00 con 0x56104f86a000 2023-11-08T16:34:49.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.935+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56104f86a800 0x56104f84b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.935+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:5 7.b 7:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56104f869180 con 0x56104f86a800 2023-11-08T16:34:49.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.935+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56104f86ac00 0x56104f84b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.935+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:6 7.1f 7:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56104f869500 con 0x56104f86ac00 2023-11-08T16:34:49.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.935+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56104f86b000 0x56104f84bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:49.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.935+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:7 7.1a 7:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56104f869880 con 0x56104f86b000 2023-11-08T16:34:49.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.935+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:8 7.1e 7:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56104f868380 con 0x56104f86a800 2023-11-08T16:34:49.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.935+0000 7ff4dfb97640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56104f86a800 0x56104f84b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.936+0000 7ff4df396640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56104f86ac00 0x56104f84b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.936+0000 7ff4deb95640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56104f86b000 0x56104f84bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:49.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.936+0000 7ff4dfb97640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56104f86a800 0x56104f84b080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=cad34e13420bab07 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.937+0000 7ff4df396640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56104f86ac00 0x56104f84b600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=dbadc7c3ed787c7b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.937+0000 7ff4dfb97640 1 -- 172.21.15.81:0/1777927239 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(4 200.00000002 [delete] v64'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56104f666b40 con 0x56104f86a000 2023-11-08T16:34:49.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.937+0000 7ff4deb95640 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56104f86b000 0x56104f84bb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=d56e22c10da23a34 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:49.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.939+0000 7ff4df396640 1 -- 172.21.15.81:0/1777927239 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(3 200.00000001 [delete] v64'102 uv101 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56104f666d80 con 0x56104f86a400 2023-11-08T16:34:49.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.939+0000 7ff4dfb97640 1 -- 172.21.15.81:0/1777927239 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(8 200.00000006 [delete] v64'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56104f666b40 con 0x56104f86a800 2023-11-08T16:34:49.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.939+0000 7ff4d4b81640 1 -- 172.21.15.81:0/1777927239 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 7.14 7:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56104f88a000 con 0x56104f86a400 2023-11-08T16:34:49.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.939+0000 7ff4dfb97640 1 -- 172.21.15.81:0/1777927239 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(5 200.00000003 [delete] v64'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56104f666b40 con 0x56104f86a800 2023-11-08T16:34:49.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.941+0000 7ff4deb95640 1 -- 172.21.15.81:0/1777927239 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(7 200.00000005 [delete] v64'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56104f6678c0 con 0x56104f86b000 2023-11-08T16:34:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4df396640 1 -- 172.21.15.81:0/1777927239 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(6 200.00000004 [delete] v64'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56104f666d80 con 0x56104f86ac00 2023-11-08T16:34:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4df396640 1 -- 172.21.15.81:0/1777927239 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v64'103 uv103 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x56104f666d80 con 0x56104f86a400 2023-11-08T16:34:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 4 Successfully wrote new journal pointer and header for rank 3:0 2023-11-08T16:34:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56104f86a000 msgr2=0x56104f84a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56104f86a000 0x56104f84a580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56104f86a800 msgr2=0x56104f84b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56104f86a800 0x56104f84b080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56104f86a400 msgr2=0x56104f84ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56104f86a400 0x56104f84ab00 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56104f86b000 msgr2=0x56104f84bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56104f86b000 0x56104f84bb80 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4df396640 1 -- 172.21.15.81:0/1777927239 reap_dead start 2023-11-08T16:34:49.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56104f86ac00 msgr2=0x56104f84b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56104f86ac00 0x56104f84b600 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561050162800 msgr2=0x56104f84a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561050162800 0x56104f84a000 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x56104f727410 tx=0x56104f7f5980 comp rx=0 tx=0).stop 2023-11-08T16:34:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 shutdown_connections 2023-11-08T16:34:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56104f86ac00 0x56104f84b600 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561050162800 0x56104f84a000 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1777927239 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56104f86b000 0x56104f84bb80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 >> 172.21.15.81:0/1777927239 conn(0x561050163400 msgr2=0x561050118000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 shutdown_connections 2023-11-08T16:34:49.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.942+0000 7ff4e1817100 1 -- 172.21.15.81:0/1777927239 wait complete. 2023-11-08T16:34:49.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 -- 172.21.15.81:0/357163082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56104f7e0000 msgr2=0x56104f3ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56104f7e0000 0x56104f3ff700 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x56104f484840 tx=0x56104f7f5800 comp rx=0 tx=0).stop 2023-11-08T16:34:49.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 -- 172.21.15.81:0/357163082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 msgr2=0x56104f3fe680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fe680 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x56104f6eb470 tx=0x56104f676060 comp rx=0 tx=0).stop 2023-11-08T16:34:49.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 -- 172.21.15.81:0/357163082 shutdown_connections 2023-11-08T16:34:49.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56104f7e0000 0x56104f3ff700 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f673400 0x56104f3fe680 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f673800 0x56104f3fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 --2- 172.21.15.81:0/357163082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f673000 0x56104f3fd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 -- 172.21.15.81:0/357163082 >> 172.21.15.81:0/357163082 conn(0x56104f672c00 msgr2=0x561050118800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 -- 172.21.15.81:0/357163082 shutdown_connections 2023-11-08T16:34:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 -- 172.21.15.81:0/357163082 wait complete. 2023-11-08T16:34:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.943+0000 7ff4e1817100 1 -- 172.21.15.81:0/1588002428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f391800 msgr2=0x56104f3fcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.944+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1588002428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f391800 0x56104f3fcb00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x56105019d1a0 tx=0x56104f676540 comp rx=0 tx=0).stop 2023-11-08T16:34:49.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.944+0000 7ff4e1817100 1 -- 172.21.15.81:0/1588002428 shutdown_connections 2023-11-08T16:34:49.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.944+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1588002428 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56104f391c00 0x56104f3fd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.944+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1588002428 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56104f391800 0x56104f3fcb00 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.944+0000 7ff4e1817100 1 --2- 172.21.15.81:0/1588002428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56104f390c00 0x56104f3fc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:49.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.944+0000 7ff4e1817100 1 -- 172.21.15.81:0/1588002428 >> 172.21.15.81:0/1588002428 conn(0x56104f391000 msgr2=0x561050119000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:49.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.944+0000 7ff4e1817100 1 -- 172.21.15.81:0/1588002428 shutdown_connections 2023-11-08T16:34:49.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:49.944+0000 7ff4e1817100 1 -- 172.21.15.81:0/1588002428 wait complete. 2023-11-08T16:34:49.968 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.166783, result: writing EResetJournal entry 2023-11-08T16:34:49.969 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-11-08T16:34:50.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.065+0000 7f45bba3c5c0 1 -- start start 2023-11-08T16:34:50.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.065+0000 7f45bba3c5c0 1 Processor -- start 2023-11-08T16:34:50.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.066+0000 7f45bba3c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152d786c00 0x56152d7f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.066+0000 7f45bba3c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152d787800 0x56152d7f0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.066+0000 7f45bba3c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152d787c00 0x56152d7f1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.066+0000 7f45bba3c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152d7cf180 con 0x56152d786c00 2023-11-08T16:34:50.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.066+0000 7f45bba3c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152d7ce140 con 0x56152d787c00 2023-11-08T16:34:50.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.066+0000 7f45bba3c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56152d7ce000 con 0x56152d787800 2023-11-08T16:34:50.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.066+0000 7f45b99b3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152d786c00 0x56152d7f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.066+0000 7f45b99b3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152d786c00 0x56152d7f0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48074/0 (socket says 172.21.15.81:48074) 2023-11-08T16:34:50.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.066+0000 7f45b99b3640 1 -- 172.21.15.81:0/1710777961 learned_addr learned my addr 172.21.15.81:0/1710777961 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.066+0000 7f45ba1b4640 1 --2- 172.21.15.81:0/1710777961 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152d787c00 0x56152d7f1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45b91b2640 1 --2- 172.21.15.81:0/1710777961 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152d787800 0x56152d7f0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45b99b3640 1 -- 172.21.15.81:0/1710777961 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152d787800 msgr2=0x56152d7f0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45b99b3640 1 --2- 172.21.15.81:0/1710777961 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152d787800 0x56152d7f0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45b99b3640 1 -- 172.21.15.81:0/1710777961 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152d787c00 msgr2=0x56152d7f1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45b99b3640 1 --2- 172.21.15.81:0/1710777961 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152d787c00 0x56152d7f1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45b99b3640 1 -- 172.21.15.81:0/1710777961 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56152d7ac9c0 con 0x56152d786c00 2023-11-08T16:34:50.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45ba1b4640 1 --2- 172.21.15.81:0/1710777961 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152d787c00 0x56152d7f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:50.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45b99b3640 1 --2- 172.21.15.81:0/1710777961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152d786c00 0x56152d7f0580 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x56152e597410 tx=0x56152e640060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c7f64daad8897f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45b71ae640 1 -- 172.21.15.81:0/1710777961 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56152d7cd1e0 con 0x56152d786c00 2023-11-08T16:34:50.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45b71ae640 1 -- 172.21.15.81:0/1710777961 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56152d7e0480 con 0x56152d786c00 2023-11-08T16:34:50.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1710777961 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56152d7acb60 con 0x56152d786c00 2023-11-08T16:34:50.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1710777961 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56152d7ac820 con 0x56152d786c00 2023-11-08T16:34:50.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45bba3c5c0 4 waiting for MDS map... 2023-11-08T16:34:50.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45b71ae640 1 -- 172.21.15.81:0/1710777961 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56152d7cd080 con 0x56152d786c00 2023-11-08T16:34:50.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.067+0000 7f45b71ae640 1 -- 172.21.15.81:0/1710777961 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56152d846c40 con 0x56152d786c00 2023-11-08T16:34:50.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.068+0000 7f45b71ae640 1 -- 172.21.15.81:0/1710777961 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x56152d7d6f00 con 0x56152d786c00 2023-11-08T16:34:50.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.068+0000 7f45bba3c5c0 4 Got MDS map 47 2023-11-08T16:34:50.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.068+0000 7f45bba3c5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:50.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.068+0000 7f45bba3c5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:50.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.069+0000 7f45bba3c5c0 1 Processor -- start 2023-11-08T16:34:50.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.069+0000 7f45bba3c5c0 1 -- start start 2023-11-08T16:34:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.069+0000 7f45bba3c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643000 0x56152d7f1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.069+0000 7f45bba3c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152e643400 0x56152d7f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.069+0000 7f45bba3c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152e643800 0x56152d7f2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.069+0000 7f45bba3c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152d7cea00 con 0x56152e643000 2023-11-08T16:34:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.069+0000 7f45bba3c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152d855180 con 0x56152e643800 2023-11-08T16:34:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.069+0000 7f45bba3c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56152d854dc0 con 0x56152e643400 2023-11-08T16:34:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.070+0000 7f45ba1b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643000 0x56152d7f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.070+0000 7f45ba1b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643000 0x56152d7f1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48088/0 (socket says 172.21.15.81:48088) 2023-11-08T16:34:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.070+0000 7f45ba1b4640 1 -- 172.21.15.81:0/2392100179 learned_addr learned my addr 172.21.15.81:0/2392100179 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.070+0000 7f45b91b2640 1 --2- 172.21.15.81:0/2392100179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152e643800 0x56152d7f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.070+0000 7f45ba1b4640 1 -- 172.21.15.81:0/2392100179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152e643400 msgr2=0x56152d7f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.070+0000 7f45ba1b4640 1 --2- 172.21.15.81:0/2392100179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152e643400 0x56152d7f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.070+0000 7f45ba1b4640 1 -- 172.21.15.81:0/2392100179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152e643800 msgr2=0x56152d7f2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.070+0000 7f45ba1b4640 1 --2- 172.21.15.81:0/2392100179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152e643800 0x56152d7f2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.070+0000 7f45ba1b4640 1 -- 172.21.15.81:0/2392100179 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56152d7ac4e0 con 0x56152e643000 2023-11-08T16:34:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45ba1b4640 1 --2- 172.21.15.81:0/2392100179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643000 0x56152d7f1600 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x56152e596a50 tx=0x56152e63e120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f675ef2e977fbde2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45b51aa640 1 -- 172.21.15.81:0/2392100179 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56152d7ccf20 con 0x56152e643000 2023-11-08T16:34:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45b51aa640 1 -- 172.21.15.81:0/2392100179 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56152d866c00 con 0x56152e643000 2023-11-08T16:34:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45b51aa640 1 -- 172.21.15.81:0/2392100179 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56152d7cd1e0 con 0x56152e643000 2023-11-08T16:34:50.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/2392100179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643000 msgr2=0x56152d7f1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/2392100179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643000 0x56152d7f1600 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x56152e596a50 tx=0x56152e63e120 comp rx=0 tx=0).stop 2023-11-08T16:34:50.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/2392100179 shutdown_connections 2023-11-08T16:34:50.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/2392100179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152e643800 0x56152d7f2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/2392100179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152e643400 0x56152d7f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/2392100179 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643000 0x56152d7f1600 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/2392100179 >> 172.21.15.81:0/2392100179 conn(0x56152e642c00 msgr2=0x56152e503000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/2392100179 shutdown_connections 2023-11-08T16:34:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.071+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/2392100179 wait complete. 2023-11-08T16:34:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.072+0000 7f45bba3c5c0 1 Processor -- start 2023-11-08T16:34:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.072+0000 7f45bba3c5c0 1 -- start start 2023-11-08T16:34:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45bba3c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152e643000 0x56152d7f3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45bba3c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152e643400 0x56152d7f2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45bba3c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643800 0x56152d7f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45bba3c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152d854c80 con 0x56152e643800 2023-11-08T16:34:50.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45bba3c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56152d7ce140 con 0x56152e643000 2023-11-08T16:34:50.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45bba3c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56152d7ceb40 con 0x56152e643400 2023-11-08T16:34:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45b91b2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643800 0x56152d7f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45b91b2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643800 0x56152d7f2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48098/0 (socket says 172.21.15.81:48098) 2023-11-08T16:34:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45b91b2640 1 -- 172.21.15.81:0/1975767493 learned_addr learned my addr 172.21.15.81:0/1975767493 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45ba1b4640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152e643000 0x56152d7f3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45b91b2640 1 -- 172.21.15.81:0/1975767493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152e643400 msgr2=0x56152d7f2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.073+0000 7f45b91b2640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152e643400 0x56152d7f2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.074+0000 7f45b91b2640 1 -- 172.21.15.81:0/1975767493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152e643000 msgr2=0x56152d7f3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.074+0000 7f45b91b2640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152e643000 0x56152d7f3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.074+0000 7f45b91b2640 1 -- 172.21.15.81:0/1975767493 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56152d7acd00 con 0x56152e643800 2023-11-08T16:34:50.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.074+0000 7f45b91b2640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643800 0x56152d7f2680 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x56152e65b5c0 tx=0x56152e63f020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.074+0000 7f45b39a7640 1 -- 172.21.15.81:0/1975767493 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56152d7ccf20 con 0x56152e643800 2023-11-08T16:34:50.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.074+0000 7f45b39a7640 1 -- 172.21.15.81:0/1975767493 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56152d7e0180 con 0x56152e643800 2023-11-08T16:34:50.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.074+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56152d7ac4e0 con 0x56152e643800 2023-11-08T16:34:50.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.074+0000 7f45b39a7640 1 -- 172.21.15.81:0/1975767493 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56152d7cd1e0 con 0x56152e643800 2023-11-08T16:34:50.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.074+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56152d7ac9c0 con 0x56152e643800 2023-11-08T16:34:50.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.074+0000 7f45bba3c5c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:50.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.076+0000 7f45b39a7640 1 -- 172.21.15.81:0/1975767493 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x56152d7e2840 con 0x56152e643800 2023-11-08T16:34:50.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.076+0000 7f45b39a7640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56152e6c0800 0x56152d7f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.076+0000 7f45b39a7640 1 -- 172.21.15.81:0/1975767493 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56152d846a80 con 0x56152e643800 2023-11-08T16:34:50.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.077+0000 7f45bba3c5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:50.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.077+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56152e6c0c00 0x56152d7f2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.077+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 7.14 7:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56152d7df880 con 0x56152e6c0c00 2023-11-08T16:34:50.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.077+0000 7f45ba1b4640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56152e6c0800 0x56152d7f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.077+0000 7f45b99b3640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56152e6c0c00 0x56152d7f2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.077+0000 7f45ba1b4640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56152e6c0800 0x56152d7f3700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x56152e596a50 tx=0x56152e63e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.078+0000 7f45b99b3640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56152e6c0c00 0x56152d7f2100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.078+0000 7f45b99b3640 1 -- 172.21.15.81:0/1975767493 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x56152e568b40 con 0x56152e6c0c00 2023-11-08T16:34:50.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.078+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:2 7.14 7:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56152d7dfc00 con 0x56152e6c0c00 2023-11-08T16:34:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.080+0000 7f45b99b3640 1 -- 172.21.15.81:0/1975767493 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v64'104 uv104 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x56152e568b40 con 0x56152e6c0c00 2023-11-08T16:34:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.080+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56152e6c1000 0x56152d7f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.080+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:3 7.1f 7:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56152daea000 con 0x56152e6c1000 2023-11-08T16:34:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.080+0000 7f45ba1b4640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56152e6c1000 0x56152d7f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.081+0000 7f45ba1b4640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56152e6c1000 0x56152d7f1b80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.082+0000 7f45ba1b4640 1 -- 172.21.15.81:0/1975767493 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x56152e569200 con 0x56152e6c1000 2023-11-08T16:34:50.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.082+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:4 7.1f 7:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56152daea380 con 0x56152e6c1000 2023-11-08T16:34:50.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.086+0000 7f45ba1b4640 1 -- 172.21.15.81:0/1975767493 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v64'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x56152e569200 con 0x56152e6c1000 2023-11-08T16:34:50.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.086+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56152e6c1400 0x56152d7f1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.086+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 7.13 7:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56152daea700 con 0x56152e6c1400 2023-11-08T16:34:50.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.086+0000 7f45b99b3640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56152e6c1400 0x56152d7f1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.087+0000 7f45b99b3640 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56152e6c1400 0x56152d7f1600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.087+0000 7f45b99b3640 1 -- 172.21.15.81:0/1975767493 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x56152e568b40 con 0x56152e6c1400 2023-11-08T16:34:50.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.087+0000 7f45bba3c5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:34:50.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.087+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:6 7.13 7:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56152daeaa80 con 0x56152e6c1400 2023-11-08T16:34:50.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45b99b3640 1 -- 172.21.15.81:0/1975767493 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v64'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x56152e568b40 con 0x56152e6c1400 2023-11-08T16:34:50.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-11-08T16:34:50.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56152e6c1400 msgr2=0x56152d7f1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56152e6c1400 0x56152d7f1600 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56152e6c0c00 msgr2=0x56152d7f2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56152e6c0c00 0x56152d7f2100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56152e6c1000 msgr2=0x56152d7f1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56152e6c1000 0x56152d7f1b80 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45b91b2640 1 -- 172.21.15.81:0/1975767493 reap_dead start 2023-11-08T16:34:50.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56152e6c0800 msgr2=0x56152d7f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56152e6c0800 0x56152d7f3700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x56152e596a50 tx=0x56152e63e000 comp rx=0 tx=0).stop 2023-11-08T16:34:50.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643800 msgr2=0x56152d7f2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643800 0x56152d7f2680 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x56152e65b5c0 tx=0x56152e63f020 comp rx=0 tx=0).stop 2023-11-08T16:34:50.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 shutdown_connections 2023-11-08T16:34:50.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56152e6c0800 0x56152d7f3700 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1975767493 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152e643800 0x56152d7f2680 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.090+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 >> 172.21.15.81:0/1975767493 conn(0x56152e642c00 msgr2=0x56152e503000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.091+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 shutdown_connections 2023-11-08T16:34:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.091+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1975767493 wait complete. 2023-11-08T16:34:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.091+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1710777961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152d786c00 msgr2=0x56152d7f0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.091+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1710777961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152d786c00 0x56152d7f0580 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x56152e597410 tx=0x56152e640060 comp rx=0 tx=0).stop 2023-11-08T16:34:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.091+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1710777961 shutdown_connections 2023-11-08T16:34:50.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.091+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1710777961 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56152d787c00 0x56152d7f1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.091+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1710777961 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56152d787800 0x56152d7f0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.091+0000 7f45bba3c5c0 1 --2- 172.21.15.81:0/1710777961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56152d786c00 0x56152d7f0580 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.091+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1710777961 >> 172.21.15.81:0/1710777961 conn(0x56152d787000 msgr2=0x56152e502800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.091+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1710777961 shutdown_connections 2023-11-08T16:34:50.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.091+0000 7f45bba3c5c0 1 -- 172.21.15.81:0/1710777961 wait complete. 2023-11-08T16:34:50.107 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.138084, result: 2023-11-08T16:34:50.107 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 0 --worker_m 7 2023-11-08T16:34:50.108 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 1 --worker_m 7 2023-11-08T16:34:50.108 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 2 --worker_m 7 2023-11-08T16:34:50.109 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 3 --worker_m 7 2023-11-08T16:34:50.109 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 4 --worker_m 7 2023-11-08T16:34:50.110 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 5 --worker_m 7 2023-11-08T16:34:50.111 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --worker_n 6 --worker_m 7 2023-11-08T16:34:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.220+0000 7fe706e195c0 1 -- start start 2023-11-08T16:34:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.220+0000 7fe706e195c0 1 Processor -- start 2023-11-08T16:34:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.221+0000 7fe706e195c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e2a60c00 0x5636e2aca580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.221+0000 7fe706e195c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e2a61800 0x5636e2acab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.221+0000 7fe706e195c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e2a61c00 0x5636e2acb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.221+0000 7fe706e195c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e2aa9180 con 0x5636e2a60c00 2023-11-08T16:34:50.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.221+0000 7fe706e195c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e2aa8140 con 0x5636e2a61800 2023-11-08T16:34:50.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.221+0000 7fe706e195c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636e2aa8000 con 0x5636e2a61c00 2023-11-08T16:34:50.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.222+0000 7fe7055bf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e2a60c00 0x5636e2aca580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.222+0000 7fe7055bf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e2a60c00 0x5636e2aca580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48114/0 (socket says 172.21.15.81:48114) 2023-11-08T16:34:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.222+0000 7fe7055bf640 1 -- 172.21.15.81:0/1875630176 learned_addr learned my addr 172.21.15.81:0/1875630176 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.222+0000 7fe704dbe640 1 --2- 172.21.15.81:0/1875630176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e2a61800 0x5636e2acab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.222+0000 7fe705dc0640 1 --2- 172.21.15.81:0/1875630176 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e2a61c00 0x5636e2acb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.223+0000 7fe7055bf640 1 -- 172.21.15.81:0/1875630176 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e2a61c00 msgr2=0x5636e2acb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.223+0000 7fe7055bf640 1 --2- 172.21.15.81:0/1875630176 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e2a61c00 0x5636e2acb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.223+0000 7fe7055bf640 1 -- 172.21.15.81:0/1875630176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e2a61800 msgr2=0x5636e2acab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.223+0000 7fe7055bf640 1 --2- 172.21.15.81:0/1875630176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e2a61800 0x5636e2acab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.223+0000 7fe7055bf640 1 -- 172.21.15.81:0/1875630176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636e2a869c0 con 0x5636e2a60c00 2023-11-08T16:34:50.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.223+0000 7fe7055bf640 1 --2- 172.21.15.81:0/1875630176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e2a60c00 0x5636e2aca580 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x5636e3875410 tx=0x5636e390e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ad957b61adce4d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.224+0000 7fe702dba640 1 -- 172.21.15.81:0/1875630176 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636e2aa71e0 con 0x5636e2a60c00 2023-11-08T16:34:50.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.224+0000 7fe702dba640 1 -- 172.21.15.81:0/1875630176 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5636e2aba480 con 0x5636e2a60c00 2023-11-08T16:34:50.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.224+0000 7fe702dba640 1 -- 172.21.15.81:0/1875630176 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636e2aa7080 con 0x5636e2a60c00 2023-11-08T16:34:50.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.224+0000 7fe706e195c0 1 -- 172.21.15.81:0/1875630176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636e2a86b60 con 0x5636e2a60c00 2023-11-08T16:34:50.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.224+0000 7fe706e195c0 1 -- 172.21.15.81:0/1875630176 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5636e2a86820 con 0x5636e2a60c00 2023-11-08T16:34:50.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.224+0000 7fe706e195c0 4 waiting for MDS map... 2023-11-08T16:34:50.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.225+0000 7fe702dba640 1 -- 172.21.15.81:0/1875630176 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x5636e2b20c40 con 0x5636e2a60c00 2023-11-08T16:34:50.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.225+0000 7fe702dba640 1 -- 172.21.15.81:0/1875630176 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x5636e2ab0f00 con 0x5636e2a60c00 2023-11-08T16:34:50.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.225+0000 7fe706e195c0 4 Got MDS map 47 2023-11-08T16:34:50.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.225+0000 7fe706e195c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:50.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.225+0000 7fe706e195c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:50.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.226+0000 7fe706e195c0 1 Processor -- start 2023-11-08T16:34:50.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.226+0000 7fe706e195c0 1 -- start start 2023-11-08T16:34:50.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.227+0000 7fe706e195c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e3927000 0x5636e2acb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.227+0000 7fe706e195c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e3927400 0x5636e2acbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.227+0000 7fe706e195c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 0x5636e2acc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.227+0000 7fe706e195c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e2aa8a00 con 0x5636e3927800 2023-11-08T16:34:50.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.227+0000 7fe706e195c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e2b2f180 con 0x5636e3927400 2023-11-08T16:34:50.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.227+0000 7fe706e195c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636e2b2edc0 con 0x5636e3927000 2023-11-08T16:34:50.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.228+0000 7fe704dbe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 0x5636e2acc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.228+0000 7fe704dbe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 0x5636e2acc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48122/0 (socket says 172.21.15.81:48122) 2023-11-08T16:34:50.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.228+0000 7fe704dbe640 1 -- 172.21.15.81:0/3601717200 learned_addr learned my addr 172.21.15.81:0/3601717200 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.228+0000 7fe705dc0640 1 --2- 172.21.15.81:0/3601717200 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e3927000 0x5636e2acb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.228+0000 7fe7055bf640 1 --2- 172.21.15.81:0/3601717200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e3927400 0x5636e2acbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.229+0000 7fe704dbe640 1 -- 172.21.15.81:0/3601717200 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e3927000 msgr2=0x5636e2acb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.229+0000 7fe704dbe640 1 --2- 172.21.15.81:0/3601717200 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e3927000 0x5636e2acb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.229+0000 7fe704dbe640 1 -- 172.21.15.81:0/3601717200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e3927400 msgr2=0x5636e2acbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.229+0000 7fe704dbe640 1 --2- 172.21.15.81:0/3601717200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e3927400 0x5636e2acbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.229+0000 7fe704dbe640 1 -- 172.21.15.81:0/3601717200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636e2a864e0 con 0x5636e3927800 2023-11-08T16:34:50.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.229+0000 7fe704dbe640 1 --2- 172.21.15.81:0/3601717200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 0x5636e2acc100 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x5636e3874a50 tx=0x5636e390e120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f3fa7a5bb30da1a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.230+0000 7fe700db6640 1 -- 172.21.15.81:0/3601717200 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636e2aa6f20 con 0x5636e3927800 2023-11-08T16:34:50.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.230+0000 7fe700db6640 1 -- 172.21.15.81:0/3601717200 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5636e3843080 con 0x5636e3927800 2023-11-08T16:34:50.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.230+0000 7fe700db6640 1 -- 172.21.15.81:0/3601717200 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636e2aa71e0 con 0x5636e3927800 2023-11-08T16:34:50.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.231+0000 7fe706e195c0 1 -- 172.21.15.81:0/3601717200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 msgr2=0x5636e2acc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.231+0000 7fe706e195c0 1 --2- 172.21.15.81:0/3601717200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 0x5636e2acc100 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x5636e3874a50 tx=0x5636e390e120 comp rx=0 tx=0).stop 2023-11-08T16:34:50.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.231+0000 7fe706e195c0 1 -- 172.21.15.81:0/3601717200 shutdown_connections 2023-11-08T16:34:50.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.231+0000 7fe706e195c0 1 --2- 172.21.15.81:0/3601717200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 0x5636e2acc100 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.231+0000 7fe706e195c0 1 --2- 172.21.15.81:0/3601717200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e3927400 0x5636e2acbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.231+0000 7fe706e195c0 1 --2- 172.21.15.81:0/3601717200 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e3927000 0x5636e2acb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.231+0000 7fe706e195c0 1 -- 172.21.15.81:0/3601717200 >> 172.21.15.81:0/3601717200 conn(0x5636e3926c00 msgr2=0x5636e37dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.231+0000 7fe706e195c0 1 -- 172.21.15.81:0/3601717200 shutdown_connections 2023-11-08T16:34:50.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.231+0000 7fe706e195c0 1 -- 172.21.15.81:0/3601717200 wait complete. 2023-11-08T16:34:50.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.232+0000 7fe706e195c0 1 Processor -- start 2023-11-08T16:34:50.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.232+0000 7fe706e195c0 1 -- start start 2023-11-08T16:34:50.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.233+0000 7fe706e195c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e3927000 0x5636e2acd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.233+0000 7fe706e195c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e3927400 0x5636e2accc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.233+0000 7fe706e195c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 0x5636e2acc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.233+0000 7fe706e195c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e2b2ec80 con 0x5636e3927800 2023-11-08T16:34:50.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.233+0000 7fe706e195c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636e2aa8140 con 0x5636e3927000 2023-11-08T16:34:50.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.233+0000 7fe706e195c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5636e2aa8b40 con 0x5636e3927400 2023-11-08T16:34:50.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.234+0000 7fe704dbe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 0x5636e2acc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.234+0000 7fe704dbe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 0x5636e2acc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48130/0 (socket says 172.21.15.81:48130) 2023-11-08T16:34:50.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.234+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 learned_addr learned my addr 172.21.15.81:0/203102485 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.234+0000 7fe705dc0640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e3927000 0x5636e2acd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.234+0000 7fe7055bf640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e3927400 0x5636e2accc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.235+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e3927400 msgr2=0x5636e2accc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.235+0000 7fe704dbe640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e3927400 0x5636e2accc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.235+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e3927000 msgr2=0x5636e2acd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.235+0000 7fe704dbe640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e3927000 0x5636e2acd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.235+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636e2a86d00 con 0x5636e3927800 2023-11-08T16:34:50.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.235+0000 7fe7055bf640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e3927400 0x5636e2accc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:34:50.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.235+0000 7fe704dbe640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 0x5636e2acc680 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x5636e3938a20 tx=0x5636e390e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.235+0000 7fe6ff5b3640 1 -- 172.21.15.81:0/203102485 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636e2aa7080 con 0x5636e3927800 2023-11-08T16:34:50.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.235+0000 7fe6ff5b3640 1 -- 172.21.15.81:0/203102485 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5636e3842f00 con 0x5636e3927800 2023-11-08T16:34:50.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.235+0000 7fe6ff5b3640 1 -- 172.21.15.81:0/203102485 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5636e2aa6420 con 0x5636e3927800 2023-11-08T16:34:50.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.235+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5636e2a864e0 con 0x5636e3927800 2023-11-08T16:34:50.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.236+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5636e2a869c0 con 0x5636e3927800 2023-11-08T16:34:50.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.236+0000 7fe706e195c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:50.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.239+0000 7fe6ff5b3640 1 -- 172.21.15.81:0/203102485 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x5636e2abd080 con 0x5636e3927800 2023-11-08T16:34:50.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.240+0000 7fe6ff5b3640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5636e3927c00 0x5636e2acd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.240+0000 7fe6ff5b3640 1 -- 172.21.15.81:0/203102485 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x5636e2b20a80 con 0x5636e3927800 2023-11-08T16:34:50.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.240+0000 7fe706e195c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:50.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.240+0000 7fe706e195c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:50.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.240+0000 7fe706e195c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:50.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.240+0000 7fe706e195c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 9 2023-11-08T16:34:50.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.240+0000 7fe706e195c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-11-08T16:34:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.240+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5636e2dd8000 0x5636e2acc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.240+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 8.0 8:00000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ab9880 con 0x5636e2dd8000 2023-11-08T16:34:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.241+0000 7fe705dc0640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5636e3927c00 0x5636e2acd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.241+0000 7fe7055bf640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5636e2dd8000 0x5636e2acc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.242+0000 7fe7055bf640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5636e2dd8000 0x5636e2acc100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.242+0000 7fe705dc0640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5636e3927c00 0x5636e2acd700 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x5636e2b52810 tx=0x5636e393cf00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.243+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 [pgnls start_epoch 64 out=81b] v62'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.243+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:2 8.0 8:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ab9c00 con 0x5636e2dd8000 2023-11-08T16:34:50.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.243+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(2 10000000206.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.243+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628294.0 2023-11-08T16:34:50.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.243+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 8.0 8:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddc000 con 0x5636e2dd8000 2023-11-08T16:34:50.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.247+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(3 10000000206.00000000 [call] v64'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.247+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5636e2dd8400 0x5636e2acbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.247+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:4 8.20 8:04000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddc380 con 0x5636e2dd8400 2023-11-08T16:34:50.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.247+0000 7f16dfc385c0 1 -- start start 2023-11-08T16:34:50.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.247+0000 7f16dfc385c0 1 Processor -- start 2023-11-08T16:34:50.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.248+0000 7fe705dc0640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5636e2dd8400 0x5636e2acbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.249+0000 7fe705dc0640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5636e2dd8400 0x5636e2acbb80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.249+0000 7f16dfc385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35aa8c00 0x563e35b12580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.249+0000 7f16dfc385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35aa9800 0x563e35b12b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.249+0000 7f16dfc385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35aa9c00 0x563e35b13080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.249+0000 7f16dfc385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35af1180 con 0x563e35aa8c00 2023-11-08T16:34:50.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.249+0000 7f16dfc385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35b77180 con 0x563e35aa9800 2023-11-08T16:34:50.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.249+0000 7f16dfc385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e35b76dc0 con 0x563e35aa9c00 2023-11-08T16:34:50.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.249+0000 7f16ddb88640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35aa8c00 0x563e35b12580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.250+0000 7f16ddb88640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35aa8c00 0x563e35b12580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48148/0 (socket says 172.21.15.81:48148) 2023-11-08T16:34:50.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.250+0000 7f16ddb88640 1 -- 172.21.15.81:0/2004944930 learned_addr learned my addr 172.21.15.81:0/2004944930 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.250+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(4 [pgnls start_epoch 64 out=81b] v62'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2dd8400 2023-11-08T16:34:50.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.250+0000 7f16dd387640 1 --2- 172.21.15.81:0/2004944930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35aa9800 0x563e35b12b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.250+0000 7f16de389640 1 --2- 172.21.15.81:0/2004944930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35aa9c00 0x563e35b13080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.251+0000 7f16ddb88640 1 -- 172.21.15.81:0/2004944930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35aa9c00 msgr2=0x563e35b13080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.251+0000 7f16ddb88640 1 --2- 172.21.15.81:0/2004944930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35aa9c00 0x563e35b13080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.251+0000 7f16ddb88640 1 -- 172.21.15.81:0/2004944930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35aa9800 msgr2=0x563e35b12b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.251+0000 7f16ddb88640 1 --2- 172.21.15.81:0/2004944930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35aa9800 0x563e35b12b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.251+0000 7f16ddb88640 1 -- 172.21.15.81:0/2004944930 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e35ace9c0 con 0x563e35aa8c00 2023-11-08T16:34:50.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.251+0000 7f16de389640 1 --2- 172.21.15.81:0/2004944930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35aa9c00 0x563e35b13080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:34:50.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.250+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:5 8.20 8:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddc700 con 0x5636e2dd8400 2023-11-08T16:34:50.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.251+0000 7f16ddb88640 1 --2- 172.21.15.81:0/2004944930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35aa8c00 0x563e35b12580 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x563e368b1410 tx=0x563e35d9a540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb0dd1c1845869ad server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.251+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(5 1000000020b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2dd8400 2023-11-08T16:34:50.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.251+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628299.0 2023-11-08T16:34:50.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.251+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:6 8.20 8:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddca80 con 0x5636e2dd8400 2023-11-08T16:34:50.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.251+0000 7f16db383640 1 -- 172.21.15.81:0/2004944930 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e35aef1e0 con 0x563e35aa8c00 2023-11-08T16:34:50.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.252+0000 7f16db383640 1 -- 172.21.15.81:0/2004944930 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563e36874180 con 0x563e35aa8c00 2023-11-08T16:34:50.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.252+0000 7f16db383640 1 -- 172.21.15.81:0/2004944930 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e35aef080 con 0x563e35aa8c00 2023-11-08T16:34:50.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.252+0000 7f16dfc385c0 1 -- 172.21.15.81:0/2004944930 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e35aceb60 con 0x563e35aa8c00 2023-11-08T16:34:50.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.252+0000 7f16dfc385c0 1 -- 172.21.15.81:0/2004944930 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x563e35ace820 con 0x563e35aa8c00 2023-11-08T16:34:50.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.252+0000 7f16dfc385c0 4 waiting for MDS map... 2023-11-08T16:34:50.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.252+0000 7f16db383640 1 -- 172.21.15.81:0/2004944930 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x563e35b68c40 con 0x563e35aa8c00 2023-11-08T16:34:50.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.252+0000 7f16db383640 1 -- 172.21.15.81:0/2004944930 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x563e35af8f00 con 0x563e35aa8c00 2023-11-08T16:34:50.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.253+0000 7f16dfc385c0 4 Got MDS map 47 2023-11-08T16:34:50.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.253+0000 7f16dfc385c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:50.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.253+0000 7f16dfc385c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:50.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.254+0000 7f16dfc385c0 1 Processor -- start 2023-11-08T16:34:50.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.254+0000 7f16dfc385c0 1 -- start start 2023-11-08T16:34:50.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.254+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(6 1000000020b.00000000 [call] v64'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2dd8400 2023-11-08T16:34:50.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.254+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5636e2dd8800 0x5636e2acb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.255+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:7 8.10 8:08000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddc380 con 0x5636e2dd8800 2023-11-08T16:34:50.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.255+0000 7fe7055bf640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5636e2dd8800 0x5636e2acb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.256+0000 7f16dfc385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35db7000 0x563e35b13600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.256+0000 7f16dfc385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35db7400 0x563e35b13b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.256+0000 7f16dfc385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35db7800 0x563e35b14100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.256+0000 7f16dfc385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35af0140 con 0x563e35db7400 2023-11-08T16:34:50.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.256+0000 7f16dfc385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35af0b40 con 0x563e35db7000 2023-11-08T16:34:50.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.256+0000 7f16dfc385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e35b77180 con 0x563e35db7800 2023-11-08T16:34:50.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.257+0000 7fe7055bf640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5636e2dd8800 0x5636e2acb600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.257+0000 7f16de389640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35db7000 0x563e35b13600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.257+0000 7f16de389640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35db7000 0x563e35b13600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:51770/0 (socket says 172.21.15.81:51770) 2023-11-08T16:34:50.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.257+0000 7f16de389640 1 -- 172.21.15.81:0/1978548170 learned_addr learned my addr 172.21.15.81:0/1978548170 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.257+0000 7f16ddb88640 1 --2- 172.21.15.81:0/1978548170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35db7400 0x563e35b13b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.257+0000 7f16dd387640 1 --2- 172.21.15.81:0/1978548170 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35db7800 0x563e35b14100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.258+0000 7f16ddb88640 1 -- 172.21.15.81:0/1978548170 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35db7800 msgr2=0x563e35b14100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.258+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(7 [pgnls start_epoch 64 out=81b] v62'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8800 2023-11-08T16:34:50.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.258+0000 7f16ddb88640 1 --2- 172.21.15.81:0/1978548170 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35db7800 0x563e35b14100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.258+0000 7f16ddb88640 1 -- 172.21.15.81:0/1978548170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35db7000 msgr2=0x563e35b13600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.258+0000 7f16ddb88640 1 --2- 172.21.15.81:0/1978548170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35db7000 0x563e35b13600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.258+0000 7f16ddb88640 1 -- 172.21.15.81:0/1978548170 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e35ace4e0 con 0x563e35db7400 2023-11-08T16:34:50.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.258+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:8 8.10 8:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddc700 con 0x5636e2dd8800 2023-11-08T16:34:50.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.258+0000 7f16ddb88640 1 --2- 172.21.15.81:0/1978548170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35db7400 0x563e35b13b80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x563e368b17a0 tx=0x563e35d9a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ab0476fcb88640 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.259+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(8 100000001fd.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8800 2023-11-08T16:34:50.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.259+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628285.0 2023-11-08T16:34:50.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.259+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:9 8.10 8:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddc000 con 0x5636e2dd8800 2023-11-08T16:34:50.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.259+0000 7f16d937f640 1 -- 172.21.15.81:0/1978548170 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e35aeedc0 con 0x563e35db7400 2023-11-08T16:34:50.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.259+0000 7f16d937f640 1 -- 172.21.15.81:0/1978548170 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563e36874c00 con 0x563e35db7400 2023-11-08T16:34:50.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.259+0000 7f16d937f640 1 -- 172.21.15.81:0/1978548170 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e35aef1e0 con 0x563e35db7400 2023-11-08T16:34:50.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.260+0000 7f16dfc385c0 1 -- 172.21.15.81:0/1978548170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35db7400 msgr2=0x563e35b13b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.260+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/1978548170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35db7400 0x563e35b13b80 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x563e368b17a0 tx=0x563e35d9a060 comp rx=0 tx=0).stop 2023-11-08T16:34:50.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.260+0000 7f48215855c0 1 -- start start 2023-11-08T16:34:50.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.260+0000 7f16dfc385c0 1 -- 172.21.15.81:0/1978548170 shutdown_connections 2023-11-08T16:34:50.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.260+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/1978548170 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35db7800 0x563e35b14100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.260+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/1978548170 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35db7400 0x563e35b13b80 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.260+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/1978548170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35db7000 0x563e35b13600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.260+0000 7f16dfc385c0 1 -- 172.21.15.81:0/1978548170 >> 172.21.15.81:0/1978548170 conn(0x563e35db6c00 msgr2=0x563e36830800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.260+0000 7f48215855c0 1 Processor -- start 2023-11-08T16:34:50.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.260+0000 7f16dfc385c0 1 -- 172.21.15.81:0/1978548170 shutdown_connections 2023-11-08T16:34:50.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.260+0000 7f16dfc385c0 1 -- 172.21.15.81:0/1978548170 wait complete. 2023-11-08T16:34:50.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.261+0000 7f16dfc385c0 1 Processor -- start 2023-11-08T16:34:50.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.262+0000 7f48215855c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564584276c00 0x5645842e0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.262+0000 7f48215855c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564584277800 0x5645842e0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.262+0000 7f48215855c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564584277c00 0x5645842e1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.262+0000 7f48215855c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645842be140 con 0x564584277c00 2023-11-08T16:34:50.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.262+0000 7f48215855c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645842bf180 con 0x564584277800 2023-11-08T16:34:50.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.262+0000 7f48215855c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645842be000 con 0x564584276c00 2023-11-08T16:34:50.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.262+0000 7f16dfc385c0 1 -- start start 2023-11-08T16:34:50.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.262+0000 7f482052c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564584277c00 0x5645842e1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.262+0000 7f482052c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564584277c00 0x5645842e1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48178/0 (socket says 172.21.15.81:48178) 2023-11-08T16:34:50.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.262+0000 7f482052c640 1 -- 172.21.15.81:0/3206251737 learned_addr learned my addr 172.21.15.81:0/3206251737 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.262+0000 7f481fd2b640 1 --2- 172.21.15.81:0/3206251737 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564584276c00 0x5645842e0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.263+0000 7f481f52a640 1 --2- 172.21.15.81:0/3206251737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564584277800 0x5645842e0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.263+0000 7f16dfc385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35db7000 0x563e35b15180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.263+0000 7f16dfc385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35db7400 0x563e35b14c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.263+0000 7f16dfc385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35db7800 0x563e35b14680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.263+0000 7f16dfc385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35b76c80 con 0x563e35db7000 2023-11-08T16:34:50.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.263+0000 7f16dfc385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e35b76b40 con 0x563e35db7800 2023-11-08T16:34:50.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.263+0000 7f16dfc385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e35b76a00 con 0x563e35db7400 2023-11-08T16:34:50.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.263+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(9 100000001fd.00000000 [call] v64'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8800 2023-11-08T16:34:50.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.263+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5636e2dd8c00 0x5636e2dee000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.263+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:10 8.30 8:0c000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ab9880 con 0x5636e2dd8c00 2023-11-08T16:34:50.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.263+0000 7f16de389640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35db7000 0x563e35b15180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7f482052c640 1 -- 172.21.15.81:0/3206251737 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564584276c00 msgr2=0x5645842e0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7f482052c640 1 --2- 172.21.15.81:0/3206251737 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564584276c00 0x5645842e0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7f16de389640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35db7000 0x563e35b15180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48192/0 (socket says 172.21.15.81:48192) 2023-11-08T16:34:50.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7f482052c640 1 -- 172.21.15.81:0/3206251737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564584277800 msgr2=0x5645842e0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7f482052c640 1 --2- 172.21.15.81:0/3206251737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564584277800 0x5645842e0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7f482052c640 1 -- 172.21.15.81:0/3206251737 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56458429c9c0 con 0x564584277c00 2023-11-08T16:34:50.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7f481f52a640 1 --2- 172.21.15.81:0/3206251737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564584277800 0x5645842e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:34:50.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7fe704dbe640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5636e2dd8c00 0x5636e2dee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7f16de389640 1 -- 172.21.15.81:0/439004518 learned_addr learned my addr 172.21.15.81:0/439004518 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7f481fd2b640 1 --2- 172.21.15.81:0/3206251737 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564584276c00 0x5645842e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:34:50.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7f16ddb88640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35db7400 0x563e35b14c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.264+0000 7f16dd387640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35db7800 0x563e35b14680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35db7800 msgr2=0x563e35b14680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7f16ddb88640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35db7800 0x563e35b14680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35db7000 msgr2=0x563e35b15180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7f16ddb88640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35db7000 0x563e35b15180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e35ace9c0 con 0x563e35db7400 2023-11-08T16:34:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7fe704dbe640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5636e2dd8c00 0x5636e2dee000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7f6ebbe175c0 1 -- start start 2023-11-08T16:34:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7f6ebbe175c0 1 Processor -- start 2023-11-08T16:34:50.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(10 [pgnls start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7fe700db6640 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:11 8.8 8:10000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddce00 con 0x5636e2dd8400 2023-11-08T16:34:50.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7f482052c640 1 --2- 172.21.15.81:0/3206251737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564584277c00 0x5645842e1080 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x564585091830 tx=0x564585052600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13c84941358c15c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.265+0000 7f16ddb88640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35db7400 0x563e35b14c00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x563e35daf710 tx=0x563e3687cc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.266+0000 7f481d526640 1 -- 172.21.15.81:0/3206251737 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645842bd1e0 con 0x564584277c00 2023-11-08T16:34:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.266+0000 7f481d526640 1 -- 172.21.15.81:0/3206251737 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645842d0480 con 0x564584277c00 2023-11-08T16:34:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.266+0000 7f481d526640 1 -- 172.21.15.81:0/3206251737 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645842bd080 con 0x564584277c00 2023-11-08T16:34:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.266+0000 7f48215855c0 1 -- 172.21.15.81:0/3206251737 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56458429cb60 con 0x564584277c00 2023-11-08T16:34:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.266+0000 7f48215855c0 1 -- 172.21.15.81:0/3206251737 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56458429c820 con 0x564584277c00 2023-11-08T16:34:50.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.266+0000 7f48215855c0 4 waiting for MDS map... 2023-11-08T16:34:50.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.266+0000 7f16d7b7c640 1 -- 172.21.15.81:0/439004518 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e35aef080 con 0x563e35db7400 2023-11-08T16:34:50.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.266+0000 7f16d7b7c640 1 -- 172.21.15.81:0/439004518 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563e36874d80 con 0x563e35db7400 2023-11-08T16:34:50.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.266+0000 7f16d7b7c640 1 -- 172.21.15.81:0/439004518 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x563e35aee420 con 0x563e35db7400 2023-11-08T16:34:50.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563e35aceb60 con 0x563e35db7400 2023-11-08T16:34:50.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f481d526640 1 -- 172.21.15.81:0/3206251737 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x564584336c40 con 0x564584277c00 2023-11-08T16:34:50.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e35ace820 con 0x563e35db7400 2023-11-08T16:34:50.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f16dfc385c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:50.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(11 [pgnls start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2dd8400 2023-11-08T16:34:50.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f481d526640 1 -- 172.21.15.81:0/3206251737 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x5645842c6f00 con 0x564584277c00 2023-11-08T16:34:50.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f48215855c0 4 Got MDS map 47 2023-11-08T16:34:50.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7fe7005b5640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5636e2df4000 0x5636e2df0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7fe7005b5640 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:12 8.28 8:14000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddd180 con 0x5636e2df4000 2023-11-08T16:34:50.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f48215855c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:50.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f48215855c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:50.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f6ebbe175c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b9892ac00 0x562b98994580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f6ebbe175c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b9892b800 0x562b98994b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f6ebbe175c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b9892bc00 0x562b98995080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f6ebbe175c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b98973180 con 0x562b9892ac00 2023-11-08T16:34:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f6ebbe175c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b989f9180 con 0x562b9892bc00 2023-11-08T16:34:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.267+0000 7f6ebbe175c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b989f8dc0 con 0x562b9892b800 2023-11-08T16:34:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.268+0000 7fea58a395c0 1 -- start start 2023-11-08T16:34:50.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.268+0000 7fe705dc0640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5636e2df4000 0x5636e2df0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.268+0000 7fea58a395c0 1 Processor -- start 2023-11-08T16:34:50.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.268+0000 7f6eb9a1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b9892ac00 0x562b98994580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.268+0000 7f6eba21b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b9892bc00 0x562b98995080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.268+0000 7f6eb9a1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b9892ac00 0x562b98994580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48202/0 (socket says 172.21.15.81:48202) 2023-11-08T16:34:50.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.268+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1941472936 learned_addr learned my addr 172.21.15.81:0/1941472936 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1941472936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b9892b800 msgr2=0x562b98994b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7fe705dc0640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5636e2df4000 0x5636e2df0680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7f48215855c0 1 Processor -- start 2023-11-08T16:34:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/1941472936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b9892b800 0x562b98994b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1941472936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b9892bc00 msgr2=0x562b98995080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/1941472936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b9892bc00 0x562b98995080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1941472936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b989509c0 con 0x562b9892ac00 2023-11-08T16:34:50.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7f48215855c0 1 -- start start 2023-11-08T16:34:50.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/1941472936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b9892ac00 0x562b98994580 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x562b99737830 tx=0x562b99702300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf25d996bf4c4df9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(12 [pgnls start_epoch 64 out=101b] v62'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2df4000 2023-11-08T16:34:50.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7fea58a395c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf151cc00 0x55fdf1586580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7fea58a395c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf151d800 0x55fdf1586b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:13 8.28 8:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ab9c00 con 0x5636e2df4000 2023-11-08T16:34:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7f6eb7215640 1 -- 172.21.15.81:0/1941472936 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562b989711e0 con 0x562b9892ac00 2023-11-08T16:34:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7f6eb7215640 1 -- 172.21.15.81:0/1941472936 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b9974aa80 con 0x562b9892ac00 2023-11-08T16:34:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7f6eb7215640 1 -- 172.21.15.81:0/1941472936 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562b98971080 con 0x562b9892ac00 2023-11-08T16:34:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7fea58a395c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf151dc00 0x55fdf1587080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7fea58a395c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf1565180 con 0x55fdf151dc00 2023-11-08T16:34:50.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7fea58a395c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf1564140 con 0x55fdf151cc00 2023-11-08T16:34:50.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.269+0000 7fea58a395c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdf1564000 con 0x55fdf151d800 2023-11-08T16:34:50.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(13 10000000202.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2df4000 2023-11-08T16:34:50.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628290.0 2023-11-08T16:34:50.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:14 8.28 8:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddca80 con 0x5636e2df4000 2023-11-08T16:34:50.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7f48215855c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458514ec00 0x5645842e1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7f48215855c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56458514f000 0x5645842e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7f48215855c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f400 0x5645842e2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7f48215855c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645842bea00 con 0x56458514ec00 2023-11-08T16:34:50.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7f48215855c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564584345180 con 0x56458514f400 2023-11-08T16:34:50.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.270+0000 7f48215855c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x564584344dc0 con 0x56458514f000 2023-11-08T16:34:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.271+0000 7f16d7b7c640 1 -- 172.21.15.81:0/439004518 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x563e35b04840 con 0x563e35db7400 2023-11-08T16:34:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.271+0000 7f481f52a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f400 0x5645842e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.271+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1941472936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b98950b60 con 0x562b9892ac00 2023-11-08T16:34:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.271+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1941472936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562b98950820 con 0x562b9892ac00 2023-11-08T16:34:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.271+0000 7f6ebbe175c0 4 waiting for MDS map... 2023-11-08T16:34:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.271+0000 7fea56133640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf151d800 0x55fdf1586b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.271+0000 7fea57135640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf151dc00 0x55fdf1587080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.271+0000 7fea56934640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf151cc00 0x55fdf1586580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.271+0000 7fea57135640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf151dc00 0x55fdf1587080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48214/0 (socket says 172.21.15.81:48214) 2023-11-08T16:34:50.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.271+0000 7fea57135640 1 -- 172.21.15.81:0/2991698649 learned_addr learned my addr 172.21.15.81:0/2991698649 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7f481f52a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f400 0x5645842e2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:51850/0 (socket says 172.21.15.81:51850) 2023-11-08T16:34:50.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7f481f52a640 1 -- 172.21.15.81:0/1374579142 learned_addr learned my addr 172.21.15.81:0/1374579142 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7f481fd2b640 1 --2- 172.21.15.81:0/1374579142 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56458514f000 0x5645842e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7f481f52a640 1 --2- 172.21.15.81:0/1374579142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458514ec00 0x5645842e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7fe900c425c0 1 -- start start 2023-11-08T16:34:50.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7fe900c425c0 1 Processor -- start 2023-11-08T16:34:50.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7fea56934640 1 -- 172.21.15.81:0/2991698649 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf151d800 msgr2=0x55fdf1586b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7fea56934640 1 --2- 172.21.15.81:0/2991698649 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf151d800 0x55fdf1586b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7fea56934640 1 -- 172.21.15.81:0/2991698649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf151dc00 msgr2=0x55fdf1587080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7fea56934640 1 --2- 172.21.15.81:0/2991698649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf151dc00 0x55fdf1587080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7fea56934640 1 -- 172.21.15.81:0/2991698649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdf1542d00 con 0x55fdf151cc00 2023-11-08T16:34:50.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7fea56934640 1 --2- 172.21.15.81:0/2991698649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf151cc00 0x55fdf1586580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55fdf2323380 tx=0x55fdf1808420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=781f0ff81e284925 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f481f52a640 1 -- 172.21.15.81:0/1374579142 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56458514f000 msgr2=0x5645842e1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f481f52a640 1 --2- 172.21.15.81:0/1374579142 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56458514f000 0x5645842e1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f481f52a640 1 -- 172.21.15.81:0/1374579142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458514ec00 msgr2=0x5645842e1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f481f52a640 1 --2- 172.21.15.81:0/1374579142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458514ec00 0x5645842e1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f481f52a640 1 -- 172.21.15.81:0/1374579142 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56458429c4e0 con 0x56458514f400 2023-11-08T16:34:50.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f481fd2b640 1 --2- 172.21.15.81:0/1374579142 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56458514f000 0x5645842e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:34:50.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.271+0000 7f6eb7215640 1 -- 172.21.15.81:0/1941472936 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x562b989eac40 con 0x562b9892ac00 2023-11-08T16:34:50.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7f6eb7215640 1 -- 172.21.15.81:0/1941472936 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x562b9897af00 con 0x562b9892ac00 2023-11-08T16:34:50.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7f6ebbe175c0 4 Got MDS map 47 2023-11-08T16:34:50.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7f6ebbe175c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:50.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.272+0000 7f6ebbe175c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:50.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7fea5412f640 1 -- 172.21.15.81:0/2991698649 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fdf15631e0 con 0x55fdf151cc00 2023-11-08T16:34:50.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7fea5412f640 1 -- 172.21.15.81:0/2991698649 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fdf1576180 con 0x55fdf151cc00 2023-11-08T16:34:50.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7fea5412f640 1 -- 172.21.15.81:0/2991698649 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fdf1563080 con 0x55fdf151cc00 2023-11-08T16:34:50.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f6ebbe175c0 1 Processor -- start 2023-11-08T16:34:50.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f16d7b7c640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x563e35f0e000 0x563e35b15700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f16d7b7c640 1 -- 172.21.15.81:0/439004518 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x563e35b68a80 con 0x563e35db7400 2023-11-08T16:34:50.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f16dfc385c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:50.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f16dfc385c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:50.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f16dfc385c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:50.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f16dfc385c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 9 2023-11-08T16:34:50.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f16dfc385c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-11-08T16:34:50.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x563e35f0e400 0x563e35b14100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:1 8.24 8:24924924::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35b01880 con 0x563e35f0e400 2023-11-08T16:34:50.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7fea58a395c0 1 -- 172.21.15.81:0/2991698649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdf15429c0 con 0x55fdf151cc00 2023-11-08T16:34:50.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7fea58a395c0 1 -- 172.21.15.81:0/2991698649 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55fdf1542b60 con 0x55fdf151cc00 2023-11-08T16:34:50.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7fea58a395c0 4 waiting for MDS map... 2023-11-08T16:34:50.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fe900c425c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1708eec00 0x55e170958580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fe900c425c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1708ef800 0x55e170958b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fe900c425c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1708efc00 0x55e170959080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fe900c425c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e170937180 con 0x55e1708ef800 2023-11-08T16:34:50.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fe900c425c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1709bd180 con 0x55e1708eec00 2023-11-08T16:34:50.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fe900c425c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1709bcdc0 con 0x55e1708efc00 2023-11-08T16:34:50.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7f16de389640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x563e35f0e000 0x563e35b15700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fea5412f640 1 -- 172.21.15.81:0/2991698649 <== mon.1 v2:172.21.15.93:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55fdf15dcc40 con 0x55fdf151cc00 2023-11-08T16:34:50.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fea5412f640 1 -- 172.21.15.81:0/2991698649 <== mon.1 v2:172.21.15.93:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55fdf156cf00 con 0x55fdf151cc00 2023-11-08T16:34:50.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fea58a395c0 4 Got MDS map 47 2023-11-08T16:34:50.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fea58a395c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:50.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fea58a395c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:50.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.274+0000 7fe8feb4c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1708eec00 0x55e170958580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f16dd387640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x563e35f0e400 0x563e35b14100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f16de389640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x563e35f0e000 0x563e35b15700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x563e35daf320 tx=0x563e35d9a2a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f481f52a640 1 --2- 172.21.15.81:0/1374579142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f400 0x5645842e2100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x564585090ae0 tx=0x564585052480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fff32bf5ebcd6fc9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.273+0000 7f6ebbe175c0 1 -- start start 2023-11-08T16:34:50.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f6ebbe175c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7000 0x562b98995600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f6ebbe175c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b997f7400 0x562b98995b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f6ebbe175c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b997f7800 0x562b98996100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f6ebbe175c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b98972a00 con 0x562b997f7000 2023-11-08T16:34:50.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f6ebbe175c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b98972b40 con 0x562b997f7400 2023-11-08T16:34:50.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f6ebbe175c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b989f9180 con 0x562b997f7800 2023-11-08T16:34:50.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f6eba21b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7000 0x562b98995600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f6eba21b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7000 0x562b98995600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48240/0 (socket says 172.21.15.81:48240) 2023-11-08T16:34:50.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f6eba21b640 1 -- 172.21.15.81:0/3785527498 learned_addr learned my addr 172.21.15.81:0/3785527498 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7fea58a395c0 1 Processor -- start 2023-11-08T16:34:50.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(14 10000000202.00000000 [call] v64'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2df4000 2023-11-08T16:34:50.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:15 8.28 8:177a762d::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddce00 con 0x5636e2df4000 2023-11-08T16:34:50.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7f16dd387640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x563e35f0e400 0x563e35b14100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7f481b522640 1 -- 172.21.15.81:0/1374579142 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645842bcf20 con 0x56458514f400 2023-11-08T16:34:50.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7f481b522640 1 -- 172.21.15.81:0/1374579142 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564585061080 con 0x56458514f400 2023-11-08T16:34:50.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7f481b522640 1 -- 172.21.15.81:0/1374579142 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645842bd1e0 con 0x56458514f400 2023-11-08T16:34:50.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7f6eb9219640 1 --2- 172.21.15.81:0/3785527498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b997f7800 0x562b98996100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7fea58a395c0 1 -- start start 2023-11-08T16:34:50.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7fe8feb4c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1708eec00 0x55e170958580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:51854/0 (socket says 172.21.15.81:51854) 2023-11-08T16:34:50.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7fe8feb4c640 1 -- 172.21.15.81:0/3787170624 learned_addr learned my addr 172.21.15.81:0/3787170624 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/3787170624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1708ef800 0x55e170958b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.275+0000 7fe8ff34d640 1 --2- 172.21.15.81:0/3787170624 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1708efc00 0x55e170959080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7fe8fe34b640 1 -- 172.21.15.81:0/3787170624 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1708efc00 msgr2=0x55e170959080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/3787170624 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1708efc00 0x55e170959080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7fe8fe34b640 1 -- 172.21.15.81:0/3787170624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1708eec00 msgr2=0x55e170958580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/3787170624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1708eec00 0x55e170958580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7fe8fe34b640 1 -- 172.21.15.81:0/3787170624 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1709149c0 con 0x55e1708ef800 2023-11-08T16:34:50.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(15 [pgnls start_epoch 64 out=81b] v64'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2df4000 2023-11-08T16:34:50.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/3785527498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b997f7400 0x562b98995b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7f6eba21b640 1 -- 172.21.15.81:0/3785527498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b997f7800 msgr2=0x562b98996100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:16 8.28 8:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddd180 con 0x5636e2df4000 2023-11-08T16:34:50.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7f6eba21b640 1 --2- 172.21.15.81:0/3785527498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b997f7800 0x562b98996100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7f6eba21b640 1 -- 172.21.15.81:0/3785527498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b997f7400 msgr2=0x562b98995b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7f6eba21b640 1 --2- 172.21.15.81:0/3785527498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b997f7400 0x562b98995b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7f6eba21b640 1 -- 172.21.15.81:0/3785527498 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b989504e0 con 0x562b997f7000 2023-11-08T16:34:50.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.276+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/3787170624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1708ef800 0x55e170958b00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x55e1717057a0 tx=0x55e1716c6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd83065ed44fd5b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/3785527498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b997f7400 0x562b98995b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:50.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7fea58a395c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf181d000 0x55fdf1587600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7fea58a395c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf181d400 0x55fdf1587b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7fea58a395c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf181d800 0x55fdf1588100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7fea58a395c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf1564a00 con 0x55fdf181d400 2023-11-08T16:34:50.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7fea58a395c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf15eb180 con 0x55fdf181d800 2023-11-08T16:34:50.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7fea58a395c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdf15eadc0 con 0x55fdf181d000 2023-11-08T16:34:50.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(16 10000000201.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2df4000 2023-11-08T16:34:50.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628289.0 2023-11-08T16:34:50.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:17 8.28 8:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddd500 con 0x5636e2df4000 2023-11-08T16:34:50.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(1 [pgnls start_epoch 64 out=49b] v62'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f0e400 2023-11-08T16:34:50.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7f16d8b7e640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x563e35f0f800 0x563e35f20000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7f16d8b7e640 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:2 8.14 8:28000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98000 con 0x563e35f0f800 2023-11-08T16:34:50.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7f48215855c0 1 -- 172.21.15.81:0/1374579142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f400 msgr2=0x5645842e2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7f48215855c0 1 --2- 172.21.15.81:0/1374579142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f400 0x5645842e2100 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x564585090ae0 tx=0x564585052480 comp rx=0 tx=0).stop 2023-11-08T16:34:50.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7f6eba21b640 1 --2- 172.21.15.81:0/3785527498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7000 0x562b98995600 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x562b99802c90 tx=0x562b99808f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205859508f3ea73f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7fe8fc347640 1 -- 172.21.15.81:0/3787170624 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e1709351e0 con 0x55e1708ef800 2023-11-08T16:34:50.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7fe8fc347640 1 -- 172.21.15.81:0/3787170624 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e170948480 con 0x55e1708ef800 2023-11-08T16:34:50.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7fe8fc347640 1 -- 172.21.15.81:0/3787170624 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e170935080 con 0x55e1708ef800 2023-11-08T16:34:50.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7fe900c425c0 1 -- 172.21.15.81:0/3787170624 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e170914b60 con 0x55e1708ef800 2023-11-08T16:34:50.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7fe900c425c0 1 -- 172.21.15.81:0/3787170624 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55e170914820 con 0x55e1708ef800 2023-11-08T16:34:50.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7fe900c425c0 4 waiting for MDS map... 2023-11-08T16:34:50.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7f48215855c0 1 -- 172.21.15.81:0/1374579142 shutdown_connections 2023-11-08T16:34:50.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7f6eb5211640 1 -- 172.21.15.81:0/3785527498 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562b989711e0 con 0x562b997f7000 2023-11-08T16:34:50.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7f6eb5211640 1 -- 172.21.15.81:0/3785527498 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b9974b800 con 0x562b997f7000 2023-11-08T16:34:50.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7f6eb5211640 1 -- 172.21.15.81:0/3785527498 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562b98971080 con 0x562b997f7000 2023-11-08T16:34:50.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7f48215855c0 1 --2- 172.21.15.81:0/1374579142 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f400 0x5645842e2100 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7f48215855c0 1 --2- 172.21.15.81:0/1374579142 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56458514f000 0x5645842e1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7f48215855c0 1 --2- 172.21.15.81:0/1374579142 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458514ec00 0x5645842e1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.277+0000 7fea56934640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf181d400 0x55fdf1587b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7fea57135640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf181d000 0x55fdf1587600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7fea57135640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf181d000 0x55fdf1587600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:49922/0 (socket says 172.21.15.81:49922) 2023-11-08T16:34:50.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7fea57135640 1 -- 172.21.15.81:0/2578796060 learned_addr learned my addr 172.21.15.81:0/2578796060 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7fea56133640 1 --2- 172.21.15.81:0/2578796060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf181d800 0x55fdf1588100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7f48215855c0 1 -- 172.21.15.81:0/1374579142 >> 172.21.15.81:0/1374579142 conn(0x56458514e800 msgr2=0x564584373800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.278+0000 7fea57135640 1 -- 172.21.15.81:0/2578796060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf181d800 msgr2=0x55fdf1588100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7f16de389640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x563e35f0f800 0x563e35f20000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7fea57135640 1 --2- 172.21.15.81:0/2578796060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf181d800 0x55fdf1588100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7fea57135640 1 -- 172.21.15.81:0/2578796060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf181d400 msgr2=0x55fdf1587b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7fea57135640 1 --2- 172.21.15.81:0/2578796060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf181d400 0x55fdf1587b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7fea57135640 1 -- 172.21.15.81:0/2578796060 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdf1542820 con 0x55fdf181d000 2023-11-08T16:34:50.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/3785527498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7000 msgr2=0x562b98995600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/3785527498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7000 0x562b98995600 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x562b99802c90 tx=0x562b99808f00 comp rx=0 tx=0).stop 2023-11-08T16:34:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7f48215855c0 1 -- 172.21.15.81:0/1374579142 shutdown_connections 2023-11-08T16:34:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/3785527498 shutdown_connections 2023-11-08T16:34:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/3785527498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b997f7800 0x562b98996100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/3785527498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b997f7400 0x562b98995b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7fe8fc347640 1 -- 172.21.15.81:0/3787170624 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55e1709aec40 con 0x55e1708ef800 2023-11-08T16:34:50.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7fe8fc347640 1 -- 172.21.15.81:0/3787170624 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55e1717cc000 con 0x55e1708ef800 2023-11-08T16:34:50.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7fe900c425c0 4 Got MDS map 47 2023-11-08T16:34:50.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7f1915c0c5c0 1 -- start start 2023-11-08T16:34:50.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(17 10000000201.00000000 [call] v64'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2df4000 2023-11-08T16:34:50.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/3785527498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7000 0x562b98995600 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7f1915c0c5c0 1 Processor -- start 2023-11-08T16:34:50.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/3785527498 >> 172.21.15.81:0/3785527498 conn(0x562b997f6c00 msgr2=0x562b996a5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7fe900c425c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:50.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.279+0000 7fe900c425c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:50.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5636e2dd9000 0x5636e2dee580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:18 8.18 8:18000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddd880 con 0x5636e2dd9000 2023-11-08T16:34:50.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7f48215855c0 1 -- 172.21.15.81:0/1374579142 wait complete. 2023-11-08T16:34:50.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7fea57135640 1 --2- 172.21.15.81:0/2578796060 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf181d000 0x55fdf1587600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55fdf22ee780 tx=0x55fdf1808120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14ccb693449a9820 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7f16de389640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x563e35f0f800 0x563e35f20000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/3785527498 shutdown_connections 2023-11-08T16:34:50.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/3785527498 wait complete. 2023-11-08T16:34:50.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7fe900c425c0 1 Processor -- start 2023-11-08T16:34:50.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7fea5212b640 1 -- 172.21.15.81:0/2578796060 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fdf1562f20 con 0x55fdf181d000 2023-11-08T16:34:50.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.281+0000 7fe900c425c0 1 -- start start 2023-11-08T16:34:50.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.281+0000 7f48215855c0 1 Processor -- start 2023-11-08T16:34:50.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.280+0000 7fe7055bf640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5636e2dd9000 0x5636e2dee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.281+0000 7f16de389640 1 -- 172.21.15.81:0/439004518 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(2 [pgnls start_epoch 64 out=49b] v62'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563e35d86d80 con 0x563e35f0f800 2023-11-08T16:34:50.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.281+0000 7f16d937f640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x563e35f0fc00 0x563e35f20580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.281+0000 7fea5212b640 1 -- 172.21.15.81:0/2578796060 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fdf22f5200 con 0x55fdf181d000 2023-11-08T16:34:50.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.281+0000 7fea5212b640 1 -- 172.21.15.81:0/2578796060 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fdf15631e0 con 0x55fdf181d000 2023-11-08T16:34:50.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.281+0000 7f16d937f640 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 8.34 8:2c000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98380 con 0x563e35f0fc00 2023-11-08T16:34:50.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.281+0000 7f48215855c0 1 -- start start 2023-11-08T16:34:50.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.281+0000 7f16ddb88640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x563e35f0fc00 0x563e35f20580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.281+0000 7fe7055bf640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5636e2dd9000 0x5636e2dee580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fea58a395c0 1 -- 172.21.15.81:0/2578796060 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf181d000 msgr2=0x55fdf1587600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fea58a395c0 1 --2- 172.21.15.81:0/2578796060 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf181d000 0x55fdf1587600 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55fdf22ee780 tx=0x55fdf1808120 comp rx=0 tx=0).stop 2023-11-08T16:34:50.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f6ebbe175c0 1 Processor -- start 2023-11-08T16:34:50.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f1915c0c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e41e0c00 0x5557e424a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f1915c0c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e41e1800 0x5557e424ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f1915c0c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e41e1c00 0x5557e424b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f1915c0c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e4229180 con 0x5557e41e1c00 2023-11-08T16:34:50.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f1915c0c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e4228140 con 0x5557e41e0c00 2023-11-08T16:34:50.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f1915c0c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557e4228000 con 0x5557e41e1800 2023-11-08T16:34:50.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fe900c425c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1717e5000 0x55e170959600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fea58a395c0 1 -- 172.21.15.81:0/2578796060 shutdown_connections 2023-11-08T16:34:50.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fea58a395c0 1 --2- 172.21.15.81:0/2578796060 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf181d800 0x55fdf1588100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fea58a395c0 1 --2- 172.21.15.81:0/2578796060 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf181d400 0x55fdf1587b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fea58a395c0 1 --2- 172.21.15.81:0/2578796060 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf181d000 0x55fdf1587600 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fea58a395c0 1 -- 172.21.15.81:0/2578796060 >> 172.21.15.81:0/2578796060 conn(0x55fdf181cc00 msgr2=0x55fdf22a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fe900c425c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5400 0x55e170959b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fe900c425c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1717e5800 0x55e17095a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fe900c425c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e170936a00 con 0x55e1717e5000 2023-11-08T16:34:50.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fe900c425c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e170936b40 con 0x55e1717e5800 2023-11-08T16:34:50.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fe900c425c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1709bd180 con 0x55e1717e5400 2023-11-08T16:34:50.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f1913f66640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e41e1c00 0x5557e424b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f1913f66640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e41e1c00 0x5557e424b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48270/0 (socket says 172.21.15.81:48270) 2023-11-08T16:34:50.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f1913f66640 1 -- 172.21.15.81:0/1618399180 learned_addr learned my addr 172.21.15.81:0/1618399180 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f16ddb88640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x563e35f0fc00 0x563e35f20580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f48215855c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458514ec00 0x5645842e3180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f48215855c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f000 0x5645842e2c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f48215855c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56458514f400 0x5645842e2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f48215855c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564584344c80 con 0x56458514ec00 2023-11-08T16:34:50.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f48215855c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5645842bf180 con 0x56458514f000 2023-11-08T16:34:50.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f48215855c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5645842beb40 con 0x56458514f400 2023-11-08T16:34:50.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7f1913765640 1 --2- 172.21.15.81:0/1618399180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e41e0c00 0x5557e424a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(18 [pgnls start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd9000 2023-11-08T16:34:50.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7f1912f64640 1 --2- 172.21.15.81:0/1618399180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e41e1800 0x5557e424ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(3 [pgnls start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563e35d86fc0 con 0x563e35f0fc00 2023-11-08T16:34:50.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fea58a395c0 1 -- 172.21.15.81:0/2578796060 shutdown_connections 2023-11-08T16:34:50.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7fea58a395c0 1 -- 172.21.15.81:0/2578796060 wait complete. 2023-11-08T16:34:50.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7fe700db6640 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:19 8.38 8:1c000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddc380 con 0x5636e2dd8000 2023-11-08T16:34:50.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7f1913f66640 1 -- 172.21.15.81:0/1618399180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e41e1800 msgr2=0x5557e424ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7f1913f66640 1 --2- 172.21.15.81:0/1618399180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e41e1800 0x5557e424ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7f1913f66640 1 -- 172.21.15.81:0/1618399180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e41e0c00 msgr2=0x5557e424a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7f1913f66640 1 --2- 172.21.15.81:0/1618399180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e41e0c00 0x5557e424a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7f1913f66640 1 -- 172.21.15.81:0/1618399180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557e42069c0 con 0x5557e41e1c00 2023-11-08T16:34:50.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7fe8fe34b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1717e5800 0x55e17095a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7fe8fe34b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1717e5800 0x55e17095a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:51880/0 (socket says 172.21.15.81:51880) 2023-11-08T16:34:50.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7fe8fe34b640 1 -- 172.21.15.81:0/3195071636 learned_addr learned my addr 172.21.15.81:0/3195071636 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.283+0000 7fe8feb4c640 1 --2- 172.21.15.81:0/3195071636 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5400 0x55e170959b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.282+0000 7f6ebbe175c0 1 -- start start 2023-11-08T16:34:50.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f481fd2b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f000 0x5645842e2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7fea58a395c0 1 Processor -- start 2023-11-08T16:34:50.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f16d8b7e640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x563e35f2e000 0x563e35f20b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f16d8b7e640 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:4 8.c 8:30000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98700 con 0x563e35f2e000 2023-11-08T16:34:50.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7fe8ff34d640 1 --2- 172.21.15.81:0/3195071636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1717e5000 0x55e170959600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(19 [pgnls start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7fe7005b5640 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:20 8.4 8:20000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddc700 con 0x5636e2dd8400 2023-11-08T16:34:50.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f481fd2b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f000 0x5645842e2c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:51884/0 (socket says 172.21.15.81:51884) 2023-11-08T16:34:50.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f481fd2b640 1 -- 172.21.15.81:0/4069498314 learned_addr learned my addr 172.21.15.81:0/4069498314 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f481f52a640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56458514f400 0x5645842e2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f481f52a640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458514ec00 0x5645842e3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f6ebbe175c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b997f7000 0x562b98997180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f6ebbe175c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7400 0x562b98996c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f6ebbe175c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b997f7800 0x562b98996680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f6ebbe175c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b989f8c80 con 0x562b997f7400 2023-11-08T16:34:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f6ebbe175c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b989f8b40 con 0x562b997f7000 2023-11-08T16:34:50.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f6ebbe175c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b989f8a00 con 0x562b997f7800 2023-11-08T16:34:50.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f1913f66640 1 --2- 172.21.15.81:0/1618399180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e41e1c00 0x5557e424b080 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x5557e4ff8b40 tx=0x5557e4fb0360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=872bd176fedcc798 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7fea58a395c0 1 -- start start 2023-11-08T16:34:50.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7fe8feb4c640 1 -- 172.21.15.81:0/3195071636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1717e5800 msgr2=0x55e17095a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7fe8feb4c640 1 --2- 172.21.15.81:0/3195071636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1717e5800 0x55e17095a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7fe8feb4c640 1 -- 172.21.15.81:0/3195071636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1717e5000 msgr2=0x55e170959600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7fe8feb4c640 1 --2- 172.21.15.81:0/3195071636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1717e5000 0x55e170959600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7fe8feb4c640 1 -- 172.21.15.81:0/3195071636 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1709144e0 con 0x55e1717e5400 2023-11-08T16:34:50.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/3195071636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1717e5800 0x55e17095a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:34:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f481fd2b640 1 -- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56458514f400 msgr2=0x5645842e2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f481fd2b640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56458514f400 0x5645842e2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f481fd2b640 1 -- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458514ec00 msgr2=0x5645842e3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f481fd2b640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56458514ec00 0x5645842e3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f481fd2b640 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56458429cd00 con 0x56458514f000 2023-11-08T16:34:50.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.284+0000 7f16dd387640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x563e35f2e000 0x563e35f20b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f1910f60640 1 -- 172.21.15.81:0/1618399180 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5557e4226f20 con 0x5557e41e1c00 2023-11-08T16:34:50.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f6eb9a1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7400 0x562b98996c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f6eb9a1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7400 0x562b98996c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48288/0 (socket says 172.21.15.81:48288) 2023-11-08T16:34:50.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1154556006 learned_addr learned my addr 172.21.15.81:0/1154556006 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7fe8feb4c640 1 --2- 172.21.15.81:0/3195071636 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5400 0x55e170959b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55e171705470 tx=0x55e1716c6240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=581784557f85a3e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(20 [pgnls start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2dd8400 2023-11-08T16:34:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7fe700db6640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5636e2df4400 0x5636e2df0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7fe700db6640 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:21 8.24 8:24000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ab9c00 con 0x5636e2df4400 2023-11-08T16:34:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f1910f60640 1 -- 172.21.15.81:0/1618399180 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5557e4fc8300 con 0x5557e41e1c00 2023-11-08T16:34:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f16dd387640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x563e35f2e000 0x563e35f20b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f1910f60640 1 -- 172.21.15.81:0/1618399180 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5557e42271e0 con 0x5557e41e1c00 2023-11-08T16:34:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f6eb9219640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b997f7800 0x562b98996680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f6eba21b640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b997f7000 0x562b98997180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1618399180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557e4206d00 con 0x5557e41e1c00 2023-11-08T16:34:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1618399180 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5557e42069c0 con 0x5557e41e1c00 2023-11-08T16:34:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7f1915c0c5c0 4 waiting for MDS map... 2023-11-08T16:34:50.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7fea58a395c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf181d000 0x55fdf1589180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7fea58a395c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf181d400 0x55fdf1588c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.285+0000 7fe8fa343640 1 -- 172.21.15.81:0/3195071636 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e170934f20 con 0x55e1717e5400 2023-11-08T16:34:50.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7fea58a395c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf181d800 0x55fdf1588680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7fea58a395c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf15eac80 con 0x55fdf181d800 2023-11-08T16:34:50.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7fe704dbe640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5636e2df4400 0x5636e2df0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7fea58a395c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdf1565180 con 0x55fdf181d400 2023-11-08T16:34:50.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7fea58a395c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdf1564b40 con 0x55fdf181d000 2023-11-08T16:34:50.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f1910f60640 1 -- 172.21.15.81:0/1618399180 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x5557e42a0c40 con 0x5557e41e1c00 2023-11-08T16:34:50.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(4 [pgnls start_epoch 64 out=81b] v62'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f2e000 2023-11-08T16:34:50.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 8.c 8:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35b01c00 con 0x563e35f2e000 2023-11-08T16:34:50.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7fe8fa343640 1 -- 172.21.15.81:0/3195071636 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1716d5b00 con 0x55e1717e5400 2023-11-08T16:34:50.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(5 100000001fc.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f2e000 2023-11-08T16:34:50.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f481fd2b640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f000 0x5645842e2c00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x56458515f620 tx=0x564585162c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f16dfc385c0 10 datascan.operator(): handling object 1099511628284.0 2023-11-08T16:34:50.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7fe8fa343640 1 -- 172.21.15.81:0/3195071636 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e1709351e0 con 0x55e1717e5400 2023-11-08T16:34:50.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:6 8.c 8:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98700 con 0x563e35f2e000 2023-11-08T16:34:50.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b997f7800 msgr2=0x562b98996680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b997f7800 0x562b98996680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b997f7000 msgr2=0x562b98997180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b997f7000 0x562b98997180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.286+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b989509c0 con 0x562b997f7400 2023-11-08T16:34:50.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7fea56133640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf181d800 0x55fdf1588680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7fea56934640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf181d400 0x55fdf1588c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7fea56934640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf181d400 0x55fdf1588c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:51900/0 (socket says 172.21.15.81:51900) 2023-11-08T16:34:50.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7f1910f60640 1 -- 172.21.15.81:0/1618399180 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x5557e50aa000 con 0x5557e41e1c00 2023-11-08T16:34:50.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7f1915c0c5c0 4 Got MDS map 47 2023-11-08T16:34:50.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 learned_addr learned my addr 172.21.15.81:0/505794715 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7f4819d1f640 1 -- 172.21.15.81:0/4069498314 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645842bcf20 con 0x56458514f000 2023-11-08T16:34:50.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7f4819d1f640 1 -- 172.21.15.81:0/4069498314 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5645842d0180 con 0x56458514f000 2023-11-08T16:34:50.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7400 0x562b98996c00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x562b99803a40 tx=0x562b99702000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7f4819d1f640 1 -- 172.21.15.81:0/4069498314 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5645842bd1e0 con 0x56458514f000 2023-11-08T16:34:50.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7fea57135640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf181d000 0x55fdf1589180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56458429c4e0 con 0x56458514f000 2023-11-08T16:34:50.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56458429c9c0 con 0x56458514f000 2023-11-08T16:34:50.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7f48215855c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:50.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7fe900c425c0 1 -- 172.21.15.81:0/3195071636 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5400 msgr2=0x55e170959b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7fe900c425c0 1 --2- 172.21.15.81:0/3195071636 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5400 0x55e170959b80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55e171705470 tx=0x55e1716c6240 comp rx=0 tx=0).stop 2023-11-08T16:34:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fe900c425c0 1 -- 172.21.15.81:0/3195071636 shutdown_connections 2023-11-08T16:34:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fe900c425c0 1 --2- 172.21.15.81:0/3195071636 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1717e5800 0x55e17095a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fe900c425c0 1 --2- 172.21.15.81:0/3195071636 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5400 0x55e170959b80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fe900c425c0 1 --2- 172.21.15.81:0/3195071636 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1717e5000 0x55e170959600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fe900c425c0 1 -- 172.21.15.81:0/3195071636 >> 172.21.15.81:0/3195071636 conn(0x55e1717e4c00 msgr2=0x55e171669800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf181d000 msgr2=0x55fdf1589180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fe900c425c0 1 -- 172.21.15.81:0/3195071636 shutdown_connections 2023-11-08T16:34:50.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fe900c425c0 1 -- 172.21.15.81:0/3195071636 wait complete. 2023-11-08T16:34:50.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fea56133640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf181d000 0x55fdf1589180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7f1915c0c5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:50.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.287+0000 7f1915c0c5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:50.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf181d400 msgr2=0x55fdf1588c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fea56133640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf181d400 0x55fdf1588c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdf1542d00 con 0x55fdf181d800 2023-11-08T16:34:50.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.288+0000 7f1915c0c5c0 1 Processor -- start 2023-11-08T16:34:50.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.289+0000 7f6eb3a0e640 1 -- 172.21.15.81:0/1154556006 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562b98970dc0 con 0x562b997f7400 2023-11-08T16:34:50.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.289+0000 7fea56133640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf181d800 0x55fdf1588680 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x55fdf22ef500 tx=0x55fdf22f2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.289+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562b98950b60 con 0x562b997f7400 2023-11-08T16:34:50.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.289+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(6 100000001fc.00000000 [call] v64'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f2e000 2023-11-08T16:34:50.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.289+0000 7fe900c425c0 1 Processor -- start 2023-11-08T16:34:50.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.289+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b98950820 con 0x562b997f7400 2023-11-08T16:34:50.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.289+0000 7f6ebbe175c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:50.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.289+0000 7f6eb3a0e640 1 -- 172.21.15.81:0/1154556006 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b9974b680 con 0x562b997f7400 2023-11-08T16:34:50.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.289+0000 7f1915c0c5c0 1 -- start start 2023-11-08T16:34:50.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.289+0000 7fe900c425c0 1 -- start start 2023-11-08T16:34:50.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:7 8.2c 8:34000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98e00 con 0x563e35f0e400 2023-11-08T16:34:50.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.289+0000 7fe704dbe640 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5636e2df4400 0x5636e2df0c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7f6eb3a0e640 1 -- 172.21.15.81:0/1154556006 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562b989711e0 con 0x562b997f7400 2023-11-08T16:34:50.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7fea50928640 1 -- 172.21.15.81:0/505794715 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fdf1562f20 con 0x55fdf181d800 2023-11-08T16:34:50.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7fea50928640 1 -- 172.21.15.81:0/505794715 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fdf22f5980 con 0x55fdf181d800 2023-11-08T16:34:50.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7fea50928640 1 -- 172.21.15.81:0/505794715 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55fdf15631e0 con 0x55fdf181d800 2023-11-08T16:34:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(7 [pgnls start_epoch 64 out=81b] v62'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f0e400 2023-11-08T16:34:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(21 [pgnls start_epoch 64 out=49b] v62'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2df4400 2023-11-08T16:34:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55fdf15429c0 con 0x55fdf181d800 2023-11-08T16:34:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7fe900c425c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1717e5000 0x55e17095b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7fe900c425c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1717e5400 0x55e17095ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7fe900c425c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5800 0x55e17095a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7fe900c425c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1709bcc80 con 0x55e1717e5400 2023-11-08T16:34:50.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7fe900c425c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1709bcb40 con 0x55e1717e5000 2023-11-08T16:34:50.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7fe900c425c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e1709bca00 con 0x55e1717e5800 2023-11-08T16:34:50.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7f1915c0c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e4fcbc00 0x5557e424b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7f1915c0c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e510c000 0x5557e424bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7f1915c0c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e510c400 0x5557e424c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7f1915c0c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e4228a00 con 0x5557e4fcbc00 2023-11-08T16:34:50.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7f1915c0c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e42af180 con 0x5557e510c400 2023-11-08T16:34:50.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.290+0000 7f1915c0c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557e42aedc0 con 0x5557e510c000 2023-11-08T16:34:50.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.291+0000 7f1912f64640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e4fcbc00 0x5557e424b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.291+0000 7f1912f64640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e4fcbc00 0x5557e424b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48312/0 (socket says 172.21.15.81:48312) 2023-11-08T16:34:50.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.291+0000 7f1912f64640 1 -- 172.21.15.81:0/1191510528 learned_addr learned my addr 172.21.15.81:0/1191510528 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.291+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:8 8.2c 8:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e99180 con 0x563e35f0e400 2023-11-08T16:34:50.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.291+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdf1542b60 con 0x55fdf181d800 2023-11-08T16:34:50.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.291+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:22 9.0 9:00000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2dddc00 con 0x5636e2dd8c00 2023-11-08T16:34:50.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.291+0000 7fea58a395c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:50.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.291+0000 7fe8fe34b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5800 0x55e17095a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7fe8fe34b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5800 0x55e17095a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:49968/0 (socket says 172.21.15.81:49968) 2023-11-08T16:34:50.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7fe8fe34b640 1 -- 172.21.15.81:0/1479994114 learned_addr learned my addr 172.21.15.81:0/1479994114 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f1913765640 1 --2- 172.21.15.81:0/1191510528 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e510c000 0x5557e424bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7fe8ff34d640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1717e5000 0x55e17095b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f1912f64640 1 -- 172.21.15.81:0/1191510528 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e510c000 msgr2=0x5557e424bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f1912f64640 1 --2- 172.21.15.81:0/1191510528 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e510c000 0x5557e424bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f1912f64640 1 -- 172.21.15.81:0/1191510528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e510c400 msgr2=0x5557e424c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f1912f64640 1 --2- 172.21.15.81:0/1191510528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e510c400 0x5557e424c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f1912f64640 1 -- 172.21.15.81:0/1191510528 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557e4206b60 con 0x5557e4fcbc00 2023-11-08T16:34:50.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f4819d1f640 1 -- 172.21.15.81:0/4069498314 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x5645842d3080 con 0x56458514f000 2023-11-08T16:34:50.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(8 10000000209.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f0e400 2023-11-08T16:34:50.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f16dfc385c0 10 datascan.operator(): handling object 1099511628297.0 2023-11-08T16:34:50.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f6eb3a0e640 1 -- 172.21.15.81:0/1154556006 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x562b98987080 con 0x562b997f7400 2023-11-08T16:34:50.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(22 [pgnls start_epoch 64 out=101b] v62'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:9 8.2c 8:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35b01880 con 0x563e35f0e400 2023-11-08T16:34:50.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:23 9.0 9:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddd880 con 0x5636e2dd8c00 2023-11-08T16:34:50.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7fe8fe34b640 1 -- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1717e5000 msgr2=0x55e17095b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1717e5000 0x55e17095b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7fe8fe34b640 1 -- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1717e5400 msgr2=0x55e17095ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1717e5400 0x55e17095ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7fe8fe34b640 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e170914d00 con 0x55e1717e5800 2023-11-08T16:34:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.292+0000 7f1912f64640 1 --2- 172.21.15.81:0/1191510528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e4fcbc00 0x5557e424b600 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x5557e4fb4ae0 tx=0x5557e50aec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b646e350f6dba8a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f190ef5c640 1 -- 172.21.15.81:0/1191510528 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5557e4226f20 con 0x5557e4fcbc00 2023-11-08T16:34:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f190ef5c640 1 -- 172.21.15.81:0/1191510528 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5557e4fc9200 con 0x5557e4fcbc00 2023-11-08T16:34:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f190ef5c640 1 -- 172.21.15.81:0/1191510528 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5557e42271e0 con 0x5557e4fcbc00 2023-11-08T16:34:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(23 10000000206.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f4819d1f640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5645851d2c00 0x5645842e3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f6eb3a0e640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562b99879800 0x562b98997700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f6eb3a0e640 1 -- 172.21.15.81:0/1154556006 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x562b989eaa80 con 0x562b997f7400 2023-11-08T16:34:50.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f4819d1f640 1 -- 172.21.15.81:0/4069498314 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x564584336a80 con 0x56458514f000 2023-11-08T16:34:50.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628294.0 2023-11-08T16:34:50.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:24 8.0 8:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddca80 con 0x5636e2dd8000 2023-11-08T16:34:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f48215855c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5800 0x55e17095a680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55e1717c6bd0 tx=0x55e1716c6540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f48215855c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f48215855c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f48215855c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 9 2023-11-08T16:34:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f48215855c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-11-08T16:34:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5645851d3000 0x5645842e2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.293+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:1 8.12 8:49249249::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5645842cf880 con 0x5645851d3000 2023-11-08T16:34:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f6ebbe175c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f6ebbe175c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f6ebbe175c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:50.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f6ebbe175c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 9 2023-11-08T16:34:50.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f6ebbe175c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-11-08T16:34:50.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562b99879c00 0x562b98996100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:1 8.36 8:6db6db6d::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98983880 con 0x562b99879c00 2023-11-08T16:34:50.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f481f52a640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5645851d2c00 0x5645842e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1191510528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e4fcbc00 msgr2=0x5557e424b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7fe8f8b40640 1 -- 172.21.15.81:0/1479994114 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e170934dc0 con 0x55e1717e5800 2023-11-08T16:34:50.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/1191510528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e4fcbc00 0x5557e424b600 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x5557e4fb4ae0 tx=0x5557e50aec00 comp rx=0 tx=0).stop 2023-11-08T16:34:50.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7fe8f8b40640 1 -- 172.21.15.81:0/1479994114 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e1716d5200 con 0x55e1717e5800 2023-11-08T16:34:50.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7fe8f8b40640 1 -- 172.21.15.81:0/1479994114 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55e170934f20 con 0x55e1717e5800 2023-11-08T16:34:50.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e1709144e0 con 0x55e1717e5800 2023-11-08T16:34:50.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1191510528 shutdown_connections 2023-11-08T16:34:50.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/1191510528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e510c400 0x5557e424c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/1191510528 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e510c000 0x5557e424bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/1191510528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e4fcbc00 0x5557e424b600 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1191510528 >> 172.21.15.81:0/1191510528 conn(0x5557e4fcb800 msgr2=0x5557e42dd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1709149c0 con 0x55e1717e5800 2023-11-08T16:34:50.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7fe900c425c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:50.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.295+0000 7f6eb9219640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562b99879c00 0x562b98996100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.295+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1191510528 shutdown_connections 2023-11-08T16:34:50.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.295+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1191510528 wait complete. 2023-11-08T16:34:50.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.295+0000 7f481f52a640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5645851d2c00 0x5645842e3700 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5645843682d0 tx=0x564585052120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.295+0000 7f481f52a640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5645851d3000 0x5645842e2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.294+0000 7f6eba21b640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562b99879800 0x562b98997700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.295+0000 7fea50928640 1 -- 172.21.15.81:0/505794715 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55fdf1578840 con 0x55fdf181d800 2023-11-08T16:34:50.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.295+0000 7f6eb9219640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562b99879c00 0x562b98996100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.296+0000 7f1915c0c5c0 1 Processor -- start 2023-11-08T16:34:50.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.296+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(1 [pgnls start_epoch 64 out=101b] v62'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562b997d8b40 con 0x562b99879c00 2023-11-08T16:34:50.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.296+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:2 8.36 8:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98983c00 con 0x562b99879c00 2023-11-08T16:34:50.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.296+0000 7f1915c0c5c0 1 -- start start 2023-11-08T16:34:50.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.296+0000 7f6eba21b640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562b99879800 0x562b98997700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x562b99802c90 tx=0x562b99702360 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.296+0000 7f481f52a640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5645851d3000 0x5645842e2100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.296+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(2 100000001f9.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562b997d8b40 con 0x562b99879c00 2023-11-08T16:34:50.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.296+0000 7f6ebbe175c0 10 datascan.operator(): handling object 1099511628281.0 2023-11-08T16:34:50.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.296+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:3 8.36 8:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98983880 con 0x562b99879c00 2023-11-08T16:34:50.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.296+0000 7fea50928640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55fdf189a800 0x55fdf1589700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7fea50928640 1 -- 172.21.15.81:0/505794715 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55fdf15dca80 con 0x55fdf181d800 2023-11-08T16:34:50.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(9 10000000209.00000000 [call] v64'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f0e400 2023-11-08T16:34:50.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x563e35f0e800 0x563e35b13b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:10 8.1c 8:38000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98700 con 0x563e35f0e800 2023-11-08T16:34:50.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7fea58a395c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:50.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7fea58a395c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:50.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7fea58a395c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:50.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7fea58a395c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 9 2023-11-08T16:34:50.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7fea58a395c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-11-08T16:34:50.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55fdf189ac00 0x55fdf1588100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:1 8.9 8:92492492::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf1575880 con 0x55fdf189ac00 2023-11-08T16:34:50.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(24 10000000206.00000000 [call,setxattr (8)] v64'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f1915c0c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e4fcb800 0x5557e424d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f1915c0c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e510d000 0x5557e424cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f1915c0c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e510cc00 0x5557e424c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f1915c0c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e4228140 con 0x5557e510d000 2023-11-08T16:34:50.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f1915c0c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557e4229180 con 0x5557e510cc00 2023-11-08T16:34:50.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f1915c0c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557e4228b40 con 0x5557e4fcb800 2023-11-08T16:34:50.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:25 9.0 9:04f22c1d::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddce00 con 0x5636e2dd8c00 2023-11-08T16:34:50.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(1 [pgnls start_epoch 64 out=49b] v62'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564585138b40 con 0x5645851d3000 2023-11-08T16:34:50.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f481ad21640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x564584600000 0x564584606000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7f481ad21640 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:2 8.32 8:4c000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460a000 con 0x564584600000 2023-11-08T16:34:50.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.297+0000 7fea57135640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55fdf189a800 0x55fdf1589700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.298+0000 7f16de389640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x563e35f0e800 0x563e35b13b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.298+0000 7f1913765640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e510d000 0x5557e424cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.298+0000 7fea56934640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55fdf189ac00 0x55fdf1588100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.298+0000 7f1912f64640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e4fcb800 0x5557e424d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.298+0000 7f1912f64640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e510cc00 0x5557e424c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.298+0000 7f1912f64640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e510cc00 0x5557e424c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:51930/0 (socket says 172.21.15.81:51930) 2023-11-08T16:34:50.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.298+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 learned_addr learned my addr 172.21.15.81:0/159202259 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.298+0000 7f482052c640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x564584600000 0x564584606000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.298+0000 7fe8f8b40640 1 -- 172.21.15.81:0/1479994114 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55e17094b080 con 0x55e1717e5800 2023-11-08T16:34:50.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(25 [pgnls start_epoch 64 out=101b] v62'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:26 9.0 9:04f22c1d:::1000000020a.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddd180 con 0x5636e2dd8c00 2023-11-08T16:34:50.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e4fcb800 msgr2=0x5557e424d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7f1913765640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e4fcb800 0x5557e424d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e510cc00 msgr2=0x5557e424c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7f1913765640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e510cc00 0x5557e424c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557e4206820 con 0x5557e510d000 2023-11-08T16:34:50.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7fea57135640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55fdf189a800 0x55fdf1589700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55fdf22ee780 tx=0x55fdf18080c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7f482052c640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x564584600000 0x564584606000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(26 1000000020a.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628298.1 2023-11-08T16:34:50.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7f1913765640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e510d000 0x5557e424cc00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x5557e4fb5860 tx=0x5557e50aed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:27 8.2a 8:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ab9880 con 0x5636e2dd8000 2023-11-08T16:34:50.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7fea56934640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55fdf189ac00 0x55fdf1588100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7f16de389640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x563e35f0e800 0x563e35b13b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7f190d759640 1 -- 172.21.15.81:0/159202259 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5557e4226f20 con 0x5557e510d000 2023-11-08T16:34:50.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7f190d759640 1 -- 172.21.15.81:0/159202259 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5557e423a180 con 0x5557e510d000 2023-11-08T16:34:50.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.299+0000 7f190d759640 1 -- 172.21.15.81:0/159202259 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5557e42262c0 con 0x5557e510d000 2023-11-08T16:34:50.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5557e42064e0 con 0x5557e510d000 2023-11-08T16:34:50.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(2 [pgnls start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564585138d80 con 0x564584600000 2023-11-08T16:34:50.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f481b522640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x564584600400 0x564584606580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f481b522640 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:3 8.a 8:50000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460a380 con 0x564584600400 2023-11-08T16:34:50.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557e42061a0 con 0x5557e510d000 2023-11-08T16:34:50.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f1915c0c5c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:50.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(3 100000001f9.00000000 [call] v64'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562b997d8b40 con 0x562b99879c00 2023-11-08T16:34:50.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f16de389640 1 -- 172.21.15.81:0/439004518 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(10 [pgnls start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563e35d86d80 con 0x563e35f0e800 2023-11-08T16:34:50.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:4 8.36 8:6fe38dc2::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98983c00 con 0x562b99879c00 2023-11-08T16:34:50.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(1 [pgnls start_epoch 64 out=49b] v62'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf189ac00 2023-11-08T16:34:50.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7fe8f8b40640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55e170c76000 0x55e17095b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7fe8f8b40640 1 -- 172.21.15.81:0/1479994114 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55e1709aea80 con 0x55e1717e5800 2023-11-08T16:34:50.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f16d937f640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x563e35f2e800 0x563e35f21600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(4 [pgnls start_epoch 64 out=81b] v64'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562b997d8b40 con 0x562b99879c00 2023-11-08T16:34:50.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:5 8.36 8:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caa000 con 0x562b99879c00 2023-11-08T16:34:50.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7fea5192a640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55fdf191c000 0x55fdf1998000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7fea5192a640 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:2 8.29 8:94000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199c000 con 0x55fdf191c000 2023-11-08T16:34:50.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f481fd2b640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x564584600400 0x564584606580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7fe900c425c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:50.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7fe900c425c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:50.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7fe900c425c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:50.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7fe900c425c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 9 2023-11-08T16:34:50.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7fe900c425c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-11-08T16:34:50.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55e170c76400 0x55e17095a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:1 8.1b 8:db6db6db::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170947880 con 0x55e170c76400 2023-11-08T16:34:50.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.300+0000 7f16d937f640 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:11 8.3c 8:3c000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98000 con 0x563e35f2e800 2023-11-08T16:34:50.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.301+0000 7f16ddb88640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x563e35f2e800 0x563e35f21600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.302+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(5 100000001f7.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562b997d8b40 con 0x562b99879c00 2023-11-08T16:34:50.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.302+0000 7fe8feb4c640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55e170c76400 0x55e17095a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.302+0000 7f6ebbe175c0 10 datascan.operator(): handling object 1099511628279.0 2023-11-08T16:34:50.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.302+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:6 8.36 8:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caa380 con 0x562b99879c00 2023-11-08T16:34:50.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.302+0000 7f16ddb88640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x563e35f2e800 0x563e35f21600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.302+0000 7fea57135640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55fdf191c000 0x55fdf1998000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.302+0000 7fe8ff34d640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55e170c76000 0x55e17095b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.303+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(27 1000000020a.00000000 [call,setxattr (8)] v64'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.303+0000 7f481fd2b640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x564584600400 0x564584606580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.303+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:28 9.0 9:05552577::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ab9c00 con 0x5636e2dd8c00 2023-11-08T16:34:50.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.303+0000 7f190d759640 1 -- 172.21.15.81:0/159202259 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x5557e423d340 con 0x5557e510d000 2023-11-08T16:34:50.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.303+0000 7fea57135640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55fdf191c000 0x55fdf1998000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.303+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(11 [pgnls start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563e35d86fc0 con 0x563e35f2e800 2023-11-08T16:34:50.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.303+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(28 [pgnls start_epoch 64 out=81b] v62'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.303+0000 7f16d8b7e640 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:12 8.2 8:40000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98a80 con 0x563e35f0f800 2023-11-08T16:34:50.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.303+0000 7f190d759640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5557e513ac00 0x5557e424d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f190d759640 1 -- 172.21.15.81:0/159202259 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x5557e42a0a80 con 0x5557e510d000 2023-11-08T16:34:50.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:29 9.0 9:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2dddc00 con 0x5636e2dd8c00 2023-11-08T16:34:50.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f1915c0c5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:50.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f1915c0c5c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:50.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f1915c0c5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:50.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f1915c0c5c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 9 2023-11-08T16:34:50.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f1915c0c5c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-11-08T16:34:50.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7fea57135640 1 -- 172.21.15.81:0/505794715 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(2 [pgnls start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdf1800d80 con 0x55fdf191c000 2023-11-08T16:34:50.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7fea5212b640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55fdf191c400 0x55fdf1998580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5557e510c800 0x5557e424c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:1 8.2d 8:b6db6db6::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4239880 con 0x5557e510c800 2023-11-08T16:34:50.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.303+0000 7f481fd2b640 1 -- 172.21.15.81:0/4069498314 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(3 [pgnls start_epoch 64 out=49b] v62'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564585138fc0 con 0x564584600400 2023-11-08T16:34:50.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7fea5212b640 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:3 8.19 8:98000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199c380 con 0x55fdf191c400 2023-11-08T16:34:50.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f1912f64640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5557e513ac00 0x5557e424d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f1912f64640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5557e510c800 0x5557e424c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7fe8feb4c640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55e170c76400 0x55e17095a100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f481ad21640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x564584600800 0x564584606b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f481ad21640 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:4 8.2a 8:54000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460a700 con 0x564584600800 2023-11-08T16:34:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(29 1000000020b.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628299.0 2023-11-08T16:34:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:30 8.20 8:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddd880 con 0x5636e2dd8400 2023-11-08T16:34:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7fea56934640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55fdf191c400 0x55fdf1998580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.304+0000 7f481f52a640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x564584600800 0x564584606b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.305+0000 7f16de389640 1 -- 172.21.15.81:0/439004518 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(12 [pgnls start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563e35d86d80 con 0x563e35f0f800 2023-11-08T16:34:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.305+0000 7f16d937f640 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:13 8.22 8:44000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98700 con 0x563e35f2e800 2023-11-08T16:34:50.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.305+0000 7f1912f64640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5557e510c800 0x5557e424c100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.305+0000 7fe8ff34d640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55e170c76000 0x55e17095b700 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55e1709e09c0 tx=0x55e1716ced20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.305+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(1 [pgnls start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c76400 2023-11-08T16:34:50.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.305+0000 7fe8f9b42640 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:2 8.3b 8:dc000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c86000 con 0x55e170c76400 2023-11-08T16:34:50.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.305+0000 7f481f52a640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x564584600800 0x564584606b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.305+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(1 [pgnls start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.305+0000 7f1912f64640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5557e513ac00 0x5557e424d700 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x5557e4fb57a0 tx=0x5557e4fb0180 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.305+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(6 100000001f7.00000000 [call] v64'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562b997d8b40 con 0x562b99879c00 2023-11-08T16:34:50.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.305+0000 7f190e75b640 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:2 8.1d 8:b8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564000 con 0x5557e510c800 2023-11-08T16:34:50.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562b98cac000 0x562b98995b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:7 8.e 8:70000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caa700 con 0x562b98cac000 2023-11-08T16:34:50.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(4 [pgnls start_epoch 64 out=101b] v64'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:5 8.2a 8:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5645842cfc00 con 0x564584600800 2023-11-08T16:34:50.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(13 [pgnls start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563e35d86fc0 con 0x563e35f2e800 2023-11-08T16:34:50.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f16d8b7e640 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:14 8.12 8:48000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98380 con 0x563e35f0e400 2023-11-08T16:34:50.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7fea56934640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55fdf191c400 0x55fdf1998580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(2 [pgnls start_epoch 64 out=49b] v62'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f190ef5c640 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:3 8.3d 8:bc000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4239880 con 0x5557e510c800 2023-11-08T16:34:50.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(2 [pgnls start_epoch 64 out=81b] v62'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c76400 2023-11-08T16:34:50.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(3 [pgnls start_epoch 64 out=81b] v62'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:3 8.3b 8:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170947c00 con 0x55e170c76400 2023-11-08T16:34:50.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(5 1000000020a.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:4 8.3d 8:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4239c00 con 0x5557e510c800 2023-11-08T16:34:50.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f48215855c0 10 datascan.operator(): handling object 1099511628298.0 2023-11-08T16:34:50.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:6 8.2a 8:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460a700 con 0x564584600800 2023-11-08T16:34:50.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.306+0000 7f6eba21b640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562b98cac000 0x562b98995b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.307+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(4 10000000205.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.307+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628293.0 2023-11-08T16:34:50.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.307+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 8.3d 8:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564000 con 0x5557e510c800 2023-11-08T16:34:50.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.307+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(14 [pgnls start_epoch 64 out=49b] v62'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f0e400 2023-11-08T16:34:50.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.307+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x563e35f0ec00 0x563e35b13600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.307+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(3 [pgnls start_epoch 64 out=101b] v62'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c400 2023-11-08T16:34:50.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.307+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:15 9.4 9:24924924::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98e00 con 0x563e35f0ec00 2023-11-08T16:34:50.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.307+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:4 8.19 8:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf1575c00 con 0x55fdf191c400 2023-11-08T16:34:50.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.308+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(3 100000001fe.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c76400 2023-11-08T16:34:50.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.308+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511628286.0 2023-11-08T16:34:50.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.308+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:4 8.3b 8:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170947880 con 0x55e170c76400 2023-11-08T16:34:50.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.308+0000 7f6eba21b640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562b98cac000 0x562b98995b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.308+0000 7f16dd387640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x563e35f0ec00 0x563e35b13600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.308+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(30 1000000020b.00000000 [call,setxattr (8)] v64'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2dd8400 2023-11-08T16:34:50.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.308+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:31 9.10 9:08000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddce00 con 0x5636e2dd9000 2023-11-08T16:34:50.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.308+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(4 10000000200.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c400 2023-11-08T16:34:50.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.308+0000 7fea58a395c0 10 datascan.operator(): handling object 1099511628288.0 2023-11-08T16:34:50.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.308+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:5 8.19 8:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199c380 con 0x55fdf191c400 2023-11-08T16:34:50.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.309+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(7 [pgnls start_epoch 64 out=49b] v62'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cac000 2023-11-08T16:34:50.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.309+0000 7f16dd387640 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x563e35f0ec00 0x563e35b13600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.309+0000 7f6eb4a10640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562b98cad800 0x562b98cba000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.309+0000 7f6eb4a10640 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:8 8.2e 8:74000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98983880 con 0x562b98cad800 2023-11-08T16:34:50.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.309+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(15 [pgnls start_epoch 64 out=81b] v60'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f0ec00 2023-11-08T16:34:50.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.309+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:16 9.4 9:24ae92ec:::100000001fb.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e99500 con 0x563e35f0ec00 2023-11-08T16:34:50.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.309+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(6 1000000020a.00000000 [call] v64'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.309+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:7 8.2a 8:57fe7535::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460ae00 con 0x564584600800 2023-11-08T16:34:50.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.310+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(16 100000001fb.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f0ec00 2023-11-08T16:34:50.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.310+0000 7f16dfc385c0 10 datascan.operator(): handling object 1099511628283.1 2023-11-08T16:34:50.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.310+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:17 8.13 8:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e99880 con 0x563e35f0fc00 2023-11-08T16:34:50.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.310+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562b98cad800 0x562b98cba000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.310+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(31 [pgnls start_epoch 64 out=81b] v62'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd9000 2023-11-08T16:34:50.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.310+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(7 [pgnls start_epoch 64 out=81b] v64'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.310+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:32 9.10 9:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0a000 con 0x5636e2dd9000 2023-11-08T16:34:50.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.310+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:8 8.2a 8:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460b180 con 0x564584600800 2023-11-08T16:34:50.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.310+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(8 100000001f6.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.310+0000 7f48215855c0 10 datascan.operator(): handling object 1099511628278.0 2023-11-08T16:34:50.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.310+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 8.2a 8:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5645842cf880 con 0x564584600800 2023-11-08T16:34:50.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.311+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(32 100000001fd.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd9000 2023-11-08T16:34:50.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.311+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628285.0 2023-11-08T16:34:50.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.311+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:33 8.10 8:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0a380 con 0x5636e2dd8800 2023-11-08T16:34:50.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.311+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562b98cad800 0x562b98cba000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.311+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(5 10000000205.00000000 [call] v64'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5557e510c400 0x5557e424bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:6 8.3 8:c0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564380 con 0x5557e510c400 2023-11-08T16:34:50.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7f1913f66640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5557e510c400 0x5557e424bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(17 100000001fb.00000000 [call,setxattr (8)] v64'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563e35d86fc0 con 0x563e35f0fc00 2023-11-08T16:34:50.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:18 9.14 9:28000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35b01c00 con 0x563e35f0ec00 2023-11-08T16:34:50.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(4 100000001fe.00000000 [call] v64'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c76400 2023-11-08T16:34:50.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1154556006 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(8 [pgnls start_epoch 64 out=49b] v62'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b997d8fc0 con 0x562b98cad800 2023-11-08T16:34:50.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7f6eb5211640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562b98cadc00 0x562b98cba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(9 100000001f6.00000000 [call] v64'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:10 8.1a 8:58000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460a700 con 0x564584600000 2023-11-08T16:34:50.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55e170c76800 0x55e170959b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:5 8.7 8:e0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e000 con 0x55e170c76800 2023-11-08T16:34:50.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.312+0000 7f6eb5211640 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 8.1e 8:78000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98983c00 con 0x562b98cadc00 2023-11-08T16:34:50.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.313+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(18 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f0ec00 2023-11-08T16:34:50.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.313+0000 7f1913f66640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5557e510c400 0x5557e424bb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.313+0000 7f6eb9219640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562b98cadc00 0x562b98cba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.313+0000 7fe8ff34d640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55e170c76800 0x55e170959b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.313+0000 7f16d937f640 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:19 9.c 9:30000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e99180 con 0x563e35f0fc00 2023-11-08T16:34:50.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.313+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(10 [pgnls start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564585138d80 con 0x564584600000 2023-11-08T16:34:50.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.313+0000 7f481b522640 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:11 8.3a 8:5c000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460bc00 con 0x564584600400 2023-11-08T16:34:50.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.313+0000 7f1913f66640 1 -- 172.21.15.81:0/159202259 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(6 [pgnls start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5557e5096d80 con 0x5557e510c400 2023-11-08T16:34:50.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.313+0000 7f190ef5c640 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:7 8.23 8:c4000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4239880 con 0x5557e510c800 2023-11-08T16:34:50.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f6eb9219640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562b98cadc00 0x562b98cba580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(7 [pgnls start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f481fd2b640 1 -- 172.21.15.81:0/4069498314 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(11 [pgnls start_epoch 64 out=49b] v62'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564585138fc0 con 0x564584600400 2023-11-08T16:34:50.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(5 10000000200.00000000 [call] v64'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c400 2023-11-08T16:34:50.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:6 8.19 8:997b7129::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199c700 con 0x55fdf191c400 2023-11-08T16:34:50.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f481ad21640 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:12 8.6 8:60000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460aa80 con 0x564584600000 2023-11-08T16:34:50.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f190e75b640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5557e513b000 0x5557e4574000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f190e75b640 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:8 8.13 8:c8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564000 con 0x5557e513b000 2023-11-08T16:34:50.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(9 [pgnls start_epoch 64 out=81b] v62'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cadc00 2023-11-08T16:34:50.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(19 [pgnls start_epoch 64 out=101b] v62'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563e35d86fc0 con 0x563e35f0fc00 2023-11-08T16:34:50.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:10 8.1e 8:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caaa80 con 0x562b98cadc00 2023-11-08T16:34:50.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(12 [pgnls start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564585138d80 con 0x564584600000 2023-11-08T16:34:50.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7fe8ff34d640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55e170c76800 0x55e170959b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:20 9.c 9:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e98e00 con 0x563e35f0fc00 2023-11-08T16:34:50.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f481b522640 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:13 8.26 8:64000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460a000 con 0x564584600800 2023-11-08T16:34:50.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7f1913765640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5557e513b000 0x5557e4574000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.314+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(6 [pgnls start_epoch 64 out=81b] v64'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c400 2023-11-08T16:34:50.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:7 8.19 8:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf1575880 con 0x55fdf191c400 2023-11-08T16:34:50.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(10 1000000020c.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cadc00 2023-11-08T16:34:50.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7f6ebbe175c0 10 datascan.operator(): handling object 1099511628300.0 2023-11-08T16:34:50.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(5 [pgnls start_epoch 64 out=101b] v62'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(13 [pgnls start_epoch 64 out=81b] v62'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:11 8.1e 8:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caae00 con 0x562b98cadc00 2023-11-08T16:34:50.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:14 8.26 8:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460ae00 con 0x564584600800 2023-11-08T16:34:50.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:6 8.7 8:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e380 con 0x55e170c76800 2023-11-08T16:34:50.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(33 100000001fd.00000000 [call,setxattr (8)] v64'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8800 2023-11-08T16:34:50.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:34 9.8 9:10000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0a700 con 0x5636e2dd8000 2023-11-08T16:34:50.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(14 100000001ff.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7f48215855c0 10 datascan.operator(): handling object 1099511628287.0 2023-11-08T16:34:50.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:15 8.26 8:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460b500 con 0x564584600800 2023-11-08T16:34:50.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(6 10000000207.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511628295.0 2023-11-08T16:34:50.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.316+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(7 100000001f8.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c400 2023-11-08T16:34:50.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.316+0000 7fea58a395c0 10 datascan.operator(): handling object 1099511628280.0 2023-11-08T16:34:50.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.316+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:8 8.19 8:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf1575c00 con 0x55fdf191c400 2023-11-08T16:34:50.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.315+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:7 8.7 8:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e700 con 0x55e170c76800 2023-11-08T16:34:50.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.316+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(34 [pgnls start_epoch 64 out=101b] v62'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.316+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:35 9.8 9:101c2845:::100000001fe.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0aa80 con 0x5636e2dd8000 2023-11-08T16:34:50.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.316+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(20 100000001fc.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563e35d86fc0 con 0x563e35f0fc00 2023-11-08T16:34:50.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.316+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(35 100000001fe.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.317+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628286.1 2023-11-08T16:34:50.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.317+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:36 8.3b 8:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0ae00 con 0x5636e2dd9000 2023-11-08T16:34:50.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.317+0000 7f16dfc385c0 10 datascan.operator(): handling object 1099511628284.0 2023-11-08T16:34:50.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.317+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:21 8.c 8:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e99c00 con 0x563e35f2e000 2023-11-08T16:34:50.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.317+0000 7f1913765640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5557e513b000 0x5557e4574000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.317+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(8 [pgnls start_epoch 64 out=81b] v64'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b000 2023-11-08T16:34:50.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.317+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(11 1000000020c.00000000 [call] v64'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cadc00 2023-11-08T16:34:50.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.317+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 8.13 8:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564700 con 0x5557e513b000 2023-11-08T16:34:50.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.318+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:12 8.3e 8:7c000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caa380 con 0x562b98cad800 2023-11-08T16:34:50.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.318+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(9 100000001fb.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b000 2023-11-08T16:34:50.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.318+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628283.0 2023-11-08T16:34:50.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.318+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:10 8.13 8:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564a80 con 0x5557e513b000 2023-11-08T16:34:50.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.319+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1154556006 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(12 [pgnls start_epoch 64 out=49b] v62'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b997d8fc0 con 0x562b98cad800 2023-11-08T16:34:50.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.319+0000 7f6eb5211640 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:13 8.1 8:80000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98cab500 con 0x562b99879c00 2023-11-08T16:34:50.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.319+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(13 [pgnls start_epoch 64 out=49b] v62'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b997d8b40 con 0x562b99879c00 2023-11-08T16:34:50.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.319+0000 7f6eb4a10640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562b98cc6000 0x562b98cbab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.319+0000 7f6eb4a10640 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:14 8.21 8:84000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98cabc00 con 0x562b98cc6000 2023-11-08T16:34:50.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.319+0000 7f6eba21b640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562b98cc6000 0x562b98cbab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.320+0000 7f6eba21b640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562b98cc6000 0x562b98cbab00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.320+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(15 100000001ff.00000000 [call] v64'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.320+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:16 8.16 8:68000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460b880 con 0x564584600800 2023-11-08T16:34:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.320+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(14 [pgnls start_epoch 64 out=49b] v62'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cc6000 2023-11-08T16:34:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(16 [pgnls start_epoch 64 out=49b] v62'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7f481b522640 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:17 8.36 8:6c000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460a700 con 0x564584600400 2023-11-08T16:34:50.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.320+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(8 100000001f8.00000000 [call] v64'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c400 2023-11-08T16:34:50.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.320+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(7 10000000207.00000000 [call] v64'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.320+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:8 8.7 8:e2e07930::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8ea80 con 0x55e170c76800 2023-11-08T16:34:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(36 100000001fe.00000000 [call,setxattr (8)] v64'7 uv7 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd9000 2023-11-08T16:34:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.320+0000 7f6eb5211640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562b98cc6800 0x562b98cbb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7f6eb5211640 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:15 8.11 8:88000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98cab880 con 0x562b98cc6800 2023-11-08T16:34:50.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:37 9.8 9:1247e94b::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0b180 con 0x5636e2dd8000 2023-11-08T16:34:50.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55fdf189b000 0x55fdf1587b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:9 8.39 8:9c000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199c380 con 0x55fdf189b000 2023-11-08T16:34:50.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7f481fd2b640 1 -- 172.21.15.81:0/4069498314 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(17 [pgnls start_epoch 64 out=101b] v64'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564585138fc0 con 0x564584600400 2023-11-08T16:34:50.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(10 100000001fb.00000000 [call] v64'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b000 2023-11-08T16:34:50.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(8 [pgnls start_epoch 64 out=81b] v64'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:9 8.7 8:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8ee00 con 0x55e170c76800 2023-11-08T16:34:50.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(21 100000001fc.00000000 [call,setxattr (8)] v64'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f2e000 2023-11-08T16:34:50.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562b98cc6800 0x562b98cbb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:11 8.33 8:cc000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564000 con 0x5557e510c400 2023-11-08T16:34:50.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.321+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(9 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.322+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511627777.0 2023-11-08T16:34:50.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.322+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:10 8.7 8:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e000 con 0x55e170c76800 2023-11-08T16:34:50.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.322+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(37 [pgnls start_epoch 64 out=101b] v62'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.322+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5645851d3400 0x5645842e1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.322+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:18 9.12 9:49249249::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5645842cfc00 con 0x5645851d3400 2023-11-08T16:34:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.322+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:38 9.8 9:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0b500 con 0x5636e2dd8000 2023-11-08T16:34:50.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.322+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:22 9.c 9:37389ade::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35f56000 con 0x563e35f0fc00 2023-11-08T16:34:50.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.322+0000 7fea56133640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55fdf189b000 0x55fdf1587b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.322+0000 7f1913f66640 1 -- 172.21.15.81:0/159202259 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(11 [pgnls start_epoch 64 out=49b] v62'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5557e5096d80 con 0x5557e510c400 2023-11-08T16:34:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.322+0000 7f190e75b640 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:12 8.b 8:d0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564e00 con 0x5557e513b000 2023-11-08T16:34:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7f6eb9a1a640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562b98cc6800 0x562b98cbb600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(38 100000001f9.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628281.1 2023-11-08T16:34:50.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:39 8.36 8:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0b880 con 0x5636e2dd8c00 2023-11-08T16:34:50.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7f482052c640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5645851d3400 0x5645842e1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(12 [pgnls start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b000 2023-11-08T16:34:50.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7f190ef5c640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5557e513b400 0x5557e4574580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7f190ef5c640 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:13 8.2b 8:d4000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4239c00 con 0x5557e513b400 2023-11-08T16:34:50.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7fea56133640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55fdf189b000 0x55fdf1587b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(22 [pgnls start_epoch 64 out=81b] v62'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563e35d86fc0 con 0x563e35f0fc00 2023-11-08T16:34:50.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:23 9.c 9:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35f56380 con 0x563e35f0fc00 2023-11-08T16:34:50.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1154556006 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(15 [pgnls start_epoch 64 out=49b] v62'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b997d8fc0 con 0x562b98cc6800 2023-11-08T16:34:50.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.323+0000 7f6eb4a10640 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:16 8.31 8:8c000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caa380 con 0x562b98cad800 2023-11-08T16:34:50.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(10 10000000001.00000000 [call] v64'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55e170c76c00 0x55e170959600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:11 8.27 8:e4000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e380 con 0x55e170c76c00 2023-11-08T16:34:50.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7f1912f64640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5557e513b400 0x5557e4574580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(9 [pgnls start_epoch 64 out=49b] v62'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdf1800fc0 con 0x55fdf189b000 2023-11-08T16:34:50.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7fea5192a640 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:10 8.5 8:a0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199ce00 con 0x55fdf189ac00 2023-11-08T16:34:50.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7fe8feb4c640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55e170c76c00 0x55e170959600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7f482052c640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5645851d3400 0x5645842e1b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(23 10000000209.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563e35d86fc0 con 0x563e35f0fc00 2023-11-08T16:34:50.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7f16dfc385c0 10 datascan.operator(): handling object 1099511628297.0 2023-11-08T16:34:50.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:24 8.2c 8:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35f56700 con 0x563e35f0e400 2023-11-08T16:34:50.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.324+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1154556006 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(16 [pgnls start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b997d8fc0 con 0x562b98cad800 2023-11-08T16:34:50.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7f6eb5211640 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:17 8.9 8:90000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caa700 con 0x562b98cac000 2023-11-08T16:34:50.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7fe8feb4c640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55e170c76c00 0x55e170959600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(11 [pgnls start_epoch 64 out=81b] v62'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c76c00 2023-11-08T16:34:50.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(18 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564585138d80 con 0x5645851d3400 2023-11-08T16:34:50.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7f1912f64640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5557e513b400 0x5557e4574580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7f481b522640 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:19 9.a 9:50000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460aa80 con 0x5645851d3400 2023-11-08T16:34:50.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:12 8.27 8:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e700 con 0x55e170c76c00 2023-11-08T16:34:50.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(12 10000000208.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c76c00 2023-11-08T16:34:50.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511628296.0 2023-11-08T16:34:50.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(10 [pgnls start_epoch 64 out=81b] v62'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf189ac00 2023-11-08T16:34:50.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:13 8.27 8:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8ea80 con 0x55e170c76c00 2023-11-08T16:34:50.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.325+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:11 8.5 8:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199c700 con 0x55fdf189ac00 2023-11-08T16:34:50.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(17 [pgnls start_epoch 64 out=49b] v62'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cac000 2023-11-08T16:34:50.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562b98cac400 0x562b98995600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:18 9.16 9:6db6db6d::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caaa80 con 0x562b98cac400 2023-11-08T16:34:50.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(13 [pgnls start_epoch 64 out=81b] v62'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5557e5096b40 con 0x5557e513b400 2023-11-08T16:34:50.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:14 8.2b 8:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564700 con 0x5557e513b400 2023-11-08T16:34:50.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(39 100000001f9.00000000 [call,setxattr (8)] v64'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:40 9.8 9:15246602::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0bc00 con 0x5636e2dd8000 2023-11-08T16:34:50.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(19 [pgnls start_epoch 64 out=101b] v62'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564585138d80 con 0x5645851d3400 2023-11-08T16:34:50.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:20 9.a 9:5097359c:::100000001fd.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460b180 con 0x5645851d3400 2023-11-08T16:34:50.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(11 10000000204.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf189ac00 2023-11-08T16:34:50.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7fea58a395c0 10 datascan.operator(): handling object 1099511628292.0 2023-11-08T16:34:50.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.326+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:12 8.5 8:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199ca80 con 0x55fdf189ac00 2023-11-08T16:34:50.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7f6eb9219640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562b98cac400 0x562b98995600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(40 [pgnls start_epoch 64 out=101b] v62'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:41 9.8 9:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0b880 con 0x5636e2dd8000 2023-11-08T16:34:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(13 10000000208.00000000 [call] v64'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c76c00 2023-11-08T16:34:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55e170c77000 0x55e170c98000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:14 8.17 8:e8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8ee00 con 0x55e170c77000 2023-11-08T16:34:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(14 10000000203.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096b40 con 0x5557e513b400 2023-11-08T16:34:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(41 10000000202.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628291.0 2023-11-08T16:34:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:15 8.2b 8:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4565180 con 0x5557e513b400 2023-11-08T16:34:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628290.0 2023-11-08T16:34:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.327+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:42 8.28 8:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0a380 con 0x5636e2df4000 2023-11-08T16:34:50.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.328+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(20 100000001fd.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564585138d80 con 0x5645851d3400 2023-11-08T16:34:50.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.328+0000 7f48215855c0 10 datascan.operator(): handling object 1099511628285.1 2023-11-08T16:34:50.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.328+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5645851d3800 0x5645842e1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.328+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:21 8.10 8:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460a000 con 0x5645851d3800 2023-11-08T16:34:50.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.328+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55e170c77000 0x55e170c98000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.328+0000 7f6eb9219640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562b98cac400 0x562b98995600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.329+0000 7f481fd2b640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5645851d3800 0x5645842e1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.329+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(42 10000000202.00000000 [call,setxattr (8)] v64'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2df4000 2023-11-08T16:34:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.329+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(18 [pgnls start_epoch 64 out=101b] v62'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cac400 2023-11-08T16:34:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.329+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:43 9.8 9:177a762d::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0a700 con 0x5636e2dd8000 2023-11-08T16:34:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.329+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:19 9.16 9:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caae00 con 0x562b98cac400 2023-11-08T16:34:50.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.329+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55e170c77000 0x55e170c98000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.329+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(43 [pgnls start_epoch 64 out=81b] v62'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.330+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:44 9.8 9:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddd180 con 0x5636e2dd8000 2023-11-08T16:34:50.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.330+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(44 10000000201.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.330+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628289.0 2023-11-08T16:34:50.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.330+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:45 8.28 8:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2dddc00 con 0x5636e2df4000 2023-11-08T16:34:50.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.330+0000 7f481fd2b640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5645851d3800 0x5645842e1600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.330+0000 7fe8fe34b640 1 -- 172.21.15.81:0/1479994114 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(14 [pgnls start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1717a6fc0 con 0x55e170c77000 2023-11-08T16:34:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.331+0000 7fe8fa343640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55e170ca2000 0x55e170c9a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.331+0000 7fe8fa343640 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:15 8.37 8:ec000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170947c00 con 0x55e170ca2000 2023-11-08T16:34:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.331+0000 7fe8ff34d640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55e170ca2000 0x55e170c9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.331+0000 7fe8ff34d640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55e170ca2000 0x55e170c9a680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.331+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(19 100000001f9.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cac400 2023-11-08T16:34:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.331+0000 7f6ebbe175c0 10 datascan.operator(): handling object 1099511628281.0 2023-11-08T16:34:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.331+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:20 8.36 8:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caa000 con 0x562b99879c00 2023-11-08T16:34:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.332+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(24 10000000209.00000000 [call,setxattr (8)] v64'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f0e400 2023-11-08T16:34:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.332+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:25 9.1c 9:38000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35f56a80 con 0x563e35f2e000 2023-11-08T16:34:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.332+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(15 [pgnls start_epoch 64 out=49b] v62'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170ca2000 2023-11-08T16:34:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.332+0000 7fe8f9b42640 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:16 8.f 8:f0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e380 con 0x55e170ca2000 2023-11-08T16:34:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.332+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(25 [pgnls start_epoch 64 out=81b] v60'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f2e000 2023-11-08T16:34:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.332+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:26 9.1c 9:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35f56e00 con 0x563e35f2e000 2023-11-08T16:34:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.332+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(16 [pgnls start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170ca2000 2023-11-08T16:34:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.332+0000 7fe8fa343640 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:17 8.2f 8:f4000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8f500 con 0x55e170c76c00 2023-11-08T16:34:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.332+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(26 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f2e000 2023-11-08T16:34:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.332+0000 7f16dfc385c0 10 datascan.operator(): handling object 1099511628278.1 2023-11-08T16:34:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.333+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:27 8.2a 8:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35f57180 con 0x563e35f0fc00 2023-11-08T16:34:50.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.333+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(17 [pgnls start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c76c00 2023-11-08T16:34:50.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.333+0000 7fe8f9b42640 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:18 8.1f 8:f8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170947880 con 0x55e170c76c00 2023-11-08T16:34:50.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.333+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(12 10000000204.00000000 [call] v64'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf189ac00 2023-11-08T16:34:50.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.333+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55fdf189b400 0x55fdf1587600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.333+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:13 8.25 8:a4000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf1575880 con 0x55fdf189b400 2023-11-08T16:34:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.333+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(18 [pgnls start_epoch 64 out=49b] v62'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c76c00 2023-11-08T16:34:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.333+0000 7fe8fa343640 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:19 8.3f 8:fc000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8f880 con 0x55e170c76800 2023-11-08T16:34:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7fea57135640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55fdf189b400 0x55fdf1587600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(15 10000000203.00000000 [call] v64'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x5557e5096b40 con 0x5557e513b400 2023-11-08T16:34:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5557e510c000 0x5557e424b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:16 8.1b 8:d8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4565500 con 0x5557e510c000 2023-11-08T16:34:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(19 [pgnls start_epoch 64 out=81b] v62'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:20 8.3f 8:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8f180 con 0x55e170c76800 2023-11-08T16:34:50.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(45 10000000201.00000000 [call,setxattr (8)] v64'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2df4000 2023-11-08T16:34:50.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:46 9.18 9:18000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddc380 con 0x5636e2df4400 2023-11-08T16:34:50.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7f481fd2b640 1 -- 172.21.15.81:0/4069498314 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(21 100000001fd.00000000 [call,setxattr (8)] v64'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564585138fc0 con 0x5645851d3800 2023-11-08T16:34:50.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(20 100000001fa.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511628282.0 2023-11-08T16:34:50.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:22 9.a 9:55a6cd83::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460ae00 con 0x5645851d3400 2023-11-08T16:34:50.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.334+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:21 8.3f 8:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e000 con 0x55e170c76800 2023-11-08T16:34:50.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.335+0000 7f1913f66640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5557e510c000 0x5557e424b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.335+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(27 100000001f6.00000000 [call,setxattr (8)] v64'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563e35d86fc0 con 0x563e35f0fc00 2023-11-08T16:34:50.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.335+0000 7fea57135640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55fdf189b400 0x55fdf1587600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.335+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(46 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2df4400 2023-11-08T16:34:50.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.335+0000 7fe700db6640 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:47 9.4 9:20000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddd500 con 0x5636e2dd8c00 2023-11-08T16:34:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.335+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:28 9.2 9:40000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35f57500 con 0x563e35f0fc00 2023-11-08T16:34:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.336+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(22 [pgnls start_epoch 64 out=101b] v62'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564585138d80 con 0x5645851d3400 2023-11-08T16:34:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.336+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:23 9.a 9:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460a380 con 0x5645851d3400 2023-11-08T16:34:50.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.336+0000 7fea57135640 1 -- 172.21.15.81:0/505794715 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(13 [pgnls start_epoch 64 out=49b] v62'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdf1800d80 con 0x55fdf189b400 2023-11-08T16:34:50.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.336+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(20 100000001f9.00000000 [call,setxattr (8)] v64'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562b997d8b40 con 0x562b99879c00 2023-11-08T16:34:50.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.336+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:21 9.16 9:6fe38dc2::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98cab180 con 0x562b98cac400 2023-11-08T16:34:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.336+0000 7fea5192a640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55fdf191c800 0x55fdf1998b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.336+0000 7fea5192a640 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:14 8.15 8:a8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199d180 con 0x55fdf191c800 2023-11-08T16:34:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.336+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(28 [pgnls start_epoch 64 out=81b] v60'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563e35d86fc0 con 0x563e35f0fc00 2023-11-08T16:34:50.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.336+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(47 [pgnls start_epoch 64 out=101b] v60'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.336+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:29 9.2 9:40d56c48:::100000001fc.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35f56700 con 0x563e35f0fc00 2023-11-08T16:34:50.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.336+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:48 9.4 9:2057972a:::100000001ff.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddca80 con 0x5636e2dd8c00 2023-11-08T16:34:50.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7f1913f66640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5557e510c000 0x5557e424b600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7f16ddb88640 1 -- 172.21.15.81:0/439004518 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(29 100000001fc.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x563e35d86fc0 con 0x563e35f0fc00 2023-11-08T16:34:50.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7f16dfc385c0 10 datascan.operator(): handling object 1099511628284.1 2023-11-08T16:34:50.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:30 8.c 8:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35f56a80 con 0x563e35f2e000 2023-11-08T16:34:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7fea56934640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55fdf191c800 0x55fdf1998b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(48 100000001ff.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628287.1 2023-11-08T16:34:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:49 8.26 8:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ddce00 con 0x5636e2dd8000 2023-11-08T16:34:50.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(23 1000000020a.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564585138d80 con 0x5645851d3400 2023-11-08T16:34:50.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7f48215855c0 10 datascan.operator(): handling object 1099511628298.0 2023-11-08T16:34:50.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:24 8.2a 8:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460bc00 con 0x564584600800 2023-11-08T16:34:50.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(21 [pgnls start_epoch 64 out=81b] v62'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cac400 2023-11-08T16:34:50.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.337+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:22 9.16 9:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98983c00 con 0x562b98cac400 2023-11-08T16:34:50.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.338+0000 7f1913f66640 1 -- 172.21.15.81:0/159202259 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(16 [pgnls start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5557e5096d80 con 0x5557e510c000 2023-11-08T16:34:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.338+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(22 100000001f7.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cac400 2023-11-08T16:34:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.338+0000 7f6ebbe175c0 10 datascan.operator(): handling object 1099511628279.0 2023-11-08T16:34:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.338+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:23 8.36 8:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98cab500 con 0x562b99879c00 2023-11-08T16:34:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.338+0000 7fea56934640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55fdf191c800 0x55fdf1998b00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.338+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5557e513b800 0x5557e4574b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.338+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:17 9.d 9:b6db6db6::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4565880 con 0x5557e513b800 2023-11-08T16:34:50.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.338+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(21 100000001fa.00000000 [call] v64'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.339+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:22 9.1b 9:db6db6db::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e380 con 0x55e170ca2000 2023-11-08T16:34:50.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.339+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(14 [pgnls start_epoch 64 out=81b] v62'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c800 2023-11-08T16:34:50.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.339+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:15 8.15 8:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf1575c00 con 0x55fdf191c800 2023-11-08T16:34:50.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.339+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(22 [pgnls start_epoch 64 out=81b] v62'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170ca2000 2023-11-08T16:34:50.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.339+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(30 100000001fc.00000000 [call,setxattr (8)] v64'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x563e35d86b40 con 0x563e35f2e000 2023-11-08T16:34:50.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.339+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:31 9.12 9:48000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x563e35e99500 con 0x563e35f0e800 2023-11-08T16:34:50.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.339+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:23 9.1b 9:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8f880 con 0x55e170ca2000 2023-11-08T16:34:50.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.339+0000 7f1913765640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5557e513b800 0x5557e4574b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7fe7055bf640 1 -- 172.21.15.81:0/203102485 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(49 100000001ff.00000000 [call,setxattr (8)] v64'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5636e3830b40 con 0x5636e2dd8000 2023-11-08T16:34:50.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:50 9.4 9:218dfa3a::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ab9c00 con 0x5636e2dd8c00 2023-11-08T16:34:50.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(23 100000001fe.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170ca2000 2023-11-08T16:34:50.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511628286.0 2023-11-08T16:34:50.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:24 8.3b 8:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e700 con 0x55e170c76400 2023-11-08T16:34:50.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(50 [pgnls start_epoch 64 out=101b] v60'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16de389640 1 -- 172.21.15.81:0/439004518 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(31 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563e35d86d80 con 0x563e35f0e800 2023-11-08T16:34:50.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:51 9.4 9:218dfa3a:::10000000200.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2ab9880 con 0x5636e2dd8c00 2023-11-08T16:34:50.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x563e35f2e000 msgr2=0x563e35f20b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x563e35f2e000 0x563e35f20b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(15 1000000020d.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c800 2023-11-08T16:34:50.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x563e35f0ec00 msgr2=0x563e35b13600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x563e35f0ec00 0x563e35b13600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x563e35f0fc00 msgr2=0x563e35f20580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x563e35f0fc00 0x563e35f20580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x563e35f0f800 msgr2=0x563e35f20000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x563e35f0f800 0x563e35f20000 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x563e35f0e800 msgr2=0x563e35b13b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7fea58a395c0 10 datascan.operator(): handling object 1099511628301.0 2023-11-08T16:34:50.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x563e35f0e800 0x563e35b13b80 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x563e35f0e400 msgr2=0x563e35b14100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x563e35f0e400 0x563e35b14100 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x563e35f2e800 msgr2=0x563e35f21600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x563e35f2e800 0x563e35f21600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.340+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:16 8.15 8:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199ce00 con 0x55fdf191c800 2023-11-08T16:34:50.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x563e35f0e000 msgr2=0x563e35b15700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x563e35f0e000 0x563e35b15700 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x563e35daf320 tx=0x563e35d9a2a0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35db7400 msgr2=0x563e35b14c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/439004518 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35db7400 0x563e35b14c00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x563e35daf710 tx=0x563e3687cc00 comp rx=0 tx=0).stop 2023-11-08T16:34:50.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 reap_dead start 2023-11-08T16:34:50.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f1913765640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5557e513b800 0x5557e4574b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 reap_dead start 2023-11-08T16:34:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 reap_dead start 2023-11-08T16:34:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dd387640 1 -- 172.21.15.81:0/439004518 reap_dead start 2023-11-08T16:34:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 shutdown_connections 2023-11-08T16:34:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 >> 172.21.15.81:0/439004518 conn(0x563e35db6c00 msgr2=0x563e36830800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 <== osd.1 v2:172.21.15.81:6800/2060798739 12 ==== osd_op_reply(51 10000000200.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5636e3830fc0 con 0x5636e2dd8c00 2023-11-08T16:34:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 shutdown_connections 2023-11-08T16:34:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7fe706e195c0 10 datascan.operator(): handling object 1099511628288.1 2023-11-08T16:34:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7f16dfc385c0 1 -- 172.21.15.81:0/439004518 wait complete. 2023-11-08T16:34:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.341+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:52 8.19 8:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5636e2e0a380 con 0x5636e2dd8400 2023-11-08T16:34:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f16dfc385c0 1 -- 172.21.15.81:0/2004944930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35aa8c00 msgr2=0x563e35b12580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/2004944930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35aa8c00 0x563e35b12580 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x563e368b1410 tx=0x563e35d9a540 comp rx=0 tx=0).stop 2023-11-08T16:34:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(24 1000000020a.00000000 [call,setxattr (8)] v64'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:25 9.a 9:57fe7535::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5645842cf880 con 0x5645851d3400 2023-11-08T16:34:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f16dfc385c0 1 -- 172.21.15.81:0/2004944930 shutdown_connections 2023-11-08T16:34:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/2004944930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x563e35aa9c00 0x563e35b13080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/2004944930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e35aa9800 0x563e35b12b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f16dfc385c0 1 --2- 172.21.15.81:0/2004944930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x563e35aa8c00 0x563e35b12580 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f16dfc385c0 1 -- 172.21.15.81:0/2004944930 >> 172.21.15.81:0/2004944930 conn(0x563e35aa9000 msgr2=0x563e36831000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(17 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b800 2023-11-08T16:34:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(23 100000001f7.00000000 [call,setxattr (8)] v64'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562b997d8b40 con 0x562b99879c00 2023-11-08T16:34:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f190e75b640 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:18 9.1d 9:b8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4239880 con 0x5557e510c800 2023-11-08T16:34:50.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f16dfc385c0 1 -- 172.21.15.81:0/2004944930 shutdown_connections 2023-11-08T16:34:50.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f16dfc385c0 1 -- 172.21.15.81:0/2004944930 wait complete. 2023-11-08T16:34:50.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.342+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:24 9.e 9:70000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caaa80 con 0x562b98cad800 2023-11-08T16:34:50.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.343+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(18 [pgnls start_epoch 64 out=101b] v62'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.343+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:19 9.1d 9:b9a363c8:::1000000020d.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564000 con 0x5557e510c800 2023-11-08T16:34:50.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.343+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(25 [pgnls start_epoch 64 out=81b] v62'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564585138d80 con 0x5645851d3400 2023-11-08T16:34:50.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.343+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(19 1000000020d.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.343+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628301.1 2023-11-08T16:34:50.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.343+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1154556006 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(24 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b997d8fc0 con 0x562b98cad800 2023-11-08T16:34:50.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.343+0000 7f6eb5211640 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:25 9.1e 9:78000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98cabc00 con 0x562b98cc6000 2023-11-08T16:34:50.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.343+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:26 9.a 9:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460aa80 con 0x5645851d3400 2023-11-08T16:34:50.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.343+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:20 8.15 8:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564380 con 0x5557e513b800 2023-11-08T16:34:50.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.344+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(25 [pgnls start_epoch 64 out=101b] v62'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cc6000 2023-11-08T16:34:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.344+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:26 9.1e 9:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caae00 con 0x562b98cc6000 2023-11-08T16:34:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.344+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(26 100000001f6.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564585138d80 con 0x5645851d3400 2023-11-08T16:34:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.344+0000 7f48215855c0 10 datascan.operator(): handling object 1099511628278.0 2023-11-08T16:34:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.344+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(26 1000000020c.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cc6000 2023-11-08T16:34:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.344+0000 7f6ebbe175c0 10 datascan.operator(): handling object 1099511628300.0 2023-11-08T16:34:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.344+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:27 8.1e 8:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caa000 con 0x562b98cadc00 2023-11-08T16:34:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.344+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:27 8.2a 8:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460b180 con 0x564584600800 2023-11-08T16:34:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.345+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(24 100000001fe.00000000 [call,setxattr (8)] v64'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c76400 2023-11-08T16:34:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.345+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55e170c77400 0x55e170c98580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.345+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:25 9.7 9:e0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8ea80 con 0x55e170c77400 2023-11-08T16:34:50.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.346+0000 7fe8feb4c640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55e170c77400 0x55e170c98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.346+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(16 1000000020d.00000000 [call] v64'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c800 2023-11-08T16:34:50.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(27 100000001f6.00000000 [call,setxattr (8)] v64'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:28 9.1a 9:58000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460ae00 con 0x564584600800 2023-11-08T16:34:50.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe705dc0640 1 -- 172.21.15.81:0/203102485 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(52 10000000200.00000000 [call,setxattr (8)] v64'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5636e3830d80 con 0x5636e2dd8400 2023-11-08T16:34:50.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5636e2df4000 msgr2=0x5636e2df0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5636e2df4000 0x5636e2df0680 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5636e2dd8c00 msgr2=0x5636e2dee000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5636e2dd8c00 0x5636e2dee000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5636e2dd8000 msgr2=0x5636e2acc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5636e2dd8000 0x5636e2acc100 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5636e2dd9000 msgr2=0x5636e2dee580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5636e2dd9000 0x5636e2dee580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5636e2dd8400 msgr2=0x5636e2acbb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5636e2dd8400 0x5636e2acbb80 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5636e2df4400 msgr2=0x5636e2df0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5636e2df4400 0x5636e2df0c00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5636e2dd8800 msgr2=0x5636e2acb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5636e2dd8800 0x5636e2acb600 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:17 8.35 8:ac000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199c700 con 0x55fdf191c000 2023-11-08T16:34:50.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(28 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7f481b522640 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:29 9.6 9:60000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460b500 con 0x5645851d3400 2023-11-08T16:34:50.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5636e3927c00 msgr2=0x5636e2acd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5636e3927c00 0x5636e2acd700 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x5636e2b52810 tx=0x5636e393cf00 comp rx=0 tx=0).stop 2023-11-08T16:34:50.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 msgr2=0x5636e2acc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 --2- 172.21.15.81:0/203102485 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e3927800 0x5636e2acc680 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x5636e3938a20 tx=0x5636e390e0c0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 reap_dead start 2023-11-08T16:34:50.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 reap_dead start 2023-11-08T16:34:50.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 shutdown_connections 2023-11-08T16:34:50.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 >> 172.21.15.81:0/203102485 conn(0x5636e3926c00 msgr2=0x5636e37dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 reap_dead start 2023-11-08T16:34:50.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe704dbe640 1 -- 172.21.15.81:0/203102485 reap_dead start 2023-11-08T16:34:50.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.347+0000 7fe8feb4c640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55e170c77400 0x55e170c98580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 shutdown_connections 2023-11-08T16:34:50.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fe706e195c0 1 -- 172.21.15.81:0/203102485 wait complete. 2023-11-08T16:34:50.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fea57135640 1 -- 172.21.15.81:0/505794715 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(17 [pgnls start_epoch 64 out=49b] v62'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdf1800d80 con 0x55fdf191c000 2023-11-08T16:34:50.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fea5192a640 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:18 8.d 8:b0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199c000 con 0x55fdf191c400 2023-11-08T16:34:50.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(29 [pgnls start_epoch 64 out=81b] v62'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564585138d80 con 0x5645851d3400 2023-11-08T16:34:50.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(25 [pgnls start_epoch 64 out=101b] v62'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c77400 2023-11-08T16:34:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:30 9.6 9:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5645842cfc00 con 0x5645851d3400 2023-11-08T16:34:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:26 9.7 9:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170947c00 con 0x55e170c77400 2023-11-08T16:34:50.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fe706e195c0 1 -- 172.21.15.81:0/1875630176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e2a60c00 msgr2=0x5636e2aca580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fe706e195c0 1 --2- 172.21.15.81:0/1875630176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e2a60c00 0x5636e2aca580 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x5636e3875410 tx=0x5636e390e420 comp rx=0 tx=0).stop 2023-11-08T16:34:50.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fe706e195c0 1 -- 172.21.15.81:0/1875630176 shutdown_connections 2023-11-08T16:34:50.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fe706e195c0 1 --2- 172.21.15.81:0/1875630176 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5636e2a61c00 0x5636e2acb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fe706e195c0 1 --2- 172.21.15.81:0/1875630176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5636e2a61800 0x5636e2acab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fe706e195c0 1 --2- 172.21.15.81:0/1875630176 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5636e2a60c00 0x5636e2aca580 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.348+0000 7fe706e195c0 1 -- 172.21.15.81:0/1875630176 >> 172.21.15.81:0/1875630176 conn(0x5636e2a61000 msgr2=0x5636e37dc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(18 [pgnls start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c400 2023-11-08T16:34:50.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7f482052c640 1 -- 172.21.15.81:0/4069498314 <== osd.5 v2:172.21.15.93:6802/2406579932 9 ==== osd_op_reply(30 100000001ff.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564585138d80 con 0x5645851d3400 2023-11-08T16:34:50.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7f48215855c0 10 datascan.operator(): handling object 1099511628287.0 2023-11-08T16:34:50.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:31 8.26 8:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460a380 con 0x564584600800 2023-11-08T16:34:50.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7fea5212b640 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:19 8.2d 8:b4000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199dc00 con 0x55fdf189b000 2023-11-08T16:34:50.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(26 10000000207.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c77400 2023-11-08T16:34:50.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511628295.0 2023-11-08T16:34:50.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:27 8.7 8:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170947880 con 0x55e170c76800 2023-11-08T16:34:50.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7fe706e195c0 1 -- 172.21.15.81:0/1875630176 shutdown_connections 2023-11-08T16:34:50.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7fe706e195c0 1 -- 172.21.15.81:0/1875630176 wait complete. 2023-11-08T16:34:50.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(20 1000000020d.00000000 [call,setxattr (8)] v64'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b800 2023-11-08T16:34:50.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:21 9.1d 9:bbb18289::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564700 con 0x5557e510c800 2023-11-08T16:34:50.663 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '1', '--worker_m', '7'] in time 0:00:00.555356, result: 2023-11-08T16:34:50.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.350+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(21 [pgnls start_epoch 64 out=101b] v62'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(19 [pgnls start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdf1800fc0 con 0x55fdf189b000 2023-11-08T16:34:50.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.349+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:20 9.9 9:92492492::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199ca80 con 0x55fdf189b000 2023-11-08T16:34:50.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.350+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(20 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdf1800fc0 con 0x55fdf189b000 2023-11-08T16:34:50.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.350+0000 7fea5212b640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55fdf191d000 0x55fdf1999600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.350+0000 7fea5212b640 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:21 9.19 9:98000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199c380 con 0x55fdf191d000 2023-11-08T16:34:50.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.350+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:22 9.1d 9:bbb18289:::10000000204.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564a80 con 0x5557e510c800 2023-11-08T16:34:50.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.351+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(27 1000000020c.00000000 [call,setxattr (8)] v64'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cadc00 2023-11-08T16:34:50.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.351+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:28 9.1e 9:7ac44a97::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98cab180 con 0x562b98cc6000 2023-11-08T16:34:50.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.351+0000 7fea56133640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55fdf191d000 0x55fdf1999600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.351+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(28 [pgnls start_epoch 64 out=81b] v62'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cc6000 2023-11-08T16:34:50.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.351+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:29 9.1e 9:7ac44a97:::10000000207.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caa700 con 0x562b98cc6000 2023-11-08T16:34:50.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.352+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(22 10000000204.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.352+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(29 10000000207.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cc6000 2023-11-08T16:34:50.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.352+0000 7f6ebbe175c0 10 datascan.operator(): handling object 1099511628295.1 2023-11-08T16:34:50.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.352+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628292.1 2023-11-08T16:34:50.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.352+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:30 8.7 8:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98cabc00 con 0x562b99879c00 2023-11-08T16:34:50.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.352+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:23 8.5 8:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564e00 con 0x5557e513b400 2023-11-08T16:34:50.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.352+0000 7fea56133640 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55fdf191d000 0x55fdf1999600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.352+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(21 [pgnls start_epoch 64 out=101b] v62'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55fdf1800fc0 con 0x55fdf191d000 2023-11-08T16:34:50.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.352+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:22 9.19 9:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199d180 con 0x55fdf191d000 2023-11-08T16:34:50.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.353+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(22 10000000200.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fdf1800fc0 con 0x55fdf191d000 2023-11-08T16:34:50.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.353+0000 7fea58a395c0 10 datascan.operator(): handling object 1099511628288.0 2023-11-08T16:34:50.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.353+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:23 8.19 8:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf1575c00 con 0x55fdf191c400 2023-11-08T16:34:50.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.355+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(31 100000001ff.00000000 [call,setxattr (8)] v64'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.355+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5645851d3c00 0x564584607600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.355+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:32 9.16 9:68000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460aa80 con 0x5645851d3c00 2023-11-08T16:34:50.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.356+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(27 10000000207.00000000 [call,setxattr (8)] v64'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.356+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(30 10000000207.00000000 [call,setxattr (8)] v64'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562b997d8b40 con 0x562b99879c00 2023-11-08T16:34:50.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.356+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:28 9.7 9:e2e07930::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8f180 con 0x55e170c77400 2023-11-08T16:34:50.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.356+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:31 9.1 9:80000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98caae00 con 0x562b98cadc00 2023-11-08T16:34:50.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.356+0000 7f481f52a640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5645851d3c00 0x564584607600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.356+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(31 [pgnls start_epoch 64 out=101b] v62'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cadc00 2023-11-08T16:34:50.676 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '0', '--worker_m', '7'] in time 0:00:00.568430, result: 2023-11-08T16:34:50.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.356+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:32 9.1 9:8410d73e:::10000000201.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce6000 con 0x562b98cadc00 2023-11-08T16:34:50.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.356+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(23 10000000204.00000000 [call,setxattr (8)] v64'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5557e5096b40 con 0x5557e513b400 2023-11-08T16:34:50.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.356+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:24 9.1d 9:bfcd3380::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4565180 con 0x5557e510c800 2023-11-08T16:34:50.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(28 [pgnls start_epoch 64 out=101b] v62'7 uv7 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c77400 2023-11-08T16:34:50.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(32 10000000201.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cadc00 2023-11-08T16:34:50.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7f6ebbe175c0 10 datascan.operator(): handling object 1099511628289.1 2023-11-08T16:34:50.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:33 8.28 8:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce6380 con 0x562b98cc6000 2023-11-08T16:34:50.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:29 9.7 9:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e000 con 0x55e170c77400 2023-11-08T16:34:50.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 11 ==== osd_op_reply(24 [pgnls start_epoch 64 out=81b] v62'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:25 9.1d 9:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564000 con 0x5557e510c800 2023-11-08T16:34:50.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(23 10000000200.00000000 [call,setxattr (8)] v64'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c400 2023-11-08T16:34:50.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7f481f52a640 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5645851d3c00 0x564584607600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:24 9.19 9:997b7129::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf1575880 con 0x55fdf191d000 2023-11-08T16:34:50.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 12 ==== osd_op_reply(25 10000000205.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628293.0 2023-11-08T16:34:50.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.357+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:26 8.3d 8:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564700 con 0x5557e510c800 2023-11-08T16:34:50.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.358+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(24 [pgnls start_epoch 64 out=81b] v62'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fdf1800fc0 con 0x55fdf191d000 2023-11-08T16:34:50.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.358+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(29 10000000001.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c77400 2023-11-08T16:34:50.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.358+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:25 9.19 9:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199ca80 con 0x55fdf191d000 2023-11-08T16:34:50.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.358+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511627777.0 2023-11-08T16:34:50.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.358+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:30 8.7 8:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e380 con 0x55e170c76800 2023-11-08T16:34:50.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.358+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(32 [pgnls start_epoch 64 out=101b] v62'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564585138b40 con 0x5645851d3c00 2023-11-08T16:34:50.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.358+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:33 9.16 9:6bb58479:::10000000206.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460b500 con 0x5645851d3c00 2023-11-08T16:34:50.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.358+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(25 100000001f8.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fdf1800fc0 con 0x55fdf191d000 2023-11-08T16:34:50.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.358+0000 7fea58a395c0 10 datascan.operator(): handling object 1099511628280.0 2023-11-08T16:34:50.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.358+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:26 8.19 8:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199c380 con 0x55fdf191c400 2023-11-08T16:34:50.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.359+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(33 10000000206.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x564585138b40 con 0x5645851d3c00 2023-11-08T16:34:50.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.359+0000 7f48215855c0 10 datascan.operator(): handling object 1099511628294.1 2023-11-08T16:34:50.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.359+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(33 10000000201.00000000 [call,setxattr (8)] v64'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cc6000 2023-11-08T16:34:50.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.359+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:34 9.1 9:848ebb0b::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce6700 con 0x562b98cadc00 2023-11-08T16:34:50.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.359+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:34 8.0 8:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56458460ae00 con 0x564584600800 2023-11-08T16:34:50.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.360+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(34 [pgnls start_epoch 64 out=81b] v62'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cadc00 2023-11-08T16:34:50.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.360+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:35 9.1 9:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce6a80 con 0x562b98cadc00 2023-11-08T16:34:50.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.360+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 13 ==== osd_op_reply(26 10000000205.00000000 [call,setxattr (8)] v64'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.360+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:27 9.3 9:c0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564a80 con 0x5557e513b800 2023-11-08T16:34:50.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.360+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(35 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cadc00 2023-11-08T16:34:50.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.360+0000 7f6ebbe175c0 10 datascan.operator(): handling object 1099511628279.1 2023-11-08T16:34:50.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.360+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(30 10000000001.00000000 [call,setxattr (8)] v64'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.360+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:31 9.7 9:e5993619::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8f880 con 0x55e170c77400 2023-11-08T16:34:50.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.360+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:36 8.36 8:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce6e00 con 0x562b99879c00 2023-11-08T16:34:50.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.361+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(27 [pgnls start_epoch 64 out=81b] v62'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b800 2023-11-08T16:34:50.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.361+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(31 [pgnls start_epoch 64 out=81b] v62'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c77400 2023-11-08T16:34:50.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.361+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:28 9.3 9:c1028715:::10000000202.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564e00 con 0x5557e513b800 2023-11-08T16:34:50.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.361+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:32 9.7 9:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8f500 con 0x55e170c77400 2023-11-08T16:34:50.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.362+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.5 v2:172.21.15.93:6802/2406579932 9 ==== osd_op_reply(26 100000001f8.00000000 [call,setxattr (8)] v64'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c400 2023-11-08T16:34:50.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.362+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:27 9.5 9:a0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199d180 con 0x55fdf189b000 2023-11-08T16:34:50.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.362+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(27 [pgnls start_epoch 64 out=81b] v62'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fdf1800fc0 con 0x55fdf189b000 2023-11-08T16:34:50.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.362+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:28 9.5 9:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf19c4000 con 0x55fdf189b000 2023-11-08T16:34:50.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.362+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(32 10000000208.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c77400 2023-11-08T16:34:50.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.362+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511628296.0 2023-11-08T16:34:50.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.362+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:33 8.27 8:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8fc00 con 0x55e170c76c00 2023-11-08T16:34:50.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.362+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(28 10000000202.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b800 2023-11-08T16:34:50.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.362+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628290.1 2023-11-08T16:34:50.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.362+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:29 8.28 8:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4565500 con 0x5557e510c800 2023-11-08T16:34:50.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.362+0000 7f481f52a640 1 -- 172.21.15.81:0/4069498314 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(34 10000000206.00000000 [call,setxattr (8)] v64'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564585138b40 con 0x564584600800 2023-11-08T16:34:50.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x564584600000 msgr2=0x564584606000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x564584600000 0x564584606000 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x564584600400 msgr2=0x564584606580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x564584600400 0x564584606580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x564584600800 msgr2=0x564584606b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x564584600800 0x564584606b00 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5645851d3c00 msgr2=0x564584607600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5645851d3c00 0x564584607600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5645851d3400 msgr2=0x5645842e1b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5645851d3400 0x5645842e1b80 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5645851d3000 msgr2=0x5645842e2100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5645851d3000 0x5645842e2100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5645851d3800 msgr2=0x5645842e1600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5645851d3800 0x5645842e1600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(28 10000000204.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fdf1800fc0 con 0x55fdf189b000 2023-11-08T16:34:50.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7fea58a395c0 10 datascan.operator(): handling object 1099511628292.0 2023-11-08T16:34:50.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:29 8.5 8:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf19c4380 con 0x55fdf189ac00 2023-11-08T16:34:50.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f481fd2b640 1 -- 172.21.15.81:0/4069498314 reap_dead start 2023-11-08T16:34:50.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5645851d2c00 msgr2=0x5645842e3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5645851d2c00 0x5645842e3700 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x5645843682d0 tx=0x564585052120 comp rx=0 tx=0).stop 2023-11-08T16:34:50.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f481fd2b640 1 -- 172.21.15.81:0/4069498314 reap_dead start 2023-11-08T16:34:50.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f000 msgr2=0x5645842e2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f000 0x5645842e2c00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x56458515f620 tx=0x564585162c00 comp rx=0 tx=0).stop 2023-11-08T16:34:50.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 shutdown_connections 2023-11-08T16:34:50.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 --2- 172.21.15.81:0/4069498314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56458514f000 0x5645842e2c00 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 >> 172.21.15.81:0/4069498314 conn(0x56458514e800 msgr2=0x564584373800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.363+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 shutdown_connections 2023-11-08T16:34:50.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f48215855c0 1 -- 172.21.15.81:0/4069498314 wait complete. 2023-11-08T16:34:50.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(36 100000001f7.00000000 [call,setxattr (8)] v64'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562b997d8b40 con 0x562b99879c00 2023-11-08T16:34:50.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562b98cac800 0x562b98cbc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:37 9.11 9:88000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce7180 con 0x562b98cac800 2023-11-08T16:34:50.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f48215855c0 1 -- 172.21.15.81:0/3206251737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564584277c00 msgr2=0x5645842e1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f48215855c0 1 --2- 172.21.15.81:0/3206251737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564584277c00 0x5645842e1080 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x564585091830 tx=0x564585052600 comp rx=0 tx=0).stop 2023-11-08T16:34:50.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f48215855c0 1 -- 172.21.15.81:0/3206251737 shutdown_connections 2023-11-08T16:34:50.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f48215855c0 1 --2- 172.21.15.81:0/3206251737 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564584277c00 0x5645842e1080 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f48215855c0 1 --2- 172.21.15.81:0/3206251737 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564584277800 0x5645842e0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f48215855c0 1 --2- 172.21.15.81:0/3206251737 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564584276c00 0x5645842e0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f48215855c0 1 -- 172.21.15.81:0/3206251737 >> 172.21.15.81:0/3206251737 conn(0x564584277000 msgr2=0x564584373000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.364+0000 7f6eba21b640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562b98cac800 0x562b98cbc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.365+0000 7f48215855c0 1 -- 172.21.15.81:0/3206251737 shutdown_connections 2023-11-08T16:34:50.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.365+0000 7f48215855c0 1 -- 172.21.15.81:0/3206251737 wait complete. 2023-11-08T16:34:50.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.365+0000 7f6eba21b640 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562b98cac800 0x562b98cbc100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.366+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 14 ==== osd_op_reply(29 10000000202.00000000 [call,setxattr (8)] v64'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.366+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:30 9.13 9:c8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4565880 con 0x5557e513b800 2023-11-08T16:34:50.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.366+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(33 10000000208.00000000 [call,setxattr (8)] v64'5 uv5 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c76c00 2023-11-08T16:34:50.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.366+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(37 [pgnls start_epoch 64 out=101b] v62'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cac800 2023-11-08T16:34:50.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.366+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:38 9.11 9:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce7500 con 0x562b98cac800 2023-11-08T16:34:50.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.366+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:34 9.17 9:e8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170947880 con 0x55e170c77000 2023-11-08T16:34:50.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.366+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(38 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cac800 2023-11-08T16:34:50.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.366+0000 7f6ebbe175c0 10 datascan.operator(): handling object 1099511628280.1 2023-11-08T16:34:50.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.366+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:39 8.19 8:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce7880 con 0x562b98cc6800 2023-11-08T16:34:50.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.367+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(30 [pgnls start_epoch 64 out=101b] v62'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b800 2023-11-08T16:34:50.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.367+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:31 9.13 9:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4239880 con 0x5557e513b800 2023-11-08T16:34:50.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.367+0000 7fe8fe34b640 1 -- 172.21.15.81:0/1479994114 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(34 [pgnls start_epoch 64 out=81b] v62'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e1717a6fc0 con 0x55e170c77000 2023-11-08T16:34:50.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.367+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:35 9.17 9:e88218fc:::10000000209.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e700 con 0x55e170c77000 2023-11-08T16:34:50.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.367+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(31 100000001fb.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b800 2023-11-08T16:34:50.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.367+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628283.0 2023-11-08T16:34:50.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.367+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:32 8.13 8:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564380 con 0x5557e513b000 2023-11-08T16:34:50.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.368+0000 7fe8fe34b640 1 -- 172.21.15.81:0/1479994114 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(35 10000000209.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6fc0 con 0x55e170c77000 2023-11-08T16:34:50.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.368+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511628297.1 2023-11-08T16:34:50.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.368+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55e170ca2400 0x55e170c98b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.368+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:36 8.2c 8:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8ea80 con 0x55e170ca2400 2023-11-08T16:34:50.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.368+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55e170ca2400 0x55e170c98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.369+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(32 100000001fb.00000000 [call,setxattr (8)] v64'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b000 2023-11-08T16:34:50.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.369+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:33 9.13 9:cb24bee2::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564a80 con 0x5557e513b800 2023-11-08T16:34:50.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.369+0000 7fe8fe34b640 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55e170ca2400 0x55e170c98b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.370+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(29 10000000204.00000000 [call,setxattr (8)] v64'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf189ac00 2023-11-08T16:34:50.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.370+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:30 9.15 9:a8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf19c4700 con 0x55fdf191c800 2023-11-08T16:34:50.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.370+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(33 [pgnls start_epoch 64 out=101b] v62'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b800 2023-11-08T16:34:50.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.370+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:34 9.13 9:cb24bee2:::100000001fa.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564e00 con 0x5557e513b800 2023-11-08T16:34:50.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.371+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(30 [pgnls start_epoch 64 out=101b] v62'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c800 2023-11-08T16:34:50.717 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '2', '--worker_m', '7'] in time 0:00:00.608797, result: 2023-11-08T16:34:50.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.371+0000 7f6eb9a1a640 1 -- 172.21.15.81:0/1154556006 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(39 100000001f8.00000000 [call,setxattr (8)] v64'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562b997d8fc0 con 0x562b98cc6800 2023-11-08T16:34:50.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.371+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:40 9.11 9:8f3fbbd0::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce7c00 con 0x562b98cac800 2023-11-08T16:34:50.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.371+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:31 9.15 9:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf19c4a80 con 0x55fdf191c800 2023-11-08T16:34:50.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.371+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(34 100000001fa.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b800 2023-11-08T16:34:50.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.371+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(40 [pgnls start_epoch 64 out=81b] v62'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cac800 2023-11-08T16:34:50.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.371+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628282.1 2023-11-08T16:34:50.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.371+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:41 9.11 9:8f3fbbd0:::1000000020c.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce6380 con 0x562b98cac800 2023-11-08T16:34:50.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.371+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:35 8.3f 8:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4565880 con 0x5557e510c400 2023-11-08T16:34:50.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.372+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(41 1000000020c.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cac800 2023-11-08T16:34:50.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.372+0000 7f6ebbe175c0 10 datascan.operator(): handling object 1099511628300.1 2023-11-08T16:34:50.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.372+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:42 8.1e 8:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce7180 con 0x562b98cadc00 2023-11-08T16:34:50.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.372+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(31 1000000020d.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c800 2023-11-08T16:34:50.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.372+0000 7fea58a395c0 10 datascan.operator(): handling object 1099511628301.0 2023-11-08T16:34:50.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.372+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:32 8.15 8:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf19c4e00 con 0x55fdf191c800 2023-11-08T16:34:50.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.374+0000 7fe8fe34b640 1 -- 172.21.15.81:0/1479994114 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(36 10000000209.00000000 [call,setxattr (8)] v64'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e1717a6fc0 con 0x55e170ca2400 2023-11-08T16:34:50.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.374+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:37 9.f 9:f0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8f180 con 0x55e170c77400 2023-11-08T16:34:50.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.374+0000 7f1913f66640 1 -- 172.21.15.81:0/159202259 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(35 100000001fa.00000000 [call,setxattr (8)] v64'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5557e5096d80 con 0x5557e510c400 2023-11-08T16:34:50.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.374+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(37 [pgnls start_epoch 64 out=81b] v60'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c77400 2023-11-08T16:34:50.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.374+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:36 9.13 9:cc6bf2c4::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4565c00 con 0x5557e513b800 2023-11-08T16:34:50.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.374+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:38 9.f 9:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e000 con 0x55e170c77400 2023-11-08T16:34:50.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(42 1000000020c.00000000 [call,setxattr (8)] v64'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562b997d8b40 con 0x562b98cadc00 2023-11-08T16:34:50.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:43 9.9 9:90000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562b98ce6000 con 0x562b98cc6000 2023-11-08T16:34:50.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6eba21b640 1 -- 172.21.15.81:0/1154556006 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(43 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562b997d8d80 con 0x562b98cc6000 2023-11-08T16:34:50.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.7 v2:172.21.15.93:6818/433691658 9 ==== osd_op_reply(36 [pgnls start_epoch 64 out=81b] v62'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b800 2023-11-08T16:34:50.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562b98cc6000 msgr2=0x562b98cbab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562b98cc6000 0x562b98cbab00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562b99879c00 msgr2=0x562b98996100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(38 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6b40 con 0x55e170c77400 2023-11-08T16:34:50.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562b99879c00 0x562b98996100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562b98cac800 msgr2=0x562b98cbc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562b98cac800 0x562b98cbc100 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562b98cadc00 msgr2=0x562b98cba580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562b98cadc00 0x562b98cba580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562b98cac400 msgr2=0x562b98995600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562b98cac400 0x562b98995600 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562b98cc6800 msgr2=0x562b98cbb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562b98cc6800 0x562b98cbb600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:37 9.13 9:cc6bf2c4:::1000000020b.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564380 con 0x5557e513b800 2023-11-08T16:34:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562b98cac000 msgr2=0x562b98995b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562b98cac000 0x562b98995b80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562b98cad800 msgr2=0x562b98cba000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562b98cad800 0x562b98cba000 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511627777.1 2023-11-08T16:34:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:39 8.7 8:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170947c00 con 0x55e170c76800 2023-11-08T16:34:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 reap_dead start 2023-11-08T16:34:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 reap_dead start 2023-11-08T16:34:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 reap_dead start 2023-11-08T16:34:50.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6eb9219640 1 -- 172.21.15.81:0/1154556006 reap_dead start 2023-11-08T16:34:50.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562b99879800 msgr2=0x562b98997700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562b99879800 0x562b98997700 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x562b99802c90 tx=0x562b99702360 comp rx=0 tx=0).stop 2023-11-08T16:34:50.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7400 msgr2=0x562b98996c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7400 0x562b98996c00 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x562b99803a40 tx=0x562b99702000 comp rx=0 tx=0).stop 2023-11-08T16:34:50.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 shutdown_connections 2023-11-08T16:34:50.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562b99879800 0x562b98997700 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1154556006 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b997f7400 0x562b98996c00 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 >> 172.21.15.81:0/1154556006 conn(0x562b997f6c00 msgr2=0x562b996a5800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.375+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(32 1000000020d.00000000 [call,setxattr (8)] v64'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c800 2023-11-08T16:34:50.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:33 9.15 9:ab3d427d::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf19c5180 con 0x55fdf191c800 2023-11-08T16:34:50.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 shutdown_connections 2023-11-08T16:34:50.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1154556006 wait complete. 2023-11-08T16:34:50.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.7 v2:172.21.15.93:6818/433691658 10 ==== osd_op_reply(37 1000000020b.00000001 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b800 2023-11-08T16:34:50.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628299.1 2023-11-08T16:34:50.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5557e513bc00 0x5557e4575080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:38 8.20 8:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564a80 con 0x5557e513bc00 2023-11-08T16:34:50.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(33 [pgnls start_epoch 64 out=101b] v62'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c800 2023-11-08T16:34:50.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.376+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:34 9.15 9:ab3d427d:::10000000203.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf19c5500 con 0x55fdf191c800 2023-11-08T16:34:50.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7f1912f64640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5557e513bc00 0x5557e4575080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1941472936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b9892ac00 msgr2=0x562b98994580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1941472936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b9892ac00 0x562b98994580 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x562b99737830 tx=0x562b99702300 comp rx=0 tx=0).stop 2023-11-08T16:34:50.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(34 10000000203.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c800 2023-11-08T16:34:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1941472936 shutdown_connections 2023-11-08T16:34:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1941472936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562b9892bc00 0x562b98995080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1941472936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562b9892b800 0x562b98994b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7f6ebbe175c0 1 --2- 172.21.15.81:0/1941472936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562b9892ac00 0x562b98994580 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7fea58a395c0 10 datascan.operator(): handling object 1099511628291.1 2023-11-08T16:34:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:35 8.2b 8:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf19c5880 con 0x55fdf189ac00 2023-11-08T16:34:50.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1941472936 >> 172.21.15.81:0/1941472936 conn(0x562b9892b000 msgr2=0x562b996a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 12 ==== osd_op_reply(39 10000000001.00000000 [call,setxattr (8)] v64'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.377+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:40 9.1f 9:f8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8ee00 con 0x55e170c76800 2023-11-08T16:34:50.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.378+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1941472936 shutdown_connections 2023-11-08T16:34:50.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.378+0000 7f6ebbe175c0 1 -- 172.21.15.81:0/1941472936 wait complete. 2023-11-08T16:34:50.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.378+0000 7f1912f64640 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5557e513bc00 0x5557e4575080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.378+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 13 ==== osd_op_reply(40 [pgnls start_epoch 64 out=81b] v62'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.378+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:41 9.1f 9:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170c8e700 con 0x55e170c76800 2023-11-08T16:34:50.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.378+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 14 ==== osd_op_reply(41 100000001fa.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.378+0000 7fe900c425c0 10 datascan.operator(): handling object 1099511628282.0 2023-11-08T16:34:50.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.378+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:42 8.3f 8:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55e170947880 con 0x55e170c76800 2023-11-08T16:34:50.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(35 10000000203.00000000 [call,setxattr (8)] v64'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf189ac00 2023-11-08T16:34:50.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:36 9.15 9:ab9be7ec::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf19c4380 con 0x55fdf191c800 2023-11-08T16:34:50.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe8ff34d640 1 -- 172.21.15.81:0/1479994114 <== osd.1 v2:172.21.15.81:6800/2060798739 15 ==== osd_op_reply(42 100000001fa.00000000 [call,setxattr (8)] v64'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55e1717a6d80 con 0x55e170c76800 2023-11-08T16:34:50.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55e170c76c00 msgr2=0x55e170959600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55e170c76c00 0x55e170959600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55e170c76800 msgr2=0x55e170959b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55e170c76800 0x55e170959b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55e170ca2000 msgr2=0x55e170c9a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55e170ca2000 0x55e170c9a680 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55e170c76400 msgr2=0x55e17095a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55e170c76400 0x55e17095a100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55e170c77400 msgr2=0x55e170c98580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55e170c77400 0x55e170c98580 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55e170ca2400 msgr2=0x55e170c98b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55e170ca2400 0x55e170c98b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 reap_dead start 2023-11-08T16:34:50.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 reap_dead start 2023-11-08T16:34:50.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe8feb4c640 1 -- 172.21.15.81:0/1479994114 reap_dead start 2023-11-08T16:34:50.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55e170c77000 msgr2=0x55e170c98000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55e170c77000 0x55e170c98000 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55e170c76000 msgr2=0x55e17095b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55e170c76000 0x55e17095b700 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x55e1709e09c0 tx=0x55e1716ced20 comp rx=0 tx=0).stop 2023-11-08T16:34:50.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5800 msgr2=0x55e17095a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5800 0x55e17095a680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55e1717c6bd0 tx=0x55e1716c6540 comp rx=0 tx=0).stop 2023-11-08T16:34:50.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 shutdown_connections 2023-11-08T16:34:50.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55e170ca2400 0x55e170c98b00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55e170c77000 0x55e170c98000 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55e170c76000 0x55e17095b700 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 --2- 172.21.15.81:0/1479994114 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1717e5800 0x55e17095a680 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 >> 172.21.15.81:0/1479994114 conn(0x55e1717e4c00 msgr2=0x55e171669800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 shutdown_connections 2023-11-08T16:34:50.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fe900c425c0 1 -- 172.21.15.81:0/1479994114 wait complete. 2023-11-08T16:34:50.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.381+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.7 v2:172.21.15.93:6818/433691658 9 ==== osd_op_reply(36 [pgnls start_epoch 64 out=81b] v62'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c800 2023-11-08T16:34:50.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:37 9.15 9:ab9be7ec:::10000000205.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf19c4700 con 0x55fdf191c800 2023-11-08T16:34:50.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(38 1000000020b.00000000 [call,setxattr (8)] v64'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5557e5096b40 con 0x5557e513bc00 2023-11-08T16:34:50.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:39 9.b 9:d0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564e00 con 0x5557e510c800 2023-11-08T16:34:50.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7fe900c425c0 1 -- 172.21.15.81:0/3787170624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1708ef800 msgr2=0x55e170958b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7fe900c425c0 1 --2- 172.21.15.81:0/3787170624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1708ef800 0x55e170958b00 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x55e1717057a0 tx=0x55e1716c6600 comp rx=0 tx=0).stop 2023-11-08T16:34:50.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.7 v2:172.21.15.93:6818/433691658 10 ==== osd_op_reply(37 10000000205.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c800 2023-11-08T16:34:50.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7fe900c425c0 1 -- 172.21.15.81:0/3787170624 shutdown_connections 2023-11-08T16:34:50.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7fe900c425c0 1 --2- 172.21.15.81:0/3787170624 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55e1708efc00 0x55e170959080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7fe900c425c0 1 --2- 172.21.15.81:0/3787170624 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55e1708ef800 0x55e170958b00 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7fe900c425c0 1 --2- 172.21.15.81:0/3787170624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e1708eec00 0x55e170958580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7fe900c425c0 1 -- 172.21.15.81:0/3787170624 >> 172.21.15.81:0/3787170624 conn(0x55e1708ef000 msgr2=0x55e171669000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 15 ==== osd_op_reply(39 [pgnls start_epoch 64 out=101b] v62'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:40 9.b 9:d2952c40:::10000000208.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4565c00 con 0x5557e510c800 2023-11-08T16:34:50.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7fea58a395c0 10 datascan.operator(): handling object 1099511628293.1 2023-11-08T16:34:50.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.382+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:38 8.3d 8:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199ce00 con 0x55fdf189b000 2023-11-08T16:34:50.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.383+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 16 ==== osd_op_reply(40 10000000208.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.383+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628296.1 2023-11-08T16:34:50.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.383+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:41 8.27 8:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4239880 con 0x5557e510c800 2023-11-08T16:34:50.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.383+0000 7fe900c425c0 1 -- 172.21.15.81:0/3787170624 shutdown_connections 2023-11-08T16:34:50.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.383+0000 7fe900c425c0 1 -- 172.21.15.81:0/3787170624 wait complete. 2023-11-08T16:34:50.761 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '3', '--worker_m', '7'] in time 0:00:00.651751, result: 2023-11-08T16:34:50.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.384+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(38 10000000205.00000000 [call,setxattr (8)] v64'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55fdf1800fc0 con 0x55fdf189b000 2023-11-08T16:34:50.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.384+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:39 9.d 9:b0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55fdf199c000 con 0x55fdf191c800 2023-11-08T16:34:50.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 17 ==== osd_op_reply(41 10000000208.00000000 [call,setxattr (8)] v64'6 uv6 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:42 9.b 9:d4a58307::::head [pgnls start_epoch 64 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4565180 con 0x5557e510c800 2023-11-08T16:34:50.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea56934640 1 -- 172.21.15.81:0/505794715 <== osd.7 v2:172.21.15.93:6818/433691658 11 ==== osd_op_reply(39 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55fdf1800b40 con 0x55fdf191c800 2023-11-08T16:34:50.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55fdf189b000 msgr2=0x55fdf1587b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55fdf189b000 0x55fdf1587b80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55fdf189b400 msgr2=0x55fdf1587600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55fdf189b400 0x55fdf1587600 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55fdf191c000 msgr2=0x55fdf1998000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55fdf191c000 0x55fdf1998000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55fdf191d000 msgr2=0x55fdf1999600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55fdf191d000 0x55fdf1999600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55fdf191c400 msgr2=0x55fdf1998580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55fdf191c400 0x55fdf1998580 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55fdf189ac00 msgr2=0x55fdf1588100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55fdf189ac00 0x55fdf1588100 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55fdf191c800 msgr2=0x55fdf1998b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55fdf191c800 0x55fdf1998b00 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 reap_dead start 2023-11-08T16:34:50.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 reap_dead start 2023-11-08T16:34:50.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea56133640 1 -- 172.21.15.81:0/505794715 reap_dead start 2023-11-08T16:34:50.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55fdf189a800 msgr2=0x55fdf1589700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55fdf189a800 0x55fdf1589700 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x55fdf22ee780 tx=0x55fdf18080c0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf181d800 msgr2=0x55fdf1588680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf181d800 0x55fdf1588680 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x55fdf22ef500 tx=0x55fdf22f2c00 comp rx=0 tx=0).stop 2023-11-08T16:34:50.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.385+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 18 ==== osd_op_reply(42 [pgnls start_epoch 64 out=81b] v62'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 shutdown_connections 2023-11-08T16:34:50.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55fdf189a800 0x55fdf1589700 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7fea58a395c0 1 --2- 172.21.15.81:0/505794715 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf181d800 0x55fdf1588680 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 >> 172.21.15.81:0/505794715 conn(0x55fdf181cc00 msgr2=0x55fdf22a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:43 9.b 9:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564000 con 0x5557e510c800 2023-11-08T16:34:50.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 shutdown_connections 2023-11-08T16:34:50.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7fea58a395c0 1 -- 172.21.15.81:0/505794715 wait complete. 2023-11-08T16:34:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.0 v2:172.21.15.81:6808/1666604866 19 ==== osd_op_reply(43 10000000203.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x5557e5096b40 con 0x5557e510c800 2023-11-08T16:34:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7f1915c0c5c0 10 datascan.operator(): handling object 1099511628291.0 2023-11-08T16:34:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:44 8.2b 8:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4564700 con 0x5557e513b400 2023-11-08T16:34:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7fea58a395c0 1 -- 172.21.15.81:0/2991698649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf151cc00 msgr2=0x55fdf1586580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.386+0000 7fea58a395c0 1 --2- 172.21.15.81:0/2991698649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf151cc00 0x55fdf1586580 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55fdf2323380 tx=0x55fdf1808420 comp rx=0 tx=0).stop 2023-11-08T16:34:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.387+0000 7fea58a395c0 1 -- 172.21.15.81:0/2991698649 shutdown_connections 2023-11-08T16:34:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.387+0000 7fea58a395c0 1 --2- 172.21.15.81:0/2991698649 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fdf151dc00 0x55fdf1587080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.387+0000 7fea58a395c0 1 --2- 172.21.15.81:0/2991698649 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fdf151d800 0x55fdf1586b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.387+0000 7fea58a395c0 1 --2- 172.21.15.81:0/2991698649 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fdf151cc00 0x55fdf1586580 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.387+0000 7fea58a395c0 1 -- 172.21.15.81:0/2991698649 >> 172.21.15.81:0/2991698649 conn(0x55fdf151d000 msgr2=0x55fdf22a6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.387+0000 7fea58a395c0 1 -- 172.21.15.81:0/2991698649 shutdown_connections 2023-11-08T16:34:50.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.387+0000 7fea58a395c0 1 -- 172.21.15.81:0/2991698649 wait complete. 2023-11-08T16:34:50.777 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '6', '--worker_m', '7'] in time 0:00:00.666031, result: 2023-11-08T16:34:50.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.389+0000 7f1912f64640 1 -- 172.21.15.81:0/159202259 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(44 10000000203.00000000 [call,setxattr (8)] v64'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x5557e5096b40 con 0x5557e513b400 2023-11-08T16:34:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.389+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:45 9.1b 9:d8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5557e4565500 con 0x5557e513b000 2023-11-08T16:34:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(45 [pgnls start_epoch 64 out=81b] v62'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x5557e5096fc0 con 0x5557e513b000 2023-11-08T16:34:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5557e510c800 msgr2=0x5557e424c100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5557e510c800 0x5557e424c100 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5557e510c400 msgr2=0x5557e424bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5557e510c400 0x5557e424bb80 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5557e513b000 msgr2=0x5557e4574000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5557e513b000 0x5557e4574000 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5557e510c000 msgr2=0x5557e424b600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x5557e510c000 0x5557e424b600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 reap_dead start 2023-11-08T16:34:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5557e513bc00 msgr2=0x5557e4575080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x5557e513bc00 0x5557e4575080 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5557e513b400 msgr2=0x5557e4574580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5557e513b400 0x5557e4574580 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5557e513b800 msgr2=0x5557e4574b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5557e513b800 0x5557e4574b00 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1913765640 1 -- 172.21.15.81:0/159202259 reap_dead start 2023-11-08T16:34:50.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5557e513ac00 msgr2=0x5557e424d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5557e513ac00 0x5557e424d700 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x5557e4fb57a0 tx=0x5557e4fb0180 comp rx=0 tx=0).stop 2023-11-08T16:34:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e510d000 msgr2=0x5557e424cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e510d000 0x5557e424cc00 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x5557e4fb5860 tx=0x5557e50aed20 comp rx=0 tx=0).stop 2023-11-08T16:34:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 shutdown_connections 2023-11-08T16:34:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5557e513ac00 0x5557e424d700 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/159202259 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e510d000 0x5557e424cc00 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.390+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 >> 172.21.15.81:0/159202259 conn(0x5557e4fcbc00 msgr2=0x5557e42dd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.391+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 shutdown_connections 2023-11-08T16:34:50.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.391+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/159202259 wait complete. 2023-11-08T16:34:50.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.391+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1618399180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e41e1c00 msgr2=0x5557e424b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.391+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/1618399180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e41e1c00 0x5557e424b080 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x5557e4ff8b40 tx=0x5557e4fb0360 comp rx=0 tx=0).stop 2023-11-08T16:34:50.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.391+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1618399180 shutdown_connections 2023-11-08T16:34:50.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.391+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/1618399180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5557e41e1c00 0x5557e424b080 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.391+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/1618399180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5557e41e1800 0x5557e424ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.391+0000 7f1915c0c5c0 1 --2- 172.21.15.81:0/1618399180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5557e41e0c00 0x5557e424a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.391+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1618399180 >> 172.21.15.81:0/1618399180 conn(0x5557e41e1000 msgr2=0x5557e42dd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.391+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1618399180 shutdown_connections 2023-11-08T16:34:50.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.391+0000 7f1915c0c5c0 1 -- 172.21.15.81:0/1618399180 wait complete. 2023-11-08T16:34:50.787 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '4', '--worker_m', '7'] in time 0:00:00.677088, result: 2023-11-08T16:34:50.787 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--worker_n', '5', '--worker_m', '7'] in time 0:00:00.677395, result: 2023-11-08T16:34:50.788 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 0 --worker_m 7 2023-11-08T16:34:50.788 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 1 --worker_m 7 2023-11-08T16:34:50.789 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 2 --worker_m 7 2023-11-08T16:34:50.790 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 3 --worker_m 7 2023-11-08T16:34:50.790 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 4 --worker_m 7 2023-11-08T16:34:50.791 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 5 --worker_m 7 2023-11-08T16:34:50.791 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --worker_n 6 --worker_m 7 2023-11-08T16:34:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.924+0000 7fb3878145c0 1 -- start start 2023-11-08T16:34:50.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.924+0000 7fb3878145c0 1 Processor -- start 2023-11-08T16:34:50.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.925+0000 7fb3878145c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614216aec00 0x561421718580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.925+0000 7fb3878145c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614216af800 0x561421718b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.925+0000 7fb3878145c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5614216afc00 0x561421719080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.925+0000 7fb3878145c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614216f7180 con 0x5614216aec00 2023-11-08T16:34:50.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.925+0000 7fb3878145c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142177d180 con 0x5614216af800 2023-11-08T16:34:50.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.926+0000 7fb3878145c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56142177cdc0 con 0x5614216afc00 2023-11-08T16:34:50.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.926+0000 7fb385ce8640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5614216afc00 0x561421719080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.926+0000 7fb385ce8640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5614216afc00 0x561421719080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:35728/0 (socket says 172.21.15.81:35728) 2023-11-08T16:34:50.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.926+0000 7fb385ce8640 1 -- 172.21.15.81:0/4118159804 learned_addr learned my addr 172.21.15.81:0/4118159804 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.926+0000 7fb3854e7640 1 --2- 172.21.15.81:0/4118159804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614216aec00 0x561421718580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.926+0000 7fb384ce6640 1 --2- 172.21.15.81:0/4118159804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614216af800 0x561421718b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.927+0000 7fb3854e7640 1 -- 172.21.15.81:0/4118159804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5614216afc00 msgr2=0x561421719080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.927+0000 7fb3854e7640 1 --2- 172.21.15.81:0/4118159804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5614216afc00 0x561421719080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.927+0000 7fb3854e7640 1 -- 172.21.15.81:0/4118159804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614216af800 msgr2=0x561421718b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.927+0000 7fb3854e7640 1 --2- 172.21.15.81:0/4118159804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614216af800 0x561421718b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.927+0000 7fb3854e7640 1 -- 172.21.15.81:0/4118159804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614216d49c0 con 0x5614216aec00 2023-11-08T16:34:50.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.928+0000 7f3d3a2175c0 1 -- start start 2023-11-08T16:34:50.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.928+0000 7f3d3a2175c0 1 Processor -- start 2023-11-08T16:34:50.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.928+0000 7fa23642b5c0 1 -- start start 2023-11-08T16:34:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.928+0000 7fa23642b5c0 1 Processor -- start 2023-11-08T16:34:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.928+0000 7fb3854e7640 1 --2- 172.21.15.81:0/4118159804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614216aec00 0x561421718580 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x5614224bca80 tx=0x5614224822a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bdc6465b542a77 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.928+0000 7fb382ce2640 1 -- 172.21.15.81:0/4118159804 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614216f4f20 con 0x5614216aec00 2023-11-08T16:34:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.928+0000 7fb382ce2640 1 -- 172.21.15.81:0/4118159804 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561422484000 con 0x5614216aec00 2023-11-08T16:34:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.928+0000 7fb382ce2640 1 -- 172.21.15.81:0/4118159804 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614216f51e0 con 0x5614216aec00 2023-11-08T16:34:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.929+0000 7fb3878145c0 1 -- 172.21.15.81:0/4118159804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614216d4d00 con 0x5614216aec00 2023-11-08T16:34:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.929+0000 7fb3878145c0 1 -- 172.21.15.81:0/4118159804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5614216d49c0 con 0x5614216aec00 2023-11-08T16:34:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.929+0000 7fb3878145c0 4 waiting for MDS map... 2023-11-08T16:34:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7fb382ce2640 1 -- 172.21.15.81:0/4118159804 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56142176ec40 con 0x5614216aec00 2023-11-08T16:34:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7fa23642b5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562635d2ec00 0x562635d98580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7fb382ce2640 1 -- 172.21.15.81:0/4118159804 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x5614216fef00 con 0x5614216aec00 2023-11-08T16:34:50.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7fa23642b5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562635d2f800 0x562635d98b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7fa23642b5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562635d2fc00 0x562635d99080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7fa23642b5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562635d77180 con 0x562635d2ec00 2023-11-08T16:34:50.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7fa23642b5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562635d76140 con 0x562635d2fc00 2023-11-08T16:34:50.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7fa23642b5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562635d76000 con 0x562635d2f800 2023-11-08T16:34:50.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7fb3878145c0 4 Got MDS map 47 2023-11-08T16:34:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7fb3878145c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7fb3878145c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7f3d3a2175c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6800 0x56091b065b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7f3d3a2175c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091bdd7c00 0x56091b065600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7f3d3a2175c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091bdd7400 0x56091b065080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7f3d3a2175c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091b043180 con 0x56091bdd6800 2023-11-08T16:34:50.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7f3d3a2175c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091b042b40 con 0x56091bdd7c00 2023-11-08T16:34:50.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.930+0000 7f3d3a2175c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56091b042140 con 0x56091bdd7400 2023-11-08T16:34:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.931+0000 7f3d37d96640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6800 0x56091b065b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.931+0000 7fa2344c8640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562635d2fc00 0x562635d99080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.931+0000 7f3d37d96640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6800 0x56091b065b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54850/0 (socket says 172.21.15.81:54850) 2023-11-08T16:34:50.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.931+0000 7f3d37d96640 1 -- 172.21.15.81:0/2762196039 learned_addr learned my addr 172.21.15.81:0/2762196039 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.931+0000 7fa2344c8640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562635d2fc00 0x562635d99080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:56902/0 (socket says 172.21.15.81:56902) 2023-11-08T16:34:50.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.931+0000 7fa2344c8640 1 -- 172.21.15.81:0/2416983088 learned_addr learned my addr 172.21.15.81:0/2416983088 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.931+0000 7fa233cc7640 1 --2- 172.21.15.81:0/2416983088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562635d2ec00 0x562635d98580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.931+0000 7f3d37595640 1 --2- 172.21.15.81:0/2762196039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091bdd7c00 0x56091b065600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.931+0000 7fb3878145c0 1 Processor -- start 2023-11-08T16:34:50.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.931+0000 7fa2334c6640 1 --2- 172.21.15.81:0/2416983088 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562635d2f800 0x562635d98b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7f3d38597640 1 --2- 172.21.15.81:0/2762196039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091bdd7400 0x56091b065080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7f3d37d96640 1 -- 172.21.15.81:0/2762196039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091bdd7400 msgr2=0x56091b065080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7fb3878145c0 1 -- start start 2023-11-08T16:34:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7f3d37d96640 1 --2- 172.21.15.81:0/2762196039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091bdd7400 0x56091b065080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7f3d37d96640 1 -- 172.21.15.81:0/2762196039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091bdd7c00 msgr2=0x56091b065600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7f3d37d96640 1 --2- 172.21.15.81:0/2762196039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091bdd7c00 0x56091b065600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7f3d37d96640 1 -- 172.21.15.81:0/2762196039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56091b020d00 con 0x56091bdd6800 2023-11-08T16:34:50.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7fa233cc7640 1 -- 172.21.15.81:0/2416983088 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562635d2f800 msgr2=0x562635d98b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7fa233cc7640 1 --2- 172.21.15.81:0/2416983088 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562635d2f800 0x562635d98b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7fa233cc7640 1 -- 172.21.15.81:0/2416983088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562635d2fc00 msgr2=0x562635d99080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7fa233cc7640 1 --2- 172.21.15.81:0/2416983088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562635d2fc00 0x562635d99080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7fa233cc7640 1 -- 172.21.15.81:0/2416983088 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562635d549c0 con 0x562635d2ec00 2023-11-08T16:34:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.932+0000 7f3d37595640 1 --2- 172.21.15.81:0/2762196039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091bdd7c00 0x56091b065600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:34:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.933+0000 7f3d38597640 1 --2- 172.21.15.81:0/2762196039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091bdd7400 0x56091b065080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.933+0000 7f3d37d96640 1 --2- 172.21.15.81:0/2762196039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6800 0x56091b065b80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x56091be1d290 tx=0x56091bde0ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6081e434604b0c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.933+0000 7fa233cc7640 1 --2- 172.21.15.81:0/2416983088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562635d2ec00 0x562635d98580 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x562636b313b0 tx=0x562636be4420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d4d411660aa4e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.933+0000 7fa2314c2640 1 -- 172.21.15.81:0/2416983088 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562635d751e0 con 0x562635d2ec00 2023-11-08T16:34:50.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.933+0000 7fa2314c2640 1 -- 172.21.15.81:0/2416983088 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562635d88180 con 0x562635d2ec00 2023-11-08T16:34:50.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.933+0000 7fa2314c2640 1 -- 172.21.15.81:0/2416983088 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562635d75080 con 0x562635d2ec00 2023-11-08T16:34:50.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.933+0000 7f3d35591640 1 -- 172.21.15.81:0/2762196039 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56091b040f20 con 0x56091bdd6800 2023-11-08T16:34:50.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7f3d35591640 1 -- 172.21.15.81:0/2762196039 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56091bed2000 con 0x56091bdd6800 2023-11-08T16:34:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7f3d35591640 1 -- 172.21.15.81:0/2762196039 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56091b0411e0 con 0x56091bdd6800 2023-11-08T16:34:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2416983088 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562635d54b60 con 0x562635d2ec00 2023-11-08T16:34:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2416983088 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562635d54820 con 0x562635d2ec00 2023-11-08T16:34:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7fa23642b5c0 4 waiting for MDS map... 2023-11-08T16:34:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7fb3878145c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248fc00 0x561421719600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7fb3878145c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614225d4000 0x561421719b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7fb3878145c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614225d4400 0x56142171a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7fb3878145c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614216f6a00 con 0x5614225d4000 2023-11-08T16:34:50.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2762196039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56091b0209c0 con 0x56091bdd6800 2023-11-08T16:34:50.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2762196039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56091b020820 con 0x56091bdd6800 2023-11-08T16:34:50.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7f3d3a2175c0 4 waiting for MDS map... 2023-11-08T16:34:50.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7fb3878145c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614216f6b40 con 0x5614225d4400 2023-11-08T16:34:50.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7fb3878145c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56142177cdc0 con 0x56142248fc00 2023-11-08T16:34:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.934+0000 7fa2314c2640 1 -- 172.21.15.81:0/2416983088 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x562635deec40 con 0x562635d2ec00 2023-11-08T16:34:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.935+0000 7fa2314c2640 1 -- 172.21.15.81:0/2416983088 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x562636bf8000 con 0x562635d2ec00 2023-11-08T16:34:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.935+0000 7fb385ce8640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248fc00 0x561421719600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.935+0000 7fb385ce8640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248fc00 0x561421719600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:35766/0 (socket says 172.21.15.81:35766) 2023-11-08T16:34:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.935+0000 7fa23642b5c0 4 Got MDS map 47 2023-11-08T16:34:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.935+0000 7fb385ce8640 1 -- 172.21.15.81:0/39562988 learned_addr learned my addr 172.21.15.81:0/39562988 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.935+0000 7fb384ce6640 1 --2- 172.21.15.81:0/39562988 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614225d4400 0x56142171a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.935+0000 7fa23642b5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:50.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.935+0000 7fa23642b5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:50.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.935+0000 7fb3854e7640 1 --2- 172.21.15.81:0/39562988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614225d4000 0x561421719b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7fb385ce8640 1 -- 172.21.15.81:0/39562988 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614225d4400 msgr2=0x56142171a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7fb385ce8640 1 --2- 172.21.15.81:0/39562988 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614225d4400 0x56142171a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7fb385ce8640 1 -- 172.21.15.81:0/39562988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614225d4000 msgr2=0x561421719b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7fb385ce8640 1 --2- 172.21.15.81:0/39562988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614225d4000 0x561421719b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7fb385ce8640 1 -- 172.21.15.81:0/39562988 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614216d4b60 con 0x56142248fc00 2023-11-08T16:34:50.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7f3d35591640 1 -- 172.21.15.81:0/2762196039 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56091b0bac40 con 0x56091bdd6800 2023-11-08T16:34:50.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7f3d35591640 1 -- 172.21.15.81:0/2762196039 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x56091b04af00 con 0x56091bdd6800 2023-11-08T16:34:50.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7fb385ce8640 1 --2- 172.21.15.81:0/39562988 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248fc00 0x561421719600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x561422582c90 tx=0x561422586c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ff258785f53f0d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7f3d3a2175c0 4 Got MDS map 47 2023-11-08T16:34:50.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7fa23642b5c0 1 Processor -- start 2023-11-08T16:34:50.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7fa23642b5c0 1 -- start start 2023-11-08T16:34:50.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7f3d3a2175c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:50.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.936+0000 7f3d3a2175c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:50.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.937+0000 7fb380cde640 1 -- 172.21.15.81:0/39562988 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614216f4f20 con 0x56142248fc00 2023-11-08T16:34:50.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.937+0000 7fb380cde640 1 -- 172.21.15.81:0/39562988 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561422484c00 con 0x56142248fc00 2023-11-08T16:34:50.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.937+0000 7fb380cde640 1 -- 172.21.15.81:0/39562988 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614216f51e0 con 0x56142248fc00 2023-11-08T16:34:50.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.937+0000 7fa23642b5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03000 0x562635d99600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.937+0000 7fa23642b5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562636c03400 0x562635d99b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fa23642b5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562636c03800 0x562635d9a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fa23642b5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562635d76a00 con 0x562636c03000 2023-11-08T16:34:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fa23642b5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562635dfd180 con 0x562636c03400 2023-11-08T16:34:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fa23642b5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562635dfcdc0 con 0x562636c03800 2023-11-08T16:34:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fb3878145c0 1 -- 172.21.15.81:0/39562988 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248fc00 msgr2=0x561421719600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fb3878145c0 1 --2- 172.21.15.81:0/39562988 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248fc00 0x561421719600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x561422582c90 tx=0x561422586c00 comp rx=0 tx=0).stop 2023-11-08T16:34:50.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fb3878145c0 1 -- 172.21.15.81:0/39562988 shutdown_connections 2023-11-08T16:34:50.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fb3878145c0 1 --2- 172.21.15.81:0/39562988 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614225d4400 0x56142171a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fb3878145c0 1 --2- 172.21.15.81:0/39562988 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614225d4000 0x561421719b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fb3878145c0 1 --2- 172.21.15.81:0/39562988 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248fc00 0x561421719600 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fb3878145c0 1 -- 172.21.15.81:0/39562988 >> 172.21.15.81:0/39562988 conn(0x56142248f800 msgr2=0x561422429800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7f3d3a2175c0 1 Processor -- start 2023-11-08T16:34:50.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fa2344c8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03000 0x562635d99600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fa2344c8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03000 0x562635d99600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54874/0 (socket says 172.21.15.81:54874) 2023-11-08T16:34:50.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fa2344c8640 1 -- 172.21.15.81:0/793950134 learned_addr learned my addr 172.21.15.81:0/793950134 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fb3878145c0 1 -- 172.21.15.81:0/39562988 shutdown_connections 2023-11-08T16:34:50.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.939+0000 7f3d3a2175c0 1 -- start start 2023-11-08T16:34:50.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.939+0000 7fa2334c6640 1 --2- 172.21.15.81:0/793950134 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562636c03800 0x562635d9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.938+0000 7fb3878145c0 1 -- 172.21.15.81:0/39562988 wait complete. 2023-11-08T16:34:50.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.939+0000 7fa233cc7640 1 --2- 172.21.15.81:0/793950134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562636c03400 0x562635d99b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.939+0000 7fa2334c6640 1 -- 172.21.15.81:0/793950134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562636c03400 msgr2=0x562635d99b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.939+0000 7fa2334c6640 1 --2- 172.21.15.81:0/793950134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562636c03400 0x562635d99b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.939+0000 7fa2334c6640 1 -- 172.21.15.81:0/793950134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03000 msgr2=0x562635d99600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.939+0000 7fa2334c6640 1 --2- 172.21.15.81:0/793950134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03000 0x562635d99600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.939+0000 7fa2334c6640 1 -- 172.21.15.81:0/793950134 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562635d544e0 con 0x562636c03800 2023-11-08T16:34:50.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.939+0000 7fb3878145c0 1 Processor -- start 2023-11-08T16:34:50.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.940+0000 7fb3878145c0 1 -- start start 2023-11-08T16:34:50.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.940+0000 7fa2334c6640 1 --2- 172.21.15.81:0/793950134 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562636c03800 0x562635d9a100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x562636beaba0 tx=0x562636c14c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c307fbcd762f1ca server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.940+0000 7f3d3a2175c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6400 0x56091b066100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.940+0000 7f3d3a2175c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091becc800 0x56091b066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.940+0000 7f3d3a2175c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091beccc00 0x56091b066c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.940+0000 7f3d3a2175c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091b042a00 con 0x56091bdd6400 2023-11-08T16:34:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.940+0000 7f3d3a2175c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091b0c8dc0 con 0x56091becc800 2023-11-08T16:34:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.940+0000 7f3d3a2175c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56091b0c9180 con 0x56091beccc00 2023-11-08T16:34:50.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.940+0000 7fa22f4be640 1 -- 172.21.15.81:0/793950134 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562635d751e0 con 0x562636c03800 2023-11-08T16:34:50.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7fa22f4be640 1 -- 172.21.15.81:0/793950134 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562635ea9080 con 0x562636c03800 2023-11-08T16:34:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.940+0000 7f3d38597640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6400 0x56091b066100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7f3d38597640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6400 0x56091b066100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54880/0 (socket says 172.21.15.81:54880) 2023-11-08T16:34:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7f3d38597640 1 -- 172.21.15.81:0/2208529789 learned_addr learned my addr 172.21.15.81:0/2208529789 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7fb3878145c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248f800 0x56142171b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7fb3878145c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614225d5000 0x56142171ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7fb3878145c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614225d4c00 0x56142171a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7fb3878145c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142177cc80 con 0x5614225d5000 2023-11-08T16:34:50.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7fb3878145c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56142177cb40 con 0x5614225d4c00 2023-11-08T16:34:50.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7fb3878145c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56142177ca00 con 0x56142248f800 2023-11-08T16:34:50.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7fb385ce8640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248f800 0x56142171b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7fb385ce8640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248f800 0x56142171b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:35790/0 (socket says 172.21.15.81:35790) 2023-11-08T16:34:50.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 learned_addr learned my addr 172.21.15.81:0/3305262764 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7fb3854e7640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614225d5000 0x56142171ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7f3d37d96640 1 --2- 172.21.15.81:0/2208529789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091becc800 0x56091b066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7f3d37595640 1 --2- 172.21.15.81:0/2208529789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091beccc00 0x56091b066c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7f3d37d96640 1 -- 172.21.15.81:0/2208529789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091beccc00 msgr2=0x56091b066c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7f3d37d96640 1 --2- 172.21.15.81:0/2208529789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091beccc00 0x56091b066c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7f3d37d96640 1 -- 172.21.15.81:0/2208529789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6400 msgr2=0x56091b066100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7f3d37d96640 1 --2- 172.21.15.81:0/2208529789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6400 0x56091b066100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7f3d37d96640 1 -- 172.21.15.81:0/2208529789 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56091b0204e0 con 0x56091becc800 2023-11-08T16:34:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.941+0000 7fa22f4be640 1 -- 172.21.15.81:0/793950134 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562635d75080 con 0x562636c03800 2023-11-08T16:34:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7fa23642b5c0 1 -- 172.21.15.81:0/793950134 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562636c03800 msgr2=0x562635d9a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/793950134 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562636c03800 0x562635d9a100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x562636beaba0 tx=0x562636c14c00 comp rx=0 tx=0).stop 2023-11-08T16:34:50.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7f7f0de0e5c0 1 -- start start 2023-11-08T16:34:50.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7f7f0de0e5c0 1 Processor -- start 2023-11-08T16:34:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614225d4c00 msgr2=0x56142171a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7fb384ce6640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614225d4c00 0x56142171a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.942+0000 7fb385ce8640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614225d4c00 0x56142171a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614225d5000 msgr2=0x56142171ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7fb385ce8640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614225d5000 0x56142171ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614216d4820 con 0x56142248f800 2023-11-08T16:34:50.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7fa23642b5c0 1 -- 172.21.15.81:0/793950134 shutdown_connections 2023-11-08T16:34:50.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/793950134 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562636c03800 0x562635d9a100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/793950134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562636c03400 0x562635d99b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/793950134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03000 0x562635d99600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7fa23642b5c0 1 -- 172.21.15.81:0/793950134 >> 172.21.15.81:0/793950134 conn(0x562636c02c00 msgr2=0x562636aab000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7fa23642b5c0 1 -- 172.21.15.81:0/793950134 shutdown_connections 2023-11-08T16:34:50.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7fa23642b5c0 1 -- 172.21.15.81:0/793950134 wait complete. 2023-11-08T16:34:50.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7fb385ce8640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248f800 0x56142171b180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x561422583950 tx=0x561422482000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.943+0000 7f3d37d96640 1 --2- 172.21.15.81:0/2208529789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091becc800 0x56091b066680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x56091be1d710 tx=0x56091bd8a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d55fd6a9b8907680 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7fb37f4db640 1 -- 172.21.15.81:0/3305262764 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614216f4420 con 0x56142248f800 2023-11-08T16:34:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7fb37f4db640 1 -- 172.21.15.81:0/3305262764 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561422484a80 con 0x56142248f800 2023-11-08T16:34:50.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7fb37f4db640 1 -- 172.21.15.81:0/3305262764 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5614216f4f20 con 0x56142248f800 2023-11-08T16:34:50.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f7f0de0e5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0b3ec00 0x55def0ba8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f7f0de0e5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0b3f800 0x55def0ba8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f7f0de0e5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0b3fc00 0x55def0ba9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f7f0de0e5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def0b87180 con 0x55def0b3ec00 2023-11-08T16:34:50.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f7f0de0e5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def0c0d180 con 0x55def0b3f800 2023-11-08T16:34:50.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f7f0de0e5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55def0c0cdc0 con 0x55def0b3fc00 2023-11-08T16:34:50.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7fa23642b5c0 1 Processor -- start 2023-11-08T16:34:50.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7fa23642b5c0 1 -- start start 2023-11-08T16:34:50.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f3d3358d640 1 -- 172.21.15.81:0/2208529789 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56091b040f20 con 0x56091becc800 2023-11-08T16:34:50.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f3d3358d640 1 -- 172.21.15.81:0/2208529789 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56091bed2c00 con 0x56091becc800 2023-11-08T16:34:50.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f3d3358d640 1 -- 172.21.15.81:0/2208529789 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56091b0411e0 con 0x56091becc800 2023-11-08T16:34:50.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f7f0b9f6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0b3ec00 0x55def0ba8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f7f0b9f6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0b3ec00 0x55def0ba8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54916/0 (socket says 172.21.15.81:54916) 2023-11-08T16:34:50.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.944+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/2245676871 learned_addr learned my addr 172.21.15.81:0/2245676871 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:50.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.945+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/2245676871 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0b3fc00 0x55def0ba9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:50.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.945+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/2245676871 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0b3fc00 msgr2=0x55def0ba9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:50.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.945+0000 7f7f0b9f6640 1 --2- 172.21.15.81:0/2245676871 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0b3fc00 0x55def0ba9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.945+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/2245676871 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0b3f800 msgr2=0x55def0ba8b00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T16:34:50.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.945+0000 7f7f0b9f6640 1 --2- 172.21.15.81:0/2245676871 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0b3f800 0x55def0ba8b00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.945+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/2245676871 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55def0b649c0 con 0x55def0b3ec00 2023-11-08T16:34:50.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.945+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/2245676871 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0b3fc00 0x55def0ba9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:34:50.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.945+0000 7f7f0b9f6640 1 --2- 172.21.15.81:0/2245676871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0b3ec00 0x55def0ba8580 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x55def19433e0 tx=0x55def0e1a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c749dd68e7aa1843 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:50.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2208529789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091becc800 msgr2=0x56091b066680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:50.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/2208529789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091becc800 0x56091b066680 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x56091be1d710 tx=0x56091bd8a060 comp rx=0 tx=0).stop 2023-11-08T16:34:50.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f7f091f1640 1 -- 172.21.15.81:0/2245676871 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55def0b851e0 con 0x55def0b3ec00 2023-11-08T16:34:50.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/2245676871 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55def0b64b60 con 0x55def0b3ec00 2023-11-08T16:34:50.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/2245676871 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55def0b64820 con 0x55def0b3ec00 2023-11-08T16:34:50.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f7f0de0e5c0 4 waiting for MDS map... 2023-11-08T16:34:50.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f7f091f1640 1 -- 172.21.15.81:0/2245676871 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55def190aa80 con 0x55def0b3ec00 2023-11-08T16:34:50.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2208529789 shutdown_connections 2023-11-08T16:34:50.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/2208529789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091beccc00 0x56091b066c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/2208529789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091becc800 0x56091b066680 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f7f091f1640 1 -- 172.21.15.81:0/2245676871 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55def0b85080 con 0x55def0b3ec00 2023-11-08T16:34:50.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7fa23642b5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562636c03000 0x562635d9b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/2208529789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6400 0x56091b066100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:50.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2208529789 >> 172.21.15.81:0/2208529789 conn(0x56091bdd7800 msgr2=0x56091b0f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:50.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7fa23642b5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562636c03400 0x562635d9ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7fa23642b5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03800 0x562635d9a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:50.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7fa23642b5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562635dfcc80 con 0x562636c03800 2023-11-08T16:34:50.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7fa23642b5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562635d76140 con 0x562636c03400 2023-11-08T16:34:50.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7fa23642b5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562635d76b40 con 0x562636c03000 2023-11-08T16:34:50.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f7f091f1640 1 -- 172.21.15.81:0/2245676871 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55def0bfec40 con 0x55def0b3ec00 2023-11-08T16:34:50.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7f7f091f1640 1 -- 172.21.15.81:0/2245676871 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55def0b8ef00 con 0x55def0b3ec00 2023-11-08T16:34:50.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.946+0000 7fa2334c6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03800 0x562635d9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7fa2334c6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03800 0x562635d9a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54918/0 (socket says 172.21.15.81:54918) 2023-11-08T16:34:51.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7fa2334c6640 1 -- 172.21.15.81:0/2688234169 learned_addr learned my addr 172.21.15.81:0/2688234169 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.945+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5614216d44e0 con 0x56142248f800 2023-11-08T16:34:51.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.945+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5614216d41a0 con 0x56142248f800 2023-11-08T16:34:51.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.945+0000 7fb3878145c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:51.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7f7f0de0e5c0 4 Got MDS map 47 2023-11-08T16:34:51.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7f7f0de0e5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:51.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7f7f0de0e5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:51.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7fa2344c8640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562636c03000 0x562635d9b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7fa2334c6640 1 -- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562636c03000 msgr2=0x562635d9b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7fa2334c6640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562636c03000 0x562635d9b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7fa2334c6640 1 -- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562636c03400 msgr2=0x562635d9ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7fa2334c6640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562636c03400 0x562635d9ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7fa2334c6640 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562635d549c0 con 0x562636c03800 2023-11-08T16:34:51.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2208529789 shutdown_connections 2023-11-08T16:34:51.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.947+0000 7fa2334c6640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03800 0x562635d9a680 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x562636beb860 tx=0x562636abf8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.948+0000 7fa22dcbb640 1 -- 172.21.15.81:0/2688234169 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562635d74420 con 0x562636c03800 2023-11-08T16:34:51.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.948+0000 7fa22dcbb640 1 -- 172.21.15.81:0/2688234169 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562635d88480 con 0x562636c03800 2023-11-08T16:34:51.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.948+0000 7fa22dcbb640 1 -- 172.21.15.81:0/2688234169 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562635d751e0 con 0x562636c03800 2023-11-08T16:34:51.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.948+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562635d54b60 con 0x562636c03800 2023-11-08T16:34:51.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.948+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2208529789 wait complete. 2023-11-08T16:34:51.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.949+0000 7f3d3a2175c0 1 Processor -- start 2023-11-08T16:34:51.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.948+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562635d541a0 con 0x562636c03800 2023-11-08T16:34:51.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.948+0000 7fa23642b5c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:51.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.948+0000 7f7f0de0e5c0 1 Processor -- start 2023-11-08T16:34:51.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.948+0000 7f7f0de0e5c0 1 -- start start 2023-11-08T16:34:51.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.949+0000 7f3d3a2175c0 1 -- start start 2023-11-08T16:34:51.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.949+0000 7fb37f4db640 1 -- 172.21.15.81:0/3305262764 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x56142170b080 con 0x56142248f800 2023-11-08T16:34:51.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.949+0000 7f7f0de0e5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0e33000 0x55def0ba9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.949+0000 7f7f0de0e5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0e33400 0x55def0ba9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.949+0000 7f7f0de0e5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 0x55def0baa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.949+0000 7f7f0de0e5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def0b86140 con 0x55def0e33000 2023-11-08T16:34:51.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.949+0000 7f7f0de0e5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def0b86b40 con 0x55def0e33400 2023-11-08T16:34:51.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.949+0000 7f7f0de0e5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55def0c0d180 con 0x55def0e33800 2023-11-08T16:34:51.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.950+0000 7f7f0c1f7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0e33000 0x55def0ba9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.950+0000 7f7f0c1f7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0e33000 0x55def0ba9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54928/0 (socket says 172.21.15.81:54928) 2023-11-08T16:34:51.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.950+0000 7fb37f4db640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x561421a18000 0x56142171b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.950+0000 7f7f0c1f7640 1 -- 172.21.15.81:0/839823099 learned_addr learned my addr 172.21.15.81:0/839823099 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.950+0000 7f7f0b1f5640 1 --2- 172.21.15.81:0/839823099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 0x55def0baa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.950+0000 7fb37f4db640 1 -- 172.21.15.81:0/3305262764 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56142176ea80 con 0x56142248f800 2023-11-08T16:34:51.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7fb3878145c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:51.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7fb3878145c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:51.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7fb3878145c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:51.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5614225d4800 0x56142171a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 7.14 7:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421707880 con 0x5614225d4800 2023-11-08T16:34:51.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7f7f0b9f6640 1 --2- 172.21.15.81:0/839823099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0e33400 0x55def0ba9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7fb3854e7640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x561421a18000 0x56142171b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7f7f0c1f7640 1 -- 172.21.15.81:0/839823099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 msgr2=0x55def0baa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/839823099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 0x55def0baa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7f7f0c1f7640 1 -- 172.21.15.81:0/839823099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0e33400 msgr2=0x55def0ba9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/839823099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0e33400 0x55def0ba9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7f3d3a2175c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091bdd7800 0x56091b066100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7f7f0c1f7640 1 -- 172.21.15.81:0/839823099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55def0b644e0 con 0x55def0e33000 2023-11-08T16:34:51.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.951+0000 7f7f0b1f5640 1 --2- 172.21.15.81:0/839823099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 0x55def0baa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:34:51.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7fb384ce6640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5614225d4800 0x56142171a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7f3d3a2175c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091becd800 0x56091b066680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7f3d3a2175c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091becd400 0x56091b066c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7f3d3a2175c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091b042b40 con 0x56091becd800 2023-11-08T16:34:51.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7f3d3a2175c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56091b043180 con 0x56091bdd7800 2023-11-08T16:34:51.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7f3d3a2175c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56091b042000 con 0x56091becd400 2023-11-08T16:34:51.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7f3d38597640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091bdd7800 0x56091b066100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7f3d38597640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091bdd7800 0x56091b066100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:56966/0 (socket says 172.21.15.81:56966) 2023-11-08T16:34:51.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7f3d38597640 1 -- 172.21.15.81:0/3366333913 learned_addr learned my addr 172.21.15.81:0/3366333913 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7f3d37d96640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091becd800 0x56091b066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/839823099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0e33000 0x55def0ba9600 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x55def0e22690 tx=0x55def0e18e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2feaa502ec46fd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.953+0000 7fb3854e7640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x561421a18000 0x56142171b700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x5614224bd920 tx=0x5614224821e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.952+0000 7f7f071ed640 1 -- 172.21.15.81:0/839823099 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55def0b851e0 con 0x55def0e33000 2023-11-08T16:34:51.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.953+0000 7f3d37595640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091becd400 0x56091b066c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.953+0000 7f7f071ed640 1 -- 172.21.15.81:0/839823099 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55def190b080 con 0x55def0e33000 2023-11-08T16:34:51.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.953+0000 7f7f071ed640 1 -- 172.21.15.81:0/839823099 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55def0b85080 con 0x55def0e33000 2023-11-08T16:34:51.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.953+0000 7fb384ce6640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5614225d4800 0x56142171a100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.953+0000 7f3d37d96640 1 -- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091becd400 msgr2=0x56091b066c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.953+0000 7f3d37d96640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091becd400 0x56091b066c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.953+0000 7f3d37d96640 1 -- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091bdd7800 msgr2=0x56091b066100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.953+0000 7f3d37d96640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091bdd7800 0x56091b066100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.953+0000 7f3d37d96640 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56091b020b60 con 0x56091becd800 2023-11-08T16:34:51.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.953+0000 7fa22dcbb640 1 -- 172.21.15.81:0/2688234169 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x562635d8a840 con 0x562636c03800 2023-11-08T16:34:51.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.954+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv104 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.954+0000 7f3d37d96640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091becd800 0x56091b066680 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x56091be1d770 tx=0x56091bee4c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.954+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/839823099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0e33000 msgr2=0x55def0ba9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.954+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/839823099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0e33000 0x55def0ba9600 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x55def0e22690 tx=0x55def0e18e40 comp rx=0 tx=0).stop 2023-11-08T16:34:51.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.954+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5614225d4400 0x561421719b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.954+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:2 7.1f 7:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421707c00 con 0x5614225d4400 2023-11-08T16:34:51.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.954+0000 7f3d31d8a640 1 -- 172.21.15.81:0/3366333913 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56091b041080 con 0x56091becd800 2023-11-08T16:34:51.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.955+0000 7f3d31d8a640 1 -- 172.21.15.81:0/3366333913 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56091bed2480 con 0x56091becd800 2023-11-08T16:34:51.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.955+0000 7f3d31d8a640 1 -- 172.21.15.81:0/3366333913 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56091b040f20 con 0x56091becd800 2023-11-08T16:34:51.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.955+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56091b020d00 con 0x56091becd800 2023-11-08T16:34:51.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.955+0000 7fa22dcbb640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562636c6e800 0x562635d9b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.955+0000 7fb385ce8640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5614225d4400 0x561421719b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.955+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56091b0209c0 con 0x56091becd800 2023-11-08T16:34:51.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.955+0000 7f3d3a2175c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:51.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.955+0000 7fa22dcbb640 1 -- 172.21.15.81:0/2688234169 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x562635deea80 con 0x562636c03800 2023-11-08T16:34:51.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7fa2344c8640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562636c6e800 0x562635d9b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.955+0000 7f01574425c0 1 -- start start 2023-11-08T16:34:51.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.955+0000 7f01574425c0 1 Processor -- start 2023-11-08T16:34:51.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/839823099 shutdown_connections 2023-11-08T16:34:51.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/839823099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 0x55def0baa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/839823099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0e33400 0x55def0ba9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/839823099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0e33000 0x55def0ba9600 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/839823099 >> 172.21.15.81:0/839823099 conn(0x55def0e32c00 msgr2=0x55def18b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7fa23642b5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:51.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7fa23642b5c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:51.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7fa23642b5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:51.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562636c6ec00 0x562635d9a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 7.14 7:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562635d87880 con 0x562636c6ec00 2023-11-08T16:34:51.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7fe23320c5c0 1 -- start start 2023-11-08T16:34:51.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7fe23320c5c0 1 Processor -- start 2023-11-08T16:34:51.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7fa2344c8640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562636c6e800 0x562635d9b700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x562636bea7b0 tx=0x562636be4120 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7fb385ce8640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5614225d4400 0x561421719b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.956+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/839823099 shutdown_connections 2023-11-08T16:34:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.957+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/839823099 wait complete. 2023-11-08T16:34:51.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.957+0000 7fa233cc7640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562636c6ec00 0x562635d9a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.957+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x56142255ad80 con 0x5614225d4400 2023-11-08T16:34:51.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.957+0000 7f01574425c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0800 0x55af33e89b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.957+0000 7f01574425c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af33e1fc00 0x55af33e88b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.957+0000 7f01574425c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af34bf1c00 0x55af33e89600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.957+0000 7f7f0de0e5c0 1 Processor -- start 2023-11-08T16:34:51.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.957+0000 7f01574425c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af33e67180 con 0x55af34bf0800 2023-11-08T16:34:51.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.957+0000 7f01574425c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af33e66b40 con 0x55af33e1fc00 2023-11-08T16:34:51.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.957+0000 7f01574425c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af33e66a00 con 0x55af34bf1c00 2023-11-08T16:34:51.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7f7f0de0e5c0 1 -- start start 2023-11-08T16:34:51.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 8.0 8:00000000::::head [pgnls-filter start_epoch 64 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a000 con 0x5614225d4800 2023-11-08T16:34:51.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7f0155bea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0800 0x55af33e89b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7f01563eb640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af34bf1c00 0x55af33e89600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7f0155bea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0800 0x55af33e89b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54952/0 (socket says 172.21.15.81:54952) 2023-11-08T16:34:51.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7f0155bea640 1 -- 172.21.15.81:0/3781527100 learned_addr learned my addr 172.21.15.81:0/3781527100 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7f01553e9640 1 --2- 172.21.15.81:0/3781527100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af33e1fc00 0x55af33e88b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7fe23320c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303ae42c00 0x56303aeadb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7fe23320c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303ae43800 0x56303aead600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7fe23320c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303ae43c00 0x56303aead080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7fe23320c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56303ae8a000 con 0x56303ae43c00 2023-11-08T16:34:51.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7fe23320c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56303ae8aa00 con 0x56303ae42c00 2023-11-08T16:34:51.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7fe23320c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56303ae8ab40 con 0x56303ae43800 2023-11-08T16:34:51.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7fa233cc7640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562636c6ec00 0x562635d9a100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.958+0000 7f3d31d8a640 1 -- 172.21.15.81:0/3366333913 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x56091b056840 con 0x56091becd800 2023-11-08T16:34:51.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7fe230db3640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303ae42c00 0x56303aeadb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7fe230db3640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303ae42c00 0x56303aeadb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:56998/0 (socket says 172.21.15.81:56998) 2023-11-08T16:34:51.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f7f0de0e5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0e33000 0x55def0bab180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f7f0de0e5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0e33400 0x55def0baac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f7f0de0e5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 0x55def0baa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f7f0de0e5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def0c0cc80 con 0x55def0e33000 2023-11-08T16:34:51.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f7f0de0e5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55def0c0cb40 con 0x55def0e33400 2023-11-08T16:34:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7fe230db3640 1 -- 172.21.15.81:0/3150314004 learned_addr learned my addr 172.21.15.81:0/3150314004 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f7f0de0e5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55def0c0ca00 con 0x55def0e33800 2023-11-08T16:34:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv104 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562636c6f000 0x562635d99b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:2 7.1f 7:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562635d87c00 con 0x562636c6f000 2023-11-08T16:34:51.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f01563eb640 1 -- 172.21.15.81:0/3781527100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af33e1fc00 msgr2=0x55af33e88b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7f7f0b1f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 0x55def0baa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7f7f0b1f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 0x55def0baa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:35884/0 (socket says 172.21.15.81:35884) 2023-11-08T16:34:51.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7f7f0b1f5640 1 -- 172.21.15.81:0/1660207497 learned_addr learned my addr 172.21.15.81:0/1660207497 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 64 out=81b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7fe230db3640 1 -- 172.21.15.81:0/3150314004 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303ae43800 msgr2=0x56303aead600 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-08T16:34:51.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7fe230db3640 1 --2- 172.21.15.81:0/3150314004 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303ae43800 0x56303aead600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7fe230db3640 1 -- 172.21.15.81:0/3150314004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303ae43c00 msgr2=0x56303aead080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7fe230db3640 1 --2- 172.21.15.81:0/3150314004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303ae43c00 0x56303aead080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7fe230db3640 1 -- 172.21.15.81:0/3150314004 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56303ae689c0 con 0x56303ae42c00 2023-11-08T16:34:51.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f01563eb640 1 --2- 172.21.15.81:0/3781527100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af33e1fc00 0x55af33e88b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:4 8.0 8:00000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a380 con 0x5614225d4800 2023-11-08T16:34:51.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f01563eb640 1 -- 172.21.15.81:0/3781527100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0800 msgr2=0x55af33e89b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f01563eb640 1 --2- 172.21.15.81:0/3781527100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0800 0x55af33e89b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0e33000 0x55def0bab180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7f7f0b9f6640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0e33400 0x55def0baac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f01563eb640 1 -- 172.21.15.81:0/3781527100 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af33e44820 con 0x55af34bf1c00 2023-11-08T16:34:51.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f01553e9640 1 --2- 172.21.15.81:0/3781527100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af33e1fc00 0x55af33e88b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:34:51.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7fa2344c8640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562636c6f000 0x562635d99b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7fe230db3640 1 --2- 172.21.15.81:0/3150314004 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303ae42c00 0x56303aeadb80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x56303bc82bd0 tx=0x56303bc38480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77fb58e74d27ecbc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.959+0000 7f3d31d8a640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56091bf39000 0x56091b36e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7f3d31d8a640 1 -- 172.21.15.81:0/3366333913 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56091b0baa80 con 0x56091becd800 2023-11-08T16:34:51.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7f4f6fe385c0 1 -- start start 2023-11-08T16:34:51.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7f4f6fe385c0 1 Processor -- start 2023-11-08T16:34:51.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(4 [pgnls-filter start_epoch 64 out=81b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.960+0000 7f01563eb640 1 --2- 172.21.15.81:0/3781527100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af34bf1c00 0x55af33e89600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55af34b9cb70 tx=0x55af34bee240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70ce616e37852fc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f3d3a2175c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:51.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f3d3a2175c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:51.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f3d3a2175c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:51.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56091becd000 0x56091b067180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 7.14 7:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b053880 con 0x56091becd000 2023-11-08T16:34:51.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f7f0b1f5640 1 -- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0e33400 msgr2=0x55def0baac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f7f0b1f5640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0e33400 0x55def0baac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f7f0b1f5640 1 -- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0e33000 msgr2=0x55def0bab180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7fb3878145c0 20 datascan.forall_objects: OSD matched oid 10000000206.00000000 2023-11-08T16:34:51.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7fb3878145c0 10 datascan.operator(): handling object 10000000206.0 2023-11-08T16:34:51.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f7f0b1f5640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0e33000 0x55def0bab180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f7f0b1f5640 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55def0b649c0 con 0x55def0e33800 2023-11-08T16:34:51.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:5 8.0 8:012cdbc7:::10000000206.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a700 con 0x5614225d4800 2023-11-08T16:34:51.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f3d38597640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56091bf39000 0x56091b36e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f3d37595640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56091becd000 0x56091b067180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f7f0b1f5640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 0x55def0baa680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55def0e23350 tx=0x55def0e18d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.961+0000 7f4f6fe385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f680c00 0x56317f6ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f4f6fe385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f681800 0x56317f6eab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f4f6fe385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56317f681c00 0x56317f6eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f4f6fe385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f6c9180 con 0x56317f681800 2023-11-08T16:34:51.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f4f6fe385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f6c8140 con 0x56317f681c00 2023-11-08T16:34:51.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f4f6fe385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56317f6c8000 con 0x56317f680c00 2023-11-08T16:34:51.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f01533e5640 1 -- 172.21.15.81:0/3781527100 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55af33e651e0 con 0x55af34bf1c00 2023-11-08T16:34:51.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7fe22e5ae640 1 -- 172.21.15.81:0/3150314004 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56303ae88f20 con 0x56303ae42c00 2023-11-08T16:34:51.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7fe22e5ae640 1 -- 172.21.15.81:0/3150314004 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56303ae9c900 con 0x56303ae42c00 2023-11-08T16:34:51.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7fe22e5ae640 1 -- 172.21.15.81:0/3150314004 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56303ae891e0 con 0x56303ae42c00 2023-11-08T16:34:51.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(5 10000000206.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f01533e5640 1 -- 172.21.15.81:0/3781527100 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af34c06000 con 0x55af34bf1c00 2023-11-08T16:34:51.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f01533e5640 1 -- 172.21.15.81:0/3781527100 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55af33e65080 con 0x55af34bf1c00 2023-11-08T16:34:51.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7fe23320c5c0 1 -- 172.21.15.81:0/3150314004 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56303ae68d00 con 0x56303ae42c00 2023-11-08T16:34:51.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7fe23320c5c0 1 -- 172.21.15.81:0/3150314004 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56303ae689c0 con 0x56303ae42c00 2023-11-08T16:34:51.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7fe23320c5c0 4 waiting for MDS map... 2023-11-08T16:34:51.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x561421a19400 0x561421719600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:6 9.0 9:012cdbc7:::10000000206.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a000 con 0x561421a19400 2023-11-08T16:34:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f01574425c0 1 -- 172.21.15.81:0/3781527100 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af33e44b60 con 0x55af34bf1c00 2023-11-08T16:34:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f01574425c0 1 -- 172.21.15.81:0/3781527100 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55af33e44680 con 0x55af34bf1c00 2023-11-08T16:34:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f01574425c0 4 waiting for MDS map... 2023-11-08T16:34:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f3d37595640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56091becd000 0x56091b067180 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f7f059ea640 1 -- 172.21.15.81:0/1660207497 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55def0b84f20 con 0x55def0e33800 2023-11-08T16:34:51.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f7f059ea640 1 -- 172.21.15.81:0/1660207497 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55def190a180 con 0x55def0e33800 2023-11-08T16:34:51.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f7f059ea640 1 -- 172.21.15.81:0/1660207497 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55def0b851e0 con 0x55def0e33800 2023-11-08T16:34:51.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55def0b64b60 con 0x55def0e33800 2023-11-08T16:34:51.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7fa2344c8640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562636c6f000 0x562635d99b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f4f6d587640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f681800 0x56317f6eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f4f6e589640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56317f681c00 0x56317f6eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f4f6d587640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f681800 0x56317f6eab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54978/0 (socket says 172.21.15.81:54978) 2023-11-08T16:34:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f4f6d587640 1 -- 172.21.15.81:0/1467809784 learned_addr learned my addr 172.21.15.81:0/1467809784 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.962+0000 7f4f6dd88640 1 --2- 172.21.15.81:0/1467809784 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f680c00 0x56317f6ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.963+0000 7f3d38597640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56091bf39000 0x56091b36e000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x56091bee3770 tx=0x56091bee4de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.963+0000 7fe22e5ae640 1 -- 172.21.15.81:0/3150314004 <== mon.1 v2:172.21.15.93:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56303af02c40 con 0x56303ae42c00 2023-11-08T16:34:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.963+0000 7fb3854e7640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x561421a19400 0x561421719600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.963+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv104 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.963+0000 7fe22e5ae640 1 -- 172.21.15.81:0/3150314004 <== mon.1 v2:172.21.15.93:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x56303afb8000 con 0x56303ae42c00 2023-11-08T16:34:51.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.963+0000 7fe23320c5c0 4 Got MDS map 47 2023-11-08T16:34:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.963+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55def0b64820 con 0x55def0e33800 2023-11-08T16:34:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.963+0000 7f7f0de0e5c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.963+0000 7fe23320c5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.963+0000 7fe23320c5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.963+0000 7fa2344c8640 1 -- 172.21.15.81:0/2688234169 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x562636bdad80 con 0x562636c6f000 2023-11-08T16:34:51.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f01533e5640 1 -- 172.21.15.81:0/3781527100 <== mon.2 v2:172.21.15.93:3301/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55af33edec40 con 0x55af34bf1c00 2023-11-08T16:34:51.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f4f6e589640 1 -- 172.21.15.81:0/1467809784 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f680c00 msgr2=0x56317f6ea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f4f6e589640 1 --2- 172.21.15.81:0/1467809784 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f680c00 0x56317f6ea580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f4f6e589640 1 -- 172.21.15.81:0/1467809784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f681800 msgr2=0x56317f6eab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f01533e5640 1 -- 172.21.15.81:0/3781527100 <== mon.2 v2:172.21.15.93:3301/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55af33e6ef00 con 0x55af34bf1c00 2023-11-08T16:34:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f4f6e589640 1 --2- 172.21.15.81:0/1467809784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f681800 0x56317f6eab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f4f6e589640 1 -- 172.21.15.81:0/1467809784 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56317f6a69c0 con 0x56317f681c00 2023-11-08T16:34:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7fb3854e7640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x561421a19400 0x561421719600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56091beccc00 0x56091b067700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:2 7.1f 7:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b053c00 con 0x56091beccc00 2023-11-08T16:34:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f4f6e589640 1 --2- 172.21.15.81:0/1467809784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56317f681c00 0x56317f6eb080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5631804829c0 tx=0x563180446ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eec5d5d9c1a1e46d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7fe23320c5c0 1 Processor -- start 2023-11-08T16:34:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7fb3854e7640 1 -- 172.21.15.81:0/3305262764 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(6 10000000206.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56142255afc0 con 0x561421a19400 2023-11-08T16:34:51.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 8.0 8:00000000::::head [pgnls-filter start_epoch 64 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0000 con 0x562636c6ec00 2023-11-08T16:34:51.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 64 out=81b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562636c6f400 0x562635d99600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:4 8.36 8:6db6db6d::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0380 con 0x562636c6f400 2023-11-08T16:34:51.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7fb3878145c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7fb3878145c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000206 2023-11-08T16:34:51.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7fb3878145c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/17 2023-11-08T16:34:51.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7fb3878145c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=17 2023-11-08T16:34:51.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:7 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a380 con 0x5614225d4800 2023-11-08T16:34:51.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f01574425c0 4 Got MDS map 47 2023-11-08T16:34:51.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f01574425c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:51.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.964+0000 7f01574425c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:51.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7f01574425c0 1 Processor -- start 2023-11-08T16:34:51.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7f3d38597640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56091beccc00 0x56091b067700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7f01574425c0 1 -- start start 2023-11-08T16:34:51.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(7 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fb3878145c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-11-08T16:34:51.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fb3878145c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/17, will insert to root 2023-11-08T16:34:51.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:8 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a700 con 0x5614225d4800 2023-11-08T16:34:51.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fa233cc7640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562636c6f400 0x562635d99600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.965+0000 7fe23320c5c0 1 -- start start 2023-11-08T16:34:51.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fe23320c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303bc55c00 0x56303aeacb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fe23320c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd78000 0x56303aeac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fe23320c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303bd78400 0x56303aeae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fe23320c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56303af11180 con 0x56303bd78000 2023-11-08T16:34:51.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fe23320c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56303af10dc0 con 0x56303bd78400 2023-11-08T16:34:51.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fe23320c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56303af10c80 con 0x56303bc55c00 2023-11-08T16:34:51.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(8 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fb3878145c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:34:51.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.966+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 7.10 7:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2aa80 con 0x5614225d4800 2023-11-08T16:34:51.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.967+0000 7fe230db3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd78000 0x56303aeac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.967+0000 7f01574425c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af34bf0000 0x55af33e89080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.967+0000 7fe2305b2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303bd78400 0x56303aeae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.967+0000 7f3d38597640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56091beccc00 0x56091b067700 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.967+0000 7f7f059ea640 1 -- 172.21.15.81:0/1660207497 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55def0b9b080 con 0x55def0e33800 2023-11-08T16:34:51.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.967+0000 7fa233cc7640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562636c6f400 0x562635d99600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.967+0000 7f01574425c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af33e1f800 0x55af33e8a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7f01574425c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af34d48000 0x55af33e8a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.967+0000 7fe2305b2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303bd78400 0x56303aeae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:57012/0 (socket says 172.21.15.81:57012) 2023-11-08T16:34:51.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.967+0000 7fe2305b2640 1 -- 172.21.15.81:0/4263880125 learned_addr learned my addr 172.21.15.81:0/4263880125 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.967+0000 7fe2315b4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303bc55c00 0x56303aeacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.967+0000 7fe230db3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd78000 0x56303aeac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54980/0 (socket says 172.21.15.81:54980) 2023-11-08T16:34:51.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 64 out=101b] v64'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562636bdab40 con 0x562636c6f400 2023-11-08T16:34:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7f0155bea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af33e1f800 0x55af33e8a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7f0155bea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af33e1f800 0x55af33e8a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54986/0 (socket says 172.21.15.81:54986) 2023-11-08T16:34:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7f0155bea640 1 -- 172.21.15.81:0/2737490222 learned_addr learned my addr 172.21.15.81:0/2737490222 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7f01574425c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af33e66140 con 0x55af33e1f800 2023-11-08T16:34:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7f3d38597640 1 -- 172.21.15.81:0/3366333913 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x56091bde8d80 con 0x56091beccc00 2023-11-08T16:34:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7f01574425c0 1 -- 172.21.15.81:0/2737490222 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af33e66000 con 0x55af34d48000 2023-11-08T16:34:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7f01574425c0 1 -- 172.21.15.81:0/2737490222 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af33eed180 con 0x55af34bf0000 2023-11-08T16:34:51.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7fa23642b5c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-11-08T16:34:51.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7fa23642b5c0 10 datascan.operator(): handling object 100000001f9.0 2023-11-08T16:34:51.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:5 8.36 8:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0700 con 0x562636c6f400 2023-11-08T16:34:51.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7f7f059ea640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55def0ea0800 0x55def0bab700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7f7f059ea640 1 -- 172.21.15.81:0/1660207497 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55def0bfea80 con 0x55def0e33800 2023-11-08T16:34:51.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7fe230db3640 1 -- 172.21.15.81:0/4263880125 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303bc55c00 msgr2=0x56303aeacb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7fe230db3640 1 --2- 172.21.15.81:0/4263880125 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303bc55c00 0x56303aeacb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7fe230db3640 1 -- 172.21.15.81:0/4263880125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303bd78400 msgr2=0x56303aeae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7fe230db3640 1 --2- 172.21.15.81:0/4263880125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303bd78400 0x56303aeae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.968+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 8.0 8:00000000::::head [pgnls-filter start_epoch 64 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37e000 con 0x56091becd000 2023-11-08T16:34:51.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f01553e9640 1 --2- 172.21.15.81:0/2737490222 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af34bf0000 0x55af33e89080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f01553e9640 1 --2- 172.21.15.81:0/2737490222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af34d48000 0x55af33e8a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7fe230db3640 1 -- 172.21.15.81:0/4263880125 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56303ae68b60 con 0x56303bd78000 2023-11-08T16:34:51.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(5 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x562636bdab40 con 0x562636c6f400 2023-11-08T16:34:51.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f0155bea640 1 -- 172.21.15.81:0/2737490222 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af34bf0000 msgr2=0x55af33e89080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f0155bea640 1 --2- 172.21.15.81:0/2737490222 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af34bf0000 0x55af33e89080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f0155bea640 1 -- 172.21.15.81:0/2737490222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af34d48000 msgr2=0x55af33e8a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562636c6fc00 0x5626360be000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f0155bea640 1 --2- 172.21.15.81:0/2737490222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af34d48000 0x55af33e8a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:6 9.16 9:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0000 con 0x562636c6fc00 2023-11-08T16:34:51.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55def0ea0800 0x55def0bab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f7f0de0e5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:51.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f7f0de0e5c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:51.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f7f0de0e5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:51.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55def0ea0c00 0x55def0baa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f0155bea640 1 -- 172.21.15.81:0/2737490222 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af33e44d00 con 0x55af33e1f800 2023-11-08T16:34:51.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7fe230db3640 1 --2- 172.21.15.81:0/4263880125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd78000 0x56303aeac580 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x56303bc83950 tx=0x56303bc38540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd6231e93df1229 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 7.14 7:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0b97880 con 0x55def0ea0c00 2023-11-08T16:34:51.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f0155bea640 1 --2- 172.21.15.81:0/2737490222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af33e1f800 0x55af33e8a100 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x55af34d04690 tx=0x55af34d06c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c83f4bb0485174 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(9 10000000000.00000000 [create,omap-set-header] v64'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fe22c5aa640 1 -- 172.21.15.81:0/4263880125 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56303ae89080 con 0x56303bd78000 2023-11-08T16:34:51.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.969+0000 7f7f0b9f6640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55def0ea0c00 0x55def0baa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fb3878145c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-11-08T16:34:51.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:10 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2ae00 con 0x5614225d4800 2023-11-08T16:34:51.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7f01513e1640 1 -- 172.21.15.81:0/2737490222 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55af33e651e0 con 0x55af33e1f800 2023-11-08T16:34:51.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7f01513e1640 1 -- 172.21.15.81:0/2737490222 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af34c06f00 con 0x55af33e1f800 2023-11-08T16:34:51.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7f01513e1640 1 -- 172.21.15.81:0/2737490222 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55af33e65080 con 0x55af33e1f800 2023-11-08T16:34:51.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fe22c5aa640 1 -- 172.21.15.81:0/4263880125 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56303afbad80 con 0x56303bd78000 2023-11-08T16:34:51.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fe22c5aa640 1 -- 172.21.15.81:0/4263880125 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56303ae88dc0 con 0x56303bd78000 2023-11-08T16:34:51.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fa2334c6640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562636c6fc00 0x5626360be000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(10 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fb3878145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=17 frag=*, r=0 2023-11-08T16:34:51.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fb3878145c0 20 datascan.read_dentry: 17_head not found in result 2023-11-08T16:34:51.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fb3878145c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000206 at 0x0x10000000000/17 with size=6291456 bytes 2023-11-08T16:34:51.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:11 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a380 con 0x5614225d4800 2023-11-08T16:34:51.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7f7f0b9f6640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55def0ea0c00 0x55def0baa100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 64 out=81b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.970+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:4 8.24 8:24924924::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37e380 con 0x56091beccc00 2023-11-08T16:34:51.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55def0ea0800 0x55def0bab700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55def0e22690 tx=0x55def0e1a180 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv104 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55def0ea1000 0x55def0ba9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:2 7.1f 7:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0b97c00 con 0x55def0ea1000 2023-11-08T16:34:51.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7f4f6b583640 1 -- 172.21.15.81:0/1467809784 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56317f6c6f20 con 0x56317f681c00 2023-11-08T16:34:51.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1467809784 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56317f6a6d00 con 0x56317f681c00 2023-11-08T16:34:51.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1467809784 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56317f6a69c0 con 0x56317f681c00 2023-11-08T16:34:51.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7f4f6fe385c0 4 waiting for MDS map... 2023-11-08T16:34:51.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7f4f6b583640 1 -- 172.21.15.81:0/1467809784 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56317f7fa300 con 0x56317f681c00 2023-11-08T16:34:51.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7f4f6b583640 1 -- 172.21.15.81:0/1467809784 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56317f6c71e0 con 0x56317f681c00 2023-11-08T16:34:51.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7fe23320c5c0 1 -- 172.21.15.81:0/4263880125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd78000 msgr2=0x56303aeac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/4263880125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd78000 0x56303aeac580 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x56303bc83950 tx=0x56303bc38540 comp rx=0 tx=0).stop 2023-11-08T16:34:51.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.971+0000 7f3d38597640 1 -- 172.21.15.81:0/3366333913 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 64 out=49b] v62'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56091bde8d80 con 0x56091beccc00 2023-11-08T16:34:51.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f01574425c0 1 -- 172.21.15.81:0/2737490222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af33e1f800 msgr2=0x55af33e8a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f01574425c0 1 --2- 172.21.15.81:0/2737490222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af33e1f800 0x55af33e8a100 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x55af34d04690 tx=0x55af34d06c00 comp rx=0 tx=0).stop 2023-11-08T16:34:51.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fe23320c5c0 1 -- 172.21.15.81:0/4263880125 shutdown_connections 2023-11-08T16:34:51.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/4263880125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303bd78400 0x56303aeae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/4263880125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd78000 0x56303aeac580 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/4263880125 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303bc55c00 0x56303aeacb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fe23320c5c0 1 -- 172.21.15.81:0/4263880125 >> 172.21.15.81:0/4263880125 conn(0x56303bc55800 msgr2=0x56303af35000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f3d3358d640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56091bf39400 0x56091b36e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f3d3358d640 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:5 8.14 8:28000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b053c00 con 0x56091bf39400 2023-11-08T16:34:51.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f01574425c0 1 -- 172.21.15.81:0/2737490222 shutdown_connections 2023-11-08T16:34:51.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f01574425c0 1 --2- 172.21.15.81:0/2737490222 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af34d48000 0x55af33e8a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f01574425c0 1 --2- 172.21.15.81:0/2737490222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af33e1f800 0x55af33e8a100 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f01574425c0 1 --2- 172.21.15.81:0/2737490222 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af34bf0000 0x55af33e89080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f01574425c0 1 -- 172.21.15.81:0/2737490222 >> 172.21.15.81:0/2737490222 conn(0x55af34bf0400 msgr2=0x55af34b9a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55def0ea1000 0x55def0ba9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fe23320c5c0 1 -- 172.21.15.81:0/4263880125 shutdown_connections 2023-11-08T16:34:51.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f01574425c0 1 -- 172.21.15.81:0/2737490222 shutdown_connections 2023-11-08T16:34:51.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f01574425c0 1 -- 172.21.15.81:0/2737490222 wait complete. 2023-11-08T16:34:51.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fe23320c5c0 1 -- 172.21.15.81:0/4263880125 wait complete. 2023-11-08T16:34:51.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f4f6b583640 1 -- 172.21.15.81:0/1467809784 <== mon.1 v2:172.21.15.93:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56317f740c40 con 0x56317f681c00 2023-11-08T16:34:51.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f4f6b583640 1 -- 172.21.15.81:0/1467809784 <== mon.1 v2:172.21.15.93:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x56317f6d0f00 con 0x56317f681c00 2023-11-08T16:34:51.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(11 10000000000.00000000 [omap-set-vals] v64'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fb3878145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-11-08T16:34:51.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7f4f6fe385c0 4 Got MDS map 47 2023-11-08T16:34:51.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fb3878145c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-11-08T16:34:51.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fb3878145c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-11-08T16:34:51.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.972+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:12 7.1f 7:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a700 con 0x5614225d4400 2023-11-08T16:34:51.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7f4f6fe385c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:51.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7fa2334c6640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562636c6fc00 0x5626360be000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7f4f6fe385c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:51.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7f01574425c0 1 Processor -- start 2023-11-08T16:34:51.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7fe23320c5c0 1 Processor -- start 2023-11-08T16:34:51.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55def0ea1000 0x55def0ba9b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(12 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x56142255ad80 con 0x5614225d4400 2023-11-08T16:34:51.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7fb3878145c0 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-11-08T16:34:51.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7fb3878145c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-11-08T16:34:51.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:13 7.1f 7:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2aa80 con 0x5614225d4400 2023-11-08T16:34:51.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7f01574425c0 1 -- start start 2023-11-08T16:34:51.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7fe23320c5c0 1 -- start start 2023-11-08T16:34:51.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7f3d37d96640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56091bf39400 0x56091b36e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.973+0000 7f4f6fe385c0 1 Processor -- start 2023-11-08T16:34:51.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fa2334c6640 1 -- 172.21.15.81:0/2688234169 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(6 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562636bdafc0 con 0x562636c6fc00 2023-11-08T16:34:51.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fa23642b5c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fa23642b5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-11-08T16:34:51.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fa23642b5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-11-08T16:34:51.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fa23642b5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-11-08T16:34:51.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7f4f6fe385c0 1 -- start start 2023-11-08T16:34:51.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:7 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0380 con 0x562636c6ec00 2023-11-08T16:34:51.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(7 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7f7f0c1f7640 1 -- 172.21.15.81:0/1660207497 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55def0cb8d80 con 0x55def0ea1000 2023-11-08T16:34:51.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 8.0 8:00000000::::head [pgnls-filter start_epoch 64 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fae000 con 0x55def0ea0c00 2023-11-08T16:34:51.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fe23320c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303bc55800 0x56303aeacb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f4f6fe385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563180443c00 0x56317f6eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(13 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x56142255ad80 con 0x5614225d4400 2023-11-08T16:34:51.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fa23642b5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fa23642b5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fa23642b5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/4, will insert to root 2023-11-08T16:34:51.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:8 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0e00 con 0x562636c6ec00 2023-11-08T16:34:51.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f01574425c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af33e1f800 0x55af33e8b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f01574425c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0000 0x55af33e8b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f01574425c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af34d49400 0x55af33e8ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f01574425c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af33eecdc0 con 0x55af34bf0000 2023-11-08T16:34:51.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f01574425c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af33eecc80 con 0x55af34d49400 2023-11-08T16:34:51.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f01574425c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55af33e67180 con 0x55af33e1f800 2023-11-08T16:34:51.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fe23320c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd79000 0x56303aeac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.974+0000 7fe23320c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303bd78c00 0x56303aeae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fe23320c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56303ae8b180 con 0x56303bd79000 2023-11-08T16:34:51.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fe23320c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56303ae8ab40 con 0x56303bc55800 2023-11-08T16:34:51.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fe23320c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56303af10c80 con 0x56303bd78c00 2023-11-08T16:34:51.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f4f6fe385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f9c6000 0x56317f6ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f4f6fe385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c6400 0x56317f6ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f4f6fe385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f6c8a00 con 0x56317f9c6400 2023-11-08T16:34:51.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f4f6fe385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f74f180 con 0x563180443c00 2023-11-08T16:34:51.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f4f6fe385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56317f74edc0 con 0x56317f9c6000 2023-11-08T16:34:51.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fb3878145c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:34:51.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:14 7.1f 7:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2ae00 con 0x5614225d4400 2023-11-08T16:34:51.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 64 out=81b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f3d37d96640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56091bf39400 0x56091b36e580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:4 8.12 8:49249249::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fae380 con 0x55def0ea1000 2023-11-08T16:34:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(8 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fa23642b5c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-11-08T16:34:51.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fa23642b5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f0155bea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0000 0x55af33e8b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b1180 con 0x562636c6ec00 2023-11-08T16:34:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fe230db3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd79000 0x56303aeac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fe230db3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd79000 0x56303aeac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54992/0 (socket says 172.21.15.81:54992) 2023-11-08T16:34:51.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 learned_addr learned my addr 172.21.15.81:0/2400868731 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f4f6d587640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c6400 0x56317f6ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f4f6d587640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c6400 0x56317f6ec100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54988/0 (socket says 172.21.15.81:54988) 2023-11-08T16:34:51.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f4f6d587640 1 -- 172.21.15.81:0/2109506518 learned_addr learned my addr 172.21.15.81:0/2109506518 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f4f6d587640 1 --2- 172.21.15.81:0/2109506518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563180443c00 0x56317f6eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f0155bea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0000 0x55af33e8b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54994/0 (socket says 172.21.15.81:54994) 2023-11-08T16:34:51.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.975+0000 7f0155bea640 1 -- 172.21.15.81:0/1907834121 learned_addr learned my addr 172.21.15.81:0/1907834121 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(9 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f4f6dd88640 1 --2- 172.21.15.81:0/2109506518 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f9c6000 0x56317f6ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7fa23642b5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-11-08T16:34:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7fa23642b5c0 20 datascan.read_dentry: 4_head not found in result 2023-11-08T16:34:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7fa23642b5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/4 with size=6291456 bytes 2023-11-08T16:34:51.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:10 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x562635d87880 con 0x562636c6ec00 2023-11-08T16:34:51.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303bd78c00 msgr2=0x56303aeae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7fe230db3640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303bd78c00 0x56303aeae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303bc55800 msgr2=0x56303aeacb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7fe230db3640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303bc55800 0x56303aeacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56303ae68d00 con 0x56303bd79000 2023-11-08T16:34:51.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f3d37d96640 1 -- 172.21.15.81:0/3366333913 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 64 out=49b] v62'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56091bde8fc0 con 0x56091bf39400 2023-11-08T16:34:51.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f01553e9640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af34d49400 0x55af33e8ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f4f6d587640 1 -- 172.21.15.81:0/2109506518 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f9c6000 msgr2=0x56317f6ebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f3d32d8c640 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:6 8.34 8:2c000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b053880 con 0x56091becd000 2023-11-08T16:34:51.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f4f6d587640 1 --2- 172.21.15.81:0/2109506518 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f9c6000 0x56317f6ebb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f4f6d587640 1 -- 172.21.15.81:0/2109506518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563180443c00 msgr2=0x56317f6eb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f4f6d587640 1 --2- 172.21.15.81:0/2109506518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563180443c00 0x56317f6eb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f4f6d587640 1 -- 172.21.15.81:0/2109506518 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56317f6a6b60 con 0x56317f9c6400 2023-11-08T16:34:51.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7fe230db3640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd79000 0x56303aeac580 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x56303bc48cc0 tx=0x56303bc38180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f4f6dd88640 1 --2- 172.21.15.81:0/2109506518 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f9c6000 0x56317f6ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:34:51.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f0155bea640 1 -- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af33e1f800 msgr2=0x55af33e8b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f0155bea640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af33e1f800 0x55af33e8b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f0155bea640 1 -- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af34d49400 msgr2=0x55af33e8ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f0155bea640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af34d49400 0x55af33e8ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.976+0000 7f0155bea640 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af33e44820 con 0x55af34bf0000 2023-11-08T16:34:51.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.977+0000 7fe22ada7640 1 -- 172.21.15.81:0/2400868731 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56303ae89080 con 0x56303bd79000 2023-11-08T16:34:51.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.977+0000 7fe22ada7640 1 -- 172.21.15.81:0/2400868731 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56303afbac00 con 0x56303bd79000 2023-11-08T16:34:51.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.977+0000 7fe22ada7640 1 -- 172.21.15.81:0/2400868731 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56303ae88dc0 con 0x56303bd79000 2023-11-08T16:34:51.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.977+0000 7f4f6d587640 1 --2- 172.21.15.81:0/2109506518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c6400 0x56317f6ec100 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x56317f9766c0 tx=0x56317f978cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfcd1608e391cb88 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.977+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56303ae689c0 con 0x56303bd79000 2023-11-08T16:34:51.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.977+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.977+0000 7f3d3358d640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56091bf39800 0x56091b36eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.977+0000 7f3d3358d640 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:7 8.c 8:30000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37e380 con 0x56091bf39800 2023-11-08T16:34:51.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.977+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56303ae68b60 con 0x56303bd79000 2023-11-08T16:34:51.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.977+0000 7fe23320c5c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:51.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f3d37595640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56091bf39800 0x56091b36eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f0155bea640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0000 0x55af33e8b180 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x55af34d05350 tx=0x55af34bee5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f4f6957f640 1 -- 172.21.15.81:0/2109506518 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56317f6c6f20 con 0x56317f9c6400 2023-11-08T16:34:51.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f7f0c1f7640 1 -- 172.21.15.81:0/1660207497 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 64 out=49b] v62'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55def0cb8d80 con 0x55def0ea1000 2023-11-08T16:34:51.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f7f071ed640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55def0fb8000 0x55def0fbe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f7f071ed640 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 8.32 8:4c000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0b97c00 con 0x55def0fb8000 2023-11-08T16:34:51.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f4f6957f640 1 -- 172.21.15.81:0/2109506518 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56317f7fb200 con 0x56317f9c6400 2023-11-08T16:34:51.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f4f6957f640 1 -- 172.21.15.81:0/2109506518 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56317f6c71e0 con 0x56317f9c6400 2023-11-08T16:34:51.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f014fbde640 1 -- 172.21.15.81:0/1907834121 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55af33e64420 con 0x55af34bf0000 2023-11-08T16:34:51.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f014fbde640 1 -- 172.21.15.81:0/1907834121 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55af34c06480 con 0x55af34bf0000 2023-11-08T16:34:51.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55af33e444e0 con 0x55af34bf0000 2023-11-08T16:34:51.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f014fbde640 1 -- 172.21.15.81:0/1907834121 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55af33e64420 con 0x55af34bf0000 2023-11-08T16:34:51.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f7f0b9f6640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55def0fb8000 0x55def0fbe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f3d37595640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56091bf39800 0x56091b36eb00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af33e441a0 con 0x55af34bf0000 2023-11-08T16:34:51.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.978+0000 7f01574425c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:51.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 64 out=81b] v64'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56091bde8b40 con 0x56091bf39800 2023-11-08T16:34:51.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f3d3a2175c0 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-11-08T16:34:51.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f3d3a2175c0 10 datascan.operator(): handling object 100000001fc.0 2023-11-08T16:34:51.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:8 8.c 8:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37e700 con 0x56091bf39800 2023-11-08T16:34:51.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/2109506518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c6400 msgr2=0x56317f6ec100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/2109506518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c6400 0x56317f6ec100 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x56317f9766c0 tx=0x56317f978cc0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/2109506518 shutdown_connections 2023-11-08T16:34:51.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/2109506518 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c6400 0x56317f6ec100 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/2109506518 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f9c6000 0x56317f6ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/2109506518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563180443c00 0x56317f6eb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/2109506518 >> 172.21.15.81:0/2109506518 conn(0x563180443800 msgr2=0x5631803fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7fe22ada7640 1 -- 172.21.15.81:0/2400868731 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x56303ae9e840 con 0x56303bd79000 2023-11-08T16:34:51.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/2109506518 shutdown_connections 2023-11-08T16:34:51.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(14 1.00000000 [create,omap-set-header] v64'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x56142255ad80 con 0x5614225d4400 2023-11-08T16:34:51.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.979+0000 7f7f0b9f6640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55def0fb8000 0x55def0fbe000 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0fb8000 2023-11-08T16:34:51.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/2109506518 wait complete. 2023-11-08T16:34:51.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7f7f069ec640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55def0fb8400 0x55def0fbe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7fb3878145c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-11-08T16:34:51.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:15 7.1f 7:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2b180 con 0x5614225d4400 2023-11-08T16:34:51.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(8 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x56091bde8b40 con 0x56091bf39800 2023-11-08T16:34:51.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 9.c 9:30604137:::100000001fc.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37e380 con 0x56091becd000 2023-11-08T16:34:51.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7f7f069ec640 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:6 8.a 8:50000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0b97880 con 0x55def0fb8400 2023-11-08T16:34:51.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7fe22ada7640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56303b1b0000 0x56303b1b6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7fe22ada7640 1 -- 172.21.15.81:0/2400868731 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56303af02a80 con 0x56303bd79000 2023-11-08T16:34:51.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7fe23320c5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:51.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7fe23320c5c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:51.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.980+0000 7fe23320c5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:51.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56303bd78800 0x56303aeae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 7.14 7:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303ae9b880 con 0x56303bd78800 2023-11-08T16:34:51.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(15 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x56142255ad80 con 0x5614225d4400 2023-11-08T16:34:51.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7f7f0b1f5640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55def0fb8400 0x55def0fbe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7f4f6fe385c0 1 Processor -- start 2023-11-08T16:34:51.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fe2315b4640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56303b1b0000 0x56303b1b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fe2305b2640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56303bd78800 0x56303aeae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fb3878145c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:34:51.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fb3878145c0 20 datascan.read_dentry: subdir_head not found in result 2023-11-08T16:34:51.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fb3878145c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-11-08T16:34:51.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:16 7.1f 7:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421707880 con 0x5614225d4400 2023-11-08T16:34:51.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f7f0b1f5640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55def0fb8400 0x55def0fbe580 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(9 100000001fc.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f3d3a2175c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f3d3a2175c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fc 2023-11-08T16:34:51.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f3d3a2175c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/7 2023-11-08T16:34:51.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f3d3a2175c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=7 2023-11-08T16:34:51.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:10 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37ea80 con 0x56091becd000 2023-11-08T16:34:51.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(10 10000000000.00000000 [omap-set-vals] v64'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fa23642b5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-11-08T16:34:51.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:11 8.36 8:6fe38dc2::::head [pgnls-filter start_epoch 64 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0a80 con 0x562636c6f400 2023-11-08T16:34:51.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 64 out=81b] v64'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562636bdab40 con 0x562636c6f400 2023-11-08T16:34:51.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fa23642b5c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-11-08T16:34:51.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7fa23642b5c0 10 datascan.operator(): handling object 100000001f7.0 2023-11-08T16:34:51.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:12 8.36 8:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0700 con 0x562636c6f400 2023-11-08T16:34:51.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7f014fbde640 1 -- 172.21.15.81:0/1907834121 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55af33e7b340 con 0x55af34bf0000 2023-11-08T16:34:51.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f014fbde640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55af34d62c00 0x55af3418a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f014fbde640 1 -- 172.21.15.81:0/1907834121 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55af33edea80 con 0x55af34bf0000 2023-11-08T16:34:51.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7fe2315b4640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56303b1b0000 0x56303b1b6000 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x56303af46990 tx=0x56303bd29bc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7fe2305b2640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56303bd78800 0x56303aeae680 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.981+0000 7f4f6fe385c0 1 -- start start 2023-11-08T16:34:51.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(10 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(12 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x562636bdab40 con 0x562636c6f400 2023-11-08T16:34:51.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:13 9.16 9:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0e00 con 0x562636c6fc00 2023-11-08T16:34:51.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f4f6fe385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563180443800 0x56317f6ed180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f3d3a2175c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f3d3a2175c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f3d3a2175c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/7, will insert to root 2023-11-08T16:34:51.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv104 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:11 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37ee00 con 0x56091becd000 2023-11-08T16:34:51.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56303bd78400 0x56303aeaec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:2 7.1f 7:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303ae9bc00 con 0x56303bd78400 2023-11-08T16:34:51.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f4f6fe385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c7000 0x56317f6ecc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f4f6fe385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f9c6c00 0x56317f6ec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f4f6fe385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f6c9180 con 0x56317f9c7000 2023-11-08T16:34:51.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f4f6fe385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317f6c8140 con 0x563180443800 2023-11-08T16:34:51.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.982+0000 7f4f6fe385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56317f6c8b40 con 0x56317f9c6c00 2023-11-08T16:34:51.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f01574425c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:51.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f01574425c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:51.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f01574425c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:51.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(11 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f3d3a2175c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-11-08T16:34:51.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f7f0b1f5640 1 -- 172.21.15.81:0/1660207497 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 64 out=49b] v62'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55def0cb8fc0 con 0x55def0fb8400 2023-11-08T16:34:51.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55af34d49000 0x55af33e8a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f3d3a2175c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:12 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37f180 con 0x56091becd000 2023-11-08T16:34:51.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 7.14 7:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af33e77880 con 0x55af34d49000 2023-11-08T16:34:51.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f7f071ed640 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:7 8.2a 8:54000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fae000 con 0x55def0ea0c00 2023-11-08T16:34:51.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f01553e9640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55af34d62c00 0x55af3418a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f4f6dd88640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c7000 0x56317f6ecc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f4f6dd88640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c7000 0x56317f6ecc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55006/0 (socket says 172.21.15.81:55006) 2023-11-08T16:34:51.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 learned_addr learned my addr 172.21.15.81:0/1920396101 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7fa2334c6640 1 -- 172.21.15.81:0/2688234169 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(13 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562636bdafc0 con 0x562636c6fc00 2023-11-08T16:34:51.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7fa23642b5c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7fa23642b5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-11-08T16:34:51.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7fa23642b5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-11-08T16:34:51.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7fa23642b5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-11-08T16:34:51.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f4f6d587640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563180443800 0x56317f6ed180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:14 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b1500 con 0x562636c6ec00 2023-11-08T16:34:51.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(7 [pgnls-filter start_epoch 64 out=101b] v64'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7fe2315b4640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56303bd78400 0x56303aeaec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f7f0de0e5c0 20 datascan.forall_objects: OSD matched oid 1000000020a.00000000 2023-11-08T16:34:51.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f7f0de0e5c0 10 datascan.operator(): handling object 1000000020a.0 2023-11-08T16:34:51.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:8 8.2a 8:55a6cd83:::1000000020a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fae700 con 0x55def0ea0c00 2023-11-08T16:34:51.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f4f6d587640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f9c6c00 0x56317f6ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f9c6c00 msgr2=0x56317f6ec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f4f6dd88640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f9c6c00 0x56317f6ec680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563180443800 msgr2=0x56317f6ed180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f4f6dd88640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563180443800 0x56317f6ed180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56317f6a6820 con 0x56317f9c7000 2023-11-08T16:34:51.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(12 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f3d3a2175c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=7 frag=*, r=0 2023-11-08T16:34:51.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f3d3a2175c0 20 datascan.read_dentry: 7_head not found in result 2023-11-08T16:34:51.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.983+0000 7f3d3a2175c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fc at 0x0x10000000000/7 with size=6291456 bytes 2023-11-08T16:34:51.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:13 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b053880 con 0x56091becd000 2023-11-08T16:34:51.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7f01553e9640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55af34d49000 0x55af33e8a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(14 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7fa23642b5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7fa23642b5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7fa23642b5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/2, will insert to root 2023-11-08T16:34:51.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:15 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b1880 con 0x562636c6ec00 2023-11-08T16:34:51.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7f4f6dd88640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c7000 0x56317f6ecc00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x56317f977440 tx=0x56317f978d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(8 1000000020a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55def0ea1800 0x55def0ba9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:9 9.a 9:55a6cd83:::1000000020a.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fae380 con 0x55def0ea1800 2023-11-08T16:34:51.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7f01553e9640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55af34d62c00 0x55af3418a000 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55af34b9c900 tx=0x55af34d07ce0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(16 1.00000000 [omap-set-vals] v64'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x56142255ad80 con 0x5614225d4400 2023-11-08T16:34:51.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7fb3878145c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-11-08T16:34:51.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x561421a19800 0x561421a34000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:17 8.20 8:04000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2aa80 con 0x561421a19800 2023-11-08T16:34:51.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7fe2315b4640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56303bd78400 0x56303aeaec00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7f4f67d7c640 1 -- 172.21.15.81:0/1920396101 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56317f6c6f20 con 0x56317f9c7000 2023-11-08T16:34:51.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7f4f67d7c640 1 -- 172.21.15.81:0/1920396101 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56317f6da180 con 0x56317f9c7000 2023-11-08T16:34:51.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.984+0000 7f4f67d7c640 1 -- 172.21.15.81:0/1920396101 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x56317f6c71e0 con 0x56317f9c7000 2023-11-08T16:34:51.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56317f6a64e0 con 0x56317f9c7000 2023-11-08T16:34:51.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56317f6a61a0 con 0x56317f9c7000 2023-11-08T16:34:51.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7f4f6fe385c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:51.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7f01553e9640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55af34d49000 0x55af33e8a680 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55def0ea1800 0x55def0ba9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7fe2315b4640 1 -- 172.21.15.81:0/2400868731 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x56303bbacd80 con 0x56303bd78400 2023-11-08T16:34:51.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7fb384ce6640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x561421a19800 0x561421a34000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 8.0 8:00000000::::head [pgnls-filter start_epoch 64 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dc000 con 0x56303bd78800 2023-11-08T16:34:51.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.985+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv104 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(13 10000000000.00000000 [omap-set-vals] v64'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7f3d3a2175c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-11-08T16:34:51.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:14 8.2c 8:34000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37e700 con 0x56091beccc00 2023-11-08T16:34:51.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(15 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fa23642b5c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-11-08T16:34:51.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fa23642b5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:16 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x562635d87880 con 0x562636c6ec00 2023-11-08T16:34:51.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55af34d48c00 0x55af33e89080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:2 7.1f 7:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af33e77c00 con 0x55af34d48c00 2023-11-08T16:34:51.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 64 out=81b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56303bd78000 0x56303aeaf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:4 8.2d 8:b6db6db6::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dc380 con 0x56303bd78000 2023-11-08T16:34:51.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(16 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fa23642b5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-11-08T16:34:51.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fa23642b5c0 20 datascan.read_dentry: 2_head not found in result 2023-11-08T16:34:51.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fa23642b5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/2 with size=6291456 bytes 2023-11-08T16:34:51.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:17 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0000 con 0x562636c6ec00 2023-11-08T16:34:51.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7fb384ce6640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x561421a19800 0x561421a34000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.986+0000 7f7f0c1f7640 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55def0ea1800 0x55def0ba9600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f3d38597640 1 -- 172.21.15.81:0/3366333913 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 64 out=81b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56091bde8d80 con 0x56091beccc00 2023-11-08T16:34:51.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7fe230db3640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56303bd78000 0x56303aeaf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f01563eb640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55af34d48c00 0x55af33e89080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f3d3a2175c0 20 datascan.forall_objects: OSD matched oid 10000000209.00000000 2023-11-08T16:34:51.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f3d3a2175c0 10 datascan.operator(): handling object 10000000209.0 2023-11-08T16:34:51.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:15 8.2c 8:37389ade:::10000000209.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37e380 con 0x56091beccc00 2023-11-08T16:34:51.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7fe230db3640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56303bd78000 0x56303aeaf180 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f7f0c1f7640 1 -- 172.21.15.81:0/1660207497 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(9 1000000020a.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55def0cb8d80 con 0x55def0ea1800 2023-11-08T16:34:51.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f7f0de0e5c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f7f0de0e5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020a 2023-11-08T16:34:51.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f7f0de0e5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/21 2023-11-08T16:34:51.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f7f0de0e5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=21 2023-11-08T16:34:51.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:10 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faee00 con 0x55def0ea0c00 2023-11-08T16:34:51.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(17 [pgnls-filter start_epoch 64 out=81b] v64'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56142255ab40 con 0x561421a19800 2023-11-08T16:34:51.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7fb3878145c0 20 datascan.forall_objects: OSD matched oid 1000000020b.00000000 2023-11-08T16:34:51.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7fb3878145c0 10 datascan.operator(): handling object 1000000020b.0 2023-11-08T16:34:51.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:18 8.20 8:05552577:::1000000020b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2ae00 con 0x561421a19800 2023-11-08T16:34:51.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.987+0000 7f4f67d7c640 1 -- 172.21.15.81:0/1920396101 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x56317f6dc840 con 0x56317f9c7000 2023-11-08T16:34:51.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f3d38597640 1 -- 172.21.15.81:0/3366333913 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(15 10000000209.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x56091bde8d80 con 0x56091beccc00 2023-11-08T16:34:51.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:16 9.c 9:37389ade:::10000000209.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37ee00 con 0x56091becd000 2023-11-08T16:34:51.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f01563eb640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55af34d48c00 0x55af33e89080 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56303bbacfc0 con 0x56303bd78000 2023-11-08T16:34:51.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(17 10000000000.00000000 [omap-set-vals] v64'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7fa23642b5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-11-08T16:34:51.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7fe22c5aa640 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 8.1d 8:b8000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303ae9b880 con 0x56303bd78000 2023-11-08T16:34:51.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:18 8.e 8:70000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0380 con 0x562636c6f000 2023-11-08T16:34:51.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(16 10000000209.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f3d3a2175c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f3d3a2175c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000209 2023-11-08T16:34:51.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f4f67d7c640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56317f9f4c00 0x56317f6ed700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f3d3a2175c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/20 2023-11-08T16:34:51.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f3d3a2175c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=20 2023-11-08T16:34:51.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:17 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37f500 con 0x56091becd000 2023-11-08T16:34:51.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(10 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(18 1000000020b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x56142255ab40 con 0x561421a19800 2023-11-08T16:34:51.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f7f0de0e5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f7f0de0e5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f7f0de0e5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/21, will insert to root 2023-11-08T16:34:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:11 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faf180 con 0x55def0ea0c00 2023-11-08T16:34:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.988+0000 7f4f67d7c640 1 -- 172.21.15.81:0/1920396101 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x56317f740a80 con 0x56317f9c7000 2023-11-08T16:34:51.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:19 9.0 9:05552577:::1000000020b.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a700 con 0x561421a19400 2023-11-08T16:34:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f01563eb640 1 -- 172.21.15.81:0/1907834121 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55af34c14d80 con 0x55af34d48c00 2023-11-08T16:34:51.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 8.0 8:00000000::::head [pgnls-filter start_epoch 64 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6000 con 0x55af34d49000 2023-11-08T16:34:51.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f4f6fe385c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:51.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f4f6fe385c0 4 datascan.main: data pool 'cephfs_data' has ID 8 2023-11-08T16:34:51.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f4f6fe385c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:34:51.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56317f9c6800 0x56317f6eb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:1 7.14 7:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317f6d9880 con 0x56317f9c6800 2023-11-08T16:34:51.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 64 out=49b] v62'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56303bbacfc0 con 0x56303bd78000 2023-11-08T16:34:51.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f4f6d587640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56317f9f4c00 0x56317f6ed700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7fa2344c8640 1 -- 172.21.15.81:0/2688234169 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(18 [pgnls-filter start_epoch 64 out=49b] v62'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562636bdad80 con 0x562636c6f000 2023-11-08T16:34:51.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7fe22bda9640 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:6 8.3d 8:bc000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dc000 con 0x56303bd78000 2023-11-08T16:34:51.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(11 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f7f0de0e5c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-11-08T16:34:51.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f7f0de0e5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:12 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faf500 con 0x55def0ea0c00 2023-11-08T16:34:51.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f4f6d587640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56317f9c6800 0x56317f6eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(17 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7fa22f4be640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5626360c3800 0x5626360c0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.989+0000 7fa22f4be640 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:19 8.2e 8:74000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x562635d87c00 con 0x5626360c3800 2023-11-08T16:34:51.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 64 out=81b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f3d3a2175c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f3d3a2175c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f3d3a2175c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/20, will insert to root 2023-11-08T16:34:51.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:18 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37f880 con 0x56091becd000 2023-11-08T16:34:51.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:4 8.9 8:92492492::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6380 con 0x55af34d48c00 2023-11-08T16:34:51.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7fb3854e7640 1 -- 172.21.15.81:0/3305262764 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(19 1000000020b.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56142255afc0 con 0x561421a19400 2023-11-08T16:34:51.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(12 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f7f0de0e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=21 frag=*, r=0 2023-11-08T16:34:51.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f7f0de0e5c0 20 datascan.read_dentry: 21_head not found in result 2023-11-08T16:34:51.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f7f0de0e5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020a at 0x0x10000000000/21 with size=6291456 bytes 2023-11-08T16:34:51.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:13 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faf880 con 0x55def0ea0c00 2023-11-08T16:34:51.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7fa2344c8640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5626360c3800 0x5626360c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7fb3878145c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7fb3878145c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020b 2023-11-08T16:34:51.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7fb3878145c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/22 2023-11-08T16:34:51.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7fb3878145c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=22 2023-11-08T16:34:51.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:20 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2b500 con 0x5614225d4800 2023-11-08T16:34:51.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 64 out=81b] v64'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56303bbacfc0 con 0x56303bd78000 2023-11-08T16:34:51.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7fe23320c5c0 20 datascan.forall_objects: OSD matched oid 10000000205.00000000 2023-11-08T16:34:51.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7fe23320c5c0 10 datascan.operator(): handling object 10000000205.0 2023-11-08T16:34:51.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:7 8.3d 8:bfcd3380:::10000000205.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dc700 con 0x56303bd78000 2023-11-08T16:34:51.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f4f6d587640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56317f9c6800 0x56317f6eb600 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(18 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f3d3a2175c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-11-08T16:34:51.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f3d3a2175c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(7 10000000205.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x56303bbacfc0 con 0x56303bd78000 2023-11-08T16:34:51.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.990+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:19 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37f180 con 0x56091becd000 2023-11-08T16:34:51.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7f01563eb640 1 -- 172.21.15.81:0/1907834121 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 64 out=49b] v62'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af34c14d80 con 0x55af34d48c00 2023-11-08T16:34:51.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:8 9.1d 9:bfcd3380:::10000000205.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dc380 con 0x56303bd78000 2023-11-08T16:34:51.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7f01513e1640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55af34d63c00 0x55af3418a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7f4f6d587640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56317f9f4c00 0x56317f6ed700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x56317f9768a0 tx=0x56317f979080 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv104 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7f01513e1640 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:5 8.29 8:94000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af33e77880 con 0x55af34d63c00 2023-11-08T16:34:51.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56317f9c6400 0x56317f6ec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:2 7.1f 7:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317f6d9c00 con 0x56317f9c6400 2023-11-08T16:34:51.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7fa2344c8640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5626360c3800 0x5626360c0680 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(8 10000000205.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56303bbacfc0 con 0x56303bd78000 2023-11-08T16:34:51.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7fe23320c5c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7fe23320c5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000205 2023-11-08T16:34:51.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7fe23320c5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/16 2023-11-08T16:34:51.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7fe23320c5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=16 2023-11-08T16:34:51.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dc000 con 0x56303bd78800 2023-11-08T16:34:51.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.991+0000 7f0155bea640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55af34d63c00 0x55af3418a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.992+0000 7f4f6e589640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56317f9c6400 0x56317f6ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.992+0000 7fa2344c8640 1 -- 172.21.15.81:0/2688234169 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(19 [pgnls-filter start_epoch 64 out=49b] v62'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562636bdad80 con 0x5626360c3800 2023-11-08T16:34:51.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.992+0000 7fa22ecbd640 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:20 8.1e 8:78000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0380 con 0x562636c6ec00 2023-11-08T16:34:51.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.992+0000 7f0155bea640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55af34d63c00 0x55af3418a580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.992+0000 7f0155bea640 1 -- 172.21.15.81:0/1907834121 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af34c14fc0 con 0x55af34d63c00 2023-11-08T16:34:51.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.992+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(20 [pgnls-filter start_epoch 64 out=81b] v64'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.992+0000 7f0150be0640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55af341b4000 0x55af3418ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.992+0000 7f0150be0640 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:6 8.19 8:98000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6000 con 0x55af341b4000 2023-11-08T16:34:51.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.993+0000 7fa23642b5c0 20 datascan.forall_objects: OSD matched oid 1000000020c.00000000 2023-11-08T16:34:51.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.993+0000 7fa23642b5c0 10 datascan.operator(): handling object 1000000020c.0 2023-11-08T16:34:51.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.993+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:21 8.1e 8:789b801b:::1000000020c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0700 con 0x562636c6ec00 2023-11-08T16:34:51.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.993+0000 7f4f6e589640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56317f9c6400 0x56317f6ec100 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.993+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(21 1000000020c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.993+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5626360c2800 0x5626360be580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.993+0000 7f01553e9640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55af341b4000 0x55af3418ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.993+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:22 9.1e 9:789b801b:::1000000020c.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0e00 con 0x5626360c2800 2023-11-08T16:34:51.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.993+0000 7f4f6e589640 1 -- 172.21.15.81:0/1920396101 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x563180522d80 con 0x56317f9c6400 2023-11-08T16:34:51.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.993+0000 7fa233cc7640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5626360c2800 0x5626360be580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.993+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:3 8.0 8:00000000::::head [pgnls-filter start_epoch 64 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6000 con 0x56317f9c6800 2023-11-08T16:34:51.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(13 10000000000.00000000 [omap-set-vals] v64'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fa233cc7640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5626360c2800 0x5626360be580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7f7f0de0e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-11-08T16:34:51.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 64 out=81b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56317f9c6000 0x56317f6ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:4 8.1b 8:db6db6db::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6380 con 0x56317f9c6000 2023-11-08T16:34:51.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7f01553e9640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55af341b4000 0x55af3418ab00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:14 8.2a 8:57fe7535::::head [pgnls-filter start_epoch 64 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fae000 con 0x55def0ea0c00 2023-11-08T16:34:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(20 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fb3878145c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fb3878145c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fb3878145c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/22, will insert to root 2023-11-08T16:34:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:21 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2b880 con 0x5614225d4800 2023-11-08T16:34:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(22 1000000020c.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562636bdab40 con 0x5626360c2800 2023-11-08T16:34:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fa23642b5c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fa23642b5c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020c 2023-11-08T16:34:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fa23642b5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/23 2023-11-08T16:34:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fa23642b5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=23 2023-11-08T16:34:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:23 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b1180 con 0x562636c6ec00 2023-11-08T16:34:51.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.994+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(14 [pgnls-filter start_epoch 64 out=81b] v64'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f7f0de0e5c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-11-08T16:34:51.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f7f0de0e5c0 10 datascan.operator(): handling object 100000001f6.0 2023-11-08T16:34:51.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(19 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f3d3a2175c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=20 frag=*, r=0 2023-11-08T16:34:51.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:15 8.2a 8:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faea80 con 0x55def0ea0c00 2023-11-08T16:34:51.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f3d3a2175c0 20 datascan.read_dentry: 20_head not found in result 2023-11-08T16:34:51.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f3d3a2175c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000209 at 0x0x10000000000/20 with size=6291456 bytes 2023-11-08T16:34:51.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:20 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37ee00 con 0x56091becd000 2023-11-08T16:34:51.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f4f6dd88640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56317f9c6000 0x56317f6ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(9 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fe23320c5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fe23320c5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fe23320c5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/16, will insert to root 2023-11-08T16:34:51.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:10 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dce00 con 0x56303bd78800 2023-11-08T16:34:51.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(21 10000000000.00000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fb3878145c0 10 datascan.find_or_create_dirfrag: read_version = 21 2023-11-08T16:34:51.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fb3878145c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:22 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421707c00 con 0x5614225d4800 2023-11-08T16:34:51.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(15 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(23 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 64 out=101b] v64'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55af34c14b40 con 0x55af341b4000 2023-11-08T16:34:51.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:16 9.a 9:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faee00 con 0x55def0ea1800 2023-11-08T16:34:51.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fa23642b5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fa23642b5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fa23642b5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/23, will insert to root 2023-11-08T16:34:51.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:24 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b1500 con 0x562636c6ec00 2023-11-08T16:34:51.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f01574425c0 20 datascan.forall_objects: OSD matched oid 10000000200.00000000 2023-11-08T16:34:51.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f01574425c0 10 datascan.operator(): handling object 10000000200.0 2023-11-08T16:34:51.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.995+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:7 8.19 8:992a6246:::10000000200.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6700 con 0x55af341b4000 2023-11-08T16:34:51.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f4f6dd88640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56317f9c6000 0x56317f6ebb80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f7f0c1f7640 1 -- 172.21.15.81:0/1660207497 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(16 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55def0cb8d80 con 0x55def0ea1800 2023-11-08T16:34:51.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f7f0de0e5c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f7f0de0e5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-11-08T16:34:51.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f7f0de0e5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-11-08T16:34:51.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f7f0de0e5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-11-08T16:34:51.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:17 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fafc00 con 0x55def0ea0c00 2023-11-08T16:34:51.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(20 10000000000.00000000 [omap-set-vals] v64'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56091bde8b40 con 0x56091becd000 2023-11-08T16:34:51.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f3d3a2175c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-11-08T16:34:51.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56091affb800 0x56091b36f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(7 10000000200.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55af34c14b40 con 0x55af341b4000 2023-11-08T16:34:51.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:8 9.19 9:992a6246:::10000000200.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6e00 con 0x55af34d49000 2023-11-08T16:34:51.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.996+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:21 8.1c 8:38000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37f500 con 0x56091affb800 2023-11-08T16:34:51.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(10 10000000000.00000000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563180522fc0 con 0x56317f9c6000 2023-11-08T16:34:51.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fe23320c5c0 10 datascan.find_or_create_dirfrag: read_version = 22 2023-11-08T16:34:51.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fe23320c5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:11 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dd180 con 0x56303bd78800 2023-11-08T16:34:51.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(8 10000000200.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f01574425c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f01574425c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000200 2023-11-08T16:34:51.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f01574425c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/11 2023-11-08T16:34:51.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f01574425c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=11 2023-11-08T16:34:51.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7180 con 0x55af34d49000 2023-11-08T16:34:51.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f4f6957f640 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:5 8.3b 8:dc000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317f6d9880 con 0x56317f9c6000 2023-11-08T16:34:51.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f3d38597640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56091affb800 0x56091b36f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(24 10000000000.00000000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fa23642b5c0 10 datascan.find_or_create_dirfrag: read_version = 22 2023-11-08T16:34:51.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fa23642b5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:25 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b1880 con 0x562636c6ec00 2023-11-08T16:34:51.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(22 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fb3878145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=22 frag=*, r=0 2023-11-08T16:34:51.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fb3878145c0 20 datascan.read_dentry: 22_head not found in result 2023-11-08T16:34:51.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fb3878145c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020b at 0x0x10000000000/22 with size=6291456 bytes 2023-11-08T16:34:51.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:23 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a380 con 0x5614225d4800 2023-11-08T16:34:51.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(17 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f7f0de0e5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f7f0de0e5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f7f0de0e5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/1, will insert to root 2023-11-08T16:34:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:18 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faf180 con 0x55def0ea0c00 2023-11-08T16:34:51.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(11 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fe23320c5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=16 frag=*, r=0 2023-11-08T16:34:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fe23320c5c0 20 datascan.read_dentry: 16_head not found in result 2023-11-08T16:34:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fe23320c5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000205 at 0x0x10000000000/16 with size=6291456 bytes 2023-11-08T16:34:51.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:12 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303ae9b880 con 0x56303bd78800 2023-11-08T16:34:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(9 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f01574425c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f01574425c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f01574425c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/11, will insert to root 2023-11-08T16:34:51.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.997+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:10 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7500 con 0x55af34d49000 2023-11-08T16:34:51.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.998+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(25 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.998+0000 7fa23642b5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=23 frag=*, r=0 2023-11-08T16:34:51.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.998+0000 7fa23642b5c0 20 datascan.read_dentry: 23_head not found in result 2023-11-08T16:34:51.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.998+0000 7fa23642b5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020c at 0x0x10000000000/23 with size=6291456 bytes 2023-11-08T16:34:51.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.998+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:26 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0000 con 0x562636c6ec00 2023-11-08T16:34:51.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.998+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 64 out=81b] v64'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563180522fc0 con 0x56317f9c6000 2023-11-08T16:34:51.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.998+0000 7f4f6fe385c0 20 datascan.forall_objects: OSD matched oid 100000001fe.00000000 2023-11-08T16:34:51.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.998+0000 7f4f6fe385c0 10 datascan.operator(): handling object 100000001fe.0 2023-11-08T16:34:51.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.998+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:6 8.3b 8:deccbd37:::100000001fe.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6700 con 0x56317f9c6000 2023-11-08T16:34:51.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.998+0000 7f3d38597640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56091affb800 0x56091b36f600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(23 10000000000.00000000 [omap-set-vals] v64'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7fb3878145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-11-08T16:34:51.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7f3d38597640 1 -- 172.21.15.81:0/3366333913 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(21 [pgnls-filter start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56091bde8d80 con 0x56091affb800 2023-11-08T16:34:51.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(6 100000001fe.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x563180522fc0 con 0x56317f9c6000 2023-11-08T16:34:51.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x561421a19c00 0x561421a34580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:24 8.10 8:08000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2aa80 con 0x561421a19c00 2023-11-08T16:34:51.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7f3d32d8c640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56091b398000 0x56091b371180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7f3d32d8c640 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:22 8.3c 8:3c000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b3a0000 con 0x56091b398000 2023-11-08T16:34:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:7 9.1b 9:deccbd37:::100000001fe.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6380 con 0x56317f9c6800 2023-11-08T16:34:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(18 10000000000.00000000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7f7f0de0e5c0 10 datascan.find_or_create_dirfrag: read_version = 23 2023-11-08T16:34:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7f7f0de0e5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:50.999+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:19 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faf500 con 0x55def0ea0c00 2023-11-08T16:34:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.000+0000 7f3d37d96640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56091b398000 0x56091b371180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.000+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(7 100000001fe.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.000+0000 7fb385ce8640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x561421a19c00 0x561421a34580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.000+0000 7f4f6fe385c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.000+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fe 2023-11-08T16:34:51.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.000+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/9 2023-11-08T16:34:51.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.000+0000 7f4f6fe385c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=9 2023-11-08T16:34:51.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.000+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:8 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6000 con 0x56317f9c6800 2023-11-08T16:34:51.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.000+0000 7fb385ce8640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x561421a19c00 0x561421a34580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.001+0000 7f3d37d96640 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56091b398000 0x56091b371180 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.001+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(24 [pgnls-filter start_epoch 64 out=81b] v64'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56142255ad80 con 0x561421a19c00 2023-11-08T16:34:51.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.001+0000 7fb3878145c0 20 datascan.forall_objects: OSD matched oid 100000001fd.00000000 2023-11-08T16:34:51.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.001+0000 7fb3878145c0 10 datascan.operator(): handling object 100000001fd.0 2023-11-08T16:34:51.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.001+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:25 8.10 8:08b99fea:::100000001fd.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2b180 con 0x561421a19c00 2023-11-08T16:34:51.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7f3d37d96640 1 -- 172.21.15.81:0/3366333913 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(22 [pgnls-filter start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56091bde8fc0 con 0x56091b398000 2023-11-08T16:34:51.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(12 10000000000.00000000 [omap-set-vals] v64'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7fe23320c5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-11-08T16:34:51.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56303b1b1400 0x56303aeaf700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7f3d3358d640 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:23 8.2 8:40000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b37e000 con 0x56091bf39400 2023-11-08T16:34:51.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:13 8.3 8:c0000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dc000 con 0x56303b1b1400 2023-11-08T16:34:51.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(10 10000000000.00000000 [omap-get-header out=274b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7f01574425c0 10 datascan.find_or_create_dirfrag: read_version = 24 2023-11-08T16:34:51.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7f01574425c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:11 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7880 con 0x55af34d49000 2023-11-08T16:34:51.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7fe2305b2640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56303b1b1400 0x56303aeaf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(25 100000001fd.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x56142255ad80 con 0x561421a19c00 2023-11-08T16:34:51.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x561421a32000 0x561421a34b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:26 9.10 9:08b99fea:::100000001fd.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2b500 con 0x561421a32000 2023-11-08T16:34:51.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.002+0000 7f3d37d96640 1 -- 172.21.15.81:0/3366333913 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56091bde8fc0 con 0x56091bf39400 2023-11-08T16:34:51.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.003+0000 7f3d32d8c640 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:24 8.22 8:44000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b3a0380 con 0x56091b398000 2023-11-08T16:34:51.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.003+0000 7fe2305b2640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56303b1b1400 0x56303aeaf700 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.003+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56303bbacb40 con 0x56303b1b1400 2023-11-08T16:34:51.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.003+0000 7fe22bda9640 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:14 8.23 8:c4000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303bc8a000 con 0x56303bd78000 2023-11-08T16:34:51.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.003+0000 7fb3854e7640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x561421a32000 0x561421a34b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.003+0000 7f3d37d96640 1 -- 172.21.15.81:0/3366333913 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(24 [pgnls-filter start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56091bde8fc0 con 0x56091b398000 2023-11-08T16:34:51.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.003+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(14 [pgnls-filter start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56303bbacfc0 con 0x56303bd78000 2023-11-08T16:34:51.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.003+0000 7fe22c5aa640 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:15 8.13 8:c8000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dca80 con 0x56303bd78800 2023-11-08T16:34:51.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.003+0000 7f3d3358d640 1 -- 172.21.15.81:0/3366333913 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:25 8.12 8:48000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56091b3a0000 con 0x56091beccc00 2023-11-08T16:34:51.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(15 [pgnls-filter start_epoch 64 out=81b] v64'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7fe23320c5c0 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-11-08T16:34:51.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7fe23320c5c0 10 datascan.operator(): handling object 100000001fb.0 2023-11-08T16:34:51.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:16 8.13 8:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dce00 con 0x56303bd78800 2023-11-08T16:34:51.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7fb3854e7640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x561421a32000 0x561421a34b00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7f3d38597640 1 -- 172.21.15.81:0/3366333913 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 64 out=49b] v62'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56091bde8d80 con 0x56091beccc00 2023-11-08T16:34:51.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(26 10000000000.00000000 [omap-set-vals] v64'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562636bdab40 con 0x562636c6ec00 2023-11-08T16:34:51.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7fa23642b5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-11-08T16:34:51.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:27 8.3e 8:7c000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360b0380 con 0x5626360c3800 2023-11-08T16:34:51.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(16 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56303b1b1c00 0x56303b1b6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.004+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:17 9.13 9:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dd500 con 0x56303b1b1c00 2023-11-08T16:34:51.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(19 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f7f0de0e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-11-08T16:34:51.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f7f0de0e5c0 20 datascan.read_dentry: 1_head not found in result 2023-11-08T16:34:51.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f7f0de0e5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/1 with size=6291456 bytes 2023-11-08T16:34:51.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:20 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faf880 con 0x55def0ea0c00 2023-11-08T16:34:51.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(8 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f4f6fe385c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f4f6fe385c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f4f6fe385c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/9, will insert to root 2023-11-08T16:34:51.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6e00 con 0x56317f9c6800 2023-11-08T16:34:51.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7fb3854e7640 1 -- 172.21.15.81:0/3305262764 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(26 100000001fd.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56142255afc0 con 0x561421a32000 2023-11-08T16:34:51.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7fb3878145c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7fb3878145c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fd 2023-11-08T16:34:51.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7fb3878145c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/8 2023-11-08T16:34:51.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7fb3878145c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=8 2023-11-08T16:34:51.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56091bf39800 msgr2=0x56091b36eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56091bf39800 0x56091b36eb00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56091becd000 msgr2=0x56091b067180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56091becd000 0x56091b067180 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56091bf39400 msgr2=0x56091b36e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56091bf39400 0x56091b36e580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56091affb800 msgr2=0x56091b36f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56091affb800 0x56091b36f600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56091beccc00 msgr2=0x56091b067700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56091beccc00 0x56091b067700 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:27 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2b880 con 0x5614225d4800 2023-11-08T16:34:51.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56091b398000 msgr2=0x56091b371180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56091b398000 0x56091b371180 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 reap_dead start 2023-11-08T16:34:51.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d37595640 1 -- 172.21.15.81:0/3366333913 reap_dead start 2023-11-08T16:34:51.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7fe2315b4640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56303b1b1c00 0x56303b1b6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56091bf39000 msgr2=0x56091b36e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56091bf39000 0x56091b36e000 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x56091bee3770 tx=0x56091bee4de0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091becd800 msgr2=0x56091b066680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091becd800 0x56091b066680 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x56091be1d770 tx=0x56091bee4c00 comp rx=0 tx=0).stop 2023-11-08T16:34:51.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(11 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f01574425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=11 frag=*, r=0 2023-11-08T16:34:51.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f01574425c0 20 datascan.read_dentry: 11_head not found in result 2023-11-08T16:34:51.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f01574425c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000200 at 0x0x10000000000/11 with size=6291456 bytes 2023-11-08T16:34:51.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.005+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:12 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7c00 con 0x55af34d49000 2023-11-08T16:34:51.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.006+0000 7fa2344c8640 1 -- 172.21.15.81:0/2688234169 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 64 out=49b] v62'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562636bdad80 con 0x5626360c3800 2023-11-08T16:34:51.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.006+0000 7fa22ecbd640 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:28 8.1 8:80000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360d6000 con 0x562636c6f400 2023-11-08T16:34:51.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.006+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 shutdown_connections 2023-11-08T16:34:51.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.006+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56091bf39000 0x56091b36e000 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.006+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/3366333913 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091becd800 0x56091b066680 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.006+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 >> 172.21.15.81:0/3366333913 conn(0x56091bdd6400 msgr2=0x56091b0f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.006+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(28 [pgnls-filter start_epoch 64 out=49b] v62'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562636bdab40 con 0x562636c6f400 2023-11-08T16:34:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.006+0000 7fa22f4be640 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:29 8.21 8:84000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360d6700 con 0x5626360c2800 2023-11-08T16:34:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.006+0000 7fe2315b4640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56303b1b1c00 0x56303b1b6580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.006+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 shutdown_connections 2023-11-08T16:34:51.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.006+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/3366333913 wait complete. 2023-11-08T16:34:51.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7fa233cc7640 1 -- 172.21.15.81:0/2688234169 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(29 [pgnls-filter start_epoch 64 out=49b] v62'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562636bdab40 con 0x5626360c2800 2023-11-08T16:34:51.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7fa22ecbd640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562636c02800 0x5626360c0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7fa22ecbd640 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:30 8.11 8:88000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360d6380 con 0x562636c02800 2023-11-08T16:34:51.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7fe2315b4640 1 -- 172.21.15.81:0/2400868731 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(17 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56303bbacd80 con 0x56303b1b1c00 2023-11-08T16:34:51.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7fe23320c5c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7fe23320c5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-11-08T16:34:51.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7fe23320c5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/6 2023-11-08T16:34:51.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7fe23320c5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=6 2023-11-08T16:34:51.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:18 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dd880 con 0x56303bd78800 2023-11-08T16:34:51.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2762196039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6800 msgr2=0x56091b065b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/2762196039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6800 0x56091b065b80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x56091be1d290 tx=0x56091bde0ae0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(20 10000000000.00000000 [omap-set-vals] v64'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f7f0de0e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-11-08T16:34:51.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:21 8.1a 8:58000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fae000 con 0x55def0fb8000 2023-11-08T16:34:51.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(9 10000000000.00000000 [omap-get-header out=274b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2762196039 shutdown_connections 2023-11-08T16:34:51.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/2762196039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56091bdd7400 0x56091b065080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/2762196039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56091bdd7c00 0x56091b065600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f3d3a2175c0 1 --2- 172.21.15.81:0/2762196039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56091bdd6800 0x56091b065b80 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2762196039 >> 172.21.15.81:0/2762196039 conn(0x56091affb000 msgr2=0x56091b0f9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.007+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0fb8000 2023-11-08T16:34:51.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.008+0000 7f4f6fe385c0 10 datascan.find_or_create_dirfrag: read_version = 26 2023-11-08T16:34:51.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.008+0000 7f4f6fe385c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.008+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:10 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7180 con 0x56317f9c6800 2023-11-08T16:34:51.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.008+0000 7f7f069ec640 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:22 8.3a 8:5c000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faea80 con 0x55def0fb8400 2023-11-08T16:34:51.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.008+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2762196039 shutdown_connections 2023-11-08T16:34:51.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.008+0000 7fa2334c6640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562636c02800 0x5626360c0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.008+0000 7f3d3a2175c0 1 -- 172.21.15.81:0/2762196039 wait complete. 2023-11-08T16:34:51.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.008+0000 7f7f0b1f5640 1 -- 172.21.15.81:0/1660207497 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 64 out=49b] v62'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55def0cb8fc0 con 0x55def0fb8400 2023-11-08T16:34:51.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.008+0000 7f7f071ed640 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:23 8.6 8:60000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0b97c00 con 0x55def0fb8000 2023-11-08T16:34:51.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.009+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0fb8000 2023-11-08T16:34:51.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.009+0000 7f7f069ec640 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:24 8.26 8:64000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fafc00 con 0x55def0ea0c00 2023-11-08T16:34:51.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.009+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(24 [pgnls-filter start_epoch 64 out=81b] v64'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.009+0000 7f7f0de0e5c0 20 datascan.forall_objects: OSD matched oid 100000001ff.00000000 2023-11-08T16:34:51.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.009+0000 7f7f0de0e5c0 10 datascan.operator(): handling object 100000001ff.0 2023-11-08T16:34:51.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.009+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(12 10000000000.00000000 [omap-set-vals] v64'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.009+0000 7f01574425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-11-08T16:34:51.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.009+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:13 8.19 8:997b7129::::head [pgnls-filter start_epoch 64 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6700 con 0x55af341b4000 2023-11-08T16:34:51.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.009+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:25 8.26 8:658f2455:::100000001ff.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fae700 con 0x55def0ea0c00 2023-11-08T16:34:51.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.009+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(27 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fa2334c6640 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562636c02800 0x5626360c0c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(25 100000001ff.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(18 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fe23320c5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fe23320c5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fe23320c5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/6, will insert to root 2023-11-08T16:34:51.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fb3878145c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fb3878145c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fb3878145c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/8, will insert to root 2023-11-08T16:34:51.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:28 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2bc00 con 0x5614225d4800 2023-11-08T16:34:51.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:19 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1ddc00 con 0x56303bd78800 2023-11-08T16:34:51.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:26 9.6 9:658f2455:::100000001ff.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faf180 con 0x55def0ea1800 2023-11-08T16:34:51.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(10 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f4f6fe385c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=9 frag=*, r=0 2023-11-08T16:34:51.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f4f6fe385c0 20 datascan.read_dentry: 9_head not found in result 2023-11-08T16:34:51.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fe at 0x0x10000000000/9 with size=6291456 bytes 2023-11-08T16:34:51.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:11 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317f6d9880 con 0x56317f9c6800 2023-11-08T16:34:51.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 64 out=81b] v64'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55af34c14b40 con 0x55af341b4000 2023-11-08T16:34:51.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(19 10000000000.00000000 [omap-get-header out=274b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f01574425c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-11-08T16:34:51.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f01574425c0 10 datascan.operator(): handling object 100000001f8.0 2023-11-08T16:34:51.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:14 8.19 8:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6a80 con 0x55af341b4000 2023-11-08T16:34:51.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fe23320c5c0 10 datascan.find_or_create_dirfrag: read_version = 27 2023-11-08T16:34:51.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fe23320c5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:20 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dd500 con 0x56303bd78800 2023-11-08T16:34:51.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fa2334c6640 1 -- 172.21.15.81:0/2688234169 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(30 [pgnls-filter start_epoch 64 out=49b] v62'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562636bdafc0 con 0x562636c02800 2023-11-08T16:34:51.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(28 10000000000.00000000 [omap-get-header out=274b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f7f0c1f7640 1 -- 172.21.15.81:0/1660207497 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(26 100000001ff.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55def0cb8d80 con 0x55def0ea1800 2023-11-08T16:34:51.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fa22f4be640 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:31 8.31 8:8c000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360d6a80 con 0x5626360c3800 2023-11-08T16:34:51.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fb3878145c0 10 datascan.find_or_create_dirfrag: read_version = 27 2023-11-08T16:34:51.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fb3878145c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:29 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a000 con 0x5614225d4800 2023-11-08T16:34:51.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f7f0de0e5c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f7f0de0e5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001ff 2023-11-08T16:34:51.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f7f0de0e5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/10 2023-11-08T16:34:51.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f7f0de0e5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=10 2023-11-08T16:34:51.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.010+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:27 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faf500 con 0x55def0ea0c00 2023-11-08T16:34:51.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.011+0000 7fa2344c8640 1 -- 172.21.15.81:0/2688234169 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(31 [pgnls-filter start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562636bdad80 con 0x5626360c3800 2023-11-08T16:34:51.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.011+0000 7fa22ecbd640 1 -- 172.21.15.81:0/2688234169 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:32 8.9 8:90000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x5626360d6a80 con 0x562636c6f000 2023-11-08T16:34:51.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.011+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(14 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x55af34c14b40 con 0x55af341b4000 2023-11-08T16:34:51.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.011+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:15 9.19 9:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6e00 con 0x55af34d49000 2023-11-08T16:34:51.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa2344c8640 1 -- 172.21.15.81:0/2688234169 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 64 out=49b] v62'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562636bdad80 con 0x562636c6f000 2023-11-08T16:34:51.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(11 10000000000.00000000 [omap-set-vals] v64'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5626360c2800 msgr2=0x5626360be580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x5626360c2800 0x5626360be580 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562636c6f400 msgr2=0x562635d99600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562636c6f400 0x562635d99600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562636c6ec00 msgr2=0x562635d9a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7f4f6fe385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-11-08T16:34:51.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562636c6ec00 0x562635d9a100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562636c6fc00 msgr2=0x5626360be000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562636c6fc00 0x5626360be000 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562636c02800 msgr2=0x5626360c0c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562636c02800 0x5626360c0c00 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562636c6f000 msgr2=0x562635d99b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562636c6f000 0x562635d99b80 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5626360c3800 msgr2=0x5626360c0680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5626360c3800 0x5626360c0680 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(15 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa2334c6640 1 -- 172.21.15.81:0/2688234169 reap_dead start 2023-11-08T16:34:51.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7f01574425c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7f01574425c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-11-08T16:34:51.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7f01574425c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-11-08T16:34:51.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7f01574425c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-11-08T16:34:51.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:16 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7180 con 0x55af34d49000 2023-11-08T16:34:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa2334c6640 1 -- 172.21.15.81:0/2688234169 reap_dead start 2023-11-08T16:34:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa2334c6640 1 -- 172.21.15.81:0/2688234169 reap_dead start 2023-11-08T16:34:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa2334c6640 1 -- 172.21.15.81:0/2688234169 reap_dead start 2023-11-08T16:34:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562636c6e800 msgr2=0x562635d9b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562636c6e800 0x562635d9b700 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x562636bea7b0 tx=0x562636be4120 comp rx=0 tx=0).stop 2023-11-08T16:34:51.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03800 msgr2=0x562635d9a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03800 0x562635d9a680 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x562636beb860 tx=0x562636abf8c0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 shutdown_connections 2023-11-08T16:34:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562636c6e800 0x562635d9b700 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2688234169 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562636c03800 0x562635d9a680 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 >> 172.21.15.81:0/2688234169 conn(0x562636c02c00 msgr2=0x562636aab000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56317f9f5400 0x56317fb08000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:12 8.7 8:e0000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6380 con 0x56317f9f5400 2023-11-08T16:34:51.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7f4f6d587640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56317f9f5400 0x56317fb08000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.012+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(20 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fe23320c5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=6 frag=*, r=0 2023-11-08T16:34:51.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fe23320c5c0 20 datascan.read_dentry: 6_head not found in result 2023-11-08T16:34:51.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fe23320c5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fb at 0x0x10000000000/6 with size=6291456 bytes 2023-11-08T16:34:51.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:21 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dc000 con 0x56303bd78800 2023-11-08T16:34:51.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 shutdown_connections 2023-11-08T16:34:51.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2688234169 wait complete. 2023-11-08T16:34:51.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(29 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(27 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7f7f0de0e5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fb3878145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=8 frag=*, r=0 2023-11-08T16:34:51.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fb3878145c0 20 datascan.read_dentry: 8_head not found in result 2023-11-08T16:34:51.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7f7f0de0e5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7f7f0de0e5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/10, will insert to root 2023-11-08T16:34:51.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:28 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0faf880 con 0x55def0ea0c00 2023-11-08T16:34:51.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fb3878145c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fd at 0x0x10000000000/8 with size=6291456 bytes 2023-11-08T16:34:51.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7f4f6d587640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56317f9f5400 0x56317fb08000 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2416983088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562635d2ec00 msgr2=0x562635d98580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2416983088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562635d2ec00 0x562635d98580 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x562636b313b0 tx=0x562636be4420 comp rx=0 tx=0).stop 2023-11-08T16:34:51.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2416983088 shutdown_connections 2023-11-08T16:34:51.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2416983088 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562635d2fc00 0x562635d99080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2416983088 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562635d2f800 0x562635d98b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7fa23642b5c0 1 --2- 172.21.15.81:0/2416983088 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562635d2ec00 0x562635d98580 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2416983088 >> 172.21.15.81:0/2416983088 conn(0x562635d2f000 msgr2=0x562636aaa800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.013+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:30 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a700 con 0x5614225d4800 2023-11-08T16:34:51.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(16 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7f01574425c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7f01574425c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7f01574425c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/3, will insert to root 2023-11-08T16:34:51.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:17 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6a80 con 0x55af34d49000 2023-11-08T16:34:51.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 64 out=101b] v64'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x563180522b40 con 0x56317f9f5400 2023-11-08T16:34:51.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2416983088 shutdown_connections 2023-11-08T16:34:51.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7fa23642b5c0 1 -- 172.21.15.81:0/2416983088 wait complete. 2023-11-08T16:34:51.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7f4f6fe385c0 20 datascan.forall_objects: OSD matched oid 10000000207.00000000 2023-11-08T16:34:51.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7f4f6fe385c0 10 datascan.operator(): handling object 10000000207.0 2023-11-08T16:34:51.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.014+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:13 8.7 8:e15a914f:::10000000207.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6000 con 0x56317f9f5400 2023-11-08T16:34:51.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.015+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(13 10000000207.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x563180522b40 con 0x56317f9f5400 2023-11-08T16:34:51.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.015+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56317f9f5c00 0x56317fb08580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.015+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:14 9.7 9:e15a914f:::10000000207.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7500 con 0x56317f9f5c00 2023-11-08T16:34:51.406 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '1', '--worker_m', '7'] in time 0:00:00.617637, result: 2023-11-08T16:34:51.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.016+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(21 10000000000.00000000 [omap-set-vals] v64'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.016+0000 7fe23320c5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-11-08T16:34:51.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.017+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:22 8.33 8:cc000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dca80 con 0x56303b1b1400 2023-11-08T16:34:51.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.017+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 18 ==== osd_op_reply(28 10000000000.00000000 [omap-get-header out=274b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.017+0000 7f4f6e589640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56317f9f5c00 0x56317fb08580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.017+0000 7f7f0de0e5c0 10 datascan.find_or_create_dirfrag: read_version = 29 2023-11-08T16:34:51.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.017+0000 7f7f0de0e5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.017+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:29 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fae000 con 0x55def0ea0c00 2023-11-08T16:34:51.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.017+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 64 out=49b] v62'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56303bbacb40 con 0x56303b1b1400 2023-11-08T16:34:51.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.017+0000 7fe22c5aa640 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:23 8.b 8:d0000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dc700 con 0x56303bd78800 2023-11-08T16:34:51.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.017+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(23 [pgnls-filter start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.018+0000 7fe22bda9640 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:24 8.2b 8:d4000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303bc8a380 con 0x56303bd78400 2023-11-08T16:34:51.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.018+0000 7f4f6e589640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56317f9f5c00 0x56317fb08580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.018+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(30 10000000000.00000000 [omap-set-vals] v64'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.018+0000 7fb3878145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-11-08T16:34:51.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.018+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:31 8.30 8:0c000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2b500 con 0x561421a19400 2023-11-08T16:34:51.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(17 10000000000.00000000 [omap-get-header out=274b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f01574425c0 10 datascan.find_or_create_dirfrag: read_version = 30 2023-11-08T16:34:51.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f01574425c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:18 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6000 con 0x55af34d49000 2023-11-08T16:34:51.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 19 ==== osd_op_reply(29 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f7f0de0e5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=10 frag=*, r=0 2023-11-08T16:34:51.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7fe2315b4640 1 -- 172.21.15.81:0/2400868731 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(24 [pgnls-filter start_epoch 64 out=81b] v64'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56303bbacd80 con 0x56303bd78400 2023-11-08T16:34:51.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f7f0de0e5c0 20 datascan.read_dentry: 10_head not found in result 2023-11-08T16:34:51.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f7f0de0e5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001ff at 0x0x10000000000/10 with size=6291456 bytes 2023-11-08T16:34:51.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:30 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0b97880 con 0x55def0ea0c00 2023-11-08T16:34:51.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7fe23320c5c0 20 datascan.forall_objects: OSD matched oid 10000000203.00000000 2023-11-08T16:34:51.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7fe23320c5c0 10 datascan.operator(): handling object 10000000203.0 2023-11-08T16:34:51.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7fb3854e7640 1 -- 172.21.15.81:0/3305262764 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(31 [pgnls-filter start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56142255afc0 con 0x561421a19400 2023-11-08T16:34:51.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:25 8.2b 8:d4a58307:::10000000203.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dd180 con 0x56303bd78400 2023-11-08T16:34:51.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7fb3804dd640 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:32 8.8 8:10000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a4a000 con 0x561421a19800 2023-11-08T16:34:51.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(18 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f01574425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-11-08T16:34:51.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f01574425c0 20 datascan.read_dentry: 3_head not found in result 2023-11-08T16:34:51.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f01574425c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/3 with size=6291456 bytes 2023-11-08T16:34:51.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.019+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:19 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6700 con 0x55af34d49000 2023-11-08T16:34:51.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.020+0000 7f4f6e589640 1 -- 172.21.15.81:0/1920396101 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(14 10000000207.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x563180522d80 con 0x56317f9f5c00 2023-11-08T16:34:51.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.020+0000 7f4f6fe385c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.020+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000207 2023-11-08T16:34:51.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.020+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/18 2023-11-08T16:34:51.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.020+0000 7f4f6fe385c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=18 2023-11-08T16:34:51.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.020+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:15 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7880 con 0x56317f9c6800 2023-11-08T16:34:51.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.020+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56142255ab40 con 0x561421a19800 2023-11-08T16:34:51.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.020+0000 7fb380cde640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561421a46000 0x561421a36680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.020+0000 7fb380cde640 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:33 8.28 8:14000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421707c00 con 0x561421a46000 2023-11-08T16:34:51.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.020+0000 7fe2315b4640 1 -- 172.21.15.81:0/2400868731 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(25 10000000203.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x56303bbacd80 con 0x56303bd78400 2023-11-08T16:34:51.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.020+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:26 9.b 9:d4a58307:::10000000203.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303ae9bc00 con 0x56303bd78000 2023-11-08T16:34:51.423 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '3', '--worker_m', '7'] in time 0:00:00.633786, result: 2023-11-08T16:34:51.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.021+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(26 10000000203.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56303bbacfc0 con 0x56303bd78000 2023-11-08T16:34:51.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.021+0000 7fe23320c5c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.021+0000 7fe23320c5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000203 2023-11-08T16:34:51.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.021+0000 7fe23320c5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/14 2023-11-08T16:34:51.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.021+0000 7fe23320c5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=14 2023-11-08T16:34:51.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.021+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:27 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303ae9b880 con 0x56303bd78800 2023-11-08T16:34:51.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.021+0000 7fb384ce6640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561421a46000 0x561421a36680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.021+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 20 ==== osd_op_reply(30 10000000000.00000000 [omap-set-vals] v64'31 uv31 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.021+0000 7f7f0de0e5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-11-08T16:34:51.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.021+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:31 8.16 8:68000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0b97c00 con 0x55def0ea0c00 2023-11-08T16:34:51.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.021+0000 7fb384ce6640 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561421a46000 0x561421a36680 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f7f0b9f6640 1 -- 172.21.15.81:0/1660207497 <== osd.3 v2:172.21.15.81:6816/2756229107 21 ==== osd_op_reply(31 [pgnls-filter start_epoch 64 out=49b] v62'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55def0cb8b40 con 0x55def0ea0c00 2023-11-08T16:34:51.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 64 out=101b] v64'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x56142255ab40 con 0x561421a46000 2023-11-08T16:34:51.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7fb3878145c0 20 datascan.forall_objects: OSD matched oid 10000000202.00000000 2023-11-08T16:34:51.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7fb3878145c0 10 datascan.operator(): handling object 10000000202.0 2023-11-08T16:34:51.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f7f069ec640 1 -- 172.21.15.81:0/1660207497 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:32 8.36 8:6c000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55def0fae000 con 0x55def0fb8400 2023-11-08T16:34:51.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(19 10000000000.00000000 [omap-set-vals] v64'32 uv32 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f01574425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-11-08T16:34:51.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:34 8.28 8:15246602:::10000000202.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a380 con 0x561421a46000 2023-11-08T16:34:51.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55af34d48000 0x55af33e8a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:20 8.39 8:9c000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7c00 con 0x55af34d48000 2023-11-08T16:34:51.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(27 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7fe23320c5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7fe23320c5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7fe23320c5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/14, will insert to root 2023-11-08T16:34:51.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(15 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:28 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dd880 con 0x56303bd78800 2023-11-08T16:34:51.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f4f6fe385c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f4f6fe385c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f4f6fe385c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/18, will insert to root 2023-11-08T16:34:51.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:16 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7c00 con 0x56317f9c6800 2023-11-08T16:34:51.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f7f0b1f5640 1 -- 172.21.15.81:0/1660207497 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(32 [pgnls-filter start_epoch 64 out=101b] v64'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55def0cb8fc0 con 0x55def0fb8400 2023-11-08T16:34:51.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55def0fb8000 msgr2=0x55def0fbe000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55def0fb8000 0x55def0fbe000 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.022+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(34 10000000202.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x56142255ab40 con 0x561421a46000 2023-11-08T16:34:51.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:35 9.8 9:15246602:::10000000202.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2bc00 con 0x5614225d4800 2023-11-08T16:34:51.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55def0fb8400 msgr2=0x55def0fbe580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55def0fb8400 0x55def0fbe580 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55def0ea0c00 msgr2=0x55def0baa100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55def0ea0c00 0x55def0baa100 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55def0ea1800 msgr2=0x55def0ba9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55def0ea1800 0x55def0ba9600 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55def0ea1000 msgr2=0x55def0ba9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55def0ea1000 0x55def0ba9b80 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0b1f5640 1 -- 172.21.15.81:0/1660207497 reap_dead start 2023-11-08T16:34:51.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0b1f5640 1 -- 172.21.15.81:0/1660207497 reap_dead start 2023-11-08T16:34:51.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55def0ea0800 msgr2=0x55def0bab700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55def0ea0800 0x55def0bab700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x55def0e22690 tx=0x55def0e1a180 comp rx=0 tx=0).stop 2023-11-08T16:34:51.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 msgr2=0x55def0baa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 0x55def0baa680 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55def0e23350 tx=0x55def0e18d20 comp rx=0 tx=0).stop 2023-11-08T16:34:51.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(28 10000000000.00000000 [omap-get-header out=274b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7fe23320c5c0 10 datascan.find_or_create_dirfrag: read_version = 32 2023-11-08T16:34:51.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7fe23320c5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:29 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1ddc00 con 0x56303bd78800 2023-11-08T16:34:51.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 18 ==== osd_op_reply(35 10000000202.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7fb3878145c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7fb3878145c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000202 2023-11-08T16:34:51.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7fb3878145c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/13 2023-11-08T16:34:51.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7fb3878145c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=13 2023-11-08T16:34:51.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:36 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a000 con 0x5614225d4800 2023-11-08T16:34:51.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 shutdown_connections 2023-11-08T16:34:51.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55def0ea0800 0x55def0bab700 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/1660207497 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0e33800 0x55def0baa680 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 >> 172.21.15.81:0/1660207497 conn(0x55def0e32c00 msgr2=0x55def18b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f01563eb640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55af34d48000 0x55af33e8a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.023+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(16 10000000000.00000000 [omap-get-header out=274b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f4f6fe385c0 10 datascan.find_or_create_dirfrag: read_version = 32 2023-11-08T16:34:51.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f4f6fe385c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:17 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7180 con 0x56317f9c6800 2023-11-08T16:34:51.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 shutdown_connections 2023-11-08T16:34:51.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/1660207497 wait complete. 2023-11-08T16:34:51.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(29 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fe23320c5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=14 frag=*, r=0 2023-11-08T16:34:51.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fe23320c5c0 20 datascan.read_dentry: 14_head not found in result 2023-11-08T16:34:51.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fe23320c5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000203 at 0x0x10000000000/14 with size=6291456 bytes 2023-11-08T16:34:51.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 19 ==== osd_op_reply(36 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fb3878145c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fb3878145c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fb3878145c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/13, will insert to root 2023-11-08T16:34:51.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:37 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a2a700 con 0x5614225d4800 2023-11-08T16:34:51.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:30 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dd500 con 0x56303bd78800 2023-11-08T16:34:51.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(17 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/2245676871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0b3ec00 msgr2=0x55def0ba8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/2245676871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0b3ec00 0x55def0ba8580 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x55def19433e0 tx=0x55def0e1a240 comp rx=0 tx=0).stop 2023-11-08T16:34:51.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 20 ==== osd_op_reply(37 10000000000.00000000 [omap-get-header out=274b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f01563eb640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55af34d48000 0x55af33e8a100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fb3878145c0 10 datascan.find_or_create_dirfrag: read_version = 32 2023-11-08T16:34:51.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7fb3878145c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f4f6fe385c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=18 frag=*, r=0 2023-11-08T16:34:51.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f4f6fe385c0 20 datascan.read_dentry: 18_head not found in result 2023-11-08T16:34:51.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000207 at 0x0x10000000000/18 with size=6291456 bytes 2023-11-08T16:34:51.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.024+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:18 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6380 con 0x56317f9c6800 2023-11-08T16:34:51.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.025+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:38 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a62000 con 0x5614225d4800 2023-11-08T16:34:51.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.025+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/2245676871 shutdown_connections 2023-11-08T16:34:51.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.025+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/2245676871 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55def0b3fc00 0x55def0ba9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.025+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/2245676871 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55def0b3f800 0x55def0ba8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.025+0000 7f7f0de0e5c0 1 --2- 172.21.15.81:0/2245676871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55def0b3ec00 0x55def0ba8580 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.025+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/2245676871 >> 172.21.15.81:0/2245676871 conn(0x55def0b3f000 msgr2=0x55def18b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.025+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/2245676871 shutdown_connections 2023-11-08T16:34:51.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.025+0000 7f01563eb640 1 -- 172.21.15.81:0/1907834121 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(20 [pgnls-filter start_epoch 64 out=49b] v62'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af34c14d80 con 0x55af34d48000 2023-11-08T16:34:51.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.025+0000 7f01513e1640 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:21 8.5 8:a0000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6380 con 0x55af34d48c00 2023-11-08T16:34:51.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.025+0000 7f7f0de0e5c0 1 -- 172.21.15.81:0/2245676871 wait complete. 2023-11-08T16:34:51.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7f01563eb640 1 -- 172.21.15.81:0/1907834121 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(21 [pgnls-filter start_epoch 64 out=81b] v64'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55af34c14d80 con 0x55af34d48c00 2023-11-08T16:34:51.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7f01574425c0 20 datascan.forall_objects: OSD matched oid 10000000204.00000000 2023-11-08T16:34:51.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7f01574425c0 10 datascan.operator(): handling object 10000000204.0 2023-11-08T16:34:51.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:22 8.5 8:a0beda95:::10000000204.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7880 con 0x55af34d48c00 2023-11-08T16:34:51.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(30 10000000000.00000000 [omap-set-vals] v64'33 uv33 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56303bbacb40 con 0x56303bd78800 2023-11-08T16:34:51.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7fe23320c5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-11-08T16:34:51.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56303b1f0400 0x56303b1b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:31 8.1b 8:d8000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56303b1dc000 con 0x56303b1f0400 2023-11-08T16:34:51.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 21 ==== osd_op_reply(38 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7fb3878145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=13 frag=*, r=0 2023-11-08T16:34:51.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7fb3878145c0 20 datascan.read_dentry: 13_head not found in result 2023-11-08T16:34:51.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7fb3878145c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000202 at 0x0x10000000000/13 with size=6291456 bytes 2023-11-08T16:34:51.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.026+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:39 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a62380 con 0x5614225d4800 2023-11-08T16:34:51.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.027+0000 7fe230db3640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56303b1f0400 0x56303b1b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.027+0000 7f01563eb640 1 -- 172.21.15.81:0/1907834121 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(22 10000000204.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55af34c14d80 con 0x55af34d48c00 2023-11-08T16:34:51.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.027+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:23 9.5 9:a0beda95:::10000000204.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6e00 con 0x55af34d48000 2023-11-08T16:34:51.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f01563eb640 1 -- 172.21.15.81:0/1907834121 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(23 10000000204.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55af34c14d80 con 0x55af34d48000 2023-11-08T16:34:51.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7fe230db3640 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56303b1f0400 0x56303b1b6b00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(18 10000000000.00000000 [omap-set-vals] v64'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f4f6fe385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-11-08T16:34:51.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:19 8.7 8:e2e07930::::head [pgnls-filter start_epoch 64 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6e00 con 0x56317f9f5400 2023-11-08T16:34:51.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f01574425c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f01574425c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000204 2023-11-08T16:34:51.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f01574425c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/15 2023-11-08T16:34:51.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f01574425c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=15 2023-11-08T16:34:51.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:24 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6380 con 0x55af34d49000 2023-11-08T16:34:51.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 64 out=81b] v64'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563180522b40 con 0x56317f9f5400 2023-11-08T16:34:51.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7fe230db3640 1 -- 172.21.15.81:0/2400868731 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(31 [pgnls-filter start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56303bbacfc0 con 0x56303b1f0400 2023-11-08T16:34:51.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 22 ==== osd_op_reply(39 10000000000.00000000 [omap-set-vals] v64'35 uv35 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fb3878145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-11-08T16:34:51.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:40 8.28 8:177a762d::::head [pgnls-filter start_epoch 64 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a62700 con 0x561421a46000 2023-11-08T16:34:51.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(24 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56303bd78000 msgr2=0x56303aeaf180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56303bd78000 0x56303aeaf180 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56303b1b1400 msgr2=0x56303aeaf700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56303b1b1400 0x56303aeaf700 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f01574425c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f01574425c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f01574425c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/15, will insert to root 2023-11-08T16:34:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:25 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7500 con 0x55af34d49000 2023-11-08T16:34:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f4f6fe385c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-11-08T16:34:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.028+0000 7f4f6fe385c0 10 datascan.operator(): handling object 10000000001.0 2023-11-08T16:34:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:20 8.7 8:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317f6d9880 con 0x56317f9f5400 2023-11-08T16:34:51.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56303bd78800 msgr2=0x56303aeae680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56303bd78800 0x56303aeae680 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56303b1f0400 msgr2=0x56303b1b6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56303b1f0400 0x56303b1b6b00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56303bd78400 msgr2=0x56303aeaec00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56303bd78400 0x56303aeaec00 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56303b1b1c00 msgr2=0x56303b1b6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56303b1b1c00 0x56303b1b6580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 reap_dead start 2023-11-08T16:34:51.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe2305b2640 1 -- 172.21.15.81:0/2400868731 reap_dead start 2023-11-08T16:34:51.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56303b1b0000 msgr2=0x56303b1b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56303b1b0000 0x56303b1b6000 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x56303af46990 tx=0x56303bd29bc0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd79000 msgr2=0x56303aeac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd79000 0x56303aeac580 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x56303bc48cc0 tx=0x56303bc38180 comp rx=0 tx=0).stop 2023-11-08T16:34:51.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(40 [pgnls-filter start_epoch 64 out=81b] v64'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56142255ab40 con 0x561421a46000 2023-11-08T16:34:51.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(25 10000000000.00000000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f01574425c0 10 datascan.find_or_create_dirfrag: read_version = 35 2023-11-08T16:34:51.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f01574425c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:26 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7880 con 0x55af34d49000 2023-11-08T16:34:51.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(20 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x563180522b40 con 0x56317f9f5400 2023-11-08T16:34:51.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:21 9.7 9:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7880 con 0x56317f9f5c00 2023-11-08T16:34:51.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 shutdown_connections 2023-11-08T16:34:51.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56303b1b0000 0x56303b1b6000 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/2400868731 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303bd79000 0x56303aeac580 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 >> 172.21.15.81:0/2400868731 conn(0x56303bc55c00 msgr2=0x56303af35000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fb3878145c0 20 datascan.forall_objects: OSD matched oid 10000000201.00000000 2023-11-08T16:34:51.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.029+0000 7fb3878145c0 10 datascan.operator(): handling object 10000000201.0 2023-11-08T16:34:51.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:41 8.28 8:177a762d:::10000000201.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a62a80 con 0x561421a46000 2023-11-08T16:34:51.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 shutdown_connections 2023-11-08T16:34:51.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fe23320c5c0 1 -- 172.21.15.81:0/2400868731 wait complete. 2023-11-08T16:34:51.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(26 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7f01574425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=15 frag=*, r=0 2023-11-08T16:34:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7f01574425c0 20 datascan.read_dentry: 15_head not found in result 2023-11-08T16:34:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7f01574425c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000204 at 0x0x10000000000/15 with size=6291456 bytes 2023-11-08T16:34:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:27 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6e00 con 0x55af34d49000 2023-11-08T16:34:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(41 10000000201.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x56142255ab40 con 0x561421a46000 2023-11-08T16:34:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7f4f6e589640 1 -- 172.21.15.81:0/1920396101 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(21 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x563180522d80 con 0x56317f9f5c00 2023-11-08T16:34:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:42 9.8 9:177a762d:::10000000201.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a63180 con 0x5614225d4800 2023-11-08T16:34:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fe23320c5c0 1 -- 172.21.15.81:0/3150314004 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303ae42c00 msgr2=0x56303aeadb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/3150314004 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303ae42c00 0x56303aeadb80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x56303bc82bd0 tx=0x56303bc38480 comp rx=0 tx=0).stop 2023-11-08T16:34:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7f4f6fe385c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-11-08T16:34:51.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-11-08T16:34:51.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7f4f6fe385c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-11-08T16:34:51.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:22 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7c00 con 0x56317f9c6800 2023-11-08T16:34:51.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fe23320c5c0 1 -- 172.21.15.81:0/3150314004 shutdown_connections 2023-11-08T16:34:51.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/3150314004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56303ae43c00 0x56303aead080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/3150314004 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56303ae43800 0x56303aead600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fe23320c5c0 1 --2- 172.21.15.81:0/3150314004 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56303ae42c00 0x56303aeadb80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.030+0000 7fe23320c5c0 1 -- 172.21.15.81:0/3150314004 >> 172.21.15.81:0/3150314004 conn(0x56303ae43000 msgr2=0x56303af34800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.031+0000 7fe23320c5c0 1 -- 172.21.15.81:0/3150314004 shutdown_connections 2023-11-08T16:34:51.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.031+0000 7fe23320c5c0 1 -- 172.21.15.81:0/3150314004 wait complete. 2023-11-08T16:34:51.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.031+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 23 ==== osd_op_reply(42 10000000201.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.031+0000 7fb3878145c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.031+0000 7fb3878145c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000201 2023-11-08T16:34:51.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.031+0000 7fb3878145c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/12 2023-11-08T16:34:51.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.031+0000 7fb3878145c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=12 2023-11-08T16:34:51.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.031+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:43 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a63500 con 0x5614225d4800 2023-11-08T16:34:51.490 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '2', '--worker_m', '7'] in time 0:00:00.701070, result: 2023-11-08T16:34:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(27 10000000000.00000000 [omap-set-vals] v64'36 uv36 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f01574425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-11-08T16:34:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55af341b4800 0x55af3418b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:28 8.25 8:a4000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7c00 con 0x55af341b4800 2023-11-08T16:34:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(22 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f4f6fe385c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f4f6fe385c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f4f6fe385c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/0, will insert to root 2023-11-08T16:34:51.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:23 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7180 con 0x56317f9c6800 2023-11-08T16:34:51.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 24 ==== osd_op_reply(43 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7fb3878145c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7fb3878145c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7fb3878145c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/12, will insert to root 2023-11-08T16:34:51.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:44 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a63880 con 0x5614225d4800 2023-11-08T16:34:51.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(23 10000000000.00000000 [omap-get-header out=274b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f4f6fe385c0 10 datascan.find_or_create_dirfrag: read_version = 36 2023-11-08T16:34:51.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f4f6fe385c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:24 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6380 con 0x56317f9c6800 2023-11-08T16:34:51.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 25 ==== osd_op_reply(44 10000000000.00000000 [omap-get-header out=274b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7fb3878145c0 10 datascan.find_or_create_dirfrag: read_version = 36 2023-11-08T16:34:51.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7fb3878145c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:45 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a63c00 con 0x5614225d4800 2023-11-08T16:34:51.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.032+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(24 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.033+0000 7f4f6fe385c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-11-08T16:34:51.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.033+0000 7f4f6fe385c0 20 datascan.read_dentry: 0_head not found in result 2023-11-08T16:34:51.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.033+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/0 with size=6291456 bytes 2023-11-08T16:34:51.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.033+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:25 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6e00 con 0x56317f9c6800 2023-11-08T16:34:51.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.033+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 26 ==== osd_op_reply(45 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.033+0000 7f0155bea640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55af341b4800 0x55af3418b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.033+0000 7fb3878145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=12 frag=*, r=0 2023-11-08T16:34:51.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.033+0000 7fb3878145c0 20 datascan.read_dentry: 12_head not found in result 2023-11-08T16:34:51.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.033+0000 7fb3878145c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000201 at 0x0x10000000000/12 with size=6291456 bytes 2023-11-08T16:34:51.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.033+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:46 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a62000 con 0x5614225d4800 2023-11-08T16:34:51.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.033+0000 7f0155bea640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55af341b4800 0x55af3418b080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.034+0000 7f0155bea640 1 -- 172.21.15.81:0/1907834121 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(28 [pgnls-filter start_epoch 64 out=49b] v62'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af34c14fc0 con 0x55af341b4800 2023-11-08T16:34:51.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.034+0000 7f01513e1640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55af341be000 0x55af3418d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.034+0000 7f01513e1640 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:29 8.15 8:a8000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af33e77c00 con 0x55af341be000 2023-11-08T16:34:51.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.034+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(25 10000000000.00000000 [omap-set-vals] v64'37 uv37 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.035+0000 7f4f6fe385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-11-08T16:34:51.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.035+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56317fb1a400 0x56317fb08b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.035+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:26 8.27 8:e4000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6000 con 0x56317fb1a400 2023-11-08T16:34:51.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.035+0000 7f01553e9640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55af341be000 0x55af3418d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.035+0000 7f4f6dd88640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56317fb1a400 0x56317fb08b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.035+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 27 ==== osd_op_reply(46 10000000000.00000000 [omap-set-vals] v64'38 uv38 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.035+0000 7fb3878145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-11-08T16:34:51.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.035+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:47 8.18 8:18000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a62380 con 0x561421a32000 2023-11-08T16:34:51.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.036+0000 7f4f6dd88640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56317fb1a400 0x56317fb08b00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.036+0000 7f01553e9640 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55af341be000 0x55af3418d700 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.036+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(26 [pgnls-filter start_epoch 64 out=81b] v64'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563180522fc0 con 0x56317fb1a400 2023-11-08T16:34:51.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.036+0000 7f4f6fe385c0 20 datascan.forall_objects: OSD matched oid 10000000208.00000000 2023-11-08T16:34:51.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.036+0000 7f4f6fe385c0 10 datascan.operator(): handling object 10000000208.0 2023-11-08T16:34:51.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.036+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:27 8.27 8:e5993619:::10000000208.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7c00 con 0x56317fb1a400 2023-11-08T16:34:51.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.036+0000 7fb3854e7640 1 -- 172.21.15.81:0/3305262764 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(47 [pgnls-filter start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56142255afc0 con 0x561421a32000 2023-11-08T16:34:51.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.036+0000 7fb3804dd640 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:48 8.38 8:1c000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a68000 con 0x5614225d4800 2023-11-08T16:34:51.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.036+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 64 out=81b] v64'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55af34c14b40 con 0x55af341be000 2023-11-08T16:34:51.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.036+0000 7f01574425c0 20 datascan.forall_objects: OSD matched oid 1000000020d.00000000 2023-11-08T16:34:51.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.036+0000 7f01574425c0 10 datascan.operator(): handling object 1000000020d.0 2023-11-08T16:34:51.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.037+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:30 8.15 8:a8e69ea8:::1000000020d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7180 con 0x55af341be000 2023-11-08T16:34:51.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.037+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(27 10000000208.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv6 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x563180522fc0 con 0x56317fb1a400 2023-11-08T16:34:51.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.037+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.3 v2:172.21.15.81:6816/2756229107 28 ==== osd_op_reply(48 [pgnls-filter start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56142255ab40 con 0x5614225d4800 2023-11-08T16:34:51.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.037+0000 7fb380cde640 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:49 8.4 8:20000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a62700 con 0x561421a19800 2023-11-08T16:34:51.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.037+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:28 9.7 9:e5993619:::10000000208.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6380 con 0x56317f9f5c00 2023-11-08T16:34:51.513 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '5', '--worker_m', '7'] in time 0:00:00.721999, result: 2023-11-08T16:34:51.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(49 [pgnls-filter start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56142255ab40 con 0x561421a19800 2023-11-08T16:34:51.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3804dd640 1 -- 172.21.15.81:0/3305262764 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:50 8.24 8:24000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x561421a68380 con 0x5614225d4400 2023-11-08T16:34:51.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6e589640 1 -- 172.21.15.81:0/1920396101 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(28 10000000208.00000000 [getxattr (30) out=30b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x563180522d80 con 0x56317f9f5c00 2023-11-08T16:34:51.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000208 2023-11-08T16:34:51.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/19 2023-11-08T16:34:51.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=19 2023-11-08T16:34:51.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:29 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7500 con 0x56317f9c6800 2023-11-08T16:34:51.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(30 1000000020d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (98) out=98b,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+144 (crc 0 0 0) 0x55af34c14b40 con 0x55af341be000 2023-11-08T16:34:51.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(29 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:31 9.15 9:a8e69ea8:::1000000020d.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af33e77880 con 0x55af341be000 2023-11-08T16:34:51.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/19, will insert to root 2023-11-08T16:34:51.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:30 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7880 con 0x56317f9c6800 2023-11-08T16:34:51.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(30 10000000000.00000000 [omap-get-header out=274b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 10 datascan.find_or_create_dirfrag: read_version = 38 2023-11-08T16:34:51.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:31 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317f6d9880 con 0x56317f9c6800 2023-11-08T16:34:51.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb385ce8640 1 -- 172.21.15.81:0/3305262764 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(50 [pgnls-filter start_epoch 64 out=49b] v62'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56142255ad80 con 0x5614225d4400 2023-11-08T16:34:51.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561421a46000 msgr2=0x561421a36680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561421a46000 0x561421a36680 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x561421a19400 msgr2=0x561421719600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x561421a19400 0x561421719600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5614225d4800 msgr2=0x56142171a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x5614225d4800 0x56142171a100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x561421a32000 msgr2=0x561421a34b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x561421a32000 0x561421a34b00 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x561421a19800 msgr2=0x561421a34000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x561421a19800 0x561421a34000 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5614225d4400 msgr2=0x561421719b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x5614225d4400 0x561421719b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.038+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x561421a19c00 msgr2=0x561421a34580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x561421a19c00 0x561421a34580 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x561421a18000 msgr2=0x56142171b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x561421a18000 0x56142171b700 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x5614224bd920 tx=0x5614224821e0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248f800 msgr2=0x56142171b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb3878145c0 1 --2- 172.21.15.81:0/3305262764 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56142248f800 0x56142171b180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x561422583950 tx=0x561422482000 comp rx=0 tx=0).stop 2023-11-08T16:34:51.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 reap_dead start 2023-11-08T16:34:51.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 reap_dead start 2023-11-08T16:34:51.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 reap_dead start 2023-11-08T16:34:51.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 shutdown_connections 2023-11-08T16:34:51.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 >> 172.21.15.81:0/3305262764 conn(0x56142248fc00 msgr2=0x561422429800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb384ce6640 1 -- 172.21.15.81:0/3305262764 reap_dead start 2023-11-08T16:34:51.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 18 ==== osd_op_reply(31 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7f4f6fe385c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=19 frag=*, r=0 2023-11-08T16:34:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7f4f6fe385c0 20 datascan.read_dentry: 19_head not found in result 2023-11-08T16:34:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000208 at 0x0x10000000000/19 with size=6291456 bytes 2023-11-08T16:34:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:32 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317f6d9c00 con 0x56317f9c6800 2023-11-08T16:34:51.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 shutdown_connections 2023-11-08T16:34:51.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(31 1000000020d.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x55af34c14b40 con 0x55af341be000 2023-11-08T16:34:51.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7f01574425c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7f01574425c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020d 2023-11-08T16:34:51.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7f01574425c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/24 2023-11-08T16:34:51.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7f01574425c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=24 2023-11-08T16:34:51.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:32 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6a80 con 0x55af34d49000 2023-11-08T16:34:51.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.039+0000 7fb3878145c0 1 -- 172.21.15.81:0/3305262764 wait complete. 2023-11-08T16:34:51.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7fb3878145c0 1 -- 172.21.15.81:0/4118159804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614216aec00 msgr2=0x561421718580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7fb3878145c0 1 --2- 172.21.15.81:0/4118159804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614216aec00 0x561421718580 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x5614224bca80 tx=0x5614224822a0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7fb3878145c0 1 -- 172.21.15.81:0/4118159804 shutdown_connections 2023-11-08T16:34:51.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7fb3878145c0 1 --2- 172.21.15.81:0/4118159804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5614216afc00 0x561421719080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7fb3878145c0 1 --2- 172.21.15.81:0/4118159804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5614216af800 0x561421718b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7fb3878145c0 1 --2- 172.21.15.81:0/4118159804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5614216aec00 0x561421718580 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7fb3878145c0 1 -- 172.21.15.81:0/4118159804 >> 172.21.15.81:0/4118159804 conn(0x5614216af000 msgr2=0x561422429000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7fb3878145c0 1 -- 172.21.15.81:0/4118159804 shutdown_connections 2023-11-08T16:34:51.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7fb3878145c0 1 -- 172.21.15.81:0/4118159804 wait complete. 2023-11-08T16:34:51.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 19 ==== osd_op_reply(32 10000000000.00000000 [omap-set-vals] v64'39 uv39 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7f4f6fe385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-11-08T16:34:51.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56317fb1ac00 0x56317fb09080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.040+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:33 8.17 8:e8000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6a80 con 0x56317fb1ac00 2023-11-08T16:34:51.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(32 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01574425c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01574425c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01574425c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/24, will insert to root 2023-11-08T16:34:51.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:33 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6000 con 0x55af34d49000 2023-11-08T16:34:51.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 18 ==== osd_op_reply(33 10000000000.00000000 [omap-get-header out=274b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01574425c0 10 datascan.find_or_create_dirfrag: read_version = 39 2023-11-08T16:34:51.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01574425c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:34 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6700 con 0x55af34d49000 2023-11-08T16:34:51.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f4f6d587640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56317fb1ac00 0x56317fb09080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 19 ==== osd_op_reply(34 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01574425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=24 frag=*, r=0 2023-11-08T16:34:51.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01574425c0 20 datascan.read_dentry: 24_head not found in result 2023-11-08T16:34:51.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01574425c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020d at 0x0x10000000000/24 with size=6291456 bytes 2023-11-08T16:34:51.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.041+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:35 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a6380 con 0x55af34d49000 2023-11-08T16:34:51.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.042+0000 7f4f6d587640 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56317fb1ac00 0x56317fb09080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.043+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(33 [pgnls-filter start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563180522b40 con 0x56317fb1ac00 2023-11-08T16:34:51.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.043+0000 7f4f68d7e640 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:34 8.37 8:ec000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317fb30000 con 0x56317f9c6800 2023-11-08T16:34:51.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.043+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.3 v2:172.21.15.81:6816/2756229107 20 ==== osd_op_reply(35 10000000000.00000000 [omap-set-vals] v64'40 uv40 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55af34c14b40 con 0x55af34d49000 2023-11-08T16:34:51.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.043+0000 7f01574425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-11-08T16:34:51.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.043+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:36 8.35 8:ac000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341a7500 con 0x55af34d63c00 2023-11-08T16:34:51.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.043+0000 7f0155bea640 1 -- 172.21.15.81:0/1907834121 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(36 [pgnls-filter start_epoch 64 out=49b] v62'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af34c14fc0 con 0x55af34d63c00 2023-11-08T16:34:51.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.043+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 20 ==== osd_op_reply(34 [pgnls-filter start_epoch 64 out=49b] v62'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.043+0000 7f4f6957f640 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:35 8.f 8:f0000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317f6d9880 con 0x56317f9c6800 2023-11-08T16:34:51.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.043+0000 7f01513e1640 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:37 8.d 8:b0000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341d8000 con 0x55af341b4000 2023-11-08T16:34:51.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.044+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 21 ==== osd_op_reply(35 [pgnls-filter start_epoch 64 out=49b] v62'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.044+0000 7f4f68d7e640 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:36 8.2f 8:f4000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6e00 con 0x56317fb1a400 2023-11-08T16:34:51.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.044+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(36 [pgnls-filter start_epoch 64 out=49b] v62'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563180522fc0 con 0x56317fb1a400 2023-11-08T16:34:51.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.044+0000 7f4f6957f640 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:37 8.1f 8:f8000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317f6d9c00 con 0x56317fb1a400 2023-11-08T16:34:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.044+0000 7f01553e9640 1 -- 172.21.15.81:0/1907834121 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(37 [pgnls-filter start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af34c14b40 con 0x55af341b4000 2023-11-08T16:34:51.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.044+0000 7f0150be0640 1 -- 172.21.15.81:0/1907834121 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:38 8.2d 8:b4000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55af341d8a80 con 0x55af34d48000 2023-11-08T16:34:51.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.044+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(37 [pgnls-filter start_epoch 64 out=49b] v62'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x563180522fc0 con 0x56317fb1a400 2023-11-08T16:34:51.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.044+0000 7f4f68d7e640 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:38 8.3f 8:fc000000::::head [pgnls-filter start_epoch 64 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7500 con 0x56317f9f5400 2023-11-08T16:34:51.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01563eb640 1 -- 172.21.15.81:0/1907834121 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(38 [pgnls-filter start_epoch 64 out=49b] v62'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55af34c14d80 con 0x55af34d48000 2023-11-08T16:34:51.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55af34d48000 msgr2=0x55af33e8a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55af34d48000 0x55af33e8a100 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55af341b4800 msgr2=0x55af3418b080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55af341b4800 0x55af3418b080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55af34d63c00 msgr2=0x55af3418a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55af34d63c00 0x55af3418a580 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55af34d49000 msgr2=0x55af33e8a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55af34d49000 0x55af33e8a680 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55af341b4000 msgr2=0x55af3418ab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55af341b4000 0x55af3418ab00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55af34d48c00 msgr2=0x55af33e89080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55af34d48c00 0x55af33e89080 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55af341be000 msgr2=0x55af3418d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55af341be000 0x55af3418d700 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f0155bea640 1 -- 172.21.15.81:0/1907834121 reap_dead start 2023-11-08T16:34:51.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f0155bea640 1 -- 172.21.15.81:0/1907834121 reap_dead start 2023-11-08T16:34:51.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 64 out=81b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x563180522b40 con 0x56317f9f5400 2023-11-08T16:34:51.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55af34d62c00 msgr2=0x55af3418a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55af34d62c00 0x55af3418a000 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x55af34b9c900 tx=0x55af34d07ce0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0000 msgr2=0x55af33e8b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0000 0x55af33e8b180 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x55af34d05350 tx=0x55af34bee5a0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f4f6fe385c0 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-11-08T16:34:51.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f4f6fe385c0 10 datascan.operator(): handling object 100000001fa.0 2023-11-08T16:34:51.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:39 8.3f 8:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6700 con 0x56317f9f5400 2023-11-08T16:34:51.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 shutdown_connections 2023-11-08T16:34:51.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55af34d62c00 0x55af3418a000 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 --2- 172.21.15.81:0/1907834121 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0000 0x55af33e8b180 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 >> 172.21.15.81:0/1907834121 conn(0x55af34bf0400 msgr2=0x55af34b9a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.554 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '0', '--worker_m', '7'] in time 0:00:00.766596, result: 2023-11-08T16:34:51.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.045+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 shutdown_connections 2023-11-08T16:34:51.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(39 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (97) out=97b,getxattr,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+143 (crc 0 0 0) 0x563180522b40 con 0x56317f9f5400 2023-11-08T16:34:51.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:40 9.1f 9:feb1c803:::100000001fa.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7180 con 0x56317f9f5400 2023-11-08T16:34:51.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(40 100000001fa.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x563180522b40 con 0x56317f9f5400 2023-11-08T16:34:51.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6fe385c0 20 datascan.operator(): loaded layout from xattr: pi: 9 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:34:51.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f01574425c0 1 -- 172.21.15.81:0/1907834121 wait complete. 2023-11-08T16:34:51.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-11-08T16:34:51.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/5 2023-11-08T16:34:51.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6fe385c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=5 2023-11-08T16:34:51.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:41 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7c00 con 0x56317f9c6800 2023-11-08T16:34:51.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 22 ==== osd_op_reply(41 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6fe385c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:34:51.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6fe385c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:34:51.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6fe385c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/5, will insert to root 2023-11-08T16:34:51.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:42 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6e00 con 0x56317f9c6800 2023-11-08T16:34:51.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f01574425c0 1 -- 172.21.15.81:0/3781527100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af34bf1c00 msgr2=0x55af33e89600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.046+0000 7f01574425c0 1 --2- 172.21.15.81:0/3781527100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af34bf1c00 0x55af33e89600 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x55af34b9cb70 tx=0x55af34bee240 comp rx=0 tx=0).stop 2023-11-08T16:34:51.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f01574425c0 1 -- 172.21.15.81:0/3781527100 shutdown_connections 2023-11-08T16:34:51.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f01574425c0 1 --2- 172.21.15.81:0/3781527100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55af34bf1c00 0x55af33e89600 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f01574425c0 1 --2- 172.21.15.81:0/3781527100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55af33e1fc00 0x55af33e88b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f01574425c0 1 --2- 172.21.15.81:0/3781527100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55af34bf0800 0x55af33e89b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f01574425c0 1 -- 172.21.15.81:0/3781527100 >> 172.21.15.81:0/3781527100 conn(0x55af33e1f000 msgr2=0x55af34b9b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 23 ==== osd_op_reply(42 10000000000.00000000 [omap-get-header out=274b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f4f6fe385c0 10 datascan.find_or_create_dirfrag: read_version = 40 2023-11-08T16:34:51.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f4f6fe385c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:34:51.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:43 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf7880 con 0x56317f9c6800 2023-11-08T16:34:51.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f01574425c0 1 -- 172.21.15.81:0/3781527100 shutdown_connections 2023-11-08T16:34:51.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f01574425c0 1 -- 172.21.15.81:0/3781527100 wait complete. 2023-11-08T16:34:51.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 24 ==== osd_op_reply(43 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f4f6fe385c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=5 frag=*, r=0 2023-11-08T16:34:51.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f4f6fe385c0 20 datascan.read_dentry: 5_head not found in result 2023-11-08T16:34:51.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f4f6fe385c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fa at 0x0x10000000000/5 with size=6291456 bytes 2023-11-08T16:34:51.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.047+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:44 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x56317faf6a80 con 0x56317f9c6800 2023-11-08T16:34:51.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6d587640 1 -- 172.21.15.81:0/1920396101 <== osd.3 v2:172.21.15.81:6816/2756229107 25 ==== osd_op_reply(44 10000000000.00000000 [omap-set-vals] v64'41 uv41 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x563180522b40 con 0x56317f9c6800 2023-11-08T16:34:51.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-11-08T16:34:51.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56317fb1a400 msgr2=0x56317fb08b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56317fb1a400 0x56317fb08b00 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56317f9f5400 msgr2=0x56317fb08000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56317f9f5400 0x56317fb08000 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56317f9c6800 msgr2=0x56317f6eb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x56317f9c6800 0x56317f6eb600 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56317f9c6000 msgr2=0x56317f6ebb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56317f9c6000 0x56317f6ebb80 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56317f9f5c00 msgr2=0x56317fb08580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56317f9f5c00 0x56317fb08580 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56317f9c6400 msgr2=0x56317f6ec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56317f9c6400 0x56317f6ec100 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56317fb1ac00 msgr2=0x56317fb09080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56317fb1ac00 0x56317fb09080 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 reap_dead start 2023-11-08T16:34:51.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 reap_dead start 2023-11-08T16:34:51.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6dd88640 1 -- 172.21.15.81:0/1920396101 reap_dead start 2023-11-08T16:34:51.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56317f9f4c00 msgr2=0x56317f6ed700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56317f9f4c00 0x56317f6ed700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x56317f9768a0 tx=0x56317f979080 comp rx=0 tx=0).stop 2023-11-08T16:34:51.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c7000 msgr2=0x56317f6ecc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c7000 0x56317f6ecc00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x56317f977440 tx=0x56317f978d20 comp rx=0 tx=0).stop 2023-11-08T16:34:51.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 shutdown_connections 2023-11-08T16:34:51.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56317f9f4c00 0x56317f6ed700 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1920396101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f9c7000 0x56317f6ecc00 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 >> 172.21.15.81:0/1920396101 conn(0x563180443c00 msgr2=0x5631803fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 shutdown_connections 2023-11-08T16:34:51.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.049+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1920396101 wait complete. 2023-11-08T16:34:51.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.050+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1467809784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56317f681c00 msgr2=0x56317f6eb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.050+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1467809784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56317f681c00 0x56317f6eb080 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x5631804829c0 tx=0x563180446ba0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.050+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1467809784 shutdown_connections 2023-11-08T16:34:51.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.050+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1467809784 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56317f681c00 0x56317f6eb080 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.050+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1467809784 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56317f681800 0x56317f6eab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.050+0000 7f4f6fe385c0 1 --2- 172.21.15.81:0/1467809784 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56317f680c00 0x56317f6ea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.050+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1467809784 >> 172.21.15.81:0/1467809784 conn(0x56317f681000 msgr2=0x5631803fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.050+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1467809784 shutdown_connections 2023-11-08T16:34:51.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.050+0000 7f4f6fe385c0 1 -- 172.21.15.81:0/1467809784 wait complete. 2023-11-08T16:34:51.575 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '4', '--worker_m', '7'] in time 0:00:00.785097, result: 2023-11-08T16:34:51.576 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--worker_n', '6', '--worker_m', '7'] in time 0:00:00.784849, result: 2023-11-08T16:34:51.576 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-11-08T16:34:51.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.684+0000 7fdb15a425c0 1 -- start start 2023-11-08T16:34:51.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.684+0000 7fdb15a425c0 1 Processor -- start 2023-11-08T16:34:51.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.685+0000 7fdb15a425c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb87e6c00 0x55cfb8850580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.685+0000 7fdb15a425c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb87e7800 0x55cfb8850b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.685+0000 7fdb15a425c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb87e7c00 0x55cfb8851080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.685+0000 7fdb15a425c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb882f180 con 0x55cfb87e7c00 2023-11-08T16:34:51.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.685+0000 7fdb15a425c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb882e140 con 0x55cfb87e7800 2023-11-08T16:34:51.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.685+0000 7fdb15a425c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfb882e000 con 0x55cfb87e6c00 2023-11-08T16:34:51.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb149eb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb87e7c00 0x55cfb8851080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb149eb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb87e7c00 0x55cfb8851080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55030/0 (socket says 172.21.15.81:55030) 2023-11-08T16:34:51.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb149eb640 1 -- 172.21.15.81:0/2728872468 learned_addr learned my addr 172.21.15.81:0/2728872468 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb141ea640 1 --2- 172.21.15.81:0/2728872468 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb87e6c00 0x55cfb8850580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb139e9640 1 --2- 172.21.15.81:0/2728872468 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb87e7800 0x55cfb8850b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb149eb640 1 -- 172.21.15.81:0/2728872468 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb87e6c00 msgr2=0x55cfb8850580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb149eb640 1 --2- 172.21.15.81:0/2728872468 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb87e6c00 0x55cfb8850580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb149eb640 1 -- 172.21.15.81:0/2728872468 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb87e7800 msgr2=0x55cfb8850b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb149eb640 1 --2- 172.21.15.81:0/2728872468 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb87e7800 0x55cfb8850b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb149eb640 1 -- 172.21.15.81:0/2728872468 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfb880c9c0 con 0x55cfb87e7c00 2023-11-08T16:34:51.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb139e9640 1 --2- 172.21.15.81:0/2728872468 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb87e7800 0x55cfb8850b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:51.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.686+0000 7fdb149eb640 1 --2- 172.21.15.81:0/2728872468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb87e7c00 0x55cfb8851080 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x55cfb95e6a50 tx=0x55cfb8ac6120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c11f5530dbc2a80 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.687+0000 7fdb119e5640 1 -- 172.21.15.81:0/2728872468 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cfb882cf20 con 0x55cfb87e7c00 2023-11-08T16:34:51.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.687+0000 7fdb119e5640 1 -- 172.21.15.81:0/2728872468 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cfb88c6300 con 0x55cfb87e7c00 2023-11-08T16:34:51.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.687+0000 7fdb119e5640 1 -- 172.21.15.81:0/2728872468 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cfb882d1e0 con 0x55cfb87e7c00 2023-11-08T16:34:51.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.687+0000 7fdb15a425c0 1 -- 172.21.15.81:0/2728872468 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfb880cd00 con 0x55cfb87e7c00 2023-11-08T16:34:51.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.687+0000 7fdb15a425c0 1 -- 172.21.15.81:0/2728872468 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55cfb880c9c0 con 0x55cfb87e7c00 2023-11-08T16:34:51.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.687+0000 7fdb15a425c0 4 waiting for MDS map... 2023-11-08T16:34:51.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.687+0000 7fdb119e5640 1 -- 172.21.15.81:0/2728872468 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55cfb88a6c40 con 0x55cfb87e7c00 2023-11-08T16:34:51.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.687+0000 7fdb119e5640 1 -- 172.21.15.81:0/2728872468 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 47) v1 ==== 3306+0+0 (secure 0 0 0) 0x55cfb8836f00 con 0x55cfb87e7c00 2023-11-08T16:34:51.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.687+0000 7fdb15a425c0 4 Got MDS map 47 2023-11-08T16:34:51.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.687+0000 7fdb15a425c0 4 datascan.main: Using metadata pool output 2023-11-08T16:34:51.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.687+0000 7fdb15a425c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:34:51.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.688+0000 7fdb15a425c0 1 Processor -- start 2023-11-08T16:34:51.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.688+0000 7fdb15a425c0 1 -- start start 2023-11-08T16:34:51.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.689+0000 7fdb15a425c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb95b3c00 0x55cfb8851600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.689+0000 7fdb15a425c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3a000 0x55cfb8851b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.689+0000 7fdb15a425c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb8b3a400 0x55cfb8852100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.689+0000 7fdb15a425c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb882ea00 con 0x55cfb8b3a000 2023-11-08T16:34:51.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.689+0000 7fdb15a425c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb88b5180 con 0x55cfb95b3c00 2023-11-08T16:34:51.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.689+0000 7fdb15a425c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfb88b4dc0 con 0x55cfb8b3a400 2023-11-08T16:34:51.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.689+0000 7fdb141ea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3a000 0x55cfb8851b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.689+0000 7fdb141ea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3a000 0x55cfb8851b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55034/0 (socket says 172.21.15.81:55034) 2023-11-08T16:34:51.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.689+0000 7fdb141ea640 1 -- 172.21.15.81:0/216116226 learned_addr learned my addr 172.21.15.81:0/216116226 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.690+0000 7fdb139e9640 1 --2- 172.21.15.81:0/216116226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb95b3c00 0x55cfb8851600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.690+0000 7fdb139e9640 1 --2- 172.21.15.81:0/216116226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb8b3a400 0x55cfb8852100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.690+0000 7fdb141ea640 1 -- 172.21.15.81:0/216116226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb8b3a400 msgr2=0x55cfb8852100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.690+0000 7fdb141ea640 1 --2- 172.21.15.81:0/216116226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb8b3a400 0x55cfb8852100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.690+0000 7fdb141ea640 1 -- 172.21.15.81:0/216116226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb95b3c00 msgr2=0x55cfb8851600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:51.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.690+0000 7fdb141ea640 1 --2- 172.21.15.81:0/216116226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb95b3c00 0x55cfb8851600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.690+0000 7fdb141ea640 1 -- 172.21.15.81:0/216116226 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfb880cb60 con 0x55cfb8b3a000 2023-11-08T16:34:51.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.690+0000 7fdb141ea640 1 --2- 172.21.15.81:0/216116226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3a000 0x55cfb8851b80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55cfb8ade960 tx=0x55cfb8aeacc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634775b043e2c2f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.690+0000 7fdb0f9e1640 1 -- 172.21.15.81:0/216116226 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cfb882cf20 con 0x55cfb8b3a000 2023-11-08T16:34:51.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.690+0000 7fdb0f9e1640 1 -- 172.21.15.81:0/216116226 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cfb88c6a80 con 0x55cfb8b3a000 2023-11-08T16:34:51.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.690+0000 7fdb0f9e1640 1 -- 172.21.15.81:0/216116226 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cfb882d1e0 con 0x55cfb8b3a000 2023-11-08T16:34:51.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.691+0000 7fdb15a425c0 1 -- 172.21.15.81:0/216116226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3a000 msgr2=0x55cfb8851b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.691+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/216116226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3a000 0x55cfb8851b80 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x55cfb8ade960 tx=0x55cfb8aeacc0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.691+0000 7fdb15a425c0 1 -- 172.21.15.81:0/216116226 shutdown_connections 2023-11-08T16:34:51.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.691+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/216116226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb8b3a400 0x55cfb8852100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.691+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/216116226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3a000 0x55cfb8851b80 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.691+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/216116226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb95b3c00 0x55cfb8851600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.691+0000 7fdb15a425c0 1 -- 172.21.15.81:0/216116226 >> 172.21.15.81:0/216116226 conn(0x55cfb95b3800 msgr2=0x55cfb9571000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.691+0000 7fdb15a425c0 1 -- 172.21.15.81:0/216116226 shutdown_connections 2023-11-08T16:34:51.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.691+0000 7fdb15a425c0 1 -- 172.21.15.81:0/216116226 wait complete. 2023-11-08T16:34:51.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.691+0000 7fdb15a425c0 1 Processor -- start 2023-11-08T16:34:51.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.692+0000 7fdb15a425c0 1 -- start start 2023-11-08T16:34:51.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.692+0000 7fdb15a425c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb95b3800 0x55cfb8853180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.692+0000 7fdb15a425c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb8b3b000 0x55cfb8852c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.692+0000 7fdb15a425c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3ac00 0x55cfb8852680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.692+0000 7fdb15a425c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb882e140 con 0x55cfb8b3ac00 2023-11-08T16:34:51.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.692+0000 7fdb15a425c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfb882f180 con 0x55cfb95b3800 2023-11-08T16:34:51.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.692+0000 7fdb15a425c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfb882eb40 con 0x55cfb8b3b000 2023-11-08T16:34:51.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.693+0000 7fdb139e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3ac00 0x55cfb8852680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.693+0000 7fdb139e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3ac00 0x55cfb8852680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55038/0 (socket says 172.21.15.81:55038) 2023-11-08T16:34:51.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.693+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 learned_addr learned my addr 172.21.15.81:0/1670434595 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:51.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.693+0000 7fdb139e9640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb95b3800 0x55cfb8853180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.693+0000 7fdb141ea640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb8b3b000 0x55cfb8852c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.693+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb8b3b000 msgr2=0x55cfb8852c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.693+0000 7fdb139e9640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb8b3b000 0x55cfb8852c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.693+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb95b3800 msgr2=0x55cfb8853180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.693+0000 7fdb139e9640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb95b3800 0x55cfb8853180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.693+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfb880c820 con 0x55cfb8b3ac00 2023-11-08T16:34:51.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.693+0000 7fdb139e9640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3ac00 0x55cfb8852680 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x55cfb8adf5c0 tx=0x55cfb8aeae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.694+0000 7fdb0e1de640 1 -- 172.21.15.81:0/1670434595 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cfb882cf20 con 0x55cfb8b3ac00 2023-11-08T16:34:51.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.694+0000 7fdb0e1de640 1 -- 172.21.15.81:0/1670434595 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cfb8840180 con 0x55cfb8b3ac00 2023-11-08T16:34:51.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.694+0000 7fdb0e1de640 1 -- 172.21.15.81:0/1670434595 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cfb882d1e0 con 0x55cfb8b3ac00 2023-11-08T16:34:51.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.694+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cfb880c4e0 con 0x55cfb8b3ac00 2023-11-08T16:34:51.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.694+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfb880c1a0 con 0x55cfb8b3ac00 2023-11-08T16:34:51.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.694+0000 7fdb15a425c0 4 datascan.init: resolving metadata pool 7 2023-11-08T16:34:51.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.695+0000 7fdb0e1de640 1 -- 172.21.15.81:0/1670434595 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55cfb8842840 con 0x55cfb8b3ac00 2023-11-08T16:34:51.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.696+0000 7fdb0e1de640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cfb8b5cc00 0x55cfb8853700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.696+0000 7fdb0e1de640 1 -- 172.21.15.81:0/1670434595 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x55cfb88a6a80 con 0x55cfb8b3ac00 2023-11-08T16:34:51.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.696+0000 7fdb15a425c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:34:51.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.696+0000 7fdb15a425c0 4 datascan.main: resolving metadata pool 7 2023-11-08T16:34:51.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.696+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55cfb8b3a800 0x55cfb8851600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.696+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:1 7.0 7:00000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3a800 2023-11-08T16:34:51.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.696+0000 7fdb141ea640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cfb8b5cc00 0x55cfb8853700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.697+0000 7fdb141ea640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cfb8b5cc00 0x55cfb8853700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55cfb8ade960 tx=0x55cfb9576060 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.697+0000 7fdb139e9640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55cfb8b3a800 0x55cfb8851600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.698+0000 7fdb139e9640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55cfb8b3a800 0x55cfb8851600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.699+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(1 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3a800 2023-11-08T16:34:51.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.699+0000 7fdb139e9640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55cfb8b5d000 0x55cfb8c4c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.699+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:2 7.10 7:08000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.699+0000 7fdb149eb640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55cfb8b5d000 0x55cfb8c4c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.699+0000 7fdb149eb640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55cfb8b5d000 0x55cfb8c4c000 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.700+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(2 [pgnls start_epoch 64 out=81b] v64'41 uv41 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.700+0000 7fdb149eb640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55cfb95b3400 0x55cfb8c4c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.700+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:3 7.8 7:10000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb95b3400 2023-11-08T16:34:51.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.700+0000 7fdb141ea640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55cfb95b3400 0x55cfb8c4c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.701+0000 7fdb141ea640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55cfb95b3400 0x55cfb8c4c580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.701+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(3 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688fc0 con 0x55cfb95b3400 2023-11-08T16:34:51.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.701+0000 7fdb141ea640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55cfb8b3bc00 0x55cfb8c4cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.701+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:4 7.18 7:18000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.701+0000 7fdb139e9640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55cfb8b3bc00 0x55cfb8c4cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.702+0000 7fdb139e9640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55cfb8b3bc00 0x55cfb8c4cb00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.702+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(4 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.702+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 7.4 7:20000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.702+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(5 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.702+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:6 7.14 7:28000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.703+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(6 [pgnls start_epoch 64 out=125b] v64'104 uv104 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.703+0000 7fdb149eb640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55cfb8b5dc00 0x55cfb8c4d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.703+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:7 7.c 7:30000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.703+0000 7fdb149eb640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55cfb8b5dc00 0x55cfb8c4d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.704+0000 7fdb149eb640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55cfb8b5dc00 0x55cfb8c4d080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.705+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(7 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.705+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:8 7.1c 7:38000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3a800 2023-11-08T16:34:51.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.706+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(8 [pgnls start_epoch 64 out=49b] v63'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3a800 2023-11-08T16:34:51.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.706+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 7.2 7:40000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.706+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(9 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.706+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:10 7.12 7:48000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.707+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(10 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.707+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:11 7.a 7:50000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.707+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(11 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.707+0000 7fdb149eb640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55cfb8bd0000 0x55cfb8c4d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.707+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:12 7.1a 7:58000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd0000 2023-11-08T16:34:51.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.708+0000 7fdb141ea640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55cfb8bd0000 0x55cfb8c4d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.708+0000 7fdb141ea640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55cfb8bd0000 0x55cfb8c4d600 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.709+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(12 [pgnls start_epoch 64 out=49b] v64'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688fc0 con 0x55cfb8bd0000 2023-11-08T16:34:51.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.709+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:13 7.6 7:60000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.710+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(13 [pgnls start_epoch 64 out=49b] v63'25 uv25 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.710+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:14 7.16 7:68000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd0000 2023-11-08T16:34:51.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.710+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(14 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688fc0 con 0x55cfb8bd0000 2023-11-08T16:34:51.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.710+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:15 7.e 7:70000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb95b3400 2023-11-08T16:34:51.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.711+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(15 [pgnls start_epoch 64 out=49b] v63'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688fc0 con 0x55cfb95b3400 2023-11-08T16:34:51.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.711+0000 7fdb141ea640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55cfb8bd1400 0x55cfb8c4db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.711+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:16 7.1e 7:78000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1400 2023-11-08T16:34:51.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.711+0000 7fdb139e9640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55cfb8bd1400 0x55cfb8c4db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.712+0000 7fdb139e9640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55cfb8bd1400 0x55cfb8c4db80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.712+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(16 [pgnls start_epoch 64 out=49b] v64'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8bd1400 2023-11-08T16:34:51.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.712+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:17 7.1 7:80000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd0000 2023-11-08T16:34:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.713+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(17 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688fc0 con 0x55cfb8bd0000 2023-11-08T16:34:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.713+0000 7fdb141ea640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55cfb8bd1800 0x55cfb8c4e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.713+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:18 7.11 7:88000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1800 2023-11-08T16:34:51.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.713+0000 7fdb149eb640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55cfb8bd1800 0x55cfb8c4e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:51.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.715+0000 7fdb149eb640 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55cfb8bd1800 0x55cfb8c4e100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:51.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.715+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(18 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8bd1800 2023-11-08T16:34:51.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.715+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:19 7.9 7:90000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.716+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(19 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.716+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:20 7.19 7:98000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.717+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(20 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.717+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:21 7.5 7:a0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1800 2023-11-08T16:34:51.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.718+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(21 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8bd1800 2023-11-08T16:34:51.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.718+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:22 7.15 7:a8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1800 2023-11-08T16:34:51.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.719+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(22 [pgnls start_epoch 64 out=73b] v63'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8bd1800 2023-11-08T16:34:51.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.719+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:23 7.d 7:b0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.720+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(23 [pgnls start_epoch 64 out=74b] v64'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.720+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:24 7.1d 7:b8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1400 2023-11-08T16:34:51.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.720+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(24 [pgnls start_epoch 64 out=49b] v63'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8bd1400 2023-11-08T16:34:51.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.720+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:25 7.3 7:c0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.721+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(25 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.721+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:26 7.13 7:c8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.721+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(26 [pgnls start_epoch 64 out=98b] v64'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.721+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:27 7.b 7:d0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1400 2023-11-08T16:34:51.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.722+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(27 [pgnls start_epoch 64 out=76b] v64'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8bd1400 2023-11-08T16:34:51.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.722+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:28 7.1b 7:d8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.722+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(28 [pgnls start_epoch 64 out=49b] v64'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.722+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:29 7.7 7:e0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.723+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(29 [pgnls start_epoch 64 out=49b] v63'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.723+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:30 7.17 7:e8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1800 2023-11-08T16:34:51.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.724+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(30 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8bd1800 2023-11-08T16:34:51.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.724+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:31 7.f 7:f0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.724+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(31 [pgnls start_epoch 64 out=73b] v64'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.724+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:32 7.1f 7:f8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.725+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(32 [pgnls start_epoch 64 out=101b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.725+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-11-08T16:34:51.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.725+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:33 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883fc00 con 0x55cfb8b5d000 2023-11-08T16:34:51.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=12470b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+12470 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-11-08T16:34:51.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-08T16:34:51.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-11-08T16:34:51.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-08T16:34:51.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-11-08T16:34:51.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-08T16:34:51.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-11-08T16:34:51.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-08T16:34:51.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-11-08T16:34:51.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-08T16:34:51.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-11-08T16:34:51.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-08T16:34:51.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-11-08T16:34:51.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.726+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:34 7.13 7:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78000 con 0x55cfb8b3bc00 2023-11-08T16:34:51.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.727+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.727+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-11-08T16:34:51.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.727+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-08T16:34:51.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.727+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-11-08T16:34:51.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.727+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-08T16:34:51.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.727+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-11-08T16:34:51.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.727+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-08T16:34:51.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.727+0000 7fdb15a425c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-11-08T16:34:51.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.727+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:35 7.1f 7:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78380 con 0x55cfb8b5dc00 2023-11-08T16:34:51.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.728+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.728+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.728+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:36 7.0 7:00000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78700 con 0x55cfb8b3a800 2023-11-08T16:34:51.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.729+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(36 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3a800 2023-11-08T16:34:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.729+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:37 7.10 7:08000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.730+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(37 [pgnls start_epoch 64 out=81b] v64'41 uv41 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.730+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:38 7.8 7:10000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb95b3400 2023-11-08T16:34:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.730+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(38 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688fc0 con 0x55cfb95b3400 2023-11-08T16:34:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.730+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:39 7.18 7:18000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.730+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(39 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.731+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:40 7.4 7:20000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.731+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(40 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.731+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:41 7.14 7:28000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.731+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(41 [pgnls start_epoch 64 out=125b] v64'104 uv104 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.731+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:42 7.c 7:30000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.732+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(42 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.732+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:43 7.1c 7:38000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3a800 2023-11-08T16:34:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.733+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(43 [pgnls start_epoch 64 out=49b] v63'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3a800 2023-11-08T16:34:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.734+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:44 7.2 7:40000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.734+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(44 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.734+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:45 7.12 7:48000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.735+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(45 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.735+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:46 7.a 7:50000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.735+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(46 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.735+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:47 7.1a 7:58000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd0000 2023-11-08T16:34:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.736+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(47 [pgnls start_epoch 64 out=49b] v64'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688fc0 con 0x55cfb8bd0000 2023-11-08T16:34:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.736+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:48 7.6 7:60000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.737+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 11 ==== osd_op_reply(48 [pgnls start_epoch 64 out=49b] v63'25 uv25 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.737+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:49 7.16 7:68000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd0000 2023-11-08T16:34:51.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.738+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(49 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688fc0 con 0x55cfb8bd0000 2023-11-08T16:34:51.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.738+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:50 7.e 7:70000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb95b3400 2023-11-08T16:34:51.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.738+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(50 [pgnls start_epoch 64 out=49b] v63'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688fc0 con 0x55cfb95b3400 2023-11-08T16:34:51.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.738+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:51 7.1e 7:78000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1400 2023-11-08T16:34:51.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.739+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(51 [pgnls start_epoch 64 out=49b] v64'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8bd1400 2023-11-08T16:34:51.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.739+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:52 7.1 7:80000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd0000 2023-11-08T16:34:51.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.740+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(52 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688fc0 con 0x55cfb8bd0000 2023-11-08T16:34:51.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.740+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:53 7.11 7:88000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1800 2023-11-08T16:34:51.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.741+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(53 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8bd1800 2023-11-08T16:34:51.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.741+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:54 7.9 7:90000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.741+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 12 ==== osd_op_reply(54 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.741+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:55 7.19 7:98000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.742+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(55 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.742+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:56 7.5 7:a0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1800 2023-11-08T16:34:51.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.743+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(56 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8bd1800 2023-11-08T16:34:51.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.743+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:57 7.15 7:a8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1800 2023-11-08T16:34:51.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.745+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(57 [pgnls start_epoch 64 out=73b] v63'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8bd1800 2023-11-08T16:34:51.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.745+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:58 7.d 7:b0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.746+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 11 ==== osd_op_reply(58 [pgnls start_epoch 64 out=74b] v64'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.746+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:59 7.1d 7:b8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1400 2023-11-08T16:34:51.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.747+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(59 [pgnls start_epoch 64 out=49b] v63'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8bd1400 2023-11-08T16:34:51.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.747+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:60 7.3 7:c0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.747+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(60 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.747+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:61 7.13 7:c8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.748+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 13 ==== osd_op_reply(61 [pgnls start_epoch 64 out=98b] v64'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.748+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:62 7.b 7:d0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1400 2023-11-08T16:34:51.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.748+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(62 [pgnls start_epoch 64 out=76b] v64'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8bd1400 2023-11-08T16:34:51.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.748+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:63 7.1b 7:d8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.749+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 14 ==== osd_op_reply(63 [pgnls start_epoch 64 out=49b] v64'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.749+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:64 7.7 7:e0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.750+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 12 ==== osd_op_reply(64 [pgnls start_epoch 64 out=49b] v63'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.750+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:65 7.17 7:e8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8bd1800 2023-11-08T16:34:51.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.751+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(65 [pgnls start_epoch 64 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8bd1800 2023-11-08T16:34:51.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.751+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:66 7.f 7:f0000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.752+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 15 ==== osd_op_reply(66 [pgnls start_epoch 64 out=73b] v64'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.752+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:67 7.1f 7:f8000000::::head [pgnls start_epoch 64 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.753+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 13 ==== osd_op_reply(67 [pgnls start_epoch 64 out=101b] v64'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.753+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-11-08T16:34:51.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.753+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:68 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78a80 con 0x55cfb8b5d000 2023-11-08T16:34:51.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=12470b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+12470 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-11-08T16:34:51.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x100000001ff for future processing to fix dnfirst 2023-11-08T16:34:51.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000200 for future processing to fix dnfirst 2023-11-08T16:34:51.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000201 for future processing to fix dnfirst 2023-11-08T16:34:51.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000202 for future processing to fix dnfirst 2023-11-08T16:34:51.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000203 for future processing to fix dnfirst 2023-11-08T16:34:51.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000204 for future processing to fix dnfirst 2023-11-08T16:34:51.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000205 for future processing to fix dnfirst 2023-11-08T16:34:51.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000206 for future processing to fix dnfirst 2023-11-08T16:34:51.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000207 for future processing to fix dnfirst 2023-11-08T16:34:51.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000208 for future processing to fix dnfirst 2023-11-08T16:34:51.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-11-08T16:34:51.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000209 for future processing to fix dnfirst 2023-11-08T16:34:51.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x1000000020a for future processing to fix dnfirst 2023-11-08T16:34:51.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x1000000020b for future processing to fix dnfirst 2023-11-08T16:34:51.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x1000000020c for future processing to fix dnfirst 2023-11-08T16:34:51.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x1000000020d for future processing to fix dnfirst 2023-11-08T16:34:51.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-11-08T16:34:51.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-11-08T16:34:51.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-11-08T16:34:51.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x100000001fa for future processing to fix dnfirst 2023-11-08T16:34:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x100000001fb for future processing to fix dnfirst 2023-11-08T16:34:51.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x100000001fc for future processing to fix dnfirst 2023-11-08T16:34:51.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x100000001fd for future processing to fix dnfirst 2023-11-08T16:34:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x100000001fe for future processing to fix dnfirst 2023-11-08T16:34:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-11-08T16:34:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-08T16:34:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-11-08T16:34:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-08T16:34:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-11-08T16:34:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-08T16:34:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-11-08T16:34:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-08T16:34:51.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-11-08T16:34:51.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-08T16:34:51.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-11-08T16:34:51.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-08T16:34:51.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-11-08T16:34:51.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.754+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:69 7.13 7:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78000 con 0x55cfb8b3bc00 2023-11-08T16:34:51.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.755+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 16 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.755+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-11-08T16:34:51.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.755+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-08T16:34:51.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.755+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-11-08T16:34:51.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.755+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-08T16:34:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.755+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-11-08T16:34:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.755+0000 7fdb15a425c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-08T16:34:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.755+0000 7fdb15a425c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-11-08T16:34:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.755+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:70 7.1f 7:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78700 con 0x55cfb8b5dc00 2023-11-08T16:34:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.756+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 14 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.756+0000 7fdb15a425c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:34:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.756+0000 7fdb15a425c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-11-08T16:34:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.756+0000 7fdb15a425c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-11-08T16:34:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.756+0000 7fdb15a425c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-11-08T16:34:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.756+0000 7fdb15a425c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-11-08T16:34:51.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.756+0000 7fdb15a425c0 10 datascan.scan_links: processing 26 injected_inos 2023-11-08T16:34:51.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.756+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-11-08T16:34:51.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.756+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-11-08T16:34:51.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.756+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:71 7.1f 7:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78380 con 0x55cfb8b5dc00 2023-11-08T16:34:51.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.757+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 15 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.757+0000 7fdb15a425c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:34:51.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.757+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.757+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:72 7.1f 7:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78e00 con 0x55cfb8b5dc00 2023-11-08T16:34:51.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.761+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 16 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v64'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.761+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-11-08T16:34:51.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.761+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-11-08T16:34:51.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.761+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:0 2023-11-08T16:34:51.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.761+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:73 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79180 con 0x55cfb8b5d000 2023-11-08T16:34:51.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.762+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.762+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=0 frag=*, r=0 2023-11-08T16:34:51.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.762+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.762+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:74 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78a80 con 0x55cfb8b5d000 2023-11-08T16:34:51.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.765+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v64'42 uv42 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.765+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-11-08T16:34:51.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.765+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-11-08T16:34:51.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.765+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:1 2023-11-08T16:34:51.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.765+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:75 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78700 con 0x55cfb8b5d000 2023-11-08T16:34:51.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.766+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(75 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.766+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=1 frag=*, r=0 2023-11-08T16:34:51.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.766+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.766+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:76 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78380 con 0x55cfb8b5d000 2023-11-08T16:34:51.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.769+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(76 10000000000.00000000 [omap-set-vals] v64'43 uv43 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.769+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-11-08T16:34:51.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.769+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-11-08T16:34:51.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.769+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:2 2023-11-08T16:34:51.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.769+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:77 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79500 con 0x55cfb8b5d000 2023-11-08T16:34:51.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.770+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.770+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=2 frag=*, r=0 2023-11-08T16:34:51.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.770+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.770+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:78 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79880 con 0x55cfb8b5d000 2023-11-08T16:34:51.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.773+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 18 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v64'44 uv44 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.773+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-11-08T16:34:51.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.773+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-11-08T16:34:51.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.773+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:3 2023-11-08T16:34:51.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.773+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:79 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.774+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 19 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.774+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=3 frag=*, r=0 2023-11-08T16:34:51.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.774+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.774+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:80 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78e00 con 0x55cfb8b5d000 2023-11-08T16:34:51.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.777+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 20 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v64'45 uv45 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.777+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-11-08T16:34:51.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.777+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-11-08T16:34:51.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.777+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:4 2023-11-08T16:34:51.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.777+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:81 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79180 con 0x55cfb8b5d000 2023-11-08T16:34:51.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.778+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 21 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.778+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=4 frag=*, r=0 2023-11-08T16:34:51.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.778+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.778+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:82 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78a80 con 0x55cfb8b5d000 2023-11-08T16:34:51.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.781+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 22 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v64'46 uv46 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.781+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-11-08T16:34:51.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.781+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x100000001fa 2023-11-08T16:34:51.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.781+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:5 2023-11-08T16:34:51.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.781+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:83 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78700 con 0x55cfb8b5d000 2023-11-08T16:34:51.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.782+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 23 ==== osd_op_reply(83 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.782+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=5 frag=*, r=0 2023-11-08T16:34:51.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.782+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.782+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:84 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78380 con 0x55cfb8b5d000 2023-11-08T16:34:51.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.786+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 24 ==== osd_op_reply(84 10000000000.00000000 [omap-set-vals] v64'47 uv47 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.786+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-11-08T16:34:51.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.786+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x100000001fb 2023-11-08T16:34:51.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.786+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:6 2023-11-08T16:34:51.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.786+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:85 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79c00 con 0x55cfb8b5d000 2023-11-08T16:34:51.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.787+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 25 ==== osd_op_reply(85 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.787+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=6 frag=*, r=0 2023-11-08T16:34:51.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.787+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.787+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:86 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79500 con 0x55cfb8b5d000 2023-11-08T16:34:51.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.790+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 26 ==== osd_op_reply(86 10000000000.00000000 [omap-set-vals] v64'48 uv48 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.790+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-11-08T16:34:51.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.790+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x100000001fc 2023-11-08T16:34:51.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.790+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:7 2023-11-08T16:34:51.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.790+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:87 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79880 con 0x55cfb8b5d000 2023-11-08T16:34:51.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.791+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 27 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.791+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=7 frag=*, r=0 2023-11-08T16:34:51.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.791+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.791+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:88 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78e00 con 0x55cfb8b5d000 2023-11-08T16:34:51.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.795+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 28 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v64'49 uv49 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.796+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-11-08T16:34:51.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.796+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x100000001fd 2023-11-08T16:34:51.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.796+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:8 2023-11-08T16:34:51.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.796+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:89 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79180 con 0x55cfb8b5d000 2023-11-08T16:34:51.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.796+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 29 ==== osd_op_reply(89 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.797+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=8 frag=*, r=0 2023-11-08T16:34:51.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.797+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.797+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:90 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78a80 con 0x55cfb8b5d000 2023-11-08T16:34:51.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.800+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 30 ==== osd_op_reply(90 10000000000.00000000 [omap-set-vals] v64'50 uv50 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.800+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-11-08T16:34:51.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.800+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x100000001fe 2023-11-08T16:34:51.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.800+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:9 2023-11-08T16:34:51.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.800+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:91 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.801+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 31 ==== osd_op_reply(91 10000000000.00000000 [omap-get-vals-by-keys out=502b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.801+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=9 frag=*, r=0 2023-11-08T16:34:51.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.801+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.801+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:92 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78700 con 0x55cfb8b5d000 2023-11-08T16:34:51.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.804+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 32 ==== osd_op_reply(92 10000000000.00000000 [omap-set-vals] v64'51 uv51 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.805+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-11-08T16:34:51.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.805+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x100000001ff 2023-11-08T16:34:51.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.805+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:10 2023-11-08T16:34:51.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.805+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:93 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78380 con 0x55cfb8b5d000 2023-11-08T16:34:51.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.805+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 33 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.806+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=10 frag=*, r=0 2023-11-08T16:34:51.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.806+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.806+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:94 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79c00 con 0x55cfb8b5d000 2023-11-08T16:34:51.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.810+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 34 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v64'52 uv52 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.810+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-11-08T16:34:51.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.810+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000200 2023-11-08T16:34:51.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.810+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:11 2023-11-08T16:34:51.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.810+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:95 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79500 con 0x55cfb8b5d000 2023-11-08T16:34:51.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.811+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 35 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.811+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=11 frag=*, r=0 2023-11-08T16:34:51.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.811+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.811+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:96 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79880 con 0x55cfb8b5d000 2023-11-08T16:34:51.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.814+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 36 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v64'53 uv53 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.814+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-11-08T16:34:51.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.814+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000201 2023-11-08T16:34:51.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.814+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:12 2023-11-08T16:34:51.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.814+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:97 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78e00 con 0x55cfb8b5d000 2023-11-08T16:34:51.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.815+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 37 ==== osd_op_reply(97 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.815+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=12 frag=*, r=0 2023-11-08T16:34:51.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.815+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.815+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:98 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79180 con 0x55cfb8b5d000 2023-11-08T16:34:51.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.819+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 38 ==== osd_op_reply(98 10000000000.00000000 [omap-set-vals] v64'54 uv54 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.819+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-11-08T16:34:51.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.819+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000202 2023-11-08T16:34:51.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.819+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:13 2023-11-08T16:34:51.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.819+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:99 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78a80 con 0x55cfb8b5d000 2023-11-08T16:34:51.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.820+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 39 ==== osd_op_reply(99 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.820+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=13 frag=*, r=0 2023-11-08T16:34:51.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.820+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.820+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:100 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78700 con 0x55cfb8b5d000 2023-11-08T16:34:51.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.824+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 40 ==== osd_op_reply(100 10000000000.00000000 [omap-set-vals] v64'55 uv55 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.824+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-11-08T16:34:51.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.824+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000203 2023-11-08T16:34:51.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.824+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:14 2023-11-08T16:34:51.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.824+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:101 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78380 con 0x55cfb8b5d000 2023-11-08T16:34:51.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.825+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 41 ==== osd_op_reply(101 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.825+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=14 frag=*, r=0 2023-11-08T16:34:51.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.825+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.825+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:102 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79c00 con 0x55cfb8b5d000 2023-11-08T16:34:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.829+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 42 ==== osd_op_reply(102 10000000000.00000000 [omap-set-vals] v64'56 uv56 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.829+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-11-08T16:34:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.829+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000204 2023-11-08T16:34:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.829+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:15 2023-11-08T16:34:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.829+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:103 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79500 con 0x55cfb8b5d000 2023-11-08T16:34:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.830+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 43 ==== osd_op_reply(103 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.830+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=15 frag=*, r=0 2023-11-08T16:34:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.830+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.830+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:104 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79880 con 0x55cfb8b5d000 2023-11-08T16:34:51.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.834+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 44 ==== osd_op_reply(104 10000000000.00000000 [omap-set-vals] v64'57 uv57 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.834+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-11-08T16:34:51.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.834+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000205 2023-11-08T16:34:51.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.834+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:16 2023-11-08T16:34:51.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.834+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:105 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78e00 con 0x55cfb8b5d000 2023-11-08T16:34:51.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.835+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 45 ==== osd_op_reply(105 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv57 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.835+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=16 frag=*, r=0 2023-11-08T16:34:51.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.835+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.835+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:106 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.839+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 46 ==== osd_op_reply(106 10000000000.00000000 [omap-set-vals] v64'58 uv58 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.839+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-11-08T16:34:51.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.839+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000206 2023-11-08T16:34:51.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.839+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:17 2023-11-08T16:34:51.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.839+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:107 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb883f880 con 0x55cfb8b5d000 2023-11-08T16:34:51.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.840+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 47 ==== osd_op_reply(107 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv58 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.840+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=17 frag=*, r=0 2023-11-08T16:34:51.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.840+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.840+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:108 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79180 con 0x55cfb8b5d000 2023-11-08T16:34:51.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.844+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 48 ==== osd_op_reply(108 10000000000.00000000 [omap-set-vals] v64'59 uv59 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.844+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-11-08T16:34:51.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.844+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000207 2023-11-08T16:34:51.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.844+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:18 2023-11-08T16:34:51.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.844+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:109 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78a80 con 0x55cfb8b5d000 2023-11-08T16:34:51.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.845+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 49 ==== osd_op_reply(109 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv59 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.845+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=18 frag=*, r=0 2023-11-08T16:34:51.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.845+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.845+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:110 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78700 con 0x55cfb8b5d000 2023-11-08T16:34:51.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.848+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 50 ==== osd_op_reply(110 10000000000.00000000 [omap-set-vals] v64'60 uv60 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.848+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-11-08T16:34:51.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.848+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000208 2023-11-08T16:34:51.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.848+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:19 2023-11-08T16:34:51.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.848+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:111 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78380 con 0x55cfb8b5d000 2023-11-08T16:34:51.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.849+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 51 ==== osd_op_reply(111 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv60 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.849+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=19 frag=*, r=0 2023-11-08T16:34:51.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.849+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.849+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:112 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79c00 con 0x55cfb8b5d000 2023-11-08T16:34:51.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.854+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 52 ==== osd_op_reply(112 10000000000.00000000 [omap-set-vals] v64'61 uv61 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.854+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-11-08T16:34:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.854+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x10000000209 2023-11-08T16:34:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.854+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:20 2023-11-08T16:34:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.854+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:113 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79500 con 0x55cfb8b5d000 2023-11-08T16:34:51.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.855+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 53 ==== osd_op_reply(113 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv61 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.855+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=20 frag=*, r=0 2023-11-08T16:34:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.855+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.855+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:114 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c79880 con 0x55cfb8b5d000 2023-11-08T16:34:51.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.858+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 54 ==== osd_op_reply(114 10000000000.00000000 [omap-set-vals] v64'62 uv62 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.858+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-11-08T16:34:51.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.858+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x1000000020a 2023-11-08T16:34:51.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.858+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:21 2023-11-08T16:34:51.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.858+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:115 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8c78e00 con 0x55cfb8b5d000 2023-11-08T16:34:51.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.859+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 55 ==== osd_op_reply(115 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv62 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.859+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=21 frag=*, r=0 2023-11-08T16:34:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.859+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.859+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:116 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8cac000 con 0x55cfb8b5d000 2023-11-08T16:34:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.863+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 56 ==== osd_op_reply(116 10000000000.00000000 [omap-set-vals] v64'63 uv63 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.863+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-11-08T16:34:51.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.863+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x1000000020b 2023-11-08T16:34:51.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.863+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:22 2023-11-08T16:34:51.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.863+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:117 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8cac380 con 0x55cfb8b5d000 2023-11-08T16:34:51.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.864+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 57 ==== osd_op_reply(117 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv63 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.864+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=22 frag=*, r=0 2023-11-08T16:34:51.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.864+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.864+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:118 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8cac700 con 0x55cfb8b5d000 2023-11-08T16:34:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.868+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 58 ==== osd_op_reply(118 10000000000.00000000 [omap-set-vals] v64'64 uv64 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.868+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-11-08T16:34:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.868+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x1000000020c 2023-11-08T16:34:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.868+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:23 2023-11-08T16:34:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.868+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:119 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8caca80 con 0x55cfb8b5d000 2023-11-08T16:34:51.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.869+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 59 ==== osd_op_reply(119 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv64 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.869+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=23 frag=*, r=0 2023-11-08T16:34:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.869+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.869+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:120 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8cace00 con 0x55cfb8b5d000 2023-11-08T16:34:51.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.872+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 60 ==== osd_op_reply(120 10000000000.00000000 [omap-set-vals] v64'65 uv65 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.872+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-11-08T16:34:51.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.872+0000 7fdb15a425c0 10 datascan.scan_links: handling injected_ino 0x1000000020d 2023-11-08T16:34:51.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.872+0000 7fdb15a425c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:24 2023-11-08T16:34:51.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.872+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:121 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8cad180 con 0x55cfb8b5d000 2023-11-08T16:34:51.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.873+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 61 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=503b] v0'0 uv65 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.873+0000 7fdb15a425c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=24 frag=*, r=0 2023-11-08T16:34:51.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.873+0000 7fdb15a425c0 20 datascan.scan_links: first is now 2 2023-11-08T16:34:51.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.873+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:122 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8cad500 con 0x55cfb8b5d000 2023-11-08T16:34:51.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.877+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.3 v2:172.21.15.81:6816/2756229107 62 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v64'66 uv66 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5d000 2023-11-08T16:34:51.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.877+0000 7fdb15a425c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-11-08T16:34:51.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.877+0000 7fdb15a425c0 10 datascan.scan_links: updating inotable 2023-11-08T16:34:51.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.877+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:123 7.13 7:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8cad880 con 0x55cfb8b3bc00 2023-11-08T16:34:51.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.878+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 17 ==== osd_op_reply(123 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.879+0000 7fdb15a425c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-11-08T16:34:51.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.879+0000 7fdb15a425c0 10 mds.0.inotable: skip_inos now [0x1000000020e~0xfffffffdf2] 2023-11-08T16:34:51.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.879+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:124 7.13 7:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8cadc00 con 0x55cfb8b3bc00 2023-11-08T16:34:51.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.883+0000 7fdb139e9640 1 -- 172.21.15.81:0/1670434595 <== osd.0 v2:172.21.15.81:6808/1666604866 18 ==== osd_op_reply(124 mds0_inotable [writefull 0~34] v64'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55cfb9688b40 con 0x55cfb8b3bc00 2023-11-08T16:34:51.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.883+0000 7fdb15a425c0 10 datascan.scan_links: updating snaptable 2023-11-08T16:34:51.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.883+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:125 7.d 7:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e64) v8 -- 0x55cfb8cad880 con 0x55cfb8b5dc00 2023-11-08T16:34:51.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb149eb640 1 -- 172.21.15.81:0/1670434595 <== osd.6 v2:172.21.15.93:6816/607960134 17 ==== osd_op_reply(125 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55cfb9688d80 con 0x55cfb8b5dc00 2023-11-08T16:34:51.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55cfb8b3bc00 msgr2=0x55cfb8c4cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55cfb8b3bc00 0x55cfb8c4cb00 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55cfb8bd1400 msgr2=0x55cfb8c4db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55cfb8bd1400 0x55cfb8c4db80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55cfb95b3400 msgr2=0x55cfb8c4c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55cfb95b3400 0x55cfb8c4c580 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55cfb8b5d000 msgr2=0x55cfb8c4c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55cfb8b5d000 0x55cfb8c4c000 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55cfb8bd0000 msgr2=0x55cfb8c4d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55cfb8bd0000 0x55cfb8c4d600 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 reap_dead start 2023-11-08T16:34:51.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55cfb8b3a800 msgr2=0x55cfb8851600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55cfb8b3a800 0x55cfb8851600 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55cfb8b5dc00 msgr2=0x55cfb8c4d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55cfb8b5dc00 0x55cfb8c4d080 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55cfb8bd1800 msgr2=0x55cfb8c4e100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55cfb8bd1800 0x55cfb8c4e100 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 reap_dead start 2023-11-08T16:34:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.885+0000 7fdb141ea640 1 -- 172.21.15.81:0/1670434595 reap_dead start 2023-11-08T16:34:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.886+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cfb8b5cc00 msgr2=0x55cfb8853700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.886+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cfb8b5cc00 0x55cfb8853700 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55cfb8ade960 tx=0x55cfb9576060 comp rx=0 tx=0).stop 2023-11-08T16:34:51.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.886+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3ac00 msgr2=0x55cfb8852680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.886+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3ac00 0x55cfb8852680 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x55cfb8adf5c0 tx=0x55cfb8aeae40 comp rx=0 tx=0).stop 2023-11-08T16:34:51.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.886+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 shutdown_connections 2023-11-08T16:34:51.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.886+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cfb8b5cc00 0x55cfb8853700 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.886+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/1670434595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb8b3ac00 0x55cfb8852680 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.886+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 >> 172.21.15.81:0/1670434595 conn(0x55cfb95b3c00 msgr2=0x55cfb9571000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.887+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 shutdown_connections 2023-11-08T16:34:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.887+0000 7fdb15a425c0 1 -- 172.21.15.81:0/1670434595 wait complete. 2023-11-08T16:34:51.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.888+0000 7fdb15a425c0 1 -- 172.21.15.81:0/2728872468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb87e7c00 msgr2=0x55cfb8851080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:51.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.888+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/2728872468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb87e7c00 0x55cfb8851080 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x55cfb95e6a50 tx=0x55cfb8ac6120 comp rx=0 tx=0).stop 2023-11-08T16:34:51.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.888+0000 7fdb15a425c0 1 -- 172.21.15.81:0/2728872468 shutdown_connections 2023-11-08T16:34:51.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.888+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/2728872468 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cfb87e7c00 0x55cfb8851080 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.888+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/2728872468 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cfb87e7800 0x55cfb8850b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.888+0000 7fdb15a425c0 1 --2- 172.21.15.81:0/2728872468 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cfb87e6c00 0x55cfb8850580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:51.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.888+0000 7fdb15a425c0 1 -- 172.21.15.81:0/2728872468 >> 172.21.15.81:0/2728872468 conn(0x55cfb87e7000 msgr2=0x55cfb9570800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:51.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.888+0000 7fdb15a425c0 1 -- 172.21.15.81:0/2728872468 shutdown_connections 2023-11-08T16:34:51.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:51.888+0000 7fdb15a425c0 1 -- 172.21.15.81:0/2728872468 wait complete. 2023-11-08T16:34:51.900 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.324003, result: 2023-11-08T16:34:51.901 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-08T16:34:52.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.119+0000 7f12a861a640 1 Processor -- start 2023-11-08T16:34:52.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.120+0000 7f12a861a640 1 -- start start 2023-11-08T16:34:52.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.121+0000 7f12a861a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f12a0151d60 0x7f12a00b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:52.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.121+0000 7f12a861a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a00cdf40 0x7f12a015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:52.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.121+0000 7f12a861a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f12a00ce8d0 0x7f12a00ced40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:52.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.121+0000 7f12a861a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a00cdb20 con 0x7f12a0151d60 2023-11-08T16:34:52.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.121+0000 7f12a861a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12a00cf280 con 0x7f12a00ce8d0 2023-11-08T16:34:52.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.121+0000 7f12a861a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a00ce3d0 con 0x7f12a00cdf40 2023-11-08T16:34:52.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.121+0000 7f12a63b6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f12a0151d60 0x7f12a00b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:52.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.121+0000 7f12a63b6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f12a0151d60 0x7f12a00b33d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55048/0 (socket says 172.21.15.81:55048) 2023-11-08T16:34:52.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.121+0000 7f12a63b6640 1 -- 172.21.15.81:0/2245531004 learned_addr learned my addr 172.21.15.81:0/2245531004 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:52.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.122+0000 7f12a5bb5640 1 --2- 172.21.15.81:0/2245531004 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a00cdf40 0x7f12a015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:52.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.122+0000 7f12a63b6640 1 -- 172.21.15.81:0/2245531004 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a00cdf40 msgr2=0x7f12a015bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:52.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.122+0000 7f12a6bb7640 1 --2- 172.21.15.81:0/2245531004 >> v2:172.21.15.93:3300/0 conn(0x7f12a00ce8d0 0x7f12a00ced40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:52.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.122+0000 7f12a63b6640 1 --2- 172.21.15.81:0/2245531004 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a00cdf40 0x7f12a015bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.122+0000 7f12a63b6640 1 -- 172.21.15.81:0/2245531004 >> v2:172.21.15.93:3300/0 conn(0x7f12a00ce8d0 msgr2=0x7f12a00ced40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:52.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.122+0000 7f12a63b6640 1 --2- 172.21.15.81:0/2245531004 >> v2:172.21.15.93:3300/0 conn(0x7f12a00ce8d0 0x7f12a00ced40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.122+0000 7f12a63b6640 1 -- 172.21.15.81:0/2245531004 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a00ccc40 con 0x7f12a0151d60 2023-11-08T16:34:52.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.122+0000 7f12a6bb7640 1 --2- 172.21.15.81:0/2245531004 >> v2:172.21.15.93:3300/0 conn(0x7f12a00ce8d0 0x7f12a00ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:52.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.123+0000 7f12a63b6640 1 --2- 172.21.15.81:0/2245531004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0151d60 0x7f12a00b33d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f12a00cb2e0 tx=0x7f129c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e2d91c40adb255 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:52.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.123+0000 7f12a53b4640 1 -- 172.21.15.81:0/2245531004 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c0317c0 con 0x7f12a0151d60 2023-11-08T16:34:52.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.123+0000 7f12a53b4640 1 -- 172.21.15.81:0/2245531004 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f129c031920 con 0x7f12a0151d60 2023-11-08T16:34:52.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.123+0000 7f12a53b4640 1 -- 172.21.15.81:0/2245531004 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f129c031cb0 con 0x7f12a0151d60 2023-11-08T16:34:52.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.124+0000 7f12a861a640 1 -- 172.21.15.81:0/2245531004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0151d60 msgr2=0x7f12a00b33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:52.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.124+0000 7f12a861a640 1 --2- 172.21.15.81:0/2245531004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0151d60 0x7f12a00b33d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f12a00cb2e0 tx=0x7f129c02b860 comp rx=0 tx=0).stop 2023-11-08T16:34:52.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.124+0000 7f12a861a640 1 -- 172.21.15.81:0/2245531004 shutdown_connections 2023-11-08T16:34:52.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.124+0000 7f12a861a640 1 --2- 172.21.15.81:0/2245531004 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a00cdf40 0x7f12a015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.124+0000 7f12a861a640 1 --2- 172.21.15.81:0/2245531004 >> v2:172.21.15.93:3300/0 conn(0x7f12a00ce8d0 0x7f12a00ced40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.124+0000 7f12a861a640 1 --2- 172.21.15.81:0/2245531004 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a0151d60 0x7f12a00b33d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.124+0000 7f12a861a640 1 -- 172.21.15.81:0/2245531004 >> 172.21.15.81:0/2245531004 conn(0x7f12a0065df0 msgr2=0x7f12a0066200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:52.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.124+0000 7f12a861a640 1 -- 172.21.15.81:0/2245531004 shutdown_connections 2023-11-08T16:34:52.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.124+0000 7f12a861a640 1 -- 172.21.15.81:0/2245531004 wait complete. 2023-11-08T16:34:52.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.126+0000 7f12a861a640 1 Processor -- start 2023-11-08T16:34:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.126+0000 7f12a861a640 1 -- start start 2023-11-08T16:34:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.127+0000 7f12a861a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a00cdf40 0x7f12a00c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.127+0000 7f12a861a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00ce8d0 0x7f12a01f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.127+0000 7f12a861a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12a0151d60 0x7f12a00c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:52.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.127+0000 7f12a861a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a0143ed0 con 0x7f12a00ce8d0 2023-11-08T16:34:52.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.127+0000 7f12a861a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12a01438e0 con 0x7f12a0151d60 2023-11-08T16:34:52.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.127+0000 7f12a861a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12a01f4700 con 0x7f12a00cdf40 2023-11-08T16:34:52.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.127+0000 7f12a5bb5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00ce8d0 0x7f12a01f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:52.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.127+0000 7f12a5bb5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00ce8d0 0x7f12a01f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55060/0 (socket says 172.21.15.81:55060) 2023-11-08T16:34:52.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.127+0000 7f12a5bb5640 1 -- 172.21.15.81:0/569396288 learned_addr learned my addr 172.21.15.81:0/569396288 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:52.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f12a63b6640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a00cdf40 0x7f12a00c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:52.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f12a5bb5640 1 -- 172.21.15.81:0/569396288 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a00cdf40 msgr2=0x7f12a00c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:52.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f12a6bb7640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12a0151d60 0x7f12a00c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:52.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f12a5bb5640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a00cdf40 0x7f12a00c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f12a5bb5640 1 -- 172.21.15.81:0/569396288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12a0151d60 msgr2=0x7f12a00c33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:52.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f12a5bb5640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12a0151d60 0x7f12a00c33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f12a5bb5640 1 -- 172.21.15.81:0/569396288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129c0094d0 con 0x7f12a00ce8d0 2023-11-08T16:34:52.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f12a63b6640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a00cdf40 0x7f12a00c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:52.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f12a5bb5640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00ce8d0 0x7f12a01f3c60 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f129000c7f0 tx=0x7f129000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:52.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f1296ffd640 1 -- 172.21.15.81:0/569396288 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290004220 con 0x7f12a00ce8d0 2023-11-08T16:34:52.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f1296ffd640 1 -- 172.21.15.81:0/569396288 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1290004380 con 0x7f12a00ce8d0 2023-11-08T16:34:52.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f12a861a640 1 -- 172.21.15.81:0/569396288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a01f49e0 con 0x7f12a00ce8d0 2023-11-08T16:34:52.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f1296ffd640 1 -- 172.21.15.81:0/569396288 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1290007860 con 0x7f12a00ce8d0 2023-11-08T16:34:52.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.128+0000 7f12a861a640 1 -- 172.21.15.81:0/569396288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12a0146690 con 0x7f12a00ce8d0 2023-11-08T16:34:52.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.129+0000 7f12a861a640 1 -- 172.21.15.81:0/569396288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12a01f4b70 con 0x7f12a00ce8d0 2023-11-08T16:34:52.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.131+0000 7f1296ffd640 1 -- 172.21.15.81:0/569396288 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1290007bd0 con 0x7f12a00ce8d0 2023-11-08T16:34:52.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.132+0000 7f1296ffd640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f128406ae70 0x7f128406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:52.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.132+0000 7f1296ffd640 1 -- 172.21.15.81:0/569396288 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6372+0+0 (secure 0 0 0) 0x7f12900898c0 con 0x7f12a00ce8d0 2023-11-08T16:34:52.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.132+0000 7f12a63b6640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f128406ae70 0x7f128406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:52.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.133+0000 7f12a63b6640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f128406ae70 0x7f128406d330 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f129c036000 tx=0x7f129c03d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:52.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.136+0000 7f1296ffd640 1 -- 172.21.15.81:0/569396288 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1290054640 con 0x7f12a00ce8d0 2023-11-08T16:34:52.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.397+0000 7f12a861a640 1 -- 172.21.15.81:0/569396288 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f12a00743f0 con 0x7f12a00ce8d0 2023-11-08T16:34:52.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.903+0000 7f1296ffd640 1 -- 172.21.15.81:0/569396288 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 3:0 v48) v1 ==== 101+0+0 (secure 0 0 0) 0x7f129001e020 con 0x7f12a00ce8d0 2023-11-08T16:34:52.905 INFO:teuthology.orchestra.run.smithi081.stderr:repaired: restoring rank 3:0 2023-11-08T16:34:52.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 -- 172.21.15.81:0/569396288 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f128406ae70 msgr2=0x7f128406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f128406ae70 0x7f128406d330 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f129c036000 tx=0x7f129c03d040 comp rx=0 tx=0).stop 2023-11-08T16:34:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 -- 172.21.15.81:0/569396288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00ce8d0 msgr2=0x7f12a01f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00ce8d0 0x7f12a01f3c60 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f129000c7f0 tx=0x7f129000ccb0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 -- 172.21.15.81:0/569396288 shutdown_connections 2023-11-08T16:34:52.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f128406ae70 0x7f128406d330 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f12a0151d60 0x7f12a00c33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f12a00ce8d0 0x7f12a01f3c60 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 --2- 172.21.15.81:0/569396288 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f12a00cdf40 0x7f12a00c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:52.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 -- 172.21.15.81:0/569396288 >> 172.21.15.81:0/569396288 conn(0x7f12a0065df0 msgr2=0x7f12a0151350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:52.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 -- 172.21.15.81:0/569396288 shutdown_connections 2023-11-08T16:34:52.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:52.907+0000 7f12a861a640 1 -- 172.21.15.81:0/569396288 wait complete. 2023-11-08T16:34:52.917 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:34:52.916+0000 7fc60eb4e640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-08T16:34:52.932 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T16:34:52.932 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T16:34:52.933 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T16:34:52.933 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:34:52.933 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T16:34:52.934 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T16:34:52.934 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T16:34:52.934 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:34:52.934 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T16:34:52.935 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T16:34:52.935 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T16:34:52.935 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:34:52.936 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T16:34:52.936 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T16:34:52.936 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T16:34:52.936 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:34:52.953 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T16:34:52.953 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T16:34:52.956 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T16:34:52.956 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T16:34:52.963 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T16:34:52.963 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T16:34:52.968 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T16:34:52.968 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T16:34:53.003 INFO:tasks.ceph.mds.a:Started 2023-11-08T16:34:53.004 DEBUG:teuthology.parallel:result is None 2023-11-08T16:34:53.011 INFO:tasks.ceph.mds.c:Started 2023-11-08T16:34:53.012 DEBUG:teuthology.parallel:result is None 2023-11-08T16:34:53.018 INFO:tasks.ceph.mds.d:Started 2023-11-08T16:34:53.018 DEBUG:teuthology.parallel:result is None 2023-11-08T16:34:53.026 INFO:tasks.ceph.mds.b:Started 2023-11-08T16:34:53.026 DEBUG:teuthology.parallel:result is None 2023-11-08T16:34:53.027 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:34:53.166 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:34:53.167 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:34:53.173 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:34:53.172+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:34:53.179 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:34:53.188 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2023-11-08T16:34:53.202 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:34:53.201+0000 7f1a3fe83640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-08T16:34:53.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.223+0000 7fb28b9d7640 1 Processor -- start 2023-11-08T16:34:53.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.223+0000 7fb28b9d7640 1 -- start start 2023-11-08T16:34:53.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.224+0000 7fb28b9d7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb2840ca480 0x7fb28415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:53.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.224+0000 7fb28b9d7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb28415ea60 0x7fb28415eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:53.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.224+0000 7fb28b9d7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb28406b630 con 0x7fb2840ade50 2023-11-08T16:34:53.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.224+0000 7fb28b9d7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2840b1860 con 0x7fb2840ca480 2023-11-08T16:34:53.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.224+0000 7fb28b9d7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2840ca910 con 0x7fb28415ea60 2023-11-08T16:34:53.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.224+0000 7fb288f72640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb2840ca480 0x7fb28415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:53.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.224+0000 7fb288f72640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb2840ca480 0x7fb28415ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55120/0 (socket says 172.21.15.81:55120) 2023-11-08T16:34:53.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.224+0000 7fb288f72640 1 -- 172.21.15.81:0/797047659 learned_addr learned my addr 172.21.15.81:0/797047659 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:53.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.224+0000 7fb289f74640 1 --2- 172.21.15.81:0/797047659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb28415ea60 0x7fb28415eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:53.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.225+0000 7fb288f72640 1 -- 172.21.15.81:0/797047659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb28415ea60 msgr2=0x7fb28415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:53.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.225+0000 7fb288f72640 1 --2- 172.21.15.81:0/797047659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb28415ea60 0x7fb28415eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:53.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.225+0000 7fb288f72640 1 -- 172.21.15.81:0/797047659 >> v1:172.21.15.93:6789/0 conn(0x7fb2840ade50 legacy=0x7fb28415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:53.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.225+0000 7fb288f72640 1 -- 172.21.15.81:0/797047659 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840cabb0 con 0x7fb2840ca480 2023-11-08T16:34:53.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.226+0000 7fb288f72640 1 --2- 172.21.15.81:0/797047659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2840ca480 0x7fb28415ad90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb274009870 tx=0x7fb27402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ba928a76eb9273 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:53.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.226+0000 7fb27bfff640 1 -- 172.21.15.81:0/797047659 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274030840 con 0x7fb2840ca480 2023-11-08T16:34:53.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.226+0000 7fb27bfff640 1 -- 172.21.15.81:0/797047659 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2740309a0 con 0x7fb2840ca480 2023-11-08T16:34:53.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.226+0000 7fb27bfff640 1 -- 172.21.15.81:0/797047659 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274030d30 con 0x7fb2840ca480 2023-11-08T16:34:53.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.226+0000 7fb28b9d7640 1 -- 172.21.15.81:0/797047659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2840ca480 msgr2=0x7fb28415ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:53.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.226+0000 7fb28b9d7640 1 --2- 172.21.15.81:0/797047659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2840ca480 0x7fb28415ad90 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb274009870 tx=0x7fb27402b980 comp rx=0 tx=0).stop 2023-11-08T16:34:53.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.227+0000 7fb28b9d7640 1 -- 172.21.15.81:0/797047659 shutdown_connections 2023-11-08T16:34:53.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.227+0000 7fb28b9d7640 1 --2- 172.21.15.81:0/797047659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb28415ea60 0x7fb28415eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:53.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.227+0000 7fb28b9d7640 1 --2- 172.21.15.81:0/797047659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2840ca480 0x7fb28415ad90 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:53.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.227+0000 7fb28b9d7640 1 -- 172.21.15.81:0/797047659 >> 172.21.15.81:0/797047659 conn(0x7fb2840cb490 msgr2=0x7fb2840cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:53.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.227+0000 7fb28b9d7640 1 -- 172.21.15.81:0/797047659 shutdown_connections 2023-11-08T16:34:53.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.227+0000 7fb28b9d7640 1 -- 172.21.15.81:0/797047659 wait complete. 2023-11-08T16:34:53.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.228+0000 7fb28b9d7640 1 Processor -- start 2023-11-08T16:34:53.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.228+0000 7fb28b9d7640 1 -- start start 2023-11-08T16:34:53.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.229+0000 7fb28b9d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2840ade50 0x7fb2841556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:53.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.229+0000 7fb28b9d7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2840ca480 0x7fb2840ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:53.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.229+0000 7fb28b9d7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb28415ea60 0x7fb2841e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:53.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.229+0000 7fb28b9d7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb284155240 con 0x7fb2840ade50 2023-11-08T16:34:53.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.229+0000 7fb28b9d7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb284154630 con 0x7fb2840ca480 2023-11-08T16:34:53.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.229+0000 7fb28b9d7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb284154910 con 0x7fb28415ea60 2023-11-08T16:34:53.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.230+0000 7fb289773640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2840ade50 0x7fb2841556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:53.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.230+0000 7fb289773640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2840ade50 0x7fb2841556d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55134/0 (socket says 172.21.15.81:55134) 2023-11-08T16:34:53.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.230+0000 7fb289f74640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb28415ea60 0x7fb2841e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:53.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.230+0000 7fb289773640 1 -- 172.21.15.81:0/1395563822 learned_addr learned my addr 172.21.15.81:0/1395563822 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:53.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.230+0000 7fb288f72640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2840ca480 0x7fb2840ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:53.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.230+0000 7fb289773640 1 -- 172.21.15.81:0/1395563822 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb28415ea60 msgr2=0x7fb2841e4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:53.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.230+0000 7fb289773640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb28415ea60 0x7fb2841e4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:53.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.230+0000 7fb289773640 1 -- 172.21.15.81:0/1395563822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2840ca480 msgr2=0x7fb2840ce040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:53.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.230+0000 7fb289773640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2840ca480 0x7fb2840ce040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:53.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.230+0000 7fb289773640 1 -- 172.21.15.81:0/1395563822 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740094d0 con 0x7fb2840ade50 2023-11-08T16:34:53.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.230+0000 7fb289773640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2840ade50 0x7fb2841556d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb28000c450 tx=0x7fb28000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:53.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.231+0000 7fb279ffb640 1 -- 172.21.15.81:0/1395563822 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2800052d0 con 0x7fb2840ade50 2023-11-08T16:34:53.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.231+0000 7fb279ffb640 1 -- 172.21.15.81:0/1395563822 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb280005430 con 0x7fb2840ade50 2023-11-08T16:34:53.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.231+0000 7fb279ffb640 1 -- 172.21.15.81:0/1395563822 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb280012870 con 0x7fb2840ade50 2023-11-08T16:34:53.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.231+0000 7fb28b9d7640 1 -- 172.21.15.81:0/1395563822 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb284154bf0 con 0x7fb2840ade50 2023-11-08T16:34:53.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.231+0000 7fb28b9d7640 1 -- 172.21.15.81:0/1395563822 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2840ce580 con 0x7fb2840ade50 2023-11-08T16:34:53.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.232+0000 7fb28b9d7640 1 -- 172.21.15.81:0/1395563822 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28404ece0 con 0x7fb2840ade50 2023-11-08T16:34:53.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.233+0000 7fb279ffb640 1 -- 172.21.15.81:0/1395563822 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb2800129d0 con 0x7fb2840ade50 2023-11-08T16:34:53.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.234+0000 7fb279ffb640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb26806aea0 0x7fb26806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:53.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.234+0000 7fb279ffb640 1 -- 172.21.15.81:0/1395563822 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fb280089320 con 0x7fb2840ade50 2023-11-08T16:34:53.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.234+0000 7fb288f72640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb26806aea0 0x7fb26806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:53.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.235+0000 7fb288f72640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb26806aea0 0x7fb26806d360 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb274036000 tx=0x7fb274008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:53.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.239+0000 7fb279ffb640 1 -- 172.21.15.81:0/1395563822 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb2800540f0 con 0x7fb2840ade50 2023-11-08T16:34:53.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.461+0000 7fb28b9d7640 1 -- 172.21.15.81:0/1395563822 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb2840cc1d0 con 0x7fb2840ade50 2023-11-08T16:34:53.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.462+0000 7fb279ffb640 1 -- 172.21.15.81:0/1395563822 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) v1 ==== 94+0+4910 (secure 0 0 0) 0x7fb280057710 con 0x7fb2840ade50 2023-11-08T16:34:53.464 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:53.464 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":51,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4913,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3492741644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3492741644},{"type":"v1","addr":"172.21.15.93:6833","nonce":3492741644}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":4916,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/119699732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":119699732},{"type":"v1","addr":"172.21.15.93:6835","nonce":119699732}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/3140373039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3140373039},{"type":"v1","addr":"172.21.15.81:6836","nonce":3140373039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:53.180983+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":0,"incarnation":51,"state":"up:replay","state_seq":1,"addr":"172.21.15.81:6837/552116415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":552116415},{"type":"v1","addr":"172.21.15.81:6837","nonce":552116415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:34:53.464 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 51 2023-11-08T16:34:53.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.464+0000 7fb28b9d7640 1 -- 172.21.15.81:0/1395563822 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb26806aea0 msgr2=0x7fb26806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:53.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.464+0000 7fb28b9d7640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb26806aea0 0x7fb26806d360 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb274036000 tx=0x7fb274008030 comp rx=0 tx=0).stop 2023-11-08T16:34:53.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.464+0000 7fb28b9d7640 1 -- 172.21.15.81:0/1395563822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2840ade50 msgr2=0x7fb2841556d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:53.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.464+0000 7fb28b9d7640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2840ade50 0x7fb2841556d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fb28000c450 tx=0x7fb28000c910 comp rx=0 tx=0).stop 2023-11-08T16:34:53.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.464+0000 7fb28b9d7640 1 -- 172.21.15.81:0/1395563822 shutdown_connections 2023-11-08T16:34:53.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.464+0000 7fb28b9d7640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb26806aea0 0x7fb26806d360 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:53.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.464+0000 7fb28b9d7640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb28415ea60 0x7fb2841e4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:53.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.464+0000 7fb28b9d7640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2840ca480 0x7fb2840ce040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:53.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.464+0000 7fb28b9d7640 1 --2- 172.21.15.81:0/1395563822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2840ade50 0x7fb2841556d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:53.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.464+0000 7fb28b9d7640 1 -- 172.21.15.81:0/1395563822 >> 172.21.15.81:0/1395563822 conn(0x7fb2840cb490 msgr2=0x7fb284071140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:53.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.465+0000 7fb28b9d7640 1 -- 172.21.15.81:0/1395563822 shutdown_connections 2023-11-08T16:34:53.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:53.465+0000 7fb28b9d7640 1 -- 172.21.15.81:0/1395563822 wait complete. 2023-11-08T16:34:53.885 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:34:44.993932+0000', 'modified': '2023-11-08T16:34:53.180983+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6600}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6600': {'gid': 6600, 'name': 'c', 'rank': 0, 'incarnation': 51, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.81:6837/552116415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6835', 'nonce': 552116415}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 552116415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8, 9], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:34:53.886 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6600:up:replay 2023-11-08T16:34:54.888 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:34:55.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.109+0000 7fea67593640 1 Processor -- start 2023-11-08T16:34:55.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.109+0000 7fea67593640 1 -- start start 2023-11-08T16:34:55.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.110+0000 7fea67593640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fea6006d5d0 0x7fea60165540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:55.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.110+0000 7fea67593640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fea60165a80 0x7fea601629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:55.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.110+0000 7fea67593640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea60162050 con 0x7fea6006b850 2023-11-08T16:34:55.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea67593640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea60164e10 con 0x7fea60165a80 2023-11-08T16:34:55.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea67593640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea60162330 con 0x7fea6006d5d0 2023-11-08T16:34:55.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea66591640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fea6006b850 0x7fea60161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46500/0 (socket says 172.21.15.81:46500) 2023-11-08T16:34:55.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea66591640 1 -- 172.21.15.81:0/126964594 learned_addr learned my addr 172.21.15.81:0/126964594 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:55.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea66d92640 1 --2- 172.21.15.81:0/126964594 >> v2:172.21.15.81:3300/0 conn(0x7fea60165a80 0x7fea601629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:55.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea65d90640 1 --2- 172.21.15.81:0/126964594 >> v2:172.21.15.93:3300/0 conn(0x7fea6006d5d0 0x7fea60165540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:55.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea6558f640 1 -- 172.21.15.81:0/126964594 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996978941 0 0) 0x7fea60162050 con 0x7fea6006b850 2023-11-08T16:34:55.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea6558f640 1 -- 172.21.15.81:0/126964594 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea4c003520 con 0x7fea6006b850 2023-11-08T16:34:55.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea66d92640 1 -- 172.21.15.81:0/126964594 >> v2:172.21.15.93:3300/0 conn(0x7fea6006d5d0 msgr2=0x7fea60165540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:55.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea66d92640 1 --2- 172.21.15.81:0/126964594 >> v2:172.21.15.93:3300/0 conn(0x7fea6006d5d0 0x7fea60165540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea66d92640 1 -- 172.21.15.81:0/126964594 >> v1:172.21.15.81:6789/0 conn(0x7fea6006b850 legacy=0x7fea60161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:55.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.111+0000 7fea66d92640 1 -- 172.21.15.81:0/126964594 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea60162610 con 0x7fea60165a80 2023-11-08T16:34:55.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.113+0000 7fea66d92640 1 --2- 172.21.15.81:0/126964594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea60165a80 0x7fea601629e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fea60071ae0 tx=0x7fea5802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3679ee9e24872db server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:55.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.113+0000 7fea6558f640 1 -- 172.21.15.81:0/126964594 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5802cc80 con 0x7fea60165a80 2023-11-08T16:34:55.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.113+0000 7fea6558f640 1 -- 172.21.15.81:0/126964594 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5802cde0 con 0x7fea60165a80 2023-11-08T16:34:55.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.113+0000 7fea6558f640 1 -- 172.21.15.81:0/126964594 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea58034640 con 0x7fea60165a80 2023-11-08T16:34:55.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.113+0000 7fea67593640 1 -- 172.21.15.81:0/126964594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea60165a80 msgr2=0x7fea601629e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:55.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.113+0000 7fea67593640 1 --2- 172.21.15.81:0/126964594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea60165a80 0x7fea601629e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fea60071ae0 tx=0x7fea5802bed0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.113+0000 7fea67593640 1 -- 172.21.15.81:0/126964594 shutdown_connections 2023-11-08T16:34:55.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.113+0000 7fea67593640 1 --2- 172.21.15.81:0/126964594 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea60165a80 0x7fea601629e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.114+0000 7fea67593640 1 --2- 172.21.15.81:0/126964594 >> v2:172.21.15.93:3300/0 conn(0x7fea6006d5d0 0x7fea60165540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.114+0000 7fea67593640 1 -- 172.21.15.81:0/126964594 >> 172.21.15.81:0/126964594 conn(0x7fea6006a4a0 msgr2=0x7fea6006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:55.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.114+0000 7fea67593640 1 -- 172.21.15.81:0/126964594 shutdown_connections 2023-11-08T16:34:55.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.114+0000 7fea67593640 1 -- 172.21.15.81:0/126964594 wait complete. 2023-11-08T16:34:55.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.115+0000 7fea67593640 1 Processor -- start 2023-11-08T16:34:55.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.116+0000 7fea67593640 1 -- start start 2023-11-08T16:34:55.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.116+0000 7fea67593640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea6006b850 0x7fea601459f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:55.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea67593640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea6006d5d0 0x7fea600b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:55.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea67593640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea60165a80 0x7fea600b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:55.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea67593640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea60154cd0 con 0x7fea6006d5d0 2023-11-08T16:34:55.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea67593640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea60150850 con 0x7fea60165a80 2023-11-08T16:34:55.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea67593640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea60150aa0 con 0x7fea6006b850 2023-11-08T16:34:55.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea65d90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea6006d5d0 0x7fea600b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:55.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea65d90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea6006d5d0 0x7fea600b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55150/0 (socket says 172.21.15.81:55150) 2023-11-08T16:34:55.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea65d90640 1 -- 172.21.15.81:0/287467303 learned_addr learned my addr 172.21.15.81:0/287467303 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:55.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea65d90640 1 -- 172.21.15.81:0/287467303 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea6006b850 msgr2=0x7fea601459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:55.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea65d90640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea6006b850 0x7fea601459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea65d90640 1 -- 172.21.15.81:0/287467303 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea60165a80 msgr2=0x7fea600b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:55.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea65d90640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea60165a80 0x7fea600b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea65d90640 1 -- 172.21.15.81:0/287467303 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea58009a70 con 0x7fea6006d5d0 2023-11-08T16:34:55.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.117+0000 7fea65d90640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea6006d5d0 0x7fea600b3500 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fea5000d800 tx=0x7fea5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:55.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.118+0000 7fea56ffd640 1 -- 172.21.15.81:0/287467303 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5000b6c0 con 0x7fea6006d5d0 2023-11-08T16:34:55.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.118+0000 7fea56ffd640 1 -- 172.21.15.81:0/287467303 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5000b820 con 0x7fea6006d5d0 2023-11-08T16:34:55.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.118+0000 7fea56ffd640 1 -- 172.21.15.81:0/287467303 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5000bbb0 con 0x7fea6006d5d0 2023-11-08T16:34:55.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.118+0000 7fea67593640 1 -- 172.21.15.81:0/287467303 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea60150d80 con 0x7fea6006d5d0 2023-11-08T16:34:55.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.118+0000 7fea67593640 1 -- 172.21.15.81:0/287467303 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea60151340 con 0x7fea6006d5d0 2023-11-08T16:34:55.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.119+0000 7fea67593640 1 -- 172.21.15.81:0/287467303 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea60067800 con 0x7fea6006d5d0 2023-11-08T16:34:55.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.120+0000 7fea56ffd640 1 -- 172.21.15.81:0/287467303 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea5000bd10 con 0x7fea6006d5d0 2023-11-08T16:34:55.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.121+0000 7fea56ffd640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea4406aea0 0x7fea4406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:55.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.121+0000 7fea56ffd640 1 -- 172.21.15.81:0/287467303 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fea50089e30 con 0x7fea6006d5d0 2023-11-08T16:34:55.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.121+0000 7fea66591640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea4406aea0 0x7fea4406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:55.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.122+0000 7fea66591640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea4406aea0 0x7fea4406d360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000f60 tx=0x7fea5c0078f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:55.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.126+0000 7fea56ffd640 1 -- 172.21.15.81:0/287467303 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea50054bf0 con 0x7fea6006d5d0 2023-11-08T16:34:55.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.358+0000 7fea67593640 1 -- 172.21.15.81:0/287467303 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea600717a0 con 0x7fea6006d5d0 2023-11-08T16:34:55.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.359+0000 7fea56ffd640 1 -- 172.21.15.81:0/287467303 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fea50058210 con 0x7fea6006d5d0 2023-11-08T16:34:55.361 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:55.361 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":53,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3027678987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3027678987},{"type":"v1","addr":"172.21.15.93:6833","nonce":3027678987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/3140373039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3140373039},{"type":"v1","addr":"172.21.15.81:6836","nonce":3140373039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3145959397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3145959397},{"type":"v1","addr":"172.21.15.93:6835","nonce":3145959397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:54.196883+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":0,"incarnation":51,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.81:6837/552116415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":552116415},{"type":"v1","addr":"172.21.15.81:6837","nonce":552116415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:34:55.362 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 53 2023-11-08T16:34:55.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 -- 172.21.15.81:0/287467303 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea4406aea0 msgr2=0x7fea4406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:55.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea4406aea0 0x7fea4406d360 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fea5c000f60 tx=0x7fea5c0078f0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 -- 172.21.15.81:0/287467303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea6006d5d0 msgr2=0x7fea600b3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:55.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea6006d5d0 0x7fea600b3500 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fea5000d800 tx=0x7fea5000dcc0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 -- 172.21.15.81:0/287467303 shutdown_connections 2023-11-08T16:34:55.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea4406aea0 0x7fea4406d360 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea60165a80 0x7fea600b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea6006d5d0 0x7fea600b3500 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 --2- 172.21.15.81:0/287467303 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea6006b850 0x7fea601459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:55.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 -- 172.21.15.81:0/287467303 >> 172.21.15.81:0/287467303 conn(0x7fea6006a4a0 msgr2=0x7fea60162980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:55.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 -- 172.21.15.81:0/287467303 shutdown_connections 2023-11-08T16:34:55.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:55.362+0000 7fea67593640 1 -- 172.21.15.81:0/287467303 wait complete. 2023-11-08T16:34:55.785 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:34:44.993932+0000', 'modified': '2023-11-08T16:34:54.196883+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6600}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6600': {'gid': 6600, 'name': 'c', 'rank': 0, 'incarnation': 51, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.81:6837/552116415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6835', 'nonce': 552116415}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 552116415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8, 9], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:34:55.785 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6600:up:rejoin 2023-11-08T16:34:56.787 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:34:57.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.007+0000 7f4769a1a640 1 Processor -- start 2023-11-08T16:34:57.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.007+0000 7f4769a1a640 1 -- start start 2023-11-08T16:34:57.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.009+0000 7f4769a1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f476415aac0 0x7f47640711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.009+0000 7f4769a1a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f47640cf420 0x7f476415aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.009+0000 7f4769a1a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4764070470 con 0x7f47640716e0 2023-11-08T16:34:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.009+0000 7f4769a1a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f476406f650 con 0x7f47640cf420 2023-11-08T16:34:57.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.009+0000 7f4769a1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f476406d580 con 0x7f476415aac0 2023-11-08T16:34:57.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.010+0000 7f4768a18640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f476415aac0 0x7f47640711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.010+0000 7f4769219640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f47640cf420 0x7f476415aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.010+0000 7f4763fff640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f47640716e0 0x7f47640ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:44336/0 (socket says 172.21.15.81:44336) 2023-11-08T16:34:57.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.010+0000 7f4763fff640 1 -- 172.21.15.81:0/388267643 learned_addr learned my addr 172.21.15.81:0/388267643 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:57.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.010+0000 7f4768a18640 1 -- 172.21.15.81:0/388267643 >> v2:172.21.15.93:3300/0 conn(0x7f47640cf420 msgr2=0x7f476415aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:57.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.010+0000 7f4768a18640 1 --2- 172.21.15.81:0/388267643 >> v2:172.21.15.93:3300/0 conn(0x7f47640cf420 0x7f476415aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.010+0000 7f4768a18640 1 -- 172.21.15.81:0/388267643 >> v1:172.21.15.93:6789/0 conn(0x7f47640716e0 legacy=0x7f47640ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.011+0000 7f4768a18640 1 -- 172.21.15.81:0/388267643 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476406ca00 con 0x7f476415aac0 2023-11-08T16:34:57.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.011+0000 7f4769219640 1 --2- 172.21.15.81:0/388267643 >> v2:172.21.15.93:3300/0 conn(0x7f47640cf420 0x7f476415aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:34:57.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.012+0000 7f4768a18640 1 --2- 172.21.15.81:0/388267643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f476415aac0 0x7f47640711a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47580097a0 tx=0x7f475802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=926a79f34f313c9e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:57.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.012+0000 7f47637fe640 1 -- 172.21.15.81:0/388267643 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475802d040 con 0x7f476415aac0 2023-11-08T16:34:57.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.012+0000 7f47637fe640 1 -- 172.21.15.81:0/388267643 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4758031810 con 0x7f476415aac0 2023-11-08T16:34:57.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.012+0000 7f47637fe640 1 -- 172.21.15.81:0/388267643 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758031ba0 con 0x7f476415aac0 2023-11-08T16:34:57.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.013+0000 7f4769a1a640 1 -- 172.21.15.81:0/388267643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f476415aac0 msgr2=0x7f47640711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.013+0000 7f4769a1a640 1 --2- 172.21.15.81:0/388267643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f476415aac0 0x7f47640711a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47580097a0 tx=0x7f475802b8b0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.013+0000 7f4769a1a640 1 -- 172.21.15.81:0/388267643 shutdown_connections 2023-11-08T16:34:57.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.013+0000 7f4769a1a640 1 --2- 172.21.15.81:0/388267643 >> v2:172.21.15.93:3300/0 conn(0x7f47640cf420 0x7f476415aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.013+0000 7f4769a1a640 1 --2- 172.21.15.81:0/388267643 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f476415aac0 0x7f47640711a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.013+0000 7f4769a1a640 1 -- 172.21.15.81:0/388267643 >> 172.21.15.81:0/388267643 conn(0x7f476406a4a0 msgr2=0x7f476406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:57.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.013+0000 7f4769a1a640 1 -- 172.21.15.81:0/388267643 shutdown_connections 2023-11-08T16:34:57.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.013+0000 7f4769a1a640 1 -- 172.21.15.81:0/388267643 wait complete. 2023-11-08T16:34:57.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.015+0000 7f4769a1a640 1 Processor -- start 2023-11-08T16:34:57.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.015+0000 7f4769a1a640 1 -- start start 2023-11-08T16:34:57.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.016+0000 7f4769a1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47640716e0 0x7f4764156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.016+0000 7f4769a1a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47640cf420 0x7f47641e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4769a1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f476415aac0 0x7f47641e5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4769a1a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47640ae850 con 0x7f47640716e0 2023-11-08T16:34:57.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4769a1a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47640ae500 con 0x7f47640cf420 2023-11-08T16:34:57.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4769a1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47641e8810 con 0x7f476415aac0 2023-11-08T16:34:57.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4768a18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47640716e0 0x7f4764156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4768a18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47640716e0 0x7f4764156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55158/0 (socket says 172.21.15.81:55158) 2023-11-08T16:34:57.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4768a18640 1 -- 172.21.15.81:0/2388655227 learned_addr learned my addr 172.21.15.81:0/2388655227 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:57.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4763fff640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47640cf420 0x7f47641e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4769219640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f476415aac0 0x7f47641e5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4768a18640 1 -- 172.21.15.81:0/2388655227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f476415aac0 msgr2=0x7f47641e5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4768a18640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f476415aac0 0x7f47641e5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4768a18640 1 -- 172.21.15.81:0/2388655227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47640cf420 msgr2=0x7f47641e7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4768a18640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47640cf420 0x7f47641e7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.017+0000 7f4768a18640 1 -- 172.21.15.81:0/2388655227 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f475c009a70 con 0x7f47640716e0 2023-11-08T16:34:57.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.018+0000 7f4768a18640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47640716e0 0x7f4764156430 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4758000c00 tx=0x7f47580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:57.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.018+0000 7f47617fa640 1 -- 172.21.15.81:0/2388655227 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758006e10 con 0x7f47640716e0 2023-11-08T16:34:57.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.018+0000 7f47617fa640 1 -- 172.21.15.81:0/2388655227 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475803ad60 con 0x7f47640716e0 2023-11-08T16:34:57.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.018+0000 7f4769a1a640 1 -- 172.21.15.81:0/2388655227 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4758009400 con 0x7f47640716e0 2023-11-08T16:34:57.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.018+0000 7f47617fa640 1 -- 172.21.15.81:0/2388655227 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4758044630 con 0x7f47640716e0 2023-11-08T16:34:57.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.018+0000 7f4769a1a640 1 -- 172.21.15.81:0/2388655227 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47641e8dd0 con 0x7f47640716e0 2023-11-08T16:34:57.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.019+0000 7f4769a1a640 1 -- 172.21.15.81:0/2388655227 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4764067800 con 0x7f47640716e0 2023-11-08T16:34:57.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.020+0000 7f47617fa640 1 -- 172.21.15.81:0/2388655227 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f475804d450 con 0x7f47640716e0 2023-11-08T16:34:57.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.021+0000 7f47617fa640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f474806aea0 0x7f474806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.021+0000 7f47617fa640 1 -- 172.21.15.81:0/2388655227 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f47580aba60 con 0x7f47640716e0 2023-11-08T16:34:57.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.022+0000 7f4763fff640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f474806aea0 0x7f474806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.022+0000 7f4763fff640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f474806aea0 0x7f474806d360 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f47640b1ef0 tx=0x7f4750009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:57.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.026+0000 7f47617fa640 1 -- 172.21.15.81:0/2388655227 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4758076810 con 0x7f47640716e0 2023-11-08T16:34:57.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.253+0000 7f4769a1a640 1 -- 172.21.15.81:0/2388655227 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f476415e100 con 0x7f47640716e0 2023-11-08T16:34:57.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.254+0000 7f47617fa640 1 -- 172.21.15.81:0/2388655227 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f4758035090 con 0x7f47640716e0 2023-11-08T16:34:57.257 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:57.257 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3027678987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3027678987},{"type":"v1","addr":"172.21.15.93:6833","nonce":3027678987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/3140373039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3140373039},{"type":"v1","addr":"172.21.15.81:6836","nonce":3140373039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3145959397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3145959397},{"type":"v1","addr":"172.21.15.93:6835","nonce":3145959397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:55.206262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/552116415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":552116415},{"type":"v1","addr":"172.21.15.81:6837","nonce":552116415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:34:57.257 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 54 2023-11-08T16:34:57.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 -- 172.21.15.81:0/2388655227 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f474806aea0 msgr2=0x7f474806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f474806aea0 0x7f474806d360 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f47640b1ef0 tx=0x7f4750009040 comp rx=0 tx=0).stop 2023-11-08T16:34:57.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 -- 172.21.15.81:0/2388655227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47640716e0 msgr2=0x7f4764156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47640716e0 0x7f4764156430 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4758000c00 tx=0x7f47580046c0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 -- 172.21.15.81:0/2388655227 shutdown_connections 2023-11-08T16:34:57.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f474806aea0 0x7f474806d360 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f476415aac0 0x7f47641e5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47640cf420 0x7f47641e7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 --2- 172.21.15.81:0/2388655227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47640716e0 0x7f4764156430 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 -- 172.21.15.81:0/2388655227 >> 172.21.15.81:0/2388655227 conn(0x7f476406a4a0 msgr2=0x7f47640cdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:57.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 -- 172.21.15.81:0/2388655227 shutdown_connections 2023-11-08T16:34:57.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.257+0000 7f4769a1a640 1 -- 172.21.15.81:0/2388655227 wait complete. 2023-11-08T16:34:57.678 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:34:44.993932+0000', 'modified': '2023-11-08T16:34:55.206262+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 65, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6600}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6600': {'gid': 6600, 'name': 'c', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6837/552116415', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6835', 'nonce': 552116415}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 552116415}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8, 9], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:34:57.679 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:34:57.679 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T16:34:57.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.904+0000 7f0560bfc640 1 Processor -- start 2023-11-08T16:34:57.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.905+0000 7f0560bfc640 1 -- start start 2023-11-08T16:34:57.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.906+0000 7f0560bfc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f055c06cf10 0x7f055c161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.906+0000 7f0560bfc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f055c0ca1a0 0x7f055c0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.906+0000 7f0560bfc640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f055c0cd580 con 0x7f055c0cc670 2023-11-08T16:34:57.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.906+0000 7f0560bfc640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f055c0c93e0 con 0x7f055c06cf10 2023-11-08T16:34:57.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.906+0000 7f0560bfc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c0c96c0 con 0x7f055c0ca1a0 2023-11-08T16:34:57.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.906+0000 7f055a59c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f055c06cf10 0x7f055c161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.907+0000 7f055a59c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f055c06cf10 0x7f055c161630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55168/0 (socket says 172.21.15.81:55168) 2023-11-08T16:34:57.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.907+0000 7f055a59c640 1 -- 172.21.15.81:0/1450613350 learned_addr learned my addr 172.21.15.81:0/1450613350 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:57.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.907+0000 7f0559d9b640 1 --2- 172.21.15.81:0/1450613350 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f055c0ca1a0 0x7f055c0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.907+0000 7f055a59c640 1 -- 172.21.15.81:0/1450613350 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f055c0ca1a0 msgr2=0x7f055c0caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:57.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.907+0000 7f055a59c640 1 --2- 172.21.15.81:0/1450613350 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f055c0ca1a0 0x7f055c0caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.907+0000 7f055a59c640 1 -- 172.21.15.81:0/1450613350 >> v1:172.21.15.93:6789/0 conn(0x7f055c0cc670 legacy=0x7f055c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:57.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.907+0000 7f055a59c640 1 -- 172.21.15.81:0/1450613350 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c0c99a0 con 0x7f055c06cf10 2023-11-08T16:34:57.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.908+0000 7f055a59c640 1 --2- 172.21.15.81:0/1450613350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f055c06cf10 0x7f055c161630 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f054c009ea0 tx=0x7f054c02bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f745bf462b27d28 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:57.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.908+0000 7f055959a640 1 -- 172.21.15.81:0/1450613350 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c02cab0 con 0x7f055c06cf10 2023-11-08T16:34:57.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.908+0000 7f055959a640 1 -- 172.21.15.81:0/1450613350 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c02cc10 con 0x7f055c06cf10 2023-11-08T16:34:57.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.908+0000 7f055959a640 1 -- 172.21.15.81:0/1450613350 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c0343f0 con 0x7f055c06cf10 2023-11-08T16:34:57.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.909+0000 7f0560bfc640 1 -- 172.21.15.81:0/1450613350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f055c06cf10 msgr2=0x7f055c161630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.909+0000 7f0560bfc640 1 --2- 172.21.15.81:0/1450613350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f055c06cf10 0x7f055c161630 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f054c009ea0 tx=0x7f054c02bf10 comp rx=0 tx=0).stop 2023-11-08T16:34:57.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.909+0000 7f0560bfc640 1 -- 172.21.15.81:0/1450613350 shutdown_connections 2023-11-08T16:34:57.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.909+0000 7f0560bfc640 1 --2- 172.21.15.81:0/1450613350 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f055c0ca1a0 0x7f055c0caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.909+0000 7f0560bfc640 1 --2- 172.21.15.81:0/1450613350 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f055c06cf10 0x7f055c161630 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.909+0000 7f0560bfc640 1 -- 172.21.15.81:0/1450613350 >> 172.21.15.81:0/1450613350 conn(0x7f055c0ceaa0 msgr2=0x7f055c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:57.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.909+0000 7f0560bfc640 1 -- 172.21.15.81:0/1450613350 shutdown_connections 2023-11-08T16:34:57.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.909+0000 7f0560bfc640 1 -- 172.21.15.81:0/1450613350 wait complete. 2023-11-08T16:34:57.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.911+0000 7f0560bfc640 1 Processor -- start 2023-11-08T16:34:57.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.911+0000 7f0560bfc640 1 -- start start 2023-11-08T16:34:57.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.912+0000 7f0560bfc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f055c06cf10 0x7f055c14cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.912+0000 7f0560bfc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f055c0ca1a0 0x7f055c0c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.912+0000 7f0560bfc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f055c0cc670 0x7f055c0c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.912+0000 7f0560bfc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c14d010 con 0x7f055c0ca1a0 2023-11-08T16:34:57.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.912+0000 7f0560bfc640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f055c14d280 con 0x7f055c0cc670 2023-11-08T16:34:57.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.912+0000 7f0560bfc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055c14d4d0 con 0x7f055c06cf10 2023-11-08T16:34:57.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.912+0000 7f0559d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f055c0ca1a0 0x7f055c0c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.912+0000 7f0559d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f055c0ca1a0 0x7f055c0c25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55174/0 (socket says 172.21.15.81:55174) 2023-11-08T16:34:57.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.912+0000 7f0559d9b640 1 -- 172.21.15.81:0/372662653 learned_addr learned my addr 172.21.15.81:0/372662653 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:57.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f055ad9d640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f055c0cc670 0x7f055c0c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f0559d9b640 1 -- 172.21.15.81:0/372662653 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f055c06cf10 msgr2=0x7f055c14cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f0559d9b640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f055c06cf10 0x7f055c14cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f0559d9b640 1 -- 172.21.15.81:0/372662653 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f055c0cc670 msgr2=0x7f055c0c7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f0559d9b640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f055c0cc670 0x7f055c0c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f0559d9b640 1 -- 172.21.15.81:0/372662653 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c009b30 con 0x7f055c0ca1a0 2023-11-08T16:34:57.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f0559d9b640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f055c0ca1a0 0x7f055c0c25f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f054400d5f0 tx=0x7f054400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:57.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f0552ffd640 1 -- 172.21.15.81:0/372662653 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05440125f0 con 0x7f055c0ca1a0 2023-11-08T16:34:57.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f0552ffd640 1 -- 172.21.15.81:0/372662653 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0544012750 con 0x7f055c0ca1a0 2023-11-08T16:34:57.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f0552ffd640 1 -- 172.21.15.81:0/372662653 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544012b00 con 0x7f055c0ca1a0 2023-11-08T16:34:57.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f0560bfc640 1 -- 172.21.15.81:0/372662653 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f055c14d7b0 con 0x7f055c0ca1a0 2023-11-08T16:34:57.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.913+0000 7f0560bfc640 1 -- 172.21.15.81:0/372662653 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c14dd70 con 0x7f055c0ca1a0 2023-11-08T16:34:57.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.915+0000 7f0552ffd640 1 -- 172.21.15.81:0/372662653 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0544012c60 con 0x7f055c0ca1a0 2023-11-08T16:34:57.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.916+0000 7f0552ffd640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f053c06ae70 0x7f053c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.916+0000 7f0552ffd640 1 -- 172.21.15.81:0/372662653 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f0544089db0 con 0x7f055c0ca1a0 2023-11-08T16:34:57.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.917+0000 7f055a59c640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f053c06ae70 0x7f053c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.917+0000 7f056378a740 1 Processor -- start 2023-11-08T16:34:57.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.917+0000 7f055a59c640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f053c06ae70 0x7f053c06d330 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f054c035000 tx=0x7f054c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:57.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.917+0000 7f056378a740 1 -- start start 2023-11-08T16:34:57.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.918+0000 7f056378a740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa69ec0 0x557c9fa3cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.918+0000 7f056378a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa6a2a0 0x557c9fa6cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.918+0000 7f056378a740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa3d300 0x557c9fa6d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.918+0000 7f056378a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9f9af7e0 con 0x557c9fa6a2a0 2023-11-08T16:34:57.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.918+0000 7f056378a740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9f9aced0 con 0x557c9fa69ec0 2023-11-08T16:34:57.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.918+0000 7f056378a740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c9f9a87c0 con 0x557c9fa3d300 2023-11-08T16:34:57.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.918+0000 7f0559d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa6a2a0 0x557c9fa6cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f0559d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa6a2a0 0x557c9fa6cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55186/0 (socket says 172.21.15.81:55186) 2023-11-08T16:34:57.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f0559d9b640 1 -- 172.21.15.81:0/3483679554 learned_addr learned my addr 172.21.15.81:0/3483679554 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:57.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f055ad9d640 1 --2- 172.21.15.81:0/3483679554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa3d300 0x557c9fa6d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f0559d9b640 1 -- 172.21.15.81:0/3483679554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa3d300 msgr2=0x557c9fa6d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:57.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f055a59c640 1 --2- 172.21.15.81:0/3483679554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa69ec0 0x557c9fa3cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f0559d9b640 1 --2- 172.21.15.81:0/3483679554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa3d300 0x557c9fa6d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f0559d9b640 1 -- 172.21.15.81:0/3483679554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa69ec0 msgr2=0x557c9fa3cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:57.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f0559d9b640 1 --2- 172.21.15.81:0/3483679554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa69ec0 0x557c9fa3cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f0559d9b640 1 -- 172.21.15.81:0/3483679554 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c9f97bb10 con 0x557c9fa6a2a0 2023-11-08T16:34:57.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f055ad9d640 1 --2- 172.21.15.81:0/3483679554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa3d300 0x557c9fa6d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:57.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f0559d9b640 1 --2- 172.21.15.81:0/3483679554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa6a2a0 0x557c9fa6cc50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0544060600 tx=0x7f0544058f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=506a351197dca4a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:57.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f053bfff640 1 -- 172.21.15.81:0/3483679554 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054405c980 con 0x557c9fa6a2a0 2023-11-08T16:34:57.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f053bfff640 1 -- 172.21.15.81:0/3483679554 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0544060b30 con 0x557c9fa6a2a0 2023-11-08T16:34:57.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.919+0000 7f053bfff640 1 -- 172.21.15.81:0/3483679554 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544054f40 con 0x557c9fa6a2a0 2023-11-08T16:34:57.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.920+0000 7f056378a740 1 -- 172.21.15.81:0/3483679554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa6a2a0 msgr2=0x557c9fa6cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.920+0000 7f056378a740 1 --2- 172.21.15.81:0/3483679554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa6a2a0 0x557c9fa6cc50 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f0544060600 tx=0x7f0544058f30 comp rx=0 tx=0).stop 2023-11-08T16:34:57.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.920+0000 7f056378a740 1 -- 172.21.15.81:0/3483679554 shutdown_connections 2023-11-08T16:34:57.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.920+0000 7f056378a740 1 --2- 172.21.15.81:0/3483679554 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa3d300 0x557c9fa6d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.920+0000 7f056378a740 1 --2- 172.21.15.81:0/3483679554 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa6a2a0 0x557c9fa6cc50 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.920+0000 7f056378a740 1 --2- 172.21.15.81:0/3483679554 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa69ec0 0x557c9fa3cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.920+0000 7f056378a740 1 -- 172.21.15.81:0/3483679554 >> 172.21.15.81:0/3483679554 conn(0x557c9fa3f0d0 msgr2=0x557c9fa87c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:57.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.920+0000 7f056378a740 1 -- 172.21.15.81:0/3483679554 shutdown_connections 2023-11-08T16:34:57.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.920+0000 7f056378a740 1 -- 172.21.15.81:0/3483679554 wait complete. 2023-11-08T16:34:57.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.921+0000 7f056378a740 1 -- start start 2023-11-08T16:34:57.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.921+0000 7f056378a740 1 Processor -- start 2023-11-08T16:34:57.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.922+0000 7f056378a740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa69ec0 0x557c9fa5b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.922+0000 7f056378a740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa6a2a0 0x557c9fa81a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.922+0000 7f056378a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa82280 0x557c9fa5bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.922+0000 7f056378a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9f945c40 con 0x557c9fa82280 2023-11-08T16:34:57.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.922+0000 7f056378a740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9f9458e0 con 0x557c9fa69ec0 2023-11-08T16:34:57.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.922+0000 7f056378a740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c9f70bf20 con 0x557c9fa6a2a0 2023-11-08T16:34:57.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.922+0000 7f056378a740 10 client.0 _ll_register_callbacks cb 0x557c9f967cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:34:57.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.922+0000 7f055ad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa82280 0x557c9fa5bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f055ad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa82280 0x557c9fa5bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55198/0 (socket says 172.21.15.81:55198) 2023-11-08T16:34:57.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f055ad9d640 1 -- 172.21.15.81:0/1722278385 learned_addr learned my addr 172.21.15.81:0/1722278385 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:57.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f0559d9b640 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa6a2a0 0x557c9fa81a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f055a59c640 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa69ec0 0x557c9fa5b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f055ad9d640 1 -- 172.21.15.81:0/1722278385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa6a2a0 msgr2=0x557c9fa81a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:57.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f055ad9d640 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa6a2a0 0x557c9fa81a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f055ad9d640 1 -- 172.21.15.81:0/1722278385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa69ec0 msgr2=0x557c9fa5b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:57.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f055ad9d640 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa69ec0 0x557c9fa5b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f055ad9d640 1 -- 172.21.15.81:0/1722278385 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c14dd70 con 0x557c9fa82280 2023-11-08T16:34:57.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f055a59c640 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa69ec0 0x557c9fa5b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:57.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f055ad9d640 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa82280 0x557c9fa5bbd0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0554004490 tx=0x7f055400e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1144288dde74b669 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:57.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f05397fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:34:57.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.923+0000 7f05397fa640 1 -- 172.21.15.81:0/1722278385 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0554005230 con 0x557c9fa82280 2023-11-08T16:34:57.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f05397fa640 1 -- 172.21.15.81:0/1722278385 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f0554005390 con 0x557c9fa82280 2023-11-08T16:34:57.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f056378a740 1 -- 172.21.15.81:0/1722278385 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c9f97cc80 con 0x557c9fa82280 2023-11-08T16:34:57.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f05397fa640 1 -- 172.21.15.81:0/1722278385 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f05540045e0 con 0x557c9fa82280 2023-11-08T16:34:57.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f056378a740 10 client.6633 fetch_fsmap learned FSMap version 54 2023-11-08T16:34:57.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f056378a740 1 -- 172.21.15.81:0/1722278385 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x557c9f9a6e40 con 0x557c9fa82280 2023-11-08T16:34:57.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f05397fa640 1 -- 172.21.15.81:0/1722278385 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557c9f97cc80 con 0x557c9fa82280 2023-11-08T16:34:57.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f05397fa640 1 -- 172.21.15.81:0/1722278385 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f055401a060 con 0x557c9fa82280 2023-11-08T16:34:57.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f056378a740 10 client.6633 fetch_fsmap finished waiting for FSMap version 54 2023-11-08T16:34:57.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f056378a740 10 client.6633 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:34:57.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f056378a740 20 client.6633 populate_metadata read hostname 'smithi081' 2023-11-08T16:34:57.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f056378a740 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x557c9faa4540 0x557c9faa6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f056378a740 4 client.6633 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:34:57.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f056378a740 1 -- 172.21.15.81:0/1722278385 --> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c9f9a17a0 con 0x557c9faa4540 2023-11-08T16:34:57.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.924+0000 7f055a59c640 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x557c9faa4540 0x557c9faa6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.925+0000 7f05397fa640 10 client.6633 ms_handle_connect on v2:172.21.15.81:6835/552116415 2023-11-08T16:34:57.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.925+0000 7f055a59c640 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x557c9faa4540 0x557c9faa6930 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6102968bddca9293 server_cookie=fa20de0b9f24262d in_seq=0 out_seq=0 2023-11-08T16:34:57.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f05397fa640 1 -- 172.21.15.81:0/1722278385 <== mds.0 v2:172.21.15.81:6835/552116415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c9f9a17a0 con 0x557c9faa4540 2023-11-08T16:34:57.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f05397fa640 10 client.6633 handle_command_reply: tid=0 2023-11-08T16:34:57.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f056378a740 1 client.6633 shutdown 2023-11-08T16:34:57.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f056378a740 1 -- 172.21.15.81:0/1722278385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa82280 msgr2=0x557c9fa5bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f056378a740 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa82280 0x557c9fa5bbd0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0554004490 tx=0x7f055400e590 comp rx=0 tx=0).stop 2023-11-08T16:34:57.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f056378a740 1 -- 172.21.15.81:0/1722278385 shutdown_connections 2023-11-08T16:34:57.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f056378a740 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x557c9faa4540 0x557c9faa6930 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f056378a740 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa82280 0x557c9fa5bbd0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f056378a740 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa6a2a0 0x557c9fa81a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f05397fa640 0 client.6633 ms_handle_reset on v2:172.21.15.81:6835/552116415 2023-11-08T16:34:57.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f056378a740 1 --2- 172.21.15.81:0/1722278385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa69ec0 0x557c9fa5b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f056378a740 1 -- 172.21.15.81:0/1722278385 >> 172.21.15.81:0/1722278385 conn(0x557c9fa3d300 msgr2=0x557c9fa40d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:57.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f056378a740 1 -- 172.21.15.81:0/1722278385 shutdown_connections 2023-11-08T16:34:57.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.926+0000 7f056378a740 1 -- 172.21.15.81:0/1722278385 wait complete. 2023-11-08T16:34:57.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.927+0000 7f056378a740 20 client.6633 trim_cache size 0 max 16384 2023-11-08T16:34:57.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.943+0000 7f056378a740 1 Processor -- start 2023-11-08T16:34:57.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.944+0000 7f056378a740 1 -- start start 2023-11-08T16:34:57.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.944+0000 7f056378a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa83480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.944+0000 7f056378a740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa6a2a0 0x557c9fa6ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.944+0000 7f056378a740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa82280 0x557c9fa408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.944+0000 7f056378a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9fa387e0 con 0x557c9fa69ec0 2023-11-08T16:34:57.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.944+0000 7f056378a740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9fa7eb80 con 0x557c9fa82280 2023-11-08T16:34:57.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.944+0000 7f056378a740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c9fa82810 con 0x557c9fa6a2a0 2023-11-08T16:34:57.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.945+0000 7f055a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa83480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.945+0000 7f055a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa83480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55202/0 (socket says 172.21.15.81:55202) 2023-11-08T16:34:57.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.945+0000 7f055a59c640 1 -- 172.21.15.81:0/3641719829 learned_addr learned my addr 172.21.15.81:0/3641719829 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:57.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.945+0000 7f0559d9b640 1 --2- 172.21.15.81:0/3641719829 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa6a2a0 0x557c9fa6ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.945+0000 7f055a59c640 1 -- 172.21.15.81:0/3641719829 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa6a2a0 msgr2=0x557c9fa6ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:57.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.945+0000 7f055a59c640 1 --2- 172.21.15.81:0/3641719829 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa6a2a0 0x557c9fa6ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.945+0000 7f055a59c640 1 -- 172.21.15.81:0/3641719829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa82280 msgr2=0x557c9fa408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:57.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.945+0000 7f055ad9d640 1 --2- 172.21.15.81:0/3641719829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa82280 0x557c9fa408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.945+0000 7f055a59c640 1 --2- 172.21.15.81:0/3641719829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa82280 0x557c9fa408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.945+0000 7f055a59c640 1 -- 172.21.15.81:0/3641719829 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c9f944b90 con 0x557c9fa69ec0 2023-11-08T16:34:57.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.945+0000 7f055ad9d640 1 --2- 172.21.15.81:0/3641719829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa82280 0x557c9fa408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:57.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.946+0000 7f055a59c640 1 --2- 172.21.15.81:0/3641719829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa83480 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f054c0407d0 tx=0x7f054c009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2cc4dc91adf0b48 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:57.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.946+0000 7f0550ff9640 1 -- 172.21.15.81:0/3641719829 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c045040 con 0x557c9fa69ec0 2023-11-08T16:34:57.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.946+0000 7f0550ff9640 1 -- 172.21.15.81:0/3641719829 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c02c930 con 0x557c9fa69ec0 2023-11-08T16:34:57.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.946+0000 7f0550ff9640 1 -- 172.21.15.81:0/3641719829 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c04f0b0 con 0x557c9fa69ec0 2023-11-08T16:34:57.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.947+0000 7f056378a740 1 -- 172.21.15.81:0/3641719829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 msgr2=0x557c9fa83480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.947+0000 7f056378a740 1 --2- 172.21.15.81:0/3641719829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa83480 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f054c0407d0 tx=0x7f054c009000 comp rx=0 tx=0).stop 2023-11-08T16:34:57.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.947+0000 7f056378a740 1 -- 172.21.15.81:0/3641719829 shutdown_connections 2023-11-08T16:34:57.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.947+0000 7f056378a740 1 --2- 172.21.15.81:0/3641719829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa82280 0x557c9fa408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.947+0000 7f056378a740 1 --2- 172.21.15.81:0/3641719829 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa6a2a0 0x557c9fa6ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.947+0000 7f056378a740 1 --2- 172.21.15.81:0/3641719829 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa83480 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.947+0000 7f056378a740 1 -- 172.21.15.81:0/3641719829 >> 172.21.15.81:0/3641719829 conn(0x557c9fa3d300 msgr2=0x557c9fa3bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:57.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.947+0000 7f056378a740 1 -- 172.21.15.81:0/3641719829 shutdown_connections 2023-11-08T16:34:57.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.947+0000 7f056378a740 1 -- 172.21.15.81:0/3641719829 wait complete. 2023-11-08T16:34:57.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.947+0000 7f056378a740 1 -- start start 2023-11-08T16:34:57.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.948+0000 7f056378a740 1 Processor -- start 2023-11-08T16:34:57.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f056378a740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa40810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f056378a740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa6a2a0 0x557c9fa40d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f056378a740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa82280 0x557c9fa419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f056378a740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9fa6d2b0 con 0x557c9fa69ec0 2023-11-08T16:34:57.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f056378a740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c9fa6de40 con 0x557c9fa6a2a0 2023-11-08T16:34:57.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f056378a740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c9fa6d060 con 0x557c9fa82280 2023-11-08T16:34:57.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f056378a740 10 client.0 _ll_register_callbacks cb 0x7f0554007c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:34:57.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f055a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa40810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f055a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa40810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55212/0 (socket says 172.21.15.81:55212) 2023-11-08T16:34:57.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f055a59c640 1 -- 172.21.15.81:0/3009712248 learned_addr learned my addr 172.21.15.81:0/3009712248 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:57.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f0559d9b640 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa6a2a0 0x557c9fa40d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f055a59c640 1 -- 172.21.15.81:0/3009712248 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa82280 msgr2=0x557c9fa419c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:57.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f055a59c640 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa82280 0x557c9fa419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f055a59c640 1 -- 172.21.15.81:0/3009712248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa6a2a0 msgr2=0x557c9fa40d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:57.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f055a59c640 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa6a2a0 0x557c9fa40d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.949+0000 7f055a59c640 1 -- 172.21.15.81:0/3009712248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f055c0c99a0 con 0x557c9fa69ec0 2023-11-08T16:34:57.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.950+0000 7f055a59c640 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa40810 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f054c0407d0 tx=0x7f054c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a56f8e41e813c39 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:57.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.950+0000 7f05397fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:34:57.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.950+0000 7f05397fa640 1 -- 172.21.15.81:0/3009712248 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c04f2c0 con 0x557c9fa69ec0 2023-11-08T16:34:57.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.950+0000 7f05397fa640 1 -- 172.21.15.81:0/3009712248 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f054c045070 con 0x557c9fa69ec0 2023-11-08T16:34:57.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.950+0000 7f056378a740 1 -- 172.21.15.81:0/3009712248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0544060b30 con 0x557c9fa69ec0 2023-11-08T16:34:57.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.950+0000 7f05397fa640 1 -- 172.21.15.81:0/3009712248 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f054c04f420 con 0x557c9fa69ec0 2023-11-08T16:34:57.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.950+0000 7f056378a740 10 client.6639 fetch_fsmap learned FSMap version 54 2023-11-08T16:34:57.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.950+0000 7f056378a740 1 -- 172.21.15.81:0/3009712248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x557c9f96a180 con 0x557c9fa69ec0 2023-11-08T16:34:57.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.951+0000 7f05397fa640 1 -- 172.21.15.81:0/3009712248 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0544060b30 con 0x557c9fa69ec0 2023-11-08T16:34:57.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.951+0000 7f05397fa640 1 -- 172.21.15.81:0/3009712248 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f054c03c070 con 0x557c9fa69ec0 2023-11-08T16:34:57.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.951+0000 7f056378a740 10 client.6639 fetch_fsmap finished waiting for FSMap version 54 2023-11-08T16:34:57.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.951+0000 7f056378a740 10 client.6639 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:34:57.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.951+0000 7f056378a740 20 client.6639 populate_metadata read hostname 'smithi081' 2023-11-08T16:34:57.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.951+0000 7f056378a740 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x557c9fa7fc40 0x557c9fa41f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:57.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.951+0000 7f056378a740 4 client.6639 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:34:57.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.951+0000 7f056378a740 1 -- 172.21.15.81:0/3009712248 --> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557c9f9a14c0 con 0x557c9fa7fc40 2023-11-08T16:34:57.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.951+0000 7f0559d9b640 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x557c9fa7fc40 0x557c9fa41f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:57.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.951+0000 7f0559d9b640 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x557c9fa7fc40 0x557c9fa41f00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8363a3c0f79b6b1b server_cookie=26921224e58296e9 in_seq=0 out_seq=0 2023-11-08T16:34:57.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.951+0000 7f05397fa640 10 client.6639 ms_handle_connect on v2:172.21.15.81:6835/552116415 2023-11-08T16:34:57.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f05397fa640 1 -- 172.21.15.81:0/3009712248 <== mds.0 v2:172.21.15.81:6835/552116415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557c9f9a14c0 con 0x557c9fa7fc40 2023-11-08T16:34:57.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f05397fa640 10 client.6639 handle_command_reply: tid=0 2023-11-08T16:34:57.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 1 client.6639 shutdown 2023-11-08T16:34:57.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 1 -- 172.21.15.81:0/3009712248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 msgr2=0x557c9fa40810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa40810 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f054c0407d0 tx=0x7f054c004ab0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 1 -- 172.21.15.81:0/3009712248 shutdown_connections 2023-11-08T16:34:57.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x557c9fa7fc40 0x557c9fa41f00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557c9fa82280 0x557c9fa419c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557c9fa6a2a0 0x557c9fa40d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 1 --2- 172.21.15.81:0/3009712248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557c9fa69ec0 0x557c9fa40810 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 1 -- 172.21.15.81:0/3009712248 >> 172.21.15.81:0/3009712248 conn(0x557c9fa3d300 msgr2=0x557c9fa97530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:57.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f05397fa640 0 client.6639 ms_handle_reset on v2:172.21.15.81:6835/552116415 2023-11-08T16:34:57.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 1 -- 172.21.15.81:0/3009712248 shutdown_connections 2023-11-08T16:34:57.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 1 -- 172.21.15.81:0/3009712248 wait complete. 2023-11-08T16:34:57.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.952+0000 7f056378a740 20 client.6639 trim_cache size 0 max 16384 2023-11-08T16:34:57.968 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:34:57.969 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:34:57.969 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:34:57.969 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 6600, 2023-11-08T16:34:57.969 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:34:57.969 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:34:57.970 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:34:57.970 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 4.7638818399999998, 2023-11-08T16:34:57.970 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 54, 2023-11-08T16:34:57.970 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 68, 2023-11-08T16:34:57.970 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 68, 2023-11-08T16:34:57.971 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 4.7814010089999996 2023-11-08T16:34:57.971 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:34:57.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 -- 172.21.15.81:0/372662653 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f053c06ae70 msgr2=0x7f053c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f053c06ae70 0x7f053c06d330 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f054c035000 tx=0x7f054c02e000 comp rx=0 tx=0).stop 2023-11-08T16:34:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 -- 172.21.15.81:0/372662653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f055c0ca1a0 msgr2=0x7f055c0c25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f055c0ca1a0 0x7f055c0c25f0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f054400d5f0 tx=0x7f054400dab0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 -- 172.21.15.81:0/372662653 shutdown_connections 2023-11-08T16:34:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f053c06ae70 0x7f053c06d330 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f055c0cc670 0x7f055c0c7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f055c0ca1a0 0x7f055c0c25f0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 --2- 172.21.15.81:0/372662653 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f055c06cf10 0x7f055c14cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:57.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 -- 172.21.15.81:0/372662653 >> 172.21.15.81:0/372662653 conn(0x7f055c0ceaa0 msgr2=0x7f055c0cac80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:57.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 -- 172.21.15.81:0/372662653 shutdown_connections 2023-11-08T16:34:57.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:57.953+0000 7f0560bfc640 1 -- 172.21.15.81:0/372662653 wait complete. 2023-11-08T16:34:57.975 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:34:58.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.172+0000 7fa97a0a9640 1 Processor -- start 2023-11-08T16:34:58.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.172+0000 7fa97a0a9640 1 -- start start 2023-11-08T16:34:58.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.174+0000 7fa97a0a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa9740ade50 0x7fa9740ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:58.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.174+0000 7fa97a0a9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa97406b630 con 0x7fa97415a680 2023-11-08T16:34:58.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.174+0000 7fa97a0a9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9740cabb0 con 0x7fa97415ea60 2023-11-08T16:34:58.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.174+0000 7fa97a0a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9740b1860 con 0x7fa9740ade50 2023-11-08T16:34:58.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.174+0000 7fa96bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa97415a680 0x7fa97415aae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46514/0 (socket says 172.21.15.81:46514) 2023-11-08T16:34:58.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.174+0000 7fa96bfff640 1 -- 172.21.15.81:0/2175032381 learned_addr learned my addr 172.21.15.81:0/2175032381 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:34:58.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.175+0000 7fa9737fe640 1 --2- 172.21.15.81:0/2175032381 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa9740ade50 0x7fa9740ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:58.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.175+0000 7fa972ffd640 1 -- 172.21.15.81:0/2175032381 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2232426172 0 0) 0x7fa97406b630 con 0x7fa97415a680 2023-11-08T16:34:58.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.175+0000 7fa972ffd640 1 -- 172.21.15.81:0/2175032381 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa960003520 con 0x7fa97415a680 2023-11-08T16:34:58.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.175+0000 7fa9737fe640 1 -- 172.21.15.81:0/2175032381 >> v1:172.21.15.93:6789/0 conn(0x7fa97415ea60 legacy=0x7fa97415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.175+0000 7fa9737fe640 1 -- 172.21.15.81:0/2175032381 >> v1:172.21.15.81:6789/0 conn(0x7fa97415a680 legacy=0x7fa97415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.175+0000 7fa9737fe640 1 -- 172.21.15.81:0/2175032381 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9740cae90 con 0x7fa9740ade50 2023-11-08T16:34:58.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.177+0000 7fa9737fe640 1 --2- 172.21.15.81:0/2175032381 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa9740ade50 0x7fa9740ca480 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa9640097d0 tx=0x7fa96402b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23ee58beda4fcab8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:58.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.177+0000 7fa972ffd640 1 -- 172.21.15.81:0/2175032381 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964030800 con 0x7fa9740ade50 2023-11-08T16:34:58.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.177+0000 7fa972ffd640 1 -- 172.21.15.81:0/2175032381 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa964030960 con 0x7fa9740ade50 2023-11-08T16:34:58.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.177+0000 7fa972ffd640 1 -- 172.21.15.81:0/2175032381 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa964030cd0 con 0x7fa9740ade50 2023-11-08T16:34:58.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.178+0000 7fa97a0a9640 1 -- 172.21.15.81:0/2175032381 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa9740ade50 msgr2=0x7fa9740ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.178+0000 7fa97a0a9640 1 --2- 172.21.15.81:0/2175032381 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa9740ade50 0x7fa9740ca480 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa9640097d0 tx=0x7fa96402b940 comp rx=0 tx=0).stop 2023-11-08T16:34:58.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.178+0000 7fa97a0a9640 1 -- 172.21.15.81:0/2175032381 shutdown_connections 2023-11-08T16:34:58.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.178+0000 7fa97a0a9640 1 --2- 172.21.15.81:0/2175032381 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa9740ade50 0x7fa9740ca480 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.178+0000 7fa97a0a9640 1 -- 172.21.15.81:0/2175032381 >> 172.21.15.81:0/2175032381 conn(0x7fa9740cb490 msgr2=0x7fa9740cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:58.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.178+0000 7fa97a0a9640 1 -- 172.21.15.81:0/2175032381 shutdown_connections 2023-11-08T16:34:58.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.178+0000 7fa97a0a9640 1 -- 172.21.15.81:0/2175032381 wait complete. 2023-11-08T16:34:58.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.180+0000 7fa97a0a9640 1 Processor -- start 2023-11-08T16:34:58.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.180+0000 7fa97a0a9640 1 -- start start 2023-11-08T16:34:58.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.181+0000 7fa97a0a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa9740ade50 0x7fa97414f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:58.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.181+0000 7fa97a0a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa97415a680 0x7fa9741f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:58.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.181+0000 7fa97a0a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97415ea60 0x7fa974153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:58.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.181+0000 7fa97a0a9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9740ccc90 con 0x7fa97415a680 2023-11-08T16:34:58.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.181+0000 7fa97a0a9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa974154f80 con 0x7fa97415ea60 2023-11-08T16:34:58.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.181+0000 7fa97a0a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9740cc940 con 0x7fa9740ade50 2023-11-08T16:34:58.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.181+0000 7fa96bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa97415a680 0x7fa9741f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:58.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.181+0000 7fa96bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa97415a680 0x7fa9741f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55216/0 (socket says 172.21.15.81:55216) 2023-11-08T16:34:58.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.181+0000 7fa96bfff640 1 -- 172.21.15.81:0/4114308391 learned_addr learned my addr 172.21.15.81:0/4114308391 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:58.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.182+0000 7fa9737fe640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa9740ade50 0x7fa97414f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:58.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.182+0000 7fa973fff640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97415ea60 0x7fa974153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:58.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.182+0000 7fa96bfff640 1 -- 172.21.15.81:0/4114308391 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa9740ade50 msgr2=0x7fa97414f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.182+0000 7fa96bfff640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa9740ade50 0x7fa97414f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.182+0000 7fa96bfff640 1 -- 172.21.15.81:0/4114308391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97415ea60 msgr2=0x7fa974153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.182+0000 7fa96bfff640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97415ea60 0x7fa974153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.182+0000 7fa96bfff640 1 -- 172.21.15.81:0/4114308391 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa964009400 con 0x7fa97415a680 2023-11-08T16:34:58.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.182+0000 7fa973fff640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97415ea60 0x7fa974153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:58.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.182+0000 7fa96bfff640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa97415a680 0x7fa9741f3700 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00c500 tx=0x7fa95c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:58.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.182+0000 7fa970ff9640 1 -- 172.21.15.81:0/4114308391 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c005ce0 con 0x7fa97415a680 2023-11-08T16:34:58.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.182+0000 7fa970ff9640 1 -- 172.21.15.81:0/4114308391 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95c005e40 con 0x7fa97415a680 2023-11-08T16:34:58.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.183+0000 7fa970ff9640 1 -- 172.21.15.81:0/4114308391 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95c007330 con 0x7fa97415a680 2023-11-08T16:34:58.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.183+0000 7fa97a0a9640 1 -- 172.21.15.81:0/4114308391 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa974154440 con 0x7fa97415a680 2023-11-08T16:34:58.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.183+0000 7fa97a0a9640 1 -- 172.21.15.81:0/4114308391 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9741549c0 con 0x7fa97415a680 2023-11-08T16:34:58.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.184+0000 7fa97a0a9640 1 -- 172.21.15.81:0/4114308391 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97404ece0 con 0x7fa97415a680 2023-11-08T16:34:58.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.185+0000 7fa970ff9640 1 -- 172.21.15.81:0/4114308391 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa95c007550 con 0x7fa97415a680 2023-11-08T16:34:58.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.186+0000 7fa970ff9640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa95406ae70 0x7fa95406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:58.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.186+0000 7fa970ff9640 1 -- 172.21.15.81:0/4114308391 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fa95c088f10 con 0x7fa97415a680 2023-11-08T16:34:58.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.186+0000 7fa9737fe640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa95406ae70 0x7fa95406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:58.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.187+0000 7fa9737fe640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa95406ae70 0x7fa95406d330 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa964036000 tx=0x7fa964008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:58.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.190+0000 7fa970ff9640 1 -- 172.21.15.81:0/4114308391 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa95c053e20 con 0x7fa97415a680 2023-11-08T16:34:58.198 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:34:58.196+0000 7f1a4368a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:34:58.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.418+0000 7fa97a0a9640 1 -- 172.21.15.81:0/4114308391 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9740cc1d0 con 0x7fa97415a680 2023-11-08T16:34:58.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.420+0000 7fa970ff9640 1 -- 172.21.15.81:0/4114308391 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fa95c017020 con 0x7fa97415a680 2023-11-08T16:34:58.422 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:34:58.422 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3027678987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3027678987},{"type":"v1","addr":"172.21.15.93:6833","nonce":3027678987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/3140373039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3140373039},{"type":"v1","addr":"172.21.15.81:6836","nonce":3140373039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3145959397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3145959397},{"type":"v1","addr":"172.21.15.93:6835","nonce":3145959397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:55.206262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/552116415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":552116415},{"type":"v1","addr":"172.21.15.81:6837","nonce":552116415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:34:58.422 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 54 2023-11-08T16:34:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.422+0000 7fa97a0a9640 1 -- 172.21.15.81:0/4114308391 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa95406ae70 msgr2=0x7fa95406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.422+0000 7fa97a0a9640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa95406ae70 0x7fa95406d330 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa964036000 tx=0x7fa964008030 comp rx=0 tx=0).stop 2023-11-08T16:34:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.422+0000 7fa97a0a9640 1 -- 172.21.15.81:0/4114308391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa97415a680 msgr2=0x7fa9741f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.422+0000 7fa97a0a9640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa97415a680 0x7fa9741f3700 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fa95c00c500 tx=0x7fa95c00c9c0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.423+0000 7fa97a0a9640 1 -- 172.21.15.81:0/4114308391 shutdown_connections 2023-11-08T16:34:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.423+0000 7fa97a0a9640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa95406ae70 0x7fa95406d330 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.423+0000 7fa97a0a9640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa97415ea60 0x7fa974153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.423+0000 7fa97a0a9640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa97415a680 0x7fa9741f3700 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.423+0000 7fa97a0a9640 1 --2- 172.21.15.81:0/4114308391 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa9740ade50 0x7fa97414f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.423+0000 7fa97a0a9640 1 -- 172.21.15.81:0/4114308391 >> 172.21.15.81:0/4114308391 conn(0x7fa9740cb490 msgr2=0x7fa97414fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.423+0000 7fa97a0a9640 1 -- 172.21.15.81:0/4114308391 shutdown_connections 2023-11-08T16:34:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.423+0000 7fa97a0a9640 1 -- 172.21.15.81:0/4114308391 wait complete. 2023-11-08T16:34:58.443 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 3, "epoch": 54, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 3, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-08T16:34:44.993932+0000", "damaged": [], "data_pools": [ 8, 9 ], "enabled": true, "epoch": 54, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_6600": { "addr": "172.21.15.81:6837/552116415", "addrs": { "addrvec": [ { "addr": "172.21.15.81:6835", "nonce": 552116415, "type": "v2" }, { "addr": "172.21.15.81:6837", "nonce": 552116415, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6600, "incarnation": 51, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 65, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 7, "modified": "2023-11-08T16:34:55.206262+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 6600 } } } ], "standbys": [ { "addr": "172.21.15.93:6833/3027678987", "addrs": { "addrvec": [ { "addr": "172.21.15.93:6832", "nonce": 3027678987, "type": "v2" }, { "addr": "172.21.15.93:6833", "nonce": 3027678987, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 52, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5494, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.81:6836/3140373039", "addrs": { "addrvec": [ { "addr": "172.21.15.81:6834", "nonce": 3140373039, "type": "v2" }, { "addr": "172.21.15.81:6836", "nonce": 3140373039, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 50, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6594, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.93:6835/3145959397", "addrs": { "addrvec": [ { "addr": "172.21.15.93:6834", "nonce": 3145959397, "type": "v2" }, { "addr": "172.21.15.93:6835", "nonce": 3145959397, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 52, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6603, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-08T16:34:58.443 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:34:58.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.656+0000 7f2fc0954640 1 Processor -- start 2023-11-08T16:34:58.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.656+0000 7f2fc0954640 1 -- start start 2023-11-08T16:34:58.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.658+0000 7f2fc0954640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fbc0ced10 0x7f2fbc0cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:58.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.658+0000 7f2fc0954640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2fbc06b540 0x7f2fbc06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:58.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.658+0000 7f2fc0954640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fbc0cf690 con 0x7f2fbc150b80 2023-11-08T16:34:58.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.658+0000 7f2fc0954640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fbc15b0a0 con 0x7f2fbc06b540 2023-11-08T16:34:58.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.658+0000 7f2fc0954640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc0b2fe0 con 0x7f2fbc0ced10 2023-11-08T16:34:58.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.658+0000 7f2fbbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2fbc06b540 0x7f2fbc06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:58.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.658+0000 7f2fbbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2fbc06b540 0x7f2fbc06b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55220/0 (socket says 172.21.15.81:55220) 2023-11-08T16:34:58.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.658+0000 7f2fbbfff640 1 -- 172.21.15.81:0/2104828860 learned_addr learned my addr 172.21.15.81:0/2104828860 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:58.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.659+0000 7f2fb2ffd640 1 --2- 172.21.15.81:0/2104828860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fbc0ced10 0x7f2fbc0cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:58.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.659+0000 7f2fbbfff640 1 -- 172.21.15.81:0/2104828860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fbc0ced10 msgr2=0x7f2fbc0cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:58.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.659+0000 7f2fbbfff640 1 --2- 172.21.15.81:0/2104828860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fbc0ced10 0x7f2fbc0cf150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.659+0000 7f2fbbfff640 1 -- 172.21.15.81:0/2104828860 >> v1:172.21.15.93:6789/0 conn(0x7f2fbc150b80 legacy=0x7f2fbc15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:34:58.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.659+0000 7f2fbbfff640 1 -- 172.21.15.81:0/2104828860 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc0b32c0 con 0x7f2fbc06b540 2023-11-08T16:34:58.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.660+0000 7f2fbbfff640 1 --2- 172.21.15.81:0/2104828860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fbc06b540 0x7f2fbc06b9d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400bb10 tx=0x7f2fb402e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8057d5b40bc13139 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:58.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.660+0000 7f2fbaffd640 1 -- 172.21.15.81:0/2104828860 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb402ee70 con 0x7f2fbc06b540 2023-11-08T16:34:58.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.660+0000 7f2fbaffd640 1 -- 172.21.15.81:0/2104828860 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb4036440 con 0x7f2fbc06b540 2023-11-08T16:34:58.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.660+0000 7f2fbaffd640 1 -- 172.21.15.81:0/2104828860 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb40367d0 con 0x7f2fbc06b540 2023-11-08T16:34:58.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.661+0000 7f2fc0954640 1 -- 172.21.15.81:0/2104828860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fbc06b540 msgr2=0x7f2fbc06b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.661+0000 7f2fc0954640 1 --2- 172.21.15.81:0/2104828860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fbc06b540 0x7f2fbc06b9d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2fb400bb10 tx=0x7f2fb402e0c0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.661+0000 7f2fc0954640 1 -- 172.21.15.81:0/2104828860 shutdown_connections 2023-11-08T16:34:58.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.661+0000 7f2fc0954640 1 --2- 172.21.15.81:0/2104828860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fbc06b540 0x7f2fbc06b9d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.661+0000 7f2fc0954640 1 --2- 172.21.15.81:0/2104828860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fbc0ced10 0x7f2fbc0cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.661+0000 7f2fc0954640 1 -- 172.21.15.81:0/2104828860 >> 172.21.15.81:0/2104828860 conn(0x7f2fbc06a4a0 msgr2=0x7f2fbc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:58.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.661+0000 7f2fc0954640 1 -- 172.21.15.81:0/2104828860 shutdown_connections 2023-11-08T16:34:58.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.661+0000 7f2fc0954640 1 -- 172.21.15.81:0/2104828860 wait complete. 2023-11-08T16:34:58.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.663+0000 7f2fc0954640 1 Processor -- start 2023-11-08T16:34:58.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.663+0000 7f2fc0954640 1 -- start start 2023-11-08T16:34:58.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.664+0000 7f2fc0954640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2fbc06b540 0x7f2fbc0af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:58.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.664+0000 7f2fc0954640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fbc0ced10 0x7f2fbc14c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:58.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.664+0000 7f2fc0954640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fbc150b80 0x7f2fbc140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:58.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.664+0000 7f2fc0954640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc1536e0 con 0x7f2fbc0ced10 2023-11-08T16:34:58.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.664+0000 7f2fc0954640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc14f170 con 0x7f2fbc06b540 2023-11-08T16:34:58.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.664+0000 7f2fc0954640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fbc071320 con 0x7f2fbc150b80 2023-11-08T16:34:58.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fb2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fbc0ced10 0x7f2fbc14c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:58.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fb2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fbc0ced10 0x7f2fbc14c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55226/0 (socket says 172.21.15.81:55226) 2023-11-08T16:34:58.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fb2ffd640 1 -- 172.21.15.81:0/2941303693 learned_addr learned my addr 172.21.15.81:0/2941303693 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:34:58.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fbb7fe640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2fbc06b540 0x7f2fbc0af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:58.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fbbfff640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fbc150b80 0x7f2fbc140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:58.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fb2ffd640 1 -- 172.21.15.81:0/2941303693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fbc150b80 msgr2=0x7f2fbc140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fb2ffd640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fbc150b80 0x7f2fbc140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fb2ffd640 1 -- 172.21.15.81:0/2941303693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2fbc06b540 msgr2=0x7f2fbc0af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fb2ffd640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2fbc06b540 0x7f2fbc0af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fb2ffd640 1 -- 172.21.15.81:0/2941303693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb400b770 con 0x7f2fbc0ced10 2023-11-08T16:34:58.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fbbfff640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fbc150b80 0x7f2fbc140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:34:58.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.665+0000 7f2fb2ffd640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fbc0ced10 0x7f2fbc14c200 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400d800 tx=0x7f2fa400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:58.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.666+0000 7f2fb8ff9640 1 -- 172.21.15.81:0/2941303693 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400b6c0 con 0x7f2fbc0ced10 2023-11-08T16:34:58.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.666+0000 7f2fb8ff9640 1 -- 172.21.15.81:0/2941303693 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa400b820 con 0x7f2fbc0ced10 2023-11-08T16:34:58.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.666+0000 7f2fc0954640 1 -- 172.21.15.81:0/2941303693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fbc0715c0 con 0x7f2fbc0ced10 2023-11-08T16:34:58.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.666+0000 7f2fb8ff9640 1 -- 172.21.15.81:0/2941303693 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400bbb0 con 0x7f2fbc0ced10 2023-11-08T16:34:58.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.666+0000 7f2fc0954640 1 -- 172.21.15.81:0/2941303693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fbc153370 con 0x7f2fbc0ced10 2023-11-08T16:34:58.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.667+0000 7f2fc0954640 1 -- 172.21.15.81:0/2941303693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fbc153850 con 0x7f2fbc0ced10 2023-11-08T16:34:58.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.668+0000 7f2fb8ff9640 1 -- 172.21.15.81:0/2941303693 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fa400bd10 con 0x7f2fbc0ced10 2023-11-08T16:34:58.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.669+0000 7f2fb8ff9640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f9c06ae70 0x7f2f9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:34:58.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.669+0000 7f2fb8ff9640 1 -- 172.21.15.81:0/2941303693 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f2fa4089f00 con 0x7f2fbc0ced10 2023-11-08T16:34:58.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.669+0000 7f2fbb7fe640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f9c06ae70 0x7f2f9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:34:58.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.670+0000 7f2fbb7fe640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f9c06ae70 0x7f2f9c06d330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2fac005520 tx=0x7f2fac009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:34:58.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.673+0000 7f2fb8ff9640 1 -- 172.21.15.81:0/2941303693 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2fa4054c20 con 0x7f2fbc0ced10 2023-11-08T16:34:58.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.902+0000 7f2fc0954640 1 -- 172.21.15.81:0/2941303693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2fbc067800 con 0x7f2fbc0ced10 2023-11-08T16:34:58.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.903+0000 7f2fb8ff9640 1 -- 172.21.15.81:0/2941303693 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) v1 ==== 53+0+91 (secure 0 0 0) 0x7f2fa4058240 con 0x7f2fbc0ced10 2023-11-08T16:34:58.905 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-11-08T16:34:58.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.905+0000 7f2fc0954640 1 -- 172.21.15.81:0/2941303693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f9c06ae70 msgr2=0x7f2f9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.906+0000 7f2fc0954640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f9c06ae70 0x7f2f9c06d330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2fac005520 tx=0x7f2fac009040 comp rx=0 tx=0).stop 2023-11-08T16:34:58.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.906+0000 7f2fc0954640 1 -- 172.21.15.81:0/2941303693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fbc0ced10 msgr2=0x7f2fbc14c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:34:58.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.906+0000 7f2fc0954640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fbc0ced10 0x7f2fbc14c200 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400d800 tx=0x7f2fa400dcc0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.906+0000 7f2fc0954640 1 -- 172.21.15.81:0/2941303693 shutdown_connections 2023-11-08T16:34:58.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.906+0000 7f2fc0954640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2f9c06ae70 0x7f2f9c06d330 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.906+0000 7f2fc0954640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fbc150b80 0x7f2fbc140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.906+0000 7f2fc0954640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fbc0ced10 0x7f2fbc14c200 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.906+0000 7f2fc0954640 1 --2- 172.21.15.81:0/2941303693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2fbc06b540 0x7f2fbc0af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:34:58.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.906+0000 7f2fc0954640 1 -- 172.21.15.81:0/2941303693 >> 172.21.15.81:0/2941303693 conn(0x7f2fbc06a4a0 msgr2=0x7f2fbc15d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:34:58.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.906+0000 7f2fc0954640 1 -- 172.21.15.81:0/2941303693 shutdown_connections 2023-11-08T16:34:58.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:34:58.906+0000 7f2fc0954640 1 -- 172.21.15.81:0/2941303693 wait complete. 2023-11-08T16:34:58.925 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:34:58.925 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:34:58.926 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:34:58.926 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:34:58.926 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:34:58.927 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:34:58.927 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:34:58.927 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:34:58.928 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:34:58.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:34:58.928 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:34:58.978 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:34:58.978 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:34:58.979 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:34:58.979 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:34:58.979 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:34:58.979 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:34:58.979 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:34:58.980 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:34:58.980 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:34:58.980 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:34:58.980 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:34:58.980 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:34:58.981 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:34:58.981 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:34:58.981 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:34:58.981 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:34:58.981 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:34:58.982 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 42011sec preferred_lft 42011sec 2023-11-08T16:34:58.982 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:34:58.982 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:34:58.982 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:34:58.983 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff 2023-11-08T16:34:58.983 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:34:58.983 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:34:58.983 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:34:58.983 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:34:58.984 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:34:58.984 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:34:58.984 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:34:58.984 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:34:58.984 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:34:58.985 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:34:58.985 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::2cac:17ff:fe64:67da/64 scope link 2023-11-08T16:34:58.985 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:34:58.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:34:58.987 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:34:59.073 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:34:59.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:34:59.077 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:34:59.131 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:34:59.131 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:34:59.131 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:34:59.131 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:34:59.132 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:34:59.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:34:59.132 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:34:59.133 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:34:59.133 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:34:59.134 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:34:59.134 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:34:59.134 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:34:59.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:34:59.317 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:34:59.369 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:34:59.369 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:34:59.370 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:34:59.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:34:59.371 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:34:59.456 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:34:59.457 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:34:59.457 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:34:59.509 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:34:59.509 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:34:59.510 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:34:59.560 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:34:59.655 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:34:59.710 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:34:59.711 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:34:59.711 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:34:59.711 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:34:59.711 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:34:59.711 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:34:59.711 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:34:59.712 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:34:59.712 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:34:59.712 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:34:59.712 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:34:59.712 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:34:59.713 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:34:59.713 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:34:59.713 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:34:59.713 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:34:59.713 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:34:59.713 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:34:59.714 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:34:59.714 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:34:59.714 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:34:59.714 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:34:59.715 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:34:59.715 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:34:59.715 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:34:59.715 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:34:59.715 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:34:59.716 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:34:59.716 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:34:59.716 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:34:59.716 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:34:59.716 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:34:59.717 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:34:59.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:34:59.718 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:34:59.770 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:34:59.770 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:34:59.777 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:34:59.854 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:34:59.899 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:34:59.899 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:34:59.899 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:34:59.899 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:34:59.900 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:34:59.900 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:34:59.900 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:34:59.900 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:34:59.900 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:34:59.901 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:34:59.901 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:34:59.901 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:34:59.901 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:34:59.901 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:34:59.902 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:34:59.902 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:34:59.902 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:34:59.902 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:34:59.902 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:34:59.903 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:34:59.903 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:34:59.903 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:34:59.903 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:34:59.903 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:34:59.904 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:34:59.904 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:34:59.904 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:34:59.904 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:34:59.904 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:34:59.904 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:34:59.905 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:34:59.905 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:34:59.905 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:34:59.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:34:59.907 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:34:59.965 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:34:59.964+0000 7fc428438480 -1 init, newargv = 0x55e60ece1230 newargc=15 2023-11-08T16:34:59.966 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[110217]: starting ceph client 2023-11-08T16:34:59.972 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[110217]: starting fuse 2023-11-08T16:34:59.993 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:34:59.994 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:34:59.994 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:34:59.995 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:34:59.995 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:34:59.995 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:34:59.996 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:34:59.996 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:34:59.997 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:34:59.997 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:34:59.997 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:34:59.998 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:34:59.998 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:34:59.998 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:34:59.999 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:34:59.999 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:35:00.000 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:35:00.000 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:35:00.000 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:35:00.001 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:35:00.001 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:35:00.001 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:35:00.002 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:35:00.002 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:35:00.002 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:35:00.003 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:35:00.003 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:35:00.004 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:35:00.004 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:35:00.188 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.110217.asok 2023-11-08T16:35:00.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:35:00.205 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.110217.asok 2023-11-08T16:35:00.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:35:00.206 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.110217.asok status 2023-11-08T16:35:00.445 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:35:00.446 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:35:00.446 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:35:00.446 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:35:00.446 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:35:00.447 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:35:00.447 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:35:00.447 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "110217", 2023-11-08T16:35:00.447 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:35:00.447 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:35:00.448 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:35:00.448 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:35:00.448 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 6657, 2023-11-08T16:35:00.448 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:35:00.448 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:35:00.449 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:35:00.449 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 6657 2023-11-08T16:35:00.449 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:35:00.449 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:35:00.449 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:35:00.450 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:35:00.450 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1852048877 2023-11-08T16:35:00.450 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:35:00.450 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:35:00.450 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:35:00.451 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:35:00.451 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:35:00.451 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1852048877 2023-11-08T16:35:00.451 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:35:00.451 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.6657 192.168.0.1:0/1852048877", 2023-11-08T16:35:00.452 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/1852048877", 2023-11-08T16:35:00.452 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:35:00.452 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 54, 2023-11-08T16:35:00.452 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 68, 2023-11-08T16:35:00.452 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:35:00.453 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:35:00.453 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:35:00.453 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:35:00.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:35:00.461 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:35:00.461 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:35:00.515 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:35:00.516 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:35:00.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:35:00.517 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:35:00.612 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-08T16:35:00.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.831+0000 7faca3b3c640 1 Processor -- start 2023-11-08T16:35:00.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.831+0000 7faca3b3c640 1 -- start start 2023-11-08T16:35:00.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.832+0000 7faca3b3c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac9c15e550 0x7fac9c0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.833+0000 7faca3b3c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c0b0e70 con 0x7fac9c15e170 2023-11-08T16:35:00.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.833+0000 7faca3b3c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c071460 con 0x7fac9c0b0a20 2023-11-08T16:35:00.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.833+0000 7faca3b3c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0703c0 con 0x7fac9c15e550 2023-11-08T16:35:00.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.833+0000 7faca18d8640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fac9c15e170 0x7fac9c15ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46526/0 (socket says 172.21.15.81:46526) 2023-11-08T16:35:00.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.833+0000 7faca18d8640 1 -- 172.21.15.81:0/2014269671 learned_addr learned my addr 172.21.15.81:0/2014269671 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:35:00.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.833+0000 7faca10d7640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac9c15e550 0x7fac9c0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.833+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632208174 0 0) 0x7fac98001110 con 0x7fac9c15e170 2023-11-08T16:35:00.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.833+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac88003520 con 0x7fac9c15e170 2023-11-08T16:35:00.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1359893630 0 0) 0x7fac88003520 con 0x7fac9c15e170 2023-11-08T16:35:00.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac9c15e550 msgr2=0x7fac9c0b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:00.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac9c15e550 0x7fac9c0b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 >> v1:172.21.15.93:6789/0 conn(0x7fac9c0b0a20 legacy=0x7fac9c0ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:00.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c0706a0 con 0x7fac9c15e170 2023-11-08T16:35:00.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fac98001800 con 0x7fac9c15e170 2023-11-08T16:35:00.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 >> v1:172.21.15.81:6789/0 conn(0x7fac9c15e170 legacy=0x7fac9c15ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:00.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac880039a0 0x7fac88003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac880047c0 0x7fac88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac88005180 0x7fac8800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c0703c0 con 0x7fac880047c0 2023-11-08T16:35:00.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8800c080 con 0x7fac88005180 2023-11-08T16:35:00.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8800c240 con 0x7fac880039a0 2023-11-08T16:35:00.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca10d7640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac9c15e550 0x7fac9c0b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:35:00.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.834+0000 7faca10d7640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac880047c0 0x7fac88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.835+0000 7faca18d8640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac880039a0 0x7fac88003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.835+0000 7faca20d9640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac88005180 0x7fac8800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.835+0000 7faca10d7640 1 -- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac880039a0 msgr2=0x7fac88003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:00.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.835+0000 7faca10d7640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac880039a0 0x7fac88003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.835+0000 7faca10d7640 1 -- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac88005180 msgr2=0x7fac8800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:00.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.835+0000 7faca10d7640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac88005180 0x7fac8800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.835+0000 7faca10d7640 1 -- 172.21.15.81:0/2014269671 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac88003520 con 0x7fac880047c0 2023-11-08T16:35:00.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.836+0000 7faca10d7640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac880047c0 0x7fac88004c40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fac8c006fd0 tx=0x7fac880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b8eba1900a2a68 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:00.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.836+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c032600 con 0x7fac880047c0 2023-11-08T16:35:00.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.836+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c02e050 con 0x7fac880047c0 2023-11-08T16:35:00.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.836+0000 7faca08d6640 1 -- 172.21.15.81:0/2014269671 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c032980 con 0x7fac880047c0 2023-11-08T16:35:00.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.837+0000 7faca3b3c640 1 -- 172.21.15.81:0/2014269671 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac880047c0 msgr2=0x7fac88004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:00.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.837+0000 7faca3b3c640 1 --2- 172.21.15.81:0/2014269671 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac880047c0 0x7fac88004c40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fac8c006fd0 tx=0x7fac880057f0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.837+0000 7faca20d9640 1 -- 172.21.15.81:0/2014269671 reap_dead start 2023-11-08T16:35:00.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.837+0000 7faca3b3c640 1 -- 172.21.15.81:0/2014269671 shutdown_connections 2023-11-08T16:35:00.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.837+0000 7faca3b3c640 1 -- 172.21.15.81:0/2014269671 >> 172.21.15.81:0/2014269671 conn(0x7fac9c0cb4e0 msgr2=0x7fac9c0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:00.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.837+0000 7faca3b3c640 1 -- 172.21.15.81:0/2014269671 shutdown_connections 2023-11-08T16:35:00.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.837+0000 7faca3b3c640 1 -- 172.21.15.81:0/2014269671 wait complete. 2023-11-08T16:35:00.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.839+0000 7faca3b3c640 1 Processor -- start 2023-11-08T16:35:00.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.839+0000 7faca3b3c640 1 -- start start 2023-11-08T16:35:00.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca3b3c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac880047c0 0x7fac9c0c3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca3b3c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac9c1f2c20 0x7fac9c152a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca3b3c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac9c152f50 0x7fac9c14efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca3b3c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1533d0 con 0x7fac9c1f2c20 2023-11-08T16:35:00.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca3b3c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c150720 con 0x7fac9c152f50 2023-11-08T16:35:00.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca3b3c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c150a00 con 0x7fac880047c0 2023-11-08T16:35:00.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca10d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac9c1f2c20 0x7fac9c152a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca10d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac9c1f2c20 0x7fac9c152a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55242/0 (socket says 172.21.15.81:55242) 2023-11-08T16:35:00.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca10d7640 1 -- 172.21.15.81:0/3056915659 learned_addr learned my addr 172.21.15.81:0/3056915659 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:00.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca20d9640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac9c152f50 0x7fac9c14efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca18d8640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac880047c0 0x7fac9c0c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca10d7640 1 -- 172.21.15.81:0/3056915659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac880047c0 msgr2=0x7fac9c0c3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:00.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca10d7640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac880047c0 0x7fac9c0c3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca10d7640 1 -- 172.21.15.81:0/3056915659 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac9c152f50 msgr2=0x7fac9c14efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:00.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca10d7640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac9c152f50 0x7fac9c14efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.840+0000 7faca10d7640 1 -- 172.21.15.81:0/3056915659 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c150ce0 con 0x7fac9c1f2c20 2023-11-08T16:35:00.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.841+0000 7faca10d7640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac9c1f2c20 0x7fac9c152a10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fac9c071b10 tx=0x7fac8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:00.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.841+0000 7fac927fc640 1 -- 172.21.15.81:0/3056915659 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c03d8f0 con 0x7fac9c1f2c20 2023-11-08T16:35:00.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.841+0000 7fac927fc640 1 -- 172.21.15.81:0/3056915659 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c03da50 con 0x7fac9c1f2c20 2023-11-08T16:35:00.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.841+0000 7fac927fc640 1 -- 172.21.15.81:0/3056915659 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c03dde0 con 0x7fac9c1f2c20 2023-11-08T16:35:00.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.841+0000 7faca3b3c640 1 -- 172.21.15.81:0/3056915659 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c150fc0 con 0x7fac9c1f2c20 2023-11-08T16:35:00.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.841+0000 7faca3b3c640 1 -- 172.21.15.81:0/3056915659 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c1f30a0 con 0x7fac9c1f2c20 2023-11-08T16:35:00.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.843+0000 7fac927fc640 1 -- 172.21.15.81:0/3056915659 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fac8c031420 con 0x7fac9c1f2c20 2023-11-08T16:35:00.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.844+0000 7fac927fc640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac8006ae70 0x7fac8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.844+0000 7fac927fc640 1 -- 172.21.15.81:0/3056915659 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fac8c0abe60 con 0x7fac9c1f2c20 2023-11-08T16:35:00.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.845+0000 7faca18d8640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac8006ae70 0x7fac8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.845+0000 7faca67af740 1 Processor -- start 2023-11-08T16:35:00.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.845+0000 7faca67af740 1 -- start start 2023-11-08T16:35:00.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.845+0000 7faca18d8640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac8006ae70 0x7fac8006d330 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fac980012b0 tx=0x7fac98009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:00.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.846+0000 7faca67af740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129730 0x55f1110fc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.846+0000 7faca67af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129b10 0x55f11112c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.846+0000 7faca67af740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f1110fce00 0x55f11112ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.846+0000 7faca67af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11106f5f0 con 0x55f111129b10 2023-11-08T16:35:00.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.846+0000 7faca67af740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11106cce0 con 0x55f1110fce00 2023-11-08T16:35:00.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.846+0000 7faca67af740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1110685d0 con 0x55f111129730 2023-11-08T16:35:00.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.846+0000 7faca10d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129b10 0x55f11112c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.847+0000 7faca10d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129b10 0x55f11112c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55252/0 (socket says 172.21.15.81:55252) 2023-11-08T16:35:00.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.847+0000 7faca10d7640 1 -- 172.21.15.81:0/692918050 learned_addr learned my addr 172.21.15.81:0/692918050 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:00.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.847+0000 7faca20d9640 1 --2- 172.21.15.81:0/692918050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f1110fce00 0x55f11112ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.847+0000 7faca18d8640 1 --2- 172.21.15.81:0/692918050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129730 0x55f1110fc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.847+0000 7faca10d7640 1 -- 172.21.15.81:0/692918050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129730 msgr2=0x55f1110fc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:00.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.847+0000 7faca10d7640 1 --2- 172.21.15.81:0/692918050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129730 0x55f1110fc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.847+0000 7faca10d7640 1 -- 172.21.15.81:0/692918050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f1110fce00 msgr2=0x55f11112ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:00.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.847+0000 7faca10d7640 1 --2- 172.21.15.81:0/692918050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f1110fce00 0x55f11112ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.847+0000 7faca10d7640 1 -- 172.21.15.81:0/692918050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f111042a60 con 0x55f111129b10 2023-11-08T16:35:00.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.847+0000 7faca10d7640 1 --2- 172.21.15.81:0/692918050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129b10 0x55f11112c4c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fac8c082610 tx=0x7fac8c07a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d7cf35376bb0ed server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:00.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.847+0000 7fac7f7fe640 1 -- 172.21.15.81:0/692918050 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c092120 con 0x55f111129b10 2023-11-08T16:35:00.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.848+0000 7fac7f7fe640 1 -- 172.21.15.81:0/692918050 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c07e990 con 0x55f111129b10 2023-11-08T16:35:00.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.848+0000 7fac7f7fe640 1 -- 172.21.15.81:0/692918050 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c082b40 con 0x55f111129b10 2023-11-08T16:35:00.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.848+0000 7faca67af740 1 -- 172.21.15.81:0/692918050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129b10 msgr2=0x55f11112c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:00.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.848+0000 7faca67af740 1 --2- 172.21.15.81:0/692918050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129b10 0x55f11112c4c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fac8c082610 tx=0x7fac8c07a260 comp rx=0 tx=0).stop 2023-11-08T16:35:00.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.848+0000 7faca67af740 1 -- 172.21.15.81:0/692918050 shutdown_connections 2023-11-08T16:35:00.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.848+0000 7faca67af740 1 --2- 172.21.15.81:0/692918050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f1110fce00 0x55f11112ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.848+0000 7faca67af740 1 --2- 172.21.15.81:0/692918050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129b10 0x55f11112c4c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.848+0000 7faca67af740 1 --2- 172.21.15.81:0/692918050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129730 0x55f1110fc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.848+0000 7faca67af740 1 -- 172.21.15.81:0/692918050 >> 172.21.15.81:0/692918050 conn(0x55f1110febd0 msgr2=0x55f1111474c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:00.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.848+0000 7faca67af740 1 -- 172.21.15.81:0/692918050 shutdown_connections 2023-11-08T16:35:00.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.848+0000 7faca67af740 1 -- 172.21.15.81:0/692918050 wait complete. 2023-11-08T16:35:00.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.849+0000 7faca67af740 1 -- start start 2023-11-08T16:35:00.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.849+0000 7faca67af740 1 Processor -- start 2023-11-08T16:35:00.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.850+0000 7faca67af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f111124db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.850+0000 7faca67af740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f11112e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.850+0000 7faca67af740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111123ec0 0x55f111123550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.850+0000 7faca67af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f111005590 con 0x55f111129730 2023-11-08T16:35:00.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.850+0000 7faca67af740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f111005230 con 0x55f111123ec0 2023-11-08T16:35:00.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.850+0000 7faca67af740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f110dcc0c0 con 0x55f111129b10 2023-11-08T16:35:00.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.850+0000 7faca67af740 10 client.0 _ll_register_callbacks cb 0x55f111027700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:35:00.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7faca18d8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f111124db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7faca18d8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f111124db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55264/0 (socket says 172.21.15.81:55264) 2023-11-08T16:35:00.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7faca18d8640 1 -- 172.21.15.81:0/1341224323 learned_addr learned my addr 172.21.15.81:0/1341224323 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:00.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7faca10d7640 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f11112e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7faca20d9640 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111123ec0 0x55f111123550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7faca18d8640 1 -- 172.21.15.81:0/1341224323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 msgr2=0x55f11112e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:00.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7faca18d8640 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f11112e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7faca18d8640 1 -- 172.21.15.81:0/1341224323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111123ec0 msgr2=0x55f111123550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:00.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7faca18d8640 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111123ec0 0x55f111123550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7faca18d8640 1 -- 172.21.15.81:0/1341224323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c1f30a0 con 0x55f111129730 2023-11-08T16:35:00.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7faca18d8640 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f111124db0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fac98004ac0 tx=0x7fac98001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6363779aa12e37c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:00.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.851+0000 7fac7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:35:00.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7fac7cff9640 1 -- 172.21.15.81:0/1341224323 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac980108e0 con 0x55f111129730 2023-11-08T16:35:00.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7fac7cff9640 1 -- 172.21.15.81:0/1341224323 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fac9801c070 con 0x55f111129730 2023-11-08T16:35:00.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7faca67af740 1 -- 172.21.15.81:0/1341224323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f11108e910 con 0x55f111129730 2023-11-08T16:35:00.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7fac7cff9640 1 -- 172.21.15.81:0/1341224323 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac98014600 con 0x55f111129730 2023-11-08T16:35:00.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7faca67af740 10 client.6669 fetch_fsmap learned FSMap version 54 2023-11-08T16:35:00.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7faca67af740 1 -- 172.21.15.81:0/1341224323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55f111003810 con 0x55f111129730 2023-11-08T16:35:00.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7fac7cff9640 1 -- 172.21.15.81:0/1341224323 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f11108e910 con 0x55f111129730 2023-11-08T16:35:00.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7fac7cff9640 1 -- 172.21.15.81:0/1341224323 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fac98014760 con 0x55f111129730 2023-11-08T16:35:00.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7faca67af740 10 client.6669 fetch_fsmap finished waiting for FSMap version 54 2023-11-08T16:35:00.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7faca67af740 10 client.6669 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-08T16:35:00.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7faca67af740 20 client.6669 populate_metadata read hostname 'smithi081' 2023-11-08T16:35:00.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.852+0000 7faca67af740 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55f111130370 0x55f111129f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.853+0000 7faca67af740 4 client.6669 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:35:00.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.853+0000 7faca67af740 1 -- 172.21.15.81:0/1341224323 --> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f11103b450 con 0x55f111130370 2023-11-08T16:35:00.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.853+0000 7faca10d7640 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55f111130370 0x55f111129f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.853+0000 7fac7cff9640 10 client.6669 ms_handle_connect on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:00.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.853+0000 7faca10d7640 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55f111130370 0x55f111129f70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0780da1b1b0fb82 server_cookie=af8331a05e543b1e in_seq=0 out_seq=0 2023-11-08T16:35:00.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7fac7cff9640 1 -- 172.21.15.81:0/1341224323 <== mds.0 v2:172.21.15.81:6835/552116415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f11103b450 con 0x55f111130370 2023-11-08T16:35:00.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7fac7cff9640 10 client.6669 handle_command_reply: tid=0 2023-11-08T16:35:00.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7faca67af740 1 client.6669 shutdown 2023-11-08T16:35:00.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7faca67af740 1 -- 172.21.15.81:0/1341224323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 msgr2=0x55f111124db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:00.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7faca67af740 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f111124db0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fac98004ac0 tx=0x7fac98001ac0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7faca67af740 1 -- 172.21.15.81:0/1341224323 shutdown_connections 2023-11-08T16:35:00.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7faca67af740 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55f111130370 0x55f111129f70 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7faca67af740 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111123ec0 0x55f111123550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7faca67af740 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f11112e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7faca67af740 1 --2- 172.21.15.81:0/1341224323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f111124db0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7fac7cff9640 0 client.6669 ms_handle_reset on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:00.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7faca67af740 1 -- 172.21.15.81:0/1341224323 >> 172.21.15.81:0/1341224323 conn(0x55f1110fce00 msgr2=0x55f1111004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:00.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7faca67af740 1 -- 172.21.15.81:0/1341224323 shutdown_connections 2023-11-08T16:35:00.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.854+0000 7faca67af740 1 -- 172.21.15.81:0/1341224323 wait complete. 2023-11-08T16:35:00.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.855+0000 7faca67af740 20 client.6669 trim_cache size 0 max 16384 2023-11-08T16:35:00.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.871+0000 7faca67af740 1 Processor -- start 2023-11-08T16:35:00.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.871+0000 7faca67af740 1 -- start start 2023-11-08T16:35:00.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.872+0000 7faca67af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111123ec0 0x55f111126d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.872+0000 7faca67af740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111129730 0x55f1111ab610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.872+0000 7faca67af740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f111147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.872+0000 7faca67af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f111100560 con 0x55f111123ec0 2023-11-08T16:35:00.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.872+0000 7faca67af740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11112c770 con 0x55f111129730 2023-11-08T16:35:00.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.872+0000 7faca67af740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f11113e9d0 con 0x55f111129b10 2023-11-08T16:35:00.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca18d8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111123ec0 0x55f111126d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca18d8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111123ec0 0x55f111126d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55270/0 (socket says 172.21.15.81:55270) 2023-11-08T16:35:00.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca18d8640 1 -- 172.21.15.81:0/3112593620 learned_addr learned my addr 172.21.15.81:0/3112593620 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:00.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca10d7640 1 --2- 172.21.15.81:0/3112593620 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111129730 0x55f1111ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca20d9640 1 --2- 172.21.15.81:0/3112593620 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f111147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca18d8640 1 -- 172.21.15.81:0/3112593620 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 msgr2=0x55f111147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:00.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca18d8640 1 --2- 172.21.15.81:0/3112593620 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f111147ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca18d8640 1 -- 172.21.15.81:0/3112593620 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111129730 msgr2=0x55f1111ab610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:00.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca18d8640 1 --2- 172.21.15.81:0/3112593620 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111129730 0x55f1111ab610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca18d8640 1 -- 172.21.15.81:0/3112593620 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f111097c60 con 0x55f111123ec0 2023-11-08T16:35:00.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca20d9640 1 --2- 172.21.15.81:0/3112593620 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f111147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:35:00.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.873+0000 7faca18d8640 1 --2- 172.21.15.81:0/3112593620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111123ec0 0x55f111126d20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fac98010620 tx=0x7fac98021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657ed606be30e1bb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:00.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7fac7ffff640 1 -- 172.21.15.81:0/3112593620 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98005420 con 0x55f111123ec0 2023-11-08T16:35:00.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7fac7ffff640 1 -- 172.21.15.81:0/3112593620 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac98014c20 con 0x55f111123ec0 2023-11-08T16:35:00.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7fac7ffff640 1 -- 172.21.15.81:0/3112593620 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac98014600 con 0x55f111123ec0 2023-11-08T16:35:00.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7faca67af740 1 -- 172.21.15.81:0/3112593620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111123ec0 msgr2=0x55f111126d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:00.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7faca67af740 1 --2- 172.21.15.81:0/3112593620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111123ec0 0x55f111126d20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fac98010620 tx=0x7fac98021000 comp rx=0 tx=0).stop 2023-11-08T16:35:00.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7faca67af740 1 -- 172.21.15.81:0/3112593620 shutdown_connections 2023-11-08T16:35:00.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7faca67af740 1 --2- 172.21.15.81:0/3112593620 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f111147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7faca67af740 1 --2- 172.21.15.81:0/3112593620 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111129730 0x55f1111ab610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7faca67af740 1 --2- 172.21.15.81:0/3112593620 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111123ec0 0x55f111126d20 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7faca67af740 1 -- 172.21.15.81:0/3112593620 >> 172.21.15.81:0/3112593620 conn(0x55f1110fce00 msgr2=0x55f1110fd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:00.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7faca67af740 1 -- 172.21.15.81:0/3112593620 shutdown_connections 2023-11-08T16:35:00.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.874+0000 7faca67af740 1 -- 172.21.15.81:0/3112593620 wait complete. 2023-11-08T16:35:00.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.875+0000 7faca67af740 1 -- start start 2023-11-08T16:35:00.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.875+0000 7faca67af740 1 Processor -- start 2023-11-08T16:35:00.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.876+0000 7faca67af740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111123ec0 0x55f111148070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.876+0000 7faca67af740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f1111b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.876+0000 7faca67af740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f1111a7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.876+0000 7faca67af740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1111287d0 con 0x55f111129730 2023-11-08T16:35:00.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.876+0000 7faca67af740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1111a8920 con 0x55f111123ec0 2023-11-08T16:35:00.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.876+0000 7faca67af740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1111b22f0 con 0x55f111129b10 2023-11-08T16:35:00.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.876+0000 7faca67af740 10 client.0 _ll_register_callbacks cb 0x7fac98010830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:35:00.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.876+0000 7faca10d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f1111b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7faca10d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f1111b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55286/0 (socket says 172.21.15.81:55286) 2023-11-08T16:35:00.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7faca10d7640 1 -- 172.21.15.81:0/3845383604 learned_addr learned my addr 172.21.15.81:0/3845383604 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:00.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7faca18d8640 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111123ec0 0x55f111148070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7faca20d9640 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f1111a7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7faca10d7640 1 -- 172.21.15.81:0/3845383604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 msgr2=0x55f1111a7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:00.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7faca10d7640 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f1111a7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7faca10d7640 1 -- 172.21.15.81:0/3845383604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111123ec0 msgr2=0x55f111148070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:00.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7faca10d7640 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111123ec0 0x55f111148070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7faca10d7640 1 -- 172.21.15.81:0/3845383604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f111003810 con 0x55f111129730 2023-11-08T16:35:00.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7faca10d7640 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f1111b19a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fac8c076d40 tx=0x7fac8c047070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20e95e80ef0649b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:00.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7fac7cff9640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:35:00.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7fac7cff9640 1 -- 172.21.15.81:0/3845383604 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac8c082b40 con 0x55f111129730 2023-11-08T16:35:00.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7fac7cff9640 1 -- 172.21.15.81:0/3845383604 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fac8c076f50 con 0x55f111129730 2023-11-08T16:35:00.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.877+0000 7faca67af740 1 -- 172.21.15.81:0/3845383604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fac8c07e990 con 0x55f111129730 2023-11-08T16:35:00.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7fac7cff9640 1 -- 172.21.15.81:0/3845383604 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac8c077110 con 0x55f111129730 2023-11-08T16:35:00.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7faca67af740 10 client.6675 fetch_fsmap learned FSMap version 54 2023-11-08T16:35:00.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7faca67af740 1 -- 172.21.15.81:0/3845383604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55f111004580 con 0x55f111129730 2023-11-08T16:35:00.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7fac7cff9640 1 -- 172.21.15.81:0/3845383604 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac8c07e990 con 0x55f111129730 2023-11-08T16:35:00.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7fac7cff9640 1 -- 172.21.15.81:0/3845383604 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fac8c04d070 con 0x55f111129730 2023-11-08T16:35:00.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7faca67af740 10 client.6675 fetch_fsmap finished waiting for FSMap version 54 2023-11-08T16:35:00.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7faca67af740 10 client.6675 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-08T16:35:00.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7faca67af740 20 client.6675 populate_metadata read hostname 'smithi081' 2023-11-08T16:35:00.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7faca67af740 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55f11115f020 0x55f111126cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:00.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7faca67af740 4 client.6675 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-08T16:35:00.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7faca67af740 1 -- 172.21.15.81:0/3845383604 --> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55f1110615b0 con 0x55f11115f020 2023-11-08T16:35:00.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.878+0000 7faca18d8640 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55f11115f020 0x55f111126cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:00.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.879+0000 7faca18d8640 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55f11115f020 0x55f111126cb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d713e8995be28c0 server_cookie=5cc55a3cce3ab264 in_seq=0 out_seq=0 2023-11-08T16:35:00.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.879+0000 7fac7cff9640 10 client.6675 ms_handle_connect on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:00.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7fac7cff9640 1 -- 172.21.15.81:0/3845383604 <== mds.0 v2:172.21.15.81:6835/552116415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f1110615b0 con 0x55f11115f020 2023-11-08T16:35:00.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7fac7cff9640 10 client.6675 handle_command_reply: tid=0 2023-11-08T16:35:00.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7faca67af740 1 client.6675 shutdown 2023-11-08T16:35:00.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7faca67af740 1 -- 172.21.15.81:0/3845383604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 msgr2=0x55f1111b19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:00.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7faca67af740 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f1111b19a0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fac8c076d40 tx=0x7fac8c047070 comp rx=0 tx=0).stop 2023-11-08T16:35:00.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7faca67af740 1 -- 172.21.15.81:0/3845383604 shutdown_connections 2023-11-08T16:35:00.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7faca67af740 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55f11115f020 0x55f111126cb0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7faca67af740 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f111129b10 0x55f1111a7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7fac7cff9640 0 client.6675 ms_handle_reset on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:00.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7faca67af740 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f111129730 0x55f1111b19a0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7faca67af740 1 --2- 172.21.15.81:0/3845383604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f111123ec0 0x55f111148070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7faca67af740 1 -- 172.21.15.81:0/3845383604 >> 172.21.15.81:0/3845383604 conn(0x55f1110fce00 msgr2=0x55f1110fb810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:00.911 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.880+0000 7f1a3fe83640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-11-08T16:35:00.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7faca67af740 1 -- 172.21.15.81:0/3845383604 shutdown_connections 2023-11-08T16:35:00.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.880+0000 7faca67af740 1 -- 172.21.15.81:0/3845383604 wait complete. 2023-11-08T16:35:00.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.881+0000 7faca67af740 20 client.6675 trim_cache size 0 max 16384 2023-11-08T16:35:00.913 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:35:00.913 INFO:teuthology.orchestra.run.smithi081.stdout: "return_code": 0, 2023-11-08T16:35:00.914 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_tag": "aeec8420-8ffe-4148-b618-a117a5a0a623", 2023-11-08T16:35:00.914 INFO:teuthology.orchestra.run.smithi081.stdout: "mode": "asynchronous" 2023-11-08T16:35:00.914 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:35:00.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 -- 172.21.15.81:0/3056915659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac8006ae70 msgr2=0x7fac8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:00.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac8006ae70 0x7fac8006d330 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fac980012b0 tx=0x7fac98009040 comp rx=0 tx=0).stop 2023-11-08T16:35:00.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 -- 172.21.15.81:0/3056915659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac9c1f2c20 msgr2=0x7fac9c152a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:00.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac9c1f2c20 0x7fac9c152a10 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fac9c071b10 tx=0x7fac8c004730 comp rx=0 tx=0).stop 2023-11-08T16:35:00.916 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.882+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x558622158100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:35:00.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 -- 172.21.15.81:0/3056915659 shutdown_connections 2023-11-08T16:35:00.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fac8006ae70 0x7fac8006d330 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fac9c152f50 0x7fac9c14efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fac9c1f2c20 0x7fac9c152a10 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 --2- 172.21.15.81:0/3056915659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fac880047c0 0x7fac9c0c3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:00.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 -- 172.21.15.81:0/3056915659 >> 172.21.15.81:0/3056915659 conn(0x7fac9c0cb4e0 msgr2=0x7fac9c06c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:00.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 -- 172.21.15.81:0/3056915659 shutdown_connections 2023-11-08T16:35:00.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:00.882+0000 7faca3b3c640 1 -- 172.21.15.81:0/3056915659 wait complete. 2023-11-08T16:35:00.921 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.884+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001f8:[<0x10000000000/3 v40>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.921 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.885+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fd [2,head] /subdir/8 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001fd:[<0x10000000000/8 v80>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001fd:[<0x10000000000/8 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.921 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.885+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [2,head] /subdir/15 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622254680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000204:[<0x10000000000/15 v136>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000204:[<0x10000000000/15 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.921 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.886+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fe [2,head] /subdir/9 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001fe:[<0x10000000000/9 v88>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001fe:[<0x10000000000/9 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.921 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.886+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [2,head] /subdir/20 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622252580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000209:[<0x10000000000/20 v176>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000209:[<0x10000000000/20 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.922 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.886+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [2,head] /subdir/21 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622252000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x1000000020a:[<0x10000000000/21 v184>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x1000000020a:[<0x10000000000/21 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.922 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.886+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [2,head] /subdir/19 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622253080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000208:[<0x10000000000/19 v168>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000208:[<0x10000000000/19 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.922 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.887+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000201 [2,head] /subdir/12 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622255700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000201:[<0x10000000000/12 v112>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000201:[<0x10000000000/12 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.922 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.887+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ff [2,head] /subdir/10 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862225c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001ff:[<0x10000000000/10 v96>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001ff:[<0x10000000000/10 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.923 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.887+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000200 [2,head] /subdir/11 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862225c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000200:[<0x10000000000/11 v104>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000200:[<0x10000000000/11 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.923 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.887+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001f7:[<0x10000000000/2 v32>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.923 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.888+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622252b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001f6:[<0x10000000000/1 v24>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.923 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.888+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001f9:[<0x10000000000/4 v48>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.923 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.888+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000205 [2,head] /subdir/16 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622254100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000205:[<0x10000000000/16 v144>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000205:[<0x10000000000/16 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.924 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.888+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [2,head] /subdir/24 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x1000000020d:[<0x10000000000/24 v200>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x1000000020d:[<0x10000000000/24 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.924 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [2,head] /subdir/22 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x1000000020b:[<0x10000000000/22 v192>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x1000000020b:[<0x10000000000/22 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.924 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /subdir/5 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001fa:[<0x10000000000/5 v56>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001fa:[<0x10000000000/5 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.924 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862225cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000001:[<0x10000000000/0 v16>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.924 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [2,head] /subdir/14 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622254c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000203:[<0x10000000000/14 v128>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000203:[<0x10000000000/14 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.925 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000202 [2,head] /subdir/13 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622255180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000202:[<0x10000000000/13 v120>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000202:[<0x10000000000/13 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.925 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [2,head] /subdir/23 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x1000000020c:[<0x10000000000/23 v198>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x1000000020c:[<0x10000000000/23 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.925 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.890+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000207 [2,head] /subdir/18 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622253600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000207:[<0x10000000000/18 v160>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000207:[<0x10000000000/18 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.925 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.890+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /subdir/7 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001fc:[<0x10000000000/7 v72>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001fc:[<0x10000000000/7 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.926 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.890+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000206 [2,head] /subdir/17 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622253b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000206:[<0x10000000000/17 v152>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000206:[<0x10000000000/17 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.926 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.891+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x55862215ed80 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-11-08T16:35:00.606015+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={6657=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x558622157600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-11-08T16:35:00.606015+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:35:00.926 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:35:00.893+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /subdir/6 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001fb:[<0x10000000000/6 v64>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001fb:[<0x10000000000/6 v1>,<0x1/subdir v7>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:35:00.927 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-08T16:35:01.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.137+0000 7f1999be7640 1 Processor -- start 2023-11-08T16:35:01.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.137+0000 7f1999be7640 1 -- start start 2023-11-08T16:35:01.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.139+0000 7f1999be7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19940cab40 0x7f19940cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.139+0000 7f1999be7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1994160b40 con 0x7f199406b250 2023-11-08T16:35:01.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.139+0000 7f1999be7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19940caf20 con 0x7f199406b6b0 2023-11-08T16:35:01.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.139+0000 7f1999be7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f199415db40 con 0x7f19940cab40 2023-11-08T16:35:01.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.139+0000 7f1992ffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f199406b250 0x7f19940b04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:38316/0 (socket says 172.21.15.81:38316) 2023-11-08T16:35:01.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.139+0000 7f1992ffd640 1 -- 172.21.15.81:0/4131114039 learned_addr learned my addr 172.21.15.81:0/4131114039 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:35:01.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19937fe640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19940cab40 0x7f19940cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488471687 0 0) 0x7f1994160b40 con 0x7f199406b250 2023-11-08T16:35:01.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1980003520 con 0x7f199406b250 2023-11-08T16:35:01.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3474908044 0 0) 0x7f1980003520 con 0x7f199406b250 2023-11-08T16:35:01.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19940cab40 msgr2=0x7f19940cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:01.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19940cab40 0x7f19940cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 >> v1:172.21.15.93:6789/0 conn(0x7f199406b6b0 legacy=0x7f1994160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:01.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f199415de20 con 0x7f199406b250 2023-11-08T16:35:01.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f197c001690 con 0x7f199406b250 2023-11-08T16:35:01.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 >> v1:172.21.15.81:6789/0 conn(0x7f199406b250 legacy=0x7f19940b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:01.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1980003a10 0x7f1980003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 0x7f1980004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19940cab40 0x7f198000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199415db40 con 0x7f19800047c0 2023-11-08T16:35:01.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1980005510 con 0x7f1980003a10 2023-11-08T16:35:01.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.140+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198000bec0 con 0x7f19940cab40 2023-11-08T16:35:01.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.141+0000 7f1992ffd640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 0x7f1980004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.141+0000 7f19937fe640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1980003a10 0x7f1980003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.141+0000 7f1992ffd640 1 -- 172.21.15.81:0/4131114039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19940cab40 msgr2=0x7f198000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:01.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.141+0000 7f1992ffd640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19940cab40 0x7f198000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.141+0000 7f1992ffd640 1 -- 172.21.15.81:0/4131114039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1980003a10 msgr2=0x7f1980003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:01.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.141+0000 7f1992ffd640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1980003a10 0x7f1980003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.141+0000 7f1992ffd640 1 -- 172.21.15.81:0/4131114039 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1980003520 con 0x7f19800047c0 2023-11-08T16:35:01.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.141+0000 7f19937fe640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1980003a10 0x7f1980003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:35:01.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.142+0000 7f1992ffd640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 0x7f1980004c40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c160 tx=0x7f19800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76f42b1d54cd4be server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:01.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.142+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c005ad0 con 0x7f19800047c0 2023-11-08T16:35:01.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.142+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197c005c30 con 0x7f19800047c0 2023-11-08T16:35:01.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.142+0000 7f19927fc640 1 -- 172.21.15.81:0/4131114039 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c02c7c0 con 0x7f19800047c0 2023-11-08T16:35:01.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.143+0000 7f1999be7640 1 -- 172.21.15.81:0/4131114039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 msgr2=0x7f1980004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:01.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.143+0000 7f1999be7640 1 --2- 172.21.15.81:0/4131114039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 0x7f1980004c40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f197c00c160 tx=0x7f19800057f0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.143+0000 7f1993fff640 1 -- 172.21.15.81:0/4131114039 reap_dead start 2023-11-08T16:35:01.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.143+0000 7f1999be7640 1 -- 172.21.15.81:0/4131114039 shutdown_connections 2023-11-08T16:35:01.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.143+0000 7f1999be7640 1 -- 172.21.15.81:0/4131114039 >> 172.21.15.81:0/4131114039 conn(0x7f19940cb4e0 msgr2=0x7f19940cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:01.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.143+0000 7f1999be7640 1 -- 172.21.15.81:0/4131114039 shutdown_connections 2023-11-08T16:35:01.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.143+0000 7f1999be7640 1 -- 172.21.15.81:0/4131114039 wait complete. 2023-11-08T16:35:01.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.145+0000 7f1999be7640 1 Processor -- start 2023-11-08T16:35:01.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.145+0000 7f1999be7640 1 -- start start 2023-11-08T16:35:01.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f1999be7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 0x7f1994151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f1999be7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1994152260 0x7f19940c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f1999be7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19940c46f0 0x7f199415fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f1999be7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19940c6250 con 0x7f19800047c0 2023-11-08T16:35:01.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f1999be7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1994151ab0 con 0x7f1994152260 2023-11-08T16:35:01.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f1999be7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19940c5bd0 con 0x7f19940c46f0 2023-11-08T16:35:01.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f19937fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 0x7f1994151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f19937fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 0x7f1994151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56790/0 (socket says 172.21.15.81:56790) 2023-11-08T16:35:01.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f19937fe640 1 -- 172.21.15.81:0/1850249047 learned_addr learned my addr 172.21.15.81:0/1850249047 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:01.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f19937fe640 1 -- 172.21.15.81:0/1850249047 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19940c46f0 msgr2=0x7f199415fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:01.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f19937fe640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19940c46f0 0x7f199415fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f19937fe640 1 -- 172.21.15.81:0/1850249047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1994152260 msgr2=0x7f19940c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:01.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f19937fe640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1994152260 0x7f19940c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.146+0000 7f19937fe640 1 -- 172.21.15.81:0/1850249047 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19940c5040 con 0x7f19800047c0 2023-11-08T16:35:01.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.147+0000 7f19937fe640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 0x7f1994151570 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f1984010890 tx=0x7f1984010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:01.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.147+0000 7f198bfff640 1 -- 172.21.15.81:0/1850249047 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198400eb60 con 0x7f19800047c0 2023-11-08T16:35:01.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.147+0000 7f198bfff640 1 -- 172.21.15.81:0/1850249047 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198400ecc0 con 0x7f19800047c0 2023-11-08T16:35:01.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.147+0000 7f1999be7640 1 -- 172.21.15.81:0/1850249047 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19940c5e70 con 0x7f19800047c0 2023-11-08T16:35:01.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.147+0000 7f198bfff640 1 -- 172.21.15.81:0/1850249047 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198401c400 con 0x7f19800047c0 2023-11-08T16:35:01.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.147+0000 7f1999be7640 1 -- 172.21.15.81:0/1850249047 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19940c6bb0 con 0x7f19800047c0 2023-11-08T16:35:01.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.149+0000 7f198bfff640 1 -- 172.21.15.81:0/1850249047 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f198401c560 con 0x7f19800047c0 2023-11-08T16:35:01.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.150+0000 7f198bfff640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f197406aea0 0x7f197406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.150+0000 7f198bfff640 1 -- 172.21.15.81:0/1850249047 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f1984089f60 con 0x7f19800047c0 2023-11-08T16:35:01.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.150+0000 7f1992ffd640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f197406aea0 0x7f197406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.151+0000 7f199c777740 1 Processor -- start 2023-11-08T16:35:01.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.151+0000 7f1992ffd640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f197406aea0 0x7f197406d360 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f197c034000 tx=0x7f197c007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:01.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.151+0000 7f199c777740 1 -- start start 2023-11-08T16:35:01.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f199c777740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf270 0x55c7ddc921d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f199c777740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf650 0x55c7ddcc2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f199c777740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddc92710 0x55c7ddcc2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f199c777740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ddc04b60 con 0x55c7ddcbf270 2023-11-08T16:35:01.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f199c777740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ddc02250 con 0x55c7ddc92710 2023-11-08T16:35:01.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f199c777740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7ddbfdb40 con 0x55c7ddcbf650 2023-11-08T16:35:01.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f19937fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf270 0x55c7ddc921d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f19937fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf270 0x55c7ddc921d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56794/0 (socket says 172.21.15.81:56794) 2023-11-08T16:35:01.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f19937fe640 1 -- 172.21.15.81:0/1761794961 learned_addr learned my addr 172.21.15.81:0/1761794961 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:01.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f19937fe640 1 -- 172.21.15.81:0/1761794961 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf650 msgr2=0x55c7ddcc2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:01.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f19937fe640 1 --2- 172.21.15.81:0/1761794961 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf650 0x55c7ddcc2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f19937fe640 1 -- 172.21.15.81:0/1761794961 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddc92710 msgr2=0x55c7ddcc2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:01.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f19937fe640 1 --2- 172.21.15.81:0/1761794961 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddc92710 0x55c7ddcc2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.152+0000 7f19937fe640 1 -- 172.21.15.81:0/1761794961 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7ddbfc1c0 con 0x55c7ddcbf270 2023-11-08T16:35:01.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.153+0000 7f19937fe640 1 --2- 172.21.15.81:0/1761794961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf270 0x55c7ddc921d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1984088070 tx=0x7f19840582d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3af42499f9891d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:01.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.153+0000 7f19897fa640 1 -- 172.21.15.81:0/1761794961 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19840701c0 con 0x55c7ddcbf270 2023-11-08T16:35:01.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.153+0000 7f19897fa640 1 -- 172.21.15.81:0/1761794961 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1984060bc0 con 0x55c7ddcbf270 2023-11-08T16:35:01.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.153+0000 7f19897fa640 1 -- 172.21.15.81:0/1761794961 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1984055080 con 0x55c7ddcbf270 2023-11-08T16:35:01.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.154+0000 7f199c777740 1 -- 172.21.15.81:0/1761794961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf270 msgr2=0x55c7ddc921d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:01.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.154+0000 7f199c777740 1 --2- 172.21.15.81:0/1761794961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf270 0x55c7ddc921d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f1984088070 tx=0x7f19840582d0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.154+0000 7f199c777740 1 -- 172.21.15.81:0/1761794961 shutdown_connections 2023-11-08T16:35:01.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.154+0000 7f199c777740 1 --2- 172.21.15.81:0/1761794961 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddc92710 0x55c7ddcc2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.154+0000 7f199c777740 1 --2- 172.21.15.81:0/1761794961 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf650 0x55c7ddcc2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.154+0000 7f199c777740 1 --2- 172.21.15.81:0/1761794961 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf270 0x55c7ddc921d0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.154+0000 7f199c777740 1 -- 172.21.15.81:0/1761794961 >> 172.21.15.81:0/1761794961 conn(0x55c7ddc94480 msgr2=0x55c7ddcdd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:01.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.154+0000 7f199c777740 1 -- 172.21.15.81:0/1761794961 shutdown_connections 2023-11-08T16:35:01.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.154+0000 7f199c777740 1 -- 172.21.15.81:0/1761794961 wait complete. 2023-11-08T16:35:01.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.154+0000 7f199c777740 1 -- start start 2023-11-08T16:35:01.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.155+0000 7f199c777740 1 Processor -- start 2023-11-08T16:35:01.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f199c777740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 0x55c7ddcdd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f199c777740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcb35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f199c777740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddcd86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f199c777740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ddb9add0 con 0x55c7ddcbf650 2023-11-08T16:35:01.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f199c777740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ddb9aa70 con 0x55c7ddcbe520 2023-11-08T16:35:01.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f199c777740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7dd8f6f90 con 0x55c7ddcbf270 2023-11-08T16:35:01.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f199c777740 10 client.0 _ll_register_callbacks cb 0x55c7ddbbce80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:35:01.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f1992ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcb35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f1992ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcb35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56800/0 (socket says 172.21.15.81:56800) 2023-11-08T16:35:01.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f1992ffd640 1 -- 172.21.15.81:0/1747860921 learned_addr learned my addr 172.21.15.81:0/1747860921 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:01.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f1993fff640 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddcd86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f1992ffd640 1 -- 172.21.15.81:0/1747860921 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 msgr2=0x55c7ddcdd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:01.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f1992ffd640 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 0x55c7ddcdd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.156+0000 7f1992ffd640 1 -- 172.21.15.81:0/1747860921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 msgr2=0x55c7ddcd86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:01.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.157+0000 7f1992ffd640 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddcd86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.157+0000 7f1992ffd640 1 -- 172.21.15.81:0/1747860921 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19940c6bb0 con 0x55c7ddcbf650 2023-11-08T16:35:01.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.157+0000 7f1993fff640 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddcd86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:35:01.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.157+0000 7f1992ffd640 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcb35f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f197c004a60 tx=0x7f197c001950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca4fac53405f9d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:01.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.157+0000 7f19727fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:35:01.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.157+0000 7f19727fc640 1 -- 172.21.15.81:0/1747860921 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197c047070 con 0x55c7ddcbf650 2023-11-08T16:35:01.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.157+0000 7f19727fc640 1 -- 172.21.15.81:0/1747860921 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f197c04ac70 con 0x55c7ddcbf650 2023-11-08T16:35:01.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.157+0000 7f199c777740 1 -- 172.21.15.81:0/1747860921 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c7ddc24400 con 0x55c7ddcbf650 2023-11-08T16:35:01.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.157+0000 7f19727fc640 1 -- 172.21.15.81:0/1747860921 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f197c04ae90 con 0x55c7ddcbf650 2023-11-08T16:35:01.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.157+0000 7f199c777740 10 client.6687 fetch_fsmap learned FSMap version 54 2023-11-08T16:35:01.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.158+0000 7f199c777740 1 -- 172.21.15.81:0/1747860921 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55c7ddb99d20 con 0x55c7ddcbf650 2023-11-08T16:35:01.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.158+0000 7f19727fc640 1 -- 172.21.15.81:0/1747860921 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7ddc24400 con 0x55c7ddcbf650 2023-11-08T16:35:01.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.158+0000 7f19727fc640 1 -- 172.21.15.81:0/1747860921 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f197c02c9d0 con 0x55c7ddcbf650 2023-11-08T16:35:01.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.158+0000 7f199c777740 10 client.6687 fetch_fsmap finished waiting for FSMap version 54 2023-11-08T16:35:01.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.158+0000 7f199c777740 10 client.6687 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-08T16:35:01.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.158+0000 7f199c777740 20 client.6687 populate_metadata read hostname 'smithi081' 2023-11-08T16:35:01.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.158+0000 7f199c777740 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55c7ddcdb060 0x55c7ddcf0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.158+0000 7f199c777740 4 client.6687 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:35:01.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.158+0000 7f199c777740 1 -- 172.21.15.81:0/1747860921 --> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7ddbf6b20 con 0x55c7ddcdb060 2023-11-08T16:35:01.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.158+0000 7f19937fe640 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55c7ddcdb060 0x55c7ddcf0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.159+0000 7f19727fc640 10 client.6687 ms_handle_connect on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:01.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.159+0000 7f19937fe640 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55c7ddcdb060 0x55c7ddcf0230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eb66850981db6b3 server_cookie=5e575eccfac5b5a0 in_seq=0 out_seq=0 2023-11-08T16:35:01.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.159+0000 7f19727fc640 1 -- 172.21.15.81:0/1747860921 <== mds.0 v2:172.21.15.81:6835/552116415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c7ddbf6b20 con 0x55c7ddcdb060 2023-11-08T16:35:01.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.159+0000 7f19727fc640 10 client.6687 handle_command_reply: tid=0 2023-11-08T16:35:01.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.159+0000 7f199c777740 1 client.6687 shutdown 2023-11-08T16:35:01.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f199c777740 1 -- 172.21.15.81:0/1747860921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 msgr2=0x55c7ddcb35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:01.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f199c777740 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcb35f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f197c004a60 tx=0x7f197c001950 comp rx=0 tx=0).stop 2023-11-08T16:35:01.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f199c777740 1 -- 172.21.15.81:0/1747860921 shutdown_connections 2023-11-08T16:35:01.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f199c777740 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55c7ddcdb060 0x55c7ddcf0230 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f199c777740 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddcd86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f199c777740 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcb35f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f199c777740 1 --2- 172.21.15.81:0/1747860921 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 0x55c7ddcdd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f19727fc640 0 client.6687 ms_handle_reset on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:01.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f199c777740 1 -- 172.21.15.81:0/1747860921 >> 172.21.15.81:0/1747860921 conn(0x55c7ddc92710 msgr2=0x55c7ddce0510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:01.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f199c777740 1 -- 172.21.15.81:0/1747860921 shutdown_connections 2023-11-08T16:35:01.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f199c777740 1 -- 172.21.15.81:0/1747860921 wait complete. 2023-11-08T16:35:01.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.160+0000 7f199c777740 20 client.6687 trim_cache size 0 max 16384 2023-11-08T16:35:01.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.177+0000 7f199c777740 1 Processor -- start 2023-11-08T16:35:01.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.177+0000 7f199c777740 1 -- start start 2023-11-08T16:35:01.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.177+0000 7f199c777740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddcd78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f199c777740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 0x55c7ddcbea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f199c777740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcbd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f199c777740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ddbfccd0 con 0x55c7ddcbf650 2023-11-08T16:35:01.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f199c777740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ddcd7e20 con 0x55c7ddcbe520 2023-11-08T16:35:01.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f199c777740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7ddcbefb0 con 0x55c7ddcbf270 2023-11-08T16:35:01.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f1993fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcbd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f1993fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcbd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56804/0 (socket says 172.21.15.81:56804) 2023-11-08T16:35:01.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f1993fff640 1 -- 172.21.15.81:0/2569088973 learned_addr learned my addr 172.21.15.81:0/2569088973 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:01.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f19937fe640 1 --2- 172.21.15.81:0/2569088973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddcd78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f1993fff640 1 -- 172.21.15.81:0/2569088973 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 msgr2=0x55c7ddcbea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:01.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f1993fff640 1 --2- 172.21.15.81:0/2569088973 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 0x55c7ddcbea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f1993fff640 1 -- 172.21.15.81:0/2569088973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 msgr2=0x55c7ddcd78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:01.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f1993fff640 1 --2- 172.21.15.81:0/2569088973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddcd78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f1993fff640 1 -- 172.21.15.81:0/2569088973 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7ddc2d750 con 0x55c7ddcbf650 2023-11-08T16:35:01.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.178+0000 7f19937fe640 1 --2- 172.21.15.81:0/2569088973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddcd78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:35:01.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.179+0000 7f1993fff640 1 --2- 172.21.15.81:0/2569088973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcbd5e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f19940c22f0 tx=0x7f198c001ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a2bfc91d31b22f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:01.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.179+0000 7f1989ffb640 1 -- 172.21.15.81:0/2569088973 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c00b070 con 0x55c7ddcbf650 2023-11-08T16:35:01.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.179+0000 7f1989ffb640 1 -- 172.21.15.81:0/2569088973 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198c00d680 con 0x55c7ddcbf650 2023-11-08T16:35:01.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.179+0000 7f1989ffb640 1 -- 172.21.15.81:0/2569088973 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c00da30 con 0x55c7ddcbf650 2023-11-08T16:35:01.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.180+0000 7f199c777740 1 -- 172.21.15.81:0/2569088973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 msgr2=0x55c7ddcbd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:01.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.180+0000 7f199c777740 1 --2- 172.21.15.81:0/2569088973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcbd5e0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f19940c22f0 tx=0x7f198c001ef0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.180+0000 7f199c777740 1 -- 172.21.15.81:0/2569088973 shutdown_connections 2023-11-08T16:35:01.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.180+0000 7f199c777740 1 --2- 172.21.15.81:0/2569088973 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcbd5e0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.180+0000 7f199c777740 1 --2- 172.21.15.81:0/2569088973 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 0x55c7ddcbea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.180+0000 7f199c777740 1 --2- 172.21.15.81:0/2569088973 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddcd78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.180+0000 7f199c777740 1 -- 172.21.15.81:0/2569088973 >> 172.21.15.81:0/2569088973 conn(0x55c7ddc92710 msgr2=0x55c7ddc99230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:01.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.180+0000 7f199c777740 1 -- 172.21.15.81:0/2569088973 shutdown_connections 2023-11-08T16:35:01.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.180+0000 7f199c777740 1 -- 172.21.15.81:0/2569088973 wait complete. 2023-11-08T16:35:01.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.180+0000 7f199c777740 1 -- start start 2023-11-08T16:35:01.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.181+0000 7f199c777740 1 Processor -- start 2023-11-08T16:35:01.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.181+0000 7f199c777740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddc98130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f199c777740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 0x55c7ddcf6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f199c777740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcde250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f199c777740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ddcdedb0 con 0x55c7ddcbf650 2023-11-08T16:35:01.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f199c777740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7ddcc2f00 con 0x55c7ddcbe520 2023-11-08T16:35:01.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f199c777740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7ddcf6720 con 0x55c7ddcbf270 2023-11-08T16:35:01.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f199c777740 10 client.0 _ll_register_callbacks cb 0x55c7ddcc5b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:35:01.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f1993fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcde250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f1993fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcde250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:56808/0 (socket says 172.21.15.81:56808) 2023-11-08T16:35:01.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f1993fff640 1 -- 172.21.15.81:0/4084803315 learned_addr learned my addr 172.21.15.81:0/4084803315 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:01.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f1993fff640 1 -- 172.21.15.81:0/4084803315 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 msgr2=0x55c7ddcf6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:01.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f1993fff640 1 --2- 172.21.15.81:0/4084803315 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 0x55c7ddcf6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f1993fff640 1 -- 172.21.15.81:0/4084803315 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 msgr2=0x55c7ddc98130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:01.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f1993fff640 1 --2- 172.21.15.81:0/4084803315 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddc98130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.182+0000 7f1993fff640 1 -- 172.21.15.81:0/4084803315 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f198c001790 con 0x55c7ddcbf650 2023-11-08T16:35:01.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f1993fff640 1 --2- 172.21.15.81:0/4084803315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcde250 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f198c007f00 tx=0x7f198c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc78b5815fc91e29 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:01.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f19727fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:35:01.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f19727fc640 1 -- 172.21.15.81:0/4084803315 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c013880 con 0x55c7ddcbf650 2023-11-08T16:35:01.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f19727fc640 1 -- 172.21.15.81:0/4084803315 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f198c0139e0 con 0x55c7ddcbf650 2023-11-08T16:35:01.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f199c777740 1 -- 172.21.15.81:0/4084803315 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1984060bc0 con 0x55c7ddcbf650 2023-11-08T16:35:01.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f19727fc640 1 -- 172.21.15.81:0/4084803315 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f198c013be0 con 0x55c7ddcbf650 2023-11-08T16:35:01.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f199c777740 10 client.6693 fetch_fsmap learned FSMap version 54 2023-11-08T16:35:01.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f199c777740 1 -- 172.21.15.81:0/4084803315 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55c7ddbbf310 con 0x55c7ddcbf650 2023-11-08T16:35:01.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f19727fc640 1 -- 172.21.15.81:0/4084803315 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1984060bc0 con 0x55c7ddcbf650 2023-11-08T16:35:01.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f19727fc640 1 -- 172.21.15.81:0/4084803315 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f198c027440 con 0x55c7ddcbf650 2023-11-08T16:35:01.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f199c777740 10 client.6693 fetch_fsmap finished waiting for FSMap version 54 2023-11-08T16:35:01.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f199c777740 10 client.6693 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-08T16:35:01.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f199c777740 20 client.6693 populate_metadata read hostname 'smithi081' 2023-11-08T16:35:01.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f199c777740 1 --2- 172.21.15.81:0/4084803315 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55c7ddd47b30 0x55c7ddc96f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:01.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f199c777740 4 client.6693 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-08T16:35:01.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.183+0000 7f199c777740 1 -- 172.21.15.81:0/4084803315 --> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c7ddbf6840 con 0x55c7ddd47b30 2023-11-08T16:35:01.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.184+0000 7f19937fe640 1 --2- 172.21.15.81:0/4084803315 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55c7ddd47b30 0x55c7ddc96f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:01.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.184+0000 7f19937fe640 1 --2- 172.21.15.81:0/4084803315 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55c7ddd47b30 0x55c7ddc96f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31515c8163dbb7d2 server_cookie=590458def4bd2f5e in_seq=0 out_seq=0 2023-11-08T16:35:01.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.184+0000 7f19727fc640 10 client.6693 ms_handle_connect on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:01.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.184+0000 7f19727fc640 1 -- 172.21.15.81:0/4084803315 <== mds.0 v2:172.21.15.81:6835/552116415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55c7ddbf6840 con 0x55c7ddd47b30 2023-11-08T16:35:01.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.184+0000 7f19727fc640 10 client.6693 handle_command_reply: tid=0 2023-11-08T16:35:01.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.184+0000 7f199c777740 1 client.6693 shutdown 2023-11-08T16:35:01.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f199c777740 1 -- 172.21.15.81:0/4084803315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 msgr2=0x55c7ddcde250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:01.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f199c777740 1 --2- 172.21.15.81:0/4084803315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcde250 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f198c007f00 tx=0x7f198c000f40 comp rx=0 tx=0).stop 2023-11-08T16:35:01.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f199c777740 1 -- 172.21.15.81:0/4084803315 shutdown_connections 2023-11-08T16:35:01.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f199c777740 1 --2- 172.21.15.81:0/4084803315 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55c7ddd47b30 0x55c7ddc96f10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f199c777740 1 --2- 172.21.15.81:0/4084803315 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7ddcbf650 0x55c7ddcde250 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f199c777740 1 --2- 172.21.15.81:0/4084803315 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7ddcbf270 0x55c7ddcf6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f19727fc640 0 client.6693 ms_handle_reset on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:01.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f199c777740 1 --2- 172.21.15.81:0/4084803315 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7ddcbe520 0x55c7ddc98130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f199c777740 1 -- 172.21.15.81:0/4084803315 >> 172.21.15.81:0/4084803315 conn(0x55c7ddc92710 msgr2=0x55c7ddcdc000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:01.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f199c777740 1 -- 172.21.15.81:0/4084803315 shutdown_connections 2023-11-08T16:35:01.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f199c777740 1 -- 172.21.15.81:0/4084803315 wait complete. 2023-11-08T16:35:01.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.185+0000 7f199c777740 20 client.6693 trim_cache size 0 max 16384 2023-11-08T16:35:01.205 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:35:01.205 INFO:teuthology.orchestra.run.smithi081.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-08T16:35:01.205 INFO:teuthology.orchestra.run.smithi081.stdout: "scrubs": { 2023-11-08T16:35:01.206 INFO:teuthology.orchestra.run.smithi081.stdout: "aeec8420-8ffe-4148-b618-a117a5a0a623": { 2023-11-08T16:35:01.206 INFO:teuthology.orchestra.run.smithi081.stdout: "path": "/", 2023-11-08T16:35:01.206 INFO:teuthology.orchestra.run.smithi081.stdout: "tag": "aeec8420-8ffe-4148-b618-a117a5a0a623", 2023-11-08T16:35:01.206 INFO:teuthology.orchestra.run.smithi081.stdout: "options": "recursive,repair" 2023-11-08T16:35:01.207 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:35:01.207 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:35:01.207 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:35:01.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 -- 172.21.15.81:0/1850249047 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f197406aea0 msgr2=0x7f197406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:01.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f197406aea0 0x7f197406d360 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f197c034000 tx=0x7f197c007000 comp rx=0 tx=0).stop 2023-11-08T16:35:01.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 -- 172.21.15.81:0/1850249047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 msgr2=0x7f1994151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:01.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 0x7f1994151570 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f1984010890 tx=0x7f1984010d50 comp rx=0 tx=0).stop 2023-11-08T16:35:01.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 -- 172.21.15.81:0/1850249047 shutdown_connections 2023-11-08T16:35:01.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f197406aea0 0x7f197406d360 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19940c46f0 0x7f199415fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1994152260 0x7f19940c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 --2- 172.21.15.81:0/1850249047 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19800047c0 0x7f1994151570 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 -- 172.21.15.81:0/1850249047 >> 172.21.15.81:0/1850249047 conn(0x7f19940cb4e0 msgr2=0x7f199406baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 -- 172.21.15.81:0/1850249047 shutdown_connections 2023-11-08T16:35:01.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:01.186+0000 7f1999be7640 1 -- 172.21.15.81:0/1850249047 wait complete. 2023-11-08T16:35:01.211 INFO:tasks.cephfs.filesystem:scrub status for tag:aeec8420-8ffe-4148-b618-a117a5a0a623 - {'path': '/', 'tag': 'aeec8420-8ffe-4148-b618-a117a5a0a623', 'options': 'recursive,repair'} 2023-11-08T16:35:17.646 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:35:17.656 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:35:31.212 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-08T16:35:31.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.437+0000 7f38d6844640 1 Processor -- start 2023-11-08T16:35:31.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.437+0000 7f38d6844640 1 -- start start 2023-11-08T16:35:31.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.438+0000 7f38d6844640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f38d00b0bf0 0x7f38d00ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.439+0000 7f38d6844640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d0071460 con 0x7f38d015e170 2023-11-08T16:35:31.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.439+0000 7f38d6844640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d00703c0 con 0x7f38d015e550 2023-11-08T16:35:31.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.439+0000 7f38d6844640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d00706a0 con 0x7f38d00b0bf0 2023-11-08T16:35:31.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.439+0000 7f38cffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f38d015e170 0x7f38d015ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:42082/0 (socket says 172.21.15.81:42082) 2023-11-08T16:35:31.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.439+0000 7f38cffff640 1 -- 172.21.15.81:0/2818088311 learned_addr learned my addr 172.21.15.81:0/2818088311 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:35:31.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.439+0000 7f38d4de1640 1 --2- 172.21.15.81:0/2818088311 >> v2:172.21.15.93:3300/0 conn(0x7f38d00b0bf0 0x7f38d00ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.439+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486507364 0 0) 0x7f38d0071460 con 0x7f38d015e170 2023-11-08T16:35:31.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.439+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38b8003520 con 0x7f38d015e170 2023-11-08T16:35:31.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 185842786 0 0) 0x7f38b8003520 con 0x7f38d015e170 2023-11-08T16:35:31.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 >> v2:172.21.15.93:3300/0 conn(0x7f38d00b0bf0 msgr2=0x7f38d00ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:31.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 --2- 172.21.15.81:0/2818088311 >> v2:172.21.15.93:3300/0 conn(0x7f38d00b0bf0 0x7f38d00ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 >> v1:172.21.15.93:6789/0 conn(0x7f38d015e550 legacy=0x7f38d00b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:31.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0070980 con 0x7f38d015e170 2023-11-08T16:35:31.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f38c0001670 con 0x7f38d015e170 2023-11-08T16:35:31.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 >> v1:172.21.15.81:6789/0 conn(0x7f38d015e170 legacy=0x7f38d015ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:31.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 --2- 172.21.15.81:0/2818088311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38b80039e0 0x7f38b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 --2- 172.21.15.81:0/2818088311 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38b80047c0 0x7f38b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 --2- 172.21.15.81:0/2818088311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b8005180 0x7f38b800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d00706a0 con 0x7f38b8005180 2023-11-08T16:35:31.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38b800c050 con 0x7f38b80039e0 2023-11-08T16:35:31.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38b800c210 con 0x7f38b80047c0 2023-11-08T16:35:31.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.440+0000 7f38d4de1640 1 --2- 172.21.15.81:0/2818088311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b8005180 0x7f38b800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.441+0000 7f38cffff640 1 --2- 172.21.15.81:0/2818088311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38b80039e0 0x7f38b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.441+0000 7f38d4de1640 1 -- 172.21.15.81:0/2818088311 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38b80047c0 msgr2=0x7f38b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:31.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.441+0000 7f38d4de1640 1 --2- 172.21.15.81:0/2818088311 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38b80047c0 0x7f38b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.441+0000 7f38d4de1640 1 -- 172.21.15.81:0/2818088311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38b80039e0 msgr2=0x7f38b8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:31.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.441+0000 7f38d4de1640 1 --2- 172.21.15.81:0/2818088311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38b80039e0 0x7f38b8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.441+0000 7f38d4de1640 1 -- 172.21.15.81:0/2818088311 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b8003520 con 0x7f38b8005180 2023-11-08T16:35:31.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.442+0000 7f38d4de1640 1 -- 172.21.15.81:0/2818088311 reap_dead start 2023-11-08T16:35:31.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.442+0000 7f38d4de1640 1 --2- 172.21.15.81:0/2818088311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b8005180 0x7f38b800b980 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009fc0 tx=0x7f38b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6fdb2ca97dd34b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:31.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.442+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c802cc60 con 0x7f38b8005180 2023-11-08T16:35:31.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.442+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c800f040 con 0x7f38b8005180 2023-11-08T16:35:31.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.442+0000 7f38ceffd640 1 -- 172.21.15.81:0/2818088311 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003990 con 0x7f38b8005180 2023-11-08T16:35:31.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.443+0000 7f38d6844640 1 -- 172.21.15.81:0/2818088311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b8005180 msgr2=0x7f38b800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:31.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.443+0000 7f38d6844640 1 --2- 172.21.15.81:0/2818088311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b8005180 0x7f38b800b980 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009fc0 tx=0x7f38b80057f0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.443+0000 7f38d6844640 1 -- 172.21.15.81:0/2818088311 shutdown_connections 2023-11-08T16:35:31.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.443+0000 7f38d6844640 1 --2- 172.21.15.81:0/2818088311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38b8005180 0x7f38b800b980 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.443+0000 7f38d6844640 1 -- 172.21.15.81:0/2818088311 >> 172.21.15.81:0/2818088311 conn(0x7f38d00cb4e0 msgr2=0x7f38d00cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:31.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.443+0000 7f38d6844640 1 -- 172.21.15.81:0/2818088311 shutdown_connections 2023-11-08T16:35:31.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.443+0000 7f38d6844640 1 -- 172.21.15.81:0/2818088311 wait complete. 2023-11-08T16:35:31.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.445+0000 7f38d6844640 1 Processor -- start 2023-11-08T16:35:31.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.445+0000 7f38d6844640 1 -- start start 2023-11-08T16:35:31.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d6844640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38b8005180 0x7f38d0158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d6844640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d015c5c0 0x7f38d0147980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d6844640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d015ca40 0x7f38d0147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d6844640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d015b060 con 0x7f38d015ca40 2023-11-08T16:35:31.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d6844640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d015b340 con 0x7f38b8005180 2023-11-08T16:35:31.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d6844640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d015b620 con 0x7f38d015c5c0 2023-11-08T16:35:31.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d4de1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d015ca40 0x7f38d0147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d4de1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d015ca40 0x7f38d0147ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45132/0 (socket says 172.21.15.81:45132) 2023-11-08T16:35:31.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d4de1640 1 -- 172.21.15.81:0/1923186404 learned_addr learned my addr 172.21.15.81:0/1923186404 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:31.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d4de1640 1 -- 172.21.15.81:0/1923186404 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d015c5c0 msgr2=0x7f38d0147980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:31.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d4de1640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d015c5c0 0x7f38d0147980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d4de1640 1 -- 172.21.15.81:0/1923186404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38b8005180 msgr2=0x7f38d0158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:31.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d4de1640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38b8005180 0x7f38d0158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.446+0000 7f38d4de1640 1 -- 172.21.15.81:0/1923186404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d015b900 con 0x7f38d015ca40 2023-11-08T16:35:31.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.447+0000 7f38d4de1640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d015ca40 0x7f38d0147ec0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f38c802c6d0 tx=0x7f38c8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:31.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.447+0000 7f38ccff9640 1 -- 172.21.15.81:0/1923186404 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8035dc0 con 0x7f38d015ca40 2023-11-08T16:35:31.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.447+0000 7f38ccff9640 1 -- 172.21.15.81:0/1923186404 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8003990 con 0x7f38d015ca40 2023-11-08T16:35:31.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.447+0000 7f38ccff9640 1 -- 172.21.15.81:0/1923186404 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8003d80 con 0x7f38d015ca40 2023-11-08T16:35:31.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.447+0000 7f38d6844640 1 -- 172.21.15.81:0/1923186404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d015bbe0 con 0x7f38d015ca40 2023-11-08T16:35:31.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.447+0000 7f38d6844640 1 -- 172.21.15.81:0/1923186404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d01594e0 con 0x7f38d015ca40 2023-11-08T16:35:31.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.449+0000 7f38ccff9640 1 -- 172.21.15.81:0/1923186404 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f38c8004030 con 0x7f38d015ca40 2023-11-08T16:35:31.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.450+0000 7f38ccff9640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38bc06aea0 0x7f38bc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.450+0000 7f38ccff9640 1 -- 172.21.15.81:0/1923186404 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f38c80ac5e0 con 0x7f38d015ca40 2023-11-08T16:35:31.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.450+0000 7f38cffff640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38bc06aea0 0x7f38bc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.451+0000 7f38cffff640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38bc06aea0 0x7f38bc06d360 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f38c0001050 tx=0x7f38c00047e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:31.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.451+0000 7f38d93d2740 1 Processor -- start 2023-11-08T16:35:31.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.451+0000 7f38d93d2740 1 -- start start 2023-11-08T16:35:31.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.452+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 0x55569d9751d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.452+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2650 0x55569d9a5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.452+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d975710 0x55569d9a5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.452+0000 7f38d93d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d8e7b60 con 0x55569d975710 2023-11-08T16:35:31.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.452+0000 7f38d93d2740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d8e5250 con 0x55569d9a2650 2023-11-08T16:35:31.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.452+0000 7f38d93d2740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55569d8e0b40 con 0x55569d9a2270 2023-11-08T16:35:31.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38d4de1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d975710 0x55569d9a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38d4de1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d975710 0x55569d9a5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45142/0 (socket says 172.21.15.81:45142) 2023-11-08T16:35:31.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38d4de1640 1 -- 172.21.15.81:0/3548465665 learned_addr learned my addr 172.21.15.81:0/3548465665 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:31.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38cffff640 1 --2- 172.21.15.81:0/3548465665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 0x55569d9751d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38cf7fe640 1 --2- 172.21.15.81:0/3548465665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2650 0x55569d9a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38d4de1640 1 -- 172.21.15.81:0/3548465665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 msgr2=0x55569d9751d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:31.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38d4de1640 1 --2- 172.21.15.81:0/3548465665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 0x55569d9751d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38d4de1640 1 -- 172.21.15.81:0/3548465665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2650 msgr2=0x55569d9a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:31.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38d4de1640 1 --2- 172.21.15.81:0/3548465665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2650 0x55569d9a5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38d4de1640 1 -- 172.21.15.81:0/3548465665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55569d8df1c0 con 0x55569d975710 2023-11-08T16:35:31.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38cffff640 1 --2- 172.21.15.81:0/3548465665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 0x55569d9751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:35:31.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38d4de1640 1 --2- 172.21.15.81:0/3548465665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d975710 0x55569d9a5540 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f38c80aa790 tx=0x7f38c807adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c52b7a5841b0c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:31.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.453+0000 7f38b67fc640 1 -- 172.21.15.81:0/3548465665 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c807f2d0 con 0x55569d975710 2023-11-08T16:35:31.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.454+0000 7f38b67fc640 1 -- 172.21.15.81:0/3548465665 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8083480 con 0x55569d975710 2023-11-08T16:35:31.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.454+0000 7f38b67fc640 1 -- 172.21.15.81:0/3548465665 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c8079520 con 0x55569d975710 2023-11-08T16:35:31.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.454+0000 7f38d93d2740 1 -- 172.21.15.81:0/3548465665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d975710 msgr2=0x55569d9a5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:31.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.454+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3548465665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d975710 0x55569d9a5540 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f38c80aa790 tx=0x7f38c807adf0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.454+0000 7f38d93d2740 1 -- 172.21.15.81:0/3548465665 shutdown_connections 2023-11-08T16:35:31.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.454+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3548465665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d975710 0x55569d9a5540 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.454+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3548465665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2650 0x55569d9a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.454+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3548465665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 0x55569d9751d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.454+0000 7f38d93d2740 1 -- 172.21.15.81:0/3548465665 >> 172.21.15.81:0/3548465665 conn(0x55569d977480 msgr2=0x55569d9c0000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:31.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.454+0000 7f38d93d2740 1 -- 172.21.15.81:0/3548465665 shutdown_connections 2023-11-08T16:35:31.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.454+0000 7f38d93d2740 1 -- 172.21.15.81:0/3548465665 wait complete. 2023-11-08T16:35:31.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.455+0000 7f38d93d2740 1 -- start start 2023-11-08T16:35:31.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.455+0000 7f38d93d2740 1 Processor -- start 2023-11-08T16:35:31.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.456+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2270 0x55569d98bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.456+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2650 0x55569d98d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d98d7e0 0x55569d98dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38d93d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d87ddd0 con 0x55569d9a2650 2023-11-08T16:35:31.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38d93d2740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d87da70 con 0x55569d9a2270 2023-11-08T16:35:31.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38d93d2740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55569d5d9f90 con 0x55569d98d7e0 2023-11-08T16:35:31.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38d93d2740 10 client.0 _ll_register_callbacks cb 0x55569d89fe80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:35:31.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2650 0x55569d98d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2650 0x55569d98d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45146/0 (socket says 172.21.15.81:45146) 2023-11-08T16:35:31.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38cf7fe640 1 -- 172.21.15.81:0/2314808174 learned_addr learned my addr 172.21.15.81:0/2314808174 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:31.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38cffff640 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2270 0x55569d98bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38d4de1640 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d98d7e0 0x55569d98dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38cf7fe640 1 -- 172.21.15.81:0/2314808174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d98d7e0 msgr2=0x55569d98dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:31.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38cf7fe640 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d98d7e0 0x55569d98dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38cf7fe640 1 -- 172.21.15.81:0/2314808174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2270 msgr2=0x55569d98bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:31.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38cf7fe640 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2270 0x55569d98bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38cf7fe640 1 -- 172.21.15.81:0/2314808174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d01594e0 con 0x55569d9a2650 2023-11-08T16:35:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.457+0000 7f38cffff640 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2270 0x55569d98bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:35:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.458+0000 7f38cf7fe640 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2650 0x55569d98d2a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f38c4001d90 tx=0x7f38c400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b0c30a90e92185 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.458+0000 7f38aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:35:31.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.458+0000 7f38aeffd640 1 -- 172.21.15.81:0/2314808174 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c400b070 con 0x55569d9a2650 2023-11-08T16:35:31.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.458+0000 7f38aeffd640 1 -- 172.21.15.81:0/2314808174 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f38c40052d0 con 0x55569d9a2650 2023-11-08T16:35:31.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.458+0000 7f38d93d2740 1 -- 172.21.15.81:0/2314808174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55569d907400 con 0x55569d9a2650 2023-11-08T16:35:31.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.458+0000 7f38aeffd640 1 -- 172.21.15.81:0/2314808174 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38c4007e20 con 0x55569d9a2650 2023-11-08T16:35:31.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.458+0000 7f38d93d2740 10 client.6705 fetch_fsmap learned FSMap version 54 2023-11-08T16:35:31.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.458+0000 7f38d93d2740 1 -- 172.21.15.81:0/2314808174 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55569d87cd20 con 0x55569d9a2650 2023-11-08T16:35:31.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.458+0000 7f38aeffd640 1 -- 172.21.15.81:0/2314808174 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55569d907400 con 0x55569d9a2650 2023-11-08T16:35:31.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.458+0000 7f38aeffd640 1 -- 172.21.15.81:0/2314808174 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f38c4003610 con 0x55569d9a2650 2023-11-08T16:35:31.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.459+0000 7f38d93d2740 10 client.6705 fetch_fsmap finished waiting for FSMap version 54 2023-11-08T16:35:31.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.459+0000 7f38d93d2740 10 client.6705 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-08T16:35:31.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.459+0000 7f38d93d2740 20 client.6705 populate_metadata read hostname 'smithi081' 2023-11-08T16:35:31.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.459+0000 7f38d93d2740 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55569d99e3e0 0x55569d99e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.459+0000 7f38d93d2740 4 client.6705 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:35:31.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.459+0000 7f38d93d2740 1 -- 172.21.15.81:0/2314808174 --> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55569d8d9b20 con 0x55569d99e3e0 2023-11-08T16:35:31.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.459+0000 7f38cffff640 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55569d99e3e0 0x55569d99e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.459+0000 7f38cffff640 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55569d99e3e0 0x55569d99e810 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11e988c9ae80fac5 server_cookie=cc8081c75f91fc6d in_seq=0 out_seq=0 2023-11-08T16:35:31.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.459+0000 7f38aeffd640 10 client.6705 ms_handle_connect on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:31.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38aeffd640 1 -- 172.21.15.81:0/2314808174 <== mds.0 v2:172.21.15.81:6835/552116415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55569d8d9b20 con 0x55569d99e3e0 2023-11-08T16:35:31.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38aeffd640 10 client.6705 handle_command_reply: tid=0 2023-11-08T16:35:31.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38d93d2740 1 client.6705 shutdown 2023-11-08T16:35:31.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38d93d2740 1 -- 172.21.15.81:0/2314808174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2650 msgr2=0x55569d98d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:31.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38d93d2740 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2650 0x55569d98d2a0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f38c4001d90 tx=0x7f38c400d520 comp rx=0 tx=0).stop 2023-11-08T16:35:31.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38d93d2740 1 -- 172.21.15.81:0/2314808174 shutdown_connections 2023-11-08T16:35:31.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38d93d2740 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55569d99e3e0 0x55569d99e810 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38d93d2740 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d98d7e0 0x55569d98dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38d93d2740 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2650 0x55569d98d2a0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38aeffd640 0 client.6705 ms_handle_reset on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:31.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38d93d2740 1 --2- 172.21.15.81:0/2314808174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2270 0x55569d98bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.460+0000 7f38d93d2740 1 -- 172.21.15.81:0/2314808174 >> 172.21.15.81:0/2314808174 conn(0x55569d975710 msgr2=0x55569d9790f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:31.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.461+0000 7f38d93d2740 1 -- 172.21.15.81:0/2314808174 shutdown_connections 2023-11-08T16:35:31.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.461+0000 7f38d93d2740 1 -- 172.21.15.81:0/2314808174 wait complete. 2023-11-08T16:35:31.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.461+0000 7f38d93d2740 20 client.6705 trim_cache size 0 max 16384 2023-11-08T16:35:31.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.477+0000 7f38d93d2740 1 Processor -- start 2023-11-08T16:35:31.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.477+0000 7f38d93d2740 1 -- start start 2023-11-08T16:35:31.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.478+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d98d7e0 0x55569d98dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.478+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2270 0x55569d9c2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.478+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2650 0x55569d978d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.478+0000 7f38d93d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d98b900 con 0x55569d9a2270 2023-11-08T16:35:31.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.478+0000 7f38d93d2740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d99f450 con 0x55569d98d7e0 2023-11-08T16:35:31.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.478+0000 7f38d93d2740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55569d974b00 con 0x55569d9a2650 2023-11-08T16:35:31.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.478+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2270 0x55569d9c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.478+0000 7f38cf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2270 0x55569d9c2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45152/0 (socket says 172.21.15.81:45152) 2023-11-08T16:35:31.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.478+0000 7f38cf7fe640 1 -- 172.21.15.81:0/3963673451 learned_addr learned my addr 172.21.15.81:0/3963673451 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:31.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38d4de1640 1 --2- 172.21.15.81:0/3963673451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2650 0x55569d978d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38cffff640 1 --2- 172.21.15.81:0/3963673451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d98d7e0 0x55569d98dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38cf7fe640 1 -- 172.21.15.81:0/3963673451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2650 msgr2=0x55569d978d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:31.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38cf7fe640 1 --2- 172.21.15.81:0/3963673451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2650 0x55569d978d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38cf7fe640 1 -- 172.21.15.81:0/3963673451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d98d7e0 msgr2=0x55569d98dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:31.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38cf7fe640 1 --2- 172.21.15.81:0/3963673451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d98d7e0 0x55569d98dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38cf7fe640 1 -- 172.21.15.81:0/3963673451 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55569d910750 con 0x55569d9a2270 2023-11-08T16:35:31.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38cffff640 1 --2- 172.21.15.81:0/3963673451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d98d7e0 0x55569d98dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:35:31.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38cf7fe640 1 --2- 172.21.15.81:0/3963673451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2270 0x55569d9c2740 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f38c4001ce0 tx=0x7f38c400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96aa7a89c9fe6ffb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:31.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38b6ffd640 1 -- 172.21.15.81:0/3963673451 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c400dea0 con 0x55569d9a2270 2023-11-08T16:35:31.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38b6ffd640 1 -- 172.21.15.81:0/3963673451 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c400da20 con 0x55569d9a2270 2023-11-08T16:35:31.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.479+0000 7f38b6ffd640 1 -- 172.21.15.81:0/3963673451 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40050d0 con 0x55569d9a2270 2023-11-08T16:35:31.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.480+0000 7f38d93d2740 1 -- 172.21.15.81:0/3963673451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2270 msgr2=0x55569d9c2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:31.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.480+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3963673451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2270 0x55569d9c2740 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f38c4001ce0 tx=0x7f38c400b000 comp rx=0 tx=0).stop 2023-11-08T16:35:31.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.480+0000 7f38d93d2740 1 -- 172.21.15.81:0/3963673451 shutdown_connections 2023-11-08T16:35:31.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.480+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3963673451 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2650 0x55569d978d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.480+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3963673451 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d9a2270 0x55569d9c2740 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.480+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3963673451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d98d7e0 0x55569d98dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.480+0000 7f38d93d2740 1 -- 172.21.15.81:0/3963673451 >> 172.21.15.81:0/3963673451 conn(0x55569d975710 msgr2=0x55569d970990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:31.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.480+0000 7f38d93d2740 1 -- 172.21.15.81:0/3963673451 shutdown_connections 2023-11-08T16:35:31.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.480+0000 7f38d93d2740 1 -- 172.21.15.81:0/3963673451 wait complete. 2023-11-08T16:35:31.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.481+0000 7f38d93d2740 1 -- start start 2023-11-08T16:35:31.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.481+0000 7f38d93d2740 1 Processor -- start 2023-11-08T16:35:31.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d98d7e0 0x55569da2aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 0x55569d9a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38d93d2740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2650 0x55569d9a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38d93d2740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569d9a5220 con 0x55569d98d7e0 2023-11-08T16:35:31.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38d93d2740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569da2ee30 con 0x55569d9a2650 2023-11-08T16:35:31.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38d93d2740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55569d9d9240 con 0x55569d9a2270 2023-11-08T16:35:31.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38d93d2740 10 client.0 _ll_register_callbacks cb 0x7f38c4007c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:35:31.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38cffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d98d7e0 0x55569da2aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38cffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d98d7e0 0x55569da2aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:45154/0 (socket says 172.21.15.81:45154) 2023-11-08T16:35:31.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38cffff640 1 -- 172.21.15.81:0/3762258815 learned_addr learned my addr 172.21.15.81:0/3762258815 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:35:31.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38d4de1640 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2650 0x55569d9a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.482+0000 7f38cf7fe640 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 0x55569d9a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38cffff640 1 -- 172.21.15.81:0/3762258815 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 msgr2=0x55569d9a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:31.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38cffff640 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 0x55569d9a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38cffff640 1 -- 172.21.15.81:0/3762258815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2650 msgr2=0x55569d9a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:35:31.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38cffff640 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2650 0x55569d9a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38cffff640 1 -- 172.21.15.81:0/3762258815 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55569d87cd20 con 0x55569d98d7e0 2023-11-08T16:35:31.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38d4de1640 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2650 0x55569d9a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:35:31.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38cf7fe640 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 0x55569d9a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:35:31.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38cffff640 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d98d7e0 0x55569da2aab0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f38c00082c0 tx=0x7f38c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed96ed44e90eaaca server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:35:31.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:35:31.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38aeffd640 1 -- 172.21.15.81:0/3762258815 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c001a040 con 0x55569d98d7e0 2023-11-08T16:35:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38aeffd640 1 -- 172.21.15.81:0/3762258815 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f38c0023610 con 0x55569d98d7e0 2023-11-08T16:35:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38d93d2740 1 -- 172.21.15.81:0/3762258815 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f38c8083480 con 0x55569d98d7e0 2023-11-08T16:35:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38aeffd640 1 -- 172.21.15.81:0/3762258815 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f38c00237d0 con 0x55569d98d7e0 2023-11-08T16:35:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38d93d2740 10 client.6711 fetch_fsmap learned FSMap version 54 2023-11-08T16:35:31.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.483+0000 7f38d93d2740 1 -- 172.21.15.81:0/3762258815 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55569d8a2310 con 0x55569d98d7e0 2023-11-08T16:35:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.484+0000 7f38aeffd640 1 -- 172.21.15.81:0/3762258815 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c8083480 con 0x55569d98d7e0 2023-11-08T16:35:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.484+0000 7f38aeffd640 1 -- 172.21.15.81:0/3762258815 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 54) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f38c0029020 con 0x55569d98d7e0 2023-11-08T16:35:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.484+0000 7f38d93d2740 10 client.6711 fetch_fsmap finished waiting for FSMap version 54 2023-11-08T16:35:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.484+0000 7f38d93d2740 10 client.6711 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.c 2023-11-08T16:35:31.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.484+0000 7f38d93d2740 20 client.6711 populate_metadata read hostname 'smithi081' 2023-11-08T16:35:31.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.484+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55569da2e730 0x55569da2b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:35:31.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.484+0000 7f38d93d2740 4 client.6711 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-08T16:35:31.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.484+0000 7f38d93d2740 1 -- 172.21.15.81:0/3762258815 --> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55569d8d9840 con 0x55569da2e730 2023-11-08T16:35:31.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.484+0000 7f38cf7fe640 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55569da2e730 0x55569da2b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:35:31.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38aeffd640 10 client.6711 ms_handle_connect on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:31.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38cf7fe640 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55569da2e730 0x55569da2b590 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dd78309685b241a server_cookie=3d8c19387b6b4bc in_seq=0 out_seq=0 2023-11-08T16:35:31.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38aeffd640 1 -- 172.21.15.81:0/3762258815 <== mds.0 v2:172.21.15.81:6835/552116415 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55569d8d9840 con 0x55569da2e730 2023-11-08T16:35:31.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38aeffd640 10 client.6711 handle_command_reply: tid=0 2023-11-08T16:35:31.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38d93d2740 1 client.6711 shutdown 2023-11-08T16:35:31.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38d93d2740 1 -- 172.21.15.81:0/3762258815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d98d7e0 msgr2=0x55569da2aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:31.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d98d7e0 0x55569da2aab0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f38c00082c0 tx=0x7f38c0000bc0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38d93d2740 1 -- 172.21.15.81:0/3762258815 shutdown_connections 2023-11-08T16:35:31.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.81:6835/552116415,v1:172.21.15.81:6837/552116415] conn(0x55569da2e730 0x55569da2b590 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55569d9a2650 0x55569d9a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55569d9a2270 0x55569d9a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38aeffd640 0 client.6711 ms_handle_reset on v2:172.21.15.81:6835/552116415 2023-11-08T16:35:31.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38d93d2740 1 --2- 172.21.15.81:0/3762258815 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55569d98d7e0 0x55569da2aab0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38d93d2740 1 -- 172.21.15.81:0/3762258815 >> 172.21.15.81:0/3762258815 conn(0x55569d975710 msgr2=0x55569d970990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:31.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38d93d2740 1 -- 172.21.15.81:0/3762258815 shutdown_connections 2023-11-08T16:35:31.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.485+0000 7f38d93d2740 1 -- 172.21.15.81:0/3762258815 wait complete. 2023-11-08T16:35:31.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d93d2740 20 client.6711 trim_cache size 0 max 16384 2023-11-08T16:35:31.508 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:35:31.508 INFO:teuthology.orchestra.run.smithi081.stdout: "status": "no active scrubs running", 2023-11-08T16:35:31.508 INFO:teuthology.orchestra.run.smithi081.stdout: "scrubs": {} 2023-11-08T16:35:31.508 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:35:31.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 -- 172.21.15.81:0/1923186404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38bc06aea0 msgr2=0x7f38bc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:31.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38bc06aea0 0x7f38bc06d360 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f38c0001050 tx=0x7f38c00047e0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 -- 172.21.15.81:0/1923186404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d015ca40 msgr2=0x7f38d0147ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:35:31.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d015ca40 0x7f38d0147ec0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f38c802c6d0 tx=0x7f38c8009750 comp rx=0 tx=0).stop 2023-11-08T16:35:31.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 -- 172.21.15.81:0/1923186404 shutdown_connections 2023-11-08T16:35:31.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38bc06aea0 0x7f38bc06d360 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d015ca40 0x7f38d0147ec0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d015c5c0 0x7f38d0147980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 --2- 172.21.15.81:0/1923186404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38b8005180 0x7f38d0158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:35:31.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 -- 172.21.15.81:0/1923186404 >> 172.21.15.81:0/1923186404 conn(0x7f38d00cb4e0 msgr2=0x7f38d00b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:35:31.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 -- 172.21.15.81:0/1923186404 shutdown_connections 2023-11-08T16:35:31.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:31.486+0000 7f38d6844640 1 -- 172.21.15.81:0/1923186404 wait complete. 2023-11-08T16:35:31.512 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-08T16:35:31.513 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/0 2023-11-08T16:35:31.513 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:35:31.514 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:35:31.514 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/0" 2023-11-08T16:35:31.514 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:35:31.515 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:35:31.515 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:35:31.515 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:35:31.516 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:35:31.516 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:35:31.516 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:35:31.517 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:35:31.517 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:35:31.517 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:35:31.518 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:35:35.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:35:35.134 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/1 2023-11-08T16:35:35.135 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:35:35.135 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:35:35.136 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/1" 2023-11-08T16:35:35.136 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:35:35.137 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:35:35.137 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:35:35.137 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:35:35.138 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:35:35.138 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:35:35.138 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:35:35.139 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:35:35.139 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:35:35.139 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:35:35.140 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:35:38.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:35:38.757 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/2 2023-11-08T16:35:38.758 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:35:38.758 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:35:38.759 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/2" 2023-11-08T16:35:38.759 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:35:38.759 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:35:38.760 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:35:38.760 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:35:38.760 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:35:38.761 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:35:38.761 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:35:38.761 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:35:38.762 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:35:38.762 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:35:38.762 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:35:42.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:35:42.425 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/3 2023-11-08T16:35:42.425 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:35:42.426 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:35:42.426 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/3" 2023-11-08T16:35:42.427 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:35:42.427 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:35:42.427 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:35:42.428 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:35:42.428 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:35:42.428 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:35:42.429 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:35:42.429 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:35:42.429 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:35:42.430 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:35:42.430 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:35:46.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:35:46.014 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/4 2023-11-08T16:35:46.015 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:35:46.015 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:35:46.016 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/4" 2023-11-08T16:35:46.016 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:35:46.016 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:35:46.017 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:35:46.017 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:35:46.018 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:35:46.018 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:35:46.018 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:35:46.019 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:35:46.019 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:35:46.019 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:35:46.020 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:35:47.794 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:35:47.801 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:35:49.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:35:49.683 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/5 2023-11-08T16:35:49.683 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:35:49.684 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:35:49.685 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/5" 2023-11-08T16:35:49.685 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:35:49.686 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:35:49.687 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:35:49.687 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:35:49.688 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:35:49.688 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:35:49.689 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:35:49.690 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:35:49.690 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:35:49.691 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:35:49.691 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:35:53.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:35:53.311 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/6 2023-11-08T16:35:53.311 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:35:53.312 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:35:53.312 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/6" 2023-11-08T16:35:53.312 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:35:53.313 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:35:53.313 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:35:53.313 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:35:53.314 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:35:53.314 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:35:53.314 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:35:53.315 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:35:53.315 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:35:53.315 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:35:53.316 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:35:56.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:35:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:35:56.960 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/7 2023-11-08T16:35:56.961 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:35:56.961 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:35:56.962 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/7" 2023-11-08T16:35:56.962 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:35:56.962 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:35:56.963 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:35:56.963 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:35:56.963 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:35:56.964 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:35:56.964 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:35:56.965 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:35:56.965 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:35:56.965 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:35:56.966 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:00.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:00.565 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/8 2023-11-08T16:36:00.566 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:00.566 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:00.567 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/8" 2023-11-08T16:36:00.567 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:00.568 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:00.568 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:00.568 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:00.569 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:00.569 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:00.569 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:00.570 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:00.570 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:00.570 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:00.571 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:04.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:04.324 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/9 2023-11-08T16:36:04.325 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:04.325 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:04.326 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/9" 2023-11-08T16:36:04.326 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:04.326 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:04.327 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:04.327 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:04.327 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:04.328 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:04.328 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:04.329 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:04.329 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:04.329 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:04.330 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:08.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:08.053 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/10 2023-11-08T16:36:08.054 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:08.054 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:08.054 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/10" 2023-11-08T16:36:08.055 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:08.055 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:08.056 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:08.056 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:08.056 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:08.057 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:08.057 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:08.057 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:08.058 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:08.058 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:08.059 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:11.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:11.664 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/11 2023-11-08T16:36:11.664 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:11.665 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:11.665 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/11" 2023-11-08T16:36:11.666 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:11.666 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:11.666 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:11.667 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:11.667 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:11.668 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:11.668 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:11.669 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:11.669 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:11.669 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:11.670 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:15.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:15.314 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/12 2023-11-08T16:36:15.315 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:15.315 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:15.316 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/12" 2023-11-08T16:36:15.316 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:15.316 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:15.317 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:15.317 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:15.317 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:15.318 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:15.318 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:15.318 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:15.319 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:15.319 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:15.319 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:17.940 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:36:17.949 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:36:18.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:18.946 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/13 2023-11-08T16:36:18.946 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:18.947 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:18.947 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/13" 2023-11-08T16:36:18.948 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:18.948 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:18.948 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:18.949 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:18.949 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:18.949 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:18.950 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:18.950 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:18.951 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:18.951 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:18.951 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:22.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:22.532 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/14 2023-11-08T16:36:22.533 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:22.533 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:22.534 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/14" 2023-11-08T16:36:22.534 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:22.534 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:22.535 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:22.535 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:22.536 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:22.536 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:22.536 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:22.537 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:22.537 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:22.538 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:22.538 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:26.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:26.121 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/15 2023-11-08T16:36:26.122 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:26.122 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:26.123 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/15" 2023-11-08T16:36:26.123 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:26.123 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:26.124 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:26.124 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:26.124 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:26.125 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:26.125 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:26.125 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:26.126 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:26.126 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:26.127 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:29.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:29.779 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/16 2023-11-08T16:36:29.780 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:29.780 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:29.781 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/16" 2023-11-08T16:36:29.781 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:29.781 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:29.782 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:29.782 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:29.783 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:29.783 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:29.783 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:29.784 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:29.784 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:29.784 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:29.785 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:33.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:33.451 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/17 2023-11-08T16:36:33.451 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:33.452 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:33.452 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/17" 2023-11-08T16:36:33.452 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:33.453 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:33.453 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:33.453 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:33.454 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:33.454 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:33.455 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:33.455 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:33.455 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:33.456 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:33.456 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:37.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:37.115 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/18 2023-11-08T16:36:37.115 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:37.116 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:37.116 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/18" 2023-11-08T16:36:37.116 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:37.117 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:37.118 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:37.118 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:37.119 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:37.120 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:37.120 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:37.121 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:37.121 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:37.122 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:37.122 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:40.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:40.801 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/19 2023-11-08T16:36:40.802 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:40.802 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:40.802 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/19" 2023-11-08T16:36:40.803 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:40.803 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:40.804 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:40.804 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:40.804 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:40.805 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:40.805 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:40.805 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:40.806 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:40.806 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:40.807 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:44.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:44.483 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/20 2023-11-08T16:36:44.484 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:44.484 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:44.484 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/20" 2023-11-08T16:36:44.485 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:44.485 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:44.485 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:44.486 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:44.486 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:44.487 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:44.487 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:44.487 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:44.488 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:44.488 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:44.488 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:48.087 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:36:48.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:48.093 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:36:48.101 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/21 2023-11-08T16:36:48.102 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:48.102 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:48.102 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/21" 2023-11-08T16:36:48.103 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:48.103 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:48.104 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:48.104 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:48.104 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:48.105 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:48.105 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:48.105 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:48.106 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:48.106 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:48.106 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:51.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:51.723 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/22 2023-11-08T16:36:51.723 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:51.724 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:51.724 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/22" 2023-11-08T16:36:51.725 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:51.725 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:51.726 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:51.726 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:51.726 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:51.727 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:51.727 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:51.727 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:51.728 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:51.728 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:51.728 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:55.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:55.358 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/23 2023-11-08T16:36:55.358 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:55.359 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:55.359 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/23" 2023-11-08T16:36:55.360 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:55.360 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:55.360 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:55.361 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:55.361 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:55.361 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:55.362 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:55.362 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:55.362 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:55.363 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:55.363 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:36:58.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:36:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:36:59.002 INFO:tasks.cephfs.mount:Validating 6291456 bytes from subdir/24 2023-11-08T16:36:59.003 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:36:59.003 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:36:59.003 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/subdir/24" 2023-11-08T16:36:59.004 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:36:59.004 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:36:59.004 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 6291456: 2023-11-08T16:36:59.005 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:36:59.005 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 6291456 2023-11-08T16:36:59.006 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:36:59.006 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:36:59.006 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:36:59.007 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:36:59.007 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:36:59.008 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:37:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:37:02.810 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T16:37:02.811 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:37:02.962 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:37:03.090 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T16:37:03.091 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:37:03.091 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:37:03.150 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:03.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.353+0000 7f51e713c640 1 Processor -- start 2023-11-08T16:37:03.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.354+0000 7f51e713c640 1 -- start start 2023-11-08T16:37:03.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.355+0000 7f51e713c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f51e015aac0 0x7f51e00711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.355+0000 7f51e713c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f51e00716e0 0x7f51e00ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.355+0000 7f51e713c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51e00cf250 0x7f51e006b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.355+0000 7f51e713c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e006f650 con 0x7f51e015aac0 2023-11-08T16:37:03.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.355+0000 7f51e713c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e00cf6c0 con 0x7f51e00716e0 2023-11-08T16:37:03.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.355+0000 7f51e713c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e0070470 con 0x7f51e00cf250 2023-11-08T16:37:03.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.355+0000 7f51e613a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f51e015aac0 0x7f51e00711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:03.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.356+0000 7f51e613a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f51e015aac0 0x7f51e00711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37332/0 (socket says 172.21.15.81:37332) 2023-11-08T16:37:03.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.356+0000 7f51e613a640 1 -- 172.21.15.81:0/2873662733 learned_addr learned my addr 172.21.15.81:0/2873662733 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:03.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.356+0000 7f51e693b640 1 --2- 172.21.15.81:0/2873662733 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51e00cf250 0x7f51e006b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:03.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.356+0000 7f51e613a640 1 -- 172.21.15.81:0/2873662733 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51e00cf250 msgr2=0x7f51e006b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:03.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.356+0000 7f51e613a640 1 --2- 172.21.15.81:0/2873662733 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51e00cf250 0x7f51e006b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.356+0000 7f51e613a640 1 -- 172.21.15.81:0/2873662733 >> v2:172.21.15.93:3300/0 conn(0x7f51e00716e0 msgr2=0x7f51e00ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:03.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.356+0000 7f51e613a640 1 --2- 172.21.15.81:0/2873662733 >> v2:172.21.15.93:3300/0 conn(0x7f51e00716e0 0x7f51e00ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.356+0000 7f51e613a640 1 -- 172.21.15.81:0/2873662733 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e006d580 con 0x7f51e015aac0 2023-11-08T16:37:03.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.357+0000 7f51e613a640 1 --2- 172.21.15.81:0/2873662733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51e015aac0 0x7f51e00711a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f51e015e790 tx=0x7f51dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249ecde7d945a447 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:03.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.357+0000 7f51e5138640 1 -- 172.21.15.81:0/2873662733 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc031870 con 0x7f51e015aac0 2023-11-08T16:37:03.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.357+0000 7f51e5138640 1 -- 172.21.15.81:0/2873662733 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc0319d0 con 0x7f51e015aac0 2023-11-08T16:37:03.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.358+0000 7f51e5138640 1 -- 172.21.15.81:0/2873662733 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc031d60 con 0x7f51e015aac0 2023-11-08T16:37:03.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.358+0000 7f51e713c640 1 -- 172.21.15.81:0/2873662733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51e015aac0 msgr2=0x7f51e00711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:03.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.358+0000 7f51e713c640 1 --2- 172.21.15.81:0/2873662733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51e015aac0 0x7f51e00711a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f51e015e790 tx=0x7f51dc02b910 comp rx=0 tx=0).stop 2023-11-08T16:37:03.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.358+0000 7f51e713c640 1 -- 172.21.15.81:0/2873662733 shutdown_connections 2023-11-08T16:37:03.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.358+0000 7f51e713c640 1 --2- 172.21.15.81:0/2873662733 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51e00cf250 0x7f51e006b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.358+0000 7f51e713c640 1 --2- 172.21.15.81:0/2873662733 >> v2:172.21.15.93:3300/0 conn(0x7f51e00716e0 0x7f51e00ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.358+0000 7f51e713c640 1 --2- 172.21.15.81:0/2873662733 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51e015aac0 0x7f51e00711a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.358+0000 7f51e713c640 1 -- 172.21.15.81:0/2873662733 >> 172.21.15.81:0/2873662733 conn(0x7f51e006a4a0 msgr2=0x7f51e006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:03.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.358+0000 7f51e713c640 1 -- 172.21.15.81:0/2873662733 shutdown_connections 2023-11-08T16:37:03.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.358+0000 7f51e713c640 1 -- 172.21.15.81:0/2873662733 wait complete. 2023-11-08T16:37:03.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.360+0000 7f51e713c640 1 Processor -- start 2023-11-08T16:37:03.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.360+0000 7f51e713c640 1 -- start start 2023-11-08T16:37:03.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.361+0000 7f51e713c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51e00716e0 0x7f51e01f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.361+0000 7f51e713c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51e00cf250 0x7f51e0146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.361+0000 7f51e713c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51e015aac0 0x7f51e00b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.361+0000 7f51e713c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e01f63f0 con 0x7f51e015aac0 2023-11-08T16:37:03.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.361+0000 7f51e713c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e01f66d0 con 0x7f51e00716e0 2023-11-08T16:37:03.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.361+0000 7f51e713c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e01f69b0 con 0x7f51e00cf250 2023-11-08T16:37:03.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e693b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51e015aac0 0x7f51e00b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:03.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e693b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51e015aac0 0x7f51e00b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37340/0 (socket says 172.21.15.81:37340) 2023-11-08T16:37:03.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e693b640 1 -- 172.21.15.81:0/3585976073 learned_addr learned my addr 172.21.15.81:0/3585976073 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:03.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e613a640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51e00716e0 0x7f51e01f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:03.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e5939640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51e00cf250 0x7f51e0146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:03.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e693b640 1 -- 172.21.15.81:0/3585976073 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51e00cf250 msgr2=0x7f51e0146550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:03.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e693b640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51e00cf250 0x7f51e0146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e693b640 1 -- 172.21.15.81:0/3585976073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51e00716e0 msgr2=0x7f51e01f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:03.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e693b640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51e00716e0 0x7f51e01f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e693b640 1 -- 172.21.15.81:0/3585976073 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0094d0 con 0x7f51e015aac0 2023-11-08T16:37:03.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e613a640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51e00716e0 0x7f51e01f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:03.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51e693b640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51e015aac0 0x7f51e00b1990 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f51d800cd70 tx=0x7f51d800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:03.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.362+0000 7f51d6ffd640 1 -- 172.21.15.81:0/3585976073 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8003b90 con 0x7f51e015aac0 2023-11-08T16:37:03.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.363+0000 7f51d6ffd640 1 -- 172.21.15.81:0/3585976073 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d8003cf0 con 0x7f51e015aac0 2023-11-08T16:37:03.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.363+0000 7f51d6ffd640 1 -- 172.21.15.81:0/3585976073 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d8004240 con 0x7f51e015aac0 2023-11-08T16:37:03.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.363+0000 7f51e713c640 1 -- 172.21.15.81:0/3585976073 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e01f6c90 con 0x7f51e015aac0 2023-11-08T16:37:03.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.363+0000 7f51e713c640 1 -- 172.21.15.81:0/3585976073 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e00acfc0 con 0x7f51e015aac0 2023-11-08T16:37:03.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.364+0000 7f51e713c640 1 -- 172.21.15.81:0/3585976073 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e0150470 con 0x7f51e015aac0 2023-11-08T16:37:03.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.365+0000 7f51d6ffd640 1 -- 172.21.15.81:0/3585976073 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f51d80043a0 con 0x7f51e015aac0 2023-11-08T16:37:03.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.366+0000 7f51d6ffd640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f51c406ae70 0x7f51c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.366+0000 7f51d6ffd640 1 -- 172.21.15.81:0/3585976073 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f51d8012070 con 0x7f51e015aac0 2023-11-08T16:37:03.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.366+0000 7f51e613a640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f51c406ae70 0x7f51c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:03.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.367+0000 7f51e613a640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f51c406ae70 0x7f51c406d330 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f51dc036000 tx=0x7f51dc03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:03.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.371+0000 7f51d6ffd640 1 -- 172.21.15.81:0/3585976073 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51e0150470 con 0x7f51e015aac0 2023-11-08T16:37:03.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.562+0000 7f51e713c640 1 -- 172.21.15.81:0/3585976073 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51e0150850 con 0x7f51e015aac0 2023-11-08T16:37:03.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.565+0000 7f51d6ffd640 1 -- 172.21.15.81:0/3585976073 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18768 (secure 0 0 0) 0x7f51e0150850 con 0x7f51e015aac0 2023-11-08T16:37:03.569 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:03.569 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":68,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:34:53.906443+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:32:41.268027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T16:32:42.449607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:32:53.335342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.e","mappings":[{"from":0,"to":6}]},{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"9.11","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/321347143":"2023-11-09T16:34:45.940255+0000","172.21.15.81:6837/1446454901":"2023-11-09T16:34:43.853227+0000","172.21.15.81:6837/494395478":"2023-11-09T16:34:53.172859+0000","172.21.15.81:6835/494395478":"2023-11-09T16:34:53.172859+0000","172.21.15.81:6835/321347143":"2023-11-09T16:34:45.940255+0000","172.21.15.81:6835/1446454901":"2023-11-09T16:34:43.853227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:03.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.568+0000 7f51e713c640 1 -- 172.21.15.81:0/3585976073 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f51c406ae70 msgr2=0x7f51c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:03.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.568+0000 7f51e713c640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f51c406ae70 0x7f51c406d330 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f51dc036000 tx=0x7f51dc03c040 comp rx=0 tx=0).stop 2023-11-08T16:37:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.568+0000 7f51e713c640 1 -- 172.21.15.81:0/3585976073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51e015aac0 msgr2=0x7f51e00b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.568+0000 7f51e713c640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51e015aac0 0x7f51e00b1990 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f51d800cd70 tx=0x7f51d800a660 comp rx=0 tx=0).stop 2023-11-08T16:37:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.568+0000 7f51e713c640 1 -- 172.21.15.81:0/3585976073 shutdown_connections 2023-11-08T16:37:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.568+0000 7f51e713c640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f51c406ae70 0x7f51c406d330 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.568+0000 7f51e713c640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51e015aac0 0x7f51e00b1990 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.568+0000 7f51e713c640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51e00cf250 0x7f51e0146550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.568+0000 7f51e713c640 1 --2- 172.21.15.81:0/3585976073 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51e00716e0 0x7f51e01f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.568+0000 7f51e713c640 1 -- 172.21.15.81:0/3585976073 >> 172.21.15.81:0/3585976073 conn(0x7f51e006a4a0 msgr2=0x7f51e0070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:03.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.568+0000 7f51e713c640 1 -- 172.21.15.81:0/3585976073 shutdown_connections 2023-11-08T16:37:03.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.569+0000 7f51e713c640 1 -- 172.21.15.81:0/3585976073 wait complete. 2023-11-08T16:37:03.589 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:32:41.268027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:32:42.449607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:32:53.335342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:37:03.590 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:37:03.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.803+0000 7f6ebf773640 1 Processor -- start 2023-11-08T16:37:03.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.803+0000 7f6ebf773640 1 -- start start 2023-11-08T16:37:03.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.804+0000 7f6ebf773640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6eb815a0e0 0x7f6eb80ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.804+0000 7f6ebf773640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb80ce960 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.805+0000 7f6ebf773640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eb80c9480 con 0x7f6eb80ca750 2023-11-08T16:37:03.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.805+0000 7f6ebf773640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb8071980 con 0x7f6eb815a0e0 2023-11-08T16:37:03.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.805+0000 7f6ebd50f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6eb815a0e0 0x7f6eb80ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:03.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.805+0000 7f6ebcd0e640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f6eb80c9bf0 0x7f6eb80ca040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51462/0 (socket says 172.21.15.81:51462) 2023-11-08T16:37:03.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.805+0000 7f6ebcd0e640 1 -- 172.21.15.81:0/261499267 learned_addr learned my addr 172.21.15.81:0/261499267 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:03.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.805+0000 7f6ebd50f640 1 -- 172.21.15.81:0/261499267 >> v1:172.21.15.93:6789/0 conn(0x7f6eb80ca750 legacy=0x7f6eb806b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:03.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.805+0000 7f6ebd50f640 1 -- 172.21.15.81:0/261499267 >> v1:172.21.15.81:6789/0 conn(0x7f6eb80c9bf0 legacy=0x7f6eb80ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:03.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.805+0000 7f6ebd50f640 1 -- 172.21.15.81:0/261499267 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb815a4c0 con 0x7f6eb815a0e0 2023-11-08T16:37:03.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.805+0000 7f6eaffff640 1 -- 172.21.15.81:0/261499267 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3920697856 0 0) 0x7f6ea8001110 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.806+0000 7f6ebd50f640 1 --2- 172.21.15.81:0/261499267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb815a0e0 0x7f6eb80ccc10 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40097a0 tx=0x7f6eb402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fc1074cb4ab634 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:03.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.807+0000 7f6eaffff640 1 -- 172.21.15.81:0/261499267 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4030610 con 0x7f6eb815a0e0 2023-11-08T16:37:03.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.807+0000 7f6eaffff640 1 -- 172.21.15.81:0/261499267 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4030770 con 0x7f6eb815a0e0 2023-11-08T16:37:03.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.807+0000 7f6eaffff640 1 -- 172.21.15.81:0/261499267 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4030b00 con 0x7f6eb815a0e0 2023-11-08T16:37:03.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.807+0000 7f6ebf773640 1 -- 172.21.15.81:0/261499267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb815a0e0 msgr2=0x7f6eb80ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:03.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.807+0000 7f6ebf773640 1 --2- 172.21.15.81:0/261499267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb815a0e0 0x7f6eb80ccc10 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40097a0 tx=0x7f6eb402b960 comp rx=0 tx=0).stop 2023-11-08T16:37:03.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.807+0000 7f6ebf773640 1 -- 172.21.15.81:0/261499267 shutdown_connections 2023-11-08T16:37:03.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.807+0000 7f6ebf773640 1 --2- 172.21.15.81:0/261499267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb815a0e0 0x7f6eb80ccc10 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.807+0000 7f6ebf773640 1 -- 172.21.15.81:0/261499267 >> 172.21.15.81:0/261499267 conn(0x7f6eb80cb580 msgr2=0x7f6eb80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:03.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.807+0000 7f6ebf773640 1 -- 172.21.15.81:0/261499267 shutdown_connections 2023-11-08T16:37:03.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.808+0000 7f6ebf773640 1 -- 172.21.15.81:0/261499267 wait complete. 2023-11-08T16:37:03.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.809+0000 7f6ebf773640 1 Processor -- start 2023-11-08T16:37:03.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.809+0000 7f6ebf773640 1 -- start start 2023-11-08T16:37:03.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.810+0000 7f6ebf773640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80c9bf0 0x7f6eb8154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.810+0000 7f6ebf773640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6eb80ca750 0x7f6eb80c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.810+0000 7f6ebf773640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb815a0e0 0x7f6eb815b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.810+0000 7f6ebf773640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80c6110 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.810+0000 7f6ebf773640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb8155d30 con 0x7f6eb815a0e0 2023-11-08T16:37:03.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.810+0000 7f6ebf773640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb8152b70 con 0x7f6eb80ca750 2023-11-08T16:37:03.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebd50f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80c9bf0 0x7f6eb8154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:03.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebd50f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80c9bf0 0x7f6eb8154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37362/0 (socket says 172.21.15.81:37362) 2023-11-08T16:37:03.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebd50f640 1 -- 172.21.15.81:0/2876010609 learned_addr learned my addr 172.21.15.81:0/2876010609 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:03.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebcd0e640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6eb80ca750 0x7f6eb80c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:03.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebdd10640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb815a0e0 0x7f6eb815b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:03.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebd50f640 1 -- 172.21.15.81:0/2876010609 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6eb80ca750 msgr2=0x7f6eb80c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:03.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebd50f640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6eb80ca750 0x7f6eb80c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebd50f640 1 -- 172.21.15.81:0/2876010609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb815a0e0 msgr2=0x7f6eb815b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:03.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebd50f640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb815a0e0 0x7f6eb815b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:03.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebd50f640 1 -- 172.21.15.81:0/2876010609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb4009400 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebdd10640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb815a0e0 0x7f6eb815b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:03.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.811+0000 7f6ebd50f640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80c9bf0 0x7f6eb8154ff0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40098c0 tx=0x7f6eb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:03.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.812+0000 7f6eadffb640 1 -- 172.21.15.81:0/2876010609 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb403e070 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.812+0000 7f6eadffb640 1 -- 172.21.15.81:0/2876010609 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4039800 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.812+0000 7f6eadffb640 1 -- 172.21.15.81:0/2876010609 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4039bb0 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.812+0000 7f6ebf773640 1 -- 172.21.15.81:0/2876010609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb8155f80 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.812+0000 7f6ebf773640 1 -- 172.21.15.81:0/2876010609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb8153000 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.813+0000 7f6ebf773640 1 -- 172.21.15.81:0/2876010609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb81564a0 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.814+0000 7f6eadffb640 1 -- 172.21.15.81:0/2876010609 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6eb4042070 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.815+0000 7f6eadffb640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06aea0 0x7f6e9c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:03.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.815+0000 7f6eadffb640 1 -- 172.21.15.81:0/2876010609 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f6eb40ab4b0 con 0x7f6eb80c9bf0 2023-11-08T16:37:03.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.815+0000 7f6ebcd0e640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06aea0 0x7f6e9c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:03.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.816+0000 7f6ebcd0e640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06aea0 0x7f6e9c06d360 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80b03e0 tx=0x7f6ea8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:03.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:03.819+0000 7f6eadffb640 1 -- 172.21.15.81:0/2876010609 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6eb81564a0 con 0x7f6eb80c9bf0 2023-11-08T16:37:04.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.009+0000 7f6ebf773640 1 -- 172.21.15.81:0/2876010609 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6eb8152940 con 0x7f6eb80c9bf0 2023-11-08T16:37:04.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.010+0000 7f6eadffb640 1 -- 172.21.15.81:0/2876010609 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6eb8152940 con 0x7f6eb80c9bf0 2023-11-08T16:37:04.013 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:37:04.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 -- 172.21.15.81:0/2876010609 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06aea0 msgr2=0x7f6e9c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06aea0 0x7f6e9c06d360 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6eb80b03e0 tx=0x7f6ea8009040 comp rx=0 tx=0).stop 2023-11-08T16:37:04.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 -- 172.21.15.81:0/2876010609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80c9bf0 msgr2=0x7f6eb8154ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80c9bf0 0x7f6eb8154ff0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f6eb40098c0 tx=0x7f6eb40046c0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 -- 172.21.15.81:0/2876010609 shutdown_connections 2023-11-08T16:37:04.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e9c06aea0 0x7f6e9c06d360 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eb815a0e0 0x7f6eb815b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6eb80ca750 0x7f6eb80c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 --2- 172.21.15.81:0/2876010609 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6eb80c9bf0 0x7f6eb8154ff0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 -- 172.21.15.81:0/2876010609 >> 172.21.15.81:0/2876010609 conn(0x7f6eb80cb580 msgr2=0x7f6eb806b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:04.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 -- 172.21.15.81:0/2876010609 shutdown_connections 2023-11-08T16:37:04.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.013+0000 7f6ebf773640 1 -- 172.21.15.81:0/2876010609 wait complete. 2023-11-08T16:37:04.033 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:37:04.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.241+0000 7fa1e2a00640 1 Processor -- start 2023-11-08T16:37:04.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.241+0000 7fa1e2a00640 1 -- start start 2023-11-08T16:37:04.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.243+0000 7fa1e2a00640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa1dc0ce0d0 0x7fa1dc0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.243+0000 7fa1e2a00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc0cda10 con 0x7fa1dc15f280 2023-11-08T16:37:04.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.243+0000 7fa1e2a00640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1dc06fa30 con 0x7fa1dc15bfc0 2023-11-08T16:37:04.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.243+0000 7fa1e2a00640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1dc0ce4b0 con 0x7fa1dc0ce0d0 2023-11-08T16:37:04.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.243+0000 7fa1e0f9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa1dc15f280 0x7fa1dc1516e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51468/0 (socket says 172.21.15.81:51468) 2023-11-08T16:37:04.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.243+0000 7fa1e0f9d640 1 -- 172.21.15.81:0/3261350178 learned_addr learned my addr 172.21.15.81:0/3261350178 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:04.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.243+0000 7fa1dbfff640 1 --2- 172.21.15.81:0/3261350178 >> v2:172.21.15.93:3300/0 conn(0x7fa1dc0ce0d0 0x7fa1dc0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531482960 0 0) 0x7fa1dc0cda10 con 0x7fa1dc15f280 2023-11-08T16:37:04.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1c4003520 con 0x7fa1dc15f280 2023-11-08T16:37:04.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3108526057 0 0) 0x7fa1c4003520 con 0x7fa1dc15f280 2023-11-08T16:37:04.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 >> v2:172.21.15.93:3300/0 conn(0x7fa1dc0ce0d0 msgr2=0x7fa1dc0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:04.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 --2- 172.21.15.81:0/3261350178 >> v2:172.21.15.93:3300/0 conn(0x7fa1dc0ce0d0 0x7fa1dc0caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 >> v1:172.21.15.93:6789/0 conn(0x7fa1dc15bfc0 legacy=0x7fa1dc15eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc06fca0 con 0x7fa1dc15f280 2023-11-08T16:37:04.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fa1d40019f0 con 0x7fa1dc15f280 2023-11-08T16:37:04.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 >> v1:172.21.15.81:6789/0 conn(0x7fa1dc15f280 legacy=0x7fa1dc1516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1c4003a30 0x7fa1c4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1c40047c0 0x7fa1c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c40051d0 0x7fa1c400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0ce4b0 con 0x7fa1c40051d0 2023-11-08T16:37:04.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c400c100 con 0x7fa1c40047c0 2023-11-08T16:37:04.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1c400c2c0 con 0x7fa1c4003a30 2023-11-08T16:37:04.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.244+0000 7fa1e0f9d640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c40051d0 0x7fa1c400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.245+0000 7fa1db7fe640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1c40047c0 0x7fa1c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.245+0000 7fa1e0f9d640 1 -- 172.21.15.81:0/3261350178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1c4003a30 msgr2=0x7fa1c4003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:04.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.245+0000 7fa1dbfff640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1c4003a30 0x7fa1c4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.245+0000 7fa1e0f9d640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1c4003a30 0x7fa1c4003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.245+0000 7fa1e0f9d640 1 -- 172.21.15.81:0/3261350178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1c40047c0 msgr2=0x7fa1c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:04.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.245+0000 7fa1e0f9d640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1c40047c0 0x7fa1c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.245+0000 7fa1e0f9d640 1 -- 172.21.15.81:0/3261350178 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c4003520 con 0x7fa1c40051d0 2023-11-08T16:37:04.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.245+0000 7fa1db7fe640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1c40047c0 0x7fa1c4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:04.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.246+0000 7fa1e0f9d640 1 -- 172.21.15.81:0/3261350178 reap_dead start 2023-11-08T16:37:04.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.246+0000 7fa1e0f9d640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c40051d0 0x7fa1c400b980 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400a760 tx=0x7fa1c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3880717a0f35891 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:04.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.246+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4034630 con 0x7fa1c40051d0 2023-11-08T16:37:04.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.246+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d402f070 con 0x7fa1c40051d0 2023-11-08T16:37:04.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.246+0000 7fa1daffd640 1 -- 172.21.15.81:0/3261350178 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4034a10 con 0x7fa1c40051d0 2023-11-08T16:37:04.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.247+0000 7fa1e2a00640 1 -- 172.21.15.81:0/3261350178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c40051d0 msgr2=0x7fa1c400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.247+0000 7fa1e2a00640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c40051d0 0x7fa1c400b980 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400a760 tx=0x7fa1c40057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.247+0000 7fa1e2a00640 1 -- 172.21.15.81:0/3261350178 shutdown_connections 2023-11-08T16:37:04.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.247+0000 7fa1e2a00640 1 --2- 172.21.15.81:0/3261350178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1c40051d0 0x7fa1c400b980 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.247+0000 7fa1e2a00640 1 -- 172.21.15.81:0/3261350178 >> 172.21.15.81:0/3261350178 conn(0x7fa1dc0cb580 msgr2=0x7fa1dc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:04.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.247+0000 7fa1e2a00640 1 -- 172.21.15.81:0/3261350178 shutdown_connections 2023-11-08T16:37:04.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.247+0000 7fa1e2a00640 1 -- 172.21.15.81:0/3261350178 wait complete. 2023-11-08T16:37:04.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.249+0000 7fa1e2a00640 1 Processor -- start 2023-11-08T16:37:04.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.249+0000 7fa1e2a00640 1 -- start start 2023-11-08T16:37:04.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.250+0000 7fa1e2a00640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1c40051d0 0x7fa1dc1f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e2a00640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1dc1530e0 0x7fa1dc0c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e2a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1dc0c9680 0x7fa1dc0c9b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e2a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0c3ee0 con 0x7fa1dc0c9680 2023-11-08T16:37:04.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e2a00640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0ca040 con 0x7fa1dc1530e0 2023-11-08T16:37:04.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e2a00640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1dc0c30a0 con 0x7fa1c40051d0 2023-11-08T16:37:04.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e0f9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1dc0c9680 0x7fa1dc0c9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e0f9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1dc0c9680 0x7fa1dc0c9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37384/0 (socket says 172.21.15.81:37384) 2023-11-08T16:37:04.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e0f9d640 1 -- 172.21.15.81:0/1084236656 learned_addr learned my addr 172.21.15.81:0/1084236656 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:04.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1db7fe640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1dc1530e0 0x7fa1dc0c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e0f9d640 1 -- 172.21.15.81:0/1084236656 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1c40051d0 msgr2=0x7fa1dc1f3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e0f9d640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1c40051d0 0x7fa1dc1f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e0f9d640 1 -- 172.21.15.81:0/1084236656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1dc1530e0 msgr2=0x7fa1dc0c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e0f9d640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1dc1530e0 0x7fa1dc0c2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1e0f9d640 1 -- 172.21.15.81:0/1084236656 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc153560 con 0x7fa1dc0c9680 2023-11-08T16:37:04.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.251+0000 7fa1db7fe640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1dc1530e0 0x7fa1dc0c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:04.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.252+0000 7fa1e0f9d640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1dc0c9680 0x7fa1dc0c9b00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400a760 tx=0x7fa1d4001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:04.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.252+0000 7fa1d8ff9640 1 -- 172.21.15.81:0/1084236656 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4001730 con 0x7fa1dc0c9680 2023-11-08T16:37:04.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.252+0000 7fa1d8ff9640 1 -- 172.21.15.81:0/1084236656 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d4001890 con 0x7fa1dc0c9680 2023-11-08T16:37:04.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.252+0000 7fa1d8ff9640 1 -- 172.21.15.81:0/1084236656 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4004d80 con 0x7fa1dc0c9680 2023-11-08T16:37:04.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.252+0000 7fa1e2a00640 1 -- 172.21.15.81:0/1084236656 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1dc0c32d0 con 0x7fa1dc0c9680 2023-11-08T16:37:04.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.252+0000 7fa1e2a00640 1 -- 172.21.15.81:0/1084236656 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1dc0c3890 con 0x7fa1dc0c9680 2023-11-08T16:37:04.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.253+0000 7fa1e2a00640 1 -- 172.21.15.81:0/1084236656 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1dc04ece0 con 0x7fa1dc0c9680 2023-11-08T16:37:04.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.254+0000 7fa1d8ff9640 1 -- 172.21.15.81:0/1084236656 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa1d4031030 con 0x7fa1dc0c9680 2023-11-08T16:37:04.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.255+0000 7fa1d8ff9640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1c806ae70 0x7fa1c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.255+0000 7fa1d8ff9640 1 -- 172.21.15.81:0/1084236656 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fa1d40aad60 con 0x7fa1dc0c9680 2023-11-08T16:37:04.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.255+0000 7fa1dbfff640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1c806ae70 0x7fa1c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.256+0000 7fa1dbfff640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1c806ae70 0x7fa1c806d330 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0c22a0 tx=0x7fa1cc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:04.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.260+0000 7fa1d8ff9640 1 -- 172.21.15.81:0/1084236656 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa1d4075b20 con 0x7fa1dc0c9680 2023-11-08T16:37:04.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.447+0000 7fa1e2a00640 1 -- 172.21.15.81:0/1084236656 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1dc0ce6b0 con 0x7fa1dc0c9680 2023-11-08T16:37:04.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.448+0000 7fa1d8ff9640 1 -- 172.21.15.81:0/1084236656 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1d4079140 con 0x7fa1dc0c9680 2023-11-08T16:37:04.451 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:37:04.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.450+0000 7fa1e2a00640 1 -- 172.21.15.81:0/1084236656 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1c806ae70 msgr2=0x7fa1c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.450+0000 7fa1e2a00640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1c806ae70 0x7fa1c806d330 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc0c22a0 tx=0x7fa1cc008040 comp rx=0 tx=0).stop 2023-11-08T16:37:04.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.450+0000 7fa1e2a00640 1 -- 172.21.15.81:0/1084236656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1dc0c9680 msgr2=0x7fa1dc0c9b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.450+0000 7fa1e2a00640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1dc0c9680 0x7fa1dc0c9b00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fa1d400a760 tx=0x7fa1d4001cb0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.450+0000 7fa1e2a00640 1 -- 172.21.15.81:0/1084236656 shutdown_connections 2023-11-08T16:37:04.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.450+0000 7fa1e2a00640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa1c806ae70 0x7fa1c806d330 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.450+0000 7fa1e2a00640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa1dc0c9680 0x7fa1dc0c9b00 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.450+0000 7fa1e2a00640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa1dc1530e0 0x7fa1dc0c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.450+0000 7fa1e2a00640 1 --2- 172.21.15.81:0/1084236656 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa1c40051d0 0x7fa1dc1f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.450+0000 7fa1e2a00640 1 -- 172.21.15.81:0/1084236656 >> 172.21.15.81:0/1084236656 conn(0x7fa1dc0cb580 msgr2=0x7fa1dc155440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:04.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.450+0000 7fa1e2a00640 1 -- 172.21.15.81:0/1084236656 shutdown_connections 2023-11-08T16:37:04.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.451+0000 7fa1e2a00640 1 -- 172.21.15.81:0/1084236656 wait complete. 2023-11-08T16:37:04.472 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:37:04.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.679+0000 7f410df05640 1 Processor -- start 2023-11-08T16:37:04.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.679+0000 7f410df05640 1 -- start start 2023-11-08T16:37:04.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.680+0000 7f410df05640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f41080b8b60 0x7f41080cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.680+0000 7f410df05640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080d3a30 0x7f41080cfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.680+0000 7f410df05640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41080b66c0 con 0x7f410815d140 2023-11-08T16:37:04.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.680+0000 7f410df05640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41080d1d40 con 0x7f41080b8b60 2023-11-08T16:37:04.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.680+0000 7f410df05640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41080b5d90 con 0x7f41080d3a30 2023-11-08T16:37:04.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.681+0000 7f4106ffd640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f41080b8b60 0x7f41080cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.681+0000 7f4106ffd640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f41080b8b60 0x7f41080cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:39064/0 (socket says 172.21.15.81:39064) 2023-11-08T16:37:04.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.681+0000 7f4106ffd640 1 -- 172.21.15.81:0/3155802179 learned_addr learned my addr 172.21.15.81:0/3155802179 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:04.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.681+0000 7f4107fff640 1 --2- 172.21.15.81:0/3155802179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080d3a30 0x7f41080cfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.682+0000 7f41067fc640 1 -- 172.21.15.81:0/3155802179 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 387362974 0 0) 0x7f41080b66c0 con 0x7f410815d140 2023-11-08T16:37:04.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.682+0000 7f41067fc640 1 -- 172.21.15.81:0/3155802179 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40f0003520 con 0x7f410815d140 2023-11-08T16:37:04.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.682+0000 7f4107fff640 1 -- 172.21.15.81:0/3155802179 >> v2:172.21.15.93:3300/0 conn(0x7f41080b8b60 msgr2=0x7f41080cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:04.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.682+0000 7f4107fff640 1 --2- 172.21.15.81:0/3155802179 >> v2:172.21.15.93:3300/0 conn(0x7f41080b8b60 0x7f41080cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.682+0000 7f4107fff640 1 -- 172.21.15.81:0/3155802179 >> v1:172.21.15.93:6789/0 conn(0x7f410815d140 legacy=0x7f41080d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.682+0000 7f4107fff640 1 -- 172.21.15.81:0/3155802179 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080b6070 con 0x7f41080d3a30 2023-11-08T16:37:04.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.683+0000 7f4107fff640 1 --2- 172.21.15.81:0/3155802179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080d3a30 0x7f41080cfdf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4100009d90 tx=0x7f410002bf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff88c302d956c01e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:04.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.683+0000 7f41067fc640 1 -- 172.21.15.81:0/3155802179 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410002cce0 con 0x7f41080d3a30 2023-11-08T16:37:04.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.683+0000 7f41067fc640 1 -- 172.21.15.81:0/3155802179 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410002ce40 con 0x7f41080d3a30 2023-11-08T16:37:04.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.683+0000 7f41067fc640 1 -- 172.21.15.81:0/3155802179 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4100034640 con 0x7f41080d3a30 2023-11-08T16:37:04.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.684+0000 7f410df05640 1 -- 172.21.15.81:0/3155802179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080d3a30 msgr2=0x7f41080cfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.684+0000 7f410df05640 1 --2- 172.21.15.81:0/3155802179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080d3a30 0x7f41080cfdf0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4100009d90 tx=0x7f410002bf30 comp rx=0 tx=0).stop 2023-11-08T16:37:04.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.684+0000 7f410df05640 1 -- 172.21.15.81:0/3155802179 shutdown_connections 2023-11-08T16:37:04.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.684+0000 7f410df05640 1 --2- 172.21.15.81:0/3155802179 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080d3a30 0x7f41080cfdf0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.684+0000 7f410df05640 1 --2- 172.21.15.81:0/3155802179 >> v2:172.21.15.93:3300/0 conn(0x7f41080b8b60 0x7f41080cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.684+0000 7f410df05640 1 -- 172.21.15.81:0/3155802179 >> 172.21.15.81:0/3155802179 conn(0x7f41080d2ea0 msgr2=0x7f41080d32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:04.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.684+0000 7f410df05640 1 -- 172.21.15.81:0/3155802179 shutdown_connections 2023-11-08T16:37:04.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.684+0000 7f410df05640 1 -- 172.21.15.81:0/3155802179 wait complete. 2023-11-08T16:37:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.686+0000 7f410df05640 1 Processor -- start 2023-11-08T16:37:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.686+0000 7f410df05640 1 -- start start 2023-11-08T16:37:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.687+0000 7f410df05640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41080b8b60 0x7f41080ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.687+0000 7f410df05640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41080d3a30 0x7f41081f2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.687+0000 7f410df05640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f410815d140 0x7f410815a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.687+0000 7f410df05640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41080cb200 con 0x7f41080b8b60 2023-11-08T16:37:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.687+0000 7f410df05640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41080caf20 con 0x7f41080d3a30 2023-11-08T16:37:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.687+0000 7f410df05640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41080c6a30 con 0x7f410815d140 2023-11-08T16:37:04.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.687+0000 7f41077fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41080b8b60 0x7f41080ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.688+0000 7f41077fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41080b8b60 0x7f41080ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37390/0 (socket says 172.21.15.81:37390) 2023-11-08T16:37:04.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.688+0000 7f41077fe640 1 -- 172.21.15.81:0/4004400604 learned_addr learned my addr 172.21.15.81:0/4004400604 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:04.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.688+0000 7f4106ffd640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41080d3a30 0x7f41081f2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.688+0000 7f4107fff640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f410815d140 0x7f410815a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.688+0000 7f41077fe640 1 -- 172.21.15.81:0/4004400604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f410815d140 msgr2=0x7f410815a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.688+0000 7f41077fe640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f410815d140 0x7f410815a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.688+0000 7f41077fe640 1 -- 172.21.15.81:0/4004400604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41080d3a30 msgr2=0x7f41081f2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.688+0000 7f41077fe640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41080d3a30 0x7f41081f2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.688+0000 7f41077fe640 1 -- 172.21.15.81:0/4004400604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100009a70 con 0x7f41080b8b60 2023-11-08T16:37:04.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.688+0000 7f41077fe640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41080b8b60 0x7f41080ca930 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f40f800c430 tx=0x7f40f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:04.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.689+0000 7f40effff640 1 -- 172.21.15.81:0/4004400604 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80052d0 con 0x7f41080b8b60 2023-11-08T16:37:04.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.689+0000 7f40effff640 1 -- 172.21.15.81:0/4004400604 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f8005450 con 0x7f41080b8b60 2023-11-08T16:37:04.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.689+0000 7f410df05640 1 -- 172.21.15.81:0/4004400604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41080c6d10 con 0x7f41080b8b60 2023-11-08T16:37:04.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.689+0000 7f40effff640 1 -- 172.21.15.81:0/4004400604 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8003830 con 0x7f41080b8b60 2023-11-08T16:37:04.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.689+0000 7f410df05640 1 -- 172.21.15.81:0/4004400604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41080c9750 con 0x7f41080b8b60 2023-11-08T16:37:04.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.690+0000 7f410df05640 1 -- 172.21.15.81:0/4004400604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41080c6890 con 0x7f41080b8b60 2023-11-08T16:37:04.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.691+0000 7f40effff640 1 -- 172.21.15.81:0/4004400604 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40f8003990 con 0x7f41080b8b60 2023-11-08T16:37:04.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.692+0000 7f40effff640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40f406ae70 0x7f40f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:04.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.692+0000 7f40effff640 1 -- 172.21.15.81:0/4004400604 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f40f8088ef0 con 0x7f41080b8b60 2023-11-08T16:37:04.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.692+0000 7f4106ffd640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40f406ae70 0x7f40f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:04.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.693+0000 7f4106ffd640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40f406ae70 0x7f40f406d330 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f410815c780 tx=0x7f40fc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:04.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.696+0000 7f40effff640 1 -- 172.21.15.81:0/4004400604 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41080c6890 con 0x7f41080b8b60 2023-11-08T16:37:04.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.887+0000 7f410df05640 1 -- 172.21.15.81:0/4004400604 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f41080c6890 con 0x7f41080b8b60 2023-11-08T16:37:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.887+0000 7f40effff640 1 -- 172.21.15.81:0/4004400604 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7f41080c6890 con 0x7f41080b8b60 2023-11-08T16:37:04.890 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:37:04.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 -- 172.21.15.81:0/4004400604 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40f406ae70 msgr2=0x7f40f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40f406ae70 0x7f40f406d330 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f410815c780 tx=0x7f40fc008040 comp rx=0 tx=0).stop 2023-11-08T16:37:04.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 -- 172.21.15.81:0/4004400604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41080b8b60 msgr2=0x7f41080ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:04.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41080b8b60 0x7f41080ca930 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f40f800c430 tx=0x7f40f800c8f0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 -- 172.21.15.81:0/4004400604 shutdown_connections 2023-11-08T16:37:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40f406ae70 0x7f40f406d330 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f410815d140 0x7f410815a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f41080d3a30 0x7f41081f2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 --2- 172.21.15.81:0/4004400604 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f41080b8b60 0x7f41080ca930 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 -- 172.21.15.81:0/4004400604 >> 172.21.15.81:0/4004400604 conn(0x7f41080d2ea0 msgr2=0x7f410806a980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 -- 172.21.15.81:0/4004400604 shutdown_connections 2023-11-08T16:37:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:04.890+0000 7f410df05640 1 -- 172.21.15.81:0/4004400604 wait complete. 2023-11-08T16:37:04.911 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:37:05.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.118+0000 7f2792e72640 1 Processor -- start 2023-11-08T16:37:05.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.118+0000 7f2792e72640 1 -- start start 2023-11-08T16:37:05.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.119+0000 7f2792e72640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c0b4950 0x7f278c06dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.119+0000 7f2792e72640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f278c06d100 0x7f278c06d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.120+0000 7f2792e72640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f278c0710a0 con 0x7f278c0af740 2023-11-08T16:37:05.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.120+0000 7f2792e72640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f278c06daf0 con 0x7f278c06d100 2023-11-08T16:37:05.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.120+0000 7f2792e72640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c071660 con 0x7f278c0b4950 2023-11-08T16:37:05.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.120+0000 7f2791e70640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c0b4950 0x7f278c06dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.120+0000 7f27940d4640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f278c06d100 0x7f278c06d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.120+0000 7f2791e70640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c0b4950 0x7f278c06dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:36018/0 (socket says 172.21.15.81:36018) 2023-11-08T16:37:05.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.120+0000 7f2791e70640 1 -- 172.21.15.81:0/2348359952 learned_addr learned my addr 172.21.15.81:0/2348359952 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:05.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.121+0000 7f2791e70640 1 -- 172.21.15.81:0/2348359952 >> v2:172.21.15.93:3300/0 conn(0x7f278c06d100 msgr2=0x7f278c06d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:05.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.121+0000 7f2791e70640 1 --2- 172.21.15.81:0/2348359952 >> v2:172.21.15.93:3300/0 conn(0x7f278c06d100 0x7f278c06d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.121+0000 7f2791e70640 1 -- 172.21.15.81:0/2348359952 >> v1:172.21.15.93:6789/0 conn(0x7f278c0af740 legacy=0x7f278c071af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:05.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.121+0000 7f2791e70640 1 -- 172.21.15.81:0/2348359952 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f278c06e210 con 0x7f278c0b4950 2023-11-08T16:37:05.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.122+0000 7f2791e70640 1 --2- 172.21.15.81:0/2348359952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c0b4950 0x7f278c06dcd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2788009870 tx=0x7f278802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50f97cdd1304888a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:05.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.123+0000 7f2790e6e640 1 -- 172.21.15.81:0/2348359952 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788030840 con 0x7f278c0b4950 2023-11-08T16:37:05.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.123+0000 7f2790e6e640 1 -- 172.21.15.81:0/2348359952 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27880309a0 con 0x7f278c0b4950 2023-11-08T16:37:05.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.123+0000 7f2790e6e640 1 -- 172.21.15.81:0/2348359952 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2788030d30 con 0x7f278c0b4950 2023-11-08T16:37:05.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.123+0000 7f2792e72640 1 -- 172.21.15.81:0/2348359952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c0b4950 msgr2=0x7f278c06dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:05.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.123+0000 7f2792e72640 1 --2- 172.21.15.81:0/2348359952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c0b4950 0x7f278c06dcd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2788009870 tx=0x7f278802b980 comp rx=0 tx=0).stop 2023-11-08T16:37:05.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.124+0000 7f2792e72640 1 -- 172.21.15.81:0/2348359952 shutdown_connections 2023-11-08T16:37:05.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.124+0000 7f2792e72640 1 --2- 172.21.15.81:0/2348359952 >> v2:172.21.15.93:3300/0 conn(0x7f278c06d100 0x7f278c06d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.124+0000 7f2792e72640 1 --2- 172.21.15.81:0/2348359952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c0b4950 0x7f278c06dcd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.124+0000 7f2792e72640 1 -- 172.21.15.81:0/2348359952 >> 172.21.15.81:0/2348359952 conn(0x7f278c099a30 msgr2=0x7f278c0d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:05.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.124+0000 7f2792e72640 1 -- 172.21.15.81:0/2348359952 shutdown_connections 2023-11-08T16:37:05.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.124+0000 7f2792e72640 1 -- 172.21.15.81:0/2348359952 wait complete. 2023-11-08T16:37:05.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.126+0000 7f2792e72640 1 Processor -- start 2023-11-08T16:37:05.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.126+0000 7f2792e72640 1 -- start start 2023-11-08T16:37:05.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.127+0000 7f2792e72640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c06d100 0x7f278c1e37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.127+0000 7f2792e72640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f278c0af740 0x7f278c1e5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.127+0000 7f2792e72640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f278c0b4950 0x7f278c146e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.127+0000 7f2792e72640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1e5480 con 0x7f278c0b4950 2023-11-08T16:37:05.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.127+0000 7f2792e72640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1e4870 con 0x7f278c0af740 2023-11-08T16:37:05.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.127+0000 7f2792e72640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f278c1e4b50 con 0x7f278c06d100 2023-11-08T16:37:05.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.128+0000 7f27940d4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f278c0b4950 0x7f278c146e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.128+0000 7f27940d4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f278c0b4950 0x7f278c146e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37406/0 (socket says 172.21.15.81:37406) 2023-11-08T16:37:05.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.128+0000 7f27940d4640 1 -- 172.21.15.81:0/1395138781 learned_addr learned my addr 172.21.15.81:0/1395138781 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:05.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.128+0000 7f279166f640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f278c0af740 0x7f278c1e5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.128+0000 7f2791e70640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c06d100 0x7f278c1e37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.128+0000 7f27940d4640 1 -- 172.21.15.81:0/1395138781 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c06d100 msgr2=0x7f278c1e37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:05.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.128+0000 7f27940d4640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c06d100 0x7f278c1e37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.128+0000 7f27940d4640 1 -- 172.21.15.81:0/1395138781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f278c0af740 msgr2=0x7f278c1e5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:05.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.128+0000 7f27940d4640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f278c0af740 0x7f278c1e5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.128+0000 7f27940d4640 1 -- 172.21.15.81:0/1395138781 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27880094d0 con 0x7f278c0b4950 2023-11-08T16:37:05.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.129+0000 7f27940d4640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f278c0b4950 0x7f278c146e90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f278400efc0 tx=0x7f278400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:05.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.129+0000 7f27827fc640 1 -- 172.21.15.81:0/1395138781 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784014480 con 0x7f278c0b4950 2023-11-08T16:37:05.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.129+0000 7f27827fc640 1 -- 172.21.15.81:0/1395138781 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278400f040 con 0x7f278c0b4950 2023-11-08T16:37:05.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.129+0000 7f27827fc640 1 -- 172.21.15.81:0/1395138781 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784014820 con 0x7f278c0b4950 2023-11-08T16:37:05.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.129+0000 7f2792e72640 1 -- 172.21.15.81:0/1395138781 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278c1e4e30 con 0x7f278c0b4950 2023-11-08T16:37:05.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.129+0000 7f2792e72640 1 -- 172.21.15.81:0/1395138781 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278c1551a0 con 0x7f278c0b4950 2023-11-08T16:37:05.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.130+0000 7f2792e72640 1 -- 172.21.15.81:0/1395138781 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f278c1e5630 con 0x7f278c0b4950 2023-11-08T16:37:05.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.131+0000 7f27827fc640 1 -- 172.21.15.81:0/1395138781 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2784014980 con 0x7f278c0b4950 2023-11-08T16:37:05.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.132+0000 7f27827fc640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277006aea0 0x7f277006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.132+0000 7f27827fc640 1 -- 172.21.15.81:0/1395138781 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f278408a3d0 con 0x7f278c0b4950 2023-11-08T16:37:05.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.132+0000 7f2791e70640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277006aea0 0x7f277006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.133+0000 7f2791e70640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277006aea0 0x7f277006d360 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2788036000 tx=0x7f278803b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:05.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.137+0000 7f27827fc640 1 -- 172.21.15.81:0/1395138781 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f278c1e5630 con 0x7f278c0b4950 2023-11-08T16:37:05.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.331+0000 7f2792e72640 1 -- 172.21.15.81:0/1395138781 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f278c1e5630 con 0x7f278c0b4950 2023-11-08T16:37:05.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.331+0000 7f27827fc640 1 -- 172.21.15.81:0/1395138781 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v68) v1 ==== 106+0+11 (secure 0 0 0) 0x7f278c1e5630 con 0x7f278c0b4950 2023-11-08T16:37:05.334 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:37:05.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 -- 172.21.15.81:0/1395138781 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277006aea0 msgr2=0x7f277006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:05.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277006aea0 0x7f277006d360 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2788036000 tx=0x7f278803b040 comp rx=0 tx=0).stop 2023-11-08T16:37:05.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 -- 172.21.15.81:0/1395138781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f278c0b4950 msgr2=0x7f278c146e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:05.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f278c0b4950 0x7f278c146e90 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f278400efc0 tx=0x7f278400c490 comp rx=0 tx=0).stop 2023-11-08T16:37:05.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 -- 172.21.15.81:0/1395138781 shutdown_connections 2023-11-08T16:37:05.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277006aea0 0x7f277006d360 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f278c0b4950 0x7f278c146e90 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f278c0af740 0x7f278c1e5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 --2- 172.21.15.81:0/1395138781 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f278c06d100 0x7f278c1e37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 -- 172.21.15.81:0/1395138781 >> 172.21.15.81:0/1395138781 conn(0x7f278c099a30 msgr2=0x7f278c02c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:05.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 -- 172.21.15.81:0/1395138781 shutdown_connections 2023-11-08T16:37:05.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.334+0000 7f2792e72640 1 -- 172.21.15.81:0/1395138781 wait complete. 2023-11-08T16:37:05.355 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:37:05.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.569+0000 7f2676fc4640 1 Processor -- start 2023-11-08T16:37:05.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.570+0000 7f2676fc4640 1 -- start start 2023-11-08T16:37:05.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.571+0000 7f2676fc4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f26700b4f80 0x7f26700b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.571+0000 7f2676fc4640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2670160d40 0x7f2670161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.571+0000 7f2676fc4640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26700b6540 con 0x7f267015cfe0 2023-11-08T16:37:05.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.571+0000 7f2676fc4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2670161ab0 con 0x7f26700b4f80 2023-11-08T16:37:05.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.571+0000 7f2676fc4640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26700d1060 con 0x7f2670160d40 2023-11-08T16:37:05.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.572+0000 7f26757c1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f26700b4f80 0x7f26700b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.572+0000 7f26757c1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f26700b4f80 0x7f26700b6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37418/0 (socket says 172.21.15.81:37418) 2023-11-08T16:37:05.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.572+0000 7f26757c1640 1 -- 172.21.15.81:0/3848132229 learned_addr learned my addr 172.21.15.81:0/3848132229 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:05.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.572+0000 7f26767c3640 1 --2- 172.21.15.81:0/3848132229 >> v2:172.21.15.93:3300/0 conn(0x7f2670160d40 0x7f2670161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.572+0000 7f26757c1640 1 -- 172.21.15.81:0/3848132229 >> v2:172.21.15.93:3300/0 conn(0x7f2670160d40 msgr2=0x7f2670161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:05.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.572+0000 7f26757c1640 1 --2- 172.21.15.81:0/3848132229 >> v2:172.21.15.93:3300/0 conn(0x7f2670160d40 0x7f2670161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.572+0000 7f26757c1640 1 -- 172.21.15.81:0/3848132229 >> v1:172.21.15.93:6789/0 conn(0x7f267015cfe0 legacy=0x7f26700b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:05.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.572+0000 7f26757c1640 1 -- 172.21.15.81:0/3848132229 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26700d1340 con 0x7f26700b4f80 2023-11-08T16:37:05.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.573+0000 7f26757c1640 1 --2- 172.21.15.81:0/3848132229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26700b4f80 0x7f26700b6000 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f267008c150 tx=0x7f266402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e196d37363f6642 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:05.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.574+0000 7f2674fc0640 1 -- 172.21.15.81:0/3848132229 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664031870 con 0x7f26700b4f80 2023-11-08T16:37:05.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.574+0000 7f2674fc0640 1 -- 172.21.15.81:0/3848132229 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26640319d0 con 0x7f26700b4f80 2023-11-08T16:37:05.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.574+0000 7f2674fc0640 1 -- 172.21.15.81:0/3848132229 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664031d60 con 0x7f26700b4f80 2023-11-08T16:37:05.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.574+0000 7f2676fc4640 1 -- 172.21.15.81:0/3848132229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26700b4f80 msgr2=0x7f26700b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:05.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.574+0000 7f2676fc4640 1 --2- 172.21.15.81:0/3848132229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26700b4f80 0x7f26700b6000 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f267008c150 tx=0x7f266402b910 comp rx=0 tx=0).stop 2023-11-08T16:37:05.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.575+0000 7f2676fc4640 1 -- 172.21.15.81:0/3848132229 shutdown_connections 2023-11-08T16:37:05.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.575+0000 7f2676fc4640 1 --2- 172.21.15.81:0/3848132229 >> v2:172.21.15.93:3300/0 conn(0x7f2670160d40 0x7f2670161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.575+0000 7f2676fc4640 1 --2- 172.21.15.81:0/3848132229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26700b4f80 0x7f26700b6000 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.575+0000 7f2676fc4640 1 -- 172.21.15.81:0/3848132229 >> 172.21.15.81:0/3848132229 conn(0x7f26700cf870 msgr2=0x7f26700cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:05.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.575+0000 7f2676fc4640 1 -- 172.21.15.81:0/3848132229 shutdown_connections 2023-11-08T16:37:05.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.575+0000 7f2676fc4640 1 -- 172.21.15.81:0/3848132229 wait complete. 2023-11-08T16:37:05.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.576+0000 7f2676fc4640 1 Processor -- start 2023-11-08T16:37:05.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.576+0000 7f2676fc4640 1 -- start start 2023-11-08T16:37:05.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.577+0000 7f2676fc4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26700b4f80 0x7f26700cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.577+0000 7f2676fc4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f267015cfe0 0x7f2670150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.577+0000 7f2676fc4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2670160d40 0x7f26700c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.577+0000 7f2676fc4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26700cbd40 con 0x7f267015cfe0 2023-11-08T16:37:05.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.577+0000 7f2676fc4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26700cc020 con 0x7f26700b4f80 2023-11-08T16:37:05.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.577+0000 7f2676fc4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26700cc300 con 0x7f2670160d40 2023-11-08T16:37:05.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f26757c1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f267015cfe0 0x7f2670150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f26757c1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f267015cfe0 0x7f2670150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37422/0 (socket says 172.21.15.81:37422) 2023-11-08T16:37:05.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f26757c1640 1 -- 172.21.15.81:0/2644466652 learned_addr learned my addr 172.21.15.81:0/2644466652 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:05.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f2675fc2640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26700b4f80 0x7f26700cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f26757c1640 1 -- 172.21.15.81:0/2644466652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2670160d40 msgr2=0x7f26700c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:05.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f26767c3640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2670160d40 0x7f26700c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f26757c1640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2670160d40 0x7f26700c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f26757c1640 1 -- 172.21.15.81:0/2644466652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26700b4f80 msgr2=0x7f26700cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:05.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f26757c1640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26700b4f80 0x7f26700cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f26757c1640 1 -- 172.21.15.81:0/2644466652 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26640094d0 con 0x7f267015cfe0 2023-11-08T16:37:05.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f2675fc2640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26700b4f80 0x7f26700cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:05.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.578+0000 7f26757c1640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f267015cfe0 0x7f2670150a80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f266402b950 tx=0x7f26640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:05.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.579+0000 7f266e7fc640 1 -- 172.21.15.81:0/2644466652 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266403ad30 con 0x7f267015cfe0 2023-11-08T16:37:05.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.579+0000 7f266e7fc640 1 -- 172.21.15.81:0/2644466652 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26640443f0 con 0x7f267015cfe0 2023-11-08T16:37:05.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.579+0000 7f266e7fc640 1 -- 172.21.15.81:0/2644466652 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26640447a0 con 0x7f267015cfe0 2023-11-08T16:37:05.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.579+0000 7f2676fc4640 1 -- 172.21.15.81:0/2644466652 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26700cc5e0 con 0x7f267015cfe0 2023-11-08T16:37:05.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.579+0000 7f2676fc4640 1 -- 172.21.15.81:0/2644466652 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26700cb9e0 con 0x7f267015cfe0 2023-11-08T16:37:05.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.580+0000 7f2676fc4640 1 -- 172.21.15.81:0/2644466652 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26700ccb00 con 0x7f267015cfe0 2023-11-08T16:37:05.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.581+0000 7f266e7fc640 1 -- 172.21.15.81:0/2644466652 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f266404d450 con 0x7f267015cfe0 2023-11-08T16:37:05.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.582+0000 7f266e7fc640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f265c06ae70 0x7f265c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:05.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.582+0000 7f266e7fc640 1 -- 172.21.15.81:0/2644466652 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f266402d040 con 0x7f267015cfe0 2023-11-08T16:37:05.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.582+0000 7f2675fc2640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f265c06ae70 0x7f265c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:05.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.583+0000 7f2675fc2640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f265c06ae70 0x7f265c06d330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2660004520 tx=0x7f2660009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:05.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.586+0000 7f266e7fc640 1 -- 172.21.15.81:0/2644466652 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26700ccb00 con 0x7f267015cfe0 2023-11-08T16:37:05.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.788+0000 7f2676fc4640 1 -- 172.21.15.81:0/2644466652 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f26700ccb00 con 0x7f267015cfe0 2023-11-08T16:37:05.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.788+0000 7f266e7fc640 1 -- 172.21.15.81:0/2644466652 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v68) v1 ==== 80+0+362 (secure 0 0 0) 0x7f26700ccb00 con 0x7f267015cfe0 2023-11-08T16:37:05.791 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/321347143 2023-11-09T16:34:45.940255+0000 2023-11-08T16:37:05.791 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/1446454901 2023-11-09T16:34:43.853227+0000 2023-11-08T16:37:05.792 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/494395478 2023-11-09T16:34:53.172859+0000 2023-11-08T16:37:05.792 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6835/494395478 2023-11-09T16:34:53.172859+0000 2023-11-08T16:37:05.792 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6835/321347143 2023-11-09T16:34:45.940255+0000 2023-11-08T16:37:05.792 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6835/1446454901 2023-11-09T16:34:43.853227+0000 2023-11-08T16:37:05.793 INFO:teuthology.orchestra.run.smithi081.stderr:listed 6 entries 2023-11-08T16:37:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 -- 172.21.15.81:0/2644466652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f265c06ae70 msgr2=0x7f265c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f265c06ae70 0x7f265c06d330 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2660004520 tx=0x7f2660009040 comp rx=0 tx=0).stop 2023-11-08T16:37:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 -- 172.21.15.81:0/2644466652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f267015cfe0 msgr2=0x7f2670150a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f267015cfe0 0x7f2670150a80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f266402b950 tx=0x7f26640046c0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 -- 172.21.15.81:0/2644466652 shutdown_connections 2023-11-08T16:37:05.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f265c06ae70 0x7f265c06d330 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2670160d40 0x7f26700c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f267015cfe0 0x7f2670150a80 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 --2- 172.21.15.81:0/2644466652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26700b4f80 0x7f26700cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 -- 172.21.15.81:0/2644466652 >> 172.21.15.81:0/2644466652 conn(0x7f26700cf870 msgr2=0x7f26700b54a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 -- 172.21.15.81:0/2644466652 shutdown_connections 2023-11-08T16:37:05.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:05.791+0000 7f2676fc4640 1 -- 172.21.15.81:0/2644466652 wait complete. 2023-11-08T16:37:05.811 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:37:05.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:05.812 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:37:05.904 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[110217]: fuse finished with error 0 and tester_r 0 2023-11-08T16:37:05.943 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:37:05.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:05.944 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:37:06.122 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:37:06.123 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:37:06.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:37:06.183 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:37:06.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:06.183 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-08T16:37:06.237 INFO:teuthology.orchestra.run.smithi093.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-08T16:37:06.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:37:06.238 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:37:06.239 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:37:06.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:37:06.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:37:06.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:06.301 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-08T16:37:06.356 INFO:teuthology.orchestra.run.smithi081.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-08T16:37:06.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:37:06.357 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:37:06.358 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:37:06.417 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:37:06.418 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:37:06.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:06.419 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-08T16:37:06.475 INFO:teuthology.orchestra.run.smithi093.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-08T16:37:06.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:37:06.478 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:37:06.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.700+0000 7f289b59e640 1 Processor -- start 2023-11-08T16:37:06.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.700+0000 7f289b59e640 1 -- start start 2023-11-08T16:37:06.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.701+0000 7f289b59e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f289c15aac0 0x7f289c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:06.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.702+0000 7f289b59e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289c070470 con 0x7f289c0716e0 2023-11-08T16:37:06.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.702+0000 7f289b59e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289c06d580 con 0x7f289c0cf420 2023-11-08T16:37:06.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.702+0000 7f289b59e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289c06f650 con 0x7f289c15aac0 2023-11-08T16:37:06.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.702+0000 7f289a59c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f289c15aac0 0x7f289c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:06.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.702+0000 7f2891d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f289c0716e0 0x7f289c0ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51482/0 (socket says 172.21.15.81:51482) 2023-11-08T16:37:06.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.702+0000 7f2891d9b640 1 -- 172.21.15.81:0/849328190 learned_addr learned my addr 172.21.15.81:0/849328190 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:06.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.702+0000 7f289a59c640 1 -- 172.21.15.81:0/849328190 >> v1:172.21.15.93:6789/0 conn(0x7f289c0cf420 legacy=0x7f289c15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:06.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.702+0000 7f289a59c640 1 -- 172.21.15.81:0/849328190 >> v1:172.21.15.81:6789/0 conn(0x7f289c0716e0 legacy=0x7f289c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:06.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.702+0000 7f289a59c640 1 -- 172.21.15.81:0/849328190 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c06ca00 con 0x7f289c15aac0 2023-11-08T16:37:06.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.703+0000 7f289a59c640 1 --2- 172.21.15.81:0/849328190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289c15aac0 0x7f289c0711a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f288c0097a0 tx=0x7f288c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c6033da8e87298 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:06.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.704+0000 7f2899d9b640 1 -- 172.21.15.81:0/849328190 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c031830 con 0x7f289c15aac0 2023-11-08T16:37:06.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.704+0000 7f2899d9b640 1 -- 172.21.15.81:0/849328190 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c031990 con 0x7f289c15aac0 2023-11-08T16:37:06.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.704+0000 7f2899d9b640 1 -- 172.21.15.81:0/849328190 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c031d20 con 0x7f289c15aac0 2023-11-08T16:37:06.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.704+0000 7f289b59e640 1 -- 172.21.15.81:0/849328190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289c15aac0 msgr2=0x7f289c0711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:06.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.704+0000 7f289b59e640 1 --2- 172.21.15.81:0/849328190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289c15aac0 0x7f289c0711a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f288c0097a0 tx=0x7f288c02b8d0 comp rx=0 tx=0).stop 2023-11-08T16:37:06.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.704+0000 7f289b59e640 1 -- 172.21.15.81:0/849328190 shutdown_connections 2023-11-08T16:37:06.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.704+0000 7f289b59e640 1 --2- 172.21.15.81:0/849328190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289c15aac0 0x7f289c0711a0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:06.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.704+0000 7f289b59e640 1 -- 172.21.15.81:0/849328190 >> 172.21.15.81:0/849328190 conn(0x7f289c06a4a0 msgr2=0x7f289c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:06.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.705+0000 7f289b59e640 1 -- 172.21.15.81:0/849328190 shutdown_connections 2023-11-08T16:37:06.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.705+0000 7f289b59e640 1 -- 172.21.15.81:0/849328190 wait complete. 2023-11-08T16:37:06.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.706+0000 7f289b59e640 1 Processor -- start 2023-11-08T16:37:06.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.707+0000 7f289b59e640 1 -- start start 2023-11-08T16:37:06.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.707+0000 7f289b59e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289c0716e0 0x7f289c149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:06.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.707+0000 7f289b59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289c0cf420 0x7f289c15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:06.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f289b59e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f289c15aac0 0x7f289c15b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:06.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f289b59e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c152710 con 0x7f289c0cf420 2023-11-08T16:37:06.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f289b59e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c151e50 con 0x7f289c0716e0 2023-11-08T16:37:06.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f289b59e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289c1520c0 con 0x7f289c15aac0 2023-11-08T16:37:06.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f2891d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289c0cf420 0x7f289c15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:06.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f2891d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289c0cf420 0x7f289c15aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37450/0 (socket says 172.21.15.81:37450) 2023-11-08T16:37:06.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f2891d9b640 1 -- 172.21.15.81:0/2931394947 learned_addr learned my addr 172.21.15.81:0/2931394947 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:06.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f289a59c640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289c0716e0 0x7f289c149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:06.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f2891d9b640 1 -- 172.21.15.81:0/2931394947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f289c15aac0 msgr2=0x7f289c15b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:06.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f2891d9b640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f289c15aac0 0x7f289c15b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:06.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f2891d9b640 1 -- 172.21.15.81:0/2931394947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289c0716e0 msgr2=0x7f289c149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:06.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f2891d9b640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289c0716e0 0x7f289c149b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:06.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f2891d9b640 1 -- 172.21.15.81:0/2931394947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c009400 con 0x7f289c0cf420 2023-11-08T16:37:06.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.708+0000 7f289a59c640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289c0716e0 0x7f289c149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:06.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.709+0000 7f2891d9b640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289c0cf420 0x7f289c15aea0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f288400c530 tx=0x7f288400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:06.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.709+0000 7f28937fe640 1 -- 172.21.15.81:0/2931394947 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884005ce0 con 0x7f289c0cf420 2023-11-08T16:37:06.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.709+0000 7f28937fe640 1 -- 172.21.15.81:0/2931394947 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2884005e40 con 0x7f289c0cf420 2023-11-08T16:37:06.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.709+0000 7f28937fe640 1 -- 172.21.15.81:0/2931394947 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884007270 con 0x7f289c0cf420 2023-11-08T16:37:06.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.709+0000 7f289b59e640 1 -- 172.21.15.81:0/2931394947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289c1523a0 con 0x7f289c0cf420 2023-11-08T16:37:06.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.709+0000 7f289b59e640 1 -- 172.21.15.81:0/2931394947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c143e70 con 0x7f289c0cf420 2023-11-08T16:37:06.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.710+0000 7f289b59e640 1 -- 172.21.15.81:0/2931394947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289c1528c0 con 0x7f289c0cf420 2023-11-08T16:37:06.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.711+0000 7f28937fe640 1 -- 172.21.15.81:0/2931394947 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28840073d0 con 0x7f289c0cf420 2023-11-08T16:37:06.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.712+0000 7f28937fe640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f287c06ae70 0x7f287c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:06.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.712+0000 7f28937fe640 1 -- 172.21.15.81:0/2931394947 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f2884088e00 con 0x7f289c0cf420 2023-11-08T16:37:06.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.712+0000 7f289a59c640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f287c06ae70 0x7f287c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:06.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.713+0000 7f289a59c640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f287c06ae70 0x7f287c06d330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f288c036000 tx=0x7f288c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:06.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.716+0000 7f28937fe640 1 -- 172.21.15.81:0/2931394947 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f289c1528c0 con 0x7f289c0cf420 2023-11-08T16:37:06.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.947+0000 7f289b59e640 1 -- 172.21.15.81:0/2931394947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f289c1528c0 con 0x7f289c0cf420 2023-11-08T16:37:06.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.948+0000 7f28937fe640 1 -- 172.21.15.81:0/2931394947 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f289c1528c0 con 0x7f289c0cf420 2023-11-08T16:37:06.951 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:06.951 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3027678987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3027678987},{"type":"v1","addr":"172.21.15.93:6833","nonce":3027678987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/3140373039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3140373039},{"type":"v1","addr":"172.21.15.81:6836","nonce":3140373039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3145959397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3145959397},{"type":"v1","addr":"172.21.15.93:6835","nonce":3145959397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:55.206262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/552116415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":552116415},{"type":"v1","addr":"172.21.15.81:6837","nonce":552116415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:37:06.952 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 54 2023-11-08T16:37:06.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.951+0000 7f289b59e640 1 -- 172.21.15.81:0/2931394947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f287c06ae70 msgr2=0x7f287c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:06.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.951+0000 7f289b59e640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f287c06ae70 0x7f287c06d330 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f288c036000 tx=0x7f288c03c040 comp rx=0 tx=0).stop 2023-11-08T16:37:06.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.951+0000 7f289b59e640 1 -- 172.21.15.81:0/2931394947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289c0cf420 msgr2=0x7f289c15aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:06.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.951+0000 7f289b59e640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289c0cf420 0x7f289c15aea0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f288400c530 tx=0x7f288400c9f0 comp rx=0 tx=0).stop 2023-11-08T16:37:06.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.952+0000 7f289b59e640 1 -- 172.21.15.81:0/2931394947 shutdown_connections 2023-11-08T16:37:06.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.952+0000 7f289b59e640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f287c06ae70 0x7f287c06d330 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:06.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.952+0000 7f289b59e640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f289c15aac0 0x7f289c15b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:06.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.952+0000 7f289b59e640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f289c0cf420 0x7f289c15aea0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:06.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.952+0000 7f289b59e640 1 --2- 172.21.15.81:0/2931394947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289c0716e0 0x7f289c149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:06.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.952+0000 7f289b59e640 1 -- 172.21.15.81:0/2931394947 >> 172.21.15.81:0/2931394947 conn(0x7f289c06a4a0 msgr2=0x7f289c0ae4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:06.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.952+0000 7f289b59e640 1 -- 172.21.15.81:0/2931394947 shutdown_connections 2023-11-08T16:37:06.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:06.952+0000 7f289b59e640 1 -- 172.21.15.81:0/2931394947 wait complete. 2023-11-08T16:37:06.973 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:07.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.183+0000 7efd96918640 1 Processor -- start 2023-11-08T16:37:07.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.183+0000 7efd96918640 1 -- start start 2023-11-08T16:37:07.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.185+0000 7efd96918640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7efd900ade50 0x7efd900ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.185+0000 7efd96918640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd9015a680 0x7efd9015aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.185+0000 7efd96918640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9006b630 con 0x7efd9015b020 2023-11-08T16:37:07.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.185+0000 7efd96918640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd900b1860 con 0x7efd900ade50 2023-11-08T16:37:07.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.185+0000 7efd96918640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd900ca9c0 con 0x7efd9015a680 2023-11-08T16:37:07.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.185+0000 7efd94eb5640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7efd9015b020 0x7efd900c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51498/0 (socket says 172.21.15.81:51498) 2023-11-08T16:37:07.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.185+0000 7efd94eb5640 1 -- 172.21.15.81:0/2946719432 learned_addr learned my addr 172.21.15.81:0/2946719432 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:07.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.185+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105729098 0 0) 0x7efd9006b630 con 0x7efd9015b020 2023-11-08T16:37:07.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.185+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd78003520 con 0x7efd9015b020 2023-11-08T16:37:07.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.185+0000 7efd8f7fe640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd9015a680 0x7efd9015aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.185+0000 7efd8ffff640 1 --2- 172.21.15.81:0/2946719432 >> v2:172.21.15.93:3300/0 conn(0x7efd900ade50 0x7efd900ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1895120925 0 0) 0x7efd78003520 con 0x7efd9015b020 2023-11-08T16:37:07.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd9015a680 msgr2=0x7efd9015aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:07.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd9015a680 0x7efd9015aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 >> v2:172.21.15.93:3300/0 conn(0x7efd900ade50 msgr2=0x7efd900ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:07.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 --2- 172.21.15.81:0/2946719432 >> v2:172.21.15.93:3300/0 conn(0x7efd900ade50 0x7efd900ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900cabb0 con 0x7efd9015b020 2023-11-08T16:37:07.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7efd880019f0 con 0x7efd9015b020 2023-11-08T16:37:07.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 >> v1:172.21.15.81:6789/0 conn(0x7efd9015b020 legacy=0x7efd900c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd78003a50 0x7efd78003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd780047c0 0x7efd78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efd78005180 0x7efd7800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7800c050 con 0x7efd780047c0 2023-11-08T16:37:07.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7800c190 con 0x7efd78005180 2023-11-08T16:37:07.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7800c350 con 0x7efd78003a50 2023-11-08T16:37:07.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8f7fe640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd9015a680 0x7efd9015aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:37:07.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8ffff640 1 --2- 172.21.15.81:0/2946719432 >> v2:172.21.15.93:3300/0 conn(0x7efd900ade50 0x7efd900ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:37:07.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.186+0000 7efd8f7fe640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd780047c0 0x7efd78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.187+0000 7efd94eb5640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efd78005180 0x7efd7800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.187+0000 7efd8ffff640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd78003a50 0x7efd78003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.187+0000 7efd8f7fe640 1 -- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd78003a50 msgr2=0x7efd78003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:07.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.187+0000 7efd8f7fe640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd78003a50 0x7efd78003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.187+0000 7efd8f7fe640 1 -- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efd78005180 msgr2=0x7efd7800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:07.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.187+0000 7efd8f7fe640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efd78005180 0x7efd7800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.187+0000 7efd8f7fe640 1 -- 172.21.15.81:0/2946719432 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd78003520 con 0x7efd780047c0 2023-11-08T16:37:07.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.188+0000 7efd8f7fe640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd780047c0 0x7efd78004c40 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7efd84006fd0 tx=0x7efd780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31cc6d3900fc73db server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:07.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.188+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84033600 con 0x7efd780047c0 2023-11-08T16:37:07.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.188+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8402f050 con 0x7efd780047c0 2023-11-08T16:37:07.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.188+0000 7efd8effd640 1 -- 172.21.15.81:0/2946719432 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd84033980 con 0x7efd780047c0 2023-11-08T16:37:07.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.189+0000 7efd96918640 1 -- 172.21.15.81:0/2946719432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd780047c0 msgr2=0x7efd78004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.189+0000 7efd96918640 1 --2- 172.21.15.81:0/2946719432 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd780047c0 0x7efd78004c40 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7efd84006fd0 tx=0x7efd780057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.189+0000 7efd94eb5640 1 -- 172.21.15.81:0/2946719432 reap_dead start 2023-11-08T16:37:07.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.189+0000 7efd96918640 1 -- 172.21.15.81:0/2946719432 shutdown_connections 2023-11-08T16:37:07.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.189+0000 7efd96918640 1 -- 172.21.15.81:0/2946719432 >> 172.21.15.81:0/2946719432 conn(0x7efd900cb490 msgr2=0x7efd900cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:07.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.189+0000 7efd96918640 1 -- 172.21.15.81:0/2946719432 shutdown_connections 2023-11-08T16:37:07.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.189+0000 7efd96918640 1 -- 172.21.15.81:0/2946719432 wait complete. 2023-11-08T16:37:07.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.191+0000 7efd96918640 1 Processor -- start 2023-11-08T16:37:07.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.191+0000 7efd96918640 1 -- start start 2023-11-08T16:37:07.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd96918640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efd780047c0 0x7efd9014ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd96918640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd901545f0 0x7efd90154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd96918640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd900af7f0 0x7efd900afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd96918640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd900b01c0 con 0x7efd901545f0 2023-11-08T16:37:07.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd96918640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd900ce470 con 0x7efd780047c0 2023-11-08T16:37:07.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd96918640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd9014f470 con 0x7efd900af7f0 2023-11-08T16:37:07.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd8f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd901545f0 0x7efd90154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd8f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd901545f0 0x7efd90154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37462/0 (socket says 172.21.15.81:37462) 2023-11-08T16:37:07.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd8f7fe640 1 -- 172.21.15.81:0/2628279203 learned_addr learned my addr 172.21.15.81:0/2628279203 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:07.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd94eb5640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd900af7f0 0x7efd900afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd8ffff640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efd780047c0 0x7efd9014ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd8f7fe640 1 -- 172.21.15.81:0/2628279203 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd900af7f0 msgr2=0x7efd900afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd8f7fe640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd900af7f0 0x7efd900afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd8f7fe640 1 -- 172.21.15.81:0/2628279203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efd780047c0 msgr2=0x7efd9014ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd8f7fe640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efd780047c0 0x7efd9014ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.192+0000 7efd8f7fe640 1 -- 172.21.15.81:0/2628279203 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900cd6b0 con 0x7efd901545f0 2023-11-08T16:37:07.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.193+0000 7efd94eb5640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd900af7f0 0x7efd900afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:07.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.193+0000 7efd8f7fe640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd901545f0 0x7efd90154a70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7efd8402bdc0 tx=0x7efd840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:07.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.193+0000 7efd8cff9640 1 -- 172.21.15.81:0/2628279203 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8403d850 con 0x7efd901545f0 2023-11-08T16:37:07.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.193+0000 7efd8cff9640 1 -- 172.21.15.81:0/2628279203 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd8403d9b0 con 0x7efd901545f0 2023-11-08T16:37:07.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.193+0000 7efd96918640 1 -- 172.21.15.81:0/2628279203 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd900cd990 con 0x7efd901545f0 2023-11-08T16:37:07.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.193+0000 7efd8cff9640 1 -- 172.21.15.81:0/2628279203 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd8403dd40 con 0x7efd901545f0 2023-11-08T16:37:07.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.193+0000 7efd96918640 1 -- 172.21.15.81:0/2628279203 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd900cdf50 con 0x7efd901545f0 2023-11-08T16:37:07.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.194+0000 7efd96918640 1 -- 172.21.15.81:0/2628279203 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd900ce0e0 con 0x7efd901545f0 2023-11-08T16:37:07.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.195+0000 7efd8cff9640 1 -- 172.21.15.81:0/2628279203 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efd84032420 con 0x7efd901545f0 2023-11-08T16:37:07.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.196+0000 7efd8cff9640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efd7c06aea0 0x7efd7c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.196+0000 7efd8cff9640 1 -- 172.21.15.81:0/2628279203 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7efd840ab5e0 con 0x7efd901545f0 2023-11-08T16:37:07.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.197+0000 7efd8ffff640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efd7c06aea0 0x7efd7c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.197+0000 7efd8ffff640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efd7c06aea0 0x7efd7c06d360 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efd900af190 tx=0x7efd80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:07.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.201+0000 7efd8cff9640 1 -- 172.21.15.81:0/2628279203 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efd840763a0 con 0x7efd901545f0 2023-11-08T16:37:07.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.391+0000 7efd96918640 1 -- 172.21.15.81:0/2628279203 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd90066ce0 con 0x7efd901545f0 2023-11-08T16:37:07.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.393+0000 7efd8cff9640 1 -- 172.21.15.81:0/2628279203 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18768 (secure 0 0 0) 0x7efd8400f040 con 0x7efd901545f0 2023-11-08T16:37:07.396 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:07.397 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":68,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:34:53.906443+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:32:41.268027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T16:32:42.449607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:32:53.335342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.e","mappings":[{"from":0,"to":6}]},{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"9.11","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/321347143":"2023-11-09T16:34:45.940255+0000","172.21.15.81:6837/1446454901":"2023-11-09T16:34:43.853227+0000","172.21.15.81:6837/494395478":"2023-11-09T16:34:53.172859+0000","172.21.15.81:6835/494395478":"2023-11-09T16:34:53.172859+0000","172.21.15.81:6835/321347143":"2023-11-09T16:34:45.940255+0000","172.21.15.81:6835/1446454901":"2023-11-09T16:34:43.853227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:07.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 -- 172.21.15.81:0/2628279203 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efd7c06aea0 msgr2=0x7efd7c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efd7c06aea0 0x7efd7c06d360 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7efd900af190 tx=0x7efd80008040 comp rx=0 tx=0).stop 2023-11-08T16:37:07.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 -- 172.21.15.81:0/2628279203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd901545f0 msgr2=0x7efd90154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd901545f0 0x7efd90154a70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7efd8402bdc0 tx=0x7efd840046c0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 -- 172.21.15.81:0/2628279203 shutdown_connections 2023-11-08T16:37:07.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efd7c06aea0 0x7efd7c06d360 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efd900af7f0 0x7efd900afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efd901545f0 0x7efd90154a70 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 --2- 172.21.15.81:0/2628279203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efd780047c0 0x7efd9014ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 -- 172.21.15.81:0/2628279203 >> 172.21.15.81:0/2628279203 conn(0x7efd900cb490 msgr2=0x7efd900b23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:07.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 -- 172.21.15.81:0/2628279203 shutdown_connections 2023-11-08T16:37:07.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.396+0000 7efd96918640 1 -- 172.21.15.81:0/2628279203 wait complete. 2023-11-08T16:37:07.418 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:32:41.268027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:32:42.449607+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:32:53.335342+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:37:07.419 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:37:07.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.631+0000 7f22d0df4640 1 Processor -- start 2023-11-08T16:37:07.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.632+0000 7f22d0df4640 1 -- start start 2023-11-08T16:37:07.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.633+0000 7f22d0df4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f22cc15cf30 0x7f22cc0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.633+0000 7f22d0df4640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f22cc0cfd90 0x7f22cc0cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.633+0000 7f22d0df4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22cc0ce0b0 con 0x7f22cc15d310 2023-11-08T16:37:07.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.633+0000 7f22d0df4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22cc0cdb70 con 0x7f22cc15cf30 2023-11-08T16:37:07.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.633+0000 7f22d0df4640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22cc0ce390 con 0x7f22cc0cfd90 2023-11-08T16:37:07.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.633+0000 7f22cb7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f22cc15cf30 0x7f22cc0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.634+0000 7f22cb7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f22cc15cf30 0x7f22cc0d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37464/0 (socket says 172.21.15.81:37464) 2023-11-08T16:37:07.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.634+0000 7f22cb7fe640 1 -- 172.21.15.81:0/3432938573 learned_addr learned my addr 172.21.15.81:0/3432938573 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:07.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.634+0000 7f22caffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f22cc15d310 0x7f22cc0cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51510/0 (socket says 172.21.15.81:51510) 2023-11-08T16:37:07.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.634+0000 7f22cb7fe640 1 -- 172.21.15.81:0/3432938573 >> v2:172.21.15.93:3300/0 conn(0x7f22cc0cfd90 msgr2=0x7f22cc0cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:07.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.634+0000 7f22cb7fe640 1 --2- 172.21.15.81:0/3432938573 >> v2:172.21.15.93:3300/0 conn(0x7f22cc0cfd90 0x7f22cc0cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.634+0000 7f22cb7fe640 1 -- 172.21.15.81:0/3432938573 >> v1:172.21.15.81:6789/0 conn(0x7f22cc15d310 legacy=0x7f22cc0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.634+0000 7f22cb7fe640 1 -- 172.21.15.81:0/3432938573 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22cc0d1be0 con 0x7f22cc15cf30 2023-11-08T16:37:07.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.635+0000 7f22cb7fe640 1 --2- 172.21.15.81:0/3432938573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22cc15cf30 0x7f22cc0d26c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009870 tx=0x7f22bc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2967cc70959f57ca server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:07.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.635+0000 7f22ca7fc640 1 -- 172.21.15.81:0/3432938573 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc030860 con 0x7f22cc15cf30 2023-11-08T16:37:07.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.635+0000 7f22ca7fc640 1 -- 172.21.15.81:0/3432938573 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc0309c0 con 0x7f22cc15cf30 2023-11-08T16:37:07.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.635+0000 7f22ca7fc640 1 -- 172.21.15.81:0/3432938573 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc030d50 con 0x7f22cc15cf30 2023-11-08T16:37:07.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.636+0000 7f22d0df4640 1 -- 172.21.15.81:0/3432938573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22cc15cf30 msgr2=0x7f22cc0d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.636+0000 7f22d0df4640 1 --2- 172.21.15.81:0/3432938573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22cc15cf30 0x7f22cc0d26c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009870 tx=0x7f22bc02b9a0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.636+0000 7f22d0df4640 1 -- 172.21.15.81:0/3432938573 shutdown_connections 2023-11-08T16:37:07.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.636+0000 7f22d0df4640 1 --2- 172.21.15.81:0/3432938573 >> v2:172.21.15.93:3300/0 conn(0x7f22cc0cfd90 0x7f22cc0cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.636+0000 7f22d0df4640 1 --2- 172.21.15.81:0/3432938573 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22cc15cf30 0x7f22cc0d26c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.636+0000 7f22d0df4640 1 -- 172.21.15.81:0/3432938573 >> 172.21.15.81:0/3432938573 conn(0x7f22cc06c040 msgr2=0x7f22cc06c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:07.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.636+0000 7f22d0df4640 1 -- 172.21.15.81:0/3432938573 shutdown_connections 2023-11-08T16:37:07.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.636+0000 7f22d0df4640 1 -- 172.21.15.81:0/3432938573 wait complete. 2023-11-08T16:37:07.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.638+0000 7f22d0df4640 1 Processor -- start 2023-11-08T16:37:07.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.638+0000 7f22d0df4640 1 -- start start 2023-11-08T16:37:07.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.639+0000 7f22d0df4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22cc0cfd90 0x7f22cc15f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.639+0000 7f22d0df4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f22cc15cf30 0x7f22cc14cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.639+0000 7f22d0df4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22cc14d220 0x7f22cc0b25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.639+0000 7f22d0df4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc14d6a0 con 0x7f22cc0cfd90 2023-11-08T16:37:07.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.639+0000 7f22d0df4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0b3320 con 0x7f22cc14d220 2023-11-08T16:37:07.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.639+0000 7f22d0df4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22cc0b3600 con 0x7f22cc15cf30 2023-11-08T16:37:07.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.639+0000 7f22cb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22cc0cfd90 0x7f22cc15f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.639+0000 7f22cb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22cc0cfd90 0x7f22cc15f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37468/0 (socket says 172.21.15.81:37468) 2023-11-08T16:37:07.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.639+0000 7f22cb7fe640 1 -- 172.21.15.81:0/3135666993 learned_addr learned my addr 172.21.15.81:0/3135666993 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:07.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22cbfff640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22cc14d220 0x7f22cc0b25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22caffd640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f22cc15cf30 0x7f22cc14cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22cb7fe640 1 -- 172.21.15.81:0/3135666993 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f22cc15cf30 msgr2=0x7f22cc14cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22cb7fe640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f22cc15cf30 0x7f22cc14cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22cb7fe640 1 -- 172.21.15.81:0/3135666993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22cc14d220 msgr2=0x7f22cc0b25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22cb7fe640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22cc14d220 0x7f22cc0b25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22cb7fe640 1 -- 172.21.15.81:0/3135666993 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22bc0094d0 con 0x7f22cc0cfd90 2023-11-08T16:37:07.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22cbfff640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22cc14d220 0x7f22cc0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:07.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22cb7fe640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22cc0cfd90 0x7f22cc15f7e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009870 tx=0x7f22bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:07.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22c3fff640 1 -- 172.21.15.81:0/3135666993 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc039d60 con 0x7f22cc0cfd90 2023-11-08T16:37:07.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22c3fff640 1 -- 172.21.15.81:0/3135666993 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22bc043400 con 0x7f22cc0cfd90 2023-11-08T16:37:07.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22c3fff640 1 -- 172.21.15.81:0/3135666993 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22bc0437b0 con 0x7f22cc0cfd90 2023-11-08T16:37:07.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.640+0000 7f22d0df4640 1 -- 172.21.15.81:0/3135666993 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22cc0b38e0 con 0x7f22cc0cfd90 2023-11-08T16:37:07.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.641+0000 7f22d0df4640 1 -- 172.21.15.81:0/3135666993 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22cc0b3ea0 con 0x7f22cc0cfd90 2023-11-08T16:37:07.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.642+0000 7f22d0df4640 1 -- 172.21.15.81:0/3135666993 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22cc04ece0 con 0x7f22cc0cfd90 2023-11-08T16:37:07.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.643+0000 7f22c3fff640 1 -- 172.21.15.81:0/3135666993 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f22bc043910 con 0x7f22cc0cfd90 2023-11-08T16:37:07.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.644+0000 7f22c3fff640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f22b006ae70 0x7f22b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:07.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.644+0000 7f22c3fff640 1 -- 172.21.15.81:0/3135666993 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f22bc03e070 con 0x7f22cc0cfd90 2023-11-08T16:37:07.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.644+0000 7f22caffd640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f22b006ae70 0x7f22b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:07.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.644+0000 7f22caffd640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f22b006ae70 0x7f22b006d330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0cc720 tx=0x7f22b4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:07.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.648+0000 7f22c3fff640 1 -- 172.21.15.81:0/3135666993 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f22bc077af0 con 0x7f22cc0cfd90 2023-11-08T16:37:07.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.838+0000 7f22d0df4640 1 -- 172.21.15.81:0/3135666993 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f22cc1f2840 con 0x7f22cc0cfd90 2023-11-08T16:37:07.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.839+0000 7f22c3fff640 1 -- 172.21.15.81:0/3135666993 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f22bc07b110 con 0x7f22cc0cfd90 2023-11-08T16:37:07.842 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:37:07.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.841+0000 7f22d0df4640 1 -- 172.21.15.81:0/3135666993 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f22b006ae70 msgr2=0x7f22b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.841+0000 7f22d0df4640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f22b006ae70 0x7f22b006d330 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f22cc0cc720 tx=0x7f22b4009040 comp rx=0 tx=0).stop 2023-11-08T16:37:07.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.841+0000 7f22d0df4640 1 -- 172.21.15.81:0/3135666993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22cc0cfd90 msgr2=0x7f22cc15f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:07.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.841+0000 7f22d0df4640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22cc0cfd90 0x7f22cc15f7e0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f22bc009870 tx=0x7f22bc004730 comp rx=0 tx=0).stop 2023-11-08T16:37:07.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.841+0000 7f22d0df4640 1 -- 172.21.15.81:0/3135666993 shutdown_connections 2023-11-08T16:37:07.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.841+0000 7f22d0df4640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f22b006ae70 0x7f22b006d330 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.841+0000 7f22d0df4640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22cc14d220 0x7f22cc0b25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.841+0000 7f22d0df4640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f22cc15cf30 0x7f22cc14cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.841+0000 7f22d0df4640 1 --2- 172.21.15.81:0/3135666993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22cc0cfd90 0x7f22cc15f7e0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:07.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.841+0000 7f22d0df4640 1 -- 172.21.15.81:0/3135666993 >> 172.21.15.81:0/3135666993 conn(0x7f22cc06c040 msgr2=0x7f22cc0cf660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:07.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.841+0000 7f22d0df4640 1 -- 172.21.15.81:0/3135666993 shutdown_connections 2023-11-08T16:37:07.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:07.842+0000 7f22d0df4640 1 -- 172.21.15.81:0/3135666993 wait complete. 2023-11-08T16:37:07.862 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:37:08.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.079+0000 7f44d6df5640 1 Processor -- start 2023-11-08T16:37:08.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.079+0000 7f44d6df5640 1 -- start start 2023-11-08T16:37:08.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.081+0000 7f44d6df5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f44d00ce390 0x7f44d015e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.081+0000 7f44d6df5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d015e8d0 con 0x7f44d006be20 2023-11-08T16:37:08.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.081+0000 7f44d6df5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d00ca2f0 con 0x7f44d00c9200 2023-11-08T16:37:08.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.081+0000 7f44d6df5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d00ce770 con 0x7f44d00ce390 2023-11-08T16:37:08.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.081+0000 7f44d4b91640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f44d00ce390 0x7f44d015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.081+0000 7f44d5392640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f44d006be20 0x7f44d00ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51526/0 (socket says 172.21.15.81:51526) 2023-11-08T16:37:08.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.081+0000 7f44d5392640 1 -- 172.21.15.81:0/4063793652 learned_addr learned my addr 172.21.15.81:0/4063793652 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:08.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.082+0000 7f44d4b91640 1 -- 172.21.15.81:0/4063793652 >> v1:172.21.15.93:6789/0 conn(0x7f44d00c9200 legacy=0x7f44d006b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:08.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.082+0000 7f44d4b91640 1 -- 172.21.15.81:0/4063793652 >> v1:172.21.15.81:6789/0 conn(0x7f44d006be20 legacy=0x7f44d00ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.082+0000 7f44d4b91640 1 -- 172.21.15.81:0/4063793652 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d00c96e0 con 0x7f44d00ce390 2023-11-08T16:37:08.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.082+0000 7f44c77fe640 1 -- 172.21.15.81:0/4063793652 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175140649 0 0) 0x7f44c8001490 con 0x7f44d006be20 2023-11-08T16:37:08.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.083+0000 7f44d4b91640 1 --2- 172.21.15.81:0/4063793652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d00ce390 0x7f44d015e390 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0097d0 tx=0x7f44cc02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0bcd98af9f7b25 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:08.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.083+0000 7f44c77fe640 1 -- 172.21.15.81:0/4063793652 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc030800 con 0x7f44d00ce390 2023-11-08T16:37:08.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.083+0000 7f44c77fe640 1 -- 172.21.15.81:0/4063793652 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc030960 con 0x7f44d00ce390 2023-11-08T16:37:08.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.083+0000 7f44c77fe640 1 -- 172.21.15.81:0/4063793652 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc030cf0 con 0x7f44d00ce390 2023-11-08T16:37:08.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.084+0000 7f44d6df5640 1 -- 172.21.15.81:0/4063793652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d00ce390 msgr2=0x7f44d015e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.084+0000 7f44d6df5640 1 --2- 172.21.15.81:0/4063793652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d00ce390 0x7f44d015e390 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f44cc0097d0 tx=0x7f44cc02b940 comp rx=0 tx=0).stop 2023-11-08T16:37:08.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.084+0000 7f44d6df5640 1 -- 172.21.15.81:0/4063793652 shutdown_connections 2023-11-08T16:37:08.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.084+0000 7f44d6df5640 1 --2- 172.21.15.81:0/4063793652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d00ce390 0x7f44d015e390 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.084+0000 7f44d6df5640 1 -- 172.21.15.81:0/4063793652 >> 172.21.15.81:0/4063793652 conn(0x7f44d00cb580 msgr2=0x7f44d00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:08.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.084+0000 7f44d6df5640 1 -- 172.21.15.81:0/4063793652 shutdown_connections 2023-11-08T16:37:08.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.084+0000 7f44d6df5640 1 -- 172.21.15.81:0/4063793652 wait complete. 2023-11-08T16:37:08.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.086+0000 7f44d6df5640 1 Processor -- start 2023-11-08T16:37:08.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.086+0000 7f44d6df5640 1 -- start start 2023-11-08T16:37:08.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.087+0000 7f44d6df5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44d006be20 0x7f44d0157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.087+0000 7f44d6df5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d00c9200 0x7f44d0153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.087+0000 7f44d6df5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44d00ce390 0x7f44d00b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.087+0000 7f44d6df5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00b45a0 con 0x7f44d00c9200 2023-11-08T16:37:08.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.087+0000 7f44d6df5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00c3d90 con 0x7f44d006be20 2023-11-08T16:37:08.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.087+0000 7f44d6df5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d00b3ac0 con 0x7f44d00ce390 2023-11-08T16:37:08.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.087+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d00c9200 0x7f44d0153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.087+0000 7f44c7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d00c9200 0x7f44d0153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37484/0 (socket says 172.21.15.81:37484) 2023-11-08T16:37:08.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.087+0000 7f44c7fff640 1 -- 172.21.15.81:0/949042158 learned_addr learned my addr 172.21.15.81:0/949042158 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.087+0000 7f44d4b91640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44d006be20 0x7f44d0157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44d5392640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44d00ce390 0x7f44d00b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44c7fff640 1 -- 172.21.15.81:0/949042158 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44d00ce390 msgr2=0x7f44d00b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44c7fff640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44d00ce390 0x7f44d00b2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44c7fff640 1 -- 172.21.15.81:0/949042158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44d006be20 msgr2=0x7f44d0157110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44c7fff640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44d006be20 0x7f44d0157110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44c7fff640 1 -- 172.21.15.81:0/949042158 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44cc009400 con 0x7f44d00c9200 2023-11-08T16:37:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44c7fff640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d00c9200 0x7f44d0153dc0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f44c000c400 tx=0x7f44c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:08.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44c57fa640 1 -- 172.21.15.81:0/949042158 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0011790 con 0x7f44d00c9200 2023-11-08T16:37:08.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44c57fa640 1 -- 172.21.15.81:0/949042158 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44c0011930 con 0x7f44d00c9200 2023-11-08T16:37:08.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44c57fa640 1 -- 172.21.15.81:0/949042158 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44c0011ce0 con 0x7f44d00c9200 2023-11-08T16:37:08.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44d6df5640 1 -- 172.21.15.81:0/949042158 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d00b3da0 con 0x7f44d00c9200 2023-11-08T16:37:08.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.088+0000 7f44d6df5640 1 -- 172.21.15.81:0/949042158 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d0154300 con 0x7f44d00c9200 2023-11-08T16:37:08.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.089+0000 7f44d6df5640 1 -- 172.21.15.81:0/949042158 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d0154490 con 0x7f44d00c9200 2023-11-08T16:37:08.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.090+0000 7f44c57fa640 1 -- 172.21.15.81:0/949042158 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44c001c460 con 0x7f44d00c9200 2023-11-08T16:37:08.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.091+0000 7f44c57fa640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44bc06ae70 0x7f44bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.091+0000 7f44c57fa640 1 -- 172.21.15.81:0/949042158 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f44c0089920 con 0x7f44d00c9200 2023-11-08T16:37:08.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.092+0000 7f44d4b91640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44bc06ae70 0x7f44bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.092+0000 7f44d4b91640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44bc06ae70 0x7f44bc06d330 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f44cc036000 tx=0x7f44cc03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:08.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.096+0000 7f44c57fa640 1 -- 172.21.15.81:0/949042158 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44d0154490 con 0x7f44d00c9200 2023-11-08T16:37:08.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.284+0000 7f44d6df5640 1 -- 172.21.15.81:0/949042158 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f44d0154490 con 0x7f44d00c9200 2023-11-08T16:37:08.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.286+0000 7f44c57fa640 1 -- 172.21.15.81:0/949042158 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7f44d0154490 con 0x7f44d00c9200 2023-11-08T16:37:08.288 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:37:08.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.288+0000 7f44d6df5640 1 -- 172.21.15.81:0/949042158 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44bc06ae70 msgr2=0x7f44bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.288+0000 7f44d6df5640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44bc06ae70 0x7f44bc06d330 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f44cc036000 tx=0x7f44cc03b040 comp rx=0 tx=0).stop 2023-11-08T16:37:08.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.288+0000 7f44d6df5640 1 -- 172.21.15.81:0/949042158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d00c9200 msgr2=0x7f44d0153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.288+0000 7f44d6df5640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d00c9200 0x7f44d0153dc0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f44c000c400 tx=0x7f44c000c8c0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.288+0000 7f44d6df5640 1 -- 172.21.15.81:0/949042158 shutdown_connections 2023-11-08T16:37:08.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.288+0000 7f44d6df5640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44bc06ae70 0x7f44bc06d330 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.288+0000 7f44d6df5640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44d00ce390 0x7f44d00b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.288+0000 7f44d6df5640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d00c9200 0x7f44d0153dc0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.288+0000 7f44d6df5640 1 --2- 172.21.15.81:0/949042158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44d006be20 0x7f44d0157110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.288+0000 7f44d6df5640 1 -- 172.21.15.81:0/949042158 >> 172.21.15.81:0/949042158 conn(0x7f44d00cb580 msgr2=0x7f44d00cd1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:08.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.289+0000 7f44d6df5640 1 -- 172.21.15.81:0/949042158 shutdown_connections 2023-11-08T16:37:08.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.289+0000 7f44d6df5640 1 -- 172.21.15.81:0/949042158 wait complete. 2023-11-08T16:37:08.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:37:08.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.517+0000 7f17e1a10640 1 Processor -- start 2023-11-08T16:37:08.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.517+0000 7f17e1a10640 1 -- start start 2023-11-08T16:37:08.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.519+0000 7f17e1a10640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f17dc0c9200 0x7f17dc15e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.519+0000 7f17e1a10640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17dc15e8d0 con 0x7f17dc0ce390 2023-11-08T16:37:08.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.519+0000 7f17e1a10640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17dc0ca2f0 con 0x7f17dc06be20 2023-11-08T16:37:08.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.519+0000 7f17e1a10640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17dc0ce770 con 0x7f17dc0c9200 2023-11-08T16:37:08.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.519+0000 7f17db7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f17dc0c9200 0x7f17dc15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.520+0000 7f17daffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f17dc0ce390 0x7f17dc06b710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51534/0 (socket says 172.21.15.81:51534) 2023-11-08T16:37:08.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.520+0000 7f17daffd640 1 -- 172.21.15.81:0/3027980900 learned_addr learned my addr 172.21.15.81:0/3027980900 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:08.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.520+0000 7f17db7fe640 1 -- 172.21.15.81:0/3027980900 >> v1:172.21.15.93:6789/0 conn(0x7f17dc06be20 legacy=0x7f17dc0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.520+0000 7f17da7fc640 1 -- 172.21.15.81:0/3027980900 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134177573 0 0) 0x7f17cc001110 con 0x7f17dc0ce390 2023-11-08T16:37:08.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.520+0000 7f17db7fe640 1 -- 172.21.15.81:0/3027980900 >> v1:172.21.15.81:6789/0 conn(0x7f17dc0ce390 legacy=0x7f17dc06b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.520+0000 7f17db7fe640 1 -- 172.21.15.81:0/3027980900 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17dc0c96e0 con 0x7f17dc0c9200 2023-11-08T16:37:08.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.521+0000 7f17db7fe640 1 --2- 172.21.15.81:0/3027980900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc0c9200 0x7f17dc15e390 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009d90 tx=0x7f17d402bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82ca7dc96ce888cd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:08.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.522+0000 7f17da7fc640 1 -- 172.21.15.81:0/3027980900 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d402cd80 con 0x7f17dc0c9200 2023-11-08T16:37:08.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.522+0000 7f17da7fc640 1 -- 172.21.15.81:0/3027980900 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17d4034440 con 0x7f17dc0c9200 2023-11-08T16:37:08.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.522+0000 7f17da7fc640 1 -- 172.21.15.81:0/3027980900 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17d40347d0 con 0x7f17dc0c9200 2023-11-08T16:37:08.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.522+0000 7f17e1a10640 1 -- 172.21.15.81:0/3027980900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc0c9200 msgr2=0x7f17dc15e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.522+0000 7f17e1a10640 1 --2- 172.21.15.81:0/3027980900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc0c9200 0x7f17dc15e390 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f17d4009d90 tx=0x7f17d402bfd0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.522+0000 7f17e1a10640 1 -- 172.21.15.81:0/3027980900 shutdown_connections 2023-11-08T16:37:08.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.522+0000 7f17e1a10640 1 --2- 172.21.15.81:0/3027980900 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc0c9200 0x7f17dc15e390 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.522+0000 7f17e1a10640 1 -- 172.21.15.81:0/3027980900 >> 172.21.15.81:0/3027980900 conn(0x7f17dc0cb580 msgr2=0x7f17dc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:08.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.522+0000 7f17e1a10640 1 -- 172.21.15.81:0/3027980900 shutdown_connections 2023-11-08T16:37:08.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.522+0000 7f17e1a10640 1 -- 172.21.15.81:0/3027980900 wait complete. 2023-11-08T16:37:08.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.524+0000 7f17e1a10640 1 Processor -- start 2023-11-08T16:37:08.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.524+0000 7f17e1a10640 1 -- start start 2023-11-08T16:37:08.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.525+0000 7f17e1a10640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc06be20 0x7f17dc15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.525+0000 7f17e1a10640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17dc0c9200 0x7f17dc14b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.525+0000 7f17e1a10640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17dc0ce390 0x7f17dc14bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.525+0000 7f17e1a10640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc1e37f0 con 0x7f17dc06be20 2023-11-08T16:37:08.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.525+0000 7f17e1a10640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc0c35b0 con 0x7f17dc0c9200 2023-11-08T16:37:08.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.525+0000 7f17e1a10640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17dc0c3890 con 0x7f17dc0ce390 2023-11-08T16:37:08.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.525+0000 7f17daffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc06be20 0x7f17dc15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.525+0000 7f17daffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc06be20 0x7f17dc15c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37492/0 (socket says 172.21.15.81:37492) 2023-11-08T16:37:08.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.525+0000 7f17daffd640 1 -- 172.21.15.81:0/2600230871 learned_addr learned my addr 172.21.15.81:0/2600230871 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:08.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.526+0000 7f17daffd640 1 -- 172.21.15.81:0/2600230871 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17dc0ce390 msgr2=0x7f17dc14bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.526+0000 7f17daffd640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17dc0ce390 0x7f17dc14bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.526+0000 7f17daffd640 1 -- 172.21.15.81:0/2600230871 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17dc0c9200 msgr2=0x7f17dc14b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.526+0000 7f17daffd640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17dc0c9200 0x7f17dc14b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.526+0000 7f17daffd640 1 -- 172.21.15.81:0/2600230871 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d4009a70 con 0x7f17dc06be20 2023-11-08T16:37:08.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.526+0000 7f17daffd640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc06be20 0x7f17dc15c740 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00c3f0 tx=0x7f17cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:08.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.526+0000 7f17d8ff9640 1 -- 172.21.15.81:0/2600230871 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc0074e0 con 0x7f17dc06be20 2023-11-08T16:37:08.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.526+0000 7f17d8ff9640 1 -- 172.21.15.81:0/2600230871 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17cc007640 con 0x7f17dc06be20 2023-11-08T16:37:08.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.526+0000 7f17d8ff9640 1 -- 172.21.15.81:0/2600230871 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17cc004a00 con 0x7f17dc06be20 2023-11-08T16:37:08.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.526+0000 7f17e1a10640 1 -- 172.21.15.81:0/2600230871 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17dc0c3b70 con 0x7f17dc06be20 2023-11-08T16:37:08.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.527+0000 7f17e1a10640 1 -- 172.21.15.81:0/2600230871 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17dc15cc80 con 0x7f17dc06be20 2023-11-08T16:37:08.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.528+0000 7f17e1a10640 1 -- 172.21.15.81:0/2600230871 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17dc1e3930 con 0x7f17dc06be20 2023-11-08T16:37:08.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.529+0000 7f17d8ff9640 1 -- 172.21.15.81:0/2600230871 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f17cc004b60 con 0x7f17dc06be20 2023-11-08T16:37:08.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.530+0000 7f17d8ff9640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17bc06ae70 0x7f17bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.530+0000 7f17d8ff9640 1 -- 172.21.15.81:0/2600230871 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f17cc0895f0 con 0x7f17dc06be20 2023-11-08T16:37:08.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.530+0000 7f17d3fff640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17bc06ae70 0x7f17bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.531+0000 7f17d3fff640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17bc06ae70 0x7f17bc06d330 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f17c4000f60 tx=0x7f17c40079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:08.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.534+0000 7f17d8ff9640 1 -- 172.21.15.81:0/2600230871 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f17dc1e3930 con 0x7f17dc06be20 2023-11-08T16:37:08.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.721+0000 7f17e1a10640 1 -- 172.21.15.81:0/2600230871 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f17dc15e8d0 con 0x7f17dc06be20 2023-11-08T16:37:08.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.722+0000 7f17d8ff9640 1 -- 172.21.15.81:0/2600230871 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7f17dc15e8d0 con 0x7f17dc06be20 2023-11-08T16:37:08.724 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:37:08.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.724+0000 7f17e1a10640 1 -- 172.21.15.81:0/2600230871 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17bc06ae70 msgr2=0x7f17bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.724+0000 7f17e1a10640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17bc06ae70 0x7f17bc06d330 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f17c4000f60 tx=0x7f17c40079b0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.724+0000 7f17e1a10640 1 -- 172.21.15.81:0/2600230871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc06be20 msgr2=0x7f17dc15c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.724+0000 7f17e1a10640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc06be20 0x7f17dc15c740 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f17cc00c3f0 tx=0x7f17cc00c8b0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.724+0000 7f17e1a10640 1 -- 172.21.15.81:0/2600230871 shutdown_connections 2023-11-08T16:37:08.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.724+0000 7f17e1a10640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f17bc06ae70 0x7f17bc06d330 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.724+0000 7f17e1a10640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17dc0ce390 0x7f17dc14bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.724+0000 7f17e1a10640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17dc0c9200 0x7f17dc14b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.724+0000 7f17e1a10640 1 --2- 172.21.15.81:0/2600230871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f17dc06be20 0x7f17dc15c740 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.724+0000 7f17e1a10640 1 -- 172.21.15.81:0/2600230871 >> 172.21.15.81:0/2600230871 conn(0x7f17dc0cb580 msgr2=0x7f17dc0c7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:08.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.724+0000 7f17e1a10640 1 -- 172.21.15.81:0/2600230871 shutdown_connections 2023-11-08T16:37:08.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.725+0000 7f17e1a10640 1 -- 172.21.15.81:0/2600230871 wait complete. 2023-11-08T16:37:08.745 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:37:08.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.944+0000 7f3ca118c640 1 Processor -- start 2023-11-08T16:37:08.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.944+0000 7f3ca118c640 1 -- start start 2023-11-08T16:37:08.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.946+0000 7f3ca118c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3c9c1612f0 0x7f3c9c06c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.946+0000 7f3ca118c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9c15d680 con 0x7f3c9c1616d0 2023-11-08T16:37:08.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.946+0000 7f3ca118c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c9c15d960 con 0x7f3c9c06cc70 2023-11-08T16:37:08.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.946+0000 7f3ca118c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c9c15d4a0 con 0x7f3c9c1612f0 2023-11-08T16:37:08.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c93fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3c9c1616d0 0x7f3c9c0c9f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51542/0 (socket says 172.21.15.81:51542) 2023-11-08T16:37:08.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c93fff640 1 -- 172.21.15.81:0/3912311421 learned_addr learned my addr 172.21.15.81:0/3912311421 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:08.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c9bfff640 1 --2- 172.21.15.81:0/3912311421 >> v2:172.21.15.93:3300/0 conn(0x7f3c9c1612f0 0x7f3c9c06c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227921637 0 0) 0x7f3c9c15d680 con 0x7f3c9c1616d0 2023-11-08T16:37:08.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c88003520 con 0x7f3c9c1616d0 2023-11-08T16:37:08.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2005312651 0 0) 0x7f3c88003520 con 0x7f3c9c1616d0 2023-11-08T16:37:08.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 >> v2:172.21.15.93:3300/0 conn(0x7f3c9c1612f0 msgr2=0x7f3c9c06c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:08.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c9b7fe640 1 --2- 172.21.15.81:0/3912311421 >> v2:172.21.15.93:3300/0 conn(0x7f3c9c1612f0 0x7f3c9c06c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 >> v1:172.21.15.93:6789/0 conn(0x7f3c9c06cc70 legacy=0x7f3c9c0cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c15dc40 con 0x7f3c9c1616d0 2023-11-08T16:37:08.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f3c840017f0 con 0x7f3c9c1616d0 2023-11-08T16:37:08.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 >> v1:172.21.15.81:6789/0 conn(0x7f3c9c1616d0 legacy=0x7f3c9c0c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.947+0000 7f3c9b7fe640 1 --2- 172.21.15.81:0/3912311421 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c88003970 0x7f3c88003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3c9b7fe640 1 --2- 172.21.15.81:0/3912311421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 0x7f3c88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3c9b7fe640 1 --2- 172.21.15.81:0/3912311421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c88005180 0x7f3c8800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c15d4a0 con 0x7f3c880047c0 2023-11-08T16:37:08.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8800c020 con 0x7f3c88005180 2023-11-08T16:37:08.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8800c1e0 con 0x7f3c88003970 2023-11-08T16:37:08.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3c93fff640 1 --2- 172.21.15.81:0/3912311421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 0x7f3c88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3c93fff640 1 -- 172.21.15.81:0/3912311421 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c88003970 msgr2=0x7f3c88003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:08.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3c93fff640 1 --2- 172.21.15.81:0/3912311421 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c88003970 0x7f3c88003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3c93fff640 1 -- 172.21.15.81:0/3912311421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c88005180 msgr2=0x7f3c8800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:08.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3ca098b640 1 --2- 172.21.15.81:0/3912311421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c88005180 0x7f3c8800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3c93fff640 1 --2- 172.21.15.81:0/3912311421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c88005180 0x7f3c8800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3c93fff640 1 -- 172.21.15.81:0/3912311421 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c88003520 con 0x7f3c880047c0 2023-11-08T16:37:08.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3ca098b640 1 --2- 172.21.15.81:0/3912311421 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c88005180 0x7f3c8800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:08.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.948+0000 7f3ca098b640 1 -- 172.21.15.81:0/3912311421 reap_dead start 2023-11-08T16:37:08.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.949+0000 7f3c93fff640 1 --2- 172.21.15.81:0/3912311421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 0x7f3c88004c40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400c190 tx=0x7f3c880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a489885bf090559 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:08.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.950+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c84007270 con 0x7f3c880047c0 2023-11-08T16:37:08.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.950+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c840073f0 con 0x7f3c880047c0 2023-11-08T16:37:08.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.950+0000 7f3c9b7fe640 1 -- 172.21.15.81:0/3912311421 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8402c8a0 con 0x7f3c880047c0 2023-11-08T16:37:08.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.950+0000 7f3ca118c640 1 -- 172.21.15.81:0/3912311421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 msgr2=0x7f3c88004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.950+0000 7f3ca118c640 1 --2- 172.21.15.81:0/3912311421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 0x7f3c88004c40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400c190 tx=0x7f3c880057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.950+0000 7f3ca118c640 1 -- 172.21.15.81:0/3912311421 shutdown_connections 2023-11-08T16:37:08.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.950+0000 7f3ca118c640 1 --2- 172.21.15.81:0/3912311421 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 0x7f3c88004c40 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.950+0000 7f3ca118c640 1 -- 172.21.15.81:0/3912311421 >> 172.21.15.81:0/3912311421 conn(0x7f3c9c0ce8d0 msgr2=0x7f3c9c0cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:08.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.951+0000 7f3ca118c640 1 -- 172.21.15.81:0/3912311421 shutdown_connections 2023-11-08T16:37:08.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.951+0000 7f3ca118c640 1 -- 172.21.15.81:0/3912311421 wait complete. 2023-11-08T16:37:08.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.952+0000 7f3ca118c640 1 Processor -- start 2023-11-08T16:37:08.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.953+0000 7f3ca118c640 1 -- start start 2023-11-08T16:37:08.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.953+0000 7f3ca118c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 0x7f3c9c1f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.953+0000 7f3ca118c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c9c1f3340 0x7f3c9c157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.953+0000 7f3ca118c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c9c15a520 0x7f3c9c15a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3ca118c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c1f6e30 con 0x7f3c880047c0 2023-11-08T16:37:08.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3ca118c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c1f37c0 con 0x7f3c9c15a520 2023-11-08T16:37:08.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3ca118c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9c1f6590 con 0x7f3c9c1f3340 2023-11-08T16:37:08.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3c9bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 0x7f3c9c1f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3c9bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 0x7f3c9c1f5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37522/0 (socket says 172.21.15.81:37522) 2023-11-08T16:37:08.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3c9bfff640 1 -- 172.21.15.81:0/3179037606 learned_addr learned my addr 172.21.15.81:0/3179037606 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:08.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3c9bfff640 1 -- 172.21.15.81:0/3179037606 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c9c1f3340 msgr2=0x7f3c9c157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3c9bfff640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c9c1f3340 0x7f3c9c157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3ca098b640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c9c15a520 0x7f3c9c15a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3c9bfff640 1 -- 172.21.15.81:0/3179037606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c9c15a520 msgr2=0x7f3c9c15a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:08.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3c9bfff640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c9c15a520 0x7f3c9c15a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:08.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.954+0000 7f3c9bfff640 1 -- 172.21.15.81:0/3179037606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c9c1f6870 con 0x7f3c880047c0 2023-11-08T16:37:08.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.955+0000 7f3c9bfff640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 0x7f3c9c1f5eb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00e860 tx=0x7f3c8c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:08.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.955+0000 7f3c997fa640 1 -- 172.21.15.81:0/3179037606 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c009600 con 0x7f3c880047c0 2023-11-08T16:37:08.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.955+0000 7f3c997fa640 1 -- 172.21.15.81:0/3179037606 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8c00c940 con 0x7f3c880047c0 2023-11-08T16:37:08.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.955+0000 7f3c997fa640 1 -- 172.21.15.81:0/3179037606 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c00ccd0 con 0x7f3c880047c0 2023-11-08T16:37:08.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.955+0000 7f3ca118c640 1 -- 172.21.15.81:0/3179037606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9c1f6b50 con 0x7f3c880047c0 2023-11-08T16:37:08.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.955+0000 7f3ca118c640 1 -- 172.21.15.81:0/3179037606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9c158530 con 0x7f3c880047c0 2023-11-08T16:37:08.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.956+0000 7f3ca118c640 1 -- 172.21.15.81:0/3179037606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c9c151750 con 0x7f3c880047c0 2023-11-08T16:37:08.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.957+0000 7f3c997fa640 1 -- 172.21.15.81:0/3179037606 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c8c013470 con 0x7f3c880047c0 2023-11-08T16:37:08.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.958+0000 7f3c997fa640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7c06ae70 0x7f3c7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:08.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.959+0000 7f3c997fa640 1 -- 172.21.15.81:0/3179037606 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f3c8c089c30 con 0x7f3c880047c0 2023-11-08T16:37:08.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.959+0000 7f3c93fff640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7c06ae70 0x7f3c7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:08.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.960+0000 7f3c93fff640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7c06ae70 0x7f3c7c06d330 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3c84034000 tx=0x7f3c84039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:08.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:08.963+0000 7f3c997fa640 1 -- 172.21.15.81:0/3179037606 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c9c151750 con 0x7f3c880047c0 2023-11-08T16:37:09.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.154+0000 7f3ca118c640 1 -- 172.21.15.81:0/3179037606 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f3c9c151750 con 0x7f3c880047c0 2023-11-08T16:37:09.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.155+0000 7f3c997fa640 1 -- 172.21.15.81:0/3179037606 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v68) v1 ==== 106+0+11 (secure 0 0 0) 0x7f3c9c151750 con 0x7f3c880047c0 2023-11-08T16:37:09.158 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:37:09.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.158+0000 7f3ca118c640 1 -- 172.21.15.81:0/3179037606 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7c06ae70 msgr2=0x7f3c7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:09.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.158+0000 7f3ca118c640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7c06ae70 0x7f3c7c06d330 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3c84034000 tx=0x7f3c84039040 comp rx=0 tx=0).stop 2023-11-08T16:37:09.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.158+0000 7f3ca118c640 1 -- 172.21.15.81:0/3179037606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 msgr2=0x7f3c9c1f5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:09.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.158+0000 7f3ca118c640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 0x7f3c9c1f5eb0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00e860 tx=0x7f3c8c00ed20 comp rx=0 tx=0).stop 2023-11-08T16:37:09.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.158+0000 7f3ca118c640 1 -- 172.21.15.81:0/3179037606 shutdown_connections 2023-11-08T16:37:09.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.158+0000 7f3ca118c640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7c06ae70 0x7f3c7c06d330 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.158+0000 7f3ca118c640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c9c15a520 0x7f3c9c15a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.158+0000 7f3ca118c640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c9c1f3340 0x7f3c9c157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.158+0000 7f3ca118c640 1 --2- 172.21.15.81:0/3179037606 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c880047c0 0x7f3c9c1f5eb0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.158+0000 7f3ca118c640 1 -- 172.21.15.81:0/3179037606 >> 172.21.15.81:0/3179037606 conn(0x7f3c9c0ce8d0 msgr2=0x7f3c9c06f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:09.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.159+0000 7f3ca118c640 1 -- 172.21.15.81:0/3179037606 shutdown_connections 2023-11-08T16:37:09.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.159+0000 7f3ca118c640 1 -- 172.21.15.81:0/3179037606 wait complete. 2023-11-08T16:37:09.179 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:37:09.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.383+0000 7f208bfff640 1 Processor -- start 2023-11-08T16:37:09.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.384+0000 7f208bfff640 1 -- start start 2023-11-08T16:37:09.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.385+0000 7f208bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f208c0cf170 0x7f208c15b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.385+0000 7f208bfff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208c070470 con 0x7f208c0ced10 2023-11-08T16:37:09.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.385+0000 7f208bfff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208c06d580 con 0x7f208c15aac0 2023-11-08T16:37:09.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.385+0000 7f208bfff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f208c06f650 con 0x7f208c0cf170 2023-11-08T16:37:09.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.386+0000 7f208b7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f208c0cf170 0x7f208c15b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:09.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.386+0000 7f208b7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f208c0cf170 0x7f208c15b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37538/0 (socket says 172.21.15.81:37538) 2023-11-08T16:37:09.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.386+0000 7f208b7fe640 1 -- 172.21.15.81:0/2207833804 learned_addr learned my addr 172.21.15.81:0/2207833804 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:09.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.386+0000 7f208b7fe640 1 -- 172.21.15.81:0/2207833804 >> v1:172.21.15.93:6789/0 conn(0x7f208c15aac0 legacy=0x7f208c0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:09.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.386+0000 7f208b7fe640 1 -- 172.21.15.81:0/2207833804 >> v1:172.21.15.81:6789/0 conn(0x7f208c0ced10 legacy=0x7f208c15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:09.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.386+0000 7f208b7fe640 1 -- 172.21.15.81:0/2207833804 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f208c06ca00 con 0x7f208c0cf170 2023-11-08T16:37:09.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.387+0000 7f208b7fe640 1 --2- 172.21.15.81:0/2207833804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f208c0cf170 0x7f208c15b5b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2084009e50 tx=0x7f208402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f081f954082bd71e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:09.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.387+0000 7f2089ffb640 1 -- 172.21.15.81:0/2207833804 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208402cb10 con 0x7f208c0cf170 2023-11-08T16:37:09.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.387+0000 7f2089ffb640 1 -- 172.21.15.81:0/2207833804 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208402cc70 con 0x7f208c0cf170 2023-11-08T16:37:09.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.387+0000 7f2089ffb640 1 -- 172.21.15.81:0/2207833804 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20840343f0 con 0x7f208c0cf170 2023-11-08T16:37:09.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.388+0000 7f208bfff640 1 -- 172.21.15.81:0/2207833804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f208c0cf170 msgr2=0x7f208c15b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:09.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.388+0000 7f208bfff640 1 --2- 172.21.15.81:0/2207833804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f208c0cf170 0x7f208c15b5b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2084009e50 tx=0x7f208402bf70 comp rx=0 tx=0).stop 2023-11-08T16:37:09.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.388+0000 7f208bfff640 1 -- 172.21.15.81:0/2207833804 shutdown_connections 2023-11-08T16:37:09.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.388+0000 7f208bfff640 1 --2- 172.21.15.81:0/2207833804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f208c0cf170 0x7f208c15b5b0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.388+0000 7f208bfff640 1 -- 172.21.15.81:0/2207833804 >> 172.21.15.81:0/2207833804 conn(0x7f208c06a4a0 msgr2=0x7f208c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:09.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.388+0000 7f208bfff640 1 -- 172.21.15.81:0/2207833804 shutdown_connections 2023-11-08T16:37:09.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.388+0000 7f208bfff640 1 -- 172.21.15.81:0/2207833804 wait complete. 2023-11-08T16:37:09.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.390+0000 7f208bfff640 1 Processor -- start 2023-11-08T16:37:09.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.390+0000 7f208bfff640 1 -- start start 2023-11-08T16:37:09.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.391+0000 7f208bfff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f208c0ced10 0x7f208c14b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.391+0000 7f208bfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f208c0cf170 0x7f208c15d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.391+0000 7f208bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f208c15aac0 0x7f208c15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.391+0000 7f208bfff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0c4570 con 0x7f208c15aac0 2023-11-08T16:37:09.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.391+0000 7f208bfff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0c4850 con 0x7f208c0ced10 2023-11-08T16:37:09.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.391+0000 7f208bfff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208c0c4b30 con 0x7f208c0cf170 2023-11-08T16:37:09.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.392+0000 7f208b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f208c15aac0 0x7f208c15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:09.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.392+0000 7f208b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f208c15aac0 0x7f208c15d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37542/0 (socket says 172.21.15.81:37542) 2023-11-08T16:37:09.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.392+0000 7f208b7fe640 1 -- 172.21.15.81:0/2772130616 learned_addr learned my addr 172.21.15.81:0/2772130616 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:09.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.392+0000 7f208a7fc640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f208c0cf170 0x7f208c15d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:09.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.392+0000 7f208affd640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f208c0ced10 0x7f208c14b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:09.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.392+0000 7f208b7fe640 1 -- 172.21.15.81:0/2772130616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f208c0cf170 msgr2=0x7f208c15d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:09.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.392+0000 7f208b7fe640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f208c0cf170 0x7f208c15d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.392+0000 7f208b7fe640 1 -- 172.21.15.81:0/2772130616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f208c0ced10 msgr2=0x7f208c14b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:09.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.392+0000 7f208b7fe640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f208c0ced10 0x7f208c14b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.392+0000 7f208b7fe640 1 -- 172.21.15.81:0/2772130616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2084009b30 con 0x7f208c15aac0 2023-11-08T16:37:09.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.392+0000 7f208b7fe640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f208c15aac0 0x7f208c15d5b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2084009e50 tx=0x7f2084004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:09.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.393+0000 7f2073fff640 1 -- 172.21.15.81:0/2772130616 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2084009510 con 0x7f208c15aac0 2023-11-08T16:37:09.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.393+0000 7f2073fff640 1 -- 172.21.15.81:0/2772130616 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f208403e450 con 0x7f208c15aac0 2023-11-08T16:37:09.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.393+0000 7f2073fff640 1 -- 172.21.15.81:0/2772130616 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f208403e7e0 con 0x7f208c15aac0 2023-11-08T16:37:09.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.393+0000 7f208bfff640 1 -- 172.21.15.81:0/2772130616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208c0c4e10 con 0x7f208c15aac0 2023-11-08T16:37:09.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.393+0000 7f208bfff640 1 -- 172.21.15.81:0/2772130616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f208c14b8e0 con 0x7f208c15aac0 2023-11-08T16:37:09.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.394+0000 7f208bfff640 1 -- 172.21.15.81:0/2772130616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208c067800 con 0x7f208c15aac0 2023-11-08T16:37:09.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.395+0000 7f2073fff640 1 -- 172.21.15.81:0/2772130616 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f208403e940 con 0x7f208c15aac0 2023-11-08T16:37:09.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.396+0000 7f2073fff640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f207806ae70 0x7f207806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.396+0000 7f2073fff640 1 -- 172.21.15.81:0/2772130616 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f20840ac900 con 0x7f208c15aac0 2023-11-08T16:37:09.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.396+0000 7f208affd640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f207806ae70 0x7f207806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:09.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.397+0000 7f208affd640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f207806ae70 0x7f207806d330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f207c006fd0 tx=0x7f207c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:09.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.400+0000 7f2073fff640 1 -- 172.21.15.81:0/2772130616 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f20840777d0 con 0x7f208c15aac0 2023-11-08T16:37:09.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.626+0000 7f208bfff640 1 -- 172.21.15.81:0/2772130616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f208c0cb880 con 0x7f208c15aac0 2023-11-08T16:37:09.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.627+0000 7f2073fff640 1 -- 172.21.15.81:0/2772130616 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4913 (secure 0 0 0) 0x7f208407adf0 con 0x7f208c15aac0 2023-11-08T16:37:09.630 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:09.630 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3027678987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3027678987},{"type":"v1","addr":"172.21.15.93:6833","nonce":3027678987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/3140373039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3140373039},{"type":"v1","addr":"172.21.15.81:6836","nonce":3140373039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3145959397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3145959397},{"type":"v1","addr":"172.21.15.93:6835","nonce":3145959397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:55.206262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/552116415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":552116415},{"type":"v1","addr":"172.21.15.81:6837","nonce":552116415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:37:09.630 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 54 2023-11-08T16:37:09.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.629+0000 7f208bfff640 1 -- 172.21.15.81:0/2772130616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f207806ae70 msgr2=0x7f207806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:09.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.629+0000 7f208bfff640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f207806ae70 0x7f207806d330 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f207c006fd0 tx=0x7f207c008040 comp rx=0 tx=0).stop 2023-11-08T16:37:09.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.629+0000 7f208bfff640 1 -- 172.21.15.81:0/2772130616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f208c15aac0 msgr2=0x7f208c15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:09.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.629+0000 7f208bfff640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f208c15aac0 0x7f208c15d5b0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2084009e50 tx=0x7f2084004a40 comp rx=0 tx=0).stop 2023-11-08T16:37:09.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.630+0000 7f208bfff640 1 -- 172.21.15.81:0/2772130616 shutdown_connections 2023-11-08T16:37:09.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.630+0000 7f208bfff640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f207806ae70 0x7f207806d330 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.630+0000 7f208bfff640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f208c15aac0 0x7f208c15d5b0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.630+0000 7f208bfff640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f208c0cf170 0x7f208c15d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.630+0000 7f208bfff640 1 --2- 172.21.15.81:0/2772130616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f208c0ced10 0x7f208c14b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.630+0000 7f208bfff640 1 -- 172.21.15.81:0/2772130616 >> 172.21.15.81:0/2772130616 conn(0x7f208c06a4a0 msgr2=0x7f208c0adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:09.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.630+0000 7f208bfff640 1 -- 172.21.15.81:0/2772130616 shutdown_connections 2023-11-08T16:37:09.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.630+0000 7f208bfff640 1 -- 172.21.15.81:0/2772130616 wait complete. 2023-11-08T16:37:09.649 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:09.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.857+0000 7f8eb9044640 1 Processor -- start 2023-11-08T16:37:09.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.857+0000 7f8eb9044640 1 -- start start 2023-11-08T16:37:09.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb9044640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8eb415d2b0 0x7f8eb4161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb9044640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8eb40c9060 0x7f8eb40cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb9044640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8eb40c94e0 0x7f8eb40ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb9044640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb415d690 con 0x7f8eb40c9060 2023-11-08T16:37:09.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb9044640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb4161c50 con 0x7f8eb40c94e0 2023-11-08T16:37:09.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb9044640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb406b980 con 0x7f8eb415d2b0 2023-11-08T16:37:09.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb259c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8eb40c9060 0x7f8eb40cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:09.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb259c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8eb40c9060 0x7f8eb40cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37548/0 (socket says 172.21.15.81:37548) 2023-11-08T16:37:09.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb259c640 1 -- 172.21.15.81:0/951586884 learned_addr learned my addr 172.21.15.81:0/951586884 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:09.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb2d9d640 1 --2- 172.21.15.81:0/951586884 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8eb415d2b0 0x7f8eb4161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:09.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb359e640 1 --2- 172.21.15.81:0/951586884 >> v2:172.21.15.93:3300/0 conn(0x7f8eb40c94e0 0x7f8eb40ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:09.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.859+0000 7f8eb259c640 1 -- 172.21.15.81:0/951586884 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8eb415d2b0 msgr2=0x7f8eb4161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:09.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.860+0000 7f8eb259c640 1 --2- 172.21.15.81:0/951586884 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8eb415d2b0 0x7f8eb4161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.860+0000 7f8eb259c640 1 -- 172.21.15.81:0/951586884 >> v2:172.21.15.93:3300/0 conn(0x7f8eb40c94e0 msgr2=0x7f8eb40ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:09.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.860+0000 7f8eb259c640 1 --2- 172.21.15.81:0/951586884 >> v2:172.21.15.93:3300/0 conn(0x7f8eb40c94e0 0x7f8eb40ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.860+0000 7f8eb259c640 1 -- 172.21.15.81:0/951586884 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb406bc60 con 0x7f8eb40c9060 2023-11-08T16:37:09.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.860+0000 7f8eb359e640 1 --2- 172.21.15.81:0/951586884 >> v2:172.21.15.93:3300/0 conn(0x7f8eb40c94e0 0x7f8eb40ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:09.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.861+0000 7f8eb259c640 1 --2- 172.21.15.81:0/951586884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8eb40c9060 0x7f8eb40cc7f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40b15c0 tx=0x7f8ea802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1471623d6e0c34e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:09.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.861+0000 7f8eb1d9b640 1 -- 172.21.15.81:0/951586884 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8031860 con 0x7f8eb40c9060 2023-11-08T16:37:09.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.861+0000 7f8eb1d9b640 1 -- 172.21.15.81:0/951586884 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea80319c0 con 0x7f8eb40c9060 2023-11-08T16:37:09.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.861+0000 7f8eb1d9b640 1 -- 172.21.15.81:0/951586884 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8031d50 con 0x7f8eb40c9060 2023-11-08T16:37:09.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.862+0000 7f8eb9044640 1 -- 172.21.15.81:0/951586884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8eb40c9060 msgr2=0x7f8eb40cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:09.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.862+0000 7f8eb9044640 1 --2- 172.21.15.81:0/951586884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8eb40c9060 0x7f8eb40cc7f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8eb40b15c0 tx=0x7f8ea802b900 comp rx=0 tx=0).stop 2023-11-08T16:37:09.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.862+0000 7f8eb9044640 1 -- 172.21.15.81:0/951586884 shutdown_connections 2023-11-08T16:37:09.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.862+0000 7f8eb9044640 1 --2- 172.21.15.81:0/951586884 >> v2:172.21.15.93:3300/0 conn(0x7f8eb40c94e0 0x7f8eb40ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.862+0000 7f8eb9044640 1 --2- 172.21.15.81:0/951586884 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8eb40c9060 0x7f8eb40cc7f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.862+0000 7f8eb9044640 1 --2- 172.21.15.81:0/951586884 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8eb415d2b0 0x7f8eb4161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.862+0000 7f8eb9044640 1 -- 172.21.15.81:0/951586884 >> 172.21.15.81:0/951586884 conn(0x7f8eb40cb490 msgr2=0x7f8eb40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:09.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.862+0000 7f8eb9044640 1 -- 172.21.15.81:0/951586884 shutdown_connections 2023-11-08T16:37:09.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.862+0000 7f8eb9044640 1 -- 172.21.15.81:0/951586884 wait complete. 2023-11-08T16:37:09.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.864+0000 7f8eb9044640 1 Processor -- start 2023-11-08T16:37:09.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.864+0000 7f8eb9044640 1 -- start start 2023-11-08T16:37:09.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.865+0000 7f8eb9044640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8eb40c9060 0x7f8eb4158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.865+0000 7f8eb9044640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8eb40c94e0 0x7f8eb40c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.865+0000 7f8eb9044640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8eb415d2b0 0x7f8eb40c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb9044640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb40b3c80 con 0x7f8eb40c9060 2023-11-08T16:37:09.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb9044640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb4158780 con 0x7f8eb40c94e0 2023-11-08T16:37:09.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb9044640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb40b3070 con 0x7f8eb415d2b0 2023-11-08T16:37:09.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8eb40c9060 0x7f8eb4158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:09.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb2d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8eb40c9060 0x7f8eb4158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37556/0 (socket says 172.21.15.81:37556) 2023-11-08T16:37:09.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb2d9d640 1 -- 172.21.15.81:0/3371929581 learned_addr learned my addr 172.21.15.81:0/3371929581 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:09.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb2d9d640 1 -- 172.21.15.81:0/3371929581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8eb415d2b0 msgr2=0x7f8eb40c81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb2d9d640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8eb415d2b0 0x7f8eb40c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb2d9d640 1 -- 172.21.15.81:0/3371929581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8eb40c94e0 msgr2=0x7f8eb40c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb259c640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8eb40c94e0 0x7f8eb40c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:09.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb2d9d640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8eb40c94e0 0x7f8eb40c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:09.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb2d9d640 1 -- 172.21.15.81:0/3371929581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea80094d0 con 0x7f8eb40c9060 2023-11-08T16:37:09.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.866+0000 7f8eb2d9d640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8eb40c9060 0x7f8eb4158240 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400d830 tx=0x7f8ea400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:09.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.867+0000 7f8e9b7fe640 1 -- 172.21.15.81:0/3371929581 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea400b6c0 con 0x7f8eb40c9060 2023-11-08T16:37:09.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.867+0000 7f8e9b7fe640 1 -- 172.21.15.81:0/3371929581 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea400b820 con 0x7f8eb40c9060 2023-11-08T16:37:09.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.867+0000 7f8e9b7fe640 1 -- 172.21.15.81:0/3371929581 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea400bbb0 con 0x7f8eb40c9060 2023-11-08T16:37:09.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.867+0000 7f8eb9044640 1 -- 172.21.15.81:0/3371929581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb40b3350 con 0x7f8eb40c9060 2023-11-08T16:37:09.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.867+0000 7f8eb9044640 1 -- 172.21.15.81:0/3371929581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb40b3910 con 0x7f8eb40c9060 2023-11-08T16:37:09.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.868+0000 7f8eb9044640 1 -- 172.21.15.81:0/3371929581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb404ece0 con 0x7f8eb40c9060 2023-11-08T16:37:09.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.869+0000 7f8e9b7fe640 1 -- 172.21.15.81:0/3371929581 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ea401e460 con 0x7f8eb40c9060 2023-11-08T16:37:09.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.870+0000 7f8e9b7fe640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ea006ae70 0x7f8ea006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:09.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.870+0000 7f8e9b7fe640 1 -- 172.21.15.81:0/3371929581 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f8ea408a1b0 con 0x7f8eb40c9060 2023-11-08T16:37:09.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.870+0000 7f8eb259c640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ea006ae70 0x7f8ea006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:09.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.871+0000 7f8eb259c640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ea006ae70 0x7f8ea006d330 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8036000 tx=0x7f8ea803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:09.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:09.876+0000 7f8e9b7fe640 1 -- 172.21.15.81:0/3371929581 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ea4054e40 con 0x7f8eb40c9060 2023-11-08T16:37:10.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.067+0000 7f8eb9044640 1 -- 172.21.15.81:0/3371929581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8eb406bf10 con 0x7f8eb40c9060 2023-11-08T16:37:10.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.069+0000 7f8e9b7fe640 1 -- 172.21.15.81:0/3371929581 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18768 (secure 0 0 0) 0x7f8ea4058460 con 0x7f8eb40c9060 2023-11-08T16:37:10.072 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:10.073 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":68,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:34:53.906443+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:32:41.268027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T16:32:42.449607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:32:53.335342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.e","mappings":[{"from":0,"to":6}]},{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"9.11","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/321347143":"2023-11-09T16:34:45.940255+0000","172.21.15.81:6837/1446454901":"2023-11-09T16:34:43.853227+0000","172.21.15.81:6837/494395478":"2023-11-09T16:34:53.172859+0000","172.21.15.81:6835/494395478":"2023-11-09T16:34:53.172859+0000","172.21.15.81:6835/321347143":"2023-11-09T16:34:45.940255+0000","172.21.15.81:6835/1446454901":"2023-11-09T16:34:43.853227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:10.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.071+0000 7f8eb9044640 1 -- 172.21.15.81:0/3371929581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ea006ae70 msgr2=0x7f8ea006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.071+0000 7f8eb9044640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ea006ae70 0x7f8ea006d330 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8036000 tx=0x7f8ea803c040 comp rx=0 tx=0).stop 2023-11-08T16:37:10.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.071+0000 7f8eb9044640 1 -- 172.21.15.81:0/3371929581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8eb40c9060 msgr2=0x7f8eb4158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.071+0000 7f8eb9044640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8eb40c9060 0x7f8eb4158240 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8ea400d830 tx=0x7f8ea400dcf0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.071+0000 7f8eb9044640 1 -- 172.21.15.81:0/3371929581 shutdown_connections 2023-11-08T16:37:10.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.071+0000 7f8eb9044640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ea006ae70 0x7f8ea006d330 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.071+0000 7f8eb9044640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8eb415d2b0 0x7f8eb40c81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.071+0000 7f8eb9044640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8eb40c94e0 0x7f8eb40c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.071+0000 7f8eb9044640 1 --2- 172.21.15.81:0/3371929581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8eb40c9060 0x7f8eb4158240 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.071+0000 7f8eb9044640 1 -- 172.21.15.81:0/3371929581 >> 172.21.15.81:0/3371929581 conn(0x7f8eb40cb490 msgr2=0x7f8eb40afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:10.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.072+0000 7f8eb9044640 1 -- 172.21.15.81:0/3371929581 shutdown_connections 2023-11-08T16:37:10.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.072+0000 7f8eb9044640 1 -- 172.21.15.81:0/3371929581 wait complete. 2023-11-08T16:37:10.095 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T16:37:10.096 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:37:10.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.298+0000 7fa3e9bb7640 1 Processor -- start 2023-11-08T16:37:10.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.298+0000 7fa3e9bb7640 1 -- start start 2023-11-08T16:37:10.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.300+0000 7fa3e9bb7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa3e40ade50 0x7fa3e40ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:10.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.300+0000 7fa3e9bb7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa3e415ea60 0x7fa3e415eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:10.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.300+0000 7fa3e9bb7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3e406b630 con 0x7fa3e415a680 2023-11-08T16:37:10.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.300+0000 7fa3e9bb7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3e40b1860 con 0x7fa3e415ea60 2023-11-08T16:37:10.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.300+0000 7fa3e9bb7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e40ca9c0 con 0x7fa3e40ade50 2023-11-08T16:37:10.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.300+0000 7fa3e3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa3e415ea60 0x7fa3e415eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:10.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.300+0000 7fa3e3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa3e415ea60 0x7fa3e415eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37562/0 (socket says 172.21.15.81:37562) 2023-11-08T16:37:10.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.300+0000 7fa3e3fff640 1 -- 172.21.15.81:0/1765772103 learned_addr learned my addr 172.21.15.81:0/1765772103 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:10.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.301+0000 7fa3e37fe640 1 --2- 172.21.15.81:0/1765772103 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa3e40ade50 0x7fa3e40ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:10.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.301+0000 7fa3e3fff640 1 -- 172.21.15.81:0/1765772103 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa3e40ade50 msgr2=0x7fa3e40ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:10.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.301+0000 7fa3e3fff640 1 --2- 172.21.15.81:0/1765772103 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa3e40ade50 0x7fa3e40ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.301+0000 7fa3e3fff640 1 -- 172.21.15.81:0/1765772103 >> v1:172.21.15.93:6789/0 conn(0x7fa3e415a680 legacy=0x7fa3e415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.301+0000 7fa3e3fff640 1 -- 172.21.15.81:0/1765772103 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e40cabb0 con 0x7fa3e415ea60 2023-11-08T16:37:10.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.302+0000 7fa3e3fff640 1 --2- 172.21.15.81:0/1765772103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3e415ea60 0x7fa3e415eef0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009e50 tx=0x7fa3dc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4469ca95e971dab server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:10.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.302+0000 7fa3e2ffd640 1 -- 172.21.15.81:0/1765772103 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc02cb00 con 0x7fa3e415ea60 2023-11-08T16:37:10.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.302+0000 7fa3e2ffd640 1 -- 172.21.15.81:0/1765772103 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc02cc60 con 0x7fa3e415ea60 2023-11-08T16:37:10.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.302+0000 7fa3e2ffd640 1 -- 172.21.15.81:0/1765772103 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc0343f0 con 0x7fa3e415ea60 2023-11-08T16:37:10.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.303+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/1765772103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3e415ea60 msgr2=0x7fa3e415eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.303+0000 7fa3e9bb7640 1 --2- 172.21.15.81:0/1765772103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3e415ea60 0x7fa3e415eef0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009e50 tx=0x7fa3dc02bf60 comp rx=0 tx=0).stop 2023-11-08T16:37:10.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.303+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/1765772103 shutdown_connections 2023-11-08T16:37:10.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.303+0000 7fa3e9bb7640 1 --2- 172.21.15.81:0/1765772103 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3e415ea60 0x7fa3e415eef0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.303+0000 7fa3e9bb7640 1 --2- 172.21.15.81:0/1765772103 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa3e40ade50 0x7fa3e40ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.303+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/1765772103 >> 172.21.15.81:0/1765772103 conn(0x7fa3e40cb490 msgr2=0x7fa3e40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:10.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.303+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/1765772103 shutdown_connections 2023-11-08T16:37:10.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.303+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/1765772103 wait complete. 2023-11-08T16:37:10.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.304+0000 7fa3e9bb7640 1 Processor -- start 2023-11-08T16:37:10.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.305+0000 7fa3e9bb7640 1 -- start start 2023-11-08T16:37:10.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.305+0000 7fa3e9bb7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3e40ade50 0x7fa3e415aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:10.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e9bb7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa3e415a680 0x7fa3e4157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:10.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e9bb7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3e415ea60 0x7fa3e41f14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:10.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e9bb7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e415b040 con 0x7fa3e415ea60 2023-11-08T16:37:10.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e9bb7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e415bd60 con 0x7fa3e40ade50 2023-11-08T16:37:10.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e9bb7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3e41583f0 con 0x7fa3e415a680 2023-11-08T16:37:10.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3e415ea60 0x7fa3e41f14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:10.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3e415ea60 0x7fa3e41f14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37574/0 (socket says 172.21.15.81:37574) 2023-11-08T16:37:10.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e3fff640 1 -- 172.21.15.81:0/69341798 learned_addr learned my addr 172.21.15.81:0/69341798 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:10.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e3fff640 1 -- 172.21.15.81:0/69341798 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa3e415a680 msgr2=0x7fa3e4157eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e3fff640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa3e415a680 0x7fa3e4157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e3fff640 1 -- 172.21.15.81:0/69341798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3e40ade50 msgr2=0x7fa3e415aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e3fff640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3e40ade50 0x7fa3e415aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.306+0000 7fa3e3fff640 1 -- 172.21.15.81:0/69341798 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc009b30 con 0x7fa3e415ea60 2023-11-08T16:37:10.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.307+0000 7fa3e3fff640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3e415ea60 0x7fa3e41f14f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc036040 tx=0x7fa3dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:10.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.307+0000 7fa3e0ff9640 1 -- 172.21.15.81:0/69341798 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc040670 con 0x7fa3e415ea60 2023-11-08T16:37:10.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.307+0000 7fa3e0ff9640 1 -- 172.21.15.81:0/69341798 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc0407f0 con 0x7fa3e415ea60 2023-11-08T16:37:10.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.307+0000 7fa3e0ff9640 1 -- 172.21.15.81:0/69341798 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc040ba0 con 0x7fa3e415ea60 2023-11-08T16:37:10.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.307+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/69341798 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3e41f1a30 con 0x7fa3e415ea60 2023-11-08T16:37:10.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.307+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/69341798 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3e415b430 con 0x7fa3e415ea60 2023-11-08T16:37:10.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.308+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/69341798 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3e404ece0 con 0x7fa3e415ea60 2023-11-08T16:37:10.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.309+0000 7fa3e0ff9640 1 -- 172.21.15.81:0/69341798 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa3dc040d00 con 0x7fa3e415ea60 2023-11-08T16:37:10.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.310+0000 7fa3e0ff9640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa3c406aea0 0x7fa3c406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:10.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.310+0000 7fa3e0ff9640 1 -- 172.21.15.81:0/69341798 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fa3dc0ac6b0 con 0x7fa3e415ea60 2023-11-08T16:37:10.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.310+0000 7fa3e37fe640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa3c406aea0 0x7fa3c406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:10.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.311+0000 7fa3e37fe640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa3c406aea0 0x7fa3c406d360 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40098f0 tx=0x7fa3d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:10.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.315+0000 7fa3e0ff9640 1 -- 172.21.15.81:0/69341798 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa3dc077460 con 0x7fa3e415ea60 2023-11-08T16:37:10.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.538+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/69341798 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3e415bed0 con 0x7fa3e415ea60 2023-11-08T16:37:10.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.539+0000 7fa3e0ff9640 1 -- 172.21.15.81:0/69341798 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fa3dc009510 con 0x7fa3e415ea60 2023-11-08T16:37:10.542 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:10.542 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3027678987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3027678987},{"type":"v1","addr":"172.21.15.93:6833","nonce":3027678987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/3140373039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3140373039},{"type":"v1","addr":"172.21.15.81:6836","nonce":3140373039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3145959397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3145959397},{"type":"v1","addr":"172.21.15.93:6835","nonce":3145959397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:55.206262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/552116415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":552116415},{"type":"v1","addr":"172.21.15.81:6837","nonce":552116415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:37:10.542 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 54 2023-11-08T16:37:10.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.541+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/69341798 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa3c406aea0 msgr2=0x7fa3c406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.541+0000 7fa3e9bb7640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa3c406aea0 0x7fa3c406d360 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa3d40098f0 tx=0x7fa3d4008040 comp rx=0 tx=0).stop 2023-11-08T16:37:10.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.541+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/69341798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3e415ea60 msgr2=0x7fa3e41f14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.541+0000 7fa3e9bb7640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3e415ea60 0x7fa3e41f14f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc036040 tx=0x7fa3dc004a40 comp rx=0 tx=0).stop 2023-11-08T16:37:10.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.542+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/69341798 shutdown_connections 2023-11-08T16:37:10.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.542+0000 7fa3e9bb7640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa3c406aea0 0x7fa3c406d360 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.542+0000 7fa3e9bb7640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa3e415ea60 0x7fa3e41f14f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.542+0000 7fa3e9bb7640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa3e415a680 0x7fa3e4157eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.542+0000 7fa3e9bb7640 1 --2- 172.21.15.81:0/69341798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa3e40ade50 0x7fa3e415aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.542+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/69341798 >> 172.21.15.81:0/69341798 conn(0x7fa3e40cb490 msgr2=0x7fa3e4158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:10.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.542+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/69341798 shutdown_connections 2023-11-08T16:37:10.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.542+0000 7fa3e9bb7640 1 -- 172.21.15.81:0/69341798 wait complete. 2023-11-08T16:37:10.563 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:37:10.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.767+0000 7fde97ae1640 1 Processor -- start 2023-11-08T16:37:10.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.767+0000 7fde97ae1640 1 -- start start 2023-11-08T16:37:10.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.769+0000 7fde97ae1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fde900b0070 0x7fde9006d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:10.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.769+0000 7fde97ae1640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde900cbee0 con 0x7fde900cd5c0 2023-11-08T16:37:10.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.769+0000 7fde97ae1640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde900b0450 con 0x7fde900d06e0 2023-11-08T16:37:10.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.769+0000 7fde97ae1640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde900cb2d0 con 0x7fde900b0070 2023-11-08T16:37:10.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.769+0000 7fde9587d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fde900b0070 0x7fde9006d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:10.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.769+0000 7fde9587d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fde900b0070 0x7fde9006d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37584/0 (socket says 172.21.15.81:37584) 2023-11-08T16:37:10.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.769+0000 7fde9587d640 1 -- 172.21.15.81:0/3068084190 learned_addr learned my addr 172.21.15.81:0/3068084190 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:10.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.770+0000 7fde9587d640 1 -- 172.21.15.81:0/3068084190 >> v1:172.21.15.93:6789/0 conn(0x7fde900d06e0 legacy=0x7fde900cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:10.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.770+0000 7fde9587d640 1 -- 172.21.15.81:0/3068084190 >> v1:172.21.15.81:6789/0 conn(0x7fde900cd5c0 legacy=0x7fde900cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.770+0000 7fde9587d640 1 -- 172.21.15.81:0/3068084190 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde900cb5b0 con 0x7fde900b0070 2023-11-08T16:37:10.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.771+0000 7fde9587d640 1 --2- 172.21.15.81:0/3068084190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde900b0070 0x7fde9006d6a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fde80009ea0 tx=0x7fde8002bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae68e460173b1f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:10.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.771+0000 7fde9487b640 1 -- 172.21.15.81:0/3068084190 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8002cb50 con 0x7fde900b0070 2023-11-08T16:37:10.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.771+0000 7fde9487b640 1 -- 172.21.15.81:0/3068084190 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8002ccb0 con 0x7fde900b0070 2023-11-08T16:37:10.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.771+0000 7fde9487b640 1 -- 172.21.15.81:0/3068084190 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde80034610 con 0x7fde900b0070 2023-11-08T16:37:10.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.772+0000 7fde97ae1640 1 -- 172.21.15.81:0/3068084190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde900b0070 msgr2=0x7fde9006d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.772+0000 7fde97ae1640 1 --2- 172.21.15.81:0/3068084190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde900b0070 0x7fde9006d6a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fde80009ea0 tx=0x7fde8002bfb0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.772+0000 7fde97ae1640 1 -- 172.21.15.81:0/3068084190 shutdown_connections 2023-11-08T16:37:10.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.772+0000 7fde97ae1640 1 --2- 172.21.15.81:0/3068084190 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde900b0070 0x7fde9006d6a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.772+0000 7fde97ae1640 1 -- 172.21.15.81:0/3068084190 >> 172.21.15.81:0/3068084190 conn(0x7fde900d0be0 msgr2=0x7fde900d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:10.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.772+0000 7fde97ae1640 1 -- 172.21.15.81:0/3068084190 shutdown_connections 2023-11-08T16:37:10.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.772+0000 7fde97ae1640 1 -- 172.21.15.81:0/3068084190 wait complete. 2023-11-08T16:37:10.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.774+0000 7fde97ae1640 1 Processor -- start 2023-11-08T16:37:10.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.774+0000 7fde97ae1640 1 -- start start 2023-11-08T16:37:10.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde97ae1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde900b0070 0x7fde90152570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:10.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde97ae1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde900cd5c0 0x7fde900c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:10.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde97ae1640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde900d06e0 0x7fde9015be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:10.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde97ae1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900c75c0 con 0x7fde900b0070 2023-11-08T16:37:10.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde97ae1640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde900c7f70 con 0x7fde900d06e0 2023-11-08T16:37:10.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde97ae1640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde900b3900 con 0x7fde900cd5c0 2023-11-08T16:37:10.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde9587d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde900b0070 0x7fde90152570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:10.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde9587d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde900b0070 0x7fde90152570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37590/0 (socket says 172.21.15.81:37590) 2023-11-08T16:37:10.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde9587d640 1 -- 172.21.15.81:0/3424761831 learned_addr learned my addr 172.21.15.81:0/3424761831 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:10.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde9587d640 1 -- 172.21.15.81:0/3424761831 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde900cd5c0 msgr2=0x7fde900c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde9587d640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde900cd5c0 0x7fde900c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde9587d640 1 -- 172.21.15.81:0/3424761831 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde900d06e0 msgr2=0x7fde9015be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde9587d640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde900d06e0 0x7fde9015be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.775+0000 7fde9587d640 1 -- 172.21.15.81:0/3424761831 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde80009b30 con 0x7fde900b0070 2023-11-08T16:37:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.776+0000 7fde9587d640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde900b0070 0x7fde90152570 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fde80009ea0 tx=0x7fde80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.776+0000 7fde8e7fc640 1 -- 172.21.15.81:0/3424761831 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8003e450 con 0x7fde900b0070 2023-11-08T16:37:10.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.776+0000 7fde8e7fc640 1 -- 172.21.15.81:0/3424761831 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde8003e5f0 con 0x7fde900b0070 2023-11-08T16:37:10.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.776+0000 7fde8e7fc640 1 -- 172.21.15.81:0/3424761831 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde8003e9a0 con 0x7fde900b0070 2023-11-08T16:37:10.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.776+0000 7fde97ae1640 1 -- 172.21.15.81:0/3424761831 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde900c7030 con 0x7fde900b0070 2023-11-08T16:37:10.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.776+0000 7fde97ae1640 1 -- 172.21.15.81:0/3424761831 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde900b3ba0 con 0x7fde900b0070 2023-11-08T16:37:10.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.778+0000 7fde97ae1640 1 -- 172.21.15.81:0/3424761831 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde900b3760 con 0x7fde900b0070 2023-11-08T16:37:10.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.778+0000 7fde8e7fc640 1 -- 172.21.15.81:0/3424761831 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fde8003eb00 con 0x7fde900b0070 2023-11-08T16:37:10.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.779+0000 7fde8e7fc640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fde7c06ae70 0x7fde7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:10.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.779+0000 7fde8e7fc640 1 -- 172.21.15.81:0/3424761831 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fde800ac830 con 0x7fde900b0070 2023-11-08T16:37:10.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.779+0000 7fde9507c640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fde7c06ae70 0x7fde7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:10.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.780+0000 7fde9507c640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fde7c06ae70 0x7fde7c06d330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fde84000f60 tx=0x7fde840079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:10.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:10.784+0000 7fde8e7fc640 1 -- 172.21.15.81:0/3424761831 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fde900b3760 con 0x7fde900b0070 2023-11-08T16:37:11.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.013+0000 7fde97ae1640 1 -- 172.21.15.81:0/3424761831 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde900b3760 con 0x7fde900b0070 2023-11-08T16:37:11.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.014+0000 7fde8e7fc640 1 -- 172.21.15.81:0/3424761831 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4913 (secure 0 0 0) 0x7fde900b3760 con 0x7fde900b0070 2023-11-08T16:37:11.017 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:11.017 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":54,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3027678987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3027678987},{"type":"v1","addr":"172.21.15.93:6833","nonce":3027678987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/3140373039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3140373039},{"type":"v1","addr":"172.21.15.81:6836","nonce":3140373039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3145959397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3145959397},{"type":"v1","addr":"172.21.15.93:6835","nonce":3145959397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:34:44.993932+0000","modified":"2023-11-08T16:34:55.206262+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":65,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"c","rank":0,"incarnation":51,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/552116415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":552116415},{"type":"v1","addr":"172.21.15.81:6837","nonce":552116415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8,9],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-08T16:37:11.018 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 54 2023-11-08T16:37:11.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 -- 172.21.15.81:0/3424761831 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fde7c06ae70 msgr2=0x7fde7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fde7c06ae70 0x7fde7c06d330 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fde84000f60 tx=0x7fde840079b0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 -- 172.21.15.81:0/3424761831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde900b0070 msgr2=0x7fde90152570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde900b0070 0x7fde90152570 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fde80009ea0 tx=0x7fde80004a40 comp rx=0 tx=0).stop 2023-11-08T16:37:11.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 -- 172.21.15.81:0/3424761831 shutdown_connections 2023-11-08T16:37:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fde7c06ae70 0x7fde7c06d330 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde900d06e0 0x7fde9015be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde900cd5c0 0x7fde900c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 --2- 172.21.15.81:0/3424761831 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde900b0070 0x7fde90152570 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 -- 172.21.15.81:0/3424761831 >> 172.21.15.81:0/3424761831 conn(0x7fde900d0be0 msgr2=0x7fde900add00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:11.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 -- 172.21.15.81:0/3424761831 shutdown_connections 2023-11-08T16:37:11.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.017+0000 7fde97ae1640 1 -- 172.21.15.81:0/3424761831 wait complete. 2023-11-08T16:37:11.041 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:11.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.251+0000 7f4378a2d640 1 Processor -- start 2023-11-08T16:37:11.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.251+0000 7f4378a2d640 1 -- start start 2023-11-08T16:37:11.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.253+0000 7f4378a2d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f437415d2b0 0x7f4374161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:11.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.253+0000 7f4378a2d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740c9060 0x7f43740cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:11.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.253+0000 7f4378a2d640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4374161c50 con 0x7f43740c94e0 2023-11-08T16:37:11.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.253+0000 7f4378a2d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f437415d690 con 0x7f437415d2b0 2023-11-08T16:37:11.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.253+0000 7f4378a2d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437406b980 con 0x7f43740c9060 2023-11-08T16:37:11.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.253+0000 7f437259c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f437415d2b0 0x7f4374161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:11.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.253+0000 7f437259c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f437415d2b0 0x7f4374161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35892/0 (socket says 172.21.15.81:35892) 2023-11-08T16:37:11.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.253+0000 7f437259c640 1 -- 172.21.15.81:0/903808799 learned_addr learned my addr 172.21.15.81:0/903808799 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:11.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.253+0000 7f436bfff640 1 --2- 172.21.15.81:0/903808799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740c9060 0x7f43740cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:11.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.254+0000 7f437259c640 1 -- 172.21.15.81:0/903808799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740c9060 msgr2=0x7f43740cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:11.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.254+0000 7f437259c640 1 --2- 172.21.15.81:0/903808799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740c9060 0x7f43740cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.254+0000 7f437259c640 1 -- 172.21.15.81:0/903808799 >> v1:172.21.15.93:6789/0 conn(0x7f43740c94e0 legacy=0x7f437406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.254+0000 7f437259c640 1 -- 172.21.15.81:0/903808799 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437406bc60 con 0x7f437415d2b0 2023-11-08T16:37:11.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.255+0000 7f437259c640 1 --2- 172.21.15.81:0/903808799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f437415d2b0 0x7f4374161710 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4364009870 tx=0x7f436402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf15ed2174b18161 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:11.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.255+0000 7f4371d9b640 1 -- 172.21.15.81:0/903808799 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364030840 con 0x7f437415d2b0 2023-11-08T16:37:11.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.255+0000 7f4371d9b640 1 -- 172.21.15.81:0/903808799 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43640309a0 con 0x7f437415d2b0 2023-11-08T16:37:11.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.255+0000 7f4371d9b640 1 -- 172.21.15.81:0/903808799 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364030d30 con 0x7f437415d2b0 2023-11-08T16:37:11.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.256+0000 7f4378a2d640 1 -- 172.21.15.81:0/903808799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f437415d2b0 msgr2=0x7f4374161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.256+0000 7f4378a2d640 1 --2- 172.21.15.81:0/903808799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f437415d2b0 0x7f4374161710 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f4364009870 tx=0x7f436402b980 comp rx=0 tx=0).stop 2023-11-08T16:37:11.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.256+0000 7f4378a2d640 1 -- 172.21.15.81:0/903808799 shutdown_connections 2023-11-08T16:37:11.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.256+0000 7f4378a2d640 1 --2- 172.21.15.81:0/903808799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740c9060 0x7f43740cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.256+0000 7f4378a2d640 1 --2- 172.21.15.81:0/903808799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f437415d2b0 0x7f4374161710 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.256+0000 7f4378a2d640 1 -- 172.21.15.81:0/903808799 >> 172.21.15.81:0/903808799 conn(0x7f43740cb490 msgr2=0x7f43740cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:11.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.256+0000 7f4378a2d640 1 -- 172.21.15.81:0/903808799 shutdown_connections 2023-11-08T16:37:11.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.256+0000 7f4378a2d640 1 -- 172.21.15.81:0/903808799 wait complete. 2023-11-08T16:37:11.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.258+0000 7f4378a2d640 1 Processor -- start 2023-11-08T16:37:11.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.258+0000 7f4378a2d640 1 -- start start 2023-11-08T16:37:11.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.259+0000 7f4378a2d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740c9060 0x7f43740c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:11.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.259+0000 7f4378a2d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740c94e0 0x7f437406f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.259+0000 7f4378a2d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f437415d2b0 0x7f43741f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.259+0000 7f4378a2d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4374070e80 con 0x7f43740c94e0 2023-11-08T16:37:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.259+0000 7f4378a2d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4374070200 con 0x7f437415d2b0 2023-11-08T16:37:11.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.259+0000 7f4378a2d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43740704e0 con 0x7f43740c9060 2023-11-08T16:37:11.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.259+0000 7f436bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740c94e0 0x7f437406f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:11.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.259+0000 7f436bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740c94e0 0x7f437406f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35904/0 (socket says 172.21.15.81:35904) 2023-11-08T16:37:11.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.259+0000 7f436bfff640 1 -- 172.21.15.81:0/282610836 learned_addr learned my addr 172.21.15.81:0/282610836 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:11.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.260+0000 7f437259c640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740c9060 0x7f43740c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:11.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.260+0000 7f436bfff640 1 -- 172.21.15.81:0/282610836 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740c9060 msgr2=0x7f43740c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.260+0000 7f4372d9d640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f437415d2b0 0x7f43741f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:11.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.260+0000 7f436bfff640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740c9060 0x7f43740c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.260+0000 7f436bfff640 1 -- 172.21.15.81:0/282610836 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f437415d2b0 msgr2=0x7f43741f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.260+0000 7f436bfff640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f437415d2b0 0x7f43741f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.260+0000 7f436bfff640 1 -- 172.21.15.81:0/282610836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43640094d0 con 0x7f43740c94e0 2023-11-08T16:37:11.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.260+0000 7f437259c640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740c9060 0x7f43740c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:11.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.260+0000 7f4372d9d640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f437415d2b0 0x7f43741f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:11.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.260+0000 7f436bfff640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740c94e0 0x7f437406f7b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f435c00d5f0 tx=0x7f435c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:11.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.260+0000 7f436affd640 1 -- 172.21.15.81:0/282610836 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c0125f0 con 0x7f43740c94e0 2023-11-08T16:37:11.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.261+0000 7f436affd640 1 -- 172.21.15.81:0/282610836 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c012750 con 0x7f43740c94e0 2023-11-08T16:37:11.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.261+0000 7f436affd640 1 -- 172.21.15.81:0/282610836 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c012b00 con 0x7f43740c94e0 2023-11-08T16:37:11.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.261+0000 7f4378a2d640 1 -- 172.21.15.81:0/282610836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43740707c0 con 0x7f43740c94e0 2023-11-08T16:37:11.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.261+0000 7f4378a2d640 1 -- 172.21.15.81:0/282610836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43740c52b0 con 0x7f43740c94e0 2023-11-08T16:37:11.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.262+0000 7f4378a2d640 1 -- 172.21.15.81:0/282610836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f437404ece0 con 0x7f43740c94e0 2023-11-08T16:37:11.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.263+0000 7f436affd640 1 -- 172.21.15.81:0/282610836 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f435c012c60 con 0x7f43740c94e0 2023-11-08T16:37:11.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.264+0000 7f436affd640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406ae70 0x7f435406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:11.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.264+0000 7f436affd640 1 -- 172.21.15.81:0/282610836 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f435c089d40 con 0x7f43740c94e0 2023-11-08T16:37:11.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.264+0000 7f437259c640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406ae70 0x7f435406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:11.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.265+0000 7f437259c640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406ae70 0x7f435406d330 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4364036000 tx=0x7f436403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:11.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.268+0000 7f436affd640 1 -- 172.21.15.81:0/282610836 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f435c054b90 con 0x7f43740c94e0 2023-11-08T16:37:11.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.458+0000 7f4378a2d640 1 -- 172.21.15.81:0/282610836 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43740b4990 con 0x7f43740c94e0 2023-11-08T16:37:11.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.461+0000 7f436affd640 1 -- 172.21.15.81:0/282610836 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18768 (secure 0 0 0) 0x7f435c0581b0 con 0x7f43740c94e0 2023-11-08T16:37:11.464 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:11.465 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":68,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:34:53.906443+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:32:41.268027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-08T16:32:42.449607+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:32:53.335342+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.e","mappings":[{"from":0,"to":6}]},{"pgid":"8.14","mappings":[{"from":0,"to":6}]},{"pgid":"9.11","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/321347143":"2023-11-09T16:34:45.940255+0000","172.21.15.81:6837/1446454901":"2023-11-09T16:34:43.853227+0000","172.21.15.81:6837/494395478":"2023-11-09T16:34:53.172859+0000","172.21.15.81:6835/494395478":"2023-11-09T16:34:53.172859+0000","172.21.15.81:6835/321347143":"2023-11-09T16:34:45.940255+0000","172.21.15.81:6835/1446454901":"2023-11-09T16:34:43.853227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:11.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.463+0000 7f4378a2d640 1 -- 172.21.15.81:0/282610836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406ae70 msgr2=0x7f435406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.463+0000 7f4378a2d640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406ae70 0x7f435406d330 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4364036000 tx=0x7f436403b040 comp rx=0 tx=0).stop 2023-11-08T16:37:11.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.464+0000 7f4378a2d640 1 -- 172.21.15.81:0/282610836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740c94e0 msgr2=0x7f437406f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.464+0000 7f4378a2d640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740c94e0 0x7f437406f7b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f435c00d5f0 tx=0x7f435c00dab0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.464+0000 7f4378a2d640 1 -- 172.21.15.81:0/282610836 shutdown_connections 2023-11-08T16:37:11.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.464+0000 7f4378a2d640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406ae70 0x7f435406d330 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.464+0000 7f4378a2d640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f437415d2b0 0x7f43741f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.464+0000 7f4378a2d640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740c94e0 0x7f437406f7b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.464+0000 7f4378a2d640 1 --2- 172.21.15.81:0/282610836 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740c9060 0x7f43740c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.464+0000 7f4378a2d640 1 -- 172.21.15.81:0/282610836 >> 172.21.15.81:0/282610836 conn(0x7f43740cb490 msgr2=0x7f43740ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:11.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.464+0000 7f4378a2d640 1 -- 172.21.15.81:0/282610836 shutdown_connections 2023-11-08T16:37:11.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.464+0000 7f4378a2d640 1 -- 172.21.15.81:0/282610836 wait complete. 2023-11-08T16:37:11.485 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:37:11.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.697+0000 7f1d3f3ff640 1 Processor -- start 2023-11-08T16:37:11.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.697+0000 7f1d3f3ff640 1 -- start start 2023-11-08T16:37:11.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.698+0000 7f1d3f3ff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1d3815ac20 0x7f1d380cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:11.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.699+0000 7f1d3f3ff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3806cd70 con 0x7f1d380ad940 2023-11-08T16:37:11.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.699+0000 7f1d3f3ff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d380ca500 con 0x7f1d3806f7c0 2023-11-08T16:37:11.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.699+0000 7f1d3f3ff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d380ae130 con 0x7f1d3815ac20 2023-11-08T16:37:11.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.699+0000 7f1d3e3fd640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1d3815ac20 0x7f1d380cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:11.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.699+0000 7f1d3ebfe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f1d380ad940 0x7f1d3806b390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34056/0 (socket says 172.21.15.81:34056) 2023-11-08T16:37:11.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.699+0000 7f1d3ebfe640 1 -- 172.21.15.81:0/1894319749 learned_addr learned my addr 172.21.15.81:0/1894319749 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:11.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.700+0000 7f1d3d3fb640 1 -- 172.21.15.81:0/1894319749 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284074370 0 0) 0x7f1d3806cd70 con 0x7f1d380ad940 2023-11-08T16:37:11.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.700+0000 7f1d3d3fb640 1 -- 172.21.15.81:0/1894319749 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d24003520 con 0x7f1d380ad940 2023-11-08T16:37:11.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.700+0000 7f1d3e3fd640 1 -- 172.21.15.81:0/1894319749 >> v1:172.21.15.93:6789/0 conn(0x7f1d3806f7c0 legacy=0x7f1d38070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.700+0000 7f1d3e3fd640 1 -- 172.21.15.81:0/1894319749 >> v1:172.21.15.81:6789/0 conn(0x7f1d380ad940 legacy=0x7f1d3806b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.700+0000 7f1d3e3fd640 1 -- 172.21.15.81:0/1894319749 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3806d030 con 0x7f1d3815ac20 2023-11-08T16:37:11.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.701+0000 7f1d3e3fd640 1 --2- 172.21.15.81:0/1894319749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d3815ac20 0x7f1d380cbaa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1d340097d0 tx=0x7f1d3402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3bee39363a5acd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:11.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.702+0000 7f1d3d3fb640 1 -- 172.21.15.81:0/1894319749 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34031830 con 0x7f1d3815ac20 2023-11-08T16:37:11.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.702+0000 7f1d3d3fb640 1 -- 172.21.15.81:0/1894319749 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d34031990 con 0x7f1d3815ac20 2023-11-08T16:37:11.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.702+0000 7f1d3d3fb640 1 -- 172.21.15.81:0/1894319749 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d34031d20 con 0x7f1d3815ac20 2023-11-08T16:37:11.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.703+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/1894319749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d3815ac20 msgr2=0x7f1d380cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.703+0000 7f1d3f3ff640 1 --2- 172.21.15.81:0/1894319749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d3815ac20 0x7f1d380cbaa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1d340097d0 tx=0x7f1d3402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.703+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/1894319749 shutdown_connections 2023-11-08T16:37:11.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.703+0000 7f1d3f3ff640 1 --2- 172.21.15.81:0/1894319749 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d3815ac20 0x7f1d380cbaa0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.703+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/1894319749 >> 172.21.15.81:0/1894319749 conn(0x7f1d380b04d0 msgr2=0x7f1d380b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:11.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.703+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/1894319749 shutdown_connections 2023-11-08T16:37:11.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.703+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/1894319749 wait complete. 2023-11-08T16:37:11.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.705+0000 7f1d3f3ff640 1 Processor -- start 2023-11-08T16:37:11.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.705+0000 7f1d3f3ff640 1 -- start start 2023-11-08T16:37:11.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3f3ff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d3806f7c0 0x7f1d380a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:11.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3f3ff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d380ad940 0x7f1d380ab790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:11.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3f3ff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1d3815ac20 0x7f1d380b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:11.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3f3ff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380aa6f0 con 0x7f1d380ad940 2023-11-08T16:37:11.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3f3ff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380aa9d0 con 0x7f1d3806f7c0 2023-11-08T16:37:11.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3f3ff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d380aacb0 con 0x7f1d3815ac20 2023-11-08T16:37:11.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3dbfc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d380ad940 0x7f1d380ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:11.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3dbfc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d380ad940 0x7f1d380ab790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35920/0 (socket says 172.21.15.81:35920) 2023-11-08T16:37:11.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3dbfc640 1 -- 172.21.15.81:0/2183440057 learned_addr learned my addr 172.21.15.81:0/2183440057 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:11.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3e3fd640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d3806f7c0 0x7f1d380a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:11.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3dbfc640 1 -- 172.21.15.81:0/2183440057 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1d3815ac20 msgr2=0x7f1d380b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3dbfc640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1d3815ac20 0x7f1d380b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3dbfc640 1 -- 172.21.15.81:0/2183440057 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d3806f7c0 msgr2=0x7f1d380a9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3dbfc640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d3806f7c0 0x7f1d380a9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3dbfc640 1 -- 172.21.15.81:0/2183440057 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d34009400 con 0x7f1d380ad940 2023-11-08T16:37:11.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.706+0000 7f1d3e3fd640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d3806f7c0 0x7f1d380a9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:11.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.707+0000 7f1d3dbfc640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d380ad940 0x7f1d380ab790 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800c400 tx=0x7f1d2800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:11.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.707+0000 7f1d2effd640 1 -- 172.21.15.81:0/2183440057 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d280052d0 con 0x7f1d380ad940 2023-11-08T16:37:11.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.707+0000 7f1d2effd640 1 -- 172.21.15.81:0/2183440057 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d28005430 con 0x7f1d380ad940 2023-11-08T16:37:11.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.707+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/2183440057 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d380aaf90 con 0x7f1d380ad940 2023-11-08T16:37:11.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.707+0000 7f1d2effd640 1 -- 172.21.15.81:0/2183440057 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d28003830 con 0x7f1d380ad940 2023-11-08T16:37:11.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.707+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/2183440057 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d380aa3e0 con 0x7f1d380ad940 2023-11-08T16:37:11.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.708+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/2183440057 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d380ab4b0 con 0x7f1d380ad940 2023-11-08T16:37:11.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.709+0000 7f1d2effd640 1 -- 172.21.15.81:0/2183440057 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1d28003990 con 0x7f1d380ad940 2023-11-08T16:37:11.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.710+0000 7f1d2effd640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1d1c06ae70 0x7f1d1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:11.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.710+0000 7f1d2effd640 1 -- 172.21.15.81:0/2183440057 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f1d28088fa0 con 0x7f1d380ad940 2023-11-08T16:37:11.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.711+0000 7f1d3e3fd640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1d1c06ae70 0x7f1d1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:11.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.711+0000 7f1d3e3fd640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1d1c06ae70 0x7f1d1c06d330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1d34036000 tx=0x7f1d34008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:11.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.715+0000 7f1d2effd640 1 -- 172.21.15.81:0/2183440057 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1d380ab4b0 con 0x7f1d380ad940 2023-11-08T16:37:11.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.929+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/2183440057 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1d380ab4b0 con 0x7f1d380ad940 2023-11-08T16:37:11.933 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:37:11.931+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:37:11.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.934+0000 7f1d2effd640 1 -- 172.21.15.81:0/2183440057 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1d380ab4b0 con 0x7f1d380ad940 2023-11-08T16:37:11.937 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:37:11.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/2183440057 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1d1c06ae70 msgr2=0x7f1d1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1d1c06ae70 0x7f1d1c06d330 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f1d34036000 tx=0x7f1d34008030 comp rx=0 tx=0).stop 2023-11-08T16:37:11.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/2183440057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d380ad940 msgr2=0x7f1d380ab790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:11.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d380ad940 0x7f1d380ab790 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f1d2800c400 tx=0x7f1d2800c8c0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/2183440057 shutdown_connections 2023-11-08T16:37:11.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1d1c06ae70 0x7f1d1c06d330 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1d3815ac20 0x7f1d380b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1d380ad940 0x7f1d380ab790 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 --2- 172.21.15.81:0/2183440057 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1d3806f7c0 0x7f1d380a9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:11.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/2183440057 >> 172.21.15.81:0/2183440057 conn(0x7f1d380b04d0 msgr2=0x7f1d380b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:11.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/2183440057 shutdown_connections 2023-11-08T16:37:11.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:11.937+0000 7f1d3f3ff640 1 -- 172.21.15.81:0/2183440057 wait complete. 2023-11-08T16:37:11.957 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T16:37:11.958 INFO:tasks.ceph.mds.c.smithi081.stderr: -9951> 2023-11-08T16:34:58.196+0000 7f1a4368a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:37:11.959 INFO:tasks.ceph.mds.c.smithi081.stderr: -9564> 2023-11-08T16:35:00.880+0000 7f1a3fe83640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-11-08T16:37:11.959 INFO:tasks.ceph.mds.c.smithi081.stderr: -9412> 2023-11-08T16:35:00.882+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x558622158100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:37:11.960 INFO:tasks.ceph.mds.c.smithi081.stderr: -9010> 2023-11-08T16:35:00.884+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /subdir/3 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224c100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001f8:[<0x10000000000/3 v40>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001f8:[<0x10000000000/3 v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.960 INFO:tasks.ceph.mds.c.smithi081.stderr: -8800> 2023-11-08T16:35:00.885+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fd [2,head] /subdir/8 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224a580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001fd:[<0x10000000000/8 v80>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001fd:[<0x10000000000/8 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.964 INFO:tasks.ceph.mds.c.smithi081.stderr: -8777> 2023-11-08T16:35:00.885+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000204 [2,head] /subdir/15 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622254680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000204:[<0x10000000000/15 v136>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000204:[<0x10000000000/15 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.964 INFO:tasks.ceph.mds.c.smithi081.stderr: -8751> 2023-11-08T16:35:00.886+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fe [2,head] /subdir/9 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001fe:[<0x10000000000/9 v88>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001fe:[<0x10000000000/9 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.964 INFO:tasks.ceph.mds.c.smithi081.stderr: -8729> 2023-11-08T16:35:00.886+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [2,head] /subdir/20 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622252580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000209:[<0x10000000000/20 v176>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000209:[<0x10000000000/20 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.965 INFO:tasks.ceph.mds.c.smithi081.stderr: -8707> 2023-11-08T16:35:00.886+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [2,head] /subdir/21 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622252000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x1000000020a:[<0x10000000000/21 v184>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x1000000020a:[<0x10000000000/21 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.965 INFO:tasks.ceph.mds.c.smithi081.stderr: -8685> 2023-11-08T16:35:00.886+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [2,head] /subdir/19 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622253080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000208:[<0x10000000000/19 v168>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000208:[<0x10000000000/19 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.965 INFO:tasks.ceph.mds.c.smithi081.stderr: -8662> 2023-11-08T16:35:00.887+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000201 [2,head] /subdir/12 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622255700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000201:[<0x10000000000/12 v112>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000201:[<0x10000000000/12 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.966 INFO:tasks.ceph.mds.c.smithi081.stderr: -8639> 2023-11-08T16:35:00.887+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001ff [2,head] /subdir/10 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862225c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001ff:[<0x10000000000/10 v96>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001ff:[<0x10000000000/10 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.966 INFO:tasks.ceph.mds.c.smithi081.stderr: -8616> 2023-11-08T16:35:00.887+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000200 [2,head] /subdir/11 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862225c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000200:[<0x10000000000/11 v104>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000200:[<0x10000000000/11 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.967 INFO:tasks.ceph.mds.c.smithi081.stderr: -8593> 2023-11-08T16:35:00.887+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /subdir/2 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224c680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001f7:[<0x10000000000/2 v32>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001f7:[<0x10000000000/2 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.967 INFO:tasks.ceph.mds.c.smithi081.stderr: -8568> 2023-11-08T16:35:00.888+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /subdir/1 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622252b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001f6:[<0x10000000000/1 v24>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001f6:[<0x10000000000/1 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.967 INFO:tasks.ceph.mds.c.smithi081.stderr: -8545> 2023-11-08T16:35:00.888+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /subdir/4 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224bb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001f9:[<0x10000000000/4 v48>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001f9:[<0x10000000000/4 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.968 INFO:tasks.ceph.mds.c.smithi081.stderr: -8523> 2023-11-08T16:35:00.888+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000205 [2,head] /subdir/16 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622254100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000205:[<0x10000000000/16 v144>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000205:[<0x10000000000/16 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.968 INFO:tasks.ceph.mds.c.smithi081.stderr: -8500> 2023-11-08T16:35:00.888+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020d [2,head] /subdir/24 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x1000000020d:[<0x10000000000/24 v200>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x1000000020d:[<0x10000000000/24 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.968 INFO:tasks.ceph.mds.c.smithi081.stderr: -8476> 2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [2,head] /subdir/22 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224d700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x1000000020b:[<0x10000000000/22 v192>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x1000000020b:[<0x10000000000/22 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.969 INFO:tasks.ceph.mds.c.smithi081.stderr: -8454> 2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /subdir/5 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001fa:[<0x10000000000/5 v56>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001fa:[<0x10000000000/5 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.969 INFO:tasks.ceph.mds.c.smithi081.stderr: -8430> 2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/0 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862225cb00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000001:[<0x10000000000/0 v16>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000001:[<0x10000000000/0 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.969 INFO:tasks.ceph.mds.c.smithi081.stderr: -8408> 2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000203 [2,head] /subdir/14 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622254c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000203:[<0x10000000000/14 v128>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000203:[<0x10000000000/14 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.970 INFO:tasks.ceph.mds.c.smithi081.stderr: -8383> 2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000202 [2,head] /subdir/13 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622255180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000202:[<0x10000000000/13 v120>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000202:[<0x10000000000/13 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.970 INFO:tasks.ceph.mds.c.smithi081.stderr: -8361> 2023-11-08T16:35:00.889+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020c [2,head] /subdir/23 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224d180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x1000000020c:[<0x10000000000/23 v198>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x1000000020c:[<0x10000000000/23 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.970 INFO:tasks.ceph.mds.c.smithi081.stderr: -8338> 2023-11-08T16:35:00.890+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000207 [2,head] /subdir/18 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622253600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000207:[<0x10000000000/18 v160>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000207:[<0x10000000000/18 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.971 INFO:tasks.ceph.mds.c.smithi081.stderr: -8322> 2023-11-08T16:35:00.890+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /subdir/7 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224ab00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001fc:[<0x10000000000/7 v72>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001fc:[<0x10000000000/7 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.971 INFO:tasks.ceph.mds.c.smithi081.stderr: -8305> 2023-11-08T16:35:00.890+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000206 [2,head] /subdir/17 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x558622253b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x10000000206:[<0x10000000000/17 v152>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x10000000206:[<0x10000000000/17 v1>,<0x1/subdir v4>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:11.971 INFO:tasks.ceph.mds.c.smithi081.stderr: -8209> 2023-11-08T16:35:00.891+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x55862215ed80 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-11-08T16:35:00.606015+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={6657=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x558622157600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-11-08T16:35:00.606015+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:37:11.972 INFO:tasks.ceph.mds.c.smithi081.stderr: -7988> 2023-11-08T16:35:00.893+0000 7f1a3fe83640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /subdir/6 auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55862224b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(9)0x100000001fb:[<0x10000000000/6 v64>,<0x1/subdir v212>]//[8]","memoryvalue":"(9)0x100000001fb:[<0x10000000000/6 v1>,<0x1/subdir v7>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:37:12.037 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:37:12.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.145+0000 7fb780615640 1 Processor -- start 2023-11-08T16:37:12.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.145+0000 7fb780615640 1 -- start start 2023-11-08T16:37:12.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.146+0000 7fb780615640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb77815e170 0x7fb77815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:12.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.146+0000 7fb780615640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb77815e550 0x7fb7780b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:12.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.146+0000 7fb780615640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb7780b0a20 0x7fb7780ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:12.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.146+0000 7fb780615640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7780b0e70 con 0x7fb7780b0a20 2023-11-08T16:37:12.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.146+0000 7fb780615640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb778071460 con 0x7fb77815e550 2023-11-08T16:37:12.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.146+0000 7fb780615640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780703c0 con 0x7fb77815e170 2023-11-08T16:37:12.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.147+0000 7fb77ebb2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb7780b0a20 0x7fb7780ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:12.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.147+0000 7fb77ebb2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb7780b0a20 0x7fb7780ad4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35952/0 (socket says 172.21.15.81:35952) 2023-11-08T16:37:12.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.147+0000 7fb77ebb2640 1 -- 172.21.15.81:0/1740393850 learned_addr learned my addr 172.21.15.81:0/1740393850 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:12.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.147+0000 7fb77e3b1640 1 --2- 172.21.15.81:0/1740393850 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb77815e170 0x7fb77815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:12.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.147+0000 7fb77dbb0640 1 --2- 172.21.15.81:0/1740393850 >> v2:172.21.15.93:3300/0 conn(0x7fb77815e550 0x7fb7780b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:12.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.147+0000 7fb77ebb2640 1 -- 172.21.15.81:0/1740393850 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb77815e170 msgr2=0x7fb77815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:12.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.147+0000 7fb77ebb2640 1 --2- 172.21.15.81:0/1740393850 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb77815e170 0x7fb77815ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:12.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.147+0000 7fb77ebb2640 1 -- 172.21.15.81:0/1740393850 >> v2:172.21.15.93:3300/0 conn(0x7fb77815e550 msgr2=0x7fb7780b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:12.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.147+0000 7fb77ebb2640 1 --2- 172.21.15.81:0/1740393850 >> v2:172.21.15.93:3300/0 conn(0x7fb77815e550 0x7fb7780b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:12.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.147+0000 7fb77ebb2640 1 -- 172.21.15.81:0/1740393850 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780706a0 con 0x7fb7780b0a20 2023-11-08T16:37:12.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.148+0000 7fb77ebb2640 1 --2- 172.21.15.81:0/1740393850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7780b0a20 0x7fb7780ad4f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb7780cc570 tx=0x7fb77002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253361fe0bfff6e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:12.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.149+0000 7fb77d3af640 1 -- 172.21.15.81:0/1740393850 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77002ca70 con 0x7fb7780b0a20 2023-11-08T16:37:12.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.149+0000 7fb77d3af640 1 -- 172.21.15.81:0/1740393850 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77002cbd0 con 0x7fb7780b0a20 2023-11-08T16:37:12.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.149+0000 7fb77d3af640 1 -- 172.21.15.81:0/1740393850 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7700343f0 con 0x7fb7780b0a20 2023-11-08T16:37:12.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.149+0000 7fb780615640 1 -- 172.21.15.81:0/1740393850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7780b0a20 msgr2=0x7fb7780ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:12.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.149+0000 7fb780615640 1 --2- 172.21.15.81:0/1740393850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7780b0a20 0x7fb7780ad4f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fb7780cc570 tx=0x7fb77002bed0 comp rx=0 tx=0).stop 2023-11-08T16:37:12.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.149+0000 7fb780615640 1 -- 172.21.15.81:0/1740393850 shutdown_connections 2023-11-08T16:37:12.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.149+0000 7fb780615640 1 --2- 172.21.15.81:0/1740393850 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7780b0a20 0x7fb7780ad4f0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:12.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.149+0000 7fb780615640 1 --2- 172.21.15.81:0/1740393850 >> v2:172.21.15.93:3300/0 conn(0x7fb77815e550 0x7fb7780b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:12.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.149+0000 7fb780615640 1 --2- 172.21.15.81:0/1740393850 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb77815e170 0x7fb77815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:12.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.149+0000 7fb780615640 1 -- 172.21.15.81:0/1740393850 >> 172.21.15.81:0/1740393850 conn(0x7fb7780cb4e0 msgr2=0x7fb7780cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:12.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.150+0000 7fb780615640 1 -- 172.21.15.81:0/1740393850 shutdown_connections 2023-11-08T16:37:12.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.150+0000 7fb780615640 1 -- 172.21.15.81:0/1740393850 wait complete. 2023-11-08T16:37:12.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.151+0000 7fb780615640 1 Processor -- start 2023-11-08T16:37:12.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.151+0000 7fb780615640 1 -- start start 2023-11-08T16:37:12.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.152+0000 7fb780615640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7780b0a20 0x7fb7781511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:12.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.152+0000 7fb780615640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb77815e170 0x7fb7780b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:12.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.152+0000 7fb780615640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb77815e550 0x7fb7781504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:12.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.152+0000 7fb780615640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780b3490 con 0x7fb7780b0a20 2023-11-08T16:37:12.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.152+0000 7fb780615640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780b1bd0 con 0x7fb77815e550 2023-11-08T16:37:12.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.152+0000 7fb780615640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780b1eb0 con 0x7fb77815e170 2023-11-08T16:37:12.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.153+0000 7fb77e3b1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7780b0a20 0x7fb7781511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:12.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.153+0000 7fb77e3b1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7780b0a20 0x7fb7781511c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35962/0 (socket says 172.21.15.81:35962) 2023-11-08T16:37:12.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.153+0000 7fb77e3b1640 1 -- 172.21.15.81:0/2190216454 learned_addr learned my addr 172.21.15.81:0/2190216454 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:12.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.153+0000 7fb77dbb0640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb77815e170 0x7fb7780b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:12.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.153+0000 7fb77ebb2640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb77815e550 0x7fb7781504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:12.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.153+0000 7fb77e3b1640 1 -- 172.21.15.81:0/2190216454 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb77815e170 msgr2=0x7fb7780b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:12.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.153+0000 7fb77e3b1640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb77815e170 0x7fb7780b2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:12.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.153+0000 7fb77e3b1640 1 -- 172.21.15.81:0/2190216454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb77815e550 msgr2=0x7fb7781504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:12.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.153+0000 7fb77e3b1640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb77815e550 0x7fb7781504d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:12.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.153+0000 7fb77e3b1640 1 -- 172.21.15.81:0/2190216454 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb770009b30 con 0x7fb7780b0a20 2023-11-08T16:37:12.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.153+0000 7fb77e3b1640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7780b0a20 0x7fb7781511c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb77400e9b0 tx=0x7fb77400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:12.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.154+0000 7fb76effd640 1 -- 172.21.15.81:0/2190216454 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774009670 con 0x7fb7780b0a20 2023-11-08T16:37:12.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.154+0000 7fb76effd640 1 -- 172.21.15.81:0/2190216454 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77400c9a0 con 0x7fb7780b0a20 2023-11-08T16:37:12.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.154+0000 7fb780615640 1 -- 172.21.15.81:0/2190216454 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7780b2190 con 0x7fb7780b0a20 2023-11-08T16:37:12.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.154+0000 7fb76effd640 1 -- 172.21.15.81:0/2190216454 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77400cd50 con 0x7fb7780b0a20 2023-11-08T16:37:12.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.154+0000 7fb780615640 1 -- 172.21.15.81:0/2190216454 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7780b2750 con 0x7fb7780b0a20 2023-11-08T16:37:12.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.155+0000 7fb780615640 1 -- 172.21.15.81:0/2190216454 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77804ece0 con 0x7fb7780b0a20 2023-11-08T16:37:12.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.156+0000 7fb76effd640 1 -- 172.21.15.81:0/2190216454 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb774013470 con 0x7fb7780b0a20 2023-11-08T16:37:12.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.156+0000 7fb76effd640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb75c06ae70 0x7fb75c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:12.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.157+0000 7fb76effd640 1 -- 172.21.15.81:0/2190216454 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6544+0+0 (secure 0 0 0) 0x7fb774089be0 con 0x7fb7780b0a20 2023-11-08T16:37:12.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.157+0000 7fb77dbb0640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb75c06ae70 0x7fb75c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:12.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.157+0000 7fb77dbb0640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb75c06ae70 0x7fb75c06d330 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb768006fd0 tx=0x7fb768008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:12.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.161+0000 7fb76effd640 1 -- 172.21.15.81:0/2190216454 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb7740548d0 con 0x7fb7780b0a20 2023-11-08T16:37:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:12.372+0000 7fb780615640 1 -- 172.21.15.81:0/2190216454 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb778070950 con 0x7fb7780b0a20 2023-11-08T16:37:13.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.013+0000 7fb76effd640 1 -- 172.21.15.81:0/2190216454 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb7740890a0 con 0x7fb7780b0a20 2023-11-08T16:37:13.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.016+0000 7fb780615640 1 -- 172.21.15.81:0/2190216454 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb75c06ae70 msgr2=0x7fb75c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.016+0000 7fb780615640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb75c06ae70 0x7fb75c06d330 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb768006fd0 tx=0x7fb768008040 comp rx=0 tx=0).stop 2023-11-08T16:37:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.016+0000 7fb780615640 1 -- 172.21.15.81:0/2190216454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7780b0a20 msgr2=0x7fb7781511c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.016+0000 7fb780615640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7780b0a20 0x7fb7781511c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb77400e9b0 tx=0x7fb77400ee70 comp rx=0 tx=0).stop 2023-11-08T16:37:13.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.016+0000 7fb780615640 1 -- 172.21.15.81:0/2190216454 shutdown_connections 2023-11-08T16:37:13.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.016+0000 7fb780615640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb75c06ae70 0x7fb75c06d330 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:13.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.016+0000 7fb780615640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb77815e550 0x7fb7781504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:13.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.016+0000 7fb780615640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb77815e170 0x7fb7780b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:13.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.016+0000 7fb780615640 1 --2- 172.21.15.81:0/2190216454 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7780b0a20 0x7fb7781511c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:13.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.016+0000 7fb780615640 1 -- 172.21.15.81:0/2190216454 >> 172.21.15.81:0/2190216454 conn(0x7fb7780cb4e0 msgr2=0x7fb77815d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.017+0000 7fb780615640 1 -- 172.21.15.81:0/2190216454 shutdown_connections 2023-11-08T16:37:13.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.017+0000 7fb780615640 1 -- 172.21.15.81:0/2190216454 wait complete. 2023-11-08T16:37:13.040 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T16:37:13.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.244+0000 7f6e4e044640 1 Processor -- start 2023-11-08T16:37:13.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.244+0000 7f6e4e044640 1 -- start start 2023-11-08T16:37:13.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.245+0000 7f6e4e044640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6e4815e170 0x7f6e4815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:13.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.246+0000 7f6e4e044640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e480b0bf0 0x7f6e480ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:13.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.246+0000 7f6e4e044640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e48071460 con 0x7f6e4815e550 2023-11-08T16:37:13.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.246+0000 7f6e4e044640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e480703c0 con 0x7f6e4815e170 2023-11-08T16:37:13.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.246+0000 7f6e4e044640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480706a0 con 0x7f6e480b0bf0 2023-11-08T16:37:13.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.246+0000 7f6e477fe640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6e4815e170 0x7f6e4815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:13.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.246+0000 7f6e477fe640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6e4815e170 0x7f6e4815ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:46428/0 (socket says 172.21.15.81:46428) 2023-11-08T16:37:13.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.246+0000 7f6e47fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e480b0bf0 0x7f6e480ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:13.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.246+0000 7f6e477fe640 1 -- 172.21.15.81:0/3461050348 learned_addr learned my addr 172.21.15.81:0/3461050348 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:13.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.247+0000 7f6e47fff640 1 -- 172.21.15.81:0/3461050348 >> v2:172.21.15.93:3300/0 conn(0x7f6e4815e170 msgr2=0x7f6e4815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:13.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.247+0000 7f6e47fff640 1 --2- 172.21.15.81:0/3461050348 >> v2:172.21.15.93:3300/0 conn(0x7f6e4815e170 0x7f6e4815ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:13.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.247+0000 7f6e47fff640 1 -- 172.21.15.81:0/3461050348 >> v1:172.21.15.93:6789/0 conn(0x7f6e4815e550 legacy=0x7f6e480b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:13.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.247+0000 7f6e47fff640 1 -- 172.21.15.81:0/3461050348 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e48070980 con 0x7f6e480b0bf0 2023-11-08T16:37:13.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.248+0000 7f6e47fff640 1 --2- 172.21.15.81:0/3461050348 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e480b0bf0 0x7f6e480ad4f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009d90 tx=0x7f6e4002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29bb6e23798aaede server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:13.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.249+0000 7f6e46ffd640 1 -- 172.21.15.81:0/3461050348 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4002cc50 con 0x7f6e480b0bf0 2023-11-08T16:37:13.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.249+0000 7f6e46ffd640 1 -- 172.21.15.81:0/3461050348 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4002cdb0 con 0x7f6e480b0bf0 2023-11-08T16:37:13.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.249+0000 7f6e46ffd640 1 -- 172.21.15.81:0/3461050348 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40034660 con 0x7f6e480b0bf0 2023-11-08T16:37:13.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.249+0000 7f6e4e044640 1 -- 172.21.15.81:0/3461050348 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e480b0bf0 msgr2=0x7f6e480ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:13.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.249+0000 7f6e4e044640 1 --2- 172.21.15.81:0/3461050348 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e480b0bf0 0x7f6e480ad4f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009d90 tx=0x7f6e4002bea0 comp rx=0 tx=0).stop 2023-11-08T16:37:13.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.249+0000 7f6e4e044640 1 -- 172.21.15.81:0/3461050348 shutdown_connections 2023-11-08T16:37:13.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.249+0000 7f6e4e044640 1 --2- 172.21.15.81:0/3461050348 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e480b0bf0 0x7f6e480ad4f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:13.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.249+0000 7f6e4e044640 1 --2- 172.21.15.81:0/3461050348 >> v2:172.21.15.93:3300/0 conn(0x7f6e4815e170 0x7f6e4815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:13.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.249+0000 7f6e4e044640 1 -- 172.21.15.81:0/3461050348 >> 172.21.15.81:0/3461050348 conn(0x7f6e480cb4e0 msgr2=0x7f6e480cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:13.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.249+0000 7f6e4e044640 1 -- 172.21.15.81:0/3461050348 shutdown_connections 2023-11-08T16:37:13.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.250+0000 7f6e4e044640 1 -- 172.21.15.81:0/3461050348 wait complete. 2023-11-08T16:37:13.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.251+0000 7f6e4e044640 1 Processor -- start 2023-11-08T16:37:13.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.251+0000 7f6e4e044640 1 -- start start 2023-11-08T16:37:13.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.252+0000 7f6e4e044640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e480b0bf0 0x7f6e48153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:13.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.252+0000 7f6e4e044640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e4815e170 0x7f6e481f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:13.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.252+0000 7f6e4e044640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4815e550 0x7f6e481f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:13.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.252+0000 7f6e4e044640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e480c35c0 con 0x7f6e4815e550 2023-11-08T16:37:13.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.252+0000 7f6e4e044640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e48154540 con 0x7f6e4815e170 2023-11-08T16:37:13.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.252+0000 7f6e4e044640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e48154040 con 0x7f6e480b0bf0 2023-11-08T16:37:13.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e47fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4815e550 0x7f6e481f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:13.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e47fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4815e550 0x7f6e481f2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35978/0 (socket says 172.21.15.81:35978) 2023-11-08T16:37:13.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e47fff640 1 -- 172.21.15.81:0/2243121017 learned_addr learned my addr 172.21.15.81:0/2243121017 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:13.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e3ffff640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e4815e170 0x7f6e481f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:13.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e477fe640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e480b0bf0 0x7f6e48153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:13.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e47fff640 1 -- 172.21.15.81:0/2243121017 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e480b0bf0 msgr2=0x7f6e48153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:13.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e47fff640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e480b0bf0 0x7f6e48153b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:13.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e47fff640 1 -- 172.21.15.81:0/2243121017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e4815e170 msgr2=0x7f6e481f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:13.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e47fff640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e4815e170 0x7f6e481f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:13.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e47fff640 1 -- 172.21.15.81:0/2243121017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e40009a70 con 0x7f6e4815e550 2023-11-08T16:37:13.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e3ffff640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e4815e170 0x7f6e481f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:13.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.253+0000 7f6e47fff640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4815e550 0x7f6e481f2a60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009d90 tx=0x7f6e40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:13.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.254+0000 7f6e44ff9640 1 -- 172.21.15.81:0/2243121017 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40040650 con 0x7f6e4815e550 2023-11-08T16:37:13.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.254+0000 7f6e44ff9640 1 -- 172.21.15.81:0/2243121017 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e400407b0 con 0x7f6e4815e550 2023-11-08T16:37:13.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.254+0000 7f6e44ff9640 1 -- 172.21.15.81:0/2243121017 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e40040b60 con 0x7f6e4815e550 2023-11-08T16:37:13.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.254+0000 7f6e4e044640 1 -- 172.21.15.81:0/2243121017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e480c6f80 con 0x7f6e4815e550 2023-11-08T16:37:13.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.254+0000 7f6e4e044640 1 -- 172.21.15.81:0/2243121017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e480c7540 con 0x7f6e4815e550 2023-11-08T16:37:13.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.255+0000 7f6e4e044640 1 -- 172.21.15.81:0/2243121017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e480c3770 con 0x7f6e4815e550 2023-11-08T16:37:13.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.256+0000 7f6e44ff9640 1 -- 172.21.15.81:0/2243121017 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e40040cc0 con 0x7f6e4815e550 2023-11-08T16:37:13.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.257+0000 7f6e44ff9640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e2806aea0 0x7f6e2806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:13.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.257+0000 7f6e44ff9640 1 -- 172.21.15.81:0/2243121017 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6544+0+0 (secure 0 0 0) 0x7f6e400ac6e0 con 0x7f6e4815e550 2023-11-08T16:37:13.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.257+0000 7f6e477fe640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e2806aea0 0x7f6e2806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:13.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.258+0000 7f6e477fe640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e2806aea0 0x7f6e2806d360 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009aa0 tx=0x7f6e38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:13.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.261+0000 7f6e44ff9640 1 -- 172.21.15.81:0/2243121017 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e480c3770 con 0x7f6e4815e550 2023-11-08T16:37:13.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:13.450+0000 7f6e4e044640 1 -- 172.21.15.81:0/2243121017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e480743f0 con 0x7f6e4815e550 2023-11-08T16:37:14.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.016+0000 7f6e44ff9640 1 -- 172.21.15.81:0/2243121017 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v70) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6e400773d0 con 0x7f6e4815e550 2023-11-08T16:37:14.019 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' removed 2023-11-08T16:37:14.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.020+0000 7f6e4e044640 1 -- 172.21.15.81:0/2243121017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e2806aea0 msgr2=0x7f6e2806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:14.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.020+0000 7f6e4e044640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e2806aea0 0x7f6e2806d360 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009aa0 tx=0x7f6e38008040 comp rx=0 tx=0).stop 2023-11-08T16:37:14.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.020+0000 7f6e4e044640 1 -- 172.21.15.81:0/2243121017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4815e550 msgr2=0x7f6e481f2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:14.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.020+0000 7f6e4e044640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4815e550 0x7f6e481f2a60 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009d90 tx=0x7f6e40004a40 comp rx=0 tx=0).stop 2023-11-08T16:37:14.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.020+0000 7f6e4e044640 1 -- 172.21.15.81:0/2243121017 shutdown_connections 2023-11-08T16:37:14.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.020+0000 7f6e4e044640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e2806aea0 0x7f6e2806d360 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:14.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.020+0000 7f6e4e044640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4815e550 0x7f6e481f2a60 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:14.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.020+0000 7f6e4e044640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e4815e170 0x7f6e481f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:14.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.020+0000 7f6e4e044640 1 --2- 172.21.15.81:0/2243121017 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e480b0bf0 0x7f6e48153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:14.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.020+0000 7f6e4e044640 1 -- 172.21.15.81:0/2243121017 >> 172.21.15.81:0/2243121017 conn(0x7f6e480cb4e0 msgr2=0x7f6e480cdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:14.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.022+0000 7f6e4e044640 1 -- 172.21.15.81:0/2243121017 shutdown_connections 2023-11-08T16:37:14.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.022+0000 7f6e4e044640 1 -- 172.21.15.81:0/2243121017 wait complete. 2023-11-08T16:37:14.047 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T16:37:14.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.258+0000 7ff329d16640 1 Processor -- start 2023-11-08T16:37:14.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.259+0000 7ff329d16640 1 -- start start 2023-11-08T16:37:14.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.260+0000 7ff329d16640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff324071890 0x7ff32415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:14.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.260+0000 7ff329d16640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff32415afd0 0x7ff32415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:14.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.260+0000 7ff329d16640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3240b09f0 con 0x7ff3240714b0 2023-11-08T16:37:14.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.260+0000 7ff329d16640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32406b250 con 0x7ff324071890 2023-11-08T16:37:14.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.260+0000 7ff329d16640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240ca360 con 0x7ff32415afd0 2023-11-08T16:37:14.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.260+0000 7ff31bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff324071890 0x7ff32415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:14.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.260+0000 7ff31bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff324071890 0x7ff32415e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35980/0 (socket says 172.21.15.81:35980) 2023-11-08T16:37:14.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.260+0000 7ff31bfff640 1 -- 172.21.15.81:0/486552005 learned_addr learned my addr 172.21.15.81:0/486552005 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:14.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.261+0000 7ff323fff640 1 --2- 172.21.15.81:0/486552005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff32415afd0 0x7ff32415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:14.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.261+0000 7ff31bfff640 1 -- 172.21.15.81:0/486552005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff32415afd0 msgr2=0x7ff32415ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:14.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.261+0000 7ff31bfff640 1 --2- 172.21.15.81:0/486552005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff32415afd0 0x7ff32415ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:14.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.261+0000 7ff31bfff640 1 -- 172.21.15.81:0/486552005 >> v1:172.21.15.81:6789/0 conn(0x7ff3240714b0 legacy=0x7ff32415a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:14.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.261+0000 7ff31bfff640 1 -- 172.21.15.81:0/486552005 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3240ca640 con 0x7ff324071890 2023-11-08T16:37:14.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.262+0000 7ff31bfff640 1 --2- 172.21.15.81:0/486552005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff324071890 0x7ff32415e9d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009870 tx=0x7ff30c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1719edd5f2339b35 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:14.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.263+0000 7ff322ffd640 1 -- 172.21.15.81:0/486552005 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c030840 con 0x7ff324071890 2023-11-08T16:37:14.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.263+0000 7ff322ffd640 1 -- 172.21.15.81:0/486552005 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c0309a0 con 0x7ff324071890 2023-11-08T16:37:14.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.263+0000 7ff322ffd640 1 -- 172.21.15.81:0/486552005 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c030d30 con 0x7ff324071890 2023-11-08T16:37:14.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.263+0000 7ff329d16640 1 -- 172.21.15.81:0/486552005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff324071890 msgr2=0x7ff32415e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:14.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.263+0000 7ff329d16640 1 --2- 172.21.15.81:0/486552005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff324071890 0x7ff32415e9d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7ff30c009870 tx=0x7ff30c02b980 comp rx=0 tx=0).stop 2023-11-08T16:37:14.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.263+0000 7ff329d16640 1 -- 172.21.15.81:0/486552005 shutdown_connections 2023-11-08T16:37:14.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.263+0000 7ff329d16640 1 --2- 172.21.15.81:0/486552005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff32415afd0 0x7ff32415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:14.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.263+0000 7ff329d16640 1 --2- 172.21.15.81:0/486552005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff324071890 0x7ff32415e9d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:14.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.263+0000 7ff329d16640 1 -- 172.21.15.81:0/486552005 >> 172.21.15.81:0/486552005 conn(0x7ff3240cb4e0 msgr2=0x7ff3240cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:14.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.263+0000 7ff329d16640 1 -- 172.21.15.81:0/486552005 shutdown_connections 2023-11-08T16:37:14.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.263+0000 7ff329d16640 1 -- 172.21.15.81:0/486552005 wait complete. 2023-11-08T16:37:14.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.265+0000 7ff329d16640 1 Processor -- start 2023-11-08T16:37:14.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.265+0000 7ff329d16640 1 -- start start 2023-11-08T16:37:14.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.266+0000 7ff329d16640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3240714b0 0x7ff32414fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:14.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.266+0000 7ff329d16640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff324071890 0x7ff324157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:14.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.266+0000 7ff329d16640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff32415afd0 0x7ff3240af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:14.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.266+0000 7ff329d16640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240cdcf0 con 0x7ff3240714b0 2023-11-08T16:37:14.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.266+0000 7ff329d16640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240cd0e0 con 0x7ff32415afd0 2023-11-08T16:37:14.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.266+0000 7ff329d16640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3240cd3c0 con 0x7ff324071890 2023-11-08T16:37:14.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.266+0000 7ff3237fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3240714b0 0x7ff32414fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:14.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff3237fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3240714b0 0x7ff32414fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35984/0 (socket says 172.21.15.81:35984) 2023-11-08T16:37:14.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff3237fe640 1 -- 172.21.15.81:0/3607332799 learned_addr learned my addr 172.21.15.81:0/3607332799 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:14.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff31bfff640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff324071890 0x7ff324157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:14.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff323fff640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff32415afd0 0x7ff3240af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:14.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff3237fe640 1 -- 172.21.15.81:0/3607332799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff324071890 msgr2=0x7ff324157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:14.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff3237fe640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff324071890 0x7ff324157a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:14.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff3237fe640 1 -- 172.21.15.81:0/3607332799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff32415afd0 msgr2=0x7ff3240af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:14.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff3237fe640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff32415afd0 0x7ff3240af850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:14.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff3237fe640 1 -- 172.21.15.81:0/3607332799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff30c0094d0 con 0x7ff3240714b0 2023-11-08T16:37:14.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff3237fe640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3240714b0 0x7ff32414fac0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff31400c450 tx=0x7ff31400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:14.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff320ff9640 1 -- 172.21.15.81:0/3607332799 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3140052d0 con 0x7ff3240714b0 2023-11-08T16:37:14.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.267+0000 7ff320ff9640 1 -- 172.21.15.81:0/3607332799 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff314005430 con 0x7ff3240714b0 2023-11-08T16:37:14.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.268+0000 7ff329d16640 1 -- 172.21.15.81:0/3607332799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3240cd6a0 con 0x7ff3240714b0 2023-11-08T16:37:14.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.268+0000 7ff320ff9640 1 -- 172.21.15.81:0/3607332799 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff314012870 con 0x7ff3240714b0 2023-11-08T16:37:14.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.268+0000 7ff329d16640 1 -- 172.21.15.81:0/3607332799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3240ce180 con 0x7ff3240714b0 2023-11-08T16:37:14.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.269+0000 7ff329d16640 1 -- 172.21.15.81:0/3607332799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3240ce310 con 0x7ff3240714b0 2023-11-08T16:37:14.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.270+0000 7ff320ff9640 1 -- 172.21.15.81:0/3607332799 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff3140129d0 con 0x7ff3240714b0 2023-11-08T16:37:14.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.271+0000 7ff320ff9640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff30406ae70 0x7ff30406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:14.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.271+0000 7ff320ff9640 1 -- 172.21.15.81:0/3607332799 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6106+0+0 (secure 0 0 0) 0x7ff314089c90 con 0x7ff3240714b0 2023-11-08T16:37:14.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.271+0000 7ff31bfff640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff30406ae70 0x7ff30406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:14.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.272+0000 7ff31bfff640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff30406ae70 0x7ff30406d330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff30c036000 tx=0x7ff30c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:14.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.274+0000 7ff320ff9640 1 -- 172.21.15.81:0/3607332799 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff314054d20 con 0x7ff3240714b0 2023-11-08T16:37:14.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:14.474+0000 7ff329d16640 1 -- 172.21.15.81:0/3607332799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff3240743f0 con 0x7ff3240714b0 2023-11-08T16:37:15.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.029+0000 7ff320ff9640 1 -- 172.21.15.81:0/3607332799 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v71) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff314058340 con 0x7ff3240714b0 2023-11-08T16:37:15.033 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' removed 2023-11-08T16:37:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.034+0000 7ff329d16640 1 -- 172.21.15.81:0/3607332799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff30406ae70 msgr2=0x7ff30406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:15.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.034+0000 7ff329d16640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff30406ae70 0x7ff30406d330 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff30c036000 tx=0x7ff30c03b040 comp rx=0 tx=0).stop 2023-11-08T16:37:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.034+0000 7ff329d16640 1 -- 172.21.15.81:0/3607332799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3240714b0 msgr2=0x7ff32414fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.034+0000 7ff329d16640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3240714b0 0x7ff32414fac0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff31400c450 tx=0x7ff31400c910 comp rx=0 tx=0).stop 2023-11-08T16:37:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.035+0000 7ff329d16640 1 -- 172.21.15.81:0/3607332799 shutdown_connections 2023-11-08T16:37:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.035+0000 7ff329d16640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff30406ae70 0x7ff30406d330 secure :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7ff30c036000 tx=0x7ff30c03b040 comp rx=0 tx=0).stop 2023-11-08T16:37:15.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.035+0000 7ff329d16640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff32415afd0 0x7ff3240af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:15.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.035+0000 7ff329d16640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff324071890 0x7ff324157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:15.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.035+0000 7ff329d16640 1 --2- 172.21.15.81:0/3607332799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff3240714b0 0x7ff32414fac0 secure :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff31400c450 tx=0x7ff31400c910 comp rx=0 tx=0).stop 2023-11-08T16:37:15.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.035+0000 7ff329d16640 1 -- 172.21.15.81:0/3607332799 >> 172.21.15.81:0/3607332799 conn(0x7ff3240cb4e0 msgr2=0x7ff3240c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:15.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.037+0000 7ff329d16640 1 -- 172.21.15.81:0/3607332799 shutdown_connections 2023-11-08T16:37:15.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.037+0000 7ff329d16640 1 -- 172.21.15.81:0/3607332799 wait complete. 2023-11-08T16:37:15.067 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-11-08T16:37:15.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.269+0000 7f76dc3d7640 1 Processor -- start 2023-11-08T16:37:15.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.269+0000 7f76dc3d7640 1 -- start start 2023-11-08T16:37:15.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76dc3d7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f76d415e170 0x7f76d415ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:15.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76dc3d7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76d40b0bf0 0x7f76d40ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:15.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76dc3d7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d4071460 con 0x7f76d415e550 2023-11-08T16:37:15.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76dc3d7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d40703c0 con 0x7f76d415e170 2023-11-08T16:37:15.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76dc3d7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40706a0 con 0x7f76d40b0bf0 2023-11-08T16:37:15.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76da173640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f76d415e170 0x7f76d415ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:15.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76d9972640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f76d415e550 0x7f76d40b04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34078/0 (socket says 172.21.15.81:34078) 2023-11-08T16:37:15.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76d9972640 1 -- 172.21.15.81:0/2984798111 learned_addr learned my addr 172.21.15.81:0/2984798111 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:15.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76da173640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f76d415e170 0x7f76d415ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:35998/0 (socket says 172.21.15.81:35998) 2023-11-08T16:37:15.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76da974640 1 --2- 172.21.15.81:0/2984798111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76d40b0bf0 0x7f76d40ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:15.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76da173640 1 -- 172.21.15.81:0/2984798111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76d40b0bf0 msgr2=0x7f76d40ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:15.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76da173640 1 --2- 172.21.15.81:0/2984798111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76d40b0bf0 0x7f76d40ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:15.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76da173640 1 -- 172.21.15.81:0/2984798111 >> v1:172.21.15.81:6789/0 conn(0x7f76d415e550 legacy=0x7f76d40b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:15.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76da173640 1 -- 172.21.15.81:0/2984798111 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d4070980 con 0x7f76d415e170 2023-11-08T16:37:15.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.271+0000 7f76d9171640 1 -- 172.21.15.81:0/2984798111 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1043769309 0 0) 0x7f76d4071460 con 0x7f76d415e550 2023-11-08T16:37:15.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.272+0000 7f76da173640 1 --2- 172.21.15.81:0/2984798111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76d415e170 0x7f76d415ac80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f76d000ba40 tx=0x7f76d002e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd3432ada743184 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:15.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.272+0000 7f76d9171640 1 -- 172.21.15.81:0/2984798111 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d002ee30 con 0x7f76d415e170 2023-11-08T16:37:15.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.272+0000 7f76d9171640 1 -- 172.21.15.81:0/2984798111 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d0036440 con 0x7f76d415e170 2023-11-08T16:37:15.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.272+0000 7f76d9171640 1 -- 172.21.15.81:0/2984798111 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d00367d0 con 0x7f76d415e170 2023-11-08T16:37:15.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.273+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2984798111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76d415e170 msgr2=0x7f76d415ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:15.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.273+0000 7f76dc3d7640 1 --2- 172.21.15.81:0/2984798111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76d415e170 0x7f76d415ac80 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f76d000ba40 tx=0x7f76d002e080 comp rx=0 tx=0).stop 2023-11-08T16:37:15.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.273+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2984798111 shutdown_connections 2023-11-08T16:37:15.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.273+0000 7f76dc3d7640 1 --2- 172.21.15.81:0/2984798111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76d40b0bf0 0x7f76d40ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:15.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.273+0000 7f76dc3d7640 1 --2- 172.21.15.81:0/2984798111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76d415e170 0x7f76d415ac80 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:15.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.273+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2984798111 >> 172.21.15.81:0/2984798111 conn(0x7f76d40cb4e0 msgr2=0x7f76d40cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:15.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.273+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2984798111 shutdown_connections 2023-11-08T16:37:15.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.273+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2984798111 wait complete. 2023-11-08T16:37:15.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.275+0000 7f76dc3d7640 1 Processor -- start 2023-11-08T16:37:15.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.275+0000 7f76dc3d7640 1 -- start start 2023-11-08T16:37:15.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76dc3d7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d40b0bf0 0x7f76d40c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:15.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76dc3d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76d415e170 0x7f76d4145e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:15.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76dc3d7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76d415e550 0x7f76d40b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:15.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76dc3d7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40c36f0 con 0x7f76d415e170 2023-11-08T16:37:15.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76dc3d7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d4151450 con 0x7f76d40b0bf0 2023-11-08T16:37:15.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76dc3d7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d40c2d50 con 0x7f76d415e550 2023-11-08T16:37:15.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76d9972640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76d415e170 0x7f76d4145e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:15.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76d9972640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76d415e170 0x7f76d4145e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36000/0 (socket says 172.21.15.81:36000) 2023-11-08T16:37:15.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76d9972640 1 -- 172.21.15.81:0/2180129092 learned_addr learned my addr 172.21.15.81:0/2180129092 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:15.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76d9972640 1 -- 172.21.15.81:0/2180129092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76d415e550 msgr2=0x7f76d40b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:15.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76d9972640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76d415e550 0x7f76d40b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:15.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76d9972640 1 -- 172.21.15.81:0/2180129092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d40b0bf0 msgr2=0x7f76d40c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:15.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76d9972640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d40b0bf0 0x7f76d40c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:15.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.276+0000 7f76d9972640 1 -- 172.21.15.81:0/2180129092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d000b6a0 con 0x7f76d415e170 2023-11-08T16:37:15.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.277+0000 7f76d9972640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76d415e170 0x7f76d4145e00 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f76c400c410 tx=0x7f76c400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:15.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.277+0000 7f76caffd640 1 -- 172.21.15.81:0/2180129092 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c40074e0 con 0x7f76d415e170 2023-11-08T16:37:15.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.277+0000 7f76caffd640 1 -- 172.21.15.81:0/2180129092 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c4007660 con 0x7f76d415e170 2023-11-08T16:37:15.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.277+0000 7f76caffd640 1 -- 172.21.15.81:0/2180129092 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4004a20 con 0x7f76d415e170 2023-11-08T16:37:15.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.277+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2180129092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d40c3030 con 0x7f76d415e170 2023-11-08T16:37:15.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.277+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2180129092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d40c6110 con 0x7f76d415e170 2023-11-08T16:37:15.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.278+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2180129092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d40743f0 con 0x7f76d415e170 2023-11-08T16:37:15.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.279+0000 7f76caffd640 1 -- 172.21.15.81:0/2180129092 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f76c4004b80 con 0x7f76d415e170 2023-11-08T16:37:15.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.280+0000 7f76caffd640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f76b806aea0 0x7f76b806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:15.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.281+0000 7f76caffd640 1 -- 172.21.15.81:0/2180129092 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5634+0+0 (secure 0 0 0) 0x7f76c40894d0 con 0x7f76d415e170 2023-11-08T16:37:15.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.281+0000 7f76da173640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f76b806aea0 0x7f76b806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:15.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.282+0000 7f76da173640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f76b806aea0 0x7f76b806d360 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f76d0038000 tx=0x7f76d0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:15.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.285+0000 7f76caffd640 1 -- 172.21.15.81:0/2180129092 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f76c4057fb0 con 0x7f76d415e170 2023-11-08T16:37:15.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:15.474+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2180129092 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f76d40c3830 con 0x7f76d415e170 2023-11-08T16:37:16.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.042+0000 7f76caffd640 1 -- 172.21.15.81:0/2180129092 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v72) v1 ==== 187+0+0 (secure 0 0 0) 0x7f76d40c3830 con 0x7f76d415e170 2023-11-08T16:37:16.045 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data_extra' removed 2023-11-08T16:37:16.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.046+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2180129092 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f76b806aea0 msgr2=0x7f76b806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.046+0000 7f76dc3d7640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f76b806aea0 0x7f76b806d360 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f76d0038000 tx=0x7f76d0030000 comp rx=0 tx=0).stop 2023-11-08T16:37:16.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.046+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2180129092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76d415e170 msgr2=0x7f76d4145e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.046+0000 7f76dc3d7640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76d415e170 0x7f76d4145e00 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f76c400c410 tx=0x7f76c400c8d0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.046+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2180129092 shutdown_connections 2023-11-08T16:37:16.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.046+0000 7f76dc3d7640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f76b806aea0 0x7f76b806d360 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.046+0000 7f76dc3d7640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f76d415e550 0x7f76d40b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.046+0000 7f76dc3d7640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f76d415e170 0x7f76d4145e00 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.046+0000 7f76dc3d7640 1 --2- 172.21.15.81:0/2180129092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76d40b0bf0 0x7f76d40c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.046+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2180129092 >> 172.21.15.81:0/2180129092 conn(0x7f76d40cb4e0 msgr2=0x7f76d415c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:16.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.047+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2180129092 shutdown_connections 2023-11-08T16:37:16.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.047+0000 7f76dc3d7640 1 -- 172.21.15.81:0/2180129092 wait complete. 2023-11-08T16:37:16.071 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T16:37:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.290+0000 7f343075a640 1 Processor -- start 2023-11-08T16:37:16.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.290+0000 7f343075a640 1 -- start start 2023-11-08T16:37:16.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.292+0000 7f343075a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f34280cd040 0x7f34280b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.292+0000 7f343075a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34280b0a20 0x7f3428160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.292+0000 7f343075a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3428160b40 con 0x7f34280cab40 2023-11-08T16:37:16.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.292+0000 7f343075a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f342806b960 con 0x7f34280cd040 2023-11-08T16:37:16.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.292+0000 7f343075a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34280cd4d0 con 0x7f34280b0a20 2023-11-08T16:37:16.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.293+0000 7f342dcf5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f34280cd040 0x7f34280b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.293+0000 7f342dcf5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f34280cd040 0x7f34280b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36006/0 (socket says 172.21.15.81:36006) 2023-11-08T16:37:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.293+0000 7f342dcf5640 1 -- 172.21.15.81:0/4238544798 learned_addr learned my addr 172.21.15.81:0/4238544798 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.293+0000 7f342e4f6640 1 --1- 172.21.15.81:0/4238544798 >> v1:172.21.15.93:6789/0 conn(0x7f34280cab40 0x7f342806b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:44320/0 (socket says 172.21.15.81:44320) 2023-11-08T16:37:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.293+0000 7f342ecf7640 1 --2- 172.21.15.81:0/4238544798 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34280b0a20 0x7f3428160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.294+0000 7f342dcf5640 1 -- 172.21.15.81:0/4238544798 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34280b0a20 msgr2=0x7f3428160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:16.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.294+0000 7f342dcf5640 1 --2- 172.21.15.81:0/4238544798 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34280b0a20 0x7f3428160fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.294+0000 7f342dcf5640 1 -- 172.21.15.81:0/4238544798 >> v1:172.21.15.93:6789/0 conn(0x7f34280cab40 legacy=0x7f342806b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.294+0000 7f342dcf5640 1 -- 172.21.15.81:0/4238544798 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34280caf20 con 0x7f34280cd040 2023-11-08T16:37:16.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.296+0000 7f342dcf5640 1 --2- 172.21.15.81:0/4238544798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34280cd040 0x7f34280b04e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f34180098a0 tx=0x7f341802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93d491fa825f293 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:16.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.296+0000 7f342d4f4640 1 -- 172.21.15.81:0/4238544798 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34180317c0 con 0x7f34280cd040 2023-11-08T16:37:16.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.296+0000 7f342d4f4640 1 -- 172.21.15.81:0/4238544798 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3418031920 con 0x7f34280cd040 2023-11-08T16:37:16.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.296+0000 7f342d4f4640 1 -- 172.21.15.81:0/4238544798 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3418031cb0 con 0x7f34280cd040 2023-11-08T16:37:16.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.297+0000 7f343075a640 1 -- 172.21.15.81:0/4238544798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34280cd040 msgr2=0x7f34280b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.297+0000 7f343075a640 1 --2- 172.21.15.81:0/4238544798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34280cd040 0x7f34280b04e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f34180098a0 tx=0x7f341802b860 comp rx=0 tx=0).stop 2023-11-08T16:37:16.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.297+0000 7f343075a640 1 -- 172.21.15.81:0/4238544798 shutdown_connections 2023-11-08T16:37:16.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.297+0000 7f343075a640 1 --2- 172.21.15.81:0/4238544798 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34280b0a20 0x7f3428160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.297+0000 7f343075a640 1 --2- 172.21.15.81:0/4238544798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34280cd040 0x7f34280b04e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.297+0000 7f343075a640 1 -- 172.21.15.81:0/4238544798 >> 172.21.15.81:0/4238544798 conn(0x7f34280cb4e0 msgr2=0x7f34280cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:16.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.298+0000 7f343075a640 1 -- 172.21.15.81:0/4238544798 shutdown_connections 2023-11-08T16:37:16.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.298+0000 7f343075a640 1 -- 172.21.15.81:0/4238544798 wait complete. 2023-11-08T16:37:16.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.300+0000 7f343075a640 1 Processor -- start 2023-11-08T16:37:16.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.300+0000 7f343075a640 1 -- start start 2023-11-08T16:37:16.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f343075a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34280b0a20 0x7f34280b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f343075a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34280cab40 0x7f34281490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f343075a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34280cd040 0x7f342806fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f343075a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3428150c60 con 0x7f34280b0a20 2023-11-08T16:37:16.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f343075a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34280c71b0 con 0x7f34280cab40 2023-11-08T16:37:16.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f343075a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34281514c0 con 0x7f34280cd040 2023-11-08T16:37:16.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f342e4f6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34280b0a20 0x7f34280b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f342e4f6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34280b0a20 0x7f34280b3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36018/0 (socket says 172.21.15.81:36018) 2023-11-08T16:37:16.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f342e4f6640 1 -- 172.21.15.81:0/3620720681 learned_addr learned my addr 172.21.15.81:0/3620720681 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:16.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f342dcf5640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34280cab40 0x7f34281490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f342ecf7640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34280cd040 0x7f342806fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f342e4f6640 1 -- 172.21.15.81:0/3620720681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34280cd040 msgr2=0x7f342806fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f342e4f6640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34280cd040 0x7f342806fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f342e4f6640 1 -- 172.21.15.81:0/3620720681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34280cab40 msgr2=0x7f34281490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f342e4f6640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34280cab40 0x7f34281490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f342e4f6640 1 -- 172.21.15.81:0/3620720681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34180094d0 con 0x7f34280b0a20 2023-11-08T16:37:16.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.301+0000 7f342dcf5640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34280cab40 0x7f34281490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:16.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.302+0000 7f342e4f6640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34280b0a20 0x7f34280b3b90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f342400c500 tx=0x7f342400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:16.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.302+0000 7f341effd640 1 -- 172.21.15.81:0/3620720681 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3424007500 con 0x7f34280b0a20 2023-11-08T16:37:16.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.302+0000 7f341effd640 1 -- 172.21.15.81:0/3620720681 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3424007660 con 0x7f34280b0a20 2023-11-08T16:37:16.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.302+0000 7f341effd640 1 -- 172.21.15.81:0/3620720681 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34240049e0 con 0x7f34280b0a20 2023-11-08T16:37:16.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.302+0000 7f343075a640 1 -- 172.21.15.81:0/3620720681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34280b4170 con 0x7f34280b0a20 2023-11-08T16:37:16.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.302+0000 7f343075a640 1 -- 172.21.15.81:0/3620720681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3428151150 con 0x7f34280b0a20 2023-11-08T16:37:16.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.304+0000 7f343075a640 1 -- 172.21.15.81:0/3620720681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f342804ece0 con 0x7f34280b0a20 2023-11-08T16:37:16.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.304+0000 7f341effd640 1 -- 172.21.15.81:0/3620720681 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3424004c00 con 0x7f34280b0a20 2023-11-08T16:37:16.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.306+0000 7f341effd640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f340c06ae70 0x7f340c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.306+0000 7f341effd640 1 -- 172.21.15.81:0/3620720681 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f3424088480 con 0x7f34280b0a20 2023-11-08T16:37:16.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.306+0000 7f342dcf5640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f340c06ae70 0x7f340c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.307+0000 7f342dcf5640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f340c06ae70 0x7f340c06d330 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3418036000 tx=0x7f341803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:16.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.310+0000 7f341effd640 1 -- 172.21.15.81:0/3620720681 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3424053860 con 0x7f34280b0a20 2023-11-08T16:37:16.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.501+0000 7f343075a640 1 -- 172.21.15.81:0/3620720681 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f34280ae060 con 0x7f34280b0a20 2023-11-08T16:37:16.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.506+0000 7f341effd640 1 -- 172.21.15.81:0/3620720681 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3424056e80 con 0x7f34280b0a20 2023-11-08T16:37:16.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 -- 172.21.15.81:0/3620720681 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f340c06ae70 msgr2=0x7f340c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f340c06ae70 0x7f340c06d330 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3418036000 tx=0x7f341803d040 comp rx=0 tx=0).stop 2023-11-08T16:37:16.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 -- 172.21.15.81:0/3620720681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34280b0a20 msgr2=0x7f34280b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34280b0a20 0x7f34280b3b90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f342400c500 tx=0x7f342400c9c0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 -- 172.21.15.81:0/3620720681 shutdown_connections 2023-11-08T16:37:16.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f340c06ae70 0x7f340c06d330 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34280cd040 0x7f342806fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34280cab40 0x7f34281490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 --2- 172.21.15.81:0/3620720681 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f34280b0a20 0x7f34280b3b90 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 -- 172.21.15.81:0/3620720681 >> 172.21.15.81:0/3620720681 conn(0x7f34280cb4e0 msgr2=0x7f342815eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:16.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 -- 172.21.15.81:0/3620720681 shutdown_connections 2023-11-08T16:37:16.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.509+0000 7f343075a640 1 -- 172.21.15.81:0/3620720681 wait complete. 2023-11-08T16:37:16.529 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers' 2023-11-08T16:37:16.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.743+0000 7fe6b92fc640 1 Processor -- start 2023-11-08T16:37:16.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.743+0000 7fe6b92fc640 1 -- start start 2023-11-08T16:37:16.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.744+0000 7fe6b92fc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe6b40b1da0 0x7fe6b40d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.744+0000 7fe6b92fc640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b415d680 con 0x7fe6b40cea60 2023-11-08T16:37:16.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.744+0000 7fe6b92fc640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b40cdca0 con 0x7fe6b40cf850 2023-11-08T16:37:16.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.744+0000 7fe6b92fc640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b40cdf80 con 0x7fe6b40b1da0 2023-11-08T16:37:16.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.745+0000 7fe6b37fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe6b40cea60 0x7fe6b40b27a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34092/0 (socket says 172.21.15.81:34092) 2023-11-08T16:37:16.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.745+0000 7fe6b37fe640 1 -- 172.21.15.81:0/1298709994 learned_addr learned my addr 172.21.15.81:0/1298709994 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:16.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.745+0000 7fe6b27fc640 1 --2- 172.21.15.81:0/1298709994 >> v2:172.21.15.93:3300/0 conn(0x7fe6b40b1da0 0x7fe6b40d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.745+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619103428 0 0) 0x7fe6b415d680 con 0x7fe6b40cea60 2023-11-08T16:37:16.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.745+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe694003520 con 0x7fe6b40cea60 2023-11-08T16:37:16.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 16869129 0 0) 0x7fe694003520 con 0x7fe6b40cea60 2023-11-08T16:37:16.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 >> v2:172.21.15.93:3300/0 conn(0x7fe6b40b1da0 msgr2=0x7fe6b40d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:16.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 --2- 172.21.15.81:0/1298709994 >> v2:172.21.15.93:3300/0 conn(0x7fe6b40b1da0 0x7fe6b40d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 >> v1:172.21.15.93:6789/0 conn(0x7fe6b40cf850 legacy=0x7fe6b40cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40ce260 con 0x7fe6b40cea60 2023-11-08T16:37:16.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fe6ac001670 con 0x7fe6b40cea60 2023-11-08T16:37:16.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 >> v1:172.21.15.81:6789/0 conn(0x7fe6b40cea60 legacy=0x7fe6b40b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 --2- 172.21.15.81:0/1298709994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe694003ad0 0x7fe694003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 --2- 172.21.15.81:0/1298709994 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6940047c0 0x7fe694004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 --2- 172.21.15.81:0/1298709994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6940052a0 0x7fe69400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40cdf80 con 0x7fe694003ad0 2023-11-08T16:37:16.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69400c170 con 0x7fe6940052a0 2023-11-08T16:37:16.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe69400c330 con 0x7fe6940047c0 2023-11-08T16:37:16.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.746+0000 7fe6b2ffd640 1 --2- 172.21.15.81:0/1298709994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe694003ad0 0x7fe694003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.747+0000 7fe6b27fc640 1 --2- 172.21.15.81:0/1298709994 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6940047c0 0x7fe694004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.747+0000 7fe6b2ffd640 1 -- 172.21.15.81:0/1298709994 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6940047c0 msgr2=0x7fe694004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:16.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.747+0000 7fe6b2ffd640 1 --2- 172.21.15.81:0/1298709994 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6940047c0 0x7fe694004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.747+0000 7fe6b37fe640 1 --2- 172.21.15.81:0/1298709994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6940052a0 0x7fe69400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.747+0000 7fe6b2ffd640 1 -- 172.21.15.81:0/1298709994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6940052a0 msgr2=0x7fe69400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:16.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.747+0000 7fe6b2ffd640 1 --2- 172.21.15.81:0/1298709994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe6940052a0 0x7fe69400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.747+0000 7fe6b37fe640 1 -- 172.21.15.81:0/1298709994 reap_dead start 2023-11-08T16:37:16.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.747+0000 7fe6b2ffd640 1 -- 172.21.15.81:0/1298709994 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe694003520 con 0x7fe694003ad0 2023-11-08T16:37:16.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.749+0000 7fe6b2ffd640 1 --2- 172.21.15.81:0/1298709994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe694003ad0 0x7fe694003ef0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4001750 tx=0x7fe6940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968d65cfdffb4419 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.749+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40052d0 con 0x7fe694003ad0 2023-11-08T16:37:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.749+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a402e050 con 0x7fe694003ad0 2023-11-08T16:37:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.749+0000 7fe6b1ffb640 1 -- 172.21.15.81:0/1298709994 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4003830 con 0x7fe694003ad0 2023-11-08T16:37:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.749+0000 7fe6b92fc640 1 -- 172.21.15.81:0/1298709994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe694003ad0 msgr2=0x7fe694003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.749+0000 7fe6b92fc640 1 --2- 172.21.15.81:0/1298709994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe694003ad0 0x7fe694003ef0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4001750 tx=0x7fe6940057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.750+0000 7fe6b92fc640 1 -- 172.21.15.81:0/1298709994 shutdown_connections 2023-11-08T16:37:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.750+0000 7fe6b92fc640 1 --2- 172.21.15.81:0/1298709994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe694003ad0 0x7fe694003ef0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.750+0000 7fe6b92fc640 1 -- 172.21.15.81:0/1298709994 >> 172.21.15.81:0/1298709994 conn(0x7fe6b40d2e40 msgr2=0x7fe6b40d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:16.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.750+0000 7fe6b92fc640 1 -- 172.21.15.81:0/1298709994 shutdown_connections 2023-11-08T16:37:16.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.750+0000 7fe6b92fc640 1 -- 172.21.15.81:0/1298709994 wait complete. 2023-11-08T16:37:16.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.751+0000 7fe6b92fc640 1 Processor -- start 2023-11-08T16:37:16.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.752+0000 7fe6b92fc640 1 -- start start 2023-11-08T16:37:16.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b92fc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe694003ad0 0x7fe6b41f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b92fc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6b415d870 0x7fe6b40b0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b92fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6b40b0560 0x7fe6b4160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b92fc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40b09e0 con 0x7fe6b40b0560 2023-11-08T16:37:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b92fc640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b41f3c90 con 0x7fe694003ad0 2023-11-08T16:37:16.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b92fc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b41f32c0 con 0x7fe6b415d870 2023-11-08T16:37:16.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6b40b0560 0x7fe6b4160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6b40b0560 0x7fe6b4160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36036/0 (socket says 172.21.15.81:36036) 2023-11-08T16:37:16.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b37fe640 1 -- 172.21.15.81:0/2025225312 learned_addr learned my addr 172.21.15.81:0/2025225312 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:16.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b2ffd640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe694003ad0 0x7fe6b41f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b37fe640 1 -- 172.21.15.81:0/2025225312 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6b415d870 msgr2=0x7fe6b40b0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b37fe640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6b415d870 0x7fe6b40b0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b37fe640 1 -- 172.21.15.81:0/2025225312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe694003ad0 msgr2=0x7fe6b41f3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b37fe640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe694003ad0 0x7fe6b41f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.753+0000 7fe6b37fe640 1 -- 172.21.15.81:0/2025225312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b415e0c0 con 0x7fe6b40b0560 2023-11-08T16:37:16.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.754+0000 7fe6b37fe640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6b40b0560 0x7fe6b4160000 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe6b408d350 tx=0x7fe6ac00d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.754+0000 7fe6ab7fe640 1 -- 172.21.15.81:0/2025225312 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac007500 con 0x7fe6b40b0560 2023-11-08T16:37:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.754+0000 7fe6ab7fe640 1 -- 172.21.15.81:0/2025225312 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac007660 con 0x7fe6b40b0560 2023-11-08T16:37:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.754+0000 7fe6ab7fe640 1 -- 172.21.15.81:0/2025225312 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac004a20 con 0x7fe6b40b0560 2023-11-08T16:37:16.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.754+0000 7fe6b92fc640 1 -- 172.21.15.81:0/2025225312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b415e3a0 con 0x7fe6b40b0560 2023-11-08T16:37:16.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.754+0000 7fe6b92fc640 1 -- 172.21.15.81:0/2025225312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b41f2c40 con 0x7fe6b40b0560 2023-11-08T16:37:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.755+0000 7fe6b92fc640 1 -- 172.21.15.81:0/2025225312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b404ece0 con 0x7fe6b40b0560 2023-11-08T16:37:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.756+0000 7fe6ab7fe640 1 -- 172.21.15.81:0/2025225312 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe6ac004b80 con 0x7fe6b40b0560 2023-11-08T16:37:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.757+0000 7fe6ab7fe640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe69806ae70 0x7fe69806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:16.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.758+0000 7fe6ab7fe640 1 -- 172.21.15.81:0/2025225312 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5201+0+0 (secure 0 0 0) 0x7fe6ac0889a0 con 0x7fe6b40b0560 2023-11-08T16:37:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.758+0000 7fe6b2ffd640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe69806ae70 0x7fe69806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.758+0000 7fe6b2ffd640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe69806ae70 0x7fe69806d330 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4033000 tx=0x7fe6a4009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:16.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:16.762+0000 7fe6ab7fe640 1 -- 172.21.15.81:0/2025225312 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe6ac057760 con 0x7fe6b40b0560 2023-11-08T16:37:17.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.022+0000 7fe6b92fc640 1 -- 172.21.15.81:0/2025225312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]} v 0) v1 -- 0x7fe6b41f3400 con 0x7fe6b40b0560 2023-11-08T16:37:17.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.048+0000 7fe6ab7fe640 1 -- 172.21.15.81:0/2025225312 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_many_files_many_workers"]}]=0 v362) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe6ac0572f0 con 0x7fe6b40b0560 2023-11-08T16:37:17.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.050+0000 7fe6b92fc640 1 -- 172.21.15.81:0/2025225312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe69806ae70 msgr2=0x7fe69806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.050+0000 7fe6b92fc640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe69806ae70 0x7fe69806d330 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4033000 tx=0x7fe6a4009030 comp rx=0 tx=0).stop 2023-11-08T16:37:17.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.050+0000 7fe6b92fc640 1 -- 172.21.15.81:0/2025225312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6b40b0560 msgr2=0x7fe6b4160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.050+0000 7fe6b92fc640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6b40b0560 0x7fe6b4160000 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fe6b408d350 tx=0x7fe6ac00d880 comp rx=0 tx=0).stop 2023-11-08T16:37:17.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.051+0000 7fe6b92fc640 1 -- 172.21.15.81:0/2025225312 shutdown_connections 2023-11-08T16:37:17.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.051+0000 7fe6b92fc640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe69806ae70 0x7fe69806d330 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.051+0000 7fe6b92fc640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6b40b0560 0x7fe6b4160000 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.051+0000 7fe6b92fc640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6b415d870 0x7fe6b40b0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.051+0000 7fe6b92fc640 1 --2- 172.21.15.81:0/2025225312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe694003ad0 0x7fe6b41f3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.051+0000 7fe6b92fc640 1 -- 172.21.15.81:0/2025225312 >> 172.21.15.81:0/2025225312 conn(0x7fe6b40d2e40 msgr2=0x7fe6b4073130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:17.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.051+0000 7fe6b92fc640 1 -- 172.21.15.81:0/2025225312 shutdown_connections 2023-11-08T16:37:17.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.051+0000 7fe6b92fc640 1 -- 172.21.15.81:0/2025225312 wait complete. 2023-11-08T16:37:17.072 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_many_files_many_workers (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-11-08T16:37:17.073 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_rebuild_simple (tasks.cephfs.test_data_scan.TestDataScan) 2023-11-08T16:37:17.074 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:17.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.286+0000 7fbd72c26640 1 Processor -- start 2023-11-08T16:37:17.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.286+0000 7fbd72c26640 1 -- start start 2023-11-08T16:37:17.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.288+0000 7fbd72c26640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 0x7fbd6c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.288+0000 7fbd72c26640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbd6c15a680 0x7fbd6c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.288+0000 7fbd72c26640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbd6c15b020 0x7fbd6c0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.288+0000 7fbd72c26640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c0b1860 con 0x7fbd6c15a680 2023-11-08T16:37:17.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.288+0000 7fbd72c26640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd6c0ca9c0 con 0x7fbd6c15b020 2023-11-08T16:37:17.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.288+0000 7fbd72c26640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c06b630 con 0x7fbd6c0ade50 2023-11-08T16:37:17.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.288+0000 7fbd6bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbd6c15a680 0x7fbd6c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.288+0000 7fbd6bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbd6c15a680 0x7fbd6c15aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36044/0 (socket says 172.21.15.81:36044) 2023-11-08T16:37:17.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.288+0000 7fbd6bfff640 1 -- 172.21.15.81:0/1941399691 learned_addr learned my addr 172.21.15.81:0/1941399691 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:17.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.289+0000 7fbd711c3640 1 --2- 172.21.15.81:0/1941399691 >> v2:172.21.15.93:3300/0 conn(0x7fbd6c15b020 0x7fbd6c0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.289+0000 7fbd709c2640 1 --2- 172.21.15.81:0/1941399691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 0x7fbd6c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.289+0000 7fbd6bfff640 1 -- 172.21.15.81:0/1941399691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 msgr2=0x7fbd6c0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:17.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.289+0000 7fbd6bfff640 1 --2- 172.21.15.81:0/1941399691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 0x7fbd6c0ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.289+0000 7fbd6bfff640 1 -- 172.21.15.81:0/1941399691 >> v2:172.21.15.93:3300/0 conn(0x7fbd6c15b020 msgr2=0x7fbd6c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:17.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.289+0000 7fbd6bfff640 1 --2- 172.21.15.81:0/1941399691 >> v2:172.21.15.93:3300/0 conn(0x7fbd6c15b020 0x7fbd6c0c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.289+0000 7fbd6bfff640 1 -- 172.21.15.81:0/1941399691 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c0cabb0 con 0x7fbd6c15a680 2023-11-08T16:37:17.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.289+0000 7fbd709c2640 1 --2- 172.21.15.81:0/1941399691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 0x7fbd6c0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:17.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.290+0000 7fbd6bfff640 1 --2- 172.21.15.81:0/1941399691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c15a680 0x7fbd6c15aae0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c06f840 tx=0x7fbd5802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fee4646a8e1cd3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:17.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.290+0000 7fbd6b7fe640 1 -- 172.21.15.81:0/1941399691 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd58031870 con 0x7fbd6c15a680 2023-11-08T16:37:17.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.290+0000 7fbd6b7fe640 1 -- 172.21.15.81:0/1941399691 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd580319d0 con 0x7fbd6c15a680 2023-11-08T16:37:17.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.290+0000 7fbd6b7fe640 1 -- 172.21.15.81:0/1941399691 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd58031d60 con 0x7fbd6c15a680 2023-11-08T16:37:17.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.291+0000 7fbd72c26640 1 -- 172.21.15.81:0/1941399691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c15a680 msgr2=0x7fbd6c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.291+0000 7fbd72c26640 1 --2- 172.21.15.81:0/1941399691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c15a680 0x7fbd6c15aae0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c06f840 tx=0x7fbd5802b910 comp rx=0 tx=0).stop 2023-11-08T16:37:17.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.291+0000 7fbd72c26640 1 -- 172.21.15.81:0/1941399691 shutdown_connections 2023-11-08T16:37:17.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.291+0000 7fbd72c26640 1 --2- 172.21.15.81:0/1941399691 >> v2:172.21.15.93:3300/0 conn(0x7fbd6c15b020 0x7fbd6c0c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.291+0000 7fbd72c26640 1 --2- 172.21.15.81:0/1941399691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c15a680 0x7fbd6c15aae0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.291+0000 7fbd72c26640 1 --2- 172.21.15.81:0/1941399691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 0x7fbd6c0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.291+0000 7fbd72c26640 1 -- 172.21.15.81:0/1941399691 >> 172.21.15.81:0/1941399691 conn(0x7fbd6c0cb490 msgr2=0x7fbd6c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.291+0000 7fbd72c26640 1 -- 172.21.15.81:0/1941399691 shutdown_connections 2023-11-08T16:37:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.291+0000 7fbd72c26640 1 -- 172.21.15.81:0/1941399691 wait complete. 2023-11-08T16:37:17.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.293+0000 7fbd72c26640 1 Processor -- start 2023-11-08T16:37:17.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.293+0000 7fbd72c26640 1 -- start start 2023-11-08T16:37:17.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.294+0000 7fbd72c26640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 0x7fbd6c1514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.294+0000 7fbd72c26640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c15a680 0x7fbd6c15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.294+0000 7fbd72c26640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd6c15b020 0x7fbd6c15dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.294+0000 7fbd72c26640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c0cc780 con 0x7fbd6c15a680 2023-11-08T16:37:17.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.294+0000 7fbd72c26640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c152220 con 0x7fbd6c15b020 2023-11-08T16:37:17.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.294+0000 7fbd72c26640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6c151f40 con 0x7fbd6c0ade50 2023-11-08T16:37:17.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.294+0000 7fbd6bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c15a680 0x7fbd6c15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.294+0000 7fbd6bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c15a680 0x7fbd6c15d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36058/0 (socket says 172.21.15.81:36058) 2023-11-08T16:37:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.294+0000 7fbd6bfff640 1 -- 172.21.15.81:0/1537027933 learned_addr learned my addr 172.21.15.81:0/1537027933 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd711c3640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd6c15b020 0x7fbd6c15dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd709c2640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 0x7fbd6c1514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd6bfff640 1 -- 172.21.15.81:0/1537027933 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 msgr2=0x7fbd6c1514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd6bfff640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 0x7fbd6c1514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd6bfff640 1 -- 172.21.15.81:0/1537027933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd6c15b020 msgr2=0x7fbd6c15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd6bfff640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd6c15b020 0x7fbd6c15dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd6bfff640 1 -- 172.21.15.81:0/1537027933 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd580094d0 con 0x7fbd6c15a680 2023-11-08T16:37:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd711c3640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd6c15b020 0x7fbd6c15dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:17.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd709c2640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 0x7fbd6c1514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:17.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd6bfff640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c15a680 0x7fbd6c15d710 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbd58000c00 tx=0x7fbd580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:17.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd697fa640 1 -- 172.21.15.81:0/1537027933 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5803abe0 con 0x7fbd6c15a680 2023-11-08T16:37:17.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.295+0000 7fbd697fa640 1 -- 172.21.15.81:0/1537027933 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5803ad40 con 0x7fbd6c15a680 2023-11-08T16:37:17.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.296+0000 7fbd697fa640 1 -- 172.21.15.81:0/1537027933 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd58044610 con 0x7fbd6c15a680 2023-11-08T16:37:17.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.296+0000 7fbd72c26640 1 -- 172.21.15.81:0/1537027933 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd6c0cca20 con 0x7fbd6c15a680 2023-11-08T16:37:17.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.296+0000 7fbd72c26640 1 -- 172.21.15.81:0/1537027933 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd6c150b20 con 0x7fbd6c15a680 2023-11-08T16:37:17.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.297+0000 7fbd72c26640 1 -- 172.21.15.81:0/1537027933 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6c0cc5e0 con 0x7fbd6c15a680 2023-11-08T16:37:17.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.298+0000 7fbd697fa640 1 -- 172.21.15.81:0/1537027933 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd5804d450 con 0x7fbd6c15a680 2023-11-08T16:37:17.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.299+0000 7fbd697fa640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbd5006ae70 0x7fbd5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.299+0000 7fbd697fa640 1 -- 172.21.15.81:0/1537027933 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5201+0+0 (secure 0 0 0) 0x7fbd5802d040 con 0x7fbd6c15a680 2023-11-08T16:37:17.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.299+0000 7fbd709c2640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbd5006ae70 0x7fbd5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.299+0000 7fbd709c2640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbd5006ae70 0x7fbd5006d330 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbd600088b0 tx=0x7fbd60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:17.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.303+0000 7fbd697fa640 1 -- 172.21.15.81:0/1537027933 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbd6c0cc5e0 con 0x7fbd6c15a680 2023-11-08T16:37:17.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.492+0000 7fbd72c26640 1 -- 172.21.15.81:0/1537027933 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd6c0cc5e0 con 0x7fbd6c15a680 2023-11-08T16:37:17.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.495+0000 7fbd697fa640 1 -- 172.21.15.81:0/1537027933 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+13310 (secure 0 0 0) 0x7fbd6c0cc5e0 con 0x7fbd6c15a680 2023-11-08T16:37:17.497 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:17.497 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":72,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:37:16.031499+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/552116415":"2023-11-09T16:37:11.932319+0000","172.21.15.81:6835/552116415":"2023-11-09T16:37:11.932319+0000","172.21.15.81:6837/321347143":"2023-11-09T16:34:45.940255+0000","172.21.15.81:6837/1446454901":"2023-11-09T16:34:43.853227+0000","172.21.15.81:6837/494395478":"2023-11-09T16:34:53.172859+0000","172.21.15.81:6835/494395478":"2023-11-09T16:34:53.172859+0000","172.21.15.81:6835/321347143":"2023-11-09T16:34:45.940255+0000","172.21.15.81:6835/1446454901":"2023-11-09T16:34:43.853227+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:17.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.498+0000 7fbd72c26640 1 -- 172.21.15.81:0/1537027933 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbd5006ae70 msgr2=0x7fbd5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.498+0000 7fbd72c26640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbd5006ae70 0x7fbd5006d330 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbd600088b0 tx=0x7fbd60008040 comp rx=0 tx=0).stop 2023-11-08T16:37:17.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.498+0000 7fbd72c26640 1 -- 172.21.15.81:0/1537027933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c15a680 msgr2=0x7fbd6c15d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.498+0000 7fbd72c26640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c15a680 0x7fbd6c15d710 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbd58000c00 tx=0x7fbd580046c0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.498+0000 7fbd72c26640 1 -- 172.21.15.81:0/1537027933 shutdown_connections 2023-11-08T16:37:17.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.498+0000 7fbd72c26640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbd5006ae70 0x7fbd5006d330 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.498+0000 7fbd72c26640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd6c15b020 0x7fbd6c15dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.498+0000 7fbd72c26640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbd6c15a680 0x7fbd6c15d710 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.498+0000 7fbd72c26640 1 --2- 172.21.15.81:0/1537027933 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbd6c0ade50 0x7fbd6c1514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.498+0000 7fbd72c26640 1 -- 172.21.15.81:0/1537027933 >> 172.21.15.81:0/1537027933 conn(0x7fbd6c0cb490 msgr2=0x7fbd6c0af170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:17.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.499+0000 7fbd72c26640 1 -- 172.21.15.81:0/1537027933 shutdown_connections 2023-11-08T16:37:17.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.499+0000 7fbd72c26640 1 -- 172.21.15.81:0/1537027933 wait complete. 2023-11-08T16:37:17.520 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:37:17.521 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:37:17.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.734+0000 7fe4a9eaf640 1 Processor -- start 2023-11-08T16:37:17.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.734+0000 7fe4a9eaf640 1 -- start start 2023-11-08T16:37:17.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.735+0000 7fe4a9eaf640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a40ccc10 0x7fe4a40ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.736+0000 7fe4a9eaf640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a40ce960 con 0x7fe4a406b710 2023-11-08T16:37:17.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.736+0000 7fe4a9eaf640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a40c9480 con 0x7fe4a415a0e0 2023-11-08T16:37:17.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.736+0000 7fe4a9eaf640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a4071980 con 0x7fe4a40ccc10 2023-11-08T16:37:17.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.736+0000 7fe4a3fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe4a406b710 0x7fe4a406bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34106/0 (socket says 172.21.15.81:34106) 2023-11-08T16:37:17.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.736+0000 7fe4a3fff640 1 -- 172.21.15.81:0/266958797 learned_addr learned my addr 172.21.15.81:0/266958797 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:17.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.736+0000 7fe4a2ffd640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a40ccc10 0x7fe4a40ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.736+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1776094151 0 0) 0x7fe49c001490 con 0x7fe4a406b710 2023-11-08T16:37:17.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe48c005730 con 0x7fe4a406b710 2023-11-08T16:37:17.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 241833466 0 0) 0x7fe48c005730 con 0x7fe4a406b710 2023-11-08T16:37:17.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a40ccc10 msgr2=0x7fe4a40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:17.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a40ccc10 0x7fe4a40ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 >> v1:172.21.15.93:6789/0 conn(0x7fe4a415a0e0 legacy=0x7fe4a40c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a415a4c0 con 0x7fe4a406b710 2023-11-08T16:37:17.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fe49c001b80 con 0x7fe4a406b710 2023-11-08T16:37:17.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 >> v1:172.21.15.81:6789/0 conn(0x7fe4a406b710 legacy=0x7fe4a406bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 0x7fe48c006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a40ccc10 0x7fe48c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe48c006f10 0x7fe48c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a4071980 con 0x7fe48c005c20 2023-11-08T16:37:17.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c00ddb0 con 0x7fe48c006f10 2023-11-08T16:37:17.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48c00df40 con 0x7fe4a40ccc10 2023-11-08T16:37:17.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.737+0000 7fe4a37fe640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 0x7fe48c006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.738+0000 7fe4a2ffd640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a40ccc10 0x7fe48c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.738+0000 7fe4a3fff640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe48c006f10 0x7fe48c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.738+0000 7fe4a37fe640 1 -- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a40ccc10 msgr2=0x7fe48c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:17.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.738+0000 7fe4a37fe640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a40ccc10 0x7fe48c0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.738+0000 7fe4a37fe640 1 -- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe48c006f10 msgr2=0x7fe48c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:17.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.738+0000 7fe4a37fe640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe48c006f10 0x7fe48c007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.738+0000 7fe4a37fe640 1 -- 172.21.15.81:0/266958797 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48c005730 con 0x7fe48c005c20 2023-11-08T16:37:17.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.738+0000 7fe4a2ffd640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a40ccc10 0x7fe48c0069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:17.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.738+0000 7fe4a3fff640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe48c006f10 0x7fe48c007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:17.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.739+0000 7fe4a37fe640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 0x7fe48c006040 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe494001750 tx=0x7fe48c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fdc99bc7876e2d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:17.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.739+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940052d0 con 0x7fe48c005c20 2023-11-08T16:37:17.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.739+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49402e050 con 0x7fe48c005c20 2023-11-08T16:37:17.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.739+0000 7fe4a27fc640 1 -- 172.21.15.81:0/266958797 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940035f0 con 0x7fe48c005c20 2023-11-08T16:37:17.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.740+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/266958797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 msgr2=0x7fe48c006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.740+0000 7fe4a9eaf640 1 --2- 172.21.15.81:0/266958797 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 0x7fe48c006040 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe494001750 tx=0x7fe48c007a00 comp rx=0 tx=0).stop 2023-11-08T16:37:17.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.740+0000 7fe4a3fff640 1 -- 172.21.15.81:0/266958797 reap_dead start 2023-11-08T16:37:17.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.740+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/266958797 shutdown_connections 2023-11-08T16:37:17.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.740+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/266958797 >> 172.21.15.81:0/266958797 conn(0x7fe4a40cb580 msgr2=0x7fe4a40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:17.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.740+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/266958797 shutdown_connections 2023-11-08T16:37:17.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.740+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/266958797 wait complete. 2023-11-08T16:37:17.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.742+0000 7fe4a9eaf640 1 Processor -- start 2023-11-08T16:37:17.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.742+0000 7fe4a9eaf640 1 -- start start 2023-11-08T16:37:17.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a9eaf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 0x7fe4a40c30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a9eaf640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a415b7e0 0x7fe4a40c7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a9eaf640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4a415cd80 0x7fe4a41f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a9eaf640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a415bce0 con 0x7fe48c005c20 2023-11-08T16:37:17.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a9eaf640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a415bfc0 con 0x7fe4a415cd80 2023-11-08T16:37:17.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a9eaf640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a415c2a0 con 0x7fe4a415b7e0 2023-11-08T16:37:17.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 0x7fe4a40c30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 0x7fe4a40c30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36084/0 (socket says 172.21.15.81:36084) 2023-11-08T16:37:17.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a37fe640 1 -- 172.21.15.81:0/614935248 learned_addr learned my addr 172.21.15.81:0/614935248 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:17.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a37fe640 1 -- 172.21.15.81:0/614935248 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a415b7e0 msgr2=0x7fe4a40c7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a37fe640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a415b7e0 0x7fe4a40c7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a37fe640 1 -- 172.21.15.81:0/614935248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4a415cd80 msgr2=0x7fe4a41f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a37fe640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4a415cd80 0x7fe4a41f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.743+0000 7fe4a37fe640 1 -- 172.21.15.81:0/614935248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a415c580 con 0x7fe48c005c20 2023-11-08T16:37:17.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.744+0000 7fe4a37fe640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 0x7fe4a40c30f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe49402bbd0 tx=0x7fe494001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:17.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.744+0000 7fe48bfff640 1 -- 172.21.15.81:0/614935248 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940076c0 con 0x7fe48c005c20 2023-11-08T16:37:17.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.744+0000 7fe48bfff640 1 -- 172.21.15.81:0/614935248 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4940069e0 con 0x7fe48c005c20 2023-11-08T16:37:17.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.744+0000 7fe48bfff640 1 -- 172.21.15.81:0/614935248 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494006d50 con 0x7fe48c005c20 2023-11-08T16:37:17.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.744+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/614935248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a415c860 con 0x7fe48c005c20 2023-11-08T16:37:17.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.744+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/614935248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a40c7970 con 0x7fe48c005c20 2023-11-08T16:37:17.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.745+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/614935248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a4071440 con 0x7fe48c005c20 2023-11-08T16:37:17.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.746+0000 7fe48bfff640 1 -- 172.21.15.81:0/614935248 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe4940035f0 con 0x7fe48c005c20 2023-11-08T16:37:17.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.747+0000 7fe48bfff640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe49006ae70 0x7fe49006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:17.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.747+0000 7fe48bfff640 1 -- 172.21.15.81:0/614935248 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5201+0+0 (secure 0 0 0) 0x7fe4940ab310 con 0x7fe48c005c20 2023-11-08T16:37:17.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.747+0000 7fe4a2ffd640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe49006ae70 0x7fe49006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:17.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.748+0000 7fe4a2ffd640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe49006ae70 0x7fe49006d330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe4a41f6010 tx=0x7fe498008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:17.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.752+0000 7fe48bfff640 1 -- 172.21.15.81:0/614935248 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe4a4071440 con 0x7fe48c005c20 2023-11-08T16:37:17.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.939+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/614935248 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4a4071440 con 0x7fe48c005c20 2023-11-08T16:37:17.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.940+0000 7fe48bfff640 1 -- 172.21.15.81:0/614935248 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4a4071440 con 0x7fe48c005c20 2023-11-08T16:37:17.942 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:37:17.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/614935248 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe49006ae70 msgr2=0x7fe49006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe49006ae70 0x7fe49006d330 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe4a41f6010 tx=0x7fe498008040 comp rx=0 tx=0).stop 2023-11-08T16:37:17.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/614935248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 msgr2=0x7fe4a40c30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:17.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 0x7fe4a40c30f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fe49402bbd0 tx=0x7fe494001120 comp rx=0 tx=0).stop 2023-11-08T16:37:17.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/614935248 shutdown_connections 2023-11-08T16:37:17.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe49006ae70 0x7fe49006d330 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4a415cd80 0x7fe4a41f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe4a415b7e0 0x7fe4a40c7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 --2- 172.21.15.81:0/614935248 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe48c005c20 0x7fe4a40c30f0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:17.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/614935248 >> 172.21.15.81:0/614935248 conn(0x7fe4a40cb580 msgr2=0x7fe4a406bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:17.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/614935248 shutdown_connections 2023-11-08T16:37:17.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:17.943+0000 7fe4a9eaf640 1 -- 172.21.15.81:0/614935248 wait complete. 2023-11-08T16:37:17.964 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple' 2023-11-08T16:37:18.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.171+0000 7f461f354640 1 Processor -- start 2023-11-08T16:37:18.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.171+0000 7f461f354640 1 -- start start 2023-11-08T16:37:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.172+0000 7f461f354640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f46180cf170 0x7f461815b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:18.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.173+0000 7f461f354640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4618070470 con 0x7f461815aac0 2023-11-08T16:37:18.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.173+0000 7f461f354640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461806d580 con 0x7f46180ced10 2023-11-08T16:37:18.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.173+0000 7f461f354640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f461806f650 con 0x7f46180cf170 2023-11-08T16:37:18.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.173+0000 7f461eb53640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f46180cf170 0x7f461815b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:18.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.173+0000 7f461e352640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f461815aac0 0x7f46180711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34116/0 (socket says 172.21.15.81:34116) 2023-11-08T16:37:18.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.173+0000 7f461eb53640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f46180cf170 0x7f461815b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36094/0 (socket says 172.21.15.81:36094) 2023-11-08T16:37:18.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.173+0000 7f461e352640 1 -- 172.21.15.81:0/104393631 learned_addr learned my addr 172.21.15.81:0/104393631 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:18.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.173+0000 7f461eb53640 1 -- 172.21.15.81:0/104393631 >> v1:172.21.15.93:6789/0 conn(0x7f46180ced10 legacy=0x7f461815aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:18.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.173+0000 7f461eb53640 1 -- 172.21.15.81:0/104393631 >> v1:172.21.15.81:6789/0 conn(0x7f461815aac0 legacy=0x7f46180711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:18.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.173+0000 7f461eb53640 1 -- 172.21.15.81:0/104393631 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461806ca00 con 0x7f46180cf170 2023-11-08T16:37:18.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.173+0000 7f461d350640 1 -- 172.21.15.81:0/104393631 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1679945302 0 0) 0x7f4618070470 con 0x7f461815aac0 2023-11-08T16:37:18.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.174+0000 7f461eb53640 1 --2- 172.21.15.81:0/104393631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46180cf170 0x7f461815b5b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f46100097a0 tx=0x7f461002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2416484cd570b3ed server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:18.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.175+0000 7f461d350640 1 -- 172.21.15.81:0/104393631 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610031830 con 0x7f46180cf170 2023-11-08T16:37:18.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.175+0000 7f461d350640 1 -- 172.21.15.81:0/104393631 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4610031990 con 0x7f46180cf170 2023-11-08T16:37:18.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.175+0000 7f461d350640 1 -- 172.21.15.81:0/104393631 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610031d20 con 0x7f46180cf170 2023-11-08T16:37:18.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.175+0000 7f461f354640 1 -- 172.21.15.81:0/104393631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46180cf170 msgr2=0x7f461815b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:18.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.175+0000 7f461f354640 1 --2- 172.21.15.81:0/104393631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46180cf170 0x7f461815b5b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f46100097a0 tx=0x7f461002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:37:18.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.175+0000 7f461f354640 1 -- 172.21.15.81:0/104393631 shutdown_connections 2023-11-08T16:37:18.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.175+0000 7f461f354640 1 --2- 172.21.15.81:0/104393631 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46180cf170 0x7f461815b5b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:18.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.175+0000 7f461f354640 1 -- 172.21.15.81:0/104393631 >> 172.21.15.81:0/104393631 conn(0x7f461806a4a0 msgr2=0x7f461806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:18.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.176+0000 7f461f354640 1 -- 172.21.15.81:0/104393631 shutdown_connections 2023-11-08T16:37:18.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.176+0000 7f461f354640 1 -- 172.21.15.81:0/104393631 wait complete. 2023-11-08T16:37:18.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.177+0000 7f461f354640 1 Processor -- start 2023-11-08T16:37:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.177+0000 7f461f354640 1 -- start start 2023-11-08T16:37:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.178+0000 7f461f354640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46180ced10 0x7f46181566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.178+0000 7f461f354640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46180cf170 0x7f46180acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.178+0000 7f461f354640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f461815aac0 0x7f46180ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:18.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.178+0000 7f461f354640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180c7d30 con 0x7f46180cf170 2023-11-08T16:37:18.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.178+0000 7f461f354640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180c7a30 con 0x7f461815aac0 2023-11-08T16:37:18.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.178+0000 7f461f354640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4618156c00 con 0x7f46180ced10 2023-11-08T16:37:18.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.179+0000 7f461db51640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46180cf170 0x7f46180acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:18.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.179+0000 7f461db51640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46180cf170 0x7f46180acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36104/0 (socket says 172.21.15.81:36104) 2023-11-08T16:37:18.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.179+0000 7f461db51640 1 -- 172.21.15.81:0/3453991015 learned_addr learned my addr 172.21.15.81:0/3453991015 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:18.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.179+0000 7f461e352640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46180ced10 0x7f46181566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:18.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.179+0000 7f461eb53640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f461815aac0 0x7f46180ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:18.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.179+0000 7f461db51640 1 -- 172.21.15.81:0/3453991015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46180ced10 msgr2=0x7f46181566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:18.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.179+0000 7f461db51640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46180ced10 0x7f46181566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:18.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.179+0000 7f461db51640 1 -- 172.21.15.81:0/3453991015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f461815aac0 msgr2=0x7f46180ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:18.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.179+0000 7f461db51640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f461815aac0 0x7f46180ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:18.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.179+0000 7f461db51640 1 -- 172.21.15.81:0/3453991015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4610009400 con 0x7f46180cf170 2023-11-08T16:37:18.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.180+0000 7f461db51640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46180cf170 0x7f46180acd30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f460800c460 tx=0x7f460800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:18.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.180+0000 7f460effd640 1 -- 172.21.15.81:0/3453991015 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608012830 con 0x7f46180cf170 2023-11-08T16:37:18.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.180+0000 7f460effd640 1 -- 172.21.15.81:0/3453991015 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46080129d0 con 0x7f46180cf170 2023-11-08T16:37:18.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.180+0000 7f460effd640 1 -- 172.21.15.81:0/3453991015 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608012d80 con 0x7f46180cf170 2023-11-08T16:37:18.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.180+0000 7f461f354640 1 -- 172.21.15.81:0/3453991015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46180c45c0 con 0x7f46180cf170 2023-11-08T16:37:18.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.180+0000 7f461f354640 1 -- 172.21.15.81:0/3453991015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46180c4b80 con 0x7f46180cf170 2023-11-08T16:37:18.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.181+0000 7f461f354640 1 -- 172.21.15.81:0/3453991015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46180c7ee0 con 0x7f46180cf170 2023-11-08T16:37:18.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.182+0000 7f460effd640 1 -- 172.21.15.81:0/3453991015 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f460801a3b0 con 0x7f46180cf170 2023-11-08T16:37:18.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.183+0000 7f460effd640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45fc06aea0 0x7f45fc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:18.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.183+0000 7f460effd640 1 -- 172.21.15.81:0/3453991015 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f46080891f0 con 0x7f46180cf170 2023-11-08T16:37:18.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.183+0000 7f461e352640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45fc06aea0 0x7f45fc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:18.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.184+0000 7f461e352640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45fc06aea0 0x7f45fc06d360 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f46140012e0 tx=0x7f4614001630 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:18.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.187+0000 7f460effd640 1 -- 172.21.15.81:0/3453991015 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f46180c7ee0 con 0x7f46180cf170 2023-11-08T16:37:18.224 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:37:18.233 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:37:18.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:18.443+0000 7f461f354640 1 -- 172.21.15.81:0/3453991015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]} v 0) v1 -- 0x7f46180c7ee0 con 0x7f46180cf170 2023-11-08T16:37:19.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.060+0000 7f460effd640 1 -- 172.21.15.81:0/3453991015 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]}]=0 v364) v1 ==== 158+0+0 (secure 0 0 0) 0x7f46180c7ee0 con 0x7f46180cf170 2023-11-08T16:37:19.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.064+0000 7f461f354640 1 -- 172.21.15.81:0/3453991015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45fc06aea0 msgr2=0x7f45fc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:19.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.064+0000 7f461f354640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45fc06aea0 0x7f45fc06d360 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f46140012e0 tx=0x7f4614001630 comp rx=0 tx=0).stop 2023-11-08T16:37:19.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.064+0000 7f461f354640 1 -- 172.21.15.81:0/3453991015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46180cf170 msgr2=0x7f46180acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:19.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.064+0000 7f461f354640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46180cf170 0x7f46180acd30 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f460800c460 tx=0x7f460800c920 comp rx=0 tx=0).stop 2023-11-08T16:37:19.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.064+0000 7f461f354640 1 -- 172.21.15.81:0/3453991015 shutdown_connections 2023-11-08T16:37:19.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.064+0000 7f461f354640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f45fc06aea0 0x7f45fc06d360 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.064+0000 7f461f354640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f461815aac0 0x7f46180ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.064+0000 7f461f354640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46180cf170 0x7f46180acd30 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.064+0000 7f461f354640 1 --2- 172.21.15.81:0/3453991015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46180ced10 0x7f46181566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.064+0000 7f461f354640 1 -- 172.21.15.81:0/3453991015 >> 172.21.15.81:0/3453991015 conn(0x7f461806a4a0 msgr2=0x7f461806a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:19.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.064+0000 7f461f354640 1 -- 172.21.15.81:0/3453991015 shutdown_connections 2023-11-08T16:37:19.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.065+0000 7f461f354640 1 -- 172.21.15.81:0/3453991015 wait complete. 2023-11-08T16:37:19.086 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T16:37:19.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.293+0000 7f56cae7c640 1 Processor -- start 2023-11-08T16:37:19.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.293+0000 7f56cae7c640 1 -- start start 2023-11-08T16:37:19.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.295+0000 7f56cae7c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f56c40cef60 0x7f56c415f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:19.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.295+0000 7f56cae7c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c40cf620 con 0x7f56c406e5a0 2023-11-08T16:37:19.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.295+0000 7f56cae7c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c40ccd90 con 0x7f56c415ad70 2023-11-08T16:37:19.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.295+0000 7f56cae7c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c40cd070 con 0x7f56c40cef60 2023-11-08T16:37:19.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.295+0000 7f56c9419640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f56c40cef60 0x7f56c415f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:19.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.295+0000 7f56c3fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f56c406e5a0 0x7f56c40ce850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34118/0 (socket says 172.21.15.81:34118) 2023-11-08T16:37:19.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.295+0000 7f56c3fff640 1 -- 172.21.15.81:0/405307269 learned_addr learned my addr 172.21.15.81:0/405307269 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:19.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.296+0000 7f56c9419640 1 -- 172.21.15.81:0/405307269 >> v1:172.21.15.93:6789/0 conn(0x7f56c415ad70 legacy=0x7f56c40cfa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:19.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.296+0000 7f56c9419640 1 -- 172.21.15.81:0/405307269 >> v1:172.21.15.81:6789/0 conn(0x7f56c406e5a0 legacy=0x7f56c40ce850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:19.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.296+0000 7f56c37fe640 1 -- 172.21.15.81:0/405307269 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4241825968 0 0) 0x7f56c40cf620 con 0x7f56c406e5a0 2023-11-08T16:37:19.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.296+0000 7f56c9419640 1 -- 172.21.15.81:0/405307269 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c40cd350 con 0x7f56c40cef60 2023-11-08T16:37:19.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.297+0000 7f56c9419640 1 --2- 172.21.15.81:0/405307269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56c40cef60 0x7f56c415f3a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009d90 tx=0x7f56bc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a105225c2e8779 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:19.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.297+0000 7f56c37fe640 1 -- 172.21.15.81:0/405307269 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc02cbd0 con 0x7f56c40cef60 2023-11-08T16:37:19.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.297+0000 7f56c37fe640 1 -- 172.21.15.81:0/405307269 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56bc02cd30 con 0x7f56c40cef60 2023-11-08T16:37:19.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.297+0000 7f56c37fe640 1 -- 172.21.15.81:0/405307269 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56bc02cea0 con 0x7f56c40cef60 2023-11-08T16:37:19.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.298+0000 7f56cae7c640 1 -- 172.21.15.81:0/405307269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56c40cef60 msgr2=0x7f56c415f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:19.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.298+0000 7f56cae7c640 1 --2- 172.21.15.81:0/405307269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56c40cef60 0x7f56c415f3a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f56bc009d90 tx=0x7f56bc02be20 comp rx=0 tx=0).stop 2023-11-08T16:37:19.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.298+0000 7f56cae7c640 1 -- 172.21.15.81:0/405307269 shutdown_connections 2023-11-08T16:37:19.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.298+0000 7f56cae7c640 1 --2- 172.21.15.81:0/405307269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56c40cef60 0x7f56c415f3a0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.298+0000 7f56cae7c640 1 -- 172.21.15.81:0/405307269 >> 172.21.15.81:0/405307269 conn(0x7f56c4065df0 msgr2=0x7f56c40661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:19.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.298+0000 7f56cae7c640 1 -- 172.21.15.81:0/405307269 shutdown_connections 2023-11-08T16:37:19.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.298+0000 7f56cae7c640 1 -- 172.21.15.81:0/405307269 wait complete. 2023-11-08T16:37:19.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.300+0000 7f56cae7c640 1 Processor -- start 2023-11-08T16:37:19.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.300+0000 7f56cae7c640 1 -- start start 2023-11-08T16:37:19.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56cae7c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56c406e5a0 0x7f56c415d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:19.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56cae7c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56c40cef60 0x7f56c4157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:19.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56cae7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56c415ad70 0x7f56c41e4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:19.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56cae7c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c415dbd0 con 0x7f56c406e5a0 2023-11-08T16:37:19.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56cae7c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c41577c0 con 0x7f56c40cef60 2023-11-08T16:37:19.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56cae7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c415c5f0 con 0x7f56c415ad70 2023-11-08T16:37:19.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56c8c18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56c406e5a0 0x7f56c415d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:19.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56c8c18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56c406e5a0 0x7f56c415d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36132/0 (socket says 172.21.15.81:36132) 2023-11-08T16:37:19.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56c8c18640 1 -- 172.21.15.81:0/1675651661 learned_addr learned my addr 172.21.15.81:0/1675651661 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:19.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56c3fff640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56c40cef60 0x7f56c4157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:19.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56c8c18640 1 -- 172.21.15.81:0/1675651661 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56c415ad70 msgr2=0x7f56c41e4c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:19.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.301+0000 7f56c8c18640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56c415ad70 0x7f56c41e4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.302+0000 7f56c8c18640 1 -- 172.21.15.81:0/1675651661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56c40cef60 msgr2=0x7f56c4157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:19.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.302+0000 7f56c8c18640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56c40cef60 0x7f56c4157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.302+0000 7f56c8c18640 1 -- 172.21.15.81:0/1675651661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56bc009a70 con 0x7f56c406e5a0 2023-11-08T16:37:19.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.302+0000 7f56c8c18640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56c406e5a0 0x7f56c415d690 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f56b800c430 tx=0x7f56b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:19.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.302+0000 7f56c17fa640 1 -- 172.21.15.81:0/1675651661 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8011790 con 0x7f56c406e5a0 2023-11-08T16:37:19.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.302+0000 7f56c17fa640 1 -- 172.21.15.81:0/1675651661 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b80118f0 con 0x7f56c406e5a0 2023-11-08T16:37:19.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.302+0000 7f56cae7c640 1 -- 172.21.15.81:0/1675651661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c415c8d0 con 0x7f56c406e5a0 2023-11-08T16:37:19.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.302+0000 7f56c17fa640 1 -- 172.21.15.81:0/1675651661 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8011ce0 con 0x7f56c406e5a0 2023-11-08T16:37:19.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.302+0000 7f56cae7c640 1 -- 172.21.15.81:0/1675651661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c415ce90 con 0x7f56c406e5a0 2023-11-08T16:37:19.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.303+0000 7f56cae7c640 1 -- 172.21.15.81:0/1675651661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c404ece0 con 0x7f56c406e5a0 2023-11-08T16:37:19.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.304+0000 7f56c17fa640 1 -- 172.21.15.81:0/1675651661 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f56b801c460 con 0x7f56c406e5a0 2023-11-08T16:37:19.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.305+0000 7f56c17fa640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56a806ae70 0x7f56a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:19.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.305+0000 7f56c17fa640 1 -- 172.21.15.81:0/1675651661 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f56b8089470 con 0x7f56c406e5a0 2023-11-08T16:37:19.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.305+0000 7f56c3fff640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56a806ae70 0x7f56a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:19.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.306+0000 7f56c3fff640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56a806ae70 0x7f56a806d330 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f56c40c4640 tx=0x7f56b0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:19.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.310+0000 7f56c17fa640 1 -- 172.21.15.81:0/1675651661 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f56b8054670 con 0x7f56c406e5a0 2023-11-08T16:37:19.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.485+0000 7f56cae7c640 1 -- 172.21.15.81:0/1675651661 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f56c406de70 con 0x7f56c406e5a0 2023-11-08T16:37:19.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.492+0000 7f56c17fa640 1 -- 172.21.15.81:0/1675651661 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f56b8057c90 con 0x7f56c406e5a0 2023-11-08T16:37:19.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 -- 172.21.15.81:0/1675651661 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56a806ae70 msgr2=0x7f56a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:19.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56a806ae70 0x7f56a806d330 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f56c40c4640 tx=0x7f56b0009040 comp rx=0 tx=0).stop 2023-11-08T16:37:19.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 -- 172.21.15.81:0/1675651661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56c406e5a0 msgr2=0x7f56c415d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:19.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56c406e5a0 0x7f56c415d690 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f56b800c430 tx=0x7f56b800c8f0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 -- 172.21.15.81:0/1675651661 shutdown_connections 2023-11-08T16:37:19.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56a806ae70 0x7f56a806d330 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56c415ad70 0x7f56c41e4c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56c40cef60 0x7f56c4157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 --2- 172.21.15.81:0/1675651661 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56c406e5a0 0x7f56c415d690 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:19.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 -- 172.21.15.81:0/1675651661 >> 172.21.15.81:0/1675651661 conn(0x7f56c4065df0 msgr2=0x7f56c40661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:19.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 -- 172.21.15.81:0/1675651661 shutdown_connections 2023-11-08T16:37:19.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:19.495+0000 7f56cae7c640 1 -- 172.21.15.81:0/1675651661 wait complete. 2023-11-08T16:37:19.514 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T16:37:19.515 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:37:19.622 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:37:19.746 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T16:37:19.747 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:37:19.747 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:37:19.807 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:37:19.807 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:37:19.808 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:37:19.866 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:37:19.866 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:37:19.867 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:37:19.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:37:19.927 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:37:19.928 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:37:19.988 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:37:19.989 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T16:37:19.989 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:37:20.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.208+0000 7f3badac9640 1 Processor -- start 2023-11-08T16:37:20.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.208+0000 7f3badac9640 1 -- start start 2023-11-08T16:37:20.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.209+0000 7f3badac9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3ba80ade50 0x7f3ba80ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:20.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.210+0000 7f3badac9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba815a680 0x7f3ba815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:20.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.210+0000 7f3badac9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba806b630 con 0x7f3ba815b020 2023-11-08T16:37:20.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.210+0000 7f3badac9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80b1860 con 0x7f3ba80ade50 2023-11-08T16:37:20.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.210+0000 7f3badac9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80ca9c0 con 0x7f3ba815a680 2023-11-08T16:37:20.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.210+0000 7f3ba7fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3ba815b020 0x7f3ba80c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34122/0 (socket says 172.21.15.81:34122) 2023-11-08T16:37:20.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.210+0000 7f3ba7fff640 1 -- 172.21.15.81:0/116836504 learned_addr learned my addr 172.21.15.81:0/116836504 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:20.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.210+0000 7f3ba6ffd640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba815a680 0x7f3ba815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:20.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.210+0000 7f3ba77fe640 1 --2- 172.21.15.81:0/116836504 >> v2:172.21.15.93:3300/0 conn(0x7f3ba80ade50 0x7f3ba80ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:20.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2944684105 0 0) 0x7f3ba806b630 con 0x7f3ba815b020 2023-11-08T16:37:20.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b90003520 con 0x7f3ba815b020 2023-11-08T16:37:20.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1435740497 0 0) 0x7f3b90003520 con 0x7f3ba815b020 2023-11-08T16:37:20.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba815a680 msgr2=0x7f3ba815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:20.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba815a680 0x7f3ba815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 >> v2:172.21.15.93:3300/0 conn(0x7f3ba80ade50 msgr2=0x7f3ba80ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 --2- 172.21.15.81:0/116836504 >> v2:172.21.15.93:3300/0 conn(0x7f3ba80ade50 0x7f3ba80ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:20.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80cabb0 con 0x7f3ba815b020 2023-11-08T16:37:20.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f3ba00019f0 con 0x7f3ba815b020 2023-11-08T16:37:20.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 >> v1:172.21.15.81:6789/0 conn(0x7f3ba815b020 legacy=0x7f3ba80c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:20.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b90003a50 0x7f3b90003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:20.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba815a680 0x7f3b900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:20.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b90004d00 0x7f3b90005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:20.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9000b980 con 0x7f3b90004d00 2023-11-08T16:37:20.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9000bac0 con 0x7f3b90003a50 2023-11-08T16:37:20.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.211+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9000bc30 con 0x7f3ba815a680 2023-11-08T16:37:20.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.212+0000 7f3ba7fff640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b90004d00 0x7f3b90005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:20.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.212+0000 7f3ba77fe640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b90003a50 0x7f3b90003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:20.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.212+0000 7f3ba6ffd640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba815a680 0x7f3b900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.212+0000 7f3ba7fff640 1 -- 172.21.15.81:0/116836504 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba815a680 msgr2=0x7f3b900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.212+0000 7f3ba7fff640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba815a680 0x7f3b900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.212+0000 7f3ba7fff640 1 -- 172.21.15.81:0/116836504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b90003a50 msgr2=0x7f3b90003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.212+0000 7f3ba7fff640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b90003a50 0x7f3b90003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:20.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.212+0000 7f3ba7fff640 1 -- 172.21.15.81:0/116836504 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b90003520 con 0x7f3b90004d00 2023-11-08T16:37:20.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.214+0000 7f3ba7fff640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b90004d00 0x7f3b90005180 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000a860 tx=0x7f3b900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3c9086a77947d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:20.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.214+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0034630 con 0x7f3b90004d00 2023-11-08T16:37:20.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.214+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba002f070 con 0x7f3b90004d00 2023-11-08T16:37:20.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.214+0000 7f3ba67fc640 1 -- 172.21.15.81:0/116836504 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00349f0 con 0x7f3b90004d00 2023-11-08T16:37:20.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.214+0000 7f3badac9640 1 -- 172.21.15.81:0/116836504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b90004d00 msgr2=0x7f3b90005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:20.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.214+0000 7f3badac9640 1 --2- 172.21.15.81:0/116836504 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b90004d00 0x7f3b90005180 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000a860 tx=0x7f3b900057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:20.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.214+0000 7f3ba7fff640 1 -- 172.21.15.81:0/116836504 reap_dead start 2023-11-08T16:37:20.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.215+0000 7f3badac9640 1 -- 172.21.15.81:0/116836504 shutdown_connections 2023-11-08T16:37:20.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.215+0000 7f3badac9640 1 -- 172.21.15.81:0/116836504 >> 172.21.15.81:0/116836504 conn(0x7f3ba80cb490 msgr2=0x7f3ba80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:20.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.215+0000 7f3badac9640 1 -- 172.21.15.81:0/116836504 shutdown_connections 2023-11-08T16:37:20.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.215+0000 7f3badac9640 1 -- 172.21.15.81:0/116836504 wait complete. 2023-11-08T16:37:20.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.217+0000 7f3badac9640 1 Processor -- start 2023-11-08T16:37:20.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.217+0000 7f3badac9640 1 -- start start 2023-11-08T16:37:20.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3badac9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b90004d00 0x7f3ba80cc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:20.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3badac9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba81569f0 0x7f3ba8152600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:20.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3badac9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ba8154b00 0x7f3ba8154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:20.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3badac9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba81f5e30 con 0x7f3ba8154b00 2023-11-08T16:37:20.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3badac9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80ccbe0 con 0x7f3b90004d00 2023-11-08T16:37:20.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3badac9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba80ccec0 con 0x7f3ba81569f0 2023-11-08T16:37:20.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3ba7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ba8154b00 0x7f3ba8154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:20.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3ba7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ba8154b00 0x7f3ba8154f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36138/0 (socket says 172.21.15.81:36138) 2023-11-08T16:37:20.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3ba7fff640 1 -- 172.21.15.81:0/3110873570 learned_addr learned my addr 172.21.15.81:0/3110873570 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:20.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3ba77fe640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b90004d00 0x7f3ba80cc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:20.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3ba6ffd640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba81569f0 0x7f3ba8152600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:20.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3ba7fff640 1 -- 172.21.15.81:0/3110873570 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba81569f0 msgr2=0x7f3ba8152600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:20.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3ba7fff640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba81569f0 0x7f3ba8152600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:20.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3ba7fff640 1 -- 172.21.15.81:0/3110873570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b90004d00 msgr2=0x7f3ba80cc5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:20.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3ba7fff640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b90004d00 0x7f3ba80cc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:20.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.218+0000 7f3ba7fff640 1 -- 172.21.15.81:0/3110873570 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80cd1a0 con 0x7f3ba8154b00 2023-11-08T16:37:20.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.219+0000 7f3ba7fff640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ba8154b00 0x7f3ba8154f80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3ba002cba0 tx=0x7f3ba0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:20.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.219+0000 7f3b8ffff640 1 -- 172.21.15.81:0/3110873570 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0034840 con 0x7f3ba8154b00 2023-11-08T16:37:20.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.219+0000 7f3b8ffff640 1 -- 172.21.15.81:0/3110873570 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba00349a0 con 0x7f3ba8154b00 2023-11-08T16:37:20.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.219+0000 7f3badac9640 1 -- 172.21.15.81:0/3110873570 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba8156e50 con 0x7f3ba8154b00 2023-11-08T16:37:20.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.219+0000 7f3b8ffff640 1 -- 172.21.15.81:0/3110873570 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0034d10 con 0x7f3ba8154b00 2023-11-08T16:37:20.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.219+0000 7f3badac9640 1 -- 172.21.15.81:0/3110873570 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba8152b40 con 0x7f3ba8154b00 2023-11-08T16:37:20.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.220+0000 7f3badac9640 1 -- 172.21.15.81:0/3110873570 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba8152d30 con 0x7f3ba8154b00 2023-11-08T16:37:20.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.221+0000 7f3b8ffff640 1 -- 172.21.15.81:0/3110873570 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3ba0031030 con 0x7f3ba8154b00 2023-11-08T16:37:20.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.222+0000 7f3b8ffff640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b9406ae70 0x7f3b9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:20.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.222+0000 7f3b8ffff640 1 -- 172.21.15.81:0/3110873570 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f3ba00ab6d0 con 0x7f3ba8154b00 2023-11-08T16:37:20.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.222+0000 7f3ba77fe640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b9406ae70 0x7f3b9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:20.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.223+0000 7f3ba77fe640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b9406ae70 0x7f3b9406d330 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3b980099d0 tx=0x7f3b98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:20.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.227+0000 7f3b8ffff640 1 -- 172.21.15.81:0/3110873570 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3ba0076960 con 0x7f3ba8154b00 2023-11-08T16:37:20.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.453+0000 7f3badac9640 1 -- 172.21.15.81:0/3110873570 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ba8152d30 con 0x7f3ba8154b00 2023-11-08T16:37:20.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.454+0000 7f3b8ffff640 1 -- 172.21.15.81:0/3110873570 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f3ba00ab030 con 0x7f3ba8154b00 2023-11-08T16:37:20.456 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:20.457 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5494,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/3027678987","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3027678987},{"type":"v1","addr":"172.21.15.93:6833","nonce":3027678987}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6836/3140373039","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":3140373039},{"type":"v1","addr":"172.21.15.81:6836","nonce":3140373039}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":6603,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3145959397","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":3145959397},{"type":"v1","addr":"172.21.15.93:6835","nonce":3145959397}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":52},{"gid":6840,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/2568502237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6835","nonce":2568502237},{"type":"v1","addr":"172.21.15.81:6837","nonce":2568502237}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[]} 2023-11-08T16:37:20.457 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 56 2023-11-08T16:37:20.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.456+0000 7f3badac9640 1 -- 172.21.15.81:0/3110873570 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b9406ae70 msgr2=0x7f3b9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:20.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.456+0000 7f3badac9640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b9406ae70 0x7f3b9406d330 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3b980099d0 tx=0x7f3b98008040 comp rx=0 tx=0).stop 2023-11-08T16:37:20.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.457+0000 7f3badac9640 1 -- 172.21.15.81:0/3110873570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ba8154b00 msgr2=0x7f3ba8154f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:20.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.457+0000 7f3badac9640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ba8154b00 0x7f3ba8154f80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3ba002cba0 tx=0x7f3ba0001d80 comp rx=0 tx=0).stop 2023-11-08T16:37:20.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.457+0000 7f3badac9640 1 -- 172.21.15.81:0/3110873570 shutdown_connections 2023-11-08T16:37:20.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.457+0000 7f3badac9640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b9406ae70 0x7f3b9406d330 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:20.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.457+0000 7f3badac9640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3ba8154b00 0x7f3ba8154f80 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:20.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.457+0000 7f3badac9640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3ba81569f0 0x7f3ba8152600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:20.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.457+0000 7f3badac9640 1 --2- 172.21.15.81:0/3110873570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b90004d00 0x7f3ba80cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:20.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.457+0000 7f3badac9640 1 -- 172.21.15.81:0/3110873570 >> 172.21.15.81:0/3110873570 conn(0x7f3ba80cb490 msgr2=0x7f3ba81e50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:20.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.457+0000 7f3badac9640 1 -- 172.21.15.81:0/3110873570 shutdown_connections 2023-11-08T16:37:20.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:20.457+0000 7f3badac9640 1 -- 172.21.15.81:0/3110873570 wait complete. 2023-11-08T16:37:20.479 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T16:37:20.480 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T16:37:20.480 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T16:37:20.480 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:37:20.481 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T16:37:20.481 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T16:37:20.481 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T16:37:20.481 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:37:20.482 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T16:37:20.482 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T16:37:20.483 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T16:37:20.483 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:37:20.483 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T16:37:20.483 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T16:37:20.484 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T16:37:20.484 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:37:20.500 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T16:37:20.500 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T16:37:20.505 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T16:37:20.505 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T16:37:20.506 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T16:37:20.507 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T16:37:20.549 INFO:tasks.ceph.mds.c:Started 2023-11-08T16:37:20.550 DEBUG:teuthology.parallel:result is None 2023-11-08T16:37:20.555 INFO:tasks.ceph.mds.b:Started 2023-11-08T16:37:20.556 DEBUG:teuthology.parallel:result is None 2023-11-08T16:37:20.562 INFO:tasks.ceph.mds.d:Started 2023-11-08T16:37:20.562 DEBUG:teuthology.parallel:result is None 2023-11-08T16:37:20.723 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2023-11-08T16:37:20.723 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:37:20.732 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:37:26.500 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T16:37:26.500 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T16:37:26.510 INFO:tasks.ceph.mds.a:Started 2023-11-08T16:37:26.511 DEBUG:teuthology.parallel:result is None 2023-11-08T16:37:26.511 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T16:37:26.677 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:37:26.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.710+0000 7fe484c72640 1 Processor -- start 2023-11-08T16:37:26.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.710+0000 7fe484c72640 1 -- start start 2023-11-08T16:37:26.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.712+0000 7fe484c72640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe480160af0 0x7fe48015cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:26.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.712+0000 7fe484c72640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe48006f850 con 0x7fe4800cb930 2023-11-08T16:37:26.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.712+0000 7fe484c72640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe480161040 con 0x7fe4800ccc40 2023-11-08T16:37:26.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.712+0000 7fe484c72640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe480161320 con 0x7fe480160af0 2023-11-08T16:37:26.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.712+0000 7fe47f7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe4800cb930 0x7fe4800cc530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59208/0 (socket says 172.21.15.81:59208) 2023-11-08T16:37:26.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.712+0000 7fe47f7fe640 1 -- 172.21.15.81:0/4115476487 learned_addr learned my addr 172.21.15.81:0/4115476487 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:26.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.712+0000 7fe47ffff640 1 --2- 172.21.15.81:0/4115476487 >> v2:172.21.15.93:3300/0 conn(0x7fe480160af0 0x7fe48015cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:26.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.712+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825897454 0 0) 0x7fe48006f850 con 0x7fe4800cb930 2023-11-08T16:37:26.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.712+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe46c003520 con 0x7fe4800cb930 2023-11-08T16:37:26.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2364550771 0 0) 0x7fe46c003520 con 0x7fe4800cb930 2023-11-08T16:37:26.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 >> v2:172.21.15.93:3300/0 conn(0x7fe480160af0 msgr2=0x7fe48015cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:26.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 --2- 172.21.15.81:0/4115476487 >> v2:172.21.15.93:3300/0 conn(0x7fe480160af0 0x7fe48015cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:26.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 >> v1:172.21.15.93:6789/0 conn(0x7fe4800ccc40 legacy=0x7fe48015c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:26.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480161600 con 0x7fe4800cb930 2023-11-08T16:37:26.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fe470001670 con 0x7fe4800cb930 2023-11-08T16:37:26.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 >> v1:172.21.15.81:6789/0 conn(0x7fe4800cb930 legacy=0x7fe4800cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:26.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe46c003970 0x7fe46c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:26.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe46c0047c0 0x7fe46c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:26.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c005180 0x7fe46c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:26.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe480161320 con 0x7fe46c005180 2023-11-08T16:37:26.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c00c020 con 0x7fe46c0047c0 2023-11-08T16:37:26.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe46c00c1e0 con 0x7fe46c003970 2023-11-08T16:37:26.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47ffff640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c005180 0x7fe46c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:26.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe477fff640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe46c0047c0 0x7fe46c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:26.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.713+0000 7fe47f7fe640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe46c003970 0x7fe46c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:26.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.714+0000 7fe47ffff640 1 -- 172.21.15.81:0/4115476487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe46c003970 msgr2=0x7fe46c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:26.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.714+0000 7fe47ffff640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe46c003970 0x7fe46c003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:26.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.714+0000 7fe47ffff640 1 -- 172.21.15.81:0/4115476487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe46c0047c0 msgr2=0x7fe46c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:26.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.714+0000 7fe47ffff640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe46c0047c0 0x7fe46c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:26.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.714+0000 7fe47ffff640 1 -- 172.21.15.81:0/4115476487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46c003520 con 0x7fe46c005180 2023-11-08T16:37:26.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.714+0000 7fe477fff640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe46c0047c0 0x7fe46c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:26.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.714+0000 7fe47ffff640 1 -- 172.21.15.81:0/4115476487 reap_dead start 2023-11-08T16:37:26.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.715+0000 7fe47ffff640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c005180 0x7fe46c00b980 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe478009fc0 tx=0x7fe46c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1727cd738fa3d74a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:26.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.715+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47802cc60 con 0x7fe46c005180 2023-11-08T16:37:26.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.715+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47800f040 con 0x7fe46c005180 2023-11-08T16:37:26.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.715+0000 7fe47effd640 1 -- 172.21.15.81:0/4115476487 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478003990 con 0x7fe46c005180 2023-11-08T16:37:26.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.715+0000 7fe484c72640 1 -- 172.21.15.81:0/4115476487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c005180 msgr2=0x7fe46c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:26.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.715+0000 7fe484c72640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c005180 0x7fe46c00b980 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fe478009fc0 tx=0x7fe46c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:26.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.716+0000 7fe484c72640 1 -- 172.21.15.81:0/4115476487 shutdown_connections 2023-11-08T16:37:26.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.716+0000 7fe484c72640 1 --2- 172.21.15.81:0/4115476487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe46c005180 0x7fe46c00b980 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:26.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.716+0000 7fe484c72640 1 -- 172.21.15.81:0/4115476487 >> 172.21.15.81:0/4115476487 conn(0x7fe4800b04d0 msgr2=0x7fe4800b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:26.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.716+0000 7fe484c72640 1 -- 172.21.15.81:0/4115476487 shutdown_connections 2023-11-08T16:37:26.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.716+0000 7fe484c72640 1 -- 172.21.15.81:0/4115476487 wait complete. 2023-11-08T16:37:26.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.717+0000 7fe484c72640 1 Processor -- start 2023-11-08T16:37:26.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.717+0000 7fe484c72640 1 -- start start 2023-11-08T16:37:26.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.718+0000 7fe484c72640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe46c005180 0x7fe480154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:26.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.718+0000 7fe484c72640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe480155940 0x7fe480155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:26.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.718+0000 7fe484c72640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe480156300 0x7fe4801579b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:26.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.718+0000 7fe484c72640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe480154f50 con 0x7fe480155940 2023-11-08T16:37:26.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.718+0000 7fe484c72640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4801f2640 con 0x7fe480156300 2023-11-08T16:37:26.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.718+0000 7fe484c72640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe480156780 con 0x7fe46c005180 2023-11-08T16:37:26.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.719+0000 7fe477fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe480155940 0x7fe480155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:26.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.719+0000 7fe477fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe480155940 0x7fe480155dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42846/0 (socket says 172.21.15.81:42846) 2023-11-08T16:37:26.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.719+0000 7fe477fff640 1 -- 172.21.15.81:0/3080582114 learned_addr learned my addr 172.21.15.81:0/3080582114 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:26.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.719+0000 7fe47ffff640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe480156300 0x7fe4801579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:26.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.719+0000 7fe47f7fe640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe46c005180 0x7fe480154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:26.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.719+0000 7fe477fff640 1 -- 172.21.15.81:0/3080582114 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe46c005180 msgr2=0x7fe480154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:26.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.719+0000 7fe477fff640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe46c005180 0x7fe480154a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:26.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.719+0000 7fe477fff640 1 -- 172.21.15.81:0/3080582114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe480156300 msgr2=0x7fe4801579b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:26.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.719+0000 7fe477fff640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe480156300 0x7fe4801579b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:26.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.719+0000 7fe477fff640 1 -- 172.21.15.81:0/3080582114 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4800c3ef0 con 0x7fe480155940 2023-11-08T16:37:26.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.719+0000 7fe477fff640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe480155940 0x7fe480155dc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe46800d7a0 tx=0x7fe46800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:26.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.720+0000 7fe47cff9640 1 -- 172.21.15.81:0/3080582114 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe468005270 con 0x7fe480155940 2023-11-08T16:37:26.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.720+0000 7fe47cff9640 1 -- 172.21.15.81:0/3080582114 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4680053d0 con 0x7fe480155940 2023-11-08T16:37:26.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.720+0000 7fe47cff9640 1 -- 172.21.15.81:0/3080582114 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46800b8f0 con 0x7fe480155940 2023-11-08T16:37:26.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.720+0000 7fe484c72640 1 -- 172.21.15.81:0/3080582114 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4800c41d0 con 0x7fe480155940 2023-11-08T16:37:26.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.720+0000 7fe484c72640 1 -- 172.21.15.81:0/3080582114 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4800c4790 con 0x7fe480155940 2023-11-08T16:37:26.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.721+0000 7fe484c72640 1 -- 172.21.15.81:0/3080582114 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe48004ece0 con 0x7fe480155940 2023-11-08T16:37:26.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.722+0000 7fe47cff9640 1 -- 172.21.15.81:0/3080582114 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe46800ba50 con 0x7fe480155940 2023-11-08T16:37:26.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.723+0000 7fe47cff9640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe46006aea0 0x7fe46006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:26.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.723+0000 7fe47cff9640 1 -- 172.21.15.81:0/3080582114 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5201+0+0 (secure 0 0 0) 0x7fe468089480 con 0x7fe480155940 2023-11-08T16:37:26.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.723+0000 7fe47f7fe640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe46006aea0 0x7fe46006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:26.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.724+0000 7fe47f7fe640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe46006aea0 0x7fe46006d360 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe470001200 tx=0x7fe470009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:26.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.727+0000 7fe47cff9640 1 -- 172.21.15.81:0/3080582114 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe468054680 con 0x7fe480155940 2023-11-08T16:37:26.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:26.933+0000 7fe484c72640 1 -- 172.21.15.81:0/3080582114 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe4800add10 con 0x7fe480155940 2023-11-08T16:37:27.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.803+0000 7fe47cff9640 1 -- 172.21.15.81:0/3080582114 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe468019310 con 0x7fe480155940 2023-11-08T16:37:27.805 INFO:teuthology.orchestra.run.smithi081.stderr: removed all blocklist entries 2023-11-08T16:37:27.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 -- 172.21.15.81:0/3080582114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe46006aea0 msgr2=0x7fe46006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe46006aea0 0x7fe46006d360 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe470001200 tx=0x7fe470009040 comp rx=0 tx=0).stop 2023-11-08T16:37:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 -- 172.21.15.81:0/3080582114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe480155940 msgr2=0x7fe480155dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe480155940 0x7fe480155dc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fe46800d7a0 tx=0x7fe46800dc60 comp rx=0 tx=0).stop 2023-11-08T16:37:27.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 -- 172.21.15.81:0/3080582114 shutdown_connections 2023-11-08T16:37:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe46006aea0 0x7fe46006d360 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe480156300 0x7fe4801579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe480155940 0x7fe480155dc0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 --2- 172.21.15.81:0/3080582114 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe46c005180 0x7fe480154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:27.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 -- 172.21.15.81:0/3080582114 >> 172.21.15.81:0/3080582114 conn(0x7fe4800b04d0 msgr2=0x7fe48006bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:27.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 -- 172.21.15.81:0/3080582114 shutdown_connections 2023-11-08T16:37:27.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:27.807+0000 7fe484c72640 1 -- 172.21.15.81:0/3080582114 wait complete. 2023-11-08T16:37:27.831 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T16:37:28.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.043+0000 7fbcc0505640 1 Processor -- start 2023-11-08T16:37:28.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.043+0000 7fbcc0505640 1 -- start start 2023-11-08T16:37:28.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.044+0000 7fbcc0505640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbcb815d3b0 0x7fbcb806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.044+0000 7fbcc0505640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbcb80cf720 0x7fbcb80c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.044+0000 7fbcc0505640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb806cc50 con 0x7fbcb806dab0 2023-11-08T16:37:28.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.044+0000 7fbcc0505640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb80c90f0 con 0x7fbcb815d3b0 2023-11-08T16:37:28.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.044+0000 7fbcc0505640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb80adb70 con 0x7fbcb80cf720 2023-11-08T16:37:28.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.045+0000 7fbcbe2a1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbcb815d3b0 0x7fbcb806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.045+0000 7fbcbe2a1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbcb815d3b0 0x7fbcb806d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42848/0 (socket says 172.21.15.81:42848) 2023-11-08T16:37:28.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.045+0000 7fbcbe2a1640 1 -- 172.21.15.81:0/2945461700 learned_addr learned my addr 172.21.15.81:0/2945461700 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:28.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.045+0000 7fbcbe2a1640 1 -- 172.21.15.81:0/2945461700 >> v2:172.21.15.93:3300/0 conn(0x7fbcb80cf720 msgr2=0x7fbcb80c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:28.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.045+0000 7fbcbdaa0640 1 --2- 172.21.15.81:0/2945461700 >> v2:172.21.15.93:3300/0 conn(0x7fbcb80cf720 0x7fbcb80c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.045+0000 7fbcbe2a1640 1 --2- 172.21.15.81:0/2945461700 >> v2:172.21.15.93:3300/0 conn(0x7fbcb80cf720 0x7fbcb80c9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.045+0000 7fbcbe2a1640 1 -- 172.21.15.81:0/2945461700 >> v1:172.21.15.81:6789/0 conn(0x7fbcb806dab0 legacy=0x7fbcb8070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.045+0000 7fbcbe2a1640 1 -- 172.21.15.81:0/2945461700 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb80c9360 con 0x7fbcb815d3b0 2023-11-08T16:37:28.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.045+0000 7fbcbd29f640 1 -- 172.21.15.81:0/2945461700 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1576163460 0 0) 0x7fbcb806cc50 con 0x7fbcb806dab0 2023-11-08T16:37:28.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.046+0000 7fbcbe2a1640 1 --2- 172.21.15.81:0/2945461700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb815d3b0 0x7fbcb806d0e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbcb806f9b0 tx=0x7fbcb402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b70817500e7059 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:28.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.047+0000 7fbcbd29f640 1 -- 172.21.15.81:0/2945461700 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4004230 con 0x7fbcb815d3b0 2023-11-08T16:37:28.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.047+0000 7fbcbd29f640 1 -- 172.21.15.81:0/2945461700 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb4004390 con 0x7fbcb815d3b0 2023-11-08T16:37:28.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.047+0000 7fbcbd29f640 1 -- 172.21.15.81:0/2945461700 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4030840 con 0x7fbcb815d3b0 2023-11-08T16:37:28.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.047+0000 7fbcc0505640 1 -- 172.21.15.81:0/2945461700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb815d3b0 msgr2=0x7fbcb806d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.047+0000 7fbcc0505640 1 --2- 172.21.15.81:0/2945461700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb815d3b0 0x7fbcb806d0e0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fbcb806f9b0 tx=0x7fbcb402b980 comp rx=0 tx=0).stop 2023-11-08T16:37:28.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.047+0000 7fbcc0505640 1 -- 172.21.15.81:0/2945461700 shutdown_connections 2023-11-08T16:37:28.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.047+0000 7fbcc0505640 1 --2- 172.21.15.81:0/2945461700 >> v2:172.21.15.93:3300/0 conn(0x7fbcb80cf720 0x7fbcb80c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.047+0000 7fbcc0505640 1 --2- 172.21.15.81:0/2945461700 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb815d3b0 0x7fbcb806d0e0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.047+0000 7fbcc0505640 1 -- 172.21.15.81:0/2945461700 >> 172.21.15.81:0/2945461700 conn(0x7fbcb80ceaa0 msgr2=0x7fbcb80cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:28.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.047+0000 7fbcc0505640 1 -- 172.21.15.81:0/2945461700 shutdown_connections 2023-11-08T16:37:28.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.047+0000 7fbcc0505640 1 -- 172.21.15.81:0/2945461700 wait complete. 2023-11-08T16:37:28.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.049+0000 7fbcc0505640 1 Processor -- start 2023-11-08T16:37:28.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.049+0000 7fbcc0505640 1 -- start start 2023-11-08T16:37:28.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.050+0000 7fbcc0505640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb806dab0 0x7fbcb814ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.050+0000 7fbcc0505640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb80cf720 0x7fbcb80c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.050+0000 7fbcc0505640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcb815d3b0 0x7fbcb80c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.050+0000 7fbcc0505640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80b19d0 con 0x7fbcb80cf720 2023-11-08T16:37:28.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.050+0000 7fbcc0505640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb8150520 con 0x7fbcb815d3b0 2023-11-08T16:37:28.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.050+0000 7fbcc0505640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb80c51a0 con 0x7fbcb806dab0 2023-11-08T16:37:28.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.051+0000 7fbcbdaa0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb80cf720 0x7fbcb80c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.051+0000 7fbcbdaa0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb80cf720 0x7fbcb80c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42854/0 (socket says 172.21.15.81:42854) 2023-11-08T16:37:28.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.051+0000 7fbcbdaa0640 1 -- 172.21.15.81:0/1662939261 learned_addr learned my addr 172.21.15.81:0/1662939261 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:28.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.051+0000 7fbcbe2a1640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb806dab0 0x7fbcb814ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.051+0000 7fbcbdaa0640 1 -- 172.21.15.81:0/1662939261 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb806dab0 msgr2=0x7fbcb814ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.051+0000 7fbcbdaa0640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb806dab0 0x7fbcb814ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.051+0000 7fbcbeaa2640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcb815d3b0 0x7fbcb80c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.051+0000 7fbcbdaa0640 1 -- 172.21.15.81:0/1662939261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcb815d3b0 msgr2=0x7fbcb80c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.051+0000 7fbcbdaa0640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcb815d3b0 0x7fbcb80c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.051+0000 7fbcbdaa0640 1 -- 172.21.15.81:0/1662939261 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb4009460 con 0x7fbcb80cf720 2023-11-08T16:37:28.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.051+0000 7fbcbdaa0640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb80cf720 0x7fbcb80c4630 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fbca800c7f0 tx=0x7fbca800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:28.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.052+0000 7fbcaeffd640 1 -- 172.21.15.81:0/1662939261 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8004220 con 0x7fbcb80cf720 2023-11-08T16:37:28.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.052+0000 7fbcaeffd640 1 -- 172.21.15.81:0/1662939261 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca80043a0 con 0x7fbcb80cf720 2023-11-08T16:37:28.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.052+0000 7fbcaeffd640 1 -- 172.21.15.81:0/1662939261 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8007860 con 0x7fbcb80cf720 2023-11-08T16:37:28.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.052+0000 7fbcc0505640 1 -- 172.21.15.81:0/1662939261 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb80c5480 con 0x7fbcb80cf720 2023-11-08T16:37:28.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.052+0000 7fbcc0505640 1 -- 172.21.15.81:0/1662939261 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb80b1660 con 0x7fbcb80cf720 2023-11-08T16:37:28.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.053+0000 7fbcc0505640 1 -- 172.21.15.81:0/1662939261 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb80b1b40 con 0x7fbcb80cf720 2023-11-08T16:37:28.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.054+0000 7fbcaeffd640 1 -- 172.21.15.81:0/1662939261 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbca8007bd0 con 0x7fbcb80cf720 2023-11-08T16:37:28.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.055+0000 7fbcaeffd640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc9c06ae70 0x7fbc9c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.055+0000 7fbcaeffd640 1 -- 172.21.15.81:0/1662939261 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbca8089260 con 0x7fbcb80cf720 2023-11-08T16:37:28.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.055+0000 7fbcbe2a1640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc9c06ae70 0x7fbc9c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.056+0000 7fbcbe2a1640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc9c06ae70 0x7fbc9c06d330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4035000 tx=0x7fbcb4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:28.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.060+0000 7fbcaeffd640 1 -- 172.21.15.81:0/1662939261 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbca80546f0 con 0x7fbcb80cf720 2023-11-08T16:37:28.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.320+0000 7fbcc0505640 1 -- 172.21.15.81:0/1662939261 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fbcb80743f0 con 0x7fbcb80cf720 2023-11-08T16:37:28.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.321+0000 7fbcaeffd640 1 -- 172.21.15.81:0/1662939261 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fbca8057d10 con 0x7fbcb80cf720 2023-11-08T16:37:28.323 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:28.323 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:37:28.323 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_dump": [ 2023-11-08T16:37:28.323 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.324 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.a", 2023-11-08T16:37:28.324 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlL0bmNxAAlrzRL4depZdd+lBB5HN09g==", 2023-11-08T16:37:28.324 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.324 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:37:28.324 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:37:28.325 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:37:28.325 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.325 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.325 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.325 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.325 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.b", 2023-11-08T16:37:28.326 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tl2cFWHhAAvwSSL9e5996l9MBwNQzs6A==", 2023-11-08T16:37:28.326 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.326 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:37:28.326 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:37:28.327 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:37:28.327 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.327 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.327 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.327 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.328 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.c", 2023-11-08T16:37:28.328 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tlwo23DBAAl1sBjOJMIQRb1fH3o9XoiQ==", 2023-11-08T16:37:28.328 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.328 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:37:28.328 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:37:28.329 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:37:28.329 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.330 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.330 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.330 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.330 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.d", 2023-11-08T16:37:28.331 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tl5DdQLxAA1aU7e9epGtKnZ6hkHZTnsQ==", 2023-11-08T16:37:28.331 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.331 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:37:28.331 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:37:28.332 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:37:28.332 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.332 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.332 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.332 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.333 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.0", 2023-11-08T16:37:28.333 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQALt0tl3kkwKRAAgDwD1eQYk7qjvV+/gyol/w==", 2023-11-08T16:37:28.333 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.333 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:37:28.333 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:37:28.333 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.334 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.334 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.334 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.334 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.1", 2023-11-08T16:37:28.335 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAPt0tlpIh+BRAAUukXDBbd9Vev1TzIrv9sRg==", 2023-11-08T16:37:28.335 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.335 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:37:28.335 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:37:28.335 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.336 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.336 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.336 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.336 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.2", 2023-11-08T16:37:28.336 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQASt0tlmzJfDhAASO7zeg/9ddmLsoOTe+MIug==", 2023-11-08T16:37:28.337 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.337 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:37:28.337 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:37:28.337 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.337 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.338 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.338 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.338 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.3", 2023-11-08T16:37:28.338 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAVt0tlGoQSGRAAQsrZhrx4kpYUm3KEb6ZUGQ==", 2023-11-08T16:37:28.338 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.339 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:37:28.339 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:37:28.339 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.339 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.340 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.340 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.340 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.4", 2023-11-08T16:37:28.340 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAct0tlOS/kIxAAxC2I47B2WeGTFZxvdnmkMg==", 2023-11-08T16:37:28.340 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.340 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:37:28.341 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:37:28.341 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.341 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.341 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.341 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.342 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.5", 2023-11-08T16:37:28.342 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAgt0tlnAgEBRAADIRSyu/bxiykCc2uiIt0kA==", 2023-11-08T16:37:28.342 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.342 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:37:28.342 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:37:28.342 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.343 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.343 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.343 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.343 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.6", 2023-11-08T16:37:28.343 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAjt0tlrdfZHBAAHGdvBBzVwXMmy8GmsZLNbg==", 2023-11-08T16:37:28.344 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.344 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:37:28.344 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:37:28.344 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.344 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.344 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.345 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.345 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.7", 2023-11-08T16:37:28.345 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAmt0tl4nN9NRAAzV9abuBfhybYCsSFnqHEYA==", 2023-11-08T16:37:28.345 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.345 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:37:28.346 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:37:28.346 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.346 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.346 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.346 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.347 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.0", 2023-11-08T16:37:28.347 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ==", 2023-11-08T16:37:28.347 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.347 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:37:28.347 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow r", 2023-11-08T16:37:28.348 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T16:37:28.348 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.348 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.348 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.348 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.1", 2023-11-08T16:37:28.349 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlgn/qGBAA+WHDN1CJ1v62EbX1GEFBWA==", 2023-11-08T16:37:28.349 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.349 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:37:28.349 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:37:28.349 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:37:28.350 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:37:28.350 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.350 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.350 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.350 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.2", 2023-11-08T16:37:28.351 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlbQmgDRAAL/b9NwCBVT+uGxuv3VAWRw==", 2023-11-08T16:37:28.351 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.351 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:37:28.351 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:37:28.351 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:37:28.352 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:37:28.352 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.352 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.352 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.352 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.3", 2023-11-08T16:37:28.352 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlah+ZIxAAg53qCmUjPtG+l++2Byn1NA==", 2023-11-08T16:37:28.353 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.353 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:37:28.353 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:37:28.353 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:37:28.353 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:37:28.354 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.354 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.354 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.354 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.admin", 2023-11-08T16:37:28.354 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAEt0tlNjJ+KRAATvWBm8Guj09sQQB2U1Afzw==", 2023-11-08T16:37:28.355 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.355 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:37:28.355 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:37:28.355 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:37:28.355 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.356 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.356 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.356 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.356 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.x", 2023-11-08T16:37:28.356 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlhZFjLRAAANBB8roHZH4z4pE4C2Aq/Q==", 2023-11-08T16:37:28.357 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.357 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:37:28.357 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2023-11-08T16:37:28.357 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.357 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.357 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:28.358 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:28.358 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.y", 2023-11-08T16:37:28.358 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlNpDaIBAAtgLFNQitjQnLny6SRY7hHw==", 2023-11-08T16:37:28.358 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:37:28.358 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:37:28.359 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2023-11-08T16:37:28.359 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:37:28.359 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.359 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:28.359 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2023-11-08T16:37:28.360 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:37:28.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 -- 172.21.15.81:0/1662939261 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc9c06ae70 msgr2=0x7fbc9c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc9c06ae70 0x7fbc9c06d330 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4035000 tx=0x7fbcb4008030 comp rx=0 tx=0).stop 2023-11-08T16:37:28.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 -- 172.21.15.81:0/1662939261 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb80cf720 msgr2=0x7fbcb80c4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb80cf720 0x7fbcb80c4630 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fbca800c7f0 tx=0x7fbca800ccb0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 -- 172.21.15.81:0/1662939261 shutdown_connections 2023-11-08T16:37:28.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbc9c06ae70 0x7fbc9c06d330 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbcb815d3b0 0x7fbcb80c4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbcb80cf720 0x7fbcb80c4630 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 --2- 172.21.15.81:0/1662939261 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbcb806dab0 0x7fbcb814ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 -- 172.21.15.81:0/1662939261 >> 172.21.15.81:0/1662939261 conn(0x7fbcb80ceaa0 msgr2=0x7fbcb80ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:28.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 -- 172.21.15.81:0/1662939261 shutdown_connections 2023-11-08T16:37:28.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.324+0000 7fbcc0505640 1 -- 172.21.15.81:0/1662939261 wait complete. 2023-11-08T16:37:28.364 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:28.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.579+0000 7f3102d97640 1 Processor -- start 2023-11-08T16:37:28.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.579+0000 7f3102d97640 1 -- start start 2023-11-08T16:37:28.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.581+0000 7f3102d97640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f30fc0ca480 0x7f30fc15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.581+0000 7f3102d97640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f30fc15ea60 0x7f30fc15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.581+0000 7f3102d97640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30fc06b630 con 0x7f30fc0ade50 2023-11-08T16:37:28.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.581+0000 7f3102d97640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc0b1860 con 0x7f30fc15ea60 2023-11-08T16:37:28.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.581+0000 7f3102d97640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc0ca910 con 0x7f30fc0ca480 2023-11-08T16:37:28.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.581+0000 7f3100b33640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f30fc0ade50 0x7f30fc15a680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59216/0 (socket says 172.21.15.81:59216) 2023-11-08T16:37:28.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.581+0000 7f3100b33640 1 -- 172.21.15.81:0/1826845791 learned_addr learned my addr 172.21.15.81:0/1826845791 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:28.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.581+0000 7f3101334640 1 --2- 172.21.15.81:0/1826845791 >> v2:172.21.15.81:3300/0 conn(0x7f30fc15ea60 0x7f30fc15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.581+0000 7f30fbfff640 1 --2- 172.21.15.81:0/1826845791 >> v2:172.21.15.93:3300/0 conn(0x7f30fc0ca480 0x7f30fc15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.582+0000 7f3101334640 1 -- 172.21.15.81:0/1826845791 >> v2:172.21.15.93:3300/0 conn(0x7f30fc0ca480 msgr2=0x7f30fc15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:28.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.582+0000 7f3101334640 1 --2- 172.21.15.81:0/1826845791 >> v2:172.21.15.93:3300/0 conn(0x7f30fc0ca480 0x7f30fc15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.582+0000 7f3101334640 1 -- 172.21.15.81:0/1826845791 >> v1:172.21.15.81:6789/0 conn(0x7f30fc0ade50 legacy=0x7f30fc15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.582+0000 7f3101334640 1 -- 172.21.15.81:0/1826845791 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc0cabb0 con 0x7f30fc15ea60 2023-11-08T16:37:28.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.582+0000 7f30fb7fe640 1 -- 172.21.15.81:0/1826845791 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 171065005 0 0) 0x7f30fc06b630 con 0x7f30fc0ade50 2023-11-08T16:37:28.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.582+0000 7f30fbfff640 1 --2- 172.21.15.81:0/1826845791 >> v2:172.21.15.93:3300/0 conn(0x7f30fc0ca480 0x7f30fc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:37:28.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.583+0000 7f3101334640 1 --2- 172.21.15.81:0/1826845791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc15ea60 0x7f30fc15eef0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f30fc073d00 tx=0x7f30f402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4492c9aeefdf21b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:28.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.583+0000 7f30fb7fe640 1 -- 172.21.15.81:0/1826845791 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4031830 con 0x7f30fc15ea60 2023-11-08T16:37:28.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.583+0000 7f30fb7fe640 1 -- 172.21.15.81:0/1826845791 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f4031990 con 0x7f30fc15ea60 2023-11-08T16:37:28.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.583+0000 7f30fb7fe640 1 -- 172.21.15.81:0/1826845791 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4031d20 con 0x7f30fc15ea60 2023-11-08T16:37:28.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.584+0000 7f3102d97640 1 -- 172.21.15.81:0/1826845791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc15ea60 msgr2=0x7f30fc15eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.584+0000 7f3102d97640 1 --2- 172.21.15.81:0/1826845791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc15ea60 0x7f30fc15eef0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f30fc073d00 tx=0x7f30f402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.584+0000 7f3102d97640 1 -- 172.21.15.81:0/1826845791 shutdown_connections 2023-11-08T16:37:28.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.584+0000 7f3102d97640 1 --2- 172.21.15.81:0/1826845791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc15ea60 0x7f30fc15eef0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.584+0000 7f3102d97640 1 --2- 172.21.15.81:0/1826845791 >> v2:172.21.15.93:3300/0 conn(0x7f30fc0ca480 0x7f30fc15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.584+0000 7f3102d97640 1 -- 172.21.15.81:0/1826845791 >> 172.21.15.81:0/1826845791 conn(0x7f30fc0cb490 msgr2=0x7f30fc0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:28.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.584+0000 7f3102d97640 1 -- 172.21.15.81:0/1826845791 shutdown_connections 2023-11-08T16:37:28.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.584+0000 7f3102d97640 1 -- 172.21.15.81:0/1826845791 wait complete. 2023-11-08T16:37:28.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.586+0000 7f3102d97640 1 Processor -- start 2023-11-08T16:37:28.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.586+0000 7f3102d97640 1 -- start start 2023-11-08T16:37:28.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3102d97640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f30fc0ade50 0x7f30fc159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3102d97640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30fc0ca480 0x7f30fc15bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3102d97640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc15ea60 0x7f30fc158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3102d97640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc159fc0 con 0x7f30fc15ea60 2023-11-08T16:37:28.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3102d97640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc157bb0 con 0x7f30fc0ca480 2023-11-08T16:37:28.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3102d97640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc157e90 con 0x7f30fc0ade50 2023-11-08T16:37:28.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3101334640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc15ea60 0x7f30fc158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3101334640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc15ea60 0x7f30fc158950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42860/0 (socket says 172.21.15.81:42860) 2023-11-08T16:37:28.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3101334640 1 -- 172.21.15.81:0/2046857100 learned_addr learned my addr 172.21.15.81:0/2046857100 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:28.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3100b33640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f30fc0ade50 0x7f30fc159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f30fbfff640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30fc0ca480 0x7f30fc15bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3101334640 1 -- 172.21.15.81:0/2046857100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f30fc0ade50 msgr2=0x7f30fc159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3101334640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f30fc0ade50 0x7f30fc159700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3101334640 1 -- 172.21.15.81:0/2046857100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30fc0ca480 msgr2=0x7f30fc15bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.587+0000 7f3101334640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30fc0ca480 0x7f30fc15bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.588+0000 7f3101334640 1 -- 172.21.15.81:0/2046857100 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f4009400 con 0x7f30fc15ea60 2023-11-08T16:37:28.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.588+0000 7f3101334640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc15ea60 0x7f30fc158950 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f30f4034040 tx=0x7f30f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:28.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.588+0000 7f30f97fa640 1 -- 172.21.15.81:0/2046857100 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f40317d0 con 0x7f30fc15ea60 2023-11-08T16:37:28.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.588+0000 7f30f97fa640 1 -- 172.21.15.81:0/2046857100 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f403a9e0 con 0x7f30fc15ea60 2023-11-08T16:37:28.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.588+0000 7f3102d97640 1 -- 172.21.15.81:0/2046857100 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc158170 con 0x7f30fc15ea60 2023-11-08T16:37:28.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.588+0000 7f30f97fa640 1 -- 172.21.15.81:0/2046857100 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f403ad90 con 0x7f30fc15ea60 2023-11-08T16:37:28.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.588+0000 7f3102d97640 1 -- 172.21.15.81:0/2046857100 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc159c40 con 0x7f30fc15ea60 2023-11-08T16:37:28.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.589+0000 7f3102d97640 1 -- 172.21.15.81:0/2046857100 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc04ece0 con 0x7f30fc15ea60 2023-11-08T16:37:28.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.590+0000 7f30f97fa640 1 -- 172.21.15.81:0/2046857100 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f30f40443f0 con 0x7f30fc15ea60 2023-11-08T16:37:28.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.592+0000 7f30f97fa640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30dc06ae70 0x7f30dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:28.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.592+0000 7f30f97fa640 1 -- 172.21.15.81:0/2046857100 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f30f40ab5d0 con 0x7f30fc15ea60 2023-11-08T16:37:28.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.592+0000 7f3100b33640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30dc06ae70 0x7f30dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:28.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.592+0000 7f3100b33640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30dc06ae70 0x7f30dc06d330 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0012e0 tx=0x7f30ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:28.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.596+0000 7f30f97fa640 1 -- 172.21.15.81:0/2046857100 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f30f4076a50 con 0x7f30fc15ea60 2023-11-08T16:37:28.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.787+0000 7f3102d97640 1 -- 172.21.15.81:0/2046857100 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f30fc0713f0 con 0x7f30fc15ea60 2023-11-08T16:37:28.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.788+0000 7f30f97fa640 1 -- 172.21.15.81:0/2046857100 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+12797 (secure 0 0 0) 0x7f30f4035090 con 0x7f30fc15ea60 2023-11-08T16:37:28.790 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:28.790 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":77,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:37:27.800115+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:28.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.790+0000 7f3102d97640 1 -- 172.21.15.81:0/2046857100 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30dc06ae70 msgr2=0x7f30dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.790+0000 7f3102d97640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30dc06ae70 0x7f30dc06d330 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0012e0 tx=0x7f30ec009040 comp rx=0 tx=0).stop 2023-11-08T16:37:28.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.790+0000 7f3102d97640 1 -- 172.21.15.81:0/2046857100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc15ea60 msgr2=0x7f30fc158950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:28.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.791+0000 7f3102d97640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc15ea60 0x7f30fc158950 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f30f4034040 tx=0x7f30f40046c0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.791+0000 7f3102d97640 1 -- 172.21.15.81:0/2046857100 shutdown_connections 2023-11-08T16:37:28.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.791+0000 7f3102d97640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f30dc06ae70 0x7f30dc06d330 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.791+0000 7f3102d97640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f30fc15ea60 0x7f30fc158950 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.791+0000 7f3102d97640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30fc0ca480 0x7f30fc15bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.791+0000 7f3102d97640 1 --2- 172.21.15.81:0/2046857100 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f30fc0ade50 0x7f30fc159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:28.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.791+0000 7f3102d97640 1 -- 172.21.15.81:0/2046857100 >> 172.21.15.81:0/2046857100 conn(0x7f30fc0cb490 msgr2=0x7f30fc157200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:28.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.791+0000 7f3102d97640 1 -- 172.21.15.81:0/2046857100 shutdown_connections 2023-11-08T16:37:28.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:28.791+0000 7f3102d97640 1 -- 172.21.15.81:0/2046857100 wait complete. 2023-11-08T16:37:28.813 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:37:28.814 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:37:29.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.026+0000 7ff0d2761640 1 Processor -- start 2023-11-08T16:37:29.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.026+0000 7ff0d2761640 1 -- start start 2023-11-08T16:37:29.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.027+0000 7ff0d2761640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0cc15a0e0 0x7ff0cc0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.027+0000 7ff0d2761640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0cc0ca720 0x7ff0cc06b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.027+0000 7ff0d2761640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0cc0ce960 con 0x7ff0cc0c9bf0 2023-11-08T16:37:29.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.027+0000 7ff0d2761640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0cc071980 con 0x7ff0cc0ca720 2023-11-08T16:37:29.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.027+0000 7ff0d2761640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0c9480 con 0x7ff0cc15a0e0 2023-11-08T16:37:29.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.028+0000 7ff0d0cfe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0cc0ca720 0x7ff0cc06b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.028+0000 7ff0d0cfe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff0cc0ca720 0x7ff0cc06b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42876/0 (socket says 172.21.15.81:42876) 2023-11-08T16:37:29.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.028+0000 7ff0d0cfe640 1 -- 172.21.15.81:0/1161103600 learned_addr learned my addr 172.21.15.81:0/1161103600 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:29.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.028+0000 7ff0cbfff640 1 --2- 172.21.15.81:0/1161103600 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0cc15a0e0 0x7ff0cc0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.028+0000 7ff0d0cfe640 1 -- 172.21.15.81:0/1161103600 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0cc15a0e0 msgr2=0x7ff0cc0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:29.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.028+0000 7ff0d0cfe640 1 --2- 172.21.15.81:0/1161103600 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0cc15a0e0 0x7ff0cc0ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.028+0000 7ff0d0cfe640 1 -- 172.21.15.81:0/1161103600 >> v1:172.21.15.93:6789/0 conn(0x7ff0cc0c9bf0 legacy=0x7ff0cc0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:29.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.028+0000 7ff0d0cfe640 1 -- 172.21.15.81:0/1161103600 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc15a4c0 con 0x7ff0cc0ca720 2023-11-08T16:37:29.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.029+0000 7ff0d0cfe640 1 --2- 172.21.15.81:0/1161103600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0cc0ca720 0x7ff0cc06b710 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009e50 tx=0x7ff0c402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=930c6024d6c91c13 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:29.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.029+0000 7ff0caffd640 1 -- 172.21.15.81:0/1161103600 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c402cb00 con 0x7ff0cc0ca720 2023-11-08T16:37:29.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.030+0000 7ff0caffd640 1 -- 172.21.15.81:0/1161103600 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c402cc60 con 0x7ff0cc0ca720 2023-11-08T16:37:29.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.030+0000 7ff0caffd640 1 -- 172.21.15.81:0/1161103600 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c40343f0 con 0x7ff0cc0ca720 2023-11-08T16:37:29.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.030+0000 7ff0d2761640 1 -- 172.21.15.81:0/1161103600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0cc0ca720 msgr2=0x7ff0cc06b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.030+0000 7ff0d2761640 1 --2- 172.21.15.81:0/1161103600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0cc0ca720 0x7ff0cc06b710 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009e50 tx=0x7ff0c402bf60 comp rx=0 tx=0).stop 2023-11-08T16:37:29.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.030+0000 7ff0d2761640 1 -- 172.21.15.81:0/1161103600 shutdown_connections 2023-11-08T16:37:29.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.030+0000 7ff0d2761640 1 --2- 172.21.15.81:0/1161103600 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0cc0ca720 0x7ff0cc06b710 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.030+0000 7ff0d2761640 1 --2- 172.21.15.81:0/1161103600 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0cc15a0e0 0x7ff0cc0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.030+0000 7ff0d2761640 1 -- 172.21.15.81:0/1161103600 >> 172.21.15.81:0/1161103600 conn(0x7ff0cc0cb580 msgr2=0x7ff0cc0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:29.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.030+0000 7ff0d2761640 1 -- 172.21.15.81:0/1161103600 shutdown_connections 2023-11-08T16:37:29.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.030+0000 7ff0d2761640 1 -- 172.21.15.81:0/1161103600 wait complete. 2023-11-08T16:37:29.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.032+0000 7ff0d2761640 1 Processor -- start 2023-11-08T16:37:29.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.032+0000 7ff0d2761640 1 -- start start 2023-11-08T16:37:29.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.033+0000 7ff0d2761640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0cc0c9bf0 0x7ff0cc0c4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.033+0000 7ff0d2761640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0cc0ca720 0x7ff0cc0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.033+0000 7ff0d2761640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0cc15a0e0 0x7ff0cc0c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.033+0000 7ff0d2761640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0c7170 con 0x7ff0cc0ca720 2023-11-08T16:37:29.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.033+0000 7ff0d2761640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0c4fb0 con 0x7ff0cc15a0e0 2023-11-08T16:37:29.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.033+0000 7ff0d2761640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0c3730 con 0x7ff0cc0c9bf0 2023-11-08T16:37:29.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.033+0000 7ff0cb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0cc0ca720 0x7ff0cc0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.033+0000 7ff0cb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0cc0ca720 0x7ff0cc0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42886/0 (socket says 172.21.15.81:42886) 2023-11-08T16:37:29.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.033+0000 7ff0cb7fe640 1 -- 172.21.15.81:0/4202504799 learned_addr learned my addr 172.21.15.81:0/4202504799 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:29.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0cbfff640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0cc0c9bf0 0x7ff0cc0c4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0d0cfe640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0cc15a0e0 0x7ff0cc0c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0cb7fe640 1 -- 172.21.15.81:0/4202504799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0cc0c9bf0 msgr2=0x7ff0cc0c4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0cb7fe640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0cc0c9bf0 0x7ff0cc0c4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0cb7fe640 1 -- 172.21.15.81:0/4202504799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0cc15a0e0 msgr2=0x7ff0cc0c6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0cb7fe640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0cc15a0e0 0x7ff0cc0c6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0cb7fe640 1 -- 172.21.15.81:0/4202504799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4009b30 con 0x7ff0cc0ca720 2023-11-08T16:37:29.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0d0cfe640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0cc15a0e0 0x7ff0cc0c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:29.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0cb7fe640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0cc0ca720 0x7ff0cc0c66f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000b3f0 tx=0x7ff0c000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:29.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0c8ff9640 1 -- 172.21.15.81:0/4202504799 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0011830 con 0x7ff0cc0ca720 2023-11-08T16:37:29.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0c8ff9640 1 -- 172.21.15.81:0/4202504799 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0011990 con 0x7ff0cc0ca720 2023-11-08T16:37:29.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0c8ff9640 1 -- 172.21.15.81:0/4202504799 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0011d40 con 0x7ff0cc0ca720 2023-11-08T16:37:29.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.034+0000 7ff0d2761640 1 -- 172.21.15.81:0/4202504799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0cc0c39d0 con 0x7ff0cc0ca720 2023-11-08T16:37:29.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.035+0000 7ff0d2761640 1 -- 172.21.15.81:0/4202504799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0cc0c3f90 con 0x7ff0cc0ca720 2023-11-08T16:37:29.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.036+0000 7ff0d2761640 1 -- 172.21.15.81:0/4202504799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0cc04ece0 con 0x7ff0cc0ca720 2023-11-08T16:37:29.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.037+0000 7ff0c8ff9640 1 -- 172.21.15.81:0/4202504799 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff0c001a400 con 0x7ff0cc0ca720 2023-11-08T16:37:29.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.037+0000 7ff0c8ff9640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0b806ae70 0x7ff0b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.038+0000 7ff0c8ff9640 1 -- 172.21.15.81:0/4202504799 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff0c0089210 con 0x7ff0cc0ca720 2023-11-08T16:37:29.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.038+0000 7ff0cbfff640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0b806ae70 0x7ff0b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.039+0000 7ff0cbfff640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0b806ae70 0x7ff0b806d330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00a970 tx=0x7ff0bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:29.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.042+0000 7ff0c8ff9640 1 -- 172.21.15.81:0/4202504799 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0c0054570 con 0x7ff0cc0ca720 2023-11-08T16:37:29.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.231+0000 7ff0d2761640 1 -- 172.21.15.81:0/4202504799 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0cc15d6e0 con 0x7ff0cc0ca720 2023-11-08T16:37:29.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.232+0000 7ff0c8ff9640 1 -- 172.21.15.81:0/4202504799 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0c0057b90 con 0x7ff0cc0ca720 2023-11-08T16:37:29.234 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:37:29.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.234+0000 7ff0d2761640 1 -- 172.21.15.81:0/4202504799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0b806ae70 msgr2=0x7ff0b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.234+0000 7ff0d2761640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0b806ae70 0x7ff0b806d330 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc00a970 tx=0x7ff0bc008040 comp rx=0 tx=0).stop 2023-11-08T16:37:29.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.235+0000 7ff0d2761640 1 -- 172.21.15.81:0/4202504799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0cc0ca720 msgr2=0x7ff0cc0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.235+0000 7ff0d2761640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0cc0ca720 0x7ff0cc0c66f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000b3f0 tx=0x7ff0c000b8b0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.235+0000 7ff0d2761640 1 -- 172.21.15.81:0/4202504799 shutdown_connections 2023-11-08T16:37:29.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.235+0000 7ff0d2761640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff0b806ae70 0x7ff0b806d330 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.235+0000 7ff0d2761640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0cc15a0e0 0x7ff0cc0c6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.235+0000 7ff0d2761640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff0cc0ca720 0x7ff0cc0c66f0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.235+0000 7ff0d2761640 1 --2- 172.21.15.81:0/4202504799 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0cc0c9bf0 0x7ff0cc0c4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.235+0000 7ff0d2761640 1 -- 172.21.15.81:0/4202504799 >> 172.21.15.81:0/4202504799 conn(0x7ff0cc0cb580 msgr2=0x7ff0cc0ca010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:29.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.235+0000 7ff0d2761640 1 -- 172.21.15.81:0/4202504799 shutdown_connections 2023-11-08T16:37:29.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.235+0000 7ff0d2761640 1 -- 172.21.15.81:0/4202504799 wait complete. 2023-11-08T16:37:29.255 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T16:37:29.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.465+0000 7f8c193ac640 1 Processor -- start 2023-11-08T16:37:29.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.465+0000 7f8c193ac640 1 -- start start 2023-11-08T16:37:29.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.466+0000 7f8c193ac640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8c1415ac20 0x7f8c140cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.466+0000 7f8c193ac640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c140ad940 0x7f8c1406b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.466+0000 7f8c193ac640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c1406cd70 con 0x7f8c1406f7c0 2023-11-08T16:37:29.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.466+0000 7f8c193ac640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c140ae130 con 0x7f8c1415ac20 2023-11-08T16:37:29.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.466+0000 7f8c193ac640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c140cb370 con 0x7f8c140ad940 2023-11-08T16:37:29.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.467+0000 7f8c13fff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8c1415ac20 0x7f8c140cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.467+0000 7f8c18bab640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c140ad940 0x7f8c1406b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.467+0000 7f8c13fff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8c1415ac20 0x7f8c140cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:55738/0 (socket says 172.21.15.81:55738) 2023-11-08T16:37:29.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.467+0000 7f8c13fff640 1 -- 172.21.15.81:0/1210997946 learned_addr learned my addr 172.21.15.81:0/1210997946 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:29.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.468+0000 7f8c13fff640 1 -- 172.21.15.81:0/1210997946 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c140ad940 msgr2=0x7f8c1406b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:29.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.468+0000 7f8c13fff640 1 --2- 172.21.15.81:0/1210997946 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c140ad940 0x7f8c1406b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.468+0000 7f8c13fff640 1 -- 172.21.15.81:0/1210997946 >> v1:172.21.15.93:6789/0 conn(0x7f8c1406f7c0 legacy=0x7f8c14070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.468+0000 7f8c13fff640 1 -- 172.21.15.81:0/1210997946 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c140ca500 con 0x7f8c1415ac20 2023-11-08T16:37:29.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.469+0000 7f8c13fff640 1 --2- 172.21.15.81:0/1210997946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c1415ac20 0x7f8c140cbaa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c040098a0 tx=0x7f8c0402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e1aeda45b6535c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:29.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.470+0000 7f8c137fe640 1 -- 172.21.15.81:0/1210997946 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04031640 con 0x7f8c1415ac20 2023-11-08T16:37:29.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.470+0000 7f8c137fe640 1 -- 172.21.15.81:0/1210997946 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c040317a0 con 0x7f8c1415ac20 2023-11-08T16:37:29.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.470+0000 7f8c137fe640 1 -- 172.21.15.81:0/1210997946 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04031b30 con 0x7f8c1415ac20 2023-11-08T16:37:29.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.470+0000 7f8c193ac640 1 -- 172.21.15.81:0/1210997946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c1415ac20 msgr2=0x7f8c140cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.470+0000 7f8c193ac640 1 --2- 172.21.15.81:0/1210997946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c1415ac20 0x7f8c140cbaa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8c040098a0 tx=0x7f8c0402b8f0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.470+0000 7f8c193ac640 1 -- 172.21.15.81:0/1210997946 shutdown_connections 2023-11-08T16:37:29.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.470+0000 7f8c193ac640 1 --2- 172.21.15.81:0/1210997946 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c140ad940 0x7f8c1406b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.470+0000 7f8c193ac640 1 --2- 172.21.15.81:0/1210997946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c1415ac20 0x7f8c140cbaa0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.470+0000 7f8c193ac640 1 -- 172.21.15.81:0/1210997946 >> 172.21.15.81:0/1210997946 conn(0x7f8c140b04d0 msgr2=0x7f8c140b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:29.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.470+0000 7f8c193ac640 1 -- 172.21.15.81:0/1210997946 shutdown_connections 2023-11-08T16:37:29.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.470+0000 7f8c193ac640 1 -- 172.21.15.81:0/1210997946 wait complete. 2023-11-08T16:37:29.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.472+0000 7f8c193ac640 1 Processor -- start 2023-11-08T16:37:29.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.472+0000 7f8c193ac640 1 -- start start 2023-11-08T16:37:29.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.473+0000 7f8c193ac640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c1406f7c0 0x7f8c14156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.473+0000 7f8c193ac640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c140ad940 0x7f8c141513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.473+0000 7f8c193ac640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c1415ac20 0x7f8c140b3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.473+0000 7f8c193ac640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c14151920 con 0x7f8c140ad940 2023-11-08T16:37:29.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.473+0000 7f8c193ac640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c14156920 con 0x7f8c1415ac20 2023-11-08T16:37:29.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.473+0000 7f8c193ac640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c14156c00 con 0x7f8c1406f7c0 2023-11-08T16:37:29.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.473+0000 7f8c0bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c140ad940 0x7f8c141513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c0bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c140ad940 0x7f8c141513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42902/0 (socket says 172.21.15.81:42902) 2023-11-08T16:37:29.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c0bfff640 1 -- 172.21.15.81:0/3931556414 learned_addr learned my addr 172.21.15.81:0/3931556414 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:29.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c18bab640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c1415ac20 0x7f8c140b3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c13fff640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c1406f7c0 0x7f8c14156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c0bfff640 1 -- 172.21.15.81:0/3931556414 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c1406f7c0 msgr2=0x7f8c14156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c0bfff640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c1406f7c0 0x7f8c14156340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c0bfff640 1 -- 172.21.15.81:0/3931556414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c1415ac20 msgr2=0x7f8c140b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c0bfff640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c1415ac20 0x7f8c140b3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c0bfff640 1 -- 172.21.15.81:0/3931556414 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c040094d0 con 0x7f8c140ad940 2023-11-08T16:37:29.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c18bab640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c1415ac20 0x7f8c140b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:29.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c0bfff640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c140ad940 0x7f8c141513e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00c3c0 tx=0x7f8bfc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:29.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.474+0000 7f8c117fa640 1 -- 172.21.15.81:0/3931556414 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc0052d0 con 0x7f8c140ad940 2023-11-08T16:37:29.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.475+0000 7f8c117fa640 1 -- 172.21.15.81:0/3931556414 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bfc005450 con 0x7f8c140ad940 2023-11-08T16:37:29.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.475+0000 7f8c193ac640 1 -- 172.21.15.81:0/3931556414 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c14156ee0 con 0x7f8c140ad940 2023-11-08T16:37:29.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.475+0000 7f8c117fa640 1 -- 172.21.15.81:0/3931556414 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc007260 con 0x7f8c140ad940 2023-11-08T16:37:29.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.475+0000 7f8c193ac640 1 -- 172.21.15.81:0/3931556414 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c141574a0 con 0x7f8c140ad940 2023-11-08T16:37:29.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.476+0000 7f8c193ac640 1 -- 172.21.15.81:0/3931556414 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1404ece0 con 0x7f8c140ad940 2023-11-08T16:37:29.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.477+0000 7f8c117fa640 1 -- 172.21.15.81:0/3931556414 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8bfc0073c0 con 0x7f8c140ad940 2023-11-08T16:37:29.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.478+0000 7f8c117fa640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bf406aea0 0x7f8bf406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.478+0000 7f8c117fa640 1 -- 172.21.15.81:0/3931556414 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8bfc088680 con 0x7f8c140ad940 2023-11-08T16:37:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.478+0000 7f8c13fff640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bf406aea0 0x7f8bf406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.479+0000 7f8c13fff640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bf406aea0 0x7f8bf406d360 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8c04036000 tx=0x7f8c0403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:29.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.482+0000 7f8c117fa640 1 -- 172.21.15.81:0/3931556414 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8bfc053a70 con 0x7f8c140ad940 2023-11-08T16:37:29.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.674+0000 7f8c193ac640 1 -- 172.21.15.81:0/3931556414 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8c140cbd90 con 0x7f8c140ad940 2023-11-08T16:37:29.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.675+0000 7f8c117fa640 1 -- 172.21.15.81:0/3931556414 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) v1 ==== 84+0+65 (secure 0 0 0) 0x7f8bfc057090 con 0x7f8c140ad940 2023-11-08T16:37:29.677 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:29.677 INFO:teuthology.orchestra.run.smithi081.stdout:[ 2023-11-08T16:37:29.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:37:29.678 INFO:teuthology.orchestra.run.smithi081.stdout: "poolnum": 1, 2023-11-08T16:37:29.678 INFO:teuthology.orchestra.run.smithi081.stdout: "poolname": ".mgr" 2023-11-08T16:37:29.678 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:29.678 INFO:teuthology.orchestra.run.smithi081.stdout:] 2023-11-08T16:37:29.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 -- 172.21.15.81:0/3931556414 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bf406aea0 msgr2=0x7f8bf406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bf406aea0 0x7f8bf406d360 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8c04036000 tx=0x7f8c0403c040 comp rx=0 tx=0).stop 2023-11-08T16:37:29.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 -- 172.21.15.81:0/3931556414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c140ad940 msgr2=0x7f8c141513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c140ad940 0x7f8c141513e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc00c3c0 tx=0x7f8bfc00c880 comp rx=0 tx=0).stop 2023-11-08T16:37:29.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 -- 172.21.15.81:0/3931556414 shutdown_connections 2023-11-08T16:37:29.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bf406aea0 0x7f8bf406d360 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c1415ac20 0x7f8c140b3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8c140ad940 0x7f8c141513e0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 --2- 172.21.15.81:0/3931556414 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8c1406f7c0 0x7f8c14156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 -- 172.21.15.81:0/3931556414 >> 172.21.15.81:0/3931556414 conn(0x7f8c140b04d0 msgr2=0x7f8c140cda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:29.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 -- 172.21.15.81:0/3931556414 shutdown_connections 2023-11-08T16:37:29.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.678+0000 7f8c193ac640 1 -- 172.21.15.81:0/3931556414 wait complete. 2023-11-08T16:37:29.697 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T16:37:29.698 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T16:37:29.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.910+0000 7fce54d1a640 1 Processor -- start 2023-11-08T16:37:29.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.910+0000 7fce54d1a640 1 -- start start 2023-11-08T16:37:29.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.912+0000 7fce54d1a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fce500cd0e0 0x7fce5006c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.912+0000 7fce54d1a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fce5015d810 0x7fce5006cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.912+0000 7fce54d1a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce5015dd90 con 0x7fce50159bb0 2023-11-08T16:37:29.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.912+0000 7fce54d1a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5015e050 con 0x7fce5015d810 2023-11-08T16:37:29.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.912+0000 7fce54d1a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce5015e330 con 0x7fce500cd0e0 2023-11-08T16:37:29.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.912+0000 7fce4ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fce5015d810 0x7fce5006cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.912+0000 7fce4ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fce5015d810 0x7fce5006cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42904/0 (socket says 172.21.15.81:42904) 2023-11-08T16:37:29.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.912+0000 7fce4ffff640 1 -- 172.21.15.81:0/278395587 learned_addr learned my addr 172.21.15.81:0/278395587 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:29.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.912+0000 7fce47fff640 1 --2- 172.21.15.81:0/278395587 >> v2:172.21.15.93:3300/0 conn(0x7fce500cd0e0 0x7fce5006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.913+0000 7fce4ffff640 1 -- 172.21.15.81:0/278395587 >> v2:172.21.15.93:3300/0 conn(0x7fce500cd0e0 msgr2=0x7fce5006c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:29.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.913+0000 7fce4ffff640 1 --2- 172.21.15.81:0/278395587 >> v2:172.21.15.93:3300/0 conn(0x7fce500cd0e0 0x7fce5006c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.913+0000 7fce4ffff640 1 -- 172.21.15.81:0/278395587 >> v1:172.21.15.93:6789/0 conn(0x7fce50159bb0 legacy=0x7fce500c91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:29.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.913+0000 7fce4ffff640 1 -- 172.21.15.81:0/278395587 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5015e610 con 0x7fce5015d810 2023-11-08T16:37:29.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.913+0000 7fce47fff640 1 --2- 172.21.15.81:0/278395587 >> v2:172.21.15.93:3300/0 conn(0x7fce500cd0e0 0x7fce5006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:37:29.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.914+0000 7fce4ffff640 1 --2- 172.21.15.81:0/278395587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce5015d810 0x7fce5006cbb0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fce500b1510 tx=0x7fce4802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17dc707eb349dbd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:29.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.914+0000 7fce4effd640 1 -- 172.21.15.81:0/278395587 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4802c9f0 con 0x7fce5015d810 2023-11-08T16:37:29.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.914+0000 7fce4effd640 1 -- 172.21.15.81:0/278395587 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce4802cb50 con 0x7fce5015d810 2023-11-08T16:37:29.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.914+0000 7fce4effd640 1 -- 172.21.15.81:0/278395587 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce480343f0 con 0x7fce5015d810 2023-11-08T16:37:29.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.915+0000 7fce54d1a640 1 -- 172.21.15.81:0/278395587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce5015d810 msgr2=0x7fce5006cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.915+0000 7fce54d1a640 1 --2- 172.21.15.81:0/278395587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce5015d810 0x7fce5006cbb0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fce500b1510 tx=0x7fce4802be50 comp rx=0 tx=0).stop 2023-11-08T16:37:29.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.915+0000 7fce54d1a640 1 -- 172.21.15.81:0/278395587 shutdown_connections 2023-11-08T16:37:29.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.915+0000 7fce54d1a640 1 --2- 172.21.15.81:0/278395587 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce5015d810 0x7fce5006cbb0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.915+0000 7fce54d1a640 1 --2- 172.21.15.81:0/278395587 >> v2:172.21.15.93:3300/0 conn(0x7fce500cd0e0 0x7fce5006c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.915+0000 7fce54d1a640 1 -- 172.21.15.81:0/278395587 >> 172.21.15.81:0/278395587 conn(0x7fce500cb410 msgr2=0x7fce500cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:29.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.915+0000 7fce54d1a640 1 -- 172.21.15.81:0/278395587 shutdown_connections 2023-11-08T16:37:29.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.915+0000 7fce54d1a640 1 -- 172.21.15.81:0/278395587 wait complete. 2023-11-08T16:37:29.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.917+0000 7fce54d1a640 1 Processor -- start 2023-11-08T16:37:29.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.917+0000 7fce54d1a640 1 -- start start 2023-11-08T16:37:29.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.918+0000 7fce54d1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce500cd0e0 0x7fce500b2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.918+0000 7fce54d1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fce50159bb0 0x7fce500c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.918+0000 7fce54d1a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce5015d810 0x7fce500c2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.918+0000 7fce54d1a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b3e00 con 0x7fce500cd0e0 2023-11-08T16:37:29.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.918+0000 7fce54d1a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b31f0 con 0x7fce5015d810 2023-11-08T16:37:29.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.918+0000 7fce54d1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce500b34d0 con 0x7fce50159bb0 2023-11-08T16:37:29.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.918+0000 7fce4f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce500cd0e0 0x7fce500b2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.918+0000 7fce4f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce500cd0e0 0x7fce500b2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42914/0 (socket says 172.21.15.81:42914) 2023-11-08T16:37:29.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.918+0000 7fce4f7fe640 1 -- 172.21.15.81:0/3654157668 learned_addr learned my addr 172.21.15.81:0/3654157668 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:29.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.919+0000 7fce4f7fe640 1 -- 172.21.15.81:0/3654157668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fce50159bb0 msgr2=0x7fce500c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.919+0000 7fce4f7fe640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fce50159bb0 0x7fce500c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.919+0000 7fce4f7fe640 1 -- 172.21.15.81:0/3654157668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce5015d810 msgr2=0x7fce500c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:29.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.919+0000 7fce4f7fe640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce5015d810 0x7fce500c2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:29.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.919+0000 7fce4f7fe640 1 -- 172.21.15.81:0/3654157668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce48009b30 con 0x7fce500cd0e0 2023-11-08T16:37:29.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.919+0000 7fce4f7fe640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce500cd0e0 0x7fce500b2730 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c430 tx=0x7fce4000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:29.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.919+0000 7fce4cff9640 1 -- 172.21.15.81:0/3654157668 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40011790 con 0x7fce500cd0e0 2023-11-08T16:37:29.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.919+0000 7fce4cff9640 1 -- 172.21.15.81:0/3654157668 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce400118f0 con 0x7fce500cd0e0 2023-11-08T16:37:29.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.919+0000 7fce54d1a640 1 -- 172.21.15.81:0/3654157668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce500b37b0 con 0x7fce500cd0e0 2023-11-08T16:37:29.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.919+0000 7fce4cff9640 1 -- 172.21.15.81:0/3654157668 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40011ce0 con 0x7fce500cd0e0 2023-11-08T16:37:29.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.920+0000 7fce54d1a640 1 -- 172.21.15.81:0/3654157668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce50144cf0 con 0x7fce500cd0e0 2023-11-08T16:37:29.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.920+0000 7fce54d1a640 1 -- 172.21.15.81:0/3654157668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce500b3fb0 con 0x7fce500cd0e0 2023-11-08T16:37:29.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.921+0000 7fce4cff9640 1 -- 172.21.15.81:0/3654157668 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce4001c460 con 0x7fce500cd0e0 2023-11-08T16:37:29.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.922+0000 7fce4cff9640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce3006aea0 0x7fce3006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:29.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.922+0000 7fce4cff9640 1 -- 172.21.15.81:0/3654157668 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fce40089290 con 0x7fce500cd0e0 2023-11-08T16:37:29.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.923+0000 7fce47fff640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce3006aea0 0x7fce3006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:29.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.923+0000 7fce47fff640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce3006aea0 0x7fce3006d360 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fce38006f00 tx=0x7fce38008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:29.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:29.927+0000 7fce4cff9640 1 -- 172.21.15.81:0/3654157668 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fce500b3fb0 con 0x7fce500cd0e0 2023-11-08T16:37:30.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.119+0000 7fce54d1a640 1 -- 172.21.15.81:0/3654157668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fce500c9db0 con 0x7fce500cd0e0 2023-11-08T16:37:30.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.834+0000 7fce4cff9640 1 -- 172.21.15.81:0/3654157668 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) v1 ==== 138+0+0 (secure 0 0 0) 0x7fce400545f0 con 0x7fce500cd0e0 2023-11-08T16:37:30.837 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' created 2023-11-08T16:37:30.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.838+0000 7fce54d1a640 1 -- 172.21.15.81:0/3654157668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce3006aea0 msgr2=0x7fce3006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:30.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.838+0000 7fce54d1a640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce3006aea0 0x7fce3006d360 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fce38006f00 tx=0x7fce38008040 comp rx=0 tx=0).stop 2023-11-08T16:37:30.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.838+0000 7fce54d1a640 1 -- 172.21.15.81:0/3654157668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce500cd0e0 msgr2=0x7fce500b2730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:30.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.838+0000 7fce54d1a640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce500cd0e0 0x7fce500b2730 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fce4000c430 tx=0x7fce4000c8f0 comp rx=0 tx=0).stop 2023-11-08T16:37:30.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.839+0000 7fce54d1a640 1 -- 172.21.15.81:0/3654157668 shutdown_connections 2023-11-08T16:37:30.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.839+0000 7fce54d1a640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fce3006aea0 0x7fce3006d360 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:30.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.839+0000 7fce54d1a640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce5015d810 0x7fce500c2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:30.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.839+0000 7fce54d1a640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fce50159bb0 0x7fce500c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:30.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.839+0000 7fce54d1a640 1 --2- 172.21.15.81:0/3654157668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fce500cd0e0 0x7fce500b2730 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:30.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.839+0000 7fce54d1a640 1 -- 172.21.15.81:0/3654157668 >> 172.21.15.81:0/3654157668 conn(0x7fce500cb410 msgr2=0x7fce5006c260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:30.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.839+0000 7fce54d1a640 1 -- 172.21.15.81:0/3654157668 shutdown_connections 2023-11-08T16:37:30.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:30.839+0000 7fce54d1a640 1 -- 172.21.15.81:0/3654157668 wait complete. 2023-11-08T16:37:30.866 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T16:37:31.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.079+0000 7f9afd75e640 1 Processor -- start 2023-11-08T16:37:31.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.079+0000 7f9afd75e640 1 -- start start 2023-11-08T16:37:31.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.080+0000 7f9afd75e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9af80b0770 0x7f9af806f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:31.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.081+0000 7f9afd75e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af806fe50 con 0x7f9af80b0b50 2023-11-08T16:37:31.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.081+0000 7f9afd75e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af80ca790 con 0x7f9af80ae000 2023-11-08T16:37:31.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.081+0000 7f9afd75e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af80cb3c0 con 0x7f9af80b0770 2023-11-08T16:37:31.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.081+0000 7f9af6ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9af80b0770 0x7f9af806f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:31.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.081+0000 7f9af6ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9af80b0770 0x7f9af806f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50286/0 (socket says 172.21.15.81:50286) 2023-11-08T16:37:31.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.081+0000 7f9af6ffd640 1 -- 172.21.15.81:0/3697395871 learned_addr learned my addr 172.21.15.81:0/3697395871 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:31.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.081+0000 7f9af6ffd640 1 -- 172.21.15.81:0/3697395871 >> v1:172.21.15.93:6789/0 conn(0x7f9af80ae000 legacy=0x7f9af80cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:31.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.081+0000 7f9af6ffd640 1 -- 172.21.15.81:0/3697395871 >> v1:172.21.15.81:6789/0 conn(0x7f9af80b0b50 legacy=0x7f9af80b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:31.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.082+0000 7f9af6ffd640 1 -- 172.21.15.81:0/3697395871 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af80cad70 con 0x7f9af80b0770 2023-11-08T16:37:31.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.081+0000 7f9af5ffb640 1 -- 172.21.15.81:0/3697395871 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359533557 0 0) 0x7f9af0001490 con 0x7f9af80b0b50 2023-11-08T16:37:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.083+0000 7f9af6ffd640 1 --2- 172.21.15.81:0/3697395871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9af80b0770 0x7f9af806f910 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009870 tx=0x7f9ae802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb4fa20be51ac36 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.083+0000 7f9af5ffb640 1 -- 172.21.15.81:0/3697395871 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8030860 con 0x7f9af80b0770 2023-11-08T16:37:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.083+0000 7f9af5ffb640 1 -- 172.21.15.81:0/3697395871 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae80309c0 con 0x7f9af80b0770 2023-11-08T16:37:31.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.083+0000 7f9af5ffb640 1 -- 172.21.15.81:0/3697395871 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8030d50 con 0x7f9af80b0770 2023-11-08T16:37:31.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.083+0000 7f9afd75e640 1 -- 172.21.15.81:0/3697395871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9af80b0770 msgr2=0x7f9af806f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:31.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.083+0000 7f9afd75e640 1 --2- 172.21.15.81:0/3697395871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9af80b0770 0x7f9af806f910 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009870 tx=0x7f9ae802b9a0 comp rx=0 tx=0).stop 2023-11-08T16:37:31.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.084+0000 7f9afd75e640 1 -- 172.21.15.81:0/3697395871 shutdown_connections 2023-11-08T16:37:31.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.084+0000 7f9afd75e640 1 --2- 172.21.15.81:0/3697395871 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9af80b0770 0x7f9af806f910 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:31.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.084+0000 7f9afd75e640 1 -- 172.21.15.81:0/3697395871 >> 172.21.15.81:0/3697395871 conn(0x7f9af80cb500 msgr2=0x7f9af80cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:31.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.084+0000 7f9afd75e640 1 -- 172.21.15.81:0/3697395871 shutdown_connections 2023-11-08T16:37:31.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.084+0000 7f9afd75e640 1 -- 172.21.15.81:0/3697395871 wait complete. 2023-11-08T16:37:31.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.085+0000 7f9afd75e640 1 Processor -- start 2023-11-08T16:37:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.086+0000 7f9afd75e640 1 -- start start 2023-11-08T16:37:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.086+0000 7f9afd75e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9af80ae000 0x7f9af8154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.086+0000 7f9afd75e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9af80b0770 0x7f9af806c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:31.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.086+0000 7f9afd75e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9af80b0b50 0x7f9af81f1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9afd75e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8159310 con 0x7f9af80ae000 2023-11-08T16:37:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9afd75e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8156140 con 0x7f9af80b0770 2023-11-08T16:37:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9afd75e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af81558a0 con 0x7f9af80b0b50 2023-11-08T16:37:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9af6ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9af80ae000 0x7f9af8154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:31.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9af6ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9af80ae000 0x7f9af8154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50294/0 (socket says 172.21.15.81:50294) 2023-11-08T16:37:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9af6ffd640 1 -- 172.21.15.81:0/788382536 learned_addr learned my addr 172.21.15.81:0/788382536 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9af6ffd640 1 -- 172.21.15.81:0/788382536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9af80b0b50 msgr2=0x7f9af81f1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9af6ffd640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9af80b0b50 0x7f9af81f1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9af6ffd640 1 -- 172.21.15.81:0/788382536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9af80b0770 msgr2=0x7f9af806c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:31.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9af67fc640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9af80b0770 0x7f9af806c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9af6ffd640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9af80b0770 0x7f9af806c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.087+0000 7f9af6ffd640 1 -- 172.21.15.81:0/788382536 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80094d0 con 0x7f9af80ae000 2023-11-08T16:37:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.088+0000 7f9af6ffd640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9af80ae000 0x7f9af8154fa0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009870 tx=0x7f9ae8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:31.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.088+0000 7f9aef7fe640 1 -- 172.21.15.81:0/788382536 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8039d60 con 0x7f9af80ae000 2023-11-08T16:37:31.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.088+0000 7f9aef7fe640 1 -- 172.21.15.81:0/788382536 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ae8043400 con 0x7f9af80ae000 2023-11-08T16:37:31.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.088+0000 7f9afd75e640 1 -- 172.21.15.81:0/788382536 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af8155b80 con 0x7f9af80ae000 2023-11-08T16:37:31.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.088+0000 7f9aef7fe640 1 -- 172.21.15.81:0/788382536 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ae80437b0 con 0x7f9af80ae000 2023-11-08T16:37:31.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.088+0000 7f9afd75e640 1 -- 172.21.15.81:0/788382536 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af81594f0 con 0x7f9af80ae000 2023-11-08T16:37:31.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.089+0000 7f9afd75e640 1 -- 172.21.15.81:0/788382536 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af80743f0 con 0x7f9af80ae000 2023-11-08T16:37:31.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.091+0000 7f9aef7fe640 1 -- 172.21.15.81:0/788382536 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ae8043910 con 0x7f9af80ae000 2023-11-08T16:37:31.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.092+0000 7f9aef7fe640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9ad806ae70 0x7f9ad806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:31.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.092+0000 7f9aef7fe640 1 -- 172.21.15.81:0/788382536 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9ae803e070 con 0x7f9af80ae000 2023-11-08T16:37:31.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.092+0000 7f9af67fc640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9ad806ae70 0x7f9ad806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:31.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.093+0000 7f9af67fc640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9ad806ae70 0x7f9ad806d330 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9af80cbff0 tx=0x7f9ae0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:31.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.096+0000 7f9aef7fe640 1 -- 172.21.15.81:0/788382536 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9ae80784c0 con 0x7f9af80ae000 2023-11-08T16:37:31.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.290+0000 7f9afd75e640 1 -- 172.21.15.81:0/788382536 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9af8159710 con 0x7f9af80ae000 2023-11-08T16:37:31.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.853+0000 7f9aef7fe640 1 -- 172.21.15.81:0/788382536 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9ae807bae0 con 0x7f9af80ae000 2023-11-08T16:37:31.855 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' created 2023-11-08T16:37:31.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 -- 172.21.15.81:0/788382536 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9ad806ae70 msgr2=0x7f9ad806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:31.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9ad806ae70 0x7f9ad806d330 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f9af80cbff0 tx=0x7f9ae0009040 comp rx=0 tx=0).stop 2023-11-08T16:37:31.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 -- 172.21.15.81:0/788382536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9af80ae000 msgr2=0x7f9af8154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:31.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9af80ae000 0x7f9af8154fa0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f9ae8009870 tx=0x7f9ae8004730 comp rx=0 tx=0).stop 2023-11-08T16:37:31.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 -- 172.21.15.81:0/788382536 shutdown_connections 2023-11-08T16:37:31.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9ad806ae70 0x7f9ad806d330 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:31.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9af80b0b50 0x7f9af81f1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:31.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9af80b0770 0x7f9af806c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:31.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 --2- 172.21.15.81:0/788382536 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9af80ae000 0x7f9af8154fa0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:31.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 -- 172.21.15.81:0/788382536 >> 172.21.15.81:0/788382536 conn(0x7f9af80cb500 msgr2=0x7f9af80cc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:31.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 -- 172.21.15.81:0/788382536 shutdown_connections 2023-11-08T16:37:31.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:31.857+0000 7f9afd75e640 1 -- 172.21.15.81:0/788382536 wait complete. 2023-11-08T16:37:31.895 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T16:37:32.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.082+0000 7f42d759e640 1 Processor -- start 2023-11-08T16:37:32.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.082+0000 7f42d759e640 1 -- start start 2023-11-08T16:37:32.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.084+0000 7f42d759e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f42d815b250 0x7f42d815a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:32.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.084+0000 7f42d759e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42d80b11f0 0x7f42d8157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:32.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.084+0000 7f42d759e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d8157a80 con 0x7f42d81576a0 2023-11-08T16:37:32.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.084+0000 7f42d759e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d815c360 con 0x7f42d815b250 2023-11-08T16:37:32.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.084+0000 7f42d759e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d815b6e0 con 0x7f42d80b11f0 2023-11-08T16:37:32.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.084+0000 7f42cffff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f42d815b250 0x7f42d815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:32.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.084+0000 7f42cffff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f42d815b250 0x7f42d815a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:48084/0 (socket says 172.21.15.81:48084) 2023-11-08T16:37:32.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.084+0000 7f42cffff640 1 -- 172.21.15.81:0/3266575263 learned_addr learned my addr 172.21.15.81:0/3266575263 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:32.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.084+0000 7f42d6d9d640 1 --2- 172.21.15.81:0/3266575263 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42d80b11f0 0x7f42d8157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:32.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.085+0000 7f42cffff640 1 -- 172.21.15.81:0/3266575263 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42d80b11f0 msgr2=0x7f42d8157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:32.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.085+0000 7f42cffff640 1 --2- 172.21.15.81:0/3266575263 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42d80b11f0 0x7f42d8157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:32.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.085+0000 7f42cffff640 1 -- 172.21.15.81:0/3266575263 >> v1:172.21.15.93:6789/0 conn(0x7f42d81576a0 legacy=0x7f42d8070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:32.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.085+0000 7f42cffff640 1 -- 172.21.15.81:0/3266575263 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d815b9c0 con 0x7f42d815b250 2023-11-08T16:37:32.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.086+0000 7f42cffff640 1 --2- 172.21.15.81:0/3266575263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42d815b250 0x7f42d815a010 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009850 tx=0x7f42c002b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d05baaad22d94338 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:32.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.087+0000 7f42d5d9b640 1 -- 172.21.15.81:0/3266575263 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0004230 con 0x7f42d815b250 2023-11-08T16:37:32.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.087+0000 7f42d5d9b640 1 -- 172.21.15.81:0/3266575263 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c0004390 con 0x7f42d815b250 2023-11-08T16:37:32.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.087+0000 7f42d5d9b640 1 -- 172.21.15.81:0/3266575263 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c0031830 con 0x7f42d815b250 2023-11-08T16:37:32.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.088+0000 7f42d759e640 1 -- 172.21.15.81:0/3266575263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42d815b250 msgr2=0x7f42d815a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:32.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.088+0000 7f42d759e640 1 --2- 172.21.15.81:0/3266575263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42d815b250 0x7f42d815a010 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f42c0009850 tx=0x7f42c002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:37:32.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.088+0000 7f42d759e640 1 -- 172.21.15.81:0/3266575263 shutdown_connections 2023-11-08T16:37:32.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.088+0000 7f42d759e640 1 --2- 172.21.15.81:0/3266575263 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42d80b11f0 0x7f42d8157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:32.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.088+0000 7f42d759e640 1 --2- 172.21.15.81:0/3266575263 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42d815b250 0x7f42d815a010 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:32.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.088+0000 7f42d759e640 1 -- 172.21.15.81:0/3266575263 >> 172.21.15.81:0/3266575263 conn(0x7f42d80ce8d0 msgr2=0x7f42d80cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:32.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.088+0000 7f42d759e640 1 -- 172.21.15.81:0/3266575263 shutdown_connections 2023-11-08T16:37:32.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.088+0000 7f42d759e640 1 -- 172.21.15.81:0/3266575263 wait complete. 2023-11-08T16:37:32.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.090+0000 7f42d759e640 1 Processor -- start 2023-11-08T16:37:32.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.090+0000 7f42d759e640 1 -- start start 2023-11-08T16:37:32.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.091+0000 7f42d759e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42d80b11f0 0x7f42d8151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:32.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.091+0000 7f42d759e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42d81576a0 0x7f42d814f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:32.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.091+0000 7f42d759e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42d80caa30 0x7f42d8155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:32.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.091+0000 7f42d759e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8152260 con 0x7f42d80caa30 2023-11-08T16:37:32.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.091+0000 7f42d759e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8154c90 con 0x7f42d80b11f0 2023-11-08T16:37:32.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.091+0000 7f42d759e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d814fe00 con 0x7f42d81576a0 2023-11-08T16:37:32.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.092+0000 7f42d6d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42d80caa30 0x7f42d8155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:32.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.092+0000 7f42d6d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42d80caa30 0x7f42d8155400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50300/0 (socket says 172.21.15.81:50300) 2023-11-08T16:37:32.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.092+0000 7f42d6d9d640 1 -- 172.21.15.81:0/3735248387 learned_addr learned my addr 172.21.15.81:0/3735248387 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:32.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.092+0000 7f42d659c640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42d80b11f0 0x7f42d8151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:32.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.092+0000 7f42cffff640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42d81576a0 0x7f42d814f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:32.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.092+0000 7f42d6d9d640 1 -- 172.21.15.81:0/3735248387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42d81576a0 msgr2=0x7f42d814f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:32.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.092+0000 7f42d6d9d640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42d81576a0 0x7f42d814f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:32.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.092+0000 7f42d6d9d640 1 -- 172.21.15.81:0/3735248387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42d80b11f0 msgr2=0x7f42d8151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:32.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.092+0000 7f42d6d9d640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42d80b11f0 0x7f42d8151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:32.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.092+0000 7f42d6d9d640 1 -- 172.21.15.81:0/3735248387 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c0009460 con 0x7f42d80caa30 2023-11-08T16:37:32.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.092+0000 7f42cffff640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42d81576a0 0x7f42d814f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:32.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.093+0000 7f42d6d9d640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42d80caa30 0x7f42d8155400 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f42d000efc0 tx=0x7f42d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:32.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.093+0000 7f42ceffd640 1 -- 172.21.15.81:0/3735248387 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d0009ca0 con 0x7f42d80caa30 2023-11-08T16:37:32.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.093+0000 7f42ceffd640 1 -- 172.21.15.81:0/3735248387 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d000f040 con 0x7f42d80caa30 2023-11-08T16:37:32.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.093+0000 7f42ceffd640 1 -- 172.21.15.81:0/3735248387 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d00146c0 con 0x7f42d80caa30 2023-11-08T16:37:32.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.093+0000 7f42d759e640 1 -- 172.21.15.81:0/3735248387 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d8153bf0 con 0x7f42d80caa30 2023-11-08T16:37:32.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.093+0000 7f42d759e640 1 -- 172.21.15.81:0/3735248387 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d8154ee0 con 0x7f42d80caa30 2023-11-08T16:37:32.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.094+0000 7f42d759e640 1 -- 172.21.15.81:0/3735248387 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d81550d0 con 0x7f42d80caa30 2023-11-08T16:37:32.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.096+0000 7f42ceffd640 1 -- 172.21.15.81:0/3735248387 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f42d0014820 con 0x7f42d80caa30 2023-11-08T16:37:32.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.096+0000 7f42ceffd640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f42b806aea0 0x7f42b806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:32.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.097+0000 7f42ceffd640 1 -- 172.21.15.81:0/3735248387 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f42d008aaf0 con 0x7f42d80caa30 2023-11-08T16:37:32.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.097+0000 7f42d659c640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f42b806aea0 0x7f42b806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:32.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.098+0000 7f42d659c640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f42b806aea0 0x7f42b806d360 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f42c8000f60 tx=0x7f42c8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:32.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.101+0000 7f42ceffd640 1 -- 172.21.15.81:0/3735248387 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f42d0055b60 con 0x7f42d80caa30 2023-11-08T16:37:32.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.321+0000 7f42d759e640 1 -- 172.21.15.81:0/3735248387 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f42d804ece0 con 0x7f42d80caa30 2023-11-08T16:37:32.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.957+0000 7f42ceffd640 1 -- 172.21.15.81:0/3735248387 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v60) v1 ==== 174+0+0 (secure 0 0 0) 0x7f42d00180c0 con 0x7f42d80caa30 2023-11-08T16:37:32.959 INFO:teuthology.orchestra.run.smithi081.stderr:new fs with metadata pool 10 and data pool 11 2023-11-08T16:37:32.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.961+0000 7f42d759e640 1 -- 172.21.15.81:0/3735248387 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f42b806aea0 msgr2=0x7f42b806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:32.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.961+0000 7f42d759e640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f42b806aea0 0x7f42b806d360 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f42c8000f60 tx=0x7f42c8009040 comp rx=0 tx=0).stop 2023-11-08T16:37:32.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.961+0000 7f42d759e640 1 -- 172.21.15.81:0/3735248387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42d80caa30 msgr2=0x7f42d8155400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:32.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.961+0000 7f42d759e640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42d80caa30 0x7f42d8155400 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f42d000efc0 tx=0x7f42d000c490 comp rx=0 tx=0).stop 2023-11-08T16:37:32.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.961+0000 7f42d759e640 1 -- 172.21.15.81:0/3735248387 shutdown_connections 2023-11-08T16:37:32.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.961+0000 7f42d759e640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f42b806aea0 0x7f42b806d360 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:32.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.961+0000 7f42d759e640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42d80caa30 0x7f42d8155400 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:32.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.961+0000 7f42d759e640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42d81576a0 0x7f42d814f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:32.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.961+0000 7f42d759e640 1 --2- 172.21.15.81:0/3735248387 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42d80b11f0 0x7f42d8151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:32.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.961+0000 7f42d759e640 1 -- 172.21.15.81:0/3735248387 >> 172.21.15.81:0/3735248387 conn(0x7f42d80ce8d0 msgr2=0x7f42d815ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:32.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.962+0000 7f42d759e640 1 -- 172.21.15.81:0/3735248387 shutdown_connections 2023-11-08T16:37:32.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:32.962+0000 7f42d759e640 1 -- 172.21.15.81:0/3735248387 wait complete. 2023-11-08T16:37:32.987 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:33.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.168+0000 7fa7d184b640 1 Processor -- start 2023-11-08T16:37:33.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.169+0000 7fa7d184b640 1 -- start start 2023-11-08T16:37:33.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.170+0000 7fa7d184b640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa7cc0ade50 0x7fa7cc0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.170+0000 7fa7d184b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7cc06b630 con 0x7fa7cc15a680 2023-11-08T16:37:33.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.170+0000 7fa7d184b640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7cc0cabb0 con 0x7fa7cc15ea60 2023-11-08T16:37:33.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.170+0000 7fa7d184b640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7cc0b1860 con 0x7fa7cc0ade50 2023-11-08T16:37:33.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.170+0000 7fa7ca7fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa7cc15a680 0x7fa7cc15ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53106/0 (socket says 172.21.15.81:53106) 2023-11-08T16:37:33.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.170+0000 7fa7ca7fc640 1 -- 172.21.15.81:0/2867442569 learned_addr learned my addr 172.21.15.81:0/2867442569 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:33.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7caffd640 1 --2- 172.21.15.81:0/2867442569 >> v2:172.21.15.93:3300/0 conn(0x7fa7cc0ade50 0x7fa7cc0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479668458 0 0) 0x7fa7cc06b630 con 0x7fa7cc15a680 2023-11-08T16:37:33.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7b4003520 con 0x7fa7cc15a680 2023-11-08T16:37:33.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4113712179 0 0) 0x7fa7b4003520 con 0x7fa7cc15a680 2023-11-08T16:37:33.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 >> v2:172.21.15.93:3300/0 conn(0x7fa7cc0ade50 msgr2=0x7fa7cc0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:33.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 --2- 172.21.15.81:0/2867442569 >> v2:172.21.15.93:3300/0 conn(0x7fa7cc0ade50 0x7fa7cc0ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 >> v1:172.21.15.93:6789/0 conn(0x7fa7cc15ea60 legacy=0x7fa7cc15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc0cae90 con 0x7fa7cc15a680 2023-11-08T16:37:33.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fa7c0001670 con 0x7fa7cc15a680 2023-11-08T16:37:33.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 >> v1:172.21.15.81:6789/0 conn(0x7fa7cc15a680 legacy=0x7fa7cc15ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7b4003a50 0x7fa7b4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b40047c0 0x7fa7b4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7b40051d0 0x7fa7b400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b400c0d0 con 0x7fa7b40047c0 2023-11-08T16:37:33.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b400c240 con 0x7fa7b40051d0 2023-11-08T16:37:33.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.171+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b400c400 con 0x7fa7b4003a50 2023-11-08T16:37:33.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.172+0000 7fa7ca7fc640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b40047c0 0x7fa7b4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.172+0000 7fa7caffd640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7b4003a50 0x7fa7b4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.172+0000 7fa7ca7fc640 1 -- 172.21.15.81:0/2867442569 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7b4003a50 msgr2=0x7fa7b4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.172+0000 7fa7ca7fc640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7b4003a50 0x7fa7b4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.172+0000 7fa7ca7fc640 1 -- 172.21.15.81:0/2867442569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7b40051d0 msgr2=0x7fa7b400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.172+0000 7fa7cb7fe640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7b40051d0 0x7fa7b400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.172+0000 7fa7ca7fc640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7b40051d0 0x7fa7b400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.172+0000 7fa7ca7fc640 1 -- 172.21.15.81:0/2867442569 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b4003520 con 0x7fa7b40047c0 2023-11-08T16:37:33.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.172+0000 7fa7caffd640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7b4003a50 0x7fa7b4003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:33.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.172+0000 7fa7cb7fe640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7b40051d0 0x7fa7b400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:33.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.172+0000 7fa7cb7fe640 1 -- 172.21.15.81:0/2867442569 reap_dead start 2023-11-08T16:37:33.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.173+0000 7fa7ca7fc640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b40047c0 0x7fa7b4004c40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c210 tx=0x7fa7b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5045263c968f5bec server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:33.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.173+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0005ad0 con 0x7fa7b40047c0 2023-11-08T16:37:33.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.173+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c0005c30 con 0x7fa7b40047c0 2023-11-08T16:37:33.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.173+0000 7fa7c9ffb640 1 -- 172.21.15.81:0/2867442569 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c002cad0 con 0x7fa7b40047c0 2023-11-08T16:37:33.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.174+0000 7fa7d184b640 1 -- 172.21.15.81:0/2867442569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b40047c0 msgr2=0x7fa7b4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.174+0000 7fa7d184b640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b40047c0 0x7fa7b4004c40 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c210 tx=0x7fa7b40057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.174+0000 7fa7d184b640 1 -- 172.21.15.81:0/2867442569 shutdown_connections 2023-11-08T16:37:33.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.174+0000 7fa7d184b640 1 --2- 172.21.15.81:0/2867442569 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7b40047c0 0x7fa7b4004c40 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.174+0000 7fa7d184b640 1 -- 172.21.15.81:0/2867442569 >> 172.21.15.81:0/2867442569 conn(0x7fa7cc0cb490 msgr2=0x7fa7cc0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:33.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.174+0000 7fa7d184b640 1 -- 172.21.15.81:0/2867442569 shutdown_connections 2023-11-08T16:37:33.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.174+0000 7fa7d184b640 1 -- 172.21.15.81:0/2867442569 wait complete. 2023-11-08T16:37:33.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.176+0000 7fa7d184b640 1 Processor -- start 2023-11-08T16:37:33.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.176+0000 7fa7d184b640 1 -- start start 2023-11-08T16:37:33.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.177+0000 7fa7d184b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7b40047c0 0x7fa7cc0b2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.177+0000 7fa7d184b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7cc147550 0x7fa7cc1479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.177+0000 7fa7d184b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7cc14a4b0 0x7fa7cc14a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.177+0000 7fa7d184b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1f66d0 con 0x7fa7cc147550 2023-11-08T16:37:33.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.177+0000 7fa7d184b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc14e180 con 0x7fa7b40047c0 2023-11-08T16:37:33.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.177+0000 7fa7d184b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc1f5e10 con 0x7fa7cc14a4b0 2023-11-08T16:37:33.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.177+0000 7fa7ca7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7cc147550 0x7fa7cc1479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.177+0000 7fa7ca7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7cc147550 0x7fa7cc1479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50310/0 (socket says 172.21.15.81:50310) 2023-11-08T16:37:33.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.177+0000 7fa7ca7fc640 1 -- 172.21.15.81:0/1961383258 learned_addr learned my addr 172.21.15.81:0/1961383258 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:33.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7caffd640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7b40047c0 0x7fa7cc0b2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7cb7fe640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7cc14a4b0 0x7fa7cc14a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7ca7fc640 1 -- 172.21.15.81:0/1961383258 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7cc14a4b0 msgr2=0x7fa7cc14a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7ca7fc640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7cc14a4b0 0x7fa7cc14a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7ca7fc640 1 -- 172.21.15.81:0/1961383258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7b40047c0 msgr2=0x7fa7cc0b2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7ca7fc640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7b40047c0 0x7fa7cc0b2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7ca7fc640 1 -- 172.21.15.81:0/1961383258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc1f6080 con 0x7fa7cc147550 2023-11-08T16:37:33.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7caffd640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7b40047c0 0x7fa7cc0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:33.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7cb7fe640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7cc14a4b0 0x7fa7cc14a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:33.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7ca7fc640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7cc147550 0x7fa7cc1479d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c210 tx=0x7fa7c002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:33.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7b3fff640 1 -- 172.21.15.81:0/1961383258 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c003c040 con 0x7fa7cc147550 2023-11-08T16:37:33.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7b3fff640 1 -- 172.21.15.81:0/1961383258 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c002cd60 con 0x7fa7cc147550 2023-11-08T16:37:33.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7b3fff640 1 -- 172.21.15.81:0/1961383258 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0007290 con 0x7fa7cc147550 2023-11-08T16:37:33.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.178+0000 7fa7d184b640 1 -- 172.21.15.81:0/1961383258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7cc1f6360 con 0x7fa7cc147550 2023-11-08T16:37:33.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.179+0000 7fa7d184b640 1 -- 172.21.15.81:0/1961383258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7cc147f10 con 0x7fa7cc147550 2023-11-08T16:37:33.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.180+0000 7fa7d184b640 1 -- 172.21.15.81:0/1961383258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7cc1f6520 con 0x7fa7cc147550 2023-11-08T16:37:33.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.181+0000 7fa7b3fff640 1 -- 172.21.15.81:0/1961383258 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7c0032070 con 0x7fa7cc147550 2023-11-08T16:37:33.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.182+0000 7fa7b3fff640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7b806ae70 0x7fa7b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.182+0000 7fa7b3fff640 1 -- 172.21.15.81:0/1961383258 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7c00aabf0 con 0x7fa7cc147550 2023-11-08T16:37:33.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.182+0000 7fa7caffd640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7b806ae70 0x7fa7b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.183+0000 7fa7caffd640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7b806ae70 0x7fa7b806d330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc063cf0 tx=0x7fa7bc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:33.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.186+0000 7fa7b3fff640 1 -- 172.21.15.81:0/1961383258 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa7c0075c00 con 0x7fa7cc147550 2023-11-08T16:37:33.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.376+0000 7fa7d184b640 1 -- 172.21.15.81:0/1961383258 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7cc04ece0 con 0x7fa7cc147550 2023-11-08T16:37:33.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.378+0000 7fa7b3fff640 1 -- 172.21.15.81:0/1961383258 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16521 (secure 0 0 0) 0x7fa7c0079220 con 0x7fa7cc147550 2023-11-08T16:37:33.381 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:33.381 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":80,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:37:32.322725+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:37:30.174436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T16:37:31.340720+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:33.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.381+0000 7fa7d184b640 1 -- 172.21.15.81:0/1961383258 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7b806ae70 msgr2=0x7fa7b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.381+0000 7fa7d184b640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7b806ae70 0x7fa7b806d330 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc063cf0 tx=0x7fa7bc008040 comp rx=0 tx=0).stop 2023-11-08T16:37:33.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.381+0000 7fa7d184b640 1 -- 172.21.15.81:0/1961383258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7cc147550 msgr2=0x7fa7cc1479d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.381+0000 7fa7d184b640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7cc147550 0x7fa7cc1479d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa7c000c210 tx=0x7fa7c002e070 comp rx=0 tx=0).stop 2023-11-08T16:37:33.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.381+0000 7fa7d184b640 1 -- 172.21.15.81:0/1961383258 shutdown_connections 2023-11-08T16:37:33.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.381+0000 7fa7d184b640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa7b806ae70 0x7fa7b806d330 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.381+0000 7fa7d184b640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa7cc14a4b0 0x7fa7cc14a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.381+0000 7fa7d184b640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7cc147550 0x7fa7cc1479d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.381+0000 7fa7d184b640 1 --2- 172.21.15.81:0/1961383258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7b40047c0 0x7fa7cc0b2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.381+0000 7fa7d184b640 1 -- 172.21.15.81:0/1961383258 >> 172.21.15.81:0/1961383258 conn(0x7fa7cc0cb490 msgr2=0x7fa7cc0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:33.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.381+0000 7fa7d184b640 1 -- 172.21.15.81:0/1961383258 shutdown_connections 2023-11-08T16:37:33.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.382+0000 7fa7d184b640 1 -- 172.21.15.81:0/1961383258 wait complete. 2023-11-08T16:37:33.402 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:33.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.609+0000 7f5f16fed640 1 Processor -- start 2023-11-08T16:37:33.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.610+0000 7f5f16fed640 1 -- start start 2023-11-08T16:37:33.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.611+0000 7f5f16fed640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5f1015aac0 0x7f5f100711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.611+0000 7f5f16fed640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f10070470 con 0x7f5f100716e0 2023-11-08T16:37:33.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.611+0000 7f5f16fed640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f1006d580 con 0x7f5f100cf420 2023-11-08T16:37:33.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.611+0000 7f5f16fed640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f1006f650 con 0x7f5f1015aac0 2023-11-08T16:37:33.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.612+0000 7f5f15feb640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5f1015aac0 0x7f5f100711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.612+0000 7f5f157ea640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5f100716e0 0x7f5f100ced10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53120/0 (socket says 172.21.15.81:53120) 2023-11-08T16:37:33.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.612+0000 7f5f157ea640 1 -- 172.21.15.81:0/660717114 learned_addr learned my addr 172.21.15.81:0/660717114 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:33.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.612+0000 7f5f15feb640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5f1015aac0 0x7f5f100711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50326/0 (socket says 172.21.15.81:50326) 2023-11-08T16:37:33.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.612+0000 7f5f15feb640 1 -- 172.21.15.81:0/660717114 >> v1:172.21.15.93:6789/0 conn(0x7f5f100cf420 legacy=0x7f5f1015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:33.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.612+0000 7f5f15feb640 1 -- 172.21.15.81:0/660717114 >> v1:172.21.15.81:6789/0 conn(0x7f5f100716e0 legacy=0x7f5f100ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.612+0000 7f5f15feb640 1 -- 172.21.15.81:0/660717114 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f1006ca00 con 0x7f5f1015aac0 2023-11-08T16:37:33.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.612+0000 7f5f14fe9640 1 -- 172.21.15.81:0/660717114 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461846175 0 0) 0x7f5f10070470 con 0x7f5f100716e0 2023-11-08T16:37:33.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.614+0000 7f5f15feb640 1 --2- 172.21.15.81:0/660717114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1015aac0 0x7f5f100711a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0097a0 tx=0x7f5f0c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2cca2c04695219 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:33.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.614+0000 7f5f14fe9640 1 -- 172.21.15.81:0/660717114 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c031830 con 0x7f5f1015aac0 2023-11-08T16:37:33.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.614+0000 7f5f14fe9640 1 -- 172.21.15.81:0/660717114 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0c031990 con 0x7f5f1015aac0 2023-11-08T16:37:33.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.614+0000 7f5f14fe9640 1 -- 172.21.15.81:0/660717114 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f0c031d20 con 0x7f5f1015aac0 2023-11-08T16:37:33.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.615+0000 7f5f16fed640 1 -- 172.21.15.81:0/660717114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1015aac0 msgr2=0x7f5f100711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.615+0000 7f5f16fed640 1 --2- 172.21.15.81:0/660717114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1015aac0 0x7f5f100711a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c0097a0 tx=0x7f5f0c02b8d0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.615+0000 7f5f16fed640 1 -- 172.21.15.81:0/660717114 shutdown_connections 2023-11-08T16:37:33.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.615+0000 7f5f16fed640 1 --2- 172.21.15.81:0/660717114 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1015aac0 0x7f5f100711a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.615+0000 7f5f16fed640 1 -- 172.21.15.81:0/660717114 >> 172.21.15.81:0/660717114 conn(0x7f5f1006a4a0 msgr2=0x7f5f1006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:33.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.615+0000 7f5f16fed640 1 -- 172.21.15.81:0/660717114 shutdown_connections 2023-11-08T16:37:33.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.615+0000 7f5f16fed640 1 -- 172.21.15.81:0/660717114 wait complete. 2023-11-08T16:37:33.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.617+0000 7f5f16fed640 1 Processor -- start 2023-11-08T16:37:33.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.617+0000 7f5f16fed640 1 -- start start 2023-11-08T16:37:33.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f16fed640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f100716e0 0x7f5f101f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f16fed640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f100cf420 0x7f5f100a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f16fed640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1015aac0 0x7f5f10146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f16fed640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100a9880 con 0x7f5f1015aac0 2023-11-08T16:37:33.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f16fed640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f100c4230 con 0x7f5f100cf420 2023-11-08T16:37:33.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f16fed640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f10146c70 con 0x7f5f100716e0 2023-11-08T16:37:33.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f167ec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1015aac0 0x7f5f10146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f167ec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1015aac0 0x7f5f10146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50336/0 (socket says 172.21.15.81:50336) 2023-11-08T16:37:33.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f167ec640 1 -- 172.21.15.81:0/3156076549 learned_addr learned my addr 172.21.15.81:0/3156076549 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:33.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f15feb640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f100716e0 0x7f5f101f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f157ea640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f100cf420 0x7f5f100a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f167ec640 1 -- 172.21.15.81:0/3156076549 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f100716e0 msgr2=0x7f5f101f11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f167ec640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f100716e0 0x7f5f101f11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f167ec640 1 -- 172.21.15.81:0/3156076549 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f100cf420 msgr2=0x7f5f100a9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f167ec640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f100cf420 0x7f5f100a9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f167ec640 1 -- 172.21.15.81:0/3156076549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c009400 con 0x7f5f1015aac0 2023-11-08T16:37:33.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.618+0000 7f5f15feb640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f100716e0 0x7f5f101f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:33.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.619+0000 7f5f167ec640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1015aac0 0x7f5f10146f30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c7b0 tx=0x7f5f0800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:33.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.619+0000 7f5f067fc640 1 -- 172.21.15.81:0/3156076549 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08003d80 con 0x7f5f1015aac0 2023-11-08T16:37:33.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.619+0000 7f5f067fc640 1 -- 172.21.15.81:0/3156076549 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f08005070 con 0x7f5f1015aac0 2023-11-08T16:37:33.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.619+0000 7f5f067fc640 1 -- 172.21.15.81:0/3156076549 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f08005420 con 0x7f5f1015aac0 2023-11-08T16:37:33.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.619+0000 7f5f16fed640 1 -- 172.21.15.81:0/3156076549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f101f1760 con 0x7f5f1015aac0 2023-11-08T16:37:33.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.619+0000 7f5f16fed640 1 -- 172.21.15.81:0/3156076549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f100a91c0 con 0x7f5f1015aac0 2023-11-08T16:37:33.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.620+0000 7f5f16fed640 1 -- 172.21.15.81:0/3156076549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f100a99c0 con 0x7f5f1015aac0 2023-11-08T16:37:33.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.621+0000 7f5f067fc640 1 -- 172.21.15.81:0/3156076549 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f08005580 con 0x7f5f1015aac0 2023-11-08T16:37:33.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.622+0000 7f5f067fc640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ef406ae70 0x7f5ef406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:33.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.622+0000 7f5f067fc640 1 -- 172.21.15.81:0/3156076549 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f08088af0 con 0x7f5f1015aac0 2023-11-08T16:37:33.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.622+0000 7f5f15feb640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ef406ae70 0x7f5ef406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:33.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.623+0000 7f5f15feb640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ef406ae70 0x7f5ef406d330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c036000 tx=0x7f5f0c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:33.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.627+0000 7f5f067fc640 1 -- 172.21.15.81:0/3156076549 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f100a99c0 con 0x7f5f1015aac0 2023-11-08T16:37:33.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.816+0000 7f5f16fed640 1 -- 172.21.15.81:0/3156076549 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5f100a99c0 con 0x7f5f1015aac0 2023-11-08T16:37:33.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.818+0000 7f5f067fc640 1 -- 172.21.15.81:0/3156076549 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16521 (secure 0 0 0) 0x7f5f100a99c0 con 0x7f5f1015aac0 2023-11-08T16:37:33.820 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:33.820 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":80,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:37:32.322725+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:37:30.174436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T16:37:31.340720+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:33.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.820+0000 7f5f16fed640 1 -- 172.21.15.81:0/3156076549 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ef406ae70 msgr2=0x7f5ef406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.820+0000 7f5f16fed640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ef406ae70 0x7f5ef406d330 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5f0c036000 tx=0x7f5f0c03c040 comp rx=0 tx=0).stop 2023-11-08T16:37:33.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.820+0000 7f5f16fed640 1 -- 172.21.15.81:0/3156076549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1015aac0 msgr2=0x7f5f10146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:33.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.820+0000 7f5f16fed640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1015aac0 0x7f5f10146f30 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f5f0800c7b0 tx=0x7f5f0800cc70 comp rx=0 tx=0).stop 2023-11-08T16:37:33.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.821+0000 7f5f16fed640 1 -- 172.21.15.81:0/3156076549 shutdown_connections 2023-11-08T16:37:33.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.821+0000 7f5f16fed640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ef406ae70 0x7f5ef406d330 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.821+0000 7f5f16fed640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f1015aac0 0x7f5f10146f30 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.821+0000 7f5f16fed640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f100cf420 0x7f5f100a9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.821+0000 7f5f16fed640 1 --2- 172.21.15.81:0/3156076549 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f100716e0 0x7f5f101f11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:33.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.821+0000 7f5f16fed640 1 -- 172.21.15.81:0/3156076549 >> 172.21.15.81:0/3156076549 conn(0x7f5f1006a4a0 msgr2=0x7f5f100cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:33.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.821+0000 7f5f16fed640 1 -- 172.21.15.81:0/3156076549 shutdown_connections 2023-11-08T16:37:33.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:33.821+0000 7f5f16fed640 1 -- 172.21.15.81:0/3156076549 wait complete. 2023-11-08T16:37:33.842 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T16:37:34.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.052+0000 7f62fa1b0640 1 Processor -- start 2023-11-08T16:37:34.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.052+0000 7f62fa1b0640 1 -- start start 2023-11-08T16:37:34.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.054+0000 7f62fa1b0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f62f40cd8d0 0x7f62f406e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:34.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.054+0000 7f62fa1b0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f62f40cdcb0 0x7f62f40cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:34.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.054+0000 7f62fa1b0640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f62f40cc820 0x7f62f415c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:34.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.054+0000 7f62fa1b0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f406d230 con 0x7f62f40cd8d0 2023-11-08T16:37:34.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.054+0000 7f62fa1b0640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62f40d0be0 con 0x7f62f40cc820 2023-11-08T16:37:34.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.054+0000 7f62fa1b0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f406cf50 con 0x7f62f40cdcb0 2023-11-08T16:37:34.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.054+0000 7f62f37fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f62f40cd8d0 0x7f62f406e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:34.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.054+0000 7f62f37fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f62f40cd8d0 0x7f62f406e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50342/0 (socket says 172.21.15.81:50342) 2023-11-08T16:37:34.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.054+0000 7f62f37fe640 1 -- 172.21.15.81:0/3559277717 learned_addr learned my addr 172.21.15.81:0/3559277717 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:34.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.055+0000 7f62f3fff640 1 --2- 172.21.15.81:0/3559277717 >> v2:172.21.15.93:3300/0 conn(0x7f62f40cc820 0x7f62f415c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:34.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.055+0000 7f62ebfff640 1 --2- 172.21.15.81:0/3559277717 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f62f40cdcb0 0x7f62f40cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:34.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.055+0000 7f62f37fe640 1 -- 172.21.15.81:0/3559277717 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f62f40cdcb0 msgr2=0x7f62f40cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:34.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.055+0000 7f62f37fe640 1 --2- 172.21.15.81:0/3559277717 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f62f40cdcb0 0x7f62f40cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.055+0000 7f62f37fe640 1 -- 172.21.15.81:0/3559277717 >> v2:172.21.15.93:3300/0 conn(0x7f62f40cc820 msgr2=0x7f62f415c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:34.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.055+0000 7f62f37fe640 1 --2- 172.21.15.81:0/3559277717 >> v2:172.21.15.93:3300/0 conn(0x7f62f40cc820 0x7f62f415c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.055+0000 7f62f37fe640 1 -- 172.21.15.81:0/3559277717 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40d0df0 con 0x7f62f40cd8d0 2023-11-08T16:37:34.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.055+0000 7f62f3fff640 1 --2- 172.21.15.81:0/3559277717 >> v2:172.21.15.93:3300/0 conn(0x7f62f40cc820 0x7f62f415c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:34.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.056+0000 7f62f37fe640 1 --2- 172.21.15.81:0/3559277717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62f40cd8d0 0x7f62f406e5a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f62f406de40 tx=0x7f62e402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9240c2696a36908f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:34.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.056+0000 7f62f2ffd640 1 -- 172.21.15.81:0/3559277717 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4031860 con 0x7f62f40cd8d0 2023-11-08T16:37:34.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.056+0000 7f62f2ffd640 1 -- 172.21.15.81:0/3559277717 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e40319c0 con 0x7f62f40cd8d0 2023-11-08T16:37:34.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.056+0000 7f62f2ffd640 1 -- 172.21.15.81:0/3559277717 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4031d50 con 0x7f62f40cd8d0 2023-11-08T16:37:34.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.057+0000 7f62fa1b0640 1 -- 172.21.15.81:0/3559277717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62f40cd8d0 msgr2=0x7f62f406e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:34.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.057+0000 7f62fa1b0640 1 --2- 172.21.15.81:0/3559277717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62f40cd8d0 0x7f62f406e5a0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f62f406de40 tx=0x7f62e402b900 comp rx=0 tx=0).stop 2023-11-08T16:37:34.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.057+0000 7f62fa1b0640 1 -- 172.21.15.81:0/3559277717 shutdown_connections 2023-11-08T16:37:34.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.057+0000 7f62fa1b0640 1 --2- 172.21.15.81:0/3559277717 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f62f40cdcb0 0x7f62f40cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.057+0000 7f62fa1b0640 1 --2- 172.21.15.81:0/3559277717 >> v2:172.21.15.93:3300/0 conn(0x7f62f40cc820 0x7f62f415c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.057+0000 7f62fa1b0640 1 --2- 172.21.15.81:0/3559277717 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62f40cd8d0 0x7f62f406e5a0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.057+0000 7f62fa1b0640 1 -- 172.21.15.81:0/3559277717 >> 172.21.15.81:0/3559277717 conn(0x7f62f4065df0 msgr2=0x7f62f40661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:34.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.057+0000 7f62fa1b0640 1 -- 172.21.15.81:0/3559277717 shutdown_connections 2023-11-08T16:37:34.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.057+0000 7f62fa1b0640 1 -- 172.21.15.81:0/3559277717 wait complete. 2023-11-08T16:37:34.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.059+0000 7f62fa1b0640 1 Processor -- start 2023-11-08T16:37:34.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.059+0000 7f62fa1b0640 1 -- start start 2023-11-08T16:37:34.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.060+0000 7f62fa1b0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62f40cc820 0x7f62f41f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:34.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.060+0000 7f62fa1b0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f62f40cd8d0 0x7f62f40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:34.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.060+0000 7f62fa1b0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62f40cdcb0 0x7f62f406c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:34.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.060+0000 7f62fa1b0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f415af00 con 0x7f62f40cc820 2023-11-08T16:37:34.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.060+0000 7f62fa1b0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f415abf0 con 0x7f62f40cdcb0 2023-11-08T16:37:34.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.060+0000 7f62fa1b0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62f41f5fb0 con 0x7f62f40cd8d0 2023-11-08T16:37:34.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.060+0000 7f62f37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62f40cc820 0x7f62f41f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:34.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.060+0000 7f62f37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62f40cc820 0x7f62f41f5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50348/0 (socket says 172.21.15.81:50348) 2023-11-08T16:37:34.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.060+0000 7f62f37fe640 1 -- 172.21.15.81:0/2267535540 learned_addr learned my addr 172.21.15.81:0/2267535540 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:34.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.061+0000 7f62f37fe640 1 -- 172.21.15.81:0/2267535540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f62f40cd8d0 msgr2=0x7f62f40ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:34.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.061+0000 7f62f37fe640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f62f40cd8d0 0x7f62f40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.061+0000 7f62f37fe640 1 -- 172.21.15.81:0/2267535540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62f40cdcb0 msgr2=0x7f62f406c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:34.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.061+0000 7f62f37fe640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62f40cdcb0 0x7f62f406c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.061+0000 7f62f37fe640 1 -- 172.21.15.81:0/2267535540 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e40094d0 con 0x7f62f40cc820 2023-11-08T16:37:34.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.061+0000 7f62f37fe640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62f40cc820 0x7f62f41f5a70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f62e402b940 tx=0x7f62e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:34.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.061+0000 7f62f0ff9640 1 -- 172.21.15.81:0/2267535540 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e4031850 con 0x7f62f40cc820 2023-11-08T16:37:34.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.061+0000 7f62f0ff9640 1 -- 172.21.15.81:0/2267535540 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62e40443f0 con 0x7f62f40cc820 2023-11-08T16:37:34.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.061+0000 7f62f0ff9640 1 -- 172.21.15.81:0/2267535540 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62e40447a0 con 0x7f62f40cc820 2023-11-08T16:37:34.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.061+0000 7f62fa1b0640 1 -- 172.21.15.81:0/2267535540 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f41f6230 con 0x7f62f40cc820 2023-11-08T16:37:34.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.061+0000 7f62fa1b0640 1 -- 172.21.15.81:0/2267535540 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f41f67f0 con 0x7f62f40cc820 2023-11-08T16:37:34.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.063+0000 7f62fa1b0640 1 -- 172.21.15.81:0/2267535540 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62f415b040 con 0x7f62f40cc820 2023-11-08T16:37:34.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.064+0000 7f62f0ff9640 1 -- 172.21.15.81:0/2267535540 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f62e404d450 con 0x7f62f40cc820 2023-11-08T16:37:34.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.064+0000 7f62f0ff9640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f62d406ae70 0x7f62d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:34.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.064+0000 7f62f0ff9640 1 -- 172.21.15.81:0/2267535540 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62e40abdd0 con 0x7f62f40cc820 2023-11-08T16:37:34.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.065+0000 7f62ebfff640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f62d406ae70 0x7f62d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:34.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.065+0000 7f62ebfff640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f62d406ae70 0x7f62d406d330 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f62dc006b10 tx=0x7f62dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:34.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.069+0000 7f62f0ff9640 1 -- 172.21.15.81:0/2267535540 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f62f415b040 con 0x7f62f40cc820 2023-11-08T16:37:34.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.298+0000 7f62fa1b0640 1 -- 172.21.15.81:0/2267535540 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f62f415b040 con 0x7f62f40cc820 2023-11-08T16:37:34.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.982+0000 7f62f0ff9640 1 -- 172.21.15.81:0/2267535540 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) v1 ==== 118+0+0 (secure 0 0 0) 0x7f62f415b040 con 0x7f62f40cc820 2023-11-08T16:37:34.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 -- 172.21.15.81:0/2267535540 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f62d406ae70 msgr2=0x7f62d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:34.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f62d406ae70 0x7f62d406d330 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f62dc006b10 tx=0x7f62dc008040 comp rx=0 tx=0).stop 2023-11-08T16:37:34.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 -- 172.21.15.81:0/2267535540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62f40cc820 msgr2=0x7f62f41f5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:34.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62f40cc820 0x7f62f41f5a70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f62e402b940 tx=0x7f62e40046c0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 -- 172.21.15.81:0/2267535540 shutdown_connections 2023-11-08T16:37:34.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f62d406ae70 0x7f62d406d330 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62f40cdcb0 0x7f62f406c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f62f40cd8d0 0x7f62f40ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 --2- 172.21.15.81:0/2267535540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f62f40cc820 0x7f62f41f5a70 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:34.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 -- 172.21.15.81:0/2267535540 >> 172.21.15.81:0/2267535540 conn(0x7f62f4065df0 msgr2=0x7f62f40c67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:34.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 -- 172.21.15.81:0/2267535540 shutdown_connections 2023-11-08T16:37:34.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:34.987+0000 7f62fa1b0640 1 -- 172.21.15.81:0/2267535540 wait complete. 2023-11-08T16:37:35.011 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T16:37:35.012 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T16:37:35.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.228+0000 7fd849941640 1 Processor -- start 2023-11-08T16:37:35.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.228+0000 7fd849941640 1 -- start start 2023-11-08T16:37:35.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.230+0000 7fd849941640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd8440ce0d0 0x7fd8440caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:35.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.230+0000 7fd849941640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8440cda10 con 0x7fd84415f280 2023-11-08T16:37:35.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.230+0000 7fd849941640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd84406fa30 con 0x7fd84415bfc0 2023-11-08T16:37:35.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.230+0000 7fd849941640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8440ce4b0 con 0x7fd8440ce0d0 2023-11-08T16:37:35.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.230+0000 7fd842ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd8440ce0d0 0x7fd8440caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:35.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.230+0000 7fd8437fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd84415f280 0x7fd8441516e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53126/0 (socket says 172.21.15.81:53126) 2023-11-08T16:37:35.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.230+0000 7fd8437fe640 1 -- 172.21.15.81:0/3206780845 learned_addr learned my addr 172.21.15.81:0/3206780845 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:35.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.231+0000 7fd841ffb640 1 -- 172.21.15.81:0/3206780845 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026207072 0 0) 0x7fd8440cda10 con 0x7fd84415f280 2023-11-08T16:37:35.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.231+0000 7fd841ffb640 1 -- 172.21.15.81:0/3206780845 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd82c003520 con 0x7fd84415f280 2023-11-08T16:37:35.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.231+0000 7fd842ffd640 1 -- 172.21.15.81:0/3206780845 >> v1:172.21.15.93:6789/0 conn(0x7fd84415bfc0 legacy=0x7fd84415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:35.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.231+0000 7fd842ffd640 1 -- 172.21.15.81:0/3206780845 >> v1:172.21.15.81:6789/0 conn(0x7fd84415f280 legacy=0x7fd8441516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:35.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.231+0000 7fd842ffd640 1 -- 172.21.15.81:0/3206780845 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84406fca0 con 0x7fd8440ce0d0 2023-11-08T16:37:35.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.232+0000 7fd842ffd640 1 --2- 172.21.15.81:0/3206780845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8440ce0d0 0x7fd8440caf20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd8340097a0 tx=0x7fd83402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2adb787cb4d926 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:35.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.233+0000 7fd841ffb640 1 -- 172.21.15.81:0/3206780845 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834030610 con 0x7fd8440ce0d0 2023-11-08T16:37:35.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.233+0000 7fd841ffb640 1 -- 172.21.15.81:0/3206780845 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd834030770 con 0x7fd8440ce0d0 2023-11-08T16:37:35.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.233+0000 7fd841ffb640 1 -- 172.21.15.81:0/3206780845 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd834030ae0 con 0x7fd8440ce0d0 2023-11-08T16:37:35.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.233+0000 7fd849941640 1 -- 172.21.15.81:0/3206780845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8440ce0d0 msgr2=0x7fd8440caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:35.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.233+0000 7fd849941640 1 --2- 172.21.15.81:0/3206780845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8440ce0d0 0x7fd8440caf20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fd8340097a0 tx=0x7fd83402b960 comp rx=0 tx=0).stop 2023-11-08T16:37:35.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.234+0000 7fd849941640 1 -- 172.21.15.81:0/3206780845 shutdown_connections 2023-11-08T16:37:35.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.234+0000 7fd849941640 1 --2- 172.21.15.81:0/3206780845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8440ce0d0 0x7fd8440caf20 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:35.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.234+0000 7fd849941640 1 -- 172.21.15.81:0/3206780845 >> 172.21.15.81:0/3206780845 conn(0x7fd8440cb580 msgr2=0x7fd8440cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:35.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.234+0000 7fd849941640 1 -- 172.21.15.81:0/3206780845 shutdown_connections 2023-11-08T16:37:35.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.234+0000 7fd849941640 1 -- 172.21.15.81:0/3206780845 wait complete. 2023-11-08T16:37:35.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.235+0000 7fd849941640 1 Processor -- start 2023-11-08T16:37:35.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.236+0000 7fd849941640 1 -- start start 2023-11-08T16:37:35.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd849941640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8440ce0d0 0x7fd84406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:35.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd849941640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84415bfc0 0x7fd84414a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:35.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd849941640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd84415f280 0x7fd84415a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:35.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd849941640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440b2de0 con 0x7fd84415bfc0 2023-11-08T16:37:35.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd849941640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8440b2f70 con 0x7fd84415f280 2023-11-08T16:37:35.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd849941640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd84414b440 con 0x7fd8440ce0d0 2023-11-08T16:37:35.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd8427fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84415bfc0 0x7fd84414a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:35.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd8427fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84415bfc0 0x7fd84414a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50364/0 (socket says 172.21.15.81:50364) 2023-11-08T16:37:35.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd8427fc640 1 -- 172.21.15.81:0/560506283 learned_addr learned my addr 172.21.15.81:0/560506283 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:35.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd842ffd640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8440ce0d0 0x7fd84406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:35.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd8437fe640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd84415f280 0x7fd84415a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:35.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd8427fc640 1 -- 172.21.15.81:0/560506283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8440ce0d0 msgr2=0x7fd84406d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:35.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd8427fc640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8440ce0d0 0x7fd84406d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:35.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd8427fc640 1 -- 172.21.15.81:0/560506283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd84415f280 msgr2=0x7fd84415a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:35.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd8427fc640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd84415f280 0x7fd84415a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:35.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.237+0000 7fd8427fc640 1 -- 172.21.15.81:0/560506283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd834009400 con 0x7fd84415bfc0 2023-11-08T16:37:35.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.238+0000 7fd8427fc640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84415bfc0 0x7fd84414a200 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c450 tx=0x7fd83800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:35.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.238+0000 7fd82bfff640 1 -- 172.21.15.81:0/560506283 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8380052d0 con 0x7fd84415bfc0 2023-11-08T16:37:35.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.238+0000 7fd82bfff640 1 -- 172.21.15.81:0/560506283 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd838005430 con 0x7fd84415bfc0 2023-11-08T16:37:35.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.238+0000 7fd849941640 1 -- 172.21.15.81:0/560506283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd84414b690 con 0x7fd84415bfc0 2023-11-08T16:37:35.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.238+0000 7fd82bfff640 1 -- 172.21.15.81:0/560506283 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd838012870 con 0x7fd84415bfc0 2023-11-08T16:37:35.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.238+0000 7fd849941640 1 -- 172.21.15.81:0/560506283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd84414bc50 con 0x7fd84415bfc0 2023-11-08T16:37:35.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.239+0000 7fd849941640 1 -- 172.21.15.81:0/560506283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd84414bde0 con 0x7fd84415bfc0 2023-11-08T16:37:35.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.240+0000 7fd82bfff640 1 -- 172.21.15.81:0/560506283 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8380129d0 con 0x7fd84415bfc0 2023-11-08T16:37:35.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.241+0000 7fd82bfff640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83006ae70 0x7fd83006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:35.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.241+0000 7fd82bfff640 1 -- 172.21.15.81:0/560506283 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd838089130 con 0x7fd84415bfc0 2023-11-08T16:37:35.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.241+0000 7fd842ffd640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83006ae70 0x7fd83006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:35.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.242+0000 7fd842ffd640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83006ae70 0x7fd83006d330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd834036000 tx=0x7fd834008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:35.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.246+0000 7fd82bfff640 1 -- 172.21.15.81:0/560506283 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8380541e0 con 0x7fd84415bfc0 2023-11-08T16:37:35.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.473+0000 7fd849941640 1 -- 172.21.15.81:0/560506283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd8440b30e0 con 0x7fd84415bfc0 2023-11-08T16:37:35.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.985+0000 7fd82bfff640 1 -- 172.21.15.81:0/560506283 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd838057800 con 0x7fd84415bfc0 2023-11-08T16:37:35.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.989+0000 7fd849941640 1 -- 172.21.15.81:0/560506283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83006ae70 msgr2=0x7fd83006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.989+0000 7fd849941640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83006ae70 0x7fd83006d330 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd834036000 tx=0x7fd834008030 comp rx=0 tx=0).stop 2023-11-08T16:37:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.989+0000 7fd849941640 1 -- 172.21.15.81:0/560506283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84415bfc0 msgr2=0x7fd84414a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.989+0000 7fd849941640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84415bfc0 0x7fd84414a200 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd83800c450 tx=0x7fd83800c910 comp rx=0 tx=0).stop 2023-11-08T16:37:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.990+0000 7fd849941640 1 -- 172.21.15.81:0/560506283 shutdown_connections 2023-11-08T16:37:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.990+0000 7fd849941640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd83006ae70 0x7fd83006d330 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:35.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.990+0000 7fd849941640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd84415f280 0x7fd84415a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:35.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.990+0000 7fd849941640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd84415bfc0 0x7fd84414a200 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:35.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.990+0000 7fd849941640 1 --2- 172.21.15.81:0/560506283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8440ce0d0 0x7fd84406d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:35.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.990+0000 7fd849941640 1 -- 172.21.15.81:0/560506283 >> 172.21.15.81:0/560506283 conn(0x7fd8440cb580 msgr2=0x7fd8440aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:35.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.990+0000 7fd849941640 1 -- 172.21.15.81:0/560506283 shutdown_connections 2023-11-08T16:37:35.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:35.990+0000 7fd849941640 1 -- 172.21.15.81:0/560506283 wait complete. 2023-11-08T16:37:36.013 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T16:37:36.014 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:37:36.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.218+0000 7f326ab7c640 1 Processor -- start 2023-11-08T16:37:36.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.218+0000 7f326ab7c640 1 -- start start 2023-11-08T16:37:36.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.219+0000 7f326ab7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640adcd0 0x7f32640ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.219+0000 7f326ab7c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f32640cc5e0 0x7f32640cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.219+0000 7f326ab7c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32640c9a70 con 0x7f32640ccf80 2023-11-08T16:37:36.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.220+0000 7f326ab7c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32640cc3e0 con 0x7f32640cc5e0 2023-11-08T16:37:36.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.220+0000 7f326ab7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326406d070 con 0x7f32640adcd0 2023-11-08T16:37:36.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.220+0000 7f3263fff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f32640cc5e0 0x7f32640cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.220+0000 7f3268918640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640adcd0 0x7f32640ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.220+0000 7f3268918640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640adcd0 0x7f32640ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:50656/0 (socket says 172.21.15.81:50656) 2023-11-08T16:37:36.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.220+0000 7f3269119640 1 --1- 172.21.15.81:0/2565349069 >> v1:172.21.15.93:6789/0 conn(0x7f32640ccf80 0x7f326415db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:57526/0 (socket says 172.21.15.81:57526) 2023-11-08T16:37:36.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.220+0000 7f3268918640 1 -- 172.21.15.81:0/2565349069 learned_addr learned my addr 172.21.15.81:0/2565349069 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:36.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.221+0000 7f3268918640 1 -- 172.21.15.81:0/2565349069 >> v2:172.21.15.93:3300/0 conn(0x7f32640cc5e0 msgr2=0x7f32640cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:36.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.221+0000 7f3268918640 1 --2- 172.21.15.81:0/2565349069 >> v2:172.21.15.93:3300/0 conn(0x7f32640cc5e0 0x7f32640cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.221+0000 7f3268918640 1 -- 172.21.15.81:0/2565349069 >> v1:172.21.15.93:6789/0 conn(0x7f32640ccf80 legacy=0x7f326415db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:36.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.221+0000 7f3268918640 1 -- 172.21.15.81:0/2565349069 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326406d350 con 0x7f32640adcd0 2023-11-08T16:37:36.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.223+0000 7f3268918640 1 --2- 172.21.15.81:0/2565349069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640adcd0 0x7f32640ca1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f325800bb10 tx=0x7f325802e150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b187e45a5b2514f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:36.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.223+0000 7f32637fe640 1 -- 172.21.15.81:0/2565349069 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258036440 con 0x7f32640adcd0 2023-11-08T16:37:36.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.223+0000 7f32637fe640 1 -- 172.21.15.81:0/2565349069 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32580365a0 con 0x7f32640adcd0 2023-11-08T16:37:36.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.223+0000 7f32637fe640 1 -- 172.21.15.81:0/2565349069 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258036930 con 0x7f32640adcd0 2023-11-08T16:37:36.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.224+0000 7f326ab7c640 1 -- 172.21.15.81:0/2565349069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640adcd0 msgr2=0x7f32640ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:36.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.224+0000 7f326ab7c640 1 --2- 172.21.15.81:0/2565349069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640adcd0 0x7f32640ca1e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f325800bb10 tx=0x7f325802e150 comp rx=0 tx=0).stop 2023-11-08T16:37:36.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.224+0000 7f326ab7c640 1 -- 172.21.15.81:0/2565349069 shutdown_connections 2023-11-08T16:37:36.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.224+0000 7f326ab7c640 1 --2- 172.21.15.81:0/2565349069 >> v2:172.21.15.93:3300/0 conn(0x7f32640cc5e0 0x7f32640cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.224+0000 7f326ab7c640 1 --2- 172.21.15.81:0/2565349069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640adcd0 0x7f32640ca1e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.224+0000 7f326ab7c640 1 -- 172.21.15.81:0/2565349069 >> 172.21.15.81:0/2565349069 conn(0x7f32640cb490 msgr2=0x7f32640cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:36.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.224+0000 7f326ab7c640 1 -- 172.21.15.81:0/2565349069 shutdown_connections 2023-11-08T16:37:36.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.224+0000 7f326ab7c640 1 -- 172.21.15.81:0/2565349069 wait complete. 2023-11-08T16:37:36.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.226+0000 7f326ab7c640 1 Processor -- start 2023-11-08T16:37:36.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.226+0000 7f326ab7c640 1 -- start start 2023-11-08T16:37:36.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f326ab7c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32640adcd0 0x7f32640c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f326ab7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640cc5e0 0x7f32640c82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f326ab7c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32640ccf80 0x7f3264159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f326ab7c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32641e55e0 con 0x7f32640adcd0 2023-11-08T16:37:36.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f326ab7c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32640c87f0 con 0x7f32640ccf80 2023-11-08T16:37:36.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f326ab7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32640c5ec0 con 0x7f32640cc5e0 2023-11-08T16:37:36.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f3268918640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32640adcd0 0x7f32640c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f3268918640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32640adcd0 0x7f32640c7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50366/0 (socket says 172.21.15.81:50366) 2023-11-08T16:37:36.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f3268918640 1 -- 172.21.15.81:0/2675762525 learned_addr learned my addr 172.21.15.81:0/2675762525 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:36.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f3269119640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32640ccf80 0x7f3264159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f3263fff640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640cc5e0 0x7f32640c82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f3268918640 1 -- 172.21.15.81:0/2675762525 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640cc5e0 msgr2=0x7f32640c82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:36.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f3268918640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640cc5e0 0x7f32640c82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f3268918640 1 -- 172.21.15.81:0/2675762525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32640ccf80 msgr2=0x7f3264159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:36.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f3268918640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32640ccf80 0x7f3264159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.227+0000 7f3268918640 1 -- 172.21.15.81:0/2675762525 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f325800b770 con 0x7f32640adcd0 2023-11-08T16:37:36.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.228+0000 7f3268918640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32640adcd0 0x7f32640c7d70 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f325800b2c0 tx=0x7f3258004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:36.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.228+0000 7f32617fa640 1 -- 172.21.15.81:0/2675762525 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258036440 con 0x7f32640adcd0 2023-11-08T16:37:36.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.228+0000 7f32617fa640 1 -- 172.21.15.81:0/2675762525 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3258034bc0 con 0x7f32640adcd0 2023-11-08T16:37:36.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.228+0000 7f32617fa640 1 -- 172.21.15.81:0/2675762525 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3258036e20 con 0x7f32640adcd0 2023-11-08T16:37:36.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.228+0000 7f326ab7c640 1 -- 172.21.15.81:0/2675762525 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32640c61a0 con 0x7f32640adcd0 2023-11-08T16:37:36.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.228+0000 7f326ab7c640 1 -- 172.21.15.81:0/2675762525 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32640c6760 con 0x7f32640adcd0 2023-11-08T16:37:36.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.229+0000 7f326ab7c640 1 -- 172.21.15.81:0/2675762525 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3264072170 con 0x7f32640adcd0 2023-11-08T16:37:36.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.230+0000 7f32617fa640 1 -- 172.21.15.81:0/2675762525 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3258041480 con 0x7f32640adcd0 2023-11-08T16:37:36.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.231+0000 7f32617fa640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f324806aea0 0x7f324806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.231+0000 7f32617fa640 1 -- 172.21.15.81:0/2675762525 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32580ac740 con 0x7f32640adcd0 2023-11-08T16:37:36.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.231+0000 7f3263fff640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f324806aea0 0x7f324806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.232+0000 7f3263fff640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f324806aea0 0x7f324806d360 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f32641f6a30 tx=0x7f3250008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:36.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.235+0000 7f32617fa640 1 -- 172.21.15.81:0/2675762525 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3258077750 con 0x7f32640adcd0 2023-11-08T16:37:36.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.460+0000 7f326ab7c640 1 -- 172.21.15.81:0/2675762525 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f326404ece0 con 0x7f32640adcd0 2023-11-08T16:37:36.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.461+0000 7f32617fa640 1 -- 172.21.15.81:0/2675762525 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f325807ad70 con 0x7f32640adcd0 2023-11-08T16:37:36.463 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:36.463 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":64,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/2125403666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2125403666},{"type":"v1","addr":"172.21.15.93:6834","nonce":2125403666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2727356524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2727356524},{"type":"v1","addr":"172.21.15.93:6835","nonce":2727356524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6936,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2639654743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2639654743},{"type":"v1","addr":"172.21.15.81:6835","nonce":2639654743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:32.323261+0000","modified":"2023-11-08T16:37:35.475263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6942":{"gid":6942,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/2162767469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2162767469},{"type":"v1","addr":"172.21.15.81:6837","nonce":2162767469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:37:36.464 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 64 2023-11-08T16:37:36.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 -- 172.21.15.81:0/2675762525 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f324806aea0 msgr2=0x7f324806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:36.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f324806aea0 0x7f324806d360 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f32641f6a30 tx=0x7f3250008040 comp rx=0 tx=0).stop 2023-11-08T16:37:36.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 -- 172.21.15.81:0/2675762525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32640adcd0 msgr2=0x7f32640c7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:36.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32640adcd0 0x7f32640c7d70 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f325800b2c0 tx=0x7f3258004730 comp rx=0 tx=0).stop 2023-11-08T16:37:36.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 -- 172.21.15.81:0/2675762525 shutdown_connections 2023-11-08T16:37:36.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f324806aea0 0x7f324806d360 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32640ccf80 0x7f3264159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32640cc5e0 0x7f32640c82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 --2- 172.21.15.81:0/2675762525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32640adcd0 0x7f32640c7d70 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 -- 172.21.15.81:0/2675762525 >> 172.21.15.81:0/2675762525 conn(0x7f32640cb490 msgr2=0x7f326415ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:36.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 -- 172.21.15.81:0/2675762525 shutdown_connections 2023-11-08T16:37:36.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.464+0000 7f326ab7c640 1 -- 172.21.15.81:0/2675762525 wait complete. 2023-11-08T16:37:36.484 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:36.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.689+0000 7f5c25eea640 1 Processor -- start 2023-11-08T16:37:36.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.689+0000 7f5c25eea640 1 -- start start 2023-11-08T16:37:36.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.691+0000 7f5c25eea640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5c2015d490 0x7f5c200d39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.691+0000 7f5c25eea640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c200d1080 0x7f5c200d14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.691+0000 7f5c25eea640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5c200d1a20 0x7f5c200cf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.691+0000 7f5c25eea640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c200d1eb0 con 0x7f5c2015d490 2023-11-08T16:37:36.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.691+0000 7f5c25eea640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c200ce890 con 0x7f5c200d1a20 2023-11-08T16:37:36.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.691+0000 7f5c25eea640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200ceb70 con 0x7f5c200d1080 2023-11-08T16:37:36.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.691+0000 7f5c24ee8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5c2015d490 0x7f5c200d39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.691+0000 7f5c24ee8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5c2015d490 0x7f5c200d39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50376/0 (socket says 172.21.15.81:50376) 2023-11-08T16:37:36.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.691+0000 7f5c24ee8640 1 -- 172.21.15.81:0/4187555005 learned_addr learned my addr 172.21.15.81:0/4187555005 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:36.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.692+0000 7f5c256e9640 1 --2- 172.21.15.81:0/4187555005 >> v2:172.21.15.93:3300/0 conn(0x7f5c200d1a20 0x7f5c200cf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.692+0000 7f5c1ffff640 1 --2- 172.21.15.81:0/4187555005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c200d1080 0x7f5c200d14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.692+0000 7f5c24ee8640 1 -- 172.21.15.81:0/4187555005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c200d1080 msgr2=0x7f5c200d14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:36.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.692+0000 7f5c24ee8640 1 --2- 172.21.15.81:0/4187555005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c200d1080 0x7f5c200d14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.692+0000 7f5c24ee8640 1 -- 172.21.15.81:0/4187555005 >> v2:172.21.15.93:3300/0 conn(0x7f5c200d1a20 msgr2=0x7f5c200cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:36.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.692+0000 7f5c24ee8640 1 --2- 172.21.15.81:0/4187555005 >> v2:172.21.15.93:3300/0 conn(0x7f5c200d1a20 0x7f5c200cf820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.692+0000 7f5c24ee8640 1 -- 172.21.15.81:0/4187555005 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c200cee50 con 0x7f5c2015d490 2023-11-08T16:37:36.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.692+0000 7f5c1ffff640 1 --2- 172.21.15.81:0/4187555005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c200d1080 0x7f5c200d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:36.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.693+0000 7f5c24ee8640 1 --2- 172.21.15.81:0/4187555005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2015d490 0x7f5c200d39e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009870 tx=0x7f5c1402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d879ad3cd6f9a00 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:36.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.693+0000 7f5c1f7fe640 1 -- 172.21.15.81:0/4187555005 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14031640 con 0x7f5c2015d490 2023-11-08T16:37:36.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.693+0000 7f5c1f7fe640 1 -- 172.21.15.81:0/4187555005 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c140317a0 con 0x7f5c2015d490 2023-11-08T16:37:36.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.693+0000 7f5c1f7fe640 1 -- 172.21.15.81:0/4187555005 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c14031b30 con 0x7f5c2015d490 2023-11-08T16:37:36.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.694+0000 7f5c25eea640 1 -- 172.21.15.81:0/4187555005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2015d490 msgr2=0x7f5c200d39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:36.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.694+0000 7f5c25eea640 1 --2- 172.21.15.81:0/4187555005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2015d490 0x7f5c200d39e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5c14009870 tx=0x7f5c1402b8f0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.694+0000 7f5c25eea640 1 -- 172.21.15.81:0/4187555005 shutdown_connections 2023-11-08T16:37:36.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.694+0000 7f5c25eea640 1 --2- 172.21.15.81:0/4187555005 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c200d1080 0x7f5c200d14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.694+0000 7f5c25eea640 1 --2- 172.21.15.81:0/4187555005 >> v2:172.21.15.93:3300/0 conn(0x7f5c200d1a20 0x7f5c200cf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.694+0000 7f5c25eea640 1 --2- 172.21.15.81:0/4187555005 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2015d490 0x7f5c200d39e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.694+0000 7f5c25eea640 1 -- 172.21.15.81:0/4187555005 >> 172.21.15.81:0/4187555005 conn(0x7f5c200d2e50 msgr2=0x7f5c200d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:36.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.694+0000 7f5c25eea640 1 -- 172.21.15.81:0/4187555005 shutdown_connections 2023-11-08T16:37:36.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.694+0000 7f5c25eea640 1 -- 172.21.15.81:0/4187555005 wait complete. 2023-11-08T16:37:36.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.696+0000 7f5c25eea640 1 Processor -- start 2023-11-08T16:37:36.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.696+0000 7f5c25eea640 1 -- start start 2023-11-08T16:37:36.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c25eea640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c200d1080 0x7f5c201f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c25eea640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c200d1a20 0x7f5c200d3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c25eea640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2015d490 0x7f5c200d3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c25eea640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200c7f70 con 0x7f5c2015d490 2023-11-08T16:37:36.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c25eea640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200c8850 con 0x7f5c200d1a20 2023-11-08T16:37:36.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c25eea640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c200c8b30 con 0x7f5c200d1080 2023-11-08T16:37:36.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c256e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2015d490 0x7f5c200d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c256e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2015d490 0x7f5c200d3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50386/0 (socket says 172.21.15.81:50386) 2023-11-08T16:37:36.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c256e9640 1 -- 172.21.15.81:0/4138309996 learned_addr learned my addr 172.21.15.81:0/4138309996 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:36.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c24ee8640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c200d1080 0x7f5c201f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c1ffff640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c200d1a20 0x7f5c200d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c256e9640 1 -- 172.21.15.81:0/4138309996 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c200d1080 msgr2=0x7f5c201f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:36.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c256e9640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c200d1080 0x7f5c201f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c256e9640 1 -- 172.21.15.81:0/4138309996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c200d1a20 msgr2=0x7f5c200d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:36.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c256e9640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c200d1a20 0x7f5c200d3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c256e9640 1 -- 172.21.15.81:0/4138309996 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c140094d0 con 0x7f5c2015d490 2023-11-08T16:37:36.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.697+0000 7f5c1ffff640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c200d1a20 0x7f5c200d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:36.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.698+0000 7f5c256e9640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2015d490 0x7f5c200d3770 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800cd70 tx=0x7f5c1800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:36.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.698+0000 7f5c1d7fa640 1 -- 172.21.15.81:0/4138309996 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18003b90 con 0x7f5c2015d490 2023-11-08T16:37:36.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.698+0000 7f5c1d7fa640 1 -- 172.21.15.81:0/4138309996 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c18003cf0 con 0x7f5c2015d490 2023-11-08T16:37:36.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.698+0000 7f5c1d7fa640 1 -- 172.21.15.81:0/4138309996 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c18004240 con 0x7f5c2015d490 2023-11-08T16:37:36.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.698+0000 7f5c25eea640 1 -- 172.21.15.81:0/4138309996 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c200c8e10 con 0x7f5c2015d490 2023-11-08T16:37:36.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.698+0000 7f5c25eea640 1 -- 172.21.15.81:0/4138309996 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c201f6dd0 con 0x7f5c2015d490 2023-11-08T16:37:36.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.699+0000 7f5c25eea640 1 -- 172.21.15.81:0/4138309996 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c2015f620 con 0x7f5c2015d490 2023-11-08T16:37:36.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.700+0000 7f5c1d7fa640 1 -- 172.21.15.81:0/4138309996 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c180043a0 con 0x7f5c2015d490 2023-11-08T16:37:36.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.701+0000 7f5c1d7fa640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c0406ae70 0x7f5c0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:36.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.701+0000 7f5c1d7fa640 1 -- 172.21.15.81:0/4138309996 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c18012070 con 0x7f5c2015d490 2023-11-08T16:37:36.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.701+0000 7f5c24ee8640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c0406ae70 0x7f5c0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:36.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.702+0000 7f5c24ee8640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c0406ae70 0x7f5c0406d330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c14036000 tx=0x7f5c1403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:36.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.705+0000 7f5c1d7fa640 1 -- 172.21.15.81:0/4138309996 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c180550f0 con 0x7f5c2015d490 2023-11-08T16:37:36.813 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:37:36.811+0000 7f06ce4dc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:37:36.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.896+0000 7f5c25eea640 1 -- 172.21.15.81:0/4138309996 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c20067800 con 0x7f5c2015d490 2023-11-08T16:37:36.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.898+0000 7f5c1d7fa640 1 -- 172.21.15.81:0/4138309996 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16508 (secure 0 0 0) 0x7f5c18058710 con 0x7f5c2015d490 2023-11-08T16:37:36.900 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:36.900 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":81,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:37:33.857577+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:37:30.174436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T16:37:31.340720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:36.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.900+0000 7f5c25eea640 1 -- 172.21.15.81:0/4138309996 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c0406ae70 msgr2=0x7f5c0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:36.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.900+0000 7f5c25eea640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c0406ae70 0x7f5c0406d330 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5c14036000 tx=0x7f5c1403c040 comp rx=0 tx=0).stop 2023-11-08T16:37:36.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.900+0000 7f5c25eea640 1 -- 172.21.15.81:0/4138309996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2015d490 msgr2=0x7f5c200d3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:36.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.900+0000 7f5c25eea640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2015d490 0x7f5c200d3770 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f5c1800cd70 tx=0x7f5c1800a660 comp rx=0 tx=0).stop 2023-11-08T16:37:36.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.900+0000 7f5c25eea640 1 -- 172.21.15.81:0/4138309996 shutdown_connections 2023-11-08T16:37:36.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.900+0000 7f5c25eea640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c0406ae70 0x7f5c0406d330 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.900+0000 7f5c25eea640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c2015d490 0x7f5c200d3770 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.900+0000 7f5c25eea640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c200d1a20 0x7f5c200d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.900+0000 7f5c25eea640 1 --2- 172.21.15.81:0/4138309996 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c200d1080 0x7f5c201f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:36.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.900+0000 7f5c25eea640 1 -- 172.21.15.81:0/4138309996 >> 172.21.15.81:0/4138309996 conn(0x7f5c200d2e50 msgr2=0x7f5c200b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:36.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.901+0000 7f5c25eea640 1 -- 172.21.15.81:0/4138309996 shutdown_connections 2023-11-08T16:37:36.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:36.901+0000 7f5c25eea640 1 -- 172.21.15.81:0/4138309996 wait complete. 2023-11-08T16:37:36.922 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T16:37:36.923 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:37:37.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.132+0000 7f962f7b6640 1 Processor -- start 2023-11-08T16:37:37.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.132+0000 7f962f7b6640 1 -- start start 2023-11-08T16:37:37.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.133+0000 7f962f7b6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f962815d2b0 0x7f9628161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.133+0000 7f962f7b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96280ccf00 0x7f962806ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.134+0000 7f962f7b6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9628161c50 con 0x7f96280c9060 2023-11-08T16:37:37.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.134+0000 7f962f7b6640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f962815d690 con 0x7f962815d2b0 2023-11-08T16:37:37.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.134+0000 7f962f7b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962806b980 con 0x7f96280ccf00 2023-11-08T16:37:37.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.134+0000 7f962d552640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f962815d2b0 0x7f9628161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.134+0000 7f962d552640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f962815d2b0 0x7f9628161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50390/0 (socket says 172.21.15.81:50390) 2023-11-08T16:37:37.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.134+0000 7f962d552640 1 -- 172.21.15.81:0/4142482976 learned_addr learned my addr 172.21.15.81:0/4142482976 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:37.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.134+0000 7f962dd53640 1 --2- 172.21.15.81:0/4142482976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96280ccf00 0x7f962806ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.134+0000 7f962d552640 1 -- 172.21.15.81:0/4142482976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96280ccf00 msgr2=0x7f962806ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:37.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.134+0000 7f962d552640 1 --2- 172.21.15.81:0/4142482976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96280ccf00 0x7f962806ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.134+0000 7f962d552640 1 -- 172.21.15.81:0/4142482976 >> v1:172.21.15.93:6789/0 conn(0x7f96280c9060 legacy=0x7f96280cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:37.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.135+0000 7f962d552640 1 -- 172.21.15.81:0/4142482976 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962806bc60 con 0x7f962815d2b0 2023-11-08T16:37:37.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.136+0000 7f962d552640 1 --2- 172.21.15.81:0/4142482976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f962815d2b0 0x7f9628161710 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9624009870 tx=0x7f962402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ba7df0ca5e2870 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:37.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.136+0000 7f961ffff640 1 -- 172.21.15.81:0/4142482976 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624031640 con 0x7f962815d2b0 2023-11-08T16:37:37.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.136+0000 7f961ffff640 1 -- 172.21.15.81:0/4142482976 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96240317a0 con 0x7f962815d2b0 2023-11-08T16:37:37.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.136+0000 7f961ffff640 1 -- 172.21.15.81:0/4142482976 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9624031b30 con 0x7f962815d2b0 2023-11-08T16:37:37.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.137+0000 7f962f7b6640 1 -- 172.21.15.81:0/4142482976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f962815d2b0 msgr2=0x7f9628161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:37.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.137+0000 7f962f7b6640 1 --2- 172.21.15.81:0/4142482976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f962815d2b0 0x7f9628161710 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9624009870 tx=0x7f962402b8f0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.137+0000 7f962f7b6640 1 -- 172.21.15.81:0/4142482976 shutdown_connections 2023-11-08T16:37:37.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.137+0000 7f962f7b6640 1 --2- 172.21.15.81:0/4142482976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96280ccf00 0x7f962806ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.137+0000 7f962f7b6640 1 --2- 172.21.15.81:0/4142482976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f962815d2b0 0x7f9628161710 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.137+0000 7f962f7b6640 1 -- 172.21.15.81:0/4142482976 >> 172.21.15.81:0/4142482976 conn(0x7f96280cb490 msgr2=0x7f96280cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:37.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.137+0000 7f962f7b6640 1 -- 172.21.15.81:0/4142482976 shutdown_connections 2023-11-08T16:37:37.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.137+0000 7f962f7b6640 1 -- 172.21.15.81:0/4142482976 wait complete. 2023-11-08T16:37:37.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.139+0000 7f962f7b6640 1 Processor -- start 2023-11-08T16:37:37.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.139+0000 7f962f7b6640 1 -- start start 2023-11-08T16:37:37.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.140+0000 7f962f7b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96280c9060 0x7f96280b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.140+0000 7f962f7b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96280ccf00 0x7f962814ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.140+0000 7f962f7b6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f962815d2b0 0x7f96280c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.140+0000 7f962f7b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962814d340 con 0x7f96280ccf00 2023-11-08T16:37:37.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.140+0000 7f962f7b6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96280c80e0 con 0x7f962815d2b0 2023-11-08T16:37:37.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.140+0000 7f962f7b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96280b1610 con 0x7f96280c9060 2023-11-08T16:37:37.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.140+0000 7f962cd51640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96280ccf00 0x7f962814ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.140+0000 7f962cd51640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96280ccf00 0x7f962814ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50392/0 (socket says 172.21.15.81:50392) 2023-11-08T16:37:37.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.140+0000 7f962cd51640 1 -- 172.21.15.81:0/1600946502 learned_addr learned my addr 172.21.15.81:0/1600946502 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:37.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f962d552640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96280c9060 0x7f96280b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f962cd51640 1 -- 172.21.15.81:0/1600946502 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96280c9060 msgr2=0x7f96280b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:37.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f962dd53640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f962815d2b0 0x7f96280c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f962cd51640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96280c9060 0x7f96280b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f962cd51640 1 -- 172.21.15.81:0/1600946502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f962815d2b0 msgr2=0x7f96280c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:37.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f962cd51640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f962815d2b0 0x7f96280c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f962cd51640 1 -- 172.21.15.81:0/1600946502 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96240094d0 con 0x7f96280ccf00 2023-11-08T16:37:37.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f962cd51640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96280ccf00 0x7f962814ce00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f961800c430 tx=0x7f961800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:37.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f961dffb640 1 -- 172.21.15.81:0/1600946502 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618011790 con 0x7f96280ccf00 2023-11-08T16:37:37.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f961dffb640 1 -- 172.21.15.81:0/1600946502 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96180118f0 con 0x7f96280ccf00 2023-11-08T16:37:37.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f961dffb640 1 -- 172.21.15.81:0/1600946502 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618011ce0 con 0x7f96280ccf00 2023-11-08T16:37:37.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f962f7b6640 1 -- 172.21.15.81:0/1600946502 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96280b18f0 con 0x7f96280ccf00 2023-11-08T16:37:37.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.141+0000 7f962f7b6640 1 -- 172.21.15.81:0/1600946502 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96280c8600 con 0x7f96280ccf00 2023-11-08T16:37:37.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.142+0000 7f962f7b6640 1 -- 172.21.15.81:0/1600946502 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96280b1a80 con 0x7f96280ccf00 2023-11-08T16:37:37.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.143+0000 7f961dffb640 1 -- 172.21.15.81:0/1600946502 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f961801c460 con 0x7f96280ccf00 2023-11-08T16:37:37.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.144+0000 7f961dffb640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f960c06ae70 0x7f960c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.144+0000 7f961dffb640 1 -- 172.21.15.81:0/1600946502 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9618089650 con 0x7f96280ccf00 2023-11-08T16:37:37.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.145+0000 7f962d552640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f960c06ae70 0x7f960c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.145+0000 7f962d552640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f960c06ae70 0x7f960c06d330 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9624036000 tx=0x7f962403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:37.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.149+0000 7f961dffb640 1 -- 172.21.15.81:0/1600946502 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9618054660 con 0x7f96280ccf00 2023-11-08T16:37:37.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.336+0000 7f962f7b6640 1 -- 172.21.15.81:0/1600946502 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f962806c220 con 0x7f960c06ae70 2023-11-08T16:37:37.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.350+0000 7f961dffb640 1 -- 172.21.15.81:0/1600946502 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287088 (secure 0 0 0) 0x7f9624031990 con 0x7f960c06ae70 2023-11-08T16:37:37.356 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:37.357 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:37:37.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.356+0000 7f962f7b6640 1 -- 172.21.15.81:0/1600946502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f960c06ae70 msgr2=0x7f960c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:37.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.356+0000 7f962f7b6640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f960c06ae70 0x7f960c06d330 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9624036000 tx=0x7f962403c040 comp rx=0 tx=0).stop 2023-11-08T16:37:37.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.356+0000 7f962f7b6640 1 -- 172.21.15.81:0/1600946502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96280ccf00 msgr2=0x7f962814ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:37.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.356+0000 7f962f7b6640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96280ccf00 0x7f962814ce00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f961800c430 tx=0x7f961800c8f0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.356+0000 7f962f7b6640 1 -- 172.21.15.81:0/1600946502 shutdown_connections 2023-11-08T16:37:37.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.356+0000 7f962f7b6640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f960c06ae70 0x7f960c06d330 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.356+0000 7f962f7b6640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f962815d2b0 0x7f96280c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.356+0000 7f962f7b6640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96280ccf00 0x7f962814ce00 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.356+0000 7f962f7b6640 1 --2- 172.21.15.81:0/1600946502 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96280c9060 0x7f96280b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.356+0000 7f962f7b6640 1 -- 172.21.15.81:0/1600946502 >> 172.21.15.81:0/1600946502 conn(0x7f96280cb490 msgr2=0x7f962806cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:37.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.357+0000 7f962f7b6640 1 -- 172.21.15.81:0/1600946502 shutdown_connections 2023-11-08T16:37:37.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.357+0000 7f962f7b6640 1 -- 172.21.15.81:0/1600946502 wait complete. 2023-11-08T16:37:37.377 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2023-11-08T16:37:36.730150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258416,"kb_used_data":8936,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716304,"statfs":{"total":773094113280,"available":772829495296,"internally_reserved":0,"allocated":9150464,"data_stored":4741676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.892224"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867992+0000","last_change":"2023-11-08T16:37:32.899110+0000","last_active":"2023-11-08T16:37:33.867992+0000","last_peered":"2023-11-08T16:37:33.867992+0000","last_clean":"2023-11-08T16:37:33.867992+0000","last_became_active":"2023-11-08T16:37:32.898593+0000","last_became_peered":"2023-11-08T16:37:32.898593+0000","last_unstale":"2023-11-08T16:37:33.867992+0000","last_undegraded":"2023-11-08T16:37:33.867992+0000","last_fullsized":"2023-11-08T16:37:33.867992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:57.888620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.700768+0000","last_change":"2023-11-08T16:37:32.890662+0000","last_active":"2023-11-08T16:37:34.700768+0000","last_peered":"2023-11-08T16:37:34.700768+0000","last_clean":"2023-11-08T16:37:34.700768+0000","last_became_active":"2023-11-08T16:37:32.890336+0000","last_became_peered":"2023-11-08T16:37:32.890336+0000","last_unstale":"2023-11-08T16:37:34.700768+0000","last_undegraded":"2023-11-08T16:37:34.700768+0000","last_fullsized":"2023-11-08T16:37:34.700768+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:53:06.026452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107549+0000","last_change":"2023-11-08T16:37:32.893246+0000","last_active":"2023-11-08T16:37:34.107549+0000","last_peered":"2023-11-08T16:37:34.107549+0000","last_clean":"2023-11-08T16:37:34.107549+0000","last_became_active":"2023-11-08T16:37:32.892768+0000","last_became_peered":"2023-11-08T16:37:32.892768+0000","last_unstale":"2023-11-08T16:37:34.107549+0000","last_undegraded":"2023-11-08T16:37:34.107549+0000","last_fullsized":"2023-11-08T16:37:34.107549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:49:36.782384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.865601+0000","last_change":"2023-11-08T16:37:32.894249+0000","last_active":"2023-11-08T16:37:33.865601+0000","last_peered":"2023-11-08T16:37:33.865601+0000","last_clean":"2023-11-08T16:37:33.865601+0000","last_became_active":"2023-11-08T16:37:32.893846+0000","last_became_peered":"2023-11-08T16:37:32.893846+0000","last_unstale":"2023-11-08T16:37:33.865601+0000","last_undegraded":"2023-11-08T16:37:33.865601+0000","last_fullsized":"2023-11-08T16:37:33.865601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:22.625004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696038+0000","last_change":"2023-11-08T16:37:32.897416+0000","last_active":"2023-11-08T16:37:34.696038+0000","last_peered":"2023-11-08T16:37:34.696038+0000","last_clean":"2023-11-08T16:37:34.696038+0000","last_became_active":"2023-11-08T16:37:32.897061+0000","last_became_peered":"2023-11-08T16:37:32.897061+0000","last_unstale":"2023-11-08T16:37:34.696038+0000","last_undegraded":"2023-11-08T16:37:34.696038+0000","last_fullsized":"2023-11-08T16:37:34.696038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:06:31.308088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698272+0000","last_change":"2023-11-08T16:37:32.896273+0000","last_active":"2023-11-08T16:37:34.698272+0000","last_peered":"2023-11-08T16:37:34.698272+0000","last_clean":"2023-11-08T16:37:34.698272+0000","last_became_active":"2023-11-08T16:37:32.895742+0000","last_became_peered":"2023-11-08T16:37:32.895742+0000","last_unstale":"2023-11-08T16:37:34.698272+0000","last_undegraded":"2023-11-08T16:37:34.698272+0000","last_fullsized":"2023-11-08T16:37:34.698272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:45.753772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696676+0000","last_change":"2023-11-08T16:37:32.890779+0000","last_active":"2023-11-08T16:37:34.696676+0000","last_peered":"2023-11-08T16:37:34.696676+0000","last_clean":"2023-11-08T16:37:34.696676+0000","last_became_active":"2023-11-08T16:37:32.890492+0000","last_became_peered":"2023-11-08T16:37:32.890492+0000","last_unstale":"2023-11-08T16:37:34.696676+0000","last_undegraded":"2023-11-08T16:37:34.696676+0000","last_fullsized":"2023-11-08T16:37:34.696676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:22.663417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698035+0000","last_change":"2023-11-08T16:37:32.892385+0000","last_active":"2023-11-08T16:37:34.698035+0000","last_peered":"2023-11-08T16:37:34.698035+0000","last_clean":"2023-11-08T16:37:34.698035+0000","last_became_active":"2023-11-08T16:37:32.892032+0000","last_became_peered":"2023-11-08T16:37:32.892032+0000","last_unstale":"2023-11-08T16:37:34.698035+0000","last_undegraded":"2023-11-08T16:37:34.698035+0000","last_fullsized":"2023-11-08T16:37:34.698035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:38:25.173471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696802+0000","last_change":"2023-11-08T16:37:32.900485+0000","last_active":"2023-11-08T16:37:34.696802+0000","last_peered":"2023-11-08T16:37:34.696802+0000","last_clean":"2023-11-08T16:37:34.696802+0000","last_became_active":"2023-11-08T16:37:32.899649+0000","last_became_peered":"2023-11-08T16:37:32.899649+0000","last_unstale":"2023-11-08T16:37:34.696802+0000","last_undegraded":"2023-11-08T16:37:34.696802+0000","last_fullsized":"2023-11-08T16:37:34.696802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:44:20.816013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696087+0000","last_change":"2023-11-08T16:37:32.896323+0000","last_active":"2023-11-08T16:37:34.696087+0000","last_peered":"2023-11-08T16:37:34.696087+0000","last_clean":"2023-11-08T16:37:34.696087+0000","last_became_active":"2023-11-08T16:37:32.895458+0000","last_became_peered":"2023-11-08T16:37:32.895458+0000","last_unstale":"2023-11-08T16:37:34.696087+0000","last_undegraded":"2023-11-08T16:37:34.696087+0000","last_fullsized":"2023-11-08T16:37:34.696087+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:26.531450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.868746+0000","last_change":"2023-11-08T16:37:32.889586+0000","last_active":"2023-11-08T16:37:33.868746+0000","last_peered":"2023-11-08T16:37:33.868746+0000","last_clean":"2023-11-08T16:37:33.868746+0000","last_became_active":"2023-11-08T16:37:32.889250+0000","last_became_peered":"2023-11-08T16:37:32.889250+0000","last_unstale":"2023-11-08T16:37:33.868746+0000","last_undegraded":"2023-11-08T16:37:33.868746+0000","last_fullsized":"2023-11-08T16:37:33.868746+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:57.489427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.865703+0000","last_change":"2023-11-08T16:37:32.891507+0000","last_active":"2023-11-08T16:37:33.865703+0000","last_peered":"2023-11-08T16:37:33.865703+0000","last_clean":"2023-11-08T16:37:33.865703+0000","last_became_active":"2023-11-08T16:37:32.891181+0000","last_became_peered":"2023-11-08T16:37:32.891181+0000","last_unstale":"2023-11-08T16:37:33.865703+0000","last_undegraded":"2023-11-08T16:37:33.865703+0000","last_fullsized":"2023-11-08T16:37:33.865703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:21.188125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864665+0000","last_change":"2023-11-08T16:37:32.903434+0000","last_active":"2023-11-08T16:37:33.864665+0000","last_peered":"2023-11-08T16:37:33.864665+0000","last_clean":"2023-11-08T16:37:33.864665+0000","last_became_active":"2023-11-08T16:37:32.902647+0000","last_became_peered":"2023-11-08T16:37:32.902647+0000","last_unstale":"2023-11-08T16:37:33.864665+0000","last_undegraded":"2023-11-08T16:37:33.864665+0000","last_fullsized":"2023-11-08T16:37:33.864665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:59:37.827654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696048+0000","last_change":"2023-11-08T16:37:32.897191+0000","last_active":"2023-11-08T16:37:34.696048+0000","last_peered":"2023-11-08T16:37:34.696048+0000","last_clean":"2023-11-08T16:37:34.696048+0000","last_became_active":"2023-11-08T16:37:32.896908+0000","last_became_peered":"2023-11-08T16:37:32.896908+0000","last_unstale":"2023-11-08T16:37:34.696048+0000","last_undegraded":"2023-11-08T16:37:34.696048+0000","last_fullsized":"2023-11-08T16:37:34.696048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:04:38.571484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864488+0000","last_change":"2023-11-08T16:37:32.899400+0000","last_active":"2023-11-08T16:37:33.864488+0000","last_peered":"2023-11-08T16:37:33.864488+0000","last_clean":"2023-11-08T16:37:33.864488+0000","last_became_active":"2023-11-08T16:37:32.898834+0000","last_became_peered":"2023-11-08T16:37:32.898834+0000","last_unstale":"2023-11-08T16:37:33.864488+0000","last_undegraded":"2023-11-08T16:37:33.864488+0000","last_fullsized":"2023-11-08T16:37:33.864488+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:00:16.203256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.695518+0000","last_change":"2023-11-08T16:37:32.898841+0000","last_active":"2023-11-08T16:37:34.695518+0000","last_peered":"2023-11-08T16:37:34.695518+0000","last_clean":"2023-11-08T16:37:34.695518+0000","last_became_active":"2023-11-08T16:37:32.898325+0000","last_became_peered":"2023-11-08T16:37:32.898325+0000","last_unstale":"2023-11-08T16:37:34.695518+0000","last_undegraded":"2023-11-08T16:37:34.695518+0000","last_fullsized":"2023-11-08T16:37:34.695518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:18.438471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696950+0000","last_change":"2023-11-08T16:37:32.899955+0000","last_active":"2023-11-08T16:37:34.696950+0000","last_peered":"2023-11-08T16:37:34.696950+0000","last_clean":"2023-11-08T16:37:34.696950+0000","last_became_active":"2023-11-08T16:37:32.899272+0000","last_became_peered":"2023-11-08T16:37:32.899272+0000","last_unstale":"2023-11-08T16:37:34.696950+0000","last_undegraded":"2023-11-08T16:37:34.696950+0000","last_fullsized":"2023-11-08T16:37:34.696950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:20.677569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.865845+0000","last_change":"2023-11-08T16:37:32.903851+0000","last_active":"2023-11-08T16:37:33.865845+0000","last_peered":"2023-11-08T16:37:33.865845+0000","last_clean":"2023-11-08T16:37:33.865845+0000","last_became_active":"2023-11-08T16:37:32.903566+0000","last_became_peered":"2023-11-08T16:37:32.903566+0000","last_unstale":"2023-11-08T16:37:33.865845+0000","last_undegraded":"2023-11-08T16:37:33.865845+0000","last_fullsized":"2023-11-08T16:37:33.865845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:18:19.754013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699850+0000","last_change":"2023-11-08T16:37:32.901267+0000","last_active":"2023-11-08T16:37:34.699850+0000","last_peered":"2023-11-08T16:37:34.699850+0000","last_clean":"2023-11-08T16:37:34.699850+0000","last_became_active":"2023-11-08T16:37:32.900971+0000","last_became_peered":"2023-11-08T16:37:32.900971+0000","last_unstale":"2023-11-08T16:37:34.699850+0000","last_undegraded":"2023-11-08T16:37:34.699850+0000","last_fullsized":"2023-11-08T16:37:34.699850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:27:30.791214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864665+0000","last_change":"2023-11-08T16:37:32.895358+0000","last_active":"2023-11-08T16:37:33.864665+0000","last_peered":"2023-11-08T16:37:33.864665+0000","last_clean":"2023-11-08T16:37:33.864665+0000","last_became_active":"2023-11-08T16:37:32.894964+0000","last_became_peered":"2023-11-08T16:37:32.894964+0000","last_unstale":"2023-11-08T16:37:33.864665+0000","last_undegraded":"2023-11-08T16:37:33.864665+0000","last_fullsized":"2023-11-08T16:37:33.864665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:11.406716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696104+0000","last_change":"2023-11-08T16:37:32.897279+0000","last_active":"2023-11-08T16:37:34.696104+0000","last_peered":"2023-11-08T16:37:34.696104+0000","last_clean":"2023-11-08T16:37:34.696104+0000","last_became_active":"2023-11-08T16:37:32.897006+0000","last_became_peered":"2023-11-08T16:37:32.897006+0000","last_unstale":"2023-11-08T16:37:34.696104+0000","last_undegraded":"2023-11-08T16:37:34.696104+0000","last_fullsized":"2023-11-08T16:37:34.696104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:38:25.173471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.695996+0000","last_change":"2023-11-08T16:37:32.899462+0000","last_active":"2023-11-08T16:37:34.695996+0000","last_peered":"2023-11-08T16:37:34.695996+0000","last_clean":"2023-11-08T16:37:34.695996+0000","last_became_active":"2023-11-08T16:37:32.898474+0000","last_became_peered":"2023-11-08T16:37:32.898474+0000","last_unstale":"2023-11-08T16:37:34.695996+0000","last_undegraded":"2023-11-08T16:37:34.695996+0000","last_fullsized":"2023-11-08T16:37:34.695996+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:32:09.784202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867719+0000","last_change":"2023-11-08T16:37:32.886798+0000","last_active":"2023-11-08T16:37:33.867719+0000","last_peered":"2023-11-08T16:37:33.867719+0000","last_clean":"2023-11-08T16:37:33.867719+0000","last_became_active":"2023-11-08T16:37:32.884586+0000","last_became_peered":"2023-11-08T16:37:32.884586+0000","last_unstale":"2023-11-08T16:37:33.867719+0000","last_undegraded":"2023-11-08T16:37:33.867719+0000","last_fullsized":"2023-11-08T16:37:33.867719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:39.440480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867091+0000","last_change":"2023-11-08T16:37:32.898640+0000","last_active":"2023-11-08T16:37:33.867091+0000","last_peered":"2023-11-08T16:37:33.867091+0000","last_clean":"2023-11-08T16:37:33.867091+0000","last_became_active":"2023-11-08T16:37:32.898325+0000","last_became_peered":"2023-11-08T16:37:32.898325+0000","last_unstale":"2023-11-08T16:37:33.867091+0000","last_undegraded":"2023-11-08T16:37:33.867091+0000","last_fullsized":"2023-11-08T16:37:33.867091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:53:17.138775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698169+0000","last_change":"2023-11-08T16:37:32.893036+0000","last_active":"2023-11-08T16:37:34.698169+0000","last_peered":"2023-11-08T16:37:34.698169+0000","last_clean":"2023-11-08T16:37:34.698169+0000","last_became_active":"2023-11-08T16:37:32.892749+0000","last_became_peered":"2023-11-08T16:37:32.892749+0000","last_unstale":"2023-11-08T16:37:34.698169+0000","last_undegraded":"2023-11-08T16:37:34.698169+0000","last_fullsized":"2023-11-08T16:37:34.698169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:17.046902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699708+0000","last_change":"2023-11-08T16:37:32.903770+0000","last_active":"2023-11-08T16:37:34.699708+0000","last_peered":"2023-11-08T16:37:34.699708+0000","last_clean":"2023-11-08T16:37:34.699708+0000","last_became_active":"2023-11-08T16:37:32.903492+0000","last_became_peered":"2023-11-08T16:37:32.903492+0000","last_unstale":"2023-11-08T16:37:34.699708+0000","last_undegraded":"2023-11-08T16:37:34.699708+0000","last_fullsized":"2023-11-08T16:37:34.699708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:39:48.141466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107775+0000","last_change":"2023-11-08T16:37:32.880801+0000","last_active":"2023-11-08T16:37:34.107775+0000","last_peered":"2023-11-08T16:37:34.107775+0000","last_clean":"2023-11-08T16:37:34.107775+0000","last_became_active":"2023-11-08T16:37:32.880372+0000","last_became_peered":"2023-11-08T16:37:32.880372+0000","last_unstale":"2023-11-08T16:37:34.107775+0000","last_undegraded":"2023-11-08T16:37:34.107775+0000","last_fullsized":"2023-11-08T16:37:34.107775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:23:26.392182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698292+0000","last_change":"2023-11-08T16:37:32.897875+0000","last_active":"2023-11-08T16:37:34.698292+0000","last_peered":"2023-11-08T16:37:34.698292+0000","last_clean":"2023-11-08T16:37:34.698292+0000","last_became_active":"2023-11-08T16:37:32.897410+0000","last_became_peered":"2023-11-08T16:37:32.897410+0000","last_unstale":"2023-11-08T16:37:34.698292+0000","last_undegraded":"2023-11-08T16:37:34.698292+0000","last_fullsized":"2023-11-08T16:37:34.698292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:23.235607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107690+0000","last_change":"2023-11-08T16:37:32.883090+0000","last_active":"2023-11-08T16:37:34.107690+0000","last_peered":"2023-11-08T16:37:34.107690+0000","last_clean":"2023-11-08T16:37:34.107690+0000","last_became_active":"2023-11-08T16:37:32.882462+0000","last_became_peered":"2023-11-08T16:37:32.882462+0000","last_unstale":"2023-11-08T16:37:34.107690+0000","last_undegraded":"2023-11-08T16:37:34.107690+0000","last_fullsized":"2023-11-08T16:37:34.107690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:36:43.046067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697452+0000","last_change":"2023-11-08T16:37:32.902198+0000","last_active":"2023-11-08T16:37:34.697452+0000","last_peered":"2023-11-08T16:37:34.697452+0000","last_clean":"2023-11-08T16:37:34.697452+0000","last_became_active":"2023-11-08T16:37:32.901808+0000","last_became_peered":"2023-11-08T16:37:32.901808+0000","last_unstale":"2023-11-08T16:37:34.697452+0000","last_undegraded":"2023-11-08T16:37:34.697452+0000","last_fullsized":"2023-11-08T16:37:34.697452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:52.952266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697906+0000","last_change":"2023-11-08T16:37:32.900759+0000","last_active":"2023-11-08T16:37:34.697906+0000","last_peered":"2023-11-08T16:37:34.697906+0000","last_clean":"2023-11-08T16:37:34.697906+0000","last_became_active":"2023-11-08T16:37:32.899852+0000","last_became_peered":"2023-11-08T16:37:32.899852+0000","last_unstale":"2023-11-08T16:37:34.697906+0000","last_undegraded":"2023-11-08T16:37:34.697906+0000","last_fullsized":"2023-11-08T16:37:34.697906+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:14:19.127984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698034+0000","last_change":"2023-11-08T16:37:32.892073+0000","last_active":"2023-11-08T16:37:34.698034+0000","last_peered":"2023-11-08T16:37:34.698034+0000","last_clean":"2023-11-08T16:37:34.698034+0000","last_became_active":"2023-11-08T16:37:32.891260+0000","last_became_peered":"2023-11-08T16:37:32.891260+0000","last_unstale":"2023-11-08T16:37:34.698034+0000","last_undegraded":"2023-11-08T16:37:34.698034+0000","last_fullsized":"2023-11-08T16:37:34.698034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:26.531450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867696+0000","last_change":"2023-11-08T16:37:32.902724+0000","last_active":"2023-11-08T16:37:33.867696+0000","last_peered":"2023-11-08T16:37:33.867696+0000","last_clean":"2023-11-08T16:37:33.867696+0000","last_became_active":"2023-11-08T16:37:32.901742+0000","last_became_peered":"2023-11-08T16:37:32.901742+0000","last_unstale":"2023-11-08T16:37:33.867696+0000","last_undegraded":"2023-11-08T16:37:33.867696+0000","last_fullsized":"2023-11-08T16:37:33.867696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:40:11.697591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.977843+0000","last_change":"2023-11-08T16:37:31.888025+0000","last_active":"2023-11-08T16:37:32.977843+0000","last_peered":"2023-11-08T16:37:32.977843+0000","last_clean":"2023-11-08T16:37:32.977843+0000","last_became_active":"2023-11-08T16:37:31.887704+0000","last_became_peered":"2023-11-08T16:37:31.887704+0000","last_unstale":"2023-11-08T16:37:32.977843+0000","last_undegraded":"2023-11-08T16:37:32.977843+0000","last_fullsized":"2023-11-08T16:37:32.977843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:44:48.286992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697065+0000","last_change":"2023-11-08T16:37:32.900803+0000","last_active":"2023-11-08T16:37:34.697065+0000","last_peered":"2023-11-08T16:37:34.697065+0000","last_clean":"2023-11-08T16:37:34.697065+0000","last_became_active":"2023-11-08T16:37:32.900477+0000","last_became_peered":"2023-11-08T16:37:32.900477+0000","last_unstale":"2023-11-08T16:37:34.697065+0000","last_undegraded":"2023-11-08T16:37:34.697065+0000","last_fullsized":"2023-11-08T16:37:34.697065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:20:48.435016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.976140+0000","last_change":"2023-11-08T16:37:31.886511+0000","last_active":"2023-11-08T16:37:32.976140+0000","last_peered":"2023-11-08T16:37:32.976140+0000","last_clean":"2023-11-08T16:37:32.976140+0000","last_became_active":"2023-11-08T16:37:31.886212+0000","last_became_peered":"2023-11-08T16:37:31.886212+0000","last_unstale":"2023-11-08T16:37:32.976140+0000","last_undegraded":"2023-11-08T16:37:32.976140+0000","last_fullsized":"2023-11-08T16:37:32.976140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:53:16.121610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699699+0000","last_change":"2023-11-08T16:37:32.896336+0000","last_active":"2023-11-08T16:37:34.699699+0000","last_peered":"2023-11-08T16:37:34.699699+0000","last_clean":"2023-11-08T16:37:34.699699+0000","last_became_active":"2023-11-08T16:37:32.896059+0000","last_became_peered":"2023-11-08T16:37:32.896059+0000","last_unstale":"2023-11-08T16:37:34.699699+0000","last_undegraded":"2023-11-08T16:37:34.699699+0000","last_fullsized":"2023-11-08T16:37:34.699699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:17.046902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.977489+0000","last_change":"2023-11-08T16:37:31.891095+0000","last_active":"2023-11-08T16:37:32.977489+0000","last_peered":"2023-11-08T16:37:32.977489+0000","last_clean":"2023-11-08T16:37:32.977489+0000","last_became_active":"2023-11-08T16:37:31.890816+0000","last_became_peered":"2023-11-08T16:37:31.890816+0000","last_unstale":"2023-11-08T16:37:32.977489+0000","last_undegraded":"2023-11-08T16:37:32.977489+0000","last_fullsized":"2023-11-08T16:37:32.977489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:22:17.010347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.868657+0000","last_change":"2023-11-08T16:37:32.896626+0000","last_active":"2023-11-08T16:37:33.868657+0000","last_peered":"2023-11-08T16:37:33.868657+0000","last_clean":"2023-11-08T16:37:33.868657+0000","last_became_active":"2023-11-08T16:37:32.896343+0000","last_became_peered":"2023-11-08T16:37:32.896343+0000","last_unstale":"2023-11-08T16:37:33.868657+0000","last_undegraded":"2023-11-08T16:37:33.868657+0000","last_fullsized":"2023-11-08T16:37:33.868657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:44.490627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.977031+0000","last_change":"2023-11-08T16:37:31.890832+0000","last_active":"2023-11-08T16:37:32.977031+0000","last_peered":"2023-11-08T16:37:32.977031+0000","last_clean":"2023-11-08T16:37:32.977031+0000","last_became_active":"2023-11-08T16:37:31.889708+0000","last_became_peered":"2023-11-08T16:37:31.889708+0000","last_unstale":"2023-11-08T16:37:32.977031+0000","last_undegraded":"2023-11-08T16:37:32.977031+0000","last_fullsized":"2023-11-08T16:37:32.977031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:14:03.002995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698199+0000","last_change":"2023-11-08T16:37:32.889872+0000","last_active":"2023-11-08T16:37:34.698199+0000","last_peered":"2023-11-08T16:37:34.698199+0000","last_clean":"2023-11-08T16:37:34.698199+0000","last_became_active":"2023-11-08T16:37:32.889571+0000","last_became_peered":"2023-11-08T16:37:32.889571+0000","last_unstale":"2023-11-08T16:37:34.698199+0000","last_undegraded":"2023-11-08T16:37:34.698199+0000","last_fullsized":"2023-11-08T16:37:34.698199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:08:17.120461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.975794+0000","last_change":"2023-11-08T16:37:31.883142+0000","last_active":"2023-11-08T16:37:32.975794+0000","last_peered":"2023-11-08T16:37:32.975794+0000","last_clean":"2023-11-08T16:37:32.975794+0000","last_became_active":"2023-11-08T16:37:31.882790+0000","last_became_peered":"2023-11-08T16:37:31.882790+0000","last_unstale":"2023-11-08T16:37:32.975794+0000","last_undegraded":"2023-11-08T16:37:32.975794+0000","last_fullsized":"2023-11-08T16:37:32.975794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:59:36.810489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697453+0000","last_change":"2023-11-08T16:37:32.887937+0000","last_active":"2023-11-08T16:37:34.697453+0000","last_peered":"2023-11-08T16:37:34.697453+0000","last_clean":"2023-11-08T16:37:34.697453+0000","last_became_active":"2023-11-08T16:37:32.887661+0000","last_became_peered":"2023-11-08T16:37:32.887661+0000","last_unstale":"2023-11-08T16:37:34.697453+0000","last_undegraded":"2023-11-08T16:37:34.697453+0000","last_fullsized":"2023-11-08T16:37:34.697453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:45.753772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.974915+0000","last_change":"2023-11-08T16:37:31.890699+0000","last_active":"2023-11-08T16:37:32.974915+0000","last_peered":"2023-11-08T16:37:32.974915+0000","last_clean":"2023-11-08T16:37:32.974915+0000","last_became_active":"2023-11-08T16:37:31.890155+0000","last_became_peered":"2023-11-08T16:37:31.890155+0000","last_unstale":"2023-11-08T16:37:32.974915+0000","last_undegraded":"2023-11-08T16:37:32.974915+0000","last_fullsized":"2023-11-08T16:37:32.974915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:16:05.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696235+0000","last_change":"2023-11-08T16:37:32.897426+0000","last_active":"2023-11-08T16:37:34.696235+0000","last_peered":"2023-11-08T16:37:34.696235+0000","last_clean":"2023-11-08T16:37:34.696235+0000","last_became_active":"2023-11-08T16:37:32.896041+0000","last_became_peered":"2023-11-08T16:37:32.896041+0000","last_unstale":"2023-11-08T16:37:34.696235+0000","last_undegraded":"2023-11-08T16:37:34.696235+0000","last_fullsized":"2023-11-08T16:37:34.696235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:07.806014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.877203+0000","last_change":"2023-11-08T16:37:31.878294+0000","last_active":"2023-11-08T16:37:32.877203+0000","last_peered":"2023-11-08T16:37:32.877203+0000","last_clean":"2023-11-08T16:37:32.877203+0000","last_became_active":"2023-11-08T16:37:31.877510+0000","last_became_peered":"2023-11-08T16:37:31.877510+0000","last_unstale":"2023-11-08T16:37:32.877203+0000","last_undegraded":"2023-11-08T16:37:32.877203+0000","last_fullsized":"2023-11-08T16:37:32.877203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:22.218442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864247+0000","last_change":"2023-11-08T16:37:32.895439+0000","last_active":"2023-11-08T16:37:33.864247+0000","last_peered":"2023-11-08T16:37:33.864247+0000","last_clean":"2023-11-08T16:37:33.864247+0000","last_became_active":"2023-11-08T16:37:32.895019+0000","last_became_peered":"2023-11-08T16:37:32.895019+0000","last_unstale":"2023-11-08T16:37:33.864247+0000","last_undegraded":"2023-11-08T16:37:33.864247+0000","last_fullsized":"2023-11-08T16:37:33.864247+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:44.490627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.888163+0000","last_change":"2023-11-08T16:37:31.880117+0000","last_active":"2023-11-08T16:37:32.888163+0000","last_peered":"2023-11-08T16:37:32.888163+0000","last_clean":"2023-11-08T16:37:32.888163+0000","last_became_active":"2023-11-08T16:37:31.879792+0000","last_became_peered":"2023-11-08T16:37:31.879792+0000","last_unstale":"2023-11-08T16:37:32.888163+0000","last_undegraded":"2023-11-08T16:37:32.888163+0000","last_fullsized":"2023-11-08T16:37:32.888163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:24.123715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699150+0000","last_change":"2023-11-08T16:37:32.878674+0000","last_active":"2023-11-08T16:37:34.699150+0000","last_peered":"2023-11-08T16:37:34.699150+0000","last_clean":"2023-11-08T16:37:34.699150+0000","last_became_active":"2023-11-08T16:37:32.878144+0000","last_became_peered":"2023-11-08T16:37:32.878144+0000","last_unstale":"2023-11-08T16:37:34.699150+0000","last_undegraded":"2023-11-08T16:37:34.699150+0000","last_fullsized":"2023-11-08T16:37:34.699150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:44:49.304157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.884773+0000","last_change":"2023-11-08T16:37:31.881702+0000","last_active":"2023-11-08T16:37:32.884773+0000","last_peered":"2023-11-08T16:37:32.884773+0000","last_clean":"2023-11-08T16:37:32.884773+0000","last_became_active":"2023-11-08T16:37:31.881409+0000","last_became_peered":"2023-11-08T16:37:31.881409+0000","last_unstale":"2023-11-08T16:37:32.884773+0000","last_undegraded":"2023-11-08T16:37:32.884773+0000","last_fullsized":"2023-11-08T16:37:32.884773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:16:26.042660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.868355+0000","last_change":"2023-11-08T16:37:32.894401+0000","last_active":"2023-11-08T16:37:33.868355+0000","last_peered":"2023-11-08T16:37:33.868355+0000","last_clean":"2023-11-08T16:37:33.868355+0000","last_became_active":"2023-11-08T16:37:32.894032+0000","last_became_peered":"2023-11-08T16:37:32.894032+0000","last_unstale":"2023-11-08T16:37:33.868355+0000","last_undegraded":"2023-11-08T16:37:33.868355+0000","last_fullsized":"2023-11-08T16:37:33.868355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:03:19.541668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.891743+0000","last_change":"2023-11-08T16:37:31.891676+0000","last_active":"2023-11-08T16:37:32.891743+0000","last_peered":"2023-11-08T16:37:32.891743+0000","last_clean":"2023-11-08T16:37:32.891743+0000","last_became_active":"2023-11-08T16:37:31.890845+0000","last_became_peered":"2023-11-08T16:37:31.890845+0000","last_unstale":"2023-11-08T16:37:32.891743+0000","last_undegraded":"2023-11-08T16:37:32.891743+0000","last_fullsized":"2023-11-08T16:37:32.891743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:09:53.121004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.700025+0000","last_change":"2023-11-08T16:37:32.897055+0000","last_active":"2023-11-08T16:37:34.700025+0000","last_peered":"2023-11-08T16:37:34.700025+0000","last_clean":"2023-11-08T16:37:34.700025+0000","last_became_active":"2023-11-08T16:37:32.896749+0000","last_became_peered":"2023-11-08T16:37:32.896749+0000","last_unstale":"2023-11-08T16:37:34.700025+0000","last_undegraded":"2023-11-08T16:37:34.700025+0000","last_fullsized":"2023-11-08T16:37:34.700025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:23.235607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.976568+0000","last_change":"2023-11-08T16:37:31.886957+0000","last_active":"2023-11-08T16:37:32.976568+0000","last_peered":"2023-11-08T16:37:32.976568+0000","last_clean":"2023-11-08T16:37:32.976568+0000","last_became_active":"2023-11-08T16:37:31.886649+0000","last_became_peered":"2023-11-08T16:37:31.886649+0000","last_unstale":"2023-11-08T16:37:32.976568+0000","last_undegraded":"2023-11-08T16:37:32.976568+0000","last_fullsized":"2023-11-08T16:37:32.976568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:43.473462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696871+0000","last_change":"2023-11-08T16:37:32.896696+0000","last_active":"2023-11-08T16:37:34.696871+0000","last_peered":"2023-11-08T16:37:34.696871+0000","last_clean":"2023-11-08T16:37:34.696871+0000","last_became_active":"2023-11-08T16:37:32.896424+0000","last_became_peered":"2023-11-08T16:37:32.896424+0000","last_unstale":"2023-11-08T16:37:34.696871+0000","last_undegraded":"2023-11-08T16:37:34.696871+0000","last_fullsized":"2023-11-08T16:37:34.696871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:37:07.862874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"80'6","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.978939+0000","last_change":"2023-11-08T16:37:31.878213+0000","last_active":"2023-11-08T16:37:32.978939+0000","last_peered":"2023-11-08T16:37:32.978939+0000","last_clean":"2023-11-08T16:37:32.978939+0000","last_became_active":"2023-11-08T16:37:31.877533+0000","last_became_peered":"2023-11-08T16:37:31.877533+0000","last_unstale":"2023-11-08T16:37:32.978939+0000","last_undegraded":"2023-11-08T16:37:32.978939+0000","last_fullsized":"2023-11-08T16:37:32.978939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:53:05.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867464+0000","last_change":"2023-11-08T16:37:32.891876+0000","last_active":"2023-11-08T16:37:33.867464+0000","last_peered":"2023-11-08T16:37:33.867464+0000","last_clean":"2023-11-08T16:37:33.867464+0000","last_became_active":"2023-11-08T16:37:32.890633+0000","last_became_peered":"2023-11-08T16:37:32.890633+0000","last_unstale":"2023-11-08T16:37:33.867464+0000","last_undegraded":"2023-11-08T16:37:33.867464+0000","last_fullsized":"2023-11-08T16:37:33.867464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:57.283084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.978016+0000","last_change":"2023-11-08T16:37:31.881877+0000","last_active":"2023-11-08T16:37:32.978016+0000","last_peered":"2023-11-08T16:37:32.978016+0000","last_clean":"2023-11-08T16:37:32.978016+0000","last_became_active":"2023-11-08T16:37:31.881552+0000","last_became_peered":"2023-11-08T16:37:31.881552+0000","last_unstale":"2023-11-08T16:37:32.978016+0000","last_undegraded":"2023-11-08T16:37:32.978016+0000","last_fullsized":"2023-11-08T16:37:32.978016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:05.800459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698965+0000","last_change":"2023-11-08T16:37:32.888800+0000","last_active":"2023-11-08T16:37:34.698965+0000","last_peered":"2023-11-08T16:37:34.698965+0000","last_clean":"2023-11-08T16:37:34.698965+0000","last_became_active":"2023-11-08T16:37:32.888427+0000","last_became_peered":"2023-11-08T16:37:32.888427+0000","last_unstale":"2023-11-08T16:37:34.698965+0000","last_undegraded":"2023-11-08T16:37:34.698965+0000","last_fullsized":"2023-11-08T16:37:34.698965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:06.642562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.891767+0000","last_change":"2023-11-08T16:37:31.890482+0000","last_active":"2023-11-08T16:37:32.891767+0000","last_peered":"2023-11-08T16:37:32.891767+0000","last_clean":"2023-11-08T16:37:32.891767+0000","last_became_active":"2023-11-08T16:37:31.889651+0000","last_became_peered":"2023-11-08T16:37:31.889651+0000","last_unstale":"2023-11-08T16:37:32.891767+0000","last_undegraded":"2023-11-08T16:37:32.891767+0000","last_fullsized":"2023-11-08T16:37:32.891767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:59.816607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864056+0000","last_change":"2023-11-08T16:37:32.902585+0000","last_active":"2023-11-08T16:37:33.864056+0000","last_peered":"2023-11-08T16:37:33.864056+0000","last_clean":"2023-11-08T16:37:33.864056+0000","last_became_active":"2023-11-08T16:37:32.902238+0000","last_became_peered":"2023-11-08T16:37:32.902238+0000","last_unstale":"2023-11-08T16:37:33.864056+0000","last_undegraded":"2023-11-08T16:37:33.864056+0000","last_fullsized":"2023-11-08T16:37:33.864056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:52:21.508270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.873103+0000","last_change":"2023-11-08T16:37:31.885217+0000","last_active":"2023-11-08T16:37:32.873103+0000","last_peered":"2023-11-08T16:37:32.873103+0000","last_clean":"2023-11-08T16:37:32.873103+0000","last_became_active":"2023-11-08T16:37:31.884920+0000","last_became_peered":"2023-11-08T16:37:31.884920+0000","last_unstale":"2023-11-08T16:37:32.873103+0000","last_undegraded":"2023-11-08T16:37:32.873103+0000","last_fullsized":"2023-11-08T16:37:32.873103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:20.170960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696247+0000","last_change":"2023-11-08T16:37:32.901124+0000","last_active":"2023-11-08T16:37:34.696247+0000","last_peered":"2023-11-08T16:37:34.696247+0000","last_clean":"2023-11-08T16:37:34.696247+0000","last_became_active":"2023-11-08T16:37:32.900154+0000","last_became_peered":"2023-11-08T16:37:32.900154+0000","last_unstale":"2023-11-08T16:37:34.696247+0000","last_undegraded":"2023-11-08T16:37:34.696247+0000","last_fullsized":"2023-11-08T16:37:34.696247+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:22.351916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.982833+0000","last_change":"2023-11-08T16:37:31.884151+0000","last_active":"2023-11-08T16:37:32.982833+0000","last_peered":"2023-11-08T16:37:32.982833+0000","last_clean":"2023-11-08T16:37:32.982833+0000","last_became_active":"2023-11-08T16:37:31.882949+0000","last_became_peered":"2023-11-08T16:37:31.882949+0000","last_unstale":"2023-11-08T16:37:32.982833+0000","last_undegraded":"2023-11-08T16:37:32.982833+0000","last_fullsized":"2023-11-08T16:37:32.982833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:34:59.821142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698219+0000","last_change":"2023-11-08T16:37:32.889628+0000","last_active":"2023-11-08T16:37:34.698219+0000","last_peered":"2023-11-08T16:37:34.698219+0000","last_clean":"2023-11-08T16:37:34.698219+0000","last_became_active":"2023-11-08T16:37:32.889352+0000","last_became_peered":"2023-11-08T16:37:32.889352+0000","last_unstale":"2023-11-08T16:37:34.698219+0000","last_undegraded":"2023-11-08T16:37:34.698219+0000","last_fullsized":"2023-11-08T16:37:34.698219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:27:30.791214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.976461+0000","last_change":"2023-11-08T16:37:31.885970+0000","last_active":"2023-11-08T16:37:32.976461+0000","last_peered":"2023-11-08T16:37:32.976461+0000","last_clean":"2023-11-08T16:37:32.976461+0000","last_became_active":"2023-11-08T16:37:31.885703+0000","last_became_peered":"2023-11-08T16:37:31.885703+0000","last_unstale":"2023-11-08T16:37:32.976461+0000","last_undegraded":"2023-11-08T16:37:32.976461+0000","last_fullsized":"2023-11-08T16:37:32.976461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:15:10.920507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.868621+0000","last_change":"2023-11-08T16:37:32.891988+0000","last_active":"2023-11-08T16:37:33.868621+0000","last_peered":"2023-11-08T16:37:33.868621+0000","last_clean":"2023-11-08T16:37:33.868621+0000","last_became_active":"2023-11-08T16:37:32.891705+0000","last_became_peered":"2023-11-08T16:37:32.891705+0000","last_unstale":"2023-11-08T16:37:33.868621+0000","last_undegraded":"2023-11-08T16:37:33.868621+0000","last_fullsized":"2023-11-08T16:37:33.868621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:52:21.508270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.979303+0000","last_change":"2023-11-08T16:37:31.891191+0000","last_active":"2023-11-08T16:37:32.979303+0000","last_peered":"2023-11-08T16:37:32.979303+0000","last_clean":"2023-11-08T16:37:32.979303+0000","last_became_active":"2023-11-08T16:37:31.890873+0000","last_became_peered":"2023-11-08T16:37:31.890873+0000","last_unstale":"2023-11-08T16:37:32.979303+0000","last_undegraded":"2023-11-08T16:37:32.979303+0000","last_fullsized":"2023-11-08T16:37:32.979303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:15:10.920507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.865813+0000","last_change":"2023-11-08T16:37:32.890877+0000","last_active":"2023-11-08T16:37:33.865813+0000","last_peered":"2023-11-08T16:37:33.865813+0000","last_clean":"2023-11-08T16:37:33.865813+0000","last_became_active":"2023-11-08T16:37:32.889313+0000","last_became_peered":"2023-11-08T16:37:32.889313+0000","last_unstale":"2023-11-08T16:37:33.865813+0000","last_undegraded":"2023-11-08T16:37:33.865813+0000","last_fullsized":"2023-11-08T16:37:33.865813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:39.874473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.977489+0000","last_change":"2023-11-08T16:37:31.892128+0000","last_active":"2023-11-08T16:37:32.977489+0000","last_peered":"2023-11-08T16:37:32.977489+0000","last_clean":"2023-11-08T16:37:32.977489+0000","last_became_active":"2023-11-08T16:37:31.891820+0000","last_became_peered":"2023-11-08T16:37:31.891820+0000","last_unstale":"2023-11-08T16:37:32.977489+0000","last_undegraded":"2023-11-08T16:37:32.977489+0000","last_fullsized":"2023-11-08T16:37:32.977489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:38.798225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696313+0000","last_change":"2023-11-08T16:37:32.896534+0000","last_active":"2023-11-08T16:37:34.696313+0000","last_peered":"2023-11-08T16:37:34.696313+0000","last_clean":"2023-11-08T16:37:34.696313+0000","last_became_active":"2023-11-08T16:37:32.896085+0000","last_became_peered":"2023-11-08T16:37:32.896085+0000","last_unstale":"2023-11-08T16:37:34.696313+0000","last_undegraded":"2023-11-08T16:37:34.696313+0000","last_fullsized":"2023-11-08T16:37:34.696313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:26:37.454187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.866566+0000","last_change":"2023-11-08T16:37:31.888250+0000","last_active":"2023-11-08T16:37:32.866566+0000","last_peered":"2023-11-08T16:37:32.866566+0000","last_clean":"2023-11-08T16:37:32.866566+0000","last_became_active":"2023-11-08T16:37:31.887364+0000","last_became_peered":"2023-11-08T16:37:31.887364+0000","last_unstale":"2023-11-08T16:37:32.866566+0000","last_undegraded":"2023-11-08T16:37:32.866566+0000","last_fullsized":"2023-11-08T16:37:32.866566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:20.170960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107632+0000","last_change":"2023-11-08T16:37:32.889356+0000","last_active":"2023-11-08T16:37:34.107632+0000","last_peered":"2023-11-08T16:37:34.107632+0000","last_clean":"2023-11-08T16:37:34.107632+0000","last_became_active":"2023-11-08T16:37:32.887921+0000","last_became_peered":"2023-11-08T16:37:32.887921+0000","last_unstale":"2023-11-08T16:37:34.107632+0000","last_undegraded":"2023-11-08T16:37:34.107632+0000","last_fullsized":"2023-11-08T16:37:34.107632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:24:35.603662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.891840+0000","last_change":"2023-11-08T16:37:31.890236+0000","last_active":"2023-11-08T16:37:32.891840+0000","last_peered":"2023-11-08T16:37:32.891840+0000","last_clean":"2023-11-08T16:37:32.891840+0000","last_became_active":"2023-11-08T16:37:31.889327+0000","last_became_peered":"2023-11-08T16:37:31.889327+0000","last_unstale":"2023-11-08T16:37:32.891840+0000","last_undegraded":"2023-11-08T16:37:32.891840+0000","last_fullsized":"2023-11-08T16:37:32.891840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:34:59.821142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107319+0000","last_change":"2023-11-08T16:37:32.898723+0000","last_active":"2023-11-08T16:37:34.107319+0000","last_peered":"2023-11-08T16:37:34.107319+0000","last_clean":"2023-11-08T16:37:34.107319+0000","last_became_active":"2023-11-08T16:37:32.897690+0000","last_became_peered":"2023-11-08T16:37:32.897690+0000","last_unstale":"2023-11-08T16:37:34.107319+0000","last_undegraded":"2023-11-08T16:37:34.107319+0000","last_fullsized":"2023-11-08T16:37:34.107319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:09:01.413694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.873413+0000","last_change":"2023-11-08T16:37:31.881532+0000","last_active":"2023-11-08T16:37:32.873413+0000","last_peered":"2023-11-08T16:37:32.873413+0000","last_clean":"2023-11-08T16:37:32.873413+0000","last_became_active":"2023-11-08T16:37:31.881220+0000","last_became_peered":"2023-11-08T16:37:31.881220+0000","last_unstale":"2023-11-08T16:37:32.873413+0000","last_undegraded":"2023-11-08T16:37:32.873413+0000","last_fullsized":"2023-11-08T16:37:32.873413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:38.857308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107599+0000","last_change":"2023-11-08T16:37:32.901904+0000","last_active":"2023-11-08T16:37:34.107599+0000","last_peered":"2023-11-08T16:37:34.107599+0000","last_clean":"2023-11-08T16:37:34.107599+0000","last_became_active":"2023-11-08T16:37:32.901589+0000","last_became_peered":"2023-11-08T16:37:32.901589+0000","last_unstale":"2023-11-08T16:37:34.107599+0000","last_undegraded":"2023-11-08T16:37:34.107599+0000","last_fullsized":"2023-11-08T16:37:34.107599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:02.805859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.887306+0000","last_change":"2023-11-08T16:37:31.885921+0000","last_active":"2023-11-08T16:37:32.887306+0000","last_peered":"2023-11-08T16:37:32.887306+0000","last_clean":"2023-11-08T16:37:32.887306+0000","last_became_active":"2023-11-08T16:37:31.885621+0000","last_became_peered":"2023-11-08T16:37:31.885621+0000","last_unstale":"2023-11-08T16:37:32.887306+0000","last_undegraded":"2023-11-08T16:37:32.887306+0000","last_fullsized":"2023-11-08T16:37:32.887306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:56.871455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864362+0000","last_change":"2023-11-08T16:37:32.896253+0000","last_active":"2023-11-08T16:37:33.864362+0000","last_peered":"2023-11-08T16:37:33.864362+0000","last_clean":"2023-11-08T16:37:33.864362+0000","last_became_active":"2023-11-08T16:37:32.895943+0000","last_became_peered":"2023-11-08T16:37:32.895943+0000","last_unstale":"2023-11-08T16:37:33.864362+0000","last_undegraded":"2023-11-08T16:37:33.864362+0000","last_fullsized":"2023-11-08T16:37:33.864362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:57.489427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.977235+0000","last_change":"2023-11-08T16:37:31.886864+0000","last_active":"2023-11-08T16:37:32.977235+0000","last_peered":"2023-11-08T16:37:32.977235+0000","last_clean":"2023-11-08T16:37:32.977235+0000","last_became_active":"2023-11-08T16:37:31.886526+0000","last_became_peered":"2023-11-08T16:37:31.886526+0000","last_unstale":"2023-11-08T16:37:32.977235+0000","last_undegraded":"2023-11-08T16:37:32.977235+0000","last_fullsized":"2023-11-08T16:37:32.977235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:16:26.042660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699708+0000","last_change":"2023-11-08T16:37:32.889339+0000","last_active":"2023-11-08T16:37:34.699708+0000","last_peered":"2023-11-08T16:37:34.699708+0000","last_clean":"2023-11-08T16:37:34.699708+0000","last_became_active":"2023-11-08T16:37:32.889059+0000","last_became_peered":"2023-11-08T16:37:32.889059+0000","last_unstale":"2023-11-08T16:37:34.699708+0000","last_undegraded":"2023-11-08T16:37:34.699708+0000","last_fullsized":"2023-11-08T16:37:34.699708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:13:41.486934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699419+0000","last_change":"2023-11-08T16:37:31.887047+0000","last_active":"2023-11-08T16:37:34.699419+0000","last_peered":"2023-11-08T16:37:34.699419+0000","last_clean":"2023-11-08T16:37:34.699419+0000","last_became_active":"2023-11-08T16:37:31.884896+0000","last_became_peered":"2023-11-08T16:37:31.884896+0000","last_unstale":"2023-11-08T16:37:34.699419+0000","last_undegraded":"2023-11-08T16:37:34.699419+0000","last_fullsized":"2023-11-08T16:37:34.699419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:18:18.736847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867883+0000","last_change":"2023-11-08T16:37:32.891927+0000","last_active":"2023-11-08T16:37:33.867883+0000","last_peered":"2023-11-08T16:37:33.867883+0000","last_clean":"2023-11-08T16:37:33.867883+0000","last_became_active":"2023-11-08T16:37:32.891570+0000","last_became_peered":"2023-11-08T16:37:32.891570+0000","last_unstale":"2023-11-08T16:37:33.867883+0000","last_undegraded":"2023-11-08T16:37:33.867883+0000","last_fullsized":"2023-11-08T16:37:33.867883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:13:26.284908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.888996+0000","last_change":"2023-11-08T16:37:31.879909+0000","last_active":"2023-11-08T16:37:32.888996+0000","last_peered":"2023-11-08T16:37:32.888996+0000","last_clean":"2023-11-08T16:37:32.888996+0000","last_became_active":"2023-11-08T16:37:31.878874+0000","last_became_peered":"2023-11-08T16:37:31.878874+0000","last_unstale":"2023-11-08T16:37:32.888996+0000","last_undegraded":"2023-11-08T16:37:32.888996+0000","last_fullsized":"2023-11-08T16:37:32.888996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:18:57.040797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.868851+0000","last_change":"2023-11-08T16:37:32.889394+0000","last_active":"2023-11-08T16:37:33.868851+0000","last_peered":"2023-11-08T16:37:33.868851+0000","last_clean":"2023-11-08T16:37:33.868851+0000","last_became_active":"2023-11-08T16:37:32.889046+0000","last_became_peered":"2023-11-08T16:37:32.889046+0000","last_unstale":"2023-11-08T16:37:33.868851+0000","last_undegraded":"2023-11-08T16:37:33.868851+0000","last_fullsized":"2023-11-08T16:37:33.868851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:00:16.203256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.883781+0000","last_change":"2023-11-08T16:37:31.889271+0000","last_active":"2023-11-08T16:37:32.883781+0000","last_peered":"2023-11-08T16:37:32.883781+0000","last_clean":"2023-11-08T16:37:32.883781+0000","last_became_active":"2023-11-08T16:37:31.888932+0000","last_became_peered":"2023-11-08T16:37:31.888932+0000","last_unstale":"2023-11-08T16:37:32.883781+0000","last_undegraded":"2023-11-08T16:37:32.883781+0000","last_fullsized":"2023-11-08T16:37:32.883781+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:56.265919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696444+0000","last_change":"2023-11-08T16:37:32.890369+0000","last_active":"2023-11-08T16:37:34.696444+0000","last_peered":"2023-11-08T16:37:34.696444+0000","last_clean":"2023-11-08T16:37:34.696444+0000","last_became_active":"2023-11-08T16:37:32.890080+0000","last_became_peered":"2023-11-08T16:37:32.890080+0000","last_unstale":"2023-11-08T16:37:34.696444+0000","last_undegraded":"2023-11-08T16:37:34.696444+0000","last_fullsized":"2023-11-08T16:37:34.696444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:11:09.318318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.885602+0000","last_change":"2023-11-08T16:37:31.888523+0000","last_active":"2023-11-08T16:37:32.885602+0000","last_peered":"2023-11-08T16:37:32.885602+0000","last_clean":"2023-11-08T16:37:32.885602+0000","last_became_active":"2023-11-08T16:37:31.888132+0000","last_became_peered":"2023-11-08T16:37:31.888132+0000","last_unstale":"2023-11-08T16:37:32.885602+0000","last_undegraded":"2023-11-08T16:37:32.885602+0000","last_fullsized":"2023-11-08T16:37:32.885602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:30.803105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697698+0000","last_change":"2023-11-08T16:37:32.895914+0000","last_active":"2023-11-08T16:37:34.697698+0000","last_peered":"2023-11-08T16:37:34.697698+0000","last_clean":"2023-11-08T16:37:34.697698+0000","last_became_active":"2023-11-08T16:37:32.895636+0000","last_became_peered":"2023-11-08T16:37:32.895636+0000","last_unstale":"2023-11-08T16:37:34.697698+0000","last_undegraded":"2023-11-08T16:37:34.697698+0000","last_fullsized":"2023-11-08T16:37:34.697698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:32:09.784202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.876843+0000","last_change":"2023-11-08T16:37:31.882693+0000","last_active":"2023-11-08T16:37:32.876843+0000","last_peered":"2023-11-08T16:37:32.876843+0000","last_clean":"2023-11-08T16:37:32.876843+0000","last_became_active":"2023-11-08T16:37:31.882403+0000","last_became_peered":"2023-11-08T16:37:31.882403+0000","last_unstale":"2023-11-08T16:37:32.876843+0000","last_undegraded":"2023-11-08T16:37:32.876843+0000","last_fullsized":"2023-11-08T16:37:32.876843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:27:29.774049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.865464+0000","last_change":"2023-11-08T16:37:32.892923+0000","last_active":"2023-11-08T16:37:33.865464+0000","last_peered":"2023-11-08T16:37:33.865464+0000","last_clean":"2023-11-08T16:37:33.865464+0000","last_became_active":"2023-11-08T16:37:32.889324+0000","last_became_peered":"2023-11-08T16:37:32.889324+0000","last_unstale":"2023-11-08T16:37:33.865464+0000","last_undegraded":"2023-11-08T16:37:33.865464+0000","last_fullsized":"2023-11-08T16:37:33.865464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:44:12.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.883572+0000","last_change":"2023-11-08T16:37:31.885170+0000","last_active":"2023-11-08T16:37:32.883572+0000","last_peered":"2023-11-08T16:37:32.883572+0000","last_clean":"2023-11-08T16:37:32.883572+0000","last_became_active":"2023-11-08T16:37:31.884855+0000","last_became_peered":"2023-11-08T16:37:31.884855+0000","last_unstale":"2023-11-08T16:37:32.883572+0000","last_undegraded":"2023-11-08T16:37:32.883572+0000","last_fullsized":"2023-11-08T16:37:32.883572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:09:53.121004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697502+0000","last_change":"2023-11-08T16:37:32.899028+0000","last_active":"2023-11-08T16:37:34.697502+0000","last_peered":"2023-11-08T16:37:34.697502+0000","last_clean":"2023-11-08T16:37:34.697502+0000","last_became_active":"2023-11-08T16:37:32.898773+0000","last_became_peered":"2023-11-08T16:37:32.898773+0000","last_unstale":"2023-11-08T16:37:34.697502+0000","last_undegraded":"2023-11-08T16:37:34.697502+0000","last_fullsized":"2023-11-08T16:37:34.697502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:18.438471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.891917+0000","last_change":"2023-11-08T16:37:31.888762+0000","last_active":"2023-11-08T16:37:32.891917+0000","last_peered":"2023-11-08T16:37:32.891917+0000","last_clean":"2023-11-08T16:37:32.891917+0000","last_became_active":"2023-11-08T16:37:31.888322+0000","last_became_peered":"2023-11-08T16:37:31.888322+0000","last_unstale":"2023-11-08T16:37:32.891917+0000","last_undegraded":"2023-11-08T16:37:32.891917+0000","last_fullsized":"2023-11-08T16:37:32.891917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:22:17.010347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107673+0000","last_change":"2023-11-08T16:37:32.902276+0000","last_active":"2023-11-08T16:37:34.107673+0000","last_peered":"2023-11-08T16:37:34.107673+0000","last_clean":"2023-11-08T16:37:34.107673+0000","last_became_active":"2023-11-08T16:37:32.902005+0000","last_became_peered":"2023-11-08T16:37:32.902005+0000","last_unstale":"2023-11-08T16:37:34.107673+0000","last_undegraded":"2023-11-08T16:37:34.107673+0000","last_fullsized":"2023-11-08T16:37:34.107673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:39:11.763525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.976897+0000","last_change":"2023-11-08T16:37:31.889695+0000","last_active":"2023-11-08T16:37:32.976897+0000","last_peered":"2023-11-08T16:37:32.976897+0000","last_clean":"2023-11-08T16:37:32.976897+0000","last_became_active":"2023-11-08T16:37:31.889172+0000","last_became_peered":"2023-11-08T16:37:31.889172+0000","last_unstale":"2023-11-08T16:37:32.976897+0000","last_undegraded":"2023-11-08T16:37:32.976897+0000","last_fullsized":"2023-11-08T16:37:32.976897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:01.029956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":229,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.870100+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:37:32.870100+0000","last_peered":"2023-11-08T16:37:32.870100+0000","last_clean":"2023-11-08T16:37:32.870100+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:37:32.870100+0000","last_undegraded":"2023-11-08T16:37:32.870100+0000","last_fullsized":"2023-11-08T16:37:32.870100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":1035499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0900000000000001,"15min":1.0860000000000001},"min":{"1min":0.67000000000000004,"5min":0.51600000000000001,"15min":0.49299999999999999},"max":{"1min":1.6879999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.876},{"interface":"front","average":{"1min":1.014,"5min":1.0880000000000001,"15min":1.111},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3819999999999999,"5min":2.7599999999999998,"15min":4.2729999999999997},"last":1.145}]},{"osd":1,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0960000000000001,"15min":1.117},"min":{"1min":0.72399999999999998,"5min":0.60199999999999998,"15min":0.57899999999999996},"max":{"1min":1.52,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.476},{"interface":"front","average":{"1min":1.129,"5min":1.2010000000000001,"15min":1.2190000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5009999999999999,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":1.2090000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.171,"15min":1.248},"min":{"1min":0.75900000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4359999999999999,"5min":1.7230000000000001,"15min":4.2249999999999996},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.208},"min":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.56299999999999994},"max":{"1min":1.524,"5min":3.4750000000000001,"15min":3.4750000000000001},"last":0.752}]},{"osd":3,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1759999999999999,"15min":1.1619999999999999},"min":{"1min":0.70299999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.589,"5min":2.964,"15min":2.964},"last":1.25},{"interface":"front","average":{"1min":1.337,"5min":1.3480000000000001,"15min":1.4490000000000001},"min":{"1min":0.86899999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.7090000000000001,"5min":3.5680000000000001,"15min":4.181},"last":1.4299999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.232,"15min":1.224},"min":{"1min":0.83099999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.53,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":1.034},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3200000000000001,"15min":1.335},"min":{"1min":0.872,"5min":0.77300000000000002,"15min":0.59099999999999997},"max":{"1min":1.73,"5min":1.9350000000000001,"15min":2.3730000000000002},"last":1.4059999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4159999999999999,"15min":1.361},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.61799999999999999},"max":{"1min":1.6930000000000001,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.333},{"interface":"front","average":{"1min":1.379,"5min":1.456,"15min":1.4630000000000001},"min":{"1min":1.0760000000000001,"5min":0.97399999999999998,"15min":0.878},"max":{"1min":1.6279999999999999,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.5249999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.53},"min":{"1min":1.129,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.7929999999999999,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":1.573},{"interface":"front","average":{"1min":1.427,"5min":1.5129999999999999,"15min":1.4830000000000001},"min":{"1min":1.018,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8180000000000001,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":1.5409999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":444607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.042,"15min":1.056},"min":{"1min":0.60399999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.51,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.095,"15min":1.0780000000000001},"min":{"1min":0.63100000000000001,"5min":0.56999999999999995,"15min":0.56200000000000006},"max":{"1min":1.5109999999999999,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.79300000000000004}]},{"osd":1,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0820000000000001,"15min":1.05},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.609},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.113,"15min":1.1160000000000001},"min":{"1min":0.71899999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3440000000000001,"5min":1.97,"15min":1.97},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.131,"15min":1.1950000000000001},"min":{"1min":0.71899999999999997,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.6890000000000001,"5min":1.804,"15min":1.804},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1240000000000001,"15min":1.163},"min":{"1min":0.80000000000000004,"5min":0.61899999999999999,"15min":0.60499999999999998},"max":{"1min":1.5429999999999999,"5min":1.7310000000000001,"15min":1.7669999999999999},"last":1.4339999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.216,"15min":1.264},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.58299999999999996},"max":{"1min":1.4710000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2370000000000001,"15min":1.292},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.67300000000000004},"max":{"1min":1.599,"5min":1.76,"15min":1.95},"last":1.0720000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3029999999999999,"15min":1.3520000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.65300000000000002},"max":{"1min":1.595,"5min":2.218,"15min":2.218},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.3380000000000001,"15min":1.4139999999999999},"min":{"1min":1.095,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.778,"5min":2.085,"15min":2.7080000000000002},"last":1.1819999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3280000000000001,"15min":1.3149999999999999},"min":{"1min":0.96799999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.845,"5min":1.8839999999999999,"15min":2.105},"last":1.587},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":1.202,"5min":1.1579999999999999,"15min":0.78500000000000003},"max":{"1min":1.9630000000000001,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.4099999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.5129999999999999,"15min":1.583},"min":{"1min":1.228,"5min":1.002,"15min":0.74299999999999999},"max":{"1min":1.7949999999999999,"5min":2.8849999999999998,"15min":3.8959999999999999},"last":1.357},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.4790000000000001,"15min":1.514},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96099999999999997},"max":{"1min":2,"5min":2,"15min":2.4889999999999999},"last":1.8300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574957,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":991232,"data_stored":444605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0529999999999999,"15min":1.0589999999999999},"min":{"1min":0.72799999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.409,"5min":1.7969999999999999,"15min":2.4609999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1,"5min":1.0780000000000001,"15min":1.111},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3460000000000001,"5min":1.593,"15min":2.3639999999999999},"last":1.544}]},{"osd":1,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.234,"15min":1.2390000000000001},"min":{"1min":0.70599999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5269999999999999,"5min":1.9610000000000001,"15min":3.8690000000000002},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.177,"15min":1.26},"min":{"1min":0.66400000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.651,"5min":1.9370000000000001,"15min":3.2349999999999999},"last":1.2669999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1399999999999999,"15min":1.2170000000000001},"min":{"1min":0.77700000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5369999999999999,"5min":1.778,"15min":2.6339999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.198,"15min":1.2150000000000001},"min":{"1min":0.89900000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4419999999999999,"5min":1.8029999999999999,"15min":2.2559999999999998},"last":1.163}]},{"osd":3,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.238,"15min":1.294},"min":{"1min":0.90900000000000003,"5min":0.67300000000000004,"15min":0.66500000000000004},"max":{"1min":1.4910000000000001,"5min":1.8620000000000001,"15min":2.8330000000000002},"last":1.46},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.246,"15min":1.298},"min":{"1min":0.753,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.5840000000000001,"5min":2.0459999999999998,"15min":2.5760000000000001},"last":1.397}]},{"osd":4,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.278,"15min":1.351},"min":{"1min":0.89300000000000002,"5min":0.76700000000000002,"15min":0.61599999999999999},"max":{"1min":1.736,"5min":2.0790000000000002,"15min":3.9790000000000001},"last":1.103},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.409,"15min":1.47},"min":{"1min":1.1910000000000001,"5min":0.97699999999999998,"15min":0.81499999999999995},"max":{"1min":1.633,"5min":2.2010000000000001,"15min":3.0209999999999999},"last":1.0569999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.399,"15min":1.4139999999999999},"min":{"1min":1.1020000000000001,"5min":0.92400000000000004,"15min":0.752},"max":{"1min":1.673,"5min":1.867,"15min":2.5550000000000002},"last":1.603},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4099999999999999,"15min":1.4910000000000001},"min":{"1min":0.999,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":1.6870000000000001,"5min":2.3210000000000002,"15min":4.1189999999999998},"last":1.337}]},{"osd":7,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.522,"15min":1.5660000000000001},"min":{"1min":1.218,"5min":1.099,"15min":1.0049999999999999},"max":{"1min":1.7789999999999999,"5min":2.129,"15min":3.1160000000000001},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.556,"5min":1.5269999999999999,"15min":1.556},"min":{"1min":1.3029999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":1.9219999999999999,"5min":2.3660000000000001,"15min":2.7050000000000001},"last":1.3080000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574956,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":991232,"data_stored":444585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.105,"15min":1.079},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.534,"5min":1.587,"15min":1.587},"last":1.597},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.091,"15min":1.0609999999999999},"min":{"1min":0.59899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.383,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.2010000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.133,"15min":1.125},"min":{"1min":0.77900000000000003,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4490000000000001,"5min":1.8660000000000001,"15min":2.0459999999999998},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.109,"5min":1.1819999999999999,"15min":1.1659999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5489999999999999,"5min":1.742,"15min":1.742},"last":1.421}]},{"osd":2,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.218,"15min":1.1950000000000001},"min":{"1min":0.69599999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.746},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.2,"15min":1.252},"min":{"1min":0.86399999999999999,"5min":0.77800000000000002,"15min":0.71699999999999997},"max":{"1min":1.474,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.129}]},{"osd":3,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.24,"15min":1.2649999999999999},"min":{"1min":0.82399999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.694,"5min":1.7689999999999999,"15min":1.9199999999999999},"last":1.661},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.236,"15min":1.2749999999999999},"min":{"1min":0.72799999999999998,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.611,"5min":1.851,"15min":1.895},"last":1.073}]},{"osd":5,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.361,"15min":1.3640000000000001},"min":{"1min":0.92700000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.736,"5min":1.8959999999999999,"15min":1.9870000000000001},"last":1.339},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4330000000000001,"15min":1.4239999999999999},"min":{"1min":1.196,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.798,"5min":1.8899999999999999,"15min":2.46},"last":1.4710000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.333,"15min":1.4059999999999999},"min":{"1min":0.98499999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7230000000000001,"5min":1.9510000000000001,"15min":2.5609999999999999},"last":1.256},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.5,"15min":1.552},"min":{"1min":1.1659999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.825,"5min":2.0310000000000001,"15min":2.6389999999999998},"last":1.514}]},{"osd":7,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.516,"15min":1.52},"min":{"1min":1.056,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.881,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.298},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.4970000000000001,"15min":1.5800000000000001},"min":{"1min":1.282,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.837,"5min":2.0670000000000002,"15min":2.6859999999999999},"last":1.3799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":444627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77100000000000002,"15min":0.81399999999999995},"min":{"1min":0.50900000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0289999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81499999999999995,"15min":0.92100000000000004},"min":{"1min":0.50800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":1.2390000000000001,"5min":1.3919999999999999,"15min":1.585},"last":0.745}]},{"osd":1,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.875,"15min":0.89700000000000002},"min":{"1min":0.65000000000000002,"5min":0.44600000000000001,"15min":0.36499999999999999},"max":{"1min":1.264,"5min":1.4770000000000001,"15min":1.4790000000000001},"last":1.101},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.93899999999999995,"15min":0.96599999999999997},"min":{"1min":0.59499999999999997,"5min":0.378,"15min":0.378},"max":{"1min":1.2629999999999999,"5min":1.4590000000000001,"15min":1.829},"last":0.69399999999999995}]},{"osd":2,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.99099999999999999,"15min":0.98699999999999999},"min":{"1min":0.60499999999999998,"5min":0.48399999999999999,"15min":0.41699999999999998},"max":{"1min":1.7490000000000001,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.5109999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.075,"15min":1.129},"min":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.377,"5min":1.6839999999999999,"15min":2.4660000000000002},"last":1.625}]},{"osd":4,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.143,"15min":1.1970000000000001},"min":{"1min":0.82099999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.49,"5min":1.8069999999999999,"15min":2.25},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2410000000000001,"15min":1.266},"min":{"1min":0.93899999999999995,"5min":0.747,"15min":0.747},"max":{"1min":1.5840000000000001,"5min":1.8280000000000001,"15min":2.1829999999999998},"last":1.3169999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3029999999999999,"15min":1.329},"min":{"1min":1.0249999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.76,"5min":1.891,"15min":2.3999999999999999},"last":1.214},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.2969999999999999,"15min":1.341},"min":{"1min":0.91600000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.669,"5min":1.7869999999999999,"15min":2.593},"last":1.5569999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.341,"15min":1.3999999999999999},"min":{"1min":0.96499999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7070000000000001,"5min":1.744,"15min":2.9569999999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.2849999999999999,"15min":1.391},"min":{"1min":1.006,"5min":0.75800000000000001,"15min":0.73399999999999999},"max":{"1min":1.5860000000000001,"5min":1.6890000000000001,"15min":2.762},"last":1.377}]},{"osd":7,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.319,"15min":1.4319999999999999},"min":{"1min":0.95599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.677,"5min":1.861,"15min":2.8700000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3720000000000001,"15min":1.4279999999999999},"min":{"1min":1.113,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.71,"5min":2.1739999999999999,"15min":2.633},"last":1.849}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":446127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79400000000000004,"15min":0.98599999999999999},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.24099999999999999},"max":{"1min":1.2030000000000001,"5min":1.706,"15min":4.5369999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.83299999999999996,"15min":0.873},"min":{"1min":0.48499999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.3620000000000001,"5min":1.611,"15min":1.611},"last":1.0449999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.95999999999999996,"15min":1.1000000000000001},"min":{"1min":0.67800000000000005,"5min":0.52300000000000002,"15min":0.379},"max":{"1min":1.4159999999999999,"5min":1.6000000000000001,"15min":4.4980000000000002},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.97499999999999998,"15min":1.0449999999999999},"min":{"1min":0.63400000000000001,"5min":0.53600000000000003,"15min":0.40300000000000002},"max":{"1min":1.3540000000000001,"5min":1.6220000000000001,"15min":1.6879999999999999},"last":1.2090000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84499999999999997,"15min":1.024},"min":{"1min":0.49099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.25,"5min":1.3879999999999999,"15min":3.3050000000000002},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0249999999999999,"15min":1.2110000000000001},"min":{"1min":0.60599999999999998,"5min":0.502,"15min":0.42999999999999999},"max":{"1min":1.3360000000000001,"5min":1.524,"15min":4.5899999999999999},"last":1.4530000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.246,"15min":1.3999999999999999},"min":{"1min":0.92200000000000004,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.593,"5min":1.5980000000000001,"15min":4.9409999999999998},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.165,"15min":1.2709999999999999},"min":{"1min":0.83399999999999996,"5min":0.72499999999999998,"15min":0.63100000000000001},"max":{"1min":1.5389999999999999,"5min":1.625,"15min":2.46},"last":1.341}]},{"osd":5,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2130000000000001,"15min":1.3149999999999999},"min":{"1min":0.88600000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4790000000000001,"5min":1.637,"15min":2.8530000000000002},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2230000000000001,"15min":1.351},"min":{"1min":0.92600000000000005,"5min":0.71099999999999997,"15min":0.69399999999999995},"max":{"1min":1.5269999999999999,"5min":1.6659999999999999,"15min":3.5680000000000001},"last":1.375}]},{"osd":6,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2470000000000001,"15min":1.393},"min":{"1min":0.93700000000000006,"5min":0.76600000000000001,"15min":0.625},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":3.536},"last":1.427},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.363,"15min":1.5009999999999999},"min":{"1min":1.095,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.74,"5min":1.74,"15min":4.7549999999999999},"last":1.6499999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2969999999999999,"15min":1.4339999999999999},"min":{"1min":1.0289999999999999,"5min":0.748,"15min":0.72999999999999998},"max":{"1min":1.5880000000000001,"5min":1.7290000000000001,"15min":2.9700000000000002},"last":1.5},{"interface":"front","average":{"1min":1.369,"5min":1.3420000000000001,"15min":1.4930000000000001},"min":{"1min":1.0900000000000001,"5min":0.81399999999999995,"15min":0.70499999999999996},"max":{"1min":1.881,"5min":2.0449999999999999,"15min":3.4950000000000001},"last":1.3100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":446743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.753,"15min":0.77300000000000002},"min":{"1min":0.42999999999999999,"5min":0.33400000000000002,"15min":0.28899999999999998},"max":{"1min":0.98699999999999999,"5min":1.542,"15min":1.542},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.86599999999999999,"15min":0.995},"min":{"1min":0.56399999999999995,"5min":0.49199999999999999,"15min":0.309},"max":{"1min":1.2230000000000001,"5min":1.458,"15min":4.0309999999999997},"last":1.2290000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.97499999999999998,"15min":1.0780000000000001},"min":{"1min":0.54800000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":3.0339999999999998},"last":1.7689999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92800000000000005,"15min":1.0620000000000001},"min":{"1min":0.55700000000000005,"5min":0.44600000000000001,"15min":0.443},"max":{"1min":1.4750000000000001,"5min":1.5169999999999999,"15min":2.6840000000000002},"last":1.0429999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":1.093},"min":{"1min":0.59399999999999997,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":1.468,"5min":1.476,"15min":4.3179999999999996},"last":1.383},{"interface":"front","average":{"1min":1.032,"5min":1.0509999999999999,"15min":1.1140000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.34999999999999998},"max":{"1min":1.5900000000000001,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":1.5169999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.155,"15min":1.2969999999999999},"min":{"1min":0.92600000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4359999999999999,"5min":1.698,"15min":3.1659999999999999},"last":1.131},{"interface":"front","average":{"1min":1.208,"5min":1.2230000000000001,"15min":1.3300000000000001},"min":{"1min":0.86899999999999999,"5min":0.71399999999999997,"15min":0.67600000000000005},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":2.734},"last":1.823}]},{"osd":5,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.2490000000000001,"15min":1.6040000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.67100000000000004},"max":{"1min":1.921,"5min":1.921,"15min":8.4730000000000008},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2809999999999999,"15min":1.5800000000000001},"min":{"1min":1.0149999999999999,"5min":0.84599999999999997,"15min":0.69399999999999995},"max":{"1min":1.6100000000000001,"5min":1.9770000000000001,"15min":8.8040000000000003},"last":1.871}]},{"osd":6,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.29,"15min":1.444},"min":{"1min":0.77300000000000002,"5min":0.69899999999999995,"15min":0.68700000000000006},"max":{"1min":1.831,"5min":1.831,"15min":3.984},"last":1.9119999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3140000000000001,"15min":1.4379999999999999},"min":{"1min":1.034,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.554,"5min":1.843,"15min":3.2450000000000001},"last":1.427}]},{"osd":7,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.298,"15min":1.514},"min":{"1min":0.91400000000000003,"5min":0.81399999999999995,"15min":0.69199999999999995},"max":{"1min":1.6919999999999999,"5min":2.0019999999999998,"15min":4.3609999999999998},"last":1.681},{"interface":"front","average":{"1min":1.462,"5min":1.3819999999999999,"15min":1.5940000000000001},"min":{"1min":0.81299999999999994,"5min":0.77700000000000002,"15min":0.70499999999999996},"max":{"1min":1.972,"5min":2.8780000000000001,"15min":5.3840000000000003},"last":1.7170000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":1034883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.873,"15min":0.81200000000000006},"min":{"1min":0.57499999999999996,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.137,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.94099999999999995,"15min":0.96799999999999997},"min":{"1min":0.52000000000000002,"5min":0.498,"15min":0.44800000000000001},"max":{"1min":1.2709999999999999,"5min":1.456,"15min":2.3340000000000001},"last":1.3200000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89400000000000002,"15min":0.93600000000000005},"min":{"1min":0.57099999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.1859999999999999,"5min":1.714,"15min":1.762},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.91600000000000004,"15min":0.998},"min":{"1min":0.55500000000000005,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":1.175,"5min":1.8260000000000001,"15min":1.879},"last":1.107}]},{"osd":3,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90700000000000003,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.34200000000000003},"max":{"1min":1.268,"5min":1.728,"15min":1.728},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1779999999999999,"15min":1.1850000000000001},"min":{"1min":0.54400000000000004,"5min":0.499,"15min":0.48499999999999999},"max":{"1min":1.544,"5min":1.9419999999999999,"15min":2.7679999999999998},"last":1.252}]},{"osd":4,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.135,"15min":1.1499999999999999},"min":{"1min":0.76300000000000001,"5min":0.67700000000000005,"15min":0.65300000000000002},"max":{"1min":1.524,"5min":1.9079999999999999,"15min":2.2559999999999998},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2729999999999999,"15min":1.29},"min":{"1min":0.96299999999999997,"5min":0.77800000000000002,"15min":0.59399999999999997},"max":{"1min":1.444,"5min":2.0659999999999998,"15min":2.867},"last":1.014}]},{"osd":5,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.244,"15min":1.2529999999999999},"min":{"1min":0.93799999999999994,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5109999999999999,"5min":2.4169999999999998,"15min":2.7280000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.2050000000000001,"15min":1.2310000000000001},"min":{"1min":0.83199999999999996,"5min":0.73299999999999998,"15min":0.67600000000000005},"max":{"1min":1.45,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.153}]},{"osd":6,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3640000000000001,"15min":1.3620000000000001},"min":{"1min":1.0469999999999999,"5min":0.79900000000000004,"15min":0.72499999999999998},"max":{"1min":1.6930000000000001,"5min":2.0489999999999999,"15min":2.6739999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3140000000000001,"15min":1.3580000000000001},"min":{"1min":0.91500000000000004,"5min":0.72999999999999998,"15min":0.64200000000000002},"max":{"1min":1.5589999999999999,"5min":2.2160000000000002,"15min":2.8260000000000001},"last":1.2090000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3979999999999999,"15min":1.4079999999999999},"min":{"1min":1.117,"5min":0.82199999999999995,"15min":0.78000000000000003},"max":{"1min":1.736,"5min":4.5670000000000002,"15min":4.5670000000000002},"last":1.385},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.349,"15min":1.3600000000000001},"min":{"1min":0.93000000000000005,"5min":0.75800000000000001,"15min":0.70199999999999996},"max":{"1min":1.6459999999999999,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":1.071}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:37:37.384 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:37:37.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.632+0000 7f4c681e7640 1 Processor -- start 2023-11-08T16:37:37.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.633+0000 7f4c681e7640 1 -- start start 2023-11-08T16:37:37.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.634+0000 7f4c681e7640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f4c600ca480 0x7f4c6015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.634+0000 7f4c681e7640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6006b630 con 0x7f4c6015ea60 2023-11-08T16:37:37.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.634+0000 7f4c681e7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c600ca910 con 0x7f4c600ade50 2023-11-08T16:37:37.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.634+0000 7f4c681e7640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c600b1860 con 0x7f4c600ca480 2023-11-08T16:37:37.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.635+0000 7f4c66784640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4c6015ea60 0x7f4c6015eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53130/0 (socket says 172.21.15.81:53130) 2023-11-08T16:37:37.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.635+0000 7f4c66784640 1 -- 172.21.15.81:0/625690523 learned_addr learned my addr 172.21.15.81:0/625690523 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:37.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.635+0000 7f4c65782640 1 --2- 172.21.15.81:0/625690523 >> v2:172.21.15.93:3300/0 conn(0x7f4c600ca480 0x7f4c6015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.635+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143628375 0 0) 0x7f4c6006b630 con 0x7f4c6015ea60 2023-11-08T16:37:37.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.635+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c48003520 con 0x7f4c6015ea60 2023-11-08T16:37:37.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2323311053 0 0) 0x7f4c48003520 con 0x7f4c6015ea60 2023-11-08T16:37:37.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 >> v2:172.21.15.93:3300/0 conn(0x7f4c600ca480 msgr2=0x7f4c6015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:37.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 --2- 172.21.15.81:0/625690523 >> v2:172.21.15.93:3300/0 conn(0x7f4c600ca480 0x7f4c6015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 >> v1:172.21.15.93:6789/0 conn(0x7f4c600ade50 legacy=0x7f4c6015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:37.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600cabb0 con 0x7f4c6015ea60 2023-11-08T16:37:37.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f4c580019f0 con 0x7f4c6015ea60 2023-11-08T16:37:37.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 >> v1:172.21.15.81:6789/0 conn(0x7f4c6015ea60 legacy=0x7f4c6015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:37.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 --2- 172.21.15.81:0/625690523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 0x7f4c48003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 --2- 172.21.15.81:0/625690523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c480047c0 0x7f4c48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 --2- 172.21.15.81:0/625690523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c480051d0 0x7f4c4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4800c0d0 con 0x7f4c48003a50 2023-11-08T16:37:37.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4800c240 con 0x7f4c480051d0 2023-11-08T16:37:37.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c4800c400 con 0x7f4c480047c0 2023-11-08T16:37:37.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c65f83640 1 --2- 172.21.15.81:0/625690523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 0x7f4c48003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.636+0000 7f4c66784640 1 --2- 172.21.15.81:0/625690523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c480051d0 0x7f4c4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.637+0000 7f4c65f83640 1 -- 172.21.15.81:0/625690523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c480047c0 msgr2=0x7f4c48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:37.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.637+0000 7f4c65f83640 1 --2- 172.21.15.81:0/625690523 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c480047c0 0x7f4c48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.637+0000 7f4c65f83640 1 -- 172.21.15.81:0/625690523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c480051d0 msgr2=0x7f4c4800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:37.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.637+0000 7f4c65f83640 1 --2- 172.21.15.81:0/625690523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c480051d0 0x7f4c4800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.637+0000 7f4c65f83640 1 -- 172.21.15.81:0/625690523 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c48003520 con 0x7f4c48003a50 2023-11-08T16:37:37.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.637+0000 7f4c66784640 1 --2- 172.21.15.81:0/625690523 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c480051d0 0x7f4c4800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:37.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.637+0000 7f4c66784640 1 -- 172.21.15.81:0/625690523 reap_dead start 2023-11-08T16:37:37.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.638+0000 7f4c65f83640 1 --2- 172.21.15.81:0/625690523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 0x7f4c48003e70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4c50001720 tx=0x7f4c480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755f7969c8bc1748 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:37.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.638+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c500052d0 con 0x7f4c48003a50 2023-11-08T16:37:37.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.638+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5002e050 con 0x7f4c48003a50 2023-11-08T16:37:37.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.638+0000 7f4c64f81640 1 -- 172.21.15.81:0/625690523 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50003830 con 0x7f4c48003a50 2023-11-08T16:37:37.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.638+0000 7f4c681e7640 1 -- 172.21.15.81:0/625690523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 msgr2=0x7f4c48003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:37.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.638+0000 7f4c681e7640 1 --2- 172.21.15.81:0/625690523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 0x7f4c48003e70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4c50001720 tx=0x7f4c480057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.639+0000 7f4c681e7640 1 -- 172.21.15.81:0/625690523 shutdown_connections 2023-11-08T16:37:37.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.639+0000 7f4c681e7640 1 --2- 172.21.15.81:0/625690523 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 0x7f4c48003e70 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.639+0000 7f4c681e7640 1 -- 172.21.15.81:0/625690523 >> 172.21.15.81:0/625690523 conn(0x7f4c600cb490 msgr2=0x7f4c600cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:37.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.639+0000 7f4c681e7640 1 -- 172.21.15.81:0/625690523 shutdown_connections 2023-11-08T16:37:37.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.639+0000 7f4c681e7640 1 -- 172.21.15.81:0/625690523 wait complete. 2023-11-08T16:37:37.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.640+0000 7f4c681e7640 1 Processor -- start 2023-11-08T16:37:37.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.641+0000 7f4c681e7640 1 -- start start 2023-11-08T16:37:37.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.641+0000 7f4c681e7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 0x7f4c6014f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.641+0000 7f4c681e7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c60150770 0x7f4c60150bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.641+0000 7f4c681e7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c60153b40 0x7f4c60153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.641+0000 7f4c681e7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c60151130 con 0x7f4c48003a50 2023-11-08T16:37:37.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.641+0000 7f4c681e7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600b1c20 con 0x7f4c60153b40 2023-11-08T16:37:37.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c681e7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c600b29a0 con 0x7f4c60150770 2023-11-08T16:37:37.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c65f83640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 0x7f4c6014f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c65f83640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 0x7f4c6014f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50412/0 (socket says 172.21.15.81:50412) 2023-11-08T16:37:37.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c65f83640 1 -- 172.21.15.81:0/417178323 learned_addr learned my addr 172.21.15.81:0/417178323 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:37.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c65782640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c60150770 0x7f4c60150bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c66784640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c60153b40 0x7f4c60153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c65f83640 1 -- 172.21.15.81:0/417178323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c60150770 msgr2=0x7f4c60150bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:37.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c65f83640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c60150770 0x7f4c60150bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c65f83640 1 -- 172.21.15.81:0/417178323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c60153b40 msgr2=0x7f4c60153fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:37.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c65f83640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c60153b40 0x7f4c60153fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c65f83640 1 -- 172.21.15.81:0/417178323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c600b1ec0 con 0x7f4c48003a50 2023-11-08T16:37:37.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.642+0000 7f4c66784640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c60153b40 0x7f4c60153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:37.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.643+0000 7f4c65f83640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 0x7f4c6014f720 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4c5002bbc0 tx=0x7f4c50001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:37.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.643+0000 7f4c567fc640 1 -- 172.21.15.81:0/417178323 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50007640 con 0x7f4c48003a50 2023-11-08T16:37:37.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.643+0000 7f4c567fc640 1 -- 172.21.15.81:0/417178323 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c500069e0 con 0x7f4c48003a50 2023-11-08T16:37:37.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.643+0000 7f4c567fc640 1 -- 172.21.15.81:0/417178323 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50006d70 con 0x7f4c48003a50 2023-11-08T16:37:37.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.643+0000 7f4c681e7640 1 -- 172.21.15.81:0/417178323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c600b21a0 con 0x7f4c48003a50 2023-11-08T16:37:37.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.643+0000 7f4c681e7640 1 -- 172.21.15.81:0/417178323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c60154500 con 0x7f4c48003a50 2023-11-08T16:37:37.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.644+0000 7f4c681e7640 1 -- 172.21.15.81:0/417178323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c600ca070 con 0x7f4c48003a50 2023-11-08T16:37:37.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.645+0000 7f4c567fc640 1 -- 172.21.15.81:0/417178323 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c500035f0 con 0x7f4c48003a50 2023-11-08T16:37:37.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.647+0000 7f4c567fc640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4c4c06aea0 0x7f4c4c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:37.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.647+0000 7f4c567fc640 1 -- 172.21.15.81:0/417178323 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c500ab390 con 0x7f4c48003a50 2023-11-08T16:37:37.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.647+0000 7f4c65782640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4c4c06aea0 0x7f4c4c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:37.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.648+0000 7f4c65782640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4c4c06aea0 0x7f4c4c06d360 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4c60151d10 tx=0x7f4c5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:37.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.651+0000 7f4c567fc640 1 -- 172.21.15.81:0/417178323 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c50076560 con 0x7f4c48003a50 2023-11-08T16:37:37.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.839+0000 7f4c681e7640 1 -- 172.21.15.81:0/417178323 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4c6006bb10 con 0x7f4c4c06aea0 2023-11-08T16:37:37.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.848+0000 7f4c567fc640 1 -- 172.21.15.81:0/417178323 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287088 (secure 0 0 0) 0x7f4c6006bb10 con 0x7f4c4c06aea0 2023-11-08T16:37:37.853 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:37.854 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:37:37.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.853+0000 7f4c681e7640 1 -- 172.21.15.81:0/417178323 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4c4c06aea0 msgr2=0x7f4c4c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:37.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.853+0000 7f4c681e7640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4c4c06aea0 0x7f4c4c06d360 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f4c60151d10 tx=0x7f4c5c008040 comp rx=0 tx=0).stop 2023-11-08T16:37:37.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.853+0000 7f4c681e7640 1 -- 172.21.15.81:0/417178323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 msgr2=0x7f4c6014f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:37.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.853+0000 7f4c681e7640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 0x7f4c6014f720 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4c5002bbc0 tx=0x7f4c50001120 comp rx=0 tx=0).stop 2023-11-08T16:37:37.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.853+0000 7f4c681e7640 1 -- 172.21.15.81:0/417178323 shutdown_connections 2023-11-08T16:37:37.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.853+0000 7f4c681e7640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4c4c06aea0 0x7f4c4c06d360 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.853+0000 7f4c681e7640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4c60153b40 0x7f4c60153fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.853+0000 7f4c681e7640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c60150770 0x7f4c60150bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.853+0000 7f4c681e7640 1 --2- 172.21.15.81:0/417178323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c48003a50 0x7f4c6014f720 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:37.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.853+0000 7f4c681e7640 1 -- 172.21.15.81:0/417178323 >> 172.21.15.81:0/417178323 conn(0x7f4c600cb490 msgr2=0x7f4c60071180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:37.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.854+0000 7f4c681e7640 1 -- 172.21.15.81:0/417178323 shutdown_connections 2023-11-08T16:37:37.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:37.854+0000 7f4c681e7640 1 -- 172.21.15.81:0/417178323 wait complete. 2023-11-08T16:37:37.874 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2023-11-08T16:37:36.730150+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258416,"kb_used_data":8936,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716304,"statfs":{"total":773094113280,"available":772829495296,"internally_reserved":0,"allocated":9150464,"data_stored":4741676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.892224"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867992+0000","last_change":"2023-11-08T16:37:32.899110+0000","last_active":"2023-11-08T16:37:33.867992+0000","last_peered":"2023-11-08T16:37:33.867992+0000","last_clean":"2023-11-08T16:37:33.867992+0000","last_became_active":"2023-11-08T16:37:32.898593+0000","last_became_peered":"2023-11-08T16:37:32.898593+0000","last_unstale":"2023-11-08T16:37:33.867992+0000","last_undegraded":"2023-11-08T16:37:33.867992+0000","last_fullsized":"2023-11-08T16:37:33.867992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:57.888620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.700768+0000","last_change":"2023-11-08T16:37:32.890662+0000","last_active":"2023-11-08T16:37:34.700768+0000","last_peered":"2023-11-08T16:37:34.700768+0000","last_clean":"2023-11-08T16:37:34.700768+0000","last_became_active":"2023-11-08T16:37:32.890336+0000","last_became_peered":"2023-11-08T16:37:32.890336+0000","last_unstale":"2023-11-08T16:37:34.700768+0000","last_undegraded":"2023-11-08T16:37:34.700768+0000","last_fullsized":"2023-11-08T16:37:34.700768+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:53:06.026452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107549+0000","last_change":"2023-11-08T16:37:32.893246+0000","last_active":"2023-11-08T16:37:34.107549+0000","last_peered":"2023-11-08T16:37:34.107549+0000","last_clean":"2023-11-08T16:37:34.107549+0000","last_became_active":"2023-11-08T16:37:32.892768+0000","last_became_peered":"2023-11-08T16:37:32.892768+0000","last_unstale":"2023-11-08T16:37:34.107549+0000","last_undegraded":"2023-11-08T16:37:34.107549+0000","last_fullsized":"2023-11-08T16:37:34.107549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:49:36.782384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.865601+0000","last_change":"2023-11-08T16:37:32.894249+0000","last_active":"2023-11-08T16:37:33.865601+0000","last_peered":"2023-11-08T16:37:33.865601+0000","last_clean":"2023-11-08T16:37:33.865601+0000","last_became_active":"2023-11-08T16:37:32.893846+0000","last_became_peered":"2023-11-08T16:37:32.893846+0000","last_unstale":"2023-11-08T16:37:33.865601+0000","last_undegraded":"2023-11-08T16:37:33.865601+0000","last_fullsized":"2023-11-08T16:37:33.865601+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:22.625004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696038+0000","last_change":"2023-11-08T16:37:32.897416+0000","last_active":"2023-11-08T16:37:34.696038+0000","last_peered":"2023-11-08T16:37:34.696038+0000","last_clean":"2023-11-08T16:37:34.696038+0000","last_became_active":"2023-11-08T16:37:32.897061+0000","last_became_peered":"2023-11-08T16:37:32.897061+0000","last_unstale":"2023-11-08T16:37:34.696038+0000","last_undegraded":"2023-11-08T16:37:34.696038+0000","last_fullsized":"2023-11-08T16:37:34.696038+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:06:31.308088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698272+0000","last_change":"2023-11-08T16:37:32.896273+0000","last_active":"2023-11-08T16:37:34.698272+0000","last_peered":"2023-11-08T16:37:34.698272+0000","last_clean":"2023-11-08T16:37:34.698272+0000","last_became_active":"2023-11-08T16:37:32.895742+0000","last_became_peered":"2023-11-08T16:37:32.895742+0000","last_unstale":"2023-11-08T16:37:34.698272+0000","last_undegraded":"2023-11-08T16:37:34.698272+0000","last_fullsized":"2023-11-08T16:37:34.698272+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:45.753772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696676+0000","last_change":"2023-11-08T16:37:32.890779+0000","last_active":"2023-11-08T16:37:34.696676+0000","last_peered":"2023-11-08T16:37:34.696676+0000","last_clean":"2023-11-08T16:37:34.696676+0000","last_became_active":"2023-11-08T16:37:32.890492+0000","last_became_peered":"2023-11-08T16:37:32.890492+0000","last_unstale":"2023-11-08T16:37:34.696676+0000","last_undegraded":"2023-11-08T16:37:34.696676+0000","last_fullsized":"2023-11-08T16:37:34.696676+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:22.663417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698035+0000","last_change":"2023-11-08T16:37:32.892385+0000","last_active":"2023-11-08T16:37:34.698035+0000","last_peered":"2023-11-08T16:37:34.698035+0000","last_clean":"2023-11-08T16:37:34.698035+0000","last_became_active":"2023-11-08T16:37:32.892032+0000","last_became_peered":"2023-11-08T16:37:32.892032+0000","last_unstale":"2023-11-08T16:37:34.698035+0000","last_undegraded":"2023-11-08T16:37:34.698035+0000","last_fullsized":"2023-11-08T16:37:34.698035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:38:25.173471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696802+0000","last_change":"2023-11-08T16:37:32.900485+0000","last_active":"2023-11-08T16:37:34.696802+0000","last_peered":"2023-11-08T16:37:34.696802+0000","last_clean":"2023-11-08T16:37:34.696802+0000","last_became_active":"2023-11-08T16:37:32.899649+0000","last_became_peered":"2023-11-08T16:37:32.899649+0000","last_unstale":"2023-11-08T16:37:34.696802+0000","last_undegraded":"2023-11-08T16:37:34.696802+0000","last_fullsized":"2023-11-08T16:37:34.696802+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:44:20.816013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696087+0000","last_change":"2023-11-08T16:37:32.896323+0000","last_active":"2023-11-08T16:37:34.696087+0000","last_peered":"2023-11-08T16:37:34.696087+0000","last_clean":"2023-11-08T16:37:34.696087+0000","last_became_active":"2023-11-08T16:37:32.895458+0000","last_became_peered":"2023-11-08T16:37:32.895458+0000","last_unstale":"2023-11-08T16:37:34.696087+0000","last_undegraded":"2023-11-08T16:37:34.696087+0000","last_fullsized":"2023-11-08T16:37:34.696087+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:26.531450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.868746+0000","last_change":"2023-11-08T16:37:32.889586+0000","last_active":"2023-11-08T16:37:33.868746+0000","last_peered":"2023-11-08T16:37:33.868746+0000","last_clean":"2023-11-08T16:37:33.868746+0000","last_became_active":"2023-11-08T16:37:32.889250+0000","last_became_peered":"2023-11-08T16:37:32.889250+0000","last_unstale":"2023-11-08T16:37:33.868746+0000","last_undegraded":"2023-11-08T16:37:33.868746+0000","last_fullsized":"2023-11-08T16:37:33.868746+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:57.489427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.865703+0000","last_change":"2023-11-08T16:37:32.891507+0000","last_active":"2023-11-08T16:37:33.865703+0000","last_peered":"2023-11-08T16:37:33.865703+0000","last_clean":"2023-11-08T16:37:33.865703+0000","last_became_active":"2023-11-08T16:37:32.891181+0000","last_became_peered":"2023-11-08T16:37:32.891181+0000","last_unstale":"2023-11-08T16:37:33.865703+0000","last_undegraded":"2023-11-08T16:37:33.865703+0000","last_fullsized":"2023-11-08T16:37:33.865703+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:21.188125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864665+0000","last_change":"2023-11-08T16:37:32.903434+0000","last_active":"2023-11-08T16:37:33.864665+0000","last_peered":"2023-11-08T16:37:33.864665+0000","last_clean":"2023-11-08T16:37:33.864665+0000","last_became_active":"2023-11-08T16:37:32.902647+0000","last_became_peered":"2023-11-08T16:37:32.902647+0000","last_unstale":"2023-11-08T16:37:33.864665+0000","last_undegraded":"2023-11-08T16:37:33.864665+0000","last_fullsized":"2023-11-08T16:37:33.864665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:59:37.827654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696048+0000","last_change":"2023-11-08T16:37:32.897191+0000","last_active":"2023-11-08T16:37:34.696048+0000","last_peered":"2023-11-08T16:37:34.696048+0000","last_clean":"2023-11-08T16:37:34.696048+0000","last_became_active":"2023-11-08T16:37:32.896908+0000","last_became_peered":"2023-11-08T16:37:32.896908+0000","last_unstale":"2023-11-08T16:37:34.696048+0000","last_undegraded":"2023-11-08T16:37:34.696048+0000","last_fullsized":"2023-11-08T16:37:34.696048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:04:38.571484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864488+0000","last_change":"2023-11-08T16:37:32.899400+0000","last_active":"2023-11-08T16:37:33.864488+0000","last_peered":"2023-11-08T16:37:33.864488+0000","last_clean":"2023-11-08T16:37:33.864488+0000","last_became_active":"2023-11-08T16:37:32.898834+0000","last_became_peered":"2023-11-08T16:37:32.898834+0000","last_unstale":"2023-11-08T16:37:33.864488+0000","last_undegraded":"2023-11-08T16:37:33.864488+0000","last_fullsized":"2023-11-08T16:37:33.864488+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:00:16.203256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.695518+0000","last_change":"2023-11-08T16:37:32.898841+0000","last_active":"2023-11-08T16:37:34.695518+0000","last_peered":"2023-11-08T16:37:34.695518+0000","last_clean":"2023-11-08T16:37:34.695518+0000","last_became_active":"2023-11-08T16:37:32.898325+0000","last_became_peered":"2023-11-08T16:37:32.898325+0000","last_unstale":"2023-11-08T16:37:34.695518+0000","last_undegraded":"2023-11-08T16:37:34.695518+0000","last_fullsized":"2023-11-08T16:37:34.695518+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:18.438471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696950+0000","last_change":"2023-11-08T16:37:32.899955+0000","last_active":"2023-11-08T16:37:34.696950+0000","last_peered":"2023-11-08T16:37:34.696950+0000","last_clean":"2023-11-08T16:37:34.696950+0000","last_became_active":"2023-11-08T16:37:32.899272+0000","last_became_peered":"2023-11-08T16:37:32.899272+0000","last_unstale":"2023-11-08T16:37:34.696950+0000","last_undegraded":"2023-11-08T16:37:34.696950+0000","last_fullsized":"2023-11-08T16:37:34.696950+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:11:20.677569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.865845+0000","last_change":"2023-11-08T16:37:32.903851+0000","last_active":"2023-11-08T16:37:33.865845+0000","last_peered":"2023-11-08T16:37:33.865845+0000","last_clean":"2023-11-08T16:37:33.865845+0000","last_became_active":"2023-11-08T16:37:32.903566+0000","last_became_peered":"2023-11-08T16:37:32.903566+0000","last_unstale":"2023-11-08T16:37:33.865845+0000","last_undegraded":"2023-11-08T16:37:33.865845+0000","last_fullsized":"2023-11-08T16:37:33.865845+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:18:19.754013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699850+0000","last_change":"2023-11-08T16:37:32.901267+0000","last_active":"2023-11-08T16:37:34.699850+0000","last_peered":"2023-11-08T16:37:34.699850+0000","last_clean":"2023-11-08T16:37:34.699850+0000","last_became_active":"2023-11-08T16:37:32.900971+0000","last_became_peered":"2023-11-08T16:37:32.900971+0000","last_unstale":"2023-11-08T16:37:34.699850+0000","last_undegraded":"2023-11-08T16:37:34.699850+0000","last_fullsized":"2023-11-08T16:37:34.699850+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:27:30.791214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864665+0000","last_change":"2023-11-08T16:37:32.895358+0000","last_active":"2023-11-08T16:37:33.864665+0000","last_peered":"2023-11-08T16:37:33.864665+0000","last_clean":"2023-11-08T16:37:33.864665+0000","last_became_active":"2023-11-08T16:37:32.894964+0000","last_became_peered":"2023-11-08T16:37:32.894964+0000","last_unstale":"2023-11-08T16:37:33.864665+0000","last_undegraded":"2023-11-08T16:37:33.864665+0000","last_fullsized":"2023-11-08T16:37:33.864665+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:11.406716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696104+0000","last_change":"2023-11-08T16:37:32.897279+0000","last_active":"2023-11-08T16:37:34.696104+0000","last_peered":"2023-11-08T16:37:34.696104+0000","last_clean":"2023-11-08T16:37:34.696104+0000","last_became_active":"2023-11-08T16:37:32.897006+0000","last_became_peered":"2023-11-08T16:37:32.897006+0000","last_unstale":"2023-11-08T16:37:34.696104+0000","last_undegraded":"2023-11-08T16:37:34.696104+0000","last_fullsized":"2023-11-08T16:37:34.696104+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:38:25.173471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.695996+0000","last_change":"2023-11-08T16:37:32.899462+0000","last_active":"2023-11-08T16:37:34.695996+0000","last_peered":"2023-11-08T16:37:34.695996+0000","last_clean":"2023-11-08T16:37:34.695996+0000","last_became_active":"2023-11-08T16:37:32.898474+0000","last_became_peered":"2023-11-08T16:37:32.898474+0000","last_unstale":"2023-11-08T16:37:34.695996+0000","last_undegraded":"2023-11-08T16:37:34.695996+0000","last_fullsized":"2023-11-08T16:37:34.695996+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:32:09.784202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867719+0000","last_change":"2023-11-08T16:37:32.886798+0000","last_active":"2023-11-08T16:37:33.867719+0000","last_peered":"2023-11-08T16:37:33.867719+0000","last_clean":"2023-11-08T16:37:33.867719+0000","last_became_active":"2023-11-08T16:37:32.884586+0000","last_became_peered":"2023-11-08T16:37:32.884586+0000","last_unstale":"2023-11-08T16:37:33.867719+0000","last_undegraded":"2023-11-08T16:37:33.867719+0000","last_fullsized":"2023-11-08T16:37:33.867719+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:25:39.440480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867091+0000","last_change":"2023-11-08T16:37:32.898640+0000","last_active":"2023-11-08T16:37:33.867091+0000","last_peered":"2023-11-08T16:37:33.867091+0000","last_clean":"2023-11-08T16:37:33.867091+0000","last_became_active":"2023-11-08T16:37:32.898325+0000","last_became_peered":"2023-11-08T16:37:32.898325+0000","last_unstale":"2023-11-08T16:37:33.867091+0000","last_undegraded":"2023-11-08T16:37:33.867091+0000","last_fullsized":"2023-11-08T16:37:33.867091+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:53:17.138775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698169+0000","last_change":"2023-11-08T16:37:32.893036+0000","last_active":"2023-11-08T16:37:34.698169+0000","last_peered":"2023-11-08T16:37:34.698169+0000","last_clean":"2023-11-08T16:37:34.698169+0000","last_became_active":"2023-11-08T16:37:32.892749+0000","last_became_peered":"2023-11-08T16:37:32.892749+0000","last_unstale":"2023-11-08T16:37:34.698169+0000","last_undegraded":"2023-11-08T16:37:34.698169+0000","last_fullsized":"2023-11-08T16:37:34.698169+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:17.046902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699708+0000","last_change":"2023-11-08T16:37:32.903770+0000","last_active":"2023-11-08T16:37:34.699708+0000","last_peered":"2023-11-08T16:37:34.699708+0000","last_clean":"2023-11-08T16:37:34.699708+0000","last_became_active":"2023-11-08T16:37:32.903492+0000","last_became_peered":"2023-11-08T16:37:32.903492+0000","last_unstale":"2023-11-08T16:37:34.699708+0000","last_undegraded":"2023-11-08T16:37:34.699708+0000","last_fullsized":"2023-11-08T16:37:34.699708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:39:48.141466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107775+0000","last_change":"2023-11-08T16:37:32.880801+0000","last_active":"2023-11-08T16:37:34.107775+0000","last_peered":"2023-11-08T16:37:34.107775+0000","last_clean":"2023-11-08T16:37:34.107775+0000","last_became_active":"2023-11-08T16:37:32.880372+0000","last_became_peered":"2023-11-08T16:37:32.880372+0000","last_unstale":"2023-11-08T16:37:34.107775+0000","last_undegraded":"2023-11-08T16:37:34.107775+0000","last_fullsized":"2023-11-08T16:37:34.107775+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:23:26.392182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698292+0000","last_change":"2023-11-08T16:37:32.897875+0000","last_active":"2023-11-08T16:37:34.698292+0000","last_peered":"2023-11-08T16:37:34.698292+0000","last_clean":"2023-11-08T16:37:34.698292+0000","last_became_active":"2023-11-08T16:37:32.897410+0000","last_became_peered":"2023-11-08T16:37:32.897410+0000","last_unstale":"2023-11-08T16:37:34.698292+0000","last_undegraded":"2023-11-08T16:37:34.698292+0000","last_fullsized":"2023-11-08T16:37:34.698292+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:23.235607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107690+0000","last_change":"2023-11-08T16:37:32.883090+0000","last_active":"2023-11-08T16:37:34.107690+0000","last_peered":"2023-11-08T16:37:34.107690+0000","last_clean":"2023-11-08T16:37:34.107690+0000","last_became_active":"2023-11-08T16:37:32.882462+0000","last_became_peered":"2023-11-08T16:37:32.882462+0000","last_unstale":"2023-11-08T16:37:34.107690+0000","last_undegraded":"2023-11-08T16:37:34.107690+0000","last_fullsized":"2023-11-08T16:37:34.107690+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:36:43.046067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697452+0000","last_change":"2023-11-08T16:37:32.902198+0000","last_active":"2023-11-08T16:37:34.697452+0000","last_peered":"2023-11-08T16:37:34.697452+0000","last_clean":"2023-11-08T16:37:34.697452+0000","last_became_active":"2023-11-08T16:37:32.901808+0000","last_became_peered":"2023-11-08T16:37:32.901808+0000","last_unstale":"2023-11-08T16:37:34.697452+0000","last_undegraded":"2023-11-08T16:37:34.697452+0000","last_fullsized":"2023-11-08T16:37:34.697452+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:30:52.952266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697906+0000","last_change":"2023-11-08T16:37:32.900759+0000","last_active":"2023-11-08T16:37:34.697906+0000","last_peered":"2023-11-08T16:37:34.697906+0000","last_clean":"2023-11-08T16:37:34.697906+0000","last_became_active":"2023-11-08T16:37:32.899852+0000","last_became_peered":"2023-11-08T16:37:32.899852+0000","last_unstale":"2023-11-08T16:37:34.697906+0000","last_undegraded":"2023-11-08T16:37:34.697906+0000","last_fullsized":"2023-11-08T16:37:34.697906+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:14:19.127984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698034+0000","last_change":"2023-11-08T16:37:32.892073+0000","last_active":"2023-11-08T16:37:34.698034+0000","last_peered":"2023-11-08T16:37:34.698034+0000","last_clean":"2023-11-08T16:37:34.698034+0000","last_became_active":"2023-11-08T16:37:32.891260+0000","last_became_peered":"2023-11-08T16:37:32.891260+0000","last_unstale":"2023-11-08T16:37:34.698034+0000","last_undegraded":"2023-11-08T16:37:34.698034+0000","last_fullsized":"2023-11-08T16:37:34.698034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:26.531450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867696+0000","last_change":"2023-11-08T16:37:32.902724+0000","last_active":"2023-11-08T16:37:33.867696+0000","last_peered":"2023-11-08T16:37:33.867696+0000","last_clean":"2023-11-08T16:37:33.867696+0000","last_became_active":"2023-11-08T16:37:32.901742+0000","last_became_peered":"2023-11-08T16:37:32.901742+0000","last_unstale":"2023-11-08T16:37:33.867696+0000","last_undegraded":"2023-11-08T16:37:33.867696+0000","last_fullsized":"2023-11-08T16:37:33.867696+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:40:11.697591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"80'4","reported_seq":16,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.977843+0000","last_change":"2023-11-08T16:37:31.888025+0000","last_active":"2023-11-08T16:37:32.977843+0000","last_peered":"2023-11-08T16:37:32.977843+0000","last_clean":"2023-11-08T16:37:32.977843+0000","last_became_active":"2023-11-08T16:37:31.887704+0000","last_became_peered":"2023-11-08T16:37:31.887704+0000","last_unstale":"2023-11-08T16:37:32.977843+0000","last_undegraded":"2023-11-08T16:37:32.977843+0000","last_fullsized":"2023-11-08T16:37:32.977843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:44:48.286992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697065+0000","last_change":"2023-11-08T16:37:32.900803+0000","last_active":"2023-11-08T16:37:34.697065+0000","last_peered":"2023-11-08T16:37:34.697065+0000","last_clean":"2023-11-08T16:37:34.697065+0000","last_became_active":"2023-11-08T16:37:32.900477+0000","last_became_peered":"2023-11-08T16:37:32.900477+0000","last_unstale":"2023-11-08T16:37:34.697065+0000","last_undegraded":"2023-11-08T16:37:34.697065+0000","last_fullsized":"2023-11-08T16:37:34.697065+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:20:48.435016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.976140+0000","last_change":"2023-11-08T16:37:31.886511+0000","last_active":"2023-11-08T16:37:32.976140+0000","last_peered":"2023-11-08T16:37:32.976140+0000","last_clean":"2023-11-08T16:37:32.976140+0000","last_became_active":"2023-11-08T16:37:31.886212+0000","last_became_peered":"2023-11-08T16:37:31.886212+0000","last_unstale":"2023-11-08T16:37:32.976140+0000","last_undegraded":"2023-11-08T16:37:32.976140+0000","last_fullsized":"2023-11-08T16:37:32.976140+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:53:16.121610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699699+0000","last_change":"2023-11-08T16:37:32.896336+0000","last_active":"2023-11-08T16:37:34.699699+0000","last_peered":"2023-11-08T16:37:34.699699+0000","last_clean":"2023-11-08T16:37:34.699699+0000","last_became_active":"2023-11-08T16:37:32.896059+0000","last_became_peered":"2023-11-08T16:37:32.896059+0000","last_unstale":"2023-11-08T16:37:34.699699+0000","last_undegraded":"2023-11-08T16:37:34.699699+0000","last_fullsized":"2023-11-08T16:37:34.699699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:46:17.046902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.977489+0000","last_change":"2023-11-08T16:37:31.891095+0000","last_active":"2023-11-08T16:37:32.977489+0000","last_peered":"2023-11-08T16:37:32.977489+0000","last_clean":"2023-11-08T16:37:32.977489+0000","last_became_active":"2023-11-08T16:37:31.890816+0000","last_became_peered":"2023-11-08T16:37:31.890816+0000","last_unstale":"2023-11-08T16:37:32.977489+0000","last_undegraded":"2023-11-08T16:37:32.977489+0000","last_fullsized":"2023-11-08T16:37:32.977489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:22:17.010347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.868657+0000","last_change":"2023-11-08T16:37:32.896626+0000","last_active":"2023-11-08T16:37:33.868657+0000","last_peered":"2023-11-08T16:37:33.868657+0000","last_clean":"2023-11-08T16:37:33.868657+0000","last_became_active":"2023-11-08T16:37:32.896343+0000","last_became_peered":"2023-11-08T16:37:32.896343+0000","last_unstale":"2023-11-08T16:37:33.868657+0000","last_undegraded":"2023-11-08T16:37:33.868657+0000","last_fullsized":"2023-11-08T16:37:33.868657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:44.490627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.977031+0000","last_change":"2023-11-08T16:37:31.890832+0000","last_active":"2023-11-08T16:37:32.977031+0000","last_peered":"2023-11-08T16:37:32.977031+0000","last_clean":"2023-11-08T16:37:32.977031+0000","last_became_active":"2023-11-08T16:37:31.889708+0000","last_became_peered":"2023-11-08T16:37:31.889708+0000","last_unstale":"2023-11-08T16:37:32.977031+0000","last_undegraded":"2023-11-08T16:37:32.977031+0000","last_fullsized":"2023-11-08T16:37:32.977031+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:14:03.002995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698199+0000","last_change":"2023-11-08T16:37:32.889872+0000","last_active":"2023-11-08T16:37:34.698199+0000","last_peered":"2023-11-08T16:37:34.698199+0000","last_clean":"2023-11-08T16:37:34.698199+0000","last_became_active":"2023-11-08T16:37:32.889571+0000","last_became_peered":"2023-11-08T16:37:32.889571+0000","last_unstale":"2023-11-08T16:37:34.698199+0000","last_undegraded":"2023-11-08T16:37:34.698199+0000","last_fullsized":"2023-11-08T16:37:34.698199+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:08:17.120461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.975794+0000","last_change":"2023-11-08T16:37:31.883142+0000","last_active":"2023-11-08T16:37:32.975794+0000","last_peered":"2023-11-08T16:37:32.975794+0000","last_clean":"2023-11-08T16:37:32.975794+0000","last_became_active":"2023-11-08T16:37:31.882790+0000","last_became_peered":"2023-11-08T16:37:31.882790+0000","last_unstale":"2023-11-08T16:37:32.975794+0000","last_undegraded":"2023-11-08T16:37:32.975794+0000","last_fullsized":"2023-11-08T16:37:32.975794+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:59:36.810489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697453+0000","last_change":"2023-11-08T16:37:32.887937+0000","last_active":"2023-11-08T16:37:34.697453+0000","last_peered":"2023-11-08T16:37:34.697453+0000","last_clean":"2023-11-08T16:37:34.697453+0000","last_became_active":"2023-11-08T16:37:32.887661+0000","last_became_peered":"2023-11-08T16:37:32.887661+0000","last_unstale":"2023-11-08T16:37:34.697453+0000","last_undegraded":"2023-11-08T16:37:34.697453+0000","last_fullsized":"2023-11-08T16:37:34.697453+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:45.753772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.974915+0000","last_change":"2023-11-08T16:37:31.890699+0000","last_active":"2023-11-08T16:37:32.974915+0000","last_peered":"2023-11-08T16:37:32.974915+0000","last_clean":"2023-11-08T16:37:32.974915+0000","last_became_active":"2023-11-08T16:37:31.890155+0000","last_became_peered":"2023-11-08T16:37:31.890155+0000","last_unstale":"2023-11-08T16:37:32.974915+0000","last_undegraded":"2023-11-08T16:37:32.974915+0000","last_fullsized":"2023-11-08T16:37:32.974915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:16:05.663691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696235+0000","last_change":"2023-11-08T16:37:32.897426+0000","last_active":"2023-11-08T16:37:34.696235+0000","last_peered":"2023-11-08T16:37:34.696235+0000","last_clean":"2023-11-08T16:37:34.696235+0000","last_became_active":"2023-11-08T16:37:32.896041+0000","last_became_peered":"2023-11-08T16:37:32.896041+0000","last_unstale":"2023-11-08T16:37:34.696235+0000","last_undegraded":"2023-11-08T16:37:34.696235+0000","last_fullsized":"2023-11-08T16:37:34.696235+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:42:07.806014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.877203+0000","last_change":"2023-11-08T16:37:31.878294+0000","last_active":"2023-11-08T16:37:32.877203+0000","last_peered":"2023-11-08T16:37:32.877203+0000","last_clean":"2023-11-08T16:37:32.877203+0000","last_became_active":"2023-11-08T16:37:31.877510+0000","last_became_peered":"2023-11-08T16:37:31.877510+0000","last_unstale":"2023-11-08T16:37:32.877203+0000","last_undegraded":"2023-11-08T16:37:32.877203+0000","last_fullsized":"2023-11-08T16:37:32.877203+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:22.218442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864247+0000","last_change":"2023-11-08T16:37:32.895439+0000","last_active":"2023-11-08T16:37:33.864247+0000","last_peered":"2023-11-08T16:37:33.864247+0000","last_clean":"2023-11-08T16:37:33.864247+0000","last_became_active":"2023-11-08T16:37:32.895019+0000","last_became_peered":"2023-11-08T16:37:32.895019+0000","last_unstale":"2023-11-08T16:37:33.864247+0000","last_undegraded":"2023-11-08T16:37:33.864247+0000","last_fullsized":"2023-11-08T16:37:33.864247+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:44.490627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.888163+0000","last_change":"2023-11-08T16:37:31.880117+0000","last_active":"2023-11-08T16:37:32.888163+0000","last_peered":"2023-11-08T16:37:32.888163+0000","last_clean":"2023-11-08T16:37:32.888163+0000","last_became_active":"2023-11-08T16:37:31.879792+0000","last_became_peered":"2023-11-08T16:37:31.879792+0000","last_unstale":"2023-11-08T16:37:32.888163+0000","last_undegraded":"2023-11-08T16:37:32.888163+0000","last_fullsized":"2023-11-08T16:37:32.888163+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:24.123715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699150+0000","last_change":"2023-11-08T16:37:32.878674+0000","last_active":"2023-11-08T16:37:34.699150+0000","last_peered":"2023-11-08T16:37:34.699150+0000","last_clean":"2023-11-08T16:37:34.699150+0000","last_became_active":"2023-11-08T16:37:32.878144+0000","last_became_peered":"2023-11-08T16:37:32.878144+0000","last_unstale":"2023-11-08T16:37:34.699150+0000","last_undegraded":"2023-11-08T16:37:34.699150+0000","last_fullsized":"2023-11-08T16:37:34.699150+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:44:49.304157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.884773+0000","last_change":"2023-11-08T16:37:31.881702+0000","last_active":"2023-11-08T16:37:32.884773+0000","last_peered":"2023-11-08T16:37:32.884773+0000","last_clean":"2023-11-08T16:37:32.884773+0000","last_became_active":"2023-11-08T16:37:31.881409+0000","last_became_peered":"2023-11-08T16:37:31.881409+0000","last_unstale":"2023-11-08T16:37:32.884773+0000","last_undegraded":"2023-11-08T16:37:32.884773+0000","last_fullsized":"2023-11-08T16:37:32.884773+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:16:26.042660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.868355+0000","last_change":"2023-11-08T16:37:32.894401+0000","last_active":"2023-11-08T16:37:33.868355+0000","last_peered":"2023-11-08T16:37:33.868355+0000","last_clean":"2023-11-08T16:37:33.868355+0000","last_became_active":"2023-11-08T16:37:32.894032+0000","last_became_peered":"2023-11-08T16:37:32.894032+0000","last_unstale":"2023-11-08T16:37:33.868355+0000","last_undegraded":"2023-11-08T16:37:33.868355+0000","last_fullsized":"2023-11-08T16:37:33.868355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:03:19.541668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.891743+0000","last_change":"2023-11-08T16:37:31.891676+0000","last_active":"2023-11-08T16:37:32.891743+0000","last_peered":"2023-11-08T16:37:32.891743+0000","last_clean":"2023-11-08T16:37:32.891743+0000","last_became_active":"2023-11-08T16:37:31.890845+0000","last_became_peered":"2023-11-08T16:37:31.890845+0000","last_unstale":"2023-11-08T16:37:32.891743+0000","last_undegraded":"2023-11-08T16:37:32.891743+0000","last_fullsized":"2023-11-08T16:37:32.891743+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:09:53.121004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.700025+0000","last_change":"2023-11-08T16:37:32.897055+0000","last_active":"2023-11-08T16:37:34.700025+0000","last_peered":"2023-11-08T16:37:34.700025+0000","last_clean":"2023-11-08T16:37:34.700025+0000","last_became_active":"2023-11-08T16:37:32.896749+0000","last_became_peered":"2023-11-08T16:37:32.896749+0000","last_unstale":"2023-11-08T16:37:34.700025+0000","last_undegraded":"2023-11-08T16:37:34.700025+0000","last_fullsized":"2023-11-08T16:37:34.700025+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:23.235607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"80'3","reported_seq":15,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.976568+0000","last_change":"2023-11-08T16:37:31.886957+0000","last_active":"2023-11-08T16:37:32.976568+0000","last_peered":"2023-11-08T16:37:32.976568+0000","last_clean":"2023-11-08T16:37:32.976568+0000","last_became_active":"2023-11-08T16:37:31.886649+0000","last_became_peered":"2023-11-08T16:37:31.886649+0000","last_unstale":"2023-11-08T16:37:32.976568+0000","last_undegraded":"2023-11-08T16:37:32.976568+0000","last_fullsized":"2023-11-08T16:37:32.976568+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:58:43.473462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696871+0000","last_change":"2023-11-08T16:37:32.896696+0000","last_active":"2023-11-08T16:37:34.696871+0000","last_peered":"2023-11-08T16:37:34.696871+0000","last_clean":"2023-11-08T16:37:34.696871+0000","last_became_active":"2023-11-08T16:37:32.896424+0000","last_became_peered":"2023-11-08T16:37:32.896424+0000","last_unstale":"2023-11-08T16:37:34.696871+0000","last_undegraded":"2023-11-08T16:37:34.696871+0000","last_fullsized":"2023-11-08T16:37:34.696871+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:37:07.862874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"80'6","reported_seq":18,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.978939+0000","last_change":"2023-11-08T16:37:31.878213+0000","last_active":"2023-11-08T16:37:32.978939+0000","last_peered":"2023-11-08T16:37:32.978939+0000","last_clean":"2023-11-08T16:37:32.978939+0000","last_became_active":"2023-11-08T16:37:31.877533+0000","last_became_peered":"2023-11-08T16:37:31.877533+0000","last_unstale":"2023-11-08T16:37:32.978939+0000","last_undegraded":"2023-11-08T16:37:32.978939+0000","last_fullsized":"2023-11-08T16:37:32.978939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:53:05.009286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867464+0000","last_change":"2023-11-08T16:37:32.891876+0000","last_active":"2023-11-08T16:37:33.867464+0000","last_peered":"2023-11-08T16:37:33.867464+0000","last_clean":"2023-11-08T16:37:33.867464+0000","last_became_active":"2023-11-08T16:37:32.890633+0000","last_became_peered":"2023-11-08T16:37:32.890633+0000","last_unstale":"2023-11-08T16:37:33.867464+0000","last_undegraded":"2023-11-08T16:37:33.867464+0000","last_fullsized":"2023-11-08T16:37:33.867464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:57.283084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.978016+0000","last_change":"2023-11-08T16:37:31.881877+0000","last_active":"2023-11-08T16:37:32.978016+0000","last_peered":"2023-11-08T16:37:32.978016+0000","last_clean":"2023-11-08T16:37:32.978016+0000","last_became_active":"2023-11-08T16:37:31.881552+0000","last_became_peered":"2023-11-08T16:37:31.881552+0000","last_unstale":"2023-11-08T16:37:32.978016+0000","last_undegraded":"2023-11-08T16:37:32.978016+0000","last_fullsized":"2023-11-08T16:37:32.978016+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:27:05.800459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698965+0000","last_change":"2023-11-08T16:37:32.888800+0000","last_active":"2023-11-08T16:37:34.698965+0000","last_peered":"2023-11-08T16:37:34.698965+0000","last_clean":"2023-11-08T16:37:34.698965+0000","last_became_active":"2023-11-08T16:37:32.888427+0000","last_became_peered":"2023-11-08T16:37:32.888427+0000","last_unstale":"2023-11-08T16:37:34.698965+0000","last_undegraded":"2023-11-08T16:37:34.698965+0000","last_fullsized":"2023-11-08T16:37:34.698965+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:35:06.642562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.891767+0000","last_change":"2023-11-08T16:37:31.890482+0000","last_active":"2023-11-08T16:37:32.891767+0000","last_peered":"2023-11-08T16:37:32.891767+0000","last_clean":"2023-11-08T16:37:32.891767+0000","last_became_active":"2023-11-08T16:37:31.889651+0000","last_became_peered":"2023-11-08T16:37:31.889651+0000","last_unstale":"2023-11-08T16:37:32.891767+0000","last_undegraded":"2023-11-08T16:37:32.891767+0000","last_fullsized":"2023-11-08T16:37:32.891767+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:50:59.816607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864056+0000","last_change":"2023-11-08T16:37:32.902585+0000","last_active":"2023-11-08T16:37:33.864056+0000","last_peered":"2023-11-08T16:37:33.864056+0000","last_clean":"2023-11-08T16:37:33.864056+0000","last_became_active":"2023-11-08T16:37:32.902238+0000","last_became_peered":"2023-11-08T16:37:32.902238+0000","last_unstale":"2023-11-08T16:37:33.864056+0000","last_undegraded":"2023-11-08T16:37:33.864056+0000","last_fullsized":"2023-11-08T16:37:33.864056+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:52:21.508270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.873103+0000","last_change":"2023-11-08T16:37:31.885217+0000","last_active":"2023-11-08T16:37:32.873103+0000","last_peered":"2023-11-08T16:37:32.873103+0000","last_clean":"2023-11-08T16:37:32.873103+0000","last_became_active":"2023-11-08T16:37:31.884920+0000","last_became_peered":"2023-11-08T16:37:31.884920+0000","last_unstale":"2023-11-08T16:37:32.873103+0000","last_undegraded":"2023-11-08T16:37:32.873103+0000","last_fullsized":"2023-11-08T16:37:32.873103+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:20.170960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696247+0000","last_change":"2023-11-08T16:37:32.901124+0000","last_active":"2023-11-08T16:37:34.696247+0000","last_peered":"2023-11-08T16:37:34.696247+0000","last_clean":"2023-11-08T16:37:34.696247+0000","last_became_active":"2023-11-08T16:37:32.900154+0000","last_became_peered":"2023-11-08T16:37:32.900154+0000","last_unstale":"2023-11-08T16:37:34.696247+0000","last_undegraded":"2023-11-08T16:37:34.696247+0000","last_fullsized":"2023-11-08T16:37:34.696247+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:38:22.351916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"80'8","reported_seq":20,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.982833+0000","last_change":"2023-11-08T16:37:31.884151+0000","last_active":"2023-11-08T16:37:32.982833+0000","last_peered":"2023-11-08T16:37:32.982833+0000","last_clean":"2023-11-08T16:37:32.982833+0000","last_became_active":"2023-11-08T16:37:31.882949+0000","last_became_peered":"2023-11-08T16:37:31.882949+0000","last_unstale":"2023-11-08T16:37:32.982833+0000","last_undegraded":"2023-11-08T16:37:32.982833+0000","last_fullsized":"2023-11-08T16:37:32.982833+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:34:59.821142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.698219+0000","last_change":"2023-11-08T16:37:32.889628+0000","last_active":"2023-11-08T16:37:34.698219+0000","last_peered":"2023-11-08T16:37:34.698219+0000","last_clean":"2023-11-08T16:37:34.698219+0000","last_became_active":"2023-11-08T16:37:32.889352+0000","last_became_peered":"2023-11-08T16:37:32.889352+0000","last_unstale":"2023-11-08T16:37:34.698219+0000","last_undegraded":"2023-11-08T16:37:34.698219+0000","last_fullsized":"2023-11-08T16:37:34.698219+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:27:30.791214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.976461+0000","last_change":"2023-11-08T16:37:31.885970+0000","last_active":"2023-11-08T16:37:32.976461+0000","last_peered":"2023-11-08T16:37:32.976461+0000","last_clean":"2023-11-08T16:37:32.976461+0000","last_became_active":"2023-11-08T16:37:31.885703+0000","last_became_peered":"2023-11-08T16:37:31.885703+0000","last_unstale":"2023-11-08T16:37:32.976461+0000","last_undegraded":"2023-11-08T16:37:32.976461+0000","last_fullsized":"2023-11-08T16:37:32.976461+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:15:10.920507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.868621+0000","last_change":"2023-11-08T16:37:32.891988+0000","last_active":"2023-11-08T16:37:33.868621+0000","last_peered":"2023-11-08T16:37:33.868621+0000","last_clean":"2023-11-08T16:37:33.868621+0000","last_became_active":"2023-11-08T16:37:32.891705+0000","last_became_peered":"2023-11-08T16:37:32.891705+0000","last_unstale":"2023-11-08T16:37:33.868621+0000","last_undegraded":"2023-11-08T16:37:33.868621+0000","last_fullsized":"2023-11-08T16:37:33.868621+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:52:21.508270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.979303+0000","last_change":"2023-11-08T16:37:31.891191+0000","last_active":"2023-11-08T16:37:32.979303+0000","last_peered":"2023-11-08T16:37:32.979303+0000","last_clean":"2023-11-08T16:37:32.979303+0000","last_became_active":"2023-11-08T16:37:31.890873+0000","last_became_peered":"2023-11-08T16:37:31.890873+0000","last_unstale":"2023-11-08T16:37:32.979303+0000","last_undegraded":"2023-11-08T16:37:32.979303+0000","last_fullsized":"2023-11-08T16:37:32.979303+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:15:10.920507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.865813+0000","last_change":"2023-11-08T16:37:32.890877+0000","last_active":"2023-11-08T16:37:33.865813+0000","last_peered":"2023-11-08T16:37:33.865813+0000","last_clean":"2023-11-08T16:37:33.865813+0000","last_became_active":"2023-11-08T16:37:32.889313+0000","last_became_peered":"2023-11-08T16:37:32.889313+0000","last_unstale":"2023-11-08T16:37:33.865813+0000","last_undegraded":"2023-11-08T16:37:33.865813+0000","last_fullsized":"2023-11-08T16:37:33.865813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:39.874473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"80'1","reported_seq":13,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.977489+0000","last_change":"2023-11-08T16:37:31.892128+0000","last_active":"2023-11-08T16:37:32.977489+0000","last_peered":"2023-11-08T16:37:32.977489+0000","last_clean":"2023-11-08T16:37:32.977489+0000","last_became_active":"2023-11-08T16:37:31.891820+0000","last_became_peered":"2023-11-08T16:37:31.891820+0000","last_unstale":"2023-11-08T16:37:32.977489+0000","last_undegraded":"2023-11-08T16:37:32.977489+0000","last_fullsized":"2023-11-08T16:37:32.977489+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:08:38.798225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696313+0000","last_change":"2023-11-08T16:37:32.896534+0000","last_active":"2023-11-08T16:37:34.696313+0000","last_peered":"2023-11-08T16:37:34.696313+0000","last_clean":"2023-11-08T16:37:34.696313+0000","last_became_active":"2023-11-08T16:37:32.896085+0000","last_became_peered":"2023-11-08T16:37:32.896085+0000","last_unstale":"2023-11-08T16:37:34.696313+0000","last_undegraded":"2023-11-08T16:37:34.696313+0000","last_fullsized":"2023-11-08T16:37:34.696313+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:26:37.454187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.866566+0000","last_change":"2023-11-08T16:37:31.888250+0000","last_active":"2023-11-08T16:37:32.866566+0000","last_peered":"2023-11-08T16:37:32.866566+0000","last_clean":"2023-11-08T16:37:32.866566+0000","last_became_active":"2023-11-08T16:37:31.887364+0000","last_became_peered":"2023-11-08T16:37:31.887364+0000","last_unstale":"2023-11-08T16:37:32.866566+0000","last_undegraded":"2023-11-08T16:37:32.866566+0000","last_fullsized":"2023-11-08T16:37:32.866566+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:07:20.170960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107632+0000","last_change":"2023-11-08T16:37:32.889356+0000","last_active":"2023-11-08T16:37:34.107632+0000","last_peered":"2023-11-08T16:37:34.107632+0000","last_clean":"2023-11-08T16:37:34.107632+0000","last_became_active":"2023-11-08T16:37:32.887921+0000","last_became_peered":"2023-11-08T16:37:32.887921+0000","last_unstale":"2023-11-08T16:37:34.107632+0000","last_undegraded":"2023-11-08T16:37:34.107632+0000","last_fullsized":"2023-11-08T16:37:34.107632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:24:35.603662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.891840+0000","last_change":"2023-11-08T16:37:31.890236+0000","last_active":"2023-11-08T16:37:32.891840+0000","last_peered":"2023-11-08T16:37:32.891840+0000","last_clean":"2023-11-08T16:37:32.891840+0000","last_became_active":"2023-11-08T16:37:31.889327+0000","last_became_peered":"2023-11-08T16:37:31.889327+0000","last_unstale":"2023-11-08T16:37:32.891840+0000","last_undegraded":"2023-11-08T16:37:32.891840+0000","last_fullsized":"2023-11-08T16:37:32.891840+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:34:59.821142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107319+0000","last_change":"2023-11-08T16:37:32.898723+0000","last_active":"2023-11-08T16:37:34.107319+0000","last_peered":"2023-11-08T16:37:34.107319+0000","last_clean":"2023-11-08T16:37:34.107319+0000","last_became_active":"2023-11-08T16:37:32.897690+0000","last_became_peered":"2023-11-08T16:37:32.897690+0000","last_unstale":"2023-11-08T16:37:34.107319+0000","last_undegraded":"2023-11-08T16:37:34.107319+0000","last_fullsized":"2023-11-08T16:37:34.107319+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:09:01.413694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.873413+0000","last_change":"2023-11-08T16:37:31.881532+0000","last_active":"2023-11-08T16:37:32.873413+0000","last_peered":"2023-11-08T16:37:32.873413+0000","last_clean":"2023-11-08T16:37:32.873413+0000","last_became_active":"2023-11-08T16:37:31.881220+0000","last_became_peered":"2023-11-08T16:37:31.881220+0000","last_unstale":"2023-11-08T16:37:32.873413+0000","last_undegraded":"2023-11-08T16:37:32.873413+0000","last_fullsized":"2023-11-08T16:37:32.873413+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:43:38.857308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107599+0000","last_change":"2023-11-08T16:37:32.901904+0000","last_active":"2023-11-08T16:37:34.107599+0000","last_peered":"2023-11-08T16:37:34.107599+0000","last_clean":"2023-11-08T16:37:34.107599+0000","last_became_active":"2023-11-08T16:37:32.901589+0000","last_became_peered":"2023-11-08T16:37:32.901589+0000","last_unstale":"2023-11-08T16:37:34.107599+0000","last_undegraded":"2023-11-08T16:37:34.107599+0000","last_fullsized":"2023-11-08T16:37:34.107599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:30:02.805859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.887306+0000","last_change":"2023-11-08T16:37:31.885921+0000","last_active":"2023-11-08T16:37:32.887306+0000","last_peered":"2023-11-08T16:37:32.887306+0000","last_clean":"2023-11-08T16:37:32.887306+0000","last_became_active":"2023-11-08T16:37:31.885621+0000","last_became_peered":"2023-11-08T16:37:31.885621+0000","last_unstale":"2023-11-08T16:37:32.887306+0000","last_undegraded":"2023-11-08T16:37:32.887306+0000","last_fullsized":"2023-11-08T16:37:32.887306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:24:56.871455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.864362+0000","last_change":"2023-11-08T16:37:32.896253+0000","last_active":"2023-11-08T16:37:33.864362+0000","last_peered":"2023-11-08T16:37:33.864362+0000","last_clean":"2023-11-08T16:37:33.864362+0000","last_became_active":"2023-11-08T16:37:32.895943+0000","last_became_peered":"2023-11-08T16:37:32.895943+0000","last_unstale":"2023-11-08T16:37:33.864362+0000","last_undegraded":"2023-11-08T16:37:33.864362+0000","last_fullsized":"2023-11-08T16:37:33.864362+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:57:57.489427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.977235+0000","last_change":"2023-11-08T16:37:31.886864+0000","last_active":"2023-11-08T16:37:32.977235+0000","last_peered":"2023-11-08T16:37:32.977235+0000","last_clean":"2023-11-08T16:37:32.977235+0000","last_became_active":"2023-11-08T16:37:31.886526+0000","last_became_peered":"2023-11-08T16:37:31.886526+0000","last_unstale":"2023-11-08T16:37:32.977235+0000","last_undegraded":"2023-11-08T16:37:32.977235+0000","last_fullsized":"2023-11-08T16:37:32.977235+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:16:26.042660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699708+0000","last_change":"2023-11-08T16:37:32.889339+0000","last_active":"2023-11-08T16:37:34.699708+0000","last_peered":"2023-11-08T16:37:34.699708+0000","last_clean":"2023-11-08T16:37:34.699708+0000","last_became_active":"2023-11-08T16:37:32.889059+0000","last_became_peered":"2023-11-08T16:37:32.889059+0000","last_unstale":"2023-11-08T16:37:34.699708+0000","last_undegraded":"2023-11-08T16:37:34.699708+0000","last_fullsized":"2023-11-08T16:37:34.699708+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:13:41.486934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.699419+0000","last_change":"2023-11-08T16:37:31.887047+0000","last_active":"2023-11-08T16:37:34.699419+0000","last_peered":"2023-11-08T16:37:34.699419+0000","last_clean":"2023-11-08T16:37:34.699419+0000","last_became_active":"2023-11-08T16:37:31.884896+0000","last_became_peered":"2023-11-08T16:37:31.884896+0000","last_unstale":"2023-11-08T16:37:34.699419+0000","last_undegraded":"2023-11-08T16:37:34.699419+0000","last_fullsized":"2023-11-08T16:37:34.699419+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:18:18.736847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.867883+0000","last_change":"2023-11-08T16:37:32.891927+0000","last_active":"2023-11-08T16:37:33.867883+0000","last_peered":"2023-11-08T16:37:33.867883+0000","last_clean":"2023-11-08T16:37:33.867883+0000","last_became_active":"2023-11-08T16:37:32.891570+0000","last_became_peered":"2023-11-08T16:37:32.891570+0000","last_unstale":"2023-11-08T16:37:33.867883+0000","last_undegraded":"2023-11-08T16:37:33.867883+0000","last_fullsized":"2023-11-08T16:37:33.867883+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:13:26.284908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.888996+0000","last_change":"2023-11-08T16:37:31.879909+0000","last_active":"2023-11-08T16:37:32.888996+0000","last_peered":"2023-11-08T16:37:32.888996+0000","last_clean":"2023-11-08T16:37:32.888996+0000","last_became_active":"2023-11-08T16:37:31.878874+0000","last_became_peered":"2023-11-08T16:37:31.878874+0000","last_unstale":"2023-11-08T16:37:32.888996+0000","last_undegraded":"2023-11-08T16:37:32.888996+0000","last_fullsized":"2023-11-08T16:37:32.888996+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:18:57.040797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.868851+0000","last_change":"2023-11-08T16:37:32.889394+0000","last_active":"2023-11-08T16:37:33.868851+0000","last_peered":"2023-11-08T16:37:33.868851+0000","last_clean":"2023-11-08T16:37:33.868851+0000","last_became_active":"2023-11-08T16:37:32.889046+0000","last_became_peered":"2023-11-08T16:37:32.889046+0000","last_unstale":"2023-11-08T16:37:33.868851+0000","last_undegraded":"2023-11-08T16:37:33.868851+0000","last_fullsized":"2023-11-08T16:37:33.868851+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:00:16.203256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.883781+0000","last_change":"2023-11-08T16:37:31.889271+0000","last_active":"2023-11-08T16:37:32.883781+0000","last_peered":"2023-11-08T16:37:32.883781+0000","last_clean":"2023-11-08T16:37:32.883781+0000","last_became_active":"2023-11-08T16:37:31.888932+0000","last_became_peered":"2023-11-08T16:37:31.888932+0000","last_unstale":"2023-11-08T16:37:32.883781+0000","last_undegraded":"2023-11-08T16:37:32.883781+0000","last_fullsized":"2023-11-08T16:37:32.883781+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:15:56.265919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.696444+0000","last_change":"2023-11-08T16:37:32.890369+0000","last_active":"2023-11-08T16:37:34.696444+0000","last_peered":"2023-11-08T16:37:34.696444+0000","last_clean":"2023-11-08T16:37:34.696444+0000","last_became_active":"2023-11-08T16:37:32.890080+0000","last_became_peered":"2023-11-08T16:37:32.890080+0000","last_unstale":"2023-11-08T16:37:34.696444+0000","last_undegraded":"2023-11-08T16:37:34.696444+0000","last_fullsized":"2023-11-08T16:37:34.696444+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:11:09.318318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.885602+0000","last_change":"2023-11-08T16:37:31.888523+0000","last_active":"2023-11-08T16:37:32.885602+0000","last_peered":"2023-11-08T16:37:32.885602+0000","last_clean":"2023-11-08T16:37:32.885602+0000","last_became_active":"2023-11-08T16:37:31.888132+0000","last_became_peered":"2023-11-08T16:37:31.888132+0000","last_unstale":"2023-11-08T16:37:32.885602+0000","last_undegraded":"2023-11-08T16:37:32.885602+0000","last_fullsized":"2023-11-08T16:37:32.885602+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:25:30.803105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697698+0000","last_change":"2023-11-08T16:37:32.895914+0000","last_active":"2023-11-08T16:37:34.697698+0000","last_peered":"2023-11-08T16:37:34.697698+0000","last_clean":"2023-11-08T16:37:34.697698+0000","last_became_active":"2023-11-08T16:37:32.895636+0000","last_became_peered":"2023-11-08T16:37:32.895636+0000","last_unstale":"2023-11-08T16:37:34.697698+0000","last_undegraded":"2023-11-08T16:37:34.697698+0000","last_fullsized":"2023-11-08T16:37:34.697698+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:32:09.784202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.876843+0000","last_change":"2023-11-08T16:37:31.882693+0000","last_active":"2023-11-08T16:37:32.876843+0000","last_peered":"2023-11-08T16:37:32.876843+0000","last_clean":"2023-11-08T16:37:32.876843+0000","last_became_active":"2023-11-08T16:37:31.882403+0000","last_became_peered":"2023-11-08T16:37:31.882403+0000","last_unstale":"2023-11-08T16:37:32.876843+0000","last_undegraded":"2023-11-08T16:37:32.876843+0000","last_fullsized":"2023-11-08T16:37:32.876843+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:27:29.774049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:33.865464+0000","last_change":"2023-11-08T16:37:32.892923+0000","last_active":"2023-11-08T16:37:33.865464+0000","last_peered":"2023-11-08T16:37:33.865464+0000","last_clean":"2023-11-08T16:37:33.865464+0000","last_became_active":"2023-11-08T16:37:32.889324+0000","last_became_peered":"2023-11-08T16:37:32.889324+0000","last_unstale":"2023-11-08T16:37:33.865464+0000","last_undegraded":"2023-11-08T16:37:33.865464+0000","last_fullsized":"2023-11-08T16:37:33.865464+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:44:12.076692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.883572+0000","last_change":"2023-11-08T16:37:31.885170+0000","last_active":"2023-11-08T16:37:32.883572+0000","last_peered":"2023-11-08T16:37:32.883572+0000","last_clean":"2023-11-08T16:37:32.883572+0000","last_became_active":"2023-11-08T16:37:31.884855+0000","last_became_peered":"2023-11-08T16:37:31.884855+0000","last_unstale":"2023-11-08T16:37:32.883572+0000","last_undegraded":"2023-11-08T16:37:32.883572+0000","last_fullsized":"2023-11-08T16:37:32.883572+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:09:53.121004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.697502+0000","last_change":"2023-11-08T16:37:32.899028+0000","last_active":"2023-11-08T16:37:34.697502+0000","last_peered":"2023-11-08T16:37:34.697502+0000","last_clean":"2023-11-08T16:37:34.697502+0000","last_became_active":"2023-11-08T16:37:32.898773+0000","last_became_peered":"2023-11-08T16:37:32.898773+0000","last_unstale":"2023-11-08T16:37:34.697502+0000","last_undegraded":"2023-11-08T16:37:34.697502+0000","last_fullsized":"2023-11-08T16:37:34.697502+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:11:18.438471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.891917+0000","last_change":"2023-11-08T16:37:31.888762+0000","last_active":"2023-11-08T16:37:32.891917+0000","last_peered":"2023-11-08T16:37:32.891917+0000","last_clean":"2023-11-08T16:37:32.891917+0000","last_became_active":"2023-11-08T16:37:31.888322+0000","last_became_peered":"2023-11-08T16:37:31.888322+0000","last_unstale":"2023-11-08T16:37:32.891917+0000","last_undegraded":"2023-11-08T16:37:32.891917+0000","last_fullsized":"2023-11-08T16:37:32.891917+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:22:17.010347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-08T16:37:34.107673+0000","last_change":"2023-11-08T16:37:32.902276+0000","last_active":"2023-11-08T16:37:34.107673+0000","last_peered":"2023-11-08T16:37:34.107673+0000","last_clean":"2023-11-08T16:37:34.107673+0000","last_became_active":"2023-11-08T16:37:32.902005+0000","last_became_peered":"2023-11-08T16:37:32.902005+0000","last_unstale":"2023-11-08T16:37:34.107673+0000","last_undegraded":"2023-11-08T16:37:34.107673+0000","last_fullsized":"2023-11-08T16:37:34.107673+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:31.836464+0000","last_clean_scrub_stamp":"2023-11-08T16:37:31.836464+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:39:11.763525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"80'2","reported_seq":14,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.976897+0000","last_change":"2023-11-08T16:37:31.889695+0000","last_active":"2023-11-08T16:37:32.976897+0000","last_peered":"2023-11-08T16:37:32.976897+0000","last_clean":"2023-11-08T16:37:32.976897+0000","last_became_active":"2023-11-08T16:37:31.889172+0000","last_became_peered":"2023-11-08T16:37:31.889172+0000","last_unstale":"2023-11-08T16:37:32.976897+0000","last_undegraded":"2023-11-08T16:37:32.976897+0000","last_fullsized":"2023-11-08T16:37:32.976897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:37:30.819298+0000","last_clean_scrub_stamp":"2023-11-08T16:37:30.819298+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:04:01.029956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":229,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-08T16:37:32.870100+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:37:32.870100+0000","last_peered":"2023-11-08T16:37:32.870100+0000","last_clean":"2023-11-08T16:37:32.870100+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:37:32.870100+0000","last_undegraded":"2023-11-08T16:37:32.870100+0000","last_fullsized":"2023-11-08T16:37:32.870100+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574955,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1589248,"data_stored":1035499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":1.0900000000000001,"15min":1.0860000000000001},"min":{"1min":0.67000000000000004,"5min":0.51600000000000001,"15min":0.49299999999999999},"max":{"1min":1.6879999999999999,"5min":3.1899999999999999,"15min":3.1899999999999999},"last":0.876},{"interface":"front","average":{"1min":1.014,"5min":1.0880000000000001,"15min":1.111},"min":{"1min":0.69399999999999995,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3819999999999999,"5min":2.7599999999999998,"15min":4.2729999999999997},"last":1.145}]},{"osd":1,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0960000000000001,"15min":1.117},"min":{"1min":0.72399999999999998,"5min":0.60199999999999998,"15min":0.57899999999999996},"max":{"1min":1.52,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.476},{"interface":"front","average":{"1min":1.129,"5min":1.2010000000000001,"15min":1.2190000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.5009999999999999,"5min":3.3740000000000001,"15min":3.3740000000000001},"last":1.2090000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.171,"15min":1.248},"min":{"1min":0.75900000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4359999999999999,"5min":1.7230000000000001,"15min":4.2249999999999996},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.208},"min":{"1min":0.71099999999999997,"5min":0.67500000000000004,"15min":0.56299999999999994},"max":{"1min":1.524,"5min":3.4750000000000001,"15min":3.4750000000000001},"last":0.752}]},{"osd":3,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1759999999999999,"15min":1.1619999999999999},"min":{"1min":0.70299999999999996,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.589,"5min":2.964,"15min":2.964},"last":1.25},{"interface":"front","average":{"1min":1.337,"5min":1.3480000000000001,"15min":1.4490000000000001},"min":{"1min":0.86899999999999999,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.7090000000000001,"5min":3.5680000000000001,"15min":4.181},"last":1.4299999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.232,"15min":1.224},"min":{"1min":0.83099999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.53,"5min":3.6669999999999998,"15min":3.6669999999999998},"last":1.034},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3200000000000001,"15min":1.335},"min":{"1min":0.872,"5min":0.77300000000000002,"15min":0.59099999999999997},"max":{"1min":1.73,"5min":1.9350000000000001,"15min":2.3730000000000002},"last":1.4059999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4159999999999999,"15min":1.361},"min":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.61799999999999999},"max":{"1min":1.6930000000000001,"5min":3.1059999999999999,"15min":3.1059999999999999},"last":1.333},{"interface":"front","average":{"1min":1.379,"5min":1.456,"15min":1.4630000000000001},"min":{"1min":1.0760000000000001,"5min":0.97399999999999998,"15min":0.878},"max":{"1min":1.6279999999999999,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.5249999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:37:14 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.5389999999999999,"15min":1.53},"min":{"1min":1.129,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.7929999999999999,"5min":3.7530000000000001,"15min":3.7530000000000001},"last":1.573},{"interface":"front","average":{"1min":1.427,"5min":1.5129999999999999,"15min":1.4830000000000001},"min":{"1min":1.018,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":1.8180000000000001,"5min":6.9560000000000004,"15min":6.9560000000000004},"last":1.5409999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574957,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":444607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.042,"15min":1.056},"min":{"1min":0.60399999999999998,"5min":0.497,"15min":0.497},"max":{"1min":1.51,"5min":2.5609999999999999,"15min":2.5609999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.095,"15min":1.0780000000000001},"min":{"1min":0.63100000000000001,"5min":0.56999999999999995,"15min":0.56200000000000006},"max":{"1min":1.5109999999999999,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":0.79300000000000004}]},{"osd":1,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.0820000000000001,"15min":1.05},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.603,"5min":1.603,"15min":1.609},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.046,"5min":1.113,"15min":1.1160000000000001},"min":{"1min":0.71899999999999997,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.3440000000000001,"5min":1.97,"15min":1.97},"last":1.1279999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.131,"15min":1.1950000000000001},"min":{"1min":0.71899999999999997,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.6890000000000001,"5min":1.804,"15min":1.804},"last":0.95899999999999996},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.1240000000000001,"15min":1.163},"min":{"1min":0.80000000000000004,"5min":0.61899999999999999,"15min":0.60499999999999998},"max":{"1min":1.5429999999999999,"5min":1.7310000000000001,"15min":1.7669999999999999},"last":1.4339999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.216,"15min":1.264},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.58299999999999996},"max":{"1min":1.4710000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.2370000000000001,"15min":1.292},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.67300000000000004},"max":{"1min":1.599,"5min":1.76,"15min":1.95},"last":1.0720000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.3029999999999999,"15min":1.3520000000000001},"min":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.65300000000000002},"max":{"1min":1.595,"5min":2.218,"15min":2.218},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.405,"5min":1.3380000000000001,"15min":1.4139999999999999},"min":{"1min":1.095,"5min":0.64900000000000002,"15min":0.64900000000000002},"max":{"1min":1.778,"5min":2.085,"15min":2.7080000000000002},"last":1.1819999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.3280000000000001,"15min":1.3149999999999999},"min":{"1min":0.96799999999999997,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.845,"5min":1.8839999999999999,"15min":2.105},"last":1.587},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.5069999999999999,"15min":1.5489999999999999},"min":{"1min":1.202,"5min":1.1579999999999999,"15min":0.78500000000000003},"max":{"1min":1.9630000000000001,"5min":2.4700000000000002,"15min":2.4700000000000002},"last":1.4099999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.5129999999999999,"15min":1.583},"min":{"1min":1.228,"5min":1.002,"15min":0.74299999999999999},"max":{"1min":1.7949999999999999,"5min":2.8849999999999998,"15min":3.8959999999999999},"last":1.357},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.4790000000000001,"15min":1.514},"min":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96099999999999997},"max":{"1min":2,"5min":2,"15min":2.4889999999999999},"last":1.8300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574957,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":991232,"data_stored":444605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":1.0529999999999999,"15min":1.0589999999999999},"min":{"1min":0.72799999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.409,"5min":1.7969999999999999,"15min":2.4609999999999999},"last":0.98199999999999998},{"interface":"front","average":{"1min":1,"5min":1.0780000000000001,"15min":1.111},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.3460000000000001,"5min":1.593,"15min":2.3639999999999999},"last":1.544}]},{"osd":1,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.234,"15min":1.2390000000000001},"min":{"1min":0.70599999999999996,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.5269999999999999,"5min":1.9610000000000001,"15min":3.8690000000000002},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.177,"15min":1.26},"min":{"1min":0.66400000000000003,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.651,"5min":1.9370000000000001,"15min":3.2349999999999999},"last":1.2669999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.1399999999999999,"15min":1.2170000000000001},"min":{"1min":0.77700000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5369999999999999,"5min":1.778,"15min":2.6339999999999999},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.198,"15min":1.2150000000000001},"min":{"1min":0.89900000000000002,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.4419999999999999,"5min":1.8029999999999999,"15min":2.2559999999999998},"last":1.163}]},{"osd":3,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.238,"15min":1.294},"min":{"1min":0.90900000000000003,"5min":0.67300000000000004,"15min":0.66500000000000004},"max":{"1min":1.4910000000000001,"5min":1.8620000000000001,"15min":2.8330000000000002},"last":1.46},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.246,"15min":1.298},"min":{"1min":0.753,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.5840000000000001,"5min":2.0459999999999998,"15min":2.5760000000000001},"last":1.397}]},{"osd":4,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.278,"15min":1.351},"min":{"1min":0.89300000000000002,"5min":0.76700000000000002,"15min":0.61599999999999999},"max":{"1min":1.736,"5min":2.0790000000000002,"15min":3.9790000000000001},"last":1.103},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.409,"15min":1.47},"min":{"1min":1.1910000000000001,"5min":0.97699999999999998,"15min":0.81499999999999995},"max":{"1min":1.633,"5min":2.2010000000000001,"15min":3.0209999999999999},"last":1.0569999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4079999999999999,"5min":1.399,"15min":1.4139999999999999},"min":{"1min":1.1020000000000001,"5min":0.92400000000000004,"15min":0.752},"max":{"1min":1.673,"5min":1.867,"15min":2.5550000000000002},"last":1.603},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.4099999999999999,"15min":1.4910000000000001},"min":{"1min":0.999,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":1.6870000000000001,"5min":2.3210000000000002,"15min":4.1189999999999998},"last":1.337}]},{"osd":7,"last update":"Wed Nov 8 16:37:10 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.522,"15min":1.5660000000000001},"min":{"1min":1.218,"5min":1.099,"15min":1.0049999999999999},"max":{"1min":1.7789999999999999,"5min":2.129,"15min":3.1160000000000001},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.556,"5min":1.5269999999999999,"15min":1.556},"min":{"1min":1.3029999999999999,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":1.9219999999999999,"5min":2.3660000000000001,"15min":2.7050000000000001},"last":1.3080000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574956,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":991232,"data_stored":444585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.105,"15min":1.079},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.534,"5min":1.587,"15min":1.587},"last":1.597},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.091,"15min":1.0609999999999999},"min":{"1min":0.59899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.383,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":1.2010000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.133,"15min":1.125},"min":{"1min":0.77900000000000003,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4490000000000001,"5min":1.8660000000000001,"15min":2.0459999999999998},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.109,"5min":1.1819999999999999,"15min":1.1659999999999999},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.5489999999999999,"5min":1.742,"15min":1.742},"last":1.421}]},{"osd":2,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.218,"15min":1.1950000000000001},"min":{"1min":0.69599999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":1.577,"5min":1.7330000000000001,"15min":1.746},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.2,"15min":1.252},"min":{"1min":0.86399999999999999,"5min":0.77800000000000002,"15min":0.71699999999999997},"max":{"1min":1.474,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.129}]},{"osd":3,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1930000000000001,"5min":1.24,"15min":1.2649999999999999},"min":{"1min":0.82399999999999995,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":1.694,"5min":1.7689999999999999,"15min":1.9199999999999999},"last":1.661},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.236,"15min":1.2749999999999999},"min":{"1min":0.72799999999999998,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.611,"5min":1.851,"15min":1.895},"last":1.073}]},{"osd":5,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.361,"15min":1.3640000000000001},"min":{"1min":0.92700000000000005,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.736,"5min":1.8959999999999999,"15min":1.9870000000000001},"last":1.339},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4330000000000001,"15min":1.4239999999999999},"min":{"1min":1.196,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.798,"5min":1.8899999999999999,"15min":2.46},"last":1.4710000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.333,"15min":1.4059999999999999},"min":{"1min":0.98499999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.7230000000000001,"5min":1.9510000000000001,"15min":2.5609999999999999},"last":1.256},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.5,"15min":1.552},"min":{"1min":1.1659999999999999,"5min":0.81200000000000006,"15min":0.81200000000000006},"max":{"1min":1.825,"5min":2.0310000000000001,"15min":2.6389999999999998},"last":1.514}]},{"osd":7,"last update":"Wed Nov 8 16:37:09 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.516,"15min":1.52},"min":{"1min":1.056,"5min":0.92300000000000004,"15min":0.92300000000000004},"max":{"1min":1.881,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.298},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.4970000000000001,"15min":1.5800000000000001},"min":{"1min":1.282,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":1.837,"5min":2.0670000000000002,"15min":2.6859999999999999},"last":1.3799999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574956,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":995328,"data_stored":444627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.77100000000000002,"15min":0.81399999999999995},"min":{"1min":0.50900000000000001,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.0289999999999999,"5min":1.9339999999999999,"15min":1.9339999999999999},"last":1.1459999999999999},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.81499999999999995,"15min":0.92100000000000004},"min":{"1min":0.50800000000000001,"5min":0.316,"15min":0.316},"max":{"1min":1.2390000000000001,"5min":1.3919999999999999,"15min":1.585},"last":0.745}]},{"osd":1,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.90200000000000002,"5min":0.875,"15min":0.89700000000000002},"min":{"1min":0.65000000000000002,"5min":0.44600000000000001,"15min":0.36499999999999999},"max":{"1min":1.264,"5min":1.4770000000000001,"15min":1.4790000000000001},"last":1.101},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.93899999999999995,"15min":0.96599999999999997},"min":{"1min":0.59499999999999997,"5min":0.378,"15min":0.378},"max":{"1min":1.2629999999999999,"5min":1.4590000000000001,"15min":1.829},"last":0.69399999999999995}]},{"osd":2,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.996,"5min":0.99099999999999999,"15min":0.98699999999999999},"min":{"1min":0.60499999999999998,"5min":0.48399999999999999,"15min":0.41699999999999998},"max":{"1min":1.7490000000000001,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":1.5109999999999999},{"interface":"front","average":{"1min":0.98599999999999999,"5min":1.075,"15min":1.129},"min":{"1min":0.53400000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.377,"5min":1.6839999999999999,"15min":2.4660000000000002},"last":1.625}]},{"osd":4,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1579999999999999,"5min":1.143,"15min":1.1970000000000001},"min":{"1min":0.82099999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.49,"5min":1.8069999999999999,"15min":2.25},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.2410000000000001,"15min":1.266},"min":{"1min":0.93899999999999995,"5min":0.747,"15min":0.747},"max":{"1min":1.5840000000000001,"5min":1.8280000000000001,"15min":2.1829999999999998},"last":1.3169999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3029999999999999,"15min":1.329},"min":{"1min":1.0249999999999999,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.76,"5min":1.891,"15min":2.3999999999999999},"last":1.214},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.2969999999999999,"15min":1.341},"min":{"1min":0.91600000000000004,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.669,"5min":1.7869999999999999,"15min":2.593},"last":1.5569999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.341,"15min":1.3999999999999999},"min":{"1min":0.96499999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.7070000000000001,"5min":1.744,"15min":2.9569999999999999},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.2849999999999999,"15min":1.391},"min":{"1min":1.006,"5min":0.75800000000000001,"15min":0.73399999999999999},"max":{"1min":1.5860000000000001,"5min":1.6890000000000001,"15min":2.762},"last":1.377}]},{"osd":7,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.319,"15min":1.4319999999999999},"min":{"1min":0.95599999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.677,"5min":1.861,"15min":2.8700000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3720000000000001,"15min":1.4279999999999999},"min":{"1min":1.113,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.71,"5min":2.1739999999999999,"15min":2.633},"last":1.849}]}]},{"osd":2,"up_from":13,"seq":55834574957,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":446127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79400000000000004,"15min":0.98599999999999999},"min":{"1min":0.46800000000000003,"5min":0.32600000000000001,"15min":0.24099999999999999},"max":{"1min":1.2030000000000001,"5min":1.706,"15min":4.5369999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.83299999999999996,"15min":0.873},"min":{"1min":0.48499999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.3620000000000001,"5min":1.611,"15min":1.611},"last":1.0449999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.95999999999999996,"15min":1.1000000000000001},"min":{"1min":0.67800000000000005,"5min":0.52300000000000002,"15min":0.379},"max":{"1min":1.4159999999999999,"5min":1.6000000000000001,"15min":4.4980000000000002},"last":1.2569999999999999},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.97499999999999998,"15min":1.0449999999999999},"min":{"1min":0.63400000000000001,"5min":0.53600000000000003,"15min":0.40300000000000002},"max":{"1min":1.3540000000000001,"5min":1.6220000000000001,"15min":1.6879999999999999},"last":1.2090000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.84499999999999997,"15min":1.024},"min":{"1min":0.49099999999999999,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.25,"5min":1.3879999999999999,"15min":3.3050000000000002},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.0249999999999999,"15min":1.2110000000000001},"min":{"1min":0.60599999999999998,"5min":0.502,"15min":0.42999999999999999},"max":{"1min":1.3360000000000001,"5min":1.524,"15min":4.5899999999999999},"last":1.4530000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.246,"15min":1.3999999999999999},"min":{"1min":0.92200000000000004,"5min":0.73499999999999999,"15min":0.73499999999999999},"max":{"1min":1.593,"5min":1.5980000000000001,"15min":4.9409999999999998},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.165,"15min":1.2709999999999999},"min":{"1min":0.83399999999999996,"5min":0.72499999999999998,"15min":0.63100000000000001},"max":{"1min":1.5389999999999999,"5min":1.625,"15min":2.46},"last":1.341}]},{"osd":5,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.2130000000000001,"15min":1.3149999999999999},"min":{"1min":0.88600000000000001,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.4790000000000001,"5min":1.637,"15min":2.8530000000000002},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2230000000000001,"15min":1.351},"min":{"1min":0.92600000000000005,"5min":0.71099999999999997,"15min":0.69399999999999995},"max":{"1min":1.5269999999999999,"5min":1.6659999999999999,"15min":3.5680000000000001},"last":1.375}]},{"osd":6,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2470000000000001,"15min":1.393},"min":{"1min":0.93700000000000006,"5min":0.76600000000000001,"15min":0.625},"max":{"1min":1.9410000000000001,"5min":1.9410000000000001,"15min":3.536},"last":1.427},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.363,"15min":1.5009999999999999},"min":{"1min":1.095,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.74,"5min":1.74,"15min":4.7549999999999999},"last":1.6499999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:37:01 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.2969999999999999,"15min":1.4339999999999999},"min":{"1min":1.0289999999999999,"5min":0.748,"15min":0.72999999999999998},"max":{"1min":1.5880000000000001,"5min":1.7290000000000001,"15min":2.9700000000000002},"last":1.5},{"interface":"front","average":{"1min":1.369,"5min":1.3420000000000001,"15min":1.4930000000000001},"min":{"1min":1.0900000000000001,"5min":0.81399999999999995,"15min":0.70499999999999996},"max":{"1min":1.881,"5min":2.0449999999999999,"15min":3.4950000000000001},"last":1.3100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1007616,"data_stored":446743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.753,"15min":0.77300000000000002},"min":{"1min":0.42999999999999999,"5min":0.33400000000000002,"15min":0.28899999999999998},"max":{"1min":0.98699999999999999,"5min":1.542,"15min":1.542},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.86599999999999999,"15min":0.995},"min":{"1min":0.56399999999999995,"5min":0.49199999999999999,"15min":0.309},"max":{"1min":1.2230000000000001,"5min":1.458,"15min":4.0309999999999997},"last":1.2290000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.97499999999999998,"15min":1.0780000000000001},"min":{"1min":0.54800000000000004,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.5069999999999999,"5min":1.5069999999999999,"15min":3.0339999999999998},"last":1.7689999999999999},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92800000000000005,"15min":1.0620000000000001},"min":{"1min":0.55700000000000005,"5min":0.44600000000000001,"15min":0.443},"max":{"1min":1.4750000000000001,"5min":1.5169999999999999,"15min":2.6840000000000002},"last":1.0429999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.97799999999999998,"15min":1.093},"min":{"1min":0.59399999999999997,"5min":0.43099999999999999,"15min":0.38900000000000001},"max":{"1min":1.468,"5min":1.476,"15min":4.3179999999999996},"last":1.383},{"interface":"front","average":{"1min":1.032,"5min":1.0509999999999999,"15min":1.1140000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.34999999999999998},"max":{"1min":1.5900000000000001,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":1.5169999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.1379999999999999,"5min":1.155,"15min":1.2969999999999999},"min":{"1min":0.92600000000000005,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.4359999999999999,"5min":1.698,"15min":3.1659999999999999},"last":1.131},{"interface":"front","average":{"1min":1.208,"5min":1.2230000000000001,"15min":1.3300000000000001},"min":{"1min":0.86899999999999999,"5min":0.71399999999999997,"15min":0.67600000000000005},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":2.734},"last":1.823}]},{"osd":5,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.2490000000000001,"15min":1.6040000000000001},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.67100000000000004},"max":{"1min":1.921,"5min":1.921,"15min":8.4730000000000008},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2809999999999999,"15min":1.5800000000000001},"min":{"1min":1.0149999999999999,"5min":0.84599999999999997,"15min":0.69399999999999995},"max":{"1min":1.6100000000000001,"5min":1.9770000000000001,"15min":8.8040000000000003},"last":1.871}]},{"osd":6,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.29,"15min":1.444},"min":{"1min":0.77300000000000002,"5min":0.69899999999999995,"15min":0.68700000000000006},"max":{"1min":1.831,"5min":1.831,"15min":3.984},"last":1.9119999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.3140000000000001,"15min":1.4379999999999999},"min":{"1min":1.034,"5min":0.72899999999999998,"15min":0.72899999999999998},"max":{"1min":1.554,"5min":1.843,"15min":3.2450000000000001},"last":1.427}]},{"osd":7,"last update":"Wed Nov 8 16:37:08 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.298,"15min":1.514},"min":{"1min":0.91400000000000003,"5min":0.81399999999999995,"15min":0.69199999999999995},"max":{"1min":1.6919999999999999,"5min":2.0019999999999998,"15min":4.3609999999999998},"last":1.681},{"interface":"front","average":{"1min":1.462,"5min":1.3819999999999999,"15min":1.5940000000000001},"min":{"1min":0.81299999999999994,"5min":0.77700000000000002,"15min":0.70499999999999996},"max":{"1min":1.972,"5min":2.8780000000000001,"15min":5.3840000000000003},"last":1.7170000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574955,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32732,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339108,"statfs":{"total":96636764160,"available":96603246592,"internally_reserved":0,"allocated":1581056,"data_stored":1034883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.873,"15min":0.81200000000000006},"min":{"1min":0.57499999999999996,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":1.137,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.94099999999999995,"15min":0.96799999999999997},"min":{"1min":0.52000000000000002,"5min":0.498,"15min":0.44800000000000001},"max":{"1min":1.2709999999999999,"5min":1.456,"15min":2.3340000000000001},"last":1.3200000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.89400000000000002,"15min":0.93600000000000005},"min":{"1min":0.57099999999999995,"5min":0.37,"15min":0.37},"max":{"1min":1.1859999999999999,"5min":1.714,"15min":1.762},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.91600000000000004,"15min":0.998},"min":{"1min":0.55500000000000005,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":1.175,"5min":1.8260000000000001,"15min":1.879},"last":1.107}]},{"osd":3,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.90700000000000003,"15min":0.93999999999999995},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.34200000000000003},"max":{"1min":1.268,"5min":1.728,"15min":1.728},"last":0.71999999999999997},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.1779999999999999,"15min":1.1850000000000001},"min":{"1min":0.54400000000000004,"5min":0.499,"15min":0.48499999999999999},"max":{"1min":1.544,"5min":1.9419999999999999,"15min":2.7679999999999998},"last":1.252}]},{"osd":4,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.095,"5min":1.135,"15min":1.1499999999999999},"min":{"1min":0.76300000000000001,"5min":0.67700000000000005,"15min":0.65300000000000002},"max":{"1min":1.524,"5min":1.9079999999999999,"15min":2.2559999999999998},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2729999999999999,"15min":1.29},"min":{"1min":0.96299999999999997,"5min":0.77800000000000002,"15min":0.59399999999999997},"max":{"1min":1.444,"5min":2.0659999999999998,"15min":2.867},"last":1.014}]},{"osd":5,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.244,"15min":1.2529999999999999},"min":{"1min":0.93799999999999994,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.5109999999999999,"5min":2.4169999999999998,"15min":2.7280000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.2050000000000001,"15min":1.2310000000000001},"min":{"1min":0.83199999999999996,"5min":0.73299999999999998,"15min":0.67600000000000005},"max":{"1min":1.45,"5min":2.8650000000000002,"15min":2.8650000000000002},"last":1.153}]},{"osd":6,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3640000000000001,"15min":1.3620000000000001},"min":{"1min":1.0469999999999999,"5min":0.79900000000000004,"15min":0.72499999999999998},"max":{"1min":1.6930000000000001,"5min":2.0489999999999999,"15min":2.6739999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.3140000000000001,"15min":1.3580000000000001},"min":{"1min":0.91500000000000004,"5min":0.72999999999999998,"15min":0.64200000000000002},"max":{"1min":1.5589999999999999,"5min":2.2160000000000002,"15min":2.8260000000000001},"last":1.2090000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:37:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.3979999999999999,"15min":1.4079999999999999},"min":{"1min":1.117,"5min":0.82199999999999995,"15min":0.78000000000000003},"max":{"1min":1.736,"5min":4.5670000000000002,"15min":4.5670000000000002},"last":1.385},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.349,"15min":1.3600000000000001},"min":{"1min":0.93000000000000005,"5min":0.75800000000000001,"15min":0.70199999999999996},"max":{"1min":1.6459999999999999,"5min":5.4450000000000003,"15min":5.4450000000000003},"last":1.071}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:37:37.880 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T16:37:37.881 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T16:37:38.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.147+0000 7f590a1b6640 1 Processor -- start 2023-11-08T16:37:38.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.148+0000 7f590a1b6640 1 -- start start 2023-11-08T16:37:38.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.149+0000 7f590a1b6640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f59040ae000 0x7f590406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.149+0000 7f590a1b6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f590406fe50 con 0x7f59040b0770 2023-11-08T16:37:38.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.149+0000 7f590a1b6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59040ca790 con 0x7f59040b0b50 2023-11-08T16:37:38.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.149+0000 7f590a1b6640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59040cb3c0 con 0x7f59040ae000 2023-11-08T16:37:38.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.149+0000 7f59037fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f59040b0770 0x7f59040cc980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53142/0 (socket says 172.21.15.81:53142) 2023-11-08T16:37:38.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.150+0000 7f59037fe640 1 -- 172.21.15.81:0/4192643547 learned_addr learned my addr 172.21.15.81:0/4192643547 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:38.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.150+0000 7f5902ffd640 1 --2- 172.21.15.81:0/4192643547 >> v2:172.21.15.93:3300/0 conn(0x7f59040ae000 0x7f590406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:38.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.150+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969067477 0 0) 0x7f58f4001110 con 0x7f59040b0770 2023-11-08T16:37:38.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.150+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58ec003520 con 0x7f59040b0770 2023-11-08T16:37:38.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.150+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 85225037 0 0) 0x7f58ec003520 con 0x7f59040b0770 2023-11-08T16:37:38.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.150+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 >> v2:172.21.15.93:3300/0 conn(0x7f59040ae000 msgr2=0x7f590406f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:38.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.150+0000 7f59027fc640 1 --2- 172.21.15.81:0/4192643547 >> v2:172.21.15.93:3300/0 conn(0x7f59040ae000 0x7f590406f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.150+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 >> v1:172.21.15.93:6789/0 conn(0x7f59040b0b50 legacy=0x7f59040b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.150+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59040cad70 con 0x7f59040b0770 2023-11-08T16:37:38.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.150+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f58f4001800 con 0x7f59040b0770 2023-11-08T16:37:38.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.150+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 >> v1:172.21.15.81:6789/0 conn(0x7f59040b0770 legacy=0x7f59040cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.151+0000 7f59027fc640 1 --2- 172.21.15.81:0/4192643547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58ec003970 0x7f58ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.151+0000 7f59027fc640 1 --2- 172.21.15.81:0/4192643547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58ec0047c0 0x7f58ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.151+0000 7f59027fc640 1 --2- 172.21.15.81:0/4192643547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58ec005180 0x7f58ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.151+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59040cb3c0 con 0x7f58ec003970 2023-11-08T16:37:38.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.151+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec00c020 con 0x7f58ec005180 2023-11-08T16:37:38.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.151+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec00c1e0 con 0x7f58ec0047c0 2023-11-08T16:37:38.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.151+0000 7f59037fe640 1 --2- 172.21.15.81:0/4192643547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58ec003970 0x7f58ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:38.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.152+0000 7f59037fe640 1 -- 172.21.15.81:0/4192643547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58ec0047c0 msgr2=0x7f58ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:38.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.152+0000 7f59037fe640 1 --2- 172.21.15.81:0/4192643547 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58ec0047c0 0x7f58ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.152+0000 7f59037fe640 1 -- 172.21.15.81:0/4192643547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58ec005180 msgr2=0x7f58ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:38.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.152+0000 7f59037fe640 1 --2- 172.21.15.81:0/4192643547 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58ec005180 0x7f58ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.152+0000 7f59037fe640 1 -- 172.21.15.81:0/4192643547 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec003520 con 0x7f58ec003970 2023-11-08T16:37:38.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.152+0000 7f5903fff640 1 -- 172.21.15.81:0/4192643547 reap_dead start 2023-11-08T16:37:38.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.153+0000 7f59037fe640 1 --2- 172.21.15.81:0/4192643547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58ec003970 0x7f58ec003d90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f58f400c190 tx=0x7f58ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d4860d5a079eff6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:38.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.153+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f4005ad0 con 0x7f58ec003970 2023-11-08T16:37:38.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.153+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f4005c50 con 0x7f58ec003970 2023-11-08T16:37:38.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.153+0000 7f59027fc640 1 -- 172.21.15.81:0/4192643547 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f402c7e0 con 0x7f58ec003970 2023-11-08T16:37:38.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.153+0000 7f590a1b6640 1 -- 172.21.15.81:0/4192643547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58ec003970 msgr2=0x7f58ec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.153+0000 7f590a1b6640 1 --2- 172.21.15.81:0/4192643547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58ec003970 0x7f58ec003d90 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f58f400c190 tx=0x7f58ec0057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.154+0000 7f590a1b6640 1 -- 172.21.15.81:0/4192643547 shutdown_connections 2023-11-08T16:37:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.154+0000 7f590a1b6640 1 --2- 172.21.15.81:0/4192643547 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58ec003970 0x7f58ec003d90 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.154+0000 7f590a1b6640 1 -- 172.21.15.81:0/4192643547 >> 172.21.15.81:0/4192643547 conn(0x7f59040cb500 msgr2=0x7f59040cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.154+0000 7f590a1b6640 1 -- 172.21.15.81:0/4192643547 shutdown_connections 2023-11-08T16:37:38.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.154+0000 7f590a1b6640 1 -- 172.21.15.81:0/4192643547 wait complete. 2023-11-08T16:37:38.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.155+0000 7f590a1b6640 1 Processor -- start 2023-11-08T16:37:38.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.156+0000 7f590a1b6640 1 -- start start 2023-11-08T16:37:38.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.156+0000 7f590a1b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58ec003970 0x7f5904150d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.156+0000 7f590a1b6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5904159c90 0x7f590415a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.156+0000 7f590a1b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59040ac8b0 0x7f59040acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.156+0000 7f590a1b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59040ad2a0 con 0x7f59040ac8b0 2023-11-08T16:37:38.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.156+0000 7f590a1b6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59041512c0 con 0x7f5904159c90 2023-11-08T16:37:38.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.156+0000 7f590a1b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590406c540 con 0x7f58ec003970 2023-11-08T16:37:38.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.157+0000 7f5903fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59040ac8b0 0x7f59040acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:38.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.157+0000 7f5903fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59040ac8b0 0x7f59040acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50428/0 (socket says 172.21.15.81:50428) 2023-11-08T16:37:38.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.157+0000 7f5903fff640 1 -- 172.21.15.81:0/3963786888 learned_addr learned my addr 172.21.15.81:0/3963786888 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:38.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.157+0000 7f5903fff640 1 -- 172.21.15.81:0/3963786888 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58ec003970 msgr2=0x7f5904150d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.157+0000 7f5903fff640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58ec003970 0x7f5904150d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.157+0000 7f5903fff640 1 -- 172.21.15.81:0/3963786888 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5904159c90 msgr2=0x7f590415a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.157+0000 7f5903fff640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5904159c90 0x7f590415a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.157+0000 7f5903fff640 1 -- 172.21.15.81:0/3963786888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f590406c820 con 0x7f59040ac8b0 2023-11-08T16:37:38.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.158+0000 7f5903fff640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59040ac8b0 0x7f59040acd60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f59040b0f30 tx=0x7f58fc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:38.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.158+0000 7f58ebfff640 1 -- 172.21.15.81:0/3963786888 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc007070 con 0x7f59040ac8b0 2023-11-08T16:37:38.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.158+0000 7f58ebfff640 1 -- 172.21.15.81:0/3963786888 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc0071d0 con 0x7f59040ac8b0 2023-11-08T16:37:38.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.158+0000 7f58ebfff640 1 -- 172.21.15.81:0/3963786888 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc007560 con 0x7f59040ac8b0 2023-11-08T16:37:38.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.158+0000 7f590a1b6640 1 -- 172.21.15.81:0/3963786888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590406cb00 con 0x7f59040ac8b0 2023-11-08T16:37:38.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.158+0000 7f590a1b6640 1 -- 172.21.15.81:0/3963786888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590415a760 con 0x7f59040ac8b0 2023-11-08T16:37:38.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.159+0000 7f590a1b6640 1 -- 172.21.15.81:0/3963786888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590415a980 con 0x7f59040ac8b0 2023-11-08T16:37:38.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.160+0000 7f58ebfff640 1 -- 172.21.15.81:0/3963786888 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58fc0076c0 con 0x7f59040ac8b0 2023-11-08T16:37:38.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.161+0000 7f58ebfff640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58f006aea0 0x7f58f006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.161+0000 7f58ebfff640 1 -- 172.21.15.81:0/3963786888 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58fc088af0 con 0x7f59040ac8b0 2023-11-08T16:37:38.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.161+0000 7f59037fe640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58f006aea0 0x7f58f006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:38.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.162+0000 7f59037fe640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58f006aea0 0x7f58f006d360 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f58f4034000 tx=0x7f58f4039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:38.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.165+0000 7f58ebfff640 1 -- 172.21.15.81:0/3963786888 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f590415a980 con 0x7f59040ac8b0 2023-11-08T16:37:38.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.424+0000 7f590a1b6640 1 -- 172.21.15.81:0/3963786888 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f590406d020 con 0x7f59040ac8b0 2023-11-08T16:37:38.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.430+0000 7f58ebfff640 1 -- 172.21.15.81:0/3963786888 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f590406d020 con 0x7f59040ac8b0 2023-11-08T16:37:38.432 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2023-11-08T16:37:38.432 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ== 2023-11-08T16:37:38.432 INFO:teuthology.orchestra.run.smithi081.stdout: caps mds = "allow" 2023-11-08T16:37:38.432 INFO:teuthology.orchestra.run.smithi081.stdout: caps mon = "allow r" 2023-11-08T16:37:38.433 INFO:teuthology.orchestra.run.smithi081.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T16:37:38.433 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2023-11-08T16:37:38.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.432+0000 7f590a1b6640 1 -- 172.21.15.81:0/3963786888 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58f006aea0 msgr2=0x7f58f006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.433+0000 7f590a1b6640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58f006aea0 0x7f58f006d360 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f58f4034000 tx=0x7f58f4039040 comp rx=0 tx=0).stop 2023-11-08T16:37:38.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.433+0000 7f590a1b6640 1 -- 172.21.15.81:0/3963786888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59040ac8b0 msgr2=0x7f59040acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.433+0000 7f590a1b6640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59040ac8b0 0x7f59040acd60 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f59040b0f30 tx=0x7f58fc00c960 comp rx=0 tx=0).stop 2023-11-08T16:37:38.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.433+0000 7f590a1b6640 1 -- 172.21.15.81:0/3963786888 shutdown_connections 2023-11-08T16:37:38.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.433+0000 7f590a1b6640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f58f006aea0 0x7f58f006d360 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.433+0000 7f590a1b6640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59040ac8b0 0x7f59040acd60 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.433+0000 7f590a1b6640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5904159c90 0x7f590415a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.433+0000 7f590a1b6640 1 --2- 172.21.15.81:0/3963786888 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58ec003970 0x7f5904150d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.433+0000 7f590a1b6640 1 -- 172.21.15.81:0/3963786888 >> 172.21.15.81:0/3963786888 conn(0x7f59040cb500 msgr2=0x7f590415df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:38.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.433+0000 7f590a1b6640 1 -- 172.21.15.81:0/3963786888 shutdown_connections 2023-11-08T16:37:38.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.433+0000 7f590a1b6640 1 -- 172.21.15.81:0/3963786888 wait complete. 2023-11-08T16:37:38.452 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:37:38.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.653+0000 7fee3a47e640 1 Processor -- start 2023-11-08T16:37:38.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.653+0000 7fee3a47e640 1 -- start start 2023-11-08T16:37:38.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.655+0000 7fee3a47e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fee3415aac0 0x7fee340711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.655+0000 7fee3a47e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fee340716e0 0x7fee340ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.655+0000 7fee3a47e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee340cf250 0x7fee3406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.655+0000 7fee3a47e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3406f650 con 0x7fee3415aac0 2023-11-08T16:37:38.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.655+0000 7fee3a47e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee340cf6c0 con 0x7fee340716e0 2023-11-08T16:37:38.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.655+0000 7fee3a47e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee34070470 con 0x7fee340cf250 2023-11-08T16:37:38.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.655+0000 7fee3947c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fee3415aac0 0x7fee340711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:38.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.655+0000 7fee3947c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fee3415aac0 0x7fee340711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50434/0 (socket says 172.21.15.81:50434) 2023-11-08T16:37:38.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.655+0000 7fee3947c640 1 -- 172.21.15.81:0/3335266651 learned_addr learned my addr 172.21.15.81:0/3335266651 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:38.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.656+0000 7fee38c7b640 1 --2- 172.21.15.81:0/3335266651 >> v2:172.21.15.93:3300/0 conn(0x7fee340716e0 0x7fee340ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:38.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.656+0000 7fee39c7d640 1 --2- 172.21.15.81:0/3335266651 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee340cf250 0x7fee3406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:38.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.656+0000 7fee3947c640 1 -- 172.21.15.81:0/3335266651 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee340cf250 msgr2=0x7fee3406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:38.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.656+0000 7fee3947c640 1 --2- 172.21.15.81:0/3335266651 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee340cf250 0x7fee3406b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.656+0000 7fee3947c640 1 -- 172.21.15.81:0/3335266651 >> v2:172.21.15.93:3300/0 conn(0x7fee340716e0 msgr2=0x7fee340ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:38.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.656+0000 7fee3947c640 1 --2- 172.21.15.81:0/3335266651 >> v2:172.21.15.93:3300/0 conn(0x7fee340716e0 0x7fee340ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.656+0000 7fee3947c640 1 -- 172.21.15.81:0/3335266651 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3406d580 con 0x7fee3415aac0 2023-11-08T16:37:38.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.656+0000 7fee38c7b640 1 --2- 172.21.15.81:0/3335266651 >> v2:172.21.15.93:3300/0 conn(0x7fee340716e0 0x7fee340ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:37:38.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.658+0000 7fee3947c640 1 --2- 172.21.15.81:0/3335266651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3415aac0 0x7fee340711a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fee3415e790 tx=0x7fee3002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=353429d7927e18aa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:38.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.658+0000 7fee2bfff640 1 -- 172.21.15.81:0/3335266651 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30031870 con 0x7fee3415aac0 2023-11-08T16:37:38.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.658+0000 7fee2bfff640 1 -- 172.21.15.81:0/3335266651 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee300319d0 con 0x7fee3415aac0 2023-11-08T16:37:38.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.658+0000 7fee2bfff640 1 -- 172.21.15.81:0/3335266651 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30031d60 con 0x7fee3415aac0 2023-11-08T16:37:38.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.659+0000 7fee3a47e640 1 -- 172.21.15.81:0/3335266651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3415aac0 msgr2=0x7fee340711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.659+0000 7fee3a47e640 1 --2- 172.21.15.81:0/3335266651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3415aac0 0x7fee340711a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fee3415e790 tx=0x7fee3002b910 comp rx=0 tx=0).stop 2023-11-08T16:37:38.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.659+0000 7fee3a47e640 1 -- 172.21.15.81:0/3335266651 shutdown_connections 2023-11-08T16:37:38.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.659+0000 7fee3a47e640 1 --2- 172.21.15.81:0/3335266651 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee340cf250 0x7fee3406b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.659+0000 7fee3a47e640 1 --2- 172.21.15.81:0/3335266651 >> v2:172.21.15.93:3300/0 conn(0x7fee340716e0 0x7fee340ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.659+0000 7fee3a47e640 1 --2- 172.21.15.81:0/3335266651 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee3415aac0 0x7fee340711a0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.659+0000 7fee3a47e640 1 -- 172.21.15.81:0/3335266651 >> 172.21.15.81:0/3335266651 conn(0x7fee3406a4a0 msgr2=0x7fee3406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:38.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.659+0000 7fee3a47e640 1 -- 172.21.15.81:0/3335266651 shutdown_connections 2023-11-08T16:37:38.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.659+0000 7fee3a47e640 1 -- 172.21.15.81:0/3335266651 wait complete. 2023-11-08T16:37:38.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.661+0000 7fee3a47e640 1 Processor -- start 2023-11-08T16:37:38.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.661+0000 7fee3a47e640 1 -- start start 2023-11-08T16:37:38.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.662+0000 7fee3a47e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee340716e0 0x7fee34151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.662+0000 7fee3a47e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee340cf250 0x7fee341530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.662+0000 7fee3a47e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee3415aac0 0x7fee3415bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.662+0000 7fee3a47e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3414d980 con 0x7fee340716e0 2023-11-08T16:37:38.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.662+0000 7fee3a47e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee341538f0 con 0x7fee340cf250 2023-11-08T16:37:38.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.662+0000 7fee3a47e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee34153bd0 con 0x7fee3415aac0 2023-11-08T16:37:38.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.662+0000 7fee3947c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee340716e0 0x7fee34151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:38.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.662+0000 7fee3947c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee340716e0 0x7fee34151170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50446/0 (socket says 172.21.15.81:50446) 2023-11-08T16:37:38.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.662+0000 7fee3947c640 1 -- 172.21.15.81:0/4202275213 learned_addr learned my addr 172.21.15.81:0/4202275213 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:38.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.663+0000 7fee3947c640 1 -- 172.21.15.81:0/4202275213 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee3415aac0 msgr2=0x7fee3415bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.663+0000 7fee3947c640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee3415aac0 0x7fee3415bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.663+0000 7fee3947c640 1 -- 172.21.15.81:0/4202275213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee340cf250 msgr2=0x7fee341530c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.663+0000 7fee3947c640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee340cf250 0x7fee341530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.663+0000 7fee3947c640 1 -- 172.21.15.81:0/4202275213 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee300094d0 con 0x7fee340716e0 2023-11-08T16:37:38.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.663+0000 7fee3947c640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee340716e0 0x7fee34151170 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fee3002b950 tx=0x7fee300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:38.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.663+0000 7fee29ffb640 1 -- 172.21.15.81:0/4202275213 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3003ad30 con 0x7fee340716e0 2023-11-08T16:37:38.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.663+0000 7fee29ffb640 1 -- 172.21.15.81:0/4202275213 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee300443f0 con 0x7fee340716e0 2023-11-08T16:37:38.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.663+0000 7fee3a47e640 1 -- 172.21.15.81:0/4202275213 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee34153eb0 con 0x7fee340716e0 2023-11-08T16:37:38.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.663+0000 7fee29ffb640 1 -- 172.21.15.81:0/4202275213 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee300447a0 con 0x7fee340716e0 2023-11-08T16:37:38.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.663+0000 7fee3a47e640 1 -- 172.21.15.81:0/4202275213 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee34154470 con 0x7fee340716e0 2023-11-08T16:37:38.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.664+0000 7fee3a47e640 1 -- 172.21.15.81:0/4202275213 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee34154660 con 0x7fee340716e0 2023-11-08T16:37:38.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.665+0000 7fee29ffb640 1 -- 172.21.15.81:0/4202275213 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fee3004d450 con 0x7fee340716e0 2023-11-08T16:37:38.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.666+0000 7fee29ffb640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee1806ae70 0x7fee1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:38.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.666+0000 7fee29ffb640 1 -- 172.21.15.81:0/4202275213 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee3002d040 con 0x7fee340716e0 2023-11-08T16:37:38.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.667+0000 7fee38c7b640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee1806ae70 0x7fee1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:38.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.667+0000 7fee38c7b640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee1806ae70 0x7fee1806d330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fee24006e80 tx=0x7fee24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:38.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.671+0000 7fee29ffb640 1 -- 172.21.15.81:0/4202275213 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fee30076c30 con 0x7fee340716e0 2023-11-08T16:37:38.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.893+0000 7fee3a47e640 1 -- 172.21.15.81:0/4202275213 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fee34067800 con 0x7fee340716e0 2023-11-08T16:37:38.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.894+0000 7fee29ffb640 1 -- 172.21.15.81:0/4202275213 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fee30035070 con 0x7fee340716e0 2023-11-08T16:37:38.895 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:38.896 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":64,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/2125403666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2125403666},{"type":"v1","addr":"172.21.15.93:6834","nonce":2125403666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2727356524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2727356524},{"type":"v1","addr":"172.21.15.93:6835","nonce":2727356524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6936,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2639654743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2639654743},{"type":"v1","addr":"172.21.15.81:6835","nonce":2639654743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:32.323261+0000","modified":"2023-11-08T16:37:35.475263+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6942":{"gid":6942,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/2162767469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2162767469},{"type":"v1","addr":"172.21.15.81:6837","nonce":2162767469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:37:38.896 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 64 2023-11-08T16:37:38.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.896+0000 7fee3a47e640 1 -- 172.21.15.81:0/4202275213 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee1806ae70 msgr2=0x7fee1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.896+0000 7fee3a47e640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee1806ae70 0x7fee1806d330 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fee24006e80 tx=0x7fee24008040 comp rx=0 tx=0).stop 2023-11-08T16:37:38.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.896+0000 7fee3a47e640 1 -- 172.21.15.81:0/4202275213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee340716e0 msgr2=0x7fee34151170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:38.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.896+0000 7fee3a47e640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee340716e0 0x7fee34151170 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fee3002b950 tx=0x7fee300046c0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.896+0000 7fee3a47e640 1 -- 172.21.15.81:0/4202275213 shutdown_connections 2023-11-08T16:37:38.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.896+0000 7fee3a47e640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fee1806ae70 0x7fee1806d330 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.896+0000 7fee3a47e640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fee3415aac0 0x7fee3415bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.896+0000 7fee3a47e640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fee340cf250 0x7fee341530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.896+0000 7fee3a47e640 1 --2- 172.21.15.81:0/4202275213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fee340716e0 0x7fee34151170 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:38.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.896+0000 7fee3a47e640 1 -- 172.21.15.81:0/4202275213 >> 172.21.15.81:0/4202275213 conn(0x7fee3406a4a0 msgr2=0x7fee34156690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:38.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.897+0000 7fee3a47e640 1 -- 172.21.15.81:0/4202275213 shutdown_connections 2023-11-08T16:37:38.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:38.897+0000 7fee3a47e640 1 -- 172.21.15.81:0/4202275213 wait complete. 2023-11-08T16:37:39.320 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:37:32.323261+0000', 'modified': '2023-11-08T16:37:35.475263+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6942}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6942': {'gid': 6942, 'name': 'a', 'rank': 0, 'incarnation': 60, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.81:6837/2162767469', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 2162767469}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 2162767469}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:37:39.320 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:37:39.321 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-08T16:37:39.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.539+0000 7f086d92e640 1 Processor -- start 2023-11-08T16:37:39.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.539+0000 7f086d92e640 1 -- start start 2023-11-08T16:37:39.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.541+0000 7f086d92e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f086815ac20 0x7f08680cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.541+0000 7f086d92e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f086806f7c0 0x7f0868070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.541+0000 7f086d92e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086806cd70 con 0x7f08680ad940 2023-11-08T16:37:39.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.541+0000 7f086d92e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08680ae130 con 0x7f086806f7c0 2023-11-08T16:37:39.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.541+0000 7f086d92e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680cb370 con 0x7f086815ac20 2023-11-08T16:37:39.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.541+0000 7f086d12d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f08680ad940 0x7f086806b390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:53154/0 (socket says 172.21.15.81:53154) 2023-11-08T16:37:39.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.541+0000 7f086d12d640 1 -- 172.21.15.81:0/520778443 learned_addr learned my addr 172.21.15.81:0/520778443 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:39.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.541+0000 7f0867fff640 1 --2- 172.21.15.81:0/520778443 >> v2:172.21.15.93:3300/0 conn(0x7f086806f7c0 0x7f0868070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.541+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294306551 0 0) 0x7f086806cd70 con 0x7f08680ad940 2023-11-08T16:37:39.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f086c92c640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f086815ac20 0x7f08680cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0858003520 con 0x7f08680ad940 2023-11-08T16:37:39.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3145630772 0 0) 0x7f0858003520 con 0x7f08680ad940 2023-11-08T16:37:39.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f086815ac20 msgr2=0x7f08680cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f086815ac20 0x7f08680cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 >> v2:172.21.15.93:3300/0 conn(0x7f086806f7c0 msgr2=0x7f0868070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 --2- 172.21.15.81:0/520778443 >> v2:172.21.15.93:3300/0 conn(0x7f086806f7c0 0x7f0868070f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680ca500 con 0x7f08680ad940 2023-11-08T16:37:39.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f08600019f0 con 0x7f08680ad940 2023-11-08T16:37:39.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f0867fff640 1 --2- 172.21.15.81:0/520778443 >> v2:172.21.15.93:3300/0 conn(0x7f086806f7c0 0x7f0868070f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:37:39.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 >> v1:172.21.15.81:6789/0 conn(0x7f08680ad940 legacy=0x7f086806b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f086815ac20 0x7f0858003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0858003f40 0x7f08580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0858004d00 0x7f0858005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680ae130 con 0x7f0858003f40 2023-11-08T16:37:39.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680cb370 con 0x7f0858004d00 2023-11-08T16:37:39.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085800bae0 con 0x7f086815ac20 2023-11-08T16:37:39.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.542+0000 7f0867fff640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0858003f40 0x7f08580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.543+0000 7f086c92c640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f086815ac20 0x7f0858003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.543+0000 7f0867fff640 1 -- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f086815ac20 msgr2=0x7f0858003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.543+0000 7f0867fff640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f086815ac20 0x7f0858003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.543+0000 7f086d12d640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0858004d00 0x7f0858005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.543+0000 7f0867fff640 1 -- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0858004d00 msgr2=0x7f0858005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.543+0000 7f0867fff640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0858004d00 0x7f0858005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.543+0000 7f0867fff640 1 -- 172.21.15.81:0/520778443 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858003520 con 0x7f0858003f40 2023-11-08T16:37:39.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.543+0000 7f086c92c640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f086815ac20 0x7f0858003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:39.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.544+0000 7f0867fff640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0858003f40 0x7f08580047c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f08540098a0 tx=0x7f08580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f99c491520aaf38 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:39.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.544+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854004230 con 0x7f0858003f40 2023-11-08T16:37:39.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.544+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0854004390 con 0x7f0858003f40 2023-11-08T16:37:39.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.544+0000 7f08677fe640 1 -- 172.21.15.81:0/520778443 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854003830 con 0x7f0858003f40 2023-11-08T16:37:39.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.545+0000 7f086d92e640 1 -- 172.21.15.81:0/520778443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0858003f40 msgr2=0x7f08580047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.545+0000 7f086d92e640 1 --2- 172.21.15.81:0/520778443 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0858003f40 0x7f08580047c0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f08540098a0 tx=0x7f08580057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.545+0000 7f086d12d640 1 -- 172.21.15.81:0/520778443 reap_dead start 2023-11-08T16:37:39.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.545+0000 7f086d92e640 1 -- 172.21.15.81:0/520778443 shutdown_connections 2023-11-08T16:37:39.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.545+0000 7f086d92e640 1 -- 172.21.15.81:0/520778443 >> 172.21.15.81:0/520778443 conn(0x7f08680b04d0 msgr2=0x7f08680b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:39.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.545+0000 7f086d92e640 1 -- 172.21.15.81:0/520778443 shutdown_connections 2023-11-08T16:37:39.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.545+0000 7f086d92e640 1 -- 172.21.15.81:0/520778443 wait complete. 2023-11-08T16:37:39.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.547+0000 7f086d92e640 1 Processor -- start 2023-11-08T16:37:39.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.547+0000 7f086d92e640 1 -- start start 2023-11-08T16:37:39.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d92e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0858003f40 0x7f08680c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d92e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08681568b0 0x7f08680acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d92e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0868155d20 0x7f0868152780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d92e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680c6980 con 0x7f0868155d20 2023-11-08T16:37:39.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d92e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0868154ff0 con 0x7f0858003f40 2023-11-08T16:37:39.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d92e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0868153630 con 0x7f08681568b0 2023-11-08T16:37:39.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d12d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0868155d20 0x7f0868152780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d12d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0868155d20 0x7f0868152780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50452/0 (socket says 172.21.15.81:50452) 2023-11-08T16:37:39.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d12d640 1 -- 172.21.15.81:0/3134547510 learned_addr learned my addr 172.21.15.81:0/3134547510 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:39.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d12d640 1 -- 172.21.15.81:0/3134547510 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08681568b0 msgr2=0x7f08680acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d12d640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08681568b0 0x7f08680acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d12d640 1 -- 172.21.15.81:0/3134547510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0858003f40 msgr2=0x7f08680c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d12d640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0858003f40 0x7f08680c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.548+0000 7f086d12d640 1 -- 172.21.15.81:0/3134547510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0868156e50 con 0x7f0868155d20 2023-11-08T16:37:39.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.549+0000 7f086d12d640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0868155d20 0x7f0868152780 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f085800bc70 tx=0x7f086000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:39.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.549+0000 7f08657fa640 1 -- 172.21.15.81:0/3134547510 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086000b860 con 0x7f0868155d20 2023-11-08T16:37:39.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.549+0000 7f08657fa640 1 -- 172.21.15.81:0/3134547510 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f086000b9c0 con 0x7f0868155d20 2023-11-08T16:37:39.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.549+0000 7f08657fa640 1 -- 172.21.15.81:0/3134547510 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f086000bd70 con 0x7f0868155d20 2023-11-08T16:37:39.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.549+0000 7f086d92e640 1 -- 172.21.15.81:0/3134547510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0868155240 con 0x7f0868155d20 2023-11-08T16:37:39.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.549+0000 7f086d92e640 1 -- 172.21.15.81:0/3134547510 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0868155800 con 0x7f0868155d20 2023-11-08T16:37:39.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.551+0000 7f08657fa640 1 -- 172.21.15.81:0/3134547510 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0860007070 con 0x7f0868155d20 2023-11-08T16:37:39.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.552+0000 7f08657fa640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f084c06aea0 0x7f084c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.552+0000 7f08657fa640 1 -- 172.21.15.81:0/3134547510 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0860089930 con 0x7f0868155d20 2023-11-08T16:37:39.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.552+0000 7f086c92c640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f084c06aea0 0x7f084c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.553+0000 7f08717fe740 1 Processor -- start 2023-11-08T16:37:39.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.553+0000 7f08717fe740 1 -- start start 2023-11-08T16:37:39.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.553+0000 7f086c92c640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f084c06aea0 0x7f084c06d360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f08681e7230 tx=0x7f085c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:39.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f08717fe740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9080 0x55cadc0abf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f08717fe740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0d9460 0x55cadc0dbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f08717fe740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0ac4c0 0x55cadc0dc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f08717fe740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cadc01e7e0 con 0x55cadc0d9080 2023-11-08T16:37:39.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f08717fe740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cadc01bed0 con 0x55cadc0d9460 2023-11-08T16:37:39.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f08717fe740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cadc0177c0 con 0x55cadc0ac4c0 2023-11-08T16:37:39.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f086c92c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9080 0x55cadc0abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f086c92c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9080 0x55cadc0abf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50456/0 (socket says 172.21.15.81:50456) 2023-11-08T16:37:39.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f086c92c640 1 -- 172.21.15.81:0/4043471678 learned_addr learned my addr 172.21.15.81:0/4043471678 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:39.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f086c92c640 1 -- 172.21.15.81:0/4043471678 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0ac4c0 msgr2=0x55cadc0dc350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f086c92c640 1 --2- 172.21.15.81:0/4043471678 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0ac4c0 0x55cadc0dc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f086c92c640 1 -- 172.21.15.81:0/4043471678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0d9460 msgr2=0x55cadc0dbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f086c92c640 1 --2- 172.21.15.81:0/4043471678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0d9460 0x55cadc0dbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.554+0000 7f086c92c640 1 -- 172.21.15.81:0/4043471678 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cadbfeab10 con 0x55cadc0d9080 2023-11-08T16:37:39.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.555+0000 7f086c92c640 1 --2- 172.21.15.81:0/4043471678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9080 0x55cadc0abf80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f085c0109e0 tx=0x7f085c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f1b43c1f1c97a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:39.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.555+0000 7f08527fc640 1 -- 172.21.15.81:0/4043471678 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c015d80 con 0x55cadc0d9080 2023-11-08T16:37:39.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.555+0000 7f08527fc640 1 -- 172.21.15.81:0/4043471678 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c01a070 con 0x55cadc0d9080 2023-11-08T16:37:39.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.555+0000 7f08527fc640 1 -- 172.21.15.81:0/4043471678 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c021640 con 0x55cadc0d9080 2023-11-08T16:37:39.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.555+0000 7f08717fe740 1 -- 172.21.15.81:0/4043471678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9080 msgr2=0x55cadc0abf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.555+0000 7f08717fe740 1 --2- 172.21.15.81:0/4043471678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9080 0x55cadc0abf80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f085c0109e0 tx=0x7f085c013000 comp rx=0 tx=0).stop 2023-11-08T16:37:39.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.556+0000 7f08717fe740 1 -- 172.21.15.81:0/4043471678 shutdown_connections 2023-11-08T16:37:39.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.556+0000 7f08717fe740 1 --2- 172.21.15.81:0/4043471678 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0ac4c0 0x55cadc0dc350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.556+0000 7f08717fe740 1 --2- 172.21.15.81:0/4043471678 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0d9460 0x55cadc0dbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.556+0000 7f08717fe740 1 --2- 172.21.15.81:0/4043471678 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9080 0x55cadc0abf80 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.556+0000 7f08717fe740 1 -- 172.21.15.81:0/4043471678 >> 172.21.15.81:0/4043471678 conn(0x55cadc0ae290 msgr2=0x55cadc0f6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:39.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.556+0000 7f08717fe740 1 -- 172.21.15.81:0/4043471678 shutdown_connections 2023-11-08T16:37:39.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.556+0000 7f08717fe740 1 -- 172.21.15.81:0/4043471678 wait complete. 2023-11-08T16:37:39.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.556+0000 7f08717fe740 1 -- start start 2023-11-08T16:37:39.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.557+0000 7f08717fe740 1 Processor -- start 2023-11-08T16:37:39.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.557+0000 7f08717fe740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0d9080 0x55cadc0c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.557+0000 7f08717fe740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9460 0x55cadc0dec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f08717fe740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc0f1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f08717fe740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cadbfb4c40 con 0x55cadc0df150 2023-11-08T16:37:39.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f08717fe740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cadbfb48e0 con 0x55cadc0d9080 2023-11-08T16:37:39.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f08717fe740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cadbd7af20 con 0x55cadc0d9460 2023-11-08T16:37:39.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f08717fe740 10 client.0 _ll_register_callbacks cb 0x55cadbfd6cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:37:39.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f086d12d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc0f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f086d12d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc0f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50460/0 (socket says 172.21.15.81:50460) 2023-11-08T16:37:39.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f086d12d640 1 -- 172.21.15.81:0/3249014694 learned_addr learned my addr 172.21.15.81:0/3249014694 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:39.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f086d12d640 1 -- 172.21.15.81:0/3249014694 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9460 msgr2=0x55cadc0dec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f086d12d640 1 --2- 172.21.15.81:0/3249014694 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9460 0x55cadc0dec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f086d12d640 1 -- 172.21.15.81:0/3249014694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0d9080 msgr2=0x55cadc0c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f086d12d640 1 --2- 172.21.15.81:0/3249014694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0d9080 0x55cadc0c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f086d12d640 1 -- 172.21.15.81:0/3249014694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f085c00e050 con 0x55cadc0df150 2023-11-08T16:37:39.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f086d12d640 1 --2- 172.21.15.81:0/3249014694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc0f1440 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0860058300 tx=0x7f0860058d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e22fd4f1dbe4ea4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:39.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.558+0000 7f08477fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:37:39.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08477fe640 1 -- 172.21.15.81:0/3249014694 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0860057cf0 con 0x55cadc0df150 2023-11-08T16:37:39.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08477fe640 1 -- 172.21.15.81:0/3249014694 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0860060900 con 0x55cadc0df150 2023-11-08T16:37:39.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08717fe740 1 -- 172.21.15.81:0/3249014694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cadbfebc80 con 0x55cadc0df150 2023-11-08T16:37:39.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08477fe640 1 -- 172.21.15.81:0/3249014694 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0860060470 con 0x55cadc0df150 2023-11-08T16:37:39.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08717fe740 10 client.7068 fetch_fsmap learned FSMap version 64 2023-11-08T16:37:39.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08717fe740 1 -- 172.21.15.81:0/3249014694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55cadc015e40 con 0x55cadc0df150 2023-11-08T16:37:39.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08477fe640 1 -- 172.21.15.81:0/3249014694 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cadbfebc80 con 0x55cadc0df150 2023-11-08T16:37:39.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08477fe640 1 -- 172.21.15.81:0/3249014694 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f08600543f0 con 0x55cadc0df150 2023-11-08T16:37:39.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08717fe740 10 client.7068 fetch_fsmap finished waiting for FSMap version 64 2023-11-08T16:37:39.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08717fe740 10 client.7068 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T16:37:39.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08717fe740 20 client.7068 populate_metadata read hostname 'smithi081' 2023-11-08T16:37:39.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08717fe740 1 --2- 172.21.15.81:0/3249014694 >> [v2:172.21.15.81:6836/2162767469,v1:172.21.15.81:6837/2162767469] conn(0x55cadc0c97a0 0x55cadc0c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08717fe740 4 client.7068 mds_command: new command op to 6942 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:37:39.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.559+0000 7f08717fe740 1 -- 172.21.15.81:0/3249014694 --> [v2:172.21.15.81:6836/2162767469,v1:172.21.15.81:6837/2162767469] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cadc0107a0 con 0x55cadc0c97a0 2023-11-08T16:37:39.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.560+0000 7f086c92c640 1 --2- 172.21.15.81:0/3249014694 >> [v2:172.21.15.81:6836/2162767469,v1:172.21.15.81:6837/2162767469] conn(0x55cadc0c97a0 0x55cadc0c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.560+0000 7f086c92c640 1 --2- 172.21.15.81:0/3249014694 >> [v2:172.21.15.81:6836/2162767469,v1:172.21.15.81:6837/2162767469] conn(0x55cadc0c97a0 0x55cadc0c9b80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dca0c2f80a40b67c server_cookie=9a914ad4874cc904 in_seq=0 out_seq=0 2023-11-08T16:37:39.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.560+0000 7f08477fe640 10 client.7068 ms_handle_connect on v2:172.21.15.81:6836/2162767469 2023-11-08T16:37:39.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08477fe640 1 -- 172.21.15.81:0/3249014694 <== mds.0 v2:172.21.15.81:6836/2162767469 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cadc0107a0 con 0x55cadc0c97a0 2023-11-08T16:37:39.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08477fe640 10 client.7068 handle_command_reply: tid=0 2023-11-08T16:37:39.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 1 client.7068 shutdown 2023-11-08T16:37:39.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 1 -- 172.21.15.81:0/3249014694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 msgr2=0x55cadc0f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 1 --2- 172.21.15.81:0/3249014694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc0f1440 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f0860058300 tx=0x7f0860058d00 comp rx=0 tx=0).stop 2023-11-08T16:37:39.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 1 -- 172.21.15.81:0/3249014694 shutdown_connections 2023-11-08T16:37:39.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 1 --2- 172.21.15.81:0/3249014694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc0f1440 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 1 --2- 172.21.15.81:0/3249014694 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9460 0x55cadc0dec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 1 --2- 172.21.15.81:0/3249014694 >> [v2:172.21.15.81:6836/2162767469,v1:172.21.15.81:6837/2162767469] conn(0x55cadc0c97a0 0x55cadc0c9b80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 1 --2- 172.21.15.81:0/3249014694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0d9080 0x55cadc0c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 1 -- 172.21.15.81:0/3249014694 >> 172.21.15.81:0/3249014694 conn(0x55cadc0ac4c0 msgr2=0x55cadc0aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:39.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08477fe640 0 client.7068 ms_handle_reset on v2:172.21.15.81:6836/2162767469 2023-11-08T16:37:39.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 1 -- 172.21.15.81:0/3249014694 shutdown_connections 2023-11-08T16:37:39.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 1 -- 172.21.15.81:0/3249014694 wait complete. 2023-11-08T16:37:39.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.561+0000 7f08717fe740 20 client.7068 trim_cache size 0 max 16384 2023-11-08T16:37:39.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.578+0000 7f08717fe740 1 Processor -- start 2023-11-08T16:37:39.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.578+0000 7f08717fe740 1 -- start start 2023-11-08T16:37:39.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.579+0000 7f08717fe740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9080 0x55cadc0eda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.579+0000 7f08717fe740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9460 0x55cadc0edf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.579+0000 7f08717fe740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0df150 0x55cadc0db850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.579+0000 7f08717fe740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cadc0d8830 con 0x55cadc0d9460 2023-11-08T16:37:39.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.579+0000 7f08717fe740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cadc0abd00 con 0x55cadc0df150 2023-11-08T16:37:39.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.579+0000 7f08717fe740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cadbfccd00 con 0x55cadc0d9080 2023-11-08T16:37:39.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.579+0000 7f0867fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9460 0x55cadc0edf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.579+0000 7f0867fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9460 0x55cadc0edf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50476/0 (socket says 172.21.15.81:50476) 2023-11-08T16:37:39.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.579+0000 7f0867fff640 1 -- 172.21.15.81:0/879290020 learned_addr learned my addr 172.21.15.81:0/879290020 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:39.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f086c92c640 1 --2- 172.21.15.81:0/879290020 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9080 0x55cadc0eda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f086d12d640 1 --2- 172.21.15.81:0/879290020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0df150 0x55cadc0db850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f0867fff640 1 -- 172.21.15.81:0/879290020 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9080 msgr2=0x55cadc0eda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f0867fff640 1 --2- 172.21.15.81:0/879290020 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9080 0x55cadc0eda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f0867fff640 1 -- 172.21.15.81:0/879290020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0df150 msgr2=0x55cadc0db850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f0867fff640 1 --2- 172.21.15.81:0/879290020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0df150 0x55cadc0db850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f0867fff640 1 -- 172.21.15.81:0/879290020 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cadbfb3b90 con 0x55cadc0d9460 2023-11-08T16:37:39.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f086c92c640 1 --2- 172.21.15.81:0/879290020 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9080 0x55cadc0eda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:39.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f0867fff640 1 --2- 172.21.15.81:0/879290020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9460 0x55cadc0edf40 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f08540099c0 tx=0x7f0854004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28521a7bab4986db server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:39.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f0852ffd640 1 -- 172.21.15.81:0/879290020 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854004230 con 0x55cadc0d9460 2023-11-08T16:37:39.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f0852ffd640 1 -- 172.21.15.81:0/879290020 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0854003610 con 0x55cadc0d9460 2023-11-08T16:37:39.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.580+0000 7f0852ffd640 1 -- 172.21.15.81:0/879290020 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08540039a0 con 0x55cadc0d9460 2023-11-08T16:37:39.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.581+0000 7f08717fe740 1 -- 172.21.15.81:0/879290020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9460 msgr2=0x55cadc0edf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.581+0000 7f08717fe740 1 --2- 172.21.15.81:0/879290020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9460 0x55cadc0edf40 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f08540099c0 tx=0x7f0854004730 comp rx=0 tx=0).stop 2023-11-08T16:37:39.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.581+0000 7f08717fe740 1 -- 172.21.15.81:0/879290020 shutdown_connections 2023-11-08T16:37:39.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.581+0000 7f08717fe740 1 --2- 172.21.15.81:0/879290020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc0df150 0x55cadc0db850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.581+0000 7f08717fe740 1 --2- 172.21.15.81:0/879290020 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0d9460 0x55cadc0edf40 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.581+0000 7f08717fe740 1 --2- 172.21.15.81:0/879290020 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9080 0x55cadc0eda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.581+0000 7f08717fe740 1 -- 172.21.15.81:0/879290020 >> 172.21.15.81:0/879290020 conn(0x55cadc0ac4c0 msgr2=0x55cadc0aae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:39.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.581+0000 7f08717fe740 1 -- 172.21.15.81:0/879290020 shutdown_connections 2023-11-08T16:37:39.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.581+0000 7f08717fe740 1 -- 172.21.15.81:0/879290020 wait complete. 2023-11-08T16:37:39.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.582+0000 7f08717fe740 1 -- start start 2023-11-08T16:37:39.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.582+0000 7f08717fe740 1 Processor -- start 2023-11-08T16:37:39.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f08717fe740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9080 0x55cadc0c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f08717fe740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc161420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f08717fe740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc110c60 0x55cadc1657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f08717fe740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cadc1653b0 con 0x55cadc0df150 2023-11-08T16:37:39.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f08717fe740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cadc0c59d0 con 0x55cadc110c60 2023-11-08T16:37:39.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f08717fe740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cadc0dc340 con 0x55cadc0d9080 2023-11-08T16:37:39.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f08717fe740 10 client.0 _ll_register_callbacks cb 0x7f0860001cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:37:39.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f0867fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc161420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f0867fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc161420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50490/0 (socket says 172.21.15.81:50490) 2023-11-08T16:37:39.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f0867fff640 1 -- 172.21.15.81:0/2382357991 learned_addr learned my addr 172.21.15.81:0/2382357991 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:39.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f0867fff640 1 -- 172.21.15.81:0/2382357991 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9080 msgr2=0x55cadc0c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f0867fff640 1 --2- 172.21.15.81:0/2382357991 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9080 0x55cadc0c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f0867fff640 1 -- 172.21.15.81:0/2382357991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc110c60 msgr2=0x55cadc1657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f0867fff640 1 --2- 172.21.15.81:0/2382357991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc110c60 0x55cadc1657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.583+0000 7f0867fff640 1 -- 172.21.15.81:0/2382357991 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0858003520 con 0x55cadc0df150 2023-11-08T16:37:39.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.584+0000 7f0867fff640 1 --2- 172.21.15.81:0/2382357991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc161420 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f0854045040 tx=0x7f085402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b071d4cdc0cfafba server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:39.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.584+0000 7f08477fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:37:39.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.584+0000 7f08477fe640 1 -- 172.21.15.81:0/2382357991 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854042cb0 con 0x55cadc0df150 2023-11-08T16:37:39.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.584+0000 7f08477fe640 1 -- 172.21.15.81:0/2382357991 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0854042e10 con 0x55cadc0df150 2023-11-08T16:37:39.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.584+0000 7f08717fe740 1 -- 172.21.15.81:0/2382357991 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f085c01a070 con 0x55cadc0df150 2023-11-08T16:37:39.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.584+0000 7f08477fe640 1 -- 172.21.15.81:0/2382357991 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f085404d420 con 0x55cadc0df150 2023-11-08T16:37:39.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.584+0000 7f08717fe740 10 client.7074 fetch_fsmap learned FSMap version 64 2023-11-08T16:37:39.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.584+0000 7f08717fe740 1 -- 172.21.15.81:0/2382357991 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55cadbfd9180 con 0x55cadc0df150 2023-11-08T16:37:39.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.584+0000 7f08477fe640 1 -- 172.21.15.81:0/2382357991 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c01a070 con 0x55cadc0df150 2023-11-08T16:37:39.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.584+0000 7f08477fe640 1 -- 172.21.15.81:0/2382357991 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 64) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f085404d580 con 0x55cadc0df150 2023-11-08T16:37:39.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.585+0000 7f08717fe740 10 client.7074 fetch_fsmap finished waiting for FSMap version 64 2023-11-08T16:37:39.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.585+0000 7f08717fe740 10 client.7074 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T16:37:39.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.585+0000 7f08717fe740 20 client.7074 populate_metadata read hostname 'smithi081' 2023-11-08T16:37:39.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.585+0000 7f08717fe740 1 --2- 172.21.15.81:0/2382357991 >> [v2:172.21.15.81:6836/2162767469,v1:172.21.15.81:6837/2162767469] conn(0x55cadc0eda70 0x55cadc0ede50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.585+0000 7f08717fe740 4 client.7074 mds_command: new command op to 6942 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:37:39.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.585+0000 7f08717fe740 1 -- 172.21.15.81:0/2382357991 --> [v2:172.21.15.81:6836/2162767469,v1:172.21.15.81:6837/2162767469] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cadc0104c0 con 0x55cadc0eda70 2023-11-08T16:37:39.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.585+0000 7f086c92c640 1 --2- 172.21.15.81:0/2382357991 >> [v2:172.21.15.81:6836/2162767469,v1:172.21.15.81:6837/2162767469] conn(0x55cadc0eda70 0x55cadc0ede50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.585+0000 7f08477fe640 10 client.7074 ms_handle_connect on v2:172.21.15.81:6836/2162767469 2023-11-08T16:37:39.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.585+0000 7f086c92c640 1 --2- 172.21.15.81:0/2382357991 >> [v2:172.21.15.81:6836/2162767469,v1:172.21.15.81:6837/2162767469] conn(0x55cadc0eda70 0x55cadc0ede50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30d76e775bcdca71 server_cookie=58a455aa3758644f in_seq=0 out_seq=0 2023-11-08T16:37:39.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08477fe640 1 -- 172.21.15.81:0/2382357991 <== mds.0 v2:172.21.15.81:6836/2162767469 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55cadc0104c0 con 0x55cadc0eda70 2023-11-08T16:37:39.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08477fe640 10 client.7074 handle_command_reply: tid=0 2023-11-08T16:37:39.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 1 client.7074 shutdown 2023-11-08T16:37:39.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 1 -- 172.21.15.81:0/2382357991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 msgr2=0x55cadc161420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 1 --2- 172.21.15.81:0/2382357991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc161420 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f0854045040 tx=0x7f085402e000 comp rx=0 tx=0).stop 2023-11-08T16:37:39.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 1 -- 172.21.15.81:0/2382357991 shutdown_connections 2023-11-08T16:37:39.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 1 --2- 172.21.15.81:0/2382357991 >> [v2:172.21.15.81:6836/2162767469,v1:172.21.15.81:6837/2162767469] conn(0x55cadc0eda70 0x55cadc0ede50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 1 --2- 172.21.15.81:0/2382357991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cadc110c60 0x55cadc1657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 1 --2- 172.21.15.81:0/2382357991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cadc0df150 0x55cadc161420 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 1 --2- 172.21.15.81:0/2382357991 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cadc0d9080 0x55cadc0c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08477fe640 0 client.7074 ms_handle_reset on v2:172.21.15.81:6836/2162767469 2023-11-08T16:37:39.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 1 -- 172.21.15.81:0/2382357991 >> 172.21.15.81:0/2382357991 conn(0x55cadc0ac4c0 msgr2=0x55cadc0a7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:39.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 1 -- 172.21.15.81:0/2382357991 shutdown_connections 2023-11-08T16:37:39.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 1 -- 172.21.15.81:0/2382357991 wait complete. 2023-11-08T16:37:39.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.586+0000 7f08717fe740 20 client.7074 trim_cache size 0 max 16384 2023-11-08T16:37:39.606 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:37:39.606 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:37:39.607 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:37:39.607 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 6942, 2023-11-08T16:37:39.607 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:37:39.607 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:37:39.607 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:37:39.607 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 6.6273244709999997, 2023-11-08T16:37:39.608 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 64, 2023-11-08T16:37:39.608 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 81, 2023-11-08T16:37:39.608 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 81, 2023-11-08T16:37:39.608 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 12.906692199 2023-11-08T16:37:39.608 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:37:39.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 -- 172.21.15.81:0/3134547510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f084c06aea0 msgr2=0x7f084c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f084c06aea0 0x7f084c06d360 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f08681e7230 tx=0x7f085c008040 comp rx=0 tx=0).stop 2023-11-08T16:37:39.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 -- 172.21.15.81:0/3134547510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0868155d20 msgr2=0x7f0868152780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0868155d20 0x7f0868152780 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f085800bc70 tx=0x7f086000df60 comp rx=0 tx=0).stop 2023-11-08T16:37:39.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 -- 172.21.15.81:0/3134547510 shutdown_connections 2023-11-08T16:37:39.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f084c06aea0 0x7f084c06d360 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0868155d20 0x7f0868152780 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08681568b0 0x7f08680acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 --2- 172.21.15.81:0/3134547510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0858003f40 0x7f08680c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 -- 172.21.15.81:0/3134547510 >> 172.21.15.81:0/3134547510 conn(0x7f08680b04d0 msgr2=0x7f08680ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:39.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 -- 172.21.15.81:0/3134547510 shutdown_connections 2023-11-08T16:37:39.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.587+0000 7f086d92e640 1 -- 172.21.15.81:0/3134547510 wait complete. 2023-11-08T16:37:39.612 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:37:39.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.806+0000 7f43794d4640 1 Processor -- start 2023-11-08T16:37:39.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.807+0000 7f43794d4640 1 -- start start 2023-11-08T16:37:39.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.808+0000 7f43794d4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f43740cc5e0 0x7f43740cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.808+0000 7f43794d4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740ccfb0 0x7f4374070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.808+0000 7f43794d4640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43740c9a70 con 0x7f43740adcd0 2023-11-08T16:37:39.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.808+0000 7f43794d4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43740cc3e0 con 0x7f43740cc5e0 2023-11-08T16:37:39.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.808+0000 7f43794d4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437406d070 con 0x7f43740ccfb0 2023-11-08T16:37:39.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.809+0000 7f43727fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f43740cc5e0 0x7f43740cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.809+0000 7f43727fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f43740cc5e0 0x7f43740cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50502/0 (socket says 172.21.15.81:50502) 2023-11-08T16:37:39.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.809+0000 7f43727fc640 1 -- 172.21.15.81:0/3988133478 learned_addr learned my addr 172.21.15.81:0/3988133478 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:39.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.809+0000 7f43737fe640 1 --2- 172.21.15.81:0/3988133478 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740ccfb0 0x7f4374070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.809+0000 7f43727fc640 1 -- 172.21.15.81:0/3988133478 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740ccfb0 msgr2=0x7f4374070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.809+0000 7f43727fc640 1 --2- 172.21.15.81:0/3988133478 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740ccfb0 0x7f4374070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.809+0000 7f43727fc640 1 -- 172.21.15.81:0/3988133478 >> v1:172.21.15.93:6789/0 conn(0x7f43740adcd0 legacy=0x7f43740ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:39.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.809+0000 7f43727fc640 1 -- 172.21.15.81:0/3988133478 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f437406d350 con 0x7f43740cc5e0 2023-11-08T16:37:39.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.810+0000 7f43727fc640 1 --2- 172.21.15.81:0/3988133478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740cc5e0 0x7f43740cca70 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f435c009870 tx=0x7f435c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15265250b93395c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:39.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.810+0000 7f4371ffb640 1 -- 172.21.15.81:0/3988133478 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c030840 con 0x7f43740cc5e0 2023-11-08T16:37:39.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.811+0000 7f4371ffb640 1 -- 172.21.15.81:0/3988133478 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c0309a0 con 0x7f43740cc5e0 2023-11-08T16:37:39.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.811+0000 7f4371ffb640 1 -- 172.21.15.81:0/3988133478 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c030d30 con 0x7f43740cc5e0 2023-11-08T16:37:39.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.811+0000 7f43794d4640 1 -- 172.21.15.81:0/3988133478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740cc5e0 msgr2=0x7f43740cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.811+0000 7f43794d4640 1 --2- 172.21.15.81:0/3988133478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740cc5e0 0x7f43740cca70 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f435c009870 tx=0x7f435c02b980 comp rx=0 tx=0).stop 2023-11-08T16:37:39.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.811+0000 7f43794d4640 1 -- 172.21.15.81:0/3988133478 shutdown_connections 2023-11-08T16:37:39.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.811+0000 7f43794d4640 1 --2- 172.21.15.81:0/3988133478 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740ccfb0 0x7f4374070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.811+0000 7f43794d4640 1 --2- 172.21.15.81:0/3988133478 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740cc5e0 0x7f43740cca70 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.811+0000 7f43794d4640 1 -- 172.21.15.81:0/3988133478 >> 172.21.15.81:0/3988133478 conn(0x7f43740cb490 msgr2=0x7f43740cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:39.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.811+0000 7f43794d4640 1 -- 172.21.15.81:0/3988133478 shutdown_connections 2023-11-08T16:37:39.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.811+0000 7f43794d4640 1 -- 172.21.15.81:0/3988133478 wait complete. 2023-11-08T16:37:39.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.813+0000 7f43794d4640 1 Processor -- start 2023-11-08T16:37:39.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.813+0000 7f43794d4640 1 -- start start 2023-11-08T16:37:39.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.814+0000 7f43794d4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740adcd0 0x7f4374070010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.814+0000 7f43794d4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43740cc5e0 0x7f4374070550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.814+0000 7f43794d4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740ccfb0 0x7f437414fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.814+0000 7f43794d4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437414dd30 con 0x7f43740adcd0 2023-11-08T16:37:39.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.814+0000 7f43794d4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f437414e010 con 0x7f43740cc5e0 2023-11-08T16:37:39.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.814+0000 7f43794d4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f437414e2f0 con 0x7f43740ccfb0 2023-11-08T16:37:39.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.814+0000 7f4372ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740adcd0 0x7f4374070010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.814+0000 7f4372ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740adcd0 0x7f4374070010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:50512/0 (socket says 172.21.15.81:50512) 2023-11-08T16:37:39.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.814+0000 7f4372ffd640 1 -- 172.21.15.81:0/2720665559 learned_addr learned my addr 172.21.15.81:0/2720665559 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:39.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f43727fc640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43740cc5e0 0x7f4374070550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f4372ffd640 1 -- 172.21.15.81:0/2720665559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740ccfb0 msgr2=0x7f437414fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f43737fe640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740ccfb0 0x7f437414fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f4372ffd640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740ccfb0 0x7f437414fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f4372ffd640 1 -- 172.21.15.81:0/2720665559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43740cc5e0 msgr2=0x7f4374070550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:39.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f4372ffd640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43740cc5e0 0x7f4374070550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:39.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f4372ffd640 1 -- 172.21.15.81:0/2720665559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c0094d0 con 0x7f43740adcd0 2023-11-08T16:37:39.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f43727fc640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43740cc5e0 0x7f4374070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:39.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f4372ffd640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740adcd0 0x7f4374070010 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f436400c730 tx=0x7f436400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:39.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f436b7fe640 1 -- 172.21.15.81:0/2720665559 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364003b70 con 0x7f43740adcd0 2023-11-08T16:37:39.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f436b7fe640 1 -- 172.21.15.81:0/2720665559 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4364003cf0 con 0x7f43740adcd0 2023-11-08T16:37:39.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f436b7fe640 1 -- 172.21.15.81:0/2720665559 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4364003ea0 con 0x7f43740adcd0 2023-11-08T16:37:39.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.815+0000 7f43794d4640 1 -- 172.21.15.81:0/2720665559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f437414e5d0 con 0x7f43740adcd0 2023-11-08T16:37:39.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.816+0000 7f43794d4640 1 -- 172.21.15.81:0/2720665559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f437414d8c0 con 0x7f43740adcd0 2023-11-08T16:37:39.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.817+0000 7f43794d4640 1 -- 172.21.15.81:0/2720665559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4374072170 con 0x7f43740adcd0 2023-11-08T16:37:39.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.818+0000 7f436b7fe640 1 -- 172.21.15.81:0/2720665559 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f436400aa90 con 0x7f43740adcd0 2023-11-08T16:37:39.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.819+0000 7f436b7fe640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406aea0 0x7f435406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:39.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.819+0000 7f436b7fe640 1 -- 172.21.15.81:0/2720665559 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4364089460 con 0x7f43740adcd0 2023-11-08T16:37:39.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.819+0000 7f43727fc640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406aea0 0x7f435406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:39.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.820+0000 7f43727fc640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406aea0 0x7f435406d360 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f435c036000 tx=0x7f435c03b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:39.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:39.824+0000 7f436b7fe640 1 -- 172.21.15.81:0/2720665559 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4364054480 con 0x7f43740adcd0 2023-11-08T16:37:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.055+0000 7f43794d4640 1 -- 172.21.15.81:0/2720665559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f43740c9ca0 con 0x7f43740adcd0 2023-11-08T16:37:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.056+0000 7f436b7fe640 1 -- 172.21.15.81:0/2720665559 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7f43740c9ca0 con 0x7f43740adcd0 2023-11-08T16:37:40.057 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T16:37:40.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.058+0000 7f43794d4640 1 -- 172.21.15.81:0/2720665559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406aea0 msgr2=0x7f435406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:40.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.058+0000 7f43794d4640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406aea0 0x7f435406d360 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f435c036000 tx=0x7f435c03b040 comp rx=0 tx=0).stop 2023-11-08T16:37:40.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.058+0000 7f43794d4640 1 -- 172.21.15.81:0/2720665559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740adcd0 msgr2=0x7f4374070010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:40.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.058+0000 7f43794d4640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740adcd0 0x7f4374070010 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f436400c730 tx=0x7f436400cbf0 comp rx=0 tx=0).stop 2023-11-08T16:37:40.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.058+0000 7f43794d4640 1 -- 172.21.15.81:0/2720665559 shutdown_connections 2023-11-08T16:37:40.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.059+0000 7f43794d4640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f435406aea0 0x7f435406d360 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:40.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.059+0000 7f43794d4640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f43740ccfb0 0x7f437414fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:40.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.059+0000 7f43794d4640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43740cc5e0 0x7f4374070550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:40.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.059+0000 7f43794d4640 1 --2- 172.21.15.81:0/2720665559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f43740adcd0 0x7f4374070010 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:40.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.059+0000 7f43794d4640 1 -- 172.21.15.81:0/2720665559 >> 172.21.15.81:0/2720665559 conn(0x7f43740cb490 msgr2=0x7f43740c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:40.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.059+0000 7f43794d4640 1 -- 172.21.15.81:0/2720665559 shutdown_connections 2023-11-08T16:37:40.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:40.059+0000 7f43794d4640 1 -- 172.21.15.81:0/2720665559 wait complete. 2023-11-08T16:37:40.080 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:37:40.080 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:37:40.081 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:37:40.081 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:37:40.081 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:37:40.081 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:37:40.082 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:37:40.082 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:37:40.082 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:37:40.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:40.083 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:37:40.135 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:37:40.136 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:37:40.136 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:37:40.136 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:37:40.136 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:37:40.136 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:37:40.137 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:37:40.137 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:37:40.137 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:37:40.138 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:37:40.138 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:37:40.138 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:37:40.138 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:37:40.138 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:37:40.139 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:37:40.139 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:37:40.139 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:37:40.139 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 41849sec preferred_lft 41849sec 2023-11-08T16:37:40.139 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:37:40.140 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:37:40.140 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:37:40.140 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff 2023-11-08T16:37:40.140 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:37:40.140 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:37:40.141 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:37:40.141 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:37:40.141 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:37:40.141 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:37:40.142 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:37:40.142 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:37:40.142 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:37:40.143 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:37:40.143 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::2cac:17ff:fe64:67da/64 scope link 2023-11-08T16:37:40.143 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:37:40.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:40.144 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:37:40.234 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:37:40.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:40.238 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:37:40.293 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:37:40.294 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:37:40.294 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:37:40.294 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:37:40.295 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:37:40.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:40.295 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:37:40.296 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:37:40.296 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:37:40.297 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:37:40.297 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:37:40.297 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:37:40.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:40.485 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:37:40.534 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:37:40.535 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:37:40.536 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:37:40.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:40.537 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:37:40.631 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:37:40.631 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:37:40.632 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:37:40.689 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:37:40.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:37:40.690 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:37:40.740 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:37:40.834 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:37:40.890 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:37:40.891 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:37:40.891 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:37:40.891 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:37:40.891 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:37:40.891 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:37:40.892 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:37:40.892 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:37:40.892 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:37:40.892 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:37:40.892 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:37:40.893 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:37:40.893 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:37:40.893 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:37:40.893 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:37:40.893 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:37:40.894 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:37:40.894 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:37:40.894 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:37:40.894 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:37:40.894 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:37:40.895 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:37:40.895 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:37:40.895 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:37:40.895 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:37:40.896 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:37:40.896 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:37:40.896 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:37:40.896 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:37:40.896 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:37:40.897 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:37:40.897 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:37:40.897 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:37:40.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:40.899 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:37:40.950 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:37:40.951 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:37:40.957 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:37:41.034 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:37:41.080 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:37:41.080 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:37:41.081 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:37:41.081 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:37:41.081 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:37:41.081 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:37:41.082 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:37:41.082 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:37:41.082 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:37:41.082 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:37:41.083 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:37:41.083 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:37:41.083 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:37:41.083 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:37:41.083 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:37:41.084 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:37:41.084 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:37:41.084 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:37:41.084 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:37:41.084 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:37:41.085 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:37:41.085 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:37:41.085 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:37:41.085 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:37:41.085 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:37:41.086 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:37:41.086 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:37:41.086 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:37:41.086 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:37:41.087 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:37:41.087 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:37:41.087 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:37:41.087 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:37:41.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:41.089 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:37:41.147 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:37:41.145+0000 7f3b8376e480 -1 init, newargv = 0x5604f286f2c0 newargc=15 2023-11-08T16:37:41.148 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[113709]: starting ceph client 2023-11-08T16:37:41.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[113709]: starting fuse 2023-11-08T16:37:41.174 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:37:41.174 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:37:41.175 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:37:41.175 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:37:41.175 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:37:41.176 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:37:41.176 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:37:41.177 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:37:41.177 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:37:41.177 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:37:41.178 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:37:41.178 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:37:41.178 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:37:41.179 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:37:41.179 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:37:41.180 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:37:41.181 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:37:41.181 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:37:41.182 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:37:41.183 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:37:41.183 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:37:41.184 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:37:41.185 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:37:41.185 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:37:41.186 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:37:41.187 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:37:41.187 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:37:41.188 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:37:41.188 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:37:41.379 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.113709.asok 2023-11-08T16:37:41.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:37:41.396 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.113709.asok 2023-11-08T16:37:41.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:41.397 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.113709.asok status 2023-11-08T16:37:41.638 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:37:41.638 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:37:41.638 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:37:41.638 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:37:41.639 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:37:41.639 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:37:41.639 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:37:41.639 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "113709", 2023-11-08T16:37:41.639 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:37:41.640 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:41.640 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:37:41.640 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:37:41.640 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 7086, 2023-11-08T16:37:41.640 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:37:41.641 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:37:41.641 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:37:41.641 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 7086 2023-11-08T16:37:41.641 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:41.642 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:37:41.642 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:37:41.642 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:37:41.642 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1653649240 2023-11-08T16:37:41.642 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:37:41.643 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:41.643 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:37:41.643 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:37:41.644 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:37:41.644 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1653649240 2023-11-08T16:37:41.644 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:37:41.645 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.7086 192.168.0.1:0/1653649240", 2023-11-08T16:37:41.645 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/1653649240", 2023-11-08T16:37:41.646 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:37:41.646 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 64, 2023-11-08T16:37:41.646 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 81, 2023-11-08T16:37:41.647 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:37:41.647 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:37:41.648 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:37:41.648 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:37:41.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:41.653 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:37:41.653 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:37:41.708 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:37:41.709 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:37:41.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:41.710 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:37:41.802 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-11-08T16:37:42.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.014+0000 7f8b5adc0640 1 Processor -- start 2023-11-08T16:37:42.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.014+0000 7f8b5adc0640 1 -- start start 2023-11-08T16:37:42.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.015+0000 7f8b5adc0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8b540c90b0 0x7f8b540c9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:42.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.015+0000 7f8b5adc0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8b540cd640 0x7f8b5415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:42.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.015+0000 7f8b5adc0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b540c99d0 con 0x7f8b5406b250 2023-11-08T16:37:42.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.015+0000 7f8b5adc0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5406bc30 con 0x7f8b540c90b0 2023-11-08T16:37:42.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.015+0000 7f8b5adc0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5406bf10 con 0x7f8b540cd640 2023-11-08T16:37:42.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.016+0000 7f8b58b5c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8b540c90b0 0x7f8b540c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:42.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.016+0000 7f8b58b5c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8b540c90b0 0x7f8b540c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39174/0 (socket says 172.21.15.81:39174) 2023-11-08T16:37:42.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.016+0000 7f8b53fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f8b5406b250 0x7f8b5415cfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47698/0 (socket says 172.21.15.81:47698) 2023-11-08T16:37:42.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.016+0000 7f8b58b5c640 1 -- 172.21.15.81:0/2004883195 learned_addr learned my addr 172.21.15.81:0/2004883195 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:42.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.016+0000 7f8b5935d640 1 --2- 172.21.15.81:0/2004883195 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8b540cd640 0x7f8b5415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:42.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.016+0000 7f8b58b5c640 1 -- 172.21.15.81:0/2004883195 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8b540cd640 msgr2=0x7f8b5415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:42.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.016+0000 7f8b58b5c640 1 --2- 172.21.15.81:0/2004883195 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8b540cd640 0x7f8b5415d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:42.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.016+0000 7f8b58b5c640 1 -- 172.21.15.81:0/2004883195 >> v1:172.21.15.81:6789/0 conn(0x7f8b5406b250 legacy=0x7f8b5415cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:42.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.016+0000 7f8b58b5c640 1 -- 172.21.15.81:0/2004883195 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5406c1f0 con 0x7f8b540c90b0 2023-11-08T16:37:42.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.016+0000 7f8b537fe640 1 -- 172.21.15.81:0/2004883195 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441536648 0 0) 0x7f8b540c99d0 con 0x7f8b5406b250 2023-11-08T16:37:42.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.017+0000 7f8b58b5c640 1 --2- 172.21.15.81:0/2004883195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b540c90b0 0x7f8b540c9490 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009870 tx=0x7f8b4802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d0b774de35f1ea5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:42.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.018+0000 7f8b537fe640 1 -- 172.21.15.81:0/2004883195 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48030840 con 0x7f8b540c90b0 2023-11-08T16:37:42.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.018+0000 7f8b537fe640 1 -- 172.21.15.81:0/2004883195 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b480309a0 con 0x7f8b540c90b0 2023-11-08T16:37:42.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.018+0000 7f8b537fe640 1 -- 172.21.15.81:0/2004883195 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48030d30 con 0x7f8b540c90b0 2023-11-08T16:37:42.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.018+0000 7f8b5adc0640 1 -- 172.21.15.81:0/2004883195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b540c90b0 msgr2=0x7f8b540c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:42.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.018+0000 7f8b5adc0640 1 --2- 172.21.15.81:0/2004883195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b540c90b0 0x7f8b540c9490 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009870 tx=0x7f8b4802b980 comp rx=0 tx=0).stop 2023-11-08T16:37:42.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.019+0000 7f8b5adc0640 1 -- 172.21.15.81:0/2004883195 shutdown_connections 2023-11-08T16:37:42.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.019+0000 7f8b5adc0640 1 --2- 172.21.15.81:0/2004883195 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8b540cd640 0x7f8b5415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:42.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.019+0000 7f8b5adc0640 1 --2- 172.21.15.81:0/2004883195 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b540c90b0 0x7f8b540c9490 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:42.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.019+0000 7f8b5adc0640 1 -- 172.21.15.81:0/2004883195 >> 172.21.15.81:0/2004883195 conn(0x7f8b540cb4e0 msgr2=0x7f8b540cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:42.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.019+0000 7f8b5adc0640 1 -- 172.21.15.81:0/2004883195 shutdown_connections 2023-11-08T16:37:42.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.019+0000 7f8b5adc0640 1 -- 172.21.15.81:0/2004883195 wait complete. 2023-11-08T16:37:42.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.020+0000 7f8b5adc0640 1 Processor -- start 2023-11-08T16:37:42.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.020+0000 7f8b5adc0640 1 -- start start 2023-11-08T16:37:42.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.021+0000 7f8b5adc0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b5406b250 0x7f8b540c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:42.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.021+0000 7f8b5adc0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b540c90b0 0x7f8b540c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:42.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.021+0000 7f8b5adc0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8b540cd640 0x7f8b54152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:42.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.021+0000 7f8b5adc0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5415bee0 con 0x7f8b5406b250 2023-11-08T16:37:42.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.021+0000 7f8b5adc0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540c46e0 con 0x7f8b540c90b0 2023-11-08T16:37:42.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.021+0000 7f8b5adc0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5415b5b0 con 0x7f8b540cd640 2023-11-08T16:37:42.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.022+0000 7f8b58b5c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b5406b250 0x7f8b540c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:42.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.022+0000 7f8b58b5c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b5406b250 0x7f8b540c5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39178/0 (socket says 172.21.15.81:39178) 2023-11-08T16:37:42.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.022+0000 7f8b58b5c640 1 -- 172.21.15.81:0/402229694 learned_addr learned my addr 172.21.15.81:0/402229694 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:42.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.022+0000 7f8b53fff640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b540c90b0 0x7f8b540c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:42.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.022+0000 7f8b58b5c640 1 -- 172.21.15.81:0/402229694 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8b540cd640 msgr2=0x7f8b54152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:42.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.022+0000 7f8b58b5c640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8b540cd640 0x7f8b54152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:42.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.022+0000 7f8b58b5c640 1 -- 172.21.15.81:0/402229694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b540c90b0 msgr2=0x7f8b540c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:42.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.022+0000 7f8b58b5c640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b540c90b0 0x7f8b540c4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:42.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.022+0000 7f8b58b5c640 1 -- 172.21.15.81:0/402229694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b480094d0 con 0x7f8b5406b250 2023-11-08T16:37:42.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.022+0000 7f8b58b5c640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b5406b250 0x7f8b540c5080 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009870 tx=0x7f8b48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:42.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.023+0000 7f8b517fa640 1 -- 172.21.15.81:0/402229694 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48039c90 con 0x7f8b5406b250 2023-11-08T16:37:42.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.023+0000 7f8b517fa640 1 -- 172.21.15.81:0/402229694 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b48039df0 con 0x7f8b5406b250 2023-11-08T16:37:42.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.023+0000 7f8b5adc0640 1 -- 172.21.15.81:0/402229694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5415b890 con 0x7f8b5406b250 2023-11-08T16:37:42.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.023+0000 7f8b517fa640 1 -- 172.21.15.81:0/402229694 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b48043620 con 0x7f8b5406b250 2023-11-08T16:37:42.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.023+0000 7f8b5adc0640 1 -- 172.21.15.81:0/402229694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b54155010 con 0x7f8b5406b250 2023-11-08T16:37:42.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.024+0000 7f8b5adc0640 1 -- 172.21.15.81:0/402229694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b540743f0 con 0x7f8b5406b250 2023-11-08T16:37:42.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.025+0000 7f8b517fa640 1 -- 172.21.15.81:0/402229694 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8b48043780 con 0x7f8b5406b250 2023-11-08T16:37:42.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.026+0000 7f8b517fa640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8b3806ae70 0x7f8b3806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:42.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.026+0000 7f8b517fa640 1 -- 172.21.15.81:0/402229694 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b4803e070 con 0x7f8b5406b250 2023-11-08T16:37:42.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.026+0000 7f8b53fff640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8b3806ae70 0x7f8b3806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:42.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.027+0000 7f8b53fff640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8b3806ae70 0x7f8b3806d330 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8b540c2160 tx=0x7f8b40009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:42.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.031+0000 7f8b517fa640 1 -- 172.21.15.81:0/402229694 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8b480777d0 con 0x7f8b5406b250 2023-11-08T16:37:42.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.225+0000 7f8b5adc0640 1 -- 172.21.15.81:0/402229694 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f8b5415b6f0 con 0x7f8b5406b250 2023-11-08T16:37:42.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.974+0000 7f8b517fa640 1 -- 172.21.15.81:0/402229694 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v82) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8b5415b6f0 con 0x7f8b5406b250 2023-11-08T16:37:42.976 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data_extra' created 2023-11-08T16:37:42.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 -- 172.21.15.81:0/402229694 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8b3806ae70 msgr2=0x7f8b3806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:42.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8b3806ae70 0x7f8b3806d330 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f8b540c2160 tx=0x7f8b40009040 comp rx=0 tx=0).stop 2023-11-08T16:37:42.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 -- 172.21.15.81:0/402229694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b5406b250 msgr2=0x7f8b540c5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:42.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b5406b250 0x7f8b540c5080 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f8b48009870 tx=0x7f8b48004730 comp rx=0 tx=0).stop 2023-11-08T16:37:42.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 -- 172.21.15.81:0/402229694 shutdown_connections 2023-11-08T16:37:42.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8b3806ae70 0x7f8b3806d330 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:42.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8b540cd640 0x7f8b54152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:42.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b540c90b0 0x7f8b540c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:42.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 --2- 172.21.15.81:0/402229694 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8b5406b250 0x7f8b540c5080 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:42.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 -- 172.21.15.81:0/402229694 >> 172.21.15.81:0/402229694 conn(0x7f8b540cb4e0 msgr2=0x7f8b540cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:42.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 -- 172.21.15.81:0/402229694 shutdown_connections 2023-11-08T16:37:42.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:42.978+0000 7f8b5adc0640 1 -- 172.21.15.81:0/402229694 wait complete. 2023-11-08T16:37:43.007 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-11-08T16:37:43.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.197+0000 7f11ec825640 1 Processor -- start 2023-11-08T16:37:43.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.197+0000 7f11ec825640 1 -- start start 2023-11-08T16:37:43.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.199+0000 7f11ec825640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f11e81612f0 0x7f11e806c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:43.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.199+0000 7f11ec825640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f11e806cc70 0x7f11e80cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:43.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.199+0000 7f11ec825640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e815d680 con 0x7f11e81616d0 2023-11-08T16:37:43.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.199+0000 7f11ec825640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e815d4a0 con 0x7f11e806cc70 2023-11-08T16:37:43.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.199+0000 7f11ec825640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e815d960 con 0x7f11e81612f0 2023-11-08T16:37:43.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.199+0000 7f11e759e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f11e806cc70 0x7f11e80cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:43.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.200+0000 7f11e759e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f11e806cc70 0x7f11e80cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39180/0 (socket says 172.21.15.81:39180) 2023-11-08T16:37:43.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.200+0000 7f11e759e640 1 -- 172.21.15.81:0/85709924 learned_addr learned my addr 172.21.15.81:0/85709924 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:43.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.200+0000 7f11e6d9d640 1 --2- 172.21.15.81:0/85709924 >> v2:172.21.15.93:3300/0 conn(0x7f11e81612f0 0x7f11e806c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:43.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.200+0000 7f11e759e640 1 -- 172.21.15.81:0/85709924 >> v2:172.21.15.93:3300/0 conn(0x7f11e81612f0 msgr2=0x7f11e806c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:43.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.200+0000 7f11e759e640 1 --2- 172.21.15.81:0/85709924 >> v2:172.21.15.93:3300/0 conn(0x7f11e81612f0 0x7f11e806c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:43.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.200+0000 7f11e759e640 1 -- 172.21.15.81:0/85709924 >> v1:172.21.15.93:6789/0 conn(0x7f11e81616d0 legacy=0x7f11e80c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:43.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.200+0000 7f11e759e640 1 -- 172.21.15.81:0/85709924 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e815dc40 con 0x7f11e806cc70 2023-11-08T16:37:43.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.200+0000 7f11e6d9d640 1 --2- 172.21.15.81:0/85709924 >> v2:172.21.15.93:3300/0 conn(0x7f11e81612f0 0x7f11e806c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:37:43.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.201+0000 7f11e759e640 1 --2- 172.21.15.81:0/85709924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e806cc70 0x7f11e80cd420 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f11e80cf8c0 tx=0x7f11e002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=960edc6b7b191f0a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:43.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.201+0000 7f11e659c640 1 -- 172.21.15.81:0/85709924 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e002ca80 con 0x7f11e806cc70 2023-11-08T16:37:43.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.202+0000 7f11e659c640 1 -- 172.21.15.81:0/85709924 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e002cbe0 con 0x7f11e806cc70 2023-11-08T16:37:43.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.202+0000 7f11e659c640 1 -- 172.21.15.81:0/85709924 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e00343f0 con 0x7f11e806cc70 2023-11-08T16:37:43.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.202+0000 7f11ec825640 1 -- 172.21.15.81:0/85709924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e806cc70 msgr2=0x7f11e80cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:43.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.202+0000 7f11ec825640 1 --2- 172.21.15.81:0/85709924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e806cc70 0x7f11e80cd420 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f11e80cf8c0 tx=0x7f11e002bee0 comp rx=0 tx=0).stop 2023-11-08T16:37:43.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.202+0000 7f11ec825640 1 -- 172.21.15.81:0/85709924 shutdown_connections 2023-11-08T16:37:43.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.202+0000 7f11ec825640 1 --2- 172.21.15.81:0/85709924 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e806cc70 0x7f11e80cd420 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:43.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.202+0000 7f11ec825640 1 --2- 172.21.15.81:0/85709924 >> v2:172.21.15.93:3300/0 conn(0x7f11e81612f0 0x7f11e806c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:43.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.202+0000 7f11ec825640 1 -- 172.21.15.81:0/85709924 >> 172.21.15.81:0/85709924 conn(0x7f11e80ce8d0 msgr2=0x7f11e80cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:43.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.202+0000 7f11ec825640 1 -- 172.21.15.81:0/85709924 shutdown_connections 2023-11-08T16:37:43.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.202+0000 7f11ec825640 1 -- 172.21.15.81:0/85709924 wait complete. 2023-11-08T16:37:43.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.204+0000 7f11ec825640 1 Processor -- start 2023-11-08T16:37:43.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.204+0000 7f11ec825640 1 -- start start 2023-11-08T16:37:43.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.205+0000 7f11ec825640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11e806cc70 0x7f11e8148490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:43.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.205+0000 7f11ec825640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e81612f0 0x7f11e80b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:43.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.205+0000 7f11ec825640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e80b2310 0x7f11e81f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:43.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.205+0000 7f11ec825640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e81489d0 con 0x7f11e80b2310 2023-11-08T16:37:43.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.205+0000 7f11ec825640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e80b3a80 con 0x7f11e806cc70 2023-11-08T16:37:43.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.205+0000 7f11ec825640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e80b2ee0 con 0x7f11e81612f0 2023-11-08T16:37:43.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.205+0000 7f11e759e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e80b2310 0x7f11e81f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:43.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.205+0000 7f11e759e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e80b2310 0x7f11e81f2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39188/0 (socket says 172.21.15.81:39188) 2023-11-08T16:37:43.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.205+0000 7f11e759e640 1 -- 172.21.15.81:0/489886482 learned_addr learned my addr 172.21.15.81:0/489886482 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:43.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11e6d9d640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11e806cc70 0x7f11e8148490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:43.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11dffff640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e81612f0 0x7f11e80b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:43.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11e759e640 1 -- 172.21.15.81:0/489886482 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e81612f0 msgr2=0x7f11e80b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:43.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11e759e640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e81612f0 0x7f11e80b3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:43.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11e759e640 1 -- 172.21.15.81:0/489886482 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11e806cc70 msgr2=0x7f11e8148490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:43.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11e759e640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11e806cc70 0x7f11e8148490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:43.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11e759e640 1 -- 172.21.15.81:0/489886482 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e0009b30 con 0x7f11e80b2310 2023-11-08T16:37:43.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11e759e640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e80b2310 0x7f11e81f2f20 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f11e0036040 tx=0x7f11e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:43.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11df7fe640 1 -- 172.21.15.81:0/489886482 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e0009510 con 0x7f11e80b2310 2023-11-08T16:37:43.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11df7fe640 1 -- 172.21.15.81:0/489886482 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e003e450 con 0x7f11e80b2310 2023-11-08T16:37:43.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11df7fe640 1 -- 172.21.15.81:0/489886482 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e003e7e0 con 0x7f11e80b2310 2023-11-08T16:37:43.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.206+0000 7f11ec825640 1 -- 172.21.15.81:0/489886482 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e80b3150 con 0x7f11e80b2310 2023-11-08T16:37:43.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.207+0000 7f11ec825640 1 -- 172.21.15.81:0/489886482 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e80b3710 con 0x7f11e80b2310 2023-11-08T16:37:43.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.208+0000 7f11ec825640 1 -- 172.21.15.81:0/489886482 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e80b3c10 con 0x7f11e80b2310 2023-11-08T16:37:43.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.209+0000 7f11df7fe640 1 -- 172.21.15.81:0/489886482 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11e002cd50 con 0x7f11e80b2310 2023-11-08T16:37:43.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.210+0000 7f11df7fe640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c806aea0 0x7f11c806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:43.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.210+0000 7f11df7fe640 1 -- 172.21.15.81:0/489886482 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6139+0+0 (secure 0 0 0) 0x7f11e00ac6f0 con 0x7f11e80b2310 2023-11-08T16:37:43.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.210+0000 7f11e6d9d640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c806aea0 0x7f11c806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:43.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.211+0000 7f11e6d9d640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c806aea0 0x7f11c806d360 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f11d8006fd0 tx=0x7f11d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:43.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.215+0000 7f11df7fe640 1 -- 172.21.15.81:0/489886482 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11e0077550 con 0x7f11e80b2310 2023-11-08T16:37:43.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:43.445+0000 7f11ec825640 1 -- 172.21.15.81:0/489886482 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f11e8067800 con 0x7f11e80b2310 2023-11-08T16:37:44.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.004+0000 7f11df7fe640 1 -- 172.21.15.81:0/489886482 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 12 to fsmap v65) v1 ==== 141+0+0 (secure 0 0 0) 0x7f11e007ab70 con 0x7f11e80b2310 2023-11-08T16:37:44.007 INFO:teuthology.orchestra.run.smithi081.stderr:added data pool 12 to fsmap 2023-11-08T16:37:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.012+0000 7f11ec825640 1 -- 172.21.15.81:0/489886482 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c806aea0 msgr2=0x7f11c806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.012+0000 7f11ec825640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c806aea0 0x7f11c806d360 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f11d8006fd0 tx=0x7f11d8008040 comp rx=0 tx=0).stop 2023-11-08T16:37:44.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.012+0000 7f11ec825640 1 -- 172.21.15.81:0/489886482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e80b2310 msgr2=0x7f11e81f2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.012+0000 7f11ec825640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e80b2310 0x7f11e81f2f20 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f11e0036040 tx=0x7f11e0004a40 comp rx=0 tx=0).stop 2023-11-08T16:37:44.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.012+0000 7f11ec825640 1 -- 172.21.15.81:0/489886482 shutdown_connections 2023-11-08T16:37:44.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.012+0000 7f11ec825640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11c806aea0 0x7f11c806d360 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.012+0000 7f11ec825640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11e80b2310 0x7f11e81f2f20 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.012+0000 7f11ec825640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11e81612f0 0x7f11e80b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.012+0000 7f11ec825640 1 --2- 172.21.15.81:0/489886482 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11e806cc70 0x7f11e8148490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.012+0000 7f11ec825640 1 -- 172.21.15.81:0/489886482 >> 172.21.15.81:0/489886482 conn(0x7f11e80ce8d0 msgr2=0x7f11e806fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.013+0000 7f11ec825640 1 -- 172.21.15.81:0/489886482 shutdown_connections 2023-11-08T16:37:44.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.014+0000 7f11ec825640 1 -- 172.21.15.81:0/489886482 wait complete. 2023-11-08T16:37:44.037 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:37:44.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.219+0000 7faa721f5640 1 Processor -- start 2023-11-08T16:37:44.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.219+0000 7faa721f5640 1 -- start start 2023-11-08T16:37:44.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.221+0000 7faa721f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ca480 0x7faa6c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.221+0000 7faa721f5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7faa6c15ea60 0x7faa6c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.221+0000 7faa721f5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa6c06b630 con 0x7faa6c0ade50 2023-11-08T16:37:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.221+0000 7faa721f5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa6c0b1860 con 0x7faa6c15ea60 2023-11-08T16:37:44.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.221+0000 7faa721f5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0ca8e0 con 0x7faa6c0ca480 2023-11-08T16:37:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.222+0000 7faa62ffd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ca480 0x7faa6c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.222+0000 7faa6bfff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7faa6c15ea60 0x7faa6c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.222+0000 7faa62ffd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ca480 0x7faa6c15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:42884/0 (socket says 172.21.15.81:42884) 2023-11-08T16:37:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.222+0000 7faa62ffd640 1 -- 172.21.15.81:0/192825861 learned_addr learned my addr 172.21.15.81:0/192825861 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.223+0000 7faa6bfff640 1 -- 172.21.15.81:0/192825861 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ca480 msgr2=0x7faa6c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.223+0000 7faa6bfff640 1 --2- 172.21.15.81:0/192825861 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ca480 0x7faa6c15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.223+0000 7faa6bfff640 1 -- 172.21.15.81:0/192825861 >> v1:172.21.15.93:6789/0 conn(0x7faa6c0ade50 legacy=0x7faa6c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.223+0000 7faa6bfff640 1 -- 172.21.15.81:0/192825861 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa6c0cabb0 con 0x7faa6c15ea60 2023-11-08T16:37:44.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.223+0000 7faa62ffd640 1 --2- 172.21.15.81:0/192825861 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ca480 0x7faa6c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:37:44.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.224+0000 7faa6bfff640 1 --2- 172.21.15.81:0/192825861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa6c15ea60 0x7faa6c15eef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faa64009d90 tx=0x7faa6402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=286965051d296f32 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.225+0000 7faa6affd640 1 -- 172.21.15.81:0/192825861 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6402cc60 con 0x7faa6c15ea60 2023-11-08T16:37:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.225+0000 7faa6affd640 1 -- 172.21.15.81:0/192825861 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6402cdc0 con 0x7faa6c15ea60 2023-11-08T16:37:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.225+0000 7faa6affd640 1 -- 172.21.15.81:0/192825861 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa64034660 con 0x7faa6c15ea60 2023-11-08T16:37:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.225+0000 7faa721f5640 1 -- 172.21.15.81:0/192825861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa6c15ea60 msgr2=0x7faa6c15eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.225+0000 7faa721f5640 1 --2- 172.21.15.81:0/192825861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa6c15ea60 0x7faa6c15eef0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faa64009d90 tx=0x7faa6402beb0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.226+0000 7faa721f5640 1 -- 172.21.15.81:0/192825861 shutdown_connections 2023-11-08T16:37:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.226+0000 7faa721f5640 1 --2- 172.21.15.81:0/192825861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa6c15ea60 0x7faa6c15eef0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.226+0000 7faa721f5640 1 --2- 172.21.15.81:0/192825861 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ca480 0x7faa6c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.226+0000 7faa721f5640 1 -- 172.21.15.81:0/192825861 >> 172.21.15.81:0/192825861 conn(0x7faa6c0cb490 msgr2=0x7faa6c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:44.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.226+0000 7faa721f5640 1 -- 172.21.15.81:0/192825861 shutdown_connections 2023-11-08T16:37:44.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.226+0000 7faa721f5640 1 -- 172.21.15.81:0/192825861 wait complete. 2023-11-08T16:37:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.227+0000 7faa721f5640 1 Processor -- start 2023-11-08T16:37:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.227+0000 7faa721f5640 1 -- start start 2023-11-08T16:37:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.228+0000 7faa721f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ade50 0x7faa6c1f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.228+0000 7faa721f5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c0ca480 0x7faa6c0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.228+0000 7faa721f5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa6c15ea60 0x7faa6c148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.228+0000 7faa721f5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0c4c90 con 0x7faa6c0ca480 2023-11-08T16:37:44.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.228+0000 7faa721f5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0c4f70 con 0x7faa6c15ea60 2023-11-08T16:37:44.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.228+0000 7faa721f5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6c0c5250 con 0x7faa6c0ade50 2023-11-08T16:37:44.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.229+0000 7faa62ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c0ca480 0x7faa6c0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.229+0000 7faa62ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c0ca480 0x7faa6c0c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39190/0 (socket says 172.21.15.81:39190) 2023-11-08T16:37:44.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.229+0000 7faa62ffd640 1 -- 172.21.15.81:0/108261272 learned_addr learned my addr 172.21.15.81:0/108261272 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:44.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.229+0000 7faa6bfff640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa6c15ea60 0x7faa6c148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.229+0000 7faa6b7fe640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ade50 0x7faa6c1f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.229+0000 7faa62ffd640 1 -- 172.21.15.81:0/108261272 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ade50 msgr2=0x7faa6c1f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.229+0000 7faa62ffd640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ade50 0x7faa6c1f1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.229+0000 7faa62ffd640 1 -- 172.21.15.81:0/108261272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa6c15ea60 msgr2=0x7faa6c148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.229+0000 7faa62ffd640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa6c15ea60 0x7faa6c148770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.229+0000 7faa62ffd640 1 -- 172.21.15.81:0/108261272 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa540094d0 con 0x7faa6c0ca480 2023-11-08T16:37:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.230+0000 7faa62ffd640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c0ca480 0x7faa6c0c42d0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e9d0 tx=0x7faa5400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.230+0000 7faa68ff9640 1 -- 172.21.15.81:0/108261272 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa5400ca60 con 0x7faa6c0ca480 2023-11-08T16:37:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.230+0000 7faa68ff9640 1 -- 172.21.15.81:0/108261272 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa5400cbc0 con 0x7faa6c0ca480 2023-11-08T16:37:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.230+0000 7faa68ff9640 1 -- 172.21.15.81:0/108261272 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa54013470 con 0x7faa6c0ca480 2023-11-08T16:37:44.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.230+0000 7faa721f5640 1 -- 172.21.15.81:0/108261272 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa64009a70 con 0x7faa6c0ca480 2023-11-08T16:37:44.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.230+0000 7faa721f5640 1 -- 172.21.15.81:0/108261272 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa6c0c5810 con 0x7faa6c0ca480 2023-11-08T16:37:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.231+0000 7faa721f5640 1 -- 172.21.15.81:0/108261272 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa6c04ece0 con 0x7faa6c0ca480 2023-11-08T16:37:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.232+0000 7faa68ff9640 1 -- 172.21.15.81:0/108261272 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faa540135d0 con 0x7faa6c0ca480 2023-11-08T16:37:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.233+0000 7faa68ff9640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa4c06ae70 0x7faa4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.233+0000 7faa68ff9640 1 -- 172.21.15.81:0/108261272 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6171+0+0 (secure 0 0 0) 0x7faa54089b50 con 0x7faa6c0ca480 2023-11-08T16:37:44.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.233+0000 7faa6b7fe640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa4c06ae70 0x7faa4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.234+0000 7faa6b7fe640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa4c06ae70 0x7faa4c06d330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faa5c000f60 tx=0x7faa5c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:44.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.237+0000 7faa68ff9640 1 -- 172.21.15.81:0/108261272 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faa54054990 con 0x7faa6c0ca480 2023-11-08T16:37:44.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.464+0000 7faa721f5640 1 -- 172.21.15.81:0/108261272 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa6c0cb140 con 0x7faa6c0ca480 2023-11-08T16:37:44.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.466+0000 7faa68ff9640 1 -- 172.21.15.81:0/108261272 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4918 (secure 0 0 0) 0x7faa54017090 con 0x7faa6c0ca480 2023-11-08T16:37:44.468 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:44.468 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":65,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/2125403666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2125403666},{"type":"v1","addr":"172.21.15.93:6834","nonce":2125403666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2727356524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2727356524},{"type":"v1","addr":"172.21.15.93:6835","nonce":2727356524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6936,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2639654743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2639654743},{"type":"v1","addr":"172.21.15.81:6835","nonce":2639654743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:32.323261+0000","modified":"2023-11-08T16:37:43.447328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6942":{"gid":6942,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/2162767469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2162767469},{"type":"v1","addr":"172.21.15.81:6837","nonce":2162767469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:37:44.468 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 65 2023-11-08T16:37:44.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 -- 172.21.15.81:0/108261272 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa4c06ae70 msgr2=0x7faa4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa4c06ae70 0x7faa4c06d330 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7faa5c000f60 tx=0x7faa5c009040 comp rx=0 tx=0).stop 2023-11-08T16:37:44.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 -- 172.21.15.81:0/108261272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c0ca480 msgr2=0x7faa6c0c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c0ca480 0x7faa6c0c42d0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7faa5400e9d0 tx=0x7faa5400ee90 comp rx=0 tx=0).stop 2023-11-08T16:37:44.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 -- 172.21.15.81:0/108261272 shutdown_connections 2023-11-08T16:37:44.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa4c06ae70 0x7faa4c06d330 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faa6c15ea60 0x7faa6c148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faa6c0ca480 0x7faa6c0c42d0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 --2- 172.21.15.81:0/108261272 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faa6c0ade50 0x7faa6c1f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 -- 172.21.15.81:0/108261272 >> 172.21.15.81:0/108261272 conn(0x7faa6c0cb490 msgr2=0x7faa6c0b2c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:44.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 -- 172.21.15.81:0/108261272 shutdown_connections 2023-11-08T16:37:44.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.469+0000 7faa721f5640 1 -- 172.21.15.81:0/108261272 wait complete. 2023-11-08T16:37:44.489 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:44.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.693+0000 7fb1a5e4f640 1 Processor -- start 2023-11-08T16:37:44.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.693+0000 7fb1a5e4f640 1 -- start start 2023-11-08T16:37:44.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.695+0000 7fb1a5e4f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb1a00ade50 0x7fb1a00ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.695+0000 7fb1a5e4f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb1a015ea60 0x7fb1a015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.695+0000 7fb1a5e4f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1a006b630 con 0x7fb1a015a680 2023-11-08T16:37:44.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.695+0000 7fb1a5e4f640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a00b1860 con 0x7fb1a015ea60 2023-11-08T16:37:44.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.695+0000 7fb1a5e4f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1a00ca9c0 con 0x7fb1a00ade50 2023-11-08T16:37:44.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.695+0000 7fb19ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb1a015ea60 0x7fb1a015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.695+0000 7fb19ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb1a015ea60 0x7fb1a015eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39192/0 (socket says 172.21.15.81:39192) 2023-11-08T16:37:44.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.695+0000 7fb19ffff640 1 -- 172.21.15.81:0/1310987934 learned_addr learned my addr 172.21.15.81:0/1310987934 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:44.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.695+0000 7fb19f7fe640 1 --2- 172.21.15.81:0/1310987934 >> v2:172.21.15.93:3300/0 conn(0x7fb1a00ade50 0x7fb1a00ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.696+0000 7fb19ffff640 1 -- 172.21.15.81:0/1310987934 >> v2:172.21.15.93:3300/0 conn(0x7fb1a00ade50 msgr2=0x7fb1a00ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:44.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.696+0000 7fb19ffff640 1 --2- 172.21.15.81:0/1310987934 >> v2:172.21.15.93:3300/0 conn(0x7fb1a00ade50 0x7fb1a00ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.696+0000 7fb19ffff640 1 -- 172.21.15.81:0/1310987934 >> v1:172.21.15.93:6789/0 conn(0x7fb1a015a680 legacy=0x7fb1a015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:44.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.696+0000 7fb19ffff640 1 -- 172.21.15.81:0/1310987934 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1a00cabb0 con 0x7fb1a015ea60 2023-11-08T16:37:44.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.697+0000 7fb19ffff640 1 --2- 172.21.15.81:0/1310987934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a015ea60 0x7fb1a015eef0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0067770 tx=0x7fb19802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d22a06c041bff88 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:44.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.697+0000 7fb19e7fc640 1 -- 172.21.15.81:0/1310987934 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19802cb20 con 0x7fb1a015ea60 2023-11-08T16:37:44.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.697+0000 7fb19e7fc640 1 -- 172.21.15.81:0/1310987934 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb19802cc80 con 0x7fb1a015ea60 2023-11-08T16:37:44.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.697+0000 7fb19e7fc640 1 -- 172.21.15.81:0/1310987934 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1980343f0 con 0x7fb1a015ea60 2023-11-08T16:37:44.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.698+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/1310987934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a015ea60 msgr2=0x7fb1a015eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.698+0000 7fb1a5e4f640 1 --2- 172.21.15.81:0/1310987934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a015ea60 0x7fb1a015eef0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0067770 tx=0x7fb19802bf80 comp rx=0 tx=0).stop 2023-11-08T16:37:44.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.698+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/1310987934 shutdown_connections 2023-11-08T16:37:44.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.698+0000 7fb1a5e4f640 1 --2- 172.21.15.81:0/1310987934 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a015ea60 0x7fb1a015eef0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.698+0000 7fb1a5e4f640 1 --2- 172.21.15.81:0/1310987934 >> v2:172.21.15.93:3300/0 conn(0x7fb1a00ade50 0x7fb1a00ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.698+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/1310987934 >> 172.21.15.81:0/1310987934 conn(0x7fb1a00cb490 msgr2=0x7fb1a00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:44.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.698+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/1310987934 shutdown_connections 2023-11-08T16:37:44.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.698+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/1310987934 wait complete. 2023-11-08T16:37:44.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.700+0000 7fb1a5e4f640 1 Processor -- start 2023-11-08T16:37:44.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.700+0000 7fb1a5e4f640 1 -- start start 2023-11-08T16:37:44.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.701+0000 7fb1a5e4f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb1a00ade50 0x7fb1a00c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.701+0000 7fb1a5e4f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1a015a680 0x7fb1a00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.701+0000 7fb1a5e4f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a015ea60 0x7fb1a00acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.701+0000 7fb1a5e4f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a015ac20 con 0x7fb1a015ea60 2023-11-08T16:37:44.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.701+0000 7fb1a5e4f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a0149aa0 con 0x7fb1a015a680 2023-11-08T16:37:44.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.701+0000 7fb1a5e4f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1a00c5590 con 0x7fb1a00ade50 2023-11-08T16:37:44.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.701+0000 7fb19ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a015ea60 0x7fb1a00acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.701+0000 7fb19ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a015ea60 0x7fb1a00acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39196/0 (socket says 172.21.15.81:39196) 2023-11-08T16:37:44.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.701+0000 7fb19ffff640 1 -- 172.21.15.81:0/3715660017 learned_addr learned my addr 172.21.15.81:0/3715660017 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:44.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb19f7fe640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb1a00ade50 0x7fb1a00c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb19effd640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1a015a680 0x7fb1a00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb19ffff640 1 -- 172.21.15.81:0/3715660017 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb1a00ade50 msgr2=0x7fb1a00c5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb19ffff640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb1a00ade50 0x7fb1a00c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb19ffff640 1 -- 172.21.15.81:0/3715660017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1a015a680 msgr2=0x7fb1a00ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb19ffff640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1a015a680 0x7fb1a00ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb19ffff640 1 -- 172.21.15.81:0/3715660017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb198009b30 con 0x7fb1a015ea60 2023-11-08T16:37:44.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb19effd640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1a015a680 0x7fb1a00ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:44.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb19ffff640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a015ea60 0x7fb1a00acdf0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb198009f40 tx=0x7fb198004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:44.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb187fff640 1 -- 172.21.15.81:0/3715660017 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb198007e70 con 0x7fb1a015ea60 2023-11-08T16:37:44.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb187fff640 1 -- 172.21.15.81:0/3715660017 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb198009510 con 0x7fb1a015ea60 2023-11-08T16:37:44.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb187fff640 1 -- 172.21.15.81:0/3715660017 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb19803e690 con 0x7fb1a015ea60 2023-11-08T16:37:44.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.702+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/3715660017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1a00cdc80 con 0x7fb1a015ea60 2023-11-08T16:37:44.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.703+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/3715660017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1a00ce240 con 0x7fb1a015ea60 2023-11-08T16:37:44.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.704+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/3715660017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1a015ad60 con 0x7fb1a015ea60 2023-11-08T16:37:44.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.705+0000 7fb187fff640 1 -- 172.21.15.81:0/3715660017 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb19803e7f0 con 0x7fb1a015ea60 2023-11-08T16:37:44.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.706+0000 7fb187fff640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb18c06ae70 0x7fb18c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:44.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.706+0000 7fb187fff640 1 -- 172.21.15.81:0/3715660017 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fb1980ac700 con 0x7fb1a015ea60 2023-11-08T16:37:44.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.706+0000 7fb19f7fe640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb18c06ae70 0x7fb18c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:44.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.707+0000 7fb19f7fe640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb18c06ae70 0x7fb18c06d330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb190006fd0 tx=0x7fb190008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:44.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.710+0000 7fb187fff640 1 -- 172.21.15.81:0/3715660017 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb1a015ad60 con 0x7fb1a015ea60 2023-11-08T16:37:44.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.902+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/3715660017 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1a015ad60 con 0x7fb1a015ea60 2023-11-08T16:37:44.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.905+0000 7fb187fff640 1 -- 172.21.15.81:0/3715660017 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+18396 (secure 0 0 0) 0x7fb1a015ad60 con 0x7fb1a015ea60 2023-11-08T16:37:44.908 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:44.908 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":83,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:37:43.446618+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:37:30.174436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T16:37:31.340720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:37:42.279296+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2e","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:44.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.908+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/3715660017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb18c06ae70 msgr2=0x7fb18c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.908+0000 7fb1a5e4f640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb18c06ae70 0x7fb18c06d330 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb190006fd0 tx=0x7fb190008040 comp rx=0 tx=0).stop 2023-11-08T16:37:44.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.908+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/3715660017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a015ea60 msgr2=0x7fb1a00acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:44.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.908+0000 7fb1a5e4f640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a015ea60 0x7fb1a00acdf0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb198009f40 tx=0x7fb198004a40 comp rx=0 tx=0).stop 2023-11-08T16:37:44.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.909+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/3715660017 shutdown_connections 2023-11-08T16:37:44.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.909+0000 7fb1a5e4f640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb18c06ae70 0x7fb18c06d330 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.909+0000 7fb1a5e4f640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb1a015ea60 0x7fb1a00acdf0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.909+0000 7fb1a5e4f640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1a015a680 0x7fb1a00ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.909+0000 7fb1a5e4f640 1 --2- 172.21.15.81:0/3715660017 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb1a00ade50 0x7fb1a00c5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:44.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.909+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/3715660017 >> 172.21.15.81:0/3715660017 conn(0x7fb1a00cb490 msgr2=0x7fb1a01e5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:44.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.909+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/3715660017 shutdown_connections 2023-11-08T16:37:44.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:44.909+0000 7fb1a5e4f640 1 -- 172.21.15.81:0/3715660017 wait complete. 2023-11-08T16:37:44.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:44.930 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout.pool -v cephfs_data_extra .) 2023-11-08T16:37:44.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:44.995 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-11-08T16:37:45.053 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:37:45.053 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:37:45.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:45.096 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-11-08T16:37:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:6+0 records in 2023-11-08T16:37:45.355 INFO:teuthology.orchestra.run.smithi081.stderr:6+0 records out 2023-11-08T16:37:45.356 INFO:teuthology.orchestra.run.smithi081.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.158019 s, 39.8 MB/s 2023-11-08T16:37:45.357 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:37:45.358 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:37:45.358 DEBUG:teuthology.orchestra.run.smithi081:> import stat 2023-11-08T16:37:45.359 DEBUG:teuthology.orchestra.run.smithi081:> import json 2023-11-08T16:37:45.359 DEBUG:teuthology.orchestra.run.smithi081:> import sys 2023-11-08T16:37:45.359 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:37:45.360 DEBUG:teuthology.orchestra.run.smithi081:> try: 2023-11-08T16:37:45.360 DEBUG:teuthology.orchestra.run.smithi081:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-11-08T16:37:45.361 DEBUG:teuthology.orchestra.run.smithi081:> except OSError as e: 2023-11-08T16:37:45.361 DEBUG:teuthology.orchestra.run.smithi081:> sys.exit(e.errno) 2023-11-08T16:37:45.361 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:37:45.362 DEBUG:teuthology.orchestra.run.smithi081:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-08T16:37:45.362 DEBUG:teuthology.orchestra.run.smithi081:> print(json.dumps( 2023-11-08T16:37:45.362 DEBUG:teuthology.orchestra.run.smithi081:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-08T16:37:45.363 DEBUG:teuthology.orchestra.run.smithi081:> indent=2)) 2023-11-08T16:37:45.363 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:37:45.510 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:37:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: "st_mode": 33188, 2023-11-08T16:37:45.510 INFO:teuthology.orchestra.run.smithi081.stdout: "st_ino": 1099511627777, 2023-11-08T16:37:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "st_dev": 40, 2023-11-08T16:37:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "st_nlink": 1, 2023-11-08T16:37:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "st_uid": 1000, 2023-11-08T16:37:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "st_gid": 1301, 2023-11-08T16:37:45.511 INFO:teuthology.orchestra.run.smithi081.stdout: "st_size": 6291456, 2023-11-08T16:37:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "st_atime": 1699461465.1945913, 2023-11-08T16:37:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "st_mtime": 1699461465.293022, 2023-11-08T16:37:45.512 INFO:teuthology.orchestra.run.smithi081.stdout: "st_ctime": 1699461465.293022 2023-11-08T16:37:45.512 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:37:45.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:37:45.523 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:37:45.524 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:37:45.621 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:45.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.832+0000 7fbf05390640 1 Processor -- start 2023-11-08T16:37:45.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.833+0000 7fbf05390640 1 -- start start 2023-11-08T16:37:45.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.834+0000 7fbf05390640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbf0015d2b0 0x7fbf00161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:45.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.834+0000 7fbf05390640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf000c9060 0x7fbf000cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:45.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.834+0000 7fbf05390640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf00161c50 con 0x7fbf000c94e0 2023-11-08T16:37:45.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.834+0000 7fbf05390640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf0015d690 con 0x7fbf0015d2b0 2023-11-08T16:37:45.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.834+0000 7fbf05390640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0006b980 con 0x7fbf000c9060 2023-11-08T16:37:45.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.835+0000 7fbefeffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbf0015d2b0 0x7fbf00161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:45.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.835+0000 7fbeff7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbf000c94e0 0x7fbf0006ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47710/0 (socket says 172.21.15.81:47710) 2023-11-08T16:37:45.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.835+0000 7fbeff7fe640 1 -- 172.21.15.81:0/1313651196 learned_addr learned my addr 172.21.15.81:0/1313651196 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:45.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.835+0000 7fbefeffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbf0015d2b0 0x7fbf00161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39200/0 (socket says 172.21.15.81:39200) 2023-11-08T16:37:45.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.835+0000 7fbef67fc640 1 --2- 172.21.15.81:0/1313651196 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf000c9060 0x7fbf000cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:45.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.835+0000 7fbefeffd640 1 -- 172.21.15.81:0/1313651196 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf000c9060 msgr2=0x7fbf000cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:45.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.835+0000 7fbefeffd640 1 --2- 172.21.15.81:0/1313651196 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf000c9060 0x7fbf000cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:45.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.835+0000 7fbefeffd640 1 -- 172.21.15.81:0/1313651196 >> v1:172.21.15.81:6789/0 conn(0x7fbf000c94e0 legacy=0x7fbf0006ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:45.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.835+0000 7fbefeffd640 1 -- 172.21.15.81:0/1313651196 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0006bc60 con 0x7fbf0015d2b0 2023-11-08T16:37:45.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.835+0000 7fbefe7fc640 1 -- 172.21.15.81:0/1313651196 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761792207 0 0) 0x7fbef8001110 con 0x7fbf000c94e0 2023-11-08T16:37:45.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.836+0000 7fbefeffd640 1 --2- 172.21.15.81:0/1313651196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf0015d2b0 0x7fbf00161710 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009d90 tx=0x7fbef002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b41ff9986e0c2aa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:45.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.837+0000 7fbefe7fc640 1 -- 172.21.15.81:0/1313651196 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef002cc50 con 0x7fbf0015d2b0 2023-11-08T16:37:45.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.837+0000 7fbefe7fc640 1 -- 172.21.15.81:0/1313651196 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef002cdb0 con 0x7fbf0015d2b0 2023-11-08T16:37:45.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.837+0000 7fbefe7fc640 1 -- 172.21.15.81:0/1313651196 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0034660 con 0x7fbf0015d2b0 2023-11-08T16:37:45.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.837+0000 7fbf05390640 1 -- 172.21.15.81:0/1313651196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf0015d2b0 msgr2=0x7fbf00161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:45.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.837+0000 7fbf05390640 1 --2- 172.21.15.81:0/1313651196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf0015d2b0 0x7fbf00161710 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fbef0009d90 tx=0x7fbef002bea0 comp rx=0 tx=0).stop 2023-11-08T16:37:45.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.837+0000 7fbf05390640 1 -- 172.21.15.81:0/1313651196 shutdown_connections 2023-11-08T16:37:45.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.837+0000 7fbf05390640 1 --2- 172.21.15.81:0/1313651196 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf000c9060 0x7fbf000cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:45.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.837+0000 7fbf05390640 1 --2- 172.21.15.81:0/1313651196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf0015d2b0 0x7fbf00161710 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:45.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.837+0000 7fbf05390640 1 -- 172.21.15.81:0/1313651196 >> 172.21.15.81:0/1313651196 conn(0x7fbf000cb490 msgr2=0x7fbf000cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:45.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.837+0000 7fbf05390640 1 -- 172.21.15.81:0/1313651196 shutdown_connections 2023-11-08T16:37:45.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.837+0000 7fbf05390640 1 -- 172.21.15.81:0/1313651196 wait complete. 2023-11-08T16:37:45.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.839+0000 7fbf05390640 1 Processor -- start 2023-11-08T16:37:45.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.839+0000 7fbf05390640 1 -- start start 2023-11-08T16:37:45.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.840+0000 7fbf05390640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf000c9060 0x7fbf0015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:45.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.840+0000 7fbf05390640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000c94e0 0x7fbf000ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:45.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.840+0000 7fbf05390640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf0015d2b0 0x7fbf000cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:45.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.840+0000 7fbf05390640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000c4320 con 0x7fbf000c94e0 2023-11-08T16:37:45.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.840+0000 7fbf05390640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf000b1700 con 0x7fbf0015d2b0 2023-11-08T16:37:45.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.840+0000 7fbf05390640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0015f6b0 con 0x7fbf000c9060 2023-11-08T16:37:45.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.840+0000 7fbef67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000c94e0 0x7fbf000ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:45.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.840+0000 7fbef67fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000c94e0 0x7fbf000ca730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39208/0 (socket says 172.21.15.81:39208) 2023-11-08T16:37:45.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.840+0000 7fbef67fc640 1 -- 172.21.15.81:0/3590246522 learned_addr learned my addr 172.21.15.81:0/3590246522 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:45.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbefeffd640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf000c9060 0x7fbf0015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:45.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbeff7fe640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf0015d2b0 0x7fbf000cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:45.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbef67fc640 1 -- 172.21.15.81:0/3590246522 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf000c9060 msgr2=0x7fbf0015f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:45.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbef67fc640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf000c9060 0x7fbf0015f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:45.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbef67fc640 1 -- 172.21.15.81:0/3590246522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf0015d2b0 msgr2=0x7fbf000cac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:45.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbef67fc640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf0015d2b0 0x7fbf000cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:45.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbef67fc640 1 -- 172.21.15.81:0/3590246522 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef0009a70 con 0x7fbf000c94e0 2023-11-08T16:37:45.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbefeffd640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf000c9060 0x7fbf0015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:45.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbef67fc640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000c94e0 0x7fbf000ca730 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c7f0 tx=0x7fbee800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:45.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbef7fff640 1 -- 172.21.15.81:0/3590246522 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8004220 con 0x7fbf000c94e0 2023-11-08T16:37:45.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbef7fff640 1 -- 172.21.15.81:0/3590246522 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbee8004380 con 0x7fbf000c94e0 2023-11-08T16:37:45.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbf05390640 1 -- 172.21.15.81:0/3590246522 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0015f990 con 0x7fbf000c94e0 2023-11-08T16:37:45.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbef7fff640 1 -- 172.21.15.81:0/3590246522 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbee8007840 con 0x7fbf000c94e0 2023-11-08T16:37:45.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.841+0000 7fbf05390640 1 -- 172.21.15.81:0/3590246522 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0015ff50 con 0x7fbf000c94e0 2023-11-08T16:37:45.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.843+0000 7fbf05390640 1 -- 172.21.15.81:0/3590246522 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0004ece0 con 0x7fbf000c94e0 2023-11-08T16:37:45.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.844+0000 7fbef7fff640 1 -- 172.21.15.81:0/3590246522 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbee8007bb0 con 0x7fbf000c94e0 2023-11-08T16:37:45.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.844+0000 7fbef7fff640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee006ae70 0x7fbee006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:45.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.845+0000 7fbef7fff640 1 -- 172.21.15.81:0/3590246522 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fbee80896d0 con 0x7fbf000c94e0 2023-11-08T16:37:45.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.845+0000 7fbefeffd640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee006ae70 0x7fbee006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:45.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.845+0000 7fbefeffd640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee006ae70 0x7fbee006d330 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbef0036000 tx=0x7fbef0030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:45.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:45.849+0000 7fbef7fff640 1 -- 172.21.15.81:0/3590246522 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbee80545b0 con 0x7fbf000c94e0 2023-11-08T16:37:46.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.040+0000 7fbf05390640 1 -- 172.21.15.81:0/3590246522 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf00161a30 con 0x7fbf000c94e0 2023-11-08T16:37:46.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.043+0000 7fbef7fff640 1 -- 172.21.15.81:0/3590246522 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+18383 (secure 0 0 0) 0x7fbee8007e60 con 0x7fbf000c94e0 2023-11-08T16:37:46.045 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:46.046 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":84,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:37:45.002417+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:37:30.174436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T16:37:31.340720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:37:42.279296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2e","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 -- 172.21.15.81:0/3590246522 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee006ae70 msgr2=0x7fbee006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee006ae70 0x7fbee006d330 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbef0036000 tx=0x7fbef0030000 comp rx=0 tx=0).stop 2023-11-08T16:37:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 -- 172.21.15.81:0/3590246522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000c94e0 msgr2=0x7fbf000ca730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000c94e0 0x7fbf000ca730 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fbee800c7f0 tx=0x7fbee800ccb0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 -- 172.21.15.81:0/3590246522 shutdown_connections 2023-11-08T16:37:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee006ae70 0x7fbee006d330 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf0015d2b0 0x7fbf000cac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000c94e0 0x7fbf000ca730 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 --2- 172.21.15.81:0/3590246522 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf000c9060 0x7fbf0015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 -- 172.21.15.81:0/3590246522 >> 172.21.15.81:0/3590246522 conn(0x7fbf000cb490 msgr2=0x7fbf0006a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:46.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 -- 172.21.15.81:0/3590246522 shutdown_connections 2023-11-08T16:37:46.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.046+0000 7fbf05390640 1 -- 172.21.15.81:0/3590246522 wait complete. 2023-11-08T16:37:46.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:37:30.174436+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:37:31.340720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:37:42.279296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:37:46.068 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:37:46.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.281+0000 7ff73ce90640 1 Processor -- start 2023-11-08T16:37:46.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.281+0000 7ff73ce90640 1 -- start start 2023-11-08T16:37:46.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.283+0000 7ff73ce90640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff7380ce0d0 0x7ff7380caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.283+0000 7ff73ce90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815bfc0 0x7ff73815eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.283+0000 7ff73ce90640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73806fa30 con 0x7ff73815f0b0 2023-11-08T16:37:46.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.283+0000 7ff73ce90640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73815f540 con 0x7ff7380ce0d0 2023-11-08T16:37:46.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.283+0000 7ff73ce90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380cda10 con 0x7ff73815bfc0 2023-11-08T16:37:46.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.283+0000 7ff72ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815bfc0 0x7ff73815eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.283+0000 7ff73659c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff7380ce0d0 0x7ff7380caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.283+0000 7ff72ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815bfc0 0x7ff73815eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:42930/0 (socket says 172.21.15.81:42930) 2023-11-08T16:37:46.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.283+0000 7ff72ffff640 1 -- 172.21.15.81:0/4120306041 learned_addr learned my addr 172.21.15.81:0/4120306041 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:46.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.284+0000 7ff72ffff640 1 -- 172.21.15.81:0/4120306041 >> v2:172.21.15.93:3300/0 conn(0x7ff7380ce0d0 msgr2=0x7ff7380caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:46.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.284+0000 7ff72ffff640 1 --2- 172.21.15.81:0/4120306041 >> v2:172.21.15.93:3300/0 conn(0x7ff7380ce0d0 0x7ff7380caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.284+0000 7ff72ffff640 1 -- 172.21.15.81:0/4120306041 >> v1:172.21.15.93:6789/0 conn(0x7ff73815f0b0 legacy=0x7ff7381516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.284+0000 7ff72ffff640 1 -- 172.21.15.81:0/4120306041 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7380ce4b0 con 0x7ff73815bfc0 2023-11-08T16:37:46.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.286+0000 7ff72ffff640 1 --2- 172.21.15.81:0/4120306041 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815bfc0 0x7ff73815eb70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff720009870 tx=0x7ff72002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25c99dbe18007cfb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:46.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.286+0000 7ff735d9b640 1 -- 172.21.15.81:0/4120306041 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720030840 con 0x7ff73815bfc0 2023-11-08T16:37:46.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.286+0000 7ff735d9b640 1 -- 172.21.15.81:0/4120306041 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7200309a0 con 0x7ff73815bfc0 2023-11-08T16:37:46.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.286+0000 7ff735d9b640 1 -- 172.21.15.81:0/4120306041 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720030d30 con 0x7ff73815bfc0 2023-11-08T16:37:46.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.287+0000 7ff73ce90640 1 -- 172.21.15.81:0/4120306041 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815bfc0 msgr2=0x7ff73815eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.287+0000 7ff73ce90640 1 --2- 172.21.15.81:0/4120306041 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815bfc0 0x7ff73815eb70 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff720009870 tx=0x7ff72002b980 comp rx=0 tx=0).stop 2023-11-08T16:37:46.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.287+0000 7ff73ce90640 1 -- 172.21.15.81:0/4120306041 shutdown_connections 2023-11-08T16:37:46.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.287+0000 7ff73ce90640 1 --2- 172.21.15.81:0/4120306041 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815bfc0 0x7ff73815eb70 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.287+0000 7ff73ce90640 1 --2- 172.21.15.81:0/4120306041 >> v2:172.21.15.93:3300/0 conn(0x7ff7380ce0d0 0x7ff7380caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.287+0000 7ff73ce90640 1 -- 172.21.15.81:0/4120306041 >> 172.21.15.81:0/4120306041 conn(0x7ff7380cb580 msgr2=0x7ff7380cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:46.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.287+0000 7ff73ce90640 1 -- 172.21.15.81:0/4120306041 shutdown_connections 2023-11-08T16:37:46.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.287+0000 7ff73ce90640 1 -- 172.21.15.81:0/4120306041 wait complete. 2023-11-08T16:37:46.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.289+0000 7ff73ce90640 1 Processor -- start 2023-11-08T16:37:46.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.289+0000 7ff73ce90640 1 -- start start 2023-11-08T16:37:46.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.290+0000 7ff73ce90640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7380ce0d0 0x7ff7380c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.290+0000 7ff73ce90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff73815bfc0 0x7ff738158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.290+0000 7ff73ce90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815f0b0 0x7ff738156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.290+0000 7ff73ce90640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7381f49d0 con 0x7ff73815bfc0 2023-11-08T16:37:46.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.290+0000 7ff73ce90640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7380c79c0 con 0x7ff7380ce0d0 2023-11-08T16:37:46.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.290+0000 7ff73ce90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7381572d0 con 0x7ff73815f0b0 2023-11-08T16:37:46.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.290+0000 7ff72ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff73815bfc0 0x7ff738158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.290+0000 7ff72ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff73815bfc0 0x7ff738158bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39224/0 (socket says 172.21.15.81:39224) 2023-11-08T16:37:46.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.290+0000 7ff72ffff640 1 -- 172.21.15.81:0/3499379364 learned_addr learned my addr 172.21.15.81:0/3499379364 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:46.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.291+0000 7ff73659c640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7380ce0d0 0x7ff7380c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.291+0000 7ff736d9d640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815f0b0 0x7ff738156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.291+0000 7ff72ffff640 1 -- 172.21.15.81:0/3499379364 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815f0b0 msgr2=0x7ff738156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.291+0000 7ff72ffff640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815f0b0 0x7ff738156760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.291+0000 7ff72ffff640 1 -- 172.21.15.81:0/3499379364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7380ce0d0 msgr2=0x7ff7380c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.291+0000 7ff72ffff640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7380ce0d0 0x7ff7380c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.291+0000 7ff72ffff640 1 -- 172.21.15.81:0/3499379364 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200094d0 con 0x7ff73815bfc0 2023-11-08T16:37:46.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.291+0000 7ff72ffff640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff73815bfc0 0x7ff738158bf0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff720000c00 tx=0x7ff720004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:46.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.292+0000 7ff72effd640 1 -- 172.21.15.81:0/3499379364 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720039c90 con 0x7ff73815bfc0 2023-11-08T16:37:46.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.292+0000 7ff72effd640 1 -- 172.21.15.81:0/3499379364 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff720039e10 con 0x7ff73815bfc0 2023-11-08T16:37:46.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.292+0000 7ff72effd640 1 -- 172.21.15.81:0/3499379364 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff720043640 con 0x7ff73815bfc0 2023-11-08T16:37:46.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.292+0000 7ff73ce90640 1 -- 172.21.15.81:0/3499379364 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7381575b0 con 0x7ff73815bfc0 2023-11-08T16:37:46.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.292+0000 7ff73ce90640 1 -- 172.21.15.81:0/3499379364 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7380c7f40 con 0x7ff73815bfc0 2023-11-08T16:37:46.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.293+0000 7ff73ce90640 1 -- 172.21.15.81:0/3499379364 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73804ece0 con 0x7ff73815bfc0 2023-11-08T16:37:46.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.294+0000 7ff72effd640 1 -- 172.21.15.81:0/3499379364 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff7200437a0 con 0x7ff73815bfc0 2023-11-08T16:37:46.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.295+0000 7ff72effd640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff71806ae70 0x7ff71806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.295+0000 7ff72effd640 1 -- 172.21.15.81:0/3499379364 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7ff72003e070 con 0x7ff73815bfc0 2023-11-08T16:37:46.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.295+0000 7ff73659c640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff71806ae70 0x7ff71806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.296+0000 7ff73659c640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff71806ae70 0x7ff71806d330 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff728009ad0 tx=0x7ff728008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:46.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.299+0000 7ff72effd640 1 -- 172.21.15.81:0/3499379364 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7200776a0 con 0x7ff73815bfc0 2023-11-08T16:37:46.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.487+0000 7ff73ce90640 1 -- 172.21.15.81:0/3499379364 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff73815afc0 con 0x7ff73815bfc0 2023-11-08T16:37:46.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.488+0000 7ff72effd640 1 -- 172.21.15.81:0/3499379364 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff72007acc0 con 0x7ff73815bfc0 2023-11-08T16:37:46.489 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:37:46.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.490+0000 7ff73ce90640 1 -- 172.21.15.81:0/3499379364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff71806ae70 msgr2=0x7ff71806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.490+0000 7ff73ce90640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff71806ae70 0x7ff71806d330 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff728009ad0 tx=0x7ff728008040 comp rx=0 tx=0).stop 2023-11-08T16:37:46.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.490+0000 7ff73ce90640 1 -- 172.21.15.81:0/3499379364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff73815bfc0 msgr2=0x7ff738158bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.490+0000 7ff73ce90640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff73815bfc0 0x7ff738158bf0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff720000c00 tx=0x7ff720004730 comp rx=0 tx=0).stop 2023-11-08T16:37:46.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.490+0000 7ff73ce90640 1 -- 172.21.15.81:0/3499379364 shutdown_connections 2023-11-08T16:37:46.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.490+0000 7ff73ce90640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff71806ae70 0x7ff71806d330 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.490+0000 7ff73ce90640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff73815f0b0 0x7ff738156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.490+0000 7ff73ce90640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff73815bfc0 0x7ff738158bf0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.490+0000 7ff73ce90640 1 --2- 172.21.15.81:0/3499379364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7380ce0d0 0x7ff7380c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.490+0000 7ff73ce90640 1 -- 172.21.15.81:0/3499379364 >> 172.21.15.81:0/3499379364 conn(0x7ff7380cb580 msgr2=0x7ff7380b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:46.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.491+0000 7ff73ce90640 1 -- 172.21.15.81:0/3499379364 shutdown_connections 2023-11-08T16:37:46.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.491+0000 7ff73ce90640 1 -- 172.21.15.81:0/3499379364 wait complete. 2023-11-08T16:37:46.511 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:37:46.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.725+0000 7f0ef9a52640 1 Processor -- start 2023-11-08T16:37:46.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.725+0000 7f0ef9a52640 1 -- start start 2023-11-08T16:37:46.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.727+0000 7f0ef9a52640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef415e1a0 0x7f0ef40caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.727+0000 7f0ef9a52640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef415e580 con 0x7f0ef40c97b0 2023-11-08T16:37:46.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.727+0000 7f0ef9a52640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef40c9cb0 con 0x7f0ef4070530 2023-11-08T16:37:46.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.727+0000 7f0ef9a52640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef40ca830 con 0x7f0ef415e1a0 2023-11-08T16:37:46.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.727+0000 7f0ef3fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f0ef40c97b0 0x7f0ef406fe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47712/0 (socket says 172.21.15.81:47712) 2023-11-08T16:37:46.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.727+0000 7f0ef3fff640 1 -- 172.21.15.81:0/628146880 learned_addr learned my addr 172.21.15.81:0/628146880 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:46.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.727+0000 7f0ef8a50640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef415e1a0 0x7f0ef40caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.727+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2848321664 0 0) 0x7f0ef415e580 con 0x7f0ef40c97b0 2023-11-08T16:37:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ee4003520 con 0x7f0ef40c97b0 2023-11-08T16:37:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2106956342 0 0) 0x7f0ee4003520 con 0x7f0ef40c97b0 2023-11-08T16:37:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef415e1a0 msgr2=0x7f0ef40caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef415e1a0 0x7f0ef40caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 >> v1:172.21.15.93:6789/0 conn(0x7f0ef4070530 legacy=0x7f0ef406bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef40c9f90 con 0x7f0ef40c97b0 2023-11-08T16:37:46.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f0ee00017f0 con 0x7f0ef40c97b0 2023-11-08T16:37:46.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 >> v1:172.21.15.81:6789/0 conn(0x7f0ef40c97b0 legacy=0x7f0ef406fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef415e1a0 0x7f0ee40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ee4003f20 0x7f0ee40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee4004d00 0x7f0ee4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef40ca830 con 0x7f0ee4004d00 2023-11-08T16:37:46.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee400bb10 con 0x7f0ee4003f20 2023-11-08T16:37:46.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee400bcd0 con 0x7f0ef415e1a0 2023-11-08T16:37:46.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.728+0000 7f0ef9251640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee4004d00 0x7f0ee4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.729+0000 7f0ef8a50640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef415e1a0 0x7f0ee40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.729+0000 7f0ef3fff640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ee4003f20 0x7f0ee40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.729+0000 7f0ef9251640 1 -- 172.21.15.81:0/628146880 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef415e1a0 msgr2=0x7f0ee40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:46.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.729+0000 7f0ef9251640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef415e1a0 0x7f0ee40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.729+0000 7f0ef9251640 1 -- 172.21.15.81:0/628146880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ee4003f20 msgr2=0x7f0ee40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:46.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.729+0000 7f0ef9251640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ee4003f20 0x7f0ee40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.729+0000 7f0ef9251640 1 -- 172.21.15.81:0/628146880 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee4003520 con 0x7f0ee4004d00 2023-11-08T16:37:46.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.730+0000 7f0ef9251640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee4004d00 0x7f0ee4005180 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0ef406fa80 tx=0x7f0ee40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967d4050181fed08 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:46.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.730+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec003b70 con 0x7f0ee4004d00 2023-11-08T16:37:46.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.731+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec003cd0 con 0x7f0ee4004d00 2023-11-08T16:37:46.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.731+0000 7f0ef37fe640 1 -- 172.21.15.81:0/628146880 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec005310 con 0x7f0ee4004d00 2023-11-08T16:37:46.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.731+0000 7f0ef9a52640 1 -- 172.21.15.81:0/628146880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee4004d00 msgr2=0x7f0ee4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.731+0000 7f0ef9a52640 1 --2- 172.21.15.81:0/628146880 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ee4004d00 0x7f0ee4005180 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f0ef406fa80 tx=0x7f0ee40057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.731+0000 7f0ef9251640 1 -- 172.21.15.81:0/628146880 reap_dead start 2023-11-08T16:37:46.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.731+0000 7f0ef9a52640 1 -- 172.21.15.81:0/628146880 shutdown_connections 2023-11-08T16:37:46.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.731+0000 7f0ef9a52640 1 -- 172.21.15.81:0/628146880 >> 172.21.15.81:0/628146880 conn(0x7f0ef40ce8d0 msgr2=0x7f0ef40cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:46.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.731+0000 7f0ef9a52640 1 -- 172.21.15.81:0/628146880 shutdown_connections 2023-11-08T16:37:46.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.731+0000 7f0ef9a52640 1 -- 172.21.15.81:0/628146880 wait complete. 2023-11-08T16:37:46.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.733+0000 7f0ef9a52640 1 Processor -- start 2023-11-08T16:37:46.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.733+0000 7f0ef9a52640 1 -- start start 2023-11-08T16:37:46.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.734+0000 7f0ef9a52640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef4144350 0x7f0ef4158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.734+0000 7f0ef9a52640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ef4158ed0 0x7f0ef40c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.734+0000 7f0ef9a52640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ef40c7ba0 0x7f0ef4150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.734+0000 7f0ef9a52640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4159350 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.734+0000 7f0ef9a52640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef40c8400 con 0x7f0ef4158ed0 2023-11-08T16:37:46.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.734+0000 7f0ef9a52640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef40c59b0 con 0x7f0ef4144350 2023-11-08T16:37:46.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.734+0000 7f0ef9251640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ef40c7ba0 0x7f0ef4150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.734+0000 7f0ef9251640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ef40c7ba0 0x7f0ef4150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39254/0 (socket says 172.21.15.81:39254) 2023-11-08T16:37:46.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.734+0000 7f0ef9251640 1 -- 172.21.15.81:0/1916842941 learned_addr learned my addr 172.21.15.81:0/1916842941 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:46.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.735+0000 7f0ef8a50640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef4144350 0x7f0ef4158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.735+0000 7f0ef3fff640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ef4158ed0 0x7f0ef40c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.735+0000 7f0ef9251640 1 -- 172.21.15.81:0/1916842941 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef4144350 msgr2=0x7f0ef4158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.735+0000 7f0ef9251640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef4144350 0x7f0ef4158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.735+0000 7f0ef9251640 1 -- 172.21.15.81:0/1916842941 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ef4158ed0 msgr2=0x7f0ef40c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.735+0000 7f0ef9251640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ef4158ed0 0x7f0ef40c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.735+0000 7f0ef9251640 1 -- 172.21.15.81:0/1916842941 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef40c5c90 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.735+0000 7f0ef9251640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ef40c7ba0 0x7f0ef4150de0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0eec02c020 tx=0x7f0eec0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:46.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.736+0000 7f0ef17fa640 1 -- 172.21.15.81:0/1916842941 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec03e070 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.736+0000 7f0ef17fa640 1 -- 172.21.15.81:0/1916842941 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec007db0 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.736+0000 7f0ef17fa640 1 -- 172.21.15.81:0/1916842941 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec006d50 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.736+0000 7f0ef9a52640 1 -- 172.21.15.81:0/1916842941 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef40c5f70 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.736+0000 7f0ef9a52640 1 -- 172.21.15.81:0/1916842941 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef4144730 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.737+0000 7f0ef9a52640 1 -- 172.21.15.81:0/1916842941 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef41592b0 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.738+0000 7f0ef17fa640 1 -- 172.21.15.81:0/1916842941 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0eec007030 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.739+0000 7f0ef17fa640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0ed806aea0 0x7f0ed806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:46.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.739+0000 7f0ef17fa640 1 -- 172.21.15.81:0/1916842941 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f0eec0aa970 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.739+0000 7f0ef8a50640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0ed806aea0 0x7f0ed806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:46.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.740+0000 7f0ef8a50640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0ed806aea0 0x7f0ed806d360 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009b00 tx=0x7f0ee8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:46.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.743+0000 7f0ef17fa640 1 -- 172.21.15.81:0/1916842941 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0eec075720 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.933+0000 7f0ef9a52640 1 -- 172.21.15.81:0/1916842941 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ef404ece0 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.934+0000 7f0ef17fa640 1 -- 172.21.15.81:0/1916842941 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0eec048070 con 0x7f0ef40c7ba0 2023-11-08T16:37:46.936 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:37:46.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 -- 172.21.15.81:0/1916842941 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0ed806aea0 msgr2=0x7f0ed806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0ed806aea0 0x7f0ed806d360 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009b00 tx=0x7f0ee8008040 comp rx=0 tx=0).stop 2023-11-08T16:37:46.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 -- 172.21.15.81:0/1916842941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ef40c7ba0 msgr2=0x7f0ef4150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:46.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ef40c7ba0 0x7f0ef4150de0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f0eec02c020 tx=0x7f0eec0014a0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 -- 172.21.15.81:0/1916842941 shutdown_connections 2023-11-08T16:37:46.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0ed806aea0 0x7f0ed806d360 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ef40c7ba0 0x7f0ef4150de0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ef4158ed0 0x7f0ef40c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 --2- 172.21.15.81:0/1916842941 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ef4144350 0x7f0ef4158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:46.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 -- 172.21.15.81:0/1916842941 >> 172.21.15.81:0/1916842941 conn(0x7f0ef40ce8d0 msgr2=0x7f0ef40cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:46.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 -- 172.21.15.81:0/1916842941 shutdown_connections 2023-11-08T16:37:46.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:46.937+0000 7f0ef9a52640 1 -- 172.21.15.81:0/1916842941 wait complete. 2023-11-08T16:37:46.955 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:37:47.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.169+0000 7f2c89d0f640 1 Processor -- start 2023-11-08T16:37:47.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.169+0000 7f2c89d0f640 1 -- start start 2023-11-08T16:37:47.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.170+0000 7f2c89d0f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2c840ccc10 0x7f2c840ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.170+0000 7f2c89d0f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c840ce960 con 0x7f2c8415a0e0 2023-11-08T16:37:47.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.170+0000 7f2c89d0f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c840c9480 con 0x7f2c8406b710 2023-11-08T16:37:47.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.170+0000 7f2c89d0f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c84071980 con 0x7f2c840ccc10 2023-11-08T16:37:47.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.171+0000 7f2c837fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2c8415a0e0 0x7f2c840c9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47716/0 (socket says 172.21.15.81:47716) 2023-11-08T16:37:47.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.171+0000 7f2c837fe640 1 -- 172.21.15.81:0/2662878023 learned_addr learned my addr 172.21.15.81:0/2662878023 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:47.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.171+0000 7f2c7bfff640 1 --2- 172.21.15.81:0/2662878023 >> v2:172.21.15.93:3300/0 conn(0x7f2c840ccc10 0x7f2c840ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.171+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237719937 0 0) 0x7f2c74001110 con 0x7f2c8415a0e0 2023-11-08T16:37:47.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.171+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c70003520 con 0x7f2c8415a0e0 2023-11-08T16:37:47.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2743477144 0 0) 0x7f2c70003520 con 0x7f2c8415a0e0 2023-11-08T16:37:47.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 >> v2:172.21.15.93:3300/0 conn(0x7f2c840ccc10 msgr2=0x7f2c840ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:47.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 --2- 172.21.15.81:0/2662878023 >> v2:172.21.15.93:3300/0 conn(0x7f2c840ccc10 0x7f2c840ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 >> v1:172.21.15.93:6789/0 conn(0x7f2c8406b710 legacy=0x7f2c8406bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8415a4c0 con 0x7f2c8415a0e0 2023-11-08T16:37:47.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f2c74001800 con 0x7f2c8415a0e0 2023-11-08T16:37:47.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 >> v1:172.21.15.81:6789/0 conn(0x7f2c8415a0e0 legacy=0x7f2c840c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c700039e0 0x7f2c70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c700047c0 0x7f2c70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c70005180 0x7f2c7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c84071980 con 0x7f2c70005180 2023-11-08T16:37:47.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7000c090 con 0x7f2c700039e0 2023-11-08T16:37:47.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7000c250 con 0x7f2c700047c0 2023-11-08T16:37:47.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.172+0000 7f2c83fff640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c70005180 0x7f2c7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.173+0000 7f2c83fff640 1 -- 172.21.15.81:0/2662878023 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c700047c0 msgr2=0x7f2c70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:47.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.173+0000 7f2c837fe640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c700039e0 0x7f2c70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.173+0000 7f2c7bfff640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c700047c0 0x7f2c70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.173+0000 7f2c83fff640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c700047c0 0x7f2c70004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.173+0000 7f2c83fff640 1 -- 172.21.15.81:0/2662878023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c700039e0 msgr2=0x7f2c70003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:47.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.173+0000 7f2c83fff640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c700039e0 0x7f2c70003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.173+0000 7f2c83fff640 1 -- 172.21.15.81:0/2662878023 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c70003520 con 0x7f2c70005180 2023-11-08T16:37:47.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.173+0000 7f2c837fe640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c700039e0 0x7f2c70003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:47.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.174+0000 7f2c83fff640 1 -- 172.21.15.81:0/2662878023 reap_dead start 2023-11-08T16:37:47.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.174+0000 7f2c83fff640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c70005180 0x7f2c7000b980 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c001e50 tx=0x7f2c700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5c6f4a3faf2008 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:47.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.174+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c003b70 con 0x7f2c70005180 2023-11-08T16:37:47.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.174+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7c003d10 con 0x7f2c70005180 2023-11-08T16:37:47.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.174+0000 7f2c82ffd640 1 -- 172.21.15.81:0/2662878023 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c005310 con 0x7f2c70005180 2023-11-08T16:37:47.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.174+0000 7f2c89d0f640 1 -- 172.21.15.81:0/2662878023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c70005180 msgr2=0x7f2c7000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.174+0000 7f2c89d0f640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c70005180 0x7f2c7000b980 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c001e50 tx=0x7f2c700057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.175+0000 7f2c89d0f640 1 -- 172.21.15.81:0/2662878023 shutdown_connections 2023-11-08T16:37:47.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.175+0000 7f2c89d0f640 1 --2- 172.21.15.81:0/2662878023 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c70005180 0x7f2c7000b980 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.175+0000 7f2c89d0f640 1 -- 172.21.15.81:0/2662878023 >> 172.21.15.81:0/2662878023 conn(0x7f2c840cb580 msgr2=0x7f2c840cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:47.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.175+0000 7f2c89d0f640 1 -- 172.21.15.81:0/2662878023 shutdown_connections 2023-11-08T16:37:47.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.175+0000 7f2c89d0f640 1 -- 172.21.15.81:0/2662878023 wait complete. 2023-11-08T16:37:47.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.176+0000 7f2c89d0f640 1 Processor -- start 2023-11-08T16:37:47.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.176+0000 7f2c89d0f640 1 -- start start 2023-11-08T16:37:47.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.177+0000 7f2c89d0f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c70005180 0x7f2c8415a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.177+0000 7f2c89d0f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c8415c950 0x7f2c840c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.177+0000 7f2c89d0f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c840b3c80 0x7f2c840b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.177+0000 7f2c89d0f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840c6850 con 0x7f2c8415c950 2023-11-08T16:37:47.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.177+0000 7f2c89d0f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840b4640 con 0x7f2c840b3c80 2023-11-08T16:37:47.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.177+0000 7f2c89d0f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8415cdd0 con 0x7f2c70005180 2023-11-08T16:37:47.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c7bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c8415c950 0x7f2c840c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c7bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c8415c950 0x7f2c840c8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39272/0 (socket says 172.21.15.81:39272) 2023-11-08T16:37:47.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c7bfff640 1 -- 172.21.15.81:0/1906493580 learned_addr learned my addr 172.21.15.81:0/1906493580 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:47.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c837fe640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c70005180 0x7f2c8415a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c83fff640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c840b3c80 0x7f2c840b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c7bfff640 1 -- 172.21.15.81:0/1906493580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c70005180 msgr2=0x7f2c8415a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c7bfff640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c70005180 0x7f2c8415a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c7bfff640 1 -- 172.21.15.81:0/1906493580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c840b3c80 msgr2=0x7f2c840b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c7bfff640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c840b3c80 0x7f2c840b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c7bfff640 1 -- 172.21.15.81:0/1906493580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c840c8e50 con 0x7f2c8415c950 2023-11-08T16:37:47.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c837fe640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c70005180 0x7f2c8415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:47.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c83fff640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c840b3c80 0x7f2c840b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:47.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.178+0000 7f2c7bfff640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c8415c950 0x7f2c840c8910 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c010890 tx=0x7f2c6c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:47.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.179+0000 7f2c80ff9640 1 -- 172.21.15.81:0/1906493580 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c009610 con 0x7f2c8415c950 2023-11-08T16:37:47.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.179+0000 7f2c80ff9640 1 -- 172.21.15.81:0/1906493580 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6c00eb60 con 0x7f2c8415c950 2023-11-08T16:37:47.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.179+0000 7f2c80ff9640 1 -- 172.21.15.81:0/1906493580 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c01c400 con 0x7f2c8415c950 2023-11-08T16:37:47.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.179+0000 7f2c89d0f640 1 -- 172.21.15.81:0/1906493580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c840c4a80 con 0x7f2c8415c950 2023-11-08T16:37:47.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.179+0000 7f2c89d0f640 1 -- 172.21.15.81:0/1906493580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8415ad00 con 0x7f2c8415c950 2023-11-08T16:37:47.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.180+0000 7f2c89d0f640 1 -- 172.21.15.81:0/1906493580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c840c65b0 con 0x7f2c8415c950 2023-11-08T16:37:47.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.181+0000 7f2c80ff9640 1 -- 172.21.15.81:0/1906493580 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2c6c01c560 con 0x7f2c8415c950 2023-11-08T16:37:47.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.182+0000 7f2c80ff9640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406aea0 0x7f2c6406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.182+0000 7f2c80ff9640 1 -- 172.21.15.81:0/1906493580 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f2c6c089d40 con 0x7f2c8415c950 2023-11-08T16:37:47.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.182+0000 7f2c837fe640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406aea0 0x7f2c6406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.183+0000 7f2c837fe640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406aea0 0x7f2c6406d360 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2c74007950 tx=0x7f2c74001110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:47.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.187+0000 7f2c80ff9640 1 -- 172.21.15.81:0/1906493580 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2c840c65b0 con 0x7f2c8415c950 2023-11-08T16:37:47.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.374+0000 7f2c89d0f640 1 -- 172.21.15.81:0/1906493580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2c840c65b0 con 0x7f2c8415c950 2023-11-08T16:37:47.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.375+0000 7f2c80ff9640 1 -- 172.21.15.81:0/1906493580 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2c840c65b0 con 0x7f2c8415c950 2023-11-08T16:37:47.377 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:37:47.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 -- 172.21.15.81:0/1906493580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406aea0 msgr2=0x7f2c6406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406aea0 0x7f2c6406d360 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2c74007950 tx=0x7f2c74001110 comp rx=0 tx=0).stop 2023-11-08T16:37:47.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 -- 172.21.15.81:0/1906493580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c8415c950 msgr2=0x7f2c840c8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c8415c950 0x7f2c840c8910 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f2c6c010890 tx=0x7f2c6c010d50 comp rx=0 tx=0).stop 2023-11-08T16:37:47.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 -- 172.21.15.81:0/1906493580 shutdown_connections 2023-11-08T16:37:47.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2c6406aea0 0x7f2c6406d360 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c840b3c80 0x7f2c840b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2c8415c950 0x7f2c840c8910 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 --2- 172.21.15.81:0/1906493580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2c70005180 0x7f2c8415a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 -- 172.21.15.81:0/1906493580 >> 172.21.15.81:0/1906493580 conn(0x7f2c840cb580 msgr2=0x7f2c84157b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:47.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 -- 172.21.15.81:0/1906493580 shutdown_connections 2023-11-08T16:37:47.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.378+0000 7f2c89d0f640 1 -- 172.21.15.81:0/1906493580 wait complete. 2023-11-08T16:37:47.397 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:37:47.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.586+0000 7ff1d5516640 1 Processor -- start 2023-11-08T16:37:47.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.586+0000 7ff1d5516640 1 -- start start 2023-11-08T16:37:47.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.588+0000 7ff1d5516640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d00ce0d0 0x7ff1d00caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.588+0000 7ff1d5516640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff1d015bfc0 0x7ff1d015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.588+0000 7ff1d5516640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d015f500 con 0x7ff1d015f0b0 2023-11-08T16:37:47.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.588+0000 7ff1d5516640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d00cda10 con 0x7ff1d015bfc0 2023-11-08T16:37:47.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.588+0000 7ff1d5516640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d006fa30 con 0x7ff1d00ce0d0 2023-11-08T16:37:47.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.588+0000 7ff1ce7fc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff1d015bfc0 0x7ff1d015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.589+0000 7ff1ce7fc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff1d015bfc0 0x7ff1d015eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:41204/0 (socket says 172.21.15.81:41204) 2023-11-08T16:37:47.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.589+0000 7ff1ce7fc640 1 -- 172.21.15.81:0/1017923524 learned_addr learned my addr 172.21.15.81:0/1017923524 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:47.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.589+0000 7ff1ceffd640 1 --2- 172.21.15.81:0/1017923524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d00ce0d0 0x7ff1d00caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.589+0000 7ff1ce7fc640 1 -- 172.21.15.81:0/1017923524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d00ce0d0 msgr2=0x7ff1d00caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:47.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.589+0000 7ff1ce7fc640 1 --2- 172.21.15.81:0/1017923524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d00ce0d0 0x7ff1d00caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.589+0000 7ff1ce7fc640 1 -- 172.21.15.81:0/1017923524 >> v1:172.21.15.93:6789/0 conn(0x7ff1d015f0b0 legacy=0x7ff1d01516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.589+0000 7ff1ce7fc640 1 -- 172.21.15.81:0/1017923524 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d00ce4b0 con 0x7ff1d015bfc0 2023-11-08T16:37:47.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.590+0000 7ff1ce7fc640 1 --2- 172.21.15.81:0/1017923524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d015bfc0 0x7ff1d015eb70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009870 tx=0x7ff1c402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=661b22cb66a7a0a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:47.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.591+0000 7ff1cdffb640 1 -- 172.21.15.81:0/1017923524 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4030840 con 0x7ff1d015bfc0 2023-11-08T16:37:47.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.591+0000 7ff1cdffb640 1 -- 172.21.15.81:0/1017923524 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c40309a0 con 0x7ff1d015bfc0 2023-11-08T16:37:47.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.591+0000 7ff1cdffb640 1 -- 172.21.15.81:0/1017923524 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4030d30 con 0x7ff1d015bfc0 2023-11-08T16:37:47.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.591+0000 7ff1d5516640 1 -- 172.21.15.81:0/1017923524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d015bfc0 msgr2=0x7ff1d015eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.591+0000 7ff1d5516640 1 --2- 172.21.15.81:0/1017923524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d015bfc0 0x7ff1d015eb70 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009870 tx=0x7ff1c402b980 comp rx=0 tx=0).stop 2023-11-08T16:37:47.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.591+0000 7ff1d5516640 1 -- 172.21.15.81:0/1017923524 shutdown_connections 2023-11-08T16:37:47.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.591+0000 7ff1d5516640 1 --2- 172.21.15.81:0/1017923524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d015bfc0 0x7ff1d015eb70 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.591+0000 7ff1d5516640 1 --2- 172.21.15.81:0/1017923524 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d00ce0d0 0x7ff1d00caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.591+0000 7ff1d5516640 1 -- 172.21.15.81:0/1017923524 >> 172.21.15.81:0/1017923524 conn(0x7ff1d00cb580 msgr2=0x7ff1d00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:47.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.592+0000 7ff1d5516640 1 -- 172.21.15.81:0/1017923524 shutdown_connections 2023-11-08T16:37:47.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.592+0000 7ff1d5516640 1 -- 172.21.15.81:0/1017923524 wait complete. 2023-11-08T16:37:47.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.593+0000 7ff1d5516640 1 Processor -- start 2023-11-08T16:37:47.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.593+0000 7ff1d5516640 1 -- start start 2023-11-08T16:37:47.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.594+0000 7ff1d5516640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d00ce0d0 0x7ff1d0146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.594+0000 7ff1d5516640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d015bfc0 0x7ff1d01f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.594+0000 7ff1d5516640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d015f0b0 0x7ff1d015a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.594+0000 7ff1d5516640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d01f5eb0 con 0x7ff1d015f0b0 2023-11-08T16:37:47.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.594+0000 7ff1d5516640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d0146b50 con 0x7ff1d00ce0d0 2023-11-08T16:37:47.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.594+0000 7ff1d5516640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d01f60c0 con 0x7ff1d015bfc0 2023-11-08T16:37:47.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.594+0000 7ff1cf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d015f0b0 0x7ff1d015a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1cf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d015f0b0 0x7ff1d015a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39284/0 (socket says 172.21.15.81:39284) 2023-11-08T16:37:47.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1cf7fe640 1 -- 172.21.15.81:0/1071711278 learned_addr learned my addr 172.21.15.81:0/1071711278 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:47.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1ceffd640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d00ce0d0 0x7ff1d0146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1cf7fe640 1 -- 172.21.15.81:0/1071711278 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d015bfc0 msgr2=0x7ff1d01f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1cf7fe640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d015bfc0 0x7ff1d01f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1cf7fe640 1 -- 172.21.15.81:0/1071711278 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d00ce0d0 msgr2=0x7ff1d0146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1cf7fe640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d00ce0d0 0x7ff1d0146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1cf7fe640 1 -- 172.21.15.81:0/1071711278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c40094d0 con 0x7ff1d015f0b0 2023-11-08T16:37:47.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1cf7fe640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d015f0b0 0x7ff1d015a050 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800c6b0 tx=0x7ff1c800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:47.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1b77fe640 1 -- 172.21.15.81:0/1071711278 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8003b70 con 0x7ff1d015f0b0 2023-11-08T16:37:47.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1b77fe640 1 -- 172.21.15.81:0/1071711278 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c8003cf0 con 0x7ff1d015f0b0 2023-11-08T16:37:47.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1d5516640 1 -- 172.21.15.81:0/1071711278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d01f63a0 con 0x7ff1d015f0b0 2023-11-08T16:37:47.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.595+0000 7ff1b77fe640 1 -- 172.21.15.81:0/1071711278 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8003ea0 con 0x7ff1d015f0b0 2023-11-08T16:37:47.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.596+0000 7ff1d5516640 1 -- 172.21.15.81:0/1071711278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d01f6960 con 0x7ff1d015f0b0 2023-11-08T16:37:47.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.597+0000 7ff1d5516640 1 -- 172.21.15.81:0/1071711278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d004ece0 con 0x7ff1d015f0b0 2023-11-08T16:37:47.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.598+0000 7ff1b77fe640 1 -- 172.21.15.81:0/1071711278 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff1c8005310 con 0x7ff1d015f0b0 2023-11-08T16:37:47.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.599+0000 7ff1b77fe640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff1bc06aea0 0x7ff1bc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:47.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.599+0000 7ff1b77fe640 1 -- 172.21.15.81:0/1071711278 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7ff1c8089470 con 0x7ff1d015f0b0 2023-11-08T16:37:47.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.599+0000 7ff1ceffd640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff1bc06aea0 0x7ff1bc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:47.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.600+0000 7ff1ceffd640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff1bc06aea0 0x7ff1bc06d360 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0006fd0 tx=0x7ff1c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:47.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.603+0000 7ff1b77fe640 1 -- 172.21.15.81:0/1071711278 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff1c80542b0 con 0x7ff1d015f0b0 2023-11-08T16:37:47.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.779+0000 7ff1d5516640 1 -- 172.21.15.81:0/1071711278 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7ff1d015f500 con 0x7ff1d015f0b0 2023-11-08T16:37:47.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.780+0000 7ff1b77fe640 1 -- 172.21.15.81:0/1071711278 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v84) v1 ==== 106+0+11 (secure 0 0 0) 0x7ff1d015f500 con 0x7ff1d015f0b0 2023-11-08T16:37:47.782 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:37:47.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.782+0000 7ff1d5516640 1 -- 172.21.15.81:0/1071711278 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff1bc06aea0 msgr2=0x7ff1bc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.782+0000 7ff1d5516640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff1bc06aea0 0x7ff1bc06d360 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0006fd0 tx=0x7ff1c0008040 comp rx=0 tx=0).stop 2023-11-08T16:37:47.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.782+0000 7ff1d5516640 1 -- 172.21.15.81:0/1071711278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d015f0b0 msgr2=0x7ff1d015a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:47.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.782+0000 7ff1d5516640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d015f0b0 0x7ff1d015a050 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800c6b0 tx=0x7ff1c800cb70 comp rx=0 tx=0).stop 2023-11-08T16:37:47.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.782+0000 7ff1d5516640 1 -- 172.21.15.81:0/1071711278 shutdown_connections 2023-11-08T16:37:47.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.783+0000 7ff1d5516640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff1bc06aea0 0x7ff1bc06d360 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.783+0000 7ff1d5516640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff1d015f0b0 0x7ff1d015a050 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.783+0000 7ff1d5516640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff1d015bfc0 0x7ff1d01f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.783+0000 7ff1d5516640 1 --2- 172.21.15.81:0/1071711278 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1d00ce0d0 0x7ff1d0146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:47.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.783+0000 7ff1d5516640 1 -- 172.21.15.81:0/1071711278 >> 172.21.15.81:0/1071711278 conn(0x7ff1d00cb580 msgr2=0x7ff1d015c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:47.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.783+0000 7ff1d5516640 1 -- 172.21.15.81:0/1071711278 shutdown_connections 2023-11-08T16:37:47.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:47.783+0000 7ff1d5516640 1 -- 172.21.15.81:0/1071711278 wait complete. 2023-11-08T16:37:47.802 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:37:48.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.018+0000 7f10b13f5640 1 Processor -- start 2023-11-08T16:37:48.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.019+0000 7f10b13f5640 1 -- start start 2023-11-08T16:37:48.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.020+0000 7f10b13f5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f10ac0cc800 0x7f10ac06ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:48.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10b13f5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f10ac0cd530 0x7f10ac0cd980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:48.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10b13f5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac0b3680 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10b13f5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac0ced60 con 0x7f10ac0cd530 2023-11-08T16:37:48.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10b13f5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac0cdec0 con 0x7f10ac0cc800 2023-11-08T16:37:48.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10ab7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f10ac0cd530 0x7f10ac0cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:48.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10ab7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f10ac0cd530 0x7f10ac0cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39296/0 (socket says 172.21.15.81:39296) 2023-11-08T16:37:48.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10ab7fe640 1 -- 172.21.15.81:0/4283219155 learned_addr learned my addr 172.21.15.81:0/4283219155 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:48.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10aaffd640 1 --2- 172.21.15.81:0/4283219155 >> v2:172.21.15.93:3300/0 conn(0x7f10ac0cc800 0x7f10ac06ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:48.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10ab7fe640 1 -- 172.21.15.81:0/4283219155 >> v2:172.21.15.93:3300/0 conn(0x7f10ac0cc800 msgr2=0x7f10ac06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:48.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10ab7fe640 1 --2- 172.21.15.81:0/4283219155 >> v2:172.21.15.93:3300/0 conn(0x7f10ac0cc800 0x7f10ac06ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10ab7fe640 1 -- 172.21.15.81:0/4283219155 >> v1:172.21.15.93:6789/0 conn(0x7f10ac0ce8d0 legacy=0x7f10ac161550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:48.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.021+0000 7f10ab7fe640 1 -- 172.21.15.81:0/4283219155 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac1610c0 con 0x7f10ac0cd530 2023-11-08T16:37:48.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.023+0000 7f10ab7fe640 1 --2- 172.21.15.81:0/4283219155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10ac0cd530 0x7f10ac0cd980 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009870 tx=0x7f10a402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fec0f324605078f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:48.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.023+0000 7f10a9ffb640 1 -- 172.21.15.81:0/4283219155 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4031870 con 0x7f10ac0cd530 2023-11-08T16:37:48.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.023+0000 7f10a9ffb640 1 -- 172.21.15.81:0/4283219155 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a40319d0 con 0x7f10ac0cd530 2023-11-08T16:37:48.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.023+0000 7f10a9ffb640 1 -- 172.21.15.81:0/4283219155 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4031d60 con 0x7f10ac0cd530 2023-11-08T16:37:48.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.024+0000 7f10b13f5640 1 -- 172.21.15.81:0/4283219155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10ac0cd530 msgr2=0x7f10ac0cd980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:48.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.024+0000 7f10b13f5640 1 --2- 172.21.15.81:0/4283219155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10ac0cd530 0x7f10ac0cd980 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009870 tx=0x7f10a402b910 comp rx=0 tx=0).stop 2023-11-08T16:37:48.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.024+0000 7f10b13f5640 1 -- 172.21.15.81:0/4283219155 shutdown_connections 2023-11-08T16:37:48.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.024+0000 7f10b13f5640 1 --2- 172.21.15.81:0/4283219155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10ac0cd530 0x7f10ac0cd980 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.024+0000 7f10b13f5640 1 --2- 172.21.15.81:0/4283219155 >> v2:172.21.15.93:3300/0 conn(0x7f10ac0cc800 0x7f10ac06ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.024+0000 7f10b13f5640 1 -- 172.21.15.81:0/4283219155 >> 172.21.15.81:0/4283219155 conn(0x7f10ac065df0 msgr2=0x7f10ac066200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:48.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.024+0000 7f10b13f5640 1 -- 172.21.15.81:0/4283219155 shutdown_connections 2023-11-08T16:37:48.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.024+0000 7f10b13f5640 1 -- 172.21.15.81:0/4283219155 wait complete. 2023-11-08T16:37:48.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.025+0000 7f10b13f5640 1 Processor -- start 2023-11-08T16:37:48.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.026+0000 7f10b13f5640 1 -- start start 2023-11-08T16:37:48.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10b13f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10ac0cc800 0x7f10ac0cbf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:48.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10b13f5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10ac0cd530 0x7f10ac15d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:48.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10b13f5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10ac0ce8d0 0x7f10ac0d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:48.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10b13f5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac149c00 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10b13f5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac15bb60 con 0x7f10ac0cd530 2023-11-08T16:37:48.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10b13f5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac15be40 con 0x7f10ac0cc800 2023-11-08T16:37:48.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10ab7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10ac0ce8d0 0x7f10ac0d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:48.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10ab7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10ac0ce8d0 0x7f10ac0d0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39312/0 (socket says 172.21.15.81:39312) 2023-11-08T16:37:48.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10ab7fe640 1 -- 172.21.15.81:0/765525761 learned_addr learned my addr 172.21.15.81:0/765525761 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:48.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10aaffd640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10ac0cc800 0x7f10ac0cbf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:48.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10aa7fc640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10ac0cd530 0x7f10ac15d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:48.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10ab7fe640 1 -- 172.21.15.81:0/765525761 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10ac0cc800 msgr2=0x7f10ac0cbf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:48.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10ab7fe640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10ac0cc800 0x7f10ac0cbf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.027+0000 7f10ab7fe640 1 -- 172.21.15.81:0/765525761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10ac0cd530 msgr2=0x7f10ac15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:48.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.028+0000 7f10ab7fe640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10ac0cd530 0x7f10ac15d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.028+0000 7f10ab7fe640 1 -- 172.21.15.81:0/765525761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40094d0 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.028+0000 7f10aaffd640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10ac0cc800 0x7f10ac0cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:48.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.028+0000 7f10ab7fe640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10ac0ce8d0 0x7f10ac0d0b60 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f10a402b950 tx=0x7f10a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:48.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.029+0000 7f10a37fe640 1 -- 172.21.15.81:0/765525761 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a403ac60 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.029+0000 7f10a37fe640 1 -- 172.21.15.81:0/765525761 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a403adc0 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.029+0000 7f10a37fe640 1 -- 172.21.15.81:0/765525761 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4044630 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.029+0000 7f10b13f5640 1 -- 172.21.15.81:0/765525761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac15c120 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.029+0000 7f10b13f5640 1 -- 172.21.15.81:0/765525761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac15c6e0 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.030+0000 7f10b13f5640 1 -- 172.21.15.81:0/765525761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac15bf80 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.031+0000 7f10a37fe640 1 -- 172.21.15.81:0/765525761 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f10a404d450 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.032+0000 7f10a37fe640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108c06ae70 0x7f108c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:48.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.032+0000 7f10a37fe640 1 -- 172.21.15.81:0/765525761 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f10a402d040 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.032+0000 7f10aaffd640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108c06ae70 0x7f108c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:48.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.033+0000 7f10aaffd640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108c06ae70 0x7f108c06d330 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f109c007330 tx=0x7f109c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:48.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.036+0000 7f10a37fe640 1 -- 172.21.15.81:0/765525761 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f10ac15bf80 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.236+0000 7f10b13f5640 1 -- 172.21.15.81:0/765525761 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f10ac15bf80 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.237+0000 7f10a37fe640 1 -- 172.21.15.81:0/765525761 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v84) v1 ==== 80+0+0 (secure 0 0 0) 0x7f10ac15bf80 con 0x7f10ac0ce8d0 2023-11-08T16:37:48.239 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:37:48.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.239+0000 7f10b13f5640 1 -- 172.21.15.81:0/765525761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108c06ae70 msgr2=0x7f108c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:48.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.239+0000 7f10b13f5640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108c06ae70 0x7f108c06d330 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f109c007330 tx=0x7f109c00b040 comp rx=0 tx=0).stop 2023-11-08T16:37:48.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.239+0000 7f10b13f5640 1 -- 172.21.15.81:0/765525761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10ac0ce8d0 msgr2=0x7f10ac0d0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:48.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.239+0000 7f10b13f5640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10ac0ce8d0 0x7f10ac0d0b60 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f10a402b950 tx=0x7f10a40046c0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.240+0000 7f10b13f5640 1 -- 172.21.15.81:0/765525761 shutdown_connections 2023-11-08T16:37:48.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.240+0000 7f10b13f5640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f108c06ae70 0x7f108c06d330 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.240+0000 7f10b13f5640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f10ac0ce8d0 0x7f10ac0d0b60 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.240+0000 7f10b13f5640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f10ac0cd530 0x7f10ac15d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.240+0000 7f10b13f5640 1 --2- 172.21.15.81:0/765525761 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f10ac0cc800 0x7f10ac0cbf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.240+0000 7f10b13f5640 1 -- 172.21.15.81:0/765525761 >> 172.21.15.81:0/765525761 conn(0x7f10ac065df0 msgr2=0x7f10ac0b16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:48.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.240+0000 7f10b13f5640 1 -- 172.21.15.81:0/765525761 shutdown_connections 2023-11-08T16:37:48.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.240+0000 7f10b13f5640 1 -- 172.21.15.81:0/765525761 wait complete. 2023-11-08T16:37:48.261 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:37:48.261 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:37:48.314 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:37:48.365 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:37:48.413 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:37:48.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.560+0000 7f07e84a9640 1 Processor -- start 2023-11-08T16:37:48.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.560+0000 7f07e84a9640 1 -- start start 2023-11-08T16:37:48.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.561+0000 7f07e84a9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f07e00ca480 0x7f07e015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:48.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.562+0000 7f07e84a9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e006b630 con 0x7f07e015ea60 2023-11-08T16:37:48.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.562+0000 7f07e84a9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e00ca910 con 0x7f07e00ade50 2023-11-08T16:37:48.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.562+0000 7f07e84a9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e00b1860 con 0x7f07e00ca480 2023-11-08T16:37:48.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.562+0000 7f07e5a44640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f07e00ca480 0x7f07e015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:48.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.562+0000 7f07e5a44640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f07e00ca480 0x7f07e015ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39320/0 (socket says 172.21.15.81:39320) 2023-11-08T16:37:48.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.562+0000 7f07e5a44640 1 -- 172.21.15.81:0/39177216 learned_addr learned my addr 172.21.15.81:0/39177216 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:48.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.562+0000 7f07e5a44640 1 -- 172.21.15.81:0/39177216 >> v1:172.21.15.93:6789/0 conn(0x7f07e00ade50 legacy=0x7f07e015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:48.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.562+0000 7f07e5a44640 1 -- 172.21.15.81:0/39177216 >> v1:172.21.15.81:6789/0 conn(0x7f07e015ea60 legacy=0x7f07e015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:48.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.562+0000 7f07e5a44640 1 -- 172.21.15.81:0/39177216 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e00cabb0 con 0x7f07e00ca480 2023-11-08T16:37:48.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.564+0000 7f07e5a44640 1 --2- 172.21.15.81:0/39177216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e00ca480 0x7f07e015ad90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009870 tx=0x7f07d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8583599b4ed48c4c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:48.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.564+0000 7f07e5243640 1 -- 172.21.15.81:0/39177216 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0030860 con 0x7f07e00ca480 2023-11-08T16:37:48.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.564+0000 7f07e5243640 1 -- 172.21.15.81:0/39177216 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d00309c0 con 0x7f07e00ca480 2023-11-08T16:37:48.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.564+0000 7f07e5243640 1 -- 172.21.15.81:0/39177216 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d0030d50 con 0x7f07e00ca480 2023-11-08T16:37:48.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.565+0000 7f07e84a9640 1 -- 172.21.15.81:0/39177216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e00ca480 msgr2=0x7f07e015ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:48.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.565+0000 7f07e84a9640 1 --2- 172.21.15.81:0/39177216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e00ca480 0x7f07e015ad90 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009870 tx=0x7f07d002b9a0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.565+0000 7f07e84a9640 1 -- 172.21.15.81:0/39177216 shutdown_connections 2023-11-08T16:37:48.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.565+0000 7f07e84a9640 1 --2- 172.21.15.81:0/39177216 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e00ca480 0x7f07e015ad90 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.565+0000 7f07e84a9640 1 -- 172.21.15.81:0/39177216 >> 172.21.15.81:0/39177216 conn(0x7f07e00cb490 msgr2=0x7f07e00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:48.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.565+0000 7f07e84a9640 1 -- 172.21.15.81:0/39177216 shutdown_connections 2023-11-08T16:37:48.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.565+0000 7f07e84a9640 1 -- 172.21.15.81:0/39177216 wait complete. 2023-11-08T16:37:48.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.567+0000 7f07e84a9640 1 Processor -- start 2023-11-08T16:37:48.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.567+0000 7f07e84a9640 1 -- start start 2023-11-08T16:37:48.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.568+0000 7f07e84a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07e00ade50 0x7f07e00acc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:48.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.568+0000 7f07e84a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07e015ea60 0x7f07e015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:48.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.568+0000 7f07e84a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e015bff0 0x7f07e00a9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:48.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.568+0000 7f07e84a9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e00c3130 con 0x7f07e015bff0 2023-11-08T16:37:48.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.568+0000 7f07e84a9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e015c470 con 0x7f07e015ea60 2023-11-08T16:37:48.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.568+0000 7f07e84a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e01f3b80 con 0x7f07e00ade50 2023-11-08T16:37:48.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.568+0000 7f07e6a46640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e015bff0 0x7f07e00a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:48.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.568+0000 7f07e6a46640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e015bff0 0x7f07e00a9320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39336/0 (socket says 172.21.15.81:39336) 2023-11-08T16:37:48.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.568+0000 7f07e6a46640 1 -- 172.21.15.81:0/874399007 learned_addr learned my addr 172.21.15.81:0/874399007 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:48.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07e6245640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07e00ade50 0x7f07e00acc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:48.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07e6a46640 1 -- 172.21.15.81:0/874399007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07e00ade50 msgr2=0x7f07e00acc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:48.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07e5a44640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07e015ea60 0x7f07e015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:48.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07e6a46640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07e00ade50 0x7f07e00acc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07e6a46640 1 -- 172.21.15.81:0/874399007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07e015ea60 msgr2=0x7f07e015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:48.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07e6a46640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07e015ea60 0x7f07e015bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07e6a46640 1 -- 172.21.15.81:0/874399007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d00094d0 con 0x7f07e015bff0 2023-11-08T16:37:48.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07e6a46640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e015bff0 0x7f07e00a9320 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f07d800c6e0 tx=0x7f07d800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:48.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07d6ffd640 1 -- 172.21.15.81:0/874399007 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8003b70 con 0x7f07e015bff0 2023-11-08T16:37:48.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07d6ffd640 1 -- 172.21.15.81:0/874399007 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d8003cf0 con 0x7f07e015bff0 2023-11-08T16:37:48.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07d6ffd640 1 -- 172.21.15.81:0/874399007 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8005310 con 0x7f07e015bff0 2023-11-08T16:37:48.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07e84a9640 1 -- 172.21.15.81:0/874399007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e00c2540 con 0x7f07e015bff0 2023-11-08T16:37:48.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.569+0000 7f07e84a9640 1 -- 172.21.15.81:0/874399007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e00c2790 con 0x7f07e015bff0 2023-11-08T16:37:48.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.571+0000 7f07e84a9640 1 -- 172.21.15.81:0/874399007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e004ece0 con 0x7f07e015bff0 2023-11-08T16:37:48.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.571+0000 7f07d6ffd640 1 -- 172.21.15.81:0/874399007 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f07d8005470 con 0x7f07e015bff0 2023-11-08T16:37:48.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.572+0000 7f07d6ffd640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07c406ae70 0x7f07c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:48.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.572+0000 7f07d6ffd640 1 -- 172.21.15.81:0/874399007 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f07d80894d0 con 0x7f07e015bff0 2023-11-08T16:37:48.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.573+0000 7f07e6245640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07c406ae70 0x7f07c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:48.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.573+0000 7f07e6245640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07c406ae70 0x7f07c406d330 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f07e006c690 tx=0x7f07dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:48.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.577+0000 7f07d6ffd640 1 -- 172.21.15.81:0/874399007 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f07d8054440 con 0x7f07e015bff0 2023-11-08T16:37:48.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.769+0000 7f07e84a9640 1 -- 172.21.15.81:0/874399007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07e00b1570 con 0x7f07e015bff0 2023-11-08T16:37:48.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.772+0000 7f07d6ffd640 1 -- 172.21.15.81:0/874399007 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+18383 (secure 0 0 0) 0x7f07d8003e60 con 0x7f07e015bff0 2023-11-08T16:37:48.774 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:48.774 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":84,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:37:45.002417+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:37:30.174436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T16:37:31.340720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:37:42.279296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2e","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:37:48.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 -- 172.21.15.81:0/874399007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07c406ae70 msgr2=0x7f07c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:48.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07c406ae70 0x7f07c406d330 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f07e006c690 tx=0x7f07dc009040 comp rx=0 tx=0).stop 2023-11-08T16:37:48.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 -- 172.21.15.81:0/874399007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e015bff0 msgr2=0x7f07e00a9320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:48.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e015bff0 0x7f07e00a9320 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f07d800c6e0 tx=0x7f07d800cba0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 -- 172.21.15.81:0/874399007 shutdown_connections 2023-11-08T16:37:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f07c406ae70 0x7f07c406d330 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f07e015bff0 0x7f07e00a9320 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f07e015ea60 0x7f07e015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 --2- 172.21.15.81:0/874399007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f07e00ade50 0x7f07e00acc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:48.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 -- 172.21.15.81:0/874399007 >> 172.21.15.81:0/874399007 conn(0x7f07e00cb490 msgr2=0x7f07e015df20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 -- 172.21.15.81:0/874399007 shutdown_connections 2023-11-08T16:37:48.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:48.775+0000 7f07e84a9640 1 -- 172.21.15.81:0/874399007 wait complete. 2023-11-08T16:37:48.794 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:37:30.174436+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:37:31.340720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:37:42.279296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:37:48.795 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:37:49.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.054+0000 7ff7ec38a640 1 Processor -- start 2023-11-08T16:37:49.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.054+0000 7ff7ec38a640 1 -- start start 2023-11-08T16:37:49.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.056+0000 7ff7ec38a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7e40ccc10 0x7ff7e40ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.056+0000 7ff7ec38a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff7e406b710 0x7ff7e406bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.056+0000 7ff7ec38a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e40ce960 con 0x7ff7e415a0e0 2023-11-08T16:37:49.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.056+0000 7ff7ec38a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e4071980 con 0x7ff7e406b710 2023-11-08T16:37:49.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.056+0000 7ff7ec38a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40c9480 con 0x7ff7e40ccc10 2023-11-08T16:37:49.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.056+0000 7ff7ea927640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff7e406b710 0x7ff7e406bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.056+0000 7ff7ea927640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff7e406b710 0x7ff7e406bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39342/0 (socket says 172.21.15.81:39342) 2023-11-08T16:37:49.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.056+0000 7ff7ea927640 1 -- 172.21.15.81:0/3843094822 learned_addr learned my addr 172.21.15.81:0/3843094822 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:49.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.057+0000 7ff7e9925640 1 --2- 172.21.15.81:0/3843094822 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7e40ccc10 0x7ff7e40ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.057+0000 7ff7ea927640 1 -- 172.21.15.81:0/3843094822 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7e40ccc10 msgr2=0x7ff7e40ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:49.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.057+0000 7ff7ea927640 1 --2- 172.21.15.81:0/3843094822 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7e40ccc10 0x7ff7e40ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.057+0000 7ff7ea927640 1 -- 172.21.15.81:0/3843094822 >> v1:172.21.15.93:6789/0 conn(0x7ff7e415a0e0 legacy=0x7ff7e40c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:49.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.057+0000 7ff7ea927640 1 -- 172.21.15.81:0/3843094822 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e415a4c0 con 0x7ff7e406b710 2023-11-08T16:37:49.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.058+0000 7ff7ea927640 1 --2- 172.21.15.81:0/3843094822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e406b710 0x7ff7e406bb80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009e50 tx=0x7ff7dc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b635c6e4327393 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:49.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.058+0000 7ff7e9124640 1 -- 172.21.15.81:0/3843094822 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc02cb00 con 0x7ff7e406b710 2023-11-08T16:37:49.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.058+0000 7ff7e9124640 1 -- 172.21.15.81:0/3843094822 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc02cc60 con 0x7ff7e406b710 2023-11-08T16:37:49.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.058+0000 7ff7e9124640 1 -- 172.21.15.81:0/3843094822 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc0343f0 con 0x7ff7e406b710 2023-11-08T16:37:49.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.059+0000 7ff7ec38a640 1 -- 172.21.15.81:0/3843094822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e406b710 msgr2=0x7ff7e406bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.059+0000 7ff7ec38a640 1 --2- 172.21.15.81:0/3843094822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e406b710 0x7ff7e406bb80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009e50 tx=0x7ff7dc02bf60 comp rx=0 tx=0).stop 2023-11-08T16:37:49.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.059+0000 7ff7ec38a640 1 -- 172.21.15.81:0/3843094822 shutdown_connections 2023-11-08T16:37:49.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.059+0000 7ff7ec38a640 1 --2- 172.21.15.81:0/3843094822 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e406b710 0x7ff7e406bb80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.059+0000 7ff7ec38a640 1 --2- 172.21.15.81:0/3843094822 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7e40ccc10 0x7ff7e40ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.059+0000 7ff7ec38a640 1 -- 172.21.15.81:0/3843094822 >> 172.21.15.81:0/3843094822 conn(0x7ff7e40cb580 msgr2=0x7ff7e40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:49.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.059+0000 7ff7ec38a640 1 -- 172.21.15.81:0/3843094822 shutdown_connections 2023-11-08T16:37:49.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.059+0000 7ff7ec38a640 1 -- 172.21.15.81:0/3843094822 wait complete. 2023-11-08T16:37:49.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.061+0000 7ff7ec38a640 1 Processor -- start 2023-11-08T16:37:49.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.061+0000 7ff7ec38a640 1 -- start start 2023-11-08T16:37:49.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.062+0000 7ff7ec38a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e406b710 0x7ff7e40b21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.062+0000 7ff7ec38a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e40ccc10 0x7ff7e415c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.062+0000 7ff7ec38a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7e415a0e0 0x7ff7e4155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.062+0000 7ff7ec38a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415bd20 con 0x7ff7e406b710 2023-11-08T16:37:49.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.062+0000 7ff7ec38a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415b0c0 con 0x7ff7e40ccc10 2023-11-08T16:37:49.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.062+0000 7ff7ec38a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415b380 con 0x7ff7e415a0e0 2023-11-08T16:37:49.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.063+0000 7ff7ea126640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e406b710 0x7ff7e40b21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.063+0000 7ff7ea126640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e406b710 0x7ff7e40b21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39352/0 (socket says 172.21.15.81:39352) 2023-11-08T16:37:49.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.063+0000 7ff7ea126640 1 -- 172.21.15.81:0/340443371 learned_addr learned my addr 172.21.15.81:0/340443371 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:49.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.063+0000 7ff7ea927640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7e415a0e0 0x7ff7e4155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.063+0000 7ff7e9925640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e40ccc10 0x7ff7e415c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.063+0000 7ff7ea126640 1 -- 172.21.15.81:0/340443371 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7e415a0e0 msgr2=0x7ff7e4155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.063+0000 7ff7ea126640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7e415a0e0 0x7ff7e4155370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.063+0000 7ff7ea126640 1 -- 172.21.15.81:0/340443371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e40ccc10 msgr2=0x7ff7e415c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.063+0000 7ff7ea126640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e40ccc10 0x7ff7e415c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.063+0000 7ff7ea126640 1 -- 172.21.15.81:0/340443371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7dc009b30 con 0x7ff7e406b710 2023-11-08T16:37:49.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.063+0000 7ff7ea126640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e406b710 0x7ff7e40b21c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000c430 tx=0x7ff7e000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:49.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.064+0000 7ff7daffd640 1 -- 172.21.15.81:0/340443371 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0011790 con 0x7ff7e406b710 2023-11-08T16:37:49.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.064+0000 7ff7daffd640 1 -- 172.21.15.81:0/340443371 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e00118f0 con 0x7ff7e406b710 2023-11-08T16:37:49.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.064+0000 7ff7daffd640 1 -- 172.21.15.81:0/340443371 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0011ca0 con 0x7ff7e406b710 2023-11-08T16:37:49.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.064+0000 7ff7ec38a640 1 -- 172.21.15.81:0/340443371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e415b660 con 0x7ff7e406b710 2023-11-08T16:37:49.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.064+0000 7ff7ec38a640 1 -- 172.21.15.81:0/340443371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e41561a0 con 0x7ff7e406b710 2023-11-08T16:37:49.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.065+0000 7ff7ec38a640 1 -- 172.21.15.81:0/340443371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e404ece0 con 0x7ff7e406b710 2023-11-08T16:37:49.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.066+0000 7ff7daffd640 1 -- 172.21.15.81:0/340443371 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff7e001c460 con 0x7ff7e406b710 2023-11-08T16:37:49.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.067+0000 7ff7daffd640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7c806ae70 0x7ff7c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.067+0000 7ff7daffd640 1 -- 172.21.15.81:0/340443371 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7ff7e00897d0 con 0x7ff7e406b710 2023-11-08T16:37:49.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.067+0000 7ff7e9925640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7c806ae70 0x7ff7c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.068+0000 7ff7e9925640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7c806ae70 0x7ff7c806d330 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff7e41f6360 tx=0x7ff7d4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:49.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.071+0000 7ff7daffd640 1 -- 172.21.15.81:0/340443371 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff7e0054610 con 0x7ff7e406b710 2023-11-08T16:37:49.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.262+0000 7ff7ec38a640 1 -- 172.21.15.81:0/340443371 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7e40c57f0 con 0x7ff7e406b710 2023-11-08T16:37:49.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.263+0000 7ff7daffd640 1 -- 172.21.15.81:0/340443371 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7e0057c30 con 0x7ff7e406b710 2023-11-08T16:37:49.265 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:37:49.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 -- 172.21.15.81:0/340443371 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7c806ae70 msgr2=0x7ff7c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7c806ae70 0x7ff7c806d330 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff7e41f6360 tx=0x7ff7d4008040 comp rx=0 tx=0).stop 2023-11-08T16:37:49.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 -- 172.21.15.81:0/340443371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e406b710 msgr2=0x7ff7e40b21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e406b710 0x7ff7e40b21c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000c430 tx=0x7ff7e000c8f0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 -- 172.21.15.81:0/340443371 shutdown_connections 2023-11-08T16:37:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7c806ae70 0x7ff7c806d330 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7e415a0e0 0x7ff7e4155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e40ccc10 0x7ff7e415c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 --2- 172.21.15.81:0/340443371 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e406b710 0x7ff7e40b21c0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 -- 172.21.15.81:0/340443371 >> 172.21.15.81:0/340443371 conn(0x7ff7e40cb580 msgr2=0x7ff7e40b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:49.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 -- 172.21.15.81:0/340443371 shutdown_connections 2023-11-08T16:37:49.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.266+0000 7ff7ec38a640 1 -- 172.21.15.81:0/340443371 wait complete. 2023-11-08T16:37:49.287 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:37:49.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.503+0000 7fc1cb883640 1 Processor -- start 2023-11-08T16:37:49.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.503+0000 7fc1cb883640 1 -- start start 2023-11-08T16:37:49.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.505+0000 7fc1cb883640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc1c415f280 0x7fc1c40caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.505+0000 7fc1cb883640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c40cda10 con 0x7fc1c40ce0d0 2023-11-08T16:37:49.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.505+0000 7fc1cb883640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c406fa30 con 0x7fc1c415bfc0 2023-11-08T16:37:49.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.505+0000 7fc1cb883640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c40ce4b0 con 0x7fc1c415f280 2023-11-08T16:37:49.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.505+0000 7fc1c9e20640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc1c415f280 0x7fc1c40caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.505+0000 7fc1c961f640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc1c40ce0d0 0x7fc1c415eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47740/0 (socket says 172.21.15.81:47740) 2023-11-08T16:37:49.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.505+0000 7fc1c961f640 1 -- 172.21.15.81:0/4288731452 learned_addr learned my addr 172.21.15.81:0/4288731452 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:49.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.506+0000 7fc1c9e20640 1 -- 172.21.15.81:0/4288731452 >> v1:172.21.15.93:6789/0 conn(0x7fc1c415bfc0 legacy=0x7fc1c41516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:49.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.506+0000 7fc1c9e20640 1 -- 172.21.15.81:0/4288731452 >> v1:172.21.15.81:6789/0 conn(0x7fc1c40ce0d0 legacy=0x7fc1c415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.506+0000 7fc1c9e20640 1 -- 172.21.15.81:0/4288731452 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c406fca0 con 0x7fc1c415f280 2023-11-08T16:37:49.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.506+0000 7fc1b7fff640 1 -- 172.21.15.81:0/4288731452 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2512007743 0 0) 0x7fc1c40cda10 con 0x7fc1c40ce0d0 2023-11-08T16:37:49.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.507+0000 7fc1c9e20640 1 --2- 172.21.15.81:0/4288731452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f280 0x7fc1c40caf20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009d90 tx=0x7fc1bc02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dffb70786d22f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:49.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.507+0000 7fc1b7fff640 1 -- 172.21.15.81:0/4288731452 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc02cc60 con 0x7fc1c415f280 2023-11-08T16:37:49.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.507+0000 7fc1b7fff640 1 -- 172.21.15.81:0/4288731452 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc02cdc0 con 0x7fc1c415f280 2023-11-08T16:37:49.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.508+0000 7fc1b7fff640 1 -- 172.21.15.81:0/4288731452 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc034660 con 0x7fc1c415f280 2023-11-08T16:37:49.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.508+0000 7fc1cb883640 1 -- 172.21.15.81:0/4288731452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f280 msgr2=0x7fc1c40caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.508+0000 7fc1cb883640 1 --2- 172.21.15.81:0/4288731452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f280 0x7fc1c40caf20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009d90 tx=0x7fc1bc02beb0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.508+0000 7fc1cb883640 1 -- 172.21.15.81:0/4288731452 shutdown_connections 2023-11-08T16:37:49.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.508+0000 7fc1cb883640 1 --2- 172.21.15.81:0/4288731452 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f280 0x7fc1c40caf20 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.508+0000 7fc1cb883640 1 -- 172.21.15.81:0/4288731452 >> 172.21.15.81:0/4288731452 conn(0x7fc1c40cb580 msgr2=0x7fc1c40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:49.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.508+0000 7fc1cb883640 1 -- 172.21.15.81:0/4288731452 shutdown_connections 2023-11-08T16:37:49.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.508+0000 7fc1cb883640 1 -- 172.21.15.81:0/4288731452 wait complete. 2023-11-08T16:37:49.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.510+0000 7fc1cb883640 1 Processor -- start 2023-11-08T16:37:49.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.510+0000 7fc1cb883640 1 -- start start 2023-11-08T16:37:49.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.511+0000 7fc1cb883640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 0x7fc1c40c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.511+0000 7fc1cb883640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415bfc0 0x7fc1c414be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.511+0000 7fc1cb883640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415f280 0x7fc1c40c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.511+0000 7fc1cb883640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c49a0 con 0x7fc1c415bfc0 2023-11-08T16:37:49.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.511+0000 7fc1cb883640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c414c380 con 0x7fc1c40ce0d0 2023-11-08T16:37:49.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.511+0000 7fc1cb883640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c4090 con 0x7fc1c415f280 2023-11-08T16:37:49.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.512+0000 7fc1c8e1e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415bfc0 0x7fc1c414be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.512+0000 7fc1c8e1e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415bfc0 0x7fc1c414be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39374/0 (socket says 172.21.15.81:39374) 2023-11-08T16:37:49.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.512+0000 7fc1c8e1e640 1 -- 172.21.15.81:0/3332014386 learned_addr learned my addr 172.21.15.81:0/3332014386 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:49.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.512+0000 7fc1c8e1e640 1 -- 172.21.15.81:0/3332014386 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415f280 msgr2=0x7fc1c40c7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.512+0000 7fc1c961f640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 0x7fc1c40c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.512+0000 7fc1c8e1e640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415f280 0x7fc1c40c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.512+0000 7fc1c8e1e640 1 -- 172.21.15.81:0/3332014386 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 msgr2=0x7fc1c40c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.512+0000 7fc1c8e1e640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 0x7fc1c40c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.512+0000 7fc1c8e1e640 1 -- 172.21.15.81:0/3332014386 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc009a70 con 0x7fc1c415bfc0 2023-11-08T16:37:49.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.512+0000 7fc1c961f640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 0x7fc1c40c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:49.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.512+0000 7fc1c8e1e640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415bfc0 0x7fc1c414be40 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c430 tx=0x7fc1b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:49.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.513+0000 7fc1b5ffb640 1 -- 172.21.15.81:0/3332014386 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8011790 con 0x7fc1c415bfc0 2023-11-08T16:37:49.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.513+0000 7fc1b5ffb640 1 -- 172.21.15.81:0/3332014386 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b80118f0 con 0x7fc1c415bfc0 2023-11-08T16:37:49.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.513+0000 7fc1b5ffb640 1 -- 172.21.15.81:0/3332014386 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8011ce0 con 0x7fc1c415bfc0 2023-11-08T16:37:49.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.513+0000 7fc1cb883640 1 -- 172.21.15.81:0/3332014386 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c40c7150 con 0x7fc1c415bfc0 2023-11-08T16:37:49.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.513+0000 7fc1cb883640 1 -- 172.21.15.81:0/3332014386 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c40c45c0 con 0x7fc1c415bfc0 2023-11-08T16:37:49.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.514+0000 7fc1cb883640 1 -- 172.21.15.81:0/3332014386 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c4068260 con 0x7fc1c415bfc0 2023-11-08T16:37:49.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.515+0000 7fc1b5ffb640 1 -- 172.21.15.81:0/3332014386 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc1b801c460 con 0x7fc1c415bfc0 2023-11-08T16:37:49.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.516+0000 7fc1b5ffb640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 0x7fc1a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.516+0000 7fc1b5ffb640 1 -- 172.21.15.81:0/3332014386 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fc1b80896f0 con 0x7fc1c415bfc0 2023-11-08T16:37:49.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.516+0000 7fc1c961f640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 0x7fc1a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.517+0000 7fc1c961f640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 0x7fc1a806d330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0000f60 tx=0x7fc1c0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:49.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.520+0000 7fc1b5ffb640 1 -- 172.21.15.81:0/3332014386 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1b80545d0 con 0x7fc1c415bfc0 2023-11-08T16:37:49.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.725+0000 7fc1cb883640 1 -- 172.21.15.81:0/3332014386 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc1c40cad70 con 0x7fc1c415bfc0 2023-11-08T16:37:49.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.726+0000 7fc1b5ffb640 1 -- 172.21.15.81:0/3332014386 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc1b8057bf0 con 0x7fc1c415bfc0 2023-11-08T16:37:49.728 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:37:49.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.728+0000 7fc1cb883640 1 -- 172.21.15.81:0/3332014386 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 msgr2=0x7fc1a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.728+0000 7fc1cb883640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 0x7fc1a806d330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0000f60 tx=0x7fc1c0009040 comp rx=0 tx=0).stop 2023-11-08T16:37:49.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.728+0000 7fc1cb883640 1 -- 172.21.15.81:0/3332014386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415bfc0 msgr2=0x7fc1c414be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.728+0000 7fc1cb883640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415bfc0 0x7fc1c414be40 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800c430 tx=0x7fc1b800c8f0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.729+0000 7fc1cb883640 1 -- 172.21.15.81:0/3332014386 shutdown_connections 2023-11-08T16:37:49.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.729+0000 7fc1cb883640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 0x7fc1a806d330 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.729+0000 7fc1cb883640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415f280 0x7fc1c40c7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.729+0000 7fc1cb883640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415bfc0 0x7fc1c414be40 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.729+0000 7fc1cb883640 1 --2- 172.21.15.81:0/3332014386 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 0x7fc1c40c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.729+0000 7fc1cb883640 1 -- 172.21.15.81:0/3332014386 >> 172.21.15.81:0/3332014386 conn(0x7fc1c40cb580 msgr2=0x7fc1c415d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:49.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.729+0000 7fc1cb883640 1 -- 172.21.15.81:0/3332014386 shutdown_connections 2023-11-08T16:37:49.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.729+0000 7fc1cb883640 1 -- 172.21.15.81:0/3332014386 wait complete. 2023-11-08T16:37:49.747 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:37:49.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.954+0000 7fada9789640 1 Processor -- start 2023-11-08T16:37:49.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.954+0000 7fada9789640 1 -- start start 2023-11-08T16:37:49.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.956+0000 7fada9789640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fada40ce390 0x7fada415e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.956+0000 7fada9789640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fada406be20 0x7fada40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.956+0000 7fada9789640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada415e8d0 con 0x7fada40c9200 2023-11-08T16:37:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.956+0000 7fada9789640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada40ca2f0 con 0x7fada40ce390 2023-11-08T16:37:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.956+0000 7fada9789640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada40ce770 con 0x7fada406be20 2023-11-08T16:37:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.956+0000 7fada2ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fada40ce390 0x7fada415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.956+0000 7fada2ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fada40ce390 0x7fada415e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39380/0 (socket says 172.21.15.81:39380) 2023-11-08T16:37:49.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.956+0000 7fada2ffd640 1 -- 172.21.15.81:0/1352236221 learned_addr learned my addr 172.21.15.81:0/1352236221 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:49.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.957+0000 7fada37fe640 1 --2- 172.21.15.81:0/1352236221 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fada406be20 0x7fada40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.957+0000 7fada2ffd640 1 -- 172.21.15.81:0/1352236221 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fada406be20 msgr2=0x7fada40ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:49.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.957+0000 7fada2ffd640 1 --2- 172.21.15.81:0/1352236221 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fada406be20 0x7fada40ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.957+0000 7fada2ffd640 1 -- 172.21.15.81:0/1352236221 >> v1:172.21.15.81:6789/0 conn(0x7fada40c9200 legacy=0x7fada406b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.957+0000 7fada2ffd640 1 -- 172.21.15.81:0/1352236221 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada40c96e0 con 0x7fada40ce390 2023-11-08T16:37:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.957+0000 7fada37fe640 1 --2- 172.21.15.81:0/1352236221 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fada406be20 0x7fada40ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:49.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.958+0000 7fada2ffd640 1 --2- 172.21.15.81:0/1352236221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fada40ce390 0x7fada415e390 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fad94009870 tx=0x7fad9402ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f99720707b6e9dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:49.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.958+0000 7fada1ffb640 1 -- 172.21.15.81:0/1352236221 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9402f830 con 0x7fada40ce390 2023-11-08T16:37:49.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.958+0000 7fada1ffb640 1 -- 172.21.15.81:0/1352236221 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad9402f990 con 0x7fada40ce390 2023-11-08T16:37:49.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.958+0000 7fada1ffb640 1 -- 172.21.15.81:0/1352236221 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9402fd20 con 0x7fada40ce390 2023-11-08T16:37:49.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.959+0000 7fada9789640 1 -- 172.21.15.81:0/1352236221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fada40ce390 msgr2=0x7fada415e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.959+0000 7fada9789640 1 --2- 172.21.15.81:0/1352236221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fada40ce390 0x7fada415e390 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fad94009870 tx=0x7fad9402ba10 comp rx=0 tx=0).stop 2023-11-08T16:37:49.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.959+0000 7fada9789640 1 -- 172.21.15.81:0/1352236221 shutdown_connections 2023-11-08T16:37:49.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.959+0000 7fada9789640 1 --2- 172.21.15.81:0/1352236221 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fada406be20 0x7fada40ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.959+0000 7fada9789640 1 --2- 172.21.15.81:0/1352236221 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fada40ce390 0x7fada415e390 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.959+0000 7fada9789640 1 -- 172.21.15.81:0/1352236221 >> 172.21.15.81:0/1352236221 conn(0x7fada40cb580 msgr2=0x7fada40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:49.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.959+0000 7fada9789640 1 -- 172.21.15.81:0/1352236221 shutdown_connections 2023-11-08T16:37:49.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.959+0000 7fada9789640 1 -- 172.21.15.81:0/1352236221 wait complete. 2023-11-08T16:37:49.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.961+0000 7fada9789640 1 Processor -- start 2023-11-08T16:37:49.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.961+0000 7fada9789640 1 -- start start 2023-11-08T16:37:49.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada9789640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fada406be20 0x7fada4152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada9789640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fada40c9200 0x7fada4156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada9789640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fada40ce390 0x7fada40c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada9789640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada40c3380 con 0x7fada406be20 2023-11-08T16:37:49.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada9789640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada4152ea0 con 0x7fada40c9200 2023-11-08T16:37:49.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada9789640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada40b4570 con 0x7fada40ce390 2023-11-08T16:37:49.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fada406be20 0x7fada4152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fada406be20 0x7fada4152960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39392/0 (socket says 172.21.15.81:39392) 2023-11-08T16:37:49.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada2ffd640 1 -- 172.21.15.81:0/350602264 learned_addr learned my addr 172.21.15.81:0/350602264 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:49.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada2ffd640 1 -- 172.21.15.81:0/350602264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fada40ce390 msgr2=0x7fada40c37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada2ffd640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fada40ce390 0x7fada40c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada2ffd640 1 -- 172.21.15.81:0/350602264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fada40c9200 msgr2=0x7fada4156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada2ffd640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fada40c9200 0x7fada4156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.962+0000 7fada2ffd640 1 -- 172.21.15.81:0/350602264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad940094d0 con 0x7fada406be20 2023-11-08T16:37:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.963+0000 7fada2ffd640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fada406be20 0x7fada4152960 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fad9402ba50 tx=0x7fad94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:49.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.963+0000 7fad8b7fe640 1 -- 172.21.15.81:0/350602264 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9402f800 con 0x7fada406be20 2023-11-08T16:37:49.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.963+0000 7fad8b7fe640 1 -- 172.21.15.81:0/350602264 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad94042480 con 0x7fada406be20 2023-11-08T16:37:49.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.963+0000 7fad8b7fe640 1 -- 172.21.15.81:0/350602264 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad94042850 con 0x7fada406be20 2023-11-08T16:37:49.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.963+0000 7fada9789640 1 -- 172.21.15.81:0/350602264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada41f3310 con 0x7fada406be20 2023-11-08T16:37:49.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.963+0000 7fada9789640 1 -- 172.21.15.81:0/350602264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada40c2cc0 con 0x7fada406be20 2023-11-08T16:37:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.964+0000 7fada9789640 1 -- 172.21.15.81:0/350602264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada40c34c0 con 0x7fada406be20 2023-11-08T16:37:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.965+0000 7fad8b7fe640 1 -- 172.21.15.81:0/350602264 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad9402d3c0 con 0x7fada406be20 2023-11-08T16:37:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.966+0000 7fad8b7fe640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad9006ae70 0x7fad9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.966+0000 7fad8b7fe640 1 -- 172.21.15.81:0/350602264 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fad9403c070 con 0x7fada406be20 2023-11-08T16:37:49.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.966+0000 7fada27fc640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad9006ae70 0x7fad9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:49.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.967+0000 7fada27fc640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad9006ae70 0x7fad9006d330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fad98000f60 tx=0x7fad980079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:49.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:49.970+0000 7fad8b7fe640 1 -- 172.21.15.81:0/350602264 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fada40c34c0 con 0x7fada406be20 2023-11-08T16:37:50.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.160+0000 7fada9789640 1 -- 172.21.15.81:0/350602264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fada40c34c0 con 0x7fada406be20 2023-11-08T16:37:50.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.161+0000 7fad8b7fe640 1 -- 172.21.15.81:0/350602264 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fada40c34c0 con 0x7fada406be20 2023-11-08T16:37:50.163 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:37:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 -- 172.21.15.81:0/350602264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad9006ae70 msgr2=0x7fad9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad9006ae70 0x7fad9006d330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fad98000f60 tx=0x7fad980079b0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 -- 172.21.15.81:0/350602264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fada406be20 msgr2=0x7fada4152960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fada406be20 0x7fada4152960 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fad9402ba50 tx=0x7fad94004730 comp rx=0 tx=0).stop 2023-11-08T16:37:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 -- 172.21.15.81:0/350602264 shutdown_connections 2023-11-08T16:37:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fad9006ae70 0x7fad9006d330 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fada40ce390 0x7fada40c37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fada40c9200 0x7fada4156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 --2- 172.21.15.81:0/350602264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fada406be20 0x7fada4152960 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 -- 172.21.15.81:0/350602264 >> 172.21.15.81:0/350602264 conn(0x7fada40cb580 msgr2=0x7fada40ae4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:50.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 -- 172.21.15.81:0/350602264 shutdown_connections 2023-11-08T16:37:50.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.163+0000 7fada9789640 1 -- 172.21.15.81:0/350602264 wait complete. 2023-11-08T16:37:50.181 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:37:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.384+0000 7f4f5f2da640 1 Processor -- start 2023-11-08T16:37:50.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.385+0000 7f4f5f2da640 1 -- start start 2023-11-08T16:37:50.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.386+0000 7f4f5f2da640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4f580ce0d0 0x7f4f580caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.386+0000 7f4f5f2da640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f580cda10 con 0x7f4f5815bfc0 2023-11-08T16:37:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.386+0000 7f4f5f2da640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f5806fa30 con 0x7f4f5815f280 2023-11-08T16:37:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.386+0000 7f4f5f2da640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580ce4b0 con 0x7f4f580ce0d0 2023-11-08T16:37:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.386+0000 7f4f5c875640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4f5815bfc0 0x7f4f5815eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47748/0 (socket says 172.21.15.81:47748) 2023-11-08T16:37:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.386+0000 7f4f5c875640 1 -- 172.21.15.81:0/3811485336 learned_addr learned my addr 172.21.15.81:0/3811485336 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3381468806 0 0) 0x7f4f580cda10 con 0x7f4f5815bfc0 2023-11-08T16:37:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f5d076640 1 --2- 172.21.15.81:0/3811485336 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4f580ce0d0 0x7f4f580caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f44003520 con 0x7f4f5815bfc0 2023-11-08T16:37:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2856968280 0 0) 0x7f4f44003520 con 0x7f4f5815bfc0 2023-11-08T16:37:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4f580ce0d0 msgr2=0x7f4f580caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 --2- 172.21.15.81:0/3811485336 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4f580ce0d0 0x7f4f580caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 >> v1:172.21.15.93:6789/0 conn(0x7f4f5815f280 legacy=0x7f4f581516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f5806fca0 con 0x7f4f5815bfc0 2023-11-08T16:37:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f4f48001670 con 0x7f4f5815bfc0 2023-11-08T16:37:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 >> v1:172.21.15.81:6789/0 conn(0x7f4f5815bfc0 legacy=0x7f4f5815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 --2- 172.21.15.81:0/3811485336 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4f580ce0d0 0x7f4f44003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 --2- 172.21.15.81:0/3811485336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f44003fe0 0x7f4f440047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 --2- 172.21.15.81:0/3811485336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 0x7f4f44005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580ce4b0 con 0x7f4f44004d00 2023-11-08T16:37:50.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4400bb40 con 0x7f4f44003fe0 2023-11-08T16:37:50.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.387+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4400bd00 con 0x7f4f580ce0d0 2023-11-08T16:37:50.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.388+0000 7f4f5d877640 1 --2- 172.21.15.81:0/3811485336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 0x7f4f44005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:50.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.388+0000 7f4f5c875640 1 --2- 172.21.15.81:0/3811485336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f44003fe0 0x7f4f440047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:50.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.388+0000 7f4f5d877640 1 -- 172.21.15.81:0/3811485336 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4f580ce0d0 msgr2=0x7f4f44003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:50.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.388+0000 7f4f5d877640 1 --2- 172.21.15.81:0/3811485336 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4f580ce0d0 0x7f4f44003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.388+0000 7f4f5d877640 1 -- 172.21.15.81:0/3811485336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f44003fe0 msgr2=0x7f4f440047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:50.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.388+0000 7f4f5d877640 1 --2- 172.21.15.81:0/3811485336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f44003fe0 0x7f4f440047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.388+0000 7f4f5d877640 1 -- 172.21.15.81:0/3811485336 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f44003520 con 0x7f4f44004d00 2023-11-08T16:37:50.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.389+0000 7f4f5d877640 1 --2- 172.21.15.81:0/3811485336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 0x7f4f44005180 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f4f50001e20 tx=0x7f4f440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ea1364459871f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:50.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.389+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f50003b70 con 0x7f4f44004d00 2023-11-08T16:37:50.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.389+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f50003cd0 con 0x7f4f44004d00 2023-11-08T16:37:50.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.389+0000 7f4f4ffff640 1 -- 172.21.15.81:0/3811485336 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f50005310 con 0x7f4f44004d00 2023-11-08T16:37:50.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.390+0000 7f4f5f2da640 1 -- 172.21.15.81:0/3811485336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 msgr2=0x7f4f44005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.390+0000 7f4f5f2da640 1 --2- 172.21.15.81:0/3811485336 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 0x7f4f44005180 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f4f50001e20 tx=0x7f4f440057f0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.390+0000 7f4f5d877640 1 -- 172.21.15.81:0/3811485336 reap_dead start 2023-11-08T16:37:50.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.390+0000 7f4f5f2da640 1 -- 172.21.15.81:0/3811485336 shutdown_connections 2023-11-08T16:37:50.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.390+0000 7f4f5f2da640 1 -- 172.21.15.81:0/3811485336 >> 172.21.15.81:0/3811485336 conn(0x7f4f580cb580 msgr2=0x7f4f580cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:50.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.390+0000 7f4f5f2da640 1 -- 172.21.15.81:0/3811485336 shutdown_connections 2023-11-08T16:37:50.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.390+0000 7f4f5f2da640 1 -- 172.21.15.81:0/3811485336 wait complete. 2023-11-08T16:37:50.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.392+0000 7f4f5f2da640 1 Processor -- start 2023-11-08T16:37:50.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.392+0000 7f4f5f2da640 1 -- start start 2023-11-08T16:37:50.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5f2da640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 0x7f4f580c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5f2da640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4f5814ff40 0x7f4f580c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5f2da640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f580c9100 0x7f4f580c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5f2da640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f581f3b40 con 0x7f4f44004d00 2023-11-08T16:37:50.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5f2da640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580c9ac0 con 0x7f4f580c9100 2023-11-08T16:37:50.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5f2da640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f581f2ec0 con 0x7f4f5814ff40 2023-11-08T16:37:50.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5d076640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 0x7f4f580c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:50.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5d076640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 0x7f4f580c38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39418/0 (socket says 172.21.15.81:39418) 2023-11-08T16:37:50.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5d076640 1 -- 172.21.15.81:0/361513648 learned_addr learned my addr 172.21.15.81:0/361513648 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:50.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5d076640 1 -- 172.21.15.81:0/361513648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4f5814ff40 msgr2=0x7f4f580c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5d076640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4f5814ff40 0x7f4f580c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5d076640 1 -- 172.21.15.81:0/361513648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f580c9100 msgr2=0x7f4f580c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5d076640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f580c9100 0x7f4f580c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.393+0000 7f4f5d076640 1 -- 172.21.15.81:0/361513648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f581f31a0 con 0x7f4f44004d00 2023-11-08T16:37:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.394+0000 7f4f5d076640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 0x7f4f580c38c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400e7f0 tx=0x7f4f5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.394+0000 7f4f4dffb640 1 -- 172.21.15.81:0/361513648 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400eea0 con 0x7f4f44004d00 2023-11-08T16:37:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.394+0000 7f4f4dffb640 1 -- 172.21.15.81:0/361513648 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f5400c940 con 0x7f4f44004d00 2023-11-08T16:37:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.394+0000 7f4f4dffb640 1 -- 172.21.15.81:0/361513648 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f5400ccd0 con 0x7f4f44004d00 2023-11-08T16:37:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.394+0000 7f4f5f2da640 1 -- 172.21.15.81:0/361513648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f581f3480 con 0x7f4f44004d00 2023-11-08T16:37:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.394+0000 7f4f5f2da640 1 -- 172.21.15.81:0/361513648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f580c8850 con 0x7f4f44004d00 2023-11-08T16:37:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.395+0000 7f4f5f2da640 1 -- 172.21.15.81:0/361513648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f5804ece0 con 0x7f4f44004d00 2023-11-08T16:37:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.396+0000 7f4f4dffb640 1 -- 172.21.15.81:0/361513648 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4f54013470 con 0x7f4f44004d00 2023-11-08T16:37:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.397+0000 7f4f4dffb640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4f3c06ae70 0x7f4f3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.397+0000 7f4f4dffb640 1 -- 172.21.15.81:0/361513648 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f4f54089880 con 0x7f4f44004d00 2023-11-08T16:37:50.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.397+0000 7f4f5c875640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4f3c06ae70 0x7f4f3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:50.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.398+0000 7f4f5c875640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4f3c06ae70 0x7f4f3c06d330 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4f580c3810 tx=0x7f4f48000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:50.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.401+0000 7f4f4dffb640 1 -- 172.21.15.81:0/361513648 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4f54054760 con 0x7f4f44004d00 2023-11-08T16:37:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.590+0000 7f4f5f2da640 1 -- 172.21.15.81:0/361513648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f4f580ce6b0 con 0x7f4f44004d00 2023-11-08T16:37:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.591+0000 7f4f4dffb640 1 -- 172.21.15.81:0/361513648 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v84) v1 ==== 106+0+11 (secure 0 0 0) 0x7f4f54057d80 con 0x7f4f44004d00 2023-11-08T16:37:50.593 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:37:50.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 -- 172.21.15.81:0/361513648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4f3c06ae70 msgr2=0x7f4f3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4f3c06ae70 0x7f4f3c06d330 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4f580c3810 tx=0x7f4f48000ed0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 -- 172.21.15.81:0/361513648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 msgr2=0x7f4f580c38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 0x7f4f580c38c0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f4f5400e7f0 tx=0x7f4f5400ecb0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 -- 172.21.15.81:0/361513648 shutdown_connections 2023-11-08T16:37:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4f3c06ae70 0x7f4f3c06d330 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f580c9100 0x7f4f580c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4f5814ff40 0x7f4f580c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 --2- 172.21.15.81:0/361513648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4f44004d00 0x7f4f580c38c0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 -- 172.21.15.81:0/361513648 >> 172.21.15.81:0/361513648 conn(0x7f4f580cb580 msgr2=0x7f4f580af810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 -- 172.21.15.81:0/361513648 shutdown_connections 2023-11-08T16:37:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.594+0000 7f4f5f2da640 1 -- 172.21.15.81:0/361513648 wait complete. 2023-11-08T16:37:50.614 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:37:50.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.828+0000 7f17361d2640 1 Processor -- start 2023-11-08T16:37:50.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.828+0000 7f17361d2640 1 -- start start 2023-11-08T16:37:50.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.830+0000 7f17361d2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1730159af0 0x7f1730070a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.830+0000 7f17361d2640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1730070f50 0x7f173015d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.830+0000 7f17361d2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1730070580 con 0x7f173015dc90 2023-11-08T16:37:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.830+0000 7f17361d2640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17300c9460 con 0x7f1730159af0 2023-11-08T16:37:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.830+0000 7f17361d2640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173015e2a0 con 0x7f1730070f50 2023-11-08T16:37:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.830+0000 7f17351d0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1730159af0 0x7f1730070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.830+0000 7f17359d1640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f173015dc90 0x7f17300cd110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:47754/0 (socket says 172.21.15.81:47754) 2023-11-08T16:37:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.830+0000 7f17359d1640 1 -- 172.21.15.81:0/3321332377 learned_addr learned my addr 172.21.15.81:0/3321332377 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.831+0000 7f17349cf640 1 --2- 172.21.15.81:0/3321332377 >> v2:172.21.15.93:3300/0 conn(0x7f1730070f50 0x7f173015d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.831+0000 7f17351d0640 1 -- 172.21.15.81:0/3321332377 >> v2:172.21.15.93:3300/0 conn(0x7f1730070f50 msgr2=0x7f173015d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.831+0000 7f17351d0640 1 --2- 172.21.15.81:0/3321332377 >> v2:172.21.15.93:3300/0 conn(0x7f1730070f50 0x7f173015d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.831+0000 7f17351d0640 1 -- 172.21.15.81:0/3321332377 >> v1:172.21.15.81:6789/0 conn(0x7f173015dc90 legacy=0x7f17300cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.831+0000 7f17351d0640 1 -- 172.21.15.81:0/3321332377 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173015e580 con 0x7f1730159af0 2023-11-08T16:37:50.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.831+0000 7f1727fff640 1 -- 172.21.15.81:0/3321332377 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006370641 0 0) 0x7f1730070580 con 0x7f173015dc90 2023-11-08T16:37:50.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.832+0000 7f17351d0640 1 --2- 172.21.15.81:0/3321332377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1730159af0 0x7f1730070a10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f17300cc4b0 tx=0x7f172c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b73fc58c9a9e122 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:50.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.832+0000 7f1727fff640 1 -- 172.21.15.81:0/3321332377 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c031830 con 0x7f1730159af0 2023-11-08T16:37:50.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.832+0000 7f1727fff640 1 -- 172.21.15.81:0/3321332377 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c031990 con 0x7f1730159af0 2023-11-08T16:37:50.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.832+0000 7f1727fff640 1 -- 172.21.15.81:0/3321332377 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c031d20 con 0x7f1730159af0 2023-11-08T16:37:50.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.833+0000 7f17361d2640 1 -- 172.21.15.81:0/3321332377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1730159af0 msgr2=0x7f1730070a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.833+0000 7f17361d2640 1 --2- 172.21.15.81:0/3321332377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1730159af0 0x7f1730070a10 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f17300cc4b0 tx=0x7f172c02b8d0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.833+0000 7f17361d2640 1 -- 172.21.15.81:0/3321332377 shutdown_connections 2023-11-08T16:37:50.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.833+0000 7f17361d2640 1 --2- 172.21.15.81:0/3321332377 >> v2:172.21.15.93:3300/0 conn(0x7f1730070f50 0x7f173015d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.833+0000 7f17361d2640 1 --2- 172.21.15.81:0/3321332377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1730159af0 0x7f1730070a10 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.833+0000 7f17361d2640 1 -- 172.21.15.81:0/3321332377 >> 172.21.15.81:0/3321332377 conn(0x7f17300b04d0 msgr2=0x7f17300b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:50.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.833+0000 7f17361d2640 1 -- 172.21.15.81:0/3321332377 shutdown_connections 2023-11-08T16:37:50.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.833+0000 7f17361d2640 1 -- 172.21.15.81:0/3321332377 wait complete. 2023-11-08T16:37:50.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.835+0000 7f17361d2640 1 Processor -- start 2023-11-08T16:37:50.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.835+0000 7f17361d2640 1 -- start start 2023-11-08T16:37:50.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17361d2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1730070f50 0x7f17301f2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17361d2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1730159af0 0x7f17301f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17361d2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173015dc90 0x7f17300ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17361d2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17300c37f0 con 0x7f1730159af0 2023-11-08T16:37:50.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17361d2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17300af240 con 0x7f173015dc90 2023-11-08T16:37:50.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17361d2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17300af520 con 0x7f1730070f50 2023-11-08T16:37:50.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17349cf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1730159af0 0x7f17301f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:50.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17349cf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1730159af0 0x7f17301f4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:39436/0 (socket says 172.21.15.81:39436) 2023-11-08T16:37:50.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17349cf640 1 -- 172.21.15.81:0/3329981168 learned_addr learned my addr 172.21.15.81:0/3329981168 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:50.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17349cf640 1 -- 172.21.15.81:0/3329981168 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1730070f50 msgr2=0x7f17301f2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17359d1640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173015dc90 0x7f17300ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:50.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17349cf640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1730070f50 0x7f17301f2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.836+0000 7f17349cf640 1 -- 172.21.15.81:0/3329981168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173015dc90 msgr2=0x7f17300ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.837+0000 7f17349cf640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173015dc90 0x7f17300ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.837+0000 7f17349cf640 1 -- 172.21.15.81:0/3329981168 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172c009400 con 0x7f1730159af0 2023-11-08T16:37:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.837+0000 7f17359d1640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173015dc90 0x7f17300ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.837+0000 7f17349cf640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1730159af0 0x7f17301f4f30 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f172000d800 tx=0x7f172000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.837+0000 7f1725ffb640 1 -- 172.21.15.81:0/3329981168 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000b6c0 con 0x7f1730159af0 2023-11-08T16:37:50.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.837+0000 7f1725ffb640 1 -- 172.21.15.81:0/3329981168 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172000b820 con 0x7f1730159af0 2023-11-08T16:37:50.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.837+0000 7f1725ffb640 1 -- 172.21.15.81:0/3329981168 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000bbb0 con 0x7f1730159af0 2023-11-08T16:37:50.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.837+0000 7f17361d2640 1 -- 172.21.15.81:0/3329981168 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17300af800 con 0x7f1730159af0 2023-11-08T16:37:50.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.837+0000 7f17361d2640 1 -- 172.21.15.81:0/3329981168 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17301f5470 con 0x7f1730159af0 2023-11-08T16:37:50.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.838+0000 7f17361d2640 1 -- 172.21.15.81:0/3329981168 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f173004ece0 con 0x7f1730159af0 2023-11-08T16:37:50.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.839+0000 7f1725ffb640 1 -- 172.21.15.81:0/3329981168 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f172000bd10 con 0x7f1730159af0 2023-11-08T16:37:50.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.840+0000 7f1725ffb640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f171406aea0 0x7f171406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:50.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.840+0000 7f1725ffb640 1 -- 172.21.15.81:0/3329981168 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f1720089d20 con 0x7f1730159af0 2023-11-08T16:37:50.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.841+0000 7f17351d0640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f171406aea0 0x7f171406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:50.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.841+0000 7f17351d0640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f171406aea0 0x7f171406d360 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f172c036000 tx=0x7f172c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:50.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:50.845+0000 7f1725ffb640 1 -- 172.21.15.81:0/3329981168 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1720054bf0 con 0x7f1730159af0 2023-11-08T16:37:51.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.047+0000 7f17361d2640 1 -- 172.21.15.81:0/3329981168 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f17300ccad0 con 0x7f1730159af0 2023-11-08T16:37:51.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.048+0000 7f1725ffb640 1 -- 172.21.15.81:0/3329981168 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v84) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1720058210 con 0x7f1730159af0 2023-11-08T16:37:51.050 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:37:51.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.050+0000 7f17361d2640 1 -- 172.21.15.81:0/3329981168 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f171406aea0 msgr2=0x7f171406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:51.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.050+0000 7f17361d2640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f171406aea0 0x7f171406d360 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f172c036000 tx=0x7f172c030000 comp rx=0 tx=0).stop 2023-11-08T16:37:51.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.050+0000 7f17361d2640 1 -- 172.21.15.81:0/3329981168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1730159af0 msgr2=0x7f17301f4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.050+0000 7f17361d2640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1730159af0 0x7f17301f4f30 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f172000d800 tx=0x7f172000dcc0 comp rx=0 tx=0).stop 2023-11-08T16:37:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.051+0000 7f17361d2640 1 -- 172.21.15.81:0/3329981168 shutdown_connections 2023-11-08T16:37:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.051+0000 7f17361d2640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f171406aea0 0x7f171406d360 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.051+0000 7f17361d2640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f173015dc90 0x7f17300ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.051+0000 7f17361d2640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1730159af0 0x7f17301f4f30 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.051+0000 7f17361d2640 1 --2- 172.21.15.81:0/3329981168 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1730070f50 0x7f17301f2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.051+0000 7f17361d2640 1 -- 172.21.15.81:0/3329981168 >> 172.21.15.81:0/3329981168 conn(0x7f17300b04d0 msgr2=0x7f173006b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.051+0000 7f17361d2640 1 -- 172.21.15.81:0/3329981168 shutdown_connections 2023-11-08T16:37:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:51.051+0000 7f17361d2640 1 -- 172.21.15.81:0/3329981168 wait complete. 2023-11-08T16:37:51.072 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:37:51.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:51.073 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:37:51.157 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[113709]: fuse finished with error 0 and tester_r 0 2023-11-08T16:37:51.165 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:37:57.267 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:37:57.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:37:57.268 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:37:57.331 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:37:57.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.546+0000 7fe1de832640 1 Processor -- start 2023-11-08T16:37:57.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.546+0000 7fe1de832640 1 -- start start 2023-11-08T16:37:57.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.548+0000 7fe1de832640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe1d80ade50 0x7fe1d80ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:57.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.548+0000 7fe1de832640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe1d815ea60 0x7fe1d815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:57.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.548+0000 7fe1de832640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d806b630 con 0x7fe1d815a680 2023-11-08T16:37:57.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.548+0000 7fe1de832640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d80b1860 con 0x7fe1d815ea60 2023-11-08T16:37:57.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.548+0000 7fe1de832640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d80ca9c0 con 0x7fe1d80ade50 2023-11-08T16:37:57.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.548+0000 7fe1dcdcf640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe1d815ea60 0x7fe1d815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:57.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.548+0000 7fe1cffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe1d815a680 0x7fe1d815ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51662/0 (socket says 172.21.15.81:51662) 2023-11-08T16:37:57.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.548+0000 7fe1cffff640 1 -- 172.21.15.81:0/1618004834 learned_addr learned my addr 172.21.15.81:0/1618004834 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:37:57.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.549+0000 7fe1dcdcf640 1 -- 172.21.15.81:0/1618004834 >> v2:172.21.15.93:3300/0 conn(0x7fe1d80ade50 msgr2=0x7fe1d80ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:57.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.549+0000 7fe1d7fff640 1 --2- 172.21.15.81:0/1618004834 >> v2:172.21.15.93:3300/0 conn(0x7fe1d80ade50 0x7fe1d80ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:57.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.549+0000 7fe1d77fe640 1 -- 172.21.15.81:0/1618004834 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646558804 0 0) 0x7fe1d806b630 con 0x7fe1d815a680 2023-11-08T16:37:57.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.549+0000 7fe1dcdcf640 1 --2- 172.21.15.81:0/1618004834 >> v2:172.21.15.93:3300/0 conn(0x7fe1d80ade50 0x7fe1d80ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:57.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.549+0000 7fe1dcdcf640 1 -- 172.21.15.81:0/1618004834 >> v1:172.21.15.81:6789/0 conn(0x7fe1d815a680 legacy=0x7fe1d815ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:57.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.549+0000 7fe1dcdcf640 1 -- 172.21.15.81:0/1618004834 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d80cabb0 con 0x7fe1d815ea60 2023-11-08T16:37:57.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.549+0000 7fe1d7fff640 1 --2- 172.21.15.81:0/1618004834 >> v2:172.21.15.93:3300/0 conn(0x7fe1d80ade50 0x7fe1d80ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:57.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.550+0000 7fe1dcdcf640 1 --2- 172.21.15.81:0/1618004834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d815ea60 0x7fe1d815eef0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8067770 tx=0x7fe1d002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d905533e76513c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:57.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.550+0000 7fe1d77fe640 1 -- 172.21.15.81:0/1618004834 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d002cc70 con 0x7fe1d815ea60 2023-11-08T16:37:57.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.550+0000 7fe1d77fe640 1 -- 172.21.15.81:0/1618004834 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d002cdd0 con 0x7fe1d815ea60 2023-11-08T16:37:57.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.550+0000 7fe1d77fe640 1 -- 172.21.15.81:0/1618004834 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0034660 con 0x7fe1d815ea60 2023-11-08T16:37:57.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.551+0000 7fe1de832640 1 -- 172.21.15.81:0/1618004834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d815ea60 msgr2=0x7fe1d815eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:57.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.551+0000 7fe1de832640 1 --2- 172.21.15.81:0/1618004834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d815ea60 0x7fe1d815eef0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8067770 tx=0x7fe1d002bec0 comp rx=0 tx=0).stop 2023-11-08T16:37:57.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.551+0000 7fe1de832640 1 -- 172.21.15.81:0/1618004834 shutdown_connections 2023-11-08T16:37:57.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.551+0000 7fe1de832640 1 --2- 172.21.15.81:0/1618004834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d815ea60 0x7fe1d815eef0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:57.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.551+0000 7fe1de832640 1 --2- 172.21.15.81:0/1618004834 >> v2:172.21.15.93:3300/0 conn(0x7fe1d80ade50 0x7fe1d80ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:57.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.551+0000 7fe1de832640 1 -- 172.21.15.81:0/1618004834 >> 172.21.15.81:0/1618004834 conn(0x7fe1d80cb490 msgr2=0x7fe1d80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:57.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.551+0000 7fe1de832640 1 -- 172.21.15.81:0/1618004834 shutdown_connections 2023-11-08T16:37:57.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.551+0000 7fe1de832640 1 -- 172.21.15.81:0/1618004834 wait complete. 2023-11-08T16:37:57.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.553+0000 7fe1de832640 1 Processor -- start 2023-11-08T16:37:57.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.553+0000 7fe1de832640 1 -- start start 2023-11-08T16:37:57.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.554+0000 7fe1de832640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1d80ade50 0x7fe1d8152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:57.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.554+0000 7fe1de832640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d815a680 0x7fe1d80ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:57.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.554+0000 7fe1de832640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1d815ea60 0x7fe1d80acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:57.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.554+0000 7fe1de832640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d8159b50 con 0x7fe1d815a680 2023-11-08T16:37:57.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.554+0000 7fe1de832640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d8143840 con 0x7fe1d815ea60 2023-11-08T16:37:57.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.554+0000 7fe1de832640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d8153480 con 0x7fe1d80ade50 2023-11-08T16:37:57.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.554+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d815a680 0x7fe1d80ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:57.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.554+0000 7fe1cffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d815a680 0x7fe1d80ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41490/0 (socket says 172.21.15.81:41490) 2023-11-08T16:37:57.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.554+0000 7fe1cffff640 1 -- 172.21.15.81:0/803286277 learned_addr learned my addr 172.21.15.81:0/803286277 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:57.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.554+0000 7fe1dcdcf640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1d815ea60 0x7fe1d80acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:57.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.554+0000 7fe1d7fff640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1d80ade50 0x7fe1d8152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:57.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.555+0000 7fe1cffff640 1 -- 172.21.15.81:0/803286277 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1d80ade50 msgr2=0x7fe1d8152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:57.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.555+0000 7fe1cffff640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1d80ade50 0x7fe1d8152f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:57.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.555+0000 7fe1cffff640 1 -- 172.21.15.81:0/803286277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1d815ea60 msgr2=0x7fe1d80acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:57.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.555+0000 7fe1cffff640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1d815ea60 0x7fe1d80acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:57.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.555+0000 7fe1cffff640 1 -- 172.21.15.81:0/803286277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d0009a70 con 0x7fe1d815a680 2023-11-08T16:37:57.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.555+0000 7fe1cffff640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d815a680 0x7fe1d80ac8b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c4d0 tx=0x7fe1c000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:57.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.555+0000 7fe1d57fa640 1 -- 172.21.15.81:0/803286277 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00074e0 con 0x7fe1d815a680 2023-11-08T16:37:57.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.555+0000 7fe1d57fa640 1 -- 172.21.15.81:0/803286277 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c0007660 con 0x7fe1d815a680 2023-11-08T16:37:57.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.555+0000 7fe1d57fa640 1 -- 172.21.15.81:0/803286277 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0004a20 con 0x7fe1d815a680 2023-11-08T16:37:57.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.555+0000 7fe1de832640 1 -- 172.21.15.81:0/803286277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d8145dc0 con 0x7fe1d815a680 2023-11-08T16:37:57.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.555+0000 7fe1de832640 1 -- 172.21.15.81:0/803286277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d81431f0 con 0x7fe1d815a680 2023-11-08T16:37:57.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.556+0000 7fe1de832640 1 -- 172.21.15.81:0/803286277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d81439f0 con 0x7fe1d815a680 2023-11-08T16:37:57.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.557+0000 7fe1d57fa640 1 -- 172.21.15.81:0/803286277 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe1c0004b80 con 0x7fe1d815a680 2023-11-08T16:37:57.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.558+0000 7fe1d57fa640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe1b806ae70 0x7fe1b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:57.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.558+0000 7fe1d57fa640 1 -- 172.21.15.81:0/803286277 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7fe1c0088a70 con 0x7fe1d815a680 2023-11-08T16:37:57.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.559+0000 7fe1d7fff640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe1b806ae70 0x7fe1b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:57.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.559+0000 7fe1d7fff640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe1b806ae70 0x7fe1b806d330 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8008850 tx=0x7fe1c8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:57.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.563+0000 7fe1d57fa640 1 -- 172.21.15.81:0/803286277 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe1d81439f0 con 0x7fe1d815a680 2023-11-08T16:37:57.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.791+0000 7fe1de832640 1 -- 172.21.15.81:0/803286277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1d81439b0 con 0x7fe1d815a680 2023-11-08T16:37:57.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.792+0000 7fe1d57fa640 1 -- 172.21.15.81:0/803286277 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4918 (secure 0 0 0) 0x7fe1c0018070 con 0x7fe1d815a680 2023-11-08T16:37:57.795 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:57.796 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":65,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/2125403666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2125403666},{"type":"v1","addr":"172.21.15.93:6834","nonce":2125403666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2727356524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2727356524},{"type":"v1","addr":"172.21.15.93:6835","nonce":2727356524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6936,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2639654743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2639654743},{"type":"v1","addr":"172.21.15.81:6835","nonce":2639654743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:32.323261+0000","modified":"2023-11-08T16:37:43.447328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6942},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6942":{"gid":6942,"name":"a","rank":0,"incarnation":60,"state":"up:active","state_seq":3,"addr":"172.21.15.81:6837/2162767469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2162767469},{"type":"v1","addr":"172.21.15.81:6837","nonce":2162767469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:37:57.796 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 65 2023-11-08T16:37:57.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.795+0000 7fe1de832640 1 -- 172.21.15.81:0/803286277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe1b806ae70 msgr2=0x7fe1b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:57.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.795+0000 7fe1de832640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe1b806ae70 0x7fe1b806d330 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8008850 tx=0x7fe1c8008040 comp rx=0 tx=0).stop 2023-11-08T16:37:57.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.795+0000 7fe1de832640 1 -- 172.21.15.81:0/803286277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d815a680 msgr2=0x7fe1d80ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:57.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.795+0000 7fe1de832640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d815a680 0x7fe1d80ac8b0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c4d0 tx=0x7fe1c000c990 comp rx=0 tx=0).stop 2023-11-08T16:37:57.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.796+0000 7fe1de832640 1 -- 172.21.15.81:0/803286277 shutdown_connections 2023-11-08T16:37:57.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.796+0000 7fe1de832640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe1b806ae70 0x7fe1b806d330 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:57.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.796+0000 7fe1de832640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe1d815ea60 0x7fe1d80acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:57.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.796+0000 7fe1de832640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe1d815a680 0x7fe1d80ac8b0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:57.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.796+0000 7fe1de832640 1 --2- 172.21.15.81:0/803286277 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe1d80ade50 0x7fe1d8152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:57.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.796+0000 7fe1de832640 1 -- 172.21.15.81:0/803286277 >> 172.21.15.81:0/803286277 conn(0x7fe1d80cb490 msgr2=0x7fe1d80b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:57.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.796+0000 7fe1de832640 1 -- 172.21.15.81:0/803286277 shutdown_connections 2023-11-08T16:37:57.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:57.796+0000 7fe1de832640 1 -- 172.21.15.81:0/803286277 wait complete. 2023-11-08T16:37:57.817 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2023-11-08T16:37:58.054 INFO:teuthology.orchestra.run.smithi081.stdout:{"message":"","return_code":0} 2023-11-08T16:37:58.055 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-08T16:37:58.055 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:37:58.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.260+0000 7f42add65640 1 Processor -- start 2023-11-08T16:37:58.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.261+0000 7f42add65640 1 -- start start 2023-11-08T16:37:58.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.262+0000 7f42add65640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 0x7f42a80ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:58.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.262+0000 7f42add65640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f42a80c9550 0x7f42a806b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:58.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.262+0000 7f42add65640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a806cc60 con 0x7f42a80c90f0 2023-11-08T16:37:58.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.262+0000 7f42add65640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a80cb3e0 con 0x7f42a80c9550 2023-11-08T16:37:58.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.262+0000 7f42add65640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80cae20 con 0x7f42a815aeb0 2023-11-08T16:37:58.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.263+0000 7f42a77fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 0x7f42a80ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:58.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.263+0000 7f42a7fff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f42a80c9550 0x7f42a806b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:58.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.263+0000 7f42a77fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 0x7f42a80ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:38336/0 (socket says 172.21.15.81:38336) 2023-11-08T16:37:58.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.263+0000 7f42a77fe640 1 -- 172.21.15.81:0/28197368 learned_addr learned my addr 172.21.15.81:0/28197368 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:58.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.264+0000 7f42a7fff640 1 -- 172.21.15.81:0/28197368 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 msgr2=0x7f42a80ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:58.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.264+0000 7f42a7fff640 1 --2- 172.21.15.81:0/28197368 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 0x7f42a80ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:58.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.264+0000 7f42a7fff640 1 -- 172.21.15.81:0/28197368 >> v1:172.21.15.93:6789/0 conn(0x7f42a80c90f0 legacy=0x7f42a806b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:58.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.264+0000 7f42a7fff640 1 -- 172.21.15.81:0/28197368 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80cb100 con 0x7f42a80c9550 2023-11-08T16:37:58.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.264+0000 7f42a77fe640 1 --2- 172.21.15.81:0/28197368 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 0x7f42a80ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:37:58.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.264+0000 7f42a6ffd640 1 -- 172.21.15.81:0/28197368 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571462032 0 0) 0x7f42a806cc60 con 0x7f42a80c90f0 2023-11-08T16:37:58.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.265+0000 7f42a7fff640 1 --2- 172.21.15.81:0/28197368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a80c9550 0x7f42a806b9c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009d90 tx=0x7f42a002beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=311a340a9a352033 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:58.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.266+0000 7f42a6ffd640 1 -- 172.21.15.81:0/28197368 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a002cc60 con 0x7f42a80c9550 2023-11-08T16:37:58.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.266+0000 7f42a6ffd640 1 -- 172.21.15.81:0/28197368 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a002cdc0 con 0x7f42a80c9550 2023-11-08T16:37:58.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.266+0000 7f42a6ffd640 1 -- 172.21.15.81:0/28197368 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0034660 con 0x7f42a80c9550 2023-11-08T16:37:58.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.267+0000 7f42add65640 1 -- 172.21.15.81:0/28197368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a80c9550 msgr2=0x7f42a806b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:58.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.267+0000 7f42add65640 1 --2- 172.21.15.81:0/28197368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a80c9550 0x7f42a806b9c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009d90 tx=0x7f42a002beb0 comp rx=0 tx=0).stop 2023-11-08T16:37:58.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.267+0000 7f42add65640 1 -- 172.21.15.81:0/28197368 shutdown_connections 2023-11-08T16:37:58.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.267+0000 7f42add65640 1 --2- 172.21.15.81:0/28197368 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a80c9550 0x7f42a806b9c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:58.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.267+0000 7f42add65640 1 --2- 172.21.15.81:0/28197368 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 0x7f42a80ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:58.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.267+0000 7f42add65640 1 -- 172.21.15.81:0/28197368 >> 172.21.15.81:0/28197368 conn(0x7f42a80ceaa0 msgr2=0x7f42a80cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:58.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.267+0000 7f42add65640 1 -- 172.21.15.81:0/28197368 shutdown_connections 2023-11-08T16:37:58.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.267+0000 7f42add65640 1 -- 172.21.15.81:0/28197368 wait complete. 2023-11-08T16:37:58.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.269+0000 7f42add65640 1 Processor -- start 2023-11-08T16:37:58.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.269+0000 7f42add65640 1 -- start start 2023-11-08T16:37:58.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.270+0000 7f42add65640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a80c90f0 0x7f42a80ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:58.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42add65640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a80c9550 0x7f42a8156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:58.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42add65640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 0x7f42a80cc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:58.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42add65640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80ce6d0 con 0x7f42a80c90f0 2023-11-08T16:37:58.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42add65640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80cd0f0 con 0x7f42a80c9550 2023-11-08T16:37:58.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42add65640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80cd3d0 con 0x7f42a815aeb0 2023-11-08T16:37:58.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42a77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a80c90f0 0x7f42a80ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:58.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42a77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a80c90f0 0x7f42a80ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41506/0 (socket says 172.21.15.81:41506) 2023-11-08T16:37:58.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42a77fe640 1 -- 172.21.15.81:0/3290468992 learned_addr learned my addr 172.21.15.81:0/3290468992 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:58.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f429ffff640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a80c9550 0x7f42a8156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:58.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42a7fff640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 0x7f42a80cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:58.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42a77fe640 1 -- 172.21.15.81:0/3290468992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 msgr2=0x7f42a80cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:58.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42a77fe640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 0x7f42a80cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:58.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42a77fe640 1 -- 172.21.15.81:0/3290468992 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a80c9550 msgr2=0x7f42a8156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:58.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42a77fe640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a80c9550 0x7f42a8156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:58.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42a77fe640 1 -- 172.21.15.81:0/3290468992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4298009460 con 0x7f42a80c90f0 2023-11-08T16:37:58.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.271+0000 7f42a7fff640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 0x7f42a80cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:58.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.272+0000 7f42a77fe640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a80c90f0 0x7f42a80ce190 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f429800e9b0 tx=0x7f429800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:58.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.272+0000 7f42a4ff9640 1 -- 172.21.15.81:0/3290468992 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429800c9a0 con 0x7f42a80c90f0 2023-11-08T16:37:58.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.272+0000 7f42a4ff9640 1 -- 172.21.15.81:0/3290468992 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429800cb00 con 0x7f42a80c90f0 2023-11-08T16:37:58.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.272+0000 7f42add65640 1 -- 172.21.15.81:0/3290468992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a0009a70 con 0x7f42a80c90f0 2023-11-08T16:37:58.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.272+0000 7f42a4ff9640 1 -- 172.21.15.81:0/3290468992 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429800ce70 con 0x7f42a80c90f0 2023-11-08T16:37:58.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.272+0000 7f42add65640 1 -- 172.21.15.81:0/3290468992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a80cd990 con 0x7f42a80c90f0 2023-11-08T16:37:58.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.273+0000 7f42add65640 1 -- 172.21.15.81:0/3290468992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a80743f0 con 0x7f42a80c90f0 2023-11-08T16:37:58.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.274+0000 7f42a4ff9640 1 -- 172.21.15.81:0/3290468992 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4298005990 con 0x7f42a80c90f0 2023-11-08T16:37:58.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.276+0000 7f42a4ff9640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806ae70 0x7f428806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:58.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.276+0000 7f42a4ff9640 1 -- 172.21.15.81:0/3290468992 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6171+0+0 (secure 0 0 0) 0x7f4298089c20 con 0x7f42a80c90f0 2023-11-08T16:37:58.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.276+0000 7f429ffff640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806ae70 0x7f428806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:58.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.277+0000 7f429ffff640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806ae70 0x7f428806d330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f42a815b5e0 tx=0x7f4290009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:58.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.280+0000 7f42a4ff9640 1 -- 172.21.15.81:0/3290468992 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4298054b00 con 0x7f42a80c90f0 2023-11-08T16:37:58.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.506+0000 7f42add65640 1 -- 172.21.15.81:0/3290468992 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f42a806e210 con 0x7f42a80c90f0 2023-11-08T16:37:58.510 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:37:58.507+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:37:58.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.511+0000 7f42a4ff9640 1 -- 172.21.15.81:0/3290468992 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4298020020 con 0x7f42a80c90f0 2023-11-08T16:37:58.514 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:37:58.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.513+0000 7f42add65640 1 -- 172.21.15.81:0/3290468992 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806ae70 msgr2=0x7f428806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:58.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.513+0000 7f42add65640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806ae70 0x7f428806d330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f42a815b5e0 tx=0x7f4290009040 comp rx=0 tx=0).stop 2023-11-08T16:37:58.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.514+0000 7f42add65640 1 -- 172.21.15.81:0/3290468992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a80c90f0 msgr2=0x7f42a80ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:58.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.514+0000 7f42add65640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a80c90f0 0x7f42a80ce190 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f429800e9b0 tx=0x7f429800ee70 comp rx=0 tx=0).stop 2023-11-08T16:37:58.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.514+0000 7f42add65640 1 -- 172.21.15.81:0/3290468992 shutdown_connections 2023-11-08T16:37:58.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.514+0000 7f42add65640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806ae70 0x7f428806d330 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:58.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.514+0000 7f42add65640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a815aeb0 0x7f42a80cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:58.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.514+0000 7f42add65640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a80c9550 0x7f42a8156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:58.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.514+0000 7f42add65640 1 --2- 172.21.15.81:0/3290468992 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a80c90f0 0x7f42a80ce190 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:58.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.514+0000 7f42add65640 1 -- 172.21.15.81:0/3290468992 >> 172.21.15.81:0/3290468992 conn(0x7f42a80ceaa0 msgr2=0x7f42a806b2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:58.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.514+0000 7f42add65640 1 -- 172.21.15.81:0/3290468992 shutdown_connections 2023-11-08T16:37:58.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:58.514+0000 7f42add65640 1 -- 172.21.15.81:0/3290468992 wait complete. 2023-11-08T16:37:58.535 INFO:tasks.ceph.mds.a.smithi081.stderr: -3538> 2023-11-08T16:37:36.811+0000 7f06ce4dc640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:37:58.539 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:37:58.545 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:37:58.605 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:37:58.684 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2023-11-08T16:37:58.684 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2023-11-08T16:37:58.684 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:37:58.685 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2023-11-08T16:37:58.685 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.685 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2023-11-08T16:37:58.685 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.685 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2023-11-08T16:37:58.686 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.686 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2023-11-08T16:37:58.686 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2023-11-08T16:37:58.686 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.686 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2023-11-08T16:37:58.687 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2023-11-08T16:37:58.687 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.687 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:37:58.687 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2023-11-08T16:37:58.687 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.688 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:37:58.688 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:37:58.688 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.689 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2023-11-08T16:37:58.689 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.689 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2023-11-08T16:37:58.689 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2023-11-08T16:37:58.690 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:37:58.690 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:37:58.690 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2023-11-08T16:37:58.691 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:37:58.691 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2023-11-08T16:37:58.691 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:37:58.691 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:37:58.692 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2023-11-08T16:37:58.692 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.692 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:37:58.692 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.692 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2023-11-08T16:37:58.693 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.693 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2023-11-08T16:37:58.693 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.693 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2023-11-08T16:37:58.693 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2023-11-08T16:37:58.694 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2023-11-08T16:37:58.694 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:37:58.694 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:37:58.694 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.695 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2023-11-08T16:37:58.695 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2023-11-08T16:37:58.695 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.695 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2023-11-08T16:37:58.696 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2023-11-08T16:37:58.696 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:37:58.696 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.696 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2023-11-08T16:37:58.696 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2023-11-08T16:37:58.696 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.697 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2023-11-08T16:37:58.697 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2023-11-08T16:37:58.697 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2023-11-08T16:37:58.697 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2023-11-08T16:37:58.698 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.698 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2023-11-08T16:37:58.698 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2023-11-08T16:37:58.698 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.698 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:37:58.699 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.699 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2023-11-08T16:37:58.699 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.700 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2023-11-08T16:37:58.700 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.700 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:37:58.700 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2023-11-08T16:37:58.700 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.700 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2023-11-08T16:37:58.701 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2023-11-08T16:37:58.701 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2023-11-08T16:37:58.701 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2023-11-08T16:37:58.701 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2023-11-08T16:37:58.701 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2023-11-08T16:37:58.702 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2023-11-08T16:37:58.702 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2023-11-08T16:37:58.702 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2023-11-08T16:37:58.702 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:37:58.702 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2023-11-08T16:37:58.703 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2023-11-08T16:37:58.703 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:37:58.703 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2023-11-08T16:37:58.703 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:37:58.703 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:37:58.704 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2023-11-08T16:37:58.704 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:37:58.704 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.704 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2023-11-08T16:37:58.704 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:37:58.704 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2023-11-08T16:37:58.705 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:37:58.705 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:37:58.705 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:37:58.705 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:37:58.705 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2023-11-08T16:37:58.706 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.706 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2023-11-08T16:37:58.706 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:37:58.706 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:37:58.706 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2023-11-08T16:37:58.707 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:37:58.707 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2023-11-08T16:37:58.707 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2023-11-08T16:37:58.707 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:37:58.707 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2023-11-08T16:37:58.707 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2023-11-08T16:37:58.708 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:37:58.708 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.708 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:37:58.708 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:37:58.709 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:37:58.709 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.709 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:37:58.709 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:37:58.709 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:37:58.709 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.710 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2023-11-08T16:37:58.710 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:37:58.710 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.710 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:37:58.710 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2023-11-08T16:37:58.711 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.711 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2023-11-08T16:37:58.711 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2023-11-08T16:37:58.711 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2023-11-08T16:37:58.711 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2023-11-08T16:37:58.711 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:37:58.712 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:37:58.712 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2023-11-08T16:37:58.712 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2023-11-08T16:37:58.712 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:37:58.712 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2023-11-08T16:37:58.712 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:37:58.713 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:37:58.713 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2023-11-08T16:37:58.713 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2023-11-08T16:37:58.713 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2023-11-08T16:37:58.713 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:37:58.713 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:37:58.714 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:37:58.714 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:37:58.714 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2023-11-08T16:37:58.714 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2023-11-08T16:37:58.714 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2023-11-08T16:37:58.715 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:37:58.715 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2023-11-08T16:37:58.715 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = False 2023-11-08T16:37:58.715 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2023-11-08T16:37:58.715 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = False 2023-11-08T16:37:58.716 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:37:58.716 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:37:58.716 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:37:58.726 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-08T16:37:58.726 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2023-11-08T16:37:58.726 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:37:58.726 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2023-11-08T16:37:58.727 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.727 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2023-11-08T16:37:58.727 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.727 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2023-11-08T16:37:58.727 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.728 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2023-11-08T16:37:58.728 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2023-11-08T16:37:58.728 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.728 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2023-11-08T16:37:58.728 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2023-11-08T16:37:58.728 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.729 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:37:58.729 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2023-11-08T16:37:58.729 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.729 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:37:58.729 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:37:58.730 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.730 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2023-11-08T16:37:58.730 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.730 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2023-11-08T16:37:58.730 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2023-11-08T16:37:58.730 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:37:58.731 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:37:58.731 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2023-11-08T16:37:58.731 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:37:58.731 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2023-11-08T16:37:58.731 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:37:58.732 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:37:58.732 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2023-11-08T16:37:58.732 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.732 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:37:58.732 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.733 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2023-11-08T16:37:58.733 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.733 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2023-11-08T16:37:58.733 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.733 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2023-11-08T16:37:58.733 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2023-11-08T16:37:58.734 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2023-11-08T16:37:58.734 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:37:58.734 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:37:58.734 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.734 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2023-11-08T16:37:58.735 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2023-11-08T16:37:58.735 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.735 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2023-11-08T16:37:58.735 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2023-11-08T16:37:58.735 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:37:58.736 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.736 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2023-11-08T16:37:58.736 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2023-11-08T16:37:58.736 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.736 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2023-11-08T16:37:58.736 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2023-11-08T16:37:58.737 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2023-11-08T16:37:58.737 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2023-11-08T16:37:58.737 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.737 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2023-11-08T16:37:58.737 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2023-11-08T16:37:58.738 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.738 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:37:58.738 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.738 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2023-11-08T16:37:58.739 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.739 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2023-11-08T16:37:58.739 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.739 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:37:58.739 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2023-11-08T16:37:58.739 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.740 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2023-11-08T16:37:58.740 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2023-11-08T16:37:58.740 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2023-11-08T16:37:58.740 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2023-11-08T16:37:58.740 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2023-11-08T16:37:58.741 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2023-11-08T16:37:58.741 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2023-11-08T16:37:58.741 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2023-11-08T16:37:58.741 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2023-11-08T16:37:58.741 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:37:58.742 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2023-11-08T16:37:58.742 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2023-11-08T16:37:58.742 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:37:58.742 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2023-11-08T16:37:58.742 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:37:58.743 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:37:58.743 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2023-11-08T16:37:58.743 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:37:58.743 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.743 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2023-11-08T16:37:58.743 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:37:58.744 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2023-11-08T16:37:58.744 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:37:58.744 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:37:58.744 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:37:58.744 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:37:58.744 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2023-11-08T16:37:58.745 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.745 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2023-11-08T16:37:58.745 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:37:58.745 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:37:58.745 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2023-11-08T16:37:58.745 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:37:58.746 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2023-11-08T16:37:58.746 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2023-11-08T16:37:58.746 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:37:58.746 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2023-11-08T16:37:58.746 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2023-11-08T16:37:58.747 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:37:58.747 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.747 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:37:58.747 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:37:58.747 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:37:58.748 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.748 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:37:58.748 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:37:58.748 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:37:58.748 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.748 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2023-11-08T16:37:58.749 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:37:58.749 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.749 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:37:58.749 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2023-11-08T16:37:58.749 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.750 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2023-11-08T16:37:58.750 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2023-11-08T16:37:58.750 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2023-11-08T16:37:58.750 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2023-11-08T16:37:58.750 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:37:58.750 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:37:58.751 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2023-11-08T16:37:58.751 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2023-11-08T16:37:58.751 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:37:58.751 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2023-11-08T16:37:58.751 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:37:58.752 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:37:58.752 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2023-11-08T16:37:58.752 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2023-11-08T16:37:58.752 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2023-11-08T16:37:58.752 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:37:58.752 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:37:58.753 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:37:58.753 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:37:58.753 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2023-11-08T16:37:58.753 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2023-11-08T16:37:58.754 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2023-11-08T16:37:58.754 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:37:58.754 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2023-11-08T16:37:58.754 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = False 2023-11-08T16:37:58.754 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2023-11-08T16:37:58.754 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = False 2023-11-08T16:37:58.755 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:37:58.755 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:37:58.755 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:37:58.771 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:37:58.779 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:37:58.935 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2023-11-08T16:37:58.936 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2023-11-08T16:37:58.936 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:37:58.936 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2023-11-08T16:37:58.936 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.936 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2023-11-08T16:37:58.937 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.937 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2023-11-08T16:37:58.937 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.937 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2023-11-08T16:37:58.937 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2023-11-08T16:37:58.938 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.938 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2023-11-08T16:37:58.938 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2023-11-08T16:37:58.938 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.938 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:37:58.939 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2023-11-08T16:37:58.939 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.939 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:37:58.939 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:37:58.939 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.939 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2023-11-08T16:37:58.940 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.940 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2023-11-08T16:37:58.940 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2023-11-08T16:37:58.940 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:37:58.941 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:37:58.941 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2023-11-08T16:37:58.941 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:37:58.941 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2023-11-08T16:37:58.941 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:37:58.942 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:37:58.942 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2023-11-08T16:37:58.942 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.942 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:37:58.942 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.943 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2023-11-08T16:37:58.943 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.943 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2023-11-08T16:37:58.943 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.943 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2023-11-08T16:37:58.944 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2023-11-08T16:37:58.944 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2023-11-08T16:37:58.944 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:37:58.944 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:37:58.944 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.945 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2023-11-08T16:37:58.945 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2023-11-08T16:37:58.945 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.945 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2023-11-08T16:37:58.945 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2023-11-08T16:37:58.945 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:37:58.946 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.946 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2023-11-08T16:37:58.946 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2023-11-08T16:37:58.946 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.946 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2023-11-08T16:37:58.947 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2023-11-08T16:37:58.947 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2023-11-08T16:37:58.947 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2023-11-08T16:37:58.947 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.947 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2023-11-08T16:37:58.948 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2023-11-08T16:37:58.948 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.948 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:37:58.948 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.948 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2023-11-08T16:37:58.948 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.949 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2023-11-08T16:37:58.949 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.949 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:37:58.949 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2023-11-08T16:37:58.949 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.950 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2023-11-08T16:37:58.950 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2023-11-08T16:37:58.950 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2023-11-08T16:37:58.950 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2023-11-08T16:37:58.950 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2023-11-08T16:37:58.951 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2023-11-08T16:37:58.951 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2023-11-08T16:37:58.951 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2023-11-08T16:37:58.951 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2023-11-08T16:37:58.951 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:37:58.951 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2023-11-08T16:37:58.952 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2023-11-08T16:37:58.952 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:37:58.952 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2023-11-08T16:37:58.952 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:37:58.952 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:37:58.953 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2023-11-08T16:37:58.953 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:37:58.953 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.953 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2023-11-08T16:37:58.953 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:37:58.954 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2023-11-08T16:37:58.954 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:37:58.954 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:37:58.954 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:37:58.954 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:37:58.955 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2023-11-08T16:37:58.955 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.955 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2023-11-08T16:37:58.955 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:37:58.955 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:37:58.956 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2023-11-08T16:37:58.956 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:37:58.956 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2023-11-08T16:37:58.956 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2023-11-08T16:37:58.956 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:37:58.957 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2023-11-08T16:37:58.957 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2023-11-08T16:37:58.957 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:37:58.957 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.957 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:37:58.958 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:37:58.958 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:37:58.958 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.958 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:37:58.958 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:37:58.958 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:37:58.959 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.959 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2023-11-08T16:37:58.959 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:37:58.959 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.959 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:37:58.960 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2023-11-08T16:37:58.960 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:37:58.960 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2023-11-08T16:37:58.960 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2023-11-08T16:37:58.960 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2023-11-08T16:37:58.961 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2023-11-08T16:37:58.961 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:37:58.961 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:37:58.961 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2023-11-08T16:37:58.961 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2023-11-08T16:37:58.962 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:37:58.962 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2023-11-08T16:37:58.962 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:37:58.962 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:37:58.962 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2023-11-08T16:37:58.963 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2023-11-08T16:37:58.963 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2023-11-08T16:37:58.963 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:37:58.963 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:37:58.963 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:37:58.963 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:37:58.964 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2023-11-08T16:37:58.964 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2023-11-08T16:37:58.964 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2023-11-08T16:37:58.964 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:37:58.964 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2023-11-08T16:37:58.965 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = False 2023-11-08T16:37:58.965 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2023-11-08T16:37:58.965 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = False 2023-11-08T16:37:58.965 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:37:58.965 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:37:58.966 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:37:58.966 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-08T16:37:58.966 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2023-11-08T16:37:58.967 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:37:58.967 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2023-11-08T16:37:58.967 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.967 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2023-11-08T16:37:58.968 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.968 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2023-11-08T16:37:58.968 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.968 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2023-11-08T16:37:58.968 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2023-11-08T16:37:58.968 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.969 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2023-11-08T16:37:58.969 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2023-11-08T16:37:58.969 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.969 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:37:58.969 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2023-11-08T16:37:58.970 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.970 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:37:58.970 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:37:58.970 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.971 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2023-11-08T16:37:58.971 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.971 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2023-11-08T16:37:58.971 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2023-11-08T16:37:58.971 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:37:58.971 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:37:58.972 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2023-11-08T16:37:58.972 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:37:58.972 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2023-11-08T16:37:58.972 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:37:58.972 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:37:58.973 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2023-11-08T16:37:58.973 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.973 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:37:58.973 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.973 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2023-11-08T16:37:58.974 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.974 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2023-11-08T16:37:58.974 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.974 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2023-11-08T16:37:58.974 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2023-11-08T16:37:58.975 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2023-11-08T16:37:58.975 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:37:58.975 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:37:58.975 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.975 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2023-11-08T16:37:58.976 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2023-11-08T16:37:58.976 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.976 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2023-11-08T16:37:58.976 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2023-11-08T16:37:58.976 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:37:58.977 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.977 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2023-11-08T16:37:58.977 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2023-11-08T16:37:58.977 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.977 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2023-11-08T16:37:58.977 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2023-11-08T16:37:58.978 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2023-11-08T16:37:58.978 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2023-11-08T16:37:58.978 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.978 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2023-11-08T16:37:58.978 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2023-11-08T16:37:58.979 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.979 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:37:58.979 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.979 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2023-11-08T16:37:58.979 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.980 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2023-11-08T16:37:58.980 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.980 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:37:58.980 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2023-11-08T16:37:58.980 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.981 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2023-11-08T16:37:58.981 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2023-11-08T16:37:58.981 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2023-11-08T16:37:58.981 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2023-11-08T16:37:58.981 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2023-11-08T16:37:58.981 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2023-11-08T16:37:58.982 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2023-11-08T16:37:58.982 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2023-11-08T16:37:58.982 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2023-11-08T16:37:58.982 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:37:58.983 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2023-11-08T16:37:58.983 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2023-11-08T16:37:58.983 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:37:58.983 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2023-11-08T16:37:58.983 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:37:58.983 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:37:58.984 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2023-11-08T16:37:58.984 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:37:58.984 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.984 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2023-11-08T16:37:58.984 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:37:58.985 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2023-11-08T16:37:58.985 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:37:58.985 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:37:58.985 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:37:58.985 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:37:58.986 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2023-11-08T16:37:58.986 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.986 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2023-11-08T16:37:58.986 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:37:58.986 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:37:58.986 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2023-11-08T16:37:58.987 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:37:58.987 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2023-11-08T16:37:58.987 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2023-11-08T16:37:58.987 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:37:58.987 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2023-11-08T16:37:58.988 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2023-11-08T16:37:58.988 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:37:58.988 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.988 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:37:58.988 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:37:58.989 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:37:58.989 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.989 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:37:58.989 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:37:58.989 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:37:58.989 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.990 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2023-11-08T16:37:58.990 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:37:58.990 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.990 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:37:58.990 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2023-11-08T16:37:58.991 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:37:58.991 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2023-11-08T16:37:58.991 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2023-11-08T16:37:58.991 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2023-11-08T16:37:58.991 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2023-11-08T16:37:58.992 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:37:58.992 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:37:58.992 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2023-11-08T16:37:58.992 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2023-11-08T16:37:58.992 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:37:58.993 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2023-11-08T16:37:58.993 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:37:58.993 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:37:58.993 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2023-11-08T16:37:58.994 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2023-11-08T16:37:58.994 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2023-11-08T16:37:58.994 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:37:58.994 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:37:58.994 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:37:58.995 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:37:58.995 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2023-11-08T16:37:58.995 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2023-11-08T16:37:58.995 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2023-11-08T16:37:58.995 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:37:58.996 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2023-11-08T16:37:58.996 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = False 2023-11-08T16:37:58.996 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2023-11-08T16:37:58.996 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = False 2023-11-08T16:37:58.996 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:37:58.997 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:37:58.997 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:37:58.999 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T16:37:59.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.107+0000 7fe04cc3af40 1 Processor -- start 2023-11-08T16:37:59.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.107+0000 7fe04cc3af40 1 -- start start 2023-11-08T16:37:59.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.108+0000 7fe04cc3af40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564f6d9d8ff0 0x564f6da82a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.108+0000 7fe04cc3af40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564f6d9def10 0x564f6da7ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.108+0000 7fe04cc3af40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6da82f40 0x564f6da7f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.108+0000 7fe04cc3af40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f6d95a340 con 0x564f6da82f40 2023-11-08T16:37:59.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.108+0000 7fe04cc3af40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f6d8f4a70 con 0x564f6d9def10 2023-11-08T16:37:59.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.108+0000 7fe04cc3af40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f6d8f4710 con 0x564f6d9d8ff0 2023-11-08T16:37:59.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.108+0000 7fe04a7e3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6da82f40 0x564f6da7f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.109+0000 7fe04a7e3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6da82f40 0x564f6da7f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41538/0 (socket says 172.21.15.81:41538) 2023-11-08T16:37:59.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.109+0000 7fe04a7e3640 1 -- 172.21.15.81:0/1134279625 learned_addr learned my addr 172.21.15.81:0/1134279625 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:59.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.109+0000 7fe0497e1640 1 --2- 172.21.15.81:0/1134279625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564f6d9def10 0x564f6da7ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.109+0000 7fe049fe2640 1 --2- 172.21.15.81:0/1134279625 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564f6d9d8ff0 0x564f6da82a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.109+0000 7fe04a7e3640 1 -- 172.21.15.81:0/1134279625 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564f6d9d8ff0 msgr2=0x564f6da82a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:59.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.109+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/1134279625 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564f6d9d8ff0 0x564f6da82a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.109+0000 7fe04a7e3640 1 -- 172.21.15.81:0/1134279625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564f6d9def10 msgr2=0x564f6da7ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:59.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.109+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/1134279625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564f6d9def10 0x564f6da7ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.109+0000 7fe04a7e3640 1 -- 172.21.15.81:0/1134279625 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f6d900680 con 0x564f6da82f40 2023-11-08T16:37:59.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.109+0000 7fe0497e1640 1 --2- 172.21.15.81:0/1134279625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564f6d9def10 0x564f6da7ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:37:59.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.109+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/1134279625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6da82f40 0x564f6da7f1d0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe04401d3d0 tx=0x7fe04404d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc16b549fd581383 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe03bfff640 1 -- 172.21.15.81:0/1134279625 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04404b930 con 0x564f6da82f40 2023-11-08T16:37:59.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe03bfff640 1 -- 172.21.15.81:0/1134279625 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe044001430 con 0x564f6da82f40 2023-11-08T16:37:59.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe03bfff640 1 -- 172.21.15.81:0/1134279625 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04405f480 con 0x564f6da82f40 2023-11-08T16:37:59.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe04cc3af40 1 -- 172.21.15.81:0/1134279625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6da82f40 msgr2=0x564f6da7f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/1134279625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6da82f40 0x564f6da7f1d0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fe04401d3d0 tx=0x7fe04404d000 comp rx=0 tx=0).stop 2023-11-08T16:37:59.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe04cc3af40 1 -- 172.21.15.81:0/1134279625 shutdown_connections 2023-11-08T16:37:59.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/1134279625 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6da82f40 0x564f6da7f1d0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/1134279625 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564f6d9def10 0x564f6da7ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/1134279625 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564f6d9d8ff0 0x564f6da82a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe04cc3af40 1 -- 172.21.15.81:0/1134279625 >> 172.21.15.81:0/1134279625 conn(0x564f6d9f5c60 msgr2=0x564f6d9a8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:59.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe04cc3af40 1 -- 172.21.15.81:0/1134279625 shutdown_connections 2023-11-08T16:37:59.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.110+0000 7fe04cc3af40 1 -- 172.21.15.81:0/1134279625 wait complete. 2023-11-08T16:37:59.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.111+0000 7fe04cc3af40 1 Processor -- start 2023-11-08T16:37:59.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.111+0000 7fe04cc3af40 1 -- start start 2023-11-08T16:37:59.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.112+0000 7fe04cc3af40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564f6d9d8ff0 0x564f6d9f0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.112+0000 7fe04cc3af40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6d9def10 0x564f6d9f0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.112+0000 7fe04cc3af40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564f6da82f40 0x564f6d9dd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.112+0000 7fe04cc3af40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f6d8e8520 con 0x564f6d9def10 2023-11-08T16:37:59.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.112+0000 7fe04cc3af40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f6d9dac40 con 0x564f6d9d8ff0 2023-11-08T16:37:59.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.112+0000 7fe04cc3af40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f6d9da960 con 0x564f6da82f40 2023-11-08T16:37:59.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.112+0000 7fe0497e1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6d9def10 0x564f6d9f0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.112+0000 7fe0497e1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6d9def10 0x564f6d9f0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41548/0 (socket says 172.21.15.81:41548) 2023-11-08T16:37:59.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.112+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 learned_addr learned my addr 172.21.15.81:0/2042972598 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:59.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe049fe2640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564f6d9d8ff0 0x564f6d9f0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564f6da82f40 0x564f6d9dd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564f6da82f40 msgr2=0x564f6d9dd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe0497e1640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564f6da82f40 0x564f6d9dd1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564f6d9d8ff0 msgr2=0x564f6d9f0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe0497e1640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564f6d9d8ff0 0x564f6d9f0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe034001e10 con 0x564f6d9def10 2023-11-08T16:37:59.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe0497e1640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6d9def10 0x564f6d9f0640 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe040005fd0 tx=0x7fe0400074b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe03affd640 1 -- 172.21.15.81:0/2042972598 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040005e70 con 0x564f6d9def10 2023-11-08T16:37:59.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe03affd640 1 -- 172.21.15.81:0/2042972598 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe040007e90 con 0x564f6d9def10 2023-11-08T16:37:59.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564f6d8f3830 con 0x564f6d9def10 2023-11-08T16:37:59.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.113+0000 7fe03affd640 1 -- 172.21.15.81:0/2042972598 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe040015840 con 0x564f6d9def10 2023-11-08T16:37:59.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.114+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f6d8f3290 con 0x564f6d9def10 2023-11-08T16:37:59.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.116+0000 7fe03affd640 1 -- 172.21.15.81:0/2042972598 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe0400159a0 con 0x564f6d9def10 2023-11-08T16:37:59.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.116+0000 7fe03affd640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe02c069850 0x7fe02c06bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.117+0000 7fe03affd640 1 -- 172.21.15.81:0/2042972598 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6257+0+0 (secure 0 0 0) 0x7fe040087cd0 con 0x564f6d9def10 2023-11-08T16:37:59.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.117+0000 7fe049fe2640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe02c069850 0x7fe02c06bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.127 INFO:teuthology.orchestra.run.smithi081.stdout:Warning: using slow linear search 2023-11-08T16:37:59.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.117+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x564f6da663f0 0x564f6db26420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.117+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:1 10.0 10:00000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x564f6da663f0 2023-11-08T16:37:59.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.117+0000 7fe049fe2640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe02c069850 0x7fe02c06bd10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe03c005fd0 tx=0x7fe03c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.118+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x564f6da663f0 0x564f6db26420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.119+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x564f6da663f0 0x564f6db26420 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.120+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(1 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.120+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7fe04406b040 0x7fe04406c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.120+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:2 10.10 10:08000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe04406b040 2023-11-08T16:37:59.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.120+0000 7fe049fe2640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7fe04406b040 0x7fe04406c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.121+0000 7fe049fe2640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7fe04406b040 0x7fe04406c430 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.122+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(2 [pgnls start_epoch 85 out=177b] v84'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.122+0000 7fe049fe2640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fe03c015be0 0x7fe03c021920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.122+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:3 10.8 10:10000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe03c015be0 2023-11-08T16:37:59.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.123+0000 7fe0497e1640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fe03c015be0 0x7fe03c021920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.123+0000 7fe0497e1640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fe03c015be0 0x7fe03c021920 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.124+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(3 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe040086040 con 0x7fe03c015be0 2023-11-08T16:37:59.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.124+0000 7fe0497e1640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7fe04001f060 0x7fe040055810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.124+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:4 10.18 10:18000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe04001f060 2023-11-08T16:37:59.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.124+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7fe04001f060 0x7fe040055810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.125+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7fe04001f060 0x7fe040055810 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.125+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(4 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe044065020 con 0x7fe04001f060 2023-11-08T16:37:59.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.125+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 10.4 10:20000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe03c015be0 2023-11-08T16:37:59.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.125+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(5 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe040086040 con 0x7fe03c015be0 2023-11-08T16:37:59.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.125+0000 7fe0497e1640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fe040056b10 0x7fe040087400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.125+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:6 10.14 10:28000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe040056b10 2023-11-08T16:37:59.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.126+0000 7fe049fe2640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fe040056b10 0x7fe040087400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.126+0000 7fe049fe2640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fe040056b10 0x7fe040087400 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.127+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(6 [pgnls start_epoch 85 out=149b] v84'21 uv21 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe040056b10 2023-11-08T16:37:59.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.127+0000 7fe049fe2640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fe03c0208b0 0x7fe03c02e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.127+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:7 10.c 10:30000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe03c0208b0 2023-11-08T16:37:59.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.127+0000 7fe0497e1640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fe03c0208b0 0x7fe03c02e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.128+0000 7fe0497e1640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fe03c0208b0 0x7fe03c02e340 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.129+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(7 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe040086040 con 0x7fe03c0208b0 2023-11-08T16:37:59.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.129+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:8 10.1c 10:38000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x564f6da663f0 2023-11-08T16:37:59.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.130+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(8 [pgnls start_epoch 85 out=73b] v80'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.130+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:9 10.2 10:40000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe040056b10 2023-11-08T16:37:59.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.131+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(9 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe040056b10 2023-11-08T16:37:59.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.131+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:10 10.12 10:48000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x564f6da663f0 2023-11-08T16:37:59.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.132+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(10 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.132+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:11 10.a 10:50000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x564f6da663f0 2023-11-08T16:37:59.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.133+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(11 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.133+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:12 10.1a 10:58000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x564f6da663f0 2023-11-08T16:37:59.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.134+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(12 [pgnls start_epoch 85 out=49b] v80'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.134+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7fe0340035f0 0x7fe04407b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.134+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:13 10.6 10:60000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe0340035f0 2023-11-08T16:37:59.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.134+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7fe0340035f0 0x7fe04407b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.136+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7fe0340035f0 0x7fe04407b420 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.137+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(13 [pgnls start_epoch 85 out=77b] v84'5 uv5 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7fe044065020 con 0x7fe0340035f0 2023-11-08T16:37:59.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.137+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:14 10.16 10:68000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x564f6da663f0 2023-11-08T16:37:59.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.137+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(14 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.138+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:15 10.e 10:70000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe04406b040 2023-11-08T16:37:59.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.138+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(15 [pgnls start_epoch 85 out=73b] v80'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.138+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:16 10.1e 10:78000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe03c015be0 2023-11-08T16:37:59.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.139+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(16 [pgnls start_epoch 85 out=49b] v80'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe040086040 con 0x7fe03c015be0 2023-11-08T16:37:59.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.139+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:17 10.1 10:80000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe04406b040 2023-11-08T16:37:59.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.140+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(17 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.140+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:18 10.11 10:88000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe0340035f0 2023-11-08T16:37:59.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.141+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(18 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe044065020 con 0x7fe0340035f0 2023-11-08T16:37:59.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.141+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:19 10.9 10:90000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe0340035f0 2023-11-08T16:37:59.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.142+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(19 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe044065020 con 0x7fe0340035f0 2023-11-08T16:37:59.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.142+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:20 10.19 10:98000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe040056b10 2023-11-08T16:37:59.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.142+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(20 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe040056b10 2023-11-08T16:37:59.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.142+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:21 10.5 10:a0000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe04001f060 2023-11-08T16:37:59.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.143+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(21 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe044065020 con 0x7fe04001f060 2023-11-08T16:37:59.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.143+0000 7fe04a7e3640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7fe04408b040 0x7fe04408b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.143+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:22 10.15 10:a8000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe04408b040 2023-11-08T16:37:59.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.143+0000 7fe049fe2640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7fe04408b040 0x7fe04408b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.144+0000 7fe049fe2640 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7fe04408b040 0x7fe04408b930 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.145+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(22 [pgnls start_epoch 85 out=97b] v80'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04408b040 2023-11-08T16:37:59.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.145+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:23 10.d 10:b0000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x564f6da663f0 2023-11-08T16:37:59.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.145+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(23 [pgnls start_epoch 85 out=74b] v80'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.145+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:24 10.1d 10:b8000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe04406b040 2023-11-08T16:37:59.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.146+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(24 [pgnls start_epoch 85 out=73b] v80'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.146+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:25 10.3 10:c0000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe04001f060 2023-11-08T16:37:59.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.147+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(25 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe044065020 con 0x7fe04001f060 2023-11-08T16:37:59.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.147+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:26 10.13 10:c8000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe040056b10 2023-11-08T16:37:59.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.148+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(26 [pgnls start_epoch 85 out=98b] v84'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe040056b10 2023-11-08T16:37:59.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.148+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:27 10.b 10:d0000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x564f6da663f0 2023-11-08T16:37:59.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.149+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(27 [pgnls start_epoch 85 out=76b] v84'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.149+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:28 10.1b 10:d8000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe04408b040 2023-11-08T16:37:59.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.149+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(28 [pgnls start_epoch 85 out=49b] v80'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04408b040 2023-11-08T16:37:59.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.149+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:29 10.7 10:e0000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x564f6da663f0 2023-11-08T16:37:59.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.150+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(29 [pgnls start_epoch 85 out=73b] v80'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.150+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:30 10.17 10:e8000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe04406b040 2023-11-08T16:37:59.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.151+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(30 [pgnls start_epoch 85 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.152+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:31 10.f 10:f0000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x564f6da663f0 2023-11-08T16:37:59.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.153+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(31 [pgnls start_epoch 85 out=73b] v80'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.153+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:32 10.1f 10:f8000000:%01:::head [pgnls start_epoch 85 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e85) v8 -- 0x564f6d9f0b80 con 0x7fe03c0208b0 2023-11-08T16:37:59.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(32 [pgnls start_epoch 85 out=101b] v84'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7fe040086040 con 0x7fe03c0208b0 2023-11-08T16:37:59.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:33 10.10 10:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6d9ddb80 con 0x7fe04406b040 2023-11-08T16:37:59.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:34 10.10 10:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6da65ac0 con 0x7fe04406b040 2023-11-08T16:37:59.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:35 10.10 10:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db26960 con 0x7fe04406b040 2023-11-08T16:37:59.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:36 10.10 10:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db27160 con 0x7fe04406b040 2023-11-08T16:37:59.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:37 10.10 10:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db27960 con 0x7fe04406b040 2023-11-08T16:37:59.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:38 10.14 10:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db28160 con 0x7fe040056b10 2023-11-08T16:37:59.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:39 10.14 10:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db28960 con 0x7fe040056b10 2023-11-08T16:37:59.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:40 10.14 10:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db29160 con 0x7fe040056b10 2023-11-08T16:37:59.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:41 10.14 10:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db29960 con 0x7fe040056b10 2023-11-08T16:37:59.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:42 10.1c 10:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2a160 con 0x564f6da663f0 2023-11-08T16:37:59.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:43 10.6 10:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db29160 con 0x7fe0340035f0 2023-11-08T16:37:59.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:44 10.e 10:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2b010 con 0x7fe04406b040 2023-11-08T16:37:59.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:45 10.15 10:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2b810 con 0x7fe04408b040 2023-11-08T16:37:59.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:46 10.15 10:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2c170 con 0x7fe04408b040 2023-11-08T16:37:59.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:47 10.d 10:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2c9d0 con 0x564f6da663f0 2023-11-08T16:37:59.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.154+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:48 10.1d 10:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2d4c0 con 0x7fe04406b040 2023-11-08T16:37:59.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.159+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(38 1.00000000.inode [delete] v85'22 uv22 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe040056b10 2023-11-08T16:37:59.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.159+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:49 10.13 10:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2c170 con 0x7fe040056b10 2023-11-08T16:37:59.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.159+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(39 200.00000000 [delete] v85'23 uv23 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe040056b10 2023-11-08T16:37:59.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.159+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:50 10.13 10:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2c170 con 0x7fe040056b10 2023-11-08T16:37:59.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.160+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(45 604.00000000 [delete] v85'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04408b040 2023-11-08T16:37:59.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.160+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(40 200.00000001 [delete] v85'24 uv24 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe040056b10 2023-11-08T16:37:59.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.160+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:51 10.b 10:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2c170 con 0x564f6da663f0 2023-11-08T16:37:59.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.160+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:52 10.7 10:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2d4c0 con 0x564f6da663f0 2023-11-08T16:37:59.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.160+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v85'6 uv6 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7fe044065020 con 0x7fe0340035f0 2023-11-08T16:37:59.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.160+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:53 10.f 10:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2d820 con 0x564f6da663f0 2023-11-08T16:37:59.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.161+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(41 606.00000000 [delete] v85'25 uv25 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe040056b10 2023-11-08T16:37:59.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.161+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:54 10.1f 10:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2d860 con 0x7fe03c0208b0 2023-11-08T16:37:59.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.163+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 11 ==== osd_op_reply(42 607.00000000 [delete] v85'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.163+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:55 10.1f 10:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e85) v8 -- 0x564f6db2dc10 con 0x7fe03c0208b0 2023-11-08T16:37:59.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.163+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(46 500.00000000 [delete] v85'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04408b040 2023-11-08T16:37:59.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.165+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(48 605.00000000 [delete] v85'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.165+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 12 ==== osd_op_reply(47 mds_snaptable [delete] v85'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.166+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 13 ==== osd_op_reply(51 mds0_sessionmap [delete] v85'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.166+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 14 ==== osd_op_reply(52 609.00000000 [delete] v85'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.166+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(33 601.00000000 [delete] v85'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.166+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 <== osd.6 v2:172.21.15.93:6816/607960134 15 ==== osd_op_reply(53 400.00000000 [delete] v85'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe044065020 con 0x564f6da663f0 2023-11-08T16:37:59.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.166+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(44 608.00000000 [delete] v85'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.167+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(34 602.00000000 [delete] v85'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.167+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(35 10000000000.00000000 [delete] v85'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.168+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(54 100.00000000.inode [delete] v85'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7fe040086040 con 0x7fe03c0208b0 2023-11-08T16:37:59.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.168+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(49 mds0_inotable [delete] v85'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe040056b10 2023-11-08T16:37:59.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.168+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(50 100.00000000 [delete] v85'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe040056b10 2023-11-08T16:37:59.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.168+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(36 600.00000000 [delete] v85'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe0497e1640 1 -- 172.21.15.81:0/2042972598 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(55 1.00000000 [delete] v85'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7fe040086040 con 0x7fe03c0208b0 2023-11-08T16:37:59.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe049fe2640 1 -- 172.21.15.81:0/2042972598 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(37 603.00000000 [delete] v85'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7fe03c01b040 con 0x7fe04406b040 2023-11-08T16:37:59.173 INFO:teuthology.orchestra.run.smithi081.stdout:Removed 23 objects 2023-11-08T16:37:59.173 INFO:teuthology.orchestra.run.smithi081.stdout:successfully purged pool cephfs_metadata 2023-11-08T16:37:59.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fe03c015be0 msgr2=0x7fe03c021920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7fe03c015be0 0x7fe03c021920 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fe040056b10 msgr2=0x7fe040087400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7fe040056b10 0x7fe040087400 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7fe04001f060 msgr2=0x7fe040055810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7fe04001f060 0x7fe040055810 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7fe04408b040 msgr2=0x7fe04408b930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7fe04408b040 0x7fe04408b930 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7fe04406b040 msgr2=0x7fe04406c430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7fe04406b040 0x7fe04406c430 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 reap_dead start 2023-11-08T16:37:59.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7fe0340035f0 msgr2=0x7fe04407b420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7fe0340035f0 0x7fe04407b420 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x564f6da663f0 msgr2=0x564f6db26420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x564f6da663f0 0x564f6db26420 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fe03c0208b0 msgr2=0x7fe03c02e340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fe03c0208b0 0x7fe03c02e340 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe02c069850 msgr2=0x7fe02c06bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe02c069850 0x7fe02c06bd10 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe03c005fd0 tx=0x7fe03c007040 comp rx=0 tx=0).stop 2023-11-08T16:37:59.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6d9def10 msgr2=0x564f6d9f0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04cc3af40 1 --2- 172.21.15.81:0/2042972598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564f6d9def10 0x564f6d9f0640 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fe040005fd0 tx=0x7fe0400074b0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.169+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 reap_dead start 2023-11-08T16:37:59.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.170+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 reap_dead start 2023-11-08T16:37:59.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.170+0000 7fe04a7e3640 1 -- 172.21.15.81:0/2042972598 reap_dead start 2023-11-08T16:37:59.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.170+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 shutdown_connections 2023-11-08T16:37:59.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.170+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 >> 172.21.15.81:0/2042972598 conn(0x564f6d9f5c60 msgr2=0x564f6d9a8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:59.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.170+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 shutdown_connections 2023-11-08T16:37:59.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.170+0000 7fe04cc3af40 1 -- 172.21.15.81:0/2042972598 wait complete. 2023-11-08T16:37:59.180 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-11-08T16:37:59.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.384+0000 7f73ddc52640 1 Processor -- start 2023-11-08T16:37:59.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.384+0000 7f73ddc52640 1 -- start start 2023-11-08T16:37:59.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.386+0000 7f73ddc52640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f73d815ad60 0x7f73d806f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.386+0000 7f73ddc52640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f73d815eef0 0x7f73d815f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.386+0000 7f73ddc52640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73d806cf30 con 0x7f73d80ad8a0 2023-11-08T16:37:59.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.386+0000 7f73ddc52640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d80c8d80 con 0x7f73d815eef0 2023-11-08T16:37:59.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.386+0000 7f73ddc52640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73d815ea60 con 0x7f73d815ad60 2023-11-08T16:37:59.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.386+0000 7f73dd451640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f73d815eef0 0x7f73d815f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.387+0000 7f73dd451640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f73d815eef0 0x7f73d815f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41556/0 (socket says 172.21.15.81:41556) 2023-11-08T16:37:59.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.387+0000 7f73dd451640 1 -- 172.21.15.81:0/4023827368 learned_addr learned my addr 172.21.15.81:0/4023827368 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:59.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.387+0000 7f73dcc50640 1 --2- 172.21.15.81:0/4023827368 >> v2:172.21.15.93:3300/0 conn(0x7f73d815ad60 0x7f73d806f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.387+0000 7f73dd451640 1 -- 172.21.15.81:0/4023827368 >> v2:172.21.15.93:3300/0 conn(0x7f73d815ad60 msgr2=0x7f73d806f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:59.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.387+0000 7f73dd451640 1 --2- 172.21.15.81:0/4023827368 >> v2:172.21.15.93:3300/0 conn(0x7f73d815ad60 0x7f73d806f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.387+0000 7f73dd451640 1 -- 172.21.15.81:0/4023827368 >> v1:172.21.15.93:6789/0 conn(0x7f73d80ad8a0 legacy=0x7f73d80cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:37:59.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.387+0000 7f73dd451640 1 -- 172.21.15.81:0/4023827368 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d80ae260 con 0x7f73d815eef0 2023-11-08T16:37:59.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.388+0000 7f73dd451640 1 --2- 172.21.15.81:0/4023827368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d815eef0 0x7f73d815f380 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f73d8071b70 tx=0x7f73d002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0123ea4bb72bce server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.388+0000 7f73d77fe640 1 -- 172.21.15.81:0/4023827368 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d002c9f0 con 0x7f73d815eef0 2023-11-08T16:37:59.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.388+0000 7f73d77fe640 1 -- 172.21.15.81:0/4023827368 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d002cb50 con 0x7f73d815eef0 2023-11-08T16:37:59.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.388+0000 7f73d77fe640 1 -- 172.21.15.81:0/4023827368 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d00343f0 con 0x7f73d815eef0 2023-11-08T16:37:59.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.389+0000 7f73ddc52640 1 -- 172.21.15.81:0/4023827368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d815eef0 msgr2=0x7f73d815f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.389+0000 7f73ddc52640 1 --2- 172.21.15.81:0/4023827368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d815eef0 0x7f73d815f380 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f73d8071b70 tx=0x7f73d002be50 comp rx=0 tx=0).stop 2023-11-08T16:37:59.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.389+0000 7f73ddc52640 1 -- 172.21.15.81:0/4023827368 shutdown_connections 2023-11-08T16:37:59.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.389+0000 7f73ddc52640 1 --2- 172.21.15.81:0/4023827368 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d815eef0 0x7f73d815f380 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.389+0000 7f73ddc52640 1 --2- 172.21.15.81:0/4023827368 >> v2:172.21.15.93:3300/0 conn(0x7f73d815ad60 0x7f73d806f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.389+0000 7f73ddc52640 1 -- 172.21.15.81:0/4023827368 >> 172.21.15.81:0/4023827368 conn(0x7f73d806a6e0 msgr2=0x7f73d80b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:37:59.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.389+0000 7f73ddc52640 1 -- 172.21.15.81:0/4023827368 shutdown_connections 2023-11-08T16:37:59.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.389+0000 7f73ddc52640 1 -- 172.21.15.81:0/4023827368 wait complete. 2023-11-08T16:37:59.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.391+0000 7f73ddc52640 1 Processor -- start 2023-11-08T16:37:59.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.391+0000 7f73ddc52640 1 -- start start 2023-11-08T16:37:59.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73ddc52640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d80ad8a0 0x7f73d814b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73ddc52640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73d815ad60 0x7f73d80b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73ddc52640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f73d815eef0 0x7f73d8154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73ddc52640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d8157f50 con 0x7f73d80ad8a0 2023-11-08T16:37:59.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73ddc52640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d814fac0 con 0x7f73d815ad60 2023-11-08T16:37:59.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73ddc52640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d81581a0 con 0x7f73d815eef0 2023-11-08T16:37:59.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73dcc50640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d80ad8a0 0x7f73d814b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73dcc50640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d80ad8a0 0x7f73d814b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41566/0 (socket says 172.21.15.81:41566) 2023-11-08T16:37:59.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73dcc50640 1 -- 172.21.15.81:0/3410239619 learned_addr learned my addr 172.21.15.81:0/3410239619 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:37:59.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73dcc50640 1 -- 172.21.15.81:0/3410239619 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f73d815eef0 msgr2=0x7f73d8154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73dcc50640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f73d815eef0 0x7f73d8154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73dcc50640 1 -- 172.21.15.81:0/3410239619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73d815ad60 msgr2=0x7f73d80b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:37:59.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73dcc50640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73d815ad60 0x7f73d80b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:37:59.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.392+0000 7f73dcc50640 1 -- 172.21.15.81:0/3410239619 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d0009b30 con 0x7f73d80ad8a0 2023-11-08T16:37:59.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.393+0000 7f73dcc50640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d80ad8a0 0x7f73d814b8c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c7f0 tx=0x7f73cc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.393+0000 7f73d57fa640 1 -- 172.21.15.81:0/3410239619 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc004220 con 0x7f73d80ad8a0 2023-11-08T16:37:59.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.393+0000 7f73d57fa640 1 -- 172.21.15.81:0/3410239619 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73cc0043a0 con 0x7f73d80ad8a0 2023-11-08T16:37:59.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.393+0000 7f73d57fa640 1 -- 172.21.15.81:0/3410239619 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73cc007860 con 0x7f73d80ad8a0 2023-11-08T16:37:59.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.393+0000 7f73ddc52640 1 -- 172.21.15.81:0/3410239619 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d8158480 con 0x7f73d80ad8a0 2023-11-08T16:37:59.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.393+0000 7f73ddc52640 1 -- 172.21.15.81:0/3410239619 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d8158a40 con 0x7f73d80ad8a0 2023-11-08T16:37:59.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.394+0000 7f73ddc52640 1 -- 172.21.15.81:0/3410239619 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d80cc010 con 0x7f73d80ad8a0 2023-11-08T16:37:59.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.395+0000 7f73d57fa640 1 -- 172.21.15.81:0/3410239619 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f73cc007bd0 con 0x7f73d80ad8a0 2023-11-08T16:37:59.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.396+0000 7f73d57fa640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f73bc06ae70 0x7f73bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:37:59.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.396+0000 7f73d57fa640 1 -- 172.21.15.81:0/3410239619 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6257+0+0 (secure 0 0 0) 0x7f73cc089860 con 0x7f73d80ad8a0 2023-11-08T16:37:59.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.396+0000 7f73d7fff640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f73bc06ae70 0x7f73bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:37:59.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.397+0000 7f73d7fff640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f73bc06ae70 0x7f73bc06d330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f73d80b3890 tx=0x7f73c4007990 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:37:59.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.400+0000 7f73d57fa640 1 -- 172.21.15.81:0/3410239619 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f73cc054640 con 0x7f73d80ad8a0 2023-11-08T16:37:59.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:37:59.631+0000 7f73ddc52640 1 -- 172.21.15.81:0/3410239619 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f73d8067800 con 0x7f73d80ad8a0 2023-11-08T16:38:00.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.524+0000 7f73d57fa640 1 -- 172.21.15.81:0/3410239619 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v68) v1 ==== 107+0+0 (secure 0 0 0) 0x7f73cc057c60 con 0x7f73d80ad8a0 2023-11-08T16:38:00.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 -- 172.21.15.81:0/3410239619 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f73bc06ae70 msgr2=0x7f73bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:00.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f73bc06ae70 0x7f73bc06d330 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f73d80b3890 tx=0x7f73c4007990 comp rx=0 tx=0).stop 2023-11-08T16:38:00.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 -- 172.21.15.81:0/3410239619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d80ad8a0 msgr2=0x7f73d814b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:00.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d80ad8a0 0x7f73d814b8c0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00c7f0 tx=0x7f73cc00ccb0 comp rx=0 tx=0).stop 2023-11-08T16:38:00.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 -- 172.21.15.81:0/3410239619 shutdown_connections 2023-11-08T16:38:00.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f73bc06ae70 0x7f73bc06d330 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:00.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f73d815eef0 0x7f73d8154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:00.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73d815ad60 0x7f73d80b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:00.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 --2- 172.21.15.81:0/3410239619 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f73d80ad8a0 0x7f73d814b8c0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:00.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 -- 172.21.15.81:0/3410239619 >> 172.21.15.81:0/3410239619 conn(0x7f73d806a6e0 msgr2=0x7f73d8156450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:00.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 -- 172.21.15.81:0/3410239619 shutdown_connections 2023-11-08T16:38:00.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.528+0000 7f73ddc52640 1 -- 172.21.15.81:0/3410239619 wait complete. 2023-11-08T16:38:00.541 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:38:00.539+0000 7f46badb8640 -1 mds.0.inotable: load_2 could not read table: -2 2023-11-08T16:38:00.541 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:38:00.539+0000 7f46badb8640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-11-08T16:38:00.542 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:38:00.539+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:38:00.542 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:38:00.540+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-08T16:38:00.553 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-08T16:38:00.554 INFO:tasks.ceph.mds.a.smithi081.stderr: -26> 2023-11-08T16:38:00.539+0000 7f46badb8640 -1 mds.0.inotable: load_2 could not read table: -2 2023-11-08T16:38:00.554 INFO:tasks.ceph.mds.a.smithi081.stderr: -24> 2023-11-08T16:38:00.539+0000 7f46badb8640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-11-08T16:38:00.599 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:38:00.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.742+0000 7ff0a5007640 1 Processor -- start 2023-11-08T16:38:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.743+0000 7ff0a5007640 1 -- start start 2023-11-08T16:38:00.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.744+0000 7ff0a5007640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a015a0e0 0x7ff0a00ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:00.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.744+0000 7ff0a5007640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a00ce960 con 0x7ff0a00c9bf0 2023-11-08T16:38:00.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.744+0000 7ff0a5007640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0a00c9480 con 0x7ff0a00ca720 2023-11-08T16:38:00.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.744+0000 7ff0a5007640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0071980 con 0x7ff0a015a0e0 2023-11-08T16:38:00.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.744+0000 7ff09e59c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff0a00c9bf0 0x7ff0a00ca010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:51696/0 (socket says 172.21.15.81:51696) 2023-11-08T16:38:00.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.744+0000 7ff09e59c640 1 -- 172.21.15.81:0/4251491557 learned_addr learned my addr 172.21.15.81:0/4251491557 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:00.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.745+0000 7ff09ed9d640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a015a0e0 0x7ff0a00ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:00.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.745+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258354169 0 0) 0x7ff088001110 con 0x7ff0a00c9bf0 2023-11-08T16:38:00.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.745+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff08c003520 con 0x7ff0a00c9bf0 2023-11-08T16:38:00.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.745+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 740808044 0 0) 0x7ff08c003520 con 0x7ff0a00c9bf0 2023-11-08T16:38:00.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.745+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a015a0e0 msgr2=0x7ff0a00ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:00.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.745+0000 7ff09dd9b640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a015a0e0 0x7ff0a00ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:00.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.745+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 >> v1:172.21.15.93:6789/0 conn(0x7ff0a00ca720 legacy=0x7ff0a006b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:00.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.745+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a015a4c0 con 0x7ff0a00c9bf0 2023-11-08T16:38:00.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.745+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ff088001800 con 0x7ff0a00c9bf0 2023-11-08T16:38:00.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.745+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 >> v1:172.21.15.81:6789/0 conn(0x7ff0a00c9bf0 legacy=0x7ff0a00ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:00.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.745+0000 7ff09dd9b640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 0x7ff08c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:00.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09dd9b640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff08c0047c0 0x7ff08c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:00.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09dd9b640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a015a0e0 0x7ff08c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:00.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0071980 con 0x7ff08c003a10 2023-11-08T16:38:00.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c00bb10 con 0x7ff08c0047c0 2023-11-08T16:38:00.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff08c00dee0 con 0x7ff0a015a0e0 2023-11-08T16:38:00.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09ed9d640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 0x7ff08c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:00.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09e59c640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff08c0047c0 0x7ff08c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:00.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09ed9d640 1 -- 172.21.15.81:0/4251491557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a015a0e0 msgr2=0x7ff08c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:00.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09ed9d640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a015a0e0 0x7ff08c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:00.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09ed9d640 1 -- 172.21.15.81:0/4251491557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff08c0047c0 msgr2=0x7ff08c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:00.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09ed9d640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff08c0047c0 0x7ff08c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:00.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09ed9d640 1 -- 172.21.15.81:0/4251491557 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08c003520 con 0x7ff08c003a10 2023-11-08T16:38:00.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.746+0000 7ff09e59c640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff08c0047c0 0x7ff08c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:00.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.747+0000 7ff09ed9d640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 0x7ff08c003e30 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff090009fc0 tx=0x7ff08c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd82f39b4a96b9e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:00.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.748+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09002cc60 con 0x7ff08c003a10 2023-11-08T16:38:00.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.748+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09000f040 con 0x7ff08c003a10 2023-11-08T16:38:00.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.748+0000 7ff09dd9b640 1 -- 172.21.15.81:0/4251491557 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090034440 con 0x7ff08c003a10 2023-11-08T16:38:00.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.748+0000 7ff0a5007640 1 -- 172.21.15.81:0/4251491557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 msgr2=0x7ff08c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:00.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.748+0000 7ff0a5007640 1 --2- 172.21.15.81:0/4251491557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 0x7ff08c003e30 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7ff090009fc0 tx=0x7ff08c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:38:00.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.748+0000 7ff09f59e640 1 -- 172.21.15.81:0/4251491557 reap_dead start 2023-11-08T16:38:00.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.748+0000 7ff0a5007640 1 -- 172.21.15.81:0/4251491557 shutdown_connections 2023-11-08T16:38:00.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.748+0000 7ff0a5007640 1 -- 172.21.15.81:0/4251491557 >> 172.21.15.81:0/4251491557 conn(0x7ff0a00cb580 msgr2=0x7ff0a00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:00.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.749+0000 7ff0a5007640 1 -- 172.21.15.81:0/4251491557 shutdown_connections 2023-11-08T16:38:00.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.749+0000 7ff0a5007640 1 -- 172.21.15.81:0/4251491557 wait complete. 2023-11-08T16:38:00.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.750+0000 7ff0a5007640 1 Processor -- start 2023-11-08T16:38:00.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.751+0000 7ff0a5007640 1 -- start start 2023-11-08T16:38:00.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.751+0000 7ff0a5007640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 0x7ff0a00c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:00.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.751+0000 7ff0a5007640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a01459e0 0x7ff0a015c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:00.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff0a5007640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a0145e40 0x7ff0a015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:00.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff0a5007640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a01f2ce0 con 0x7ff08c003a10 2023-11-08T16:38:00.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff0a5007640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a0159350 con 0x7ff0a0145e40 2023-11-08T16:38:00.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff0a5007640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0a00afc60 con 0x7ff0a01459e0 2023-11-08T16:38:00.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff09ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 0x7ff0a00c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:00.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff09ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 0x7ff0a00c4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:41612/0 (socket says 172.21.15.81:41612) 2023-11-08T16:38:00.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff09ed9d640 1 -- 172.21.15.81:0/227096959 learned_addr learned my addr 172.21.15.81:0/227096959 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:00.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff09e59c640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a01459e0 0x7ff0a015c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:00.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff09ed9d640 1 -- 172.21.15.81:0/227096959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a01459e0 msgr2=0x7ff0a015c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:00.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff09ed9d640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a01459e0 0x7ff0a015c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:00.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff09f59e640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a0145e40 0x7ff0a015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:00.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff09ed9d640 1 -- 172.21.15.81:0/227096959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a0145e40 msgr2=0x7ff0a015cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:00.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff09ed9d640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a0145e40 0x7ff0a015cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:00.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.752+0000 7ff09ed9d640 1 -- 172.21.15.81:0/227096959 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a00ac9e0 con 0x7ff08c003a10 2023-11-08T16:38:00.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.753+0000 7ff09ed9d640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 0x7ff0a00c4f40 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff09000e230 tx=0x7ff090009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:00.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.753+0000 7ff0977fe640 1 -- 172.21.15.81:0/227096959 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090034bd0 con 0x7ff08c003a10 2023-11-08T16:38:00.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.753+0000 7ff0977fe640 1 -- 172.21.15.81:0/227096959 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff090034d30 con 0x7ff08c003a10 2023-11-08T16:38:00.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.753+0000 7ff0a5007640 1 -- 172.21.15.81:0/227096959 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0a00accc0 con 0x7ff08c003a10 2023-11-08T16:38:00.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.753+0000 7ff0977fe640 1 -- 172.21.15.81:0/227096959 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff090034ea0 con 0x7ff08c003a10 2023-11-08T16:38:00.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.753+0000 7ff0a5007640 1 -- 172.21.15.81:0/227096959 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a00acfa0 con 0x7ff08c003a10 2023-11-08T16:38:00.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.754+0000 7ff0a5007640 1 -- 172.21.15.81:0/227096959 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0a004ece0 con 0x7ff08c003a10 2023-11-08T16:38:00.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.755+0000 7ff0977fe640 1 -- 172.21.15.81:0/227096959 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff090040450 con 0x7ff08c003a10 2023-11-08T16:38:00.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.756+0000 7ff0977fe640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff08006ae70 0x7ff08006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:00.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.756+0000 7ff0977fe640 1 -- 172.21.15.81:0/227096959 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x7ff0900abf20 con 0x7ff08c003a10 2023-11-08T16:38:00.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.756+0000 7ff09e59c640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff08006ae70 0x7ff08006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:00.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.757+0000 7ff09e59c640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff08006ae70 0x7ff08006d330 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff0a015d3f0 tx=0x7ff088001110 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:00.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.760+0000 7ff0977fe640 1 -- 172.21.15.81:0/227096959 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff090076f00 con 0x7ff08c003a10 2023-11-08T16:38:00.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:00.978+0000 7ff0a5007640 1 -- 172.21.15.81:0/227096959 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7ff0a00cc3b0 con 0x7ff08c003a10 2023-11-08T16:38:01.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.557+0000 7ff0977fe640 1 -- 172.21.15.81:0/227096959 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v71) v1 ==== 162+0+0 (secure 0 0 0) 0x7ff09007a520 con 0x7ff08c003a10 2023-11-08T16:38:01.560 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-08T16:38:01.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 -- 172.21.15.81:0/227096959 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff08006ae70 msgr2=0x7ff08006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:01.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff08006ae70 0x7ff08006d330 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ff0a015d3f0 tx=0x7ff088001110 comp rx=0 tx=0).stop 2023-11-08T16:38:01.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 -- 172.21.15.81:0/227096959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 msgr2=0x7ff0a00c4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:01.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 0x7ff0a00c4f40 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff09000e230 tx=0x7ff090009750 comp rx=0 tx=0).stop 2023-11-08T16:38:01.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 -- 172.21.15.81:0/227096959 shutdown_connections 2023-11-08T16:38:01.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff08006ae70 0x7ff08006d330 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:01.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0a0145e40 0x7ff0a015cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:01.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0a01459e0 0x7ff0a015c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:01.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 --2- 172.21.15.81:0/227096959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff08c003a10 0x7ff0a00c4f40 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:01.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 -- 172.21.15.81:0/227096959 >> 172.21.15.81:0/227096959 conn(0x7ff0a00cb580 msgr2=0x7ff0a006b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:01.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 -- 172.21.15.81:0/227096959 shutdown_connections 2023-11-08T16:38:01.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.561+0000 7ff0a5007640 1 -- 172.21.15.81:0/227096959 wait complete. 2023-11-08T16:38:01.584 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:01.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.836+0000 7f89dbfff640 1 Processor -- start 2023-11-08T16:38:01.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.836+0000 7f89dbfff640 1 -- start start 2023-11-08T16:38:01.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.838+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89dc0ced10 0x7f89dc0cf150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:01.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.838+0000 7f89dbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f89dc06b540 0x7f89dc06b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:01.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.838+0000 7f89dbfff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89dc0cf690 con 0x7f89dc150b80 2023-11-08T16:38:01.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.838+0000 7f89dbfff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc15b0a0 con 0x7f89dc06b540 2023-11-08T16:38:01.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.838+0000 7f89dbfff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0b2fe0 con 0x7f89dc0ced10 2023-11-08T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.839+0000 7f89db7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f89dc06b540 0x7f89dc06b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.839+0000 7f89db7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f89dc06b540 0x7f89dc06b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59244/0 (socket says 172.21.15.81:59244) 2023-11-08T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.839+0000 7f89db7fe640 1 -- 172.21.15.81:0/2340765165 learned_addr learned my addr 172.21.15.81:0/2340765165 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.839+0000 7f89d3fff640 1 --2- 172.21.15.81:0/2340765165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89dc0ced10 0x7f89dc0cf150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:01.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.839+0000 7f89db7fe640 1 -- 172.21.15.81:0/2340765165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89dc0ced10 msgr2=0x7f89dc0cf150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:01.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.839+0000 7f89db7fe640 1 --2- 172.21.15.81:0/2340765165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89dc0ced10 0x7f89dc0cf150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:01.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.839+0000 7f89db7fe640 1 -- 172.21.15.81:0/2340765165 >> v1:172.21.15.93:6789/0 conn(0x7f89dc150b80 legacy=0x7f89dc15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:01.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.839+0000 7f89db7fe640 1 -- 172.21.15.81:0/2340765165 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc0b32c0 con 0x7f89dc06b540 2023-11-08T16:38:01.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.840+0000 7f89db7fe640 1 --2- 172.21.15.81:0/2340765165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89dc06b540 0x7f89dc06b9d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f89d4009e50 tx=0x7f89d402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e59ae052a1051a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:01.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.841+0000 7f89da7fc640 1 -- 172.21.15.81:0/2340765165 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d402ca70 con 0x7f89dc06b540 2023-11-08T16:38:01.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.841+0000 7f89da7fc640 1 -- 172.21.15.81:0/2340765165 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d402cbd0 con 0x7f89dc06b540 2023-11-08T16:38:01.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.841+0000 7f89da7fc640 1 -- 172.21.15.81:0/2340765165 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d40343f0 con 0x7f89dc06b540 2023-11-08T16:38:01.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.841+0000 7f89dbfff640 1 -- 172.21.15.81:0/2340765165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89dc06b540 msgr2=0x7f89dc06b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.841+0000 7f89dbfff640 1 --2- 172.21.15.81:0/2340765165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89dc06b540 0x7f89dc06b9d0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f89d4009e50 tx=0x7f89d402bed0 comp rx=0 tx=0).stop 2023-11-08T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.841+0000 7f89dbfff640 1 -- 172.21.15.81:0/2340765165 shutdown_connections 2023-11-08T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.841+0000 7f89dbfff640 1 --2- 172.21.15.81:0/2340765165 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89dc06b540 0x7f89dc06b9d0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.841+0000 7f89dbfff640 1 --2- 172.21.15.81:0/2340765165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89dc0ced10 0x7f89dc0cf150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:01.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.841+0000 7f89dbfff640 1 -- 172.21.15.81:0/2340765165 >> 172.21.15.81:0/2340765165 conn(0x7f89dc06a4a0 msgr2=0x7f89dc06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.841+0000 7f89dbfff640 1 -- 172.21.15.81:0/2340765165 shutdown_connections 2023-11-08T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.841+0000 7f89dbfff640 1 -- 172.21.15.81:0/2340765165 wait complete. 2023-11-08T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.843+0000 7f89dbfff640 1 Processor -- start 2023-11-08T16:38:01.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.843+0000 7f89dbfff640 1 -- start start 2023-11-08T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.844+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89dc06b540 0x7f89dc13c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.844+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89dc0ced10 0x7f89dc0cd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.844+0000 7f89dbfff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89dc150b80 0x7f89dc0af550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.844+0000 7f89dbfff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0cbd80 con 0x7f89dc06b540 2023-11-08T16:38:01.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.844+0000 7f89dbfff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0ccbe0 con 0x7f89dc150b80 2023-11-08T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.844+0000 7f89dbfff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0cbfd0 con 0x7f89dc0ced10 2023-11-08T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.844+0000 7f89daffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89dc06b540 0x7f89dc13c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.844+0000 7f89daffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89dc06b540 0x7f89dc13c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59246/0 (socket says 172.21.15.81:59246) 2023-11-08T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.844+0000 7f89daffd640 1 -- 172.21.15.81:0/2515829648 learned_addr learned my addr 172.21.15.81:0/2515829648 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:01.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.844+0000 7f89db7fe640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89dc150b80 0x7f89dc0af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89daffd640 1 -- 172.21.15.81:0/2515829648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89dc0ced10 msgr2=0x7f89dc0cd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89daffd640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89dc0ced10 0x7f89dc0cd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89daffd640 1 -- 172.21.15.81:0/2515829648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89dc150b80 msgr2=0x7f89dc0af550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89daffd640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89dc150b80 0x7f89dc0af550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:01.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89daffd640 1 -- 172.21.15.81:0/2515829648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d4009b30 con 0x7f89dc06b540 2023-11-08T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89db7fe640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89dc150b80 0x7f89dc0af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89daffd640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89dc06b540 0x7f89dc13c290 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00c440 tx=0x7f89cc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89d8ff9640 1 -- 172.21.15.81:0/2515829648 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc005480 con 0x7f89dc06b540 2023-11-08T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89d8ff9640 1 -- 172.21.15.81:0/2515829648 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89cc0035f0 con 0x7f89dc06b540 2023-11-08T16:38:01.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89dbfff640 1 -- 172.21.15.81:0/2515829648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89dc0cc2b0 con 0x7f89dc06b540 2023-11-08T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89d8ff9640 1 -- 172.21.15.81:0/2515829648 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89cc0039c0 con 0x7f89dc06b540 2023-11-08T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.845+0000 7f89dbfff640 1 -- 172.21.15.81:0/2515829648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89dc0cc870 con 0x7f89dc06b540 2023-11-08T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.846+0000 7f89dbfff640 1 -- 172.21.15.81:0/2515829648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89dc067800 con 0x7f89dc06b540 2023-11-08T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.847+0000 7f89d8ff9640 1 -- 172.21.15.81:0/2515829648 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f89cc003b20 con 0x7f89dc06b540 2023-11-08T16:38:01.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.848+0000 7f89d8ff9640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f89bc06ae70 0x7f89bc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:01.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.848+0000 7f89d8ff9640 1 -- 172.21.15.81:0/2515829648 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x7f89cc088e90 con 0x7f89dc06b540 2023-11-08T16:38:01.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.848+0000 7f89d3fff640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f89bc06ae70 0x7f89bc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:01.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.849+0000 7f89d3fff640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f89bc06ae70 0x7f89bc06d330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0ca9b0 tx=0x7f89c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:01.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:01.852+0000 7f89d8ff9640 1 -- 172.21.15.81:0/2515829648 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f89cc053c20 con 0x7f89dc06b540 2023-11-08T16:38:02.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.082+0000 7f89dbfff640 1 -- 172.21.15.81:0/2515829648 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89dc06a980 con 0x7f89dc06b540 2023-11-08T16:38:02.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.084+0000 7f89d8ff9640 1 -- 172.21.15.81:0/2515829648 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f89cc08a090 con 0x7f89dc06b540 2023-11-08T16:38:02.087 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:02.087 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/2125403666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2125403666},{"type":"v1","addr":"172.21.15.93:6834","nonce":2125403666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2727356524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2727356524},{"type":"v1","addr":"172.21.15.93:6835","nonce":2727356524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6936,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2639654743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2639654743},{"type":"v1","addr":"172.21.15.81:6835","nonce":2639654743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1764545681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1764545681},{"type":"v1","addr":"172.21.15.81:6837","nonce":1764545681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:00.980798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:02.087 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 71 2023-11-08T16:38:02.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.086+0000 7f89dbfff640 1 -- 172.21.15.81:0/2515829648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f89bc06ae70 msgr2=0x7f89bc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.086+0000 7f89dbfff640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f89bc06ae70 0x7f89bc06d330 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f89dc0ca9b0 tx=0x7f89c4008040 comp rx=0 tx=0).stop 2023-11-08T16:38:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.086+0000 7f89dbfff640 1 -- 172.21.15.81:0/2515829648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89dc06b540 msgr2=0x7f89dc13c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.086+0000 7f89dbfff640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89dc06b540 0x7f89dc13c290 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f89cc00c440 tx=0x7f89cc00c900 comp rx=0 tx=0).stop 2023-11-08T16:38:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.086+0000 7f89dbfff640 1 -- 172.21.15.81:0/2515829648 shutdown_connections 2023-11-08T16:38:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.086+0000 7f89dbfff640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f89bc06ae70 0x7f89bc06d330 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.087+0000 7f89dbfff640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89dc150b80 0x7f89dc0af550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.087+0000 7f89dbfff640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89dc0ced10 0x7f89dc0cd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.087+0000 7f89dbfff640 1 --2- 172.21.15.81:0/2515829648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f89dc06b540 0x7f89dc13c290 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.087+0000 7f89dbfff640 1 -- 172.21.15.81:0/2515829648 >> 172.21.15.81:0/2515829648 conn(0x7f89dc06a4a0 msgr2=0x7f89dc0ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:02.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.087+0000 7f89dbfff640 1 -- 172.21.15.81:0/2515829648 shutdown_connections 2023-11-08T16:38:02.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.087+0000 7f89dbfff640 1 -- 172.21.15.81:0/2515829648 wait complete. 2023-11-08T16:38:02.108 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T16:38:02.108 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:02.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.310+0000 7f8fc7b00640 1 Processor -- start 2023-11-08T16:38:02.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.310+0000 7f8fc7b00640 1 -- start start 2023-11-08T16:38:02.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.312+0000 7f8fc7b00640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8fc015d1a0 0x7f8fc00d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.312+0000 7f8fc7b00640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8fc00d13e0 0x7f8fc00d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.312+0000 7f8fc7b00640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc00d1d80 con 0x7f8fc00cf430 2023-11-08T16:38:02.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.312+0000 7f8fc7b00640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc00b2450 con 0x7f8fc015d1a0 2023-11-08T16:38:02.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.312+0000 7f8fc7b00640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc00ce800 con 0x7f8fc00d13e0 2023-11-08T16:38:02.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.313+0000 7f8fc589c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8fc015d1a0 0x7f8fc00d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.313+0000 7f8fc509b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8fc00d13e0 0x7f8fc00d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.313+0000 7f8fc589c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8fc015d1a0 0x7f8fc00d3920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:59336/0 (socket says 172.21.15.81:59336) 2023-11-08T16:38:02.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.313+0000 7f8fc589c640 1 -- 172.21.15.81:0/1191472977 learned_addr learned my addr 172.21.15.81:0/1191472977 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:02.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.313+0000 7f8fc509b640 1 -- 172.21.15.81:0/1191472977 >> v2:172.21.15.93:3300/0 conn(0x7f8fc015d1a0 msgr2=0x7f8fc00d3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:02.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.313+0000 7f8fc509b640 1 --2- 172.21.15.81:0/1191472977 >> v2:172.21.15.93:3300/0 conn(0x7f8fc015d1a0 0x7f8fc00d3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.313+0000 7f8fc509b640 1 -- 172.21.15.81:0/1191472977 >> v1:172.21.15.93:6789/0 conn(0x7f8fc00cf430 legacy=0x7f8fc00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.313+0000 7f8fc509b640 1 -- 172.21.15.81:0/1191472977 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc00ceae0 con 0x7f8fc00d13e0 2023-11-08T16:38:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.315+0000 7f8fc509b640 1 --2- 172.21.15.81:0/1191472977 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8fc00d13e0 0x7f8fc00d1840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00097a0 tx=0x7f8fb002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6892514b19e69430 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.316+0000 7f8fc489a640 1 -- 172.21.15.81:0/1191472977 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb002d040 con 0x7f8fc00d13e0 2023-11-08T16:38:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.316+0000 7f8fc489a640 1 -- 172.21.15.81:0/1191472977 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0031810 con 0x7f8fc00d13e0 2023-11-08T16:38:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.316+0000 7f8fc489a640 1 -- 172.21.15.81:0/1191472977 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0031ba0 con 0x7f8fc00d13e0 2023-11-08T16:38:02.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.316+0000 7f8fc7b00640 1 -- 172.21.15.81:0/1191472977 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8fc00d13e0 msgr2=0x7f8fc00d1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.316+0000 7f8fc7b00640 1 --2- 172.21.15.81:0/1191472977 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8fc00d13e0 0x7f8fc00d1840 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f8fb00097a0 tx=0x7f8fb002b8b0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.316+0000 7f8fc7b00640 1 -- 172.21.15.81:0/1191472977 shutdown_connections 2023-11-08T16:38:02.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.316+0000 7f8fc7b00640 1 --2- 172.21.15.81:0/1191472977 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8fc00d13e0 0x7f8fc00d1840 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.316+0000 7f8fc7b00640 1 --2- 172.21.15.81:0/1191472977 >> v2:172.21.15.93:3300/0 conn(0x7f8fc015d1a0 0x7f8fc00d3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.316+0000 7f8fc7b00640 1 -- 172.21.15.81:0/1191472977 >> 172.21.15.81:0/1191472977 conn(0x7f8fc00d2e00 msgr2=0x7f8fc00d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.316+0000 7f8fc7b00640 1 -- 172.21.15.81:0/1191472977 shutdown_connections 2023-11-08T16:38:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.317+0000 7f8fc7b00640 1 -- 172.21.15.81:0/1191472977 wait complete. 2023-11-08T16:38:02.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.318+0000 7f8fc7b00640 1 Processor -- start 2023-11-08T16:38:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.318+0000 7f8fc7b00640 1 -- start start 2023-11-08T16:38:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.319+0000 7f8fc7b00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc00cf430 0x7f8fc01f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.319+0000 7f8fc7b00640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8fc00d13e0 0x7f8fc01f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.319+0000 7f8fc7b00640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fc015d1a0 0x7f8fc00cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.319+0000 7f8fc7b00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc009fbc0 con 0x7f8fc00cf430 2023-11-08T16:38:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.319+0000 7f8fc7b00640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc009ed70 con 0x7f8fc015d1a0 2023-11-08T16:38:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.319+0000 7f8fc7b00640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc009ea90 con 0x7f8fc00d13e0 2023-11-08T16:38:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.320+0000 7f8fc589c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc00cf430 0x7f8fc01f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.320+0000 7f8fc589c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc00cf430 0x7f8fc01f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59258/0 (socket says 172.21.15.81:59258) 2023-11-08T16:38:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.320+0000 7f8fc589c640 1 -- 172.21.15.81:0/3885316166 learned_addr learned my addr 172.21.15.81:0/3885316166 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:02.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.320+0000 7f8fc509b640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8fc00d13e0 0x7f8fc01f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.320+0000 7f8fc609d640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fc015d1a0 0x7f8fc00cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.320+0000 7f8fc589c640 1 -- 172.21.15.81:0/3885316166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8fc00d13e0 msgr2=0x7f8fc01f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.320+0000 7f8fc589c640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8fc00d13e0 0x7f8fc01f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.320+0000 7f8fc589c640 1 -- 172.21.15.81:0/3885316166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fc015d1a0 msgr2=0x7f8fc00cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.320+0000 7f8fc589c640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fc015d1a0 0x7f8fc00cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.320+0000 7f8fc589c640 1 -- 172.21.15.81:0/3885316166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb0009400 con 0x7f8fc00cf430 2023-11-08T16:38:02.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.320+0000 7f8fc609d640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fc015d1a0 0x7f8fc00cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:02.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.321+0000 7f8fc589c640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc00cf430 0x7f8fc01f5c00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00e7d0 tx=0x7f8fbc00ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:02.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.321+0000 7f8fb67fc640 1 -- 172.21.15.81:0/3885316166 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc009560 con 0x7f8fc00cf430 2023-11-08T16:38:02.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.321+0000 7f8fb67fc640 1 -- 172.21.15.81:0/3885316166 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc00c940 con 0x7f8fc00cf430 2023-11-08T16:38:02.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.321+0000 7f8fc7b00640 1 -- 172.21.15.81:0/3885316166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc009f500 con 0x7f8fc00cf430 2023-11-08T16:38:02.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.321+0000 7f8fb67fc640 1 -- 172.21.15.81:0/3885316166 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc00eee0 con 0x7f8fc00cf430 2023-11-08T16:38:02.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.321+0000 7f8fc7b00640 1 -- 172.21.15.81:0/3885316166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc00cc420 con 0x7f8fc00cf430 2023-11-08T16:38:02.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.322+0000 7f8fc7b00640 1 -- 172.21.15.81:0/3885316166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc004ece0 con 0x7f8fc00cf430 2023-11-08T16:38:02.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.323+0000 7f8fb67fc640 1 -- 172.21.15.81:0/3885316166 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8fbc00ccf0 con 0x7f8fc00cf430 2023-11-08T16:38:02.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.324+0000 7f8fb67fc640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8fa406ae70 0x7f8fa406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.324+0000 7f8fb67fc640 1 -- 172.21.15.81:0/3885316166 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x7f8fbc089cd0 con 0x7f8fc00cf430 2023-11-08T16:38:02.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.324+0000 7f8fc509b640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8fa406ae70 0x7f8fa406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.325+0000 7f8fc509b640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8fa406ae70 0x7f8fa406d330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0036000 tx=0x7f8fb003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:02.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.329+0000 7f8fb67fc640 1 -- 172.21.15.81:0/3885316166 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8fbc054af0 con 0x7f8fc00cf430 2023-11-08T16:38:02.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.563+0000 7f8fc7b00640 1 -- 172.21.15.81:0/3885316166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fc00b15a0 con 0x7f8fc00cf430 2023-11-08T16:38:02.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.564+0000 7f8fb67fc640 1 -- 172.21.15.81:0/3885316166 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f8fbc021090 con 0x7f8fc00cf430 2023-11-08T16:38:02.567 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:02.567 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/2125403666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2125403666},{"type":"v1","addr":"172.21.15.93:6834","nonce":2125403666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2727356524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2727356524},{"type":"v1","addr":"172.21.15.93:6835","nonce":2727356524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6936,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2639654743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2639654743},{"type":"v1","addr":"172.21.15.81:6835","nonce":2639654743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1764545681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1764545681},{"type":"v1","addr":"172.21.15.81:6837","nonce":1764545681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:00.980798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:02.568 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 71 2023-11-08T16:38:02.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 -- 172.21.15.81:0/3885316166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8fa406ae70 msgr2=0x7f8fa406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8fa406ae70 0x7f8fa406d330 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0036000 tx=0x7f8fb003c040 comp rx=0 tx=0).stop 2023-11-08T16:38:02.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 -- 172.21.15.81:0/3885316166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc00cf430 msgr2=0x7f8fc01f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc00cf430 0x7f8fc01f5c00 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00e7d0 tx=0x7f8fbc00ec90 comp rx=0 tx=0).stop 2023-11-08T16:38:02.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 -- 172.21.15.81:0/3885316166 shutdown_connections 2023-11-08T16:38:02.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8fa406ae70 0x7f8fa406d330 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fc015d1a0 0x7f8fc00cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8fc00d13e0 0x7f8fc01f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 --2- 172.21.15.81:0/3885316166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8fc00cf430 0x7f8fc01f5c00 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 -- 172.21.15.81:0/3885316166 >> 172.21.15.81:0/3885316166 conn(0x7f8fc00d2e00 msgr2=0x7f8fc006ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:02.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 -- 172.21.15.81:0/3885316166 shutdown_connections 2023-11-08T16:38:02.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.567+0000 7f8fc7b00640 1 -- 172.21.15.81:0/3885316166 wait complete. 2023-11-08T16:38:02.586 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:02.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.798+0000 7fa554e75640 1 Processor -- start 2023-11-08T16:38:02.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.798+0000 7fa554e75640 1 -- start start 2023-11-08T16:38:02.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.799+0000 7fa554e75640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5500ca480 0x7fa55015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.799+0000 7fa554e75640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa55015ea60 0x7fa55015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.799+0000 7fa554e75640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55006b630 con 0x7fa5500ade50 2023-11-08T16:38:02.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.799+0000 7fa554e75640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5500b1860 con 0x7fa55015ea60 2023-11-08T16:38:02.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.799+0000 7fa554e75640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5500ca8e0 con 0x7fa5500ca480 2023-11-08T16:38:02.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.800+0000 7fa54ed9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa55015ea60 0x7fa55015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.800+0000 7fa545d9b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5500ca480 0x7fa55015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.800+0000 7fa54ed9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa55015ea60 0x7fa55015eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:59354/0 (socket says 172.21.15.81:59354) 2023-11-08T16:38:02.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.800+0000 7fa54ed9d640 1 -- 172.21.15.81:0/425023575 learned_addr learned my addr 172.21.15.81:0/425023575 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:02.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.801+0000 7fa54ed9d640 1 -- 172.21.15.81:0/425023575 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5500ca480 msgr2=0x7fa55015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:02.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.801+0000 7fa54ed9d640 1 --2- 172.21.15.81:0/425023575 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5500ca480 0x7fa55015ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.801+0000 7fa54ed9d640 1 -- 172.21.15.81:0/425023575 >> v1:172.21.15.93:6789/0 conn(0x7fa5500ade50 legacy=0x7fa55015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.801+0000 7fa54ed9d640 1 -- 172.21.15.81:0/425023575 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5500cabb0 con 0x7fa55015ea60 2023-11-08T16:38:02.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.802+0000 7fa54ed9d640 1 --2- 172.21.15.81:0/425023575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa55015ea60 0x7fa55015eef0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa548009870 tx=0x7fa54802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eaf49b0316216a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:02.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.802+0000 7fa54dd9b640 1 -- 172.21.15.81:0/425023575 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548030840 con 0x7fa55015ea60 2023-11-08T16:38:02.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.802+0000 7fa54dd9b640 1 -- 172.21.15.81:0/425023575 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5480309a0 con 0x7fa55015ea60 2023-11-08T16:38:02.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.802+0000 7fa54dd9b640 1 -- 172.21.15.81:0/425023575 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa548030d30 con 0x7fa55015ea60 2023-11-08T16:38:02.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.803+0000 7fa554e75640 1 -- 172.21.15.81:0/425023575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa55015ea60 msgr2=0x7fa55015eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.803+0000 7fa554e75640 1 --2- 172.21.15.81:0/425023575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa55015ea60 0x7fa55015eef0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa548009870 tx=0x7fa54802b980 comp rx=0 tx=0).stop 2023-11-08T16:38:02.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.803+0000 7fa554e75640 1 -- 172.21.15.81:0/425023575 shutdown_connections 2023-11-08T16:38:02.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.803+0000 7fa554e75640 1 --2- 172.21.15.81:0/425023575 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa55015ea60 0x7fa55015eef0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.803+0000 7fa554e75640 1 --2- 172.21.15.81:0/425023575 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5500ca480 0x7fa55015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.803+0000 7fa554e75640 1 -- 172.21.15.81:0/425023575 >> 172.21.15.81:0/425023575 conn(0x7fa5500cb490 msgr2=0x7fa5500cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:02.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.803+0000 7fa554e75640 1 -- 172.21.15.81:0/425023575 shutdown_connections 2023-11-08T16:38:02.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.803+0000 7fa554e75640 1 -- 172.21.15.81:0/425023575 wait complete. 2023-11-08T16:38:02.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.805+0000 7fa554e75640 1 Processor -- start 2023-11-08T16:38:02.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.805+0000 7fa554e75640 1 -- start start 2023-11-08T16:38:02.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa554e75640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5500ade50 0x7fa550155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa554e75640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5500ca480 0x7fa5501505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa554e75640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa55015ea60 0x7fa55015db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa554e75640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa550156630 con 0x7fa5500ca480 2023-11-08T16:38:02.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa554e75640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa550156910 con 0x7fa5500ade50 2023-11-08T16:38:02.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa554e75640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa550156bf0 con 0x7fa55015ea60 2023-11-08T16:38:02.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa545d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5500ca480 0x7fa5501505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa545d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5500ca480 0x7fa5501505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59264/0 (socket says 172.21.15.81:59264) 2023-11-08T16:38:02.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa545d9b640 1 -- 172.21.15.81:0/1133682332 learned_addr learned my addr 172.21.15.81:0/1133682332 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:02.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa54ed9d640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa55015ea60 0x7fa55015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa54e59c640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5500ade50 0x7fa550155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa545d9b640 1 -- 172.21.15.81:0/1133682332 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa55015ea60 msgr2=0x7fa55015db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.806+0000 7fa545d9b640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa55015ea60 0x7fa55015db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.807+0000 7fa545d9b640 1 -- 172.21.15.81:0/1133682332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5500ade50 msgr2=0x7fa550155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:02.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.807+0000 7fa545d9b640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5500ade50 0x7fa550155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:02.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.807+0000 7fa545d9b640 1 -- 172.21.15.81:0/1133682332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5480094d0 con 0x7fa5500ca480 2023-11-08T16:38:02.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.807+0000 7fa54e59c640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5500ade50 0x7fa550155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:02.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.807+0000 7fa545d9b640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5500ca480 0x7fa5501505e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa53800e9b0 tx=0x7fa53800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:02.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.807+0000 7fa5477fe640 1 -- 172.21.15.81:0/1133682332 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa538009670 con 0x7fa5500ca480 2023-11-08T16:38:02.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.807+0000 7fa5477fe640 1 -- 172.21.15.81:0/1133682332 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa53800c9a0 con 0x7fa5500ca480 2023-11-08T16:38:02.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.807+0000 7fa5477fe640 1 -- 172.21.15.81:0/1133682332 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa53800cd50 con 0x7fa5500ca480 2023-11-08T16:38:02.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.807+0000 7fa554e75640 1 -- 172.21.15.81:0/1133682332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa550156ed0 con 0x7fa5500ca480 2023-11-08T16:38:02.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.807+0000 7fa554e75640 1 -- 172.21.15.81:0/1133682332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa550150b20 con 0x7fa5500ca480 2023-11-08T16:38:02.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.808+0000 7fa554e75640 1 -- 172.21.15.81:0/1133682332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55004ece0 con 0x7fa5500ca480 2023-11-08T16:38:02.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.809+0000 7fa5477fe640 1 -- 172.21.15.81:0/1133682332 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa538013470 con 0x7fa5500ca480 2023-11-08T16:38:02.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.810+0000 7fa5477fe640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa53006ae70 0x7fa53006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:02.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.810+0000 7fa5477fe640 1 -- 172.21.15.81:0/1133682332 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x7fa538089a50 con 0x7fa5500ca480 2023-11-08T16:38:02.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.811+0000 7fa54e59c640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa53006ae70 0x7fa53006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:02.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.811+0000 7fa54e59c640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa53006ae70 0x7fa53006d330 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa55015e8e0 tx=0x7fa540009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:02.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:02.815+0000 7fa5477fe640 1 -- 172.21.15.81:0/1133682332 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa538054910 con 0x7fa5500ca480 2023-11-08T16:38:03.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.026+0000 7fa554e75640 1 -- 172.21.15.81:0/1133682332 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa5500cc1d0 con 0x7fa5500ca480 2023-11-08T16:38:03.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.027+0000 7fa5477fe640 1 -- 172.21.15.81:0/1133682332 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4910 (secure 0 0 0) 0x7fa538017090 con 0x7fa5500ca480 2023-11-08T16:38:03.029 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:03.030 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/2125403666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2125403666},{"type":"v1","addr":"172.21.15.93:6834","nonce":2125403666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2727356524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2727356524},{"type":"v1","addr":"172.21.15.93:6835","nonce":2727356524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6936,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2639654743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2639654743},{"type":"v1","addr":"172.21.15.81:6835","nonce":2639654743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1764545681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1764545681},{"type":"v1","addr":"172.21.15.81:6837","nonce":1764545681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:00.980798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:03.030 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 71 2023-11-08T16:38:03.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 -- 172.21.15.81:0/1133682332 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa53006ae70 msgr2=0x7fa53006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa53006ae70 0x7fa53006d330 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa55015e8e0 tx=0x7fa540009040 comp rx=0 tx=0).stop 2023-11-08T16:38:03.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 -- 172.21.15.81:0/1133682332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5500ca480 msgr2=0x7fa5501505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5500ca480 0x7fa5501505e0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa53800e9b0 tx=0x7fa53800ee70 comp rx=0 tx=0).stop 2023-11-08T16:38:03.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 -- 172.21.15.81:0/1133682332 shutdown_connections 2023-11-08T16:38:03.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa53006ae70 0x7fa53006d330 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa55015ea60 0x7fa55015db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5500ca480 0x7fa5501505e0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 --2- 172.21.15.81:0/1133682332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5500ade50 0x7fa550155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 -- 172.21.15.81:0/1133682332 >> 172.21.15.81:0/1133682332 conn(0x7fa5500cb490 msgr2=0x7fa55015cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:03.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 -- 172.21.15.81:0/1133682332 shutdown_connections 2023-11-08T16:38:03.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.029+0000 7fa554e75640 1 -- 172.21.15.81:0/1133682332 wait complete. 2023-11-08T16:38:03.050 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:03.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.245+0000 7f37bc1b6640 1 Processor -- start 2023-11-08T16:38:03.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.245+0000 7f37bc1b6640 1 -- start start 2023-11-08T16:38:03.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.247+0000 7f37bc1b6640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f37b4161710 0x7f37b40cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.247+0000 7f37bc1b6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b4161bc0 con 0x7f37b40ccd30 2023-11-08T16:38:03.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.247+0000 7f37bc1b6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37b406b980 con 0x7f37b415d2b0 2023-11-08T16:38:03.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.247+0000 7f37bc1b6640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37b40cd1c0 con 0x7f37b4161710 2023-11-08T16:38:03.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.247+0000 7f37ba753640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f37b40ccd30 0x7f37b406ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50322/0 (socket says 172.21.15.81:50322) 2023-11-08T16:38:03.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.247+0000 7f37ba753640 1 -- 172.21.15.81:0/2878958498 learned_addr learned my addr 172.21.15.81:0/2878958498 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:03.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.247+0000 7f37b9751640 1 --2- 172.21.15.81:0/2878958498 >> v2:172.21.15.93:3300/0 conn(0x7f37b4161710 0x7f37b40cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1469339101 0 0) 0x7f37b4161bc0 con 0x7f37b40ccd30 2023-11-08T16:38:03.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37a0003520 con 0x7f37b40ccd30 2023-11-08T16:38:03.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2723447435 0 0) 0x7f37a0003520 con 0x7f37b40ccd30 2023-11-08T16:38:03.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 >> v2:172.21.15.93:3300/0 conn(0x7f37b4161710 msgr2=0x7f37b40cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:03.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 --2- 172.21.15.81:0/2878958498 >> v2:172.21.15.93:3300/0 conn(0x7f37b4161710 0x7f37b40cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 >> v1:172.21.15.93:6789/0 conn(0x7f37b415d2b0 legacy=0x7f37b40c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b406bc60 con 0x7f37b40ccd30 2023-11-08T16:38:03.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f37ac0019f0 con 0x7f37b40ccd30 2023-11-08T16:38:03.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 >> v1:172.21.15.81:6789/0 conn(0x7f37b40ccd30 legacy=0x7f37b406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 --2- 172.21.15.81:0/2878958498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37a0003a50 0x7f37a0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 --2- 172.21.15.81:0/2878958498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 0x7f37a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 --2- 172.21.15.81:0/2878958498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37a00051d0 0x7f37a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40cd1c0 con 0x7f37a00047c0 2023-11-08T16:38:03.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a000c100 con 0x7f37a00051d0 2023-11-08T16:38:03.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a000c2c0 con 0x7f37a0003a50 2023-11-08T16:38:03.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.248+0000 7f37b9751640 1 --2- 172.21.15.81:0/2878958498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 0x7f37a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.249+0000 7f37b9f52640 1 --2- 172.21.15.81:0/2878958498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37a0003a50 0x7f37a0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.249+0000 7f37ba753640 1 --2- 172.21.15.81:0/2878958498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37a00051d0 0x7f37a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.249+0000 7f37b9751640 1 -- 172.21.15.81:0/2878958498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37a0003a50 msgr2=0x7f37a0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:03.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.249+0000 7f37b9751640 1 --2- 172.21.15.81:0/2878958498 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37a0003a50 0x7f37a0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.249+0000 7f37b9751640 1 -- 172.21.15.81:0/2878958498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37a00051d0 msgr2=0x7f37a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:03.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.249+0000 7f37b9751640 1 --2- 172.21.15.81:0/2878958498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37a00051d0 0x7f37a000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.249+0000 7f37b9751640 1 -- 172.21.15.81:0/2878958498 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a0003520 con 0x7f37a00047c0 2023-11-08T16:38:03.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.249+0000 7f37ba753640 1 -- 172.21.15.81:0/2878958498 reap_dead start 2023-11-08T16:38:03.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.250+0000 7f37b9751640 1 --2- 172.21.15.81:0/2878958498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 0x7f37a0004c40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009a90 tx=0x7f37a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743672b17471c88 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:03.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.250+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a402c680 con 0x7f37a00047c0 2023-11-08T16:38:03.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.250+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a402c7e0 con 0x7f37a00047c0 2023-11-08T16:38:03.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.250+0000 7f37b8f50640 1 -- 172.21.15.81:0/2878958498 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a402cb70 con 0x7f37a00047c0 2023-11-08T16:38:03.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.251+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2878958498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 msgr2=0x7f37a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.251+0000 7f37bc1b6640 1 --2- 172.21.15.81:0/2878958498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 0x7f37a0004c40 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f37a4009a90 tx=0x7f37a00057f0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.251+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2878958498 shutdown_connections 2023-11-08T16:38:03.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.251+0000 7f37bc1b6640 1 --2- 172.21.15.81:0/2878958498 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 0x7f37a0004c40 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.251+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2878958498 >> 172.21.15.81:0/2878958498 conn(0x7f37b40cb490 msgr2=0x7f37b40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:03.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.251+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2878958498 shutdown_connections 2023-11-08T16:38:03.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.251+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2878958498 wait complete. 2023-11-08T16:38:03.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.253+0000 7f37bc1b6640 1 Processor -- start 2023-11-08T16:38:03.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.253+0000 7f37bc1b6640 1 -- start start 2023-11-08T16:38:03.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.254+0000 7f37bc1b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 0x7f37b40c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.254+0000 7f37bc1b6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37b4156d20 0x7f37b415b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.254+0000 7f37bc1b6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37b414e0c0 0x7f37b41555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.254+0000 7f37bc1b6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b4159070 con 0x7f37a00047c0 2023-11-08T16:38:03.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.254+0000 7f37bc1b6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40c5b80 con 0x7f37b4156d20 2023-11-08T16:38:03.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.254+0000 7f37bc1b6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37b40c5e60 con 0x7f37b414e0c0 2023-11-08T16:38:03.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.255+0000 7f37b9f52640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 0x7f37b40c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.255+0000 7f37b9f52640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 0x7f37b40c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59284/0 (socket says 172.21.15.81:59284) 2023-11-08T16:38:03.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.255+0000 7f37b9f52640 1 -- 172.21.15.81:0/2053441655 learned_addr learned my addr 172.21.15.81:0/2053441655 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:03.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.255+0000 7f37b9751640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37b4156d20 0x7f37b415b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.255+0000 7f37b9f52640 1 -- 172.21.15.81:0/2053441655 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37b414e0c0 msgr2=0x7f37b41555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.255+0000 7f37b9f52640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37b414e0c0 0x7f37b41555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.255+0000 7f37b9f52640 1 -- 172.21.15.81:0/2053441655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37b4156d20 msgr2=0x7f37b415b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.255+0000 7f37b9f52640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37b4156d20 0x7f37b415b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.255+0000 7f37b9f52640 1 -- 172.21.15.81:0/2053441655 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b40c6140 con 0x7f37a00047c0 2023-11-08T16:38:03.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.255+0000 7f37b9f52640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 0x7f37b40c64c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f37b0007fd0 tx=0x7f37b000e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:03.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.256+0000 7f37aa7fc640 1 -- 172.21.15.81:0/2053441655 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b00052e0 con 0x7f37a00047c0 2023-11-08T16:38:03.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.256+0000 7f37aa7fc640 1 -- 172.21.15.81:0/2053441655 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b0005440 con 0x7f37a00047c0 2023-11-08T16:38:03.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.256+0000 7f37aa7fc640 1 -- 172.21.15.81:0/2053441655 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b0003810 con 0x7f37a00047c0 2023-11-08T16:38:03.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.256+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2053441655 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b4156800 con 0x7f37a00047c0 2023-11-08T16:38:03.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.256+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2053441655 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b415ba40 con 0x7f37a00047c0 2023-11-08T16:38:03.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.257+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2053441655 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b404ece0 con 0x7f37a00047c0 2023-11-08T16:38:03.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.258+0000 7f37aa7fc640 1 -- 172.21.15.81:0/2053441655 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f37b0003970 con 0x7f37a00047c0 2023-11-08T16:38:03.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.259+0000 7f37aa7fc640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f379806ae70 0x7f379806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.259+0000 7f37aa7fc640 1 -- 172.21.15.81:0/2053441655 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x7f37b0089120 con 0x7f37a00047c0 2023-11-08T16:38:03.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.259+0000 7f37b9751640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f379806ae70 0x7f379806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.260+0000 7f37b9751640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f379806ae70 0x7f379806d330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f37a4036000 tx=0x7f37a4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:03.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.265+0000 7f37aa7fc640 1 -- 172.21.15.81:0/2053441655 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f37b0053f60 con 0x7f37a00047c0 2023-11-08T16:38:03.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.492+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2053441655 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37b40b4990 con 0x7f37a00047c0 2023-11-08T16:38:03.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.494+0000 7f37aa7fc640 1 -- 172.21.15.81:0/2053441655 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f37b0057580 con 0x7f37a00047c0 2023-11-08T16:38:03.496 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:03.497 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/2125403666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2125403666},{"type":"v1","addr":"172.21.15.93:6834","nonce":2125403666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2727356524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2727356524},{"type":"v1","addr":"172.21.15.93:6835","nonce":2727356524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6936,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2639654743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2639654743},{"type":"v1","addr":"172.21.15.81:6835","nonce":2639654743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1764545681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1764545681},{"type":"v1","addr":"172.21.15.81:6837","nonce":1764545681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:00.980798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:03.497 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 71 2023-11-08T16:38:03.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2053441655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f379806ae70 msgr2=0x7f379806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f379806ae70 0x7f379806d330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f37a4036000 tx=0x7f37a4030000 comp rx=0 tx=0).stop 2023-11-08T16:38:03.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2053441655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 msgr2=0x7f37b40c64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 0x7f37b40c64c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f37b0007fd0 tx=0x7f37b000e8a0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2053441655 shutdown_connections 2023-11-08T16:38:03.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f379806ae70 0x7f379806d330 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f37b414e0c0 0x7f37b41555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f37b4156d20 0x7f37b415b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 --2- 172.21.15.81:0/2053441655 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f37a00047c0 0x7f37b40c64c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2053441655 >> 172.21.15.81:0/2053441655 conn(0x7f37b40cb490 msgr2=0x7f37b406c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:03.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2053441655 shutdown_connections 2023-11-08T16:38:03.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.496+0000 7f37bc1b6640 1 -- 172.21.15.81:0/2053441655 wait complete. 2023-11-08T16:38:03.518 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:03.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.722+0000 7f6e8415e640 1 Processor -- start 2023-11-08T16:38:03.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.722+0000 7f6e8415e640 1 -- start start 2023-11-08T16:38:03.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.724+0000 7f6e8415e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6e7c0ca480 0x7f6e7c15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.724+0000 7f6e8415e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e7c15ea60 0x7f6e7c15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.724+0000 7f6e8415e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c06b630 con 0x7f6e7c0ade50 2023-11-08T16:38:03.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.724+0000 7f6e8415e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c0b1860 con 0x7f6e7c0ca480 2023-11-08T16:38:03.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.724+0000 7f6e8415e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0ca910 con 0x7f6e7c15ea60 2023-11-08T16:38:03.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.724+0000 7f6e816f9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6e7c0ca480 0x7f6e7c15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.724+0000 7f6e826fb640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e7c15ea60 0x7f6e7c15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.724+0000 7f6e81efa640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f6e7c0ade50 0x7f6e7c15a680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:53280/0 (socket says 172.21.15.81:53280) 2023-11-08T16:38:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.724+0000 7f6e816f9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6e7c0ca480 0x7f6e7c15ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:59398/0 (socket says 172.21.15.81:59398) 2023-11-08T16:38:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.724+0000 7f6e81efa640 1 -- 172.21.15.81:0/2196027932 learned_addr learned my addr 172.21.15.81:0/2196027932 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.725+0000 7f6e80ef8640 1 -- 172.21.15.81:0/2196027932 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3768936695 0 0) 0x7f6e7c06b630 con 0x7f6e7c0ade50 2023-11-08T16:38:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.725+0000 7f6e80ef8640 1 -- 172.21.15.81:0/2196027932 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e68003520 con 0x7f6e7c0ade50 2023-11-08T16:38:03.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.725+0000 7f6e826fb640 1 -- 172.21.15.81:0/2196027932 >> v2:172.21.15.93:3300/0 conn(0x7f6e7c0ca480 msgr2=0x7f6e7c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:03.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.725+0000 7f6e826fb640 1 --2- 172.21.15.81:0/2196027932 >> v2:172.21.15.93:3300/0 conn(0x7f6e7c0ca480 0x7f6e7c15ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.725+0000 7f6e826fb640 1 -- 172.21.15.81:0/2196027932 >> v1:172.21.15.93:6789/0 conn(0x7f6e7c0ade50 legacy=0x7f6e7c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.725+0000 7f6e826fb640 1 -- 172.21.15.81:0/2196027932 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c0cabb0 con 0x7f6e7c15ea60 2023-11-08T16:38:03.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.725+0000 7f6e816f9640 1 --2- 172.21.15.81:0/2196027932 >> v2:172.21.15.93:3300/0 conn(0x7f6e7c0ca480 0x7f6e7c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:38:03.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.727+0000 7f6e826fb640 1 --2- 172.21.15.81:0/2196027932 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e7c15ea60 0x7f6e7c15eef0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009d90 tx=0x7f6e7402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0293fde079ed8a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:03.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.727+0000 7f6e80ef8640 1 -- 172.21.15.81:0/2196027932 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7402cc50 con 0x7f6e7c15ea60 2023-11-08T16:38:03.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.727+0000 7f6e80ef8640 1 -- 172.21.15.81:0/2196027932 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7402cdb0 con 0x7f6e7c15ea60 2023-11-08T16:38:03.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.727+0000 7f6e80ef8640 1 -- 172.21.15.81:0/2196027932 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74034640 con 0x7f6e7c15ea60 2023-11-08T16:38:03.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.728+0000 7f6e8415e640 1 -- 172.21.15.81:0/2196027932 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e7c15ea60 msgr2=0x7f6e7c15eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.728+0000 7f6e8415e640 1 --2- 172.21.15.81:0/2196027932 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e7c15ea60 0x7f6e7c15eef0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6e74009d90 tx=0x7f6e7402bea0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.728+0000 7f6e8415e640 1 -- 172.21.15.81:0/2196027932 shutdown_connections 2023-11-08T16:38:03.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.728+0000 7f6e8415e640 1 --2- 172.21.15.81:0/2196027932 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e7c15ea60 0x7f6e7c15eef0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.728+0000 7f6e8415e640 1 --2- 172.21.15.81:0/2196027932 >> v2:172.21.15.93:3300/0 conn(0x7f6e7c0ca480 0x7f6e7c15ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.728+0000 7f6e8415e640 1 -- 172.21.15.81:0/2196027932 >> 172.21.15.81:0/2196027932 conn(0x7f6e7c0cb490 msgr2=0x7f6e7c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:03.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.728+0000 7f6e8415e640 1 -- 172.21.15.81:0/2196027932 shutdown_connections 2023-11-08T16:38:03.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.728+0000 7f6e8415e640 1 -- 172.21.15.81:0/2196027932 wait complete. 2023-11-08T16:38:03.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.730+0000 7f6e8415e640 1 Processor -- start 2023-11-08T16:38:03.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.730+0000 7f6e8415e640 1 -- start start 2023-11-08T16:38:03.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e8415e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e7c0ade50 0x7f6e7c1e5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e8415e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e7c0ca480 0x7f6e7c0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e8415e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e7c15ea60 0x7f6e7c0ab0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e8415e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1e6cf0 con 0x7f6e7c0ade50 2023-11-08T16:38:03.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e8415e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1e6170 con 0x7f6e7c15ea60 2023-11-08T16:38:03.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e8415e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c1e6450 con 0x7f6e7c0ca480 2023-11-08T16:38:03.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e81efa640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e7c0ade50 0x7f6e7c1e5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e81efa640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e7c0ade50 0x7f6e7c1e5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59292/0 (socket says 172.21.15.81:59292) 2023-11-08T16:38:03.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e81efa640 1 -- 172.21.15.81:0/3284958517 learned_addr learned my addr 172.21.15.81:0/3284958517 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:03.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e826fb640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e7c15ea60 0x7f6e7c0ab0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e81efa640 1 -- 172.21.15.81:0/3284958517 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e7c0ca480 msgr2=0x7f6e7c0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e81efa640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e7c0ca480 0x7f6e7c0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e81efa640 1 -- 172.21.15.81:0/3284958517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e7c15ea60 msgr2=0x7f6e7c0ab0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e81efa640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e7c15ea60 0x7f6e7c0ab0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.731+0000 7f6e81efa640 1 -- 172.21.15.81:0/3284958517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c009400 con 0x7f6e7c0ade50 2023-11-08T16:38:03.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.732+0000 7f6e81efa640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e7c0ade50 0x7f6e7c1e5b90 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800c430 tx=0x7f6e7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:03.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.732+0000 7f6e727fc640 1 -- 172.21.15.81:0/3284958517 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e78005ce0 con 0x7f6e7c0ade50 2023-11-08T16:38:03.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.732+0000 7f6e727fc640 1 -- 172.21.15.81:0/3284958517 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e78005e40 con 0x7f6e7c0ade50 2023-11-08T16:38:03.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.732+0000 7f6e727fc640 1 -- 172.21.15.81:0/3284958517 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e780072f0 con 0x7f6e7c0ade50 2023-11-08T16:38:03.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.732+0000 7f6e8415e640 1 -- 172.21.15.81:0/3284958517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e74009a70 con 0x7f6e7c0ade50 2023-11-08T16:38:03.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.732+0000 7f6e8415e640 1 -- 172.21.15.81:0/3284958517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c1e6a10 con 0x7f6e7c0ade50 2023-11-08T16:38:03.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.733+0000 7f6e8415e640 1 -- 172.21.15.81:0/3284958517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c0a84a0 con 0x7f6e7c0ade50 2023-11-08T16:38:03.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.734+0000 7f6e727fc640 1 -- 172.21.15.81:0/3284958517 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e78007450 con 0x7f6e7c0ade50 2023-11-08T16:38:03.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.735+0000 7f6e727fc640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006aea0 0x7f6e6006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:03.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.735+0000 7f6e727fc640 1 -- 172.21.15.81:0/3284958517 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x7f6e78088f70 con 0x7f6e7c0ade50 2023-11-08T16:38:03.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.735+0000 7f6e816f9640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006aea0 0x7f6e6006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:03.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.737+0000 7f6e816f9640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006aea0 0x7f6e6006d360 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c006e80 tx=0x7f6e6c006dd0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:03.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.744+0000 7f6e727fc640 1 -- 172.21.15.81:0/3284958517 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e7c0a84a0 con 0x7f6e7c0ade50 2023-11-08T16:38:03.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.958+0000 7f6e8415e640 1 -- 172.21.15.81:0/3284958517 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e7c0a84a0 con 0x7f6e7c0ade50 2023-11-08T16:38:03.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.960+0000 7f6e727fc640 1 -- 172.21.15.81:0/3284958517 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4910 (secure 0 0 0) 0x7f6e7c0a84a0 con 0x7f6e7c0ade50 2023-11-08T16:38:03.962 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:03.963 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":71,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/2125403666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2125403666},{"type":"v1","addr":"172.21.15.93:6834","nonce":2125403666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":6933,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2727356524","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":2727356524},{"type":"v1","addr":"172.21.15.93:6835","nonce":2727356524}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":6936,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2639654743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2639654743},{"type":"v1","addr":"172.21.15.81:6835","nonce":2639654743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7230,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1764545681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1764545681},{"type":"v1","addr":"172.21.15.81:6837","nonce":1764545681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:00.980798+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":86,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:03.963 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 71 2023-11-08T16:38:03.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 -- 172.21.15.81:0/3284958517 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006aea0 msgr2=0x7f6e6006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006aea0 0x7f6e6006d360 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c006e80 tx=0x7f6e6c006dd0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 -- 172.21.15.81:0/3284958517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e7c0ade50 msgr2=0x7f6e7c1e5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:03.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e7c0ade50 0x7f6e7c1e5b90 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800c430 tx=0x7f6e7800c8f0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 -- 172.21.15.81:0/3284958517 shutdown_connections 2023-11-08T16:38:03.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e6006aea0 0x7f6e6006d360 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e7c15ea60 0x7f6e7c0ab0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e7c0ca480 0x7f6e7c0acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 --2- 172.21.15.81:0/3284958517 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e7c0ade50 0x7f6e7c1e5b90 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:03.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 -- 172.21.15.81:0/3284958517 >> 172.21.15.81:0/3284958517 conn(0x7f6e7c0cb490 msgr2=0x7f6e7c15dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:03.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 -- 172.21.15.81:0/3284958517 shutdown_connections 2023-11-08T16:38:03.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:03.962+0000 7f6e8415e640 1 -- 172.21.15.81:0/3284958517 wait complete. 2023-11-08T16:38:03.982 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-11-08T16:38:04.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.078+0000 7f46af24c100 1 -- start start 2023-11-08T16:38:04.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.078+0000 7f46af24c100 1 Processor -- start 2023-11-08T16:38:04.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46af24c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88910c00 0x55ff8897c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46af24c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88911800 0x55ff8897cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46af24c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff88911c00 0x55ff8897d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46af24c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8895b180 con 0x55ff88911800 2023-11-08T16:38:04.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46af24c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff889df180 con 0x55ff88911c00 2023-11-08T16:38:04.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46af24c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff889dedc0 con 0x55ff88910c00 2023-11-08T16:38:04.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46aca54640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88911800 0x55ff8897cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46aca54640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88911800 0x55ff8897cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59304/0 (socket says 172.21.15.81:59304) 2023-11-08T16:38:04.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46aca54640 1 -- 172.21.15.81:0/2756484457 learned_addr learned my addr 172.21.15.81:0/2756484457 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46aca54640 1 -- 172.21.15.81:0/2756484457 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88910c00 msgr2=0x55ff8897c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46aca54640 1 --2- 172.21.15.81:0/2756484457 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88910c00 0x55ff8897c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46aca54640 1 -- 172.21.15.81:0/2756484457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff88911c00 msgr2=0x55ff8897d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46aca54640 1 --2- 172.21.15.81:0/2756484457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff88911c00 0x55ff8897d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.079+0000 7f46aca54640 1 -- 172.21.15.81:0/2756484457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff889369c0 con 0x55ff88911800 2023-11-08T16:38:04.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46aca54640 1 --2- 172.21.15.81:0/2756484457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88911800 0x55ff8897cb00 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x55ff8972ab70 tx=0x55ff88be8120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e194326d0aa81408 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46aaa50640 1 -- 172.21.15.81:0/2756484457 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff88958f20 con 0x55ff88911800 2023-11-08T16:38:04.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46aaa50640 1 -- 172.21.15.81:0/2756484457 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff896e4000 con 0x55ff88911800 2023-11-08T16:38:04.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46af24c100 1 -- 172.21.15.81:0/2756484457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff88936d00 con 0x55ff88911800 2023-11-08T16:38:04.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46af24c100 1 -- 172.21.15.81:0/2756484457 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55ff889369c0 con 0x55ff88911800 2023-11-08T16:38:04.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46af24c100 4 waiting for MDS map... 2023-11-08T16:38:04.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46aaa50640 1 -- 172.21.15.81:0/2756484457 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff889591e0 con 0x55ff88911800 2023-11-08T16:38:04.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46aaa50640 1 -- 172.21.15.81:0/2756484457 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x55ff8893c380 con 0x55ff88911800 2023-11-08T16:38:04.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46aaa50640 1 -- 172.21.15.81:0/2756484457 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 71) v1 ==== 3306+0+0 (secure 0 0 0) 0x55ff88962f00 con 0x55ff88911800 2023-11-08T16:38:04.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46af24c100 4 Got MDS map 71 2023-11-08T16:38:04.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46af24c100 10 main: main 2023-11-08T16:38:04.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.080+0000 7f46af24c100 4 main: connecting to RADOS... 2023-11-08T16:38:04.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.081+0000 7f46af24c100 1 Processor -- start 2023-11-08T16:38:04.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.081+0000 7f46af24c100 1 -- start start 2023-11-08T16:38:04.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.082+0000 7f46af24c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff896e3c00 0x55ff8897d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.082+0000 7f46af24c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88c52000 0x55ff8897db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.082+0000 7f46af24c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88c52400 0x55ff8897e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.082+0000 7f46af24c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8895a140 con 0x55ff88c52400 2023-11-08T16:38:04.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.082+0000 7f46af24c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff8895ab40 con 0x55ff896e3c00 2023-11-08T16:38:04.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.082+0000 7f46af24c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff889dedc0 con 0x55ff88c52000 2023-11-08T16:38:04.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.082+0000 7f46aca54640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88c52400 0x55ff8897e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.082+0000 7f46aca54640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88c52400 0x55ff8897e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59320/0 (socket says 172.21.15.81:59320) 2023-11-08T16:38:04.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.082+0000 7f46aca54640 1 -- 172.21.15.81:0/4225923472 learned_addr learned my addr 172.21.15.81:0/4225923472 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46ada56640 1 --2- 172.21.15.81:0/4225923472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff896e3c00 0x55ff8897d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46aca54640 1 -- 172.21.15.81:0/4225923472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88c52000 msgr2=0x55ff8897db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46aca54640 1 --2- 172.21.15.81:0/4225923472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88c52000 0x55ff8897db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46aca54640 1 -- 172.21.15.81:0/4225923472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff896e3c00 msgr2=0x55ff8897d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46aca54640 1 --2- 172.21.15.81:0/4225923472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff896e3c00 0x55ff8897d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46aca54640 1 -- 172.21.15.81:0/4225923472 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff88936b60 con 0x55ff88c52400 2023-11-08T16:38:04.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46ada56640 1 --2- 172.21.15.81:0/4225923472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff896e3c00 0x55ff8897d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:04.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46aca54640 1 --2- 172.21.15.81:0/4225923472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88c52400 0x55ff8897e100 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x55ff8972aff0 tx=0x55ff88be81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14d06a002a7cfb00 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46a8a4c640 1 -- 172.21.15.81:0/4225923472 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff88959080 con 0x55ff88c52400 2023-11-08T16:38:04.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46a8a4c640 1 -- 172.21.15.81:0/4225923472 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff896e4300 con 0x55ff88c52400 2023-11-08T16:38:04.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46a8a4c640 1 -- 172.21.15.81:0/4225923472 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff88958dc0 con 0x55ff88c52400 2023-11-08T16:38:04.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46af24c100 1 -- 172.21.15.81:0/4225923472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88c52400 msgr2=0x55ff8897e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.083+0000 7f46af24c100 1 --2- 172.21.15.81:0/4225923472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88c52400 0x55ff8897e100 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x55ff8972aff0 tx=0x55ff88be81e0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.084+0000 7f46af24c100 1 -- 172.21.15.81:0/4225923472 shutdown_connections 2023-11-08T16:38:04.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.084+0000 7f46af24c100 1 --2- 172.21.15.81:0/4225923472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88c52400 0x55ff8897e100 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.084+0000 7f46af24c100 1 --2- 172.21.15.81:0/4225923472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88c52000 0x55ff8897db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.084+0000 7f46af24c100 1 --2- 172.21.15.81:0/4225923472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff896e3c00 0x55ff8897d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.084+0000 7f46af24c100 1 -- 172.21.15.81:0/4225923472 >> 172.21.15.81:0/4225923472 conn(0x55ff896e3800 msgr2=0x55ff89688800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.084+0000 7f46af24c100 1 -- 172.21.15.81:0/4225923472 shutdown_connections 2023-11-08T16:38:04.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.084+0000 7f46af24c100 1 -- 172.21.15.81:0/4225923472 wait complete. 2023-11-08T16:38:04.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.084+0000 7f46af24c100 1 Processor -- start 2023-11-08T16:38:04.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.084+0000 7f46af24c100 1 -- start start 2023-11-08T16:38:04.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.085+0000 7f46af24c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff896e3800 0x55ff8897f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.085+0000 7f46af24c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff88c53000 0x55ff8897ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.085+0000 7f46af24c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88c52c00 0x55ff8897e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.085+0000 7f46af24c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff889dec80 con 0x55ff896e3800 2023-11-08T16:38:04.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.085+0000 7f46af24c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ff889deb40 con 0x55ff88c53000 2023-11-08T16:38:04.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.085+0000 7f46af24c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ff889dea00 con 0x55ff88c52c00 2023-11-08T16:38:04.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.085+0000 7f46ada56640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff896e3800 0x55ff8897f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.085+0000 7f46ada56640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff896e3800 0x55ff8897f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59330/0 (socket says 172.21.15.81:59330) 2023-11-08T16:38:04.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.085+0000 7f46ada56640 1 -- 172.21.15.81:0/1505759456 learned_addr learned my addr 172.21.15.81:0/1505759456 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46aca54640 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88c52c00 0x55ff8897e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46ad255640 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff88c53000 0x55ff8897ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46ada56640 1 -- 172.21.15.81:0/1505759456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88c52c00 msgr2=0x55ff8897e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46ada56640 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88c52c00 0x55ff8897e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46ada56640 1 -- 172.21.15.81:0/1505759456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff88c53000 msgr2=0x55ff8897ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46ada56640 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff88c53000 0x55ff8897ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46ada56640 1 -- 172.21.15.81:0/1505759456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ff88936d00 con 0x55ff896e3800 2023-11-08T16:38:04.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46ada56640 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff896e3800 0x55ff8897f180 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x55ff896def90 tx=0x55ff88c04d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46a7249640 1 -- 172.21.15.81:0/1505759456 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff88959080 con 0x55ff896e3800 2023-11-08T16:38:04.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46a7249640 1 -- 172.21.15.81:0/1505759456 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ff896e5980 con 0x55ff896e3800 2023-11-08T16:38:04.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46a7249640 1 -- 172.21.15.81:0/1505759456 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55ff88958dc0 con 0x55ff896e3800 2023-11-08T16:38:04.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46af24c100 1 -- 172.21.15.81:0/1505759456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ff889369c0 con 0x55ff896e3800 2023-11-08T16:38:04.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46af24c100 1 -- 172.21.15.81:0/1505759456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ff88936b60 con 0x55ff896e3800 2023-11-08T16:38:04.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.086+0000 7f46af24c100 4 main: resolving pool 10 2023-11-08T16:38:04.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.088+0000 7f46a7249640 1 -- 172.21.15.81:0/1505759456 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55ff8896e840 con 0x55ff896e3800 2023-11-08T16:38:04.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.089+0000 7f46a7249640 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55ff88c70c00 0x55ff8897f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.089+0000 7f46a7249640 1 -- 172.21.15.81:0/1505759456 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x55ff8893c540 con 0x55ff896e3800 2023-11-08T16:38:04.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.089+0000 7f46af24c100 4 main: creating IoCtx.. 2023-11-08T16:38:04.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.089+0000 7f46af24c100 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55ff88c52800 0x55ff8897d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.089+0000 7f46af24c100 1 -- 172.21.15.81:0/1505759456 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:1 10.b 10:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55ff8896b880 con 0x55ff88c52800 2023-11-08T16:38:04.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.089+0000 7f46ad255640 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55ff88c70c00 0x55ff8897f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.090+0000 7f46ad255640 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55ff88c70c00 0x55ff8897f700 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55ff8969e240 tx=0x55ff88c05e60 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.090+0000 7f46aca54640 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55ff88c52800 0x55ff8897d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.091+0000 7f46aca54640 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55ff88c52800 0x55ff8897d600 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46aca54640 1 -- 172.21.15.81:0/1505759456 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v86'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55ff896d8b40 con 0x55ff88c52800 2023-11-08T16:38:04.106 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:38:04.107 INFO:teuthology.orchestra.run.smithi081.stdout: "0": { 2023-11-08T16:38:04.107 INFO:teuthology.orchestra.run.smithi081.stdout: "data": {}, 2023-11-08T16:38:04.107 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:38:04.107 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:38:04.108 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:38:04.108 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:04.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46af24c100 1 -- 172.21.15.81:0/1505759456 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55ff88c52800 msgr2=0x55ff8897d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46af24c100 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55ff88c52800 0x55ff8897d600 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46af24c100 1 -- 172.21.15.81:0/1505759456 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55ff88c70c00 msgr2=0x55ff8897f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46af24c100 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55ff88c70c00 0x55ff8897f700 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55ff8969e240 tx=0x55ff88c05e60 comp rx=0 tx=0).stop 2023-11-08T16:38:04.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46af24c100 1 -- 172.21.15.81:0/1505759456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff896e3800 msgr2=0x55ff8897f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46af24c100 1 --2- 172.21.15.81:0/1505759456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff896e3800 0x55ff8897f180 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x55ff896def90 tx=0x55ff88c04d80 comp rx=0 tx=0).stop 2023-11-08T16:38:04.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46ad255640 1 -- 172.21.15.81:0/1505759456 reap_dead start 2023-11-08T16:38:04.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46af24c100 1 -- 172.21.15.81:0/1505759456 shutdown_connections 2023-11-08T16:38:04.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46af24c100 1 -- 172.21.15.81:0/1505759456 >> 172.21.15.81:0/1505759456 conn(0x55ff896e3c00 msgr2=0x55ff89688800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46af24c100 1 -- 172.21.15.81:0/1505759456 shutdown_connections 2023-11-08T16:38:04.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.094+0000 7f46af24c100 1 -- 172.21.15.81:0/1505759456 wait complete. 2023-11-08T16:38:04.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.095+0000 7f46af24c100 1 -- 172.21.15.81:0/2756484457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88911800 msgr2=0x55ff8897cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.095+0000 7f46af24c100 1 --2- 172.21.15.81:0/2756484457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88911800 0x55ff8897cb00 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x55ff8972ab70 tx=0x55ff88be8120 comp rx=0 tx=0).stop 2023-11-08T16:38:04.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.095+0000 7f46af24c100 1 -- 172.21.15.81:0/2756484457 shutdown_connections 2023-11-08T16:38:04.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.095+0000 7f46af24c100 1 --2- 172.21.15.81:0/2756484457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ff88911c00 0x55ff8897d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.095+0000 7f46af24c100 1 --2- 172.21.15.81:0/2756484457 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55ff88911800 0x55ff8897cb00 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.095+0000 7f46af24c100 1 --2- 172.21.15.81:0/2756484457 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55ff88910c00 0x55ff8897c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.095+0000 7f46af24c100 1 -- 172.21.15.81:0/2756484457 >> 172.21.15.81:0/2756484457 conn(0x55ff88911000 msgr2=0x55ff89689000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.095+0000 7f46af24c100 1 -- 172.21.15.81:0/2756484457 shutdown_connections 2023-11-08T16:38:04.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.095+0000 7f46af24c100 1 -- 172.21.15.81:0/2756484457 wait complete. 2023-11-08T16:38:04.114 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.132263, result: { "0": { "data": {}, "result": 0 } } 2023-11-08T16:38:04.114 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-11-08T16:38:04.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.207+0000 7f13a9a0f100 1 -- start start 2023-11-08T16:38:04.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.207+0000 7f13a9a0f100 1 Processor -- start 2023-11-08T16:38:04.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.208+0000 7f13a9a0f100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d1dcc00 0x557a7d249b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.208+0000 7f13a9a0f100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d1dd800 0x557a7d249600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.208+0000 7f13a9a0f100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d1ddc00 0x557a7d249080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.208+0000 7f13a9a0f100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a7d2ab180 con 0x557a7d1dd800 2023-11-08T16:38:04.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.208+0000 7f13a9a0f100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a7d2aadc0 con 0x557a7d1dcc00 2023-11-08T16:38:04.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.208+0000 7f13a9a0f100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a7d226a00 con 0x557a7d1ddc00 2023-11-08T16:38:04.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.208+0000 7f13a6db2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d1dd800 0x557a7d249600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.208+0000 7f13a6db2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d1dd800 0x557a7d249600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59350/0 (socket says 172.21.15.81:59350) 2023-11-08T16:38:04.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.208+0000 7f13a6db2640 1 -- 172.21.15.81:0/2181087576 learned_addr learned my addr 172.21.15.81:0/2181087576 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.208+0000 7f13a75b3640 1 --2- 172.21.15.81:0/2181087576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d1dcc00 0x557a7d249b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a7db4640 1 --2- 172.21.15.81:0/2181087576 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d1ddc00 0x557a7d249080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a6db2640 1 -- 172.21.15.81:0/2181087576 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d1ddc00 msgr2=0x557a7d249080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a6db2640 1 --2- 172.21.15.81:0/2181087576 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d1ddc00 0x557a7d249080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a6db2640 1 -- 172.21.15.81:0/2181087576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d1dcc00 msgr2=0x557a7d249b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a6db2640 1 --2- 172.21.15.81:0/2181087576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d1dcc00 0x557a7d249b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a6db2640 1 -- 172.21.15.81:0/2181087576 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a7d2029c0 con 0x557a7d1dd800 2023-11-08T16:38:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a75b3640 1 --2- 172.21.15.81:0/2181087576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d1dcc00 0x557a7d249b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a6db2640 1 --2- 172.21.15.81:0/2181087576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d1dd800 0x557a7d249600 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x557a7dff6ba0 tx=0x557a7d356ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8858d988e1025401 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a4dae640 1 -- 172.21.15.81:0/2181087576 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a7d2251e0 con 0x557a7d1dd800 2023-11-08T16:38:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a4dae640 1 -- 172.21.15.81:0/2181087576 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557a7d4c8000 con 0x557a7d1dd800 2023-11-08T16:38:04.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a4dae640 1 -- 172.21.15.81:0/2181087576 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a7d225080 con 0x557a7d1dd800 2023-11-08T16:38:04.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a9a0f100 1 -- 172.21.15.81:0/2181087576 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a7d202b60 con 0x557a7d1dd800 2023-11-08T16:38:04.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a9a0f100 1 -- 172.21.15.81:0/2181087576 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x557a7d202820 con 0x557a7d1dd800 2023-11-08T16:38:04.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a9a0f100 4 waiting for MDS map... 2023-11-08T16:38:04.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.209+0000 7f13a4dae640 1 -- 172.21.15.81:0/2181087576 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x557a7d208380 con 0x557a7d1dd800 2023-11-08T16:38:04.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.210+0000 7f13a4dae640 1 -- 172.21.15.81:0/2181087576 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 71) v1 ==== 3306+0+0 (secure 0 0 0) 0x557a7d22ef00 con 0x557a7d1dd800 2023-11-08T16:38:04.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.210+0000 7f13a9a0f100 4 Got MDS map 71 2023-11-08T16:38:04.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.210+0000 7f13a9a0f100 10 main: main 2023-11-08T16:38:04.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.210+0000 7f13a9a0f100 4 main: connecting to RADOS... 2023-11-08T16:38:04.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.211+0000 7f13a9a0f100 1 Processor -- start 2023-11-08T16:38:04.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.211+0000 7f13a9a0f100 1 -- start start 2023-11-08T16:38:04.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.211+0000 7f13a9a0f100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d4c7000 0x557a7d248b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.211+0000 7f13a9a0f100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d4c7400 0x557a7d248580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.211+0000 7f13a9a0f100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 0x557a7d24a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.211+0000 7f13a9a0f100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a7d226b40 con 0x557a7d4c7800 2023-11-08T16:38:04.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.211+0000 7f13a9a0f100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a7d2aadc0 con 0x557a7d4c7400 2023-11-08T16:38:04.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.211+0000 7f13a9a0f100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a7d2aac80 con 0x557a7d4c7000 2023-11-08T16:38:04.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a6db2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 0x557a7d24a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a6db2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 0x557a7d24a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59366/0 (socket says 172.21.15.81:59366) 2023-11-08T16:38:04.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a6db2640 1 -- 172.21.15.81:0/509886925 learned_addr learned my addr 172.21.15.81:0/509886925 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a7db4640 1 --2- 172.21.15.81:0/509886925 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d4c7000 0x557a7d248b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a75b3640 1 --2- 172.21.15.81:0/509886925 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d4c7400 0x557a7d248580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a6db2640 1 -- 172.21.15.81:0/509886925 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d4c7000 msgr2=0x557a7d248b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a6db2640 1 --2- 172.21.15.81:0/509886925 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d4c7000 0x557a7d248b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a6db2640 1 -- 172.21.15.81:0/509886925 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d4c7400 msgr2=0x557a7d248580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a6db2640 1 --2- 172.21.15.81:0/509886925 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d4c7400 0x557a7d248580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a6db2640 1 -- 172.21.15.81:0/509886925 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a7d2024e0 con 0x557a7d4c7800 2023-11-08T16:38:04.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a7db4640 1 --2- 172.21.15.81:0/509886925 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d4c7000 0x557a7d248b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:04.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.212+0000 7f13a6db2640 1 --2- 172.21.15.81:0/509886925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 0x557a7d24a100 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x557a7d536090 tx=0x557a7df6c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=920bf5a7562c1a45 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a2daa640 1 -- 172.21.15.81:0/509886925 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a7d224dc0 con 0x557a7d4c7800 2023-11-08T16:38:04.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a2daa640 1 -- 172.21.15.81:0/509886925 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557a7d4c8f00 con 0x557a7d4c7800 2023-11-08T16:38:04.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a2daa640 1 -- 172.21.15.81:0/509886925 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a7d2251e0 con 0x557a7d4c7800 2023-11-08T16:38:04.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a9a0f100 1 -- 172.21.15.81:0/509886925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 msgr2=0x557a7d24a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/509886925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 0x557a7d24a100 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x557a7d536090 tx=0x557a7df6c000 comp rx=0 tx=0).stop 2023-11-08T16:38:04.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a9a0f100 1 -- 172.21.15.81:0/509886925 shutdown_connections 2023-11-08T16:38:04.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/509886925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 0x557a7d24a100 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/509886925 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d4c7400 0x557a7d248580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/509886925 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d4c7000 0x557a7d248b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a9a0f100 1 -- 172.21.15.81:0/509886925 >> 172.21.15.81:0/509886925 conn(0x557a7d4c6c00 msgr2=0x557a7df67000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a9a0f100 1 -- 172.21.15.81:0/509886925 shutdown_connections 2023-11-08T16:38:04.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.213+0000 7f13a9a0f100 1 -- 172.21.15.81:0/509886925 wait complete. 2023-11-08T16:38:04.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.214+0000 7f13a9a0f100 1 Processor -- start 2023-11-08T16:38:04.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.214+0000 7f13a9a0f100 1 -- start start 2023-11-08T16:38:04.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.215+0000 7f13a9a0f100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d4c7000 0x557a7d248b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.215+0000 7f13a9a0f100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d4c7400 0x557a7d248580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.215+0000 7f13a9a0f100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 0x557a7d24a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.215+0000 7f13a9a0f100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a7d2aab40 con 0x557a7d4c7800 2023-11-08T16:38:04.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.215+0000 7f13a9a0f100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a7d2aaa00 con 0x557a7d4c7000 2023-11-08T16:38:04.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.215+0000 7f13a9a0f100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a7d2aa8c0 con 0x557a7d4c7400 2023-11-08T16:38:04.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.215+0000 7f13a6db2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 0x557a7d24a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.215+0000 7f13a6db2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 0x557a7d24a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59380/0 (socket says 172.21.15.81:59380) 2023-11-08T16:38:04.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.215+0000 7f13a6db2640 1 -- 172.21.15.81:0/1485213695 learned_addr learned my addr 172.21.15.81:0/1485213695 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.215+0000 7f13a75b3640 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d4c7400 0x557a7d248580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a6db2640 1 -- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d4c7400 msgr2=0x557a7d248580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a7db4640 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d4c7000 0x557a7d248b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a6db2640 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d4c7400 0x557a7d248580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a6db2640 1 -- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d4c7000 msgr2=0x557a7d248b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a6db2640 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d4c7000 0x557a7d248b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a6db2640 1 -- 172.21.15.81:0/1485213695 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a7d2029c0 con 0x557a7d4c7800 2023-11-08T16:38:04.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a75b3640 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d4c7400 0x557a7d248580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:04.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a6db2640 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 0x557a7d24a100 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x557a7d536300 tx=0x557a7d4e0c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a15a7640 1 -- 172.21.15.81:0/1485213695 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a7d225080 con 0x557a7d4c7800 2023-11-08T16:38:04.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a15a7640 1 -- 172.21.15.81:0/1485213695 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557a7d4c8d80 con 0x557a7d4c7800 2023-11-08T16:38:04.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a15a7640 1 -- 172.21.15.81:0/1485213695 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x557a7d224420 con 0x557a7d4c7800 2023-11-08T16:38:04.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a9a0f100 1 -- 172.21.15.81:0/1485213695 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557a7d202b60 con 0x557a7d4c7800 2023-11-08T16:38:04.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a9a0f100 1 -- 172.21.15.81:0/1485213695 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a7d2021a0 con 0x557a7d4c7800 2023-11-08T16:38:04.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.216+0000 7f13a9a0f100 4 main: resolving pool 10 2023-11-08T16:38:04.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.218+0000 7f13a15a7640 1 -- 172.21.15.81:0/1485213695 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x557a7d23a840 con 0x557a7d4c7800 2023-11-08T16:38:04.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.218+0000 7f13a15a7640 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x557a7d632000 0x557a7d638000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.219+0000 7f13a15a7640 1 -- 172.21.15.81:0/1485213695 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x557a7d208540 con 0x557a7d4c7800 2023-11-08T16:38:04.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.219+0000 7f13a9a0f100 4 main: creating IoCtx.. 2023-11-08T16:38:04.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.219+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x557a7d632400 0x557a7d24a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.219+0000 7f13a7db4640 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x557a7d632000 0x557a7d638000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.219+0000 7f13a9a0f100 1 -- 172.21.15.81:0/1485213695 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:1 10.d 10:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x557a7d237880 con 0x557a7d632400 2023-11-08T16:38:04.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.219+0000 7f13a7db4640 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x557a7d632000 0x557a7d638000 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x557a7df360f0 tx=0x557a7d4e0cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.219+0000 7f13a75b3640 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x557a7d632400 0x557a7d24a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.220+0000 7f13a75b3640 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x557a7d632400 0x557a7d24a680 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a75b3640 1 -- 172.21.15.81:0/1485213695 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v86'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x557a7dfa8b40 con 0x557a7d632400 2023-11-08T16:38:04.239 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:38:04.239 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:38:04.239 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:38:04.240 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:04.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 -- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x557a7d632400 msgr2=0x557a7d24a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x557a7d632400 0x557a7d24a680 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 -- 172.21.15.81:0/1485213695 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x557a7d632000 msgr2=0x557a7d638000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x557a7d632000 0x557a7d638000 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x557a7df360f0 tx=0x557a7d4e0cc0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 -- 172.21.15.81:0/1485213695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 msgr2=0x557a7d24a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/1485213695 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d4c7800 0x557a7d24a100 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x557a7d536300 tx=0x557a7d4e0c00 comp rx=0 tx=0).stop 2023-11-08T16:38:04.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a75b3640 1 -- 172.21.15.81:0/1485213695 reap_dead start 2023-11-08T16:38:04.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 -- 172.21.15.81:0/1485213695 shutdown_connections 2023-11-08T16:38:04.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 -- 172.21.15.81:0/1485213695 >> 172.21.15.81:0/1485213695 conn(0x557a7d4c6c00 msgr2=0x557a7df67000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 -- 172.21.15.81:0/1485213695 shutdown_connections 2023-11-08T16:38:04.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 -- 172.21.15.81:0/1485213695 wait complete. 2023-11-08T16:38:04.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 -- 172.21.15.81:0/2181087576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d1dd800 msgr2=0x557a7d249600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/2181087576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d1dd800 0x557a7d249600 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x557a7dff6ba0 tx=0x557a7d356ae0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 -- 172.21.15.81:0/2181087576 shutdown_connections 2023-11-08T16:38:04.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/2181087576 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x557a7d1ddc00 0x557a7d249080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/2181087576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x557a7d1dd800 0x557a7d249600 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.225+0000 7f13a9a0f100 1 --2- 172.21.15.81:0/2181087576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a7d1dcc00 0x557a7d249b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.226+0000 7f13a9a0f100 1 -- 172.21.15.81:0/2181087576 >> 172.21.15.81:0/2181087576 conn(0x557a7d1dd000 msgr2=0x557a7df66800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.226+0000 7f13a9a0f100 1 -- 172.21.15.81:0/2181087576 shutdown_connections 2023-11-08T16:38:04.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.226+0000 7f13a9a0f100 1 -- 172.21.15.81:0/2181087576 wait complete. 2023-11-08T16:38:04.246 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.131408, result: { "result": 0 } 2023-11-08T16:38:04.246 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-11-08T16:38:04.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.343+0000 7fb49d23b100 1 -- start start 2023-11-08T16:38:04.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.343+0000 7fb49d23b100 1 Processor -- start 2023-11-08T16:38:04.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.344+0000 7fb49d23b100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e5f408c00 0x558e5f474580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.344+0000 7fb49d23b100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e5f409800 0x558e5f474b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.344+0000 7fb49d23b100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e5f409c00 0x558e5f475080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.344+0000 7fb49d23b100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f453180 con 0x558e5f409c00 2023-11-08T16:38:04.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.344+0000 7fb49d23b100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f452140 con 0x558e5f408c00 2023-11-08T16:38:04.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.344+0000 7fb49d23b100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e5f452000 con 0x558e5f409800 2023-11-08T16:38:04.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.344+0000 7fb49ba88640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e5f409c00 0x558e5f475080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.344+0000 7fb49ba88640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e5f409c00 0x558e5f475080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59406/0 (socket says 172.21.15.81:59406) 2023-11-08T16:38:04.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.344+0000 7fb49ba88640 1 -- 172.21.15.81:0/3464625851 learned_addr learned my addr 172.21.15.81:0/3464625851 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.344+0000 7fb49b287640 1 --2- 172.21.15.81:0/3464625851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e5f408c00 0x558e5f474580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb49aa86640 1 --2- 172.21.15.81:0/3464625851 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e5f409800 0x558e5f474b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb49ba88640 1 -- 172.21.15.81:0/3464625851 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e5f409800 msgr2=0x558e5f474b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb49ba88640 1 --2- 172.21.15.81:0/3464625851 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e5f409800 0x558e5f474b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb49ba88640 1 -- 172.21.15.81:0/3464625851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e5f408c00 msgr2=0x558e5f474580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb49ba88640 1 --2- 172.21.15.81:0/3464625851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e5f408c00 0x558e5f474580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb49ba88640 1 -- 172.21.15.81:0/3464625851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e5f42e9c0 con 0x558e5f409c00 2023-11-08T16:38:04.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb49ba88640 1 --2- 172.21.15.81:0/3464625851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e5f409c00 0x558e5f475080 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x558e6022ab70 tx=0x558e602b6120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8faa0895898d5a9f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb498a82640 1 -- 172.21.15.81:0/3464625851 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e5f450f20 con 0x558e5f409c00 2023-11-08T16:38:04.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb498a82640 1 -- 172.21.15.81:0/3464625851 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558e601e4300 con 0x558e5f409c00 2023-11-08T16:38:04.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb498a82640 1 -- 172.21.15.81:0/3464625851 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e5f4511e0 con 0x558e5f409c00 2023-11-08T16:38:04.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb49d23b100 1 -- 172.21.15.81:0/3464625851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e5f42ed00 con 0x558e5f409c00 2023-11-08T16:38:04.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb49d23b100 1 -- 172.21.15.81:0/3464625851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x558e5f42e9c0 con 0x558e5f409c00 2023-11-08T16:38:04.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.345+0000 7fb49d23b100 4 waiting for MDS map... 2023-11-08T16:38:04.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.346+0000 7fb498a82640 1 -- 172.21.15.81:0/3464625851 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x558e5f434380 con 0x558e5f409c00 2023-11-08T16:38:04.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.346+0000 7fb498a82640 1 -- 172.21.15.81:0/3464625851 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 71) v1 ==== 3306+0+0 (secure 0 0 0) 0x558e5f45b200 con 0x558e5f409c00 2023-11-08T16:38:04.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.346+0000 7fb49d23b100 4 Got MDS map 71 2023-11-08T16:38:04.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.346+0000 7fb49d23b100 10 main: main 2023-11-08T16:38:04.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.346+0000 7fb49d23b100 4 main: connecting to RADOS... 2023-11-08T16:38:04.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.347+0000 7fb49d23b100 1 Processor -- start 2023-11-08T16:38:04.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.347+0000 7fb49d23b100 1 -- start start 2023-11-08T16:38:04.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.347+0000 7fb49d23b100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e601e7c00 0x558e5f475600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49d23b100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e60316000 0x558e5f475b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49d23b100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e60316400 0x558e5f476100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49d23b100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f452a00 con 0x558e60316400 2023-11-08T16:38:04.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49d23b100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f4d7180 con 0x558e601e7c00 2023-11-08T16:38:04.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49d23b100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e5f4d6dc0 con 0x558e60316000 2023-11-08T16:38:04.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49aa86640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e60316400 0x558e5f476100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49aa86640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e60316400 0x558e5f476100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59412/0 (socket says 172.21.15.81:59412) 2023-11-08T16:38:04.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49aa86640 1 -- 172.21.15.81:0/206193542 learned_addr learned my addr 172.21.15.81:0/206193542 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49aa86640 1 -- 172.21.15.81:0/206193542 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e60316000 msgr2=0x558e5f475b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49b287640 1 --2- 172.21.15.81:0/206193542 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e60316000 0x558e5f475b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49aa86640 1 --2- 172.21.15.81:0/206193542 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e60316000 0x558e5f475b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49aa86640 1 -- 172.21.15.81:0/206193542 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e601e7c00 msgr2=0x558e5f475600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49aa86640 1 --2- 172.21.15.81:0/206193542 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e601e7c00 0x558e5f475600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49aa86640 1 -- 172.21.15.81:0/206193542 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e5f42eb60 con 0x558e60316400 2023-11-08T16:38:04.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.348+0000 7fb49aa86640 1 --2- 172.21.15.81:0/206193542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e60316400 0x558e5f476100 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x558e60320120 tx=0x558e601e2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3183148e636992c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb496a7e640 1 -- 172.21.15.81:0/206193542 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e5f450f20 con 0x558e60316400 2023-11-08T16:38:04.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb496a7e640 1 -- 172.21.15.81:0/206193542 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558e5f464900 con 0x558e60316400 2023-11-08T16:38:04.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb496a7e640 1 -- 172.21.15.81:0/206193542 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e5f4511e0 con 0x558e60316400 2023-11-08T16:38:04.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb49d23b100 1 -- 172.21.15.81:0/206193542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e60316400 msgr2=0x558e5f476100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb49d23b100 1 --2- 172.21.15.81:0/206193542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e60316400 0x558e5f476100 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x558e60320120 tx=0x558e601e2c00 comp rx=0 tx=0).stop 2023-11-08T16:38:04.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb49d23b100 1 -- 172.21.15.81:0/206193542 shutdown_connections 2023-11-08T16:38:04.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb49d23b100 1 --2- 172.21.15.81:0/206193542 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e60316400 0x558e5f476100 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb49d23b100 1 --2- 172.21.15.81:0/206193542 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e60316000 0x558e5f475b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb49d23b100 1 --2- 172.21.15.81:0/206193542 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e601e7c00 0x558e5f475600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb49d23b100 1 -- 172.21.15.81:0/206193542 >> 172.21.15.81:0/206193542 conn(0x558e601e7800 msgr2=0x558e60193000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb49d23b100 1 -- 172.21.15.81:0/206193542 shutdown_connections 2023-11-08T16:38:04.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.349+0000 7fb49d23b100 1 -- 172.21.15.81:0/206193542 wait complete. 2023-11-08T16:38:04.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.350+0000 7fb49d23b100 1 Processor -- start 2023-11-08T16:38:04.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.350+0000 7fb49d23b100 1 -- start start 2023-11-08T16:38:04.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.351+0000 7fb49d23b100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e601e7800 0x558e5f477180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.351+0000 7fb49d23b100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e60317000 0x558e5f476c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.351+0000 7fb49d23b100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e60316c00 0x558e5f476680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.351+0000 7fb49d23b100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f452140 con 0x558e601e7800 2023-11-08T16:38:04.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.351+0000 7fb49d23b100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e5f453180 con 0x558e60316c00 2023-11-08T16:38:04.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.351+0000 7fb49d23b100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e5f452b40 con 0x558e60317000 2023-11-08T16:38:04.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.351+0000 7fb49aa86640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e601e7800 0x558e5f477180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.351+0000 7fb49aa86640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e601e7800 0x558e5f477180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59420/0 (socket says 172.21.15.81:59420) 2023-11-08T16:38:04.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.351+0000 7fb49aa86640 1 -- 172.21.15.81:0/4255968551 learned_addr learned my addr 172.21.15.81:0/4255968551 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49b287640 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e60317000 0x558e5f476c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49aa86640 1 -- 172.21.15.81:0/4255968551 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e60317000 msgr2=0x558e5f476c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49aa86640 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e60317000 0x558e5f476c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49aa86640 1 -- 172.21.15.81:0/4255968551 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e60316c00 msgr2=0x558e5f476680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49aa86640 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e60316c00 0x558e5f476680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49aa86640 1 -- 172.21.15.81:0/4255968551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e5f42e820 con 0x558e601e7800 2023-11-08T16:38:04.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49aa86640 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e601e7800 0x558e5f477180 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x558e60320de0 tx=0x558e602b6420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49527b640 1 -- 172.21.15.81:0/4255968551 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e5f450420 con 0x558e601e7800 2023-11-08T16:38:04.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49527b640 1 -- 172.21.15.81:0/4255968551 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558e601e4780 con 0x558e601e7800 2023-11-08T16:38:04.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49527b640 1 -- 172.21.15.81:0/4255968551 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x558e5f450f20 con 0x558e601e7800 2023-11-08T16:38:04.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49d23b100 1 -- 172.21.15.81:0/4255968551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558e5f42e4e0 con 0x558e601e7800 2023-11-08T16:38:04.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49d23b100 1 -- 172.21.15.81:0/4255968551 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e5f42e1a0 con 0x558e601e7800 2023-11-08T16:38:04.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.352+0000 7fb49d23b100 4 main: resolving pool 10 2023-11-08T16:38:04.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.354+0000 7fb49527b640 1 -- 172.21.15.81:0/4255968551 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x558e5f466840 con 0x558e601e7800 2023-11-08T16:38:04.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.355+0000 7fb49527b640 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x558e5f760000 0x558e5f477700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.355+0000 7fb49527b640 1 -- 172.21.15.81:0/4255968551 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x558e5f434540 con 0x558e601e7800 2023-11-08T16:38:04.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.355+0000 7fb49d23b100 4 main: creating IoCtx.. 2023-11-08T16:38:04.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.355+0000 7fb49d23b100 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x558e60316800 0x558e5f476100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.355+0000 7fb49d23b100 1 -- 172.21.15.81:0/4255968551 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:1 10.13 10:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x558e5f463880 con 0x558e60316800 2023-11-08T16:38:04.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.355+0000 7fb49b287640 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x558e5f760000 0x558e5f477700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.355+0000 7fb49aa86640 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x558e60316800 0x558e5f476100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.356+0000 7fb49b287640 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x558e5f760000 0x558e5f477700 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x558e601620f0 tx=0x558e601e3da0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.356+0000 7fb49aa86640 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x558e60316800 0x558e5f476100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.382 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:38:04.383 INFO:teuthology.orchestra.run.smithi081.stdout: "0": { 2023-11-08T16:38:04.383 INFO:teuthology.orchestra.run.smithi081.stdout: "data": {}, 2023-11-08T16:38:04.383 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:38:04.383 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:38:04.384 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:38:04.385 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:04.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.359+0000 7fb49aa86640 1 -- 172.21.15.81:0/4255968551 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v86'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x558e6016ab40 con 0x558e60316800 2023-11-08T16:38:04.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 -- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x558e60316800 msgr2=0x558e5f476100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x558e60316800 0x558e5f476100 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 -- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x558e5f760000 msgr2=0x558e5f477700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x558e5f760000 0x558e5f477700 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x558e601620f0 tx=0x558e601e3da0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 -- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e601e7800 msgr2=0x558e5f477180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 --2- 172.21.15.81:0/4255968551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e601e7800 0x558e5f477180 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x558e60320de0 tx=0x558e602b6420 comp rx=0 tx=0).stop 2023-11-08T16:38:04.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49b287640 1 -- 172.21.15.81:0/4255968551 reap_dead start 2023-11-08T16:38:04.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 -- 172.21.15.81:0/4255968551 shutdown_connections 2023-11-08T16:38:04.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 -- 172.21.15.81:0/4255968551 >> 172.21.15.81:0/4255968551 conn(0x558e601e7c00 msgr2=0x558e60193000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 -- 172.21.15.81:0/4255968551 shutdown_connections 2023-11-08T16:38:04.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 -- 172.21.15.81:0/4255968551 wait complete. 2023-11-08T16:38:04.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 -- 172.21.15.81:0/3464625851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e5f409c00 msgr2=0x558e5f475080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 --2- 172.21.15.81:0/3464625851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e5f409c00 0x558e5f475080 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x558e6022ab70 tx=0x558e602b6120 comp rx=0 tx=0).stop 2023-11-08T16:38:04.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.360+0000 7fb49d23b100 1 -- 172.21.15.81:0/3464625851 shutdown_connections 2023-11-08T16:38:04.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.361+0000 7fb49d23b100 1 --2- 172.21.15.81:0/3464625851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x558e5f409c00 0x558e5f475080 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.361+0000 7fb49d23b100 1 --2- 172.21.15.81:0/3464625851 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x558e5f409800 0x558e5f474b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.361+0000 7fb49d23b100 1 --2- 172.21.15.81:0/3464625851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558e5f408c00 0x558e5f474580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.361+0000 7fb49d23b100 1 -- 172.21.15.81:0/3464625851 >> 172.21.15.81:0/3464625851 conn(0x558e5f409000 msgr2=0x558e60192800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.361+0000 7fb49d23b100 1 -- 172.21.15.81:0/3464625851 shutdown_connections 2023-11-08T16:38:04.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.361+0000 7fb49d23b100 1 -- 172.21.15.81:0/3464625851 wait complete. 2023-11-08T16:38:04.393 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.146470, result: { "0": { "data": {}, "result": 0 } } 2023-11-08T16:38:04.393 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-11-08T16:38:04.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.502+0000 7fc631b33100 1 -- start start 2023-11-08T16:38:04.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.502+0000 7fc631b33100 1 Processor -- start 2023-11-08T16:38:04.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.503+0000 7fc631b33100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecec26c00 0x55cecec92580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.503+0000 7fc631b33100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecec27800 0x55cecec92b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.503+0000 7fc631b33100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecec27c00 0x55cecec93080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.503+0000 7fc631b33100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cecec71180 con 0x55cecec26c00 2023-11-08T16:38:04.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.503+0000 7fc631b33100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cecec70140 con 0x55cecec27c00 2023-11-08T16:38:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.503+0000 7fc631b33100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cecec70000 con 0x55cecec27800 2023-11-08T16:38:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.503+0000 7fc6302db640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecec26c00 0x55cecec92580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.503+0000 7fc6302db640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecec26c00 0x55cecec92580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59426/0 (socket says 172.21.15.81:59426) 2023-11-08T16:38:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.503+0000 7fc6302db640 1 -- 172.21.15.81:0/658004031 learned_addr learned my addr 172.21.15.81:0/658004031 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc630adc640 1 --2- 172.21.15.81:0/658004031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecec27c00 0x55cecec93080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc6302db640 1 -- 172.21.15.81:0/658004031 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecec27800 msgr2=0x55cecec92b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc6302db640 1 --2- 172.21.15.81:0/658004031 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecec27800 0x55cecec92b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc6302db640 1 -- 172.21.15.81:0/658004031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecec27c00 msgr2=0x55cecec93080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc6302db640 1 --2- 172.21.15.81:0/658004031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecec27c00 0x55cecec93080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc6302db640 1 -- 172.21.15.81:0/658004031 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cecec4c9c0 con 0x55cecec26c00 2023-11-08T16:38:04.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc6302db640 1 --2- 172.21.15.81:0/658004031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecec26c00 0x55cecec92580 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x55cecfa44870 tx=0x55cecf9fa360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823f084d677a1e66 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc62dad6640 1 -- 172.21.15.81:0/658004031 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cecec6ef20 con 0x55cecec26c00 2023-11-08T16:38:04.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc62dad6640 1 -- 172.21.15.81:0/658004031 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cecfa0c300 con 0x55cecec26c00 2023-11-08T16:38:04.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc631b33100 1 -- 172.21.15.81:0/658004031 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cecec4cd00 con 0x55cecec26c00 2023-11-08T16:38:04.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc631b33100 1 -- 172.21.15.81:0/658004031 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55cecec4c9c0 con 0x55cecec26c00 2023-11-08T16:38:04.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc631b33100 4 waiting for MDS map... 2023-11-08T16:38:04.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.504+0000 7fc62dad6640 1 -- 172.21.15.81:0/658004031 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cecec6f1e0 con 0x55cecec26c00 2023-11-08T16:38:04.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.505+0000 7fc62dad6640 1 -- 172.21.15.81:0/658004031 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x55cecec52380 con 0x55cecec26c00 2023-11-08T16:38:04.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.505+0000 7fc62dad6640 1 -- 172.21.15.81:0/658004031 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 71) v1 ==== 3306+0+0 (secure 0 0 0) 0x55cecec78f00 con 0x55cecec26c00 2023-11-08T16:38:04.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.505+0000 7fc631b33100 4 Got MDS map 71 2023-11-08T16:38:04.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.505+0000 7fc631b33100 10 main: JournalTool::main 2023-11-08T16:38:04.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.505+0000 7fc631b33100 4 main: JournalTool: connecting to RADOS... 2023-11-08T16:38:04.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.506+0000 7fc631b33100 1 Processor -- start 2023-11-08T16:38:04.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.506+0000 7fc631b33100 1 -- start start 2023-11-08T16:38:04.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.506+0000 7fc631b33100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfa0fc00 0x55cecec93600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc631b33100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48000 0x55cecec93b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc631b33100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfb48400 0x55cecec94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc631b33100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cecec70a00 con 0x55cecfb48400 2023-11-08T16:38:04.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc631b33100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cececf5180 con 0x55cecfa0fc00 2023-11-08T16:38:04.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc631b33100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cececf4dc0 con 0x55cecfb48000 2023-11-08T16:38:04.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc62fada640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfb48400 0x55cecec94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc62fada640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfb48400 0x55cecec94100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59440/0 (socket says 172.21.15.81:59440) 2023-11-08T16:38:04.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc62fada640 1 -- 172.21.15.81:0/737016466 learned_addr learned my addr 172.21.15.81:0/737016466 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc630adc640 1 --2- 172.21.15.81:0/737016466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfa0fc00 0x55cecec93600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc62fada640 1 -- 172.21.15.81:0/737016466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48000 msgr2=0x55cecec93b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc62fada640 1 --2- 172.21.15.81:0/737016466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48000 0x55cecec93b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc62fada640 1 -- 172.21.15.81:0/737016466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfa0fc00 msgr2=0x55cecec93600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc62fada640 1 --2- 172.21.15.81:0/737016466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfa0fc00 0x55cecec93600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.507+0000 7fc62fada640 1 -- 172.21.15.81:0/737016466 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cecec4cb60 con 0x55cecfb48400 2023-11-08T16:38:04.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc62fada640 1 --2- 172.21.15.81:0/737016466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfb48400 0x55cecec94100 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x55cecfb520c0 tx=0x55cecf97ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb37915a0d74e5d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc62bad2640 1 -- 172.21.15.81:0/737016466 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cecec6ef20 con 0x55cecfb48400 2023-11-08T16:38:04.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc62bad2640 1 -- 172.21.15.81:0/737016466 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cecfa0d080 con 0x55cecfb48400 2023-11-08T16:38:04.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc62bad2640 1 -- 172.21.15.81:0/737016466 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cecec6f1e0 con 0x55cecfb48400 2023-11-08T16:38:04.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc631b33100 1 -- 172.21.15.81:0/737016466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfb48400 msgr2=0x55cecec94100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc631b33100 1 --2- 172.21.15.81:0/737016466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfb48400 0x55cecec94100 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x55cecfb520c0 tx=0x55cecf97ec00 comp rx=0 tx=0).stop 2023-11-08T16:38:04.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc631b33100 1 -- 172.21.15.81:0/737016466 shutdown_connections 2023-11-08T16:38:04.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc631b33100 1 --2- 172.21.15.81:0/737016466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfb48400 0x55cecec94100 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc631b33100 1 --2- 172.21.15.81:0/737016466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48000 0x55cecec93b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc631b33100 1 --2- 172.21.15.81:0/737016466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfa0fc00 0x55cecec93600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc631b33100 1 -- 172.21.15.81:0/737016466 >> 172.21.15.81:0/737016466 conn(0x55cecfa0f800 msgr2=0x55cecf9a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc631b33100 1 -- 172.21.15.81:0/737016466 shutdown_connections 2023-11-08T16:38:04.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.508+0000 7fc631b33100 1 -- 172.21.15.81:0/737016466 wait complete. 2023-11-08T16:38:04.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.509+0000 7fc631b33100 1 Processor -- start 2023-11-08T16:38:04.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.509+0000 7fc631b33100 1 -- start start 2023-11-08T16:38:04.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.510+0000 7fc631b33100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfa0f800 0x55cecec95180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.510+0000 7fc631b33100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfb49000 0x55cecec94c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.510+0000 7fc631b33100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48c00 0x55cecec94680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.510+0000 7fc631b33100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cecec70140 con 0x55cecfa0f800 2023-11-08T16:38:04.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.510+0000 7fc631b33100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cecec71180 con 0x55cecfb49000 2023-11-08T16:38:04.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.510+0000 7fc631b33100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cecec70b40 con 0x55cecfb48c00 2023-11-08T16:38:04.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.510+0000 7fc630adc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfa0f800 0x55cecec95180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.510+0000 7fc630adc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfa0f800 0x55cecec95180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59456/0 (socket says 172.21.15.81:59456) 2023-11-08T16:38:04.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.510+0000 7fc630adc640 1 -- 172.21.15.81:0/3648406211 learned_addr learned my addr 172.21.15.81:0/3648406211 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc62fada640 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48c00 0x55cecec94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc6302db640 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfb49000 0x55cecec94c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc630adc640 1 -- 172.21.15.81:0/3648406211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48c00 msgr2=0x55cecec94680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc630adc640 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48c00 0x55cecec94680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc630adc640 1 -- 172.21.15.81:0/3648406211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfb49000 msgr2=0x55cecec94c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc630adc640 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfb49000 0x55cecec94c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc630adc640 1 -- 172.21.15.81:0/3648406211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cecec4c820 con 0x55cecfa0f800 2023-11-08T16:38:04.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc630adc640 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfa0f800 0x55cecec95180 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x55cecfb53350 tx=0x55cecf97ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc62a2cf640 1 -- 172.21.15.81:0/3648406211 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cecec6ef20 con 0x55cecfa0f800 2023-11-08T16:38:04.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc62a2cf640 1 -- 172.21.15.81:0/3648406211 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cecec82180 con 0x55cecfa0f800 2023-11-08T16:38:04.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc631b33100 1 -- 172.21.15.81:0/3648406211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55cecec4c4e0 con 0x55cecfa0f800 2023-11-08T16:38:04.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc62a2cf640 1 -- 172.21.15.81:0/3648406211 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cecec6f1e0 con 0x55cecfa0f800 2023-11-08T16:38:04.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc631b33100 1 -- 172.21.15.81:0/3648406211 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cecec4c1a0 con 0x55cecfa0f800 2023-11-08T16:38:04.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.511+0000 7fc631b33100 4 main: JournalTool: resolving pool 10 2023-11-08T16:38:04.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.513+0000 7fc62a2cf640 1 -- 172.21.15.81:0/3648406211 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55cececb7080 con 0x55cecfa0f800 2023-11-08T16:38:04.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.514+0000 7fc62a2cf640 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cecfb66c00 0x55cecec95700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.514+0000 7fc62a2cf640 1 -- 172.21.15.81:0/3648406211 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x55cecec52540 con 0x55cecfa0f800 2023-11-08T16:38:04.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.514+0000 7fc631b33100 4 main: JournalTool: creating IoCtx.. 2023-11-08T16:38:04.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.514+0000 7fc631b33100 4 main: Executing for rank 0 2023-11-08T16:38:04.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.514+0000 7fc6302db640 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cecfb66c00 0x55cecec95700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.514+0000 7fc6302db640 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cecfb66c00 0x55cecec95700 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55cecfa45170 tx=0x55cecf9fa420 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.515+0000 7fc631b33100 1 -- start start 2023-11-08T16:38:04.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.515+0000 7fc631b33100 1 Processor -- start 2023-11-08T16:38:04.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.516+0000 7fc631b33100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48400 0x55cecec94100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.516+0000 7fc631b33100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfb48000 0x55cecec93b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.516+0000 7fc631b33100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecefea000 0x55cecec93600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.516+0000 7fc631b33100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cececf4dc0 con 0x55cecefea000 2023-11-08T16:38:04.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.516+0000 7fc631b33100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cececf5180 con 0x55cecfb48000 2023-11-08T16:38:04.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.516+0000 7fc631b33100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cececf4c80 con 0x55cecfb48400 2023-11-08T16:38:04.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.516+0000 7fc62fada640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecefea000 0x55cecec93600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.516+0000 7fc62fada640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecefea000 0x55cecec93600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59466/0 (socket says 172.21.15.81:59466) 2023-11-08T16:38:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.516+0000 7fc62fada640 1 -- 172.21.15.81:0/3714570874 learned_addr learned my addr 172.21.15.81:0/3714570874 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc6302db640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfb48000 0x55cecec93b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc630adc640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48400 0x55cecec94100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc62fada640 1 -- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48400 msgr2=0x55cecec94100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc62fada640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48400 0x55cecec94100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc62fada640 1 -- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfb48000 msgr2=0x55cecec93b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc62fada640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfb48000 0x55cecec93b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc62fada640 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cecfb931e0 con 0x55cecefea000 2023-11-08T16:38:04.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc6302db640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfb48000 0x55cecec93b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc630adc640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48400 0x55cecec94100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc62fada640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecefea000 0x55cecec93600 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x55cecef51140 tx=0x55cecf9fa1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff36f2ed64359913 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc6272c9640 1 -- 172.21.15.81:0/3714570874 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cecec6e420 con 0x55cecefea000 2023-11-08T16:38:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc6272c9640 1 -- 172.21.15.81:0/3714570874 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55cecfa0c480 con 0x55cecefea000 2023-11-08T16:38:04.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc6272c9640 1 -- 172.21.15.81:0/3714570874 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55cecec6e9a0 con 0x55cecefea000 2023-11-08T16:38:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cecfb93040 con 0x55cecefea000 2023-11-08T16:38:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55cecfb92ea0 con 0x55cecefea000 2023-11-08T16:38:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc631b33100 4 waiting for MDS map... 2023-11-08T16:38:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.517+0000 7fc6272c9640 1 -- 172.21.15.81:0/3714570874 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x55cecec52a80 con 0x55cecefea000 2023-11-08T16:38:04.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.518+0000 7fc6272c9640 1 -- 172.21.15.81:0/3714570874 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 71) v1 ==== 3306+0+0 (secure 0 0 0) 0x55cecefdcf00 con 0x55cecefea000 2023-11-08T16:38:04.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.518+0000 7fc631b33100 4 Got MDS map 71 2023-11-08T16:38:04.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.518+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55cecefea400 0x55cecf002000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.518+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:1 10.f 10:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55cecf006000 con 0x55cecefea400 2023-11-08T16:38:04.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.518+0000 7fc630adc640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55cecefea400 0x55cecf002000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.519+0000 7fc630adc640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55cecefea400 0x55cecf002000 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=efe57fc548bcc7cd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.522+0000 7fc630adc640 1 -- 172.21.15.81:0/3714570874 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v86'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55cecfadeb40 con 0x55cecefea400 2023-11-08T16:38:04.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.522+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55cecefea800 0x55cecf002580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.522+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:2 10.14 10:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55cecf006380 con 0x55cecefea800 2023-11-08T16:38:04.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.523+0000 7fc6302db640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55cecefea800 0x55cecf002580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.523+0000 7fc6302db640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55cecefea800 0x55cecf002580 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=c4f9ce72c4dd9f22 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc6302db640 1 -- 172.21.15.81:0/3714570874 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v86'26 uv26 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55cecfaded80 con 0x55cecefea800 2023-11-08T16:38:04.547 INFO:teuthology.orchestra.run.smithi081.stdout:writing EResetJournal entry 2023-11-08T16:38:04.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:3 10.14 10:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55cecf006700 con 0x55cecefea800 2023-11-08T16:38:04.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55cecefeac00 0x55cecf002b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:4 10.1b 10:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55cecf006a80 con 0x55cecefeac00 2023-11-08T16:38:04.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:5 10.b 10:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55cecf006e00 con 0x55cecefea400 2023-11-08T16:38:04.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55cecefeb000 0x55cecf003080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:6 10.1f 10:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55cecf007180 con 0x55cecefeb000 2023-11-08T16:38:04.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:7 10.1a 10:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55cecf007500 con 0x55cecefea400 2023-11-08T16:38:04.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55cecefeb400 0x55cecf003600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:8 10.1e 10:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55cecf007880 con 0x55cecefeb400 2023-11-08T16:38:04.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc630adc640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55cecefeac00 0x55cecf002b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc6302db640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55cecefeb000 0x55cecf003080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.525+0000 7fc62fada640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55cecefeb400 0x55cecf003600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.526+0000 7fc630adc640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55cecefeac00 0x55cecf002b00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=828f1bbef89313ad server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.526+0000 7fc62fada640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55cecefeb400 0x55cecf003600 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=8d5ba3604119b64a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.526+0000 7fc6302db640 1 -- 172.21.15.81:0/3714570874 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(3 200.00000001 [delete] v86'27 uv26 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55cecfaded80 con 0x55cecefea800 2023-11-08T16:38:04.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.527+0000 7fc6302db640 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55cecefeb000 0x55cecf003080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=cabacf42b2125129 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.528+0000 7fc62fada640 1 -- 172.21.15.81:0/3714570874 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(8 200.00000006 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55cecfadf440 con 0x55cecefeb400 2023-11-08T16:38:04.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.528+0000 7fc630adc640 1 -- 172.21.15.81:0/3714570874 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(5 200.00000003 [delete] v86'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55cecfadeb40 con 0x55cecefea400 2023-11-08T16:38:04.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.529+0000 7fc630adc640 1 -- 172.21.15.81:0/3714570874 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(4 200.00000002 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55cecfadeb40 con 0x55cecefeac00 2023-11-08T16:38:04.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.529+0000 7fc625ac6640 1 -- 172.21.15.81:0/3714570874 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:9 10.14 10:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55cecf006380 con 0x55cecefea800 2023-11-08T16:38:04.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.529+0000 7fc630adc640 1 -- 172.21.15.81:0/3714570874 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(7 200.00000005 [delete] v86'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55cecfadeb40 con 0x55cecefea400 2023-11-08T16:38:04.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc6302db640 1 -- 172.21.15.81:0/3714570874 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(6 200.00000004 [delete] v86'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55cecfaded80 con 0x55cecefeb000 2023-11-08T16:38:04.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc6302db640 1 -- 172.21.15.81:0/3714570874 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v86'28 uv28 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55cecfaded80 con 0x55cecefea800 2023-11-08T16:38:04.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 4 Successfully wrote new journal pointer and header for rank 4:0 2023-11-08T16:38:04.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55cecefeb400 msgr2=0x55cecf003600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55cecefeb400 0x55cecf003600 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55cecefea800 msgr2=0x55cecf002580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55cecefea800 0x55cecf002580 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55cecefeac00 msgr2=0x55cecf002b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55cecefeac00 0x55cecf002b00 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55cecefea400 msgr2=0x55cecf002000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55cecefea400 0x55cecf002000 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55cecefeb000 msgr2=0x55cecf003080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55cecefeb000 0x55cecf003080 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc62fada640 1 -- 172.21.15.81:0/3714570874 reap_dead start 2023-11-08T16:38:04.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecefea000 msgr2=0x55cecec93600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.530+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecefea000 0x55cecec93600 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x55cecef51140 tx=0x55cecf9fa1e0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 shutdown_connections 2023-11-08T16:38:04.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 --2- 172.21.15.81:0/3714570874 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecefea000 0x55cecec93600 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 >> 172.21.15.81:0/3714570874 conn(0x55cecfb48800 msgr2=0x55cecf9a3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 shutdown_connections 2023-11-08T16:38:04.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 -- 172.21.15.81:0/3714570874 wait complete. 2023-11-08T16:38:04.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 -- 172.21.15.81:0/3648406211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cecfb66c00 msgr2=0x55cecec95700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cecfb66c00 0x55cecec95700 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x55cecfa45170 tx=0x55cecf9fa420 comp rx=0 tx=0).stop 2023-11-08T16:38:04.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 -- 172.21.15.81:0/3648406211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfa0f800 msgr2=0x55cecec95180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfa0f800 0x55cecec95180 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x55cecfb53350 tx=0x55cecf97ede0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 -- 172.21.15.81:0/3648406211 shutdown_connections 2023-11-08T16:38:04.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55cecfb66c00 0x55cecec95700 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecfb48c00 0x55cecec94680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecfb49000 0x55cecec94c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 --2- 172.21.15.81:0/3648406211 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecfa0f800 0x55cecec95180 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 -- 172.21.15.81:0/3648406211 >> 172.21.15.81:0/3648406211 conn(0x55cecfa0fc00 msgr2=0x55cecf9a1800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 -- 172.21.15.81:0/3648406211 shutdown_connections 2023-11-08T16:38:04.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.531+0000 7fc631b33100 1 -- 172.21.15.81:0/3648406211 wait complete. 2023-11-08T16:38:04.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.532+0000 7fc631b33100 1 -- 172.21.15.81:0/658004031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecec26c00 msgr2=0x55cecec92580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.532+0000 7fc631b33100 1 --2- 172.21.15.81:0/658004031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecec26c00 0x55cecec92580 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x55cecfa44870 tx=0x55cecf9fa360 comp rx=0 tx=0).stop 2023-11-08T16:38:04.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.532+0000 7fc631b33100 1 -- 172.21.15.81:0/658004031 shutdown_connections 2023-11-08T16:38:04.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.532+0000 7fc631b33100 1 --2- 172.21.15.81:0/658004031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55cecec27c00 0x55cecec93080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.532+0000 7fc631b33100 1 --2- 172.21.15.81:0/658004031 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55cecec27800 0x55cecec92b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.532+0000 7fc631b33100 1 --2- 172.21.15.81:0/658004031 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55cecec26c00 0x55cecec92580 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.532+0000 7fc631b33100 1 -- 172.21.15.81:0/658004031 >> 172.21.15.81:0/658004031 conn(0x55cecec27000 msgr2=0x55cecf9a1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.532+0000 7fc631b33100 1 -- 172.21.15.81:0/658004031 shutdown_connections 2023-11-08T16:38:04.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.532+0000 7fc631b33100 1 -- 172.21.15.81:0/658004031 wait complete. 2023-11-08T16:38:04.565 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.171539, result: writing EResetJournal entry 2023-11-08T16:38:04.565 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-11-08T16:38:04.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.662+0000 7f07fb0305c0 1 -- start start 2023-11-08T16:38:04.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.662+0000 7f07fb0305c0 1 Processor -- start 2023-11-08T16:38:04.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.663+0000 7f07fb0305c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a780ac00 0x5618a7875b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.663+0000 7f07fb0305c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a780b800 0x5618a7875600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.663+0000 7f07fb0305c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a780bc00 0x5618a7875080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.663+0000 7f07fb0305c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618a7852000 con 0x5618a780ac00 2023-11-08T16:38:04.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.663+0000 7f07fb0305c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618a7852a00 con 0x5618a780bc00 2023-11-08T16:38:04.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.663+0000 7f07fb0305c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618a7852b40 con 0x5618a780b800 2023-11-08T16:38:04.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.663+0000 7f07f88e7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a780ac00 0x5618a7875b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.663+0000 7f07f88e7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a780ac00 0x5618a7875b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59472/0 (socket says 172.21.15.81:59472) 2023-11-08T16:38:04.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.663+0000 7f07f88e7640 1 -- 172.21.15.81:0/2578106486 learned_addr learned my addr 172.21.15.81:0/2578106486 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.663+0000 7f07f80e6640 1 --2- 172.21.15.81:0/2578106486 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a780b800 0x5618a7875600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07f90e8640 1 --2- 172.21.15.81:0/2578106486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a780bc00 0x5618a7875080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07f88e7640 1 -- 172.21.15.81:0/2578106486 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a780b800 msgr2=0x5618a7875600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07f88e7640 1 --2- 172.21.15.81:0/2578106486 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a780b800 0x5618a7875600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07f88e7640 1 -- 172.21.15.81:0/2578106486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a780bc00 msgr2=0x5618a7875080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07f88e7640 1 --2- 172.21.15.81:0/2578106486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a780bc00 0x5618a7875080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07f88e7640 1 -- 172.21.15.81:0/2578106486 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618a78309c0 con 0x5618a780ac00 2023-11-08T16:38:04.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07f88e7640 1 --2- 172.21.15.81:0/2578106486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a780ac00 0x5618a7875b80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x5618a861b830 tx=0x5618a85e2600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=742d02d19d75c478 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07f60e2640 1 -- 172.21.15.81:0/2578106486 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5618a78511e0 con 0x5618a780ac00 2023-11-08T16:38:04.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07f60e2640 1 -- 172.21.15.81:0/2578106486 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5618a86c2000 con 0x5618a780ac00 2023-11-08T16:38:04.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07f60e2640 1 -- 172.21.15.81:0/2578106486 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5618a7851080 con 0x5618a780ac00 2023-11-08T16:38:04.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2578106486 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618a7830b60 con 0x5618a780ac00 2023-11-08T16:38:04.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2578106486 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5618a7830820 con 0x5618a780ac00 2023-11-08T16:38:04.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.664+0000 7f07fb0305c0 4 waiting for MDS map... 2023-11-08T16:38:04.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.665+0000 7f07f60e2640 1 -- 172.21.15.81:0/2578106486 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x5618a78cac40 con 0x5618a780ac00 2023-11-08T16:38:04.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.665+0000 7f07f60e2640 1 -- 172.21.15.81:0/2578106486 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 71) v1 ==== 3306+0+0 (secure 0 0 0) 0x5618a785af00 con 0x5618a780ac00 2023-11-08T16:38:04.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.665+0000 7f07fb0305c0 4 Got MDS map 71 2023-11-08T16:38:04.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.665+0000 7f07fb0305c0 4 datascan.main: Using metadata pool output 2023-11-08T16:38:04.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.665+0000 7f07fb0305c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:38:04.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.666+0000 7f07fb0305c0 1 Processor -- start 2023-11-08T16:38:04.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.666+0000 7f07fb0305c0 1 -- start start 2023-11-08T16:38:04.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07fb0305c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1000 0x5618a7874b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07fb0305c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a86c1400 0x5618a7874580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07fb0305c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a86c1800 0x5618a7876100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07fb0305c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618a78d8dc0 con 0x5618a86c1000 2023-11-08T16:38:04.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07fb0305c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618a78d8c80 con 0x5618a86c1400 2023-11-08T16:38:04.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07fb0305c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618a78d8b40 con 0x5618a86c1800 2023-11-08T16:38:04.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07f90e8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1000 0x5618a7874b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07f90e8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1000 0x5618a7874b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59486/0 (socket says 172.21.15.81:59486) 2023-11-08T16:38:04.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07f90e8640 1 -- 172.21.15.81:0/3116174220 learned_addr learned my addr 172.21.15.81:0/3116174220 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07f88e7640 1 --2- 172.21.15.81:0/3116174220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a86c1400 0x5618a7874580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07f90e8640 1 -- 172.21.15.81:0/3116174220 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a86c1800 msgr2=0x5618a7876100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07f90e8640 1 --2- 172.21.15.81:0/3116174220 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a86c1800 0x5618a7876100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07f90e8640 1 -- 172.21.15.81:0/3116174220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a86c1400 msgr2=0x5618a7874580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07f90e8640 1 --2- 172.21.15.81:0/3116174220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a86c1400 0x5618a7874580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.667+0000 7f07f90e8640 1 -- 172.21.15.81:0/3116174220 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618a78304e0 con 0x5618a86c1000 2023-11-08T16:38:04.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.668+0000 7f07f90e8640 1 --2- 172.21.15.81:0/3116174220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1000 0x5618a7874b00 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x5618a861ab70 tx=0x5618a85e23c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd99c0427bd66a65 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.668+0000 7f07f40de640 1 -- 172.21.15.81:0/3116174220 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5618a7850f20 con 0x5618a86c1000 2023-11-08T16:38:04.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.668+0000 7f07f40de640 1 -- 172.21.15.81:0/3116174220 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5618a86c2f00 con 0x5618a86c1000 2023-11-08T16:38:04.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.668+0000 7f07f40de640 1 -- 172.21.15.81:0/3116174220 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5618a78511e0 con 0x5618a86c1000 2023-11-08T16:38:04.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.668+0000 7f07fb0305c0 1 -- 172.21.15.81:0/3116174220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1000 msgr2=0x5618a7874b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.668+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/3116174220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1000 0x5618a7874b00 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x5618a861ab70 tx=0x5618a85e23c0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.668+0000 7f07fb0305c0 1 -- 172.21.15.81:0/3116174220 shutdown_connections 2023-11-08T16:38:04.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.668+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/3116174220 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a86c1800 0x5618a7876100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.668+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/3116174220 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a86c1400 0x5618a7874580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.668+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/3116174220 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1000 0x5618a7874b00 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.668+0000 7f07fb0305c0 1 -- 172.21.15.81:0/3116174220 >> 172.21.15.81:0/3116174220 conn(0x5618a86c0c00 msgr2=0x5618a8587000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.669+0000 7f07fb0305c0 1 -- 172.21.15.81:0/3116174220 shutdown_connections 2023-11-08T16:38:04.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.669+0000 7f07fb0305c0 1 -- 172.21.15.81:0/3116174220 wait complete. 2023-11-08T16:38:04.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.669+0000 7f07fb0305c0 1 Processor -- start 2023-11-08T16:38:04.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.669+0000 7f07fb0305c0 1 -- start start 2023-11-08T16:38:04.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.670+0000 7f07fb0305c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a86c1000 0x5618a7874b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.670+0000 7f07fb0305c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a86c1400 0x5618a7874580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.670+0000 7f07fb0305c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1800 0x5618a7876100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.670+0000 7f07fb0305c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618a78d8a00 con 0x5618a86c1800 2023-11-08T16:38:04.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.670+0000 7f07fb0305c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5618a7852a00 con 0x5618a86c1400 2023-11-08T16:38:04.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.670+0000 7f07fb0305c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5618a78d9180 con 0x5618a86c1000 2023-11-08T16:38:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.670+0000 7f07f80e6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1800 0x5618a7876100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f80e6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1800 0x5618a7876100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59500/0 (socket says 172.21.15.81:59500) 2023-11-08T16:38:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f80e6640 1 -- 172.21.15.81:0/2688951014 learned_addr learned my addr 172.21.15.81:0/2688951014 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f90e8640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a86c1000 0x5618a7874b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f88e7640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a86c1400 0x5618a7874580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f80e6640 1 -- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a86c1000 msgr2=0x5618a7874b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f80e6640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a86c1000 0x5618a7874b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f80e6640 1 -- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a86c1400 msgr2=0x5618a7874580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f80e6640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a86c1400 0x5618a7874580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f80e6640 1 -- 172.21.15.81:0/2688951014 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5618a7830d00 con 0x5618a86c1800 2023-11-08T16:38:04.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f88e7640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a86c1400 0x5618a7874580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:04.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f80e6640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1800 0x5618a7876100 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x5618a86c97a0 tx=0x5618a86e8c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f28db640 1 -- 172.21.15.81:0/2688951014 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5618a7850f20 con 0x5618a86c1800 2023-11-08T16:38:04.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07f28db640 1 -- 172.21.15.81:0/2688951014 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5618a86c3800 con 0x5618a86c1800 2023-11-08T16:38:04.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.671+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5618a78304e0 con 0x5618a86c1800 2023-11-08T16:38:04.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.672+0000 7f07f28db640 1 -- 172.21.15.81:0/2688951014 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5618a78511e0 con 0x5618a86c1800 2023-11-08T16:38:04.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.672+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5618a78309c0 con 0x5618a86c1800 2023-11-08T16:38:04.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.672+0000 7f07fb0305c0 4 datascan.init: resolving metadata pool 10 2023-11-08T16:38:04.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.673+0000 7f07f28db640 1 -- 172.21.15.81:0/2688951014 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x5618a7867340 con 0x5618a86c1800 2023-11-08T16:38:04.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.674+0000 7f07f28db640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5618a8570800 0x5618a7b60000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.674+0000 7f07f28db640 1 -- 172.21.15.81:0/2688951014 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x5618a78caa80 con 0x5618a86c1800 2023-11-08T16:38:04.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.674+0000 7f07fb0305c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:38:04.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.674+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5618a8570c00 0x5618a7876680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.674+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:1 10.14 10:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x5618a7863880 con 0x5618a8570c00 2023-11-08T16:38:04.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.674+0000 7f07f90e8640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5618a8570800 0x5618a7b60000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.674+0000 7f07f88e7640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5618a8570c00 0x5618a7876680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.675+0000 7f07f90e8640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5618a8570800 0x5618a7b60000 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x5618a861ab70 tx=0x5618a85e2000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.675+0000 7f07f88e7640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5618a8570c00 0x5618a7876680 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.675+0000 7f07f88e7640 1 -- 172.21.15.81:0/2688951014 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x5618a85e6b40 con 0x5618a8570c00 2023-11-08T16:38:04.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.675+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:2 10.14 10:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x5618a7863c00 con 0x5618a8570c00 2023-11-08T16:38:04.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.677+0000 7f07f88e7640 1 -- 172.21.15.81:0/2688951014 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v86'29 uv29 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x5618a85e6b40 con 0x5618a8570c00 2023-11-08T16:38:04.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.677+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5618a8571000 0x5618a7876c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.677+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:3 10.1f 10:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x5618a7b74000 con 0x5618a8571000 2023-11-08T16:38:04.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.678+0000 7f07f90e8640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5618a8571000 0x5618a7876c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.678+0000 7f07f90e8640 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5618a8571000 0x5618a7876c00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.679+0000 7f07f90e8640 1 -- 172.21.15.81:0/2688951014 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x5618a85e7200 con 0x5618a8571000 2023-11-08T16:38:04.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.679+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:4 10.1f 10:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x5618a7b74380 con 0x5618a8571000 2023-11-08T16:38:04.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.682+0000 7f07f90e8640 1 -- 172.21.15.81:0/2688951014 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v86'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x5618a85e7200 con 0x5618a8571000 2023-11-08T16:38:04.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.682+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:5 10.13 10:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x5618a7b74700 con 0x5618a8570c00 2023-11-08T16:38:04.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.683+0000 7f07f88e7640 1 -- 172.21.15.81:0/2688951014 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5618a85e6b40 con 0x5618a8570c00 2023-11-08T16:38:04.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.683+0000 7f07fb0305c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:38:04.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.683+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:6 10.13 10:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x5618a7b74a80 con 0x5618a8570c00 2023-11-08T16:38:04.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07f88e7640 1 -- 172.21.15.81:0/2688951014 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v86'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x5618a85e6b40 con 0x5618a8570c00 2023-11-08T16:38:04.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-11-08T16:38:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5618a8570c00 msgr2=0x5618a7876680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x5618a8570c00 0x5618a7876680 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5618a8571000 msgr2=0x5618a7876c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x5618a8571000 0x5618a7876c00 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5618a8570800 msgr2=0x5618a7b60000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5618a8570800 0x5618a7b60000 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x5618a861ab70 tx=0x5618a85e2000 comp rx=0 tx=0).stop 2023-11-08T16:38:04.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1800 msgr2=0x5618a7876100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/2688951014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a86c1800 0x5618a7876100 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x5618a86c97a0 tx=0x5618a86e8c00 comp rx=0 tx=0).stop 2023-11-08T16:38:04.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07f80e6640 1 -- 172.21.15.81:0/2688951014 reap_dead start 2023-11-08T16:38:04.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07f80e6640 1 -- 172.21.15.81:0/2688951014 reap_dead start 2023-11-08T16:38:04.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 shutdown_connections 2023-11-08T16:38:04.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 >> 172.21.15.81:0/2688951014 conn(0x5618a86c0c00 msgr2=0x5618a8587000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 shutdown_connections 2023-11-08T16:38:04.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.686+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2688951014 wait complete. 2023-11-08T16:38:04.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.687+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2578106486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a780ac00 msgr2=0x5618a7875b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.687+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/2578106486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a780ac00 0x5618a7875b80 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x5618a861b830 tx=0x5618a85e2600 comp rx=0 tx=0).stop 2023-11-08T16:38:04.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.687+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2578106486 shutdown_connections 2023-11-08T16:38:04.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.687+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/2578106486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5618a780bc00 0x5618a7875080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.687+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/2578106486 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5618a780b800 0x5618a7875600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.687+0000 7f07fb0305c0 1 --2- 172.21.15.81:0/2578106486 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5618a780ac00 0x5618a7875b80 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.687+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2578106486 >> 172.21.15.81:0/2578106486 conn(0x5618a780b000 msgr2=0x5618a8586800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.687+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2578106486 shutdown_connections 2023-11-08T16:38:04.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.687+0000 7f07fb0305c0 1 -- 172.21.15.81:0/2578106486 wait complete. 2023-11-08T16:38:04.710 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.144368, result: 2023-11-08T16:38:04.710 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-11-08T16:38:04.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.805+0000 7fa7f1e135c0 1 -- start start 2023-11-08T16:38:04.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.805+0000 7fa7f1e135c0 1 Processor -- start 2023-11-08T16:38:04.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.806+0000 7fa7f1e135c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464756400 0x55b4639e9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.806+0000 7fa7f1e135c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464756000 0x55b4639e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.806+0000 7fa7f1e135c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b46397f800 0x55b4639e9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.806+0000 7fa7f1e135c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4639c7180 con 0x55b464756400 2023-11-08T16:38:04.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.806+0000 7fa7f1e135c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4639c6b40 con 0x55b464756000 2023-11-08T16:38:04.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.806+0000 7fa7f1e135c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4639c6000 con 0x55b46397f800 2023-11-08T16:38:04.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.807+0000 7fa7ef9cc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464756400 0x55b4639e9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.807+0000 7fa7ef9cc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464756400 0x55b4639e9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59524/0 (socket says 172.21.15.81:59524) 2023-11-08T16:38:04.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.807+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3307682207 learned_addr learned my addr 172.21.15.81:0/3307682207 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.807+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3307682207 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b46397f800 msgr2=0x55b4639e9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.807+0000 7fa7ef1cb640 1 --2- 172.21.15.81:0/3307682207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464756000 0x55b4639e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.807+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/3307682207 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b46397f800 0x55b4639e9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.807+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3307682207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464756000 msgr2=0x55b4639e8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.807+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/3307682207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464756000 0x55b4639e8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.807+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3307682207 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4639a44e0 con 0x55b464756400 2023-11-08T16:38:04.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/3307682207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464756400 0x55b4639e9600 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x55b464799260 tx=0x55b464842000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2902087de51c3aa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7ed1c7640 1 -- 172.21.15.81:0/3307682207 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b4639c4f20 con 0x55b464756400 2023-11-08T16:38:04.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7ed1c7640 1 -- 172.21.15.81:0/3307682207 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b464844000 con 0x55b464756400 2023-11-08T16:38:04.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7ed1c7640 1 -- 172.21.15.81:0/3307682207 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b4639c51e0 con 0x55b464756400 2023-11-08T16:38:04.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3307682207 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b4639a41a0 con 0x55b464756400 2023-11-08T16:38:04.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3307682207 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b464846000 con 0x55b464756400 2023-11-08T16:38:04.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7f1e135c0 4 waiting for MDS map... 2023-11-08T16:38:04.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7ed1c7640 1 -- 172.21.15.81:0/3307682207 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x55b463a3ec40 con 0x55b464756400 2023-11-08T16:38:04.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7ed1c7640 1 -- 172.21.15.81:0/3307682207 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 71) v1 ==== 3306+0+0 (secure 0 0 0) 0x55b4639cef00 con 0x55b464756400 2023-11-08T16:38:04.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7f1e135c0 4 Got MDS map 71 2023-11-08T16:38:04.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7f1e135c0 4 datascan.main: Using metadata pool output 2023-11-08T16:38:04.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.808+0000 7fa7f1e135c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:38:04.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.809+0000 7fa7f1e135c0 1 Processor -- start 2023-11-08T16:38:04.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.809+0000 7fa7f1e135c0 1 -- start start 2023-11-08T16:38:04.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.810+0000 7fa7f1e135c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b46397fc00 0x55b4639ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.810+0000 7fa7f1e135c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464840800 0x55b4639ea680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.810+0000 7fa7f1e135c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464840c00 0x55b4639eac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.810+0000 7fa7f1e135c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4639c6a00 con 0x55b464840800 2023-11-08T16:38:04.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.810+0000 7fa7f1e135c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b463a4cdc0 con 0x55b464840c00 2023-11-08T16:38:04.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.810+0000 7fa7f1e135c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b463a4d180 con 0x55b46397fc00 2023-11-08T16:38:04.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.810+0000 7fa7ef9cc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464840800 0x55b4639ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.810+0000 7fa7ef9cc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464840800 0x55b4639ea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59532/0 (socket says 172.21.15.81:59532) 2023-11-08T16:38:04.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.810+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/2722085872 learned_addr learned my addr 172.21.15.81:0/2722085872 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.811+0000 7fa7f01cd640 1 --2- 172.21.15.81:0/2722085872 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b46397fc00 0x55b4639ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.811+0000 7fa7ef1cb640 1 --2- 172.21.15.81:0/2722085872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464840c00 0x55b4639eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.811+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/2722085872 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b46397fc00 msgr2=0x55b4639ea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.811+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/2722085872 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b46397fc00 0x55b4639ea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.811+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/2722085872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464840c00 msgr2=0x55b4639eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:04.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.811+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/2722085872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464840c00 0x55b4639eac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.811+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/2722085872 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4648461a0 con 0x55b464840800 2023-11-08T16:38:04.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.811+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/2722085872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464840800 0x55b4639ea680 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x55b46485e5d0 tx=0x55b464701ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a99908ec55e59de server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.811+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/2722085872 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b4639c4f20 con 0x55b464840800 2023-11-08T16:38:04.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.811+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/2722085872 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b464844c00 con 0x55b464840800 2023-11-08T16:38:04.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.811+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/2722085872 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b4639c51e0 con 0x55b464840800 2023-11-08T16:38:04.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.812+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/2722085872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464840800 msgr2=0x55b4639ea680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.812+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/2722085872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464840800 0x55b4639ea680 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x55b46485e5d0 tx=0x55b464701ec0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.812+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/2722085872 shutdown_connections 2023-11-08T16:38:04.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.812+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/2722085872 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464840c00 0x55b4639eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.812+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/2722085872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464840800 0x55b4639ea680 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.812+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/2722085872 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b46397fc00 0x55b4639ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.812+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/2722085872 >> 172.21.15.81:0/2722085872 conn(0x55b46397ec00 msgr2=0x55b464709800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.812+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/2722085872 shutdown_connections 2023-11-08T16:38:04.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.812+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/2722085872 wait complete. 2023-11-08T16:38:04.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.813+0000 7fa7f1e135c0 1 Processor -- start 2023-11-08T16:38:04.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.813+0000 7fa7f1e135c0 1 -- start start 2023-11-08T16:38:04.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.813+0000 7fa7f1e135c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b46397ec00 0x55b4639ea100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.813+0000 7fa7f1e135c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b464841800 0x55b4639ea680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.813+0000 7fa7f1e135c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464841400 0x55b4639eac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.813+0000 7fa7f1e135c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4639c6b40 con 0x55b46397ec00 2023-11-08T16:38:04.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7f1e135c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4639c7180 con 0x55b464841400 2023-11-08T16:38:04.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7f1e135c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b4639c6140 con 0x55b464841800 2023-11-08T16:38:04.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7f01cd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b46397ec00 0x55b4639ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7f01cd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b46397ec00 0x55b4639ea100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59546/0 (socket says 172.21.15.81:59546) 2023-11-08T16:38:04.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 learned_addr learned my addr 172.21.15.81:0/3863440677 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:04.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7ef1cb640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464841400 0x55b4639eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b464841800 0x55b4639ea680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b464841800 msgr2=0x55b4639ea680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7f01cd640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b464841800 0x55b4639ea680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464841400 msgr2=0x55b4639eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7f01cd640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464841400 0x55b4639eac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b464846340 con 0x55b46397ec00 2023-11-08T16:38:04.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.814+0000 7fa7f01cd640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b46397ec00 0x55b4639ea100 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x55b46485f9b0 tx=0x55b464860e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.815+0000 7fa7e99c0640 1 -- 172.21.15.81:0/3863440677 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b4639c4f20 con 0x55b46397ec00 2023-11-08T16:38:04.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.815+0000 7fa7e99c0640 1 -- 172.21.15.81:0/3863440677 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b4639d8900 con 0x55b46397ec00 2023-11-08T16:38:04.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.815+0000 7fa7e99c0640 1 -- 172.21.15.81:0/3863440677 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b4639c51e0 con 0x55b46397ec00 2023-11-08T16:38:04.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.815+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b4648464e0 con 0x55b46397ec00 2023-11-08T16:38:04.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.815+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b464846340 con 0x55b46397ec00 2023-11-08T16:38:04.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.815+0000 7fa7f1e135c0 4 datascan.init: resolving metadata pool 10 2023-11-08T16:38:04.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.817+0000 7fa7e99c0640 1 -- 172.21.15.81:0/3863440677 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55b4639da840 con 0x55b46397ec00 2023-11-08T16:38:04.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.817+0000 7fa7e99c0640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b4648b7c00 0x55b463ce4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.817+0000 7fa7e99c0640 1 -- 172.21.15.81:0/3863440677 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x55b463a3ea80 con 0x55b46397ec00 2023-11-08T16:38:04.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.817+0000 7fa7f1e135c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:38:04.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.817+0000 7fa7f1e135c0 4 datascan.main: data pool 'cephfs_data' has ID 11 2023-11-08T16:38:04.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.817+0000 7fa7f1e135c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:38:04.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.817+0000 7fa7f1e135c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 12 2023-11-08T16:38:04.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.817+0000 7fa7f1e135c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-11-08T16:38:04.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.817+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b464841000 0x55b4639eb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.817+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:1 11.0 11:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b4639d7880 con 0x55b464841000 2023-11-08T16:38:04.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.817+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b4648b7c00 0x55b463ce4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.818+0000 7fa7ef1cb640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b464841000 0x55b4639eb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.818+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b4648b7c00 0x55b463ce4000 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x55b46485e5d0 tx=0x55b464860c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.818+0000 7fa7ef1cb640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b464841000 0x55b4639eb180 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.819+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(1 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b464841000 2023-11-08T16:38:04.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.819+0000 7fa7ea9c2640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b463cf0000 0x55b463ce4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.819+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:2 11.20 11:04000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa000 con 0x55b463cf0000 2023-11-08T16:38:04.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.820+0000 7fa7f01cd640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b463cf0000 0x55b463ce4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.821+0000 7fa7f01cd640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b463cf0000 0x55b463ce4580 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.821+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(2 [pgnls start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.822+0000 7fa7eb1c3640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b463cf0400 0x55b463ce4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.822+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:3 11.10 11:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa380 con 0x55b463cf0400 2023-11-08T16:38:04.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.822+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b463cf0400 0x55b463ce4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.822+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b463cf0400 0x55b463ce4b00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.823+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(3 [pgnls start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.823+0000 7fa7ea9c2640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b463cf0800 0x55b463ce5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.823+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:4 11.30 11:0c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa700 con 0x55b463cf0800 2023-11-08T16:38:04.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.823+0000 7fa7ef1cb640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b463cf0800 0x55b463ce5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.824+0000 7fa7ef1cb640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b463cf0800 0x55b463ce5080 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.825+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(4 [pgnls start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.825+0000 7fa7eb1c3640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b463cf1000 0x55b463ce5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.825+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:5 11.8 11:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa700 con 0x55b463cf1000 2023-11-08T16:38:04.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.825+0000 7fa7f01cd640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b463cf1000 0x55b463ce5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.826+0000 7fa7f01cd640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b463cf1000 0x55b463ce5b80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.826+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(5 [pgnls start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1000 2023-11-08T16:38:04.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.826+0000 7fa7ea9c2640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b463cf0c00 0x55b463ce5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.826+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:6 11.28 11:14000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfaa80 con 0x55b463cf0c00 2023-11-08T16:38:04.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.827+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b463cf0c00 0x55b463ce5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.828+0000 7fa7ef9cc640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b463cf0c00 0x55b463ce5600 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.828+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(6 [pgnls start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.828+0000 7fa7eb1c3640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b463cf1400 0x55b463ce6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.828+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:7 11.18 11:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b4639d7880 con 0x55b463cf1400 2023-11-08T16:38:04.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.829+0000 7fa7ef1cb640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b463cf1400 0x55b463ce6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.829+0000 7fa7ef1cb640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b463cf1400 0x55b463ce6100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.829+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(7 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.829+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:8 11.38 11:1c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa000 con 0x55b463cf0000 2023-11-08T16:38:04.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.830+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(8 [pgnls start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.830+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:9 11.4 11:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb180 con 0x55b463cf0c00 2023-11-08T16:38:04.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.831+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(9 [pgnls start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.831+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:10 11.24 11:24000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa700 con 0x55b463cf0800 2023-11-08T16:38:04.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.832+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(10 [pgnls start_epoch 86 out=49b] v84'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.832+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:11 11.14 11:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb500 con 0x55b463cf0000 2023-11-08T16:38:04.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.833+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(11 [pgnls start_epoch 86 out=49b] v84'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.833+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:12 11.34 11:2c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfae00 con 0x55b464841000 2023-11-08T16:38:04.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.834+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(12 [pgnls start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b464841000 2023-11-08T16:38:04.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.834+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:13 11.c 11:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb880 con 0x55b464841000 2023-11-08T16:38:04.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.834+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(13 [pgnls start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b464841000 2023-11-08T16:38:04.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.834+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:14 11.2c 11:34000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa700 con 0x55b463cf0400 2023-11-08T16:38:04.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.835+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(14 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.835+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:15 11.1c 11:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfaa80 con 0x55b463cf0000 2023-11-08T16:38:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.835+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(15 [pgnls start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.836+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:16 11.3c 11:3c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfae00 con 0x55b464841000 2023-11-08T16:38:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.836+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(16 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b464841000 2023-11-08T16:38:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.836+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:17 11.2 11:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb180 con 0x55b463cf1400 2023-11-08T16:38:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.836+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(17 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.836+0000 7fa7ea9c2640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b463cf1800 0x55b463ce6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.836+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:18 11.22 11:44000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa380 con 0x55b463cf1800 2023-11-08T16:38:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.837+0000 7fa7f01cd640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b463cf1800 0x55b463ce6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.838+0000 7fa7f01cd640 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b463cf1800 0x55b463ce6680 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:04.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.839+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(18 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.839+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:19 11.12 11:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfbc00 con 0x55b463cf0c00 2023-11-08T16:38:04.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.840+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(19 [pgnls start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.840+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:20 11.32 11:4c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b4639d7880 con 0x55b463cf1400 2023-11-08T16:38:04.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.840+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(20 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.840+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:21 11.a 11:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb880 con 0x55b463cf1000 2023-11-08T16:38:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.841+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(21 [pgnls start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1000 2023-11-08T16:38:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.841+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:22 11.2a 11:54000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb500 con 0x55b463cf0800 2023-11-08T16:38:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.842+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(22 [pgnls start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.842+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:23 11.1a 11:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfae00 con 0x55b463cf1800 2023-11-08T16:38:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.842+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(23 [pgnls start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.843+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:24 11.3a 11:5c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfaa80 con 0x55b463cf0000 2023-11-08T16:38:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.843+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(24 [pgnls start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.844+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:25 11.6 11:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb180 con 0x55b463cf0400 2023-11-08T16:38:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.844+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(25 [pgnls start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.844+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:26 11.26 11:64000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa380 con 0x55b463cf1800 2023-11-08T16:38:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.845+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(26 [pgnls start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.845+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:27 11.16 11:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa000 con 0x55b463cf1800 2023-11-08T16:38:04.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.846+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(27 [pgnls start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.846+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:28 11.36 11:6c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb880 con 0x55b463cf0800 2023-11-08T16:38:04.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.847+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(28 [pgnls start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.847+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:29 11.e 11:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b4639d7880 con 0x55b463cf0800 2023-11-08T16:38:04.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.848+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(29 [pgnls start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.848+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:30 11.2e 11:74000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfae00 con 0x55b464841000 2023-11-08T16:38:04.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.849+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(30 [pgnls start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b464841000 2023-11-08T16:38:04.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.849+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:31 11.1e 11:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c000 con 0x55b463cf0c00 2023-11-08T16:38:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.850+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(31 [pgnls start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.850+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:32 11.3e 11:7c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfaa80 con 0x55b463cf0400 2023-11-08T16:38:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.851+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(32 [pgnls start_epoch 86 out=49b] v84'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.851+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:33 11.1 11:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c380 con 0x55b463cf0000 2023-11-08T16:38:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.852+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(33 [pgnls start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.852+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:34 11.21 11:84000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c000 con 0x55b463cf1800 2023-11-08T16:38:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.853+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(34 [pgnls start_epoch 86 out=49b] v84'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.853+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:35 11.11 11:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c700 con 0x55b463cf1400 2023-11-08T16:38:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.854+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(35 [pgnls start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.854+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:36 11.31 11:8c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d500 con 0x55b463cf0400 2023-11-08T16:38:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.854+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(36 [pgnls start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.855+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:37 11.9 11:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1ca80 con 0x55b463cf1000 2023-11-08T16:38:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.855+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(37 [pgnls start_epoch 86 out=49b] v84'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1000 2023-11-08T16:38:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.855+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:38 11.29 11:94000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d880 con 0x55b463cf0c00 2023-11-08T16:38:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.856+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(38 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.856+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:39 11.19 11:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1ce00 con 0x55b463cf0400 2023-11-08T16:38:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.857+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(39 [pgnls start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.857+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:40 11.39 11:9c000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1dc00 con 0x55b463cf1800 2023-11-08T16:38:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.858+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(40 [pgnls start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.858+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:41 11.5 11:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d180 con 0x55b463cf0c00 2023-11-08T16:38:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.859+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(41 [pgnls start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.859+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:42 11.25 11:a4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa700 con 0x55b463cf1000 2023-11-08T16:38:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.860+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(42 [pgnls start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1000 2023-11-08T16:38:04.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.860+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:43 11.15 11:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfaa80 con 0x55b463cf1400 2023-11-08T16:38:04.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.860+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 5 ==== osd_op_reply(43 [pgnls start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.861+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:44 11.35 11:ac000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfbc00 con 0x55b463cf0c00 2023-11-08T16:38:04.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.862+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(44 [pgnls start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.862+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:45 11.d 11:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa380 con 0x55b463cf1400 2023-11-08T16:38:04.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.862+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 6 ==== osd_op_reply(45 [pgnls start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.863+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:46 11.2d 11:b4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb180 con 0x55b463cf0000 2023-11-08T16:38:04.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.864+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(46 [pgnls start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.864+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:47 11.1d 11:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa000 con 0x55b463cf0c00 2023-11-08T16:38:04.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.865+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(47 [pgnls start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.865+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:48 11.3d 11:bc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d500 con 0x55b463cf1000 2023-11-08T16:38:04.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.865+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(48 [pgnls start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1000 2023-11-08T16:38:04.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.865+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:49 11.3 11:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb500 con 0x55b463cf0000 2023-11-08T16:38:04.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.866+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(49 [pgnls start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.866+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:50 11.23 11:c4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d880 con 0x55b463cf1000 2023-11-08T16:38:04.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.867+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(50 [pgnls start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1000 2023-11-08T16:38:04.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.867+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:51 11.13 11:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb880 con 0x55b463cf1800 2023-11-08T16:38:04.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.868+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(51 [pgnls start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.868+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:52 11.33 11:cc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1ce00 con 0x55b463cf0400 2023-11-08T16:38:04.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.869+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(52 [pgnls start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.869+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:53 11.b 11:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfae00 con 0x55b463cf1000 2023-11-08T16:38:04.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.869+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(53 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1000 2023-11-08T16:38:04.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.870+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:54 11.2b 11:d4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d180 con 0x55b463cf0800 2023-11-08T16:38:04.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.871+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(54 [pgnls start_epoch 86 out=49b] v84'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.871+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:55 11.1b 11:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b4639d7880 con 0x55b463cf0800 2023-11-08T16:38:04.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.872+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(55 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.872+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:56 11.3b 11:dc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c380 con 0x55b463cf1400 2023-11-08T16:38:04.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.873+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 7 ==== osd_op_reply(56 [pgnls start_epoch 86 out=49b] v84'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.873+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:57 11.7 11:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1dc00 con 0x55b463cf1800 2023-11-08T16:38:04.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.874+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(57 [pgnls start_epoch 86 out=81b] v84'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.874+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:58 11.7 11:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b4639d7c00 con 0x55b463cf1800 2023-11-08T16:38:04.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.875+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 9 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.876+0000 7fa7f1e135c0 10 datascan.operator(): handling object 1099511627777.0 2023-11-08T16:38:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.876+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:59 11.7 11:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d20000 con 0x55b463cf1800 2023-11-08T16:38:04.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.880+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 10 ==== osd_op_reply(59 10000000001.00000000 [call] v86'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.880+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:60 11.27 11:e4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d20380 con 0x55b463cf0800 2023-11-08T16:38:04.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.881+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(60 [pgnls start_epoch 86 out=49b] v84'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.881+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:61 11.17 11:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d500 con 0x55b463cf0c00 2023-11-08T16:38:04.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.882+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 9 ==== osd_op_reply(61 [pgnls start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.882+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:62 11.37 11:ec000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c000 con 0x55b463cf1400 2023-11-08T16:38:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.883+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 8 ==== osd_op_reply(62 [pgnls start_epoch 86 out=49b] v84'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.883+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:63 11.f 11:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d880 con 0x55b463cf0c00 2023-11-08T16:38:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.884+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 10 ==== osd_op_reply(63 [pgnls start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.884+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:64 11.2f 11:f4000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1ca80 con 0x55b463cf1400 2023-11-08T16:38:04.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.885+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 9 ==== osd_op_reply(64 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.885+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:65 11.1f 11:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa700 con 0x55b463cf1400 2023-11-08T16:38:04.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.886+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 10 ==== osd_op_reply(65 [pgnls start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.886+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:66 11.3f 11:fc000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c700 con 0x55b463cf0c00 2023-11-08T16:38:04.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.887+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 11 ==== osd_op_reply(66 [pgnls start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.887+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:67 12.0 12:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d20700 con 0x55b463cf0800 2023-11-08T16:38:04.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.888+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(67 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.888+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:68 12.10 12:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1ce00 con 0x55b463cf0400 2023-11-08T16:38:04.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.889+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(68 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.889+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:69 12.8 12:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb180 con 0x55b463cf1000 2023-11-08T16:38:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.890+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(69 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1000 2023-11-08T16:38:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.890+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:70 12.18 12:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d500 con 0x55b464841000 2023-11-08T16:38:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.891+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(70 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b464841000 2023-11-08T16:38:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.891+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:71 12.4 12:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb500 con 0x55b463cf0000 2023-11-08T16:38:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.892+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(71 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.892+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:72 12.14 12:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d880 con 0x55b463cf1800 2023-11-08T16:38:04.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.893+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 11 ==== osd_op_reply(72 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.893+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:73 12.c 12:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d20000 con 0x55b463cf0400 2023-11-08T16:38:04.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.894+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(73 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.894+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:74 12.1c 12:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c700 con 0x55b463cf0400 2023-11-08T16:38:04.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.895+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(74 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.895+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:75 12.2 12:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1dc00 con 0x55b463cf0800 2023-11-08T16:38:04.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.897+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(75 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.897+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:76 12.12 12:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b4639d7880 con 0x55b463cf1000 2023-11-08T16:38:04.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.898+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(76 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1000 2023-11-08T16:38:04.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.898+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:77 12.a 12:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d180 con 0x55b463cf1400 2023-11-08T16:38:04.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.899+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 11 ==== osd_op_reply(77 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.899+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:78 12.1a 12:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfbc00 con 0x55b463cf0800 2023-11-08T16:38:04.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.900+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(78 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.900+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:79 12.6 12:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c380 con 0x55b464841000 2023-11-08T16:38:04.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.900+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(79 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b464841000 2023-11-08T16:38:04.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.900+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:80 12.16 12:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa000 con 0x55b463cf0000 2023-11-08T16:38:04.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.902+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(80 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.902+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:81 12.e 12:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb880 con 0x55b463cf0400 2023-11-08T16:38:04.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.902+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 11 ==== osd_op_reply(81 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.902+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:82 12.1e 12:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfaa80 con 0x55b463cf0800 2023-11-08T16:38:04.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.903+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(82 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.903+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:83 12.1 12:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfae00 con 0x55b463cf0000 2023-11-08T16:38:04.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.904+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 11 ==== osd_op_reply(83 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.904+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:84 12.11 12:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfa380 con 0x55b463cf1800 2023-11-08T16:38:04.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.905+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 12 ==== osd_op_reply(84 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.906+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:85 12.9 12:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb180 con 0x55b463cf1400 2023-11-08T16:38:04.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.906+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.2 v2:172.21.15.81:6824/3759311428 12 ==== osd_op_reply(85 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf1400 2023-11-08T16:38:04.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.906+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:86 12.19 12:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d20700 con 0x55b463cf0400 2023-11-08T16:38:04.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.907+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 12 ==== osd_op_reply(86 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.907+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:87 12.5 12:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfb500 con 0x55b463cf0000 2023-11-08T16:38:04.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.908+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 12 ==== osd_op_reply(87 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.908+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:88 12.15 12:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d20380 con 0x55b463cf0000 2023-11-08T16:38:04.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.909+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 13 ==== osd_op_reply(88 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.910+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:89 12.d 12:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b4639d7c00 con 0x55b463cf0c00 2023-11-08T16:38:04.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.911+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.7 v2:172.21.15.93:6818/433691658 12 ==== osd_op_reply(89 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0c00 2023-11-08T16:38:04.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.911+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:90 12.1d 12:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d20e00 con 0x55b463cf0400 2023-11-08T16:38:04.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.912+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 13 ==== osd_op_reply(90 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.912+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:91 12.3 12:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d20000 con 0x55b463cf0400 2023-11-08T16:38:04.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.913+0000 7fa7ef9cc640 1 -- 172.21.15.81:0/3863440677 <== osd.0 v2:172.21.15.81:6808/1666604866 14 ==== osd_op_reply(91 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4fc0 con 0x55b463cf0400 2023-11-08T16:38:04.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.913+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:92 12.13 12:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d21180 con 0x55b464841000 2023-11-08T16:38:04.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.914+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(92 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b464841000 2023-11-08T16:38:04.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.914+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:93 12.b 12:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d20700 con 0x55b463cf1000 2023-11-08T16:38:04.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.914+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(93 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1000 2023-11-08T16:38:04.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.915+0000 7fa7ea9c2640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:94 12.1b 12:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c000 con 0x55b463cf0800 2023-11-08T16:38:04.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.916+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.4 v2:172.21.15.93:6800/3259756279 13 ==== osd_op_reply(94 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4b40 con 0x55b463cf0800 2023-11-08T16:38:04.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.916+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:95 12.7 12:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d20e00 con 0x55b464841000 2023-11-08T16:38:04.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.917+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(95 [pgnls start_epoch 86 out=81b] v84'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b4646e4b40 con 0x55b464841000 2023-11-08T16:38:04.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.917+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:96 12.7 12:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d20a80 con 0x55b464841000 2023-11-08T16:38:04.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.918+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(96 10000000001.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b4646e4b40 con 0x55b464841000 2023-11-08T16:38:04.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.918+0000 7fa7f1e135c0 10 datascan.operator(): handling object 1099511627777.0 2023-11-08T16:38:04.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.918+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:97 11.7 11:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463cfbc00 con 0x55b463cf1800 2023-11-08T16:38:04.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.921+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 13 ==== osd_op_reply(97 10000000001.00000000 [call,setxattr (8)] v86'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.921+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:98 12.17 12:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b4639d7880 con 0x55b463cf0000 2023-11-08T16:38:04.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.923+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 14 ==== osd_op_reply(98 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.923+0000 7fa7eb1c3640 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:99 12.f 12:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d21500 con 0x55b463cf0000 2023-11-08T16:38:04.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.924+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 15 ==== osd_op_reply(99 [pgnls start_epoch 86 out=81b] v84'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.924+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:100 12.f 12:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c380 con 0x55b463cf0000 2023-11-08T16:38:04.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.925+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 16 ==== osd_op_reply(100 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.925+0000 7fa7f1e135c0 10 datascan.operator(): handling object 1099511627777.1 2023-11-08T16:38:04.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.925+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:101 11.7 11:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1c000 con 0x55b463cf1800 2023-11-08T16:38:04.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.929+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.5 v2:172.21.15.93:6802/2406579932 14 ==== osd_op_reply(101 10000000001.00000000 [call,setxattr (8)] v86'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf1800 2023-11-08T16:38:04.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.929+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:102 12.1f 12:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55b463d1d880 con 0x55b463cf0000 2023-11-08T16:38:04.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f01cd640 1 -- 172.21.15.81:0/3863440677 <== osd.6 v2:172.21.15.93:6816/607960134 17 ==== osd_op_reply(102 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b4646e4d80 con 0x55b463cf0000 2023-11-08T16:38:04.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b463cf0400 msgr2=0x55b463ce4b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b463cf0400 0x55b463ce4b00 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b464841000 msgr2=0x55b4639eb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b464841000 0x55b4639eb180 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b463cf1400 msgr2=0x55b463ce6100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b463cf1400 0x55b463ce6100 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b463cf1000 msgr2=0x55b463ce5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b463cf1000 0x55b463ce5b80 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b463cf0800 msgr2=0x55b463ce5080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b463cf0800 0x55b463ce5080 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b463cf1800 msgr2=0x55b463ce6680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b463cf1800 0x55b463ce6680 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b463cf0000 msgr2=0x55b463ce4580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b463cf0000 0x55b463ce4580 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b463cf0c00 msgr2=0x55b463ce5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b463cf0c00 0x55b463ce5600 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 reap_dead start 2023-11-08T16:38:04.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b4648b7c00 msgr2=0x55b463ce4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 reap_dead start 2023-11-08T16:38:04.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b4648b7c00 0x55b463ce4000 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x55b46485e5d0 tx=0x55b464860c00 comp rx=0 tx=0).stop 2023-11-08T16:38:04.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b46397ec00 msgr2=0x55b4639ea100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 reap_dead start 2023-11-08T16:38:04.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3863440677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b46397ec00 0x55b4639ea100 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x55b46485f9b0 tx=0x55b464860e40 comp rx=0 tx=0).stop 2023-11-08T16:38:04.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.930+0000 7fa7ef1cb640 1 -- 172.21.15.81:0/3863440677 reap_dead start 2023-11-08T16:38:04.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.931+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 shutdown_connections 2023-11-08T16:38:04.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.931+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 >> 172.21.15.81:0/3863440677 conn(0x55b46397fc00 msgr2=0x55b464709800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.931+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 shutdown_connections 2023-11-08T16:38:04.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.931+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3863440677 wait complete. 2023-11-08T16:38:04.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.932+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3307682207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464756400 msgr2=0x55b4639e9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:04.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.932+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3307682207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464756400 0x55b4639e9600 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x55b464799260 tx=0x55b464842000 comp rx=0 tx=0).stop 2023-11-08T16:38:04.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.933+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3307682207 shutdown_connections 2023-11-08T16:38:04.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.933+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3307682207 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b46397f800 0x55b4639e9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.933+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3307682207 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b464756000 0x55b4639e8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.933+0000 7fa7f1e135c0 1 --2- 172.21.15.81:0/3307682207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b464756400 0x55b4639e9600 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:04.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.933+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3307682207 >> 172.21.15.81:0/3307682207 conn(0x55b46397f000 msgr2=0x55b464709000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:04.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.933+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3307682207 shutdown_connections 2023-11-08T16:38:04.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:04.933+0000 7fa7f1e135c0 1 -- 172.21.15.81:0/3307682207 wait complete. 2023-11-08T16:38:04.946 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.236045, result: 2023-11-08T16:38:04.946 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-11-08T16:38:05.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.048+0000 7f8da85715c0 1 -- start start 2023-11-08T16:38:05.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.048+0000 7f8da85715c0 1 Processor -- start 2023-11-08T16:38:05.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.049+0000 7f8da85715c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562039b98400 0x562039b23600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.049+0000 7f8da85715c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562039ab9c00 0x562039b23b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.049+0000 7f8da85715c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562039b98800 0x562039b23080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.049+0000 7f8da85715c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562039b01180 con 0x562039b98400 2023-11-08T16:38:05.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.049+0000 7f8da85715c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562039b00b40 con 0x562039b98800 2023-11-08T16:38:05.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.049+0000 7f8da85715c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562039b00000 con 0x562039ab9c00 2023-11-08T16:38:05.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.050+0000 7f8da6d17640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562039b98400 0x562039b23600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.050+0000 7f8da6d17640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562039b98400 0x562039b23600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59566/0 (socket says 172.21.15.81:59566) 2023-11-08T16:38:05.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.050+0000 7f8da6d17640 1 -- 172.21.15.81:0/4099366890 learned_addr learned my addr 172.21.15.81:0/4099366890 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:05.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.050+0000 7f8da7518640 1 --2- 172.21.15.81:0/4099366890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562039b98800 0x562039b23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.050+0000 7f8da6516640 1 --2- 172.21.15.81:0/4099366890 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562039ab9c00 0x562039b23b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.050+0000 7f8da6d17640 1 -- 172.21.15.81:0/4099366890 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562039ab9c00 msgr2=0x562039b23b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.050+0000 7f8da6d17640 1 --2- 172.21.15.81:0/4099366890 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562039ab9c00 0x562039b23b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.050+0000 7f8da6d17640 1 -- 172.21.15.81:0/4099366890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562039b98800 msgr2=0x562039b23080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.050+0000 7f8da6d17640 1 --2- 172.21.15.81:0/4099366890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562039b98800 0x562039b23080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.050+0000 7f8da6d17640 1 -- 172.21.15.81:0/4099366890 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562039ade4e0 con 0x562039b98400 2023-11-08T16:38:05.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.050+0000 7f8da6d17640 1 --2- 172.21.15.81:0/4099366890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562039b98400 0x562039b23600 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x56203a8c5260 tx=0x56203a970300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6533144be428305f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.051+0000 7f8da4512640 1 -- 172.21.15.81:0/4099366890 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562039aff1e0 con 0x562039b98400 2023-11-08T16:38:05.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.051+0000 7f8da4512640 1 -- 172.21.15.81:0/4099366890 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56203a97a000 con 0x562039b98400 2023-11-08T16:38:05.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.051+0000 7f8da4512640 1 -- 172.21.15.81:0/4099366890 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562039afef20 con 0x562039b98400 2023-11-08T16:38:05.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.051+0000 7f8da85715c0 1 -- 172.21.15.81:0/4099366890 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562039ade1a0 con 0x562039b98400 2023-11-08T16:38:05.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.051+0000 7f8da85715c0 1 -- 172.21.15.81:0/4099366890 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x56203a97c000 con 0x562039b98400 2023-11-08T16:38:05.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.051+0000 7f8da85715c0 4 waiting for MDS map... 2023-11-08T16:38:05.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.051+0000 7f8da4512640 1 -- 172.21.15.81:0/4099366890 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x562039b78c40 con 0x562039b98400 2023-11-08T16:38:05.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.051+0000 7f8da4512640 1 -- 172.21.15.81:0/4099366890 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 71) v1 ==== 3306+0+0 (secure 0 0 0) 0x562039b08f00 con 0x562039b98400 2023-11-08T16:38:05.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.051+0000 7f8da85715c0 4 Got MDS map 71 2023-11-08T16:38:05.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.051+0000 7f8da85715c0 4 datascan.main: Using metadata pool output 2023-11-08T16:38:05.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.051+0000 7f8da85715c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:38:05.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.052+0000 7f8da85715c0 1 Processor -- start 2023-11-08T16:38:05.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.052+0000 7f8da85715c0 1 -- start start 2023-11-08T16:38:05.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.053+0000 7f8da85715c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56203a976c00 0x562039b22b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.053+0000 7f8da85715c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.053+0000 7f8da85715c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56203a977400 0x562039b24680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.053+0000 7f8da85715c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562039b00a00 con 0x56203a977000 2023-11-08T16:38:05.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.053+0000 7f8da85715c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562039b00140 con 0x56203a977400 2023-11-08T16:38:05.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.053+0000 7f8da85715c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562039b86dc0 con 0x56203a976c00 2023-11-08T16:38:05.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.053+0000 7f8da6d17640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.053+0000 7f8da6d17640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59580/0 (socket says 172.21.15.81:59580) 2023-11-08T16:38:05.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.053+0000 7f8da6d17640 1 -- 172.21.15.81:0/3059872311 learned_addr learned my addr 172.21.15.81:0/3059872311 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:05.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.054+0000 7f8da6d17640 1 -- 172.21.15.81:0/3059872311 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56203a976c00 msgr2=0x562039b22b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.054+0000 7f8da6d17640 1 --2- 172.21.15.81:0/3059872311 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56203a976c00 0x562039b22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.054+0000 7f8da6d17640 1 -- 172.21.15.81:0/3059872311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56203a977400 msgr2=0x562039b24680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.054+0000 7f8da6d17640 1 --2- 172.21.15.81:0/3059872311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56203a977400 0x562039b24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.054+0000 7f8da6d17640 1 -- 172.21.15.81:0/3059872311 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56203a97c1a0 con 0x56203a977000 2023-11-08T16:38:05.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.054+0000 7f8da6d17640 1 --2- 172.21.15.81:0/3059872311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x56203a9945d0 tx=0x56203a970060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3d7d2afed59e50 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.054+0000 7f8da250e640 1 -- 172.21.15.81:0/3059872311 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562039aff1e0 con 0x56203a977000 2023-11-08T16:38:05.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.054+0000 7f8da250e640 1 -- 172.21.15.81:0/3059872311 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56203a97a300 con 0x56203a977000 2023-11-08T16:38:05.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.054+0000 7f8da250e640 1 -- 172.21.15.81:0/3059872311 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562039afef20 con 0x56203a977000 2023-11-08T16:38:05.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.054+0000 7f8da85715c0 1 -- 172.21.15.81:0/3059872311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 msgr2=0x562039b24100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.054+0000 7f8da85715c0 1 --2- 172.21.15.81:0/3059872311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x56203a9945d0 tx=0x56203a970060 comp rx=0 tx=0).stop 2023-11-08T16:38:05.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.055+0000 7f8da85715c0 1 -- 172.21.15.81:0/3059872311 shutdown_connections 2023-11-08T16:38:05.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.055+0000 7f8da85715c0 1 --2- 172.21.15.81:0/3059872311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56203a977400 0x562039b24680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.055+0000 7f8da85715c0 1 --2- 172.21.15.81:0/3059872311 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.055+0000 7f8da85715c0 1 --2- 172.21.15.81:0/3059872311 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56203a976c00 0x562039b22b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.055+0000 7f8da85715c0 1 -- 172.21.15.81:0/3059872311 >> 172.21.15.81:0/3059872311 conn(0x562039b98000 msgr2=0x56203a843800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:05.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.055+0000 7f8da85715c0 1 -- 172.21.15.81:0/3059872311 shutdown_connections 2023-11-08T16:38:05.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.055+0000 7f8da85715c0 1 -- 172.21.15.81:0/3059872311 wait complete. 2023-11-08T16:38:05.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.055+0000 7f8da85715c0 1 Processor -- start 2023-11-08T16:38:05.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.056+0000 7f8da85715c0 1 -- start start 2023-11-08T16:38:05.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.056+0000 7f8da85715c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56203a976c00 0x562039b22b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.056+0000 7f8da85715c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.056+0000 7f8da85715c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56203a977400 0x562039b24680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.056+0000 7f8da85715c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562039b87180 con 0x56203a977000 2023-11-08T16:38:05.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.056+0000 7f8da85715c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562039b86c80 con 0x56203a977400 2023-11-08T16:38:05.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.056+0000 7f8da85715c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562039b00b40 con 0x56203a976c00 2023-11-08T16:38:05.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da6d17640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da6d17640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59584/0 (socket says 172.21.15.81:59584) 2023-11-08T16:38:05.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 learned_addr learned my addr 172.21.15.81:0/347892247 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:05.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da7518640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56203a976c00 0x562039b22b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da6516640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56203a977400 0x562039b24680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56203a976c00 msgr2=0x562039b22b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da6d17640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56203a976c00 0x562039b22b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56203a977400 msgr2=0x562039b24680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da6d17640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56203a977400 0x562039b24680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56203a97c340 con 0x56203a977000 2023-11-08T16:38:05.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da7518640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56203a976c00 0x562039b22b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:05.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da6d17640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x56203a994840 tx=0x56203a996c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.057+0000 7f8da0d0b640 1 -- 172.21.15.81:0/347892247 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562039afedc0 con 0x56203a977000 2023-11-08T16:38:05.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.058+0000 7f8da0d0b640 1 -- 172.21.15.81:0/347892247 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56203a97a480 con 0x56203a977000 2023-11-08T16:38:05.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.058+0000 7f8da0d0b640 1 -- 172.21.15.81:0/347892247 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562039aff1e0 con 0x56203a977000 2023-11-08T16:38:05.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.058+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56203a97c4e0 con 0x56203a977000 2023-11-08T16:38:05.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.058+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56203a97c680 con 0x56203a977000 2023-11-08T16:38:05.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.058+0000 7f8da85715c0 4 datascan.init: resolving metadata pool 10 2023-11-08T16:38:05.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.059+0000 7f8da0d0b640 1 -- 172.21.15.81:0/347892247 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x562039b14b00 con 0x56203a977000 2023-11-08T16:38:05.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.060+0000 7f8da0d0b640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56203a9f0c00 0x562039b25700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.060+0000 7f8da0d0b640 1 -- 172.21.15.81:0/347892247 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x562039b78a80 con 0x56203a977000 2023-11-08T16:38:05.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.060+0000 7f8da85715c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:38:05.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.060+0000 7f8da85715c0 4 datascan.main: data pool 'cephfs_data' has ID 11 2023-11-08T16:38:05.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.060+0000 7f8da85715c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:38:05.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.060+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56203a977800 0x562039b24c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.060+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:1 10.14 10:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11880 con 0x56203a977800 2023-11-08T16:38:05.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.060+0000 7f8da7518640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56203a9f0c00 0x562039b25700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.060+0000 7f8da6516640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56203a977800 0x562039b24c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.061+0000 7f8da7518640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56203a9f0c00 0x562039b25700 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x562039baa180 tx=0x56203a996cc0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.061+0000 7f8da6516640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56203a977800 0x562039b24c00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.061+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv29 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.061+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56203a977c00 0x562039b25180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.061+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:2 10.1f 10:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11c00 con 0x56203a977c00 2023-11-08T16:38:05.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.062+0000 7f8da7518640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56203a977c00 0x562039b25180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.063+0000 7f8da7518640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56203a977c00 0x562039b25180 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.064+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.064+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:3 11.0 11:00000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26000 con 0x56203a977800 2023-11-08T16:38:05.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.065+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(3 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.065+0000 7f8da1d0d640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56203a9f1000 0x562039e30000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.065+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:4 11.20 11:04000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11880 con 0x56203a9f1000 2023-11-08T16:38:05.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.066+0000 7f8da6d17640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56203a9f1000 0x562039e30000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.067+0000 7f8da6d17640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56203a9f1000 0x562039e30000 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.067+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.067+0000 7f8da250e640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56203a9f1400 0x562039e30580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.067+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 11.10 11:08000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26000 con 0x56203a9f1400 2023-11-08T16:38:05.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.068+0000 7f8da6516640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56203a9f1400 0x562039e30580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.068+0000 7f8da6516640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56203a9f1400 0x562039e30580 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.069+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.069+0000 7f8da1d0d640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56203a9f1800 0x562039e30b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.069+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:6 11.30 11:0c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26e00 con 0x56203a9f1800 2023-11-08T16:38:05.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.070+0000 7f8da7518640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56203a9f1800 0x562039e30b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.070+0000 7f8da7518640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56203a9f1800 0x562039e30b00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.071+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.071+0000 7f8da250e640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562039e44000 0x562039e31600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.071+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:7 11.8 11:10000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26e00 con 0x562039e44000 2023-11-08T16:38:05.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.072+0000 7f8da6d17640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562039e44000 0x562039e31600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.072+0000 7f8da6d17640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562039e44000 0x562039e31600 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.073+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.073+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:8 11.28 11:14000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27180 con 0x56203a977c00 2023-11-08T16:38:05.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.074+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.074+0000 7f8da250e640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562039e44400 0x562039e31b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.074+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:9 11.18 11:18000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11c00 con 0x562039e44400 2023-11-08T16:38:05.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.074+0000 7f8da6516640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562039e44400 0x562039e31b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.075+0000 7f8da6516640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562039e44400 0x562039e31b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.075+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(9 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.075+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:10 11.38 11:1c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27500 con 0x56203a9f1000 2023-11-08T16:38:05.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.076+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.076+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:11 11.4 11:20000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26e00 con 0x56203a977c00 2023-11-08T16:38:05.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.077+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.077+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:12 11.24 11:24000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27880 con 0x56203a9f1800 2023-11-08T16:38:05.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.078+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(12 [pgnls-filter start_epoch 86 out=49b] v84'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.078+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:13 11.14 11:28000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27500 con 0x56203a9f1000 2023-11-08T16:38:05.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.079+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(13 [pgnls-filter start_epoch 86 out=49b] v84'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.079+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:14 11.34 11:2c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27c00 con 0x56203a977800 2023-11-08T16:38:05.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.080+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.080+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:15 11.c 11:30000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11880 con 0x56203a977800 2023-11-08T16:38:05.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.080+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.080+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:16 11.2c 11:34000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26000 con 0x56203a9f1400 2023-11-08T16:38:05.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.081+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(16 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.081+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:17 11.1c 11:38000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26e00 con 0x56203a9f1000 2023-11-08T16:38:05.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.082+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(17 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.082+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:18 11.3c 11:3c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27c00 con 0x56203a977800 2023-11-08T16:38:05.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.082+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(18 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.082+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:19 11.2 11:40000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27880 con 0x562039e44400 2023-11-08T16:38:05.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.082+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.083+0000 7f8da1d0d640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56203a9f1c00 0x562039e31080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.083+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:20 11.22 11:44000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27180 con 0x56203a9f1c00 2023-11-08T16:38:05.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.083+0000 7f8da7518640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56203a9f1c00 0x562039e31080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.084+0000 7f8da7518640 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56203a9f1c00 0x562039e31080 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.085+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(20 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.085+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:21 11.12 11:48000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26000 con 0x56203a977c00 2023-11-08T16:38:05.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.086+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.086+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:22 11.32 11:4c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11c00 con 0x562039e44400 2023-11-08T16:38:05.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.087+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(22 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.087+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:23 11.a 11:50000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27c00 con 0x562039e44000 2023-11-08T16:38:05.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.088+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.088+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:24 11.2a 11:54000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58000 con 0x56203a9f1800 2023-11-08T16:38:05.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.089+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(24 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.089+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:25 11.1a 11:58000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27880 con 0x56203a9f1c00 2023-11-08T16:38:05.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.090+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(25 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.090+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:26 11.3a 11:5c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58380 con 0x56203a9f1000 2023-11-08T16:38:05.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.091+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(26 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.091+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:27 11.6 11:60000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11880 con 0x56203a9f1400 2023-11-08T16:38:05.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.091+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(27 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.091+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:28 11.26 11:64000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58700 con 0x56203a9f1c00 2023-11-08T16:38:05.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.092+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.092+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:29 11.16 11:68000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27500 con 0x56203a9f1c00 2023-11-08T16:38:05.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.093+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(29 [pgnls-filter start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.093+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:30 11.36 11:6c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58a80 con 0x56203a9f1800 2023-11-08T16:38:05.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.094+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(30 [pgnls-filter start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.094+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:31 11.e 11:70000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26e00 con 0x56203a9f1800 2023-11-08T16:38:05.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.095+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(31 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.095+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:32 11.2e 11:74000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58e00 con 0x56203a977800 2023-11-08T16:38:05.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.096+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(32 [pgnls-filter start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.096+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:33 11.1e 11:78000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27c00 con 0x56203a977c00 2023-11-08T16:38:05.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.097+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(33 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.097+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:34 11.3e 11:7c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58700 con 0x56203a9f1400 2023-11-08T16:38:05.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.098+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(34 [pgnls-filter start_epoch 86 out=49b] v84'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.098+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:35 11.1 11:80000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27180 con 0x56203a9f1000 2023-11-08T16:38:05.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.099+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(35 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.099+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:36 11.21 11:84000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58a80 con 0x56203a9f1c00 2023-11-08T16:38:05.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.100+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(36 [pgnls-filter start_epoch 86 out=49b] v84'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.100+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:37 11.11 11:88000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26000 con 0x562039e44400 2023-11-08T16:38:05.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.100+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(37 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.100+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:38 11.31 11:8c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58380 con 0x56203a9f1400 2023-11-08T16:38:05.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.101+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(38 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.101+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:39 11.9 11:90000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27880 con 0x562039e44000 2023-11-08T16:38:05.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.102+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 86 out=49b] v84'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.102+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:40 11.29 11:94000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27500 con 0x56203a977c00 2023-11-08T16:38:05.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.103+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(40 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.103+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:41 11.19 11:98000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58e00 con 0x56203a9f1400 2023-11-08T16:38:05.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.104+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(41 [pgnls-filter start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.104+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:42 11.39 11:9c000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26e00 con 0x56203a9f1c00 2023-11-08T16:38:05.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.105+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(42 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.105+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:43 11.5 11:a0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58700 con 0x56203a977c00 2023-11-08T16:38:05.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.106+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(43 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.106+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:44 11.25 11:a4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27c00 con 0x562039e44000 2023-11-08T16:38:05.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.107+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(44 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.107+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:45 11.15 11:a8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58000 con 0x562039e44400 2023-11-08T16:38:05.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.108+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.108+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:46 11.35 11:ac000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59500 con 0x56203a977c00 2023-11-08T16:38:05.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.109+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(46 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.109+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:47 11.d 11:b0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59180 con 0x562039e44400 2023-11-08T16:38:05.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.109+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 6 ==== osd_op_reply(47 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.109+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:48 11.2d 11:b4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59880 con 0x56203a9f1000 2023-11-08T16:38:05.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.111+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(48 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.111+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:49 11.1d 11:b8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26000 con 0x56203a977c00 2023-11-08T16:38:05.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.112+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 9 ==== osd_op_reply(49 [pgnls-filter start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.112+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:50 11.3d 11:bc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59c00 con 0x562039e44000 2023-11-08T16:38:05.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.113+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(50 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.113+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:51 11.3 11:c0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11c00 con 0x56203a9f1000 2023-11-08T16:38:05.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.114+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(51 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.114+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:52 11.23 11:c4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27180 con 0x562039e44000 2023-11-08T16:38:05.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.115+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(52 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.115+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:53 11.13 11:c8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11880 con 0x56203a9f1c00 2023-11-08T16:38:05.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.116+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.116+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:54 11.33 11:cc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27880 con 0x56203a9f1400 2023-11-08T16:38:05.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.117+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(54 [pgnls-filter start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.117+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:55 11.b 11:d0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27500 con 0x562039e44000 2023-11-08T16:38:05.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.118+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(55 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.118+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:56 11.2b 11:d4000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27c00 con 0x56203a9f1800 2023-11-08T16:38:05.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.119+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(56 [pgnls-filter start_epoch 86 out=49b] v84'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.119+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:57 11.1b 11:d8000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26e00 con 0x56203a9f1800 2023-11-08T16:38:05.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.120+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(57 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.120+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:58 11.3b 11:dc000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59880 con 0x562039e44400 2023-11-08T16:38:05.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.120+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 7 ==== osd_op_reply(58 [pgnls-filter start_epoch 86 out=49b] v84'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.120+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:59 11.7 11:e0000000::::head [pgnls-filter start_epoch 86 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26000 con 0x56203a9f1c00 2023-11-08T16:38:05.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.121+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(59 [pgnls-filter start_epoch 86 out=81b] v86'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.122+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:60 11.0 11:00000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26380 con 0x56203a977800 2023-11-08T16:38:05.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.122+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(60 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.122+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:61 11.20 11:04000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58380 con 0x56203a9f1000 2023-11-08T16:38:05.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.124+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(61 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.124+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:62 11.10 11:08000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59c00 con 0x56203a9f1400 2023-11-08T16:38:05.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.125+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(62 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.125+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:63 11.30 11:0c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58e00 con 0x56203a9f1800 2023-11-08T16:38:05.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.126+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(63 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.126+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:64 11.8 11:10000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58a80 con 0x562039e44000 2023-11-08T16:38:05.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.127+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(64 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.127+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:65 11.28 11:14000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58000 con 0x56203a977c00 2023-11-08T16:38:05.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.128+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 10 ==== osd_op_reply(65 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.128+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:66 11.18 11:18000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58700 con 0x562039e44400 2023-11-08T16:38:05.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.129+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 8 ==== osd_op_reply(66 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.129+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:67 11.38 11:1c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59180 con 0x56203a9f1000 2023-11-08T16:38:05.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.130+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(67 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.131+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:68 11.4 11:20000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59500 con 0x56203a977c00 2023-11-08T16:38:05.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.132+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.132+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:69 11.24 11:24000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59880 con 0x56203a9f1800 2023-11-08T16:38:05.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.133+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(69 [pgnls-filter start_epoch 86 out=49b] v84'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.133+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:70 11.14 11:28000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58e00 con 0x56203a9f1000 2023-11-08T16:38:05.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.134+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 11 ==== osd_op_reply(70 [pgnls-filter start_epoch 86 out=49b] v84'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.134+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:71 11.34 11:2c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27880 con 0x56203a977800 2023-11-08T16:38:05.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.135+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(71 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.135+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:72 11.c 11:30000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58000 con 0x56203a977800 2023-11-08T16:38:05.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.136+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(72 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.136+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:73 11.2c 11:34000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59500 con 0x56203a9f1400 2023-11-08T16:38:05.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.137+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(73 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.137+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:74 11.1c 11:38000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27c00 con 0x56203a9f1000 2023-11-08T16:38:05.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.138+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 12 ==== osd_op_reply(74 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.138+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:75 11.3c 11:3c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58380 con 0x56203a977800 2023-11-08T16:38:05.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.139+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(75 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.140+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:76 11.2 11:40000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26e00 con 0x562039e44400 2023-11-08T16:38:05.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.140+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 9 ==== osd_op_reply(76 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.140+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:77 11.22 11:44000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58a80 con 0x56203a9f1c00 2023-11-08T16:38:05.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.141+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.142+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:78 11.12 11:48000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26000 con 0x56203a977c00 2023-11-08T16:38:05.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.143+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 12 ==== osd_op_reply(78 [pgnls-filter start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.143+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:79 11.32 11:4c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59180 con 0x562039e44400 2023-11-08T16:38:05.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.143+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 10 ==== osd_op_reply(79 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.144+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:80 11.a 11:50000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27180 con 0x562039e44000 2023-11-08T16:38:05.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.144+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(80 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.144+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:81 11.2a 11:54000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59c00 con 0x56203a9f1800 2023-11-08T16:38:05.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.146+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.146+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:82 11.1a 11:58000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27500 con 0x56203a9f1c00 2023-11-08T16:38:05.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.147+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 10 ==== osd_op_reply(82 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.147+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:83 11.3a 11:5c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58700 con 0x56203a9f1000 2023-11-08T16:38:05.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.148+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 13 ==== osd_op_reply(83 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.148+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:84 11.6 11:60000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11880 con 0x56203a9f1400 2023-11-08T16:38:05.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.149+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(84 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.149+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:85 11.26 11:64000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58000 con 0x56203a9f1c00 2023-11-08T16:38:05.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.150+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 11 ==== osd_op_reply(85 [pgnls-filter start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.150+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:86 11.16 11:68000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11c00 con 0x56203a9f1c00 2023-11-08T16:38:05.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.152+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.152+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:87 11.36 11:6c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59500 con 0x56203a9f1800 2023-11-08T16:38:05.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.153+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(87 [pgnls-filter start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.153+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:88 11.e 11:70000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59880 con 0x56203a9f1800 2023-11-08T16:38:05.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.153+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(88 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.153+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:89 11.2e 11:74000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58380 con 0x56203a977800 2023-11-08T16:38:05.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.154+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(89 [pgnls-filter start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.154+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:90 11.1e 11:78000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58a80 con 0x56203a977c00 2023-11-08T16:38:05.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.156+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.156+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:91 11.3e 11:7c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26380 con 0x56203a9f1400 2023-11-08T16:38:05.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.157+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 11 ==== osd_op_reply(91 [pgnls-filter start_epoch 86 out=49b] v84'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.157+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:92 11.1 11:80000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59c00 con 0x56203a9f1000 2023-11-08T16:38:05.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.158+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 14 ==== osd_op_reply(92 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.158+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:93 11.21 11:84000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27880 con 0x56203a9f1c00 2023-11-08T16:38:05.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.159+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 13 ==== osd_op_reply(93 [pgnls-filter start_epoch 86 out=49b] v84'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.159+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:94 11.11 11:88000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58000 con 0x562039e44400 2023-11-08T16:38:05.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.160+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 11 ==== osd_op_reply(94 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.160+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:95 11.31 11:8c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6a000 con 0x56203a9f1400 2023-11-08T16:38:05.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.161+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 12 ==== osd_op_reply(95 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.161+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:96 11.9 11:90000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59500 con 0x562039e44000 2023-11-08T16:38:05.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.162+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 86 out=49b] v84'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.162+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:97 11.29 11:94000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6a380 con 0x56203a977c00 2023-11-08T16:38:05.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.163+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 14 ==== osd_op_reply(97 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.163+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:98 11.19 11:98000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11c00 con 0x56203a9f1400 2023-11-08T16:38:05.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.164+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 13 ==== osd_op_reply(98 [pgnls-filter start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.164+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:99 11.39 11:9c000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6a700 con 0x56203a9f1c00 2023-11-08T16:38:05.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.165+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 14 ==== osd_op_reply(99 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.165+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:100 11.5 11:a0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26000 con 0x56203a977c00 2023-11-08T16:38:05.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.166+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 86 out=49b] v84'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.166+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:101 11.25 11:a4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6aa80 con 0x562039e44000 2023-11-08T16:38:05.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.167+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(101 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.167+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:102 11.15 11:a8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27500 con 0x562039e44400 2023-11-08T16:38:05.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.168+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 12 ==== osd_op_reply(102 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.168+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:103 11.35 11:ac000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6ae00 con 0x56203a977c00 2023-11-08T16:38:05.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.169+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 16 ==== osd_op_reply(103 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.170+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:104 11.d 11:b0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58e00 con 0x562039e44400 2023-11-08T16:38:05.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.170+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 13 ==== osd_op_reply(104 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.170+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:105 11.2d 11:b4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6b180 con 0x56203a9f1000 2023-11-08T16:38:05.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.172+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 15 ==== osd_op_reply(105 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.172+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:106 11.1d 11:b8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58700 con 0x56203a977c00 2023-11-08T16:38:05.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.173+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 17 ==== osd_op_reply(106 [pgnls-filter start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.173+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:107 11.3d 11:bc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6b500 con 0x562039e44000 2023-11-08T16:38:05.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.174+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(107 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.174+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:108 11.3 11:c0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59c00 con 0x56203a9f1000 2023-11-08T16:38:05.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.175+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.6 v2:172.21.15.93:6816/607960134 16 ==== osd_op_reply(108 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x56203a9f1000 2023-11-08T16:38:05.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.175+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:109 11.23 11:c4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6b880 con 0x562039e44000 2023-11-08T16:38:05.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.176+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(109 [pgnls-filter start_epoch 86 out=49b] v84'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.176+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:110 11.13 11:c8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59500 con 0x56203a9f1c00 2023-11-08T16:38:05.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.177+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 15 ==== osd_op_reply(110 [pgnls-filter start_epoch 86 out=49b] v84'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.177+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:111 11.33 11:cc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6bc00 con 0x56203a9f1400 2023-11-08T16:38:05.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.178+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.0 v2:172.21.15.81:6808/1666604866 14 ==== osd_op_reply(111 [pgnls-filter start_epoch 86 out=49b] v84'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x56203a9f1400 2023-11-08T16:38:05.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.178+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:112 11.b 11:d0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e59180 con 0x562039e44000 2023-11-08T16:38:05.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.179+0000 7f8da6d17640 1 -- 172.21.15.81:0/347892247 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(112 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96afc0 con 0x562039e44000 2023-11-08T16:38:05.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.179+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:113 11.2b 11:d4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11880 con 0x56203a9f1800 2023-11-08T16:38:05.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.180+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 13 ==== osd_op_reply(113 [pgnls-filter start_epoch 86 out=49b] v84'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.181+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:114 11.1b 11:d8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58380 con 0x56203a9f1800 2023-11-08T16:38:05.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.182+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 14 ==== osd_op_reply(114 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.182+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:115 11.3b 11:dc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039b11c00 con 0x562039e44400 2023-11-08T16:38:05.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.183+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 14 ==== osd_op_reply(115 [pgnls-filter start_epoch 86 out=49b] v84'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.183+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:116 11.7 11:e0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e58000 con 0x56203a9f1c00 2023-11-08T16:38:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.184+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 16 ==== osd_op_reply(116 [pgnls-filter start_epoch 86 out=81b] v86'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.184+0000 7f8da85715c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-11-08T16:38:05.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.184+0000 7f8da85715c0 10 datascan.operator(): handling object 10000000001.0 2023-11-08T16:38:05.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.184+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:117 11.7 11:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26700 con 0x56203a9f1c00 2023-11-08T16:38:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.185+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.5 v2:172.21.15.93:6802/2406579932 17 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (103) out=103b,getxattr,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+149 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1c00 2023-11-08T16:38:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.185+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:118 12.7 12:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e4e000 con 0x56203a977800 2023-11-08T16:38:05.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.186+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.1 v2:172.21.15.81:6800/2060798739 12 ==== osd_op_reply(118 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x56203a96ab40 con 0x56203a977800 2023-11-08T16:38:05.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.186+0000 7f8da85715c0 20 datascan.operator(): loaded layout from xattr: pi: 12 os: 4194304 sc: 1 su: 4194304 2023-11-08T16:38:05.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.186+0000 7f8da85715c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-11-08T16:38:05.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.186+0000 7f8da85715c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/sixmegs 2023-11-08T16:38:05.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.186+0000 7f8da85715c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=sixmegs 2023-11-08T16:38:05.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.186+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:119 10.10 10:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e4e380 con 0x56203a9f1800 2023-11-08T16:38:05.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.187+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 15 ==== osd_op_reply(119 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.187+0000 7f8da85715c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-11-08T16:38:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.187+0000 7f8da85715c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/sixmegs, will insert to root 2023-11-08T16:38:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.187+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:120 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e4e700 con 0x56203a9f1800 2023-11-08T16:38:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.188+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 16 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.188+0000 7f8da85715c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:38:05.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.188+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:121 10.10 10:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e4ea80 con 0x56203a9f1800 2023-11-08T16:38:05.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.193+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 17 ==== osd_op_reply(121 10000000000.00000000 [create,omap-set-header] v86'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.193+0000 7f8da85715c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-11-08T16:38:05.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.193+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:122 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e4ee00 con 0x56203a9f1800 2023-11-08T16:38:05.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.195+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 18 ==== osd_op_reply(122 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.195+0000 7f8da85715c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-11-08T16:38:05.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.195+0000 7f8da85715c0 20 datascan.read_dentry: sixmegs_head not found in result 2023-11-08T16:38:05.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.195+0000 7f8da85715c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/sixmegs with size=6291456 bytes 2023-11-08T16:38:05.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.195+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:123 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e4f180 con 0x56203a9f1800 2023-11-08T16:38:05.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.198+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 19 ==== osd_op_reply(123 10000000000.00000000 [omap-set-vals] v86'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.198+0000 7f8da85715c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-11-08T16:38:05.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.198+0000 7f8da85715c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-11-08T16:38:05.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.198+0000 7f8da85715c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-11-08T16:38:05.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.198+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:124 10.1f 10:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e4f500 con 0x56203a977c00 2023-11-08T16:38:05.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.200+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 18 ==== osd_op_reply(124 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.200+0000 7f8da85715c0 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-11-08T16:38:05.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.200+0000 7f8da85715c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-11-08T16:38:05.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.200+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:125 10.1f 10:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e4f880 con 0x56203a977c00 2023-11-08T16:38:05.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.200+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 19 ==== osd_op_reply(125 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.200+0000 7f8da85715c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:38:05.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.200+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:126 10.1f 10:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e4fc00 con 0x56203a977c00 2023-11-08T16:38:05.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.204+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 20 ==== osd_op_reply(126 1.00000000 [create,omap-set-header] v86'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.204+0000 7f8da85715c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-11-08T16:38:05.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.204+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:127 10.1f 10:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e4e380 con 0x56203a977c00 2023-11-08T16:38:05.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.205+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 21 ==== osd_op_reply(127 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.205+0000 7f8da85715c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:38:05.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.205+0000 7f8da85715c0 20 datascan.read_dentry: subdir_head not found in result 2023-11-08T16:38:05.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.205+0000 7f8da85715c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-11-08T16:38:05.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.205+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:128 10.1f 10:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e4e700 con 0x56203a977c00 2023-11-08T16:38:05.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.208+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 22 ==== osd_op_reply(128 1.00000000 [omap-set-vals] v86'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.209+0000 7f8da85715c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-11-08T16:38:05.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.209+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:129 11.27 11:e4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26a80 con 0x56203a9f1800 2023-11-08T16:38:05.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.210+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.4 v2:172.21.15.93:6800/3259756279 20 ==== osd_op_reply(129 [pgnls-filter start_epoch 86 out=49b] v84'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a9f1800 2023-11-08T16:38:05.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.210+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:130 11.17 11:e8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27c00 con 0x56203a977c00 2023-11-08T16:38:05.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.211+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 23 ==== osd_op_reply(130 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.211+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:131 11.37 11:ec000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6a380 con 0x562039e44400 2023-11-08T16:38:05.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.212+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 15 ==== osd_op_reply(131 [pgnls-filter start_epoch 86 out=49b] v84'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.212+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:132 11.f 11:f0000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e27180 con 0x56203a977c00 2023-11-08T16:38:05.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.213+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 24 ==== osd_op_reply(132 [pgnls-filter start_epoch 86 out=49b] v84'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.213+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:133 11.2f 11:f4000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6a700 con 0x562039e44400 2023-11-08T16:38:05.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.213+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 16 ==== osd_op_reply(133 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.214+0000 7f8da1d0d640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:134 11.1f 11:f8000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e26e00 con 0x562039e44400 2023-11-08T16:38:05.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.214+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 <== osd.2 v2:172.21.15.81:6824/3759311428 17 ==== osd_op_reply(134 [pgnls-filter start_epoch 86 out=49b] v84'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ab40 con 0x562039e44400 2023-11-08T16:38:05.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.214+0000 7f8da250e640 1 -- 172.21.15.81:0/347892247 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:135 11.3f 11:fc000000::::head [pgnls-filter start_epoch 86 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x562039e6ae00 con 0x56203a977c00 2023-11-08T16:38:05.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.215+0000 7f8da7518640 1 -- 172.21.15.81:0/347892247 <== osd.7 v2:172.21.15.93:6818/433691658 25 ==== osd_op_reply(135 [pgnls-filter start_epoch 86 out=49b] v84'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x56203a96ad80 con 0x56203a977c00 2023-11-08T16:38:05.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56203a9f1400 msgr2=0x562039e30580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56203a9f1400 0x562039e30580 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56203a977800 msgr2=0x562039b24c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x56203a977800 0x562039b24c00 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562039e44400 msgr2=0x562039e31b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562039e44400 0x562039e31b80 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562039e44000 msgr2=0x562039e31600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562039e44000 0x562039e31600 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56203a9f1800 msgr2=0x562039e30b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56203a9f1800 0x562039e30b00 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56203a9f1c00 msgr2=0x562039e31080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x56203a9f1c00 0x562039e31080 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56203a9f1000 msgr2=0x562039e30000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x56203a9f1000 0x562039e30000 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56203a977c00 msgr2=0x562039b25180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x56203a977c00 0x562039b25180 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 reap_dead start 2023-11-08T16:38:05.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 reap_dead start 2023-11-08T16:38:05.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56203a9f0c00 msgr2=0x562039b25700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da6516640 1 -- 172.21.15.81:0/347892247 reap_dead start 2023-11-08T16:38:05.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56203a9f0c00 0x562039b25700 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x562039baa180 tx=0x56203a996cc0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 msgr2=0x562039b24100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.216+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x56203a994840 tx=0x56203a996c00 comp rx=0 tx=0).stop 2023-11-08T16:38:05.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.217+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 shutdown_connections 2023-11-08T16:38:05.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.217+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x56203a9f0c00 0x562039b25700 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.217+0000 7f8da85715c0 1 --2- 172.21.15.81:0/347892247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56203a977000 0x562039b24100 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.217+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 >> 172.21.15.81:0/347892247 conn(0x562039b98000 msgr2=0x56203a843800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:05.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.217+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 shutdown_connections 2023-11-08T16:38:05.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.217+0000 7f8da85715c0 1 -- 172.21.15.81:0/347892247 wait complete. 2023-11-08T16:38:05.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.218+0000 7f8da85715c0 1 -- 172.21.15.81:0/4099366890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562039b98400 msgr2=0x562039b23600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.218+0000 7f8da85715c0 1 --2- 172.21.15.81:0/4099366890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562039b98400 0x562039b23600 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x56203a8c5260 tx=0x56203a970300 comp rx=0 tx=0).stop 2023-11-08T16:38:05.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.218+0000 7f8da85715c0 1 -- 172.21.15.81:0/4099366890 shutdown_connections 2023-11-08T16:38:05.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.218+0000 7f8da85715c0 1 --2- 172.21.15.81:0/4099366890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562039b98800 0x562039b23080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.219+0000 7f8da85715c0 1 --2- 172.21.15.81:0/4099366890 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562039ab9c00 0x562039b23b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.219+0000 7f8da85715c0 1 --2- 172.21.15.81:0/4099366890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562039b98400 0x562039b23600 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.219+0000 7f8da85715c0 1 -- 172.21.15.81:0/4099366890 >> 172.21.15.81:0/4099366890 conn(0x562039ab9000 msgr2=0x56203a843000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:05.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.219+0000 7f8da85715c0 1 -- 172.21.15.81:0/4099366890 shutdown_connections 2023-11-08T16:38:05.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.219+0000 7f8da85715c0 1 -- 172.21.15.81:0/4099366890 wait complete. 2023-11-08T16:38:05.229 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.282765, result: 2023-11-08T16:38:05.230 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-11-08T16:38:05.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.332+0000 7f4fb34385c0 1 -- start start 2023-11-08T16:38:05.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.332+0000 7f4fb34385c0 1 Processor -- start 2023-11-08T16:38:05.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.333+0000 7f4fb34385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91eb74c00 0x55f91ebde580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.333+0000 7f4fb34385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91eb75800 0x55f91ebdeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.333+0000 7f4fb34385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91eb75c00 0x55f91ebdf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.333+0000 7f4fb34385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ebbd180 con 0x55f91eb74c00 2023-11-08T16:38:05.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.333+0000 7f4fb34385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ebbca00 con 0x55f91eb75800 2023-11-08T16:38:05.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.333+0000 7f4fb34385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f91ebbc140 con 0x55f91eb75c00 2023-11-08T16:38:05.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.333+0000 7f4fb1388640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91eb74c00 0x55f91ebde580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.333+0000 7f4fb1388640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91eb74c00 0x55f91ebde580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59596/0 (socket says 172.21.15.81:59596) 2023-11-08T16:38:05.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.333+0000 7f4fb1388640 1 -- 172.21.15.81:0/271035608 learned_addr learned my addr 172.21.15.81:0/271035608 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:05.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/271035608 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91eb75800 0x55f91ebdeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/271035608 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91eb75c00 0x55f91ebdf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4fb1388640 1 -- 172.21.15.81:0/271035608 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91eb75c00 msgr2=0x55f91ebdf080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4fb1388640 1 --2- 172.21.15.81:0/271035608 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91eb75c00 0x55f91ebdf080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4fb1388640 1 -- 172.21.15.81:0/271035608 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91eb75800 msgr2=0x55f91ebdeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4fb1388640 1 --2- 172.21.15.81:0/271035608 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91eb75800 0x55f91ebdeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4fb1388640 1 -- 172.21.15.81:0/271035608 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91eb9a9c0 con 0x55f91eb74c00 2023-11-08T16:38:05.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4fb1388640 1 --2- 172.21.15.81:0/271035608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91eb74c00 0x55f91ebde580 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x55f91f987590 tx=0x55f91fa2a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1c7870bde0660 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4faeb83640 1 -- 172.21.15.81:0/271035608 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f91ebbb1e0 con 0x55f91eb74c00 2023-11-08T16:38:05.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4faeb83640 1 -- 172.21.15.81:0/271035608 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f91ebce480 con 0x55f91eb74c00 2023-11-08T16:38:05.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4faeb83640 1 -- 172.21.15.81:0/271035608 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f91ebbb080 con 0x55f91eb74c00 2023-11-08T16:38:05.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4fb34385c0 1 -- 172.21.15.81:0/271035608 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91eb9a820 con 0x55f91eb74c00 2023-11-08T16:38:05.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4fb34385c0 1 -- 172.21.15.81:0/271035608 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f91eb9a4e0 con 0x55f91eb74c00 2023-11-08T16:38:05.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.334+0000 7f4fb34385c0 4 waiting for MDS map... 2023-11-08T16:38:05.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.335+0000 7f4faeb83640 1 -- 172.21.15.81:0/271035608 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x55f91ec34c40 con 0x55f91eb74c00 2023-11-08T16:38:05.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.335+0000 7f4faeb83640 1 -- 172.21.15.81:0/271035608 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 71) v1 ==== 3306+0+0 (secure 0 0 0) 0x55f91ebc4f00 con 0x55f91eb74c00 2023-11-08T16:38:05.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.335+0000 7f4fb34385c0 4 Got MDS map 71 2023-11-08T16:38:05.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.335+0000 7f4fb34385c0 4 datascan.main: Using metadata pool output 2023-11-08T16:38:05.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.335+0000 7f4fb34385c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:38:05.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.336+0000 7f4fb34385c0 1 Processor -- start 2023-11-08T16:38:05.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.336+0000 7f4fb34385c0 1 -- start start 2023-11-08T16:38:05.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb34385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91fa2f000 0x55f91ebdf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb34385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91fa2f400 0x55f91ebdfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb34385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb34385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ebbc000 con 0x55f91fa2f800 2023-11-08T16:38:05.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb34385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ec43180 con 0x55f91fa2f400 2023-11-08T16:38:05.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb34385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f91ec42dc0 con 0x55f91fa2f000 2023-11-08T16:38:05.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb0b87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb0b87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59602/0 (socket says 172.21.15.81:59602) 2023-11-08T16:38:05.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2332806202 learned_addr learned my addr 172.21.15.81:0/2332806202 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:05.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2332806202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91fa2f000 msgr2=0x55f91ebdf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/2332806202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91fa2f000 0x55f91ebdf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2332806202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91fa2f400 0x55f91ebdfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.337+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2332806202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91fa2f400 msgr2=0x55f91ebdfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.338+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/2332806202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91fa2f400 0x55f91ebdfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.338+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2332806202 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91eb9a1a0 con 0x55f91fa2f800 2023-11-08T16:38:05.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.338+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/2332806202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0100 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x55f91f986a80 tx=0x55f91fa2a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce062604333e6cf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.338+0000 7f4facb7f640 1 -- 172.21.15.81:0/2332806202 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f91ebbb1e0 con 0x55f91fa2f800 2023-11-08T16:38:05.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.338+0000 7f4facb7f640 1 -- 172.21.15.81:0/2332806202 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f91f950f00 con 0x55f91fa2f800 2023-11-08T16:38:05.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.338+0000 7f4facb7f640 1 -- 172.21.15.81:0/2332806202 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f91ebbb080 con 0x55f91fa2f800 2023-11-08T16:38:05.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.338+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2332806202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 msgr2=0x55f91ebe0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.338+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2332806202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0100 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x55f91f986a80 tx=0x55f91fa2a120 comp rx=0 tx=0).stop 2023-11-08T16:38:05.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.338+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2332806202 shutdown_connections 2023-11-08T16:38:05.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.339+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2332806202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0100 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.339+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2332806202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91fa2f400 0x55f91ebdfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.339+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2332806202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91fa2f000 0x55f91ebdf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.339+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2332806202 >> 172.21.15.81:0/2332806202 conn(0x55f91fa2ec00 msgr2=0x55f91f8f1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:05.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.339+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2332806202 shutdown_connections 2023-11-08T16:38:05.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.339+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2332806202 wait complete. 2023-11-08T16:38:05.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.341+0000 7f4fb34385c0 1 Processor -- start 2023-11-08T16:38:05.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.341+0000 7f4fb34385c0 1 -- start start 2023-11-08T16:38:05.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.341+0000 7f4fb34385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91fa2f000 0x55f91ebe1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.341+0000 7f4fb34385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91fa2f400 0x55f91ebe0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.341+0000 7f4fb34385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.341+0000 7f4fb34385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ec42c80 con 0x55f91fa2f800 2023-11-08T16:38:05.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.341+0000 7f4fb34385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ebbca00 con 0x55f91fa2f400 2023-11-08T16:38:05.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.341+0000 7f4fb34385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f91ebbcb40 con 0x55f91fa2f000 2023-11-08T16:38:05.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb0b87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb0b87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59610/0 (socket says 172.21.15.81:59610) 2023-11-08T16:38:05.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 learned_addr learned my addr 172.21.15.81:0/2892982243 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:05.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91fa2f000 0x55f91ebe1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91fa2f400 0x55f91ebe0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91fa2f000 msgr2=0x55f91ebe1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91fa2f000 0x55f91ebe1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91fa2f400 msgr2=0x55f91ebe0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91fa2f400 0x55f91ebe0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91eb9ad00 con 0x55f91fa2f800 2023-11-08T16:38:05.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91fa2f000 0x55f91ebe1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:05.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.342+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0680 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x55f91fa42bd0 tx=0x55f91fa2a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.343+0000 7f4fab37c640 1 -- 172.21.15.81:0/2892982243 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f91ebbb1e0 con 0x55f91fa2f800 2023-11-08T16:38:05.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.343+0000 7f4fab37c640 1 -- 172.21.15.81:0/2892982243 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f91f950d80 con 0x55f91fa2f800 2023-11-08T16:38:05.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.343+0000 7f4fab37c640 1 -- 172.21.15.81:0/2892982243 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f91ebbb080 con 0x55f91fa2f800 2023-11-08T16:38:05.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.343+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f91eb9a1a0 con 0x55f91fa2f800 2023-11-08T16:38:05.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.343+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91eb9a9c0 con 0x55f91fa2f800 2023-11-08T16:38:05.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.343+0000 7f4fb34385c0 4 datascan.init: resolving metadata pool 10 2023-11-08T16:38:05.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.345+0000 7f4fab37c640 1 -- 172.21.15.81:0/2892982243 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55f91ebd1080 con 0x55f91fa2f800 2023-11-08T16:38:05.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.345+0000 7f4fab37c640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55f91fa2fc00 0x55f91ebe1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.345+0000 7f4fab37c640 1 -- 172.21.15.81:0/2892982243 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x55f91ec34a80 con 0x55f91fa2f800 2023-11-08T16:38:05.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.346+0000 7f4fb34385c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:38:05.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.346+0000 7f4fb34385c0 4 datascan.main: resolving metadata pool 10 2023-11-08T16:38:05.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.346+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55f91eece000 0x55f91ebe0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.346+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55f91fa2fc00 0x55f91ebe1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.346+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:1 10.0 10:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eece000 2023-11-08T16:38:05.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.346+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55f91eece000 0x55f91ebe0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.346+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55f91fa2fc00 0x55f91ebe1700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x55f91f9873b0 tx=0x55f91fa2d1a0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.347+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55f91eece000 0x55f91ebe0100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.348+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(1 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.348+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55f91fa2e400 0x55f91eed8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.348+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:2 10.10 10:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91fa2e400 2023-11-08T16:38:05.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.349+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55f91fa2e400 0x55f91eed8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.349+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55f91fa2e400 0x55f91eed8000 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.350+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(2 [pgnls start_epoch 86 out=81b] v86'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.350+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55f91eecfc00 0x55f91eed8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.350+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:3 10.8 10:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eecfc00 2023-11-08T16:38:05.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.351+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55f91eecfc00 0x55f91eed8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.351+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55f91eecfc00 0x55f91eed8580 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.352+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(3 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eecfc00 2023-11-08T16:38:05.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.352+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55f91fa2e800 0x55f91eed8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.352+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:4 10.18 10:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91fa2e800 2023-11-08T16:38:05.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.352+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55f91fa2e800 0x55f91eed8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.352+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55f91fa2e800 0x55f91eed8b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.353+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(4 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91fa2e800 2023-11-08T16:38:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.353+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 10.4 10:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eecfc00 2023-11-08T16:38:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.353+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(5 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eecfc00 2023-11-08T16:38:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.353+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55f91eee8000 0x55f91eed9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.353+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:6 10.14 10:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eee8000 2023-11-08T16:38:05.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.354+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55f91eee8000 0x55f91eed9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.354+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55f91eee8000 0x55f91eed9080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.354+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(6 [pgnls start_epoch 86 out=125b] v86'29 uv29 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.354+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55f91eee9000 0x55f91eed9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.354+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:7 10.c 10:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eee9000 2023-11-08T16:38:05.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.355+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55f91eee9000 0x55f91eed9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.356+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55f91eee9000 0x55f91eed9600 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.357+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(7 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eee9000 2023-11-08T16:38:05.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.357+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:8 10.1c 10:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eece000 2023-11-08T16:38:05.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.358+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(8 [pgnls start_epoch 86 out=49b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.358+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:9 10.2 10:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eee8000 2023-11-08T16:38:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.358+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(9 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.358+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:10 10.12 10:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eece000 2023-11-08T16:38:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.359+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(10 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.359+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:11 10.a 10:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eece000 2023-11-08T16:38:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.360+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(11 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.360+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:12 10.1a 10:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eece000 2023-11-08T16:38:05.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.360+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(12 [pgnls start_epoch 86 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.360+0000 7f4fb1388640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55f91eee8400 0x55f91eeda100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.360+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:13 10.6 10:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eee8400 2023-11-08T16:38:05.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.361+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55f91eee8400 0x55f91eeda100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.362+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55f91eee8400 0x55f91eeda100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.363+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(13 [pgnls start_epoch 86 out=49b] v85'6 uv6 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91eee8400 2023-11-08T16:38:05.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.363+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:14 10.16 10:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eece000 2023-11-08T16:38:05.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.364+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(14 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.364+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:15 10.e 10:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91fa2e400 2023-11-08T16:38:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.365+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(15 [pgnls start_epoch 86 out=49b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.365+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:16 10.1e 10:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eecfc00 2023-11-08T16:38:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.365+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(16 [pgnls start_epoch 86 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eecfc00 2023-11-08T16:38:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.365+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:17 10.1 10:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91fa2e400 2023-11-08T16:38:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.366+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(17 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.366+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:18 10.11 10:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eee8400 2023-11-08T16:38:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.367+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(18 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91eee8400 2023-11-08T16:38:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.367+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:19 10.9 10:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eee8400 2023-11-08T16:38:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.368+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(19 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91eee8400 2023-11-08T16:38:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.368+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:20 10.19 10:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eee8000 2023-11-08T16:38:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.369+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(20 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.369+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:21 10.5 10:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91fa2e800 2023-11-08T16:38:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.369+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(21 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91fa2e800 2023-11-08T16:38:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.369+0000 7f4fb0b87640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55f91ec55400 0x55f91eedb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.369+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:22 10.15 10:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91ec55400 2023-11-08T16:38:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.370+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55f91ec55400 0x55f91eedb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.370+0000 7f4fb1b89640 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55f91ec55400 0x55f91eedb180 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.371+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(22 [pgnls start_epoch 86 out=73b] v85'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f91f946d80 con 0x55f91ec55400 2023-11-08T16:38:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.371+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:23 10.d 10:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eece000 2023-11-08T16:38:05.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.372+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(23 [pgnls start_epoch 86 out=74b] v86'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.372+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:24 10.1d 10:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91fa2e400 2023-11-08T16:38:05.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.373+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(24 [pgnls start_epoch 86 out=49b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.373+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:25 10.3 10:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91fa2e800 2023-11-08T16:38:05.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.373+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(25 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91fa2e800 2023-11-08T16:38:05.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.373+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:26 10.13 10:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eee8000 2023-11-08T16:38:05.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.374+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(26 [pgnls start_epoch 86 out=98b] v86'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.374+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:27 10.b 10:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eece000 2023-11-08T16:38:05.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.375+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(27 [pgnls start_epoch 86 out=76b] v86'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.375+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:28 10.1b 10:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91ec55400 2023-11-08T16:38:05.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.376+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(28 [pgnls start_epoch 86 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91ec55400 2023-11-08T16:38:05.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.376+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:29 10.7 10:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eece000 2023-11-08T16:38:05.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.377+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(29 [pgnls start_epoch 86 out=49b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.377+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:30 10.17 10:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91fa2e400 2023-11-08T16:38:05.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.378+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(30 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.378+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:31 10.f 10:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eece000 2023-11-08T16:38:05.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.379+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(31 [pgnls start_epoch 86 out=73b] v86'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.379+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:32 10.1f 10:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcd880 con 0x55f91eee9000 2023-11-08T16:38:05.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.380+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(32 [pgnls start_epoch 86 out=101b] v86'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eee9000 2023-11-08T16:38:05.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.380+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-11-08T16:38:05.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.380+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:33 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ebcdc00 con 0x55f91fa2e400 2023-11-08T16:38:05.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+509 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:38:05.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-11-08T16:38:05.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-08T16:38:05.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-11-08T16:38:05.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-08T16:38:05.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-11-08T16:38:05.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-08T16:38:05.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-11-08T16:38:05.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-08T16:38:05.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-11-08T16:38:05.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-08T16:38:05.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-11-08T16:38:05.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-08T16:38:05.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-11-08T16:38:05.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.381+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:34 10.13 10:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02000 con 0x55f91eee8000 2023-11-08T16:38:05.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.382+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.382+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-11-08T16:38:05.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.382+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-08T16:38:05.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.382+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-11-08T16:38:05.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.382+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-08T16:38:05.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.382+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-11-08T16:38:05.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.382+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-08T16:38:05.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.382+0000 7f4fb34385c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-11-08T16:38:05.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.382+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:35 10.1f 10:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eee9000 2023-11-08T16:38:05.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.383+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eee9000 2023-11-08T16:38:05.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.383+0000 7f4fb34385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:38:05.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.383+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:36 10.0 10:00000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02700 con 0x55f91eece000 2023-11-08T16:38:05.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.384+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 11 ==== osd_op_reply(36 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.384+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:37 10.10 10:08000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91fa2e400 2023-11-08T16:38:05.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.385+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(37 [pgnls start_epoch 86 out=81b] v86'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.386+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:38 10.8 10:10000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eecfc00 2023-11-08T16:38:05.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.386+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(38 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eecfc00 2023-11-08T16:38:05.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.386+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:39 10.18 10:18000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91fa2e800 2023-11-08T16:38:05.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.387+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(39 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91fa2e800 2023-11-08T16:38:05.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.387+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:40 10.4 10:20000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eecfc00 2023-11-08T16:38:05.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.388+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(40 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eecfc00 2023-11-08T16:38:05.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.388+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:41 10.14 10:28000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eee8000 2023-11-08T16:38:05.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.388+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(41 [pgnls start_epoch 86 out=125b] v86'29 uv29 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.388+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:42 10.c 10:30000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eee9000 2023-11-08T16:38:05.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.390+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(42 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eee9000 2023-11-08T16:38:05.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.390+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:43 10.1c 10:38000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eece000 2023-11-08T16:38:05.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.391+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 12 ==== osd_op_reply(43 [pgnls start_epoch 86 out=49b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.391+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:44 10.2 10:40000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eee8000 2023-11-08T16:38:05.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.391+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(44 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.391+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:45 10.12 10:48000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eece000 2023-11-08T16:38:05.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.393+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 13 ==== osd_op_reply(45 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.393+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:46 10.a 10:50000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eece000 2023-11-08T16:38:05.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.394+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 14 ==== osd_op_reply(46 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.394+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:47 10.1a 10:58000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eece000 2023-11-08T16:38:05.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.395+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 15 ==== osd_op_reply(47 [pgnls start_epoch 86 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.395+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:48 10.6 10:60000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eee8400 2023-11-08T16:38:05.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.396+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(48 [pgnls start_epoch 86 out=49b] v85'6 uv6 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91eee8400 2023-11-08T16:38:05.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.396+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:49 10.16 10:68000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eece000 2023-11-08T16:38:05.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.397+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 16 ==== osd_op_reply(49 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.397+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:50 10.e 10:70000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91fa2e400 2023-11-08T16:38:05.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.398+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(50 [pgnls start_epoch 86 out=49b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.398+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:51 10.1e 10:78000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eecfc00 2023-11-08T16:38:05.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.399+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(51 [pgnls start_epoch 86 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eecfc00 2023-11-08T16:38:05.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.399+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:52 10.1 10:80000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91fa2e400 2023-11-08T16:38:05.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.401+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(52 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.401+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:53 10.11 10:88000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eee8400 2023-11-08T16:38:05.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.401+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(53 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91eee8400 2023-11-08T16:38:05.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.401+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:54 10.9 10:90000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eee8400 2023-11-08T16:38:05.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.402+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(54 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91eee8400 2023-11-08T16:38:05.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.402+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:55 10.19 10:98000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eee8000 2023-11-08T16:38:05.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.403+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(55 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.403+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:56 10.5 10:a0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91fa2e800 2023-11-08T16:38:05.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.404+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.2 v2:172.21.15.81:6824/3759311428 5 ==== osd_op_reply(56 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91fa2e800 2023-11-08T16:38:05.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.404+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:57 10.15 10:a8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91ec55400 2023-11-08T16:38:05.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.405+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(57 [pgnls start_epoch 86 out=73b] v85'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f91f946d80 con 0x55f91ec55400 2023-11-08T16:38:05.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.405+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:58 10.d 10:b0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eece000 2023-11-08T16:38:05.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.406+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 17 ==== osd_op_reply(58 [pgnls start_epoch 86 out=74b] v86'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.406+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:59 10.1d 10:b8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91fa2e400 2023-11-08T16:38:05.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.408+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(59 [pgnls start_epoch 86 out=49b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.408+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:60 10.3 10:c0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91fa2e800 2023-11-08T16:38:05.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.408+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 <== osd.2 v2:172.21.15.81:6824/3759311428 6 ==== osd_op_reply(60 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946fc0 con 0x55f91fa2e800 2023-11-08T16:38:05.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.408+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:61 10.13 10:c8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eee8000 2023-11-08T16:38:05.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.409+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(61 [pgnls start_epoch 86 out=98b] v86'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.409+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:62 10.b 10:d0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eece000 2023-11-08T16:38:05.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.410+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 18 ==== osd_op_reply(62 [pgnls start_epoch 86 out=76b] v86'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.410+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:63 10.1b 10:d8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91ec55400 2023-11-08T16:38:05.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.411+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(63 [pgnls start_epoch 86 out=49b] v86'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91ec55400 2023-11-08T16:38:05.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.411+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:64 10.7 10:e0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eece000 2023-11-08T16:38:05.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.412+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 19 ==== osd_op_reply(64 [pgnls start_epoch 86 out=49b] v85'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.412+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:65 10.17 10:e8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91fa2e400 2023-11-08T16:38:05.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.413+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(65 [pgnls start_epoch 86 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.413+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:66 10.f 10:f0000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eece000 2023-11-08T16:38:05.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.414+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 20 ==== osd_op_reply(66 [pgnls start_epoch 86 out=73b] v86'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.414+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:67 10.1f 10:f8000000::::head [pgnls start_epoch 86 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eee9000 2023-11-08T16:38:05.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.415+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(67 [pgnls start_epoch 86 out=101b] v86'12 uv12 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eee9000 2023-11-08T16:38:05.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.415+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-11-08T16:38:05.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.415+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:68 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02a80 con 0x55f91fa2e400 2023-11-08T16:38:05.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=509b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+509 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:38:05.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-11-08T16:38:05.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-11-08T16:38:05.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-08T16:38:05.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-11-08T16:38:05.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-08T16:38:05.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-11-08T16:38:05.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-08T16:38:05.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-11-08T16:38:05.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-08T16:38:05.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-11-08T16:38:05.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-08T16:38:05.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-11-08T16:38:05.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-08T16:38:05.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-11-08T16:38:05.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.417+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:69 10.13 10:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02700 con 0x55f91eee8000 2023-11-08T16:38:05.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.418+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.418+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-11-08T16:38:05.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.418+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-08T16:38:05.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.418+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-11-08T16:38:05.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.418+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-08T16:38:05.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.418+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-11-08T16:38:05.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.418+0000 7f4fb34385c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-08T16:38:05.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.418+0000 7f4fb34385c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-11-08T16:38:05.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.418+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:70 10.1f 10:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02000 con 0x55f91eee9000 2023-11-08T16:38:05.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.419+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=508b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eee9000 2023-11-08T16:38:05.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.419+0000 7f4fb34385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:38:05.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.419+0000 7f4fb34385c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-11-08T16:38:05.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.419+0000 7f4fb34385c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-11-08T16:38:05.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.419+0000 7f4fb34385c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-11-08T16:38:05.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.419+0000 7f4fb34385c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-11-08T16:38:05.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.419+0000 7f4fb34385c0 10 datascan.scan_links: processing 2 injected_inos 2023-11-08T16:38:05.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.419+0000 7f4fb34385c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-11-08T16:38:05.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.419+0000 7f4fb34385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-11-08T16:38:05.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.419+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:71 10.1f 10:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02e00 con 0x55f91eee9000 2023-11-08T16:38:05.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.420+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=507b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+507 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eee9000 2023-11-08T16:38:05.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.421+0000 7f4fb34385c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:38:05.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.421+0000 7f4fb34385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:38:05.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.421+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:72 10.1f 10:ff5b34d6:::1.00000000:head [omap-set-vals in=507b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef03180 con 0x55f91eee9000 2023-11-08T16:38:05.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.424+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v86'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eee9000 2023-11-08T16:38:05.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.424+0000 7f4fb34385c0 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-11-08T16:38:05.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.424+0000 7f4fb34385c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-11-08T16:38:05.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.424+0000 7f4fb34385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:sixmegs 2023-11-08T16:38:05.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.424+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:73 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef03500 con 0x55f91fa2e400 2023-11-08T16:38:05.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.425+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 13 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=508b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+508 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.425+0000 7f4fb34385c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-11-08T16:38:05.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.425+0000 7f4fb34385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:38:05.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.425+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:74 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02a80 con 0x55f91fa2e400 2023-11-08T16:38:05.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.430+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.4 v2:172.21.15.93:6800/3259756279 14 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v86'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f91f946d80 con 0x55f91fa2e400 2023-11-08T16:38:05.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.430+0000 7f4fb34385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-11-08T16:38:05.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.430+0000 7f4fb34385c0 10 datascan.scan_links: updating inotable 2023-11-08T16:38:05.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.430+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:75 10.13 10:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02700 con 0x55f91eee8000 2023-11-08T16:38:05.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.431+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(75 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.431+0000 7f4fb34385c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-11-08T16:38:05.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.431+0000 7f4fb34385c0 10 mds.0.inotable: skip_inos now [0x10000000002~0xfffffffffe] 2023-11-08T16:38:05.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.431+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:76 10.13 10:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef03500 con 0x55f91eee8000 2023-11-08T16:38:05.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.435+0000 7f4fb1b89640 1 -- 172.21.15.81:0/2892982243 <== osd.1 v2:172.21.15.81:6800/2060798739 12 ==== osd_op_reply(76 mds0_inotable [writefull 0~34] v86'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55f91f946d80 con 0x55f91eee8000 2023-11-08T16:38:05.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.435+0000 7f4fb34385c0 10 datascan.scan_links: updating snaptable 2023-11-08T16:38:05.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.435+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:77 10.d 10:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86) v8 -- 0x55f91ef02380 con 0x55f91eece000 2023-11-08T16:38:05.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.436+0000 7f4fb1388640 1 -- 172.21.15.81:0/2892982243 <== osd.6 v2:172.21.15.93:6816/607960134 21 ==== osd_op_reply(77 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55f91f946b40 con 0x55f91eece000 2023-11-08T16:38:05.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55f91eecfc00 msgr2=0x55f91eed8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55f91eecfc00 0x55f91eed8580 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55f91eee8000 msgr2=0x55f91eed9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55f91eee8000 0x55f91eed9080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55f91fa2e800 msgr2=0x55f91eed8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55f91fa2e800 0x55f91eed8b00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55f91ec55400 msgr2=0x55f91eedb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55f91ec55400 0x55f91eedb180 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55f91fa2e400 msgr2=0x55f91eed8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55f91fa2e400 0x55f91eed8000 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55f91eee8400 msgr2=0x55f91eeda100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55f91eee8400 0x55f91eeda100 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55f91eece000 msgr2=0x55f91ebe0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55f91eece000 0x55f91ebe0100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 reap_dead start 2023-11-08T16:38:05.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55f91eee9000 msgr2=0x55f91eed9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55f91eee9000 0x55f91eed9600 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 reap_dead start 2023-11-08T16:38:05.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 reap_dead start 2023-11-08T16:38:05.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb0b87640 1 -- 172.21.15.81:0/2892982243 reap_dead start 2023-11-08T16:38:05.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55f91fa2fc00 msgr2=0x55f91ebe1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55f91fa2fc00 0x55f91ebe1700 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x55f91f9873b0 tx=0x55f91fa2d1a0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 msgr2=0x55f91ebe0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.437+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0680 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x55f91fa42bd0 tx=0x55f91fa2a060 comp rx=0 tx=0).stop 2023-11-08T16:38:05.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.438+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 shutdown_connections 2023-11-08T16:38:05.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.438+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55f91fa2fc00 0x55f91ebe1700 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.438+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/2892982243 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91fa2f800 0x55f91ebe0680 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.438+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 >> 172.21.15.81:0/2892982243 conn(0x55f91fa2ec00 msgr2=0x55f91f8f1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:05.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.438+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 shutdown_connections 2023-11-08T16:38:05.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.438+0000 7f4fb34385c0 1 -- 172.21.15.81:0/2892982243 wait complete. 2023-11-08T16:38:05.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.439+0000 7f4fb34385c0 1 -- 172.21.15.81:0/271035608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91eb74c00 msgr2=0x55f91ebde580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.439+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/271035608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91eb74c00 0x55f91ebde580 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x55f91f987590 tx=0x55f91fa2a3c0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.439+0000 7f4fb34385c0 1 -- 172.21.15.81:0/271035608 shutdown_connections 2023-11-08T16:38:05.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.439+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/271035608 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f91eb75c00 0x55f91ebdf080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.439+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/271035608 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f91eb75800 0x55f91ebdeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.439+0000 7f4fb34385c0 1 --2- 172.21.15.81:0/271035608 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f91eb74c00 0x55f91ebde580 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.439+0000 7f4fb34385c0 1 -- 172.21.15.81:0/271035608 >> 172.21.15.81:0/271035608 conn(0x55f91eb75000 msgr2=0x55f91f8f0800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:05.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.440+0000 7f4fb34385c0 1 -- 172.21.15.81:0/271035608 shutdown_connections 2023-11-08T16:38:05.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.440+0000 7f4fb34385c0 1 -- 172.21.15.81:0/271035608 wait complete. 2023-11-08T16:38:05.452 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.222902, result: 2023-11-08T16:38:05.453 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-08T16:38:05.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.672+0000 7f8623ae0640 1 Processor -- start 2023-11-08T16:38:05.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.672+0000 7f8623ae0640 1 -- start start 2023-11-08T16:38:05.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.674+0000 7f8623ae0640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f861c0cd2a0 0x7f861c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.674+0000 7f8623ae0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c90f0 0x7f861c06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.674+0000 7f8623ae0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f861c06fa00 con 0x7f861c0cc8e0 2023-11-08T16:38:05.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.674+0000 7f8623ae0640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f861c06b620 con 0x7f861c0cd2a0 2023-11-08T16:38:05.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.674+0000 7f8623ae0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861c0caec0 con 0x7f861c0c90f0 2023-11-08T16:38:05.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.674+0000 7f862187c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f861c0cc8e0 0x7f861c164500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50380/0 (socket says 172.21.15.81:50380) 2023-11-08T16:38:05.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.674+0000 7f862187c640 1 -- 172.21.15.81:0/3165611158 learned_addr learned my addr 172.21.15.81:0/3165611158 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:05.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.674+0000 7f862107b640 1 --2- 172.21.15.81:0/3165611158 >> v2:172.21.15.93:3300/0 conn(0x7f861c0cd2a0 0x7f861c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.674+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150605469 0 0) 0x7f861c06fa00 con 0x7f861c0cc8e0 2023-11-08T16:38:05.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.674+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8608003520 con 0x7f861c0cc8e0 2023-11-08T16:38:05.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.674+0000 7f862207d640 1 --2- 172.21.15.81:0/3165611158 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c90f0 0x7f861c06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1253800720 0 0) 0x7f8608003520 con 0x7f861c0cc8e0 2023-11-08T16:38:05.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c90f0 msgr2=0x7f861c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 --2- 172.21.15.81:0/3165611158 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c90f0 0x7f861c06bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 >> v2:172.21.15.93:3300/0 conn(0x7f861c0cd2a0 msgr2=0x7f861c06bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 --2- 172.21.15.81:0/3165611158 >> v2:172.21.15.93:3300/0 conn(0x7f861c0cd2a0 0x7f861c06bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c0cb1a0 con 0x7f861c0cc8e0 2023-11-08T16:38:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f86180017f0 con 0x7f861c0cc8e0 2023-11-08T16:38:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 >> v1:172.21.15.81:6789/0 conn(0x7f861c0cc8e0 legacy=0x7f861c164500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 --2- 172.21.15.81:0/3165611158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 0x7f8608003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 --2- 172.21.15.81:0/3165611158 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c90f0 0x7f86080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 --2- 172.21.15.81:0/3165611158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8608004d00 0x7f8608005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c06b620 con 0x7f8608003970 2023-11-08T16:38:05.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c0caec0 con 0x7f8608004d00 2023-11-08T16:38:05.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f860800b980 con 0x7f861c0c90f0 2023-11-08T16:38:05.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.675+0000 7f862187c640 1 --2- 172.21.15.81:0/3165611158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 0x7f8608003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.676+0000 7f862207d640 1 --2- 172.21.15.81:0/3165611158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8608004d00 0x7f8608005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.676+0000 7f862187c640 1 -- 172.21.15.81:0/3165611158 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c90f0 msgr2=0x7f86080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.676+0000 7f862187c640 1 --2- 172.21.15.81:0/3165611158 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c90f0 0x7f86080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.676+0000 7f862187c640 1 -- 172.21.15.81:0/3165611158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8608004d00 msgr2=0x7f8608005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:05.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.676+0000 7f862187c640 1 --2- 172.21.15.81:0/3165611158 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8608004d00 0x7f8608005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.676+0000 7f862187c640 1 -- 172.21.15.81:0/3165611158 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8608003520 con 0x7f8608003970 2023-11-08T16:38:05.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.677+0000 7f862187c640 1 --2- 172.21.15.81:0/3165611158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 0x7f8608003d90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f861800c210 tx=0x7f86080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=270301f050dd0ab0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.677+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861802cad0 con 0x7f8608003970 2023-11-08T16:38:05.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.677+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f861802cc30 con 0x7f8608003970 2023-11-08T16:38:05.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.677+0000 7f862087a640 1 -- 172.21.15.81:0/3165611158 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8618007070 con 0x7f8608003970 2023-11-08T16:38:05.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.678+0000 7f8623ae0640 1 -- 172.21.15.81:0/3165611158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 msgr2=0x7f8608003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.678+0000 7f8623ae0640 1 --2- 172.21.15.81:0/3165611158 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 0x7f8608003d90 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f861800c210 tx=0x7f86080057f0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.678+0000 7f862207d640 1 -- 172.21.15.81:0/3165611158 reap_dead start 2023-11-08T16:38:05.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.678+0000 7f8623ae0640 1 -- 172.21.15.81:0/3165611158 shutdown_connections 2023-11-08T16:38:05.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.678+0000 7f8623ae0640 1 -- 172.21.15.81:0/3165611158 >> 172.21.15.81:0/3165611158 conn(0x7f861c0ceaa0 msgr2=0x7f861c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:05.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.678+0000 7f8623ae0640 1 -- 172.21.15.81:0/3165611158 shutdown_connections 2023-11-08T16:38:05.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.678+0000 7f8623ae0640 1 -- 172.21.15.81:0/3165611158 wait complete. 2023-11-08T16:38:05.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.680+0000 7f8623ae0640 1 Processor -- start 2023-11-08T16:38:05.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.680+0000 7f8623ae0640 1 -- start start 2023-11-08T16:38:05.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.681+0000 7f8623ae0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 0x7f861c151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.681+0000 7f8623ae0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f861c161fe0 0x7f861c162460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.681+0000 7f8623ae0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c7f10 0x7f861c0c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.681+0000 7f8623ae0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c1629a0 con 0x7f8608003970 2023-11-08T16:38:05.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.681+0000 7f8623ae0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f861c1524b0 con 0x7f861c161fe0 2023-11-08T16:38:05.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.681+0000 7f8623ae0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f861c161460 con 0x7f861c0c7f10 2023-11-08T16:38:05.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.681+0000 7f862187c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 0x7f861c151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.681+0000 7f862187c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 0x7f861c151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59620/0 (socket says 172.21.15.81:59620) 2023-11-08T16:38:05.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.681+0000 7f862187c640 1 -- 172.21.15.81:0/3239174834 learned_addr learned my addr 172.21.15.81:0/3239174834 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:05.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f862187c640 1 -- 172.21.15.81:0/3239174834 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c7f10 msgr2=0x7f861c0c83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f862207d640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c7f10 0x7f861c0c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f862107b640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f861c161fe0 0x7f861c162460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f862187c640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c7f10 0x7f861c0c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f862187c640 1 -- 172.21.15.81:0/3239174834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f861c161fe0 msgr2=0x7f861c162460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:05.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f862187c640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f861c161fe0 0x7f861c162460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:05.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f862187c640 1 -- 172.21.15.81:0/3239174834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f861c1603c0 con 0x7f8608003970 2023-11-08T16:38:05.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f862187c640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 0x7f861c151f70 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8618000ef0 tx=0x7f8618001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f86127fc640 1 -- 172.21.15.81:0/3239174834 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f861803c040 con 0x7f8608003970 2023-11-08T16:38:05.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f86127fc640 1 -- 172.21.15.81:0/3239174834 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8618005cb0 con 0x7f8608003970 2023-11-08T16:38:05.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f86127fc640 1 -- 172.21.15.81:0/3239174834 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86180072b0 con 0x7f8608003970 2023-11-08T16:38:05.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f8623ae0640 1 -- 172.21.15.81:0/3239174834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f861c1606a0 con 0x7f8608003970 2023-11-08T16:38:05.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.682+0000 7f8623ae0640 1 -- 172.21.15.81:0/3239174834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f861c160c60 con 0x7f8608003970 2023-11-08T16:38:05.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.684+0000 7f8623ae0640 1 -- 172.21.15.81:0/3239174834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f861c164b90 con 0x7f8608003970 2023-11-08T16:38:05.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.684+0000 7f86127fc640 1 -- 172.21.15.81:0/3239174834 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8618032070 con 0x7f8608003970 2023-11-08T16:38:05.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.685+0000 7f86127fc640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f860006aea0 0x7f860006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:05.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.685+0000 7f86127fc640 1 -- 172.21.15.81:0/3239174834 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x7f86180ab0c0 con 0x7f8608003970 2023-11-08T16:38:05.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.686+0000 7f862107b640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f860006aea0 0x7f860006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:05.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.686+0000 7f862107b640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f860006aea0 0x7f860006d360 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f860c009910 tx=0x7f860c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:05.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.690+0000 7f86127fc640 1 -- 172.21.15.81:0/3239174834 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8618075ee0 con 0x7f8608003970 2023-11-08T16:38:05.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:05.950+0000 7f8623ae0640 1 -- 172.21.15.81:0/3239174834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7f861c0743f0 con 0x7f8608003970 2023-11-08T16:38:06.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.595+0000 7f86127fc640 1 -- 172.21.15.81:0/3239174834 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 4:0 v72) v1 ==== 101+0+0 (secure 0 0 0) 0x7f8618079500 con 0x7f8608003970 2023-11-08T16:38:06.598 INFO:teuthology.orchestra.run.smithi081.stderr:repaired: restoring rank 4:0 2023-11-08T16:38:06.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.598+0000 7f8623ae0640 1 -- 172.21.15.81:0/3239174834 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f860006aea0 msgr2=0x7f860006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:06.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.598+0000 7f8623ae0640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f860006aea0 0x7f860006d360 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f860c009910 tx=0x7f860c008040 comp rx=0 tx=0).stop 2023-11-08T16:38:06.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.598+0000 7f8623ae0640 1 -- 172.21.15.81:0/3239174834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 msgr2=0x7f861c151f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:06.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.598+0000 7f8623ae0640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 0x7f861c151f70 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8618000ef0 tx=0x7f8618001160 comp rx=0 tx=0).stop 2023-11-08T16:38:06.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.598+0000 7f8623ae0640 1 -- 172.21.15.81:0/3239174834 shutdown_connections 2023-11-08T16:38:06.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.598+0000 7f8623ae0640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f860006aea0 0x7f860006d360 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:06.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.598+0000 7f8623ae0640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f861c0c7f10 0x7f861c0c83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:06.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.598+0000 7f8623ae0640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f861c161fe0 0x7f861c162460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:06.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.599+0000 7f8623ae0640 1 --2- 172.21.15.81:0/3239174834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8608003970 0x7f861c151f70 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:06.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.599+0000 7f8623ae0640 1 -- 172.21.15.81:0/3239174834 >> 172.21.15.81:0/3239174834 conn(0x7f861c0ceaa0 msgr2=0x7f861c06bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:06.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.599+0000 7f8623ae0640 1 -- 172.21.15.81:0/3239174834 shutdown_connections 2023-11-08T16:38:06.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.599+0000 7f8623ae0640 1 -- 172.21.15.81:0/3239174834 wait complete. 2023-11-08T16:38:06.613 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:38:06.611+0000 7f385b035640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-08T16:38:06.624 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T16:38:06.624 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T16:38:06.624 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T16:38:06.625 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:38:06.625 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T16:38:06.625 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T16:38:06.626 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T16:38:06.626 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:38:06.626 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T16:38:06.626 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T16:38:06.627 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T16:38:06.627 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:38:06.627 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T16:38:06.628 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T16:38:06.628 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T16:38:06.628 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:38:06.642 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T16:38:06.643 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T16:38:06.644 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T16:38:06.644 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T16:38:06.647 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T16:38:06.647 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T16:38:06.648 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T16:38:06.648 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T16:38:06.693 INFO:tasks.ceph.mds.c:Started 2023-11-08T16:38:06.693 DEBUG:teuthology.parallel:result is None 2023-11-08T16:38:06.697 INFO:tasks.ceph.mds.d:Started 2023-11-08T16:38:06.697 DEBUG:teuthology.parallel:result is None 2023-11-08T16:38:06.699 INFO:tasks.ceph.mds.a:Started 2023-11-08T16:38:06.700 DEBUG:teuthology.parallel:result is None 2023-11-08T16:38:06.705 INFO:tasks.ceph.mds.b:Started 2023-11-08T16:38:06.705 DEBUG:teuthology.parallel:result is None 2023-11-08T16:38:06.705 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:06.851 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:38:06.856 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:38:06.857 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:38:06.860 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2023-11-08T16:38:06.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.891+0000 7f46e14a9640 1 Processor -- start 2023-11-08T16:38:06.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.891+0000 7f46e14a9640 1 -- start start 2023-11-08T16:38:06.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.892+0000 7f46e14a9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f46dc0cf1a0 0x7f46dc065df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:06.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.892+0000 7f46e14a9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46dc0b4210 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.892+0000 7f46e14a9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46dc0b44f0 con 0x7f46dc15d4c0 2023-11-08T16:38:06.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.892+0000 7f46e14a9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46dc0664f0 con 0x7f46dc0cf1a0 2023-11-08T16:38:06.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.893+0000 7f46db7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f46dc0b4fd0 0x7f46dc0af2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50398/0 (socket says 172.21.15.81:50398) 2023-11-08T16:38:06.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.893+0000 7f46db7fe640 1 -- 172.21.15.81:0/2417325284 learned_addr learned my addr 172.21.15.81:0/2417325284 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:06.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.893+0000 7f46daffd640 1 --2- 172.21.15.81:0/2417325284 >> v2:172.21.15.93:3300/0 conn(0x7f46dc0cf1a0 0x7f46dc065df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:06.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.893+0000 7f46d9ffb640 1 -- 172.21.15.81:0/2417325284 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290144996 0 0) 0x7f46dc0b4210 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.893+0000 7f46d9ffb640 1 -- 172.21.15.81:0/2417325284 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46c4003520 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.893+0000 7f46d9ffb640 1 -- 172.21.15.81:0/2417325284 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2216402720 0 0) 0x7f46dc0b44f0 con 0x7f46dc15d4c0 2023-11-08T16:38:06.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.893+0000 7f46d9ffb640 1 -- 172.21.15.81:0/2417325284 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46dc0b4210 con 0x7f46dc15d4c0 2023-11-08T16:38:06.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.894+0000 7f46daffd640 1 -- 172.21.15.81:0/2417325284 >> v1:172.21.15.93:6789/0 conn(0x7f46dc15d4c0 legacy=0x7f46dc0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:06.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.894+0000 7f46daffd640 1 -- 172.21.15.81:0/2417325284 >> v1:172.21.15.81:6789/0 conn(0x7f46dc0b4fd0 legacy=0x7f46dc0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:06.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.894+0000 7f46daffd640 1 -- 172.21.15.81:0/2417325284 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46dc0b47d0 con 0x7f46dc0cf1a0 2023-11-08T16:38:06.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.895+0000 7f46daffd640 1 --2- 172.21.15.81:0/2417325284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46dc0cf1a0 0x7f46dc065df0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009790 tx=0x7f46cc02b6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef4b1299963bb476 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:06.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.895+0000 7f46d9ffb640 1 -- 172.21.15.81:0/2417325284 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d40019f0 con 0x7f46dc0cf1a0 2023-11-08T16:38:06.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.895+0000 7f46d9ffb640 1 -- 172.21.15.81:0/2417325284 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc02e050 con 0x7f46dc0cf1a0 2023-11-08T16:38:06.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.895+0000 7f46d9ffb640 1 -- 172.21.15.81:0/2417325284 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc030730 con 0x7f46dc0cf1a0 2023-11-08T16:38:06.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.895+0000 7f46e14a9640 1 -- 172.21.15.81:0/2417325284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46dc0cf1a0 msgr2=0x7f46dc065df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:06.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.895+0000 7f46e14a9640 1 --2- 172.21.15.81:0/2417325284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46dc0cf1a0 0x7f46dc065df0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009790 tx=0x7f46cc02b6b0 comp rx=0 tx=0).stop 2023-11-08T16:38:06.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.896+0000 7f46e14a9640 1 -- 172.21.15.81:0/2417325284 shutdown_connections 2023-11-08T16:38:06.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.896+0000 7f46e14a9640 1 --2- 172.21.15.81:0/2417325284 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46dc0cf1a0 0x7f46dc065df0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:06.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.896+0000 7f46e14a9640 1 -- 172.21.15.81:0/2417325284 >> 172.21.15.81:0/2417325284 conn(0x7f46dc071540 msgr2=0x7f46dc071950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:06.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.896+0000 7f46e14a9640 1 -- 172.21.15.81:0/2417325284 shutdown_connections 2023-11-08T16:38:06.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.896+0000 7f46e14a9640 1 -- 172.21.15.81:0/2417325284 wait complete. 2023-11-08T16:38:06.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.897+0000 7f46e14a9640 1 Processor -- start 2023-11-08T16:38:06.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.898+0000 7f46e14a9640 1 -- start start 2023-11-08T16:38:06.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.898+0000 7f46e14a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46dc0b4fd0 0x7f46dc0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:06.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.898+0000 7f46e14a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46dc0cf1a0 0x7f46dc0cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:06.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.898+0000 7f46e14a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46dc15d4c0 0x7f46dc0cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:06.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.898+0000 7f46e14a9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0c69b0 con 0x7f46dc15d4c0 2023-11-08T16:38:06.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.898+0000 7f46e14a9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0cbdc0 con 0x7f46dc0cf1a0 2023-11-08T16:38:06.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.898+0000 7f46e14a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46dc0cc0a0 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.899+0000 7f46daffd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46dc0b4fd0 0x7f46dc0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:06.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.899+0000 7f46db7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46dc15d4c0 0x7f46dc0cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:06.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.899+0000 7f46da7fc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46dc0cf1a0 0x7f46dc0cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:06.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.899+0000 7f46daffd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46dc0b4fd0 0x7f46dc0c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:57468/0 (socket says 172.21.15.81:57468) 2023-11-08T16:38:06.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.899+0000 7f46daffd640 1 -- 172.21.15.81:0/1097293241 learned_addr learned my addr 172.21.15.81:0/1097293241 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:06.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.899+0000 7f46daffd640 1 -- 172.21.15.81:0/1097293241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46dc0cf1a0 msgr2=0x7f46dc0cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:06.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.899+0000 7f46daffd640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46dc0cf1a0 0x7f46dc0cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:06.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.899+0000 7f46daffd640 1 -- 172.21.15.81:0/1097293241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46dc15d4c0 msgr2=0x7f46dc0cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:06.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.899+0000 7f46daffd640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46dc15d4c0 0x7f46dc0cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:06.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.899+0000 7f46daffd640 1 -- 172.21.15.81:0/1097293241 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46cc0093a0 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.899+0000 7f46db7fe640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46dc15d4c0 0x7f46dc0cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:38:06.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.900+0000 7f46daffd640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46dc0b4fd0 0x7f46dc0c6470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f46cc035040 tx=0x7f46cc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:06.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.900+0000 7f46c37fe640 1 -- 172.21.15.81:0/1097293241 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc005da0 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.900+0000 7f46c37fe640 1 -- 172.21.15.81:0/1097293241 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46cc006030 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.900+0000 7f46c37fe640 1 -- 172.21.15.81:0/1097293241 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46cc0063e0 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.900+0000 7f46e14a9640 1 -- 172.21.15.81:0/1097293241 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46dc0c6e20 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.900+0000 7f46e14a9640 1 -- 172.21.15.81:0/1097293241 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46dc0ac310 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.901+0000 7f46e14a9640 1 -- 172.21.15.81:0/1097293241 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46dc15d000 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.902+0000 7f46c37fe640 1 -- 172.21.15.81:0/1097293241 <== mon.2 v2:172.21.15.93:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f46cc039040 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.903+0000 7f46c37fe640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f46c806ae70 0x7f46c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:06.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.903+0000 7f46c37fe640 1 -- 172.21.15.81:0/1097293241 <== mon.2 v2:172.21.15.93:3301/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6343+0+0 (secure 0 0 0) 0x7f46cc0ab190 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.903+0000 7f46da7fc640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f46c806ae70 0x7f46c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:06.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.904+0000 7f46da7fc640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f46c806ae70 0x7f46c806d330 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0c9570 tx=0x7f46d0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:06.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:06.909+0000 7f46c37fe640 1 -- 172.21.15.81:0/1097293241 <== mon.2 v2:172.21.15.93:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f46cc076100 con 0x7f46dc0b4fd0 2023-11-08T16:38:06.976 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:38:06.973+0000 7f5cc09c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:38:07.002 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:38:07.000+0000 7f5ebc3c1640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-08T16:38:07.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.150+0000 7f46e14a9640 1 -- 172.21.15.81:0/1097293241 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46dc0c6c80 con 0x7f46dc0b4fd0 2023-11-08T16:38:07.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.152+0000 7f46c37fe640 1 -- 172.21.15.81:0/1097293241 <== mon.2 v2:172.21.15.93:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f46dc0c6c80 con 0x7f46dc0b4fd0 2023-11-08T16:38:07.155 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:07.155 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":75,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/2125403666","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":2125403666},{"type":"v1","addr":"172.21.15.93:6834","nonce":2125403666}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1527116581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1527116581},{"type":"v1","addr":"172.21.15.93:6833","nonce":1527116581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":7380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2591582634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2591582634},{"type":"v1","addr":"172.21.15.81:6835","nonce":2591582634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:06.985185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"a","rank":0,"incarnation":75,"state":"up:replay","state_seq":1,"addr":"172.21.15.81:6837/1843050442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1843050442},{"type":"v1","addr":"172.21.15.81:6837","nonce":1843050442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:07.155 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 75 2023-11-08T16:38:07.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 -- 172.21.15.81:0/1097293241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f46c806ae70 msgr2=0x7f46c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:07.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f46c806ae70 0x7f46c806d330 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0c9570 tx=0x7f46d0009040 comp rx=0 tx=0).stop 2023-11-08T16:38:07.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 -- 172.21.15.81:0/1097293241 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46dc0b4fd0 msgr2=0x7f46dc0c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:07.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46dc0b4fd0 0x7f46dc0c6470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f46cc035040 tx=0x7f46cc0046c0 comp rx=0 tx=0).stop 2023-11-08T16:38:07.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 -- 172.21.15.81:0/1097293241 shutdown_connections 2023-11-08T16:38:07.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f46c806ae70 0x7f46c806d330 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:07.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f46dc15d4c0 0x7f46dc0cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:07.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46dc0cf1a0 0x7f46dc0cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:07.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 --2- 172.21.15.81:0/1097293241 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f46dc0b4fd0 0x7f46dc0c6470 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:07.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 -- 172.21.15.81:0/1097293241 >> 172.21.15.81:0/1097293241 conn(0x7f46dc071540 msgr2=0x7f46dc0ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:07.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 -- 172.21.15.81:0/1097293241 shutdown_connections 2023-11-08T16:38:07.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:07.155+0000 7f46e14a9640 1 -- 172.21.15.81:0/1097293241 wait complete. 2023-11-08T16:38:07.578 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:37:59.633801+0000', 'modified': '2023-11-08T16:38:06.985185+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7383}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7383': {'gid': 7383, 'name': 'a', 'rank': 0, 'incarnation': 75, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.81:6837/1843050442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 1843050442}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 1843050442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11, 12], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:38:07.578 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7383:up:replay 2023-11-08T16:38:08.579 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:08.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.804+0000 7f5e6c960640 1 Processor -- start 2023-11-08T16:38:08.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.804+0000 7f5e6c960640 1 -- start start 2023-11-08T16:38:08.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.805+0000 7f5e6c960640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e680b6e60 0x7f5e68168160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:08.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.805+0000 7f5e6c960640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e680d1ea0 con 0x7f5e680d39e0 2023-11-08T16:38:08.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.806+0000 7f5e6c960640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e680ce890 con 0x7f5e6815d490 2023-11-08T16:38:08.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.806+0000 7f5e6c960640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680d3e40 con 0x7f5e680b6e60 2023-11-08T16:38:08.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.806+0000 7f5e6659c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5e680d39e0 0x7f5e680d1790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50410/0 (socket says 172.21.15.81:50410) 2023-11-08T16:38:08.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.806+0000 7f5e6659c640 1 -- 172.21.15.81:0/2175574561 learned_addr learned my addr 172.21.15.81:0/2175574561 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:08.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.806+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287370987 0 0) 0x7f5e680d1ea0 con 0x7f5e680d39e0 2023-11-08T16:38:08.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.806+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e50003520 con 0x7f5e680d39e0 2023-11-08T16:38:08.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.806+0000 7f5e6759e640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e680b6e60 0x7f5e68168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:08.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4043222343 0 0) 0x7f5e50003520 con 0x7f5e680d39e0 2023-11-08T16:38:08.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e680b6e60 msgr2=0x7f5e68168160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:08.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e680b6e60 0x7f5e68168160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:08.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 >> v1:172.21.15.93:6789/0 conn(0x7f5e6815d490 legacy=0x7f5e680d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:08.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e680ceb70 con 0x7f5e680d39e0 2023-11-08T16:38:08.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f5e5c001670 con 0x7f5e680d39e0 2023-11-08T16:38:08.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 >> v1:172.21.15.81:6789/0 conn(0x7f5e680d39e0 legacy=0x7f5e680d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:08.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e50003a80 0x7f5e50003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:08.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e680b6e60 0x7f5e500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:08.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e50004e20 0x7f5e5000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:08.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680d3e40 con 0x7f5e50003a80 2023-11-08T16:38:08.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e500054d0 con 0x7f5e50004e20 2023-11-08T16:38:08.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50005690 con 0x7f5e680b6e60 2023-11-08T16:38:08.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e66d9d640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e50003a80 0x7f5e50003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:08.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e6659c640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e680b6e60 0x7f5e500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:08.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e66d9d640 1 -- 172.21.15.81:0/2175574561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e680b6e60 msgr2=0x7f5e500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:08.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e66d9d640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e680b6e60 0x7f5e500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:08.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e66d9d640 1 -- 172.21.15.81:0/2175574561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e50004e20 msgr2=0x7f5e5000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:08.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e66d9d640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e50004e20 0x7f5e5000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:08.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e66d9d640 1 -- 172.21.15.81:0/2175574561 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e50003520 con 0x7f5e50003a80 2023-11-08T16:38:08.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.807+0000 7f5e6659c640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e680b6e60 0x7f5e500047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:08.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.808+0000 7f5e66d9d640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e50003a80 0x7f5e50003ea0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f5e58001750 tx=0x7f5e500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af6d8d37c503b21 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:08.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.809+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e580052d0 con 0x7f5e50003a80 2023-11-08T16:38:08.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.809+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5802e050 con 0x7f5e50003a80 2023-11-08T16:38:08.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.809+0000 7f5e65d9b640 1 -- 172.21.15.81:0/2175574561 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e580035f0 con 0x7f5e50003a80 2023-11-08T16:38:08.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.809+0000 7f5e6c960640 1 -- 172.21.15.81:0/2175574561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e50003a80 msgr2=0x7f5e50003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:08.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.809+0000 7f5e6c960640 1 --2- 172.21.15.81:0/2175574561 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e50003a80 0x7f5e50003ea0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f5e58001750 tx=0x7f5e500057f0 comp rx=0 tx=0).stop 2023-11-08T16:38:08.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.809+0000 7f5e6759e640 1 -- 172.21.15.81:0/2175574561 reap_dead start 2023-11-08T16:38:08.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.809+0000 7f5e6c960640 1 -- 172.21.15.81:0/2175574561 shutdown_connections 2023-11-08T16:38:08.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.809+0000 7f5e6c960640 1 -- 172.21.15.81:0/2175574561 >> 172.21.15.81:0/2175574561 conn(0x7f5e680d2e50 msgr2=0x7f5e680d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:08.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.809+0000 7f5e6c960640 1 -- 172.21.15.81:0/2175574561 shutdown_connections 2023-11-08T16:38:08.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.809+0000 7f5e6c960640 1 -- 172.21.15.81:0/2175574561 wait complete. 2023-11-08T16:38:08.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.811+0000 7f5e6c960640 1 Processor -- start 2023-11-08T16:38:08.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.811+0000 7f5e6c960640 1 -- start start 2023-11-08T16:38:08.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.812+0000 7f5e6c960640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e50003a80 0x7f5e681f2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:08.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.812+0000 7f5e6c960640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e680cf650 0x7f5e680cfad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:08.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.812+0000 7f5e6c960640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e6815f5b0 0x7f5e6815fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:08.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.812+0000 7f5e6c960640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680d0010 con 0x7f5e6815f5b0 2023-11-08T16:38:08.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.812+0000 7f5e6c960640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e681535a0 con 0x7f5e680cf650 2023-11-08T16:38:08.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.812+0000 7f5e6c960640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e680c68b0 con 0x7f5e50003a80 2023-11-08T16:38:08.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e6759e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e6815f5b0 0x7f5e6815fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:08.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e6759e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e6815f5b0 0x7f5e6815fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59676/0 (socket says 172.21.15.81:59676) 2023-11-08T16:38:08.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e6759e640 1 -- 172.21.15.81:0/2254559580 learned_addr learned my addr 172.21.15.81:0/2254559580 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:08.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e66d9d640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e50003a80 0x7f5e681f2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:08.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e6759e640 1 -- 172.21.15.81:0/2254559580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e50003a80 msgr2=0x7f5e681f2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:08.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e6659c640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e680cf650 0x7f5e680cfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:08.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e6759e640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e50003a80 0x7f5e681f2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:08.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e6759e640 1 -- 172.21.15.81:0/2254559580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e680cf650 msgr2=0x7f5e680cfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:08.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e6759e640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e680cf650 0x7f5e680cfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:08.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e6759e640 1 -- 172.21.15.81:0/2254559580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e681f2bc0 con 0x7f5e6815f5b0 2023-11-08T16:38:08.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e6759e640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e6815f5b0 0x7f5e6815fa30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f5e680b72f0 tx=0x7f5e6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:08.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.813+0000 7f5e4f7fe640 1 -- 172.21.15.81:0/2254559580 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e60018070 con 0x7f5e6815f5b0 2023-11-08T16:38:08.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.814+0000 7f5e4f7fe640 1 -- 172.21.15.81:0/2254559580 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e6000ebc0 con 0x7f5e6815f5b0 2023-11-08T16:38:08.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.814+0000 7f5e6c960640 1 -- 172.21.15.81:0/2254559580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e681602e0 con 0x7f5e6815f5b0 2023-11-08T16:38:08.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.814+0000 7f5e4f7fe640 1 -- 172.21.15.81:0/2254559580 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e60003990 con 0x7f5e6815f5b0 2023-11-08T16:38:08.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.814+0000 7f5e6c960640 1 -- 172.21.15.81:0/2254559580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e681608a0 con 0x7f5e6815f5b0 2023-11-08T16:38:08.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.815+0000 7f5e6c960640 1 -- 172.21.15.81:0/2254559580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e680c6680 con 0x7f5e6815f5b0 2023-11-08T16:38:08.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.816+0000 7f5e4f7fe640 1 -- 172.21.15.81:0/2254559580 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5e60014420 con 0x7f5e6815f5b0 2023-11-08T16:38:08.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.817+0000 7f5e4f7fe640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5e5406aea0 0x7f5e5406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:08.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.817+0000 7f5e4f7fe640 1 -- 172.21.15.81:0/2254559580 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f5e60089770 con 0x7f5e6815f5b0 2023-11-08T16:38:08.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.817+0000 7f5e66d9d640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5e5406aea0 0x7f5e5406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:08.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.818+0000 7f5e66d9d640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5e5406aea0 0x7f5e5406d360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5e58033000 tx=0x7f5e58009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:08.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:08.821+0000 7f5e4f7fe640 1 -- 172.21.15.81:0/2254559580 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5e680c6680 con 0x7f5e6815f5b0 2023-11-08T16:38:09.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.049+0000 7f5e6c960640 1 -- 172.21.15.81:0/2254559580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e680c6680 con 0x7f5e6815f5b0 2023-11-08T16:38:09.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.050+0000 7f5e4f7fe640 1 -- 172.21.15.81:0/2254559580 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f5e680c6680 con 0x7f5e6815f5b0 2023-11-08T16:38:09.054 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:09.054 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":77,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1527116581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1527116581},{"type":"v1","addr":"172.21.15.93:6833","nonce":1527116581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2617548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2617548066},{"type":"v1","addr":"172.21.15.93:6835","nonce":2617548066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":7380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2591582634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2591582634},{"type":"v1","addr":"172.21.15.81:6835","nonce":2591582634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:08.002123+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"a","rank":0,"incarnation":75,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.81:6837/1843050442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1843050442},{"type":"v1","addr":"172.21.15.81:6837","nonce":1843050442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:09.054 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 77 2023-11-08T16:38:09.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 -- 172.21.15.81:0/2254559580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5e5406aea0 msgr2=0x7f5e5406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:09.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5e5406aea0 0x7f5e5406d360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f5e58033000 tx=0x7f5e58009030 comp rx=0 tx=0).stop 2023-11-08T16:38:09.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 -- 172.21.15.81:0/2254559580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e6815f5b0 msgr2=0x7f5e6815fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:09.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e6815f5b0 0x7f5e6815fa30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f5e680b72f0 tx=0x7f5e6000e4b0 comp rx=0 tx=0).stop 2023-11-08T16:38:09.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 -- 172.21.15.81:0/2254559580 shutdown_connections 2023-11-08T16:38:09.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5e5406aea0 0x7f5e5406d360 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:09.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5e6815f5b0 0x7f5e6815fa30 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:09.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e680cf650 0x7f5e680cfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:09.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 --2- 172.21.15.81:0/2254559580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5e50003a80 0x7f5e681f2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:09.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 -- 172.21.15.81:0/2254559580 >> 172.21.15.81:0/2254559580 conn(0x7f5e680d2e50 msgr2=0x7f5e6806a760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:09.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 -- 172.21.15.81:0/2254559580 shutdown_connections 2023-11-08T16:38:09.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:09.053+0000 7f5e6c960640 1 -- 172.21.15.81:0/2254559580 wait complete. 2023-11-08T16:38:09.477 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:37:59.633801+0000', 'modified': '2023-11-08T16:38:08.002123+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7383}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7383': {'gid': 7383, 'name': 'a', 'rank': 0, 'incarnation': 75, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.81:6837/1843050442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 1843050442}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 1843050442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11, 12], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:38:09.477 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7383:up:rejoin 2023-11-08T16:38:10.479 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:10.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.699+0000 7f7bb7347640 1 Processor -- start 2023-11-08T16:38:10.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.699+0000 7f7bb7347640 1 -- start start 2023-11-08T16:38:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.701+0000 7f7bb7347640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7bb0150b80 0x7f7bb015a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.701+0000 7f7bb7347640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7bb00ced10 0x7f7bb00cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.701+0000 7f7bb7347640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bb00b2fe0 con 0x7f7bb0070760 2023-11-08T16:38:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.701+0000 7f7bb7347640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb00cf6c0 con 0x7f7bb00ced10 2023-11-08T16:38:10.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.701+0000 7f7bb7347640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bb00b32c0 con 0x7f7bb0150b80 2023-11-08T16:38:10.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.701+0000 7f7bb5b44640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7bb00ced10 0x7f7bb00cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:10.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.701+0000 7f7bb5b44640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7bb00ced10 0x7f7bb00cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59690/0 (socket says 172.21.15.81:59690) 2023-11-08T16:38:10.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.701+0000 7f7bb5b44640 1 -- 172.21.15.81:0/2321850369 learned_addr learned my addr 172.21.15.81:0/2321850369 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:10.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.702+0000 7f7bb5b44640 1 -- 172.21.15.81:0/2321850369 >> v2:172.21.15.93:3300/0 conn(0x7f7bb0150b80 msgr2=0x7f7bb015a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:10.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.702+0000 7f7bb5b44640 1 --2- 172.21.15.81:0/2321850369 >> v2:172.21.15.93:3300/0 conn(0x7f7bb0150b80 0x7f7bb015a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:10.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.702+0000 7f7bb5b44640 1 -- 172.21.15.81:0/2321850369 >> v1:172.21.15.93:6789/0 conn(0x7f7bb0070760 legacy=0x7f7bb0070b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:10.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.702+0000 7f7bb5b44640 1 -- 172.21.15.81:0/2321850369 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb00b35a0 con 0x7f7bb00ced10 2023-11-08T16:38:10.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.703+0000 7f7bb5b44640 1 --2- 172.21.15.81:0/2321850369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb00ced10 0x7f7bb00cf180 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00cf8d0 tx=0x7f7ba002db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2b591ae724072f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:10.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.703+0000 7f7bb5343640 1 -- 172.21.15.81:0/2321850369 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba002f8e0 con 0x7f7bb00ced10 2023-11-08T16:38:10.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.703+0000 7f7bb5343640 1 -- 172.21.15.81:0/2321850369 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba002fa40 con 0x7f7bb00ced10 2023-11-08T16:38:10.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.703+0000 7f7bb5343640 1 -- 172.21.15.81:0/2321850369 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba002fdd0 con 0x7f7bb00ced10 2023-11-08T16:38:10.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.703+0000 7f7bb7347640 1 -- 172.21.15.81:0/2321850369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb00ced10 msgr2=0x7f7bb00cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:10.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.703+0000 7f7bb7347640 1 --2- 172.21.15.81:0/2321850369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb00ced10 0x7f7bb00cf180 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00cf8d0 tx=0x7f7ba002db20 comp rx=0 tx=0).stop 2023-11-08T16:38:10.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.704+0000 7f7bb7347640 1 -- 172.21.15.81:0/2321850369 shutdown_connections 2023-11-08T16:38:10.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.704+0000 7f7bb7347640 1 --2- 172.21.15.81:0/2321850369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb00ced10 0x7f7bb00cf180 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:10.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.704+0000 7f7bb7347640 1 --2- 172.21.15.81:0/2321850369 >> v2:172.21.15.93:3300/0 conn(0x7f7bb0150b80 0x7f7bb015a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:10.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.704+0000 7f7bb7347640 1 -- 172.21.15.81:0/2321850369 >> 172.21.15.81:0/2321850369 conn(0x7f7bb006a4a0 msgr2=0x7f7bb006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:10.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.704+0000 7f7bb7347640 1 -- 172.21.15.81:0/2321850369 shutdown_connections 2023-11-08T16:38:10.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.704+0000 7f7bb7347640 1 -- 172.21.15.81:0/2321850369 wait complete. 2023-11-08T16:38:10.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.705+0000 7f7bb7347640 1 Processor -- start 2023-11-08T16:38:10.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.705+0000 7f7bb7347640 1 -- start start 2023-11-08T16:38:10.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.706+0000 7f7bb7347640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7bb0070760 0x7f7bb0159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:10.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.706+0000 7f7bb7347640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb00ced10 0x7f7bb0157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:10.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.706+0000 7f7bb7347640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb0150b80 0x7f7bb01570c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:10.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.706+0000 7f7bb7347640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0158270 con 0x7f7bb0150b80 2023-11-08T16:38:10.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.706+0000 7f7bb7347640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0145e90 con 0x7f7bb00ced10 2023-11-08T16:38:10.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.706+0000 7f7bb7347640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb0157600 con 0x7f7bb0070760 2023-11-08T16:38:10.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.707+0000 7f7bb6b46640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb0150b80 0x7f7bb01570c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:10.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.707+0000 7f7bb6b46640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb0150b80 0x7f7bb01570c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:59698/0 (socket says 172.21.15.81:59698) 2023-11-08T16:38:10.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.707+0000 7f7bb6b46640 1 -- 172.21.15.81:0/4058928410 learned_addr learned my addr 172.21.15.81:0/4058928410 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:10.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.707+0000 7f7bb6345640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7bb0070760 0x7f7bb0159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:10.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.707+0000 7f7bb5b44640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb00ced10 0x7f7bb0157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:10.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.707+0000 7f7bb6b46640 1 -- 172.21.15.81:0/4058928410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7bb0070760 msgr2=0x7f7bb0159760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:10.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.707+0000 7f7bb6b46640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7bb0070760 0x7f7bb0159760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:10.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.707+0000 7f7bb6b46640 1 -- 172.21.15.81:0/4058928410 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb00ced10 msgr2=0x7f7bb0157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:10.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.707+0000 7f7bb6b46640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb00ced10 0x7f7bb0157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:10.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.707+0000 7f7bb6b46640 1 -- 172.21.15.81:0/4058928410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba00094d0 con 0x7f7bb0150b80 2023-11-08T16:38:10.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.708+0000 7f7bb6b46640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb0150b80 0x7f7bb01570c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800c780 tx=0x7f7ba800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:10.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.708+0000 7f7ba6ffd640 1 -- 172.21.15.81:0/4058928410 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8003d80 con 0x7f7bb0150b80 2023-11-08T16:38:10.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.708+0000 7f7ba6ffd640 1 -- 172.21.15.81:0/4058928410 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba800ce90 con 0x7f7bb0150b80 2023-11-08T16:38:10.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.708+0000 7f7bb7347640 1 -- 172.21.15.81:0/4058928410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bb01584a0 con 0x7f7bb0150b80 2023-11-08T16:38:10.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.708+0000 7f7ba6ffd640 1 -- 172.21.15.81:0/4058928410 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80052b0 con 0x7f7bb0150b80 2023-11-08T16:38:10.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.708+0000 7f7bb7347640 1 -- 172.21.15.81:0/4058928410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb0158a60 con 0x7f7bb0150b80 2023-11-08T16:38:10.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.709+0000 7f7bb7347640 1 -- 172.21.15.81:0/4058928410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bb0158c50 con 0x7f7bb0150b80 2023-11-08T16:38:10.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.710+0000 7f7ba6ffd640 1 -- 172.21.15.81:0/4058928410 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7ba8005410 con 0x7f7bb0150b80 2023-11-08T16:38:10.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.711+0000 7f7ba6ffd640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b9406ae70 0x7f7b9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:10.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.711+0000 7f7ba6ffd640 1 -- 172.21.15.81:0/4058928410 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f7ba8089100 con 0x7f7bb0150b80 2023-11-08T16:38:10.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.711+0000 7f7bb6345640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b9406ae70 0x7f7b9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:10.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.712+0000 7f7bb6345640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b9406ae70 0x7f7b9406d330 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00b810 tx=0x7f7bac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:10.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.716+0000 7f7ba6ffd640 1 -- 172.21.15.81:0/4058928410 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7ba8053ed0 con 0x7f7bb0150b80 2023-11-08T16:38:10.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.943+0000 7f7bb7347640 1 -- 172.21.15.81:0/4058928410 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bb0067800 con 0x7f7bb0150b80 2023-11-08T16:38:10.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.944+0000 7f7ba6ffd640 1 -- 172.21.15.81:0/4058928410 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f7ba8017090 con 0x7f7bb0150b80 2023-11-08T16:38:10.946 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:10.946 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":78,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1527116581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1527116581},{"type":"v1","addr":"172.21.15.93:6833","nonce":1527116581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2617548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2617548066},{"type":"v1","addr":"172.21.15.93:6835","nonce":2617548066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":7380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2591582634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2591582634},{"type":"v1","addr":"172.21.15.81:6835","nonce":2591582634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:09.004709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/1843050442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1843050442},{"type":"v1","addr":"172.21.15.81:6837","nonce":1843050442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:10.947 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 78 2023-11-08T16:38:10.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 -- 172.21.15.81:0/4058928410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b9406ae70 msgr2=0x7f7b9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:10.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b9406ae70 0x7f7b9406d330 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00b810 tx=0x7f7bac008040 comp rx=0 tx=0).stop 2023-11-08T16:38:10.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 -- 172.21.15.81:0/4058928410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb0150b80 msgr2=0x7f7bb01570c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:10.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb0150b80 0x7f7bb01570c0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f7ba800c780 tx=0x7f7ba800cc40 comp rx=0 tx=0).stop 2023-11-08T16:38:10.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 -- 172.21.15.81:0/4058928410 shutdown_connections 2023-11-08T16:38:10.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b9406ae70 0x7f7b9406d330 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:10.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7bb0150b80 0x7f7bb01570c0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:10.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7bb00ced10 0x7f7bb0157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:10.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 --2- 172.21.15.81:0/4058928410 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7bb0070760 0x7f7bb0159760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:10.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 -- 172.21.15.81:0/4058928410 >> 172.21.15.81:0/4058928410 conn(0x7f7bb006a4a0 msgr2=0x7f7bb006a880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:10.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 -- 172.21.15.81:0/4058928410 shutdown_connections 2023-11-08T16:38:10.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:10.947+0000 7f7bb7347640 1 -- 172.21.15.81:0/4058928410 wait complete. 2023-11-08T16:38:11.368 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 78, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:37:59.633801+0000', 'modified': '2023-11-08T16:38:09.004709+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7383}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7383': {'gid': 7383, 'name': 'a', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6837/1843050442', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 1843050442}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 1843050442}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11, 12], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:38:11.369 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:38:11.369 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-08T16:38:11.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.588+0000 7f51a2f89640 1 Processor -- start 2023-11-08T16:38:11.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.589+0000 7f51a2f89640 1 -- start start 2023-11-08T16:38:11.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.590+0000 7f51a2f89640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f519c0cd2a0 0x7f519c06bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.590+0000 7f51a2f89640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519c0c90f0 0x7f519c06bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.590+0000 7f51a2f89640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519c06fa00 con 0x7f519c0cc8e0 2023-11-08T16:38:11.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.590+0000 7f51a2f89640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c06b620 con 0x7f519c0cd2a0 2023-11-08T16:38:11.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.590+0000 7f51a2f89640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0caec0 con 0x7f519c0c90f0 2023-11-08T16:38:11.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.590+0000 7f519bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f519c0cd2a0 0x7f519c06bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.591+0000 7f519bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f519c0cd2a0 0x7f519c06bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60012/0 (socket says 172.21.15.81:60012) 2023-11-08T16:38:11.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.591+0000 7f519bfff640 1 -- 172.21.15.81:0/3278819503 learned_addr learned my addr 172.21.15.81:0/3278819503 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:11.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.591+0000 7f51a0d25640 1 --1- 172.21.15.81:0/3278819503 >> v1:172.21.15.81:6789/0 conn(0x7f519c0cc8e0 0x7f519c164500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49184/0 (socket says 172.21.15.81:49184) 2023-11-08T16:38:11.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.591+0000 7f51a1526640 1 --2- 172.21.15.81:0/3278819503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519c0c90f0 0x7f519c06bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.591+0000 7f519bfff640 1 -- 172.21.15.81:0/3278819503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519c0c90f0 msgr2=0x7f519c06bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.591+0000 7f519bfff640 1 --2- 172.21.15.81:0/3278819503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519c0c90f0 0x7f519c06bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.591+0000 7f519bfff640 1 -- 172.21.15.81:0/3278819503 >> v1:172.21.15.81:6789/0 conn(0x7f519c0cc8e0 legacy=0x7f519c164500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.591+0000 7f519bfff640 1 -- 172.21.15.81:0/3278819503 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c0cb1a0 con 0x7f519c0cd2a0 2023-11-08T16:38:11.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.591+0000 7f51a1526640 1 --2- 172.21.15.81:0/3278819503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519c0c90f0 0x7f519c06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:11.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.592+0000 7f519bfff640 1 --2- 172.21.15.81:0/3278819503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f519c0cd2a0 0x7f519c06bab0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5190009870 tx=0x7f519002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fca6de3c6c0d8c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:11.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.592+0000 7f519b7fe640 1 -- 172.21.15.81:0/3278819503 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190031870 con 0x7f519c0cd2a0 2023-11-08T16:38:11.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.592+0000 7f519b7fe640 1 -- 172.21.15.81:0/3278819503 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51900319d0 con 0x7f519c0cd2a0 2023-11-08T16:38:11.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.592+0000 7f519b7fe640 1 -- 172.21.15.81:0/3278819503 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190031d60 con 0x7f519c0cd2a0 2023-11-08T16:38:11.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.593+0000 7f51a2f89640 1 -- 172.21.15.81:0/3278819503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f519c0cd2a0 msgr2=0x7f519c06bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.593+0000 7f51a2f89640 1 --2- 172.21.15.81:0/3278819503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f519c0cd2a0 0x7f519c06bab0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f5190009870 tx=0x7f519002b910 comp rx=0 tx=0).stop 2023-11-08T16:38:11.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.593+0000 7f51a2f89640 1 -- 172.21.15.81:0/3278819503 shutdown_connections 2023-11-08T16:38:11.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.593+0000 7f51a2f89640 1 --2- 172.21.15.81:0/3278819503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519c0c90f0 0x7f519c06bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.593+0000 7f51a2f89640 1 --2- 172.21.15.81:0/3278819503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f519c0cd2a0 0x7f519c06bab0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.593+0000 7f51a2f89640 1 -- 172.21.15.81:0/3278819503 >> 172.21.15.81:0/3278819503 conn(0x7f519c0ceaa0 msgr2=0x7f519c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:11.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.593+0000 7f51a2f89640 1 -- 172.21.15.81:0/3278819503 shutdown_connections 2023-11-08T16:38:11.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.593+0000 7f51a2f89640 1 -- 172.21.15.81:0/3278819503 wait complete. 2023-11-08T16:38:11.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.595+0000 7f51a2f89640 1 Processor -- start 2023-11-08T16:38:11.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.595+0000 7f51a2f89640 1 -- start start 2023-11-08T16:38:11.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.596+0000 7f51a2f89640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f519c0c90f0 0x7f519c1f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.596+0000 7f51a2f89640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519c0cc8e0 0x7f519c0c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.596+0000 7f51a2f89640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f519c0cd2a0 0x7f519c0c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.596+0000 7f51a2f89640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c1f7e80 con 0x7f519c0c90f0 2023-11-08T16:38:11.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.596+0000 7f51a2f89640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c160b30 con 0x7f519c0cd2a0 2023-11-08T16:38:11.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.596+0000 7f51a2f89640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c160e10 con 0x7f519c0cc8e0 2023-11-08T16:38:11.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.596+0000 7f51a0d25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f519c0c90f0 0x7f519c1f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.596+0000 7f51a0d25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f519c0c90f0 0x7f519c1f7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60014/0 (socket says 172.21.15.81:60014) 2023-11-08T16:38:11.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.596+0000 7f51a0d25640 1 -- 172.21.15.81:0/3590625252 learned_addr learned my addr 172.21.15.81:0/3590625252 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:11.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.597+0000 7f51a0d25640 1 -- 172.21.15.81:0/3590625252 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519c0cc8e0 msgr2=0x7f519c0c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.597+0000 7f51a0d25640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519c0cc8e0 0x7f519c0c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.597+0000 7f51a0d25640 1 -- 172.21.15.81:0/3590625252 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f519c0cd2a0 msgr2=0x7f519c0c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.597+0000 7f51a0d25640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f519c0cd2a0 0x7f519c0c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.597+0000 7f51a0d25640 1 -- 172.21.15.81:0/3590625252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900094d0 con 0x7f519c0c90f0 2023-11-08T16:38:11.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.597+0000 7f51a0d25640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f519c0c90f0 0x7f519c1f7940 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c770 tx=0x7f518c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:11.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.597+0000 7f51997fa640 1 -- 172.21.15.81:0/3590625252 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c00ce80 con 0x7f519c0c90f0 2023-11-08T16:38:11.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.597+0000 7f51997fa640 1 -- 172.21.15.81:0/3590625252 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c00a640 con 0x7f519c0c90f0 2023-11-08T16:38:11.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.597+0000 7f51997fa640 1 -- 172.21.15.81:0/3590625252 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c00a9b0 con 0x7f519c0c90f0 2023-11-08T16:38:11.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.597+0000 7f51a2f89640 1 -- 172.21.15.81:0/3590625252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c1610f0 con 0x7f519c0c90f0 2023-11-08T16:38:11.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.597+0000 7f51a2f89640 1 -- 172.21.15.81:0/3590625252 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c1616b0 con 0x7f519c0c90f0 2023-11-08T16:38:11.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.599+0000 7f51997fa640 1 -- 172.21.15.81:0/3590625252 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f518c00ab10 con 0x7f519c0c90f0 2023-11-08T16:38:11.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.600+0000 7f51997fa640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f518806ae70 0x7f518806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.600+0000 7f51997fa640 1 -- 172.21.15.81:0/3590625252 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f518c089170 con 0x7f519c0c90f0 2023-11-08T16:38:11.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.600+0000 7f519bfff640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f518806ae70 0x7f518806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.601+0000 7f51a5bf7740 1 Processor -- start 2023-11-08T16:38:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.601+0000 7f51a5bf7740 1 -- start start 2023-11-08T16:38:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.601+0000 7f519bfff640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f518806ae70 0x7f518806d330 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5190036000 tx=0x7f5190030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:11.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a6bbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e92a0 0x56107a6ebc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6bc300 0x56107a6ec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a62e7e0 con 0x56107a6e8ec0 2023-11-08T16:38:11.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a62bed0 con 0x56107a6e92a0 2023-11-08T16:38:11.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56107a6277c0 con 0x56107a6bc300 2023-11-08T16:38:11.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a0d25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a6bbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a0d25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a6bbdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60018/0 (socket says 172.21.15.81:60018) 2023-11-08T16:38:11.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a0d25640 1 -- 172.21.15.81:0/631724940 learned_addr learned my addr 172.21.15.81:0/631724940 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:11.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a0d25640 1 -- 172.21.15.81:0/631724940 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6bc300 msgr2=0x56107a6ec190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a0d25640 1 --2- 172.21.15.81:0/631724940 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6bc300 0x56107a6ec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a0d25640 1 -- 172.21.15.81:0/631724940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e92a0 msgr2=0x56107a6ebc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a0d25640 1 --2- 172.21.15.81:0/631724940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e92a0 0x56107a6ebc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.602+0000 7f51a0d25640 1 -- 172.21.15.81:0/631724940 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56107a5fab10 con 0x56107a6e8ec0 2023-11-08T16:38:11.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.603+0000 7f51a0d25640 1 --2- 172.21.15.81:0/631724940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a6bbdc0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f518c005fd0 tx=0x7f518c058440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d658652134a4f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:11.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.603+0000 7f5182ffd640 1 -- 172.21.15.81:0/631724940 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c00adc0 con 0x56107a6e8ec0 2023-11-08T16:38:11.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.603+0000 7f5182ffd640 1 -- 172.21.15.81:0/631724940 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c055fa0 con 0x56107a6e8ec0 2023-11-08T16:38:11.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.603+0000 7f5182ffd640 1 -- 172.21.15.81:0/631724940 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0570f0 con 0x56107a6e8ec0 2023-11-08T16:38:11.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.604+0000 7f51a5bf7740 1 -- 172.21.15.81:0/631724940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 msgr2=0x56107a6bbdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.604+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/631724940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a6bbdc0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f518c005fd0 tx=0x7f518c058440 comp rx=0 tx=0).stop 2023-11-08T16:38:11.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.604+0000 7f51a5bf7740 1 -- 172.21.15.81:0/631724940 shutdown_connections 2023-11-08T16:38:11.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.604+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/631724940 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6bc300 0x56107a6ec190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.604+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/631724940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e92a0 0x56107a6ebc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.604+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/631724940 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a6bbdc0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.604+0000 7f51a5bf7740 1 -- 172.21.15.81:0/631724940 >> 172.21.15.81:0/631724940 conn(0x56107a6be0d0 msgr2=0x56107a706c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:11.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.604+0000 7f51a5bf7740 1 -- 172.21.15.81:0/631724940 shutdown_connections 2023-11-08T16:38:11.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.604+0000 7f51a5bf7740 1 -- 172.21.15.81:0/631724940 wait complete. 2023-11-08T16:38:11.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.604+0000 7f51a5bf7740 1 -- start start 2023-11-08T16:38:11.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.605+0000 7f51a5bf7740 1 Processor -- start 2023-11-08T16:38:11.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6e8ec0 0x56107a7013d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e92a0 0x56107a6ee390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a701c00 0x56107a6ee8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a5c4c40 con 0x56107a701c00 2023-11-08T16:38:11.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a5c48e0 con 0x56107a6e92a0 2023-11-08T16:38:11.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56107a38af20 con 0x56107a6e8ec0 2023-11-08T16:38:11.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a5bf7740 10 client.0 _ll_register_callbacks cb 0x56107a5e6cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:38:11.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a1526640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a701c00 0x56107a6ee8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a1526640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a701c00 0x56107a6ee8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60022/0 (socket says 172.21.15.81:60022) 2023-11-08T16:38:11.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a1526640 1 -- 172.21.15.81:0/534430353 learned_addr learned my addr 172.21.15.81:0/534430353 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:11.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a1526640 1 -- 172.21.15.81:0/534430353 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6e8ec0 msgr2=0x56107a7013d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f519bfff640 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e92a0 0x56107a6ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a1526640 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6e8ec0 0x56107a7013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a1526640 1 -- 172.21.15.81:0/534430353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e92a0 msgr2=0x56107a6ee390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a1526640 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e92a0 0x56107a6ee390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.606+0000 7f51a1526640 1 -- 172.21.15.81:0/534430353 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c1616b0 con 0x56107a701c00 2023-11-08T16:38:11.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.607+0000 7f51a1526640 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a701c00 0x56107a6ee8d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f519400af60 tx=0x7f519400af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ddd68d40edeab4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:11.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.607+0000 7f517b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:38:11.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.607+0000 7f517b7fe640 1 -- 172.21.15.81:0/534430353 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940115f0 con 0x56107a701c00 2023-11-08T16:38:11.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.607+0000 7f517b7fe640 1 -- 172.21.15.81:0/534430353 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f5194011750 con 0x56107a701c00 2023-11-08T16:38:11.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.607+0000 7f51a5bf7740 1 -- 172.21.15.81:0/534430353 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56107a5fbc80 con 0x56107a701c00 2023-11-08T16:38:11.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.607+0000 7f517b7fe640 1 -- 172.21.15.81:0/534430353 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5194011910 con 0x56107a701c00 2023-11-08T16:38:11.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.607+0000 7f51a5bf7740 10 client.7422 fetch_fsmap learned FSMap version 78 2023-11-08T16:38:11.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.608+0000 7f51a5bf7740 1 -- 172.21.15.81:0/534430353 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x56107a625e40 con 0x56107a701c00 2023-11-08T16:38:11.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.608+0000 7f517b7fe640 1 -- 172.21.15.81:0/534430353 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56107a5fbc80 con 0x56107a701c00 2023-11-08T16:38:11.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.608+0000 7f517b7fe640 1 -- 172.21.15.81:0/534430353 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 78) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f5194011a70 con 0x56107a701c00 2023-11-08T16:38:11.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.608+0000 7f51a5bf7740 10 client.7422 fetch_fsmap finished waiting for FSMap version 78 2023-11-08T16:38:11.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.608+0000 7f51a5bf7740 10 client.7422 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T16:38:11.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.608+0000 7f51a5bf7740 20 client.7422 populate_metadata read hostname 'smithi081' 2023-11-08T16:38:11.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.608+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x56107a6fd750 0x56107a6fdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.608+0000 7f51a5bf7740 4 client.7422 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:38:11.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.608+0000 7f51a5bf7740 1 -- 172.21.15.81:0/534430353 --> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56107a6207a0 con 0x56107a6fd750 2023-11-08T16:38:11.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.608+0000 7f51a0d25640 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x56107a6fd750 0x56107a6fdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.609+0000 7f51a0d25640 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x56107a6fd750 0x56107a6fdb30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2d774bdd33c3f2 server_cookie=d386985c04e2880e in_seq=0 out_seq=0 2023-11-08T16:38:11.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.609+0000 7f517b7fe640 10 client.7422 ms_handle_connect on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:11.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f517b7fe640 1 -- 172.21.15.81:0/534430353 <== mds.0 v2:172.21.15.81:6836/1843050442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56107a6207a0 con 0x56107a6fd750 2023-11-08T16:38:11.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f517b7fe640 10 client.7422 handle_command_reply: tid=0 2023-11-08T16:38:11.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 1 client.7422 shutdown 2023-11-08T16:38:11.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 1 -- 172.21.15.81:0/534430353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a701c00 msgr2=0x56107a6ee8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a701c00 0x56107a6ee8d0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f519400af60 tx=0x7f519400af90 comp rx=0 tx=0).stop 2023-11-08T16:38:11.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 1 -- 172.21.15.81:0/534430353 shutdown_connections 2023-11-08T16:38:11.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x56107a6fd750 0x56107a6fdb30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a701c00 0x56107a6ee8d0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e92a0 0x56107a6ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/534430353 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6e8ec0 0x56107a7013d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 1 -- 172.21.15.81:0/534430353 >> 172.21.15.81:0/534430353 conn(0x56107a6bc300 msgr2=0x56107a6bfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:11.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f517b7fe640 0 client.7422 ms_handle_reset on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:11.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 1 -- 172.21.15.81:0/534430353 shutdown_connections 2023-11-08T16:38:11.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 1 -- 172.21.15.81:0/534430353 wait complete. 2023-11-08T16:38:11.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.610+0000 7f51a5bf7740 20 client.7422 trim_cache size 0 max 16384 2023-11-08T16:38:11.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.627+0000 7f51a5bf7740 1 Processor -- start 2023-11-08T16:38:11.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.627+0000 7f51a5bf7740 1 -- start start 2023-11-08T16:38:11.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.628+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e8ec0 0x56107a6bae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.628+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e92a0 0x56107a6bb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.628+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a701c00 0x56107a6e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.628+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a6ee620 con 0x56107a6e92a0 2023-11-08T16:38:11.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.628+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a6c0930 con 0x56107a6e8ec0 2023-11-08T16:38:11.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.628+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56107a6ff3d0 con 0x56107a701c00 2023-11-08T16:38:11.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.628+0000 7f519bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e92a0 0x56107a6bb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.628+0000 7f519bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e92a0 0x56107a6bb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60038/0 (socket says 172.21.15.81:60038) 2023-11-08T16:38:11.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.628+0000 7f519bfff640 1 -- 172.21.15.81:0/1675323050 learned_addr learned my addr 172.21.15.81:0/1675323050 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:11.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f51a0d25640 1 --2- 172.21.15.81:0/1675323050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e8ec0 0x56107a6bae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f51a1526640 1 --2- 172.21.15.81:0/1675323050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a701c00 0x56107a6e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f519bfff640 1 -- 172.21.15.81:0/1675323050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a701c00 msgr2=0x56107a6e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f519bfff640 1 --2- 172.21.15.81:0/1675323050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a701c00 0x56107a6e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f519bfff640 1 -- 172.21.15.81:0/1675323050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e8ec0 msgr2=0x56107a6bae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f519bfff640 1 --2- 172.21.15.81:0/1675323050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e8ec0 0x56107a6bae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f519bfff640 1 -- 172.21.15.81:0/1675323050 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56107a5c3b90 con 0x56107a6e92a0 2023-11-08T16:38:11.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f51a1526640 1 --2- 172.21.15.81:0/1675323050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a701c00 0x56107a6e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:11.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f519bfff640 1 --2- 172.21.15.81:0/1675323050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e92a0 0x56107a6bb380 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f51900090b0 tx=0x7f519002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81920ebb31f6450 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:11.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f51837fe640 1 -- 172.21.15.81:0/1675323050 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519003ad30 con 0x56107a6e92a0 2023-11-08T16:38:11.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f51837fe640 1 -- 172.21.15.81:0/1675323050 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519004a070 con 0x56107a6e92a0 2023-11-08T16:38:11.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.629+0000 7f51837fe640 1 -- 172.21.15.81:0/1675323050 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5190004250 con 0x56107a6e92a0 2023-11-08T16:38:11.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.630+0000 7f51a5bf7740 1 -- 172.21.15.81:0/1675323050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e92a0 msgr2=0x56107a6bb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.630+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/1675323050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e92a0 0x56107a6bb380 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f51900090b0 tx=0x7f519002e000 comp rx=0 tx=0).stop 2023-11-08T16:38:11.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.630+0000 7f51a5bf7740 1 -- 172.21.15.81:0/1675323050 shutdown_connections 2023-11-08T16:38:11.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.630+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/1675323050 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a701c00 0x56107a6e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.630+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/1675323050 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e92a0 0x56107a6bb380 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.630+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/1675323050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a6e8ec0 0x56107a6bae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.630+0000 7f51a5bf7740 1 -- 172.21.15.81:0/1675323050 >> 172.21.15.81:0/1675323050 conn(0x56107a6bc300 msgr2=0x56107a701280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:11.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.630+0000 7f51a5bf7740 1 -- 172.21.15.81:0/1675323050 shutdown_connections 2023-11-08T16:38:11.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.630+0000 7f51a5bf7740 1 -- 172.21.15.81:0/1675323050 wait complete. 2023-11-08T16:38:11.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.631+0000 7f51a5bf7740 1 -- start start 2023-11-08T16:38:11.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.631+0000 7f51a5bf7740 1 Processor -- start 2023-11-08T16:38:11.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a70a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6e92a0 0x56107a776660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a5bf7740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a701c00 0x56107a6c0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a776240 con 0x56107a6e8ec0 2023-11-08T16:38:11.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56107a6c1770 con 0x56107a701c00 2023-11-08T16:38:11.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a5bf7740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56107a6ed2c0 con 0x56107a6e92a0 2023-11-08T16:38:11.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a5bf7740 10 client.0 _ll_register_callbacks cb 0x7f519400a0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:38:11.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a0d25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a70a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a0d25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a70a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60054/0 (socket says 172.21.15.81:60054) 2023-11-08T16:38:11.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a0d25640 1 -- 172.21.15.81:0/3670478615 learned_addr learned my addr 172.21.15.81:0/3670478615 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:11.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a0d25640 1 -- 172.21.15.81:0/3670478615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6e92a0 msgr2=0x56107a776660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a0d25640 1 --2- 172.21.15.81:0/3670478615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6e92a0 0x56107a776660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a0d25640 1 -- 172.21.15.81:0/3670478615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a701c00 msgr2=0x56107a6c0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a0d25640 1 --2- 172.21.15.81:0/3670478615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a701c00 0x56107a6c0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.632+0000 7f51a0d25640 1 -- 172.21.15.81:0/3670478615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5190040070 con 0x56107a6e8ec0 2023-11-08T16:38:11.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f51a0d25640 1 --2- 172.21.15.81:0/3670478615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a70a160 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f518c05f9d0 tx=0x7f518c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7bab4f20693e3c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:11.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f517b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:38:11.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f517b7fe640 1 -- 172.21.15.81:0/3670478615 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0579d0 con 0x56107a6e8ec0 2023-11-08T16:38:11.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f517b7fe640 1 -- 172.21.15.81:0/3670478615 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f518c0870b0 con 0x56107a6e8ec0 2023-11-08T16:38:11.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f51a5bf7740 1 -- 172.21.15.81:0/3670478615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f518c055fa0 con 0x56107a6e8ec0 2023-11-08T16:38:11.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f517b7fe640 1 -- 172.21.15.81:0/3670478615 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f518c054310 con 0x56107a6e8ec0 2023-11-08T16:38:11.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f51a5bf7740 10 client.7428 fetch_fsmap learned FSMap version 78 2023-11-08T16:38:11.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f51a5bf7740 1 -- 172.21.15.81:0/3670478615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x56107a5e9180 con 0x56107a6e8ec0 2023-11-08T16:38:11.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f517b7fe640 1 -- 172.21.15.81:0/3670478615 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c055fa0 con 0x56107a6e8ec0 2023-11-08T16:38:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f517b7fe640 1 -- 172.21.15.81:0/3670478615 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 78) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f518c0570f0 con 0x56107a6e8ec0 2023-11-08T16:38:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f51a5bf7740 10 client.7428 fetch_fsmap finished waiting for FSMap version 78 2023-11-08T16:38:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f51a5bf7740 10 client.7428 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-08T16:38:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f51a5bf7740 20 client.7428 populate_metadata read hostname 'smithi081' 2023-11-08T16:38:11.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/3670478615 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x56107a6bae40 0x56107a6bb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f51a5bf7740 4 client.7428 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:38:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.633+0000 7f51a5bf7740 1 -- 172.21.15.81:0/3670478615 --> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56107a6204c0 con 0x56107a6bae40 2023-11-08T16:38:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.634+0000 7f519bfff640 1 --2- 172.21.15.81:0/3670478615 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x56107a6bae40 0x56107a6bb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.634+0000 7f517b7fe640 10 client.7428 ms_handle_connect on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:11.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.634+0000 7f519bfff640 1 --2- 172.21.15.81:0/3670478615 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x56107a6bae40 0x56107a6bb220 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf2379a3a7aa021 server_cookie=e8a3d08d13163086 in_seq=0 out_seq=0 2023-11-08T16:38:11.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.634+0000 7f517b7fe640 1 -- 172.21.15.81:0/3670478615 <== mds.0 v2:172.21.15.81:6836/1843050442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x56107a6204c0 con 0x56107a6bae40 2023-11-08T16:38:11.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.634+0000 7f517b7fe640 10 client.7428 handle_command_reply: tid=0 2023-11-08T16:38:11.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.634+0000 7f51a5bf7740 1 client.7428 shutdown 2023-11-08T16:38:11.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a5bf7740 1 -- 172.21.15.81:0/3670478615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 msgr2=0x56107a70a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/3670478615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a70a160 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f518c05f9d0 tx=0x7f518c01e040 comp rx=0 tx=0).stop 2023-11-08T16:38:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a5bf7740 1 -- 172.21.15.81:0/3670478615 shutdown_connections 2023-11-08T16:38:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/3670478615 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x56107a6bae40 0x56107a6bb220 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/3670478615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56107a701c00 0x56107a6c0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/3670478615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x56107a6e92a0 0x56107a776660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a5bf7740 1 --2- 172.21.15.81:0/3670478615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56107a6e8ec0 0x56107a70a160 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f517b7fe640 0 client.7428 ms_handle_reset on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a5bf7740 1 -- 172.21.15.81:0/3670478615 >> 172.21.15.81:0/3670478615 conn(0x56107a6bc300 msgr2=0x56107a6b7560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a5bf7740 1 -- 172.21.15.81:0/3670478615 shutdown_connections 2023-11-08T16:38:11.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a5bf7740 1 -- 172.21.15.81:0/3670478615 wait complete. 2023-11-08T16:38:11.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a5bf7740 20 client.7428 trim_cache size 0 max 16384 2023-11-08T16:38:11.667 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:38:11.667 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:38:11.668 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:38:11.668 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 7383, 2023-11-08T16:38:11.668 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:38:11.668 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:38:11.668 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:38:11.669 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 4.6415318350000003, 2023-11-08T16:38:11.669 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 78, 2023-11-08T16:38:11.669 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 90, 2023-11-08T16:38:11.669 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 90, 2023-11-08T16:38:11.669 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 4.7753535200000004 2023-11-08T16:38:11.670 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:38:11.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a2f89640 1 -- 172.21.15.81:0/3590625252 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f518806ae70 msgr2=0x7f518806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a2f89640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f518806ae70 0x7f518806d330 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f5190036000 tx=0x7f5190030000 comp rx=0 tx=0).stop 2023-11-08T16:38:11.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a2f89640 1 -- 172.21.15.81:0/3590625252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f519c0c90f0 msgr2=0x7f519c1f7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.635+0000 7f51a2f89640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f519c0c90f0 0x7f519c1f7940 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f518c00c770 tx=0x7f518c00cc30 comp rx=0 tx=0).stop 2023-11-08T16:38:11.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.636+0000 7f51a2f89640 1 -- 172.21.15.81:0/3590625252 shutdown_connections 2023-11-08T16:38:11.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.636+0000 7f51a2f89640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f518806ae70 0x7f518806d330 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.636+0000 7f51a2f89640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f519c0cd2a0 0x7f519c0c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.636+0000 7f51a2f89640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f519c0cc8e0 0x7f519c0c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.636+0000 7f51a2f89640 1 --2- 172.21.15.81:0/3590625252 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f519c0c90f0 0x7f519c1f7940 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.636+0000 7f51a2f89640 1 -- 172.21.15.81:0/3590625252 >> 172.21.15.81:0/3590625252 conn(0x7f519c0ceaa0 msgr2=0x7f519c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:11.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.636+0000 7f51a2f89640 1 -- 172.21.15.81:0/3590625252 shutdown_connections 2023-11-08T16:38:11.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.636+0000 7f51a2f89640 1 -- 172.21.15.81:0/3590625252 wait complete. 2023-11-08T16:38:11.676 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:11.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.890+0000 7f48f759e640 1 Processor -- start 2023-11-08T16:38:11.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.890+0000 7f48f759e640 1 -- start start 2023-11-08T16:38:11.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.892+0000 7f48f759e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f48f815abb0 0x7f48f80c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.892+0000 7f48f759e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48f80ca520 0x7f48f80aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.892+0000 7f48f759e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48f815e830 con 0x7f48f80cd770 2023-11-08T16:38:11.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.892+0000 7f48f759e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48f815f4b0 con 0x7f48f815abb0 2023-11-08T16:38:11.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.892+0000 7f48f759e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80ca990 con 0x7f48f80ca520 2023-11-08T16:38:11.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.892+0000 7f48f5d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f48f80cd770 0x7f48f80ae790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49200/0 (socket says 172.21.15.81:49200) 2023-11-08T16:38:11.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.892+0000 7f48f5d9b640 1 -- 172.21.15.81:0/3872085089 learned_addr learned my addr 172.21.15.81:0/3872085089 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:11.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.892+0000 7f48f659c640 1 --2- 172.21.15.81:0/3872085089 >> v2:172.21.15.93:3300/0 conn(0x7f48f815abb0 0x7f48f80c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f6d9d640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48f80ca520 0x7f48f80aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3942943560 0 0) 0x7f48f815e830 con 0x7f48f80cd770 2023-11-08T16:38:11.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48e0003520 con 0x7f48f80cd770 2023-11-08T16:38:11.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1657123795 0 0) 0x7f48e0003520 con 0x7f48f80cd770 2023-11-08T16:38:11.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48f80ca520 msgr2=0x7f48f80aeea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48f80ca520 0x7f48f80aeea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 >> v2:172.21.15.93:3300/0 conn(0x7f48f815abb0 msgr2=0x7f48f80c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 --2- 172.21.15.81:0/3872085089 >> v2:172.21.15.93:3300/0 conn(0x7f48f815abb0 0x7f48f80c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f815eb10 con 0x7f48f80cd770 2023-11-08T16:38:11.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f48ec001670 con 0x7f48f80cd770 2023-11-08T16:38:11.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 >> v1:172.21.15.81:6789/0 conn(0x7f48f80cd770 legacy=0x7f48f80ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48e0003a50 0x7f48e0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48f80ca520 0x7f48e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48e0004d00 0x7f48e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f815f4b0 con 0x7f48e0004d00 2023-11-08T16:38:11.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80ca990 con 0x7f48e0003a50 2023-11-08T16:38:11.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.893+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e000ba50 con 0x7f48f80ca520 2023-11-08T16:38:11.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.894+0000 7f48f6d9d640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48e0004d00 0x7f48e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.894+0000 7f48f5d9b640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48f80ca520 0x7f48e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.894+0000 7f48f659c640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48e0003a50 0x7f48e0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.894+0000 7f48f6d9d640 1 -- 172.21.15.81:0/3872085089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48f80ca520 msgr2=0x7f48e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.894+0000 7f48f6d9d640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48f80ca520 0x7f48e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.894+0000 7f48f6d9d640 1 -- 172.21.15.81:0/3872085089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48e0003a50 msgr2=0x7f48e0003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:11.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.894+0000 7f48f6d9d640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48e0003a50 0x7f48e0003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.894+0000 7f48f6d9d640 1 -- 172.21.15.81:0/3872085089 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e0003520 con 0x7f48e0004d00 2023-11-08T16:38:11.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.895+0000 7f48f6d9d640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48e0004d00 0x7f48e0005180 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009fc0 tx=0x7f48e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676fcc159bc1b10f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:11.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.895+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f002cc20 con 0x7f48e0004d00 2023-11-08T16:38:11.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.895+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f000f040 con 0x7f48e0004d00 2023-11-08T16:38:11.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.895+0000 7f48f559a640 1 -- 172.21.15.81:0/3872085089 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f0034440 con 0x7f48e0004d00 2023-11-08T16:38:11.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.896+0000 7f48f759e640 1 -- 172.21.15.81:0/3872085089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48e0004d00 msgr2=0x7f48e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.896+0000 7f48f759e640 1 --2- 172.21.15.81:0/3872085089 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48e0004d00 0x7f48e0005180 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f48f0009fc0 tx=0x7f48e00057f0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.896+0000 7f48f6d9d640 1 -- 172.21.15.81:0/3872085089 reap_dead start 2023-11-08T16:38:11.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.896+0000 7f48f759e640 1 -- 172.21.15.81:0/3872085089 shutdown_connections 2023-11-08T16:38:11.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.896+0000 7f48f759e640 1 -- 172.21.15.81:0/3872085089 >> 172.21.15.81:0/3872085089 conn(0x7f48f80cc410 msgr2=0x7f48f80cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:11.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.896+0000 7f48f759e640 1 -- 172.21.15.81:0/3872085089 shutdown_connections 2023-11-08T16:38:11.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.896+0000 7f48f759e640 1 -- 172.21.15.81:0/3872085089 wait complete. 2023-11-08T16:38:11.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.898+0000 7f48f759e640 1 Processor -- start 2023-11-08T16:38:11.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.898+0000 7f48f759e640 1 -- start start 2023-11-08T16:38:11.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.899+0000 7f48f759e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48e0004d00 0x7f48f8154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.899+0000 7f48f759e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48f80b2d00 0x7f48f80c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.899+0000 7f48f759e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48f80b02e0 0x7f48f80b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.899+0000 7f48f759e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80ac8a0 con 0x7f48f80b2d00 2023-11-08T16:38:11.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.899+0000 7f48f759e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80ad420 con 0x7f48f80b02e0 2023-11-08T16:38:11.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.899+0000 7f48f759e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f80b0ca0 con 0x7f48e0004d00 2023-11-08T16:38:11.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.899+0000 7f48f5d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48f80b2d00 0x7f48f80c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.899+0000 7f48f5d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48f80b2d00 0x7f48f80c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60064/0 (socket says 172.21.15.81:60064) 2023-11-08T16:38:11.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.899+0000 7f48f5d9b640 1 -- 172.21.15.81:0/2567502097 learned_addr learned my addr 172.21.15.81:0/2567502097 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:11.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.899+0000 7f48f659c640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48e0004d00 0x7f48f8154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.899+0000 7f48f6d9d640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48f80b02e0 0x7f48f80b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.900+0000 7f48f5d9b640 1 -- 172.21.15.81:0/2567502097 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48e0004d00 msgr2=0x7f48f8154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.900+0000 7f48f5d9b640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48e0004d00 0x7f48f8154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.900+0000 7f48f5d9b640 1 -- 172.21.15.81:0/2567502097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48f80b02e0 msgr2=0x7f48f80b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:11.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.900+0000 7f48f5d9b640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48f80b02e0 0x7f48f80b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:11.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.900+0000 7f48f5d9b640 1 -- 172.21.15.81:0/2567502097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f80acaf0 con 0x7f48f80b2d00 2023-11-08T16:38:11.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.900+0000 7f48f5d9b640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48f80b2d00 0x7f48f80c5220 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0017a0 tx=0x7f48ec00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:11.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.900+0000 7f48e6ffd640 1 -- 172.21.15.81:0/2567502097 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00c870 con 0x7f48f80b2d00 2023-11-08T16:38:11.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.900+0000 7f48e6ffd640 1 -- 172.21.15.81:0/2567502097 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec00c9f0 con 0x7f48f80b2d00 2023-11-08T16:38:11.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.900+0000 7f48e6ffd640 1 -- 172.21.15.81:0/2567502097 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec00cda0 con 0x7f48f80b2d00 2023-11-08T16:38:11.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.900+0000 7f48f759e640 1 -- 172.21.15.81:0/2567502097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48f80acdd0 con 0x7f48f80b2d00 2023-11-08T16:38:11.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.900+0000 7f48f759e640 1 -- 172.21.15.81:0/2567502097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48f80c5760 con 0x7f48f80b2d00 2023-11-08T16:38:11.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.901+0000 7f48f759e640 1 -- 172.21.15.81:0/2567502097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48f8067800 con 0x7f48f80b2d00 2023-11-08T16:38:11.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.903+0000 7f48e6ffd640 1 -- 172.21.15.81:0/2567502097 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f48ec01d460 con 0x7f48f80b2d00 2023-11-08T16:38:11.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.903+0000 7f48e6ffd640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f48d806aea0 0x7f48d806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:11.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.903+0000 7f48e6ffd640 1 -- 172.21.15.81:0/2567502097 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f48ec089930 con 0x7f48f80b2d00 2023-11-08T16:38:11.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.904+0000 7f48f659c640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f48d806aea0 0x7f48d806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:11.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.904+0000 7f48f659c640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f48d806aea0 0x7f48d806d360 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009a10 tx=0x7f48e8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:11.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:11.908+0000 7f48e6ffd640 1 -- 172.21.15.81:0/2567502097 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f48ec054670 con 0x7f48f80b2d00 2023-11-08T16:38:12.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.136+0000 7f48f759e640 1 -- 172.21.15.81:0/2567502097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48f80c58f0 con 0x7f48f80b2d00 2023-11-08T16:38:12.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.137+0000 7f48e6ffd640 1 -- 172.21.15.81:0/2567502097 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f48ec057c90 con 0x7f48f80b2d00 2023-11-08T16:38:12.139 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:12.139 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":78,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1527116581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1527116581},{"type":"v1","addr":"172.21.15.93:6833","nonce":1527116581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2617548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2617548066},{"type":"v1","addr":"172.21.15.93:6835","nonce":2617548066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":7380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2591582634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2591582634},{"type":"v1","addr":"172.21.15.81:6835","nonce":2591582634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:09.004709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/1843050442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1843050442},{"type":"v1","addr":"172.21.15.81:6837","nonce":1843050442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:12.140 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 78 2023-11-08T16:38:12.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.139+0000 7f48f759e640 1 -- 172.21.15.81:0/2567502097 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f48d806aea0 msgr2=0x7f48d806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:12.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.139+0000 7f48f759e640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f48d806aea0 0x7f48d806d360 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009a10 tx=0x7f48e8008040 comp rx=0 tx=0).stop 2023-11-08T16:38:12.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.139+0000 7f48f759e640 1 -- 172.21.15.81:0/2567502097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48f80b2d00 msgr2=0x7f48f80c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:12.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.139+0000 7f48f759e640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48f80b2d00 0x7f48f80c5220 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f48ec0017a0 tx=0x7f48ec00ecd0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.140+0000 7f48f759e640 1 -- 172.21.15.81:0/2567502097 shutdown_connections 2023-11-08T16:38:12.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.140+0000 7f48f759e640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f48d806aea0 0x7f48d806d360 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.140+0000 7f48f759e640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f48f80b02e0 0x7f48f80b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.140+0000 7f48f759e640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f48f80b2d00 0x7f48f80c5220 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.140+0000 7f48f759e640 1 --2- 172.21.15.81:0/2567502097 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f48e0004d00 0x7f48f8154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.140+0000 7f48f759e640 1 -- 172.21.15.81:0/2567502097 >> 172.21.15.81:0/2567502097 conn(0x7f48f80cc410 msgr2=0x7f48f80cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:12.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.140+0000 7f48f759e640 1 -- 172.21.15.81:0/2567502097 shutdown_connections 2023-11-08T16:38:12.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.140+0000 7f48f759e640 1 -- 172.21.15.81:0/2567502097 wait complete. 2023-11-08T16:38:12.160 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 4, "epoch": 78, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 4, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-08T16:37:59.633801+0000", "damaged": [], "data_pools": [ 11, 12 ], "enabled": true, "epoch": 78, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_7383": { "addr": "172.21.15.81:6837/1843050442", "addrs": { "addrvec": [ { "addr": "172.21.15.81:6836", "nonce": 1843050442, "type": "v2" }, { "addr": "172.21.15.81:6837", "nonce": 1843050442, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7383, "incarnation": 75, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 88, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 10, "modified": "2023-11-08T16:38:09.004709+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 7383 } } } ], "standbys": [ { "addr": "172.21.15.93:6833/1527116581", "addrs": { "addrvec": [ { "addr": "172.21.15.93:6832", "nonce": 1527116581, "type": "v2" }, { "addr": "172.21.15.93:6833", "nonce": 1527116581, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 75, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5857, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.93:6835/2617548066", "addrs": { "addrvec": [ { "addr": "172.21.15.93:6834", "nonce": 2617548066, "type": "v2" }, { "addr": "172.21.15.93:6835", "nonce": 2617548066, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 76, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 5860, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.81:6835/2591582634", "addrs": { "addrvec": [ { "addr": "172.21.15.81:6834", "nonce": 2591582634, "type": "v2" }, { "addr": "172.21.15.81:6835", "nonce": 2591582634, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 74, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 7380, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-08T16:38:12.160 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:38:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.373+0000 7f2193fce640 1 Processor -- start 2023-11-08T16:38:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.373+0000 7f2193fce640 1 -- start start 2023-11-08T16:38:12.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.375+0000 7f2193fce640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218c15d2b0 0x7f218c161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:12.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.375+0000 7f2193fce640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f218c0c9060 0x7f218c0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:12.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.375+0000 7f2193fce640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f218c0c94e0 0x7f218c0ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:12.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.375+0000 7f2193fce640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218c15d690 con 0x7f218c0c94e0 2023-11-08T16:38:12.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.375+0000 7f2193fce640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f218c161c50 con 0x7f218c0c9060 2023-11-08T16:38:12.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.375+0000 7f2193fce640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c06b980 con 0x7f218c15d2b0 2023-11-08T16:38:12.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.375+0000 7f219256b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f218c0c94e0 0x7f218c0ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:12.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.375+0000 7f219256b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f218c0c94e0 0x7f218c0ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60068/0 (socket says 172.21.15.81:60068) 2023-11-08T16:38:12.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.375+0000 7f219256b640 1 -- 172.21.15.81:0/503909161 learned_addr learned my addr 172.21.15.81:0/503909161 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:12.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.376+0000 7f2191569640 1 --2- 172.21.15.81:0/503909161 >> v2:172.21.15.93:3300/0 conn(0x7f218c0c9060 0x7f218c0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.376+0000 7f2191d6a640 1 --2- 172.21.15.81:0/503909161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218c15d2b0 0x7f218c161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.376+0000 7f219256b640 1 -- 172.21.15.81:0/503909161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218c15d2b0 msgr2=0x7f218c161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.376+0000 7f219256b640 1 --2- 172.21.15.81:0/503909161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218c15d2b0 0x7f218c161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.376+0000 7f219256b640 1 -- 172.21.15.81:0/503909161 >> v2:172.21.15.93:3300/0 conn(0x7f218c0c9060 msgr2=0x7f218c0cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.376+0000 7f219256b640 1 --2- 172.21.15.81:0/503909161 >> v2:172.21.15.93:3300/0 conn(0x7f218c0c9060 0x7f218c0cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.376+0000 7f219256b640 1 -- 172.21.15.81:0/503909161 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218c06bc60 con 0x7f218c0c94e0 2023-11-08T16:38:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.377+0000 7f219256b640 1 --2- 172.21.15.81:0/503909161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218c0c94e0 0x7f218c0ccd30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f218c06ddc0 tx=0x7f218402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ece8d93fa2a06cf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.377+0000 7f2190d68640 1 -- 172.21.15.81:0/503909161 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218402ca70 con 0x7f218c0c94e0 2023-11-08T16:38:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.377+0000 7f2190d68640 1 -- 172.21.15.81:0/503909161 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218402cbd0 con 0x7f218c0c94e0 2023-11-08T16:38:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.377+0000 7f2190d68640 1 -- 172.21.15.81:0/503909161 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21840343f0 con 0x7f218c0c94e0 2023-11-08T16:38:12.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.378+0000 7f2193fce640 1 -- 172.21.15.81:0/503909161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218c0c94e0 msgr2=0x7f218c0ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:12.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.378+0000 7f2193fce640 1 --2- 172.21.15.81:0/503909161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218c0c94e0 0x7f218c0ccd30 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f218c06ddc0 tx=0x7f218402bed0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.378+0000 7f2193fce640 1 -- 172.21.15.81:0/503909161 shutdown_connections 2023-11-08T16:38:12.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.378+0000 7f2193fce640 1 --2- 172.21.15.81:0/503909161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218c0c94e0 0x7f218c0ccd30 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.378+0000 7f2193fce640 1 --2- 172.21.15.81:0/503909161 >> v2:172.21.15.93:3300/0 conn(0x7f218c0c9060 0x7f218c0cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.378+0000 7f2193fce640 1 --2- 172.21.15.81:0/503909161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218c15d2b0 0x7f218c161710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.378+0000 7f2193fce640 1 -- 172.21.15.81:0/503909161 >> 172.21.15.81:0/503909161 conn(0x7f218c0cb490 msgr2=0x7f218c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:12.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.378+0000 7f2193fce640 1 -- 172.21.15.81:0/503909161 shutdown_connections 2023-11-08T16:38:12.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.378+0000 7f2193fce640 1 -- 172.21.15.81:0/503909161 wait complete. 2023-11-08T16:38:12.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.380+0000 7f2193fce640 1 Processor -- start 2023-11-08T16:38:12.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.380+0000 7f2193fce640 1 -- start start 2023-11-08T16:38:12.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.381+0000 7f2193fce640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218c0c9060 0x7f218c152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:12.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.381+0000 7f2193fce640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f218c0c94e0 0x7f218c15f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:12.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.381+0000 7f2193fce640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218c15d2b0 0x7f218c0af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:12.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.381+0000 7f2193fce640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c15d890 con 0x7f218c0c9060 2023-11-08T16:38:12.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.381+0000 7f2193fce640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218c15fcb0 con 0x7f218c0c94e0 2023-11-08T16:38:12.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.381+0000 7f2193fce640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218c15ff90 con 0x7f218c15d2b0 2023-11-08T16:38:12.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.381+0000 7f2191d6a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218c0c9060 0x7f218c152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:12.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.381+0000 7f2191d6a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218c0c9060 0x7f218c152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60072/0 (socket says 172.21.15.81:60072) 2023-11-08T16:38:12.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.381+0000 7f2191d6a640 1 -- 172.21.15.81:0/3051148943 learned_addr learned my addr 172.21.15.81:0/3051148943 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:12.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f219256b640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218c15d2b0 0x7f218c0af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:12.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f2191d6a640 1 -- 172.21.15.81:0/3051148943 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218c15d2b0 msgr2=0x7f218c0af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:12.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f2191d6a640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218c15d2b0 0x7f218c0af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f2191d6a640 1 -- 172.21.15.81:0/3051148943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f218c0c94e0 msgr2=0x7f218c15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:12.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f2191d6a640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f218c0c94e0 0x7f218c15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f2191d6a640 1 -- 172.21.15.81:0/3051148943 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2184009b30 con 0x7f218c0c9060 2023-11-08T16:38:12.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f2191d6a640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218c0c9060 0x7f218c152f60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f218800d800 tx=0x7f218800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:12.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f21827fc640 1 -- 172.21.15.81:0/3051148943 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218800b6c0 con 0x7f218c0c9060 2023-11-08T16:38:12.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f21827fc640 1 -- 172.21.15.81:0/3051148943 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218800b820 con 0x7f218c0c9060 2023-11-08T16:38:12.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f2193fce640 1 -- 172.21.15.81:0/3051148943 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218c160270 con 0x7f218c0c9060 2023-11-08T16:38:12.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f21827fc640 1 -- 172.21.15.81:0/3051148943 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218800bb90 con 0x7f218c0c9060 2023-11-08T16:38:12.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.382+0000 7f2193fce640 1 -- 172.21.15.81:0/3051148943 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218c1534a0 con 0x7f218c0c9060 2023-11-08T16:38:12.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.383+0000 7f2193fce640 1 -- 172.21.15.81:0/3051148943 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f218c04ece0 con 0x7f218c0c9060 2023-11-08T16:38:12.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.384+0000 7f21827fc640 1 -- 172.21.15.81:0/3051148943 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f218800bcf0 con 0x7f218c0c9060 2023-11-08T16:38:12.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.386+0000 7f21827fc640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f217006ae70 0x7f217006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:12.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.386+0000 7f21827fc640 1 -- 172.21.15.81:0/3051148943 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f2188089e70 con 0x7f218c0c9060 2023-11-08T16:38:12.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.386+0000 7f2191569640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f217006ae70 0x7f217006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.387+0000 7f2191569640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f217006ae70 0x7f217006d330 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f217c006fd0 tx=0x7f217c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:12.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.390+0000 7f21827fc640 1 -- 172.21.15.81:0/3051148943 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2188054c50 con 0x7f218c0c9060 2023-11-08T16:38:12.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.620+0000 7f2193fce640 1 -- 172.21.15.81:0/3051148943 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f218c0b4990 con 0x7f218c0c9060 2023-11-08T16:38:12.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.621+0000 7f21827fc640 1 -- 172.21.15.81:0/3051148943 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v78) v1 ==== 53+0+91 (secure 0 0 0) 0x7f2188058270 con 0x7f218c0c9060 2023-11-08T16:38:12.624 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-11-08T16:38:12.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.623+0000 7f2193fce640 1 -- 172.21.15.81:0/3051148943 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f217006ae70 msgr2=0x7f217006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:12.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.623+0000 7f2193fce640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f217006ae70 0x7f217006d330 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f217c006fd0 tx=0x7f217c008040 comp rx=0 tx=0).stop 2023-11-08T16:38:12.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.623+0000 7f2193fce640 1 -- 172.21.15.81:0/3051148943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218c0c9060 msgr2=0x7f218c152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:12.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.623+0000 7f2193fce640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218c0c9060 0x7f218c152f60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f218800d800 tx=0x7f218800dcc0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.624+0000 7f2193fce640 1 -- 172.21.15.81:0/3051148943 shutdown_connections 2023-11-08T16:38:12.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.624+0000 7f2193fce640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f217006ae70 0x7f217006d330 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.624+0000 7f2193fce640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218c15d2b0 0x7f218c0af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.624+0000 7f2193fce640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f218c0c94e0 0x7f218c15f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.624+0000 7f2193fce640 1 --2- 172.21.15.81:0/3051148943 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218c0c9060 0x7f218c152f60 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:12.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.624+0000 7f2193fce640 1 -- 172.21.15.81:0/3051148943 >> 172.21.15.81:0/3051148943 conn(0x7f218c0cb490 msgr2=0x7f218c160830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:12.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.624+0000 7f2193fce640 1 -- 172.21.15.81:0/3051148943 shutdown_connections 2023-11-08T16:38:12.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:12.624+0000 7f2193fce640 1 -- 172.21.15.81:0/3051148943 wait complete. 2023-11-08T16:38:12.645 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:38:12.646 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:38:12.646 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:38:12.646 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:38:12.647 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:38:12.647 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:38:12.647 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:38:12.648 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:38:12.648 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:38:12.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:12.649 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:38:12.701 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:38:12.701 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:38:12.701 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:38:12.702 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:38:12.702 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:38:12.702 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:38:12.702 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:38:12.703 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:38:12.703 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:38:12.704 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:38:12.704 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:38:12.704 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:38:12.705 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:38:12.705 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:38:12.705 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:38:12.706 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:38:12.706 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:38:12.706 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 41817sec preferred_lft 41817sec 2023-11-08T16:38:12.707 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:38:12.708 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:38:12.708 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:38:12.708 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff 2023-11-08T16:38:12.709 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:38:12.709 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:38:12.709 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:38:12.710 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:38:12.710 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:38:12.710 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:38:12.711 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:38:12.711 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:38:12.712 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:38:12.712 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:38:12.712 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::2cac:17ff:fe64:67da/64 scope link 2023-11-08T16:38:12.712 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:38:12.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:12.714 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:38:12.813 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:38:12.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:12.818 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:38:12.870 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:38:12.871 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:38:12.871 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:38:12.871 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:38:12.871 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:38:12.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:12.872 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:38:12.872 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:38:12.873 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:38:12.873 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:38:12.873 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:38:12.874 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:38:13.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:13.046 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:38:13.099 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:38:13.100 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:38:13.101 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:38:13.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:13.102 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:38:13.200 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:38:13.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:38:13.201 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:38:13.257 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:38:13.258 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:38:13.258 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:38:13.309 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:38:13.402 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:38:13.460 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:38:13.461 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:38:13.461 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:38:13.461 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:38:13.461 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:38:13.462 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:38:13.462 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:38:13.462 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:38:13.462 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:38:13.462 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:38:13.463 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:38:13.463 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:38:13.463 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:38:13.463 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:38:13.463 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:38:13.464 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:38:13.464 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:38:13.464 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:38:13.464 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:38:13.464 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:38:13.465 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:38:13.465 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:38:13.465 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:38:13.465 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:38:13.465 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:38:13.466 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:38:13.466 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:38:13.466 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:38:13.466 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:38:13.467 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:38:13.467 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:38:13.467 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:38:13.467 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:38:13.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:13.469 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:38:13.524 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:38:13.524 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:38:13.530 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:38:13.606 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:38:13.650 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:38:13.650 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:38:13.650 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:38:13.651 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:38:13.651 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:38:13.651 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:38:13.651 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:38:13.651 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:38:13.651 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:38:13.652 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:38:13.652 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:38:13.652 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:38:13.652 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:38:13.653 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:38:13.653 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:38:13.653 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:38:13.653 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:38:13.653 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:38:13.653 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:38:13.654 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:38:13.654 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:38:13.654 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:38:13.654 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:38:13.655 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:38:13.655 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:38:13.655 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:38:13.655 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:38:13.655 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:38:13.656 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:38:13.656 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:38:13.656 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:38:13.656 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:38:13.656 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:38:13.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:13.658 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:38:13.717 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:38:13.715+0000 7fb396041480 -1 init, newargv = 0x55f882b7bf80 newargc=15 2023-11-08T16:38:13.717 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[116226]: starting ceph client 2023-11-08T16:38:13.724 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[116226]: starting fuse 2023-11-08T16:38:13.745 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:38:13.745 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:38:13.746 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:38:13.746 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:38:13.747 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:38:13.747 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:38:13.747 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:38:13.748 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:38:13.748 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:38:13.748 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:38:13.749 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:38:13.749 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:38:13.749 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:38:13.750 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:38:13.750 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:38:13.751 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:38:13.751 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:38:13.751 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:38:13.752 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:38:13.752 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:38:13.752 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:38:13.753 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:38:13.753 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:38:13.754 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:38:13.754 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:38:13.755 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:38:13.755 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:38:13.756 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:38:13.756 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:38:13.952 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.116226.asok 2023-11-08T16:38:13.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:38:13.970 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.116226.asok 2023-11-08T16:38:13.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:13.971 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.116226.asok status 2023-11-08T16:38:14.208 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:38:14.208 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:38:14.209 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:38:14.209 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:38:14.209 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:38:14.210 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:38:14.210 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:38:14.210 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "116226", 2023-11-08T16:38:14.210 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:38:14.211 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:38:14.211 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:38:14.211 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:38:14.211 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 7446, 2023-11-08T16:38:14.211 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:38:14.212 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:38:14.212 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:38:14.212 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 7446 2023-11-08T16:38:14.212 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:38:14.212 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:38:14.213 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:38:14.213 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:38:14.213 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2583057309 2023-11-08T16:38:14.213 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:38:14.214 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:38:14.214 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:38:14.214 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:38:14.214 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:38:14.214 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2583057309 2023-11-08T16:38:14.215 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:38:14.215 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.7446 192.168.0.1:0/2583057309", 2023-11-08T16:38:14.215 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/2583057309", 2023-11-08T16:38:14.215 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:38:14.215 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 78, 2023-11-08T16:38:14.216 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 90, 2023-11-08T16:38:14.216 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:38:14.216 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:38:14.216 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:38:14.216 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:38:14.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:14.223 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:38:14.224 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:38:14.277 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:38:14.278 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:38:14.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:14.279 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:38:14.374 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive 2023-11-08T16:38:14.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.588+0000 7fa07d490640 1 Processor -- start 2023-11-08T16:38:14.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.588+0000 7fa07d490640 1 -- start start 2023-11-08T16:38:14.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.589+0000 7fa07d490640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa078071890 0x7fa07815e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.589+0000 7fa07d490640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa07815afd0 0x7fa07815ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.589+0000 7fa07d490640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0780b09f0 con 0x7fa0780714b0 2023-11-08T16:38:14.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.589+0000 7fa07d490640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07806b250 con 0x7fa07815afd0 2023-11-08T16:38:14.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.589+0000 7fa07d490640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0780ca360 con 0x7fa078071890 2023-11-08T16:38:14.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.590+0000 7fa0777fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa07815afd0 0x7fa07815ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.590+0000 7fa0777fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa07815afd0 0x7fa07815ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60088/0 (socket says 172.21.15.81:60088) 2023-11-08T16:38:14.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.590+0000 7fa0777fe640 1 -- 172.21.15.81:0/140573956 learned_addr learned my addr 172.21.15.81:0/140573956 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:14.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.590+0000 7fa0777fe640 1 -- 172.21.15.81:0/140573956 >> v2:172.21.15.93:3300/0 conn(0x7fa078071890 msgr2=0x7fa07815e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.590+0000 7fa0777fe640 1 --2- 172.21.15.81:0/140573956 >> v2:172.21.15.93:3300/0 conn(0x7fa078071890 0x7fa07815e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.590+0000 7fa0777fe640 1 -- 172.21.15.81:0/140573956 >> v1:172.21.15.93:6789/0 conn(0x7fa0780714b0 legacy=0x7fa07815a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.590+0000 7fa0777fe640 1 -- 172.21.15.81:0/140573956 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0780ca640 con 0x7fa07815afd0 2023-11-08T16:38:14.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.592+0000 7fa0777fe640 1 --2- 172.21.15.81:0/140573956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa07815afd0 0x7fa07815ef10 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa07815e930 tx=0x7fa07002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79c8d1eaf7b7816 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.592+0000 7fa0767fc640 1 -- 172.21.15.81:0/140573956 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07002ca80 con 0x7fa07815afd0 2023-11-08T16:38:14.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.592+0000 7fa0767fc640 1 -- 172.21.15.81:0/140573956 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07002cbe0 con 0x7fa07815afd0 2023-11-08T16:38:14.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.592+0000 7fa0767fc640 1 -- 172.21.15.81:0/140573956 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0700343f0 con 0x7fa07815afd0 2023-11-08T16:38:14.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.593+0000 7fa07d490640 1 -- 172.21.15.81:0/140573956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa07815afd0 msgr2=0x7fa07815ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.593+0000 7fa07d490640 1 --2- 172.21.15.81:0/140573956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa07815afd0 0x7fa07815ef10 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fa07815e930 tx=0x7fa07002bee0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.593+0000 7fa07d490640 1 -- 172.21.15.81:0/140573956 shutdown_connections 2023-11-08T16:38:14.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.593+0000 7fa07d490640 1 --2- 172.21.15.81:0/140573956 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa07815afd0 0x7fa07815ef10 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.593+0000 7fa07d490640 1 --2- 172.21.15.81:0/140573956 >> v2:172.21.15.93:3300/0 conn(0x7fa078071890 0x7fa07815e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.593+0000 7fa07d490640 1 -- 172.21.15.81:0/140573956 >> 172.21.15.81:0/140573956 conn(0x7fa0780cb4e0 msgr2=0x7fa0780cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.593+0000 7fa07d490640 1 -- 172.21.15.81:0/140573956 shutdown_connections 2023-11-08T16:38:14.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.593+0000 7fa07d490640 1 -- 172.21.15.81:0/140573956 wait complete. 2023-11-08T16:38:14.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.595+0000 7fa07d490640 1 Processor -- start 2023-11-08T16:38:14.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.595+0000 7fa07d490640 1 -- start start 2023-11-08T16:38:14.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.596+0000 7fa07d490640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0780714b0 0x7fa0780c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.596+0000 7fa07d490640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa078071890 0x7fa078153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa07d490640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa07815afd0 0x7fa07814c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa07d490640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0781551c0 con 0x7fa07815afd0 2023-11-08T16:38:14.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa07d490640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0780c7580 con 0x7fa078071890 2023-11-08T16:38:14.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa07d490640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0781542c0 con 0x7fa0780714b0 2023-11-08T16:38:14.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa0777fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa07815afd0 0x7fa07814c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa0777fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa07815afd0 0x7fa07814c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60098/0 (socket says 172.21.15.81:60098) 2023-11-08T16:38:14.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa0777fe640 1 -- 172.21.15.81:0/1058153748 learned_addr learned my addr 172.21.15.81:0/1058153748 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:14.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa076ffd640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0780714b0 0x7fa0780c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa06ffff640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa078071890 0x7fa078153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa0777fe640 1 -- 172.21.15.81:0/1058153748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0780714b0 msgr2=0x7fa0780c7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa0777fe640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0780714b0 0x7fa0780c7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa0777fe640 1 -- 172.21.15.81:0/1058153748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa078071890 msgr2=0x7fa078153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa0777fe640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa078071890 0x7fa078153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.597+0000 7fa0777fe640 1 -- 172.21.15.81:0/1058153748 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa070009b30 con 0x7fa07815afd0 2023-11-08T16:38:14.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.598+0000 7fa0777fe640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa07815afd0 0x7fa07814c5f0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa070009f40 tx=0x7fa070004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.598+0000 7fa06f7fe640 1 -- 172.21.15.81:0/1058153748 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07003e450 con 0x7fa07815afd0 2023-11-08T16:38:14.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.598+0000 7fa06f7fe640 1 -- 172.21.15.81:0/1058153748 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07003e5d0 con 0x7fa07815afd0 2023-11-08T16:38:14.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.598+0000 7fa06f7fe640 1 -- 172.21.15.81:0/1058153748 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07003e980 con 0x7fa07815afd0 2023-11-08T16:38:14.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.598+0000 7fa07d490640 1 -- 172.21.15.81:0/1058153748 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa078154540 con 0x7fa07815afd0 2023-11-08T16:38:14.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.598+0000 7fa07d490640 1 -- 172.21.15.81:0/1058153748 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa078154b00 con 0x7fa07815afd0 2023-11-08T16:38:14.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.600+0000 7fa06f7fe640 1 -- 172.21.15.81:0/1058153748 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa07002cd50 con 0x7fa07815afd0 2023-11-08T16:38:14.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.602+0000 7fa06f7fe640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 0x7fa05806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.602+0000 7fa06f7fe640 1 -- 172.21.15.81:0/1058153748 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fa0700ac5b0 con 0x7fa07815afd0 2023-11-08T16:38:14.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.602+0000 7fa076ffd640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 0x7fa05806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.602+0000 7fa080027740 1 Processor -- start 2023-11-08T16:38:14.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.602+0000 7fa080027740 1 -- start start 2023-11-08T16:38:14.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.603+0000 7fa076ffd640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 0x7fa05806d330 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa068007fd0 tx=0x7fa068009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.603+0000 7fa080027740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778730 0x5590c674b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.603+0000 7fa080027740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778b10 0x5590c677b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.603+0000 7fa080027740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c674be00 0x5590c677ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.603+0000 7fa080027740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590c66be5f0 con 0x5590c6778b10 2023-11-08T16:38:14.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.603+0000 7fa080027740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590c66bbce0 con 0x5590c674be00 2023-11-08T16:38:14.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.603+0000 7fa080027740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590c66b75d0 con 0x5590c6778730 2023-11-08T16:38:14.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.604+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778b10 0x5590c677b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.604+0000 7fa06ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778b10 0x5590c677b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60100/0 (socket says 172.21.15.81:60100) 2023-11-08T16:38:14.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.604+0000 7fa06ffff640 1 -- 172.21.15.81:0/3797219062 learned_addr learned my addr 172.21.15.81:0/3797219062 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:14.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.604+0000 7fa076ffd640 1 --2- 172.21.15.81:0/3797219062 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778730 0x5590c674b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.604+0000 7fa0777fe640 1 --2- 172.21.15.81:0/3797219062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c674be00 0x5590c677ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.604+0000 7fa06ffff640 1 -- 172.21.15.81:0/3797219062 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778730 msgr2=0x5590c674b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.604+0000 7fa06ffff640 1 --2- 172.21.15.81:0/3797219062 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778730 0x5590c674b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.604+0000 7fa06ffff640 1 -- 172.21.15.81:0/3797219062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c674be00 msgr2=0x5590c677ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.604+0000 7fa06ffff640 1 --2- 172.21.15.81:0/3797219062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c674be00 0x5590c677ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.604+0000 7fa06ffff640 1 -- 172.21.15.81:0/3797219062 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590c6691a60 con 0x5590c6778b10 2023-11-08T16:38:14.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.604+0000 7fa06ffff640 1 --2- 172.21.15.81:0/3797219062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778b10 0x5590c677b4c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa06000b7c0 tx=0x7fa06000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7b3c03bf567a9c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.605+0000 7fa06cff9640 1 -- 172.21.15.81:0/3797219062 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000d950 con 0x5590c6778b10 2023-11-08T16:38:14.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.605+0000 7fa06cff9640 1 -- 172.21.15.81:0/3797219062 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06000dad0 con 0x5590c6778b10 2023-11-08T16:38:14.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.605+0000 7fa06cff9640 1 -- 172.21.15.81:0/3797219062 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06000de80 con 0x5590c6778b10 2023-11-08T16:38:14.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.605+0000 7fa080027740 1 -- 172.21.15.81:0/3797219062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778b10 msgr2=0x5590c677b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.605+0000 7fa080027740 1 --2- 172.21.15.81:0/3797219062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778b10 0x5590c677b4c0 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa06000b7c0 tx=0x7fa06000bc80 comp rx=0 tx=0).stop 2023-11-08T16:38:14.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.606+0000 7fa080027740 1 -- 172.21.15.81:0/3797219062 shutdown_connections 2023-11-08T16:38:14.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.606+0000 7fa080027740 1 --2- 172.21.15.81:0/3797219062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c674be00 0x5590c677ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.606+0000 7fa080027740 1 --2- 172.21.15.81:0/3797219062 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778b10 0x5590c677b4c0 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.606+0000 7fa080027740 1 --2- 172.21.15.81:0/3797219062 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778730 0x5590c674b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.606+0000 7fa080027740 1 -- 172.21.15.81:0/3797219062 >> 172.21.15.81:0/3797219062 conn(0x5590c674dbd0 msgr2=0x5590c67964c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.606+0000 7fa080027740 1 -- 172.21.15.81:0/3797219062 shutdown_connections 2023-11-08T16:38:14.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.606+0000 7fa080027740 1 -- 172.21.15.81:0/3797219062 wait complete. 2023-11-08T16:38:14.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.606+0000 7fa080027740 1 -- start start 2023-11-08T16:38:14.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.607+0000 7fa080027740 1 Processor -- start 2023-11-08T16:38:14.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa080027740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6778730 0x5590c6797030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa080027740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c6775b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa080027740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6782300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa080027740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590c6654590 con 0x5590c6781e80 2023-11-08T16:38:14.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa080027740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590c6654230 con 0x5590c6778730 2023-11-08T16:38:14.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa080027740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590c641b0c0 con 0x5590c6778b10 2023-11-08T16:38:14.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa080027740 10 client.0 _ll_register_callbacks cb 0x5590c6676700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:38:14.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa0777fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6782300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa0777fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6782300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60106/0 (socket says 172.21.15.81:60106) 2023-11-08T16:38:14.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa0777fe640 1 -- 172.21.15.81:0/3217777907 learned_addr learned my addr 172.21.15.81:0/3217777907 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:14.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa0777fe640 1 -- 172.21.15.81:0/3217777907 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 msgr2=0x5590c6775b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa0777fe640 1 --2- 172.21.15.81:0/3217777907 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c6775b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa0777fe640 1 -- 172.21.15.81:0/3217777907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6778730 msgr2=0x5590c6797030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa0777fe640 1 --2- 172.21.15.81:0/3217777907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6778730 0x5590c6797030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.608+0000 7fa0777fe640 1 -- 172.21.15.81:0/3217777907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06000e040 con 0x5590c6781e80 2023-11-08T16:38:14.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.609+0000 7fa0777fe640 1 --2- 172.21.15.81:0/3217777907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6782300 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa0700070b0 tx=0x7fa07007aa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8df1726398d552 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.609+0000 7fa055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:38:14.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.609+0000 7fa055ffb640 1 -- 172.21.15.81:0/3217777907 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07007a7c0 con 0x5590c6781e80 2023-11-08T16:38:14.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.609+0000 7fa055ffb640 1 -- 172.21.15.81:0/3217777907 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fa07007f220 con 0x5590c6781e80 2023-11-08T16:38:14.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.609+0000 7fa080027740 1 -- 172.21.15.81:0/3217777907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5590c66dd910 con 0x5590c6781e80 2023-11-08T16:38:14.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.609+0000 7fa055ffb640 1 -- 172.21.15.81:0/3217777907 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0700833d0 con 0x5590c6781e80 2023-11-08T16:38:14.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.609+0000 7fa080027740 10 client.7458 fetch_fsmap learned FSMap version 78 2023-11-08T16:38:14.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.609+0000 7fa080027740 1 -- 172.21.15.81:0/3217777907 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x5590c6652810 con 0x5590c6781e80 2023-11-08T16:38:14.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.609+0000 7fa055ffb640 1 -- 172.21.15.81:0/3217777907 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5590c66dd910 con 0x5590c6781e80 2023-11-08T16:38:14.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.609+0000 7fa055ffb640 1 -- 172.21.15.81:0/3217777907 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 78) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fa07004f020 con 0x5590c6781e80 2023-11-08T16:38:14.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.610+0000 7fa080027740 10 client.7458 fetch_fsmap finished waiting for FSMap version 78 2023-11-08T16:38:14.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.610+0000 7fa080027740 10 client.7458 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-08T16:38:14.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.610+0000 7fa080027740 20 client.7458 populate_metadata read hostname 'smithi081' 2023-11-08T16:38:14.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.610+0000 7fa080027740 1 --2- 172.21.15.81:0/3217777907 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x5590c6780da0 0x5590c67811d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.610+0000 7fa080027740 4 client.7458 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:38:14.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.610+0000 7fa080027740 1 -- 172.21.15.81:0/3217777907 --> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590c668a450 con 0x5590c6780da0 2023-11-08T16:38:14.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.610+0000 7fa076ffd640 1 --2- 172.21.15.81:0/3217777907 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x5590c6780da0 0x5590c67811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.610+0000 7fa076ffd640 1 --2- 172.21.15.81:0/3217777907 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x5590c6780da0 0x5590c67811d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51aba1b0ef5baa89 server_cookie=b9059740d806c6c4 in_seq=0 out_seq=0 2023-11-08T16:38:14.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.610+0000 7fa055ffb640 10 client.7458 ms_handle_connect on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:14.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa055ffb640 1 -- 172.21.15.81:0/3217777907 <== mds.0 v2:172.21.15.81:6836/1843050442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590c668a450 con 0x5590c6780da0 2023-11-08T16:38:14.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa055ffb640 10 client.7458 handle_command_reply: tid=0 2023-11-08T16:38:14.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa080027740 1 client.7458 shutdown 2023-11-08T16:38:14.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa080027740 1 -- 172.21.15.81:0/3217777907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 msgr2=0x5590c6782300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa080027740 1 --2- 172.21.15.81:0/3217777907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6782300 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa0700070b0 tx=0x7fa07007aa30 comp rx=0 tx=0).stop 2023-11-08T16:38:14.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa080027740 1 -- 172.21.15.81:0/3217777907 shutdown_connections 2023-11-08T16:38:14.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa080027740 1 --2- 172.21.15.81:0/3217777907 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x5590c6780da0 0x5590c67811d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa080027740 1 --2- 172.21.15.81:0/3217777907 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6782300 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa055ffb640 0 client.7458 ms_handle_reset on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa080027740 1 --2- 172.21.15.81:0/3217777907 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c6775b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa080027740 1 --2- 172.21.15.81:0/3217777907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6778730 0x5590c6797030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.611+0000 7fa080027740 1 -- 172.21.15.81:0/3217777907 >> 172.21.15.81:0/3217777907 conn(0x5590c674be00 msgr2=0x5590c674f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.612+0000 7fa080027740 1 -- 172.21.15.81:0/3217777907 shutdown_connections 2023-11-08T16:38:14.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.612+0000 7fa080027740 1 -- 172.21.15.81:0/3217777907 wait complete. 2023-11-08T16:38:14.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.612+0000 7fa080027740 20 client.7458 trim_cache size 0 max 16384 2023-11-08T16:38:14.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.628+0000 7fa080027740 1 Processor -- start 2023-11-08T16:38:14.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.628+0000 7fa080027740 1 -- start start 2023-11-08T16:38:14.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.629+0000 7fa080027740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6778730 0x5590c67b7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.629+0000 7fa080027740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c67a8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.629+0000 7fa080027740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6797000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.629+0000 7fa080027740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590c67759c0 con 0x5590c6781e80 2023-11-08T16:38:14.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.629+0000 7fa080027740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590c67526f0 con 0x5590c6778730 2023-11-08T16:38:14.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.629+0000 7fa080027740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590c6752a40 con 0x5590c6778b10 2023-11-08T16:38:14.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.629+0000 7fa0777fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6797000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.629+0000 7fa0777fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6797000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60122/0 (socket says 172.21.15.81:60122) 2023-11-08T16:38:14.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.630+0000 7fa0777fe640 1 -- 172.21.15.81:0/1271997845 learned_addr learned my addr 172.21.15.81:0/1271997845 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:14.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.630+0000 7fa076ffd640 1 --2- 172.21.15.81:0/1271997845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6778730 0x5590c67b7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.630+0000 7fa06ffff640 1 --2- 172.21.15.81:0/1271997845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c67a8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.630+0000 7fa0777fe640 1 -- 172.21.15.81:0/1271997845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 msgr2=0x5590c67a8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.630+0000 7fa0777fe640 1 --2- 172.21.15.81:0/1271997845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c67a8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.630+0000 7fa0777fe640 1 -- 172.21.15.81:0/1271997845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6778730 msgr2=0x5590c67b7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.630+0000 7fa0777fe640 1 --2- 172.21.15.81:0/1271997845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6778730 0x5590c67b7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.630+0000 7fa0777fe640 1 -- 172.21.15.81:0/1271997845 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590c66e6c60 con 0x5590c6781e80 2023-11-08T16:38:14.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.630+0000 7fa06ffff640 1 --2- 172.21.15.81:0/1271997845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c67a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:14.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.630+0000 7fa0777fe640 1 --2- 172.21.15.81:0/1271997845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6797000 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa07002c4f0 tx=0x7fa070049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cf24d73bbaeda01 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.630+0000 7fa06d7fa640 1 -- 172.21.15.81:0/1271997845 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07007aea0 con 0x5590c6781e80 2023-11-08T16:38:14.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.631+0000 7fa06d7fa640 1 -- 172.21.15.81:0/1271997845 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07007f220 con 0x5590c6781e80 2023-11-08T16:38:14.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.631+0000 7fa06d7fa640 1 -- 172.21.15.81:0/1271997845 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0700831e0 con 0x5590c6781e80 2023-11-08T16:38:14.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.631+0000 7fa080027740 1 -- 172.21.15.81:0/1271997845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 msgr2=0x5590c6797000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.631+0000 7fa080027740 1 --2- 172.21.15.81:0/1271997845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6797000 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa07002c4f0 tx=0x7fa070049000 comp rx=0 tx=0).stop 2023-11-08T16:38:14.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.631+0000 7fa080027740 1 -- 172.21.15.81:0/1271997845 shutdown_connections 2023-11-08T16:38:14.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.631+0000 7fa080027740 1 --2- 172.21.15.81:0/1271997845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6781e80 0x5590c6797000 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.631+0000 7fa080027740 1 --2- 172.21.15.81:0/1271997845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c67a8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.631+0000 7fa080027740 1 --2- 172.21.15.81:0/1271997845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6778730 0x5590c67b7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.631+0000 7fa080027740 1 -- 172.21.15.81:0/1271997845 >> 172.21.15.81:0/1271997845 conn(0x5590c674be00 msgr2=0x5590c674aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.631+0000 7fa080027740 1 -- 172.21.15.81:0/1271997845 shutdown_connections 2023-11-08T16:38:14.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.631+0000 7fa080027740 1 -- 172.21.15.81:0/1271997845 wait complete. 2023-11-08T16:38:14.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.632+0000 7fa080027740 1 -- start start 2023-11-08T16:38:14.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.632+0000 7fa080027740 1 Processor -- start 2023-11-08T16:38:14.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.633+0000 7fa080027740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778730 0x5590c67989b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.633+0000 7fa080027740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c6798ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.633+0000 7fa080027740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6781e80 0x5590c6801300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.633+0000 7fa080027740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590c67826e0 con 0x5590c6778730 2023-11-08T16:38:14.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.633+0000 7fa080027740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590c67f4d00 con 0x5590c6781e80 2023-11-08T16:38:14.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.633+0000 7fa080027740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590c6802420 con 0x5590c6778b10 2023-11-08T16:38:14.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.633+0000 7fa080027740 10 client.0 _ll_register_callbacks cb 0x7fa070007310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:38:14.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.633+0000 7fa076ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778730 0x5590c67989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa076ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778730 0x5590c67989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60126/0 (socket says 172.21.15.81:60126) 2023-11-08T16:38:14.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa076ffd640 1 -- 172.21.15.81:0/506370135 learned_addr learned my addr 172.21.15.81:0/506370135 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:14.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa06ffff640 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c6798ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa0777fe640 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6781e80 0x5590c6801300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa076ffd640 1 -- 172.21.15.81:0/506370135 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 msgr2=0x5590c6798ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa076ffd640 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c6798ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa076ffd640 1 -- 172.21.15.81:0/506370135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6781e80 msgr2=0x5590c6801300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa076ffd640 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6781e80 0x5590c6801300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa076ffd640 1 -- 172.21.15.81:0/506370135 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06000e040 con 0x5590c6778730 2023-11-08T16:38:14.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa0777fe640 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6781e80 0x5590c6801300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:14.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa076ffd640 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778730 0x5590c67989b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x5590c6781850 tx=0x7fa068000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65dcacd7de3ded85 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.634+0000 7fa055ffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:38:14.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa055ffb640 1 -- 172.21.15.81:0/506370135 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa06800ecb0 con 0x5590c6778730 2023-11-08T16:38:14.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa055ffb640 1 -- 172.21.15.81:0/506370135 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fa06800ee10 con 0x5590c6778730 2023-11-08T16:38:14.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa080027740 1 -- 172.21.15.81:0/506370135 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa06000dad0 con 0x5590c6778730 2023-11-08T16:38:14.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa055ffb640 1 -- 172.21.15.81:0/506370135 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa0680035f0 con 0x5590c6778730 2023-11-08T16:38:14.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa080027740 10 client.7464 fetch_fsmap learned FSMap version 78 2023-11-08T16:38:14.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa080027740 1 -- 172.21.15.81:0/506370135 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x5590c6653580 con 0x5590c6778730 2023-11-08T16:38:14.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa055ffb640 1 -- 172.21.15.81:0/506370135 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa06000dad0 con 0x5590c6778730 2023-11-08T16:38:14.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa055ffb640 1 -- 172.21.15.81:0/506370135 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 78) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fa068003750 con 0x5590c6778730 2023-11-08T16:38:14.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa080027740 10 client.7464 fetch_fsmap finished waiting for FSMap version 78 2023-11-08T16:38:14.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa080027740 10 client.7464 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-08T16:38:14.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa080027740 20 client.7464 populate_metadata read hostname 'smithi081' 2023-11-08T16:38:14.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa080027740 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x5590c67a8ea0 0x5590c6801840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa080027740 4 client.7464 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-08T16:38:14.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.635+0000 7fa080027740 1 -- 172.21.15.81:0/506370135 --> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x5590c66b05b0 con 0x5590c67a8ea0 2023-11-08T16:38:14.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.636+0000 7fa06ffff640 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x5590c67a8ea0 0x5590c6801840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.636+0000 7fa06ffff640 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x5590c67a8ea0 0x5590c6801840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86051dd45bc0704b server_cookie=658046c5fb4d7798 in_seq=0 out_seq=0 2023-11-08T16:38:14.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.636+0000 7fa055ffb640 10 client.7464 ms_handle_connect on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:14.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa055ffb640 1 -- 172.21.15.81:0/506370135 <== mds.0 v2:172.21.15.81:6836/1843050442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5590c66b05b0 con 0x5590c67a8ea0 2023-11-08T16:38:14.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa055ffb640 10 client.7464 handle_command_reply: tid=0 2023-11-08T16:38:14.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa080027740 1 client.7464 shutdown 2023-11-08T16:38:14.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa080027740 1 -- 172.21.15.81:0/506370135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778730 msgr2=0x5590c67989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa080027740 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778730 0x5590c67989b0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x5590c6781850 tx=0x7fa068000fb0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa080027740 1 -- 172.21.15.81:0/506370135 shutdown_connections 2023-11-08T16:38:14.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa080027740 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x5590c67a8ea0 0x5590c6801840 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa080027740 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5590c6781e80 0x5590c6801300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa080027740 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5590c6778b10 0x5590c6798ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa055ffb640 0 client.7464 ms_handle_reset on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:14.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa080027740 1 --2- 172.21.15.81:0/506370135 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5590c6778730 0x5590c67989b0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa080027740 1 -- 172.21.15.81:0/506370135 >> 172.21.15.81:0/506370135 conn(0x5590c674be00 msgr2=0x5590c6775cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa080027740 1 -- 172.21.15.81:0/506370135 shutdown_connections 2023-11-08T16:38:14.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.637+0000 7fa080027740 1 -- 172.21.15.81:0/506370135 wait complete. 2023-11-08T16:38:14.677 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:38:14.638+0000 7f5ebc3c1640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-11-08T16:38:14.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.638+0000 7fa080027740 20 client.7464 trim_cache size 0 max 16384 2023-11-08T16:38:14.677 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:38:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: "return_code": 0, 2023-11-08T16:38:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_tag": "490117e8-9ec4-465d-acd8-42fcc0a84e82", 2023-11-08T16:38:14.678 INFO:teuthology.orchestra.run.smithi081.stdout: "mode": "asynchronous" 2023-11-08T16:38:14.678 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:38:14.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.638+0000 7fa07d490640 1 -- 172.21.15.81:0/1058153748 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 msgr2=0x7fa05806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.638+0000 7fa07d490640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 0x7fa05806d330 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa068007fd0 tx=0x7fa068009040 comp rx=0 tx=0).stop 2023-11-08T16:38:14.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.638+0000 7fa07d490640 1 -- 172.21.15.81:0/1058153748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa07815afd0 msgr2=0x7fa07814c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.638+0000 7fa07d490640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa07815afd0 0x7fa07814c5f0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa070009f40 tx=0x7fa070004a40 comp rx=0 tx=0).stop 2023-11-08T16:38:14.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.638+0000 7fa07d490640 1 -- 172.21.15.81:0/1058153748 shutdown_connections 2023-11-08T16:38:14.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.639+0000 7fa07d490640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 0x7fa05806d330 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.639+0000 7fa07d490640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa07815afd0 0x7fa07814c5f0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.639+0000 7fa07d490640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa078071890 0x7fa078153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.639+0000 7fa07d490640 1 --2- 172.21.15.81:0/1058153748 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa0780714b0 0x7fa0780c7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.639+0000 7fa07d490640 1 -- 172.21.15.81:0/1058153748 >> 172.21.15.81:0/1058153748 conn(0x7fa0780cb4e0 msgr2=0x7fa078156af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.639+0000 7fa07d490640 1 -- 172.21.15.81:0/1058153748 shutdown_connections 2023-11-08T16:38:14.681 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:38:14.639+0000 7f5ebc3c1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55e2d9b11b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:38:14.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.639+0000 7fa07d490640 1 -- 172.21.15.81:0/1058153748 wait complete. 2023-11-08T16:38:14.682 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:38:14.640+0000 7f5ebc3c1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e2d9b74680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(12)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[11]","memoryvalue":"(12)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:38:14.683 INFO:tasks.ceph.mds.a.smithi081.stderr:2023-11-08T16:38:14.644+0000 7f5ebc3c1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x55e2d9b04d80 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-11-08T16:38:14.369769+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={7446=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55e2d9b11600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-11-08T16:38:14.369769+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:38:14.684 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-08T16:38:14.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.898+0000 7f0e044da640 1 Processor -- start 2023-11-08T16:38:14.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.899+0000 7f0e044da640 1 -- start start 2023-11-08T16:38:14.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.900+0000 7f0e044da640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f0dfc160690 0x7f0dfc06bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.900+0000 7f0e044da640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dfc0d1ac0 con 0x7f0dfc0cb330 2023-11-08T16:38:14.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.900+0000 7f0e044da640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dfc0cf430 con 0x7f0dfc0d05e0 2023-11-08T16:38:14.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.900+0000 7f0e044da640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dfc0d0a70 con 0x7f0dfc160690 2023-11-08T16:38:14.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.900+0000 7f0e01a75640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f0dfc0cb330 0x7f0dfc0cfed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49204/0 (socket says 172.21.15.81:49204) 2023-11-08T16:38:14.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.900+0000 7f0e01a75640 1 -- 172.21.15.81:0/1723883097 learned_addr learned my addr 172.21.15.81:0/1723883097 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:14.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723159081 0 0) 0x7f0dfc0d1ac0 con 0x7f0dfc0cb330 2023-11-08T16:38:14.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e02276640 1 --2- 172.21.15.81:0/1723883097 >> v2:172.21.15.93:3300/0 conn(0x7f0dfc160690 0x7f0dfc06bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0de4003520 con 0x7f0dfc0cb330 2023-11-08T16:38:14.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 10707213 0 0) 0x7f0de4003520 con 0x7f0dfc0cb330 2023-11-08T16:38:14.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 >> v2:172.21.15.93:3300/0 conn(0x7f0dfc160690 msgr2=0x7f0dfc06bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 --2- 172.21.15.81:0/1723883097 >> v2:172.21.15.93:3300/0 conn(0x7f0dfc160690 0x7f0dfc06bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 >> v1:172.21.15.93:6789/0 conn(0x7f0dfc0d05e0 legacy=0x7f0dfc0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc06c4e0 con 0x7f0dfc0cb330 2023-11-08T16:38:14.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f0df0001670 con 0x7f0dfc0cb330 2023-11-08T16:38:14.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 >> v1:172.21.15.81:6789/0 conn(0x7f0dfc0cb330 legacy=0x7f0dfc0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 --2- 172.21.15.81:0/1723883097 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0de40039e0 0x7f0de4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 --2- 172.21.15.81:0/1723883097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0de40047c0 0x7f0de4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 --2- 172.21.15.81:0/1723883097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0de40051f0 0x7f0de400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0d0a70 con 0x7f0de40047c0 2023-11-08T16:38:14.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de400c150 con 0x7f0de40051f0 2023-11-08T16:38:14.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.901+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de400c310 con 0x7f0de40039e0 2023-11-08T16:38:14.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.902+0000 7f0e01a75640 1 --2- 172.21.15.81:0/1723883097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0de40047c0 0x7f0de4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.902+0000 7f0e02276640 1 --2- 172.21.15.81:0/1723883097 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0de40039e0 0x7f0de4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.902+0000 7f0e02a77640 1 --2- 172.21.15.81:0/1723883097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0de40051f0 0x7f0de400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.902+0000 7f0e01a75640 1 -- 172.21.15.81:0/1723883097 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0de40039e0 msgr2=0x7f0de4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.902+0000 7f0e01a75640 1 --2- 172.21.15.81:0/1723883097 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0de40039e0 0x7f0de4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.902+0000 7f0e01a75640 1 -- 172.21.15.81:0/1723883097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0de40051f0 msgr2=0x7f0de400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.902+0000 7f0e01a75640 1 --2- 172.21.15.81:0/1723883097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0de40051f0 0x7f0de400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.902+0000 7f0e01a75640 1 -- 172.21.15.81:0/1723883097 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de4003520 con 0x7f0de40047c0 2023-11-08T16:38:14.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.902+0000 7f0e02a77640 1 -- 172.21.15.81:0/1723883097 reap_dead start 2023-11-08T16:38:14.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.903+0000 7f0e01a75640 1 --2- 172.21.15.81:0/1723883097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0de40047c0 0x7f0de4004c40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0df000c240 tx=0x7f0de40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa562d347612dc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.903+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df0005ad0 con 0x7f0de40047c0 2023-11-08T16:38:14.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.903+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df0005c30 con 0x7f0de40047c0 2023-11-08T16:38:14.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.903+0000 7f0e01274640 1 -- 172.21.15.81:0/1723883097 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df002caf0 con 0x7f0de40047c0 2023-11-08T16:38:14.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.904+0000 7f0e044da640 1 -- 172.21.15.81:0/1723883097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0de40047c0 msgr2=0x7f0de4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.904+0000 7f0e044da640 1 --2- 172.21.15.81:0/1723883097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0de40047c0 0x7f0de4004c40 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0df000c240 tx=0x7f0de40057f0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.904+0000 7f0e044da640 1 -- 172.21.15.81:0/1723883097 shutdown_connections 2023-11-08T16:38:14.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.904+0000 7f0e044da640 1 --2- 172.21.15.81:0/1723883097 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0de40047c0 0x7f0de4004c40 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.904+0000 7f0e044da640 1 -- 172.21.15.81:0/1723883097 >> 172.21.15.81:0/1723883097 conn(0x7f0dfc0d0ce0 msgr2=0x7f0dfc0d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.904+0000 7f0e044da640 1 -- 172.21.15.81:0/1723883097 shutdown_connections 2023-11-08T16:38:14.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.904+0000 7f0e044da640 1 -- 172.21.15.81:0/1723883097 wait complete. 2023-11-08T16:38:14.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.906+0000 7f0e044da640 1 Processor -- start 2023-11-08T16:38:14.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.906+0000 7f0e044da640 1 -- start start 2023-11-08T16:38:14.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.907+0000 7f0e044da640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0de40047c0 0x7f0dfc0b6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.907+0000 7f0e044da640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0dfc0c4be0 0x7f0dfc0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.907+0000 7f0e044da640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dfc0b1650 0x7f0dfc0b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.907+0000 7f0e044da640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0c55a0 con 0x7f0dfc0b1650 2023-11-08T16:38:14.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.907+0000 7f0e044da640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0b2040 con 0x7f0de40047c0 2023-11-08T16:38:14.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.907+0000 7f0e044da640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0b6a40 con 0x7f0dfc0c4be0 2023-11-08T16:38:14.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.907+0000 7f0e02a77640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dfc0b1650 0x7f0dfc0b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.907+0000 7f0e02a77640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dfc0b1650 0x7f0dfc0b1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60132/0 (socket says 172.21.15.81:60132) 2023-11-08T16:38:14.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.907+0000 7f0e02a77640 1 -- 172.21.15.81:0/910632010 learned_addr learned my addr 172.21.15.81:0/910632010 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:14.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.908+0000 7f0e02276640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0de40047c0 0x7f0dfc0b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.908+0000 7f0e01a75640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0dfc0c4be0 0x7f0dfc0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.908+0000 7f0e02a77640 1 -- 172.21.15.81:0/910632010 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0dfc0c4be0 msgr2=0x7f0dfc0c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.908+0000 7f0e02a77640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0dfc0c4be0 0x7f0dfc0c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.908+0000 7f0e02a77640 1 -- 172.21.15.81:0/910632010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0de40047c0 msgr2=0x7f0dfc0b6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.908+0000 7f0e02a77640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0de40047c0 0x7f0dfc0b6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.908+0000 7f0e02a77640 1 -- 172.21.15.81:0/910632010 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0b22d0 con 0x7f0dfc0b1650 2023-11-08T16:38:14.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.908+0000 7f0e02a77640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dfc0b1650 0x7f0dfc0b1b00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc067580 tx=0x7f0df400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.908+0000 7f0dfaffd640 1 -- 172.21.15.81:0/910632010 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df400b070 con 0x7f0dfc0b1650 2023-11-08T16:38:14.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.908+0000 7f0dfaffd640 1 -- 172.21.15.81:0/910632010 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df40035f0 con 0x7f0dfc0b1650 2023-11-08T16:38:14.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.908+0000 7f0dfaffd640 1 -- 172.21.15.81:0/910632010 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df40039a0 con 0x7f0dfc0b1650 2023-11-08T16:38:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.909+0000 7f0e044da640 1 -- 172.21.15.81:0/910632010 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dfc0b25b0 con 0x7f0dfc0b1650 2023-11-08T16:38:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.909+0000 7f0e044da640 1 -- 172.21.15.81:0/910632010 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc0b2b70 con 0x7f0dfc0b1650 2023-11-08T16:38:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.911+0000 7f0dfaffd640 1 -- 172.21.15.81:0/910632010 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0df4003b00 con 0x7f0dfc0b1650 2023-11-08T16:38:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.912+0000 7f0dfaffd640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0de806ae70 0x7f0de806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.912+0000 7f0dfaffd640 1 -- 172.21.15.81:0/910632010 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f0df4088f20 con 0x7f0dfc0b1650 2023-11-08T16:38:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.912+0000 7f0e02276640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0de806ae70 0x7f0de806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.913+0000 7f0e0706c740 1 Processor -- start 2023-11-08T16:38:14.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.913+0000 7f0e02276640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0de806ae70 0x7f0de806d330 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc063cf0 tx=0x7f0dec00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.913+0000 7f0e0706c740 1 -- start start 2023-11-08T16:38:14.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.914+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa270 0x55a4f74cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.914+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa650 0x55a4f74fd000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.914+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f74cd710 0x55a4f74fd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.914+0000 7f0e0706c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f743fb60 con 0x55a4f74fa650 2023-11-08T16:38:14.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.914+0000 7f0e0706c740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f743d250 con 0x55a4f74fa270 2023-11-08T16:38:14.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.914+0000 7f0e0706c740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4f7438b40 con 0x55a4f74cd710 2023-11-08T16:38:14.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.914+0000 7f0e01a75640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa650 0x55a4f74fd000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.914+0000 7f0e01a75640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa650 0x55a4f74fd000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60146/0 (socket says 172.21.15.81:60146) 2023-11-08T16:38:14.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.914+0000 7f0e01a75640 1 -- 172.21.15.81:0/3873142226 learned_addr learned my addr 172.21.15.81:0/3873142226 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:14.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.914+0000 7f0e02276640 1 --2- 172.21.15.81:0/3873142226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa270 0x55a4f74cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.915+0000 7f0e01a75640 1 -- 172.21.15.81:0/3873142226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f74cd710 msgr2=0x55a4f74fd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.914+0000 7f0e02a77640 1 --2- 172.21.15.81:0/3873142226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f74cd710 0x55a4f74fd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.915+0000 7f0e01a75640 1 --2- 172.21.15.81:0/3873142226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f74cd710 0x55a4f74fd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.915+0000 7f0e01a75640 1 -- 172.21.15.81:0/3873142226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa270 msgr2=0x55a4f74cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.915+0000 7f0e01a75640 1 --2- 172.21.15.81:0/3873142226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa270 0x55a4f74cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.915+0000 7f0e01a75640 1 -- 172.21.15.81:0/3873142226 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4f74371c0 con 0x55a4f74fa650 2023-11-08T16:38:14.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.915+0000 7f0e02276640 1 --2- 172.21.15.81:0/3873142226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa270 0x55a4f74cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:14.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.915+0000 7f0e01a75640 1 --2- 172.21.15.81:0/3873142226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa650 0x55a4f74fd000 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0df002c450 tx=0x7f0df0007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57439d1352edc23f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.915+0000 7f0de3fff640 1 -- 172.21.15.81:0/3873142226 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df003c070 con 0x55a4f74fa650 2023-11-08T16:38:14.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.915+0000 7f0de3fff640 1 -- 172.21.15.81:0/3873142226 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df003d040 con 0x55a4f74fa650 2023-11-08T16:38:14.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.915+0000 7f0de3fff640 1 -- 172.21.15.81:0/3873142226 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df002cad0 con 0x55a4f74fa650 2023-11-08T16:38:14.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.916+0000 7f0e0706c740 1 -- 172.21.15.81:0/3873142226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa650 msgr2=0x55a4f74fd000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.916+0000 7f0e0706c740 1 --2- 172.21.15.81:0/3873142226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa650 0x55a4f74fd000 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f0df002c450 tx=0x7f0df0007000 comp rx=0 tx=0).stop 2023-11-08T16:38:14.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.916+0000 7f0e0706c740 1 -- 172.21.15.81:0/3873142226 shutdown_connections 2023-11-08T16:38:14.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.916+0000 7f0e0706c740 1 --2- 172.21.15.81:0/3873142226 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f74cd710 0x55a4f74fd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.916+0000 7f0e0706c740 1 --2- 172.21.15.81:0/3873142226 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa650 0x55a4f74fd000 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.916+0000 7f0e0706c740 1 --2- 172.21.15.81:0/3873142226 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa270 0x55a4f74cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.916+0000 7f0e0706c740 1 -- 172.21.15.81:0/3873142226 >> 172.21.15.81:0/3873142226 conn(0x55a4f74cf480 msgr2=0x55a4f7518000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.916+0000 7f0e0706c740 1 -- 172.21.15.81:0/3873142226 shutdown_connections 2023-11-08T16:38:14.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.916+0000 7f0e0706c740 1 -- 172.21.15.81:0/3873142226 wait complete. 2023-11-08T16:38:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.917+0000 7f0e0706c740 1 -- start start 2023-11-08T16:38:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.917+0000 7f0e0706c740 1 Processor -- start 2023-11-08T16:38:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.918+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f7510c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.918+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 0x55a4f751b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.918+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f7511790 0x55a4f7527f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.918+0000 7f0e0706c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f73d5dd0 con 0x55a4f74fa270 2023-11-08T16:38:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.918+0000 7f0e0706c740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f73d5a70 con 0x55a4f74fa650 2023-11-08T16:38:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.918+0000 7f0e0706c740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4f7131f90 con 0x55a4f7511790 2023-11-08T16:38:14.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.918+0000 7f0e0706c740 10 client.0 _ll_register_callbacks cb 0x55a4f73f7e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:38:14.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.918+0000 7f0e02276640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f7510c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0e02276640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f7510c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60162/0 (socket says 172.21.15.81:60162) 2023-11-08T16:38:14.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0e02276640 1 -- 172.21.15.81:0/632822522 learned_addr learned my addr 172.21.15.81:0/632822522 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:14.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0e01a75640 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 0x55a4f751b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0e02276640 1 -- 172.21.15.81:0/632822522 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f7511790 msgr2=0x55a4f7527f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0e02a77640 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f7511790 0x55a4f7527f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0e02276640 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f7511790 0x55a4f7527f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0e02276640 1 -- 172.21.15.81:0/632822522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 msgr2=0x55a4f751b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0e02276640 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 0x55a4f751b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0e02276640 1 -- 172.21.15.81:0/632822522 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de4003520 con 0x55a4f74fa270 2023-11-08T16:38:14.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0e01a75640 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 0x55a4f751b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:14.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0e02276640 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f7510c70 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0dec016b40 tx=0x7f0dec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a98d91d07494860 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0de17fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:38:14.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.919+0000 7f0de17fa640 1 -- 172.21.15.81:0/632822522 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec017c50 con 0x55a4f74fa270 2023-11-08T16:38:14.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0de17fa640 1 -- 172.21.15.81:0/632822522 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f0dec017db0 con 0x55a4f74fa270 2023-11-08T16:38:14.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0e0706c740 1 -- 172.21.15.81:0/632822522 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a4f745f400 con 0x55a4f74fa270 2023-11-08T16:38:14.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0de17fa640 1 -- 172.21.15.81:0/632822522 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0dec023410 con 0x55a4f74fa270 2023-11-08T16:38:14.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0e0706c740 10 client.7476 fetch_fsmap learned FSMap version 78 2023-11-08T16:38:14.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0e0706c740 1 -- 172.21.15.81:0/632822522 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55a4f73d4d20 con 0x55a4f74fa270 2023-11-08T16:38:14.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0de17fa640 1 -- 172.21.15.81:0/632822522 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a4f745f400 con 0x55a4f74fa270 2023-11-08T16:38:14.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0de17fa640 1 -- 172.21.15.81:0/632822522 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 78) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f0dec014070 con 0x55a4f74fa270 2023-11-08T16:38:14.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0e0706c740 10 client.7476 fetch_fsmap finished waiting for FSMap version 78 2023-11-08T16:38:14.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0e0706c740 10 client.7476 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-08T16:38:14.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0e0706c740 20 client.7476 populate_metadata read hostname 'smithi081' 2023-11-08T16:38:14.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0e0706c740 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55a4f74ff510 0x55a4f74ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0e0706c740 4 client.7476 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:38:14.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.920+0000 7f0e0706c740 1 -- 172.21.15.81:0/632822522 --> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4f7431b20 con 0x55a4f74ff510 2023-11-08T16:38:14.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.921+0000 7f0e01a75640 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55a4f74ff510 0x55a4f74ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.921+0000 7f0de17fa640 10 client.7476 ms_handle_connect on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:14.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.921+0000 7f0e01a75640 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55a4f74ff510 0x55a4f74ff8f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c22907851eb89318 server_cookie=8f50117c661a2209 in_seq=0 out_seq=0 2023-11-08T16:38:14.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0de17fa640 1 -- 172.21.15.81:0/632822522 <== mds.0 v2:172.21.15.81:6836/1843050442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a4f7431b20 con 0x55a4f74ff510 2023-11-08T16:38:14.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0de17fa640 10 client.7476 handle_command_reply: tid=0 2023-11-08T16:38:14.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0e0706c740 1 client.7476 shutdown 2023-11-08T16:38:14.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0e0706c740 1 -- 172.21.15.81:0/632822522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 msgr2=0x55a4f7510c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0e0706c740 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f7510c70 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f0dec016b40 tx=0x7f0dec014000 comp rx=0 tx=0).stop 2023-11-08T16:38:14.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0e0706c740 1 -- 172.21.15.81:0/632822522 shutdown_connections 2023-11-08T16:38:14.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0e0706c740 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55a4f74ff510 0x55a4f74ff8f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0e0706c740 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f7511790 0x55a4f7527f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0e0706c740 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 0x55a4f751b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0de17fa640 0 client.7476 ms_handle_reset on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0e0706c740 1 --2- 172.21.15.81:0/632822522 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f7510c70 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0e0706c740 1 -- 172.21.15.81:0/632822522 >> 172.21.15.81:0/632822522 conn(0x55a4f74cd710 msgr2=0x55a4f74d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0e0706c740 1 -- 172.21.15.81:0/632822522 shutdown_connections 2023-11-08T16:38:14.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.922+0000 7f0e0706c740 1 -- 172.21.15.81:0/632822522 wait complete. 2023-11-08T16:38:14.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.923+0000 7f0e0706c740 20 client.7476 trim_cache size 0 max 16384 2023-11-08T16:38:14.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.939+0000 7f0e0706c740 1 Processor -- start 2023-11-08T16:38:14.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.939+0000 7f0e0706c740 1 -- start start 2023-11-08T16:38:14.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f74ff8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 0x55a4f74c8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f7511790 0x55a4f750f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e0706c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f751b950 con 0x55a4f74fa270 2023-11-08T16:38:14.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e0706c740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f74c8ed0 con 0x55a4f74fa650 2023-11-08T16:38:14.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e0706c740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4f75113a0 con 0x55a4f7511790 2023-11-08T16:38:14.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e02276640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f74ff8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e02276640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f74ff8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60170/0 (socket says 172.21.15.81:60170) 2023-11-08T16:38:14.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e02276640 1 -- 172.21.15.81:0/1729184574 learned_addr learned my addr 172.21.15.81:0/1729184574 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:14.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e01a75640 1 --2- 172.21.15.81:0/1729184574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 0x55a4f74c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e02276640 1 -- 172.21.15.81:0/1729184574 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f7511790 msgr2=0x55a4f750f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e02276640 1 --2- 172.21.15.81:0/1729184574 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f7511790 0x55a4f750f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e02276640 1 -- 172.21.15.81:0/1729184574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 msgr2=0x55a4f74c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e02276640 1 --2- 172.21.15.81:0/1729184574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 0x55a4f74c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.940+0000 7f0e02276640 1 -- 172.21.15.81:0/1729184574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4f7468750 con 0x55a4f74fa270 2023-11-08T16:38:14.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.941+0000 7f0e01a75640 1 --2- 172.21.15.81:0/1729184574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 0x55a4f74c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:14.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.941+0000 7f0e02276640 1 --2- 172.21.15.81:0/1729184574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f74ff8f0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0045b0 tx=0x7f0dec018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1854e2e6f8a62662 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.941+0000 7f0df8ff9640 1 -- 172.21.15.81:0/1729184574 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec017c50 con 0x55a4f74fa270 2023-11-08T16:38:14.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.941+0000 7f0df8ff9640 1 -- 172.21.15.81:0/1729184574 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec01e050 con 0x55a4f74fa270 2023-11-08T16:38:14.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.941+0000 7f0df8ff9640 1 -- 172.21.15.81:0/1729184574 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec02b640 con 0x55a4f74fa270 2023-11-08T16:38:14.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.942+0000 7f0e0706c740 1 -- 172.21.15.81:0/1729184574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 msgr2=0x55a4f74ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.942+0000 7f0e0706c740 1 --2- 172.21.15.81:0/1729184574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f74ff8f0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f0dec0045b0 tx=0x7f0dec018000 comp rx=0 tx=0).stop 2023-11-08T16:38:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.942+0000 7f0e0706c740 1 -- 172.21.15.81:0/1729184574 shutdown_connections 2023-11-08T16:38:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.942+0000 7f0e0706c740 1 --2- 172.21.15.81:0/1729184574 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f7511790 0x55a4f750f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.942+0000 7f0e0706c740 1 --2- 172.21.15.81:0/1729184574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa650 0x55a4f74c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.942+0000 7f0e0706c740 1 --2- 172.21.15.81:0/1729184574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f74fa270 0x55a4f74ff8f0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.942+0000 7f0e0706c740 1 -- 172.21.15.81:0/1729184574 >> 172.21.15.81:0/1729184574 conn(0x55a4f74cd710 msgr2=0x55a4f7500100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.942+0000 7f0e0706c740 1 -- 172.21.15.81:0/1729184574 shutdown_connections 2023-11-08T16:38:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.942+0000 7f0e0706c740 1 -- 172.21.15.81:0/1729184574 wait complete. 2023-11-08T16:38:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.942+0000 7f0e0706c740 1 -- start start 2023-11-08T16:38:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.943+0000 7f0e0706c740 1 Processor -- start 2023-11-08T16:38:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa270 0x55a4f7518030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f74fa650 0x55a4f750d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e0706c740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f7511790 0x55a4f750ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e0706c740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f7548d20 con 0x55a4f7511790 2023-11-08T16:38:14.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e0706c740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4f758cc80 con 0x55a4f74fa270 2023-11-08T16:38:14.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e0706c740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4f758b7f0 con 0x55a4f74fa650 2023-11-08T16:38:14.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e0706c740 10 client.0 _ll_register_callbacks cb 0x7f0dec009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:38:14.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e02a77640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f7511790 0x55a4f750ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e02a77640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f7511790 0x55a4f750ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60180/0 (socket says 172.21.15.81:60180) 2023-11-08T16:38:14.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e02a77640 1 -- 172.21.15.81:0/1418929714 learned_addr learned my addr 172.21.15.81:0/1418929714 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:14.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e01a75640 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f74fa650 0x55a4f750d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e02276640 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa270 0x55a4f7518030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e02a77640 1 -- 172.21.15.81:0/1418929714 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f74fa650 msgr2=0x55a4f750d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e02a77640 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f74fa650 0x55a4f750d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e02a77640 1 -- 172.21.15.81:0/1418929714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa270 msgr2=0x55a4f7518030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:14.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e02a77640 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa270 0x55a4f7518030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.944+0000 7f0e02a77640 1 -- 172.21.15.81:0/1418929714 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4f73d4d20 con 0x55a4f7511790 2023-11-08T16:38:14.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0e02a77640 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f7511790 0x55a4f750ded0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0df405f780 tx=0x7f0df4057400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d46a372dc0f3e7ac server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:14.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0de17fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:38:14.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0de17fa640 1 -- 172.21.15.81:0/1418929714 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df4083020 con 0x55a4f7511790 2023-11-08T16:38:14.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0de17fa640 1 -- 172.21.15.81:0/1418929714 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f0df405bb00 con 0x55a4f7511790 2023-11-08T16:38:14.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0e0706c740 1 -- 172.21.15.81:0/1418929714 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0df003d040 con 0x55a4f7511790 2023-11-08T16:38:14.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0de17fa640 1 -- 172.21.15.81:0/1418929714 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0df405fcb0 con 0x55a4f7511790 2023-11-08T16:38:14.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0e0706c740 10 client.7482 fetch_fsmap learned FSMap version 78 2023-11-08T16:38:14.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0e0706c740 1 -- 172.21.15.81:0/1418929714 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55a4f73fa310 con 0x55a4f7511790 2023-11-08T16:38:14.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0de17fa640 1 -- 172.21.15.81:0/1418929714 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df003d040 con 0x55a4f7511790 2023-11-08T16:38:14.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0de17fa640 1 -- 172.21.15.81:0/1418929714 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 78) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f0df406f180 con 0x55a4f7511790 2023-11-08T16:38:14.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0e0706c740 10 client.7482 fetch_fsmap finished waiting for FSMap version 78 2023-11-08T16:38:14.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0e0706c740 10 client.7482 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-08T16:38:14.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0e0706c740 20 client.7482 populate_metadata read hostname 'smithi081' 2023-11-08T16:38:14.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0e0706c740 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55a4f75856f0 0x55a4f7531980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:14.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0e0706c740 4 client.7482 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-08T16:38:14.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.945+0000 7f0e0706c740 1 -- 172.21.15.81:0/1418929714 --> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a4f7431840 con 0x55a4f75856f0 2023-11-08T16:38:14.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.946+0000 7f0e02276640 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55a4f75856f0 0x55a4f7531980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:14.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.946+0000 7f0e02276640 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55a4f75856f0 0x55a4f7531980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e46f49f4fb117c9 server_cookie=356a889ef4b5950d in_seq=0 out_seq=0 2023-11-08T16:38:14.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.946+0000 7f0de17fa640 10 client.7482 ms_handle_connect on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:14.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.946+0000 7f0de17fa640 1 -- 172.21.15.81:0/1418929714 <== mds.0 v2:172.21.15.81:6836/1843050442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55a4f7431840 con 0x55a4f75856f0 2023-11-08T16:38:14.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.946+0000 7f0de17fa640 10 client.7482 handle_command_reply: tid=0 2023-11-08T16:38:14.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.946+0000 7f0e0706c740 1 client.7482 shutdown 2023-11-08T16:38:14.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.946+0000 7f0e0706c740 1 -- 172.21.15.81:0/1418929714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f7511790 msgr2=0x55a4f750ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.946+0000 7f0e0706c740 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f7511790 0x55a4f750ded0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f0df405f780 tx=0x7f0df4057400 comp rx=0 tx=0).stop 2023-11-08T16:38:14.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e0706c740 1 -- 172.21.15.81:0/1418929714 shutdown_connections 2023-11-08T16:38:14.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e0706c740 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55a4f75856f0 0x55a4f7531980 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e0706c740 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55a4f7511790 0x55a4f750ded0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e0706c740 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55a4f74fa650 0x55a4f750d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e0706c740 1 --2- 172.21.15.81:0/1418929714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a4f74fa270 0x55a4f7518030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0de17fa640 0 client.7482 ms_handle_reset on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:14.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e0706c740 1 -- 172.21.15.81:0/1418929714 >> 172.21.15.81:0/1418929714 conn(0x55a4f74cd710 msgr2=0x55a4f74fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e0706c740 1 -- 172.21.15.81:0/1418929714 shutdown_connections 2023-11-08T16:38:14.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e0706c740 1 -- 172.21.15.81:0/1418929714 wait complete. 2023-11-08T16:38:14.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e0706c740 20 client.7482 trim_cache size 0 max 16384 2023-11-08T16:38:14.972 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:38:14.972 INFO:teuthology.orchestra.run.smithi081.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-08T16:38:14.972 INFO:teuthology.orchestra.run.smithi081.stdout: "scrubs": { 2023-11-08T16:38:14.973 INFO:teuthology.orchestra.run.smithi081.stdout: "490117e8-9ec4-465d-acd8-42fcc0a84e82": { 2023-11-08T16:38:14.973 INFO:teuthology.orchestra.run.smithi081.stdout: "path": "/", 2023-11-08T16:38:14.973 INFO:teuthology.orchestra.run.smithi081.stdout: "tag": "490117e8-9ec4-465d-acd8-42fcc0a84e82", 2023-11-08T16:38:14.973 INFO:teuthology.orchestra.run.smithi081.stdout: "options": "recursive,repair" 2023-11-08T16:38:14.974 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:38:14.974 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:38:14.974 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:38:14.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e044da640 1 -- 172.21.15.81:0/910632010 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0de806ae70 msgr2=0x7f0de806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e044da640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0de806ae70 0x7f0de806d330 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc063cf0 tx=0x7f0dec00b040 comp rx=0 tx=0).stop 2023-11-08T16:38:14.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e044da640 1 -- 172.21.15.81:0/910632010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dfc0b1650 msgr2=0x7f0dfc0b1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:14.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.947+0000 7f0e044da640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dfc0b1650 0x7f0dfc0b1b00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc067580 tx=0x7f0df400d890 comp rx=0 tx=0).stop 2023-11-08T16:38:14.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.948+0000 7f0e044da640 1 -- 172.21.15.81:0/910632010 shutdown_connections 2023-11-08T16:38:14.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.948+0000 7f0e044da640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0de806ae70 0x7f0de806d330 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.948+0000 7f0e044da640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0dfc0b1650 0x7f0dfc0b1b00 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.948+0000 7f0e044da640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0dfc0c4be0 0x7f0dfc0c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.948+0000 7f0e044da640 1 --2- 172.21.15.81:0/910632010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0de40047c0 0x7f0dfc0b6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:14.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.948+0000 7f0e044da640 1 -- 172.21.15.81:0/910632010 >> 172.21.15.81:0/910632010 conn(0x7f0dfc0d0ce0 msgr2=0x7f0dfc0ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:14.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.948+0000 7f0e044da640 1 -- 172.21.15.81:0/910632010 shutdown_connections 2023-11-08T16:38:14.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:14.948+0000 7f0e044da640 1 -- 172.21.15.81:0/910632010 wait complete. 2023-11-08T16:38:14.979 INFO:tasks.cephfs.filesystem:scrub status for tag:490117e8-9ec4-465d-acd8-42fcc0a84e82 - {'path': '/', 'tag': '490117e8-9ec4-465d-acd8-42fcc0a84e82', 'options': 'recursive,repair'} 2023-11-08T16:38:18.553 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:38:18.563 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:38:44.980 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-08T16:38:45.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.201+0000 7fa731bee640 1 Processor -- start 2023-11-08T16:38:45.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.201+0000 7fa731bee640 1 -- start start 2023-11-08T16:38:45.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.203+0000 7fa731bee640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa72c071890 0x7fa72c15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.203+0000 7fa731bee640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa72c15afd0 0x7fa72c15ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.203+0000 7fa731bee640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c0b09f0 con 0x7fa72c0714b0 2023-11-08T16:38:45.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.203+0000 7fa731bee640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72c06b250 con 0x7fa72c15afd0 2023-11-08T16:38:45.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.203+0000 7fa731bee640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72c0ca360 con 0x7fa72c071890 2023-11-08T16:38:45.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.203+0000 7fa72bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa72c15afd0 0x7fa72c15ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.203+0000 7fa72bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa72c15afd0 0x7fa72c15ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60302/0 (socket says 172.21.15.81:60302) 2023-11-08T16:38:45.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.203+0000 7fa72b7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa72c0714b0 0x7fa72c15a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46120/0 (socket says 172.21.15.81:46120) 2023-11-08T16:38:45.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.203+0000 7fa72bfff640 1 -- 172.21.15.81:0/447470035 learned_addr learned my addr 172.21.15.81:0/447470035 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:45.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.204+0000 7fa722ffd640 1 --2- 172.21.15.81:0/447470035 >> v2:172.21.15.93:3300/0 conn(0x7fa72c071890 0x7fa72c15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.204+0000 7fa72bfff640 1 -- 172.21.15.81:0/447470035 >> v2:172.21.15.93:3300/0 conn(0x7fa72c071890 msgr2=0x7fa72c15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:45.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.204+0000 7fa72bfff640 1 --2- 172.21.15.81:0/447470035 >> v2:172.21.15.93:3300/0 conn(0x7fa72c071890 0x7fa72c15e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.204+0000 7fa72bfff640 1 -- 172.21.15.81:0/447470035 >> v1:172.21.15.81:6789/0 conn(0x7fa72c0714b0 legacy=0x7fa72c15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:45.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.204+0000 7fa72affd640 1 -- 172.21.15.81:0/447470035 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3339442292 0 0) 0x7fa72c0b09f0 con 0x7fa72c0714b0 2023-11-08T16:38:45.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.204+0000 7fa72bfff640 1 -- 172.21.15.81:0/447470035 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c0ca640 con 0x7fa72c15afd0 2023-11-08T16:38:45.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.204+0000 7fa722ffd640 1 --2- 172.21.15.81:0/447470035 >> v2:172.21.15.93:3300/0 conn(0x7fa72c071890 0x7fa72c15e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:45.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.205+0000 7fa72bfff640 1 --2- 172.21.15.81:0/447470035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa72c15afd0 0x7fa72c15ef10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa72c15e930 tx=0x7fa72402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec53a7b11f0dde3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:45.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.205+0000 7fa72affd640 1 -- 172.21.15.81:0/447470035 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724030860 con 0x7fa72c15afd0 2023-11-08T16:38:45.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.205+0000 7fa72affd640 1 -- 172.21.15.81:0/447470035 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7240309c0 con 0x7fa72c15afd0 2023-11-08T16:38:45.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.205+0000 7fa72affd640 1 -- 172.21.15.81:0/447470035 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724030d50 con 0x7fa72c15afd0 2023-11-08T16:38:45.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.206+0000 7fa731bee640 1 -- 172.21.15.81:0/447470035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa72c15afd0 msgr2=0x7fa72c15ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:45.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.206+0000 7fa731bee640 1 --2- 172.21.15.81:0/447470035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa72c15afd0 0x7fa72c15ef10 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fa72c15e930 tx=0x7fa72402b9a0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.206+0000 7fa731bee640 1 -- 172.21.15.81:0/447470035 shutdown_connections 2023-11-08T16:38:45.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.206+0000 7fa731bee640 1 --2- 172.21.15.81:0/447470035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa72c15afd0 0x7fa72c15ef10 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.206+0000 7fa731bee640 1 --2- 172.21.15.81:0/447470035 >> v2:172.21.15.93:3300/0 conn(0x7fa72c071890 0x7fa72c15e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.206+0000 7fa731bee640 1 -- 172.21.15.81:0/447470035 >> 172.21.15.81:0/447470035 conn(0x7fa72c0cb4e0 msgr2=0x7fa72c0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:45.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.206+0000 7fa731bee640 1 -- 172.21.15.81:0/447470035 shutdown_connections 2023-11-08T16:38:45.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.206+0000 7fa731bee640 1 -- 172.21.15.81:0/447470035 wait complete. 2023-11-08T16:38:45.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.207+0000 7fa731bee640 1 Processor -- start 2023-11-08T16:38:45.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.208+0000 7fa731bee640 1 -- start start 2023-11-08T16:38:45.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.208+0000 7fa731bee640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa72c0714b0 0x7fa72c14afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.208+0000 7fa731bee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa72c071890 0x7fa72c0b3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.208+0000 7fa731bee640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa72c15afd0 0x7fa72c0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.208+0000 7fa731bee640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0b3920 con 0x7fa72c071890 2023-11-08T16:38:45.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.208+0000 7fa731bee640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0b3c00 con 0x7fa72c15afd0 2023-11-08T16:38:45.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.208+0000 7fa731bee640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0b3ee0 con 0x7fa72c0714b0 2023-11-08T16:38:45.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.209+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa72c071890 0x7fa72c0b3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.209+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa72c071890 0x7fa72c0b3240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60308/0 (socket says 172.21.15.81:60308) 2023-11-08T16:38:45.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.209+0000 7fa722ffd640 1 -- 172.21.15.81:0/934511586 learned_addr learned my addr 172.21.15.81:0/934511586 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:45.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.209+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa72c0714b0 0x7fa72c14afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.209+0000 7fa722ffd640 1 -- 172.21.15.81:0/934511586 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa72c0714b0 msgr2=0x7fa72c14afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:45.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.209+0000 7fa722ffd640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa72c0714b0 0x7fa72c14afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.209+0000 7fa722ffd640 1 -- 172.21.15.81:0/934511586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa72c15afd0 msgr2=0x7fa72c0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:45.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.209+0000 7fa722ffd640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa72c15afd0 0x7fa72c0b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.209+0000 7fa722ffd640 1 -- 172.21.15.81:0/934511586 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7240094d0 con 0x7fa72c071890 2023-11-08T16:38:45.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.209+0000 7fa722ffd640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa72c071890 0x7fa72c0b3240 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c7f0 tx=0x7fa71400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:45.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.210+0000 7fa728ff9640 1 -- 172.21.15.81:0/934511586 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa714007a30 con 0x7fa72c071890 2023-11-08T16:38:45.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.210+0000 7fa728ff9640 1 -- 172.21.15.81:0/934511586 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa714007b90 con 0x7fa72c071890 2023-11-08T16:38:45.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.210+0000 7fa728ff9640 1 -- 172.21.15.81:0/934511586 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71400ce40 con 0x7fa72c071890 2023-11-08T16:38:45.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.210+0000 7fa731bee640 1 -- 172.21.15.81:0/934511586 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c0b41c0 con 0x7fa72c071890 2023-11-08T16:38:45.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.210+0000 7fa731bee640 1 -- 172.21.15.81:0/934511586 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c0b1cf0 con 0x7fa72c071890 2023-11-08T16:38:45.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.212+0000 7fa728ff9640 1 -- 172.21.15.81:0/934511586 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7140113f0 con 0x7fa72c071890 2023-11-08T16:38:45.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.213+0000 7fa728ff9640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa70c06ae70 0x7fa70c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.213+0000 7fa728ff9640 1 -- 172.21.15.81:0/934511586 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fa714089b70 con 0x7fa72c071890 2023-11-08T16:38:45.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.213+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa70c06ae70 0x7fa70c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.214+0000 7fa73477e740 1 Processor -- start 2023-11-08T16:38:45.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.214+0000 7fa73477e740 1 -- start start 2023-11-08T16:38:45.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.214+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa70c06ae70 0x7fa70c06d330 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0012e0 tx=0x7fa71c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:45.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.215+0000 7fa73477e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55270 0x55aac4f281d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.215+0000 7fa73477e740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f55650 0x55aac4f58000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.215+0000 7fa73477e740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f28710 0x55aac4f58540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.215+0000 7fa73477e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac4e9ab60 con 0x55aac4f55270 2023-11-08T16:38:45.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.215+0000 7fa73477e740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac4e98250 con 0x55aac4f28710 2023-11-08T16:38:45.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.215+0000 7fa73477e740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac4e93b40 con 0x55aac4f55650 2023-11-08T16:38:45.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.215+0000 7fa72b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55270 0x55aac4f281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.215+0000 7fa72b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55270 0x55aac4f281d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60324/0 (socket says 172.21.15.81:60324) 2023-11-08T16:38:45.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.215+0000 7fa72b7fe640 1 -- 172.21.15.81:0/1591378998 learned_addr learned my addr 172.21.15.81:0/1591378998 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:45.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.216+0000 7fa722ffd640 1 --2- 172.21.15.81:0/1591378998 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f55650 0x55aac4f58000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.216+0000 7fa72bfff640 1 --2- 172.21.15.81:0/1591378998 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f28710 0x55aac4f58540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.216+0000 7fa72b7fe640 1 -- 172.21.15.81:0/1591378998 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f55650 msgr2=0x55aac4f58000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:45.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.216+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/1591378998 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f55650 0x55aac4f58000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.216+0000 7fa72b7fe640 1 -- 172.21.15.81:0/1591378998 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f28710 msgr2=0x55aac4f58540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:45.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.216+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/1591378998 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f28710 0x55aac4f58540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.216+0000 7fa72b7fe640 1 -- 172.21.15.81:0/1591378998 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aac4e921c0 con 0x55aac4f55270 2023-11-08T16:38:45.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.216+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/1591378998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55270 0x55aac4f281d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa71c005f90 tx=0x7fa71c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed4e484ba7c39da server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:45.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.216+0000 7fa7217fa640 1 -- 172.21.15.81:0/1591378998 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c007850 con 0x55aac4f55270 2023-11-08T16:38:45.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.216+0000 7fa7217fa640 1 -- 172.21.15.81:0/1591378998 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c0079b0 con 0x55aac4f55270 2023-11-08T16:38:45.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.216+0000 7fa7217fa640 1 -- 172.21.15.81:0/1591378998 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c020650 con 0x55aac4f55270 2023-11-08T16:38:45.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.217+0000 7fa73477e740 1 -- 172.21.15.81:0/1591378998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55270 msgr2=0x55aac4f281d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:45.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.217+0000 7fa73477e740 1 --2- 172.21.15.81:0/1591378998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55270 0x55aac4f281d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fa71c005f90 tx=0x7fa71c010000 comp rx=0 tx=0).stop 2023-11-08T16:38:45.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.217+0000 7fa73477e740 1 -- 172.21.15.81:0/1591378998 shutdown_connections 2023-11-08T16:38:45.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.217+0000 7fa73477e740 1 --2- 172.21.15.81:0/1591378998 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f28710 0x55aac4f58540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.217+0000 7fa73477e740 1 --2- 172.21.15.81:0/1591378998 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f55650 0x55aac4f58000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.217+0000 7fa73477e740 1 --2- 172.21.15.81:0/1591378998 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55270 0x55aac4f281d0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.217+0000 7fa73477e740 1 -- 172.21.15.81:0/1591378998 >> 172.21.15.81:0/1591378998 conn(0x55aac4f2a480 msgr2=0x55aac4f73000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:45.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.217+0000 7fa73477e740 1 -- 172.21.15.81:0/1591378998 shutdown_connections 2023-11-08T16:38:45.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.217+0000 7fa73477e740 1 -- 172.21.15.81:0/1591378998 wait complete. 2023-11-08T16:38:45.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.218+0000 7fa73477e740 1 -- start start 2023-11-08T16:38:45.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.218+0000 7fa73477e740 1 Processor -- start 2023-11-08T16:38:45.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.219+0000 7fa73477e740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f55270 0x55aac4f45540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.219+0000 7fa73477e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f86f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.219+0000 7fa73477e740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f42d70 0x55aac4f60340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.219+0000 7fa73477e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac4e30dd0 con 0x55aac4f55650 2023-11-08T16:38:45.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.219+0000 7fa73477e740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac4e30a70 con 0x55aac4f42d70 2023-11-08T16:38:45.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.219+0000 7fa73477e740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac4b8cf90 con 0x55aac4f55270 2023-11-08T16:38:45.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.219+0000 7fa73477e740 10 client.0 _ll_register_callbacks cb 0x55aac4e52e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:38:45.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.219+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f86f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.219+0000 7fa722ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f86f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60332/0 (socket says 172.21.15.81:60332) 2023-11-08T16:38:45.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.219+0000 7fa722ffd640 1 -- 172.21.15.81:0/3640693349 learned_addr learned my addr 172.21.15.81:0/3640693349 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:45.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa722ffd640 1 -- 172.21.15.81:0/3640693349 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f55270 msgr2=0x55aac4f45540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:45.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f55270 0x55aac4f45540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa72bfff640 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f42d70 0x55aac4f60340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa722ffd640 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f55270 0x55aac4f45540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa722ffd640 1 -- 172.21.15.81:0/3640693349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f42d70 msgr2=0x55aac4f60340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:45.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa722ffd640 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f42d70 0x55aac4f60340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa722ffd640 1 -- 172.21.15.81:0/3640693349 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71c005d00 con 0x55aac4f55650 2023-11-08T16:38:45.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f55270 0x55aac4f45540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:45.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa722ffd640 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f86f20 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa714007410 tx=0x7fa71405c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3cac9c3fe1d0a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:45.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa70a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:38:45.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3640693349 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7140569d0 con 0x55aac4f55650 2023-11-08T16:38:45.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3640693349 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fa714089040 con 0x55aac4f55650 2023-11-08T16:38:45.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa73477e740 1 -- 172.21.15.81:0/3640693349 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aac4eba400 con 0x55aac4f55650 2023-11-08T16:38:45.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3640693349 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa714072020 con 0x55aac4f55650 2023-11-08T16:38:45.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa73477e740 10 client.7497 fetch_fsmap learned FSMap version 78 2023-11-08T16:38:45.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.220+0000 7fa73477e740 1 -- 172.21.15.81:0/3640693349 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55aac4e2fd20 con 0x55aac4f55650 2023-11-08T16:38:45.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.221+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3640693349 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aac4eba400 con 0x55aac4f55650 2023-11-08T16:38:45.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.221+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3640693349 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 78) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fa714054360 con 0x55aac4f55650 2023-11-08T16:38:45.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.221+0000 7fa73477e740 10 client.7497 fetch_fsmap finished waiting for FSMap version 78 2023-11-08T16:38:45.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.221+0000 7fa73477e740 10 client.7497 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-08T16:38:45.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.221+0000 7fa73477e740 20 client.7497 populate_metadata read hostname 'smithi081' 2023-11-08T16:38:45.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.221+0000 7fa73477e740 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55aac4f5b000 0x55aac4f5b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.221+0000 7fa73477e740 4 client.7497 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:38:45.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.221+0000 7fa73477e740 1 -- 172.21.15.81:0/3640693349 --> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aac4e8cb20 con 0x55aac4f5b000 2023-11-08T16:38:45.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.221+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55aac4f5b000 0x55aac4f5b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.222+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55aac4f5b000 0x55aac4f5b450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6525b7ea8a05d948 server_cookie=edd5dc822fac09ac in_seq=0 out_seq=0 2023-11-08T16:38:45.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.222+0000 7fa70a7fc640 10 client.7497 ms_handle_connect on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:45.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.222+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3640693349 <== mds.0 v2:172.21.15.81:6836/1843050442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aac4e8cb20 con 0x55aac4f5b000 2023-11-08T16:38:45.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.222+0000 7fa70a7fc640 10 client.7497 handle_command_reply: tid=0 2023-11-08T16:38:45.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.222+0000 7fa73477e740 1 client.7497 shutdown 2023-11-08T16:38:45.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa73477e740 1 -- 172.21.15.81:0/3640693349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 msgr2=0x55aac4f86f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:45.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa73477e740 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f86f20 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fa714007410 tx=0x7fa71405c780 comp rx=0 tx=0).stop 2023-11-08T16:38:45.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa73477e740 1 -- 172.21.15.81:0/3640693349 shutdown_connections 2023-11-08T16:38:45.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa73477e740 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55aac4f5b000 0x55aac4f5b450 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa73477e740 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f42d70 0x55aac4f60340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa73477e740 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f86f20 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa73477e740 1 --2- 172.21.15.81:0/3640693349 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f55270 0x55aac4f45540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa70a7fc640 0 client.7497 ms_handle_reset on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:45.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa73477e740 1 -- 172.21.15.81:0/3640693349 >> 172.21.15.81:0/3640693349 conn(0x55aac4f28710 msgr2=0x55aac4f57ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:45.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa73477e740 1 -- 172.21.15.81:0/3640693349 shutdown_connections 2023-11-08T16:38:45.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa73477e740 1 -- 172.21.15.81:0/3640693349 wait complete. 2023-11-08T16:38:45.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.223+0000 7fa73477e740 20 client.7497 trim_cache size 0 max 16384 2023-11-08T16:38:45.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.239+0000 7fa73477e740 1 Processor -- start 2023-11-08T16:38:45.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.239+0000 7fa73477e740 1 -- start start 2023-11-08T16:38:45.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.240+0000 7fa73477e740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f42d70 0x55aac4f5ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.240+0000 7fa73477e740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f55270 0x55aac4f2be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.240+0000 7fa73477e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f2c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.240+0000 7fa73477e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac4f5c5c0 con 0x55aac4f55650 2023-11-08T16:38:45.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.240+0000 7fa73477e740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac4f761a0 con 0x55aac4f55270 2023-11-08T16:38:45.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.240+0000 7fa73477e740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac4f60520 con 0x55aac4f42d70 2023-11-08T16:38:45.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.241+0000 7fa72bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f2c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.241+0000 7fa72bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f2c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60338/0 (socket says 172.21.15.81:60338) 2023-11-08T16:38:45.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.241+0000 7fa72bfff640 1 -- 172.21.15.81:0/1492978514 learned_addr learned my addr 172.21.15.81:0/1492978514 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:45.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.241+0000 7fa72bfff640 1 -- 172.21.15.81:0/1492978514 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f42d70 msgr2=0x55aac4f5ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:45.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.241+0000 7fa72bfff640 1 --2- 172.21.15.81:0/1492978514 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f42d70 0x55aac4f5ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.241+0000 7fa72bfff640 1 -- 172.21.15.81:0/1492978514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f55270 msgr2=0x55aac4f2be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:45.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.241+0000 7fa72bfff640 1 --2- 172.21.15.81:0/1492978514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f55270 0x55aac4f2be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.241+0000 7fa72bfff640 1 -- 172.21.15.81:0/1492978514 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aac4f2ce20 con 0x55aac4f55650 2023-11-08T16:38:45.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.241+0000 7fa72bfff640 1 --2- 172.21.15.81:0/1492978514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f2c350 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa72402b9e0 tx=0x7fa724030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882a549d058b2399 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:45.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.241+0000 7fa721ffb640 1 -- 172.21.15.81:0/1492978514 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724030b30 con 0x55aac4f55650 2023-11-08T16:38:45.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.242+0000 7fa721ffb640 1 -- 172.21.15.81:0/1492978514 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa724030cb0 con 0x55aac4f55650 2023-11-08T16:38:45.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.242+0000 7fa721ffb640 1 -- 172.21.15.81:0/1492978514 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724039690 con 0x55aac4f55650 2023-11-08T16:38:45.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.242+0000 7fa73477e740 1 -- 172.21.15.81:0/1492978514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 msgr2=0x55aac4f2c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:45.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.242+0000 7fa73477e740 1 --2- 172.21.15.81:0/1492978514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f2c350 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fa72402b9e0 tx=0x7fa724030000 comp rx=0 tx=0).stop 2023-11-08T16:38:45.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.242+0000 7fa73477e740 1 -- 172.21.15.81:0/1492978514 shutdown_connections 2023-11-08T16:38:45.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.242+0000 7fa73477e740 1 --2- 172.21.15.81:0/1492978514 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f2c350 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.242+0000 7fa73477e740 1 --2- 172.21.15.81:0/1492978514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f55270 0x55aac4f2be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.242+0000 7fa73477e740 1 --2- 172.21.15.81:0/1492978514 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f42d70 0x55aac4f5ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.242+0000 7fa73477e740 1 -- 172.21.15.81:0/1492978514 >> 172.21.15.81:0/1492978514 conn(0x55aac4f28710 msgr2=0x55aac4f5b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:45.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.242+0000 7fa73477e740 1 -- 172.21.15.81:0/1492978514 shutdown_connections 2023-11-08T16:38:45.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.242+0000 7fa73477e740 1 -- 172.21.15.81:0/1492978514 wait complete. 2023-11-08T16:38:45.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.243+0000 7fa73477e740 1 -- start start 2023-11-08T16:38:45.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.243+0000 7fa73477e740 1 Processor -- start 2023-11-08T16:38:45.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.244+0000 7fa73477e740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f42d70 0x55aac4fe99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.244+0000 7fa73477e740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f55270 0x55aac4fa1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.244+0000 7fa73477e740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f76ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.244+0000 7fa73477e740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac4f766b0 con 0x55aac4f55650 2023-11-08T16:38:45.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.244+0000 7fa73477e740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac4fe9f10 con 0x55aac4f55270 2023-11-08T16:38:45.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.244+0000 7fa73477e740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac4fa2670 con 0x55aac4f42d70 2023-11-08T16:38:45.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.244+0000 7fa73477e740 10 client.0 _ll_register_callbacks cb 0x7fa714058340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:38:45.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.244+0000 7fa72bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f76ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.244+0000 7fa72bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f76ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60340/0 (socket says 172.21.15.81:60340) 2023-11-08T16:38:45.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.244+0000 7fa72bfff640 1 -- 172.21.15.81:0/3939378289 learned_addr learned my addr 172.21.15.81:0/3939378289 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:45.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa72bfff640 1 -- 172.21.15.81:0/3939378289 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f42d70 msgr2=0x55aac4fe99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:45.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa722ffd640 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f55270 0x55aac4fa1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa72bfff640 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f42d70 0x55aac4fe99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa72bfff640 1 -- 172.21.15.81:0/3939378289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f55270 msgr2=0x55aac4fa1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:45.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa72bfff640 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f55270 0x55aac4fa1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa72bfff640 1 -- 172.21.15.81:0/3939378289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c0ca640 con 0x55aac4f55650 2023-11-08T16:38:45.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa72bfff640 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f76ad0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa724009960 tx=0x7fa724004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b6579f4b8c4981 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:45.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa70a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:38:45.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3939378289 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa724003810 con 0x55aac4f55650 2023-11-08T16:38:45.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3939378289 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fa72403e070 con 0x55aac4f55650 2023-11-08T16:38:45.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa73477e740 1 -- 172.21.15.81:0/3939378289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa71c0079b0 con 0x55aac4f55650 2023-11-08T16:38:45.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3939378289 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa724030e20 con 0x55aac4f55650 2023-11-08T16:38:45.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa73477e740 10 client.7503 fetch_fsmap learned FSMap version 78 2023-11-08T16:38:45.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.245+0000 7fa73477e740 1 -- 172.21.15.81:0/3939378289 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=2+}) v3 -- 0x55aac4ec3750 con 0x55aac4f55650 2023-11-08T16:38:45.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.246+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3939378289 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c0079b0 con 0x55aac4f55650 2023-11-08T16:38:45.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.246+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3939378289 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 78) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fa724003970 con 0x55aac4f55650 2023-11-08T16:38:45.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.246+0000 7fa73477e740 10 client.7503 fetch_fsmap finished waiting for FSMap version 78 2023-11-08T16:38:45.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.246+0000 7fa73477e740 10 client.7503 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2023-11-08T16:38:45.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.246+0000 7fa73477e740 20 client.7503 populate_metadata read hostname 'smithi081' 2023-11-08T16:38:45.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.246+0000 7fa73477e740 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55aac4f77010 0x55aac4f2be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:45.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.246+0000 7fa73477e740 4 client.7503 mds_command: new command op to 7383 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-08T16:38:45.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.246+0000 7fa73477e740 1 -- 172.21.15.81:0/3939378289 --> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aac4e8c840 con 0x55aac4f77010 2023-11-08T16:38:45.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.246+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55aac4f77010 0x55aac4f2be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:45.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa72b7fe640 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55aac4f77010 0x55aac4f2be10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb649254a1f698fb server_cookie=3305272ce28a11a4 in_seq=0 out_seq=0 2023-11-08T16:38:45.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa70a7fc640 10 client.7503 ms_handle_connect on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:45.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa70a7fc640 1 -- 172.21.15.81:0/3939378289 <== mds.0 v2:172.21.15.81:6836/1843050442 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55aac4e8c840 con 0x55aac4f77010 2023-11-08T16:38:45.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa70a7fc640 10 client.7503 handle_command_reply: tid=0 2023-11-08T16:38:45.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa73477e740 1 client.7503 shutdown 2023-11-08T16:38:45.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa73477e740 1 -- 172.21.15.81:0/3939378289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 msgr2=0x55aac4f76ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:45.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa73477e740 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f76ad0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fa724009960 tx=0x7fa724004730 comp rx=0 tx=0).stop 2023-11-08T16:38:45.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa73477e740 1 -- 172.21.15.81:0/3939378289 shutdown_connections 2023-11-08T16:38:45.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa73477e740 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.81:6836/1843050442,v1:172.21.15.81:6837/1843050442] conn(0x55aac4f77010 0x55aac4f2be10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa73477e740 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55aac4f55650 0x55aac4f76ad0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa73477e740 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55aac4f55270 0x55aac4fa1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa73477e740 1 --2- 172.21.15.81:0/3939378289 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55aac4f42d70 0x55aac4fe99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa73477e740 1 -- 172.21.15.81:0/3939378289 >> 172.21.15.81:0/3939378289 conn(0x55aac4f28710 msgr2=0x55aac4f95820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:45.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa70a7fc640 0 client.7503 ms_handle_reset on v2:172.21.15.81:6836/1843050442 2023-11-08T16:38:45.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa73477e740 1 -- 172.21.15.81:0/3939378289 shutdown_connections 2023-11-08T16:38:45.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.247+0000 7fa73477e740 1 -- 172.21.15.81:0/3939378289 wait complete. 2023-11-08T16:38:45.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa73477e740 20 client.7503 trim_cache size 0 max 16384 2023-11-08T16:38:45.264 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:38:45.264 INFO:teuthology.orchestra.run.smithi081.stdout: "status": "no active scrubs running", 2023-11-08T16:38:45.264 INFO:teuthology.orchestra.run.smithi081.stdout: "scrubs": {} 2023-11-08T16:38:45.264 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:38:45.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 -- 172.21.15.81:0/934511586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa70c06ae70 msgr2=0x7fa70c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:45.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa70c06ae70 0x7fa70c06d330 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa71c0012e0 tx=0x7fa71c009040 comp rx=0 tx=0).stop 2023-11-08T16:38:45.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 -- 172.21.15.81:0/934511586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa72c071890 msgr2=0x7fa72c0b3240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:45.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa72c071890 0x7fa72c0b3240 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fa71400c7f0 tx=0x7fa71400cbf0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 -- 172.21.15.81:0/934511586 shutdown_connections 2023-11-08T16:38:45.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa70c06ae70 0x7fa70c06d330 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa72c15afd0 0x7fa72c0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa72c071890 0x7fa72c0b3240 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 --2- 172.21.15.81:0/934511586 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa72c0714b0 0x7fa72c14afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 -- 172.21.15.81:0/934511586 >> 172.21.15.81:0/934511586 conn(0x7fa72c0cb4e0 msgr2=0x7fa72c0ca880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 -- 172.21.15.81:0/934511586 shutdown_connections 2023-11-08T16:38:45.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45.248+0000 7fa731bee640 1 -- 172.21.15.81:0/934511586 wait complete. 2023-11-08T16:38:45.269 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-08T16:38:45.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:45.270 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo ls subdir) 2023-11-08T16:38:45.363 INFO:teuthology.orchestra.run.smithi081.stdout:sixmegs 2023-11-08T16:38:45.367 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:38:45.367 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:38:45.368 DEBUG:teuthology.orchestra.run.smithi081:> import stat 2023-11-08T16:38:45.368 DEBUG:teuthology.orchestra.run.smithi081:> import json 2023-11-08T16:38:45.369 DEBUG:teuthology.orchestra.run.smithi081:> import sys 2023-11-08T16:38:45.369 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:38:45.369 DEBUG:teuthology.orchestra.run.smithi081:> try: 2023-11-08T16:38:45.370 DEBUG:teuthology.orchestra.run.smithi081:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-11-08T16:38:45.370 DEBUG:teuthology.orchestra.run.smithi081:> except OSError as e: 2023-11-08T16:38:45.370 DEBUG:teuthology.orchestra.run.smithi081:> sys.exit(e.errno) 2023-11-08T16:38:45.371 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:38:45.371 DEBUG:teuthology.orchestra.run.smithi081:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-11-08T16:38:45.371 DEBUG:teuthology.orchestra.run.smithi081:> print(json.dumps( 2023-11-08T16:38:45.372 DEBUG:teuthology.orchestra.run.smithi081:> dict([(a, getattr(s, a)) for a in attrs]), 2023-11-08T16:38:45.372 DEBUG:teuthology.orchestra.run.smithi081:> indent=2)) 2023-11-08T16:38:45.372 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:38:45.540 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:38:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: "st_mode": 33088, 2023-11-08T16:38:45.540 INFO:teuthology.orchestra.run.smithi081.stdout: "st_ino": 1099511627777, 2023-11-08T16:38:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: "st_dev": 40, 2023-11-08T16:38:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: "st_nlink": 1, 2023-11-08T16:38:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: "st_uid": 0, 2023-11-08T16:38:45.541 INFO:teuthology.orchestra.run.smithi081.stdout: "st_gid": 0, 2023-11-08T16:38:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "st_size": 6291456, 2023-11-08T16:38:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "st_atime": 1699461478.0, 2023-11-08T16:38:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "st_mtime": 1699461478.0, 2023-11-08T16:38:45.542 INFO:teuthology.orchestra.run.smithi081.stdout: "st_ctime": 1699461478.0 2023-11-08T16:38:45.542 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:38:45.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:38:45.557 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T16:38:45.558 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:38:45.724 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:38:45.849 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T16:38:45.850 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:38:45.850 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:38:45.908 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:38:46.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.111+0000 7fc7ef07c640 1 Processor -- start 2023-11-08T16:38:46.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.111+0000 7fc7ef07c640 1 -- start start 2023-11-08T16:38:46.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.112+0000 7fc7ef07c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc7e80cc5e0 0x7fc7e80cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.113+0000 7fc7ef07c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc7e80ccf80 0x7fc7e8070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.113+0000 7fc7ef07c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e80c9a70 con 0x7fc7e80adcd0 2023-11-08T16:38:46.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.113+0000 7fc7ef07c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7e80cc3e0 con 0x7fc7e80ccf80 2023-11-08T16:38:46.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.113+0000 7fc7ef07c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e806d070 con 0x7fc7e80cc5e0 2023-11-08T16:38:46.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.113+0000 7fc7ece18640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fc7e80adcd0 0x7fc7e80ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:44684/0 (socket says 172.21.15.81:44684) 2023-11-08T16:38:46.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.113+0000 7fc7ece18640 1 -- 172.21.15.81:0/2837492165 learned_addr learned my addr 172.21.15.81:0/2837492165 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:46.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.113+0000 7fc7ed619640 1 --2- 172.21.15.81:0/2837492165 >> v2:172.21.15.93:3300/0 conn(0x7fc7e80ccf80 0x7fc7e8070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.113+0000 7fc7e7fff640 1 --2- 172.21.15.81:0/2837492165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc7e80cc5e0 0x7fc7e80cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.114+0000 7fc7e77fe640 1 -- 172.21.15.81:0/2837492165 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042381158 0 0) 0x7fc7e80c9a70 con 0x7fc7e80adcd0 2023-11-08T16:38:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.114+0000 7fc7e77fe640 1 -- 172.21.15.81:0/2837492165 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7d0003520 con 0x7fc7e80adcd0 2023-11-08T16:38:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.114+0000 7fc7ed619640 1 -- 172.21.15.81:0/2837492165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc7e80cc5e0 msgr2=0x7fc7e80cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.114+0000 7fc7ed619640 1 --2- 172.21.15.81:0/2837492165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc7e80cc5e0 0x7fc7e80cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.114+0000 7fc7ed619640 1 -- 172.21.15.81:0/2837492165 >> v1:172.21.15.93:6789/0 conn(0x7fc7e80adcd0 legacy=0x7fc7e80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:46.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.114+0000 7fc7ed619640 1 -- 172.21.15.81:0/2837492165 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e806d350 con 0x7fc7e80ccf80 2023-11-08T16:38:46.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.116+0000 7fc7ed619640 1 --2- 172.21.15.81:0/2837492165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7e80ccf80 0x7fc7e8070050 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009d90 tx=0x7fc7e002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2e1326771e8dda server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:46.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.116+0000 7fc7e77fe640 1 -- 172.21.15.81:0/2837492165 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e002cc50 con 0x7fc7e80ccf80 2023-11-08T16:38:46.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.116+0000 7fc7e77fe640 1 -- 172.21.15.81:0/2837492165 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e002cdb0 con 0x7fc7e80ccf80 2023-11-08T16:38:46.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.116+0000 7fc7e77fe640 1 -- 172.21.15.81:0/2837492165 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0034640 con 0x7fc7e80ccf80 2023-11-08T16:38:46.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.117+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2837492165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7e80ccf80 msgr2=0x7fc7e8070050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:46.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.117+0000 7fc7ef07c640 1 --2- 172.21.15.81:0/2837492165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7e80ccf80 0x7fc7e8070050 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009d90 tx=0x7fc7e002bea0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.117+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2837492165 shutdown_connections 2023-11-08T16:38:46.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.117+0000 7fc7ef07c640 1 --2- 172.21.15.81:0/2837492165 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7e80ccf80 0x7fc7e8070050 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.117+0000 7fc7ef07c640 1 --2- 172.21.15.81:0/2837492165 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc7e80cc5e0 0x7fc7e80cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.117+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2837492165 >> 172.21.15.81:0/2837492165 conn(0x7fc7e80cb490 msgr2=0x7fc7e80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:46.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.117+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2837492165 shutdown_connections 2023-11-08T16:38:46.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.117+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2837492165 wait complete. 2023-11-08T16:38:46.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.119+0000 7fc7ef07c640 1 Processor -- start 2023-11-08T16:38:46.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.119+0000 7fc7ef07c640 1 -- start start 2023-11-08T16:38:46.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ef07c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7e80adcd0 0x7fc7e81f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ef07c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7e80cc5e0 0x7fc7e8151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ef07c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc7e80ccf80 0x7fc7e80b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ef07c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80a9600 con 0x7fc7e80adcd0 2023-11-08T16:38:46.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ef07c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80a98e0 con 0x7fc7e80cc5e0 2023-11-08T16:38:46.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ef07c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80a9bc0 con 0x7fc7e80ccf80 2023-11-08T16:38:46.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ece18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7e80adcd0 0x7fc7e81f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ece18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7e80adcd0 0x7fc7e81f3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60352/0 (socket says 172.21.15.81:60352) 2023-11-08T16:38:46.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ece18640 1 -- 172.21.15.81:0/2778610664 learned_addr learned my addr 172.21.15.81:0/2778610664 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:46.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7e7fff640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7e80cc5e0 0x7fc7e8151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ed619640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc7e80ccf80 0x7fc7e80b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ece18640 1 -- 172.21.15.81:0/2778610664 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc7e80ccf80 msgr2=0x7fc7e80b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:46.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ece18640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc7e80ccf80 0x7fc7e80b34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ece18640 1 -- 172.21.15.81:0/2778610664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7e80cc5e0 msgr2=0x7fc7e8151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:46.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ece18640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7e80cc5e0 0x7fc7e8151e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.120+0000 7fc7ece18640 1 -- 172.21.15.81:0/2778610664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e0009a70 con 0x7fc7e80adcd0 2023-11-08T16:38:46.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.121+0000 7fc7ece18640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7e80adcd0 0x7fc7e81f3c60 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00c390 tx=0x7fc7dc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:46.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.121+0000 7fc7e57fa640 1 -- 172.21.15.81:0/2778610664 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc005ce0 con 0x7fc7e80adcd0 2023-11-08T16:38:46.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.121+0000 7fc7e57fa640 1 -- 172.21.15.81:0/2778610664 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7dc005e40 con 0x7fc7e80adcd0 2023-11-08T16:38:46.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.121+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2778610664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e80a9ea0 con 0x7fc7e80adcd0 2023-11-08T16:38:46.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.121+0000 7fc7e57fa640 1 -- 172.21.15.81:0/2778610664 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc0072f0 con 0x7fc7e80adcd0 2023-11-08T16:38:46.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.121+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2778610664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e81523b0 con 0x7fc7e80adcd0 2023-11-08T16:38:46.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.122+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2778610664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e8072170 con 0x7fc7e80adcd0 2023-11-08T16:38:46.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.123+0000 7fc7e57fa640 1 -- 172.21.15.81:0/2778610664 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc7dc007450 con 0x7fc7e80adcd0 2023-11-08T16:38:46.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.124+0000 7fc7e57fa640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc7d406ae70 0x7fc7d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.124+0000 7fc7e57fa640 1 -- 172.21.15.81:0/2778610664 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fc7dc088fa0 con 0x7fc7e80adcd0 2023-11-08T16:38:46.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.124+0000 7fc7e7fff640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc7d406ae70 0x7fc7d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.125+0000 7fc7e7fff640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc7d406ae70 0x7fc7d406d330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80c9960 tx=0x7fc7d8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:46.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.130+0000 7fc7e57fa640 1 -- 172.21.15.81:0/2778610664 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc7dc053ce0 con 0x7fc7e80adcd0 2023-11-08T16:38:46.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.320+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2778610664 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7e80aa3c0 con 0x7fc7e80adcd0 2023-11-08T16:38:46.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.321+0000 7fc7e57fa640 1 -- 172.21.15.81:0/2778610664 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18770 (secure 0 0 0) 0x7fc7dc017020 con 0x7fc7e80adcd0 2023-11-08T16:38:46.323 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:46.324 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":90,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:38:07.597635+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:37:30.174436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T16:37:31.340720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:37:42.279296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2e","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6836/1764545681":"2023-11-09T16:38:06.974694+0000","172.21.15.81:6837/1764545681":"2023-11-09T16:38:06.974694+0000","172.21.15.81:6837/457413856":"2023-11-09T16:38:00.541280+0000","172.21.15.81:6836/457413856":"2023-11-09T16:38:00.541280+0000","172.21.15.81:6837/2162767469":"2023-11-09T16:37:58.508547+0000","172.21.15.81:6836/2162767469":"2023-11-09T16:37:58.508547+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:38:46.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2778610664 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc7d406ae70 msgr2=0x7fc7d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:46.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc7d406ae70 0x7fc7d406d330 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc7e80c9960 tx=0x7fc7d8008040 comp rx=0 tx=0).stop 2023-11-08T16:38:46.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2778610664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7e80adcd0 msgr2=0x7fc7e81f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:46.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7e80adcd0 0x7fc7e81f3c60 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc00c390 tx=0x7fc7dc00c850 comp rx=0 tx=0).stop 2023-11-08T16:38:46.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2778610664 shutdown_connections 2023-11-08T16:38:46.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc7d406ae70 0x7fc7d406d330 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc7e80ccf80 0x7fc7e80b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc7e80cc5e0 0x7fc7e8151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 --2- 172.21.15.81:0/2778610664 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc7e80adcd0 0x7fc7e81f3c60 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2778610664 >> 172.21.15.81:0/2778610664 conn(0x7fc7e80cb490 msgr2=0x7fc7e815cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:46.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2778610664 shutdown_connections 2023-11-08T16:38:46.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.324+0000 7fc7ef07c640 1 -- 172.21.15.81:0/2778610664 wait complete. 2023-11-08T16:38:46.345 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:37:30.174436+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:37:31.340720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:37:42.279296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:38:46.346 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:38:46.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.562+0000 7fc1cba07640 1 Processor -- start 2023-11-08T16:38:46.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.562+0000 7fc1cba07640 1 -- start start 2023-11-08T16:38:46.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.563+0000 7fc1cba07640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc1c40ce0d0 0x7fc1c40caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.563+0000 7fc1cba07640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415bfc0 0x7fc1c415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.563+0000 7fc1cba07640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc1c415f0b0 0x7fc1c41516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.563+0000 7fc1cba07640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c415f540 con 0x7fc1c415f0b0 2023-11-08T16:38:46.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.563+0000 7fc1cba07640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c406fa30 con 0x7fc1c40ce0d0 2023-11-08T16:38:46.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.563+0000 7fc1cba07640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40ce4b0 con 0x7fc1c415bfc0 2023-11-08T16:38:46.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.564+0000 7fc1c9fa4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc1c415f0b0 0x7fc1c41516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.564+0000 7fc1c9fa4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc1c415f0b0 0x7fc1c41516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60356/0 (socket says 172.21.15.81:60356) 2023-11-08T16:38:46.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.564+0000 7fc1c9fa4640 1 -- 172.21.15.81:0/10597326 learned_addr learned my addr 172.21.15.81:0/10597326 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:46.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.564+0000 7fc1c97a3640 1 --2- 172.21.15.81:0/10597326 >> v2:172.21.15.93:3300/0 conn(0x7fc1c40ce0d0 0x7fc1c40caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.564+0000 7fc1c8fa2640 1 --2- 172.21.15.81:0/10597326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415bfc0 0x7fc1c415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.564+0000 7fc1c9fa4640 1 -- 172.21.15.81:0/10597326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415bfc0 msgr2=0x7fc1c415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:46.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.564+0000 7fc1c9fa4640 1 --2- 172.21.15.81:0/10597326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415bfc0 0x7fc1c415eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.564+0000 7fc1c9fa4640 1 -- 172.21.15.81:0/10597326 >> v2:172.21.15.93:3300/0 conn(0x7fc1c40ce0d0 msgr2=0x7fc1c40caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.564+0000 7fc1c9fa4640 1 --2- 172.21.15.81:0/10597326 >> v2:172.21.15.93:3300/0 conn(0x7fc1c40ce0d0 0x7fc1c40caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.564+0000 7fc1c9fa4640 1 -- 172.21.15.81:0/10597326 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c406fca0 con 0x7fc1c415f0b0 2023-11-08T16:38:46.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.565+0000 7fc1c9fa4640 1 --2- 172.21.15.81:0/10597326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f0b0 0x7fc1c41516e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc1c406f3a0 tx=0x7fc1bc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486222ef24434e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:46.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.566+0000 7fc1bbfff640 1 -- 172.21.15.81:0/10597326 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc02ca80 con 0x7fc1c415f0b0 2023-11-08T16:38:46.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.566+0000 7fc1bbfff640 1 -- 172.21.15.81:0/10597326 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc02cbe0 con 0x7fc1c415f0b0 2023-11-08T16:38:46.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.566+0000 7fc1bbfff640 1 -- 172.21.15.81:0/10597326 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0343f0 con 0x7fc1c415f0b0 2023-11-08T16:38:46.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.566+0000 7fc1cba07640 1 -- 172.21.15.81:0/10597326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f0b0 msgr2=0x7fc1c41516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:46.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.566+0000 7fc1cba07640 1 --2- 172.21.15.81:0/10597326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f0b0 0x7fc1c41516e0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fc1c406f3a0 tx=0x7fc1bc02bee0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.566+0000 7fc1cba07640 1 -- 172.21.15.81:0/10597326 shutdown_connections 2023-11-08T16:38:46.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.566+0000 7fc1cba07640 1 --2- 172.21.15.81:0/10597326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415bfc0 0x7fc1c415eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.566+0000 7fc1cba07640 1 --2- 172.21.15.81:0/10597326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f0b0 0x7fc1c41516e0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.567+0000 7fc1cba07640 1 --2- 172.21.15.81:0/10597326 >> v2:172.21.15.93:3300/0 conn(0x7fc1c40ce0d0 0x7fc1c40caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.567+0000 7fc1cba07640 1 -- 172.21.15.81:0/10597326 >> 172.21.15.81:0/10597326 conn(0x7fc1c40cb580 msgr2=0x7fc1c40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:46.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.567+0000 7fc1cba07640 1 -- 172.21.15.81:0/10597326 shutdown_connections 2023-11-08T16:38:46.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.567+0000 7fc1cba07640 1 -- 172.21.15.81:0/10597326 wait complete. 2023-11-08T16:38:46.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.568+0000 7fc1cba07640 1 Processor -- start 2023-11-08T16:38:46.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.568+0000 7fc1cba07640 1 -- start start 2023-11-08T16:38:46.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.569+0000 7fc1cba07640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 0x7fc1c40c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.569+0000 7fc1cba07640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415bfc0 0x7fc1c415a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.569+0000 7fc1cba07640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f0b0 0x7fc1c415d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.569+0000 7fc1cba07640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c3170 con 0x7fc1c415f0b0 2023-11-08T16:38:46.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.569+0000 7fc1cba07640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c4b70 con 0x7fc1c40ce0d0 2023-11-08T16:38:46.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.569+0000 7fc1cba07640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40c27d0 con 0x7fc1c415bfc0 2023-11-08T16:38:46.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c9fa4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f0b0 0x7fc1c415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c9fa4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f0b0 0x7fc1c415d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60358/0 (socket says 172.21.15.81:60358) 2023-11-08T16:38:46.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c9fa4640 1 -- 172.21.15.81:0/1652067291 learned_addr learned my addr 172.21.15.81:0/1652067291 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:46.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c97a3640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 0x7fc1c40c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c8fa2640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415bfc0 0x7fc1c415a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c9fa4640 1 -- 172.21.15.81:0/1652067291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415bfc0 msgr2=0x7fc1c415a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:46.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c9fa4640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415bfc0 0x7fc1c415a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c9fa4640 1 -- 172.21.15.81:0/1652067291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 msgr2=0x7fc1c40c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:46.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c9fa4640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 0x7fc1c40c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c9fa4640 1 -- 172.21.15.81:0/1652067291 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1bc009b30 con 0x7fc1c415f0b0 2023-11-08T16:38:46.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c97a3640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 0x7fc1c40c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:46.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c8fa2640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415bfc0 0x7fc1c415a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:46.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.570+0000 7fc1c9fa4640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f0b0 0x7fc1c415d720 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009f40 tx=0x7fc1bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:46.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.571+0000 7fc1b9ffb640 1 -- 172.21.15.81:0/1652067291 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc009510 con 0x7fc1c415f0b0 2023-11-08T16:38:46.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.571+0000 7fc1b9ffb640 1 -- 172.21.15.81:0/1652067291 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1bc03e450 con 0x7fc1c415f0b0 2023-11-08T16:38:46.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.571+0000 7fc1b9ffb640 1 -- 172.21.15.81:0/1652067291 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc03e7e0 con 0x7fc1c415f0b0 2023-11-08T16:38:46.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.571+0000 7fc1cba07640 1 -- 172.21.15.81:0/1652067291 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c40c2ab0 con 0x7fc1c415f0b0 2023-11-08T16:38:46.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.571+0000 7fc1cba07640 1 -- 172.21.15.81:0/1652067291 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c40c4880 con 0x7fc1c415f0b0 2023-11-08T16:38:46.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.572+0000 7fc1cba07640 1 -- 172.21.15.81:0/1652067291 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c404ece0 con 0x7fc1c415f0b0 2023-11-08T16:38:46.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.573+0000 7fc1b9ffb640 1 -- 172.21.15.81:0/1652067291 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc1bc02cd50 con 0x7fc1c415f0b0 2023-11-08T16:38:46.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.574+0000 7fc1b9ffb640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 0x7fc1a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:46.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.574+0000 7fc1b9ffb640 1 -- 172.21.15.81:0/1652067291 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fc1bc0ac7c0 con 0x7fc1c415f0b0 2023-11-08T16:38:46.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.574+0000 7fc1c97a3640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 0x7fc1a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:46.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.575+0000 7fc1c97a3640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 0x7fc1a806d330 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0006fd0 tx=0x7fc1c0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:46.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.578+0000 7fc1b9ffb640 1 -- 172.21.15.81:0/1652067291 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1bc077620 con 0x7fc1c415f0b0 2023-11-08T16:38:46.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.769+0000 7fc1cba07640 1 -- 172.21.15.81:0/1652067291 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1c40ce6b0 con 0x7fc1c415f0b0 2023-11-08T16:38:46.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.770+0000 7fc1b9ffb640 1 -- 172.21.15.81:0/1652067291 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1bc081070 con 0x7fc1c415f0b0 2023-11-08T16:38:46.772 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:38:46.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 -- 172.21.15.81:0/1652067291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 msgr2=0x7fc1a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:46.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 0x7fc1a806d330 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0006fd0 tx=0x7fc1c0008040 comp rx=0 tx=0).stop 2023-11-08T16:38:46.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 -- 172.21.15.81:0/1652067291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f0b0 msgr2=0x7fc1c415d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:46.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f0b0 0x7fc1c415d720 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc009f40 tx=0x7fc1bc004a40 comp rx=0 tx=0).stop 2023-11-08T16:38:46.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 -- 172.21.15.81:0/1652067291 shutdown_connections 2023-11-08T16:38:46.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc1a806ae70 0x7fc1a806d330 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1c415f0b0 0x7fc1c415d720 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1c415bfc0 0x7fc1c415a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 --2- 172.21.15.81:0/1652067291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1c40ce0d0 0x7fc1c40c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:46.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 -- 172.21.15.81:0/1652067291 >> 172.21.15.81:0/1652067291 conn(0x7fc1c40cb580 msgr2=0x7fc1c415aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:46.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 -- 172.21.15.81:0/1652067291 shutdown_connections 2023-11-08T16:38:46.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:46.773+0000 7fc1cba07640 1 -- 172.21.15.81:0/1652067291 wait complete. 2023-11-08T16:38:46.791 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:38:47.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.004+0000 7f96f5385640 1 Processor -- start 2023-11-08T16:38:47.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.004+0000 7f96f5385640 1 -- start start 2023-11-08T16:38:47.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.006+0000 7f96f5385640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f96f015cf30 0x7f96f00d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.006+0000 7f96f5385640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f00ce0b0 con 0x7f96f00cfd90 2023-11-08T16:38:47.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.006+0000 7f96f5385640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f00ce390 con 0x7f96f015d310 2023-11-08T16:38:47.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.006+0000 7f96f5385640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96f00d1be0 con 0x7f96f015cf30 2023-11-08T16:38:47.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.006+0000 7f96f4b84640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f96f00cfd90 0x7f96f00b5f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46122/0 (socket says 172.21.15.81:46122) 2023-11-08T16:38:47.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.006+0000 7f96f4b84640 1 -- 172.21.15.81:0/4123520347 learned_addr learned my addr 172.21.15.81:0/4123520347 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:47.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543775230 0 0) 0x7f96f00ce0b0 con 0x7f96f00cfd90 2023-11-08T16:38:47.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96d8003520 con 0x7f96f00cfd90 2023-11-08T16:38:47.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96effff640 1 --2- 172.21.15.81:0/4123520347 >> v2:172.21.15.93:3300/0 conn(0x7f96f015cf30 0x7f96f00d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 524605027 0 0) 0x7f96d8003520 con 0x7f96f00cfd90 2023-11-08T16:38:47.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 >> v2:172.21.15.93:3300/0 conn(0x7f96f015cf30 msgr2=0x7f96f00d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:47.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 --2- 172.21.15.81:0/4123520347 >> v2:172.21.15.93:3300/0 conn(0x7f96f015cf30 0x7f96f00d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 >> v1:172.21.15.93:6789/0 conn(0x7f96f015d310 legacy=0x7f96f00cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f00d1ec0 con 0x7f96f00cfd90 2023-11-08T16:38:47.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f96e80019f0 con 0x7f96f00cfd90 2023-11-08T16:38:47.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 >> v1:172.21.15.81:6789/0 conn(0x7f96f00cfd90 legacy=0x7f96f00b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96d80039c0 0x7f96d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96d80047c0 0x7f96d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 0x7f96d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00d1be0 con 0x7f96d80051d0 2023-11-08T16:38:47.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d800c120 con 0x7f96d80047c0 2023-11-08T16:38:47.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.007+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d800c2e0 con 0x7f96d80039c0 2023-11-08T16:38:47.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.008+0000 7f96f4b84640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 0x7f96d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.008+0000 7f96effff640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96d80039c0 0x7f96d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.008+0000 7f96ef7fe640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96d80047c0 0x7f96d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.008+0000 7f96f4b84640 1 -- 172.21.15.81:0/4123520347 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96d80039c0 msgr2=0x7f96d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:47.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.008+0000 7f96f4b84640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96d80039c0 0x7f96d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.008+0000 7f96f4b84640 1 -- 172.21.15.81:0/4123520347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96d80047c0 msgr2=0x7f96d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:47.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.008+0000 7f96f4b84640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96d80047c0 0x7f96d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.008+0000 7f96f4b84640 1 -- 172.21.15.81:0/4123520347 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d8003520 con 0x7f96d80051d0 2023-11-08T16:38:47.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.008+0000 7f96effff640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96d80039c0 0x7f96d8003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:47.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.008+0000 7f96ef7fe640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96d80047c0 0x7f96d8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:47.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.009+0000 7f96f4b84640 1 -- 172.21.15.81:0/4123520347 reap_dead start 2023-11-08T16:38:47.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.009+0000 7f96f4b84640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 0x7f96d800b980 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f96e800a760 tx=0x7f96d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1731a2b11e46d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:47.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.009+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8034630 con 0x7f96d80051d0 2023-11-08T16:38:47.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.009+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e802f070 con 0x7f96d80051d0 2023-11-08T16:38:47.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.009+0000 7f96eeffd640 1 -- 172.21.15.81:0/4123520347 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e8034a10 con 0x7f96d80051d0 2023-11-08T16:38:47.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.010+0000 7f96f5385640 1 -- 172.21.15.81:0/4123520347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 msgr2=0x7f96d800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.010+0000 7f96f5385640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 0x7f96d800b980 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f96e800a760 tx=0x7f96d80057f0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.010+0000 7f96f5385640 1 -- 172.21.15.81:0/4123520347 shutdown_connections 2023-11-08T16:38:47.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.010+0000 7f96f5385640 1 --2- 172.21.15.81:0/4123520347 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 0x7f96d800b980 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.010+0000 7f96f5385640 1 -- 172.21.15.81:0/4123520347 >> 172.21.15.81:0/4123520347 conn(0x7f96f006c040 msgr2=0x7f96f006c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:47.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.010+0000 7f96f5385640 1 -- 172.21.15.81:0/4123520347 shutdown_connections 2023-11-08T16:38:47.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.010+0000 7f96f5385640 1 -- 172.21.15.81:0/4123520347 wait complete. 2023-11-08T16:38:47.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.012+0000 7f96f5385640 1 Processor -- start 2023-11-08T16:38:47.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.012+0000 7f96f5385640 1 -- start start 2023-11-08T16:38:47.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96f5385640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 0x7f96f01563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96f5385640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f00c8650 0x7f96f00b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96f5385640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96f00c7cc0 0x7f96f00b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96f5385640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00c8b50 con 0x7f96d80051d0 2023-11-08T16:38:47.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96f5385640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00c8e30 con 0x7f96f00c8650 2023-11-08T16:38:47.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96f5385640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f00c9110 con 0x7f96f00c7cc0 2023-11-08T16:38:47.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96effff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 0x7f96f01563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96effff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 0x7f96f01563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60368/0 (socket says 172.21.15.81:60368) 2023-11-08T16:38:47.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96effff640 1 -- 172.21.15.81:0/2374889823 learned_addr learned my addr 172.21.15.81:0/2374889823 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:47.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96f4b84640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96f00c7cc0 0x7f96f00b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96ef7fe640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f00c8650 0x7f96f00b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96effff640 1 -- 172.21.15.81:0/2374889823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96f00c7cc0 msgr2=0x7f96f00b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96effff640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96f00c7cc0 0x7f96f00b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96effff640 1 -- 172.21.15.81:0/2374889823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f00c8650 msgr2=0x7f96f00b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.013+0000 7f96effff640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f00c8650 0x7f96f00b64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.014+0000 7f96effff640 1 -- 172.21.15.81:0/2374889823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f00c93f0 con 0x7f96d80051d0 2023-11-08T16:38:47.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.014+0000 7f96ef7fe640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f00c8650 0x7f96f00b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:47.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.014+0000 7f96effff640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 0x7f96f01563a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e7e0 tx=0x7f96e000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:47.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.014+0000 7f96ecff9640 1 -- 172.21.15.81:0/2374889823 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e000c6d0 con 0x7f96d80051d0 2023-11-08T16:38:47.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.014+0000 7f96ecff9640 1 -- 172.21.15.81:0/2374889823 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e000c830 con 0x7f96d80051d0 2023-11-08T16:38:47.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.014+0000 7f96f5385640 1 -- 172.21.15.81:0/2374889823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f00ccd20 con 0x7f96d80051d0 2023-11-08T16:38:47.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.014+0000 7f96ecff9640 1 -- 172.21.15.81:0/2374889823 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e000cba0 con 0x7f96d80051d0 2023-11-08T16:38:47.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.014+0000 7f96f5385640 1 -- 172.21.15.81:0/2374889823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f00c6980 con 0x7f96d80051d0 2023-11-08T16:38:47.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.015+0000 7f96f5385640 1 -- 172.21.15.81:0/2374889823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f0156c40 con 0x7f96d80051d0 2023-11-08T16:38:47.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.016+0000 7f96ecff9640 1 -- 172.21.15.81:0/2374889823 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f96e000cd00 con 0x7f96d80051d0 2023-11-08T16:38:47.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.017+0000 7f96ecff9640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f96dc06aea0 0x7f96dc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.017+0000 7f96ecff9640 1 -- 172.21.15.81:0/2374889823 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f96e0089730 con 0x7f96d80051d0 2023-11-08T16:38:47.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.017+0000 7f96ef7fe640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f96dc06aea0 0x7f96dc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.018+0000 7f96ef7fe640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f96dc06aea0 0x7f96dc06d360 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f96e4007fd0 tx=0x7f96e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:47.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.022+0000 7f96ecff9640 1 -- 172.21.15.81:0/2374889823 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f96e0054470 con 0x7f96d80051d0 2023-11-08T16:38:47.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.210+0000 7f96f5385640 1 -- 172.21.15.81:0/2374889823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96f00d2170 con 0x7f96d80051d0 2023-11-08T16:38:47.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.211+0000 7f96ecff9640 1 -- 172.21.15.81:0/2374889823 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96e0057a90 con 0x7f96d80051d0 2023-11-08T16:38:47.213 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:38:47.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.214+0000 7f96f5385640 1 -- 172.21.15.81:0/2374889823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f96dc06aea0 msgr2=0x7f96dc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.214+0000 7f96f5385640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f96dc06aea0 0x7f96dc06d360 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f96e4007fd0 tx=0x7f96e4009040 comp rx=0 tx=0).stop 2023-11-08T16:38:47.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.214+0000 7f96f5385640 1 -- 172.21.15.81:0/2374889823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 msgr2=0x7f96f01563a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.214+0000 7f96f5385640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 0x7f96f01563a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f96e000e7e0 tx=0x7f96e000eca0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.214+0000 7f96f5385640 1 -- 172.21.15.81:0/2374889823 shutdown_connections 2023-11-08T16:38:47.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.214+0000 7f96f5385640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f96dc06aea0 0x7f96dc06d360 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.214+0000 7f96f5385640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f96f00c7cc0 0x7f96f00b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.214+0000 7f96f5385640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f96f00c8650 0x7f96f00b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.214+0000 7f96f5385640 1 --2- 172.21.15.81:0/2374889823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f96d80051d0 0x7f96f01563a0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.214+0000 7f96f5385640 1 -- 172.21.15.81:0/2374889823 >> 172.21.15.81:0/2374889823 conn(0x7f96f006c040 msgr2=0x7f96f015c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:47.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.214+0000 7f96f5385640 1 -- 172.21.15.81:0/2374889823 shutdown_connections 2023-11-08T16:38:47.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.215+0000 7f96f5385640 1 -- 172.21.15.81:0/2374889823 wait complete. 2023-11-08T16:38:47.234 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:38:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.450+0000 7f363cca2640 1 Processor -- start 2023-11-08T16:38:47.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.450+0000 7f363cca2640 1 -- start start 2023-11-08T16:38:47.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.452+0000 7f363cca2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f36380ce0d0 0x7f36380caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.452+0000 7f363cca2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f363815f280 0x7f36381516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.452+0000 7f363cca2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363806fa30 con 0x7f363815bfc0 2023-11-08T16:38:47.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.452+0000 7f363cca2640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36380cda10 con 0x7f36380ce0d0 2023-11-08T16:38:47.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.452+0000 7f363cca2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36380ce4b0 con 0x7f363815f280 2023-11-08T16:38:47.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.452+0000 7f363659c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f36380ce0d0 0x7f36380caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.452+0000 7f363659c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f36380ce0d0 0x7f36380caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60372/0 (socket says 172.21.15.81:60372) 2023-11-08T16:38:47.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.452+0000 7f363659c640 1 -- 172.21.15.81:0/933689662 learned_addr learned my addr 172.21.15.81:0/933689662 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:47.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.453+0000 7f3636d9d640 1 --2- 172.21.15.81:0/933689662 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f363815f280 0x7f36381516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.453+0000 7f363659c640 1 -- 172.21.15.81:0/933689662 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f363815f280 msgr2=0x7f36381516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:47.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.453+0000 7f363659c640 1 --2- 172.21.15.81:0/933689662 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f363815f280 0x7f36381516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.453+0000 7f363659c640 1 -- 172.21.15.81:0/933689662 >> v1:172.21.15.81:6789/0 conn(0x7f363815bfc0 legacy=0x7f363815eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.453+0000 7f363659c640 1 -- 172.21.15.81:0/933689662 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363806fca0 con 0x7f36380ce0d0 2023-11-08T16:38:47.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.454+0000 7f363659c640 1 --2- 172.21.15.81:0/933689662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36380ce0d0 0x7f36380caf20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3628009870 tx=0x7f362802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e82a651ece1d3b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:47.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.454+0000 7f3635d9b640 1 -- 172.21.15.81:0/933689662 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628030840 con 0x7f36380ce0d0 2023-11-08T16:38:47.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.454+0000 7f3635d9b640 1 -- 172.21.15.81:0/933689662 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36280309a0 con 0x7f36380ce0d0 2023-11-08T16:38:47.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.454+0000 7f3635d9b640 1 -- 172.21.15.81:0/933689662 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628030d30 con 0x7f36380ce0d0 2023-11-08T16:38:47.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.455+0000 7f363cca2640 1 -- 172.21.15.81:0/933689662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36380ce0d0 msgr2=0x7f36380caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.455+0000 7f363cca2640 1 --2- 172.21.15.81:0/933689662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36380ce0d0 0x7f36380caf20 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f3628009870 tx=0x7f362802b980 comp rx=0 tx=0).stop 2023-11-08T16:38:47.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.455+0000 7f363cca2640 1 -- 172.21.15.81:0/933689662 shutdown_connections 2023-11-08T16:38:47.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.455+0000 7f363cca2640 1 --2- 172.21.15.81:0/933689662 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f363815f280 0x7f36381516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.455+0000 7f363cca2640 1 --2- 172.21.15.81:0/933689662 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36380ce0d0 0x7f36380caf20 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.455+0000 7f363cca2640 1 -- 172.21.15.81:0/933689662 >> 172.21.15.81:0/933689662 conn(0x7f36380cb580 msgr2=0x7f36380cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:47.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.455+0000 7f363cca2640 1 -- 172.21.15.81:0/933689662 shutdown_connections 2023-11-08T16:38:47.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.455+0000 7f363cca2640 1 -- 172.21.15.81:0/933689662 wait complete. 2023-11-08T16:38:47.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.457+0000 7f363cca2640 1 Processor -- start 2023-11-08T16:38:47.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.457+0000 7f363cca2640 1 -- start start 2023-11-08T16:38:47.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.458+0000 7f363cca2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36380ce0d0 0x7f36380aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.458+0000 7f363cca2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f363815bfc0 0x7f36380c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.458+0000 7f363cca2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f363815f280 0x7f36380c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.458+0000 7f363cca2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36380c2940 con 0x7f36380ce0d0 2023-11-08T16:38:47.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.458+0000 7f363cca2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36380b2860 con 0x7f363815f280 2023-11-08T16:38:47.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.458+0000 7f363cca2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36380b2ab0 con 0x7f363815bfc0 2023-11-08T16:38:47.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.458+0000 7f363659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36380ce0d0 0x7f36380aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.458+0000 7f363659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36380ce0d0 0x7f36380aef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60386/0 (socket says 172.21.15.81:60386) 2023-11-08T16:38:47.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.458+0000 7f363659c640 1 -- 172.21.15.81:0/4076069883 learned_addr learned my addr 172.21.15.81:0/4076069883 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:47.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.459+0000 7f363659c640 1 -- 172.21.15.81:0/4076069883 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f363815bfc0 msgr2=0x7f36380c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.459+0000 7f363659c640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f363815bfc0 0x7f36380c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.459+0000 7f363659c640 1 -- 172.21.15.81:0/4076069883 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f363815f280 msgr2=0x7f36380c34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.459+0000 7f363659c640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f363815f280 0x7f36380c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.459+0000 7f363659c640 1 -- 172.21.15.81:0/4076069883 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36280094d0 con 0x7f36380ce0d0 2023-11-08T16:38:47.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.459+0000 7f363659c640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36380ce0d0 0x7f36380aef70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3628000c00 tx=0x7f3628004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:47.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.459+0000 7f362effd640 1 -- 172.21.15.81:0/4076069883 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628039a90 con 0x7f36380ce0d0 2023-11-08T16:38:47.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.459+0000 7f362effd640 1 -- 172.21.15.81:0/4076069883 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3628039bf0 con 0x7f36380ce0d0 2023-11-08T16:38:47.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.459+0000 7f362effd640 1 -- 172.21.15.81:0/4076069883 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3628043410 con 0x7f36380ce0d0 2023-11-08T16:38:47.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.459+0000 7f363cca2640 1 -- 172.21.15.81:0/4076069883 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36380b2d90 con 0x7f36380ce0d0 2023-11-08T16:38:47.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.459+0000 7f363cca2640 1 -- 172.21.15.81:0/4076069883 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36380b3350 con 0x7f36380ce0d0 2023-11-08T16:38:47.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.461+0000 7f363cca2640 1 -- 172.21.15.81:0/4076069883 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363804ece0 con 0x7f36380ce0d0 2023-11-08T16:38:47.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.461+0000 7f362effd640 1 -- 172.21.15.81:0/4076069883 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f362802f390 con 0x7f36380ce0d0 2023-11-08T16:38:47.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.462+0000 7f362effd640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f361806ae70 0x7f361806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.462+0000 7f362effd640 1 -- 172.21.15.81:0/4076069883 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f36280abb80 con 0x7f36380ce0d0 2023-11-08T16:38:47.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.463+0000 7f362ffff640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f361806ae70 0x7f361806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.463+0000 7f362ffff640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f361806ae70 0x7f361806d330 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3620000f60 tx=0x7f36200079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:47.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.467+0000 7f362effd640 1 -- 172.21.15.81:0/4076069883 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36280768c0 con 0x7f36380ce0d0 2023-11-08T16:38:47.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.655+0000 7f363cca2640 1 -- 172.21.15.81:0/4076069883 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36380ce6b0 con 0x7f36380ce0d0 2023-11-08T16:38:47.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.655+0000 7f362effd640 1 -- 172.21.15.81:0/4076069883 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3628048020 con 0x7f36380ce0d0 2023-11-08T16:38:47.657 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:38:47.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.659+0000 7f363cca2640 1 -- 172.21.15.81:0/4076069883 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f361806ae70 msgr2=0x7f361806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.659+0000 7f363cca2640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f361806ae70 0x7f361806d330 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3620000f60 tx=0x7f36200079b0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.659+0000 7f363cca2640 1 -- 172.21.15.81:0/4076069883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36380ce0d0 msgr2=0x7f36380aef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.659+0000 7f363cca2640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36380ce0d0 0x7f36380aef70 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f3628000c00 tx=0x7f3628004730 comp rx=0 tx=0).stop 2023-11-08T16:38:47.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.659+0000 7f363cca2640 1 -- 172.21.15.81:0/4076069883 shutdown_connections 2023-11-08T16:38:47.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.659+0000 7f363cca2640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f361806ae70 0x7f361806d330 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.659+0000 7f363cca2640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f363815f280 0x7f36380c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.659+0000 7f363cca2640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f363815bfc0 0x7f36380c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.659+0000 7f363cca2640 1 --2- 172.21.15.81:0/4076069883 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36380ce0d0 0x7f36380aef70 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.659+0000 7f363cca2640 1 -- 172.21.15.81:0/4076069883 >> 172.21.15.81:0/4076069883 conn(0x7f36380cb580 msgr2=0x7f363815d460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:47.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.660+0000 7f363cca2640 1 -- 172.21.15.81:0/4076069883 shutdown_connections 2023-11-08T16:38:47.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.660+0000 7f363cca2640 1 -- 172.21.15.81:0/4076069883 wait complete. 2023-11-08T16:38:47.679 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:38:47.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.892+0000 7f2488e4d640 1 Processor -- start 2023-11-08T16:38:47.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.892+0000 7f2488e4d640 1 -- start start 2023-11-08T16:38:47.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.893+0000 7f2488e4d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24841612f0 0x7f248406c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.894+0000 7f2488e4d640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248415d680 con 0x7f24841616d0 2023-11-08T16:38:47.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.894+0000 7f2488e4d640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248415d960 con 0x7f248406cc70 2023-11-08T16:38:47.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.894+0000 7f2488e4d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248415d4a0 con 0x7f24841612f0 2023-11-08T16:38:47.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.894+0000 7f247bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f24841616d0 0x7f24840c9f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46136/0 (socket says 172.21.15.81:46136) 2023-11-08T16:38:47.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.894+0000 7f247bfff640 1 -- 172.21.15.81:0/4267701540 learned_addr learned my addr 172.21.15.81:0/4267701540 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:47.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.894+0000 7f24837fe640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24841612f0 0x7f248406c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.894+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325403715 0 0) 0x7f248415d680 con 0x7f24841616d0 2023-11-08T16:38:47.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2470003520 con 0x7f24841616d0 2023-11-08T16:38:47.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4159005675 0 0) 0x7f2470003520 con 0x7f24841616d0 2023-11-08T16:38:47.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24841612f0 msgr2=0x7f248406c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:47.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24841612f0 0x7f248406c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 >> v1:172.21.15.93:6789/0 conn(0x7f248406cc70 legacy=0x7f24840cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248415dc40 con 0x7f24841616d0 2023-11-08T16:38:47.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f246c0017f0 con 0x7f24841616d0 2023-11-08T16:38:47.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 >> v1:172.21.15.81:6789/0 conn(0x7f24841616d0 legacy=0x7f24840c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24841612f0 0x7f24700039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2470003f20 0x7f24700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2470004d00 0x7f2470005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248415d4a0 con 0x7f2470004d00 2023-11-08T16:38:47.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247000ba60 con 0x7f2470003f20 2023-11-08T16:38:47.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247000bc20 con 0x7f24841612f0 2023-11-08T16:38:47.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.895+0000 7f2483fff640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2470004d00 0x7f2470005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.896+0000 7f24837fe640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24841612f0 0x7f24700039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.896+0000 7f247bfff640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2470003f20 0x7f24700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.896+0000 7f2483fff640 1 -- 172.21.15.81:0/4267701540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24841612f0 msgr2=0x7f24700039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:47.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.896+0000 7f2483fff640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24841612f0 0x7f24700039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.896+0000 7f2483fff640 1 -- 172.21.15.81:0/4267701540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2470003f20 msgr2=0x7f24700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:47.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.896+0000 7f2483fff640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2470003f20 0x7f24700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.896+0000 7f2483fff640 1 -- 172.21.15.81:0/4267701540 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2470003520 con 0x7f2470004d00 2023-11-08T16:38:47.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.897+0000 7f2483fff640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2470004d00 0x7f2470005180 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f247c001e50 tx=0x7f24700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8410c526c2ad04 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:47.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.897+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c003b70 con 0x7f2470004d00 2023-11-08T16:38:47.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.897+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c003cd0 con 0x7f2470004d00 2023-11-08T16:38:47.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.897+0000 7f2482ffd640 1 -- 172.21.15.81:0/4267701540 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c005310 con 0x7f2470004d00 2023-11-08T16:38:47.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.898+0000 7f2488e4d640 1 -- 172.21.15.81:0/4267701540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2470004d00 msgr2=0x7f2470005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.898+0000 7f2488e4d640 1 --2- 172.21.15.81:0/4267701540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2470004d00 0x7f2470005180 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f247c001e50 tx=0x7f24700057f0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.898+0000 7f2483fff640 1 -- 172.21.15.81:0/4267701540 reap_dead start 2023-11-08T16:38:47.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.898+0000 7f2488e4d640 1 -- 172.21.15.81:0/4267701540 shutdown_connections 2023-11-08T16:38:47.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.898+0000 7f2488e4d640 1 -- 172.21.15.81:0/4267701540 >> 172.21.15.81:0/4267701540 conn(0x7f24840ce8d0 msgr2=0x7f24840cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:47.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.898+0000 7f2488e4d640 1 -- 172.21.15.81:0/4267701540 shutdown_connections 2023-11-08T16:38:47.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.898+0000 7f2488e4d640 1 -- 172.21.15.81:0/4267701540 wait complete. 2023-11-08T16:38:47.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.900+0000 7f2488e4d640 1 Processor -- start 2023-11-08T16:38:47.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.900+0000 7f2488e4d640 1 -- start start 2023-11-08T16:38:47.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f2488e4d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2470004d00 0x7f2484158520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f2488e4d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24840c6870 0x7f24840b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f2488e4d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24840b4360 0x7f248406f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f2488e4d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24840b4970 con 0x7f24840c6870 2023-11-08T16:38:47.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f2488e4d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24840c5ab0 con 0x7f2470004d00 2023-11-08T16:38:47.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f2488e4d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24840c5d90 con 0x7f24840b4360 2023-11-08T16:38:47.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f247bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24840c6870 0x7f24840b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f247bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24840c6870 0x7f24840b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60414/0 (socket says 172.21.15.81:60414) 2023-11-08T16:38:47.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f247bfff640 1 -- 172.21.15.81:0/3977743035 learned_addr learned my addr 172.21.15.81:0/3977743035 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:47.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f247bfff640 1 -- 172.21.15.81:0/3977743035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24840b4360 msgr2=0x7f248406f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f2483fff640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24840b4360 0x7f248406f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f247bfff640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24840b4360 0x7f248406f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f247bfff640 1 -- 172.21.15.81:0/3977743035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2470004d00 msgr2=0x7f2484158520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:47.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.901+0000 7f24837fe640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2470004d00 0x7f2484158520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.902+0000 7f247bfff640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2470004d00 0x7f2484158520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:47.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.902+0000 7f247bfff640 1 -- 172.21.15.81:0/3977743035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24840c6070 con 0x7f24840c6870 2023-11-08T16:38:47.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.902+0000 7f24837fe640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2470004d00 0x7f2484158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:47.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.902+0000 7f247bfff640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24840c6870 0x7f24840b3e20 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f246c00e780 tx=0x7f246c00ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:47.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.902+0000 7f2480ff9640 1 -- 172.21.15.81:0/3977743035 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c00ee90 con 0x7f24840c6870 2023-11-08T16:38:47.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.902+0000 7f2480ff9640 1 -- 172.21.15.81:0/3977743035 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246c0072e0 con 0x7f24840c6870 2023-11-08T16:38:47.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.902+0000 7f2488e4d640 1 -- 172.21.15.81:0/3977743035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24840c6350 con 0x7f24840c6870 2023-11-08T16:38:47.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.902+0000 7f2480ff9640 1 -- 172.21.15.81:0/3977743035 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246c0057a0 con 0x7f24840c6870 2023-11-08T16:38:47.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.902+0000 7f2488e4d640 1 -- 172.21.15.81:0/3977743035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24840b1bf0 con 0x7f24840c6870 2023-11-08T16:38:47.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.904+0000 7f2488e4d640 1 -- 172.21.15.81:0/3977743035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24840b4740 con 0x7f24840c6870 2023-11-08T16:38:47.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.904+0000 7f2480ff9640 1 -- 172.21.15.81:0/3977743035 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f246c005900 con 0x7f24840c6870 2023-11-08T16:38:47.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.906+0000 7f2480ff9640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f246406ae70 0x7f246406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:47.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.906+0000 7f2480ff9640 1 -- 172.21.15.81:0/3977743035 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f246c089a90 con 0x7f24840c6870 2023-11-08T16:38:47.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.906+0000 7f24837fe640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f246406ae70 0x7f246406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:47.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.907+0000 7f24837fe640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f246406ae70 0x7f246406d330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2474009a00 tx=0x7f2474008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:47.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:47.911+0000 7f2480ff9640 1 -- 172.21.15.81:0/3977743035 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f246c0547d0 con 0x7f24840c6870 2023-11-08T16:38:48.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.102+0000 7f2488e4d640 1 -- 172.21.15.81:0/3977743035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f248404ece0 con 0x7f24840c6870 2023-11-08T16:38:48.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.103+0000 7f2480ff9640 1 -- 172.21.15.81:0/3977743035 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v90) v1 ==== 106+0+11 (secure 0 0 0) 0x7f246c01a310 con 0x7f24840c6870 2023-11-08T16:38:48.105 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:38:48.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 -- 172.21.15.81:0/3977743035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f246406ae70 msgr2=0x7f246406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:48.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f246406ae70 0x7f246406d330 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2474009a00 tx=0x7f2474008040 comp rx=0 tx=0).stop 2023-11-08T16:38:48.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 -- 172.21.15.81:0/3977743035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24840c6870 msgr2=0x7f24840b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:48.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24840c6870 0x7f24840b3e20 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f246c00e780 tx=0x7f246c00ec40 comp rx=0 tx=0).stop 2023-11-08T16:38:48.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 -- 172.21.15.81:0/3977743035 shutdown_connections 2023-11-08T16:38:48.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f246406ae70 0x7f246406d330 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:48.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24840b4360 0x7f248406f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:48.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24840c6870 0x7f24840b3e20 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:48.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 --2- 172.21.15.81:0/3977743035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2470004d00 0x7f2484158520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:48.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 -- 172.21.15.81:0/3977743035 >> 172.21.15.81:0/3977743035 conn(0x7f24840ce8d0 msgr2=0x7f248415c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:48.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 -- 172.21.15.81:0/3977743035 shutdown_connections 2023-11-08T16:38:48.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.106+0000 7f2488e4d640 1 -- 172.21.15.81:0/3977743035 wait complete. 2023-11-08T16:38:48.127 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:38:48.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.339+0000 7f09c01cc640 1 Processor -- start 2023-11-08T16:38:48.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.340+0000 7f09c01cc640 1 -- start start 2023-11-08T16:38:48.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.341+0000 7f09c01cc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f09b80d01d0 0x7f09b8167f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:48.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.341+0000 7f09c01cc640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b80b6100 con 0x7f09b80d2b70 2023-11-08T16:38:48.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.341+0000 7f09c01cc640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b80d1b50 con 0x7f09b815d310 2023-11-08T16:38:48.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.341+0000 7f09c01cc640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b80b6480 con 0x7f09b80d01d0 2023-11-08T16:38:48.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.341+0000 7f09be769640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f09b80d01d0 0x7f09b8167f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:48.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.341+0000 7f09be769640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f09b80d01d0 0x7f09b8167f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60430/0 (socket says 172.21.15.81:60430) 2023-11-08T16:38:48.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.341+0000 7f09be769640 1 -- 172.21.15.81:0/590846672 learned_addr learned my addr 172.21.15.81:0/590846672 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:48.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.342+0000 7f09be769640 1 -- 172.21.15.81:0/590846672 >> v1:172.21.15.93:6789/0 conn(0x7f09b815d310 legacy=0x7f09b80d1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:48.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.342+0000 7f09be769640 1 -- 172.21.15.81:0/590846672 >> v1:172.21.15.81:6789/0 conn(0x7f09b80d2b70 legacy=0x7f09b80cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:48.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.342+0000 7f09be769640 1 -- 172.21.15.81:0/590846672 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b80d1e30 con 0x7f09b80d01d0 2023-11-08T16:38:48.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.343+0000 7f09be769640 1 --2- 172.21.15.81:0/590846672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09b80d01d0 0x7f09b8167f20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009e50 tx=0x7f09b002c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7209c0bff92cd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:48.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.343+0000 7f09bcf66640 1 -- 172.21.15.81:0/590846672 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b002cbb0 con 0x7f09b80d01d0 2023-11-08T16:38:48.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.343+0000 7f09bcf66640 1 -- 172.21.15.81:0/590846672 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b002cd10 con 0x7f09b80d01d0 2023-11-08T16:38:48.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.343+0000 7f09bcf66640 1 -- 172.21.15.81:0/590846672 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0034610 con 0x7f09b80d01d0 2023-11-08T16:38:48.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.344+0000 7f09c01cc640 1 -- 172.21.15.81:0/590846672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09b80d01d0 msgr2=0x7f09b8167f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:48.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.344+0000 7f09c01cc640 1 --2- 172.21.15.81:0/590846672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09b80d01d0 0x7f09b8167f20 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009e50 tx=0x7f09b002c010 comp rx=0 tx=0).stop 2023-11-08T16:38:48.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.344+0000 7f09c01cc640 1 -- 172.21.15.81:0/590846672 shutdown_connections 2023-11-08T16:38:48.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.344+0000 7f09c01cc640 1 --2- 172.21.15.81:0/590846672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09b80d01d0 0x7f09b8167f20 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:48.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.344+0000 7f09c01cc640 1 -- 172.21.15.81:0/590846672 >> 172.21.15.81:0/590846672 conn(0x7f09b806aac0 msgr2=0x7f09b806aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:48.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.344+0000 7f09c01cc640 1 -- 172.21.15.81:0/590846672 shutdown_connections 2023-11-08T16:38:48.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.344+0000 7f09c01cc640 1 -- 172.21.15.81:0/590846672 wait complete. 2023-11-08T16:38:48.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.346+0000 7f09c01cc640 1 Processor -- start 2023-11-08T16:38:48.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.346+0000 7f09c01cc640 1 -- start start 2023-11-08T16:38:48.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09c01cc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09b80d01d0 0x7f09b815ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:48.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09c01cc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09b80d2b70 0x7f09b80c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:48.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09c01cc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09b815d310 0x7f09b80c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:48.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09c01cc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80c8b80 con 0x7f09b80d2b70 2023-11-08T16:38:48.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09c01cc640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80c7c60 con 0x7f09b80d01d0 2023-11-08T16:38:48.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09c01cc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80c7f00 con 0x7f09b815d310 2023-11-08T16:38:48.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09bd767640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09b80d2b70 0x7f09b80c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:48.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09bd767640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09b80d2b70 0x7f09b80c9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60440/0 (socket says 172.21.15.81:60440) 2023-11-08T16:38:48.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09bd767640 1 -- 172.21.15.81:0/925117033 learned_addr learned my addr 172.21.15.81:0/925117033 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:48.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09be769640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09b815d310 0x7f09b80c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:48.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09bdf68640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09b80d01d0 0x7f09b815ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:48.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09bd767640 1 -- 172.21.15.81:0/925117033 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09b815d310 msgr2=0x7f09b80c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:48.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09bd767640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09b815d310 0x7f09b80c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:48.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09bd767640 1 -- 172.21.15.81:0/925117033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09b80d01d0 msgr2=0x7f09b815ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:48.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09bd767640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09b80d01d0 0x7f09b815ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:48.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09bd767640 1 -- 172.21.15.81:0/925117033 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0009b30 con 0x7f09b80d2b70 2023-11-08T16:38:48.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.347+0000 7f09bdf68640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09b80d01d0 0x7f09b815ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:48.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.348+0000 7f09bd767640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09b80d2b70 0x7f09b80c9160 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c450 tx=0x7f09a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:48.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.348+0000 7f09ae7fc640 1 -- 172.21.15.81:0/925117033 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a80052d0 con 0x7f09b80d2b70 2023-11-08T16:38:48.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.348+0000 7f09ae7fc640 1 -- 172.21.15.81:0/925117033 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a8005430 con 0x7f09b80d2b70 2023-11-08T16:38:48.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.348+0000 7f09c01cc640 1 -- 172.21.15.81:0/925117033 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b80c81e0 con 0x7f09b80d2b70 2023-11-08T16:38:48.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.348+0000 7f09ae7fc640 1 -- 172.21.15.81:0/925117033 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a8012870 con 0x7f09b80d2b70 2023-11-08T16:38:48.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.348+0000 7f09c01cc640 1 -- 172.21.15.81:0/925117033 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b80c87a0 con 0x7f09b80d2b70 2023-11-08T16:38:48.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.349+0000 7f09c01cc640 1 -- 172.21.15.81:0/925117033 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b80cb1a0 con 0x7f09b80d2b70 2023-11-08T16:38:48.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.350+0000 7f09ae7fc640 1 -- 172.21.15.81:0/925117033 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f09a80129d0 con 0x7f09b80d2b70 2023-11-08T16:38:48.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.351+0000 7f09ae7fc640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099c06ae70 0x7f099c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:48.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.351+0000 7f09ae7fc640 1 -- 172.21.15.81:0/925117033 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f09a8089430 con 0x7f09b80d2b70 2023-11-08T16:38:48.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.351+0000 7f09bdf68640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099c06ae70 0x7f099c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:48.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.352+0000 7f09bdf68640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099c06ae70 0x7f099c06d330 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f09b400a830 tx=0x7f09b4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:48.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.355+0000 7f09ae7fc640 1 -- 172.21.15.81:0/925117033 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f09a8054200 con 0x7f09b80d2b70 2023-11-08T16:38:48.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.557+0000 7f09c01cc640 1 -- 172.21.15.81:0/925117033 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f09b80743f0 con 0x7f09b80d2b70 2023-11-08T16:38:48.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.558+0000 7f09ae7fc640 1 -- 172.21.15.81:0/925117033 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v90) v1 ==== 80+0+364 (secure 0 0 0) 0x7f09a8057820 con 0x7f09b80d2b70 2023-11-08T16:38:48.560 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6836/1764545681 2023-11-09T16:38:06.974694+0000 2023-11-08T16:38:48.560 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/1764545681 2023-11-09T16:38:06.974694+0000 2023-11-08T16:38:48.560 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/457413856 2023-11-09T16:38:00.541280+0000 2023-11-08T16:38:48.561 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6836/457413856 2023-11-09T16:38:00.541280+0000 2023-11-08T16:38:48.561 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/2162767469 2023-11-09T16:37:58.508547+0000 2023-11-08T16:38:48.561 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6836/2162767469 2023-11-09T16:37:58.508547+0000 2023-11-08T16:38:48.561 INFO:teuthology.orchestra.run.smithi081.stderr:listed 6 entries 2023-11-08T16:38:48.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.560+0000 7f09c01cc640 1 -- 172.21.15.81:0/925117033 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099c06ae70 msgr2=0x7f099c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:48.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.560+0000 7f09c01cc640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099c06ae70 0x7f099c06d330 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f09b400a830 tx=0x7f09b4008040 comp rx=0 tx=0).stop 2023-11-08T16:38:48.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.560+0000 7f09c01cc640 1 -- 172.21.15.81:0/925117033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09b80d2b70 msgr2=0x7f09b80c9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:48.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.561+0000 7f09c01cc640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09b80d2b70 0x7f09b80c9160 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c450 tx=0x7f09a800c910 comp rx=0 tx=0).stop 2023-11-08T16:38:48.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.561+0000 7f09c01cc640 1 -- 172.21.15.81:0/925117033 shutdown_connections 2023-11-08T16:38:48.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.561+0000 7f09c01cc640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099c06ae70 0x7f099c06d330 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:48.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.561+0000 7f09c01cc640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09b815d310 0x7f09b80c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:48.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.561+0000 7f09c01cc640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09b80d2b70 0x7f09b80c9160 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:48.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.561+0000 7f09c01cc640 1 --2- 172.21.15.81:0/925117033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09b80d01d0 0x7f09b815ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:48.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.561+0000 7f09c01cc640 1 -- 172.21.15.81:0/925117033 >> 172.21.15.81:0/925117033 conn(0x7f09b806aac0 msgr2=0x7f09b80cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:48.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.561+0000 7f09c01cc640 1 -- 172.21.15.81:0/925117033 shutdown_connections 2023-11-08T16:38:48.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:48.561+0000 7f09c01cc640 1 -- 172.21.15.81:0/925117033 wait complete. 2023-11-08T16:38:48.582 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:38:48.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:48.583 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:38:48.676 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[116226]: fuse finished with error 0 and tester_r 0 2023-11-08T16:38:48.682 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:38:48.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:48.683 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:38:48.718 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:38:48.760 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:38:48.761 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:38:48.767 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:38:48.818 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:38:48.819 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:38:48.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:48.820 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-08T16:38:48.912 INFO:teuthology.orchestra.run.smithi093.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-08T16:38:48.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:38:48.914 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:38:48.914 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:38:48.976 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:38:48.977 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:38:48.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:48.978 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-08T16:38:49.029 INFO:teuthology.orchestra.run.smithi081.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-08T16:38:49.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:38:49.031 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:38:49.031 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:38:49.090 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:38:49.091 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:38:49.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:38:49.091 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-08T16:38:49.145 INFO:teuthology.orchestra.run.smithi093.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-08T16:38:49.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:38:49.147 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:49.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.365+0000 7f0900210640 1 Processor -- start 2023-11-08T16:38:49.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.365+0000 7f0900210640 1 -- start start 2023-11-08T16:38:49.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.366+0000 7f0900210640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 0x7f08f80ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.366+0000 7f0900210640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f08f815a680 0x7f08f815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.366+0000 7f0900210640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f806b630 con 0x7f08f815b020 2023-11-08T16:38:49.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.366+0000 7f0900210640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f80b1860 con 0x7f08f815a680 2023-11-08T16:38:49.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.366+0000 7f0900210640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f80ca9c0 con 0x7f08f80ade50 2023-11-08T16:38:49.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.367+0000 7f08fdfac640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 0x7f08f80ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:49.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.367+0000 7f08fdfac640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 0x7f08f80ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:57908/0 (socket says 172.21.15.81:57908) 2023-11-08T16:38:49.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.367+0000 7f08fdfac640 1 -- 172.21.15.81:0/2713014559 learned_addr learned my addr 172.21.15.81:0/2713014559 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:49.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.367+0000 7f08fd7ab640 1 --2- 172.21.15.81:0/2713014559 >> v2:172.21.15.93:3300/0 conn(0x7f08f815a680 0x7f08f815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:49.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.368+0000 7f08fdfac640 1 -- 172.21.15.81:0/2713014559 >> v2:172.21.15.93:3300/0 conn(0x7f08f815a680 msgr2=0x7f08f815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:49.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.368+0000 7f08fdfac640 1 --2- 172.21.15.81:0/2713014559 >> v2:172.21.15.93:3300/0 conn(0x7f08f815a680 0x7f08f815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.368+0000 7f08fdfac640 1 -- 172.21.15.81:0/2713014559 >> v1:172.21.15.93:6789/0 conn(0x7f08f815b020 legacy=0x7f08f80c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:49.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.368+0000 7f08fdfac640 1 -- 172.21.15.81:0/2713014559 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f80cabb0 con 0x7f08f80ade50 2023-11-08T16:38:49.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.369+0000 7f08fdfac640 1 --2- 172.21.15.81:0/2713014559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 0x7f08f80ca480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009870 tx=0x7f08f402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15442d9a0940694 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:49.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.369+0000 7f08fcfaa640 1 -- 172.21.15.81:0/2713014559 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4030840 con 0x7f08f80ade50 2023-11-08T16:38:49.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.369+0000 7f08fcfaa640 1 -- 172.21.15.81:0/2713014559 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f40309a0 con 0x7f08f80ade50 2023-11-08T16:38:49.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.369+0000 7f08fcfaa640 1 -- 172.21.15.81:0/2713014559 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4030d30 con 0x7f08f80ade50 2023-11-08T16:38:49.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.370+0000 7f0900210640 1 -- 172.21.15.81:0/2713014559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 msgr2=0x7f08f80ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:49.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.370+0000 7f0900210640 1 --2- 172.21.15.81:0/2713014559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 0x7f08f80ca480 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f08f4009870 tx=0x7f08f402b980 comp rx=0 tx=0).stop 2023-11-08T16:38:49.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.370+0000 7f0900210640 1 -- 172.21.15.81:0/2713014559 shutdown_connections 2023-11-08T16:38:49.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.370+0000 7f0900210640 1 --2- 172.21.15.81:0/2713014559 >> v2:172.21.15.93:3300/0 conn(0x7f08f815a680 0x7f08f815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.370+0000 7f0900210640 1 --2- 172.21.15.81:0/2713014559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 0x7f08f80ca480 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.370+0000 7f0900210640 1 -- 172.21.15.81:0/2713014559 >> 172.21.15.81:0/2713014559 conn(0x7f08f80cb490 msgr2=0x7f08f80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:49.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.370+0000 7f0900210640 1 -- 172.21.15.81:0/2713014559 shutdown_connections 2023-11-08T16:38:49.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.370+0000 7f0900210640 1 -- 172.21.15.81:0/2713014559 wait complete. 2023-11-08T16:38:49.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.372+0000 7f0900210640 1 Processor -- start 2023-11-08T16:38:49.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.373+0000 7f0900210640 1 -- start start 2023-11-08T16:38:49.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.374+0000 7f0900210640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 0x7f08f81f2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.374+0000 7f0900210640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08f815a680 0x7f08f8157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.374+0000 7f0900210640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f815b020 0x7f08f80c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.374+0000 7f0900210640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f815c930 con 0x7f08f815b020 2023-11-08T16:38:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.374+0000 7f0900210640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f815cc10 con 0x7f08f815a680 2023-11-08T16:38:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.374+0000 7f0900210640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f815cef0 con 0x7f08f80ade50 2023-11-08T16:38:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.374+0000 7f08fe7ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f815b020 0x7f08f80c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.374+0000 7f08fe7ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f815b020 0x7f08f80c2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60454/0 (socket says 172.21.15.81:60454) 2023-11-08T16:38:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.374+0000 7f08fe7ad640 1 -- 172.21.15.81:0/3567727201 learned_addr learned my addr 172.21.15.81:0/3567727201 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:49.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.374+0000 7f08fdfac640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 0x7f08f81f2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:49.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.374+0000 7f08fd7ab640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08f815a680 0x7f08f8157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:49.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.375+0000 7f08fe7ad640 1 -- 172.21.15.81:0/3567727201 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 msgr2=0x7f08f81f2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:49.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.375+0000 7f08fe7ad640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 0x7f08f81f2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.375+0000 7f08fe7ad640 1 -- 172.21.15.81:0/3567727201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08f815a680 msgr2=0x7f08f8157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:49.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.375+0000 7f08fe7ad640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08f815a680 0x7f08f8157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.375+0000 7f08fe7ad640 1 -- 172.21.15.81:0/3567727201 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40094d0 con 0x7f08f815b020 2023-11-08T16:38:49.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.375+0000 7f08fe7ad640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f815b020 0x7f08f80c2ec0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f08f000c6b0 tx=0x7f08f000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:49.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.375+0000 7f08ee7fc640 1 -- 172.21.15.81:0/3567727201 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f0003b70 con 0x7f08f815b020 2023-11-08T16:38:49.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.375+0000 7f08ee7fc640 1 -- 172.21.15.81:0/3567727201 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f0003cf0 con 0x7f08f815b020 2023-11-08T16:38:49.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.375+0000 7f08ee7fc640 1 -- 172.21.15.81:0/3567727201 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f0003ea0 con 0x7f08f815b020 2023-11-08T16:38:49.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.375+0000 7f0900210640 1 -- 172.21.15.81:0/3567727201 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f815d1d0 con 0x7f08f815b020 2023-11-08T16:38:49.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.375+0000 7f0900210640 1 -- 172.21.15.81:0/3567727201 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f81f3060 con 0x7f08f815b020 2023-11-08T16:38:49.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.377+0000 7f0900210640 1 -- 172.21.15.81:0/3567727201 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f815d6f0 con 0x7f08f815b020 2023-11-08T16:38:49.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.377+0000 7f08ee7fc640 1 -- 172.21.15.81:0/3567727201 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f08f0005310 con 0x7f08f815b020 2023-11-08T16:38:49.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.378+0000 7f08ee7fc640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08dc06ae70 0x7f08dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.379+0000 7f08ee7fc640 1 -- 172.21.15.81:0/3567727201 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f08f0089730 con 0x7f08f815b020 2023-11-08T16:38:49.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.379+0000 7f08fdfac640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08dc06ae70 0x7f08dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:49.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.379+0000 7f08fdfac640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08dc06ae70 0x7f08dc06d330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f08f4036000 tx=0x7f08f4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:49.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.383+0000 7f08ee7fc640 1 -- 172.21.15.81:0/3567727201 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f08f815d6f0 con 0x7f08f815b020 2023-11-08T16:38:49.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.609+0000 7f0900210640 1 -- 172.21.15.81:0/3567727201 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08f815d6f0 con 0x7f08f815b020 2023-11-08T16:38:49.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.610+0000 7f08ee7fc640 1 -- 172.21.15.81:0/3567727201 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f08f815d6f0 con 0x7f08f815b020 2023-11-08T16:38:49.612 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:49.613 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":78,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1527116581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1527116581},{"type":"v1","addr":"172.21.15.93:6833","nonce":1527116581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2617548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2617548066},{"type":"v1","addr":"172.21.15.93:6835","nonce":2617548066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":7380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2591582634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2591582634},{"type":"v1","addr":"172.21.15.81:6835","nonce":2591582634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:09.004709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/1843050442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1843050442},{"type":"v1","addr":"172.21.15.81:6837","nonce":1843050442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:49.613 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 78 2023-11-08T16:38:49.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.613+0000 7f0900210640 1 -- 172.21.15.81:0/3567727201 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08dc06ae70 msgr2=0x7f08dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:49.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.613+0000 7f0900210640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08dc06ae70 0x7f08dc06d330 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f08f4036000 tx=0x7f08f4008030 comp rx=0 tx=0).stop 2023-11-08T16:38:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.613+0000 7f0900210640 1 -- 172.21.15.81:0/3567727201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f815b020 msgr2=0x7f08f80c2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.613+0000 7f0900210640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f815b020 0x7f08f80c2ec0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f08f000c6b0 tx=0x7f08f000cb70 comp rx=0 tx=0).stop 2023-11-08T16:38:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.613+0000 7f0900210640 1 -- 172.21.15.81:0/3567727201 shutdown_connections 2023-11-08T16:38:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.613+0000 7f0900210640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f08dc06ae70 0x7f08dc06d330 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.613+0000 7f0900210640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f08f815b020 0x7f08f80c2ec0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.613+0000 7f0900210640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f08f815a680 0x7f08f8157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.613+0000 7f0900210640 1 --2- 172.21.15.81:0/3567727201 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f08f80ade50 0x7f08f81f2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.613+0000 7f0900210640 1 -- 172.21.15.81:0/3567727201 >> 172.21.15.81:0/3567727201 conn(0x7f08f80cb490 msgr2=0x7f08f80ae560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:49.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.614+0000 7f0900210640 1 -- 172.21.15.81:0/3567727201 shutdown_connections 2023-11-08T16:38:49.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.614+0000 7f0900210640 1 -- 172.21.15.81:0/3567727201 wait complete. 2023-11-08T16:38:49.635 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:38:49.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.835+0000 7fb564ffb640 1 Processor -- start 2023-11-08T16:38:49.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.835+0000 7fb564ffb640 1 -- start start 2023-11-08T16:38:49.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.837+0000 7fb564ffb640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb5600adcd0 0x7fb5600ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.837+0000 7fb564ffb640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb5600cc5e0 0x7fb5600cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.837+0000 7fb564ffb640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5600c9a70 con 0x7fb5600ccfb0 2023-11-08T16:38:49.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.837+0000 7fb564ffb640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5600cc3e0 con 0x7fb5600cc5e0 2023-11-08T16:38:49.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.837+0000 7fb564ffb640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb56006d070 con 0x7fb5600adcd0 2023-11-08T16:38:49.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.837+0000 7fb557fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb5600cc5e0 0x7fb5600cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:49.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.837+0000 7fb55ed9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb5600ccfb0 0x7fb56015db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:46158/0 (socket says 172.21.15.81:46158) 2023-11-08T16:38:49.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.837+0000 7fb55ed9d640 1 -- 172.21.15.81:0/3748845187 learned_addr learned my addr 172.21.15.81:0/3748845187 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:49.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.837+0000 7fb55e59c640 1 --2- 172.21.15.81:0/3748845187 >> v2:172.21.15.93:3300/0 conn(0x7fb5600adcd0 0x7fb5600ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:49.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.838+0000 7fb55dd9b640 1 -- 172.21.15.81:0/3748845187 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 64326618 0 0) 0x7fb5600c9a70 con 0x7fb5600ccfb0 2023-11-08T16:38:49.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.838+0000 7fb55dd9b640 1 -- 172.21.15.81:0/3748845187 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb54c003520 con 0x7fb5600ccfb0 2023-11-08T16:38:49.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.838+0000 7fb557fff640 1 -- 172.21.15.81:0/3748845187 >> v2:172.21.15.93:3300/0 conn(0x7fb5600adcd0 msgr2=0x7fb5600ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:49.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.838+0000 7fb557fff640 1 --2- 172.21.15.81:0/3748845187 >> v2:172.21.15.93:3300/0 conn(0x7fb5600adcd0 0x7fb5600ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.838+0000 7fb557fff640 1 -- 172.21.15.81:0/3748845187 >> v1:172.21.15.81:6789/0 conn(0x7fb5600ccfb0 legacy=0x7fb56015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:49.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.838+0000 7fb557fff640 1 -- 172.21.15.81:0/3748845187 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb56006d350 con 0x7fb5600cc5e0 2023-11-08T16:38:49.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.839+0000 7fb557fff640 1 --2- 172.21.15.81:0/3748845187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5600cc5e0 0x7fb5600cca70 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fb560067770 tx=0x7fb54802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7649ff0e0f2d18 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:49.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.839+0000 7fb55dd9b640 1 -- 172.21.15.81:0/3748845187 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548031830 con 0x7fb5600cc5e0 2023-11-08T16:38:49.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.839+0000 7fb55dd9b640 1 -- 172.21.15.81:0/3748845187 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb548031990 con 0x7fb5600cc5e0 2023-11-08T16:38:49.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.839+0000 7fb55dd9b640 1 -- 172.21.15.81:0/3748845187 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb548031d20 con 0x7fb5600cc5e0 2023-11-08T16:38:49.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.840+0000 7fb564ffb640 1 -- 172.21.15.81:0/3748845187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5600cc5e0 msgr2=0x7fb5600cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:49.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.840+0000 7fb564ffb640 1 --2- 172.21.15.81:0/3748845187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5600cc5e0 0x7fb5600cca70 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7fb560067770 tx=0x7fb54802b8d0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.840+0000 7fb564ffb640 1 -- 172.21.15.81:0/3748845187 shutdown_connections 2023-11-08T16:38:49.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.840+0000 7fb564ffb640 1 --2- 172.21.15.81:0/3748845187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5600cc5e0 0x7fb5600cca70 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.840+0000 7fb564ffb640 1 --2- 172.21.15.81:0/3748845187 >> v2:172.21.15.93:3300/0 conn(0x7fb5600adcd0 0x7fb5600ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.840+0000 7fb564ffb640 1 -- 172.21.15.81:0/3748845187 >> 172.21.15.81:0/3748845187 conn(0x7fb5600cb490 msgr2=0x7fb5600cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:49.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.840+0000 7fb564ffb640 1 -- 172.21.15.81:0/3748845187 shutdown_connections 2023-11-08T16:38:49.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.840+0000 7fb564ffb640 1 -- 172.21.15.81:0/3748845187 wait complete. 2023-11-08T16:38:49.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.842+0000 7fb564ffb640 1 Processor -- start 2023-11-08T16:38:49.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.842+0000 7fb564ffb640 1 -- start start 2023-11-08T16:38:49.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb564ffb640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5600adcd0 0x7fb5600b6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb564ffb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5600cc5e0 0x7fb560155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb564ffb640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb5600ccfb0 0x7fb5600b4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb564ffb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600b6470 con 0x7fb5600cc5e0 2023-11-08T16:38:49.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb564ffb640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600b5860 con 0x7fb5600adcd0 2023-11-08T16:38:49.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb564ffb640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600b5b40 con 0x7fb5600ccfb0 2023-11-08T16:38:49.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb557fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5600cc5e0 0x7fb560155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:49.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb557fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5600cc5e0 0x7fb560155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60470/0 (socket says 172.21.15.81:60470) 2023-11-08T16:38:49.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb557fff640 1 -- 172.21.15.81:0/2395772764 learned_addr learned my addr 172.21.15.81:0/2395772764 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:49.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb557fff640 1 -- 172.21.15.81:0/2395772764 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb5600ccfb0 msgr2=0x7fb5600b4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:49.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb557fff640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb5600ccfb0 0x7fb5600b4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb557fff640 1 -- 172.21.15.81:0/2395772764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5600adcd0 msgr2=0x7fb5600b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:49.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb557fff640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5600adcd0 0x7fb5600b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:49.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.843+0000 7fb557fff640 1 -- 172.21.15.81:0/2395772764 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb548009400 con 0x7fb5600cc5e0 2023-11-08T16:38:49.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.844+0000 7fb557fff640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5600cc5e0 0x7fb560155eb0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fb548009890 tx=0x7fb5480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:49.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.844+0000 7fb556ffd640 1 -- 172.21.15.81:0/2395772764 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54803a6f0 con 0x7fb5600cc5e0 2023-11-08T16:38:49.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.844+0000 7fb556ffd640 1 -- 172.21.15.81:0/2395772764 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54803a870 con 0x7fb5600cc5e0 2023-11-08T16:38:49.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.844+0000 7fb556ffd640 1 -- 172.21.15.81:0/2395772764 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54803ac20 con 0x7fb5600cc5e0 2023-11-08T16:38:49.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.844+0000 7fb564ffb640 1 -- 172.21.15.81:0/2395772764 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5600b5e20 con 0x7fb5600cc5e0 2023-11-08T16:38:49.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.844+0000 7fb564ffb640 1 -- 172.21.15.81:0/2395772764 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5600b6e40 con 0x7fb5600cc5e0 2023-11-08T16:38:49.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.845+0000 7fb564ffb640 1 -- 172.21.15.81:0/2395772764 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb560072170 con 0x7fb5600cc5e0 2023-11-08T16:38:49.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.846+0000 7fb556ffd640 1 -- 172.21.15.81:0/2395772764 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb548043410 con 0x7fb5600cc5e0 2023-11-08T16:38:49.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.847+0000 7fb556ffd640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb54006ae70 0x7fb54006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:49.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.847+0000 7fb556ffd640 1 -- 172.21.15.81:0/2395772764 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fb5480ab600 con 0x7fb5600cc5e0 2023-11-08T16:38:49.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.847+0000 7fb55e59c640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb54006ae70 0x7fb54006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:49.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.848+0000 7fb55e59c640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb54006ae70 0x7fb54006d330 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb550006fd0 tx=0x7fb550008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:49.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:49.851+0000 7fb556ffd640 1 -- 172.21.15.81:0/2395772764 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb548076470 con 0x7fb5600cc5e0 2023-11-08T16:38:50.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.042+0000 7fb564ffb640 1 -- 172.21.15.81:0/2395772764 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb56004ece0 con 0x7fb5600cc5e0 2023-11-08T16:38:50.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.045+0000 7fb556ffd640 1 -- 172.21.15.81:0/2395772764 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18770 (secure 0 0 0) 0x7fb548035090 con 0x7fb5600cc5e0 2023-11-08T16:38:50.048 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:50.049 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":90,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:38:07.597635+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:37:30.174436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T16:37:31.340720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:37:42.279296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2e","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6836/1764545681":"2023-11-09T16:38:06.974694+0000","172.21.15.81:6837/1764545681":"2023-11-09T16:38:06.974694+0000","172.21.15.81:6837/457413856":"2023-11-09T16:38:00.541280+0000","172.21.15.81:6836/457413856":"2023-11-09T16:38:00.541280+0000","172.21.15.81:6837/2162767469":"2023-11-09T16:37:58.508547+0000","172.21.15.81:6836/2162767469":"2023-11-09T16:37:58.508547+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:38:50.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.048+0000 7fb564ffb640 1 -- 172.21.15.81:0/2395772764 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb54006ae70 msgr2=0x7fb54006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.048+0000 7fb564ffb640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb54006ae70 0x7fb54006d330 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb550006fd0 tx=0x7fb550008040 comp rx=0 tx=0).stop 2023-11-08T16:38:50.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.048+0000 7fb564ffb640 1 -- 172.21.15.81:0/2395772764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5600cc5e0 msgr2=0x7fb560155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.048+0000 7fb564ffb640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5600cc5e0 0x7fb560155eb0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fb548009890 tx=0x7fb5480046c0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.048+0000 7fb564ffb640 1 -- 172.21.15.81:0/2395772764 shutdown_connections 2023-11-08T16:38:50.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.048+0000 7fb564ffb640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb54006ae70 0x7fb54006d330 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.048+0000 7fb564ffb640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb5600ccfb0 0x7fb5600b4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.048+0000 7fb564ffb640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb5600cc5e0 0x7fb560155eb0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.048+0000 7fb564ffb640 1 --2- 172.21.15.81:0/2395772764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb5600adcd0 0x7fb5600b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.048+0000 7fb564ffb640 1 -- 172.21.15.81:0/2395772764 >> 172.21.15.81:0/2395772764 conn(0x7fb5600cb490 msgr2=0x7fb5600aea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.048+0000 7fb564ffb640 1 -- 172.21.15.81:0/2395772764 shutdown_connections 2023-11-08T16:38:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.049+0000 7fb564ffb640 1 -- 172.21.15.81:0/2395772764 wait complete. 2023-11-08T16:38:50.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:37:30.174436+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:37:31.340720+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:37:42.279296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:38:50.070 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:38:50.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.279+0000 7fdc42d63640 1 Processor -- start 2023-11-08T16:38:50.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.279+0000 7fdc42d63640 1 -- start start 2023-11-08T16:38:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.281+0000 7fdc42d63640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdc3c15e1a0 0x7fdc3c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.281+0000 7fdc42d63640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fdc3c070530 0x7fdc3c06bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.281+0000 7fdc42d63640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3c15e580 con 0x7fdc3c0c97b0 2023-11-08T16:38:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.281+0000 7fdc42d63640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3c0ca830 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.281+0000 7fdc42d63640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3c0c9cb0 con 0x7fdc3c070530 2023-11-08T16:38:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.281+0000 7fdc41d61640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdc3c15e1a0 0x7fdc3c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.281+0000 7fdc41d61640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdc3c15e1a0 0x7fdc3c0caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60486/0 (socket says 172.21.15.81:60486) 2023-11-08T16:38:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.281+0000 7fdc41d61640 1 -- 172.21.15.81:0/3596081677 learned_addr learned my addr 172.21.15.81:0/3596081677 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.281+0000 7fdc42562640 1 --2- 172.21.15.81:0/3596081677 >> v2:172.21.15.93:3300/0 conn(0x7fdc3c070530 0x7fdc3c06bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.282+0000 7fdc41d61640 1 -- 172.21.15.81:0/3596081677 >> v2:172.21.15.93:3300/0 conn(0x7fdc3c070530 msgr2=0x7fdc3c06bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:50.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.282+0000 7fdc41d61640 1 --2- 172.21.15.81:0/3596081677 >> v2:172.21.15.93:3300/0 conn(0x7fdc3c070530 0x7fdc3c06bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.282+0000 7fdc41d61640 1 -- 172.21.15.81:0/3596081677 >> v1:172.21.15.93:6789/0 conn(0x7fdc3c0c97b0 legacy=0x7fdc3c06fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:50.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.282+0000 7fdc41d61640 1 -- 172.21.15.81:0/3596081677 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3c0c9f90 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.283+0000 7fdc41d61640 1 --2- 172.21.15.81:0/3596081677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc3c15e1a0 0x7fdc3c0caa30 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0b4ac0 tx=0x7fdc3802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63e7e2468a0904d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:50.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.283+0000 7fdc40d5f640 1 -- 172.21.15.81:0/3596081677 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380317e0 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.283+0000 7fdc40d5f640 1 -- 172.21.15.81:0/3596081677 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc38031940 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.283+0000 7fdc40d5f640 1 -- 172.21.15.81:0/3596081677 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38031cd0 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.284+0000 7fdc42d63640 1 -- 172.21.15.81:0/3596081677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc3c15e1a0 msgr2=0x7fdc3c0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.284+0000 7fdc42d63640 1 --2- 172.21.15.81:0/3596081677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc3c15e1a0 0x7fdc3c0caa30 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0b4ac0 tx=0x7fdc3802b880 comp rx=0 tx=0).stop 2023-11-08T16:38:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.284+0000 7fdc42d63640 1 -- 172.21.15.81:0/3596081677 shutdown_connections 2023-11-08T16:38:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.284+0000 7fdc42d63640 1 --2- 172.21.15.81:0/3596081677 >> v2:172.21.15.93:3300/0 conn(0x7fdc3c070530 0x7fdc3c06bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.284+0000 7fdc42d63640 1 --2- 172.21.15.81:0/3596081677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc3c15e1a0 0x7fdc3c0caa30 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.284+0000 7fdc42d63640 1 -- 172.21.15.81:0/3596081677 >> 172.21.15.81:0/3596081677 conn(0x7fdc3c0ce8d0 msgr2=0x7fdc3c0cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.284+0000 7fdc42d63640 1 -- 172.21.15.81:0/3596081677 shutdown_connections 2023-11-08T16:38:50.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.284+0000 7fdc42d63640 1 -- 172.21.15.81:0/3596081677 wait complete. 2023-11-08T16:38:50.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.286+0000 7fdc42d63640 1 Processor -- start 2023-11-08T16:38:50.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.286+0000 7fdc42d63640 1 -- start start 2023-11-08T16:38:50.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.287+0000 7fdc42d63640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc3c070530 0x7fdc3c0cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.287+0000 7fdc42d63640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c0c97b0 0x7fdc3c146140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.287+0000 7fdc42d63640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc3c15e1a0 0x7fdc3c15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.287+0000 7fdc42d63640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c0cc410 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.287+0000 7fdc42d63640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c0ce320 con 0x7fdc3c0c97b0 2023-11-08T16:38:50.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.287+0000 7fdc42d63640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc3c0cc660 con 0x7fdc3c070530 2023-11-08T16:38:50.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.287+0000 7fdc42562640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc3c15e1a0 0x7fdc3c15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.287+0000 7fdc42562640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc3c15e1a0 0x7fdc3c15e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60494/0 (socket says 172.21.15.81:60494) 2023-11-08T16:38:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.287+0000 7fdc42562640 1 -- 172.21.15.81:0/366368841 learned_addr learned my addr 172.21.15.81:0/366368841 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.287+0000 7fdc41d61640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc3c070530 0x7fdc3c0cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc41560640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c0c97b0 0x7fdc3c146140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc42562640 1 -- 172.21.15.81:0/366368841 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc3c070530 msgr2=0x7fdc3c0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc42562640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc3c070530 0x7fdc3c0cdde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc42562640 1 -- 172.21.15.81:0/366368841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c0c97b0 msgr2=0x7fdc3c146140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc42562640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c0c97b0 0x7fdc3c146140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc42562640 1 -- 172.21.15.81:0/366368841 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc380094d0 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc41560640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c0c97b0 0x7fdc3c146140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:50.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc42562640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc3c15e1a0 0x7fdc3c15e990 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fdc34007e70 tx=0x7fdc3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:50.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc327fc640 1 -- 172.21.15.81:0/366368841 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34014620 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc327fc640 1 -- 172.21.15.81:0/366368841 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc340147a0 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc42d63640 1 -- 172.21.15.81:0/366368841 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc3c0cc940 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc327fc640 1 -- 172.21.15.81:0/366368841 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34014b50 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.288+0000 7fdc42d63640 1 -- 172.21.15.81:0/366368841 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc3c0ccf00 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.289+0000 7fdc42d63640 1 -- 172.21.15.81:0/366368841 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc3c04ece0 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.290+0000 7fdc327fc640 1 -- 172.21.15.81:0/366368841 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc34014cb0 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.291+0000 7fdc327fc640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2006ae70 0x7fdc2006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.291+0000 7fdc327fc640 1 -- 172.21.15.81:0/366368841 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fdc3408a330 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.292+0000 7fdc41d61640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2006ae70 0x7fdc2006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.292+0000 7fdc41d61640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2006ae70 0x7fdc2006d330 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdc38036000 tx=0x7fdc3803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:50.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.296+0000 7fdc327fc640 1 -- 172.21.15.81:0/366368841 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc34055070 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.484+0000 7fdc42d63640 1 -- 172.21.15.81:0/366368841 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc3c0cf480 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.485+0000 7fdc327fc640 1 -- 172.21.15.81:0/366368841 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc34058690 con 0x7fdc3c15e1a0 2023-11-08T16:38:50.488 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:38:50.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 -- 172.21.15.81:0/366368841 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2006ae70 msgr2=0x7fdc2006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2006ae70 0x7fdc2006d330 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fdc38036000 tx=0x7fdc3803d040 comp rx=0 tx=0).stop 2023-11-08T16:38:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 -- 172.21.15.81:0/366368841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc3c15e1a0 msgr2=0x7fdc3c15e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc3c15e1a0 0x7fdc3c15e990 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fdc34007e70 tx=0x7fdc3400e4b0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 -- 172.21.15.81:0/366368841 shutdown_connections 2023-11-08T16:38:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2006ae70 0x7fdc2006d330 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc3c15e1a0 0x7fdc3c15e990 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc3c0c97b0 0x7fdc3c146140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 --2- 172.21.15.81:0/366368841 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc3c070530 0x7fdc3c0cdde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 -- 172.21.15.81:0/366368841 >> 172.21.15.81:0/366368841 conn(0x7fdc3c0ce8d0 msgr2=0x7fdc3c06fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 -- 172.21.15.81:0/366368841 shutdown_connections 2023-11-08T16:38:50.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.488+0000 7fdc42d63640 1 -- 172.21.15.81:0/366368841 wait complete. 2023-11-08T16:38:50.508 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:38:50.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.725+0000 7f0b29ccf640 1 Processor -- start 2023-11-08T16:38:50.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.725+0000 7f0b29ccf640 1 -- start start 2023-11-08T16:38:50.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.726+0000 7f0b29ccf640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b240ce0d0 0x7f0b240caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.727+0000 7f0b29ccf640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f0b2415bfc0 0x7f0b2415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.727+0000 7f0b29ccf640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b2415f500 con 0x7f0b2415f0b0 2023-11-08T16:38:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.727+0000 7f0b29ccf640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b240cda10 con 0x7f0b2415bfc0 2023-11-08T16:38:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.727+0000 7f0b29ccf640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b2406fa30 con 0x7f0b240ce0d0 2023-11-08T16:38:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.727+0000 7f0b237fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b240ce0d0 0x7f0b240caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.728+0000 7f0b23fff640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f0b2415f0b0 0x7f0b241516e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:44748/0 (socket says 172.21.15.81:44748) 2023-11-08T16:38:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.728+0000 7f0b23fff640 1 -- 172.21.15.81:0/716168983 learned_addr learned my addr 172.21.15.81:0/716168983 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.728+0000 7f0b22ffd640 1 --2- 172.21.15.81:0/716168983 >> v2:172.21.15.93:3300/0 conn(0x7f0b2415bfc0 0x7f0b2415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.728+0000 7f0b237fe640 1 -- 172.21.15.81:0/716168983 >> v2:172.21.15.93:3300/0 conn(0x7f0b2415bfc0 msgr2=0x7f0b2415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.728+0000 7f0b227fc640 1 -- 172.21.15.81:0/716168983 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559821737 0 0) 0x7f0b1c001490 con 0x7f0b2415f0b0 2023-11-08T16:38:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.728+0000 7f0b237fe640 1 --2- 172.21.15.81:0/716168983 >> v2:172.21.15.93:3300/0 conn(0x7f0b2415bfc0 0x7f0b2415eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.728+0000 7f0b237fe640 1 -- 172.21.15.81:0/716168983 >> v1:172.21.15.93:6789/0 conn(0x7f0b2415f0b0 legacy=0x7f0b241516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.728+0000 7f0b237fe640 1 -- 172.21.15.81:0/716168983 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b240ce4b0 con 0x7f0b240ce0d0 2023-11-08T16:38:50.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.730+0000 7f0b237fe640 1 --2- 172.21.15.81:0/716168983 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b240ce0d0 0x7f0b240caf20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0b140097a0 tx=0x7f0b1402b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1644548d680188 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:50.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.730+0000 7f0b227fc640 1 -- 172.21.15.81:0/716168983 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14031830 con 0x7f0b240ce0d0 2023-11-08T16:38:50.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.730+0000 7f0b227fc640 1 -- 172.21.15.81:0/716168983 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b14031990 con 0x7f0b240ce0d0 2023-11-08T16:38:50.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.731+0000 7f0b227fc640 1 -- 172.21.15.81:0/716168983 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14031d20 con 0x7f0b240ce0d0 2023-11-08T16:38:50.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.731+0000 7f0b29ccf640 1 -- 172.21.15.81:0/716168983 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b240ce0d0 msgr2=0x7f0b240caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.731+0000 7f0b29ccf640 1 --2- 172.21.15.81:0/716168983 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b240ce0d0 0x7f0b240caf20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0b140097a0 tx=0x7f0b1402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.731+0000 7f0b29ccf640 1 -- 172.21.15.81:0/716168983 shutdown_connections 2023-11-08T16:38:50.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.731+0000 7f0b29ccf640 1 --2- 172.21.15.81:0/716168983 >> v2:172.21.15.93:3300/0 conn(0x7f0b2415bfc0 0x7f0b2415eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.731+0000 7f0b29ccf640 1 --2- 172.21.15.81:0/716168983 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b240ce0d0 0x7f0b240caf20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.731+0000 7f0b29ccf640 1 -- 172.21.15.81:0/716168983 >> 172.21.15.81:0/716168983 conn(0x7f0b240cb580 msgr2=0x7f0b240cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:50.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.731+0000 7f0b29ccf640 1 -- 172.21.15.81:0/716168983 shutdown_connections 2023-11-08T16:38:50.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.731+0000 7f0b29ccf640 1 -- 172.21.15.81:0/716168983 wait complete. 2023-11-08T16:38:50.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.733+0000 7f0b29ccf640 1 Processor -- start 2023-11-08T16:38:50.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.733+0000 7f0b29ccf640 1 -- start start 2023-11-08T16:38:50.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.734+0000 7f0b29ccf640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b240ce0d0 0x7f0b240c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.734+0000 7f0b29ccf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b2415bfc0 0x7f0b240c1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.734+0000 7f0b29ccf640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b2415f0b0 0x7f0b241f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.734+0000 7f0b29ccf640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b24144dc0 con 0x7f0b2415bfc0 2023-11-08T16:38:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.734+0000 7f0b29ccf640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240c8200 con 0x7f0b240ce0d0 2023-11-08T16:38:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.734+0000 7f0b29ccf640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240c84e0 con 0x7f0b2415f0b0 2023-11-08T16:38:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b22ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b2415bfc0 0x7f0b240c1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b22ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b2415bfc0 0x7f0b240c1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:60504/0 (socket says 172.21.15.81:60504) 2023-11-08T16:38:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b22ffd640 1 -- 172.21.15.81:0/1218331396 learned_addr learned my addr 172.21.15.81:0/1218331396 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b237fe640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b240ce0d0 0x7f0b240c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b23fff640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b2415f0b0 0x7f0b241f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b22ffd640 1 -- 172.21.15.81:0/1218331396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b2415f0b0 msgr2=0x7f0b241f4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b22ffd640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b2415f0b0 0x7f0b241f4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b22ffd640 1 -- 172.21.15.81:0/1218331396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b240ce0d0 msgr2=0x7f0b240c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b22ffd640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b240ce0d0 0x7f0b240c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b22ffd640 1 -- 172.21.15.81:0/1218331396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b14009400 con 0x7f0b2415bfc0 2023-11-08T16:38:50.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b23fff640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b2415f0b0 0x7f0b241f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:38:50.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.735+0000 7f0b237fe640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b240ce0d0 0x7f0b240c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:38:50.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.736+0000 7f0b22ffd640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b2415bfc0 0x7f0b240c1d70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800e980 tx=0x7f0b1800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:50.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.736+0000 7f0b0bfff640 1 -- 172.21.15.81:0/1218331396 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18009670 con 0x7f0b2415bfc0 2023-11-08T16:38:50.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.736+0000 7f0b0bfff640 1 -- 172.21.15.81:0/1218331396 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b1800ca00 con 0x7f0b2415bfc0 2023-11-08T16:38:50.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.736+0000 7f0b29ccf640 1 -- 172.21.15.81:0/1218331396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b240c87c0 con 0x7f0b2415bfc0 2023-11-08T16:38:50.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.736+0000 7f0b0bfff640 1 -- 172.21.15.81:0/1218331396 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b1800cd90 con 0x7f0b2415bfc0 2023-11-08T16:38:50.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.736+0000 7f0b29ccf640 1 -- 172.21.15.81:0/1218331396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b240c22b0 con 0x7f0b2415bfc0 2023-11-08T16:38:50.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.737+0000 7f0b29ccf640 1 -- 172.21.15.81:0/1218331396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b2404ece0 con 0x7f0b2415bfc0 2023-11-08T16:38:50.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.738+0000 7f0b0bfff640 1 -- 172.21.15.81:0/1218331396 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b18013470 con 0x7f0b2415bfc0 2023-11-08T16:38:50.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.739+0000 7f0b0bfff640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1006ae70 0x7f0b1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:50.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.739+0000 7f0b0bfff640 1 -- 172.21.15.81:0/1218331396 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f0b18089c50 con 0x7f0b2415bfc0 2023-11-08T16:38:50.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.739+0000 7f0b237fe640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1006ae70 0x7f0b1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:50.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.740+0000 7f0b237fe640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1006ae70 0x7f0b1006d330 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0b14036000 tx=0x7f0b1403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:50.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.743+0000 7f0b0bfff640 1 -- 172.21.15.81:0/1218331396 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0b18054990 con 0x7f0b2415bfc0 2023-11-08T16:38:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.938+0000 7f0b29ccf640 1 -- 172.21.15.81:0/1218331396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0b240ce6b0 con 0x7f0b2415bfc0 2023-11-08T16:38:50.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.939+0000 7f0b0bfff640 1 -- 172.21.15.81:0/1218331396 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0b18017020 con 0x7f0b2415bfc0 2023-11-08T16:38:50.941 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:38:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.941+0000 7f0b29ccf640 1 -- 172.21.15.81:0/1218331396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1006ae70 msgr2=0x7f0b1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.941+0000 7f0b29ccf640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1006ae70 0x7f0b1006d330 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0b14036000 tx=0x7f0b1403c040 comp rx=0 tx=0).stop 2023-11-08T16:38:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.941+0000 7f0b29ccf640 1 -- 172.21.15.81:0/1218331396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b2415bfc0 msgr2=0x7f0b240c1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.941+0000 7f0b29ccf640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b2415bfc0 0x7f0b240c1d70 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800e980 tx=0x7f0b1800ee40 comp rx=0 tx=0).stop 2023-11-08T16:38:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.941+0000 7f0b29ccf640 1 -- 172.21.15.81:0/1218331396 shutdown_connections 2023-11-08T16:38:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.941+0000 7f0b29ccf640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0b1006ae70 0x7f0b1006d330 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.941+0000 7f0b29ccf640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0b2415f0b0 0x7f0b241f4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.941+0000 7f0b29ccf640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0b2415bfc0 0x7f0b240c1d70 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.941+0000 7f0b29ccf640 1 --2- 172.21.15.81:0/1218331396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0b240ce0d0 0x7f0b240c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.941+0000 7f0b29ccf640 1 -- 172.21.15.81:0/1218331396 >> 172.21.15.81:0/1218331396 conn(0x7f0b240cb580 msgr2=0x7f0b2415cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:50.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.941+0000 7f0b29ccf640 1 -- 172.21.15.81:0/1218331396 shutdown_connections 2023-11-08T16:38:50.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:50.942+0000 7f0b29ccf640 1 -- 172.21.15.81:0/1218331396 wait complete. 2023-11-08T16:38:50.964 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:38:51.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.177+0000 7fb831a9a640 1 Processor -- start 2023-11-08T16:38:51.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.177+0000 7fb831a9a640 1 -- start start 2023-11-08T16:38:51.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.178+0000 7fb831a9a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c15e1a0 0x7fb82c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb831a9a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb82c0c97b0 0x7fb82c06fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb831a9a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb82c070360 0x7fb82c15e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb831a9a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82c0ca830 con 0x7fb82c0c97b0 2023-11-08T16:38:51.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb831a9a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82c0707d0 con 0x7fb82c070360 2023-11-08T16:38:51.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb831a9a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c15e580 con 0x7fb82c15e1a0 2023-11-08T16:38:51.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb82bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb82c0c97b0 0x7fb82c06fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb82bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb82c0c97b0 0x7fb82c06fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55624/0 (socket says 172.21.15.81:55624) 2023-11-08T16:38:51.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb82bfff640 1 -- 172.21.15.81:0/4115537543 learned_addr learned my addr 172.21.15.81:0/4115537543 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:51.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb830a98640 1 --2- 172.21.15.81:0/4115537543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c15e1a0 0x7fb82c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb82bfff640 1 -- 172.21.15.81:0/4115537543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c15e1a0 msgr2=0x7fb82c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:51.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb831299640 1 --2- 172.21.15.81:0/4115537543 >> v2:172.21.15.93:3300/0 conn(0x7fb82c070360 0x7fb82c15e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.179+0000 7fb82bfff640 1 --2- 172.21.15.81:0/4115537543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c15e1a0 0x7fb82c0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.180+0000 7fb82bfff640 1 -- 172.21.15.81:0/4115537543 >> v2:172.21.15.93:3300/0 conn(0x7fb82c070360 msgr2=0x7fb82c15e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:51.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.180+0000 7fb82bfff640 1 --2- 172.21.15.81:0/4115537543 >> v2:172.21.15.93:3300/0 conn(0x7fb82c070360 0x7fb82c15e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.180+0000 7fb82bfff640 1 -- 172.21.15.81:0/4115537543 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c0c9cb0 con 0x7fb82c0c97b0 2023-11-08T16:38:51.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.180+0000 7fb830a98640 1 --2- 172.21.15.81:0/4115537543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c15e1a0 0x7fb82c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:51.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.180+0000 7fb831299640 1 --2- 172.21.15.81:0/4115537543 >> v2:172.21.15.93:3300/0 conn(0x7fb82c070360 0x7fb82c15e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:51.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.181+0000 7fb82bfff640 1 --2- 172.21.15.81:0/4115537543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c97b0 0x7fb82c06fe20 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0ce760 tx=0x7fb81802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3966c07d2c349af5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:51.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.181+0000 7fb82b7fe640 1 -- 172.21.15.81:0/4115537543 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb81802e070 con 0x7fb82c0c97b0 2023-11-08T16:38:51.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.181+0000 7fb82b7fe640 1 -- 172.21.15.81:0/4115537543 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818004030 con 0x7fb82c0c97b0 2023-11-08T16:38:51.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.181+0000 7fb82b7fe640 1 -- 172.21.15.81:0/4115537543 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818004420 con 0x7fb82c0c97b0 2023-11-08T16:38:51.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.181+0000 7fb831a9a640 1 -- 172.21.15.81:0/4115537543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c97b0 msgr2=0x7fb82c06fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:51.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.181+0000 7fb831a9a640 1 --2- 172.21.15.81:0/4115537543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c97b0 0x7fb82c06fe20 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7fb82c0ce760 tx=0x7fb81802b840 comp rx=0 tx=0).stop 2023-11-08T16:38:51.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.182+0000 7fb831a9a640 1 -- 172.21.15.81:0/4115537543 shutdown_connections 2023-11-08T16:38:51.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.182+0000 7fb831a9a640 1 --2- 172.21.15.81:0/4115537543 >> v2:172.21.15.93:3300/0 conn(0x7fb82c070360 0x7fb82c15e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.182+0000 7fb831a9a640 1 --2- 172.21.15.81:0/4115537543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c97b0 0x7fb82c06fe20 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.182+0000 7fb831a9a640 1 --2- 172.21.15.81:0/4115537543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c15e1a0 0x7fb82c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.182+0000 7fb831a9a640 1 -- 172.21.15.81:0/4115537543 >> 172.21.15.81:0/4115537543 conn(0x7fb82c0ce8d0 msgr2=0x7fb82c0cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:51.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.182+0000 7fb831a9a640 1 -- 172.21.15.81:0/4115537543 shutdown_connections 2023-11-08T16:38:51.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.182+0000 7fb831a9a640 1 -- 172.21.15.81:0/4115537543 wait complete. 2023-11-08T16:38:51.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.183+0000 7fb831a9a640 1 Processor -- start 2023-11-08T16:38:51.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.184+0000 7fb831a9a640 1 -- start start 2023-11-08T16:38:51.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.184+0000 7fb831a9a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c070360 0x7fb82c1e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.184+0000 7fb831a9a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c97b0 0x7fb82c1f4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.184+0000 7fb831a9a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c15e1a0 0x7fb82c152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.184+0000 7fb831a9a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c0c8530 con 0x7fb82c0c97b0 2023-11-08T16:38:51.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.184+0000 7fb831a9a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c1f4f40 con 0x7fb82c070360 2023-11-08T16:38:51.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.184+0000 7fb831a9a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c0c7920 con 0x7fb82c15e1a0 2023-11-08T16:38:51.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.185+0000 7fb82bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c97b0 0x7fb82c1f4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.185+0000 7fb82bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c97b0 0x7fb82c1f4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55636/0 (socket says 172.21.15.81:55636) 2023-11-08T16:38:51.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.185+0000 7fb82bfff640 1 -- 172.21.15.81:0/4021403377 learned_addr learned my addr 172.21.15.81:0/4021403377 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:51.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.185+0000 7fb831299640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c15e1a0 0x7fb82c152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.185+0000 7fb830a98640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c070360 0x7fb82c1e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.185+0000 7fb82bfff640 1 -- 172.21.15.81:0/4021403377 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c15e1a0 msgr2=0x7fb82c152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:51.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.185+0000 7fb82bfff640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c15e1a0 0x7fb82c152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.185+0000 7fb82bfff640 1 -- 172.21.15.81:0/4021403377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c070360 msgr2=0x7fb82c1e4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:51.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.185+0000 7fb82bfff640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c070360 0x7fb82c1e4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.185+0000 7fb82bfff640 1 -- 172.21.15.81:0/4021403377 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb818009460 con 0x7fb82c0c97b0 2023-11-08T16:38:51.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.185+0000 7fb830a98640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c070360 0x7fb82c1e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:51.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.186+0000 7fb82bfff640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c97b0 0x7fb82c1f4a00 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb818035040 tx=0x7fb8180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:51.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.186+0000 7fb8297fa640 1 -- 172.21.15.81:0/4021403377 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8180039a0 con 0x7fb82c0c97b0 2023-11-08T16:38:51.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.186+0000 7fb8297fa640 1 -- 172.21.15.81:0/4021403377 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818003b00 con 0x7fb82c0c97b0 2023-11-08T16:38:51.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.186+0000 7fb8297fa640 1 -- 172.21.15.81:0/4021403377 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818003e90 con 0x7fb82c0c97b0 2023-11-08T16:38:51.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.186+0000 7fb831a9a640 1 -- 172.21.15.81:0/4021403377 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82c0c7c00 con 0x7fb82c0c97b0 2023-11-08T16:38:51.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.186+0000 7fb831a9a640 1 -- 172.21.15.81:0/4021403377 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c0c81c0 con 0x7fb82c0c97b0 2023-11-08T16:38:51.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.187+0000 7fb831a9a640 1 -- 172.21.15.81:0/4021403377 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82c0c86a0 con 0x7fb82c0c97b0 2023-11-08T16:38:51.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.188+0000 7fb8297fa640 1 -- 172.21.15.81:0/4021403377 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb818006040 con 0x7fb82c0c97b0 2023-11-08T16:38:51.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.189+0000 7fb8297fa640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006aea0 0x7fb81006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.189+0000 7fb8297fa640 1 -- 172.21.15.81:0/4021403377 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fb8180ab3e0 con 0x7fb82c0c97b0 2023-11-08T16:38:51.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.189+0000 7fb830a98640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006aea0 0x7fb81006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.190+0000 7fb830a98640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006aea0 0x7fb81006d360 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb820006fd0 tx=0x7fb820008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:51.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.194+0000 7fb8297fa640 1 -- 172.21.15.81:0/4021403377 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb818076120 con 0x7fb82c0c97b0 2023-11-08T16:38:51.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.382+0000 7fb831a9a640 1 -- 172.21.15.81:0/4021403377 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb82c04ece0 con 0x7fb82c0c97b0 2023-11-08T16:38:51.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.383+0000 7fb8297fa640 1 -- 172.21.15.81:0/4021403377 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb818079740 con 0x7fb82c0c97b0 2023-11-08T16:38:51.386 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:38:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.385+0000 7fb831a9a640 1 -- 172.21.15.81:0/4021403377 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006aea0 msgr2=0x7fb81006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.385+0000 7fb831a9a640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006aea0 0x7fb81006d360 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb820006fd0 tx=0x7fb820008040 comp rx=0 tx=0).stop 2023-11-08T16:38:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.385+0000 7fb831a9a640 1 -- 172.21.15.81:0/4021403377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c97b0 msgr2=0x7fb82c1f4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:51.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.385+0000 7fb831a9a640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c97b0 0x7fb82c1f4a00 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7fb818035040 tx=0x7fb8180046c0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.386+0000 7fb831a9a640 1 -- 172.21.15.81:0/4021403377 shutdown_connections 2023-11-08T16:38:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.386+0000 7fb831a9a640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb81006aea0 0x7fb81006d360 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.386+0000 7fb831a9a640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb82c15e1a0 0x7fb82c152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.386+0000 7fb831a9a640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb82c0c97b0 0x7fb82c1f4a00 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.386+0000 7fb831a9a640 1 --2- 172.21.15.81:0/4021403377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c070360 0x7fb82c1e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.386+0000 7fb831a9a640 1 -- 172.21.15.81:0/4021403377 >> 172.21.15.81:0/4021403377 conn(0x7fb82c0ce8d0 msgr2=0x7fb82c0ca970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:51.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.386+0000 7fb831a9a640 1 -- 172.21.15.81:0/4021403377 shutdown_connections 2023-11-08T16:38:51.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.386+0000 7fb831a9a640 1 -- 172.21.15.81:0/4021403377 wait complete. 2023-11-08T16:38:51.407 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:38:51.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.621+0000 7f6e351d0640 1 Processor -- start 2023-11-08T16:38:51.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.622+0000 7f6e351d0640 1 -- start start 2023-11-08T16:38:51.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.623+0000 7f6e351d0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e300ce0d0 0x7f6e300caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.623+0000 7f6e351d0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6e3015bfc0 0x7f6e3015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.623+0000 7f6e351d0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3015f500 con 0x7f6e3015f0b0 2023-11-08T16:38:51.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.623+0000 7f6e351d0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e300cda10 con 0x7f6e3015bfc0 2023-11-08T16:38:51.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.623+0000 7f6e351d0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3006fa30 con 0x7f6e300ce0d0 2023-11-08T16:38:51.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.623+0000 7f6e2e59c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6e3015bfc0 0x7f6e3015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.624+0000 7f6e2f59e640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f6e3015f0b0 0x7f6e301516e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54884/0 (socket says 172.21.15.81:54884) 2023-11-08T16:38:51.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.624+0000 7f6e2f59e640 1 -- 172.21.15.81:0/2738275270 learned_addr learned my addr 172.21.15.81:0/2738275270 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:51.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.624+0000 7f6e2ed9d640 1 --2- 172.21.15.81:0/2738275270 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e300ce0d0 0x7f6e300caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.624+0000 7f6e2e59c640 1 -- 172.21.15.81:0/2738275270 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e300ce0d0 msgr2=0x7f6e300caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:51.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.624+0000 7f6e2e59c640 1 --2- 172.21.15.81:0/2738275270 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e300ce0d0 0x7f6e300caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.624+0000 7f6e2e59c640 1 -- 172.21.15.81:0/2738275270 >> v1:172.21.15.81:6789/0 conn(0x7f6e3015f0b0 legacy=0x7f6e301516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:51.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.624+0000 7f6e2e59c640 1 -- 172.21.15.81:0/2738275270 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e300ce4b0 con 0x7f6e3015bfc0 2023-11-08T16:38:51.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.625+0000 7f6e2e59c640 1 --2- 172.21.15.81:0/2738275270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e3015bfc0 0x7f6e3015eb70 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f6e180097a0 tx=0x7f6e1802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7d1c23bf92075d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:51.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.625+0000 7f6e2dd9b640 1 -- 172.21.15.81:0/2738275270 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1802d040 con 0x7f6e3015bfc0 2023-11-08T16:38:51.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.625+0000 7f6e2dd9b640 1 -- 172.21.15.81:0/2738275270 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e18031810 con 0x7f6e3015bfc0 2023-11-08T16:38:51.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.625+0000 7f6e2dd9b640 1 -- 172.21.15.81:0/2738275270 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18031ba0 con 0x7f6e3015bfc0 2023-11-08T16:38:51.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.626+0000 7f6e351d0640 1 -- 172.21.15.81:0/2738275270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e3015bfc0 msgr2=0x7f6e3015eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:51.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.626+0000 7f6e351d0640 1 --2- 172.21.15.81:0/2738275270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e3015bfc0 0x7f6e3015eb70 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f6e180097a0 tx=0x7f6e1802b8b0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.626+0000 7f6e351d0640 1 -- 172.21.15.81:0/2738275270 shutdown_connections 2023-11-08T16:38:51.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.626+0000 7f6e351d0640 1 --2- 172.21.15.81:0/2738275270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e3015bfc0 0x7f6e3015eb70 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.626+0000 7f6e351d0640 1 --2- 172.21.15.81:0/2738275270 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e300ce0d0 0x7f6e300caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.626+0000 7f6e351d0640 1 -- 172.21.15.81:0/2738275270 >> 172.21.15.81:0/2738275270 conn(0x7f6e300cb580 msgr2=0x7f6e300cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:51.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.626+0000 7f6e351d0640 1 -- 172.21.15.81:0/2738275270 shutdown_connections 2023-11-08T16:38:51.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.626+0000 7f6e351d0640 1 -- 172.21.15.81:0/2738275270 wait complete. 2023-11-08T16:38:51.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.628+0000 7f6e351d0640 1 Processor -- start 2023-11-08T16:38:51.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.628+0000 7f6e351d0640 1 -- start start 2023-11-08T16:38:51.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.629+0000 7f6e351d0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e300ce0d0 0x7f6e3006d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.629+0000 7f6e351d0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e3015bfc0 0x7f6e300afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.629+0000 7f6e351d0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e3015f0b0 0x7f6e3006c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.629+0000 7f6e351d0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3006be70 con 0x7f6e300ce0d0 2023-11-08T16:38:51.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.629+0000 7f6e351d0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300b14f0 con 0x7f6e3015bfc0 2023-11-08T16:38:51.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.629+0000 7f6e351d0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300b1790 con 0x7f6e3015f0b0 2023-11-08T16:38:51.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.629+0000 7f6e2ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e300ce0d0 0x7f6e3006d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.629+0000 7f6e2ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e300ce0d0 0x7f6e3006d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55650/0 (socket says 172.21.15.81:55650) 2023-11-08T16:38:51.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.629+0000 7f6e2ed9d640 1 -- 172.21.15.81:0/2439178411 learned_addr learned my addr 172.21.15.81:0/2439178411 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:51.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e2f59e640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e3015f0b0 0x7f6e3006c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e2e59c640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e3015bfc0 0x7f6e300afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e2ed9d640 1 -- 172.21.15.81:0/2439178411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e3015f0b0 msgr2=0x7f6e3006c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:51.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e2ed9d640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e3015f0b0 0x7f6e3006c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e2ed9d640 1 -- 172.21.15.81:0/2439178411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e3015bfc0 msgr2=0x7f6e300afb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:51.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e2ed9d640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e3015bfc0 0x7f6e300afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e2ed9d640 1 -- 172.21.15.81:0/2439178411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e18009400 con 0x7f6e300ce0d0 2023-11-08T16:38:51.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e2ed9d640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e300ce0d0 0x7f6e3006d150 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6e20007ef0 tx=0x7f6e2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:51.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e277fe640 1 -- 172.21.15.81:0/2439178411 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20014620 con 0x7f6e300ce0d0 2023-11-08T16:38:51.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e277fe640 1 -- 172.21.15.81:0/2439178411 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e20014780 con 0x7f6e300ce0d0 2023-11-08T16:38:51.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e277fe640 1 -- 172.21.15.81:0/2439178411 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20014b30 con 0x7f6e300ce0d0 2023-11-08T16:38:51.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.630+0000 7f6e351d0640 1 -- 172.21.15.81:0/2439178411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3006b7b0 con 0x7f6e300ce0d0 2023-11-08T16:38:51.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.631+0000 7f6e351d0640 1 -- 172.21.15.81:0/2439178411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e30152730 con 0x7f6e300ce0d0 2023-11-08T16:38:51.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.632+0000 7f6e351d0640 1 -- 172.21.15.81:0/2439178411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e3006c0e0 con 0x7f6e300ce0d0 2023-11-08T16:38:51.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.633+0000 7f6e277fe640 1 -- 172.21.15.81:0/2439178411 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e20014c90 con 0x7f6e300ce0d0 2023-11-08T16:38:51.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.634+0000 7f6e277fe640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e1006ae70 0x7f6e1006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:51.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.634+0000 7f6e277fe640 1 -- 172.21.15.81:0/2439178411 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f6e2008a2e0 con 0x7f6e300ce0d0 2023-11-08T16:38:51.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.634+0000 7f6e2e59c640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e1006ae70 0x7f6e1006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:51.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.635+0000 7f6e2e59c640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e1006ae70 0x7f6e1006d330 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6e18036000 tx=0x7f6e1803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:51.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.638+0000 7f6e277fe640 1 -- 172.21.15.81:0/2439178411 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e200550b0 con 0x7f6e300ce0d0 2023-11-08T16:38:51.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.831+0000 7f6e351d0640 1 -- 172.21.15.81:0/2439178411 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f6e3004ece0 con 0x7f6e300ce0d0 2023-11-08T16:38:51.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.832+0000 7f6e277fe640 1 -- 172.21.15.81:0/2439178411 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v90) v1 ==== 106+0+11 (secure 0 0 0) 0x7f6e200586d0 con 0x7f6e300ce0d0 2023-11-08T16:38:51.835 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:38:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 -- 172.21.15.81:0/2439178411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e1006ae70 msgr2=0x7f6e1006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e1006ae70 0x7f6e1006d330 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f6e18036000 tx=0x7f6e1803c040 comp rx=0 tx=0).stop 2023-11-08T16:38:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 -- 172.21.15.81:0/2439178411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e300ce0d0 msgr2=0x7f6e3006d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e300ce0d0 0x7f6e3006d150 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f6e20007ef0 tx=0x7f6e2000e4b0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 -- 172.21.15.81:0/2439178411 shutdown_connections 2023-11-08T16:38:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e1006ae70 0x7f6e1006d330 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e3015f0b0 0x7f6e3006c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e3015bfc0 0x7f6e300afb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 --2- 172.21.15.81:0/2439178411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e300ce0d0 0x7f6e3006d150 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:51.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 -- 172.21.15.81:0/2439178411 >> 172.21.15.81:0/2439178411 conn(0x7f6e300cb580 msgr2=0x7f6e3015ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:51.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 -- 172.21.15.81:0/2439178411 shutdown_connections 2023-11-08T16:38:51.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:51.835+0000 7f6e351d0640 1 -- 172.21.15.81:0/2439178411 wait complete. 2023-11-08T16:38:51.856 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:52.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.061+0000 7f218ac00640 1 Processor -- start 2023-11-08T16:38:52.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.061+0000 7f218ac00640 1 -- start start 2023-11-08T16:38:52.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.063+0000 7f218ac00640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f21840c9ae0 0x7f218406f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.063+0000 7f218ac00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f218406d3f0 0x7f218406c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.063+0000 7f218ac00640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218406c850 con 0x7f218406bec0 2023-11-08T16:38:52.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.063+0000 7f218ac00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21840c9ec0 con 0x7f218406d3f0 2023-11-08T16:38:52.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.063+0000 7f218ac00640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21840cc710 con 0x7f21840c9ae0 2023-11-08T16:38:52.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.063+0000 7f218919d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f218406d3f0 0x7f218406c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.063+0000 7f218919d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f218406d3f0 0x7f218406c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55664/0 (socket says 172.21.15.81:55664) 2023-11-08T16:38:52.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.063+0000 7f218919d640 1 -- 172.21.15.81:0/3448090491 learned_addr learned my addr 172.21.15.81:0/3448090491 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:52.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.064+0000 7f218919d640 1 -- 172.21.15.81:0/3448090491 >> v2:172.21.15.93:3300/0 conn(0x7f21840c9ae0 msgr2=0x7f218406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:52.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.064+0000 7f218919d640 1 --2- 172.21.15.81:0/3448090491 >> v2:172.21.15.93:3300/0 conn(0x7f21840c9ae0 0x7f218406f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.064+0000 7f218919d640 1 -- 172.21.15.81:0/3448090491 >> v1:172.21.15.93:6789/0 conn(0x7f218406bec0 legacy=0x7f218406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:52.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.064+0000 7f218919d640 1 -- 172.21.15.81:0/3448090491 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21840cc9f0 con 0x7f218406d3f0 2023-11-08T16:38:52.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.065+0000 7f218919d640 1 --2- 172.21.15.81:0/3448090491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218406d3f0 0x7f218406c310 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f21840b14a0 tx=0x7f217c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dedf803ecb62ac server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:52.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.065+0000 7f21837fe640 1 -- 172.21.15.81:0/3448090491 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c02c9f0 con 0x7f218406d3f0 2023-11-08T16:38:52.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.065+0000 7f21837fe640 1 -- 172.21.15.81:0/3448090491 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217c02cb50 con 0x7f218406d3f0 2023-11-08T16:38:52.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.065+0000 7f21837fe640 1 -- 172.21.15.81:0/3448090491 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217c0343f0 con 0x7f218406d3f0 2023-11-08T16:38:52.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.066+0000 7f218ac00640 1 -- 172.21.15.81:0/3448090491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218406d3f0 msgr2=0x7f218406c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.066+0000 7f218ac00640 1 --2- 172.21.15.81:0/3448090491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218406d3f0 0x7f218406c310 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f21840b14a0 tx=0x7f217c02be50 comp rx=0 tx=0).stop 2023-11-08T16:38:52.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.066+0000 7f218ac00640 1 -- 172.21.15.81:0/3448090491 shutdown_connections 2023-11-08T16:38:52.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.066+0000 7f218ac00640 1 --2- 172.21.15.81:0/3448090491 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218406d3f0 0x7f218406c310 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.066+0000 7f218ac00640 1 --2- 172.21.15.81:0/3448090491 >> v2:172.21.15.93:3300/0 conn(0x7f21840c9ae0 0x7f218406f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.066+0000 7f218ac00640 1 -- 172.21.15.81:0/3448090491 >> 172.21.15.81:0/3448090491 conn(0x7f21840cb490 msgr2=0x7f21840cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:52.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.066+0000 7f218ac00640 1 -- 172.21.15.81:0/3448090491 shutdown_connections 2023-11-08T16:38:52.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.066+0000 7f218ac00640 1 -- 172.21.15.81:0/3448090491 wait complete. 2023-11-08T16:38:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.068+0000 7f218ac00640 1 Processor -- start 2023-11-08T16:38:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.068+0000 7f218ac00640 1 -- start start 2023-11-08T16:38:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f218ac00640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218406bec0 0x7f2184152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f218ac00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218406d3f0 0x7f21840b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f218ac00640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21840c9ae0 0x7f21840ae5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f218ac00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21840b3a30 con 0x7f218406d3f0 2023-11-08T16:38:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f218ac00640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21840c5b80 con 0x7f21840c9ae0 2023-11-08T16:38:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f218ac00640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21840c5e60 con 0x7f218406bec0 2023-11-08T16:38:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f2183fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218406d3f0 0x7f21840b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f2183fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218406d3f0 0x7f21840b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55680/0 (socket says 172.21.15.81:55680) 2023-11-08T16:38:52.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f2183fff640 1 -- 172.21.15.81:0/2397117427 learned_addr learned my addr 172.21.15.81:0/2397117427 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:52.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f2183fff640 1 -- 172.21.15.81:0/2397117427 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218406bec0 msgr2=0x7f2184152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f218919d640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21840c9ae0 0x7f21840ae5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f218899c640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218406bec0 0x7f2184152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f2183fff640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218406bec0 0x7f2184152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f2183fff640 1 -- 172.21.15.81:0/2397117427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21840c9ae0 msgr2=0x7f21840ae5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.069+0000 7f2183fff640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21840c9ae0 0x7f21840ae5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.070+0000 7f2183fff640 1 -- 172.21.15.81:0/2397117427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c009b30 con 0x7f218406d3f0 2023-11-08T16:38:52.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.070+0000 7f218919d640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21840c9ae0 0x7f21840ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:52.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.070+0000 7f2183fff640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218406d3f0 0x7f21840b2fb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f217000b3f0 tx=0x7f217000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:52.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.070+0000 7f21817fa640 1 -- 172.21.15.81:0/2397117427 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170011830 con 0x7f218406d3f0 2023-11-08T16:38:52.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.070+0000 7f21817fa640 1 -- 172.21.15.81:0/2397117427 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2170011990 con 0x7f218406d3f0 2023-11-08T16:38:52.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.070+0000 7f218ac00640 1 -- 172.21.15.81:0/2397117427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21840c6140 con 0x7f218406d3f0 2023-11-08T16:38:52.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.070+0000 7f21817fa640 1 -- 172.21.15.81:0/2397117427 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170011d40 con 0x7f218406d3f0 2023-11-08T16:38:52.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.070+0000 7f218ac00640 1 -- 172.21.15.81:0/2397117427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2184153290 con 0x7f218406d3f0 2023-11-08T16:38:52.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.071+0000 7f218ac00640 1 -- 172.21.15.81:0/2397117427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2184071be0 con 0x7f218406d3f0 2023-11-08T16:38:52.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.072+0000 7f21817fa640 1 -- 172.21.15.81:0/2397117427 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f217001a400 con 0x7f218406d3f0 2023-11-08T16:38:52.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.073+0000 7f21817fa640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f216806ae70 0x7f216806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.073+0000 7f21817fa640 1 -- 172.21.15.81:0/2397117427 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f21700898a0 con 0x7f218406d3f0 2023-11-08T16:38:52.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.073+0000 7f218899c640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f216806ae70 0x7f216806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.074+0000 7f218899c640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f216806ae70 0x7f216806d330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f217800a810 tx=0x7f2178008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:52.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.078+0000 7f21817fa640 1 -- 172.21.15.81:0/2397117427 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f21700545e0 con 0x7f218406d3f0 2023-11-08T16:38:52.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.307+0000 7f218ac00640 1 -- 172.21.15.81:0/2397117427 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f218404ece0 con 0x7f218406d3f0 2023-11-08T16:38:52.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.308+0000 7f21817fa640 1 -- 172.21.15.81:0/2397117427 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f2170057c00 con 0x7f218406d3f0 2023-11-08T16:38:52.311 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:52.311 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":78,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1527116581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1527116581},{"type":"v1","addr":"172.21.15.93:6833","nonce":1527116581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2617548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2617548066},{"type":"v1","addr":"172.21.15.93:6835","nonce":2617548066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":7380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2591582634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2591582634},{"type":"v1","addr":"172.21.15.81:6835","nonce":2591582634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:09.004709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/1843050442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1843050442},{"type":"v1","addr":"172.21.15.81:6837","nonce":1843050442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:52.312 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 78 2023-11-08T16:38:52.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 -- 172.21.15.81:0/2397117427 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f216806ae70 msgr2=0x7f216806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f216806ae70 0x7f216806d330 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f217800a810 tx=0x7f2178008040 comp rx=0 tx=0).stop 2023-11-08T16:38:52.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 -- 172.21.15.81:0/2397117427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218406d3f0 msgr2=0x7f21840b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218406d3f0 0x7f21840b2fb0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f217000b3f0 tx=0x7f217000b8b0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 -- 172.21.15.81:0/2397117427 shutdown_connections 2023-11-08T16:38:52.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f216806ae70 0x7f216806d330 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f21840c9ae0 0x7f21840ae5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f218406d3f0 0x7f21840b2fb0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 --2- 172.21.15.81:0/2397117427 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f218406bec0 0x7f2184152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 -- 172.21.15.81:0/2397117427 >> 172.21.15.81:0/2397117427 conn(0x7f21840cb490 msgr2=0x7f21840cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:52.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 -- 172.21.15.81:0/2397117427 shutdown_connections 2023-11-08T16:38:52.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.311+0000 7f218ac00640 1 -- 172.21.15.81:0/2397117427 wait complete. 2023-11-08T16:38:52.332 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:38:52.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.541+0000 7fd4f0e9a640 1 Processor -- start 2023-11-08T16:38:52.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.541+0000 7fd4f0e9a640 1 -- start start 2023-11-08T16:38:52.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.542+0000 7fd4f0e9a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd4ec0ced10 0x7fd4ec0cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.542+0000 7fd4f0e9a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ec0cf6b0 con 0x7fd4ec15aea0 2023-11-08T16:38:52.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.542+0000 7fd4f0e9a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ec070470 con 0x7fd4ec15aac0 2023-11-08T16:38:52.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.542+0000 7fd4f0e9a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec06f650 con 0x7fd4ec0ced10 2023-11-08T16:38:52.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.543+0000 7fd4ebfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd4ec15aea0 0x7fd4ec15b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54890/0 (socket says 172.21.15.81:54890) 2023-11-08T16:38:52.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.543+0000 7fd4ebfff640 1 -- 172.21.15.81:0/1391372222 learned_addr learned my addr 172.21.15.81:0/1391372222 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:52.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.543+0000 7fd4eaffd640 1 --2- 172.21.15.81:0/1391372222 >> v2:172.21.15.81:3300/0 conn(0x7fd4ec0ced10 0x7fd4ec0cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.543+0000 7fd4ea7fc640 1 -- 172.21.15.81:0/1391372222 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349936290 0 0) 0x7fd4ec0cf6b0 con 0x7fd4ec15aea0 2023-11-08T16:38:52.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.543+0000 7fd4ea7fc640 1 -- 172.21.15.81:0/1391372222 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4d4003520 con 0x7fd4ec15aea0 2023-11-08T16:38:52.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.543+0000 7fd4eaffd640 1 -- 172.21.15.81:0/1391372222 >> v1:172.21.15.93:6789/0 conn(0x7fd4ec15aac0 legacy=0x7fd4ec0711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.543+0000 7fd4eaffd640 1 -- 172.21.15.81:0/1391372222 >> v1:172.21.15.81:6789/0 conn(0x7fd4ec15aea0 legacy=0x7fd4ec15b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.543+0000 7fd4eaffd640 1 -- 172.21.15.81:0/1391372222 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec06d580 con 0x7fd4ec0ced10 2023-11-08T16:38:52.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.545+0000 7fd4eaffd640 1 --2- 172.21.15.81:0/1391372222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4ec0ced10 0x7fd4ec0cf170 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00097a0 tx=0x7fd4e002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40ddd36a4379b22 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:52.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.545+0000 7fd4ea7fc640 1 -- 172.21.15.81:0/1391372222 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0030610 con 0x7fd4ec0ced10 2023-11-08T16:38:52.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.545+0000 7fd4ea7fc640 1 -- 172.21.15.81:0/1391372222 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e0030770 con 0x7fd4ec0ced10 2023-11-08T16:38:52.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.545+0000 7fd4ea7fc640 1 -- 172.21.15.81:0/1391372222 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0030ae0 con 0x7fd4ec0ced10 2023-11-08T16:38:52.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.545+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/1391372222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4ec0ced10 msgr2=0x7fd4ec0cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.545+0000 7fd4f0e9a640 1 --2- 172.21.15.81:0/1391372222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4ec0ced10 0x7fd4ec0cf170 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00097a0 tx=0x7fd4e002b960 comp rx=0 tx=0).stop 2023-11-08T16:38:52.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.546+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/1391372222 shutdown_connections 2023-11-08T16:38:52.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.546+0000 7fd4f0e9a640 1 --2- 172.21.15.81:0/1391372222 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4ec0ced10 0x7fd4ec0cf170 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.546+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/1391372222 >> 172.21.15.81:0/1391372222 conn(0x7fd4ec06a4a0 msgr2=0x7fd4ec06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:52.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.546+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/1391372222 shutdown_connections 2023-11-08T16:38:52.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.546+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/1391372222 wait complete. 2023-11-08T16:38:52.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.547+0000 7fd4f0e9a640 1 Processor -- start 2023-11-08T16:38:52.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.547+0000 7fd4f0e9a640 1 -- start start 2023-11-08T16:38:52.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.548+0000 7fd4f0e9a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd4ec0ced10 0x7fd4ec0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.548+0000 7fd4f0e9a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4ec15aac0 0x7fd4ec13fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.548+0000 7fd4f0e9a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4ec15aea0 0x7fd4ec1424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.548+0000 7fd4f0e9a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec140220 con 0x7fd4ec15aac0 2023-11-08T16:38:52.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.548+0000 7fd4f0e9a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0b2a90 con 0x7fd4ec15aea0 2023-11-08T16:38:52.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.548+0000 7fd4f0e9a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec14d870 con 0x7fd4ec0ced10 2023-11-08T16:38:52.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.549+0000 7fd4eaffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4ec15aac0 0x7fd4ec13fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.549+0000 7fd4eaffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4ec15aac0 0x7fd4ec13fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55698/0 (socket says 172.21.15.81:55698) 2023-11-08T16:38:52.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.549+0000 7fd4eaffd640 1 -- 172.21.15.81:0/2853296481 learned_addr learned my addr 172.21.15.81:0/2853296481 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:52.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.549+0000 7fd4ebfff640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4ec15aea0 0x7fd4ec1424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.549+0000 7fd4eb7fe640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd4ec0ced10 0x7fd4ec0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.549+0000 7fd4eaffd640 1 -- 172.21.15.81:0/2853296481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd4ec0ced10 msgr2=0x7fd4ec0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.549+0000 7fd4eaffd640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd4ec0ced10 0x7fd4ec0b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.549+0000 7fd4eaffd640 1 -- 172.21.15.81:0/2853296481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4ec15aea0 msgr2=0x7fd4ec1424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.549+0000 7fd4eaffd640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4ec15aea0 0x7fd4ec1424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.549+0000 7fd4eaffd640 1 -- 172.21.15.81:0/2853296481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e0009400 con 0x7fd4ec15aac0 2023-11-08T16:38:52.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.550+0000 7fd4eaffd640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4ec15aac0 0x7fd4ec13fce0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00097a0 tx=0x7fd4e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:52.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.550+0000 7fd4d3fff640 1 -- 172.21.15.81:0/2853296481 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0039c90 con 0x7fd4ec15aac0 2023-11-08T16:38:52.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.550+0000 7fd4d3fff640 1 -- 172.21.15.81:0/2853296481 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e003f040 con 0x7fd4ec15aac0 2023-11-08T16:38:52.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.550+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/2853296481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec14e4c0 con 0x7fd4ec15aac0 2023-11-08T16:38:52.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.550+0000 7fd4d3fff640 1 -- 172.21.15.81:0/2853296481 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0006e10 con 0x7fd4ec15aac0 2023-11-08T16:38:52.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.550+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/2853296481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec0b2720 con 0x7fd4ec15aac0 2023-11-08T16:38:52.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.551+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/2853296481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ec0b28e0 con 0x7fd4ec15aac0 2023-11-08T16:38:52.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.552+0000 7fd4d3fff640 1 -- 172.21.15.81:0/2853296481 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd4e0004030 con 0x7fd4ec15aac0 2023-11-08T16:38:52.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.553+0000 7fd4d3fff640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd4d806ae70 0x7fd4d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.553+0000 7fd4d3fff640 1 -- 172.21.15.81:0/2853296481 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7fd4e00ab7d0 con 0x7fd4ec15aac0 2023-11-08T16:38:52.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.553+0000 7fd4eb7fe640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd4d806ae70 0x7fd4d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.554+0000 7fd4eb7fe640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd4d806ae70 0x7fd4d806d330 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc000f60 tx=0x7fd4dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:52.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.558+0000 7fd4d3fff640 1 -- 172.21.15.81:0/2853296481 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd4e00765d0 con 0x7fd4ec15aac0 2023-11-08T16:38:52.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.747+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/2853296481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4ec067800 con 0x7fd4ec15aac0 2023-11-08T16:38:52.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.750+0000 7fd4d3fff640 1 -- 172.21.15.81:0/2853296481 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18770 (secure 0 0 0) 0x7fd4e0048090 con 0x7fd4ec15aac0 2023-11-08T16:38:52.753 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:52.753 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":90,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:38:07.597635+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:37:30.174436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T16:37:31.340720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:37:42.279296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2e","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6836/1764545681":"2023-11-09T16:38:06.974694+0000","172.21.15.81:6837/1764545681":"2023-11-09T16:38:06.974694+0000","172.21.15.81:6837/457413856":"2023-11-09T16:38:00.541280+0000","172.21.15.81:6836/457413856":"2023-11-09T16:38:00.541280+0000","172.21.15.81:6837/2162767469":"2023-11-09T16:37:58.508547+0000","172.21.15.81:6836/2162767469":"2023-11-09T16:37:58.508547+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:38:52.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/2853296481 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd4d806ae70 msgr2=0x7fd4d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd4d806ae70 0x7fd4d806d330 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc000f60 tx=0x7fd4dc009040 comp rx=0 tx=0).stop 2023-11-08T16:38:52.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/2853296481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4ec15aac0 msgr2=0x7fd4ec13fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4ec15aac0 0x7fd4ec13fce0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fd4e00097a0 tx=0x7fd4e00046c0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/2853296481 shutdown_connections 2023-11-08T16:38:52.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd4d806ae70 0x7fd4d806d330 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd4ec15aea0 0x7fd4ec1424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd4ec15aac0 0x7fd4ec13fce0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 --2- 172.21.15.81:0/2853296481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd4ec0ced10 0x7fd4ec0b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/2853296481 >> 172.21.15.81:0/2853296481 conn(0x7fd4ec06a4a0 msgr2=0x7fd4ec0aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:52.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/2853296481 shutdown_connections 2023-11-08T16:38:52.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.753+0000 7fd4f0e9a640 1 -- 172.21.15.81:0/2853296481 wait complete. 2023-11-08T16:38:52.773 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T16:38:52.774 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:52.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.975+0000 7f55f780e640 1 Processor -- start 2023-11-08T16:38:52.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.975+0000 7f55f780e640 1 -- start start 2023-11-08T16:38:52.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.976+0000 7f55f780e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55f00adcd0 0x7f55f00ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.976+0000 7f55f780e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f55f0070050 0x7f55f00704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.976+0000 7f55f780e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f00c9a70 con 0x7f55f00cc5e0 2023-11-08T16:38:52.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.976+0000 7f55f780e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f00cc3e0 con 0x7f55f0070050 2023-11-08T16:38:52.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.976+0000 7f55f780e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f006d070 con 0x7f55f00adcd0 2023-11-08T16:38:52.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.976+0000 7f55f5dab640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f55f0070050 0x7f55f00704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.977+0000 7f55f5dab640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f55f0070050 0x7f55f00704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55700/0 (socket says 172.21.15.81:55700) 2023-11-08T16:38:52.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.977+0000 7f55f5dab640 1 -- 172.21.15.81:0/3556286414 learned_addr learned my addr 172.21.15.81:0/3556286414 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:52.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.977+0000 7f55f55aa640 1 --2- 172.21.15.81:0/3556286414 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55f00adcd0 0x7f55f00ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.977+0000 7f55f5dab640 1 -- 172.21.15.81:0/3556286414 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55f00adcd0 msgr2=0x7f55f00ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:52.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.977+0000 7f55f5dab640 1 --2- 172.21.15.81:0/3556286414 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55f00adcd0 0x7f55f00ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.977+0000 7f55f5dab640 1 -- 172.21.15.81:0/3556286414 >> v1:172.21.15.93:6789/0 conn(0x7f55f00cc5e0 legacy=0x7f55f00cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:52.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.977+0000 7f55f5dab640 1 -- 172.21.15.81:0/3556286414 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f006d350 con 0x7f55f0070050 2023-11-08T16:38:52.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.977+0000 7f55f55aa640 1 --2- 172.21.15.81:0/3556286414 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55f00adcd0 0x7f55f00ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:52.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.978+0000 7f55f5dab640 1 --2- 172.21.15.81:0/3556286414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f0070050 0x7f55f00704e0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009e50 tx=0x7f55e802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d52f6424e008ca3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:52.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.978+0000 7f55e7fff640 1 -- 172.21.15.81:0/3556286414 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e802ca70 con 0x7f55f0070050 2023-11-08T16:38:52.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.979+0000 7f55e7fff640 1 -- 172.21.15.81:0/3556286414 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e802cbd0 con 0x7f55f0070050 2023-11-08T16:38:52.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.979+0000 7f55e7fff640 1 -- 172.21.15.81:0/3556286414 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e80343f0 con 0x7f55f0070050 2023-11-08T16:38:52.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.979+0000 7f55f780e640 1 -- 172.21.15.81:0/3556286414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f0070050 msgr2=0x7f55f00704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.979+0000 7f55f780e640 1 --2- 172.21.15.81:0/3556286414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f0070050 0x7f55f00704e0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f55e8009e50 tx=0x7f55e802bed0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.979+0000 7f55f780e640 1 -- 172.21.15.81:0/3556286414 shutdown_connections 2023-11-08T16:38:52.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.979+0000 7f55f780e640 1 --2- 172.21.15.81:0/3556286414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f0070050 0x7f55f00704e0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.979+0000 7f55f780e640 1 --2- 172.21.15.81:0/3556286414 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55f00adcd0 0x7f55f00ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.979+0000 7f55f780e640 1 -- 172.21.15.81:0/3556286414 >> 172.21.15.81:0/3556286414 conn(0x7f55f00cb490 msgr2=0x7f55f00cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:52.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.979+0000 7f55f780e640 1 -- 172.21.15.81:0/3556286414 shutdown_connections 2023-11-08T16:38:52.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.979+0000 7f55f780e640 1 -- 172.21.15.81:0/3556286414 wait complete. 2023-11-08T16:38:52.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.981+0000 7f55f780e640 1 Processor -- start 2023-11-08T16:38:52.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.981+0000 7f55f780e640 1 -- start start 2023-11-08T16:38:52.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.982+0000 7f55f780e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55f0070050 0x7f55f0154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.982+0000 7f55f780e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55f00adcd0 0x7f55f0157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.982+0000 7f55f780e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f00cc5e0 0x7f55f00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.982+0000 7f55f780e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f0157be0 con 0x7f55f00cc5e0 2023-11-08T16:38:52.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.982+0000 7f55f780e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00b36e0 con 0x7f55f00adcd0 2023-11-08T16:38:52.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.982+0000 7f55f780e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00b39c0 con 0x7f55f0070050 2023-11-08T16:38:52.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.982+0000 7f55f5dab640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f00cc5e0 0x7f55f00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.982+0000 7f55f5dab640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f00cc5e0 0x7f55f00ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55716/0 (socket says 172.21.15.81:55716) 2023-11-08T16:38:52.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.982+0000 7f55f5dab640 1 -- 172.21.15.81:0/241025204 learned_addr learned my addr 172.21.15.81:0/241025204 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:52.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55f55aa640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55f0070050 0x7f55f0154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55f4da9640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55f00adcd0 0x7f55f0157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55f5dab640 1 -- 172.21.15.81:0/241025204 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55f0070050 msgr2=0x7f55f0154fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55f5dab640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55f0070050 0x7f55f0154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55f5dab640 1 -- 172.21.15.81:0/241025204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55f00adcd0 msgr2=0x7f55f0157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:52.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55f5dab640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55f00adcd0 0x7f55f0157290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:52.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55f5dab640 1 -- 172.21.15.81:0/241025204 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e8009b30 con 0x7f55f00cc5e0 2023-11-08T16:38:52.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55f5dab640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f00cc5e0 0x7f55f00ac8b0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f55e8036040 tx=0x7f55e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:52.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55e5ffb640 1 -- 172.21.15.81:0/241025204 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e803e450 con 0x7f55f00cc5e0 2023-11-08T16:38:52.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55e5ffb640 1 -- 172.21.15.81:0/241025204 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e803e5d0 con 0x7f55f00cc5e0 2023-11-08T16:38:52.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55e5ffb640 1 -- 172.21.15.81:0/241025204 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e803e980 con 0x7f55f00cc5e0 2023-11-08T16:38:52.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.983+0000 7f55f780e640 1 -- 172.21.15.81:0/241025204 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f00b3ca0 con 0x7f55f00cc5e0 2023-11-08T16:38:52.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.984+0000 7f55f780e640 1 -- 172.21.15.81:0/241025204 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f01554e0 con 0x7f55f00cc5e0 2023-11-08T16:38:52.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.985+0000 7f55f780e640 1 -- 172.21.15.81:0/241025204 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f0072170 con 0x7f55f00cc5e0 2023-11-08T16:38:52.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.986+0000 7f55e5ffb640 1 -- 172.21.15.81:0/241025204 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f55e803eae0 con 0x7f55f00cc5e0 2023-11-08T16:38:52.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.986+0000 7f55e5ffb640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f55d406ae70 0x7f55d406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:52.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.986+0000 7f55e5ffb640 1 -- 172.21.15.81:0/241025204 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7f55e80ac8a0 con 0x7f55f00cc5e0 2023-11-08T16:38:52.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.987+0000 7f55f55aa640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f55d406ae70 0x7f55d406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:52.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.987+0000 7f55f55aa640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f55d406ae70 0x7f55d406d330 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b840 tx=0x7f55ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:52.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:52.991+0000 7f55e5ffb640 1 -- 172.21.15.81:0/241025204 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f55e80775e0 con 0x7f55f00cc5e0 2023-11-08T16:38:53.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.223+0000 7f55f780e640 1 -- 172.21.15.81:0/241025204 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55f00c6b30 con 0x7f55f00cc5e0 2023-11-08T16:38:53.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.224+0000 7f55e5ffb640 1 -- 172.21.15.81:0/241025204 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f55f00c6b30 con 0x7f55f00cc5e0 2023-11-08T16:38:53.227 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:53.227 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":78,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1527116581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1527116581},{"type":"v1","addr":"172.21.15.93:6833","nonce":1527116581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2617548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2617548066},{"type":"v1","addr":"172.21.15.93:6835","nonce":2617548066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":7380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2591582634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2591582634},{"type":"v1","addr":"172.21.15.81:6835","nonce":2591582634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:09.004709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/1843050442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1843050442},{"type":"v1","addr":"172.21.15.81:6837","nonce":1843050442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:53.228 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 78 2023-11-08T16:38:53.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.227+0000 7f55f780e640 1 -- 172.21.15.81:0/241025204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f55d406ae70 msgr2=0x7f55d406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.227+0000 7f55f780e640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f55d406ae70 0x7f55d406d330 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b840 tx=0x7f55ec008040 comp rx=0 tx=0).stop 2023-11-08T16:38:53.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.227+0000 7f55f780e640 1 -- 172.21.15.81:0/241025204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f00cc5e0 msgr2=0x7f55f00ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.227+0000 7f55f780e640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f00cc5e0 0x7f55f00ac8b0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f55e8036040 tx=0x7f55e8004a40 comp rx=0 tx=0).stop 2023-11-08T16:38:53.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.227+0000 7f55f780e640 1 -- 172.21.15.81:0/241025204 shutdown_connections 2023-11-08T16:38:53.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.227+0000 7f55f780e640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f55d406ae70 0x7f55d406d330 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.227+0000 7f55f780e640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55f00cc5e0 0x7f55f00ac8b0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.227+0000 7f55f780e640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55f00adcd0 0x7f55f0157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.227+0000 7f55f780e640 1 --2- 172.21.15.81:0/241025204 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55f0070050 0x7f55f0154fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.227+0000 7f55f780e640 1 -- 172.21.15.81:0/241025204 >> 172.21.15.81:0/241025204 conn(0x7f55f00cb490 msgr2=0x7f55f00c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:53.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.227+0000 7f55f780e640 1 -- 172.21.15.81:0/241025204 shutdown_connections 2023-11-08T16:38:53.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.228+0000 7f55f780e640 1 -- 172.21.15.81:0/241025204 wait complete. 2023-11-08T16:38:53.249 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:38:53.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.454+0000 7fef07db6640 1 Processor -- start 2023-11-08T16:38:53.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.454+0000 7fef07db6640 1 -- start start 2023-11-08T16:38:53.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.455+0000 7fef07db6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0006bdf0 0x7fef0006c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.455+0000 7fef07db6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef0006b630 con 0x7fef000ca480 2023-11-08T16:38:53.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.455+0000 7fef07db6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef000ca910 con 0x7fef000ade50 2023-11-08T16:38:53.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.455+0000 7fef07db6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef000b1860 con 0x7fef0006bdf0 2023-11-08T16:38:53.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.456+0000 7fef05351640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fef000ca480 0x7fef0015ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54902/0 (socket says 172.21.15.81:54902) 2023-11-08T16:38:53.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.456+0000 7fef05351640 1 -- 172.21.15.81:0/3293121129 learned_addr learned my addr 172.21.15.81:0/3293121129 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:53.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.456+0000 7fef06353640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0006bdf0 0x7fef0006c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.456+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723767065 0 0) 0x7fef0006b630 con 0x7fef000ca480 2023-11-08T16:38:53.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.456+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeec003520 con 0x7fef000ca480 2023-11-08T16:38:53.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 205307792 0 0) 0x7feeec003520 con 0x7fef000ca480 2023-11-08T16:38:53.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0006bdf0 msgr2=0x7fef0006c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:53.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0006bdf0 0x7fef0006c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 >> v1:172.21.15.93:6789/0 conn(0x7fef000ade50 legacy=0x7fef0015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000cabb0 con 0x7fef000ca480 2023-11-08T16:38:53.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7feef0001670 con 0x7fef000ca480 2023-11-08T16:38:53.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 >> v1:172.21.15.81:6789/0 conn(0x7fef000ca480 legacy=0x7fef0015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 0x7feeec003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feeec0047c0 0x7feeec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0006bdf0 0x7feeec005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec00ff20 con 0x7feeec003a80 2023-11-08T16:38:53.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec010060 con 0x7feeec0047c0 2023-11-08T16:38:53.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec010220 con 0x7fef0006bdf0 2023-11-08T16:38:53.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef05b52640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 0x7feeec003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.457+0000 7fef05351640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feeec0047c0 0x7feeec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.458+0000 7fef05b52640 1 -- 172.21.15.81:0/3293121129 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0006bdf0 msgr2=0x7feeec005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:53.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.458+0000 7fef05b52640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0006bdf0 0x7feeec005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.458+0000 7fef05b52640 1 -- 172.21.15.81:0/3293121129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feeec0047c0 msgr2=0x7feeec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:53.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.458+0000 7fef05b52640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feeec0047c0 0x7feeec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.458+0000 7fef05b52640 1 -- 172.21.15.81:0/3293121129 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec003520 con 0x7feeec003a80 2023-11-08T16:38:53.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.458+0000 7fef05351640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feeec0047c0 0x7feeec004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:38:53.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.459+0000 7fef05b52640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 0x7feeec003ea0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7feefc001750 tx=0x7feeec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdcf9c48bb0719e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:53.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.459+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0052d0 con 0x7feeec003a80 2023-11-08T16:38:53.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.459+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc02e050 con 0x7feeec003a80 2023-11-08T16:38:53.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.459+0000 7fef04b50640 1 -- 172.21.15.81:0/3293121129 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0035f0 con 0x7feeec003a80 2023-11-08T16:38:53.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.460+0000 7fef07db6640 1 -- 172.21.15.81:0/3293121129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 msgr2=0x7feeec003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.460+0000 7fef07db6640 1 --2- 172.21.15.81:0/3293121129 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 0x7feeec003ea0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7feefc001750 tx=0x7feeec0057f0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.460+0000 7fef06353640 1 -- 172.21.15.81:0/3293121129 reap_dead start 2023-11-08T16:38:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.460+0000 7fef07db6640 1 -- 172.21.15.81:0/3293121129 shutdown_connections 2023-11-08T16:38:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.460+0000 7fef07db6640 1 -- 172.21.15.81:0/3293121129 >> 172.21.15.81:0/3293121129 conn(0x7fef000cb490 msgr2=0x7fef000cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:53.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.460+0000 7fef07db6640 1 -- 172.21.15.81:0/3293121129 shutdown_connections 2023-11-08T16:38:53.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.460+0000 7fef07db6640 1 -- 172.21.15.81:0/3293121129 wait complete. 2023-11-08T16:38:53.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.461+0000 7fef07db6640 1 Processor -- start 2023-11-08T16:38:53.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.462+0000 7fef07db6640 1 -- start start 2023-11-08T16:38:53.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.462+0000 7fef07db6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 0x7fef000ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef07db6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef0014e400 0x7fef000c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef07db6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0014de40 0x7fef000c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef07db6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef000c5730 con 0x7feeec003a80 2023-11-08T16:38:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef07db6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef001557c0 con 0x7fef0014e400 2023-11-08T16:38:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef07db6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef00154f20 con 0x7fef0014de40 2023-11-08T16:38:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef05b52640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 0x7fef000ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef05b52640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 0x7fef000ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55730/0 (socket says 172.21.15.81:55730) 2023-11-08T16:38:53.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef05b52640 1 -- 172.21.15.81:0/3265219706 learned_addr learned my addr 172.21.15.81:0/3265219706 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:53.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef05351640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef0014e400 0x7fef000c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef06353640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0014de40 0x7fef000c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef05b52640 1 -- 172.21.15.81:0/3265219706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0014de40 msgr2=0x7fef000c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef05b52640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0014de40 0x7fef000c51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef05b52640 1 -- 172.21.15.81:0/3265219706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef0014e400 msgr2=0x7fef000c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef05b52640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef0014e400 0x7fef000c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef05b52640 1 -- 172.21.15.81:0/3265219706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef00155200 con 0x7feeec003a80 2023-11-08T16:38:53.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.463+0000 7fef05351640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef0014e400 0x7fef000c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:53.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.464+0000 7fef05b52640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 0x7fef000ccb60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7feefc008fd0 tx=0x7feefc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:53.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.464+0000 7feef67fc640 1 -- 172.21.15.81:0/3265219706 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0076c0 con 0x7feeec003a80 2023-11-08T16:38:53.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.464+0000 7feef67fc640 1 -- 172.21.15.81:0/3265219706 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc007a30 con 0x7feeec003a80 2023-11-08T16:38:53.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.464+0000 7fef07db6640 1 -- 172.21.15.81:0/3265219706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef001554e0 con 0x7feeec003a80 2023-11-08T16:38:53.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.464+0000 7feef67fc640 1 -- 172.21.15.81:0/3265219706 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc006c00 con 0x7feeec003a80 2023-11-08T16:38:53.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.464+0000 7fef07db6640 1 -- 172.21.15.81:0/3265219706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef000cd0a0 con 0x7feeec003a80 2023-11-08T16:38:53.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.465+0000 7fef07db6640 1 -- 172.21.15.81:0/3265219706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef000cd230 con 0x7feeec003a80 2023-11-08T16:38:53.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.466+0000 7feef67fc640 1 -- 172.21.15.81:0/3265219706 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feefc0035f0 con 0x7feeec003a80 2023-11-08T16:38:53.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.467+0000 7feef67fc640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feee406ae70 0x7feee406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.467+0000 7feef67fc640 1 -- 172.21.15.81:0/3265219706 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7feefc0ab720 con 0x7feeec003a80 2023-11-08T16:38:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.467+0000 7fef05351640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feee406ae70 0x7feee406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.468+0000 7fef05351640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feee406ae70 0x7feee406d330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7feef0001eb0 tx=0x7feef0000ef0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:53.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.472+0000 7feef67fc640 1 -- 172.21.15.81:0/3265219706 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7feefc07a010 con 0x7feeec003a80 2023-11-08T16:38:53.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.696+0000 7fef07db6640 1 -- 172.21.15.81:0/3265219706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef0004ece0 con 0x7feeec003a80 2023-11-08T16:38:53.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.698+0000 7feef67fc640 1 -- 172.21.15.81:0/3265219706 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+4919 (secure 0 0 0) 0x7feefc079ba0 con 0x7feeec003a80 2023-11-08T16:38:53.701 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:53.701 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":78,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1527116581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1527116581},{"type":"v1","addr":"172.21.15.93:6833","nonce":1527116581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2617548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2617548066},{"type":"v1","addr":"172.21.15.93:6835","nonce":2617548066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":7380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2591582634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2591582634},{"type":"v1","addr":"172.21.15.81:6835","nonce":2591582634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":78,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:37:59.633801+0000","modified":"2023-11-08T16:38:09.004709+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7383},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7383":{"gid":7383,"name":"a","rank":0,"incarnation":75,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/1843050442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1843050442},{"type":"v1","addr":"172.21.15.81:6837","nonce":1843050442}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11,12],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-08T16:38:53.701 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 78 2023-11-08T16:38:53.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.700+0000 7fef07db6640 1 -- 172.21.15.81:0/3265219706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feee406ae70 msgr2=0x7feee406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.700+0000 7fef07db6640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feee406ae70 0x7feee406d330 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7feef0001eb0 tx=0x7feef0000ef0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.701+0000 7fef07db6640 1 -- 172.21.15.81:0/3265219706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 msgr2=0x7fef000ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.701+0000 7fef07db6640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 0x7fef000ccb60 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7feefc008fd0 tx=0x7feefc001120 comp rx=0 tx=0).stop 2023-11-08T16:38:53.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.701+0000 7fef07db6640 1 -- 172.21.15.81:0/3265219706 shutdown_connections 2023-11-08T16:38:53.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.701+0000 7fef07db6640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feee406ae70 0x7feee406d330 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.701+0000 7fef07db6640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fef0014de40 0x7fef000c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.701+0000 7fef07db6640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef0014e400 0x7fef000c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.701+0000 7fef07db6640 1 --2- 172.21.15.81:0/3265219706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feeec003a80 0x7fef000ccb60 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.701+0000 7fef07db6640 1 -- 172.21.15.81:0/3265219706 >> 172.21.15.81:0/3265219706 conn(0x7fef000cb490 msgr2=0x7fef000b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:53.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.701+0000 7fef07db6640 1 -- 172.21.15.81:0/3265219706 shutdown_connections 2023-11-08T16:38:53.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.701+0000 7fef07db6640 1 -- 172.21.15.81:0/3265219706 wait complete. 2023-11-08T16:38:53.721 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:38:53.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.925+0000 7ff083fff640 1 Processor -- start 2023-11-08T16:38:53.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.925+0000 7ff083fff640 1 -- start start 2023-11-08T16:38:53.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.927+0000 7ff083fff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff0840c9d00 0x7ff0840b2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.927+0000 7ff083fff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0840caec0 con 0x7ff0840cf230 2023-11-08T16:38:53.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.927+0000 7ff083fff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0840ca2b0 con 0x7ff084159b00 2023-11-08T16:38:53.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.927+0000 7ff083fff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0840ca590 con 0x7ff0840c9d00 2023-11-08T16:38:53.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.927+0000 7ff07a7fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff0840cf230 0x7ff0840c95f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54908/0 (socket says 172.21.15.81:54908) 2023-11-08T16:38:53.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.927+0000 7ff07a7fc640 1 -- 172.21.15.81:0/2075654239 learned_addr learned my addr 172.21.15.81:0/2075654239 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:53.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0837fe640 1 --2- 172.21.15.81:0/2075654239 >> v2:172.21.15.93:3300/0 conn(0x7ff0840c9d00 0x7ff0840b2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743887875 0 0) 0x7ff0840caec0 con 0x7ff0840cf230 2023-11-08T16:38:53.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff070003520 con 0x7ff0840cf230 2023-11-08T16:38:53.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 357687581 0 0) 0x7ff070003520 con 0x7ff0840cf230 2023-11-08T16:38:53.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 >> v2:172.21.15.93:3300/0 conn(0x7ff0840c9d00 msgr2=0x7ff0840b2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:53.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 --2- 172.21.15.81:0/2075654239 >> v2:172.21.15.93:3300/0 conn(0x7ff0840c9d00 0x7ff0840b2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 >> v1:172.21.15.93:6789/0 conn(0x7ff084159b00 legacy=0x7ff0840ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0840ca870 con 0x7ff0840cf230 2023-11-08T16:38:53.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ff06c001670 con 0x7ff0840cf230 2023-11-08T16:38:53.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 >> v1:172.21.15.81:6789/0 conn(0x7ff0840cf230 legacy=0x7ff0840c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 --2- 172.21.15.81:0/2075654239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff070003970 0x7ff070003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 --2- 172.21.15.81:0/2075654239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0700047c0 0x7ff070004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 --2- 172.21.15.81:0/2075654239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 0x7ff07000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840ca590 con 0x7ff070005180 2023-11-08T16:38:53.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07000c0d0 con 0x7ff0700047c0 2023-11-08T16:38:53.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07000c290 con 0x7ff070003970 2023-11-08T16:38:53.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.928+0000 7ff0837fe640 1 --2- 172.21.15.81:0/2075654239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 0x7ff07000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.929+0000 7ff07a7fc640 1 --2- 172.21.15.81:0/2075654239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0700047c0 0x7ff070004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.929+0000 7ff082ffd640 1 --2- 172.21.15.81:0/2075654239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff070003970 0x7ff070003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.929+0000 7ff0837fe640 1 -- 172.21.15.81:0/2075654239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff070003970 msgr2=0x7ff070003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:53.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.929+0000 7ff0837fe640 1 --2- 172.21.15.81:0/2075654239 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff070003970 0x7ff070003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.929+0000 7ff0837fe640 1 -- 172.21.15.81:0/2075654239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0700047c0 msgr2=0x7ff070004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:53.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.929+0000 7ff0837fe640 1 --2- 172.21.15.81:0/2075654239 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0700047c0 0x7ff070004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.929+0000 7ff0837fe640 1 -- 172.21.15.81:0/2075654239 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff070003520 con 0x7ff070005180 2023-11-08T16:38:53.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.930+0000 7ff0837fe640 1 -- 172.21.15.81:0/2075654239 reap_dead start 2023-11-08T16:38:53.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.930+0000 7ff0837fe640 1 --2- 172.21.15.81:0/2075654239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 0x7ff07000b980 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff07c007360 tx=0x7ff0700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7c1811da31f637 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:53.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.930+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c02ccf0 con 0x7ff070005180 2023-11-08T16:38:53.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.930+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c02ce90 con 0x7ff070005180 2023-11-08T16:38:53.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.930+0000 7ff0827fc640 1 -- 172.21.15.81:0/2075654239 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c003bd0 con 0x7ff070005180 2023-11-08T16:38:53.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.931+0000 7ff083fff640 1 -- 172.21.15.81:0/2075654239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 msgr2=0x7ff07000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.931+0000 7ff083fff640 1 --2- 172.21.15.81:0/2075654239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 0x7ff07000b980 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7ff07c007360 tx=0x7ff0700057f0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.931+0000 7ff083fff640 1 -- 172.21.15.81:0/2075654239 shutdown_connections 2023-11-08T16:38:53.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.931+0000 7ff083fff640 1 --2- 172.21.15.81:0/2075654239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 0x7ff07000b980 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.931+0000 7ff083fff640 1 -- 172.21.15.81:0/2075654239 >> 172.21.15.81:0/2075654239 conn(0x7ff0840b0880 msgr2=0x7ff0840b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:53.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.931+0000 7ff083fff640 1 -- 172.21.15.81:0/2075654239 shutdown_connections 2023-11-08T16:38:53.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.931+0000 7ff083fff640 1 -- 172.21.15.81:0/2075654239 wait complete. 2023-11-08T16:38:53.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.932+0000 7ff083fff640 1 Processor -- start 2023-11-08T16:38:53.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.933+0000 7ff083fff640 1 -- start start 2023-11-08T16:38:53.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.933+0000 7ff083fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 0x7ff0841584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.933+0000 7ff083fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0840cbc60 0x7ff0840cd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.933+0000 7ff083fff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0840c7570 0x7ff0840c7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.933+0000 7ff083fff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840c7f40 con 0x7ff070005180 2023-11-08T16:38:53.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.933+0000 7ff083fff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840cc160 con 0x7ff0840c7570 2023-11-08T16:38:53.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.933+0000 7ff083fff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840cc440 con 0x7ff0840cbc60 2023-11-08T16:38:53.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.934+0000 7ff082ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 0x7ff0841584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.934+0000 7ff082ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 0x7ff0841584f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55752/0 (socket says 172.21.15.81:55752) 2023-11-08T16:38:53.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.934+0000 7ff082ffd640 1 -- 172.21.15.81:0/1483914792 learned_addr learned my addr 172.21.15.81:0/1483914792 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:53.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.934+0000 7ff07a7fc640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0840cbc60 0x7ff0840cd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.934+0000 7ff0837fe640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0840c7570 0x7ff0840c7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.934+0000 7ff082ffd640 1 -- 172.21.15.81:0/1483914792 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0840cbc60 msgr2=0x7ff0840cd710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.934+0000 7ff082ffd640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0840cbc60 0x7ff0840cd710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.934+0000 7ff082ffd640 1 -- 172.21.15.81:0/1483914792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0840c7570 msgr2=0x7ff0840c7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:53.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.934+0000 7ff082ffd640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0840c7570 0x7ff0840c7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:53.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.934+0000 7ff082ffd640 1 -- 172.21.15.81:0/1483914792 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0840cc720 con 0x7ff070005180 2023-11-08T16:38:53.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.934+0000 7ff082ffd640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 0x7ff0841584f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ff07400e4d0 tx=0x7ff07400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:53.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.935+0000 7ff07bfff640 1 -- 172.21.15.81:0/1483914792 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0740052e0 con 0x7ff070005180 2023-11-08T16:38:53.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.935+0000 7ff07bfff640 1 -- 172.21.15.81:0/1483914792 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff074005440 con 0x7ff070005180 2023-11-08T16:38:53.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.935+0000 7ff07bfff640 1 -- 172.21.15.81:0/1483914792 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074003810 con 0x7ff070005180 2023-11-08T16:38:53.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.935+0000 7ff083fff640 1 -- 172.21.15.81:0/1483914792 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0840cca00 con 0x7ff070005180 2023-11-08T16:38:53.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.935+0000 7ff083fff640 1 -- 172.21.15.81:0/1483914792 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0840cdc50 con 0x7ff070005180 2023-11-08T16:38:53.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.936+0000 7ff083fff640 1 -- 172.21.15.81:0/1483914792 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0840ccbf0 con 0x7ff070005180 2023-11-08T16:38:53.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.937+0000 7ff07bfff640 1 -- 172.21.15.81:0/1483914792 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff074003970 con 0x7ff070005180 2023-11-08T16:38:53.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.938+0000 7ff07bfff640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff06406ae70 0x7ff06406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:53.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.938+0000 7ff07bfff640 1 -- 172.21.15.81:0/1483914792 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7ff074089720 con 0x7ff070005180 2023-11-08T16:38:53.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.938+0000 7ff07a7fc640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff06406ae70 0x7ff06406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:53.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.939+0000 7ff07a7fc640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff06406ae70 0x7ff06406d330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff0840cd640 tx=0x7ff06c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:53.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:53.942+0000 7ff07bfff640 1 -- 172.21.15.81:0/1483914792 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff0840ccbf0 con 0x7ff070005180 2023-11-08T16:38:54.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.132+0000 7ff083fff640 1 -- 172.21.15.81:0/1483914792 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0840ad470 con 0x7ff070005180 2023-11-08T16:38:54.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.136+0000 7ff07bfff640 1 -- 172.21.15.81:0/1483914792 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+18770 (secure 0 0 0) 0x7ff0840ad470 con 0x7ff070005180 2023-11-08T16:38:54.139 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:38:54.139 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":90,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:38:07.597635+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:37:30.174436+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-08T16:37:31.340720+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:37:42.279296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.2e","mappings":[{"from":6,"to":0}]},{"pgid":"11.3e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6836/1764545681":"2023-11-09T16:38:06.974694+0000","172.21.15.81:6837/1764545681":"2023-11-09T16:38:06.974694+0000","172.21.15.81:6837/457413856":"2023-11-09T16:38:00.541280+0000","172.21.15.81:6836/457413856":"2023-11-09T16:38:00.541280+0000","172.21.15.81:6837/2162767469":"2023-11-09T16:37:58.508547+0000","172.21.15.81:6836/2162767469":"2023-11-09T16:37:58.508547+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:38:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.138+0000 7ff083fff640 1 -- 172.21.15.81:0/1483914792 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff06406ae70 msgr2=0x7ff06406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.138+0000 7ff083fff640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff06406ae70 0x7ff06406d330 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff0840cd640 tx=0x7ff06c009040 comp rx=0 tx=0).stop 2023-11-08T16:38:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.138+0000 7ff083fff640 1 -- 172.21.15.81:0/1483914792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 msgr2=0x7ff0841584f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.138+0000 7ff083fff640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 0x7ff0841584f0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7ff07400e4d0 tx=0x7ff07400e990 comp rx=0 tx=0).stop 2023-11-08T16:38:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.138+0000 7ff083fff640 1 -- 172.21.15.81:0/1483914792 shutdown_connections 2023-11-08T16:38:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.138+0000 7ff083fff640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff06406ae70 0x7ff06406d330 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.138+0000 7ff083fff640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff0840c7570 0x7ff0840c7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.138+0000 7ff083fff640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff0840cbc60 0x7ff0840cd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.138+0000 7ff083fff640 1 --2- 172.21.15.81:0/1483914792 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff070005180 0x7ff0841584f0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.138+0000 7ff083fff640 1 -- 172.21.15.81:0/1483914792 >> 172.21.15.81:0/1483914792 conn(0x7ff0840b0880 msgr2=0x7ff08415a240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:54.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.139+0000 7ff083fff640 1 -- 172.21.15.81:0/1483914792 shutdown_connections 2023-11-08T16:38:54.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.139+0000 7ff083fff640 1 -- 172.21.15.81:0/1483914792 wait complete. 2023-11-08T16:38:54.161 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:38:54.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.373+0000 7efe0e444640 1 Processor -- start 2023-11-08T16:38:54.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.373+0000 7efe0e444640 1 -- start start 2023-11-08T16:38:54.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.374+0000 7efe0e444640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efe080cf720 0x7efe0806d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.374+0000 7efe0e444640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe080ca130 0x7efe08070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.374+0000 7efe0e444640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0806cc50 con 0x7efe0815d3b0 2023-11-08T16:38:54.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.374+0000 7efe0e444640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe080c90f0 con 0x7efe080cf720 2023-11-08T16:38:54.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.374+0000 7efe0e444640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe080adb70 con 0x7efe080ca130 2023-11-08T16:38:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.375+0000 7efe077fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efe080cf720 0x7efe0806d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.375+0000 7efe077fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efe080cf720 0x7efe0806d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55764/0 (socket says 172.21.15.81:55764) 2023-11-08T16:38:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.375+0000 7efe077fe640 1 -- 172.21.15.81:0/3099424151 learned_addr learned my addr 172.21.15.81:0/3099424151 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.375+0000 7efe0c9e1640 1 --2- 172.21.15.81:0/3099424151 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe080ca130 0x7efe08070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.375+0000 7efe077fe640 1 -- 172.21.15.81:0/3099424151 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe080ca130 msgr2=0x7efe08070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.375+0000 7efe077fe640 1 --2- 172.21.15.81:0/3099424151 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe080ca130 0x7efe08070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.375+0000 7efe077fe640 1 -- 172.21.15.81:0/3099424151 >> v1:172.21.15.93:6789/0 conn(0x7efe0815d3b0 legacy=0x7efe080c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.375+0000 7efe077fe640 1 -- 172.21.15.81:0/3099424151 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe080c9360 con 0x7efe080cf720 2023-11-08T16:38:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.377+0000 7efe077fe640 1 --2- 172.21.15.81:0/3099424151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe080cf720 0x7efe0806d0e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00bb10 tx=0x7efdfc02e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd0f47f9b8432f82 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.377+0000 7efe06ffd640 1 -- 172.21.15.81:0/3099424151 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc036440 con 0x7efe080cf720 2023-11-08T16:38:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.377+0000 7efe06ffd640 1 -- 172.21.15.81:0/3099424151 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc0365a0 con 0x7efe080cf720 2023-11-08T16:38:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.377+0000 7efe06ffd640 1 -- 172.21.15.81:0/3099424151 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc036930 con 0x7efe080cf720 2023-11-08T16:38:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.378+0000 7efe0e444640 1 -- 172.21.15.81:0/3099424151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe080cf720 msgr2=0x7efe0806d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.378+0000 7efe0e444640 1 --2- 172.21.15.81:0/3099424151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe080cf720 0x7efe0806d0e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00bb10 tx=0x7efdfc02e150 comp rx=0 tx=0).stop 2023-11-08T16:38:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.378+0000 7efe0e444640 1 -- 172.21.15.81:0/3099424151 shutdown_connections 2023-11-08T16:38:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.378+0000 7efe0e444640 1 --2- 172.21.15.81:0/3099424151 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe080ca130 0x7efe08070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.378+0000 7efe0e444640 1 --2- 172.21.15.81:0/3099424151 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe080cf720 0x7efe0806d0e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.378+0000 7efe0e444640 1 -- 172.21.15.81:0/3099424151 >> 172.21.15.81:0/3099424151 conn(0x7efe080ceaa0 msgr2=0x7efe080cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:54.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.378+0000 7efe0e444640 1 -- 172.21.15.81:0/3099424151 shutdown_connections 2023-11-08T16:38:54.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.378+0000 7efe0e444640 1 -- 172.21.15.81:0/3099424151 wait complete. 2023-11-08T16:38:54.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.380+0000 7efe0e444640 1 Processor -- start 2023-11-08T16:38:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.380+0000 7efe0e444640 1 -- start start 2023-11-08T16:38:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.381+0000 7efe0e444640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe080ca130 0x7efe080c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.381+0000 7efe0e444640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe080cf720 0x7efe08149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.381+0000 7efe0e444640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0815d3b0 0x7efe080c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.381+0000 7efe0e444640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe080c7c70 con 0x7efe080cf720 2023-11-08T16:38:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.381+0000 7efe0e444640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe080c7060 con 0x7efe0815d3b0 2023-11-08T16:38:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.381+0000 7efe0e444640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe080c7340 con 0x7efe080ca130 2023-11-08T16:38:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.381+0000 7efe077fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe080cf720 0x7efe08149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.382+0000 7efe077fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe080cf720 0x7efe08149e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55766/0 (socket says 172.21.15.81:55766) 2023-11-08T16:38:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.382+0000 7efe077fe640 1 -- 172.21.15.81:0/1458934035 learned_addr learned my addr 172.21.15.81:0/1458934035 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.382+0000 7efe0c9e1640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0815d3b0 0x7efe080c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.382+0000 7efe077fe640 1 -- 172.21.15.81:0/1458934035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe080ca130 msgr2=0x7efe080c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.382+0000 7efe07fff640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe080ca130 0x7efe080c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.382+0000 7efe077fe640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe080ca130 0x7efe080c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.382+0000 7efe077fe640 1 -- 172.21.15.81:0/1458934035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0815d3b0 msgr2=0x7efe080c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.382+0000 7efe077fe640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0815d3b0 0x7efe080c25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.382+0000 7efe077fe640 1 -- 172.21.15.81:0/1458934035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdfc00b770 con 0x7efe080cf720 2023-11-08T16:38:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.382+0000 7efe0c9e1640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0815d3b0 0x7efe080c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.382+0000 7efe077fe640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe080cf720 0x7efe08149e60 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7efdfc02e190 tx=0x7efdfc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.383+0000 7efe04ff9640 1 -- 172.21.15.81:0/1458934035 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc036440 con 0x7efe080cf720 2023-11-08T16:38:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.383+0000 7efe04ff9640 1 -- 172.21.15.81:0/1458934035 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdfc034ba0 con 0x7efe080cf720 2023-11-08T16:38:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.383+0000 7efe0e444640 1 -- 172.21.15.81:0/1458934035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe080c7620 con 0x7efe080cf720 2023-11-08T16:38:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.383+0000 7efe04ff9640 1 -- 172.21.15.81:0/1458934035 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdfc041460 con 0x7efe080cf720 2023-11-08T16:38:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.383+0000 7efe0e444640 1 -- 172.21.15.81:0/1458934035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe080c6ac0 con 0x7efe080cf720 2023-11-08T16:38:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.384+0000 7efe0e444640 1 -- 172.21.15.81:0/1458934035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe080c7de0 con 0x7efe080cf720 2023-11-08T16:38:54.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.385+0000 7efe04ff9640 1 -- 172.21.15.81:0/1458934035 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efdfc049360 con 0x7efe080cf720 2023-11-08T16:38:54.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.386+0000 7efe04ff9640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efdf406ae70 0x7efdf406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.386+0000 7efe04ff9640 1 -- 172.21.15.81:0/1458934035 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 6429+0+0 (secure 0 0 0) 0x7efdfc0abd10 con 0x7efe080cf720 2023-11-08T16:38:54.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.387+0000 7efe07fff640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efdf406ae70 0x7efdf406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.387+0000 7efe07fff640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efdf406ae70 0x7efdf406d330 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efdf8000fb0 tx=0x7efdf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:54.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.391+0000 7efe04ff9640 1 -- 172.21.15.81:0/1458934035 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efdfc0b1020 con 0x7efe080cf720 2023-11-08T16:38:54.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.615+0000 7efe0e444640 1 -- 172.21.15.81:0/1458934035 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7efe080743f0 con 0x7efe080cf720 2023-11-08T16:38:54.619 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:38:54.617+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:38:54.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.620+0000 7efe04ff9640 1 -- 172.21.15.81:0/1458934035 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v79) v1 ==== 161+0+0 (secure 0 0 0) 0x7efdfc03b090 con 0x7efe080cf720 2023-11-08T16:38:54.623 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:38:54.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.623+0000 7efe0e444640 1 -- 172.21.15.81:0/1458934035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efdf406ae70 msgr2=0x7efdf406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:54.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.623+0000 7efe0e444640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efdf406ae70 0x7efdf406d330 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7efdf8000fb0 tx=0x7efdf8009040 comp rx=0 tx=0).stop 2023-11-08T16:38:54.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.623+0000 7efe0e444640 1 -- 172.21.15.81:0/1458934035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe080cf720 msgr2=0x7efe08149e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:54.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.623+0000 7efe0e444640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe080cf720 0x7efe08149e60 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7efdfc02e190 tx=0x7efdfc0046c0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.623+0000 7efe0e444640 1 -- 172.21.15.81:0/1458934035 shutdown_connections 2023-11-08T16:38:54.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.623+0000 7efe0e444640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efdf406ae70 0x7efdf406d330 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.623+0000 7efe0e444640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0815d3b0 0x7efe080c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.623+0000 7efe0e444640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efe080cf720 0x7efe08149e60 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.623+0000 7efe0e444640 1 --2- 172.21.15.81:0/1458934035 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efe080ca130 0x7efe080c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.623+0000 7efe0e444640 1 -- 172.21.15.81:0/1458934035 >> 172.21.15.81:0/1458934035 conn(0x7efe080ceaa0 msgr2=0x7efe080730a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:54.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.623+0000 7efe0e444640 1 -- 172.21.15.81:0/1458934035 shutdown_connections 2023-11-08T16:38:54.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.624+0000 7efe0e444640 1 -- 172.21.15.81:0/1458934035 wait complete. 2023-11-08T16:38:54.641 INFO:tasks.ceph.mds.a.smithi081.stderr: -4564> 2023-11-08T16:38:07.000+0000 7f5ebc3c1640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-08T16:38:54.644 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T16:38:54.652 INFO:tasks.ceph.mds.a.smithi081.stderr: -2671> 2023-11-08T16:38:14.638+0000 7f5ebc3c1640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-11-08T16:38:54.653 INFO:tasks.ceph.mds.a.smithi081.stderr: -2509> 2023-11-08T16:38:14.639+0000 7f5ebc3c1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55e2d9b11b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:38:54.653 INFO:tasks.ceph.mds.a.smithi081.stderr: -2454> 2023-11-08T16:38:14.640+0000 7f5ebc3c1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55e2d9b74680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(12)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[11]","memoryvalue":"(12)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:38:54.653 INFO:tasks.ceph.mds.a.smithi081.stderr: -2188> 2023-11-08T16:38:14.644+0000 7f5ebc3c1640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v5 ap=5 snaprealm=0x55e2d9b04d80 DIRTYPARENT f(v1 1=0+1) n(v0 rc2023-11-08T16:38:14.369769+0000 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={7446=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55e2d9b11600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(10)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 rc2023-11-08T16:38:14.369769+0000 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:38:54.715 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:38:54.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.837+0000 7f4944916640 1 Processor -- start 2023-11-08T16:38:54.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.837+0000 7f4944916640 1 -- start start 2023-11-08T16:38:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f4944916640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f494015e170 0x7f494015ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f4944916640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f494015e550 0x7f49400b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f4944916640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f49400b0a20 0x7f49400ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f4944916640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49400b0e70 con 0x7f49400b0a20 2023-11-08T16:38:54.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f4944916640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4940071460 con 0x7f494015e170 2023-11-08T16:38:54.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f4944916640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49400703c0 con 0x7f494015e550 2023-11-08T16:38:54.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f493ed9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f49400b0a20 0x7f49400ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f493ed9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f49400b0a20 0x7f49400ad4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55806/0 (socket says 172.21.15.81:55806) 2023-11-08T16:38:54.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f493ed9d640 1 -- 172.21.15.81:0/3247320411 learned_addr learned my addr 172.21.15.81:0/3247320411 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:54.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f493ed9d640 1 -- 172.21.15.81:0/3247320411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f494015e550 msgr2=0x7f49400b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:54.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f493e59c640 1 --2- 172.21.15.81:0/3247320411 >> v2:172.21.15.93:3300/0 conn(0x7f494015e170 0x7f494015ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.839+0000 7f4937fff640 1 --2- 172.21.15.81:0/3247320411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f494015e550 0x7f49400b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.840+0000 7f493ed9d640 1 --2- 172.21.15.81:0/3247320411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f494015e550 0x7f49400b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.840+0000 7f493ed9d640 1 -- 172.21.15.81:0/3247320411 >> v2:172.21.15.93:3300/0 conn(0x7f494015e170 msgr2=0x7f494015ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.840+0000 7f493ed9d640 1 --2- 172.21.15.81:0/3247320411 >> v2:172.21.15.93:3300/0 conn(0x7f494015e170 0x7f494015ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.840+0000 7f493ed9d640 1 -- 172.21.15.81:0/3247320411 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49400706a0 con 0x7f49400b0a20 2023-11-08T16:38:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.840+0000 7f493e59c640 1 --2- 172.21.15.81:0/3247320411 >> v2:172.21.15.93:3300/0 conn(0x7f494015e170 0x7f494015ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.841+0000 7f493ed9d640 1 --2- 172.21.15.81:0/3247320411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49400b0a20 0x7f49400ad4f0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f494006d3e0 tx=0x7f493802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22267a570fba887a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.841+0000 7f493dd9b640 1 -- 172.21.15.81:0/3247320411 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493802ca70 con 0x7f49400b0a20 2023-11-08T16:38:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.841+0000 7f493dd9b640 1 -- 172.21.15.81:0/3247320411 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493802cbd0 con 0x7f49400b0a20 2023-11-08T16:38:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.841+0000 7f493dd9b640 1 -- 172.21.15.81:0/3247320411 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49380343f0 con 0x7f49400b0a20 2023-11-08T16:38:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.842+0000 7f4944916640 1 -- 172.21.15.81:0/3247320411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49400b0a20 msgr2=0x7f49400ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.842+0000 7f4944916640 1 --2- 172.21.15.81:0/3247320411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49400b0a20 0x7f49400ad4f0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f494006d3e0 tx=0x7f493802bed0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.842+0000 7f4944916640 1 -- 172.21.15.81:0/3247320411 shutdown_connections 2023-11-08T16:38:54.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.842+0000 7f4944916640 1 --2- 172.21.15.81:0/3247320411 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f494015e550 0x7f49400b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.842+0000 7f4944916640 1 --2- 172.21.15.81:0/3247320411 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f49400b0a20 0x7f49400ad4f0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.842+0000 7f4944916640 1 --2- 172.21.15.81:0/3247320411 >> v2:172.21.15.93:3300/0 conn(0x7f494015e170 0x7f494015ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.842+0000 7f4944916640 1 -- 172.21.15.81:0/3247320411 >> 172.21.15.81:0/3247320411 conn(0x7f49400cb4e0 msgr2=0x7f49400cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:54.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.842+0000 7f4944916640 1 -- 172.21.15.81:0/3247320411 shutdown_connections 2023-11-08T16:38:54.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.842+0000 7f4944916640 1 -- 172.21.15.81:0/3247320411 wait complete. 2023-11-08T16:38:54.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.844+0000 7f4944916640 1 Processor -- start 2023-11-08T16:38:54.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.844+0000 7f4944916640 1 -- start start 2023-11-08T16:38:54.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.845+0000 7f4944916640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f49400b0a20 0x7f49400b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.845+0000 7f4944916640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494015e170 0x7f49401485b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.845+0000 7f4944916640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f494015e550 0x7f4940148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.845+0000 7f4944916640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400c5ed0 con 0x7f494015e550 2023-11-08T16:38:54.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.845+0000 7f4944916640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49400c61b0 con 0x7f494015e170 2023-11-08T16:38:54.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.845+0000 7f4944916640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49400c6490 con 0x7f49400b0a20 2023-11-08T16:38:54.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.845+0000 7f493ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f494015e550 0x7f4940148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.845+0000 7f493ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f494015e550 0x7f4940148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55816/0 (socket says 172.21.15.81:55816) 2023-11-08T16:38:54.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.845+0000 7f493ed9d640 1 -- 172.21.15.81:0/2781174851 learned_addr learned my addr 172.21.15.81:0/2781174851 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:54.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.846+0000 7f493e59c640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f49400b0a20 0x7f49400b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.846+0000 7f4937fff640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494015e170 0x7f49401485b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.846+0000 7f493ed9d640 1 -- 172.21.15.81:0/2781174851 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f49400b0a20 msgr2=0x7f49400b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:54.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.846+0000 7f493ed9d640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f49400b0a20 0x7f49400b3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.846+0000 7f493ed9d640 1 -- 172.21.15.81:0/2781174851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494015e170 msgr2=0x7f49401485b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:54.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.846+0000 7f493ed9d640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494015e170 0x7f49401485b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:54.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.846+0000 7f493ed9d640 1 -- 172.21.15.81:0/2781174851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4938009b30 con 0x7f494015e550 2023-11-08T16:38:54.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.846+0000 7f493ed9d640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f494015e550 0x7f4940148af0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4938036040 tx=0x7f4938004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:54.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.846+0000 7f4936ffd640 1 -- 172.21.15.81:0/2781174851 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4938009510 con 0x7f494015e550 2023-11-08T16:38:54.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.847+0000 7f4936ffd640 1 -- 172.21.15.81:0/2781174851 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f493803e450 con 0x7f494015e550 2023-11-08T16:38:54.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.847+0000 7f4936ffd640 1 -- 172.21.15.81:0/2781174851 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493803e800 con 0x7f494015e550 2023-11-08T16:38:54.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.847+0000 7f4944916640 1 -- 172.21.15.81:0/2781174851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49400c6770 con 0x7f494015e550 2023-11-08T16:38:54.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.847+0000 7f4944916640 1 -- 172.21.15.81:0/2781174851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49400c9be0 con 0x7f494015e550 2023-11-08T16:38:54.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.848+0000 7f4944916640 1 -- 172.21.15.81:0/2781174851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494004ece0 con 0x7f494015e550 2023-11-08T16:38:54.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.849+0000 7f4936ffd640 1 -- 172.21.15.81:0/2781174851 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f493803e960 con 0x7f494015e550 2023-11-08T16:38:54.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.849+0000 7f4936ffd640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f492006ae70 0x7f492006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:54.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.850+0000 7f4936ffd640 1 -- 172.21.15.81:0/2781174851 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6515+0+0 (secure 0 0 0) 0x7f49380ac800 con 0x7f494015e550 2023-11-08T16:38:54.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.850+0000 7f493e59c640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f492006ae70 0x7f492006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.850+0000 7f493e59c640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f492006ae70 0x7f492006d330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f49400c2220 tx=0x7f4930008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:54.854+0000 7f4936ffd640 1 -- 172.21.15.81:0/2781174851 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4938077740 con 0x7f494015e550 2023-11-08T16:38:55.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.070+0000 7f4944916640 1 -- 172.21.15.81:0/2781174851 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f49400ccc60 con 0x7f494015e550 2023-11-08T16:38:55.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.623+0000 7f4936ffd640 1 -- 172.21.15.81:0/2781174851 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v80) v1 ==== 104+0+0 (secure 0 0 0) 0x7f493807ad60 con 0x7f494015e550 2023-11-08T16:38:55.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.628+0000 7f4944916640 1 -- 172.21.15.81:0/2781174851 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f492006ae70 msgr2=0x7f492006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:55.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.628+0000 7f4944916640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f492006ae70 0x7f492006d330 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f49400c2220 tx=0x7f4930008040 comp rx=0 tx=0).stop 2023-11-08T16:38:55.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.628+0000 7f4944916640 1 -- 172.21.15.81:0/2781174851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f494015e550 msgr2=0x7f4940148af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:55.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.628+0000 7f4944916640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f494015e550 0x7f4940148af0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f4938036040 tx=0x7f4938004a40 comp rx=0 tx=0).stop 2023-11-08T16:38:55.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.628+0000 7f4944916640 1 -- 172.21.15.81:0/2781174851 shutdown_connections 2023-11-08T16:38:55.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.628+0000 7f4944916640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f492006ae70 0x7f492006d330 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:55.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.628+0000 7f4944916640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f494015e550 0x7f4940148af0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:55.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.628+0000 7f4944916640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f494015e170 0x7f49401485b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:55.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.628+0000 7f4944916640 1 --2- 172.21.15.81:0/2781174851 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f49400b0a20 0x7f49400b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:55.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.628+0000 7f4944916640 1 -- 172.21.15.81:0/2781174851 >> 172.21.15.81:0/2781174851 conn(0x7f49400cb4e0 msgr2=0x7f49400cd710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:55.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.629+0000 7f4944916640 1 -- 172.21.15.81:0/2781174851 shutdown_connections 2023-11-08T16:38:55.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.629+0000 7f4944916640 1 -- 172.21.15.81:0/2781174851 wait complete. 2023-11-08T16:38:55.652 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T16:38:55.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.853+0000 7ff139d39640 1 Processor -- start 2023-11-08T16:38:55.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.853+0000 7ff139d39640 1 -- start start 2023-11-08T16:38:55.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.855+0000 7ff139d39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 0x7ff13415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:55.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.855+0000 7ff139d39640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff13415afd0 0x7ff13415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:55.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.855+0000 7ff139d39640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1340b09f0 con 0x7ff1340714b0 2023-11-08T16:38:55.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.855+0000 7ff139d39640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff13406b250 con 0x7ff13415afd0 2023-11-08T16:38:55.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.855+0000 7ff139d39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340ca360 con 0x7ff134071890 2023-11-08T16:38:55.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.856+0000 7ff132ffd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 0x7ff13415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:55.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.856+0000 7ff132ffd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 0x7ff13415e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:57940/0 (socket says 172.21.15.81:57940) 2023-11-08T16:38:55.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.856+0000 7ff132ffd640 1 -- 172.21.15.81:0/3251323385 learned_addr learned my addr 172.21.15.81:0/3251323385 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:55.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.856+0000 7ff133fff640 1 --2- 172.21.15.81:0/3251323385 >> v2:172.21.15.93:3300/0 conn(0x7ff13415afd0 0x7ff13415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:55.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.856+0000 7ff132ffd640 1 -- 172.21.15.81:0/3251323385 >> v2:172.21.15.93:3300/0 conn(0x7ff13415afd0 msgr2=0x7ff13415ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:55.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.856+0000 7ff132ffd640 1 --2- 172.21.15.81:0/3251323385 >> v2:172.21.15.93:3300/0 conn(0x7ff13415afd0 0x7ff13415ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:55.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.856+0000 7ff132ffd640 1 -- 172.21.15.81:0/3251323385 >> v1:172.21.15.93:6789/0 conn(0x7ff1340714b0 legacy=0x7ff13415a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:55.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.856+0000 7ff132ffd640 1 -- 172.21.15.81:0/3251323385 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1340ca640 con 0x7ff134071890 2023-11-08T16:38:55.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.858+0000 7ff132ffd640 1 --2- 172.21.15.81:0/3251323385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 0x7ff13415e9d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff128009870 tx=0x7ff12802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=833580494e1ec23b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:55.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.858+0000 7ff1327fc640 1 -- 172.21.15.81:0/3251323385 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128031640 con 0x7ff134071890 2023-11-08T16:38:55.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.858+0000 7ff1327fc640 1 -- 172.21.15.81:0/3251323385 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1280317a0 con 0x7ff134071890 2023-11-08T16:38:55.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.858+0000 7ff1327fc640 1 -- 172.21.15.81:0/3251323385 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff128031b30 con 0x7ff134071890 2023-11-08T16:38:55.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.859+0000 7ff139d39640 1 -- 172.21.15.81:0/3251323385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 msgr2=0x7ff13415e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:55.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.859+0000 7ff139d39640 1 --2- 172.21.15.81:0/3251323385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 0x7ff13415e9d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff128009870 tx=0x7ff12802b8f0 comp rx=0 tx=0).stop 2023-11-08T16:38:55.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.859+0000 7ff139d39640 1 -- 172.21.15.81:0/3251323385 shutdown_connections 2023-11-08T16:38:55.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.859+0000 7ff139d39640 1 --2- 172.21.15.81:0/3251323385 >> v2:172.21.15.93:3300/0 conn(0x7ff13415afd0 0x7ff13415ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:55.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.859+0000 7ff139d39640 1 --2- 172.21.15.81:0/3251323385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 0x7ff13415e9d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:55.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.859+0000 7ff139d39640 1 -- 172.21.15.81:0/3251323385 >> 172.21.15.81:0/3251323385 conn(0x7ff1340cb4e0 msgr2=0x7ff1340cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:55.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.859+0000 7ff139d39640 1 -- 172.21.15.81:0/3251323385 shutdown_connections 2023-11-08T16:38:55.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.859+0000 7ff139d39640 1 -- 172.21.15.81:0/3251323385 wait complete. 2023-11-08T16:38:55.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.861+0000 7ff139d39640 1 Processor -- start 2023-11-08T16:38:55.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.861+0000 7ff139d39640 1 -- start start 2023-11-08T16:38:55.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.863+0000 7ff139d39640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1340714b0 0x7ff13414fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:55.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.863+0000 7ff139d39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 0x7ff13414f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.863+0000 7ff139d39640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff13415afd0 0x7ff1340c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.863+0000 7ff139d39640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340c6070 con 0x7ff13415afd0 2023-11-08T16:38:55.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.863+0000 7ff139d39640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340c6e90 con 0x7ff1340714b0 2023-11-08T16:38:55.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.863+0000 7ff139d39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1341504e0 con 0x7ff134071890 2023-11-08T16:38:55.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.863+0000 7ff133fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff13415afd0 0x7ff1340c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:55.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.863+0000 7ff133fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff13415afd0 0x7ff1340c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55826/0 (socket says 172.21.15.81:55826) 2023-11-08T16:38:55.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.863+0000 7ff133fff640 1 -- 172.21.15.81:0/1275463144 learned_addr learned my addr 172.21.15.81:0/1275463144 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:55.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.863+0000 7ff1337fe640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1340714b0 0x7ff13414fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:55.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.863+0000 7ff132ffd640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 0x7ff13414f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:55.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.864+0000 7ff133fff640 1 -- 172.21.15.81:0/1275463144 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 msgr2=0x7ff13414f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:55.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.864+0000 7ff133fff640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 0x7ff13414f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:55.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.864+0000 7ff133fff640 1 -- 172.21.15.81:0/1275463144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1340714b0 msgr2=0x7ff13414fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:55.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.864+0000 7ff133fff640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1340714b0 0x7ff13414fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:55.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.864+0000 7ff133fff640 1 -- 172.21.15.81:0/1275463144 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1280094d0 con 0x7ff13415afd0 2023-11-08T16:38:55.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.864+0000 7ff1337fe640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1340714b0 0x7ff13414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:38:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.864+0000 7ff133fff640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff13415afd0 0x7ff1340c5b30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00efc0 tx=0x7ff12c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.864+0000 7ff11bfff640 1 -- 172.21.15.81:0/1275463144 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c014480 con 0x7ff13415afd0 2023-11-08T16:38:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.864+0000 7ff11bfff640 1 -- 172.21.15.81:0/1275463144 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c00f040 con 0x7ff13415afd0 2023-11-08T16:38:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.864+0000 7ff139d39640 1 -- 172.21.15.81:0/1275463144 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1340c6280 con 0x7ff13415afd0 2023-11-08T16:38:55.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.864+0000 7ff11bfff640 1 -- 172.21.15.81:0/1275463144 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c014820 con 0x7ff13415afd0 2023-11-08T16:38:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.865+0000 7ff139d39640 1 -- 172.21.15.81:0/1275463144 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1340c6840 con 0x7ff13415afd0 2023-11-08T16:38:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.866+0000 7ff139d39640 1 -- 172.21.15.81:0/1275463144 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1340c7020 con 0x7ff13415afd0 2023-11-08T16:38:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.866+0000 7ff11bfff640 1 -- 172.21.15.81:0/1275463144 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff12c014980 con 0x7ff13415afd0 2023-11-08T16:38:55.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.867+0000 7ff11bfff640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff12006ae70 0x7ff12006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:55.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.868+0000 7ff11bfff640 1 -- 172.21.15.81:0/1275463144 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 6515+0+0 (secure 0 0 0) 0x7ff12c08a520 con 0x7ff13415afd0 2023-11-08T16:38:55.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.868+0000 7ff1337fe640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff12006ae70 0x7ff12006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:55.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.868+0000 7ff1337fe640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff12006ae70 0x7ff12006d330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff1341f4090 tx=0x7ff124009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:55.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:55.872+0000 7ff11bfff640 1 -- 172.21.15.81:0/1275463144 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff12c055290 con 0x7ff13415afd0 2023-11-08T16:38:56.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:56.052+0000 7ff139d39640 1 -- 172.21.15.81:0/1275463144 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff1340743f0 con 0x7ff13415afd0 2023-11-08T16:38:57.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.024+0000 7ff11bfff640 1 -- 172.21.15.81:0/1275463144 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff12c0588b0 con 0x7ff13415afd0 2023-11-08T16:38:57.028 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' removed 2023-11-08T16:38:57.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.028+0000 7ff139d39640 1 -- 172.21.15.81:0/1275463144 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff12006ae70 msgr2=0x7ff12006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:57.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.028+0000 7ff139d39640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff12006ae70 0x7ff12006d330 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7ff1341f4090 tx=0x7ff124009040 comp rx=0 tx=0).stop 2023-11-08T16:38:57.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.028+0000 7ff139d39640 1 -- 172.21.15.81:0/1275463144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff13415afd0 msgr2=0x7ff1340c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:57.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.028+0000 7ff139d39640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff13415afd0 0x7ff1340c5b30 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00efc0 tx=0x7ff12c00c490 comp rx=0 tx=0).stop 2023-11-08T16:38:57.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.028+0000 7ff139d39640 1 -- 172.21.15.81:0/1275463144 shutdown_connections 2023-11-08T16:38:57.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.029+0000 7ff139d39640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff12006ae70 0x7ff12006d330 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:57.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.029+0000 7ff139d39640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff13415afd0 0x7ff1340c5b30 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:57.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.029+0000 7ff139d39640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff134071890 0x7ff13414f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:57.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.029+0000 7ff139d39640 1 --2- 172.21.15.81:0/1275463144 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff1340714b0 0x7ff13414fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:57.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.029+0000 7ff139d39640 1 -- 172.21.15.81:0/1275463144 >> 172.21.15.81:0/1275463144 conn(0x7ff1340cb4e0 msgr2=0x7ff134159a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:57.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.029+0000 7ff139d39640 1 -- 172.21.15.81:0/1275463144 shutdown_connections 2023-11-08T16:38:57.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.029+0000 7ff139d39640 1 -- 172.21.15.81:0/1275463144 wait complete. 2023-11-08T16:38:57.057 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T16:38:57.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.256+0000 7fbe661b8640 1 Processor -- start 2023-11-08T16:38:57.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.256+0000 7fbe661b8640 1 -- start start 2023-11-08T16:38:57.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.258+0000 7fbe661b8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbe600b4ec0 0x7fbe600b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:57.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.258+0000 7fbe661b8640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbe6015d320 0x7fbe600b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:57.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.258+0000 7fbe661b8640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe600d23c0 con 0x7fbe6015cf40 2023-11-08T16:38:57.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.258+0000 7fbe661b8640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe600b1f30 con 0x7fbe600b4ec0 2023-11-08T16:38:57.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.258+0000 7fbe661b8640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe600cdf10 con 0x7fbe6015d320 2023-11-08T16:38:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.258+0000 7fbe651b6640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbe6015cf40 0x7fbe600b47b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54924/0 (socket says 172.21.15.81:54924) 2023-11-08T16:38:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.258+0000 7fbe649b5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbe600b4ec0 0x7fbe600b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.258+0000 7fbe651b6640 1 -- 172.21.15.81:0/4284850885 learned_addr learned my addr 172.21.15.81:0/4284850885 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:57.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.259+0000 7fbe649b5640 1 -- 172.21.15.81:0/4284850885 >> v2:172.21.15.93:3300/0 conn(0x7fbe6015d320 msgr2=0x7fbe600b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:57.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.259+0000 7fbe659b7640 1 --2- 172.21.15.81:0/4284850885 >> v2:172.21.15.93:3300/0 conn(0x7fbe6015d320 0x7fbe600b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:57.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.259+0000 7fbe649b5640 1 --2- 172.21.15.81:0/4284850885 >> v2:172.21.15.93:3300/0 conn(0x7fbe6015d320 0x7fbe600b5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:57.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.259+0000 7fbe57fff640 1 -- 172.21.15.81:0/4284850885 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794597957 0 0) 0x7fbe600d23c0 con 0x7fbe6015cf40 2023-11-08T16:38:57.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.259+0000 7fbe649b5640 1 -- 172.21.15.81:0/4284850885 >> v1:172.21.15.81:6789/0 conn(0x7fbe6015cf40 legacy=0x7fbe600b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:57.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.259+0000 7fbe649b5640 1 -- 172.21.15.81:0/4284850885 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe600ce1f0 con 0x7fbe600b4ec0 2023-11-08T16:38:57.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.259+0000 7fbe659b7640 1 --2- 172.21.15.81:0/4284850885 >> v2:172.21.15.93:3300/0 conn(0x7fbe6015d320 0x7fbe600b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:38:57.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.260+0000 7fbe649b5640 1 --2- 172.21.15.81:0/4284850885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe600b4ec0 0x7fbe600b21a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fbe6006a520 tx=0x7fbe5c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc1615402b44f444 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:57.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.260+0000 7fbe57fff640 1 -- 172.21.15.81:0/4284850885 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c031830 con 0x7fbe600b4ec0 2023-11-08T16:38:57.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.260+0000 7fbe57fff640 1 -- 172.21.15.81:0/4284850885 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe5c031990 con 0x7fbe600b4ec0 2023-11-08T16:38:57.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.260+0000 7fbe57fff640 1 -- 172.21.15.81:0/4284850885 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe5c031d20 con 0x7fbe600b4ec0 2023-11-08T16:38:57.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.261+0000 7fbe661b8640 1 -- 172.21.15.81:0/4284850885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe600b4ec0 msgr2=0x7fbe600b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:57.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.261+0000 7fbe661b8640 1 --2- 172.21.15.81:0/4284850885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe600b4ec0 0x7fbe600b21a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fbe6006a520 tx=0x7fbe5c02b8d0 comp rx=0 tx=0).stop 2023-11-08T16:38:57.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.261+0000 7fbe661b8640 1 -- 172.21.15.81:0/4284850885 shutdown_connections 2023-11-08T16:38:57.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.261+0000 7fbe661b8640 1 --2- 172.21.15.81:0/4284850885 >> v2:172.21.15.93:3300/0 conn(0x7fbe6015d320 0x7fbe600b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:57.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.261+0000 7fbe661b8640 1 --2- 172.21.15.81:0/4284850885 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe600b4ec0 0x7fbe600b21a0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:57.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.261+0000 7fbe661b8640 1 -- 172.21.15.81:0/4284850885 >> 172.21.15.81:0/4284850885 conn(0x7fbe600cf7b0 msgr2=0x7fbe600cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:57.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.261+0000 7fbe661b8640 1 -- 172.21.15.81:0/4284850885 shutdown_connections 2023-11-08T16:38:57.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.261+0000 7fbe661b8640 1 -- 172.21.15.81:0/4284850885 wait complete. 2023-11-08T16:38:57.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.263+0000 7fbe661b8640 1 Processor -- start 2023-11-08T16:38:57.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.263+0000 7fbe661b8640 1 -- start start 2023-11-08T16:38:57.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe661b8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe600b4ec0 0x7fbe600c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:57.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe661b8640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe6015cf40 0x7fbe600ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:57.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe661b8640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe6015d320 0x7fbe6014d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:57.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe661b8640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe6014dbc0 con 0x7fbe600b4ec0 2023-11-08T16:38:57.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe661b8640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe600c8300 con 0x7fbe6015cf40 2023-11-08T16:38:57.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe661b8640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe600caa80 con 0x7fbe6015d320 2023-11-08T16:38:57.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe651b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe600b4ec0 0x7fbe600c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:57.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe651b6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe600b4ec0 0x7fbe600c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55852/0 (socket says 172.21.15.81:55852) 2023-11-08T16:38:57.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe651b6640 1 -- 172.21.15.81:0/2687108279 learned_addr learned my addr 172.21.15.81:0/2687108279 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:57.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe651b6640 1 -- 172.21.15.81:0/2687108279 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe6015d320 msgr2=0x7fbe6014d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:57.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe651b6640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe6015d320 0x7fbe6014d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:57.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe651b6640 1 -- 172.21.15.81:0/2687108279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe6015cf40 msgr2=0x7fbe600ccc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:57.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe651b6640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe6015cf40 0x7fbe600ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:57.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe651b6640 1 -- 172.21.15.81:0/2687108279 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c009400 con 0x7fbe600b4ec0 2023-11-08T16:38:57.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.264+0000 7fbe651b6640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe600b4ec0 0x7fbe600c7cb0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000c4d0 tx=0x7fbe5000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:57.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.265+0000 7fbe55ffb640 1 -- 172.21.15.81:0/2687108279 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe500074e0 con 0x7fbe600b4ec0 2023-11-08T16:38:57.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.265+0000 7fbe55ffb640 1 -- 172.21.15.81:0/2687108279 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe50007640 con 0x7fbe600b4ec0 2023-11-08T16:38:57.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.265+0000 7fbe55ffb640 1 -- 172.21.15.81:0/2687108279 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe50004a20 con 0x7fbe600b4ec0 2023-11-08T16:38:57.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.265+0000 7fbe661b8640 1 -- 172.21.15.81:0/2687108279 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe600cad60 con 0x7fbe600b4ec0 2023-11-08T16:38:57.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.265+0000 7fbe661b8640 1 -- 172.21.15.81:0/2687108279 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe600cb320 con 0x7fbe600b4ec0 2023-11-08T16:38:57.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.266+0000 7fbe661b8640 1 -- 172.21.15.81:0/2687108279 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe6004ece0 con 0x7fbe600b4ec0 2023-11-08T16:38:57.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.267+0000 7fbe55ffb640 1 -- 172.21.15.81:0/2687108279 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbe50004b80 con 0x7fbe600b4ec0 2023-11-08T16:38:57.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.268+0000 7fbe55ffb640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbe4c06ae70 0x7fbe4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:57.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.268+0000 7fbe55ffb640 1 -- 172.21.15.81:0/2687108279 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 6077+0+0 (secure 0 0 0) 0x7fbe500889d0 con 0x7fbe600b4ec0 2023-11-08T16:38:57.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.268+0000 7fbe649b5640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbe4c06ae70 0x7fbe4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:57.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.269+0000 7fbe649b5640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbe4c06ae70 0x7fbe4c06d330 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c036000 tx=0x7fbe5c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:57.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.272+0000 7fbe55ffb640 1 -- 172.21.15.81:0/2687108279 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe50053870 con 0x7fbe600b4ec0 2023-11-08T16:38:57.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:57.461+0000 7fbe661b8640 1 -- 172.21.15.81:0/2687108279 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbe600cb510 con 0x7fbe600b4ec0 2023-11-08T16:38:58.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.028+0000 7fbe55ffb640 1 -- 172.21.15.81:0/2687108279 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbe50018020 con 0x7fbe600b4ec0 2023-11-08T16:38:58.031 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' removed 2023-11-08T16:38:58.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.031+0000 7fbe661b8640 1 -- 172.21.15.81:0/2687108279 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbe4c06ae70 msgr2=0x7fbe4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:58.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.031+0000 7fbe661b8640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbe4c06ae70 0x7fbe4c06d330 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c036000 tx=0x7fbe5c030000 comp rx=0 tx=0).stop 2023-11-08T16:38:58.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.031+0000 7fbe661b8640 1 -- 172.21.15.81:0/2687108279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe600b4ec0 msgr2=0x7fbe600c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:58.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.031+0000 7fbe661b8640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe600b4ec0 0x7fbe600c7cb0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000c4d0 tx=0x7fbe5000c990 comp rx=0 tx=0).stop 2023-11-08T16:38:58.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.032+0000 7fbe661b8640 1 -- 172.21.15.81:0/2687108279 shutdown_connections 2023-11-08T16:38:58.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.032+0000 7fbe661b8640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbe4c06ae70 0x7fbe4c06d330 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:58.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.032+0000 7fbe661b8640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe6015d320 0x7fbe6014d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:58.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.032+0000 7fbe661b8640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe6015cf40 0x7fbe600ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:58.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.032+0000 7fbe661b8640 1 --2- 172.21.15.81:0/2687108279 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe600b4ec0 0x7fbe600c7cb0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:58.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.032+0000 7fbe661b8640 1 -- 172.21.15.81:0/2687108279 >> 172.21.15.81:0/2687108279 conn(0x7fbe600cf7b0 msgr2=0x7fbe600b2730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:58.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.032+0000 7fbe661b8640 1 -- 172.21.15.81:0/2687108279 shutdown_connections 2023-11-08T16:38:58.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.032+0000 7fbe661b8640 1 -- 172.21.15.81:0/2687108279 wait complete. 2023-11-08T16:38:58.063 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-11-08T16:38:58.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.260+0000 7f20af021640 1 Processor -- start 2023-11-08T16:38:58.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.261+0000 7f20af021640 1 -- start start 2023-11-08T16:38:58.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.262+0000 7f20af021640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f20a815d2d0 0x7f20a80cff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:58.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.262+0000 7f20af021640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f20a80d0d80 0x7f20a80d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:58.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.262+0000 7f20af021640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a80d3a80 con 0x7f20a80d19e0 2023-11-08T16:38:58.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.262+0000 7f20af021640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a80d1e50 con 0x7f20a80d0d80 2023-11-08T16:38:58.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.262+0000 7f20af021640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80d3d20 con 0x7f20a815d2d0 2023-11-08T16:38:58.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.263+0000 7f20acdbd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f20a815d2d0 0x7f20a80cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:58.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.263+0000 7f20acdbd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f20a815d2d0 0x7f20a80cff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:57970/0 (socket says 172.21.15.81:57970) 2023-11-08T16:38:58.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.263+0000 7f20ad5be640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f20a80d19e0 0x7f20a80cdad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:59398/0 (socket says 172.21.15.81:59398) 2023-11-08T16:38:58.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.263+0000 7f20acdbd640 1 -- 172.21.15.81:0/3746961135 learned_addr learned my addr 172.21.15.81:0/3746961135 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:58.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.263+0000 7f20a7fff640 1 --2- 172.21.15.81:0/3746961135 >> v2:172.21.15.93:3300/0 conn(0x7f20a80d0d80 0x7f20a80d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:58.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.263+0000 7f20a77fe640 1 -- 172.21.15.81:0/3746961135 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084899876 0 0) 0x7f20a80d3a80 con 0x7f20a80d19e0 2023-11-08T16:38:58.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.263+0000 7f20acdbd640 1 -- 172.21.15.81:0/3746961135 >> v2:172.21.15.93:3300/0 conn(0x7f20a80d0d80 msgr2=0x7f20a80d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:58.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.264+0000 7f20acdbd640 1 --2- 172.21.15.81:0/3746961135 >> v2:172.21.15.93:3300/0 conn(0x7f20a80d0d80 0x7f20a80d14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:58.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.264+0000 7f20acdbd640 1 -- 172.21.15.81:0/3746961135 >> v1:172.21.15.93:6789/0 conn(0x7f20a80d19e0 legacy=0x7f20a80cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:58.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.264+0000 7f20acdbd640 1 -- 172.21.15.81:0/3746961135 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a80b28c0 con 0x7f20a815d2d0 2023-11-08T16:38:58.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.265+0000 7f20acdbd640 1 --2- 172.21.15.81:0/3746961135 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f20a815d2d0 0x7f20a80cff00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2098009ea0 tx=0x7f209802bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37bf44967d63937c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:58.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.265+0000 7f20a77fe640 1 -- 172.21.15.81:0/3746961135 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209802cb40 con 0x7f20a815d2d0 2023-11-08T16:38:58.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.265+0000 7f20a77fe640 1 -- 172.21.15.81:0/3746961135 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209802cca0 con 0x7f20a815d2d0 2023-11-08T16:38:58.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.265+0000 7f20a77fe640 1 -- 172.21.15.81:0/3746961135 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098034610 con 0x7f20a815d2d0 2023-11-08T16:38:58.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.266+0000 7f20af021640 1 -- 172.21.15.81:0/3746961135 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f20a815d2d0 msgr2=0x7f20a80cff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:58.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.266+0000 7f20af021640 1 --2- 172.21.15.81:0/3746961135 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f20a815d2d0 0x7f20a80cff00 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2098009ea0 tx=0x7f209802bfa0 comp rx=0 tx=0).stop 2023-11-08T16:38:58.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.266+0000 7f20af021640 1 -- 172.21.15.81:0/3746961135 shutdown_connections 2023-11-08T16:38:58.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.266+0000 7f20af021640 1 --2- 172.21.15.81:0/3746961135 >> v2:172.21.15.93:3300/0 conn(0x7f20a80d0d80 0x7f20a80d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:58.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.266+0000 7f20af021640 1 --2- 172.21.15.81:0/3746961135 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f20a815d2d0 0x7f20a80cff00 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:58.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.266+0000 7f20af021640 1 -- 172.21.15.81:0/3746961135 >> 172.21.15.81:0/3746961135 conn(0x7f20a80d2ec0 msgr2=0x7f20a80d32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:58.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.266+0000 7f20af021640 1 -- 172.21.15.81:0/3746961135 shutdown_connections 2023-11-08T16:38:58.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.266+0000 7f20af021640 1 -- 172.21.15.81:0/3746961135 wait complete. 2023-11-08T16:38:58.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.268+0000 7f20af021640 1 Processor -- start 2023-11-08T16:38:58.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.268+0000 7f20af021640 1 -- start start 2023-11-08T16:38:58.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.269+0000 7f20af021640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a80d0d80 0x7f20a80c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:58.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.269+0000 7f20af021640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f20a80d19e0 0x7f20a806b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:58.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.269+0000 7f20af021640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a815d2d0 0x7f20a8159ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:58.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.269+0000 7f20af021640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a806bf70 con 0x7f20a80d0d80 2023-11-08T16:38:58.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.269+0000 7f20af021640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a806c250 con 0x7f20a815d2d0 2023-11-08T16:38:58.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.269+0000 7f20af021640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a80b67c0 con 0x7f20a80d19e0 2023-11-08T16:38:58.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.269+0000 7f20acdbd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a80d0d80 0x7f20a80c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:58.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.269+0000 7f20acdbd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a80d0d80 0x7f20a80c9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55862/0 (socket says 172.21.15.81:55862) 2023-11-08T16:38:58.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.269+0000 7f20acdbd640 1 -- 172.21.15.81:0/3380691680 learned_addr learned my addr 172.21.15.81:0/3380691680 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:58.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.270+0000 7f20acdbd640 1 -- 172.21.15.81:0/3380691680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f20a80d19e0 msgr2=0x7f20a806b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:58.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.270+0000 7f20acdbd640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f20a80d19e0 0x7f20a806b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:58.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.270+0000 7f20acdbd640 1 -- 172.21.15.81:0/3380691680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a815d2d0 msgr2=0x7f20a8159ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:58.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.270+0000 7f20acdbd640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a815d2d0 0x7f20a8159ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:58.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.270+0000 7f20acdbd640 1 -- 172.21.15.81:0/3380691680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2098009b30 con 0x7f20a80d0d80 2023-11-08T16:38:58.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.270+0000 7f20acdbd640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a80d0d80 0x7f20a80c9100 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2098000f80 tx=0x7f2098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:58.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.270+0000 7f20a57fa640 1 -- 172.21.15.81:0/3380691680 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098040670 con 0x7f20a80d0d80 2023-11-08T16:38:58.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.270+0000 7f20a57fa640 1 -- 172.21.15.81:0/3380691680 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20980407d0 con 0x7f20a80d0d80 2023-11-08T16:38:58.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.270+0000 7f20a57fa640 1 -- 172.21.15.81:0/3380691680 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098040b80 con 0x7f20a80d0d80 2023-11-08T16:38:58.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.270+0000 7f20af021640 1 -- 172.21.15.81:0/3380691680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a80b6aa0 con 0x7f20a80d0d80 2023-11-08T16:38:58.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.271+0000 7f20af021640 1 -- 172.21.15.81:0/3380691680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a80c9640 con 0x7f20a80d0d80 2023-11-08T16:38:58.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.272+0000 7f20af021640 1 -- 172.21.15.81:0/3380691680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a80b6600 con 0x7f20a80d0d80 2023-11-08T16:38:58.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.273+0000 7f20a57fa640 1 -- 172.21.15.81:0/3380691680 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2098040ce0 con 0x7f20a80d0d80 2023-11-08T16:38:58.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.274+0000 7f20a57fa640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f208806ae70 0x7f208806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:58.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.274+0000 7f20a57fa640 1 -- 172.21.15.81:0/3380691680 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f20980ac690 con 0x7f20a80d0d80 2023-11-08T16:38:58.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.274+0000 7f20a7fff640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f208806ae70 0x7f208806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:58.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.275+0000 7f20a7fff640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f208806ae70 0x7f208806d330 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2090006a90 tx=0x7f2090006a20 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:58.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.278+0000 7f20a57fa640 1 -- 172.21.15.81:0/3380691680 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2098077830 con 0x7f20a80d0d80 2023-11-08T16:38:58.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:58.476+0000 7f20af021640 1 -- 172.21.15.81:0/3380691680 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f20a80c97d0 con 0x7f20a80d0d80 2023-11-08T16:38:59.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.055+0000 7f20a57fa640 1 -- 172.21.15.81:0/3380691680 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v94) v1 ==== 187+0+0 (secure 0 0 0) 0x7f2098009510 con 0x7f20a80d0d80 2023-11-08T16:38:59.058 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data_extra' removed 2023-11-08T16:38:59.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.058+0000 7f20af021640 1 -- 172.21.15.81:0/3380691680 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f208806ae70 msgr2=0x7f208806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:59.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.058+0000 7f20af021640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f208806ae70 0x7f208806d330 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2090006a90 tx=0x7f2090006a20 comp rx=0 tx=0).stop 2023-11-08T16:38:59.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.058+0000 7f20af021640 1 -- 172.21.15.81:0/3380691680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a80d0d80 msgr2=0x7f20a80c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:59.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.058+0000 7f20af021640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a80d0d80 0x7f20a80c9100 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f2098000f80 tx=0x7f2098004a40 comp rx=0 tx=0).stop 2023-11-08T16:38:59.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.059+0000 7f20af021640 1 -- 172.21.15.81:0/3380691680 shutdown_connections 2023-11-08T16:38:59.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.059+0000 7f20af021640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f208806ae70 0x7f208806d330 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.059+0000 7f20af021640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a815d2d0 0x7f20a8159ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.059+0000 7f20af021640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f20a80d19e0 0x7f20a806b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.059+0000 7f20af021640 1 --2- 172.21.15.81:0/3380691680 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f20a80d0d80 0x7f20a80c9100 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.059+0000 7f20af021640 1 -- 172.21.15.81:0/3380691680 >> 172.21.15.81:0/3380691680 conn(0x7f20a80d2ec0 msgr2=0x7f20a80cbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:59.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.059+0000 7f20af021640 1 -- 172.21.15.81:0/3380691680 shutdown_connections 2023-11-08T16:38:59.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.059+0000 7f20af021640 1 -- 172.21.15.81:0/3380691680 wait complete. 2023-11-08T16:38:59.082 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T16:38:59.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.281+0000 7f57b5890640 1 Processor -- start 2023-11-08T16:38:59.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.281+0000 7f57b5890640 1 -- start start 2023-11-08T16:38:59.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.282+0000 7f57b5890640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f57b015e550 0x7f57b00b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.282+0000 7f57b5890640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57b00b0a20 0x7f57b00ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.282+0000 7f57b5890640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b00b0e50 con 0x7f57b015e170 2023-11-08T16:38:59.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.282+0000 7f57b5890640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b0071460 con 0x7f57b015e550 2023-11-08T16:38:59.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.282+0000 7f57b5890640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00703c0 con 0x7f57b00b0a20 2023-11-08T16:38:59.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.283+0000 7f57a7fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f57b015e550 0x7f57b00b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:59.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.283+0000 7f57a7fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f57b015e550 0x7f57b00b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55876/0 (socket says 172.21.15.81:55876) 2023-11-08T16:38:59.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.283+0000 7f57a7fff640 1 -- 172.21.15.81:0/2452607376 learned_addr learned my addr 172.21.15.81:0/2452607376 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:59.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.283+0000 7f57af7fe640 1 --2- 172.21.15.81:0/2452607376 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57b00b0a20 0x7f57b00ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:59.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.283+0000 7f57a7fff640 1 -- 172.21.15.81:0/2452607376 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57b00b0a20 msgr2=0x7f57b00ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:59.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.283+0000 7f57a7fff640 1 --2- 172.21.15.81:0/2452607376 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57b00b0a20 0x7f57b00ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.283+0000 7f57a7fff640 1 -- 172.21.15.81:0/2452607376 >> v1:172.21.15.93:6789/0 conn(0x7f57b015e170 legacy=0x7f57b015ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:59.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.283+0000 7f57a7fff640 1 -- 172.21.15.81:0/2452607376 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b00706a0 con 0x7f57b015e550 2023-11-08T16:38:59.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.284+0000 7f57a7fff640 1 --2- 172.21.15.81:0/2452607376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b015e550 0x7f57b00b04e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f57b00cfb60 tx=0x7f579802e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abb91aff5c5ec6b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:59.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.285+0000 7f57ae7fc640 1 -- 172.21.15.81:0/2452607376 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579802ee50 con 0x7f57b015e550 2023-11-08T16:38:59.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.285+0000 7f57ae7fc640 1 -- 172.21.15.81:0/2452607376 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798036440 con 0x7f57b015e550 2023-11-08T16:38:59.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.285+0000 7f57ae7fc640 1 -- 172.21.15.81:0/2452607376 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980367d0 con 0x7f57b015e550 2023-11-08T16:38:59.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.285+0000 7f57b5890640 1 -- 172.21.15.81:0/2452607376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b015e550 msgr2=0x7f57b00b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:59.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.285+0000 7f57b5890640 1 --2- 172.21.15.81:0/2452607376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b015e550 0x7f57b00b04e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f57b00cfb60 tx=0x7f579802e0a0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.286+0000 7f57b5890640 1 -- 172.21.15.81:0/2452607376 shutdown_connections 2023-11-08T16:38:59.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.286+0000 7f57b5890640 1 --2- 172.21.15.81:0/2452607376 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57b00b0a20 0x7f57b00ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.286+0000 7f57b5890640 1 --2- 172.21.15.81:0/2452607376 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b015e550 0x7f57b00b04e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.286+0000 7f57b5890640 1 -- 172.21.15.81:0/2452607376 >> 172.21.15.81:0/2452607376 conn(0x7f57b00cb4e0 msgr2=0x7f57b00cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:59.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.286+0000 7f57b5890640 1 -- 172.21.15.81:0/2452607376 shutdown_connections 2023-11-08T16:38:59.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.286+0000 7f57b5890640 1 -- 172.21.15.81:0/2452607376 wait complete. 2023-11-08T16:38:59.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.287+0000 7f57b5890640 1 Processor -- start 2023-11-08T16:38:59.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.287+0000 7f57b5890640 1 -- start start 2023-11-08T16:38:59.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.288+0000 7f57b5890640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57b00b0a20 0x7f57b00c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.288+0000 7f57b5890640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b015e170 0x7f57b01f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.288+0000 7f57b5890640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57b015e550 0x7f57b01eece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.288+0000 7f57b5890640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b01527e0 con 0x7f57b015e170 2023-11-08T16:38:59.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.288+0000 7f57b5890640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00c5440 con 0x7f57b015e550 2023-11-08T16:38:59.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.288+0000 7f57b5890640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b01efc80 con 0x7f57b00b0a20 2023-11-08T16:38:59.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.289+0000 7f57a7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b015e170 0x7f57b01f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:59.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.289+0000 7f57a7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b015e170 0x7f57b01f2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55880/0 (socket says 172.21.15.81:55880) 2023-11-08T16:38:59.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.289+0000 7f57a7fff640 1 -- 172.21.15.81:0/630456183 learned_addr learned my addr 172.21.15.81:0/630456183 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:59.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.289+0000 7f57a7fff640 1 -- 172.21.15.81:0/630456183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57b00b0a20 msgr2=0x7f57b00c4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:59.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.289+0000 7f57a7fff640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57b00b0a20 0x7f57b00c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.289+0000 7f57a7fff640 1 -- 172.21.15.81:0/630456183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57b015e550 msgr2=0x7f57b01eece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:59.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.289+0000 7f57a7fff640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57b015e550 0x7f57b01eece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.289+0000 7f57a7fff640 1 -- 172.21.15.81:0/630456183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579800b770 con 0x7f57b015e170 2023-11-08T16:38:59.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.289+0000 7f57a7fff640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b015e170 0x7f57b01f2bc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f579800b2c0 tx=0x7f57980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:59.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.289+0000 7f57acff9640 1 -- 172.21.15.81:0/630456183 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798041850 con 0x7f57b015e170 2023-11-08T16:38:59.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.290+0000 7f57acff9640 1 -- 172.21.15.81:0/630456183 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57980419b0 con 0x7f57b015e170 2023-11-08T16:38:59.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.290+0000 7f57acff9640 1 -- 172.21.15.81:0/630456183 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5798041d60 con 0x7f57b015e170 2023-11-08T16:38:59.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.290+0000 7f57b5890640 1 -- 172.21.15.81:0/630456183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b01ef6c0 con 0x7f57b015e170 2023-11-08T16:38:59.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.290+0000 7f57b5890640 1 -- 172.21.15.81:0/630456183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b01f23c0 con 0x7f57b015e170 2023-11-08T16:38:59.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.291+0000 7f57b5890640 1 -- 172.21.15.81:0/630456183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b004ece0 con 0x7f57b015e170 2023-11-08T16:38:59.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.292+0000 7f57acff9640 1 -- 172.21.15.81:0/630456183 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f579803d040 con 0x7f57b015e170 2023-11-08T16:38:59.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.293+0000 7f57acff9640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f579006ae70 0x7f579006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.293+0000 7f57acff9640 1 -- 172.21.15.81:0/630456183 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f57980adf90 con 0x7f57b015e170 2023-11-08T16:38:59.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.293+0000 7f57aeffd640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f579006ae70 0x7f579006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:59.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.294+0000 7f57aeffd640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f579006ae70 0x7f579006d330 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f57a0003cc0 tx=0x7f57a00079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:59.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.297+0000 7f57acff9640 1 -- 172.21.15.81:0/630456183 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57980e0ca0 con 0x7f57b015e170 2023-11-08T16:38:59.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.491+0000 7f57b5890640 1 -- 172.21.15.81:0/630456183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f57b00ccc60 con 0x7f57b015e170 2023-11-08T16:38:59.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.498+0000 7f57acff9640 1 -- 172.21.15.81:0/630456183 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f57980792c0 con 0x7f57b015e170 2023-11-08T16:38:59.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 -- 172.21.15.81:0/630456183 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f579006ae70 msgr2=0x7f579006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:59.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f579006ae70 0x7f579006d330 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f57a0003cc0 tx=0x7f57a00079b0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 -- 172.21.15.81:0/630456183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b015e170 msgr2=0x7f57b01f2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:59.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b015e170 0x7f57b01f2bc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f579800b2c0 tx=0x7f57980046c0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 -- 172.21.15.81:0/630456183 shutdown_connections 2023-11-08T16:38:59.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f579006ae70 0x7f579006d330 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57b015e550 0x7f57b01eece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57b015e170 0x7f57b01f2bc0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 --2- 172.21.15.81:0/630456183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57b00b0a20 0x7f57b00c4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 -- 172.21.15.81:0/630456183 >> 172.21.15.81:0/630456183 conn(0x7f57b00cb4e0 msgr2=0x7f57b00cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:59.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 -- 172.21.15.81:0/630456183 shutdown_connections 2023-11-08T16:38:59.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.501+0000 7f57b5890640 1 -- 172.21.15.81:0/630456183 wait complete. 2023-11-08T16:38:59.521 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple' 2023-11-08T16:38:59.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.726+0000 7fc303e7c640 1 Processor -- start 2023-11-08T16:38:59.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.726+0000 7fc303e7c640 1 -- start start 2023-11-08T16:38:59.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.728+0000 7fc303e7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc2fc0adf20 0x7fc2fc165230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.728+0000 7fc303e7c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc2fc165770 0x7fc2fc06b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.728+0000 7fc303e7c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2fc165b80 con 0x7fc2fc0adb40 2023-11-08T16:38:59.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.728+0000 7fc303e7c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc0cb340 con 0x7fc2fc165770 2023-11-08T16:38:59.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.728+0000 7fc303e7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0ce3f0 con 0x7fc2fc0adf20 2023-11-08T16:38:59.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.728+0000 7fc301c18640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc2fc0adb40 0x7fc2fc0c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54934/0 (socket says 172.21.15.81:54934) 2023-11-08T16:38:59.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.728+0000 7fc301c18640 1 -- 172.21.15.81:0/2049432143 learned_addr learned my addr 172.21.15.81:0/2049432143 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:38:59.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.728+0000 7fc302419640 1 --2- 172.21.15.81:0/2049432143 >> v2:172.21.15.81:3300/0 conn(0x7fc2fc165770 0x7fc2fc06b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:59.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.729+0000 7fc301417640 1 --2- 172.21.15.81:0/2049432143 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc2fc0adf20 0x7fc2fc165230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:59.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.729+0000 7fc302419640 1 -- 172.21.15.81:0/2049432143 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc2fc0adf20 msgr2=0x7fc2fc165230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:38:59.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.729+0000 7fc302419640 1 --2- 172.21.15.81:0/2049432143 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc2fc0adf20 0x7fc2fc165230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.729+0000 7fc302419640 1 -- 172.21.15.81:0/2049432143 >> v1:172.21.15.81:6789/0 conn(0x7fc2fc0adb40 legacy=0x7fc2fc0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:59.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.729+0000 7fc302419640 1 -- 172.21.15.81:0/2049432143 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc0cac80 con 0x7fc2fc165770 2023-11-08T16:38:59.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.729+0000 7fc300c16640 1 -- 172.21.15.81:0/2049432143 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 366015949 0 0) 0x7fc2fc165b80 con 0x7fc2fc0adb40 2023-11-08T16:38:59.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.730+0000 7fc302419640 1 --2- 172.21.15.81:0/2049432143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2fc165770 0x7fc2fc06b910 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009d90 tx=0x7fc2f402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d968bde77177caea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:59.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.730+0000 7fc300c16640 1 -- 172.21.15.81:0/2049432143 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f402cbc0 con 0x7fc2fc165770 2023-11-08T16:38:59.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.731+0000 7fc300c16640 1 -- 172.21.15.81:0/2049432143 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f402cd20 con 0x7fc2fc165770 2023-11-08T16:38:59.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.731+0000 7fc300c16640 1 -- 172.21.15.81:0/2049432143 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4034660 con 0x7fc2fc165770 2023-11-08T16:38:59.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.731+0000 7fc303e7c640 1 -- 172.21.15.81:0/2049432143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2fc165770 msgr2=0x7fc2fc06b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:59.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.731+0000 7fc303e7c640 1 --2- 172.21.15.81:0/2049432143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2fc165770 0x7fc2fc06b910 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009d90 tx=0x7fc2f402be10 comp rx=0 tx=0).stop 2023-11-08T16:38:59.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.731+0000 7fc303e7c640 1 -- 172.21.15.81:0/2049432143 shutdown_connections 2023-11-08T16:38:59.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.731+0000 7fc303e7c640 1 --2- 172.21.15.81:0/2049432143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2fc165770 0x7fc2fc06b910 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.731+0000 7fc303e7c640 1 --2- 172.21.15.81:0/2049432143 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc2fc0adf20 0x7fc2fc165230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.731+0000 7fc303e7c640 1 -- 172.21.15.81:0/2049432143 >> 172.21.15.81:0/2049432143 conn(0x7fc2fc0cb490 msgr2=0x7fc2fc0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:38:59.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.731+0000 7fc303e7c640 1 -- 172.21.15.81:0/2049432143 shutdown_connections 2023-11-08T16:38:59.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.731+0000 7fc303e7c640 1 -- 172.21.15.81:0/2049432143 wait complete. 2023-11-08T16:38:59.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.733+0000 7fc303e7c640 1 Processor -- start 2023-11-08T16:38:59.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.733+0000 7fc303e7c640 1 -- start start 2023-11-08T16:38:59.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.734+0000 7fc303e7c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2fc0adb40 0x7fc2fc0c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.734+0000 7fc303e7c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2fc0adf20 0x7fc2fc1f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.734+0000 7fc303e7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc2fc165770 0x7fc2fc1f77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.734+0000 7fc303e7c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc1f7df0 con 0x7fc2fc0adb40 2023-11-08T16:38:59.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.734+0000 7fc303e7c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0c53b0 con 0x7fc2fc0adf20 2023-11-08T16:38:59.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.734+0000 7fc303e7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0c3260 con 0x7fc2fc165770 2023-11-08T16:38:59.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.734+0000 7fc301c18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2fc0adb40 0x7fc2fc0c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:59.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.734+0000 7fc301c18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2fc0adb40 0x7fc2fc0c4e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55900/0 (socket says 172.21.15.81:55900) 2023-11-08T16:38:59.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.734+0000 7fc301c18640 1 -- 172.21.15.81:0/2469386938 learned_addr learned my addr 172.21.15.81:0/2469386938 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:38:59.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc301417640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2fc0adf20 0x7fc2fc1f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:59.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc302419640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc2fc165770 0x7fc2fc1f77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:59.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc301c18640 1 -- 172.21.15.81:0/2469386938 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc2fc165770 msgr2=0x7fc2fc1f77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:59.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc301c18640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc2fc165770 0x7fc2fc1f77e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc301c18640 1 -- 172.21.15.81:0/2469386938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2fc0adf20 msgr2=0x7fc2fc1f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:38:59.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc301c18640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2fc0adf20 0x7fc2fc1f72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:38:59.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc301c18640 1 -- 172.21.15.81:0/2469386938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f4009a70 con 0x7fc2fc0adb40 2023-11-08T16:38:59.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc301c18640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2fc0adb40 0x7fc2fc0c4e70 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800c430 tx=0x7fc2f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:59.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc2f27fc640 1 -- 172.21.15.81:0/2469386938 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80074e0 con 0x7fc2fc0adb40 2023-11-08T16:38:59.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc2f27fc640 1 -- 172.21.15.81:0/2469386938 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f8007660 con 0x7fc2fc0adb40 2023-11-08T16:38:59.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc2f27fc640 1 -- 172.21.15.81:0/2469386938 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f8004a20 con 0x7fc2fc0adb40 2023-11-08T16:38:59.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc303e7c640 1 -- 172.21.15.81:0/2469386938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc0b41e0 con 0x7fc2fc0adb40 2023-11-08T16:38:59.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.735+0000 7fc303e7c640 1 -- 172.21.15.81:0/2469386938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc0b3670 con 0x7fc2fc0adb40 2023-11-08T16:38:59.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.737+0000 7fc303e7c640 1 -- 172.21.15.81:0/2469386938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc04ece0 con 0x7fc2fc0adb40 2023-11-08T16:38:59.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.738+0000 7fc2f27fc640 1 -- 172.21.15.81:0/2469386938 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc2f8004b80 con 0x7fc2fc0adb40 2023-11-08T16:38:59.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.739+0000 7fc2f27fc640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc2e006ae70 0x7fc2e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:38:59.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.739+0000 7fc2f27fc640 1 -- 172.21.15.81:0/2469386938 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5201+0+0 (secure 0 0 0) 0x7fc2f80890f0 con 0x7fc2fc0adb40 2023-11-08T16:38:59.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.739+0000 7fc301417640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc2e006ae70 0x7fc2e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:38:59.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.740+0000 7fc301417640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc2e006ae70 0x7fc2e006d330 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0b04f0 tx=0x7fc2ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:38:59.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:38:59.743+0000 7fc2f27fc640 1 -- 172.21.15.81:0/2469386938 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc2f80542f0 con 0x7fc2fc0adb40 2023-11-08T16:39:00.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.002+0000 7fc303e7c640 1 -- 172.21.15.81:0/2469386938 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]} v 0) v1 -- 0x7fc2fc0b3610 con 0x7fc2fc0adb40 2023-11-08T16:39:00.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.058+0000 7fc2f27fc640 1 -- 172.21.15.81:0/2469386938 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_rebuild_simple"]}]=0 v442) v1 ==== 155+0+0 (secure 0 0 0) 0x7fc2f8057910 con 0x7fc2fc0adb40 2023-11-08T16:39:00.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.061+0000 7fc303e7c640 1 -- 172.21.15.81:0/2469386938 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc2e006ae70 msgr2=0x7fc2e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.061+0000 7fc303e7c640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc2e006ae70 0x7fc2e006d330 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc0b04f0 tx=0x7fc2ec008040 comp rx=0 tx=0).stop 2023-11-08T16:39:00.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.061+0000 7fc303e7c640 1 -- 172.21.15.81:0/2469386938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2fc0adb40 msgr2=0x7fc2fc0c4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.061+0000 7fc303e7c640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2fc0adb40 0x7fc2fc0c4e70 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fc2f800c430 tx=0x7fc2f800c8f0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.061+0000 7fc303e7c640 1 -- 172.21.15.81:0/2469386938 shutdown_connections 2023-11-08T16:39:00.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.061+0000 7fc303e7c640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc2e006ae70 0x7fc2e006d330 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.061+0000 7fc303e7c640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc2fc165770 0x7fc2fc1f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.061+0000 7fc303e7c640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc2fc0adf20 0x7fc2fc1f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.061+0000 7fc303e7c640 1 --2- 172.21.15.81:0/2469386938 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc2fc0adb40 0x7fc2fc0c4e70 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.061+0000 7fc303e7c640 1 -- 172.21.15.81:0/2469386938 >> 172.21.15.81:0/2469386938 conn(0x7fc2fc0cb490 msgr2=0x7fc2fc0afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:00.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.061+0000 7fc303e7c640 1 -- 172.21.15.81:0/2469386938 shutdown_connections 2023-11-08T16:39:00.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.062+0000 7fc303e7c640 1 -- 172.21.15.81:0/2469386938 wait complete. 2023-11-08T16:39:00.082 INFO:tasks.cephfs_test_runner:test_extra_data_pool_rebuild_simple (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-11-08T16:39:00.082 INFO:tasks.cephfs_test_runner:Starting test: test_extra_data_pool_stashed_layout (tasks.cephfs.test_data_scan.TestDataScan) 2023-11-08T16:39:00.083 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:39:00.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.278+0000 7f14f334c640 1 Processor -- start 2023-11-08T16:39:00.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.278+0000 7f14f334c640 1 -- start start 2023-11-08T16:39:00.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.279+0000 7f14f334c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f14ec0ade50 0x7f14ec0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.279+0000 7f14f334c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14ec15ea60 0x7f14ec15eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.279+0000 7f14f334c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14ec06b630 con 0x7f14ec15a680 2023-11-08T16:39:00.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.279+0000 7f14f334c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14ec0b1860 con 0x7f14ec0ade50 2023-11-08T16:39:00.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.279+0000 7f14f334c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec0ca9c0 con 0x7f14ec15ea60 2023-11-08T16:39:00.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.279+0000 7f14f10e8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f14ec0ade50 0x7f14ec0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.279+0000 7f14f10e8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f14ec0ade50 0x7f14ec0ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55904/0 (socket says 172.21.15.81:55904) 2023-11-08T16:39:00.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.279+0000 7f14f10e8640 1 -- 172.21.15.81:0/2485845878 learned_addr learned my addr 172.21.15.81:0/2485845878 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:00.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.280+0000 7f14f18e9640 1 --2- 172.21.15.81:0/2485845878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14ec15ea60 0x7f14ec15eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.280+0000 7f14f10e8640 1 -- 172.21.15.81:0/2485845878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14ec15ea60 msgr2=0x7f14ec15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:00.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.280+0000 7f14f10e8640 1 --2- 172.21.15.81:0/2485845878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14ec15ea60 0x7f14ec15eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.280+0000 7f14f10e8640 1 -- 172.21.15.81:0/2485845878 >> v1:172.21.15.93:6789/0 conn(0x7f14ec15a680 legacy=0x7f14ec15ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:00.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.280+0000 7f14f10e8640 1 -- 172.21.15.81:0/2485845878 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ec0cabb0 con 0x7f14ec0ade50 2023-11-08T16:39:00.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.280+0000 7f14f18e9640 1 --2- 172.21.15.81:0/2485845878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14ec15ea60 0x7f14ec15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:00.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.281+0000 7f14f10e8640 1 --2- 172.21.15.81:0/2485845878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14ec0ade50 0x7f14ec0ca480 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009e50 tx=0x7f14e402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1787094be6caedc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:00.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.281+0000 7f14ebfff640 1 -- 172.21.15.81:0/2485845878 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e402ca80 con 0x7f14ec0ade50 2023-11-08T16:39:00.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.281+0000 7f14ebfff640 1 -- 172.21.15.81:0/2485845878 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14e402cbe0 con 0x7f14ec0ade50 2023-11-08T16:39:00.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.281+0000 7f14ebfff640 1 -- 172.21.15.81:0/2485845878 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14e40343f0 con 0x7f14ec0ade50 2023-11-08T16:39:00.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.282+0000 7f14f334c640 1 -- 172.21.15.81:0/2485845878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14ec0ade50 msgr2=0x7f14ec0ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.282+0000 7f14f334c640 1 --2- 172.21.15.81:0/2485845878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14ec0ade50 0x7f14ec0ca480 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f14e4009e50 tx=0x7f14e402bee0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.282+0000 7f14f334c640 1 -- 172.21.15.81:0/2485845878 shutdown_connections 2023-11-08T16:39:00.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.282+0000 7f14f334c640 1 --2- 172.21.15.81:0/2485845878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14ec15ea60 0x7f14ec15eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.282+0000 7f14f334c640 1 --2- 172.21.15.81:0/2485845878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14ec0ade50 0x7f14ec0ca480 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.282+0000 7f14f334c640 1 -- 172.21.15.81:0/2485845878 >> 172.21.15.81:0/2485845878 conn(0x7f14ec0cb490 msgr2=0x7f14ec0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:00.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.282+0000 7f14f334c640 1 -- 172.21.15.81:0/2485845878 shutdown_connections 2023-11-08T16:39:00.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.282+0000 7f14f334c640 1 -- 172.21.15.81:0/2485845878 wait complete. 2023-11-08T16:39:00.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.283+0000 7f14f334c640 1 Processor -- start 2023-11-08T16:39:00.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.284+0000 7f14f334c640 1 -- start start 2023-11-08T16:39:00.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f334c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14ec0ade50 0x7f14ec152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f334c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14ec15a680 0x7f14ec14ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f334c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14ec15ea60 0x7f14ec15d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f334c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec156950 con 0x7f14ec15ea60 2023-11-08T16:39:00.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f334c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec15db70 con 0x7f14ec15a680 2023-11-08T16:39:00.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f334c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14ec14f3a0 con 0x7f14ec0ade50 2023-11-08T16:39:00.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f18e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14ec15ea60 0x7f14ec15d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f18e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14ec15ea60 0x7f14ec15d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55916/0 (socket says 172.21.15.81:55916) 2023-11-08T16:39:00.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f18e9640 1 -- 172.21.15.81:0/2136554579 learned_addr learned my addr 172.21.15.81:0/2136554579 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:00.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f18e9640 1 -- 172.21.15.81:0/2136554579 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14ec0ade50 msgr2=0x7f14ec152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f10e8640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14ec0ade50 0x7f14ec152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f18e9640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14ec0ade50 0x7f14ec152530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f18e9640 1 -- 172.21.15.81:0/2136554579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14ec15a680 msgr2=0x7f14ec14ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f18e9640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14ec15a680 0x7f14ec14ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.285+0000 7f14f18e9640 1 -- 172.21.15.81:0/2136554579 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e4009b30 con 0x7f14ec15ea60 2023-11-08T16:39:00.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.286+0000 7f14f18e9640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14ec15ea60 0x7f14ec15d630 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00c7f0 tx=0x7f14dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:00.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.286+0000 7f14e9ffb640 1 -- 172.21.15.81:0/2136554579 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc004230 con 0x7f14ec15ea60 2023-11-08T16:39:00.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.286+0000 7f14e9ffb640 1 -- 172.21.15.81:0/2136554579 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14dc004390 con 0x7f14ec15ea60 2023-11-08T16:39:00.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.286+0000 7f14e9ffb640 1 -- 172.21.15.81:0/2136554579 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14dc007860 con 0x7f14ec15ea60 2023-11-08T16:39:00.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.286+0000 7f14f334c640 1 -- 172.21.15.81:0/2136554579 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14ec14f680 con 0x7f14ec15ea60 2023-11-08T16:39:00.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.286+0000 7f14f334c640 1 -- 172.21.15.81:0/2136554579 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14ec14fc40 con 0x7f14ec15ea60 2023-11-08T16:39:00.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.287+0000 7f14f334c640 1 -- 172.21.15.81:0/2136554579 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14ec04ece0 con 0x7f14ec15ea60 2023-11-08T16:39:00.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.288+0000 7f14e9ffb640 1 -- 172.21.15.81:0/2136554579 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f14dc007bd0 con 0x7f14ec15ea60 2023-11-08T16:39:00.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.289+0000 7f14e9ffb640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f14d806ae70 0x7f14d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.289+0000 7f14e9ffb640 1 -- 172.21.15.81:0/2136554579 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f14dc089810 con 0x7f14ec15ea60 2023-11-08T16:39:00.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.289+0000 7f14f10e8640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f14d806ae70 0x7f14d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.290+0000 7f14f10e8640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f14d806ae70 0x7f14d806d330 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f14e4035000 tx=0x7f14e4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:00.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.293+0000 7f14e9ffb640 1 -- 172.21.15.81:0/2136554579 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f14dc054c00 con 0x7f14ec15ea60 2023-11-08T16:39:00.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.482+0000 7f14f334c640 1 -- 172.21.15.81:0/2136554579 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14ec0cc1d0 con 0x7f14ec15ea60 2023-11-08T16:39:00.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.484+0000 7f14e9ffb640 1 -- 172.21.15.81:0/2136554579 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+13315 (secure 0 0 0) 0x7f14dc015090 con 0x7f14ec15ea60 2023-11-08T16:39:00.487 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:00.487 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":94,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:38:59.045371+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/1843050442":"2023-11-09T16:38:54.618203+0000","172.21.15.81:6836/1764545681":"2023-11-09T16:38:06.974694+0000","172.21.15.81:6837/1764545681":"2023-11-09T16:38:06.974694+0000","172.21.15.81:6837/457413856":"2023-11-09T16:38:00.541280+0000","172.21.15.81:6836/457413856":"2023-11-09T16:38:00.541280+0000","172.21.15.81:6836/1843050442":"2023-11-09T16:38:54.618203+0000","172.21.15.81:6837/2162767469":"2023-11-09T16:37:58.508547+0000","172.21.15.81:6836/2162767469":"2023-11-09T16:37:58.508547+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:39:00.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 -- 172.21.15.81:0/2136554579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f14d806ae70 msgr2=0x7f14d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f14d806ae70 0x7f14d806d330 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f14e4035000 tx=0x7f14e4030000 comp rx=0 tx=0).stop 2023-11-08T16:39:00.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 -- 172.21.15.81:0/2136554579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14ec15ea60 msgr2=0x7f14ec15d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14ec15ea60 0x7f14ec15d630 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f14dc00c7f0 tx=0x7f14dc00ccb0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 -- 172.21.15.81:0/2136554579 shutdown_connections 2023-11-08T16:39:00.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f14d806ae70 0x7f14d806d330 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14ec15ea60 0x7f14ec15d630 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14ec15a680 0x7f14ec14ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 --2- 172.21.15.81:0/2136554579 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14ec0ade50 0x7f14ec152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 -- 172.21.15.81:0/2136554579 >> 172.21.15.81:0/2136554579 conn(0x7f14ec0cb490 msgr2=0x7f14ec0b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:00.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 -- 172.21.15.81:0/2136554579 shutdown_connections 2023-11-08T16:39:00.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.487+0000 7f14f334c640 1 -- 172.21.15.81:0/2136554579 wait complete. 2023-11-08T16:39:00.510 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:39:00.510 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:39:00.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.730+0000 7f9632d55640 1 Processor -- start 2023-11-08T16:39:00.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.730+0000 7f9632d55640 1 -- start start 2023-11-08T16:39:00.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.732+0000 7f9632d55640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f962c15bfc0 0x7f962c0caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.732+0000 7f9632d55640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962c0cda10 con 0x7f962c0ce0d0 2023-11-08T16:39:00.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.732+0000 7f9632d55640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f962c06fa30 con 0x7f962c15f280 2023-11-08T16:39:00.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.732+0000 7f9632d55640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0ce4b0 con 0x7f962c15bfc0 2023-11-08T16:39:00.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.732+0000 7f9630af1640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f962c0ce0d0 0x7f962c15eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54944/0 (socket says 172.21.15.81:54944) 2023-11-08T16:39:00.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.732+0000 7f9630af1640 1 -- 172.21.15.81:0/3224331959 learned_addr learned my addr 172.21.15.81:0/3224331959 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:00.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.732+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3369525256 0 0) 0x7f962c0cda10 con 0x7f962c0ce0d0 2023-11-08T16:39:00.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.732+0000 7f962bfff640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f962c15bfc0 0x7f962c0caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f961c003520 con 0x7f962c0ce0d0 2023-11-08T16:39:00.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2147429598 0 0) 0x7f961c003520 con 0x7f962c0ce0d0 2023-11-08T16:39:00.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f962c15bfc0 msgr2=0x7f962c0caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:00.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f962c15bfc0 0x7f962c0caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 >> v1:172.21.15.93:6789/0 conn(0x7f962c15f280 legacy=0x7f962c1516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c06fca0 con 0x7f962c0ce0d0 2023-11-08T16:39:00.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f9620001670 con 0x7f962c0ce0d0 2023-11-08T16:39:00.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 >> v1:172.21.15.81:6789/0 conn(0x7f962c0ce0d0 legacy=0x7f962c15eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f962c15bfc0 0x7f961c003a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f961c003fe0 0x7f961c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f961c004d00 0x7f961c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0ce4b0 con 0x7f961c003fe0 2023-11-08T16:39:00.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f961c00bb40 con 0x7f961c004d00 2023-11-08T16:39:00.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f961c00bd00 con 0x7f962c15bfc0 2023-11-08T16:39:00.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.733+0000 7f962bfff640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f961c003fe0 0x7f961c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.734+0000 7f962bfff640 1 -- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f962c15bfc0 msgr2=0x7f961c003a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:00.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.734+0000 7f9630af1640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f962c15bfc0 0x7f961c003a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.734+0000 7f96312f2640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f961c004d00 0x7f961c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.734+0000 7f962bfff640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f962c15bfc0 0x7f961c003a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.734+0000 7f962bfff640 1 -- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f961c004d00 msgr2=0x7f961c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:00.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.734+0000 7f962bfff640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f961c004d00 0x7f961c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.734+0000 7f962bfff640 1 -- 172.21.15.81:0/3224331959 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f961c003520 con 0x7f961c003fe0 2023-11-08T16:39:00.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.734+0000 7f96312f2640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f961c004d00 0x7f961c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:00.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.735+0000 7f962bfff640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f961c003fe0 0x7f961c0047c0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9618009ac0 tx=0x7f961c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbeb95025f8e6b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:00.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.735+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961802c6b0 con 0x7f961c003fe0 2023-11-08T16:39:00.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.735+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f961802c810 con 0x7f961c003fe0 2023-11-08T16:39:00.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.735+0000 7f962b7fe640 1 -- 172.21.15.81:0/3224331959 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f961802cba0 con 0x7f961c003fe0 2023-11-08T16:39:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.736+0000 7f9632d55640 1 -- 172.21.15.81:0/3224331959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f961c003fe0 msgr2=0x7f961c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.736+0000 7f9632d55640 1 --2- 172.21.15.81:0/3224331959 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f961c003fe0 0x7f961c0047c0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f9618009ac0 tx=0x7f961c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.736+0000 7f96312f2640 1 -- 172.21.15.81:0/3224331959 reap_dead start 2023-11-08T16:39:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.736+0000 7f9632d55640 1 -- 172.21.15.81:0/3224331959 shutdown_connections 2023-11-08T16:39:00.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.736+0000 7f9632d55640 1 -- 172.21.15.81:0/3224331959 >> 172.21.15.81:0/3224331959 conn(0x7f962c0cb580 msgr2=0x7f962c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:00.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.736+0000 7f9632d55640 1 -- 172.21.15.81:0/3224331959 shutdown_connections 2023-11-08T16:39:00.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.736+0000 7f9632d55640 1 -- 172.21.15.81:0/3224331959 wait complete. 2023-11-08T16:39:00.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.738+0000 7f9632d55640 1 Processor -- start 2023-11-08T16:39:00.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.738+0000 7f9632d55640 1 -- start start 2023-11-08T16:39:00.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.739+0000 7f9632d55640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f961c003fe0 0x7f962c1541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.739+0000 7f9632d55640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f962c1f4b60 0x7f962c157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.739+0000 7f9632d55640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f962c154830 0x7f962c1f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.739+0000 7f9632d55640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c156e30 con 0x7f962c1f4b60 2023-11-08T16:39:00.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.739+0000 7f9632d55640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0b3da0 con 0x7f962c154830 2023-11-08T16:39:00.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.739+0000 7f9632d55640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f962c0b4040 con 0x7f961c003fe0 2023-11-08T16:39:00.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.739+0000 7f962bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f962c1f4b60 0x7f962c157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.739+0000 7f962bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f962c1f4b60 0x7f962c157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55934/0 (socket says 172.21.15.81:55934) 2023-11-08T16:39:00.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.739+0000 7f962bfff640 1 -- 172.21.15.81:0/499777473 learned_addr learned my addr 172.21.15.81:0/499777473 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:00.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f9630af1640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f961c003fe0 0x7f962c1541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f962bfff640 1 -- 172.21.15.81:0/499777473 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f961c003fe0 msgr2=0x7f962c1541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f962bfff640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f961c003fe0 0x7f962c1541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f96312f2640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f962c154830 0x7f962c1f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f962bfff640 1 -- 172.21.15.81:0/499777473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f962c154830 msgr2=0x7f962c1f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f962bfff640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f962c154830 0x7f962c1f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f962bfff640 1 -- 172.21.15.81:0/499777473 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f962c0b4320 con 0x7f962c1f4b60 2023-11-08T16:39:00.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f9630af1640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f961c003fe0 0x7f962c1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:00.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f96312f2640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f962c154830 0x7f962c1f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:00.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f962bfff640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f962c1f4b60 0x7f962c157560 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f961802c250 tx=0x7f9618004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:00.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f96297fa640 1 -- 172.21.15.81:0/499777473 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618003810 con 0x7f962c1f4b60 2023-11-08T16:39:00.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f96297fa640 1 -- 172.21.15.81:0/499777473 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9618003970 con 0x7f962c1f4b60 2023-11-08T16:39:00.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f96297fa640 1 -- 172.21.15.81:0/499777473 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9618003ce0 con 0x7f962c1f4b60 2023-11-08T16:39:00.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.740+0000 7f9632d55640 1 -- 172.21.15.81:0/499777473 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f962c157040 con 0x7f962c1f4b60 2023-11-08T16:39:00.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.741+0000 7f9632d55640 1 -- 172.21.15.81:0/499777473 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f962c154cb0 con 0x7f962c1f4b60 2023-11-08T16:39:00.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.742+0000 7f9632d55640 1 -- 172.21.15.81:0/499777473 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f962c0b4180 con 0x7f962c1f4b60 2023-11-08T16:39:00.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.743+0000 7f96297fa640 1 -- 172.21.15.81:0/499777473 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9618034440 con 0x7f962c1f4b60 2023-11-08T16:39:00.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.744+0000 7f96297fa640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f961006ae70 0x7f961006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:00.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.744+0000 7f96297fa640 1 -- 172.21.15.81:0/499777473 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f96180ab770 con 0x7f962c1f4b60 2023-11-08T16:39:00.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.744+0000 7f9630af1640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f961006ae70 0x7f961006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:00.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.745+0000 7f9630af1640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f961006ae70 0x7f961006d330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9620001eb0 tx=0x7f96200014e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:00.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.748+0000 7f96297fa640 1 -- 172.21.15.81:0/499777473 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f962c0b4180 con 0x7f962c1f4b60 2023-11-08T16:39:00.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.939+0000 7f9632d55640 1 -- 172.21.15.81:0/499777473 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f962c0b4180 con 0x7f962c1f4b60 2023-11-08T16:39:00.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.940+0000 7f96297fa640 1 -- 172.21.15.81:0/499777473 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f962c0b4180 con 0x7f962c1f4b60 2023-11-08T16:39:00.942 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:39:00.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 -- 172.21.15.81:0/499777473 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f961006ae70 msgr2=0x7f961006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f961006ae70 0x7f961006d330 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9620001eb0 tx=0x7f96200014e0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 -- 172.21.15.81:0/499777473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f962c1f4b60 msgr2=0x7f962c157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:00.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f962c1f4b60 0x7f962c157560 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f961802c250 tx=0x7f9618004730 comp rx=0 tx=0).stop 2023-11-08T16:39:00.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 -- 172.21.15.81:0/499777473 shutdown_connections 2023-11-08T16:39:00.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f961006ae70 0x7f961006d330 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f962c154830 0x7f962c1f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f962c1f4b60 0x7f962c157560 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 --2- 172.21.15.81:0/499777473 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f961c003fe0 0x7f962c1541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:00.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 -- 172.21.15.81:0/499777473 >> 172.21.15.81:0/499777473 conn(0x7f962c0cb580 msgr2=0x7f962c0afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:00.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 -- 172.21.15.81:0/499777473 shutdown_connections 2023-11-08T16:39:00.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:00.942+0000 7f9632d55640 1 -- 172.21.15.81:0/499777473 wait complete. 2023-11-08T16:39:00.963 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout' 2023-11-08T16:39:01.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.169+0000 7f42ad894640 1 Processor -- start 2023-11-08T16:39:01.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.170+0000 7f42ad894640 1 -- start start 2023-11-08T16:39:01.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.171+0000 7f42ad894640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f42a80b0070 0x7f42a806d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:01.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.171+0000 7f42ad894640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a80d06e0 0x7f42a80cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:01.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.171+0000 7f42ad894640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a80cdf60 con 0x7f42a80cd5c0 2023-11-08T16:39:01.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.171+0000 7f42ad894640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a80cbee0 con 0x7f42a80b0070 2023-11-08T16:39:01.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.171+0000 7f42ad894640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80b0450 con 0x7f42a80d06e0 2023-11-08T16:39:01.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.172+0000 7f42a67fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f42a80cd5c0 0x7f42a80cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:43410/0 (socket says 172.21.15.81:43410) 2023-11-08T16:39:01.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.172+0000 7f42a67fc640 1 -- 172.21.15.81:0/3406179543 learned_addr learned my addr 172.21.15.81:0/3406179543 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:01.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.172+0000 7f42a6ffd640 1 --2- 172.21.15.81:0/3406179543 >> v2:172.21.15.93:3300/0 conn(0x7f42a80b0070 0x7f42a806d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:01.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.172+0000 7f42a77fe640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a80d06e0 0x7f42a80cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:01.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.172+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 411570981 0 0) 0x7f42a80cdf60 con 0x7f42a80cd5c0 2023-11-08T16:39:01.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.172+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4294003520 con 0x7f42a80cd5c0 2023-11-08T16:39:01.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333300690 0 0) 0x7f4294003520 con 0x7f42a80cd5c0 2023-11-08T16:39:01.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a80d06e0 msgr2=0x7f42a80cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:01.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a80d06e0 0x7f42a80cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:01.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 >> v2:172.21.15.93:3300/0 conn(0x7f42a80b0070 msgr2=0x7f42a806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:01.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 --2- 172.21.15.81:0/3406179543 >> v2:172.21.15.93:3300/0 conn(0x7f42a80b0070 0x7f42a806d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:01.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80cb2d0 con 0x7f42a80cd5c0 2023-11-08T16:39:01.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f4290001670 con 0x7f42a80cd5c0 2023-11-08T16:39:01.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 >> v1:172.21.15.81:6789/0 conn(0x7f42a80cd5c0 legacy=0x7f42a80cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:01.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4294003a50 0x7f4294003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:01.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a80d06e0 0x7f42940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:01.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4294004d00 0x7f4294005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:01.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80cbee0 con 0x7f4294004d00 2023-11-08T16:39:01.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80b0450 con 0x7f4294003a50 2023-11-08T16:39:01.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429400b9b0 con 0x7f42a80d06e0 2023-11-08T16:39:01.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a77fe640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4294004d00 0x7f4294005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:01.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a67fc640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a80d06e0 0x7f42940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:01.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.173+0000 7f42a6ffd640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4294003a50 0x7f4294003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:01.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.174+0000 7f42a77fe640 1 -- 172.21.15.81:0/3406179543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a80d06e0 msgr2=0x7f42940047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:01.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.174+0000 7f42a77fe640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a80d06e0 0x7f42940047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:01.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.174+0000 7f42a77fe640 1 -- 172.21.15.81:0/3406179543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4294003a50 msgr2=0x7f4294003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:01.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.174+0000 7f42a77fe640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4294003a50 0x7f4294003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:01.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.174+0000 7f42a77fe640 1 -- 172.21.15.81:0/3406179543 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4294003520 con 0x7f4294004d00 2023-11-08T16:39:01.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.175+0000 7f42a77fe640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4294004d00 0x7f4294005180 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009ac0 tx=0x7f42940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9bb58b72ea426a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:01.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.175+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a002c6b0 con 0x7f4294004d00 2023-11-08T16:39:01.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.175+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a002c810 con 0x7f4294004d00 2023-11-08T16:39:01.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.175+0000 7f42a5ffb640 1 -- 172.21.15.81:0/3406179543 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a002cba0 con 0x7f4294004d00 2023-11-08T16:39:01.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.176+0000 7f42ad894640 1 -- 172.21.15.81:0/3406179543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4294004d00 msgr2=0x7f4294005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:01.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.176+0000 7f42ad894640 1 --2- 172.21.15.81:0/3406179543 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4294004d00 0x7f4294005180 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f42a0009ac0 tx=0x7f42940057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:01.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.176+0000 7f42a77fe640 1 -- 172.21.15.81:0/3406179543 reap_dead start 2023-11-08T16:39:01.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.176+0000 7f42ad894640 1 -- 172.21.15.81:0/3406179543 shutdown_connections 2023-11-08T16:39:01.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.176+0000 7f42ad894640 1 -- 172.21.15.81:0/3406179543 >> 172.21.15.81:0/3406179543 conn(0x7f42a80d0be0 msgr2=0x7f42a80d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:01.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.176+0000 7f42ad894640 1 -- 172.21.15.81:0/3406179543 shutdown_connections 2023-11-08T16:39:01.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.176+0000 7f42ad894640 1 -- 172.21.15.81:0/3406179543 wait complete. 2023-11-08T16:39:01.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.177+0000 7f42ad894640 1 Processor -- start 2023-11-08T16:39:01.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.178+0000 7f42ad894640 1 -- start start 2023-11-08T16:39:01.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.178+0000 7f42ad894640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a81ecdb0 0x7f42a80b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:01.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42ad894640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a81f36c0 0x7f42a806e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:01.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42ad894640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a806e870 0x7f42a80c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:01.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42ad894640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a81ed190 con 0x7f42a81ecdb0 2023-11-08T16:39:01.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42ad894640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a806ecf0 con 0x7f42a81f36c0 2023-11-08T16:39:01.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42ad894640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80b6110 con 0x7f42a806e870 2023-11-08T16:39:01.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42a6ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a81ecdb0 0x7f42a80b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:01.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42a6ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a81ecdb0 0x7f42a80b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49582/0 (socket says 172.21.15.81:49582) 2023-11-08T16:39:01.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42a6ffd640 1 -- 172.21.15.81:0/3555488502 learned_addr learned my addr 172.21.15.81:0/3555488502 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:01.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42a6ffd640 1 -- 172.21.15.81:0/3555488502 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a806e870 msgr2=0x7f42a80c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:01.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42a6ffd640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a806e870 0x7f42a80c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:01.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42a6ffd640 1 -- 172.21.15.81:0/3555488502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a81f36c0 msgr2=0x7f42a806e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:01.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42a6ffd640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a81f36c0 0x7f42a806e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:01.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.179+0000 7f42a6ffd640 1 -- 172.21.15.81:0/3555488502 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80c7dc0 con 0x7f42a81ecdb0 2023-11-08T16:39:01.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.180+0000 7f42a6ffd640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a81ecdb0 0x7f42a80b5bd0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4298007f30 tx=0x7f4298011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:01.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.180+0000 7f429ffff640 1 -- 172.21.15.81:0/3555488502 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429800e030 con 0x7f42a81ecdb0 2023-11-08T16:39:01.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.180+0000 7f429ffff640 1 -- 172.21.15.81:0/3555488502 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4298016470 con 0x7f42a81ecdb0 2023-11-08T16:39:01.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.180+0000 7f429ffff640 1 -- 172.21.15.81:0/3555488502 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4298016820 con 0x7f42a81ecdb0 2023-11-08T16:39:01.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.180+0000 7f42ad894640 1 -- 172.21.15.81:0/3555488502 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a80b5740 con 0x7f42a81ecdb0 2023-11-08T16:39:01.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.180+0000 7f42ad894640 1 -- 172.21.15.81:0/3555488502 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a80b4e10 con 0x7f42a81ecdb0 2023-11-08T16:39:01.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.181+0000 7f42ad894640 1 -- 172.21.15.81:0/3555488502 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a804ece0 con 0x7f42a81ecdb0 2023-11-08T16:39:01.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.182+0000 7f429ffff640 1 -- 172.21.15.81:0/3555488502 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4298016980 con 0x7f42a81ecdb0 2023-11-08T16:39:01.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.183+0000 7f429ffff640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806aea0 0x7f428806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:01.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.184+0000 7f429ffff640 1 -- 172.21.15.81:0/3555488502 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f429800f070 con 0x7f42a81ecdb0 2023-11-08T16:39:01.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.184+0000 7f42a67fc640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806aea0 0x7f428806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:01.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.184+0000 7f42a67fc640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806aea0 0x7f428806d360 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f42a806ef40 tx=0x7f4290001e10 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:01.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.188+0000 7f429ffff640 1 -- 172.21.15.81:0/3555488502 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4298055610 con 0x7f42a81ecdb0 2023-11-08T16:39:01.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:01.448+0000 7f42ad894640 1 -- 172.21.15.81:0/3555488502 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]} v 0) v1 -- 0x7f42a804e500 con 0x7f42a81ecdb0 2023-11-08T16:39:02.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.075+0000 7f429ffff640 1 -- 172.21.15.81:0/3555488502 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]}]=0 v444) v1 ==== 158+0+0 (secure 0 0 0) 0x7f4298058c30 con 0x7f42a81ecdb0 2023-11-08T16:39:02.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 -- 172.21.15.81:0/3555488502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806aea0 msgr2=0x7f428806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:02.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806aea0 0x7f428806d360 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f42a806ef40 tx=0x7f4290001e10 comp rx=0 tx=0).stop 2023-11-08T16:39:02.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 -- 172.21.15.81:0/3555488502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a81ecdb0 msgr2=0x7f42a80b5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:02.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a81ecdb0 0x7f42a80b5bd0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4298007f30 tx=0x7f4298011660 comp rx=0 tx=0).stop 2023-11-08T16:39:02.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 -- 172.21.15.81:0/3555488502 shutdown_connections 2023-11-08T16:39:02.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f428806aea0 0x7f428806d360 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f42a806e870 0x7f42a80c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42a81f36c0 0x7f42a806e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 --2- 172.21.15.81:0/3555488502 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f42a81ecdb0 0x7f42a80b5bd0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 -- 172.21.15.81:0/3555488502 >> 172.21.15.81:0/3555488502 conn(0x7f42a80d0be0 msgr2=0x7f42a80ae170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:02.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 -- 172.21.15.81:0/3555488502 shutdown_connections 2023-11-08T16:39:02.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.079+0000 7f42ad894640 1 -- 172.21.15.81:0/3555488502 wait complete. 2023-11-08T16:39:02.103 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T16:39:02.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.314+0000 7f84e71a9640 1 Processor -- start 2023-11-08T16:39:02.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.314+0000 7f84e71a9640 1 -- start start 2023-11-08T16:39:02.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.316+0000 7f84e71a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 0x7f84e00caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:02.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.316+0000 7f84e71a9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f84e015bfc0 0x7f84e015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.316+0000 7f84e71a9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84e015f500 con 0x7f84e015f0b0 2023-11-08T16:39:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.316+0000 7f84e71a9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84e00cda10 con 0x7f84e015bfc0 2023-11-08T16:39:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.316+0000 7f84e71a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e006fa30 con 0x7f84e00ce0d0 2023-11-08T16:39:02.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.316+0000 7f84e4f45640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 0x7f84e00caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.317+0000 7f84e4f45640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 0x7f84e00caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:48020/0 (socket says 172.21.15.81:48020) 2023-11-08T16:39:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.317+0000 7f84e4f45640 1 -- 172.21.15.81:0/650873868 learned_addr learned my addr 172.21.15.81:0/650873868 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.317+0000 7f84dffff640 1 --2- 172.21.15.81:0/650873868 >> v2:172.21.15.93:3300/0 conn(0x7f84e015bfc0 0x7f84e015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:02.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.317+0000 7f84e4f45640 1 -- 172.21.15.81:0/650873868 >> v2:172.21.15.93:3300/0 conn(0x7f84e015bfc0 msgr2=0x7f84e015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:02.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.317+0000 7f84e4f45640 1 --2- 172.21.15.81:0/650873868 >> v2:172.21.15.93:3300/0 conn(0x7f84e015bfc0 0x7f84e015eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.317+0000 7f84e4f45640 1 -- 172.21.15.81:0/650873868 >> v1:172.21.15.93:6789/0 conn(0x7f84e015f0b0 legacy=0x7f84e01516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:02.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.317+0000 7f84e4f45640 1 -- 172.21.15.81:0/650873868 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e00ce4b0 con 0x7f84e00ce0d0 2023-11-08T16:39:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.318+0000 7f84e4f45640 1 --2- 172.21.15.81:0/650873868 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 0x7f84e00caf20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009870 tx=0x7f84d402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca22fe365670720 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.319+0000 7f84df7fe640 1 -- 172.21.15.81:0/650873868 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4030840 con 0x7f84e00ce0d0 2023-11-08T16:39:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.319+0000 7f84df7fe640 1 -- 172.21.15.81:0/650873868 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d40309a0 con 0x7f84e00ce0d0 2023-11-08T16:39:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.319+0000 7f84df7fe640 1 -- 172.21.15.81:0/650873868 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d4030d30 con 0x7f84e00ce0d0 2023-11-08T16:39:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.320+0000 7f84e71a9640 1 -- 172.21.15.81:0/650873868 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 msgr2=0x7f84e00caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.320+0000 7f84e71a9640 1 --2- 172.21.15.81:0/650873868 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 0x7f84e00caf20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009870 tx=0x7f84d402b980 comp rx=0 tx=0).stop 2023-11-08T16:39:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.320+0000 7f84e71a9640 1 -- 172.21.15.81:0/650873868 shutdown_connections 2023-11-08T16:39:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.320+0000 7f84e71a9640 1 --2- 172.21.15.81:0/650873868 >> v2:172.21.15.93:3300/0 conn(0x7f84e015bfc0 0x7f84e015eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.320+0000 7f84e71a9640 1 --2- 172.21.15.81:0/650873868 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 0x7f84e00caf20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.320+0000 7f84e71a9640 1 -- 172.21.15.81:0/650873868 >> 172.21.15.81:0/650873868 conn(0x7f84e00cb580 msgr2=0x7f84e00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:02.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.320+0000 7f84e71a9640 1 -- 172.21.15.81:0/650873868 shutdown_connections 2023-11-08T16:39:02.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.320+0000 7f84e71a9640 1 -- 172.21.15.81:0/650873868 wait complete. 2023-11-08T16:39:02.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.321+0000 7f84e71a9640 1 Processor -- start 2023-11-08T16:39:02.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.322+0000 7f84e71a9640 1 -- start start 2023-11-08T16:39:02.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84e71a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 0x7f84e01eff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:02.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84e71a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84e015bfc0 0x7f84e00c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:02.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84e71a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84e015f0b0 0x7f84e00c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:02.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84e71a9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e00c7b60 con 0x7f84e015bfc0 2023-11-08T16:39:02.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84e71a9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e00c6ee0 con 0x7f84e015f0b0 2023-11-08T16:39:02.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84e71a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84e00c71c0 con 0x7f84e00ce0d0 2023-11-08T16:39:02.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84dffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84e015bfc0 0x7f84e00c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:02.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84dffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84e015bfc0 0x7f84e00c65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49594/0 (socket says 172.21.15.81:49594) 2023-11-08T16:39:02.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84dffff640 1 -- 172.21.15.81:0/3205330701 learned_addr learned my addr 172.21.15.81:0/3205330701 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:02.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84e4f45640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 0x7f84e01eff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:02.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84dffff640 1 -- 172.21.15.81:0/3205330701 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 msgr2=0x7f84e01eff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:02.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84dffff640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 0x7f84e01eff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84e5746640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84e015f0b0 0x7f84e00c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:02.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84dffff640 1 -- 172.21.15.81:0/3205330701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84e015f0b0 msgr2=0x7f84e00c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:02.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84dffff640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84e015f0b0 0x7f84e00c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.323+0000 7f84dffff640 1 -- 172.21.15.81:0/3205330701 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d40094d0 con 0x7f84e015bfc0 2023-11-08T16:39:02.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.324+0000 7f84dffff640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84e015bfc0 0x7f84e00c65e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e980 tx=0x7f84cc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:02.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.324+0000 7f84dd7fa640 1 -- 172.21.15.81:0/3205330701 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc009670 con 0x7f84e015bfc0 2023-11-08T16:39:02.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.324+0000 7f84dd7fa640 1 -- 172.21.15.81:0/3205330701 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84cc00ca00 con 0x7f84e015bfc0 2023-11-08T16:39:02.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.324+0000 7f84dd7fa640 1 -- 172.21.15.81:0/3205330701 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84cc00cd90 con 0x7f84e015bfc0 2023-11-08T16:39:02.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.324+0000 7f84e71a9640 1 -- 172.21.15.81:0/3205330701 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e00c74a0 con 0x7f84e015bfc0 2023-11-08T16:39:02.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.324+0000 7f84e71a9640 1 -- 172.21.15.81:0/3205330701 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e01f04d0 con 0x7f84e015bfc0 2023-11-08T16:39:02.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.325+0000 7f84e71a9640 1 -- 172.21.15.81:0/3205330701 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e00c7ca0 con 0x7f84e015bfc0 2023-11-08T16:39:02.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.326+0000 7f84dd7fa640 1 -- 172.21.15.81:0/3205330701 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f84cc013470 con 0x7f84e015bfc0 2023-11-08T16:39:02.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.327+0000 7f84dd7fa640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84c406ae70 0x7f84c406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:02.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.327+0000 7f84dd7fa640 1 -- 172.21.15.81:0/3205330701 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f84cc0896b0 con 0x7f84e015bfc0 2023-11-08T16:39:02.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.328+0000 7f84e4f45640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84c406ae70 0x7f84c406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:02.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.328+0000 7f84e4f45640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84c406ae70 0x7f84c406d330 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f84d4036000 tx=0x7f84d403b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:02.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.332+0000 7f84dd7fa640 1 -- 172.21.15.81:0/3205330701 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f84cc0548b0 con 0x7f84e015bfc0 2023-11-08T16:39:02.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.524+0000 7f84e71a9640 1 -- 172.21.15.81:0/3205330701 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f84e004ece0 con 0x7f84e015bfc0 2023-11-08T16:39:02.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.530+0000 7f84dd7fa640 1 -- 172.21.15.81:0/3205330701 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f84cc057ed0 con 0x7f84e015bfc0 2023-11-08T16:39:02.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 -- 172.21.15.81:0/3205330701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84c406ae70 msgr2=0x7f84c406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:02.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84c406ae70 0x7f84c406d330 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f84d4036000 tx=0x7f84d403b040 comp rx=0 tx=0).stop 2023-11-08T16:39:02.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 -- 172.21.15.81:0/3205330701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84e015bfc0 msgr2=0x7f84e00c65e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:02.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84e015bfc0 0x7f84e00c65e0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f84cc00e980 tx=0x7f84cc00ee40 comp rx=0 tx=0).stop 2023-11-08T16:39:02.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 -- 172.21.15.81:0/3205330701 shutdown_connections 2023-11-08T16:39:02.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f84c406ae70 0x7f84c406d330 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84e015f0b0 0x7f84e00c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84e015bfc0 0x7f84e00c65e0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 --2- 172.21.15.81:0/3205330701 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84e00ce0d0 0x7f84e01eff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:02.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 -- 172.21.15.81:0/3205330701 >> 172.21.15.81:0/3205330701 conn(0x7f84e00cb580 msgr2=0x7f84e00c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:02.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 -- 172.21.15.81:0/3205330701 shutdown_connections 2023-11-08T16:39:02.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:02.533+0000 7f84e71a9640 1 -- 172.21.15.81:0/3205330701 wait complete. 2023-11-08T16:39:02.555 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T16:39:02.555 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:39:02.665 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:39:02.789 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T16:39:02.789 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:39:02.790 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:39:02.853 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:39:02.854 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:39:02.855 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:39:02.915 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:39:02.916 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:39:02.916 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:39:02.977 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:39:02.978 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:39:02.978 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:39:03.038 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:39:03.038 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T16:39:03.039 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:03.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.262+0000 7f279bc6f640 1 Processor -- start 2023-11-08T16:39:03.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.262+0000 7f279bc6f640 1 -- start start 2023-11-08T16:39:03.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279bc6f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f27940ade50 0x7f27940ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:03.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279bc6f640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f279415a680 0x7f279415ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:03.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279bc6f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279406b630 con 0x7f279415b050 2023-11-08T16:39:03.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279bc6f640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27940b1860 con 0x7f279415a680 2023-11-08T16:39:03.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279bc6f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27940ca9c0 con 0x7f27940ade50 2023-11-08T16:39:03.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279a20c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f279415b050 0x7f27940c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:43412/0 (socket says 172.21.15.81:43412) 2023-11-08T16:39:03.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279920a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f279415a680 0x7f279415ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:03.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279a20c640 1 -- 172.21.15.81:0/2548229598 learned_addr learned my addr 172.21.15.81:0/2548229598 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:03.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f2799a0b640 1 --2- 172.21.15.81:0/2548229598 >> v2:172.21.15.93:3300/0 conn(0x7f27940ade50 0x7f27940ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:03.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279920a640 1 -- 172.21.15.81:0/2548229598 >> v2:172.21.15.93:3300/0 conn(0x7f27940ade50 msgr2=0x7f27940ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:03.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279920a640 1 --2- 172.21.15.81:0/2548229598 >> v2:172.21.15.93:3300/0 conn(0x7f27940ade50 0x7f27940ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279920a640 1 -- 172.21.15.81:0/2548229598 >> v1:172.21.15.81:6789/0 conn(0x7f279415b050 legacy=0x7f27940c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:03.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.264+0000 7f279920a640 1 -- 172.21.15.81:0/2548229598 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27940cabb0 con 0x7f279415a680 2023-11-08T16:39:03.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.265+0000 7f279920a640 1 --2- 172.21.15.81:0/2548229598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279415a680 0x7f279415ab10 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2794067770 tx=0x7f278402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=933b8dd008cad0b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:03.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.265+0000 7f2798a09640 1 -- 172.21.15.81:0/2548229598 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784031830 con 0x7f279415a680 2023-11-08T16:39:03.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.265+0000 7f2798a09640 1 -- 172.21.15.81:0/2548229598 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2784031990 con 0x7f279415a680 2023-11-08T16:39:03.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.265+0000 7f2798a09640 1 -- 172.21.15.81:0/2548229598 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2784031d20 con 0x7f279415a680 2023-11-08T16:39:03.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.265+0000 7f279bc6f640 1 -- 172.21.15.81:0/2548229598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279415a680 msgr2=0x7f279415ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:03.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.265+0000 7f279bc6f640 1 --2- 172.21.15.81:0/2548229598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279415a680 0x7f279415ab10 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f2794067770 tx=0x7f278402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.266+0000 7f279bc6f640 1 -- 172.21.15.81:0/2548229598 shutdown_connections 2023-11-08T16:39:03.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.266+0000 7f279bc6f640 1 --2- 172.21.15.81:0/2548229598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279415a680 0x7f279415ab10 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.266+0000 7f279bc6f640 1 --2- 172.21.15.81:0/2548229598 >> v2:172.21.15.93:3300/0 conn(0x7f27940ade50 0x7f27940ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.266+0000 7f279bc6f640 1 -- 172.21.15.81:0/2548229598 >> 172.21.15.81:0/2548229598 conn(0x7f27940cb490 msgr2=0x7f27940cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:03.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.266+0000 7f279bc6f640 1 -- 172.21.15.81:0/2548229598 shutdown_connections 2023-11-08T16:39:03.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.266+0000 7f279bc6f640 1 -- 172.21.15.81:0/2548229598 wait complete. 2023-11-08T16:39:03.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.267+0000 7f279bc6f640 1 Processor -- start 2023-11-08T16:39:03.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.267+0000 7f279bc6f640 1 -- start start 2023-11-08T16:39:03.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.268+0000 7f279bc6f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27940ade50 0x7f27940c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:03.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.268+0000 7f279bc6f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f279415a680 0x7f2794146010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:03.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.268+0000 7f279bc6f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279415b050 0x7f2794146550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:03.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.268+0000 7f279bc6f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2794144f70 con 0x7f279415b050 2023-11-08T16:39:03.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.268+0000 7f279bc6f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2794145250 con 0x7f27940ade50 2023-11-08T16:39:03.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.268+0000 7f279bc6f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2794145530 con 0x7f279415a680 2023-11-08T16:39:03.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f279a20c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279415b050 0x7f2794146550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:03.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f279a20c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279415b050 0x7f2794146550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49606/0 (socket says 172.21.15.81:49606) 2023-11-08T16:39:03.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f279a20c640 1 -- 172.21.15.81:0/328238711 learned_addr learned my addr 172.21.15.81:0/328238711 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:03.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f279920a640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f279415a680 0x7f2794146010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:03.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f279a20c640 1 -- 172.21.15.81:0/328238711 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f279415a680 msgr2=0x7f2794146010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:03.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f2799a0b640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27940ade50 0x7f27940c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:03.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f279a20c640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f279415a680 0x7f2794146010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f279a20c640 1 -- 172.21.15.81:0/328238711 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27940ade50 msgr2=0x7f27940c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:03.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f279a20c640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27940ade50 0x7f27940c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f279a20c640 1 -- 172.21.15.81:0/328238711 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2784009400 con 0x7f279415b050 2023-11-08T16:39:03.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f279920a640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f279415a680 0x7f2794146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:03.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.269+0000 7f279a20c640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279415b050 0x7f2794146550 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f278c00ca80 tx=0x7f278c00cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:03.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.270+0000 7f278a7fc640 1 -- 172.21.15.81:0/328238711 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c00ac30 con 0x7f279415b050 2023-11-08T16:39:03.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.270+0000 7f278a7fc640 1 -- 172.21.15.81:0/328238711 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c00ad90 con 0x7f279415b050 2023-11-08T16:39:03.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.270+0000 7f278a7fc640 1 -- 172.21.15.81:0/328238711 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c012600 con 0x7f279415b050 2023-11-08T16:39:03.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.270+0000 7f279bc6f640 1 -- 172.21.15.81:0/328238711 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2794145810 con 0x7f279415b050 2023-11-08T16:39:03.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.270+0000 7f279bc6f640 1 -- 172.21.15.81:0/328238711 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27940c91b0 con 0x7f279415b050 2023-11-08T16:39:03.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.271+0000 7f279bc6f640 1 -- 172.21.15.81:0/328238711 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2794145d30 con 0x7f279415b050 2023-11-08T16:39:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.272+0000 7f278a7fc640 1 -- 172.21.15.81:0/328238711 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f278c007070 con 0x7f279415b050 2023-11-08T16:39:03.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.273+0000 7f278a7fc640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277806ae70 0x7f277806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:03.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.273+0000 7f278a7fc640 1 -- 172.21.15.81:0/328238711 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f278c0891f0 con 0x7f279415b050 2023-11-08T16:39:03.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.273+0000 7f2799a0b640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277806ae70 0x7f277806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:03.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.274+0000 7f2799a0b640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277806ae70 0x7f277806d330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2790006fd0 tx=0x7f2790008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:03.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.277+0000 7f278a7fc640 1 -- 172.21.15.81:0/328238711 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f278c054540 con 0x7f279415b050 2023-11-08T16:39:03.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.502+0000 7f279bc6f640 1 -- 172.21.15.81:0/328238711 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f279404ece0 con 0x7f279415b050 2023-11-08T16:39:03.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.503+0000 7f278a7fc640 1 -- 172.21.15.81:0/328238711 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+3699 (secure 0 0 0) 0x7f278c057b60 con 0x7f279415b050 2023-11-08T16:39:03.506 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:03.506 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":80,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5857,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1527116581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1527116581},{"type":"v1","addr":"172.21.15.93:6833","nonce":1527116581}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":75},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/2617548066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":2617548066},{"type":"v1","addr":"172.21.15.93:6835","nonce":2617548066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":7380,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/2591582634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":2591582634},{"type":"v1","addr":"172.21.15.81:6835","nonce":2591582634}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":74},{"gid":7623,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1423541977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1423541977},{"type":"v1","addr":"172.21.15.81:6837","nonce":1423541977}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":80}],"filesystems":[]} 2023-11-08T16:39:03.507 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 80 2023-11-08T16:39:03.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 -- 172.21.15.81:0/328238711 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277806ae70 msgr2=0x7f277806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:03.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277806ae70 0x7f277806d330 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f2790006fd0 tx=0x7f2790008040 comp rx=0 tx=0).stop 2023-11-08T16:39:03.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 -- 172.21.15.81:0/328238711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279415b050 msgr2=0x7f2794146550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:03.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279415b050 0x7f2794146550 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f278c00ca80 tx=0x7f278c00cf40 comp rx=0 tx=0).stop 2023-11-08T16:39:03.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 -- 172.21.15.81:0/328238711 shutdown_connections 2023-11-08T16:39:03.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f277806ae70 0x7f277806d330 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f279415b050 0x7f2794146550 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f279415a680 0x7f2794146010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 --2- 172.21.15.81:0/328238711 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f27940ade50 0x7f27940c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 -- 172.21.15.81:0/328238711 >> 172.21.15.81:0/328238711 conn(0x7f27940cb490 msgr2=0x7f27940cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:03.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 -- 172.21.15.81:0/328238711 shutdown_connections 2023-11-08T16:39:03.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.506+0000 7f279bc6f640 1 -- 172.21.15.81:0/328238711 wait complete. 2023-11-08T16:39:03.527 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T16:39:03.528 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T16:39:03.528 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T16:39:03.528 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:39:03.528 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T16:39:03.529 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T16:39:03.529 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T16:39:03.530 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:39:03.530 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T16:39:03.531 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T16:39:03.531 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T16:39:03.531 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:39:03.532 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T16:39:03.532 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T16:39:03.532 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T16:39:03.532 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:39:03.547 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T16:39:03.547 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T16:39:03.549 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T16:39:03.549 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T16:39:03.554 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T16:39:03.555 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T16:39:03.560 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T16:39:03.560 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T16:39:03.599 INFO:tasks.ceph.mds.c:Started 2023-11-08T16:39:03.599 DEBUG:teuthology.parallel:result is None 2023-11-08T16:39:03.606 INFO:tasks.ceph.mds.a:Started 2023-11-08T16:39:03.607 DEBUG:teuthology.parallel:result is None 2023-11-08T16:39:03.608 INFO:tasks.ceph.mds.d:Started 2023-11-08T16:39:03.610 DEBUG:teuthology.parallel:result is None 2023-11-08T16:39:03.617 INFO:tasks.ceph.mds.b:Started 2023-11-08T16:39:03.617 DEBUG:teuthology.parallel:result is None 2023-11-08T16:39:03.618 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T16:39:03.758 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:39:03.760 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:39:03.768 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:39:03.770 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2023-11-08T16:39:03.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.803+0000 7fea5d8e0640 1 Processor -- start 2023-11-08T16:39:03.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.803+0000 7fea5d8e0640 1 -- start start 2023-11-08T16:39:03.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.804+0000 7fea5d8e0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fea580c9580 0x7fea580b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:03.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.804+0000 7fea5d8e0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5806cc60 con 0x7fea5815aeb0 2023-11-08T16:39:03.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.804+0000 7fea5d8e0640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea580cabb0 con 0x7fea580c90f0 2023-11-08T16:39:03.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.804+0000 7fea5d8e0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea580cb3e0 con 0x7fea580c9580 2023-11-08T16:39:03.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.804+0000 7fea56ffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fea5815aeb0 0x7fea580ca4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:43438/0 (socket says 172.21.15.81:43438) 2023-11-08T16:39:03.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.804+0000 7fea56ffd640 1 -- 172.21.15.81:0/4282460618 learned_addr learned my addr 172.21.15.81:0/4282460618 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:03.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.804+0000 7fea577fe640 1 --2- 172.21.15.81:0/4282460618 >> v2:172.21.15.81:3300/0 conn(0x7fea580c9580 0x7fea580b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:03.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.805+0000 7fea567fc640 1 -- 172.21.15.81:0/4282460618 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373474073 0 0) 0x7fea5806cc60 con 0x7fea5815aeb0 2023-11-08T16:39:03.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.805+0000 7fea577fe640 1 -- 172.21.15.81:0/4282460618 >> v1:172.21.15.93:6789/0 conn(0x7fea580c90f0 legacy=0x7fea580b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:03.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.805+0000 7fea577fe640 1 -- 172.21.15.81:0/4282460618 >> v1:172.21.15.81:6789/0 conn(0x7fea5815aeb0 legacy=0x7fea580ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:03.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.805+0000 7fea577fe640 1 -- 172.21.15.81:0/4282460618 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea580cae20 con 0x7fea580c9580 2023-11-08T16:39:03.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.806+0000 7fea577fe640 1 --2- 172.21.15.81:0/4282460618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea580c9580 0x7fea580b2000 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fea50009d90 tx=0x7fea5002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2552ded331fb0b93 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:03.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.806+0000 7fea567fc640 1 -- 172.21.15.81:0/4282460618 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5002cd00 con 0x7fea580c9580 2023-11-08T16:39:03.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.806+0000 7fea567fc640 1 -- 172.21.15.81:0/4282460618 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea5002ce60 con 0x7fea580c9580 2023-11-08T16:39:03.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.806+0000 7fea567fc640 1 -- 172.21.15.81:0/4282460618 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50034660 con 0x7fea580c9580 2023-11-08T16:39:03.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.806+0000 7fea5d8e0640 1 -- 172.21.15.81:0/4282460618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea580c9580 msgr2=0x7fea580b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:03.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.806+0000 7fea5d8e0640 1 --2- 172.21.15.81:0/4282460618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea580c9580 0x7fea580b2000 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fea50009d90 tx=0x7fea5002bf50 comp rx=0 tx=0).stop 2023-11-08T16:39:03.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.806+0000 7fea5d8e0640 1 -- 172.21.15.81:0/4282460618 shutdown_connections 2023-11-08T16:39:03.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.806+0000 7fea5d8e0640 1 --2- 172.21.15.81:0/4282460618 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea580c9580 0x7fea580b2000 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.807+0000 7fea5d8e0640 1 -- 172.21.15.81:0/4282460618 >> 172.21.15.81:0/4282460618 conn(0x7fea580ceaa0 msgr2=0x7fea580cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:03.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.807+0000 7fea5d8e0640 1 -- 172.21.15.81:0/4282460618 shutdown_connections 2023-11-08T16:39:03.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.807+0000 7fea5d8e0640 1 -- 172.21.15.81:0/4282460618 wait complete. 2023-11-08T16:39:03.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.808+0000 7fea5d8e0640 1 Processor -- start 2023-11-08T16:39:03.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.808+0000 7fea5d8e0640 1 -- start start 2023-11-08T16:39:03.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.809+0000 7fea5d8e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea580c90f0 0x7fea58155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:03.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.809+0000 7fea5d8e0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea580c9580 0x7fea58155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:03.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.809+0000 7fea5d8e0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea5815aeb0 0x7fea5815b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:03.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.809+0000 7fea5d8e0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea58156c20 con 0x7fea5815aeb0 2023-11-08T16:39:03.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.809+0000 7fea5d8e0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea58155fa0 con 0x7fea580c9580 2023-11-08T16:39:03.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.809+0000 7fea5d8e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea58156280 con 0x7fea580c90f0 2023-11-08T16:39:03.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.809+0000 7fea577fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea5815aeb0 0x7fea5815b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:03.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.809+0000 7fea577fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea5815aeb0 0x7fea5815b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49654/0 (socket says 172.21.15.81:49654) 2023-11-08T16:39:03.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.809+0000 7fea577fe640 1 -- 172.21.15.81:0/62298123 learned_addr learned my addr 172.21.15.81:0/62298123 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:03.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.810+0000 7fea56ffd640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea580c90f0 0x7fea58155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:03.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.810+0000 7fea4ffff640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea580c9580 0x7fea58155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:03.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.810+0000 7fea577fe640 1 -- 172.21.15.81:0/62298123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea580c90f0 msgr2=0x7fea58155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:03.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.810+0000 7fea577fe640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea580c90f0 0x7fea58155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.810+0000 7fea577fe640 1 -- 172.21.15.81:0/62298123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea580c9580 msgr2=0x7fea58155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:03.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.810+0000 7fea577fe640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea580c9580 0x7fea58155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:03.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.810+0000 7fea577fe640 1 -- 172.21.15.81:0/62298123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea50009a70 con 0x7fea5815aeb0 2023-11-08T16:39:03.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.810+0000 7fea577fe640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea5815aeb0 0x7fea5815b290 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fea50009d90 tx=0x7fea50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:03.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.811+0000 7fea54ff9640 1 -- 172.21.15.81:0/62298123 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50003b90 con 0x7fea5815aeb0 2023-11-08T16:39:03.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.811+0000 7fea54ff9640 1 -- 172.21.15.81:0/62298123 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea50003cf0 con 0x7fea5815aeb0 2023-11-08T16:39:03.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.811+0000 7fea54ff9640 1 -- 172.21.15.81:0/62298123 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50004230 con 0x7fea5815aeb0 2023-11-08T16:39:03.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.811+0000 7fea5d8e0640 1 -- 172.21.15.81:0/62298123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea58156560 con 0x7fea5815aeb0 2023-11-08T16:39:03.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.811+0000 7fea5d8e0640 1 -- 172.21.15.81:0/62298123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea581502f0 con 0x7fea5815aeb0 2023-11-08T16:39:03.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.812+0000 7fea5d8e0640 1 -- 172.21.15.81:0/62298123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea58156d60 con 0x7fea5815aeb0 2023-11-08T16:39:03.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.813+0000 7fea54ff9640 1 -- 172.21.15.81:0/62298123 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea5003b050 con 0x7fea5815aeb0 2023-11-08T16:39:03.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.814+0000 7fea54ff9640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea3806f280 0x7fea38071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:03.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.814+0000 7fea54ff9640 1 -- 172.21.15.81:0/62298123 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5201+0+0 (secure 0 0 0) 0x7fea500abd00 con 0x7fea5815aeb0 2023-11-08T16:39:03.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.814+0000 7fea56ffd640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea3806f280 0x7fea38071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:03.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.814+0000 7fea56ffd640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea3806f280 0x7fea38071740 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fea480010e0 tx=0x7fea48009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:03.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:03.818+0000 7fea54ff9640 1 -- 172.21.15.81:0/62298123 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea58156d60 con 0x7fea5815aeb0 2023-11-08T16:39:04.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:04.019+0000 7fea5d8e0640 1 -- 172.21.15.81:0/62298123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fea58156d60 con 0x7fea5815aeb0 2023-11-08T16:39:05.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.122+0000 7fea54ff9640 1 -- 172.21.15.81:0/62298123 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v99) v1 ==== 97+0+0 (secure 0 0 0) 0x7fea58156d60 con 0x7fea5815aeb0 2023-11-08T16:39:05.124 INFO:teuthology.orchestra.run.smithi081.stderr: removed all blocklist entries 2023-11-08T16:39:05.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 -- 172.21.15.81:0/62298123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea3806f280 msgr2=0x7fea38071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea3806f280 0x7fea38071740 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fea480010e0 tx=0x7fea48009040 comp rx=0 tx=0).stop 2023-11-08T16:39:05.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 -- 172.21.15.81:0/62298123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea5815aeb0 msgr2=0x7fea5815b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea5815aeb0 0x7fea5815b290 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fea50009d90 tx=0x7fea50004a40 comp rx=0 tx=0).stop 2023-11-08T16:39:05.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 -- 172.21.15.81:0/62298123 shutdown_connections 2023-11-08T16:39:05.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea3806f280 0x7fea38071740 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea5815aeb0 0x7fea5815b290 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea580c9580 0x7fea58155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 --2- 172.21.15.81:0/62298123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea580c90f0 0x7fea58155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 -- 172.21.15.81:0/62298123 >> 172.21.15.81:0/62298123 conn(0x7fea580ceaa0 msgr2=0x7fea580b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:05.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 -- 172.21.15.81:0/62298123 shutdown_connections 2023-11-08T16:39:05.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.127+0000 7fea5d8e0640 1 -- 172.21.15.81:0/62298123 wait complete. 2023-11-08T16:39:05.150 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T16:39:05.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.361+0000 7f05a3a90640 1 Processor -- start 2023-11-08T16:39:05.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.361+0000 7f05a3a90640 1 -- start start 2023-11-08T16:39:05.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.363+0000 7f05a3a90640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f059c0ccd80 0x7f059c161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:05.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.363+0000 7f05a3a90640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f059c0ca1a0 0x7f059c0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:05.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.363+0000 7f05a3a90640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c0cd580 con 0x7f059c06cf10 2023-11-08T16:39:05.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.363+0000 7f05a3a90640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c0ca5d0 con 0x7f059c0ca1a0 2023-11-08T16:39:05.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.363+0000 7f05a3a90640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c0c93e0 con 0x7f059c0ccd80 2023-11-08T16:39:05.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.363+0000 7f05a202d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f059c0ca1a0 0x7f059c0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:05.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.363+0000 7f05a182c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f059c06cf10 0x7f059c0cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:43452/0 (socket says 172.21.15.81:43452) 2023-11-08T16:39:05.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.363+0000 7f05a182c640 1 -- 172.21.15.81:0/264353039 learned_addr learned my addr 172.21.15.81:0/264353039 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:05.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.363+0000 7f05a102b640 1 --2- 172.21.15.81:0/264353039 >> v2:172.21.15.93:3300/0 conn(0x7f059c0ccd80 0x7f059c161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:05.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.364+0000 7f05a202d640 1 -- 172.21.15.81:0/264353039 >> v2:172.21.15.93:3300/0 conn(0x7f059c0ccd80 msgr2=0x7f059c161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:05.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.364+0000 7f05a202d640 1 --2- 172.21.15.81:0/264353039 >> v2:172.21.15.93:3300/0 conn(0x7f059c0ccd80 0x7f059c161630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.364+0000 7f05a202d640 1 -- 172.21.15.81:0/264353039 >> v1:172.21.15.81:6789/0 conn(0x7f059c06cf10 legacy=0x7f059c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.364+0000 7f05a202d640 1 -- 172.21.15.81:0/264353039 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c0c96c0 con 0x7f059c0ca1a0 2023-11-08T16:39:05.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.364+0000 7f05a102b640 1 --2- 172.21.15.81:0/264353039 >> v2:172.21.15.93:3300/0 conn(0x7f059c0ccd80 0x7f059c161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:39:05.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.365+0000 7f05a202d640 1 --2- 172.21.15.81:0/264353039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c0ca1a0 0x7f059c0caeb0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f059c0c9050 tx=0x7f059402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b1498a1fe134b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:05.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.365+0000 7f05a082a640 1 -- 172.21.15.81:0/264353039 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059402cc70 con 0x7f059c0ca1a0 2023-11-08T16:39:05.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.365+0000 7f05a082a640 1 -- 172.21.15.81:0/264353039 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059402cdd0 con 0x7f059c0ca1a0 2023-11-08T16:39:05.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.365+0000 7f05a082a640 1 -- 172.21.15.81:0/264353039 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0594034660 con 0x7f059c0ca1a0 2023-11-08T16:39:05.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.366+0000 7f05a3a90640 1 -- 172.21.15.81:0/264353039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c0ca1a0 msgr2=0x7f059c0caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.366+0000 7f05a3a90640 1 --2- 172.21.15.81:0/264353039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c0ca1a0 0x7f059c0caeb0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f059c0c9050 tx=0x7f059402bec0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.366+0000 7f05a3a90640 1 -- 172.21.15.81:0/264353039 shutdown_connections 2023-11-08T16:39:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.366+0000 7f05a3a90640 1 --2- 172.21.15.81:0/264353039 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c0ca1a0 0x7f059c0caeb0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.366+0000 7f05a3a90640 1 --2- 172.21.15.81:0/264353039 >> v2:172.21.15.93:3300/0 conn(0x7f059c0ccd80 0x7f059c161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.366+0000 7f05a3a90640 1 -- 172.21.15.81:0/264353039 >> 172.21.15.81:0/264353039 conn(0x7f059c0ceaa0 msgr2=0x7f059c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:05.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.366+0000 7f05a3a90640 1 -- 172.21.15.81:0/264353039 shutdown_connections 2023-11-08T16:39:05.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.366+0000 7f05a3a90640 1 -- 172.21.15.81:0/264353039 wait complete. 2023-11-08T16:39:05.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.368+0000 7f05a3a90640 1 Processor -- start 2023-11-08T16:39:05.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.368+0000 7f05a3a90640 1 -- start start 2023-11-08T16:39:05.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.369+0000 7f05a3a90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c06cf10 0x7f059c0c6ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:05.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.369+0000 7f05a3a90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f059c0ca1a0 0x7f059c1f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.369+0000 7f05a3a90640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f059c0ccd80 0x7f059c0c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.369+0000 7f05a3a90640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c7680 con 0x7f059c06cf10 2023-11-08T16:39:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.369+0000 7f05a3a90640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c1f39f0 con 0x7f059c0ccd80 2023-11-08T16:39:05.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.369+0000 7f05a3a90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c0c70c0 con 0x7f059c0ca1a0 2023-11-08T16:39:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.369+0000 7f05a182c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c06cf10 0x7f059c0c6ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.369+0000 7f05a182c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c06cf10 0x7f059c0c6ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49672/0 (socket says 172.21.15.81:49672) 2023-11-08T16:39:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.369+0000 7f05a182c640 1 -- 172.21.15.81:0/969000166 learned_addr learned my addr 172.21.15.81:0/969000166 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:05.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.370+0000 7f05a182c640 1 -- 172.21.15.81:0/969000166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f059c0ca1a0 msgr2=0x7f059c1f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.370+0000 7f05a182c640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f059c0ca1a0 0x7f059c1f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.370+0000 7f05a182c640 1 -- 172.21.15.81:0/969000166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f059c0ccd80 msgr2=0x7f059c0c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.370+0000 7f05a182c640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f059c0ccd80 0x7f059c0c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.370+0000 7f05a182c640 1 -- 172.21.15.81:0/969000166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0594009a70 con 0x7f059c06cf10 2023-11-08T16:39:05.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.370+0000 7f05a182c640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c06cf10 0x7f059c0c6ae0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f059800c480 tx=0x7f059800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.370+0000 7f05927fc640 1 -- 172.21.15.81:0/969000166 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0598007480 con 0x7f059c06cf10 2023-11-08T16:39:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.370+0000 7f05927fc640 1 -- 172.21.15.81:0/969000166 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05980075e0 con 0x7f059c06cf10 2023-11-08T16:39:05.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.370+0000 7f05927fc640 1 -- 172.21.15.81:0/969000166 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05980059a0 con 0x7f059c06cf10 2023-11-08T16:39:05.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.370+0000 7f05a3a90640 1 -- 172.21.15.81:0/969000166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c0c73a0 con 0x7f059c06cf10 2023-11-08T16:39:05.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.370+0000 7f05a3a90640 1 -- 172.21.15.81:0/969000166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c0c27d0 con 0x7f059c06cf10 2023-11-08T16:39:05.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.371+0000 7f05a3a90640 1 -- 172.21.15.81:0/969000166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c0c2350 con 0x7f059c06cf10 2023-11-08T16:39:05.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.372+0000 7f05927fc640 1 -- 172.21.15.81:0/969000166 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0598005b00 con 0x7f059c06cf10 2023-11-08T16:39:05.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.373+0000 7f05927fc640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f058006aea0 0x7f058006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:05.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.373+0000 7f05927fc640 1 -- 172.21.15.81:0/969000166 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f05980888a0 con 0x7f059c06cf10 2023-11-08T16:39:05.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.374+0000 7f05a102b640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f058006aea0 0x7f058006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:05.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.374+0000 7f05a102b640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f058006aea0 0x7f058006d360 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f058c006fd0 tx=0x7f058c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:05.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.377+0000 7f05927fc640 1 -- 172.21.15.81:0/969000166 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f059c0c2350 con 0x7f059c06cf10 2023-11-08T16:39:05.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.640+0000 7f05a3a90640 1 -- 172.21.15.81:0/969000166 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f059c0c2350 con 0x7f059c06cf10 2023-11-08T16:39:05.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.641+0000 7f05927fc640 1 -- 172.21.15.81:0/969000166 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f059c0c2350 con 0x7f059c06cf10 2023-11-08T16:39:05.643 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:05.643 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:39:05.643 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_dump": [ 2023-11-08T16:39:05.643 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.643 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.a", 2023-11-08T16:39:05.644 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlL0bmNxAAlrzRL4depZdd+lBB5HN09g==", 2023-11-08T16:39:05.644 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.644 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:39:05.644 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:39:05.645 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:39:05.645 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.645 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.645 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.646 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.646 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.b", 2023-11-08T16:39:05.646 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tl2cFWHhAAvwSSL9e5996l9MBwNQzs6A==", 2023-11-08T16:39:05.646 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.646 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:39:05.647 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:39:05.647 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:39:05.647 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.647 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.648 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.648 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.648 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.c", 2023-11-08T16:39:05.648 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tlwo23DBAAl1sBjOJMIQRb1fH3o9XoiQ==", 2023-11-08T16:39:05.648 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.649 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:39:05.649 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:39:05.649 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:39:05.649 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.649 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.650 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.650 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.650 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.d", 2023-11-08T16:39:05.650 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tl5DdQLxAA1aU7e9epGtKnZ6hkHZTnsQ==", 2023-11-08T16:39:05.651 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.651 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:39:05.651 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:39:05.651 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:39:05.651 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.652 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.652 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.652 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.652 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.0", 2023-11-08T16:39:05.652 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQALt0tl3kkwKRAAgDwD1eQYk7qjvV+/gyol/w==", 2023-11-08T16:39:05.653 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.653 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:39:05.653 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:39:05.653 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.653 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.654 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.654 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.654 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.1", 2023-11-08T16:39:05.654 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAPt0tlpIh+BRAAUukXDBbd9Vev1TzIrv9sRg==", 2023-11-08T16:39:05.654 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.655 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:39:05.655 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:39:05.655 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.655 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.656 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.656 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.656 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.2", 2023-11-08T16:39:05.656 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQASt0tlmzJfDhAASO7zeg/9ddmLsoOTe+MIug==", 2023-11-08T16:39:05.656 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.657 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:39:05.657 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:39:05.657 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.657 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.657 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.658 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.658 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.3", 2023-11-08T16:39:05.658 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAVt0tlGoQSGRAAQsrZhrx4kpYUm3KEb6ZUGQ==", 2023-11-08T16:39:05.658 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.658 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:39:05.659 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:39:05.659 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.659 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.659 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.659 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.660 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.4", 2023-11-08T16:39:05.660 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAct0tlOS/kIxAAxC2I47B2WeGTFZxvdnmkMg==", 2023-11-08T16:39:05.660 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.660 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:39:05.660 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:39:05.661 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.661 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.661 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.661 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.661 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.5", 2023-11-08T16:39:05.662 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAgt0tlnAgEBRAADIRSyu/bxiykCc2uiIt0kA==", 2023-11-08T16:39:05.662 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.662 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:39:05.662 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:39:05.662 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.663 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.663 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.663 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.663 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.6", 2023-11-08T16:39:05.664 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAjt0tlrdfZHBAAHGdvBBzVwXMmy8GmsZLNbg==", 2023-11-08T16:39:05.664 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.664 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:39:05.664 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:39:05.664 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.665 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.665 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.665 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.665 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.7", 2023-11-08T16:39:05.665 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAmt0tl4nN9NRAAzV9abuBfhybYCsSFnqHEYA==", 2023-11-08T16:39:05.666 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.666 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:39:05.666 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:39:05.666 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.667 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.667 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.667 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.667 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.0", 2023-11-08T16:39:05.667 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ==", 2023-11-08T16:39:05.668 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.668 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:39:05.668 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow r", 2023-11-08T16:39:05.668 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T16:39:05.668 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.669 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.669 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.669 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.1", 2023-11-08T16:39:05.669 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlgn/qGBAA+WHDN1CJ1v62EbX1GEFBWA==", 2023-11-08T16:39:05.670 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.670 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:39:05.670 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:39:05.670 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:39:05.670 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:39:05.671 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.671 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.671 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.671 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.2", 2023-11-08T16:39:05.672 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlbQmgDRAAL/b9NwCBVT+uGxuv3VAWRw==", 2023-11-08T16:39:05.672 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.672 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:39:05.672 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:39:05.672 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:39:05.672 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:39:05.673 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.673 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.673 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.673 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.3", 2023-11-08T16:39:05.673 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlah+ZIxAAg53qCmUjPtG+l++2Byn1NA==", 2023-11-08T16:39:05.674 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.674 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:39:05.674 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:39:05.674 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:39:05.674 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:39:05.675 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.675 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.675 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.675 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.admin", 2023-11-08T16:39:05.675 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAEt0tlNjJ+KRAATvWBm8Guj09sQQB2U1Afzw==", 2023-11-08T16:39:05.675 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.676 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:39:05.676 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:39:05.676 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:39:05.676 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.676 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.676 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.677 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.677 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.x", 2023-11-08T16:39:05.677 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlhZFjLRAAANBB8roHZH4z4pE4C2Aq/Q==", 2023-11-08T16:39:05.677 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.677 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:39:05.678 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2023-11-08T16:39:05.678 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.678 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.678 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:05.678 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:05.679 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.y", 2023-11-08T16:39:05.679 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlNpDaIBAAtgLFNQitjQnLny6SRY7hHw==", 2023-11-08T16:39:05.679 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:39:05.679 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:39:05.679 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2023-11-08T16:39:05.680 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:39:05.680 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.680 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:05.680 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2023-11-08T16:39:05.680 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:39:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.643+0000 7f05a3a90640 1 -- 172.21.15.81:0/969000166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f058006aea0 msgr2=0x7f058006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.644+0000 7f05a3a90640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f058006aea0 0x7f058006d360 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f058c006fd0 tx=0x7f058c008040 comp rx=0 tx=0).stop 2023-11-08T16:39:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.644+0000 7f05a3a90640 1 -- 172.21.15.81:0/969000166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c06cf10 msgr2=0x7f059c0c6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.644+0000 7f05a3a90640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c06cf10 0x7f059c0c6ae0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f059800c480 tx=0x7f059800c940 comp rx=0 tx=0).stop 2023-11-08T16:39:05.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.644+0000 7f05a3a90640 1 -- 172.21.15.81:0/969000166 shutdown_connections 2023-11-08T16:39:05.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.644+0000 7f05a3a90640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f058006aea0 0x7f058006d360 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.644+0000 7f05a3a90640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f059c0ccd80 0x7f059c0c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.644+0000 7f05a3a90640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f059c0ca1a0 0x7f059c1f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.644+0000 7f05a3a90640 1 --2- 172.21.15.81:0/969000166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f059c06cf10 0x7f059c0c6ae0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.644+0000 7f05a3a90640 1 -- 172.21.15.81:0/969000166 >> 172.21.15.81:0/969000166 conn(0x7f059c0ceaa0 msgr2=0x7f059c06f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:05.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.644+0000 7f05a3a90640 1 -- 172.21.15.81:0/969000166 shutdown_connections 2023-11-08T16:39:05.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.644+0000 7f05a3a90640 1 -- 172.21.15.81:0/969000166 wait complete. 2023-11-08T16:39:05.685 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:39:05.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.899+0000 7f7d19e26640 1 Processor -- start 2023-11-08T16:39:05.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.899+0000 7f7d19e26640 1 -- start start 2023-11-08T16:39:05.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.901+0000 7f7d19e26640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7d140adf20 0x7f7d14165230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:05.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.901+0000 7f7d19e26640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d14165b90 con 0x7f7d140adb40 2023-11-08T16:39:05.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.901+0000 7f7d19e26640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d140ce3f0 con 0x7f7d14165770 2023-11-08T16:39:05.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.901+0000 7f7d19e26640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d140cb340 con 0x7f7d140adf20 2023-11-08T16:39:05.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.901+0000 7f7d137fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f7d140adb40 0x7f7d140c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:43462/0 (socket says 172.21.15.81:43462) 2023-11-08T16:39:05.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.901+0000 7f7d137fe640 1 -- 172.21.15.81:0/111129598 learned_addr learned my addr 172.21.15.81:0/111129598 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:05.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.901+0000 7f7d0bfff640 1 --2- 172.21.15.81:0/111129598 >> v2:172.21.15.81:3300/0 conn(0x7f7d140adf20 0x7f7d14165230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:05.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.902+0000 7f7d0bfff640 1 -- 172.21.15.81:0/111129598 >> v1:172.21.15.93:6789/0 conn(0x7f7d14165770 legacy=0x7f7d1406b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:05.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.902+0000 7f7d0bfff640 1 -- 172.21.15.81:0/111129598 >> v1:172.21.15.81:6789/0 conn(0x7f7d140adb40 legacy=0x7f7d140c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.902+0000 7f7d12ffd640 1 -- 172.21.15.81:0/111129598 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3144696926 0 0) 0x7f7d14165b90 con 0x7f7d140adb40 2023-11-08T16:39:05.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.902+0000 7f7d0bfff640 1 -- 172.21.15.81:0/111129598 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d140cac80 con 0x7f7d140adf20 2023-11-08T16:39:05.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.903+0000 7f7d0bfff640 1 --2- 172.21.15.81:0/111129598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d140adf20 0x7f7d14165230 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0097a0 tx=0x7f7cfc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b3f397a3aeb226 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:05.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.903+0000 7f7d12ffd640 1 -- 172.21.15.81:0/111129598 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc02e070 con 0x7f7d140adf20 2023-11-08T16:39:05.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.903+0000 7f7d12ffd640 1 -- 172.21.15.81:0/111129598 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0315a0 con 0x7f7d140adf20 2023-11-08T16:39:05.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.903+0000 7f7d12ffd640 1 -- 172.21.15.81:0/111129598 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc031930 con 0x7f7d140adf20 2023-11-08T16:39:05.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.904+0000 7f7d19e26640 1 -- 172.21.15.81:0/111129598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d140adf20 msgr2=0x7f7d14165230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.904+0000 7f7d19e26640 1 --2- 172.21.15.81:0/111129598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d140adf20 0x7f7d14165230 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc0097a0 tx=0x7f7cfc02b840 comp rx=0 tx=0).stop 2023-11-08T16:39:05.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.904+0000 7f7d19e26640 1 -- 172.21.15.81:0/111129598 shutdown_connections 2023-11-08T16:39:05.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.904+0000 7f7d19e26640 1 --2- 172.21.15.81:0/111129598 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d140adf20 0x7f7d14165230 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.904+0000 7f7d19e26640 1 -- 172.21.15.81:0/111129598 >> 172.21.15.81:0/111129598 conn(0x7f7d140cb490 msgr2=0x7f7d140cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:05.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.904+0000 7f7d19e26640 1 -- 172.21.15.81:0/111129598 shutdown_connections 2023-11-08T16:39:05.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.904+0000 7f7d19e26640 1 -- 172.21.15.81:0/111129598 wait complete. 2023-11-08T16:39:05.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.906+0000 7f7d19e26640 1 Processor -- start 2023-11-08T16:39:05.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.906+0000 7f7d19e26640 1 -- start start 2023-11-08T16:39:05.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d19e26640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d140adb40 0x7f7d14153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:05.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d19e26640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7d140adf20 0x7f7d140c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:05.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d19e26640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d14165770 0x7f7d140c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:05.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d19e26640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d14153ba0 con 0x7f7d14165770 2023-11-08T16:39:05.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d19e26640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d141625e0 con 0x7f7d140adb40 2023-11-08T16:39:05.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d19e26640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140c29e0 con 0x7f7d140adf20 2023-11-08T16:39:05.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d13fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d14165770 0x7f7d140c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:05.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d13fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d14165770 0x7f7d140c4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49692/0 (socket says 172.21.15.81:49692) 2023-11-08T16:39:05.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d13fff640 1 -- 172.21.15.81:0/2557618400 learned_addr learned my addr 172.21.15.81:0/2557618400 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:05.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d0bfff640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7d140adf20 0x7f7d140c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:05.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d137fe640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d140adb40 0x7f7d14153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:05.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d13fff640 1 -- 172.21.15.81:0/2557618400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7d140adf20 msgr2=0x7f7d140c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d13fff640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7d140adf20 0x7f7d140c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d13fff640 1 -- 172.21.15.81:0/2557618400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d140adb40 msgr2=0x7f7d14153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:05.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d13fff640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d140adb40 0x7f7d14153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:05.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.907+0000 7f7d13fff640 1 -- 172.21.15.81:0/2557618400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc009400 con 0x7f7d14165770 2023-11-08T16:39:05.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.908+0000 7f7d13fff640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d14165770 0x7f7d140c4cb0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00c780 tx=0x7f7d0c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:05.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.908+0000 7f7d10ff9640 1 -- 172.21.15.81:0/2557618400 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c004b60 con 0x7f7d14165770 2023-11-08T16:39:05.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.908+0000 7f7d10ff9640 1 -- 172.21.15.81:0/2557618400 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c00ce90 con 0x7f7d14165770 2023-11-08T16:39:05.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.908+0000 7f7d10ff9640 1 -- 172.21.15.81:0/2557618400 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c00a850 con 0x7f7d14165770 2023-11-08T16:39:05.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.908+0000 7f7d19e26640 1 -- 172.21.15.81:0/2557618400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d14162020 con 0x7f7d14165770 2023-11-08T16:39:05.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.908+0000 7f7d19e26640 1 -- 172.21.15.81:0/2557618400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d140c2670 con 0x7f7d14165770 2023-11-08T16:39:05.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.909+0000 7f7d19e26640 1 -- 172.21.15.81:0/2557618400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1404ece0 con 0x7f7d14165770 2023-11-08T16:39:05.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.910+0000 7f7d10ff9640 1 -- 172.21.15.81:0/2557618400 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7d0c00a9b0 con 0x7f7d14165770 2023-11-08T16:39:05.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.911+0000 7f7d10ff9640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7cf406ae70 0x7f7cf406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:05.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.911+0000 7f7d10ff9640 1 -- 172.21.15.81:0/2557618400 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7d0c089490 con 0x7f7d14165770 2023-11-08T16:39:05.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.911+0000 7f7d137fe640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7cf406ae70 0x7f7cf406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:05.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.912+0000 7f7d137fe640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7cf406ae70 0x7f7cf406d330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7d04000f60 tx=0x7f7d04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:05.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:05.916+0000 7f7d10ff9640 1 -- 172.21.15.81:0/2557618400 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7d0c054880 con 0x7f7d14165770 2023-11-08T16:39:06.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.104+0000 7f7d19e26640 1 -- 172.21.15.81:0/2557618400 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d140c2b50 con 0x7f7d14165770 2023-11-08T16:39:06.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.106+0000 7f7d10ff9640 1 -- 172.21.15.81:0/2557618400 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) v1 ==== 74+0+12798 (secure 0 0 0) 0x7f7d0c057ea0 con 0x7f7d14165770 2023-11-08T16:39:06.107 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:06.108 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":99,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:39:05.119772+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:39:06.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.108+0000 7f7d19e26640 1 -- 172.21.15.81:0/2557618400 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7cf406ae70 msgr2=0x7f7cf406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.108+0000 7f7d19e26640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7cf406ae70 0x7f7cf406d330 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f7d04000f60 tx=0x7f7d04009040 comp rx=0 tx=0).stop 2023-11-08T16:39:06.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.108+0000 7f7d19e26640 1 -- 172.21.15.81:0/2557618400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d14165770 msgr2=0x7f7d140c4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.108+0000 7f7d19e26640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d14165770 0x7f7d140c4cb0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c00c780 tx=0x7f7d0c00cc40 comp rx=0 tx=0).stop 2023-11-08T16:39:06.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.109+0000 7f7d19e26640 1 -- 172.21.15.81:0/2557618400 shutdown_connections 2023-11-08T16:39:06.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.109+0000 7f7d19e26640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7cf406ae70 0x7f7cf406d330 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.109+0000 7f7d19e26640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7d14165770 0x7f7d140c4cb0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.109+0000 7f7d19e26640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7d140adf20 0x7f7d140c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.109+0000 7f7d19e26640 1 --2- 172.21.15.81:0/2557618400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7d140adb40 0x7f7d14153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.109+0000 7f7d19e26640 1 -- 172.21.15.81:0/2557618400 >> 172.21.15.81:0/2557618400 conn(0x7f7d140cb490 msgr2=0x7f7d140afd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:06.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.109+0000 7f7d19e26640 1 -- 172.21.15.81:0/2557618400 shutdown_connections 2023-11-08T16:39:06.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.109+0000 7f7d19e26640 1 -- 172.21.15.81:0/2557618400 wait complete. 2023-11-08T16:39:06.130 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:39:06.131 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:39:06.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.347+0000 7efee6b1a640 1 Processor -- start 2023-11-08T16:39:06.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.347+0000 7efee6b1a640 1 -- start start 2023-11-08T16:39:06.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.349+0000 7efee6b1a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efee0070420 0x7efee0071db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.349+0000 7efee6b1a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7efee006b7d0 0x7efee006bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.349+0000 7efee6b1a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee00cd0c0 con 0x7efee006e1b0 2023-11-08T16:39:06.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.349+0000 7efee6b1a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee00cdd40 con 0x7efee0070420 2023-11-08T16:39:06.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.349+0000 7efee6b1a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee006ea50 con 0x7efee006b7d0 2023-11-08T16:39:06.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.349+0000 7efee5b18640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efee0070420 0x7efee0071db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:06.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.349+0000 7efee5317640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7efee006e1b0 0x7efee006b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:43472/0 (socket says 172.21.15.81:43472) 2023-11-08T16:39:06.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.349+0000 7efee5b18640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efee0070420 0x7efee0071db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49700/0 (socket says 172.21.15.81:49700) 2023-11-08T16:39:06.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.349+0000 7efee5317640 1 -- 172.21.15.81:0/3652081691 learned_addr learned my addr 172.21.15.81:0/3652081691 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:06.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.350+0000 7efee6319640 1 --2- 172.21.15.81:0/3652081691 >> v2:172.21.15.93:3300/0 conn(0x7efee006b7d0 0x7efee006bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:06.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.350+0000 7efee5b18640 1 -- 172.21.15.81:0/3652081691 >> v2:172.21.15.93:3300/0 conn(0x7efee006b7d0 msgr2=0x7efee006bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:06.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.350+0000 7efee5b18640 1 --2- 172.21.15.81:0/3652081691 >> v2:172.21.15.93:3300/0 conn(0x7efee006b7d0 0x7efee006bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.350+0000 7efee5b18640 1 -- 172.21.15.81:0/3652081691 >> v1:172.21.15.81:6789/0 conn(0x7efee006e1b0 legacy=0x7efee006b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.350+0000 7efee5b18640 1 -- 172.21.15.81:0/3652081691 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee00cd3a0 con 0x7efee0070420 2023-11-08T16:39:06.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.350+0000 7efee4b16640 1 -- 172.21.15.81:0/3652081691 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280280069 0 0) 0x7efee00cd0c0 con 0x7efee006e1b0 2023-11-08T16:39:06.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.351+0000 7efee5b18640 1 --2- 172.21.15.81:0/3652081691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efee0070420 0x7efee0071db0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7efee009ce20 tx=0x7efedc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=979e828970a07f50 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:06.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.351+0000 7efee4b16640 1 -- 172.21.15.81:0/3652081691 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc031830 con 0x7efee0070420 2023-11-08T16:39:06.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.351+0000 7efee4b16640 1 -- 172.21.15.81:0/3652081691 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc031990 con 0x7efee0070420 2023-11-08T16:39:06.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.351+0000 7efee4b16640 1 -- 172.21.15.81:0/3652081691 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc031d20 con 0x7efee0070420 2023-11-08T16:39:06.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.352+0000 7efee6b1a640 1 -- 172.21.15.81:0/3652081691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efee0070420 msgr2=0x7efee0071db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.352+0000 7efee6b1a640 1 --2- 172.21.15.81:0/3652081691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efee0070420 0x7efee0071db0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7efee009ce20 tx=0x7efedc02b8d0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.352+0000 7efee6b1a640 1 -- 172.21.15.81:0/3652081691 shutdown_connections 2023-11-08T16:39:06.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.352+0000 7efee6b1a640 1 --2- 172.21.15.81:0/3652081691 >> v2:172.21.15.93:3300/0 conn(0x7efee006b7d0 0x7efee006bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.352+0000 7efee6b1a640 1 --2- 172.21.15.81:0/3652081691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efee0070420 0x7efee0071db0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.352+0000 7efee6b1a640 1 -- 172.21.15.81:0/3652081691 >> 172.21.15.81:0/3652081691 conn(0x7efee00b0c10 msgr2=0x7efee00b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:06.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.352+0000 7efee6b1a640 1 -- 172.21.15.81:0/3652081691 shutdown_connections 2023-11-08T16:39:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.352+0000 7efee6b1a640 1 -- 172.21.15.81:0/3652081691 wait complete. 2023-11-08T16:39:06.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.354+0000 7efee6b1a640 1 Processor -- start 2023-11-08T16:39:06.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.354+0000 7efee6b1a640 1 -- start start 2023-11-08T16:39:06.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee6b1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efee006b7d0 0x7efee00a97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee6b1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efee006e1b0 0x7efee0153f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee6b1a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efee0070420 0x7efee015c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee6b1a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee0155660 con 0x7efee006b7d0 2023-11-08T16:39:06.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee6b1a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee0155af0 con 0x7efee0070420 2023-11-08T16:39:06.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee6b1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee0154a50 con 0x7efee006e1b0 2023-11-08T16:39:06.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee5b18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efee006b7d0 0x7efee00a97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:06.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee5b18640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efee006b7d0 0x7efee00a97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49708/0 (socket says 172.21.15.81:49708) 2023-11-08T16:39:06.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee5b18640 1 -- 172.21.15.81:0/3219031264 learned_addr learned my addr 172.21.15.81:0/3219031264 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:06.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee5b18640 1 -- 172.21.15.81:0/3219031264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efee006e1b0 msgr2=0x7efee0153f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee5b18640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efee006e1b0 0x7efee0153f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee5b18640 1 -- 172.21.15.81:0/3219031264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efee0070420 msgr2=0x7efee015c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee5b18640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efee0070420 0x7efee015c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.355+0000 7efee5b18640 1 -- 172.21.15.81:0/3219031264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc009400 con 0x7efee006b7d0 2023-11-08T16:39:06.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.356+0000 7efee5b18640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efee006b7d0 0x7efee00a97b0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7efedc009890 tx=0x7efedc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:06.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.356+0000 7efed67fc640 1 -- 172.21.15.81:0/3219031264 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc03ba70 con 0x7efee006b7d0 2023-11-08T16:39:06.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.356+0000 7efed67fc640 1 -- 172.21.15.81:0/3219031264 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efedc03bbf0 con 0x7efee006b7d0 2023-11-08T16:39:06.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.356+0000 7efed67fc640 1 -- 172.21.15.81:0/3219031264 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc045400 con 0x7efee006b7d0 2023-11-08T16:39:06.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.356+0000 7efee6b1a640 1 -- 172.21.15.81:0/3219031264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee0154d30 con 0x7efee006b7d0 2023-11-08T16:39:06.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.356+0000 7efee6b1a640 1 -- 172.21.15.81:0/3219031264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee01552f0 con 0x7efee006b7d0 2023-11-08T16:39:06.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.357+0000 7efee6b1a640 1 -- 172.21.15.81:0/3219031264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee00685e0 con 0x7efee006b7d0 2023-11-08T16:39:06.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.358+0000 7efed67fc640 1 -- 172.21.15.81:0/3219031264 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efedc04e440 con 0x7efee006b7d0 2023-11-08T16:39:06.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.359+0000 7efed67fc640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efec406aea0 0x7efec406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.359+0000 7efed67fc640 1 -- 172.21.15.81:0/3219031264 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efedc0ab5d0 con 0x7efee006b7d0 2023-11-08T16:39:06.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.359+0000 7efee5317640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efec406aea0 0x7efec406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:06.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.360+0000 7efee5317640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efec406aea0 0x7efec406d360 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efed0007fd0 tx=0x7efed0007920 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:06.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.363+0000 7efed67fc640 1 -- 172.21.15.81:0/3219031264 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efedc076930 con 0x7efee006b7d0 2023-11-08T16:39:06.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.554+0000 7efee6b1a640 1 -- 172.21.15.81:0/3219031264 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efee004ece0 con 0x7efee006b7d0 2023-11-08T16:39:06.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.555+0000 7efed67fc640 1 -- 172.21.15.81:0/3219031264 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) v1 ==== 93+0+10 (secure 0 0 0) 0x7efedc079f50 con 0x7efee006b7d0 2023-11-08T16:39:06.557 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:39:06.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.557+0000 7efee6b1a640 1 -- 172.21.15.81:0/3219031264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efec406aea0 msgr2=0x7efec406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.557+0000 7efee6b1a640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efec406aea0 0x7efec406d360 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7efed0007fd0 tx=0x7efed0007920 comp rx=0 tx=0).stop 2023-11-08T16:39:06.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.557+0000 7efee6b1a640 1 -- 172.21.15.81:0/3219031264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efee006b7d0 msgr2=0x7efee00a97b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.557+0000 7efee6b1a640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efee006b7d0 0x7efee00a97b0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7efedc009890 tx=0x7efedc0046c0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.558+0000 7efee6b1a640 1 -- 172.21.15.81:0/3219031264 shutdown_connections 2023-11-08T16:39:06.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.558+0000 7efee6b1a640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efec406aea0 0x7efec406d360 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.558+0000 7efee6b1a640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efee0070420 0x7efee015c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.558+0000 7efee6b1a640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efee006e1b0 0x7efee0153f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.558+0000 7efee6b1a640 1 --2- 172.21.15.81:0/3219031264 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efee006b7d0 0x7efee00a97b0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.558+0000 7efee6b1a640 1 -- 172.21.15.81:0/3219031264 >> 172.21.15.81:0/3219031264 conn(0x7efee00b0c10 msgr2=0x7efee00aeaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:06.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.558+0000 7efee6b1a640 1 -- 172.21.15.81:0/3219031264 shutdown_connections 2023-11-08T16:39:06.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.558+0000 7efee6b1a640 1 -- 172.21.15.81:0/3219031264 wait complete. 2023-11-08T16:39:06.579 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T16:39:06.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.786+0000 7f9cdbf25640 1 Processor -- start 2023-11-08T16:39:06.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.786+0000 7f9cdbf25640 1 -- start start 2023-11-08T16:39:06.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.788+0000 7f9cdbf25640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9cd40c56b0 0x7f9cd4159530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.788+0000 7f9cdbf25640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9cd40c90f0 0x7f9cd40c9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.788+0000 7f9cdbf25640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd40c9a90 con 0x7f9cd406b2b0 2023-11-08T16:39:06.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.788+0000 7f9cdbf25640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd406d440 con 0x7f9cd40c56b0 2023-11-08T16:39:06.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.788+0000 7f9cdbf25640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd406be50 con 0x7f9cd40c90f0 2023-11-08T16:39:06.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.788+0000 7f9cd9cc1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9cd40c56b0 0x7f9cd4159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:06.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.788+0000 7f9cd9cc1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9cd40c56b0 0x7f9cd4159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49720/0 (socket says 172.21.15.81:49720) 2023-11-08T16:39:06.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.788+0000 7f9cd9cc1640 1 -- 172.21.15.81:0/674543745 learned_addr learned my addr 172.21.15.81:0/674543745 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:06.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.788+0000 7f9cd94c0640 1 --2- 172.21.15.81:0/674543745 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9cd40c90f0 0x7f9cd40c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:06.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.788+0000 7f9cd9cc1640 1 -- 172.21.15.81:0/674543745 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9cd40c90f0 msgr2=0x7f9cd40c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:06.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.789+0000 7f9cd9cc1640 1 --2- 172.21.15.81:0/674543745 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9cd40c90f0 0x7f9cd40c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.789+0000 7f9cd9cc1640 1 -- 172.21.15.81:0/674543745 >> v1:172.21.15.81:6789/0 conn(0x7f9cd406b2b0 legacy=0x7f9cd406b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.789+0000 7f9cd9cc1640 1 -- 172.21.15.81:0/674543745 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd406ce80 con 0x7f9cd40c56b0 2023-11-08T16:39:06.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.789+0000 7f9cd8cbf640 1 -- 172.21.15.81:0/674543745 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4232206171 0 0) 0x7f9cd40c9a90 con 0x7f9cd406b2b0 2023-11-08T16:39:06.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.789+0000 7f9cd94c0640 1 --2- 172.21.15.81:0/674543745 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9cd40c90f0 0x7f9cd40c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:06.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.790+0000 7f9cd9cc1640 1 --2- 172.21.15.81:0/674543745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd40c56b0 0x7f9cd4159530 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009850 tx=0x7f9cd002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2d55aa75f1c6e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.790+0000 7f9cd8cbf640 1 -- 172.21.15.81:0/674543745 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0004230 con 0x7f9cd40c56b0 2023-11-08T16:39:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.790+0000 7f9cd8cbf640 1 -- 172.21.15.81:0/674543745 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd0004390 con 0x7f9cd40c56b0 2023-11-08T16:39:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.790+0000 7f9cd8cbf640 1 -- 172.21.15.81:0/674543745 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0031830 con 0x7f9cd40c56b0 2023-11-08T16:39:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.790+0000 7f9cdbf25640 1 -- 172.21.15.81:0/674543745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd40c56b0 msgr2=0x7f9cd4159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.790+0000 7f9cdbf25640 1 --2- 172.21.15.81:0/674543745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd40c56b0 0x7f9cd4159530 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009850 tx=0x7f9cd002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.791+0000 7f9cdbf25640 1 -- 172.21.15.81:0/674543745 shutdown_connections 2023-11-08T16:39:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.791+0000 7f9cdbf25640 1 --2- 172.21.15.81:0/674543745 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9cd40c90f0 0x7f9cd40c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.791+0000 7f9cdbf25640 1 --2- 172.21.15.81:0/674543745 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd40c56b0 0x7f9cd4159530 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.791+0000 7f9cdbf25640 1 -- 172.21.15.81:0/674543745 >> 172.21.15.81:0/674543745 conn(0x7f9cd40ceaa0 msgr2=0x7f9cd40cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:06.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.791+0000 7f9cdbf25640 1 -- 172.21.15.81:0/674543745 shutdown_connections 2023-11-08T16:39:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.791+0000 7f9cdbf25640 1 -- 172.21.15.81:0/674543745 wait complete. 2023-11-08T16:39:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.792+0000 7f9cdbf25640 1 Processor -- start 2023-11-08T16:39:06.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.793+0000 7f9cdbf25640 1 -- start start 2023-11-08T16:39:06.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.793+0000 7f9cdbf25640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9cd406b2b0 0x7f9cd414bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.793+0000 7f9cdbf25640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9cd40c56b0 0x7f9cd40c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cdbf25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd40c90f0 0x7f9cd40c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cdbf25640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd4165dd0 con 0x7f9cd40c90f0 2023-11-08T16:39:06.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cdbf25640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd414c000 con 0x7f9cd406b2b0 2023-11-08T16:39:06.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cdbf25640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd4165af0 con 0x7f9cd40c56b0 2023-11-08T16:39:06.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cda4c2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd40c90f0 0x7f9cd40c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:06.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cda4c2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd40c90f0 0x7f9cd40c2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49734/0 (socket says 172.21.15.81:49734) 2023-11-08T16:39:06.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cda4c2640 1 -- 172.21.15.81:0/422828426 learned_addr learned my addr 172.21.15.81:0/422828426 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:06.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cd9cc1640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9cd406b2b0 0x7f9cd414bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cda4c2640 1 -- 172.21.15.81:0/422828426 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9cd40c56b0 msgr2=0x7f9cd40c25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cd94c0640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9cd40c56b0 0x7f9cd40c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cda4c2640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9cd40c56b0 0x7f9cd40c25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cda4c2640 1 -- 172.21.15.81:0/422828426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9cd406b2b0 msgr2=0x7f9cd414bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:06.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cda4c2640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9cd406b2b0 0x7f9cd414bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cda4c2640 1 -- 172.21.15.81:0/422828426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd0009460 con 0x7f9cd40c90f0 2023-11-08T16:39:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.794+0000 7f9cd9cc1640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9cd406b2b0 0x7f9cd414bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.795+0000 7f9cda4c2640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd40c90f0 0x7f9cd40c2b10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00c740 tx=0x7f9ccc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.795+0000 7f9cca7fc640 1 -- 172.21.15.81:0/422828426 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc003da0 con 0x7f9cd40c90f0 2023-11-08T16:39:06.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.795+0000 7f9cca7fc640 1 -- 172.21.15.81:0/422828426 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc00ce50 con 0x7f9cd40c90f0 2023-11-08T16:39:06.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.795+0000 7f9cca7fc640 1 -- 172.21.15.81:0/422828426 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc005290 con 0x7f9cd40c90f0 2023-11-08T16:39:06.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.795+0000 7f9cdbf25640 1 -- 172.21.15.81:0/422828426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd40c6220 con 0x7f9cd40c90f0 2023-11-08T16:39:06.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.795+0000 7f9cdbf25640 1 -- 172.21.15.81:0/422828426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd40c67e0 con 0x7f9cd40c90f0 2023-11-08T16:39:06.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.796+0000 7f9cdbf25640 1 -- 172.21.15.81:0/422828426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd40c6080 con 0x7f9cd40c90f0 2023-11-08T16:39:06.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.797+0000 7f9cca7fc640 1 -- 172.21.15.81:0/422828426 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ccc0053f0 con 0x7f9cd40c90f0 2023-11-08T16:39:06.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.798+0000 7f9cca7fc640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cb806ae70 0x7f9cb806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:06.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.798+0000 7f9cca7fc640 1 -- 172.21.15.81:0/422828426 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9ccc088a10 con 0x7f9cd40c90f0 2023-11-08T16:39:06.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.798+0000 7f9cd9cc1640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cb806ae70 0x7f9cb806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:06.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.799+0000 7f9cd9cc1640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cb806ae70 0x7f9cb806d330 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0035000 tx=0x7f9cd0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:06.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.802+0000 7f9cca7fc640 1 -- 172.21.15.81:0/422828426 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9cd40c6080 con 0x7f9cd40c90f0 2023-11-08T16:39:06.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.998+0000 7f9cdbf25640 1 -- 172.21.15.81:0/422828426 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9cd40c6080 con 0x7f9cd40c90f0 2023-11-08T16:39:07.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:06.998+0000 7f9cca7fc640 1 -- 172.21.15.81:0/422828426 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9cd40c6080 con 0x7f9cd40c90f0 2023-11-08T16:39:07.000 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:07.001 INFO:teuthology.orchestra.run.smithi081.stdout:[ 2023-11-08T16:39:07.001 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:39:07.001 INFO:teuthology.orchestra.run.smithi081.stdout: "poolnum": 1, 2023-11-08T16:39:07.001 INFO:teuthology.orchestra.run.smithi081.stdout: "poolname": ".mgr" 2023-11-08T16:39:07.001 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:07.001 INFO:teuthology.orchestra.run.smithi081.stdout:] 2023-11-08T16:39:07.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 -- 172.21.15.81:0/422828426 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cb806ae70 msgr2=0x7f9cb806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:07.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cb806ae70 0x7f9cb806d330 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0035000 tx=0x7f9cd0008030 comp rx=0 tx=0).stop 2023-11-08T16:39:07.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 -- 172.21.15.81:0/422828426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd40c90f0 msgr2=0x7f9cd40c2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:07.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd40c90f0 0x7f9cd40c2b10 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00c740 tx=0x7f9ccc00cc00 comp rx=0 tx=0).stop 2023-11-08T16:39:07.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 -- 172.21.15.81:0/422828426 shutdown_connections 2023-11-08T16:39:07.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9cb806ae70 0x7f9cb806d330 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:07.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9cd40c90f0 0x7f9cd40c2b10 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:07.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9cd40c56b0 0x7f9cd40c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:07.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 --2- 172.21.15.81:0/422828426 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9cd406b2b0 0x7f9cd414bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:07.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 -- 172.21.15.81:0/422828426 >> 172.21.15.81:0/422828426 conn(0x7f9cd40ceaa0 msgr2=0x7f9cd40cd010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:07.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 -- 172.21.15.81:0/422828426 shutdown_connections 2023-11-08T16:39:07.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.001+0000 7f9cdbf25640 1 -- 172.21.15.81:0/422828426 wait complete. 2023-11-08T16:39:07.021 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T16:39:07.022 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T16:39:07.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.235+0000 7febfff00640 1 Processor -- start 2023-11-08T16:39:07.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.236+0000 7febfff00640 1 -- start start 2023-11-08T16:39:07.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.237+0000 7febfff00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febf815aff0 0x7febf80cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:07.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.237+0000 7febfff00640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febf806d610 0x7febf806dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:07.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.237+0000 7febfff00640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf80cfdb0 con 0x7febf80d1870 2023-11-08T16:39:07.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.237+0000 7febfff00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf80cbeb0 con 0x7febf815aff0 2023-11-08T16:39:07.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.237+0000 7febfff00640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf80cc190 con 0x7febf806d610 2023-11-08T16:39:07.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.238+0000 7febfdc9c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febf815aff0 0x7febf80cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:07.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.238+0000 7febfdc9c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febf815aff0 0x7febf80cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49742/0 (socket says 172.21.15.81:49742) 2023-11-08T16:39:07.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.238+0000 7febfdc9c640 1 -- 172.21.15.81:0/4171958793 learned_addr learned my addr 172.21.15.81:0/4171958793 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:07.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.238+0000 7febfdc9c640 1 -- 172.21.15.81:0/4171958793 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febf806d610 msgr2=0x7febf806dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:07.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.238+0000 7febfdc9c640 1 --2- 172.21.15.81:0/4171958793 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febf806d610 0x7febf806dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:07.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.238+0000 7febfdc9c640 1 -- 172.21.15.81:0/4171958793 >> v1:172.21.15.93:6789/0 conn(0x7febf80d1870 legacy=0x7febf80cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:07.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.238+0000 7febfdc9c640 1 -- 172.21.15.81:0/4171958793 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf80cc470 con 0x7febf815aff0 2023-11-08T16:39:07.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.239+0000 7febfdc9c640 1 --2- 172.21.15.81:0/4171958793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf815aff0 0x7febf80cf4d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7febf4009850 tx=0x7febf402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b66c07638304d592 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:07.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.239+0000 7febfcc9a640 1 -- 172.21.15.81:0/4171958793 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4004230 con 0x7febf815aff0 2023-11-08T16:39:07.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.240+0000 7febfcc9a640 1 -- 172.21.15.81:0/4171958793 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf4004390 con 0x7febf815aff0 2023-11-08T16:39:07.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.240+0000 7febfcc9a640 1 -- 172.21.15.81:0/4171958793 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf4031830 con 0x7febf815aff0 2023-11-08T16:39:07.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.240+0000 7febfff00640 1 -- 172.21.15.81:0/4171958793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf815aff0 msgr2=0x7febf80cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:07.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.240+0000 7febfff00640 1 --2- 172.21.15.81:0/4171958793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf815aff0 0x7febf80cf4d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7febf4009850 tx=0x7febf402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:39:07.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.240+0000 7febfff00640 1 -- 172.21.15.81:0/4171958793 shutdown_connections 2023-11-08T16:39:07.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.240+0000 7febfff00640 1 --2- 172.21.15.81:0/4171958793 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febf806d610 0x7febf806dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:07.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.240+0000 7febfff00640 1 --2- 172.21.15.81:0/4171958793 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf815aff0 0x7febf80cf4d0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:07.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.240+0000 7febfff00640 1 -- 172.21.15.81:0/4171958793 >> 172.21.15.81:0/4171958793 conn(0x7febf80d0ce0 msgr2=0x7febf80d10f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:07.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.240+0000 7febfff00640 1 -- 172.21.15.81:0/4171958793 shutdown_connections 2023-11-08T16:39:07.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.240+0000 7febfff00640 1 -- 172.21.15.81:0/4171958793 wait complete. 2023-11-08T16:39:07.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.242+0000 7febfff00640 1 Processor -- start 2023-11-08T16:39:07.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.242+0000 7febfff00640 1 -- start start 2023-11-08T16:39:07.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.243+0000 7febfff00640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febf806d610 0x7febf815d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:07.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.243+0000 7febfff00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf80d1870 0x7febf815b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:07.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.243+0000 7febfff00640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf815aff0 0x7febf815ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:07.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.243+0000 7febfff00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf8159620 con 0x7febf80d1870 2023-11-08T16:39:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.243+0000 7febfff00640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf815c160 con 0x7febf815aff0 2023-11-08T16:39:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.243+0000 7febfff00640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf815c440 con 0x7febf806d610 2023-11-08T16:39:07.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.243+0000 7febfd49b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf80d1870 0x7febf815b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:07.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.244+0000 7febfd49b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf80d1870 0x7febf815b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49750/0 (socket says 172.21.15.81:49750) 2023-11-08T16:39:07.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.244+0000 7febfd49b640 1 -- 172.21.15.81:0/1460137865 learned_addr learned my addr 172.21.15.81:0/1460137865 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:07.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.244+0000 7febfdc9c640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febf806d610 0x7febf815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:07.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.244+0000 7febfe49d640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf815aff0 0x7febf815ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:07.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.244+0000 7febfd49b640 1 -- 172.21.15.81:0/1460137865 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febf806d610 msgr2=0x7febf815d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.244+0000 7febfd49b640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febf806d610 0x7febf815d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.244+0000 7febfd49b640 1 -- 172.21.15.81:0/1460137865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf815aff0 msgr2=0x7febf815ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.244+0000 7febfd49b640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf815aff0 0x7febf815ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.244+0000 7febfd49b640 1 -- 172.21.15.81:0/1460137865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf4009460 con 0x7febf80d1870 2023-11-08T16:39:07.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.244+0000 7febfe49d640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf815aff0 0x7febf815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:07.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.244+0000 7febfd49b640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf80d1870 0x7febf815b550 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7febec00c3f0 tx=0x7febec00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:07.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.245+0000 7febea7fc640 1 -- 172.21.15.81:0/1460137865 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec0035f0 con 0x7febf80d1870 2023-11-08T16:39:07.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.245+0000 7febea7fc640 1 -- 172.21.15.81:0/1460137865 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec003750 con 0x7febf80d1870 2023-11-08T16:39:07.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.245+0000 7febea7fc640 1 -- 172.21.15.81:0/1460137865 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec003b00 con 0x7febf80d1870 2023-11-08T16:39:07.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.245+0000 7febfff00640 1 -- 172.21.15.81:0/1460137865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf80b08e0 con 0x7febf80d1870 2023-11-08T16:39:07.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.245+0000 7febfff00640 1 -- 172.21.15.81:0/1460137865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf80b4250 con 0x7febf80d1870 2023-11-08T16:39:07.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.246+0000 7febfff00640 1 -- 172.21.15.81:0/1460137865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf80b0740 con 0x7febf80d1870 2023-11-08T16:39:07.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.247+0000 7febea7fc640 1 -- 172.21.15.81:0/1460137865 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7febec003c60 con 0x7febf80d1870 2023-11-08T16:39:07.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.248+0000 7febea7fc640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febdc06ae70 0x7febdc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:07.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.248+0000 7febea7fc640 1 -- 172.21.15.81:0/1460137865 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 4857+0+0 (secure 0 0 0) 0x7febec088680 con 0x7febf80d1870 2023-11-08T16:39:07.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.248+0000 7febfdc9c640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febdc06ae70 0x7febdc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:07.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.249+0000 7febfdc9c640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febdc06ae70 0x7febdc06d330 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7febf4035000 tx=0x7febf4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:07.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.253+0000 7febea7fc640 1 -- 172.21.15.81:0/1460137865 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7febf80b0740 con 0x7febf80d1870 2023-11-08T16:39:07.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:07.442+0000 7febfff00640 1 -- 172.21.15.81:0/1460137865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7febf80b0e00 con 0x7febf80d1870 2023-11-08T16:39:08.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.140+0000 7febea7fc640 1 -- 172.21.15.81:0/1460137865 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) v1 ==== 138+0+0 (secure 0 0 0) 0x7febf80b0e00 con 0x7febf80d1870 2023-11-08T16:39:08.143 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' created 2023-11-08T16:39:08.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 -- 172.21.15.81:0/1460137865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febdc06ae70 msgr2=0x7febdc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:08.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febdc06ae70 0x7febdc06d330 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7febf4035000 tx=0x7febf4008030 comp rx=0 tx=0).stop 2023-11-08T16:39:08.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 -- 172.21.15.81:0/1460137865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf80d1870 msgr2=0x7febf815b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:08.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf80d1870 0x7febf815b550 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7febec00c3f0 tx=0x7febec00c8b0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 -- 172.21.15.81:0/1460137865 shutdown_connections 2023-11-08T16:39:08.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7febdc06ae70 0x7febdc06d330 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febf815aff0 0x7febf815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febf80d1870 0x7febf815b550 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 --2- 172.21.15.81:0/1460137865 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febf806d610 0x7febf815d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 -- 172.21.15.81:0/1460137865 >> 172.21.15.81:0/1460137865 conn(0x7febf80d0ce0 msgr2=0x7febf80afc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:08.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 -- 172.21.15.81:0/1460137865 shutdown_connections 2023-11-08T16:39:08.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.144+0000 7febfff00640 1 -- 172.21.15.81:0/1460137865 wait complete. 2023-11-08T16:39:08.170 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T16:39:08.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.373+0000 7f3c95856640 1 Processor -- start 2023-11-08T16:39:08.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.374+0000 7f3c95856640 1 -- start start 2023-11-08T16:39:08.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.375+0000 7f3c95856640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c900cda90 0x7f3c900cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:08.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.375+0000 7f3c95856640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3c9006bd40 0x7f3c9006c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:08.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.375+0000 7f3c95856640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c900ce430 con 0x7f3c90159d10 2023-11-08T16:39:08.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.376+0000 7f3c95856640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c900ca850 con 0x7f3c9006bd40 2023-11-08T16:39:08.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.376+0000 7f3c95856640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900c9380 con 0x7f3c900cda90 2023-11-08T16:39:08.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.376+0000 7f3c8effd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3c90159d10 0x7f3c900ca140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:43498/0 (socket says 172.21.15.81:43498) 2023-11-08T16:39:08.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.376+0000 7f3c8effd640 1 -- 172.21.15.81:0/2728651666 learned_addr learned my addr 172.21.15.81:0/2728651666 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:08.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.376+0000 7f3c8f7fe640 1 --2- 172.21.15.81:0/2728651666 >> v2:172.21.15.93:3300/0 conn(0x7f3c9006bd40 0x7f3c9006c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:08.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.376+0000 7f3c8e7fc640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c900cda90 0x7f3c900cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:08.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.376+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68541788 0 0) 0x7f3c900ce430 con 0x7f3c90159d10 2023-11-08T16:39:08.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.376+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c7c003520 con 0x7f3c90159d10 2023-11-08T16:39:08.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1601734522 0 0) 0x7f3c7c003520 con 0x7f3c90159d10 2023-11-08T16:39:08.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c900cda90 msgr2=0x7f3c900cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:08.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c900cda90 0x7f3c900cdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 >> v2:172.21.15.93:3300/0 conn(0x7f3c9006bd40 msgr2=0x7f3c9006c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:08.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 --2- 172.21.15.81:0/2728651666 >> v2:172.21.15.93:3300/0 conn(0x7f3c9006bd40 0x7f3c9006c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c900c9660 con 0x7f3c90159d10 2023-11-08T16:39:08.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f3c800019f0 con 0x7f3c90159d10 2023-11-08T16:39:08.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 >> v1:172.21.15.81:6789/0 conn(0x7f3c90159d10 legacy=0x7f3c900ca140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:08.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c7c003970 0x7f3c7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:08.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c0047c0 0x7f3c7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:08.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c900cda90 0x7f3c7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:08.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900ca850 con 0x7f3c7c003970 2023-11-08T16:39:08.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c900c9380 con 0x7f3c7c0047c0 2023-11-08T16:39:08.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7c00db90 con 0x7f3c900cda90 2023-11-08T16:39:08.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.377+0000 7f3c8effd640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c7c003970 0x7f3c7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:08.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.378+0000 7f3c8e7fc640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c0047c0 0x7f3c7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:08.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.378+0000 7f3c8f7fe640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c900cda90 0x7f3c7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:08.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.378+0000 7f3c8effd640 1 -- 172.21.15.81:0/2728651666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c900cda90 msgr2=0x7f3c7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:08.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.378+0000 7f3c8effd640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c900cda90 0x7f3c7c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.378+0000 7f3c8effd640 1 -- 172.21.15.81:0/2728651666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c0047c0 msgr2=0x7f3c7c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:08.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.378+0000 7f3c8effd640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c0047c0 0x7f3c7c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.378+0000 7f3c8effd640 1 -- 172.21.15.81:0/2728651666 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c7c003520 con 0x7f3c7c003970 2023-11-08T16:39:08.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.379+0000 7f3c8effd640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c7c003970 0x7f3c7c003d90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000a860 tx=0x7f3c7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfd592925dd78e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:08.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.379+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c80034630 con 0x7f3c7c003970 2023-11-08T16:39:08.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.379+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c8002f070 con 0x7f3c7c003970 2023-11-08T16:39:08.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.379+0000 7f3c8dffb640 1 -- 172.21.15.81:0/2728651666 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c80034a10 con 0x7f3c7c003970 2023-11-08T16:39:08.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.380+0000 7f3c95856640 1 -- 172.21.15.81:0/2728651666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c7c003970 msgr2=0x7f3c7c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:08.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.380+0000 7f3c95856640 1 --2- 172.21.15.81:0/2728651666 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c7c003970 0x7f3c7c003d90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000a860 tx=0x7f3c7c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.380+0000 7f3c8f7fe640 1 -- 172.21.15.81:0/2728651666 reap_dead start 2023-11-08T16:39:08.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.380+0000 7f3c95856640 1 -- 172.21.15.81:0/2728651666 shutdown_connections 2023-11-08T16:39:08.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.380+0000 7f3c95856640 1 -- 172.21.15.81:0/2728651666 >> 172.21.15.81:0/2728651666 conn(0x7f3c900cb500 msgr2=0x7f3c900cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:08.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.380+0000 7f3c95856640 1 -- 172.21.15.81:0/2728651666 shutdown_connections 2023-11-08T16:39:08.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.380+0000 7f3c95856640 1 -- 172.21.15.81:0/2728651666 wait complete. 2023-11-08T16:39:08.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.382+0000 7f3c95856640 1 Processor -- start 2023-11-08T16:39:08.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.382+0000 7f3c95856640 1 -- start start 2023-11-08T16:39:08.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c95856640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c003970 0x7f3c90143610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:08.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c95856640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c900acae0 0x7f3c90145910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:08.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c95856640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c90144d20 0x7f3c900ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:08.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c95856640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c90143ca0 con 0x7f3c900acae0 2023-11-08T16:39:08.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c95856640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c90143f80 con 0x7f3c7c003970 2023-11-08T16:39:08.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c95856640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c90144260 con 0x7f3c90144d20 2023-11-08T16:39:08.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c900acae0 0x7f3c90145910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:08.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c8e7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c900acae0 0x7f3c90145910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49768/0 (socket says 172.21.15.81:49768) 2023-11-08T16:39:08.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c8e7fc640 1 -- 172.21.15.81:0/4216422042 learned_addr learned my addr 172.21.15.81:0/4216422042 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:08.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c8effd640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c003970 0x7f3c90143610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:08.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c8e7fc640 1 -- 172.21.15.81:0/4216422042 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c90144d20 msgr2=0x7f3c900ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:08.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c8f7fe640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c90144d20 0x7f3c900ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:08.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c8e7fc640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c90144d20 0x7f3c900ae4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c8e7fc640 1 -- 172.21.15.81:0/4216422042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c003970 msgr2=0x7f3c90143610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:08.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.383+0000 7f3c8e7fc640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c003970 0x7f3c90143610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:08.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.384+0000 7f3c8e7fc640 1 -- 172.21.15.81:0/4216422042 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c90144540 con 0x7f3c900acae0 2023-11-08T16:39:08.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.384+0000 7f3c8effd640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c003970 0x7f3c90143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:08.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.384+0000 7f3c8e7fc640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c900acae0 0x7f3c90145910 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3c78011540 tx=0x7f3c78011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:08.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.384+0000 7f3c87fff640 1 -- 172.21.15.81:0/4216422042 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78015930 con 0x7f3c900acae0 2023-11-08T16:39:08.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.384+0000 7f3c87fff640 1 -- 172.21.15.81:0/4216422042 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c78015a90 con 0x7f3c900acae0 2023-11-08T16:39:08.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.384+0000 7f3c87fff640 1 -- 172.21.15.81:0/4216422042 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78015e60 con 0x7f3c900acae0 2023-11-08T16:39:08.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.384+0000 7f3c95856640 1 -- 172.21.15.81:0/4216422042 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c90144820 con 0x7f3c900acae0 2023-11-08T16:39:08.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.384+0000 7f3c95856640 1 -- 172.21.15.81:0/4216422042 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c90145e50 con 0x7f3c900acae0 2023-11-08T16:39:08.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.385+0000 7f3c95856640 1 -- 172.21.15.81:0/4216422042 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c900743f0 con 0x7f3c900acae0 2023-11-08T16:39:08.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.387+0000 7f3c87fff640 1 -- 172.21.15.81:0/4216422042 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3c7800f460 con 0x7f3c900acae0 2023-11-08T16:39:08.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.388+0000 7f3c87fff640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7006ae70 0x7f3c7006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:08.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.388+0000 7f3c87fff640 1 -- 172.21.15.81:0/4216422042 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f3c78089c40 con 0x7f3c900acae0 2023-11-08T16:39:08.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.388+0000 7f3c8effd640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7006ae70 0x7f3c7006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:08.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.389+0000 7f3c8effd640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7006ae70 0x7f3c7006d330 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3c80034000 tx=0x7f3c80009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:08.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.393+0000 7f3c87fff640 1 -- 172.21.15.81:0/4216422042 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3c78054ed0 con 0x7f3c900acae0 2023-11-08T16:39:08.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:08.579+0000 7f3c95856640 1 -- 172.21.15.81:0/4216422042 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3c9006b880 con 0x7f3c900acae0 2023-11-08T16:39:09.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.146+0000 7f3c87fff640 1 -- 172.21.15.81:0/4216422042 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3c780584f0 con 0x7f3c900acae0 2023-11-08T16:39:09.148 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' created 2023-11-08T16:39:09.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.150+0000 7f3c95856640 1 -- 172.21.15.81:0/4216422042 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7006ae70 msgr2=0x7f3c7006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:09.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.150+0000 7f3c95856640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7006ae70 0x7f3c7006d330 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f3c80034000 tx=0x7f3c80009030 comp rx=0 tx=0).stop 2023-11-08T16:39:09.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.150+0000 7f3c95856640 1 -- 172.21.15.81:0/4216422042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c900acae0 msgr2=0x7f3c90145910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:09.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.150+0000 7f3c95856640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c900acae0 0x7f3c90145910 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f3c78011540 tx=0x7f3c78011a00 comp rx=0 tx=0).stop 2023-11-08T16:39:09.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.150+0000 7f3c95856640 1 -- 172.21.15.81:0/4216422042 shutdown_connections 2023-11-08T16:39:09.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.150+0000 7f3c95856640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3c7006ae70 0x7f3c7006d330 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:09.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.150+0000 7f3c95856640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3c90144d20 0x7f3c900ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:09.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.150+0000 7f3c95856640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3c900acae0 0x7f3c90145910 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:09.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.150+0000 7f3c95856640 1 --2- 172.21.15.81:0/4216422042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3c7c003970 0x7f3c90143610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:09.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.150+0000 7f3c95856640 1 -- 172.21.15.81:0/4216422042 >> 172.21.15.81:0/4216422042 conn(0x7f3c900cb500 msgr2=0x7f3c9006c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:09.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.150+0000 7f3c95856640 1 -- 172.21.15.81:0/4216422042 shutdown_connections 2023-11-08T16:39:09.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.151+0000 7f3c95856640 1 -- 172.21.15.81:0/4216422042 wait complete. 2023-11-08T16:39:09.185 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T16:39:09.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.369+0000 7f4ec5c56640 1 Processor -- start 2023-11-08T16:39:09.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.369+0000 7f4ec5c56640 1 -- start start 2023-11-08T16:39:09.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.370+0000 7f4ec5c56640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec00ce0d0 0x7f4ec00caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:09.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.370+0000 7f4ec5c56640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec00cda10 con 0x7f4ec015bfc0 2023-11-08T16:39:09.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.370+0000 7f4ec5c56640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec006fa30 con 0x7f4ec015f280 2023-11-08T16:39:09.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.370+0000 7f4ec5c56640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec00ce4b0 con 0x7f4ec00ce0d0 2023-11-08T16:39:09.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.371+0000 7f4eb7fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4ec015bfc0 0x7f4ec015eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:43500/0 (socket says 172.21.15.81:43500) 2023-11-08T16:39:09.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.371+0000 7f4eb7fff640 1 -- 172.21.15.81:0/57784065 learned_addr learned my addr 172.21.15.81:0/57784065 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:09.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.371+0000 7f4ebf7fe640 1 --2- 172.21.15.81:0/57784065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec00ce0d0 0x7f4ec00caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:09.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.371+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963453027 0 0) 0x7f4ec00cda10 con 0x7f4ec015bfc0 2023-11-08T16:39:09.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.371+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eac003520 con 0x7f4ec015bfc0 2023-11-08T16:39:09.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 694634856 0 0) 0x7f4eac003520 con 0x7f4ec015bfc0 2023-11-08T16:39:09.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec00ce0d0 msgr2=0x7f4ec00caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:09.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 --2- 172.21.15.81:0/57784065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec00ce0d0 0x7f4ec00caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:09.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 >> v1:172.21.15.93:6789/0 conn(0x7f4ec015f280 legacy=0x7f4ec01516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:09.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec006fca0 con 0x7f4ec015bfc0 2023-11-08T16:39:09.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f4ea8001670 con 0x7f4ec015bfc0 2023-11-08T16:39:09.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 >> v1:172.21.15.81:6789/0 conn(0x7f4ec015bfc0 legacy=0x7f4ec015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:09.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 --2- 172.21.15.81:0/57784065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 0x7f4eac003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:09.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 --2- 172.21.15.81:0/57784065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec00ce0d0 0x7f4eac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:09.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 --2- 172.21.15.81:0/57784065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eac004d00 0x7f4eac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:09.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec00ce4b0 con 0x7f4eac003a10 2023-11-08T16:39:09.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac00ba90 con 0x7f4eac004d00 2023-11-08T16:39:09.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac00bc50 con 0x7f4ec00ce0d0 2023-11-08T16:39:09.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebf7fe640 1 --2- 172.21.15.81:0/57784065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 0x7f4eac003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:09.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebf7fe640 1 -- 172.21.15.81:0/57784065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec00ce0d0 msgr2=0x7f4eac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:09.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebf7fe640 1 --2- 172.21.15.81:0/57784065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec00ce0d0 0x7f4eac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:09.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebf7fe640 1 -- 172.21.15.81:0/57784065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eac004d00 msgr2=0x7f4eac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:09.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebf7fe640 1 --2- 172.21.15.81:0/57784065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eac004d00 0x7f4eac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:09.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.372+0000 7f4ebf7fe640 1 -- 172.21.15.81:0/57784065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac003520 con 0x7f4eac003a10 2023-11-08T16:39:09.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.373+0000 7f4ebf7fe640 1 --2- 172.21.15.81:0/57784065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 0x7f4eac003e30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009a90 tx=0x7f4eac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb070cdd4ba22fa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:09.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.374+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb002c600 con 0x7f4eac003a10 2023-11-08T16:39:09.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.374+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb002c760 con 0x7f4eac003a10 2023-11-08T16:39:09.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.374+0000 7f4ebeffd640 1 -- 172.21.15.81:0/57784065 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb002caf0 con 0x7f4eac003a10 2023-11-08T16:39:09.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.374+0000 7f4ec5c56640 1 -- 172.21.15.81:0/57784065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 msgr2=0x7f4eac003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:09.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.374+0000 7f4ec5c56640 1 --2- 172.21.15.81:0/57784065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 0x7f4eac003e30 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0009a90 tx=0x7f4eac0057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:09.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.374+0000 7f4ebffff640 1 -- 172.21.15.81:0/57784065 reap_dead start 2023-11-08T16:39:09.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.374+0000 7f4ec5c56640 1 -- 172.21.15.81:0/57784065 shutdown_connections 2023-11-08T16:39:09.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.374+0000 7f4ec5c56640 1 -- 172.21.15.81:0/57784065 >> 172.21.15.81:0/57784065 conn(0x7f4ec00cb580 msgr2=0x7f4ec00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:09.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.374+0000 7f4ec5c56640 1 -- 172.21.15.81:0/57784065 shutdown_connections 2023-11-08T16:39:09.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.374+0000 7f4ec5c56640 1 -- 172.21.15.81:0/57784065 wait complete. 2023-11-08T16:39:09.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.376+0000 7f4ec5c56640 1 Processor -- start 2023-11-08T16:39:09.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.376+0000 7f4ec5c56640 1 -- start start 2023-11-08T16:39:09.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.377+0000 7f4ec5c56640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 0x7f4ec014a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:09.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.377+0000 7f4ec5c56640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ec0147a00 0x7f4ec0156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:09.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.377+0000 7f4ec5c56640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec0156da0 0x7f4ec0155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:09.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.377+0000 7f4ec5c56640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec00c6610 con 0x7f4eac003a10 2023-11-08T16:39:09.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.377+0000 7f4ec5c56640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0157220 con 0x7f4ec0147a00 2023-11-08T16:39:09.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.377+0000 7f4ec5c56640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0156460 con 0x7f4ec0156da0 2023-11-08T16:39:09.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.377+0000 7f4ebf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 0x7f4ec014a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:09.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.377+0000 7f4ebf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 0x7f4ec014a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49786/0 (socket says 172.21.15.81:49786) 2023-11-08T16:39:09.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.377+0000 7f4ebf7fe640 1 -- 172.21.15.81:0/3737491710 learned_addr learned my addr 172.21.15.81:0/3737491710 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:09.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4eb7fff640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ec0147a00 0x7f4ec0156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:09.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4ebffff640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec0156da0 0x7f4ec0155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:09.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4ebf7fe640 1 -- 172.21.15.81:0/3737491710 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec0156da0 msgr2=0x7f4ec0155810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:09.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4ebf7fe640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec0156da0 0x7f4ec0155810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:09.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4ebf7fe640 1 -- 172.21.15.81:0/3737491710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ec0147a00 msgr2=0x7f4ec0156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:09.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4ebf7fe640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ec0147a00 0x7f4ec0156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:09.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4ebf7fe640 1 -- 172.21.15.81:0/3737491710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec00c5c70 con 0x7f4eac003a10 2023-11-08T16:39:09.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4ebffff640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec0156da0 0x7f4ec0155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:09.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4ebf7fe640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 0x7f4ec014a3a0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002c1a0 tx=0x7f4eb0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:09.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4ebcff9640 1 -- 172.21.15.81:0/3737491710 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0003810 con 0x7f4eac003a10 2023-11-08T16:39:09.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4ebcff9640 1 -- 172.21.15.81:0/3737491710 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb0003970 con 0x7f4eac003a10 2023-11-08T16:39:09.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.378+0000 7f4ec5c56640 1 -- 172.21.15.81:0/3737491710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec00c5f50 con 0x7f4eac003a10 2023-11-08T16:39:09.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.379+0000 7f4ebcff9640 1 -- 172.21.15.81:0/3737491710 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb0003d00 con 0x7f4eac003a10 2023-11-08T16:39:09.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.379+0000 7f4ec5c56640 1 -- 172.21.15.81:0/3737491710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec014a8e0 con 0x7f4eac003a10 2023-11-08T16:39:09.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.380+0000 7f4ec5c56640 1 -- 172.21.15.81:0/3737491710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec00c5a40 con 0x7f4eac003a10 2023-11-08T16:39:09.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.381+0000 7f4ebcff9640 1 -- 172.21.15.81:0/3737491710 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4eb0034440 con 0x7f4eac003a10 2023-11-08T16:39:09.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.382+0000 7f4ebcff9640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4ea006ae70 0x7f4ea006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:09.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.382+0000 7f4ebcff9640 1 -- 172.21.15.81:0/3737491710 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4eb00ab3b0 con 0x7f4eac003a10 2023-11-08T16:39:09.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.382+0000 7f4eb7fff640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4ea006ae70 0x7f4ea006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:09.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.383+0000 7f4eb7fff640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4ea006ae70 0x7f4ea006d330 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00b3d50 tx=0x7f4ea80016e0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:09.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.387+0000 7f4ebcff9640 1 -- 172.21.15.81:0/3737491710 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4eb00764b0 con 0x7f4eac003a10 2023-11-08T16:39:09.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:09.617+0000 7f4ec5c56640 1 -- 172.21.15.81:0/3737491710 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4ec00c6880 con 0x7f4eac003a10 2023-11-08T16:39:10.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.209+0000 7f4ebcff9640 1 -- 172.21.15.81:0/3737491710 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v82) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4eb0003e60 con 0x7f4eac003a10 2023-11-08T16:39:10.213 INFO:teuthology.orchestra.run.smithi081.stderr:new fs with metadata pool 13 and data pool 14 2023-11-08T16:39:10.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.223+0000 7f4ec5c56640 1 -- 172.21.15.81:0/3737491710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4ea006ae70 msgr2=0x7f4ea006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.223+0000 7f4ec5c56640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4ea006ae70 0x7f4ea006d330 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00b3d50 tx=0x7f4ea80016e0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.223+0000 7f4ec5c56640 1 -- 172.21.15.81:0/3737491710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 msgr2=0x7f4ec014a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.223+0000 7f4ec5c56640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 0x7f4ec014a3a0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f4eb002c1a0 tx=0x7f4eb0004730 comp rx=0 tx=0).stop 2023-11-08T16:39:10.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.226+0000 7f4ec5c56640 1 -- 172.21.15.81:0/3737491710 shutdown_connections 2023-11-08T16:39:10.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.226+0000 7f4ec5c56640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4ea006ae70 0x7f4ea006d330 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.226+0000 7f4ec5c56640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4ec0156da0 0x7f4ec0155810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.226+0000 7f4ec5c56640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ec0147a00 0x7f4ec0156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.226+0000 7f4ec5c56640 1 --2- 172.21.15.81:0/3737491710 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4eac003a10 0x7f4ec014a3a0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.226+0000 7f4ec5c56640 1 -- 172.21.15.81:0/3737491710 >> 172.21.15.81:0/3737491710 conn(0x7f4ec00cb580 msgr2=0x7f4ec00cd290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:10.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.227+0000 7f4ec5c56640 1 -- 172.21.15.81:0/3737491710 shutdown_connections 2023-11-08T16:39:10.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.227+0000 7f4ec5c56640 1 -- 172.21.15.81:0/3737491710 wait complete. 2023-11-08T16:39:10.258 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:39:10.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.429+0000 7f8bf2c58640 1 Processor -- start 2023-11-08T16:39:10.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.429+0000 7f8bf2c58640 1 -- start start 2023-11-08T16:39:10.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.430+0000 7f8bf2c58640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8bec161710 0x7f8bec0cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.430+0000 7f8bf2c58640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8bec0ccd30 0x7f8bec06ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.430+0000 7f8bf2c58640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bec161bc0 con 0x7f8bec15d2b0 2023-11-08T16:39:10.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.430+0000 7f8bf2c58640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bec0cd1c0 con 0x7f8bec161710 2023-11-08T16:39:10.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.430+0000 7f8bf2c58640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec15d690 con 0x7f8bec0ccd30 2023-11-08T16:39:10.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.431+0000 7f8bebfff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8bec161710 0x7f8bec0cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.431+0000 7f8bebfff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8bec161710 0x7f8bec0cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:54262/0 (socket says 172.21.15.81:54262) 2023-11-08T16:39:10.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.431+0000 7f8bebfff640 1 -- 172.21.15.81:0/3814373018 learned_addr learned my addr 172.21.15.81:0/3814373018 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:10.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.431+0000 7f8bf11f5640 1 --2- 172.21.15.81:0/3814373018 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8bec0ccd30 0x7f8bec06ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.432+0000 7f8bebfff640 1 -- 172.21.15.81:0/3814373018 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8bec0ccd30 msgr2=0x7f8bec06ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:10.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.432+0000 7f8bebfff640 1 --2- 172.21.15.81:0/3814373018 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8bec0ccd30 0x7f8bec06ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.432+0000 7f8bebfff640 1 -- 172.21.15.81:0/3814373018 >> v1:172.21.15.93:6789/0 conn(0x7f8bec15d2b0 legacy=0x7f8bec0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.432+0000 7f8bebfff640 1 -- 172.21.15.81:0/3814373018 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec06b980 con 0x7f8bec161710 2023-11-08T16:39:10.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.433+0000 7f8bebfff640 1 --2- 172.21.15.81:0/3814373018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8bec161710 0x7f8bec0cc7f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009870 tx=0x7f8bd802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fff28ed11b42219 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:10.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.434+0000 7f8beb7fe640 1 -- 172.21.15.81:0/3814373018 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8030840 con 0x7f8bec161710 2023-11-08T16:39:10.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.434+0000 7f8beb7fe640 1 -- 172.21.15.81:0/3814373018 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd80309a0 con 0x7f8bec161710 2023-11-08T16:39:10.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.434+0000 7f8beb7fe640 1 -- 172.21.15.81:0/3814373018 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8030d30 con 0x7f8bec161710 2023-11-08T16:39:10.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.434+0000 7f8bf2c58640 1 -- 172.21.15.81:0/3814373018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8bec161710 msgr2=0x7f8bec0cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.434+0000 7f8bf2c58640 1 --2- 172.21.15.81:0/3814373018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8bec161710 0x7f8bec0cc7f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009870 tx=0x7f8bd802b980 comp rx=0 tx=0).stop 2023-11-08T16:39:10.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.434+0000 7f8bf2c58640 1 -- 172.21.15.81:0/3814373018 shutdown_connections 2023-11-08T16:39:10.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.434+0000 7f8bf2c58640 1 --2- 172.21.15.81:0/3814373018 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8bec0ccd30 0x7f8bec06ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.434+0000 7f8bf2c58640 1 --2- 172.21.15.81:0/3814373018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8bec161710 0x7f8bec0cc7f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.434+0000 7f8bf2c58640 1 -- 172.21.15.81:0/3814373018 >> 172.21.15.81:0/3814373018 conn(0x7f8bec0cb490 msgr2=0x7f8bec0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:10.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.435+0000 7f8bf2c58640 1 -- 172.21.15.81:0/3814373018 shutdown_connections 2023-11-08T16:39:10.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.435+0000 7f8bf2c58640 1 -- 172.21.15.81:0/3814373018 wait complete. 2023-11-08T16:39:10.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.436+0000 7f8bf2c58640 1 Processor -- start 2023-11-08T16:39:10.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.436+0000 7f8bf2c58640 1 -- start start 2023-11-08T16:39:10.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.437+0000 7f8bf2c58640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8bec0ccd30 0x7f8bec1f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.437+0000 7f8bf2c58640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bec15d2b0 0x7f8bec1f5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.437+0000 7f8bf2c58640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8bec161710 0x7f8bec0ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.437+0000 7f8bf2c58640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec15aef0 con 0x7f8bec15d2b0 2023-11-08T16:39:10.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.437+0000 7f8bf2c58640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec1f64a0 con 0x7f8bec161710 2023-11-08T16:39:10.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.437+0000 7f8bf2c58640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bec1f6700 con 0x7f8bec0ccd30 2023-11-08T16:39:10.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.438+0000 7f8bebfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bec15d2b0 0x7f8bec1f5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.438+0000 7f8bebfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bec15d2b0 0x7f8bec1f5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49792/0 (socket says 172.21.15.81:49792) 2023-11-08T16:39:10.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.438+0000 7f8bebfff640 1 -- 172.21.15.81:0/2037304896 learned_addr learned my addr 172.21.15.81:0/2037304896 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:10.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.438+0000 7f8bf11f5640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8bec161710 0x7f8bec0ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.438+0000 7f8bf09f4640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8bec0ccd30 0x7f8bec1f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.438+0000 7f8bebfff640 1 -- 172.21.15.81:0/2037304896 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8bec0ccd30 msgr2=0x7f8bec1f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.438+0000 7f8bebfff640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8bec0ccd30 0x7f8bec1f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.438+0000 7f8bebfff640 1 -- 172.21.15.81:0/2037304896 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8bec161710 msgr2=0x7f8bec0ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.438+0000 7f8bebfff640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8bec161710 0x7f8bec0ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.438+0000 7f8bebfff640 1 -- 172.21.15.81:0/2037304896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd80094d0 con 0x7f8bec15d2b0 2023-11-08T16:39:10.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.439+0000 7f8bebfff640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bec15d2b0 0x7f8bec1f5f60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8bd802b9c0 tx=0x7f8bd8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:10.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.439+0000 7f8be97fa640 1 -- 172.21.15.81:0/2037304896 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd803e040 con 0x7f8bec15d2b0 2023-11-08T16:39:10.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.439+0000 7f8be97fa640 1 -- 172.21.15.81:0/2037304896 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd8039cb0 con 0x7f8bec15d2b0 2023-11-08T16:39:10.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.439+0000 7f8be97fa640 1 -- 172.21.15.81:0/2037304896 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd8044630 con 0x7f8bec15d2b0 2023-11-08T16:39:10.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.439+0000 7f8bf2c58640 1 -- 172.21.15.81:0/2037304896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bec1f69e0 con 0x7f8bec15d2b0 2023-11-08T16:39:10.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.439+0000 7f8bf2c58640 1 -- 172.21.15.81:0/2037304896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bec15ab80 con 0x7f8bec15d2b0 2023-11-08T16:39:10.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.440+0000 7f8bf2c58640 1 -- 172.21.15.81:0/2037304896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bec15b060 con 0x7f8bec15d2b0 2023-11-08T16:39:10.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.441+0000 7f8be97fa640 1 -- 172.21.15.81:0/2037304896 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8bd8043390 con 0x7f8bec15d2b0 2023-11-08T16:39:10.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.442+0000 7f8be97fa640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bd006ae70 0x7f8bd006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.442+0000 7f8be97fa640 1 -- 172.21.15.81:0/2037304896 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bd80aba80 con 0x7f8bec15d2b0 2023-11-08T16:39:10.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.442+0000 7f8bf09f4640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bd006ae70 0x7f8bd006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.443+0000 7f8bf09f4640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bd006ae70 0x7f8bd006d330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8be0001640 tx=0x7f8be0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:10.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.446+0000 7f8be97fa640 1 -- 172.21.15.81:0/2037304896 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8bd8076a90 con 0x7f8bec15d2b0 2023-11-08T16:39:10.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.635+0000 7f8bf2c58640 1 -- 172.21.15.81:0/2037304896 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bec04ece0 con 0x7f8bec15d2b0 2023-11-08T16:39:10.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.638+0000 7f8be97fa640 1 -- 172.21.15.81:0/2037304896 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16476 (secure 0 0 0) 0x7f8bd8034090 con 0x7f8bec15d2b0 2023-11-08T16:39:10.640 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:10.640 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":102,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:39:09.619184+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:39:07.492834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T16:39:08.634345+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:39:10.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.640+0000 7f8bf2c58640 1 -- 172.21.15.81:0/2037304896 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bd006ae70 msgr2=0x7f8bd006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.641+0000 7f8bf2c58640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bd006ae70 0x7f8bd006d330 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8be0001640 tx=0x7f8be0009040 comp rx=0 tx=0).stop 2023-11-08T16:39:10.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.641+0000 7f8bf2c58640 1 -- 172.21.15.81:0/2037304896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bec15d2b0 msgr2=0x7f8bec1f5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.641+0000 7f8bf2c58640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bec15d2b0 0x7f8bec1f5f60 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f8bd802b9c0 tx=0x7f8bd8004730 comp rx=0 tx=0).stop 2023-11-08T16:39:10.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.641+0000 7f8bf2c58640 1 -- 172.21.15.81:0/2037304896 shutdown_connections 2023-11-08T16:39:10.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.641+0000 7f8bf2c58640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8bd006ae70 0x7f8bd006d330 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.641+0000 7f8bf2c58640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8bec161710 0x7f8bec0ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.641+0000 7f8bf2c58640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8bec15d2b0 0x7f8bec1f5f60 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.641+0000 7f8bf2c58640 1 --2- 172.21.15.81:0/2037304896 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8bec0ccd30 0x7f8bec1f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.641+0000 7f8bf2c58640 1 -- 172.21.15.81:0/2037304896 >> 172.21.15.81:0/2037304896 conn(0x7f8bec0cb490 msgr2=0x7f8bec06c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:10.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.641+0000 7f8bf2c58640 1 -- 172.21.15.81:0/2037304896 shutdown_connections 2023-11-08T16:39:10.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.641+0000 7f8bf2c58640 1 -- 172.21.15.81:0/2037304896 wait complete. 2023-11-08T16:39:10.660 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:39:10.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.874+0000 7fd6f0bb6640 1 Processor -- start 2023-11-08T16:39:10.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.874+0000 7fd6f0bb6640 1 -- start start 2023-11-08T16:39:10.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.876+0000 7fd6f0bb6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6ec0adcd0 0x7fd6ec0ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.876+0000 7fd6f0bb6640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd6ec070050 0x7fd6ec0704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.876+0000 7fd6f0bb6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec0c9a70 con 0x7fd6ec0cc5e0 2023-11-08T16:39:10.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.876+0000 7fd6f0bb6640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ec0cc3e0 con 0x7fd6ec070050 2023-11-08T16:39:10.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.876+0000 7fd6f0bb6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec06d070 con 0x7fd6ec0adcd0 2023-11-08T16:39:10.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.876+0000 7fd6ead9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd6ec070050 0x7fd6ec0704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.877+0000 7fd6ea59c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6ec0adcd0 0x7fd6ec0ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.877+0000 7fd6ead9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd6ec070050 0x7fd6ec0704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:54278/0 (socket says 172.21.15.81:54278) 2023-11-08T16:39:10.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.877+0000 7fd6ead9d640 1 -- 172.21.15.81:0/4294054739 learned_addr learned my addr 172.21.15.81:0/4294054739 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:10.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.877+0000 7fd6ead9d640 1 -- 172.21.15.81:0/4294054739 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6ec0adcd0 msgr2=0x7fd6ec0ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:10.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.877+0000 7fd6ead9d640 1 --2- 172.21.15.81:0/4294054739 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6ec0adcd0 0x7fd6ec0ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.877+0000 7fd6ead9d640 1 -- 172.21.15.81:0/4294054739 >> v1:172.21.15.93:6789/0 conn(0x7fd6ec0cc5e0 legacy=0x7fd6ec0cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.877+0000 7fd6ead9d640 1 -- 172.21.15.81:0/4294054739 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec06d350 con 0x7fd6ec070050 2023-11-08T16:39:10.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.879+0000 7fd6ead9d640 1 --2- 172.21.15.81:0/4294054739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6ec070050 0x7fd6ec0704e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009e50 tx=0x7fd6e402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b16638979684a573 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:10.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.880+0000 7fd6e959a640 1 -- 172.21.15.81:0/4294054739 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e402cb00 con 0x7fd6ec070050 2023-11-08T16:39:10.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.880+0000 7fd6e959a640 1 -- 172.21.15.81:0/4294054739 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e402cc60 con 0x7fd6ec070050 2023-11-08T16:39:10.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.880+0000 7fd6e959a640 1 -- 172.21.15.81:0/4294054739 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e40343f0 con 0x7fd6ec070050 2023-11-08T16:39:10.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.881+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/4294054739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6ec070050 msgr2=0x7fd6ec0704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.881+0000 7fd6f0bb6640 1 --2- 172.21.15.81:0/4294054739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6ec070050 0x7fd6ec0704e0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009e50 tx=0x7fd6e402bf60 comp rx=0 tx=0).stop 2023-11-08T16:39:10.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.881+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/4294054739 shutdown_connections 2023-11-08T16:39:10.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.881+0000 7fd6f0bb6640 1 --2- 172.21.15.81:0/4294054739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6ec070050 0x7fd6ec0704e0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.881+0000 7fd6f0bb6640 1 --2- 172.21.15.81:0/4294054739 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6ec0adcd0 0x7fd6ec0ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.881+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/4294054739 >> 172.21.15.81:0/4294054739 conn(0x7fd6ec0cb490 msgr2=0x7fd6ec0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:10.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.881+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/4294054739 shutdown_connections 2023-11-08T16:39:10.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.881+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/4294054739 wait complete. 2023-11-08T16:39:10.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.883+0000 7fd6f0bb6640 1 Processor -- start 2023-11-08T16:39:10.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.883+0000 7fd6f0bb6640 1 -- start start 2023-11-08T16:39:10.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.884+0000 7fd6f0bb6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6ec070050 0x7fd6ec158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.884+0000 7fd6f0bb6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6ec0adcd0 0x7fd6ec15c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.884+0000 7fd6f0bb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ec0cc5e0 0x7fd6ec0708b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.884+0000 7fd6f0bb6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec15c0a0 con 0x7fd6ec0cc5e0 2023-11-08T16:39:10.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.884+0000 7fd6f0bb6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec14f140 con 0x7fd6ec0adcd0 2023-11-08T16:39:10.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.884+0000 7fd6f0bb6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec1585e0 con 0x7fd6ec070050 2023-11-08T16:39:10.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.884+0000 7fd6ead9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ec0cc5e0 0x7fd6ec0708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.884+0000 7fd6ead9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ec0cc5e0 0x7fd6ec0708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:49796/0 (socket says 172.21.15.81:49796) 2023-11-08T16:39:10.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.884+0000 7fd6ead9d640 1 -- 172.21.15.81:0/2734918712 learned_addr learned my addr 172.21.15.81:0/2734918712 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:10.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.884+0000 7fd6ea59c640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6ec070050 0x7fd6ec158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.884+0000 7fd6e9d9b640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6ec0adcd0 0x7fd6ec15c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.885+0000 7fd6ead9d640 1 -- 172.21.15.81:0/2734918712 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6ec070050 msgr2=0x7fd6ec158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.885+0000 7fd6ead9d640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6ec070050 0x7fd6ec158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.885+0000 7fd6ead9d640 1 -- 172.21.15.81:0/2734918712 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6ec0adcd0 msgr2=0x7fd6ec15c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:10.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.885+0000 7fd6ead9d640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6ec0adcd0 0x7fd6ec15c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:10.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.885+0000 7fd6ead9d640 1 -- 172.21.15.81:0/2734918712 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e4009b30 con 0x7fd6ec0cc5e0 2023-11-08T16:39:10.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.885+0000 7fd6ead9d640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ec0cc5e0 0x7fd6ec0708b0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009e50 tx=0x7fd6e4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:10.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.885+0000 7fd6daffd640 1 -- 172.21.15.81:0/2734918712 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4009510 con 0x7fd6ec0cc5e0 2023-11-08T16:39:10.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.885+0000 7fd6daffd640 1 -- 172.21.15.81:0/2734918712 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e403e450 con 0x7fd6ec0cc5e0 2023-11-08T16:39:10.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.885+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/2734918712 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec15b800 con 0x7fd6ec0cc5e0 2023-11-08T16:39:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.885+0000 7fd6daffd640 1 -- 172.21.15.81:0/2734918712 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e403e820 con 0x7fd6ec0cc5e0 2023-11-08T16:39:10.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.885+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/2734918712 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec15bdc0 con 0x7fd6ec0cc5e0 2023-11-08T16:39:10.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.887+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/2734918712 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec072170 con 0x7fd6ec0cc5e0 2023-11-08T16:39:10.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.888+0000 7fd6daffd640 1 -- 172.21.15.81:0/2734918712 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6e403e980 con 0x7fd6ec0cc5e0 2023-11-08T16:39:10.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.889+0000 7fd6daffd640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6cc06ae70 0x7fd6cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:10.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.889+0000 7fd6daffd640 1 -- 172.21.15.81:0/2734918712 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6e40ac380 con 0x7fd6ec0cc5e0 2023-11-08T16:39:10.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.889+0000 7fd6ea59c640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6cc06ae70 0x7fd6cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:10.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.890+0000 7fd6ea59c640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6cc06ae70 0x7fd6cc06d330 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009ad0 tx=0x7fd6dc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:10.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:10.894+0000 7fd6daffd640 1 -- 172.21.15.81:0/2734918712 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6e4077420 con 0x7fd6ec0cc5e0 2023-11-08T16:39:11.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.083+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/2734918712 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6ec04ece0 con 0x7fd6ec0cc5e0 2023-11-08T16:39:11.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.084+0000 7fd6daffd640 1 -- 172.21.15.81:0/2734918712 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16476 (secure 0 0 0) 0x7fd6e407aa40 con 0x7fd6ec0cc5e0 2023-11-08T16:39:11.086 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:11.087 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":102,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:39:09.619184+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:39:07.492834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T16:39:08.634345+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:39:11.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/2734918712 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6cc06ae70 msgr2=0x7fd6cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:11.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6cc06ae70 0x7fd6cc06d330 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc009ad0 tx=0x7fd6dc008040 comp rx=0 tx=0).stop 2023-11-08T16:39:11.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/2734918712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ec0cc5e0 msgr2=0x7fd6ec0708b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:11.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ec0cc5e0 0x7fd6ec0708b0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4009e50 tx=0x7fd6e4004a40 comp rx=0 tx=0).stop 2023-11-08T16:39:11.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/2734918712 shutdown_connections 2023-11-08T16:39:11.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6cc06ae70 0x7fd6cc06d330 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:11.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6ec0cc5e0 0x7fd6ec0708b0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:11.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6ec0adcd0 0x7fd6ec15c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:11.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 --2- 172.21.15.81:0/2734918712 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6ec070050 0x7fd6ec158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:11.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/2734918712 >> 172.21.15.81:0/2734918712 conn(0x7fd6ec0cb490 msgr2=0x7fd6ec0cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:11.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/2734918712 shutdown_connections 2023-11-08T16:39:11.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.087+0000 7fd6f0bb6640 1 -- 172.21.15.81:0/2734918712 wait complete. 2023-11-08T16:39:11.106 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T16:39:11.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.316+0000 7ff303fff640 1 Processor -- start 2023-11-08T16:39:11.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.316+0000 7ff303fff640 1 -- start start 2023-11-08T16:39:11.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.317+0000 7ff303fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff30415d310 0x7ff3040d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:11.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.317+0000 7ff303fff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3040ce0b0 con 0x7ff3040cfd90 2023-11-08T16:39:11.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.318+0000 7ff303fff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3040ce390 con 0x7ff30415cf30 2023-11-08T16:39:11.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.318+0000 7ff303fff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040d1be0 con 0x7ff30415d310 2023-11-08T16:39:11.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.318+0000 7ff3037fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff3040cfd90 0x7ff3040b5f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34172/0 (socket says 172.21.15.81:34172) 2023-11-08T16:39:11.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.318+0000 7ff3037fe640 1 -- 172.21.15.81:0/286502472 learned_addr learned my addr 172.21.15.81:0/286502472 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:11.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.318+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865833807 0 0) 0x7ff3040ce0b0 con 0x7ff3040cfd90 2023-11-08T16:39:11.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.318+0000 7ff3027fc640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff30415d310 0x7ff3040d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:11.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.318+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2ec003520 con 0x7ff3040cfd90 2023-11-08T16:39:11.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2188796623 0 0) 0x7ff2ec003520 con 0x7ff3040cfd90 2023-11-08T16:39:11.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff30415d310 msgr2=0x7ff3040d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:11.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff30415d310 0x7ff3040d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:11.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 >> v1:172.21.15.93:6789/0 conn(0x7ff30415cf30 legacy=0x7ff3040cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:11.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3040d1ec0 con 0x7ff3040cfd90 2023-11-08T16:39:11.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ff2fc0019f0 con 0x7ff3040cfd90 2023-11-08T16:39:11.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 >> v1:172.21.15.81:6789/0 conn(0x7ff3040cfd90 legacy=0x7ff3040b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:11.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2ec0039a0 0x7ff2ec003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:11.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2ec004300 0x7ff2ec0047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:11.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2ec004fe0 0x7ff2ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:11.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040d1be0 con 0x7ff2ec004300 2023-11-08T16:39:11.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec00bec0 con 0x7ff2ec004fe0 2023-11-08T16:39:11.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ec00c030 con 0x7ff2ec0039a0 2023-11-08T16:39:11.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff3027fc640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2ec004300 0x7ff2ec0047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:11.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff302ffd640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2ec0039a0 0x7ff2ec003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:11.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff3027fc640 1 -- 172.21.15.81:0/286502472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2ec0039a0 msgr2=0x7ff2ec003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:11.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.319+0000 7ff3027fc640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2ec0039a0 0x7ff2ec003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:11.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.320+0000 7ff3027fc640 1 -- 172.21.15.81:0/286502472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2ec004fe0 msgr2=0x7ff2ec00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:11.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.320+0000 7ff3027fc640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff2ec004fe0 0x7ff2ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:11.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.320+0000 7ff3027fc640 1 -- 172.21.15.81:0/286502472 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec003520 con 0x7ff2ec004300 2023-11-08T16:39:11.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.320+0000 7ff302ffd640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2ec0039a0 0x7ff2ec003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:11.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.321+0000 7ff3027fc640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2ec004300 0x7ff2ec0047e0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009ac0 tx=0x7ff2ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f61de54c732512 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:11.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.321+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f802c6b0 con 0x7ff2ec004300 2023-11-08T16:39:11.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.321+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f802c810 con 0x7ff2ec004300 2023-11-08T16:39:11.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.321+0000 7ff301ffb640 1 -- 172.21.15.81:0/286502472 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f802cba0 con 0x7ff2ec004300 2023-11-08T16:39:11.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.321+0000 7ff303fff640 1 -- 172.21.15.81:0/286502472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2ec004300 msgr2=0x7ff2ec0047e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:11.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.321+0000 7ff303fff640 1 --2- 172.21.15.81:0/286502472 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff2ec004300 0x7ff2ec0047e0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009ac0 tx=0x7ff2ec0057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:11.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.321+0000 7ff3037fe640 1 -- 172.21.15.81:0/286502472 reap_dead start 2023-11-08T16:39:11.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.321+0000 7ff303fff640 1 -- 172.21.15.81:0/286502472 shutdown_connections 2023-11-08T16:39:11.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.321+0000 7ff303fff640 1 -- 172.21.15.81:0/286502472 >> 172.21.15.81:0/286502472 conn(0x7ff30406c040 msgr2=0x7ff30406c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:11.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.322+0000 7ff303fff640 1 -- 172.21.15.81:0/286502472 shutdown_connections 2023-11-08T16:39:11.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.322+0000 7ff303fff640 1 -- 172.21.15.81:0/286502472 wait complete. 2023-11-08T16:39:11.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.323+0000 7ff303fff640 1 Processor -- start 2023-11-08T16:39:11.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.323+0000 7ff303fff640 1 -- start start 2023-11-08T16:39:11.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.324+0000 7ff303fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2ec004300 0x7ff3041efb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:11.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.324+0000 7ff303fff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3040b3510 0x7ff30415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:11.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.324+0000 7ff303fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff30415f860 0x7ff3040ca710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:11.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.324+0000 7ff303fff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30415fce0 con 0x7ff30415f860 2023-11-08T16:39:11.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.324+0000 7ff303fff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040b10d0 con 0x7ff3040b3510 2023-11-08T16:39:11.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.324+0000 7ff303fff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3041f00d0 con 0x7ff2ec004300 2023-11-08T16:39:11.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff3037fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff30415f860 0x7ff3040ca710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:11.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff3037fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff30415f860 0x7ff3040ca710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42558/0 (socket says 172.21.15.81:42558) 2023-11-08T16:39:11.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff3037fe640 1 -- 172.21.15.81:0/186313490 learned_addr learned my addr 172.21.15.81:0/186313490 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:11.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff3027fc640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3040b3510 0x7ff30415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:11.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff3037fe640 1 -- 172.21.15.81:0/186313490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2ec004300 msgr2=0x7ff3041efb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:11.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff3037fe640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2ec004300 0x7ff3041efb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:11.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff3037fe640 1 -- 172.21.15.81:0/186313490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3040b3510 msgr2=0x7ff30415f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:11.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff3037fe640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3040b3510 0x7ff30415f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:11.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff3037fe640 1 -- 172.21.15.81:0/186313490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3041f2860 con 0x7ff30415f860 2023-11-08T16:39:11.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff3037fe640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff30415f860 0x7ff3040ca710 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0012e0 tx=0x7ff2fc00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:11.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff2eb7fe640 1 -- 172.21.15.81:0/186313490 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00a610 con 0x7ff30415f860 2023-11-08T16:39:11.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff2eb7fe640 1 -- 172.21.15.81:0/186313490 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc00a7b0 con 0x7ff30415f860 2023-11-08T16:39:11.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff2eb7fe640 1 -- 172.21.15.81:0/186313490 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc00ab60 con 0x7ff30415f860 2023-11-08T16:39:11.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.325+0000 7ff303fff640 1 -- 172.21.15.81:0/186313490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3041f2b40 con 0x7ff30415f860 2023-11-08T16:39:11.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.326+0000 7ff303fff640 1 -- 172.21.15.81:0/186313490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3041f3100 con 0x7ff30415f860 2023-11-08T16:39:11.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.327+0000 7ff303fff640 1 -- 172.21.15.81:0/186313490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30404ece0 con 0x7ff30415f860 2023-11-08T16:39:11.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.328+0000 7ff2eb7fe640 1 -- 172.21.15.81:0/186313490 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff2fc00acc0 con 0x7ff30415f860 2023-11-08T16:39:11.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.329+0000 7ff2eb7fe640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff2f006ae70 0x7ff2f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:11.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.329+0000 7ff2eb7fe640 1 -- 172.21.15.81:0/186313490 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2fc088f10 con 0x7ff30415f860 2023-11-08T16:39:11.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.329+0000 7ff302ffd640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff2f006ae70 0x7ff2f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:11.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.330+0000 7ff302ffd640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff2f006ae70 0x7ff2f006d330 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40016d0 tx=0x7ff2f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:11.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.334+0000 7ff2eb7fe640 1 -- 172.21.15.81:0/186313490 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff2fc053f20 con 0x7ff30415f860 2023-11-08T16:39:11.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:11.556+0000 7ff303fff640 1 -- 172.21.15.81:0/186313490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff304066100 con 0x7ff30415f860 2023-11-08T16:39:12.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.221+0000 7ff2eb7fe640 1 -- 172.21.15.81:0/186313490 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v85) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff2fc01a070 con 0x7ff30415f860 2023-11-08T16:39:12.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.225+0000 7ff303fff640 1 -- 172.21.15.81:0/186313490 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff2f006ae70 msgr2=0x7ff2f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:12.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.225+0000 7ff303fff640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff2f006ae70 0x7ff2f006d330 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40016d0 tx=0x7ff2f4009040 comp rx=0 tx=0).stop 2023-11-08T16:39:12.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.225+0000 7ff303fff640 1 -- 172.21.15.81:0/186313490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff30415f860 msgr2=0x7ff3040ca710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:12.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.225+0000 7ff303fff640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff30415f860 0x7ff3040ca710 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0012e0 tx=0x7ff2fc00cf00 comp rx=0 tx=0).stop 2023-11-08T16:39:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.226+0000 7ff303fff640 1 -- 172.21.15.81:0/186313490 shutdown_connections 2023-11-08T16:39:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.226+0000 7ff303fff640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff2f006ae70 0x7ff2f006d330 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.226+0000 7ff303fff640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff30415f860 0x7ff3040ca710 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.226+0000 7ff303fff640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff3040b3510 0x7ff30415f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.226+0000 7ff303fff640 1 --2- 172.21.15.81:0/186313490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff2ec004300 0x7ff3041efb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.226+0000 7ff303fff640 1 -- 172.21.15.81:0/186313490 >> 172.21.15.81:0/186313490 conn(0x7ff30406c040 msgr2=0x7ff3040c6830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.226+0000 7ff303fff640 1 -- 172.21.15.81:0/186313490 shutdown_connections 2023-11-08T16:39:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.226+0000 7ff303fff640 1 -- 172.21.15.81:0/186313490 wait complete. 2023-11-08T16:39:12.251 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T16:39:12.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T16:39:12.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.470+0000 7f4c0e2e5640 1 Processor -- start 2023-11-08T16:39:12.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.470+0000 7f4c0e2e5640 1 -- start start 2023-11-08T16:39:12.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.471+0000 7f4c0e2e5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f4c08162ed0 0x7f4c081652c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:12.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.471+0000 7f4c0e2e5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080ce960 con 0x7f4c0815a0e0 2023-11-08T16:39:12.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.471+0000 7f4c0e2e5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c080c9480 con 0x7f4c080ccc10 2023-11-08T16:39:12.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.471+0000 7f4c0e2e5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c08071980 con 0x7f4c08162ed0 2023-11-08T16:39:12.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.472+0000 7f4c07fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f4c0815a0e0 0x7f4c080c9bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34188/0 (socket says 172.21.15.81:34188) 2023-11-08T16:39:12.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.472+0000 7f4c07fff640 1 -- 172.21.15.81:0/2717645905 learned_addr learned my addr 172.21.15.81:0/2717645905 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:12.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.472+0000 7f4c0c882640 1 --2- 172.21.15.81:0/2717645905 >> v2:172.21.15.93:3300/0 conn(0x7f4c08162ed0 0x7f4c081652c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:12.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.472+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120611639 0 0) 0x7f4bf8001110 con 0x7f4c0815a0e0 2023-11-08T16:39:12.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.472+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bf4003520 con 0x7f4c0815a0e0 2023-11-08T16:39:12.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.472+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1529671933 0 0) 0x7f4bf4003520 con 0x7f4c0815a0e0 2023-11-08T16:39:12.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.472+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 >> v2:172.21.15.93:3300/0 conn(0x7f4c08162ed0 msgr2=0x7f4c081652c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:12.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.472+0000 7f4c077fe640 1 --2- 172.21.15.81:0/2717645905 >> v2:172.21.15.93:3300/0 conn(0x7f4c08162ed0 0x7f4c081652c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:12.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.472+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 >> v1:172.21.15.93:6789/0 conn(0x7f4c080ccc10 legacy=0x7f4c080ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:12.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.472+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c0815a4c0 con 0x7f4c0815a0e0 2023-11-08T16:39:12.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.472+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f4bf8001800 con 0x7f4c0815a0e0 2023-11-08T16:39:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 >> v1:172.21.15.81:6789/0 conn(0x7f4c0815a0e0 legacy=0x7f4c080c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c077fe640 1 --2- 172.21.15.81:0/2717645905 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4bf40039e0 0x7f4bf4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c077fe640 1 --2- 172.21.15.81:0/2717645905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bf40047c0 0x7f4bf4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c077fe640 1 --2- 172.21.15.81:0/2717645905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bf4005180 0x7f4bf400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:12.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c08071980 con 0x7f4bf4005180 2023-11-08T16:39:12.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf400c090 con 0x7f4bf40047c0 2023-11-08T16:39:12.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf400c250 con 0x7f4bf40039e0 2023-11-08T16:39:12.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c0c882640 1 --2- 172.21.15.81:0/2717645905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bf4005180 0x7f4bf400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:12.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c0c882640 1 -- 172.21.15.81:0/2717645905 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4bf40039e0 msgr2=0x7f4bf4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:12.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4bfffff640 1 --2- 172.21.15.81:0/2717645905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bf40047c0 0x7f4bf4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:12.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c0c882640 1 --2- 172.21.15.81:0/2717645905 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4bf40039e0 0x7f4bf4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:12.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c0c882640 1 -- 172.21.15.81:0/2717645905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bf40047c0 msgr2=0x7f4bf4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:12.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c0c882640 1 --2- 172.21.15.81:0/2717645905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bf40047c0 0x7f4bf4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:12.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.473+0000 7f4c0c882640 1 -- 172.21.15.81:0/2717645905 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4003520 con 0x7f4bf4005180 2023-11-08T16:39:12.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.474+0000 7f4c0c882640 1 -- 172.21.15.81:0/2717645905 reap_dead start 2023-11-08T16:39:12.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.474+0000 7f4c0c882640 1 --2- 172.21.15.81:0/2717645905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bf4005180 0x7f4bf400b980 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009fc0 tx=0x7f4bf40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf7e8ef25df6851 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:12.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.475+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0002ee70 con 0x7f4bf4005180 2023-11-08T16:39:12.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.475+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c0000f040 con 0x7f4bf4005180 2023-11-08T16:39:12.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.475+0000 7f4c077fe640 1 -- 172.21.15.81:0/2717645905 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c00003bd0 con 0x7f4bf4005180 2023-11-08T16:39:12.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.475+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/2717645905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bf4005180 msgr2=0x7f4bf400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:12.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.475+0000 7f4c0e2e5640 1 --2- 172.21.15.81:0/2717645905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bf4005180 0x7f4bf400b980 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4c00009fc0 tx=0x7f4bf40057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:12.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.475+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/2717645905 shutdown_connections 2023-11-08T16:39:12.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.475+0000 7f4c0e2e5640 1 --2- 172.21.15.81:0/2717645905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4bf4005180 0x7f4bf400b980 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:12.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.475+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/2717645905 >> 172.21.15.81:0/2717645905 conn(0x7f4c080cb580 msgr2=0x7f4c080cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:12.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.475+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/2717645905 shutdown_connections 2023-11-08T16:39:12.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.475+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/2717645905 wait complete. 2023-11-08T16:39:12.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.477+0000 7f4c0e2e5640 1 Processor -- start 2023-11-08T16:39:12.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.477+0000 7f4c0e2e5640 1 -- start start 2023-11-08T16:39:12.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.478+0000 7f4c0e2e5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bf4005180 0x7f4c081f3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:12.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.478+0000 7f4c0e2e5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c081f27c0 0x7f4c08158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:12.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.478+0000 7f4c0e2e5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c08158e70 0x7f4c0815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:12.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.478+0000 7f4c0e2e5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c08159380 con 0x7f4c081f27c0 2023-11-08T16:39:12.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.478+0000 7f4c0e2e5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c081f3090 con 0x7f4bf4005180 2023-11-08T16:39:12.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.478+0000 7f4c0e2e5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c081f3370 con 0x7f4c08158e70 2023-11-08T16:39:12.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.478+0000 7f4bfffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c081f27c0 0x7f4c08158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:12.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4bfffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c081f27c0 0x7f4c08158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42580/0 (socket says 172.21.15.81:42580) 2023-11-08T16:39:12.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4bfffff640 1 -- 172.21.15.81:0/174081795 learned_addr learned my addr 172.21.15.81:0/174081795 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:12.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4c0c882640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c08158e70 0x7f4c0815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:12.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4bfffff640 1 -- 172.21.15.81:0/174081795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c08158e70 msgr2=0x7f4c0815c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:12.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4c07fff640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bf4005180 0x7f4c081f3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:12.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4bfffff640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c08158e70 0x7f4c0815c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:12.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4bfffff640 1 -- 172.21.15.81:0/174081795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bf4005180 msgr2=0x7f4c081f3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:12.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4bfffff640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bf4005180 0x7f4c081f3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:12.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4bfffff640 1 -- 172.21.15.81:0/174081795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c08159660 con 0x7f4c081f27c0 2023-11-08T16:39:12.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4bfffff640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c081f27c0 0x7f4c08158930 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000d4c0 tx=0x7f4bf000d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:12.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4c057fa640 1 -- 172.21.15.81:0/174081795 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0005480 con 0x7f4c081f27c0 2023-11-08T16:39:12.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4c057fa640 1 -- 172.21.15.81:0/174081795 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0003610 con 0x7f4c081f27c0 2023-11-08T16:39:12.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.479+0000 7f4c057fa640 1 -- 172.21.15.81:0/174081795 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf00039e0 con 0x7f4c081f27c0 2023-11-08T16:39:12.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.480+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/174081795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c08158050 con 0x7f4c081f27c0 2023-11-08T16:39:12.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.480+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/174081795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c081f2e20 con 0x7f4c081f27c0 2023-11-08T16:39:12.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.481+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/174081795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c08071160 con 0x7f4c081f27c0 2023-11-08T16:39:12.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.482+0000 7f4c057fa640 1 -- 172.21.15.81:0/174081795 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4bf0003b40 con 0x7f4c081f27c0 2023-11-08T16:39:12.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.483+0000 7f4c057fa640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4be806ae70 0x7f4be806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:12.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.483+0000 7f4c057fa640 1 -- 172.21.15.81:0/174081795 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bf00895e0 con 0x7f4c081f27c0 2023-11-08T16:39:12.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.483+0000 7f4c07fff640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4be806ae70 0x7f4be806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:12.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.484+0000 7f4c07fff640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4be806ae70 0x7f4be806d330 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8001370 tx=0x7f4bf8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:12.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.488+0000 7f4c057fa640 1 -- 172.21.15.81:0/174081795 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4c08071160 con 0x7f4c081f27c0 2023-11-08T16:39:12.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:12.715+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/174081795 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4c08071160 con 0x7f4c081f27c0 2023-11-08T16:39:13.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.232+0000 7f4c057fa640 1 -- 172.21.15.81:0/174081795 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v86) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4c08071160 con 0x7f4c081f27c0 2023-11-08T16:39:13.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.237+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/174081795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4be806ae70 msgr2=0x7f4be806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.237+0000 7f4c0e2e5640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4be806ae70 0x7f4be806d330 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8001370 tx=0x7f4bf8009040 comp rx=0 tx=0).stop 2023-11-08T16:39:13.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.237+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/174081795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c081f27c0 msgr2=0x7f4c08158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.237+0000 7f4c0e2e5640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c081f27c0 0x7f4c08158930 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4bf000d4c0 tx=0x7f4bf000d980 comp rx=0 tx=0).stop 2023-11-08T16:39:13.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.237+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/174081795 shutdown_connections 2023-11-08T16:39:13.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.237+0000 7f4c0e2e5640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4be806ae70 0x7f4be806d330 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.237+0000 7f4c0e2e5640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f4c08158e70 0x7f4c0815c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.237+0000 7f4c0e2e5640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4c081f27c0 0x7f4c08158930 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.237+0000 7f4c0e2e5640 1 --2- 172.21.15.81:0/174081795 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bf4005180 0x7f4c081f3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.237+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/174081795 >> 172.21.15.81:0/174081795 conn(0x7f4c080cb580 msgr2=0x7f4c08162430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:13.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.238+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/174081795 shutdown_connections 2023-11-08T16:39:13.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.238+0000 7f4c0e2e5640 1 -- 172.21.15.81:0/174081795 wait complete. 2023-11-08T16:39:13.263 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T16:39:13.264 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:13.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.478+0000 7feffa8c9640 1 Processor -- start 2023-11-08T16:39:13.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.478+0000 7feffa8c9640 1 -- start start 2023-11-08T16:39:13.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.480+0000 7feffa8c9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7feff415d2b0 0x7feff4161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.480+0000 7feffa8c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feff40ccf00 0x7feff406ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.480+0000 7feffa8c9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff4161c50 con 0x7feff40c9060 2023-11-08T16:39:13.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.480+0000 7feffa8c9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff415d690 con 0x7feff415d2b0 2023-11-08T16:39:13.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.480+0000 7feffa8c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff406b980 con 0x7feff40ccf00 2023-11-08T16:39:13.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.480+0000 7feff3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7feff415d2b0 0x7feff4161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:13.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.480+0000 7feff37fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7feff40c9060 0x7feff40cc7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34200/0 (socket says 172.21.15.81:34200) 2023-11-08T16:39:13.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.480+0000 7feff3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7feff415d2b0 0x7feff4161710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42594/0 (socket says 172.21.15.81:42594) 2023-11-08T16:39:13.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.480+0000 7feff37fe640 1 -- 172.21.15.81:0/3225206286 learned_addr learned my addr 172.21.15.81:0/3225206286 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:13.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.481+0000 7feff8e66640 1 --2- 172.21.15.81:0/3225206286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feff40ccf00 0x7feff406ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:13.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.481+0000 7feff3fff640 1 -- 172.21.15.81:0/3225206286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feff40ccf00 msgr2=0x7feff406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:13.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.481+0000 7feff3fff640 1 --2- 172.21.15.81:0/3225206286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feff40ccf00 0x7feff406ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.481+0000 7feff3fff640 1 -- 172.21.15.81:0/3225206286 >> v1:172.21.15.81:6789/0 conn(0x7feff40c9060 legacy=0x7feff40cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.481+0000 7feff2ffd640 1 -- 172.21.15.81:0/3225206286 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750166815 0 0) 0x7fefe8001110 con 0x7feff40c9060 2023-11-08T16:39:13.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.481+0000 7feff3fff640 1 -- 172.21.15.81:0/3225206286 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff406bc60 con 0x7feff415d2b0 2023-11-08T16:39:13.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.482+0000 7feff3fff640 1 --2- 172.21.15.81:0/3225206286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feff415d2b0 0x7feff4161710 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fefe40097a0 tx=0x7fefe402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cccf59074e8aeef server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:13.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.482+0000 7feff2ffd640 1 -- 172.21.15.81:0/3225206286 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4031830 con 0x7feff415d2b0 2023-11-08T16:39:13.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.482+0000 7feff2ffd640 1 -- 172.21.15.81:0/3225206286 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe4031990 con 0x7feff415d2b0 2023-11-08T16:39:13.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.482+0000 7feff2ffd640 1 -- 172.21.15.81:0/3225206286 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe4031d20 con 0x7feff415d2b0 2023-11-08T16:39:13.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.483+0000 7feffa8c9640 1 -- 172.21.15.81:0/3225206286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feff415d2b0 msgr2=0x7feff4161710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.483+0000 7feffa8c9640 1 --2- 172.21.15.81:0/3225206286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feff415d2b0 0x7feff4161710 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fefe40097a0 tx=0x7fefe402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.483+0000 7feffa8c9640 1 -- 172.21.15.81:0/3225206286 shutdown_connections 2023-11-08T16:39:13.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.483+0000 7feffa8c9640 1 --2- 172.21.15.81:0/3225206286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feff40ccf00 0x7feff406ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.483+0000 7feffa8c9640 1 --2- 172.21.15.81:0/3225206286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feff415d2b0 0x7feff4161710 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.483+0000 7feffa8c9640 1 -- 172.21.15.81:0/3225206286 >> 172.21.15.81:0/3225206286 conn(0x7feff40cb490 msgr2=0x7feff40cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:13.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.483+0000 7feffa8c9640 1 -- 172.21.15.81:0/3225206286 shutdown_connections 2023-11-08T16:39:13.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.483+0000 7feffa8c9640 1 -- 172.21.15.81:0/3225206286 wait complete. 2023-11-08T16:39:13.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.484+0000 7feffa8c9640 1 Processor -- start 2023-11-08T16:39:13.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.485+0000 7feffa8c9640 1 -- start start 2023-11-08T16:39:13.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.485+0000 7feffa8c9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feff40c9060 0x7feff40b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feffa8c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feff40ccf00 0x7feff415fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feffa8c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feff415d2b0 0x7feff4146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feffa8c9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff415f7f0 con 0x7feff40ccf00 2023-11-08T16:39:13.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feffa8c9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff40b1fb0 con 0x7feff40c9060 2023-11-08T16:39:13.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feffa8c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff41601c0 con 0x7feff415d2b0 2023-11-08T16:39:13.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feff37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feff40ccf00 0x7feff415fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:13.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feff37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feff40ccf00 0x7feff415fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42596/0 (socket says 172.21.15.81:42596) 2023-11-08T16:39:13.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feff37fe640 1 -- 172.21.15.81:0/23725559 learned_addr learned my addr 172.21.15.81:0/23725559 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:13.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feff37fe640 1 -- 172.21.15.81:0/23725559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feff415d2b0 msgr2=0x7feff4146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feff37fe640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feff415d2b0 0x7feff4146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feff37fe640 1 -- 172.21.15.81:0/23725559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feff40c9060 msgr2=0x7feff40b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feff37fe640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feff40c9060 0x7feff40b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feff37fe640 1 -- 172.21.15.81:0/23725559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe4009400 con 0x7feff40ccf00 2023-11-08T16:39:13.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.486+0000 7feff37fe640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feff40ccf00 0x7feff415fc80 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fefe800c7b0 tx=0x7fefe800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:13.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.487+0000 7feff0ff9640 1 -- 172.21.15.81:0/23725559 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe80072d0 con 0x7feff40ccf00 2023-11-08T16:39:13.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.487+0000 7feff0ff9640 1 -- 172.21.15.81:0/23725559 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe8007450 con 0x7feff40ccf00 2023-11-08T16:39:13.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.487+0000 7feff0ff9640 1 -- 172.21.15.81:0/23725559 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe80047e0 con 0x7feff40ccf00 2023-11-08T16:39:13.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.487+0000 7feffa8c9640 1 -- 172.21.15.81:0/23725559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff415ebe0 con 0x7feff40ccf00 2023-11-08T16:39:13.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.487+0000 7feffa8c9640 1 -- 172.21.15.81:0/23725559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff415ef70 con 0x7feff40ccf00 2023-11-08T16:39:13.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.488+0000 7feffa8c9640 1 -- 172.21.15.81:0/23725559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff404ece0 con 0x7feff40ccf00 2023-11-08T16:39:13.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.489+0000 7feff0ff9640 1 -- 172.21.15.81:0/23725559 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fefe8004960 con 0x7feff40ccf00 2023-11-08T16:39:13.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.490+0000 7feff0ff9640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fefe006aea0 0x7fefe006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.490+0000 7feff0ff9640 1 -- 172.21.15.81:0/23725559 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefe80885f0 con 0x7feff40ccf00 2023-11-08T16:39:13.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.490+0000 7feff3fff640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fefe006aea0 0x7fefe006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:13.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.491+0000 7feff3fff640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fefe006aea0 0x7fefe006d360 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fefe4036000 tx=0x7fefe4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:13.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.494+0000 7feff0ff9640 1 -- 172.21.15.81:0/23725559 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fefe80536a0 con 0x7feff40ccf00 2023-11-08T16:39:13.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.721+0000 7feffa8c9640 1 -- 172.21.15.81:0/23725559 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feff4161c50 con 0x7feff40ccf00 2023-11-08T16:39:13.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.722+0000 7feff0ff9640 1 -- 172.21.15.81:0/23725559 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4912 (secure 0 0 0) 0x7feff4161c50 con 0x7feff40ccf00 2023-11-08T16:39:13.724 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:13.724 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":86,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1157756119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1157756119},{"type":"v1","addr":"172.21.15.93:6833","nonce":1157756119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1607105255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1607105255},{"type":"v1","addr":"172.21.15.81:6835","nonce":1607105255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3988749175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3988749175},{"type":"v1","addr":"172.21.15.81:6837","nonce":3988749175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:09.619973+0000","modified":"2023-11-08T16:39:12.717402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7713},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7713":{"gid":7713,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6835/691341796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":691341796},{"type":"v1","addr":"172.21.15.93:6835","nonce":691341796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:39:13.725 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 86 2023-11-08T16:39:13.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.724+0000 7feffa8c9640 1 -- 172.21.15.81:0/23725559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fefe006aea0 msgr2=0x7fefe006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.724+0000 7feffa8c9640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fefe006aea0 0x7fefe006d360 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fefe4036000 tx=0x7fefe4030000 comp rx=0 tx=0).stop 2023-11-08T16:39:13.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.724+0000 7feffa8c9640 1 -- 172.21.15.81:0/23725559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feff40ccf00 msgr2=0x7feff415fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.724+0000 7feffa8c9640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feff40ccf00 0x7feff415fc80 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fefe800c7b0 tx=0x7fefe800cc70 comp rx=0 tx=0).stop 2023-11-08T16:39:13.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.724+0000 7feffa8c9640 1 -- 172.21.15.81:0/23725559 shutdown_connections 2023-11-08T16:39:13.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.724+0000 7feffa8c9640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fefe006aea0 0x7fefe006d360 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.724+0000 7feffa8c9640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7feff415d2b0 0x7feff4146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.724+0000 7feffa8c9640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7feff40ccf00 0x7feff415fc80 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.724+0000 7feffa8c9640 1 --2- 172.21.15.81:0/23725559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7feff40c9060 0x7feff40b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.724+0000 7feffa8c9640 1 -- 172.21.15.81:0/23725559 >> 172.21.15.81:0/23725559 conn(0x7feff40cb490 msgr2=0x7feff40ae4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:13.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.725+0000 7feffa8c9640 1 -- 172.21.15.81:0/23725559 shutdown_connections 2023-11-08T16:39:13.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.725+0000 7feffa8c9640 1 -- 172.21.15.81:0/23725559 wait complete. 2023-11-08T16:39:13.745 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:39:13.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.955+0000 7fdb4042d640 1 Processor -- start 2023-11-08T16:39:13.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.955+0000 7fdb4042d640 1 -- start start 2023-11-08T16:39:13.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.957+0000 7fdb4042d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fdb3815adc0 0x7fdb380b2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.957+0000 7fdb4042d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdb380709c0 0x7fdb380cf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.957+0000 7fdb4042d640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb380ce970 con 0x7fdb380cf620 2023-11-08T16:39:13.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.957+0000 7fdb4042d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb380cecc0 con 0x7fdb380709c0 2023-11-08T16:39:13.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.957+0000 7fdb4042d640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb38070e70 con 0x7fdb3815adc0 2023-11-08T16:39:13.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.957+0000 7fdb3d9c8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdb380709c0 0x7fdb380cf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:13.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.957+0000 7fdb3d9c8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdb380709c0 0x7fdb380cf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42606/0 (socket says 172.21.15.81:42606) 2023-11-08T16:39:13.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.957+0000 7fdb3d9c8640 1 -- 172.21.15.81:0/3472174306 learned_addr learned my addr 172.21.15.81:0/3472174306 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:13.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.957+0000 7fdb3e1c9640 1 --2- 172.21.15.81:0/3472174306 >> v2:172.21.15.93:3300/0 conn(0x7fdb3815adc0 0x7fdb380b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:13.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.957+0000 7fdb3d9c8640 1 -- 172.21.15.81:0/3472174306 >> v2:172.21.15.93:3300/0 conn(0x7fdb3815adc0 msgr2=0x7fdb380b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:13.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.958+0000 7fdb3d9c8640 1 --2- 172.21.15.81:0/3472174306 >> v2:172.21.15.93:3300/0 conn(0x7fdb3815adc0 0x7fdb380b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.958+0000 7fdb3d9c8640 1 -- 172.21.15.81:0/3472174306 >> v1:172.21.15.81:6789/0 conn(0x7fdb380cf620 legacy=0x7fdb3806a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.958+0000 7fdb3d9c8640 1 -- 172.21.15.81:0/3472174306 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb38071110 con 0x7fdb380709c0 2023-11-08T16:39:13.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.958+0000 7fdb3e1c9640 1 --2- 172.21.15.81:0/3472174306 >> v2:172.21.15.93:3300/0 conn(0x7fdb3815adc0 0x7fdb380b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:13.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.959+0000 7fdb3d9c8640 1 --2- 172.21.15.81:0/3472174306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb380709c0 0x7fdb380cf0e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fdb380686e0 tx=0x7fdb2802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65ed9fc68f7473ad server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:13.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.959+0000 7fdb3d1c7640 1 -- 172.21.15.81:0/3472174306 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb280317e0 con 0x7fdb380709c0 2023-11-08T16:39:13.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.959+0000 7fdb3d1c7640 1 -- 172.21.15.81:0/3472174306 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb28031940 con 0x7fdb380709c0 2023-11-08T16:39:13.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.959+0000 7fdb3d1c7640 1 -- 172.21.15.81:0/3472174306 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28031cd0 con 0x7fdb380709c0 2023-11-08T16:39:13.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.959+0000 7fdb4042d640 1 -- 172.21.15.81:0/3472174306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb380709c0 msgr2=0x7fdb380cf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.959+0000 7fdb4042d640 1 --2- 172.21.15.81:0/3472174306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb380709c0 0x7fdb380cf0e0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fdb380686e0 tx=0x7fdb2802b880 comp rx=0 tx=0).stop 2023-11-08T16:39:13.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.960+0000 7fdb4042d640 1 -- 172.21.15.81:0/3472174306 shutdown_connections 2023-11-08T16:39:13.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.960+0000 7fdb4042d640 1 --2- 172.21.15.81:0/3472174306 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb380709c0 0x7fdb380cf0e0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.960+0000 7fdb4042d640 1 --2- 172.21.15.81:0/3472174306 >> v2:172.21.15.93:3300/0 conn(0x7fdb3815adc0 0x7fdb380b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.960+0000 7fdb4042d640 1 -- 172.21.15.81:0/3472174306 >> 172.21.15.81:0/3472174306 conn(0x7fdb380b1600 msgr2=0x7fdb380b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:13.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.960+0000 7fdb4042d640 1 -- 172.21.15.81:0/3472174306 shutdown_connections 2023-11-08T16:39:13.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.960+0000 7fdb4042d640 1 -- 172.21.15.81:0/3472174306 wait complete. 2023-11-08T16:39:13.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.961+0000 7fdb4042d640 1 Processor -- start 2023-11-08T16:39:13.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.961+0000 7fdb4042d640 1 -- start start 2023-11-08T16:39:13.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.962+0000 7fdb4042d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb380709c0 0x7fdb381f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.962+0000 7fdb4042d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb380cf620 0x7fdb38150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.962+0000 7fdb4042d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb3815adc0 0x7fdb381e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.962+0000 7fdb4042d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c5ac0 con 0x7fdb380709c0 2023-11-08T16:39:13.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.962+0000 7fdb4042d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3806e3e0 con 0x7fdb3815adc0 2023-11-08T16:39:13.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.962+0000 7fdb4042d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c3ff0 con 0x7fdb380cf620 2023-11-08T16:39:13.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3e1c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb380709c0 0x7fdb381f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:13.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3e1c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb380709c0 0x7fdb381f24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42622/0 (socket says 172.21.15.81:42622) 2023-11-08T16:39:13.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3e1c9640 1 -- 172.21.15.81:0/1961100616 learned_addr learned my addr 172.21.15.81:0/1961100616 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:13.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3d9c8640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb380cf620 0x7fdb38150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:13.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3e9ca640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb3815adc0 0x7fdb381e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:13.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3e1c9640 1 -- 172.21.15.81:0/1961100616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb380cf620 msgr2=0x7fdb38150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3e1c9640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb380cf620 0x7fdb38150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3e1c9640 1 -- 172.21.15.81:0/1961100616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb3815adc0 msgr2=0x7fdb381e46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:13.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3e1c9640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb3815adc0 0x7fdb381e46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:13.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3e1c9640 1 -- 172.21.15.81:0/1961100616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280094d0 con 0x7fdb380709c0 2023-11-08T16:39:13.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3d9c8640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb380cf620 0x7fdb38150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:13.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.963+0000 7fdb3e1c9640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb380709c0 0x7fdb381f24c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400c7f0 tx=0x7fdb3400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:13.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.964+0000 7fdb2effd640 1 -- 172.21.15.81:0/1961100616 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34007820 con 0x7fdb380709c0 2023-11-08T16:39:13.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.964+0000 7fdb2effd640 1 -- 172.21.15.81:0/1961100616 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34007980 con 0x7fdb380709c0 2023-11-08T16:39:13.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.964+0000 7fdb4042d640 1 -- 172.21.15.81:0/1961100616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb380c57e0 con 0x7fdb380709c0 2023-11-08T16:39:13.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.964+0000 7fdb2effd640 1 -- 172.21.15.81:0/1961100616 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34007cf0 con 0x7fdb380709c0 2023-11-08T16:39:13.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.964+0000 7fdb4042d640 1 -- 172.21.15.81:0/1961100616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb380c44d0 con 0x7fdb380709c0 2023-11-08T16:39:13.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.965+0000 7fdb4042d640 1 -- 172.21.15.81:0/1961100616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3804ece0 con 0x7fdb380709c0 2023-11-08T16:39:13.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.966+0000 7fdb2effd640 1 -- 172.21.15.81:0/1961100616 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdb340113f0 con 0x7fdb380709c0 2023-11-08T16:39:13.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.967+0000 7fdb2effd640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb1c06ae70 0x7fdb1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:13.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.967+0000 7fdb2effd640 1 -- 172.21.15.81:0/1961100616 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb34089930 con 0x7fdb380709c0 2023-11-08T16:39:13.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.967+0000 7fdb3d9c8640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb1c06ae70 0x7fdb1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.968+0000 7fdb3d9c8640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb1c06ae70 0x7fdb1c06d330 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdb28036000 tx=0x7fdb2803d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:13.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:13.973+0000 7fdb2effd640 1 -- 172.21.15.81:0/1961100616 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdb340549d0 con 0x7fdb380709c0 2023-11-08T16:39:14.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.164+0000 7fdb4042d640 1 -- 172.21.15.81:0/1961100616 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb3804e490 con 0x7fdb380709c0 2023-11-08T16:39:14.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.167+0000 7fdb2effd640 1 -- 172.21.15.81:0/1961100616 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16463 (secure 0 0 0) 0x7fdb3401f090 con 0x7fdb380709c0 2023-11-08T16:39:14.169 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:14.170 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":103,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:39:11.209195+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:39:07.492834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T16:39:08.634345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:39:14.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.170+0000 7fdb4042d640 1 -- 172.21.15.81:0/1961100616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb1c06ae70 msgr2=0x7fdb1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.170+0000 7fdb4042d640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb1c06ae70 0x7fdb1c06d330 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdb28036000 tx=0x7fdb2803d040 comp rx=0 tx=0).stop 2023-11-08T16:39:14.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.170+0000 7fdb4042d640 1 -- 172.21.15.81:0/1961100616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb380709c0 msgr2=0x7fdb381f24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.170+0000 7fdb4042d640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb380709c0 0x7fdb381f24c0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400c7f0 tx=0x7fdb3400ccb0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.170+0000 7fdb4042d640 1 -- 172.21.15.81:0/1961100616 shutdown_connections 2023-11-08T16:39:14.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.170+0000 7fdb4042d640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdb1c06ae70 0x7fdb1c06d330 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.170+0000 7fdb4042d640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb3815adc0 0x7fdb381e46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.170+0000 7fdb4042d640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdb380cf620 0x7fdb38150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.170+0000 7fdb4042d640 1 --2- 172.21.15.81:0/1961100616 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdb380709c0 0x7fdb381f24c0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.170+0000 7fdb4042d640 1 -- 172.21.15.81:0/1961100616 >> 172.21.15.81:0/1961100616 conn(0x7fdb380b1600 msgr2=0x7fdb380cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:14.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.171+0000 7fdb4042d640 1 -- 172.21.15.81:0/1961100616 shutdown_connections 2023-11-08T16:39:14.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.171+0000 7fdb4042d640 1 -- 172.21.15.81:0/1961100616 wait complete. 2023-11-08T16:39:14.193 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T16:39:14.193 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:39:14.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.406+0000 7fbe1c99a640 1 Processor -- start 2023-11-08T16:39:14.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.406+0000 7fbe1c99a640 1 -- start start 2023-11-08T16:39:14.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe1c99a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1806b850 0x7fbe1806d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe1c99a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbe181629e0 0x7fbe18162e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe1c99a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe18162050 con 0x7fbe18161820 2023-11-08T16:39:14.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe1c99a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe18164e10 con 0x7fbe181629e0 2023-11-08T16:39:14.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe1c99a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe18162330 con 0x7fbe1806b850 2023-11-08T16:39:14.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe17fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbe181629e0 0x7fbe18162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe0effd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbe18161820 0x7fbe18165540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34208/0 (socket says 172.21.15.81:34208) 2023-11-08T16:39:14.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe17fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbe181629e0 0x7fbe18162e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42634/0 (socket says 172.21.15.81:42634) 2023-11-08T16:39:14.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe0effd640 1 -- 172.21.15.81:0/3694192856 learned_addr learned my addr 172.21.15.81:0/3694192856 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:14.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe177fe640 1 --2- 172.21.15.81:0/3694192856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1806b850 0x7fbe1806d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe17fff640 1 -- 172.21.15.81:0/3694192856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1806b850 msgr2=0x7fbe1806d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:14.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe17fff640 1 --2- 172.21.15.81:0/3694192856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1806b850 0x7fbe1806d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.408+0000 7fbe17fff640 1 -- 172.21.15.81:0/3694192856 >> v1:172.21.15.81:6789/0 conn(0x7fbe18161820 legacy=0x7fbe18165540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.409+0000 7fbe17fff640 1 -- 172.21.15.81:0/3694192856 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe18162610 con 0x7fbe181629e0 2023-11-08T16:39:14.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.409+0000 7fbe177fe640 1 --2- 172.21.15.81:0/3694192856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1806b850 0x7fbe1806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:14.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.409+0000 7fbe16ffd640 1 -- 172.21.15.81:0/3694192856 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319740186 0 0) 0x7fbe18162050 con 0x7fbe18161820 2023-11-08T16:39:14.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.410+0000 7fbe17fff640 1 --2- 172.21.15.81:0/3694192856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe181629e0 0x7fbe18162e70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009d90 tx=0x7fbe1002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d55189fa637473 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:14.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.410+0000 7fbe16ffd640 1 -- 172.21.15.81:0/3694192856 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe1002cc50 con 0x7fbe181629e0 2023-11-08T16:39:14.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.410+0000 7fbe16ffd640 1 -- 172.21.15.81:0/3694192856 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1002cdb0 con 0x7fbe181629e0 2023-11-08T16:39:14.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.410+0000 7fbe16ffd640 1 -- 172.21.15.81:0/3694192856 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10034660 con 0x7fbe181629e0 2023-11-08T16:39:14.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.410+0000 7fbe1c99a640 1 -- 172.21.15.81:0/3694192856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe181629e0 msgr2=0x7fbe18162e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.410+0000 7fbe1c99a640 1 --2- 172.21.15.81:0/3694192856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe181629e0 0x7fbe18162e70 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fbe10009d90 tx=0x7fbe1002bea0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.411+0000 7fbe1c99a640 1 -- 172.21.15.81:0/3694192856 shutdown_connections 2023-11-08T16:39:14.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.411+0000 7fbe1c99a640 1 --2- 172.21.15.81:0/3694192856 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe181629e0 0x7fbe18162e70 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.411+0000 7fbe1c99a640 1 --2- 172.21.15.81:0/3694192856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe1806b850 0x7fbe1806d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.411+0000 7fbe1c99a640 1 -- 172.21.15.81:0/3694192856 >> 172.21.15.81:0/3694192856 conn(0x7fbe1806a4a0 msgr2=0x7fbe1806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:14.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.411+0000 7fbe1c99a640 1 -- 172.21.15.81:0/3694192856 shutdown_connections 2023-11-08T16:39:14.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.411+0000 7fbe1c99a640 1 -- 172.21.15.81:0/3694192856 wait complete. 2023-11-08T16:39:14.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.412+0000 7fbe1c99a640 1 Processor -- start 2023-11-08T16:39:14.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.412+0000 7fbe1c99a640 1 -- start start 2023-11-08T16:39:14.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.413+0000 7fbe1c99a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1806b850 0x7fbe181f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.413+0000 7fbe1c99a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe18161820 0x7fbe181fa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.413+0000 7fbe1c99a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe181629e0 0x7fbe180b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.413+0000 7fbe1c99a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe18151270 con 0x7fbe1806b850 2023-11-08T16:39:14.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.413+0000 7fbe1c99a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180b3360 con 0x7fbe181629e0 2023-11-08T16:39:14.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.413+0000 7fbe1c99a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180b2da0 con 0x7fbe18161820 2023-11-08T16:39:14.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.414+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1806b850 0x7fbe181f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.414+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1806b850 0x7fbe181f9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42640/0 (socket says 172.21.15.81:42640) 2023-11-08T16:39:14.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.414+0000 7fbe177fe640 1 -- 172.21.15.81:0/2022186615 learned_addr learned my addr 172.21.15.81:0/2022186615 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:14.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.414+0000 7fbe17fff640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe181629e0 0x7fbe180b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.414+0000 7fbe0effd640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe18161820 0x7fbe181fa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.414+0000 7fbe177fe640 1 -- 172.21.15.81:0/2022186615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe18161820 msgr2=0x7fbe181fa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.414+0000 7fbe177fe640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe18161820 0x7fbe181fa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.414+0000 7fbe177fe640 1 -- 172.21.15.81:0/2022186615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe181629e0 msgr2=0x7fbe180b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.414+0000 7fbe177fe640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe181629e0 0x7fbe180b2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.414+0000 7fbe177fe640 1 -- 172.21.15.81:0/2022186615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe10009a70 con 0x7fbe1806b850 2023-11-08T16:39:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.414+0000 7fbe17fff640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe181629e0 0x7fbe180b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.415+0000 7fbe177fe640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1806b850 0x7fbe181f9e40 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c820 tx=0x7fbe0800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.415+0000 7fbe14ff9640 1 -- 172.21.15.81:0/2022186615 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe08007a30 con 0x7fbe1806b850 2023-11-08T16:39:14.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.415+0000 7fbe14ff9640 1 -- 172.21.15.81:0/2022186615 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe08007b90 con 0x7fbe1806b850 2023-11-08T16:39:14.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.415+0000 7fbe1c99a640 1 -- 172.21.15.81:0/2022186615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe180b3080 con 0x7fbe1806b850 2023-11-08T16:39:14.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.415+0000 7fbe14ff9640 1 -- 172.21.15.81:0/2022186615 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0800ce70 con 0x7fbe1806b850 2023-11-08T16:39:14.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.415+0000 7fbe1c99a640 1 -- 172.21.15.81:0/2022186615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe181f6660 con 0x7fbe1806b850 2023-11-08T16:39:14.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.416+0000 7fbe1c99a640 1 -- 172.21.15.81:0/2022186615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe18150cf0 con 0x7fbe1806b850 2023-11-08T16:39:14.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.417+0000 7fbe14ff9640 1 -- 172.21.15.81:0/2022186615 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbe080113f0 con 0x7fbe1806b850 2023-11-08T16:39:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.418+0000 7fbe14ff9640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbdf806ae70 0x7fbdf806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.418+0000 7fbe14ff9640 1 -- 172.21.15.81:0/2022186615 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe08089800 con 0x7fbe1806b850 2023-11-08T16:39:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.418+0000 7fbe0effd640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbdf806ae70 0x7fbdf806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.419+0000 7fbe0effd640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbdf806ae70 0x7fbdf806d330 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbe00007fd0 tx=0x7fbe00009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:14.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.422+0000 7fbe14ff9640 1 -- 172.21.15.81:0/2022186615 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbe18150cf0 con 0x7fbe1806b850 2023-11-08T16:39:14.551 INFO:tasks.ceph.mds.b.smithi093.stderr:2023-11-08T16:39:14.548+0000 7f9a3f5c1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:39:14.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.606+0000 7fbe1c99a640 1 -- 172.21.15.81:0/2022186615 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe18165c50 con 0x7fbdf806ae70 2023-11-08T16:39:14.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.620+0000 7fbe14ff9640 1 -- 172.21.15.81:0/2022186615 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287546 (secure 0 0 0) 0x7fbe00001c70 con 0x7fbdf806ae70 2023-11-08T16:39:14.624 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:14.627 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:39:14.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 -- 172.21.15.81:0/2022186615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbdf806ae70 msgr2=0x7fbdf806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbdf806ae70 0x7fbdf806d330 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbe00007fd0 tx=0x7fbe00009040 comp rx=0 tx=0).stop 2023-11-08T16:39:14.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 -- 172.21.15.81:0/2022186615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1806b850 msgr2=0x7fbe181f9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1806b850 0x7fbe181f9e40 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbe0800c820 tx=0x7fbe0800cc20 comp rx=0 tx=0).stop 2023-11-08T16:39:14.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 -- 172.21.15.81:0/2022186615 shutdown_connections 2023-11-08T16:39:14.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbdf806ae70 0x7fbdf806d330 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbe181629e0 0x7fbe180b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbe18161820 0x7fbe181fa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 --2- 172.21.15.81:0/2022186615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbe1806b850 0x7fbe181f9e40 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 -- 172.21.15.81:0/2022186615 >> 172.21.15.81:0/2022186615 conn(0x7fbe1806a4a0 msgr2=0x7fbe18163e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:14.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 -- 172.21.15.81:0/2022186615 shutdown_connections 2023-11-08T16:39:14.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.627+0000 7fbe1c99a640 1 -- 172.21.15.81:0/2022186615 wait complete. 2023-11-08T16:39:14.646 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2023-11-08T16:39:12.763467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260564,"kb_used_data":11084,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714156,"statfs":{"total":773094113280,"available":772827295744,"internally_reserved":0,"allocated":11350016,"data_stored":5812088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.620834"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.245630+0000","last_change":"2023-11-08T16:39:10.241105+0000","last_active":"2023-11-08T16:39:10.245630+0000","last_peered":"2023-11-08T16:39:10.245630+0000","last_clean":"2023-11-08T16:39:10.245630+0000","last_became_active":"2023-11-08T16:39:10.240677+0000","last_became_peered":"2023-11-08T16:39:10.240677+0000","last_unstale":"2023-11-08T16:39:10.245630+0000","last_undegraded":"2023-11-08T16:39:10.245630+0000","last_fullsized":"2023-11-08T16:39:10.245630+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:38.165884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225035+0000","last_change":"2023-11-08T16:39:10.225715+0000","last_active":"2023-11-08T16:39:11.225035+0000","last_peered":"2023-11-08T16:39:11.225035+0000","last_clean":"2023-11-08T16:39:11.225035+0000","last_became_active":"2023-11-08T16:39:10.225346+0000","last_became_peered":"2023-11-08T16:39:10.225346+0000","last_unstale":"2023-11-08T16:39:11.225035+0000","last_undegraded":"2023-11-08T16:39:11.225035+0000","last_fullsized":"2023-11-08T16:39:11.225035+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:57:03.464872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.232685+0000","last_change":"2023-11-08T16:39:10.226516+0000","last_active":"2023-11-08T16:39:10.232685+0000","last_peered":"2023-11-08T16:39:10.232685+0000","last_clean":"2023-11-08T16:39:10.232685+0000","last_became_active":"2023-11-08T16:39:10.225660+0000","last_became_peered":"2023-11-08T16:39:10.225660+0000","last_unstale":"2023-11-08T16:39:10.232685+0000","last_undegraded":"2023-11-08T16:39:10.232685+0000","last_fullsized":"2023-11-08T16:39:10.232685+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:32.776251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.243310+0000","last_change":"2023-11-08T16:39:10.230188+0000","last_active":"2023-11-08T16:39:10.243310+0000","last_peered":"2023-11-08T16:39:10.243310+0000","last_clean":"2023-11-08T16:39:10.243310+0000","last_became_active":"2023-11-08T16:39:10.229930+0000","last_became_peered":"2023-11-08T16:39:10.229930+0000","last_unstale":"2023-11-08T16:39:10.243310+0000","last_undegraded":"2023-11-08T16:39:10.243310+0000","last_fullsized":"2023-11-08T16:39:10.243310+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:25.931399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.226744+0000","last_change":"2023-11-08T16:39:10.222049+0000","last_active":"2023-11-08T16:39:11.226744+0000","last_peered":"2023-11-08T16:39:11.226744+0000","last_clean":"2023-11-08T16:39:11.226744+0000","last_became_active":"2023-11-08T16:39:10.221396+0000","last_became_peered":"2023-11-08T16:39:10.221396+0000","last_unstale":"2023-11-08T16:39:11.226744+0000","last_undegraded":"2023-11-08T16:39:11.226744+0000","last_fullsized":"2023-11-08T16:39:11.226744+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:19:02.824312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.239537+0000","last_change":"2023-11-08T16:39:10.234031+0000","last_active":"2023-11-08T16:39:10.239537+0000","last_peered":"2023-11-08T16:39:10.239537+0000","last_clean":"2023-11-08T16:39:10.239537+0000","last_became_active":"2023-11-08T16:39:10.232189+0000","last_became_peered":"2023-11-08T16:39:10.232189+0000","last_unstale":"2023-11-08T16:39:10.239537+0000","last_undegraded":"2023-11-08T16:39:10.239537+0000","last_fullsized":"2023-11-08T16:39:10.239537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:07:01.023231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228093+0000","last_change":"2023-11-08T16:39:10.227019+0000","last_active":"2023-11-08T16:39:11.228093+0000","last_peered":"2023-11-08T16:39:11.228093+0000","last_clean":"2023-11-08T16:39:11.228093+0000","last_became_active":"2023-11-08T16:39:10.226700+0000","last_became_peered":"2023-11-08T16:39:10.226700+0000","last_unstale":"2023-11-08T16:39:11.228093+0000","last_undegraded":"2023-11-08T16:39:11.228093+0000","last_fullsized":"2023-11-08T16:39:11.228093+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:32.776251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.242242+0000","last_change":"2023-11-08T16:39:10.236830+0000","last_active":"2023-11-08T16:39:10.242242+0000","last_peered":"2023-11-08T16:39:10.242242+0000","last_clean":"2023-11-08T16:39:10.242242+0000","last_became_active":"2023-11-08T16:39:10.236497+0000","last_became_peered":"2023-11-08T16:39:10.236497+0000","last_unstale":"2023-11-08T16:39:10.242242+0000","last_undegraded":"2023-11-08T16:39:10.242242+0000","last_fullsized":"2023-11-08T16:39:10.242242+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:41.564607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225939+0000","last_change":"2023-11-08T16:39:10.222150+0000","last_active":"2023-11-08T16:39:11.225939+0000","last_peered":"2023-11-08T16:39:11.225939+0000","last_clean":"2023-11-08T16:39:11.225939+0000","last_became_active":"2023-11-08T16:39:10.221845+0000","last_became_peered":"2023-11-08T16:39:10.221845+0000","last_unstale":"2023-11-08T16:39:11.225939+0000","last_undegraded":"2023-11-08T16:39:11.225939+0000","last_fullsized":"2023-11-08T16:39:11.225939+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:58:52.139332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.232980+0000","last_change":"2023-11-08T16:39:10.228413+0000","last_active":"2023-11-08T16:39:10.232980+0000","last_peered":"2023-11-08T16:39:10.232980+0000","last_clean":"2023-11-08T16:39:10.232980+0000","last_became_active":"2023-11-08T16:39:10.228115+0000","last_became_peered":"2023-11-08T16:39:10.228115+0000","last_unstale":"2023-11-08T16:39:10.232980+0000","last_undegraded":"2023-11-08T16:39:10.232980+0000","last_fullsized":"2023-11-08T16:39:10.232980+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:43.946508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224756+0000","last_change":"2023-11-08T16:39:10.223495+0000","last_active":"2023-11-08T16:39:11.224756+0000","last_peered":"2023-11-08T16:39:11.224756+0000","last_clean":"2023-11-08T16:39:11.224756+0000","last_became_active":"2023-11-08T16:39:10.223208+0000","last_became_peered":"2023-11-08T16:39:10.223208+0000","last_unstale":"2023-11-08T16:39:11.224756+0000","last_undegraded":"2023-11-08T16:39:11.224756+0000","last_fullsized":"2023-11-08T16:39:11.224756+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:22.834895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.233991+0000","last_change":"2023-11-08T16:39:10.229501+0000","last_active":"2023-11-08T16:39:10.233991+0000","last_peered":"2023-11-08T16:39:10.233991+0000","last_clean":"2023-11-08T16:39:10.233991+0000","last_became_active":"2023-11-08T16:39:10.228488+0000","last_became_peered":"2023-11-08T16:39:10.228488+0000","last_unstale":"2023-11-08T16:39:10.233991+0000","last_undegraded":"2023-11-08T16:39:10.233991+0000","last_fullsized":"2023-11-08T16:39:10.233991+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:46:26.608103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.242204+0000","last_change":"2023-11-08T16:39:10.236629+0000","last_active":"2023-11-08T16:39:10.242204+0000","last_peered":"2023-11-08T16:39:10.242204+0000","last_clean":"2023-11-08T16:39:10.242204+0000","last_became_active":"2023-11-08T16:39:10.236317+0000","last_became_peered":"2023-11-08T16:39:10.236317+0000","last_unstale":"2023-11-08T16:39:10.242204+0000","last_undegraded":"2023-11-08T16:39:10.242204+0000","last_fullsized":"2023-11-08T16:39:10.242204+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:51.892821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.223631+0000","last_change":"2023-11-08T16:39:10.242802+0000","last_active":"2023-11-08T16:39:11.223631+0000","last_peered":"2023-11-08T16:39:11.223631+0000","last_clean":"2023-11-08T16:39:11.223631+0000","last_became_active":"2023-11-08T16:39:10.242496+0000","last_became_peered":"2023-11-08T16:39:10.242496+0000","last_unstale":"2023-11-08T16:39:11.223631+0000","last_undegraded":"2023-11-08T16:39:11.223631+0000","last_fullsized":"2023-11-08T16:39:11.223631+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:53.831527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.242331+0000","last_change":"2023-11-08T16:39:10.239182+0000","last_active":"2023-11-08T16:39:10.242331+0000","last_peered":"2023-11-08T16:39:10.242331+0000","last_clean":"2023-11-08T16:39:10.242331+0000","last_became_active":"2023-11-08T16:39:10.238590+0000","last_became_peered":"2023-11-08T16:39:10.238590+0000","last_unstale":"2023-11-08T16:39:10.242331+0000","last_undegraded":"2023-11-08T16:39:10.242331+0000","last_fullsized":"2023-11-08T16:39:10.242331+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:13.821481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.231348+0000","last_change":"2023-11-08T16:39:10.219542+0000","last_active":"2023-11-08T16:39:10.231348+0000","last_peered":"2023-11-08T16:39:10.231348+0000","last_clean":"2023-11-08T16:39:10.231348+0000","last_became_active":"2023-11-08T16:39:10.218943+0000","last_became_peered":"2023-11-08T16:39:10.218943+0000","last_unstale":"2023-11-08T16:39:10.231348+0000","last_undegraded":"2023-11-08T16:39:10.231348+0000","last_fullsized":"2023-11-08T16:39:10.231348+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:34:32.079590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.238181+0000","last_change":"2023-11-08T16:39:10.227281+0000","last_active":"2023-11-08T16:39:10.238181+0000","last_peered":"2023-11-08T16:39:10.238181+0000","last_clean":"2023-11-08T16:39:10.238181+0000","last_became_active":"2023-11-08T16:39:10.226996+0000","last_became_peered":"2023-11-08T16:39:10.226996+0000","last_unstale":"2023-11-08T16:39:10.238181+0000","last_undegraded":"2023-11-08T16:39:10.238181+0000","last_fullsized":"2023-11-08T16:39:10.238181+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:24:05.328966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.230611+0000","last_change":"2023-11-08T16:39:10.222675+0000","last_active":"2023-11-08T16:39:10.230611+0000","last_peered":"2023-11-08T16:39:10.230611+0000","last_clean":"2023-11-08T16:39:10.230611+0000","last_became_active":"2023-11-08T16:39:10.222141+0000","last_became_peered":"2023-11-08T16:39:10.222141+0000","last_unstale":"2023-11-08T16:39:10.230611+0000","last_undegraded":"2023-11-08T16:39:10.230611+0000","last_fullsized":"2023-11-08T16:39:10.230611+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:22.834895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225095+0000","last_change":"2023-11-08T16:39:10.242809+0000","last_active":"2023-11-08T16:39:11.225095+0000","last_peered":"2023-11-08T16:39:11.225095+0000","last_clean":"2023-11-08T16:39:11.225095+0000","last_became_active":"2023-11-08T16:39:10.241875+0000","last_became_peered":"2023-11-08T16:39:10.241875+0000","last_unstale":"2023-11-08T16:39:11.225095+0000","last_undegraded":"2023-11-08T16:39:11.225095+0000","last_fullsized":"2023-11-08T16:39:11.225095+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:38.165884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.243393+0000","last_change":"2023-11-08T16:39:10.235161+0000","last_active":"2023-11-08T16:39:10.243393+0000","last_peered":"2023-11-08T16:39:10.243393+0000","last_clean":"2023-11-08T16:39:10.243393+0000","last_became_active":"2023-11-08T16:39:10.234261+0000","last_became_peered":"2023-11-08T16:39:10.234261+0000","last_unstale":"2023-11-08T16:39:10.243393+0000","last_undegraded":"2023-11-08T16:39:10.243393+0000","last_fullsized":"2023-11-08T16:39:10.243393+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:52.527800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.226572+0000","last_change":"2023-11-08T16:39:10.222066+0000","last_active":"2023-11-08T16:39:11.226572+0000","last_peered":"2023-11-08T16:39:11.226572+0000","last_clean":"2023-11-08T16:39:11.226572+0000","last_became_active":"2023-11-08T16:39:10.221554+0000","last_became_peered":"2023-11-08T16:39:10.221554+0000","last_unstale":"2023-11-08T16:39:11.226572+0000","last_undegraded":"2023-11-08T16:39:11.226572+0000","last_fullsized":"2023-11-08T16:39:11.226572+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:07:01.023231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.223837+0000","last_change":"2023-11-08T16:39:10.242700+0000","last_active":"2023-11-08T16:39:11.223837+0000","last_peered":"2023-11-08T16:39:11.223837+0000","last_clean":"2023-11-08T16:39:11.223837+0000","last_became_active":"2023-11-08T16:39:10.242404+0000","last_became_peered":"2023-11-08T16:39:10.242404+0000","last_unstale":"2023-11-08T16:39:11.223837+0000","last_undegraded":"2023-11-08T16:39:11.223837+0000","last_fullsized":"2023-11-08T16:39:11.223837+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:51:42.471499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.288880+0000","last_change":"2023-11-08T16:39:10.221382+0000","last_active":"2023-11-08T16:39:11.288880+0000","last_peered":"2023-11-08T16:39:11.288880+0000","last_clean":"2023-11-08T16:39:11.288880+0000","last_became_active":"2023-11-08T16:39:10.220883+0000","last_became_peered":"2023-11-08T16:39:10.220883+0000","last_unstale":"2023-11-08T16:39:11.288880+0000","last_undegraded":"2023-11-08T16:39:11.288880+0000","last_fullsized":"2023-11-08T16:39:11.288880+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:47:12.014262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.232634+0000","last_change":"2023-11-08T16:39:10.226047+0000","last_active":"2023-11-08T16:39:10.232634+0000","last_peered":"2023-11-08T16:39:10.232634+0000","last_clean":"2023-11-08T16:39:10.232634+0000","last_became_active":"2023-11-08T16:39:10.225706+0000","last_became_peered":"2023-11-08T16:39:10.225706+0000","last_unstale":"2023-11-08T16:39:10.232634+0000","last_undegraded":"2023-11-08T16:39:10.232634+0000","last_fullsized":"2023-11-08T16:39:10.232634+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:49:36.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225473+0000","last_change":"2023-11-08T16:39:10.211514+0000","last_active":"2023-11-08T16:39:11.225473+0000","last_peered":"2023-11-08T16:39:11.225473+0000","last_clean":"2023-11-08T16:39:11.225473+0000","last_became_active":"2023-11-08T16:39:10.210746+0000","last_became_peered":"2023-11-08T16:39:10.210746+0000","last_unstale":"2023-11-08T16:39:11.225473+0000","last_undegraded":"2023-11-08T16:39:11.225473+0000","last_fullsized":"2023-11-08T16:39:11.225473+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:47:12.014262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224642+0000","last_change":"2023-11-08T16:39:10.243326+0000","last_active":"2023-11-08T16:39:11.224642+0000","last_peered":"2023-11-08T16:39:11.224642+0000","last_clean":"2023-11-08T16:39:11.224642+0000","last_became_active":"2023-11-08T16:39:10.242532+0000","last_became_peered":"2023-11-08T16:39:10.242532+0000","last_unstale":"2023-11-08T16:39:11.224642+0000","last_undegraded":"2023-11-08T16:39:11.224642+0000","last_fullsized":"2023-11-08T16:39:11.224642+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:32.776251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.232742+0000","last_change":"2023-11-08T16:39:10.225108+0000","last_active":"2023-11-08T16:39:10.232742+0000","last_peered":"2023-11-08T16:39:10.232742+0000","last_clean":"2023-11-08T16:39:10.232742+0000","last_became_active":"2023-11-08T16:39:10.222292+0000","last_became_peered":"2023-11-08T16:39:10.222292+0000","last_unstale":"2023-11-08T16:39:10.232742+0000","last_undegraded":"2023-11-08T16:39:10.232742+0000","last_fullsized":"2023-11-08T16:39:10.232742+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:41:25.445412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.247598+0000","last_change":"2023-11-08T16:39:10.243222+0000","last_active":"2023-11-08T16:39:10.247598+0000","last_peered":"2023-11-08T16:39:10.247598+0000","last_clean":"2023-11-08T16:39:10.247598+0000","last_became_active":"2023-11-08T16:39:10.242398+0000","last_became_peered":"2023-11-08T16:39:10.242398+0000","last_unstale":"2023-11-08T16:39:10.247598+0000","last_undegraded":"2023-11-08T16:39:10.247598+0000","last_fullsized":"2023-11-08T16:39:10.247598+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:01.474655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228131+0000","last_change":"2023-11-08T16:39:10.220276+0000","last_active":"2023-11-08T16:39:11.228131+0000","last_peered":"2023-11-08T16:39:11.228131+0000","last_clean":"2023-11-08T16:39:11.228131+0000","last_became_active":"2023-11-08T16:39:10.219959+0000","last_became_peered":"2023-11-08T16:39:10.219959+0000","last_unstale":"2023-11-08T16:39:11.228131+0000","last_undegraded":"2023-11-08T16:39:11.228131+0000","last_fullsized":"2023-11-08T16:39:11.228131+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:22.834895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224079+0000","last_change":"2023-11-08T16:39:10.216773+0000","last_active":"2023-11-08T16:39:11.224079+0000","last_peered":"2023-11-08T16:39:11.224079+0000","last_clean":"2023-11-08T16:39:11.224079+0000","last_became_active":"2023-11-08T16:39:10.215761+0000","last_became_peered":"2023-11-08T16:39:10.215761+0000","last_unstale":"2023-11-08T16:39:11.224079+0000","last_undegraded":"2023-11-08T16:39:11.224079+0000","last_fullsized":"2023-11-08T16:39:11.224079+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:49:36.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.226530+0000","last_change":"2023-11-08T16:39:10.234935+0000","last_active":"2023-11-08T16:39:11.226530+0000","last_peered":"2023-11-08T16:39:11.226530+0000","last_clean":"2023-11-08T16:39:11.226530+0000","last_became_active":"2023-11-08T16:39:10.232089+0000","last_became_peered":"2023-11-08T16:39:10.232089+0000","last_unstale":"2023-11-08T16:39:11.226530+0000","last_undegraded":"2023-11-08T16:39:11.226530+0000","last_fullsized":"2023-11-08T16:39:11.226530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:46:23.223886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.231370+0000","last_change":"2023-11-08T16:39:10.219794+0000","last_active":"2023-11-08T16:39:10.231370+0000","last_peered":"2023-11-08T16:39:10.231370+0000","last_clean":"2023-11-08T16:39:10.231370+0000","last_became_active":"2023-11-08T16:39:10.218461+0000","last_became_peered":"2023-11-08T16:39:10.218461+0000","last_unstale":"2023-11-08T16:39:10.231370+0000","last_undegraded":"2023-11-08T16:39:10.231370+0000","last_fullsized":"2023-11-08T16:39:10.231370+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:51:04.503788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.237432+0000","last_change":"2023-11-08T16:39:10.225266+0000","last_active":"2023-11-08T16:39:10.237432+0000","last_peered":"2023-11-08T16:39:10.237432+0000","last_clean":"2023-11-08T16:39:10.237432+0000","last_became_active":"2023-11-08T16:39:10.224993+0000","last_became_peered":"2023-11-08T16:39:10.224993+0000","last_unstale":"2023-11-08T16:39:10.237432+0000","last_undegraded":"2023-11-08T16:39:10.237432+0000","last_fullsized":"2023-11-08T16:39:10.237432+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:05:29.168734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"102'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289312+0000","last_change":"2023-11-08T16:39:09.187878+0000","last_active":"2023-11-08T16:39:11.289312+0000","last_peered":"2023-11-08T16:39:11.289312+0000","last_clean":"2023-11-08T16:39:11.289312+0000","last_became_active":"2023-11-08T16:39:09.187197+0000","last_became_peered":"2023-11-08T16:39:09.187197+0000","last_unstale":"2023-11-08T16:39:11.289312+0000","last_undegraded":"2023-11-08T16:39:11.289312+0000","last_fullsized":"2023-11-08T16:39:11.289312+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:00.469842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.236142+0000","last_change":"2023-11-08T16:39:10.224562+0000","last_active":"2023-11-08T16:39:10.236142+0000","last_peered":"2023-11-08T16:39:10.236142+0000","last_clean":"2023-11-08T16:39:10.236142+0000","last_became_active":"2023-11-08T16:39:10.224266+0000","last_became_peered":"2023-11-08T16:39:10.224266+0000","last_unstale":"2023-11-08T16:39:10.236142+0000","last_undegraded":"2023-11-08T16:39:10.236142+0000","last_fullsized":"2023-11-08T16:39:10.236142+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:16:53.407457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224991+0000","last_change":"2023-11-08T16:39:09.187022+0000","last_active":"2023-11-08T16:39:11.224991+0000","last_peered":"2023-11-08T16:39:11.224991+0000","last_clean":"2023-11-08T16:39:11.224991+0000","last_became_active":"2023-11-08T16:39:09.185770+0000","last_became_peered":"2023-11-08T16:39:09.185770+0000","last_unstale":"2023-11-08T16:39:11.224991+0000","last_undegraded":"2023-11-08T16:39:11.224991+0000","last_fullsized":"2023-11-08T16:39:11.224991+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:58.270995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228573+0000","last_change":"2023-11-08T16:39:10.230250+0000","last_active":"2023-11-08T16:39:11.228573+0000","last_peered":"2023-11-08T16:39:11.228573+0000","last_clean":"2023-11-08T16:39:11.228573+0000","last_became_active":"2023-11-08T16:39:10.229969+0000","last_became_peered":"2023-11-08T16:39:10.229969+0000","last_unstale":"2023-11-08T16:39:11.228573+0000","last_undegraded":"2023-11-08T16:39:11.228573+0000","last_fullsized":"2023-11-08T16:39:11.228573+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:46:38.004348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228055+0000","last_change":"2023-11-08T16:39:09.172522+0000","last_active":"2023-11-08T16:39:11.228055+0000","last_peered":"2023-11-08T16:39:11.228055+0000","last_clean":"2023-11-08T16:39:11.228055+0000","last_became_active":"2023-11-08T16:39:09.172097+0000","last_became_peered":"2023-11-08T16:39:09.172097+0000","last_unstale":"2023-11-08T16:39:11.228055+0000","last_undegraded":"2023-11-08T16:39:11.228055+0000","last_fullsized":"2023-11-08T16:39:11.228055+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:47:11.009449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.219751+0000","last_change":"2023-11-08T16:39:10.214575+0000","last_active":"2023-11-08T16:39:10.219751+0000","last_peered":"2023-11-08T16:39:10.219751+0000","last_clean":"2023-11-08T16:39:10.219751+0000","last_became_active":"2023-11-08T16:39:10.214052+0000","last_became_peered":"2023-11-08T16:39:10.214052+0000","last_unstale":"2023-11-08T16:39:10.219751+0000","last_undegraded":"2023-11-08T16:39:10.219751+0000","last_fullsized":"2023-11-08T16:39:10.219751+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:41:07.662978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289640+0000","last_change":"2023-11-08T16:39:09.180548+0000","last_active":"2023-11-08T16:39:11.289640+0000","last_peered":"2023-11-08T16:39:11.289640+0000","last_clean":"2023-11-08T16:39:11.289640+0000","last_became_active":"2023-11-08T16:39:09.179872+0000","last_became_peered":"2023-11-08T16:39:09.179872+0000","last_unstale":"2023-11-08T16:39:11.289640+0000","last_undegraded":"2023-11-08T16:39:11.289640+0000","last_fullsized":"2023-11-08T16:39:11.289640+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:44:57.514158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.240311+0000","last_change":"2023-11-08T16:39:10.232964+0000","last_active":"2023-11-08T16:39:10.240311+0000","last_peered":"2023-11-08T16:39:10.240311+0000","last_clean":"2023-11-08T16:39:10.240311+0000","last_became_active":"2023-11-08T16:39:10.232681+0000","last_became_peered":"2023-11-08T16:39:10.232681+0000","last_unstale":"2023-11-08T16:39:10.240311+0000","last_undegraded":"2023-11-08T16:39:10.240311+0000","last_fullsized":"2023-11-08T16:39:10.240311+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:54:41.044887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.226761+0000","last_change":"2023-11-08T16:39:09.186518+0000","last_active":"2023-11-08T16:39:11.226761+0000","last_peered":"2023-11-08T16:39:11.226761+0000","last_clean":"2023-11-08T16:39:11.226761+0000","last_became_active":"2023-11-08T16:39:09.185088+0000","last_became_peered":"2023-11-08T16:39:09.185088+0000","last_unstale":"2023-11-08T16:39:11.226761+0000","last_undegraded":"2023-11-08T16:39:11.226761+0000","last_fullsized":"2023-11-08T16:39:11.226761+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:54:40.040074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289066+0000","last_change":"2023-11-08T16:39:10.216338+0000","last_active":"2023-11-08T16:39:11.289066+0000","last_peered":"2023-11-08T16:39:11.289066+0000","last_clean":"2023-11-08T16:39:11.289066+0000","last_became_active":"2023-11-08T16:39:10.216020+0000","last_became_peered":"2023-11-08T16:39:10.216020+0000","last_unstale":"2023-11-08T16:39:11.289066+0000","last_undegraded":"2023-11-08T16:39:11.289066+0000","last_fullsized":"2023-11-08T16:39:11.289066+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:59.275808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.243703+0000","last_change":"2023-11-08T16:39:09.169965+0000","last_active":"2023-11-08T16:39:10.243703+0000","last_peered":"2023-11-08T16:39:10.243703+0000","last_clean":"2023-11-08T16:39:10.243703+0000","last_became_active":"2023-11-08T16:39:09.168823+0000","last_became_peered":"2023-11-08T16:39:09.168823+0000","last_unstale":"2023-11-08T16:39:10.243703+0000","last_undegraded":"2023-11-08T16:39:10.243703+0000","last_fullsized":"2023-11-08T16:39:10.243703+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:47:11.009449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.239560+0000","last_change":"2023-11-08T16:39:10.234973+0000","last_active":"2023-11-08T16:39:10.239560+0000","last_peered":"2023-11-08T16:39:10.239560+0000","last_clean":"2023-11-08T16:39:10.239560+0000","last_became_active":"2023-11-08T16:39:10.231757+0000","last_became_peered":"2023-11-08T16:39:10.231757+0000","last_unstale":"2023-11-08T16:39:10.239560+0000","last_undegraded":"2023-11-08T16:39:10.239560+0000","last_fullsized":"2023-11-08T16:39:10.239560+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:00:21.625962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.222563+0000","last_change":"2023-11-08T16:39:09.168677+0000","last_active":"2023-11-08T16:39:10.222563+0000","last_peered":"2023-11-08T16:39:10.222563+0000","last_clean":"2023-11-08T16:39:10.222563+0000","last_became_active":"2023-11-08T16:39:09.165890+0000","last_became_peered":"2023-11-08T16:39:09.165890+0000","last_unstale":"2023-11-08T16:39:10.222563+0000","last_undegraded":"2023-11-08T16:39:10.222563+0000","last_fullsized":"2023-11-08T16:39:10.222563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:58:51.134519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.230891+0000","last_change":"2023-11-08T16:39:10.226104+0000","last_active":"2023-11-08T16:39:10.230891+0000","last_peered":"2023-11-08T16:39:10.230891+0000","last_clean":"2023-11-08T16:39:10.230891+0000","last_became_active":"2023-11-08T16:39:10.219988+0000","last_became_peered":"2023-11-08T16:39:10.219988+0000","last_unstale":"2023-11-08T16:39:10.230891+0000","last_undegraded":"2023-11-08T16:39:10.230891+0000","last_fullsized":"2023-11-08T16:39:10.230891+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:48:48.116487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.227836+0000","last_change":"2023-11-08T16:39:09.175817+0000","last_active":"2023-11-08T16:39:11.227836+0000","last_peered":"2023-11-08T16:39:11.227836+0000","last_clean":"2023-11-08T16:39:11.227836+0000","last_became_active":"2023-11-08T16:39:09.175306+0000","last_became_peered":"2023-11-08T16:39:09.175306+0000","last_unstale":"2023-11-08T16:39:11.227836+0000","last_undegraded":"2023-11-08T16:39:11.227836+0000","last_fullsized":"2023-11-08T16:39:11.227836+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:54:40.040074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225144+0000","last_change":"2023-11-08T16:39:10.231814+0000","last_active":"2023-11-08T16:39:11.225144+0000","last_peered":"2023-11-08T16:39:11.225144+0000","last_clean":"2023-11-08T16:39:11.225144+0000","last_became_active":"2023-11-08T16:39:10.231528+0000","last_became_peered":"2023-11-08T16:39:10.231528+0000","last_unstale":"2023-11-08T16:39:11.225144+0000","last_undegraded":"2023-11-08T16:39:11.225144+0000","last_fullsized":"2023-11-08T16:39:11.225144+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:46:38.004348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225581+0000","last_change":"2023-11-08T16:39:09.185547+0000","last_active":"2023-11-08T16:39:11.225581+0000","last_peered":"2023-11-08T16:39:11.225581+0000","last_clean":"2023-11-08T16:39:11.225581+0000","last_became_active":"2023-11-08T16:39:09.183003+0000","last_became_peered":"2023-11-08T16:39:09.183003+0000","last_unstale":"2023-11-08T16:39:11.225581+0000","last_undegraded":"2023-11-08T16:39:11.225581+0000","last_fullsized":"2023-11-08T16:39:11.225581+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:54:40.040074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228201+0000","last_change":"2023-11-08T16:39:10.224065+0000","last_active":"2023-11-08T16:39:11.228201+0000","last_peered":"2023-11-08T16:39:11.228201+0000","last_clean":"2023-11-08T16:39:11.228201+0000","last_became_active":"2023-11-08T16:39:10.223779+0000","last_became_peered":"2023-11-08T16:39:10.223779+0000","last_unstale":"2023-11-08T16:39:11.228201+0000","last_undegraded":"2023-11-08T16:39:11.228201+0000","last_fullsized":"2023-11-08T16:39:11.228201+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:53:06.069562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.225950+0000","last_change":"2023-11-08T16:39:09.173054+0000","last_active":"2023-11-08T16:39:10.225950+0000","last_peered":"2023-11-08T16:39:10.225950+0000","last_clean":"2023-11-08T16:39:10.225950+0000","last_became_active":"2023-11-08T16:39:09.172799+0000","last_became_peered":"2023-11-08T16:39:09.172799+0000","last_unstale":"2023-11-08T16:39:10.225950+0000","last_undegraded":"2023-11-08T16:39:10.225950+0000","last_fullsized":"2023-11-08T16:39:10.225950+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:07:00.018418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.241199+0000","last_change":"2023-11-08T16:39:10.237632+0000","last_active":"2023-11-08T16:39:10.241199+0000","last_peered":"2023-11-08T16:39:10.241199+0000","last_clean":"2023-11-08T16:39:10.241199+0000","last_became_active":"2023-11-08T16:39:10.237353+0000","last_became_peered":"2023-11-08T16:39:10.237353+0000","last_unstale":"2023-11-08T16:39:10.241199+0000","last_undegraded":"2023-11-08T16:39:10.241199+0000","last_fullsized":"2023-11-08T16:39:10.241199+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:53:06.069562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"102'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225688+0000","last_change":"2023-11-08T16:39:09.184567+0000","last_active":"2023-11-08T16:39:11.225688+0000","last_peered":"2023-11-08T16:39:11.225688+0000","last_clean":"2023-11-08T16:39:11.225688+0000","last_became_active":"2023-11-08T16:39:09.184246+0000","last_became_peered":"2023-11-08T16:39:09.184246+0000","last_unstale":"2023-11-08T16:39:11.225688+0000","last_undegraded":"2023-11-08T16:39:11.225688+0000","last_fullsized":"2023-11-08T16:39:11.225688+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:51.522987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.245516+0000","last_change":"2023-11-08T16:39:10.242482+0000","last_active":"2023-11-08T16:39:10.245516+0000","last_peered":"2023-11-08T16:39:10.245516+0000","last_clean":"2023-11-08T16:39:10.245516+0000","last_became_active":"2023-11-08T16:39:10.242187+0000","last_became_peered":"2023-11-08T16:39:10.242187+0000","last_unstale":"2023-11-08T16:39:10.245516+0000","last_undegraded":"2023-11-08T16:39:10.245516+0000","last_fullsized":"2023-11-08T16:39:10.245516+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:14:35.821692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"102'6","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.245782+0000","last_change":"2023-11-08T16:39:09.181877+0000","last_active":"2023-11-08T16:39:10.245782+0000","last_peered":"2023-11-08T16:39:10.245782+0000","last_clean":"2023-11-08T16:39:10.245782+0000","last_became_active":"2023-11-08T16:39:09.181312+0000","last_became_peered":"2023-11-08T16:39:09.181312+0000","last_unstale":"2023-11-08T16:39:10.245782+0000","last_undegraded":"2023-11-08T16:39:10.245782+0000","last_fullsized":"2023-11-08T16:39:10.245782+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:20.289964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.244230+0000","last_change":"2023-11-08T16:39:10.240472+0000","last_active":"2023-11-08T16:39:10.244230+0000","last_peered":"2023-11-08T16:39:10.244230+0000","last_clean":"2023-11-08T16:39:10.244230+0000","last_became_active":"2023-11-08T16:39:10.240113+0000","last_became_peered":"2023-11-08T16:39:10.240113+0000","last_unstale":"2023-11-08T16:39:10.244230+0000","last_undegraded":"2023-11-08T16:39:10.244230+0000","last_fullsized":"2023-11-08T16:39:10.244230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:01:39.873637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228308+0000","last_change":"2023-11-08T16:39:09.180771+0000","last_active":"2023-11-08T16:39:11.228308+0000","last_peered":"2023-11-08T16:39:11.228308+0000","last_clean":"2023-11-08T16:39:11.228308+0000","last_became_active":"2023-11-08T16:39:09.180449+0000","last_became_peered":"2023-11-08T16:39:09.180449+0000","last_unstale":"2023-11-08T16:39:11.228308+0000","last_undegraded":"2023-11-08T16:39:11.228308+0000","last_fullsized":"2023-11-08T16:39:11.228308+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:30:11.665802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.223754+0000","last_change":"2023-11-08T16:39:10.217002+0000","last_active":"2023-11-08T16:39:11.223754+0000","last_peered":"2023-11-08T16:39:11.223754+0000","last_clean":"2023-11-08T16:39:11.223754+0000","last_became_active":"2023-11-08T16:39:10.214516+0000","last_became_peered":"2023-11-08T16:39:10.214516+0000","last_unstale":"2023-11-08T16:39:11.223754+0000","last_undegraded":"2023-11-08T16:39:11.223754+0000","last_fullsized":"2023-11-08T16:39:11.223754+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:30:12.670615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.198905+0000","last_change":"2023-11-08T16:39:09.183592+0000","last_active":"2023-11-08T16:39:10.198905+0000","last_peered":"2023-11-08T16:39:10.198905+0000","last_clean":"2023-11-08T16:39:10.198905+0000","last_became_active":"2023-11-08T16:39:09.183313+0000","last_became_peered":"2023-11-08T16:39:09.183313+0000","last_unstale":"2023-11-08T16:39:10.198905+0000","last_undegraded":"2023-11-08T16:39:10.198905+0000","last_fullsized":"2023-11-08T16:39:10.198905+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:06:09.999691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.239228+0000","last_change":"2023-11-08T16:39:10.231934+0000","last_active":"2023-11-08T16:39:10.239228+0000","last_peered":"2023-11-08T16:39:10.239228+0000","last_clean":"2023-11-08T16:39:10.239228+0000","last_became_active":"2023-11-08T16:39:10.231667+0000","last_became_peered":"2023-11-08T16:39:10.231667+0000","last_unstale":"2023-11-08T16:39:10.239228+0000","last_undegraded":"2023-11-08T16:39:10.239228+0000","last_fullsized":"2023-11-08T16:39:10.239228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:09:54.424406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.216479+0000","last_change":"2023-11-08T16:39:09.185054+0000","last_active":"2023-11-08T16:39:10.216479+0000","last_peered":"2023-11-08T16:39:10.216479+0000","last_clean":"2023-11-08T16:39:10.216479+0000","last_became_active":"2023-11-08T16:39:09.184699+0000","last_became_peered":"2023-11-08T16:39:09.184699+0000","last_unstale":"2023-11-08T16:39:10.216479+0000","last_undegraded":"2023-11-08T16:39:10.216479+0000","last_fullsized":"2023-11-08T16:39:10.216479+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:57:02.460058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.230692+0000","last_change":"2023-11-08T16:39:10.224794+0000","last_active":"2023-11-08T16:39:10.230692+0000","last_peered":"2023-11-08T16:39:10.230692+0000","last_clean":"2023-11-08T16:39:10.230692+0000","last_became_active":"2023-11-08T16:39:10.224431+0000","last_became_peered":"2023-11-08T16:39:10.224431+0000","last_unstale":"2023-11-08T16:39:10.230692+0000","last_undegraded":"2023-11-08T16:39:10.230692+0000","last_fullsized":"2023-11-08T16:39:10.230692+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:34.587030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"102'8","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.244792+0000","last_change":"2023-11-08T16:39:09.162958+0000","last_active":"2023-11-08T16:39:10.244792+0000","last_peered":"2023-11-08T16:39:10.244792+0000","last_clean":"2023-11-08T16:39:10.244792+0000","last_became_active":"2023-11-08T16:39:09.162581+0000","last_became_peered":"2023-11-08T16:39:09.162581+0000","last_unstale":"2023-11-08T16:39:10.244792+0000","last_undegraded":"2023-11-08T16:39:10.244792+0000","last_fullsized":"2023-11-08T16:39:10.244792+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:54.503796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.237451+0000","last_change":"2023-11-08T16:39:10.226079+0000","last_active":"2023-11-08T16:39:10.237451+0000","last_peered":"2023-11-08T16:39:10.237451+0000","last_clean":"2023-11-08T16:39:10.237451+0000","last_became_active":"2023-11-08T16:39:10.224515+0000","last_became_peered":"2023-11-08T16:39:10.224515+0000","last_unstale":"2023-11-08T16:39:10.237451+0000","last_undegraded":"2023-11-08T16:39:10.237451+0000","last_fullsized":"2023-11-08T16:39:10.237451+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:54:07.505364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289736+0000","last_change":"2023-11-08T16:39:09.184787+0000","last_active":"2023-11-08T16:39:11.289736+0000","last_peered":"2023-11-08T16:39:11.289736+0000","last_clean":"2023-11-08T16:39:11.289736+0000","last_became_active":"2023-11-08T16:39:09.183611+0000","last_became_peered":"2023-11-08T16:39:09.183611+0000","last_unstale":"2023-11-08T16:39:11.289736+0000","last_undegraded":"2023-11-08T16:39:11.289736+0000","last_fullsized":"2023-11-08T16:39:11.289736+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:24.926586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289610+0000","last_change":"2023-11-08T16:39:10.206877+0000","last_active":"2023-11-08T16:39:11.289610+0000","last_peered":"2023-11-08T16:39:11.289610+0000","last_clean":"2023-11-08T16:39:11.289610+0000","last_became_active":"2023-11-08T16:39:10.206224+0000","last_became_peered":"2023-11-08T16:39:10.206224+0000","last_unstale":"2023-11-08T16:39:11.289610+0000","last_undegraded":"2023-11-08T16:39:11.289610+0000","last_fullsized":"2023-11-08T16:39:11.289610+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:53.831527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228415+0000","last_change":"2023-11-08T16:39:09.189356+0000","last_active":"2023-11-08T16:39:11.228415+0000","last_peered":"2023-11-08T16:39:11.228415+0000","last_clean":"2023-11-08T16:39:11.228415+0000","last_became_active":"2023-11-08T16:39:09.186501+0000","last_became_peered":"2023-11-08T16:39:09.186501+0000","last_unstale":"2023-11-08T16:39:11.228415+0000","last_undegraded":"2023-11-08T16:39:11.228415+0000","last_fullsized":"2023-11-08T16:39:11.228415+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:58.270995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224844+0000","last_change":"2023-11-08T16:39:10.229070+0000","last_active":"2023-11-08T16:39:11.224844+0000","last_peered":"2023-11-08T16:39:11.224844+0000","last_clean":"2023-11-08T16:39:11.224844+0000","last_became_active":"2023-11-08T16:39:10.228754+0000","last_became_peered":"2023-11-08T16:39:10.228754+0000","last_unstale":"2023-11-08T16:39:11.224844+0000","last_undegraded":"2023-11-08T16:39:11.224844+0000","last_fullsized":"2023-11-08T16:39:11.224844+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:53:06.069562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225920+0000","last_change":"2023-11-08T16:39:09.185406+0000","last_active":"2023-11-08T16:39:11.225920+0000","last_peered":"2023-11-08T16:39:11.225920+0000","last_clean":"2023-11-08T16:39:11.225920+0000","last_became_active":"2023-11-08T16:39:09.185069+0000","last_became_peered":"2023-11-08T16:39:09.185069+0000","last_unstale":"2023-11-08T16:39:11.225920+0000","last_undegraded":"2023-11-08T16:39:11.225920+0000","last_fullsized":"2023-11-08T16:39:11.225920+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:00.469842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.231389+0000","last_change":"2023-11-08T16:39:10.220489+0000","last_active":"2023-11-08T16:39:10.231389+0000","last_peered":"2023-11-08T16:39:10.231389+0000","last_clean":"2023-11-08T16:39:10.231389+0000","last_became_active":"2023-11-08T16:39:10.220123+0000","last_became_peered":"2023-11-08T16:39:10.220123+0000","last_unstale":"2023-11-08T16:39:10.231389+0000","last_undegraded":"2023-11-08T16:39:10.231389+0000","last_fullsized":"2023-11-08T16:39:10.231389+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:19:02.824312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225198+0000","last_change":"2023-11-08T16:39:09.183311+0000","last_active":"2023-11-08T16:39:11.225198+0000","last_peered":"2023-11-08T16:39:11.225198+0000","last_clean":"2023-11-08T16:39:11.225198+0000","last_became_active":"2023-11-08T16:39:09.183038+0000","last_became_peered":"2023-11-08T16:39:09.183038+0000","last_unstale":"2023-11-08T16:39:11.225198+0000","last_undegraded":"2023-11-08T16:39:11.225198+0000","last_fullsized":"2023-11-08T16:39:11.225198+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:24.926586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225823+0000","last_change":"2023-11-08T16:39:10.227910+0000","last_active":"2023-11-08T16:39:11.225823+0000","last_peered":"2023-11-08T16:39:11.225823+0000","last_clean":"2023-11-08T16:39:11.225823+0000","last_became_active":"2023-11-08T16:39:10.227089+0000","last_became_peered":"2023-11-08T16:39:10.227089+0000","last_unstale":"2023-11-08T16:39:11.225823+0000","last_undegraded":"2023-11-08T16:39:11.225823+0000","last_fullsized":"2023-11-08T16:39:11.225823+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:43.391913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.212076+0000","last_change":"2023-11-08T16:39:09.172863+0000","last_active":"2023-11-08T16:39:10.212076+0000","last_peered":"2023-11-08T16:39:10.212076+0000","last_clean":"2023-11-08T16:39:10.212076+0000","last_became_active":"2023-11-08T16:39:09.172536+0000","last_became_peered":"2023-11-08T16:39:09.172536+0000","last_unstale":"2023-11-08T16:39:10.212076+0000","last_undegraded":"2023-11-08T16:39:10.212076+0000","last_fullsized":"2023-11-08T16:39:10.212076+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:15:17.786066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224904+0000","last_change":"2023-11-08T16:39:10.233062+0000","last_active":"2023-11-08T16:39:11.224904+0000","last_peered":"2023-11-08T16:39:11.224904+0000","last_clean":"2023-11-08T16:39:11.224904+0000","last_became_active":"2023-11-08T16:39:10.232791+0000","last_became_peered":"2023-11-08T16:39:10.232791+0000","last_unstale":"2023-11-08T16:39:11.224904+0000","last_undegraded":"2023-11-08T16:39:11.224904+0000","last_fullsized":"2023-11-08T16:39:11.224904+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:55.508609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225937+0000","last_change":"2023-11-08T16:39:09.183397+0000","last_active":"2023-11-08T16:39:11.225937+0000","last_peered":"2023-11-08T16:39:11.225937+0000","last_clean":"2023-11-08T16:39:11.225937+0000","last_became_active":"2023-11-08T16:39:09.183081+0000","last_became_peered":"2023-11-08T16:39:09.183081+0000","last_unstale":"2023-11-08T16:39:11.225937+0000","last_undegraded":"2023-11-08T16:39:11.225937+0000","last_fullsized":"2023-11-08T16:39:11.225937+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:14:34.816878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.237104+0000","last_change":"2023-11-08T16:39:10.226471+0000","last_active":"2023-11-08T16:39:10.237104+0000","last_peered":"2023-11-08T16:39:10.237104+0000","last_clean":"2023-11-08T16:39:10.237104+0000","last_became_active":"2023-11-08T16:39:10.225106+0000","last_became_peered":"2023-11-08T16:39:10.225106+0000","last_unstale":"2023-11-08T16:39:10.237104+0000","last_undegraded":"2023-11-08T16:39:10.237104+0000","last_fullsized":"2023-11-08T16:39:10.237104+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:30:02.487471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.212223+0000","last_change":"2023-11-08T16:39:09.184775+0000","last_active":"2023-11-08T16:39:10.212223+0000","last_peered":"2023-11-08T16:39:10.212223+0000","last_clean":"2023-11-08T16:39:10.212223+0000","last_became_active":"2023-11-08T16:39:09.184484+0000","last_became_peered":"2023-11-08T16:39:09.184484+0000","last_unstale":"2023-11-08T16:39:10.212223+0000","last_undegraded":"2023-11-08T16:39:10.212223+0000","last_fullsized":"2023-11-08T16:39:10.212223+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:54:53.437907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289526+0000","last_change":"2023-11-08T16:39:10.208466+0000","last_active":"2023-11-08T16:39:11.289526+0000","last_peered":"2023-11-08T16:39:11.289526+0000","last_clean":"2023-11-08T16:39:11.289526+0000","last_became_active":"2023-11-08T16:39:10.207971+0000","last_became_peered":"2023-11-08T16:39:10.207971+0000","last_unstale":"2023-11-08T16:39:11.289526+0000","last_undegraded":"2023-11-08T16:39:11.289526+0000","last_fullsized":"2023-11-08T16:39:11.289526+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:46:23.223886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228449+0000","last_change":"2023-11-08T16:39:09.188262+0000","last_active":"2023-11-08T16:39:11.228449+0000","last_peered":"2023-11-08T16:39:11.228449+0000","last_clean":"2023-11-08T16:39:11.228449+0000","last_became_active":"2023-11-08T16:39:09.186561+0000","last_became_peered":"2023-11-08T16:39:09.186561+0000","last_unstale":"2023-11-08T16:39:11.228449+0000","last_undegraded":"2023-11-08T16:39:11.228449+0000","last_fullsized":"2023-11-08T16:39:11.228449+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:49:35.299981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.235369+0000","last_change":"2023-11-08T16:39:10.223848+0000","last_active":"2023-11-08T16:39:10.235369+0000","last_peered":"2023-11-08T16:39:10.235369+0000","last_clean":"2023-11-08T16:39:10.235369+0000","last_became_active":"2023-11-08T16:39:10.223505+0000","last_became_peered":"2023-11-08T16:39:10.223505+0000","last_unstale":"2023-11-08T16:39:10.235369+0000","last_undegraded":"2023-11-08T16:39:10.235369+0000","last_fullsized":"2023-11-08T16:39:10.235369+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:01:21.503000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225134+0000","last_change":"2023-11-08T16:39:09.188211+0000","last_active":"2023-11-08T16:39:11.225134+0000","last_peered":"2023-11-08T16:39:11.225134+0000","last_clean":"2023-11-08T16:39:11.225134+0000","last_became_active":"2023-11-08T16:39:09.187938+0000","last_became_peered":"2023-11-08T16:39:09.187938+0000","last_unstale":"2023-11-08T16:39:11.225134+0000","last_undegraded":"2023-11-08T16:39:11.225134+0000","last_fullsized":"2023-11-08T16:39:11.225134+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:42.387100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.232405+0000","last_change":"2023-11-08T16:39:10.221431+0000","last_active":"2023-11-08T16:39:10.232405+0000","last_peered":"2023-11-08T16:39:10.232405+0000","last_clean":"2023-11-08T16:39:10.232405+0000","last_became_active":"2023-11-08T16:39:10.221168+0000","last_became_peered":"2023-11-08T16:39:10.221168+0000","last_unstale":"2023-11-08T16:39:10.232405+0000","last_undegraded":"2023-11-08T16:39:10.232405+0000","last_fullsized":"2023-11-08T16:39:10.232405+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:20:23.511414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.226126+0000","last_change":"2023-11-08T16:39:09.184420+0000","last_active":"2023-11-08T16:39:11.226126+0000","last_peered":"2023-11-08T16:39:11.226126+0000","last_clean":"2023-11-08T16:39:11.226126+0000","last_became_active":"2023-11-08T16:39:09.183724+0000","last_became_peered":"2023-11-08T16:39:09.183724+0000","last_unstale":"2023-11-08T16:39:11.226126+0000","last_undegraded":"2023-11-08T16:39:11.226126+0000","last_fullsized":"2023-11-08T16:39:11.226126+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:24.926586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225583+0000","last_change":"2023-11-08T16:39:10.223514+0000","last_active":"2023-11-08T16:39:11.225583+0000","last_peered":"2023-11-08T16:39:11.225583+0000","last_clean":"2023-11-08T16:39:11.225583+0000","last_became_active":"2023-11-08T16:39:10.223211+0000","last_became_peered":"2023-11-08T16:39:10.223211+0000","last_unstale":"2023-11-08T16:39:11.225583+0000","last_undegraded":"2023-11-08T16:39:11.225583+0000","last_fullsized":"2023-11-08T16:39:11.225583+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:59.275808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289754+0000","last_change":"2023-11-08T16:39:09.180750+0000","last_active":"2023-11-08T16:39:11.289754+0000","last_peered":"2023-11-08T16:39:11.289754+0000","last_clean":"2023-11-08T16:39:11.289754+0000","last_became_active":"2023-11-08T16:39:09.180237+0000","last_became_peered":"2023-11-08T16:39:09.180237+0000","last_unstale":"2023-11-08T16:39:11.289754+0000","last_undegraded":"2023-11-08T16:39:11.289754+0000","last_fullsized":"2023-11-08T16:39:11.289754+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:54:40.040074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.219326+0000","last_change":"2023-11-08T16:39:10.216556+0000","last_active":"2023-11-08T16:39:10.219326+0000","last_peered":"2023-11-08T16:39:10.219326+0000","last_clean":"2023-11-08T16:39:10.219326+0000","last_became_active":"2023-11-08T16:39:10.216243+0000","last_became_peered":"2023-11-08T16:39:10.216243+0000","last_unstale":"2023-11-08T16:39:10.219326+0000","last_undegraded":"2023-11-08T16:39:10.219326+0000","last_fullsized":"2023-11-08T16:39:10.219326+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:15:56.431930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.196722+0000","last_change":"2023-11-08T16:39:09.183557+0000","last_active":"2023-11-08T16:39:10.196722+0000","last_peered":"2023-11-08T16:39:10.196722+0000","last_clean":"2023-11-08T16:39:10.196722+0000","last_became_active":"2023-11-08T16:39:09.183300+0000","last_became_peered":"2023-11-08T16:39:09.183300+0000","last_unstale":"2023-11-08T16:39:10.196722+0000","last_undegraded":"2023-11-08T16:39:10.196722+0000","last_fullsized":"2023-11-08T16:39:10.196722+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:16:37.233758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.245746+0000","last_change":"2023-11-08T16:39:10.242852+0000","last_active":"2023-11-08T16:39:10.245746+0000","last_peered":"2023-11-08T16:39:10.245746+0000","last_clean":"2023-11-08T16:39:10.245746+0000","last_became_active":"2023-11-08T16:39:10.241501+0000","last_became_peered":"2023-11-08T16:39:10.241501+0000","last_unstale":"2023-11-08T16:39:10.245746+0000","last_undegraded":"2023-11-08T16:39:10.245746+0000","last_fullsized":"2023-11-08T16:39:10.245746+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:46:38.004348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225040+0000","last_change":"2023-11-08T16:39:09.185269+0000","last_active":"2023-11-08T16:39:11.225040+0000","last_peered":"2023-11-08T16:39:11.225040+0000","last_clean":"2023-11-08T16:39:11.225040+0000","last_became_active":"2023-11-08T16:39:09.184979+0000","last_became_peered":"2023-11-08T16:39:09.184979+0000","last_unstale":"2023-11-08T16:39:11.225040+0000","last_undegraded":"2023-11-08T16:39:11.225040+0000","last_fullsized":"2023-11-08T16:39:11.225040+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:40.559793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289162+0000","last_change":"2023-11-08T16:39:10.235772+0000","last_active":"2023-11-08T16:39:11.289162+0000","last_peered":"2023-11-08T16:39:11.289162+0000","last_clean":"2023-11-08T16:39:11.289162+0000","last_became_active":"2023-11-08T16:39:10.235507+0000","last_became_peered":"2023-11-08T16:39:10.235507+0000","last_unstale":"2023-11-08T16:39:11.289162+0000","last_undegraded":"2023-11-08T16:39:11.289162+0000","last_fullsized":"2023-11-08T16:39:11.289162+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:43.391913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289584+0000","last_change":"2023-11-08T16:39:09.187527+0000","last_active":"2023-11-08T16:39:11.289584+0000","last_peered":"2023-11-08T16:39:11.289584+0000","last_clean":"2023-11-08T16:39:11.289584+0000","last_became_active":"2023-11-08T16:39:09.184253+0000","last_became_peered":"2023-11-08T16:39:09.184253+0000","last_unstale":"2023-11-08T16:39:11.289584+0000","last_undegraded":"2023-11-08T16:39:11.289584+0000","last_fullsized":"2023-11-08T16:39:11.289584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:40.559793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.237084+0000","last_change":"2023-11-08T16:39:10.225772+0000","last_active":"2023-11-08T16:39:10.237084+0000","last_peered":"2023-11-08T16:39:10.237084+0000","last_clean":"2023-11-08T16:39:10.237084+0000","last_became_active":"2023-11-08T16:39:10.225489+0000","last_became_peered":"2023-11-08T16:39:10.225489+0000","last_unstale":"2023-11-08T16:39:10.237084+0000","last_undegraded":"2023-11-08T16:39:10.237084+0000","last_fullsized":"2023-11-08T16:39:10.237084+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:32:49.327179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.227979+0000","last_change":"2023-11-08T16:39:09.175591+0000","last_active":"2023-11-08T16:39:11.227979+0000","last_peered":"2023-11-08T16:39:11.227979+0000","last_clean":"2023-11-08T16:39:11.227979+0000","last_became_active":"2023-11-08T16:39:09.174815+0000","last_became_peered":"2023-11-08T16:39:09.174815+0000","last_unstale":"2023-11-08T16:39:11.227979+0000","last_undegraded":"2023-11-08T16:39:11.227979+0000","last_fullsized":"2023-11-08T16:39:11.227979+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:54.503796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228340+0000","last_change":"2023-11-08T16:39:10.226800+0000","last_active":"2023-11-08T16:39:11.228340+0000","last_peered":"2023-11-08T16:39:11.228340+0000","last_clean":"2023-11-08T16:39:11.228340+0000","last_became_active":"2023-11-08T16:39:10.226497+0000","last_became_peered":"2023-11-08T16:39:10.226497+0000","last_unstale":"2023-11-08T16:39:11.228340+0000","last_undegraded":"2023-11-08T16:39:11.228340+0000","last_fullsized":"2023-11-08T16:39:11.228340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:38.165884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.214238+0000","last_change":"2023-11-08T16:39:09.184176+0000","last_active":"2023-11-08T16:39:10.214238+0000","last_peered":"2023-11-08T16:39:10.214238+0000","last_clean":"2023-11-08T16:39:10.214238+0000","last_became_active":"2023-11-08T16:39:09.183888+0000","last_became_peered":"2023-11-08T16:39:09.183888+0000","last_unstale":"2023-11-08T16:39:10.214238+0000","last_undegraded":"2023-11-08T16:39:10.214238+0000","last_fullsized":"2023-11-08T16:39:10.214238+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:44:57.514158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.196885+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:39:10.196885+0000","last_peered":"2023-11-08T16:39:10.196885+0000","last_clean":"2023-11-08T16:39:10.196885+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:39:10.196885+0000","last_undegraded":"2023-11-08T16:39:10.196885+0000","last_fullsized":"2023-11-08T16:39:10.196885+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574974,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33008,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338832,"statfs":{"total":96636764160,"available":96602963968,"internally_reserved":0,"allocated":1871872,"data_stored":1172678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0609999999999999,"15min":1.0840000000000001},"min":{"1min":0.70699999999999996,"5min":0.51600000000000001,"15min":0.49299999999999999},"max":{"1min":1.4670000000000001,"5min":1.6879999999999999,"15min":3.1899999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0700000000000001,"15min":1.109},"min":{"1min":0.64400000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.595,"5min":1.7629999999999999,"15min":4.2729999999999997},"last":0.93400000000000005}]},{"osd":1,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.091,"15min":1.1160000000000001},"min":{"1min":0.77400000000000002,"5min":0.60199999999999998,"15min":0.57899999999999996},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.8340000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.1459999999999999,"15min":1.204},"min":{"1min":0.622,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6459999999999999,"5min":2.1419999999999999,"15min":3.3740000000000001},"last":1.4490000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.159,"15min":1.236},"min":{"1min":0.70999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5389999999999999,"5min":1.619,"15min":4.2249999999999996},"last":1.147},{"interface":"front","average":{"1min":1.264,"5min":1.2010000000000001,"15min":1.212},"min":{"1min":0.752,"5min":0.67500000000000004,"15min":0.56299999999999994},"max":{"1min":1.6910000000000001,"5min":1.79,"15min":3.4750000000000001},"last":1.3180000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.135,"15min":1.1599999999999999},"min":{"1min":0.751,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4179999999999999,"5min":1.7669999999999999,"15min":2.964},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.333,"15min":1.4379999999999999},"min":{"1min":0.98099999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.7490000000000001,"5min":2.25,"15min":4.181},"last":1.5109999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1859999999999999,"15min":1.222},"min":{"1min":0.90700000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.5529999999999999,"5min":1.6970000000000001,"15min":3.6669999999999998},"last":1.097},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3069999999999999,"15min":1.333},"min":{"1min":1.0089999999999999,"5min":0.77300000000000002,"15min":0.59099999999999997},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.3730000000000002},"last":1.3520000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3819999999999999,"15min":1.3640000000000001},"min":{"1min":1.0129999999999999,"5min":0.89200000000000002,"15min":0.61799999999999999},"max":{"1min":1.865,"5min":2.3559999999999999,"15min":3.1059999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4530000000000001,"15min":1.46},"min":{"1min":1.1930000000000001,"5min":1.0269999999999999,"15min":0.878},"max":{"1min":1.825,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.4159999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.516,"15min":1.53},"min":{"1min":1.208,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.919,"5min":1.923,"15min":3.7530000000000001},"last":1.474},{"interface":"front","average":{"1min":1.496,"5min":1.4570000000000001,"15min":1.484},"min":{"1min":1.087,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0430000000000001,"5min":2.3540000000000001,"15min":6.9560000000000004},"last":1.5669999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":578170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.052,"15min":1.0629999999999999},"min":{"1min":0.874,"5min":0.497,"15min":0.497},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":2.5609999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.0980000000000001,"15min":1.0860000000000001},"min":{"1min":0.95999999999999996,"5min":0.63100000000000001,"15min":0.56200000000000006},"max":{"1min":1.502,"5min":1.6819999999999999,"15min":2.7519999999999998},"last":1.484}]},{"osd":1,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1020000000000001,"15min":1.0669999999999999},"min":{"1min":0.81999999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.875},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1160000000000001,"15min":1.125},"min":{"1min":0.90600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.97},"last":0.93600000000000005}]},{"osd":2,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.171,"15min":1.1859999999999999},"min":{"1min":0.84199999999999997,"5min":0.71599999999999997,"15min":0.56799999999999995},"max":{"1min":1.698,"5min":1.804,"15min":1.804},"last":1.571},{"interface":"front","average":{"1min":1.282,"5min":1.2,"15min":1.171},"min":{"1min":0.97699999999999998,"5min":0.72299999999999998,"15min":0.60499999999999998},"max":{"1min":1.5509999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.1990000000000001,"15min":1.2470000000000001},"min":{"1min":1.022,"5min":0.63200000000000001,"15min":0.58299999999999996},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.1120000000000001},"last":1.347},{"interface":"front","average":{"1min":1.363,"5min":1.278,"15min":1.292},"min":{"1min":0.873,"5min":0.68200000000000005,"15min":0.67300000000000004},"max":{"1min":1.6439999999999999,"5min":1.76,"15min":1.95},"last":1.7509999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2949999999999999,"15min":1.339},"min":{"1min":0.84999999999999998,"5min":0.81799999999999995,"15min":0.65300000000000002},"max":{"1min":1.778,"5min":1.778,"15min":2.218},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3660000000000001,"15min":1.4059999999999999},"min":{"1min":1.028,"5min":0.86899999999999999,"15min":0.64900000000000002},"max":{"1min":1.9019999999999999,"5min":1.9590000000000001,"15min":2.7080000000000002},"last":1.681}]},{"osd":5,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3440000000000001,"15min":1.3240000000000001},"min":{"1min":0.91500000000000004,"5min":0.879,"15min":0.80500000000000005},"max":{"1min":1.6599999999999999,"5min":1.994,"15min":2.105},"last":1.264},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4950000000000001,"15min":1.5369999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.78500000000000003},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.4700000000000002},"last":1.7789999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.5029999999999999,"15min":1.5680000000000001},"min":{"1min":1.3069999999999999,"5min":1.002,"15min":0.74299999999999999},"max":{"1min":2.024,"5min":2.1309999999999998,"15min":3.8959999999999999},"last":1.635},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.498,"15min":1.51},"min":{"1min":1.1479999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":2.069,"5min":2.0870000000000002,"15min":2.4889999999999999},"last":1.407}]}]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":582254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.046,"15min":1.0609999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.575,"5min":1.639,"15min":2.4609999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.069,"15min":1.1100000000000001},"min":{"1min":0.74299999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5660000000000001,"5min":1.593,"15min":2.3639999999999999},"last":1.5700000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2130000000000001,"15min":1.2330000000000001},"min":{"1min":0.77100000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6180000000000001,"5min":1.9610000000000001,"15min":3.8690000000000002},"last":1.532},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.1879999999999999,"15min":1.2509999999999999},"min":{"1min":0.90000000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4950000000000001,"5min":1.9370000000000001,"15min":3.2349999999999999},"last":1.377}]},{"osd":2,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1559999999999999,"15min":1.212},"min":{"1min":0.66100000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5580000000000001,"5min":1.764,"15min":2.6339999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.21,"15min":1.208},"min":{"1min":0.70499999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.577,"5min":1.8029999999999999,"15min":2.2559999999999998},"last":1.3220000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.25,"15min":1.29},"min":{"1min":0.97799999999999998,"5min":0.67300000000000004,"15min":0.66500000000000004},"max":{"1min":1.6519999999999999,"5min":1.843,"15min":2.8330000000000002},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.264,"15min":1.2969999999999999},"min":{"1min":0.84599999999999997,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.8440000000000001,"5min":2.0459999999999998,"15min":2.5760000000000001},"last":1.083}]},{"osd":4,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.262,"15min":1.3380000000000001},"min":{"1min":0.89700000000000002,"5min":0.85499999999999998,"15min":0.61599999999999999},"max":{"1min":1.5389999999999999,"5min":2.0790000000000002,"15min":3.9790000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4159999999999999,"15min":1.4650000000000001},"min":{"1min":1.0569999999999999,"5min":0.98499999999999999,"15min":0.81499999999999995},"max":{"1min":1.8029999999999999,"5min":2.0409999999999999,"15min":3.0209999999999999},"last":1.2010000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4039999999999999,"15min":1.415},"min":{"1min":0.99299999999999999,"5min":0.92400000000000004,"15min":0.752},"max":{"1min":1.7749999999999999,"5min":1.867,"15min":2.5550000000000002},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.3899999999999999,"15min":1.4810000000000001},"min":{"1min":0.98499999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":1.657,"5min":2.1619999999999999,"15min":4.1189999999999998},"last":1.1319999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.52,"15min":1.5620000000000001},"min":{"1min":1.2729999999999999,"5min":1.099,"15min":1.0049999999999999},"max":{"1min":2.0219999999999998,"5min":2.129,"15min":3.1160000000000001},"last":1.512},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5329999999999999,"15min":1.5549999999999999},"min":{"1min":1.1870000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":2.0800000000000001,"5min":2.3660000000000001,"15min":2.7050000000000001},"last":1.4810000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":582414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0820000000000001,"15min":1.0760000000000001},"min":{"1min":0.73199999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1259999999999999,"15min":1.0800000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.45400000000000001},"max":{"1min":1.548,"5min":1.772,"15min":1.772},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.127,"15min":1.1240000000000001},"min":{"1min":0.874,"5min":0.57299999999999995,"15min":0.56999999999999995},"max":{"1min":1.51,"5min":1.587,"15min":2.0459999999999998},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1739999999999999,"15min":1.1679999999999999},"min":{"1min":0.79100000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.462,"5min":1.742,"15min":1.742},"last":0.84599999999999997}]},{"osd":2,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2350000000000001,"15min":1.204},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.6299999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.2330000000000001,"15min":1.25},"min":{"1min":0.92900000000000005,"5min":0.78100000000000003,"15min":0.71699999999999997},"max":{"1min":1.6020000000000001,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.92900000000000005}]},{"osd":3,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.278,"15min":1.276},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79100000000000004},"max":{"1min":1.758,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.23,"5min":1.266,"15min":1.2729999999999999},"min":{"1min":0.95199999999999996,"5min":0.72499999999999998,"15min":0.65100000000000002},"max":{"1min":1.6359999999999999,"5min":1.851,"15min":1.895},"last":1.1459999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":0.90500000000000003,"5min":0.68100000000000005,"15min":0.58799999999999997},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.224},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.464,"15min":1.4319999999999999},"min":{"1min":0.879,"5min":0.86499999999999999,"15min":0.61899999999999999},"max":{"1min":1.7669999999999999,"5min":2.1970000000000001,"15min":2.46},"last":0.879}]},{"osd":6,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3660000000000001,"15min":1.3979999999999999},"min":{"1min":1.089,"5min":0.75800000000000001,"15min":0.74299999999999999},"max":{"1min":1.772,"5min":1.9510000000000001,"15min":2.5609999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.512,"15min":1.5389999999999999},"min":{"1min":1.032,"5min":0.83899999999999997,"15min":0.81200000000000006},"max":{"1min":2.4780000000000002,"5min":2.4780000000000002,"15min":2.6389999999999998},"last":1.369}]},{"osd":7,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.5469999999999999,"15min":1.528},"min":{"1min":1.3029999999999999,"5min":1.056,"15min":0.92300000000000004},"max":{"1min":1.859,"5min":2.1040000000000001,"15min":2.2639999999999998},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.522,"15min":1.569},"min":{"1min":1.2430000000000001,"5min":1.0209999999999999,"15min":0.84799999999999998},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.6859999999999999},"last":1.4179999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":575954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.53100000000000003,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":1.29,"5min":1.4930000000000001,"15min":1.9339999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.85799999999999998,"15min":0.90700000000000003},"min":{"1min":0.46000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.4710000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.0389999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.58599999999999997,"5min":0.40899999999999997,"15min":0.36499999999999999},"max":{"1min":1.179,"5min":1.4770000000000001,"15min":1.4790000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.63400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.425,"5min":1.425,"15min":1.829},"last":1.0940000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0129999999999999,"15min":0.998},"min":{"1min":0.624,"5min":0.48399999999999999,"15min":0.41699999999999998},"max":{"1min":1.474,"5min":1.7490000000000001,"15min":2.0779999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.0840000000000001,"15min":1.1180000000000001},"min":{"1min":0.66300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.6419999999999999,"5min":1.7270000000000001,"15min":2.4660000000000002},"last":1.26}]},{"osd":4,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.149,"15min":1.1830000000000001},"min":{"1min":0.73199999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5720000000000001,"5min":1.8069999999999999,"15min":2.25},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.747},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.1829999999999998},"last":1.1830000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3220000000000001,"15min":1.329},"min":{"1min":1.006,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.3999999999999999},"last":1.296},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.337,"15min":1.339},"min":{"1min":0.996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.714,"5min":1.714,"15min":2.593},"last":1.4430000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3740000000000001,"15min":1.391},"min":{"1min":0.877,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.776,"5min":1.776,"15min":2.9569999999999999},"last":1.355},{"interface":"front","average":{"1min":1.29,"5min":1.3049999999999999,"15min":1.3680000000000001},"min":{"1min":0.83699999999999997,"5min":0.75800000000000001,"15min":0.73399999999999999},"max":{"1min":1.796,"5min":1.8919999999999999,"15min":2.762},"last":0.83599999999999997}]},{"osd":7,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3260000000000001,"15min":1.409},"min":{"1min":0.92200000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6850000000000001,"5min":1.861,"15min":2.8700000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.484,"5min":1.4019999999999999,"15min":1.4239999999999999},"min":{"1min":1.032,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.121,"5min":2.121,"15min":2.633},"last":1.401}]}]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":575954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79500000000000004,"15min":0.93999999999999995},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.24099999999999999},"max":{"1min":1.264,"5min":1.2929999999999999,"15min":4.5369999999999999},"last":0.753},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.87,"15min":0.878},"min":{"1min":0.55700000000000005,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.94099999999999995}]},{"osd":1,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0349999999999999,"15min":1.0800000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.379},"max":{"1min":1.609,"5min":1.7689999999999999,"15min":4.4980000000000002},"last":0.871},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.0589999999999999,"15min":1.052},"min":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.40300000000000002},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.1659999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.86599999999999999,"15min":0.98499999999999999},"min":{"1min":0.50800000000000001,"5min":0.39300000000000002,"15min":0.35499999999999998},"max":{"1min":1.194,"5min":1.4870000000000001,"15min":3.3050000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.0660000000000001,"15min":1.173},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.42999999999999999},"max":{"1min":1.635,"5min":1.819,"15min":4.5899999999999999},"last":1.3109999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2809999999999999,"15min":1.3700000000000001},"min":{"1min":0.76100000000000001,"5min":0.74299999999999999,"15min":0.73499999999999999},"max":{"1min":1.829,"5min":2.2160000000000002,"15min":4.9409999999999998},"last":1.421},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2130000000000001,"15min":1.258},"min":{"1min":0.85499999999999998,"5min":0.72499999999999998,"15min":0.63100000000000001},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.46},"last":1.357}]},{"osd":5,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.252,"15min":1.3009999999999999},"min":{"1min":0.871,"5min":0.78200000000000003,"15min":0.77200000000000002},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":2.8530000000000002},"last":1.45},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.252,"15min":1.3260000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.69399999999999995},"max":{"1min":1.7689999999999999,"5min":2.5390000000000001,"15min":3.5680000000000001},"last":1.5289999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.264,"15min":1.361},"min":{"1min":0.877,"5min":0.81000000000000005,"15min":0.625},"max":{"1min":1.6200000000000001,"5min":1.9710000000000001,"15min":3.536},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3859999999999999,"15min":1.47},"min":{"1min":0.88500000000000001,"5min":0.755,"15min":0.72199999999999998},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":4.7549999999999999},"last":1.212}]},{"osd":7,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3280000000000001,"15min":1.4099999999999999},"min":{"1min":0.745,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.9700000000000002},"last":1.516},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3640000000000001,"15min":1.4570000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.70499999999999996},"max":{"1min":1.702,"5min":2.129,"15min":3.4950000000000001},"last":1.7789999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":576024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.28899999999999998},"max":{"1min":1.1739999999999999,"5min":1.542,"15min":1.542},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.877,"15min":0.96699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.309},"max":{"1min":1.47,"5min":1.47,"15min":4.0309999999999997},"last":0.57599999999999996}]},{"osd":2,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0029999999999999,"15min":1.054},"min":{"1min":0.48899999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.462,"5min":1.7689999999999999,"15min":3.0339999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.95099999999999996,"15min":1.028},"min":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":1.542,"5min":1.542,"15min":2.6840000000000002},"last":0.46899999999999997}]},{"osd":3,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.997,"15min":1.0629999999999999},"min":{"1min":0.49399999999999999,"5min":0.46899999999999997,"15min":0.38900000000000001},"max":{"1min":1.335,"5min":1.5129999999999999,"15min":4.3179999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.032,"15min":1.075},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.274,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1850000000000001,"15min":1.2649999999999999},"min":{"1min":0.79600000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.907,"5min":1.907,"15min":3.1659999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.202,"5min":1.222,"15min":1.2989999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.492,"5min":1.823,"15min":2.734},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.254,"15min":1.5129999999999999},"min":{"1min":0.83299999999999996,"5min":0.72399999999999998,"15min":0.67100000000000004},"max":{"1min":1.784,"5min":1.921,"15min":8.4730000000000008},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.278,"15min":1.498},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.8320000000000001,"5min":1.871,"15min":8.8040000000000003},"last":0.68700000000000006}]},{"osd":6,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3049999999999999,"15min":1.4059999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7529999999999999,"5min":1.9119999999999999,"15min":3.984},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.373,"5min":1.329,"15min":1.409},"min":{"1min":0.81599999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":3.2450000000000001},"last":0.875}]},{"osd":7,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.298,"15min":1.4550000000000001},"min":{"1min":0.81399999999999995,"5min":0.80700000000000005,"15min":0.69199999999999995},"max":{"1min":1.853,"5min":1.853,"15min":4.3609999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.391,"15min":1.5349999999999999},"min":{"1min":0.74299999999999999,"5min":0.72799999999999998,"15min":0.70499999999999996},"max":{"1min":1.903,"5min":2.8780000000000001,"15min":5.3840000000000003},"last":0.74299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1875968,"data_stored":1168640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.872,"15min":0.81899999999999995},"min":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.29399999999999998},"max":{"1min":1.208,"5min":1.3680000000000001,"15min":1.6220000000000001},"last":0.996},{"interface":"front","average":{"1min":1.02,"5min":0.95399999999999996,"15min":0.96799999999999997},"min":{"1min":0.71699999999999997,"5min":0.52000000000000002,"15min":0.44800000000000001},"max":{"1min":1.3939999999999999,"5min":1.722,"15min":2.3340000000000001},"last":1.048}]},{"osd":2,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.93500000000000005,"15min":0.93200000000000005},"min":{"1min":0.61899999999999999,"5min":0.42799999999999999,"15min":0.37},"max":{"1min":1.5860000000000001,"5min":1.6910000000000001,"15min":1.762},"last":0.87},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.92600000000000005,"15min":0.98299999999999998},"min":{"1min":0.65800000000000003,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":1.3400000000000001,"5min":1.613,"15min":1.879},"last":1.3400000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91800000000000004,"15min":0.93899999999999995},"min":{"1min":0.57399999999999995,"5min":0.48399999999999999,"15min":0.34200000000000003},"max":{"1min":1.55,"5min":1.72,"15min":1.728},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.194,"5min":1.2050000000000001,"15min":1.1850000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.573,"5min":1.8740000000000001,"15min":2.7679999999999998},"last":1.1719999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.171,"15min":1.1599999999999999},"min":{"1min":0.85199999999999998,"5min":0.76300000000000001,"15min":0.65300000000000002},"max":{"1min":1.496,"5min":2.0489999999999999,"15min":2.2559999999999998},"last":1.204},{"interface":"front","average":{"1min":1.286,"5min":1.2709999999999999,"15min":1.2829999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.59399999999999997},"max":{"1min":1.7190000000000001,"5min":1.8859999999999999,"15min":2.867},"last":1.095}]},{"osd":5,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.2529999999999999,"15min":1.2629999999999999},"min":{"1min":0.98799999999999999,"5min":0.79600000000000004,"15min":0.67900000000000005},"max":{"1min":1.5629999999999999,"5min":1.98,"15min":2.7280000000000002},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.2170000000000001,"15min":1.232},"min":{"1min":0.95499999999999996,"5min":0.74399999999999999,"15min":0.67600000000000005},"max":{"1min":1.579,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.1339999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.371,"15min":1.3600000000000001},"min":{"1min":1.022,"5min":0.79900000000000004,"15min":0.72499999999999998},"max":{"1min":1.865,"5min":1.9399999999999999,"15min":2.6739999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3120000000000001,"15min":1.3540000000000001},"min":{"1min":0.97799999999999998,"5min":0.88200000000000001,"15min":0.64200000000000002},"max":{"1min":1.6779999999999999,"5min":2.1280000000000001,"15min":2.8260000000000001},"last":1.258}]},{"osd":7,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.375,"15min":1.407},"min":{"1min":0.92900000000000005,"5min":0.75,"15min":0.75},"max":{"1min":1.8169999999999999,"5min":2.0150000000000001,"15min":4.5670000000000002},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.337,"15min":1.359},"min":{"1min":0.95099999999999996,"5min":0.85299999999999998,"15min":0.70199999999999996},"max":{"1min":1.7669999999999999,"5min":2.6070000000000002,"15min":5.4450000000000003},"last":1.502}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:39:14.654 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:39:14.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.900+0000 7f5c4b490640 1 Processor -- start 2023-11-08T16:39:14.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.900+0000 7f5c4b490640 1 -- start start 2023-11-08T16:39:14.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.902+0000 7f5c4b490640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5c4415d2b0 0x7f5c44161710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.902+0000 7f5c4b490640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c44161c50 con 0x7f5c440ccf00 2023-11-08T16:39:14.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.902+0000 7f5c4b490640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c4406b980 con 0x7f5c440c9060 2023-11-08T16:39:14.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.902+0000 7f5c4b490640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c4415d690 con 0x7f5c4415d2b0 2023-11-08T16:39:14.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.902+0000 7f5c49a2d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5c440ccf00 0x7f5c4406ca20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34222/0 (socket says 172.21.15.81:34222) 2023-11-08T16:39:14.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.902+0000 7f5c49a2d640 1 -- 172.21.15.81:0/593393980 learned_addr learned my addr 172.21.15.81:0/593393980 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:14.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.902+0000 7f5c4922c640 1 --2- 172.21.15.81:0/593393980 >> v2:172.21.15.93:3300/0 conn(0x7f5c4415d2b0 0x7f5c44161710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697800909 0 0) 0x7f5c3c001490 con 0x7f5c440ccf00 2023-11-08T16:39:14.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c30003520 con 0x7f5c440ccf00 2023-11-08T16:39:14.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 784475561 0 0) 0x7f5c30003520 con 0x7f5c440ccf00 2023-11-08T16:39:14.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 >> v2:172.21.15.93:3300/0 conn(0x7f5c4415d2b0 msgr2=0x7f5c44161710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:14.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 --2- 172.21.15.81:0/593393980 >> v2:172.21.15.93:3300/0 conn(0x7f5c4415d2b0 0x7f5c44161710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 >> v1:172.21.15.93:6789/0 conn(0x7f5c440c9060 legacy=0x7f5c440cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4406bc60 con 0x7f5c440ccf00 2023-11-08T16:39:14.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f5c3c001b80 con 0x7f5c440ccf00 2023-11-08T16:39:14.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 >> v1:172.21.15.81:6789/0 conn(0x7f5c440ccf00 legacy=0x7f5c4406ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 --2- 172.21.15.81:0/593393980 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c30003aa0 0x7f5c30003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 --2- 172.21.15.81:0/593393980 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c300047c0 0x7f5c30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 --2- 172.21.15.81:0/593393980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c300052a0 0x7f5c3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4415d690 con 0x7f5c300052a0 2023-11-08T16:39:14.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3000c150 con 0x7f5c30003aa0 2023-11-08T16:39:14.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3000c310 con 0x7f5c300047c0 2023-11-08T16:39:14.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.903+0000 7f5c49a2d640 1 --2- 172.21.15.81:0/593393980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c300052a0 0x7f5c3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.904+0000 7f5c4922c640 1 --2- 172.21.15.81:0/593393980 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c30003aa0 0x7f5c30003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.904+0000 7f5c49a2d640 1 -- 172.21.15.81:0/593393980 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c300047c0 msgr2=0x7f5c30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:14.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.904+0000 7f5c49a2d640 1 --2- 172.21.15.81:0/593393980 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c300047c0 0x7f5c30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.904+0000 7f5c49a2d640 1 -- 172.21.15.81:0/593393980 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c30003aa0 msgr2=0x7f5c30003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:14.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.904+0000 7f5c49a2d640 1 --2- 172.21.15.81:0/593393980 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c30003aa0 0x7f5c30003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.904+0000 7f5c49a2d640 1 -- 172.21.15.81:0/593393980 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c30003520 con 0x7f5c300052a0 2023-11-08T16:39:14.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.904+0000 7f5c4922c640 1 --2- 172.21.15.81:0/593393980 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c30003aa0 0x7f5c30003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:14.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.905+0000 7f5c49a2d640 1 -- 172.21.15.81:0/593393980 reap_dead start 2023-11-08T16:39:14.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.905+0000 7f5c49a2d640 1 --2- 172.21.15.81:0/593393980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c300052a0 0x7f5c3000b980 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c002230 tx=0x7f5c300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e13bfbb6cea675 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:14.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.905+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c0072d0 con 0x7f5c300052a0 2023-11-08T16:39:14.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.905+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c02f070 con 0x7f5c300052a0 2023-11-08T16:39:14.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.905+0000 7f5c3bfff640 1 -- 172.21.15.81:0/593393980 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c007670 con 0x7f5c300052a0 2023-11-08T16:39:14.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.906+0000 7f5c4b490640 1 -- 172.21.15.81:0/593393980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c300052a0 msgr2=0x7f5c3000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.906+0000 7f5c4b490640 1 --2- 172.21.15.81:0/593393980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c300052a0 0x7f5c3000b980 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c002230 tx=0x7f5c300057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.906+0000 7f5c4b490640 1 -- 172.21.15.81:0/593393980 shutdown_connections 2023-11-08T16:39:14.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.906+0000 7f5c4b490640 1 --2- 172.21.15.81:0/593393980 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c300052a0 0x7f5c3000b980 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.906+0000 7f5c4b490640 1 -- 172.21.15.81:0/593393980 >> 172.21.15.81:0/593393980 conn(0x7f5c440cb490 msgr2=0x7f5c440cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:14.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.906+0000 7f5c4b490640 1 -- 172.21.15.81:0/593393980 shutdown_connections 2023-11-08T16:39:14.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.906+0000 7f5c4b490640 1 -- 172.21.15.81:0/593393980 wait complete. 2023-11-08T16:39:14.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.908+0000 7f5c4b490640 1 Processor -- start 2023-11-08T16:39:14.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.908+0000 7f5c4b490640 1 -- start start 2023-11-08T16:39:14.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c4b490640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c300052a0 0x7f5c440b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c4b490640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c44149c10 0x7f5c44148e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c4b490640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c440b1b00 0x7f5c441493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c4b490640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4415ff50 con 0x7f5c440b1b00 2023-11-08T16:39:14.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c4b490640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4415f2d0 con 0x7f5c300052a0 2023-11-08T16:39:14.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c4b490640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c4415f5b0 con 0x7f5c44149c10 2023-11-08T16:39:14.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c49a2d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c440b1b00 0x7f5c441493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c49a2d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c440b1b00 0x7f5c441493a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42654/0 (socket says 172.21.15.81:42654) 2023-11-08T16:39:14.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c49a2d640 1 -- 172.21.15.81:0/4056263241 learned_addr learned my addr 172.21.15.81:0/4056263241 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:14.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c4922c640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c300052a0 0x7f5c440b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c49a2d640 1 -- 172.21.15.81:0/4056263241 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c44149c10 msgr2=0x7f5c44148e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c49a2d640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c44149c10 0x7f5c44148e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c49a2d640 1 -- 172.21.15.81:0/4056263241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c300052a0 msgr2=0x7f5c440b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:14.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c49a2d640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c300052a0 0x7f5c440b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:14.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c49a2d640 1 -- 172.21.15.81:0/4056263241 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4415f890 con 0x7f5c440b1b00 2023-11-08T16:39:14.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.909+0000 7f5c4922c640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c300052a0 0x7f5c440b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:14.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.910+0000 7f5c49a2d640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c440b1b00 0x7f5c441493a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00a820 tx=0x7f5c3c001e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.910+0000 7f5c39ffb640 1 -- 172.21.15.81:0/4056263241 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c001ad0 con 0x7f5c440b1b00 2023-11-08T16:39:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.910+0000 7f5c39ffb640 1 -- 172.21.15.81:0/4056263241 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3c004b60 con 0x7f5c440b1b00 2023-11-08T16:39:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.910+0000 7f5c39ffb640 1 -- 172.21.15.81:0/4056263241 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3c03e460 con 0x7f5c440b1b00 2023-11-08T16:39:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.910+0000 7f5c4b490640 1 -- 172.21.15.81:0/4056263241 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4415fb70 con 0x7f5c440b1b00 2023-11-08T16:39:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.910+0000 7f5c4b490640 1 -- 172.21.15.81:0/4056263241 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c440c80f0 con 0x7f5c440b1b00 2023-11-08T16:39:14.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.911+0000 7f5c4b490640 1 -- 172.21.15.81:0/4056263241 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c44160090 con 0x7f5c440b1b00 2023-11-08T16:39:14.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.912+0000 7f5c39ffb640 1 -- 172.21.15.81:0/4056263241 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c3c031030 con 0x7f5c440b1b00 2023-11-08T16:39:14.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.913+0000 7f5c39ffb640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c2806aea0 0x7f5c2806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:14.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.913+0000 7f5c39ffb640 1 -- 172.21.15.81:0/4056263241 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c3c0aa730 con 0x7f5c440b1b00 2023-11-08T16:39:14.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.914+0000 7f5c4922c640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c2806aea0 0x7f5c2806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:14.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.914+0000 7f5c4922c640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c2806aea0 0x7f5c2806d360 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009aa0 tx=0x7f5c40008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:14.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:14.918+0000 7f5c39ffb640 1 -- 172.21.15.81:0/4056263241 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5c44160090 con 0x7f5c440b1b00 2023-11-08T16:39:15.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.092+0000 7f5c4b490640 1 -- 172.21.15.81:0/4056263241 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5c4406c220 con 0x7f5c2806aea0 2023-11-08T16:39:15.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.112+0000 7f5c39ffb640 1 -- 172.21.15.81:0/4056263241 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287546 (secure 0 0 0) 0x7f5c400119a0 con 0x7f5c2806aea0 2023-11-08T16:39:15.116 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:15.119 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:39:15.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 -- 172.21.15.81:0/4056263241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c2806aea0 msgr2=0x7f5c2806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c2806aea0 0x7f5c2806d360 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5c40009aa0 tx=0x7f5c40008040 comp rx=0 tx=0).stop 2023-11-08T16:39:15.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 -- 172.21.15.81:0/4056263241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c440b1b00 msgr2=0x7f5c441493a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c440b1b00 0x7f5c441493a0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c00a820 tx=0x7f5c3c001e40 comp rx=0 tx=0).stop 2023-11-08T16:39:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 -- 172.21.15.81:0/4056263241 shutdown_connections 2023-11-08T16:39:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5c2806aea0 0x7f5c2806d360 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5c440b1b00 0x7f5c441493a0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5c44149c10 0x7f5c44148e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 --2- 172.21.15.81:0/4056263241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c300052a0 0x7f5c440b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 -- 172.21.15.81:0/4056263241 >> 172.21.15.81:0/4056263241 conn(0x7f5c440cb490 msgr2=0x7f5c440af440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:15.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 -- 172.21.15.81:0/4056263241 shutdown_connections 2023-11-08T16:39:15.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.118+0000 7f5c4b490640 1 -- 172.21.15.81:0/4056263241 wait complete. 2023-11-08T16:39:15.138 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2023-11-08T16:39:14.764277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":138,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260564,"kb_used_data":11084,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714156,"statfs":{"total":773094113280,"available":772827295744,"internally_reserved":0,"allocated":11350016,"data_stored":5812088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001837"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.245630+0000","last_change":"2023-11-08T16:39:10.241105+0000","last_active":"2023-11-08T16:39:10.245630+0000","last_peered":"2023-11-08T16:39:10.245630+0000","last_clean":"2023-11-08T16:39:10.245630+0000","last_became_active":"2023-11-08T16:39:10.240677+0000","last_became_peered":"2023-11-08T16:39:10.240677+0000","last_unstale":"2023-11-08T16:39:10.245630+0000","last_undegraded":"2023-11-08T16:39:10.245630+0000","last_fullsized":"2023-11-08T16:39:10.245630+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:38.165884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225035+0000","last_change":"2023-11-08T16:39:10.225715+0000","last_active":"2023-11-08T16:39:11.225035+0000","last_peered":"2023-11-08T16:39:11.225035+0000","last_clean":"2023-11-08T16:39:11.225035+0000","last_became_active":"2023-11-08T16:39:10.225346+0000","last_became_peered":"2023-11-08T16:39:10.225346+0000","last_unstale":"2023-11-08T16:39:11.225035+0000","last_undegraded":"2023-11-08T16:39:11.225035+0000","last_fullsized":"2023-11-08T16:39:11.225035+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:57:03.464872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.232685+0000","last_change":"2023-11-08T16:39:10.226516+0000","last_active":"2023-11-08T16:39:10.232685+0000","last_peered":"2023-11-08T16:39:10.232685+0000","last_clean":"2023-11-08T16:39:10.232685+0000","last_became_active":"2023-11-08T16:39:10.225660+0000","last_became_peered":"2023-11-08T16:39:10.225660+0000","last_unstale":"2023-11-08T16:39:10.232685+0000","last_undegraded":"2023-11-08T16:39:10.232685+0000","last_fullsized":"2023-11-08T16:39:10.232685+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:32.776251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.243310+0000","last_change":"2023-11-08T16:39:10.230188+0000","last_active":"2023-11-08T16:39:10.243310+0000","last_peered":"2023-11-08T16:39:10.243310+0000","last_clean":"2023-11-08T16:39:10.243310+0000","last_became_active":"2023-11-08T16:39:10.229930+0000","last_became_peered":"2023-11-08T16:39:10.229930+0000","last_unstale":"2023-11-08T16:39:10.243310+0000","last_undegraded":"2023-11-08T16:39:10.243310+0000","last_fullsized":"2023-11-08T16:39:10.243310+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:25.931399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.226744+0000","last_change":"2023-11-08T16:39:10.222049+0000","last_active":"2023-11-08T16:39:11.226744+0000","last_peered":"2023-11-08T16:39:11.226744+0000","last_clean":"2023-11-08T16:39:11.226744+0000","last_became_active":"2023-11-08T16:39:10.221396+0000","last_became_peered":"2023-11-08T16:39:10.221396+0000","last_unstale":"2023-11-08T16:39:11.226744+0000","last_undegraded":"2023-11-08T16:39:11.226744+0000","last_fullsized":"2023-11-08T16:39:11.226744+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:19:02.824312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.239537+0000","last_change":"2023-11-08T16:39:10.234031+0000","last_active":"2023-11-08T16:39:10.239537+0000","last_peered":"2023-11-08T16:39:10.239537+0000","last_clean":"2023-11-08T16:39:10.239537+0000","last_became_active":"2023-11-08T16:39:10.232189+0000","last_became_peered":"2023-11-08T16:39:10.232189+0000","last_unstale":"2023-11-08T16:39:10.239537+0000","last_undegraded":"2023-11-08T16:39:10.239537+0000","last_fullsized":"2023-11-08T16:39:10.239537+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:07:01.023231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228093+0000","last_change":"2023-11-08T16:39:10.227019+0000","last_active":"2023-11-08T16:39:11.228093+0000","last_peered":"2023-11-08T16:39:11.228093+0000","last_clean":"2023-11-08T16:39:11.228093+0000","last_became_active":"2023-11-08T16:39:10.226700+0000","last_became_peered":"2023-11-08T16:39:10.226700+0000","last_unstale":"2023-11-08T16:39:11.228093+0000","last_undegraded":"2023-11-08T16:39:11.228093+0000","last_fullsized":"2023-11-08T16:39:11.228093+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:32.776251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.242242+0000","last_change":"2023-11-08T16:39:10.236830+0000","last_active":"2023-11-08T16:39:10.242242+0000","last_peered":"2023-11-08T16:39:10.242242+0000","last_clean":"2023-11-08T16:39:10.242242+0000","last_became_active":"2023-11-08T16:39:10.236497+0000","last_became_peered":"2023-11-08T16:39:10.236497+0000","last_unstale":"2023-11-08T16:39:10.242242+0000","last_undegraded":"2023-11-08T16:39:10.242242+0000","last_fullsized":"2023-11-08T16:39:10.242242+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:41.564607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225939+0000","last_change":"2023-11-08T16:39:10.222150+0000","last_active":"2023-11-08T16:39:11.225939+0000","last_peered":"2023-11-08T16:39:11.225939+0000","last_clean":"2023-11-08T16:39:11.225939+0000","last_became_active":"2023-11-08T16:39:10.221845+0000","last_became_peered":"2023-11-08T16:39:10.221845+0000","last_unstale":"2023-11-08T16:39:11.225939+0000","last_undegraded":"2023-11-08T16:39:11.225939+0000","last_fullsized":"2023-11-08T16:39:11.225939+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:58:52.139332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.232980+0000","last_change":"2023-11-08T16:39:10.228413+0000","last_active":"2023-11-08T16:39:10.232980+0000","last_peered":"2023-11-08T16:39:10.232980+0000","last_clean":"2023-11-08T16:39:10.232980+0000","last_became_active":"2023-11-08T16:39:10.228115+0000","last_became_peered":"2023-11-08T16:39:10.228115+0000","last_unstale":"2023-11-08T16:39:10.232980+0000","last_undegraded":"2023-11-08T16:39:10.232980+0000","last_fullsized":"2023-11-08T16:39:10.232980+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:43.946508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224756+0000","last_change":"2023-11-08T16:39:10.223495+0000","last_active":"2023-11-08T16:39:11.224756+0000","last_peered":"2023-11-08T16:39:11.224756+0000","last_clean":"2023-11-08T16:39:11.224756+0000","last_became_active":"2023-11-08T16:39:10.223208+0000","last_became_peered":"2023-11-08T16:39:10.223208+0000","last_unstale":"2023-11-08T16:39:11.224756+0000","last_undegraded":"2023-11-08T16:39:11.224756+0000","last_fullsized":"2023-11-08T16:39:11.224756+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:22.834895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.233991+0000","last_change":"2023-11-08T16:39:10.229501+0000","last_active":"2023-11-08T16:39:10.233991+0000","last_peered":"2023-11-08T16:39:10.233991+0000","last_clean":"2023-11-08T16:39:10.233991+0000","last_became_active":"2023-11-08T16:39:10.228488+0000","last_became_peered":"2023-11-08T16:39:10.228488+0000","last_unstale":"2023-11-08T16:39:10.233991+0000","last_undegraded":"2023-11-08T16:39:10.233991+0000","last_fullsized":"2023-11-08T16:39:10.233991+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:46:26.608103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.242204+0000","last_change":"2023-11-08T16:39:10.236629+0000","last_active":"2023-11-08T16:39:10.242204+0000","last_peered":"2023-11-08T16:39:10.242204+0000","last_clean":"2023-11-08T16:39:10.242204+0000","last_became_active":"2023-11-08T16:39:10.236317+0000","last_became_peered":"2023-11-08T16:39:10.236317+0000","last_unstale":"2023-11-08T16:39:10.242204+0000","last_undegraded":"2023-11-08T16:39:10.242204+0000","last_fullsized":"2023-11-08T16:39:10.242204+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:40:51.892821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.223631+0000","last_change":"2023-11-08T16:39:10.242802+0000","last_active":"2023-11-08T16:39:11.223631+0000","last_peered":"2023-11-08T16:39:11.223631+0000","last_clean":"2023-11-08T16:39:11.223631+0000","last_became_active":"2023-11-08T16:39:10.242496+0000","last_became_peered":"2023-11-08T16:39:10.242496+0000","last_unstale":"2023-11-08T16:39:11.223631+0000","last_undegraded":"2023-11-08T16:39:11.223631+0000","last_fullsized":"2023-11-08T16:39:11.223631+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:53.831527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.242331+0000","last_change":"2023-11-08T16:39:10.239182+0000","last_active":"2023-11-08T16:39:10.242331+0000","last_peered":"2023-11-08T16:39:10.242331+0000","last_clean":"2023-11-08T16:39:10.242331+0000","last_became_active":"2023-11-08T16:39:10.238590+0000","last_became_peered":"2023-11-08T16:39:10.238590+0000","last_unstale":"2023-11-08T16:39:10.242331+0000","last_undegraded":"2023-11-08T16:39:10.242331+0000","last_fullsized":"2023-11-08T16:39:10.242331+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:14:13.821481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.231348+0000","last_change":"2023-11-08T16:39:10.219542+0000","last_active":"2023-11-08T16:39:10.231348+0000","last_peered":"2023-11-08T16:39:10.231348+0000","last_clean":"2023-11-08T16:39:10.231348+0000","last_became_active":"2023-11-08T16:39:10.218943+0000","last_became_peered":"2023-11-08T16:39:10.218943+0000","last_unstale":"2023-11-08T16:39:10.231348+0000","last_undegraded":"2023-11-08T16:39:10.231348+0000","last_fullsized":"2023-11-08T16:39:10.231348+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:34:32.079590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.238181+0000","last_change":"2023-11-08T16:39:10.227281+0000","last_active":"2023-11-08T16:39:10.238181+0000","last_peered":"2023-11-08T16:39:10.238181+0000","last_clean":"2023-11-08T16:39:10.238181+0000","last_became_active":"2023-11-08T16:39:10.226996+0000","last_became_peered":"2023-11-08T16:39:10.226996+0000","last_unstale":"2023-11-08T16:39:10.238181+0000","last_undegraded":"2023-11-08T16:39:10.238181+0000","last_fullsized":"2023-11-08T16:39:10.238181+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:24:05.328966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.230611+0000","last_change":"2023-11-08T16:39:10.222675+0000","last_active":"2023-11-08T16:39:10.230611+0000","last_peered":"2023-11-08T16:39:10.230611+0000","last_clean":"2023-11-08T16:39:10.230611+0000","last_became_active":"2023-11-08T16:39:10.222141+0000","last_became_peered":"2023-11-08T16:39:10.222141+0000","last_unstale":"2023-11-08T16:39:10.230611+0000","last_undegraded":"2023-11-08T16:39:10.230611+0000","last_fullsized":"2023-11-08T16:39:10.230611+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:22.834895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225095+0000","last_change":"2023-11-08T16:39:10.242809+0000","last_active":"2023-11-08T16:39:11.225095+0000","last_peered":"2023-11-08T16:39:11.225095+0000","last_clean":"2023-11-08T16:39:11.225095+0000","last_became_active":"2023-11-08T16:39:10.241875+0000","last_became_peered":"2023-11-08T16:39:10.241875+0000","last_unstale":"2023-11-08T16:39:11.225095+0000","last_undegraded":"2023-11-08T16:39:11.225095+0000","last_fullsized":"2023-11-08T16:39:11.225095+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:38.165884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.243393+0000","last_change":"2023-11-08T16:39:10.235161+0000","last_active":"2023-11-08T16:39:10.243393+0000","last_peered":"2023-11-08T16:39:10.243393+0000","last_clean":"2023-11-08T16:39:10.243393+0000","last_became_active":"2023-11-08T16:39:10.234261+0000","last_became_peered":"2023-11-08T16:39:10.234261+0000","last_unstale":"2023-11-08T16:39:10.243393+0000","last_undegraded":"2023-11-08T16:39:10.243393+0000","last_fullsized":"2023-11-08T16:39:10.243393+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:52.527800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.226572+0000","last_change":"2023-11-08T16:39:10.222066+0000","last_active":"2023-11-08T16:39:11.226572+0000","last_peered":"2023-11-08T16:39:11.226572+0000","last_clean":"2023-11-08T16:39:11.226572+0000","last_became_active":"2023-11-08T16:39:10.221554+0000","last_became_peered":"2023-11-08T16:39:10.221554+0000","last_unstale":"2023-11-08T16:39:11.226572+0000","last_undegraded":"2023-11-08T16:39:11.226572+0000","last_fullsized":"2023-11-08T16:39:11.226572+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:07:01.023231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.223837+0000","last_change":"2023-11-08T16:39:10.242700+0000","last_active":"2023-11-08T16:39:11.223837+0000","last_peered":"2023-11-08T16:39:11.223837+0000","last_clean":"2023-11-08T16:39:11.223837+0000","last_became_active":"2023-11-08T16:39:10.242404+0000","last_became_peered":"2023-11-08T16:39:10.242404+0000","last_unstale":"2023-11-08T16:39:11.223837+0000","last_undegraded":"2023-11-08T16:39:11.223837+0000","last_fullsized":"2023-11-08T16:39:11.223837+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:51:42.471499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.288880+0000","last_change":"2023-11-08T16:39:10.221382+0000","last_active":"2023-11-08T16:39:11.288880+0000","last_peered":"2023-11-08T16:39:11.288880+0000","last_clean":"2023-11-08T16:39:11.288880+0000","last_became_active":"2023-11-08T16:39:10.220883+0000","last_became_peered":"2023-11-08T16:39:10.220883+0000","last_unstale":"2023-11-08T16:39:11.288880+0000","last_undegraded":"2023-11-08T16:39:11.288880+0000","last_fullsized":"2023-11-08T16:39:11.288880+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:47:12.014262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.232634+0000","last_change":"2023-11-08T16:39:10.226047+0000","last_active":"2023-11-08T16:39:10.232634+0000","last_peered":"2023-11-08T16:39:10.232634+0000","last_clean":"2023-11-08T16:39:10.232634+0000","last_became_active":"2023-11-08T16:39:10.225706+0000","last_became_peered":"2023-11-08T16:39:10.225706+0000","last_unstale":"2023-11-08T16:39:10.232634+0000","last_undegraded":"2023-11-08T16:39:10.232634+0000","last_fullsized":"2023-11-08T16:39:10.232634+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:49:36.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225473+0000","last_change":"2023-11-08T16:39:10.211514+0000","last_active":"2023-11-08T16:39:11.225473+0000","last_peered":"2023-11-08T16:39:11.225473+0000","last_clean":"2023-11-08T16:39:11.225473+0000","last_became_active":"2023-11-08T16:39:10.210746+0000","last_became_peered":"2023-11-08T16:39:10.210746+0000","last_unstale":"2023-11-08T16:39:11.225473+0000","last_undegraded":"2023-11-08T16:39:11.225473+0000","last_fullsized":"2023-11-08T16:39:11.225473+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:47:12.014262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224642+0000","last_change":"2023-11-08T16:39:10.243326+0000","last_active":"2023-11-08T16:39:11.224642+0000","last_peered":"2023-11-08T16:39:11.224642+0000","last_clean":"2023-11-08T16:39:11.224642+0000","last_became_active":"2023-11-08T16:39:10.242532+0000","last_became_peered":"2023-11-08T16:39:10.242532+0000","last_unstale":"2023-11-08T16:39:11.224642+0000","last_undegraded":"2023-11-08T16:39:11.224642+0000","last_fullsized":"2023-11-08T16:39:11.224642+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:39:32.776251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.232742+0000","last_change":"2023-11-08T16:39:10.225108+0000","last_active":"2023-11-08T16:39:10.232742+0000","last_peered":"2023-11-08T16:39:10.232742+0000","last_clean":"2023-11-08T16:39:10.232742+0000","last_became_active":"2023-11-08T16:39:10.222292+0000","last_became_peered":"2023-11-08T16:39:10.222292+0000","last_unstale":"2023-11-08T16:39:10.232742+0000","last_undegraded":"2023-11-08T16:39:10.232742+0000","last_fullsized":"2023-11-08T16:39:10.232742+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:41:25.445412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.247598+0000","last_change":"2023-11-08T16:39:10.243222+0000","last_active":"2023-11-08T16:39:10.247598+0000","last_peered":"2023-11-08T16:39:10.247598+0000","last_clean":"2023-11-08T16:39:10.247598+0000","last_became_active":"2023-11-08T16:39:10.242398+0000","last_became_peered":"2023-11-08T16:39:10.242398+0000","last_unstale":"2023-11-08T16:39:10.247598+0000","last_undegraded":"2023-11-08T16:39:10.247598+0000","last_fullsized":"2023-11-08T16:39:10.247598+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:01.474655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228131+0000","last_change":"2023-11-08T16:39:10.220276+0000","last_active":"2023-11-08T16:39:11.228131+0000","last_peered":"2023-11-08T16:39:11.228131+0000","last_clean":"2023-11-08T16:39:11.228131+0000","last_became_active":"2023-11-08T16:39:10.219959+0000","last_became_peered":"2023-11-08T16:39:10.219959+0000","last_unstale":"2023-11-08T16:39:11.228131+0000","last_undegraded":"2023-11-08T16:39:11.228131+0000","last_fullsized":"2023-11-08T16:39:11.228131+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:50:22.834895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224079+0000","last_change":"2023-11-08T16:39:10.216773+0000","last_active":"2023-11-08T16:39:11.224079+0000","last_peered":"2023-11-08T16:39:11.224079+0000","last_clean":"2023-11-08T16:39:11.224079+0000","last_became_active":"2023-11-08T16:39:10.215761+0000","last_became_peered":"2023-11-08T16:39:10.215761+0000","last_unstale":"2023-11-08T16:39:11.224079+0000","last_undegraded":"2023-11-08T16:39:11.224079+0000","last_fullsized":"2023-11-08T16:39:11.224079+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:49:36.304794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.226530+0000","last_change":"2023-11-08T16:39:10.234935+0000","last_active":"2023-11-08T16:39:11.226530+0000","last_peered":"2023-11-08T16:39:11.226530+0000","last_clean":"2023-11-08T16:39:11.226530+0000","last_became_active":"2023-11-08T16:39:10.232089+0000","last_became_peered":"2023-11-08T16:39:10.232089+0000","last_unstale":"2023-11-08T16:39:11.226530+0000","last_undegraded":"2023-11-08T16:39:11.226530+0000","last_fullsized":"2023-11-08T16:39:11.226530+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:46:23.223886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.231370+0000","last_change":"2023-11-08T16:39:10.219794+0000","last_active":"2023-11-08T16:39:10.231370+0000","last_peered":"2023-11-08T16:39:10.231370+0000","last_clean":"2023-11-08T16:39:10.231370+0000","last_became_active":"2023-11-08T16:39:10.218461+0000","last_became_peered":"2023-11-08T16:39:10.218461+0000","last_unstale":"2023-11-08T16:39:10.231370+0000","last_undegraded":"2023-11-08T16:39:10.231370+0000","last_fullsized":"2023-11-08T16:39:10.231370+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:51:04.503788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.237432+0000","last_change":"2023-11-08T16:39:10.225266+0000","last_active":"2023-11-08T16:39:10.237432+0000","last_peered":"2023-11-08T16:39:10.237432+0000","last_clean":"2023-11-08T16:39:10.237432+0000","last_became_active":"2023-11-08T16:39:10.224993+0000","last_became_peered":"2023-11-08T16:39:10.224993+0000","last_unstale":"2023-11-08T16:39:10.237432+0000","last_undegraded":"2023-11-08T16:39:10.237432+0000","last_fullsized":"2023-11-08T16:39:10.237432+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:05:29.168734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"102'4","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289312+0000","last_change":"2023-11-08T16:39:09.187878+0000","last_active":"2023-11-08T16:39:11.289312+0000","last_peered":"2023-11-08T16:39:11.289312+0000","last_clean":"2023-11-08T16:39:11.289312+0000","last_became_active":"2023-11-08T16:39:09.187197+0000","last_became_peered":"2023-11-08T16:39:09.187197+0000","last_unstale":"2023-11-08T16:39:11.289312+0000","last_undegraded":"2023-11-08T16:39:11.289312+0000","last_fullsized":"2023-11-08T16:39:11.289312+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:00.469842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.236142+0000","last_change":"2023-11-08T16:39:10.224562+0000","last_active":"2023-11-08T16:39:10.236142+0000","last_peered":"2023-11-08T16:39:10.236142+0000","last_clean":"2023-11-08T16:39:10.236142+0000","last_became_active":"2023-11-08T16:39:10.224266+0000","last_became_peered":"2023-11-08T16:39:10.224266+0000","last_unstale":"2023-11-08T16:39:10.236142+0000","last_undegraded":"2023-11-08T16:39:10.236142+0000","last_fullsized":"2023-11-08T16:39:10.236142+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:16:53.407457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224991+0000","last_change":"2023-11-08T16:39:09.187022+0000","last_active":"2023-11-08T16:39:11.224991+0000","last_peered":"2023-11-08T16:39:11.224991+0000","last_clean":"2023-11-08T16:39:11.224991+0000","last_became_active":"2023-11-08T16:39:09.185770+0000","last_became_peered":"2023-11-08T16:39:09.185770+0000","last_unstale":"2023-11-08T16:39:11.224991+0000","last_undegraded":"2023-11-08T16:39:11.224991+0000","last_fullsized":"2023-11-08T16:39:11.224991+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:58.270995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228573+0000","last_change":"2023-11-08T16:39:10.230250+0000","last_active":"2023-11-08T16:39:11.228573+0000","last_peered":"2023-11-08T16:39:11.228573+0000","last_clean":"2023-11-08T16:39:11.228573+0000","last_became_active":"2023-11-08T16:39:10.229969+0000","last_became_peered":"2023-11-08T16:39:10.229969+0000","last_unstale":"2023-11-08T16:39:11.228573+0000","last_undegraded":"2023-11-08T16:39:11.228573+0000","last_fullsized":"2023-11-08T16:39:11.228573+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:46:38.004348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228055+0000","last_change":"2023-11-08T16:39:09.172522+0000","last_active":"2023-11-08T16:39:11.228055+0000","last_peered":"2023-11-08T16:39:11.228055+0000","last_clean":"2023-11-08T16:39:11.228055+0000","last_became_active":"2023-11-08T16:39:09.172097+0000","last_became_peered":"2023-11-08T16:39:09.172097+0000","last_unstale":"2023-11-08T16:39:11.228055+0000","last_undegraded":"2023-11-08T16:39:11.228055+0000","last_fullsized":"2023-11-08T16:39:11.228055+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:47:11.009449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.219751+0000","last_change":"2023-11-08T16:39:10.214575+0000","last_active":"2023-11-08T16:39:10.219751+0000","last_peered":"2023-11-08T16:39:10.219751+0000","last_clean":"2023-11-08T16:39:10.219751+0000","last_became_active":"2023-11-08T16:39:10.214052+0000","last_became_peered":"2023-11-08T16:39:10.214052+0000","last_unstale":"2023-11-08T16:39:10.219751+0000","last_undegraded":"2023-11-08T16:39:10.219751+0000","last_fullsized":"2023-11-08T16:39:10.219751+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:41:07.662978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289640+0000","last_change":"2023-11-08T16:39:09.180548+0000","last_active":"2023-11-08T16:39:11.289640+0000","last_peered":"2023-11-08T16:39:11.289640+0000","last_clean":"2023-11-08T16:39:11.289640+0000","last_became_active":"2023-11-08T16:39:09.179872+0000","last_became_peered":"2023-11-08T16:39:09.179872+0000","last_unstale":"2023-11-08T16:39:11.289640+0000","last_undegraded":"2023-11-08T16:39:11.289640+0000","last_fullsized":"2023-11-08T16:39:11.289640+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:44:57.514158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.240311+0000","last_change":"2023-11-08T16:39:10.232964+0000","last_active":"2023-11-08T16:39:10.240311+0000","last_peered":"2023-11-08T16:39:10.240311+0000","last_clean":"2023-11-08T16:39:10.240311+0000","last_became_active":"2023-11-08T16:39:10.232681+0000","last_became_peered":"2023-11-08T16:39:10.232681+0000","last_unstale":"2023-11-08T16:39:10.240311+0000","last_undegraded":"2023-11-08T16:39:10.240311+0000","last_fullsized":"2023-11-08T16:39:10.240311+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:54:41.044887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.226761+0000","last_change":"2023-11-08T16:39:09.186518+0000","last_active":"2023-11-08T16:39:11.226761+0000","last_peered":"2023-11-08T16:39:11.226761+0000","last_clean":"2023-11-08T16:39:11.226761+0000","last_became_active":"2023-11-08T16:39:09.185088+0000","last_became_peered":"2023-11-08T16:39:09.185088+0000","last_unstale":"2023-11-08T16:39:11.226761+0000","last_undegraded":"2023-11-08T16:39:11.226761+0000","last_fullsized":"2023-11-08T16:39:11.226761+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:54:40.040074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289066+0000","last_change":"2023-11-08T16:39:10.216338+0000","last_active":"2023-11-08T16:39:11.289066+0000","last_peered":"2023-11-08T16:39:11.289066+0000","last_clean":"2023-11-08T16:39:11.289066+0000","last_became_active":"2023-11-08T16:39:10.216020+0000","last_became_peered":"2023-11-08T16:39:10.216020+0000","last_unstale":"2023-11-08T16:39:11.289066+0000","last_undegraded":"2023-11-08T16:39:11.289066+0000","last_fullsized":"2023-11-08T16:39:11.289066+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:59.275808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"102'1","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.243703+0000","last_change":"2023-11-08T16:39:09.169965+0000","last_active":"2023-11-08T16:39:10.243703+0000","last_peered":"2023-11-08T16:39:10.243703+0000","last_clean":"2023-11-08T16:39:10.243703+0000","last_became_active":"2023-11-08T16:39:09.168823+0000","last_became_peered":"2023-11-08T16:39:09.168823+0000","last_unstale":"2023-11-08T16:39:10.243703+0000","last_undegraded":"2023-11-08T16:39:10.243703+0000","last_fullsized":"2023-11-08T16:39:10.243703+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:47:11.009449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.239560+0000","last_change":"2023-11-08T16:39:10.234973+0000","last_active":"2023-11-08T16:39:10.239560+0000","last_peered":"2023-11-08T16:39:10.239560+0000","last_clean":"2023-11-08T16:39:10.239560+0000","last_became_active":"2023-11-08T16:39:10.231757+0000","last_became_peered":"2023-11-08T16:39:10.231757+0000","last_unstale":"2023-11-08T16:39:10.239560+0000","last_undegraded":"2023-11-08T16:39:10.239560+0000","last_fullsized":"2023-11-08T16:39:10.239560+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:00:21.625962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.222563+0000","last_change":"2023-11-08T16:39:09.168677+0000","last_active":"2023-11-08T16:39:10.222563+0000","last_peered":"2023-11-08T16:39:10.222563+0000","last_clean":"2023-11-08T16:39:10.222563+0000","last_became_active":"2023-11-08T16:39:09.165890+0000","last_became_peered":"2023-11-08T16:39:09.165890+0000","last_unstale":"2023-11-08T16:39:10.222563+0000","last_undegraded":"2023-11-08T16:39:10.222563+0000","last_fullsized":"2023-11-08T16:39:10.222563+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:58:51.134519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.230891+0000","last_change":"2023-11-08T16:39:10.226104+0000","last_active":"2023-11-08T16:39:10.230891+0000","last_peered":"2023-11-08T16:39:10.230891+0000","last_clean":"2023-11-08T16:39:10.230891+0000","last_became_active":"2023-11-08T16:39:10.219988+0000","last_became_peered":"2023-11-08T16:39:10.219988+0000","last_unstale":"2023-11-08T16:39:10.230891+0000","last_undegraded":"2023-11-08T16:39:10.230891+0000","last_fullsized":"2023-11-08T16:39:10.230891+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:48:48.116487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.227836+0000","last_change":"2023-11-08T16:39:09.175817+0000","last_active":"2023-11-08T16:39:11.227836+0000","last_peered":"2023-11-08T16:39:11.227836+0000","last_clean":"2023-11-08T16:39:11.227836+0000","last_became_active":"2023-11-08T16:39:09.175306+0000","last_became_peered":"2023-11-08T16:39:09.175306+0000","last_unstale":"2023-11-08T16:39:11.227836+0000","last_undegraded":"2023-11-08T16:39:11.227836+0000","last_fullsized":"2023-11-08T16:39:11.227836+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:54:40.040074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225144+0000","last_change":"2023-11-08T16:39:10.231814+0000","last_active":"2023-11-08T16:39:11.225144+0000","last_peered":"2023-11-08T16:39:11.225144+0000","last_clean":"2023-11-08T16:39:11.225144+0000","last_became_active":"2023-11-08T16:39:10.231528+0000","last_became_peered":"2023-11-08T16:39:10.231528+0000","last_unstale":"2023-11-08T16:39:11.225144+0000","last_undegraded":"2023-11-08T16:39:11.225144+0000","last_fullsized":"2023-11-08T16:39:11.225144+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:46:38.004348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225581+0000","last_change":"2023-11-08T16:39:09.185547+0000","last_active":"2023-11-08T16:39:11.225581+0000","last_peered":"2023-11-08T16:39:11.225581+0000","last_clean":"2023-11-08T16:39:11.225581+0000","last_became_active":"2023-11-08T16:39:09.183003+0000","last_became_peered":"2023-11-08T16:39:09.183003+0000","last_unstale":"2023-11-08T16:39:11.225581+0000","last_undegraded":"2023-11-08T16:39:11.225581+0000","last_fullsized":"2023-11-08T16:39:11.225581+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:54:40.040074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228201+0000","last_change":"2023-11-08T16:39:10.224065+0000","last_active":"2023-11-08T16:39:11.228201+0000","last_peered":"2023-11-08T16:39:11.228201+0000","last_clean":"2023-11-08T16:39:11.228201+0000","last_became_active":"2023-11-08T16:39:10.223779+0000","last_became_peered":"2023-11-08T16:39:10.223779+0000","last_unstale":"2023-11-08T16:39:11.228201+0000","last_undegraded":"2023-11-08T16:39:11.228201+0000","last_fullsized":"2023-11-08T16:39:11.228201+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:53:06.069562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.225950+0000","last_change":"2023-11-08T16:39:09.173054+0000","last_active":"2023-11-08T16:39:10.225950+0000","last_peered":"2023-11-08T16:39:10.225950+0000","last_clean":"2023-11-08T16:39:10.225950+0000","last_became_active":"2023-11-08T16:39:09.172799+0000","last_became_peered":"2023-11-08T16:39:09.172799+0000","last_unstale":"2023-11-08T16:39:10.225950+0000","last_undegraded":"2023-11-08T16:39:10.225950+0000","last_fullsized":"2023-11-08T16:39:10.225950+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:07:00.018418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.241199+0000","last_change":"2023-11-08T16:39:10.237632+0000","last_active":"2023-11-08T16:39:10.241199+0000","last_peered":"2023-11-08T16:39:10.241199+0000","last_clean":"2023-11-08T16:39:10.241199+0000","last_became_active":"2023-11-08T16:39:10.237353+0000","last_became_peered":"2023-11-08T16:39:10.237353+0000","last_unstale":"2023-11-08T16:39:10.241199+0000","last_undegraded":"2023-11-08T16:39:10.241199+0000","last_fullsized":"2023-11-08T16:39:10.241199+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:53:06.069562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"102'3","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225688+0000","last_change":"2023-11-08T16:39:09.184567+0000","last_active":"2023-11-08T16:39:11.225688+0000","last_peered":"2023-11-08T16:39:11.225688+0000","last_clean":"2023-11-08T16:39:11.225688+0000","last_became_active":"2023-11-08T16:39:09.184246+0000","last_became_peered":"2023-11-08T16:39:09.184246+0000","last_unstale":"2023-11-08T16:39:11.225688+0000","last_undegraded":"2023-11-08T16:39:11.225688+0000","last_fullsized":"2023-11-08T16:39:11.225688+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:22:51.522987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.245516+0000","last_change":"2023-11-08T16:39:10.242482+0000","last_active":"2023-11-08T16:39:10.245516+0000","last_peered":"2023-11-08T16:39:10.245516+0000","last_clean":"2023-11-08T16:39:10.245516+0000","last_became_active":"2023-11-08T16:39:10.242187+0000","last_became_peered":"2023-11-08T16:39:10.242187+0000","last_unstale":"2023-11-08T16:39:10.245516+0000","last_undegraded":"2023-11-08T16:39:10.245516+0000","last_fullsized":"2023-11-08T16:39:10.245516+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:14:35.821692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"102'6","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.245782+0000","last_change":"2023-11-08T16:39:09.181877+0000","last_active":"2023-11-08T16:39:10.245782+0000","last_peered":"2023-11-08T16:39:10.245782+0000","last_clean":"2023-11-08T16:39:10.245782+0000","last_became_active":"2023-11-08T16:39:09.181312+0000","last_became_peered":"2023-11-08T16:39:09.181312+0000","last_unstale":"2023-11-08T16:39:10.245782+0000","last_undegraded":"2023-11-08T16:39:10.245782+0000","last_fullsized":"2023-11-08T16:39:10.245782+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:30:20.289964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.244230+0000","last_change":"2023-11-08T16:39:10.240472+0000","last_active":"2023-11-08T16:39:10.244230+0000","last_peered":"2023-11-08T16:39:10.244230+0000","last_clean":"2023-11-08T16:39:10.244230+0000","last_became_active":"2023-11-08T16:39:10.240113+0000","last_became_peered":"2023-11-08T16:39:10.240113+0000","last_unstale":"2023-11-08T16:39:10.244230+0000","last_undegraded":"2023-11-08T16:39:10.244230+0000","last_fullsized":"2023-11-08T16:39:10.244230+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:01:39.873637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228308+0000","last_change":"2023-11-08T16:39:09.180771+0000","last_active":"2023-11-08T16:39:11.228308+0000","last_peered":"2023-11-08T16:39:11.228308+0000","last_clean":"2023-11-08T16:39:11.228308+0000","last_became_active":"2023-11-08T16:39:09.180449+0000","last_became_peered":"2023-11-08T16:39:09.180449+0000","last_unstale":"2023-11-08T16:39:11.228308+0000","last_undegraded":"2023-11-08T16:39:11.228308+0000","last_fullsized":"2023-11-08T16:39:11.228308+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:30:11.665802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.223754+0000","last_change":"2023-11-08T16:39:10.217002+0000","last_active":"2023-11-08T16:39:11.223754+0000","last_peered":"2023-11-08T16:39:11.223754+0000","last_clean":"2023-11-08T16:39:11.223754+0000","last_became_active":"2023-11-08T16:39:10.214516+0000","last_became_peered":"2023-11-08T16:39:10.214516+0000","last_unstale":"2023-11-08T16:39:11.223754+0000","last_undegraded":"2023-11-08T16:39:11.223754+0000","last_fullsized":"2023-11-08T16:39:11.223754+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:30:12.670615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.198905+0000","last_change":"2023-11-08T16:39:09.183592+0000","last_active":"2023-11-08T16:39:10.198905+0000","last_peered":"2023-11-08T16:39:10.198905+0000","last_clean":"2023-11-08T16:39:10.198905+0000","last_became_active":"2023-11-08T16:39:09.183313+0000","last_became_peered":"2023-11-08T16:39:09.183313+0000","last_unstale":"2023-11-08T16:39:10.198905+0000","last_undegraded":"2023-11-08T16:39:10.198905+0000","last_fullsized":"2023-11-08T16:39:10.198905+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:06:09.999691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.239228+0000","last_change":"2023-11-08T16:39:10.231934+0000","last_active":"2023-11-08T16:39:10.239228+0000","last_peered":"2023-11-08T16:39:10.239228+0000","last_clean":"2023-11-08T16:39:10.239228+0000","last_became_active":"2023-11-08T16:39:10.231667+0000","last_became_peered":"2023-11-08T16:39:10.231667+0000","last_unstale":"2023-11-08T16:39:10.239228+0000","last_undegraded":"2023-11-08T16:39:10.239228+0000","last_fullsized":"2023-11-08T16:39:10.239228+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:09:54.424406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.216479+0000","last_change":"2023-11-08T16:39:09.185054+0000","last_active":"2023-11-08T16:39:10.216479+0000","last_peered":"2023-11-08T16:39:10.216479+0000","last_clean":"2023-11-08T16:39:10.216479+0000","last_became_active":"2023-11-08T16:39:09.184699+0000","last_became_peered":"2023-11-08T16:39:09.184699+0000","last_unstale":"2023-11-08T16:39:10.216479+0000","last_undegraded":"2023-11-08T16:39:10.216479+0000","last_fullsized":"2023-11-08T16:39:10.216479+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:57:02.460058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.230692+0000","last_change":"2023-11-08T16:39:10.224794+0000","last_active":"2023-11-08T16:39:10.230692+0000","last_peered":"2023-11-08T16:39:10.230692+0000","last_clean":"2023-11-08T16:39:10.230692+0000","last_became_active":"2023-11-08T16:39:10.224431+0000","last_became_peered":"2023-11-08T16:39:10.224431+0000","last_unstale":"2023-11-08T16:39:10.230692+0000","last_undegraded":"2023-11-08T16:39:10.230692+0000","last_fullsized":"2023-11-08T16:39:10.230692+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:17:34.587030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"102'8","reported_seq":19,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.244792+0000","last_change":"2023-11-08T16:39:09.162958+0000","last_active":"2023-11-08T16:39:10.244792+0000","last_peered":"2023-11-08T16:39:10.244792+0000","last_clean":"2023-11-08T16:39:10.244792+0000","last_became_active":"2023-11-08T16:39:09.162581+0000","last_became_peered":"2023-11-08T16:39:09.162581+0000","last_unstale":"2023-11-08T16:39:10.244792+0000","last_undegraded":"2023-11-08T16:39:10.244792+0000","last_fullsized":"2023-11-08T16:39:10.244792+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:54.503796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.237451+0000","last_change":"2023-11-08T16:39:10.226079+0000","last_active":"2023-11-08T16:39:10.237451+0000","last_peered":"2023-11-08T16:39:10.237451+0000","last_clean":"2023-11-08T16:39:10.237451+0000","last_became_active":"2023-11-08T16:39:10.224515+0000","last_became_peered":"2023-11-08T16:39:10.224515+0000","last_unstale":"2023-11-08T16:39:10.237451+0000","last_undegraded":"2023-11-08T16:39:10.237451+0000","last_fullsized":"2023-11-08T16:39:10.237451+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:54:07.505364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289736+0000","last_change":"2023-11-08T16:39:09.184787+0000","last_active":"2023-11-08T16:39:11.289736+0000","last_peered":"2023-11-08T16:39:11.289736+0000","last_clean":"2023-11-08T16:39:11.289736+0000","last_became_active":"2023-11-08T16:39:09.183611+0000","last_became_peered":"2023-11-08T16:39:09.183611+0000","last_unstale":"2023-11-08T16:39:11.289736+0000","last_undegraded":"2023-11-08T16:39:11.289736+0000","last_fullsized":"2023-11-08T16:39:11.289736+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:24.926586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289610+0000","last_change":"2023-11-08T16:39:10.206877+0000","last_active":"2023-11-08T16:39:11.289610+0000","last_peered":"2023-11-08T16:39:11.289610+0000","last_clean":"2023-11-08T16:39:11.289610+0000","last_became_active":"2023-11-08T16:39:10.206224+0000","last_became_peered":"2023-11-08T16:39:10.206224+0000","last_unstale":"2023-11-08T16:39:11.289610+0000","last_undegraded":"2023-11-08T16:39:11.289610+0000","last_fullsized":"2023-11-08T16:39:11.289610+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:22:53.831527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228415+0000","last_change":"2023-11-08T16:39:09.189356+0000","last_active":"2023-11-08T16:39:11.228415+0000","last_peered":"2023-11-08T16:39:11.228415+0000","last_clean":"2023-11-08T16:39:11.228415+0000","last_became_active":"2023-11-08T16:39:09.186501+0000","last_became_peered":"2023-11-08T16:39:09.186501+0000","last_unstale":"2023-11-08T16:39:11.228415+0000","last_undegraded":"2023-11-08T16:39:11.228415+0000","last_fullsized":"2023-11-08T16:39:11.228415+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:58.270995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224844+0000","last_change":"2023-11-08T16:39:10.229070+0000","last_active":"2023-11-08T16:39:11.224844+0000","last_peered":"2023-11-08T16:39:11.224844+0000","last_clean":"2023-11-08T16:39:11.224844+0000","last_became_active":"2023-11-08T16:39:10.228754+0000","last_became_peered":"2023-11-08T16:39:10.228754+0000","last_unstale":"2023-11-08T16:39:11.224844+0000","last_undegraded":"2023-11-08T16:39:11.224844+0000","last_fullsized":"2023-11-08T16:39:11.224844+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:53:06.069562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"102'1","reported_seq":13,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225920+0000","last_change":"2023-11-08T16:39:09.185406+0000","last_active":"2023-11-08T16:39:11.225920+0000","last_peered":"2023-11-08T16:39:11.225920+0000","last_clean":"2023-11-08T16:39:11.225920+0000","last_became_active":"2023-11-08T16:39:09.185069+0000","last_became_peered":"2023-11-08T16:39:09.185069+0000","last_unstale":"2023-11-08T16:39:11.225920+0000","last_undegraded":"2023-11-08T16:39:11.225920+0000","last_fullsized":"2023-11-08T16:39:11.225920+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:35:00.469842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.231389+0000","last_change":"2023-11-08T16:39:10.220489+0000","last_active":"2023-11-08T16:39:10.231389+0000","last_peered":"2023-11-08T16:39:10.231389+0000","last_clean":"2023-11-08T16:39:10.231389+0000","last_became_active":"2023-11-08T16:39:10.220123+0000","last_became_peered":"2023-11-08T16:39:10.220123+0000","last_unstale":"2023-11-08T16:39:10.231389+0000","last_undegraded":"2023-11-08T16:39:10.231389+0000","last_fullsized":"2023-11-08T16:39:10.231389+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:19:02.824312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225198+0000","last_change":"2023-11-08T16:39:09.183311+0000","last_active":"2023-11-08T16:39:11.225198+0000","last_peered":"2023-11-08T16:39:11.225198+0000","last_clean":"2023-11-08T16:39:11.225198+0000","last_became_active":"2023-11-08T16:39:09.183038+0000","last_became_peered":"2023-11-08T16:39:09.183038+0000","last_unstale":"2023-11-08T16:39:11.225198+0000","last_undegraded":"2023-11-08T16:39:11.225198+0000","last_fullsized":"2023-11-08T16:39:11.225198+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:24.926586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225823+0000","last_change":"2023-11-08T16:39:10.227910+0000","last_active":"2023-11-08T16:39:11.225823+0000","last_peered":"2023-11-08T16:39:11.225823+0000","last_clean":"2023-11-08T16:39:11.225823+0000","last_became_active":"2023-11-08T16:39:10.227089+0000","last_became_peered":"2023-11-08T16:39:10.227089+0000","last_unstale":"2023-11-08T16:39:11.225823+0000","last_undegraded":"2023-11-08T16:39:11.225823+0000","last_fullsized":"2023-11-08T16:39:11.225823+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:43.391913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.212076+0000","last_change":"2023-11-08T16:39:09.172863+0000","last_active":"2023-11-08T16:39:10.212076+0000","last_peered":"2023-11-08T16:39:10.212076+0000","last_clean":"2023-11-08T16:39:10.212076+0000","last_became_active":"2023-11-08T16:39:09.172536+0000","last_became_peered":"2023-11-08T16:39:09.172536+0000","last_unstale":"2023-11-08T16:39:10.212076+0000","last_undegraded":"2023-11-08T16:39:10.212076+0000","last_fullsized":"2023-11-08T16:39:10.212076+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:15:17.786066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.224904+0000","last_change":"2023-11-08T16:39:10.233062+0000","last_active":"2023-11-08T16:39:11.224904+0000","last_peered":"2023-11-08T16:39:11.224904+0000","last_clean":"2023-11-08T16:39:11.224904+0000","last_became_active":"2023-11-08T16:39:10.232791+0000","last_became_peered":"2023-11-08T16:39:10.232791+0000","last_unstale":"2023-11-08T16:39:11.224904+0000","last_undegraded":"2023-11-08T16:39:11.224904+0000","last_fullsized":"2023-11-08T16:39:11.224904+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:55.508609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225937+0000","last_change":"2023-11-08T16:39:09.183397+0000","last_active":"2023-11-08T16:39:11.225937+0000","last_peered":"2023-11-08T16:39:11.225937+0000","last_clean":"2023-11-08T16:39:11.225937+0000","last_became_active":"2023-11-08T16:39:09.183081+0000","last_became_peered":"2023-11-08T16:39:09.183081+0000","last_unstale":"2023-11-08T16:39:11.225937+0000","last_undegraded":"2023-11-08T16:39:11.225937+0000","last_fullsized":"2023-11-08T16:39:11.225937+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:14:34.816878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.237104+0000","last_change":"2023-11-08T16:39:10.226471+0000","last_active":"2023-11-08T16:39:10.237104+0000","last_peered":"2023-11-08T16:39:10.237104+0000","last_clean":"2023-11-08T16:39:10.237104+0000","last_became_active":"2023-11-08T16:39:10.225106+0000","last_became_peered":"2023-11-08T16:39:10.225106+0000","last_unstale":"2023-11-08T16:39:10.237104+0000","last_undegraded":"2023-11-08T16:39:10.237104+0000","last_fullsized":"2023-11-08T16:39:10.237104+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:30:02.487471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.212223+0000","last_change":"2023-11-08T16:39:09.184775+0000","last_active":"2023-11-08T16:39:10.212223+0000","last_peered":"2023-11-08T16:39:10.212223+0000","last_clean":"2023-11-08T16:39:10.212223+0000","last_became_active":"2023-11-08T16:39:09.184484+0000","last_became_peered":"2023-11-08T16:39:09.184484+0000","last_unstale":"2023-11-08T16:39:10.212223+0000","last_undegraded":"2023-11-08T16:39:10.212223+0000","last_fullsized":"2023-11-08T16:39:10.212223+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:54:53.437907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289526+0000","last_change":"2023-11-08T16:39:10.208466+0000","last_active":"2023-11-08T16:39:11.289526+0000","last_peered":"2023-11-08T16:39:11.289526+0000","last_clean":"2023-11-08T16:39:11.289526+0000","last_became_active":"2023-11-08T16:39:10.207971+0000","last_became_peered":"2023-11-08T16:39:10.207971+0000","last_unstale":"2023-11-08T16:39:11.289526+0000","last_undegraded":"2023-11-08T16:39:11.289526+0000","last_fullsized":"2023-11-08T16:39:11.289526+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:46:23.223886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"102'2","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228449+0000","last_change":"2023-11-08T16:39:09.188262+0000","last_active":"2023-11-08T16:39:11.228449+0000","last_peered":"2023-11-08T16:39:11.228449+0000","last_clean":"2023-11-08T16:39:11.228449+0000","last_became_active":"2023-11-08T16:39:09.186561+0000","last_became_peered":"2023-11-08T16:39:09.186561+0000","last_unstale":"2023-11-08T16:39:11.228449+0000","last_undegraded":"2023-11-08T16:39:11.228449+0000","last_fullsized":"2023-11-08T16:39:11.228449+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:49:35.299981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.235369+0000","last_change":"2023-11-08T16:39:10.223848+0000","last_active":"2023-11-08T16:39:10.235369+0000","last_peered":"2023-11-08T16:39:10.235369+0000","last_clean":"2023-11-08T16:39:10.235369+0000","last_became_active":"2023-11-08T16:39:10.223505+0000","last_became_peered":"2023-11-08T16:39:10.223505+0000","last_unstale":"2023-11-08T16:39:10.235369+0000","last_undegraded":"2023-11-08T16:39:10.235369+0000","last_fullsized":"2023-11-08T16:39:10.235369+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:01:21.503000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225134+0000","last_change":"2023-11-08T16:39:09.188211+0000","last_active":"2023-11-08T16:39:11.225134+0000","last_peered":"2023-11-08T16:39:11.225134+0000","last_clean":"2023-11-08T16:39:11.225134+0000","last_became_active":"2023-11-08T16:39:09.187938+0000","last_became_peered":"2023-11-08T16:39:09.187938+0000","last_unstale":"2023-11-08T16:39:11.225134+0000","last_undegraded":"2023-11-08T16:39:11.225134+0000","last_fullsized":"2023-11-08T16:39:11.225134+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:42.387100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.232405+0000","last_change":"2023-11-08T16:39:10.221431+0000","last_active":"2023-11-08T16:39:10.232405+0000","last_peered":"2023-11-08T16:39:10.232405+0000","last_clean":"2023-11-08T16:39:10.232405+0000","last_became_active":"2023-11-08T16:39:10.221168+0000","last_became_peered":"2023-11-08T16:39:10.221168+0000","last_unstale":"2023-11-08T16:39:10.232405+0000","last_undegraded":"2023-11-08T16:39:10.232405+0000","last_fullsized":"2023-11-08T16:39:10.232405+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:20:23.511414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.226126+0000","last_change":"2023-11-08T16:39:09.184420+0000","last_active":"2023-11-08T16:39:11.226126+0000","last_peered":"2023-11-08T16:39:11.226126+0000","last_clean":"2023-11-08T16:39:11.226126+0000","last_became_active":"2023-11-08T16:39:09.183724+0000","last_became_peered":"2023-11-08T16:39:09.183724+0000","last_unstale":"2023-11-08T16:39:11.226126+0000","last_undegraded":"2023-11-08T16:39:11.226126+0000","last_fullsized":"2023-11-08T16:39:11.226126+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:18:24.926586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225583+0000","last_change":"2023-11-08T16:39:10.223514+0000","last_active":"2023-11-08T16:39:11.225583+0000","last_peered":"2023-11-08T16:39:11.225583+0000","last_clean":"2023-11-08T16:39:11.225583+0000","last_became_active":"2023-11-08T16:39:10.223211+0000","last_became_peered":"2023-11-08T16:39:10.223211+0000","last_unstale":"2023-11-08T16:39:11.225583+0000","last_undegraded":"2023-11-08T16:39:11.225583+0000","last_fullsized":"2023-11-08T16:39:11.225583+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:36:59.275808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289754+0000","last_change":"2023-11-08T16:39:09.180750+0000","last_active":"2023-11-08T16:39:11.289754+0000","last_peered":"2023-11-08T16:39:11.289754+0000","last_clean":"2023-11-08T16:39:11.289754+0000","last_became_active":"2023-11-08T16:39:09.180237+0000","last_became_peered":"2023-11-08T16:39:09.180237+0000","last_unstale":"2023-11-08T16:39:11.289754+0000","last_undegraded":"2023-11-08T16:39:11.289754+0000","last_fullsized":"2023-11-08T16:39:11.289754+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:54:40.040074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.219326+0000","last_change":"2023-11-08T16:39:10.216556+0000","last_active":"2023-11-08T16:39:10.219326+0000","last_peered":"2023-11-08T16:39:10.219326+0000","last_clean":"2023-11-08T16:39:10.219326+0000","last_became_active":"2023-11-08T16:39:10.216243+0000","last_became_peered":"2023-11-08T16:39:10.216243+0000","last_unstale":"2023-11-08T16:39:10.219326+0000","last_undegraded":"2023-11-08T16:39:10.219326+0000","last_fullsized":"2023-11-08T16:39:10.219326+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:15:56.431930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.196722+0000","last_change":"2023-11-08T16:39:09.183557+0000","last_active":"2023-11-08T16:39:10.196722+0000","last_peered":"2023-11-08T16:39:10.196722+0000","last_clean":"2023-11-08T16:39:10.196722+0000","last_became_active":"2023-11-08T16:39:09.183300+0000","last_became_peered":"2023-11-08T16:39:09.183300+0000","last_unstale":"2023-11-08T16:39:10.196722+0000","last_undegraded":"2023-11-08T16:39:10.196722+0000","last_fullsized":"2023-11-08T16:39:10.196722+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:16:37.233758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.245746+0000","last_change":"2023-11-08T16:39:10.242852+0000","last_active":"2023-11-08T16:39:10.245746+0000","last_peered":"2023-11-08T16:39:10.245746+0000","last_clean":"2023-11-08T16:39:10.245746+0000","last_became_active":"2023-11-08T16:39:10.241501+0000","last_became_peered":"2023-11-08T16:39:10.241501+0000","last_unstale":"2023-11-08T16:39:10.245746+0000","last_undegraded":"2023-11-08T16:39:10.245746+0000","last_fullsized":"2023-11-08T16:39:10.245746+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:46:38.004348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.225040+0000","last_change":"2023-11-08T16:39:09.185269+0000","last_active":"2023-11-08T16:39:11.225040+0000","last_peered":"2023-11-08T16:39:11.225040+0000","last_clean":"2023-11-08T16:39:11.225040+0000","last_became_active":"2023-11-08T16:39:09.184979+0000","last_became_peered":"2023-11-08T16:39:09.184979+0000","last_unstale":"2023-11-08T16:39:11.225040+0000","last_undegraded":"2023-11-08T16:39:11.225040+0000","last_fullsized":"2023-11-08T16:39:11.225040+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:40.559793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289162+0000","last_change":"2023-11-08T16:39:10.235772+0000","last_active":"2023-11-08T16:39:11.289162+0000","last_peered":"2023-11-08T16:39:11.289162+0000","last_clean":"2023-11-08T16:39:11.289162+0000","last_became_active":"2023-11-08T16:39:10.235507+0000","last_became_peered":"2023-11-08T16:39:10.235507+0000","last_unstale":"2023-11-08T16:39:11.289162+0000","last_undegraded":"2023-11-08T16:39:11.289162+0000","last_fullsized":"2023-11-08T16:39:11.289162+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:58:43.391913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.289584+0000","last_change":"2023-11-08T16:39:09.187527+0000","last_active":"2023-11-08T16:39:11.289584+0000","last_peered":"2023-11-08T16:39:11.289584+0000","last_clean":"2023-11-08T16:39:11.289584+0000","last_became_active":"2023-11-08T16:39:09.184253+0000","last_became_peered":"2023-11-08T16:39:09.184253+0000","last_unstale":"2023-11-08T16:39:11.289584+0000","last_undegraded":"2023-11-08T16:39:11.289584+0000","last_fullsized":"2023-11-08T16:39:11.289584+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:55:40.559793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.237084+0000","last_change":"2023-11-08T16:39:10.225772+0000","last_active":"2023-11-08T16:39:10.237084+0000","last_peered":"2023-11-08T16:39:10.237084+0000","last_clean":"2023-11-08T16:39:10.237084+0000","last_became_active":"2023-11-08T16:39:10.225489+0000","last_became_peered":"2023-11-08T16:39:10.225489+0000","last_unstale":"2023-11-08T16:39:10.237084+0000","last_undegraded":"2023-11-08T16:39:10.237084+0000","last_fullsized":"2023-11-08T16:39:10.237084+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:32:49.327179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.227979+0000","last_change":"2023-11-08T16:39:09.175591+0000","last_active":"2023-11-08T16:39:11.227979+0000","last_peered":"2023-11-08T16:39:11.227979+0000","last_clean":"2023-11-08T16:39:11.227979+0000","last_became_active":"2023-11-08T16:39:09.174815+0000","last_became_peered":"2023-11-08T16:39:09.174815+0000","last_unstale":"2023-11-08T16:39:11.227979+0000","last_undegraded":"2023-11-08T16:39:11.227979+0000","last_fullsized":"2023-11-08T16:39:11.227979+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:49:54.503796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":11,"reported_epoch":103,"state":"active+clean","last_fresh":"2023-11-08T16:39:11.228340+0000","last_change":"2023-11-08T16:39:10.226800+0000","last_active":"2023-11-08T16:39:11.228340+0000","last_peered":"2023-11-08T16:39:11.228340+0000","last_clean":"2023-11-08T16:39:11.228340+0000","last_became_active":"2023-11-08T16:39:10.226497+0000","last_became_peered":"2023-11-08T16:39:10.226497+0000","last_unstale":"2023-11-08T16:39:11.228340+0000","last_undegraded":"2023-11-08T16:39:11.228340+0000","last_fullsized":"2023-11-08T16:39:11.228340+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:09.140409+0000","last_clean_scrub_stamp":"2023-11-08T16:39:09.140409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:19:38.165884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.214238+0000","last_change":"2023-11-08T16:39:09.184176+0000","last_active":"2023-11-08T16:39:10.214238+0000","last_peered":"2023-11-08T16:39:10.214238+0000","last_clean":"2023-11-08T16:39:10.214238+0000","last_became_active":"2023-11-08T16:39:09.183888+0000","last_became_peered":"2023-11-08T16:39:09.183888+0000","last_unstale":"2023-11-08T16:39:10.214238+0000","last_undegraded":"2023-11-08T16:39:10.214238+0000","last_fullsized":"2023-11-08T16:39:10.214238+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:39:08.135596+0000","last_clean_scrub_stamp":"2023-11-08T16:39:08.135596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:44:57.514158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":252,"reported_epoch":102,"state":"active+clean","last_fresh":"2023-11-08T16:39:10.196885+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:39:10.196885+0000","last_peered":"2023-11-08T16:39:10.196885+0000","last_clean":"2023-11-08T16:39:10.196885+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:39:10.196885+0000","last_undegraded":"2023-11-08T16:39:10.196885+0000","last_fullsized":"2023-11-08T16:39:10.196885+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574974,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33008,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338832,"statfs":{"total":96636764160,"available":96602963968,"internally_reserved":0,"allocated":1871872,"data_stored":1172678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.0609999999999999,"15min":1.0840000000000001},"min":{"1min":0.70699999999999996,"5min":0.51600000000000001,"15min":0.49299999999999999},"max":{"1min":1.4670000000000001,"5min":1.6879999999999999,"15min":3.1899999999999999},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.0700000000000001,"15min":1.109},"min":{"1min":0.64400000000000002,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.595,"5min":1.7629999999999999,"15min":4.2729999999999997},"last":0.93400000000000005}]},{"osd":1,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.091,"15min":1.1160000000000001},"min":{"1min":0.77400000000000002,"5min":0.60199999999999998,"15min":0.57899999999999996},"max":{"1min":1.7050000000000001,"5min":1.7050000000000001,"15min":1.8340000000000001},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.1459999999999999,"15min":1.204},"min":{"1min":0.622,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6459999999999999,"5min":2.1419999999999999,"15min":3.3740000000000001},"last":1.4490000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.159,"15min":1.236},"min":{"1min":0.70999999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.5389999999999999,"5min":1.619,"15min":4.2249999999999996},"last":1.147},{"interface":"front","average":{"1min":1.264,"5min":1.2010000000000001,"15min":1.212},"min":{"1min":0.752,"5min":0.67500000000000004,"15min":0.56299999999999994},"max":{"1min":1.6910000000000001,"5min":1.79,"15min":3.4750000000000001},"last":1.3180000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.135,"15min":1.1599999999999999},"min":{"1min":0.751,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4179999999999999,"5min":1.7669999999999999,"15min":2.964},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.391,"5min":1.333,"15min":1.4379999999999999},"min":{"1min":0.98099999999999998,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.7490000000000001,"5min":2.25,"15min":4.181},"last":1.5109999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1859999999999999,"15min":1.222},"min":{"1min":0.90700000000000003,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.5529999999999999,"5min":1.6970000000000001,"15min":3.6669999999999998},"last":1.097},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3069999999999999,"15min":1.333},"min":{"1min":1.0089999999999999,"5min":0.77300000000000002,"15min":0.59099999999999997},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":2.3730000000000002},"last":1.3520000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.373,"5min":1.3819999999999999,"15min":1.3640000000000001},"min":{"1min":1.0129999999999999,"5min":0.89200000000000002,"15min":0.61799999999999999},"max":{"1min":1.865,"5min":2.3559999999999999,"15min":3.1059999999999999},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4530000000000001,"15min":1.46},"min":{"1min":1.1930000000000001,"5min":1.0269999999999999,"15min":0.878},"max":{"1min":1.825,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.4159999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:38:14 2023","interfaces":[{"interface":"back","average":{"1min":1.5229999999999999,"5min":1.516,"15min":1.53},"min":{"1min":1.208,"5min":0.94799999999999995,"15min":0.94799999999999995},"max":{"1min":1.919,"5min":1.923,"15min":3.7530000000000001},"last":1.474},{"interface":"front","average":{"1min":1.496,"5min":1.4570000000000001,"15min":1.484},"min":{"1min":1.087,"5min":0.73999999999999999,"15min":0.73999999999999999},"max":{"1min":2.0430000000000001,"5min":2.3540000000000001,"15min":6.9560000000000004},"last":1.5669999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32432,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339408,"statfs":{"total":96636764160,"available":96603553792,"internally_reserved":0,"allocated":1277952,"data_stored":578170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.052,"15min":1.0629999999999999},"min":{"1min":0.874,"5min":0.497,"15min":0.497},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":2.5609999999999999},"last":1.6100000000000001},{"interface":"front","average":{"1min":1.196,"5min":1.0980000000000001,"15min":1.0860000000000001},"min":{"1min":0.95999999999999996,"5min":0.63100000000000001,"15min":0.56200000000000006},"max":{"1min":1.502,"5min":1.6819999999999999,"15min":2.7519999999999998},"last":1.484}]},{"osd":1,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1020000000000001,"15min":1.0669999999999999},"min":{"1min":0.81999999999999995,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.875},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.1160000000000001,"15min":1.125},"min":{"1min":0.90600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.7290000000000001,"5min":1.7290000000000001,"15min":1.97},"last":0.93600000000000005}]},{"osd":2,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.171,"15min":1.1859999999999999},"min":{"1min":0.84199999999999997,"5min":0.71599999999999997,"15min":0.56799999999999995},"max":{"1min":1.698,"5min":1.804,"15min":1.804},"last":1.571},{"interface":"front","average":{"1min":1.282,"5min":1.2,"15min":1.171},"min":{"1min":0.97699999999999998,"5min":0.72299999999999998,"15min":0.60499999999999998},"max":{"1min":1.5509999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.73399999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.1990000000000001,"15min":1.2470000000000001},"min":{"1min":1.022,"5min":0.63200000000000001,"15min":0.58299999999999996},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":2.1120000000000001},"last":1.347},{"interface":"front","average":{"1min":1.363,"5min":1.278,"15min":1.292},"min":{"1min":0.873,"5min":0.68200000000000005,"15min":0.67300000000000004},"max":{"1min":1.6439999999999999,"5min":1.76,"15min":1.95},"last":1.7509999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.2949999999999999,"15min":1.339},"min":{"1min":0.84999999999999998,"5min":0.81799999999999995,"15min":0.65300000000000002},"max":{"1min":1.778,"5min":1.778,"15min":2.218},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3660000000000001,"15min":1.4059999999999999},"min":{"1min":1.028,"5min":0.86899999999999999,"15min":0.64900000000000002},"max":{"1min":1.9019999999999999,"5min":1.9590000000000001,"15min":2.7080000000000002},"last":1.681}]},{"osd":5,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.377,"5min":1.3440000000000001,"15min":1.3240000000000001},"min":{"1min":0.91500000000000004,"5min":0.879,"15min":0.80500000000000005},"max":{"1min":1.6599999999999999,"5min":1.994,"15min":2.105},"last":1.264},{"interface":"front","average":{"1min":1.4670000000000001,"5min":1.4950000000000001,"15min":1.5369999999999999},"min":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.78500000000000003},"max":{"1min":2.1280000000000001,"5min":2.1280000000000001,"15min":2.4700000000000002},"last":1.7789999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5289999999999999,"5min":1.5029999999999999,"15min":1.5680000000000001},"min":{"1min":1.3069999999999999,"5min":1.002,"15min":0.74299999999999999},"max":{"1min":2.024,"5min":2.1309999999999998,"15min":3.8959999999999999},"last":1.635},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.498,"15min":1.51},"min":{"1min":1.1479999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":2.069,"5min":2.0870000000000002,"15min":2.4889999999999999},"last":1.407}]}]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1269760,"data_stored":582254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.046,"15min":1.0609999999999999},"min":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.575,"5min":1.639,"15min":2.4609999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.069,"15min":1.1100000000000001},"min":{"1min":0.74299999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.5660000000000001,"5min":1.593,"15min":2.3639999999999999},"last":1.5700000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2130000000000001,"15min":1.2330000000000001},"min":{"1min":0.77100000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.6180000000000001,"5min":1.9610000000000001,"15min":3.8690000000000002},"last":1.532},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.1879999999999999,"15min":1.2509999999999999},"min":{"1min":0.90000000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.4950000000000001,"5min":1.9370000000000001,"15min":3.2349999999999999},"last":1.377}]},{"osd":2,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.1559999999999999,"15min":1.212},"min":{"1min":0.66100000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.5580000000000001,"5min":1.764,"15min":2.6339999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.21,"15min":1.208},"min":{"1min":0.70499999999999996,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.577,"5min":1.8029999999999999,"15min":2.2559999999999998},"last":1.3220000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.25,"15min":1.29},"min":{"1min":0.97799999999999998,"5min":0.67300000000000004,"15min":0.66500000000000004},"max":{"1min":1.6519999999999999,"5min":1.843,"15min":2.8330000000000002},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.264,"15min":1.2969999999999999},"min":{"1min":0.84599999999999997,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.8440000000000001,"5min":2.0459999999999998,"15min":2.5760000000000001},"last":1.083}]},{"osd":4,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.262,"15min":1.3380000000000001},"min":{"1min":0.89700000000000002,"5min":0.85499999999999998,"15min":0.61599999999999999},"max":{"1min":1.5389999999999999,"5min":2.0790000000000002,"15min":3.9790000000000001},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4159999999999999,"15min":1.4650000000000001},"min":{"1min":1.0569999999999999,"5min":0.98499999999999999,"15min":0.81499999999999995},"max":{"1min":1.8029999999999999,"5min":2.0409999999999999,"15min":3.0209999999999999},"last":1.2010000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.4039999999999999,"15min":1.415},"min":{"1min":0.99299999999999999,"5min":0.92400000000000004,"15min":0.752},"max":{"1min":1.7749999999999999,"5min":1.867,"15min":2.5550000000000002},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.3899999999999999,"15min":1.4810000000000001},"min":{"1min":0.98499999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":1.657,"5min":2.1619999999999999,"15min":4.1189999999999998},"last":1.1319999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:38:13 2023","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.52,"15min":1.5620000000000001},"min":{"1min":1.2729999999999999,"5min":1.099,"15min":1.0049999999999999},"max":{"1min":2.0219999999999998,"5min":2.129,"15min":3.1160000000000001},"last":1.512},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.5329999999999999,"15min":1.5549999999999999},"min":{"1min":1.1870000000000001,"5min":0.96999999999999997,"15min":0.96999999999999997},"max":{"1min":2.0800000000000001,"5min":2.3660000000000001,"15min":2.7050000000000001},"last":1.4810000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1277952,"data_stored":582414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0820000000000001,"15min":1.0760000000000001},"min":{"1min":0.73199999999999998,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1259999999999999,"15min":1.0800000000000001},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.45400000000000001},"max":{"1min":1.548,"5min":1.772,"15min":1.772},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.127,"15min":1.1240000000000001},"min":{"1min":0.874,"5min":0.57299999999999995,"15min":0.56999999999999995},"max":{"1min":1.51,"5min":1.587,"15min":2.0459999999999998},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.1739999999999999,"15min":1.1679999999999999},"min":{"1min":0.79100000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.462,"5min":1.742,"15min":1.742},"last":0.84599999999999997}]},{"osd":2,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2350000000000001,"15min":1.204},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.6299999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.234,"5min":1.2330000000000001,"15min":1.25},"min":{"1min":0.92900000000000005,"5min":0.78100000000000003,"15min":0.71699999999999997},"max":{"1min":1.6020000000000001,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.92900000000000005}]},{"osd":3,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.278,"15min":1.276},"min":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79100000000000004},"max":{"1min":1.758,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.23,"5min":1.266,"15min":1.2729999999999999},"min":{"1min":0.95199999999999996,"5min":0.72499999999999998,"15min":0.65100000000000002},"max":{"1min":1.6359999999999999,"5min":1.851,"15min":1.895},"last":1.1459999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3740000000000001,"15min":1.3680000000000001},"min":{"1min":0.90500000000000003,"5min":0.68100000000000005,"15min":0.58799999999999997},"max":{"1min":2.286,"5min":2.286,"15min":2.286},"last":1.224},{"interface":"front","average":{"1min":1.4399999999999999,"5min":1.464,"15min":1.4319999999999999},"min":{"1min":0.879,"5min":0.86499999999999999,"15min":0.61899999999999999},"max":{"1min":1.7669999999999999,"5min":2.1970000000000001,"15min":2.46},"last":0.879}]},{"osd":6,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3660000000000001,"15min":1.3979999999999999},"min":{"1min":1.089,"5min":0.75800000000000001,"15min":0.74299999999999999},"max":{"1min":1.772,"5min":1.9510000000000001,"15min":2.5609999999999999},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.512,"15min":1.5389999999999999},"min":{"1min":1.032,"5min":0.83899999999999997,"15min":0.81200000000000006},"max":{"1min":2.4780000000000002,"5min":2.4780000000000002,"15min":2.6389999999999998},"last":1.369}]},{"osd":7,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.5640000000000001,"5min":1.5469999999999999,"15min":1.528},"min":{"1min":1.3029999999999999,"5min":1.056,"15min":0.92300000000000004},"max":{"1min":1.859,"5min":2.1040000000000001,"15min":2.2639999999999998},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.575,"5min":1.522,"15min":1.569},"min":{"1min":1.2430000000000001,"5min":1.0209999999999999,"15min":0.84799999999999998},"max":{"1min":2.1070000000000002,"5min":2.1070000000000002,"15min":2.6859999999999999},"last":1.4179999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":575954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.80900000000000005,"15min":0.81499999999999995},"min":{"1min":0.53100000000000003,"5min":0.36599999999999999,"15min":0.33800000000000002},"max":{"1min":1.29,"5min":1.4930000000000001,"15min":1.9339999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.85799999999999998,"15min":0.90700000000000003},"min":{"1min":0.46000000000000002,"5min":0.316,"15min":0.316},"max":{"1min":1.4710000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":1.0389999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.58599999999999997,"5min":0.40899999999999997,"15min":0.36499999999999999},"max":{"1min":1.179,"5min":1.4770000000000001,"15min":1.4790000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.0509999999999999,"5min":0.95999999999999996,"15min":0.96799999999999997},"min":{"1min":0.63400000000000001,"5min":0.378,"15min":0.378},"max":{"1min":1.425,"5min":1.425,"15min":1.829},"last":1.0940000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.0129999999999999,"15min":0.998},"min":{"1min":0.624,"5min":0.48399999999999999,"15min":0.41699999999999998},"max":{"1min":1.474,"5min":1.7490000000000001,"15min":2.0779999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.0840000000000001,"15min":1.1180000000000001},"min":{"1min":0.66300000000000003,"5min":0.504,"15min":0.504},"max":{"1min":1.6419999999999999,"5min":1.7270000000000001,"15min":2.4660000000000002},"last":1.26}]},{"osd":4,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.149,"15min":1.1830000000000001},"min":{"1min":0.73199999999999998,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.5720000000000001,"5min":1.8069999999999999,"15min":2.25},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.2669999999999999,"15min":1.2689999999999999},"min":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.747},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.1829999999999998},"last":1.1830000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.3220000000000001,"15min":1.329},"min":{"1min":1.006,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":2.2229999999999999,"5min":2.2229999999999999,"15min":2.3999999999999999},"last":1.296},{"interface":"front","average":{"1min":1.4179999999999999,"5min":1.337,"15min":1.339},"min":{"1min":0.996,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.714,"5min":1.714,"15min":2.593},"last":1.4430000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3740000000000001,"15min":1.391},"min":{"1min":0.877,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.776,"5min":1.776,"15min":2.9569999999999999},"last":1.355},{"interface":"front","average":{"1min":1.29,"5min":1.3049999999999999,"15min":1.3680000000000001},"min":{"1min":0.83699999999999997,"5min":0.75800000000000001,"15min":0.73399999999999999},"max":{"1min":1.796,"5min":1.8919999999999999,"15min":2.762},"last":0.83599999999999997}]},{"osd":7,"last update":"Wed Nov 8 16:39:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.3260000000000001,"15min":1.409},"min":{"1min":0.92200000000000004,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.6850000000000001,"5min":1.861,"15min":2.8700000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.484,"5min":1.4019999999999999,"15min":1.4239999999999999},"min":{"1min":1.032,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.121,"5min":2.121,"15min":2.633},"last":1.401}]}]},{"osd":2,"up_from":13,"seq":55834574976,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1257472,"data_stored":575954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.79500000000000004,"15min":0.93999999999999995},"min":{"1min":0.40100000000000002,"5min":0.32900000000000001,"15min":0.24099999999999999},"max":{"1min":1.264,"5min":1.2929999999999999,"15min":4.5369999999999999},"last":0.753},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.87,"15min":0.878},"min":{"1min":0.55700000000000005,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.94099999999999995}]},{"osd":1,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.0349999999999999,"15min":1.0800000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.379},"max":{"1min":1.609,"5min":1.7689999999999999,"15min":4.4980000000000002},"last":0.871},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.0589999999999999,"15min":1.052},"min":{"1min":0.52800000000000002,"5min":0.52400000000000002,"15min":0.40300000000000002},"max":{"1min":2.1909999999999998,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.1659999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.86599999999999999,"15min":0.98499999999999999},"min":{"1min":0.50800000000000001,"5min":0.39300000000000002,"15min":0.35499999999999998},"max":{"1min":1.194,"5min":1.4870000000000001,"15min":3.3050000000000002},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.097,"5min":1.0660000000000001,"15min":1.173},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.42999999999999999},"max":{"1min":1.635,"5min":1.819,"15min":4.5899999999999999},"last":1.3109999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.2809999999999999,"15min":1.3700000000000001},"min":{"1min":0.76100000000000001,"5min":0.74299999999999999,"15min":0.73499999999999999},"max":{"1min":1.829,"5min":2.2160000000000002,"15min":4.9409999999999998},"last":1.421},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2130000000000001,"15min":1.258},"min":{"1min":0.85499999999999998,"5min":0.72499999999999998,"15min":0.63100000000000001},"max":{"1min":2.0739999999999998,"5min":2.0739999999999998,"15min":2.46},"last":1.357}]},{"osd":5,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.252,"15min":1.3009999999999999},"min":{"1min":0.871,"5min":0.78200000000000003,"15min":0.77200000000000002},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":2.8530000000000002},"last":1.45},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.252,"15min":1.3260000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.69399999999999995},"max":{"1min":1.7689999999999999,"5min":2.5390000000000001,"15min":3.5680000000000001},"last":1.5289999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.264,"15min":1.361},"min":{"1min":0.877,"5min":0.81000000000000005,"15min":0.625},"max":{"1min":1.6200000000000001,"5min":1.9710000000000001,"15min":3.536},"last":1.6990000000000001},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.3859999999999999,"15min":1.47},"min":{"1min":0.88500000000000001,"5min":0.755,"15min":0.72199999999999998},"max":{"1min":2.2719999999999998,"5min":2.2719999999999998,"15min":4.7549999999999999},"last":1.212}]},{"osd":7,"last update":"Wed Nov 8 16:39:06 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3280000000000001,"15min":1.4099999999999999},"min":{"1min":0.745,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.9700000000000002},"last":1.516},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3640000000000001,"15min":1.4570000000000001},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.70499999999999996},"max":{"1min":1.702,"5min":2.129,"15min":3.4950000000000001},"last":1.7789999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574976,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1261568,"data_stored":576024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.76800000000000002,"15min":0.76500000000000001},"min":{"1min":0.34499999999999997,"5min":0.33400000000000002,"15min":0.28899999999999998},"max":{"1min":1.1739999999999999,"5min":1.542,"15min":1.542},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.98099999999999998,"5min":0.877,"15min":0.96699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.309},"max":{"1min":1.47,"5min":1.47,"15min":4.0309999999999997},"last":0.57599999999999996}]},{"osd":2,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0029999999999999,"15min":1.054},"min":{"1min":0.48899999999999999,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.462,"5min":1.7689999999999999,"15min":3.0339999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.95099999999999996,"15min":1.028},"min":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":1.542,"5min":1.542,"15min":2.6840000000000002},"last":0.46899999999999997}]},{"osd":3,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":0.997,"15min":1.0629999999999999},"min":{"1min":0.49399999999999999,"5min":0.46899999999999997,"15min":0.38900000000000001},"max":{"1min":1.335,"5min":1.5129999999999999,"15min":4.3179999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.032,"15min":1.075},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.274,"5min":8.5570000000000004,"15min":8.5570000000000004},"last":0.38100000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.1850000000000001,"15min":1.2649999999999999},"min":{"1min":0.79600000000000004,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.907,"5min":1.907,"15min":3.1659999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.202,"5min":1.222,"15min":1.2989999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.492,"5min":1.823,"15min":2.734},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.254,"15min":1.5129999999999999},"min":{"1min":0.83299999999999996,"5min":0.72399999999999998,"15min":0.67100000000000004},"max":{"1min":1.784,"5min":1.921,"15min":8.4730000000000008},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.278,"15min":1.498},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.8320000000000001,"5min":1.871,"15min":8.8040000000000003},"last":0.68700000000000006}]},{"osd":6,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.3049999999999999,"15min":1.4059999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.7529999999999999,"5min":1.9119999999999999,"15min":3.984},"last":0.64400000000000002},{"interface":"front","average":{"1min":1.373,"5min":1.329,"15min":1.409},"min":{"1min":0.81599999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":3.2450000000000001},"last":0.875}]},{"osd":7,"last update":"Wed Nov 8 16:39:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.298,"15min":1.4550000000000001},"min":{"1min":0.81399999999999995,"5min":0.80700000000000005,"15min":0.69199999999999995},"max":{"1min":1.853,"5min":1.853,"15min":4.3609999999999998},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.391,"15min":1.5349999999999999},"min":{"1min":0.74299999999999999,"5min":0.72799999999999998,"15min":0.70499999999999996},"max":{"1min":1.903,"5min":2.8780000000000001,"15min":5.3840000000000003},"last":0.74299999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574974,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1875968,"data_stored":1168640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.872,"15min":0.81899999999999995},"min":{"1min":0.53900000000000003,"5min":0.42099999999999999,"15min":0.29399999999999998},"max":{"1min":1.208,"5min":1.3680000000000001,"15min":1.6220000000000001},"last":0.996},{"interface":"front","average":{"1min":1.02,"5min":0.95399999999999996,"15min":0.96799999999999997},"min":{"1min":0.71699999999999997,"5min":0.52000000000000002,"15min":0.44800000000000001},"max":{"1min":1.3939999999999999,"5min":1.722,"15min":2.3340000000000001},"last":1.048}]},{"osd":2,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.93500000000000005,"15min":0.93200000000000005},"min":{"1min":0.61899999999999999,"5min":0.42799999999999999,"15min":0.37},"max":{"1min":1.5860000000000001,"5min":1.6910000000000001,"15min":1.762},"last":0.87},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.92600000000000005,"15min":0.98299999999999998},"min":{"1min":0.65800000000000003,"5min":0.45700000000000002,"15min":0.36799999999999999},"max":{"1min":1.3400000000000001,"5min":1.613,"15min":1.879},"last":1.3400000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":0.96099999999999997,"5min":0.91800000000000004,"15min":0.93899999999999995},"min":{"1min":0.57399999999999995,"5min":0.48399999999999999,"15min":0.34200000000000003},"max":{"1min":1.55,"5min":1.72,"15min":1.728},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.194,"5min":1.2050000000000001,"15min":1.1850000000000001},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.573,"5min":1.8740000000000001,"15min":2.7679999999999998},"last":1.1719999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.171,"15min":1.1599999999999999},"min":{"1min":0.85199999999999998,"5min":0.76300000000000001,"15min":0.65300000000000002},"max":{"1min":1.496,"5min":2.0489999999999999,"15min":2.2559999999999998},"last":1.204},{"interface":"front","average":{"1min":1.286,"5min":1.2709999999999999,"15min":1.2829999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.59399999999999997},"max":{"1min":1.7190000000000001,"5min":1.8859999999999999,"15min":2.867},"last":1.095}]},{"osd":5,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.2529999999999999,"15min":1.2629999999999999},"min":{"1min":0.98799999999999999,"5min":0.79600000000000004,"15min":0.67900000000000005},"max":{"1min":1.5629999999999999,"5min":1.98,"15min":2.7280000000000002},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.224,"5min":1.2170000000000001,"15min":1.232},"min":{"1min":0.95499999999999996,"5min":0.74399999999999999,"15min":0.67600000000000005},"max":{"1min":1.579,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.1339999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.371,"15min":1.3600000000000001},"min":{"1min":1.022,"5min":0.79900000000000004,"15min":0.72499999999999998},"max":{"1min":1.865,"5min":1.9399999999999999,"15min":2.6739999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3120000000000001,"15min":1.3540000000000001},"min":{"1min":0.97799999999999998,"5min":0.88200000000000001,"15min":0.64200000000000002},"max":{"1min":1.6779999999999999,"5min":2.1280000000000001,"15min":2.8260000000000001},"last":1.258}]},{"osd":7,"last update":"Wed Nov 8 16:39:09 2023","interfaces":[{"interface":"back","average":{"1min":1.4399999999999999,"5min":1.375,"15min":1.407},"min":{"1min":0.92900000000000005,"5min":0.75,"15min":0.75},"max":{"1min":1.8169999999999999,"5min":2.0150000000000001,"15min":4.5670000000000002},"last":1.5369999999999999},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.337,"15min":1.359},"min":{"1min":0.95099999999999996,"5min":0.85299999999999998,"15min":0.70199999999999996},"max":{"1min":1.7669999999999999,"5min":2.6070000000000002,"15min":5.4450000000000003},"last":1.502}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:39:15.144 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T16:39:15.144 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T16:39:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.402+0000 7f1f7354c640 1 Processor -- start 2023-11-08T16:39:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.402+0000 7f1f7354c640 1 -- start start 2023-11-08T16:39:15.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.404+0000 7f1f7354c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1f6c0b0770 0x7f1f6c06f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.404+0000 7f1f7354c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f6c0ae000 0x7f1f6c0cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.404+0000 7f1f7354c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6c06fe50 con 0x7f1f6c0b0b50 2023-11-08T16:39:15.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.404+0000 7f1f7354c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f6c0cb3c0 con 0x7f1f6c0b0770 2023-11-08T16:39:15.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.404+0000 7f1f7354c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0ca580 con 0x7f1f6c0ae000 2023-11-08T16:39:15.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.404+0000 7f1f712e8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1f6c0b0770 0x7f1f6c06f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:15.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.404+0000 7f1f712e8640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1f6c0b0770 0x7f1f6c06f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42664/0 (socket says 172.21.15.81:42664) 2023-11-08T16:39:15.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.404+0000 7f1f712e8640 1 -- 172.21.15.81:0/3734482423 learned_addr learned my addr 172.21.15.81:0/3734482423 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:15.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.404+0000 7f1f71ae9640 1 --1- 172.21.15.81:0/3734482423 >> v1:172.21.15.81:6789/0 conn(0x7f1f6c0b0b50 0x7f1f6c0b3b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34224/0 (socket says 172.21.15.81:34224) 2023-11-08T16:39:15.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.405+0000 7f1f70ae7640 1 --2- 172.21.15.81:0/3734482423 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f6c0ae000 0x7f1f6c0cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:15.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.405+0000 7f1f712e8640 1 -- 172.21.15.81:0/3734482423 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f6c0ae000 msgr2=0x7f1f6c0cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.405+0000 7f1f712e8640 1 --2- 172.21.15.81:0/3734482423 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f6c0ae000 0x7f1f6c0cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.405+0000 7f1f712e8640 1 -- 172.21.15.81:0/3734482423 >> v1:172.21.15.81:6789/0 conn(0x7f1f6c0b0b50 legacy=0x7f1f6c0b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.405+0000 7f1f63fff640 1 -- 172.21.15.81:0/3734482423 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168495693 0 0) 0x7f1f64001490 con 0x7f1f6c0b0b50 2023-11-08T16:39:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.405+0000 7f1f712e8640 1 -- 172.21.15.81:0/3734482423 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c0ccec0 con 0x7f1f6c0b0770 2023-11-08T16:39:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.405+0000 7f1f70ae7640 1 --2- 172.21.15.81:0/3734482423 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f6c0ae000 0x7f1f6c0cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:15.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.406+0000 7f1f712e8640 1 --2- 172.21.15.81:0/3734482423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f6c0b0770 0x7f1f6c06f910 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009870 tx=0x7f1f6802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf85f8330abf3d19 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:15.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.406+0000 7f1f63fff640 1 -- 172.21.15.81:0/3734482423 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68030840 con 0x7f1f6c0b0770 2023-11-08T16:39:15.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.406+0000 7f1f63fff640 1 -- 172.21.15.81:0/3734482423 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f680309a0 con 0x7f1f6c0b0770 2023-11-08T16:39:15.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.406+0000 7f1f63fff640 1 -- 172.21.15.81:0/3734482423 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68030d30 con 0x7f1f6c0b0770 2023-11-08T16:39:15.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.407+0000 7f1f7354c640 1 -- 172.21.15.81:0/3734482423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f6c0b0770 msgr2=0x7f1f6c06f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.407+0000 7f1f7354c640 1 --2- 172.21.15.81:0/3734482423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f6c0b0770 0x7f1f6c06f910 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f1f68009870 tx=0x7f1f6802b980 comp rx=0 tx=0).stop 2023-11-08T16:39:15.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.407+0000 7f1f7354c640 1 -- 172.21.15.81:0/3734482423 shutdown_connections 2023-11-08T16:39:15.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.407+0000 7f1f7354c640 1 --2- 172.21.15.81:0/3734482423 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f6c0ae000 0x7f1f6c0cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.407+0000 7f1f7354c640 1 --2- 172.21.15.81:0/3734482423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f6c0b0770 0x7f1f6c06f910 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.407+0000 7f1f7354c640 1 -- 172.21.15.81:0/3734482423 >> 172.21.15.81:0/3734482423 conn(0x7f1f6c0cb500 msgr2=0x7f1f6c0cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:15.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.407+0000 7f1f7354c640 1 -- 172.21.15.81:0/3734482423 shutdown_connections 2023-11-08T16:39:15.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.407+0000 7f1f7354c640 1 -- 172.21.15.81:0/3734482423 wait complete. 2023-11-08T16:39:15.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.409+0000 7f1f7354c640 1 Processor -- start 2023-11-08T16:39:15.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.409+0000 7f1f7354c640 1 -- start start 2023-11-08T16:39:15.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f7354c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f6c0ae000 0x7f1f6c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f7354c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f6c0b0770 0x7f1f6c0c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f7354c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f6c0b0b50 0x7f1f6c0c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f7354c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0c7290 con 0x7f1f6c0ae000 2023-11-08T16:39:15.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f7354c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0c7bf0 con 0x7f1f6c0b0b50 2023-11-08T16:39:15.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f7354c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c0c6610 con 0x7f1f6c0b0770 2023-11-08T16:39:15.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f712e8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f6c0ae000 0x7f1f6c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:15.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f712e8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f6c0ae000 0x7f1f6c0c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42672/0 (socket says 172.21.15.81:42672) 2023-11-08T16:39:15.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f712e8640 1 -- 172.21.15.81:0/4142434617 learned_addr learned my addr 172.21.15.81:0/4142434617 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:15.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f712e8640 1 -- 172.21.15.81:0/4142434617 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f6c0b0770 msgr2=0x7f1f6c0c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f712e8640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f6c0b0770 0x7f1f6c0c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f712e8640 1 -- 172.21.15.81:0/4142434617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f6c0b0b50 msgr2=0x7f1f6c0c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f712e8640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f6c0b0b50 0x7f1f6c0c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.410+0000 7f1f712e8640 1 -- 172.21.15.81:0/4142434617 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f680094d0 con 0x7f1f6c0ae000 2023-11-08T16:39:15.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.411+0000 7f1f712e8640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f6c0ae000 0x7f1f6c0c76b0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1f6802b9c0 tx=0x7f1f68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:15.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.411+0000 7f1f61ffb640 1 -- 172.21.15.81:0/4142434617 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68030810 con 0x7f1f6c0ae000 2023-11-08T16:39:15.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.411+0000 7f1f61ffb640 1 -- 172.21.15.81:0/4142434617 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f68044480 con 0x7f1f6c0ae000 2023-11-08T16:39:15.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.411+0000 7f1f61ffb640 1 -- 172.21.15.81:0/4142434617 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68044850 con 0x7f1f6c0ae000 2023-11-08T16:39:15.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.411+0000 7f1f7354c640 1 -- 172.21.15.81:0/4142434617 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6c0c68f0 con 0x7f1f6c0ae000 2023-11-08T16:39:15.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.411+0000 7f1f7354c640 1 -- 172.21.15.81:0/4142434617 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f6c0c6eb0 con 0x7f1f6c0ae000 2023-11-08T16:39:15.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.412+0000 7f1f7354c640 1 -- 172.21.15.81:0/4142434617 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6c0743f0 con 0x7f1f6c0ae000 2023-11-08T16:39:15.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.413+0000 7f1f61ffb640 1 -- 172.21.15.81:0/4142434617 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f6802f390 con 0x7f1f6c0ae000 2023-11-08T16:39:15.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.414+0000 7f1f61ffb640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f5006ae70 0x7f1f5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.414+0000 7f1f61ffb640 1 -- 172.21.15.81:0/4142434617 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f6803e070 con 0x7f1f6c0ae000 2023-11-08T16:39:15.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.414+0000 7f1f70ae7640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f5006ae70 0x7f1f5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:15.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.415+0000 7f1f70ae7640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f5006ae70 0x7f1f5006d330 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:15.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.419+0000 7f1f61ffb640 1 -- 172.21.15.81:0/4142434617 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f68077060 con 0x7f1f6c0ae000 2023-11-08T16:39:15.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.675+0000 7f1f7354c640 1 -- 172.21.15.81:0/4142434617 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1f6c0c9b90 con 0x7f1f6c0ae000 2023-11-08T16:39:15.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.679+0000 7f1f61ffb640 1 -- 172.21.15.81:0/4142434617 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1f680340b0 con 0x7f1f6c0ae000 2023-11-08T16:39:15.681 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2023-11-08T16:39:15.681 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ== 2023-11-08T16:39:15.681 INFO:teuthology.orchestra.run.smithi081.stdout: caps mds = "allow" 2023-11-08T16:39:15.681 INFO:teuthology.orchestra.run.smithi081.stdout: caps mon = "allow r" 2023-11-08T16:39:15.682 INFO:teuthology.orchestra.run.smithi081.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T16:39:15.682 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2023-11-08T16:39:15.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.682+0000 7f1f7354c640 1 -- 172.21.15.81:0/4142434617 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f5006ae70 msgr2=0x7f1f5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.682+0000 7f1f7354c640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f5006ae70 0x7f1f5006d330 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006fd0 tx=0x7f1f5c008040 comp rx=0 tx=0).stop 2023-11-08T16:39:15.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.682+0000 7f1f7354c640 1 -- 172.21.15.81:0/4142434617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f6c0ae000 msgr2=0x7f1f6c0c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.682+0000 7f1f7354c640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f6c0ae000 0x7f1f6c0c76b0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f1f6802b9c0 tx=0x7f1f68004730 comp rx=0 tx=0).stop 2023-11-08T16:39:15.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.683+0000 7f1f7354c640 1 -- 172.21.15.81:0/4142434617 shutdown_connections 2023-11-08T16:39:15.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.683+0000 7f1f7354c640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f5006ae70 0x7f1f5006d330 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.683+0000 7f1f7354c640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f6c0b0b50 0x7f1f6c0c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.683+0000 7f1f7354c640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f6c0b0770 0x7f1f6c0c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.683+0000 7f1f7354c640 1 --2- 172.21.15.81:0/4142434617 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f6c0ae000 0x7f1f6c0c76b0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.683+0000 7f1f7354c640 1 -- 172.21.15.81:0/4142434617 >> 172.21.15.81:0/4142434617 conn(0x7f1f6c0cb500 msgr2=0x7f1f6c0b4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:15.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.683+0000 7f1f7354c640 1 -- 172.21.15.81:0/4142434617 shutdown_connections 2023-11-08T16:39:15.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.683+0000 7f1f7354c640 1 -- 172.21.15.81:0/4142434617 wait complete. 2023-11-08T16:39:15.702 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:15.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.912+0000 7f61b7fff640 1 Processor -- start 2023-11-08T16:39:15.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.912+0000 7f61b7fff640 1 -- start start 2023-11-08T16:39:15.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.914+0000 7f61b7fff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f61b80d39e0 0x7f61b80d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.914+0000 7f61b7fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61b80b5e50 0x7f61b80b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.914+0000 7f61b7fff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b80ce890 con 0x7f61b815d490 2023-11-08T16:39:15.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.914+0000 7f61b7fff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b80d3e40 con 0x7f61b80d39e0 2023-11-08T16:39:15.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.914+0000 7f61b7fff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b80ceb70 con 0x7f61b80b5e50 2023-11-08T16:39:15.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.915+0000 7f61b67fc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f61b80d39e0 0x7f61b80d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:15.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.915+0000 7f61b67fc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f61b80d39e0 0x7f61b80d1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:56492/0 (socket says 172.21.15.81:56492) 2023-11-08T16:39:15.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.915+0000 7f61b77fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61b80b5e50 0x7f61b80b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:15.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.915+0000 7f61b67fc640 1 -- 172.21.15.81:0/2947456563 learned_addr learned my addr 172.21.15.81:0/2947456563 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:15.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.915+0000 7f61b77fe640 1 -- 172.21.15.81:0/2947456563 >> v2:172.21.15.93:3300/0 conn(0x7f61b80d39e0 msgr2=0x7f61b80d1790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:15.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.915+0000 7f61b77fe640 1 --2- 172.21.15.81:0/2947456563 >> v2:172.21.15.93:3300/0 conn(0x7f61b80d39e0 0x7f61b80d1790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.915+0000 7f61b77fe640 1 -- 172.21.15.81:0/2947456563 >> v1:172.21.15.93:6789/0 conn(0x7f61b815d490 legacy=0x7f61b80d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.915+0000 7f61b77fe640 1 -- 172.21.15.81:0/2947456563 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b80cee50 con 0x7f61b80b5e50 2023-11-08T16:39:15.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.915+0000 7f61b67fc640 1 --2- 172.21.15.81:0/2947456563 >> v2:172.21.15.93:3300/0 conn(0x7f61b80d39e0 0x7f61b80d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:39:15.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.917+0000 7f61b77fe640 1 --2- 172.21.15.81:0/2947456563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61b80b5e50 0x7f61b80b62e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f61b00097a0 tx=0x7f61b002b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c69ba4908e36df7b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:15.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.917+0000 7f61b5ffb640 1 -- 172.21.15.81:0/2947456563 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b002d040 con 0x7f61b80b5e50 2023-11-08T16:39:15.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.917+0000 7f61b5ffb640 1 -- 172.21.15.81:0/2947456563 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b0031810 con 0x7f61b80b5e50 2023-11-08T16:39:15.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.917+0000 7f61b5ffb640 1 -- 172.21.15.81:0/2947456563 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0031ba0 con 0x7f61b80b5e50 2023-11-08T16:39:15.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.918+0000 7f61b7fff640 1 -- 172.21.15.81:0/2947456563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61b80b5e50 msgr2=0x7f61b80b62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.918+0000 7f61b7fff640 1 --2- 172.21.15.81:0/2947456563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61b80b5e50 0x7f61b80b62e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f61b00097a0 tx=0x7f61b002b8b0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.918+0000 7f61b7fff640 1 -- 172.21.15.81:0/2947456563 shutdown_connections 2023-11-08T16:39:15.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.918+0000 7f61b7fff640 1 --2- 172.21.15.81:0/2947456563 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61b80b5e50 0x7f61b80b62e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.918+0000 7f61b7fff640 1 --2- 172.21.15.81:0/2947456563 >> v2:172.21.15.93:3300/0 conn(0x7f61b80d39e0 0x7f61b80d1790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.918+0000 7f61b7fff640 1 -- 172.21.15.81:0/2947456563 >> 172.21.15.81:0/2947456563 conn(0x7f61b80d2e50 msgr2=0x7f61b80d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:15.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.918+0000 7f61b7fff640 1 -- 172.21.15.81:0/2947456563 shutdown_connections 2023-11-08T16:39:15.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.918+0000 7f61b7fff640 1 -- 172.21.15.81:0/2947456563 wait complete. 2023-11-08T16:39:15.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.920+0000 7f61b7fff640 1 Processor -- start 2023-11-08T16:39:15.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.920+0000 7f61b7fff640 1 -- start start 2023-11-08T16:39:15.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.921+0000 7f61b7fff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61b80b5e50 0x7f61b80c79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.921+0000 7f61b7fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61b80d39e0 0x7f61b80c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.921+0000 7f61b7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61b815d490 0x7f61b80c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.921+0000 7f61b7fff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b8159920 con 0x7f61b815d490 2023-11-08T16:39:15.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.921+0000 7f61b7fff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b815a780 con 0x7f61b80b5e50 2023-11-08T16:39:15.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.921+0000 7f61b7fff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b80c98c0 con 0x7f61b80d39e0 2023-11-08T16:39:15.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.922+0000 7f61b77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61b815d490 0x7f61b80c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:15.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.922+0000 7f61b77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61b815d490 0x7f61b80c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42682/0 (socket says 172.21.15.81:42682) 2023-11-08T16:39:15.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.922+0000 7f61b77fe640 1 -- 172.21.15.81:0/3451866123 learned_addr learned my addr 172.21.15.81:0/3451866123 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:15.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.922+0000 7f61b6ffd640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61b80b5e50 0x7f61b80c79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:15.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.922+0000 7f61b67fc640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61b80d39e0 0x7f61b80c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:15.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.922+0000 7f61b77fe640 1 -- 172.21.15.81:0/3451866123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61b80d39e0 msgr2=0x7f61b80c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.922+0000 7f61b77fe640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61b80d39e0 0x7f61b80c8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.922+0000 7f61b77fe640 1 -- 172.21.15.81:0/3451866123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61b80b5e50 msgr2=0x7f61b80c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:15.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.922+0000 7f61b77fe640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61b80b5e50 0x7f61b80c79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:15.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.922+0000 7f61b77fe640 1 -- 172.21.15.81:0/3451866123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a00094d0 con 0x7f61b815d490 2023-11-08T16:39:15.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.922+0000 7f61b77fe640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61b815d490 0x7f61b80c9090 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f61b002b8f0 tx=0x7f61b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:15.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.923+0000 7f61af7fe640 1 -- 172.21.15.81:0/3451866123 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b003ace0 con 0x7f61b815d490 2023-11-08T16:39:15.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.923+0000 7f61af7fe640 1 -- 172.21.15.81:0/3451866123 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b003ae40 con 0x7f61b815d490 2023-11-08T16:39:15.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.923+0000 7f61af7fe640 1 -- 172.21.15.81:0/3451866123 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b0042630 con 0x7f61b815d490 2023-11-08T16:39:15.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.923+0000 7f61b7fff640 1 -- 172.21.15.81:0/3451866123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b0009400 con 0x7f61b815d490 2023-11-08T16:39:15.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.923+0000 7f61b7fff640 1 -- 172.21.15.81:0/3451866123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b8159b70 con 0x7f61b815d490 2023-11-08T16:39:15.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.924+0000 7f61b7fff640 1 -- 172.21.15.81:0/3451866123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b815a910 con 0x7f61b815d490 2023-11-08T16:39:15.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.925+0000 7f61af7fe640 1 -- 172.21.15.81:0/3451866123 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f61b0042790 con 0x7f61b815d490 2023-11-08T16:39:15.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.926+0000 7f61af7fe640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f619806ae70 0x7f619806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:15.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.926+0000 7f61af7fe640 1 -- 172.21.15.81:0/3451866123 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61b00ab4d0 con 0x7f61b815d490 2023-11-08T16:39:15.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.926+0000 7f61b6ffd640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f619806ae70 0x7f619806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:15.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.927+0000 7f61b6ffd640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f619806ae70 0x7f619806d330 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f61a80012e0 tx=0x7f61a8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:15.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:15.930+0000 7f61af7fe640 1 -- 172.21.15.81:0/3451866123 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f61b0076570 con 0x7f61b815d490 2023-11-08T16:39:16.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.157+0000 7f61b7fff640 1 -- 172.21.15.81:0/3451866123 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61b8067800 con 0x7f61b815d490 2023-11-08T16:39:16.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.158+0000 7f61af7fe640 1 -- 172.21.15.81:0/3451866123 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 86 v86) v1 ==== 94+0+4912 (secure 0 0 0) 0x7f61b0079b90 con 0x7f61b815d490 2023-11-08T16:39:16.160 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:16.160 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":86,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1157756119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1157756119},{"type":"v1","addr":"172.21.15.93:6833","nonce":1157756119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1607105255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1607105255},{"type":"v1","addr":"172.21.15.81:6835","nonce":1607105255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3988749175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3988749175},{"type":"v1","addr":"172.21.15.81:6837","nonce":3988749175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":86,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:09.619973+0000","modified":"2023-11-08T16:39:12.717402+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7713},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7713":{"gid":7713,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6835/691341796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":691341796},{"type":"v1","addr":"172.21.15.93:6835","nonce":691341796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:39:16.160 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 86 2023-11-08T16:39:16.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.160+0000 7f61b7fff640 1 -- 172.21.15.81:0/3451866123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f619806ae70 msgr2=0x7f619806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.160+0000 7f61b7fff640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f619806ae70 0x7f619806d330 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f61a80012e0 tx=0x7f61a8009040 comp rx=0 tx=0).stop 2023-11-08T16:39:16.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.160+0000 7f61b7fff640 1 -- 172.21.15.81:0/3451866123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61b815d490 msgr2=0x7f61b80c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.160+0000 7f61b7fff640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61b815d490 0x7f61b80c9090 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f61b002b8f0 tx=0x7f61b00046c0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.161+0000 7f61b7fff640 1 -- 172.21.15.81:0/3451866123 shutdown_connections 2023-11-08T16:39:16.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.161+0000 7f61b7fff640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f619806ae70 0x7f619806d330 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.161+0000 7f61b7fff640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f61b815d490 0x7f61b80c9090 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.161+0000 7f61b7fff640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f61b80d39e0 0x7f61b80c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.161+0000 7f61b7fff640 1 --2- 172.21.15.81:0/3451866123 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61b80b5e50 0x7f61b80c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.161+0000 7f61b7fff640 1 -- 172.21.15.81:0/3451866123 >> 172.21.15.81:0/3451866123 conn(0x7f61b80d2e50 msgr2=0x7f61b80d1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:16.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.161+0000 7f61b7fff640 1 -- 172.21.15.81:0/3451866123 shutdown_connections 2023-11-08T16:39:16.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.161+0000 7f61b7fff640 1 -- 172.21.15.81:0/3451866123 wait complete. 2023-11-08T16:39:16.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 86, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:39:09.619973+0000', 'modified': '2023-11-08T16:39:12.717402+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7713}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7713': {'gid': 7713, 'name': 'b', 'rank': 0, 'incarnation': 82, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.93:6835/691341796', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6834', 'nonce': 691341796}, {'type': 'v1', 'addr': '172.21.15.93:6835', 'nonce': 691341796}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:39:16.585 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:39:16.585 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-08T16:39:16.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.808+0000 7fda5e2af640 1 Processor -- start 2023-11-08T16:39:16.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.808+0000 7fda5e2af640 1 -- start start 2023-11-08T16:39:16.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.809+0000 7fda5e2af640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fda5815aeb0 0x7fda580ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.810+0000 7fda5e2af640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5806cc60 con 0x7fda580c9580 2023-11-08T16:39:16.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.810+0000 7fda5e2af640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda580cae20 con 0x7fda580c90f0 2023-11-08T16:39:16.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.810+0000 7fda5e2af640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda580cb3e0 con 0x7fda5815aeb0 2023-11-08T16:39:16.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.810+0000 7fda57fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fda5815aeb0 0x7fda580ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.810+0000 7fda5c84c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fda580c9580 0x7fda5806b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34232/0 (socket says 172.21.15.81:34232) 2023-11-08T16:39:16.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.810+0000 7fda5c84c640 1 -- 172.21.15.81:0/3845270271 learned_addr learned my addr 172.21.15.81:0/3845270271 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:16.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.810+0000 7fda57fff640 1 -- 172.21.15.81:0/3845270271 >> v1:172.21.15.93:6789/0 conn(0x7fda580c90f0 legacy=0x7fda5806b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:16.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.810+0000 7fda56ffd640 1 -- 172.21.15.81:0/3845270271 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1833714689 0 0) 0x7fda5806cc60 con 0x7fda580c9580 2023-11-08T16:39:16.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.810+0000 7fda57fff640 1 -- 172.21.15.81:0/3845270271 >> v1:172.21.15.81:6789/0 conn(0x7fda580c9580 legacy=0x7fda5806b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.810+0000 7fda57fff640 1 -- 172.21.15.81:0/3845270271 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda580cb100 con 0x7fda5815aeb0 2023-11-08T16:39:16.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.812+0000 7fda57fff640 1 --2- 172.21.15.81:0/3845270271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda5815aeb0 0x7fda580ca4a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009790 tx=0x7fda4c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8e9f0b109cde736 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:16.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.812+0000 7fda56ffd640 1 -- 172.21.15.81:0/3845270271 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c02d040 con 0x7fda5815aeb0 2023-11-08T16:39:16.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.812+0000 7fda56ffd640 1 -- 172.21.15.81:0/3845270271 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4c031810 con 0x7fda5815aeb0 2023-11-08T16:39:16.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.812+0000 7fda56ffd640 1 -- 172.21.15.81:0/3845270271 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c031ba0 con 0x7fda5815aeb0 2023-11-08T16:39:16.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.813+0000 7fda5e2af640 1 -- 172.21.15.81:0/3845270271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda5815aeb0 msgr2=0x7fda580ca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.813+0000 7fda5e2af640 1 --2- 172.21.15.81:0/3845270271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda5815aeb0 0x7fda580ca4a0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009790 tx=0x7fda4c02b8b0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.813+0000 7fda5e2af640 1 -- 172.21.15.81:0/3845270271 shutdown_connections 2023-11-08T16:39:16.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.813+0000 7fda5e2af640 1 --2- 172.21.15.81:0/3845270271 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda5815aeb0 0x7fda580ca4a0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.813+0000 7fda5e2af640 1 -- 172.21.15.81:0/3845270271 >> 172.21.15.81:0/3845270271 conn(0x7fda580ceaa0 msgr2=0x7fda580cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:16.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.813+0000 7fda5e2af640 1 -- 172.21.15.81:0/3845270271 shutdown_connections 2023-11-08T16:39:16.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.813+0000 7fda5e2af640 1 -- 172.21.15.81:0/3845270271 wait complete. 2023-11-08T16:39:16.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.815+0000 7fda5e2af640 1 Processor -- start 2023-11-08T16:39:16.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.815+0000 7fda5e2af640 1 -- start start 2023-11-08T16:39:16.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.816+0000 7fda5e2af640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda580c90f0 0x7fda580c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.816+0000 7fda5e2af640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda580c9580 0x7fda58158d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.816+0000 7fda5e2af640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fda5815aeb0 0x7fda580cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.816+0000 7fda5e2af640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda580ac8b0 con 0x7fda580c9580 2023-11-08T16:39:16.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.816+0000 7fda5e2af640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda58159240 con 0x7fda580c90f0 2023-11-08T16:39:16.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.816+0000 7fda5e2af640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda581f6680 con 0x7fda5815aeb0 2023-11-08T16:39:16.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.816+0000 7fda577fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda580c9580 0x7fda58158d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.816+0000 7fda577fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda580c9580 0x7fda58158d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42698/0 (socket says 172.21.15.81:42698) 2023-11-08T16:39:16.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.816+0000 7fda577fe640 1 -- 172.21.15.81:0/4201020422 learned_addr learned my addr 172.21.15.81:0/4201020422 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:16.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.817+0000 7fda57fff640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda580c90f0 0x7fda580c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.817+0000 7fda577fe640 1 -- 172.21.15.81:0/4201020422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fda5815aeb0 msgr2=0x7fda580cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.817+0000 7fda5c84c640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fda5815aeb0 0x7fda580cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.817+0000 7fda577fe640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fda5815aeb0 0x7fda580cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.817+0000 7fda577fe640 1 -- 172.21.15.81:0/4201020422 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda580c90f0 msgr2=0x7fda580c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.817+0000 7fda577fe640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda580c90f0 0x7fda580c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.817+0000 7fda577fe640 1 -- 172.21.15.81:0/4201020422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda4c0093a0 con 0x7fda580c9580 2023-11-08T16:39:16.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.817+0000 7fda57fff640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda580c90f0 0x7fda580c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:16.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.817+0000 7fda577fe640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda580c9580 0x7fda58158d00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fda4800c430 tx=0x7fda4800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:16.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.818+0000 7fda54ff9640 1 -- 172.21.15.81:0/4201020422 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48011790 con 0x7fda580c9580 2023-11-08T16:39:16.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.818+0000 7fda54ff9640 1 -- 172.21.15.81:0/4201020422 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda480118f0 con 0x7fda580c9580 2023-11-08T16:39:16.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.818+0000 7fda5e2af640 1 -- 172.21.15.81:0/4201020422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda581f6960 con 0x7fda580c9580 2023-11-08T16:39:16.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.818+0000 7fda54ff9640 1 -- 172.21.15.81:0/4201020422 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48011ce0 con 0x7fda580c9580 2023-11-08T16:39:16.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.818+0000 7fda5e2af640 1 -- 172.21.15.81:0/4201020422 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda581f6f20 con 0x7fda580c9580 2023-11-08T16:39:16.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.820+0000 7fda54ff9640 1 -- 172.21.15.81:0/4201020422 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fda4801c460 con 0x7fda580c9580 2023-11-08T16:39:16.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.821+0000 7fda54ff9640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda4406ae70 0x7fda4406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.821+0000 7fda54ff9640 1 -- 172.21.15.81:0/4201020422 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda48089610 con 0x7fda580c9580 2023-11-08T16:39:16.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.821+0000 7fda57fff640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda4406ae70 0x7fda4406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.822+0000 7fda60e3f740 1 Processor -- start 2023-11-08T16:39:16.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.822+0000 7fda60e3f740 1 -- start start 2023-11-08T16:39:16.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.822+0000 7fda57fff640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda4406ae70 0x7fda4406d330 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fda4c036000 tx=0x7fda4c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:16.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.823+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef3c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.823+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f52a0 0x55c3ef3f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.823+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3c8300 0x55c3ef3f8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.823+0000 7fda60e3f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef33a7e0 con 0x55c3ef3f52a0 2023-11-08T16:39:16.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.823+0000 7fda60e3f740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef337ed0 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.823+0000 7fda60e3f740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef3337c0 con 0x55c3ef3c8300 2023-11-08T16:39:16.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.823+0000 7fda577fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f52a0 0x55c3ef3f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.823+0000 7fda577fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f52a0 0x55c3ef3f7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42706/0 (socket says 172.21.15.81:42706) 2023-11-08T16:39:16.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.823+0000 7fda577fe640 1 -- 172.21.15.81:0/4035308351 learned_addr learned my addr 172.21.15.81:0/4035308351 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:16.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.823+0000 7fda57fff640 1 --2- 172.21.15.81:0/4035308351 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef3c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.824+0000 7fda577fe640 1 -- 172.21.15.81:0/4035308351 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3c8300 msgr2=0x55c3ef3f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:16.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.824+0000 7fda577fe640 1 --2- 172.21.15.81:0/4035308351 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3c8300 0x55c3ef3f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.824+0000 7fda577fe640 1 -- 172.21.15.81:0/4035308351 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 msgr2=0x55c3ef3c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:16.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.824+0000 7fda577fe640 1 --2- 172.21.15.81:0/4035308351 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef3c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.824+0000 7fda577fe640 1 -- 172.21.15.81:0/4035308351 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3ef306b10 con 0x55c3ef3f52a0 2023-11-08T16:39:16.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.824+0000 7fda57fff640 1 --2- 172.21.15.81:0/4035308351 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef3c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:16.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.824+0000 7fda577fe640 1 --2- 172.21.15.81:0/4035308351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f52a0 0x55c3ef3f7c50 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fda480547f0 tx=0x7fda480589f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e77d726c1d363c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:16.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.824+0000 7fda3dffb640 1 -- 172.21.15.81:0/4035308351 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda480605f0 con 0x55c3ef3f52a0 2023-11-08T16:39:16.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.824+0000 7fda3dffb640 1 -- 172.21.15.81:0/4035308351 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4805c170 con 0x55c3ef3f52a0 2023-11-08T16:39:16.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.824+0000 7fda3dffb640 1 -- 172.21.15.81:0/4035308351 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48054a00 con 0x55c3ef3f52a0 2023-11-08T16:39:16.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.825+0000 7fda60e3f740 1 -- 172.21.15.81:0/4035308351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f52a0 msgr2=0x55c3ef3f7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.825+0000 7fda60e3f740 1 --2- 172.21.15.81:0/4035308351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f52a0 0x55c3ef3f7c50 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fda480547f0 tx=0x7fda480589f0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.825+0000 7fda60e3f740 1 -- 172.21.15.81:0/4035308351 shutdown_connections 2023-11-08T16:39:16.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.825+0000 7fda60e3f740 1 --2- 172.21.15.81:0/4035308351 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3c8300 0x55c3ef3f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.825+0000 7fda60e3f740 1 --2- 172.21.15.81:0/4035308351 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f52a0 0x55c3ef3f7c50 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.825+0000 7fda60e3f740 1 --2- 172.21.15.81:0/4035308351 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef3c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.825+0000 7fda60e3f740 1 -- 172.21.15.81:0/4035308351 >> 172.21.15.81:0/4035308351 conn(0x55c3ef3ca0d0 msgr2=0x55c3ef412c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:16.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.825+0000 7fda60e3f740 1 -- 172.21.15.81:0/4035308351 shutdown_connections 2023-11-08T16:39:16.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.825+0000 7fda60e3f740 1 -- 172.21.15.81:0/4035308351 wait complete. 2023-11-08T16:39:16.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.826+0000 7fda60e3f740 1 -- start start 2023-11-08T16:39:16.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.826+0000 7fda60e3f740 1 Processor -- start 2023-11-08T16:39:16.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.827+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef3f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.827+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f52a0 0x55c3ef423140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.827+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef423680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.827+0000 7fda60e3f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef2d0c40 con 0x55c3ef40d5d0 2023-11-08T16:39:16.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.827+0000 7fda60e3f740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef2d08e0 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.827+0000 7fda60e3f740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef096f20 con 0x55c3ef3f52a0 2023-11-08T16:39:16.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.827+0000 7fda60e3f740 10 client.0 _ll_register_callbacks cb 0x55c3ef2f2cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:39:16.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.827+0000 7fda5c84c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef423680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.827+0000 7fda5c84c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef423680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42708/0 (socket says 172.21.15.81:42708) 2023-11-08T16:39:16.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.827+0000 7fda5c84c640 1 -- 172.21.15.81:0/1157827456 learned_addr learned my addr 172.21.15.81:0/1157827456 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:16.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda577fe640 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f52a0 0x55c3ef423140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda57fff640 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef3f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda5c84c640 1 -- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f52a0 msgr2=0x55c3ef423140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:16.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda5c84c640 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f52a0 0x55c3ef423140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda5c84c640 1 -- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 msgr2=0x55c3ef3f9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:16.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda5c84c640 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef3f9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda5c84c640 1 -- 172.21.15.81:0/1157827456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda581f6f20 con 0x55c3ef40d5d0 2023-11-08T16:39:16.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda577fe640 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f52a0 0x55c3ef423140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:16.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda57fff640 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef3f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:16.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda5c84c640 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef423680 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fda5000d4b0 tx=0x7fda5000d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e7e67a33fccd47 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:16.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:39:16.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda36ffd640 1 -- 172.21.15.81:0/1157827456 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda500070d0 con 0x55c3ef40d5d0 2023-11-08T16:39:16.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda36ffd640 1 -- 172.21.15.81:0/1157827456 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda50013a80 con 0x55c3ef40d5d0 2023-11-08T16:39:16.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.828+0000 7fda60e3f740 1 -- 172.21.15.81:0/1157827456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3ef307c80 con 0x55c3ef40d5d0 2023-11-08T16:39:16.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.829+0000 7fda36ffd640 1 -- 172.21.15.81:0/1157827456 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda50001760 con 0x55c3ef40d5d0 2023-11-08T16:39:16.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.829+0000 7fda60e3f740 10 client.7854 fetch_fsmap learned FSMap version 86 2023-11-08T16:39:16.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.829+0000 7fda60e3f740 1 -- 172.21.15.81:0/1157827456 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55c3ef331e40 con 0x55c3ef40d5d0 2023-11-08T16:39:16.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.829+0000 7fda36ffd640 1 -- 172.21.15.81:0/1157827456 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c3ef307c80 con 0x55c3ef40d5d0 2023-11-08T16:39:16.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.829+0000 7fda36ffd640 1 -- 172.21.15.81:0/1157827456 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fda50013c40 con 0x55c3ef40d5d0 2023-11-08T16:39:16.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.829+0000 7fda60e3f740 10 client.7854 fetch_fsmap finished waiting for FSMap version 86 2023-11-08T16:39:16.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.829+0000 7fda60e3f740 10 client.7854 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T16:39:16.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.829+0000 7fda60e3f740 20 client.7854 populate_metadata read hostname 'smithi081' 2023-11-08T16:39:16.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.829+0000 7fda60e3f740 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:6834/691341796,v1:172.21.15.93:6835/691341796] conn(0x55c3ef42c7c0 0x55c3ef42cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.829+0000 7fda60e3f740 4 client.7854 mds_command: new command op to 7713 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:39:16.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.829+0000 7fda60e3f740 1 -- 172.21.15.81:0/1157827456 --> [v2:172.21.15.93:6834/691341796,v1:172.21.15.93:6835/691341796] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3ef32c7a0 con 0x55c3ef42c7c0 2023-11-08T16:39:16.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.830+0000 7fda57fff640 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:6834/691341796,v1:172.21.15.93:6835/691341796] conn(0x55c3ef42c7c0 0x55c3ef42cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.831+0000 7fda57fff640 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:6834/691341796,v1:172.21.15.93:6835/691341796] conn(0x55c3ef42c7c0 0x55c3ef42cba0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ad6378b20a30c server_cookie=a99e53910c8a1617 in_seq=0 out_seq=0 2023-11-08T16:39:16.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.831+0000 7fda36ffd640 10 client.7854 ms_handle_connect on v2:172.21.15.93:6834/691341796 2023-11-08T16:39:16.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda36ffd640 1 -- 172.21.15.81:0/1157827456 <== mds.0 v2:172.21.15.93:6834/691341796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c3ef32c7a0 con 0x55c3ef42c7c0 2023-11-08T16:39:16.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda36ffd640 10 client.7854 handle_command_reply: tid=0 2023-11-08T16:39:16.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda60e3f740 1 client.7854 shutdown 2023-11-08T16:39:16.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda60e3f740 1 -- 172.21.15.81:0/1157827456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 msgr2=0x55c3ef423680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda60e3f740 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef423680 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fda5000d4b0 tx=0x7fda5000d970 comp rx=0 tx=0).stop 2023-11-08T16:39:16.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda60e3f740 1 -- 172.21.15.81:0/1157827456 shutdown_connections 2023-11-08T16:39:16.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda60e3f740 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:6834/691341796,v1:172.21.15.93:6835/691341796] conn(0x55c3ef42c7c0 0x55c3ef42cba0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda60e3f740 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef423680 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda60e3f740 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f52a0 0x55c3ef423140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda60e3f740 1 --2- 172.21.15.81:0/1157827456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef3f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda36ffd640 0 client.7854 ms_handle_reset on v2:172.21.15.93:6834/691341796 2023-11-08T16:39:16.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda60e3f740 1 -- 172.21.15.81:0/1157827456 >> 172.21.15.81:0/1157827456 conn(0x55c3ef3c8300 msgr2=0x55c3ef40cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:16.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.833+0000 7fda60e3f740 1 -- 172.21.15.81:0/1157827456 shutdown_connections 2023-11-08T16:39:16.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.834+0000 7fda60e3f740 1 -- 172.21.15.81:0/1157827456 wait complete. 2023-11-08T16:39:16.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.834+0000 7fda60e3f740 20 client.7854 trim_cache size 0 max 16384 2023-11-08T16:39:16.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.850+0000 7fda60e3f740 1 Processor -- start 2023-11-08T16:39:16.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.850+0000 7fda60e3f740 1 -- start start 2023-11-08T16:39:16.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.851+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f4ec0 0x55c3ef415530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.851+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f52a0 0x55c3ef40dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.851+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef3f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.851+0000 7fda60e3f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef3c8920 con 0x55c3ef40d5d0 2023-11-08T16:39:16.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.851+0000 7fda60e3f740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef4139c0 con 0x55c3ef3f52a0 2023-11-08T16:39:16.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.851+0000 7fda60e3f740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef345330 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.852+0000 7fda5c84c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef3f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.852+0000 7fda5c84c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef3f37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42710/0 (socket says 172.21.15.81:42710) 2023-11-08T16:39:16.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.852+0000 7fda5c84c640 1 -- 172.21.15.81:0/914667015 learned_addr learned my addr 172.21.15.81:0/914667015 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:16.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.852+0000 7fda577fe640 1 --2- 172.21.15.81:0/914667015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f52a0 0x55c3ef40dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.852+0000 7fda57fff640 1 --2- 172.21.15.81:0/914667015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f4ec0 0x55c3ef415530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.852+0000 7fda5c84c640 1 -- 172.21.15.81:0/914667015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f4ec0 msgr2=0x55c3ef415530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:16.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.852+0000 7fda5c84c640 1 --2- 172.21.15.81:0/914667015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f4ec0 0x55c3ef415530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.852+0000 7fda5c84c640 1 -- 172.21.15.81:0/914667015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f52a0 msgr2=0x55c3ef40dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:16.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.852+0000 7fda5c84c640 1 --2- 172.21.15.81:0/914667015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f52a0 0x55c3ef40dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.852+0000 7fda5c84c640 1 -- 172.21.15.81:0/914667015 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3ef2cfb90 con 0x55c3ef40d5d0 2023-11-08T16:39:16.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.852+0000 7fda577fe640 1 --2- 172.21.15.81:0/914667015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f52a0 0x55c3ef40dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:16.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.853+0000 7fda5c84c640 1 --2- 172.21.15.81:0/914667015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef3f37c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fda5001a000 tx=0x7fda50006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3338cc46def4195d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:16.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.853+0000 7fda3e7fc640 1 -- 172.21.15.81:0/914667015 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50013470 con 0x55c3ef40d5d0 2023-11-08T16:39:16.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.853+0000 7fda3e7fc640 1 -- 172.21.15.81:0/914667015 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda500135d0 con 0x55c3ef40d5d0 2023-11-08T16:39:16.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.853+0000 7fda3e7fc640 1 -- 172.21.15.81:0/914667015 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda50013960 con 0x55c3ef40d5d0 2023-11-08T16:39:16.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.853+0000 7fda60e3f740 1 -- 172.21.15.81:0/914667015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 msgr2=0x55c3ef3f37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.853+0000 7fda60e3f740 1 --2- 172.21.15.81:0/914667015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef3f37c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fda5001a000 tx=0x7fda50006000 comp rx=0 tx=0).stop 2023-11-08T16:39:16.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.853+0000 7fda60e3f740 1 -- 172.21.15.81:0/914667015 shutdown_connections 2023-11-08T16:39:16.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.853+0000 7fda60e3f740 1 --2- 172.21.15.81:0/914667015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef40d5d0 0x55c3ef3f37c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.853+0000 7fda60e3f740 1 --2- 172.21.15.81:0/914667015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef3f52a0 0x55c3ef40dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.853+0000 7fda60e3f740 1 --2- 172.21.15.81:0/914667015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f4ec0 0x55c3ef415530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.853+0000 7fda60e3f740 1 -- 172.21.15.81:0/914667015 >> 172.21.15.81:0/914667015 conn(0x55c3ef3c8300 msgr2=0x55c3ef3c3560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:16.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.854+0000 7fda60e3f740 1 -- 172.21.15.81:0/914667015 shutdown_connections 2023-11-08T16:39:16.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.854+0000 7fda60e3f740 1 -- 172.21.15.81:0/914667015 wait complete. 2023-11-08T16:39:16.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.854+0000 7fda60e3f740 1 -- start start 2023-11-08T16:39:16.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.854+0000 7fda60e3f740 1 Processor -- start 2023-11-08T16:39:16.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.855+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef40c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.855+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f52a0 0x55c3ef3f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.855+0000 7fda60e3f740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef40d5d0 0x55c3ef3f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda60e3f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef47fca0 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda60e3f740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef3c36f0 con 0x55c3ef40d5d0 2023-11-08T16:39:16.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda60e3f740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c3ef414390 con 0x55c3ef3f52a0 2023-11-08T16:39:16.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda60e3f740 10 client.0 _ll_register_callbacks cb 0x7fda50007eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:39:16.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda57fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef40c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda57fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef40c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42720/0 (socket says 172.21.15.81:42720) 2023-11-08T16:39:16.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda57fff640 1 -- 172.21.15.81:0/3741080985 learned_addr learned my addr 172.21.15.81:0/3741080985 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:16.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda57fff640 1 -- 172.21.15.81:0/3741080985 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f52a0 msgr2=0x55c3ef3f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:16.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda57fff640 1 --2- 172.21.15.81:0/3741080985 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f52a0 0x55c3ef3f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda57fff640 1 -- 172.21.15.81:0/3741080985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef40d5d0 msgr2=0x55c3ef3f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:16.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda57fff640 1 --2- 172.21.15.81:0/3741080985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef40d5d0 0x55c3ef3f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda57fff640 1 -- 172.21.15.81:0/3741080985 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3ef331e40 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda57fff640 1 --2- 172.21.15.81:0/3741080985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef40c260 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fda4c034040 tx=0x7fda4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52deb11b6901a00d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:16.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.856+0000 7fda36ffd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:39:16.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda36ffd640 1 -- 172.21.15.81:0/3741080985 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4c049040 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda36ffd640 1 -- 172.21.15.81:0/3741080985 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda4c051240 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda60e3f740 1 -- 172.21.15.81:0/3741080985 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fda4805c170 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda36ffd640 1 -- 172.21.15.81:0/3741080985 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda4c051400 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda60e3f740 10 client.7860 fetch_fsmap learned FSMap version 86 2023-11-08T16:39:16.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda60e3f740 1 -- 172.21.15.81:0/3741080985 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=86,monmap=2+}) v3 -- 0x55c3ef2f5180 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda36ffd640 1 -- 172.21.15.81:0/3741080985 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4805c170 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda36ffd640 1 -- 172.21.15.81:0/3741080985 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 86) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fda4c051560 con 0x55c3ef3f4ec0 2023-11-08T16:39:16.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda60e3f740 10 client.7860 fetch_fsmap finished waiting for FSMap version 86 2023-11-08T16:39:16.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda60e3f740 10 client.7860 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-08T16:39:16.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda60e3f740 20 client.7860 populate_metadata read hostname 'smithi081' 2023-11-08T16:39:16.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda60e3f740 1 --2- 172.21.15.81:0/3741080985 >> [v2:172.21.15.93:6834/691341796,v1:172.21.15.93:6835/691341796] conn(0x55c3ef47f080 0x55c3ef416160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:16.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda60e3f740 4 client.7860 mds_command: new command op to 7713 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:39:16.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.857+0000 7fda60e3f740 1 -- 172.21.15.81:0/3741080985 --> [v2:172.21.15.93:6834/691341796,v1:172.21.15.93:6835/691341796] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c3ef32c4c0 con 0x55c3ef47f080 2023-11-08T16:39:16.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.858+0000 7fda577fe640 1 --2- 172.21.15.81:0/3741080985 >> [v2:172.21.15.93:6834/691341796,v1:172.21.15.93:6835/691341796] conn(0x55c3ef47f080 0x55c3ef416160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:16.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.859+0000 7fda577fe640 1 --2- 172.21.15.81:0/3741080985 >> [v2:172.21.15.93:6834/691341796,v1:172.21.15.93:6835/691341796] conn(0x55c3ef47f080 0x55c3ef416160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb6fb2ba36c29b66 server_cookie=470a196333ea4f15 in_seq=0 out_seq=0 2023-11-08T16:39:16.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.859+0000 7fda36ffd640 10 client.7860 ms_handle_connect on v2:172.21.15.93:6834/691341796 2023-11-08T16:39:16.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.859+0000 7fda36ffd640 1 -- 172.21.15.81:0/3741080985 <== mds.0 v2:172.21.15.93:6834/691341796 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55c3ef32c4c0 con 0x55c3ef47f080 2023-11-08T16:39:16.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.859+0000 7fda36ffd640 10 client.7860 handle_command_reply: tid=0 2023-11-08T16:39:16.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.859+0000 7fda60e3f740 1 client.7860 shutdown 2023-11-08T16:39:16.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda60e3f740 1 -- 172.21.15.81:0/3741080985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f4ec0 msgr2=0x55c3ef40c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda60e3f740 1 --2- 172.21.15.81:0/3741080985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef40c260 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fda4c034040 tx=0x7fda4c004730 comp rx=0 tx=0).stop 2023-11-08T16:39:16.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda60e3f740 1 -- 172.21.15.81:0/3741080985 shutdown_connections 2023-11-08T16:39:16.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda60e3f740 1 --2- 172.21.15.81:0/3741080985 >> [v2:172.21.15.93:6834/691341796,v1:172.21.15.93:6835/691341796] conn(0x55c3ef47f080 0x55c3ef416160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda60e3f740 1 --2- 172.21.15.81:0/3741080985 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3ef40d5d0 0x55c3ef3f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda60e3f740 1 --2- 172.21.15.81:0/3741080985 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c3ef3f52a0 0x55c3ef3f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda36ffd640 0 client.7860 ms_handle_reset on v2:172.21.15.93:6834/691341796 2023-11-08T16:39:16.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda60e3f740 1 --2- 172.21.15.81:0/3741080985 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c3ef3f4ec0 0x55c3ef40c260 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda60e3f740 1 -- 172.21.15.81:0/3741080985 >> 172.21.15.81:0/3741080985 conn(0x55c3ef3c8300 msgr2=0x55c3ef40dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:16.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda60e3f740 1 -- 172.21.15.81:0/3741080985 shutdown_connections 2023-11-08T16:39:16.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda60e3f740 1 -- 172.21.15.81:0/3741080985 wait complete. 2023-11-08T16:39:16.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.860+0000 7fda60e3f740 20 client.7860 trim_cache size 0 max 16384 2023-11-08T16:39:16.877 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:39:16.877 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:39:16.877 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:39:16.877 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 7713, 2023-11-08T16:39:16.878 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:39:16.878 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:39:16.878 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:39:16.878 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 6.6445823339999999, 2023-11-08T16:39:16.878 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 86, 2023-11-08T16:39:16.879 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 103, 2023-11-08T16:39:16.879 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 103, 2023-11-08T16:39:16.879 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 13.085653141 2023-11-08T16:39:16.879 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:39:16.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 -- 172.21.15.81:0/4201020422 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda4406ae70 msgr2=0x7fda4406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda4406ae70 0x7fda4406d330 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fda4c036000 tx=0x7fda4c03c040 comp rx=0 tx=0).stop 2023-11-08T16:39:16.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 -- 172.21.15.81:0/4201020422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda580c9580 msgr2=0x7fda58158d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:16.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda580c9580 0x7fda58158d00 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fda4800c430 tx=0x7fda4800c8f0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 -- 172.21.15.81:0/4201020422 shutdown_connections 2023-11-08T16:39:16.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda4406ae70 0x7fda4406d330 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fda5815aeb0 0x7fda580cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda580c9580 0x7fda58158d00 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 --2- 172.21.15.81:0/4201020422 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda580c90f0 0x7fda580c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:16.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 -- 172.21.15.81:0/4201020422 >> 172.21.15.81:0/4201020422 conn(0x7fda580ceaa0 msgr2=0x7fda580b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:16.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 -- 172.21.15.81:0/4201020422 shutdown_connections 2023-11-08T16:39:16.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:16.861+0000 7fda5e2af640 1 -- 172.21.15.81:0/4201020422 wait complete. 2023-11-08T16:39:16.884 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:39:17.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.093+0000 7fc33d84b640 1 Processor -- start 2023-11-08T16:39:17.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.093+0000 7fc33d84b640 1 -- start start 2023-11-08T16:39:17.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.094+0000 7fc33d84b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc3380ade50 0x7fc3380ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:17.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc33d84b640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fc33815a680 0x7fc33815ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:17.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc33d84b640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc33806b630 con 0x7fc33815b050 2023-11-08T16:39:17.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc33d84b640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3380b1860 con 0x7fc3380ade50 2023-11-08T16:39:17.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc33d84b640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3380ca9c0 con 0x7fc33815a680 2023-11-08T16:39:17.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc336ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc3380ade50 0x7fc3380ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:17.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc336ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc3380ade50 0x7fc3380ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42728/0 (socket says 172.21.15.81:42728) 2023-11-08T16:39:17.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc336ffd640 1 -- 172.21.15.81:0/944121297 learned_addr learned my addr 172.21.15.81:0/944121297 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:17.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc3367fc640 1 --2- 172.21.15.81:0/944121297 >> v2:172.21.15.93:3300/0 conn(0x7fc33815a680 0x7fc33815ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:17.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc336ffd640 1 -- 172.21.15.81:0/944121297 >> v2:172.21.15.93:3300/0 conn(0x7fc33815a680 msgr2=0x7fc33815ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:17.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc336ffd640 1 --2- 172.21.15.81:0/944121297 >> v2:172.21.15.93:3300/0 conn(0x7fc33815a680 0x7fc33815ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:17.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc336ffd640 1 -- 172.21.15.81:0/944121297 >> v1:172.21.15.93:6789/0 conn(0x7fc33815b050 legacy=0x7fc3380c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:17.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc336ffd640 1 -- 172.21.15.81:0/944121297 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3380cabb0 con 0x7fc3380ade50 2023-11-08T16:39:17.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.095+0000 7fc3367fc640 1 --2- 172.21.15.81:0/944121297 >> v2:172.21.15.93:3300/0 conn(0x7fc33815a680 0x7fc33815ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:17.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.097+0000 7fc336ffd640 1 --2- 172.21.15.81:0/944121297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3380ade50 0x7fc3380ca480 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc338073d00 tx=0x7fc32802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474b14400711ea3a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:17.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.097+0000 7fc335ffb640 1 -- 172.21.15.81:0/944121297 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328031870 con 0x7fc3380ade50 2023-11-08T16:39:17.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.097+0000 7fc335ffb640 1 -- 172.21.15.81:0/944121297 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3280319d0 con 0x7fc3380ade50 2023-11-08T16:39:17.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.097+0000 7fc335ffb640 1 -- 172.21.15.81:0/944121297 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc328031d60 con 0x7fc3380ade50 2023-11-08T16:39:17.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.098+0000 7fc33d84b640 1 -- 172.21.15.81:0/944121297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3380ade50 msgr2=0x7fc3380ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:17.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.098+0000 7fc33d84b640 1 --2- 172.21.15.81:0/944121297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3380ade50 0x7fc3380ca480 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fc338073d00 tx=0x7fc32802b910 comp rx=0 tx=0).stop 2023-11-08T16:39:17.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.098+0000 7fc33d84b640 1 -- 172.21.15.81:0/944121297 shutdown_connections 2023-11-08T16:39:17.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.098+0000 7fc33d84b640 1 --2- 172.21.15.81:0/944121297 >> v2:172.21.15.93:3300/0 conn(0x7fc33815a680 0x7fc33815ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:17.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.098+0000 7fc33d84b640 1 --2- 172.21.15.81:0/944121297 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc3380ade50 0x7fc3380ca480 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:17.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.098+0000 7fc33d84b640 1 -- 172.21.15.81:0/944121297 >> 172.21.15.81:0/944121297 conn(0x7fc3380cb490 msgr2=0x7fc3380cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:17.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.098+0000 7fc33d84b640 1 -- 172.21.15.81:0/944121297 shutdown_connections 2023-11-08T16:39:17.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.098+0000 7fc33d84b640 1 -- 172.21.15.81:0/944121297 wait complete. 2023-11-08T16:39:17.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.100+0000 7fc33d84b640 1 Processor -- start 2023-11-08T16:39:17.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.101+0000 7fc33d84b640 1 -- start start 2023-11-08T16:39:17.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc33d84b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3380ade50 0x7fc3381535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:17.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc33d84b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc33815a680 0x7fc3381540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:17.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc33d84b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc33815b050 0x7fc3380c4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:17.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc33d84b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338153190 con 0x7fc33815b050 2023-11-08T16:39:17.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc33d84b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc338152510 con 0x7fc3380ade50 2023-11-08T16:39:17.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc33d84b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3381527f0 con 0x7fc33815a680 2023-11-08T16:39:17.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc3377fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc33815b050 0x7fc3380c4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:17.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc3377fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc33815b050 0x7fc3380c4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42730/0 (socket says 172.21.15.81:42730) 2023-11-08T16:39:17.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc3377fe640 1 -- 172.21.15.81:0/1072064895 learned_addr learned my addr 172.21.15.81:0/1072064895 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:17.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc336ffd640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3380ade50 0x7fc3381535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:17.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc3377fe640 1 -- 172.21.15.81:0/1072064895 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc33815a680 msgr2=0x7fc3381540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:17.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc3377fe640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc33815a680 0x7fc3381540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:17.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc3377fe640 1 -- 172.21.15.81:0/1072064895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3380ade50 msgr2=0x7fc3381535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:17.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc3377fe640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3380ade50 0x7fc3381535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:17.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc3377fe640 1 -- 172.21.15.81:0/1072064895 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3280094d0 con 0x7fc33815b050 2023-11-08T16:39:17.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.102+0000 7fc336ffd640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3380ade50 0x7fc3381535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:17.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.103+0000 7fc3377fe640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc33815b050 0x7fc3380c4f00 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc33000b990 tx=0x7fc33000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:17.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.103+0000 7fc31ffff640 1 -- 172.21.15.81:0/1072064895 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33000cba0 con 0x7fc33815b050 2023-11-08T16:39:17.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.103+0000 7fc31ffff640 1 -- 172.21.15.81:0/1072064895 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33000cd00 con 0x7fc33815b050 2023-11-08T16:39:17.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.103+0000 7fc31ffff640 1 -- 172.21.15.81:0/1072064895 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc330014680 con 0x7fc33815b050 2023-11-08T16:39:17.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.103+0000 7fc33d84b640 1 -- 172.21.15.81:0/1072064895 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc338152ad0 con 0x7fc33815b050 2023-11-08T16:39:17.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.103+0000 7fc33d84b640 1 -- 172.21.15.81:0/1072064895 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3381545f0 con 0x7fc33815b050 2023-11-08T16:39:17.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.104+0000 7fc33d84b640 1 -- 172.21.15.81:0/1072064895 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3381532d0 con 0x7fc33815b050 2023-11-08T16:39:17.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.106+0000 7fc31ffff640 1 -- 172.21.15.81:0/1072064895 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc3300147e0 con 0x7fc33815b050 2023-11-08T16:39:17.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.106+0000 7fc31ffff640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc32406ae70 0x7fc32406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:17.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.107+0000 7fc31ffff640 1 -- 172.21.15.81:0/1072064895 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3300894c0 con 0x7fc33815b050 2023-11-08T16:39:17.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.107+0000 7fc336ffd640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc32406ae70 0x7fc32406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:17.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.107+0000 7fc336ffd640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc32406ae70 0x7fc32406d330 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc328036000 tx=0x7fc32803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:17.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.112+0000 7fc31ffff640 1 -- 172.21.15.81:0/1072064895 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc330054600 con 0x7fc33815b050 2023-11-08T16:39:17.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.347+0000 7fc33d84b640 1 -- 172.21.15.81:0/1072064895 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc33804ece0 con 0x7fc33815b050 2023-11-08T16:39:17.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.348+0000 7fc31ffff640 1 -- 172.21.15.81:0/1072064895 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v86) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc330019060 con 0x7fc33815b050 2023-11-08T16:39:17.349 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T16:39:17.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 -- 172.21.15.81:0/1072064895 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc32406ae70 msgr2=0x7fc32406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:17.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc32406ae70 0x7fc32406d330 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc328036000 tx=0x7fc32803c040 comp rx=0 tx=0).stop 2023-11-08T16:39:17.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 -- 172.21.15.81:0/1072064895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc33815b050 msgr2=0x7fc3380c4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:17.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc33815b050 0x7fc3380c4f00 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fc33000b990 tx=0x7fc33000be50 comp rx=0 tx=0).stop 2023-11-08T16:39:17.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 -- 172.21.15.81:0/1072064895 shutdown_connections 2023-11-08T16:39:17.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc32406ae70 0x7fc32406d330 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:17.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc33815b050 0x7fc3380c4f00 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:17.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc33815a680 0x7fc3381540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:17.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 --2- 172.21.15.81:0/1072064895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3380ade50 0x7fc3381535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:17.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 -- 172.21.15.81:0/1072064895 >> 172.21.15.81:0/1072064895 conn(0x7fc3380cb490 msgr2=0x7fc3380b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:17.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 -- 172.21.15.81:0/1072064895 shutdown_connections 2023-11-08T16:39:17.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:17.351+0000 7fc33d84b640 1 -- 172.21.15.81:0/1072064895 wait complete. 2023-11-08T16:39:17.370 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:39:17.370 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:39:17.371 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:39:17.371 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:39:17.371 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:39:17.372 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:39:17.372 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:39:17.373 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:39:17.373 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:39:17.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:17.374 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:39:17.429 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:39:17.429 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:39:17.430 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:39:17.430 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:39:17.430 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:39:17.430 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:39:17.430 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:39:17.431 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:39:17.431 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:39:17.431 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:39:17.431 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:39:17.431 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:39:17.431 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:39:17.432 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:39:17.432 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:39:17.432 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:39:17.432 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:39:17.432 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 41752sec preferred_lft 41752sec 2023-11-08T16:39:17.433 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:39:17.433 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:39:17.433 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:39:17.433 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff 2023-11-08T16:39:17.433 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:39:17.434 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:39:17.434 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:39:17.434 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:39:17.434 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:39:17.434 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:39:17.434 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:39:17.435 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:39:17.435 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:39:17.435 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:39:17.435 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::2cac:17ff:fe64:67da/64 scope link 2023-11-08T16:39:17.435 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:39:17.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:17.437 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:39:17.525 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:39:17.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:17.530 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:39:17.580 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:39:17.581 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:39:17.581 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:39:17.581 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:39:17.581 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:39:17.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:17.583 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:39:17.583 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:39:17.584 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:39:17.584 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:39:17.584 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:39:17.585 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:39:17.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:17.772 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:39:17.824 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:39:17.825 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:39:17.826 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:39:17.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:17.827 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:39:17.925 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:39:17.925 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:39:17.926 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:39:17.980 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:39:17.981 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:39:17.981 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:39:18.031 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:39:18.114 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:39:18.171 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:39:18.221 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:39:18.221 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:39:18.221 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:39:18.221 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:39:18.222 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:39:18.222 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:39:18.222 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:39:18.222 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:39:18.222 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:39:18.223 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:39:18.223 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:39:18.223 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:39:18.223 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:39:18.223 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:39:18.224 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:39:18.224 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:39:18.224 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:39:18.224 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:39:18.224 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:39:18.225 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:39:18.225 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:39:18.225 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:39:18.225 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:39:18.225 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:39:18.226 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:39:18.226 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:39:18.226 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:39:18.226 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:39:18.226 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:39:18.227 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:39:18.227 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:39:18.227 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:39:18.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:18.228 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:39:18.287 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:39:18.288 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:39:18.293 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:39:18.371 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:39:18.414 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:39:18.414 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:39:18.415 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:39:18.415 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:39:18.415 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:39:18.415 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:39:18.415 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:39:18.416 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:39:18.416 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:39:18.416 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:39:18.416 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:39:18.417 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:39:18.417 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:39:18.417 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:39:18.417 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:39:18.417 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:39:18.418 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:39:18.418 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:39:18.418 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:39:18.418 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:39:18.419 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:39:18.419 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:39:18.419 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:39:18.419 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:39:18.420 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:39:18.420 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:39:18.420 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:39:18.420 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:39:18.420 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:39:18.421 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:39:18.421 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:39:18.421 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:39:18.421 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:39:18.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:18.423 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:39:18.479 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:39:18.478+0000 7f293c437480 -1 init, newargv = 0x561d73e15120 newargc=15 2023-11-08T16:39:18.479 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[119215]: starting ceph client 2023-11-08T16:39:18.490 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[119215]: starting fuse 2023-11-08T16:39:18.509 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:39:18.509 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:39:18.510 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:39:18.510 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:39:18.511 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:39:18.511 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:39:18.511 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:39:18.512 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:39:18.512 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:39:18.512 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:39:18.513 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:39:18.513 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:39:18.514 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:39:18.514 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:39:18.514 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:39:18.515 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:39:18.515 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:39:18.515 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:39:18.516 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:39:18.516 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:39:18.517 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:39:18.517 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:39:18.517 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:39:18.518 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:39:18.518 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:39:18.519 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:39:18.519 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:39:18.519 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:39:18.520 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:18.714 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.119215.asok 2023-11-08T16:39:18.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:18.732 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.119215.asok 2023-11-08T16:39:18.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:18.732 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.119215.asok status 2023-11-08T16:39:18.892 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:39:18.902 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:39:18.975 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:39:18.975 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:39:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:39:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:39:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:39:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:39:18.976 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:39:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "119215", 2023-11-08T16:39:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:39:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:39:18.977 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:39:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 7872, 2023-11-08T16:39:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:39:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:39:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:39:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 7872 2023-11-08T16:39:18.978 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:39:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:39:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:39:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3425294959 2023-11-08T16:39:18.979 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:39:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:39:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:39:18.980 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 3425294959 2023-11-08T16:39:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:39:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.7872 192.168.0.1:0/3425294959", 2023-11-08T16:39:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/3425294959", 2023-11-08T16:39:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:39:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 86, 2023-11-08T16:39:18.981 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 103, 2023-11-08T16:39:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:39:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:39:18.982 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:39:18.982 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:39:18.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:18.990 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:39:18.990 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:39:19.033 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:39:19.034 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:39:19.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:19.034 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:39:19.173 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_extra --pg_num_min 64 2023-11-08T16:39:19.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.392+0000 7f3b7cb25640 1 Processor -- start 2023-11-08T16:39:19.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.393+0000 7f3b7cb25640 1 -- start start 2023-11-08T16:39:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.394+0000 7f3b7cb25640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b780714b0 0x7f3b7815a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:19.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.394+0000 7f3b7cb25640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b78071890 0x7f3b7815ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:19.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b7cb25640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b7815b340 con 0x7f3b780adf00 2023-11-08T16:39:19.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b7cb25640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b7806b250 con 0x7f3b780714b0 2023-11-08T16:39:19.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b7cb25640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b780b09f0 con 0x7f3b78071890 2023-11-08T16:39:19.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b7659c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b780714b0 0x7f3b7815a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:19.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b76d9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3b780adf00 0x7f3b780c90b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:34236/0 (socket says 172.21.15.81:34236) 2023-11-08T16:39:19.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b76d9d640 1 -- 172.21.15.81:0/329691607 learned_addr learned my addr 172.21.15.81:0/329691607 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:19.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b6ffff640 1 --2- 172.21.15.81:0/329691607 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b78071890 0x7f3b7815ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:19.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b75d9b640 1 -- 172.21.15.81:0/329691607 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922448276 0 0) 0x7f3b7815b340 con 0x7f3b780adf00 2023-11-08T16:39:19.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b75d9b640 1 -- 172.21.15.81:0/329691607 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b64003520 con 0x7f3b780adf00 2023-11-08T16:39:19.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b7659c640 1 -- 172.21.15.81:0/329691607 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b78071890 msgr2=0x7f3b7815ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:19.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b7659c640 1 --2- 172.21.15.81:0/329691607 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b78071890 0x7f3b7815ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b7659c640 1 -- 172.21.15.81:0/329691607 >> v1:172.21.15.81:6789/0 conn(0x7f3b780adf00 legacy=0x7f3b780c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b7659c640 1 -- 172.21.15.81:0/329691607 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b780ca360 con 0x7f3b780714b0 2023-11-08T16:39:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.395+0000 7f3b6ffff640 1 --2- 172.21.15.81:0/329691607 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b78071890 0x7f3b7815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:19.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.396+0000 7f3b7659c640 1 --2- 172.21.15.81:0/329691607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b780714b0 0x7f3b7815a8c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f3b680097a0 tx=0x7f3b6802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb8395b1c9d36bf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:19.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.397+0000 7f3b75d9b640 1 -- 172.21.15.81:0/329691607 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6802d040 con 0x7f3b780714b0 2023-11-08T16:39:19.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.397+0000 7f3b75d9b640 1 -- 172.21.15.81:0/329691607 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b68031810 con 0x7f3b780714b0 2023-11-08T16:39:19.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.397+0000 7f3b75d9b640 1 -- 172.21.15.81:0/329691607 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68031ba0 con 0x7f3b780714b0 2023-11-08T16:39:19.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.397+0000 7f3b7cb25640 1 -- 172.21.15.81:0/329691607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b780714b0 msgr2=0x7f3b7815a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:19.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.397+0000 7f3b7cb25640 1 --2- 172.21.15.81:0/329691607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b780714b0 0x7f3b7815a8c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f3b680097a0 tx=0x7f3b6802b8b0 comp rx=0 tx=0).stop 2023-11-08T16:39:19.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.398+0000 7f3b7cb25640 1 -- 172.21.15.81:0/329691607 shutdown_connections 2023-11-08T16:39:19.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.398+0000 7f3b7cb25640 1 --2- 172.21.15.81:0/329691607 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b78071890 0x7f3b7815ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:19.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.398+0000 7f3b7cb25640 1 --2- 172.21.15.81:0/329691607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b780714b0 0x7f3b7815a8c0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:19.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.398+0000 7f3b7cb25640 1 -- 172.21.15.81:0/329691607 >> 172.21.15.81:0/329691607 conn(0x7f3b780cb4e0 msgr2=0x7f3b780cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:19.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.398+0000 7f3b7cb25640 1 -- 172.21.15.81:0/329691607 shutdown_connections 2023-11-08T16:39:19.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.398+0000 7f3b7cb25640 1 -- 172.21.15.81:0/329691607 wait complete. 2023-11-08T16:39:19.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.400+0000 7f3b7cb25640 1 Processor -- start 2023-11-08T16:39:19.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.400+0000 7f3b7cb25640 1 -- start start 2023-11-08T16:39:19.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.401+0000 7f3b7cb25640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b780714b0 0x7f3b780b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:19.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.401+0000 7f3b7cb25640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b78071890 0x7f3b7815c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:19.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.401+0000 7f3b7cb25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b780adf00 0x7f3b78148a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:19.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.401+0000 7f3b7cb25640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7815bcb0 con 0x7f3b780adf00 2023-11-08T16:39:19.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.401+0000 7f3b7cb25640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7815b0a0 con 0x7f3b78071890 2023-11-08T16:39:19.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.401+0000 7f3b7cb25640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7815b380 con 0x7f3b780714b0 2023-11-08T16:39:19.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.401+0000 7f3b76d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b780adf00 0x7f3b78148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:19.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.401+0000 7f3b76d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b780adf00 0x7f3b78148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42754/0 (socket says 172.21.15.81:42754) 2023-11-08T16:39:19.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.401+0000 7f3b76d9d640 1 -- 172.21.15.81:0/2430490585 learned_addr learned my addr 172.21.15.81:0/2430490585 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:19.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.401+0000 7f3b7659c640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b780714b0 0x7f3b780b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:19.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.401+0000 7f3b6ffff640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b78071890 0x7f3b7815c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:19.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b76d9d640 1 -- 172.21.15.81:0/2430490585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b780714b0 msgr2=0x7f3b780b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:19.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b76d9d640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b780714b0 0x7f3b780b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:19.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b76d9d640 1 -- 172.21.15.81:0/2430490585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b78071890 msgr2=0x7f3b7815c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:19.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b76d9d640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b78071890 0x7f3b7815c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:19.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b76d9d640 1 -- 172.21.15.81:0/2430490585 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b68009400 con 0x7f3b780adf00 2023-11-08T16:39:19.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b6ffff640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b78071890 0x7f3b7815c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:19.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b76d9d640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b780adf00 0x7f3b78148a70 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000cae0 tx=0x7f3b7000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:19.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b6effd640 1 -- 172.21.15.81:0/2430490585 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7000ac40 con 0x7f3b780adf00 2023-11-08T16:39:19.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b6effd640 1 -- 172.21.15.81:0/2430490585 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7000ada0 con 0x7f3b780adf00 2023-11-08T16:39:19.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b6effd640 1 -- 172.21.15.81:0/2430490585 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70012600 con 0x7f3b780adf00 2023-11-08T16:39:19.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b7cb25640 1 -- 172.21.15.81:0/2430490585 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7815b660 con 0x7f3b780adf00 2023-11-08T16:39:19.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.402+0000 7f3b7cb25640 1 -- 172.21.15.81:0/2430490585 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7815c140 con 0x7f3b780adf00 2023-11-08T16:39:19.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.403+0000 7f3b7cb25640 1 -- 172.21.15.81:0/2430490585 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b780c9670 con 0x7f3b780adf00 2023-11-08T16:39:19.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.404+0000 7f3b6effd640 1 -- 172.21.15.81:0/2430490585 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3b70007070 con 0x7f3b780adf00 2023-11-08T16:39:19.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.405+0000 7f3b6effd640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b5806ae70 0x7f3b5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:19.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.405+0000 7f3b6effd640 1 -- 172.21.15.81:0/2430490585 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b700894c0 con 0x7f3b780adf00 2023-11-08T16:39:19.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.406+0000 7f3b7659c640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b5806ae70 0x7f3b5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:19.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.407+0000 7f3b7659c640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b5806ae70 0x7f3b5806d330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3b68036000 tx=0x7f3b6803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:19.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.410+0000 7f3b6effd640 1 -- 172.21.15.81:0/2430490585 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b70054580 con 0x7f3b780adf00 2023-11-08T16:39:19.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:19.601+0000 7f3b7cb25640 1 -- 172.21.15.81:0/2430490585 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64} v 0) v1 -- 0x7f3b780b0e60 con 0x7f3b780adf00 2023-11-08T16:39:20.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.284+0000 7f3b6effd640 1 -- 172.21.15.81:0/2430490585 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_extra", "pg_num_min": 64}]=0 pool 'cephfs_data_extra' created v104) v1 ==== 142+0+0 (secure 0 0 0) 0x7f3b780b0e60 con 0x7f3b780adf00 2023-11-08T16:39:20.286 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data_extra' created 2023-11-08T16:39:20.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 -- 172.21.15.81:0/2430490585 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b5806ae70 msgr2=0x7f3b5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:20.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b5806ae70 0x7f3b5806d330 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3b68036000 tx=0x7f3b6803c040 comp rx=0 tx=0).stop 2023-11-08T16:39:20.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 -- 172.21.15.81:0/2430490585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b780adf00 msgr2=0x7f3b78148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:20.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b780adf00 0x7f3b78148a70 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f3b7000cae0 tx=0x7f3b7000a060 comp rx=0 tx=0).stop 2023-11-08T16:39:20.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 -- 172.21.15.81:0/2430490585 shutdown_connections 2023-11-08T16:39:20.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b5806ae70 0x7f3b5806d330 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:20.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b780adf00 0x7f3b78148a70 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:20.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b78071890 0x7f3b7815c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:20.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 --2- 172.21.15.81:0/2430490585 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b780714b0 0x7f3b780b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:20.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 -- 172.21.15.81:0/2430490585 >> 172.21.15.81:0/2430490585 conn(0x7f3b780cb4e0 msgr2=0x7f3b780c7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:20.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 -- 172.21.15.81:0/2430490585 shutdown_connections 2023-11-08T16:39:20.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.288+0000 7f3b7cb25640 1 -- 172.21.15.81:0/2430490585 wait complete. 2023-11-08T16:39:20.316 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_extra 2023-11-08T16:39:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.511+0000 7f515473c640 1 Processor -- start 2023-11-08T16:39:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.511+0000 7f515473c640 1 -- start start 2023-11-08T16:39:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.513+0000 7f515473c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f514c159b40 0x7f514c06cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.513+0000 7f515473c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f514c0c9830 0x7f514c15dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.513+0000 7f515473c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f514c0c9dd0 con 0x7f514c159f20 2023-11-08T16:39:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.513+0000 7f515473c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f514c06d2e0 con 0x7f514c0c9830 2023-11-08T16:39:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.513+0000 7f515473c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0ca0b0 con 0x7f514c159b40 2023-11-08T16:39:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.513+0000 7f5152cd9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f514c0c9830 0x7f514c15dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.513+0000 7f5152cd9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f514c0c9830 0x7f514c15dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42756/0 (socket says 172.21.15.81:42756) 2023-11-08T16:39:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.514+0000 7f5152cd9640 1 -- 172.21.15.81:0/156181866 learned_addr learned my addr 172.21.15.81:0/156181866 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.514+0000 7f51524d8640 1 --2- 172.21.15.81:0/156181866 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f514c159b40 0x7f514c06cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:20.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.514+0000 7f5152cd9640 1 -- 172.21.15.81:0/156181866 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f514c159b40 msgr2=0x7f514c06cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:20.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.514+0000 7f5152cd9640 1 --2- 172.21.15.81:0/156181866 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f514c159b40 0x7f514c06cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:20.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.514+0000 7f5152cd9640 1 -- 172.21.15.81:0/156181866 >> v1:172.21.15.93:6789/0 conn(0x7f514c159f20 legacy=0x7f514c0700d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.514+0000 7f5152cd9640 1 -- 172.21.15.81:0/156181866 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0ca390 con 0x7f514c0c9830 2023-11-08T16:39:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.515+0000 7f5152cd9640 1 --2- 172.21.15.81:0/156181866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f514c0c9830 0x7f514c15dea0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5144009e50 tx=0x7f514402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d67a9aecbcc5bdf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.515+0000 7f51514d6640 1 -- 172.21.15.81:0/156181866 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514402cb00 con 0x7f514c0c9830 2023-11-08T16:39:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.515+0000 7f51514d6640 1 -- 172.21.15.81:0/156181866 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514402cc60 con 0x7f514c0c9830 2023-11-08T16:39:20.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.516+0000 7f51514d6640 1 -- 172.21.15.81:0/156181866 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51440343f0 con 0x7f514c0c9830 2023-11-08T16:39:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.516+0000 7f515473c640 1 -- 172.21.15.81:0/156181866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f514c0c9830 msgr2=0x7f514c15dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.516+0000 7f515473c640 1 --2- 172.21.15.81:0/156181866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f514c0c9830 0x7f514c15dea0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f5144009e50 tx=0x7f514402bf60 comp rx=0 tx=0).stop 2023-11-08T16:39:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.516+0000 7f515473c640 1 -- 172.21.15.81:0/156181866 shutdown_connections 2023-11-08T16:39:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.516+0000 7f515473c640 1 --2- 172.21.15.81:0/156181866 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f514c0c9830 0x7f514c15dea0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:20.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.516+0000 7f515473c640 1 --2- 172.21.15.81:0/156181866 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f514c159b40 0x7f514c06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.516+0000 7f515473c640 1 -- 172.21.15.81:0/156181866 >> 172.21.15.81:0/156181866 conn(0x7f514c0b05b0 msgr2=0x7f514c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.516+0000 7f515473c640 1 -- 172.21.15.81:0/156181866 shutdown_connections 2023-11-08T16:39:20.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.516+0000 7f515473c640 1 -- 172.21.15.81:0/156181866 wait complete. 2023-11-08T16:39:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.518+0000 7f515473c640 1 Processor -- start 2023-11-08T16:39:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.518+0000 7f515473c640 1 -- start start 2023-11-08T16:39:20.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.519+0000 7f515473c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f514c0c9830 0x7f514c0c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:20.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.519+0000 7f515473c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f514c159b40 0x7f514c0c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:20.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.519+0000 7f515473c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f514c159f20 0x7f514c1f2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:20.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.519+0000 7f515473c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0c6430 con 0x7f514c0c9830 2023-11-08T16:39:20.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.519+0000 7f515473c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f514c0affc0 con 0x7f514c159f20 2023-11-08T16:39:20.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.519+0000 7f515473c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f514c1f2cc0 con 0x7f514c159b40 2023-11-08T16:39:20.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.519+0000 7f51524d8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f514c0c9830 0x7f514c0c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:20.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.519+0000 7f51524d8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f514c0c9830 0x7f514c0c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:42758/0 (socket says 172.21.15.81:42758) 2023-11-08T16:39:20.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.519+0000 7f51524d8640 1 -- 172.21.15.81:0/814977801 learned_addr learned my addr 172.21.15.81:0/814977801 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:20.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f5152cd9640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f514c159f20 0x7f514c1f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:20.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f5151cd7640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f514c159b40 0x7f514c0c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:20.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f51524d8640 1 -- 172.21.15.81:0/814977801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f514c159b40 msgr2=0x7f514c0c5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:20.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f51524d8640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f514c159b40 0x7f514c0c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:20.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f51524d8640 1 -- 172.21.15.81:0/814977801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f514c159f20 msgr2=0x7f514c1f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:20.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f51524d8640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f514c159f20 0x7f514c1f2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:20.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f51524d8640 1 -- 172.21.15.81:0/814977801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5144009b30 con 0x7f514c0c9830 2023-11-08T16:39:20.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f51524d8640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f514c0c9830 0x7f514c0c8760 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f514800d620 tx=0x7f514800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:20.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f5142ffd640 1 -- 172.21.15.81:0/814977801 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514800b620 con 0x7f514c0c9830 2023-11-08T16:39:20.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f5142ffd640 1 -- 172.21.15.81:0/814977801 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f514800b780 con 0x7f514c0c9830 2023-11-08T16:39:20.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f5142ffd640 1 -- 172.21.15.81:0/814977801 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514800bb30 con 0x7f514c0c9830 2023-11-08T16:39:20.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.520+0000 7f515473c640 1 -- 172.21.15.81:0/814977801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f514c0c71f0 con 0x7f514c0c9830 2023-11-08T16:39:20.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.521+0000 7f515473c640 1 -- 172.21.15.81:0/814977801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f514c0c6950 con 0x7f514c0c9830 2023-11-08T16:39:20.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.522+0000 7f515473c640 1 -- 172.21.15.81:0/814977801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514c04ece0 con 0x7f514c0c9830 2023-11-08T16:39:20.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.523+0000 7f5142ffd640 1 -- 172.21.15.81:0/814977801 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f514800bc90 con 0x7f514c0c9830 2023-11-08T16:39:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.524+0000 7f5142ffd640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513006ae70 0x7f513006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.524+0000 7f5142ffd640 1 -- 172.21.15.81:0/814977801 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f514808a790 con 0x7f514c0c9830 2023-11-08T16:39:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.524+0000 7f5151cd7640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513006ae70 0x7f513006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:20.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.525+0000 7f5151cd7640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513006ae70 0x7f513006d330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f514c159310 tx=0x7f513c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:20.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.528+0000 7f5142ffd640 1 -- 172.21.15.81:0/814977801 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f51480556a0 con 0x7f514c0c9830 2023-11-08T16:39:20.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:20.758+0000 7f515473c640 1 -- 172.21.15.81:0/814977801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"} v 0) v1 -- 0x7f514c0ccc60 con 0x7f514c0c9830 2023-11-08T16:39:21.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.304+0000 7f5142ffd640 1 -- 172.21.15.81:0/814977801 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_extra"}]=0 added data pool 15 to fsmap v87) v1 ==== 141+0+0 (secure 0 0 0) 0x7f5148058cc0 con 0x7f514c0c9830 2023-11-08T16:39:21.309 INFO:teuthology.orchestra.run.smithi081.stderr:added data pool 15 to fsmap 2023-11-08T16:39:21.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.312+0000 7f515473c640 1 -- 172.21.15.81:0/814977801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513006ae70 msgr2=0x7f513006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.312+0000 7f515473c640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513006ae70 0x7f513006d330 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f514c159310 tx=0x7f513c009040 comp rx=0 tx=0).stop 2023-11-08T16:39:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.312+0000 7f515473c640 1 -- 172.21.15.81:0/814977801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f514c0c9830 msgr2=0x7f514c0c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.312+0000 7f515473c640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f514c0c9830 0x7f514c0c8760 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f514800d620 tx=0x7f514800dae0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.313+0000 7f515473c640 1 -- 172.21.15.81:0/814977801 shutdown_connections 2023-11-08T16:39:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.313+0000 7f515473c640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513006ae70 0x7f513006d330 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.313+0000 7f515473c640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f514c159f20 0x7f514c1f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.313+0000 7f515473c640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f514c159b40 0x7f514c0c5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.313+0000 7f515473c640 1 --2- 172.21.15.81:0/814977801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f514c0c9830 0x7f514c0c8760 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.313+0000 7f515473c640 1 -- 172.21.15.81:0/814977801 >> 172.21.15.81:0/814977801 conn(0x7f514c0b05b0 msgr2=0x7f514c071590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.314+0000 7f515473c640 1 -- 172.21.15.81:0/814977801 shutdown_connections 2023-11-08T16:39:21.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.314+0000 7f515473c640 1 -- 172.21.15.81:0/814977801 wait complete. 2023-11-08T16:39:21.339 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:21.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.523+0000 7f7c6d22e640 1 Processor -- start 2023-11-08T16:39:21.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.523+0000 7f7c6d22e640 1 -- start start 2023-11-08T16:39:21.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.525+0000 7f7c6d22e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7c6806d5d0 0x7f7c68165540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.525+0000 7f7c6d22e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c68165a80 0x7f7c681629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.525+0000 7f7c6d22e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c68162050 con 0x7f7c6806b850 2023-11-08T16:39:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.525+0000 7f7c6d22e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c68164e10 con 0x7f7c6806d5d0 2023-11-08T16:39:21.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.525+0000 7f7c6d22e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c68162330 con 0x7f7c68165a80 2023-11-08T16:39:21.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.526+0000 7f7c677fe640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7c6806d5d0 0x7f7c68165540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:21.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.526+0000 7f7c677fe640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7c6806d5d0 0x7f7c68165540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:37732/0 (socket says 172.21.15.81:37732) 2023-11-08T16:39:21.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.526+0000 7f7c6ca2d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c68165a80 0x7f7c681629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:21.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.526+0000 7f7c677fe640 1 -- 172.21.15.81:0/35426266 learned_addr learned my addr 172.21.15.81:0/35426266 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:21.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.526+0000 7f7c677fe640 1 -- 172.21.15.81:0/35426266 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c68165a80 msgr2=0x7f7c681629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:21.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.526+0000 7f7c677fe640 1 --2- 172.21.15.81:0/35426266 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c68165a80 0x7f7c681629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.526+0000 7f7c677fe640 1 -- 172.21.15.81:0/35426266 >> v1:172.21.15.93:6789/0 conn(0x7f7c6806b850 legacy=0x7f7c68161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:21.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.526+0000 7f7c677fe640 1 -- 172.21.15.81:0/35426266 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c68162610 con 0x7f7c6806d5d0 2023-11-08T16:39:21.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.528+0000 7f7c677fe640 1 --2- 172.21.15.81:0/35426266 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c6806d5d0 0x7f7c68165540 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009e50 tx=0x7f7c5802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe3617d92a017f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:21.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.528+0000 7f7c66ffd640 1 -- 172.21.15.81:0/35426266 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5802cb00 con 0x7f7c6806d5d0 2023-11-08T16:39:21.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.528+0000 7f7c66ffd640 1 -- 172.21.15.81:0/35426266 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5802cc60 con 0x7f7c6806d5d0 2023-11-08T16:39:21.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.528+0000 7f7c66ffd640 1 -- 172.21.15.81:0/35426266 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c580343f0 con 0x7f7c6806d5d0 2023-11-08T16:39:21.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.529+0000 7f7c6d22e640 1 -- 172.21.15.81:0/35426266 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c6806d5d0 msgr2=0x7f7c68165540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:21.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.529+0000 7f7c6d22e640 1 --2- 172.21.15.81:0/35426266 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c6806d5d0 0x7f7c68165540 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009e50 tx=0x7f7c5802bf60 comp rx=0 tx=0).stop 2023-11-08T16:39:21.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.529+0000 7f7c6d22e640 1 -- 172.21.15.81:0/35426266 shutdown_connections 2023-11-08T16:39:21.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.529+0000 7f7c6d22e640 1 --2- 172.21.15.81:0/35426266 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c68165a80 0x7f7c681629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.529+0000 7f7c6d22e640 1 --2- 172.21.15.81:0/35426266 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c6806d5d0 0x7f7c68165540 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.529+0000 7f7c6d22e640 1 -- 172.21.15.81:0/35426266 >> 172.21.15.81:0/35426266 conn(0x7f7c6806a4a0 msgr2=0x7f7c6806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:21.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.529+0000 7f7c6d22e640 1 -- 172.21.15.81:0/35426266 shutdown_connections 2023-11-08T16:39:21.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.529+0000 7f7c6d22e640 1 -- 172.21.15.81:0/35426266 wait complete. 2023-11-08T16:39:21.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.530+0000 7f7c6d22e640 1 Processor -- start 2023-11-08T16:39:21.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.531+0000 7f7c6d22e640 1 -- start start 2023-11-08T16:39:21.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.531+0000 7f7c6d22e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c6806b850 0x7f7c68162bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:21.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.531+0000 7f7c6d22e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c6806d5d0 0x7f7c6815d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:21.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c6d22e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c68165a80 0x7f7c68155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:21.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c6d22e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6815dc30 con 0x7f7c6806d5d0 2023-11-08T16:39:21.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c6d22e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6815df10 con 0x7f7c68165a80 2023-11-08T16:39:21.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c6d22e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6815e1f0 con 0x7f7c6806b850 2023-11-08T16:39:21.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c677fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c6806d5d0 0x7f7c6815d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:21.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c677fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c6806d5d0 0x7f7c6815d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36404/0 (socket says 172.21.15.81:36404) 2023-11-08T16:39:21.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c677fe640 1 -- 172.21.15.81:0/766802727 learned_addr learned my addr 172.21.15.81:0/766802727 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:21.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c67fff640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c6806b850 0x7f7c68162bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:21.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c6ca2d640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c68165a80 0x7f7c68155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:21.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c677fe640 1 -- 172.21.15.81:0/766802727 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c6806b850 msgr2=0x7f7c68162bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:21.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c677fe640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c6806b850 0x7f7c68162bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c677fe640 1 -- 172.21.15.81:0/766802727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c68165a80 msgr2=0x7f7c68155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:21.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c677fe640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c68165a80 0x7f7c68155dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.532+0000 7f7c677fe640 1 -- 172.21.15.81:0/766802727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c58009b30 con 0x7f7c6806d5d0 2023-11-08T16:39:21.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.533+0000 7f7c677fe640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c6806d5d0 0x7f7c6815d170 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7c58000f80 tx=0x7f7c58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:21.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.533+0000 7f7c64ff9640 1 -- 172.21.15.81:0/766802727 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5803e450 con 0x7f7c6806d5d0 2023-11-08T16:39:21.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.533+0000 7f7c64ff9640 1 -- 172.21.15.81:0/766802727 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5803e5f0 con 0x7f7c6806d5d0 2023-11-08T16:39:21.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.533+0000 7f7c6d22e640 1 -- 172.21.15.81:0/766802727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6815e4d0 con 0x7f7c6806d5d0 2023-11-08T16:39:21.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.533+0000 7f7c64ff9640 1 -- 172.21.15.81:0/766802727 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5803e9a0 con 0x7f7c6806d5d0 2023-11-08T16:39:21.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.533+0000 7f7c6d22e640 1 -- 172.21.15.81:0/766802727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c68156cc0 con 0x7f7c6806d5d0 2023-11-08T16:39:21.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.534+0000 7f7c6d22e640 1 -- 172.21.15.81:0/766802727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6815e9f0 con 0x7f7c6806d5d0 2023-11-08T16:39:21.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.535+0000 7f7c64ff9640 1 -- 172.21.15.81:0/766802727 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c5803eb00 con 0x7f7c6806d5d0 2023-11-08T16:39:21.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.536+0000 7f7c64ff9640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c4806ae70 0x7f7c4806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:21.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.536+0000 7f7c64ff9640 1 -- 172.21.15.81:0/766802727 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f7c580ac5b0 con 0x7f7c6806d5d0 2023-11-08T16:39:21.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.536+0000 7f7c67fff640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c4806ae70 0x7f7c4806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:21.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.537+0000 7f7c67fff640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c4806ae70 0x7f7c4806d330 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7c50007fd0 tx=0x7f7c50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:21.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.540+0000 7f7c64ff9640 1 -- 172.21.15.81:0/766802727 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7c6815e9f0 con 0x7f7c6806d5d0 2023-11-08T16:39:21.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.765+0000 7f7c6d22e640 1 -- 172.21.15.81:0/766802727 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c6815e9f0 con 0x7f7c6806d5d0 2023-11-08T16:39:21.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.766+0000 7f7c64ff9640 1 -- 172.21.15.81:0/766802727 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4915 (secure 0 0 0) 0x7f7c6815e9f0 con 0x7f7c6806d5d0 2023-11-08T16:39:21.768 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:21.768 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":87,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1157756119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1157756119},{"type":"v1","addr":"172.21.15.93:6833","nonce":1157756119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1607105255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1607105255},{"type":"v1","addr":"172.21.15.81:6835","nonce":1607105255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3988749175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3988749175},{"type":"v1","addr":"172.21.15.81:6837","nonce":3988749175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:09.619973+0000","modified":"2023-11-08T16:39:20.760892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7713},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7713":{"gid":7713,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6835/691341796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":691341796},{"type":"v1","addr":"172.21.15.93:6835","nonce":691341796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:39:21.768 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 87 2023-11-08T16:39:21.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.768+0000 7f7c6d22e640 1 -- 172.21.15.81:0/766802727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c4806ae70 msgr2=0x7f7c4806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:21.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.768+0000 7f7c6d22e640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c4806ae70 0x7f7c4806d330 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7c50007fd0 tx=0x7f7c50009040 comp rx=0 tx=0).stop 2023-11-08T16:39:21.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.768+0000 7f7c6d22e640 1 -- 172.21.15.81:0/766802727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c6806d5d0 msgr2=0x7f7c6815d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:21.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.768+0000 7f7c6d22e640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c6806d5d0 0x7f7c6815d170 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f7c58000f80 tx=0x7f7c58004a40 comp rx=0 tx=0).stop 2023-11-08T16:39:21.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.768+0000 7f7c6d22e640 1 -- 172.21.15.81:0/766802727 shutdown_connections 2023-11-08T16:39:21.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.768+0000 7f7c6d22e640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7c4806ae70 0x7f7c4806d330 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.768+0000 7f7c6d22e640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c68165a80 0x7f7c68155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.768+0000 7f7c6d22e640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7c6806d5d0 0x7f7c6815d170 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.768+0000 7f7c6d22e640 1 --2- 172.21.15.81:0/766802727 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7c6806b850 0x7f7c68162bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:21.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.768+0000 7f7c6d22e640 1 -- 172.21.15.81:0/766802727 >> 172.21.15.81:0/766802727 conn(0x7f7c6806a4a0 msgr2=0x7f7c680cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:21.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.768+0000 7f7c6d22e640 1 -- 172.21.15.81:0/766802727 shutdown_connections 2023-11-08T16:39:21.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.769+0000 7f7c6d22e640 1 -- 172.21.15.81:0/766802727 wait complete. 2023-11-08T16:39:21.788 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:39:22.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.998+0000 7f7ba1226640 1 Processor -- start 2023-11-08T16:39:22.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:21.999+0000 7f7ba1226640 1 -- start start 2023-11-08T16:39:22.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.000+0000 7f7ba1226640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b9c0afb90 0x7f7b9c0ce870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:22.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.000+0000 7f7ba1226640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c06c3e0 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.000+0000 7f7ba1226640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c06c6c0 con 0x7f7b9c0b2720 2023-11-08T16:39:22.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.000+0000 7f7ba1226640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9c06c9a0 con 0x7f7b9c0afb90 2023-11-08T16:39:22.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.000+0000 7f7b93fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b9c0afb90 0x7f7b9c0ce870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:22.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.001+0000 7f7b93fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b9c0afb90 0x7f7b9c0ce870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36408/0 (socket says 172.21.15.81:36408) 2023-11-08T16:39:22.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.001+0000 7f7b93fff640 1 -- 172.21.15.81:0/1586711119 learned_addr learned my addr 172.21.15.81:0/1586711119 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:22.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.001+0000 7f7b9ad9d640 1 --1- 172.21.15.81:0/1586711119 >> v1:172.21.15.81:6789/0 conn(0x7f7b9c06e7c0 0x7f7b9c06d8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:43036/0 (socket says 172.21.15.81:43036) 2023-11-08T16:39:22.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.001+0000 7f7b93fff640 1 -- 172.21.15.81:0/1586711119 >> v1:172.21.15.93:6789/0 conn(0x7f7b9c0b2720 legacy=0x7f7b9c160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:22.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.001+0000 7f7b93fff640 1 -- 172.21.15.81:0/1586711119 >> v1:172.21.15.81:6789/0 conn(0x7f7b9c06e7c0 legacy=0x7f7b9c06d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:22.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.001+0000 7f7b93fff640 1 -- 172.21.15.81:0/1586711119 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c06cc80 con 0x7f7b9c0afb90 2023-11-08T16:39:22.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.002+0000 7f7b93fff640 1 --2- 172.21.15.81:0/1586711119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b9c0afb90 0x7f7b9c0ce870 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009870 tx=0x7f7b8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d985cf7659dd5a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:22.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.002+0000 7f7b9a59c640 1 -- 172.21.15.81:0/1586711119 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84031870 con 0x7f7b9c0afb90 2023-11-08T16:39:22.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.003+0000 7f7b9a59c640 1 -- 172.21.15.81:0/1586711119 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b840319d0 con 0x7f7b9c0afb90 2023-11-08T16:39:22.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.003+0000 7f7b9a59c640 1 -- 172.21.15.81:0/1586711119 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b84031d60 con 0x7f7b9c0afb90 2023-11-08T16:39:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.003+0000 7f7ba1226640 1 -- 172.21.15.81:0/1586711119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b9c0afb90 msgr2=0x7f7b9c0ce870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.003+0000 7f7ba1226640 1 --2- 172.21.15.81:0/1586711119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b9c0afb90 0x7f7b9c0ce870 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f7b84009870 tx=0x7f7b8402b910 comp rx=0 tx=0).stop 2023-11-08T16:39:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.003+0000 7f7ba1226640 1 -- 172.21.15.81:0/1586711119 shutdown_connections 2023-11-08T16:39:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.003+0000 7f7ba1226640 1 --2- 172.21.15.81:0/1586711119 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b9c0afb90 0x7f7b9c0ce870 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:22.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.003+0000 7f7ba1226640 1 -- 172.21.15.81:0/1586711119 >> 172.21.15.81:0/1586711119 conn(0x7f7b9c0cd720 msgr2=0x7f7b9c0cdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:22.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.003+0000 7f7ba1226640 1 -- 172.21.15.81:0/1586711119 shutdown_connections 2023-11-08T16:39:22.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.003+0000 7f7ba1226640 1 -- 172.21.15.81:0/1586711119 wait complete. 2023-11-08T16:39:22.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.005+0000 7f7ba1226640 1 Processor -- start 2023-11-08T16:39:22.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.005+0000 7f7ba1226640 1 -- start start 2023-11-08T16:39:22.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.006+0000 7f7ba1226640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b9c06e7c0 0x7f7b9c14fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:22.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.006+0000 7f7ba1226640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b9c0afb90 0x7f7b9c0c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:22.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.006+0000 7f7ba1226640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b9c0b2720 0x7f7b9c0c69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:22.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.006+0000 7f7ba1226640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c150bf0 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.006+0000 7f7ba1226640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c0b38c0 con 0x7f7b9c0afb90 2023-11-08T16:39:22.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.006+0000 7f7ba1226640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c0c8960 con 0x7f7b9c0b2720 2023-11-08T16:39:22.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.006+0000 7f7b9ad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b9c06e7c0 0x7f7b9c14fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:22.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.006+0000 7f7b9ad9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b9c06e7c0 0x7f7b9c14fab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36424/0 (socket says 172.21.15.81:36424) 2023-11-08T16:39:22.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.006+0000 7f7b9ad9d640 1 -- 172.21.15.81:0/4227014204 learned_addr learned my addr 172.21.15.81:0/4227014204 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:22.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b93fff640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b9c0afb90 0x7f7b9c0c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:22.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b9ad9d640 1 -- 172.21.15.81:0/4227014204 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b9c0b2720 msgr2=0x7f7b9c0c69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:22.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b9b59e640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b9c0b2720 0x7f7b9c0c69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:22.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b9ad9d640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b9c0b2720 0x7f7b9c0c69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:22.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b9ad9d640 1 -- 172.21.15.81:0/4227014204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b9c0afb90 msgr2=0x7f7b9c0c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:22.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b9ad9d640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b9c0afb90 0x7f7b9c0c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b9ad9d640 1 -- 172.21.15.81:0/4227014204 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b840094d0 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b93fff640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b9c0afb90 0x7f7b9c0c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b9ad9d640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b9c06e7c0 0x7f7b9c14fab0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c4d0 tx=0x7f7b8c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b937fe640 1 -- 172.21.15.81:0/4227014204 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c007480 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b937fe640 1 -- 172.21.15.81:0/4227014204 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b8c0075e0 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7b937fe640 1 -- 172.21.15.81:0/4227014204 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b8c0049e0 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.007+0000 7f7ba1226640 1 -- 172.21.15.81:0/4227014204 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c150530 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.008+0000 7f7ba1226640 1 -- 172.21.15.81:0/4227014204 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c14fff0 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.009+0000 7f7ba1226640 1 -- 172.21.15.81:0/4227014204 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c150390 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.010+0000 7f7b937fe640 1 -- 172.21.15.81:0/4227014204 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b8c004b40 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.011+0000 7f7b937fe640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7c06ae70 0x7f7b7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.011+0000 7f7b937fe640 1 -- 172.21.15.81:0/4227014204 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f7b8c0888d0 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.011+0000 7f7b93fff640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7c06ae70 0x7f7b7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.012+0000 7f7b93fff640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7c06ae70 0x7f7b7c06d330 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7b84036000 tx=0x7f7b8403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.015+0000 7f7b937fe640 1 -- 172.21.15.81:0/4227014204 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b9c150390 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.206+0000 7f7ba1226640 1 -- 172.21.15.81:0/4227014204 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b9c150390 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.209+0000 7f7b937fe640 1 -- 172.21.15.81:0/4227014204 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+18254 (secure 0 0 0) 0x7f7b9c150390 con 0x7f7b9c06e7c0 2023-11-08T16:39:22.211 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:22.211 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":105,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:39:20.760286+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:39:07.492834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T16:39:08.634345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:39:19.654916+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:39:22.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 -- 172.21.15.81:0/4227014204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7c06ae70 msgr2=0x7f7b7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:22.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7c06ae70 0x7f7b7c06d330 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f7b84036000 tx=0x7f7b8403c040 comp rx=0 tx=0).stop 2023-11-08T16:39:22.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 -- 172.21.15.81:0/4227014204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b9c06e7c0 msgr2=0x7f7b9c14fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:22.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b9c06e7c0 0x7f7b9c14fab0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7b8c00c4d0 tx=0x7f7b8c00c990 comp rx=0 tx=0).stop 2023-11-08T16:39:22.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 -- 172.21.15.81:0/4227014204 shutdown_connections 2023-11-08T16:39:22.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b7c06ae70 0x7f7b7c06d330 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:22.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b9c0b2720 0x7f7b9c0c69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:22.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b9c0afb90 0x7f7b9c0c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:22.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 --2- 172.21.15.81:0/4227014204 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b9c06e7c0 0x7f7b9c14fab0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:22.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 -- 172.21.15.81:0/4227014204 >> 172.21.15.81:0/4227014204 conn(0x7f7b9c0cd720 msgr2=0x7f7b9c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:22.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 -- 172.21.15.81:0/4227014204 shutdown_connections 2023-11-08T16:39:22.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:22.212+0000 7f7ba1226640 1 -- 172.21.15.81:0/4227014204 wait complete. 2023-11-08T16:39:22.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:22.233 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stripey) 2023-11-08T16:39:22.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:22.301 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.dir.layout -v 'stripe_unit=65536 stripe_count=4 object_size=262144 pool=cephfs_data_extra' ./stripey) 2023-11-08T16:39:22.364 INFO:tasks.cephfs.mount:Writing 1048576 bytes to stripey/flushed_file_0 2023-11-08T16:39:22.365 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:39:22.365 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:22.366 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-11-08T16:39:22.366 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:39:22.366 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 1048576): 2023-11-08T16:39:22.367 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:22.367 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:39:22.367 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:23.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:23.156 INFO:tasks.cephfs.mount:Validating 1048576 bytes from stripey/flushed_file_0 2023-11-08T16:39:23.156 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:39:23.157 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:23.157 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-11-08T16:39:23.158 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:39:23.158 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:39:23.158 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 1048576: 2023-11-08T16:39:23.159 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:39:23.159 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 1048576 2023-11-08T16:39:23.160 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:39:23.160 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:39:23.160 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:23.161 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:39:23.161 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:39:23.161 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:23.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:23.939 INFO:tasks.cephfs.mount:Writing 524288 bytes to stripey/flushed_file_1 2023-11-08T16:39:23.939 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:39:23.940 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:23.940 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-11-08T16:39:23.941 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:39:23.941 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 524288): 2023-11-08T16:39:23.941 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:23.942 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:39:23.942 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:24.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:24.443 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/flushed_file_1 2023-11-08T16:39:24.444 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:39:24.444 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:24.444 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-11-08T16:39:24.445 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:39:24.445 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:39:24.445 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 524288: 2023-11-08T16:39:24.446 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:39:24.446 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 524288 2023-11-08T16:39:24.447 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:39:24.447 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:39:24.447 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:24.447 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:39:24.448 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:39:24.448 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:24.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:24.971 INFO:tasks.cephfs.mount:Writing 917504 bytes to stripey/flushed_file_2 2023-11-08T16:39:24.972 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:39:24.972 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:24.973 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-11-08T16:39:24.973 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:39:24.973 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 917504): 2023-11-08T16:39:24.974 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:24.974 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:39:24.974 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:25.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:25.727 INFO:tasks.cephfs.mount:Validating 917504 bytes from stripey/flushed_file_2 2023-11-08T16:39:25.728 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:39:25.728 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:25.728 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-11-08T16:39:25.729 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:39:25.729 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:39:25.730 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 917504: 2023-11-08T16:39:25.730 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:39:25.730 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 917504 2023-11-08T16:39:25.731 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:39:25.731 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:39:25.731 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:25.732 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:39:25.732 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:39:25.732 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:26.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:26.478 INFO:tasks.cephfs.mount:Writing 917503 bytes to stripey/flushed_file_3 2023-11-08T16:39:26.479 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:39:26.479 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:26.480 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-11-08T16:39:26.481 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:39:26.481 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 917503): 2023-11-08T16:39:26.482 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:26.482 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:39:26.483 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:27.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:27.242 INFO:tasks.cephfs.mount:Validating 917503 bytes from stripey/flushed_file_3 2023-11-08T16:39:27.242 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:39:27.243 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:27.243 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-11-08T16:39:27.243 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:39:27.243 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:39:27.244 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 917503: 2023-11-08T16:39:27.244 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:39:27.244 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 917503 2023-11-08T16:39:27.245 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:39:27.245 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:39:27.245 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:27.246 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:39:27.246 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:39:27.246 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:27.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:27.981 INFO:tasks.cephfs.mount:Writing 1441792 bytes to stripey/flushed_file_4 2023-11-08T16:39:27.981 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:39:27.982 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:27.982 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-11-08T16:39:27.982 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:39:27.983 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 1441792): 2023-11-08T16:39:27.983 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:27.983 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:39:27.984 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:29.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:29.048 INFO:tasks.cephfs.mount:Validating 1441792 bytes from stripey/flushed_file_4 2023-11-08T16:39:29.049 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:39:29.049 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:29.049 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-11-08T16:39:29.050 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:39:29.050 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:39:29.050 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 1441792: 2023-11-08T16:39:29.051 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:39:29.051 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 1441792 2023-11-08T16:39:29.052 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:39:29.052 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:39:29.052 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:29.053 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:39:29.053 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:39:29.053 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:30.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:30.068 INFO:tasks.cephfs.mount:Writing 1441793 bytes to stripey/flushed_file_5 2023-11-08T16:39:30.069 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:39:30.069 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:30.070 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-11-08T16:39:30.070 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:39:30.070 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 1441793): 2023-11-08T16:39:30.071 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:30.071 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:39:30.072 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:31.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:31.120 INFO:tasks.cephfs.mount:Validating 1441793 bytes from stripey/flushed_file_5 2023-11-08T16:39:31.121 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:39:31.122 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:31.122 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-11-08T16:39:31.122 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:39:31.123 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:39:31.123 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 1441793: 2023-11-08T16:39:31.123 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:39:31.124 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 1441793 2023-11-08T16:39:31.124 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:39:31.124 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:39:31.125 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:31.125 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:39:31.125 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:39:31.126 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:32.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:32.151 INFO:tasks.cephfs.mount:Writing 1572864 bytes to stripey/flushed_file_6 2023-11-08T16:39:32.152 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:39:32.152 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:32.152 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-11-08T16:39:32.153 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:39:32.153 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 1572864): 2023-11-08T16:39:32.154 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:32.154 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:39:32.154 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:33.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:33.282 INFO:tasks.cephfs.mount:Validating 1572864 bytes from stripey/flushed_file_6 2023-11-08T16:39:33.283 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:39:33.283 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:33.284 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-11-08T16:39:33.284 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:39:33.284 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:39:33.285 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 1572864: 2023-11-08T16:39:33.285 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:39:33.285 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 1572864 2023-11-08T16:39:33.286 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:39:33.286 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:39:33.286 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:33.287 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:39:33.287 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:39:33.287 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:34.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:34.361 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:34.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.607+0000 7fe551052640 1 Processor -- start 2023-11-08T16:39:34.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.607+0000 7fe551052640 1 -- start start 2023-11-08T16:39:34.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.609+0000 7fe551052640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe54c0ced10 0x7fe54c0cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:34.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.609+0000 7fe551052640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54c15b0a0 con 0x7fe54c150b80 2023-11-08T16:39:34.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.609+0000 7fe551052640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54c0b2fe0 con 0x7fe54c06b540 2023-11-08T16:39:34.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.609+0000 7fe551052640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54c0cf6c0 con 0x7fe54c0ced10 2023-11-08T16:39:34.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.609+0000 7fe543fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe54c0ced10 0x7fe54c0cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:34.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.609+0000 7fe54bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe54c150b80 0x7fe54c15a990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59456/0 (socket says 172.21.15.81:59456) 2023-11-08T16:39:34.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.609+0000 7fe54bfff640 1 -- 172.21.15.81:0/2559947423 learned_addr learned my addr 172.21.15.81:0/2559947423 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:34.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.609+0000 7fe543fff640 1 -- 172.21.15.81:0/2559947423 >> v1:172.21.15.93:6789/0 conn(0x7fe54c06b540 legacy=0x7fe54c06b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:34.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.609+0000 7fe543fff640 1 -- 172.21.15.81:0/2559947423 >> v1:172.21.15.81:6789/0 conn(0x7fe54c150b80 legacy=0x7fe54c15a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:34.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.609+0000 7fe543fff640 1 -- 172.21.15.81:0/2559947423 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54c0b32c0 con 0x7fe54c0ced10 2023-11-08T16:39:34.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.609+0000 7fe54b7fe640 1 -- 172.21.15.81:0/2559947423 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1310278123 0 0) 0x7fe54c15b0a0 con 0x7fe54c150b80 2023-11-08T16:39:34.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.610+0000 7fe543fff640 1 --2- 172.21.15.81:0/2559947423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c0ced10 0x7fe54c0cf180 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe5340097a0 tx=0x7fe53402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=297e7ae9b775b1db server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:34.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.611+0000 7fe54b7fe640 1 -- 172.21.15.81:0/2559947423 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534031830 con 0x7fe54c0ced10 2023-11-08T16:39:34.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.611+0000 7fe54b7fe640 1 -- 172.21.15.81:0/2559947423 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe534031990 con 0x7fe54c0ced10 2023-11-08T16:39:34.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.611+0000 7fe54b7fe640 1 -- 172.21.15.81:0/2559947423 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534031d20 con 0x7fe54c0ced10 2023-11-08T16:39:34.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.611+0000 7fe551052640 1 -- 172.21.15.81:0/2559947423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c0ced10 msgr2=0x7fe54c0cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:34.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.611+0000 7fe551052640 1 --2- 172.21.15.81:0/2559947423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c0ced10 0x7fe54c0cf180 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe5340097a0 tx=0x7fe53402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:39:34.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.611+0000 7fe551052640 1 -- 172.21.15.81:0/2559947423 shutdown_connections 2023-11-08T16:39:34.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.611+0000 7fe551052640 1 --2- 172.21.15.81:0/2559947423 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c0ced10 0x7fe54c0cf180 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:34.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.611+0000 7fe551052640 1 -- 172.21.15.81:0/2559947423 >> 172.21.15.81:0/2559947423 conn(0x7fe54c06a4a0 msgr2=0x7fe54c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:34.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.611+0000 7fe551052640 1 -- 172.21.15.81:0/2559947423 shutdown_connections 2023-11-08T16:39:34.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.611+0000 7fe551052640 1 -- 172.21.15.81:0/2559947423 wait complete. 2023-11-08T16:39:34.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.613+0000 7fe551052640 1 Processor -- start 2023-11-08T16:39:34.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.613+0000 7fe551052640 1 -- start start 2023-11-08T16:39:34.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.614+0000 7fe551052640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c06b540 0x7fe54c14a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:34.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.614+0000 7fe551052640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe54c0ced10 0x7fe54c14d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:34.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.614+0000 7fe551052640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe54c150b80 0x7fe54c15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:34.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.614+0000 7fe551052640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c14cd30 con 0x7fe54c06b540 2023-11-08T16:39:34.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.614+0000 7fe551052640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c14c120 con 0x7fe54c0ced10 2023-11-08T16:39:34.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.614+0000 7fe551052640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54c14c400 con 0x7fe54c150b80 2023-11-08T16:39:34.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.614+0000 7fe54bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c06b540 0x7fe54c14a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:34.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.614+0000 7fe54bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c06b540 0x7fe54c14a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43418/0 (socket says 172.21.15.81:43418) 2023-11-08T16:39:34.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.614+0000 7fe54bfff640 1 -- 172.21.15.81:0/721208171 learned_addr learned my addr 172.21.15.81:0/721208171 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:34.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe550851640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe54c150b80 0x7fe54c15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:34.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe54bfff640 1 -- 172.21.15.81:0/721208171 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe54c150b80 msgr2=0x7fe54c15be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:34.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe54bfff640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe54c150b80 0x7fe54c15be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:34.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe54bfff640 1 -- 172.21.15.81:0/721208171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe54c0ced10 msgr2=0x7fe54c14d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:34.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe54bfff640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe54c0ced10 0x7fe54c14d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:34.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe54bfff640 1 -- 172.21.15.81:0/721208171 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe534009400 con 0x7fe54c06b540 2023-11-08T16:39:34.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe550851640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe54c150b80 0x7fe54c15be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:34.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe54bfff640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c06b540 0x7fe54c14a040 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c430 tx=0x7fe53c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:34.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe5497fa640 1 -- 172.21.15.81:0/721208171 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c0074e0 con 0x7fe54c06b540 2023-11-08T16:39:34.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe5497fa640 1 -- 172.21.15.81:0/721208171 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe53c007660 con 0x7fe54c06b540 2023-11-08T16:39:34.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe5497fa640 1 -- 172.21.15.81:0/721208171 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53c004a20 con 0x7fe54c06b540 2023-11-08T16:39:34.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe551052640 1 -- 172.21.15.81:0/721208171 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54c14c6e0 con 0x7fe54c06b540 2023-11-08T16:39:34.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.615+0000 7fe551052640 1 -- 172.21.15.81:0/721208171 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54c14a580 con 0x7fe54c06b540 2023-11-08T16:39:34.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.617+0000 7fe551052640 1 -- 172.21.15.81:0/721208171 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54c067800 con 0x7fe54c06b540 2023-11-08T16:39:34.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.617+0000 7fe5497fa640 1 -- 172.21.15.81:0/721208171 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe53c004b80 con 0x7fe54c06b540 2023-11-08T16:39:34.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.618+0000 7fe5497fa640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe53006aea0 0x7fe53006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:34.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.618+0000 7fe5497fa640 1 -- 172.21.15.81:0/721208171 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7fe53c089780 con 0x7fe54c06b540 2023-11-08T16:39:34.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.619+0000 7fe543fff640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe53006aea0 0x7fe53006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:34.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.619+0000 7fe543fff640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe53006aea0 0x7fe53006d360 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe534036000 tx=0x7fe534030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:34.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.623+0000 7fe5497fa640 1 -- 172.21.15.81:0/721208171 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe53c054560 con 0x7fe54c06b540 2023-11-08T16:39:34.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.831+0000 7fe551052640 1 -- 172.21.15.81:0/721208171 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe54c0b2710 con 0x7fe54c06b540 2023-11-08T16:39:34.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.832+0000 7fe5497fa640 1 -- 172.21.15.81:0/721208171 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4915 (secure 0 0 0) 0x7fe53c057b80 con 0x7fe54c06b540 2023-11-08T16:39:34.834 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:34.835 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":87,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1157756119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1157756119},{"type":"v1","addr":"172.21.15.93:6833","nonce":1157756119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1607105255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1607105255},{"type":"v1","addr":"172.21.15.81:6835","nonce":1607105255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":7707,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3988749175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3988749175},{"type":"v1","addr":"172.21.15.81:6837","nonce":3988749175}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:09.619973+0000","modified":"2023-11-08T16:39:20.760892+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7713},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7713":{"gid":7713,"name":"b","rank":0,"incarnation":82,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6835/691341796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":691341796},{"type":"v1","addr":"172.21.15.93:6835","nonce":691341796}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:39:34.835 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 87 2023-11-08T16:39:34.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 -- 172.21.15.81:0/721208171 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe53006aea0 msgr2=0x7fe53006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:34.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe53006aea0 0x7fe53006d360 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fe534036000 tx=0x7fe534030000 comp rx=0 tx=0).stop 2023-11-08T16:39:34.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 -- 172.21.15.81:0/721208171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c06b540 msgr2=0x7fe54c14a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:34.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c06b540 0x7fe54c14a040 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00c430 tx=0x7fe53c00c8f0 comp rx=0 tx=0).stop 2023-11-08T16:39:34.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 -- 172.21.15.81:0/721208171 shutdown_connections 2023-11-08T16:39:34.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe53006aea0 0x7fe53006d360 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:34.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe54c150b80 0x7fe54c15be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:34.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe54c0ced10 0x7fe54c14d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:34.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 --2- 172.21.15.81:0/721208171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe54c06b540 0x7fe54c14a040 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:34.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 -- 172.21.15.81:0/721208171 >> 172.21.15.81:0/721208171 conn(0x7fe54c06a4a0 msgr2=0x7fe54c15d500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:34.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 -- 172.21.15.81:0/721208171 shutdown_connections 2023-11-08T16:39:34.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:34.835+0000 7fe551052640 1 -- 172.21.15.81:0/721208171 wait complete. 2023-11-08T16:39:34.854 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2023-11-08T16:39:35.122 INFO:teuthology.orchestra.run.smithi093.stdout:{"message":"","return_code":0} 2023-11-08T16:39:35.123 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-08T16:39:35.123 INFO:tasks.cephfs.mount:Writing 524288 bytes to stripey/unflushed_file 2023-11-08T16:39:35.123 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:39:35.124 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:35.124 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file" 2023-11-08T16:39:35.124 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'w'"'"') as f: 2023-11-08T16:39:35.125 DEBUG:teuthology.orchestra.run.smithi081:> for i in range(0, 524288): 2023-11-08T16:39:35.125 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:35.126 DEBUG:teuthology.orchestra.run.smithi081:> f.write(chr(val)) 2023-11-08T16:39:35.126 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:35.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:35.592 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/unflushed_file 2023-11-08T16:39:35.593 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:39:35.593 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:39:35.594 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file" 2023-11-08T16:39:35.594 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:39:35.594 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:39:35.595 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 524288: 2023-11-08T16:39:35.595 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:39:35.595 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 524288 2023-11-08T16:39:35.596 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:39:35.596 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:39:35.597 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:39:35.597 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:39:35.597 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:39:35.598 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:36.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:36.128 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:39:36.129 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:39:36.129 DEBUG:teuthology.orchestra.run.smithi081:> import stat 2023-11-08T16:39:36.130 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:39:36.130 DEBUG:teuthology.orchestra.run.smithi081:> print(os.stat("/home/ubuntu/cephtest/mnt.0/stripey/unflushed_file").st_ino) 2023-11-08T16:39:36.130 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:39:36.316 INFO:teuthology.orchestra.run.smithi081.stdout:1099511628284 2023-11-08T16:39:36.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:39:36.330 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:39:36.330 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:39:36.427 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:39:36.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.634+0000 7fc06d4e0640 1 Processor -- start 2023-11-08T16:39:36.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.634+0000 7fc06d4e0640 1 -- start start 2023-11-08T16:39:36.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.635+0000 7fc06d4e0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc0680ade50 0x7fc0680ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:36.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.635+0000 7fc06d4e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc06815a680 0x7fc06815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:36.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.635+0000 7fc06d4e0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06806b630 con 0x7fc06815b020 2023-11-08T16:39:36.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.635+0000 7fc06d4e0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680b1860 con 0x7fc0680ade50 2023-11-08T16:39:36.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.635+0000 7fc06d4e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680ca9c0 con 0x7fc06815a680 2023-11-08T16:39:36.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.636+0000 7fc066ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc0680ade50 0x7fc0680ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:36.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.636+0000 7fc0677fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc06815b020 0x7fc0680c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59466/0 (socket says 172.21.15.81:59466) 2023-11-08T16:39:36.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.636+0000 7fc0677fe640 1 -- 172.21.15.81:0/583399723 learned_addr learned my addr 172.21.15.81:0/583399723 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:36.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.636+0000 7fc0667fc640 1 -- 172.21.15.81:0/583399723 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 415619639 0 0) 0x7fc06806b630 con 0x7fc06815b020 2023-11-08T16:39:36.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.636+0000 7fc05ffff640 1 --2- 172.21.15.81:0/583399723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc06815a680 0x7fc06815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:36.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.636+0000 7fc0667fc640 1 -- 172.21.15.81:0/583399723 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc054003520 con 0x7fc06815b020 2023-11-08T16:39:36.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.636+0000 7fc066ffd640 1 -- 172.21.15.81:0/583399723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc06815a680 msgr2=0x7fc06815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:36.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.636+0000 7fc066ffd640 1 --2- 172.21.15.81:0/583399723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc06815a680 0x7fc06815aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:36.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.636+0000 7fc066ffd640 1 -- 172.21.15.81:0/583399723 >> v1:172.21.15.81:6789/0 conn(0x7fc06815b020 legacy=0x7fc0680c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:36.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.636+0000 7fc066ffd640 1 -- 172.21.15.81:0/583399723 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680cabb0 con 0x7fc0680ade50 2023-11-08T16:39:36.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.638+0000 7fc066ffd640 1 --2- 172.21.15.81:0/583399723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0680ade50 0x7fc0680ca480 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc0580097a0 tx=0x7fc05802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf131bf6a312509 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:36.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.638+0000 7fc0667fc640 1 -- 172.21.15.81:0/583399723 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05802d040 con 0x7fc0680ade50 2023-11-08T16:39:36.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.638+0000 7fc0667fc640 1 -- 172.21.15.81:0/583399723 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc058031810 con 0x7fc0680ade50 2023-11-08T16:39:36.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.638+0000 7fc0667fc640 1 -- 172.21.15.81:0/583399723 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058031b80 con 0x7fc0680ade50 2023-11-08T16:39:36.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.638+0000 7fc06d4e0640 1 -- 172.21.15.81:0/583399723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0680ade50 msgr2=0x7fc0680ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:36.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.638+0000 7fc06d4e0640 1 --2- 172.21.15.81:0/583399723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0680ade50 0x7fc0680ca480 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc0580097a0 tx=0x7fc05802b8b0 comp rx=0 tx=0).stop 2023-11-08T16:39:36.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.639+0000 7fc06d4e0640 1 -- 172.21.15.81:0/583399723 shutdown_connections 2023-11-08T16:39:36.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.639+0000 7fc06d4e0640 1 --2- 172.21.15.81:0/583399723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc06815a680 0x7fc06815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:36.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.639+0000 7fc06d4e0640 1 --2- 172.21.15.81:0/583399723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc0680ade50 0x7fc0680ca480 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:36.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.639+0000 7fc06d4e0640 1 -- 172.21.15.81:0/583399723 >> 172.21.15.81:0/583399723 conn(0x7fc0680cb490 msgr2=0x7fc0680cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:36.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.639+0000 7fc06d4e0640 1 -- 172.21.15.81:0/583399723 shutdown_connections 2023-11-08T16:39:36.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.639+0000 7fc06d4e0640 1 -- 172.21.15.81:0/583399723 wait complete. 2023-11-08T16:39:36.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.640+0000 7fc06d4e0640 1 Processor -- start 2023-11-08T16:39:36.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.641+0000 7fc06d4e0640 1 -- start start 2023-11-08T16:39:36.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.641+0000 7fc06d4e0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0680ade50 0x7fc0680b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:36.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc06d4e0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc06815a680 0x7fc068152a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:36.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc06d4e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc06815b020 0x7fc068152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:36.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc06d4e0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0681f3930 con 0x7fc06815a680 2023-11-08T16:39:36.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc06d4e0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0681534d0 con 0x7fc0680ade50 2023-11-08T16:39:36.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc06d4e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0681f6120 con 0x7fc06815b020 2023-11-08T16:39:36.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc06815a680 0x7fc068152a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:36.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc05ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc06815a680 0x7fc068152a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43434/0 (socket says 172.21.15.81:43434) 2023-11-08T16:39:36.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc05ffff640 1 -- 172.21.15.81:0/332210338 learned_addr learned my addr 172.21.15.81:0/332210338 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:36.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc05ffff640 1 -- 172.21.15.81:0/332210338 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc06815b020 msgr2=0x7fc068152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:36.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc05ffff640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc06815b020 0x7fc068152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:36.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc05ffff640 1 -- 172.21.15.81:0/332210338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0680ade50 msgr2=0x7fc0680b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:36.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc066ffd640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0680ade50 0x7fc0680b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:36.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc05ffff640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0680ade50 0x7fc0680b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:36.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.642+0000 7fc05ffff640 1 -- 172.21.15.81:0/332210338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc058009400 con 0x7fc06815a680 2023-11-08T16:39:36.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.643+0000 7fc05ffff640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc06815a680 0x7fc068152a50 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc05000c7f0 tx=0x7fc05000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:36.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.643+0000 7fc05f7fe640 1 -- 172.21.15.81:0/332210338 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050004220 con 0x7fc06815a680 2023-11-08T16:39:36.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.643+0000 7fc05f7fe640 1 -- 172.21.15.81:0/332210338 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0500043a0 con 0x7fc06815a680 2023-11-08T16:39:36.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.643+0000 7fc06d4e0640 1 -- 172.21.15.81:0/332210338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0681f6400 con 0x7fc06815a680 2023-11-08T16:39:36.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.643+0000 7fc05f7fe640 1 -- 172.21.15.81:0/332210338 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050007860 con 0x7fc06815a680 2023-11-08T16:39:36.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.643+0000 7fc06d4e0640 1 -- 172.21.15.81:0/332210338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0681f69c0 con 0x7fc06815a680 2023-11-08T16:39:36.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.644+0000 7fc06d4e0640 1 -- 172.21.15.81:0/332210338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0681f6bb0 con 0x7fc06815a680 2023-11-08T16:39:36.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.645+0000 7fc05f7fe640 1 -- 172.21.15.81:0/332210338 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc050007bd0 con 0x7fc06815a680 2023-11-08T16:39:36.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.646+0000 7fc05f7fe640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc04806ae70 0x7fc04806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:36.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.646+0000 7fc05f7fe640 1 -- 172.21.15.81:0/332210338 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7fc0500897d0 con 0x7fc06815a680 2023-11-08T16:39:36.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.646+0000 7fc066ffd640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc04806ae70 0x7fc04806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:36.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.647+0000 7fc066ffd640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc04806ae70 0x7fc04806d330 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc058036000 tx=0x7fc058008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:36.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.650+0000 7fc05f7fe640 1 -- 172.21.15.81:0/332210338 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc050054640 con 0x7fc06815a680 2023-11-08T16:39:36.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.844+0000 7fc06d4e0640 1 -- 172.21.15.81:0/332210338 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc06804ece0 con 0x7fc06815a680 2023-11-08T16:39:36.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.846+0000 7fc05f7fe640 1 -- 172.21.15.81:0/332210338 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+18241 (secure 0 0 0) 0x7fc050057c60 con 0x7fc06815a680 2023-11-08T16:39:36.848 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:36.848 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":106,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:39:22.303415+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:39:07.492834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T16:39:08.634345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:39:19.654916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:39:36.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 -- 172.21.15.81:0/332210338 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc04806ae70 msgr2=0x7fc04806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:36.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc04806ae70 0x7fc04806d330 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fc058036000 tx=0x7fc058008030 comp rx=0 tx=0).stop 2023-11-08T16:39:36.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 -- 172.21.15.81:0/332210338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc06815a680 msgr2=0x7fc068152a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:36.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc06815a680 0x7fc068152a50 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc05000c7f0 tx=0x7fc05000ccb0 comp rx=0 tx=0).stop 2023-11-08T16:39:36.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 -- 172.21.15.81:0/332210338 shutdown_connections 2023-11-08T16:39:36.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc04806ae70 0x7fc04806d330 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:36.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc06815b020 0x7fc068152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:36.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc06815a680 0x7fc068152a50 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:36.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 --2- 172.21.15.81:0/332210338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc0680ade50 0x7fc0680b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:36.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 -- 172.21.15.81:0/332210338 >> 172.21.15.81:0/332210338 conn(0x7fc0680cb490 msgr2=0x7fc0680af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:36.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 -- 172.21.15.81:0/332210338 shutdown_connections 2023-11-08T16:39:36.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:36.849+0000 7fc06d4e0640 1 -- 172.21.15.81:0/332210338 wait complete. 2023-11-08T16:39:36.868 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:39:07.492834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:39:08.634345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:39:19.654916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:39:36.869 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:39:37.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.087+0000 7f95f1f97640 1 Processor -- start 2023-11-08T16:39:37.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.087+0000 7f95f1f97640 1 -- start start 2023-11-08T16:39:37.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.089+0000 7f95f1f97640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15a0e0 0x7f95ec0ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.089+0000 7f95f1f97640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec0ce960 con 0x7f95ec0c9bf0 2023-11-08T16:39:37.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.089+0000 7f95f1f97640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ec0c9480 con 0x7f95ec0ca720 2023-11-08T16:39:37.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.089+0000 7f95f1f97640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec071980 con 0x7f95ec15a0e0 2023-11-08T16:39:37.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95e3fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f95ec0c9bf0 0x7f95ec0ca010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59482/0 (socket says 172.21.15.81:59482) 2023-11-08T16:39:37.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95e3fff640 1 -- 172.21.15.81:0/4001671145 learned_addr learned my addr 172.21.15.81:0/4001671145 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:37.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95eb7fe640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15a0e0 0x7f95ec0ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 68072669 0 0) 0x7f95d4001110 con 0x7f95ec0c9bf0 2023-11-08T16:39:37.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95d8003520 con 0x7f95ec0c9bf0 2023-11-08T16:39:37.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2933140016 0 0) 0x7f95d8003520 con 0x7f95ec0c9bf0 2023-11-08T16:39:37.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15a0e0 msgr2=0x7f95ec0ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:37.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95eaffd640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15a0e0 0x7f95ec0ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 >> v1:172.21.15.93:6789/0 conn(0x7f95ec0ca720 legacy=0x7f95ec06b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec15a4c0 con 0x7f95ec0c9bf0 2023-11-08T16:39:37.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f95d4001800 con 0x7f95ec0c9bf0 2023-11-08T16:39:37.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.090+0000 7f95eb7fe640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15a0e0 0x7f95ec0ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:39:37.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 >> v1:172.21.15.81:6789/0 conn(0x7f95ec0c9bf0 legacy=0x7f95ec0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95eaffd640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95d8003a10 0x7f95d8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95eaffd640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15a0e0 0x7f95d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95eaffd640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 0x7f95d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec071980 con 0x7f95d8004d00 2023-11-08T16:39:37.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d800bb20 con 0x7f95d8003a10 2023-11-08T16:39:37.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95d800bce0 con 0x7f95ec15a0e0 2023-11-08T16:39:37.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95ebfff640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 0x7f95d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95eb7fe640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95d8003a10 0x7f95d8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95e3fff640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15a0e0 0x7f95d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95ebfff640 1 -- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15a0e0 msgr2=0x7f95d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:37.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95ebfff640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15a0e0 0x7f95d80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95ebfff640 1 -- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95d8003a10 msgr2=0x7f95d8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:37.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95ebfff640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95d8003a10 0x7f95d8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95ebfff640 1 -- 172.21.15.81:0/4001671145 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc009400 con 0x7f95d8004d00 2023-11-08T16:39:37.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.091+0000 7f95e3fff640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15a0e0 0x7f95d80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:37.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.092+0000 7f95ebfff640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 0x7f95d8005180 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f95e4001e50 tx=0x7f95d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6868c8afcd2792f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:37.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.093+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4003b70 con 0x7f95d8004d00 2023-11-08T16:39:37.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.093+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e4003cd0 con 0x7f95d8004d00 2023-11-08T16:39:37.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.093+0000 7f95eaffd640 1 -- 172.21.15.81:0/4001671145 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4005310 con 0x7f95d8004d00 2023-11-08T16:39:37.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.093+0000 7f95f1f97640 1 -- 172.21.15.81:0/4001671145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 msgr2=0x7f95d8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.093+0000 7f95f1f97640 1 --2- 172.21.15.81:0/4001671145 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 0x7f95d8005180 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f95e4001e50 tx=0x7f95d80057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.093+0000 7f95ebfff640 1 -- 172.21.15.81:0/4001671145 reap_dead start 2023-11-08T16:39:37.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.094+0000 7f95f1f97640 1 -- 172.21.15.81:0/4001671145 shutdown_connections 2023-11-08T16:39:37.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.094+0000 7f95f1f97640 1 -- 172.21.15.81:0/4001671145 >> 172.21.15.81:0/4001671145 conn(0x7f95ec0cb580 msgr2=0x7f95ec0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:37.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.094+0000 7f95f1f97640 1 -- 172.21.15.81:0/4001671145 shutdown_connections 2023-11-08T16:39:37.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.094+0000 7f95f1f97640 1 -- 172.21.15.81:0/4001671145 wait complete. 2023-11-08T16:39:37.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.095+0000 7f95f1f97640 1 Processor -- start 2023-11-08T16:39:37.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.096+0000 7f95f1f97640 1 -- start start 2023-11-08T16:39:37.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.096+0000 7f95f1f97640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 0x7f95ec1f2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.096+0000 7f95f1f97640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95ec14f030 0x7f95ec15c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.096+0000 7f95f1f97640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15ac10 0x7f95ec15b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.096+0000 7f95f1f97640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec1f31d0 con 0x7f95d8004d00 2023-11-08T16:39:37.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.097+0000 7f95f1f97640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec1f2240 con 0x7f95ec14f030 2023-11-08T16:39:37.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.097+0000 7f95f1f97640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ec1f2800 con 0x7f95ec15ac10 2023-11-08T16:39:37.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.097+0000 7f95eb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 0x7f95ec1f2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.097+0000 7f95eb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 0x7f95ec1f2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43456/0 (socket says 172.21.15.81:43456) 2023-11-08T16:39:37.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.097+0000 7f95eb7fe640 1 -- 172.21.15.81:0/3798960125 learned_addr learned my addr 172.21.15.81:0/3798960125 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:37.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.097+0000 7f95eb7fe640 1 -- 172.21.15.81:0/3798960125 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15ac10 msgr2=0x7f95ec15b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.097+0000 7f95eb7fe640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15ac10 0x7f95ec15b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.097+0000 7f95eb7fe640 1 -- 172.21.15.81:0/3798960125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95ec14f030 msgr2=0x7f95ec15c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.097+0000 7f95eb7fe640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95ec14f030 0x7f95ec15c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.097+0000 7f95eb7fe640 1 -- 172.21.15.81:0/3798960125 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ec15d220 con 0x7f95d8004d00 2023-11-08T16:39:37.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.097+0000 7f95eb7fe640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 0x7f95ec1f2c90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0118b0 tx=0x7f95dc011d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:37.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.098+0000 7f95e8ff9640 1 -- 172.21.15.81:0/3798960125 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc00fb70 con 0x7f95d8004d00 2023-11-08T16:39:37.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.098+0000 7f95e8ff9640 1 -- 172.21.15.81:0/3798960125 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95dc00fcd0 con 0x7f95d8004d00 2023-11-08T16:39:37.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.098+0000 7f95e8ff9640 1 -- 172.21.15.81:0/3798960125 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95dc009e50 con 0x7f95d8004d00 2023-11-08T16:39:37.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.098+0000 7f95f1f97640 1 -- 172.21.15.81:0/3798960125 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95ec15d500 con 0x7f95d8004d00 2023-11-08T16:39:37.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.098+0000 7f95f1f97640 1 -- 172.21.15.81:0/3798960125 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ec15ce10 con 0x7f95d8004d00 2023-11-08T16:39:37.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.099+0000 7f95f1f97640 1 -- 172.21.15.81:0/3798960125 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ec04ece0 con 0x7f95d8004d00 2023-11-08T16:39:37.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.100+0000 7f95e8ff9640 1 -- 172.21.15.81:0/3798960125 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f95dc015460 con 0x7f95d8004d00 2023-11-08T16:39:37.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.101+0000 7f95e8ff9640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95cc06ae70 0x7f95cc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.101+0000 7f95e8ff9640 1 -- 172.21.15.81:0/3798960125 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f95dc089d80 con 0x7f95d8004d00 2023-11-08T16:39:37.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.101+0000 7f95e3fff640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95cc06ae70 0x7f95cc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.102+0000 7f95e3fff640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95cc06ae70 0x7f95cc06d330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0c8750 tx=0x7f95d4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:37.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.105+0000 7f95e8ff9640 1 -- 172.21.15.81:0/3798960125 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f95dc054c80 con 0x7f95d8004d00 2023-11-08T16:39:37.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.294+0000 7f95f1f97640 1 -- 172.21.15.81:0/3798960125 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95ec06d030 con 0x7f95d8004d00 2023-11-08T16:39:37.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.295+0000 7f95e8ff9640 1 -- 172.21.15.81:0/3798960125 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f95dc0220a0 con 0x7f95d8004d00 2023-11-08T16:39:37.297 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:39:37.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.298+0000 7f95f1f97640 1 -- 172.21.15.81:0/3798960125 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95cc06ae70 msgr2=0x7f95cc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.298+0000 7f95f1f97640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95cc06ae70 0x7f95cc06d330 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f95ec0c8750 tx=0x7f95d4009040 comp rx=0 tx=0).stop 2023-11-08T16:39:37.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.298+0000 7f95f1f97640 1 -- 172.21.15.81:0/3798960125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 msgr2=0x7f95ec1f2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.298+0000 7f95f1f97640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 0x7f95ec1f2c90 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0118b0 tx=0x7f95dc011d70 comp rx=0 tx=0).stop 2023-11-08T16:39:37.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.298+0000 7f95f1f97640 1 -- 172.21.15.81:0/3798960125 shutdown_connections 2023-11-08T16:39:37.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.299+0000 7f95f1f97640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f95cc06ae70 0x7f95cc06d330 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.299+0000 7f95f1f97640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f95ec15ac10 0x7f95ec15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.299+0000 7f95f1f97640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95ec14f030 0x7f95ec15c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.299+0000 7f95f1f97640 1 --2- 172.21.15.81:0/3798960125 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f95d8004d00 0x7f95ec1f2c90 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.299+0000 7f95f1f97640 1 -- 172.21.15.81:0/3798960125 >> 172.21.15.81:0/3798960125 conn(0x7f95ec0cb580 msgr2=0x7f95ec0b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:37.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.299+0000 7f95f1f97640 1 -- 172.21.15.81:0/3798960125 shutdown_connections 2023-11-08T16:39:37.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.299+0000 7f95f1f97640 1 -- 172.21.15.81:0/3798960125 wait complete. 2023-11-08T16:39:37.319 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:39:37.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.537+0000 7f58022e1640 1 Processor -- start 2023-11-08T16:39:37.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.537+0000 7f58022e1640 1 -- start start 2023-11-08T16:39:37.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.538+0000 7f58022e1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f57fc0cd8d0 0x7f57fc06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.538+0000 7f58022e1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57fc0cc9f0 0x7f57fc15c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.538+0000 7f58022e1640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57fc06cf50 con 0x7f57fc0cdcb0 2023-11-08T16:39:37.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.538+0000 7f58022e1640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57fc06d230 con 0x7f57fc0cd8d0 2023-11-08T16:39:37.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.538+0000 7f58022e1640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc0d0be0 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.538+0000 7f57fbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f57fc0cd8d0 0x7f57fc06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.539+0000 7f57fbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f57fc0cd8d0 0x7f57fc06e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43462/0 (socket says 172.21.15.81:43462) 2023-11-08T16:39:37.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.539+0000 7f57fbfff640 1 -- 172.21.15.81:0/1492383272 learned_addr learned my addr 172.21.15.81:0/1492383272 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:37.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.539+0000 7f57fb7fe640 1 --1- 172.21.15.81:0/1492383272 >> v1:172.21.15.81:6789/0 conn(0x7f57fc0cdcb0 0x7f57fc0cc2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59494/0 (socket says 172.21.15.81:59494) 2023-11-08T16:39:37.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.539+0000 7f580087e640 1 --2- 172.21.15.81:0/1492383272 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57fc0cc9f0 0x7f57fc15c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.539+0000 7f57fbfff640 1 -- 172.21.15.81:0/1492383272 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57fc0cc9f0 msgr2=0x7f57fc15c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:37.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.539+0000 7f57fbfff640 1 --2- 172.21.15.81:0/1492383272 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57fc0cc9f0 0x7f57fc15c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.539+0000 7f57fbfff640 1 -- 172.21.15.81:0/1492383272 >> v1:172.21.15.81:6789/0 conn(0x7f57fc0cdcb0 legacy=0x7f57fc0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.539+0000 7f57fbfff640 1 -- 172.21.15.81:0/1492383272 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57fc0d0df0 con 0x7f57fc0cd8d0 2023-11-08T16:39:37.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.539+0000 7f580087e640 1 --2- 172.21.15.81:0/1492383272 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57fc0cc9f0 0x7f57fc15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:37.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.540+0000 7f57fbfff640 1 --2- 172.21.15.81:0/1492383272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57fc0cd8d0 0x7f57fc06e5a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e50 tx=0x7f57ec02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236c8a45c8223039 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:37.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.540+0000 7f57faffd640 1 -- 172.21.15.81:0/1492383272 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec02caf0 con 0x7f57fc0cd8d0 2023-11-08T16:39:37.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.541+0000 7f57faffd640 1 -- 172.21.15.81:0/1492383272 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec02cc50 con 0x7f57fc0cd8d0 2023-11-08T16:39:37.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.541+0000 7f57faffd640 1 -- 172.21.15.81:0/1492383272 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0343f0 con 0x7f57fc0cd8d0 2023-11-08T16:39:37.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.541+0000 7f58022e1640 1 -- 172.21.15.81:0/1492383272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57fc0cd8d0 msgr2=0x7f57fc06e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.541+0000 7f58022e1640 1 --2- 172.21.15.81:0/1492383272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57fc0cd8d0 0x7f57fc06e5a0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e50 tx=0x7f57ec02bf50 comp rx=0 tx=0).stop 2023-11-08T16:39:37.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.541+0000 7f58022e1640 1 -- 172.21.15.81:0/1492383272 shutdown_connections 2023-11-08T16:39:37.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.541+0000 7f58022e1640 1 --2- 172.21.15.81:0/1492383272 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57fc0cc9f0 0x7f57fc15c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.541+0000 7f58022e1640 1 --2- 172.21.15.81:0/1492383272 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57fc0cd8d0 0x7f57fc06e5a0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.541+0000 7f58022e1640 1 -- 172.21.15.81:0/1492383272 >> 172.21.15.81:0/1492383272 conn(0x7f57fc065df0 msgr2=0x7f57fc0661d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:37.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.541+0000 7f58022e1640 1 -- 172.21.15.81:0/1492383272 shutdown_connections 2023-11-08T16:39:37.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.541+0000 7f58022e1640 1 -- 172.21.15.81:0/1492383272 wait complete. 2023-11-08T16:39:37.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.543+0000 7f58022e1640 1 Processor -- start 2023-11-08T16:39:37.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.543+0000 7f58022e1640 1 -- start start 2023-11-08T16:39:37.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.544+0000 7f58022e1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57fc0cc9f0 0x7f57fc159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.544+0000 7f58022e1640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc0cd8d0 0x7f57fc15c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.544+0000 7f58022e1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57fc0cdcb0 0x7f57fc0c6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.544+0000 7f58022e1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc159dc0 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.544+0000 7f58022e1640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1e5890 con 0x7f57fc0cd8d0 2023-11-08T16:39:37.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.544+0000 7f58022e1640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc15c300 con 0x7f57fc0cdcb0 2023-11-08T16:39:37.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.544+0000 7f57fbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57fc0cc9f0 0x7f57fc159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.544+0000 7f57fbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57fc0cc9f0 0x7f57fc159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43468/0 (socket says 172.21.15.81:43468) 2023-11-08T16:39:37.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.544+0000 7f57fbfff640 1 -- 172.21.15.81:0/3556021480 learned_addr learned my addr 172.21.15.81:0/3556021480 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:37.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.545+0000 7f57fbfff640 1 -- 172.21.15.81:0/3556021480 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57fc0cdcb0 msgr2=0x7f57fc0c6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.545+0000 7f57fbfff640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57fc0cdcb0 0x7f57fc0c6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.545+0000 7f57fbfff640 1 -- 172.21.15.81:0/3556021480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc0cd8d0 msgr2=0x7f57fc15c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.545+0000 7f57fbfff640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc0cd8d0 0x7f57fc15c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.545+0000 7f57fbfff640 1 -- 172.21.15.81:0/3556021480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ec009b30 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.545+0000 7f57fbfff640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57fc0cc9f0 0x7f57fc159880 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f57ec000f80 tx=0x7f57ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:37.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.545+0000 7f57f8ff9640 1 -- 172.21.15.81:0/3556021480 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec03e450 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.545+0000 7f57f8ff9640 1 -- 172.21.15.81:0/3556021480 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec03e5f0 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.545+0000 7f57f8ff9640 1 -- 172.21.15.81:0/3556021480 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec03e9a0 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.545+0000 7f58022e1640 1 -- 172.21.15.81:0/3556021480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc15b730 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.546+0000 7f58022e1640 1 -- 172.21.15.81:0/3556021480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc15bcb0 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.547+0000 7f58022e1640 1 -- 172.21.15.81:0/3556021480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc04ece0 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.548+0000 7f57f8ff9640 1 -- 172.21.15.81:0/3556021480 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f57ec03eb00 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.549+0000 7f57f8ff9640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57dc06ae70 0x7f57dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.549+0000 7f57f8ff9640 1 -- 172.21.15.81:0/3556021480 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f57ec0ac670 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.549+0000 7f57fb7fe640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57dc06ae70 0x7f57dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.549+0000 7f57fb7fe640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57dc06ae70 0x7f57dc06d330 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f57e4007fd0 tx=0x7f57e4007980 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:37.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.553+0000 7f57f8ff9640 1 -- 172.21.15.81:0/3556021480 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f57ec0777c0 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.742+0000 7f58022e1640 1 -- 172.21.15.81:0/3556021480 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57fc06de70 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.743+0000 7f57f8ff9640 1 -- 172.21.15.81:0/3556021480 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f57ec03edb0 con 0x7f57fc0cc9f0 2023-11-08T16:39:37.745 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:39:37.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 -- 172.21.15.81:0/3556021480 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57dc06ae70 msgr2=0x7f57dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57dc06ae70 0x7f57dc06d330 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f57e4007fd0 tx=0x7f57e4007980 comp rx=0 tx=0).stop 2023-11-08T16:39:37.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 -- 172.21.15.81:0/3556021480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57fc0cc9f0 msgr2=0x7f57fc159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57fc0cc9f0 0x7f57fc159880 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f57ec000f80 tx=0x7f57ec004a40 comp rx=0 tx=0).stop 2023-11-08T16:39:37.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 -- 172.21.15.81:0/3556021480 shutdown_connections 2023-11-08T16:39:37.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57dc06ae70 0x7f57dc06d330 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f57fc0cdcb0 0x7f57fc0c6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc0cd8d0 0x7f57fc15c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 --2- 172.21.15.81:0/3556021480 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f57fc0cc9f0 0x7f57fc159880 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 -- 172.21.15.81:0/3556021480 >> 172.21.15.81:0/3556021480 conn(0x7f57fc065df0 msgr2=0x7f57fc06b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:37.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 -- 172.21.15.81:0/3556021480 shutdown_connections 2023-11-08T16:39:37.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.746+0000 7f58022e1640 1 -- 172.21.15.81:0/3556021480 wait complete. 2023-11-08T16:39:37.768 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:39:37.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.972+0000 7f0ebb064640 1 Processor -- start 2023-11-08T16:39:37.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.972+0000 7f0ebb064640 1 -- start start 2023-11-08T16:39:37.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.974+0000 7f0ebb064640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40ce390 0x7f0eb415e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.974+0000 7f0ebb064640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0eb40c9200 0x7f0eb406b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.974+0000 7f0ebb064640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0eb415e8d0 con 0x7f0eb406bc50 2023-11-08T16:39:37.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.974+0000 7f0ebb064640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0eb406c100 con 0x7f0eb40c9200 2023-11-08T16:39:37.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.974+0000 7f0ebb064640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40ca2f0 con 0x7f0eb40ce390 2023-11-08T16:39:37.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.974+0000 7f0eb3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0eb40c9200 0x7f0eb406b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.974+0000 7f0eb3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0eb40c9200 0x7f0eb406b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43482/0 (socket says 172.21.15.81:43482) 2023-11-08T16:39:37.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.974+0000 7f0eb3fff640 1 -- 172.21.15.81:0/2128487551 learned_addr learned my addr 172.21.15.81:0/2128487551 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.975+0000 7f0eb8e00640 1 --2- 172.21.15.81:0/2128487551 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40ce390 0x7f0eb415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.975+0000 7f0eb3fff640 1 -- 172.21.15.81:0/2128487551 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40ce390 msgr2=0x7f0eb415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.975+0000 7f0eb3fff640 1 --2- 172.21.15.81:0/2128487551 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40ce390 0x7f0eb415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.975+0000 7f0eb3fff640 1 -- 172.21.15.81:0/2128487551 >> v1:172.21.15.93:6789/0 conn(0x7f0eb406bc50 legacy=0x7f0eb4160e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:37.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.975+0000 7f0eb3fff640 1 -- 172.21.15.81:0/2128487551 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40ad8c0 con 0x7f0eb40c9200 2023-11-08T16:39:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.975+0000 7f0eb8e00640 1 --2- 172.21.15.81:0/2128487551 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40ce390 0x7f0eb415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.976+0000 7f0eb3fff640 1 --2- 172.21.15.81:0/2128487551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40c9200 0x7f0eb406b710 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009870 tx=0x7f0ea002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7cec6c1518fea2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.976+0000 7f0eb37fe640 1 -- 172.21.15.81:0/2128487551 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0030840 con 0x7f0eb40c9200 2023-11-08T16:39:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.976+0000 7f0eb37fe640 1 -- 172.21.15.81:0/2128487551 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ea00309a0 con 0x7f0eb40c9200 2023-11-08T16:39:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.976+0000 7f0eb37fe640 1 -- 172.21.15.81:0/2128487551 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ea0030d30 con 0x7f0eb40c9200 2023-11-08T16:39:37.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.977+0000 7f0ebb064640 1 -- 172.21.15.81:0/2128487551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40c9200 msgr2=0x7f0eb406b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.977+0000 7f0ebb064640 1 --2- 172.21.15.81:0/2128487551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40c9200 0x7f0eb406b710 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009870 tx=0x7f0ea002b980 comp rx=0 tx=0).stop 2023-11-08T16:39:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.977+0000 7f0ebb064640 1 -- 172.21.15.81:0/2128487551 shutdown_connections 2023-11-08T16:39:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.977+0000 7f0ebb064640 1 --2- 172.21.15.81:0/2128487551 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40c9200 0x7f0eb406b710 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.977+0000 7f0ebb064640 1 --2- 172.21.15.81:0/2128487551 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40ce390 0x7f0eb415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.977+0000 7f0ebb064640 1 -- 172.21.15.81:0/2128487551 >> 172.21.15.81:0/2128487551 conn(0x7f0eb40cb580 msgr2=0x7f0eb40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:37.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.977+0000 7f0ebb064640 1 -- 172.21.15.81:0/2128487551 shutdown_connections 2023-11-08T16:39:37.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.977+0000 7f0ebb064640 1 -- 172.21.15.81:0/2128487551 wait complete. 2023-11-08T16:39:37.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.979+0000 7f0ebb064640 1 Processor -- start 2023-11-08T16:39:37.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.979+0000 7f0ebb064640 1 -- start start 2023-11-08T16:39:37.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.980+0000 7f0ebb064640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0eb406bc50 0x7f0eb4154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.980+0000 7f0ebb064640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40c9200 0x7f0eb40b1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.980+0000 7f0ebb064640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40ce390 0x7f0eb40b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.980+0000 7f0ebb064640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40b2ff0 con 0x7f0eb40ce390 2023-11-08T16:39:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.980+0000 7f0ebb064640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40b2a30 con 0x7f0eb406bc50 2023-11-08T16:39:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.980+0000 7f0ebb064640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb40b2d10 con 0x7f0eb40c9200 2023-11-08T16:39:37.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.980+0000 7f0eb9601640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40ce390 0x7f0eb40b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.980+0000 7f0eb9601640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40ce390 0x7f0eb40b31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43498/0 (socket says 172.21.15.81:43498) 2023-11-08T16:39:37.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.980+0000 7f0eb9601640 1 -- 172.21.15.81:0/3065301819 learned_addr learned my addr 172.21.15.81:0/3065301819 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:37.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0eb3fff640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40c9200 0x7f0eb40b1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0eb8e00640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0eb406bc50 0x7f0eb4154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0eb9601640 1 -- 172.21.15.81:0/3065301819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40c9200 msgr2=0x7f0eb40b1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0eb9601640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40c9200 0x7f0eb40b1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0eb9601640 1 -- 172.21.15.81:0/3065301819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0eb406bc50 msgr2=0x7f0eb4154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0eb9601640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0eb406bc50 0x7f0eb4154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0eb9601640 1 -- 172.21.15.81:0/3065301819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea00094d0 con 0x7f0eb40ce390 2023-11-08T16:39:37.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0eb9601640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40ce390 0x7f0eb40b31e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c780 tx=0x7f0eac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:37.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0eb17fa640 1 -- 172.21.15.81:0/3065301819 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac004b60 con 0x7f0eb40ce390 2023-11-08T16:39:37.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0eb17fa640 1 -- 172.21.15.81:0/3065301819 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eac00ce90 con 0x7f0eb40ce390 2023-11-08T16:39:37.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0ebb064640 1 -- 172.21.15.81:0/3065301819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eb4155fa0 con 0x7f0eb40ce390 2023-11-08T16:39:37.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0eb17fa640 1 -- 172.21.15.81:0/3065301819 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eac00a850 con 0x7f0eb40ce390 2023-11-08T16:39:37.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.981+0000 7f0ebb064640 1 -- 172.21.15.81:0/3065301819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eb415c620 con 0x7f0eb40ce390 2023-11-08T16:39:37.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.982+0000 7f0ebb064640 1 -- 172.21.15.81:0/3065301819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eb41564c0 con 0x7f0eb40ce390 2023-11-08T16:39:37.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.984+0000 7f0eb17fa640 1 -- 172.21.15.81:0/3065301819 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0eac00a9b0 con 0x7f0eb40ce390 2023-11-08T16:39:37.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.985+0000 7f0eb17fa640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 0x7f0e9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:37.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.985+0000 7f0eb17fa640 1 -- 172.21.15.81:0/3065301819 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f0eac0898e0 con 0x7f0eb40ce390 2023-11-08T16:39:37.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.985+0000 7f0eb8e00640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 0x7f0e9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:37.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.986+0000 7f0eb8e00640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 0x7f0e9806d330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800a950 tx=0x7f0ea8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:37.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:37.989+0000 7f0eb17fa640 1 -- 172.21.15.81:0/3065301819 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0eb41564c0 con 0x7f0eb40ce390 2023-11-08T16:39:38.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.181+0000 7f0ebb064640 1 -- 172.21.15.81:0/3065301819 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0eb41564c0 con 0x7f0eb40ce390 2023-11-08T16:39:38.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.182+0000 7f0eb17fa640 1 -- 172.21.15.81:0/3065301819 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0eb41564c0 con 0x7f0eb40ce390 2023-11-08T16:39:38.183 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:39:38.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.184+0000 7f0ebb064640 1 -- 172.21.15.81:0/3065301819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 msgr2=0x7f0e9806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:38.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.184+0000 7f0ebb064640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 0x7f0e9806d330 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0ea800a950 tx=0x7f0ea8008040 comp rx=0 tx=0).stop 2023-11-08T16:39:38.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.184+0000 7f0ebb064640 1 -- 172.21.15.81:0/3065301819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40ce390 msgr2=0x7f0eb40b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:38.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.184+0000 7f0ebb064640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40ce390 0x7f0eb40b31e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f0eac00c780 tx=0x7f0eac00cc40 comp rx=0 tx=0).stop 2023-11-08T16:39:38.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.184+0000 7f0ebb064640 1 -- 172.21.15.81:0/3065301819 shutdown_connections 2023-11-08T16:39:38.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.184+0000 7f0ebb064640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0e9806ae70 0x7f0e9806d330 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.184+0000 7f0ebb064640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0eb40ce390 0x7f0eb40b31e0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.184+0000 7f0ebb064640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0eb40c9200 0x7f0eb40b1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.184+0000 7f0ebb064640 1 --2- 172.21.15.81:0/3065301819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0eb406bc50 0x7f0eb4154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.184+0000 7f0ebb064640 1 -- 172.21.15.81:0/3065301819 >> 172.21.15.81:0/3065301819 conn(0x7f0eb40cb580 msgr2=0x7f0eb4071500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:38.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.185+0000 7f0ebb064640 1 -- 172.21.15.81:0/3065301819 shutdown_connections 2023-11-08T16:39:38.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.185+0000 7f0ebb064640 1 -- 172.21.15.81:0/3065301819 wait complete. 2023-11-08T16:39:38.204 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:39:38.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.418+0000 7efc5a6d4640 1 Processor -- start 2023-11-08T16:39:38.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.418+0000 7efc5a6d4640 1 -- start start 2023-11-08T16:39:38.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.419+0000 7efc5a6d4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc5415a0e0 0x7efc540ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.419+0000 7efc5a6d4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efc540c9bf0 0x7efc540ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.419+0000 7efc5a6d4640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc540ce960 con 0x7efc540ca550 2023-11-08T16:39:38.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.419+0000 7efc5a6d4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc54071980 con 0x7efc540c9bf0 2023-11-08T16:39:38.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.419+0000 7efc5a6d4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc540c9480 con 0x7efc5415a0e0 2023-11-08T16:39:38.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.420+0000 7efc537fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efc540c9bf0 0x7efc540ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:38.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.420+0000 7efc537fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efc540c9bf0 0x7efc540ca010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43504/0 (socket says 172.21.15.81:43504) 2023-11-08T16:39:38.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.420+0000 7efc537fe640 1 -- 172.21.15.81:0/2325178621 learned_addr learned my addr 172.21.15.81:0/2325178621 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:38.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.420+0000 7efc53fff640 1 --2- 172.21.15.81:0/2325178621 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc5415a0e0 0x7efc540ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:38.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.420+0000 7efc537fe640 1 -- 172.21.15.81:0/2325178621 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc5415a0e0 msgr2=0x7efc540ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:38.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.420+0000 7efc537fe640 1 --2- 172.21.15.81:0/2325178621 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc5415a0e0 0x7efc540ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.420+0000 7efc537fe640 1 -- 172.21.15.81:0/2325178621 >> v1:172.21.15.93:6789/0 conn(0x7efc540ca550 legacy=0x7efc5415dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:38.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.420+0000 7efc537fe640 1 -- 172.21.15.81:0/2325178621 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc5415a4c0 con 0x7efc540c9bf0 2023-11-08T16:39:38.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.421+0000 7efc537fe640 1 --2- 172.21.15.81:0/2325178621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc540c9bf0 0x7efc540ca010 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009870 tx=0x7efc3c02db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57d2e4604e1002e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:38.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.422+0000 7efc52ffd640 1 -- 172.21.15.81:0/2325178621 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c02e940 con 0x7efc540c9bf0 2023-11-08T16:39:38.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.422+0000 7efc52ffd640 1 -- 172.21.15.81:0/2325178621 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c02eaa0 con 0x7efc540c9bf0 2023-11-08T16:39:38.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.422+0000 7efc52ffd640 1 -- 172.21.15.81:0/2325178621 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c02ee30 con 0x7efc540c9bf0 2023-11-08T16:39:38.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.422+0000 7efc5a6d4640 1 -- 172.21.15.81:0/2325178621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc540c9bf0 msgr2=0x7efc540ca010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:38.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.422+0000 7efc5a6d4640 1 --2- 172.21.15.81:0/2325178621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc540c9bf0 0x7efc540ca010 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009870 tx=0x7efc3c02db90 comp rx=0 tx=0).stop 2023-11-08T16:39:38.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.423+0000 7efc5a6d4640 1 -- 172.21.15.81:0/2325178621 shutdown_connections 2023-11-08T16:39:38.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.423+0000 7efc5a6d4640 1 --2- 172.21.15.81:0/2325178621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc540c9bf0 0x7efc540ca010 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.423+0000 7efc5a6d4640 1 --2- 172.21.15.81:0/2325178621 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc5415a0e0 0x7efc540ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.423+0000 7efc5a6d4640 1 -- 172.21.15.81:0/2325178621 >> 172.21.15.81:0/2325178621 conn(0x7efc540cb580 msgr2=0x7efc540cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:38.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.423+0000 7efc5a6d4640 1 -- 172.21.15.81:0/2325178621 shutdown_connections 2023-11-08T16:39:38.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.423+0000 7efc5a6d4640 1 -- 172.21.15.81:0/2325178621 wait complete. 2023-11-08T16:39:38.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.424+0000 7efc5a6d4640 1 Processor -- start 2023-11-08T16:39:38.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.425+0000 7efc5a6d4640 1 -- start start 2023-11-08T16:39:38.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.425+0000 7efc5a6d4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc540c9bf0 0x7efc54156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.425+0000 7efc5a6d4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc540ca550 0x7efc541581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc5a6d4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc5415a0e0 0x7efc541586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc5a6d4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc541f2b50 con 0x7efc540c9bf0 2023-11-08T16:39:38.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc5a6d4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc541f2df0 con 0x7efc5415a0e0 2023-11-08T16:39:38.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc5a6d4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc541f30d0 con 0x7efc540ca550 2023-11-08T16:39:38.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc53fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc540c9bf0 0x7efc54156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:38.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc53fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc540c9bf0 0x7efc54156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43512/0 (socket says 172.21.15.81:43512) 2023-11-08T16:39:38.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc53fff640 1 -- 172.21.15.81:0/1225133857 learned_addr learned my addr 172.21.15.81:0/1225133857 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:38.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc537fe640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc540ca550 0x7efc541581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:38.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc58c71640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc5415a0e0 0x7efc541586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:38.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc53fff640 1 -- 172.21.15.81:0/1225133857 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc540ca550 msgr2=0x7efc541581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:38.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc53fff640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc540ca550 0x7efc541581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc53fff640 1 -- 172.21.15.81:0/1225133857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc5415a0e0 msgr2=0x7efc541586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:38.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc53fff640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc5415a0e0 0x7efc541586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.426+0000 7efc53fff640 1 -- 172.21.15.81:0/1225133857 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3c0094d0 con 0x7efc540c9bf0 2023-11-08T16:39:38.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.427+0000 7efc53fff640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc540c9bf0 0x7efc54156bd0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7efc4400d5f0 tx=0x7efc4400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:38.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.427+0000 7efc50ff9640 1 -- 172.21.15.81:0/1225133857 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc440125f0 con 0x7efc540c9bf0 2023-11-08T16:39:38.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.427+0000 7efc50ff9640 1 -- 172.21.15.81:0/1225133857 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44012750 con 0x7efc540c9bf0 2023-11-08T16:39:38.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.427+0000 7efc50ff9640 1 -- 172.21.15.81:0/1225133857 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44012b00 con 0x7efc540c9bf0 2023-11-08T16:39:38.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.427+0000 7efc5a6d4640 1 -- 172.21.15.81:0/1225133857 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc541f33b0 con 0x7efc540c9bf0 2023-11-08T16:39:38.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.427+0000 7efc5a6d4640 1 -- 172.21.15.81:0/1225133857 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc54157110 con 0x7efc540c9bf0 2023-11-08T16:39:38.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.428+0000 7efc5a6d4640 1 -- 172.21.15.81:0/1225133857 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc5404ece0 con 0x7efc540c9bf0 2023-11-08T16:39:38.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.429+0000 7efc50ff9640 1 -- 172.21.15.81:0/1225133857 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efc44012c60 con 0x7efc540c9bf0 2023-11-08T16:39:38.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.430+0000 7efc50ff9640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efc3406ae70 0x7efc3406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.430+0000 7efc50ff9640 1 -- 172.21.15.81:0/1225133857 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7efc44089d20 con 0x7efc540c9bf0 2023-11-08T16:39:38.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.430+0000 7efc537fe640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efc3406ae70 0x7efc3406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:38.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.431+0000 7efc537fe640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efc3406ae70 0x7efc3406d330 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efc3c038000 tx=0x7efc3c032000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:38.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.434+0000 7efc50ff9640 1 -- 172.21.15.81:0/1225133857 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7efc44054b90 con 0x7efc540c9bf0 2023-11-08T16:39:38.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.616+0000 7efc5a6d4640 1 -- 172.21.15.81:0/1225133857 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7efc540cc3b0 con 0x7efc540c9bf0 2023-11-08T16:39:38.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.617+0000 7efc50ff9640 1 -- 172.21.15.81:0/1225133857 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v106) v1 ==== 106+0+11 (secure 0 0 0) 0x7efc440581b0 con 0x7efc540c9bf0 2023-11-08T16:39:38.619 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:39:38.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.619+0000 7efc5a6d4640 1 -- 172.21.15.81:0/1225133857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efc3406ae70 msgr2=0x7efc3406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:38.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.619+0000 7efc5a6d4640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efc3406ae70 0x7efc3406d330 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efc3c038000 tx=0x7efc3c032000 comp rx=0 tx=0).stop 2023-11-08T16:39:38.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.619+0000 7efc5a6d4640 1 -- 172.21.15.81:0/1225133857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc540c9bf0 msgr2=0x7efc54156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:38.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.619+0000 7efc5a6d4640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc540c9bf0 0x7efc54156bd0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7efc4400d5f0 tx=0x7efc4400dab0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.620+0000 7efc5a6d4640 1 -- 172.21.15.81:0/1225133857 shutdown_connections 2023-11-08T16:39:38.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.620+0000 7efc5a6d4640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efc3406ae70 0x7efc3406d330 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.620+0000 7efc5a6d4640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efc5415a0e0 0x7efc541586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.620+0000 7efc5a6d4640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efc540ca550 0x7efc541581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.620+0000 7efc5a6d4640 1 --2- 172.21.15.81:0/1225133857 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efc540c9bf0 0x7efc54156bd0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.620+0000 7efc5a6d4640 1 -- 172.21.15.81:0/1225133857 >> 172.21.15.81:0/1225133857 conn(0x7efc540cb580 msgr2=0x7efc540b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:38.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.620+0000 7efc5a6d4640 1 -- 172.21.15.81:0/1225133857 shutdown_connections 2023-11-08T16:39:38.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.620+0000 7efc5a6d4640 1 -- 172.21.15.81:0/1225133857 wait complete. 2023-11-08T16:39:38.640 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:39:38.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.850+0000 7faab0525640 1 Processor -- start 2023-11-08T16:39:38.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.851+0000 7faab0525640 1 -- start start 2023-11-08T16:39:38.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.852+0000 7faab0525640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faaa815d310 0x7faaa80d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.852+0000 7faab0525640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7faaa80d1300 0x7faaa80cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.852+0000 7faab0525640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faaa80b6100 con 0x7faaa80d0000 2023-11-08T16:39:38.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.852+0000 7faab0525640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa80b6480 con 0x7faaa815d310 2023-11-08T16:39:38.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.852+0000 7faab0525640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaa80d1b50 con 0x7faaa80d1300 2023-11-08T16:39:38.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.852+0000 7faaae2c1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faaa815d310 0x7faaa80d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:38.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.852+0000 7faaae2c1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faaa815d310 0x7faaa80d2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43522/0 (socket says 172.21.15.81:43522) 2023-11-08T16:39:38.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.852+0000 7faaae2c1640 1 -- 172.21.15.81:0/2191624818 learned_addr learned my addr 172.21.15.81:0/2191624818 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:38.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.853+0000 7faaae2c1640 1 -- 172.21.15.81:0/2191624818 >> v2:172.21.15.93:3300/0 conn(0x7faaa80d1300 msgr2=0x7faaa80cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:38.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.853+0000 7faaadac0640 1 --2- 172.21.15.81:0/2191624818 >> v2:172.21.15.93:3300/0 conn(0x7faaa80d1300 0x7faaa80cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:38.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.853+0000 7faaae2c1640 1 --2- 172.21.15.81:0/2191624818 >> v2:172.21.15.93:3300/0 conn(0x7faaa80d1300 0x7faaa80cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.853+0000 7faaae2c1640 1 -- 172.21.15.81:0/2191624818 >> v1:172.21.15.93:6789/0 conn(0x7faaa80d0000 legacy=0x7faaa80b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:38.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.853+0000 7faaae2c1640 1 -- 172.21.15.81:0/2191624818 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80d1e30 con 0x7faaa815d310 2023-11-08T16:39:38.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.854+0000 7faaae2c1640 1 --2- 172.21.15.81:0/2191624818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaa815d310 0x7faaa80d2b70 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7faaa808c570 tx=0x7faaa402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=200c581794a62177 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:38.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.854+0000 7faaad2bf640 1 -- 172.21.15.81:0/2191624818 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4030860 con 0x7faaa815d310 2023-11-08T16:39:38.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.854+0000 7faaad2bf640 1 -- 172.21.15.81:0/2191624818 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa40309c0 con 0x7faaa815d310 2023-11-08T16:39:38.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.854+0000 7faaad2bf640 1 -- 172.21.15.81:0/2191624818 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4030d50 con 0x7faaa815d310 2023-11-08T16:39:38.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.855+0000 7faab0525640 1 -- 172.21.15.81:0/2191624818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaa815d310 msgr2=0x7faaa80d2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:38.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.855+0000 7faab0525640 1 --2- 172.21.15.81:0/2191624818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaa815d310 0x7faaa80d2b70 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7faaa808c570 tx=0x7faaa402b9a0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.855+0000 7faab0525640 1 -- 172.21.15.81:0/2191624818 shutdown_connections 2023-11-08T16:39:38.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.855+0000 7faab0525640 1 --2- 172.21.15.81:0/2191624818 >> v2:172.21.15.93:3300/0 conn(0x7faaa80d1300 0x7faaa80cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.855+0000 7faab0525640 1 --2- 172.21.15.81:0/2191624818 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaa815d310 0x7faaa80d2b70 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.855+0000 7faab0525640 1 -- 172.21.15.81:0/2191624818 >> 172.21.15.81:0/2191624818 conn(0x7faaa806aac0 msgr2=0x7faaa806aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:38.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.855+0000 7faab0525640 1 -- 172.21.15.81:0/2191624818 shutdown_connections 2023-11-08T16:39:38.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.855+0000 7faab0525640 1 -- 172.21.15.81:0/2191624818 wait complete. 2023-11-08T16:39:38.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.858+0000 7faab0525640 1 Processor -- start 2023-11-08T16:39:38.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.858+0000 7faab0525640 1 -- start start 2023-11-08T16:39:38.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.859+0000 7faab0525640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaa80d0000 0x7faaa814be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.859+0000 7faab0525640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faaa80d1300 0x7faaa80b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.859+0000 7faab0525640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaa815d310 0x7faaa815e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.859+0000 7faab0525640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa815ea50 con 0x7faaa80d0000 2023-11-08T16:39:38.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.859+0000 7faab0525640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa815ed30 con 0x7faaa80d1300 2023-11-08T16:39:38.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.859+0000 7faab0525640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaa815f010 con 0x7faaa815d310 2023-11-08T16:39:38.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.859+0000 7faaae2c1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaa80d0000 0x7faaa814be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:38.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.859+0000 7faaae2c1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaa80d0000 0x7faaa814be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43528/0 (socket says 172.21.15.81:43528) 2023-11-08T16:39:38.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.859+0000 7faaae2c1640 1 -- 172.21.15.81:0/2532887894 learned_addr learned my addr 172.21.15.81:0/2532887894 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:38.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faaadac0640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faaa80d1300 0x7faaa80b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:38.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faaae2c1640 1 -- 172.21.15.81:0/2532887894 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaa815d310 msgr2=0x7faaa815e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:38.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faaae2c1640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaa815d310 0x7faaa815e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faaae2c1640 1 -- 172.21.15.81:0/2532887894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faaa80d1300 msgr2=0x7faaa80b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:38.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faaae2c1640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faaa80d1300 0x7faaa80b53e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:38.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faaae2c1640 1 -- 172.21.15.81:0/2532887894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa40094d0 con 0x7faaa80d0000 2023-11-08T16:39:38.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faaadac0640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faaa80d1300 0x7faaa80b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:38.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faaae2c1640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaa80d0000 0x7faaa814be50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7faaa4000c00 tx=0x7faaa4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:38.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faa9effd640 1 -- 172.21.15.81:0/2532887894 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4030830 con 0x7faaa80d0000 2023-11-08T16:39:38.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faa9effd640 1 -- 172.21.15.81:0/2532887894 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa4043400 con 0x7faaa80d0000 2023-11-08T16:39:38.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faab0525640 1 -- 172.21.15.81:0/2532887894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa815f2f0 con 0x7faaa80d0000 2023-11-08T16:39:38.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.860+0000 7faa9effd640 1 -- 172.21.15.81:0/2532887894 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa40437b0 con 0x7faaa80d0000 2023-11-08T16:39:38.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.861+0000 7faab0525640 1 -- 172.21.15.81:0/2532887894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa80b5920 con 0x7faaa80d0000 2023-11-08T16:39:38.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.862+0000 7faab0525640 1 -- 172.21.15.81:0/2532887894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaa815f7b0 con 0x7faaa80d0000 2023-11-08T16:39:38.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.863+0000 7faa9effd640 1 -- 172.21.15.81:0/2532887894 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faaa4043910 con 0x7faaa80d0000 2023-11-08T16:39:38.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.863+0000 7faa9effd640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa8c06ae70 0x7faa8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:38.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.864+0000 7faa9effd640 1 -- 172.21.15.81:0/2532887894 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7faaa403e070 con 0x7faaa80d0000 2023-11-08T16:39:38.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.864+0000 7faaadac0640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa8c06ae70 0x7faa8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:38.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.864+0000 7faaadac0640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa8c06ae70 0x7faa8c06d330 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faaa815a9b0 tx=0x7faa98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:38.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:38.868+0000 7faa9effd640 1 -- 172.21.15.81:0/2532887894 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faaa4077b50 con 0x7faaa80d0000 2023-11-08T16:39:39.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.068+0000 7faab0525640 1 -- 172.21.15.81:0/2532887894 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faaa80743f0 con 0x7faaa80d0000 2023-11-08T16:39:39.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.069+0000 7faa9effd640 1 -- 172.21.15.81:0/2532887894 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) v1 ==== 80+0+0 (secure 0 0 0) 0x7faaa407b170 con 0x7faaa80d0000 2023-11-08T16:39:39.071 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:39:39.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 -- 172.21.15.81:0/2532887894 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa8c06ae70 msgr2=0x7faa8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa8c06ae70 0x7faa8c06d330 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faaa815a9b0 tx=0x7faa98008040 comp rx=0 tx=0).stop 2023-11-08T16:39:39.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 -- 172.21.15.81:0/2532887894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaa80d0000 msgr2=0x7faaa814be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaa80d0000 0x7faaa814be50 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7faaa4000c00 tx=0x7faaa4004730 comp rx=0 tx=0).stop 2023-11-08T16:39:39.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 -- 172.21.15.81:0/2532887894 shutdown_connections 2023-11-08T16:39:39.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faa8c06ae70 0x7faa8c06d330 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faaa815d310 0x7faaa815e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faaa80d1300 0x7faaa80b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 --2- 172.21.15.81:0/2532887894 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faaa80d0000 0x7faaa814be50 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 -- 172.21.15.81:0/2532887894 >> 172.21.15.81:0/2532887894 conn(0x7faaa806aac0 msgr2=0x7faaa80cc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:39.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 -- 172.21.15.81:0/2532887894 shutdown_connections 2023-11-08T16:39:39.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.072+0000 7faab0525640 1 -- 172.21.15.81:0/2532887894 wait complete. 2023-11-08T16:39:39.091 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:39:39.091 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:39:39.140 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:39:39.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.397+0000 7f4480856640 1 Processor -- start 2023-11-08T16:39:39.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.397+0000 7f4480856640 1 -- start start 2023-11-08T16:39:39.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.398+0000 7f4480856640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f447c0ade50 0x7f447c0ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:39.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.398+0000 7f4480856640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f447c15a680 0x7f447c15aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:39.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.398+0000 7f4480856640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f447c06b630 con 0x7f447c15b020 2023-11-08T16:39:39.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.399+0000 7f4480856640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f447c0b1860 con 0x7f447c15a680 2023-11-08T16:39:39.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.399+0000 7f4480856640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c0ca9c0 con 0x7f447c0ade50 2023-11-08T16:39:39.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.399+0000 7f4473fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f447c15a680 0x7f447c15aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:39.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.399+0000 7f447ad9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f447c15b020 0x7f447c0c9060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59510/0 (socket says 172.21.15.81:59510) 2023-11-08T16:39:39.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.399+0000 7f447ad9d640 1 -- 172.21.15.81:0/3904915837 learned_addr learned my addr 172.21.15.81:0/3904915837 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:39.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.399+0000 7f447a59c640 1 --2- 172.21.15.81:0/3904915837 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f447c0ade50 0x7f447c0ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:39.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.399+0000 7f4473fff640 1 -- 172.21.15.81:0/3904915837 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f447c0ade50 msgr2=0x7f447c0ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:39.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.399+0000 7f4473fff640 1 --2- 172.21.15.81:0/3904915837 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f447c0ade50 0x7f447c0ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.399+0000 7f4473fff640 1 -- 172.21.15.81:0/3904915837 >> v1:172.21.15.81:6789/0 conn(0x7f447c15b020 legacy=0x7f447c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.399+0000 7f4473fff640 1 -- 172.21.15.81:0/3904915837 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f447c0cabb0 con 0x7f447c15a680 2023-11-08T16:39:39.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.401+0000 7f4473fff640 1 --2- 172.21.15.81:0/3904915837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447c15a680 0x7f447c15aae0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f44640097a0 tx=0x7f446402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2874f3d78a966f97 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:39.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.401+0000 7f4479d9b640 1 -- 172.21.15.81:0/3904915837 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446402d040 con 0x7f447c15a680 2023-11-08T16:39:39.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.401+0000 7f4479d9b640 1 -- 172.21.15.81:0/3904915837 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4464031810 con 0x7f447c15a680 2023-11-08T16:39:39.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.401+0000 7f4479d9b640 1 -- 172.21.15.81:0/3904915837 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464031ba0 con 0x7f447c15a680 2023-11-08T16:39:39.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.402+0000 7f4480856640 1 -- 172.21.15.81:0/3904915837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447c15a680 msgr2=0x7f447c15aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.402+0000 7f4480856640 1 --2- 172.21.15.81:0/3904915837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447c15a680 0x7f447c15aae0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f44640097a0 tx=0x7f446402b8b0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.402+0000 7f4480856640 1 -- 172.21.15.81:0/3904915837 shutdown_connections 2023-11-08T16:39:39.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.402+0000 7f4480856640 1 --2- 172.21.15.81:0/3904915837 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447c15a680 0x7f447c15aae0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.402+0000 7f4480856640 1 --2- 172.21.15.81:0/3904915837 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f447c0ade50 0x7f447c0ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.402+0000 7f4480856640 1 -- 172.21.15.81:0/3904915837 >> 172.21.15.81:0/3904915837 conn(0x7f447c0cb490 msgr2=0x7f447c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:39.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.402+0000 7f4480856640 1 -- 172.21.15.81:0/3904915837 shutdown_connections 2023-11-08T16:39:39.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.402+0000 7f4480856640 1 -- 172.21.15.81:0/3904915837 wait complete. 2023-11-08T16:39:39.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.404+0000 7f4480856640 1 Processor -- start 2023-11-08T16:39:39.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.405+0000 7f4480856640 1 -- start start 2023-11-08T16:39:39.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.406+0000 7f4480856640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f447c0ade50 0x7f447c154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:39.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.406+0000 7f4480856640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447c15a680 0x7f447c1587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:39.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.406+0000 7f4480856640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f447c15b020 0x7f447c14adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:39.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.406+0000 7f4480856640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c1582d0 con 0x7f447c15a680 2023-11-08T16:39:39.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.406+0000 7f4480856640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f447c155e00 con 0x7f447c0ade50 2023-11-08T16:39:39.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.406+0000 7f4480856640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f447c155000 con 0x7f447c15b020 2023-11-08T16:39:39.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.406+0000 7f4473fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447c15a680 0x7f447c1587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:39.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.406+0000 7f4473fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447c15a680 0x7f447c1587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43550/0 (socket says 172.21.15.81:43550) 2023-11-08T16:39:39.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.406+0000 7f4473fff640 1 -- 172.21.15.81:0/2444806128 learned_addr learned my addr 172.21.15.81:0/2444806128 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:39.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f447a59c640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f447c0ade50 0x7f447c154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:39.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f4473fff640 1 -- 172.21.15.81:0/2444806128 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f447c15b020 msgr2=0x7f447c14adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f447ad9d640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f447c15b020 0x7f447c14adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:39.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f4473fff640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f447c15b020 0x7f447c14adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f4473fff640 1 -- 172.21.15.81:0/2444806128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f447c0ade50 msgr2=0x7f447c154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f4473fff640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f447c0ade50 0x7f447c154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f4473fff640 1 -- 172.21.15.81:0/2444806128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464009400 con 0x7f447c15a680 2023-11-08T16:39:39.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f447a59c640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f447c0ade50 0x7f447c154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:39.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f4473fff640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447c15a680 0x7f447c1587b0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f446402b8f0 tx=0x7f44640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:39.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f4472ffd640 1 -- 172.21.15.81:0/2444806128 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446403ace0 con 0x7f447c15a680 2023-11-08T16:39:39.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f4472ffd640 1 -- 172.21.15.81:0/2444806128 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446403ae40 con 0x7f447c15a680 2023-11-08T16:39:39.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f4472ffd640 1 -- 172.21.15.81:0/2444806128 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4464044630 con 0x7f447c15a680 2023-11-08T16:39:39.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.407+0000 7f4480856640 1 -- 172.21.15.81:0/2444806128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f447c1552e0 con 0x7f447c15a680 2023-11-08T16:39:39.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.408+0000 7f4480856640 1 -- 172.21.15.81:0/2444806128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f447c1558a0 con 0x7f447c15a680 2023-11-08T16:39:39.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.409+0000 7f4480856640 1 -- 172.21.15.81:0/2444806128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f447c04ece0 con 0x7f447c15a680 2023-11-08T16:39:39.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.410+0000 7f4472ffd640 1 -- 172.21.15.81:0/2444806128 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f446404d450 con 0x7f447c15a680 2023-11-08T16:39:39.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.410+0000 7f4472ffd640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f445c06aea0 0x7f445c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:39.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.410+0000 7f4472ffd640 1 -- 172.21.15.81:0/2444806128 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f44640abc00 con 0x7f447c15a680 2023-11-08T16:39:39.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.411+0000 7f447a59c640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f445c06aea0 0x7f445c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:39.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.411+0000 7f447a59c640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f445c06aea0 0x7f445c06d360 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f446c006fd0 tx=0x7f446c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:39.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.415+0000 7f4472ffd640 1 -- 172.21.15.81:0/2444806128 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44640769f0 con 0x7f447c15a680 2023-11-08T16:39:39.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.606+0000 7f4480856640 1 -- 172.21.15.81:0/2444806128 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f447c0cc1d0 con 0x7f447c15a680 2023-11-08T16:39:39.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.608+0000 7f4472ffd640 1 -- 172.21.15.81:0/2444806128 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+18241 (secure 0 0 0) 0x7f446407a010 con 0x7f447c15a680 2023-11-08T16:39:39.610 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:39.610 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":106,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:39:22.303415+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:39:07.492834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T16:39:08.634345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:39:19.654916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:39:39.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.610+0000 7f4480856640 1 -- 172.21.15.81:0/2444806128 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f445c06aea0 msgr2=0x7f445c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.610+0000 7f4480856640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f445c06aea0 0x7f445c06d360 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f446c006fd0 tx=0x7f446c008040 comp rx=0 tx=0).stop 2023-11-08T16:39:39.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.610+0000 7f4480856640 1 -- 172.21.15.81:0/2444806128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447c15a680 msgr2=0x7f447c1587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.610+0000 7f4480856640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447c15a680 0x7f447c1587b0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f446402b8f0 tx=0x7f44640046c0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.611+0000 7f4480856640 1 -- 172.21.15.81:0/2444806128 shutdown_connections 2023-11-08T16:39:39.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.611+0000 7f4480856640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f445c06aea0 0x7f445c06d360 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.611+0000 7f4480856640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f447c15b020 0x7f447c14adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.611+0000 7f4480856640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f447c15a680 0x7f447c1587b0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.611+0000 7f4480856640 1 --2- 172.21.15.81:0/2444806128 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f447c0ade50 0x7f447c154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.611+0000 7f4480856640 1 -- 172.21.15.81:0/2444806128 >> 172.21.15.81:0/2444806128 conn(0x7f447c0cb490 msgr2=0x7f447c0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:39.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.611+0000 7f4480856640 1 -- 172.21.15.81:0/2444806128 shutdown_connections 2023-11-08T16:39:39.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.611+0000 7f4480856640 1 -- 172.21.15.81:0/2444806128 wait complete. 2023-11-08T16:39:39.630 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:39:07.492834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:39:08.634345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:39:19.654916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:39:39.632 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:39:39.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.847+0000 7f09a9bf5640 1 Processor -- start 2023-11-08T16:39:39.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.848+0000 7f09a9bf5640 1 -- start start 2023-11-08T16:39:39.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.849+0000 7f09a9bf5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f09a40ce0d0 0x7f09a40caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:39.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.849+0000 7f09a9bf5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a40cda10 con 0x7f09a415bfc0 2023-11-08T16:39:39.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.849+0000 7f09a9bf5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09a406fa30 con 0x7f09a415f280 2023-11-08T16:39:39.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.849+0000 7f09a9bf5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09a40ce4b0 con 0x7f09a40ce0d0 2023-11-08T16:39:39.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.849+0000 7f09a37fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f09a40ce0d0 0x7f09a40caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:39.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.849+0000 7f09a2ffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f09a415bfc0 0x7f09a415eb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59526/0 (socket says 172.21.15.81:59526) 2023-11-08T16:39:39.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.849+0000 7f09a2ffd640 1 -- 172.21.15.81:0/1107778905 learned_addr learned my addr 172.21.15.81:0/1107778905 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:39.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.849+0000 7f09a37fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f09a40ce0d0 0x7f09a40caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43556/0 (socket says 172.21.15.81:43556) 2023-11-08T16:39:39.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.850+0000 7f09a37fe640 1 -- 172.21.15.81:0/1107778905 >> v1:172.21.15.93:6789/0 conn(0x7f09a415f280 legacy=0x7f09a41516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:39.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.850+0000 7f09a37fe640 1 -- 172.21.15.81:0/1107778905 >> v1:172.21.15.81:6789/0 conn(0x7f09a415bfc0 legacy=0x7f09a415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.850+0000 7f09a37fe640 1 -- 172.21.15.81:0/1107778905 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a406fca0 con 0x7f09a40ce0d0 2023-11-08T16:39:39.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.851+0000 7f09a37fe640 1 --2- 172.21.15.81:0/1107778905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09a40ce0d0 0x7f09a40caf20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f09940097a0 tx=0x7f099402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a41e3ac1e8d0b5f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:39.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.851+0000 7f09a27fc640 1 -- 172.21.15.81:0/1107778905 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994031830 con 0x7f09a40ce0d0 2023-11-08T16:39:39.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.851+0000 7f09a27fc640 1 -- 172.21.15.81:0/1107778905 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0994031990 con 0x7f09a40ce0d0 2023-11-08T16:39:39.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.851+0000 7f09a27fc640 1 -- 172.21.15.81:0/1107778905 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0994031d20 con 0x7f09a40ce0d0 2023-11-08T16:39:39.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.852+0000 7f09a9bf5640 1 -- 172.21.15.81:0/1107778905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09a40ce0d0 msgr2=0x7f09a40caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.852+0000 7f09a9bf5640 1 --2- 172.21.15.81:0/1107778905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09a40ce0d0 0x7f09a40caf20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f09940097a0 tx=0x7f099402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.852+0000 7f09a9bf5640 1 -- 172.21.15.81:0/1107778905 shutdown_connections 2023-11-08T16:39:39.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.852+0000 7f09a9bf5640 1 --2- 172.21.15.81:0/1107778905 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09a40ce0d0 0x7f09a40caf20 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.852+0000 7f09a9bf5640 1 -- 172.21.15.81:0/1107778905 >> 172.21.15.81:0/1107778905 conn(0x7f09a40cb580 msgr2=0x7f09a40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:39.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.852+0000 7f09a9bf5640 1 -- 172.21.15.81:0/1107778905 shutdown_connections 2023-11-08T16:39:39.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.852+0000 7f09a9bf5640 1 -- 172.21.15.81:0/1107778905 wait complete. 2023-11-08T16:39:39.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.854+0000 7f09a9bf5640 1 Processor -- start 2023-11-08T16:39:39.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.854+0000 7f09a9bf5640 1 -- start start 2023-11-08T16:39:39.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.855+0000 7f09a9bf5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09a40ce0d0 0x7f09a414ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:39.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.855+0000 7f09a9bf5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09a415bfc0 0x7f09a415a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:39.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.855+0000 7f09a9bf5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09a415f280 0x7f09a415a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:39.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.855+0000 7f09a9bf5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40b40d0 con 0x7f09a40ce0d0 2023-11-08T16:39:39.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.855+0000 7f09a9bf5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40c4700 con 0x7f09a415f280 2023-11-08T16:39:39.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.855+0000 7f09a9bf5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09a40c4970 con 0x7f09a415bfc0 2023-11-08T16:39:39.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.855+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09a40ce0d0 0x7f09a414ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:39.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.855+0000 7f09a37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09a40ce0d0 0x7f09a414ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43570/0 (socket says 172.21.15.81:43570) 2023-11-08T16:39:39.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.855+0000 7f09a37fe640 1 -- 172.21.15.81:0/3458558744 learned_addr learned my addr 172.21.15.81:0/3458558744 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:39.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.855+0000 7f09a3fff640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09a415f280 0x7f09a415a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:39.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.855+0000 7f09a2ffd640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09a415bfc0 0x7f09a415a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:39.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f09a37fe640 1 -- 172.21.15.81:0/3458558744 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09a415bfc0 msgr2=0x7f09a415a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f09a37fe640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09a415bfc0 0x7f09a415a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f09a37fe640 1 -- 172.21.15.81:0/3458558744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09a415f280 msgr2=0x7f09a415a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:39.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f09a37fe640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09a415f280 0x7f09a415a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:39.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f09a37fe640 1 -- 172.21.15.81:0/3458558744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0994009400 con 0x7f09a40ce0d0 2023-11-08T16:39:39.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f09a3fff640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09a415f280 0x7f09a415a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:39.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f09a37fe640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09a40ce0d0 0x7f09a414ef30 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f09940098c0 tx=0x7f09940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:39.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f098bfff640 1 -- 172.21.15.81:0/3458558744 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099403ba70 con 0x7f09a40ce0d0 2023-11-08T16:39:39.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f098bfff640 1 -- 172.21.15.81:0/3458558744 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099403bbf0 con 0x7f09a40ce0d0 2023-11-08T16:39:39.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f098bfff640 1 -- 172.21.15.81:0/3458558744 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09940453f0 con 0x7f09a40ce0d0 2023-11-08T16:39:39.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f09a9bf5640 1 -- 172.21.15.81:0/3458558744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a40c4c50 con 0x7f09a40ce0d0 2023-11-08T16:39:39.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.856+0000 7f09a9bf5640 1 -- 172.21.15.81:0/3458558744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a40c5210 con 0x7f09a40ce0d0 2023-11-08T16:39:39.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.858+0000 7f09a9bf5640 1 -- 172.21.15.81:0/3458558744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a40c5400 con 0x7f09a40ce0d0 2023-11-08T16:39:39.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.858+0000 7f098bfff640 1 -- 172.21.15.81:0/3458558744 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f099404e450 con 0x7f09a40ce0d0 2023-11-08T16:39:39.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.859+0000 7f098bfff640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099006ae70 0x7f099006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:39.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.859+0000 7f098bfff640 1 -- 172.21.15.81:0/3458558744 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f09940ab9a0 con 0x7f09a40ce0d0 2023-11-08T16:39:39.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.860+0000 7f09a2ffd640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099006ae70 0x7f099006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:39.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.860+0000 7f09a2ffd640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099006ae70 0x7f099006d330 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f09a406d8f0 tx=0x7f0998009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:39.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:39.863+0000 7f098bfff640 1 -- 172.21.15.81:0/3458558744 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0994076940 con 0x7f09a40ce0d0 2023-11-08T16:39:40.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.051+0000 7f09a9bf5640 1 -- 172.21.15.81:0/3458558744 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f09a404ece0 con 0x7f09a40ce0d0 2023-11-08T16:39:40.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.052+0000 7f098bfff640 1 -- 172.21.15.81:0/3458558744 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v106) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0994079f60 con 0x7f09a40ce0d0 2023-11-08T16:39:40.054 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:39:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.054+0000 7f09a9bf5640 1 -- 172.21.15.81:0/3458558744 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099006ae70 msgr2=0x7f099006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.054+0000 7f09a9bf5640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099006ae70 0x7f099006d330 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f09a406d8f0 tx=0x7f0998009040 comp rx=0 tx=0).stop 2023-11-08T16:39:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.054+0000 7f09a9bf5640 1 -- 172.21.15.81:0/3458558744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09a40ce0d0 msgr2=0x7f09a414ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.054+0000 7f09a9bf5640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09a40ce0d0 0x7f09a414ef30 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f09940098c0 tx=0x7f09940046c0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.054+0000 7f09a9bf5640 1 -- 172.21.15.81:0/3458558744 shutdown_connections 2023-11-08T16:39:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.054+0000 7f09a9bf5640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f099006ae70 0x7f099006d330 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.054+0000 7f09a9bf5640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09a415f280 0x7f09a415a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.054+0000 7f09a9bf5640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09a415bfc0 0x7f09a415a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.054+0000 7f09a9bf5640 1 --2- 172.21.15.81:0/3458558744 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09a40ce0d0 0x7f09a414ef30 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.054+0000 7f09a9bf5640 1 -- 172.21.15.81:0/3458558744 >> 172.21.15.81:0/3458558744 conn(0x7f09a40cb580 msgr2=0x7f09a40ae990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.055+0000 7f09a9bf5640 1 -- 172.21.15.81:0/3458558744 shutdown_connections 2023-11-08T16:39:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.055+0000 7f09a9bf5640 1 -- 172.21.15.81:0/3458558744 wait complete. 2023-11-08T16:39:40.074 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:39:40.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.283+0000 7f3b18bc9640 1 Processor -- start 2023-11-08T16:39:40.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.284+0000 7f3b18bc9640 1 -- start start 2023-11-08T16:39:40.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.285+0000 7f3b18bc9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3b140ce390 0x7f3b1415e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.285+0000 7f3b18bc9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b1406be20 0x7f3b140ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.285+0000 7f3b18bc9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1415e8d0 con 0x7f3b140c9200 2023-11-08T16:39:40.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.285+0000 7f3b18bc9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b140ca2f0 con 0x7f3b1406be20 2023-11-08T16:39:40.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.285+0000 7f3b18bc9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b140ce770 con 0x7f3b140ce390 2023-11-08T16:39:40.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.286+0000 7f3b12d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b1406be20 0x7f3b140ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.286+0000 7f3b12d9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b1406be20 0x7f3b140ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43576/0 (socket says 172.21.15.81:43576) 2023-11-08T16:39:40.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.286+0000 7f3b12d9d640 1 -- 172.21.15.81:0/2911529369 learned_addr learned my addr 172.21.15.81:0/2911529369 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:40.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.286+0000 7f3b1259c640 1 --2- 172.21.15.81:0/2911529369 >> v2:172.21.15.93:3300/0 conn(0x7f3b140ce390 0x7f3b1415e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.286+0000 7f3b12d9d640 1 -- 172.21.15.81:0/2911529369 >> v2:172.21.15.93:3300/0 conn(0x7f3b140ce390 msgr2=0x7f3b1415e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:40.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.286+0000 7f3b12d9d640 1 --2- 172.21.15.81:0/2911529369 >> v2:172.21.15.93:3300/0 conn(0x7f3b140ce390 0x7f3b1415e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.286+0000 7f3b12d9d640 1 -- 172.21.15.81:0/2911529369 >> v1:172.21.15.93:6789/0 conn(0x7f3b140c9200 legacy=0x7f3b1406b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:40.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.286+0000 7f3b12d9d640 1 -- 172.21.15.81:0/2911529369 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b140c96e0 con 0x7f3b1406be20 2023-11-08T16:39:40.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.287+0000 7f3b12d9d640 1 --2- 172.21.15.81:0/2911529369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b1406be20 0x7f3b140ac8b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009e50 tx=0x7f3b0c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779e262e0142563d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:40.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.287+0000 7f3b11d9b640 1 -- 172.21.15.81:0/2911529369 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c02cb10 con 0x7f3b1406be20 2023-11-08T16:39:40.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.288+0000 7f3b11d9b640 1 -- 172.21.15.81:0/2911529369 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0c02cc70 con 0x7f3b1406be20 2023-11-08T16:39:40.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.288+0000 7f3b11d9b640 1 -- 172.21.15.81:0/2911529369 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c0343f0 con 0x7f3b1406be20 2023-11-08T16:39:40.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.288+0000 7f3b18bc9640 1 -- 172.21.15.81:0/2911529369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b1406be20 msgr2=0x7f3b140ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.288+0000 7f3b18bc9640 1 --2- 172.21.15.81:0/2911529369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b1406be20 0x7f3b140ac8b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009e50 tx=0x7f3b0c02bf70 comp rx=0 tx=0).stop 2023-11-08T16:39:40.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.288+0000 7f3b18bc9640 1 -- 172.21.15.81:0/2911529369 shutdown_connections 2023-11-08T16:39:40.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.288+0000 7f3b18bc9640 1 --2- 172.21.15.81:0/2911529369 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b1406be20 0x7f3b140ac8b0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.288+0000 7f3b18bc9640 1 --2- 172.21.15.81:0/2911529369 >> v2:172.21.15.93:3300/0 conn(0x7f3b140ce390 0x7f3b1415e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.288+0000 7f3b18bc9640 1 -- 172.21.15.81:0/2911529369 >> 172.21.15.81:0/2911529369 conn(0x7f3b140cb580 msgr2=0x7f3b140cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:40.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.288+0000 7f3b18bc9640 1 -- 172.21.15.81:0/2911529369 shutdown_connections 2023-11-08T16:39:40.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.288+0000 7f3b18bc9640 1 -- 172.21.15.81:0/2911529369 wait complete. 2023-11-08T16:39:40.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.290+0000 7f3b18bc9640 1 Processor -- start 2023-11-08T16:39:40.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.290+0000 7f3b18bc9640 1 -- start start 2023-11-08T16:39:40.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.291+0000 7f3b18bc9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b1406be20 0x7f3b1415be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.291+0000 7f3b18bc9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b140c9200 0x7f3b14155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.291+0000 7f3b18bc9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b140ce390 0x7f3b140c7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.291+0000 7f3b18bc9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140b3c40 con 0x7f3b140ce390 2023-11-08T16:39:40.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.291+0000 7f3b18bc9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b14159620 con 0x7f3b1406be20 2023-11-08T16:39:40.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.291+0000 7f3b18bc9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1415c3c0 con 0x7f3b140c9200 2023-11-08T16:39:40.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.291+0000 7f3b12d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b140ce390 0x7f3b140c7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.291+0000 7f3b12d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b140ce390 0x7f3b140c7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43580/0 (socket says 172.21.15.81:43580) 2023-11-08T16:39:40.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.291+0000 7f3b12d9d640 1 -- 172.21.15.81:0/1929952603 learned_addr learned my addr 172.21.15.81:0/1929952603 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:40.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.292+0000 7f3b1259c640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b1406be20 0x7f3b1415be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.292+0000 7f3b09d9b640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b140c9200 0x7f3b14155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.292+0000 7f3b12d9d640 1 -- 172.21.15.81:0/1929952603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b140c9200 msgr2=0x7f3b14155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.292+0000 7f3b12d9d640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b140c9200 0x7f3b14155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.292+0000 7f3b12d9d640 1 -- 172.21.15.81:0/1929952603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b1406be20 msgr2=0x7f3b1415be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.292+0000 7f3b12d9d640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b1406be20 0x7f3b1415be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.292+0000 7f3b12d9d640 1 -- 172.21.15.81:0/1929952603 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c009b30 con 0x7f3b140ce390 2023-11-08T16:39:40.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.292+0000 7f3b09d9b640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b140c9200 0x7f3b14155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:40.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.292+0000 7f3b12d9d640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b140ce390 0x7f3b140c7be0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c036040 tx=0x7f3b0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:40.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.292+0000 7f3b0b7fe640 1 -- 172.21.15.81:0/1929952603 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c009510 con 0x7f3b140ce390 2023-11-08T16:39:40.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.293+0000 7f3b0b7fe640 1 -- 172.21.15.81:0/1929952603 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b0c03e450 con 0x7f3b140ce390 2023-11-08T16:39:40.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.293+0000 7f3b18bc9640 1 -- 172.21.15.81:0/1929952603 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b141562d0 con 0x7f3b140ce390 2023-11-08T16:39:40.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.293+0000 7f3b0b7fe640 1 -- 172.21.15.81:0/1929952603 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b0c03e7e0 con 0x7f3b140ce390 2023-11-08T16:39:40.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.293+0000 7f3b18bc9640 1 -- 172.21.15.81:0/1929952603 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b14155760 con 0x7f3b140ce390 2023-11-08T16:39:40.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.294+0000 7f3b18bc9640 1 -- 172.21.15.81:0/1929952603 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b14156460 con 0x7f3b140ce390 2023-11-08T16:39:40.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.295+0000 7f3b0b7fe640 1 -- 172.21.15.81:0/1929952603 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3b0c03e940 con 0x7f3b140ce390 2023-11-08T16:39:40.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.295+0000 7f3b0b7fe640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3af406ae70 0x7f3af406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.296+0000 7f3b0b7fe640 1 -- 172.21.15.81:0/1929952603 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f3b0c0ac500 con 0x7f3b140ce390 2023-11-08T16:39:40.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.296+0000 7f3b1259c640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3af406ae70 0x7f3af406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.296+0000 7f3b1259c640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3af406ae70 0x7f3af406d330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3b04006fd0 tx=0x7f3b04008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:40.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.300+0000 7f3b0b7fe640 1 -- 172.21.15.81:0/1929952603 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b0c0772e0 con 0x7f3b140ce390 2023-11-08T16:39:40.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.490+0000 7f3b18bc9640 1 -- 172.21.15.81:0/1929952603 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3b1415e8d0 con 0x7f3b140ce390 2023-11-08T16:39:40.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.490+0000 7f3b0b7fe640 1 -- 172.21.15.81:0/1929952603 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v106) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b1415e8d0 con 0x7f3b140ce390 2023-11-08T16:39:40.492 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:39:40.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.493+0000 7f3b18bc9640 1 -- 172.21.15.81:0/1929952603 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3af406ae70 msgr2=0x7f3af406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.493+0000 7f3b18bc9640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3af406ae70 0x7f3af406d330 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3b04006fd0 tx=0x7f3b04008040 comp rx=0 tx=0).stop 2023-11-08T16:39:40.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.493+0000 7f3b18bc9640 1 -- 172.21.15.81:0/1929952603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b140ce390 msgr2=0x7f3b140c7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.493+0000 7f3b18bc9640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b140ce390 0x7f3b140c7be0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c036040 tx=0x7f3b0c004a40 comp rx=0 tx=0).stop 2023-11-08T16:39:40.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.493+0000 7f3b18bc9640 1 -- 172.21.15.81:0/1929952603 shutdown_connections 2023-11-08T16:39:40.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.493+0000 7f3b18bc9640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3af406ae70 0x7f3af406d330 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.493+0000 7f3b18bc9640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b140ce390 0x7f3b140c7be0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.493+0000 7f3b18bc9640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b140c9200 0x7f3b14155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.493+0000 7f3b18bc9640 1 --2- 172.21.15.81:0/1929952603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b1406be20 0x7f3b1415be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.493+0000 7f3b18bc9640 1 -- 172.21.15.81:0/1929952603 >> 172.21.15.81:0/1929952603 conn(0x7f3b140cb580 msgr2=0x7f3b14071550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:40.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.494+0000 7f3b18bc9640 1 -- 172.21.15.81:0/1929952603 shutdown_connections 2023-11-08T16:39:40.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.494+0000 7f3b18bc9640 1 -- 172.21.15.81:0/1929952603 wait complete. 2023-11-08T16:39:40.513 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:39:40.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.726+0000 7ff72307c640 1 Processor -- start 2023-11-08T16:39:40.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.726+0000 7ff72307c640 1 -- start start 2023-11-08T16:39:40.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.728+0000 7ff72307c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff71c0ce390 0x7ff71c15e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.728+0000 7ff72307c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff71c06be20 0x7ff71c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.728+0000 7ff72307c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff71c15e8d0 con 0x7ff71c0c9200 2023-11-08T16:39:40.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.728+0000 7ff72307c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71c0ca2f0 con 0x7ff71c0ce390 2023-11-08T16:39:40.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.728+0000 7ff72307c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff71c0ce770 con 0x7ff71c06be20 2023-11-08T16:39:40.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.728+0000 7ff720e18640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff71c0ce390 0x7ff71c15e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.728+0000 7ff71bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff71c0c9200 0x7ff71c06b710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:59536/0 (socket says 172.21.15.81:59536) 2023-11-08T16:39:40.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.728+0000 7ff71bfff640 1 -- 172.21.15.81:0/3577437422 learned_addr learned my addr 172.21.15.81:0/3577437422 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:40.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.729+0000 7ff721619640 1 --2- 172.21.15.81:0/3577437422 >> v2:172.21.15.93:3300/0 conn(0x7ff71c06be20 0x7ff71c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.729+0000 7ff720e18640 1 -- 172.21.15.81:0/3577437422 >> v2:172.21.15.93:3300/0 conn(0x7ff71c06be20 msgr2=0x7ff71c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:40.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.729+0000 7ff720e18640 1 --2- 172.21.15.81:0/3577437422 >> v2:172.21.15.93:3300/0 conn(0x7ff71c06be20 0x7ff71c0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.729+0000 7ff720e18640 1 -- 172.21.15.81:0/3577437422 >> v1:172.21.15.81:6789/0 conn(0x7ff71c0c9200 legacy=0x7ff71c06b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.729+0000 7ff720e18640 1 -- 172.21.15.81:0/3577437422 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff71c0c96e0 con 0x7ff71c0ce390 2023-11-08T16:39:40.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.729+0000 7ff71b7fe640 1 -- 172.21.15.81:0/3577437422 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504380826 0 0) 0x7ff708001110 con 0x7ff71c0c9200 2023-11-08T16:39:40.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.730+0000 7ff720e18640 1 --2- 172.21.15.81:0/3577437422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff71c0ce390 0x7ff71c15e390 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff7100097a0 tx=0x7ff71002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68054874c5d4298 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:40.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.730+0000 7ff71b7fe640 1 -- 172.21.15.81:0/3577437422 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710030610 con 0x7ff71c0ce390 2023-11-08T16:39:40.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.730+0000 7ff71b7fe640 1 -- 172.21.15.81:0/3577437422 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff710030770 con 0x7ff71c0ce390 2023-11-08T16:39:40.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.730+0000 7ff71b7fe640 1 -- 172.21.15.81:0/3577437422 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff710030b00 con 0x7ff71c0ce390 2023-11-08T16:39:40.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.731+0000 7ff72307c640 1 -- 172.21.15.81:0/3577437422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff71c0ce390 msgr2=0x7ff71c15e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.731+0000 7ff72307c640 1 --2- 172.21.15.81:0/3577437422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff71c0ce390 0x7ff71c15e390 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7ff7100097a0 tx=0x7ff71002b960 comp rx=0 tx=0).stop 2023-11-08T16:39:40.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.731+0000 7ff72307c640 1 -- 172.21.15.81:0/3577437422 shutdown_connections 2023-11-08T16:39:40.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.731+0000 7ff72307c640 1 --2- 172.21.15.81:0/3577437422 >> v2:172.21.15.93:3300/0 conn(0x7ff71c06be20 0x7ff71c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.731+0000 7ff72307c640 1 --2- 172.21.15.81:0/3577437422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff71c0ce390 0x7ff71c15e390 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.731+0000 7ff72307c640 1 -- 172.21.15.81:0/3577437422 >> 172.21.15.81:0/3577437422 conn(0x7ff71c0cb580 msgr2=0x7ff71c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:40.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.731+0000 7ff72307c640 1 -- 172.21.15.81:0/3577437422 shutdown_connections 2023-11-08T16:39:40.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.731+0000 7ff72307c640 1 -- 172.21.15.81:0/3577437422 wait complete. 2023-11-08T16:39:40.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.733+0000 7ff72307c640 1 Processor -- start 2023-11-08T16:39:40.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.733+0000 7ff72307c640 1 -- start start 2023-11-08T16:39:40.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.734+0000 7ff72307c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff71c06be20 0x7ff71c155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.734+0000 7ff72307c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff71c0c9200 0x7ff71c1e3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.734+0000 7ff72307c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff71c0ce390 0x7ff71c1e3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.734+0000 7ff72307c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0b2f60 con 0x7ff71c0c9200 2023-11-08T16:39:40.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.734+0000 7ff72307c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0b2350 con 0x7ff71c0ce390 2023-11-08T16:39:40.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.734+0000 7ff72307c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff71c0b2630 con 0x7ff71c06be20 2023-11-08T16:39:40.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.735+0000 7ff71bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff71c0c9200 0x7ff71c1e3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.735+0000 7ff71bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff71c0c9200 0x7ff71c1e3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43602/0 (socket says 172.21.15.81:43602) 2023-11-08T16:39:40.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.735+0000 7ff71bfff640 1 -- 172.21.15.81:0/1703370930 learned_addr learned my addr 172.21.15.81:0/1703370930 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:40.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.735+0000 7ff720e18640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff71c06be20 0x7ff71c155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.735+0000 7ff721619640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff71c0ce390 0x7ff71c1e3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.735+0000 7ff71bfff640 1 -- 172.21.15.81:0/1703370930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff71c06be20 msgr2=0x7ff71c155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.735+0000 7ff71bfff640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff71c06be20 0x7ff71c155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.735+0000 7ff71bfff640 1 -- 172.21.15.81:0/1703370930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff71c0ce390 msgr2=0x7ff71c1e3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.735+0000 7ff71bfff640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff71c0ce390 0x7ff71c1e3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.735+0000 7ff71bfff640 1 -- 172.21.15.81:0/1703370930 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff710009400 con 0x7ff71c0c9200 2023-11-08T16:39:40.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.735+0000 7ff720e18640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff71c06be20 0x7ff71c155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:40.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.736+0000 7ff71bfff640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff71c0c9200 0x7ff71c1e3780 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ff70800c780 tx=0x7ff70800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:40.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.736+0000 7ff7197fa640 1 -- 172.21.15.81:0/1703370930 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080072d0 con 0x7ff71c0c9200 2023-11-08T16:39:40.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.736+0000 7ff7197fa640 1 -- 172.21.15.81:0/1703370930 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff708007430 con 0x7ff71c0c9200 2023-11-08T16:39:40.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.736+0000 7ff72307c640 1 -- 172.21.15.81:0/1703370930 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff71c0b2910 con 0x7ff71c0c9200 2023-11-08T16:39:40.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.736+0000 7ff7197fa640 1 -- 172.21.15.81:0/1703370930 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708001d10 con 0x7ff71c0c9200 2023-11-08T16:39:40.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.736+0000 7ff72307c640 1 -- 172.21.15.81:0/1703370930 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff71c155db0 con 0x7ff71c0c9200 2023-11-08T16:39:40.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.737+0000 7ff72307c640 1 -- 172.21.15.81:0/1703370930 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71c0b30d0 con 0x7ff71c0c9200 2023-11-08T16:39:40.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.738+0000 7ff7197fa640 1 -- 172.21.15.81:0/1703370930 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff7080047e0 con 0x7ff71c0c9200 2023-11-08T16:39:40.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.739+0000 7ff7197fa640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff70006ae70 0x7ff70006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:40.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.739+0000 7ff7197fa640 1 -- 172.21.15.81:0/1703370930 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7ff708088ac0 con 0x7ff71c0c9200 2023-11-08T16:39:40.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.739+0000 7ff720e18640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff70006ae70 0x7ff70006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:40.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.740+0000 7ff720e18640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff70006ae70 0x7ff70006d330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff710036000 tx=0x7ff71003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:40.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.743+0000 7ff7197fa640 1 -- 172.21.15.81:0/1703370930 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff708053a60 con 0x7ff71c0c9200 2023-11-08T16:39:40.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.933+0000 7ff72307c640 1 -- 172.21.15.81:0/1703370930 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff71c15e8d0 con 0x7ff71c0c9200 2023-11-08T16:39:40.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.934+0000 7ff7197fa640 1 -- 172.21.15.81:0/1703370930 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v106) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff71c15e8d0 con 0x7ff71c0c9200 2023-11-08T16:39:40.935 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:39:40.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.936+0000 7ff72307c640 1 -- 172.21.15.81:0/1703370930 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff70006ae70 msgr2=0x7ff70006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.936+0000 7ff72307c640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff70006ae70 0x7ff70006d330 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff710036000 tx=0x7ff71003b040 comp rx=0 tx=0).stop 2023-11-08T16:39:40.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.936+0000 7ff72307c640 1 -- 172.21.15.81:0/1703370930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff71c0c9200 msgr2=0x7ff71c1e3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:40.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.936+0000 7ff72307c640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff71c0c9200 0x7ff71c1e3780 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7ff70800c780 tx=0x7ff70800cc40 comp rx=0 tx=0).stop 2023-11-08T16:39:40.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.937+0000 7ff72307c640 1 -- 172.21.15.81:0/1703370930 shutdown_connections 2023-11-08T16:39:40.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.937+0000 7ff72307c640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff70006ae70 0x7ff70006d330 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.937+0000 7ff72307c640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff71c0ce390 0x7ff71c1e3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.937+0000 7ff72307c640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff71c0c9200 0x7ff71c1e3780 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.937+0000 7ff72307c640 1 --2- 172.21.15.81:0/1703370930 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff71c06be20 0x7ff71c155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:40.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.937+0000 7ff72307c640 1 -- 172.21.15.81:0/1703370930 >> 172.21.15.81:0/1703370930 conn(0x7ff71c0cb580 msgr2=0x7ff71c0ae470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:40.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.937+0000 7ff72307c640 1 -- 172.21.15.81:0/1703370930 shutdown_connections 2023-11-08T16:39:40.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:40.937+0000 7ff72307c640 1 -- 172.21.15.81:0/1703370930 wait complete. 2023-11-08T16:39:40.957 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:39:41.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.167+0000 7f1c85ffc640 1 Processor -- start 2023-11-08T16:39:41.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.168+0000 7f1c85ffc640 1 -- start start 2023-11-08T16:39:41.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.169+0000 7f1c85ffc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1c800c2020 0x7f1c8006f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.169+0000 7f1c85ffc640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c8006cae0 con 0x7f1c80161730 2023-11-08T16:39:41.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.169+0000 7f1c85ffc640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c800cdb00 con 0x7f1c800cdf90 2023-11-08T16:39:41.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.169+0000 7f1c85ffc640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c80161b10 con 0x7f1c800c2020 2023-11-08T16:39:41.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7f7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f1c80161730 0x7f1c8006d250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60584/0 (socket says 172.21.15.81:60584) 2023-11-08T16:39:41.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7f7fe640 1 -- 172.21.15.81:0/1900089442 learned_addr learned my addr 172.21.15.81:0/1900089442 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:41.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1298359348 0 0) 0x7f1c8006cae0 con 0x7f1c80161730 2023-11-08T16:39:41.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c6c003520 con 0x7f1c80161730 2023-11-08T16:39:41.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7ffff640 1 --2- 172.21.15.81:0/1900089442 >> v2:172.21.15.93:3300/0 conn(0x7f1c800c2020 0x7f1c8006f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 424374472 0 0) 0x7f1c6c003520 con 0x7f1c80161730 2023-11-08T16:39:41.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 >> v2:172.21.15.93:3300/0 conn(0x7f1c800c2020 msgr2=0x7f1c8006f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:41.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7e7fc640 1 --2- 172.21.15.81:0/1900089442 >> v2:172.21.15.93:3300/0 conn(0x7f1c800c2020 0x7f1c8006f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 >> v1:172.21.15.93:6789/0 conn(0x7f1c800cdf90 legacy=0x7f1c800c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:41.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c800ce3d0 con 0x7f1c80161730 2023-11-08T16:39:41.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f1c700019f0 con 0x7f1c80161730 2023-11-08T16:39:41.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.170+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 >> v1:172.21.15.81:6789/0 conn(0x7f1c80161730 legacy=0x7f1c8006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:41.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7e7fc640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1c6c003a30 0x7f1c6c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7e7fc640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 0x7f1c6c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7e7fc640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c6c0052a0 0x7f1c6c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c80161b10 con 0x7f1c6c0047c0 2023-11-08T16:39:41.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c00c170 con 0x7f1c6c0052a0 2023-11-08T16:39:41.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c00c330 con 0x7f1c6c003a30 2023-11-08T16:39:41.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7effd640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 0x7f1c6c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7ffff640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c6c0052a0 0x7f1c6c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7f7fe640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1c6c003a30 0x7f1c6c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7effd640 1 -- 172.21.15.81:0/1900089442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1c6c003a30 msgr2=0x7f1c6c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:41.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7effd640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1c6c003a30 0x7f1c6c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7effd640 1 -- 172.21.15.81:0/1900089442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c6c0052a0 msgr2=0x7f1c6c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:41.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7effd640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c6c0052a0 0x7f1c6c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7effd640 1 -- 172.21.15.81:0/1900089442 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c003520 con 0x7f1c6c0047c0 2023-11-08T16:39:41.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7ffff640 1 -- 172.21.15.81:0/1900089442 reap_dead start 2023-11-08T16:39:41.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.171+0000 7f1c7f7fe640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1c6c003a30 0x7f1c6c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:41.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.173+0000 7f1c7effd640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 0x7f1c6c004c40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f1c68003d80 tx=0x7f1c6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=249568ab87c0ace6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:41.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.173+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68032600 con 0x7f1c6c0047c0 2023-11-08T16:39:41.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.173+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c6802e050 con 0x7f1c6c0047c0 2023-11-08T16:39:41.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.173+0000 7f1c7e7fc640 1 -- 172.21.15.81:0/1900089442 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680329e0 con 0x7f1c6c0047c0 2023-11-08T16:39:41.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.174+0000 7f1c85ffc640 1 -- 172.21.15.81:0/1900089442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 msgr2=0x7f1c6c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:41.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.174+0000 7f1c85ffc640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 0x7f1c6c004c40 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f1c68003d80 tx=0x7f1c6c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.174+0000 7f1c85ffc640 1 -- 172.21.15.81:0/1900089442 shutdown_connections 2023-11-08T16:39:41.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.174+0000 7f1c85ffc640 1 --2- 172.21.15.81:0/1900089442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 0x7f1c6c004c40 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.174+0000 7f1c85ffc640 1 -- 172.21.15.81:0/1900089442 >> 172.21.15.81:0/1900089442 conn(0x7f1c800cb580 msgr2=0x7f1c800cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:41.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.174+0000 7f1c85ffc640 1 -- 172.21.15.81:0/1900089442 shutdown_connections 2023-11-08T16:39:41.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.174+0000 7f1c85ffc640 1 -- 172.21.15.81:0/1900089442 wait complete. 2023-11-08T16:39:41.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.176+0000 7f1c85ffc640 1 Processor -- start 2023-11-08T16:39:41.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.176+0000 7f1c85ffc640 1 -- start start 2023-11-08T16:39:41.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.177+0000 7f1c85ffc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 0x7f1c80159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.177+0000 7f1c85ffc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1c800c6f90 0x7f1c800c7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.177+0000 7f1c85ffc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c8015ad70 0x7f1c8015b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.177+0000 7f1c85ffc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800c5f60 con 0x7f1c6c0047c0 2023-11-08T16:39:41.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.177+0000 7f1c85ffc640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c800b2e60 con 0x7f1c8015ad70 2023-11-08T16:39:41.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.177+0000 7f1c85ffc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c8015a320 con 0x7f1c800c6f90 2023-11-08T16:39:41.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.177+0000 7f1c7f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 0x7f1c80159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.177+0000 7f1c7f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 0x7f1c80159c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54488/0 (socket says 172.21.15.81:54488) 2023-11-08T16:39:41.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.177+0000 7f1c7f7fe640 1 -- 172.21.15.81:0/3243946398 learned_addr learned my addr 172.21.15.81:0/3243946398 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:41.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c7effd640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1c800c6f90 0x7f1c800c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c7ffff640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c8015ad70 0x7f1c8015b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c7f7fe640 1 -- 172.21.15.81:0/3243946398 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1c800c6f90 msgr2=0x7f1c800c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:41.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c7f7fe640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1c800c6f90 0x7f1c800c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c7f7fe640 1 -- 172.21.15.81:0/3243946398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c8015ad70 msgr2=0x7f1c8015b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:41.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c7f7fe640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c8015ad70 0x7f1c8015b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c7f7fe640 1 -- 172.21.15.81:0/3243946398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c8015a8e0 con 0x7f1c6c0047c0 2023-11-08T16:39:41.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c7ffff640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c8015ad70 0x7f1c8015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:41.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c7f7fe640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 0x7f1c80159c00 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f1c700020b0 tx=0x7f1c7000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:41.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c77fff640 1 -- 172.21.15.81:0/3243946398 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70004d60 con 0x7f1c6c0047c0 2023-11-08T16:39:41.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c77fff640 1 -- 172.21.15.81:0/3243946398 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c7000be20 con 0x7f1c6c0047c0 2023-11-08T16:39:41.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c77fff640 1 -- 172.21.15.81:0/3243946398 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c70007310 con 0x7f1c6c0047c0 2023-11-08T16:39:41.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.178+0000 7f1c85ffc640 1 -- 172.21.15.81:0/3243946398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8015a140 con 0x7f1c6c0047c0 2023-11-08T16:39:41.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.179+0000 7f1c85ffc640 1 -- 172.21.15.81:0/3243946398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c800b2b80 con 0x7f1c6c0047c0 2023-11-08T16:39:41.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.180+0000 7f1c85ffc640 1 -- 172.21.15.81:0/3243946398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c8004ece0 con 0x7f1c6c0047c0 2023-11-08T16:39:41.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.181+0000 7f1c77fff640 1 -- 172.21.15.81:0/3243946398 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1c70007470 con 0x7f1c6c0047c0 2023-11-08T16:39:41.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.182+0000 7f1c77fff640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1c6006aea0 0x7f1c6006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.182+0000 7f1c77fff640 1 -- 172.21.15.81:0/3243946398 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f1c70089750 con 0x7f1c6c0047c0 2023-11-08T16:39:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.182+0000 7f1c7effd640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1c6006aea0 0x7f1c6006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.183+0000 7f1c7effd640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1c6006aea0 0x7f1c6006d360 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1c68034000 tx=0x7f1c68009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:41.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.186+0000 7f1c77fff640 1 -- 172.21.15.81:0/3243946398 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1c70054650 con 0x7f1c6c0047c0 2023-11-08T16:39:41.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.374+0000 7f1c85ffc640 1 -- 172.21.15.81:0/3243946398 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f1c8004e490 con 0x7f1c6c0047c0 2023-11-08T16:39:41.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.375+0000 7f1c77fff640 1 -- 172.21.15.81:0/3243946398 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v106) v1 ==== 106+0+11 (secure 0 0 0) 0x7f1c70087e60 con 0x7f1c6c0047c0 2023-11-08T16:39:41.377 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:39:41.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 -- 172.21.15.81:0/3243946398 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1c6006aea0 msgr2=0x7f1c6006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:41.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1c6006aea0 0x7f1c6006d360 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1c68034000 tx=0x7f1c68009030 comp rx=0 tx=0).stop 2023-11-08T16:39:41.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 -- 172.21.15.81:0/3243946398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 msgr2=0x7f1c80159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:41.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 0x7f1c80159c00 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f1c700020b0 tx=0x7f1c7000b4a0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 -- 172.21.15.81:0/3243946398 shutdown_connections 2023-11-08T16:39:41.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1c6006aea0 0x7f1c6006d360 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c8015ad70 0x7f1c8015b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1c800c6f90 0x7f1c800c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 --2- 172.21.15.81:0/3243946398 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1c6c0047c0 0x7f1c80159c00 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 -- 172.21.15.81:0/3243946398 >> 172.21.15.81:0/3243946398 conn(0x7f1c800cb580 msgr2=0x7f1c8015d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:41.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 -- 172.21.15.81:0/3243946398 shutdown_connections 2023-11-08T16:39:41.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.378+0000 7f1c85ffc640 1 -- 172.21.15.81:0/3243946398 wait complete. 2023-11-08T16:39:41.398 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:39:41.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.615+0000 7f8ad4fb6640 1 Processor -- start 2023-11-08T16:39:41.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.616+0000 7f8ad4fb6640 1 -- start start 2023-11-08T16:39:41.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.617+0000 7f8ad4fb6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8ad00cd620 0x7f8ad00d04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.617+0000 7f8ad4fb6640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8ad006b2c0 0x7f8ad006b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.617+0000 7f8ad4fb6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad0161bb0 con 0x7f8ad006e6d0 2023-11-08T16:39:41.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.617+0000 7f8ad4fb6640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad0160fa0 con 0x7f8ad00cd620 2023-11-08T16:39:41.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.617+0000 7f8ad4fb6640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad0161280 con 0x7f8ad006b2c0 2023-11-08T16:39:41.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.618+0000 7f8ace59c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8ad00cd620 0x7f8ad00d04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.618+0000 7f8ace59c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8ad00cd620 0x7f8ad00d04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54496/0 (socket says 172.21.15.81:54496) 2023-11-08T16:39:41.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.618+0000 7f8ace59c640 1 -- 172.21.15.81:0/2097665187 learned_addr learned my addr 172.21.15.81:0/2097665187 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:41.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.618+0000 7f8aced9d640 1 --2- 172.21.15.81:0/2097665187 >> v2:172.21.15.93:3300/0 conn(0x7f8ad006b2c0 0x7f8ad006b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.618+0000 7f8ace59c640 1 -- 172.21.15.81:0/2097665187 >> v2:172.21.15.93:3300/0 conn(0x7f8ad006b2c0 msgr2=0x7f8ad006b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:41.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.618+0000 7f8ace59c640 1 --2- 172.21.15.81:0/2097665187 >> v2:172.21.15.93:3300/0 conn(0x7f8ad006b2c0 0x7f8ad006b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.618+0000 7f8ace59c640 1 -- 172.21.15.81:0/2097665187 >> v1:172.21.15.93:6789/0 conn(0x7f8ad006e6d0 legacy=0x7f8ad00cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:41.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.618+0000 7f8ace59c640 1 -- 172.21.15.81:0/2097665187 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad0161560 con 0x7f8ad00cd620 2023-11-08T16:39:41.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.620+0000 7f8ace59c640 1 --2- 172.21.15.81:0/2097665187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ad00cd620 0x7f8ad00d04b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00afa70 tx=0x7f8ac002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b008c5a15fd4c33 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:41.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.620+0000 7f8acdd9b640 1 -- 172.21.15.81:0/2097665187 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac00317e0 con 0x7f8ad00cd620 2023-11-08T16:39:41.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.620+0000 7f8acdd9b640 1 -- 172.21.15.81:0/2097665187 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac0031940 con 0x7f8ad00cd620 2023-11-08T16:39:41.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.620+0000 7f8acdd9b640 1 -- 172.21.15.81:0/2097665187 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0031cd0 con 0x7f8ad00cd620 2023-11-08T16:39:41.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.621+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2097665187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ad00cd620 msgr2=0x7f8ad00d04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:41.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.621+0000 7f8ad4fb6640 1 --2- 172.21.15.81:0/2097665187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ad00cd620 0x7f8ad00d04b0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00afa70 tx=0x7f8ac002b880 comp rx=0 tx=0).stop 2023-11-08T16:39:41.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.621+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2097665187 shutdown_connections 2023-11-08T16:39:41.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.621+0000 7f8ad4fb6640 1 --2- 172.21.15.81:0/2097665187 >> v2:172.21.15.93:3300/0 conn(0x7f8ad006b2c0 0x7f8ad006b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.621+0000 7f8ad4fb6640 1 --2- 172.21.15.81:0/2097665187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ad00cd620 0x7f8ad00d04b0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.621+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2097665187 >> 172.21.15.81:0/2097665187 conn(0x7f8ad006dac0 msgr2=0x7f8ad006dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:41.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.621+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2097665187 shutdown_connections 2023-11-08T16:39:41.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.621+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2097665187 wait complete. 2023-11-08T16:39:41.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.623+0000 7f8ad4fb6640 1 Processor -- start 2023-11-08T16:39:41.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.623+0000 7f8ad4fb6640 1 -- start start 2023-11-08T16:39:41.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.624+0000 7f8ad4fb6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8ad006b2c0 0x7f8ad00b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.624+0000 7f8ad4fb6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8ad006e6d0 0x7f8ad0154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.624+0000 7f8ad4fb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ad00cd620 0x7f8ad015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.624+0000 7f8ad4fb6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad00b5810 con 0x7f8ad00cd620 2023-11-08T16:39:41.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.624+0000 7f8ad4fb6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad00b2c10 con 0x7f8ad006e6d0 2023-11-08T16:39:41.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.624+0000 7f8ad4fb6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad00b51c0 con 0x7f8ad006b2c0 2023-11-08T16:39:41.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.624+0000 7f8aced9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ad00cd620 0x7f8ad015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.624+0000 7f8aced9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ad00cd620 0x7f8ad015d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54502/0 (socket says 172.21.15.81:54502) 2023-11-08T16:39:41.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.624+0000 7f8aced9d640 1 -- 172.21.15.81:0/2606236964 learned_addr learned my addr 172.21.15.81:0/2606236964 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:41.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8ace59c640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8ad006b2c0 0x7f8ad00b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8ac7fff640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8ad006e6d0 0x7f8ad0154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8aced9d640 1 -- 172.21.15.81:0/2606236964 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8ad006b2c0 msgr2=0x7f8ad00b5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:41.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8aced9d640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8ad006b2c0 0x7f8ad00b5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8aced9d640 1 -- 172.21.15.81:0/2606236964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8ad006e6d0 msgr2=0x7f8ad0154d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:41.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8aced9d640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8ad006e6d0 0x7f8ad0154d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8aced9d640 1 -- 172.21.15.81:0/2606236964 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac00094d0 con 0x7f8ad00cd620 2023-11-08T16:39:41.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8aced9d640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ad00cd620 0x7f8ad015d7d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8007e60 tx=0x7f8ac800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:41.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8ac6ffd640 1 -- 172.21.15.81:0/2606236964 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8014420 con 0x7f8ad00cd620 2023-11-08T16:39:41.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8ac6ffd640 1 -- 172.21.15.81:0/2606236964 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac80145a0 con 0x7f8ad00cd620 2023-11-08T16:39:41.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8ac6ffd640 1 -- 172.21.15.81:0/2606236964 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8014950 con 0x7f8ad00cd620 2023-11-08T16:39:41.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.625+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2606236964 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad00b54a0 con 0x7f8ad00cd620 2023-11-08T16:39:41.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.626+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2606236964 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad01554c0 con 0x7f8ad00cd620 2023-11-08T16:39:41.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.627+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2606236964 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad01556e0 con 0x7f8ad00cd620 2023-11-08T16:39:41.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.627+0000 7f8ac6ffd640 1 -- 172.21.15.81:0/2606236964 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8ac8014ab0 con 0x7f8ad00cd620 2023-11-08T16:39:41.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.628+0000 7f8ac6ffd640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ab006ae70 0x7f8ab006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:41.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.628+0000 7f8ac6ffd640 1 -- 172.21.15.81:0/2606236964 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f8ac808a180 con 0x7f8ad00cd620 2023-11-08T16:39:41.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.629+0000 7f8ace59c640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ab006ae70 0x7f8ab006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:41.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.629+0000 7f8ace59c640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ab006ae70 0x7f8ab006d330 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0036000 tx=0x7f8ac003d040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:41.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.633+0000 7f8ac6ffd640 1 -- 172.21.15.81:0/2606236964 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8ad01556e0 con 0x7f8ad00cd620 2023-11-08T16:39:41.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.837+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2606236964 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8ad01556e0 con 0x7f8ad00cd620 2023-11-08T16:39:41.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.838+0000 7f8ac6ffd640 1 -- 172.21.15.81:0/2606236964 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v106) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8ad01556e0 con 0x7f8ad00cd620 2023-11-08T16:39:41.839 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:39:41.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.840+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2606236964 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ab006ae70 msgr2=0x7f8ab006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:41.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.840+0000 7f8ad4fb6640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ab006ae70 0x7f8ab006d330 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0036000 tx=0x7f8ac003d040 comp rx=0 tx=0).stop 2023-11-08T16:39:41.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.840+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2606236964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ad00cd620 msgr2=0x7f8ad015d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:41.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.840+0000 7f8ad4fb6640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ad00cd620 0x7f8ad015d7d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f8ac8007e60 tx=0x7f8ac800e4b0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.840+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2606236964 shutdown_connections 2023-11-08T16:39:41.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.840+0000 7f8ad4fb6640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8ab006ae70 0x7f8ab006d330 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.840+0000 7f8ad4fb6640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8ad00cd620 0x7f8ad015d7d0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.840+0000 7f8ad4fb6640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8ad006e6d0 0x7f8ad0154d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.840+0000 7f8ad4fb6640 1 --2- 172.21.15.81:0/2606236964 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8ad006b2c0 0x7f8ad00b5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:41.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.840+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2606236964 >> 172.21.15.81:0/2606236964 conn(0x7f8ad006dac0 msgr2=0x7f8ad00c70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:41.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.841+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2606236964 shutdown_connections 2023-11-08T16:39:41.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:41.841+0000 7f8ad4fb6640 1 -- 172.21.15.81:0/2606236964 wait complete. 2023-11-08T16:39:41.859 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:39:41.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:41.860 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:39:41.949 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[119215]: fuse finished with error 0 and tester_r 0 2023-11-08T16:39:41.955 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:39:48.057 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:39:48.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:39:48.058 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:39:48.119 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:39:48.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.326+0000 7fa5aadc4640 1 Processor -- start 2023-11-08T16:39:48.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.326+0000 7fa5aadc4640 1 -- start start 2023-11-08T16:39:48.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.328+0000 7fa5aadc4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa5a40ad940 0x7fa5a40cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:48.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.328+0000 7fa5aadc4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a406cd70 con 0x7fa5a415ac20 2023-11-08T16:39:48.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.328+0000 7fa5aadc4640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a40ca500 con 0x7fa5a406f7c0 2023-11-08T16:39:48.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.328+0000 7fa5aadc4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a40ae130 con 0x7fa5a40ad940 2023-11-08T16:39:48.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.328+0000 7fa5a9dc2640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa5a415ac20 0x7fa5a4070f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60592/0 (socket says 172.21.15.81:60592) 2023-11-08T16:39:48.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.328+0000 7fa5a9dc2640 1 -- 172.21.15.81:0/2408822364 learned_addr learned my addr 172.21.15.81:0/2408822364 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:48.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.328+0000 7fa5aa5c3640 1 --2- 172.21.15.81:0/2408822364 >> v2:172.21.15.81:3300/0 conn(0x7fa5a40ad940 0x7fa5a40cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:48.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.329+0000 7fa5aa5c3640 1 -- 172.21.15.81:0/2408822364 >> v1:172.21.15.93:6789/0 conn(0x7fa5a406f7c0 legacy=0x7fa5a406b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:48.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.329+0000 7fa5aa5c3640 1 -- 172.21.15.81:0/2408822364 >> v1:172.21.15.81:6789/0 conn(0x7fa5a415ac20 legacy=0x7fa5a4070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:48.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.329+0000 7fa5aa5c3640 1 -- 172.21.15.81:0/2408822364 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a406d030 con 0x7fa5a40ad940 2023-11-08T16:39:48.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.329+0000 7fa5a8dc0640 1 -- 172.21.15.81:0/2408822364 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966457742 0 0) 0x7fa5a406cd70 con 0x7fa5a415ac20 2023-11-08T16:39:48.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.330+0000 7fa5aa5c3640 1 --2- 172.21.15.81:0/2408822364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5a40ad940 0x7fa5a40cbaa0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009d90 tx=0x7fa59c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94091e2c7515e1c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:48.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.330+0000 7fa5a8dc0640 1 -- 172.21.15.81:0/2408822364 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c02cbd0 con 0x7fa5a40ad940 2023-11-08T16:39:48.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.330+0000 7fa5a8dc0640 1 -- 172.21.15.81:0/2408822364 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59c02cd30 con 0x7fa5a40ad940 2023-11-08T16:39:48.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.330+0000 7fa5a8dc0640 1 -- 172.21.15.81:0/2408822364 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59c02cea0 con 0x7fa5a40ad940 2023-11-08T16:39:48.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.331+0000 7fa5aadc4640 1 -- 172.21.15.81:0/2408822364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5a40ad940 msgr2=0x7fa5a40cbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:48.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.331+0000 7fa5aadc4640 1 --2- 172.21.15.81:0/2408822364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5a40ad940 0x7fa5a40cbaa0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009d90 tx=0x7fa59c02be20 comp rx=0 tx=0).stop 2023-11-08T16:39:48.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.331+0000 7fa5aadc4640 1 -- 172.21.15.81:0/2408822364 shutdown_connections 2023-11-08T16:39:48.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.331+0000 7fa5aadc4640 1 --2- 172.21.15.81:0/2408822364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5a40ad940 0x7fa5a40cbaa0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:48.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.331+0000 7fa5aadc4640 1 -- 172.21.15.81:0/2408822364 >> 172.21.15.81:0/2408822364 conn(0x7fa5a40b04d0 msgr2=0x7fa5a40b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:48.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.331+0000 7fa5aadc4640 1 -- 172.21.15.81:0/2408822364 shutdown_connections 2023-11-08T16:39:48.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.331+0000 7fa5aadc4640 1 -- 172.21.15.81:0/2408822364 wait complete. 2023-11-08T16:39:48.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.333+0000 7fa5aadc4640 1 Processor -- start 2023-11-08T16:39:48.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.333+0000 7fa5aadc4640 1 -- start start 2023-11-08T16:39:48.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5aadc4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5a406f7c0 0x7fa5a41e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:48.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5aadc4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5a40ad940 0x7fa5a40a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:48.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5aadc4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5a415ac20 0x7fa5a40a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:48.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5aadc4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40ab360 con 0x7fa5a40ad940 2023-11-08T16:39:48.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5aadc4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40c4ec0 con 0x7fa5a415ac20 2023-11-08T16:39:48.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5aadc4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40c51a0 con 0x7fa5a406f7c0 2023-11-08T16:39:48.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5a95c1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5a40ad940 0x7fa5a40a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:48.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5a95c1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5a40ad940 0x7fa5a40a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54510/0 (socket says 172.21.15.81:54510) 2023-11-08T16:39:48.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5a95c1640 1 -- 172.21.15.81:0/3059071329 learned_addr learned my addr 172.21.15.81:0/3059071329 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:48.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5aa5c3640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5a415ac20 0x7fa5a40a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:48.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5a95c1640 1 -- 172.21.15.81:0/3059071329 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5a406f7c0 msgr2=0x7fa5a41e09f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:48.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5a9dc2640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5a406f7c0 0x7fa5a41e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:48.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5a95c1640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5a406f7c0 0x7fa5a41e09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:48.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5a95c1640 1 -- 172.21.15.81:0/3059071329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5a415ac20 msgr2=0x7fa5a40a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:48.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5a95c1640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5a415ac20 0x7fa5a40a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:48.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5a95c1640 1 -- 172.21.15.81:0/3059071329 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c009a70 con 0x7fa5a40ad940 2023-11-08T16:39:48.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.334+0000 7fa5aa5c3640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5a415ac20 0x7fa5a40a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:48.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.335+0000 7fa5a95c1640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5a40ad940 0x7fa5a40a84a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fa59400c430 tx=0x7fa59400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:48.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.335+0000 7fa59a7fc640 1 -- 172.21.15.81:0/3059071329 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594011790 con 0x7fa5a40ad940 2023-11-08T16:39:48.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.335+0000 7fa59a7fc640 1 -- 172.21.15.81:0/3059071329 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5940118f0 con 0x7fa5a40ad940 2023-11-08T16:39:48.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.335+0000 7fa59a7fc640 1 -- 172.21.15.81:0/3059071329 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa594011ce0 con 0x7fa5a40ad940 2023-11-08T16:39:48.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.335+0000 7fa5aadc4640 1 -- 172.21.15.81:0/3059071329 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a40ab600 con 0x7fa5a40ad940 2023-11-08T16:39:48.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.335+0000 7fa5aadc4640 1 -- 172.21.15.81:0/3059071329 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a40c4b20 con 0x7fa5a40ad940 2023-11-08T16:39:48.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.337+0000 7fa5aadc4640 1 -- 172.21.15.81:0/3059071329 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a40abb20 con 0x7fa5a40ad940 2023-11-08T16:39:48.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.338+0000 7fa59a7fc640 1 -- 172.21.15.81:0/3059071329 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa59401c460 con 0x7fa5a40ad940 2023-11-08T16:39:48.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.338+0000 7fa59a7fc640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa58806ae70 0x7fa58806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:48.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.338+0000 7fa59a7fc640 1 -- 172.21.15.81:0/3059071329 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6229+0+0 (secure 0 0 0) 0x7fa594089750 con 0x7fa5a40ad940 2023-11-08T16:39:48.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.339+0000 7fa5a9dc2640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa58806ae70 0x7fa58806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:48.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.339+0000 7fa5a9dc2640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa58806ae70 0x7fa58806d330 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0000f60 tx=0x7fa5a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:48.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.343+0000 7fa59a7fc640 1 -- 172.21.15.81:0/3059071329 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa5a40abb20 con 0x7fa5a40ad940 2023-11-08T16:39:48.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.573+0000 7fa5aadc4640 1 -- 172.21.15.81:0/3059071329 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa5a40abb20 con 0x7fa5a40ad940 2023-11-08T16:39:48.577 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:39:48.574+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:39:48.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.577+0000 7fa59a7fc640 1 -- 172.21.15.81:0/3059071329 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa5a40abb20 con 0x7fa5a40ad940 2023-11-08T16:39:48.580 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:39:48.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 -- 172.21.15.81:0/3059071329 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa58806ae70 msgr2=0x7fa58806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:48.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa58806ae70 0x7fa58806d330 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa5a0000f60 tx=0x7fa5a0009040 comp rx=0 tx=0).stop 2023-11-08T16:39:48.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 -- 172.21.15.81:0/3059071329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5a40ad940 msgr2=0x7fa5a40a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:48.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5a40ad940 0x7fa5a40a84a0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fa59400c430 tx=0x7fa59400c8f0 comp rx=0 tx=0).stop 2023-11-08T16:39:48.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 -- 172.21.15.81:0/3059071329 shutdown_connections 2023-11-08T16:39:48.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa58806ae70 0x7fa58806d330 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:48.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa5a415ac20 0x7fa5a40a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:48.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa5a40ad940 0x7fa5a40a84a0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:48.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 --2- 172.21.15.81:0/3059071329 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa5a406f7c0 0x7fa5a41e09f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:48.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 -- 172.21.15.81:0/3059071329 >> 172.21.15.81:0/3059071329 conn(0x7fa5a40b04d0 msgr2=0x7fa5a40b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:48.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 -- 172.21.15.81:0/3059071329 shutdown_connections 2023-11-08T16:39:48.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:48.581+0000 7fa5aadc4640 1 -- 172.21.15.81:0/3059071329 wait complete. 2023-11-08T16:39:48.604 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:39:48.612 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:39:48.693 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2023-11-08T16:39:48.771 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-08T16:39:48.772 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2023-11-08T16:39:48.772 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:39:48.772 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2023-11-08T16:39:48.772 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.773 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2023-11-08T16:39:48.773 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.773 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2023-11-08T16:39:48.773 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.773 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2023-11-08T16:39:48.774 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2023-11-08T16:39:48.774 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.774 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2023-11-08T16:39:48.774 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2023-11-08T16:39:48.774 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.774 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:39:48.775 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2023-11-08T16:39:48.775 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.775 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:39:48.775 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:39:48.775 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.776 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2023-11-08T16:39:48.776 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.776 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2023-11-08T16:39:48.776 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2023-11-08T16:39:48.776 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:39:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:39:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2023-11-08T16:39:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:39:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2023-11-08T16:39:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:39:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:39:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2023-11-08T16:39:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:39:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2023-11-08T16:39:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2023-11-08T16:39:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2023-11-08T16:39:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2023-11-08T16:39:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2023-11-08T16:39:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:39:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:39:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.781 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2023-11-08T16:39:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2023-11-08T16:39:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2023-11-08T16:39:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2023-11-08T16:39:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:39:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2023-11-08T16:39:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2023-11-08T16:39:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2023-11-08T16:39:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2023-11-08T16:39:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2023-11-08T16:39:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2023-11-08T16:39:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2023-11-08T16:39:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2023-11-08T16:39:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:39:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2023-11-08T16:39:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2023-11-08T16:39:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:39:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2023-11-08T16:39:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2023-11-08T16:39:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2023-11-08T16:39:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2023-11-08T16:39:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2023-11-08T16:39:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2023-11-08T16:39:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2023-11-08T16:39:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2023-11-08T16:39:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2023-11-08T16:39:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2023-11-08T16:39:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:39:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2023-11-08T16:39:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2023-11-08T16:39:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:39:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2023-11-08T16:39:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:39:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:39:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2023-11-08T16:39:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:39:48.790 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.790 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2023-11-08T16:39:48.790 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:39:48.790 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2023-11-08T16:39:48.790 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:39:48.791 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:39:48.791 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:39:48.791 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:39:48.791 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2023-11-08T16:39:48.791 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.793 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2023-11-08T16:39:48.793 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:39:48.794 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:39:48.794 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2023-11-08T16:39:48.794 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:39:48.794 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2023-11-08T16:39:48.794 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2023-11-08T16:39:48.795 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:39:48.795 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2023-11-08T16:39:48.795 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2023-11-08T16:39:48.795 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:39:48.795 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.796 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:39:48.796 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:39:48.796 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:39:48.796 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.796 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:39:48.797 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:39:48.797 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:39:48.797 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.797 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2023-11-08T16:39:48.797 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:39:48.798 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.798 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:39:48.798 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2023-11-08T16:39:48.798 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:48.798 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2023-11-08T16:39:48.799 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2023-11-08T16:39:48.799 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2023-11-08T16:39:48.799 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2023-11-08T16:39:48.799 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:39:48.799 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:39:48.800 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2023-11-08T16:39:48.800 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2023-11-08T16:39:48.800 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:39:48.800 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2023-11-08T16:39:48.800 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:39:48.800 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:39:48.801 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2023-11-08T16:39:48.801 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2023-11-08T16:39:48.801 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2023-11-08T16:39:48.801 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:39:48.801 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:39:48.802 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:39:48.802 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:39:48.802 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2023-11-08T16:39:48.802 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2023-11-08T16:39:48.802 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2023-11-08T16:39:48.803 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:39:48.803 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2023-11-08T16:39:48.803 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = False 2023-11-08T16:39:48.803 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2023-11-08T16:39:48.803 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = False 2023-11-08T16:39:48.804 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:39:48.804 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:39:48.804 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:39:48.805 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2023-11-08T16:39:48.805 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2023-11-08T16:39:48.805 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:39:48.805 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2023-11-08T16:39:48.805 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.806 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2023-11-08T16:39:48.806 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.806 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2023-11-08T16:39:48.806 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.806 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2023-11-08T16:39:48.807 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2023-11-08T16:39:48.807 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.807 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2023-11-08T16:39:48.807 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2023-11-08T16:39:48.807 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.808 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:39:48.808 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2023-11-08T16:39:48.808 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.808 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:39:48.808 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:39:48.809 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.809 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2023-11-08T16:39:48.809 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.809 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2023-11-08T16:39:48.809 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2023-11-08T16:39:48.810 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:39:48.810 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:39:48.810 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2023-11-08T16:39:48.810 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:39:48.810 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2023-11-08T16:39:48.811 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:39:48.811 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:39:48.811 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2023-11-08T16:39:48.811 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.811 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:39:48.811 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.812 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2023-11-08T16:39:48.812 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.812 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2023-11-08T16:39:48.812 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.812 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2023-11-08T16:39:48.813 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2023-11-08T16:39:48.813 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2023-11-08T16:39:48.813 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:39:48.813 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:39:48.813 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.814 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2023-11-08T16:39:48.814 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2023-11-08T16:39:48.814 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.814 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2023-11-08T16:39:48.814 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2023-11-08T16:39:48.815 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:39:48.815 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.815 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2023-11-08T16:39:48.815 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2023-11-08T16:39:48.815 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.816 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2023-11-08T16:39:48.816 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2023-11-08T16:39:48.816 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2023-11-08T16:39:48.816 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2023-11-08T16:39:48.816 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.816 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2023-11-08T16:39:48.817 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2023-11-08T16:39:48.817 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.817 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:39:48.817 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.817 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2023-11-08T16:39:48.817 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.818 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2023-11-08T16:39:48.818 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.818 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:39:48.818 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2023-11-08T16:39:48.818 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.819 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2023-11-08T16:39:48.819 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2023-11-08T16:39:48.819 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2023-11-08T16:39:48.819 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2023-11-08T16:39:48.820 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2023-11-08T16:39:48.820 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2023-11-08T16:39:48.820 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2023-11-08T16:39:48.820 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2023-11-08T16:39:48.820 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2023-11-08T16:39:48.821 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:39:48.821 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2023-11-08T16:39:48.821 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2023-11-08T16:39:48.821 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:39:48.821 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2023-11-08T16:39:48.821 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:39:48.822 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:39:48.822 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2023-11-08T16:39:48.822 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:39:48.822 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.822 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2023-11-08T16:39:48.823 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:39:48.823 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2023-11-08T16:39:48.823 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:39:48.824 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:39:48.824 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:39:48.824 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:39:48.824 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2023-11-08T16:39:48.824 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.824 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2023-11-08T16:39:48.825 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:39:48.825 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:39:48.825 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2023-11-08T16:39:48.825 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:39:48.826 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2023-11-08T16:39:48.826 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2023-11-08T16:39:48.826 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:39:48.826 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2023-11-08T16:39:48.826 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2023-11-08T16:39:48.827 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:39:48.827 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.827 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:39:48.827 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:39:48.827 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:39:48.828 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.828 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:39:48.828 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:39:48.828 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:39:48.828 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.828 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2023-11-08T16:39:48.829 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:39:48.829 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.829 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:39:48.829 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2023-11-08T16:39:48.829 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:48.830 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2023-11-08T16:39:48.830 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2023-11-08T16:39:48.830 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2023-11-08T16:39:48.830 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2023-11-08T16:39:48.830 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:39:48.831 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:39:48.831 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2023-11-08T16:39:48.831 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2023-11-08T16:39:48.831 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:39:48.831 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2023-11-08T16:39:48.832 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:39:48.832 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:39:48.832 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2023-11-08T16:39:48.832 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2023-11-08T16:39:48.832 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2023-11-08T16:39:48.833 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:39:48.833 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:39:48.833 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:39:48.833 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:39:48.833 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2023-11-08T16:39:48.834 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2023-11-08T16:39:48.834 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2023-11-08T16:39:48.834 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:39:48.834 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2023-11-08T16:39:48.834 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = False 2023-11-08T16:39:48.835 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2023-11-08T16:39:48.835 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = False 2023-11-08T16:39:48.835 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:39:48.835 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:39:48.835 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:39:48.838 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:39:48.849 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:39:49.008 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2023-11-08T16:39:49.008 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2023-11-08T16:39:49.009 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:39:49.009 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2023-11-08T16:39:49.009 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.009 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2023-11-08T16:39:49.009 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.009 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2023-11-08T16:39:49.010 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.010 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2023-11-08T16:39:49.010 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2023-11-08T16:39:49.010 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.010 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2023-11-08T16:39:49.011 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2023-11-08T16:39:49.011 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.011 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:39:49.011 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2023-11-08T16:39:49.011 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.012 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:39:49.012 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:39:49.012 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.012 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2023-11-08T16:39:49.012 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.013 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2023-11-08T16:39:49.013 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2023-11-08T16:39:49.013 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:39:49.013 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:39:49.013 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2023-11-08T16:39:49.013 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:39:49.014 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2023-11-08T16:39:49.014 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:39:49.014 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:39:49.014 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2023-11-08T16:39:49.015 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.015 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:39:49.015 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.015 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2023-11-08T16:39:49.015 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.015 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2023-11-08T16:39:49.016 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.016 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2023-11-08T16:39:49.016 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2023-11-08T16:39:49.016 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2023-11-08T16:39:49.016 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:39:49.017 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:39:49.017 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.017 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2023-11-08T16:39:49.017 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2023-11-08T16:39:49.017 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.018 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2023-11-08T16:39:49.018 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2023-11-08T16:39:49.018 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:39:49.018 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.018 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2023-11-08T16:39:49.019 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2023-11-08T16:39:49.019 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.019 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2023-11-08T16:39:49.019 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2023-11-08T16:39:49.019 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2023-11-08T16:39:49.019 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2023-11-08T16:39:49.020 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.020 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2023-11-08T16:39:49.020 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2023-11-08T16:39:49.020 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.020 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:39:49.021 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.021 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2023-11-08T16:39:49.021 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.021 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2023-11-08T16:39:49.021 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.022 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:39:49.022 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2023-11-08T16:39:49.022 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.022 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2023-11-08T16:39:49.022 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2023-11-08T16:39:49.023 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2023-11-08T16:39:49.023 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2023-11-08T16:39:49.023 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2023-11-08T16:39:49.023 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2023-11-08T16:39:49.023 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2023-11-08T16:39:49.023 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2023-11-08T16:39:49.024 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2023-11-08T16:39:49.024 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:39:49.024 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2023-11-08T16:39:49.024 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2023-11-08T16:39:49.024 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:39:49.024 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2023-11-08T16:39:49.025 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:39:49.025 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:39:49.025 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2023-11-08T16:39:49.025 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:39:49.026 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.026 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2023-11-08T16:39:49.026 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:39:49.026 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2023-11-08T16:39:49.026 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:39:49.027 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:39:49.027 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:39:49.027 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:39:49.027 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2023-11-08T16:39:49.027 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.027 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2023-11-08T16:39:49.028 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:39:49.028 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:39:49.028 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2023-11-08T16:39:49.028 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:39:49.028 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2023-11-08T16:39:49.029 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2023-11-08T16:39:49.029 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:39:49.029 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2023-11-08T16:39:49.029 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2023-11-08T16:39:49.029 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:39:49.030 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.030 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:39:49.030 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:39:49.030 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:39:49.030 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.030 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:39:49.031 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:39:49.031 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:39:49.031 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.031 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2023-11-08T16:39:49.031 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:39:49.032 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.032 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:39:49.032 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2023-11-08T16:39:49.032 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:49.032 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2023-11-08T16:39:49.032 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2023-11-08T16:39:49.033 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2023-11-08T16:39:49.033 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2023-11-08T16:39:49.033 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:39:49.033 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:39:49.033 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2023-11-08T16:39:49.034 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2023-11-08T16:39:49.034 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:39:49.034 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2023-11-08T16:39:49.034 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:39:49.034 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:39:49.035 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2023-11-08T16:39:49.035 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2023-11-08T16:39:49.035 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2023-11-08T16:39:49.035 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:39:49.035 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:39:49.036 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:39:49.036 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:39:49.036 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2023-11-08T16:39:49.036 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2023-11-08T16:39:49.036 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2023-11-08T16:39:49.036 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:39:49.037 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2023-11-08T16:39:49.037 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = False 2023-11-08T16:39:49.037 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2023-11-08T16:39:49.037 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = False 2023-11-08T16:39:49.037 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:39:49.038 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:39:49.038 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:39:49.038 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:39:49.048 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:39:49.050 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-08T16:39:49.050 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2023-11-08T16:39:49.050 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:39:49.050 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2023-11-08T16:39:49.051 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.051 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2023-11-08T16:39:49.051 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.051 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2023-11-08T16:39:49.051 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.052 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2023-11-08T16:39:49.052 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2023-11-08T16:39:49.052 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.052 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2023-11-08T16:39:49.052 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2023-11-08T16:39:49.053 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.053 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:39:49.053 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2023-11-08T16:39:49.053 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.053 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:39:49.054 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:39:49.054 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.054 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2023-11-08T16:39:49.054 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.054 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2023-11-08T16:39:49.055 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2023-11-08T16:39:49.055 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:39:49.055 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:39:49.055 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2023-11-08T16:39:49.055 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:39:49.055 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2023-11-08T16:39:49.056 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:39:49.056 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:39:49.056 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2023-11-08T16:39:49.056 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.056 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:39:49.057 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.057 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2023-11-08T16:39:49.057 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.057 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2023-11-08T16:39:49.057 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.058 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2023-11-08T16:39:49.058 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2023-11-08T16:39:49.058 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2023-11-08T16:39:49.058 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:39:49.058 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:39:49.059 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.059 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2023-11-08T16:39:49.059 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2023-11-08T16:39:49.059 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.059 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2023-11-08T16:39:49.059 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2023-11-08T16:39:49.060 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:39:49.060 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.060 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2023-11-08T16:39:49.060 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2023-11-08T16:39:49.060 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.061 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2023-11-08T16:39:49.061 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2023-11-08T16:39:49.061 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2023-11-08T16:39:49.061 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2023-11-08T16:39:49.061 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.062 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2023-11-08T16:39:49.062 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2023-11-08T16:39:49.062 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.062 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:39:49.062 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.063 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2023-11-08T16:39:49.063 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.063 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2023-11-08T16:39:49.063 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.063 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:39:49.064 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2023-11-08T16:39:49.064 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.064 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2023-11-08T16:39:49.064 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2023-11-08T16:39:49.064 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2023-11-08T16:39:49.065 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2023-11-08T16:39:49.065 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2023-11-08T16:39:49.065 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2023-11-08T16:39:49.065 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2023-11-08T16:39:49.065 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2023-11-08T16:39:49.065 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2023-11-08T16:39:49.066 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:39:49.066 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2023-11-08T16:39:49.066 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2023-11-08T16:39:49.066 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:39:49.066 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2023-11-08T16:39:49.067 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:39:49.067 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:39:49.067 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2023-11-08T16:39:49.067 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:39:49.067 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.067 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2023-11-08T16:39:49.068 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:39:49.068 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2023-11-08T16:39:49.068 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:39:49.068 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:39:49.068 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:39:49.069 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:39:49.069 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2023-11-08T16:39:49.069 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.069 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2023-11-08T16:39:49.069 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:39:49.070 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:39:49.070 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2023-11-08T16:39:49.070 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:39:49.070 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2023-11-08T16:39:49.071 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2023-11-08T16:39:49.071 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:39:49.071 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2023-11-08T16:39:49.071 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2023-11-08T16:39:49.071 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:39:49.071 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.072 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:39:49.072 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:39:49.072 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:39:49.072 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.072 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:39:49.073 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:39:49.073 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:39:49.073 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.073 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2023-11-08T16:39:49.073 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:39:49.074 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.074 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:39:49.074 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2023-11-08T16:39:49.074 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:39:49.074 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2023-11-08T16:39:49.075 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2023-11-08T16:39:49.075 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2023-11-08T16:39:49.075 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2023-11-08T16:39:49.075 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:39:49.075 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:39:49.075 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2023-11-08T16:39:49.076 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2023-11-08T16:39:49.076 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:39:49.076 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2023-11-08T16:39:49.076 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:39:49.076 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:39:49.077 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2023-11-08T16:39:49.077 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2023-11-08T16:39:49.077 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2023-11-08T16:39:49.077 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:39:49.077 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:39:49.078 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:39:49.078 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:39:49.078 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2023-11-08T16:39:49.078 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2023-11-08T16:39:49.078 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2023-11-08T16:39:49.079 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:39:49.079 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2023-11-08T16:39:49.079 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = False 2023-11-08T16:39:49.079 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2023-11-08T16:39:49.079 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = False 2023-11-08T16:39:49.080 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:39:49.080 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:39:49.080 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:39:49.092 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T16:39:49.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.223+0000 7f4bbbf55f40 1 Processor -- start 2023-11-08T16:39:49.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.223+0000 7f4bbbf55f40 1 -- start start 2023-11-08T16:39:49.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.224+0000 7f4bbbf55f40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1120 0x55c418384470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.224+0000 7f4bbbf55f40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c4182e1500 0x55c418385f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.224+0000 7f4bbbf55f40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c4182e1a90 0x55c418386470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.224+0000 7f4bbbf55f40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c418261340 con 0x55c4182e1120 2023-11-08T16:39:49.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.224+0000 7f4bbbf55f40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4181fba70 con 0x55c4182e1500 2023-11-08T16:39:49.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.224+0000 7f4bbbf55f40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4181fb710 con 0x55c4182e1a90 2023-11-08T16:39:49.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.224+0000 7f4bb9a57640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1120 0x55c418384470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.224+0000 7f4bb9a57640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1120 0x55c418384470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54520/0 (socket says 172.21.15.81:54520) 2023-11-08T16:39:49.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.224+0000 7f4bb9a57640 1 -- 172.21.15.81:0/2114920691 learned_addr learned my addr 172.21.15.81:0/2114920691 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:49.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.225+0000 7f4bb9a57640 1 -- 172.21.15.81:0/2114920691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c4182e1a90 msgr2=0x55c418386470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:49.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.225+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/2114920691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c4182e1a90 0x55c418386470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.225+0000 7f4bb9a57640 1 -- 172.21.15.81:0/2114920691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c4182e1500 msgr2=0x55c418385f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:49.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.225+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/2114920691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c4182e1500 0x55c418385f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.225+0000 7f4bb9a57640 1 -- 172.21.15.81:0/2114920691 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac00a160 con 0x55c4182e1120 2023-11-08T16:39:49.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.225+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/2114920691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1120 0x55c418384470 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4bac000f80 tx=0x7f4bac04c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6caed030df3c425 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.225+0000 7f4bb3fff640 1 -- 172.21.15.81:0/2114920691 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac005220 con 0x55c4182e1120 2023-11-08T16:39:49.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.225+0000 7f4bb3fff640 1 -- 172.21.15.81:0/2114920691 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac0053a0 con 0x55c4182e1120 2023-11-08T16:39:49.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.225+0000 7f4bb3fff640 1 -- 172.21.15.81:0/2114920691 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac052900 con 0x55c4182e1120 2023-11-08T16:39:49.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.226+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/2114920691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1120 msgr2=0x55c418384470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.226+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/2114920691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1120 0x55c418384470 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4bac000f80 tx=0x7f4bac04c070 comp rx=0 tx=0).stop 2023-11-08T16:39:49.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.226+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/2114920691 shutdown_connections 2023-11-08T16:39:49.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.226+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/2114920691 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c4182e1a90 0x55c418386470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.226+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/2114920691 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c4182e1500 0x55c418385f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.226+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/2114920691 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1120 0x55c418384470 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.226+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/2114920691 >> 172.21.15.81:0/2114920691 conn(0x55c4182faa50 msgr2=0x55c4182fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:49.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.226+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/2114920691 shutdown_connections 2023-11-08T16:39:49.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.226+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/2114920691 wait complete. 2023-11-08T16:39:49.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.227+0000 7f4bbbf55f40 1 Processor -- start 2023-11-08T16:39:49.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.227+0000 7f4bbbf55f40 1 -- start start 2023-11-08T16:39:49.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.228+0000 7f4bbbf55f40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c4182e1120 0x55c4182f6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.228+0000 7f4bbbf55f40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1500 0x55c418386090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.228+0000 7f4bbbf55f40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c4182e1a90 0x55c4182d95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.228+0000 7f4bbbf55f40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4181ef520 con 0x55c4182e1500 2023-11-08T16:39:49.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.228+0000 7f4bbbf55f40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4182dcfc0 con 0x55c4182e1120 2023-11-08T16:39:49.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.228+0000 7f4bbbf55f40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c4182dd280 con 0x55c4182e1a90 2023-11-08T16:39:49.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.228+0000 7f4bb9256640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1500 0x55c418386090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.228+0000 7f4bb9256640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1500 0x55c418386090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54534/0 (socket says 172.21.15.81:54534) 2023-11-08T16:39:49.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.228+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 learned_addr learned my addr 172.21.15.81:0/3926327607 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:49.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.229+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c4182e1a90 msgr2=0x55c4182d95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.229+0000 7f4bb9256640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c4182e1a90 0x55c4182d95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.229+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c4182e1120 msgr2=0x55c4182f6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.229+0000 7f4bb9256640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c4182e1120 0x55c4182f6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.229+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c418221670 con 0x55c4182e1500 2023-11-08T16:39:49.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.229+0000 7f4bb9256640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1500 0x55c418386090 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40030a0 tx=0x7f4ba40086c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.229+0000 7f4bb2ffd640 1 -- 172.21.15.81:0/3926327607 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba400f040 con 0x55c4182e1500 2023-11-08T16:39:49.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.229+0000 7f4bb2ffd640 1 -- 172.21.15.81:0/3926327607 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba4013620 con 0x55c4182e1500 2023-11-08T16:39:49.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.229+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c418207680 con 0x55c4182e1500 2023-11-08T16:39:49.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.229+0000 7f4bb2ffd640 1 -- 172.21.15.81:0/3926327607 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba40139b0 con 0x55c4182e1500 2023-11-08T16:39:49.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.229+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4181fa830 con 0x55c4182e1500 2023-11-08T16:39:49.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.231+0000 7f4bb2ffd640 1 -- 172.21.15.81:0/3926327607 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4ba4013b10 con 0x55c4182e1500 2023-11-08T16:39:49.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.232+0000 7f4bb2ffd640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4b9c069820 0x7f4b9c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.232+0000 7f4bb2ffd640 1 -- 172.21.15.81:0/3926327607 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f4ba4088590 con 0x55c4182e1500 2023-11-08T16:39:49.241 INFO:teuthology.orchestra.run.smithi081.stdout:Warning: using slow linear search 2023-11-08T16:39:49.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.233+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4b9c069820 0x7f4b9c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.233+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55c4182fae30 0x55c41842ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.233+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:1 13.0 13:00000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x55c4182fae30 2023-11-08T16:39:49.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.233+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4b9c069820 0x7f4b9c06bce0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x55c4182f85a0 tx=0x7f4bac005de0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.233+0000 7f4bba258640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55c4182fae30 0x55c41842ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.234+0000 7f4bba258640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55c4182fae30 0x55c41842ea20 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.236+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(1 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bb400d2a0 con 0x55c4182fae30 2023-11-08T16:39:49.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.236+0000 7f4bba258640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f4bb400c230 0x7f4bb400c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.236+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:2 13.10 13:08000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb400c230 2023-11-08T16:39:49.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.236+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f4bb400c230 0x7f4bb400c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.237+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f4bb400c230 0x7f4bb400c6b0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.237+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(2 [pgnls start_epoch 109 out=177b] v106'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb400c230 2023-11-08T16:39:49.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.237+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f4bac052bd0 0x7f4bac004600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.237+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:3 13.8 13:10000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bac052bd0 2023-11-08T16:39:49.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.237+0000 7f4bb9256640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f4bac052bd0 0x7f4bac004600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.238+0000 7f4bb9256640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f4bac052bd0 0x7f4bac004600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.238+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(3 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ba4019020 con 0x7f4bac052bd0 2023-11-08T16:39:49.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.238+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:4 13.18 13:18000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x55c4182fae30 2023-11-08T16:39:49.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.239+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(4 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bb400d2a0 con 0x55c4182fae30 2023-11-08T16:39:49.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.239+0000 7f4bba258640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f4bb400cbf0 0x7f4bb40118a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.239+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:5 13.4 13:20000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.240+0000 7f4bba258640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f4bb400cbf0 0x7f4bb40118a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.241+0000 7f4bba258640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f4bb400cbf0 0x7f4bb40118a0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.242+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(5 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.242+0000 7f4bba258640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7f4bb4019460 0x7f4bb40198a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.242+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:6 13.14 13:28000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb4019460 2023-11-08T16:39:49.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.242+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7f4bb4019460 0x7f4bb40198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.243+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7f4bb4019460 0x7f4bb40198a0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.243+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(6 [pgnls start_epoch 109 out=149b] v107'42 uv42 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb4019460 2023-11-08T16:39:49.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.243+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:7 13.c 13:30000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb400c230 2023-11-08T16:39:49.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.244+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(7 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb400c230 2023-11-08T16:39:49.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.244+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:8 13.1c 13:38000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.245+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(8 [pgnls start_epoch 109 out=73b] v102'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.245+0000 7f4bba258640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f4bb4015060 0x7f4bb4015950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.245+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:9 13.2 13:40000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb4015060 2023-11-08T16:39:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.245+0000 7f4bb9256640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f4bb4015060 0x7f4bb4015950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.246+0000 7f4bb9256640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f4bb4015060 0x7f4bb4015950 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.253+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(9 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ba4019020 con 0x7f4bb4015060 2023-11-08T16:39:49.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.253+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:10 13.12 13:48000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb4019460 2023-11-08T16:39:49.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.254+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(10 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb4019460 2023-11-08T16:39:49.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.254+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:11 13.a 13:50000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bac052bd0 2023-11-08T16:39:49.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.254+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(11 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ba4019020 con 0x7f4bac052bd0 2023-11-08T16:39:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.254+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:12 13.1a 13:58000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb400c230 2023-11-08T16:39:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.255+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(12 [pgnls start_epoch 109 out=49b] v102'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb400c230 2023-11-08T16:39:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.255+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f4bac05b950 0x7f4bac066460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.255+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:13 13.6 13:60000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bac05b950 2023-11-08T16:39:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.256+0000 7f4bba258640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f4bac05b950 0x7f4bac066460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.257+0000 7f4bba258640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f4bac05b950 0x7f4bac066460 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.265+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(13 [pgnls start_epoch 109 out=77b] v107'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bac05b950 2023-11-08T16:39:49.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.265+0000 7f4bba258640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7f4bb402c940 0x7f4bb402b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.265+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:14 13.16 13:68000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb402c940 2023-11-08T16:39:49.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.265+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7f4bb402c940 0x7f4bb402b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.266+0000 7f4bb9a57640 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7f4bb402c940 0x7f4bb402b3c0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.267+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(14 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb402c940 2023-11-08T16:39:49.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.267+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:15 13.e 13:70000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x55c4182fae30 2023-11-08T16:39:49.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.268+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(15 [pgnls start_epoch 109 out=73b] v102'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f4bb400d2a0 con 0x55c4182fae30 2023-11-08T16:39:49.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.268+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:16 13.1e 13:78000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bac05b950 2023-11-08T16:39:49.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.269+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(16 [pgnls start_epoch 109 out=49b] v102'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bac05b950 2023-11-08T16:39:49.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.269+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:17 13.1 13:80000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.271+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(17 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.271+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:18 13.11 13:88000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb400c230 2023-11-08T16:39:49.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.271+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(18 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb400c230 2023-11-08T16:39:49.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.271+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:19 13.9 13:90000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb4015060 2023-11-08T16:39:49.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.272+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(19 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ba4019020 con 0x7f4bb4015060 2023-11-08T16:39:49.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.272+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:20 13.19 13:98000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb402c940 2023-11-08T16:39:49.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.273+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(20 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb402c940 2023-11-08T16:39:49.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.273+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:21 13.5 13:a0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bac05b950 2023-11-08T16:39:49.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.274+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(21 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bac05b950 2023-11-08T16:39:49.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.274+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:22 13.15 13:a8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bac05b950 2023-11-08T16:39:49.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.275+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(22 [pgnls start_epoch 109 out=97b] v102'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bac05b950 2023-11-08T16:39:49.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.275+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:23 13.d 13:b0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bac05b950 2023-11-08T16:39:49.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.276+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(23 [pgnls start_epoch 109 out=74b] v102'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bac05b950 2023-11-08T16:39:49.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.276+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:24 13.1d 13:b8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x55c4182fae30 2023-11-08T16:39:49.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.278+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(24 [pgnls start_epoch 109 out=73b] v102'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f4bb400d2a0 con 0x55c4182fae30 2023-11-08T16:39:49.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.278+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:25 13.3 13:c0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb4019460 2023-11-08T16:39:49.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.278+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(25 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb4019460 2023-11-08T16:39:49.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.278+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:26 13.13 13:c8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x55c4182fae30 2023-11-08T16:39:49.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.280+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(26 [pgnls start_epoch 109 out=98b] v106'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7f4bb400d2a0 con 0x55c4182fae30 2023-11-08T16:39:49.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.280+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:27 13.b 13:d0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb4015060 2023-11-08T16:39:49.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.281+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(27 [pgnls start_epoch 109 out=76b] v106'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f4ba4019020 con 0x7f4bb4015060 2023-11-08T16:39:49.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.281+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:28 13.1b 13:d8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bac05b950 2023-11-08T16:39:49.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.282+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(28 [pgnls start_epoch 109 out=49b] v102'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bac05b950 2023-11-08T16:39:49.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.283+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:29 13.7 13:e0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x55c4182fae30 2023-11-08T16:39:49.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.284+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(29 [pgnls start_epoch 109 out=73b] v102'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f4bb400d2a0 con 0x55c4182fae30 2023-11-08T16:39:49.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.284+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:30 13.17 13:e8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb4015060 2023-11-08T16:39:49.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.285+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(30 [pgnls start_epoch 109 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f4ba4019020 con 0x7f4bb4015060 2023-11-08T16:39:49.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.285+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:31 13.f 13:f0000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.286+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(31 [pgnls start_epoch 109 out=73b] v102'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.286+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:32 13.1f 13:f8000000:%01:::head [pgnls start_epoch 109 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e109) v8 -- 0x55c4183865d0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(32 [pgnls start_epoch 109 out=101b] v106'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:33 13.10 13:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c4182f2f70 con 0x7f4bb400c230 2023-11-08T16:39:49.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:34 13.10 13:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c4182dbb50 con 0x7f4bb400c230 2023-11-08T16:39:49.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:35 13.10 13:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c41842f400 con 0x7f4bb400c230 2023-11-08T16:39:49.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:36 13.10 13:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c41842fc00 con 0x7f4bb400c230 2023-11-08T16:39:49.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:37 13.10 13:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418430400 con 0x7f4bb400c230 2023-11-08T16:39:49.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:38 13.14 13:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418430c00 con 0x7f4bb4019460 2023-11-08T16:39:49.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:39 13.14 13:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418431400 con 0x7f4bb4019460 2023-11-08T16:39:49.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:40 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418431c00 con 0x7f4bb4019460 2023-11-08T16:39:49.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:41 13.14 13:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418432400 con 0x7f4bb4019460 2023-11-08T16:39:49.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:42 13.1c 13:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418432c00 con 0x7f4bb400cbf0 2023-11-08T16:39:49.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:43 13.6 13:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418433400 con 0x7f4bac05b950 2023-11-08T16:39:49.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.287+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:44 13.e 13:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418433c60 con 0x55c4182fae30 2023-11-08T16:39:49.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.288+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:45 13.15 13:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c4184346e0 con 0x7f4bac05b950 2023-11-08T16:39:49.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.288+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:46 13.15 13:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418434f60 con 0x7f4bac05b950 2023-11-08T16:39:49.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.288+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:47 13.d 13:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c4184358f0 con 0x7f4bac05b950 2023-11-08T16:39:49.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.288+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:48 13.1d 13:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418436390 con 0x55c4182fae30 2023-11-08T16:39:49.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.294+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(48 605.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x55c4182fae30 2023-11-08T16:39:49.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.294+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:49 13.13 13:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418433c60 con 0x55c4182fae30 2023-11-08T16:39:49.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.294+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v109'8 uv8 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bac05b950 2023-11-08T16:39:49.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.294+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(33 601.00000000 [delete] v109'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb400c230 2023-11-08T16:39:49.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.294+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:50 13.13 13:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418433c60 con 0x55c4182fae30 2023-11-08T16:39:49.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.295+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:51 13.b 13:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418436350 con 0x7f4bb4015060 2023-11-08T16:39:49.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.295+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(42 607.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.295+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:52 13.7 13:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418433c60 con 0x55c4182fae30 2023-11-08T16:39:49.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.295+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(38 1.00000000.inode [delete] v109'43 uv43 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb4019460 2023-11-08T16:39:49.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.295+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(39 200.00000000 [delete] v109'44 uv44 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb4019460 2023-11-08T16:39:49.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.295+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:53 13.f 13:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418433c60 con 0x7f4bb400cbf0 2023-11-08T16:39:49.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.295+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:54 13.1f 13:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418431070 con 0x7f4bb400cbf0 2023-11-08T16:39:49.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.296+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(34 602.00000000 [delete] v109'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb400c230 2023-11-08T16:39:49.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.296+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:55 13.1f 13:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e109) v8 -- 0x55c418433c60 con 0x7f4bb400cbf0 2023-11-08T16:39:49.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.296+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(35 10000000000.00000000 [delete] v109'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb400c230 2023-11-08T16:39:49.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.296+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(36 600.00000000 [delete] v109'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb400c230 2023-11-08T16:39:49.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.297+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(40 200.00000001 [delete] v109'45 uv45 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb4019460 2023-11-08T16:39:49.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.297+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(44 608.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x55c4182fae30 2023-11-08T16:39:49.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.297+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(45 604.00000000 [delete] v109'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bac05b950 2023-11-08T16:39:49.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.297+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(46 500.00000000 [delete] v109'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bac05b950 2023-11-08T16:39:49.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.297+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(47 mds_snaptable [delete] v109'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bac05b950 2023-11-08T16:39:49.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.297+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(37 603.00000000 [delete] v109'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb400c230 2023-11-08T16:39:49.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.299+0000 7f4bb9a57640 1 -- 172.21.15.81:0/3926327607 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(41 606.00000000 [delete] v109'46 uv46 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bac0539b0 con 0x7f4bb4019460 2023-11-08T16:39:49.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.300+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.7 v2:172.21.15.93:6818/433691658 9 ==== osd_op_reply(49 mds0_inotable [delete] v109'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x55c4182fae30 2023-11-08T16:39:49.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.300+0000 7f4bb9256640 1 -- 172.21.15.81:0/3926327607 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(51 mds0_sessionmap [delete] v109'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f4ba4019020 con 0x7f4bb4015060 2023-11-08T16:39:49.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.300+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(54 100.00000000.inode [delete] v109'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.301+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(55 1.00000000 [delete] v109'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.301+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.7 v2:172.21.15.93:6818/433691658 10 ==== osd_op_reply(50 100.00000000 [delete] v109'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x55c4182fae30 2023-11-08T16:39:49.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.301+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.7 v2:172.21.15.93:6818/433691658 11 ==== osd_op_reply(52 609.00000000 [delete] v109'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x55c4182fae30 2023-11-08T16:39:49.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.301+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(53 400.00000000 [delete] v109'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f4bb400d2a0 con 0x7f4bb400cbf0 2023-11-08T16:39:49.305 INFO:teuthology.orchestra.run.smithi081.stdout:Removed 23 objects 2023-11-08T16:39:49.306 INFO:teuthology.orchestra.run.smithi081.stdout:successfully purged pool cephfs_metadata 2023-11-08T16:39:49.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.301+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7f4bb4019460 msgr2=0x7f4bb40198a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.301+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7f4bb4019460 0x7f4bb40198a0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.301+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f4bb400c230 msgr2=0x7f4bb400c6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.301+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f4bb400c230 0x7f4bb400c6b0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f4bac052bd0 msgr2=0x7f4bac004600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f4bac052bd0 0x7f4bac004600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7f4bb402c940 msgr2=0x7f4bb402b3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7f4bb402c940 0x7f4bb402b3c0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 reap_dead start 2023-11-08T16:39:49.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f4bac05b950 msgr2=0x7f4bac066460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f4bac05b950 0x7f4bac066460 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f4bb4015060 msgr2=0x7f4bb4015950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7f4bb4015060 0x7f4bb4015950 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f4bb400cbf0 msgr2=0x7f4bb40118a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f4bb400cbf0 0x7f4bb40118a0 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55c4182fae30 msgr2=0x55c41842ea20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55c4182fae30 0x55c41842ea20 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bba258640 1 -- 172.21.15.81:0/3926327607 reap_dead start 2023-11-08T16:39:49.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4b9c069820 msgr2=0x7f4b9c06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4b9c069820 0x7f4b9c06bce0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x55c4182f85a0 tx=0x7f4bac005de0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1500 msgr2=0x55c418386090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1500 0x55c418386090 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40030a0 tx=0x7f4ba40086c0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 shutdown_connections 2023-11-08T16:39:49.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f4b9c069820 0x7f4b9c06bce0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 --2- 172.21.15.81:0/3926327607 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c4182e1500 0x55c418386090 secure :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4ba40030a0 tx=0x7f4ba40086c0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.302+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 >> 172.21.15.81:0/3926327607 conn(0x55c4182faa50 msgr2=0x55c418388940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:49.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.303+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 shutdown_connections 2023-11-08T16:39:49.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.303+0000 7f4bbbf55f40 1 -- 172.21.15.81:0/3926327607 wait complete. 2023-11-08T16:39:49.313 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs reset cephfs --yes-i-really-mean-it 2023-11-08T16:39:49.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.525+0000 7f56de1d0640 1 Processor -- start 2023-11-08T16:39:49.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.525+0000 7f56de1d0640 1 -- start start 2023-11-08T16:39:49.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.527+0000 7f56de1d0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f56d8071890 0x7f56d815e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.527+0000 7f56de1d0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56d815afd0 0x7f56d815ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.527+0000 7f56de1d0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d80b09f0 con 0x7f56d80714b0 2023-11-08T16:39:49.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.527+0000 7f56de1d0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d806b250 con 0x7f56d8071890 2023-11-08T16:39:49.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.527+0000 7f56de1d0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d80ca360 con 0x7f56d815afd0 2023-11-08T16:39:49.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.527+0000 7f56cffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f56d8071890 0x7f56d815e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.527+0000 7f56d77fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f56d80714b0 0x7f56d815a8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60596/0 (socket says 172.21.15.81:60596) 2023-11-08T16:39:49.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.527+0000 7f56d77fe640 1 -- 172.21.15.81:0/2676275174 learned_addr learned my addr 172.21.15.81:0/2676275174 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:49.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.528+0000 7f56d7fff640 1 --2- 172.21.15.81:0/2676275174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56d815afd0 0x7f56d815ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.528+0000 7f56cffff640 1 -- 172.21.15.81:0/2676275174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56d815afd0 msgr2=0x7f56d815ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:49.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.528+0000 7f56cffff640 1 --2- 172.21.15.81:0/2676275174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56d815afd0 0x7f56d815ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.528+0000 7f56cffff640 1 -- 172.21.15.81:0/2676275174 >> v1:172.21.15.81:6789/0 conn(0x7f56d80714b0 legacy=0x7f56d815a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.528+0000 7f56cffff640 1 -- 172.21.15.81:0/2676275174 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d80ca640 con 0x7f56d8071890 2023-11-08T16:39:49.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.528+0000 7f56d6ffd640 1 -- 172.21.15.81:0/2676275174 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626820911 0 0) 0x7f56d80b09f0 con 0x7f56d80714b0 2023-11-08T16:39:49.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.529+0000 7f56cffff640 1 --2- 172.21.15.81:0/2676275174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d8071890 0x7f56d815e9d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f56c00097a0 tx=0x7f56c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dc9b13991ffb20a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.530+0000 7f56d6ffd640 1 -- 172.21.15.81:0/2676275174 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0031830 con 0x7f56d8071890 2023-11-08T16:39:49.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.530+0000 7f56d6ffd640 1 -- 172.21.15.81:0/2676275174 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c0031990 con 0x7f56d8071890 2023-11-08T16:39:49.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.530+0000 7f56d6ffd640 1 -- 172.21.15.81:0/2676275174 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c0031d20 con 0x7f56d8071890 2023-11-08T16:39:49.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.530+0000 7f56de1d0640 1 -- 172.21.15.81:0/2676275174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d8071890 msgr2=0x7f56d815e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.530+0000 7f56de1d0640 1 --2- 172.21.15.81:0/2676275174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d8071890 0x7f56d815e9d0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f56c00097a0 tx=0x7f56c002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.530+0000 7f56de1d0640 1 -- 172.21.15.81:0/2676275174 shutdown_connections 2023-11-08T16:39:49.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.530+0000 7f56de1d0640 1 --2- 172.21.15.81:0/2676275174 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56d815afd0 0x7f56d815ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.530+0000 7f56de1d0640 1 --2- 172.21.15.81:0/2676275174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d8071890 0x7f56d815e9d0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.530+0000 7f56de1d0640 1 -- 172.21.15.81:0/2676275174 >> 172.21.15.81:0/2676275174 conn(0x7f56d80cb4e0 msgr2=0x7f56d80cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:49.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.530+0000 7f56de1d0640 1 -- 172.21.15.81:0/2676275174 shutdown_connections 2023-11-08T16:39:49.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.530+0000 7f56de1d0640 1 -- 172.21.15.81:0/2676275174 wait complete. 2023-11-08T16:39:49.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.532+0000 7f56de1d0640 1 Processor -- start 2023-11-08T16:39:49.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.532+0000 7f56de1d0640 1 -- start start 2023-11-08T16:39:49.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.533+0000 7f56de1d0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d80714b0 0x7f56d80c9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.533+0000 7f56de1d0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56d8071890 0x7f56d80c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.533+0000 7f56de1d0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56d815afd0 0x7f56d80c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.533+0000 7f56de1d0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8154040 con 0x7f56d80714b0 2023-11-08T16:39:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.533+0000 7f56de1d0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8154320 con 0x7f56d815afd0 2023-11-08T16:39:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.533+0000 7f56de1d0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8154600 con 0x7f56d8071890 2023-11-08T16:39:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.533+0000 7f56d77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d80714b0 0x7f56d80c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.534+0000 7f56d77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d80714b0 0x7f56d80c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54546/0 (socket says 172.21.15.81:54546) 2023-11-08T16:39:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.534+0000 7f56d77fe640 1 -- 172.21.15.81:0/1859769475 learned_addr learned my addr 172.21.15.81:0/1859769475 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:49.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.534+0000 7f56cffff640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56d8071890 0x7f56d80c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.534+0000 7f56d77fe640 1 -- 172.21.15.81:0/1859769475 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56d8071890 msgr2=0x7f56d80c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.534+0000 7f56d77fe640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56d8071890 0x7f56d80c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.534+0000 7f56d77fe640 1 -- 172.21.15.81:0/1859769475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56d815afd0 msgr2=0x7f56d80c6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:49.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.534+0000 7f56d7fff640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56d815afd0 0x7f56d80c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.534+0000 7f56d77fe640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56d815afd0 0x7f56d80c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:49.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.534+0000 7f56d77fe640 1 -- 172.21.15.81:0/1859769475 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c0009400 con 0x7f56d80714b0 2023-11-08T16:39:49.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.534+0000 7f56d77fe640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d80714b0 0x7f56d80c9800 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f56c800c4d0 tx=0x7f56c800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.534+0000 7f56d4ff9640 1 -- 172.21.15.81:0/1859769475 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c80074e0 con 0x7f56d80714b0 2023-11-08T16:39:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.535+0000 7f56d4ff9640 1 -- 172.21.15.81:0/1859769475 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c8007660 con 0x7f56d80714b0 2023-11-08T16:39:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.535+0000 7f56de1d0640 1 -- 172.21.15.81:0/1859769475 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d81548e0 con 0x7f56d80714b0 2023-11-08T16:39:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.535+0000 7f56d4ff9640 1 -- 172.21.15.81:0/1859769475 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c8004a20 con 0x7f56d80714b0 2023-11-08T16:39:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.535+0000 7f56de1d0640 1 -- 172.21.15.81:0/1859769475 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d8153cd0 con 0x7f56d80714b0 2023-11-08T16:39:49.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.536+0000 7f56de1d0640 1 -- 172.21.15.81:0/1859769475 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d80b0e60 con 0x7f56d80714b0 2023-11-08T16:39:49.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.537+0000 7f56d4ff9640 1 -- 172.21.15.81:0/1859769475 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f56c8004b80 con 0x7f56d80714b0 2023-11-08T16:39:49.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.538+0000 7f56d4ff9640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56b806ae70 0x7f56b806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:49.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.538+0000 7f56d4ff9640 1 -- 172.21.15.81:0/1859769475 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f56c8088e80 con 0x7f56d80714b0 2023-11-08T16:39:49.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.538+0000 7f56cffff640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56b806ae70 0x7f56b806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:49.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.539+0000 7f56cffff640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56b806ae70 0x7f56b806d330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f56c0036000 tx=0x7f56c003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:49.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.543+0000 7f56d4ff9640 1 -- 172.21.15.81:0/1859769475 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f56d80b0e60 con 0x7f56d80714b0 2023-11-08T16:39:49.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:49.777+0000 7f56de1d0640 1 -- 172.21.15.81:0/1859769475 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f56d80b0e60 con 0x7f56d80714b0 2023-11-08T16:39:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.591+0000 7f56d4ff9640 1 -- 172.21.15.81:0/1859769475 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs reset", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v90) v1 ==== 107+0+0 (secure 0 0 0) 0x7f56d80b0e60 con 0x7f56d80714b0 2023-11-08T16:39:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.594+0000 7f56de1d0640 1 -- 172.21.15.81:0/1859769475 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56b806ae70 msgr2=0x7f56b806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.594+0000 7f56de1d0640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56b806ae70 0x7f56b806d330 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f56c0036000 tx=0x7f56c003c040 comp rx=0 tx=0).stop 2023-11-08T16:39:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.594+0000 7f56de1d0640 1 -- 172.21.15.81:0/1859769475 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d80714b0 msgr2=0x7f56d80c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.594+0000 7f56de1d0640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d80714b0 0x7f56d80c9800 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f56c800c4d0 tx=0x7f56c800c990 comp rx=0 tx=0).stop 2023-11-08T16:39:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.594+0000 7f56de1d0640 1 -- 172.21.15.81:0/1859769475 shutdown_connections 2023-11-08T16:39:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.594+0000 7f56de1d0640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f56b806ae70 0x7f56b806d330 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.594+0000 7f56de1d0640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56d815afd0 0x7f56d80c6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.594+0000 7f56de1d0640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f56d8071890 0x7f56d80c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:50.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.594+0000 7f56de1d0640 1 --2- 172.21.15.81:0/1859769475 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56d80714b0 0x7f56d80c9800 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:50.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.594+0000 7f56de1d0640 1 -- 172.21.15.81:0/1859769475 >> 172.21.15.81:0/1859769475 conn(0x7f56d80cb4e0 msgr2=0x7f56d81586f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:50.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.595+0000 7f56de1d0640 1 -- 172.21.15.81:0/1859769475 shutdown_connections 2023-11-08T16:39:50.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.595+0000 7f56de1d0640 1 -- 172.21.15.81:0/1859769475 wait complete. 2023-11-08T16:39:50.607 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:39:50.605+0000 7fdbdf475640 -1 mds.0.inotable: load_2 could not read table: -2 2023-11-08T16:39:50.607 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:39:50.605+0000 7fdbdf475640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-11-08T16:39:50.608 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:39:50.605+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:39:50.608 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:39:50.605+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 mds daemon damaged (MDS_DAMAGE) 2023-11-08T16:39:50.619 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-08T16:39:50.624 INFO:tasks.ceph.mds.c.smithi081.stderr: -29> 2023-11-08T16:39:50.605+0000 7fdbdf475640 -1 mds.0.inotable: load_2 could not read table: -2 2023-11-08T16:39:50.625 INFO:tasks.ceph.mds.c.smithi081.stderr: -28> 2023-11-08T16:39:50.605+0000 7fdbdf475640 -1 log_channel(cluster) log [ERR] : error reading table object 'mds0_inotable' -2 ((2) No such file or directory) 2023-11-08T16:39:50.665 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:39:50.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.805+0000 7f152f9a9640 1 Processor -- start 2023-11-08T16:39:50.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.806+0000 7f152f9a9640 1 -- start start 2023-11-08T16:39:50.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.807+0000 7f152f9a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15280ce0d0 0x7f15280caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:50.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.807+0000 7f152f9a9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f152815bfc0 0x7f152815eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:50.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.807+0000 7f152f9a9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152815f500 con 0x7f152815f0b0 2023-11-08T16:39:50.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.807+0000 7f152f9a9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15280cda10 con 0x7f152815bfc0 2023-11-08T16:39:50.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.807+0000 7f152f9a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152806fa30 con 0x7f15280ce0d0 2023-11-08T16:39:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.807+0000 7f152cf44640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f152815bfc0 0x7f152815eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.807+0000 7f152cf44640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f152815bfc0 0x7f152815eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54566/0 (socket says 172.21.15.81:54566) 2023-11-08T16:39:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.807+0000 7f152cf44640 1 -- 172.21.15.81:0/2991373355 learned_addr learned my addr 172.21.15.81:0/2991373355 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:50.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.808+0000 7f152d745640 1 --2- 172.21.15.81:0/2991373355 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15280ce0d0 0x7f15280caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:50.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.808+0000 7f152cf44640 1 -- 172.21.15.81:0/2991373355 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15280ce0d0 msgr2=0x7f15280caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:50.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.808+0000 7f152cf44640 1 --2- 172.21.15.81:0/2991373355 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15280ce0d0 0x7f15280caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:50.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.808+0000 7f152cf44640 1 -- 172.21.15.81:0/2991373355 >> v1:172.21.15.93:6789/0 conn(0x7f152815f0b0 legacy=0x7f15281516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:50.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.808+0000 7f152cf44640 1 -- 172.21.15.81:0/2991373355 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15280ce4b0 con 0x7f152815bfc0 2023-11-08T16:39:50.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.809+0000 7f152cf44640 1 --2- 172.21.15.81:0/2991373355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f152815bfc0 0x7f152815eb70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1518009870 tx=0x7f151802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871112f0c7a2e11b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:50.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.809+0000 7f151ffff640 1 -- 172.21.15.81:0/2991373355 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518030840 con 0x7f152815bfc0 2023-11-08T16:39:50.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.809+0000 7f151ffff640 1 -- 172.21.15.81:0/2991373355 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15180309a0 con 0x7f152815bfc0 2023-11-08T16:39:50.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.809+0000 7f151ffff640 1 -- 172.21.15.81:0/2991373355 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518030d30 con 0x7f152815bfc0 2023-11-08T16:39:50.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.810+0000 7f152f9a9640 1 -- 172.21.15.81:0/2991373355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f152815bfc0 msgr2=0x7f152815eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:50.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.810+0000 7f152f9a9640 1 --2- 172.21.15.81:0/2991373355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f152815bfc0 0x7f152815eb70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f1518009870 tx=0x7f151802b980 comp rx=0 tx=0).stop 2023-11-08T16:39:50.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.810+0000 7f152f9a9640 1 -- 172.21.15.81:0/2991373355 shutdown_connections 2023-11-08T16:39:50.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.810+0000 7f152f9a9640 1 --2- 172.21.15.81:0/2991373355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f152815bfc0 0x7f152815eb70 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:50.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.810+0000 7f152f9a9640 1 --2- 172.21.15.81:0/2991373355 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15280ce0d0 0x7f15280caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:50.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.810+0000 7f152f9a9640 1 -- 172.21.15.81:0/2991373355 >> 172.21.15.81:0/2991373355 conn(0x7f15280cb580 msgr2=0x7f15280cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:50.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.810+0000 7f152f9a9640 1 -- 172.21.15.81:0/2991373355 shutdown_connections 2023-11-08T16:39:50.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.810+0000 7f152f9a9640 1 -- 172.21.15.81:0/2991373355 wait complete. 2023-11-08T16:39:50.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.812+0000 7f152f9a9640 1 Processor -- start 2023-11-08T16:39:50.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.812+0000 7f152f9a9640 1 -- start start 2023-11-08T16:39:50.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.813+0000 7f152f9a9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15280ce0d0 0x7f15280c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:50.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.813+0000 7f152f9a9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f152815bfc0 0x7f15280c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:50.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.813+0000 7f152f9a9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152815f0b0 0x7f152815a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:50.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.813+0000 7f152f9a9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280c8200 con 0x7f152815bfc0 2023-11-08T16:39:50.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.813+0000 7f152f9a9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15280c84a0 con 0x7f152815f0b0 2023-11-08T16:39:50.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.813+0000 7f152f9a9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15280c8780 con 0x7f15280ce0d0 2023-11-08T16:39:50.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.813+0000 7f152cf44640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f152815bfc0 0x7f15280c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:50.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.813+0000 7f152cf44640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f152815bfc0 0x7f15280c63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54570/0 (socket says 172.21.15.81:54570) 2023-11-08T16:39:50.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.813+0000 7f152cf44640 1 -- 172.21.15.81:0/2575266404 learned_addr learned my addr 172.21.15.81:0/2575266404 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:50.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f152df46640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152815f0b0 0x7f152815a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:50.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f152d745640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15280ce0d0 0x7f15280c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:50.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f152cf44640 1 -- 172.21.15.81:0/2575266404 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15280ce0d0 msgr2=0x7f15280c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:50.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f152cf44640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15280ce0d0 0x7f15280c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:50.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f152cf44640 1 -- 172.21.15.81:0/2575266404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152815f0b0 msgr2=0x7f152815a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:50.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f152cf44640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152815f0b0 0x7f152815a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:50.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f152cf44640 1 -- 172.21.15.81:0/2575266404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15180094d0 con 0x7f152815bfc0 2023-11-08T16:39:50.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f152df46640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152815f0b0 0x7f152815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:50.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f152cf44640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f152815bfc0 0x7f15280c63c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f1518000c00 tx=0x7f1518004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:50.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f151dffb640 1 -- 172.21.15.81:0/2575266404 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518039c90 con 0x7f152815bfc0 2023-11-08T16:39:50.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f151dffb640 1 -- 172.21.15.81:0/2575266404 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1518039df0 con 0x7f152815bfc0 2023-11-08T16:39:50.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f152f9a9640 1 -- 172.21.15.81:0/2575266404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15280c8a60 con 0x7f152815bfc0 2023-11-08T16:39:50.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f151dffb640 1 -- 172.21.15.81:0/2575266404 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518043640 con 0x7f152815bfc0 2023-11-08T16:39:50.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.814+0000 7f152f9a9640 1 -- 172.21.15.81:0/2575266404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15280c4a50 con 0x7f152815bfc0 2023-11-08T16:39:50.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.816+0000 7f152f9a9640 1 -- 172.21.15.81:0/2575266404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15280c2630 con 0x7f152815bfc0 2023-11-08T16:39:50.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.817+0000 7f151dffb640 1 -- 172.21.15.81:0/2575266404 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15180437a0 con 0x7f152815bfc0 2023-11-08T16:39:50.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.818+0000 7f151dffb640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f150c06ae70 0x7f150c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:50.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.818+0000 7f151dffb640 1 -- 172.21.15.81:0/2575266404 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f151803e070 con 0x7f152815bfc0 2023-11-08T16:39:50.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.818+0000 7f152d745640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f150c06ae70 0x7f150c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:50.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.819+0000 7f152d745640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f150c06ae70 0x7f150c06d330 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1524006fd0 tx=0x7f1524008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:50.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:50.823+0000 7f151dffb640 1 -- 172.21.15.81:0/2575266404 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15280c2630 con 0x7f152815bfc0 2023-11-08T16:39:51.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.045+0000 7f152f9a9640 1 -- 172.21.15.81:0/2575266404 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f15280c2630 con 0x7f152815bfc0 2023-11-08T16:39:51.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.613+0000 7f151dffb640 1 -- 172.21.15.81:0/2575266404 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v93) v1 ==== 162+0+0 (secure 0 0 0) 0x7f15280c2630 con 0x7f152815bfc0 2023-11-08T16:39:51.616 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-08T16:39:51.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.616+0000 7f152f9a9640 1 -- 172.21.15.81:0/2575266404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f150c06ae70 msgr2=0x7f150c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:51.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.616+0000 7f152f9a9640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f150c06ae70 0x7f150c06d330 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1524006fd0 tx=0x7f1524008040 comp rx=0 tx=0).stop 2023-11-08T16:39:51.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.616+0000 7f152f9a9640 1 -- 172.21.15.81:0/2575266404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f152815bfc0 msgr2=0x7f15280c63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:51.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.616+0000 7f152f9a9640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f152815bfc0 0x7f15280c63c0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f1518000c00 tx=0x7f1518004730 comp rx=0 tx=0).stop 2023-11-08T16:39:51.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.617+0000 7f152f9a9640 1 -- 172.21.15.81:0/2575266404 shutdown_connections 2023-11-08T16:39:51.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.617+0000 7f152f9a9640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f150c06ae70 0x7f150c06d330 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:51.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.617+0000 7f152f9a9640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f152815f0b0 0x7f152815a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:51.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.617+0000 7f152f9a9640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f152815bfc0 0x7f15280c63c0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:51.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.617+0000 7f152f9a9640 1 --2- 172.21.15.81:0/2575266404 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f15280ce0d0 0x7f15280c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:51.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.617+0000 7f152f9a9640 1 -- 172.21.15.81:0/2575266404 >> 172.21.15.81:0/2575266404 conn(0x7f15280cb580 msgr2=0x7f15280af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:51.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.617+0000 7f152f9a9640 1 -- 172.21.15.81:0/2575266404 shutdown_connections 2023-11-08T16:39:51.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.617+0000 7f152f9a9640 1 -- 172.21.15.81:0/2575266404 wait complete. 2023-11-08T16:39:51.641 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.894+0000 7fdc70864640 1 Processor -- start 2023-11-08T16:39:51.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.895+0000 7fdc70864640 1 -- start start 2023-11-08T16:39:51.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.896+0000 7fdc70864640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c0b4fd0 0x7fdc6c065df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:51.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.896+0000 7fdc70864640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6c0b4210 con 0x7fdc6c15d4c0 2023-11-08T16:39:51.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.896+0000 7fdc70864640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6c0b44f0 con 0x7fdc6c0cf1a0 2023-11-08T16:39:51.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.896+0000 7fdc70864640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0664f0 con 0x7fdc6c0b4fd0 2023-11-08T16:39:51.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.896+0000 7fdc69d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fdc6c15d4c0 0x7fdc6c0af2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56514/0 (socket says 172.21.15.81:56514) 2023-11-08T16:39:51.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.896+0000 7fdc69d9b640 1 -- 172.21.15.81:0/2234837481 learned_addr learned my addr 172.21.15.81:0/2234837481 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:51.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6ad9d640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c0b4fd0 0x7fdc6c065df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:51.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1588502262 0 0) 0x7fdc6c0b4210 con 0x7fdc6c15d4c0 2023-11-08T16:39:51.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc58003520 con 0x7fdc6c15d4c0 2023-11-08T16:39:51.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3240458555 0 0) 0x7fdc58003520 con 0x7fdc6c15d4c0 2023-11-08T16:39:51.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c0b4fd0 msgr2=0x7fdc6c065df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:51.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c0b4fd0 0x7fdc6c065df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:51.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 >> v1:172.21.15.93:6789/0 conn(0x7fdc6c0cf1a0 legacy=0x7fdc6c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:51.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c0b47d0 con 0x7fdc6c15d4c0 2023-11-08T16:39:51.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fdc54001670 con 0x7fdc6c15d4c0 2023-11-08T16:39:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 >> v1:172.21.15.81:6789/0 conn(0x7fdc6c15d4c0 legacy=0x7fdc6c0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc58003a80 0x7fdc58003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 0x7fdc58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c0b4fd0 0x7fdc5800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:51.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0664f0 con 0x7fdc580047c0 2023-11-08T16:39:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc580054e0 con 0x7fdc58003a80 2023-11-08T16:39:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.897+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc58005650 con 0x7fdc6c0b4fd0 2023-11-08T16:39:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.898+0000 7fdc69d9b640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 0x7fdc58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.898+0000 7fdc6ad9d640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c0b4fd0 0x7fdc5800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:51.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.898+0000 7fdc6a59c640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc58003a80 0x7fdc58003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:51.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.898+0000 7fdc69d9b640 1 -- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c0b4fd0 msgr2=0x7fdc5800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:51.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.898+0000 7fdc69d9b640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c0b4fd0 0x7fdc5800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:51.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.898+0000 7fdc69d9b640 1 -- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc58003a80 msgr2=0x7fdc58003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:51.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.898+0000 7fdc69d9b640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc58003a80 0x7fdc58003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:51.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.898+0000 7fdc69d9b640 1 -- 172.21.15.81:0/2234837481 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc58003520 con 0x7fdc580047c0 2023-11-08T16:39:51.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.898+0000 7fdc6ad9d640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c0b4fd0 0x7fdc5800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:51.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.898+0000 7fdc6a59c640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc58003a80 0x7fdc58003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:51.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.899+0000 7fdc69d9b640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 0x7fdc58004c40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400c240 tx=0x7fdc580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c9c8833def25a08 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:51.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.899+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc54005ad0 con 0x7fdc580047c0 2023-11-08T16:39:51.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.899+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc54005c30 con 0x7fdc580047c0 2023-11-08T16:39:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.899+0000 7fdc6959a640 1 -- 172.21.15.81:0/2234837481 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5402c870 con 0x7fdc580047c0 2023-11-08T16:39:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.900+0000 7fdc70864640 1 -- 172.21.15.81:0/2234837481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 msgr2=0x7fdc58004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.900+0000 7fdc70864640 1 --2- 172.21.15.81:0/2234837481 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 0x7fdc58004c40 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400c240 tx=0x7fdc580057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.900+0000 7fdc6ad9d640 1 -- 172.21.15.81:0/2234837481 reap_dead start 2023-11-08T16:39:51.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.900+0000 7fdc70864640 1 -- 172.21.15.81:0/2234837481 shutdown_connections 2023-11-08T16:39:51.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.900+0000 7fdc70864640 1 -- 172.21.15.81:0/2234837481 >> 172.21.15.81:0/2234837481 conn(0x7fdc6c071540 msgr2=0x7fdc6c071950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:51.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.900+0000 7fdc70864640 1 -- 172.21.15.81:0/2234837481 shutdown_connections 2023-11-08T16:39:51.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.900+0000 7fdc70864640 1 -- 172.21.15.81:0/2234837481 wait complete. 2023-11-08T16:39:51.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.902+0000 7fdc70864640 1 Processor -- start 2023-11-08T16:39:51.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.902+0000 7fdc70864640 1 -- start start 2023-11-08T16:39:51.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.903+0000 7fdc70864640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 0x7fdc6c0c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:51.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.903+0000 7fdc70864640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc6c0c69b0 0x7fdc6c1f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:51.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.903+0000 7fdc70864640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c1f2cf0 0x7fdc6c1f3170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:51.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.903+0000 7fdc70864640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c098f60 con 0x7fdc580047c0 2023-11-08T16:39:51.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.903+0000 7fdc70864640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0c7c60 con 0x7fdc6c0c69b0 2023-11-08T16:39:51.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.903+0000 7fdc70864640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c1f36b0 con 0x7fdc6c1f2cf0 2023-11-08T16:39:51.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.903+0000 7fdc6a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 0x7fdc6c0c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:51.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.903+0000 7fdc6a59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 0x7fdc6c0c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38036/0 (socket says 172.21.15.81:38036) 2023-11-08T16:39:51.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.903+0000 7fdc6a59c640 1 -- 172.21.15.81:0/2852906752 learned_addr learned my addr 172.21.15.81:0/2852906752 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:51.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.904+0000 7fdc69d9b640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc6c0c69b0 0x7fdc6c1f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:51.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.904+0000 7fdc6a59c640 1 -- 172.21.15.81:0/2852906752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c1f2cf0 msgr2=0x7fdc6c1f3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:51.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.904+0000 7fdc6ad9d640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c1f2cf0 0x7fdc6c1f3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:51.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.904+0000 7fdc6a59c640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c1f2cf0 0x7fdc6c1f3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:51.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.904+0000 7fdc6a59c640 1 -- 172.21.15.81:0/2852906752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc6c0c69b0 msgr2=0x7fdc6c1f27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:51.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.904+0000 7fdc6a59c640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc6c0c69b0 0x7fdc6c1f27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:51.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.904+0000 7fdc6a59c640 1 -- 172.21.15.81:0/2852906752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c0c8300 con 0x7fdc580047c0 2023-11-08T16:39:51.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.904+0000 7fdc6a59c640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 0x7fdc6c0c6470 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00db10 tx=0x7fdc5c002130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:51.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.904+0000 7fdc62ffd640 1 -- 172.21.15.81:0/2852906752 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c017070 con 0x7fdc580047c0 2023-11-08T16:39:51.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.905+0000 7fdc62ffd640 1 -- 172.21.15.81:0/2852906752 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5c003bf0 con 0x7fdc580047c0 2023-11-08T16:39:51.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.905+0000 7fdc70864640 1 -- 172.21.15.81:0/2852906752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c0c6fe0 con 0x7fdc580047c0 2023-11-08T16:39:51.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.905+0000 7fdc62ffd640 1 -- 172.21.15.81:0/2852906752 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c006ba0 con 0x7fdc580047c0 2023-11-08T16:39:51.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.905+0000 7fdc70864640 1 -- 172.21.15.81:0/2852906752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c0c75a0 con 0x7fdc580047c0 2023-11-08T16:39:51.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.906+0000 7fdc70864640 1 -- 172.21.15.81:0/2852906752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c0c6d90 con 0x7fdc580047c0 2023-11-08T16:39:51.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.907+0000 7fdc62ffd640 1 -- 172.21.15.81:0/2852906752 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc5c006d00 con 0x7fdc580047c0 2023-11-08T16:39:51.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.907+0000 7fdc62ffd640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc4c06ae70 0x7fdc4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:51.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.908+0000 7fdc62ffd640 1 -- 172.21.15.81:0/2852906752 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fdc5c089520 con 0x7fdc580047c0 2023-11-08T16:39:51.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.908+0000 7fdc69d9b640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc4c06ae70 0x7fdc4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:51.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.909+0000 7fdc69d9b640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc4c06ae70 0x7fdc4c06d330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdc54034000 tx=0x7fdc54009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:51.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:51.912+0000 7fdc62ffd640 1 -- 172.21.15.81:0/2852906752 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc5c054300 con 0x7fdc580047c0 2023-11-08T16:39:52.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.140+0000 7fdc70864640 1 -- 172.21.15.81:0/2852906752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc6c04ece0 con 0x7fdc580047c0 2023-11-08T16:39:52.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.141+0000 7fdc62ffd640 1 -- 172.21.15.81:0/2852906752 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4911 (secure 0 0 0) 0x7fdc5c057920 con 0x7fdc580047c0 2023-11-08T16:39:52.143 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:52.144 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":93,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1157756119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1157756119},{"type":"v1","addr":"172.21.15.93:6833","nonce":1157756119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/4124283270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":4124283270},{"type":"v1","addr":"172.21.15.93:6835","nonce":4124283270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":7701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1607105255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1607105255},{"type":"v1","addr":"172.21.15.81:6835","nonce":1607105255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8028,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3163533823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3163533823},{"type":"v1","addr":"172.21.15.81:6837","nonce":3163533823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:51.048090+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:39:52.144 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 93 2023-11-08T16:39:52.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 -- 172.21.15.81:0/2852906752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc4c06ae70 msgr2=0x7fdc4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:52.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc4c06ae70 0x7fdc4c06d330 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdc54034000 tx=0x7fdc54009030 comp rx=0 tx=0).stop 2023-11-08T16:39:52.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 -- 172.21.15.81:0/2852906752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 msgr2=0x7fdc6c0c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:52.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 0x7fdc6c0c6470 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00db10 tx=0x7fdc5c002130 comp rx=0 tx=0).stop 2023-11-08T16:39:52.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 -- 172.21.15.81:0/2852906752 shutdown_connections 2023-11-08T16:39:52.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc4c06ae70 0x7fdc4c06d330 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc6c1f2cf0 0x7fdc6c1f3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc6c0c69b0 0x7fdc6c1f27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 --2- 172.21.15.81:0/2852906752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc580047c0 0x7fdc6c0c6470 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 -- 172.21.15.81:0/2852906752 >> 172.21.15.81:0/2852906752 conn(0x7fdc6c071540 msgr2=0x7fdc6c0af7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:52.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 -- 172.21.15.81:0/2852906752 shutdown_connections 2023-11-08T16:39:52.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.143+0000 7fdc70864640 1 -- 172.21.15.81:0/2852906752 wait complete. 2023-11-08T16:39:52.165 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-11-08T16:39:52.165 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:52.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.374+0000 7fe15307c640 1 Processor -- start 2023-11-08T16:39:52.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.375+0000 7fe15307c640 1 -- start start 2023-11-08T16:39:52.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.376+0000 7fe15307c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe14c165940 0x7fe14c16c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:52.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.376+0000 7fe15307c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14c0ce3f0 con 0x7fe14c0adf20 2023-11-08T16:39:52.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.376+0000 7fe15307c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe14c0cac80 con 0x7fe14c0adb40 2023-11-08T16:39:52.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.376+0000 7fe15307c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe14c0cb340 con 0x7fe14c165940 2023-11-08T16:39:52.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.377+0000 7fe14bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe14c0adf20 0x7fe14c165230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56528/0 (socket says 172.21.15.81:56528) 2023-11-08T16:39:52.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.377+0000 7fe14bfff640 1 -- 172.21.15.81:0/1504879902 learned_addr learned my addr 172.21.15.81:0/1504879902 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:52.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.377+0000 7fe151619640 1 --2- 172.21.15.81:0/1504879902 >> v2:172.21.15.81:3300/0 conn(0x7fe14c165940 0x7fe14c16c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:52.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.377+0000 7fe14b7fe640 1 -- 172.21.15.81:0/1504879902 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456386856 0 0) 0x7fe14c0ce3f0 con 0x7fe14c0adf20 2023-11-08T16:39:52.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.377+0000 7fe14b7fe640 1 -- 172.21.15.81:0/1504879902 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe13c003520 con 0x7fe14c0adf20 2023-11-08T16:39:52.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.377+0000 7fe151619640 1 -- 172.21.15.81:0/1504879902 >> v1:172.21.15.93:6789/0 conn(0x7fe14c0adb40 legacy=0x7fe14c0c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:52.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.377+0000 7fe151619640 1 -- 172.21.15.81:0/1504879902 >> v1:172.21.15.81:6789/0 conn(0x7fe14c0adf20 legacy=0x7fe14c165230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:52.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.377+0000 7fe151619640 1 -- 172.21.15.81:0/1504879902 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c0caf60 con 0x7fe14c165940 2023-11-08T16:39:52.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.379+0000 7fe151619640 1 --2- 172.21.15.81:0/1504879902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe14c165940 0x7fe14c16c560 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe144009d90 tx=0x7fe14402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa33837caf4a2f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:52.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.379+0000 7fe14b7fe640 1 -- 172.21.15.81:0/1504879902 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14402cc60 con 0x7fe14c165940 2023-11-08T16:39:52.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.379+0000 7fe14b7fe640 1 -- 172.21.15.81:0/1504879902 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14402cdc0 con 0x7fe14c165940 2023-11-08T16:39:52.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.379+0000 7fe14b7fe640 1 -- 172.21.15.81:0/1504879902 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144034660 con 0x7fe14c165940 2023-11-08T16:39:52.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.380+0000 7fe15307c640 1 -- 172.21.15.81:0/1504879902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe14c165940 msgr2=0x7fe14c16c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:52.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.380+0000 7fe15307c640 1 --2- 172.21.15.81:0/1504879902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe14c165940 0x7fe14c16c560 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe144009d90 tx=0x7fe14402beb0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.380+0000 7fe15307c640 1 -- 172.21.15.81:0/1504879902 shutdown_connections 2023-11-08T16:39:52.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.380+0000 7fe15307c640 1 --2- 172.21.15.81:0/1504879902 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe14c165940 0x7fe14c16c560 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.380+0000 7fe15307c640 1 -- 172.21.15.81:0/1504879902 >> 172.21.15.81:0/1504879902 conn(0x7fe14c0cb490 msgr2=0x7fe14c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:52.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.380+0000 7fe15307c640 1 -- 172.21.15.81:0/1504879902 shutdown_connections 2023-11-08T16:39:52.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.380+0000 7fe15307c640 1 -- 172.21.15.81:0/1504879902 wait complete. 2023-11-08T16:39:52.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.382+0000 7fe15307c640 1 Processor -- start 2023-11-08T16:39:52.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.382+0000 7fe15307c640 1 -- start start 2023-11-08T16:39:52.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe15307c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe14c0adb40 0x7fe14c0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:52.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe15307c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe14c0adf20 0x7fe14c154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:52.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe15307c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe14c165940 0x7fe14c0c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:52.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe15307c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0c8d30 con 0x7fe14c0adf20 2023-11-08T16:39:52.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe15307c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0c4f50 con 0x7fe14c0adb40 2023-11-08T16:39:52.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe15307c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14c0c4340 con 0x7fe14c165940 2023-11-08T16:39:52.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe14bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe14c0adf20 0x7fe14c154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:52.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe14bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe14c0adf20 0x7fe14c154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38062/0 (socket says 172.21.15.81:38062) 2023-11-08T16:39:52.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe14bfff640 1 -- 172.21.15.81:0/4209285869 learned_addr learned my addr 172.21.15.81:0/4209285869 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:52.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe14bfff640 1 -- 172.21.15.81:0/4209285869 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe14c165940 msgr2=0x7fe14c0c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:52.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe14bfff640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe14c165940 0x7fe14c0c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe14bfff640 1 -- 172.21.15.81:0/4209285869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe14c0adb40 msgr2=0x7fe14c0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:52.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe14bfff640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe14c0adb40 0x7fe14c0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.383+0000 7fe14bfff640 1 -- 172.21.15.81:0/4209285869 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe144009a70 con 0x7fe14c0adf20 2023-11-08T16:39:52.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.384+0000 7fe14bfff640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe14c0adf20 0x7fe14c154e80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe13800c4d0 tx=0x7fe13800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:52.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.384+0000 7fe1497fa640 1 -- 172.21.15.81:0/4209285869 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138005ce0 con 0x7fe14c0adf20 2023-11-08T16:39:52.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.384+0000 7fe1497fa640 1 -- 172.21.15.81:0/4209285869 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe138005e40 con 0x7fe14c0adf20 2023-11-08T16:39:52.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.384+0000 7fe1497fa640 1 -- 172.21.15.81:0/4209285869 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138007270 con 0x7fe14c0adf20 2023-11-08T16:39:52.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.384+0000 7fe15307c640 1 -- 172.21.15.81:0/4209285869 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe14c0c4620 con 0x7fe14c0adf20 2023-11-08T16:39:52.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.384+0000 7fe15307c640 1 -- 172.21.15.81:0/4209285869 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe14c0c4be0 con 0x7fe14c0adf20 2023-11-08T16:39:52.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.385+0000 7fe15307c640 1 -- 172.21.15.81:0/4209285869 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe14c04ece0 con 0x7fe14c0adf20 2023-11-08T16:39:52.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.386+0000 7fe1497fa640 1 -- 172.21.15.81:0/4209285869 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe1380073d0 con 0x7fe14c0adf20 2023-11-08T16:39:52.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.387+0000 7fe1497fa640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe13006aec0 0x7fe13006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:52.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.387+0000 7fe1497fa640 1 -- 172.21.15.81:0/4209285869 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fe138088dd0 con 0x7fe14c0adf20 2023-11-08T16:39:52.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.387+0000 7fe150e18640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe13006aec0 0x7fe13006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:52.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.388+0000 7fe150e18640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe13006aec0 0x7fe13006d380 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe140000f60 tx=0x7fe1400079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:52.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.392+0000 7fe1497fa640 1 -- 172.21.15.81:0/4209285869 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe138053cc0 con 0x7fe14c0adf20 2023-11-08T16:39:52.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.616+0000 7fe15307c640 1 -- 172.21.15.81:0/4209285869 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe14c0b4990 con 0x7fe14c0adf20 2023-11-08T16:39:52.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.621+0000 7fe1497fa640 1 -- 172.21.15.81:0/4209285869 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4911 (secure 0 0 0) 0x7fe138017070 con 0x7fe14c0adf20 2023-11-08T16:39:52.624 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:52.624 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":93,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1157756119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1157756119},{"type":"v1","addr":"172.21.15.93:6833","nonce":1157756119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/4124283270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":4124283270},{"type":"v1","addr":"172.21.15.93:6835","nonce":4124283270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":7701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1607105255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1607105255},{"type":"v1","addr":"172.21.15.81:6835","nonce":1607105255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8028,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3163533823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3163533823},{"type":"v1","addr":"172.21.15.81:6837","nonce":3163533823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:51.048090+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:39:52.624 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 93 2023-11-08T16:39:52.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.623+0000 7fe15307c640 1 -- 172.21.15.81:0/4209285869 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe13006aec0 msgr2=0x7fe13006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:52.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.623+0000 7fe15307c640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe13006aec0 0x7fe13006d380 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe140000f60 tx=0x7fe1400079b0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.623+0000 7fe15307c640 1 -- 172.21.15.81:0/4209285869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe14c0adf20 msgr2=0x7fe14c154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:52.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.623+0000 7fe15307c640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe14c0adf20 0x7fe14c154e80 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fe13800c4d0 tx=0x7fe13800c990 comp rx=0 tx=0).stop 2023-11-08T16:39:52.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.624+0000 7fe15307c640 1 -- 172.21.15.81:0/4209285869 shutdown_connections 2023-11-08T16:39:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.624+0000 7fe15307c640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe13006aec0 0x7fe13006d380 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.624+0000 7fe15307c640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe14c165940 0x7fe14c0c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.624+0000 7fe15307c640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe14c0adf20 0x7fe14c154e80 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.624+0000 7fe15307c640 1 --2- 172.21.15.81:0/4209285869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe14c0adb40 0x7fe14c0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.624+0000 7fe15307c640 1 -- 172.21.15.81:0/4209285869 >> 172.21.15.81:0/4209285869 conn(0x7fe14c0cb490 msgr2=0x7fe14c0cb870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.624+0000 7fe15307c640 1 -- 172.21.15.81:0/4209285869 shutdown_connections 2023-11-08T16:39:52.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.624+0000 7fe15307c640 1 -- 172.21.15.81:0/4209285869 wait complete. 2023-11-08T16:39:52.645 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:52.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.851+0000 7f5815000640 1 Processor -- start 2023-11-08T16:39:52.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.851+0000 7f5815000640 1 -- start start 2023-11-08T16:39:52.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.853+0000 7f5815000640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f58100ca480 0x7f581015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:52.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.853+0000 7f5815000640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f581015ea60 0x7f581015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:52.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.853+0000 7f5815000640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581006b630 con 0x7f58100ade50 2023-11-08T16:39:52.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.853+0000 7f5815000640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58100b1860 con 0x7f581015ea60 2023-11-08T16:39:52.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.853+0000 7f5815000640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58100ca910 con 0x7f58100ca480 2023-11-08T16:39:52.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.853+0000 7f580ed9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f581015ea60 0x7f581015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:52.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.853+0000 7f580ed9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f581015ea60 0x7f581015eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38068/0 (socket says 172.21.15.81:38068) 2023-11-08T16:39:52.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.853+0000 7f580ed9d640 1 -- 172.21.15.81:0/2322085389 learned_addr learned my addr 172.21.15.81:0/2322085389 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:52.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.853+0000 7f580ed9d640 1 -- 172.21.15.81:0/2322085389 >> v2:172.21.15.93:3300/0 conn(0x7f58100ca480 msgr2=0x7f581015ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:52.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.853+0000 7f580ed9d640 1 --2- 172.21.15.81:0/2322085389 >> v2:172.21.15.93:3300/0 conn(0x7f58100ca480 0x7f581015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.853+0000 7f580ed9d640 1 -- 172.21.15.81:0/2322085389 >> v1:172.21.15.93:6789/0 conn(0x7f58100ade50 legacy=0x7f581015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:52.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.854+0000 7f580ed9d640 1 -- 172.21.15.81:0/2322085389 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58100cabb0 con 0x7f581015ea60 2023-11-08T16:39:52.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.855+0000 7f580ed9d640 1 --2- 172.21.15.81:0/2322085389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f581015ea60 0x7f581015eef0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f5810073d00 tx=0x7f580802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7888efd46a50d4cf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:52.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.855+0000 7f580d59a640 1 -- 172.21.15.81:0/2322085389 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580802ca80 con 0x7f581015ea60 2023-11-08T16:39:52.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.855+0000 7f580d59a640 1 -- 172.21.15.81:0/2322085389 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580802cbe0 con 0x7f581015ea60 2023-11-08T16:39:52.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.855+0000 7f580d59a640 1 -- 172.21.15.81:0/2322085389 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58080343f0 con 0x7f581015ea60 2023-11-08T16:39:52.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.855+0000 7f5815000640 1 -- 172.21.15.81:0/2322085389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f581015ea60 msgr2=0x7f581015eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:52.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.855+0000 7f5815000640 1 --2- 172.21.15.81:0/2322085389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f581015ea60 0x7f581015eef0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f5810073d00 tx=0x7f580802bee0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.856+0000 7f5815000640 1 -- 172.21.15.81:0/2322085389 shutdown_connections 2023-11-08T16:39:52.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.856+0000 7f5815000640 1 --2- 172.21.15.81:0/2322085389 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f581015ea60 0x7f581015eef0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.856+0000 7f5815000640 1 --2- 172.21.15.81:0/2322085389 >> v2:172.21.15.93:3300/0 conn(0x7f58100ca480 0x7f581015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.856+0000 7f5815000640 1 -- 172.21.15.81:0/2322085389 >> 172.21.15.81:0/2322085389 conn(0x7f58100cb490 msgr2=0x7f58100cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:52.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.856+0000 7f5815000640 1 -- 172.21.15.81:0/2322085389 shutdown_connections 2023-11-08T16:39:52.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.856+0000 7f5815000640 1 -- 172.21.15.81:0/2322085389 wait complete. 2023-11-08T16:39:52.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.857+0000 7f5815000640 1 Processor -- start 2023-11-08T16:39:52.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.857+0000 7f5815000640 1 -- start start 2023-11-08T16:39:52.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.858+0000 7f5815000640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58100ade50 0x7f5810166630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:52.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.858+0000 7f5815000640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58100ca480 0x7f581014f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:52.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.858+0000 7f5815000640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f581015ea60 0x7f58100cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:52.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.858+0000 7f5815000640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581014e4b0 con 0x7f58100ade50 2023-11-08T16:39:52.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.858+0000 7f5815000640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581014e790 con 0x7f581015ea60 2023-11-08T16:39:52.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.858+0000 7f5815000640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581014ea70 con 0x7f58100ca480 2023-11-08T16:39:52.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.859+0000 7f580e59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58100ade50 0x7f5810166630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:52.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.859+0000 7f580e59c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58100ade50 0x7f5810166630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38082/0 (socket says 172.21.15.81:38082) 2023-11-08T16:39:52.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.859+0000 7f580e59c640 1 -- 172.21.15.81:0/2165766093 learned_addr learned my addr 172.21.15.81:0/2165766093 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:52.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.859+0000 7f580dd9b640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58100ca480 0x7f581014f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:52.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.859+0000 7f580ed9d640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f581015ea60 0x7f58100cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:52.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.859+0000 7f580e59c640 1 -- 172.21.15.81:0/2165766093 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58100ca480 msgr2=0x7f581014f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:52.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.859+0000 7f580e59c640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58100ca480 0x7f581014f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.859+0000 7f580e59c640 1 -- 172.21.15.81:0/2165766093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f581015ea60 msgr2=0x7f58100cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:52.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.859+0000 7f580e59c640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f581015ea60 0x7f58100cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:52.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.859+0000 7f580e59c640 1 -- 172.21.15.81:0/2165766093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5808009b30 con 0x7f58100ade50 2023-11-08T16:39:52.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.859+0000 7f580e59c640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58100ade50 0x7f5810166630 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f580000c430 tx=0x7f580000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:52.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.860+0000 7f57feffd640 1 -- 172.21.15.81:0/2165766093 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800011790 con 0x7f58100ade50 2023-11-08T16:39:52.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.860+0000 7f57feffd640 1 -- 172.21.15.81:0/2165766093 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58000118f0 con 0x7f58100ade50 2023-11-08T16:39:52.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.860+0000 7f57feffd640 1 -- 172.21.15.81:0/2165766093 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5800011ce0 con 0x7f58100ade50 2023-11-08T16:39:52.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.860+0000 7f5815000640 1 -- 172.21.15.81:0/2165766093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581014ed50 con 0x7f58100ade50 2023-11-08T16:39:52.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.860+0000 7f5815000640 1 -- 172.21.15.81:0/2165766093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581014fa90 con 0x7f58100ade50 2023-11-08T16:39:52.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.861+0000 7f5815000640 1 -- 172.21.15.81:0/2165766093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581014f270 con 0x7f58100ade50 2023-11-08T16:39:52.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.862+0000 7f57feffd640 1 -- 172.21.15.81:0/2165766093 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f580001c460 con 0x7f58100ade50 2023-11-08T16:39:52.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.863+0000 7f57feffd640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57f006ae70 0x7f57f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:52.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.863+0000 7f57feffd640 1 -- 172.21.15.81:0/2165766093 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f5800089930 con 0x7f58100ade50 2023-11-08T16:39:52.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.863+0000 7f580dd9b640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57f006ae70 0x7f57f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:52.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.864+0000 7f580dd9b640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57f006ae70 0x7f57f006d330 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f58100c93a0 tx=0x7f5804008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:52.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:52.867+0000 7f57feffd640 1 -- 172.21.15.81:0/2165766093 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f581014f270 con 0x7f58100ade50 2023-11-08T16:39:53.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.093+0000 7f5815000640 1 -- 172.21.15.81:0/2165766093 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f581014f270 con 0x7f58100ade50 2023-11-08T16:39:53.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.095+0000 7f57feffd640 1 -- 172.21.15.81:0/2165766093 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4911 (secure 0 0 0) 0x7f581014f270 con 0x7f58100ade50 2023-11-08T16:39:53.097 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:53.098 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":93,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1157756119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1157756119},{"type":"v1","addr":"172.21.15.93:6833","nonce":1157756119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/4124283270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":4124283270},{"type":"v1","addr":"172.21.15.93:6835","nonce":4124283270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":7701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1607105255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1607105255},{"type":"v1","addr":"172.21.15.81:6835","nonce":1607105255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8028,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3163533823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3163533823},{"type":"v1","addr":"172.21.15.81:6837","nonce":3163533823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:51.048090+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:39:53.098 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 93 2023-11-08T16:39:53.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.097+0000 7f5815000640 1 -- 172.21.15.81:0/2165766093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57f006ae70 msgr2=0x7f57f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.097+0000 7f5815000640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57f006ae70 0x7f57f006d330 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f58100c93a0 tx=0x7f5804008040 comp rx=0 tx=0).stop 2023-11-08T16:39:53.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.097+0000 7f5815000640 1 -- 172.21.15.81:0/2165766093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58100ade50 msgr2=0x7f5810166630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.097+0000 7f5815000640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58100ade50 0x7f5810166630 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f580000c430 tx=0x7f580000c8f0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.097+0000 7f5815000640 1 -- 172.21.15.81:0/2165766093 shutdown_connections 2023-11-08T16:39:53.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.097+0000 7f5815000640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f57f006ae70 0x7f57f006d330 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.097+0000 7f5815000640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f581015ea60 0x7f58100cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.097+0000 7f5815000640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58100ca480 0x7f581014f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.097+0000 7f5815000640 1 --2- 172.21.15.81:0/2165766093 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58100ade50 0x7f5810166630 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.097+0000 7f5815000640 1 -- 172.21.15.81:0/2165766093 >> 172.21.15.81:0/2165766093 conn(0x7f58100cb490 msgr2=0x7f58101533d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:53.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.098+0000 7f5815000640 1 -- 172.21.15.81:0/2165766093 shutdown_connections 2023-11-08T16:39:53.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.098+0000 7f5815000640 1 -- 172.21.15.81:0/2165766093 wait complete. 2023-11-08T16:39:53.119 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:53.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.315+0000 7f340a168640 1 Processor -- start 2023-11-08T16:39:53.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.315+0000 7f340a168640 1 -- start start 2023-11-08T16:39:53.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.317+0000 7f340a168640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f340406b0c0 0x7f34040c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.317+0000 7f340a168640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34040c5030 con 0x7f340415b6d0 2023-11-08T16:39:53.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.317+0000 7f340a168640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34040c61f0 con 0x7f34040cb980 2023-11-08T16:39:53.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.317+0000 7f340a168640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34040c4420 con 0x7f340406b0c0 2023-11-08T16:39:53.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.317+0000 7f34037fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f340415b6d0 0x7f340415bab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56544/0 (socket says 172.21.15.81:56544) 2023-11-08T16:39:53.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.317+0000 7f34037fe640 1 -- 172.21.15.81:0/191441067 learned_addr learned my addr 172.21.15.81:0/191441067 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:53.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f3402ffd640 1 --2- 172.21.15.81:0/191441067 >> v2:172.21.15.93:3300/0 conn(0x7f340406b0c0 0x7f34040c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:53.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1908192751 0 0) 0x7f34040c5030 con 0x7f340415b6d0 2023-11-08T16:39:53.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33ec003520 con 0x7f340415b6d0 2023-11-08T16:39:53.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1467369634 0 0) 0x7f33ec003520 con 0x7f340415b6d0 2023-11-08T16:39:53.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 >> v2:172.21.15.93:3300/0 conn(0x7f340406b0c0 msgr2=0x7f34040c5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:53.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 --2- 172.21.15.81:0/191441067 >> v2:172.21.15.93:3300/0 conn(0x7f340406b0c0 0x7f34040c5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 >> v1:172.21.15.93:6789/0 conn(0x7f34040cb980 legacy=0x7f3404164c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34040c4700 con 0x7f340415b6d0 2023-11-08T16:39:53.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f33f4001670 con 0x7f340415b6d0 2023-11-08T16:39:53.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 >> v1:172.21.15.81:6789/0 conn(0x7f340415b6d0 legacy=0x7f340415bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 --2- 172.21.15.81:0/191441067 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33ec003aa0 0x7f33ec003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 --2- 172.21.15.81:0/191441067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33ec0047c0 0x7f33ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 --2- 172.21.15.81:0/191441067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 0x7f33ec00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34040c4420 con 0x7f33ec0052a0 2023-11-08T16:39:53.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec00c1f0 con 0x7f33ec0047c0 2023-11-08T16:39:53.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.318+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33ec00c3b0 con 0x7f33ec003aa0 2023-11-08T16:39:53.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.319+0000 7f3403fff640 1 --2- 172.21.15.81:0/191441067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 0x7f33ec00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:53.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.319+0000 7f3403fff640 1 -- 172.21.15.81:0/191441067 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33ec003aa0 msgr2=0x7f33ec003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:53.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.319+0000 7f3403fff640 1 --2- 172.21.15.81:0/191441067 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33ec003aa0 0x7f33ec003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.319+0000 7f3403fff640 1 -- 172.21.15.81:0/191441067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33ec0047c0 msgr2=0x7f33ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:53.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.319+0000 7f3403fff640 1 --2- 172.21.15.81:0/191441067 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33ec0047c0 0x7f33ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.319+0000 7f3403fff640 1 -- 172.21.15.81:0/191441067 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec003520 con 0x7f33ec0052a0 2023-11-08T16:39:53.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.320+0000 7f3403fff640 1 -- 172.21.15.81:0/191441067 reap_dead start 2023-11-08T16:39:53.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.320+0000 7f3403fff640 1 --2- 172.21.15.81:0/191441067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 0x7f33ec00b980 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f33fc001e20 tx=0x7f33ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5888d8b99e77352e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:53.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.321+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc003b70 con 0x7f33ec0052a0 2023-11-08T16:39:53.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.321+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc003cd0 con 0x7f33ec0052a0 2023-11-08T16:39:53.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.321+0000 7f34027fc640 1 -- 172.21.15.81:0/191441067 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc005310 con 0x7f33ec0052a0 2023-11-08T16:39:53.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.321+0000 7f340a168640 1 -- 172.21.15.81:0/191441067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 msgr2=0x7f33ec00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.321+0000 7f340a168640 1 --2- 172.21.15.81:0/191441067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 0x7f33ec00b980 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f33fc001e20 tx=0x7f33ec0057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.321+0000 7f340a168640 1 -- 172.21.15.81:0/191441067 shutdown_connections 2023-11-08T16:39:53.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.321+0000 7f340a168640 1 --2- 172.21.15.81:0/191441067 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 0x7f33ec00b980 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.321+0000 7f340a168640 1 -- 172.21.15.81:0/191441067 >> 172.21.15.81:0/191441067 conn(0x7f34040cbe40 msgr2=0x7f34040cc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:53.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.322+0000 7f340a168640 1 -- 172.21.15.81:0/191441067 shutdown_connections 2023-11-08T16:39:53.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.322+0000 7f340a168640 1 -- 172.21.15.81:0/191441067 wait complete. 2023-11-08T16:39:53.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.323+0000 7f340a168640 1 Processor -- start 2023-11-08T16:39:53.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.324+0000 7f340a168640 1 -- start start 2023-11-08T16:39:53.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.324+0000 7f340a168640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 0x7f3404150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.324+0000 7f340a168640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34040710d0 0x7f3404071530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.324+0000 7f340a168640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34040c6fc0 0x7f34040c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.324+0000 7f340a168640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3404151560 con 0x7f33ec0052a0 2023-11-08T16:39:53.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.324+0000 7f340a168640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3404071a70 con 0x7f34040c6fc0 2023-11-08T16:39:53.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.324+0000 7f340a168640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34040b1620 con 0x7f34040710d0 2023-11-08T16:39:53.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.325+0000 7f34037fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 0x7f3404150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:53.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.325+0000 7f34037fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 0x7f3404150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38100/0 (socket says 172.21.15.81:38100) 2023-11-08T16:39:53.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.325+0000 7f34037fe640 1 -- 172.21.15.81:0/2287404291 learned_addr learned my addr 172.21.15.81:0/2287404291 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:53.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.325+0000 7f3402ffd640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34040710d0 0x7f3404071530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:53.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.325+0000 7f34037fe640 1 -- 172.21.15.81:0/2287404291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34040710d0 msgr2=0x7f3404071530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.325+0000 7f34037fe640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34040710d0 0x7f3404071530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.325+0000 7f34037fe640 1 -- 172.21.15.81:0/2287404291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34040c6fc0 msgr2=0x7f34040c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.325+0000 7f34037fe640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34040c6fc0 0x7f34040c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.325+0000 7f34037fe640 1 -- 172.21.15.81:0/2287404291 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340415d140 con 0x7f33ec0052a0 2023-11-08T16:39:53.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.325+0000 7f34037fe640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 0x7f3404150920 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f33f400c760 tx=0x7f33f400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:53.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.326+0000 7f33ebfff640 1 -- 172.21.15.81:0/2287404291 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40072d0 con 0x7f33ec0052a0 2023-11-08T16:39:53.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.326+0000 7f33ebfff640 1 -- 172.21.15.81:0/2287404291 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33f4007430 con 0x7f33ec0052a0 2023-11-08T16:39:53.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.326+0000 7f340a168640 1 -- 172.21.15.81:0/2287404291 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340415d420 con 0x7f33ec0052a0 2023-11-08T16:39:53.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.326+0000 7f33ebfff640 1 -- 172.21.15.81:0/2287404291 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33f40047e0 con 0x7f33ec0052a0 2023-11-08T16:39:53.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.326+0000 7f340a168640 1 -- 172.21.15.81:0/2287404291 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34040b12b0 con 0x7f33ec0052a0 2023-11-08T16:39:53.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.327+0000 7f340a168640 1 -- 172.21.15.81:0/2287404291 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340406d0c0 con 0x7f33ec0052a0 2023-11-08T16:39:53.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.328+0000 7f33ebfff640 1 -- 172.21.15.81:0/2287404291 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f33f4004940 con 0x7f33ec0052a0 2023-11-08T16:39:53.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.329+0000 7f33ebfff640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33f006ae70 0x7f33f006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.329+0000 7f33ebfff640 1 -- 172.21.15.81:0/2287404291 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f33f4088bd0 con 0x7f33ec0052a0 2023-11-08T16:39:53.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.329+0000 7f3402ffd640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33f006ae70 0x7f33f006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:53.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.330+0000 7f3402ffd640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33f006ae70 0x7f33f006d330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f34040c6de0 tx=0x7f33f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:53.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.333+0000 7f33ebfff640 1 -- 172.21.15.81:0/2287404291 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f33f40539c0 con 0x7f33ec0052a0 2023-11-08T16:39:53.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.545+0000 7f340a168640 1 -- 172.21.15.81:0/2287404291 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3404073930 con 0x7f33ec0052a0 2023-11-08T16:39:53.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.547+0000 7f33ebfff640 1 -- 172.21.15.81:0/2287404291 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4911 (secure 0 0 0) 0x7f33f4018070 con 0x7f33ec0052a0 2023-11-08T16:39:53.549 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:53.550 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":93,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1157756119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1157756119},{"type":"v1","addr":"172.21.15.93:6833","nonce":1157756119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/4124283270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":4124283270},{"type":"v1","addr":"172.21.15.93:6835","nonce":4124283270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":7701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1607105255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1607105255},{"type":"v1","addr":"172.21.15.81:6835","nonce":1607105255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8028,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3163533823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3163533823},{"type":"v1","addr":"172.21.15.81:6837","nonce":3163533823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:51.048090+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:39:53.550 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 93 2023-11-08T16:39:53.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 -- 172.21.15.81:0/2287404291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33f006ae70 msgr2=0x7f33f006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33f006ae70 0x7f33f006d330 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f34040c6de0 tx=0x7f33f8008040 comp rx=0 tx=0).stop 2023-11-08T16:39:53.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 -- 172.21.15.81:0/2287404291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 msgr2=0x7f3404150920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 0x7f3404150920 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f33f400c760 tx=0x7f33f400cc20 comp rx=0 tx=0).stop 2023-11-08T16:39:53.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 -- 172.21.15.81:0/2287404291 shutdown_connections 2023-11-08T16:39:53.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f33f006ae70 0x7f33f006d330 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f34040c6fc0 0x7f34040c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f34040710d0 0x7f3404071530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 --2- 172.21.15.81:0/2287404291 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33ec0052a0 0x7f3404150920 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 -- 172.21.15.81:0/2287404291 >> 172.21.15.81:0/2287404291 conn(0x7f34040cbe40 msgr2=0x7f34040ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:53.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 -- 172.21.15.81:0/2287404291 shutdown_connections 2023-11-08T16:39:53.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.549+0000 7f340a168640 1 -- 172.21.15.81:0/2287404291 wait complete. 2023-11-08T16:39:53.570 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:53.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.767+0000 7fab46ef3640 1 Processor -- start 2023-11-08T16:39:53.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.768+0000 7fab46ef3640 1 -- start start 2023-11-08T16:39:53.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.769+0000 7fab46ef3640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fab400ced10 0x7fab400cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.769+0000 7fab46ef3640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab400cf6b0 con 0x7fab4015aea0 2023-11-08T16:39:53.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.769+0000 7fab46ef3640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab40070470 con 0x7fab4015aac0 2023-11-08T16:39:53.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.769+0000 7fab46ef3640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4006f650 con 0x7fab400ced10 2023-11-08T16:39:53.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab466f2640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fab4015aea0 0x7fab4015b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56548/0 (socket says 172.21.15.81:56548) 2023-11-08T16:39:53.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab466f2640 1 -- 172.21.15.81:0/3277786531 learned_addr learned my addr 172.21.15.81:0/3277786531 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:53.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463772158 0 0) 0x7fab400cf6b0 con 0x7fab4015aea0 2023-11-08T16:39:53.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fab2c003520 con 0x7fab4015aea0 2023-11-08T16:39:53.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab456f0640 1 --2- 172.21.15.81:0/3277786531 >> v2:172.21.15.93:3300/0 conn(0x7fab400ced10 0x7fab400cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:53.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3908619068 0 0) 0x7fab2c003520 con 0x7fab4015aea0 2023-11-08T16:39:53.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 >> v2:172.21.15.93:3300/0 conn(0x7fab400ced10 msgr2=0x7fab400cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:53.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab44eef640 1 --2- 172.21.15.81:0/3277786531 >> v2:172.21.15.93:3300/0 conn(0x7fab400ced10 0x7fab400cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 >> v1:172.21.15.93:6789/0 conn(0x7fab4015aac0 legacy=0x7fab400711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:53.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4006d580 con 0x7fab4015aea0 2023-11-08T16:39:53.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fab38001670 con 0x7fab4015aea0 2023-11-08T16:39:53.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.770+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 >> v1:172.21.15.81:6789/0 conn(0x7fab4015aea0 legacy=0x7fab4015b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab44eef640 1 --2- 172.21.15.81:0/3277786531 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab2c003aa0 0x7fab2c003ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab44eef640 1 --2- 172.21.15.81:0/3277786531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab2c0047c0 0x7fab2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab44eef640 1 --2- 172.21.15.81:0/3277786531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab2c0053d0 0x7fab2c00ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4006f650 con 0x7fab2c0047c0 2023-11-08T16:39:53.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c00c2e0 con 0x7fab2c0053d0 2023-11-08T16:39:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab2c00c4a0 con 0x7fab2c003aa0 2023-11-08T16:39:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab456f0640 1 --2- 172.21.15.81:0/3277786531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab2c0047c0 0x7fab2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab456f0640 1 -- 172.21.15.81:0/3277786531 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab2c003aa0 msgr2=0x7fab2c003ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab45ef1640 1 --2- 172.21.15.81:0/3277786531 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab2c003aa0 0x7fab2c003ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:53.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab456f0640 1 --2- 172.21.15.81:0/3277786531 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab2c003aa0 0x7fab2c003ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab456f0640 1 -- 172.21.15.81:0/3277786531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab2c0053d0 msgr2=0x7fab2c00ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:53.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab456f0640 1 --2- 172.21.15.81:0/3277786531 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab2c0053d0 0x7fab2c00ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab456f0640 1 -- 172.21.15.81:0/3277786531 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2c003520 con 0x7fab2c0047c0 2023-11-08T16:39:53.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab466f2640 1 -- 172.21.15.81:0/3277786531 reap_dead start 2023-11-08T16:39:53.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.771+0000 7fab45ef1640 1 --2- 172.21.15.81:0/3277786531 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab2c003aa0 0x7fab2c003ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:53.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.772+0000 7fab456f0640 1 --2- 172.21.15.81:0/3277786531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab2c0047c0 0x7fab2c004c40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fab3000a9a0 tx=0x7fab2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b947e28557b7ebd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:53.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.773+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30030400 con 0x7fab2c0047c0 2023-11-08T16:39:53.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.773+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab30030560 con 0x7fab2c0047c0 2023-11-08T16:39:53.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.773+0000 7fab44eef640 1 -- 172.21.15.81:0/3277786531 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab30030950 con 0x7fab2c0047c0 2023-11-08T16:39:53.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.773+0000 7fab46ef3640 1 -- 172.21.15.81:0/3277786531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab2c0047c0 msgr2=0x7fab2c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.773+0000 7fab46ef3640 1 --2- 172.21.15.81:0/3277786531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab2c0047c0 0x7fab2c004c40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fab3000a9a0 tx=0x7fab2c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.773+0000 7fab46ef3640 1 -- 172.21.15.81:0/3277786531 shutdown_connections 2023-11-08T16:39:53.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.773+0000 7fab46ef3640 1 --2- 172.21.15.81:0/3277786531 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab2c0047c0 0x7fab2c004c40 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.773+0000 7fab46ef3640 1 -- 172.21.15.81:0/3277786531 >> 172.21.15.81:0/3277786531 conn(0x7fab4006a4a0 msgr2=0x7fab4006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:53.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.773+0000 7fab46ef3640 1 -- 172.21.15.81:0/3277786531 shutdown_connections 2023-11-08T16:39:53.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.773+0000 7fab46ef3640 1 -- 172.21.15.81:0/3277786531 wait complete. 2023-11-08T16:39:53.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.775+0000 7fab46ef3640 1 Processor -- start 2023-11-08T16:39:53.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.775+0000 7fab46ef3640 1 -- start start 2023-11-08T16:39:53.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.776+0000 7fab46ef3640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab2c0047c0 0x7fab4015d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.776+0000 7fab46ef3640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab401518b0 0x7fab40151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.776+0000 7fab46ef3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab4014fd90 0x7fab40150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.776+0000 7fab46ef3640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab40150810 con 0x7fab4014fd90 2023-11-08T16:39:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.776+0000 7fab46ef3640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab40150af0 con 0x7fab401518b0 2023-11-08T16:39:53.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.776+0000 7fab46ef3640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab40150dd0 con 0x7fab2c0047c0 2023-11-08T16:39:53.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.776+0000 7fab466f2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab4014fd90 0x7fab40150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:53.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.776+0000 7fab466f2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab4014fd90 0x7fab40150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38110/0 (socket says 172.21.15.81:38110) 2023-11-08T16:39:53.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.776+0000 7fab466f2640 1 -- 172.21.15.81:0/3288091081 learned_addr learned my addr 172.21.15.81:0/3288091081 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:53.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab466f2640 1 -- 172.21.15.81:0/3288091081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab2c0047c0 msgr2=0x7fab4015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab466f2640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab2c0047c0 0x7fab4015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab466f2640 1 -- 172.21.15.81:0/3288091081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab401518b0 msgr2=0x7fab40151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab456f0640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab401518b0 0x7fab40151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:53.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab466f2640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab401518b0 0x7fab40151d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:53.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab466f2640 1 -- 172.21.15.81:0/3288091081 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab401510b0 con 0x7fab4014fd90 2023-11-08T16:39:53.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab466f2640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab4014fd90 0x7fab40150210 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fab400cf9f0 tx=0x7fab3800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:53.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab367fc640 1 -- 172.21.15.81:0/3288091081 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38007440 con 0x7fab4014fd90 2023-11-08T16:39:53.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab367fc640 1 -- 172.21.15.81:0/3288091081 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab380075a0 con 0x7fab4014fd90 2023-11-08T16:39:53.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab46ef3640 1 -- 172.21.15.81:0/3288091081 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab40151390 con 0x7fab4014fd90 2023-11-08T16:39:53.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab367fc640 1 -- 172.21.15.81:0/3288091081 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab38004a20 con 0x7fab4014fd90 2023-11-08T16:39:53.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.777+0000 7fab46ef3640 1 -- 172.21.15.81:0/3288091081 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4015db70 con 0x7fab4014fd90 2023-11-08T16:39:53.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.779+0000 7fab46ef3640 1 -- 172.21.15.81:0/3288091081 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab400743f0 con 0x7fab4014fd90 2023-11-08T16:39:53.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.779+0000 7fab367fc640 1 -- 172.21.15.81:0/3288091081 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fab38004b80 con 0x7fab4014fd90 2023-11-08T16:39:53.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.780+0000 7fab367fc640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fab2406aea0 0x7fab2406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:53.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.780+0000 7fab367fc640 1 -- 172.21.15.81:0/3288091081 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fab38089160 con 0x7fab4014fd90 2023-11-08T16:39:53.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.781+0000 7fab45ef1640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fab2406aea0 0x7fab2406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:53.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.781+0000 7fab45ef1640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fab2406aea0 0x7fab2406d360 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fab4014f410 tx=0x7fab3c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:53.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.785+0000 7fab367fc640 1 -- 172.21.15.81:0/3288091081 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fab38053f50 con 0x7fab4014fd90 2023-11-08T16:39:53.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.992+0000 7fab46ef3640 1 -- 172.21.15.81:0/3288091081 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab40067800 con 0x7fab4014fd90 2023-11-08T16:39:53.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.994+0000 7fab367fc640 1 -- 172.21.15.81:0/3288091081 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) v1 ==== 94+0+4911 (secure 0 0 0) 0x7fab38057570 con 0x7fab4014fd90 2023-11-08T16:39:53.997 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:53.997 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":93,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1157756119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1157756119},{"type":"v1","addr":"172.21.15.93:6833","nonce":1157756119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6130,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/4124283270","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":4124283270},{"type":"v1","addr":"172.21.15.93:6835","nonce":4124283270}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":89},{"gid":7701,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1607105255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1607105255},{"type":"v1","addr":"172.21.15.81:6835","nonce":1607105255}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":8028,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/3163533823","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3163533823},{"type":"v1","addr":"172.21.15.81:6837","nonce":3163533823}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":93,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:51.048090+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":110,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{},"failed":[],"damaged":[0],"stopped":[],"info":{},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:39:53.997 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 93 2023-11-08T16:39:53.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.996+0000 7fab46ef3640 1 -- 172.21.15.81:0/3288091081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fab2406aea0 msgr2=0x7fab2406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.996+0000 7fab46ef3640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fab2406aea0 0x7fab2406d360 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fab4014f410 tx=0x7fab3c009040 comp rx=0 tx=0).stop 2023-11-08T16:39:53.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.996+0000 7fab46ef3640 1 -- 172.21.15.81:0/3288091081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab4014fd90 msgr2=0x7fab40150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:53.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.996+0000 7fab46ef3640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab4014fd90 0x7fab40150210 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fab400cf9f0 tx=0x7fab3800c960 comp rx=0 tx=0).stop 2023-11-08T16:39:54.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.996+0000 7fab46ef3640 1 -- 172.21.15.81:0/3288091081 shutdown_connections 2023-11-08T16:39:54.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.996+0000 7fab46ef3640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fab2406aea0 0x7fab2406d360 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.996+0000 7fab46ef3640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fab4014fd90 0x7fab40150210 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.996+0000 7fab46ef3640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fab401518b0 0x7fab40151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.996+0000 7fab46ef3640 1 --2- 172.21.15.81:0/3288091081 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fab2c0047c0 0x7fab4015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.996+0000 7fab46ef3640 1 -- 172.21.15.81:0/3288091081 >> 172.21.15.81:0/3288091081 conn(0x7fab4006a4a0 msgr2=0x7fab400aeed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.997+0000 7fab46ef3640 1 -- 172.21.15.81:0/3288091081 shutdown_connections 2023-11-08T16:39:54.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:53.997+0000 7fab46ef3640 1 -- 172.21.15.81:0/3288091081 wait complete. 2023-11-08T16:39:54.017 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset session 2023-11-08T16:39:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.115+0000 7efd7700c100 1 -- start start 2023-11-08T16:39:54.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.115+0000 7efd7700c100 1 Processor -- start 2023-11-08T16:39:54.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.116+0000 7efd7700c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a666c800 0x55c7a590fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.116+0000 7efd7700c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a666dc00 0x55c7a590f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.116+0000 7efd7700c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666d400 0x55c7a590f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.116+0000 7efd7700c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a58ed180 con 0x55c7a666d400 2023-11-08T16:39:54.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.116+0000 7efd7700c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a58ecb40 con 0x55c7a666c800 2023-11-08T16:39:54.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.116+0000 7efd7700c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7a58ec000 con 0x55c7a666dc00 2023-11-08T16:39:54.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.116+0000 7efd753ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666d400 0x55c7a590f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.116+0000 7efd753ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666d400 0x55c7a590f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38130/0 (socket says 172.21.15.81:38130) 2023-11-08T16:39:54.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.116+0000 7efd753ad640 1 -- 172.21.15.81:0/4122489405 learned_addr learned my addr 172.21.15.81:0/4122489405 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.117+0000 7efd743ab640 1 --2- 172.21.15.81:0/4122489405 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a666dc00 0x55c7a590f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.117+0000 7efd74bac640 1 --2- 172.21.15.81:0/4122489405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a666c800 0x55c7a590fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.117+0000 7efd753ad640 1 -- 172.21.15.81:0/4122489405 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a666dc00 msgr2=0x55c7a590f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.117+0000 7efd753ad640 1 --2- 172.21.15.81:0/4122489405 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a666dc00 0x55c7a590f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.117+0000 7efd753ad640 1 -- 172.21.15.81:0/4122489405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a666c800 msgr2=0x55c7a590fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.117+0000 7efd753ad640 1 --2- 172.21.15.81:0/4122489405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a666c800 0x55c7a590fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.117+0000 7efd753ad640 1 -- 172.21.15.81:0/4122489405 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7a58c84e0 con 0x55c7a666d400 2023-11-08T16:39:54.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.117+0000 7efd74bac640 1 --2- 172.21.15.81:0/4122489405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a666c800 0x55c7a590fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:54.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.117+0000 7efd753ad640 1 --2- 172.21.15.81:0/4122489405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666d400 0x55c7a590f080 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x55c7a66c8ae0 tx=0x55c7a6766060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e6471d22f718a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.117+0000 7efd723a7640 1 -- 172.21.15.81:0/4122489405 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c7a58eadc0 con 0x55c7a666d400 2023-11-08T16:39:54.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.118+0000 7efd723a7640 1 -- 172.21.15.81:0/4122489405 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7a668c000 con 0x55c7a666d400 2023-11-08T16:39:54.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.118+0000 7efd723a7640 1 -- 172.21.15.81:0/4122489405 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c7a58eaf20 con 0x55c7a666d400 2023-11-08T16:39:54.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.118+0000 7efd7700c100 1 -- 172.21.15.81:0/4122489405 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7a58c8820 con 0x55c7a666d400 2023-11-08T16:39:54.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.118+0000 7efd7700c100 1 -- 172.21.15.81:0/4122489405 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c7a58c8680 con 0x55c7a666d400 2023-11-08T16:39:54.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.118+0000 7efd7700c100 4 waiting for MDS map... 2023-11-08T16:39:54.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.118+0000 7efd723a7640 1 -- 172.21.15.81:0/4122489405 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x55c7a58ce380 con 0x55c7a666d400 2023-11-08T16:39:54.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.118+0000 7efd723a7640 1 -- 172.21.15.81:0/4122489405 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 93) v1 ==== 3306+0+0 (secure 0 0 0) 0x55c7a58f4f00 con 0x55c7a666d400 2023-11-08T16:39:54.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.118+0000 7efd7700c100 4 Got MDS map 93 2023-11-08T16:39:54.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.118+0000 7efd7700c100 10 main: main 2023-11-08T16:39:54.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.118+0000 7efd7700c100 4 main: connecting to RADOS... 2023-11-08T16:39:54.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.119+0000 7efd7700c100 1 Processor -- start 2023-11-08T16:39:54.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.119+0000 7efd7700c100 1 -- start start 2023-11-08T16:39:54.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.120+0000 7efd7700c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666c400 0x55c7a5910100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.120+0000 7efd7700c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a67c6000 0x55c7a5910680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.120+0000 7efd7700c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a67c6400 0x55c7a5910c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.120+0000 7efd7700c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a58eca00 con 0x55c7a666c400 2023-11-08T16:39:54.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.120+0000 7efd7700c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a5970dc0 con 0x55c7a67c6000 2023-11-08T16:39:54.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.120+0000 7efd7700c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7a5971180 con 0x55c7a67c6400 2023-11-08T16:39:54.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.121+0000 7efd743ab640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666c400 0x55c7a5910100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.121+0000 7efd743ab640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666c400 0x55c7a5910100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38132/0 (socket says 172.21.15.81:38132) 2023-11-08T16:39:54.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.121+0000 7efd743ab640 1 -- 172.21.15.81:0/93140581 learned_addr learned my addr 172.21.15.81:0/93140581 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.121+0000 7efd74bac640 1 --2- 172.21.15.81:0/93140581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a67c6000 0x55c7a5910680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.121+0000 7efd743ab640 1 -- 172.21.15.81:0/93140581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a67c6400 msgr2=0x55c7a5910c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.121+0000 7efd743ab640 1 --2- 172.21.15.81:0/93140581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a67c6400 0x55c7a5910c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.121+0000 7efd743ab640 1 -- 172.21.15.81:0/93140581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a67c6000 msgr2=0x55c7a5910680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.121+0000 7efd743ab640 1 --2- 172.21.15.81:0/93140581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a67c6000 0x55c7a5910680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.121+0000 7efd743ab640 1 -- 172.21.15.81:0/93140581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7a58c84e0 con 0x55c7a666c400 2023-11-08T16:39:54.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.122+0000 7efd743ab640 1 --2- 172.21.15.81:0/93140581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666c400 0x55c7a5910100 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x55c7a67d4030 tx=0x55c7a6784c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed18d127bc377141 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.122+0000 7efd703a3640 1 -- 172.21.15.81:0/93140581 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c7a58eadc0 con 0x55c7a666c400 2023-11-08T16:39:54.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.122+0000 7efd703a3640 1 -- 172.21.15.81:0/93140581 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7a668cc00 con 0x55c7a666c400 2023-11-08T16:39:54.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.122+0000 7efd703a3640 1 -- 172.21.15.81:0/93140581 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c7a58eaf20 con 0x55c7a666c400 2023-11-08T16:39:54.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.122+0000 7efd7700c100 1 -- 172.21.15.81:0/93140581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666c400 msgr2=0x55c7a5910100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.122+0000 7efd7700c100 1 --2- 172.21.15.81:0/93140581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666c400 0x55c7a5910100 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x55c7a67d4030 tx=0x55c7a6784c00 comp rx=0 tx=0).stop 2023-11-08T16:39:54.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.122+0000 7efd7700c100 1 -- 172.21.15.81:0/93140581 shutdown_connections 2023-11-08T16:39:54.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.122+0000 7efd7700c100 1 --2- 172.21.15.81:0/93140581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a67c6400 0x55c7a5910c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.122+0000 7efd7700c100 1 --2- 172.21.15.81:0/93140581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a67c6000 0x55c7a5910680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.122+0000 7efd7700c100 1 --2- 172.21.15.81:0/93140581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666c400 0x55c7a5910100 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.123+0000 7efd7700c100 1 -- 172.21.15.81:0/93140581 >> 172.21.15.81:0/93140581 conn(0x55c7a666d800 msgr2=0x55c7a662d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.123+0000 7efd7700c100 1 -- 172.21.15.81:0/93140581 shutdown_connections 2023-11-08T16:39:54.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.123+0000 7efd7700c100 1 -- 172.21.15.81:0/93140581 wait complete. 2023-11-08T16:39:54.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.123+0000 7efd7700c100 1 Processor -- start 2023-11-08T16:39:54.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.123+0000 7efd7700c100 1 -- start start 2023-11-08T16:39:54.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.124+0000 7efd7700c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a666d800 0x55c7a5910100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.124+0000 7efd7700c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a67c7000 0x55c7a5910680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.124+0000 7efd7700c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a67c6c00 0x55c7a5910c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.124+0000 7efd7700c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a5970c80 con 0x55c7a67c6c00 2023-11-08T16:39:54.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.124+0000 7efd7700c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7a5970b40 con 0x55c7a67c7000 2023-11-08T16:39:54.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.124+0000 7efd7700c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c7a5970a00 con 0x55c7a666d800 2023-11-08T16:39:54.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.124+0000 7efd743ab640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a67c6c00 0x55c7a5910c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.124+0000 7efd743ab640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a67c6c00 0x55c7a5910c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38136/0 (socket says 172.21.15.81:38136) 2023-11-08T16:39:54.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.124+0000 7efd743ab640 1 -- 172.21.15.81:0/295199729 learned_addr learned my addr 172.21.15.81:0/295199729 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd743ab640 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a666d800 0x55c7a5910100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd74bac640 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a67c7000 0x55c7a5910680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd743ab640 1 -- 172.21.15.81:0/295199729 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a666d800 msgr2=0x55c7a5910100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd743ab640 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a666d800 0x55c7a5910100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd743ab640 1 -- 172.21.15.81:0/295199729 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a67c7000 msgr2=0x55c7a5910680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd743ab640 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a67c7000 0x55c7a5910680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd743ab640 1 -- 172.21.15.81:0/295199729 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7a58c81a0 con 0x55c7a67c6c00 2023-11-08T16:39:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd743ab640 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a67c6c00 0x55c7a5910c00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x55c7a67d52f0 tx=0x55c7a6625f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd6eba0640 1 -- 172.21.15.81:0/295199729 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c7a58eadc0 con 0x55c7a67c6c00 2023-11-08T16:39:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd6eba0640 1 -- 172.21.15.81:0/295199729 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c7a668d500 con 0x55c7a67c6c00 2023-11-08T16:39:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd6eba0640 1 -- 172.21.15.81:0/295199729 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c7a58eaf20 con 0x55c7a67c6c00 2023-11-08T16:39:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.125+0000 7efd7700c100 1 -- 172.21.15.81:0/295199729 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c7a58c8b60 con 0x55c7a67c6c00 2023-11-08T16:39:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.126+0000 7efd7700c100 1 -- 172.21.15.81:0/295199729 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c7a58c8d00 con 0x55c7a67c6c00 2023-11-08T16:39:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.126+0000 7efd7700c100 4 main: resolving pool 13 2023-11-08T16:39:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.127+0000 7efd6eba0640 1 -- 172.21.15.81:0/295199729 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55c7a5900840 con 0x55c7a67c6c00 2023-11-08T16:39:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.128+0000 7efd6eba0640 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55c7a5bfc000 0x55c7a5bfe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.128+0000 7efd6eba0640 1 -- 172.21.15.81:0/295199729 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x55c7a58ce540 con 0x55c7a67c6c00 2023-11-08T16:39:54.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.128+0000 7efd7700c100 4 main: creating IoCtx.. 2023-11-08T16:39:54.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.128+0000 7efd7700c100 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55c7a67c6800 0x55c7a5911180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.128+0000 7efd7700c100 1 -- 172.21.15.81:0/295199729 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:1 13.b 13:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55c7a58fd880 con 0x55c7a67c6800 2023-11-08T16:39:54.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.128+0000 7efd74bac640 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55c7a5bfc000 0x55c7a5bfe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.128+0000 7efd74bac640 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55c7a5bfc000 0x55c7a5bfe000 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55c7a666a390 tx=0x55c7a6785ec0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.129+0000 7efd743ab640 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55c7a67c6800 0x55c7a5911180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.129+0000 7efd743ab640 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55c7a67c6800 0x55c7a5911180 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd743ab640 1 -- 172.21.15.81:0/295199729 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v110'5 uv5 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x55c7a6606b40 con 0x55c7a67c6800 2023-11-08T16:39:54.145 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:39:54.145 INFO:teuthology.orchestra.run.smithi081.stdout: "0": { 2023-11-08T16:39:54.145 INFO:teuthology.orchestra.run.smithi081.stdout: "data": {}, 2023-11-08T16:39:54.146 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:39:54.146 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:54.146 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:39:54.146 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:54.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd7700c100 1 -- 172.21.15.81:0/295199729 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55c7a67c6800 msgr2=0x55c7a5911180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd7700c100 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55c7a67c6800 0x55c7a5911180 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd7700c100 1 -- 172.21.15.81:0/295199729 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55c7a5bfc000 msgr2=0x55c7a5bfe000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd7700c100 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55c7a5bfc000 0x55c7a5bfe000 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55c7a666a390 tx=0x55c7a6785ec0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd7700c100 1 -- 172.21.15.81:0/295199729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a67c6c00 msgr2=0x55c7a5910c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd7700c100 1 --2- 172.21.15.81:0/295199729 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a67c6c00 0x55c7a5910c00 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x55c7a67d52f0 tx=0x55c7a6625f20 comp rx=0 tx=0).stop 2023-11-08T16:39:54.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd74bac640 1 -- 172.21.15.81:0/295199729 reap_dead start 2023-11-08T16:39:54.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd7700c100 1 -- 172.21.15.81:0/295199729 shutdown_connections 2023-11-08T16:39:54.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd7700c100 1 -- 172.21.15.81:0/295199729 >> 172.21.15.81:0/295199729 conn(0x55c7a666c400 msgr2=0x55c7a662d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd7700c100 1 -- 172.21.15.81:0/295199729 shutdown_connections 2023-11-08T16:39:54.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.132+0000 7efd7700c100 1 -- 172.21.15.81:0/295199729 wait complete. 2023-11-08T16:39:54.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.133+0000 7efd7700c100 1 -- 172.21.15.81:0/4122489405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666d400 msgr2=0x55c7a590f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.133+0000 7efd7700c100 1 --2- 172.21.15.81:0/4122489405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666d400 0x55c7a590f080 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x55c7a66c8ae0 tx=0x55c7a6766060 comp rx=0 tx=0).stop 2023-11-08T16:39:54.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.133+0000 7efd7700c100 1 -- 172.21.15.81:0/4122489405 shutdown_connections 2023-11-08T16:39:54.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.133+0000 7efd7700c100 1 --2- 172.21.15.81:0/4122489405 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c7a666d400 0x55c7a590f080 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.133+0000 7efd7700c100 1 --2- 172.21.15.81:0/4122489405 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c7a666dc00 0x55c7a590f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.133+0000 7efd7700c100 1 --2- 172.21.15.81:0/4122489405 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c7a666c800 0x55c7a590fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.133+0000 7efd7700c100 1 -- 172.21.15.81:0/4122489405 >> 172.21.15.81:0/4122489405 conn(0x55c7a58a3000 msgr2=0x55c7a662d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.133+0000 7efd7700c100 1 -- 172.21.15.81:0/4122489405 shutdown_connections 2023-11-08T16:39:54.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.133+0000 7efd7700c100 1 -- 172.21.15.81:0/4122489405 wait complete. 2023-11-08T16:39:54.152 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'session'] in time 0:00:00.134935, result: { "0": { "data": {}, "result": 0 } } 2023-11-08T16:39:54.153 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset snap 2023-11-08T16:39:54.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.238+0000 7f7147638100 1 -- start start 2023-11-08T16:39:54.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.238+0000 7f7147638100 1 Processor -- start 2023-11-08T16:39:54.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.239+0000 7f7147638100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612240bcc00 0x561224128580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.243 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.239+0000 7f7147638100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612240bd800 0x561224128b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.239+0000 7f7147638100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612240bdc00 0x561224129080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.239+0000 7f7147638100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561224107180 con 0x5612240bd800 2023-11-08T16:39:54.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.239+0000 7f7147638100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561224106140 con 0x5612240bcc00 2023-11-08T16:39:54.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.239+0000 7f7147638100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x561224106000 con 0x5612240bdc00 2023-11-08T16:39:54.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.239+0000 7f7144d87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612240bd800 0x561224128b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.239+0000 7f7144d87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612240bd800 0x561224128b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38160/0 (socket says 172.21.15.81:38160) 2023-11-08T16:39:54.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.239+0000 7f7144d87640 1 -- 172.21.15.81:0/3254805991 learned_addr learned my addr 172.21.15.81:0/3254805991 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7145d89640 1 --2- 172.21.15.81:0/3254805991 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612240bdc00 0x561224129080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7144d87640 1 -- 172.21.15.81:0/3254805991 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612240bdc00 msgr2=0x561224129080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7144d87640 1 --2- 172.21.15.81:0/3254805991 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612240bdc00 0x561224129080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7145588640 1 --2- 172.21.15.81:0/3254805991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612240bcc00 0x561224128580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7144d87640 1 -- 172.21.15.81:0/3254805991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612240bcc00 msgr2=0x561224128580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7144d87640 1 --2- 172.21.15.81:0/3254805991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612240bcc00 0x561224128580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7144d87640 1 -- 172.21.15.81:0/3254805991 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612240e29c0 con 0x5612240bd800 2023-11-08T16:39:54.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7144d87640 1 --2- 172.21.15.81:0/3254805991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612240bd800 0x561224128b00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x561224e16cf0 tx=0x56122439a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ef88404bf928e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7142d83640 1 -- 172.21.15.81:0/3254805991 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5612241051e0 con 0x5612240bd800 2023-11-08T16:39:54.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7142d83640 1 -- 172.21.15.81:0/3254805991 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561224118480 con 0x5612240bd800 2023-11-08T16:39:54.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7142d83640 1 -- 172.21.15.81:0/3254805991 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561224105080 con 0x5612240bd800 2023-11-08T16:39:54.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7147638100 1 -- 172.21.15.81:0/3254805991 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612240e2b60 con 0x5612240bd800 2023-11-08T16:39:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7147638100 1 -- 172.21.15.81:0/3254805991 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5612240e2820 con 0x5612240bd800 2023-11-08T16:39:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.240+0000 7f7147638100 4 waiting for MDS map... 2023-11-08T16:39:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.241+0000 7f7142d83640 1 -- 172.21.15.81:0/3254805991 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x5612240e8380 con 0x5612240bd800 2023-11-08T16:39:54.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.241+0000 7f7142d83640 1 -- 172.21.15.81:0/3254805991 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 93) v1 ==== 3306+0+0 (secure 0 0 0) 0x56122410ef00 con 0x5612240bd800 2023-11-08T16:39:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.241+0000 7f7147638100 4 Got MDS map 93 2023-11-08T16:39:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.241+0000 7f7147638100 10 main: main 2023-11-08T16:39:54.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.241+0000 7f7147638100 4 main: connecting to RADOS... 2023-11-08T16:39:54.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.242+0000 7f7147638100 1 Processor -- start 2023-11-08T16:39:54.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.242+0000 7f7147638100 1 -- start start 2023-11-08T16:39:54.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.242+0000 7f7147638100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612243a9000 0x561224129600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.242+0000 7f7147638100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612243a9400 0x561224129b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.242+0000 7f7147638100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9800 0x56122412a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.242+0000 7f7147638100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561224106a00 con 0x5612243a9800 2023-11-08T16:39:54.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.242+0000 7f7147638100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122418b180 con 0x5612243a9400 2023-11-08T16:39:54.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.242+0000 7f7147638100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56122418adc0 con 0x5612243a9000 2023-11-08T16:39:54.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7144d87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9800 0x56122412a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7144d87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9800 0x56122412a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38172/0 (socket says 172.21.15.81:38172) 2023-11-08T16:39:54.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7144d87640 1 -- 172.21.15.81:0/2004471777 learned_addr learned my addr 172.21.15.81:0/2004471777 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7144d87640 1 -- 172.21.15.81:0/2004471777 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612243a9000 msgr2=0x561224129600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7144d87640 1 --2- 172.21.15.81:0/2004471777 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612243a9000 0x561224129600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7144d87640 1 -- 172.21.15.81:0/2004471777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612243a9400 msgr2=0x561224129b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7144d87640 1 --2- 172.21.15.81:0/2004471777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612243a9400 0x561224129b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7144d87640 1 -- 172.21.15.81:0/2004471777 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612240e24e0 con 0x5612243a9800 2023-11-08T16:39:54.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7144d87640 1 --2- 172.21.15.81:0/2004471777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9800 0x56122412a100 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x561224408120 tx=0x56122439a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97333a82ebc25981 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7140d7f640 1 -- 172.21.15.81:0/2004471777 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561224104dc0 con 0x5612243a9800 2023-11-08T16:39:54.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7140d7f640 1 -- 172.21.15.81:0/2004471777 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561224e8cd80 con 0x5612243a9800 2023-11-08T16:39:54.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.243+0000 7f7140d7f640 1 -- 172.21.15.81:0/2004471777 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5612241051e0 con 0x5612243a9800 2023-11-08T16:39:54.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.244+0000 7f7147638100 1 -- 172.21.15.81:0/2004471777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9800 msgr2=0x56122412a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.244+0000 7f7147638100 1 --2- 172.21.15.81:0/2004471777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9800 0x56122412a100 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x561224408120 tx=0x56122439a060 comp rx=0 tx=0).stop 2023-11-08T16:39:54.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.244+0000 7f7147638100 1 -- 172.21.15.81:0/2004471777 shutdown_connections 2023-11-08T16:39:54.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.244+0000 7f7147638100 1 --2- 172.21.15.81:0/2004471777 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9800 0x56122412a100 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.244+0000 7f7147638100 1 --2- 172.21.15.81:0/2004471777 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612243a9400 0x561224129b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.244+0000 7f7147638100 1 --2- 172.21.15.81:0/2004471777 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612243a9000 0x561224129600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.244+0000 7f7147638100 1 -- 172.21.15.81:0/2004471777 >> 172.21.15.81:0/2004471777 conn(0x5612243a8c00 msgr2=0x561224e47000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.244+0000 7f7147638100 1 -- 172.21.15.81:0/2004471777 shutdown_connections 2023-11-08T16:39:54.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.244+0000 7f7147638100 1 -- 172.21.15.81:0/2004471777 wait complete. 2023-11-08T16:39:54.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.245+0000 7f7147638100 1 Processor -- start 2023-11-08T16:39:54.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.245+0000 7f7147638100 1 -- start start 2023-11-08T16:39:54.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.245+0000 7f7147638100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9000 0x56122412b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7147638100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612243a9400 0x56122412ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7147638100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612243a9800 0x56122412a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7147638100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56122418ac80 con 0x5612243a9000 2023-11-08T16:39:54.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7147638100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561224106140 con 0x5612243a9800 2023-11-08T16:39:54.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7147638100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x561224106b40 con 0x5612243a9400 2023-11-08T16:39:54.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7145d89640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9000 0x56122412b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7145d89640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9000 0x56122412b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38178/0 (socket says 172.21.15.81:38178) 2023-11-08T16:39:54.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7145d89640 1 -- 172.21.15.81:0/815453553 learned_addr learned my addr 172.21.15.81:0/815453553 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7145d89640 1 -- 172.21.15.81:0/815453553 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612243a9400 msgr2=0x56122412ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7145d89640 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612243a9400 0x56122412ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7145d89640 1 -- 172.21.15.81:0/815453553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612243a9800 msgr2=0x56122412a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7145d89640 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612243a9800 0x56122412a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7145d89640 1 -- 172.21.15.81:0/815453553 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5612240e29c0 con 0x5612243a9000 2023-11-08T16:39:54.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.246+0000 7f7145d89640 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9000 0x56122412b180 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x561224e16420 tx=0x56122439a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.247+0000 7f713f57c640 1 -- 172.21.15.81:0/815453553 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561224104f20 con 0x5612243a9000 2023-11-08T16:39:54.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.247+0000 7f713f57c640 1 -- 172.21.15.81:0/815453553 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561224e8d800 con 0x5612243a9000 2023-11-08T16:39:54.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.247+0000 7f713f57c640 1 -- 172.21.15.81:0/815453553 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x561224104dc0 con 0x5612243a9000 2023-11-08T16:39:54.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.247+0000 7f7147638100 1 -- 172.21.15.81:0/815453553 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5612240e2b60 con 0x5612243a9000 2023-11-08T16:39:54.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.247+0000 7f7147638100 1 -- 172.21.15.81:0/815453553 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5612240e21a0 con 0x5612243a9000 2023-11-08T16:39:54.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.247+0000 7f7147638100 4 main: resolving pool 13 2023-11-08T16:39:54.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.248+0000 7f713f57c640 1 -- 172.21.15.81:0/815453553 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x56122411b080 con 0x5612243a9000 2023-11-08T16:39:54.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.249+0000 7f713f57c640 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5612243a9c00 0x56122412b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.249+0000 7f713f57c640 1 -- 172.21.15.81:0/815453553 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x5612240e8540 con 0x5612243a9000 2023-11-08T16:39:54.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.249+0000 7f7147638100 4 main: creating IoCtx.. 2023-11-08T16:39:54.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.249+0000 7f7145588640 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5612243a9c00 0x56122412b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.249+0000 7f7147638100 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56122449c000 0x56122412a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.249+0000 7f7147638100 1 -- 172.21.15.81:0/815453553 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:1 13.d 13:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x561224117880 con 0x56122449c000 2023-11-08T16:39:54.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.250+0000 7f7144d87640 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56122449c000 0x56122412a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.250+0000 7f7145588640 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5612243a9c00 0x56122412b700 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x56122419a0c0 tx=0x561224520000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.251+0000 7f7144d87640 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56122449c000 0x56122412a100 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.265 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:39:54.266 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:39:54.266 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:39:54.266 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:54.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.254+0000 7f7144d87640 1 -- 172.21.15.81:0/815453553 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v110'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x561224f5cb40 con 0x56122449c000 2023-11-08T16:39:54.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.254+0000 7f7147638100 1 -- 172.21.15.81:0/815453553 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56122449c000 msgr2=0x56122412a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.254+0000 7f7147638100 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x56122449c000 0x56122412a100 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.254+0000 7f7147638100 1 -- 172.21.15.81:0/815453553 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5612243a9c00 msgr2=0x56122412b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.254+0000 7f7147638100 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x5612243a9c00 0x56122412b700 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x56122419a0c0 tx=0x561224520000 comp rx=0 tx=0).stop 2023-11-08T16:39:54.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.254+0000 7f7147638100 1 -- 172.21.15.81:0/815453553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9000 msgr2=0x56122412b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.254+0000 7f7147638100 1 --2- 172.21.15.81:0/815453553 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612243a9000 0x56122412b180 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x561224e16420 tx=0x56122439a240 comp rx=0 tx=0).stop 2023-11-08T16:39:54.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.254+0000 7f7145588640 1 -- 172.21.15.81:0/815453553 reap_dead start 2023-11-08T16:39:54.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 -- 172.21.15.81:0/815453553 shutdown_connections 2023-11-08T16:39:54.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 -- 172.21.15.81:0/815453553 >> 172.21.15.81:0/815453553 conn(0x5612243a8c00 msgr2=0x561224e47000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 -- 172.21.15.81:0/815453553 shutdown_connections 2023-11-08T16:39:54.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 -- 172.21.15.81:0/815453553 wait complete. 2023-11-08T16:39:54.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 -- 172.21.15.81:0/3254805991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612240bd800 msgr2=0x561224128b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 --2- 172.21.15.81:0/3254805991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612240bd800 0x561224128b00 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x561224e16cf0 tx=0x56122439a480 comp rx=0 tx=0).stop 2023-11-08T16:39:54.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 -- 172.21.15.81:0/3254805991 shutdown_connections 2023-11-08T16:39:54.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 --2- 172.21.15.81:0/3254805991 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5612240bdc00 0x561224129080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 --2- 172.21.15.81:0/3254805991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5612240bd800 0x561224128b00 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 --2- 172.21.15.81:0/3254805991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5612240bcc00 0x561224128580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 -- 172.21.15.81:0/3254805991 >> 172.21.15.81:0/3254805991 conn(0x5612240bd000 msgr2=0x561224e46800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 -- 172.21.15.81:0/3254805991 shutdown_connections 2023-11-08T16:39:54.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.255+0000 7f7147638100 1 -- 172.21.15.81:0/3254805991 wait complete. 2023-11-08T16:39:54.272 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'snap'] in time 0:00:00.119670, result: { "result": 0 } 2023-11-08T16:39:54.273 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 cephfs:0 reset inode 2023-11-08T16:39:54.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.364+0000 7f657fa0c100 1 -- start start 2023-11-08T16:39:54.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.365+0000 7f657fa0c100 1 Processor -- start 2023-11-08T16:39:54.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.365+0000 7f657fa0c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44aacc00 0x555a44b18580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.365+0000 7f657fa0c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555a44aad800 0x555a44b18b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.365+0000 7f657fa0c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44aadc00 0x555a44b19080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.365+0000 7f657fa0c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a44af7180 con 0x555a44aacc00 2023-11-08T16:39:54.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.365+0000 7f657fa0c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a44af6140 con 0x555a44aadc00 2023-11-08T16:39:54.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.365+0000 7f657fa0c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a44af6000 con 0x555a44aad800 2023-11-08T16:39:54.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.366+0000 7f657d613640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44aacc00 0x555a44b18580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.366+0000 7f657d613640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44aacc00 0x555a44b18580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38206/0 (socket says 172.21.15.81:38206) 2023-11-08T16:39:54.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.366+0000 7f657d613640 1 -- 172.21.15.81:0/4113494845 learned_addr learned my addr 172.21.15.81:0/4113494845 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.366+0000 7f657d613640 1 -- 172.21.15.81:0/4113494845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555a44aad800 msgr2=0x555a44b18b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.366+0000 7f657d613640 1 --2- 172.21.15.81:0/4113494845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555a44aad800 0x555a44b18b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.366+0000 7f657d613640 1 -- 172.21.15.81:0/4113494845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44aadc00 msgr2=0x555a44b19080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.366+0000 7f657d613640 1 --2- 172.21.15.81:0/4113494845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44aadc00 0x555a44b19080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.366+0000 7f657d613640 1 -- 172.21.15.81:0/4113494845 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a44ad29c0 con 0x555a44aacc00 2023-11-08T16:39:54.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.366+0000 7f657d613640 1 --2- 172.21.15.81:0/4113494845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44aacc00 0x555a44b18580 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x555a458c4bd0 tx=0x555a44c26d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9e3f0023fb828a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.366+0000 7f657ae0e640 1 -- 172.21.15.81:0/4113494845 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a44af51e0 con 0x555a44aacc00 2023-11-08T16:39:54.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.367+0000 7f657ae0e640 1 -- 172.21.15.81:0/4113494845 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a44b08480 con 0x555a44aacc00 2023-11-08T16:39:54.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.367+0000 7f657ae0e640 1 -- 172.21.15.81:0/4113494845 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a44af5080 con 0x555a44aacc00 2023-11-08T16:39:54.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.367+0000 7f657fa0c100 1 -- 172.21.15.81:0/4113494845 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a44ad2b60 con 0x555a44aacc00 2023-11-08T16:39:54.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.367+0000 7f657fa0c100 1 -- 172.21.15.81:0/4113494845 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555a44ad2820 con 0x555a44aacc00 2023-11-08T16:39:54.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.367+0000 7f657fa0c100 4 waiting for MDS map... 2023-11-08T16:39:54.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.367+0000 7f657ae0e640 1 -- 172.21.15.81:0/4113494845 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x555a44ad8380 con 0x555a44aacc00 2023-11-08T16:39:54.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.367+0000 7f657ae0e640 1 -- 172.21.15.81:0/4113494845 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 93) v1 ==== 3306+0+0 (secure 0 0 0) 0x555a44aff200 con 0x555a44aacc00 2023-11-08T16:39:54.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.367+0000 7f657fa0c100 4 Got MDS map 93 2023-11-08T16:39:54.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.367+0000 7f657fa0c100 10 main: main 2023-11-08T16:39:54.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.367+0000 7f657fa0c100 4 main: connecting to RADOS... 2023-11-08T16:39:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.368+0000 7f657fa0c100 1 Processor -- start 2023-11-08T16:39:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.368+0000 7f657fa0c100 1 -- start start 2023-11-08T16:39:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657fa0c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555a44d95000 0x555a44b19600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657fa0c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95400 0x555a44b19b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657fa0c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44d95800 0x555a44b1a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657fa0c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a44af6a00 con 0x555a44d95400 2023-11-08T16:39:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657fa0c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a44b7b180 con 0x555a44d95800 2023-11-08T16:39:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657fa0c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a44b7adc0 con 0x555a44d95000 2023-11-08T16:39:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657d613640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95400 0x555a44b19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657d613640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95400 0x555a44b19b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38220/0 (socket says 172.21.15.81:38220) 2023-11-08T16:39:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657d613640 1 -- 172.21.15.81:0/3037821801 learned_addr learned my addr 172.21.15.81:0/3037821801 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657d613640 1 -- 172.21.15.81:0/3037821801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555a44d95000 msgr2=0x555a44b19600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657d613640 1 --2- 172.21.15.81:0/3037821801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555a44d95000 0x555a44b19600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657d613640 1 -- 172.21.15.81:0/3037821801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44d95800 msgr2=0x555a44b1a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657d613640 1 --2- 172.21.15.81:0/3037821801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44d95800 0x555a44b1a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.369+0000 7f657d613640 1 -- 172.21.15.81:0/3037821801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a44ad24e0 con 0x555a44d95400 2023-11-08T16:39:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.370+0000 7f657d613640 1 --2- 172.21.15.81:0/3037821801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95400 0x555a44b19b80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x555a44dfe0c0 tx=0x555a44c26c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dd4bcd77c4ddbe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.370+0000 7f6578e0a640 1 -- 172.21.15.81:0/3037821801 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a44af4dc0 con 0x555a44d95400 2023-11-08T16:39:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.370+0000 7f6578e0a640 1 -- 172.21.15.81:0/3037821801 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a44b8ac00 con 0x555a44d95400 2023-11-08T16:39:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.370+0000 7f6578e0a640 1 -- 172.21.15.81:0/3037821801 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a44af51e0 con 0x555a44d95400 2023-11-08T16:39:54.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.370+0000 7f657fa0c100 1 -- 172.21.15.81:0/3037821801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95400 msgr2=0x555a44b19b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.370+0000 7f657fa0c100 1 --2- 172.21.15.81:0/3037821801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95400 0x555a44b19b80 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x555a44dfe0c0 tx=0x555a44c26c60 comp rx=0 tx=0).stop 2023-11-08T16:39:54.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.370+0000 7f657fa0c100 1 -- 172.21.15.81:0/3037821801 shutdown_connections 2023-11-08T16:39:54.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.370+0000 7f657fa0c100 1 --2- 172.21.15.81:0/3037821801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44d95800 0x555a44b1a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.370+0000 7f657fa0c100 1 --2- 172.21.15.81:0/3037821801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95400 0x555a44b19b80 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.370+0000 7f657fa0c100 1 --2- 172.21.15.81:0/3037821801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555a44d95000 0x555a44b19600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.370+0000 7f657fa0c100 1 -- 172.21.15.81:0/3037821801 >> 172.21.15.81:0/3037821801 conn(0x555a44d94c00 msgr2=0x555a45837000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.371+0000 7f657fa0c100 1 -- 172.21.15.81:0/3037821801 shutdown_connections 2023-11-08T16:39:54.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.371+0000 7f657fa0c100 1 -- 172.21.15.81:0/3037821801 wait complete. 2023-11-08T16:39:54.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.371+0000 7f657fa0c100 1 Processor -- start 2023-11-08T16:39:54.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.371+0000 7f657fa0c100 1 -- start start 2023-11-08T16:39:54.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.372+0000 7f657fa0c100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44d95000 0x555a44b1b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.372+0000 7f657fa0c100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555a44d95400 0x555a44b1ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.372+0000 7f657fa0c100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95800 0x555a44b1a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.372+0000 7f657fa0c100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a44b7ac80 con 0x555a44d95800 2023-11-08T16:39:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.372+0000 7f657fa0c100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a44af6140 con 0x555a44d95000 2023-11-08T16:39:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.372+0000 7f657fa0c100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a44af6b40 con 0x555a44d95400 2023-11-08T16:39:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.372+0000 7f657ce12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95800 0x555a44b1a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.372+0000 7f657ce12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95800 0x555a44b1a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38234/0 (socket says 172.21.15.81:38234) 2023-11-08T16:39:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.372+0000 7f657ce12640 1 -- 172.21.15.81:0/1488807096 learned_addr learned my addr 172.21.15.81:0/1488807096 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f657de14640 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44d95000 0x555a44b1b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f657ce12640 1 -- 172.21.15.81:0/1488807096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555a44d95400 msgr2=0x555a44b1ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f657ce12640 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555a44d95400 0x555a44b1ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f657ce12640 1 -- 172.21.15.81:0/1488807096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44d95000 msgr2=0x555a44b1b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f657ce12640 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44d95000 0x555a44b1b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f657ce12640 1 -- 172.21.15.81:0/1488807096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a44ad29c0 con 0x555a44d95800 2023-11-08T16:39:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f657de14640 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44d95000 0x555a44b1b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f657ce12640 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95800 0x555a44b1a680 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x555a44dfeea0 tx=0x555a44dace40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f6577607640 1 -- 172.21.15.81:0/1488807096 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a44af4dc0 con 0x555a44d95800 2023-11-08T16:39:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f6577607640 1 -- 172.21.15.81:0/1488807096 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a44b8b800 con 0x555a44d95800 2023-11-08T16:39:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f6577607640 1 -- 172.21.15.81:0/1488807096 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555a44af51e0 con 0x555a44d95800 2023-11-08T16:39:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.373+0000 7f657fa0c100 1 -- 172.21.15.81:0/1488807096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555a44ad2b60 con 0x555a44d95800 2023-11-08T16:39:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.374+0000 7f657fa0c100 1 -- 172.21.15.81:0/1488807096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a44ad2820 con 0x555a44d95800 2023-11-08T16:39:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.374+0000 7f657fa0c100 4 main: resolving pool 13 2023-11-08T16:39:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.375+0000 7f6577607640 1 -- 172.21.15.81:0/1488807096 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x555a44b0a840 con 0x555a44d95800 2023-11-08T16:39:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.376+0000 7f6577607640 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x555a44e14800 0x555a44b1b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.376+0000 7f6577607640 1 -- 172.21.15.81:0/1488807096 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x555a44ad8540 con 0x555a44d95800 2023-11-08T16:39:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.376+0000 7f657fa0c100 4 main: creating IoCtx.. 2023-11-08T16:39:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.376+0000 7f657fa0c100 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x555a44e14c00 0x555a44b1a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.376+0000 7f657fa0c100 1 -- 172.21.15.81:0/1488807096 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:1 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555a44b07880 con 0x555a44e14c00 2023-11-08T16:39:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.376+0000 7f657de14640 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x555a44e14800 0x555a44b1b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.377+0000 7f657d613640 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x555a44e14c00 0x555a44b1a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.377+0000 7f657de14640 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x555a44e14800 0x555a44b1b700 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x555a44b3c7b0 tx=0x555a44b72900 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.378+0000 7f657d613640 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x555a44e14c00 0x555a44b1a100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657d613640 1 -- 172.21.15.81:0/1488807096 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v110'6 uv6 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x555a45880b40 con 0x555a44e14c00 2023-11-08T16:39:54.393 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:39:54.393 INFO:teuthology.orchestra.run.smithi081.stdout: "0": { 2023-11-08T16:39:54.394 INFO:teuthology.orchestra.run.smithi081.stdout: "data": {}, 2023-11-08T16:39:54.394 INFO:teuthology.orchestra.run.smithi081.stdout: "result": 0 2023-11-08T16:39:54.394 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:39:54.394 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:39:54.394 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:54.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 -- 172.21.15.81:0/1488807096 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x555a44e14c00 msgr2=0x555a44b1a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x555a44e14c00 0x555a44b1a100 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 -- 172.21.15.81:0/1488807096 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x555a44e14800 msgr2=0x555a44b1b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x555a44e14800 0x555a44b1b700 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x555a44b3c7b0 tx=0x555a44b72900 comp rx=0 tx=0).stop 2023-11-08T16:39:54.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 -- 172.21.15.81:0/1488807096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95800 msgr2=0x555a44b1a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 --2- 172.21.15.81:0/1488807096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44d95800 0x555a44b1a680 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x555a44dfeea0 tx=0x555a44dace40 comp rx=0 tx=0).stop 2023-11-08T16:39:54.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657ce12640 1 -- 172.21.15.81:0/1488807096 reap_dead start 2023-11-08T16:39:54.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 -- 172.21.15.81:0/1488807096 shutdown_connections 2023-11-08T16:39:54.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 -- 172.21.15.81:0/1488807096 >> 172.21.15.81:0/1488807096 conn(0x555a44d94c00 msgr2=0x555a45837000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 -- 172.21.15.81:0/1488807096 shutdown_connections 2023-11-08T16:39:54.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 -- 172.21.15.81:0/1488807096 wait complete. 2023-11-08T16:39:54.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 -- 172.21.15.81:0/4113494845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44aacc00 msgr2=0x555a44b18580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 --2- 172.21.15.81:0/4113494845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44aacc00 0x555a44b18580 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x555a458c4bd0 tx=0x555a44c26d20 comp rx=0 tx=0).stop 2023-11-08T16:39:54.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 -- 172.21.15.81:0/4113494845 shutdown_connections 2023-11-08T16:39:54.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 --2- 172.21.15.81:0/4113494845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555a44aadc00 0x555a44b19080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 --2- 172.21.15.81:0/4113494845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555a44aad800 0x555a44b18b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 --2- 172.21.15.81:0/4113494845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555a44aacc00 0x555a44b18580 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.381+0000 7f657fa0c100 1 -- 172.21.15.81:0/4113494845 >> 172.21.15.81:0/4113494845 conn(0x555a44aad000 msgr2=0x555a45836800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.382+0000 7f657fa0c100 1 -- 172.21.15.81:0/4113494845 shutdown_connections 2023-11-08T16:39:54.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.382+0000 7f657fa0c100 1 -- 172.21.15.81:0/4113494845 wait complete. 2023-11-08T16:39:54.400 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'cephfs:0', 'reset', 'inode'] in time 0:00:00.127549, result: { "0": { "data": {}, "result": 0 } } 2023-11-08T16:39:54.401 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force 2023-11-08T16:39:54.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.496+0000 7f157e613100 1 -- start start 2023-11-08T16:39:54.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.496+0000 7f157e613100 1 Processor -- start 2023-11-08T16:39:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.496+0000 7f157e613100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d46e000 0x55b92d3f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157e613100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d38dc00 0x55b92d3f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157e613100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d46e800 0x55b92d3f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157e613100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d3d7180 con 0x55b92d38dc00 2023-11-08T16:39:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157e613100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d3d6b40 con 0x55b92d46e000 2023-11-08T16:39:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157e613100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b92d3d6000 con 0x55b92d46e800 2023-11-08T16:39:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157ba12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d38dc00 0x55b92d3f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157ba12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d38dc00 0x55b92d3f9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38250/0 (socket says 172.21.15.81:38250) 2023-11-08T16:39:54.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157ba12640 1 -- 172.21.15.81:0/3316769196 learned_addr learned my addr 172.21.15.81:0/3316769196 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157ba12640 1 -- 172.21.15.81:0/3316769196 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d46e800 msgr2=0x55b92d3f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157ba12640 1 --2- 172.21.15.81:0/3316769196 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d46e800 0x55b92d3f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157ba12640 1 -- 172.21.15.81:0/3316769196 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d46e000 msgr2=0x55b92d3f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157ba12640 1 --2- 172.21.15.81:0/3316769196 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d46e000 0x55b92d3f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157ba12640 1 -- 172.21.15.81:0/3316769196 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b92d3b24e0 con 0x55b92d38dc00 2023-11-08T16:39:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.497+0000 7f157ba12640 1 --2- 172.21.15.81:0/3316769196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d38dc00 0x55b92d3f9b80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x55b92e196ba0 tx=0x55b92d66ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a94cbbc735199c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.498+0000 7f1579a0e640 1 -- 172.21.15.81:0/3316769196 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b92d3d4f20 con 0x55b92d38dc00 2023-11-08T16:39:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.498+0000 7f1579a0e640 1 -- 172.21.15.81:0/3316769196 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b92d680000 con 0x55b92d38dc00 2023-11-08T16:39:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.498+0000 7f1579a0e640 1 -- 172.21.15.81:0/3316769196 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b92d3d51e0 con 0x55b92d38dc00 2023-11-08T16:39:54.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.498+0000 7f157e613100 1 -- 172.21.15.81:0/3316769196 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b92d3b21a0 con 0x55b92d38dc00 2023-11-08T16:39:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.498+0000 7f157e613100 1 -- 172.21.15.81:0/3316769196 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b92d46c000 con 0x55b92d38dc00 2023-11-08T16:39:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.498+0000 7f157e613100 4 waiting for MDS map... 2023-11-08T16:39:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.498+0000 7f1579a0e640 1 -- 172.21.15.81:0/3316769196 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x55b92d3b8380 con 0x55b92d38dc00 2023-11-08T16:39:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.498+0000 7f1579a0e640 1 -- 172.21.15.81:0/3316769196 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 93) v1 ==== 3306+0+0 (secure 0 0 0) 0x55b92d3df200 con 0x55b92d38dc00 2023-11-08T16:39:54.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.498+0000 7f157e613100 4 Got MDS map 93 2023-11-08T16:39:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.498+0000 7f157e613100 10 main: JournalTool::main 2023-11-08T16:39:54.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.498+0000 7f157e613100 4 main: JournalTool: connecting to RADOS... 2023-11-08T16:39:54.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.499+0000 7f157e613100 1 Processor -- start 2023-11-08T16:39:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.499+0000 7f157e613100 1 -- start start 2023-11-08T16:39:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.500+0000 7f157e613100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d678c00 0x55b92d3f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.500+0000 7f157e613100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d679000 0x55b92d3fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.500+0000 7f157e613100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d679400 0x55b92d3fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.500+0000 7f157e613100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d3d6a00 con 0x55b92d679400 2023-11-08T16:39:54.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.500+0000 7f157e613100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d3d6140 con 0x55b92d679000 2023-11-08T16:39:54.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.500+0000 7f157e613100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b92d45adc0 con 0x55b92d678c00 2023-11-08T16:39:54.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.500+0000 7f157ba12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d679400 0x55b92d3fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.500+0000 7f157ba12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d679400 0x55b92d3fa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38256/0 (socket says 172.21.15.81:38256) 2023-11-08T16:39:54.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.500+0000 7f157ba12640 1 -- 172.21.15.81:0/1028495780 learned_addr learned my addr 172.21.15.81:0/1028495780 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.501+0000 7f157ca14640 1 --2- 172.21.15.81:0/1028495780 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d678c00 0x55b92d3f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.501+0000 7f157ba12640 1 -- 172.21.15.81:0/1028495780 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d678c00 msgr2=0x55b92d3f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.501+0000 7f157ba12640 1 --2- 172.21.15.81:0/1028495780 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d678c00 0x55b92d3f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.501+0000 7f157ba12640 1 -- 172.21.15.81:0/1028495780 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d679000 msgr2=0x55b92d3fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.501+0000 7f157ba12640 1 --2- 172.21.15.81:0/1028495780 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d679000 0x55b92d3fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.501+0000 7f157ba12640 1 -- 172.21.15.81:0/1028495780 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b92d46c1a0 con 0x55b92d679400 2023-11-08T16:39:54.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.501+0000 7f157ba12640 1 --2- 172.21.15.81:0/1028495780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d679400 0x55b92d3fa680 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x55b92d6ec120 tx=0x55b92e11c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d222ee929e4d499 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.501+0000 7f1577a0a640 1 -- 172.21.15.81:0/1028495780 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b92d3d5080 con 0x55b92d679400 2023-11-08T16:39:54.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.501+0000 7f1577a0a640 1 -- 172.21.15.81:0/1028495780 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b92d680f00 con 0x55b92d679400 2023-11-08T16:39:54.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.501+0000 7f1577a0a640 1 -- 172.21.15.81:0/1028495780 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b92d3d4f20 con 0x55b92d679400 2023-11-08T16:39:54.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.502+0000 7f157e613100 1 -- 172.21.15.81:0/1028495780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d679400 msgr2=0x55b92d3fa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.502+0000 7f157e613100 1 --2- 172.21.15.81:0/1028495780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d679400 0x55b92d3fa680 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x55b92d6ec120 tx=0x55b92e11c120 comp rx=0 tx=0).stop 2023-11-08T16:39:54.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.502+0000 7f157e613100 1 -- 172.21.15.81:0/1028495780 shutdown_connections 2023-11-08T16:39:54.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.502+0000 7f157e613100 1 --2- 172.21.15.81:0/1028495780 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d679400 0x55b92d3fa680 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.502+0000 7f157e613100 1 --2- 172.21.15.81:0/1028495780 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d679000 0x55b92d3fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.502+0000 7f157e613100 1 --2- 172.21.15.81:0/1028495780 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d678c00 0x55b92d3f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.502+0000 7f157e613100 1 -- 172.21.15.81:0/1028495780 >> 172.21.15.81:0/1028495780 conn(0x55b92d46e400 msgr2=0x55b92e117800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.502+0000 7f157e613100 1 -- 172.21.15.81:0/1028495780 shutdown_connections 2023-11-08T16:39:54.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.502+0000 7f157e613100 1 -- 172.21.15.81:0/1028495780 wait complete. 2023-11-08T16:39:54.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.502+0000 7f157e613100 1 Processor -- start 2023-11-08T16:39:54.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.503+0000 7f157e613100 1 -- start start 2023-11-08T16:39:54.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.503+0000 7f157e613100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d678c00 0x55b92d3f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.503+0000 7f157e613100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d679000 0x55b92d3fa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.503+0000 7f157e613100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d679400 0x55b92d3fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.503+0000 7f157e613100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d45b180 con 0x55b92d678c00 2023-11-08T16:39:54.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.503+0000 7f157e613100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d45ac80 con 0x55b92d679000 2023-11-08T16:39:54.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.503+0000 7f157e613100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b92d45ab40 con 0x55b92d679400 2023-11-08T16:39:54.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f157ca14640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d678c00 0x55b92d3f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f157ca14640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d678c00 0x55b92d3f8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38258/0 (socket says 172.21.15.81:38258) 2023-11-08T16:39:54.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f157ca14640 1 -- 172.21.15.81:0/2053673082 learned_addr learned my addr 172.21.15.81:0/2053673082 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f157ca14640 1 -- 172.21.15.81:0/2053673082 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d679400 msgr2=0x55b92d3fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f157ca14640 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d679400 0x55b92d3fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f157ca14640 1 -- 172.21.15.81:0/2053673082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d679000 msgr2=0x55b92d3fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f157ca14640 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d679000 0x55b92d3fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f157ca14640 1 -- 172.21.15.81:0/2053673082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b92d46c340 con 0x55b92d678c00 2023-11-08T16:39:54.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f157ca14640 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d678c00 0x55b92d3f8b00 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x55b92d6ed410 tx=0x55b92d698f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f1576207640 1 -- 172.21.15.81:0/2053673082 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b92d3d5080 con 0x55b92d678c00 2023-11-08T16:39:54.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f1576207640 1 -- 172.21.15.81:0/2053673082 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b92d681980 con 0x55b92d678c00 2023-11-08T16:39:54.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f1576207640 1 -- 172.21.15.81:0/2053673082 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b92d3d4f20 con 0x55b92d678c00 2023-11-08T16:39:54.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.504+0000 7f157e613100 1 -- 172.21.15.81:0/2053673082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b92d46c4e0 con 0x55b92d678c00 2023-11-08T16:39:54.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.505+0000 7f157e613100 1 -- 172.21.15.81:0/2053673082 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b92d46c340 con 0x55b92d678c00 2023-11-08T16:39:54.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.505+0000 7f157e613100 4 main: JournalTool: resolving pool 13 2023-11-08T16:39:54.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.506+0000 7f1576207640 1 -- 172.21.15.81:0/2053673082 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55b92d41cb00 con 0x55b92d678c00 2023-11-08T16:39:54.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.507+0000 7f1576207640 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b92d6f1400 0x55b92d3fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.507+0000 7f1576207640 1 -- 172.21.15.81:0/2053673082 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x55b92d3b8540 con 0x55b92d678c00 2023-11-08T16:39:54.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.507+0000 7f157e613100 4 main: JournalTool: creating IoCtx.. 2023-11-08T16:39:54.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.507+0000 7f157e613100 4 main: Executing for rank 0 2023-11-08T16:39:54.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.507+0000 7f157c213640 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b92d6f1400 0x55b92d3fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.508+0000 7f157c213640 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b92d6f1400 0x55b92d3fb700 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55b92d480b10 tx=0x55b92e11d8c0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.508+0000 7f157e613100 1 -- start start 2023-11-08T16:39:54.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.508+0000 7f157e613100 1 Processor -- start 2023-11-08T16:39:54.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.509+0000 7f157e613100 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d679c00 0x55b92d3fac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.509+0000 7f157e613100 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d6f1800 0x55b92d3fb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.509+0000 7f157e613100 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d6f1c00 0x55b92d84e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.509+0000 7f157e613100 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d3d6b40 con 0x55b92d6f1c00 2023-11-08T16:39:54.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.509+0000 7f157e613100 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b92d45ab40 con 0x55b92d6f1800 2023-11-08T16:39:54.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.509+0000 7f157e613100 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b92d45ac80 con 0x55b92d679c00 2023-11-08T16:39:54.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.509+0000 7f157ba12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d6f1c00 0x55b92d84e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.509+0000 7f157ba12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d6f1c00 0x55b92d84e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38270/0 (socket says 172.21.15.81:38270) 2023-11-08T16:39:54.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.509+0000 7f157ba12640 1 -- 172.21.15.81:0/1017471444 learned_addr learned my addr 172.21.15.81:0/1017471444 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157ca14640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d679c00 0x55b92d3fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157c213640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d6f1800 0x55b92d3fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157ba12640 1 -- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d679c00 msgr2=0x55b92d3fac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157ba12640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d679c00 0x55b92d3fac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157ba12640 1 -- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d6f1800 msgr2=0x55b92d3fb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157ba12640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d6f1800 0x55b92d3fb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157ba12640 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b92d3b2d00 con 0x55b92d6f1c00 2023-11-08T16:39:54.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157ca14640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d679c00 0x55b92d3fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:54.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157ba12640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d6f1c00 0x55b92d84e000 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x55b92d72c360 tx=0x55b92d698c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e397ce18785bd82c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f1573201640 1 -- 172.21.15.81:0/1017471444 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b92d3d51e0 con 0x55b92d6f1c00 2023-11-08T16:39:54.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f1573201640 1 -- 172.21.15.81:0/1017471444 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b92d680d80 con 0x55b92d6f1c00 2023-11-08T16:39:54.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f1573201640 1 -- 172.21.15.81:0/1017471444 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b92d3d4420 con 0x55b92d6f1c00 2023-11-08T16:39:54.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b92d3b24e0 con 0x55b92d6f1c00 2023-11-08T16:39:54.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b92d3b21a0 con 0x55b92d6f1c00 2023-11-08T16:39:54.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.510+0000 7f157e613100 4 waiting for MDS map... 2023-11-08T16:39:54.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.511+0000 7f1573201640 1 -- 172.21.15.81:0/1017471444 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x55b92d3b8a80 con 0x55b92d6f1c00 2023-11-08T16:39:54.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.511+0000 7f1573201640 1 -- 172.21.15.81:0/1017471444 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 93) v1 ==== 3306+0+0 (secure 0 0 0) 0x55b92d840f00 con 0x55b92d6f1c00 2023-11-08T16:39:54.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.511+0000 7f157e613100 4 Got MDS map 93 2023-11-08T16:39:54.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.511+0000 7f157e613100 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b92d86a000 0x55b92d84e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.511+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:1 13.f 13:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b92d868380 con 0x55b92d86a000 2023-11-08T16:39:54.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.512+0000 7f157ca14640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b92d86a000 0x55b92d84e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.513+0000 7f157ca14640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b92d86a000 0x55b92d84e580 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=67950416b22c647f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.517+0000 7f157ca14640 1 -- 172.21.15.81:0/1017471444 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v110'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55b92d66ab40 con 0x55b92d86a000 2023-11-08T16:39:54.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.517+0000 7f157e613100 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b92d86a400 0x55b92d84eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.517+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:2 13.14 13:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b92d868700 con 0x55b92d86a400 2023-11-08T16:39:54.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.517+0000 7f157c213640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b92d86a400 0x55b92d84eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.518+0000 7f157c213640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b92d86a400 0x55b92d84eb00 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=cd231f47c52c0413 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.520+0000 7f157c213640 1 -- 172.21.15.81:0/1017471444 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v110'47 uv47 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55b92d66ad80 con 0x55b92d86a400 2023-11-08T16:39:54.533 INFO:teuthology.orchestra.run.smithi081.stdout:writing EResetJournal entry 2023-11-08T16:39:54.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:3 13.14 13:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b92d868a80 con 0x55b92d86a400 2023-11-08T16:39:54.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157e613100 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b92d86a800 0x55b92d84f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:4 13.1b 13:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b92d868700 con 0x55b92d86a800 2023-11-08T16:39:54.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157e613100 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b92d86ac00 0x55b92d84f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:5 13.b 13:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b92d868a80 con 0x55b92d86ac00 2023-11-08T16:39:54.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:6 13.1f 13:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b92d868e00 con 0x55b92d86a000 2023-11-08T16:39:54.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157e613100 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b92d86b000 0x55b92d84fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:7 13.1a 13:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b92d869180 con 0x55b92d86b000 2023-11-08T16:39:54.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:8 13.1e 13:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b92d869500 con 0x55b92d86a800 2023-11-08T16:39:54.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157ba12640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b92d86b000 0x55b92d84fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157c213640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b92d86ac00 0x55b92d84f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.521+0000 7f157ca14640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b92d86a800 0x55b92d84f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.522+0000 7f157ba12640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b92d86b000 0x55b92d84fb80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=ed94505d120297b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.523+0000 7f157c213640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b92d86ac00 0x55b92d84f600 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=a67d2287a8da09a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.523+0000 7f157ca14640 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b92d86a800 0x55b92d84f080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=4834ab2dd2d31450 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.523+0000 7f157ba12640 1 -- 172.21.15.81:0/1017471444 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(7 200.00000005 [delete] v110'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b92d66afc0 con 0x55b92d86b000 2023-11-08T16:39:54.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.524+0000 7f157c213640 1 -- 172.21.15.81:0/1017471444 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(3 200.00000001 [delete] v110'48 uv47 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b92d66ad80 con 0x55b92d86a400 2023-11-08T16:39:54.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.524+0000 7f157ca14640 1 -- 172.21.15.81:0/1017471444 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(6 200.00000004 [delete] v110'9 uv8 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b92d66ab40 con 0x55b92d86a000 2023-11-08T16:39:54.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.526+0000 7f157ca14640 1 -- 172.21.15.81:0/1017471444 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(4 200.00000002 [delete] v110'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b92d66ab40 con 0x55b92d86a800 2023-11-08T16:39:54.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.526+0000 7f15719fe640 1 -- 172.21.15.81:0/1017471444 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:9 13.14 13:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b92d868380 con 0x55b92d86a400 2023-11-08T16:39:54.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.526+0000 7f157c213640 1 -- 172.21.15.81:0/1017471444 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(5 200.00000003 [delete] v110'6 uv5 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b92d66ad80 con 0x55b92d86ac00 2023-11-08T16:39:54.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.527+0000 7f157ca14640 1 -- 172.21.15.81:0/1017471444 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(8 200.00000006 [delete] v110'2 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b92d66ab40 con 0x55b92d86a800 2023-11-08T16:39:54.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.528+0000 7f157c213640 1 -- 172.21.15.81:0/1017471444 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v110'49 uv49 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x55b92d66ad80 con 0x55b92d86a400 2023-11-08T16:39:54.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 4 Successfully wrote new journal pointer and header for rank 5:0 2023-11-08T16:39:54.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b92d86a400 msgr2=0x55b92d84eb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b92d86a400 0x55b92d84eb00 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b92d86b000 msgr2=0x55b92d84fb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b92d86b000 0x55b92d84fb80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b92d86a800 msgr2=0x55b92d84f080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b92d86a800 0x55b92d84f080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b92d86ac00 msgr2=0x55b92d84f600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b92d86ac00 0x55b92d84f600 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b92d86a000 msgr2=0x55b92d84e580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b92d86a000 0x55b92d84e580 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d6f1c00 msgr2=0x55b92d84e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 --2- 172.21.15.81:0/1017471444 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d6f1c00 0x55b92d84e000 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x55b92d72c360 tx=0x55b92d698c60 comp rx=0 tx=0).stop 2023-11-08T16:39:54.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157ba12640 1 -- 172.21.15.81:0/1017471444 reap_dead start 2023-11-08T16:39:54.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157ba12640 1 -- 172.21.15.81:0/1017471444 reap_dead start 2023-11-08T16:39:54.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157ba12640 1 -- 172.21.15.81:0/1017471444 reap_dead start 2023-11-08T16:39:54.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 shutdown_connections 2023-11-08T16:39:54.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 >> 172.21.15.81:0/1017471444 conn(0x55b92d679800 msgr2=0x55b92d83c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 shutdown_connections 2023-11-08T16:39:54.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/1017471444 wait complete. 2023-11-08T16:39:54.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/2053673082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b92d6f1400 msgr2=0x55b92d3fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b92d6f1400 0x55b92d3fb700 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x55b92d480b10 tx=0x55b92e11d8c0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 -- 172.21.15.81:0/2053673082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d678c00 msgr2=0x55b92d3f8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.529+0000 7f157e613100 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d678c00 0x55b92d3f8b00 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x55b92d6ed410 tx=0x55b92d698f60 comp rx=0 tx=0).stop 2023-11-08T16:39:54.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 -- 172.21.15.81:0/2053673082 shutdown_connections 2023-11-08T16:39:54.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b92d6f1400 0x55b92d3fb700 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d679400 0x55b92d3fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d679000 0x55b92d3fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 --2- 172.21.15.81:0/2053673082 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d678c00 0x55b92d3f8b00 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 -- 172.21.15.81:0/2053673082 >> 172.21.15.81:0/2053673082 conn(0x55b92d46e400 msgr2=0x55b92e117800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 -- 172.21.15.81:0/2053673082 shutdown_connections 2023-11-08T16:39:54.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 -- 172.21.15.81:0/2053673082 wait complete. 2023-11-08T16:39:54.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 -- 172.21.15.81:0/3316769196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d38dc00 msgr2=0x55b92d3f9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 --2- 172.21.15.81:0/3316769196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d38dc00 0x55b92d3f9b80 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x55b92e196ba0 tx=0x55b92d66ec00 comp rx=0 tx=0).stop 2023-11-08T16:39:54.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 -- 172.21.15.81:0/3316769196 shutdown_connections 2023-11-08T16:39:54.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 --2- 172.21.15.81:0/3316769196 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b92d46e800 0x55b92d3f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 --2- 172.21.15.81:0/3316769196 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b92d38dc00 0x55b92d3f9b80 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 --2- 172.21.15.81:0/3316769196 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b92d46e000 0x55b92d3f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 -- 172.21.15.81:0/3316769196 >> 172.21.15.81:0/3316769196 conn(0x55b92d38d000 msgr2=0x55b92e117000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 -- 172.21.15.81:0/3316769196 shutdown_connections 2023-11-08T16:39:54.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.530+0000 7f157e613100 1 -- 172.21.15.81:0/3316769196 wait complete. 2023-11-08T16:39:54.551 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force'] in time 0:00:00.149865, result: writing EResetJournal entry 2023-11-08T16:39:54.551 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init 2023-11-08T16:39:54.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.649+0000 7f3f8b20c5c0 1 -- start start 2023-11-08T16:39:54.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.649+0000 7f3f8b20c5c0 1 Processor -- start 2023-11-08T16:39:54.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.650+0000 7f3f8b20c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f70ec00 0x55943f778580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.650+0000 7f3f8b20c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f70f800 0x55943f778b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.650+0000 7f3f8b20c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f70fc00 0x55943f779080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.650+0000 7f3f8b20c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943f757180 con 0x55943f70f800 2023-11-08T16:39:54.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.650+0000 7f3f8b20c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943f756140 con 0x55943f70fc00 2023-11-08T16:39:54.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.650+0000 7f3f8b20c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55943f756000 con 0x55943f70ec00 2023-11-08T16:39:54.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.650+0000 7f3f8866e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f70f800 0x55943f778b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.650+0000 7f3f8866e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f70f800 0x55943f778b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38274/0 (socket says 172.21.15.81:38274) 2023-11-08T16:39:54.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.650+0000 7f3f8866e640 1 -- 172.21.15.81:0/3785788707 learned_addr learned my addr 172.21.15.81:0/3785788707 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f89670640 1 --2- 172.21.15.81:0/3785788707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f70fc00 0x55943f779080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8866e640 1 -- 172.21.15.81:0/3785788707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f70ec00 msgr2=0x55943f778580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8866e640 1 --2- 172.21.15.81:0/3785788707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f70ec00 0x55943f778580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8866e640 1 -- 172.21.15.81:0/3785788707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f70fc00 msgr2=0x55943f779080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8866e640 1 --2- 172.21.15.81:0/3785788707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f70fc00 0x55943f779080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8866e640 1 -- 172.21.15.81:0/3785788707 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55943f7349c0 con 0x55943f70f800 2023-11-08T16:39:54.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f89670640 1 --2- 172.21.15.81:0/3785788707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f70fc00 0x55943f779080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:54.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8866e640 1 --2- 172.21.15.81:0/3785788707 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f70f800 0x55943f778b00 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x5594405154d0 tx=0x55943f9f8300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc024fc4877f6a57 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8666a640 1 -- 172.21.15.81:0/3785788707 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55943f7551e0 con 0x55943f70f800 2023-11-08T16:39:54.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8666a640 1 -- 172.21.15.81:0/3785788707 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55943f768480 con 0x55943f70f800 2023-11-08T16:39:54.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8666a640 1 -- 172.21.15.81:0/3785788707 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55943f755080 con 0x55943f70f800 2023-11-08T16:39:54.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3785788707 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55943f734b60 con 0x55943f70f800 2023-11-08T16:39:54.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3785788707 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55943f734820 con 0x55943f70f800 2023-11-08T16:39:54.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.651+0000 7f3f8b20c5c0 4 waiting for MDS map... 2023-11-08T16:39:54.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.652+0000 7f3f8666a640 1 -- 172.21.15.81:0/3785788707 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x55943f7cec40 con 0x55943f70f800 2023-11-08T16:39:54.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.652+0000 7f3f8666a640 1 -- 172.21.15.81:0/3785788707 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 93) v1 ==== 3306+0+0 (secure 0 0 0) 0x55943f75ef00 con 0x55943f70f800 2023-11-08T16:39:54.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.652+0000 7f3f8b20c5c0 4 Got MDS map 93 2023-11-08T16:39:54.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.652+0000 7f3f8b20c5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:39:54.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.652+0000 7f3f8b20c5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:39:54.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.653+0000 7f3f8b20c5c0 1 Processor -- start 2023-11-08T16:39:54.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.653+0000 7f3f8b20c5c0 1 -- start start 2023-11-08T16:39:54.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f8b20c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f9ff000 0x55943f779600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f8b20c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff400 0x55943f779b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f8b20c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f9ff800 0x55943f77a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f8b20c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943f756a00 con 0x55943f9ff400 2023-11-08T16:39:54.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f8b20c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943f7dd180 con 0x55943f9ff800 2023-11-08T16:39:54.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f8b20c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55943f7dcdc0 con 0x55943f9ff000 2023-11-08T16:39:54.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f88e6f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff400 0x55943f779b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f88e6f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff400 0x55943f779b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38282/0 (socket says 172.21.15.81:38282) 2023-11-08T16:39:54.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f88e6f640 1 -- 172.21.15.81:0/3538600751 learned_addr learned my addr 172.21.15.81:0/3538600751 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f89670640 1 --2- 172.21.15.81:0/3538600751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f9ff000 0x55943f779600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f88e6f640 1 -- 172.21.15.81:0/3538600751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f9ff000 msgr2=0x55943f779600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f88e6f640 1 --2- 172.21.15.81:0/3538600751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f9ff000 0x55943f779600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f88e6f640 1 -- 172.21.15.81:0/3538600751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f9ff800 msgr2=0x55943f77a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f88e6f640 1 --2- 172.21.15.81:0/3538600751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f9ff800 0x55943f77a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.654+0000 7f3f88e6f640 1 -- 172.21.15.81:0/3538600751 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55943f7344e0 con 0x55943f9ff400 2023-11-08T16:39:54.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.655+0000 7f3f88e6f640 1 --2- 172.21.15.81:0/3538600751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff400 0x55943f779b80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x55943f9f2a20 tx=0x55943fa16d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec12d3195f7ad629 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.655+0000 7f3f84666640 1 -- 172.21.15.81:0/3538600751 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55943f7551e0 con 0x55943f9ff400 2023-11-08T16:39:54.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.655+0000 7f3f84666640 1 -- 172.21.15.81:0/3538600751 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55943f768900 con 0x55943f9ff400 2023-11-08T16:39:54.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.655+0000 7f3f84666640 1 -- 172.21.15.81:0/3538600751 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55943f755080 con 0x55943f9ff400 2023-11-08T16:39:54.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.655+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3538600751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff400 msgr2=0x55943f779b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.655+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/3538600751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff400 0x55943f779b80 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x55943f9f2a20 tx=0x55943fa16d20 comp rx=0 tx=0).stop 2023-11-08T16:39:54.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.655+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3538600751 shutdown_connections 2023-11-08T16:39:54.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.655+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/3538600751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f9ff800 0x55943f77a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.655+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/3538600751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff400 0x55943f779b80 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.655+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/3538600751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f9ff000 0x55943f779600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.655+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3538600751 >> 172.21.15.81:0/3538600751 conn(0x55943f9fec00 msgr2=0x55944048b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.656+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3538600751 shutdown_connections 2023-11-08T16:39:54.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.656+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3538600751 wait complete. 2023-11-08T16:39:54.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.656+0000 7f3f8b20c5c0 1 Processor -- start 2023-11-08T16:39:54.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.656+0000 7f3f8b20c5c0 1 -- start start 2023-11-08T16:39:54.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.657+0000 7f3f8b20c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff000 0x55943f77b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.657+0000 7f3f8b20c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f9ff400 0x55943f77ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.657+0000 7f3f8b20c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f9ff800 0x55943f77a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.657+0000 7f3f8b20c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943f7dcc80 con 0x55943f9ff000 2023-11-08T16:39:54.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.657+0000 7f3f8b20c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943f756140 con 0x55943f9ff400 2023-11-08T16:39:54.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.657+0000 7f3f8b20c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55943f756b40 con 0x55943f9ff800 2023-11-08T16:39:54.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.657+0000 7f3f89670640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff000 0x55943f77b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.657+0000 7f3f89670640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff000 0x55943f77b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38298/0 (socket says 172.21.15.81:38298) 2023-11-08T16:39:54.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.657+0000 7f3f89670640 1 -- 172.21.15.81:0/1190688835 learned_addr learned my addr 172.21.15.81:0/1190688835 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f88e6f640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f9ff400 0x55943f77ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f8866e640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f9ff800 0x55943f77a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f89670640 1 -- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f9ff800 msgr2=0x55943f77a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f89670640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f9ff800 0x55943f77a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f89670640 1 -- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f9ff400 msgr2=0x55943f77ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f89670640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f9ff400 0x55943f77ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f89670640 1 -- 172.21.15.81:0/1190688835 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55943f7349c0 con 0x55943f9ff000 2023-11-08T16:39:54.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f88e6f640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f9ff400 0x55943f77ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:54.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f89670640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff000 0x55943f77b180 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x559440514a50 tx=0x55943f9f8120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f82e63640 1 -- 172.21.15.81:0/1190688835 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55943f754f20 con 0x55943f9ff000 2023-11-08T16:39:54.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f82e63640 1 -- 172.21.15.81:0/1190688835 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5594404e6300 con 0x55943f9ff000 2023-11-08T16:39:54.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55943f734b60 con 0x55943f9ff000 2023-11-08T16:39:54.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.658+0000 7f3f82e63640 1 -- 172.21.15.81:0/1190688835 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55943f7551e0 con 0x55943f9ff000 2023-11-08T16:39:54.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.659+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55943f7341a0 con 0x55943f9ff000 2023-11-08T16:39:54.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.659+0000 7f3f8b20c5c0 4 datascan.init: resolving metadata pool 13 2023-11-08T16:39:54.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.660+0000 7f3f82e63640 1 -- 172.21.15.81:0/1190688835 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55943f76b080 con 0x55943f9ff000 2023-11-08T16:39:54.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.661+0000 7f3f82e63640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55943fa76800 0x55943f77b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.661+0000 7f3f82e63640 1 -- 172.21.15.81:0/1190688835 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x55943f7cea80 con 0x55943f9ff000 2023-11-08T16:39:54.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.661+0000 7f3f8b20c5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:39:54.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.661+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55943fa76c00 0x55943f77a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.661+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55943f767880 con 0x55943fa76c00 2023-11-08T16:39:54.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.661+0000 7f3f88e6f640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55943fa76800 0x55943f77b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.662+0000 7f3f8866e640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55943fa76c00 0x55943f77a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.662+0000 7f3f88e6f640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55943fa76800 0x55943f77b700 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x55943f9f2a20 tx=0x55943fa16c00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.662+0000 7f3f8866e640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55943fa76c00 0x55943f77a100 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.663+0000 7f3f8866e640 1 -- 172.21.15.81:0/1190688835 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x5594404d4b40 con 0x55943fa76c00 2023-11-08T16:39:54.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.663+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:2 13.14 13:287034c4:::1.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55943f767c00 con 0x55943fa76c00 2023-11-08T16:39:54.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.666+0000 7f3f8866e640 1 -- 172.21.15.81:0/1190688835 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~582] v110'50 uv50 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x5594404d4b40 con 0x55943fa76c00 2023-11-08T16:39:54.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.666+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55943fa77000 0x55943f779b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.666+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:3 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55943fb74000 con 0x55943fa77000 2023-11-08T16:39:54.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.667+0000 7f3f89670640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55943fa77000 0x55943f779b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.668+0000 7f3f89670640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55943fa77000 0x55943f779b80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.668+0000 7f3f89670640 1 -- 172.21.15.81:0/1190688835 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x5594404d5200 con 0x55943fa77000 2023-11-08T16:39:54.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.669+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:4 13.1f 13:fe07bba1:::100.00000000.inode:head [writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55943fb74380 con 0x55943fa77000 2023-11-08T16:39:54.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.672+0000 7f3f89670640 1 -- 172.21.15.81:0/1190688835 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~582] v110'10 uv10 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x5594404d5200 con 0x55943fa77000 2023-11-08T16:39:54.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.672+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55943fa77400 0x55943f779600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.672+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:5 13.13 13:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55943fb74700 con 0x55943fa77400 2023-11-08T16:39:54.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.673+0000 7f3f88e6f640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55943fa77400 0x55943f779600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.674+0000 7f3f88e6f640 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55943fa77400 0x55943f779600 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.675+0000 7f3f88e6f640 1 -- 172.21.15.81:0/1190688835 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5594404d58c0 con 0x55943fa77400 2023-11-08T16:39:54.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.675+0000 7f3f8b20c5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:39:54.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.675+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:6 13.13 13:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55943fb74a80 con 0x55943fa77400 2023-11-08T16:39:54.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f88e6f640 1 -- 172.21.15.81:0/1190688835 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v110'7 uv7 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x5594404d58c0 con 0x55943fa77400 2023-11-08T16:39:54.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f8b20c5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-11-08T16:39:54.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55943fa76c00 msgr2=0x55943f77a100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55943fa76c00 0x55943f77a100 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55943fa77000 msgr2=0x55943f779b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55943fa77000 0x55943f779b80 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55943fa77400 msgr2=0x55943f779600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55943fa77400 0x55943f779600 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f88e6f640 1 -- 172.21.15.81:0/1190688835 reap_dead start 2023-11-08T16:39:54.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55943fa76800 msgr2=0x55943f77b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55943fa76800 0x55943f77b700 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x55943f9f2a20 tx=0x55943fa16c00 comp rx=0 tx=0).stop 2023-11-08T16:39:54.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff000 msgr2=0x55943f77b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.677+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff000 0x55943f77b180 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x559440514a50 tx=0x55943f9f8120 comp rx=0 tx=0).stop 2023-11-08T16:39:54.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 shutdown_connections 2023-11-08T16:39:54.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55943fa76800 0x55943f77b700 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/1190688835 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f9ff000 0x55943f77b180 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 >> 172.21.15.81:0/1190688835 conn(0x55943f9fec00 msgr2=0x55944048b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 shutdown_connections 2023-11-08T16:39:54.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/1190688835 wait complete. 2023-11-08T16:39:54.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3785788707 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f70f800 msgr2=0x55943f778b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/3785788707 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f70f800 0x55943f778b00 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x5594405154d0 tx=0x55943f9f8300 comp rx=0 tx=0).stop 2023-11-08T16:39:54.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3785788707 shutdown_connections 2023-11-08T16:39:54.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/3785788707 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55943f70fc00 0x55943f779080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/3785788707 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55943f70f800 0x55943f778b00 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 --2- 172.21.15.81:0/3785788707 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55943f70ec00 0x55943f778580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3785788707 >> 172.21.15.81:0/3785788707 conn(0x55943f70f000 msgr2=0x55944048a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.678+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3785788707 shutdown_connections 2023-11-08T16:39:54.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.679+0000 7f3f8b20c5c0 1 -- 172.21.15.81:0/3785788707 wait complete. 2023-11-08T16:39:54.690 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init'] in time 0:00:00.139175, result: 2023-11-08T16:39:54.691 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-11-08T16:39:54.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.785+0000 7fc9de64c5c0 1 -- start start 2023-11-08T16:39:54.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.785+0000 7fc9de64c5c0 1 Processor -- start 2023-11-08T16:39:54.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.786+0000 7fc9de64c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d1f342c00 0x555d1f3ac580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.786+0000 7fc9de64c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d1f343800 0x555d1f3acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.786+0000 7fc9de64c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d1f343c00 0x555d1f3ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.786+0000 7fc9de64c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1f38b180 con 0x555d1f342c00 2023-11-08T16:39:54.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.786+0000 7fc9de64c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1f38a140 con 0x555d1f343800 2023-11-08T16:39:54.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.786+0000 7fc9de64c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d1f38a000 con 0x555d1f343c00 2023-11-08T16:39:54.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.786+0000 7fc9dc655640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d1f342c00 0x555d1f3ac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.786+0000 7fc9dc655640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d1f342c00 0x555d1f3ac580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38320/0 (socket says 172.21.15.81:38320) 2023-11-08T16:39:54.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.786+0000 7fc9dc655640 1 -- 172.21.15.81:0/2717590877 learned_addr learned my addr 172.21.15.81:0/2717590877 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.786+0000 7fc9dce56640 1 --2- 172.21.15.81:0/2717590877 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d1f343c00 0x555d1f3ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.786+0000 7fc9dc655640 1 -- 172.21.15.81:0/2717590877 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d1f343c00 msgr2=0x555d1f3ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9dc655640 1 --2- 172.21.15.81:0/2717590877 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d1f343c00 0x555d1f3ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9dc655640 1 -- 172.21.15.81:0/2717590877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d1f343800 msgr2=0x555d1f3acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9dc655640 1 --2- 172.21.15.81:0/2717590877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d1f343800 0x555d1f3acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9dc655640 1 -- 172.21.15.81:0/2717590877 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d1f3689c0 con 0x555d1f342c00 2023-11-08T16:39:54.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9dc655640 1 --2- 172.21.15.81:0/2717590877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d1f342c00 0x555d1f3ac580 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x555d2014f410 tx=0x555d2010cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=242eb481dbdf1a58 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9d9e50640 1 -- 172.21.15.81:0/2717590877 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555d1f3891e0 con 0x555d1f342c00 2023-11-08T16:39:54.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9d9e50640 1 -- 172.21.15.81:0/2717590877 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d1f39c480 con 0x555d1f342c00 2023-11-08T16:39:54.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9d9e50640 1 -- 172.21.15.81:0/2717590877 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555d1f389080 con 0x555d1f342c00 2023-11-08T16:39:54.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2717590877 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d1f368b60 con 0x555d1f342c00 2023-11-08T16:39:54.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2717590877 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x555d1f368820 con 0x555d1f342c00 2023-11-08T16:39:54.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9de64c5c0 4 waiting for MDS map... 2023-11-08T16:39:54.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.787+0000 7fc9d9e50640 1 -- 172.21.15.81:0/2717590877 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x555d1f402c40 con 0x555d1f342c00 2023-11-08T16:39:54.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.788+0000 7fc9d9e50640 1 -- 172.21.15.81:0/2717590877 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 93) v1 ==== 3306+0+0 (secure 0 0 0) 0x555d1f392f00 con 0x555d1f342c00 2023-11-08T16:39:54.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.788+0000 7fc9de64c5c0 4 Got MDS map 93 2023-11-08T16:39:54.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.788+0000 7fc9de64c5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:39:54.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.788+0000 7fc9de64c5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:39:54.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.789+0000 7fc9de64c5c0 1 Processor -- start 2023-11-08T16:39:54.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.789+0000 7fc9de64c5c0 1 -- start start 2023-11-08T16:39:54.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.789+0000 7fc9de64c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd000 0x555d1f3ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.789+0000 7fc9de64c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d201fd400 0x555d1f3adb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.789+0000 7fc9de64c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d201fd800 0x555d1f3ae100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.789+0000 7fc9de64c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1f38aa00 con 0x555d201fd000 2023-11-08T16:39:54.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.789+0000 7fc9de64c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1f411180 con 0x555d201fd400 2023-11-08T16:39:54.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.789+0000 7fc9de64c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d1f410dc0 con 0x555d201fd800 2023-11-08T16:39:54.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.790+0000 7fc9dce56640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd000 0x555d1f3ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.790+0000 7fc9dce56640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd000 0x555d1f3ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38324/0 (socket says 172.21.15.81:38324) 2023-11-08T16:39:54.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.790+0000 7fc9dce56640 1 -- 172.21.15.81:0/1630371183 learned_addr learned my addr 172.21.15.81:0/1630371183 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.790+0000 7fc9dce56640 1 -- 172.21.15.81:0/1630371183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d201fd800 msgr2=0x555d1f3ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.790+0000 7fc9dce56640 1 --2- 172.21.15.81:0/1630371183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d201fd800 0x555d1f3ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.790+0000 7fc9dce56640 1 -- 172.21.15.81:0/1630371183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d201fd400 msgr2=0x555d1f3adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:54.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.790+0000 7fc9dce56640 1 --2- 172.21.15.81:0/1630371183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d201fd400 0x555d1f3adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.790+0000 7fc9dce56640 1 -- 172.21.15.81:0/1630371183 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d1f3684e0 con 0x555d201fd000 2023-11-08T16:39:54.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.790+0000 7fc9dce56640 1 --2- 172.21.15.81:0/1630371183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd000 0x555d1f3ad600 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x555d2014ea80 tx=0x555d200d20c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca3253ac33759ad server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/1630371183 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555d1f388f20 con 0x555d201fd000 2023-11-08T16:39:54.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/1630371183 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d1f422f00 con 0x555d201fd000 2023-11-08T16:39:54.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/1630371183 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555d1f3891e0 con 0x555d201fd000 2023-11-08T16:39:54.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/1630371183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd000 msgr2=0x555d1f3ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/1630371183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd000 0x555d1f3ad600 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x555d2014ea80 tx=0x555d200d20c0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/1630371183 shutdown_connections 2023-11-08T16:39:54.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/1630371183 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d201fd800 0x555d1f3ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/1630371183 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d201fd400 0x555d1f3adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/1630371183 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd000 0x555d1f3ad600 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/1630371183 >> 172.21.15.81:0/1630371183 conn(0x555d201fcc00 msgr2=0x555d200cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:54.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/1630371183 shutdown_connections 2023-11-08T16:39:54.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.791+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/1630371183 wait complete. 2023-11-08T16:39:54.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.792+0000 7fc9de64c5c0 1 Processor -- start 2023-11-08T16:39:54.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.792+0000 7fc9de64c5c0 1 -- start start 2023-11-08T16:39:54.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.793+0000 7fc9de64c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d201fd000 0x555d1f3af180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.793+0000 7fc9de64c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d201fd400 0x555d1f3aec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.793+0000 7fc9de64c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd800 0x555d1f3ae680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.793+0000 7fc9de64c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1f410c80 con 0x555d201fd800 2023-11-08T16:39:54.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.793+0000 7fc9de64c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d1f38a140 con 0x555d201fd400 2023-11-08T16:39:54.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.793+0000 7fc9de64c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x555d1f38ab40 con 0x555d201fd000 2023-11-08T16:39:54.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.793+0000 7fc9dbe54640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd800 0x555d1f3ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.793+0000 7fc9dbe54640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd800 0x555d1f3ae680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38328/0 (socket says 172.21.15.81:38328) 2023-11-08T16:39:54.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.793+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 learned_addr learned my addr 172.21.15.81:0/2588455022 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:54.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d201fd000 msgr2=0x555d1f3af180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.793+0000 7fc9dc655640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d201fd400 0x555d1f3aec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9dbe54640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d201fd000 0x555d1f3af180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d201fd400 msgr2=0x555d1f3aec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:54.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9dbe54640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d201fd400 0x555d1f3aec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:54.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d1f368d00 con 0x555d201fd800 2023-11-08T16:39:54.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9dbe54640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd800 0x555d1f3ae680 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x555d2011d5c0 tx=0x555d20218d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9d6649640 1 -- 172.21.15.81:0/2588455022 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555d1f388f20 con 0x555d201fd800 2023-11-08T16:39:54.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9d6649640 1 -- 172.21.15.81:0/2588455022 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555d1f423800 con 0x555d201fd800 2023-11-08T16:39:54.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9d6649640 1 -- 172.21.15.81:0/2588455022 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x555d1f3891e0 con 0x555d201fd800 2023-11-08T16:39:54.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x555d1f3684e0 con 0x555d201fd800 2023-11-08T16:39:54.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d1f3689c0 con 0x555d201fd800 2023-11-08T16:39:54.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.794+0000 7fc9de64c5c0 4 datascan.init: resolving metadata pool 13 2023-11-08T16:39:54.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.796+0000 7fc9d6649640 1 -- 172.21.15.81:0/2588455022 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x555d1f39e840 con 0x555d201fd800 2023-11-08T16:39:54.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9d6649640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x555d2027c800 0x555d1f3af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9d6649640 1 -- 172.21.15.81:0/2588455022 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x555d1f402a80 con 0x555d201fd800 2023-11-08T16:39:54.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9de64c5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:39:54.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9de64c5c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2023-11-08T16:39:54.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9de64c5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:39:54.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9de64c5c0 4 datascan.main: data pool 'cephfs_data_extra' has ID 15 2023-11-08T16:39:54.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9de64c5c0 4 datascan.main: opening data pool 'cephfs_data_extra' 2023-11-08T16:39:54.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x555d2027cc00 0x555d1f3ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:1 14.0 14:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39b880 con 0x555d2027cc00 2023-11-08T16:39:54.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9dce56640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x555d2027c800 0x555d1f3af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9dc655640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x555d2027cc00 0x555d1f3ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.797+0000 7fc9dce56640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x555d2027c800 0x555d1f3af700 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x555d2014ea80 tx=0x555d200c5f80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.798+0000 7fc9dc655640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x555d2027cc00 0x555d1f3ae100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.798+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(1 [pgnls start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.798+0000 7fc9d764b640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x555d1f628000 0x555d1f6aa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.798+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:2 14.20 14:04000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4000 con 0x555d1f628000 2023-11-08T16:39:54.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.799+0000 7fc9dce56640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x555d1f628000 0x555d1f6aa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.799+0000 7fc9dce56640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x555d1f628000 0x555d1f6aa000 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.799+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(2 [pgnls start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.800+0000 7fc9d7e4c640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x555d1f628400 0x555d1f6aa580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.800+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:3 14.10 14:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d1f628400 2023-11-08T16:39:54.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.800+0000 7fc9dc655640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x555d1f628400 0x555d1f6aa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.800+0000 7fc9dc655640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x555d1f628400 0x555d1f6aa580 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.801+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(3 [pgnls start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.801+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:4 14.30 14:0c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4700 con 0x555d1f628000 2023-11-08T16:39:54.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.801+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(4 [pgnls start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.801+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:5 14.8 14:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4000 con 0x555d1f628000 2023-11-08T16:39:54.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.801+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(5 [pgnls start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.801+0000 7fc9d764b640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x555d1f628800 0x555d1f6aab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.801+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:6 14.28 14:14000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4a80 con 0x555d1f628800 2023-11-08T16:39:54.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.802+0000 7fc9dbe54640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x555d1f628800 0x555d1f6aab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.802+0000 7fc9dbe54640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x555d1f628800 0x555d1f6aab00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.803+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(6 [pgnls start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.803+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:7 14.18 14:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4700 con 0x555d2027cc00 2023-11-08T16:39:54.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.803+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(7 [pgnls start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.803+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:8 14.38 14:1c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d1f628400 2023-11-08T16:39:54.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.803+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(8 [pgnls start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.803+0000 7fc9d7e4c640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x555d1f629000 0x555d1f6ab600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.803+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:9 14.4 14:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4e00 con 0x555d1f629000 2023-11-08T16:39:54.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.804+0000 7fc9dce56640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x555d1f629000 0x555d1f6ab600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.805+0000 7fc9dce56640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x555d1f629000 0x555d1f6ab600 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.806+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(9 [pgnls start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:54.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.806+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:10 14.24 14:24000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4700 con 0x555d1f628400 2023-11-08T16:39:54.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.806+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(10 [pgnls start_epoch 110 out=49b] v106'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.806+0000 7fc9d7e4c640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x555d1f629400 0x555d1f6abb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.806+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:11 14.14 14:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5180 con 0x555d1f629400 2023-11-08T16:39:54.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.807+0000 7fc9dc655640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x555d1f629400 0x555d1f6abb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.808+0000 7fc9dc655640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x555d1f629400 0x555d1f6abb80 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.808+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(11 [pgnls start_epoch 110 out=49b] v106'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.808+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:12 14.34 14:2c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39b880 con 0x555d2027cc00 2023-11-08T16:39:54.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.809+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(12 [pgnls start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.809+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:13 14.c 14:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5500 con 0x555d1f628000 2023-11-08T16:39:54.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.809+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(13 [pgnls start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.809+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:14 14.2c 14:34000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4000 con 0x555d1f628400 2023-11-08T16:39:54.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.810+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(14 [pgnls start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.810+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:15 14.1c 14:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d1f628000 2023-11-08T16:39:54.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.810+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(15 [pgnls start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.810+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:16 14.3c 14:3c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4e00 con 0x555d1f628400 2023-11-08T16:39:54.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.811+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(16 [pgnls start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.811+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:17 14.2 14:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4700 con 0x555d1f629000 2023-11-08T16:39:54.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.811+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(17 [pgnls start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:54.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.811+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:18 14.22 14:44000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5500 con 0x555d1f629400 2023-11-08T16:39:54.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.812+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(18 [pgnls start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.812+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:19 14.12 14:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5180 con 0x555d2027cc00 2023-11-08T16:39:54.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.813+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(19 [pgnls start_epoch 110 out=49b] v106'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.813+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:20 14.32 14:4c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39b880 con 0x555d1f629400 2023-11-08T16:39:54.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.814+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(20 [pgnls start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.814+0000 7fc9d7e4c640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x555d1f629800 0x555d1f6ac100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.814+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:21 14.a 14:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5880 con 0x555d1f629800 2023-11-08T16:39:54.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.815+0000 7fc9dbe54640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x555d1f629800 0x555d1f6ac100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.816+0000 7fc9dbe54640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x555d1f629800 0x555d1f6ac100 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.816+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(21 [pgnls start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.816+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:22 14.2a 14:54000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4a80 con 0x555d1f629400 2023-11-08T16:39:54.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.817+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(22 [pgnls start_epoch 110 out=81b] v106'10 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.818+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:23 14.2a 14:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39bc00 con 0x555d1f629400 2023-11-08T16:39:54.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.818+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(23 100000001f6.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.819+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628278.0 2023-11-08T16:39:54.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.819+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:24 14.2a 14:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5500 con 0x555d1f629400 2023-11-08T16:39:54.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.823+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(24 100000001f6.00000000 [call] v110'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.823+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:25 14.1a 14:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5180 con 0x555d1f628000 2023-11-08T16:39:54.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.823+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(25 [pgnls start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.823+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:26 14.3a 14:5c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5880 con 0x555d1f628000 2023-11-08T16:39:54.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.824+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(26 [pgnls start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.824+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:27 14.6 14:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d1f628000 2023-11-08T16:39:54.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.824+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(27 [pgnls start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.824+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:28 14.26 14:64000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4000 con 0x555d1f629400 2023-11-08T16:39:54.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.825+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(28 [pgnls start_epoch 110 out=49b] v106'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.825+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:29 14.16 14:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4700 con 0x555d1f628800 2023-11-08T16:39:54.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.826+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(29 [pgnls start_epoch 110 out=49b] v106'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.826+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:30 14.36 14:6c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4e00 con 0x555d2027cc00 2023-11-08T16:39:54.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.827+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 5 ==== osd_op_reply(30 [pgnls start_epoch 110 out=101b] v106'17 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.827+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:31 14.36 14:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5c00 con 0x555d2027cc00 2023-11-08T16:39:54.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.827+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 6 ==== osd_op_reply(31 100000001f9.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.827+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628281.0 2023-11-08T16:39:54.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.827+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:32 14.36 14:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4a80 con 0x555d2027cc00 2023-11-08T16:39:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.829+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 7 ==== osd_op_reply(32 100000001f9.00000000 [call] v110'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.829+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:33 14.36 14:6fe38dc2::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5500 con 0x555d2027cc00 2023-11-08T16:39:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.830+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 8 ==== osd_op_reply(33 [pgnls start_epoch 110 out=81b] v110'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.830+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:34 14.36 14:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4000 con 0x555d2027cc00 2023-11-08T16:39:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.830+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 9 ==== osd_op_reply(34 100000001f7.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.830+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628279.0 2023-11-08T16:39:54.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.830+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:35 14.36 14:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4e00 con 0x555d2027cc00 2023-11-08T16:39:54.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.832+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 10 ==== osd_op_reply(35 100000001f7.00000000 [call] v110'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.832+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:36 14.e 14:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39bc00 con 0x555d1f629400 2023-11-08T16:39:54.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.833+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(36 [pgnls start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.833+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:37 14.2e 14:74000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5180 con 0x555d1f628800 2023-11-08T16:39:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.833+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(37 [pgnls start_epoch 110 out=49b] v106'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.833+0000 7fc9d764b640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x555d1f628c00 0x555d1f6ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.834+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:38 14.1e 14:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39b880 con 0x555d1f628c00 2023-11-08T16:39:54.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.834+0000 7fc9dce56640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x555d1f628c00 0x555d1f6ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:54.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.835+0000 7fc9dce56640 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x555d1f628c00 0x555d1f6ab080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:54.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.836+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(38 [pgnls start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628c00 2023-11-08T16:39:54.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.836+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:39 14.3e 14:7c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5880 con 0x555d1f629800 2023-11-08T16:39:54.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.837+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(39 [pgnls start_epoch 110 out=49b] v106'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.837+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:40 14.1 14:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4700 con 0x555d1f628800 2023-11-08T16:39:54.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.838+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(40 [pgnls start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.838+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:41 14.21 14:84000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea000 con 0x555d1f629800 2023-11-08T16:39:54.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.839+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(41 [pgnls start_epoch 110 out=49b] v106'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.839+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:42 14.11 14:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5180 con 0x555d1f629400 2023-11-08T16:39:54.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.840+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(42 [pgnls start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.840+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:43 14.31 14:8c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea380 con 0x555d1f628400 2023-11-08T16:39:54.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.840+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(43 [pgnls start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.841+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:44 14.9 14:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5880 con 0x555d1f629800 2023-11-08T16:39:54.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.841+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(44 [pgnls start_epoch 110 out=49b] v106'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.841+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:45 14.29 14:94000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea700 con 0x555d1f629000 2023-11-08T16:39:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.842+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(45 [pgnls start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.842+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:46 14.19 14:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5c00 con 0x555d1f629000 2023-11-08T16:39:54.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.844+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(46 [pgnls start_epoch 110 out=81b] v106'11 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:54.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.844+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:47 14.19 14:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4a80 con 0x555d1f629000 2023-11-08T16:39:54.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.845+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(47 100000001f8.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:54.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.845+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628280.0 2023-11-08T16:39:54.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.845+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:48 14.19 14:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5500 con 0x555d1f629000 2023-11-08T16:39:54.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.848+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(48 100000001f8.00000000 [call] v110'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:54.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.848+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:49 14.39 14:9c000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4000 con 0x555d1f628c00 2023-11-08T16:39:54.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.849+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(49 [pgnls start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628c00 2023-11-08T16:39:54.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.849+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:50 14.5 14:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea700 con 0x555d1f628000 2023-11-08T16:39:54.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.850+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(50 [pgnls start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.850+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:51 14.25 14:a4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eaa80 con 0x555d2027cc00 2023-11-08T16:39:54.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.851+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 11 ==== osd_op_reply(51 [pgnls start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.851+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:52 14.15 14:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea380 con 0x555d1f628c00 2023-11-08T16:39:54.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.852+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(52 [pgnls start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628c00 2023-11-08T16:39:54.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.852+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:53 14.35 14:ac000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eae00 con 0x555d1f629400 2023-11-08T16:39:54.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.853+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(53 [pgnls start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.853+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:54 14.d 14:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb500 con 0x555d1f629000 2023-11-08T16:39:54.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.854+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(54 [pgnls start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:54.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.854+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:55 14.2d 14:b4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb180 con 0x555d1f629400 2023-11-08T16:39:54.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.855+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(55 [pgnls start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.855+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:56 14.1d 14:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d1f628000 2023-11-08T16:39:54.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.856+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(56 [pgnls start_epoch 110 out=49b] v106'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.856+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:57 14.3d 14:bc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea700 con 0x555d1f628800 2023-11-08T16:39:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.857+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(57 [pgnls start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.857+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:58 14.3 14:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5c00 con 0x555d1f628000 2023-11-08T16:39:54.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.858+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 11 ==== osd_op_reply(58 [pgnls start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.858+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:59 14.23 14:c4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea380 con 0x555d1f628c00 2023-11-08T16:39:54.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.859+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(59 [pgnls start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628c00 2023-11-08T16:39:54.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.859+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:60 14.13 14:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4a80 con 0x555d2027cc00 2023-11-08T16:39:54.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.859+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 12 ==== osd_op_reply(60 [pgnls start_epoch 110 out=81b] v106'16 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.860+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:61 14.13 14:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4e00 con 0x555d2027cc00 2023-11-08T16:39:54.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.860+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 13 ==== osd_op_reply(61 100000001fb.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.860+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628283.0 2023-11-08T16:39:54.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.860+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:62 14.13 14:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5180 con 0x555d2027cc00 2023-11-08T16:39:54.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.864+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 14 ==== osd_op_reply(62 100000001fb.00000000 [call] v110'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.864+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:63 14.33 14:cc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39bc00 con 0x555d1f628400 2023-11-08T16:39:54.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.865+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(63 [pgnls start_epoch 110 out=49b] v106'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.865+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:64 14.b 14:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39b880 con 0x555d1f628000 2023-11-08T16:39:54.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.866+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 12 ==== osd_op_reply(64 [pgnls start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.866+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:65 14.2b 14:d4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb500 con 0x555d1f629400 2023-11-08T16:39:54.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.867+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(65 [pgnls start_epoch 110 out=49b] v106'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.867+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:66 14.1b 14:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eaa80 con 0x555d1f628c00 2023-11-08T16:39:54.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.868+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(66 [pgnls start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628c00 2023-11-08T16:39:54.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.868+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:67 14.3b 14:dc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb880 con 0x555d1f629400 2023-11-08T16:39:54.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.869+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 13 ==== osd_op_reply(67 [pgnls start_epoch 110 out=49b] v106'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.869+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:68 14.7 14:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eae00 con 0x555d1f629800 2023-11-08T16:39:54.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.871+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(68 [pgnls start_epoch 110 out=81b] v106'6 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.871+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:69 14.7 14:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb180 con 0x555d1f629800 2023-11-08T16:39:54.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.872+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(69 10000000001.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.872+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511627777.0 2023-11-08T16:39:54.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.872+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:70 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb500 con 0x555d1f629800 2023-11-08T16:39:54.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.876+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(70 10000000001.00000000 [call] v110'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.876+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:71 14.27 14:e4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea000 con 0x555d1f629800 2023-11-08T16:39:54.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.877+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(71 [pgnls start_epoch 110 out=49b] v106'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.877+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:72 14.17 14:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4a80 con 0x555d1f628400 2023-11-08T16:39:54.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.878+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(72 [pgnls start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.878+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:73 14.37 14:ec000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5500 con 0x555d1f629400 2023-11-08T16:39:54.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.879+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 14 ==== osd_op_reply(73 [pgnls start_epoch 110 out=49b] v106'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.879+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:74 14.f 14:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4e00 con 0x555d1f628c00 2023-11-08T16:39:54.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.881+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(74 [pgnls start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628c00 2023-11-08T16:39:54.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.881+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:75 14.2f 14:f4000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4000 con 0x555d1f628000 2023-11-08T16:39:54.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.882+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 13 ==== osd_op_reply(75 [pgnls start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.882+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:76 14.1f 14:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5180 con 0x555d1f628000 2023-11-08T16:39:54.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.883+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 14 ==== osd_op_reply(76 [pgnls start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.883+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:77 14.3f 14:fc000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d1f628800 2023-11-08T16:39:54.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.884+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(77 [pgnls start_epoch 110 out=81b] v106'9 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.884+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:78 14.3f 14:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea700 con 0x555d1f628800 2023-11-08T16:39:54.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.885+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(78 100000001fa.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.885+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628282.0 2023-11-08T16:39:54.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.885+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:79 14.3f 14:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ebc00 con 0x555d1f628800 2023-11-08T16:39:54.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.889+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(79 100000001fa.00000000 [call] v110'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.889+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:80 15.0 15:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d2027cc00 2023-11-08T16:39:54.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.890+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 15 ==== osd_op_reply(80 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.890+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:81 15.10 15:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5c00 con 0x555d1f629400 2023-11-08T16:39:54.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.891+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 15 ==== osd_op_reply(81 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.891+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:82 15.8 15:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4700 con 0x555d2027cc00 2023-11-08T16:39:54.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.892+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 16 ==== osd_op_reply(82 [pgnls start_epoch 110 out=81b] v106'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.892+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:83 15.8 15:1247e94b:::100000001f9.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eae00 con 0x555d2027cc00 2023-11-08T16:39:54.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.893+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 17 ==== osd_op_reply(83 100000001f9.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.893+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628281.1 2023-11-08T16:39:54.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.893+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:84 14.36 14:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb180 con 0x555d2027cc00 2023-11-08T16:39:54.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.897+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 18 ==== osd_op_reply(84 100000001f9.00000000 [call,setxattr (8)] v110'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.897+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:85 15.18 15:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb500 con 0x555d1f628800 2023-11-08T16:39:54.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.898+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(85 [pgnls start_epoch 110 out=81b] v106'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.898+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:86 15.18 15:1cd44203:::100000001fb.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea000 con 0x555d1f628800 2023-11-08T16:39:54.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.899+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(86 100000001fb.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.899+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628283.2 2023-11-08T16:39:54.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.899+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:87 14.13 14:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea380 con 0x555d2027cc00 2023-11-08T16:39:54.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.903+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 19 ==== osd_op_reply(87 100000001fb.00000000 [call,setxattr (8)] v110'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.903+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:88 15.4 15:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eaa80 con 0x555d1f628400 2023-11-08T16:39:54.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.904+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(88 [pgnls start_epoch 110 out=81b] v106'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.904+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:89 15.4 15:24ae92ec:::100000001fb.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39b880 con 0x555d1f628400 2023-11-08T16:39:54.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.905+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(89 100000001fb.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.905+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628283.1 2023-11-08T16:39:54.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.905+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:90 14.13 14:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb880 con 0x555d2027cc00 2023-11-08T16:39:54.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.909+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 20 ==== osd_op_reply(90 100000001fb.00000000 [call,setxattr (8)] v110'19 uv19 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.909+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:91 15.14 15:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eae00 con 0x555d1f629000 2023-11-08T16:39:54.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.910+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(91 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:54.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.910+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:92 15.c 15:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5880 con 0x555d1f629800 2023-11-08T16:39:54.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.911+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 9 ==== osd_op_reply(92 [pgnls start_epoch 110 out=81b] v106'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.911+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:93 15.c 15:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb180 con 0x555d1f629800 2023-11-08T16:39:54.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.912+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 10 ==== osd_op_reply(93 100000001fc.00000000 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.913+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628284.0 2023-11-08T16:39:54.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.913+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:94 14.c 14:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea380 con 0x555d1f628000 2023-11-08T16:39:54.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.917+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 15 ==== osd_op_reply(94 100000001fc.00000000 [call,setxattr (8)] v110'7 uv7 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.917+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:95 15.1c 15:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4a80 con 0x555d1f628400 2023-11-08T16:39:54.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.918+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(95 [pgnls start_epoch 110 out=101b] v106'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.918+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:96 15.1c 15:398da958:::100000001f7.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5500 con 0x555d1f628400 2023-11-08T16:39:54.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.919+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 12 ==== osd_op_reply(96 100000001f7.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.919+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628279.2 2023-11-08T16:39:54.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.919+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:97 14.36 14:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d2027cc00 2023-11-08T16:39:54.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.923+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 21 ==== osd_op_reply(97 100000001f7.00000000 [call,setxattr (8)] v110'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.923+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:98 15.1c 15:3c0114ac::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5c00 con 0x555d1f628400 2023-11-08T16:39:54.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.924+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 13 ==== osd_op_reply(98 [pgnls start_epoch 110 out=101b] v106'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.924+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:99 15.1c 15:3c0114ac:::100000001fa.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4700 con 0x555d1f628400 2023-11-08T16:39:54.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.925+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 14 ==== osd_op_reply(99 100000001fa.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.925+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628282.4 2023-11-08T16:39:54.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.925+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:100 14.3f 14:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea700 con 0x555d1f628800 2023-11-08T16:39:54.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.928+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(100 100000001fa.00000000 [call,setxattr (8)] v110'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.928+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:101 15.1c 15:3d6db04b::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ebc00 con 0x555d1f628400 2023-11-08T16:39:54.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.929+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 15 ==== osd_op_reply(101 [pgnls start_epoch 110 out=81b] v106'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.929+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:102 15.1c 15:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb500 con 0x555d1f628400 2023-11-08T16:39:54.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.930+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 16 ==== osd_op_reply(102 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:54.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.930+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628278.1 2023-11-08T16:39:54.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.930+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:103 14.2a 14:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea000 con 0x555d1f629400 2023-11-08T16:39:54.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.934+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 16 ==== osd_op_reply(103 100000001f6.00000000 [call,setxattr (8)] v110'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.935+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:104 15.2 15:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb180 con 0x555d2027cc00 2023-11-08T16:39:54.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.936+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 22 ==== osd_op_reply(104 [pgnls start_epoch 110 out=101b] v106'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.936+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:105 15.2 15:40d56c48:::100000001fc.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5880 con 0x555d2027cc00 2023-11-08T16:39:54.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.937+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 23 ==== osd_op_reply(105 100000001fc.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.937+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628284.1 2023-11-08T16:39:54.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.937+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:106 14.c 14:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eaa80 con 0x555d1f628000 2023-11-08T16:39:54.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.940+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 16 ==== osd_op_reply(106 100000001fc.00000000 [call,setxattr (8)] v110'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.940+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:107 15.2 15:46f89f8a::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb880 con 0x555d2027cc00 2023-11-08T16:39:54.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.941+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 24 ==== osd_op_reply(107 [pgnls start_epoch 110 out=81b] v106'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.941+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:108 15.2 15:46f89f8a:::100000001f9.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ebc00 con 0x555d2027cc00 2023-11-08T16:39:54.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.942+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 25 ==== osd_op_reply(108 100000001f9.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.942+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628281.5 2023-11-08T16:39:54.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.942+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:109 14.36 14:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb500 con 0x555d2027cc00 2023-11-08T16:39:54.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.946+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 26 ==== osd_op_reply(109 100000001f9.00000000 [call,setxattr (8)] v110'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.946+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:110 15.12 15:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4a80 con 0x555d1f628800 2023-11-08T16:39:54.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.947+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(110 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.947+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:111 15.a 15:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4e00 con 0x555d2027cc00 2023-11-08T16:39:54.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.948+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 27 ==== osd_op_reply(111 [pgnls start_epoch 110 out=101b] v106'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.948+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:112 15.a 15:544c5ef0:::100000001fc.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5500 con 0x555d2027cc00 2023-11-08T16:39:54.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.949+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 28 ==== osd_op_reply(112 100000001fc.00000003 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.949+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628284.3 2023-11-08T16:39:54.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.949+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:113 14.c 14:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d1f628000 2023-11-08T16:39:54.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.952+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 17 ==== osd_op_reply(113 100000001fc.00000000 [call,setxattr (8)] v110'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.952+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:114 15.a 15:56c65d28::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5c00 con 0x555d2027cc00 2023-11-08T16:39:54.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.953+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 29 ==== osd_op_reply(114 [pgnls start_epoch 110 out=101b] v106'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.953+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:115 15.a 15:56c65d28:::100000001fa.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4700 con 0x555d2027cc00 2023-11-08T16:39:54.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.954+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 30 ==== osd_op_reply(115 100000001fa.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.954+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628282.6 2023-11-08T16:39:54.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.954+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:116 14.3f 14:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39b880 con 0x555d1f628800 2023-11-08T16:39:54.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.958+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(116 100000001fa.00000000 [call,setxattr (8)] v110'12 uv12 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:54.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.958+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:117 15.a 15:57fe7535::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152000 con 0x555d2027cc00 2023-11-08T16:39:54.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.959+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 31 ==== osd_op_reply(117 [pgnls start_epoch 110 out=81b] v106'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.959+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:118 15.a 15:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152380 con 0x555d2027cc00 2023-11-08T16:39:54.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.960+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 32 ==== osd_op_reply(118 100000001f6.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.960+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628278.0 2023-11-08T16:39:54.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.960+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:119 14.2a 14:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152700 con 0x555d1f629400 2023-11-08T16:39:54.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.964+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 17 ==== osd_op_reply(119 100000001f6.00000000 [call,setxattr (8)] v110'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:54.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.964+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:120 15.1a 15:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152a80 con 0x555d1f629800 2023-11-08T16:39:54.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.964+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 11 ==== osd_op_reply(120 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.964+0000 7fc9d7e4c640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:121 15.6 15:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4000 con 0x555d1f628000 2023-11-08T16:39:54.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.965+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 18 ==== osd_op_reply(121 [pgnls start_epoch 110 out=81b] v106'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.965+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:122 15.6 15:613bf181:::100000001f7.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152e00 con 0x555d1f628000 2023-11-08T16:39:54.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.966+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 19 ==== osd_op_reply(122 100000001f7.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.966+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628279.3 2023-11-08T16:39:54.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.967+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:123 14.36 14:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153180 con 0x555d2027cc00 2023-11-08T16:39:54.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.970+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 33 ==== osd_op_reply(123 100000001f7.00000000 [call,setxattr (8)] v110'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.970+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:124 15.16 15:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153500 con 0x555d1f628000 2023-11-08T16:39:54.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.971+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 20 ==== osd_op_reply(124 [pgnls start_epoch 110 out=101b] v106'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.971+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:125 15.16 15:6df493a7:::100000001f9.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153880 con 0x555d1f628000 2023-11-08T16:39:54.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.972+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 21 ==== osd_op_reply(125 100000001f9.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.972+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628281.6 2023-11-08T16:39:54.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.972+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:126 14.36 14:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153c00 con 0x555d2027cc00 2023-11-08T16:39:54.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.976+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 34 ==== osd_op_reply(126 100000001f9.00000000 [call,setxattr (8)] v110'24 uv24 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.976+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:127 15.16 15:6ee5e2c4::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d1f628000 2023-11-08T16:39:54.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.977+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 22 ==== osd_op_reply(127 [pgnls start_epoch 110 out=101b] v106'5 uv5 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.977+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:128 15.16 15:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4000 con 0x555d1f628000 2023-11-08T16:39:54.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.978+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 23 ==== osd_op_reply(128 100000001f9.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.978+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628281.0 2023-11-08T16:39:54.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.978+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:129 14.36 14:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5880 con 0x555d2027cc00 2023-11-08T16:39:54.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.981+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 35 ==== osd_op_reply(129 100000001f9.00000000 [call,setxattr (8)] v110'25 uv25 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.981+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:130 15.16 15:6fe38dc2::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4e00 con 0x555d1f628000 2023-11-08T16:39:54.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.982+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 24 ==== osd_op_reply(130 [pgnls start_epoch 110 out=81b] v106'5 uv5 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.982+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:131 15.16 15:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5500 con 0x555d1f628000 2023-11-08T16:39:54.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.983+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 25 ==== osd_op_reply(131 100000001f7.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:54.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.983+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628279.0 2023-11-08T16:39:54.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.983+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:132 14.36 14:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eae00 con 0x555d2027cc00 2023-11-08T16:39:54.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.987+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 36 ==== osd_op_reply(132 100000001f7.00000000 [call,setxattr (8)] v110'26 uv26 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.987+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:133 15.e 15:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea380 con 0x555d2027cc00 2023-11-08T16:39:54.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.988+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 37 ==== osd_op_reply(133 [pgnls start_epoch 110 out=101b] v106'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.988+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:134 15.e 15:72395991:::10000000001.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eaa80 con 0x555d2027cc00 2023-11-08T16:39:54.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.989+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 38 ==== osd_op_reply(134 10000000001.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.989+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511627777.3 2023-11-08T16:39:54.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.989+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:135 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea000 con 0x555d1f629800 2023-11-08T16:39:54.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.993+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 12 ==== osd_op_reply(135 10000000001.00000000 [call,setxattr (8)] v110'8 uv8 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:54.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.993+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:136 15.e 15:728aa123::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb180 con 0x555d2027cc00 2023-11-08T16:39:54.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.993+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 39 ==== osd_op_reply(136 [pgnls start_epoch 110 out=101b] v106'3 uv3 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.994+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:137 15.e 15:728aa123:::100000001f6.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb880 con 0x555d2027cc00 2023-11-08T16:39:54.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.994+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 40 ==== osd_op_reply(137 100000001f6.00000003 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:54.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.994+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628278.3 2023-11-08T16:39:54.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.995+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:138 14.2a 14:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ebc00 con 0x555d1f629400 2023-11-08T16:39:55.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.999+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 18 ==== osd_op_reply(138 100000001f6.00000000 [call,setxattr (8)] v110'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:54.999+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:139 15.e 15:7780a862::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb500 con 0x555d2027cc00 2023-11-08T16:39:55.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.000+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 41 ==== osd_op_reply(139 [pgnls start_epoch 110 out=81b] v106'3 uv3 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.000+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:140 15.e 15:7780a862:::100000001fa.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152380 con 0x555d2027cc00 2023-11-08T16:39:55.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.001+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 42 ==== osd_op_reply(140 100000001fa.00000003 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.001+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628282.3 2023-11-08T16:39:55.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.001+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:141 14.3f 14:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152700 con 0x555d1f628800 2023-11-08T16:39:55.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.004+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(141 100000001fa.00000000 [call,setxattr (8)] v110'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.005+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:142 15.1e 15:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153180 con 0x555d2027cc00 2023-11-08T16:39:55.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.006+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 43 ==== osd_op_reply(142 [pgnls start_epoch 110 out=101b] v106'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.006+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:143 15.1e 15:7e4d8c81:::10000000001.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153c00 con 0x555d2027cc00 2023-11-08T16:39:55.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.006+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 44 ==== osd_op_reply(143 10000000001.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.007+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511627777.2 2023-11-08T16:39:55.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.007+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:144 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5880 con 0x555d1f629800 2023-11-08T16:39:55.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.011+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 13 ==== osd_op_reply(144 10000000001.00000000 [call,setxattr (8)] v110'9 uv9 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:55.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.011+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:145 15.1e 15:7e8133fb::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5c00 con 0x555d2027cc00 2023-11-08T16:39:55.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.012+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 45 ==== osd_op_reply(145 [pgnls start_epoch 110 out=81b] v106'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.012+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:146 15.1e 15:7e8133fb:::100000001fb.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4700 con 0x555d2027cc00 2023-11-08T16:39:55.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.013+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 46 ==== osd_op_reply(146 100000001fb.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.013+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628283.6 2023-11-08T16:39:55.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.013+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:147 14.13 14:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eae00 con 0x555d2027cc00 2023-11-08T16:39:55.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.017+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 47 ==== osd_op_reply(147 100000001fb.00000000 [call,setxattr (8)] v110'20 uv20 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.017+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:148 15.1 15:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea380 con 0x555d1f629000 2023-11-08T16:39:55.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.018+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(148 [pgnls start_epoch 110 out=81b] v106'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.018+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:149 15.1 15:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eaa80 con 0x555d1f629000 2023-11-08T16:39:55.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.019+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(149 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.019+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628279.1 2023-11-08T16:39:55.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.019+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:150 14.36 14:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb180 con 0x555d2027cc00 2023-11-08T16:39:55.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.022+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 48 ==== osd_op_reply(150 100000001f7.00000000 [call,setxattr (8)] v110'27 uv27 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.022+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:151 15.11 15:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb880 con 0x555d2027cc00 2023-11-08T16:39:55.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.024+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 49 ==== osd_op_reply(151 [pgnls start_epoch 110 out=81b] v106'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.024+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:152 15.11 15:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ebc00 con 0x555d2027cc00 2023-11-08T16:39:55.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.025+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 50 ==== osd_op_reply(152 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.025+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628280.1 2023-11-08T16:39:55.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.025+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:153 14.19 14:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb500 con 0x555d1f629000 2023-11-08T16:39:55.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.029+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 11 ==== osd_op_reply(153 100000001f8.00000000 [call,setxattr (8)] v110'13 uv13 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.029+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:154 15.9 15:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152a80 con 0x555d1f629000 2023-11-08T16:39:55.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.030+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 12 ==== osd_op_reply(154 [pgnls start_epoch 110 out=81b] v106'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.030+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:155 15.9 15:94f3e7b4:::100000001fa.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152700 con 0x555d1f629000 2023-11-08T16:39:55.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.031+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 13 ==== osd_op_reply(155 100000001fa.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.031+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628282.2 2023-11-08T16:39:55.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.031+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:156 14.3f 14:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152e00 con 0x555d1f628800 2023-11-08T16:39:55.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.035+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(156 100000001fa.00000000 [call,setxattr (8)] v110'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.035+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:157 15.19 15:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153500 con 0x555d1f629400 2023-11-08T16:39:55.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.036+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 19 ==== osd_op_reply(157 [pgnls start_epoch 110 out=81b] v106'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.036+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:158 15.19 15:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153880 con 0x555d1f629400 2023-11-08T16:39:55.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.037+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 20 ==== osd_op_reply(158 100000001f8.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.037+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628280.0 2023-11-08T16:39:55.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.037+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:159 14.19 14:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152000 con 0x555d1f629000 2023-11-08T16:39:55.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.042+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 14 ==== osd_op_reply(159 100000001f8.00000000 [call,setxattr (8)] v110'14 uv14 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.042+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:160 15.5 15:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4a80 con 0x555d1f629000 2023-11-08T16:39:55.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.043+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 15 ==== osd_op_reply(160 [pgnls start_epoch 110 out=81b] v106'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.043+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:161 15.5 15:a238cdfc:::100000001f9.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5880 con 0x555d1f629000 2023-11-08T16:39:55.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.044+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 16 ==== osd_op_reply(161 100000001f9.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.044+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628281.2 2023-11-08T16:39:55.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.045+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:162 14.36 14:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea700 con 0x555d2027cc00 2023-11-08T16:39:55.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.049+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 51 ==== osd_op_reply(162 100000001f9.00000000 [call,setxattr (8)] v110'28 uv28 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.049+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:163 15.15 15:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea000 con 0x555d1f628c00 2023-11-08T16:39:55.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.050+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(163 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628c00 2023-11-08T16:39:55.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.050+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:164 15.d 15:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39bc00 con 0x555d1f628c00 2023-11-08T16:39:55.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.051+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(164 [pgnls start_epoch 110 out=81b] v106'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628c00 2023-11-08T16:39:55.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.051+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:165 15.d 15:b0a6c804:::100000001fc.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39b880 con 0x555d1f628c00 2023-11-08T16:39:55.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.052+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.7 v2:172.21.15.93:6818/433691658 9 ==== osd_op_reply(165 100000001fc.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628c00 2023-11-08T16:39:55.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.052+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628284.2 2023-11-08T16:39:55.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.053+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:166 14.c 14:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152a80 con 0x555d1f628000 2023-11-08T16:39:55.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.057+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.0 v2:172.21.15.81:6808/1666604866 26 ==== osd_op_reply(166 100000001fc.00000000 [call,setxattr (8)] v110'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628000 2023-11-08T16:39:55.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.057+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:167 15.1d 15:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152700 con 0x555d1f629400 2023-11-08T16:39:55.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.058+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 21 ==== osd_op_reply(167 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.058+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:168 15.3 15:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152380 con 0x555d1f629400 2023-11-08T16:39:55.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.059+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 22 ==== osd_op_reply(168 [pgnls start_epoch 110 out=101b] v106'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.059+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:169 15.3 15:c0957981:::100000001fa.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea380 con 0x555d1f629400 2023-11-08T16:39:55.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.060+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 23 ==== osd_op_reply(169 100000001fa.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.060+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628282.5 2023-11-08T16:39:55.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.060+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:170 14.3f 14:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eaa80 con 0x555d1f628800 2023-11-08T16:39:55.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.064+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(170 100000001fa.00000000 [call,setxattr (8)] v110'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.064+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:171 15.3 15:c5d8c51d::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb500 con 0x555d1f629400 2023-11-08T16:39:55.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.066+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 24 ==== osd_op_reply(171 [pgnls start_epoch 110 out=81b] v106'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.066+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:172 15.3 15:c5d8c51d:::100000001fb.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d1f629400 2023-11-08T16:39:55.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.067+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 25 ==== osd_op_reply(172 100000001fb.00000004 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.067+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628283.4 2023-11-08T16:39:55.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.067+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:173 14.13 14:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4000 con 0x555d2027cc00 2023-11-08T16:39:55.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.071+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 52 ==== osd_op_reply(173 100000001fb.00000000 [call,setxattr (8)] v110'21 uv21 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.071+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:174 15.13 15:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4e00 con 0x555d1f628400 2023-11-08T16:39:55.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.072+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 17 ==== osd_op_reply(174 [pgnls start_epoch 110 out=101b] v106'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:55.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.072+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:175 15.13 15:c9e7fe79:::100000001f9.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5500 con 0x555d1f628400 2023-11-08T16:39:55.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.073+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 18 ==== osd_op_reply(175 100000001f9.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:55.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.073+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628281.3 2023-11-08T16:39:55.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.073+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:176 14.36 14:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153c00 con 0x555d2027cc00 2023-11-08T16:39:55.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.077+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 53 ==== osd_op_reply(176 100000001f9.00000000 [call,setxattr (8)] v110'29 uv29 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.077+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:177 15.13 15:ca8fdbba::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153500 con 0x555d1f628400 2023-11-08T16:39:55.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.078+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 19 ==== osd_op_reply(177 [pgnls start_epoch 110 out=101b] v106'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:55.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.078+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:178 15.13 15:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153880 con 0x555d1f628400 2023-11-08T16:39:55.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.079+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 20 ==== osd_op_reply(178 100000001fb.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:55.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.079+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628283.0 2023-11-08T16:39:55.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.079+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:179 14.13 14:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152700 con 0x555d2027cc00 2023-11-08T16:39:55.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.083+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 54 ==== osd_op_reply(179 100000001fb.00000000 [call,setxattr (8)] v110'22 uv22 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.083+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:180 15.13 15:cb24bee2::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f724000 con 0x555d1f628400 2023-11-08T16:39:55.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.084+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 21 ==== osd_op_reply(180 [pgnls start_epoch 110 out=81b] v106'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:55.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.084+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:181 15.13 15:cb24bee2:::100000001fa.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f724380 con 0x555d1f628400 2023-11-08T16:39:55.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.085+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 22 ==== osd_op_reply(181 100000001fa.00000001 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:55.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.085+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628282.1 2023-11-08T16:39:55.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.085+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:182 14.3f 14:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f724700 con 0x555d1f628800 2023-11-08T16:39:55.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.089+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(182 100000001fa.00000000 [call,setxattr (8)] v110'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.089+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:183 15.b 15:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f724a80 con 0x555d2027cc00 2023-11-08T16:39:55.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.090+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 55 ==== osd_op_reply(183 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.090+0000 7fc9d764b640 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:184 15.1b 15:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153180 con 0x555d1f628c00 2023-11-08T16:39:55.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.091+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.7 v2:172.21.15.93:6818/433691658 10 ==== osd_op_reply(184 [pgnls start_epoch 110 out=81b] v106'1 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628c00 2023-11-08T16:39:55.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.091+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:185 15.1b 15:de1cf3c0:::100000001fb.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f724e00 con 0x555d1f628c00 2023-11-08T16:39:55.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.092+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.7 v2:172.21.15.93:6818/433691658 11 ==== osd_op_reply(185 100000001fb.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f628c00 2023-11-08T16:39:55.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.092+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628283.3 2023-11-08T16:39:55.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.092+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:186 14.13 14:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f725180 con 0x555d2027cc00 2023-11-08T16:39:55.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.096+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 56 ==== osd_op_reply(186 100000001fb.00000000 [call,setxattr (8)] v110'23 uv23 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.096+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:187 15.7 15:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f725500 con 0x555d1f628400 2023-11-08T16:39:55.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.097+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 23 ==== osd_op_reply(187 [pgnls start_epoch 110 out=101b] v106'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:55.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.097+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:188 15.7 15:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f725880 con 0x555d1f628400 2023-11-08T16:39:55.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.098+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 24 ==== osd_op_reply(188 10000000001.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:55.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.098+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511627777.0 2023-11-08T16:39:55.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.098+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:189 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f725c00 con 0x555d1f629800 2023-11-08T16:39:55.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.103+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 14 ==== osd_op_reply(189 10000000001.00000000 [call,setxattr (8)] v110'10 uv10 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:55.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.103+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:190 15.7 15:e7ecf421::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f724000 con 0x555d1f628400 2023-11-08T16:39:55.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.103+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 25 ==== osd_op_reply(190 [pgnls start_epoch 110 out=81b] v106'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:55.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.103+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:191 15.7 15:e7ecf421:::100000001f9.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f724380 con 0x555d1f628400 2023-11-08T16:39:55.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.104+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.1 v2:172.21.15.81:6800/2060798739 26 ==== osd_op_reply(191 100000001f9.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f628400 2023-11-08T16:39:55.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.104+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628281.4 2023-11-08T16:39:55.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.104+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:192 14.36 14:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f724a80 con 0x555d2027cc00 2023-11-08T16:39:55.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.107+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 57 ==== osd_op_reply(192 100000001f9.00000000 [call,setxattr (8)] v110'30 uv30 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.107+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:193 15.17 15:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152380 con 0x555d1f629400 2023-11-08T16:39:55.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.109+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 26 ==== osd_op_reply(193 [pgnls start_epoch 110 out=101b] v106'2 uv2 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.109+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:194 15.17 15:ee7e3069:::100000001f9.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153c00 con 0x555d1f629400 2023-11-08T16:39:55.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.110+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 27 ==== osd_op_reply(194 100000001f9.00000007 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.110+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628281.7 2023-11-08T16:39:55.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.110+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:195 14.36 14:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153500 con 0x555d2027cc00 2023-11-08T16:39:55.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.113+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 58 ==== osd_op_reply(195 100000001f9.00000000 [call,setxattr (8)] v110'31 uv31 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.114+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:196 15.17 15:eff9002a::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153880 con 0x555d1f629400 2023-11-08T16:39:55.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.116+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 28 ==== osd_op_reply(196 [pgnls start_epoch 110 out=81b] v106'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.116+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:197 15.17 15:eff9002a:::100000001f8.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152700 con 0x555d1f629400 2023-11-08T16:39:55.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.116+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 29 ==== osd_op_reply(197 100000001f8.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.116+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628280.3 2023-11-08T16:39:55.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.116+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:198 14.19 14:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d1f629000 2023-11-08T16:39:55.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.120+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 17 ==== osd_op_reply(198 100000001f8.00000000 [call,setxattr (8)] v110'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.120+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:199 15.f 15:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4000 con 0x555d1f629000 2023-11-08T16:39:55.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.121+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 18 ==== osd_op_reply(199 [pgnls start_epoch 110 out=101b] v106'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.121+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:200 15.f 15:f26fdfa9:::100000001f8.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4e00 con 0x555d1f629000 2023-11-08T16:39:55.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.122+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 19 ==== osd_op_reply(200 100000001f8.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.122+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628280.2 2023-11-08T16:39:55.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.122+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:201 14.19 14:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5500 con 0x555d1f629000 2023-11-08T16:39:55.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.126+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 20 ==== osd_op_reply(201 100000001f8.00000000 [call,setxattr (8)] v110'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.126+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:202 15.f 15:f376f0dc::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea380 con 0x555d1f629000 2023-11-08T16:39:55.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.127+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 21 ==== osd_op_reply(202 [pgnls start_epoch 110 out=101b] v106'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.127+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:203 15.f 15:f376f0dc:::100000001fb.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6eb500 con 0x555d1f629000 2023-11-08T16:39:55.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.128+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 22 ==== osd_op_reply(203 100000001fb.00000005 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.128+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628283.5 2023-11-08T16:39:55.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.128+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:204 14.13 14:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4a80 con 0x555d2027cc00 2023-11-08T16:39:55.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.132+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 59 ==== osd_op_reply(204 100000001fb.00000000 [call,setxattr (8)] v110'24 uv24 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.132+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:205 15.f 15:f526278d::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b5880 con 0x555d1f629000 2023-11-08T16:39:55.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.133+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 23 ==== osd_op_reply(205 [pgnls start_epoch 110 out=101b] v106'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.133+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:206 15.f 15:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6b4380 con 0x555d1f629000 2023-11-08T16:39:55.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.134+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 24 ==== osd_op_reply(206 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.135+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511627777.1 2023-11-08T16:39:55.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.135+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:207 14.7 14:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f724e00 con 0x555d1f629800 2023-11-08T16:39:55.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.139+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.5 v2:172.21.15.93:6802/2406579932 15 ==== osd_op_reply(207 10000000001.00000000 [call,setxattr (8)] v110'11 uv11 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f629800 2023-11-08T16:39:55.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.139+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:208 15.f 15:f776b802::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152000 con 0x555d1f629000 2023-11-08T16:39:55.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.140+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 25 ==== osd_op_reply(208 [pgnls start_epoch 110 out=81b] v106'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.140+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:209 15.f 15:f776b802:::100000001fb.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20152a80 con 0x555d1f629000 2023-11-08T16:39:55.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.141+0000 7fc9dce56640 1 -- 172.21.15.81:0/2588455022 <== osd.6 v2:172.21.15.93:6816/607960134 26 ==== osd_op_reply(209 100000001fb.00000007 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118d80 con 0x555d1f629000 2023-11-08T16:39:55.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.141+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628283.7 2023-11-08T16:39:55.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.141+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:210 14.13 14:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d20153180 con 0x555d2027cc00 2023-11-08T16:39:55.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.144+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.2 v2:172.21.15.81:6824/3759311428 60 ==== osd_op_reply(210 100000001fb.00000000 [call,setxattr (8)] v110'25 uv25 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d2027cc00 2023-11-08T16:39:55.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.144+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:211 15.1f 15:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39bc00 con 0x555d1f628800 2023-11-08T16:39:55.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.146+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 18 ==== osd_op_reply(211 [pgnls start_epoch 110 out=101b] v106'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.146+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:212 15.1f 15:faedc8eb:::100000001f6.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f39b880 con 0x555d1f628800 2023-11-08T16:39:55.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.146+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 19 ==== osd_op_reply(212 100000001f6.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.147+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628278.2 2023-11-08T16:39:55.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.147+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:213 14.2a 14:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f6ea000 con 0x555d1f629400 2023-11-08T16:39:55.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.152+0000 7fc9dc655640 1 -- 172.21.15.81:0/2588455022 <== osd.4 v2:172.21.15.93:6800/3259756279 30 ==== osd_op_reply(213 100000001f6.00000000 [call,setxattr (8)] v110'15 uv15 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118b40 con 0x555d1f629400 2023-11-08T16:39:55.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.152+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:214 15.1f 15:fb032386::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f72e000 con 0x555d1f628800 2023-11-08T16:39:55.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.152+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 20 ==== osd_op_reply(214 [pgnls start_epoch 110 out=101b] v106'4 uv4 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.152+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:215 15.1f 15:fb032386:::100000001fa.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f72e380 con 0x555d1f628800 2023-11-08T16:39:55.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.153+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 21 ==== osd_op_reply(215 100000001fa.00000007 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.153+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628282.7 2023-11-08T16:39:55.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.153+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:216 14.3f 14:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f72e700 con 0x555d1f628800 2023-11-08T16:39:55.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.157+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 22 ==== osd_op_reply(216 100000001fa.00000000 [call,setxattr (8)] v110'17 uv17 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.157+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:217 15.1f 15:feb1c803::::head [pgnls start_epoch 110 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f72ea80 con 0x555d1f628800 2023-11-08T16:39:55.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.158+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 23 ==== osd_op_reply(217 [pgnls start_epoch 110 out=81b] v106'4 uv4 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.158+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:218 15.1f 15:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f72ee00 con 0x555d1f628800 2023-11-08T16:39:55.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.159+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 24 ==== osd_op_reply(218 100000001fa.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.159+0000 7fc9de64c5c0 10 datascan.operator(): handling object 1099511628282.0 2023-11-08T16:39:55.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.159+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:219 14.3f 14:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b,setxattr scan_pool_id (8) in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x555d1f72f180 con 0x555d1f628800 2023-11-08T16:39:55.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 <== osd.3 v2:172.21.15.81:6816/2756229107 25 ==== osd_op_reply(219 100000001fa.00000000 [call,setxattr (8)] v110'18 uv18 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x555d20118fc0 con 0x555d1f628800 2023-11-08T16:39:55.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x555d1f628000 msgr2=0x555d1f6aa000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x555d1f628000 0x555d1f6aa000 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x555d1f628400 msgr2=0x555d1f6aa580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x555d1f628400 0x555d1f6aa580 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x555d2027cc00 msgr2=0x555d1f3ae100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x555d2027cc00 0x555d1f3ae100 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x555d1f628800 msgr2=0x555d1f6aab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x555d1f628800 0x555d1f6aab00 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x555d1f629400 msgr2=0x555d1f6abb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x555d1f629400 0x555d1f6abb80 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 reap_dead start 2023-11-08T16:39:55.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x555d1f629800 msgr2=0x555d1f6ac100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x555d1f629800 0x555d1f6ac100 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 reap_dead start 2023-11-08T16:39:55.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x555d1f629000 msgr2=0x555d1f6ab600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x555d1f629000 0x555d1f6ab600 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x555d1f628c00 msgr2=0x555d1f6ab080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x555d1f628c00 0x555d1f6ab080 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.163+0000 7fc9dbe54640 1 -- 172.21.15.81:0/2588455022 reap_dead start 2023-11-08T16:39:55.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.164+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x555d2027c800 msgr2=0x555d1f3af700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.164+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x555d2027c800 0x555d1f3af700 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x555d2014ea80 tx=0x555d200c5f80 comp rx=0 tx=0).stop 2023-11-08T16:39:55.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.164+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd800 msgr2=0x555d1f3ae680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.164+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd800 0x555d1f3ae680 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x555d2011d5c0 tx=0x555d20218d20 comp rx=0 tx=0).stop 2023-11-08T16:39:55.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.164+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 shutdown_connections 2023-11-08T16:39:55.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.164+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x555d2027c800 0x555d1f3af700 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.164+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2588455022 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d201fd800 0x555d1f3ae680 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.164+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 >> 172.21.15.81:0/2588455022 conn(0x555d201fcc00 msgr2=0x555d200cd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:55.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.165+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 shutdown_connections 2023-11-08T16:39:55.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.165+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2588455022 wait complete. 2023-11-08T16:39:55.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.166+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2717590877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d1f342c00 msgr2=0x555d1f3ac580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.166+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2717590877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d1f342c00 0x555d1f3ac580 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x555d2014f410 tx=0x555d2010cc00 comp rx=0 tx=0).stop 2023-11-08T16:39:55.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.166+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2717590877 shutdown_connections 2023-11-08T16:39:55.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.166+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2717590877 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x555d1f343c00 0x555d1f3ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.166+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2717590877 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d1f343800 0x555d1f3acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.166+0000 7fc9de64c5c0 1 --2- 172.21.15.81:0/2717590877 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x555d1f342c00 0x555d1f3ac580 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.166+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2717590877 >> 172.21.15.81:0/2717590877 conn(0x555d1f343000 msgr2=0x555d200cc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:55.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.166+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2717590877 shutdown_connections 2023-11-08T16:39:55.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.167+0000 7fc9de64c5c0 1 -- 172.21.15.81:0/2717590877 wait complete. 2023-11-08T16:39:55.182 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.490927, result: 2023-11-08T16:39:55.182 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-11-08T16:39:55.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.284+0000 7f93bae3c5c0 1 -- start start 2023-11-08T16:39:55.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.284+0000 7f93bae3c5c0 1 Processor -- start 2023-11-08T16:39:55.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.285+0000 7f93bae3c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97cfac00 0x562a97d64580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.285+0000 7f93bae3c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97cfb800 0x562a97d64b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.285+0000 7f93bae3c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97cfbc00 0x562a97d65080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.285+0000 7f93bae3c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a97d43180 con 0x562a97cfac00 2023-11-08T16:39:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.285+0000 7f93bae3c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a97d42140 con 0x562a97cfb800 2023-11-08T16:39:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.285+0000 7f93bae3c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a97d42000 con 0x562a97cfbc00 2023-11-08T16:39:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.285+0000 7f93b8e20640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97cfac00 0x562a97d64580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.285+0000 7f93b8e20640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97cfac00 0x562a97d64580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38354/0 (socket says 172.21.15.81:38354) 2023-11-08T16:39:55.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.285+0000 7f93b8e20640 1 -- 172.21.15.81:0/1868084048 learned_addr learned my addr 172.21.15.81:0/1868084048 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:55.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b861f640 1 --2- 172.21.15.81:0/1868084048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97cfb800 0x562a97d64b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b8e20640 1 -- 172.21.15.81:0/1868084048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97cfbc00 msgr2=0x562a97d65080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:55.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b9621640 1 --2- 172.21.15.81:0/1868084048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97cfbc00 0x562a97d65080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b8e20640 1 --2- 172.21.15.81:0/1868084048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97cfbc00 0x562a97d65080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b8e20640 1 -- 172.21.15.81:0/1868084048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97cfb800 msgr2=0x562a97d64b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:55.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b8e20640 1 --2- 172.21.15.81:0/1868084048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97cfb800 0x562a97d64b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b8e20640 1 -- 172.21.15.81:0/1868084048 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a97d209c0 con 0x562a97cfac00 2023-11-08T16:39:55.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b861f640 1 --2- 172.21.15.81:0/1868084048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97cfb800 0x562a97d64b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:55.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b8e20640 1 --2- 172.21.15.81:0/1868084048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97cfac00 0x562a97d64580 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x562a98b034a0 tx=0x562a97fd8420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1b97a22894fcf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b661b640 1 -- 172.21.15.81:0/1868084048 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a97d411e0 con 0x562a97cfac00 2023-11-08T16:39:55.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b661b640 1 -- 172.21.15.81:0/1868084048 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562a97d54480 con 0x562a97cfac00 2023-11-08T16:39:55.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93b661b640 1 -- 172.21.15.81:0/1868084048 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a97d41080 con 0x562a97cfac00 2023-11-08T16:39:55.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/1868084048 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a97d20b60 con 0x562a97cfac00 2023-11-08T16:39:55.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.286+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/1868084048 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562a97d20820 con 0x562a97cfac00 2023-11-08T16:39:55.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.287+0000 7f93bae3c5c0 4 waiting for MDS map... 2023-11-08T16:39:55.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.287+0000 7f93b661b640 1 -- 172.21.15.81:0/1868084048 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x562a97dbac40 con 0x562a97cfac00 2023-11-08T16:39:55.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.287+0000 7f93b661b640 1 -- 172.21.15.81:0/1868084048 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 93) v1 ==== 3306+0+0 (secure 0 0 0) 0x562a97d4af00 con 0x562a97cfac00 2023-11-08T16:39:55.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.287+0000 7f93bae3c5c0 4 Got MDS map 93 2023-11-08T16:39:55.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.287+0000 7f93bae3c5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:39:55.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.287+0000 7f93bae3c5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:39:55.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.288+0000 7f93bae3c5c0 1 Processor -- start 2023-11-08T16:39:55.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.288+0000 7f93bae3c5c0 1 -- start start 2023-11-08T16:39:55.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.289+0000 7f93bae3c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97fe3000 0x562a97d65600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.289+0000 7f93bae3c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3400 0x562a97d65b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.289+0000 7f93bae3c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97fe3800 0x562a97d66100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.289+0000 7f93bae3c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a97d42a00 con 0x562a97fe3400 2023-11-08T16:39:55.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.289+0000 7f93bae3c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a97dc9180 con 0x562a97fe3800 2023-11-08T16:39:55.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.289+0000 7f93bae3c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a97dc8dc0 con 0x562a97fe3000 2023-11-08T16:39:55.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.289+0000 7f93b8e20640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3400 0x562a97d65b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.289+0000 7f93b8e20640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3400 0x562a97d65b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38368/0 (socket says 172.21.15.81:38368) 2023-11-08T16:39:55.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.289+0000 7f93b8e20640 1 -- 172.21.15.81:0/3146560202 learned_addr learned my addr 172.21.15.81:0/3146560202 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:55.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.289+0000 7f93b9621640 1 --2- 172.21.15.81:0/3146560202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97fe3000 0x562a97d65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.290+0000 7f93b861f640 1 --2- 172.21.15.81:0/3146560202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97fe3800 0x562a97d66100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.290+0000 7f93b8e20640 1 -- 172.21.15.81:0/3146560202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97fe3000 msgr2=0x562a97d65600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:55.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.290+0000 7f93b8e20640 1 --2- 172.21.15.81:0/3146560202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97fe3000 0x562a97d65600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.290+0000 7f93b8e20640 1 -- 172.21.15.81:0/3146560202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97fe3800 msgr2=0x562a97d66100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:55.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.290+0000 7f93b8e20640 1 --2- 172.21.15.81:0/3146560202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97fe3800 0x562a97d66100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.290+0000 7f93b8e20640 1 -- 172.21.15.81:0/3146560202 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a97d204e0 con 0x562a97fe3400 2023-11-08T16:39:55.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.290+0000 7f93b8e20640 1 --2- 172.21.15.81:0/3146560202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3400 0x562a97d65b80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x562a97ff6930 tx=0x562a97fd8060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b483fe8bb98cd1e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.290+0000 7f93b4617640 1 -- 172.21.15.81:0/3146560202 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a97d40dc0 con 0x562a97fe3400 2023-11-08T16:39:55.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.290+0000 7f93b4617640 1 -- 172.21.15.81:0/3146560202 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562a97dda900 con 0x562a97fe3400 2023-11-08T16:39:55.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.290+0000 7f93b4617640 1 -- 172.21.15.81:0/3146560202 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a97d411e0 con 0x562a97fe3400 2023-11-08T16:39:55.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.291+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3146560202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3400 msgr2=0x562a97d65b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.291+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3146560202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3400 0x562a97d65b80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x562a97ff6930 tx=0x562a97fd8060 comp rx=0 tx=0).stop 2023-11-08T16:39:55.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.291+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3146560202 shutdown_connections 2023-11-08T16:39:55.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.291+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3146560202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97fe3800 0x562a97d66100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.291+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3146560202 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3400 0x562a97d65b80 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.291+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3146560202 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97fe3000 0x562a97d65600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.291+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3146560202 >> 172.21.15.81:0/3146560202 conn(0x562a97fe2c00 msgr2=0x562a98a85000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:55.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.291+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3146560202 shutdown_connections 2023-11-08T16:39:55.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.291+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3146560202 wait complete. 2023-11-08T16:39:55.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.292+0000 7f93bae3c5c0 1 Processor -- start 2023-11-08T16:39:55.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.292+0000 7f93bae3c5c0 1 -- start start 2023-11-08T16:39:55.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.292+0000 7f93bae3c5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97fe3000 0x562a97d67180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.292+0000 7f93bae3c5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97fe3400 0x562a97d66c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.292+0000 7f93bae3c5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3800 0x562a97d66680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.292+0000 7f93bae3c5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a97dc8c80 con 0x562a97fe3800 2023-11-08T16:39:55.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.292+0000 7f93bae3c5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a97d42140 con 0x562a97fe3000 2023-11-08T16:39:55.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.292+0000 7f93bae3c5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a97d42b40 con 0x562a97fe3400 2023-11-08T16:39:55.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.293+0000 7f93b861f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3800 0x562a97d66680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.293+0000 7f93b861f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3800 0x562a97d66680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38374/0 (socket says 172.21.15.81:38374) 2023-11-08T16:39:55.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.293+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 learned_addr learned my addr 172.21.15.81:0/3660701353 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:55.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.293+0000 7f93b9621640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97fe3000 0x562a97d67180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.293+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97fe3400 msgr2=0x562a97d66c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.293+0000 7f93b861f640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97fe3400 0x562a97d66c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.293+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97fe3000 msgr2=0x562a97d67180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.293+0000 7f93b861f640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97fe3000 0x562a97d67180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.293+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a97d209c0 con 0x562a97fe3800 2023-11-08T16:39:55.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.293+0000 7f93b861f640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3800 0x562a97d66680 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x562a98b02a80 tx=0x562a97fd81e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.294+0000 7f93b2e14640 1 -- 172.21.15.81:0/3660701353 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a97d40f20 con 0x562a97fe3800 2023-11-08T16:39:55.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.294+0000 7f93b2e14640 1 -- 172.21.15.81:0/3660701353 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562a97dda300 con 0x562a97fe3800 2023-11-08T16:39:55.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.294+0000 7f93b2e14640 1 -- 172.21.15.81:0/3660701353 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562a97d40dc0 con 0x562a97fe3800 2023-11-08T16:39:55.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.294+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562a97d20b60 con 0x562a97fe3800 2023-11-08T16:39:55.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.294+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a97d201a0 con 0x562a97fe3800 2023-11-08T16:39:55.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.294+0000 7f93bae3c5c0 4 datascan.init: resolving metadata pool 13 2023-11-08T16:39:55.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.296+0000 7f93b2e14640 1 -- 172.21.15.81:0/3660701353 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x562a97d57080 con 0x562a97fe3800 2023-11-08T16:39:55.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.296+0000 7f93b2e14640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562a97fe3c00 0x562a97d67700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.296+0000 7f93b2e14640 1 -- 172.21.15.81:0/3660701353 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x562a97dbaa80 con 0x562a97fe3800 2023-11-08T16:39:55.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.296+0000 7f93bae3c5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:39:55.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.296+0000 7f93bae3c5c0 4 datascan.main: data pool 'cephfs_data' has ID 14 2023-11-08T16:39:55.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.296+0000 7f93bae3c5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:39:55.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.296+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562a980e0000 0x562a97d66100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.296+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:1 13.14 13:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a97d53880 con 0x562a980e0000 2023-11-08T16:39:55.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.297+0000 7f93b9621640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562a97fe3c00 0x562a97d67700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.297+0000 7f93b8e20640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562a980e0000 0x562a97d66100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.297+0000 7f93b8e20640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562a980e0000 0x562a97d66100 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.297+0000 7f93b9621640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562a97fe3c00 0x562a97d67700 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x562a97dec810 tx=0x562a98164000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.298+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv50 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.298+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562a980e0400 0x562a97d65b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.298+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:2 13.1f 13:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a97d53c00 con 0x562a980e0400 2023-11-08T16:39:55.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.298+0000 7f93b9621640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562a980e0400 0x562a97d65b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.300+0000 7f93b9621640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562a980e0400 0x562a97d65b80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.301+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.301+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562a980e0800 0x562a97d65600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.301+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:3 14.0 14:00000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170000 con 0x562a980e0800 2023-11-08T16:39:55.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.301+0000 7f93b8e20640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562a980e0800 0x562a97d65600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.301+0000 7f93b8e20640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562a980e0800 0x562a97d65600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.302+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.302+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:4 14.20 14:04000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170e00 con 0x562a980e0000 2023-11-08T16:39:55.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.302+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.302+0000 7f93b4617640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562a98176000 0x562a9817c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.302+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:5 14.10 14:08000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a97d53880 con 0x562a98176000 2023-11-08T16:39:55.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.303+0000 7f93b861f640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562a98176000 0x562a9817c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.303+0000 7f93b861f640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562a98176000 0x562a9817c000 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.304+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.304+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:6 14.30 14:0c000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170000 con 0x562a980e0000 2023-11-08T16:39:55.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.304+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(6 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.304+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:7 14.8 14:10000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171180 con 0x562a980e0000 2023-11-08T16:39:55.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.304+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(7 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.304+0000 7f93b3e16640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562a98176400 0x562a9817c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.304+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:8 14.28 14:14000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170e00 con 0x562a98176400 2023-11-08T16:39:55.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.305+0000 7f93b9621640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562a98176400 0x562a9817c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.305+0000 7f93b9621640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562a98176400 0x562a9817c580 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.305+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176400 2023-11-08T16:39:55.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.305+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:9 14.18 14:18000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171500 con 0x562a980e0800 2023-11-08T16:39:55.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.306+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.306+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:10 14.38 14:1c000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170e00 con 0x562a98176000 2023-11-08T16:39:55.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.306+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.306+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:11 14.4 14:20000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170e00 con 0x562a980e0400 2023-11-08T16:39:55.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.307+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.307+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:12 14.24 14:24000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171880 con 0x562a98176000 2023-11-08T16:39:55.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.307+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 110 out=49b] v106'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.308+0000 7f93b4617640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562a98176800 0x562a9817cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.308+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:13 14.14 14:28000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171c00 con 0x562a98176800 2023-11-08T16:39:55.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.308+0000 7f93b8e20640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562a98176800 0x562a9817cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.309+0000 7f93b8e20640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562a98176800 0x562a9817cb00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.310+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 110 out=49b] v106'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.310+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:14 14.34 14:2c000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a97d53c00 con 0x562a980e0800 2023-11-08T16:39:55.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.310+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(14 [pgnls-filter start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.311+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:15 14.c 14:30000000::::head [pgnls-filter start_epoch 110 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a97d53880 con 0x562a980e0000 2023-11-08T16:39:55.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.311+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(15 [pgnls-filter start_epoch 110 out=81b] v110'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.311+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:16 14.0 14:00000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170380 con 0x562a980e0800 2023-11-08T16:39:55.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.311+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(16 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.311+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:17 14.20 14:04000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170000 con 0x562a980e0000 2023-11-08T16:39:55.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.312+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.312+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:18 14.10 14:08000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a9819a000 con 0x562a98176000 2023-11-08T16:39:55.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.312+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.312+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:19 14.30 14:0c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171180 con 0x562a980e0000 2023-11-08T16:39:55.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.313+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(19 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.313+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:20 14.8 14:10000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a9819a380 con 0x562a980e0000 2023-11-08T16:39:55.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.313+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(20 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.313+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:21 14.28 14:14000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171500 con 0x562a98176400 2023-11-08T16:39:55.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.313+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176400 2023-11-08T16:39:55.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.313+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:22 14.18 14:18000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a9819a700 con 0x562a980e0800 2023-11-08T16:39:55.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.314+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.314+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:23 14.38 14:1c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171c00 con 0x562a98176000 2023-11-08T16:39:55.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.314+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(23 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.314+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:24 14.4 14:20000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a9819aa80 con 0x562a980e0400 2023-11-08T16:39:55.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.315+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(24 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.315+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:25 14.24 14:24000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170e00 con 0x562a98176000 2023-11-08T16:39:55.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.315+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(25 [pgnls-filter start_epoch 110 out=49b] v106'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.315+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:26 14.14 14:28000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a9819aa80 con 0x562a98176800 2023-11-08T16:39:55.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.316+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(26 [pgnls-filter start_epoch 110 out=49b] v106'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.316+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:27 14.34 14:2c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171500 con 0x562a980e0800 2023-11-08T16:39:55.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.317+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 6 ==== osd_op_reply(27 [pgnls-filter start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.317+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:28 14.c 14:30000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171c00 con 0x562a980e0000 2023-11-08T16:39:55.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.318+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(28 [pgnls-filter start_epoch 110 out=81b] v110'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.318+0000 7f93bae3c5c0 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-11-08T16:39:55.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.318+0000 7f93bae3c5c0 10 datascan.operator(): handling object 100000001fc.0 2023-11-08T16:39:55.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.318+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:29 14.c 14:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170700 con 0x562a980e0000 2023-11-08T16:39:55.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.318+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(29 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+46 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.318+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562a980e1000 0x562a9817d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.318+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:30 15.c 15:30604137:::100000001fc.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171500 con 0x562a980e1000 2023-11-08T16:39:55.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.319+0000 7f93b861f640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562a980e1000 0x562a9817d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.320+0000 7f93b861f640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562a980e1000 0x562a9817d600 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.321+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(30 100000001fc.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a980e1000 2023-11-08T16:39:55.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.321+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:31 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171c00 con 0x562a980e0400 2023-11-08T16:39:55.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.322+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(31 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.322+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:39:55.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.322+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:32 13.1f 13:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170a80 con 0x562a980e0400 2023-11-08T16:39:55.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.325+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(32 1.00000000 [create,omap-set-header] v110'11 uv11 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.325+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-11-08T16:39:55.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.325+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:33 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6000 con 0x562a980e0400 2023-11-08T16:39:55.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.326+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(33 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.326+0000 7f93bae3c5c0 20 datascan.read_dentry: oid=1.00000000 dname=lost+found frag=*, r=0 2023-11-08T16:39:55.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.326+0000 7f93bae3c5c0 20 datascan.read_dentry: lost+found_head not found in result 2023-11-08T16:39:55.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.326+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:34 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6380 con 0x562a980e0400 2023-11-08T16:39:55.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.329+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(34 1.00000000 [omap-set-vals] v110'12 uv12 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.329+0000 7f93bae3c5c0 20 datascan.inject_linkage: Injected dentry 0x0x1/lost+found pointing to 0x0x4 2023-11-08T16:39:55.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.329+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:35 13.3 13:c0ce80d4:::4.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6700 con 0x562a980e0000 2023-11-08T16:39:55.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.330+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 11 ==== osd_op_reply(35 4.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.330+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:39:55.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.330+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:36 13.3 13:c0ce80d4:::4.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6a80 con 0x562a980e0000 2023-11-08T16:39:55.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.333+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 12 ==== osd_op_reply(36 4.00000000 [create,omap-set-header] v110'1 uv1 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.333+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x4 2023-11-08T16:39:55.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.333+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:37 13.3 13:c0ce80d4:::4.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6e00 con 0x562a980e0000 2023-11-08T16:39:55.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.335+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 13 ==== osd_op_reply(37 4.00000000 [omap-set-vals] v110'2 uv2 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.335+0000 7f93bae3c5c0 20 datascan.inject_linkage: Injected dentry 0x0x4/100000001fc pointing to 0x0x100000001fc 2023-11-08T16:39:55.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.336+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:38 14.2c 14:34000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7180 con 0x562a98176000 2023-11-08T16:39:55.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.336+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(38 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.336+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:39 14.1c 14:38000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170380 con 0x562a980e0000 2023-11-08T16:39:55.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.336+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 14 ==== osd_op_reply(39 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.337+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:40 14.3c 14:3c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171880 con 0x562a98176000 2023-11-08T16:39:55.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.337+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(40 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.337+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:41 14.2 14:40000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170000 con 0x562a980e0400 2023-11-08T16:39:55.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.338+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(41 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.338+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:42 14.22 14:44000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a97d53c00 con 0x562a98176800 2023-11-08T16:39:55.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.339+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(42 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.339+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:43 14.12 14:48000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171180 con 0x562a980e0800 2023-11-08T16:39:55.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.339+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 7 ==== osd_op_reply(43 [pgnls-filter start_epoch 110 out=49b] v106'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.340+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:44 14.32 14:4c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a97d53880 con 0x562a98176800 2023-11-08T16:39:55.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.341+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(44 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.341+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:45 14.a 14:50000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170700 con 0x562a980e1000 2023-11-08T16:39:55.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.342+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(45 [pgnls-filter start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a980e1000 2023-11-08T16:39:55.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.342+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:46 14.2a 14:54000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171880 con 0x562a98176800 2023-11-08T16:39:55.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.343+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(46 [pgnls-filter start_epoch 110 out=81b] v110'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.343+0000 7f93bae3c5c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-11-08T16:39:55.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.343+0000 7f93bae3c5c0 10 datascan.operator(): handling object 100000001f6.0 2023-11-08T16:39:55.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.343+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:47 14.2a 14:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6000 con 0x562a98176800 2023-11-08T16:39:55.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.345+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(47 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.345+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:48 15.a 15:57fe7535:::100000001f6.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6700 con 0x562a980e0800 2023-11-08T16:39:55.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.345+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 8 ==== osd_op_reply(48 100000001f6.00000000 [getxattr (30) out=30b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.346+0000 7f93bae3c5c0 20 datascan.operator(): loaded layout from xattr: pi: 15 os: 262144 sc: 4 su: 65536 2023-11-08T16:39:55.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.346+0000 7f93bae3c5c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-11-08T16:39:55.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.346+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:49 15.a 15:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6a80 con 0x562a980e0800 2023-11-08T16:39:55.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.346+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 9 ==== osd_op_reply(49 100000001f6.00000000 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.346+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:50 15.1c 15:3d6db04b:::100000001f6.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6e00 con 0x562a98176000 2023-11-08T16:39:55.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.347+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(50 100000001f6.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.347+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:51 15.1f 15:faedc8eb:::100000001f6.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7180 con 0x562a98176400 2023-11-08T16:39:55.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.347+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(51 100000001f6.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176400 2023-11-08T16:39:55.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.348+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:52 15.e 15:728aa123:::100000001f6.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7500 con 0x562a980e0800 2023-11-08T16:39:55.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.348+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 10 ==== osd_op_reply(52 100000001f6.00000003 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.348+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-11-08T16:39:55.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.348+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_1 2023-11-08T16:39:55.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.348+0000 7f93bae3c5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_1 2023-11-08T16:39:55.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.348+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:53 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7880 con 0x562a98176000 2023-11-08T16:39:55.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.349+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(53 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.349+0000 7f93bae3c5c0 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-11-08T16:39:55.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.349+0000 7f93bae3c5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_1, will insert to root 2023-11-08T16:39:55.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.349+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:54 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7c00 con 0x562a98176000 2023-11-08T16:39:55.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.349+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(54 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.349+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:39:55.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.349+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:55 13.10 13:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ac000 con 0x562a98176000 2023-11-08T16:39:55.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.352+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 12 ==== osd_op_reply(55 10000000000.00000000 [create,omap-set-header] v110'16 uv16 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.352+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-11-08T16:39:55.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.352+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:56 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ac380 con 0x562a98176000 2023-11-08T16:39:55.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.353+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 13 ==== osd_op_reply(56 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.353+0000 7f93bae3c5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_1 frag=*, r=0 2023-11-08T16:39:55.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.353+0000 7f93bae3c5c0 20 datascan.read_dentry: flushed_file_1_head not found in result 2023-11-08T16:39:55.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.353+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/flushed_file_1 with size=524288 bytes 2023-11-08T16:39:55.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.353+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:57 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ac700 con 0x562a98176000 2023-11-08T16:39:55.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.355+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 14 ==== osd_op_reply(57 10000000000.00000000 [omap-set-vals] v110'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.355+0000 7f93bae3c5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_1 pointing to 0x0x100000001f6 2023-11-08T16:39:55.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.355+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/stripey 2023-11-08T16:39:55.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.355+0000 7f93bae3c5c0 20 datascan.get_frag_of: dirino=0x1 target_dname=stripey 2023-11-08T16:39:55.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.355+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:58 13.1f 13:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981aca80 con 0x562a980e0400 2023-11-08T16:39:55.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.357+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(58 1.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 154+0+0 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.357+0000 7f93bae3c5c0 10 datascan.get_frag_of: No backtrace on '1.00000000' 2023-11-08T16:39:55.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.357+0000 7f93bae3c5c0 4 datascan.get_frag_of: No backtrace on '1.00000000': cannot determine fragtree 2023-11-08T16:39:55.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.357+0000 7f93bae3c5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/stripey, will insert to root 2023-11-08T16:39:55.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.357+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:59 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ace00 con 0x562a980e0400 2023-11-08T16:39:55.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.358+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(59 1.00000000 [omap-get-header out=274b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.358+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: read_version = 12 2023-11-08T16:39:55.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.358+0000 7f93bae3c5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x1 * 2023-11-08T16:39:55.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.358+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:60 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ad180 con 0x562a980e0400 2023-11-08T16:39:55.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.360+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 11 ==== osd_op_reply(60 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv12 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.360+0000 7f93bae3c5c0 20 datascan.read_dentry: oid=1.00000000 dname=stripey frag=*, r=0 2023-11-08T16:39:55.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.360+0000 7f93bae3c5c0 20 datascan.read_dentry: stripey_head not found in result 2023-11-08T16:39:55.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.360+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:stripey 2023-11-08T16:39:55.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.360+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:61 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ad500 con 0x562a980e0400 2023-11-08T16:39:55.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.363+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 12 ==== osd_op_reply(61 1.00000000 [omap-set-vals] v110'13 uv13 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.363+0000 7f93bae3c5c0 20 datascan.inject_linkage: Injected dentry 0x0x1/stripey pointing to 0x0x10000000000 2023-11-08T16:39:55.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.363+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:62 14.1a 14:58000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ad880 con 0x562a980e0000 2023-11-08T16:39:55.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.364+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 15 ==== osd_op_reply(62 [pgnls-filter start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.364+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:63 14.3a 14:5c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171c00 con 0x562a980e0000 2023-11-08T16:39:55.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.365+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 16 ==== osd_op_reply(63 [pgnls-filter start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.365+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:64 14.6 14:60000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170380 con 0x562a980e0000 2023-11-08T16:39:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.366+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.366+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:65 14.26 14:64000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170a80 con 0x562a98176800 2023-11-08T16:39:55.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.367+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(65 [pgnls-filter start_epoch 110 out=49b] v106'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.367+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:66 14.16 14:68000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171180 con 0x562a98176400 2023-11-08T16:39:55.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.367+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(66 [pgnls-filter start_epoch 110 out=49b] v106'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176400 2023-11-08T16:39:55.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.367+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:67 14.36 14:6c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ad880 con 0x562a980e0800 2023-11-08T16:39:55.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.368+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 11 ==== osd_op_reply(67 [pgnls-filter start_epoch 110 out=101b] v110'31 uv31 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.368+0000 7f93bae3c5c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-11-08T16:39:55.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.368+0000 7f93bae3c5c0 10 datascan.operator(): handling object 100000001f9.0 2023-11-08T16:39:55.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.369+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:68 14.36 14:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981adc00 con 0x562a980e0800 2023-11-08T16:39:55.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.369+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 12 ==== osd_op_reply(68 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv31 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.369+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:69 15.16 15:6ee5e2c4:::100000001f9.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ac000 con 0x562a980e0000 2023-11-08T16:39:55.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.370+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 18 ==== osd_op_reply(69 100000001f9.00000000 [getxattr (30) out=30b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.370+0000 7f93bae3c5c0 20 datascan.operator(): loaded layout from xattr: pi: 15 os: 262144 sc: 4 su: 65536 2023-11-08T16:39:55.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.370+0000 7f93bae3c5c0 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-11-08T16:39:55.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.370+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:70 15.7 15:e7ecf421:::100000001f9.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6000 con 0x562a98176000 2023-11-08T16:39:55.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.371+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 15 ==== osd_op_reply(70 100000001f9.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.371+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:71 15.2 15:46f89f8a:::100000001f9.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6380 con 0x562a980e0800 2023-11-08T16:39:55.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.372+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 13 ==== osd_op_reply(71 100000001f9.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.372+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:72 15.16 15:6df493a7:::100000001f9.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6700 con 0x562a980e0000 2023-11-08T16:39:55.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.373+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 19 ==== osd_op_reply(72 100000001f9.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.373+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:73 15.17 15:ee7e3069:::100000001f9.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6a80 con 0x562a98176800 2023-11-08T16:39:55.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.374+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(73 100000001f9.00000007 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.374+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-11-08T16:39:55.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.374+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_4 2023-11-08T16:39:55.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.374+0000 7f93bae3c5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_4 2023-11-08T16:39:55.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.374+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:74 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6e00 con 0x562a98176000 2023-11-08T16:39:55.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.375+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 16 ==== osd_op_reply(74 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.375+0000 7f93bae3c5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:39:55.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.375+0000 7f93bae3c5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:39:55.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.375+0000 7f93bae3c5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_4, will insert to root 2023-11-08T16:39:55.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.375+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:75 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7180 con 0x562a98176000 2023-11-08T16:39:55.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.376+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 17 ==== osd_op_reply(75 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.376+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-11-08T16:39:55.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.376+0000 7f93bae3c5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:39:55.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.376+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:76 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7500 con 0x562a98176000 2023-11-08T16:39:55.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.376+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 18 ==== osd_op_reply(76 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.376+0000 7f93bae3c5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_4 frag=*, r=0 2023-11-08T16:39:55.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.377+0000 7f93bae3c5c0 20 datascan.read_dentry: flushed_file_4_head not found in result 2023-11-08T16:39:55.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.377+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/flushed_file_4 with size=1441792 bytes 2023-11-08T16:39:55.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.377+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:77 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7880 con 0x562a98176000 2023-11-08T16:39:55.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.380+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 19 ==== osd_op_reply(77 10000000000.00000000 [omap-set-vals] v110'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.380+0000 7f93bae3c5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_4 pointing to 0x0x100000001f9 2023-11-08T16:39:55.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.380+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:78 14.36 14:6fe38dc2::::head [pgnls-filter start_epoch 110 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7c00 con 0x562a980e0800 2023-11-08T16:39:55.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.380+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 14 ==== osd_op_reply(78 [pgnls-filter start_epoch 110 out=81b] v110'31 uv31 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.381+0000 7f93bae3c5c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-11-08T16:39:55.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.381+0000 7f93bae3c5c0 10 datascan.operator(): handling object 100000001f7.0 2023-11-08T16:39:55.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.381+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:79 14.36 14:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6000 con 0x562a980e0800 2023-11-08T16:39:55.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.381+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 15 ==== osd_op_reply(79 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.382+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:80 15.16 15:6fe38dc2:::100000001f7.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7180 con 0x562a980e0000 2023-11-08T16:39:55.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.382+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 20 ==== osd_op_reply(80 100000001f7.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.382+0000 7f93bae3c5c0 20 datascan.operator(): loaded layout from xattr: pi: 15 os: 262144 sc: 4 su: 65536 2023-11-08T16:39:55.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.382+0000 7f93bae3c5c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-11-08T16:39:55.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.382+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:81 15.16 15:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ac380 con 0x562a980e0000 2023-11-08T16:39:55.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.383+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 21 ==== osd_op_reply(81 100000001f7.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.383+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:82 15.1 15:848ebb0b:::100000001f7.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ac700 con 0x562a980e0400 2023-11-08T16:39:55.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.384+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 13 ==== osd_op_reply(82 100000001f7.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.385+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:83 15.1c 15:398da958:::100000001f7.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ad880 con 0x562a98176000 2023-11-08T16:39:55.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.385+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 20 ==== osd_op_reply(83 100000001f7.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.385+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:84 15.6 15:613bf181:::100000001f7.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ad180 con 0x562a980e0000 2023-11-08T16:39:55.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.386+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 22 ==== osd_op_reply(84 100000001f7.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.386+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-11-08T16:39:55.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.386+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_2 2023-11-08T16:39:55.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.386+0000 7f93bae3c5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_2 2023-11-08T16:39:55.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.386+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:85 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981adc00 con 0x562a98176000 2023-11-08T16:39:55.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.387+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 21 ==== osd_op_reply(85 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.387+0000 7f93bae3c5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:39:55.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.387+0000 7f93bae3c5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:39:55.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.387+0000 7f93bae3c5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_2, will insert to root 2023-11-08T16:39:55.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.387+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:86 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ac000 con 0x562a98176000 2023-11-08T16:39:55.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.388+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 22 ==== osd_op_reply(86 10000000000.00000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.388+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-11-08T16:39:55.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.388+0000 7f93bae3c5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:39:55.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.388+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:87 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171c00 con 0x562a98176000 2023-11-08T16:39:55.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.389+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 23 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.389+0000 7f93bae3c5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_2 frag=*, r=0 2023-11-08T16:39:55.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.389+0000 7f93bae3c5c0 20 datascan.read_dentry: flushed_file_2_head not found in result 2023-11-08T16:39:55.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.389+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/flushed_file_2 with size=917504 bytes 2023-11-08T16:39:55.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.389+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:88 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170380 con 0x562a98176000 2023-11-08T16:39:55.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.393+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 24 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v110'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.393+0000 7f93bae3c5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_2 pointing to 0x0x100000001f7 2023-11-08T16:39:55.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.393+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:89 14.e 14:70000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170a80 con 0x562a98176800 2023-11-08T16:39:55.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.394+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(89 [pgnls-filter start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.394+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:90 14.2e 14:74000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170e00 con 0x562a98176400 2023-11-08T16:39:55.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.395+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 110 out=49b] v106'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176400 2023-11-08T16:39:55.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.396+0000 7f93b4617640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562a98176c00 0x562a9817d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.396+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:91 14.1e 14:78000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981aca80 con 0x562a98176c00 2023-11-08T16:39:55.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.397+0000 7f93b9621640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562a98176c00 0x562a9817d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.398+0000 7f93b9621640 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562a98176c00 0x562a9817d080 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.399+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(91 [pgnls-filter start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176c00 2023-11-08T16:39:55.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.399+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:92 14.3e 14:7c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171500 con 0x562a980e1000 2023-11-08T16:39:55.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.400+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(92 [pgnls-filter start_epoch 110 out=49b] v106'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a980e1000 2023-11-08T16:39:55.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.400+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:93 14.1 14:80000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ace00 con 0x562a98176400 2023-11-08T16:39:55.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.401+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(93 [pgnls-filter start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176400 2023-11-08T16:39:55.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.401+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:94 14.21 14:84000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a97d53c00 con 0x562a980e1000 2023-11-08T16:39:55.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.402+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(94 [pgnls-filter start_epoch 110 out=49b] v106'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a980e1000 2023-11-08T16:39:55.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.402+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:95 14.11 14:88000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171880 con 0x562a98176800 2023-11-08T16:39:55.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.404+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(95 [pgnls-filter start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.404+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:96 14.31 14:8c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ad500 con 0x562a98176000 2023-11-08T16:39:55.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.405+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 25 ==== osd_op_reply(96 [pgnls-filter start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.405+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:97 14.9 14:90000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170000 con 0x562a980e1000 2023-11-08T16:39:55.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.406+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(97 [pgnls-filter start_epoch 110 out=49b] v106'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a980e1000 2023-11-08T16:39:55.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.406+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:98 14.29 14:94000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ac700 con 0x562a980e0400 2023-11-08T16:39:55.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.407+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 14 ==== osd_op_reply(98 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.407+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:99 14.19 14:98000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170700 con 0x562a980e0400 2023-11-08T16:39:55.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.408+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 15 ==== osd_op_reply(99 [pgnls-filter start_epoch 110 out=81b] v110'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.408+0000 7f93bae3c5c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-11-08T16:39:55.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.408+0000 7f93bae3c5c0 10 datascan.operator(): handling object 100000001f8.0 2023-11-08T16:39:55.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.408+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:100 14.19 14:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7880 con 0x562a980e0400 2023-11-08T16:39:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.410+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 16 ==== osd_op_reply(100 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.410+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:101 15.19 15:997b7129:::100000001f8.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6380 con 0x562a98176800 2023-11-08T16:39:55.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.411+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(101 100000001f8.00000000 [getxattr (30) out=30b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.411+0000 7f93bae3c5c0 20 datascan.operator(): loaded layout from xattr: pi: 15 os: 262144 sc: 4 su: 65536 2023-11-08T16:39:55.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.411+0000 7f93bae3c5c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-11-08T16:39:55.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.411+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:102 15.19 15:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6700 con 0x562a98176800 2023-11-08T16:39:55.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.412+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(102 100000001f8.00000000 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.412+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:103 15.11 15:893be98d:::100000001f8.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171180 con 0x562a980e0800 2023-11-08T16:39:55.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.413+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 16 ==== osd_op_reply(103 100000001f8.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.413+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:104 15.f 15:f26fdfa9:::100000001f8.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170e00 con 0x562a980e0400 2023-11-08T16:39:55.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.414+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 17 ==== osd_op_reply(104 100000001f8.00000002 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.415+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:105 15.17 15:eff9002a:::100000001f8.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171c00 con 0x562a98176800 2023-11-08T16:39:55.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.416+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 13 ==== osd_op_reply(105 100000001f8.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.416+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-11-08T16:39:55.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.416+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_3 2023-11-08T16:39:55.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.416+0000 7f93bae3c5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_3 2023-11-08T16:39:55.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.416+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:106 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170380 con 0x562a98176000 2023-11-08T16:39:55.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.417+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 26 ==== osd_op_reply(106 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.417+0000 7f93bae3c5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:39:55.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.417+0000 7f93bae3c5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:39:55.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.417+0000 7f93bae3c5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_3, will insert to root 2023-11-08T16:39:55.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.417+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:107 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171500 con 0x562a98176000 2023-11-08T16:39:55.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.418+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 27 ==== osd_op_reply(107 10000000000.00000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.418+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-11-08T16:39:55.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.418+0000 7f93bae3c5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:39:55.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.418+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:108 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6a80 con 0x562a98176000 2023-11-08T16:39:55.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.419+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 28 ==== osd_op_reply(108 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.419+0000 7f93bae3c5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_3 frag=*, r=0 2023-11-08T16:39:55.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.419+0000 7f93bae3c5c0 20 datascan.read_dentry: flushed_file_3_head not found in result 2023-11-08T16:39:55.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.419+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/flushed_file_3 with size=917503 bytes 2023-11-08T16:39:55.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.419+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:109 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7c00 con 0x562a98176000 2023-11-08T16:39:55.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.422+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 29 ==== osd_op_reply(109 10000000000.00000000 [omap-set-vals] v110'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.422+0000 7f93bae3c5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_3 pointing to 0x0x100000001f8 2023-11-08T16:39:55.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.422+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:110 14.39 14:9c000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6e00 con 0x562a98176c00 2023-11-08T16:39:55.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.423+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(110 [pgnls-filter start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176c00 2023-11-08T16:39:55.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.423+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:111 14.5 14:a0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a97d53880 con 0x562a980e0000 2023-11-08T16:39:55.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.424+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 23 ==== osd_op_reply(111 [pgnls-filter start_epoch 110 out=49b] v106'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.424+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:112 14.25 14:a4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981aca80 con 0x562a980e0800 2023-11-08T16:39:55.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.425+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 17 ==== osd_op_reply(112 [pgnls-filter start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.426+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:113 14.15 14:a8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ba000 con 0x562a98176c00 2023-11-08T16:39:55.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.427+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(113 [pgnls-filter start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176c00 2023-11-08T16:39:55.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.427+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:114 14.35 14:ac000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ace00 con 0x562a98176800 2023-11-08T16:39:55.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.428+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 14 ==== osd_op_reply(114 [pgnls-filter start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.428+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:115 14.d 14:b0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ba380 con 0x562a980e0400 2023-11-08T16:39:55.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.430+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 18 ==== osd_op_reply(115 [pgnls-filter start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.430+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:116 14.2d 14:b4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ad880 con 0x562a98176800 2023-11-08T16:39:55.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.430+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 15 ==== osd_op_reply(116 [pgnls-filter start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.431+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:117 14.1d 14:b8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ba700 con 0x562a980e0000 2023-11-08T16:39:55.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.432+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 24 ==== osd_op_reply(117 [pgnls-filter start_epoch 110 out=49b] v106'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.432+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:118 14.3d 14:bc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981adc00 con 0x562a98176400 2023-11-08T16:39:55.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.433+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(118 [pgnls-filter start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176400 2023-11-08T16:39:55.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.433+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:119 14.3 14:c0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981baa80 con 0x562a980e0000 2023-11-08T16:39:55.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.434+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 25 ==== osd_op_reply(119 [pgnls-filter start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.434+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:120 14.23 14:c4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ac000 con 0x562a98176c00 2023-11-08T16:39:55.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.435+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(120 [pgnls-filter start_epoch 110 out=49b] v106'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176c00 2023-11-08T16:39:55.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.435+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:121 14.13 14:c8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981bae00 con 0x562a980e0800 2023-11-08T16:39:55.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.436+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 18 ==== osd_op_reply(121 [pgnls-filter start_epoch 110 out=81b] v110'25 uv25 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.436+0000 7f93bae3c5c0 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-11-08T16:39:55.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.436+0000 7f93bae3c5c0 10 datascan.operator(): handling object 100000001fb.0 2023-11-08T16:39:55.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.436+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:122 14.13 14:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6000 con 0x562a980e0800 2023-11-08T16:39:55.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.437+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 19 ==== osd_op_reply(122 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv25 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.437+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:123 15.13 15:ca8fdbba:::100000001fb.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6380 con 0x562a98176000 2023-11-08T16:39:55.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.438+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 30 ==== osd_op_reply(123 100000001fb.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.438+0000 7f93bae3c5c0 20 datascan.operator(): loaded layout from xattr: pi: 15 os: 262144 sc: 4 su: 65536 2023-11-08T16:39:55.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.438+0000 7f93bae3c5c0 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-11-08T16:39:55.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.438+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:124 15.3 15:c5d8c51d:::100000001fb.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170a80 con 0x562a98176800 2023-11-08T16:39:55.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.439+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 16 ==== osd_op_reply(124 100000001fb.00000004 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.440+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:125 15.f 15:f376f0dc:::100000001fb.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171880 con 0x562a980e0400 2023-11-08T16:39:55.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.441+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 19 ==== osd_op_reply(125 100000001fb.00000005 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.441+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:126 15.1e 15:7e8133fb:::100000001fb.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ac380 con 0x562a980e0800 2023-11-08T16:39:55.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.442+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 20 ==== osd_op_reply(126 100000001fb.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.442+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:127 15.f 15:f776b802:::100000001fb.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ad180 con 0x562a980e0400 2023-11-08T16:39:55.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.443+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 20 ==== osd_op_reply(127 100000001fb.00000007 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.443+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-11-08T16:39:55.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.443+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_6 2023-11-08T16:39:55.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.443+0000 7f93bae3c5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_6 2023-11-08T16:39:55.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.443+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:128 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6a80 con 0x562a98176000 2023-11-08T16:39:55.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.444+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 31 ==== osd_op_reply(128 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.444+0000 7f93bae3c5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:39:55.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.444+0000 7f93bae3c5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:39:55.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.444+0000 7f93bae3c5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_6, will insert to root 2023-11-08T16:39:55.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.444+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:129 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170000 con 0x562a98176000 2023-11-08T16:39:55.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.445+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 32 ==== osd_op_reply(129 10000000000.00000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.445+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-11-08T16:39:55.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.445+0000 7f93bae3c5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:39:55.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.445+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:130 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170380 con 0x562a98176000 2023-11-08T16:39:55.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.446+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 33 ==== osd_op_reply(130 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.446+0000 7f93bae3c5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_6 frag=*, r=0 2023-11-08T16:39:55.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.446+0000 7f93bae3c5c0 20 datascan.read_dentry: flushed_file_6_head not found in result 2023-11-08T16:39:55.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.446+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fb at 0x0x10000000000/flushed_file_6 with size=1572864 bytes 2023-11-08T16:39:55.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.446+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:131 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171500 con 0x562a98176000 2023-11-08T16:39:55.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.449+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 34 ==== osd_op_reply(131 10000000000.00000000 [omap-set-vals] v110'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.449+0000 7f93bae3c5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_6 pointing to 0x0x100000001fb 2023-11-08T16:39:55.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.449+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:132 14.33 14:cc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171180 con 0x562a98176000 2023-11-08T16:39:55.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.450+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 35 ==== osd_op_reply(132 [pgnls-filter start_epoch 110 out=49b] v106'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.450+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:133 14.b 14:d0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981bb180 con 0x562a980e0000 2023-11-08T16:39:55.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.451+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 26 ==== osd_op_reply(133 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.452+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:134 14.2b 14:d4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ba700 con 0x562a98176800 2023-11-08T16:39:55.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.454+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 17 ==== osd_op_reply(134 [pgnls-filter start_epoch 110 out=49b] v106'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.454+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:135 14.1b 14:d8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981bb500 con 0x562a98176c00 2023-11-08T16:39:55.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.454+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(135 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176c00 2023-11-08T16:39:55.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.454+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:136 14.3b 14:dc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7c00 con 0x562a98176800 2023-11-08T16:39:55.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.456+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 18 ==== osd_op_reply(136 [pgnls-filter start_epoch 110 out=49b] v106'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.456+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:137 14.7 14:e0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981bb880 con 0x562a980e1000 2023-11-08T16:39:55.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.458+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(137 [pgnls-filter start_epoch 110 out=81b] v110'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a980e1000 2023-11-08T16:39:55.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.458+0000 7f93bae3c5c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-11-08T16:39:55.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.458+0000 7f93bae3c5c0 10 datascan.operator(): handling object 10000000001.0 2023-11-08T16:39:55.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.458+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:138 14.7 14:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171c00 con 0x562a980e1000 2023-11-08T16:39:55.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.459+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(138 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a980e1000 2023-11-08T16:39:55.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.459+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:139 15.7 15:e2e07930:::10000000001.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170e00 con 0x562a98176000 2023-11-08T16:39:55.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.459+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 36 ==== osd_op_reply(139 10000000001.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.460+0000 7f93bae3c5c0 20 datascan.operator(): loaded layout from xattr: pi: 15 os: 262144 sc: 4 su: 65536 2023-11-08T16:39:55.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.460+0000 7f93bae3c5c0 10 datascan.operator(): calculating striped size from complete objs: 0, partial objs: 4 2023-11-08T16:39:55.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.460+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:140 15.7 15:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ad500 con 0x562a98176000 2023-11-08T16:39:55.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.460+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 37 ==== osd_op_reply(140 10000000001.00000000 [stat out=16b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.461+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:141 15.f 15:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981aca80 con 0x562a980e0400 2023-11-08T16:39:55.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.462+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.6 v2:172.21.15.93:6816/607960134 21 ==== osd_op_reply(141 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0d80 con 0x562a980e0400 2023-11-08T16:39:55.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.462+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:142 15.1e 15:7e4d8c81:::10000000001.00000002:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ace00 con 0x562a980e0800 2023-11-08T16:39:55.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.463+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 21 ==== osd_op_reply(142 10000000001.00000002 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.463+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:143 15.e 15:72395991:::10000000001.00000003:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ad880 con 0x562a980e0800 2023-11-08T16:39:55.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.464+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 22 ==== osd_op_reply(143 10000000001.00000003 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.464+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-11-08T16:39:55.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.464+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_0 2023-11-08T16:39:55.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.464+0000 7f93bae3c5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_0 2023-11-08T16:39:55.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.464+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:144 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981ac700 con 0x562a98176000 2023-11-08T16:39:55.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.465+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 38 ==== osd_op_reply(144 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.465+0000 7f93bae3c5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:39:55.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.465+0000 7f93bae3c5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:39:55.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.465+0000 7f93bae3c5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_0, will insert to root 2023-11-08T16:39:55.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.465+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:145 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a97d53c00 con 0x562a98176000 2023-11-08T16:39:55.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.466+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 39 ==== osd_op_reply(145 10000000000.00000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.466+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: read_version = 21 2023-11-08T16:39:55.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.466+0000 7f93bae3c5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:39:55.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.466+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:146 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a97d53880 con 0x562a98176000 2023-11-08T16:39:55.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.467+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 40 ==== osd_op_reply(146 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.467+0000 7f93bae3c5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_0 frag=*, r=0 2023-11-08T16:39:55.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.467+0000 7f93bae3c5c0 20 datascan.read_dentry: flushed_file_0_head not found in result 2023-11-08T16:39:55.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.467+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/flushed_file_0 with size=1048576 bytes 2023-11-08T16:39:55.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.467+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:147 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981bb880 con 0x562a98176000 2023-11-08T16:39:55.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.471+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 41 ==== osd_op_reply(147 10000000000.00000000 [omap-set-vals] v110'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.471+0000 7f93bae3c5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_0 pointing to 0x0x10000000001 2023-11-08T16:39:55.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.471+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:148 14.27 14:e4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981baa80 con 0x562a980e1000 2023-11-08T16:39:55.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.473+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(148 [pgnls-filter start_epoch 110 out=49b] v106'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a980e1000 2023-11-08T16:39:55.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.473+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:149 14.17 14:e8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98b0a000 con 0x562a98176000 2023-11-08T16:39:55.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.474+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 42 ==== osd_op_reply(149 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.474+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:150 14.37 14:ec000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6380 con 0x562a98176800 2023-11-08T16:39:55.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.475+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 19 ==== osd_op_reply(150 [pgnls-filter start_epoch 110 out=49b] v106'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.475+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:151 14.f 14:f0000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98b0a380 con 0x562a98176c00 2023-11-08T16:39:55.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.476+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(151 [pgnls-filter start_epoch 110 out=49b] v106'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176c00 2023-11-08T16:39:55.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.476+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:152 14.2f 14:f4000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6700 con 0x562a980e0000 2023-11-08T16:39:55.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.477+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 27 ==== osd_op_reply(152 [pgnls-filter start_epoch 110 out=49b] v106'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.477+0000 7f93b4617640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:153 14.1f 14:f8000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98b0a700 con 0x562a980e0000 2023-11-08T16:39:55.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.478+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.0 v2:172.21.15.81:6808/1666604866 28 ==== osd_op_reply(153 [pgnls-filter start_epoch 110 out=49b] v106'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0000 2023-11-08T16:39:55.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.478+0000 7f93b3e16640 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:154 14.3f 14:fc000000::::head [pgnls-filter start_epoch 110 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a7180 con 0x562a98176400 2023-11-08T16:39:55.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.480+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(154 [pgnls-filter start_epoch 110 out=81b] v110'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176400 2023-11-08T16:39:55.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.480+0000 7f93bae3c5c0 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-11-08T16:39:55.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.480+0000 7f93bae3c5c0 10 datascan.operator(): handling object 100000001fa.0 2023-11-08T16:39:55.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.480+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:155 14.3f 14:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981bae00 con 0x562a98176400 2023-11-08T16:39:55.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.481+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(155 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr (111) out=111b,getxattr,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176400 2023-11-08T16:39:55.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.481+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:156 15.1f 15:feb1c803:::100000001fa.00000000:head [getxattr layout in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981bbc00 con 0x562a98176400 2023-11-08T16:39:55.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.482+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(156 100000001fa.00000000 [getxattr (30) out=30b] v0'0 uv4 ondisk = 0) v8 ==== 164+0+30 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176400 2023-11-08T16:39:55.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.482+0000 7f93bae3c5c0 20 datascan.operator(): loaded layout from xattr: pi: 15 os: 262144 sc: 4 su: 65536 2023-11-08T16:39:55.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.482+0000 7f93bae3c5c0 10 datascan.operator(): calculating striped size from complete objs: 4, partial objs: 4 2023-11-08T16:39:55.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.482+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:157 15.1c 15:3c0114ac:::100000001fa.00000004:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170000 con 0x562a98176000 2023-11-08T16:39:55.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.483+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 43 ==== osd_op_reply(157 100000001fa.00000004 [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.483+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:158 15.3 15:c0957981:::100000001fa.00000005:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170380 con 0x562a98176800 2023-11-08T16:39:55.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.484+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.4 v2:172.21.15.93:6800/3259756279 20 ==== osd_op_reply(158 100000001fa.00000005 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a98176800 2023-11-08T16:39:55.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.484+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:159 15.a 15:56c65d28:::100000001fa.00000006:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171500 con 0x562a980e0800 2023-11-08T16:39:55.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.485+0000 7f93b8e20640 1 -- 172.21.15.81:0/3660701353 <== osd.2 v2:172.21.15.81:6824/3759311428 23 ==== osd_op_reply(159 100000001fa.00000006 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0b40 con 0x562a980e0800 2023-11-08T16:39:55.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.485+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:160 15.1f 15:fb032386:::100000001fa.00000007:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171180 con 0x562a98176400 2023-11-08T16:39:55.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.486+0000 7f93b9621640 1 -- 172.21.15.81:0/3660701353 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(160 100000001fa.00000007 [stat out=16b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x562a98ad0d80 con 0x562a98176400 2023-11-08T16:39:55.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.486+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-11-08T16:39:55.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.486+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/flushed_file_5 2023-11-08T16:39:55.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.486+0000 7f93bae3c5c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=flushed_file_5 2023-11-08T16:39:55.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.486+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:161 13.10 13:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170700 con 0x562a98176000 2023-11-08T16:39:55.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.487+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 44 ==== osd_op_reply(161 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.487+0000 7f93bae3c5c0 10 datascan.get_frag_of: No backtrace on '10000000000.00000000' 2023-11-08T16:39:55.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.487+0000 7f93bae3c5c0 4 datascan.get_frag_of: No backtrace on '10000000000.00000000': cannot determine fragtree 2023-11-08T16:39:55.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.487+0000 7f93bae3c5c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/flushed_file_5, will insert to root 2023-11-08T16:39:55.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.487+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:162 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98171c00 con 0x562a98176000 2023-11-08T16:39:55.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.488+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 45 ==== osd_op_reply(162 10000000000.00000000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.488+0000 7f93bae3c5c0 10 datascan.find_or_create_dirfrag: read_version = 22 2023-11-08T16:39:55.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.488+0000 7f93bae3c5c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2023-11-08T16:39:55.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.488+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:163 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a98170e00 con 0x562a98176000 2023-11-08T16:39:55.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.489+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 46 ==== osd_op_reply(163 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.489+0000 7f93bae3c5c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_5 frag=*, r=0 2023-11-08T16:39:55.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.489+0000 7f93bae3c5c0 20 datascan.read_dentry: flushed_file_5_head not found in result 2023-11-08T16:39:55.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.489+0000 7f93bae3c5c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fa at 0x0x10000000000/flushed_file_5 with size=1441793 bytes 2023-11-08T16:39:55.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.489+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:164 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x562a981a6a80 con 0x562a98176000 2023-11-08T16:39:55.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 <== osd.1 v2:172.21.15.81:6800/2060798739 47 ==== osd_op_reply(164 10000000000.00000000 [omap-set-vals] v110'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562a98ad0fc0 con 0x562a98176000 2023-11-08T16:39:55.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_5 pointing to 0x0x100000001fa 2023-11-08T16:39:55.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562a980e0000 msgr2=0x562a97d66100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562a980e0000 0x562a97d66100 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562a98176000 msgr2=0x562a9817c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562a98176000 0x562a9817c000 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562a980e0800 msgr2=0x562a97d65600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562a980e0800 0x562a97d65600 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562a98176400 msgr2=0x562a9817c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562a98176400 0x562a9817c580 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562a98176800 msgr2=0x562a9817cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562a98176800 0x562a9817cb00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.492+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562a980e1000 msgr2=0x562a9817d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562a980e1000 0x562a9817d600 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562a980e0400 msgr2=0x562a97d65b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562a980e0400 0x562a97d65b80 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562a98176c00 msgr2=0x562a9817d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562a98176c00 0x562a9817d080 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562a97fe3c00 msgr2=0x562a97d67700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562a97fe3c00 0x562a97d67700 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x562a97dec810 tx=0x562a98164000 comp rx=0 tx=0).stop 2023-11-08T16:39:55.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3800 msgr2=0x562a97d66680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/3660701353 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97fe3800 0x562a97d66680 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x562a98b02a80 tx=0x562a97fd81e0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 reap_dead start 2023-11-08T16:39:55.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 reap_dead start 2023-11-08T16:39:55.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 reap_dead start 2023-11-08T16:39:55.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 reap_dead start 2023-11-08T16:39:55.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.493+0000 7f93b861f640 1 -- 172.21.15.81:0/3660701353 reap_dead start 2023-11-08T16:39:55.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.494+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 shutdown_connections 2023-11-08T16:39:55.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.494+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 >> 172.21.15.81:0/3660701353 conn(0x562a97fe2c00 msgr2=0x562a98a85000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:55.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.494+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 shutdown_connections 2023-11-08T16:39:55.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.494+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/3660701353 wait complete. 2023-11-08T16:39:55.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.495+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/1868084048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97cfac00 msgr2=0x562a97d64580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.495+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/1868084048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97cfac00 0x562a97d64580 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x562a98b034a0 tx=0x562a97fd8420 comp rx=0 tx=0).stop 2023-11-08T16:39:55.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.495+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/1868084048 shutdown_connections 2023-11-08T16:39:55.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.495+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/1868084048 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562a97cfbc00 0x562a97d65080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.495+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/1868084048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562a97cfb800 0x562a97d64b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.495+0000 7f93bae3c5c0 1 --2- 172.21.15.81:0/1868084048 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562a97cfac00 0x562a97d64580 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.495+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/1868084048 >> 172.21.15.81:0/1868084048 conn(0x562a97cfb000 msgr2=0x562a98a84800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:55.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.496+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/1868084048 shutdown_connections 2023-11-08T16:39:55.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.496+0000 7f93bae3c5c0 1 -- 172.21.15.81:0/1868084048 wait complete. 2023-11-08T16:39:55.510 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:00.327521, result: 2023-11-08T16:39:55.510 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-11-08T16:39:55.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.611+0000 7f22354145c0 1 -- start start 2023-11-08T16:39:55.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.611+0000 7f22354145c0 1 Processor -- start 2023-11-08T16:39:55.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.612+0000 7f22354145c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d2ea000 0x55b57c57b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.612+0000 7f22354145c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57c511c00 0x55b57c57bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.612+0000 7f22354145c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d2ea800 0x55b57c57b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.612+0000 7f22354145c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57c558b40 con 0x55b57c511c00 2023-11-08T16:39:55.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.612+0000 7f22354145c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57c558a00 con 0x55b57d2ea000 2023-11-08T16:39:55.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.612+0000 7f22354145c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b57c558140 con 0x55b57d2ea800 2023-11-08T16:39:55.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.613+0000 7f22328ed640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57c511c00 0x55b57c57bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.613+0000 7f22328ed640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57c511c00 0x55b57c57bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38390/0 (socket says 172.21.15.81:38390) 2023-11-08T16:39:55.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.613+0000 7f22328ed640 1 -- 172.21.15.81:0/693984581 learned_addr learned my addr 172.21.15.81:0/693984581 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:55.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.613+0000 7f22338ef640 1 --2- 172.21.15.81:0/693984581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d2ea800 0x55b57c57b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.613+0000 7f22330ee640 1 --2- 172.21.15.81:0/693984581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d2ea000 0x55b57c57b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.613+0000 7f22328ed640 1 -- 172.21.15.81:0/693984581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d2ea800 msgr2=0x55b57c57b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:55.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.613+0000 7f22328ed640 1 --2- 172.21.15.81:0/693984581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d2ea800 0x55b57c57b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.613+0000 7f22328ed640 1 -- 172.21.15.81:0/693984581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d2ea000 msgr2=0x55b57c57b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:55.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.613+0000 7f22328ed640 1 --2- 172.21.15.81:0/693984581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d2ea000 0x55b57c57b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.613+0000 7f22328ed640 1 -- 172.21.15.81:0/693984581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b57c5364e0 con 0x55b57c511c00 2023-11-08T16:39:55.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.613+0000 7f22330ee640 1 --2- 172.21.15.81:0/693984581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d2ea000 0x55b57c57b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:39:55.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.614+0000 7f22328ed640 1 --2- 172.21.15.81:0/693984581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57c511c00 0x55b57c57bb80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x55b57d329620 tx=0x55b57d2e4600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08fa4108a7da729 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.614+0000 7f22308e9640 1 -- 172.21.15.81:0/693984581 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b57c556f20 con 0x55b57c511c00 2023-11-08T16:39:55.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.614+0000 7f22308e9640 1 -- 172.21.15.81:0/693984581 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b57d2f0000 con 0x55b57c511c00 2023-11-08T16:39:55.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.614+0000 7f22308e9640 1 -- 172.21.15.81:0/693984581 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b57c5571e0 con 0x55b57c511c00 2023-11-08T16:39:55.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.614+0000 7f22354145c0 1 -- 172.21.15.81:0/693984581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b57c5361a0 con 0x55b57c511c00 2023-11-08T16:39:55.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.614+0000 7f22354145c0 1 -- 172.21.15.81:0/693984581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b57d2f2000 con 0x55b57c511c00 2023-11-08T16:39:55.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.614+0000 7f22354145c0 4 waiting for MDS map... 2023-11-08T16:39:55.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.614+0000 7f22308e9640 1 -- 172.21.15.81:0/693984581 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x55b57c5d0c40 con 0x55b57c511c00 2023-11-08T16:39:55.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.614+0000 7f22308e9640 1 -- 172.21.15.81:0/693984581 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 93) v1 ==== 3306+0+0 (secure 0 0 0) 0x55b57c560f00 con 0x55b57c511c00 2023-11-08T16:39:55.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.614+0000 7f22354145c0 4 Got MDS map 93 2023-11-08T16:39:55.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.615+0000 7f22354145c0 4 datascan.main: Using metadata pool output 2023-11-08T16:39:55.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.615+0000 7f22354145c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:39:55.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.615+0000 7f22354145c0 1 Processor -- start 2023-11-08T16:39:55.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.616+0000 7f22354145c0 1 -- start start 2023-11-08T16:39:55.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.616+0000 7f22354145c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dcc00 0x55b57c57ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.616+0000 7f22354145c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d3dd000 0x55b57c57c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.616+0000 7f22354145c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d3dd400 0x55b57c57c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.616+0000 7f22354145c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57c558000 con 0x55b57d3dcc00 2023-11-08T16:39:55.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.616+0000 7f22354145c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57c5df180 con 0x55b57d3dd000 2023-11-08T16:39:55.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.616+0000 7f22354145c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b57c5dedc0 con 0x55b57d3dd400 2023-11-08T16:39:55.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22338ef640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dcc00 0x55b57c57ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22338ef640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dcc00 0x55b57c57ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38396/0 (socket says 172.21.15.81:38396) 2023-11-08T16:39:55.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22338ef640 1 -- 172.21.15.81:0/888369947 learned_addr learned my addr 172.21.15.81:0/888369947 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:55.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22338ef640 1 -- 172.21.15.81:0/888369947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d3dd400 msgr2=0x55b57c57c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:55.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22328ed640 1 --2- 172.21.15.81:0/888369947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d3dd400 0x55b57c57c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22330ee640 1 --2- 172.21.15.81:0/888369947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d3dd000 0x55b57c57c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22338ef640 1 --2- 172.21.15.81:0/888369947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d3dd400 0x55b57c57c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22338ef640 1 -- 172.21.15.81:0/888369947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d3dd000 msgr2=0x55b57c57c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:55.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22338ef640 1 --2- 172.21.15.81:0/888369947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d3dd000 0x55b57c57c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22338ef640 1 -- 172.21.15.81:0/888369947 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b57d2f21a0 con 0x55b57d3dcc00 2023-11-08T16:39:55.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22330ee640 1 --2- 172.21.15.81:0/888369947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d3dd000 0x55b57c57c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:55.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f22338ef640 1 --2- 172.21.15.81:0/888369947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dcc00 0x55b57c57ab00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x55b57d3ecc90 tx=0x55b57d3f6c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=896fbb7ce73f81a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.617+0000 7f222e8e5640 1 -- 172.21.15.81:0/888369947 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b57c556f20 con 0x55b57d3dcc00 2023-11-08T16:39:55.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.618+0000 7f222e8e5640 1 -- 172.21.15.81:0/888369947 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b57d2f0d80 con 0x55b57d3dcc00 2023-11-08T16:39:55.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.618+0000 7f222e8e5640 1 -- 172.21.15.81:0/888369947 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b57c5571e0 con 0x55b57d3dcc00 2023-11-08T16:39:55.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.618+0000 7f22354145c0 1 -- 172.21.15.81:0/888369947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dcc00 msgr2=0x55b57c57ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.618+0000 7f22354145c0 1 --2- 172.21.15.81:0/888369947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dcc00 0x55b57c57ab00 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x55b57d3ecc90 tx=0x55b57d3f6c00 comp rx=0 tx=0).stop 2023-11-08T16:39:55.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.618+0000 7f22354145c0 1 -- 172.21.15.81:0/888369947 shutdown_connections 2023-11-08T16:39:55.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.618+0000 7f22354145c0 1 --2- 172.21.15.81:0/888369947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d3dd400 0x55b57c57c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.618+0000 7f22354145c0 1 --2- 172.21.15.81:0/888369947 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d3dd000 0x55b57c57c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.618+0000 7f22354145c0 1 --2- 172.21.15.81:0/888369947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dcc00 0x55b57c57ab00 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.618+0000 7f22354145c0 1 -- 172.21.15.81:0/888369947 >> 172.21.15.81:0/888369947 conn(0x55b57d2ea400 msgr2=0x55b57d28d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:55.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.618+0000 7f22354145c0 1 -- 172.21.15.81:0/888369947 shutdown_connections 2023-11-08T16:39:55.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.618+0000 7f22354145c0 1 -- 172.21.15.81:0/888369947 wait complete. 2023-11-08T16:39:55.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.619+0000 7f22354145c0 1 Processor -- start 2023-11-08T16:39:55.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.619+0000 7f22354145c0 1 -- start start 2023-11-08T16:39:55.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22354145c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d3dcc00 0x55b57c57ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22354145c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dd000 0x55b57c57c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22354145c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d3dd400 0x55b57c57c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22354145c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57c5dec80 con 0x55b57d3dd000 2023-11-08T16:39:55.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22354145c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57c558a00 con 0x55b57d3dd400 2023-11-08T16:39:55.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22354145c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b57c559180 con 0x55b57d3dcc00 2023-11-08T16:39:55.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22330ee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dd000 0x55b57c57c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22330ee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dd000 0x55b57c57c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38410/0 (socket says 172.21.15.81:38410) 2023-11-08T16:39:55.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 learned_addr learned my addr 172.21.15.81:0/1593609689 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:55.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d3dd400 0x55b57c57c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d3dcc00 msgr2=0x55b57c57ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22330ee640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d3dcc00 0x55b57c57ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d3dd400 msgr2=0x55b57c57c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.620+0000 7f22330ee640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d3dd400 0x55b57c57c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.621+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b57d2f2340 con 0x55b57d3dd000 2023-11-08T16:39:55.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.621+0000 7f22330ee640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dd000 0x55b57c57c100 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x55b57d328ea0 tx=0x55b57d2e4240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.621+0000 7f222d0e2640 1 -- 172.21.15.81:0/1593609689 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b57c556dc0 con 0x55b57d3dd000 2023-11-08T16:39:55.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.621+0000 7f222d0e2640 1 -- 172.21.15.81:0/1593609689 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b57d2f1680 con 0x55b57d3dd000 2023-11-08T16:39:55.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.621+0000 7f222d0e2640 1 -- 172.21.15.81:0/1593609689 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b57c556f20 con 0x55b57d3dd000 2023-11-08T16:39:55.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.621+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b57d2f24e0 con 0x55b57d3dd000 2023-11-08T16:39:55.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.621+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b57d2f2340 con 0x55b57d3dd000 2023-11-08T16:39:55.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.621+0000 7f22354145c0 4 datascan.init: resolving metadata pool 13 2023-11-08T16:39:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.623+0000 7f222d0e2640 1 -- 172.21.15.81:0/1593609689 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55b57c56c840 con 0x55b57d3dd000 2023-11-08T16:39:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.624+0000 7f222d0e2640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b57c874000 0x55b57c57d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.624+0000 7f222d0e2640 1 -- 172.21.15.81:0/1593609689 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x55b57c5d0a80 con 0x55b57d3dd000 2023-11-08T16:39:55.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.624+0000 7f22354145c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:39:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.624+0000 7f22354145c0 4 datascan.main: resolving metadata pool 13 2023-11-08T16:39:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.624+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b57d3dd800 0x55b57c57cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.624+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57d3dd800 2023-11-08T16:39:55.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.624+0000 7f22338ef640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b57c874000 0x55b57c57d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.625+0000 7f22338ef640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b57c874000 0x55b57c57d700 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55b57d3ecc90 tx=0x55b57d2e4540 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.625+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b57d3dd800 0x55b57c57cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.626+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b57d3dd800 0x55b57c57cc00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.627+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(1 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.627+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b57c874400 0x55b57c886000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.627+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:2 13.10 13:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874400 2023-11-08T16:39:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.627+0000 7f22338ef640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b57c874400 0x55b57c886000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.628+0000 7f22338ef640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b57c874400 0x55b57c886000 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.628+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(2 [pgnls start_epoch 110 out=81b] v110'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.628+0000 7f22338ef640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b57c875000 0x55b57c57a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.628+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:3 13.8 13:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875000 2023-11-08T16:39:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.628+0000 7f22330ee640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b57c875000 0x55b57c57a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.629+0000 7f22330ee640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b57c875000 0x55b57c57a580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.629+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(3 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875000 2023-11-08T16:39:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.629+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:4 13.18 13:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57d3dd800 2023-11-08T16:39:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.630+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(4 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.630+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b57c874800 0x55b57c886580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.630+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:5 13.4 13:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874800 2023-11-08T16:39:55.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.631+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b57c874800 0x55b57c886580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.632+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b57c874800 0x55b57c886580 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.633+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(5 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.633+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b57c874c00 0x55b57c886b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.633+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:6 13.14 13:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874c00 2023-11-08T16:39:55.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.633+0000 7f22338ef640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b57c874c00 0x55b57c886b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.634+0000 7f22338ef640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b57c874c00 0x55b57c886b00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.634+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(6 [pgnls start_epoch 110 out=125b] v110'50 uv50 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874c00 2023-11-08T16:39:55.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.634+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:7 13.c 13:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874400 2023-11-08T16:39:55.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.635+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(7 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.635+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:8 13.1c 13:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874800 2023-11-08T16:39:55.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.636+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(8 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.636+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b57c875800 0x55b57c887080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.636+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:9 13.2 13:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875800 2023-11-08T16:39:55.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.636+0000 7f22330ee640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b57c875800 0x55b57c887080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.637+0000 7f22330ee640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b57c875800 0x55b57c887080 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.638+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(9 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875800 2023-11-08T16:39:55.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.638+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:10 13.12 13:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874c00 2023-11-08T16:39:55.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.638+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(10 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874c00 2023-11-08T16:39:55.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.639+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:11 13.a 13:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875000 2023-11-08T16:39:55.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.639+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(11 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875000 2023-11-08T16:39:55.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.639+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:12 13.1a 13:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874400 2023-11-08T16:39:55.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.639+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(12 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.639+0000 7f22338ef640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b57c875400 0x55b57c888100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.639+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:13 13.6 13:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875400 2023-11-08T16:39:55.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.640+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b57c875400 0x55b57c888100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.641+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b57c875400 0x55b57c888100 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.642+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(13 [pgnls start_epoch 110 out=49b] v109'8 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.642+0000 7f22328ed640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b57c875c00 0x55b57c887600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.642+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:14 13.16 13:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875c00 2023-11-08T16:39:55.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.643+0000 7f22338ef640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b57c875c00 0x55b57c887600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.643+0000 7f22338ef640 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b57c875c00 0x55b57c887600 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:55.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.644+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(14 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c875c00 2023-11-08T16:39:55.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.644+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:15 13.e 13:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57d3dd800 2023-11-08T16:39:55.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.645+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(15 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.645+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:16 13.1e 13:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875400 2023-11-08T16:39:55.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.645+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(16 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.645+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:17 13.1 13:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874800 2023-11-08T16:39:55.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.646+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(17 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.646+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:18 13.11 13:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874400 2023-11-08T16:39:55.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.647+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(18 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.647+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:19 13.9 13:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875800 2023-11-08T16:39:55.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.648+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(19 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875800 2023-11-08T16:39:55.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.648+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:20 13.19 13:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875c00 2023-11-08T16:39:55.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.648+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(20 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c875c00 2023-11-08T16:39:55.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.648+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:21 13.5 13:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875400 2023-11-08T16:39:55.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.649+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(21 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.649+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:22 13.15 13:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875400 2023-11-08T16:39:55.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.650+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(22 [pgnls start_epoch 110 out=73b] v109'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.650+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:23 13.d 13:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875400 2023-11-08T16:39:55.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.650+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(23 [pgnls start_epoch 110 out=74b] v110'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.650+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:24 13.1d 13:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57d3dd800 2023-11-08T16:39:55.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.652+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(24 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.652+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:25 13.3 13:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874c00 2023-11-08T16:39:55.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.652+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(25 [pgnls start_epoch 110 out=71b] v110'2 uv2 ondisk = 1) v8 ==== 144+0+71 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874c00 2023-11-08T16:39:55.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.652+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:26 13.13 13:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57d3dd800 2023-11-08T16:39:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.653+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(26 [pgnls start_epoch 110 out=98b] v110'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.653+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:27 13.b 13:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875800 2023-11-08T16:39:55.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.654+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(27 [pgnls start_epoch 110 out=76b] v110'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875800 2023-11-08T16:39:55.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.654+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:28 13.1b 13:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875400 2023-11-08T16:39:55.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.655+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(28 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.655+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:29 13.7 13:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57d3dd800 2023-11-08T16:39:55.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.656+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(29 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.656+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:30 13.17 13:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875800 2023-11-08T16:39:55.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.657+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(30 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875800 2023-11-08T16:39:55.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.657+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:31 13.f 13:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874800 2023-11-08T16:39:55.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.660+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(31 [pgnls start_epoch 110 out=73b] v110'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.660+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:32 13.1f 13:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874800 2023-11-08T16:39:55.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.660+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(32 [pgnls start_epoch 110 out=101b] v110'13 uv13 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.660+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-11-08T16:39:55.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.660+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:33 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569c00 con 0x55b57c874400 2023-11-08T16:39:55.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=3582b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+3582 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-11-08T16:39:55.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-08T16:39:55.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-11-08T16:39:55.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-08T16:39:55.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-11-08T16:39:55.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-08T16:39:55.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-11-08T16:39:55.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-08T16:39:55.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-11-08T16:39:55.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-08T16:39:55.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object 4.00000000 2023-11-08T16:39:55.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.661+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:34 13.3 13:c0ce80d4:::4.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8000 con 0x55b57c874c00 2023-11-08T16:39:55.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.662+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(34 4.00000000 [omap-get-vals out=513b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+513 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874c00 2023-11-08T16:39:55.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.662+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.662+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-11-08T16:39:55.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.662+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-08T16:39:55.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.662+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-11-08T16:39:55.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.662+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:35 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8380 con 0x55b57d3dd800 2023-11-08T16:39:55.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.663+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(35 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.663+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-11-08T16:39:55.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.663+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-08T16:39:55.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.663+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-11-08T16:39:55.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.663+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-08T16:39:55.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.663+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-11-08T16:39:55.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.663+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-08T16:39:55.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.663+0000 7f22354145c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-11-08T16:39:55.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.663+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:36 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8700 con 0x55b57c874800 2023-11-08T16:39:55.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.664+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(36 1.00000000 [omap-get-vals out=1016b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+1016 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.664+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.664+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.664+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:37 13.0 13:00000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57d3dd800 2023-11-08T16:39:55.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.665+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(37 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.665+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:38 13.10 13:08000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874400 2023-11-08T16:39:55.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.666+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(38 [pgnls start_epoch 110 out=81b] v110'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.666+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:39 13.8 13:10000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875000 2023-11-08T16:39:55.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.667+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(39 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875000 2023-11-08T16:39:55.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.667+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:40 13.18 13:18000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57d3dd800 2023-11-08T16:39:55.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.668+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 9 ==== osd_op_reply(40 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.668+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:41 13.4 13:20000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874800 2023-11-08T16:39:55.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.669+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(41 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.669+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:42 13.14 13:28000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874c00 2023-11-08T16:39:55.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.670+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(42 [pgnls start_epoch 110 out=125b] v110'50 uv50 ondisk = 1) v8 ==== 144+0+125 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874c00 2023-11-08T16:39:55.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.670+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:43 13.c 13:30000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874400 2023-11-08T16:39:55.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.670+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(43 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.670+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:44 13.1c 13:38000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874800 2023-11-08T16:39:55.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.671+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(44 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.671+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:45 13.2 13:40000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875800 2023-11-08T16:39:55.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.673+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(45 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875800 2023-11-08T16:39:55.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.673+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:46 13.12 13:48000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874c00 2023-11-08T16:39:55.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.673+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(46 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874c00 2023-11-08T16:39:55.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.673+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:47 13.a 13:50000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875000 2023-11-08T16:39:55.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.674+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(47 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875000 2023-11-08T16:39:55.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.674+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:48 13.1a 13:58000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874400 2023-11-08T16:39:55.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.675+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(48 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.675+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:49 13.6 13:60000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875400 2023-11-08T16:39:55.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.676+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(49 [pgnls start_epoch 110 out=49b] v109'8 uv8 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.676+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:50 13.16 13:68000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875c00 2023-11-08T16:39:55.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.677+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(50 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c875c00 2023-11-08T16:39:55.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.677+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:51 13.e 13:70000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57d3dd800 2023-11-08T16:39:55.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.678+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 10 ==== osd_op_reply(51 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.678+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:52 13.1e 13:78000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875400 2023-11-08T16:39:55.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.679+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(52 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.679+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:53 13.1 13:80000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874800 2023-11-08T16:39:55.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.680+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(53 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.680+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:54 13.11 13:88000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874400 2023-11-08T16:39:55.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.680+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(54 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.680+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:55 13.9 13:90000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875800 2023-11-08T16:39:55.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.681+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(55 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875800 2023-11-08T16:39:55.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.681+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:56 13.19 13:98000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875c00 2023-11-08T16:39:55.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.682+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(56 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c875c00 2023-11-08T16:39:55.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.682+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:57 13.5 13:a0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875400 2023-11-08T16:39:55.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.683+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(57 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.683+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:58 13.15 13:a8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875400 2023-11-08T16:39:55.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.684+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(58 [pgnls start_epoch 110 out=73b] v109'6 uv6 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.684+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:59 13.d 13:b0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875400 2023-11-08T16:39:55.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.685+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(59 [pgnls start_epoch 110 out=74b] v110'3 uv3 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.685+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:60 13.1d 13:b8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57d3dd800 2023-11-08T16:39:55.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.686+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 11 ==== osd_op_reply(60 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.686+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:61 13.3 13:c0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874c00 2023-11-08T16:39:55.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.687+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(61 [pgnls start_epoch 110 out=71b] v110'2 uv2 ondisk = 1) v8 ==== 144+0+71 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874c00 2023-11-08T16:39:55.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.687+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:62 13.13 13:c8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57d3dd800 2023-11-08T16:39:55.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.688+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 12 ==== osd_op_reply(62 [pgnls start_epoch 110 out=98b] v110'7 uv7 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.689+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:63 13.b 13:d0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875800 2023-11-08T16:39:55.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.690+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(63 [pgnls start_epoch 110 out=76b] v110'6 uv5 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875800 2023-11-08T16:39:55.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.690+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:64 13.1b 13:d8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875400 2023-11-08T16:39:55.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.690+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(64 [pgnls start_epoch 110 out=49b] v110'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.691+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:65 13.7 13:e0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57d3dd800 2023-11-08T16:39:55.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.691+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 13 ==== osd_op_reply(65 [pgnls start_epoch 110 out=49b] v109'3 uv3 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.692+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:66 13.17 13:e8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c875800 2023-11-08T16:39:55.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.693+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(66 [pgnls start_epoch 110 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b57d3cefc0 con 0x55b57c875800 2023-11-08T16:39:55.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.693+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:67 13.f 13:f0000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874800 2023-11-08T16:39:55.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.695+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(67 [pgnls start_epoch 110 out=73b] v110'4 uv4 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.695+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:68 13.1f 13:f8000000::::head [pgnls start_epoch 110 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874800 2023-11-08T16:39:55.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 11 ==== osd_op_reply(68 [pgnls start_epoch 110 out=101b] v110'13 uv13 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-11-08T16:39:55.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:69 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8380 con 0x55b57c874400 2023-11-08T16:39:55.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(69 10000000000.00000000 [omap-get-vals out=3582b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+3582 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-11-08T16:39:55.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-11-08T16:39:55.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-11-08T16:39:55.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-11-08T16:39:55.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-11-08T16:39:55.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: adding 0x100000001fa for future processing to fix dnfirst 2023-11-08T16:39:55.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 20 datascan.scan_links: adding 0x100000001fb for future processing to fix dnfirst 2023-11-08T16:39:55.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-11-08T16:39:55.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-08T16:39:55.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-11-08T16:39:55.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-08T16:39:55.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-11-08T16:39:55.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-08T16:39:55.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-11-08T16:39:55.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-08T16:39:55.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-11-08T16:39:55.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-08T16:39:55.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object 4.00000000 2023-11-08T16:39:55.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.696+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:70 13.3 13:c0ce80d4:::4.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8700 con 0x55b57c874c00 2023-11-08T16:39:55.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.697+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(70 4.00000000 [omap-get-vals out=513b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+513 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874c00 2023-11-08T16:39:55.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.697+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.697+0000 7f22354145c0 20 datascan.scan_links: adding 0x100000001fc for future processing to fix dnfirst 2023-11-08T16:39:55.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.697+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-11-08T16:39:55.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.697+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-08T16:39:55.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.697+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-11-08T16:39:55.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.697+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:71 13.13 13:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8000 con 0x55b57d3dd800 2023-11-08T16:39:55.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.699+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 14 ==== osd_op_reply(71 100.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.699+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-11-08T16:39:55.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.699+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-08T16:39:55.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.699+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-11-08T16:39:55.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.699+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-08T16:39:55.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.699+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-11-08T16:39:55.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.699+0000 7f22354145c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-08T16:39:55.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.699+0000 7f22354145c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-11-08T16:39:55.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.699+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:72 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874800 2023-11-08T16:39:55.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 12 ==== osd_op_reply(72 1.00000000 [omap-get-vals out=1016b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+1016 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22354145c0 20 datascan.scan_links: adding 0x4 for future processing to fix dnfirst 2023-11-08T16:39:55.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22354145c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:39:55.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22354145c0 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-11-08T16:39:55.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22354145c0 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-11-08T16:39:55.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22354145c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-11-08T16:39:55.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22354145c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-11-08T16:39:55.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22354145c0 10 datascan.scan_links: processing 10 injected_inos 2023-11-08T16:39:55.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22354145c0 10 datascan.scan_links: handling injected_ino 0x4 2023-11-08T16:39:55.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22354145c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:lost+found 2023-11-08T16:39:55.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.700+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:73 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569c00 con 0x55b57c874800 2023-11-08T16:39:55.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.702+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 13 ==== osd_op_reply(73 1.00000000 [omap-get-vals-by-keys out=511b] v0'0 uv13 ondisk = 0) v8 ==== 154+0+511 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.702+0000 7f22354145c0 20 datascan.read_dentry: oid=1.00000000 dname=lost+found frag=*, r=0 2023-11-08T16:39:55.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.702+0000 7f22354145c0 20 datascan.scan_links: first is now 2 2023-11-08T16:39:55.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.702+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:74 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=511b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8a80 con 0x55b57c874800 2023-11-08T16:39:55.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.706+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 14 ==== osd_op_reply(74 1.00000000 [omap-set-vals] v110'14 uv14 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.706+0000 7f22354145c0 20 datascan.inject_linkage: Injected dentry 0x0x1/lost+found pointing to 0x0x4 2023-11-08T16:39:55.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.706+0000 7f22354145c0 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-11-08T16:39:55.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.706+0000 7f22354145c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:stripey 2023-11-08T16:39:55.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.706+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:75 13.1f 13:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8000 con 0x55b57c874800 2023-11-08T16:39:55.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.708+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 15 ==== osd_op_reply(75 1.00000000 [omap-get-vals-by-keys out=508b] v0'0 uv14 ondisk = 0) v8 ==== 154+0+508 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.708+0000 7f22354145c0 20 datascan.read_dentry: oid=1.00000000 dname=stripey frag=*, r=0 2023-11-08T16:39:55.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.708+0000 7f22354145c0 20 datascan.scan_links: first is now 2 2023-11-08T16:39:55.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.708+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:76 13.1f 13:ff5b34d6:::1.00000000:head [omap-set-vals in=508b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8e00 con 0x55b57c874800 2023-11-08T16:39:55.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.712+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.6 v2:172.21.15.93:6816/607960134 16 ==== osd_op_reply(76 1.00000000 [omap-set-vals] v110'15 uv15 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c874800 2023-11-08T16:39:55.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.712+0000 7f22354145c0 20 datascan.inject_linkage: Injected dentry 0x0x1/stripey pointing to 0x0x10000000000 2023-11-08T16:39:55.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.712+0000 7f22354145c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-11-08T16:39:55.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.712+0000 7f22354145c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_0 2023-11-08T16:39:55.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.712+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:77 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a9180 con 0x55b57c874400 2023-11-08T16:39:55.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.713+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(77 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.713+0000 7f22354145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_0 frag=*, r=0 2023-11-08T16:39:55.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.713+0000 7f22354145c0 20 datascan.scan_links: first is now 2 2023-11-08T16:39:55.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.713+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:78 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874400 2023-11-08T16:39:55.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.716+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 12 ==== osd_op_reply(78 10000000000.00000000 [omap-set-vals] v110'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.716+0000 7f22354145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_0 pointing to 0x0x10000000001 2023-11-08T16:39:55.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.716+0000 7f22354145c0 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-11-08T16:39:55.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.716+0000 7f22354145c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_1 2023-11-08T16:39:55.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.716+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:79 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569c00 con 0x55b57c874400 2023-11-08T16:39:55.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.717+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 13 ==== osd_op_reply(79 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.717+0000 7f22354145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_1 frag=*, r=0 2023-11-08T16:39:55.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.717+0000 7f22354145c0 20 datascan.scan_links: first is now 2 2023-11-08T16:39:55.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.717+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:80 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8380 con 0x55b57c874400 2023-11-08T16:39:55.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.720+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 14 ==== osd_op_reply(80 10000000000.00000000 [omap-set-vals] v110'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.721+0000 7f22354145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_1 pointing to 0x0x100000001f6 2023-11-08T16:39:55.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.721+0000 7f22354145c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-11-08T16:39:55.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.721+0000 7f22354145c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_2 2023-11-08T16:39:55.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.721+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:81 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8700 con 0x55b57c874400 2023-11-08T16:39:55.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.721+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 15 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.721+0000 7f22354145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_2 frag=*, r=0 2023-11-08T16:39:55.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.721+0000 7f22354145c0 20 datascan.scan_links: first is now 2 2023-11-08T16:39:55.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.721+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:82 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a9180 con 0x55b57c874400 2023-11-08T16:39:55.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.724+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 16 ==== osd_op_reply(82 10000000000.00000000 [omap-set-vals] v110'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.724+0000 7f22354145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_2 pointing to 0x0x100000001f7 2023-11-08T16:39:55.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.725+0000 7f22354145c0 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-11-08T16:39:55.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.725+0000 7f22354145c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_3 2023-11-08T16:39:55.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.725+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:83 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a9500 con 0x55b57c874400 2023-11-08T16:39:55.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.725+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 17 ==== osd_op_reply(83 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.725+0000 7f22354145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_3 frag=*, r=0 2023-11-08T16:39:55.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.725+0000 7f22354145c0 20 datascan.scan_links: first is now 2 2023-11-08T16:39:55.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.726+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:84 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a9880 con 0x55b57c874400 2023-11-08T16:39:55.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.729+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 18 ==== osd_op_reply(84 10000000000.00000000 [omap-set-vals] v110'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.729+0000 7f22354145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_3 pointing to 0x0x100000001f8 2023-11-08T16:39:55.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.729+0000 7f22354145c0 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-11-08T16:39:55.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.729+0000 7f22354145c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_4 2023-11-08T16:39:55.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.729+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:85 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a9c00 con 0x55b57c874400 2023-11-08T16:39:55.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.730+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 19 ==== osd_op_reply(85 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.730+0000 7f22354145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_4 frag=*, r=0 2023-11-08T16:39:55.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.730+0000 7f22354145c0 20 datascan.scan_links: first is now 2 2023-11-08T16:39:55.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.730+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:86 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8380 con 0x55b57c874400 2023-11-08T16:39:55.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.733+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 20 ==== osd_op_reply(86 10000000000.00000000 [omap-set-vals] v110'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.733+0000 7f22354145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_4 pointing to 0x0x100000001f9 2023-11-08T16:39:55.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.733+0000 7f22354145c0 10 datascan.scan_links: handling injected_ino 0x100000001fa 2023-11-08T16:39:55.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.733+0000 7f22354145c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_5 2023-11-08T16:39:55.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.733+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:87 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8700 con 0x55b57c874400 2023-11-08T16:39:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.734+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 21 ==== osd_op_reply(87 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.734+0000 7f22354145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_5 frag=*, r=0 2023-11-08T16:39:55.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.734+0000 7f22354145c0 20 datascan.scan_links: first is now 2 2023-11-08T16:39:55.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.734+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:88 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c874400 2023-11-08T16:39:55.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.737+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 22 ==== osd_op_reply(88 10000000000.00000000 [omap-set-vals] v110'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.737+0000 7f22354145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_5 pointing to 0x0x100000001fa 2023-11-08T16:39:55.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.737+0000 7f22354145c0 10 datascan.scan_links: handling injected_ino 0x100000001fb 2023-11-08T16:39:55.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.737+0000 7f22354145c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:flushed_file_6 2023-11-08T16:39:55.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.737+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:89 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=27b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569c00 con 0x55b57c874400 2023-11-08T16:39:55.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.738+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 23 ==== osd_op_reply(89 10000000000.00000000 [omap-get-vals-by-keys out=515b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+515 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.738+0000 7f22354145c0 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed_file_6 frag=*, r=0 2023-11-08T16:39:55.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.738+0000 7f22354145c0 20 datascan.scan_links: first is now 2 2023-11-08T16:39:55.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.738+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:90 13.10 13:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=515b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a9180 con 0x55b57c874400 2023-11-08T16:39:55.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.742+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.1 v2:172.21.15.81:6800/2060798739 24 ==== osd_op_reply(90 10000000000.00000000 [omap-set-vals] v110'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874400 2023-11-08T16:39:55.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.742+0000 7f22354145c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/flushed_file_6 pointing to 0x0x100000001fb 2023-11-08T16:39:55.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.742+0000 7f22354145c0 10 datascan.scan_links: handling injected_ino 0x100000001fc 2023-11-08T16:39:55.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.742+0000 7f22354145c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x4:100000001fc 2023-11-08T16:39:55.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.742+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:91 13.3 13:c0ce80d4:::4.00000000:head [omap-get-vals-by-keys in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a9500 con 0x55b57c874c00 2023-11-08T16:39:55.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.743+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(91 4.00000000 [omap-get-vals-by-keys out=512b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+512 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874c00 2023-11-08T16:39:55.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.743+0000 7f22354145c0 20 datascan.read_dentry: oid=4.00000000 dname=100000001fc frag=*, r=0 2023-11-08T16:39:55.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.743+0000 7f22354145c0 20 datascan.scan_links: first is now 2 2023-11-08T16:39:55.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.743+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:92 13.3 13:c0ce80d4:::4.00000000:head [omap-set-vals in=512b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a9880 con 0x55b57c874c00 2023-11-08T16:39:55.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.747+0000 7f22338ef640 1 -- 172.21.15.81:0/1593609689 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(92 4.00000000 [omap-set-vals] v110'3 uv3 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55b57d3ced80 con 0x55b57c874c00 2023-11-08T16:39:55.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.747+0000 7f22354145c0 20 datascan.inject_linkage: Injected dentry 0x0x4/100000001fc pointing to 0x0x100000001fc 2023-11-08T16:39:55.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.747+0000 7f22354145c0 10 datascan.scan_links: updating inotable 2023-11-08T16:39:55.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.747+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:93 13.13 13:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a9c00 con 0x55b57d3dd800 2023-11-08T16:39:55.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.748+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 15 ==== osd_op_reply(93 mds0_inotable [read 0~34 out=34b] v0'0 uv6 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.748+0000 7f22354145c0 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-11-08T16:39:55.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.748+0000 7f22354145c0 10 mds.0.inotable: skip_inos now [0x100000001fd~0xfffffffe03] 2023-11-08T16:39:55.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.748+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:94 13.13 13:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c8a8380 con 0x55b57d3dd800 2023-11-08T16:39:55.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.752+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.7 v2:172.21.15.93:6818/433691658 16 ==== osd_op_reply(94 mds0_inotable [writefull 0~34] v110'8 uv8 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57d3dd800 2023-11-08T16:39:55.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.752+0000 7f22354145c0 10 datascan.scan_links: updating snaptable 2023-11-08T16:39:55.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.752+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:95 13.d 13:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e110) v8 -- 0x55b57c569880 con 0x55b57c875400 2023-11-08T16:39:55.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22328ed640 1 -- 172.21.15.81:0/1593609689 <== osd.4 v2:172.21.15.93:6800/3259756279 13 ==== osd_op_reply(95 mds_snaptable [read 0~70 out=70b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55b57d3ceb40 con 0x55b57c875400 2023-11-08T16:39:55.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b57c874c00 msgr2=0x55b57c886b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b57c874c00 0x55b57c886b00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b57c874400 msgr2=0x55b57c886000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b57c874400 0x55b57c886000 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b57c875000 msgr2=0x55b57c57a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b57c875000 0x55b57c57a580 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b57c875c00 msgr2=0x55b57c887600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b57c875c00 0x55b57c887600 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 reap_dead start 2023-11-08T16:39:55.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b57c875400 msgr2=0x55b57c888100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b57c875400 0x55b57c888100 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b57c875800 msgr2=0x55b57c887080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b57c875800 0x55b57c887080 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b57c874800 msgr2=0x55b57c886580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b57c874800 0x55b57c886580 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b57d3dd800 msgr2=0x55b57c57cc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b57d3dd800 0x55b57c57cc00 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22330ee640 1 -- 172.21.15.81:0/1593609689 reap_dead start 2023-11-08T16:39:55.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b57c874000 msgr2=0x55b57c57d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b57c874000 0x55b57c57d700 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x55b57d3ecc90 tx=0x55b57d2e4540 comp rx=0 tx=0).stop 2023-11-08T16:39:55.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dd000 msgr2=0x55b57c57c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dd000 0x55b57c57c100 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x55b57d328ea0 tx=0x55b57d2e4240 comp rx=0 tx=0).stop 2023-11-08T16:39:55.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 shutdown_connections 2023-11-08T16:39:55.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b57c874000 0x55b57c57d700 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 --2- 172.21.15.81:0/1593609689 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57d3dd000 0x55b57c57c100 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.755+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 >> 172.21.15.81:0/1593609689 conn(0x55b57d2ea400 msgr2=0x55b57d28d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:55.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.756+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 shutdown_connections 2023-11-08T16:39:55.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.756+0000 7f22354145c0 1 -- 172.21.15.81:0/1593609689 wait complete. 2023-11-08T16:39:55.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.757+0000 7f22354145c0 1 -- 172.21.15.81:0/693984581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57c511c00 msgr2=0x55b57c57bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:55.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.757+0000 7f22354145c0 1 --2- 172.21.15.81:0/693984581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57c511c00 0x55b57c57bb80 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x55b57d329620 tx=0x55b57d2e4600 comp rx=0 tx=0).stop 2023-11-08T16:39:55.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.757+0000 7f22354145c0 1 -- 172.21.15.81:0/693984581 shutdown_connections 2023-11-08T16:39:55.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.757+0000 7f22354145c0 1 --2- 172.21.15.81:0/693984581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b57d2ea800 0x55b57c57b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.757+0000 7f22354145c0 1 --2- 172.21.15.81:0/693984581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b57c511c00 0x55b57c57bb80 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.757+0000 7f22354145c0 1 --2- 172.21.15.81:0/693984581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b57d2ea000 0x55b57c57b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.757+0000 7f22354145c0 1 -- 172.21.15.81:0/693984581 >> 172.21.15.81:0/693984581 conn(0x55b57c511000 msgr2=0x55b57d28b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:55.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.757+0000 7f22354145c0 1 -- 172.21.15.81:0/693984581 shutdown_connections 2023-11-08T16:39:55.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.757+0000 7f22354145c0 1 -- 172.21.15.81:0/693984581 wait complete. 2023-11-08T16:39:55.772 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.262132, result: 2023-11-08T16:39:55.773 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds repaired 0 2023-11-08T16:39:55.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.991+0000 7fdcfcbdc640 1 Processor -- start 2023-11-08T16:39:55.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.991+0000 7fdcfcbdc640 1 -- start start 2023-11-08T16:39:55.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.992+0000 7fdcfcbdc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fdcf806cf10 0x7fdcf8161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.992+0000 7fdcfcbdc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdcf80ca1a0 0x7fdcf80caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:55.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.992+0000 7fdcfcbdc640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcf80cd580 con 0x7fdcf80cc670 2023-11-08T16:39:55.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.992+0000 7fdcfcbdc640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf80c93e0 con 0x7fdcf80ca1a0 2023-11-08T16:39:55.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.992+0000 7fdcfcbdc640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcf80c96c0 con 0x7fdcf806cf10 2023-11-08T16:39:55.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.993+0000 7fdceffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdcf80ca1a0 0x7fdcf80caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.993+0000 7fdceffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdcf80ca1a0 0x7fdcf80caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38414/0 (socket says 172.21.15.81:38414) 2023-11-08T16:39:55.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.993+0000 7fdceffff640 1 -- 172.21.15.81:0/881485166 learned_addr learned my addr 172.21.15.81:0/881485166 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:55.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.993+0000 7fdceffff640 1 -- 172.21.15.81:0/881485166 >> v2:172.21.15.93:3300/0 conn(0x7fdcf806cf10 msgr2=0x7fdcf8161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:55.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.993+0000 7fdcf659c640 1 --2- 172.21.15.81:0/881485166 >> v2:172.21.15.93:3300/0 conn(0x7fdcf806cf10 0x7fdcf8161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:55.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.993+0000 7fdceffff640 1 --2- 172.21.15.81:0/881485166 >> v2:172.21.15.93:3300/0 conn(0x7fdcf806cf10 0x7fdcf8161630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:55.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.993+0000 7fdceffff640 1 -- 172.21.15.81:0/881485166 >> v1:172.21.15.93:6789/0 conn(0x7fdcf80cc670 legacy=0x7fdcf80cca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:55.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.993+0000 7fdceffff640 1 -- 172.21.15.81:0/881485166 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcf80c99a0 con 0x7fdcf80ca1a0 2023-11-08T16:39:55.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.994+0000 7fdceffff640 1 --2- 172.21.15.81:0/881485166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf80ca1a0 0x7fdcf80caeb0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80726d0 tx=0x7fdce002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5d6d006dc92057 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:56.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.995+0000 7fdcf5d9b640 1 -- 172.21.15.81:0/881485166 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0031870 con 0x7fdcf80ca1a0 2023-11-08T16:39:56.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.995+0000 7fdcf5d9b640 1 -- 172.21.15.81:0/881485166 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdce00319d0 con 0x7fdcf80ca1a0 2023-11-08T16:39:56.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.995+0000 7fdcf5d9b640 1 -- 172.21.15.81:0/881485166 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdce0031d60 con 0x7fdcf80ca1a0 2023-11-08T16:39:56.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.995+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/881485166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf80ca1a0 msgr2=0x7fdcf80caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:56.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.995+0000 7fdcfcbdc640 1 --2- 172.21.15.81:0/881485166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf80ca1a0 0x7fdcf80caeb0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fdcf80726d0 tx=0x7fdce002b910 comp rx=0 tx=0).stop 2023-11-08T16:39:56.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.995+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/881485166 shutdown_connections 2023-11-08T16:39:56.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.996+0000 7fdcfcbdc640 1 --2- 172.21.15.81:0/881485166 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf80ca1a0 0x7fdcf80caeb0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.996+0000 7fdcfcbdc640 1 --2- 172.21.15.81:0/881485166 >> v2:172.21.15.93:3300/0 conn(0x7fdcf806cf10 0x7fdcf8161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.996+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/881485166 >> 172.21.15.81:0/881485166 conn(0x7fdcf80ceaa0 msgr2=0x7fdcf80cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:56.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.996+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/881485166 shutdown_connections 2023-11-08T16:39:56.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.996+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/881485166 wait complete. 2023-11-08T16:39:56.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.997+0000 7fdcfcbdc640 1 Processor -- start 2023-11-08T16:39:56.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.997+0000 7fdcfcbdc640 1 -- start start 2023-11-08T16:39:56.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.998+0000 7fdcfcbdc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdcf806cf10 0x7fdcf814d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:56.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.998+0000 7fdcfcbdc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcf80ca1a0 0x7fdcf80b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:56.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.998+0000 7fdcfcbdc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf80cc670 0x7fdcf80b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:56.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.998+0000 7fdcfcbdc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf814dbc0 con 0x7fdcf80cc670 2023-11-08T16:39:56.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.998+0000 7fdcfcbdc640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80c43d0 con 0x7fdcf806cf10 2023-11-08T16:39:56.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.998+0000 7fdcfcbdc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf80c46b0 con 0x7fdcf80ca1a0 2023-11-08T16:39:56.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.999+0000 7fdcf6d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf80cc670 0x7fdcf80b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:56.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.999+0000 7fdcf6d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf80cc670 0x7fdcf80b1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38420/0 (socket says 172.21.15.81:38420) 2023-11-08T16:39:56.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.999+0000 7fdcf6d9d640 1 -- 172.21.15.81:0/2527287952 learned_addr learned my addr 172.21.15.81:0/2527287952 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:56.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.999+0000 7fdcf659c640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdcf806cf10 0x7fdcf814d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:56.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.999+0000 7fdceffff640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcf80ca1a0 0x7fdcf80b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:56.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.999+0000 7fdcf6d9d640 1 -- 172.21.15.81:0/2527287952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcf80ca1a0 msgr2=0x7fdcf80b1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:56.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.999+0000 7fdcf6d9d640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcf80ca1a0 0x7fdcf80b1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.999+0000 7fdcf6d9d640 1 -- 172.21.15.81:0/2527287952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdcf806cf10 msgr2=0x7fdcf814d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:56.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.999+0000 7fdcf6d9d640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdcf806cf10 0x7fdcf814d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:55.999+0000 7fdcf6d9d640 1 -- 172.21.15.81:0/2527287952 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce00094d0 con 0x7fdcf80cc670 2023-11-08T16:39:56.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.000+0000 7fdcf6d9d640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf80cc670 0x7fdcf80b1bc0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c780 tx=0x7fdcf000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:56.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.000+0000 7fdceeffd640 1 -- 172.21.15.81:0/2527287952 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf0004b60 con 0x7fdcf80cc670 2023-11-08T16:39:56.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.000+0000 7fdceeffd640 1 -- 172.21.15.81:0/2527287952 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf000ce90 con 0x7fdcf80cc670 2023-11-08T16:39:56.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.000+0000 7fdceeffd640 1 -- 172.21.15.81:0/2527287952 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000a850 con 0x7fdcf80cc670 2023-11-08T16:39:56.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.000+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/2527287952 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcf80c4990 con 0x7fdcf80cc670 2023-11-08T16:39:56.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.000+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/2527287952 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcf814d680 con 0x7fdcf80cc670 2023-11-08T16:39:56.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.001+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/2527287952 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcf80ca5b0 con 0x7fdcf80cc670 2023-11-08T16:39:56.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.002+0000 7fdceeffd640 1 -- 172.21.15.81:0/2527287952 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdcf000a9b0 con 0x7fdcf80cc670 2023-11-08T16:39:56.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.003+0000 7fdceeffd640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdcd806ae70 0x7fdcd806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:56.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.003+0000 7fdceeffd640 1 -- 172.21.15.81:0/2527287952 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x7fdcf0089a80 con 0x7fdcf80cc670 2023-11-08T16:39:56.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.003+0000 7fdcf659c640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdcd806ae70 0x7fdcd806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:56.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.004+0000 7fdcf659c640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdcd806ae70 0x7fdcd806d330 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdce800a8b0 tx=0x7fdce8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:56.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.008+0000 7fdceeffd640 1 -- 172.21.15.81:0/2527287952 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcf0054860 con 0x7fdcf80cc670 2023-11-08T16:39:56.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.269+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/2527287952 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "mds repaired", "role": "0"} v 0) v1 -- 0x7fdcf814dd00 con 0x7fdcf80cc670 2023-11-08T16:39:56.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.652+0000 7fdceeffd640 1 -- 172.21.15.81:0/2527287952 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "mds repaired", "role": "0"}]=0 repaired: restoring rank 5:0 v94) v1 ==== 101+0+0 (secure 0 0 0) 0x7fdcf814dd00 con 0x7fdcf80cc670 2023-11-08T16:39:56.655 INFO:teuthology.orchestra.run.smithi081.stderr:repaired: restoring rank 5:0 2023-11-08T16:39:56.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.655+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/2527287952 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdcd806ae70 msgr2=0x7fdcd806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:56.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.655+0000 7fdcfcbdc640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdcd806ae70 0x7fdcd806d330 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdce800a8b0 tx=0x7fdce8008040 comp rx=0 tx=0).stop 2023-11-08T16:39:56.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.655+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/2527287952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf80cc670 msgr2=0x7fdcf80b1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:56.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.656+0000 7fdcfcbdc640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf80cc670 0x7fdcf80b1bc0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fdcf000c780 tx=0x7fdcf000cc40 comp rx=0 tx=0).stop 2023-11-08T16:39:56.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.656+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/2527287952 shutdown_connections 2023-11-08T16:39:56.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.656+0000 7fdcfcbdc640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdcd806ae70 0x7fdcd806d330 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.656+0000 7fdcfcbdc640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcf80cc670 0x7fdcf80b1bc0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.656+0000 7fdcfcbdc640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcf80ca1a0 0x7fdcf80b1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.656+0000 7fdcfcbdc640 1 --2- 172.21.15.81:0/2527287952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdcf806cf10 0x7fdcf814d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.656+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/2527287952 >> 172.21.15.81:0/2527287952 conn(0x7fdcf80ceaa0 msgr2=0x7fdcf80ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:56.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.656+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/2527287952 shutdown_connections 2023-11-08T16:39:56.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.656+0000 7fdcfcbdc640 1 -- 172.21.15.81:0/2527287952 wait complete. 2023-11-08T16:39:56.671 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:39:56.669+0000 7f43dc8aa640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-08T16:39:56.682 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T16:39:56.683 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T16:39:56.683 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T16:39:56.683 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:39:56.684 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T16:39:56.684 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T16:39:56.684 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T16:39:56.684 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:39:56.685 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T16:39:56.685 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T16:39:56.685 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T16:39:56.685 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:39:56.686 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T16:39:56.686 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T16:39:56.686 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T16:39:56.687 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:39:56.700 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T16:39:56.700 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T16:39:56.701 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T16:39:56.701 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T16:39:56.702 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T16:39:56.703 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T16:39:56.706 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T16:39:56.706 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T16:39:56.747 INFO:tasks.ceph.mds.a:Started 2023-11-08T16:39:56.748 DEBUG:teuthology.parallel:result is None 2023-11-08T16:39:56.752 INFO:tasks.ceph.mds.c:Started 2023-11-08T16:39:56.752 DEBUG:teuthology.parallel:result is None 2023-11-08T16:39:56.754 INFO:tasks.ceph.mds.b:Started 2023-11-08T16:39:56.755 DEBUG:teuthology.parallel:result is None 2023-11-08T16:39:56.762 INFO:tasks.ceph.mds.d:Started 2023-11-08T16:39:56.762 DEBUG:teuthology.parallel:result is None 2023-11-08T16:39:56.763 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:56.904 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:39:56.911 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:39:56.926 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2023-11-08T16:39:56.926 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:39:56.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.956+0000 7f28eca34640 1 Processor -- start 2023-11-08T16:39:56.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.957+0000 7f28eca34640 1 -- start start 2023-11-08T16:39:56.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.958+0000 7f28eca34640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28e80d1af0 0x7f28e80d3920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:56.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.958+0000 7f28eca34640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f28e80b2c30 0x7f28e80b3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:56.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.958+0000 7f28eca34640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28e80b2450 con 0x7f28e815d1a0 2023-11-08T16:39:56.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.958+0000 7f28eca34640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28e80ce800 con 0x7f28e80b2c30 2023-11-08T16:39:56.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.958+0000 7f28eca34640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80ceae0 con 0x7f28e80d1af0 2023-11-08T16:39:56.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.958+0000 7f28e659c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f28e815d1a0 0x7f28e80d13e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56604/0 (socket says 172.21.15.81:56604) 2023-11-08T16:39:56.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.958+0000 7f28e659c640 1 -- 172.21.15.81:0/943754856 learned_addr learned my addr 172.21.15.81:0/943754856 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:56.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.958+0000 7f28e6d9d640 1 --2- 172.21.15.81:0/943754856 >> v2:172.21.15.81:3300/0 conn(0x7f28e80b2c30 0x7f28e80b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:56.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.958+0000 7f28ddd9b640 1 --2- 172.21.15.81:0/943754856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28e80d1af0 0x7f28e80d3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:56.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.959+0000 7f28e5d9b640 1 -- 172.21.15.81:0/943754856 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349300551 0 0) 0x7f28e80b2450 con 0x7f28e815d1a0 2023-11-08T16:39:56.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.959+0000 7f28e5d9b640 1 -- 172.21.15.81:0/943754856 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28d4003520 con 0x7f28e815d1a0 2023-11-08T16:39:56.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.959+0000 7f28ddd9b640 1 -- 172.21.15.81:0/943754856 >> v2:172.21.15.81:3300/0 conn(0x7f28e80b2c30 msgr2=0x7f28e80b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:56.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.959+0000 7f28ddd9b640 1 --2- 172.21.15.81:0/943754856 >> v2:172.21.15.81:3300/0 conn(0x7f28e80b2c30 0x7f28e80b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.959+0000 7f28ddd9b640 1 -- 172.21.15.81:0/943754856 >> v1:172.21.15.81:6789/0 conn(0x7f28e815d1a0 legacy=0x7f28e80d13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:56.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.959+0000 7f28ddd9b640 1 -- 172.21.15.81:0/943754856 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e80b2710 con 0x7f28e80d1af0 2023-11-08T16:39:56.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.960+0000 7f28ddd9b640 1 --2- 172.21.15.81:0/943754856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28e80d1af0 0x7f28e80d3920 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f28d80097a0 tx=0x7f28d802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f85a3b8ff6b6da6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:56.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.960+0000 7f28e5d9b640 1 -- 172.21.15.81:0/943754856 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d802d040 con 0x7f28e80d1af0 2023-11-08T16:39:56.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.960+0000 7f28e5d9b640 1 -- 172.21.15.81:0/943754856 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d8031810 con 0x7f28e80d1af0 2023-11-08T16:39:56.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.960+0000 7f28e5d9b640 1 -- 172.21.15.81:0/943754856 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d8031ba0 con 0x7f28e80d1af0 2023-11-08T16:39:56.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.961+0000 7f28eca34640 1 -- 172.21.15.81:0/943754856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28e80d1af0 msgr2=0x7f28e80d3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:56.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.961+0000 7f28eca34640 1 --2- 172.21.15.81:0/943754856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28e80d1af0 0x7f28e80d3920 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f28d80097a0 tx=0x7f28d802b8b0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.961+0000 7f28eca34640 1 -- 172.21.15.81:0/943754856 shutdown_connections 2023-11-08T16:39:56.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.961+0000 7f28eca34640 1 --2- 172.21.15.81:0/943754856 >> v2:172.21.15.81:3300/0 conn(0x7f28e80b2c30 0x7f28e80b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.961+0000 7f28eca34640 1 --2- 172.21.15.81:0/943754856 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28e80d1af0 0x7f28e80d3920 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.961+0000 7f28eca34640 1 -- 172.21.15.81:0/943754856 >> 172.21.15.81:0/943754856 conn(0x7f28e80d2e00 msgr2=0x7f28e80d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:56.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.961+0000 7f28eca34640 1 -- 172.21.15.81:0/943754856 shutdown_connections 2023-11-08T16:39:56.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.961+0000 7f28eca34640 1 -- 172.21.15.81:0/943754856 wait complete. 2023-11-08T16:39:56.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.963+0000 7f28eca34640 1 Processor -- start 2023-11-08T16:39:56.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.963+0000 7f28eca34640 1 -- start start 2023-11-08T16:39:56.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28eca34640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28e80b2c30 0x7f28e80c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:56.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28eca34640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28e80d1af0 0x7f28e81f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:56.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28eca34640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28e815d1a0 0x7f28e80c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:56.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28eca34640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80ccb80 con 0x7f28e815d1a0 2023-11-08T16:39:56.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28eca34640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80cbfe0 con 0x7f28e80d1af0 2023-11-08T16:39:56.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28eca34640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28e80cc250 con 0x7f28e80b2c30 2023-11-08T16:39:56.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28e6d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28e815d1a0 0x7f28e80c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:56.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28e659c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28e80b2c30 0x7f28e80c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:56.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28e6d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28e815d1a0 0x7f28e80c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38446/0 (socket says 172.21.15.81:38446) 2023-11-08T16:39:56.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28e6d9d640 1 -- 172.21.15.81:0/3188914364 learned_addr learned my addr 172.21.15.81:0/3188914364 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:56.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28ddd9b640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28e80d1af0 0x7f28e81f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:56.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.964+0000 7f28e6d9d640 1 -- 172.21.15.81:0/3188914364 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28e80b2c30 msgr2=0x7f28e80c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:56.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.965+0000 7f28e6d9d640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28e80b2c30 0x7f28e80c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.965+0000 7f28e6d9d640 1 -- 172.21.15.81:0/3188914364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28e80d1af0 msgr2=0x7f28e81f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:56.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.965+0000 7f28e6d9d640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28e80d1af0 0x7f28e81f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:56.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.965+0000 7f28e6d9d640 1 -- 172.21.15.81:0/3188914364 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d8009400 con 0x7f28e815d1a0 2023-11-08T16:39:56.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.965+0000 7f28e6d9d640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28e815d1a0 0x7f28e80c9ed0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009d30 tx=0x7f28e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:56.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.965+0000 7f28df7fe640 1 -- 172.21.15.81:0/3188914364 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e0009ba0 con 0x7f28e815d1a0 2023-11-08T16:39:56.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.965+0000 7f28df7fe640 1 -- 172.21.15.81:0/3188914364 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e000f040 con 0x7f28e815d1a0 2023-11-08T16:39:56.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.965+0000 7f28df7fe640 1 -- 172.21.15.81:0/3188914364 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e00146c0 con 0x7f28e815d1a0 2023-11-08T16:39:56.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.965+0000 7f28eca34640 1 -- 172.21.15.81:0/3188914364 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28e80cc530 con 0x7f28e815d1a0 2023-11-08T16:39:56.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.965+0000 7f28eca34640 1 -- 172.21.15.81:0/3188914364 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28e80c9860 con 0x7f28e815d1a0 2023-11-08T16:39:56.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.967+0000 7f28eca34640 1 -- 172.21.15.81:0/3188914364 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28e804ece0 con 0x7f28e815d1a0 2023-11-08T16:39:56.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.967+0000 7f28df7fe640 1 -- 172.21.15.81:0/3188914364 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28e0014820 con 0x7f28e815d1a0 2023-11-08T16:39:56.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.968+0000 7f28df7fe640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28c806ae70 0x7f28c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:56.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.968+0000 7f28df7fe640 1 -- 172.21.15.81:0/3188914364 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6401+0+0 (secure 0 0 0) 0x7f28e008a1b0 con 0x7f28e815d1a0 2023-11-08T16:39:56.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.969+0000 7f28e659c640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28c806ae70 0x7f28c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:56.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.969+0000 7f28e659c640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28c806ae70 0x7f28c806d330 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f28d00010e0 tx=0x7f28d0007be0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:56.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:56.973+0000 7f28df7fe640 1 -- 172.21.15.81:0/3188914364 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f28e00550e0 con 0x7f28e815d1a0 2023-11-08T16:39:57.019 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:39:57.017+0000 7f5cc09c9640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:39:57.049 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:39:57.046+0000 7f9540baf640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-08T16:39:57.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.217+0000 7f28eca34640 1 -- 172.21.15.81:0/3188914364 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f28e815b920 con 0x7f28e815d1a0 2023-11-08T16:39:57.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.218+0000 7f28df7fe640 1 -- 172.21.15.81:0/3188914364 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 97 v97) v1 ==== 94+0+4914 (secure 0 0 0) 0x7f28e0058700 con 0x7f28e815d1a0 2023-11-08T16:39:57.221 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:57.221 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":97,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6019,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6833/1157756119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":1157756119},{"type":"v1","addr":"172.21.15.93:6833","nonce":1157756119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":81},{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/728323060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":728323060},{"type":"v1","addr":"172.21.15.93:6834","nonce":728323060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":8187,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/699802320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":699802320},{"type":"v1","addr":"172.21.15.81:6835","nonce":699802320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":97,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:57.029216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"c","rank":0,"incarnation":97,"state":"up:replay","state_seq":1,"addr":"172.21.15.81:6837/3197735576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3197735576},{"type":"v1","addr":"172.21.15.81:6837","nonce":3197735576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:39:57.221 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 97 2023-11-08T16:39:57.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 -- 172.21.15.81:0/3188914364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28c806ae70 msgr2=0x7f28c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:57.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28c806ae70 0x7f28c806d330 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f28d00010e0 tx=0x7f28d0007be0 comp rx=0 tx=0).stop 2023-11-08T16:39:57.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 -- 172.21.15.81:0/3188914364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28e815d1a0 msgr2=0x7f28e80c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:57.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28e815d1a0 0x7f28e80c9ed0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f28e0009d30 tx=0x7f28e000c490 comp rx=0 tx=0).stop 2023-11-08T16:39:57.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 -- 172.21.15.81:0/3188914364 shutdown_connections 2023-11-08T16:39:57.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f28c806ae70 0x7f28c806d330 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:57.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f28e815d1a0 0x7f28e80c9ed0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:57.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f28e80d1af0 0x7f28e81f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:57.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 --2- 172.21.15.81:0/3188914364 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f28e80b2c30 0x7f28e80c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:57.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 -- 172.21.15.81:0/3188914364 >> 172.21.15.81:0/3188914364 conn(0x7f28e80d2e00 msgr2=0x7f28e815c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:57.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 -- 172.21.15.81:0/3188914364 shutdown_connections 2023-11-08T16:39:57.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:57.221+0000 7f28eca34640 1 -- 172.21.15.81:0/3188914364 wait complete. 2023-11-08T16:39:57.644 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 97, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:39:49.779667+0000', 'modified': '2023-11-08T16:39:57.029216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8190}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8190': {'gid': 8190, 'name': 'c', 'rank': 0, 'incarnation': 97, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.81:6837/3197735576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 3197735576}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 3197735576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14, 15], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:39:57.645 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8190:up:replay 2023-11-08T16:39:58.646 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:39:58.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.866+0000 7f192a6ec640 1 Processor -- start 2023-11-08T16:39:58.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.866+0000 7f192a6ec640 1 -- start start 2023-11-08T16:39:58.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.868+0000 7f192a6ec640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1924071060 0x7f1924164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:58.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.868+0000 7f192a6ec640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19240c9a70 con 0x7f19240adcd0 2023-11-08T16:39:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.868+0000 7f192a6ec640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19240cc3e0 con 0x7f19240cc5e0 2023-11-08T16:39:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.868+0000 7f192a6ec640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f192406d070 con 0x7f1924071060 2023-11-08T16:39:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.868+0000 7f1923fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f19240adcd0 0x7f19240ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56614/0 (socket says 172.21.15.81:56614) 2023-11-08T16:39:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.868+0000 7f1923fff640 1 -- 172.21.15.81:0/3409658872 learned_addr learned my addr 172.21.15.81:0/3409658872 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:39:58.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.868+0000 7f1928c89640 1 --2- 172.21.15.81:0/3409658872 >> v2:172.21.15.81:3300/0 conn(0x7f1924071060 0x7f1924164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:58.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.868+0000 7f1922ffd640 1 -- 172.21.15.81:0/3409658872 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4052895150 0 0) 0x7f19240c9a70 con 0x7f19240adcd0 2023-11-08T16:39:58.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.868+0000 7f1922ffd640 1 -- 172.21.15.81:0/3409658872 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f190c003520 con 0x7f19240adcd0 2023-11-08T16:39:58.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.868+0000 7f1928c89640 1 -- 172.21.15.81:0/3409658872 >> v1:172.21.15.93:6789/0 conn(0x7f19240cc5e0 legacy=0x7f19240cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:39:58.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.868+0000 7f1928c89640 1 -- 172.21.15.81:0/3409658872 >> v1:172.21.15.81:6789/0 conn(0x7f19240adcd0 legacy=0x7f19240ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:58.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.869+0000 7f1928c89640 1 -- 172.21.15.81:0/3409658872 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f192406d350 con 0x7f1924071060 2023-11-08T16:39:58.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.870+0000 7f1928c89640 1 --2- 172.21.15.81:0/3409658872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1924071060 0x7f1924164eb0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f191c009d90 tx=0x7f191c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301ed832318cc91c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:58.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.870+0000 7f1922ffd640 1 -- 172.21.15.81:0/3409658872 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c02cc60 con 0x7f1924071060 2023-11-08T16:39:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.870+0000 7f1922ffd640 1 -- 172.21.15.81:0/3409658872 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f191c02cdc0 con 0x7f1924071060 2023-11-08T16:39:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.870+0000 7f1922ffd640 1 -- 172.21.15.81:0/3409658872 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c034660 con 0x7f1924071060 2023-11-08T16:39:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.870+0000 7f192a6ec640 1 -- 172.21.15.81:0/3409658872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1924071060 msgr2=0x7f1924164eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.870+0000 7f192a6ec640 1 --2- 172.21.15.81:0/3409658872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1924071060 0x7f1924164eb0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f191c009d90 tx=0x7f191c02beb0 comp rx=0 tx=0).stop 2023-11-08T16:39:58.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.871+0000 7f192a6ec640 1 -- 172.21.15.81:0/3409658872 shutdown_connections 2023-11-08T16:39:58.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.871+0000 7f192a6ec640 1 --2- 172.21.15.81:0/3409658872 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1924071060 0x7f1924164eb0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:58.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.871+0000 7f192a6ec640 1 -- 172.21.15.81:0/3409658872 >> 172.21.15.81:0/3409658872 conn(0x7f19240cb490 msgr2=0x7f19240cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:58.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.871+0000 7f192a6ec640 1 -- 172.21.15.81:0/3409658872 shutdown_connections 2023-11-08T16:39:58.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.871+0000 7f192a6ec640 1 -- 172.21.15.81:0/3409658872 wait complete. 2023-11-08T16:39:58.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.872+0000 7f192a6ec640 1 Processor -- start 2023-11-08T16:39:58.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.872+0000 7f192a6ec640 1 -- start start 2023-11-08T16:39:58.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.873+0000 7f192a6ec640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1924071060 0x7f19241584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:58.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.873+0000 7f192a6ec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240adcd0 0x7f1924147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:58.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.873+0000 7f192a6ec640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19240cc5e0 0x7f1924165510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:58.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.873+0000 7f192a6ec640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1924165a50 con 0x7f19240adcd0 2023-11-08T16:39:58.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.873+0000 7f192a6ec640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19240b2fd0 con 0x7f1924071060 2023-11-08T16:39:58.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.873+0000 7f192a6ec640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1924158aa0 con 0x7f19240cc5e0 2023-11-08T16:39:58.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f19237fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240adcd0 0x7f1924147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:58.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f19237fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240adcd0 0x7f1924147fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38460/0 (socket says 172.21.15.81:38460) 2023-11-08T16:39:58.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f19237fe640 1 -- 172.21.15.81:0/565811055 learned_addr learned my addr 172.21.15.81:0/565811055 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:39:58.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f1923fff640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1924071060 0x7f19241584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:58.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f1928c89640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19240cc5e0 0x7f1924165510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:58.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f19237fe640 1 -- 172.21.15.81:0/565811055 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19240cc5e0 msgr2=0x7f1924165510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:58.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f19237fe640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19240cc5e0 0x7f1924165510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:58.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f19237fe640 1 -- 172.21.15.81:0/565811055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1924071060 msgr2=0x7f19241584c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:58.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f19237fe640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1924071060 0x7f19241584c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:58.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f19237fe640 1 -- 172.21.15.81:0/565811055 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c009a70 con 0x7f19240adcd0 2023-11-08T16:39:58.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f1923fff640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1924071060 0x7f19241584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:39:58.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.874+0000 7f19237fe640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240adcd0 0x7f1924147fd0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f191800b420 tx=0x7f191800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:58.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.875+0000 7f1920ff9640 1 -- 172.21.15.81:0/565811055 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918011830 con 0x7f19240adcd0 2023-11-08T16:39:58.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.875+0000 7f1920ff9640 1 -- 172.21.15.81:0/565811055 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1918011990 con 0x7f19240adcd0 2023-11-08T16:39:58.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.875+0000 7f1920ff9640 1 -- 172.21.15.81:0/565811055 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1918011d40 con 0x7f19240adcd0 2023-11-08T16:39:58.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.875+0000 7f192a6ec640 1 -- 172.21.15.81:0/565811055 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1924158d80 con 0x7f19240adcd0 2023-11-08T16:39:58.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.875+0000 7f192a6ec640 1 -- 172.21.15.81:0/565811055 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1924159340 con 0x7f19240adcd0 2023-11-08T16:39:58.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.876+0000 7f192a6ec640 1 -- 172.21.15.81:0/565811055 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1924072170 con 0x7f19240adcd0 2023-11-08T16:39:58.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.877+0000 7f1920ff9640 1 -- 172.21.15.81:0/565811055 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f191801b460 con 0x7f19240adcd0 2023-11-08T16:39:58.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.878+0000 7f1920ff9640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191006ae70 0x7f191006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:39:58.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.878+0000 7f1920ff9640 1 -- 172.21.15.81:0/565811055 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f19180895e0 con 0x7f19240adcd0 2023-11-08T16:39:58.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.878+0000 7f1923fff640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191006ae70 0x7f191006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:39:58.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.879+0000 7f1923fff640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191006ae70 0x7f191006d330 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f19140011e0 tx=0x7f1914009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:39:58.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:58.883+0000 7f1920ff9640 1 -- 172.21.15.81:0/565811055 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1918054250 con 0x7f19240adcd0 2023-11-08T16:39:59.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.114+0000 7f192a6ec640 1 -- 172.21.15.81:0/565811055 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f192404ece0 con 0x7f19240adcd0 2023-11-08T16:39:59.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.115+0000 7f1920ff9640 1 -- 172.21.15.81:0/565811055 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) v1 ==== 94+0+4914 (secure 0 0 0) 0x7f1918057870 con 0x7f19240adcd0 2023-11-08T16:39:59.117 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:39:59.117 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":99,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/728323060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":728323060},{"type":"v1","addr":"172.21.15.93:6834","nonce":728323060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3481704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":3481704447},{"type":"v1","addr":"172.21.15.93:6835","nonce":3481704447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8187,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/699802320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":699802320},{"type":"v1","addr":"172.21.15.81:6835","nonce":699802320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:58.046397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"c","rank":0,"incarnation":97,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.81:6837/3197735576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3197735576},{"type":"v1","addr":"172.21.15.81:6837","nonce":3197735576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:39:59.117 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 99 2023-11-08T16:39:59.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 -- 172.21.15.81:0/565811055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191006ae70 msgr2=0x7f191006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:59.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191006ae70 0x7f191006d330 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f19140011e0 tx=0x7f1914009040 comp rx=0 tx=0).stop 2023-11-08T16:39:59.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 -- 172.21.15.81:0/565811055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240adcd0 msgr2=0x7f1924147fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:39:59.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240adcd0 0x7f1924147fd0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f191800b420 tx=0x7f191800b8e0 comp rx=0 tx=0).stop 2023-11-08T16:39:59.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 -- 172.21.15.81:0/565811055 shutdown_connections 2023-11-08T16:39:59.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f191006ae70 0x7f191006d330 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:59.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f19240cc5e0 0x7f1924165510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:59.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f19240adcd0 0x7f1924147fd0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:59.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 --2- 172.21.15.81:0/565811055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1924071060 0x7f19241584c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:39:59.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 -- 172.21.15.81:0/565811055 >> 172.21.15.81:0/565811055 conn(0x7f19240cb490 msgr2=0x7f19240ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:39:59.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 -- 172.21.15.81:0/565811055 shutdown_connections 2023-11-08T16:39:59.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:39:59.118+0000 7f192a6ec640 1 -- 172.21.15.81:0/565811055 wait complete. 2023-11-08T16:39:59.541 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:39:49.779667+0000', 'modified': '2023-11-08T16:39:58.046397+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8190}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8190': {'gid': 8190, 'name': 'c', 'rank': 0, 'incarnation': 97, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.81:6837/3197735576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 3197735576}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 3197735576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14, 15], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:39:59.542 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8190:up:rejoin 2023-11-08T16:40:00.543 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:40:00.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.765+0000 7f138d52c640 1 Processor -- start 2023-11-08T16:40:00.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.765+0000 7f138d52c640 1 -- start start 2023-11-08T16:40:00.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.766+0000 7f138d52c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f13880ade50 0x7f13880ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:00.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f138d52c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f138815a680 0x7f138815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:00.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f138d52c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f138815b020 0x7f13880c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:00.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f138d52c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880b1860 con 0x7f138815b020 2023-11-08T16:40:00.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f138d52c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13880ca9c0 con 0x7f13880ade50 2023-11-08T16:40:00.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f138d52c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138806b630 con 0x7f138815a680 2023-11-08T16:40:00.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f13877fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f138815b020 0x7f13880c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:00.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f13877fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f138815b020 0x7f13880c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38470/0 (socket says 172.21.15.81:38470) 2023-11-08T16:40:00.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f13877fe640 1 -- 172.21.15.81:0/982369879 learned_addr learned my addr 172.21.15.81:0/982369879 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:00.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f13867fc640 1 --2- 172.21.15.81:0/982369879 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f138815a680 0x7f138815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:00.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f1386ffd640 1 --2- 172.21.15.81:0/982369879 >> v2:172.21.15.93:3300/0 conn(0x7f13880ade50 0x7f13880ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:00.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f13877fe640 1 -- 172.21.15.81:0/982369879 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f138815a680 msgr2=0x7f138815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:00.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f13877fe640 1 --2- 172.21.15.81:0/982369879 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f138815a680 0x7f138815aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:00.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.767+0000 7f13877fe640 1 -- 172.21.15.81:0/982369879 >> v2:172.21.15.93:3300/0 conn(0x7f13880ade50 msgr2=0x7f13880ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:00.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.768+0000 7f13877fe640 1 --2- 172.21.15.81:0/982369879 >> v2:172.21.15.93:3300/0 conn(0x7f13880ade50 0x7f13880ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:00.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.768+0000 7f13877fe640 1 -- 172.21.15.81:0/982369879 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880cabb0 con 0x7f138815b020 2023-11-08T16:40:00.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.769+0000 7f13877fe640 1 --2- 172.21.15.81:0/982369879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138815b020 0x7f13880c9060 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f1388067770 tx=0x7f138002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb48fd117c9cef73 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:00.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.769+0000 7f1385ffb640 1 -- 172.21.15.81:0/982369879 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138002ca70 con 0x7f138815b020 2023-11-08T16:40:00.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.769+0000 7f1385ffb640 1 -- 172.21.15.81:0/982369879 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138002cbd0 con 0x7f138815b020 2023-11-08T16:40:00.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.769+0000 7f1385ffb640 1 -- 172.21.15.81:0/982369879 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13800343f0 con 0x7f138815b020 2023-11-08T16:40:00.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.770+0000 7f138d52c640 1 -- 172.21.15.81:0/982369879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138815b020 msgr2=0x7f13880c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:00.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.770+0000 7f138d52c640 1 --2- 172.21.15.81:0/982369879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138815b020 0x7f13880c9060 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f1388067770 tx=0x7f138002bed0 comp rx=0 tx=0).stop 2023-11-08T16:40:00.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.770+0000 7f138d52c640 1 -- 172.21.15.81:0/982369879 shutdown_connections 2023-11-08T16:40:00.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.770+0000 7f138d52c640 1 --2- 172.21.15.81:0/982369879 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f138815a680 0x7f138815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:00.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.770+0000 7f138d52c640 1 --2- 172.21.15.81:0/982369879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138815b020 0x7f13880c9060 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:00.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.770+0000 7f138d52c640 1 --2- 172.21.15.81:0/982369879 >> v2:172.21.15.93:3300/0 conn(0x7f13880ade50 0x7f13880ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:00.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.770+0000 7f138d52c640 1 -- 172.21.15.81:0/982369879 >> 172.21.15.81:0/982369879 conn(0x7f13880cb490 msgr2=0x7f13880cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:00.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.770+0000 7f138d52c640 1 -- 172.21.15.81:0/982369879 shutdown_connections 2023-11-08T16:40:00.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.770+0000 7f138d52c640 1 -- 172.21.15.81:0/982369879 wait complete. 2023-11-08T16:40:00.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.772+0000 7f138d52c640 1 Processor -- start 2023-11-08T16:40:00.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.772+0000 7f138d52c640 1 -- start start 2023-11-08T16:40:00.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f138d52c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13880ade50 0x7f13880cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:00.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f138d52c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138815a680 0x7f13881efee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:00.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f138d52c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f138815b020 0x7f13880aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:00.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f138d52c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13881f38d0 con 0x7f138815a680 2023-11-08T16:40:00.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f138d52c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13880cc7a0 con 0x7f138815b020 2023-11-08T16:40:00.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f138d52c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13880cca80 con 0x7f13880ade50 2023-11-08T16:40:00.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f13867fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138815a680 0x7f13881efee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:00.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f13867fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138815a680 0x7f13881efee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38484/0 (socket says 172.21.15.81:38484) 2023-11-08T16:40:00.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f13867fc640 1 -- 172.21.15.81:0/3585599417 learned_addr learned my addr 172.21.15.81:0/3585599417 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:00.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f13877fe640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f138815b020 0x7f13880aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:00.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f13867fc640 1 -- 172.21.15.81:0/3585599417 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13880ade50 msgr2=0x7f13880cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:00.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f1386ffd640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13880ade50 0x7f13880cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:00.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f13867fc640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13880ade50 0x7f13880cd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:00.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f13867fc640 1 -- 172.21.15.81:0/3585599417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f138815b020 msgr2=0x7f13880aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:00.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f13867fc640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f138815b020 0x7f13880aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:00.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f13867fc640 1 -- 172.21.15.81:0/3585599417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1380009b30 con 0x7f138815a680 2023-11-08T16:40:00.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.773+0000 7f13877fe640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f138815b020 0x7f13880aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:00.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.774+0000 7f13867fc640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138815a680 0x7f13881efee0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f137c00d800 tx=0x7f137c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:00.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.774+0000 7f136f7fe640 1 -- 172.21.15.81:0/3585599417 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00b6c0 con 0x7f138815a680 2023-11-08T16:40:00.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.774+0000 7f136f7fe640 1 -- 172.21.15.81:0/3585599417 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c00b820 con 0x7f138815a680 2023-11-08T16:40:00.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.774+0000 7f136f7fe640 1 -- 172.21.15.81:0/3585599417 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00bbb0 con 0x7f138815a680 2023-11-08T16:40:00.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.774+0000 7f138d52c640 1 -- 172.21.15.81:0/3585599417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13880ccd60 con 0x7f138815a680 2023-11-08T16:40:00.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.774+0000 7f138d52c640 1 -- 172.21.15.81:0/3585599417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13880af3f0 con 0x7f138815a680 2023-11-08T16:40:00.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.775+0000 7f138d52c640 1 -- 172.21.15.81:0/3585599417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13880cc5e0 con 0x7f138815a680 2023-11-08T16:40:00.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.776+0000 7f136f7fe640 1 -- 172.21.15.81:0/3585599417 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f137c00bd10 con 0x7f138815a680 2023-11-08T16:40:00.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.777+0000 7f136f7fe640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 0x7f137406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:00.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.777+0000 7f136f7fe640 1 -- 172.21.15.81:0/3585599417 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f137c089f60 con 0x7f138815a680 2023-11-08T16:40:00.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.778+0000 7f1386ffd640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 0x7f137406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:00.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.778+0000 7f1386ffd640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 0x7f137406d330 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1378006fd0 tx=0x7f1378008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:00.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:00.782+0000 7f136f7fe640 1 -- 172.21.15.81:0/3585599417 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f137c054c60 con 0x7f138815a680 2023-11-08T16:40:01.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.011+0000 7f138d52c640 1 -- 172.21.15.81:0/3585599417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f138804ece0 con 0x7f138815a680 2023-11-08T16:40:01.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.012+0000 7f136f7fe640 1 -- 172.21.15.81:0/3585599417 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4916 (secure 0 0 0) 0x7f137c058280 con 0x7f138815a680 2023-11-08T16:40:01.014 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:01.014 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":100,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/728323060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":728323060},{"type":"v1","addr":"172.21.15.93:6834","nonce":728323060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3481704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":3481704447},{"type":"v1","addr":"172.21.15.93:6835","nonce":3481704447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8187,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/699802320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":699802320},{"type":"v1","addr":"172.21.15.81:6835","nonce":699802320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:59.048664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/3197735576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3197735576},{"type":"v1","addr":"172.21.15.81:6837","nonce":3197735576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:40:01.014 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 100 2023-11-08T16:40:01.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.014+0000 7f138d52c640 1 -- 172.21.15.81:0/3585599417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 msgr2=0x7f137406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.014+0000 7f138d52c640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 0x7f137406d330 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1378006fd0 tx=0x7f1378008040 comp rx=0 tx=0).stop 2023-11-08T16:40:01.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.014+0000 7f138d52c640 1 -- 172.21.15.81:0/3585599417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138815a680 msgr2=0x7f13881efee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.014+0000 7f138d52c640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138815a680 0x7f13881efee0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f137c00d800 tx=0x7f137c00dcc0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.015+0000 7f138d52c640 1 -- 172.21.15.81:0/3585599417 shutdown_connections 2023-11-08T16:40:01.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.015+0000 7f138d52c640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f137406ae70 0x7f137406d330 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.015+0000 7f138d52c640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f138815b020 0x7f13880aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.015+0000 7f138d52c640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f138815a680 0x7f13881efee0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.015+0000 7f138d52c640 1 --2- 172.21.15.81:0/3585599417 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f13880ade50 0x7f13880cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.015+0000 7f138d52c640 1 -- 172.21.15.81:0/3585599417 >> 172.21.15.81:0/3585599417 conn(0x7f13880cb490 msgr2=0x7f1388159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:01.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.015+0000 7f138d52c640 1 -- 172.21.15.81:0/3585599417 shutdown_connections 2023-11-08T16:40:01.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.015+0000 7f138d52c640 1 -- 172.21.15.81:0/3585599417 wait complete. 2023-11-08T16:40:01.437 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 100, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:39:49.779667+0000', 'modified': '2023-11-08T16:39:59.048664+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 112, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8190}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8190': {'gid': 8190, 'name': 'c', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6837/3197735576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 3197735576}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 3197735576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14, 15], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:40:01.438 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:40:01.438 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T16:40:01.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.660+0000 7f143bb05640 1 Processor -- start 2023-11-08T16:40:01.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.660+0000 7f143bb05640 1 -- start start 2023-11-08T16:40:01.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.661+0000 7f143bb05640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f143415d3b0 0x7f143406d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f143bb05640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f14340ca130 0x7f1434070c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f143bb05640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143406cc50 con 0x7f14340cf720 2023-11-08T16:40:01.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f143bb05640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14340c90f0 con 0x7f143415d3b0 2023-11-08T16:40:01.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f143bb05640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14340adb70 con 0x7f14340ca130 2023-11-08T16:40:01.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f14398a1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f143415d3b0 0x7f143406d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f14390a0640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f14340cf720 0x7f14340c9a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50828/0 (socket says 172.21.15.81:50828) 2023-11-08T16:40:01.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f14390a0640 1 -- 172.21.15.81:0/1801591090 learned_addr learned my addr 172.21.15.81:0/1801591090 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:01.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f143a0a2640 1 --2- 172.21.15.81:0/1801591090 >> v2:172.21.15.93:3300/0 conn(0x7f14340ca130 0x7f1434070c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f14398a1640 1 -- 172.21.15.81:0/1801591090 >> v2:172.21.15.93:3300/0 conn(0x7f14340ca130 msgr2=0x7f1434070c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:01.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f14398a1640 1 --2- 172.21.15.81:0/1801591090 >> v2:172.21.15.93:3300/0 conn(0x7f14340ca130 0x7f1434070c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f14398a1640 1 -- 172.21.15.81:0/1801591090 >> v1:172.21.15.81:6789/0 conn(0x7f14340cf720 legacy=0x7f14340c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f14398a1640 1 -- 172.21.15.81:0/1801591090 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340c9360 con 0x7f143415d3b0 2023-11-08T16:40:01.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.662+0000 7f143a0a2640 1 --2- 172.21.15.81:0/1801591090 >> v2:172.21.15.93:3300/0 conn(0x7f14340ca130 0x7f1434070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:01.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.664+0000 7f14398a1640 1 --2- 172.21.15.81:0/1801591090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f143415d3b0 0x7f143406d0e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f143406baf0 tx=0x7f143002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ceaf2b45635627 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:01.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.664+0000 7f143889f640 1 -- 172.21.15.81:0/1801591090 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143002d040 con 0x7f143415d3b0 2023-11-08T16:40:01.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.664+0000 7f143889f640 1 -- 172.21.15.81:0/1801591090 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1430031810 con 0x7f143415d3b0 2023-11-08T16:40:01.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.664+0000 7f143889f640 1 -- 172.21.15.81:0/1801591090 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430031ba0 con 0x7f143415d3b0 2023-11-08T16:40:01.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.664+0000 7f143bb05640 1 -- 172.21.15.81:0/1801591090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f143415d3b0 msgr2=0x7f143406d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.664+0000 7f143bb05640 1 --2- 172.21.15.81:0/1801591090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f143415d3b0 0x7f143406d0e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f143406baf0 tx=0x7f143002b8b0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.665+0000 7f143bb05640 1 -- 172.21.15.81:0/1801591090 shutdown_connections 2023-11-08T16:40:01.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.665+0000 7f143bb05640 1 --2- 172.21.15.81:0/1801591090 >> v2:172.21.15.93:3300/0 conn(0x7f14340ca130 0x7f1434070c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.665+0000 7f143bb05640 1 --2- 172.21.15.81:0/1801591090 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f143415d3b0 0x7f143406d0e0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.665+0000 7f143bb05640 1 -- 172.21.15.81:0/1801591090 >> 172.21.15.81:0/1801591090 conn(0x7f14340ceaa0 msgr2=0x7f14340cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:01.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.665+0000 7f143bb05640 1 -- 172.21.15.81:0/1801591090 shutdown_connections 2023-11-08T16:40:01.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.665+0000 7f143bb05640 1 -- 172.21.15.81:0/1801591090 wait complete. 2023-11-08T16:40:01.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.666+0000 7f143bb05640 1 Processor -- start 2023-11-08T16:40:01.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.667+0000 7f143bb05640 1 -- start start 2023-11-08T16:40:01.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f143bb05640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14340ca130 0x7f14340c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f143bb05640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14340cf720 0x7f14340706c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f143bb05640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143415d3b0 0x7f14340b2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f143bb05640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340b27d0 con 0x7f14340ca130 2023-11-08T16:40:01.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f143bb05640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340b1b50 con 0x7f14340cf720 2023-11-08T16:40:01.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f143bb05640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14340b1e30 con 0x7f143415d3b0 2023-11-08T16:40:01.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f14398a1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14340ca130 0x7f14340c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f14398a1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14340ca130 0x7f14340c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54862/0 (socket says 172.21.15.81:54862) 2023-11-08T16:40:01.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f14398a1640 1 -- 172.21.15.81:0/1479769293 learned_addr learned my addr 172.21.15.81:0/1479769293 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:01.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f14390a0640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14340cf720 0x7f14340706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f14398a1640 1 -- 172.21.15.81:0/1479769293 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143415d3b0 msgr2=0x7f14340b2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f14398a1640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143415d3b0 0x7f14340b2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f14398a1640 1 -- 172.21.15.81:0/1479769293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14340cf720 msgr2=0x7f14340706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f14398a1640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14340cf720 0x7f14340706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f14398a1640 1 -- 172.21.15.81:0/1479769293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14300093a0 con 0x7f14340ca130 2023-11-08T16:40:01.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.668+0000 7f14390a0640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14340cf720 0x7f14340706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:01.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.669+0000 7f14398a1640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14340ca130 0x7f14340c8580 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1430000c00 tx=0x7f14300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:01.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.669+0000 7f14267fc640 1 -- 172.21.15.81:0/1479769293 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143003ad60 con 0x7f14340ca130 2023-11-08T16:40:01.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.669+0000 7f14267fc640 1 -- 172.21.15.81:0/1479769293 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14300443f0 con 0x7f14340ca130 2023-11-08T16:40:01.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.669+0000 7f143bb05640 1 -- 172.21.15.81:0/1479769293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14340b2110 con 0x7f14340ca130 2023-11-08T16:40:01.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.669+0000 7f14267fc640 1 -- 172.21.15.81:0/1479769293 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14300447a0 con 0x7f14340ca130 2023-11-08T16:40:01.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.669+0000 7f143bb05640 1 -- 172.21.15.81:0/1479769293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1434153f10 con 0x7f14340ca130 2023-11-08T16:40:01.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.671+0000 7f14267fc640 1 -- 172.21.15.81:0/1479769293 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f143004d450 con 0x7f14340ca130 2023-11-08T16:40:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.672+0000 7f14267fc640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141806ae70 0x7f141806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.672+0000 7f14267fc640 1 -- 172.21.15.81:0/1479769293 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f14300abbd0 con 0x7f14340ca130 2023-11-08T16:40:01.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.672+0000 7f14390a0640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141806ae70 0x7f141806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.673+0000 7f14390a0640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141806ae70 0x7f141806d330 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f14340c7160 tx=0x7f1428009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:01.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.674+0000 7f143e772740 1 Processor -- start 2023-11-08T16:40:01.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.675+0000 7f143e772740 1 -- start start 2023-11-08T16:40:01.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.675+0000 7f143e772740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a36ec0 0x5631a3a09dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.675+0000 7f143e772740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a372a0 0x5631a3a39c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.675+0000 7f143e772740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a0a300 0x5631a3a3a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.675+0000 7f143e772740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a397c7e0 con 0x5631a3a372a0 2023-11-08T16:40:01.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f143e772740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a3979ed0 con 0x5631a3a0a300 2023-11-08T16:40:01.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f143e772740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631a39757c0 con 0x5631a3a36ec0 2023-11-08T16:40:01.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f14390a0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a372a0 0x5631a3a39c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f14390a0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a372a0 0x5631a3a39c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54866/0 (socket says 172.21.15.81:54866) 2023-11-08T16:40:01.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f14390a0640 1 -- 172.21.15.81:0/2803568652 learned_addr learned my addr 172.21.15.81:0/2803568652 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:01.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f14398a1640 1 --2- 172.21.15.81:0/2803568652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a36ec0 0x5631a3a09dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f143a0a2640 1 --2- 172.21.15.81:0/2803568652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a0a300 0x5631a3a3a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f14390a0640 1 -- 172.21.15.81:0/2803568652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a36ec0 msgr2=0x5631a3a09dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:01.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f14390a0640 1 --2- 172.21.15.81:0/2803568652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a36ec0 0x5631a3a09dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f14390a0640 1 -- 172.21.15.81:0/2803568652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a0a300 msgr2=0x5631a3a3a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:01.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f14390a0640 1 --2- 172.21.15.81:0/2803568652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a0a300 0x5631a3a3a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f14390a0640 1 -- 172.21.15.81:0/2803568652 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631a3948b10 con 0x5631a3a372a0 2023-11-08T16:40:01.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f14398a1640 1 --2- 172.21.15.81:0/2803568652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a36ec0 0x5631a3a09dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:01.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.676+0000 7f143a0a2640 1 --2- 172.21.15.81:0/2803568652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a0a300 0x5631a3a3a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:01.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.677+0000 7f14390a0640 1 --2- 172.21.15.81:0/2803568652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a372a0 0x5631a3a39c50 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f142800de30 tx=0x7f1428010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c420d0d4fc5cb63d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:01.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.677+0000 7f14177fe640 1 -- 172.21.15.81:0/2803568652 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428012820 con 0x5631a3a372a0 2023-11-08T16:40:01.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.677+0000 7f14177fe640 1 -- 172.21.15.81:0/2803568652 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1428012980 con 0x5631a3a372a0 2023-11-08T16:40:01.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.677+0000 7f14177fe640 1 -- 172.21.15.81:0/2803568652 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428012cf0 con 0x5631a3a372a0 2023-11-08T16:40:01.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.678+0000 7f143e772740 1 -- 172.21.15.81:0/2803568652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a372a0 msgr2=0x5631a3a39c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.678+0000 7f143e772740 1 --2- 172.21.15.81:0/2803568652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a372a0 0x5631a3a39c50 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f142800de30 tx=0x7f1428010000 comp rx=0 tx=0).stop 2023-11-08T16:40:01.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.678+0000 7f143e772740 1 -- 172.21.15.81:0/2803568652 shutdown_connections 2023-11-08T16:40:01.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.678+0000 7f143e772740 1 --2- 172.21.15.81:0/2803568652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a0a300 0x5631a3a3a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.678+0000 7f143e772740 1 --2- 172.21.15.81:0/2803568652 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a372a0 0x5631a3a39c50 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.678+0000 7f143e772740 1 --2- 172.21.15.81:0/2803568652 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a36ec0 0x5631a3a09dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.678+0000 7f143e772740 1 -- 172.21.15.81:0/2803568652 >> 172.21.15.81:0/2803568652 conn(0x5631a3a0c0d0 msgr2=0x5631a3a54c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:01.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.678+0000 7f143e772740 1 -- 172.21.15.81:0/2803568652 shutdown_connections 2023-11-08T16:40:01.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.678+0000 7f143e772740 1 -- 172.21.15.81:0/2803568652 wait complete. 2023-11-08T16:40:01.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.678+0000 7f143e772740 1 -- start start 2023-11-08T16:40:01.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.679+0000 7f143e772740 1 Processor -- start 2023-11-08T16:40:01.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f143e772740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a38e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f143e772740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a372a0 0x5631a3a65270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f143e772740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a393c0 0x5631a3a657b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f143e772740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a3912c40 con 0x5631a3a36ec0 2023-11-08T16:40:01.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f143e772740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a39128e0 con 0x5631a3a393c0 2023-11-08T16:40:01.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f143e772740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631a36d8f20 con 0x5631a3a372a0 2023-11-08T16:40:01.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f143e772740 10 client.0 _ll_register_callbacks cb 0x5631a3934cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:40:01.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f14398a1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a38e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f14398a1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a38e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54878/0 (socket says 172.21.15.81:54878) 2023-11-08T16:40:01.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f14398a1640 1 -- 172.21.15.81:0/349305200 learned_addr learned my addr 172.21.15.81:0/349305200 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:01.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f143a0a2640 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a393c0 0x5631a3a657b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f14398a1640 1 -- 172.21.15.81:0/349305200 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a372a0 msgr2=0x5631a3a65270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:01.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f14398a1640 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a372a0 0x5631a3a65270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f14398a1640 1 -- 172.21.15.81:0/349305200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a393c0 msgr2=0x5631a3a657b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:01.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f14398a1640 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a393c0 0x5631a3a657b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.680+0000 7f14398a1640 1 -- 172.21.15.81:0/349305200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143406cc50 con 0x5631a3a36ec0 2023-11-08T16:40:01.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.681+0000 7f14398a1640 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a38e80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f143002b8f0 tx=0x7f143007ae80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f5c9cf1551fc75 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:01.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.681+0000 7f1414ff9640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:40:01.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.681+0000 7f1414ff9640 1 -- 172.21.15.81:0/349305200 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143007ab20 con 0x5631a3a36ec0 2023-11-08T16:40:01.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.681+0000 7f1414ff9640 1 -- 172.21.15.81:0/349305200 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f14300829c0 con 0x5631a3a36ec0 2023-11-08T16:40:01.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.681+0000 7f143e772740 1 -- 172.21.15.81:0/349305200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5631a3949c80 con 0x5631a3a36ec0 2023-11-08T16:40:01.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.681+0000 7f1414ff9640 1 -- 172.21.15.81:0/349305200 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1430079bb0 con 0x5631a3a36ec0 2023-11-08T16:40:01.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.681+0000 7f143e772740 10 client.8244 fetch_fsmap learned FSMap version 100 2023-11-08T16:40:01.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.681+0000 7f143e772740 1 -- 172.21.15.81:0/349305200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x5631a3973e40 con 0x5631a3a36ec0 2023-11-08T16:40:01.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.681+0000 7f1414ff9640 1 -- 172.21.15.81:0/349305200 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5631a3949c80 con 0x5631a3a36ec0 2023-11-08T16:40:01.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.681+0000 7f1414ff9640 1 -- 172.21.15.81:0/349305200 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 100) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f1430082530 con 0x5631a3a36ec0 2023-11-08T16:40:01.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.682+0000 7f143e772740 10 client.8244 fetch_fsmap finished waiting for FSMap version 100 2023-11-08T16:40:01.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.682+0000 7f143e772740 10 client.8244 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:40:01.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.682+0000 7f143e772740 20 client.8244 populate_metadata read hostname 'smithi081' 2023-11-08T16:40:01.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.682+0000 7f143e772740 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5631a3a6e870 0x5631a3a72850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.682+0000 7f143e772740 4 client.8244 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:40:01.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.682+0000 7f143e772740 1 -- 172.21.15.81:0/349305200 --> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5631a396e7a0 con 0x5631a3a6e870 2023-11-08T16:40:01.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.682+0000 7f14390a0640 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5631a3a6e870 0x5631a3a72850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.683+0000 7f1414ff9640 10 client.8244 ms_handle_connect on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:01.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.682+0000 7f14390a0640 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5631a3a6e870 0x5631a3a72850 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac2b8d2c8dd75228 server_cookie=27254d73e76b6afc in_seq=0 out_seq=0 2023-11-08T16:40:01.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.683+0000 7f1414ff9640 1 -- 172.21.15.81:0/349305200 <== mds.0 v2:172.21.15.81:6836/3197735576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5631a396e7a0 con 0x5631a3a6e870 2023-11-08T16:40:01.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.683+0000 7f1414ff9640 10 client.8244 handle_command_reply: tid=0 2023-11-08T16:40:01.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.683+0000 7f143e772740 1 client.8244 shutdown 2023-11-08T16:40:01.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f143e772740 1 -- 172.21.15.81:0/349305200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 msgr2=0x5631a3a38e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f143e772740 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a38e80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f143002b8f0 tx=0x7f143007ae80 comp rx=0 tx=0).stop 2023-11-08T16:40:01.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f143e772740 1 -- 172.21.15.81:0/349305200 shutdown_connections 2023-11-08T16:40:01.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f143e772740 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a393c0 0x5631a3a657b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f143e772740 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5631a3a6e870 0x5631a3a72850 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f143e772740 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a372a0 0x5631a3a65270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f143e772740 1 --2- 172.21.15.81:0/349305200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a38e80 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f143e772740 1 -- 172.21.15.81:0/349305200 >> 172.21.15.81:0/349305200 conn(0x5631a3a0a300 msgr2=0x5631a3a0de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:01.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f1414ff9640 0 client.8244 ms_handle_reset on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:01.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f143e772740 1 -- 172.21.15.81:0/349305200 shutdown_connections 2023-11-08T16:40:01.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f143e772740 1 -- 172.21.15.81:0/349305200 wait complete. 2023-11-08T16:40:01.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.684+0000 7f143e772740 20 client.8244 trim_cache size 0 max 16384 2023-11-08T16:40:01.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.701+0000 7f143e772740 1 Processor -- start 2023-11-08T16:40:01.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.701+0000 7f143e772740 1 -- start start 2023-11-08T16:40:01.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.702+0000 7f143e772740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a58350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.702+0000 7f143e772740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a372a0 0x5631a3a64df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.702+0000 7f143e772740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a393c0 0x5631a3a65330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.702+0000 7f143e772740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a3a366b0 con 0x5631a3a36ec0 2023-11-08T16:40:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.702+0000 7f143e772740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a3a58d30 con 0x5631a3a372a0 2023-11-08T16:40:01.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.702+0000 7f143e772740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631a3a0e3f0 con 0x5631a3a393c0 2023-11-08T16:40:01.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.702+0000 7f14398a1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a58350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.702+0000 7f14398a1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a58350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54890/0 (socket says 172.21.15.81:54890) 2023-11-08T16:40:01.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.702+0000 7f14398a1640 1 -- 172.21.15.81:0/2622296213 learned_addr learned my addr 172.21.15.81:0/2622296213 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:01.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.703+0000 7f14390a0640 1 --2- 172.21.15.81:0/2622296213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a372a0 0x5631a3a64df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.703+0000 7f14398a1640 1 -- 172.21.15.81:0/2622296213 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a393c0 msgr2=0x5631a3a65330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:01.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.703+0000 7f14398a1640 1 --2- 172.21.15.81:0/2622296213 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a393c0 0x5631a3a65330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.703+0000 7f14398a1640 1 -- 172.21.15.81:0/2622296213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a372a0 msgr2=0x5631a3a64df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:01.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.703+0000 7f14398a1640 1 --2- 172.21.15.81:0/2622296213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a372a0 0x5631a3a64df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.703+0000 7f14398a1640 1 -- 172.21.15.81:0/2622296213 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5631a3911b90 con 0x5631a3a36ec0 2023-11-08T16:40:01.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.703+0000 7f14390a0640 1 --2- 172.21.15.81:0/2622296213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a372a0 0x5631a3a64df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:01.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.703+0000 7f14398a1640 1 --2- 172.21.15.81:0/2622296213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a58350 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1430076500 tx=0x7f1430090000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745964ace05d6099 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:01.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.703+0000 7f1417fff640 1 -- 172.21.15.81:0/2622296213 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430079bb0 con 0x5631a3a36ec0 2023-11-08T16:40:01.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.703+0000 7f1417fff640 1 -- 172.21.15.81:0/2622296213 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14300829c0 con 0x5631a3a36ec0 2023-11-08T16:40:01.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.703+0000 7f1417fff640 1 -- 172.21.15.81:0/2622296213 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143007a920 con 0x5631a3a36ec0 2023-11-08T16:40:01.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.704+0000 7f143e772740 1 -- 172.21.15.81:0/2622296213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 msgr2=0x5631a3a58350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.704+0000 7f143e772740 1 --2- 172.21.15.81:0/2622296213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a58350 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f1430076500 tx=0x7f1430090000 comp rx=0 tx=0).stop 2023-11-08T16:40:01.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.704+0000 7f143e772740 1 -- 172.21.15.81:0/2622296213 shutdown_connections 2023-11-08T16:40:01.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.704+0000 7f143e772740 1 --2- 172.21.15.81:0/2622296213 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a393c0 0x5631a3a65330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.704+0000 7f143e772740 1 --2- 172.21.15.81:0/2622296213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a372a0 0x5631a3a64df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.704+0000 7f143e772740 1 --2- 172.21.15.81:0/2622296213 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a58350 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.704+0000 7f143e772740 1 -- 172.21.15.81:0/2622296213 >> 172.21.15.81:0/2622296213 conn(0x5631a3a0a300 msgr2=0x5631a3a54c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:01.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.704+0000 7f143e772740 1 -- 172.21.15.81:0/2622296213 shutdown_connections 2023-11-08T16:40:01.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.704+0000 7f143e772740 1 -- 172.21.15.81:0/2622296213 wait complete. 2023-11-08T16:40:01.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.705+0000 7f143e772740 1 -- start start 2023-11-08T16:40:01.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.705+0000 7f143e772740 1 Processor -- start 2023-11-08T16:40:01.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.706+0000 7f143e772740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a0df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.706+0000 7f143e772740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a372a0 0x5631a3abee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.706+0000 7f143e772740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a393c0 0x5631a3a58350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.706+0000 7f143e772740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a3a37a00 con 0x5631a3a36ec0 2023-11-08T16:40:01.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.706+0000 7f143e772740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5631a3a6e310 con 0x5631a3a393c0 2023-11-08T16:40:01.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.706+0000 7f143e772740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5631a3abf360 con 0x5631a3a372a0 2023-11-08T16:40:01.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.706+0000 7f143e772740 10 client.0 _ll_register_callbacks cb 0x7f143003aec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:40:01.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.706+0000 7f14398a1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a0df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.706+0000 7f14398a1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a0df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54892/0 (socket says 172.21.15.81:54892) 2023-11-08T16:40:01.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.706+0000 7f14398a1640 1 -- 172.21.15.81:0/3490612756 learned_addr learned my addr 172.21.15.81:0/3490612756 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:01.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f14390a0640 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a372a0 0x5631a3abee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f143a0a2640 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a393c0 0x5631a3a58350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f14398a1640 1 -- 172.21.15.81:0/3490612756 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a372a0 msgr2=0x5631a3abee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:01.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f14398a1640 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a372a0 0x5631a3abee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f14398a1640 1 -- 172.21.15.81:0/3490612756 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a393c0 msgr2=0x5631a3a58350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:01.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f14398a1640 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a393c0 0x5631a3a58350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f14398a1640 1 -- 172.21.15.81:0/3490612756 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5631a3973e40 con 0x5631a3a36ec0 2023-11-08T16:40:01.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f143a0a2640 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a393c0 0x5631a3a58350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:01.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f14398a1640 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a0df00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f14300abeb0 tx=0x7f14300a9d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa35d245be2b145f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:01.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f1414ff9640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:40:01.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f1414ff9640 1 -- 172.21.15.81:0/3490612756 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430079bb0 con 0x5631a3a36ec0 2023-11-08T16:40:01.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f1414ff9640 1 -- 172.21.15.81:0/3490612756 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f143008fdd0 con 0x5631a3a36ec0 2023-11-08T16:40:01.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f143e772740 1 -- 172.21.15.81:0/3490612756 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1428012980 con 0x5631a3a36ec0 2023-11-08T16:40:01.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f1414ff9640 1 -- 172.21.15.81:0/3490612756 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1430076500 con 0x5631a3a36ec0 2023-11-08T16:40:01.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f143e772740 10 client.8250 fetch_fsmap learned FSMap version 100 2023-11-08T16:40:01.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.707+0000 7f143e772740 1 -- 172.21.15.81:0/3490612756 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x5631a3937180 con 0x5631a3a36ec0 2023-11-08T16:40:01.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.708+0000 7f1414ff9640 1 -- 172.21.15.81:0/3490612756 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1428012980 con 0x5631a3a36ec0 2023-11-08T16:40:01.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.708+0000 7f1414ff9640 1 -- 172.21.15.81:0/3490612756 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 100) v1 ==== 3302+0+0 (secure 0 0 0) 0x7f1430076660 con 0x5631a3a36ec0 2023-11-08T16:40:01.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.708+0000 7f143e772740 10 client.8250 fetch_fsmap finished waiting for FSMap version 100 2023-11-08T16:40:01.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.708+0000 7f143e772740 10 client.8250 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:40:01.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.708+0000 7f143e772740 20 client.8250 populate_metadata read hostname 'smithi081' 2023-11-08T16:40:01.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.708+0000 7f143e772740 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5631a3a61360 0x5631a3a61740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.708+0000 7f143e772740 4 client.8250 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:40:01.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.708+0000 7f143e772740 1 -- 172.21.15.81:0/3490612756 --> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5631a396e4c0 con 0x5631a3a61360 2023-11-08T16:40:01.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.708+0000 7f14390a0640 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5631a3a61360 0x5631a3a61740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.708+0000 7f14390a0640 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5631a3a61360 0x5631a3a61740 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=538a7cc183e070b9 server_cookie=9ae06a711b5943b8 in_seq=0 out_seq=0 2023-11-08T16:40:01.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f1414ff9640 10 client.8250 ms_handle_connect on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:01.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f1414ff9640 1 -- 172.21.15.81:0/3490612756 <== mds.0 v2:172.21.15.81:6836/3197735576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5631a396e4c0 con 0x5631a3a61360 2023-11-08T16:40:01.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f1414ff9640 10 client.8250 handle_command_reply: tid=0 2023-11-08T16:40:01.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 1 client.8250 shutdown 2023-11-08T16:40:01.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 1 -- 172.21.15.81:0/3490612756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 msgr2=0x5631a3a0df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a0df00 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f14300abeb0 tx=0x7f14300a9d50 comp rx=0 tx=0).stop 2023-11-08T16:40:01.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 1 -- 172.21.15.81:0/3490612756 shutdown_connections 2023-11-08T16:40:01.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5631a3a393c0 0x5631a3a58350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5631a3a61360 0x5631a3a61740 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5631a3a372a0 0x5631a3abee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 1 --2- 172.21.15.81:0/3490612756 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5631a3a36ec0 0x5631a3a0df00 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f1414ff9640 0 client.8250 ms_handle_reset on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:01.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 1 -- 172.21.15.81:0/3490612756 >> 172.21.15.81:0/3490612756 conn(0x5631a3a0a300 msgr2=0x5631a3a54c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:01.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 1 -- 172.21.15.81:0/3490612756 shutdown_connections 2023-11-08T16:40:01.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 1 -- 172.21.15.81:0/3490612756 wait complete. 2023-11-08T16:40:01.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.709+0000 7f143e772740 20 client.8250 trim_cache size 0 max 16384 2023-11-08T16:40:01.729 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:40:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:40:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:40:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 8190, 2023-11-08T16:40:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:40:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:40:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:40:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 4.6723773550000001, 2023-11-08T16:40:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 100, 2023-11-08T16:40:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 114, 2023-11-08T16:40:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 114, 2023-11-08T16:40:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 4.7940764949999997 2023-11-08T16:40:01.731 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:40:01.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 -- 172.21.15.81:0/1479769293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141806ae70 msgr2=0x7f141806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141806ae70 0x7f141806d330 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f14340c7160 tx=0x7f1428009040 comp rx=0 tx=0).stop 2023-11-08T16:40:01.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 -- 172.21.15.81:0/1479769293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14340ca130 msgr2=0x7f14340c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14340ca130 0x7f14340c8580 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1430000c00 tx=0x7f14300046c0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 -- 172.21.15.81:0/1479769293 shutdown_connections 2023-11-08T16:40:01.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f141806ae70 0x7f141806d330 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f143415d3b0 0x7f14340b2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f14340cf720 0x7f14340706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 --2- 172.21.15.81:0/1479769293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14340ca130 0x7f14340c8580 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 -- 172.21.15.81:0/1479769293 >> 172.21.15.81:0/1479769293 conn(0x7f14340ceaa0 msgr2=0x7f14340cb250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:01.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 -- 172.21.15.81:0/1479769293 shutdown_connections 2023-11-08T16:40:01.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.710+0000 7f143bb05640 1 -- 172.21.15.81:0/1479769293 wait complete. 2023-11-08T16:40:01.735 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:40:01.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.929+0000 7fc89359e640 1 Processor -- start 2023-11-08T16:40:01.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.930+0000 7fc89359e640 1 -- start start 2023-11-08T16:40:01.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.931+0000 7fc89359e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc8940d39e0 0x7fc8940d1790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.931+0000 7fc89359e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8940ce890 con 0x7fc89415d490 2023-11-08T16:40:01.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.931+0000 7fc89359e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8940ceb70 con 0x7fc8940b5e50 2023-11-08T16:40:01.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.931+0000 7fc89359e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8940d3e40 con 0x7fc8940d39e0 2023-11-08T16:40:01.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.931+0000 7fc89259c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fc89415d490 0x7fc8940d1080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50838/0 (socket says 172.21.15.81:50838) 2023-11-08T16:40:01.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.931+0000 7fc89259c640 1 -- 172.21.15.81:0/487974288 learned_addr learned my addr 172.21.15.81:0/487974288 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:01.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.932+0000 7fc889d9b640 1 --2- 172.21.15.81:0/487974288 >> v2:172.21.15.81:3300/0 conn(0x7fc8940d39e0 0x7fc8940d1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.932+0000 7fc891d9b640 1 -- 172.21.15.81:0/487974288 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3256524127 0 0) 0x7fc8940ce890 con 0x7fc89415d490 2023-11-08T16:40:01.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.932+0000 7fc889d9b640 1 -- 172.21.15.81:0/487974288 >> v1:172.21.15.93:6789/0 conn(0x7fc8940b5e50 legacy=0x7fc8940b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.932+0000 7fc889d9b640 1 -- 172.21.15.81:0/487974288 >> v1:172.21.15.81:6789/0 conn(0x7fc89415d490 legacy=0x7fc8940d1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.932+0000 7fc889d9b640 1 -- 172.21.15.81:0/487974288 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8940cee50 con 0x7fc8940d39e0 2023-11-08T16:40:01.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.933+0000 7fc889d9b640 1 --2- 172.21.15.81:0/487974288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8940d39e0 0x7fc8940d1790 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0097a0 tx=0x7fc87c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cada64f464d30f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:01.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.934+0000 7fc891d9b640 1 -- 172.21.15.81:0/487974288 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c030610 con 0x7fc8940d39e0 2023-11-08T16:40:01.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.934+0000 7fc891d9b640 1 -- 172.21.15.81:0/487974288 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c030770 con 0x7fc8940d39e0 2023-11-08T16:40:01.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.934+0000 7fc891d9b640 1 -- 172.21.15.81:0/487974288 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c030b00 con 0x7fc8940d39e0 2023-11-08T16:40:01.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.934+0000 7fc89359e640 1 -- 172.21.15.81:0/487974288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8940d39e0 msgr2=0x7fc8940d1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.934+0000 7fc89359e640 1 --2- 172.21.15.81:0/487974288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8940d39e0 0x7fc8940d1790 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fc87c0097a0 tx=0x7fc87c02b960 comp rx=0 tx=0).stop 2023-11-08T16:40:01.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.934+0000 7fc89359e640 1 -- 172.21.15.81:0/487974288 shutdown_connections 2023-11-08T16:40:01.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.934+0000 7fc89359e640 1 --2- 172.21.15.81:0/487974288 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8940d39e0 0x7fc8940d1790 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.934+0000 7fc89359e640 1 -- 172.21.15.81:0/487974288 >> 172.21.15.81:0/487974288 conn(0x7fc8940d2e50 msgr2=0x7fc8940d3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:01.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.934+0000 7fc89359e640 1 -- 172.21.15.81:0/487974288 shutdown_connections 2023-11-08T16:40:01.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.934+0000 7fc89359e640 1 -- 172.21.15.81:0/487974288 wait complete. 2023-11-08T16:40:01.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.936+0000 7fc89359e640 1 Processor -- start 2023-11-08T16:40:01.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.936+0000 7fc89359e640 1 -- start start 2023-11-08T16:40:01.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.937+0000 7fc89359e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8940b5e50 0x7fc89415fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.937+0000 7fc89359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8940d39e0 0x7fc894152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.937+0000 7fc89359e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc89415d490 0x7fc8940cf780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.937+0000 7fc89359e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8940cb0b0 con 0x7fc8940d39e0 2023-11-08T16:40:01.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.937+0000 7fc89359e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89415ffd0 con 0x7fc8940b5e50 2023-11-08T16:40:01.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.937+0000 7fc89359e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8940ca4a0 con 0x7fc89415d490 2023-11-08T16:40:01.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.938+0000 7fc889d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8940d39e0 0x7fc894152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.938+0000 7fc889d9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8940d39e0 0x7fc894152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54896/0 (socket says 172.21.15.81:54896) 2023-11-08T16:40:01.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.938+0000 7fc889d9b640 1 -- 172.21.15.81:0/4007793000 learned_addr learned my addr 172.21.15.81:0/4007793000 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:01.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.938+0000 7fc89259c640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8940b5e50 0x7fc89415fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.938+0000 7fc892d9d640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc89415d490 0x7fc8940cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.938+0000 7fc889d9b640 1 -- 172.21.15.81:0/4007793000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc89415d490 msgr2=0x7fc8940cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.938+0000 7fc889d9b640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc89415d490 0x7fc8940cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.938+0000 7fc889d9b640 1 -- 172.21.15.81:0/4007793000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8940b5e50 msgr2=0x7fc89415fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:01.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.938+0000 7fc889d9b640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8940b5e50 0x7fc89415fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:01.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.938+0000 7fc889d9b640 1 -- 172.21.15.81:0/4007793000 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c009400 con 0x7fc8940d39e0 2023-11-08T16:40:01.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.938+0000 7fc892d9d640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc89415d490 0x7fc8940cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:01.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.939+0000 7fc889d9b640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8940d39e0 0x7fc894152270 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fc87c02bf70 tx=0x7fc87c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:01.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.939+0000 7fc88b7fe640 1 -- 172.21.15.81:0/4007793000 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c03e040 con 0x7fc8940d39e0 2023-11-08T16:40:01.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.939+0000 7fc88b7fe640 1 -- 172.21.15.81:0/4007793000 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c039cb0 con 0x7fc8940d39e0 2023-11-08T16:40:01.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.939+0000 7fc88b7fe640 1 -- 172.21.15.81:0/4007793000 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c044630 con 0x7fc8940d39e0 2023-11-08T16:40:01.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.939+0000 7fc89359e640 1 -- 172.21.15.81:0/4007793000 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8940ca780 con 0x7fc8940d39e0 2023-11-08T16:40:01.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.939+0000 7fc89359e640 1 -- 172.21.15.81:0/4007793000 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8940cad40 con 0x7fc8940d39e0 2023-11-08T16:40:01.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.940+0000 7fc89359e640 1 -- 172.21.15.81:0/4007793000 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc894152a90 con 0x7fc8940d39e0 2023-11-08T16:40:01.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.941+0000 7fc88b7fe640 1 -- 172.21.15.81:0/4007793000 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc87c04d3b0 con 0x7fc8940d39e0 2023-11-08T16:40:01.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.942+0000 7fc88b7fe640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc87406aef0 0x7fc87406d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:01.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.942+0000 7fc88b7fe640 1 -- 172.21.15.81:0/4007793000 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fc87c0abfa0 con 0x7fc8940d39e0 2023-11-08T16:40:01.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.942+0000 7fc89259c640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc87406aef0 0x7fc87406d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:01.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.943+0000 7fc89259c640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc87406aef0 0x7fc87406d3b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc884000f60 tx=0x7fc884007d00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:01.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:01.946+0000 7fc88b7fe640 1 -- 172.21.15.81:0/4007793000 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc894152a90 con 0x7fc8940d39e0 2023-11-08T16:40:02.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.172+0000 7fc89359e640 1 -- 172.21.15.81:0/4007793000 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc894152a90 con 0x7fc8940d39e0 2023-11-08T16:40:02.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.173+0000 7fc88b7fe640 1 -- 172.21.15.81:0/4007793000 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4916 (secure 0 0 0) 0x7fc894152a90 con 0x7fc8940d39e0 2023-11-08T16:40:02.175 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:02.175 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":100,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/728323060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":728323060},{"type":"v1","addr":"172.21.15.93:6834","nonce":728323060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3481704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":3481704447},{"type":"v1","addr":"172.21.15.93:6835","nonce":3481704447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8187,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/699802320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":699802320},{"type":"v1","addr":"172.21.15.81:6835","nonce":699802320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:59.048664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/3197735576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3197735576},{"type":"v1","addr":"172.21.15.81:6837","nonce":3197735576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:40:02.175 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 100 2023-11-08T16:40:02.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 -- 172.21.15.81:0/4007793000 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc87406aef0 msgr2=0x7fc87406d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:02.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc87406aef0 0x7fc87406d3b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fc884000f60 tx=0x7fc884007d00 comp rx=0 tx=0).stop 2023-11-08T16:40:02.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 -- 172.21.15.81:0/4007793000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8940d39e0 msgr2=0x7fc894152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:02.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8940d39e0 0x7fc894152270 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fc87c02bf70 tx=0x7fc87c004730 comp rx=0 tx=0).stop 2023-11-08T16:40:02.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 -- 172.21.15.81:0/4007793000 shutdown_connections 2023-11-08T16:40:02.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc87406aef0 0x7fc87406d3b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc89415d490 0x7fc8940cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc8940d39e0 0x7fc894152270 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 --2- 172.21.15.81:0/4007793000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc8940b5e50 0x7fc89415fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 -- 172.21.15.81:0/4007793000 >> 172.21.15.81:0/4007793000 conn(0x7fc8940d2e50 msgr2=0x7fc8940b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:02.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 -- 172.21.15.81:0/4007793000 shutdown_connections 2023-11-08T16:40:02.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.176+0000 7fc89359e640 1 -- 172.21.15.81:0/4007793000 wait complete. 2023-11-08T16:40:02.197 INFO:tasks.cephfs.test_data_scan:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 5, "epoch": 100, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 5, "mdsmap": { "bal_rank_mask": "-1", "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-11-08T16:39:49.779667+0000", "damaged": [], "data_pools": [ 14, 15 ], "enabled": true, "epoch": 100, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true, "refuse_client_session": false, "refuse_standby_for_another_fs": false }, "fs_name": "cephfs", "in": [ 0 ], "info": { "gid_8190": { "addr": "172.21.15.81:6837/3197735576", "addrs": { "addrvec": [ { "addr": "172.21.15.81:6836", "nonce": 3197735576, "type": "v2" }, { "addr": "172.21.15.81:6837", "nonce": 3197735576, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8190, "incarnation": 97, "join_fscid": -1, "name": "c", "rank": 0, "state": "up:active", "state_seq": 4 } }, "last_failure": 0, "last_failure_osd_epoch": 112, "max_file_size": 1099511627776, "max_mds": 1, "max_xattr_size": 65536, "metadata_pool": 13, "modified": "2023-11-08T16:39:59.048664+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 8190 } } } ], "standbys": [ { "addr": "172.21.15.93:6834/728323060", "addrs": { "addrvec": [ { "addr": "172.21.15.93:6832", "nonce": 728323060, "type": "v2" }, { "addr": "172.21.15.93:6834", "nonce": 728323060, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 97, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6196, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.93:6835/3481704447", "addrs": { "addrvec": [ { "addr": "172.21.15.93:6833", "nonce": 3481704447, "type": "v2" }, { "addr": "172.21.15.93:6835", "nonce": 3481704447, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 98, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 6199, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.81:6835/699802320", "addrs": { "addrvec": [ { "addr": "172.21.15.81:6834", "nonce": 699802320, "type": "v2" }, { "addr": "172.21.15.81:6835", "nonce": 699802320, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_11": "minor log segments", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 96, "export_targets": [], "features": 4540138322906710015, "flags": 0, "gid": 8187, "incarnation": 0, "join_fscid": -1, "name": "a", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-11-08T16:40:02.198 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:40:02.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.394+0000 7fcd8759e640 1 Processor -- start 2023-11-08T16:40:02.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.394+0000 7fcd8759e640 1 -- start start 2023-11-08T16:40:02.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.396+0000 7fcd8759e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcd880ced10 0x7fcd880cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:02.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.396+0000 7fcd8759e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fcd8815aea0 0x7fcd8815b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:02.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.396+0000 7fcd8759e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd880cf6b0 con 0x7fcd8815aac0 2023-11-08T16:40:02.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.396+0000 7fcd8759e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd8806f650 con 0x7fcd880ced10 2023-11-08T16:40:02.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.396+0000 7fcd8759e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd88070470 con 0x7fcd8815aea0 2023-11-08T16:40:02.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.396+0000 7fcd8659c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcd8815aac0 0x7fcd880711a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50842/0 (socket says 172.21.15.81:50842) 2023-11-08T16:40:02.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.396+0000 7fcd8659c640 1 -- 172.21.15.81:0/2670461770 learned_addr learned my addr 172.21.15.81:0/2670461770 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:02.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.396+0000 7fcd7ffff640 1 --2- 172.21.15.81:0/2670461770 >> v2:172.21.15.81:3300/0 conn(0x7fcd880ced10 0x7fcd880cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:02.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.397+0000 7fcd86d9d640 1 --2- 172.21.15.81:0/2670461770 >> v2:172.21.15.93:3300/0 conn(0x7fcd8815aea0 0x7fcd8815b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:02.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.397+0000 7fcd85d9b640 1 -- 172.21.15.81:0/2670461770 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186913911 0 0) 0x7fcd880cf6b0 con 0x7fcd8815aac0 2023-11-08T16:40:02.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.397+0000 7fcd85d9b640 1 -- 172.21.15.81:0/2670461770 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd74003520 con 0x7fcd8815aac0 2023-11-08T16:40:02.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.397+0000 7fcd7ffff640 1 -- 172.21.15.81:0/2670461770 >> v2:172.21.15.93:3300/0 conn(0x7fcd8815aea0 msgr2=0x7fcd8815b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:02.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.397+0000 7fcd7ffff640 1 --2- 172.21.15.81:0/2670461770 >> v2:172.21.15.93:3300/0 conn(0x7fcd8815aea0 0x7fcd8815b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.397+0000 7fcd7ffff640 1 -- 172.21.15.81:0/2670461770 >> v1:172.21.15.81:6789/0 conn(0x7fcd8815aac0 legacy=0x7fcd880711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:02.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.397+0000 7fcd7ffff640 1 -- 172.21.15.81:0/2670461770 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd8806d580 con 0x7fcd880ced10 2023-11-08T16:40:02.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.398+0000 7fcd7ffff640 1 --2- 172.21.15.81:0/2670461770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd880ced10 0x7fcd880cf170 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fcd880703e0 tx=0x7fcd7002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d51d785bb1a70e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:02.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.398+0000 7fcd85d9b640 1 -- 172.21.15.81:0/2670461770 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70031830 con 0x7fcd880ced10 2023-11-08T16:40:02.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.398+0000 7fcd85d9b640 1 -- 172.21.15.81:0/2670461770 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd70031990 con 0x7fcd880ced10 2023-11-08T16:40:02.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.398+0000 7fcd85d9b640 1 -- 172.21.15.81:0/2670461770 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd70031d20 con 0x7fcd880ced10 2023-11-08T16:40:02.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.399+0000 7fcd8759e640 1 -- 172.21.15.81:0/2670461770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd880ced10 msgr2=0x7fcd880cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:02.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.399+0000 7fcd8759e640 1 --2- 172.21.15.81:0/2670461770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd880ced10 0x7fcd880cf170 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fcd880703e0 tx=0x7fcd7002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.399+0000 7fcd8759e640 1 -- 172.21.15.81:0/2670461770 shutdown_connections 2023-11-08T16:40:02.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.399+0000 7fcd8759e640 1 --2- 172.21.15.81:0/2670461770 >> v2:172.21.15.93:3300/0 conn(0x7fcd8815aea0 0x7fcd8815b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.399+0000 7fcd8759e640 1 --2- 172.21.15.81:0/2670461770 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd880ced10 0x7fcd880cf170 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.399+0000 7fcd8759e640 1 -- 172.21.15.81:0/2670461770 >> 172.21.15.81:0/2670461770 conn(0x7fcd8806a4a0 msgr2=0x7fcd8806a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:02.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.399+0000 7fcd8759e640 1 -- 172.21.15.81:0/2670461770 shutdown_connections 2023-11-08T16:40:02.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.399+0000 7fcd8759e640 1 -- 172.21.15.81:0/2670461770 wait complete. 2023-11-08T16:40:02.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.401+0000 7fcd8759e640 1 Processor -- start 2023-11-08T16:40:02.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.401+0000 7fcd8759e640 1 -- start start 2023-11-08T16:40:02.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8759e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd880ced10 0x7fcd88156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:02.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8759e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd8815aac0 0x7fcd880a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:02.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8759e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd8815aea0 0x7fcd8815cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:02.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8759e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8815bd00 con 0x7fcd880ced10 2023-11-08T16:40:02.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8759e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8815bfa0 con 0x7fcd8815aac0 2023-11-08T16:40:02.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8759e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd8815c280 con 0x7fcd8815aea0 2023-11-08T16:40:02.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd880ced10 0x7fcd88156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:02.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd880ced10 0x7fcd88156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54906/0 (socket says 172.21.15.81:54906) 2023-11-08T16:40:02.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8659c640 1 -- 172.21.15.81:0/2379298990 learned_addr learned my addr 172.21.15.81:0/2379298990 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:02.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8659c640 1 -- 172.21.15.81:0/2379298990 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd8815aea0 msgr2=0x7fcd8815cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:02.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8659c640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd8815aea0 0x7fcd8815cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8659c640 1 -- 172.21.15.81:0/2379298990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd8815aac0 msgr2=0x7fcd880a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:02.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8659c640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd8815aac0 0x7fcd880a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.402+0000 7fcd8659c640 1 -- 172.21.15.81:0/2379298990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd70009400 con 0x7fcd880ced10 2023-11-08T16:40:02.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.403+0000 7fcd8659c640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd880ced10 0x7fcd88156100 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800c500 tx=0x7fcd7800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:02.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.403+0000 7fcd7effd640 1 -- 172.21.15.81:0/2379298990 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78005ce0 con 0x7fcd880ced10 2023-11-08T16:40:02.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.403+0000 7fcd7effd640 1 -- 172.21.15.81:0/2379298990 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd78005e40 con 0x7fcd880ced10 2023-11-08T16:40:02.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.403+0000 7fcd7effd640 1 -- 172.21.15.81:0/2379298990 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd78007330 con 0x7fcd880ced10 2023-11-08T16:40:02.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.403+0000 7fcd8759e640 1 -- 172.21.15.81:0/2379298990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd8815c560 con 0x7fcd880ced10 2023-11-08T16:40:02.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.403+0000 7fcd8759e640 1 -- 172.21.15.81:0/2379298990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd88156640 con 0x7fcd880ced10 2023-11-08T16:40:02.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.404+0000 7fcd8759e640 1 -- 172.21.15.81:0/2379298990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd8815ca80 con 0x7fcd880ced10 2023-11-08T16:40:02.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.405+0000 7fcd7effd640 1 -- 172.21.15.81:0/2379298990 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcd78007550 con 0x7fcd880ced10 2023-11-08T16:40:02.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.406+0000 7fcd7effd640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6806ae70 0x7fcd6806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:02.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.406+0000 7fcd7effd640 1 -- 172.21.15.81:0/2379298990 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fcd78088fd0 con 0x7fcd880ced10 2023-11-08T16:40:02.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.406+0000 7fcd7ffff640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6806ae70 0x7fcd6806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:02.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.407+0000 7fcd7ffff640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6806ae70 0x7fcd6806d330 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcd70036000 tx=0x7fcd70030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:02.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.410+0000 7fcd7effd640 1 -- 172.21.15.81:0/2379298990 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcd8815ca80 con 0x7fcd880ced10 2023-11-08T16:40:02.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.622+0000 7fcd8759e640 1 -- 172.21.15.81:0/2379298990 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcd8815ca80 con 0x7fcd880ced10 2023-11-08T16:40:02.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.623+0000 7fcd7effd640 1 -- 172.21.15.81:0/2379298990 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v100) v1 ==== 53+0+91 (secure 0 0 0) 0x7fcd8815ca80 con 0x7fcd880ced10 2023-11-08T16:40:02.625 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_extra ] 2023-11-08T16:40:02.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.625+0000 7fcd8759e640 1 -- 172.21.15.81:0/2379298990 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6806ae70 msgr2=0x7fcd6806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:02.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.625+0000 7fcd8759e640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6806ae70 0x7fcd6806d330 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcd70036000 tx=0x7fcd70030000 comp rx=0 tx=0).stop 2023-11-08T16:40:02.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.625+0000 7fcd8759e640 1 -- 172.21.15.81:0/2379298990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd880ced10 msgr2=0x7fcd88156100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:02.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.625+0000 7fcd8759e640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd880ced10 0x7fcd88156100 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800c500 tx=0x7fcd7800c9c0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.626+0000 7fcd8759e640 1 -- 172.21.15.81:0/2379298990 shutdown_connections 2023-11-08T16:40:02.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.626+0000 7fcd8759e640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcd6806ae70 0x7fcd6806d330 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.626+0000 7fcd8759e640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcd8815aea0 0x7fcd8815cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.626+0000 7fcd8759e640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd8815aac0 0x7fcd880a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.626+0000 7fcd8759e640 1 --2- 172.21.15.81:0/2379298990 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcd880ced10 0x7fcd88156100 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:02.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.626+0000 7fcd8759e640 1 -- 172.21.15.81:0/2379298990 >> 172.21.15.81:0/2379298990 conn(0x7fcd8806a4a0 msgr2=0x7fcd880ae8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:02.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.626+0000 7fcd8759e640 1 -- 172.21.15.81:0/2379298990 shutdown_connections 2023-11-08T16:40:02.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:02.626+0000 7fcd8759e640 1 -- 172.21.15.81:0/2379298990 wait complete. 2023-11-08T16:40:02.646 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:40:02.646 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:40:02.647 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:40:02.647 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:40:02.647 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:40:02.648 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:40:02.648 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:40:02.648 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:40:02.648 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:40:02.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:02.649 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:40:02.699 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:40:02.699 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:40:02.700 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:40:02.700 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:40:02.700 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:40:02.700 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:40:02.700 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:40:02.701 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:40:02.701 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:40:02.701 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:40:02.701 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:40:02.701 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:40:02.702 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:40:02.702 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:40:02.702 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:40:02.702 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:40:02.702 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:40:02.703 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 41707sec preferred_lft 41707sec 2023-11-08T16:40:02.703 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:40:02.703 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:40:02.703 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:40:02.703 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff 2023-11-08T16:40:02.704 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:40:02.704 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:40:02.704 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:40:02.704 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:40:02.704 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:40:02.704 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:40:02.705 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:40:02.705 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:40:02.705 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:40:02.705 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:40:02.705 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::2cac:17ff:fe64:67da/64 scope link 2023-11-08T16:40:02.705 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:40:02.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:02.706 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:40:02.793 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:40:02.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:02.797 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:40:02.852 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:40:02.852 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:40:02.852 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:40:02.852 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:40:02.853 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:40:02.853 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:02.853 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:40:02.854 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:40:02.854 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:40:02.855 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:40:02.855 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:40:02.855 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:40:03.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:03.038 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:40:03.088 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:40:03.089 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:40:03.089 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:40:03.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:03.090 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:40:03.179 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:40:03.179 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:40:03.179 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:40:03.236 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:40:03.237 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:40:03.237 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:40:03.291 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:40:03.379 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:40:03.436 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:40:03.436 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:40:03.436 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:40:03.437 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:40:03.437 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:40:03.437 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:40:03.437 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:40:03.437 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:40:03.438 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:40:03.438 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:40:03.438 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:40:03.438 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:40:03.438 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:40:03.439 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:40:03.439 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:40:03.439 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:40:03.439 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:40:03.439 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:40:03.440 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:40:03.440 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:40:03.440 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:40:03.440 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:40:03.440 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:40:03.441 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:40:03.441 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:40:03.441 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:40:03.441 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:40:03.441 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:40:03.441 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:40:03.442 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:40:03.442 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:40:03.442 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:40:03.442 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:40:03.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:03.444 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:40:03.500 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:40:03.501 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:40:03.507 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:40:03.584 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:40:03.628 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:40:03.628 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:40:03.628 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:40:03.629 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:40:03.629 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:40:03.629 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:40:03.629 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:40:03.629 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:40:03.630 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:40:03.630 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:40:03.630 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:40:03.630 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:40:03.630 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:40:03.631 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:40:03.631 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:40:03.631 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:40:03.631 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:40:03.631 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:40:03.632 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:40:03.632 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:40:03.632 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:40:03.632 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:40:03.632 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:40:03.633 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:40:03.633 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:40:03.633 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:40:03.633 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:40:03.633 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:40:03.634 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:40:03.634 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:40:03.634 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:40:03.634 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:40:03.634 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:40:03.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:03.636 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:40:03.694 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:40:03.693+0000 7f7545c3a480 -1 init, newargv = 0x55ad869e8640 newargc=15 2023-11-08T16:40:03.695 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[121968]: starting ceph client 2023-11-08T16:40:03.703 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[121968]: starting fuse 2023-11-08T16:40:03.724 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:40:03.725 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:40:03.725 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:40:03.725 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:40:03.726 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:40:03.727 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:40:03.727 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:40:03.727 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:40:03.728 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:40:03.728 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:40:03.729 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:40:03.729 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:40:03.730 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:40:03.730 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:40:03.730 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:40:03.731 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:40:03.732 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:40:03.732 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:40:03.733 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:40:03.733 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:40:03.733 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:40:03.734 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:40:03.734 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:40:03.735 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:40:03.735 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:40:03.736 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:40:03.736 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:40:03.737 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:40:03.737 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:40:03.920 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.121968.asok 2023-11-08T16:40:03.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:40:03.936 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.121968.asok 2023-11-08T16:40:03.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:03.937 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.121968.asok status 2023-11-08T16:40:04.174 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:40:04.175 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:40:04.175 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:40:04.175 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:40:04.176 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:40:04.176 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:40:04.176 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:40:04.176 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "121968", 2023-11-08T16:40:04.176 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:40:04.177 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:40:04.177 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:40:04.177 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:40:04.177 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 8274, 2023-11-08T16:40:04.177 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:40:04.178 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:40:04.178 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:40:04.178 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 8274 2023-11-08T16:40:04.178 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:40:04.178 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:40:04.179 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:40:04.179 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:40:04.179 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 4196814751 2023-11-08T16:40:04.179 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:40:04.180 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:40:04.180 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:40:04.180 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:40:04.180 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:40:04.181 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 4196814751 2023-11-08T16:40:04.181 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:40:04.181 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.8274 192.168.0.1:0/4196814751", 2023-11-08T16:40:04.181 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/4196814751", 2023-11-08T16:40:04.181 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:40:04.182 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 100, 2023-11-08T16:40:04.182 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 114, 2023-11-08T16:40:04.182 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:40:04.182 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:40:04.182 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:40:04.183 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:40:04.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:04.190 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:40:04.191 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:40:04.246 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:40:04.247 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:40:04.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:04.248 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:40:04.342 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub start / repair,recursive 2023-11-08T16:40:04.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.559+0000 7febb589b640 1 Processor -- start 2023-11-08T16:40:04.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.559+0000 7febb589b640 1 -- start start 2023-11-08T16:40:04.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.561+0000 7febb589b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febb00cd040 0x7febb00b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.561+0000 7febb589b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb0160b40 con 0x7febb00cab40 2023-11-08T16:40:04.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.561+0000 7febb589b640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febb00caf20 con 0x7febb00b0a20 2023-11-08T16:40:04.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.561+0000 7febb589b640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febb006b960 con 0x7febb00cd040 2023-11-08T16:40:04.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.561+0000 7febae7fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7febb00cd040 0x7febb00b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.561+0000 7febaeffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7febb00cab40 0x7febb006b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50856/0 (socket says 172.21.15.81:50856) 2023-11-08T16:40:04.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.561+0000 7febaeffd640 1 -- 172.21.15.81:0/2554515414 learned_addr learned my addr 172.21.15.81:0/2554515414 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:04.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.562+0000 7febae7fc640 1 -- 172.21.15.81:0/2554515414 >> v1:172.21.15.93:6789/0 conn(0x7febb00b0a20 legacy=0x7febb0160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.562+0000 7febadffb640 1 -- 172.21.15.81:0/2554515414 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 89135821 0 0) 0x7febb0160b40 con 0x7febb00cab40 2023-11-08T16:40:04.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.562+0000 7febae7fc640 1 -- 172.21.15.81:0/2554515414 >> v1:172.21.15.81:6789/0 conn(0x7febb00cab40 legacy=0x7febb006b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.562+0000 7febae7fc640 1 -- 172.21.15.81:0/2554515414 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb00b0e30 con 0x7febb00cd040 2023-11-08T16:40:04.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.563+0000 7febae7fc640 1 --2- 172.21.15.81:0/2554515414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb00cd040 0x7febb00b04e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7feba40097a0 tx=0x7feba402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280de812b1c7393b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.563+0000 7febadffb640 1 -- 172.21.15.81:0/2554515414 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba402e070 con 0x7febb00cd040 2023-11-08T16:40:04.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.563+0000 7febadffb640 1 -- 172.21.15.81:0/2554515414 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba40315a0 con 0x7febb00cd040 2023-11-08T16:40:04.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.563+0000 7febadffb640 1 -- 172.21.15.81:0/2554515414 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4031930 con 0x7febb00cd040 2023-11-08T16:40:04.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.564+0000 7febb589b640 1 -- 172.21.15.81:0/2554515414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb00cd040 msgr2=0x7febb00b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.564+0000 7febb589b640 1 --2- 172.21.15.81:0/2554515414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb00cd040 0x7febb00b04e0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7feba40097a0 tx=0x7feba402b840 comp rx=0 tx=0).stop 2023-11-08T16:40:04.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.564+0000 7febb589b640 1 -- 172.21.15.81:0/2554515414 shutdown_connections 2023-11-08T16:40:04.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.564+0000 7febb589b640 1 --2- 172.21.15.81:0/2554515414 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb00cd040 0x7febb00b04e0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.564+0000 7febb589b640 1 -- 172.21.15.81:0/2554515414 >> 172.21.15.81:0/2554515414 conn(0x7febb00cb4e0 msgr2=0x7febb00cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.564+0000 7febb589b640 1 -- 172.21.15.81:0/2554515414 shutdown_connections 2023-11-08T16:40:04.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.564+0000 7febb589b640 1 -- 172.21.15.81:0/2554515414 wait complete. 2023-11-08T16:40:04.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.566+0000 7febb589b640 1 Processor -- start 2023-11-08T16:40:04.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.566+0000 7febb589b640 1 -- start start 2023-11-08T16:40:04.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.567+0000 7febb589b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb00b0a20 0x7febb014f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.567+0000 7febb589b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb00cab40 0x7febb00c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.567+0000 7febb589b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febb00cd040 0x7febb00c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.567+0000 7febb589b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb00c97d0 con 0x7febb00b0a20 2023-11-08T16:40:04.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.567+0000 7febb589b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febb00c8b50 con 0x7febb00cd040 2023-11-08T16:40:04.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.567+0000 7febb589b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febb00c8e30 con 0x7febb00cab40 2023-11-08T16:40:04.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.567+0000 7febaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb00b0a20 0x7febb014f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.567+0000 7febaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb00b0a20 0x7febb014f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54932/0 (socket says 172.21.15.81:54932) 2023-11-08T16:40:04.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.567+0000 7febaeffd640 1 -- 172.21.15.81:0/2540782690 learned_addr learned my addr 172.21.15.81:0/2540782690 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:04.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.568+0000 7febaeffd640 1 -- 172.21.15.81:0/2540782690 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb00cab40 msgr2=0x7febb00c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.568+0000 7febaeffd640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb00cab40 0x7febb00c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.568+0000 7febaeffd640 1 -- 172.21.15.81:0/2540782690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febb00cd040 msgr2=0x7febb00c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.568+0000 7febaeffd640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febb00cd040 0x7febb00c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.568+0000 7febaeffd640 1 -- 172.21.15.81:0/2540782690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba4009400 con 0x7febb00b0a20 2023-11-08T16:40:04.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.568+0000 7febaeffd640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb00b0a20 0x7febb014f120 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7feba000c4a0 tx=0x7feba000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.568+0000 7feb97fff640 1 -- 172.21.15.81:0/2540782690 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba00074e0 con 0x7febb00b0a20 2023-11-08T16:40:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.568+0000 7feb97fff640 1 -- 172.21.15.81:0/2540782690 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba0007640 con 0x7febb00b0a20 2023-11-08T16:40:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.568+0000 7feb97fff640 1 -- 172.21.15.81:0/2540782690 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0004a00 con 0x7febb00b0a20 2023-11-08T16:40:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.568+0000 7febb589b640 1 -- 172.21.15.81:0/2540782690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febb00c9110 con 0x7febb00b0a20 2023-11-08T16:40:04.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.568+0000 7febb589b640 1 -- 172.21.15.81:0/2540782690 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb00c6070 con 0x7febb00b0a20 2023-11-08T16:40:04.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.570+0000 7feb97fff640 1 -- 172.21.15.81:0/2540782690 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feba0004b60 con 0x7febb00b0a20 2023-11-08T16:40:04.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.571+0000 7feb97fff640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06aea0 0x7feb9c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.571+0000 7feb97fff640 1 -- 172.21.15.81:0/2540782690 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7feba0088a60 con 0x7febb00b0a20 2023-11-08T16:40:04.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.572+0000 7febae7fc640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06aea0 0x7feb9c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.572+0000 7febb8431740 1 Processor -- start 2023-11-08T16:40:04.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.572+0000 7febae7fc640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06aea0 0x7feb9c06d360 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feba4036000 tx=0x7feba4030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.572+0000 7febb8431740 1 -- start start 2023-11-08T16:40:04.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.573+0000 7febb8431740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7730 0x55fb484ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.573+0000 7febb8431740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 0x55fb484fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.573+0000 7febb8431740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484cae00 0x55fb484faa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.573+0000 7febb8431740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb4843d5f0 con 0x55fb484f7730 2023-11-08T16:40:04.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.573+0000 7febb8431740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb4843ace0 con 0x55fb484cae00 2023-11-08T16:40:04.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.573+0000 7febb8431740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb484365d0 con 0x55fb484f7b10 2023-11-08T16:40:04.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.574+0000 7febaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7730 0x55fb484ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.574+0000 7febaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7730 0x55fb484ca8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54940/0 (socket says 172.21.15.81:54940) 2023-11-08T16:40:04.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.574+0000 7febaeffd640 1 -- 172.21.15.81:0/1726119879 learned_addr learned my addr 172.21.15.81:0/1726119879 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:04.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.574+0000 7febaeffd640 1 -- 172.21.15.81:0/1726119879 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 msgr2=0x55fb484fa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.574+0000 7febaeffd640 1 --2- 172.21.15.81:0/1726119879 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 0x55fb484fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.574+0000 7febaeffd640 1 -- 172.21.15.81:0/1726119879 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484cae00 msgr2=0x55fb484faa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.574+0000 7febaeffd640 1 --2- 172.21.15.81:0/1726119879 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484cae00 0x55fb484faa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.574+0000 7febaeffd640 1 -- 172.21.15.81:0/1726119879 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb48410a60 con 0x55fb484f7730 2023-11-08T16:40:04.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.574+0000 7febaeffd640 1 --2- 172.21.15.81:0/1726119879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7730 0x55fb484ca8c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7feba0086be0 tx=0x7feba0057c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a038f54f7601075f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.574+0000 7feb95ffb640 1 -- 172.21.15.81:0/1726119879 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba005f7b0 con 0x55fb484f7730 2023-11-08T16:40:04.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.575+0000 7feb95ffb640 1 -- 172.21.15.81:0/1726119879 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba0004e10 con 0x55fb484f7730 2023-11-08T16:40:04.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.575+0000 7feb95ffb640 1 -- 172.21.15.81:0/1726119879 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0053bc0 con 0x55fb484f7730 2023-11-08T16:40:04.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.575+0000 7febb8431740 1 -- 172.21.15.81:0/1726119879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7730 msgr2=0x55fb484ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.575+0000 7febb8431740 1 --2- 172.21.15.81:0/1726119879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7730 0x55fb484ca8c0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7feba0086be0 tx=0x7feba0057c70 comp rx=0 tx=0).stop 2023-11-08T16:40:04.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.575+0000 7febb8431740 1 -- 172.21.15.81:0/1726119879 shutdown_connections 2023-11-08T16:40:04.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.575+0000 7febb8431740 1 --2- 172.21.15.81:0/1726119879 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484cae00 0x55fb484faa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.575+0000 7febb8431740 1 --2- 172.21.15.81:0/1726119879 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 0x55fb484fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.575+0000 7febb8431740 1 --2- 172.21.15.81:0/1726119879 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7730 0x55fb484ca8c0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.575+0000 7febb8431740 1 -- 172.21.15.81:0/1726119879 >> 172.21.15.81:0/1726119879 conn(0x55fb484ccbd0 msgr2=0x55fb485154c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.575+0000 7febb8431740 1 -- 172.21.15.81:0/1726119879 shutdown_connections 2023-11-08T16:40:04.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.575+0000 7febb8431740 1 -- 172.21.15.81:0/1726119879 wait complete. 2023-11-08T16:40:04.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.576+0000 7febb8431740 1 -- start start 2023-11-08T16:40:04.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.576+0000 7febb8431740 1 Processor -- start 2023-11-08T16:40:04.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.577+0000 7febb8431740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 0x55fb484f9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.577+0000 7febb8431740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 0x55fb484f0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.577+0000 7febb8431740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb484fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.577+0000 7febb8431740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb483d3590 con 0x55fb484f3170 2023-11-08T16:40:04.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.577+0000 7febb8431740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb483d3230 con 0x55fb484f7730 2023-11-08T16:40:04.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.577+0000 7febb8431740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb4819a0c0 con 0x55fb484f7b10 2023-11-08T16:40:04.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.577+0000 7febb8431740 10 client.0 _ll_register_callbacks cb 0x55fb483f5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:40:04.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.578+0000 7febaf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb484fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.578+0000 7febaf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb484fbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54954/0 (socket says 172.21.15.81:54954) 2023-11-08T16:40:04.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.578+0000 7febaf7fe640 1 -- 172.21.15.81:0/2211499091 learned_addr learned my addr 172.21.15.81:0/2211499091 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:04.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.578+0000 7febaf7fe640 1 -- 172.21.15.81:0/2211499091 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 msgr2=0x55fb484f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.578+0000 7febaf7fe640 1 --2- 172.21.15.81:0/2211499091 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 0x55fb484f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.578+0000 7febaf7fe640 1 -- 172.21.15.81:0/2211499091 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 msgr2=0x55fb484f9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.578+0000 7febaf7fe640 1 --2- 172.21.15.81:0/2211499091 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 0x55fb484f9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.578+0000 7febaf7fe640 1 -- 172.21.15.81:0/2211499091 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febb00c6070 con 0x55fb484f3170 2023-11-08T16:40:04.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.578+0000 7febaf7fe640 1 --2- 172.21.15.81:0/2211499091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb484fbcf0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7feba800c160 tx=0x7feba800c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89b989a7c14fd4e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.578+0000 7feb8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:40:04.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.578+0000 7feb8e7fc640 1 -- 172.21.15.81:0/2211499091 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8003b70 con 0x55fb484f3170 2023-11-08T16:40:04.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7feb8e7fc640 1 -- 172.21.15.81:0/2211499091 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7feba8003cd0 con 0x55fb484f3170 2023-11-08T16:40:04.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7febb8431740 1 -- 172.21.15.81:0/2211499091 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb4845c910 con 0x55fb484f3170 2023-11-08T16:40:04.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7feb8e7fc640 1 -- 172.21.15.81:0/2211499091 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7feba8001630 con 0x55fb484f3170 2023-11-08T16:40:04.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7febb8431740 10 client.8289 fetch_fsmap learned FSMap version 100 2023-11-08T16:40:04.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7febb8431740 1 -- 172.21.15.81:0/2211499091 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x55fb483d1810 con 0x55fb484f3170 2023-11-08T16:40:04.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7feb8e7fc640 1 -- 172.21.15.81:0/2211499091 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fb4845c910 con 0x55fb484f3170 2023-11-08T16:40:04.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7feb8e7fc640 1 -- 172.21.15.81:0/2211499091 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 100) v1 ==== 3302+0+0 (secure 0 0 0) 0x7feba8006b50 con 0x55fb484f3170 2023-11-08T16:40:04.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7febb8431740 10 client.8289 fetch_fsmap finished waiting for FSMap version 100 2023-11-08T16:40:04.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7febb8431740 10 client.8289 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-08T16:40:04.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7febb8431740 20 client.8289 populate_metadata read hostname 'smithi081' 2023-11-08T16:40:04.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7febb8431740 1 --2- 172.21.15.81:0/2211499091 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55fb484e19b0 0x55fb484fc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7febb8431740 4 client.8289 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:40:04.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.579+0000 7febb8431740 1 -- 172.21.15.81:0/2211499091 --> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb48409450 con 0x55fb484e19b0 2023-11-08T16:40:04.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.580+0000 7febaeffd640 1 --2- 172.21.15.81:0/2211499091 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55fb484e19b0 0x55fb484fc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.580+0000 7febaeffd640 1 --2- 172.21.15.81:0/2211499091 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55fb484e19b0 0x55fb484fc230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c8159224e01c08c server_cookie=14d773e6c6970cce in_seq=0 out_seq=0 2023-11-08T16:40:04.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.580+0000 7feb8e7fc640 10 client.8289 ms_handle_connect on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:04.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7feb8e7fc640 1 -- 172.21.15.81:0/2211499091 <== mds.0 v2:172.21.15.81:6836/3197735576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb48409450 con 0x55fb484e19b0 2023-11-08T16:40:04.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7feb8e7fc640 10 client.8289 handle_command_reply: tid=0 2023-11-08T16:40:04.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 1 client.8289 shutdown 2023-11-08T16:40:04.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 1 -- 172.21.15.81:0/2211499091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 msgr2=0x55fb484fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 1 --2- 172.21.15.81:0/2211499091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb484fbcf0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7feba800c160 tx=0x7feba800c620 comp rx=0 tx=0).stop 2023-11-08T16:40:04.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 1 -- 172.21.15.81:0/2211499091 shutdown_connections 2023-11-08T16:40:04.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 1 --2- 172.21.15.81:0/2211499091 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb484fbcf0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 1 --2- 172.21.15.81:0/2211499091 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55fb484e19b0 0x55fb484fc230 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 1 --2- 172.21.15.81:0/2211499091 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 0x55fb484f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 1 --2- 172.21.15.81:0/2211499091 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 0x55fb484f9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7feb8e7fc640 0 client.8289 ms_handle_reset on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:04.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 1 -- 172.21.15.81:0/2211499091 >> 172.21.15.81:0/2211499091 conn(0x55fb484cae00 msgr2=0x55fb484ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 1 -- 172.21.15.81:0/2211499091 shutdown_connections 2023-11-08T16:40:04.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 1 -- 172.21.15.81:0/2211499091 wait complete. 2023-11-08T16:40:04.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.581+0000 7febb8431740 20 client.8289 trim_cache size 0 max 16384 2023-11-08T16:40:04.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.597+0000 7febb8431740 1 Processor -- start 2023-11-08T16:40:04.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.598+0000 7febb8431740 1 -- start start 2023-11-08T16:40:04.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.598+0000 7febb8431740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f3170 0x55fb484ce230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.598+0000 7febb8431740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 0x55fb484ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.598+0000 7febb8431740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7b10 0x55fb484cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.598+0000 7febb8431740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb484f22d0 con 0x55fb484f7b10 2023-11-08T16:40:04.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.598+0000 7febb8431740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb484ef210 con 0x55fb484f7730 2023-11-08T16:40:04.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.598+0000 7febb8431740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb484cb940 con 0x55fb484f3170 2023-11-08T16:40:04.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.599+0000 7febaf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7b10 0x55fb484cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.599+0000 7febaf7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7b10 0x55fb484cf3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54960/0 (socket says 172.21.15.81:54960) 2023-11-08T16:40:04.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.599+0000 7febaf7fe640 1 -- 172.21.15.81:0/1455709296 learned_addr learned my addr 172.21.15.81:0/1455709296 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:04.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.599+0000 7febaf7fe640 1 -- 172.21.15.81:0/1455709296 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f3170 msgr2=0x55fb484ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.599+0000 7febaf7fe640 1 --2- 172.21.15.81:0/1455709296 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f3170 0x55fb484ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.599+0000 7febaf7fe640 1 -- 172.21.15.81:0/1455709296 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 msgr2=0x55fb484ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.599+0000 7febaf7fe640 1 --2- 172.21.15.81:0/1455709296 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 0x55fb484ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.599+0000 7febaf7fe640 1 -- 172.21.15.81:0/1455709296 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb48465c60 con 0x55fb484f7b10 2023-11-08T16:40:04.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.599+0000 7febaf7fe640 1 --2- 172.21.15.81:0/1455709296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7b10 0x55fb484cf3d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7feba8007cd0 tx=0x7feba8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a473118a6c92ed server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7feb967fc640 1 -- 172.21.15.81:0/1455709296 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba8025030 con 0x55fb484f7b10 2023-11-08T16:40:04.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7feb967fc640 1 -- 172.21.15.81:0/1455709296 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba800b070 con 0x55fb484f7b10 2023-11-08T16:40:04.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7feb967fc640 1 -- 172.21.15.81:0/1455709296 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba800cb20 con 0x55fb484f7b10 2023-11-08T16:40:04.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7febb8431740 1 -- 172.21.15.81:0/1455709296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7b10 msgr2=0x55fb484cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7febb8431740 1 --2- 172.21.15.81:0/1455709296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7b10 0x55fb484cf3d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7feba8007cd0 tx=0x7feba8011000 comp rx=0 tx=0).stop 2023-11-08T16:40:04.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7febb8431740 1 -- 172.21.15.81:0/1455709296 shutdown_connections 2023-11-08T16:40:04.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7febb8431740 1 --2- 172.21.15.81:0/1455709296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f7b10 0x55fb484cf3d0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7febb8431740 1 --2- 172.21.15.81:0/1455709296 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 0x55fb484ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7febb8431740 1 --2- 172.21.15.81:0/1455709296 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f3170 0x55fb484ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7febb8431740 1 -- 172.21.15.81:0/1455709296 >> 172.21.15.81:0/1455709296 conn(0x55fb484cae00 msgr2=0x55fb484f5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7febb8431740 1 -- 172.21.15.81:0/1455709296 shutdown_connections 2023-11-08T16:40:04.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.600+0000 7febb8431740 1 -- 172.21.15.81:0/1455709296 wait complete. 2023-11-08T16:40:04.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.601+0000 7febb8431740 1 -- start start 2023-11-08T16:40:04.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.601+0000 7febb8431740 1 Processor -- start 2023-11-08T16:40:04.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.602+0000 7febb8431740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb4850ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.602+0000 7febb8431740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 0x55fb4857fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.602+0000 7febb8431740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 0x55fb484ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.602+0000 7febb8431740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb484f9340 con 0x55fb484f3170 2023-11-08T16:40:04.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.602+0000 7febb8431740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb4850b3b0 con 0x55fb484f7730 2023-11-08T16:40:04.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.602+0000 7febb8431740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb48518dd0 con 0x55fb484f7b10 2023-11-08T16:40:04.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.602+0000 7febb8431740 10 client.0 _ll_register_callbacks cb 0x7feba8001510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:40:04.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.602+0000 7febaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb4850ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7febaeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb4850ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54976/0 (socket says 172.21.15.81:54976) 2023-11-08T16:40:04.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7febaeffd640 1 -- 172.21.15.81:0/4098537603 learned_addr learned my addr 172.21.15.81:0/4098537603 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:04.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7febaeffd640 1 -- 172.21.15.81:0/4098537603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 msgr2=0x55fb484ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7febaeffd640 1 --2- 172.21.15.81:0/4098537603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 0x55fb484ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7febaeffd640 1 -- 172.21.15.81:0/4098537603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 msgr2=0x55fb4857fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7febaeffd640 1 --2- 172.21.15.81:0/4098537603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 0x55fb4857fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7febaeffd640 1 -- 172.21.15.81:0/4098537603 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb483d1810 con 0x55fb484f3170 2023-11-08T16:40:04.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7febaeffd640 1 --2- 172.21.15.81:0/4098537603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb4850ae50 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7feba0007fd0 tx=0x7feba000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90f07f70f6f7398 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7feb8e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:40:04.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7feb8e7fc640 1 -- 172.21.15.81:0/4098537603 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba0086040 con 0x55fb484f3170 2023-11-08T16:40:04.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7feb8e7fc640 1 -- 172.21.15.81:0/4098537603 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7feba0085040 con 0x55fb484f3170 2023-11-08T16:40:04.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7febb8431740 1 -- 172.21.15.81:0/4098537603 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feba0004e10 con 0x55fb484f3170 2023-11-08T16:40:04.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.603+0000 7feb8e7fc640 1 -- 172.21.15.81:0/4098537603 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7feba005f830 con 0x55fb484f3170 2023-11-08T16:40:04.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.604+0000 7febb8431740 10 client.8295 fetch_fsmap learned FSMap version 100 2023-11-08T16:40:04.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.604+0000 7febb8431740 1 -- 172.21.15.81:0/4098537603 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x55fb483d2580 con 0x55fb484f3170 2023-11-08T16:40:04.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.604+0000 7feb8e7fc640 1 -- 172.21.15.81:0/4098537603 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba0004e10 con 0x55fb484f3170 2023-11-08T16:40:04.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.604+0000 7feb8e7fc640 1 -- 172.21.15.81:0/4098537603 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 100) v1 ==== 3302+0+0 (secure 0 0 0) 0x7feba0053bc0 con 0x55fb484f3170 2023-11-08T16:40:04.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.604+0000 7febb8431740 10 client.8295 fetch_fsmap finished waiting for FSMap version 100 2023-11-08T16:40:04.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.604+0000 7febb8431740 10 client.8295 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-08T16:40:04.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.604+0000 7febb8431740 20 client.8295 populate_metadata read hostname 'smithi081' 2023-11-08T16:40:04.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.604+0000 7febb8431740 1 --2- 172.21.15.81:0/4098537603 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55fb484f70b0 0x55fb4852e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.604+0000 7febb8431740 4 client.8295 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-08T16:40:04.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.604+0000 7febb8431740 1 -- 172.21.15.81:0/4098537603 --> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55fb4842f5b0 con 0x55fb484f70b0 2023-11-08T16:40:04.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.604+0000 7febae7fc640 1 --2- 172.21.15.81:0/4098537603 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55fb484f70b0 0x55fb4852e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.605+0000 7febae7fc640 1 --2- 172.21.15.81:0/4098537603 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55fb484f70b0 0x55fb4852e820 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5feb3a8b98f5f975 server_cookie=66bcc852cd15d0f9 in_seq=0 out_seq=0 2023-11-08T16:40:04.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.605+0000 7feb8e7fc640 10 client.8295 ms_handle_connect on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:04.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7feb8e7fc640 1 -- 172.21.15.81:0/4098537603 <== mds.0 v2:172.21.15.81:6836/3197735576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fb4842f5b0 con 0x55fb484f70b0 2023-11-08T16:40:04.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7feb8e7fc640 10 client.8295 handle_command_reply: tid=0 2023-11-08T16:40:04.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 1 client.8295 shutdown 2023-11-08T16:40:04.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 1 -- 172.21.15.81:0/4098537603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 msgr2=0x55fb4850ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 1 --2- 172.21.15.81:0/4098537603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb4850ae50 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7feba0007fd0 tx=0x7feba000cf90 comp rx=0 tx=0).stop 2023-11-08T16:40:04.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 1 -- 172.21.15.81:0/4098537603 shutdown_connections 2023-11-08T16:40:04.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 1 --2- 172.21.15.81:0/4098537603 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55fb484f70b0 0x55fb4852e820 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 1 --2- 172.21.15.81:0/4098537603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55fb484f7b10 0x55fb484ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 1 --2- 172.21.15.81:0/4098537603 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55fb484f7730 0x55fb4857fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 1 --2- 172.21.15.81:0/4098537603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55fb484f3170 0x55fb4850ae50 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 1 -- 172.21.15.81:0/4098537603 >> 172.21.15.81:0/4098537603 conn(0x55fb484cae00 msgr2=0x55fb484df760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7feb8e7fc640 0 client.8295 ms_handle_reset on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:04.622 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.606+0000 7f9540baf640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x4 2023-11-08T16:40:04.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 1 -- 172.21.15.81:0/4098537603 shutdown_connections 2023-11-08T16:40:04.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 1 -- 172.21.15.81:0/4098537603 wait complete. 2023-11-08T16:40:04.623 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.606+0000 7f9540baf640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-11-08T16:40:04.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.606+0000 7febb8431740 20 client.8295 trim_cache size 0 max 16384 2023-11-08T16:40:04.624 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:40:04.624 INFO:teuthology.orchestra.run.smithi081.stdout: "return_code": 0, 2023-11-08T16:40:04.624 INFO:teuthology.orchestra.run.smithi081.stdout: "scrub_tag": "a6e3f8ec-acc8-4573-ae28-3fef5c33d28e", 2023-11-08T16:40:04.624 INFO:teuthology.orchestra.run.smithi081.stdout: "mode": "asynchronous" 2023-11-08T16:40:04.625 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:40:04.625 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.607+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /stripey/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55f906dbdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/stripey v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:40:04.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.607+0000 7febb589b640 1 -- 172.21.15.81:0/2540782690 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06aea0 msgr2=0x7feb9c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.607+0000 7febb589b640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06aea0 0x7feb9c06d360 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7feba4036000 tx=0x7feba4030000 comp rx=0 tx=0).stop 2023-11-08T16:40:04.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.607+0000 7febb589b640 1 -- 172.21.15.81:0/2540782690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb00b0a20 msgr2=0x7febb014f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.607+0000 7febb589b640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb00b0a20 0x7febb014f120 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7feba000c4a0 tx=0x7feba000c960 comp rx=0 tx=0).stop 2023-11-08T16:40:04.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.608+0000 7febb589b640 1 -- 172.21.15.81:0/2540782690 shutdown_connections 2023-11-08T16:40:04.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.608+0000 7febb589b640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7feb9c06aea0 0x7feb9c06d360 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.608+0000 7febb589b640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7febb00cd040 0x7febb00c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.608+0000 7febb589b640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7febb00cab40 0x7febb00c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.608+0000 7febb589b640 1 --2- 172.21.15.81:0/2540782690 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7febb00b0a20 0x7febb014f120 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.608+0000 7febb589b640 1 -- 172.21.15.81:0/2540782690 >> 172.21.15.81:0/2540782690 conn(0x7febb00cb4e0 msgr2=0x7febb015e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.628 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.608+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x4 [...2,head] /lost+found/ auth v1 pv6 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55f906dbe100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x4:[<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:40:04.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.608+0000 7febb589b640 1 -- 172.21.15.81:0/2540782690 shutdown_connections 2023-11-08T16:40:04.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.608+0000 7febb589b640 1 -- 172.21.15.81:0/2540782690 wait complete. 2023-11-08T16:40:04.630 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.609+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /stripey/flushed_file_4 auth v1 ap=1 DIRTYPARENT s=1441792 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e19180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001f9:[<0x10000000000/flushed_file_4 v70>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001f9:[<0x10000000000/flushed_file_4 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:04.630 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.610+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /stripey/flushed_file_6 auth v1 ap=1 DIRTYPARENT s=1572864 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e18680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001fb:[<0x10000000000/flushed_file_6 v68>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001fb:[<0x10000000000/flushed_file_6 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:04.630 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.610+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /stripey/flushed_file_2 auth v1 ap=1 DIRTYPARENT s=917504 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e3db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001f7:[<0x10000000000/flushed_file_2 v56>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001f7:[<0x10000000000/flushed_file_2 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:04.631 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.610+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /stripey/flushed_file_3 auth v1 ap=1 DIRTYPARENT s=917503 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e19700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001f8:[<0x10000000000/flushed_file_3 v62>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001f8:[<0x10000000000/flushed_file_3 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:04.631 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.611+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /stripey/flushed_file_0 auth v1 ap=1 DIRTYPARENT s=1048576 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e3c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x10000000001:[<0x10000000000/flushed_file_0 v36>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x10000000001:[<0x10000000000/flushed_file_0 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:04.631 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.611+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /stripey/flushed_file_1 auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e3d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001f6:[<0x10000000000/flushed_file_1 v48>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001f6:[<0x10000000000/flushed_file_1 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:04.631 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.611+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /lost+found/100000001fc auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e3c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x100000001fc:[<0x4/100000001fc v1>,<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-11-08T16:40:04.631 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.611+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /stripey/flushed_file_5 auth v1 ap=1 DIRTYPARENT s=1441793 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e18c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001fa:[<0x10000000000/flushed_file_5 v54>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001fa:[<0x10000000000/flushed_file_5 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:04.631 INFO:tasks.ceph.mds.c.smithi081.stderr:2023-11-08T16:40:04.621+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v7 ap=5 snaprealm=0x55f906db6fc0 DIRTYPARENT f(v1 2=0+2) n(v0 rc2023-11-08T16:40:04.337118+0000 2=0+2)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={8274=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55f906dbd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 2=0+2)","ondisk_value.rstat":"n(v0 3=0+3)","memory_value.dirstat":"f(v1 2=0+2)","memory_value.rstat":"n(v0 rc2023-11-08T16:40:04.337118+0000 2=0+2)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:40:04.632 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2023-11-08T16:40:04.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.824+0000 7ff803a6b640 1 Processor -- start 2023-11-08T16:40:04.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.825+0000 7ff803a6b640 1 -- start start 2023-11-08T16:40:04.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.826+0000 7ff803a6b640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff7fc0add40 0x7ff7fc06cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.826+0000 7ff803a6b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc0ce5d0 0x7ff7fc0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.826+0000 7ff803a6b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7fc165f90 con 0x7ff7fc0caa80 2023-11-08T16:40:04.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.826+0000 7ff803a6b640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7fc0cb280 con 0x7ff7fc0add40 2023-11-08T16:40:04.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.826+0000 7ff803a6b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc06d400 con 0x7ff7fc0ce5d0 2023-11-08T16:40:04.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.826+0000 7ff801006640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff7fc0caa80 0x7ff7fc0cdec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:50860/0 (socket says 172.21.15.81:50860) 2023-11-08T16:40:04.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.826+0000 7ff801006640 1 -- 172.21.15.81:0/3033101812 learned_addr learned my addr 172.21.15.81:0/3033101812 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:04.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff802008640 1 --2- 172.21.15.81:0/3033101812 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc0ce5d0 0x7ff7fc0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff801807640 1 --2- 172.21.15.81:0/3033101812 >> v2:172.21.15.93:3300/0 conn(0x7ff7fc0add40 0x7ff7fc06cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806248615 0 0) 0x7ff7fc165f90 con 0x7ff7fc0caa80 2023-11-08T16:40:04.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7e8003520 con 0x7ff7fc0caa80 2023-11-08T16:40:04.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3517309238 0 0) 0x7ff7e8003520 con 0x7ff7fc0caa80 2023-11-08T16:40:04.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc0ce5d0 msgr2=0x7ff7fc0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 --2- 172.21.15.81:0/3033101812 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc0ce5d0 0x7ff7fc0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 >> v2:172.21.15.93:3300/0 conn(0x7ff7fc0add40 msgr2=0x7ff7fc06cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 --2- 172.21.15.81:0/3033101812 >> v2:172.21.15.93:3300/0 conn(0x7ff7fc0add40 0x7ff7fc06cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc0ae120 con 0x7ff7fc0caa80 2023-11-08T16:40:04.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ff7ec001670 con 0x7ff7fc0caa80 2023-11-08T16:40:04.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 >> v1:172.21.15.81:6789/0 conn(0x7ff7fc0caa80 legacy=0x7ff7fc0cdec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 --2- 172.21.15.81:0/3033101812 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e80039e0 0x7ff7e8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 --2- 172.21.15.81:0/3033101812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e80047c0 0x7ff7e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 --2- 172.21.15.81:0/3033101812 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc0ce5d0 0x7ff7e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0cb280 con 0x7ff7e80047c0 2023-11-08T16:40:04.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc06d400 con 0x7ff7e80039e0 2023-11-08T16:40:04.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.827+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e800bab0 con 0x7ff7fc0ce5d0 2023-11-08T16:40:04.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.828+0000 7ff801006640 1 --2- 172.21.15.81:0/3033101812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e80047c0 0x7ff7e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.828+0000 7ff801006640 1 -- 172.21.15.81:0/3033101812 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc0ce5d0 msgr2=0x7ff7e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.828+0000 7ff801006640 1 --2- 172.21.15.81:0/3033101812 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc0ce5d0 0x7ff7e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.828+0000 7ff801006640 1 -- 172.21.15.81:0/3033101812 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e80039e0 msgr2=0x7ff7e8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.828+0000 7ff801006640 1 --2- 172.21.15.81:0/3033101812 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e80039e0 0x7ff7e8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.828+0000 7ff801006640 1 -- 172.21.15.81:0/3033101812 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e8003520 con 0x7ff7e80047c0 2023-11-08T16:40:04.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.829+0000 7ff801006640 1 --2- 172.21.15.81:0/3033101812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e80047c0 0x7ff7e8004c40 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec001eb0 tx=0x7ff7e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2152152ea2c30ea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.829+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec005ad0 con 0x7ff7e80047c0 2023-11-08T16:40:04.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.830+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec005c30 con 0x7ff7e80047c0 2023-11-08T16:40:04.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.830+0000 7ff800805640 1 -- 172.21.15.81:0/3033101812 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec02c890 con 0x7ff7e80047c0 2023-11-08T16:40:04.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.830+0000 7ff803a6b640 1 -- 172.21.15.81:0/3033101812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e80047c0 msgr2=0x7ff7e8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.830+0000 7ff803a6b640 1 --2- 172.21.15.81:0/3033101812 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7e80047c0 0x7ff7e8004c40 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec001eb0 tx=0x7ff7e80057f0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.830+0000 7ff802008640 1 -- 172.21.15.81:0/3033101812 reap_dead start 2023-11-08T16:40:04.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.830+0000 7ff803a6b640 1 -- 172.21.15.81:0/3033101812 shutdown_connections 2023-11-08T16:40:04.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.830+0000 7ff803a6b640 1 -- 172.21.15.81:0/3033101812 >> 172.21.15.81:0/3033101812 conn(0x7ff7fc0cb4e0 msgr2=0x7ff7fc0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.830+0000 7ff803a6b640 1 -- 172.21.15.81:0/3033101812 shutdown_connections 2023-11-08T16:40:04.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.830+0000 7ff803a6b640 1 -- 172.21.15.81:0/3033101812 wait complete. 2023-11-08T16:40:04.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.833+0000 7ff803a6b640 1 Processor -- start 2023-11-08T16:40:04.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.833+0000 7ff803a6b640 1 -- start start 2023-11-08T16:40:04.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.834+0000 7ff803a6b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e80047c0 0x7ff7fc153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.834+0000 7ff803a6b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc1f7210 0x7ff7fc163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.834+0000 7ff803a6b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc164020 0x7ff7fc0c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.834+0000 7ff803a6b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc1644a0 con 0x7ff7fc1f7210 2023-11-08T16:40:04.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.834+0000 7ff803a6b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0b2130 con 0x7ff7e80047c0 2023-11-08T16:40:04.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.834+0000 7ff803a6b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0b2c70 con 0x7ff7fc164020 2023-11-08T16:40:04.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.835+0000 7ff801006640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc1f7210 0x7ff7fc163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.835+0000 7ff801006640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc1f7210 0x7ff7fc163ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54990/0 (socket says 172.21.15.81:54990) 2023-11-08T16:40:04.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.835+0000 7ff801006640 1 -- 172.21.15.81:0/849090250 learned_addr learned my addr 172.21.15.81:0/849090250 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:04.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.835+0000 7ff801807640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e80047c0 0x7ff7fc153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.835+0000 7ff802008640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc164020 0x7ff7fc0c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.835+0000 7ff801006640 1 -- 172.21.15.81:0/849090250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc164020 msgr2=0x7ff7fc0c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.835+0000 7ff801006640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc164020 0x7ff7fc0c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.835+0000 7ff801006640 1 -- 172.21.15.81:0/849090250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e80047c0 msgr2=0x7ff7fc153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.835+0000 7ff801006640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e80047c0 0x7ff7fc153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.835+0000 7ff801006640 1 -- 172.21.15.81:0/849090250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc154490 con 0x7ff7fc1f7210 2023-11-08T16:40:04.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.835+0000 7ff801807640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e80047c0 0x7ff7fc153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:04.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.836+0000 7ff801006640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc1f7210 0x7ff7fc163ae0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec001eb0 tx=0x7ff7fc166120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.837+0000 7ff7f27fc640 1 -- 172.21.15.81:0/849090250 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec039030 con 0x7ff7fc1f7210 2023-11-08T16:40:04.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.837+0000 7ff7f27fc640 1 -- 172.21.15.81:0/849090250 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec02ccc0 con 0x7ff7fc1f7210 2023-11-08T16:40:04.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.837+0000 7ff7f27fc640 1 -- 172.21.15.81:0/849090250 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec03c040 con 0x7ff7fc1f7210 2023-11-08T16:40:04.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.837+0000 7ff803a6b640 1 -- 172.21.15.81:0/849090250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7fc162e90 con 0x7ff7fc1f7210 2023-11-08T16:40:04.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.838+0000 7ff803a6b640 1 -- 172.21.15.81:0/849090250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc0b2620 con 0x7ff7fc1f7210 2023-11-08T16:40:04.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.839+0000 7ff7f27fc640 1 -- 172.21.15.81:0/849090250 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff7ec032070 con 0x7ff7fc1f7210 2023-11-08T16:40:04.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.840+0000 7ff7f27fc640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7e006aec0 0x7ff7e006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.840+0000 7ff7f27fc640 1 -- 172.21.15.81:0/849090250 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7ff7ec0aab40 con 0x7ff7fc1f7210 2023-11-08T16:40:04.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.841+0000 7ff801807640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7e006aec0 0x7ff7e006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.841+0000 7ff8065fd740 1 Processor -- start 2023-11-08T16:40:04.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.842+0000 7ff8065fd740 1 -- start start 2023-11-08T16:40:04.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.842+0000 7ff801807640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7e006aec0 0x7ff7e006d380 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8007520 tx=0x7ff7f800b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.842+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231270 0x5626cb2041d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.842+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231650 0x5626cb234000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.842+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb204710 0x5626cb234540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.842+0000 7ff8065fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cb176b60 con 0x5626cb231270 2023-11-08T16:40:04.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.842+0000 7ff8065fd740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cb174250 con 0x5626cb231650 2023-11-08T16:40:04.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.842+0000 7ff8065fd740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626cb16fb40 con 0x5626cb204710 2023-11-08T16:40:04.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.843+0000 7ff801807640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231270 0x5626cb2041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.843+0000 7ff801807640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231270 0x5626cb2041d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54994/0 (socket says 172.21.15.81:54994) 2023-11-08T16:40:04.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.843+0000 7ff801807640 1 -- 172.21.15.81:0/4291717063 learned_addr learned my addr 172.21.15.81:0/4291717063 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:04.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.843+0000 7ff801807640 1 -- 172.21.15.81:0/4291717063 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb204710 msgr2=0x5626cb234540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.843+0000 7ff801807640 1 --2- 172.21.15.81:0/4291717063 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb204710 0x5626cb234540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.843+0000 7ff801807640 1 -- 172.21.15.81:0/4291717063 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231650 msgr2=0x5626cb234000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.843+0000 7ff801807640 1 --2- 172.21.15.81:0/4291717063 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231650 0x5626cb234000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.843+0000 7ff801807640 1 -- 172.21.15.81:0/4291717063 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626cb16e1c0 con 0x5626cb231270 2023-11-08T16:40:04.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.844+0000 7ff801807640 1 --2- 172.21.15.81:0/4291717063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231270 0x5626cb2041d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8016790 tx=0x7ff7f8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5eac45438934ac server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.844+0000 7ff7df7fe640 1 -- 172.21.15.81:0/4291717063 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8014ad0 con 0x5626cb231270 2023-11-08T16:40:04.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.844+0000 7ff7df7fe640 1 -- 172.21.15.81:0/4291717063 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f8014c30 con 0x5626cb231270 2023-11-08T16:40:04.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.844+0000 7ff7df7fe640 1 -- 172.21.15.81:0/4291717063 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8021400 con 0x5626cb231270 2023-11-08T16:40:04.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.845+0000 7ff8065fd740 1 -- 172.21.15.81:0/4291717063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231270 msgr2=0x5626cb2041d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.845+0000 7ff8065fd740 1 --2- 172.21.15.81:0/4291717063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231270 0x5626cb2041d0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8016790 tx=0x7ff7f8012000 comp rx=0 tx=0).stop 2023-11-08T16:40:04.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.845+0000 7ff8065fd740 1 -- 172.21.15.81:0/4291717063 shutdown_connections 2023-11-08T16:40:04.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.845+0000 7ff8065fd740 1 --2- 172.21.15.81:0/4291717063 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb204710 0x5626cb234540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.845+0000 7ff8065fd740 1 --2- 172.21.15.81:0/4291717063 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231650 0x5626cb234000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.845+0000 7ff8065fd740 1 --2- 172.21.15.81:0/4291717063 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231270 0x5626cb2041d0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.845+0000 7ff8065fd740 1 -- 172.21.15.81:0/4291717063 >> 172.21.15.81:0/4291717063 conn(0x5626cb206480 msgr2=0x5626cb24f000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.845+0000 7ff8065fd740 1 -- 172.21.15.81:0/4291717063 shutdown_connections 2023-11-08T16:40:04.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.845+0000 7ff8065fd740 1 -- 172.21.15.81:0/4291717063 wait complete. 2023-11-08T16:40:04.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.846+0000 7ff8065fd740 1 -- start start 2023-11-08T16:40:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.846+0000 7ff8065fd740 1 Processor -- start 2023-11-08T16:40:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.847+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb231270 0x5626cb24f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.847+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb24d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.847+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb236560 0x5626cb2369e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.847+0000 7ff8065fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cb10cdd0 con 0x5626cb231650 2023-11-08T16:40:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.847+0000 7ff8065fd740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cb10ca70 con 0x5626cb236560 2023-11-08T16:40:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.847+0000 7ff8065fd740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626cae68f90 con 0x5626cb231270 2023-11-08T16:40:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.847+0000 7ff8065fd740 10 client.0 _ll_register_callbacks cb 0x5626cb12ee80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:40:04.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.847+0000 7ff801006640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb24d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.847+0000 7ff801006640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb24d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:54996/0 (socket says 172.21.15.81:54996) 2023-11-08T16:40:04.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.847+0000 7ff801006640 1 -- 172.21.15.81:0/3965066939 learned_addr learned my addr 172.21.15.81:0/3965066939 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:04.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff801006640 1 -- 172.21.15.81:0/3965066939 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb231270 msgr2=0x5626cb24f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff801006640 1 --2- 172.21.15.81:0/3965066939 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb231270 0x5626cb24f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff801006640 1 -- 172.21.15.81:0/3965066939 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb236560 msgr2=0x5626cb2369e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff801006640 1 --2- 172.21.15.81:0/3965066939 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb236560 0x5626cb2369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff801006640 1 -- 172.21.15.81:0/3965066939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f80164a0 con 0x5626cb231650 2023-11-08T16:40:04.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff801006640 1 --2- 172.21.15.81:0/3965066939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb24d0c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec001930 tx=0x7ff7ec02cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8c3893777c9584 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff7dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:40:04.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff7dcff9640 1 -- 172.21.15.81:0/3965066939 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec079ae0 con 0x5626cb231650 2023-11-08T16:40:04.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff7dcff9640 1 -- 172.21.15.81:0/3965066939 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7ff7ec07d7d0 con 0x5626cb231650 2023-11-08T16:40:04.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff8065fd740 1 -- 172.21.15.81:0/3965066939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5626cb196400 con 0x5626cb231650 2023-11-08T16:40:04.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff7dcff9640 1 -- 172.21.15.81:0/3965066939 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7ec081980 con 0x5626cb231650 2023-11-08T16:40:04.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff8065fd740 10 client.8307 fetch_fsmap learned FSMap version 100 2023-11-08T16:40:04.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.848+0000 7ff8065fd740 1 -- 172.21.15.81:0/3965066939 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x5626cb10bd20 con 0x5626cb231650 2023-11-08T16:40:04.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.849+0000 7ff7dcff9640 1 -- 172.21.15.81:0/3965066939 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5626cb196400 con 0x5626cb231650 2023-11-08T16:40:04.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.849+0000 7ff7dcff9640 1 -- 172.21.15.81:0/3965066939 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 100) v1 ==== 3302+0+0 (secure 0 0 0) 0x7ff7ec048060 con 0x5626cb231650 2023-11-08T16:40:04.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.849+0000 7ff8065fd740 10 client.8307 fetch_fsmap finished waiting for FSMap version 100 2023-11-08T16:40:04.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.849+0000 7ff8065fd740 10 client.8307 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-08T16:40:04.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.849+0000 7ff8065fd740 20 client.8307 populate_metadata read hostname 'smithi081' 2023-11-08T16:40:04.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.849+0000 7ff8065fd740 1 --2- 172.21.15.81:0/3965066939 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5626cb262b60 0x5626cb262f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.849+0000 7ff8065fd740 4 client.8307 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:40:04.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.849+0000 7ff8065fd740 1 -- 172.21.15.81:0/3965066939 --> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5626cb168b20 con 0x5626cb262b60 2023-11-08T16:40:04.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.849+0000 7ff801807640 1 --2- 172.21.15.81:0/3965066939 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5626cb262b60 0x5626cb262f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.850+0000 7ff801807640 1 --2- 172.21.15.81:0/3965066939 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5626cb262b60 0x5626cb262f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89a596f5061f5a8e server_cookie=46a22fec69692eea in_seq=0 out_seq=0 2023-11-08T16:40:04.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.850+0000 7ff7dcff9640 10 client.8307 ms_handle_connect on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:04.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.850+0000 7ff7dcff9640 1 -- 172.21.15.81:0/3965066939 <== mds.0 v2:172.21.15.81:6836/3197735576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5626cb168b20 con 0x5626cb262b60 2023-11-08T16:40:04.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.850+0000 7ff7dcff9640 10 client.8307 handle_command_reply: tid=0 2023-11-08T16:40:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.850+0000 7ff8065fd740 1 client.8307 shutdown 2023-11-08T16:40:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff8065fd740 1 -- 172.21.15.81:0/3965066939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 msgr2=0x5626cb24d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff8065fd740 1 --2- 172.21.15.81:0/3965066939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb24d0c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec001930 tx=0x7ff7ec02cef0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff8065fd740 1 -- 172.21.15.81:0/3965066939 shutdown_connections 2023-11-08T16:40:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff8065fd740 1 --2- 172.21.15.81:0/3965066939 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb236560 0x5626cb2369e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff8065fd740 1 --2- 172.21.15.81:0/3965066939 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb24d0c0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff8065fd740 1 --2- 172.21.15.81:0/3965066939 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5626cb262b60 0x5626cb262f40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff8065fd740 1 --2- 172.21.15.81:0/3965066939 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb231270 0x5626cb24f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff7dcff9640 0 client.8307 ms_handle_reset on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:04.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff8065fd740 1 -- 172.21.15.81:0/3965066939 >> 172.21.15.81:0/3965066939 conn(0x5626cb204710 msgr2=0x5626cb208120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff8065fd740 1 -- 172.21.15.81:0/3965066939 shutdown_connections 2023-11-08T16:40:04.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff8065fd740 1 -- 172.21.15.81:0/3965066939 wait complete. 2023-11-08T16:40:04.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.851+0000 7ff8065fd740 20 client.8307 trim_cache size 0 max 16384 2023-11-08T16:40:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.867+0000 7ff8065fd740 1 Processor -- start 2023-11-08T16:40:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.867+0000 7ff8065fd740 1 -- start start 2023-11-08T16:40:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.868+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231270 0x5626cb225430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.868+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb2489b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.868+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb236560 0x5626cb234100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.868+0000 7ff8065fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cb20b840 con 0x5626cb231650 2023-11-08T16:40:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.868+0000 7ff8065fd740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cb2040d0 con 0x5626cb231270 2023-11-08T16:40:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.868+0000 7ff8065fd740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626cb20b230 con 0x5626cb236560 2023-11-08T16:40:04.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.869+0000 7ff801006640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb2489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.869+0000 7ff801006640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb2489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55006/0 (socket says 172.21.15.81:55006) 2023-11-08T16:40:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.869+0000 7ff801006640 1 -- 172.21.15.81:0/3556899976 learned_addr learned my addr 172.21.15.81:0/3556899976 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.869+0000 7ff801006640 1 -- 172.21.15.81:0/3556899976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb236560 msgr2=0x5626cb234100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.869+0000 7ff801006640 1 --2- 172.21.15.81:0/3556899976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb236560 0x5626cb234100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.869+0000 7ff801006640 1 -- 172.21.15.81:0/3556899976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231270 msgr2=0x5626cb225430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.869+0000 7ff801006640 1 --2- 172.21.15.81:0/3556899976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231270 0x5626cb225430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.869+0000 7ff801006640 1 -- 172.21.15.81:0/3556899976 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5626cb19f750 con 0x5626cb231650 2023-11-08T16:40:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.869+0000 7ff801006640 1 --2- 172.21.15.81:0/3556899976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb2489b0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec081450 tx=0x7ff7ec043070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3f85ba8bcd72d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.869+0000 7ff7dffff640 1 -- 172.21.15.81:0/3556899976 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec081980 con 0x5626cb231650 2023-11-08T16:40:04.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.870+0000 7ff7dffff640 1 -- 172.21.15.81:0/3556899976 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec078dc0 con 0x5626cb231650 2023-11-08T16:40:04.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.870+0000 7ff7dffff640 1 -- 172.21.15.81:0/3556899976 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec079b20 con 0x5626cb231650 2023-11-08T16:40:04.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.870+0000 7ff8065fd740 1 -- 172.21.15.81:0/3556899976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 msgr2=0x5626cb2489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.870+0000 7ff8065fd740 1 --2- 172.21.15.81:0/3556899976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb2489b0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec081450 tx=0x7ff7ec043070 comp rx=0 tx=0).stop 2023-11-08T16:40:04.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.870+0000 7ff8065fd740 1 -- 172.21.15.81:0/3556899976 shutdown_connections 2023-11-08T16:40:04.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.870+0000 7ff8065fd740 1 --2- 172.21.15.81:0/3556899976 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb236560 0x5626cb234100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.870+0000 7ff8065fd740 1 --2- 172.21.15.81:0/3556899976 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb231650 0x5626cb2489b0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.870+0000 7ff8065fd740 1 --2- 172.21.15.81:0/3556899976 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231270 0x5626cb225430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.870+0000 7ff8065fd740 1 -- 172.21.15.81:0/3556899976 >> 172.21.15.81:0/3556899976 conn(0x5626cb204710 msgr2=0x5626cb230140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.870+0000 7ff8065fd740 1 -- 172.21.15.81:0/3556899976 shutdown_connections 2023-11-08T16:40:04.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.870+0000 7ff8065fd740 1 -- 172.21.15.81:0/3556899976 wait complete. 2023-11-08T16:40:04.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.871+0000 7ff8065fd740 1 -- start start 2023-11-08T16:40:04.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.871+0000 7ff8065fd740 1 Processor -- start 2023-11-08T16:40:04.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.872+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb231270 0x5626cb252c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.872+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231650 0x5626cb24c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.872+0000 7ff8065fd740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb236560 0x5626cb24cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.872+0000 7ff8065fd740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cb272410 con 0x5626cb236560 2023-11-08T16:40:04.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.872+0000 7ff8065fd740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5626cb20a7f0 con 0x5626cb231650 2023-11-08T16:40:04.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.872+0000 7ff8065fd740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5626cb2225a0 con 0x5626cb231270 2023-11-08T16:40:04.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.872+0000 7ff8065fd740 10 client.0 _ll_register_callbacks cb 0x7ff7ec001760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:40:04.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff802008640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb236560 0x5626cb24cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff802008640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb236560 0x5626cb24cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:55012/0 (socket says 172.21.15.81:55012) 2023-11-08T16:40:04.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff802008640 1 -- 172.21.15.81:0/625563714 learned_addr learned my addr 172.21.15.81:0/625563714 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:04.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff801807640 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb231270 0x5626cb252c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff801006640 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231650 0x5626cb24c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff802008640 1 -- 172.21.15.81:0/625563714 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb231270 msgr2=0x5626cb252c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff802008640 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb231270 0x5626cb252c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff802008640 1 -- 172.21.15.81:0/625563714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231650 msgr2=0x5626cb24c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:04.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff802008640 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231650 0x5626cb24c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff802008640 1 -- 172.21.15.81:0/625563714 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f80164a0 con 0x5626cb236560 2023-11-08T16:40:04.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff802008640 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb236560 0x5626cb24cdb0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc165000 tx=0x7ff7f4011630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bf5ebe78ce8c1a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:04.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.873+0000 7ff7dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:40:04.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.874+0000 7ff7dcff9640 1 -- 172.21.15.81:0/625563714 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f4004030 con 0x5626cb236560 2023-11-08T16:40:04.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.874+0000 7ff7dcff9640 1 -- 172.21.15.81:0/625563714 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7ff7f4004190 con 0x5626cb236560 2023-11-08T16:40:04.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.874+0000 7ff8065fd740 1 -- 172.21.15.81:0/625563714 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7f8014c30 con 0x5626cb236560 2023-11-08T16:40:04.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.874+0000 7ff7dcff9640 1 -- 172.21.15.81:0/625563714 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7f4004350 con 0x5626cb236560 2023-11-08T16:40:04.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.874+0000 7ff8065fd740 10 client.8313 fetch_fsmap learned FSMap version 100 2023-11-08T16:40:04.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.874+0000 7ff8065fd740 1 -- 172.21.15.81:0/625563714 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x5626cb131310 con 0x5626cb236560 2023-11-08T16:40:04.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.874+0000 7ff7dcff9640 1 -- 172.21.15.81:0/625563714 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f8014c30 con 0x5626cb236560 2023-11-08T16:40:04.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.874+0000 7ff7dcff9640 1 -- 172.21.15.81:0/625563714 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 100) v1 ==== 3302+0+0 (secure 0 0 0) 0x7ff7f401e430 con 0x5626cb236560 2023-11-08T16:40:04.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.875+0000 7ff8065fd740 10 client.8313 fetch_fsmap finished waiting for FSMap version 100 2023-11-08T16:40:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.875+0000 7ff8065fd740 10 client.8313 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-08T16:40:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.875+0000 7ff8065fd740 20 client.8313 populate_metadata read hostname 'smithi081' 2023-11-08T16:40:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.875+0000 7ff8065fd740 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5626cb2094b0 0x5626cb2098c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.875+0000 7ff8065fd740 4 client.8313 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-08T16:40:04.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.875+0000 7ff8065fd740 1 -- 172.21.15.81:0/625563714 --> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5626cb168840 con 0x5626cb2094b0 2023-11-08T16:40:04.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.875+0000 7ff801807640 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5626cb2094b0 0x5626cb2098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:04.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.875+0000 7ff7dcff9640 10 client.8313 ms_handle_connect on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:04.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.875+0000 7ff801807640 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5626cb2094b0 0x5626cb2098c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2712475fa60b6ee server_cookie=8b5e5ff9946850 in_seq=0 out_seq=0 2023-11-08T16:40:04.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff7dcff9640 1 -- 172.21.15.81:0/625563714 <== mds.0 v2:172.21.15.81:6836/3197735576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x5626cb168840 con 0x5626cb2094b0 2023-11-08T16:40:04.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff7dcff9640 10 client.8313 handle_command_reply: tid=0 2023-11-08T16:40:04.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 1 client.8313 shutdown 2023-11-08T16:40:04.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 1 -- 172.21.15.81:0/625563714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb236560 msgr2=0x5626cb24cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb236560 0x5626cb24cdb0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc165000 tx=0x7ff7f4011630 comp rx=0 tx=0).stop 2023-11-08T16:40:04.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 1 -- 172.21.15.81:0/625563714 shutdown_connections 2023-11-08T16:40:04.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5626cb236560 0x5626cb24cdb0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5626cb231650 0x5626cb24c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x5626cb2094b0 0x5626cb2098c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 1 --2- 172.21.15.81:0/625563714 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5626cb231270 0x5626cb252c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff7dcff9640 0 client.8313 ms_handle_reset on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 1 -- 172.21.15.81:0/625563714 >> 172.21.15.81:0/625563714 conn(0x5626cb204710 msgr2=0x5626cb230140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 1 -- 172.21.15.81:0/625563714 shutdown_connections 2023-11-08T16:40:04.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 1 -- 172.21.15.81:0/625563714 wait complete. 2023-11-08T16:40:04.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.876+0000 7ff8065fd740 20 client.8313 trim_cache size 0 max 16384 2023-11-08T16:40:04.891 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:40:04.891 INFO:teuthology.orchestra.run.smithi081.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-08T16:40:04.891 INFO:teuthology.orchestra.run.smithi081.stdout: "scrubs": { 2023-11-08T16:40:04.892 INFO:teuthology.orchestra.run.smithi081.stdout: "a6e3f8ec-acc8-4573-ae28-3fef5c33d28e": { 2023-11-08T16:40:04.892 INFO:teuthology.orchestra.run.smithi081.stdout: "path": "/", 2023-11-08T16:40:04.892 INFO:teuthology.orchestra.run.smithi081.stdout: "tag": "a6e3f8ec-acc8-4573-ae28-3fef5c33d28e", 2023-11-08T16:40:04.892 INFO:teuthology.orchestra.run.smithi081.stdout: "options": "recursive,repair" 2023-11-08T16:40:04.892 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:40:04.893 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:40:04.893 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:40:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 -- 172.21.15.81:0/849090250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7e006aec0 msgr2=0x7ff7e006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7e006aec0 0x7ff7e006d380 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8007520 tx=0x7ff7f800b040 comp rx=0 tx=0).stop 2023-11-08T16:40:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 -- 172.21.15.81:0/849090250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc1f7210 msgr2=0x7ff7fc163ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:04.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc1f7210 0x7ff7fc163ae0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec001eb0 tx=0x7ff7fc166120 comp rx=0 tx=0).stop 2023-11-08T16:40:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 -- 172.21.15.81:0/849090250 shutdown_connections 2023-11-08T16:40:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff7e006aec0 0x7ff7e006d380 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff7fc164020 0x7ff7fc0c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff7fc1f7210 0x7ff7fc163ae0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 --2- 172.21.15.81:0/849090250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7e80047c0 0x7ff7fc153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:04.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 -- 172.21.15.81:0/849090250 >> 172.21.15.81:0/849090250 conn(0x7ff7fc0cb4e0 msgr2=0x7ff7fc0cc5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:04.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 -- 172.21.15.81:0/849090250 shutdown_connections 2023-11-08T16:40:04.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:04.877+0000 7ff803a6b640 1 -- 172.21.15.81:0/849090250 wait complete. 2023-11-08T16:40:04.897 INFO:tasks.cephfs.filesystem:scrub status for tag:a6e3f8ec-acc8-4573-ae28-3fef5c33d28e - {'path': '/', 'tag': 'a6e3f8ec-acc8-4573-ae28-3fef5c33d28e', 'options': 'recursive,repair'} 2023-11-08T16:40:19.176 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:40:19.186 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:40:34.901 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 scrub status 2023-11-08T16:40:35.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.117+0000 7fd6f6875640 1 Processor -- start 2023-11-08T16:40:35.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.117+0000 7fd6f6875640 1 -- start start 2023-11-08T16:40:35.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.118+0000 7fd6f6875640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd6f00714b0 0x7fd6f015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.119+0000 7fd6f6875640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f00b09f0 con 0x7fd6f0071890 2023-11-08T16:40:35.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.119+0000 7fd6f6875640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f00c9ed0 con 0x7fd6f00adf00 2023-11-08T16:40:35.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.119+0000 7fd6f6875640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f006b250 con 0x7fd6f00714b0 2023-11-08T16:40:35.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.119+0000 7fd6ef7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd6f0071890 0x7fd6f00c90b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54718/0 (socket says 172.21.15.81:54718) 2023-11-08T16:40:35.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.119+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/3296163967 learned_addr learned my addr 172.21.15.81:0/3296163967 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:35.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.119+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104163355 0 0) 0x7fd6f00b09f0 con 0x7fd6f0071890 2023-11-08T16:40:35.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.119+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d8003520 con 0x7fd6f0071890 2023-11-08T16:40:35.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6effff640 1 --2- 172.21.15.81:0/3296163967 >> v2:172.21.15.93:3300/0 conn(0x7fd6f00714b0 0x7fd6f015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 881139711 0 0) 0x7fd6d8003520 con 0x7fd6f0071890 2023-11-08T16:40:35.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 >> v2:172.21.15.93:3300/0 conn(0x7fd6f00714b0 msgr2=0x7fd6f015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 --2- 172.21.15.81:0/3296163967 >> v2:172.21.15.93:3300/0 conn(0x7fd6f00714b0 0x7fd6f015a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 >> v1:172.21.15.93:6789/0 conn(0x7fd6f00adf00 legacy=0x7fd6f00c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f00ca360 con 0x7fd6f0071890 2023-11-08T16:40:35.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fd6e4001670 con 0x7fd6f0071890 2023-11-08T16:40:35.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 >> v1:172.21.15.81:6789/0 conn(0x7fd6f0071890 legacy=0x7fd6f00c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:35.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 --2- 172.21.15.81:0/3296163967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6d80039e0 0x7fd6d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 --2- 172.21.15.81:0/3296163967 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6d80047c0 0x7fd6d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 --2- 172.21.15.81:0/3296163967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80051d0 0x7fd6d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d800c0f0 con 0x7fd6d80051d0 2023-11-08T16:40:35.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d800c260 con 0x7fd6d80039e0 2023-11-08T16:40:35.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d800c420 con 0x7fd6d80047c0 2023-11-08T16:40:35.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6f4e12640 1 --2- 172.21.15.81:0/3296163967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80051d0 0x7fd6d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.120+0000 7fd6effff640 1 --2- 172.21.15.81:0/3296163967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6d80039e0 0x7fd6d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.121+0000 7fd6ef7fe640 1 --2- 172.21.15.81:0/3296163967 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6d80047c0 0x7fd6d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.121+0000 7fd6f4e12640 1 -- 172.21.15.81:0/3296163967 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6d80047c0 msgr2=0x7fd6d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.121+0000 7fd6f4e12640 1 --2- 172.21.15.81:0/3296163967 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6d80047c0 0x7fd6d8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.121+0000 7fd6f4e12640 1 -- 172.21.15.81:0/3296163967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6d80039e0 msgr2=0x7fd6d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.121+0000 7fd6f4e12640 1 --2- 172.21.15.81:0/3296163967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6d80039e0 0x7fd6d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.121+0000 7fd6f4e12640 1 -- 172.21.15.81:0/3296163967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d8003520 con 0x7fd6d80051d0 2023-11-08T16:40:35.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.122+0000 7fd6f4e12640 1 -- 172.21.15.81:0/3296163967 reap_dead start 2023-11-08T16:40:35.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.122+0000 7fd6f4e12640 1 --2- 172.21.15.81:0/3296163967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80051d0 0x7fd6d800b980 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80044b0 tx=0x7fd6d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f3e213ad30a697 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:35.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.122+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8030620 con 0x7fd6d80051d0 2023-11-08T16:40:35.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.122+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e8030780 con 0x7fd6d80051d0 2023-11-08T16:40:35.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.122+0000 7fd6eeffd640 1 -- 172.21.15.81:0/3296163967 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8030b10 con 0x7fd6d80051d0 2023-11-08T16:40:35.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.123+0000 7fd6f6875640 1 -- 172.21.15.81:0/3296163967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80051d0 msgr2=0x7fd6d800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:35.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.123+0000 7fd6f6875640 1 --2- 172.21.15.81:0/3296163967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80051d0 0x7fd6d800b980 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd6e80044b0 tx=0x7fd6d80057f0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.123+0000 7fd6f6875640 1 -- 172.21.15.81:0/3296163967 shutdown_connections 2023-11-08T16:40:35.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.123+0000 7fd6f6875640 1 --2- 172.21.15.81:0/3296163967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6d80051d0 0x7fd6d800b980 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.123+0000 7fd6f6875640 1 -- 172.21.15.81:0/3296163967 >> 172.21.15.81:0/3296163967 conn(0x7fd6f00cb4e0 msgr2=0x7fd6f00cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:35.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.123+0000 7fd6f6875640 1 -- 172.21.15.81:0/3296163967 shutdown_connections 2023-11-08T16:40:35.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.123+0000 7fd6f6875640 1 -- 172.21.15.81:0/3296163967 wait complete. 2023-11-08T16:40:35.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.125+0000 7fd6f6875640 1 Processor -- start 2023-11-08T16:40:35.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.125+0000 7fd6f6875640 1 -- start start 2023-11-08T16:40:35.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6f6875640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6d80051d0 0x7fd6f014cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6f6875640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f00ac8b0 0x7fd6f00acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6f6875640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6f015b750 0x7fd6f015bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6f6875640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00ad270 con 0x7fd6f00ac8b0 2023-11-08T16:40:35.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6f6875640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f015c110 con 0x7fd6f015b750 2023-11-08T16:40:35.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6f6875640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f015c3c0 con 0x7fd6d80051d0 2023-11-08T16:40:35.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f00ac8b0 0x7fd6f00acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f00ac8b0 0x7fd6f00acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44712/0 (socket says 172.21.15.81:44712) 2023-11-08T16:40:35.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/1581923752 learned_addr learned my addr 172.21.15.81:0/1581923752 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:35.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6effff640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6d80051d0 0x7fd6f014cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/1581923752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6d80051d0 msgr2=0x7fd6f014cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:35.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6ef7fe640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6d80051d0 0x7fd6f014cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/1581923752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6f015b750 msgr2=0x7fd6f015bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:35.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6ef7fe640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6f015b750 0x7fd6f015bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.126+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/1581923752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f015c6a0 con 0x7fd6f00ac8b0 2023-11-08T16:40:35.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.127+0000 7fd6ef7fe640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f00ac8b0 0x7fd6f00acd30 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400d7e0 tx=0x7fd6e400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:35.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.127+0000 7fd6ecff9640 1 -- 172.21.15.81:0/1581923752 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e40074e0 con 0x7fd6f00ac8b0 2023-11-08T16:40:35.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.127+0000 7fd6ecff9640 1 -- 172.21.15.81:0/1581923752 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e4007640 con 0x7fd6f00ac8b0 2023-11-08T16:40:35.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.127+0000 7fd6ecff9640 1 -- 172.21.15.81:0/1581923752 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4004a20 con 0x7fd6f00ac8b0 2023-11-08T16:40:35.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.127+0000 7fd6f6875640 1 -- 172.21.15.81:0/1581923752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f015c980 con 0x7fd6f00ac8b0 2023-11-08T16:40:35.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.127+0000 7fd6f6875640 1 -- 172.21.15.81:0/1581923752 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f01543b0 con 0x7fd6f00ac8b0 2023-11-08T16:40:35.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.129+0000 7fd6ecff9640 1 -- 172.21.15.81:0/1581923752 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6e4004b80 con 0x7fd6f00ac8b0 2023-11-08T16:40:35.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.130+0000 7fd6ecff9640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae70 0x7fd6dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.130+0000 7fd6ecff9640 1 -- 172.21.15.81:0/1581923752 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fd6e4089320 con 0x7fd6f00ac8b0 2023-11-08T16:40:35.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.130+0000 7fd6effff640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae70 0x7fd6dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.131+0000 7fd6f9409740 1 Processor -- start 2023-11-08T16:40:35.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.131+0000 7fd6effff640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae70 0x7fd6dc06d330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0006fd0 tx=0x7fd6e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:35.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.131+0000 7fd6f9409740 1 -- start start 2023-11-08T16:40:35.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.132+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400270 0x55d02a3d31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.132+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400650 0x55d02a403000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.132+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a3d3710 0x55d02a403540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.132+0000 7fd6f9409740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02a345b60 con 0x55d02a400270 2023-11-08T16:40:35.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.132+0000 7fd6f9409740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02a343250 con 0x55d02a3d3710 2023-11-08T16:40:35.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.132+0000 7fd6f9409740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d02a33eb40 con 0x55d02a400650 2023-11-08T16:40:35.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.132+0000 7fd6effff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400270 0x55d02a3d31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.132+0000 7fd6effff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400270 0x55d02a3d31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44714/0 (socket says 172.21.15.81:44714) 2023-11-08T16:40:35.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.132+0000 7fd6effff640 1 -- 172.21.15.81:0/3061523141 learned_addr learned my addr 172.21.15.81:0/3061523141 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:35.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.133+0000 7fd6effff640 1 -- 172.21.15.81:0/3061523141 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400650 msgr2=0x55d02a403000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.133+0000 7fd6effff640 1 --2- 172.21.15.81:0/3061523141 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400650 0x55d02a403000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.133+0000 7fd6f4e12640 1 --2- 172.21.15.81:0/3061523141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a3d3710 0x55d02a403540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.133+0000 7fd6effff640 1 -- 172.21.15.81:0/3061523141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a3d3710 msgr2=0x55d02a403540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.133+0000 7fd6effff640 1 --2- 172.21.15.81:0/3061523141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a3d3710 0x55d02a403540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.133+0000 7fd6effff640 1 -- 172.21.15.81:0/3061523141 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d02a33d1c0 con 0x55d02a400270 2023-11-08T16:40:35.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.133+0000 7fd6effff640 1 --2- 172.21.15.81:0/3061523141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400270 0x55d02a3d31d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0010920 tx=0x7fd6e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b86c20f4a431967 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:35.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.133+0000 7fd6d5ffb640 1 -- 172.21.15.81:0/3061523141 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0014da0 con 0x55d02a400270 2023-11-08T16:40:35.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.133+0000 7fd6d5ffb640 1 -- 172.21.15.81:0/3061523141 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e001a070 con 0x55d02a400270 2023-11-08T16:40:35.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.133+0000 7fd6d5ffb640 1 -- 172.21.15.81:0/3061523141 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0021640 con 0x55d02a400270 2023-11-08T16:40:35.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.134+0000 7fd6f9409740 1 -- 172.21.15.81:0/3061523141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400270 msgr2=0x55d02a3d31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:35.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.134+0000 7fd6f9409740 1 --2- 172.21.15.81:0/3061523141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400270 0x55d02a3d31d0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0010920 tx=0x7fd6e0013000 comp rx=0 tx=0).stop 2023-11-08T16:40:35.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.134+0000 7fd6f9409740 1 -- 172.21.15.81:0/3061523141 shutdown_connections 2023-11-08T16:40:35.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.134+0000 7fd6f9409740 1 --2- 172.21.15.81:0/3061523141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a3d3710 0x55d02a403540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.134+0000 7fd6f9409740 1 --2- 172.21.15.81:0/3061523141 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400650 0x55d02a403000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.134+0000 7fd6f9409740 1 --2- 172.21.15.81:0/3061523141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400270 0x55d02a3d31d0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.134+0000 7fd6f9409740 1 -- 172.21.15.81:0/3061523141 >> 172.21.15.81:0/3061523141 conn(0x55d02a3d5480 msgr2=0x55d02a41e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:35.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.134+0000 7fd6f9409740 1 -- 172.21.15.81:0/3061523141 shutdown_connections 2023-11-08T16:40:35.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.134+0000 7fd6f9409740 1 -- 172.21.15.81:0/3061523141 wait complete. 2023-11-08T16:40:35.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.135+0000 7fd6f9409740 1 -- start start 2023-11-08T16:40:35.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.135+0000 7fd6f9409740 1 Processor -- start 2023-11-08T16:40:35.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.136+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400270 0x55d02a3f4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.136+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a400650 0x55d02a415a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.136+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a415fb0 0x55d02a4180b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.136+0000 7fd6f9409740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02a2dbdd0 con 0x55d02a415fb0 2023-11-08T16:40:35.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.136+0000 7fd6f9409740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02a2dba70 con 0x55d02a400650 2023-11-08T16:40:35.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.136+0000 7fd6f9409740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d02a037f90 con 0x55d02a400270 2023-11-08T16:40:35.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.136+0000 7fd6f9409740 10 client.0 _ll_register_callbacks cb 0x55d02a2fde80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:40:35.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.136+0000 7fd6f4e12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a415fb0 0x55d02a4180b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.136+0000 7fd6f4e12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a415fb0 0x55d02a4180b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44726/0 (socket says 172.21.15.81:44726) 2023-11-08T16:40:35.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.136+0000 7fd6f4e12640 1 -- 172.21.15.81:0/109943858 learned_addr learned my addr 172.21.15.81:0/109943858 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:35.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.137+0000 7fd6effff640 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400270 0x55d02a3f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.137+0000 7fd6ef7fe640 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a400650 0x55d02a415a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.137+0000 7fd6f4e12640 1 -- 172.21.15.81:0/109943858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400270 msgr2=0x55d02a3f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.137+0000 7fd6f4e12640 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400270 0x55d02a3f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.137+0000 7fd6f4e12640 1 -- 172.21.15.81:0/109943858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a400650 msgr2=0x55d02a415a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.137+0000 7fd6f4e12640 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a400650 0x55d02a415a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.137+0000 7fd6f4e12640 1 -- 172.21.15.81:0/109943858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e000e030 con 0x55d02a415fb0 2023-11-08T16:40:35.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.137+0000 7fd6f4e12640 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a415fb0 0x55d02a4180b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd6e802bf80 tx=0x7fd6e8003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5cc2cf6082e963 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:35.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.137+0000 7fd6ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:40:35.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6ceffd640 1 -- 172.21.15.81:0/109943858 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e803e040 con 0x55d02a415fb0 2023-11-08T16:40:35.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6f9409740 1 -- 172.21.15.81:0/109943858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d02a365400 con 0x55d02a415fb0 2023-11-08T16:40:35.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6ceffd640 1 -- 172.21.15.81:0/109943858 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fd6e8006d50 con 0x55d02a415fb0 2023-11-08T16:40:35.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6ceffd640 1 -- 172.21.15.81:0/109943858 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6e8039450 con 0x55d02a415fb0 2023-11-08T16:40:35.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6f9409740 10 client.8325 fetch_fsmap learned FSMap version 100 2023-11-08T16:40:35.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6f9409740 1 -- 172.21.15.81:0/109943858 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x55d02a2dad20 con 0x55d02a415fb0 2023-11-08T16:40:35.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6ceffd640 1 -- 172.21.15.81:0/109943858 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d02a365400 con 0x55d02a415fb0 2023-11-08T16:40:35.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6ceffd640 1 -- 172.21.15.81:0/109943858 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 100) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fd6e8041030 con 0x55d02a415fb0 2023-11-08T16:40:35.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6f9409740 10 client.8325 fetch_fsmap finished waiting for FSMap version 100 2023-11-08T16:40:35.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6f9409740 10 client.8325 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-08T16:40:35.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6f9409740 20 client.8325 populate_metadata read hostname 'smithi081' 2023-11-08T16:40:35.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6f9409740 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55d02a42e290 0x55d02a42e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6f9409740 4 client.8325 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:40:35.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.138+0000 7fd6f9409740 1 -- 172.21.15.81:0/109943858 --> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d02a337b20 con 0x55d02a42e290 2023-11-08T16:40:35.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.139+0000 7fd6effff640 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55d02a42e290 0x55d02a42e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.139+0000 7fd6ceffd640 10 client.8325 ms_handle_connect on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:35.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.139+0000 7fd6effff640 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55d02a42e290 0x55d02a42e6a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0a6a092f458b7f2 server_cookie=77e6cd0143a0ca2b in_seq=0 out_seq=0 2023-11-08T16:40:35.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6ceffd640 1 -- 172.21.15.81:0/109943858 <== mds.0 v2:172.21.15.81:6836/3197735576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d02a337b20 con 0x55d02a42e290 2023-11-08T16:40:35.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6ceffd640 10 client.8325 handle_command_reply: tid=0 2023-11-08T16:40:35.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6f9409740 1 client.8325 shutdown 2023-11-08T16:40:35.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6f9409740 1 -- 172.21.15.81:0/109943858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a415fb0 msgr2=0x55d02a4180b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:35.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6f9409740 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a415fb0 0x55d02a4180b0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fd6e802bf80 tx=0x7fd6e8003ae0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6f9409740 1 -- 172.21.15.81:0/109943858 shutdown_connections 2023-11-08T16:40:35.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6f9409740 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a415fb0 0x55d02a4180b0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6f9409740 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a400650 0x55d02a415a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6f9409740 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55d02a42e290 0x55d02a42e6a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6f9409740 1 --2- 172.21.15.81:0/109943858 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400270 0x55d02a3f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6f9409740 1 -- 172.21.15.81:0/109943858 >> 172.21.15.81:0/109943858 conn(0x55d02a3d3710 msgr2=0x55d02a3d7120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:35.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6ceffd640 0 client.8325 ms_handle_reset on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:35.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6f9409740 1 -- 172.21.15.81:0/109943858 shutdown_connections 2023-11-08T16:40:35.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.140+0000 7fd6f9409740 1 -- 172.21.15.81:0/109943858 wait complete. 2023-11-08T16:40:35.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.141+0000 7fd6f9409740 20 client.8325 trim_cache size 0 max 16384 2023-11-08T16:40:35.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.157+0000 7fd6f9409740 1 Processor -- start 2023-11-08T16:40:35.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.157+0000 7fd6f9409740 1 -- start start 2023-11-08T16:40:35.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.157+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a400270 0x55d02a440bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.157+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a421510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.157+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a415fb0 0x55d02a421a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.157+0000 7fd6f9409740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02a440a50 con 0x55d02a400650 2023-11-08T16:40:35.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.157+0000 7fd6f9409740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02a419a60 con 0x55d02a400270 2023-11-08T16:40:35.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.157+0000 7fd6f9409740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d02a4060a0 con 0x55d02a415fb0 2023-11-08T16:40:35.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.158+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a421510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.158+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a421510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44728/0 (socket says 172.21.15.81:44728) 2023-11-08T16:40:35.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.158+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/1008089755 learned_addr learned my addr 172.21.15.81:0/1008089755 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:35.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.158+0000 7fd6effff640 1 --2- 172.21.15.81:0/1008089755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a400270 0x55d02a440bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.158+0000 7fd6f4e12640 1 --2- 172.21.15.81:0/1008089755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a415fb0 0x55d02a421a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.158+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/1008089755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a415fb0 msgr2=0x55d02a421a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.158+0000 7fd6ef7fe640 1 --2- 172.21.15.81:0/1008089755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a415fb0 0x55d02a421a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.158+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/1008089755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a400270 msgr2=0x55d02a440bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.158+0000 7fd6ef7fe640 1 --2- 172.21.15.81:0/1008089755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a400270 0x55d02a440bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.158+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/1008089755 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d02a36e750 con 0x55d02a400650 2023-11-08T16:40:35.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.158+0000 7fd6effff640 1 --2- 172.21.15.81:0/1008089755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a400270 0x55d02a440bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:35.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.159+0000 7fd6ef7fe640 1 --2- 172.21.15.81:0/1008089755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a421510 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400b4a0 tx=0x7fd6e4057920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42805104628dc1f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:35.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.159+0000 7fd6d67fc640 1 -- 172.21.15.81:0/1008089755 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e406f070 con 0x55d02a400650 2023-11-08T16:40:35.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.159+0000 7fd6d67fc640 1 -- 172.21.15.81:0/1008089755 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e4057ce0 con 0x55d02a400650 2023-11-08T16:40:35.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.159+0000 7fd6d67fc640 1 -- 172.21.15.81:0/1008089755 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e405be30 con 0x55d02a400650 2023-11-08T16:40:35.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.159+0000 7fd6f9409740 1 -- 172.21.15.81:0/1008089755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 msgr2=0x55d02a421510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:35.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.159+0000 7fd6f9409740 1 --2- 172.21.15.81:0/1008089755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a421510 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400b4a0 tx=0x7fd6e4057920 comp rx=0 tx=0).stop 2023-11-08T16:40:35.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.160+0000 7fd6f9409740 1 -- 172.21.15.81:0/1008089755 shutdown_connections 2023-11-08T16:40:35.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.160+0000 7fd6f9409740 1 --2- 172.21.15.81:0/1008089755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a415fb0 0x55d02a421a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.160+0000 7fd6f9409740 1 --2- 172.21.15.81:0/1008089755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a421510 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.160+0000 7fd6f9409740 1 --2- 172.21.15.81:0/1008089755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a400270 0x55d02a440bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.160+0000 7fd6f9409740 1 -- 172.21.15.81:0/1008089755 >> 172.21.15.81:0/1008089755 conn(0x55d02a3d3710 msgr2=0x55d02a3ce990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:35.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.160+0000 7fd6f9409740 1 -- 172.21.15.81:0/1008089755 shutdown_connections 2023-11-08T16:40:35.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.160+0000 7fd6f9409740 1 -- 172.21.15.81:0/1008089755 wait complete. 2023-11-08T16:40:35.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.160+0000 7fd6f9409740 1 -- start start 2023-11-08T16:40:35.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.160+0000 7fd6f9409740 1 Processor -- start 2023-11-08T16:40:35.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.161+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400270 0x55d02a488870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.161+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a3fe510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.161+0000 7fd6f9409740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a415fb0 0x55d02a3fea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.161+0000 7fd6f9409740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02a3ff1f0 con 0x55d02a400650 2023-11-08T16:40:35.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.161+0000 7fd6f9409740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d02a48cad0 con 0x55d02a415fb0 2023-11-08T16:40:35.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.161+0000 7fd6f9409740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d02a41e9a0 con 0x55d02a400270 2023-11-08T16:40:35.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6f9409740 10 client.0 _ll_register_callbacks cb 0x7fd6d80046a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:40:35.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a3fe510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6ef7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a3fe510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:44730/0 (socket says 172.21.15.81:44730) 2023-11-08T16:40:35.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/3424495925 learned_addr learned my addr 172.21.15.81:0/3424495925 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:35.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/3424495925 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400270 msgr2=0x55d02a488870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6ef7fe640 1 --2- 172.21.15.81:0/3424495925 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400270 0x55d02a488870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/3424495925 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a415fb0 msgr2=0x55d02a3fea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:35.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6ef7fe640 1 --2- 172.21.15.81:0/3424495925 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a415fb0 0x55d02a3fea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6ef7fe640 1 -- 172.21.15.81:0/3424495925 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f01543b0 con 0x55d02a400650 2023-11-08T16:40:35.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6ef7fe640 1 --2- 172.21.15.81:0/3424495925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a3fe510 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400d7b0 tx=0x7fd6e4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29accf24f9f47dbe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:35.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:40:35.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.162+0000 7fd6ceffd640 1 -- 172.21.15.81:0/3424495925 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e406f040 con 0x55d02a400650 2023-11-08T16:40:35.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6ceffd640 1 -- 172.21.15.81:0/3424495925 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fd6e405ffe0 con 0x55d02a400650 2023-11-08T16:40:35.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6f9409740 1 -- 172.21.15.81:0/3424495925 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd6e001a070 con 0x55d02a400650 2023-11-08T16:40:35.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6ceffd640 1 -- 172.21.15.81:0/3424495925 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=100) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd6e405be30 con 0x55d02a400650 2023-11-08T16:40:35.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6f9409740 10 client.8331 fetch_fsmap learned FSMap version 100 2023-11-08T16:40:35.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6f9409740 1 -- 172.21.15.81:0/3424495925 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=100,monmap=2+}) v3 -- 0x55d02a300310 con 0x55d02a400650 2023-11-08T16:40:35.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6ceffd640 1 -- 172.21.15.81:0/3424495925 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e001a070 con 0x55d02a400650 2023-11-08T16:40:35.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6ceffd640 1 -- 172.21.15.81:0/3424495925 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 100) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fd6e405fb50 con 0x55d02a400650 2023-11-08T16:40:35.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6f9409740 10 client.8331 fetch_fsmap finished waiting for FSMap version 100 2023-11-08T16:40:35.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6f9409740 10 client.8331 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.c 2023-11-08T16:40:35.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6f9409740 20 client.8331 populate_metadata read hostname 'smithi081' 2023-11-08T16:40:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6f9409740 1 --2- 172.21.15.81:0/3424495925 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55d02a42e7a0 0x55d02a420570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6f9409740 4 client.8331 mds_command: new command op to 8190 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2023-11-08T16:40:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6f9409740 1 -- 172.21.15.81:0/3424495925 --> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d02a337840 con 0x55d02a42e7a0 2023-11-08T16:40:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.163+0000 7fd6effff640 1 --2- 172.21.15.81:0/3424495925 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55d02a42e7a0 0x55d02a420570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:35.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6effff640 1 --2- 172.21.15.81:0/3424495925 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55d02a42e7a0 0x55d02a420570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef01d53706a68e32 server_cookie=9cd7a0052d397e84 in_seq=0 out_seq=0 2023-11-08T16:40:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6ceffd640 10 client.8331 ms_handle_connect on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6ceffd640 1 -- 172.21.15.81:0/3424495925 <== mds.0 v2:172.21.15.81:6836/3197735576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d02a337840 con 0x55d02a42e7a0 2023-11-08T16:40:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6ceffd640 10 client.8331 handle_command_reply: tid=0 2023-11-08T16:40:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6f9409740 1 client.8331 shutdown 2023-11-08T16:40:35.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6f9409740 1 -- 172.21.15.81:0/3424495925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 msgr2=0x55d02a3fe510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:35.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6f9409740 1 --2- 172.21.15.81:0/3424495925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a3fe510 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400d7b0 tx=0x7fd6e4001a00 comp rx=0 tx=0).stop 2023-11-08T16:40:35.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6f9409740 1 -- 172.21.15.81:0/3424495925 shutdown_connections 2023-11-08T16:40:35.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6f9409740 1 --2- 172.21.15.81:0/3424495925 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d02a415fb0 0x55d02a3fea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6f9409740 1 --2- 172.21.15.81:0/3424495925 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d02a400650 0x55d02a3fe510 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6f9409740 1 --2- 172.21.15.81:0/3424495925 >> [v2:172.21.15.81:6836/3197735576,v1:172.21.15.81:6837/3197735576] conn(0x55d02a42e7a0 0x55d02a420570 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6f9409740 1 --2- 172.21.15.81:0/3424495925 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d02a400270 0x55d02a488870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6f9409740 1 -- 172.21.15.81:0/3424495925 >> 172.21.15.81:0/3424495925 conn(0x55d02a3d3710 msgr2=0x55d02a421510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:35.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.164+0000 7fd6ceffd640 0 client.8331 ms_handle_reset on v2:172.21.15.81:6836/3197735576 2023-11-08T16:40:35.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f9409740 1 -- 172.21.15.81:0/3424495925 shutdown_connections 2023-11-08T16:40:35.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f9409740 1 -- 172.21.15.81:0/3424495925 wait complete. 2023-11-08T16:40:35.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f9409740 20 client.8331 trim_cache size 0 max 16384 2023-11-08T16:40:35.186 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:40:35.186 INFO:teuthology.orchestra.run.smithi081.stdout: "status": "no active scrubs running", 2023-11-08T16:40:35.186 INFO:teuthology.orchestra.run.smithi081.stdout: "scrubs": {} 2023-11-08T16:40:35.187 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:40:35.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f6875640 1 -- 172.21.15.81:0/1581923752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae70 msgr2=0x7fd6dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:35.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f6875640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae70 0x7fd6dc06d330 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0006fd0 tx=0x7fd6e0008040 comp rx=0 tx=0).stop 2023-11-08T16:40:35.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f6875640 1 -- 172.21.15.81:0/1581923752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f00ac8b0 msgr2=0x7fd6f00acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:35.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f6875640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f00ac8b0 0x7fd6f00acd30 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400d7e0 tx=0x7fd6e400dca0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f6875640 1 -- 172.21.15.81:0/1581923752 shutdown_connections 2023-11-08T16:40:35.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f6875640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae70 0x7fd6dc06d330 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f6875640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6f015b750 0x7fd6f015bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f6875640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f00ac8b0 0x7fd6f00acd30 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f6875640 1 --2- 172.21.15.81:0/1581923752 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6d80051d0 0x7fd6f014cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:35.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.165+0000 7fd6f6875640 1 -- 172.21.15.81:0/1581923752 >> 172.21.15.81:0/1581923752 conn(0x7fd6f00cb4e0 msgr2=0x7fd6f00ad700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:35.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.166+0000 7fd6f6875640 1 -- 172.21.15.81:0/1581923752 shutdown_connections 2023-11-08T16:40:35.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:35.166+0000 7fd6f6875640 1 -- 172.21.15.81:0/1581923752 wait complete. 2023-11-08T16:40:35.191 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-08T16:40:35.191 INFO:tasks.cephfs.mount:Validating 1048576 bytes from stripey/flushed_file_0 2023-11-08T16:40:35.192 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:40:35.192 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:40:35.192 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_0" 2023-11-08T16:40:35.193 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:40:35.193 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:40:35.193 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 1048576: 2023-11-08T16:40:35.194 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:40:35.194 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 1048576 2023-11-08T16:40:35.195 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:40:35.195 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:40:35.195 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:40:35.196 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:40:35.196 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:40:35.196 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:40:36.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:40:36.023 INFO:tasks.cephfs.mount:Validating 524288 bytes from stripey/flushed_file_1 2023-11-08T16:40:36.024 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:40:36.025 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:40:36.025 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_1" 2023-11-08T16:40:36.026 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:40:36.026 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:40:36.027 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 524288: 2023-11-08T16:40:36.028 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:40:36.028 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 524288 2023-11-08T16:40:36.029 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:40:36.029 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:40:36.029 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:40:36.030 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:40:36.030 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:40:36.030 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:40:36.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:40:36.564 INFO:tasks.cephfs.mount:Validating 917504 bytes from stripey/flushed_file_2 2023-11-08T16:40:36.565 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:40:36.565 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:40:36.565 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_2" 2023-11-08T16:40:36.566 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:40:36.566 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:40:36.567 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 917504: 2023-11-08T16:40:36.567 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:40:36.567 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 917504 2023-11-08T16:40:36.568 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:40:36.568 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:40:36.568 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:40:36.570 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:40:36.570 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:40:36.570 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:40:37.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:40:37.320 INFO:tasks.cephfs.mount:Validating 917503 bytes from stripey/flushed_file_3 2023-11-08T16:40:37.320 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:40:37.321 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:40:37.321 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_3" 2023-11-08T16:40:37.322 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:40:37.322 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:40:37.322 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 917503: 2023-11-08T16:40:37.323 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:40:37.323 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 917503 2023-11-08T16:40:37.323 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:40:37.324 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:40:37.324 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:40:37.324 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:40:37.325 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:40:37.325 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:40:38.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:40:38.081 INFO:tasks.cephfs.mount:Validating 1441792 bytes from stripey/flushed_file_4 2023-11-08T16:40:38.082 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:40:38.082 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:40:38.083 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_4" 2023-11-08T16:40:38.083 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:40:38.084 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:40:38.084 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 1441792: 2023-11-08T16:40:38.084 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:40:38.085 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 1441792 2023-11-08T16:40:38.085 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:40:38.086 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:40:38.086 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:40:38.086 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:40:38.087 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:40:38.087 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:40:39.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:40:39.117 INFO:tasks.cephfs.mount:Validating 1441793 bytes from stripey/flushed_file_5 2023-11-08T16:40:39.117 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:40:39.118 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:40:39.118 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_5" 2023-11-08T16:40:39.118 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:40:39.119 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:40:39.119 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 1441793: 2023-11-08T16:40:39.119 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:40:39.120 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 1441793 2023-11-08T16:40:39.120 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:40:39.120 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:40:39.121 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:40:39.121 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:40:39.121 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:40:39.122 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:40:40.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:40:40.170 INFO:tasks.cephfs.mount:Validating 1572864 bytes from stripey/flushed_file_6 2023-11-08T16:40:40.171 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:40:40.171 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:40:40.172 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/stripey/flushed_file_6" 2023-11-08T16:40:40.172 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:40:40.172 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:40:40.173 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 1572864: 2023-11-08T16:40:40.173 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:40:40.173 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 1572864 2023-11-08T16:40:40.174 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:40:40.174 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:40:40.175 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:40:40.175 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:40:40.175 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:40:40.176 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:40:41.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:40:41.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:41.294 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo ls lost+found) 2023-11-08T16:40:41.433 INFO:teuthology.orchestra.run.smithi081.stdout:100000001fc 2023-11-08T16:40:41.437 INFO:tasks.cephfs.mount:Validating 524288 bytes from lost+found/100000001fc 2023-11-08T16:40:41.438 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:40:41.438 DEBUG:teuthology.orchestra.run.smithi081:> import zlib 2023-11-08T16:40:41.439 DEBUG:teuthology.orchestra.run.smithi081:> path = "/home/ubuntu/cephtest/mnt.0/lost+found/100000001fc" 2023-11-08T16:40:41.439 DEBUG:teuthology.orchestra.run.smithi081:> with open(path, '"'"'r'"'"') as f: 2023-11-08T16:40:41.439 DEBUG:teuthology.orchestra.run.smithi081:> bytes = f.read() 2023-11-08T16:40:41.440 DEBUG:teuthology.orchestra.run.smithi081:> if len(bytes) != 524288: 2023-11-08T16:40:41.440 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad length {0} vs. expected {1}".format( 2023-11-08T16:40:41.441 DEBUG:teuthology.orchestra.run.smithi081:> len(bytes), 524288 2023-11-08T16:40:41.441 DEBUG:teuthology.orchestra.run.smithi081:> )) 2023-11-08T16:40:41.441 DEBUG:teuthology.orchestra.run.smithi081:> for i, b in enumerate(bytes): 2023-11-08T16:40:41.442 DEBUG:teuthology.orchestra.run.smithi081:> val = zlib.crc32(str(i).encode('"'"'utf-8'"'"')) & 7 2023-11-08T16:40:41.442 DEBUG:teuthology.orchestra.run.smithi081:> if b != chr(val): 2023-11-08T16:40:41.442 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Bad data at offset {0}".format(i)) 2023-11-08T16:40:41.443 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:40:41.658 INFO:teuthology.orchestra.run.smithi081.stderr:Traceback (most recent call last): 2023-11-08T16:40:41.659 INFO:teuthology.orchestra.run.smithi081.stderr: File "", line 13, in 2023-11-08T16:40:41.659 INFO:teuthology.orchestra.run.smithi081.stderr:RuntimeError: Bad data at offset 65536 2023-11-08T16:40:41.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:41 stdin-killer INFO: command exited with status 1: exiting normally with same code! 2023-11-08T16:40:41.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:40:41.678 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T16:40:41.679 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:40:41.827 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:40:41.962 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T16:40:41.962 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:40:41.963 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:40:42.022 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:40:42.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.235+0000 7f3b1ab90640 1 Processor -- start 2023-11-08T16:40:42.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.235+0000 7f3b1ab90640 1 -- start start 2023-11-08T16:40:42.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.237+0000 7f3b1ab90640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b141446a0 0x7f3b140b55e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.237+0000 7f3b1ab90640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3b1406e9c0 0x7f3b1406edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.237+0000 7f3b1ab90640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1406f300 con 0x7f3b140716a0 2023-11-08T16:40:42.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.237+0000 7f3b1ab90640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b14144a80 con 0x7f3b141446a0 2023-11-08T16:40:42.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.237+0000 7f3b1ab90640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b140b4820 con 0x7f3b1406e9c0 2023-11-08T16:40:42.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.237+0000 7f3b1892c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b141446a0 0x7f3b140b55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:42.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.238+0000 7f3b1892c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3b141446a0 0x7f3b140b55e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57092/0 (socket says 172.21.15.81:57092) 2023-11-08T16:40:42.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.238+0000 7f3b1892c640 1 -- 172.21.15.81:0/786562672 learned_addr learned my addr 172.21.15.81:0/786562672 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:42.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.238+0000 7f3b0bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f3b140716a0 0x7f3b1406a600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55550/0 (socket says 172.21.15.81:55550) 2023-11-08T16:40:42.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.238+0000 7f3b1912d640 1 --2- 172.21.15.81:0/786562672 >> v2:172.21.15.93:3300/0 conn(0x7f3b1406e9c0 0x7f3b1406edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:42.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.238+0000 7f3b1892c640 1 -- 172.21.15.81:0/786562672 >> v2:172.21.15.93:3300/0 conn(0x7f3b1406e9c0 msgr2=0x7f3b1406edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:42.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.238+0000 7f3b1892c640 1 --2- 172.21.15.81:0/786562672 >> v2:172.21.15.93:3300/0 conn(0x7f3b1406e9c0 0x7f3b1406edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.238+0000 7f3b1892c640 1 -- 172.21.15.81:0/786562672 >> v1:172.21.15.81:6789/0 conn(0x7f3b140716a0 legacy=0x7f3b1406a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.238+0000 7f3b1892c640 1 -- 172.21.15.81:0/786562672 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b140b4b00 con 0x7f3b141446a0 2023-11-08T16:40:42.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.238+0000 7f3b0b7fe640 1 -- 172.21.15.81:0/786562672 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289649123 0 0) 0x7f3b1406f300 con 0x7f3b140716a0 2023-11-08T16:40:42.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.238+0000 7f3b1912d640 1 --2- 172.21.15.81:0/786562672 >> v2:172.21.15.93:3300/0 conn(0x7f3b1406e9c0 0x7f3b1406edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:42.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.239+0000 7f3b1892c640 1 --2- 172.21.15.81:0/786562672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b141446a0 0x7f3b140b55e0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f3b14066cd0 tx=0x7f3b1002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5e4f39b5ce55fd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:42.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.240+0000 7f3b0b7fe640 1 -- 172.21.15.81:0/786562672 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1002cb20 con 0x7f3b141446a0 2023-11-08T16:40:42.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.240+0000 7f3b0b7fe640 1 -- 172.21.15.81:0/786562672 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1002cc80 con 0x7f3b141446a0 2023-11-08T16:40:42.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.240+0000 7f3b0b7fe640 1 -- 172.21.15.81:0/786562672 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b100343f0 con 0x7f3b141446a0 2023-11-08T16:40:42.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.240+0000 7f3b1ab90640 1 -- 172.21.15.81:0/786562672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b141446a0 msgr2=0x7f3b140b55e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.240+0000 7f3b1ab90640 1 --2- 172.21.15.81:0/786562672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b141446a0 0x7f3b140b55e0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f3b14066cd0 tx=0x7f3b1002bf80 comp rx=0 tx=0).stop 2023-11-08T16:40:42.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.241+0000 7f3b1ab90640 1 -- 172.21.15.81:0/786562672 shutdown_connections 2023-11-08T16:40:42.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.241+0000 7f3b1ab90640 1 --2- 172.21.15.81:0/786562672 >> v2:172.21.15.93:3300/0 conn(0x7f3b1406e9c0 0x7f3b1406edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.241+0000 7f3b1ab90640 1 --2- 172.21.15.81:0/786562672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b141446a0 0x7f3b140b55e0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.241+0000 7f3b1ab90640 1 -- 172.21.15.81:0/786562672 >> 172.21.15.81:0/786562672 conn(0x7f3b1406b1c0 msgr2=0x7f3b1406b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:42.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.241+0000 7f3b1ab90640 1 -- 172.21.15.81:0/786562672 shutdown_connections 2023-11-08T16:40:42.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.241+0000 7f3b1ab90640 1 -- 172.21.15.81:0/786562672 wait complete. 2023-11-08T16:40:42.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.242+0000 7f3b1ab90640 1 Processor -- start 2023-11-08T16:40:42.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.243+0000 7f3b1ab90640 1 -- start start 2023-11-08T16:40:42.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.243+0000 7f3b1ab90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b1406e9c0 0x7f3b14141790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.243+0000 7f3b1ab90640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b140716a0 0x7f3b140c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.243+0000 7f3b1ab90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b141446a0 0x7f3b140c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.243+0000 7f3b1ab90640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b141f4ec0 con 0x7f3b1406e9c0 2023-11-08T16:40:42.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.243+0000 7f3b1ab90640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140c4c70 con 0x7f3b140716a0 2023-11-08T16:40:42.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.243+0000 7f3b1ab90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b140c4f50 con 0x7f3b141446a0 2023-11-08T16:40:42.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.244+0000 7f3b1892c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b1406e9c0 0x7f3b14141790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:42.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.244+0000 7f3b1892c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b1406e9c0 0x7f3b14141790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57094/0 (socket says 172.21.15.81:57094) 2023-11-08T16:40:42.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.244+0000 7f3b1892c640 1 -- 172.21.15.81:0/3108570464 learned_addr learned my addr 172.21.15.81:0/3108570464 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:42.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.244+0000 7f3b0bfff640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b140716a0 0x7f3b140c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:42.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.244+0000 7f3b1912d640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b141446a0 0x7f3b140c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:42.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.244+0000 7f3b1892c640 1 -- 172.21.15.81:0/3108570464 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b141446a0 msgr2=0x7f3b140c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.244+0000 7f3b1892c640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b141446a0 0x7f3b140c6400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.244+0000 7f3b1892c640 1 -- 172.21.15.81:0/3108570464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b140716a0 msgr2=0x7f3b140c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.244+0000 7f3b1892c640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b140716a0 0x7f3b140c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.244+0000 7f3b1892c640 1 -- 172.21.15.81:0/3108570464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b10009b30 con 0x7f3b1406e9c0 2023-11-08T16:40:42.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.244+0000 7f3b0bfff640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b140716a0 0x7f3b140c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:42.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.245+0000 7f3b1892c640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b1406e9c0 0x7f3b14141790 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f3b10000f80 tx=0x7f3b10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:42.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.245+0000 7f3b097fa640 1 -- 172.21.15.81:0/3108570464 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b10009510 con 0x7f3b1406e9c0 2023-11-08T16:40:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.245+0000 7f3b097fa640 1 -- 172.21.15.81:0/3108570464 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1003e450 con 0x7f3b1406e9c0 2023-11-08T16:40:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.245+0000 7f3b097fa640 1 -- 172.21.15.81:0/3108570464 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1003e800 con 0x7f3b1406e9c0 2023-11-08T16:40:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.245+0000 7f3b1ab90640 1 -- 172.21.15.81:0/3108570464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b140c5230 con 0x7f3b1406e9c0 2023-11-08T16:40:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.245+0000 7f3b1ab90640 1 -- 172.21.15.81:0/3108570464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b14141cd0 con 0x7f3b1406e9c0 2023-11-08T16:40:42.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.246+0000 7f3b1ab90640 1 -- 172.21.15.81:0/3108570464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b140c56f0 con 0x7f3b1406e9c0 2023-11-08T16:40:42.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.247+0000 7f3b097fa640 1 -- 172.21.15.81:0/3108570464 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3b1003e960 con 0x7f3b1406e9c0 2023-11-08T16:40:42.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.248+0000 7f3b097fa640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0006ae70 0x7f3b0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.248+0000 7f3b097fa640 1 -- 172.21.15.81:0/3108570464 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f3b100ac620 con 0x7f3b1406e9c0 2023-11-08T16:40:42.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.248+0000 7f3b0bfff640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0006ae70 0x7f3b0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:42.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.249+0000 7f3b0bfff640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0006ae70 0x7f3b0006d330 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3b140b2b90 tx=0x7f3b04009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:42.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.253+0000 7f3b097fa640 1 -- 172.21.15.81:0/3108570464 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3b100773b0 con 0x7f3b1406e9c0 2023-11-08T16:40:42.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.429+0000 7f3b1ab90640 1 -- 172.21.15.81:0/3108570464 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b1404ece0 con 0x7f3b1406e9c0 2023-11-08T16:40:42.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.431+0000 7f3b097fa640 1 -- 172.21.15.81:0/3108570464 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18822 (secure 0 0 0) 0x7f3b1007a9d0 con 0x7f3b1406e9c0 2023-11-08T16:40:42.435 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:42.435 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":114,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:39:57.654547+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:39:07.492834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T16:39:08.634345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:39:19.654916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6836/3163533823":"2023-11-09T16:39:57.018336+0000","172.21.15.81:6836/3988749175":"2023-11-09T16:39:50.607108+0000","172.21.15.81:6837/3163533823":"2023-11-09T16:39:57.018336+0000","172.21.15.93:6835/691341796":"2023-11-09T16:39:48.576045+0000","172.21.15.81:6837/3988749175":"2023-11-09T16:39:50.607108+0000","172.21.15.93:6834/691341796":"2023-11-09T16:39:48.576045+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:40:42.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 -- 172.21.15.81:0/3108570464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0006ae70 msgr2=0x7f3b0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0006ae70 0x7f3b0006d330 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f3b140b2b90 tx=0x7f3b04009040 comp rx=0 tx=0).stop 2023-11-08T16:40:42.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 -- 172.21.15.81:0/3108570464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b1406e9c0 msgr2=0x7f3b14141790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b1406e9c0 0x7f3b14141790 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f3b10000f80 tx=0x7f3b10004a40 comp rx=0 tx=0).stop 2023-11-08T16:40:42.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 -- 172.21.15.81:0/3108570464 shutdown_connections 2023-11-08T16:40:42.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3b0006ae70 0x7f3b0006d330 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3b141446a0 0x7f3b140c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b140716a0 0x7f3b140c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 --2- 172.21.15.81:0/3108570464 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3b1406e9c0 0x7f3b14141790 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 -- 172.21.15.81:0/3108570464 >> 172.21.15.81:0/3108570464 conn(0x7f3b1406b1c0 msgr2=0x7f3b1406a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:42.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 -- 172.21.15.81:0/3108570464 shutdown_connections 2023-11-08T16:40:42.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.434+0000 7f3b1ab90640 1 -- 172.21.15.81:0/3108570464 wait complete. 2023-11-08T16:40:42.455 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:39:07.492834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:39:08.634345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:39:19.654916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:40:42.456 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:40:42.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.673+0000 7f8cb1ffc640 1 Processor -- start 2023-11-08T16:40:42.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.673+0000 7f8cb1ffc640 1 -- start start 2023-11-08T16:40:42.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.675+0000 7f8cb1ffc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8cac15d000 0x7f8cac0b2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.675+0000 7f8cb1ffc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac0b8730 0x7f8cac0cdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.675+0000 7f8cb1ffc640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cac06c100 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.675+0000 7f8cb1ffc640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac06d110 con 0x7f8cac15d000 2023-11-08T16:40:42.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.675+0000 7f8cb1ffc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0ce470 con 0x7f8cac0b8730 2023-11-08T16:40:42.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.675+0000 7f8ca2ffd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac0b8730 0x7f8cac0cdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:42.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.676+0000 7f8cab7fe640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8cac15d000 0x7f8cac0b2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:42.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.676+0000 7f8ca2ffd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac0b8730 0x7f8cac0cdaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:53816/0 (socket says 172.21.15.81:53816) 2023-11-08T16:40:42.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.676+0000 7f8ca2ffd640 1 -- 172.21.15.81:0/3988730832 learned_addr learned my addr 172.21.15.81:0/3988730832 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:42.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.676+0000 7f8cab7fe640 1 -- 172.21.15.81:0/3988730832 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac0b8730 msgr2=0x7f8cac0cdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:42.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.676+0000 7f8cab7fe640 1 --2- 172.21.15.81:0/3988730832 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac0b8730 0x7f8cac0cdaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.676+0000 7f8cab7fe640 1 -- 172.21.15.81:0/3988730832 >> v1:172.21.15.93:6789/0 conn(0x7f8cac0cdfe0 legacy=0x7f8cac160b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.676+0000 7f8cab7fe640 1 -- 172.21.15.81:0/3988730832 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac06c350 con 0x7f8cac15d000 2023-11-08T16:40:42.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.676+0000 7f8ca2ffd640 1 --2- 172.21.15.81:0/3988730832 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac0b8730 0x7f8cac0cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:40:42.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.677+0000 7f8cab7fe640 1 --2- 172.21.15.81:0/3988730832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cac15d000 0x7f8cac0b2000 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009dc0 tx=0x7f8c9c02be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5584da87b476998 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:42.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.678+0000 7f8caaffd640 1 -- 172.21.15.81:0/3988730832 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c02cc40 con 0x7f8cac15d000 2023-11-08T16:40:42.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.678+0000 7f8caaffd640 1 -- 172.21.15.81:0/3988730832 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9c02cda0 con 0x7f8cac15d000 2023-11-08T16:40:42.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.678+0000 7f8caaffd640 1 -- 172.21.15.81:0/3988730832 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9c034660 con 0x7f8cac15d000 2023-11-08T16:40:42.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.678+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/3988730832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cac15d000 msgr2=0x7f8cac0b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.678+0000 7f8cb1ffc640 1 --2- 172.21.15.81:0/3988730832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cac15d000 0x7f8cac0b2000 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c009dc0 tx=0x7f8c9c02be90 comp rx=0 tx=0).stop 2023-11-08T16:40:42.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.679+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/3988730832 shutdown_connections 2023-11-08T16:40:42.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.679+0000 7f8cb1ffc640 1 --2- 172.21.15.81:0/3988730832 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac0b8730 0x7f8cac0cdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.679+0000 7f8cb1ffc640 1 --2- 172.21.15.81:0/3988730832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cac15d000 0x7f8cac0b2000 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.679+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/3988730832 >> 172.21.15.81:0/3988730832 conn(0x7f8cac0d2e10 msgr2=0x7f8cac0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:42.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.679+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/3988730832 shutdown_connections 2023-11-08T16:40:42.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.679+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/3988730832 wait complete. 2023-11-08T16:40:42.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.680+0000 7f8cb1ffc640 1 Processor -- start 2023-11-08T16:40:42.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.680+0000 7f8cb1ffc640 1 -- start start 2023-11-08T16:40:42.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.681+0000 7f8cb1ffc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cac0b8730 0x7f8cac0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.681+0000 7f8cb1ffc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cac0cdfe0 0x7f8cac14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.681+0000 7f8cb1ffc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac15d000 0x7f8cac0cb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.681+0000 7f8cb1ffc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0cccb0 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.681+0000 7f8cb1ffc640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac14c650 con 0x7f8cac0b8730 2023-11-08T16:40:42.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.681+0000 7f8cb1ffc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0cc030 con 0x7f8cac15d000 2023-11-08T16:40:42.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8ca2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cac0cdfe0 0x7f8cac14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:42.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8ca2ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cac0cdfe0 0x7f8cac14c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57104/0 (socket says 172.21.15.81:57104) 2023-11-08T16:40:42.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8ca2ffd640 1 -- 172.21.15.81:0/2423527892 learned_addr learned my addr 172.21.15.81:0/2423527892 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:42.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8cabfff640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac15d000 0x7f8cac0cb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:42.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8ca2ffd640 1 -- 172.21.15.81:0/2423527892 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac15d000 msgr2=0x7f8cac0cb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8ca2ffd640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac15d000 0x7f8cac0cb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8ca2ffd640 1 -- 172.21.15.81:0/2423527892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cac0b8730 msgr2=0x7f8cac0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8ca2ffd640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cac0b8730 0x7f8cac0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8ca2ffd640 1 -- 172.21.15.81:0/2423527892 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c940094d0 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8cabfff640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac15d000 0x7f8cac0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:42.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8ca2ffd640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cac0cdfe0 0x7f8cac14c110 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400e9d0 tx=0x7f8c9400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:42.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.682+0000 7f8ca8ff9640 1 -- 172.21.15.81:0/2423527892 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9400ca60 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.683+0000 7f8ca8ff9640 1 -- 172.21.15.81:0/2423527892 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9400cbc0 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.683+0000 7f8ca8ff9640 1 -- 172.21.15.81:0/2423527892 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c94013470 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.683+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/2423527892 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9c009a70 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.683+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/2423527892 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac0cc5f0 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.684+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/2423527892 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac04ece0 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.685+0000 7f8ca8ff9640 1 -- 172.21.15.81:0/2423527892 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8c940135d0 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.686+0000 7f8ca8ff9640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8c8c06ae70 0x7f8c8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.686+0000 7f8ca8ff9640 1 -- 172.21.15.81:0/2423527892 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f8c94089d30 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.686+0000 7f8cab7fe640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8c8c06ae70 0x7f8c8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:42.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.687+0000 7f8cab7fe640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8c8c06ae70 0x7f8c8c06d330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c036000 tx=0x7f8c9c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:42.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.690+0000 7f8ca8ff9640 1 -- 172.21.15.81:0/2423527892 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8c94054a30 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.880+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/2423527892 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8cac0ccdf0 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.881+0000 7f8ca8ff9640 1 -- 172.21.15.81:0/2423527892 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8cac0ccdf0 con 0x7f8cac0cdfe0 2023-11-08T16:40:42.883 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:40:42.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.883+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/2423527892 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8c8c06ae70 msgr2=0x7f8c8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.883+0000 7f8cb1ffc640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8c8c06ae70 0x7f8c8c06d330 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c036000 tx=0x7f8c9c02e000 comp rx=0 tx=0).stop 2023-11-08T16:40:42.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.883+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/2423527892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cac0cdfe0 msgr2=0x7f8cac14c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:42.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.883+0000 7f8cb1ffc640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cac0cdfe0 0x7f8cac14c110 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400e9d0 tx=0x7f8c9400ee90 comp rx=0 tx=0).stop 2023-11-08T16:40:42.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.884+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/2423527892 shutdown_connections 2023-11-08T16:40:42.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.884+0000 7f8cb1ffc640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8c8c06ae70 0x7f8c8c06d330 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.884+0000 7f8cb1ffc640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8cac15d000 0x7f8cac0cb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.884+0000 7f8cb1ffc640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8cac0cdfe0 0x7f8cac14c110 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.884+0000 7f8cb1ffc640 1 --2- 172.21.15.81:0/2423527892 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8cac0b8730 0x7f8cac0cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:42.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.884+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/2423527892 >> 172.21.15.81:0/2423527892 conn(0x7f8cac0d2e10 msgr2=0x7f8cac0b43f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:42.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.884+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/2423527892 shutdown_connections 2023-11-08T16:40:42.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:42.884+0000 7f8cb1ffc640 1 -- 172.21.15.81:0/2423527892 wait complete. 2023-11-08T16:40:42.904 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:40:43.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.112+0000 7fdc4675e640 1 Processor -- start 2023-11-08T16:40:43.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.112+0000 7fdc4675e640 1 -- start start 2023-11-08T16:40:43.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.114+0000 7fdc4675e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fdc400cdf90 0x7fdc400c1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.114+0000 7fdc4675e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdc400c1e50 0x7fdc4006f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.114+0000 7fdc4675e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4006cae0 con 0x7fdc40161730 2023-11-08T16:40:43.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.114+0000 7fdc4675e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc400c22e0 con 0x7fdc400c1e50 2023-11-08T16:40:43.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.114+0000 7fdc4675e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc400cdb00 con 0x7fdc400cdf90 2023-11-08T16:40:43.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.114+0000 7fdc44cfb640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdc400c1e50 0x7fdc4006f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.114+0000 7fdc44cfb640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdc400c1e50 0x7fdc4006f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57112/0 (socket says 172.21.15.81:57112) 2023-11-08T16:40:43.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.114+0000 7fdc44cfb640 1 -- 172.21.15.81:0/3202401469 learned_addr learned my addr 172.21.15.81:0/3202401469 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:43.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.114+0000 7fdc44cfb640 1 -- 172.21.15.81:0/3202401469 >> v2:172.21.15.93:3300/0 conn(0x7fdc400cdf90 msgr2=0x7fdc400c1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:43.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.115+0000 7fdc3f7fe640 1 --2- 172.21.15.81:0/3202401469 >> v2:172.21.15.93:3300/0 conn(0x7fdc400cdf90 0x7fdc400c1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.115+0000 7fdc44cfb640 1 --2- 172.21.15.81:0/3202401469 >> v2:172.21.15.93:3300/0 conn(0x7fdc400cdf90 0x7fdc400c1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.115+0000 7fdc3effd640 1 -- 172.21.15.81:0/3202401469 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1036792892 0 0) 0x7fdc4006cae0 con 0x7fdc40161730 2023-11-08T16:40:43.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.115+0000 7fdc44cfb640 1 -- 172.21.15.81:0/3202401469 >> v1:172.21.15.81:6789/0 conn(0x7fdc40161730 legacy=0x7fdc4006d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.115+0000 7fdc44cfb640 1 -- 172.21.15.81:0/3202401469 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc40161b10 con 0x7fdc400c1e50 2023-11-08T16:40:43.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.115+0000 7fdc3f7fe640 1 --2- 172.21.15.81:0/3202401469 >> v2:172.21.15.93:3300/0 conn(0x7fdc400cdf90 0x7fdc400c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:43.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.116+0000 7fdc44cfb640 1 --2- 172.21.15.81:0/3202401469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc400c1e50 0x7fdc4006f990 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009e50 tx=0x7fdc3802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db52163c6744038 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:43.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.116+0000 7fdc3effd640 1 -- 172.21.15.81:0/3202401469 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3802cb10 con 0x7fdc400c1e50 2023-11-08T16:40:43.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.116+0000 7fdc3effd640 1 -- 172.21.15.81:0/3202401469 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc3802cc70 con 0x7fdc400c1e50 2023-11-08T16:40:43.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.116+0000 7fdc3effd640 1 -- 172.21.15.81:0/3202401469 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380343f0 con 0x7fdc400c1e50 2023-11-08T16:40:43.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.116+0000 7fdc4675e640 1 -- 172.21.15.81:0/3202401469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc400c1e50 msgr2=0x7fdc4006f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.116+0000 7fdc4675e640 1 --2- 172.21.15.81:0/3202401469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc400c1e50 0x7fdc4006f990 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009e50 tx=0x7fdc3802bf70 comp rx=0 tx=0).stop 2023-11-08T16:40:43.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.117+0000 7fdc4675e640 1 -- 172.21.15.81:0/3202401469 shutdown_connections 2023-11-08T16:40:43.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.117+0000 7fdc4675e640 1 --2- 172.21.15.81:0/3202401469 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc400c1e50 0x7fdc4006f990 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.117+0000 7fdc4675e640 1 --2- 172.21.15.81:0/3202401469 >> v2:172.21.15.93:3300/0 conn(0x7fdc400cdf90 0x7fdc400c1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.117+0000 7fdc4675e640 1 -- 172.21.15.81:0/3202401469 >> 172.21.15.81:0/3202401469 conn(0x7fdc400cb580 msgr2=0x7fdc400cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:43.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.117+0000 7fdc4675e640 1 -- 172.21.15.81:0/3202401469 shutdown_connections 2023-11-08T16:40:43.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.117+0000 7fdc4675e640 1 -- 172.21.15.81:0/3202401469 wait complete. 2023-11-08T16:40:43.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.118+0000 7fdc4675e640 1 Processor -- start 2023-11-08T16:40:43.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.118+0000 7fdc4675e640 1 -- start start 2023-11-08T16:40:43.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.119+0000 7fdc4675e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc400c1e50 0x7fdc40153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.119+0000 7fdc4675e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc400cdf90 0x7fdc400b1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.119+0000 7fdc4675e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc40161730 0x7fdc400b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.119+0000 7fdc4675e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc401540c0 con 0x7fdc400cdf90 2023-11-08T16:40:43.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.119+0000 7fdc4675e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400b28c0 con 0x7fdc40161730 2023-11-08T16:40:43.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.119+0000 7fdc4675e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400b2ba0 con 0x7fdc400c1e50 2023-11-08T16:40:43.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.120+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc400cdf90 0x7fdc400b1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.120+0000 7fdc3f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc400cdf90 0x7fdc400b1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57114/0 (socket says 172.21.15.81:57114) 2023-11-08T16:40:43.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.120+0000 7fdc3f7fe640 1 -- 172.21.15.81:0/3633343823 learned_addr learned my addr 172.21.15.81:0/3633343823 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:43.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.120+0000 7fdc3f7fe640 1 -- 172.21.15.81:0/3633343823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc400c1e50 msgr2=0x7fdc40153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.120+0000 7fdc3f7fe640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc400c1e50 0x7fdc40153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.120+0000 7fdc3f7fe640 1 -- 172.21.15.81:0/3633343823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc40161730 msgr2=0x7fdc400b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.120+0000 7fdc3f7fe640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc40161730 0x7fdc400b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.120+0000 7fdc3f7fe640 1 -- 172.21.15.81:0/3633343823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38009b30 con 0x7fdc400cdf90 2023-11-08T16:40:43.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.120+0000 7fdc3f7fe640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc400cdf90 0x7fdc400b1de0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c7f0 tx=0x7fdc3400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:43.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.121+0000 7fdc3cff9640 1 -- 172.21.15.81:0/3633343823 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc34007a30 con 0x7fdc400cdf90 2023-11-08T16:40:43.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.121+0000 7fdc3cff9640 1 -- 172.21.15.81:0/3633343823 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc34007b90 con 0x7fdc400cdf90 2023-11-08T16:40:43.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.121+0000 7fdc3cff9640 1 -- 172.21.15.81:0/3633343823 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3400ce40 con 0x7fdc400cdf90 2023-11-08T16:40:43.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.121+0000 7fdc4675e640 1 -- 172.21.15.81:0/3633343823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc400b2e80 con 0x7fdc400cdf90 2023-11-08T16:40:43.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.121+0000 7fdc4675e640 1 -- 172.21.15.81:0/3633343823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc400b3440 con 0x7fdc400cdf90 2023-11-08T16:40:43.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.122+0000 7fdc4675e640 1 -- 172.21.15.81:0/3633343823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc4004ece0 con 0x7fdc400cdf90 2023-11-08T16:40:43.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.123+0000 7fdc3cff9640 1 -- 172.21.15.81:0/3633343823 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdc340113f0 con 0x7fdc400cdf90 2023-11-08T16:40:43.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.123+0000 7fdc3cff9640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2c06ae70 0x7fdc2c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.124+0000 7fdc3cff9640 1 -- 172.21.15.81:0/3633343823 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fdc34089b60 con 0x7fdc400cdf90 2023-11-08T16:40:43.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.124+0000 7fdc3ffff640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2c06ae70 0x7fdc2c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.124+0000 7fdc3ffff640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2c06ae70 0x7fdc2c06d330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdc401fa930 tx=0x7fdc30007930 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:43.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.128+0000 7fdc3cff9640 1 -- 172.21.15.81:0/3633343823 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdc34054860 con 0x7fdc400cdf90 2023-11-08T16:40:43.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.303+0000 7fdc4675e640 1 -- 172.21.15.81:0/3633343823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdc4004e490 con 0x7fdc400cdf90 2023-11-08T16:40:43.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.304+0000 7fdc3cff9640 1 -- 172.21.15.81:0/3633343823 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdc34057e80 con 0x7fdc400cdf90 2023-11-08T16:40:43.307 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:40:43.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 -- 172.21.15.81:0/3633343823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2c06ae70 msgr2=0x7fdc2c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2c06ae70 0x7fdc2c06d330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdc401fa930 tx=0x7fdc30007930 comp rx=0 tx=0).stop 2023-11-08T16:40:43.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 -- 172.21.15.81:0/3633343823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc400cdf90 msgr2=0x7fdc400b1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc400cdf90 0x7fdc400b1de0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fdc3400c7f0 tx=0x7fdc3400cbf0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 -- 172.21.15.81:0/3633343823 shutdown_connections 2023-11-08T16:40:43.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdc2c06ae70 0x7fdc2c06d330 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdc40161730 0x7fdc400b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdc400cdf90 0x7fdc400b1de0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 --2- 172.21.15.81:0/3633343823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdc400c1e50 0x7fdc40153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 -- 172.21.15.81:0/3633343823 >> 172.21.15.81:0/3633343823 conn(0x7fdc400cb580 msgr2=0x7fdc400704d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:43.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 -- 172.21.15.81:0/3633343823 shutdown_connections 2023-11-08T16:40:43.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.307+0000 7fdc4675e640 1 -- 172.21.15.81:0/3633343823 wait complete. 2023-11-08T16:40:43.329 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:40:43.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.541+0000 7f9b5abb6640 1 Processor -- start 2023-11-08T16:40:43.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.541+0000 7f9b5abb6640 1 -- start start 2023-11-08T16:40:43.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.542+0000 7f9b5abb6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b540ce0d0 0x7f9b540caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.542+0000 7f9b5abb6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9b5415f280 0x7f9b541516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.542+0000 7f9b5abb6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5406fa30 con 0x7f9b5415bfc0 2023-11-08T16:40:43.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.542+0000 7f9b5abb6640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b540cda10 con 0x7f9b5415f280 2023-11-08T16:40:43.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.542+0000 7f9b5abb6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b540ce4b0 con 0x7f9b540ce0d0 2023-11-08T16:40:43.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.543+0000 7f9b59153640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9b5415f280 0x7f9b541516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.543+0000 7f9b59153640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9b5415f280 0x7f9b541516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57130/0 (socket says 172.21.15.81:57130) 2023-11-08T16:40:43.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.543+0000 7f9b59153640 1 -- 172.21.15.81:0/147904422 learned_addr learned my addr 172.21.15.81:0/147904422 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:43.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.543+0000 7f9b58952640 1 --2- 172.21.15.81:0/147904422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b540ce0d0 0x7f9b540caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.543+0000 7f9b59153640 1 -- 172.21.15.81:0/147904422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b540ce0d0 msgr2=0x7f9b540caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:43.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.543+0000 7f9b59153640 1 --2- 172.21.15.81:0/147904422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b540ce0d0 0x7f9b540caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.543+0000 7f9b59153640 1 -- 172.21.15.81:0/147904422 >> v1:172.21.15.93:6789/0 conn(0x7f9b5415bfc0 legacy=0x7f9b5415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.543+0000 7f9b59153640 1 -- 172.21.15.81:0/147904422 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5406fca0 con 0x7f9b5415f280 2023-11-08T16:40:43.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.544+0000 7f9b59153640 1 --2- 172.21.15.81:0/147904422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5415f280 0x7f9b541516e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009e50 tx=0x7f9b4c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4630ac8335c7bf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:43.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.545+0000 7f9b537fe640 1 -- 172.21.15.81:0/147904422 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c02cb00 con 0x7f9b5415f280 2023-11-08T16:40:43.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.545+0000 7f9b537fe640 1 -- 172.21.15.81:0/147904422 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c02cc60 con 0x7f9b5415f280 2023-11-08T16:40:43.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.545+0000 7f9b537fe640 1 -- 172.21.15.81:0/147904422 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c0343f0 con 0x7f9b5415f280 2023-11-08T16:40:43.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.545+0000 7f9b5abb6640 1 -- 172.21.15.81:0/147904422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5415f280 msgr2=0x7f9b541516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.545+0000 7f9b5abb6640 1 --2- 172.21.15.81:0/147904422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5415f280 0x7f9b541516e0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009e50 tx=0x7f9b4c02bf60 comp rx=0 tx=0).stop 2023-11-08T16:40:43.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.545+0000 7f9b5abb6640 1 -- 172.21.15.81:0/147904422 shutdown_connections 2023-11-08T16:40:43.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.545+0000 7f9b5abb6640 1 --2- 172.21.15.81:0/147904422 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5415f280 0x7f9b541516e0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.545+0000 7f9b5abb6640 1 --2- 172.21.15.81:0/147904422 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b540ce0d0 0x7f9b540caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.545+0000 7f9b5abb6640 1 -- 172.21.15.81:0/147904422 >> 172.21.15.81:0/147904422 conn(0x7f9b540cb580 msgr2=0x7f9b540cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:43.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.545+0000 7f9b5abb6640 1 -- 172.21.15.81:0/147904422 shutdown_connections 2023-11-08T16:40:43.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.545+0000 7f9b5abb6640 1 -- 172.21.15.81:0/147904422 wait complete. 2023-11-08T16:40:43.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.547+0000 7f9b5abb6640 1 Processor -- start 2023-11-08T16:40:43.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.547+0000 7f9b5abb6640 1 -- start start 2023-11-08T16:40:43.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.548+0000 7f9b5abb6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b540ce0d0 0x7f9b540c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.548+0000 7f9b5abb6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b5415bfc0 0x7f9b540c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.548+0000 7f9b5abb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5415f280 0x7f9b540c9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.548+0000 7f9b5abb6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b541f6eb0 con 0x7f9b5415f280 2023-11-08T16:40:43.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.548+0000 7f9b5abb6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b541f6bd0 con 0x7f9b5415bfc0 2023-11-08T16:40:43.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.548+0000 7f9b5abb6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b540c2760 con 0x7f9b540ce0d0 2023-11-08T16:40:43.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.548+0000 7f9b59153640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5415f280 0x7f9b540c9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.548+0000 7f9b59153640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5415f280 0x7f9b540c9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57142/0 (socket says 172.21.15.81:57142) 2023-11-08T16:40:43.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.548+0000 7f9b59153640 1 -- 172.21.15.81:0/1420245944 learned_addr learned my addr 172.21.15.81:0/1420245944 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:43.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.549+0000 7f9b58952640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b540ce0d0 0x7f9b540c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.549+0000 7f9b53fff640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b5415bfc0 0x7f9b540c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.549+0000 7f9b59153640 1 -- 172.21.15.81:0/1420245944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b540ce0d0 msgr2=0x7f9b540c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.549+0000 7f9b59153640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b540ce0d0 0x7f9b540c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.549+0000 7f9b59153640 1 -- 172.21.15.81:0/1420245944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b5415bfc0 msgr2=0x7f9b540c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.549+0000 7f9b59153640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b5415bfc0 0x7f9b540c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.549+0000 7f9b59153640 1 -- 172.21.15.81:0/1420245944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b4c009b30 con 0x7f9b5415f280 2023-11-08T16:40:43.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.549+0000 7f9b59153640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5415f280 0x7f9b540c9d50 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009e50 tx=0x7f9b4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:43.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.549+0000 7f9b517fa640 1 -- 172.21.15.81:0/1420245944 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c040670 con 0x7f9b5415f280 2023-11-08T16:40:43.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.550+0000 7f9b517fa640 1 -- 172.21.15.81:0/1420245944 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b4c0407d0 con 0x7f9b5415f280 2023-11-08T16:40:43.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.550+0000 7f9b5abb6640 1 -- 172.21.15.81:0/1420245944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b540c2a40 con 0x7f9b5415f280 2023-11-08T16:40:43.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.550+0000 7f9b517fa640 1 -- 172.21.15.81:0/1420245944 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b4c040b80 con 0x7f9b5415f280 2023-11-08T16:40:43.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.550+0000 7f9b5abb6640 1 -- 172.21.15.81:0/1420245944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b541f5f30 con 0x7f9b5415f280 2023-11-08T16:40:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.551+0000 7f9b5abb6640 1 -- 172.21.15.81:0/1420245944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b540c2f60 con 0x7f9b5415f280 2023-11-08T16:40:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.552+0000 7f9b517fa640 1 -- 172.21.15.81:0/1420245944 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9b4c040ce0 con 0x7f9b5415f280 2023-11-08T16:40:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.553+0000 7f9b517fa640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b4006ae70 0x7f9b4006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.553+0000 7f9b517fa640 1 -- 172.21.15.81:0/1420245944 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f9b4c0ac980 con 0x7f9b5415f280 2023-11-08T16:40:43.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.553+0000 7f9b58952640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b4006ae70 0x7f9b4006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.554+0000 7f9b58952640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b4006ae70 0x7f9b4006d330 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9b48006fd0 tx=0x7f9b48008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:43.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.557+0000 7f9b517fa640 1 -- 172.21.15.81:0/1420245944 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9b540c2f60 con 0x7f9b5415f280 2023-11-08T16:40:43.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.748+0000 7f9b5abb6640 1 -- 172.21.15.81:0/1420245944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9b540c2f60 con 0x7f9b5415f280 2023-11-08T16:40:43.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.749+0000 7f9b517fa640 1 -- 172.21.15.81:0/1420245944 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9b540c2f60 con 0x7f9b5415f280 2023-11-08T16:40:43.752 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:40:43.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.751+0000 7f9b5abb6640 1 -- 172.21.15.81:0/1420245944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b4006ae70 msgr2=0x7f9b4006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.751+0000 7f9b5abb6640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b4006ae70 0x7f9b4006d330 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9b48006fd0 tx=0x7f9b48008040 comp rx=0 tx=0).stop 2023-11-08T16:40:43.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.751+0000 7f9b5abb6640 1 -- 172.21.15.81:0/1420245944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5415f280 msgr2=0x7f9b540c9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.751+0000 7f9b5abb6640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5415f280 0x7f9b540c9d50 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f9b4c009e50 tx=0x7f9b4c004a40 comp rx=0 tx=0).stop 2023-11-08T16:40:43.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.752+0000 7f9b5abb6640 1 -- 172.21.15.81:0/1420245944 shutdown_connections 2023-11-08T16:40:43.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.752+0000 7f9b5abb6640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9b4006ae70 0x7f9b4006d330 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.752+0000 7f9b5abb6640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9b5415f280 0x7f9b540c9d50 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.752+0000 7f9b5abb6640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9b5415bfc0 0x7f9b540c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.752+0000 7f9b5abb6640 1 --2- 172.21.15.81:0/1420245944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9b540ce0d0 0x7f9b540c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.752+0000 7f9b5abb6640 1 -- 172.21.15.81:0/1420245944 >> 172.21.15.81:0/1420245944 conn(0x7f9b540cb580 msgr2=0x7f9b5406b710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:43.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.752+0000 7f9b5abb6640 1 -- 172.21.15.81:0/1420245944 shutdown_connections 2023-11-08T16:40:43.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.752+0000 7f9b5abb6640 1 -- 172.21.15.81:0/1420245944 wait complete. 2023-11-08T16:40:43.773 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:40:43.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.984+0000 7f1410301640 1 Processor -- start 2023-11-08T16:40:43.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.984+0000 7f1410301640 1 -- start start 2023-11-08T16:40:43.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.985+0000 7f1410301640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f140815a0e0 0x7f14080ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.986+0000 7f1410301640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f14080ca750 0x7f140806b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.986+0000 7f1410301640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14080ce960 con 0x7f14080c9bf0 2023-11-08T16:40:43.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.986+0000 7f1410301640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1408071980 con 0x7f14080ca750 2023-11-08T16:40:43.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.986+0000 7f1410301640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14080c9480 con 0x7f140815a0e0 2023-11-08T16:40:43.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.986+0000 7f140e89e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f14080ca750 0x7f140806b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.986+0000 7f140d89c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f14080c9bf0 0x7f14080ca040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55568/0 (socket says 172.21.15.81:55568) 2023-11-08T16:40:43.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.986+0000 7f140d89c640 1 -- 172.21.15.81:0/3754782862 learned_addr learned my addr 172.21.15.81:0/3754782862 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:43.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.986+0000 7f140e09d640 1 --2- 172.21.15.81:0/3754782862 >> v2:172.21.15.93:3300/0 conn(0x7f140815a0e0 0x7f14080ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:43.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.986+0000 7f140e89e640 1 -- 172.21.15.81:0/3754782862 >> v2:172.21.15.93:3300/0 conn(0x7f140815a0e0 msgr2=0x7f14080ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:43.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.986+0000 7f140e89e640 1 --2- 172.21.15.81:0/3754782862 >> v2:172.21.15.93:3300/0 conn(0x7f140815a0e0 0x7f14080ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.986+0000 7f140e89e640 1 -- 172.21.15.81:0/3754782862 >> v1:172.21.15.81:6789/0 conn(0x7f14080c9bf0 legacy=0x7f14080ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.987+0000 7f140e89e640 1 -- 172.21.15.81:0/3754782862 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f140815a4c0 con 0x7f14080ca750 2023-11-08T16:40:43.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.987+0000 7f140d09b640 1 -- 172.21.15.81:0/3754782862 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2083632470 0 0) 0x7f13f8001110 con 0x7f14080c9bf0 2023-11-08T16:40:43.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.988+0000 7f140e89e640 1 --2- 172.21.15.81:0/3754782862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14080ca750 0x7f140806b710 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1408067640 tx=0x7f140002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79e2d980511c611 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:43.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.988+0000 7f140d09b640 1 -- 172.21.15.81:0/3754782862 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140002cd10 con 0x7f14080ca750 2023-11-08T16:40:43.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.988+0000 7f140d09b640 1 -- 172.21.15.81:0/3754782862 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140002ce70 con 0x7f14080ca750 2023-11-08T16:40:43.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.988+0000 7f140d09b640 1 -- 172.21.15.81:0/3754782862 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1400034640 con 0x7f14080ca750 2023-11-08T16:40:43.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.989+0000 7f1410301640 1 -- 172.21.15.81:0/3754782862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14080ca750 msgr2=0x7f140806b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:43.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.989+0000 7f1410301640 1 --2- 172.21.15.81:0/3754782862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14080ca750 0x7f140806b710 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f1408067640 tx=0x7f140002bf60 comp rx=0 tx=0).stop 2023-11-08T16:40:43.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.989+0000 7f1410301640 1 -- 172.21.15.81:0/3754782862 shutdown_connections 2023-11-08T16:40:43.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.989+0000 7f1410301640 1 --2- 172.21.15.81:0/3754782862 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14080ca750 0x7f140806b710 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.989+0000 7f1410301640 1 --2- 172.21.15.81:0/3754782862 >> v2:172.21.15.93:3300/0 conn(0x7f140815a0e0 0x7f14080ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:43.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.989+0000 7f1410301640 1 -- 172.21.15.81:0/3754782862 >> 172.21.15.81:0/3754782862 conn(0x7f14080cb580 msgr2=0x7f14080cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:43.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.989+0000 7f1410301640 1 -- 172.21.15.81:0/3754782862 shutdown_connections 2023-11-08T16:40:43.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.989+0000 7f1410301640 1 -- 172.21.15.81:0/3754782862 wait complete. 2023-11-08T16:40:43.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.991+0000 7f1410301640 1 Processor -- start 2023-11-08T16:40:43.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.991+0000 7f1410301640 1 -- start start 2023-11-08T16:40:43.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f1410301640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14080c9bf0 0x7f14080cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f1410301640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14080ca750 0x7f1408152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f1410301640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f140815a0e0 0x7f1408152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:43.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f1410301640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1408156ca0 con 0x7f14080ca750 2023-11-08T16:40:43.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f1410301640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14080c2760 con 0x7f140815a0e0 2023-11-08T16:40:43.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f1410301640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1408153a50 con 0x7f14080c9bf0 2023-11-08T16:40:43.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f140d89c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14080ca750 0x7f1408152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:44.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f140d89c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14080ca750 0x7f1408152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57164/0 (socket says 172.21.15.81:57164) 2023-11-08T16:40:44.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f140d89c640 1 -- 172.21.15.81:0/4020081903 learned_addr learned my addr 172.21.15.81:0/4020081903 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:44.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f140e09d640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14080c9bf0 0x7f14080cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:44.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f140d89c640 1 -- 172.21.15.81:0/4020081903 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14080c9bf0 msgr2=0x7f14080cd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:44.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f140d89c640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14080c9bf0 0x7f14080cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f140e89e640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f140815a0e0 0x7f1408152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:44.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f140d89c640 1 -- 172.21.15.81:0/4020081903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f140815a0e0 msgr2=0x7f1408152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:44.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f140d89c640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f140815a0e0 0x7f1408152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.992+0000 7f140d89c640 1 -- 172.21.15.81:0/4020081903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1400009a70 con 0x7f14080ca750 2023-11-08T16:40:44.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.993+0000 7f140d89c640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14080ca750 0x7f1408152220 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f13f800c780 tx=0x7f13f800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:44.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.993+0000 7f13feffd640 1 -- 172.21.15.81:0/4020081903 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f80072d0 con 0x7f14080ca750 2023-11-08T16:40:44.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.993+0000 7f13feffd640 1 -- 172.21.15.81:0/4020081903 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f8007450 con 0x7f14080ca750 2023-11-08T16:40:44.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.993+0000 7f1410301640 1 -- 172.21.15.81:0/4020081903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1408153d30 con 0x7f14080ca750 2023-11-08T16:40:44.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.993+0000 7f13feffd640 1 -- 172.21.15.81:0/4020081903 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f80047e0 con 0x7f14080ca750 2023-11-08T16:40:44.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.993+0000 7f1410301640 1 -- 172.21.15.81:0/4020081903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1408154010 con 0x7f14080ca750 2023-11-08T16:40:44.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.994+0000 7f1410301640 1 -- 172.21.15.81:0/4020081903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14080c2930 con 0x7f14080ca750 2023-11-08T16:40:44.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.995+0000 7f13feffd640 1 -- 172.21.15.81:0/4020081903 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f13f8004960 con 0x7f14080ca750 2023-11-08T16:40:44.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.996+0000 7f13feffd640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f13ec06ae70 0x7f13ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:44.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.996+0000 7f13feffd640 1 -- 172.21.15.81:0/4020081903 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f13f8088d00 con 0x7f14080ca750 2023-11-08T16:40:44.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.997+0000 7f140e09d640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f13ec06ae70 0x7f13ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:44.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:43.997+0000 7f140e09d640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f13ec06ae70 0x7f13ec06d330 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1404006fd0 tx=0x7f1404008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:44.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.000+0000 7f13feffd640 1 -- 172.21.15.81:0/4020081903 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f13f8053a90 con 0x7f14080ca750 2023-11-08T16:40:44.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.188+0000 7f1410301640 1 -- 172.21.15.81:0/4020081903 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f14080ce930 con 0x7f14080ca750 2023-11-08T16:40:44.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.188+0000 7f13feffd640 1 -- 172.21.15.81:0/4020081903 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v114) v1 ==== 106+0+11 (secure 0 0 0) 0x7f13f8018020 con 0x7f14080ca750 2023-11-08T16:40:44.191 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:40:44.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.191+0000 7f1410301640 1 -- 172.21.15.81:0/4020081903 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f13ec06ae70 msgr2=0x7f13ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:44.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.191+0000 7f1410301640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f13ec06ae70 0x7f13ec06d330 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f1404006fd0 tx=0x7f1404008040 comp rx=0 tx=0).stop 2023-11-08T16:40:44.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.191+0000 7f1410301640 1 -- 172.21.15.81:0/4020081903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14080ca750 msgr2=0x7f1408152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:44.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.191+0000 7f1410301640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14080ca750 0x7f1408152220 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f13f800c780 tx=0x7f13f800cc40 comp rx=0 tx=0).stop 2023-11-08T16:40:44.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.191+0000 7f1410301640 1 -- 172.21.15.81:0/4020081903 shutdown_connections 2023-11-08T16:40:44.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.191+0000 7f1410301640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f13ec06ae70 0x7f13ec06d330 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.191+0000 7f1410301640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f140815a0e0 0x7f1408152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.191+0000 7f1410301640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f14080ca750 0x7f1408152220 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.191+0000 7f1410301640 1 --2- 172.21.15.81:0/4020081903 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f14080c9bf0 0x7f14080cd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.191+0000 7f1410301640 1 -- 172.21.15.81:0/4020081903 >> 172.21.15.81:0/4020081903 conn(0x7f14080cb580 msgr2=0x7f14080b4170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:44.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.191+0000 7f1410301640 1 -- 172.21.15.81:0/4020081903 shutdown_connections 2023-11-08T16:40:44.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.192+0000 7f1410301640 1 -- 172.21.15.81:0/4020081903 wait complete. 2023-11-08T16:40:44.211 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:40:44.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.412+0000 7ffbcc3ee640 1 Processor -- start 2023-11-08T16:40:44.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.413+0000 7ffbcc3ee640 1 -- start start 2023-11-08T16:40:44.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.414+0000 7ffbcc3ee640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ffbc415aeb0 0x7ffbc40ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:44.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.414+0000 7ffbcc3ee640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc406cc60 con 0x7ffbc40c9580 2023-11-08T16:40:44.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.414+0000 7ffbcc3ee640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc40cae20 con 0x7ffbc40c90f0 2023-11-08T16:40:44.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.414+0000 7ffbcc3ee640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc40cb3e0 con 0x7ffbc415aeb0 2023-11-08T16:40:44.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.415+0000 7ffbca18a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ffbc415aeb0 0x7ffbc40ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:44.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.415+0000 7ffbca18a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ffbc415aeb0 0x7ffbc40ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57172/0 (socket says 172.21.15.81:57172) 2023-11-08T16:40:44.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.415+0000 7ffbca18a640 1 -- 172.21.15.81:0/2640201682 learned_addr learned my addr 172.21.15.81:0/2640201682 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:44.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.415+0000 7ffbca98b640 1 --1- 172.21.15.81:0/2640201682 >> v1:172.21.15.81:6789/0 conn(0x7ffbc40c9580 0x7ffbc406b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55570/0 (socket says 172.21.15.81:55570) 2023-11-08T16:40:44.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.415+0000 7ffbca18a640 1 -- 172.21.15.81:0/2640201682 >> v1:172.21.15.93:6789/0 conn(0x7ffbc40c90f0 legacy=0x7ffbc406b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:44.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.415+0000 7ffbca18a640 1 -- 172.21.15.81:0/2640201682 >> v1:172.21.15.81:6789/0 conn(0x7ffbc40c9580 legacy=0x7ffbc406b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:44.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.415+0000 7ffbca18a640 1 -- 172.21.15.81:0/2640201682 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40cb100 con 0x7ffbc415aeb0 2023-11-08T16:40:44.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.416+0000 7ffbca18a640 1 --2- 172.21.15.81:0/2640201682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbc415aeb0 0x7ffbc40ca4a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009850 tx=0x7ffbc002ba00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e16dbeb2526de28 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:44.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.416+0000 7ffbc9188640 1 -- 172.21.15.81:0/2640201682 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0004230 con 0x7ffbc415aeb0 2023-11-08T16:40:44.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.417+0000 7ffbc9188640 1 -- 172.21.15.81:0/2640201682 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc0004390 con 0x7ffbc415aeb0 2023-11-08T16:40:44.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.417+0000 7ffbc9188640 1 -- 172.21.15.81:0/2640201682 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc002f820 con 0x7ffbc415aeb0 2023-11-08T16:40:44.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.417+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2640201682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbc415aeb0 msgr2=0x7ffbc40ca4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:44.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.417+0000 7ffbcc3ee640 1 --2- 172.21.15.81:0/2640201682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbc415aeb0 0x7ffbc40ca4a0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009850 tx=0x7ffbc002ba00 comp rx=0 tx=0).stop 2023-11-08T16:40:44.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.417+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2640201682 shutdown_connections 2023-11-08T16:40:44.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.417+0000 7ffbcc3ee640 1 --2- 172.21.15.81:0/2640201682 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbc415aeb0 0x7ffbc40ca4a0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.417+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2640201682 >> 172.21.15.81:0/2640201682 conn(0x7ffbc40ceaa0 msgr2=0x7ffbc40cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:44.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.417+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2640201682 shutdown_connections 2023-11-08T16:40:44.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.417+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2640201682 wait complete. 2023-11-08T16:40:44.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.419+0000 7ffbcc3ee640 1 Processor -- start 2023-11-08T16:40:44.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.419+0000 7ffbcc3ee640 1 -- start start 2023-11-08T16:40:44.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.420+0000 7ffbcc3ee640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffbc40c90f0 0x7ffbc414ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:44.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.420+0000 7ffbcc3ee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbc40c9580 0x7ffbc40cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:44.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.420+0000 7ffbcc3ee640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc415aeb0 0x7ffbc40cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:44.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.420+0000 7ffbcc3ee640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc415bd00 con 0x7ffbc40c9580 2023-11-08T16:40:44.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.420+0000 7ffbcc3ee640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc415bf10 con 0x7ffbc415aeb0 2023-11-08T16:40:44.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.420+0000 7ffbcc3ee640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc415c1f0 con 0x7ffbc40c90f0 2023-11-08T16:40:44.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.420+0000 7ffbc9989640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbc40c9580 0x7ffbc40cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:44.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.420+0000 7ffbc9989640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbc40c9580 0x7ffbc40cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57184/0 (socket says 172.21.15.81:57184) 2023-11-08T16:40:44.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.420+0000 7ffbc9989640 1 -- 172.21.15.81:0/2715639173 learned_addr learned my addr 172.21.15.81:0/2715639173 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:44.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbca18a640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffbc40c90f0 0x7ffbc414ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:44.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbc9989640 1 -- 172.21.15.81:0/2715639173 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffbc40c90f0 msgr2=0x7ffbc414ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:44.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbca98b640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc415aeb0 0x7ffbc40cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:44.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbc9989640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffbc40c90f0 0x7ffbc414ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbc9989640 1 -- 172.21.15.81:0/2715639173 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc415aeb0 msgr2=0x7ffbc40cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:44.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbc9989640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc415aeb0 0x7ffbc40cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbc9989640 1 -- 172.21.15.81:0/2715639173 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc0009460 con 0x7ffbc40c9580 2023-11-08T16:40:44.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbc9989640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbc40c9580 0x7ffbc40cd590 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c450 tx=0x7ffbb400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:44.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbbaffd640 1 -- 172.21.15.81:0/2715639173 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40052d0 con 0x7ffbc40c9580 2023-11-08T16:40:44.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbbaffd640 1 -- 172.21.15.81:0/2715639173 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb4005430 con 0x7ffbc40c9580 2023-11-08T16:40:44.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbbaffd640 1 -- 172.21.15.81:0/2715639173 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4012870 con 0x7ffbc40c9580 2023-11-08T16:40:44.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2715639173 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc415c4d0 con 0x7ffbc40c9580 2023-11-08T16:40:44.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.421+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2715639173 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc40cc910 con 0x7ffbc40c9580 2023-11-08T16:40:44.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.422+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2715639173 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc415c9f0 con 0x7ffbc40c9580 2023-11-08T16:40:44.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.423+0000 7ffbbaffd640 1 -- 172.21.15.81:0/2715639173 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffbb40129d0 con 0x7ffbc40c9580 2023-11-08T16:40:44.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.424+0000 7ffbbaffd640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffba806ae70 0x7ffba806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:44.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.424+0000 7ffbbaffd640 1 -- 172.21.15.81:0/2715639173 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7ffbb4089580 con 0x7ffbc40c9580 2023-11-08T16:40:44.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.425+0000 7ffbca18a640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffba806ae70 0x7ffba806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:44.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.425+0000 7ffbca18a640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffba806ae70 0x7ffba806d330 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0035000 tx=0x7ffbc0008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:44.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.429+0000 7ffbbaffd640 1 -- 172.21.15.81:0/2715639173 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffbc415c9f0 con 0x7ffbc40c9580 2023-11-08T16:40:44.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.630+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2715639173 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffbc415c9f0 con 0x7ffbc40c9580 2023-11-08T16:40:44.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.631+0000 7ffbbaffd640 1 -- 172.21.15.81:0/2715639173 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v114) v1 ==== 80+0+364 (secure 0 0 0) 0x7ffbc415c9f0 con 0x7ffbc40c9580 2023-11-08T16:40:44.634 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6836/3163533823 2023-11-09T16:39:57.018336+0000 2023-11-08T16:40:44.634 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6836/3988749175 2023-11-09T16:39:50.607108+0000 2023-11-08T16:40:44.635 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/3163533823 2023-11-09T16:39:57.018336+0000 2023-11-08T16:40:44.635 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.93:6835/691341796 2023-11-09T16:39:48.576045+0000 2023-11-08T16:40:44.635 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/3988749175 2023-11-09T16:39:50.607108+0000 2023-11-08T16:40:44.635 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.93:6834/691341796 2023-11-09T16:39:48.576045+0000 2023-11-08T16:40:44.636 INFO:teuthology.orchestra.run.smithi081.stderr:listed 6 entries 2023-11-08T16:40:44.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2715639173 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffba806ae70 msgr2=0x7ffba806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:44.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffba806ae70 0x7ffba806d330 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0035000 tx=0x7ffbc0008030 comp rx=0 tx=0).stop 2023-11-08T16:40:44.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2715639173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbc40c9580 msgr2=0x7ffbc40cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:44.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbc40c9580 0x7ffbc40cd590 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c450 tx=0x7ffbb400c910 comp rx=0 tx=0).stop 2023-11-08T16:40:44.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2715639173 shutdown_connections 2023-11-08T16:40:44.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffba806ae70 0x7ffba806d330 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc415aeb0 0x7ffbc40cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffbc40c9580 0x7ffbc40cd590 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 --2- 172.21.15.81:0/2715639173 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffbc40c90f0 0x7ffbc414ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:44.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2715639173 >> 172.21.15.81:0/2715639173 conn(0x7ffbc40ceaa0 msgr2=0x7ffbc40c9980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:44.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2715639173 shutdown_connections 2023-11-08T16:40:44.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:44.634+0000 7ffbcc3ee640 1 -- 172.21.15.81:0/2715639173 wait complete. 2023-11-08T16:40:44.655 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:40:44.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:44.656 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:40:44.743 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[121968]: fuse finished with error 0 and tester_r 0 2023-11-08T16:40:44.750 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:40:44.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:44.750 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:40:44.929 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:40:44.930 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:40:44.990 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:40:44.991 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:40:44.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:44.992 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-08T16:40:45.048 INFO:teuthology.orchestra.run.smithi093.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-08T16:40:45.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:40:45.050 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:40:45.050 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:40:45.114 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:40:45.114 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:40:45.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:45.115 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-08T16:40:45.167 INFO:teuthology.orchestra.run.smithi081.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-08T16:40:45.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:40:45.169 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:40:45.169 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:40:45.229 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:40:45.230 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:40:45.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:40:45.231 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-08T16:40:45.283 INFO:teuthology.orchestra.run.smithi093.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-08T16:40:45.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:40:45.284 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:40:45.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.503+0000 7fb490a85640 1 Processor -- start 2023-11-08T16:40:45.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.503+0000 7fb490a85640 1 -- start start 2023-11-08T16:40:45.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.504+0000 7fb490a85640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb48c15aac0 0x7fb48c0711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.504+0000 7fb490a85640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb48c0716e0 0x7fb48c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.504+0000 7fb490a85640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48c070470 con 0x7fb48c0cf250 2023-11-08T16:40:45.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.504+0000 7fb490a85640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c06f650 con 0x7fb48c15aac0 2023-11-08T16:40:45.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.505+0000 7fb490a85640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48c0cf6c0 con 0x7fb48c0716e0 2023-11-08T16:40:45.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.505+0000 7fb48b7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb48c15aac0 0x7fb48c0711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:45.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.505+0000 7fb48bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb48c0cf250 0x7fb48c06b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55586/0 (socket says 172.21.15.81:55586) 2023-11-08T16:40:45.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.505+0000 7fb48bfff640 1 -- 172.21.15.81:0/61420388 learned_addr learned my addr 172.21.15.81:0/61420388 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:45.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.505+0000 7fb48b7fe640 1 -- 172.21.15.81:0/61420388 >> v2:172.21.15.93:3300/0 conn(0x7fb48c0716e0 msgr2=0x7fb48c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:45.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.505+0000 7fb48b7fe640 1 --2- 172.21.15.81:0/61420388 >> v2:172.21.15.93:3300/0 conn(0x7fb48c0716e0 0x7fb48c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.505+0000 7fb48b7fe640 1 -- 172.21.15.81:0/61420388 >> v1:172.21.15.81:6789/0 conn(0x7fb48c0cf250 legacy=0x7fb48c06b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:45.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.505+0000 7fb48b7fe640 1 -- 172.21.15.81:0/61420388 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48c06d580 con 0x7fb48c15aac0 2023-11-08T16:40:45.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.506+0000 7fb48b7fe640 1 --2- 172.21.15.81:0/61420388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb48c15aac0 0x7fb48c0711a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fb48c15e790 tx=0x7fb47c02b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3edd91378d5c65f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:45.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.507+0000 7fb48affd640 1 -- 172.21.15.81:0/61420388 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c030710 con 0x7fb48c15aac0 2023-11-08T16:40:45.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.507+0000 7fb48affd640 1 -- 172.21.15.81:0/61420388 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb47c02e050 con 0x7fb48c15aac0 2023-11-08T16:40:45.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.507+0000 7fb48affd640 1 -- 172.21.15.81:0/61420388 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb47c030a90 con 0x7fb48c15aac0 2023-11-08T16:40:45.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.507+0000 7fb490a85640 1 -- 172.21.15.81:0/61420388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb48c15aac0 msgr2=0x7fb48c0711a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:45.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.507+0000 7fb490a85640 1 --2- 172.21.15.81:0/61420388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb48c15aac0 0x7fb48c0711a0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fb48c15e790 tx=0x7fb47c02b710 comp rx=0 tx=0).stop 2023-11-08T16:40:45.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.507+0000 7fb490a85640 1 -- 172.21.15.81:0/61420388 shutdown_connections 2023-11-08T16:40:45.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.507+0000 7fb490a85640 1 --2- 172.21.15.81:0/61420388 >> v2:172.21.15.93:3300/0 conn(0x7fb48c0716e0 0x7fb48c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.507+0000 7fb490a85640 1 --2- 172.21.15.81:0/61420388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb48c15aac0 0x7fb48c0711a0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.507+0000 7fb490a85640 1 -- 172.21.15.81:0/61420388 >> 172.21.15.81:0/61420388 conn(0x7fb48c06a4a0 msgr2=0x7fb48c06a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:45.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.507+0000 7fb490a85640 1 -- 172.21.15.81:0/61420388 shutdown_connections 2023-11-08T16:40:45.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.507+0000 7fb490a85640 1 -- 172.21.15.81:0/61420388 wait complete. 2023-11-08T16:40:45.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.509+0000 7fb490a85640 1 Processor -- start 2023-11-08T16:40:45.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.509+0000 7fb490a85640 1 -- start start 2023-11-08T16:40:45.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.510+0000 7fb490a85640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb48c0716e0 0x7fb48c14f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.510+0000 7fb490a85640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb48c0cf250 0x7fb48c13e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.510+0000 7fb490a85640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb48c15aac0 0x7fb48c13e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.510+0000 7fb490a85640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c15b8b0 con 0x7fb48c15aac0 2023-11-08T16:40:45.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.510+0000 7fb490a85640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c14d800 con 0x7fb48c0cf250 2023-11-08T16:40:45.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.510+0000 7fb490a85640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c14fab0 con 0x7fb48c0716e0 2023-11-08T16:40:45.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.510+0000 7fb48bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb48c15aac0 0x7fb48c13e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:45.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.510+0000 7fb48bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb48c15aac0 0x7fb48c13e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57204/0 (socket says 172.21.15.81:57204) 2023-11-08T16:40:45.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.510+0000 7fb48bfff640 1 -- 172.21.15.81:0/566154122 learned_addr learned my addr 172.21.15.81:0/566154122 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:45.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb48b7fe640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb48c0716e0 0x7fb48c14f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:45.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb48bfff640 1 -- 172.21.15.81:0/566154122 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb48c0716e0 msgr2=0x7fb48c14f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:45.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb482ffd640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb48c0cf250 0x7fb48c13e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:45.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb48bfff640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb48c0716e0 0x7fb48c14f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb48bfff640 1 -- 172.21.15.81:0/566154122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb48c0cf250 msgr2=0x7fb48c13e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:45.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb48bfff640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb48c0cf250 0x7fb48c13e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb48bfff640 1 -- 172.21.15.81:0/566154122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47c009400 con 0x7fb48c15aac0 2023-11-08T16:40:45.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb482ffd640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb48c0cf250 0x7fb48c13e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:45.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb48bfff640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb48c15aac0 0x7fb48c13e930 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fb48400caf0 tx=0x7fb4840020f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:45.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb488ff9640 1 -- 172.21.15.81:0/566154122 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400a760 con 0x7fb48c15aac0 2023-11-08T16:40:45.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb488ff9640 1 -- 172.21.15.81:0/566154122 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48400a8e0 con 0x7fb48c15aac0 2023-11-08T16:40:45.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb488ff9640 1 -- 172.21.15.81:0/566154122 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48400acd0 con 0x7fb48c15aac0 2023-11-08T16:40:45.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.511+0000 7fb490a85640 1 -- 172.21.15.81:0/566154122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c14fd90 con 0x7fb48c15aac0 2023-11-08T16:40:45.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.512+0000 7fb490a85640 1 -- 172.21.15.81:0/566154122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c150350 con 0x7fb48c15aac0 2023-11-08T16:40:45.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.513+0000 7fb490a85640 1 -- 172.21.15.81:0/566154122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c067800 con 0x7fb48c15aac0 2023-11-08T16:40:45.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.514+0000 7fb488ff9640 1 -- 172.21.15.81:0/566154122 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb4840070d0 con 0x7fb48c15aac0 2023-11-08T16:40:45.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.514+0000 7fb488ff9640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 0x7fb46c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.515+0000 7fb488ff9640 1 -- 172.21.15.81:0/566154122 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fb484089160 con 0x7fb48c15aac0 2023-11-08T16:40:45.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.515+0000 7fb48b7fe640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 0x7fb46c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:45.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.516+0000 7fb48b7fe640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 0x7fb46c06d330 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb47c034000 tx=0x7fb47c008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:45.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.519+0000 7fb488ff9640 1 -- 172.21.15.81:0/566154122 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb484053e60 con 0x7fb48c15aac0 2023-11-08T16:40:45.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.746+0000 7fb490a85640 1 -- 172.21.15.81:0/566154122 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb48c15e300 con 0x7fb48c15aac0 2023-11-08T16:40:45.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.747+0000 7fb488ff9640 1 -- 172.21.15.81:0/566154122 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4916 (secure 0 0 0) 0x7fb484018090 con 0x7fb48c15aac0 2023-11-08T16:40:45.750 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:45.750 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":100,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/728323060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":728323060},{"type":"v1","addr":"172.21.15.93:6834","nonce":728323060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3481704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":3481704447},{"type":"v1","addr":"172.21.15.93:6835","nonce":3481704447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8187,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/699802320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":699802320},{"type":"v1","addr":"172.21.15.81:6835","nonce":699802320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:59.048664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/3197735576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3197735576},{"type":"v1","addr":"172.21.15.81:6837","nonce":3197735576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:40:45.750 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 100 2023-11-08T16:40:45.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.749+0000 7fb490a85640 1 -- 172.21.15.81:0/566154122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 msgr2=0x7fb46c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:45.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.749+0000 7fb490a85640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 0x7fb46c06d330 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb47c034000 tx=0x7fb47c008030 comp rx=0 tx=0).stop 2023-11-08T16:40:45.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.749+0000 7fb490a85640 1 -- 172.21.15.81:0/566154122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb48c15aac0 msgr2=0x7fb48c13e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:45.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.749+0000 7fb490a85640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb48c15aac0 0x7fb48c13e930 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fb48400caf0 tx=0x7fb4840020f0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.750+0000 7fb490a85640 1 -- 172.21.15.81:0/566154122 shutdown_connections 2023-11-08T16:40:45.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.750+0000 7fb490a85640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46c06ae70 0x7fb46c06d330 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.750+0000 7fb490a85640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb48c15aac0 0x7fb48c13e930 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.750+0000 7fb490a85640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb48c0cf250 0x7fb48c13e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.750+0000 7fb490a85640 1 --2- 172.21.15.81:0/566154122 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb48c0716e0 0x7fb48c14f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.750+0000 7fb490a85640 1 -- 172.21.15.81:0/566154122 >> 172.21.15.81:0/566154122 conn(0x7fb48c06a4a0 msgr2=0x7fb48c156b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:45.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.750+0000 7fb490a85640 1 -- 172.21.15.81:0/566154122 shutdown_connections 2023-11-08T16:40:45.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.750+0000 7fb490a85640 1 -- 172.21.15.81:0/566154122 wait complete. 2023-11-08T16:40:45.772 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:40:45.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.976+0000 7f2b85475640 1 Processor -- start 2023-11-08T16:40:45.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.976+0000 7f2b85475640 1 -- start start 2023-11-08T16:40:45.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.978+0000 7f2b85475640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2b800adcd0 0x7f2b800ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.978+0000 7f2b85475640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2b80070050 0x7f2b800704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.978+0000 7f2b85475640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b800c9a70 con 0x7f2b800cc5e0 2023-11-08T16:40:45.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.978+0000 7f2b85475640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b800cc3e0 con 0x7f2b800adcd0 2023-11-08T16:40:45.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.978+0000 7f2b85475640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8006d070 con 0x7f2b80070050 2023-11-08T16:40:45.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.978+0000 7f2b7effd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2b800adcd0 0x7f2b800ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:45.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.978+0000 7f2b7effd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2b800adcd0 0x7f2b800ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57216/0 (socket says 172.21.15.81:57216) 2023-11-08T16:40:45.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.978+0000 7f2b7effd640 1 -- 172.21.15.81:0/1955150014 learned_addr learned my addr 172.21.15.81:0/1955150014 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:45.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.979+0000 7f2b7f7fe640 1 --2- 172.21.15.81:0/1955150014 >> v2:172.21.15.93:3300/0 conn(0x7f2b80070050 0x7f2b800704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:45.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.979+0000 7f2b7effd640 1 -- 172.21.15.81:0/1955150014 >> v2:172.21.15.93:3300/0 conn(0x7f2b80070050 msgr2=0x7f2b800704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:45.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.979+0000 7f2b7effd640 1 --2- 172.21.15.81:0/1955150014 >> v2:172.21.15.93:3300/0 conn(0x7f2b80070050 0x7f2b800704e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.979+0000 7f2b7effd640 1 -- 172.21.15.81:0/1955150014 >> v1:172.21.15.93:6789/0 conn(0x7f2b800cc5e0 legacy=0x7f2b800cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:45.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.979+0000 7f2b7effd640 1 -- 172.21.15.81:0/1955150014 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8006d350 con 0x7f2b800adcd0 2023-11-08T16:40:45.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.979+0000 7f2b7f7fe640 1 --2- 172.21.15.81:0/1955150014 >> v2:172.21.15.93:3300/0 conn(0x7f2b80070050 0x7f2b800704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:45.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.980+0000 7f2b7effd640 1 --2- 172.21.15.81:0/1955150014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b800adcd0 0x7f2b800ca1e0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f2b8006d910 tx=0x7f2b7002b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b217b0519ca10d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:45.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.980+0000 7f2b7e7fc640 1 -- 172.21.15.81:0/1955150014 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70031890 con 0x7f2b800adcd0 2023-11-08T16:40:45.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.980+0000 7f2b7e7fc640 1 -- 172.21.15.81:0/1955150014 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b700319f0 con 0x7f2b800adcd0 2023-11-08T16:40:45.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.980+0000 7f2b7e7fc640 1 -- 172.21.15.81:0/1955150014 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70031d80 con 0x7f2b800adcd0 2023-11-08T16:40:45.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.981+0000 7f2b85475640 1 -- 172.21.15.81:0/1955150014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b800adcd0 msgr2=0x7f2b800ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:45.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.981+0000 7f2b85475640 1 --2- 172.21.15.81:0/1955150014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b800adcd0 0x7f2b800ca1e0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f2b8006d910 tx=0x7f2b7002b930 comp rx=0 tx=0).stop 2023-11-08T16:40:45.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.981+0000 7f2b85475640 1 -- 172.21.15.81:0/1955150014 shutdown_connections 2023-11-08T16:40:45.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.981+0000 7f2b85475640 1 --2- 172.21.15.81:0/1955150014 >> v2:172.21.15.93:3300/0 conn(0x7f2b80070050 0x7f2b800704e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.981+0000 7f2b85475640 1 --2- 172.21.15.81:0/1955150014 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b800adcd0 0x7f2b800ca1e0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.981+0000 7f2b85475640 1 -- 172.21.15.81:0/1955150014 >> 172.21.15.81:0/1955150014 conn(0x7f2b800cb490 msgr2=0x7f2b800cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:45.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.981+0000 7f2b85475640 1 -- 172.21.15.81:0/1955150014 shutdown_connections 2023-11-08T16:40:45.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.981+0000 7f2b85475640 1 -- 172.21.15.81:0/1955150014 wait complete. 2023-11-08T16:40:45.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.983+0000 7f2b85475640 1 Processor -- start 2023-11-08T16:40:45.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.983+0000 7f2b85475640 1 -- start start 2023-11-08T16:40:45.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.984+0000 7f2b85475640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b80070050 0x7f2b80156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.984+0000 7f2b85475640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b800adcd0 0x7f2b800b3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.984+0000 7f2b85475640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b800cc5e0 0x7f2b800b3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.984+0000 7f2b85475640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b800c2ca0 con 0x7f2b80070050 2023-11-08T16:40:45.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.984+0000 7f2b85475640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b80156bd0 con 0x7f2b800adcd0 2023-11-08T16:40:45.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.984+0000 7f2b85475640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b801597d0 con 0x7f2b800cc5e0 2023-11-08T16:40:45.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.985+0000 7f2b7effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b80070050 0x7f2b80156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:45.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.985+0000 7f2b7effd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b80070050 0x7f2b80156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57226/0 (socket says 172.21.15.81:57226) 2023-11-08T16:40:45.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.985+0000 7f2b7effd640 1 -- 172.21.15.81:0/2005779565 learned_addr learned my addr 172.21.15.81:0/2005779565 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:45.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.985+0000 7f2b7effd640 1 -- 172.21.15.81:0/2005779565 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b800cc5e0 msgr2=0x7f2b800b3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:45.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.985+0000 7f2b7effd640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b800cc5e0 0x7f2b800b3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.985+0000 7f2b7effd640 1 -- 172.21.15.81:0/2005779565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b800adcd0 msgr2=0x7f2b800b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:45.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.985+0000 7f2b7effd640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b800adcd0 0x7f2b800b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:45.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.985+0000 7f2b7effd640 1 -- 172.21.15.81:0/2005779565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b700094d0 con 0x7f2b80070050 2023-11-08T16:40:45.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.986+0000 7f2b7effd640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b80070050 0x7f2b80156690 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f2b70000c00 tx=0x7f2b700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:45.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.986+0000 7f2b77fff640 1 -- 172.21.15.81:0/2005779565 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7003ad30 con 0x7f2b80070050 2023-11-08T16:40:45.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.986+0000 7f2b77fff640 1 -- 172.21.15.81:0/2005779565 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b7003ae90 con 0x7f2b80070050 2023-11-08T16:40:45.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.986+0000 7f2b77fff640 1 -- 172.21.15.81:0/2005779565 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b70044630 con 0x7f2b80070050 2023-11-08T16:40:45.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.986+0000 7f2b85475640 1 -- 172.21.15.81:0/2005779565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b80157d70 con 0x7f2b80070050 2023-11-08T16:40:45.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.986+0000 7f2b85475640 1 -- 172.21.15.81:0/2005779565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b800c25e0 con 0x7f2b80070050 2023-11-08T16:40:45.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.987+0000 7f2b85475640 1 -- 172.21.15.81:0/2005779565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b80072170 con 0x7f2b80070050 2023-11-08T16:40:45.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.988+0000 7f2b77fff640 1 -- 172.21.15.81:0/2005779565 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b70044790 con 0x7f2b80070050 2023-11-08T16:40:45.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.989+0000 7f2b77fff640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b6006ae70 0x7f2b6006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:45.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.989+0000 7f2b77fff640 1 -- 172.21.15.81:0/2005779565 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f2b700ac2c0 con 0x7f2b80070050 2023-11-08T16:40:45.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.989+0000 7f2b767fc640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b6006ae70 0x7f2b6006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:45.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.990+0000 7f2b767fc640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b6006ae70 0x7f2b6006d330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2b80070a60 tx=0x7f2b680079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:46.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:45.993+0000 7f2b77fff640 1 -- 172.21.15.81:0/2005779565 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b700770e0 con 0x7f2b80070050 2023-11-08T16:40:46.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.186+0000 7f2b85475640 1 -- 172.21.15.81:0/2005779565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b8004ece0 con 0x7f2b80070050 2023-11-08T16:40:46.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.189+0000 7f2b77fff640 1 -- 172.21.15.81:0/2005779565 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18822 (secure 0 0 0) 0x7f2b7007a700 con 0x7f2b80070050 2023-11-08T16:40:46.192 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:46.193 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":114,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:39:57.654547+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:39:07.492834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T16:39:08.634345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:39:19.654916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6836/3163533823":"2023-11-09T16:39:57.018336+0000","172.21.15.81:6836/3988749175":"2023-11-09T16:39:50.607108+0000","172.21.15.81:6837/3163533823":"2023-11-09T16:39:57.018336+0000","172.21.15.93:6835/691341796":"2023-11-09T16:39:48.576045+0000","172.21.15.81:6837/3988749175":"2023-11-09T16:39:50.607108+0000","172.21.15.93:6834/691341796":"2023-11-09T16:39:48.576045+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:40:46.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.192+0000 7f2b85475640 1 -- 172.21.15.81:0/2005779565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b6006ae70 msgr2=0x7f2b6006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.192+0000 7f2b85475640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b6006ae70 0x7f2b6006d330 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2b80070a60 tx=0x7f2b680079b0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.192+0000 7f2b85475640 1 -- 172.21.15.81:0/2005779565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b80070050 msgr2=0x7f2b80156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.192+0000 7f2b85475640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b80070050 0x7f2b80156690 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f2b70000c00 tx=0x7f2b700046c0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.192+0000 7f2b85475640 1 -- 172.21.15.81:0/2005779565 shutdown_connections 2023-11-08T16:40:46.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.192+0000 7f2b85475640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b6006ae70 0x7f2b6006d330 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.192+0000 7f2b85475640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b800cc5e0 0x7f2b800b3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.192+0000 7f2b85475640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b800adcd0 0x7f2b800b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.192+0000 7f2b85475640 1 --2- 172.21.15.81:0/2005779565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b80070050 0x7f2b80156690 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.192+0000 7f2b85475640 1 -- 172.21.15.81:0/2005779565 >> 172.21.15.81:0/2005779565 conn(0x7f2b800cb490 msgr2=0x7f2b800c9040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:46.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.193+0000 7f2b85475640 1 -- 172.21.15.81:0/2005779565 shutdown_connections 2023-11-08T16:40:46.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.193+0000 7f2b85475640 1 -- 172.21.15.81:0/2005779565 wait complete. 2023-11-08T16:40:46.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:39:07.492834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:39:08.634345+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '103', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data_extra', 'create_time': '2023-11-08T16:39:19.654916+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:40:46.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:40:46.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.425+0000 7f8de8026640 1 Processor -- start 2023-11-08T16:40:46.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.425+0000 7f8de8026640 1 -- start start 2023-11-08T16:40:46.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.427+0000 7f8de8026640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8de00ce0d0 0x7f8de00caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.427+0000 7f8de8026640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8de015f280 0x7f8de01516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.427+0000 7f8de8026640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de006fa30 con 0x7f8de015bfc0 2023-11-08T16:40:46.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.427+0000 7f8de8026640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de00cda10 con 0x7f8de015f280 2023-11-08T16:40:46.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.427+0000 7f8de8026640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00ce4b0 con 0x7f8de00ce0d0 2023-11-08T16:40:46.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.427+0000 7f8de65c3640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8de015f280 0x7f8de01516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:46.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.427+0000 7f8de65c3640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8de015f280 0x7f8de01516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57236/0 (socket says 172.21.15.81:57236) 2023-11-08T16:40:46.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.427+0000 7f8de65c3640 1 -- 172.21.15.81:0/468478453 learned_addr learned my addr 172.21.15.81:0/468478453 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:46.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.428+0000 7f8de5dc2640 1 --2- 172.21.15.81:0/468478453 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8de00ce0d0 0x7f8de00caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:46.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.428+0000 7f8de65c3640 1 -- 172.21.15.81:0/468478453 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8de00ce0d0 msgr2=0x7f8de00caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:46.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.428+0000 7f8de65c3640 1 --2- 172.21.15.81:0/468478453 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8de00ce0d0 0x7f8de00caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.428+0000 7f8de65c3640 1 -- 172.21.15.81:0/468478453 >> v1:172.21.15.93:6789/0 conn(0x7f8de015bfc0 legacy=0x7f8de015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.428+0000 7f8de65c3640 1 -- 172.21.15.81:0/468478453 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de006fca0 con 0x7f8de015f280 2023-11-08T16:40:46.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.429+0000 7f8de65c3640 1 --2- 172.21.15.81:0/468478453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8de015f280 0x7f8de01516e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009e50 tx=0x7f8dd802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb78016079a616b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:46.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.429+0000 7f8de4dc0640 1 -- 172.21.15.81:0/468478453 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd802cb00 con 0x7f8de015f280 2023-11-08T16:40:46.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.429+0000 7f8de4dc0640 1 -- 172.21.15.81:0/468478453 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd802cc60 con 0x7f8de015f280 2023-11-08T16:40:46.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.429+0000 7f8de4dc0640 1 -- 172.21.15.81:0/468478453 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd80343f0 con 0x7f8de015f280 2023-11-08T16:40:46.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.430+0000 7f8de8026640 1 -- 172.21.15.81:0/468478453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8de015f280 msgr2=0x7f8de01516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.430+0000 7f8de8026640 1 --2- 172.21.15.81:0/468478453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8de015f280 0x7f8de01516e0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009e50 tx=0x7f8dd802bf60 comp rx=0 tx=0).stop 2023-11-08T16:40:46.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.430+0000 7f8de8026640 1 -- 172.21.15.81:0/468478453 shutdown_connections 2023-11-08T16:40:46.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.430+0000 7f8de8026640 1 --2- 172.21.15.81:0/468478453 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8de015f280 0x7f8de01516e0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.430+0000 7f8de8026640 1 --2- 172.21.15.81:0/468478453 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8de00ce0d0 0x7f8de00caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.430+0000 7f8de8026640 1 -- 172.21.15.81:0/468478453 >> 172.21.15.81:0/468478453 conn(0x7f8de00cb580 msgr2=0x7f8de00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:46.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.431+0000 7f8de8026640 1 -- 172.21.15.81:0/468478453 shutdown_connections 2023-11-08T16:40:46.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.431+0000 7f8de8026640 1 -- 172.21.15.81:0/468478453 wait complete. 2023-11-08T16:40:46.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.432+0000 7f8de8026640 1 Processor -- start 2023-11-08T16:40:46.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.432+0000 7f8de8026640 1 -- start start 2023-11-08T16:40:46.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.433+0000 7f8de8026640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8de00ce0d0 0x7f8de00c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.433+0000 7f8de8026640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8de015bfc0 0x7f8de00c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.433+0000 7f8de8026640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8de015f280 0x7f8de0149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.433+0000 7f8de8026640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00c55f0 con 0x7f8de015f280 2023-11-08T16:40:46.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.433+0000 7f8de8026640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00b14f0 con 0x7f8de00ce0d0 2023-11-08T16:40:46.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.433+0000 7f8de8026640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de00b17b0 con 0x7f8de015bfc0 2023-11-08T16:40:46.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.434+0000 7f8de65c3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8de015f280 0x7f8de0149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:46.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.434+0000 7f8de65c3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8de015f280 0x7f8de0149a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57252/0 (socket says 172.21.15.81:57252) 2023-11-08T16:40:46.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.434+0000 7f8de65c3640 1 -- 172.21.15.81:0/185576741 learned_addr learned my addr 172.21.15.81:0/185576741 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:46.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.434+0000 7f8de5dc2640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8de00ce0d0 0x7f8de00c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:46.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.434+0000 7f8de55c1640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8de015bfc0 0x7f8de00c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:46.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.434+0000 7f8de65c3640 1 -- 172.21.15.81:0/185576741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8de015bfc0 msgr2=0x7f8de00c9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.434+0000 7f8de65c3640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8de015bfc0 0x7f8de00c9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.434+0000 7f8de65c3640 1 -- 172.21.15.81:0/185576741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8de00ce0d0 msgr2=0x7f8de00c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.434+0000 7f8de65c3640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8de00ce0d0 0x7f8de00c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.434+0000 7f8de65c3640 1 -- 172.21.15.81:0/185576741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd8009b30 con 0x7f8de015f280 2023-11-08T16:40:46.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.435+0000 7f8de65c3640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8de015f280 0x7f8de0149a90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009e50 tx=0x7f8dd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:46.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.435+0000 7f8dd67fc640 1 -- 172.21.15.81:0/185576741 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8040670 con 0x7f8de015f280 2023-11-08T16:40:46.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.435+0000 7f8dd67fc640 1 -- 172.21.15.81:0/185576741 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd80407d0 con 0x7f8de015f280 2023-11-08T16:40:46.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.435+0000 7f8de8026640 1 -- 172.21.15.81:0/185576741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de00b1a90 con 0x7f8de015f280 2023-11-08T16:40:46.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.435+0000 7f8dd67fc640 1 -- 172.21.15.81:0/185576741 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8040b80 con 0x7f8de015f280 2023-11-08T16:40:46.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.435+0000 7f8de8026640 1 -- 172.21.15.81:0/185576741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de00b2050 con 0x7f8de015f280 2023-11-08T16:40:46.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.436+0000 7f8de8026640 1 -- 172.21.15.81:0/185576741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de004ece0 con 0x7f8de015f280 2023-11-08T16:40:46.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.437+0000 7f8dd67fc640 1 -- 172.21.15.81:0/185576741 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8dd8040ce0 con 0x7f8de015f280 2023-11-08T16:40:46.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.438+0000 7f8dd67fc640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8dc406ae70 0x7f8dc406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.439+0000 7f8dd67fc640 1 -- 172.21.15.81:0/185576741 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f8dd80aca10 con 0x7f8de015f280 2023-11-08T16:40:46.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.439+0000 7f8de5dc2640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8dc406ae70 0x7f8dc406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:46.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.439+0000 7f8de5dc2640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8dc406ae70 0x7f8dc406d330 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc006fd0 tx=0x7f8ddc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:46.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.443+0000 7f8dd67fc640 1 -- 172.21.15.81:0/185576741 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8dd8077710 con 0x7f8de015f280 2023-11-08T16:40:46.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.633+0000 7f8de8026640 1 -- 172.21.15.81:0/185576741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8de00ce6b0 con 0x7f8de015f280 2023-11-08T16:40:46.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.634+0000 7f8dd67fc640 1 -- 172.21.15.81:0/185576741 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v114) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8dd8009510 con 0x7f8de015f280 2023-11-08T16:40:46.637 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:40:46.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.636+0000 7f8de8026640 1 -- 172.21.15.81:0/185576741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8dc406ae70 msgr2=0x7f8dc406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.636+0000 7f8de8026640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8dc406ae70 0x7f8dc406d330 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc006fd0 tx=0x7f8ddc008040 comp rx=0 tx=0).stop 2023-11-08T16:40:46.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.636+0000 7f8de8026640 1 -- 172.21.15.81:0/185576741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8de015f280 msgr2=0x7f8de0149a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.636+0000 7f8de8026640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8de015f280 0x7f8de0149a90 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8009e50 tx=0x7f8dd8004a40 comp rx=0 tx=0).stop 2023-11-08T16:40:46.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.636+0000 7f8de8026640 1 -- 172.21.15.81:0/185576741 shutdown_connections 2023-11-08T16:40:46.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.636+0000 7f8de8026640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8dc406ae70 0x7f8dc406d330 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.636+0000 7f8de8026640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8de015f280 0x7f8de0149a90 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.636+0000 7f8de8026640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8de015bfc0 0x7f8de00c9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.636+0000 7f8de8026640 1 --2- 172.21.15.81:0/185576741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8de00ce0d0 0x7f8de00c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.636+0000 7f8de8026640 1 -- 172.21.15.81:0/185576741 >> 172.21.15.81:0/185576741 conn(0x7f8de00cb580 msgr2=0x7f8de006c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:46.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.637+0000 7f8de8026640 1 -- 172.21.15.81:0/185576741 shutdown_connections 2023-11-08T16:40:46.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.637+0000 7f8de8026640 1 -- 172.21.15.81:0/185576741 wait complete. 2023-11-08T16:40:46.656 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:40:46.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.870+0000 7fbb1bfff640 1 Processor -- start 2023-11-08T16:40:46.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.870+0000 7fbb1bfff640 1 -- start start 2023-11-08T16:40:46.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.872+0000 7fbb1bfff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbb1c15cf30 0x7fbb1c0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.872+0000 7fbb1bfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb1c0cfd90 0x7fbb1c0cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.872+0000 7fbb1bfff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1c0ce0b0 con 0x7fbb1c15d310 2023-11-08T16:40:46.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.872+0000 7fbb1bfff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cdb70 con 0x7fbb1c15cf30 2023-11-08T16:40:46.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.872+0000 7fbb1bfff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0ce390 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.873+0000 7fbb1b7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb1c0cfd90 0x7fbb1c0cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:46.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.873+0000 7fbb13fff640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fbb1c15d310 0x7fbb1c0cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50634/0 (socket says 172.21.15.81:50634) 2023-11-08T16:40:46.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.873+0000 7fbb13fff640 1 -- 172.21.15.81:0/3577146190 learned_addr learned my addr 172.21.15.81:0/3577146190 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:46.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.873+0000 7fbb1b7fe640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb1c0cfd90 0x7fbb1c0cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:53918/0 (socket says 172.21.15.81:53918) 2023-11-08T16:40:46.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.873+0000 7fbb1affd640 1 --2- 172.21.15.81:0/3577146190 >> v2:172.21.15.93:3300/0 conn(0x7fbb1c15cf30 0x7fbb1c0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:46.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.874+0000 7fbb1b7fe640 1 -- 172.21.15.81:0/3577146190 >> v2:172.21.15.93:3300/0 conn(0x7fbb1c15cf30 msgr2=0x7fbb1c0d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:46.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.874+0000 7fbb1b7fe640 1 --2- 172.21.15.81:0/3577146190 >> v2:172.21.15.93:3300/0 conn(0x7fbb1c15cf30 0x7fbb1c0d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.874+0000 7fbb1b7fe640 1 -- 172.21.15.81:0/3577146190 >> v1:172.21.15.93:6789/0 conn(0x7fbb1c15d310 legacy=0x7fbb1c0cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.874+0000 7fbb1b7fe640 1 -- 172.21.15.81:0/3577146190 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c0d1be0 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.874+0000 7fbb1affd640 1 --2- 172.21.15.81:0/3577146190 >> v2:172.21.15.93:3300/0 conn(0x7fbb1c15cf30 0x7fbb1c0d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:40:46.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.875+0000 7fbb1b7fe640 1 --2- 172.21.15.81:0/3577146190 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb1c0cfd90 0x7fbb1c0cd630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009dc0 tx=0x7fbb1402be90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a20e266895c14720 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:46.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.876+0000 7fbb1a7fc640 1 -- 172.21.15.81:0/3577146190 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb1402cc40 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.876+0000 7fbb1a7fc640 1 -- 172.21.15.81:0/3577146190 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb1402cda0 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.876+0000 7fbb1a7fc640 1 -- 172.21.15.81:0/3577146190 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14034660 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.877+0000 7fbb1bfff640 1 -- 172.21.15.81:0/3577146190 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb1c0cfd90 msgr2=0x7fbb1c0cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.877+0000 7fbb1bfff640 1 --2- 172.21.15.81:0/3577146190 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb1c0cfd90 0x7fbb1c0cd630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009dc0 tx=0x7fbb1402be90 comp rx=0 tx=0).stop 2023-11-08T16:40:46.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.877+0000 7fbb1bfff640 1 -- 172.21.15.81:0/3577146190 shutdown_connections 2023-11-08T16:40:46.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.877+0000 7fbb1bfff640 1 --2- 172.21.15.81:0/3577146190 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb1c0cfd90 0x7fbb1c0cd630 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.877+0000 7fbb1bfff640 1 --2- 172.21.15.81:0/3577146190 >> v2:172.21.15.93:3300/0 conn(0x7fbb1c15cf30 0x7fbb1c0d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.877+0000 7fbb1bfff640 1 -- 172.21.15.81:0/3577146190 >> 172.21.15.81:0/3577146190 conn(0x7fbb1c06c040 msgr2=0x7fbb1c06c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:46.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.877+0000 7fbb1bfff640 1 -- 172.21.15.81:0/3577146190 shutdown_connections 2023-11-08T16:40:46.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.877+0000 7fbb1bfff640 1 -- 172.21.15.81:0/3577146190 wait complete. 2023-11-08T16:40:46.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.879+0000 7fbb1bfff640 1 Processor -- start 2023-11-08T16:40:46.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.879+0000 7fbb1bfff640 1 -- start start 2023-11-08T16:40:46.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb1c0cfd90 0x7fbb1c0b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1bfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb1c15cf30 0x7fbb1c0c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1bfff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c0c67e0 0x7fbb1c14a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1bfff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0c7a20 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1bfff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0b3b50 con 0x7fbb1c0c67e0 2023-11-08T16:40:46.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1bfff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0c8380 con 0x7fbb1c15cf30 2023-11-08T16:40:46.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb1c0cfd90 0x7fbb1c0b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:46.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb1c0cfd90 0x7fbb1c0b3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57262/0 (socket says 172.21.15.81:57262) 2023-11-08T16:40:46.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1affd640 1 -- 172.21.15.81:0/1862866644 learned_addr learned my addr 172.21.15.81:0/1862866644 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:46.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1b7fe640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c0c67e0 0x7fbb1c14a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:46.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1affd640 1 -- 172.21.15.81:0/1862866644 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb1c15cf30 msgr2=0x7fbb1c0c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1affd640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb1c15cf30 0x7fbb1c0c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1affd640 1 -- 172.21.15.81:0/1862866644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c0c67e0 msgr2=0x7fbb1c14a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:46.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1affd640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c0c67e0 0x7fbb1c14a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:46.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.880+0000 7fbb1affd640 1 -- 172.21.15.81:0/1862866644 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c009400 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.881+0000 7fbb1affd640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb1c0cfd90 0x7fbb1c0b3610 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00ec40 tx=0x7fbb0c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:46.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.881+0000 7fbb137fe640 1 -- 172.21.15.81:0/1862866644 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c009760 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.881+0000 7fbb137fe640 1 -- 172.21.15.81:0/1862866644 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c00cc30 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.881+0000 7fbb137fe640 1 -- 172.21.15.81:0/1862866644 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c013470 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.881+0000 7fbb1bfff640 1 -- 172.21.15.81:0/1862866644 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb14009a70 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.881+0000 7fbb1bfff640 1 -- 172.21.15.81:0/1862866644 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c0c7080 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.882+0000 7fbb1bfff640 1 -- 172.21.15.81:0/1862866644 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c04ece0 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.883+0000 7fbb137fe640 1 -- 172.21.15.81:0/1862866644 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb0c0135d0 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.884+0000 7fbb137fe640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbafc06ae70 0x7fbafc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:46.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.884+0000 7fbb137fe640 1 -- 172.21.15.81:0/1862866644 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fbb0c089ba0 con 0x7fbb1c0cfd90 2023-11-08T16:40:46.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.884+0000 7fbb13fff640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbafc06ae70 0x7fbafc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:46.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.885+0000 7fbb13fff640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbafc06ae70 0x7fbafc06d330 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c14d670 tx=0x7fbb04007a40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:46.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:46.888+0000 7fbb137fe640 1 -- 172.21.15.81:0/1862866644 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb0c054940 con 0x7fbb1c0cfd90 2023-11-08T16:40:47.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.080+0000 7fbb1bfff640 1 -- 172.21.15.81:0/1862866644 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbb1c0d0ce0 con 0x7fbb1c0cfd90 2023-11-08T16:40:47.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.081+0000 7fbb137fe640 1 -- 172.21.15.81:0/1862866644 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v114) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbb0c017020 con 0x7fbb1c0cfd90 2023-11-08T16:40:47.084 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:40:47.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 -- 172.21.15.81:0/1862866644 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbafc06ae70 msgr2=0x7fbafc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbafc06ae70 0x7fbafc06d330 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c14d670 tx=0x7fbb04007a40 comp rx=0 tx=0).stop 2023-11-08T16:40:47.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 -- 172.21.15.81:0/1862866644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb1c0cfd90 msgr2=0x7fbb1c0b3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb1c0cfd90 0x7fbb1c0b3610 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00ec40 tx=0x7fbb0c00c520 comp rx=0 tx=0).stop 2023-11-08T16:40:47.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 -- 172.21.15.81:0/1862866644 shutdown_connections 2023-11-08T16:40:47.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbafc06ae70 0x7fbafc06d330 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb1c0c67e0 0x7fbb1c14a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb1c15cf30 0x7fbb1c0c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 --2- 172.21.15.81:0/1862866644 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb1c0cfd90 0x7fbb1c0b3610 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 -- 172.21.15.81:0/1862866644 >> 172.21.15.81:0/1862866644 conn(0x7fbb1c06c040 msgr2=0x7fbb1c0b7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:47.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 -- 172.21.15.81:0/1862866644 shutdown_connections 2023-11-08T16:40:47.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.084+0000 7fbb1bfff640 1 -- 172.21.15.81:0/1862866644 wait complete. 2023-11-08T16:40:47.105 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:40:47.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.320+0000 7f81e9eb4640 1 Processor -- start 2023-11-08T16:40:47.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.321+0000 7f81e9eb4640 1 -- start start 2023-11-08T16:40:47.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.322+0000 7f81e9eb4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f81e415cf30 0x7f81e40d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.322+0000 7f81e9eb4640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f81e415d310 0x7f81e40cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.322+0000 7f81e9eb4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e40ce0b0 con 0x7f81e40cf680 2023-11-08T16:40:47.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.322+0000 7f81e9eb4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e40cdb70 con 0x7f81e415cf30 2023-11-08T16:40:47.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.322+0000 7f81e9eb4640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e40ce390 con 0x7f81e415d310 2023-11-08T16:40:47.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.322+0000 7f81e96b3640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f81e40cf680 0x7f81e40b5f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55590/0 (socket says 172.21.15.81:55590) 2023-11-08T16:40:47.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.322+0000 7f81e96b3640 1 -- 172.21.15.81:0/1553070001 learned_addr learned my addr 172.21.15.81:0/1553070001 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:47.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.322+0000 7f81e8eb2640 1 --2- 172.21.15.81:0/1553070001 >> v2:172.21.15.81:3300/0 conn(0x7f81e415cf30 0x7f81e40d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:47.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.323+0000 7f81e3fff640 1 --2- 172.21.15.81:0/1553070001 >> v2:172.21.15.93:3300/0 conn(0x7f81e415d310 0x7f81e40cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:47.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.323+0000 7f81e8eb2640 1 -- 172.21.15.81:0/1553070001 >> v2:172.21.15.93:3300/0 conn(0x7f81e415d310 msgr2=0x7f81e40cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:47.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.323+0000 7f81e8eb2640 1 --2- 172.21.15.81:0/1553070001 >> v2:172.21.15.93:3300/0 conn(0x7f81e415d310 0x7f81e40cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.323+0000 7f81e8eb2640 1 -- 172.21.15.81:0/1553070001 >> v1:172.21.15.81:6789/0 conn(0x7f81e40cf680 legacy=0x7f81e40b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.323+0000 7f81e8eb2640 1 -- 172.21.15.81:0/1553070001 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e40d1be0 con 0x7f81e415cf30 2023-11-08T16:40:47.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.324+0000 7f81e8eb2640 1 --2- 172.21.15.81:0/1553070001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e415cf30 0x7f81e40d26c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0097a0 tx=0x7f81cc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=941a03f01f8b56d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:47.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.324+0000 7f81e37fe640 1 -- 172.21.15.81:0/1553070001 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc031830 con 0x7f81e415cf30 2023-11-08T16:40:47.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.324+0000 7f81e37fe640 1 -- 172.21.15.81:0/1553070001 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc031990 con 0x7f81e415cf30 2023-11-08T16:40:47.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.324+0000 7f81e37fe640 1 -- 172.21.15.81:0/1553070001 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc031d20 con 0x7f81e415cf30 2023-11-08T16:40:47.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.325+0000 7f81e9eb4640 1 -- 172.21.15.81:0/1553070001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e415cf30 msgr2=0x7f81e40d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.325+0000 7f81e9eb4640 1 --2- 172.21.15.81:0/1553070001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e415cf30 0x7f81e40d26c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0097a0 tx=0x7f81cc02b8d0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.325+0000 7f81e9eb4640 1 -- 172.21.15.81:0/1553070001 shutdown_connections 2023-11-08T16:40:47.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.325+0000 7f81e9eb4640 1 --2- 172.21.15.81:0/1553070001 >> v2:172.21.15.93:3300/0 conn(0x7f81e415d310 0x7f81e40cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.325+0000 7f81e9eb4640 1 --2- 172.21.15.81:0/1553070001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e415cf30 0x7f81e40d26c0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.325+0000 7f81e9eb4640 1 -- 172.21.15.81:0/1553070001 >> 172.21.15.81:0/1553070001 conn(0x7f81e406c040 msgr2=0x7f81e406c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:47.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.325+0000 7f81e9eb4640 1 -- 172.21.15.81:0/1553070001 shutdown_connections 2023-11-08T16:40:47.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.325+0000 7f81e9eb4640 1 -- 172.21.15.81:0/1553070001 wait complete. 2023-11-08T16:40:47.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.327+0000 7f81e9eb4640 1 Processor -- start 2023-11-08T16:40:47.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.327+0000 7f81e9eb4640 1 -- start start 2023-11-08T16:40:47.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.328+0000 7f81e9eb4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f81e40cf680 0x7f81e40c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.328+0000 7f81e9eb4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e415cf30 0x7f81e40b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.328+0000 7f81e9eb4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81e40c8f70 0x7f81e415f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.328+0000 7f81e9eb4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40b5240 con 0x7f81e415cf30 2023-11-08T16:40:47.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.328+0000 7f81e9eb4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40c9420 con 0x7f81e40c8f70 2023-11-08T16:40:47.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.328+0000 7f81e9eb4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e40c6980 con 0x7f81e40cf680 2023-11-08T16:40:47.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.328+0000 7f81e3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e415cf30 0x7f81e40b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:47.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.328+0000 7f81e3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e415cf30 0x7f81e40b4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57288/0 (socket says 172.21.15.81:57288) 2023-11-08T16:40:47.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.328+0000 7f81e3fff640 1 -- 172.21.15.81:0/2497449927 learned_addr learned my addr 172.21.15.81:0/2497449927 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:47.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e96b3640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81e40c8f70 0x7f81e415f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:47.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e3fff640 1 -- 172.21.15.81:0/2497449927 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f81e40cf680 msgr2=0x7f81e40c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e3fff640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f81e40cf680 0x7f81e40c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e3fff640 1 -- 172.21.15.81:0/2497449927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81e40c8f70 msgr2=0x7f81e415f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e3fff640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81e40c8f70 0x7f81e415f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e3fff640 1 -- 172.21.15.81:0/2497449927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc009400 con 0x7f81e415cf30 2023-11-08T16:40:47.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e96b3640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81e40c8f70 0x7f81e415f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:47.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e3fff640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e415cf30 0x7f81e40b4b30 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f81d4007de0 tx=0x7f81d400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:47.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e17fa640 1 -- 172.21.15.81:0/2497449927 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4014620 con 0x7f81e415cf30 2023-11-08T16:40:47.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e17fa640 1 -- 172.21.15.81:0/2497449927 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d400ee90 con 0x7f81e415cf30 2023-11-08T16:40:47.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e17fa640 1 -- 172.21.15.81:0/2497449927 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d40149a0 con 0x7f81e415cf30 2023-11-08T16:40:47.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e9eb4640 1 -- 172.21.15.81:0/2497449927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e40c6c60 con 0x7f81e415cf30 2023-11-08T16:40:47.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.329+0000 7f81e9eb4640 1 -- 172.21.15.81:0/2497449927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e40c7220 con 0x7f81e415cf30 2023-11-08T16:40:47.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.331+0000 7f81e9eb4640 1 -- 172.21.15.81:0/2497449927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e404ece0 con 0x7f81e415cf30 2023-11-08T16:40:47.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.332+0000 7f81e17fa640 1 -- 172.21.15.81:0/2497449927 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f81d4014b00 con 0x7f81e415cf30 2023-11-08T16:40:47.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.332+0000 7f81e17fa640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f81c406aea0 0x7f81c406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.333+0000 7f81e17fa640 1 -- 172.21.15.81:0/2497449927 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f81d408a330 con 0x7f81e415cf30 2023-11-08T16:40:47.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.333+0000 7f81e8eb2640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f81c406aea0 0x7f81c406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:47.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.333+0000 7f81e8eb2640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f81c406aea0 0x7f81c406d360 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f81cc036000 tx=0x7f81cc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:47.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.337+0000 7f81e17fa640 1 -- 172.21.15.81:0/2497449927 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f81d4055040 con 0x7f81e415cf30 2023-11-08T16:40:47.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.525+0000 7f81e9eb4640 1 -- 172.21.15.81:0/2497449927 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81e40ced30 con 0x7f81e415cf30 2023-11-08T16:40:47.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.525+0000 7f81e17fa640 1 -- 172.21.15.81:0/2497449927 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v114) v1 ==== 100+0+11 (secure 0 0 0) 0x7f81d4014db0 con 0x7f81e415cf30 2023-11-08T16:40:47.528 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:40:47.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 -- 172.21.15.81:0/2497449927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f81c406aea0 msgr2=0x7f81c406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f81c406aea0 0x7f81c406d360 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f81cc036000 tx=0x7f81cc030000 comp rx=0 tx=0).stop 2023-11-08T16:40:47.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 -- 172.21.15.81:0/2497449927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e415cf30 msgr2=0x7f81e40b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e415cf30 0x7f81e40b4b30 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f81d4007de0 tx=0x7f81d400e4b0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 -- 172.21.15.81:0/2497449927 shutdown_connections 2023-11-08T16:40:47.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f81c406aea0 0x7f81c406d360 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81e40c8f70 0x7f81e415f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f81e415cf30 0x7f81e40b4b30 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 --2- 172.21.15.81:0/2497449927 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f81e40cf680 0x7f81e40c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 -- 172.21.15.81:0/2497449927 >> 172.21.15.81:0/2497449927 conn(0x7f81e406c040 msgr2=0x7f81e40730a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:47.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 -- 172.21.15.81:0/2497449927 shutdown_connections 2023-11-08T16:40:47.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.528+0000 7f81e9eb4640 1 -- 172.21.15.81:0/2497449927 wait complete. 2023-11-08T16:40:47.548 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_extra pg_num 2023-11-08T16:40:47.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.767+0000 7f32cea13640 1 Processor -- start 2023-11-08T16:40:47.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.767+0000 7f32cea13640 1 -- start start 2023-11-08T16:40:47.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.769+0000 7f32cea13640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f32c81612f0 0x7f32c806c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.769+0000 7f32cea13640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f32c81616d0 0x7f32c80c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.769+0000 7f32cea13640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32c815d680 con 0x7f32c806cc70 2023-11-08T16:40:47.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.769+0000 7f32cea13640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c815d4a0 con 0x7f32c81616d0 2023-11-08T16:40:47.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.769+0000 7f32cea13640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32c815d960 con 0x7f32c81612f0 2023-11-08T16:40:47.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.769+0000 7f32cd210640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f32c81616d0 0x7f32c80c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:47.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.769+0000 7f32cd210640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f32c81616d0 0x7f32c80c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57300/0 (socket says 172.21.15.81:57300) 2023-11-08T16:40:47.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.769+0000 7f32cd210640 1 -- 172.21.15.81:0/2501498914 learned_addr learned my addr 172.21.15.81:0/2501498914 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:47.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.770+0000 7f32cda11640 1 --2- 172.21.15.81:0/2501498914 >> v2:172.21.15.93:3300/0 conn(0x7f32c81612f0 0x7f32c806c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:47.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.770+0000 7f32cd210640 1 -- 172.21.15.81:0/2501498914 >> v2:172.21.15.93:3300/0 conn(0x7f32c81612f0 msgr2=0x7f32c806c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:47.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.770+0000 7f32cd210640 1 --2- 172.21.15.81:0/2501498914 >> v2:172.21.15.93:3300/0 conn(0x7f32c81612f0 0x7f32c806c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.770+0000 7f32cd210640 1 -- 172.21.15.81:0/2501498914 >> v1:172.21.15.93:6789/0 conn(0x7f32c806cc70 legacy=0x7f32c80ca450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:47.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.770+0000 7f32cd210640 1 -- 172.21.15.81:0/2501498914 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c815dc40 con 0x7f32c81616d0 2023-11-08T16:40:47.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.771+0000 7f32cd210640 1 --2- 172.21.15.81:0/2501498914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32c81616d0 0x7f32c80c9f10 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f32c80cf8c0 tx=0x7f32b802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80312c9c3839655 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:47.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.771+0000 7f32cca0f640 1 -- 172.21.15.81:0/2501498914 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8004230 con 0x7f32c81616d0 2023-11-08T16:40:47.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.771+0000 7f32cca0f640 1 -- 172.21.15.81:0/2501498914 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b8004390 con 0x7f32c81616d0 2023-11-08T16:40:47.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.771+0000 7f32cca0f640 1 -- 172.21.15.81:0/2501498914 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b8003810 con 0x7f32c81616d0 2023-11-08T16:40:47.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.772+0000 7f32cea13640 1 -- 172.21.15.81:0/2501498914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32c81616d0 msgr2=0x7f32c80c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.772+0000 7f32cea13640 1 --2- 172.21.15.81:0/2501498914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32c81616d0 0x7f32c80c9f10 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f32c80cf8c0 tx=0x7f32b802b8f0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.772+0000 7f32cea13640 1 -- 172.21.15.81:0/2501498914 shutdown_connections 2023-11-08T16:40:47.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.772+0000 7f32cea13640 1 --2- 172.21.15.81:0/2501498914 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32c81616d0 0x7f32c80c9f10 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.772+0000 7f32cea13640 1 --2- 172.21.15.81:0/2501498914 >> v2:172.21.15.93:3300/0 conn(0x7f32c81612f0 0x7f32c806c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.772+0000 7f32cea13640 1 -- 172.21.15.81:0/2501498914 >> 172.21.15.81:0/2501498914 conn(0x7f32c80ce8d0 msgr2=0x7f32c80cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:47.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.772+0000 7f32cea13640 1 -- 172.21.15.81:0/2501498914 shutdown_connections 2023-11-08T16:40:47.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.772+0000 7f32cea13640 1 -- 172.21.15.81:0/2501498914 wait complete. 2023-11-08T16:40:47.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.774+0000 7f32cea13640 1 Processor -- start 2023-11-08T16:40:47.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.774+0000 7f32cea13640 1 -- start start 2023-11-08T16:40:47.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.775+0000 7f32cea13640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c806cc70 0x7f32c81f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.775+0000 7f32cea13640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32c81612f0 0x7f32c815f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.775+0000 7f32cea13640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32c81f5d60 0x7f32c815fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.775+0000 7f32cea13640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81f62b0 con 0x7f32c81f5d60 2023-11-08T16:40:47.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.775+0000 7f32cea13640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81f6590 con 0x7f32c806cc70 2023-11-08T16:40:47.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.775+0000 7f32cea13640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32c81f6870 con 0x7f32c81612f0 2023-11-08T16:40:47.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.776+0000 7f32ce212640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32c81f5d60 0x7f32c815fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:47.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.776+0000 7f32ce212640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32c81f5d60 0x7f32c815fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57314/0 (socket says 172.21.15.81:57314) 2023-11-08T16:40:47.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.776+0000 7f32ce212640 1 -- 172.21.15.81:0/3375494630 learned_addr learned my addr 172.21.15.81:0/3375494630 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:47.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.776+0000 7f32cda11640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c806cc70 0x7f32c81f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:47.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.776+0000 7f32cd210640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32c81612f0 0x7f32c815f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:47.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.776+0000 7f32ce212640 1 -- 172.21.15.81:0/3375494630 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32c81612f0 msgr2=0x7f32c815f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.776+0000 7f32ce212640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32c81612f0 0x7f32c815f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.776+0000 7f32ce212640 1 -- 172.21.15.81:0/3375494630 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c806cc70 msgr2=0x7f32c81f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.776+0000 7f32ce212640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c806cc70 0x7f32c81f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.776+0000 7f32ce212640 1 -- 172.21.15.81:0/3375494630 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b8009460 con 0x7f32c81f5d60 2023-11-08T16:40:47.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.776+0000 7f32ce212640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32c81f5d60 0x7f32c815fbc0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c780 tx=0x7f32c000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:47.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.777+0000 7f32be7fc640 1 -- 172.21.15.81:0/3375494630 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0004b60 con 0x7f32c81f5d60 2023-11-08T16:40:47.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.777+0000 7f32be7fc640 1 -- 172.21.15.81:0/3375494630 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c000ce90 con 0x7f32c81f5d60 2023-11-08T16:40:47.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.777+0000 7f32be7fc640 1 -- 172.21.15.81:0/3375494630 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c000a850 con 0x7f32c81f5d60 2023-11-08T16:40:47.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.777+0000 7f32cea13640 1 -- 172.21.15.81:0/3375494630 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c81f6b50 con 0x7f32c81f5d60 2023-11-08T16:40:47.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.777+0000 7f32cea13640 1 -- 172.21.15.81:0/3375494630 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c81f2580 con 0x7f32c81f5d60 2023-11-08T16:40:47.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.778+0000 7f32cea13640 1 -- 172.21.15.81:0/3375494630 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32c8156bc0 con 0x7f32c81f5d60 2023-11-08T16:40:47.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.779+0000 7f32be7fc640 1 -- 172.21.15.81:0/3375494630 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32c000a9b0 con 0x7f32c81f5d60 2023-11-08T16:40:47.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.780+0000 7f32be7fc640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ac06aea0 0x7f32ac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:47.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.780+0000 7f32be7fc640 1 -- 172.21.15.81:0/3375494630 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f32c0089ae0 con 0x7f32c81f5d60 2023-11-08T16:40:47.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.780+0000 7f32cda11640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ac06aea0 0x7f32ac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:47.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.781+0000 7f32cda11640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ac06aea0 0x7f32ac06d360 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f32c400a970 tx=0x7f32c4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:47.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.785+0000 7f32be7fc640 1 -- 172.21.15.81:0/3375494630 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32c8156bc0 con 0x7f32c81f5d60 2023-11-08T16:40:47.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.976+0000 7f32cea13640 1 -- 172.21.15.81:0/3375494630 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"} v 0) v1 -- 0x7f32c8156bc0 con 0x7f32c81f5d60 2023-11-08T16:40:47.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.976+0000 7f32be7fc640 1 -- 172.21.15.81:0/3375494630 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_extra", "var": "pg_num"}]=0 v114) v1 ==== 106+0+11 (secure 0 0 0) 0x7f32c8156bc0 con 0x7f32c81f5d60 2023-11-08T16:40:47.979 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:40:47.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 -- 172.21.15.81:0/3375494630 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ac06aea0 msgr2=0x7f32ac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ac06aea0 0x7f32ac06d360 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f32c400a970 tx=0x7f32c4008040 comp rx=0 tx=0).stop 2023-11-08T16:40:47.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 -- 172.21.15.81:0/3375494630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32c81f5d60 msgr2=0x7f32c815fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:47.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32c81f5d60 0x7f32c815fbc0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c780 tx=0x7f32c000cc40 comp rx=0 tx=0).stop 2023-11-08T16:40:47.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 -- 172.21.15.81:0/3375494630 shutdown_connections 2023-11-08T16:40:47.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ac06aea0 0x7f32ac06d360 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f32c81f5d60 0x7f32c815fbc0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f32c81612f0 0x7f32c815f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 --2- 172.21.15.81:0/3375494630 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32c806cc70 0x7f32c81f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:47.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 -- 172.21.15.81:0/3375494630 >> 172.21.15.81:0/3375494630 conn(0x7f32c80ce8d0 msgr2=0x7f32c80cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:47.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 -- 172.21.15.81:0/3375494630 shutdown_connections 2023-11-08T16:40:47.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:47.979+0000 7f32cea13640 1 -- 172.21.15.81:0/3375494630 wait complete. 2023-11-08T16:40:48.002 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:40:48.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.211+0000 7fecb1716640 1 Processor -- start 2023-11-08T16:40:48.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.212+0000 7fecb1716640 1 -- start start 2023-11-08T16:40:48.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.213+0000 7fecb1716640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fecac0ca480 0x7fecac15ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:48.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.213+0000 7fecb1716640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecac06b630 con 0x7fecac15ea60 2023-11-08T16:40:48.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.213+0000 7fecb1716640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecac0ca910 con 0x7fecac0ade50 2023-11-08T16:40:48.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.213+0000 7fecb1716640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecac0b1860 con 0x7fecac0ca480 2023-11-08T16:40:48.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.214+0000 7fecab7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fecac15ea60 0x7fecac15eef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55604/0 (socket says 172.21.15.81:55604) 2023-11-08T16:40:48.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.214+0000 7fecab7fe640 1 -- 172.21.15.81:0/4089851239 learned_addr learned my addr 172.21.15.81:0/4089851239 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:48.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.214+0000 7feca27fc640 1 --2- 172.21.15.81:0/4089851239 >> v2:172.21.15.81:3300/0 conn(0x7fecac0ca480 0x7fecac15ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.214+0000 7fecaa7fc640 1 -- 172.21.15.81:0/4089851239 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3377083613 0 0) 0x7fecac06b630 con 0x7fecac15ea60 2023-11-08T16:40:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.214+0000 7feca27fc640 1 -- 172.21.15.81:0/4089851239 >> v1:172.21.15.93:6789/0 conn(0x7fecac0ade50 legacy=0x7fecac15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.214+0000 7feca27fc640 1 -- 172.21.15.81:0/4089851239 >> v1:172.21.15.81:6789/0 conn(0x7fecac15ea60 legacy=0x7fecac15eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.214+0000 7feca27fc640 1 -- 172.21.15.81:0/4089851239 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac0cabb0 con 0x7fecac0ca480 2023-11-08T16:40:48.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.215+0000 7feca27fc640 1 --2- 172.21.15.81:0/4089851239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecac0ca480 0x7fecac15ad90 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fec940097a0 tx=0x7fec9402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95f1922df5d8c2a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:48.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.216+0000 7fecaa7fc640 1 -- 172.21.15.81:0/4089851239 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94031830 con 0x7fecac0ca480 2023-11-08T16:40:48.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.216+0000 7fecaa7fc640 1 -- 172.21.15.81:0/4089851239 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec94031990 con 0x7fecac0ca480 2023-11-08T16:40:48.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.216+0000 7fecaa7fc640 1 -- 172.21.15.81:0/4089851239 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94031d20 con 0x7fecac0ca480 2023-11-08T16:40:48.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.216+0000 7fecb1716640 1 -- 172.21.15.81:0/4089851239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecac0ca480 msgr2=0x7fecac15ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.216+0000 7fecb1716640 1 --2- 172.21.15.81:0/4089851239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecac0ca480 0x7fecac15ad90 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fec940097a0 tx=0x7fec9402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.216+0000 7fecb1716640 1 -- 172.21.15.81:0/4089851239 shutdown_connections 2023-11-08T16:40:48.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.216+0000 7fecb1716640 1 --2- 172.21.15.81:0/4089851239 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecac0ca480 0x7fecac15ad90 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.216+0000 7fecb1716640 1 -- 172.21.15.81:0/4089851239 >> 172.21.15.81:0/4089851239 conn(0x7fecac0cb490 msgr2=0x7fecac0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:48.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.217+0000 7fecb1716640 1 -- 172.21.15.81:0/4089851239 shutdown_connections 2023-11-08T16:40:48.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.217+0000 7fecb1716640 1 -- 172.21.15.81:0/4089851239 wait complete. 2023-11-08T16:40:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.218+0000 7fecb1716640 1 Processor -- start 2023-11-08T16:40:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.218+0000 7fecb1716640 1 -- start start 2023-11-08T16:40:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.219+0000 7fecb1716640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecac0ade50 0x7fecac1588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.219+0000 7fecb1716640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecac0ca480 0x7fecac14f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:48.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.219+0000 7fecb1716640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fecac15ea60 0x7fecac159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:48.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.219+0000 7fecb1716640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac158420 con 0x7fecac0ade50 2023-11-08T16:40:48.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.219+0000 7fecb1716640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecac14fad0 con 0x7fecac0ca480 2023-11-08T16:40:48.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.219+0000 7fecb1716640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecac15a3e0 con 0x7fecac15ea60 2023-11-08T16:40:48.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.220+0000 7fecaaffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecac0ade50 0x7fecac1588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:48.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.220+0000 7fecaaffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecac0ade50 0x7fecac1588b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57330/0 (socket says 172.21.15.81:57330) 2023-11-08T16:40:48.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.220+0000 7fecaaffd640 1 -- 172.21.15.81:0/344417111 learned_addr learned my addr 172.21.15.81:0/344417111 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:48.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.220+0000 7fecaaffd640 1 -- 172.21.15.81:0/344417111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fecac15ea60 msgr2=0x7fecac159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.220+0000 7fecaaffd640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fecac15ea60 0x7fecac159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.220+0000 7fecaaffd640 1 -- 172.21.15.81:0/344417111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecac0ca480 msgr2=0x7fecac14f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.220+0000 7feca27fc640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecac0ca480 0x7fecac14f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:48.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.220+0000 7fecaaffd640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecac0ca480 0x7fecac14f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.220+0000 7fecaaffd640 1 -- 172.21.15.81:0/344417111 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec94009400 con 0x7fecac0ade50 2023-11-08T16:40:48.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.220+0000 7fecaaffd640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecac0ade50 0x7fecac1588b0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00c460 tx=0x7fec9c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:48.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.221+0000 7feca3fff640 1 -- 172.21.15.81:0/344417111 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c011790 con 0x7fecac0ade50 2023-11-08T16:40:48.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.221+0000 7feca3fff640 1 -- 172.21.15.81:0/344417111 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec9c011930 con 0x7fecac0ade50 2023-11-08T16:40:48.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.221+0000 7feca3fff640 1 -- 172.21.15.81:0/344417111 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec9c011ce0 con 0x7fecac0ade50 2023-11-08T16:40:48.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.221+0000 7fecb1716640 1 -- 172.21.15.81:0/344417111 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecac150490 con 0x7fecac0ade50 2023-11-08T16:40:48.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.221+0000 7fecb1716640 1 -- 172.21.15.81:0/344417111 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecac157dd0 con 0x7fecac0ade50 2023-11-08T16:40:48.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.222+0000 7fecb1716640 1 -- 172.21.15.81:0/344417111 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecac04ece0 con 0x7fecac0ade50 2023-11-08T16:40:48.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.223+0000 7feca3fff640 1 -- 172.21.15.81:0/344417111 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fec9c01c460 con 0x7fecac0ade50 2023-11-08T16:40:48.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.224+0000 7feca3fff640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fec8c06ae70 0x7fec8c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:48.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.224+0000 7feca3fff640 1 -- 172.21.15.81:0/344417111 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fec9c089820 con 0x7fecac0ade50 2023-11-08T16:40:48.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.224+0000 7feca27fc640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fec8c06ae70 0x7fec8c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:48.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.225+0000 7feca27fc640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fec8c06ae70 0x7fec8c06d330 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fec94036000 tx=0x7fec94008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:48.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.228+0000 7feca3fff640 1 -- 172.21.15.81:0/344417111 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fec9c0545c0 con 0x7fecac0ade50 2023-11-08T16:40:48.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.456+0000 7fecb1716640 1 -- 172.21.15.81:0/344417111 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecac15e470 con 0x7fecac0ade50 2023-11-08T16:40:48.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.457+0000 7feca3fff640 1 -- 172.21.15.81:0/344417111 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4916 (secure 0 0 0) 0x7fec9c087e60 con 0x7fecac0ade50 2023-11-08T16:40:48.462 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:48.463 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":100,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/728323060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":728323060},{"type":"v1","addr":"172.21.15.93:6834","nonce":728323060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3481704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":3481704447},{"type":"v1","addr":"172.21.15.93:6835","nonce":3481704447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8187,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/699802320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":699802320},{"type":"v1","addr":"172.21.15.81:6835","nonce":699802320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:59.048664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/3197735576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3197735576},{"type":"v1","addr":"172.21.15.81:6837","nonce":3197735576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:40:48.464 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 100 2023-11-08T16:40:48.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.460+0000 7fecb1716640 1 -- 172.21.15.81:0/344417111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fec8c06ae70 msgr2=0x7fec8c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.460+0000 7fecb1716640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fec8c06ae70 0x7fec8c06d330 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fec94036000 tx=0x7fec94008030 comp rx=0 tx=0).stop 2023-11-08T16:40:48.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.460+0000 7fecb1716640 1 -- 172.21.15.81:0/344417111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecac0ade50 msgr2=0x7fecac1588b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.460+0000 7fecb1716640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecac0ade50 0x7fecac1588b0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fec9c00c460 tx=0x7fec9c00c920 comp rx=0 tx=0).stop 2023-11-08T16:40:48.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.460+0000 7fecb1716640 1 -- 172.21.15.81:0/344417111 shutdown_connections 2023-11-08T16:40:48.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.460+0000 7fecb1716640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fec8c06ae70 0x7fec8c06d330 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.460+0000 7fecb1716640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fecac15ea60 0x7fecac159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.460+0000 7fecb1716640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fecac0ca480 0x7fecac14f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.460+0000 7fecb1716640 1 --2- 172.21.15.81:0/344417111 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fecac0ade50 0x7fecac1588b0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.460+0000 7fecb1716640 1 -- 172.21.15.81:0/344417111 >> 172.21.15.81:0/344417111 conn(0x7fecac0cb490 msgr2=0x7fecac0b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:48.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.460+0000 7fecb1716640 1 -- 172.21.15.81:0/344417111 shutdown_connections 2023-11-08T16:40:48.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.461+0000 7fecb1716640 1 -- 172.21.15.81:0/344417111 wait complete. 2023-11-08T16:40:48.482 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:40:48.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.689+0000 7fa795e61640 1 Processor -- start 2023-11-08T16:40:48.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.689+0000 7fa795e61640 1 -- start start 2023-11-08T16:40:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.691+0000 7fa795e61640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa7900ade50 0x7fa7900ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.691+0000 7fa795e61640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 0x7fa79015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:48.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.691+0000 7fa795e61640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79006b630 con 0x7fa79015a680 2023-11-08T16:40:48.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.691+0000 7fa795e61640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7900b1860 con 0x7fa7900ade50 2023-11-08T16:40:48.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.691+0000 7fa795e61640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7900ca9c0 con 0x7fa79015ea60 2023-11-08T16:40:48.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.691+0000 7fa78ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 0x7fa79015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:48.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.691+0000 7fa78ffff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 0x7fa79015eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:53946/0 (socket says 172.21.15.81:53946) 2023-11-08T16:40:48.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.691+0000 7fa786ffd640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fa79015a680 0x7fa79015ab10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50674/0 (socket says 172.21.15.81:50674) 2023-11-08T16:40:48.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.691+0000 7fa78ffff640 1 -- 172.21.15.81:0/3370697170 learned_addr learned my addr 172.21.15.81:0/3370697170 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:48.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.691+0000 7fa78f7fe640 1 --2- 172.21.15.81:0/3370697170 >> v2:172.21.15.93:3300/0 conn(0x7fa7900ade50 0x7fa7900ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:48.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.692+0000 7fa78ffff640 1 -- 172.21.15.81:0/3370697170 >> v2:172.21.15.93:3300/0 conn(0x7fa7900ade50 msgr2=0x7fa7900ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:48.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.692+0000 7fa78ffff640 1 --2- 172.21.15.81:0/3370697170 >> v2:172.21.15.93:3300/0 conn(0x7fa7900ade50 0x7fa7900ca480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.692+0000 7fa78ffff640 1 -- 172.21.15.81:0/3370697170 >> v1:172.21.15.93:6789/0 conn(0x7fa79015a680 legacy=0x7fa79015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.692+0000 7fa78ffff640 1 -- 172.21.15.81:0/3370697170 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7900cabb0 con 0x7fa79015ea60 2023-11-08T16:40:48.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.694+0000 7fa78ffff640 1 --2- 172.21.15.81:0/3370697170 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 0x7fa79015eef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa788009e50 tx=0x7fa78802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b95f19d7bfb9cbc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:48.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.694+0000 7fa78effd640 1 -- 172.21.15.81:0/3370697170 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78802cb00 con 0x7fa79015ea60 2023-11-08T16:40:48.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.694+0000 7fa78effd640 1 -- 172.21.15.81:0/3370697170 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78802cc60 con 0x7fa79015ea60 2023-11-08T16:40:48.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.694+0000 7fa78effd640 1 -- 172.21.15.81:0/3370697170 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7880343f0 con 0x7fa79015ea60 2023-11-08T16:40:48.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.695+0000 7fa795e61640 1 -- 172.21.15.81:0/3370697170 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 msgr2=0x7fa79015eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.695+0000 7fa795e61640 1 --2- 172.21.15.81:0/3370697170 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 0x7fa79015eef0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa788009e50 tx=0x7fa78802bf60 comp rx=0 tx=0).stop 2023-11-08T16:40:48.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.695+0000 7fa795e61640 1 -- 172.21.15.81:0/3370697170 shutdown_connections 2023-11-08T16:40:48.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.695+0000 7fa795e61640 1 --2- 172.21.15.81:0/3370697170 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 0x7fa79015eef0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.695+0000 7fa795e61640 1 --2- 172.21.15.81:0/3370697170 >> v2:172.21.15.93:3300/0 conn(0x7fa7900ade50 0x7fa7900ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.695+0000 7fa795e61640 1 -- 172.21.15.81:0/3370697170 >> 172.21.15.81:0/3370697170 conn(0x7fa7900cb490 msgr2=0x7fa7900cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:48.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.695+0000 7fa795e61640 1 -- 172.21.15.81:0/3370697170 shutdown_connections 2023-11-08T16:40:48.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.695+0000 7fa795e61640 1 -- 172.21.15.81:0/3370697170 wait complete. 2023-11-08T16:40:48.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.697+0000 7fa795e61640 1 Processor -- start 2023-11-08T16:40:48.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.697+0000 7fa795e61640 1 -- start start 2023-11-08T16:40:48.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.698+0000 7fa795e61640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7900ade50 0x7fa790155a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:48.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.698+0000 7fa795e61640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa79015a680 0x7fa79014f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:48.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.698+0000 7fa795e61640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 0x7fa79014e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:48.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.698+0000 7fa795e61640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79014e800 con 0x7fa7900ade50 2023-11-08T16:40:48.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.698+0000 7fa795e61640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa790156090 con 0x7fa79015a680 2023-11-08T16:40:48.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.698+0000 7fa795e61640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79015bf40 con 0x7fa79015ea60 2023-11-08T16:40:48.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.698+0000 7fa78f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7900ade50 0x7fa790155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:48.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.698+0000 7fa78f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7900ade50 0x7fa790155a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57342/0 (socket says 172.21.15.81:57342) 2023-11-08T16:40:48.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.698+0000 7fa78f7fe640 1 -- 172.21.15.81:0/2394216878 learned_addr learned my addr 172.21.15.81:0/2394216878 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:48.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.699+0000 7fa78ffff640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 0x7fa79014e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:48.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.699+0000 7fa786ffd640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa79015a680 0x7fa79014f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:48.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.699+0000 7fa78f7fe640 1 -- 172.21.15.81:0/2394216878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 msgr2=0x7fa79014e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.699+0000 7fa78f7fe640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 0x7fa79014e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.699+0000 7fa78f7fe640 1 -- 172.21.15.81:0/2394216878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa79015a680 msgr2=0x7fa79014f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.699+0000 7fa78f7fe640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa79015a680 0x7fa79014f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.699+0000 7fa78f7fe640 1 -- 172.21.15.81:0/2394216878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa788009b30 con 0x7fa7900ade50 2023-11-08T16:40:48.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.699+0000 7fa786ffd640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa79015a680 0x7fa79014f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:40:48.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.699+0000 7fa78f7fe640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7900ade50 0x7fa790155a80 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa78000e9b0 tx=0x7fa78000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:48.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.700+0000 7fa78cff9640 1 -- 172.21.15.81:0/2394216878 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780009670 con 0x7fa7900ade50 2023-11-08T16:40:48.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.700+0000 7fa78cff9640 1 -- 172.21.15.81:0/2394216878 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78000c9a0 con 0x7fa7900ade50 2023-11-08T16:40:48.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.700+0000 7fa78cff9640 1 -- 172.21.15.81:0/2394216878 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78000cd50 con 0x7fa7900ade50 2023-11-08T16:40:48.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.700+0000 7fa795e61640 1 -- 172.21.15.81:0/2394216878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa79015c220 con 0x7fa7900ade50 2023-11-08T16:40:48.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.700+0000 7fa795e61640 1 -- 172.21.15.81:0/2394216878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79015c7e0 con 0x7fa7900ade50 2023-11-08T16:40:48.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.701+0000 7fa795e61640 1 -- 172.21.15.81:0/2394216878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79004ece0 con 0x7fa7900ade50 2023-11-08T16:40:48.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.702+0000 7fa78cff9640 1 -- 172.21.15.81:0/2394216878 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa780013470 con 0x7fa7900ade50 2023-11-08T16:40:48.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.703+0000 7fa78cff9640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa77406ae70 0x7fa77406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:48.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.703+0000 7fa78cff9640 1 -- 172.21.15.81:0/2394216878 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fa780089b70 con 0x7fa7900ade50 2023-11-08T16:40:48.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.703+0000 7fa786ffd640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa77406ae70 0x7fa77406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:48.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.704+0000 7fa786ffd640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa77406ae70 0x7fa77406d330 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa778007fd0 tx=0x7fa778007d40 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:48.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.707+0000 7fa78cff9640 1 -- 172.21.15.81:0/2394216878 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa780054910 con 0x7fa7900ade50 2023-11-08T16:40:48.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.900+0000 7fa795e61640 1 -- 172.21.15.81:0/2394216878 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7900cc1d0 con 0x7fa7900ade50 2023-11-08T16:40:48.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.901+0000 7fa78cff9640 1 -- 172.21.15.81:0/2394216878 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18822 (secure 0 0 0) 0x7fa780017090 con 0x7fa7900ade50 2023-11-08T16:40:48.905 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:48.905 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":114,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:39:57.654547+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:39:07.492834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T16:39:08.634345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:39:19.654916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6836/3163533823":"2023-11-09T16:39:57.018336+0000","172.21.15.81:6836/3988749175":"2023-11-09T16:39:50.607108+0000","172.21.15.81:6837/3163533823":"2023-11-09T16:39:57.018336+0000","172.21.15.93:6835/691341796":"2023-11-09T16:39:48.576045+0000","172.21.15.81:6837/3988749175":"2023-11-09T16:39:50.607108+0000","172.21.15.93:6834/691341796":"2023-11-09T16:39:48.576045+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:40:48.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 -- 172.21.15.81:0/2394216878 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa77406ae70 msgr2=0x7fa77406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa77406ae70 0x7fa77406d330 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa778007fd0 tx=0x7fa778007d40 comp rx=0 tx=0).stop 2023-11-08T16:40:48.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 -- 172.21.15.81:0/2394216878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7900ade50 msgr2=0x7fa790155a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:48.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7900ade50 0x7fa790155a80 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa78000e9b0 tx=0x7fa78000ee70 comp rx=0 tx=0).stop 2023-11-08T16:40:48.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 -- 172.21.15.81:0/2394216878 shutdown_connections 2023-11-08T16:40:48.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa77406ae70 0x7fa77406d330 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa79015ea60 0x7fa79014e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa79015a680 0x7fa79014f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 --2- 172.21.15.81:0/2394216878 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa7900ade50 0x7fa790155a80 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:48.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 -- 172.21.15.81:0/2394216878 >> 172.21.15.81:0/2394216878 conn(0x7fa7900cb490 msgr2=0x7fa790159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:48.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 -- 172.21.15.81:0/2394216878 shutdown_connections 2023-11-08T16:40:48.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:48.904+0000 7fa795e61640 1 -- 172.21.15.81:0/2394216878 wait complete. 2023-11-08T16:40:48.927 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T16:40:48.927 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:40:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.125+0000 7f5894c5d640 1 Processor -- start 2023-11-08T16:40:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.126+0000 7f5894c5d640 1 -- start start 2023-11-08T16:40:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.127+0000 7f5894c5d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f58900adcd0 0x7f58900ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.127+0000 7f5894c5d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5890070050 0x7f58900704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.127+0000 7f5894c5d640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58900c9a70 con 0x7f58900cc5e0 2023-11-08T16:40:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.127+0000 7f5894c5d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58900cc3e0 con 0x7f5890070050 2023-11-08T16:40:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.127+0000 7f5894c5d640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f589006d070 con 0x7f58900adcd0 2023-11-08T16:40:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.127+0000 7f5887fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f58900cc5e0 0x7f58900cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55620/0 (socket says 172.21.15.81:55620) 2023-11-08T16:40:49.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.127+0000 7f5887fff640 1 -- 172.21.15.81:0/4239769127 learned_addr learned my addr 172.21.15.81:0/4239769127 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:49.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.127+0000 7f588ed9d640 1 --2- 172.21.15.81:0/4239769127 >> v2:172.21.15.81:3300/0 conn(0x7f5890070050 0x7f58900704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.127+0000 7f588e59c640 1 --2- 172.21.15.81:0/4239769127 >> v2:172.21.15.93:3300/0 conn(0x7f58900adcd0 0x7f58900ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.127+0000 7f588dd9b640 1 -- 172.21.15.81:0/4239769127 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813132393 0 0) 0x7f58900c9a70 con 0x7f58900cc5e0 2023-11-08T16:40:49.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.128+0000 7f588ed9d640 1 -- 172.21.15.81:0/4239769127 >> v2:172.21.15.93:3300/0 conn(0x7f58900adcd0 msgr2=0x7f58900ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:49.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.128+0000 7f588ed9d640 1 --2- 172.21.15.81:0/4239769127 >> v2:172.21.15.93:3300/0 conn(0x7f58900adcd0 0x7f58900ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.128+0000 7f588ed9d640 1 -- 172.21.15.81:0/4239769127 >> v1:172.21.15.81:6789/0 conn(0x7f58900cc5e0 legacy=0x7f58900cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.128+0000 7f588ed9d640 1 -- 172.21.15.81:0/4239769127 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f589006d350 con 0x7f5890070050 2023-11-08T16:40:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.128+0000 7f588e59c640 1 --2- 172.21.15.81:0/4239769127 >> v2:172.21.15.93:3300/0 conn(0x7f58900adcd0 0x7f58900ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.129+0000 7f588ed9d640 1 --2- 172.21.15.81:0/4239769127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5890070050 0x7f58900704e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f5890067770 tx=0x7f588802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76ebeb3b19c0561 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.129+0000 7f588dd9b640 1 -- 172.21.15.81:0/4239769127 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588802cc70 con 0x7f5890070050 2023-11-08T16:40:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.129+0000 7f588dd9b640 1 -- 172.21.15.81:0/4239769127 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588802cdd0 con 0x7f5890070050 2023-11-08T16:40:49.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.129+0000 7f588dd9b640 1 -- 172.21.15.81:0/4239769127 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5888034660 con 0x7f5890070050 2023-11-08T16:40:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.130+0000 7f5894c5d640 1 -- 172.21.15.81:0/4239769127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5890070050 msgr2=0x7f58900704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.130+0000 7f5894c5d640 1 --2- 172.21.15.81:0/4239769127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5890070050 0x7f58900704e0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f5890067770 tx=0x7f588802bec0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.130+0000 7f5894c5d640 1 -- 172.21.15.81:0/4239769127 shutdown_connections 2023-11-08T16:40:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.130+0000 7f5894c5d640 1 --2- 172.21.15.81:0/4239769127 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5890070050 0x7f58900704e0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.130+0000 7f5894c5d640 1 --2- 172.21.15.81:0/4239769127 >> v2:172.21.15.93:3300/0 conn(0x7f58900adcd0 0x7f58900ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.130+0000 7f5894c5d640 1 -- 172.21.15.81:0/4239769127 >> 172.21.15.81:0/4239769127 conn(0x7f58900cb490 msgr2=0x7f58900cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.130+0000 7f5894c5d640 1 -- 172.21.15.81:0/4239769127 shutdown_connections 2023-11-08T16:40:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.130+0000 7f5894c5d640 1 -- 172.21.15.81:0/4239769127 wait complete. 2023-11-08T16:40:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.131+0000 7f5894c5d640 1 Processor -- start 2023-11-08T16:40:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.132+0000 7f5894c5d640 1 -- start start 2023-11-08T16:40:49.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5894c5d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5890070050 0x7f58900b27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5894c5d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58900adcd0 0x7f5890156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5894c5d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58900cc5e0 0x7f58901e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5894c5d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58900b39a0 con 0x7f58900adcd0 2023-11-08T16:40:49.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5894c5d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58900b2d90 con 0x7f5890070050 2023-11-08T16:40:49.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5894c5d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58900b3070 con 0x7f58900cc5e0 2023-11-08T16:40:49.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5887fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58900adcd0 0x7f5890156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5887fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58900adcd0 0x7f5890156020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57352/0 (socket says 172.21.15.81:57352) 2023-11-08T16:40:49.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5887fff640 1 -- 172.21.15.81:0/369637521 learned_addr learned my addr 172.21.15.81:0/369637521 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:49.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5887fff640 1 -- 172.21.15.81:0/369637521 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58900cc5e0 msgr2=0x7f58901e5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f588ed9d640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58900cc5e0 0x7f58901e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f588e59c640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5890070050 0x7f58900b27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5887fff640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58900cc5e0 0x7f58901e5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5887fff640 1 -- 172.21.15.81:0/369637521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5890070050 msgr2=0x7f58900b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5887fff640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5890070050 0x7f58900b27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.133+0000 7f5887fff640 1 -- 172.21.15.81:0/369637521 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5888009a70 con 0x7f58900adcd0 2023-11-08T16:40:49.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.134+0000 7f5887fff640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58900adcd0 0x7f5890156020 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f587800c430 tx=0x7f587800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:49.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.134+0000 7f5886ffd640 1 -- 172.21.15.81:0/369637521 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58780074e0 con 0x7f58900adcd0 2023-11-08T16:40:49.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.134+0000 7f5886ffd640 1 -- 172.21.15.81:0/369637521 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5878007660 con 0x7f58900adcd0 2023-11-08T16:40:49.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.134+0000 7f5886ffd640 1 -- 172.21.15.81:0/369637521 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878004a20 con 0x7f58900adcd0 2023-11-08T16:40:49.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.134+0000 7f5894c5d640 1 -- 172.21.15.81:0/369637521 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58900b3350 con 0x7f58900adcd0 2023-11-08T16:40:49.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.134+0000 7f5894c5d640 1 -- 172.21.15.81:0/369637521 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58900b3e30 con 0x7f58900adcd0 2023-11-08T16:40:49.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.135+0000 7f5894c5d640 1 -- 172.21.15.81:0/369637521 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5890072170 con 0x7f58900adcd0 2023-11-08T16:40:49.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.136+0000 7f5886ffd640 1 -- 172.21.15.81:0/369637521 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5878004b80 con 0x7f58900adcd0 2023-11-08T16:40:49.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.137+0000 7f5886ffd640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f587006aea0 0x7f587006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.138+0000 7f5886ffd640 1 -- 172.21.15.81:0/369637521 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f58780897e0 con 0x7f58900adcd0 2023-11-08T16:40:49.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.138+0000 7f588e59c640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f587006aea0 0x7f587006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.138+0000 7f588e59c640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f587006aea0 0x7f587006d360 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5880008850 tx=0x7f5880008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:49.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.142+0000 7f5886ffd640 1 -- 172.21.15.81:0/369637521 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f58780544e0 con 0x7f58900adcd0 2023-11-08T16:40:49.323 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:40:49.333 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:40:49.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.371+0000 7f5894c5d640 1 -- 172.21.15.81:0/369637521 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f589004ece0 con 0x7f58900adcd0 2023-11-08T16:40:49.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.372+0000 7f5886ffd640 1 -- 172.21.15.81:0/369637521 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4916 (secure 0 0 0) 0x7f5878057b00 con 0x7f58900adcd0 2023-11-08T16:40:49.376 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:49.376 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":100,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/728323060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":728323060},{"type":"v1","addr":"172.21.15.93:6834","nonce":728323060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3481704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":3481704447},{"type":"v1","addr":"172.21.15.93:6835","nonce":3481704447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8187,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/699802320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":699802320},{"type":"v1","addr":"172.21.15.81:6835","nonce":699802320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:59.048664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/3197735576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3197735576},{"type":"v1","addr":"172.21.15.81:6837","nonce":3197735576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:40:49.377 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 100 2023-11-08T16:40:49.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 -- 172.21.15.81:0/369637521 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f587006aea0 msgr2=0x7f587006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f587006aea0 0x7f587006d360 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f5880008850 tx=0x7f5880008040 comp rx=0 tx=0).stop 2023-11-08T16:40:49.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 -- 172.21.15.81:0/369637521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58900adcd0 msgr2=0x7f5890156020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58900adcd0 0x7f5890156020 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f587800c430 tx=0x7f587800c8f0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 -- 172.21.15.81:0/369637521 shutdown_connections 2023-11-08T16:40:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f587006aea0 0x7f587006d360 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f58900cc5e0 0x7f58901e5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f58900adcd0 0x7f5890156020 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 --2- 172.21.15.81:0/369637521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5890070050 0x7f58900b27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 -- 172.21.15.81:0/369637521 >> 172.21.15.81:0/369637521 conn(0x7f58900cb490 msgr2=0x7f58900c5370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:49.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 -- 172.21.15.81:0/369637521 shutdown_connections 2023-11-08T16:40:49.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.374+0000 7f5894c5d640 1 -- 172.21.15.81:0/369637521 wait complete. 2023-11-08T16:40:49.395 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:40:49.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.648+0000 7fe50f916640 1 Processor -- start 2023-11-08T16:40:49.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.648+0000 7fe50f916640 1 -- start start 2023-11-08T16:40:49.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.649+0000 7fe50f916640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe5080adcd0 0x7fe5080ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.649+0000 7fe50f916640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080cc5e0 0x7fe5080cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.649+0000 7fe50f916640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5080c9a70 con 0x7fe5080ccf80 2023-11-08T16:40:49.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.649+0000 7fe50f916640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5080cc3e0 con 0x7fe5080adcd0 2023-11-08T16:40:49.652 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.649+0000 7fe50f916640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50806d070 con 0x7fe5080cc5e0 2023-11-08T16:40:49.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.650+0000 7fe50ceb1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080cc5e0 0x7fe5080cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.650+0000 7fe50d6b2640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe5080adcd0 0x7fe5080ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.650+0000 7fe50deb3640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7fe5080ccf80 0x7fe50815db80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:50688/0 (socket says 172.21.15.81:50688) 2023-11-08T16:40:49.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.650+0000 7fe50ceb1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080cc5e0 0x7fe5080cca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:53964/0 (socket says 172.21.15.81:53964) 2023-11-08T16:40:49.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.650+0000 7fe50deb3640 1 -- 172.21.15.81:0/1300080254 learned_addr learned my addr 172.21.15.81:0/1300080254 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:49.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.651+0000 7fe50d6b2640 1 -- 172.21.15.81:0/1300080254 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080cc5e0 msgr2=0x7fe5080cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:49.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.651+0000 7fe50d6b2640 1 --2- 172.21.15.81:0/1300080254 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080cc5e0 0x7fe5080cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.651+0000 7fe50d6b2640 1 -- 172.21.15.81:0/1300080254 >> v1:172.21.15.93:6789/0 conn(0x7fe5080ccf80 legacy=0x7fe50815db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.651+0000 7fe50d6b2640 1 -- 172.21.15.81:0/1300080254 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50806d350 con 0x7fe5080adcd0 2023-11-08T16:40:49.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.651+0000 7fe4fffff640 1 -- 172.21.15.81:0/1300080254 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507029996 0 0) 0x7fe5080c9a70 con 0x7fe5080ccf80 2023-11-08T16:40:49.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.651+0000 7fe50ceb1640 1 --2- 172.21.15.81:0/1300080254 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080cc5e0 0x7fe5080cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:40:49.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.653+0000 7fe50d6b2640 1 --2- 172.21.15.81:0/1300080254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5080adcd0 0x7fe5080ca1e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe5040097a0 tx=0x7fe50402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbd47f9608ba77cf server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.653+0000 7fe4fffff640 1 -- 172.21.15.81:0/1300080254 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50402d040 con 0x7fe5080adcd0 2023-11-08T16:40:49.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.653+0000 7fe4fffff640 1 -- 172.21.15.81:0/1300080254 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe504031810 con 0x7fe5080adcd0 2023-11-08T16:40:49.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.653+0000 7fe4fffff640 1 -- 172.21.15.81:0/1300080254 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504031ba0 con 0x7fe5080adcd0 2023-11-08T16:40:49.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.654+0000 7fe50f916640 1 -- 172.21.15.81:0/1300080254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5080adcd0 msgr2=0x7fe5080ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.654+0000 7fe50f916640 1 --2- 172.21.15.81:0/1300080254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5080adcd0 0x7fe5080ca1e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe5040097a0 tx=0x7fe50402b8b0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.654+0000 7fe50f916640 1 -- 172.21.15.81:0/1300080254 shutdown_connections 2023-11-08T16:40:49.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.654+0000 7fe50f916640 1 --2- 172.21.15.81:0/1300080254 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080cc5e0 0x7fe5080cca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.654+0000 7fe50f916640 1 --2- 172.21.15.81:0/1300080254 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5080adcd0 0x7fe5080ca1e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.654+0000 7fe50f916640 1 -- 172.21.15.81:0/1300080254 >> 172.21.15.81:0/1300080254 conn(0x7fe5080cb490 msgr2=0x7fe5080cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:49.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.654+0000 7fe50f916640 1 -- 172.21.15.81:0/1300080254 shutdown_connections 2023-11-08T16:40:49.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.654+0000 7fe50f916640 1 -- 172.21.15.81:0/1300080254 wait complete. 2023-11-08T16:40:49.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.656+0000 7fe50f916640 1 Processor -- start 2023-11-08T16:40:49.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.656+0000 7fe50f916640 1 -- start start 2023-11-08T16:40:49.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.657+0000 7fe50f916640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5080adcd0 0x7fe5081f0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.657+0000 7fe50f916640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5080cc5e0 0x7fe5080b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.657+0000 7fe50f916640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080ccf80 0x7fe5080b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.657+0000 7fe50f916640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080aaf10 con 0x7fe5080adcd0 2023-11-08T16:40:49.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.657+0000 7fe50f916640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080ab1f0 con 0x7fe5080cc5e0 2023-11-08T16:40:49.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.657+0000 7fe50f916640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5080ab4d0 con 0x7fe5080ccf80 2023-11-08T16:40:49.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.657+0000 7fe50d6b2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5080adcd0 0x7fe5081f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.657+0000 7fe50d6b2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5080adcd0 0x7fe5081f0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57354/0 (socket says 172.21.15.81:57354) 2023-11-08T16:40:49.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.657+0000 7fe50d6b2640 1 -- 172.21.15.81:0/864625742 learned_addr learned my addr 172.21.15.81:0/864625742 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:49.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.657+0000 7fe50deb3640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080ccf80 0x7fe5080b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe50ceb1640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5080cc5e0 0x7fe5080b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe50d6b2640 1 -- 172.21.15.81:0/864625742 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080ccf80 msgr2=0x7fe5080b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe50d6b2640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080ccf80 0x7fe5080b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe50d6b2640 1 -- 172.21.15.81:0/864625742 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5080cc5e0 msgr2=0x7fe5080b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe50d6b2640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5080cc5e0 0x7fe5080b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe50d6b2640 1 -- 172.21.15.81:0/864625742 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4f8009400 con 0x7fe5080adcd0 2023-11-08T16:40:49.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe50ceb1640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5080cc5e0 0x7fe5080b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:49.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe50d6b2640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5080adcd0 0x7fe5081f0150 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe5040097a0 tx=0x7fe5040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:49.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe4fdffb640 1 -- 172.21.15.81:0/864625742 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504006e10 con 0x7fe5080adcd0 2023-11-08T16:40:49.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe4fdffb640 1 -- 172.21.15.81:0/864625742 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50403ad60 con 0x7fe5080adcd0 2023-11-08T16:40:49.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe50f916640 1 -- 172.21.15.81:0/864625742 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe504009400 con 0x7fe5080adcd0 2023-11-08T16:40:49.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.658+0000 7fe4fdffb640 1 -- 172.21.15.81:0/864625742 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504044610 con 0x7fe5080adcd0 2023-11-08T16:40:49.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.659+0000 7fe50f916640 1 -- 172.21.15.81:0/864625742 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5080aba90 con 0x7fe5080adcd0 2023-11-08T16:40:49.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.660+0000 7fe50f916640 1 -- 172.21.15.81:0/864625742 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe508072170 con 0x7fe5080adcd0 2023-11-08T16:40:49.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.661+0000 7fe4fdffb640 1 -- 172.21.15.81:0/864625742 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe50404d450 con 0x7fe5080adcd0 2023-11-08T16:40:49.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.661+0000 7fe4fdffb640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4ec06ae70 0x7fe4ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:49.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.662+0000 7fe4fdffb640 1 -- 172.21.15.81:0/864625742 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7fe5040abb40 con 0x7fe5080adcd0 2023-11-08T16:40:49.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.662+0000 7fe50ceb1640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4ec06ae70 0x7fe4ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:49.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.662+0000 7fe50ceb1640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4ec06ae70 0x7fe4ec06d330 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe5080ccb90 tx=0x7fe4f8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:49.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.666+0000 7fe4fdffb640 1 -- 172.21.15.81:0/864625742 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe5040768e0 con 0x7fe5080adcd0 2023-11-08T16:40:49.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.893+0000 7fe50f916640 1 -- 172.21.15.81:0/864625742 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe50804ece0 con 0x7fe5080adcd0 2023-11-08T16:40:49.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.894+0000 7fe4fdffb640 1 -- 172.21.15.81:0/864625742 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) v1 ==== 95+0+4916 (secure 0 0 0) 0x7fe504079f00 con 0x7fe5080adcd0 2023-11-08T16:40:49.900 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:49.900 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":100,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/728323060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":728323060},{"type":"v1","addr":"172.21.15.93:6834","nonce":728323060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3481704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":3481704447},{"type":"v1","addr":"172.21.15.93:6835","nonce":3481704447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8187,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/699802320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":699802320},{"type":"v1","addr":"172.21.15.81:6835","nonce":699802320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[{"mdsmap":{"epoch":100,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:39:49.779667+0000","modified":"2023-11-08T16:39:59.048664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":112,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8190":{"gid":8190,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/3197735576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":3197735576},{"type":"v1","addr":"172.21.15.81:6837","nonce":3197735576}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14,15],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-08T16:40:49.902 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 100 2023-11-08T16:40:49.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 -- 172.21.15.81:0/864625742 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4ec06ae70 msgr2=0x7fe4ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4ec06ae70 0x7fe4ec06d330 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fe5080ccb90 tx=0x7fe4f8008040 comp rx=0 tx=0).stop 2023-11-08T16:40:49.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 -- 172.21.15.81:0/864625742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5080adcd0 msgr2=0x7fe5081f0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:49.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5080adcd0 0x7fe5081f0150 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe5040097a0 tx=0x7fe5040046c0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 -- 172.21.15.81:0/864625742 shutdown_connections 2023-11-08T16:40:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe4ec06ae70 0x7fe4ec06d330 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe5080ccf80 0x7fe5080b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5080cc5e0 0x7fe5080b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 --2- 172.21.15.81:0/864625742 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe5080adcd0 0x7fe5081f0150 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:49.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 -- 172.21.15.81:0/864625742 >> 172.21.15.81:0/864625742 conn(0x7fe5080cb490 msgr2=0x7fe50815cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:49.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 -- 172.21.15.81:0/864625742 shutdown_connections 2023-11-08T16:40:49.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:49.897+0000 7fe50f916640 1 -- 172.21.15.81:0/864625742 wait complete. 2023-11-08T16:40:49.919 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:40:50.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.132+0000 7f4175778640 1 Processor -- start 2023-11-08T16:40:50.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.132+0000 7f4175778640 1 -- start start 2023-11-08T16:40:50.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.133+0000 7f4175778640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f4170071060 0x7f4170164eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.133+0000 7f4175778640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41700c9a70 con 0x7f41700cc5e0 2023-11-08T16:40:50.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.133+0000 7f4175778640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41700cc3e0 con 0x7f41700adcd0 2023-11-08T16:40:50.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.133+0000 7f4175778640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417006d070 con 0x7f4170071060 2023-11-08T16:40:50.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.134+0000 7f4167fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f41700cc5e0 0x7f41700cca70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55634/0 (socket says 172.21.15.81:55634) 2023-11-08T16:40:50.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.134+0000 7f4167fff640 1 -- 172.21.15.81:0/1994841823 learned_addr learned my addr 172.21.15.81:0/1994841823 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:50.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.134+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 827169299 0 0) 0x7f41700c9a70 con 0x7f41700cc5e0 2023-11-08T16:40:50.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.134+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f415c005730 con 0x7f41700cc5e0 2023-11-08T16:40:50.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.134+0000 7f416f7fe640 1 --2- 172.21.15.81:0/1994841823 >> v2:172.21.15.93:3300/0 conn(0x7f4170071060 0x7f4170164eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.134+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2033888861 0 0) 0x7f415c005730 con 0x7f41700cc5e0 2023-11-08T16:40:50.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.134+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 >> v2:172.21.15.93:3300/0 conn(0x7f4170071060 msgr2=0x7f4170164eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:50.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.134+0000 7f416e7fc640 1 --2- 172.21.15.81:0/1994841823 >> v2:172.21.15.93:3300/0 conn(0x7f4170071060 0x7f4170164eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.134+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 >> v1:172.21.15.93:6789/0 conn(0x7f41700adcd0 legacy=0x7f41700ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:50.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f417006d350 con 0x7f41700cc5e0 2023-11-08T16:40:50.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f4158001670 con 0x7f41700cc5e0 2023-11-08T16:40:50.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416f7fe640 1 --2- 172.21.15.81:0/1994841823 >> v2:172.21.15.93:3300/0 conn(0x7f4170071060 0x7f4170164eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:50.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 >> v1:172.21.15.81:6789/0 conn(0x7f41700cc5e0 legacy=0x7f41700cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416e7fc640 1 --2- 172.21.15.81:0/1994841823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f415c005c60 0x7f415c006080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416e7fc640 1 --2- 172.21.15.81:0/1994841823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f415c0069d0 0x7f415c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416e7fc640 1 --2- 172.21.15.81:0/1994841823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f415c007590 0x7f415c00dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417006d070 con 0x7f415c0069d0 2023-11-08T16:40:50.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c00e490 con 0x7f415c005c60 2023-11-08T16:40:50.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c00e650 con 0x7f415c007590 2023-11-08T16:40:50.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f4167fff640 1 --2- 172.21.15.81:0/1994841823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f415c0069d0 0x7f415c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f4167fff640 1 -- 172.21.15.81:0/1994841823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f415c007590 msgr2=0x7f415c00dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:50.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416f7fe640 1 --2- 172.21.15.81:0/1994841823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f415c007590 0x7f415c00dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f416effd640 1 --2- 172.21.15.81:0/1994841823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f415c005c60 0x7f415c006080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f4167fff640 1 --2- 172.21.15.81:0/1994841823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f415c007590 0x7f415c00dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f4167fff640 1 -- 172.21.15.81:0/1994841823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f415c005c60 msgr2=0x7f415c006080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:50.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.135+0000 7f4167fff640 1 --2- 172.21.15.81:0/1994841823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f415c005c60 0x7f415c006080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.136+0000 7f4167fff640 1 -- 172.21.15.81:0/1994841823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c005730 con 0x7f415c0069d0 2023-11-08T16:40:50.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.136+0000 7f416f7fe640 1 -- 172.21.15.81:0/1994841823 reap_dead start 2023-11-08T16:40:50.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.137+0000 7f4167fff640 1 --2- 172.21.15.81:0/1994841823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f415c0069d0 0x7f415c006e50 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f415800c240 tx=0x7f415c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86eaf3743a811177 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:50.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.137+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158005ad0 con 0x7f415c0069d0 2023-11-08T16:40:50.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.137+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4158005c30 con 0x7f415c0069d0 2023-11-08T16:40:50.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.137+0000 7f416e7fc640 1 -- 172.21.15.81:0/1994841823 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415802caf0 con 0x7f415c0069d0 2023-11-08T16:40:50.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.137+0000 7f4175778640 1 -- 172.21.15.81:0/1994841823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f415c0069d0 msgr2=0x7f415c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.137+0000 7f4175778640 1 --2- 172.21.15.81:0/1994841823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f415c0069d0 0x7f415c006e50 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f415800c240 tx=0x7f415c007a00 comp rx=0 tx=0).stop 2023-11-08T16:40:50.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.138+0000 7f4175778640 1 -- 172.21.15.81:0/1994841823 shutdown_connections 2023-11-08T16:40:50.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.138+0000 7f4175778640 1 --2- 172.21.15.81:0/1994841823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f415c0069d0 0x7f415c006e50 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.138+0000 7f4175778640 1 -- 172.21.15.81:0/1994841823 >> 172.21.15.81:0/1994841823 conn(0x7f41700cb490 msgr2=0x7f41700cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:50.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.138+0000 7f4175778640 1 -- 172.21.15.81:0/1994841823 shutdown_connections 2023-11-08T16:40:50.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.138+0000 7f4175778640 1 -- 172.21.15.81:0/1994841823 wait complete. 2023-11-08T16:40:50.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.139+0000 7f4175778640 1 Processor -- start 2023-11-08T16:40:50.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.140+0000 7f4175778640 1 -- start start 2023-11-08T16:40:50.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.140+0000 7f4175778640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f415c0069d0 0x7f4170154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.140+0000 7f4175778640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41700c5ae0 0x7f4170157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.140+0000 7f4175778640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4170158150 0x7f4170156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.140+0000 7f4175778640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41701585d0 con 0x7f4170158150 2023-11-08T16:40:50.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.140+0000 7f4175778640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41700b3670 con 0x7f415c0069d0 2023-11-08T16:40:50.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.140+0000 7f4175778640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4170158a60 con 0x7f41700c5ae0 2023-11-08T16:40:50.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.141+0000 7f416f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4170158150 0x7f4170156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.141+0000 7f416f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4170158150 0x7f4170156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57368/0 (socket says 172.21.15.81:57368) 2023-11-08T16:40:50.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.141+0000 7f416f7fe640 1 -- 172.21.15.81:0/2819680897 learned_addr learned my addr 172.21.15.81:0/2819680897 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:50.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.141+0000 7f4167fff640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41700c5ae0 0x7f4170157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.141+0000 7f416f7fe640 1 -- 172.21.15.81:0/2819680897 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41700c5ae0 msgr2=0x7f4170157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.141+0000 7f416f7fe640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41700c5ae0 0x7f4170157c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.141+0000 7f416effd640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f415c0069d0 0x7f4170154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.141+0000 7f416f7fe640 1 -- 172.21.15.81:0/2819680897 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f415c0069d0 msgr2=0x7f4170154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.141+0000 7f416f7fe640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f415c0069d0 0x7f4170154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.141+0000 7f416f7fe640 1 -- 172.21.15.81:0/2819680897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41701553e0 con 0x7f4170158150 2023-11-08T16:40:50.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.142+0000 7f416f7fe640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4170158150 0x7f4170156590 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f41700cb440 tx=0x7f416800b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:50.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.142+0000 7f41677fe640 1 -- 172.21.15.81:0/2819680897 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168008e80 con 0x7f4170158150 2023-11-08T16:40:50.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.142+0000 7f41677fe640 1 -- 172.21.15.81:0/2819680897 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416800be50 con 0x7f4170158150 2023-11-08T16:40:50.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.142+0000 7f41677fe640 1 -- 172.21.15.81:0/2819680897 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4168009800 con 0x7f4170158150 2023-11-08T16:40:50.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.142+0000 7f4175778640 1 -- 172.21.15.81:0/2819680897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4170157170 con 0x7f4170158150 2023-11-08T16:40:50.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.142+0000 7f4175778640 1 -- 172.21.15.81:0/2819680897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41700b3300 con 0x7f4170158150 2023-11-08T16:40:50.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.143+0000 7f4175778640 1 -- 172.21.15.81:0/2819680897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4170072170 con 0x7f4170158150 2023-11-08T16:40:50.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.144+0000 7f41677fe640 1 -- 172.21.15.81:0/2819680897 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4168009960 con 0x7f4170158150 2023-11-08T16:40:50.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.145+0000 7f41677fe640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f415006aea0 0x7f415006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.145+0000 7f41677fe640 1 -- 172.21.15.81:0/2819680897 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f4168089ac0 con 0x7f4170158150 2023-11-08T16:40:50.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.145+0000 7f416effd640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f415006aea0 0x7f415006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.146+0000 7f416effd640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f415006aea0 0x7f415006d360 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f4160009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:50.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.149+0000 7f41677fe640 1 -- 172.21.15.81:0/2819680897 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f4168054850 con 0x7f4170158150 2023-11-08T16:40:50.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.340+0000 7f4175778640 1 -- 172.21.15.81:0/2819680897 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f417004ece0 con 0x7f4170158150 2023-11-08T16:40:50.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.342+0000 7f41677fe640 1 -- 172.21.15.81:0/2819680897 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) v1 ==== 74+0+18822 (secure 0 0 0) 0x7f4168057e70 con 0x7f4170158150 2023-11-08T16:40:50.346 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:50.347 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":114,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:39:57.654547+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:39:07.492834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-08T16:39:08.634345+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data_extra","create_time":"2023-11-08T16:39:19.654916+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.18","mappings":[{"from":1,"to":2}]},{"pgid":"14.1b","mappings":[{"from":1,"to":7}]},{"pgid":"14.26","mappings":[{"from":1,"to":7}]},{"pgid":"14.2a","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6836/3163533823":"2023-11-09T16:39:57.018336+0000","172.21.15.81:6836/3988749175":"2023-11-09T16:39:50.607108+0000","172.21.15.81:6837/3163533823":"2023-11-09T16:39:57.018336+0000","172.21.15.93:6835/691341796":"2023-11-09T16:39:48.576045+0000","172.21.15.81:6837/3988749175":"2023-11-09T16:39:50.607108+0000","172.21.15.93:6834/691341796":"2023-11-09T16:39:48.576045+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:40:50.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 -- 172.21.15.81:0/2819680897 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f415006aea0 msgr2=0x7f415006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f415006aea0 0x7f415006d360 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f4160009040 comp rx=0 tx=0).stop 2023-11-08T16:40:50.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 -- 172.21.15.81:0/2819680897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4170158150 msgr2=0x7f4170156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4170158150 0x7f4170156590 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f41700cb440 tx=0x7f416800b7a0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 -- 172.21.15.81:0/2819680897 shutdown_connections 2023-11-08T16:40:50.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f415006aea0 0x7f415006d360 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f4170158150 0x7f4170156590 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41700c5ae0 0x7f4170157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 --2- 172.21.15.81:0/2819680897 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f415c0069d0 0x7f4170154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 -- 172.21.15.81:0/2819680897 >> 172.21.15.81:0/2819680897 conn(0x7f41700cb490 msgr2=0x7f41700cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:50.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 -- 172.21.15.81:0/2819680897 shutdown_connections 2023-11-08T16:40:50.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.346+0000 7f4175778640 1 -- 172.21.15.81:0/2819680897 wait complete. 2023-11-08T16:40:50.366 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:40:50.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.574+0000 7f63a9a58640 1 Processor -- start 2023-11-08T16:40:50.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.575+0000 7f63a9a58640 1 -- start start 2023-11-08T16:40:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.576+0000 7f63a9a58640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63a406cf10 0x7f63a4161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.576+0000 7f63a9a58640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f63a40ccd80 0x7f63a40caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.576+0000 7f63a9a58640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a40cd580 con 0x7f63a40ca1a0 2023-11-08T16:40:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.576+0000 7f63a9a58640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a40c93e0 con 0x7f63a406cf10 2023-11-08T16:40:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.576+0000 7f63a9a58640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a40c96c0 con 0x7f63a40ccd80 2023-11-08T16:40:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.577+0000 7f63a2ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f63a406cf10 0x7f63a4161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.577+0000 7f639a7fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f63a40ca1a0 0x7f63a40cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:55640/0 (socket says 172.21.15.81:55640) 2023-11-08T16:40:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.577+0000 7f639a7fc640 1 -- 172.21.15.81:0/3185666267 learned_addr learned my addr 172.21.15.81:0/3185666267 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.577+0000 7f63a37fe640 1 --2- 172.21.15.81:0/3185666267 >> v2:172.21.15.93:3300/0 conn(0x7f63a40ccd80 0x7f63a40caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.577+0000 7f63a2ffd640 1 -- 172.21.15.81:0/3185666267 >> v2:172.21.15.93:3300/0 conn(0x7f63a40ccd80 msgr2=0x7f63a40caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.577+0000 7f63a2ffd640 1 --2- 172.21.15.81:0/3185666267 >> v2:172.21.15.93:3300/0 conn(0x7f63a40ccd80 0x7f63a40caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.577+0000 7f63a2ffd640 1 -- 172.21.15.81:0/3185666267 >> v1:172.21.15.81:6789/0 conn(0x7f63a40ca1a0 legacy=0x7f63a40cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.577+0000 7f63a2ffd640 1 -- 172.21.15.81:0/3185666267 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a40c99a0 con 0x7f63a406cf10 2023-11-08T16:40:50.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.578+0000 7f63a2ffd640 1 --2- 172.21.15.81:0/3185666267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63a406cf10 0x7f63a4161630 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f63a40cea40 tx=0x7f639402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4e63eb75ca9726a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:50.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.578+0000 7f63a27fc640 1 -- 172.21.15.81:0/3185666267 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394031830 con 0x7f63a406cf10 2023-11-08T16:40:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.579+0000 7f63a27fc640 1 -- 172.21.15.81:0/3185666267 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6394031990 con 0x7f63a406cf10 2023-11-08T16:40:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.579+0000 7f63a27fc640 1 -- 172.21.15.81:0/3185666267 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394031d20 con 0x7f63a406cf10 2023-11-08T16:40:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.579+0000 7f63a9a58640 1 -- 172.21.15.81:0/3185666267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63a406cf10 msgr2=0x7f63a4161630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.579+0000 7f63a9a58640 1 --2- 172.21.15.81:0/3185666267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63a406cf10 0x7f63a4161630 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f63a40cea40 tx=0x7f639402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.579+0000 7f63a9a58640 1 -- 172.21.15.81:0/3185666267 shutdown_connections 2023-11-08T16:40:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.579+0000 7f63a9a58640 1 --2- 172.21.15.81:0/3185666267 >> v2:172.21.15.93:3300/0 conn(0x7f63a40ccd80 0x7f63a40caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.579+0000 7f63a9a58640 1 --2- 172.21.15.81:0/3185666267 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63a406cf10 0x7f63a4161630 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.579+0000 7f63a9a58640 1 -- 172.21.15.81:0/3185666267 >> 172.21.15.81:0/3185666267 conn(0x7f63a40ceaa0 msgr2=0x7f63a40cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.579+0000 7f63a9a58640 1 -- 172.21.15.81:0/3185666267 shutdown_connections 2023-11-08T16:40:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.579+0000 7f63a9a58640 1 -- 172.21.15.81:0/3185666267 wait complete. 2023-11-08T16:40:50.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.581+0000 7f63a9a58640 1 Processor -- start 2023-11-08T16:40:50.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.581+0000 7f63a9a58640 1 -- start start 2023-11-08T16:40:50.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.582+0000 7f63a9a58640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63a406cf10 0x7f63a415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.582+0000 7f63a9a58640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63a40ca1a0 0x7f63a40cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.582+0000 7f63a9a58640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63a40ccd80 0x7f63a40cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.582+0000 7f63a9a58640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a41f3450 con 0x7f63a40ca1a0 2023-11-08T16:40:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.582+0000 7f63a9a58640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a41f2880 con 0x7f63a40ccd80 2023-11-08T16:40:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.582+0000 7f63a9a58640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a415b7d0 con 0x7f63a406cf10 2023-11-08T16:40:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.582+0000 7f639a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63a40ca1a0 0x7f63a40cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.582+0000 7f639a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63a40ca1a0 0x7f63a40cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:57400/0 (socket says 172.21.15.81:57400) 2023-11-08T16:40:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.582+0000 7f639a7fc640 1 -- 172.21.15.81:0/3229620506 learned_addr learned my addr 172.21.15.81:0/3229620506 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f63a2ffd640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63a406cf10 0x7f63a415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f63a37fe640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63a40ccd80 0x7f63a40cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f639a7fc640 1 -- 172.21.15.81:0/3229620506 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63a406cf10 msgr2=0x7f63a415b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f639a7fc640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63a406cf10 0x7f63a415b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f639a7fc640 1 -- 172.21.15.81:0/3229620506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63a40ccd80 msgr2=0x7f63a40cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f639a7fc640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63a40ccd80 0x7f63a40cb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f639a7fc640 1 -- 172.21.15.81:0/3229620506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63940093a0 con 0x7f63a40ca1a0 2023-11-08T16:40:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f639a7fc640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63a40ca1a0 0x7f63a40cb240 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f638c00c4a0 tx=0x7f638c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f63a0ff9640 1 -- 172.21.15.81:0/3229620506 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c005ce0 con 0x7f63a40ca1a0 2023-11-08T16:40:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f63a0ff9640 1 -- 172.21.15.81:0/3229620506 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638c005e40 con 0x7f63a40ca1a0 2023-11-08T16:40:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f63a0ff9640 1 -- 172.21.15.81:0/3229620506 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638c007270 con 0x7f63a40ca1a0 2023-11-08T16:40:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.583+0000 7f63a9a58640 1 -- 172.21.15.81:0/3229620506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a41f2b20 con 0x7f63a40ca1a0 2023-11-08T16:40:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.584+0000 7f63a9a58640 1 -- 172.21.15.81:0/3229620506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a41f30e0 con 0x7f63a40ca1a0 2023-11-08T16:40:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.585+0000 7f63a9a58640 1 -- 172.21.15.81:0/3229620506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63a41f3600 con 0x7f63a40ca1a0 2023-11-08T16:40:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.585+0000 7f63a0ff9640 1 -- 172.21.15.81:0/3229620506 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f638c0073d0 con 0x7f63a40ca1a0 2023-11-08T16:40:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.586+0000 7f63a0ff9640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f638406aea0 0x7f638406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.587+0000 7f63a0ff9640 1 -- 172.21.15.81:0/3229620506 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6487+0+0 (secure 0 0 0) 0x7f638c088dd0 con 0x7f63a40ca1a0 2023-11-08T16:40:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.587+0000 7f63a2ffd640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f638406aea0 0x7f638406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.588+0000 7f63a2ffd640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f638406aea0 0x7f638406d360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6394036000 tx=0x7f639403c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:50.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.591+0000 7f63a0ff9640 1 -- 172.21.15.81:0/3229620506 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f63a41f3600 con 0x7f63a40ca1a0 2023-11-08T16:40:50.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.818+0000 7f63a9a58640 1 -- 172.21.15.81:0/3229620506 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f63a40743f0 con 0x7f63a40ca1a0 2023-11-08T16:40:50.822 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:40:50.819+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:40:50.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.823+0000 7f63a0ff9640 1 -- 172.21.15.81:0/3229620506 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) v1 ==== 161+0+0 (secure 0 0 0) 0x7f638c017020 con 0x7f63a40ca1a0 2023-11-08T16:40:50.826 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:40:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 -- 172.21.15.81:0/3229620506 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f638406aea0 msgr2=0x7f638406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f638406aea0 0x7f638406d360 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6394036000 tx=0x7f639403c040 comp rx=0 tx=0).stop 2023-11-08T16:40:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 -- 172.21.15.81:0/3229620506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63a40ca1a0 msgr2=0x7f63a40cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63a40ca1a0 0x7f63a40cb240 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f638c00c4a0 tx=0x7f638c00c960 comp rx=0 tx=0).stop 2023-11-08T16:40:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 -- 172.21.15.81:0/3229620506 shutdown_connections 2023-11-08T16:40:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f638406aea0 0x7f638406d360 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f63a40ccd80 0x7f63a40cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f63a40ca1a0 0x7f63a40cb240 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 --2- 172.21.15.81:0/3229620506 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f63a406cf10 0x7f63a415b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:50.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 -- 172.21.15.81:0/3229620506 >> 172.21.15.81:0/3229620506 conn(0x7f63a40ceaa0 msgr2=0x7f63a41601a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:50.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 -- 172.21.15.81:0/3229620506 shutdown_connections 2023-11-08T16:40:50.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:50.830+0000 7f63a9a58640 1 -- 172.21.15.81:0/3229620506 wait complete. 2023-11-08T16:40:50.838 INFO:tasks.ceph.mds.c.smithi081.stderr: -6509> 2023-11-08T16:39:57.046+0000 7f9540baf640 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-11-08T16:40:50.843 INFO:tasks.ceph.mds.c.smithi081.stderr: -4629> 2023-11-08T16:40:04.606+0000 7f9540baf640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x4 2023-11-08T16:40:50.843 INFO:tasks.ceph.mds.c.smithi081.stderr: -4589> 2023-11-08T16:40:04.606+0000 7f9540baf640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-11-08T16:40:50.844 INFO:tasks.ceph.mds.c.smithi081.stderr: -4443> 2023-11-08T16:40:04.607+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /stripey/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55f906dbdb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x10000000000:[<0x1/stripey v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:40:50.844 INFO:tasks.ceph.mds.c.smithi081.stderr: -4331> 2023-11-08T16:40:04.608+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x4 [...2,head] /lost+found/ auth v1 pv6 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55f906dbe100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x4:[<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:40:50.845 INFO:tasks.ceph.mds.c.smithi081.stderr: -4116> 2023-11-08T16:40:04.609+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f9 [2,head] /stripey/flushed_file_4 auth v1 ap=1 DIRTYPARENT s=1441792 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e19180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001f9:[<0x10000000000/flushed_file_4 v70>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001f9:[<0x10000000000/flushed_file_4 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:50.846 INFO:tasks.ceph.mds.c.smithi081.stderr: -4092> 2023-11-08T16:40:04.610+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fb [2,head] /stripey/flushed_file_6 auth v1 ap=1 DIRTYPARENT s=1572864 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e18680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001fb:[<0x10000000000/flushed_file_6 v68>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001fb:[<0x10000000000/flushed_file_6 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:50.846 INFO:tasks.ceph.mds.c.smithi081.stderr: -4069> 2023-11-08T16:40:04.610+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f7 [2,head] /stripey/flushed_file_2 auth v1 ap=1 DIRTYPARENT s=917504 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e3db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001f7:[<0x10000000000/flushed_file_2 v56>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001f7:[<0x10000000000/flushed_file_2 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:50.846 INFO:tasks.ceph.mds.c.smithi081.stderr: -4044> 2023-11-08T16:40:04.610+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f8 [2,head] /stripey/flushed_file_3 auth v1 ap=1 DIRTYPARENT s=917503 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e19700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001f8:[<0x10000000000/flushed_file_3 v62>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001f8:[<0x10000000000/flushed_file_3 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:50.846 INFO:tasks.ceph.mds.c.smithi081.stderr: -4018> 2023-11-08T16:40:04.611+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /stripey/flushed_file_0 auth v1 ap=1 DIRTYPARENT s=1048576 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e3c580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x10000000001:[<0x10000000000/flushed_file_0 v36>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x10000000001:[<0x10000000000/flushed_file_0 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:50.847 INFO:tasks.ceph.mds.c.smithi081.stderr: -3999> 2023-11-08T16:40:04.611+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001f6 [2,head] /stripey/flushed_file_1 auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e3d600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001f6:[<0x10000000000/flushed_file_1 v48>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001f6:[<0x10000000000/flushed_file_1 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:50.847 INFO:tasks.ceph.mds.c.smithi081.stderr: -3983> 2023-11-08T16:40:04.611+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fc [2,head] /lost+found/100000001fc auth v1 ap=1 DIRTYPARENT s=524288 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e3c000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(15)0x100000001fc:[<0x4/100000001fc v1>,<0x1/lost+found v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2023-11-08T16:40:50.847 INFO:tasks.ceph.mds.c.smithi081.stderr: -3966> 2023-11-08T16:40:04.611+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000001fa [2,head] /stripey/flushed_file_5 auth v1 ap=1 DIRTYPARENT s=1441793 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x55f906e18c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(15)0x100000001fa:[<0x10000000000/flushed_file_5 v54>,<0x1/stripey v75>]//[14]","memoryvalue":"(15)0x100000001fa:[<0x10000000000/flushed_file_5 v1>,<0x1/stripey v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-08T16:40:50.847 INFO:tasks.ceph.mds.c.smithi081.stderr: -3478> 2023-11-08T16:40:04.621+0000 7f9540baf640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v7 ap=5 snaprealm=0x55f906db6fc0 DIRTYPARENT f(v1 2=0+2) n(v0 rc2023-11-08T16:40:04.337118+0000 2=0+2)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) caps={8274=pLsXs/-@3} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x55f906dbd080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(13)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 2=0+2)","ondisk_value.rstat":"n(v0 3=0+3)","memory_value.dirstat":"f(v1 2=0+2)","memory_value.rstat":"n(v0 rc2023-11-08T16:40:04.337118+0000 2=0+2)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-11-08T16:40:50.852 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T16:40:50.903 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:40:51.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.046+0000 7fcf96fae640 1 Processor -- start 2023-11-08T16:40:51.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.046+0000 7fcf96fae640 1 -- start start 2023-11-08T16:40:51.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf96fae640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcf900cc810 0x7fcf9006fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:51.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf96fae640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9006c430 con 0x7fcf9015d1d0 2023-11-08T16:40:51.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf96fae640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf9006b820 con 0x7fcf9015cdf0 2023-11-08T16:40:51.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf96fae640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf9006bb00 con 0x7fcf900cc810 2023-11-08T16:40:51.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf957ab640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcf9015d1d0 0x7fcf9006f6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49584/0 (socket says 172.21.15.81:49584) 2023-11-08T16:40:51.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf957ab640 1 -- 172.21.15.81:0/2193837991 learned_addr learned my addr 172.21.15.81:0/2193837991 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:51.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf967ad640 1 --2- 172.21.15.81:0/2193837991 >> v2:172.21.15.81:3300/0 conn(0x7fcf900cc810 0x7fcf9006fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf94faa640 1 -- 172.21.15.81:0/2193837991 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1125684639 0 0) 0x7fcf9006c430 con 0x7fcf9015d1d0 2023-11-08T16:40:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf94faa640 1 -- 172.21.15.81:0/2193837991 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf78003520 con 0x7fcf9015d1d0 2023-11-08T16:40:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf967ad640 1 -- 172.21.15.81:0/2193837991 >> v1:172.21.15.93:6789/0 conn(0x7fcf9015cdf0 legacy=0x7fcf9006c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf967ad640 1 -- 172.21.15.81:0/2193837991 >> v1:172.21.15.81:6789/0 conn(0x7fcf9015d1d0 legacy=0x7fcf9006f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.048+0000 7fcf967ad640 1 -- 172.21.15.81:0/2193837991 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf9006bde0 con 0x7fcf900cc810 2023-11-08T16:40:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.049+0000 7fcf967ad640 1 --2- 172.21.15.81:0/2193837991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf900cc810 0x7fcf9006fdd0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009d90 tx=0x7fcf8802be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52887d64b3a007ce server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.049+0000 7fcf94faa640 1 -- 172.21.15.81:0/2193837991 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8802cbe0 con 0x7fcf900cc810 2023-11-08T16:40:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.049+0000 7fcf94faa640 1 -- 172.21.15.81:0/2193837991 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8802cd40 con 0x7fcf900cc810 2023-11-08T16:40:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.049+0000 7fcf94faa640 1 -- 172.21.15.81:0/2193837991 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88034660 con 0x7fcf900cc810 2023-11-08T16:40:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.050+0000 7fcf96fae640 1 -- 172.21.15.81:0/2193837991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf900cc810 msgr2=0x7fcf9006fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.050+0000 7fcf96fae640 1 --2- 172.21.15.81:0/2193837991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf900cc810 0x7fcf9006fdd0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009d90 tx=0x7fcf8802be30 comp rx=0 tx=0).stop 2023-11-08T16:40:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.050+0000 7fcf96fae640 1 -- 172.21.15.81:0/2193837991 shutdown_connections 2023-11-08T16:40:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.050+0000 7fcf96fae640 1 --2- 172.21.15.81:0/2193837991 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf900cc810 0x7fcf9006fdd0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.050+0000 7fcf96fae640 1 -- 172.21.15.81:0/2193837991 >> 172.21.15.81:0/2193837991 conn(0x7fcf9006a6e0 msgr2=0x7fcf900b0600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:51.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.050+0000 7fcf96fae640 1 -- 172.21.15.81:0/2193837991 shutdown_connections 2023-11-08T16:40:51.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.050+0000 7fcf96fae640 1 -- 172.21.15.81:0/2193837991 wait complete. 2023-11-08T16:40:51.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.052+0000 7fcf96fae640 1 Processor -- start 2023-11-08T16:40:51.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.052+0000 7fcf96fae640 1 -- start start 2023-11-08T16:40:51.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf96fae640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf900cc810 0x7fcf9014db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:51.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf96fae640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf9015cdf0 0x7fcf9015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:51.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf96fae640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf9015d1d0 0x7fcf9015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:51.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf96fae640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90150c90 con 0x7fcf9015d1d0 2023-11-08T16:40:51.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf96fae640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9014f9a0 con 0x7fcf900cc810 2023-11-08T16:40:51.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf96fae640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9014fc80 con 0x7fcf9015cdf0 2023-11-08T16:40:51.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf967ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf9015d1d0 0x7fcf9015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:51.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf967ad640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf9015d1d0 0x7fcf9015bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53414/0 (socket says 172.21.15.81:53414) 2023-11-08T16:40:51.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf967ad640 1 -- 172.21.15.81:0/3124126773 learned_addr learned my addr 172.21.15.81:0/3124126773 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:51.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf967ad640 1 -- 172.21.15.81:0/3124126773 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf9015cdf0 msgr2=0x7fcf9015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:51.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf95fac640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf900cc810 0x7fcf9014db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:51.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf967ad640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf9015cdf0 0x7fcf9015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:51.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf967ad640 1 -- 172.21.15.81:0/3124126773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf900cc810 msgr2=0x7fcf9014db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:51.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf967ad640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf900cc810 0x7fcf9014db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:51.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf967ad640 1 -- 172.21.15.81:0/3124126773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf88009a70 con 0x7fcf9015d1d0 2023-11-08T16:40:51.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.053+0000 7fcf95fac640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf900cc810 0x7fcf9014db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:51.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.054+0000 7fcf967ad640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf9015d1d0 0x7fcf9015bf00 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009d90 tx=0x7fcf88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:51.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.054+0000 7fcf867fc640 1 -- 172.21.15.81:0/3124126773 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88040650 con 0x7fcf9015d1d0 2023-11-08T16:40:51.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.054+0000 7fcf867fc640 1 -- 172.21.15.81:0/3124126773 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf880407b0 con 0x7fcf9015d1d0 2023-11-08T16:40:51.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.054+0000 7fcf867fc640 1 -- 172.21.15.81:0/3124126773 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88040b60 con 0x7fcf9015d1d0 2023-11-08T16:40:51.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.054+0000 7fcf96fae640 1 -- 172.21.15.81:0/3124126773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9014ff60 con 0x7fcf9015d1d0 2023-11-08T16:40:51.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.054+0000 7fcf96fae640 1 -- 172.21.15.81:0/3124126773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf90150520 con 0x7fcf9015d1d0 2023-11-08T16:40:51.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.055+0000 7fcf96fae640 1 -- 172.21.15.81:0/3124126773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf90067800 con 0x7fcf9015d1d0 2023-11-08T16:40:51.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.056+0000 7fcf867fc640 1 -- 172.21.15.81:0/3124126773 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcf88040cc0 con 0x7fcf9015d1d0 2023-11-08T16:40:51.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.058+0000 7fcf867fc640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf7c06ae70 0x7fcf7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:51.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.058+0000 7fcf867fc640 1 -- 172.21.15.81:0/3124126773 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6573+0+0 (secure 0 0 0) 0x7fcf880ac810 con 0x7fcf9015d1d0 2023-11-08T16:40:51.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.058+0000 7fcf95fac640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf7c06ae70 0x7fcf7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:51.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.058+0000 7fcf95fac640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf7c06ae70 0x7fcf7c06d330 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcf80000f60 tx=0x7fcf80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:51.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.062+0000 7fcf867fc640 1 -- 172.21.15.81:0/3124126773 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf880774c0 con 0x7fcf9015d1d0 2023-11-08T16:40:51.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.296+0000 7fcf96fae640 1 -- 172.21.15.81:0/3124126773 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcf900cb8f0 con 0x7fcf9015d1d0 2023-11-08T16:40:51.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.826+0000 7fcf867fc640 1 -- 172.21.15.81:0/3124126773 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v102) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcf8807aae0 con 0x7fcf9015d1d0 2023-11-08T16:40:51.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 -- 172.21.15.81:0/3124126773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf7c06ae70 msgr2=0x7fcf7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:51.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf7c06ae70 0x7fcf7c06d330 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fcf80000f60 tx=0x7fcf80009040 comp rx=0 tx=0).stop 2023-11-08T16:40:51.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 -- 172.21.15.81:0/3124126773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf9015d1d0 msgr2=0x7fcf9015bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:51.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf9015d1d0 0x7fcf9015bf00 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009d90 tx=0x7fcf88004a40 comp rx=0 tx=0).stop 2023-11-08T16:40:51.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 -- 172.21.15.81:0/3124126773 shutdown_connections 2023-11-08T16:40:51.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf7c06ae70 0x7fcf7c06d330 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:51.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf9015d1d0 0x7fcf9015bf00 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:51.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf9015cdf0 0x7fcf9015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:51.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 --2- 172.21.15.81:0/3124126773 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf900cc810 0x7fcf9014db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:51.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 -- 172.21.15.81:0/3124126773 >> 172.21.15.81:0/3124126773 conn(0x7fcf9006a6e0 msgr2=0x7fcf900ccfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:51.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 -- 172.21.15.81:0/3124126773 shutdown_connections 2023-11-08T16:40:51.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:51.830+0000 7fcf96fae640 1 -- 172.21.15.81:0/3124126773 wait complete. 2023-11-08T16:40:51.854 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T16:40:52.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.066+0000 7fa693fee640 1 Processor -- start 2023-11-08T16:40:52.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.066+0000 7fa693fee640 1 -- start start 2023-11-08T16:40:52.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.068+0000 7fa693fee640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa68c0714b0 0x7fa68c15a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:52.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.068+0000 7fa693fee640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa68c071890 0x7fa68c15ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:52.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.068+0000 7fa693fee640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68c15b340 con 0x7fa68c0adf00 2023-11-08T16:40:52.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.068+0000 7fa693fee640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68c06b250 con 0x7fa68c071890 2023-11-08T16:40:52.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.068+0000 7fa693fee640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68c0b09f0 con 0x7fa68c0714b0 2023-11-08T16:40:52.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.068+0000 7fa69258b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa68c0adf00 0x7fa68c0c90b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49596/0 (socket says 172.21.15.81:49596) 2023-11-08T16:40:52.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.068+0000 7fa691589640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa68c071890 0x7fa68c15ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:52.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.068+0000 7fa69258b640 1 -- 172.21.15.81:0/3229987648 learned_addr learned my addr 172.21.15.81:0/3229987648 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:52.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.069+0000 7fa691d8a640 1 --2- 172.21.15.81:0/3229987648 >> v2:172.21.15.93:3300/0 conn(0x7fa68c0714b0 0x7fa68c15a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:52.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.069+0000 7fa691589640 1 -- 172.21.15.81:0/3229987648 >> v2:172.21.15.93:3300/0 conn(0x7fa68c0714b0 msgr2=0x7fa68c15a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.069+0000 7fa691589640 1 --2- 172.21.15.81:0/3229987648 >> v2:172.21.15.93:3300/0 conn(0x7fa68c0714b0 0x7fa68c15a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.069+0000 7fa691589640 1 -- 172.21.15.81:0/3229987648 >> v1:172.21.15.81:6789/0 conn(0x7fa68c0adf00 legacy=0x7fa68c0c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.069+0000 7fa691589640 1 -- 172.21.15.81:0/3229987648 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68c0ca360 con 0x7fa68c071890 2023-11-08T16:40:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.069+0000 7fa690d88640 1 -- 172.21.15.81:0/3229987648 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370205350 0 0) 0x7fa68c15b340 con 0x7fa68c0adf00 2023-11-08T16:40:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.070+0000 7fa691589640 1 --2- 172.21.15.81:0/3229987648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa68c071890 0x7fa68c15ae00 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa68c06c3f0 tx=0x7fa68002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33cc0bae62f6645f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.070+0000 7fa690d88640 1 -- 172.21.15.81:0/3229987648 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680031830 con 0x7fa68c071890 2023-11-08T16:40:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.070+0000 7fa690d88640 1 -- 172.21.15.81:0/3229987648 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa680031990 con 0x7fa68c071890 2023-11-08T16:40:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.070+0000 7fa690d88640 1 -- 172.21.15.81:0/3229987648 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680031d20 con 0x7fa68c071890 2023-11-08T16:40:52.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.071+0000 7fa693fee640 1 -- 172.21.15.81:0/3229987648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa68c071890 msgr2=0x7fa68c15ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:52.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.071+0000 7fa693fee640 1 --2- 172.21.15.81:0/3229987648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa68c071890 0x7fa68c15ae00 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa68c06c3f0 tx=0x7fa68002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:40:52.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.071+0000 7fa693fee640 1 -- 172.21.15.81:0/3229987648 shutdown_connections 2023-11-08T16:40:52.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.071+0000 7fa693fee640 1 --2- 172.21.15.81:0/3229987648 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa68c071890 0x7fa68c15ae00 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:52.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.071+0000 7fa693fee640 1 --2- 172.21.15.81:0/3229987648 >> v2:172.21.15.93:3300/0 conn(0x7fa68c0714b0 0x7fa68c15a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:52.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.071+0000 7fa693fee640 1 -- 172.21.15.81:0/3229987648 >> 172.21.15.81:0/3229987648 conn(0x7fa68c0cb4e0 msgr2=0x7fa68c0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:52.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.071+0000 7fa693fee640 1 -- 172.21.15.81:0/3229987648 shutdown_connections 2023-11-08T16:40:52.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.071+0000 7fa693fee640 1 -- 172.21.15.81:0/3229987648 wait complete. 2023-11-08T16:40:52.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.073+0000 7fa693fee640 1 Processor -- start 2023-11-08T16:40:52.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.073+0000 7fa693fee640 1 -- start start 2023-11-08T16:40:52.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa693fee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa68c0714b0 0x7fa68c0c4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:52.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa693fee640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa68c071890 0x7fa68c0c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:52.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa693fee640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa68c0adf00 0x7fa68c0cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:52.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa693fee640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c0c7bc0 con 0x7fa68c0714b0 2023-11-08T16:40:52.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa693fee640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c0c6b00 con 0x7fa68c0adf00 2023-11-08T16:40:52.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa693fee640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68c0c7290 con 0x7fa68c071890 2023-11-08T16:40:52.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa691d8a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa68c0714b0 0x7fa68c0c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:52.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa691d8a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa68c0714b0 0x7fa68c0c4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53428/0 (socket says 172.21.15.81:53428) 2023-11-08T16:40:52.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa691d8a640 1 -- 172.21.15.81:0/3072182136 learned_addr learned my addr 172.21.15.81:0/3072182136 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:52.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa691d8a640 1 -- 172.21.15.81:0/3072182136 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa68c071890 msgr2=0x7fa68c0c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:52.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa691d8a640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa68c071890 0x7fa68c0c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:52.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa691d8a640 1 -- 172.21.15.81:0/3072182136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa68c0adf00 msgr2=0x7fa68c0cd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:52.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa691d8a640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa68c0adf00 0x7fa68c0cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:52.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa691d8a640 1 -- 172.21.15.81:0/3072182136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa680009400 con 0x7fa68c0714b0 2023-11-08T16:40:52.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa691d8a640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa68c0714b0 0x7fa68c0c4570 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fa68800d800 tx=0x7fa68800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:52.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa67e7fc640 1 -- 172.21.15.81:0/3072182136 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800b6c0 con 0x7fa68c0714b0 2023-11-08T16:40:52.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa67e7fc640 1 -- 172.21.15.81:0/3072182136 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68800b820 con 0x7fa68c0714b0 2023-11-08T16:40:52.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa693fee640 1 -- 172.21.15.81:0/3072182136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa68c0c7570 con 0x7fa68c0714b0 2023-11-08T16:40:52.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa67e7fc640 1 -- 172.21.15.81:0/3072182136 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800bbb0 con 0x7fa68c0714b0 2023-11-08T16:40:52.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.075+0000 7fa693fee640 1 -- 172.21.15.81:0/3072182136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa68c0c4ab0 con 0x7fa68c0714b0 2023-11-08T16:40:52.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.076+0000 7fa693fee640 1 -- 172.21.15.81:0/3072182136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68c0743f0 con 0x7fa68c0714b0 2023-11-08T16:40:52.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.077+0000 7fa67e7fc640 1 -- 172.21.15.81:0/3072182136 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa68800bd10 con 0x7fa68c0714b0 2023-11-08T16:40:52.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.078+0000 7fa67e7fc640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa67006ae70 0x7fa67006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:52.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.078+0000 7fa67e7fc640 1 -- 172.21.15.81:0/3072182136 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6573+0+0 (secure 0 0 0) 0x7fa688089fb0 con 0x7fa68c0714b0 2023-11-08T16:40:52.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.078+0000 7fa691589640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa67006ae70 0x7fa67006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:52.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.079+0000 7fa691589640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa67006ae70 0x7fa67006d330 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa680036000 tx=0x7fa680030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:52.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.083+0000 7fa67e7fc640 1 -- 172.21.15.81:0/3072182136 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa688054c60 con 0x7fa68c0714b0 2023-11-08T16:40:52.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:52.276+0000 7fa693fee640 1 -- 172.21.15.81:0/3072182136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa68c06c8b0 con 0x7fa68c0714b0 2023-11-08T16:40:53.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.114+0000 7fa67e7fc640 1 -- 172.21.15.81:0/3072182136 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa688058280 con 0x7fa68c0714b0 2023-11-08T16:40:53.117 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' removed 2023-11-08T16:40:53.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.120+0000 7fa693fee640 1 -- 172.21.15.81:0/3072182136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa67006ae70 msgr2=0x7fa67006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:53.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.120+0000 7fa693fee640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa67006ae70 0x7fa67006d330 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa680036000 tx=0x7fa680030000 comp rx=0 tx=0).stop 2023-11-08T16:40:53.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.120+0000 7fa693fee640 1 -- 172.21.15.81:0/3072182136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa68c0714b0 msgr2=0x7fa68c0c4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:53.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.120+0000 7fa693fee640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa68c0714b0 0x7fa68c0c4570 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fa68800d800 tx=0x7fa68800dcc0 comp rx=0 tx=0).stop 2023-11-08T16:40:53.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.121+0000 7fa693fee640 1 -- 172.21.15.81:0/3072182136 shutdown_connections 2023-11-08T16:40:53.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.121+0000 7fa693fee640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa67006ae70 0x7fa67006d330 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:53.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.121+0000 7fa693fee640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa68c0adf00 0x7fa68c0cd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:53.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.121+0000 7fa693fee640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa68c071890 0x7fa68c0c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:53.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.121+0000 7fa693fee640 1 --2- 172.21.15.81:0/3072182136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa68c0714b0 0x7fa68c0c4570 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:53.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.121+0000 7fa693fee640 1 -- 172.21.15.81:0/3072182136 >> 172.21.15.81:0/3072182136 conn(0x7fa68c0cb4e0 msgr2=0x7fa68c06b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:53.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.121+0000 7fa693fee640 1 -- 172.21.15.81:0/3072182136 shutdown_connections 2023-11-08T16:40:53.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.121+0000 7fa693fee640 1 -- 172.21.15.81:0/3072182136 wait complete. 2023-11-08T16:40:53.145 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T16:40:53.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.353+0000 7fb801f6e640 1 Processor -- start 2023-11-08T16:40:53.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.353+0000 7fb801f6e640 1 -- start start 2023-11-08T16:40:53.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.355+0000 7fb801f6e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb7fc0cab40 0x7fb7fc0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:53.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.355+0000 7fb801f6e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7fc160b40 con 0x7fb7fc06b250 2023-11-08T16:40:53.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.355+0000 7fb801f6e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7fc0caf20 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.355+0000 7fb801f6e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7fc15db40 con 0x7fb7fc0cab40 2023-11-08T16:40:53.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.355+0000 7fb7fb7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb7fc0cab40 0x7fb7fc0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:53.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.355+0000 7fb7f3fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb7fc06b250 0x7fb7fc0b04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49608/0 (socket says 172.21.15.81:49608) 2023-11-08T16:40:53.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.355+0000 7fb7f3fff640 1 -- 172.21.15.81:0/2994855540 learned_addr learned my addr 172.21.15.81:0/2994855540 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:53.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.356+0000 7fb7fb7fe640 1 -- 172.21.15.81:0/2994855540 >> v1:172.21.15.93:6789/0 conn(0x7fb7fc0b0bf0 legacy=0x7fb7fc160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:53.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.356+0000 7fb7fb7fe640 1 -- 172.21.15.81:0/2994855540 >> v1:172.21.15.81:6789/0 conn(0x7fb7fc06b250 legacy=0x7fb7fc0b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:53.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.356+0000 7fb7faffd640 1 -- 172.21.15.81:0/2994855540 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1442774736 0 0) 0x7fb7fc160b40 con 0x7fb7fc06b250 2023-11-08T16:40:53.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.356+0000 7fb7fb7fe640 1 -- 172.21.15.81:0/2994855540 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc15de20 con 0x7fb7fc0cab40 2023-11-08T16:40:53.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.357+0000 7fb7fb7fe640 1 --2- 172.21.15.81:0/2994855540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7fc0cab40 0x7fb7fc0cd040 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0097a0 tx=0x7fb7ec02b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f3be1288ba290e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:53.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.357+0000 7fb7faffd640 1 -- 172.21.15.81:0/2994855540 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec02f050 con 0x7fb7fc0cab40 2023-11-08T16:40:53.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.357+0000 7fb7faffd640 1 -- 172.21.15.81:0/2994855540 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7ec031700 con 0x7fb7fc0cab40 2023-11-08T16:40:53.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.357+0000 7fb7faffd640 1 -- 172.21.15.81:0/2994855540 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec031a90 con 0x7fb7fc0cab40 2023-11-08T16:40:53.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.358+0000 7fb801f6e640 1 -- 172.21.15.81:0/2994855540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7fc0cab40 msgr2=0x7fb7fc0cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:53.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.358+0000 7fb801f6e640 1 --2- 172.21.15.81:0/2994855540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7fc0cab40 0x7fb7fc0cd040 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec0097a0 tx=0x7fb7ec02b7a0 comp rx=0 tx=0).stop 2023-11-08T16:40:53.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.358+0000 7fb801f6e640 1 -- 172.21.15.81:0/2994855540 shutdown_connections 2023-11-08T16:40:53.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.358+0000 7fb801f6e640 1 --2- 172.21.15.81:0/2994855540 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7fc0cab40 0x7fb7fc0cd040 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:53.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.358+0000 7fb801f6e640 1 -- 172.21.15.81:0/2994855540 >> 172.21.15.81:0/2994855540 conn(0x7fb7fc0cb4e0 msgr2=0x7fb7fc0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:53.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.358+0000 7fb801f6e640 1 -- 172.21.15.81:0/2994855540 shutdown_connections 2023-11-08T16:40:53.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.358+0000 7fb801f6e640 1 -- 172.21.15.81:0/2994855540 wait complete. 2023-11-08T16:40:53.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.360+0000 7fb801f6e640 1 Processor -- start 2023-11-08T16:40:53.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.360+0000 7fb801f6e640 1 -- start start 2023-11-08T16:40:53.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.361+0000 7fb801f6e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb7fc06b250 0x7fb7fc0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:53.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.361+0000 7fb801f6e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7fc0b0bf0 0x7fb7fc0c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:53.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.361+0000 7fb801f6e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7fc0cab40 0x7fb7fc14def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:53.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.361+0000 7fb801f6e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc0c7040 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.361+0000 7fb801f6e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc0c6d30 con 0x7fb7fc0cab40 2023-11-08T16:40:53.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.361+0000 7fb801f6e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc0c6380 con 0x7fb7fc06b250 2023-11-08T16:40:53.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.361+0000 7fb7f3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7fc0b0bf0 0x7fb7fc0c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:53.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.361+0000 7fb7f3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7fc0b0bf0 0x7fb7fc0c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53452/0 (socket says 172.21.15.81:53452) 2023-11-08T16:40:53.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.361+0000 7fb7f3fff640 1 -- 172.21.15.81:0/1493893333 learned_addr learned my addr 172.21.15.81:0/1493893333 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:53.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7fb7fe640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb7fc06b250 0x7fb7fc0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:53.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7fbfff640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7fc0cab40 0x7fb7fc14def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:53.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7f3fff640 1 -- 172.21.15.81:0/1493893333 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb7fc06b250 msgr2=0x7fb7fc0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:53.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7f3fff640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb7fc06b250 0x7fb7fc0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:53.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7f3fff640 1 -- 172.21.15.81:0/1493893333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7fc0cab40 msgr2=0x7fb7fc14def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:53.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7f3fff640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7fc0cab40 0x7fb7fc14def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:53.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7f3fff640 1 -- 172.21.15.81:0/1493893333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7ec009400 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7fb7fe640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb7fc06b250 0x7fb7fc0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:53.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7fbfff640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7fc0cab40 0x7fb7fc14def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:53.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7f3fff640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7fc0b0bf0 0x7fb7fc0c5b30 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400c4d0 tx=0x7fb7e400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:53.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7f8ff9640 1 -- 172.21.15.81:0/1493893333 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e40074e0 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7f8ff9640 1 -- 172.21.15.81:0/1493893333 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e4007660 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb7f8ff9640 1 -- 172.21.15.81:0/1493893333 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e4004a20 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb801f6e640 1 -- 172.21.15.81:0/1493893333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc0c6660 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.362+0000 7fb801f6e640 1 -- 172.21.15.81:0/1493893333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc0c6070 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.363+0000 7fb801f6e640 1 -- 172.21.15.81:0/1493893333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7fc0c7180 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.364+0000 7fb7f8ff9640 1 -- 172.21.15.81:0/1493893333 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb7e4004b80 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.365+0000 7fb7f8ff9640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb7dc06ae70 0x7fb7dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:53.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.365+0000 7fb7f8ff9640 1 -- 172.21.15.81:0/1493893333 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6135+0+0 (secure 0 0 0) 0x7fb7e4088990 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.366+0000 7fb7fb7fe640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb7dc06ae70 0x7fb7dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:53.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.366+0000 7fb7fb7fe640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb7dc06ae70 0x7fb7dc06d330 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec035000 tx=0x7fb7ec008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:53.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.370+0000 7fb7f8ff9640 1 -- 172.21.15.81:0/1493893333 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb7fc0c7180 con 0x7fb7fc0b0bf0 2023-11-08T16:40:53.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:53.565+0000 7fb801f6e640 1 -- 172.21.15.81:0/1493893333 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb7fc0c7180 con 0x7fb7fc0b0bf0 2023-11-08T16:40:54.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.135+0000 7fb7f8ff9640 1 -- 172.21.15.81:0/1493893333 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v117) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb7fc0c7180 con 0x7fb7fc0b0bf0 2023-11-08T16:40:54.137 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' removed 2023-11-08T16:40:54.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.138+0000 7fb801f6e640 1 -- 172.21.15.81:0/1493893333 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb7dc06ae70 msgr2=0x7fb7dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.139+0000 7fb801f6e640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb7dc06ae70 0x7fb7dc06d330 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec035000 tx=0x7fb7ec008030 comp rx=0 tx=0).stop 2023-11-08T16:40:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.139+0000 7fb801f6e640 1 -- 172.21.15.81:0/1493893333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7fc0b0bf0 msgr2=0x7fb7fc0c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.139+0000 7fb801f6e640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7fc0b0bf0 0x7fb7fc0c5b30 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fb7e400c4d0 tx=0x7fb7e400c990 comp rx=0 tx=0).stop 2023-11-08T16:40:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.139+0000 7fb801f6e640 1 -- 172.21.15.81:0/1493893333 shutdown_connections 2023-11-08T16:40:54.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.139+0000 7fb801f6e640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb7dc06ae70 0x7fb7dc06d330 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.139+0000 7fb801f6e640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb7fc0cab40 0x7fb7fc14def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.139+0000 7fb801f6e640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb7fc0b0bf0 0x7fb7fc0c5b30 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.139+0000 7fb801f6e640 1 --2- 172.21.15.81:0/1493893333 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb7fc06b250 0x7fb7fc0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:54.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.139+0000 7fb801f6e640 1 -- 172.21.15.81:0/1493893333 >> 172.21.15.81:0/1493893333 conn(0x7fb7fc0cb4e0 msgr2=0x7fb7fc06f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:54.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.139+0000 7fb801f6e640 1 -- 172.21.15.81:0/1493893333 shutdown_connections 2023-11-08T16:40:54.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.139+0000 7fb801f6e640 1 -- 172.21.15.81:0/1493893333 wait complete. 2023-11-08T16:40:54.161 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_extra cephfs_data_extra --yes-i-really-really-mean-it 2023-11-08T16:40:54.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.368+0000 7f17a7f4c640 1 Processor -- start 2023-11-08T16:40:54.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.368+0000 7f17a7f4c640 1 -- start start 2023-11-08T16:40:54.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.369+0000 7f17a7f4c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c97c0 0x7f17a0165370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:54.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.369+0000 7f17a7f4c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a006b960 con 0x7f17a00b0550 2023-11-08T16:40:54.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.369+0000 7f17a7f4c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a006bc30 con 0x7f17a00b0930 2023-11-08T16:40:54.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.369+0000 7f17a7f4c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a006bf10 con 0x7f17a00c97c0 2023-11-08T16:40:54.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.370+0000 7f17a5ce8640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f17a00b0550 0x7f17a006b250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49616/0 (socket says 172.21.15.81:49616) 2023-11-08T16:40:54.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.370+0000 7f17a5ce8640 1 -- 172.21.15.81:0/3257285051 learned_addr learned my addr 172.21.15.81:0/3257285051 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:54.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.370+0000 7f17a64e9640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c97c0 0x7f17a0165370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:54.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.370+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316847687 0 0) 0x7f17a006b960 con 0x7f17a00b0550 2023-11-08T16:40:54.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.370+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f178c003520 con 0x7f17a00b0550 2023-11-08T16:40:54.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.370+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1736384824 0 0) 0x7f178c003520 con 0x7f17a00b0550 2023-11-08T16:40:54.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.370+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c97c0 msgr2=0x7f17a0165370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:54.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.370+0000 7f17a4ce6640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c97c0 0x7f17a0165370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:54.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 >> v1:172.21.15.93:6789/0 conn(0x7f17a00b0930 legacy=0x7f17a00c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:54.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a006c1f0 con 0x7f17a00b0550 2023-11-08T16:40:54.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f179c001670 con 0x7f17a00b0550 2023-11-08T16:40:54.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 >> v1:172.21.15.81:6789/0 conn(0x7f17a00b0550 legacy=0x7f17a006b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:54.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a4ce6640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 0x7f178c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:54.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a4ce6640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f178c0047c0 0x7f178c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:54.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a4ce6640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c97c0 0x7f178c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:54.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a006bf10 con 0x7f178c0039a0 2023-11-08T16:40:54.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178c00ba60 con 0x7f178c0047c0 2023-11-08T16:40:54.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f178c00de30 con 0x7f17a00c97c0 2023-11-08T16:40:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a5ce8640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 0x7f178c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.371+0000 7f17a54e7640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f178c0047c0 0x7f178c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.372+0000 7f17a64e9640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c97c0 0x7f178c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:54.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.372+0000 7f17a5ce8640 1 -- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c97c0 msgr2=0x7f178c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.372+0000 7f17a5ce8640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c97c0 0x7f178c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.372+0000 7f17a5ce8640 1 -- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f178c0047c0 msgr2=0x7f178c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.372+0000 7f17a5ce8640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f178c0047c0 0x7f178c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.372+0000 7f17a5ce8640 1 -- 172.21.15.81:0/3257285051 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f178c003520 con 0x7f178c0039a0 2023-11-08T16:40:54.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.372+0000 7f17a54e7640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f178c0047c0 0x7f178c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.372+0000 7f17a64e9640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c97c0 0x7f178c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.373+0000 7f17a5ce8640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 0x7f178c003dc0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f179c00c210 tx=0x7f178c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ebbe3f8c80dda3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.373+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c005ad0 con 0x7f178c0039a0 2023-11-08T16:40:54.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.373+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c005c30 con 0x7f178c0039a0 2023-11-08T16:40:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.373+0000 7f17a4ce6640 1 -- 172.21.15.81:0/3257285051 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c02c870 con 0x7f178c0039a0 2023-11-08T16:40:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.373+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3257285051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 msgr2=0x7f178c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.373+0000 7f17a7f4c640 1 --2- 172.21.15.81:0/3257285051 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 0x7f178c003dc0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f179c00c210 tx=0x7f178c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:40:54.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.374+0000 7f17a64e9640 1 -- 172.21.15.81:0/3257285051 reap_dead start 2023-11-08T16:40:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.374+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3257285051 shutdown_connections 2023-11-08T16:40:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.374+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3257285051 >> 172.21.15.81:0/3257285051 conn(0x7f17a00cb4e0 msgr2=0x7f17a00cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.374+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3257285051 shutdown_connections 2023-11-08T16:40:54.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.374+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3257285051 wait complete. 2023-11-08T16:40:54.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.376+0000 7f17a7f4c640 1 Processor -- start 2023-11-08T16:40:54.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.376+0000 7f17a7f4c640 1 -- start start 2023-11-08T16:40:54.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.377+0000 7f17a7f4c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 0x7f17a00b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.377+0000 7f17a7f4c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c5b30 0x7f17a014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.377+0000 7f17a7f4c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17a00b15f0 0x7f17a00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.377+0000 7f17a7f4c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a00c64f0 con 0x7f178c0039a0 2023-11-08T16:40:54.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.377+0000 7f17a7f4c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a015cb60 con 0x7f17a00b15f0 2023-11-08T16:40:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.377+0000 7f17a7f4c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a014e980 con 0x7f17a00c5b30 2023-11-08T16:40:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.377+0000 7f17a5ce8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 0x7f17a00b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.377+0000 7f17a5ce8640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 0x7f17a00b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53456/0 (socket says 172.21.15.81:53456) 2023-11-08T16:40:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.377+0000 7f17a5ce8640 1 -- 172.21.15.81:0/3453125994 learned_addr learned my addr 172.21.15.81:0/3453125994 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:54.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.378+0000 7f17a64e9640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17a00b15f0 0x7f17a00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.378+0000 7f17a54e7640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c5b30 0x7f17a014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.378+0000 7f17a5ce8640 1 -- 172.21.15.81:0/3453125994 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c5b30 msgr2=0x7f17a014e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.378+0000 7f17a5ce8640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c5b30 0x7f17a014e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:54.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.378+0000 7f17a5ce8640 1 -- 172.21.15.81:0/3453125994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17a00b15f0 msgr2=0x7f17a00ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.378+0000 7f17a5ce8640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17a00b15f0 0x7f17a00ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.378+0000 7f17a5ce8640 1 -- 172.21.15.81:0/3453125994 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a00c5f90 con 0x7f178c0039a0 2023-11-08T16:40:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.378+0000 7f17a5ce8640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 0x7f17a00b41d0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f179c02c410 tx=0x7f179c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.378+0000 7f17927fc640 1 -- 172.21.15.81:0/3453125994 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c03c040 con 0x7f178c0039a0 2023-11-08T16:40:54.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.379+0000 7f17927fc640 1 -- 172.21.15.81:0/3453125994 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f179c02cca0 con 0x7f178c0039a0 2023-11-08T16:40:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.379+0000 7f17927fc640 1 -- 172.21.15.81:0/3453125994 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f179c0072b0 con 0x7f178c0039a0 2023-11-08T16:40:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.379+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3453125994 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a015bf50 con 0x7f178c0039a0 2023-11-08T16:40:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.379+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3453125994 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a015c510 con 0x7f178c0039a0 2023-11-08T16:40:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.380+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3453125994 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a00743f0 con 0x7f178c0039a0 2023-11-08T16:40:54.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.381+0000 7f17927fc640 1 -- 172.21.15.81:0/3453125994 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f179c032070 con 0x7f178c0039a0 2023-11-08T16:40:54.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.382+0000 7f17927fc640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f178406ae70 0x7f178406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:54.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.382+0000 7f17927fc640 1 -- 172.21.15.81:0/3453125994 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5605+0+0 (secure 0 0 0) 0x7f179c0aaaf0 con 0x7f178c0039a0 2023-11-08T16:40:54.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.382+0000 7f17a54e7640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f178406ae70 0x7f178406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:54.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.383+0000 7f17a54e7640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f178406ae70 0x7f178406d330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f17a0071670 tx=0x7f1794009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:54.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.386+0000 7f17927fc640 1 -- 172.21.15.81:0/3453125994 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f179c075b60 con 0x7f178c0039a0 2023-11-08T16:40:54.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:54.580+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3453125994 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f17a00b13d0 con 0x7f178c0039a0 2023-11-08T16:40:55.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.146+0000 7f17927fc640 1 -- 172.21.15.81:0/3453125994 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_extra", "pool2": "cephfs_data_extra", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_extra' removed v118) v1 ==== 187+0+0 (secure 0 0 0) 0x7f179c046070 con 0x7f178c0039a0 2023-11-08T16:40:55.148 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data_extra' removed 2023-11-08T16:40:55.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.149+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3453125994 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f178406ae70 msgr2=0x7f178406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:55.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.149+0000 7f17a7f4c640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f178406ae70 0x7f178406d330 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f17a0071670 tx=0x7f1794009040 comp rx=0 tx=0).stop 2023-11-08T16:40:55.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.149+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3453125994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 msgr2=0x7f17a00b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:55.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.149+0000 7f17a7f4c640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 0x7f17a00b41d0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f179c02c410 tx=0x7f179c001a00 comp rx=0 tx=0).stop 2023-11-08T16:40:55.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.150+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3453125994 shutdown_connections 2023-11-08T16:40:55.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.150+0000 7f17a7f4c640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f178406ae70 0x7f178406d330 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.150+0000 7f17a7f4c640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17a00b15f0 0x7f17a00ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.150+0000 7f17a7f4c640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f17a00c5b30 0x7f17a014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.150+0000 7f17a7f4c640 1 --2- 172.21.15.81:0/3453125994 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f178c0039a0 0x7f17a00b41d0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.150+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3453125994 >> 172.21.15.81:0/3453125994 conn(0x7f17a00cb4e0 msgr2=0x7f17a00cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:55.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.150+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3453125994 shutdown_connections 2023-11-08T16:40:55.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.150+0000 7f17a7f4c640 1 -- 172.21.15.81:0/3453125994 wait complete. 2023-11-08T16:40:55.178 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T16:40:55.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.376+0000 7f2b60c87640 1 Processor -- start 2023-11-08T16:40:55.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.376+0000 7f2b60c87640 1 -- start start 2023-11-08T16:40:55.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.377+0000 7f2b60c87640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b5c15e170 0x7f2b5c15ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.377+0000 7f2b60c87640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2b5c15e550 0x7f2b5c0b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.377+0000 7f2b60c87640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b5c0b0e70 con 0x7f2b5c0b0a20 2023-11-08T16:40:55.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.377+0000 7f2b60c87640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b5c071460 con 0x7f2b5c15e550 2023-11-08T16:40:55.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.377+0000 7f2b60c87640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c0703c0 con 0x7f2b5c15e170 2023-11-08T16:40:55.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.378+0000 7f2b53fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2b5c15e550 0x7f2b5c0b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.378+0000 7f2b53fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2b5c15e550 0x7f2b5c0b04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53462/0 (socket says 172.21.15.81:53462) 2023-11-08T16:40:55.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.378+0000 7f2b53fff640 1 -- 172.21.15.81:0/2906327092 learned_addr learned my addr 172.21.15.81:0/2906327092 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:55.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.378+0000 7f2b5a59c640 1 --2- 172.21.15.81:0/2906327092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b5c15e170 0x7f2b5c15ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.378+0000 7f2b53fff640 1 -- 172.21.15.81:0/2906327092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b5c15e170 msgr2=0x7f2b5c15ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:55.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.378+0000 7f2b53fff640 1 --2- 172.21.15.81:0/2906327092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b5c15e170 0x7f2b5c15ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.378+0000 7f2b53fff640 1 -- 172.21.15.81:0/2906327092 >> v1:172.21.15.93:6789/0 conn(0x7f2b5c0b0a20 legacy=0x7f2b5c0ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:55.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.378+0000 7f2b53fff640 1 -- 172.21.15.81:0/2906327092 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b5c0706a0 con 0x7f2b5c15e550 2023-11-08T16:40:55.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.379+0000 7f2b53fff640 1 --2- 172.21.15.81:0/2906327092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b5c15e550 0x7f2b5c0b04e0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009870 tx=0x7f2b4402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcb718d37eca8ce server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:55.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.379+0000 7f2b59d9b640 1 -- 172.21.15.81:0/2906327092 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44031640 con 0x7f2b5c15e550 2023-11-08T16:40:55.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.379+0000 7f2b59d9b640 1 -- 172.21.15.81:0/2906327092 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b440317a0 con 0x7f2b5c15e550 2023-11-08T16:40:55.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.379+0000 7f2b59d9b640 1 -- 172.21.15.81:0/2906327092 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44031b30 con 0x7f2b5c15e550 2023-11-08T16:40:55.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.380+0000 7f2b60c87640 1 -- 172.21.15.81:0/2906327092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b5c15e550 msgr2=0x7f2b5c0b04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:55.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.380+0000 7f2b60c87640 1 --2- 172.21.15.81:0/2906327092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b5c15e550 0x7f2b5c0b04e0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009870 tx=0x7f2b4402b8f0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.380+0000 7f2b60c87640 1 -- 172.21.15.81:0/2906327092 shutdown_connections 2023-11-08T16:40:55.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.380+0000 7f2b60c87640 1 --2- 172.21.15.81:0/2906327092 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b5c15e550 0x7f2b5c0b04e0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.380+0000 7f2b60c87640 1 --2- 172.21.15.81:0/2906327092 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b5c15e170 0x7f2b5c15ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.380+0000 7f2b60c87640 1 -- 172.21.15.81:0/2906327092 >> 172.21.15.81:0/2906327092 conn(0x7f2b5c0cb4e0 msgr2=0x7f2b5c0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:55.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.380+0000 7f2b60c87640 1 -- 172.21.15.81:0/2906327092 shutdown_connections 2023-11-08T16:40:55.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.380+0000 7f2b60c87640 1 -- 172.21.15.81:0/2906327092 wait complete. 2023-11-08T16:40:55.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.382+0000 7f2b60c87640 1 Processor -- start 2023-11-08T16:40:55.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.382+0000 7f2b60c87640 1 -- start start 2023-11-08T16:40:55.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.383+0000 7f2b60c87640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b5c0b0a20 0x7f2b5c1e7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.383+0000 7f2b60c87640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b5c15e170 0x7f2b5c13e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.383+0000 7f2b60c87640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b5c15e550 0x7f2b5c0b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.383+0000 7f2b60c87640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c1e78d0 con 0x7f2b5c15e170 2023-11-08T16:40:55.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.383+0000 7f2b60c87640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c13f2e0 con 0x7f2b5c0b0a20 2023-11-08T16:40:55.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.383+0000 7f2b60c87640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5c1e4130 con 0x7f2b5c15e550 2023-11-08T16:40:55.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.383+0000 7f2b53fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b5c15e170 0x7f2b5c13e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.383+0000 7f2b53fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b5c15e170 0x7f2b5c13e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53470/0 (socket says 172.21.15.81:53470) 2023-11-08T16:40:55.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.383+0000 7f2b53fff640 1 -- 172.21.15.81:0/3672135623 learned_addr learned my addr 172.21.15.81:0/3672135623 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:55.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.384+0000 7f2b53fff640 1 -- 172.21.15.81:0/3672135623 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b5c15e550 msgr2=0x7f2b5c0b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:55.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.384+0000 7f2b53fff640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b5c15e550 0x7f2b5c0b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.384+0000 7f2b5a59c640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b5c0b0a20 0x7f2b5c1e7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.384+0000 7f2b53fff640 1 -- 172.21.15.81:0/3672135623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b5c0b0a20 msgr2=0x7f2b5c1e7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:55.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.384+0000 7f2b53fff640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b5c0b0a20 0x7f2b5c1e7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.384+0000 7f2b53fff640 1 -- 172.21.15.81:0/3672135623 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b440094d0 con 0x7f2b5c15e170 2023-11-08T16:40:55.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.384+0000 7f2b53fff640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b5c15e170 0x7f2b5c13e840 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009870 tx=0x7f2b440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:55.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.384+0000 7f2b52ffd640 1 -- 172.21.15.81:0/3672135623 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4403ac60 con 0x7f2b5c15e170 2023-11-08T16:40:55.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.384+0000 7f2b52ffd640 1 -- 172.21.15.81:0/3672135623 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4403adc0 con 0x7f2b5c15e170 2023-11-08T16:40:55.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.384+0000 7f2b52ffd640 1 -- 172.21.15.81:0/3672135623 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44044630 con 0x7f2b5c15e170 2023-11-08T16:40:55.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.384+0000 7f2b60c87640 1 -- 172.21.15.81:0/3672135623 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b5c1e4410 con 0x7f2b5c15e170 2023-11-08T16:40:55.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.385+0000 7f2b60c87640 1 -- 172.21.15.81:0/3672135623 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b5c13ed80 con 0x7f2b5c15e170 2023-11-08T16:40:55.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.386+0000 7f2b60c87640 1 -- 172.21.15.81:0/3672135623 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b5c1e4270 con 0x7f2b5c15e170 2023-11-08T16:40:55.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.387+0000 7f2b52ffd640 1 -- 172.21.15.81:0/3672135623 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b4402f050 con 0x7f2b5c15e170 2023-11-08T16:40:55.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.387+0000 7f2b52ffd640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b3c06ae70 0x7f2b3c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.388+0000 7f2b52ffd640 1 -- 172.21.15.81:0/3672135623 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f2b440ab1b0 con 0x7f2b5c15e170 2023-11-08T16:40:55.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.388+0000 7f2b5a59c640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b3c06ae70 0x7f2b3c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.388+0000 7f2b5a59c640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b3c06ae70 0x7f2b3c06d330 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c006fd0 tx=0x7f2b4c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:55.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.392+0000 7f2b52ffd640 1 -- 172.21.15.81:0/3672135623 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2b5c1e4270 con 0x7f2b5c15e170 2023-11-08T16:40:55.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.582+0000 7f2b60c87640 1 -- 172.21.15.81:0/3672135623 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2b5c0b0e70 con 0x7f2b5c15e170 2023-11-08T16:40:55.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.589+0000 7f2b52ffd640 1 -- 172.21.15.81:0/3672135623 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2b5c0b0e70 con 0x7f2b5c15e170 2023-11-08T16:40:55.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 -- 172.21.15.81:0/3672135623 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b3c06ae70 msgr2=0x7f2b3c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:55.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b3c06ae70 0x7f2b3c06d330 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c006fd0 tx=0x7f2b4c008040 comp rx=0 tx=0).stop 2023-11-08T16:40:55.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 -- 172.21.15.81:0/3672135623 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b5c15e170 msgr2=0x7f2b5c13e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:55.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b5c15e170 0x7f2b5c13e840 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f2b44009870 tx=0x7f2b440046c0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 -- 172.21.15.81:0/3672135623 shutdown_connections 2023-11-08T16:40:55.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b3c06ae70 0x7f2b3c06d330 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b5c15e550 0x7f2b5c0b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b5c15e170 0x7f2b5c13e840 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 --2- 172.21.15.81:0/3672135623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b5c0b0a20 0x7f2b5c1e7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 -- 172.21.15.81:0/3672135623 >> 172.21.15.81:0/3672135623 conn(0x7f2b5c0cb4e0 msgr2=0x7f2b5c0cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:55.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 -- 172.21.15.81:0/3672135623 shutdown_connections 2023-11-08T16:40:55.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.592+0000 7f2b60c87640 1 -- 172.21.15.81:0/3672135623 wait complete. 2023-11-08T16:40:55.612 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout' 2023-11-08T16:40:55.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.822+0000 7f66715ee640 1 Processor -- start 2023-11-08T16:40:55.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.822+0000 7f66715ee640 1 -- start start 2023-11-08T16:40:55.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.823+0000 7f66715ee640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f666c06bdf0 0x7f666c06c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.823+0000 7f66715ee640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c06b630 con 0x7f666c0ca480 2023-11-08T16:40:55.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.823+0000 7f66715ee640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f666c0ca910 con 0x7f666c0ade50 2023-11-08T16:40:55.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.823+0000 7f66715ee640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666c0b1860 con 0x7f666c06bdf0 2023-11-08T16:40:55.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.823+0000 7f666a7fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f666c0ca480 0x7f666c15ad90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49632/0 (socket says 172.21.15.81:49632) 2023-11-08T16:40:55.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.823+0000 7f666a7fc640 1 -- 172.21.15.81:0/789920521 learned_addr learned my addr 172.21.15.81:0/789920521 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:55.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808615350 0 0) 0x7f666c06b630 con 0x7f666c0ca480 2023-11-08T16:40:55.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6654003520 con 0x7f666c0ca480 2023-11-08T16:40:55.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f666b7fe640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f666c06bdf0 0x7f666c06c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1095853725 0 0) 0x7f6654003520 con 0x7f666c0ca480 2023-11-08T16:40:55.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f666c06bdf0 msgr2=0x7f666c06c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:55.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f6669ffb640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f666c06bdf0 0x7f666c06c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 >> v1:172.21.15.93:6789/0 conn(0x7f666c0ade50 legacy=0x7f666c15a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:55.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c0cabb0 con 0x7f666c0ca480 2023-11-08T16:40:55.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f66600019f0 con 0x7f666c0ca480 2023-11-08T16:40:55.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 >> v1:172.21.15.81:6789/0 conn(0x7f666c0ca480 legacy=0x7f666c15ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:55.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f6669ffb640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6654003a80 0x7f6654003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.824+0000 7f6669ffb640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f666c06bdf0 0x7f66540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f6669ffb640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6654004d70 0x7f66540051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665400bb90 con 0x7f6654004d70 2023-11-08T16:40:55.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665400bd00 con 0x7f6654003a80 2023-11-08T16:40:55.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665400bec0 con 0x7f666c06bdf0 2023-11-08T16:40:55.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f666b7fe640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6654004d70 0x7f66540051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f666affd640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6654003a80 0x7f6654003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f666b7fe640 1 -- 172.21.15.81:0/789920521 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f666c06bdf0 msgr2=0x7f66540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:55.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f666a7fc640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f666c06bdf0 0x7f66540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f666b7fe640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f666c06bdf0 0x7f66540047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f666b7fe640 1 -- 172.21.15.81:0/789920521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6654003a80 msgr2=0x7f6654003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:55.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f666b7fe640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6654003a80 0x7f6654003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.825+0000 7f666b7fe640 1 -- 172.21.15.81:0/789920521 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6654003520 con 0x7f6654004d70 2023-11-08T16:40:55.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.826+0000 7f666b7fe640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6654004d70 0x7f66540051f0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f666400ba00 tx=0x7f66540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f82c4a0469712f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:55.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.827+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66640335f0 con 0x7f6654004d70 2023-11-08T16:40:55.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.827+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6664033750 con 0x7f6654004d70 2023-11-08T16:40:55.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.827+0000 7f6669ffb640 1 -- 172.21.15.81:0/789920521 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6664033ae0 con 0x7f6654004d70 2023-11-08T16:40:55.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.827+0000 7f66715ee640 1 -- 172.21.15.81:0/789920521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6654004d70 msgr2=0x7f66540051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:55.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.827+0000 7f66715ee640 1 --2- 172.21.15.81:0/789920521 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6654004d70 0x7f66540051f0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f666400ba00 tx=0x7f66540057f0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.827+0000 7f666b7fe640 1 -- 172.21.15.81:0/789920521 reap_dead start 2023-11-08T16:40:55.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.827+0000 7f66715ee640 1 -- 172.21.15.81:0/789920521 shutdown_connections 2023-11-08T16:40:55.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.827+0000 7f66715ee640 1 -- 172.21.15.81:0/789920521 >> 172.21.15.81:0/789920521 conn(0x7f666c0cb490 msgr2=0x7f666c0cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:55.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.827+0000 7f66715ee640 1 -- 172.21.15.81:0/789920521 shutdown_connections 2023-11-08T16:40:55.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.828+0000 7f66715ee640 1 -- 172.21.15.81:0/789920521 wait complete. 2023-11-08T16:40:55.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.829+0000 7f66715ee640 1 Processor -- start 2023-11-08T16:40:55.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.829+0000 7f66715ee640 1 -- start start 2023-11-08T16:40:55.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.830+0000 7f66715ee640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6654004d70 0x7f666c15e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.830+0000 7f66715ee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f666c15cb20 0x7f666c0c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.830+0000 7f66715ee640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f666c1f5c60 0x7f666c1f6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.830+0000 7f66715ee640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c1f6650 con 0x7f666c15cb20 2023-11-08T16:40:55.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.830+0000 7f66715ee640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666c1f2590 con 0x7f666c1f5c60 2023-11-08T16:40:55.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.830+0000 7f66715ee640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666c15d020 con 0x7f6654004d70 2023-11-08T16:40:55.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.831+0000 7f666a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f666c15cb20 0x7f666c0c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.831+0000 7f666a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f666c15cb20 0x7f666c0c51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53484/0 (socket says 172.21.15.81:53484) 2023-11-08T16:40:55.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.831+0000 7f666a7fc640 1 -- 172.21.15.81:0/2100723512 learned_addr learned my addr 172.21.15.81:0/2100723512 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:55.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.831+0000 7f666b7fe640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f666c1f5c60 0x7f666c1f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.831+0000 7f666affd640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6654004d70 0x7f666c15e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.831+0000 7f666a7fc640 1 -- 172.21.15.81:0/2100723512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6654004d70 msgr2=0x7f666c15e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:55.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.831+0000 7f666a7fc640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6654004d70 0x7f666c15e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.831+0000 7f666a7fc640 1 -- 172.21.15.81:0/2100723512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f666c1f5c60 msgr2=0x7f666c1f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:55.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.831+0000 7f666a7fc640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f666c1f5c60 0x7f666c1f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:55.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.831+0000 7f666a7fc640 1 -- 172.21.15.81:0/2100723512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f666c15d300 con 0x7f666c15cb20 2023-11-08T16:40:55.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.831+0000 7f666a7fc640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f666c15cb20 0x7f666c0c51f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f6660002160 tx=0x7f666000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:55.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.832+0000 7f66537fe640 1 -- 172.21.15.81:0/2100723512 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660004d60 con 0x7f666c15cb20 2023-11-08T16:40:55.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.832+0000 7f66537fe640 1 -- 172.21.15.81:0/2100723512 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66600070d0 con 0x7f666c15cb20 2023-11-08T16:40:55.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.832+0000 7f66537fe640 1 -- 172.21.15.81:0/2100723512 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6660007480 con 0x7f666c15cb20 2023-11-08T16:40:55.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.832+0000 7f66715ee640 1 -- 172.21.15.81:0/2100723512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f666c15d5e0 con 0x7f666c15cb20 2023-11-08T16:40:55.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.832+0000 7f66715ee640 1 -- 172.21.15.81:0/2100723512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666c15dba0 con 0x7f666c15cb20 2023-11-08T16:40:55.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.833+0000 7f66715ee640 1 -- 172.21.15.81:0/2100723512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666c15ddb0 con 0x7f666c15cb20 2023-11-08T16:40:55.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.834+0000 7f66537fe640 1 -- 172.21.15.81:0/2100723512 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f66600075e0 con 0x7f666c15cb20 2023-11-08T16:40:55.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.835+0000 7f66537fe640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f665806aea0 0x7f665806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:55.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.835+0000 7f66537fe640 1 -- 172.21.15.81:0/2100723512 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f6660089720 con 0x7f666c15cb20 2023-11-08T16:40:55.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.836+0000 7f666affd640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f665806aea0 0x7f665806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:55.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.836+0000 7f666affd640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f665806aea0 0x7f665806d360 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f665c007fd0 tx=0x7f665c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:55.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:55.840+0000 7f66537fe640 1 -- 172.21.15.81:0/2100723512 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f66600549d0 con 0x7f666c15cb20 2023-11-08T16:40:56.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.104+0000 7f66715ee640 1 -- 172.21.15.81:0/2100723512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]} v 0) v1 -- 0x7f666c04ece0 con 0x7f666c15cb20 2023-11-08T16:40:56.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.149+0000 7f66537fe640 1 -- 172.21.15.81:0/2100723512 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_extra_data_pool_stashed_layout"]}]=0 v529) v1 ==== 155+0+0 (secure 0 0 0) 0x7f66600190e0 con 0x7f666c15cb20 2023-11-08T16:40:56.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 -- 172.21.15.81:0/2100723512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f665806aea0 msgr2=0x7f665806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f665806aea0 0x7f665806d360 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f665c007fd0 tx=0x7f665c009040 comp rx=0 tx=0).stop 2023-11-08T16:40:56.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 -- 172.21.15.81:0/2100723512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f666c15cb20 msgr2=0x7f666c0c51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f666c15cb20 0x7f666c0c51f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f6660002160 tx=0x7f666000b4a0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 -- 172.21.15.81:0/2100723512 shutdown_connections 2023-11-08T16:40:56.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f665806aea0 0x7f665806d360 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f666c1f5c60 0x7f666c1f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f666c15cb20 0x7f666c0c51f0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 --2- 172.21.15.81:0/2100723512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6654004d70 0x7f666c15e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 -- 172.21.15.81:0/2100723512 >> 172.21.15.81:0/2100723512 conn(0x7f666c0cb490 msgr2=0x7f666c0cd810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:56.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 -- 172.21.15.81:0/2100723512 shutdown_connections 2023-11-08T16:40:56.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.152+0000 7f66715ee640 1 -- 172.21.15.81:0/2100723512 wait complete. 2023-11-08T16:40:56.172 INFO:tasks.cephfs_test_runner:test_extra_data_pool_stashed_layout (tasks.cephfs.test_data_scan.TestDataScan) ... ok 2023-11-08T16:40:56.172 INFO:tasks.cephfs_test_runner:Starting test: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) That when injecting a dentry into a fragmented directory, we put it in the right fragment. 2023-11-08T16:40:56.172 INFO:tasks.cephfs_test_runner:test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-11-08T16:40:56.173 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:40:56.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.379+0000 7f25041c5640 1 Processor -- start 2023-11-08T16:40:56.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.379+0000 7f25041c5640 1 -- start start 2023-11-08T16:40:56.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.380+0000 7f25041c5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc0cf850 0x7f24fc0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.380+0000 7f25041c5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24fc15d680 con 0x7f24fc0b1da0 2023-11-08T16:40:56.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.380+0000 7f25041c5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24fc0cdca0 con 0x7f24fc0cea60 2023-11-08T16:40:56.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.380+0000 7f25041c5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc0cdf80 con 0x7f24fc0cf850 2023-11-08T16:40:56.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2501760640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f24fc0b1da0 0x7f24fc0cfc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49636/0 (socket says 172.21.15.81:49636) 2023-11-08T16:40:56.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2501760640 1 -- 172.21.15.81:0/3891611436 learned_addr learned my addr 172.21.15.81:0/3891611436 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:56.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2501f61640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc0cf850 0x7f24fc0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875469685 0 0) 0x7f24fc15d680 con 0x7f24fc0b1da0 2023-11-08T16:40:56.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24e4003520 con 0x7f24fc0b1da0 2023-11-08T16:40:56.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4159379101 0 0) 0x7f24e4003520 con 0x7f24fc0b1da0 2023-11-08T16:40:56.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc0cf850 msgr2=0x7f24fc0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:56.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2500f5f640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc0cf850 0x7f24fc0d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 >> v1:172.21.15.93:6789/0 conn(0x7f24fc0cea60 legacy=0x7f24fc0b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc0ce260 con 0x7f24fc0b1da0 2023-11-08T16:40:56.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f24f8001670 con 0x7f24fc0b1da0 2023-11-08T16:40:56.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.381+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 >> v1:172.21.15.81:6789/0 conn(0x7f24fc0b1da0 legacy=0x7f24fc0cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2500f5f640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24e4003a80 0x7f24e4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2500f5f640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc0cf850 0x7f24e40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2500f5f640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24e4004dd0 0x7f24e4005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc0cdf80 con 0x7f24e4004dd0 2023-11-08T16:40:56.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e400bbe0 con 0x7f24e4003a80 2023-11-08T16:40:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e400bda0 con 0x7f24fc0cf850 2023-11-08T16:40:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2502762640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24e4004dd0 0x7f24e4005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2501f61640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24e4003a80 0x7f24e4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2501760640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc0cf850 0x7f24e40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2502762640 1 -- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc0cf850 msgr2=0x7f24e40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:56.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2502762640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc0cf850 0x7f24e40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2502762640 1 -- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24e4003a80 msgr2=0x7f24e4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:56.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2502762640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24e4003a80 0x7f24e4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2502762640 1 -- 172.21.15.81:0/3891611436 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e4003520 con 0x7f24e4004dd0 2023-11-08T16:40:56.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.382+0000 7f2501f61640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24e4003a80 0x7f24e4003ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:40:56.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.384+0000 7f2502762640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24e4004dd0 0x7f24e4005250 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f24f4001e50 tx=0x7f24e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6c8f9a10e0b0a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:56.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.384+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4003b70 con 0x7f24e4004dd0 2023-11-08T16:40:56.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.384+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f4003cd0 con 0x7f24e4004dd0 2023-11-08T16:40:56.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.384+0000 7f2500f5f640 1 -- 172.21.15.81:0/3891611436 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4005310 con 0x7f24e4004dd0 2023-11-08T16:40:56.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.384+0000 7f25041c5640 1 -- 172.21.15.81:0/3891611436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24e4004dd0 msgr2=0x7f24e4005250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.384+0000 7f25041c5640 1 --2- 172.21.15.81:0/3891611436 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24e4004dd0 0x7f24e4005250 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f24f4001e50 tx=0x7f24e40057f0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.384+0000 7f2502762640 1 -- 172.21.15.81:0/3891611436 reap_dead start 2023-11-08T16:40:56.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.384+0000 7f25041c5640 1 -- 172.21.15.81:0/3891611436 shutdown_connections 2023-11-08T16:40:56.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.384+0000 7f25041c5640 1 -- 172.21.15.81:0/3891611436 >> 172.21.15.81:0/3891611436 conn(0x7f24fc0d2e40 msgr2=0x7f24fc0d3250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:56.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.385+0000 7f25041c5640 1 -- 172.21.15.81:0/3891611436 shutdown_connections 2023-11-08T16:40:56.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.385+0000 7f25041c5640 1 -- 172.21.15.81:0/3891611436 wait complete. 2023-11-08T16:40:56.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.387+0000 7f25041c5640 1 Processor -- start 2023-11-08T16:40:56.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.387+0000 7f25041c5640 1 -- start start 2023-11-08T16:40:56.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.388+0000 7f25041c5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24e4004dd0 0x7f24fc1f0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.388+0000 7f25041c5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc15c030 0x7f24fc1f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.388+0000 7f25041c5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc1f2e40 0x7f24fc0c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.388+0000 7f25041c5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc1f3de0 con 0x7f24fc15c030 2023-11-08T16:40:56.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.388+0000 7f25041c5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc1f7310 con 0x7f24e4004dd0 2023-11-08T16:40:56.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.388+0000 7f25041c5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc1f3820 con 0x7f24fc1f2e40 2023-11-08T16:40:56.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2501760640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc15c030 0x7f24fc1f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2501760640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc15c030 0x7f24fc1f6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53516/0 (socket says 172.21.15.81:53516) 2023-11-08T16:40:56.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2501760640 1 -- 172.21.15.81:0/4153737275 learned_addr learned my addr 172.21.15.81:0/4153737275 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:56.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2501f61640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24e4004dd0 0x7f24fc1f0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2502762640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc1f2e40 0x7f24fc0c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2501760640 1 -- 172.21.15.81:0/4153737275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc1f2e40 msgr2=0x7f24fc0c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2501760640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc1f2e40 0x7f24fc0c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2501760640 1 -- 172.21.15.81:0/4153737275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24e4004dd0 msgr2=0x7f24fc1f0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2501760640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24e4004dd0 0x7f24fc1f0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2501760640 1 -- 172.21.15.81:0/4153737275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc1f3b00 con 0x7f24fc15c030 2023-11-08T16:40:56.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2502762640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc1f2e40 0x7f24fc0c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:56.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.389+0000 7f2501760640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc15c030 0x7f24fc1f6dd0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f24f800e730 tx=0x7f24f800eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:56.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.390+0000 7f24f27fc640 1 -- 172.21.15.81:0/4153737275 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f800c6c0 con 0x7f24fc15c030 2023-11-08T16:40:56.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.390+0000 7f24f27fc640 1 -- 172.21.15.81:0/4153737275 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24f800c840 con 0x7f24fc15c030 2023-11-08T16:40:56.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.390+0000 7f24f27fc640 1 -- 172.21.15.81:0/4153737275 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f800cbf0 con 0x7f24fc15c030 2023-11-08T16:40:56.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.390+0000 7f25041c5640 1 -- 172.21.15.81:0/4153737275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24fc1f65d0 con 0x7f24fc15c030 2023-11-08T16:40:56.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.390+0000 7f25041c5640 1 -- 172.21.15.81:0/4153737275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24fc1f32f0 con 0x7f24fc15c030 2023-11-08T16:40:56.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.391+0000 7f25041c5640 1 -- 172.21.15.81:0/4153737275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24fc1f6430 con 0x7f24fc15c030 2023-11-08T16:40:56.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.392+0000 7f24f27fc640 1 -- 172.21.15.81:0/4153737275 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24f800cd50 con 0x7f24fc15c030 2023-11-08T16:40:56.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.393+0000 7f24f27fc640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24e806ae70 0x7f24e806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.393+0000 7f24f27fc640 1 -- 172.21.15.81:0/4153737275 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f24f8089460 con 0x7f24fc15c030 2023-11-08T16:40:56.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.393+0000 7f2501f61640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24e806ae70 0x7f24e806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.394+0000 7f2501f61640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24e806ae70 0x7f24e806d330 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f24ec006fd0 tx=0x7f24ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:56.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.397+0000 7f24f27fc640 1 -- 172.21.15.81:0/4153737275 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24fc1f6430 con 0x7f24fc15c030 2023-11-08T16:40:56.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.590+0000 7f25041c5640 1 -- 172.21.15.81:0/4153737275 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24fc1f6af0 con 0x7f24fc15c030 2023-11-08T16:40:56.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.592+0000 7f24f27fc640 1 -- 172.21.15.81:0/4153737275 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+13324 (secure 0 0 0) 0x7f24fc1f6af0 con 0x7f24fc15c030 2023-11-08T16:40:56.594 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:56.594 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":118,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:40:55.136616+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6836/3197735576":"2023-11-09T16:40:50.820832+0000","172.21.15.81:6836/3163533823":"2023-11-09T16:39:57.018336+0000","172.21.15.81:6836/3988749175":"2023-11-09T16:39:50.607108+0000","172.21.15.81:6837/3197735576":"2023-11-09T16:40:50.820832+0000","172.21.15.81:6837/3163533823":"2023-11-09T16:39:57.018336+0000","172.21.15.93:6835/691341796":"2023-11-09T16:39:48.576045+0000","172.21.15.81:6837/3988749175":"2023-11-09T16:39:50.607108+0000","172.21.15.93:6834/691341796":"2023-11-09T16:39:48.576045+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:40:56.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.594+0000 7f25041c5640 1 -- 172.21.15.81:0/4153737275 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24e806ae70 msgr2=0x7f24e806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.594+0000 7f25041c5640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24e806ae70 0x7f24e806d330 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f24ec006fd0 tx=0x7f24ec008040 comp rx=0 tx=0).stop 2023-11-08T16:40:56.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.595+0000 7f25041c5640 1 -- 172.21.15.81:0/4153737275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc15c030 msgr2=0x7f24fc1f6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.595+0000 7f25041c5640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc15c030 0x7f24fc1f6dd0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f24f800e730 tx=0x7f24f800eb30 comp rx=0 tx=0).stop 2023-11-08T16:40:56.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.595+0000 7f25041c5640 1 -- 172.21.15.81:0/4153737275 shutdown_connections 2023-11-08T16:40:56.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.595+0000 7f25041c5640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24e806ae70 0x7f24e806d330 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.595+0000 7f25041c5640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24fc1f2e40 0x7f24fc0c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.595+0000 7f25041c5640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24fc15c030 0x7f24fc1f6dd0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.595+0000 7f25041c5640 1 --2- 172.21.15.81:0/4153737275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24e4004dd0 0x7f24fc1f0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.595+0000 7f25041c5640 1 -- 172.21.15.81:0/4153737275 >> 172.21.15.81:0/4153737275 conn(0x7f24fc0d2e40 msgr2=0x7f24fc0d1420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:56.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.595+0000 7f25041c5640 1 -- 172.21.15.81:0/4153737275 shutdown_connections 2023-11-08T16:40:56.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.595+0000 7f25041c5640 1 -- 172.21.15.81:0/4153737275 wait complete. 2023-11-08T16:40:56.617 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:40:56.617 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:40:56.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.824+0000 7f3fef4e0640 1 Processor -- start 2023-11-08T16:40:56.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.824+0000 7f3fef4e0640 1 -- start start 2023-11-08T16:40:56.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.825+0000 7f3fef4e0640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f3fe80ce0d0 0x7f3fe80caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.825+0000 7f3fef4e0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3fe815bfc0 0x7f3fe815eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.825+0000 7f3fef4e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3fe815f0b0 0x7f3fe81516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.825+0000 7f3fef4e0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe815f4e0 con 0x7f3fe815bfc0 2023-11-08T16:40:56.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.825+0000 7f3fef4e0640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe806fa30 con 0x7f3fe80ce0d0 2023-11-08T16:40:56.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.825+0000 7f3fef4e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80ce4b0 con 0x7f3fe815f0b0 2023-11-08T16:40:56.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.826+0000 7f3feca7b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3fe815bfc0 0x7f3fe815eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.826+0000 7f3feca7b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f3fe815bfc0 0x7f3fe815eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53528/0 (socket says 172.21.15.81:53528) 2023-11-08T16:40:56.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.826+0000 7f3feca7b640 1 -- 172.21.15.81:0/831418953 learned_addr learned my addr 172.21.15.81:0/831418953 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:56.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.826+0000 7f3fed27c640 1 --2- 172.21.15.81:0/831418953 >> v2:172.21.15.93:3300/0 conn(0x7f3fe80ce0d0 0x7f3fe80caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.826+0000 7f3feda7d640 1 --2- 172.21.15.81:0/831418953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3fe815f0b0 0x7f3fe81516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.826+0000 7f3feca7b640 1 -- 172.21.15.81:0/831418953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3fe815f0b0 msgr2=0x7f3fe81516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:56.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.826+0000 7f3feca7b640 1 --2- 172.21.15.81:0/831418953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3fe815f0b0 0x7f3fe81516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.826+0000 7f3feca7b640 1 -- 172.21.15.81:0/831418953 >> v2:172.21.15.93:3300/0 conn(0x7f3fe80ce0d0 msgr2=0x7f3fe80caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:56.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.826+0000 7f3feca7b640 1 --2- 172.21.15.81:0/831418953 >> v2:172.21.15.93:3300/0 conn(0x7f3fe80ce0d0 0x7f3fe80caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.826+0000 7f3feca7b640 1 -- 172.21.15.81:0/831418953 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe806fca0 con 0x7f3fe815bfc0 2023-11-08T16:40:56.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.826+0000 7f3feda7d640 1 --2- 172.21.15.81:0/831418953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3fe815f0b0 0x7f3fe81516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:56.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.827+0000 7f3feca7b640 1 --2- 172.21.15.81:0/831418953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe815bfc0 0x7f3fe815eb70 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f3fe806f3a0 tx=0x7f3fd802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a5c0b1ab8644a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:56.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.828+0000 7f3fdffff640 1 -- 172.21.15.81:0/831418953 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8031870 con 0x7f3fe815bfc0 2023-11-08T16:40:56.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.828+0000 7f3fdffff640 1 -- 172.21.15.81:0/831418953 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd80319d0 con 0x7f3fe815bfc0 2023-11-08T16:40:56.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.828+0000 7f3fdffff640 1 -- 172.21.15.81:0/831418953 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8031d60 con 0x7f3fe815bfc0 2023-11-08T16:40:56.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.828+0000 7f3fef4e0640 1 -- 172.21.15.81:0/831418953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe815bfc0 msgr2=0x7f3fe815eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.828+0000 7f3fef4e0640 1 --2- 172.21.15.81:0/831418953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe815bfc0 0x7f3fe815eb70 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f3fe806f3a0 tx=0x7f3fd802b910 comp rx=0 tx=0).stop 2023-11-08T16:40:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.829+0000 7f3fef4e0640 1 -- 172.21.15.81:0/831418953 shutdown_connections 2023-11-08T16:40:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.829+0000 7f3fef4e0640 1 --2- 172.21.15.81:0/831418953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3fe815f0b0 0x7f3fe81516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.829+0000 7f3fef4e0640 1 --2- 172.21.15.81:0/831418953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe815bfc0 0x7f3fe815eb70 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.829+0000 7f3fef4e0640 1 --2- 172.21.15.81:0/831418953 >> v2:172.21.15.93:3300/0 conn(0x7f3fe80ce0d0 0x7f3fe80caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.829+0000 7f3fef4e0640 1 -- 172.21.15.81:0/831418953 >> 172.21.15.81:0/831418953 conn(0x7f3fe80cb580 msgr2=0x7f3fe80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.829+0000 7f3fef4e0640 1 -- 172.21.15.81:0/831418953 shutdown_connections 2023-11-08T16:40:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.829+0000 7f3fef4e0640 1 -- 172.21.15.81:0/831418953 wait complete. 2023-11-08T16:40:56.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.830+0000 7f3fef4e0640 1 Processor -- start 2023-11-08T16:40:56.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.831+0000 7f3fef4e0640 1 -- start start 2023-11-08T16:40:56.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3fef4e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3fe80ce0d0 0x7f3fe80b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3fef4e0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fe815bfc0 0x7f3fe8153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3fef4e0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe815f0b0 0x7f3fe81f2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3fef4e0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80c36b0 con 0x7f3fe815f0b0 2023-11-08T16:40:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3fef4e0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81f3110 con 0x7f3fe815bfc0 2023-11-08T16:40:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3fef4e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe81f33f0 con 0x7f3fe80ce0d0 2023-11-08T16:40:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3feda7d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe815f0b0 0x7f3fe81f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3feda7d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe815f0b0 0x7f3fe81f2850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53544/0 (socket says 172.21.15.81:53544) 2023-11-08T16:40:56.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3feda7d640 1 -- 172.21.15.81:0/605907532 learned_addr learned my addr 172.21.15.81:0/605907532 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:56.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3feca7b640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fe815bfc0 0x7f3fe8153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3feda7d640 1 -- 172.21.15.81:0/605907532 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3fe80ce0d0 msgr2=0x7f3fe80b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3feda7d640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3fe80ce0d0 0x7f3fe80b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3feda7d640 1 -- 172.21.15.81:0/605907532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fe815bfc0 msgr2=0x7f3fe8153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:56.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3feda7d640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fe815bfc0 0x7f3fe8153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:56.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3feda7d640 1 -- 172.21.15.81:0/605907532 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd80094d0 con 0x7f3fe815f0b0 2023-11-08T16:40:56.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.832+0000 7f3feca7b640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fe815bfc0 0x7f3fe8153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:56.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.833+0000 7f3feda7d640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe815f0b0 0x7f3fe81f2850 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000cef0 tx=0x7f3fe000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:56.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.833+0000 7f3fddffb640 1 -- 172.21.15.81:0/605907532 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000d840 con 0x7f3fe815f0b0 2023-11-08T16:40:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.833+0000 7f3fddffb640 1 -- 172.21.15.81:0/605907532 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe000d9c0 con 0x7f3fe815f0b0 2023-11-08T16:40:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.833+0000 7f3fddffb640 1 -- 172.21.15.81:0/605907532 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe000dd70 con 0x7f3fe815f0b0 2023-11-08T16:40:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.833+0000 7f3fef4e0640 1 -- 172.21.15.81:0/605907532 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe81f36d0 con 0x7f3fe815f0b0 2023-11-08T16:40:56.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.833+0000 7f3fef4e0640 1 -- 172.21.15.81:0/605907532 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe81538b0 con 0x7f3fe815f0b0 2023-11-08T16:40:56.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.834+0000 7f3fef4e0640 1 -- 172.21.15.81:0/605907532 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe81f3860 con 0x7f3fe815f0b0 2023-11-08T16:40:56.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.835+0000 7f3fddffb640 1 -- 172.21.15.81:0/605907532 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3fe001e3b0 con 0x7f3fe815f0b0 2023-11-08T16:40:56.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.836+0000 7f3fddffb640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3fcc06ae70 0x7f3fcc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:56.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.836+0000 7f3fddffb640 1 -- 172.21.15.81:0/605907532 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f3fe0089f00 con 0x7f3fe815f0b0 2023-11-08T16:40:56.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.836+0000 7f3fed27c640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3fcc06ae70 0x7f3fcc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:56.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.837+0000 7f3fed27c640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3fcc06ae70 0x7f3fcc06d330 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80c9a40 tx=0x7f3fe4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:56.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:56.840+0000 7f3fddffb640 1 -- 172.21.15.81:0/605907532 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f3fe0055260 con 0x7f3fe815f0b0 2023-11-08T16:40:57.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.031+0000 7f3fef4e0640 1 -- 172.21.15.81:0/605907532 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fe804ece0 con 0x7f3fe815f0b0 2023-11-08T16:40:57.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.033+0000 7f3fddffb640 1 -- 172.21.15.81:0/605907532 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fe001a0a0 con 0x7f3fe815f0b0 2023-11-08T16:40:57.034 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:40:57.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 -- 172.21.15.81:0/605907532 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3fcc06ae70 msgr2=0x7f3fcc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:57.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3fcc06ae70 0x7f3fcc06d330 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3fe80c9a40 tx=0x7f3fe4008040 comp rx=0 tx=0).stop 2023-11-08T16:40:57.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 -- 172.21.15.81:0/605907532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe815f0b0 msgr2=0x7f3fe81f2850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:57.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe815f0b0 0x7f3fe81f2850 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f3fe000cef0 tx=0x7f3fe000a710 comp rx=0 tx=0).stop 2023-11-08T16:40:57.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 -- 172.21.15.81:0/605907532 shutdown_connections 2023-11-08T16:40:57.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f3fcc06ae70 0x7f3fcc06d330 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:57.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f3fe815f0b0 0x7f3fe81f2850 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:57.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3fe815bfc0 0x7f3fe8153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:57.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 --2- 172.21.15.81:0/605907532 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f3fe80ce0d0 0x7f3fe80b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:57.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 -- 172.21.15.81:0/605907532 >> 172.21.15.81:0/605907532 conn(0x7f3fe80cb580 msgr2=0x7f3fe815b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:57.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 -- 172.21.15.81:0/605907532 shutdown_connections 2023-11-08T16:40:57.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.037+0000 7f3fef4e0640 1 -- 172.21.15.81:0/605907532 wait complete. 2023-11-08T16:40:57.058 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection' 2023-11-08T16:40:57.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.257+0000 7f9c6359e640 1 Processor -- start 2023-11-08T16:40:57.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.257+0000 7f9c6359e640 1 -- start start 2023-11-08T16:40:57.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.259+0000 7f9c6359e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c640cf170 0x7f9c6415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:57.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.259+0000 7f9c6359e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c64070470 con 0x7f9c640ced10 2023-11-08T16:40:57.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.259+0000 7f9c6359e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6406d580 con 0x7f9c6415aac0 2023-11-08T16:40:57.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.259+0000 7f9c6359e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6406f650 con 0x7f9c640cf170 2023-11-08T16:40:57.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.259+0000 7f9c5bfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f9c640ced10 0x7f9c6415aea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49644/0 (socket says 172.21.15.81:49644) 2023-11-08T16:40:57.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.259+0000 7f9c5bfff640 1 -- 172.21.15.81:0/1539829489 learned_addr learned my addr 172.21.15.81:0/1539829489 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:57.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c62d9d640 1 --2- 172.21.15.81:0/1539829489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c640cf170 0x7f9c6415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:57.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927551876 0 0) 0x7f9c64070470 con 0x7f9c640ced10 2023-11-08T16:40:57.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c50003520 con 0x7f9c640ced10 2023-11-08T16:40:57.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1730430878 0 0) 0x7f9c50003520 con 0x7f9c640ced10 2023-11-08T16:40:57.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c640cf170 msgr2=0x7f9c6415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:57.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 --2- 172.21.15.81:0/1539829489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c640cf170 0x7f9c6415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:57.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 >> v1:172.21.15.93:6789/0 conn(0x7f9c6415aac0 legacy=0x7f9c640711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:57.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6406ca00 con 0x7f9c640ced10 2023-11-08T16:40:57.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f9c4c001670 con 0x7f9c640ced10 2023-11-08T16:40:57.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 >> v1:172.21.15.81:6789/0 conn(0x7f9c640ced10 legacy=0x7f9c6415aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:57.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 --2- 172.21.15.81:0/1539829489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c50003a80 0x7f9c50003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:57.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 --2- 172.21.15.81:0/1539829489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c500047c0 0x7f9c50004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:57.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 --2- 172.21.15.81:0/1539829489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c640cf170 0x7f9c50005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:57.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6406f650 con 0x7f9c500047c0 2023-11-08T16:40:57.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5000bb40 con 0x7f9c50003a80 2023-11-08T16:40:57.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5000bd00 con 0x7f9c640cf170 2023-11-08T16:40:57.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.260+0000 7f9c5bfff640 1 --2- 172.21.15.81:0/1539829489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c500047c0 0x7f9c50004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:57.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.261+0000 7f9c6259c640 1 --2- 172.21.15.81:0/1539829489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c50003a80 0x7f9c50003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:57.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.261+0000 7f9c5bfff640 1 -- 172.21.15.81:0/1539829489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c640cf170 msgr2=0x7f9c50005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:57.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.261+0000 7f9c5bfff640 1 --2- 172.21.15.81:0/1539829489 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c640cf170 0x7f9c50005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:57.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.261+0000 7f9c5bfff640 1 -- 172.21.15.81:0/1539829489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c50003a80 msgr2=0x7f9c50003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:57.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.261+0000 7f9c5bfff640 1 --2- 172.21.15.81:0/1539829489 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c50003a80 0x7f9c50003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:57.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.261+0000 7f9c5bfff640 1 -- 172.21.15.81:0/1539829489 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c50003520 con 0x7f9c500047c0 2023-11-08T16:40:57.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.262+0000 7f9c5bfff640 1 --2- 172.21.15.81:0/1539829489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c500047c0 0x7f9c50004c40 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00c0e0 tx=0x7f9c500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=730dda3e58b0cfd4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:57.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.262+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c005ad0 con 0x7f9c500047c0 2023-11-08T16:40:57.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.262+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c005c50 con 0x7f9c500047c0 2023-11-08T16:40:57.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.262+0000 7f9c61d9b640 1 -- 172.21.15.81:0/1539829489 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c02c7d0 con 0x7f9c500047c0 2023-11-08T16:40:57.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.263+0000 7f9c6359e640 1 -- 172.21.15.81:0/1539829489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c500047c0 msgr2=0x7f9c50004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:57.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.263+0000 7f9c6359e640 1 --2- 172.21.15.81:0/1539829489 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c500047c0 0x7f9c50004c40 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c00c0e0 tx=0x7f9c500057f0 comp rx=0 tx=0).stop 2023-11-08T16:40:57.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.263+0000 7f9c62d9d640 1 -- 172.21.15.81:0/1539829489 reap_dead start 2023-11-08T16:40:57.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.263+0000 7f9c6359e640 1 -- 172.21.15.81:0/1539829489 shutdown_connections 2023-11-08T16:40:57.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.263+0000 7f9c6359e640 1 -- 172.21.15.81:0/1539829489 >> 172.21.15.81:0/1539829489 conn(0x7f9c6406a4a0 msgr2=0x7f9c6406a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:57.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.263+0000 7f9c6359e640 1 -- 172.21.15.81:0/1539829489 shutdown_connections 2023-11-08T16:40:57.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.263+0000 7f9c6359e640 1 -- 172.21.15.81:0/1539829489 wait complete. 2023-11-08T16:40:57.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.265+0000 7f9c6359e640 1 Processor -- start 2023-11-08T16:40:57.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.265+0000 7f9c6359e640 1 -- start start 2023-11-08T16:40:57.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.266+0000 7f9c6359e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c500047c0 0x7f9c641503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:57.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.266+0000 7f9c6359e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c6415aea0 0x7f9c640b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:57.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.266+0000 7f9c6359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640acb10 0x7f9c64147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:57.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.266+0000 7f9c6359e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6415b410 con 0x7f9c640acb10 2023-11-08T16:40:57.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.266+0000 7f9c6359e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6415b6f0 con 0x7f9c500047c0 2023-11-08T16:40:57.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.266+0000 7f9c6359e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6415b9d0 con 0x7f9c6415aea0 2023-11-08T16:40:57.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.267+0000 7f9c62d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640acb10 0x7f9c64147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:57.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.267+0000 7f9c62d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640acb10 0x7f9c64147fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53566/0 (socket says 172.21.15.81:53566) 2023-11-08T16:40:57.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.267+0000 7f9c62d9d640 1 -- 172.21.15.81:0/3543450528 learned_addr learned my addr 172.21.15.81:0/3543450528 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:57.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.267+0000 7f9c6259c640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c500047c0 0x7f9c641503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:57.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.267+0000 7f9c62d9d640 1 -- 172.21.15.81:0/3543450528 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c6415aea0 msgr2=0x7f9c640b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:57.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.267+0000 7f9c62d9d640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c6415aea0 0x7f9c640b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:57.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.267+0000 7f9c62d9d640 1 -- 172.21.15.81:0/3543450528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c500047c0 msgr2=0x7f9c641503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:57.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.267+0000 7f9c62d9d640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c500047c0 0x7f9c641503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:57.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.267+0000 7f9c62d9d640 1 -- 172.21.15.81:0/3543450528 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6415bcb0 con 0x7f9c640acb10 2023-11-08T16:40:57.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.267+0000 7f9c62d9d640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640acb10 0x7f9c64147fd0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f9c640cf9f0 tx=0x7f9c5c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:57.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.268+0000 7f9c5affd640 1 -- 172.21.15.81:0/3543450528 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c010040 con 0x7f9c640acb10 2023-11-08T16:40:57.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.268+0000 7f9c5affd640 1 -- 172.21.15.81:0/3543450528 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5c00bce0 con 0x7f9c640acb10 2023-11-08T16:40:57.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.268+0000 7f9c5affd640 1 -- 172.21.15.81:0/3543450528 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5c014640 con 0x7f9c640acb10 2023-11-08T16:40:57.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.268+0000 7f9c6359e640 1 -- 172.21.15.81:0/3543450528 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6415bf90 con 0x7f9c640acb10 2023-11-08T16:40:57.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.268+0000 7f9c6359e640 1 -- 172.21.15.81:0/3543450528 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c640b2e30 con 0x7f9c640acb10 2023-11-08T16:40:57.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.269+0000 7f9c6359e640 1 -- 172.21.15.81:0/3543450528 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6415c180 con 0x7f9c640acb10 2023-11-08T16:40:57.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.270+0000 7f9c5affd640 1 -- 172.21.15.81:0/3543450528 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9c5c0147a0 con 0x7f9c640acb10 2023-11-08T16:40:57.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.271+0000 7f9c5affd640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c4406ae70 0x7f9c4406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:57.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.271+0000 7f9c5affd640 1 -- 172.21.15.81:0/3543450528 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f9c5c089b90 con 0x7f9c640acb10 2023-11-08T16:40:57.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.271+0000 7f9c6259c640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c4406ae70 0x7f9c4406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:57.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.272+0000 7f9c6259c640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c4406ae70 0x7f9c4406d330 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005490 tx=0x7f9c54009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:57.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.275+0000 7f9c5affd640 1 -- 172.21.15.81:0/3543450528 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9c5c054ec0 con 0x7f9c640acb10 2023-11-08T16:40:57.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:57.533+0000 7f9c6359e640 1 -- 172.21.15.81:0/3543450528 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]} v 0) v1 -- 0x7f9c64067800 con 0x7f9c640acb10 2023-11-08T16:40:58.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.162+0000 7f9c5affd640 1 -- 172.21.15.81:0/3543450528 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]}]=0 v531) v1 ==== 148+0+0 (secure 0 0 0) 0x7f9c5c0180d0 con 0x7f9c640acb10 2023-11-08T16:40:58.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 -- 172.21.15.81:0/3543450528 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c4406ae70 msgr2=0x7f9c4406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:58.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c4406ae70 0x7f9c4406d330 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f9c54005490 tx=0x7f9c54009040 comp rx=0 tx=0).stop 2023-11-08T16:40:58.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 -- 172.21.15.81:0/3543450528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640acb10 msgr2=0x7f9c64147fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:58.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640acb10 0x7f9c64147fd0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f9c640cf9f0 tx=0x7f9c5c00b480 comp rx=0 tx=0).stop 2023-11-08T16:40:58.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 -- 172.21.15.81:0/3543450528 shutdown_connections 2023-11-08T16:40:58.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9c4406ae70 0x7f9c4406d330 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9c640acb10 0x7f9c64147fd0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9c6415aea0 0x7f9c640b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 --2- 172.21.15.81:0/3543450528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c500047c0 0x7f9c641503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 -- 172.21.15.81:0/3543450528 >> 172.21.15.81:0/3543450528 conn(0x7f9c6406a4a0 msgr2=0x7f9c64157830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:58.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 -- 172.21.15.81:0/3543450528 shutdown_connections 2023-11-08T16:40:58.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.166+0000 7f9c6359e640 1 -- 172.21.15.81:0/3543450528 wait complete. 2023-11-08T16:40:58.189 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-08T16:40:58.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.409+0000 7fb473fff640 1 Processor -- start 2023-11-08T16:40:58.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.409+0000 7fb473fff640 1 -- start start 2023-11-08T16:40:58.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.411+0000 7fb473fff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb4741576a0 0x7fb47415a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:58.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.411+0000 7fb473fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb4740b11f0 0x7fb474157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:58.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.411+0000 7fb473fff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb474157a80 con 0x7fb47415b250 2023-11-08T16:40:58.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.411+0000 7fb473fff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb47415c360 con 0x7fb4740b11f0 2023-11-08T16:40:58.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.411+0000 7fb473fff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb47415b6e0 con 0x7fb4741576a0 2023-11-08T16:40:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.411+0000 7fb4727fc640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fb47415b250 0x7fb474070c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49656/0 (socket says 172.21.15.81:49656) 2023-11-08T16:40:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.411+0000 7fb4737fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb4740b11f0 0x7fb474157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:58.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.411+0000 7fb4727fc640 1 -- 172.21.15.81:0/313538174 learned_addr learned my addr 172.21.15.81:0/313538174 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.411+0000 7fb472ffd640 1 --2- 172.21.15.81:0/313538174 >> v2:172.21.15.93:3300/0 conn(0x7fb4741576a0 0x7fb47415a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.412+0000 7fb471ffb640 1 -- 172.21.15.81:0/313538174 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2513675663 0 0) 0x7fb474157a80 con 0x7fb47415b250 2023-11-08T16:40:58.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.412+0000 7fb471ffb640 1 -- 172.21.15.81:0/313538174 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb45c003520 con 0x7fb47415b250 2023-11-08T16:40:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.412+0000 7fb4737fe640 1 -- 172.21.15.81:0/313538174 >> v2:172.21.15.93:3300/0 conn(0x7fb4741576a0 msgr2=0x7fb47415a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.412+0000 7fb4737fe640 1 --2- 172.21.15.81:0/313538174 >> v2:172.21.15.93:3300/0 conn(0x7fb4741576a0 0x7fb47415a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.412+0000 7fb4737fe640 1 -- 172.21.15.81:0/313538174 >> v1:172.21.15.81:6789/0 conn(0x7fb47415b250 legacy=0x7fb474070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.412+0000 7fb4737fe640 1 -- 172.21.15.81:0/313538174 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47415b9c0 con 0x7fb4740b11f0 2023-11-08T16:40:58.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.413+0000 7fb4737fe640 1 --2- 172.21.15.81:0/313538174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4740b11f0 0x7fb474157cd0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fb47406d810 tx=0x7fb46c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c11bfdb925cef0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.413+0000 7fb471ffb640 1 -- 172.21.15.81:0/313538174 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c02cbf0 con 0x7fb4740b11f0 2023-11-08T16:40:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.413+0000 7fb471ffb640 1 -- 172.21.15.81:0/313538174 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46c02cd50 con 0x7fb4740b11f0 2023-11-08T16:40:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.413+0000 7fb471ffb640 1 -- 172.21.15.81:0/313538174 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46c034660 con 0x7fb4740b11f0 2023-11-08T16:40:58.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.414+0000 7fb473fff640 1 -- 172.21.15.81:0/313538174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4740b11f0 msgr2=0x7fb474157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.414+0000 7fb473fff640 1 --2- 172.21.15.81:0/313538174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4740b11f0 0x7fb474157cd0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fb47406d810 tx=0x7fb46c02be40 comp rx=0 tx=0).stop 2023-11-08T16:40:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.414+0000 7fb473fff640 1 -- 172.21.15.81:0/313538174 shutdown_connections 2023-11-08T16:40:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.414+0000 7fb473fff640 1 --2- 172.21.15.81:0/313538174 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4740b11f0 0x7fb474157cd0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.414+0000 7fb473fff640 1 --2- 172.21.15.81:0/313538174 >> v2:172.21.15.93:3300/0 conn(0x7fb4741576a0 0x7fb47415a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.414+0000 7fb473fff640 1 -- 172.21.15.81:0/313538174 >> 172.21.15.81:0/313538174 conn(0x7fb4740ce8d0 msgr2=0x7fb4740cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:58.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.414+0000 7fb473fff640 1 -- 172.21.15.81:0/313538174 shutdown_connections 2023-11-08T16:40:58.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.414+0000 7fb473fff640 1 -- 172.21.15.81:0/313538174 wait complete. 2023-11-08T16:40:58.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.416+0000 7fb473fff640 1 Processor -- start 2023-11-08T16:40:58.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.416+0000 7fb473fff640 1 -- start start 2023-11-08T16:40:58.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.417+0000 7fb473fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4740b11f0 0x7fb4741f08f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:58.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.417+0000 7fb473fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4741576a0 0x7fb4741534f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:58.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.417+0000 7fb473fff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4740c5910 0x7fb4741edc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.417+0000 7fb473fff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4741f0f00 con 0x7fb4740b11f0 2023-11-08T16:40:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.417+0000 7fb473fff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4741f11e0 con 0x7fb4740c5910 2023-11-08T16:40:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.417+0000 7fb473fff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4741f14c0 con 0x7fb4741576a0 2023-11-08T16:40:58.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.417+0000 7fb472ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4740b11f0 0x7fb4741f08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.417+0000 7fb472ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4740b11f0 0x7fb4741f08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53580/0 (socket says 172.21.15.81:53580) 2023-11-08T16:40:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.417+0000 7fb472ffd640 1 -- 172.21.15.81:0/446001823 learned_addr learned my addr 172.21.15.81:0/446001823 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.417+0000 7fb4737fe640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4740c5910 0x7fb4741edc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.417+0000 7fb472ffd640 1 -- 172.21.15.81:0/446001823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4741576a0 msgr2=0x7fb4741534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:58.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.418+0000 7fb4727fc640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4741576a0 0x7fb4741534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.418+0000 7fb472ffd640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4741576a0 0x7fb4741534f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.418+0000 7fb472ffd640 1 -- 172.21.15.81:0/446001823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4740c5910 msgr2=0x7fb4741edc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.418+0000 7fb472ffd640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4740c5910 0x7fb4741edc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.418+0000 7fb472ffd640 1 -- 172.21.15.81:0/446001823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46c009a70 con 0x7fb4740b11f0 2023-11-08T16:40:58.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.418+0000 7fb472ffd640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4740b11f0 0x7fb4741f08f0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb46400d800 tx=0x7fb46400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:58.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.418+0000 7fb45b7fe640 1 -- 172.21.15.81:0/446001823 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46400b6c0 con 0x7fb4740b11f0 2023-11-08T16:40:58.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.418+0000 7fb45b7fe640 1 -- 172.21.15.81:0/446001823 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb46400b820 con 0x7fb4740b11f0 2023-11-08T16:40:58.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.418+0000 7fb45b7fe640 1 -- 172.21.15.81:0/446001823 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb46400bbb0 con 0x7fb4740b11f0 2023-11-08T16:40:58.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.418+0000 7fb473fff640 1 -- 172.21.15.81:0/446001823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4741f17a0 con 0x7fb4740b11f0 2023-11-08T16:40:58.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.418+0000 7fb473fff640 1 -- 172.21.15.81:0/446001823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb474153a30 con 0x7fb4740b11f0 2023-11-08T16:40:58.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.419+0000 7fb473fff640 1 -- 172.21.15.81:0/446001823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47404ece0 con 0x7fb4740b11f0 2023-11-08T16:40:58.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.420+0000 7fb45b7fe640 1 -- 172.21.15.81:0/446001823 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb46400bd10 con 0x7fb4740b11f0 2023-11-08T16:40:58.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.421+0000 7fb45b7fe640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46006ae70 0x7fb46006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:58.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.421+0000 7fb45b7fe640 1 -- 172.21.15.81:0/446001823 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5201+0+0 (secure 0 0 0) 0x7fb464089a60 con 0x7fb4740b11f0 2023-11-08T16:40:58.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.422+0000 7fb4727fc640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46006ae70 0x7fb46006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:58.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.422+0000 7fb4727fc640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46006ae70 0x7fb46006d330 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb4741f0e30 tx=0x7fb468009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:58.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.426+0000 7fb45b7fe640 1 -- 172.21.15.81:0/446001823 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb464054c60 con 0x7fb4740b11f0 2023-11-08T16:40:58.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.615+0000 7fb473fff640 1 -- 172.21.15.81:0/446001823 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb4741f1040 con 0x7fb4740b11f0 2023-11-08T16:40:58.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.620+0000 7fb45b7fe640 1 -- 172.21.15.81:0/446001823 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb4741f1040 con 0x7fb4740b11f0 2023-11-08T16:40:58.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.623+0000 7fb473fff640 1 -- 172.21.15.81:0/446001823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46006ae70 msgr2=0x7fb46006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:58.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.623+0000 7fb473fff640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46006ae70 0x7fb46006d330 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fb4741f0e30 tx=0x7fb468009040 comp rx=0 tx=0).stop 2023-11-08T16:40:58.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.623+0000 7fb473fff640 1 -- 172.21.15.81:0/446001823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4740b11f0 msgr2=0x7fb4741f08f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:58.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.623+0000 7fb473fff640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4740b11f0 0x7fb4741f08f0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fb46400d800 tx=0x7fb46400dcc0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.623+0000 7fb473fff640 1 -- 172.21.15.81:0/446001823 shutdown_connections 2023-11-08T16:40:58.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.623+0000 7fb473fff640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb46006ae70 0x7fb46006d330 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.624+0000 7fb473fff640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4740c5910 0x7fb4741edc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.624+0000 7fb473fff640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb4741576a0 0x7fb4741534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.624+0000 7fb473fff640 1 --2- 172.21.15.81:0/446001823 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb4740b11f0 0x7fb4741f08f0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:58.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.624+0000 7fb473fff640 1 -- 172.21.15.81:0/446001823 >> 172.21.15.81:0/446001823 conn(0x7fb4740ce8d0 msgr2=0x7fb474157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:58.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.624+0000 7fb473fff640 1 -- 172.21.15.81:0/446001823 shutdown_connections 2023-11-08T16:40:58.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:58.624+0000 7fb473fff640 1 -- 172.21.15.81:0/446001823 wait complete. 2023-11-08T16:40:58.645 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T16:40:58.645 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:40:58.761 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:40:58.886 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T16:40:58.887 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:40:58.887 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:40:58.946 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:40:58.947 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:40:58.947 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:40:59.008 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:40:59.009 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:40:59.009 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:40:59.068 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:40:59.069 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:40:59.070 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:40:59.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:40:59.130 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-08T16:40:59.130 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:40:59.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.351+0000 7f16058d1640 1 Processor -- start 2023-11-08T16:40:59.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.351+0000 7f16058d1640 1 -- start start 2023-11-08T16:40:59.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.353+0000 7f16058d1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f160006d5d0 0x7f1600165540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.353+0000 7f16058d1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1600165a80 0x7f16001629e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.353+0000 7f16058d1640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1600162050 con 0x7f160006b850 2023-11-08T16:40:59.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.353+0000 7f16058d1640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1600164e10 con 0x7f160006d5d0 2023-11-08T16:40:59.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.353+0000 7f16058d1640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1600162330 con 0x7f1600165a80 2023-11-08T16:40:59.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.353+0000 7f15fffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f160006d5d0 0x7f1600165540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.353+0000 7f16048cf640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f160006b850 0x7f1600161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:49668/0 (socket says 172.21.15.81:49668) 2023-11-08T16:40:59.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.353+0000 7f16048cf640 1 -- 172.21.15.81:0/1698384890 learned_addr learned my addr 172.21.15.81:0/1698384890 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:40:59.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.354+0000 7f16050d0640 1 --2- 172.21.15.81:0/1698384890 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1600165a80 0x7f16001629e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.354+0000 7f15fffff640 1 -- 172.21.15.81:0/1698384890 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1600165a80 msgr2=0x7f16001629e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:59.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.354+0000 7f15fffff640 1 --2- 172.21.15.81:0/1698384890 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1600165a80 0x7f16001629e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.354+0000 7f15fffff640 1 -- 172.21.15.81:0/1698384890 >> v1:172.21.15.81:6789/0 conn(0x7f160006b850 legacy=0x7f1600161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:59.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.354+0000 7f15fffff640 1 -- 172.21.15.81:0/1698384890 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1600162610 con 0x7f160006d5d0 2023-11-08T16:40:59.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.354+0000 7f16050d0640 1 --2- 172.21.15.81:0/1698384890 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1600165a80 0x7f16001629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:40:59.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.355+0000 7f15fffff640 1 --2- 172.21.15.81:0/1698384890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160006d5d0 0x7f1600165540 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0097a0 tx=0x7f15ec02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248725473724c44a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:59.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.355+0000 7f15ff7fe640 1 -- 172.21.15.81:0/1698384890 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec02d040 con 0x7f160006d5d0 2023-11-08T16:40:59.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.355+0000 7f15ff7fe640 1 -- 172.21.15.81:0/1698384890 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec031810 con 0x7f160006d5d0 2023-11-08T16:40:59.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.355+0000 7f15ff7fe640 1 -- 172.21.15.81:0/1698384890 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec031ba0 con 0x7f160006d5d0 2023-11-08T16:40:59.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.356+0000 7f16058d1640 1 -- 172.21.15.81:0/1698384890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160006d5d0 msgr2=0x7f1600165540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:59.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.356+0000 7f16058d1640 1 --2- 172.21.15.81:0/1698384890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160006d5d0 0x7f1600165540 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0097a0 tx=0x7f15ec02b8b0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.356+0000 7f16058d1640 1 -- 172.21.15.81:0/1698384890 shutdown_connections 2023-11-08T16:40:59.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.356+0000 7f16058d1640 1 --2- 172.21.15.81:0/1698384890 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1600165a80 0x7f16001629e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.356+0000 7f16058d1640 1 --2- 172.21.15.81:0/1698384890 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f160006d5d0 0x7f1600165540 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.356+0000 7f16058d1640 1 -- 172.21.15.81:0/1698384890 >> 172.21.15.81:0/1698384890 conn(0x7f160006a4a0 msgr2=0x7f160006a8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:59.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.356+0000 7f16058d1640 1 -- 172.21.15.81:0/1698384890 shutdown_connections 2023-11-08T16:40:59.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.356+0000 7f16058d1640 1 -- 172.21.15.81:0/1698384890 wait complete. 2023-11-08T16:40:59.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.358+0000 7f16058d1640 1 Processor -- start 2023-11-08T16:40:59.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.358+0000 7f16058d1640 1 -- start start 2023-11-08T16:40:59.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.359+0000 7f16058d1640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f160006b850 0x7f1600153ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.359+0000 7f16058d1640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f160006d5d0 0x7f160014f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.359+0000 7f16058d1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1600165a80 0x7f16001509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.359+0000 7f16058d1640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1600155810 con 0x7f1600165a80 2023-11-08T16:40:59.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.359+0000 7f16058d1640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1600153d90 con 0x7f160006d5d0 2023-11-08T16:40:59.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.359+0000 7f16058d1640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160014fcd0 con 0x7f160006b850 2023-11-08T16:40:59.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.359+0000 7f16050d0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1600165a80 0x7f16001509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.359+0000 7f16050d0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1600165a80 0x7f16001509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53608/0 (socket says 172.21.15.81:53608) 2023-11-08T16:40:59.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.359+0000 7f16050d0640 1 -- 172.21.15.81:0/1808903723 learned_addr learned my addr 172.21.15.81:0/1808903723 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:59.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f16048cf640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f160006b850 0x7f1600153ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f15fffff640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f160006d5d0 0x7f160014f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f16050d0640 1 -- 172.21.15.81:0/1808903723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f160006b850 msgr2=0x7f1600153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:59.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f16050d0640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f160006b850 0x7f1600153ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f16050d0640 1 -- 172.21.15.81:0/1808903723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f160006d5d0 msgr2=0x7f160014f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:59.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f16050d0640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f160006d5d0 0x7f160014f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f16050d0640 1 -- 172.21.15.81:0/1808903723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec009400 con 0x7f1600165a80 2023-11-08T16:40:59.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f16050d0640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1600165a80 0x7f16001509e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f15f800cd50 tx=0x7f15f800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:59.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f15fd7fa640 1 -- 172.21.15.81:0/1808903723 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f800f680 con 0x7f1600165a80 2023-11-08T16:40:59.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f15fd7fa640 1 -- 172.21.15.81:0/1808903723 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f800f800 con 0x7f1600165a80 2023-11-08T16:40:59.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f15fd7fa640 1 -- 172.21.15.81:0/1808903723 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f800fbb0 con 0x7f1600165a80 2023-11-08T16:40:59.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.360+0000 7f16058d1640 1 -- 172.21.15.81:0/1808903723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1600153ab0 con 0x7f1600165a80 2023-11-08T16:40:59.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.361+0000 7f16058d1640 1 -- 172.21.15.81:0/1808903723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1600155d20 con 0x7f1600165a80 2023-11-08T16:40:59.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.362+0000 7f16058d1640 1 -- 172.21.15.81:0/1808903723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1600067800 con 0x7f1600165a80 2023-11-08T16:40:59.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.363+0000 7f15fd7fa640 1 -- 172.21.15.81:0/1808903723 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15f800b040 con 0x7f1600165a80 2023-11-08T16:40:59.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.363+0000 7f15fd7fa640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15e406ae70 0x7f15e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.364+0000 7f15fd7fa640 1 -- 172.21.15.81:0/1808903723 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5201+0+0 (secure 0 0 0) 0x7f15f8089ed0 con 0x7f1600165a80 2023-11-08T16:40:59.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.364+0000 7f16048cf640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15e406ae70 0x7f15e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.364+0000 7f16048cf640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15e406ae70 0x7f15e406d330 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f15f4005880 tx=0x7f15f4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:59.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.368+0000 7f15fd7fa640 1 -- 172.21.15.81:0/1808903723 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f15f80550d0 con 0x7f1600165a80 2023-11-08T16:40:59.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.598+0000 7f16058d1640 1 -- 172.21.15.81:0/1808903723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f16000717a0 con 0x7f1600165a80 2023-11-08T16:40:59.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.599+0000 7f15fd7fa640 1 -- 172.21.15.81:0/1808903723 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+3695 (secure 0 0 0) 0x7f15f801d090 con 0x7f1600165a80 2023-11-08T16:40:59.601 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:40:59.601 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":102,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6196,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/728323060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":728323060},{"type":"v1","addr":"172.21.15.93:6834","nonce":728323060}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6199,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/3481704447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":3481704447},{"type":"v1","addr":"172.21.15.93:6835","nonce":3481704447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":98},{"gid":8187,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/699802320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":699802320},{"type":"v1","addr":"172.21.15.81:6835","nonce":699802320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96},{"gid":8460,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/1883285447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1883285447},{"type":"v1","addr":"172.21.15.81:6837","nonce":1883285447}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":102}],"filesystems":[]} 2023-11-08T16:40:59.602 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 102 2023-11-08T16:40:59.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.601+0000 7f16058d1640 1 -- 172.21.15.81:0/1808903723 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15e406ae70 msgr2=0x7f15e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:59.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.601+0000 7f16058d1640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15e406ae70 0x7f15e406d330 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f15f4005880 tx=0x7f15f4009040 comp rx=0 tx=0).stop 2023-11-08T16:40:59.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.601+0000 7f16058d1640 1 -- 172.21.15.81:0/1808903723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1600165a80 msgr2=0x7f16001509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:59.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.601+0000 7f16058d1640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1600165a80 0x7f16001509e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f15f800cd50 tx=0x7f15f800a650 comp rx=0 tx=0).stop 2023-11-08T16:40:59.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.602+0000 7f16058d1640 1 -- 172.21.15.81:0/1808903723 shutdown_connections 2023-11-08T16:40:59.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.602+0000 7f16058d1640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f15e406ae70 0x7f15e406d330 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.602+0000 7f16058d1640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1600165a80 0x7f16001509e0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.602+0000 7f16058d1640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f160006d5d0 0x7f160014f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.602+0000 7f16058d1640 1 --2- 172.21.15.81:0/1808903723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f160006b850 0x7f1600153ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.602+0000 7f16058d1640 1 -- 172.21.15.81:0/1808903723 >> 172.21.15.81:0/1808903723 conn(0x7f160006a4a0 msgr2=0x7f1600159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:59.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.602+0000 7f16058d1640 1 -- 172.21.15.81:0/1808903723 shutdown_connections 2023-11-08T16:40:59.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.602+0000 7f16058d1640 1 -- 172.21.15.81:0/1808903723 wait complete. 2023-11-08T16:40:59.623 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-08T16:40:59.623 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-08T16:40:59.623 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T16:40:59.624 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:40:59.624 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-08T16:40:59.624 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-08T16:40:59.625 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T16:40:59.625 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:40:59.625 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-08T16:40:59.625 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-08T16:40:59.626 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T16:40:59.626 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:40:59.626 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-08T16:40:59.627 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-08T16:40:59.627 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T16:40:59.627 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:40:59.641 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T16:40:59.642 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-08T16:40:59.645 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T16:40:59.645 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-08T16:40:59.647 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T16:40:59.647 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-08T16:40:59.650 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T16:40:59.651 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-08T16:40:59.692 INFO:tasks.ceph.mds.a:Started 2023-11-08T16:40:59.693 DEBUG:teuthology.parallel:result is None 2023-11-08T16:40:59.699 INFO:tasks.ceph.mds.b:Started 2023-11-08T16:40:59.699 DEBUG:teuthology.parallel:result is None 2023-11-08T16:40:59.700 INFO:tasks.ceph.mds.c:Started 2023-11-08T16:40:59.700 DEBUG:teuthology.parallel:result is None 2023-11-08T16:40:59.706 INFO:tasks.ceph.mds.d:Started 2023-11-08T16:40:59.706 DEBUG:teuthology.parallel:result is None 2023-11-08T16:40:59.707 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-08T16:40:59.851 INFO:tasks.ceph.mds.a.smithi081.stdout:starting mds.a at 2023-11-08T16:40:59.852 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:40:59.870 INFO:tasks.ceph.mds.b.smithi093.stdout:starting mds.b at 2023-11-08T16:40:59.871 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:40:59.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.894+0000 7fba6b475640 1 Processor -- start 2023-11-08T16:40:59.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.894+0000 7fba6b475640 1 -- start start 2023-11-08T16:40:59.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.895+0000 7fba6b475640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fba6406cf10 0x7fba64161630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.895+0000 7fba6b475640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fba640ca1a0 0x7fba640caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.895+0000 7fba6b475640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba640cc670 0x7fba640cca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.895+0000 7fba6b475640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba640cd580 con 0x7fba640ca1a0 2023-11-08T16:40:59.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.895+0000 7fba6b475640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba640ccfc0 con 0x7fba6406cf10 2023-11-08T16:40:59.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.895+0000 7fba6b475640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640c93e0 con 0x7fba640cc670 2023-11-08T16:40:59.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.896+0000 7fba68a10640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fba640ca1a0 0x7fba640caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.896+0000 7fba68a10640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fba640ca1a0 0x7fba640caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53658/0 (socket says 172.21.15.81:53658) 2023-11-08T16:40:59.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.896+0000 7fba69a12640 1 --2- 172.21.15.81:0/1598483015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba640cc670 0x7fba640cca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.896+0000 7fba68a10640 1 -- 172.21.15.81:0/1598483015 learned_addr learned my addr 172.21.15.81:0/1598483015 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:59.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.896+0000 7fba69211640 1 --2- 172.21.15.81:0/1598483015 >> v2:172.21.15.93:3300/0 conn(0x7fba6406cf10 0x7fba64161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.896+0000 7fba68a10640 1 -- 172.21.15.81:0/1598483015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba640cc670 msgr2=0x7fba640cca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:59.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.896+0000 7fba68a10640 1 --2- 172.21.15.81:0/1598483015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba640cc670 0x7fba640cca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.896+0000 7fba68a10640 1 -- 172.21.15.81:0/1598483015 >> v2:172.21.15.93:3300/0 conn(0x7fba6406cf10 msgr2=0x7fba64161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:40:59.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.896+0000 7fba68a10640 1 --2- 172.21.15.81:0/1598483015 >> v2:172.21.15.93:3300/0 conn(0x7fba6406cf10 0x7fba64161630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.896+0000 7fba68a10640 1 -- 172.21.15.81:0/1598483015 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba640c96c0 con 0x7fba640ca1a0 2023-11-08T16:40:59.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.897+0000 7fba68a10640 1 --2- 172.21.15.81:0/1598483015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba640ca1a0 0x7fba640caeb0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fba640726d0 tx=0x7fba5402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31a432ed5fb8f8e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:59.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.897+0000 7fba5bfff640 1 -- 172.21.15.81:0/1598483015 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54031860 con 0x7fba640ca1a0 2023-11-08T16:40:59.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.898+0000 7fba5bfff640 1 -- 172.21.15.81:0/1598483015 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba540319c0 con 0x7fba640ca1a0 2023-11-08T16:40:59.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.898+0000 7fba5bfff640 1 -- 172.21.15.81:0/1598483015 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba54031d50 con 0x7fba640ca1a0 2023-11-08T16:40:59.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.898+0000 7fba6b475640 1 -- 172.21.15.81:0/1598483015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba640ca1a0 msgr2=0x7fba640caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:59.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.898+0000 7fba6b475640 1 --2- 172.21.15.81:0/1598483015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba640ca1a0 0x7fba640caeb0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fba640726d0 tx=0x7fba5402b900 comp rx=0 tx=0).stop 2023-11-08T16:40:59.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.898+0000 7fba6b475640 1 -- 172.21.15.81:0/1598483015 shutdown_connections 2023-11-08T16:40:59.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.898+0000 7fba6b475640 1 --2- 172.21.15.81:0/1598483015 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba640cc670 0x7fba640cca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.898+0000 7fba6b475640 1 --2- 172.21.15.81:0/1598483015 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba640ca1a0 0x7fba640caeb0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.898+0000 7fba6b475640 1 --2- 172.21.15.81:0/1598483015 >> v2:172.21.15.93:3300/0 conn(0x7fba6406cf10 0x7fba64161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.898+0000 7fba6b475640 1 -- 172.21.15.81:0/1598483015 >> 172.21.15.81:0/1598483015 conn(0x7fba640ceaa0 msgr2=0x7fba640cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:40:59.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.898+0000 7fba6b475640 1 -- 172.21.15.81:0/1598483015 shutdown_connections 2023-11-08T16:40:59.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.898+0000 7fba6b475640 1 -- 172.21.15.81:0/1598483015 wait complete. 2023-11-08T16:40:59.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.900+0000 7fba6b475640 1 Processor -- start 2023-11-08T16:40:59.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.900+0000 7fba6b475640 1 -- start start 2023-11-08T16:40:59.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.901+0000 7fba6b475640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba6406cf10 0x7fba640c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.901+0000 7fba6b475640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba640ca1a0 0x7fba640b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.901+0000 7fba6b475640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba640cc670 0x7fba640c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.901+0000 7fba6b475640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640c3a50 con 0x7fba640cc670 2023-11-08T16:40:59.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.901+0000 7fba6b475640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba640c3d30 con 0x7fba640ca1a0 2023-11-08T16:40:59.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.901+0000 7fba6b475640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba640c4010 con 0x7fba6406cf10 2023-11-08T16:40:59.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.901+0000 7fba69a12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba640cc670 0x7fba640c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.901+0000 7fba69a12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba640cc670 0x7fba640c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:53660/0 (socket says 172.21.15.81:53660) 2023-11-08T16:40:59.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.901+0000 7fba69a12640 1 -- 172.21.15.81:0/3487841326 learned_addr learned my addr 172.21.15.81:0/3487841326 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:40:59.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba69211640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba6406cf10 0x7fba640c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba68a10640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba640ca1a0 0x7fba640b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba69a12640 1 -- 172.21.15.81:0/3487841326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba6406cf10 msgr2=0x7fba640c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:59.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba69a12640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba6406cf10 0x7fba640c33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba69a12640 1 -- 172.21.15.81:0/3487841326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba640ca1a0 msgr2=0x7fba640b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:40:59.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba69a12640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba640ca1a0 0x7fba640b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:40:59.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba69a12640 1 -- 172.21.15.81:0/3487841326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba540094d0 con 0x7fba640cc670 2023-11-08T16:40:59.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba69211640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba6406cf10 0x7fba640c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:40:59.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba68a10640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba640ca1a0 0x7fba640b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:40:59.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba69a12640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba640cc670 0x7fba640c4af0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00efc0 tx=0x7fba5c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:59.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba59ffb640 1 -- 172.21.15.81:0/3487841326 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c009ba0 con 0x7fba640cc670 2023-11-08T16:40:59.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba59ffb640 1 -- 172.21.15.81:0/3487841326 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba5c00f040 con 0x7fba640cc670 2023-11-08T16:40:59.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba59ffb640 1 -- 172.21.15.81:0/3487841326 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba5c0146c0 con 0x7fba640cc670 2023-11-08T16:40:59.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.902+0000 7fba6b475640 1 -- 172.21.15.81:0/3487841326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba640c42f0 con 0x7fba640cc670 2023-11-08T16:40:59.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.903+0000 7fba6b475640 1 -- 172.21.15.81:0/3487841326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba640b1e30 con 0x7fba640cc670 2023-11-08T16:40:59.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.904+0000 7fba6b475640 1 -- 172.21.15.81:0/3487841326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba640c9c60 con 0x7fba640cc670 2023-11-08T16:40:59.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.905+0000 7fba59ffb640 1 -- 172.21.15.81:0/3487841326 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fba5c014820 con 0x7fba640cc670 2023-11-08T16:40:59.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.906+0000 7fba59ffb640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fba4806aea0 0x7fba4806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:40:59.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.906+0000 7fba59ffb640 1 -- 172.21.15.81:0/3487841326 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5201+0+0 (secure 0 0 0) 0x7fba5c089ef0 con 0x7fba640cc670 2023-11-08T16:40:59.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.906+0000 7fba69211640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fba4806aea0 0x7fba4806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:40:59.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.907+0000 7fba69211640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fba4806aea0 0x7fba4806d360 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fba60009a60 tx=0x7fba60008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:40:59.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:40:59.910+0000 7fba59ffb640 1 -- 172.21.15.81:0/3487841326 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fba640c9c60 con 0x7fba640cc670 2023-11-08T16:41:00.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:00.114+0000 7fba6b475640 1 -- 172.21.15.81:0/3487841326 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fba640c4810 con 0x7fba640cc670 2023-11-08T16:41:01.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.204+0000 7fba59ffb640 1 -- 172.21.15.81:0/3487841326 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v123) v1 ==== 97+0+0 (secure 0 0 0) 0x7fba640c4810 con 0x7fba640cc670 2023-11-08T16:41:01.206 INFO:teuthology.orchestra.run.smithi081.stderr: removed all blocklist entries 2023-11-08T16:41:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 -- 172.21.15.81:0/3487841326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fba4806aea0 msgr2=0x7fba4806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fba4806aea0 0x7fba4806d360 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fba60009a60 tx=0x7fba60008040 comp rx=0 tx=0).stop 2023-11-08T16:41:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 -- 172.21.15.81:0/3487841326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba640cc670 msgr2=0x7fba640c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba640cc670 0x7fba640c4af0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fba5c00efc0 tx=0x7fba5c00c490 comp rx=0 tx=0).stop 2023-11-08T16:41:01.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 -- 172.21.15.81:0/3487841326 shutdown_connections 2023-11-08T16:41:01.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fba4806aea0 0x7fba4806d360 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fba640cc670 0x7fba640c4af0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba640ca1a0 0x7fba640b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 --2- 172.21.15.81:0/3487841326 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fba6406cf10 0x7fba640c33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 -- 172.21.15.81:0/3487841326 >> 172.21.15.81:0/3487841326 conn(0x7fba640ceaa0 msgr2=0x7fba64070f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:01.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 -- 172.21.15.81:0/3487841326 shutdown_connections 2023-11-08T16:41:01.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.208+0000 7fba6b475640 1 -- 172.21.15.81:0/3487841326 wait complete. 2023-11-08T16:41:01.230 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-08T16:41:01.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.441+0000 7f65b29ce640 1 Processor -- start 2023-11-08T16:41:01.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.441+0000 7f65b29ce640 1 -- start start 2023-11-08T16:41:01.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.443+0000 7f65b29ce640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f65ac15aeb0 0x7f65ac0ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.443+0000 7f65b29ce640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ac06cc60 con 0x7f65ac0c9580 2023-11-08T16:41:01.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.443+0000 7f65b29ce640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ac0cae20 con 0x7f65ac0c90f0 2023-11-08T16:41:01.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.443+0000 7f65b29ce640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ac0cb3e0 con 0x7f65ac15aeb0 2023-11-08T16:41:01.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.443+0000 7f65b0f6b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f65ac0c9580 0x7f65ac06b9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32992/0 (socket says 172.21.15.81:32992) 2023-11-08T16:41:01.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.443+0000 7f65b0f6b640 1 -- 172.21.15.81:0/2152919314 learned_addr learned my addr 172.21.15.81:0/2152919314 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:01.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.443+0000 7f65abfff640 1 --2- 172.21.15.81:0/2152919314 >> v2:172.21.15.93:3300/0 conn(0x7f65ac15aeb0 0x7f65ac0ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.443+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963242586 0 0) 0x7f65ac06cc60 con 0x7f65ac0c9580 2023-11-08T16:41:01.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6594003520 con 0x7f65ac0c9580 2023-11-08T16:41:01.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 12158874 0 0) 0x7f6594003520 con 0x7f65ac0c9580 2023-11-08T16:41:01.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 >> v2:172.21.15.93:3300/0 conn(0x7f65ac15aeb0 msgr2=0x7f65ac0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:01.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 --2- 172.21.15.81:0/2152919314 >> v2:172.21.15.93:3300/0 conn(0x7f65ac15aeb0 0x7f65ac0ca4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 >> v1:172.21.15.93:6789/0 conn(0x7f65ac0c90f0 legacy=0x7f65ac06b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac0cb100 con 0x7f65ac0c9580 2023-11-08T16:41:01.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f65a40019f0 con 0x7f65ac0c9580 2023-11-08T16:41:01.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 >> v1:172.21.15.81:6789/0 conn(0x7f65ac0c9580 legacy=0x7f65ac06b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 --2- 172.21.15.81:0/2152919314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65940039c0 0x7f6594003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 --2- 172.21.15.81:0/2152919314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65940047c0 0x7f6594004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 --2- 172.21.15.81:0/2152919314 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6594005180 0x7f659400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0cb3e0 con 0x7f65940039c0 2023-11-08T16:41:01.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659400c080 con 0x7f65940047c0 2023-11-08T16:41:01.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659400c240 con 0x7f6594005180 2023-11-08T16:41:01.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.444+0000 7f65abfff640 1 --2- 172.21.15.81:0/2152919314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65940039c0 0x7f6594003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.445+0000 7f65abfff640 1 -- 172.21.15.81:0/2152919314 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6594005180 msgr2=0x7f659400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:01.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.445+0000 7f65abfff640 1 --2- 172.21.15.81:0/2152919314 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6594005180 0x7f659400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.445+0000 7f65abfff640 1 -- 172.21.15.81:0/2152919314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65940047c0 msgr2=0x7f6594004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:01.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.445+0000 7f65ab7fe640 1 --2- 172.21.15.81:0/2152919314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65940047c0 0x7f6594004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.445+0000 7f65abfff640 1 --2- 172.21.15.81:0/2152919314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65940047c0 0x7f6594004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.445+0000 7f65abfff640 1 -- 172.21.15.81:0/2152919314 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6594003520 con 0x7f65940039c0 2023-11-08T16:41:01.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.445+0000 7f65b0f6b640 1 -- 172.21.15.81:0/2152919314 reap_dead start 2023-11-08T16:41:01.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.445+0000 7f65ab7fe640 1 --2- 172.21.15.81:0/2152919314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65940047c0 0x7f6594004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:01.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.446+0000 7f65abfff640 1 --2- 172.21.15.81:0/2152919314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65940039c0 0x7f6594003de0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f659c009ab0 tx=0x7f65940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d40060e17bd12f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:01.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.446+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c005990 con 0x7f65940039c0 2023-11-08T16:41:01.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.446+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c005af0 con 0x7f65940039c0 2023-11-08T16:41:01.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.446+0000 7f65aaffd640 1 -- 172.21.15.81:0/2152919314 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c005ea0 con 0x7f65940039c0 2023-11-08T16:41:01.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.447+0000 7f65b29ce640 1 -- 172.21.15.81:0/2152919314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65940039c0 msgr2=0x7f6594003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.447+0000 7f65b29ce640 1 --2- 172.21.15.81:0/2152919314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65940039c0 0x7f6594003de0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f659c009ab0 tx=0x7f65940057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.447+0000 7f65b29ce640 1 -- 172.21.15.81:0/2152919314 shutdown_connections 2023-11-08T16:41:01.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.447+0000 7f65b29ce640 1 --2- 172.21.15.81:0/2152919314 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65940039c0 0x7f6594003de0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.447+0000 7f65b29ce640 1 -- 172.21.15.81:0/2152919314 >> 172.21.15.81:0/2152919314 conn(0x7f65ac0ceaa0 msgr2=0x7f65ac0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:01.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.447+0000 7f65b29ce640 1 -- 172.21.15.81:0/2152919314 shutdown_connections 2023-11-08T16:41:01.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.447+0000 7f65b29ce640 1 -- 172.21.15.81:0/2152919314 wait complete. 2023-11-08T16:41:01.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.449+0000 7f65b29ce640 1 Processor -- start 2023-11-08T16:41:01.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.449+0000 7f65b29ce640 1 -- start start 2023-11-08T16:41:01.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65b29ce640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65940039c0 0x7f65ac0c3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65b29ce640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ac0b3ec0 0x7f65ac0ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65b29ce640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f65ac151e40 0x7f65ac0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65b29ce640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0c3590 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65b29ce640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0adad0 con 0x7f65940039c0 2023-11-08T16:41:01.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65b29ce640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0b16b0 con 0x7f65ac151e40 2023-11-08T16:41:01.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ac0b3ec0 0x7f65ac0ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ac0b3ec0 0x7f65ac0ad2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58672/0 (socket says 172.21.15.81:58672) 2023-11-08T16:41:01.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65ab7fe640 1 -- 172.21.15.81:0/3353750628 learned_addr learned my addr 172.21.15.81:0/3353750628 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:01.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65abfff640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65940039c0 0x7f65ac0c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65ab7fe640 1 -- 172.21.15.81:0/3353750628 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f65ac151e40 msgr2=0x7f65ac0ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65ab7fe640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f65ac151e40 0x7f65ac0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65ab7fe640 1 -- 172.21.15.81:0/3353750628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65940039c0 msgr2=0x7f65ac0c3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65ab7fe640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65940039c0 0x7f65ac0c3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65ab7fe640 1 -- 172.21.15.81:0/3353750628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac0ad800 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.450+0000 7f65abfff640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65940039c0 0x7f65ac0c3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:01.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.451+0000 7f65ab7fe640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ac0b3ec0 0x7f65ac0ad2c0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f65a000d400 tx=0x7f65a000d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:01.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.451+0000 7f65a8ff9640 1 -- 172.21.15.81:0/3353750628 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a000b070 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.451+0000 7f65a8ff9640 1 -- 172.21.15.81:0/3353750628 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a0003610 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.451+0000 7f65b29ce640 1 -- 172.21.15.81:0/3353750628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ac0b1950 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.451+0000 7f65a8ff9640 1 -- 172.21.15.81:0/3353750628 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a0003a00 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.451+0000 7f65b29ce640 1 -- 172.21.15.81:0/3353750628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ac0b1f10 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.452+0000 7f65b29ce640 1 -- 172.21.15.81:0/3353750628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ac0b20d0 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.453+0000 7f65a8ff9640 1 -- 172.21.15.81:0/3353750628 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f65a0003b60 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.454+0000 7f65a8ff9640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f659806aea0 0x7f659806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.455+0000 7f65a8ff9640 1 -- 172.21.15.81:0/3353750628 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f65a0088aa0 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.455+0000 7f65abfff640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f659806aea0 0x7f659806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.455+0000 7f65abfff640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f659806aea0 0x7f659806d360 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f659c035000 tx=0x7f659c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:01.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.459+0000 7f65a8ff9640 1 -- 172.21.15.81:0/3353750628 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f65a0053f40 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.719+0000 7f65b29ce640 1 -- 172.21.15.81:0/3353750628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f65ac0743f0 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.720+0000 7f65a8ff9640 1 -- 172.21.15.81:0/3353750628 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f65a0018020 con 0x7f65ac0b3ec0 2023-11-08T16:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:41:01.722 INFO:teuthology.orchestra.run.smithi081.stdout: "auth_dump": [ 2023-11-08T16:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.a", 2023-11-08T16:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlL0bmNxAAlrzRL4depZdd+lBB5HN09g==", 2023-11-08T16:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.723 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.724 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.b", 2023-11-08T16:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tl2cFWHhAAvwSSL9e5996l9MBwNQzs6A==", 2023-11-08T16:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.725 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.726 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.c", 2023-11-08T16:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tlwo23DBAAl1sBjOJMIQRb1fH3o9XoiQ==", 2023-11-08T16:41:01.727 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.728 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mds.d", 2023-11-08T16:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAGt0tl5DdQLxAA1aU7e9epGtKnZ6hkHZTnsQ==", 2023-11-08T16:41:01.729 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.730 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.731 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.0", 2023-11-08T16:41:01.732 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQALt0tl3kkwKRAAgDwD1eQYk7qjvV+/gyol/w==", 2023-11-08T16:41:01.732 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.732 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:41:01.732 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:41:01.732 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.733 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.733 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.733 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.733 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.1", 2023-11-08T16:41:01.733 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAPt0tlpIh+BRAAUukXDBbd9Vev1TzIrv9sRg==", 2023-11-08T16:41:01.734 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.734 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:41:01.734 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:41:01.734 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.734 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.735 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.735 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.735 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.2", 2023-11-08T16:41:01.735 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQASt0tlmzJfDhAASO7zeg/9ddmLsoOTe+MIug==", 2023-11-08T16:41:01.735 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.736 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:41:01.736 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:41:01.736 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.736 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.736 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.736 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.737 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.3", 2023-11-08T16:41:01.737 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAVt0tlGoQSGRAAQsrZhrx4kpYUm3KEb6ZUGQ==", 2023-11-08T16:41:01.737 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.738 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:41:01.738 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:41:01.738 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.738 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.738 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.739 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.739 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.4", 2023-11-08T16:41:01.739 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAct0tlOS/kIxAAxC2I47B2WeGTFZxvdnmkMg==", 2023-11-08T16:41:01.739 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.739 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:41:01.739 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:41:01.740 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.740 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.740 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.740 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.740 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.5", 2023-11-08T16:41:01.741 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAgt0tlnAgEBRAADIRSyu/bxiykCc2uiIt0kA==", 2023-11-08T16:41:01.741 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.742 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:41:01.742 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:41:01.743 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.743 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.743 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.743 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.743 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.6", 2023-11-08T16:41:01.744 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAjt0tlrdfZHBAAHGdvBBzVwXMmy8GmsZLNbg==", 2023-11-08T16:41:01.744 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.744 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:41:01.744 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:41:01.744 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.745 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.745 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.745 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.745 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "osd.7", 2023-11-08T16:41:01.745 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAmt0tl4nN9NRAAzV9abuBfhybYCsSFnqHEYA==", 2023-11-08T16:41:01.746 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.746 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow profile osd", 2023-11-08T16:41:01.746 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile osd", 2023-11-08T16:41:01.746 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.747 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.747 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.747 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.747 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.0", 2023-11-08T16:41:01.747 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ==", 2023-11-08T16:41:01.748 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.748 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:41:01.748 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow r", 2023-11-08T16:41:01.748 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-08T16:41:01.748 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.748 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.749 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.749 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.1", 2023-11-08T16:41:01.749 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlgn/qGBAA+WHDN1CJ1v62EbX1GEFBWA==", 2023-11-08T16:41:01.749 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.750 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:41:01.750 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:41:01.750 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:41:01.750 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:41:01.750 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.750 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.751 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.751 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.2", 2023-11-08T16:41:01.751 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlbQmgDRAAL/b9NwCBVT+uGxuv3VAWRw==", 2023-11-08T16:41:01.751 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.752 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:41:01.752 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:41:01.752 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:41:01.752 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:41:01.752 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.752 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.753 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.753 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.3", 2023-11-08T16:41:01.753 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAHt0tlah+ZIxAAg53qCmUjPtG+l++2Byn1NA==", 2023-11-08T16:41:01.753 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.753 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow", 2023-11-08T16:41:01.754 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow r", 2023-11-08T16:41:01.754 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow rw", 2023-11-08T16:41:01.754 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow rwx" 2023-11-08T16:41:01.754 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.754 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.755 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.755 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "client.admin", 2023-11-08T16:41:01.755 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAEt0tlNjJ+KRAATvWBm8Guj09sQQB2U1Afzw==", 2023-11-08T16:41:01.755 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.756 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:41:01.756 INFO:teuthology.orchestra.run.smithi081.stdout: "mgr": "allow *", 2023-11-08T16:41:01.756 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow *", 2023-11-08T16:41:01.756 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.756 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.757 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.757 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.757 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.x", 2023-11-08T16:41:01.757 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlhZFjLRAAANBB8roHZH4z4pE4C2Aq/Q==", 2023-11-08T16:41:01.757 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.758 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:41:01.758 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2023-11-08T16:41:01.758 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.758 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.758 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:01.759 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:01.759 INFO:teuthology.orchestra.run.smithi081.stdout: "entity": "mgr.y", 2023-11-08T16:41:01.759 INFO:teuthology.orchestra.run.smithi081.stdout: "key": "AQAFt0tlNpDaIBAAtgLFNQitjQnLny6SRY7hHw==", 2023-11-08T16:41:01.759 INFO:teuthology.orchestra.run.smithi081.stdout: "caps": { 2023-11-08T16:41:01.759 INFO:teuthology.orchestra.run.smithi081.stdout: "mds": "allow *", 2023-11-08T16:41:01.760 INFO:teuthology.orchestra.run.smithi081.stdout: "mon": "allow profile mgr", 2023-11-08T16:41:01.760 INFO:teuthology.orchestra.run.smithi081.stdout: "osd": "allow *" 2023-11-08T16:41:01.760 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.760 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:01.760 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2023-11-08T16:41:01.761 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:41:01.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 -- 172.21.15.81:0/3353750628 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f659806aea0 msgr2=0x7f659806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f659806aea0 0x7f659806d360 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f659c035000 tx=0x7f659c030000 comp rx=0 tx=0).stop 2023-11-08T16:41:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 -- 172.21.15.81:0/3353750628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ac0b3ec0 msgr2=0x7f65ac0ad2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ac0b3ec0 0x7f65ac0ad2c0 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f65a000d400 tx=0x7f65a000d8c0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 -- 172.21.15.81:0/3353750628 shutdown_connections 2023-11-08T16:41:01.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f659806aea0 0x7f659806d360 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f65ac151e40 0x7f65ac0ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65ac0b3ec0 0x7f65ac0ad2c0 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 --2- 172.21.15.81:0/3353750628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65940039c0 0x7f65ac0c3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 -- 172.21.15.81:0/3353750628 >> 172.21.15.81:0/3353750628 conn(0x7f65ac0ceaa0 msgr2=0x7f65ac159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:01.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 -- 172.21.15.81:0/3353750628 shutdown_connections 2023-11-08T16:41:01.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.723+0000 7f65b29ce640 1 -- 172.21.15.81:0/3353750628 wait complete. 2023-11-08T16:41:01.765 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:41:01.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.975+0000 7f893dd39640 1 Processor -- start 2023-11-08T16:41:01.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.976+0000 7f893dd39640 1 -- start start 2023-11-08T16:41:01.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.977+0000 7f893dd39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89380ca480 0x7f893815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.977+0000 7f893dd39640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893806b630 con 0x7f89380ade50 2023-11-08T16:41:01.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.977+0000 7f893dd39640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89380ca8e0 con 0x7f893815ea60 2023-11-08T16:41:01.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.977+0000 7f893dd39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89380b1860 con 0x7f89380ca480 2023-11-08T16:41:01.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.977+0000 7f89377fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f89380ade50 0x7f893815a680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:33000/0 (socket says 172.21.15.81:33000) 2023-11-08T16:41:01.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.977+0000 7f89377fe640 1 -- 172.21.15.81:0/3962824385 learned_addr learned my addr 172.21.15.81:0/3962824385 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:01.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f892ffff640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89380ca480 0x7f893815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079177135 0 0) 0x7f893806b630 con 0x7f89380ade50 2023-11-08T16:41:01.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8924003520 con 0x7f89380ade50 2023-11-08T16:41:01.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 602474847 0 0) 0x7f8924003520 con 0x7f89380ade50 2023-11-08T16:41:01.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89380ca480 msgr2=0x7f893815ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:01.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89380ca480 0x7f893815ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 >> v1:172.21.15.93:6789/0 conn(0x7f893815ea60 legacy=0x7f893815eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89380cabb0 con 0x7f89380ade50 2023-11-08T16:41:01.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f8928001670 con 0x7f89380ade50 2023-11-08T16:41:01.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 >> v1:172.21.15.81:6789/0 conn(0x7f89380ade50 legacy=0x7f893815a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89380ca480 0x7f8924003a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 0x7f89240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924004d50 0x7f89240051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892400bb90 con 0x7f8924004050 2023-11-08T16:41:01.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892400bd00 con 0x7f8924004d50 2023-11-08T16:41:01.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.978+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f892400bec0 con 0x7f89380ca480 2023-11-08T16:41:01.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.979+0000 7f892ffff640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 0x7f89240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.979+0000 7f892ffff640 1 -- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89380ca480 msgr2=0x7f8924003a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:01.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.979+0000 7f89377fe640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89380ca480 0x7f8924003a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.979+0000 7f8937fff640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924004d50 0x7f89240051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.979+0000 7f892ffff640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f89380ca480 0x7f8924003a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.979+0000 7f892ffff640 1 -- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924004d50 msgr2=0x7f89240051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:01.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.979+0000 7f892ffff640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924004d50 0x7f89240051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.979+0000 7f892ffff640 1 -- 172.21.15.81:0/3962824385 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8924003520 con 0x7f8924004050 2023-11-08T16:41:01.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.979+0000 7f8937fff640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8924004d50 0x7f89240051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:01.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.980+0000 7f892ffff640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 0x7f89240047c0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f8920009ac0 tx=0x7f89240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258b6f7e345c5e3b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:01.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.980+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892002c6b0 con 0x7f8924004050 2023-11-08T16:41:01.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.980+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892002c810 con 0x7f8924004050 2023-11-08T16:41:01.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.980+0000 7f8936ffd640 1 -- 172.21.15.81:0/3962824385 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f892002cba0 con 0x7f8924004050 2023-11-08T16:41:01.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.981+0000 7f893dd39640 1 -- 172.21.15.81:0/3962824385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 msgr2=0x7f89240047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.981+0000 7f893dd39640 1 --2- 172.21.15.81:0/3962824385 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 0x7f89240047c0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f8920009ac0 tx=0x7f89240057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.981+0000 7f8937fff640 1 -- 172.21.15.81:0/3962824385 reap_dead start 2023-11-08T16:41:01.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.981+0000 7f893dd39640 1 -- 172.21.15.81:0/3962824385 shutdown_connections 2023-11-08T16:41:01.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.981+0000 7f893dd39640 1 -- 172.21.15.81:0/3962824385 >> 172.21.15.81:0/3962824385 conn(0x7f89380cb490 msgr2=0x7f89380cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:01.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.981+0000 7f893dd39640 1 -- 172.21.15.81:0/3962824385 shutdown_connections 2023-11-08T16:41:01.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.981+0000 7f893dd39640 1 -- 172.21.15.81:0/3962824385 wait complete. 2023-11-08T16:41:01.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.983+0000 7f893dd39640 1 Processor -- start 2023-11-08T16:41:01.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.983+0000 7f893dd39640 1 -- start start 2023-11-08T16:41:01.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.984+0000 7f893dd39640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 0x7f893815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.984+0000 7f893dd39640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89380c52c0 0x7f8938159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.984+0000 7f893dd39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f893815a290 0x7f89380aef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:01.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.984+0000 7f893dd39640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893815b490 con 0x7f8924004050 2023-11-08T16:41:01.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.984+0000 7f893dd39640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893815bdf0 con 0x7f89380c52c0 2023-11-08T16:41:01.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.984+0000 7f893dd39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893815a810 con 0x7f893815a290 2023-11-08T16:41:01.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.984+0000 7f89377fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 0x7f893815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.984+0000 7f89377fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 0x7f893815b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58686/0 (socket says 172.21.15.81:58686) 2023-11-08T16:41:01.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.984+0000 7f89377fe640 1 -- 172.21.15.81:0/350030253 learned_addr learned my addr 172.21.15.81:0/350030253 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:01.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f8937fff640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f893815a290 0x7f89380aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f89377fe640 1 -- 172.21.15.81:0/350030253 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f893815a290 msgr2=0x7f89380aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f892ffff640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89380c52c0 0x7f8938159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:01.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f89377fe640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f893815a290 0x7f89380aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f89377fe640 1 -- 172.21.15.81:0/350030253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89380c52c0 msgr2=0x7f8938159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:01.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f89377fe640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89380c52c0 0x7f8938159d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:01.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f89377fe640 1 -- 172.21.15.81:0/350030253 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893815aaf0 con 0x7f8924004050 2023-11-08T16:41:01.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f89377fe640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 0x7f893815b8b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f892800d780 tx=0x7f892800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:01.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f8934ff9640 1 -- 172.21.15.81:0/350030253 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89280074e0 con 0x7f8924004050 2023-11-08T16:41:01.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f8934ff9640 1 -- 172.21.15.81:0/350030253 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f892800de90 con 0x7f8924004050 2023-11-08T16:41:01.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f893dd39640 1 -- 172.21.15.81:0/350030253 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893815add0 con 0x7f8924004050 2023-11-08T16:41:01.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.985+0000 7f8934ff9640 1 -- 172.21.15.81:0/350030253 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89280049e0 con 0x7f8924004050 2023-11-08T16:41:01.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.986+0000 7f893dd39640 1 -- 172.21.15.81:0/350030253 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89381e69c0 con 0x7f8924004050 2023-11-08T16:41:01.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.987+0000 7f893dd39640 1 -- 172.21.15.81:0/350030253 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893815b5d0 con 0x7f8924004050 2023-11-08T16:41:01.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.987+0000 7f8934ff9640 1 -- 172.21.15.81:0/350030253 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8928004b40 con 0x7f8924004050 2023-11-08T16:41:01.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.988+0000 7f8934ff9640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f891806ae70 0x7f891806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.988+0000 7f8934ff9640 1 -- 172.21.15.81:0/350030253 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8928088d10 con 0x7f8924004050 2023-11-08T16:41:02.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.988+0000 7f892ffff640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f891806ae70 0x7f891806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.989+0000 7f892ffff640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f891806ae70 0x7f891806d330 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8920036000 tx=0x7f8920030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:02.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:01.992+0000 7f8934ff9640 1 -- 172.21.15.81:0/350030253 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f893815b5d0 con 0x7f8924004050 2023-11-08T16:41:02.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.187+0000 7f893dd39640 1 -- 172.21.15.81:0/350030253 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f893815b5d0 con 0x7f8924004050 2023-11-08T16:41:02.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.189+0000 7f8934ff9640 1 -- 172.21.15.81:0/350030253 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+12807 (secure 0 0 0) 0x7f893815b5d0 con 0x7f8924004050 2023-11-08T16:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:02.191 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":123,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:01.200635+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":104,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":107,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:41:02.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 -- 172.21.15.81:0/350030253 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f891806ae70 msgr2=0x7f891806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:02.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f891806ae70 0x7f891806d330 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f8920036000 tx=0x7f8920030000 comp rx=0 tx=0).stop 2023-11-08T16:41:02.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 -- 172.21.15.81:0/350030253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 msgr2=0x7f893815b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:02.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 0x7f893815b8b0 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f892800d780 tx=0x7f892800dc40 comp rx=0 tx=0).stop 2023-11-08T16:41:02.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 -- 172.21.15.81:0/350030253 shutdown_connections 2023-11-08T16:41:02.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f891806ae70 0x7f891806d330 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f893815a290 0x7f89380aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f89380c52c0 0x7f8938159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 --2- 172.21.15.81:0/350030253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8924004050 0x7f893815b8b0 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 -- 172.21.15.81:0/350030253 >> 172.21.15.81:0/350030253 conn(0x7f89380cb490 msgr2=0x7f893806c580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:02.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 -- 172.21.15.81:0/350030253 shutdown_connections 2023-11-08T16:41:02.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.192+0000 7f893dd39640 1 -- 172.21.15.81:0/350030253 wait complete. 2023-11-08T16:41:02.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:41:02.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:41:02.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.425+0000 7f55aaee5640 1 Processor -- start 2023-11-08T16:41:02.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.425+0000 7f55aaee5640 1 -- start start 2023-11-08T16:41:02.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.426+0000 7f55aaee5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f55a40ce0d0 0x7f55a40caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55aaee5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55a415bfc0 0x7f55a415eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55aaee5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f55a415f0b0 0x7f55a41516e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55aaee5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a415f540 con 0x7f55a40ce0d0 2023-11-08T16:41:02.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55aaee5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55a406fa30 con 0x7f55a415f0b0 2023-11-08T16:41:02.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55aaee5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40ce4b0 con 0x7f55a415bfc0 2023-11-08T16:41:02.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55a8c81640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f55a40ce0d0 0x7f55a40caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55a8c81640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f55a40ce0d0 0x7f55a40caf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58688/0 (socket says 172.21.15.81:58688) 2023-11-08T16:41:02.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55a8c81640 1 -- 172.21.15.81:0/3701192978 learned_addr learned my addr 172.21.15.81:0/3701192978 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:02.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55a3fff640 1 --2- 172.21.15.81:0/3701192978 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55a415bfc0 0x7f55a415eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55a8c81640 1 -- 172.21.15.81:0/3701192978 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55a415bfc0 msgr2=0x7f55a415eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:02.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55a8c81640 1 --2- 172.21.15.81:0/3701192978 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55a415bfc0 0x7f55a415eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55a9482640 1 --2- 172.21.15.81:0/3701192978 >> v2:172.21.15.93:3300/0 conn(0x7f55a415f0b0 0x7f55a41516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55a8c81640 1 -- 172.21.15.81:0/3701192978 >> v2:172.21.15.93:3300/0 conn(0x7f55a415f0b0 msgr2=0x7f55a41516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:02.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55a8c81640 1 --2- 172.21.15.81:0/3701192978 >> v2:172.21.15.93:3300/0 conn(0x7f55a415f0b0 0x7f55a41516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.427+0000 7f55a8c81640 1 -- 172.21.15.81:0/3701192978 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a406fca0 con 0x7f55a40ce0d0 2023-11-08T16:41:02.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.428+0000 7f55a8c81640 1 --2- 172.21.15.81:0/3701192978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a40ce0d0 0x7f55a40caf20 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f55a406f920 tx=0x7f559802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee86f9ea0527892 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:02.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.429+0000 7f55a37fe640 1 -- 172.21.15.81:0/3701192978 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598031860 con 0x7f55a40ce0d0 2023-11-08T16:41:02.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.429+0000 7f55a37fe640 1 -- 172.21.15.81:0/3701192978 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55980319c0 con 0x7f55a40ce0d0 2023-11-08T16:41:02.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.429+0000 7f55a37fe640 1 -- 172.21.15.81:0/3701192978 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598031d50 con 0x7f55a40ce0d0 2023-11-08T16:41:02.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.429+0000 7f55aaee5640 1 -- 172.21.15.81:0/3701192978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a40ce0d0 msgr2=0x7f55a40caf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:02.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.429+0000 7f55aaee5640 1 --2- 172.21.15.81:0/3701192978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a40ce0d0 0x7f55a40caf20 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f55a406f920 tx=0x7f559802b900 comp rx=0 tx=0).stop 2023-11-08T16:41:02.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.429+0000 7f55aaee5640 1 -- 172.21.15.81:0/3701192978 shutdown_connections 2023-11-08T16:41:02.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.429+0000 7f55aaee5640 1 --2- 172.21.15.81:0/3701192978 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55a415bfc0 0x7f55a415eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.429+0000 7f55aaee5640 1 --2- 172.21.15.81:0/3701192978 >> v2:172.21.15.93:3300/0 conn(0x7f55a415f0b0 0x7f55a41516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.429+0000 7f55aaee5640 1 --2- 172.21.15.81:0/3701192978 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a40ce0d0 0x7f55a40caf20 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.429+0000 7f55aaee5640 1 -- 172.21.15.81:0/3701192978 >> 172.21.15.81:0/3701192978 conn(0x7f55a40cb580 msgr2=0x7f55a40cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:02.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.430+0000 7f55aaee5640 1 -- 172.21.15.81:0/3701192978 shutdown_connections 2023-11-08T16:41:02.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.430+0000 7f55aaee5640 1 -- 172.21.15.81:0/3701192978 wait complete. 2023-11-08T16:41:02.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.431+0000 7f55aaee5640 1 Processor -- start 2023-11-08T16:41:02.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.431+0000 7f55aaee5640 1 -- start start 2023-11-08T16:41:02.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.432+0000 7f55aaee5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a40ce0d0 0x7f55a40b3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.432+0000 7f55aaee5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55a415bfc0 0x7f55a4157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.432+0000 7f55aaee5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55a415f0b0 0x7f55a40c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.432+0000 7f55aaee5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40c2ab0 con 0x7f55a40ce0d0 2023-11-08T16:41:02.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.432+0000 7f55aaee5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a41575d0 con 0x7f55a415bfc0 2023-11-08T16:41:02.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.432+0000 7f55aaee5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55a40b42d0 con 0x7f55a415f0b0 2023-11-08T16:41:02.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.433+0000 7f55a8c81640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a40ce0d0 0x7f55a40b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.433+0000 7f55a8c81640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a40ce0d0 0x7f55a40b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58692/0 (socket says 172.21.15.81:58692) 2023-11-08T16:41:02.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.433+0000 7f55a8c81640 1 -- 172.21.15.81:0/3442076889 learned_addr learned my addr 172.21.15.81:0/3442076889 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:02.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.433+0000 7f55a8c81640 1 -- 172.21.15.81:0/3442076889 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55a415f0b0 msgr2=0x7f55a40c51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:02.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.433+0000 7f55a8c81640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55a415f0b0 0x7f55a40c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.433+0000 7f55a8c81640 1 -- 172.21.15.81:0/3442076889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55a415bfc0 msgr2=0x7f55a4157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:02.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.433+0000 7f55a8c81640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55a415bfc0 0x7f55a4157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.433+0000 7f55a8c81640 1 -- 172.21.15.81:0/3442076889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55980094d0 con 0x7f55a40ce0d0 2023-11-08T16:41:02.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.433+0000 7f55a8c81640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a40ce0d0 0x7f55a40b3730 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f5598000c00 tx=0x7f55980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:02.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.433+0000 7f55a17fa640 1 -- 172.21.15.81:0/3442076889 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559803ad10 con 0x7f55a40ce0d0 2023-11-08T16:41:02.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.434+0000 7f55a17fa640 1 -- 172.21.15.81:0/3442076889 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559803ae70 con 0x7f55a40ce0d0 2023-11-08T16:41:02.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.434+0000 7f55a17fa640 1 -- 172.21.15.81:0/3442076889 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598044610 con 0x7f55a40ce0d0 2023-11-08T16:41:02.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.434+0000 7f55aaee5640 1 -- 172.21.15.81:0/3442076889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a40b3d10 con 0x7f55a40ce0d0 2023-11-08T16:41:02.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.434+0000 7f55aaee5640 1 -- 172.21.15.81:0/3442076889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a40c26d0 con 0x7f55a40ce0d0 2023-11-08T16:41:02.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.435+0000 7f55aaee5640 1 -- 172.21.15.81:0/3442076889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a404ece0 con 0x7f55a40ce0d0 2023-11-08T16:41:02.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.436+0000 7f55a17fa640 1 -- 172.21.15.81:0/3442076889 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5598044770 con 0x7f55a40ce0d0 2023-11-08T16:41:02.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.436+0000 7f55a17fa640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558806ae70 0x7f558806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.437+0000 7f55a17fa640 1 -- 172.21.15.81:0/3442076889 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f55980abb10 con 0x7f55a40ce0d0 2023-11-08T16:41:02.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.437+0000 7f55a3fff640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558806ae70 0x7f558806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.437+0000 7f55a3fff640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558806ae70 0x7f558806d330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5590006b10 tx=0x7f5590008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:02.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.441+0000 7f55a17fa640 1 -- 172.21.15.81:0/3442076889 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5598076f90 con 0x7f55a40ce0d0 2023-11-08T16:41:02.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.628+0000 7f55aaee5640 1 -- 172.21.15.81:0/3442076889 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f55a40ce6b0 con 0x7f55a40ce0d0 2023-11-08T16:41:02.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.629+0000 7f55a17fa640 1 -- 172.21.15.81:0/3442076889 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f559807a5b0 con 0x7f55a40ce0d0 2023-11-08T16:41:02.631 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:41:02.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.631+0000 7f55aaee5640 1 -- 172.21.15.81:0/3442076889 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558806ae70 msgr2=0x7f558806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:02.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.631+0000 7f55aaee5640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558806ae70 0x7f558806d330 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f5590006b10 tx=0x7f5590008040 comp rx=0 tx=0).stop 2023-11-08T16:41:02.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.631+0000 7f55aaee5640 1 -- 172.21.15.81:0/3442076889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a40ce0d0 msgr2=0x7f55a40b3730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:02.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.631+0000 7f55aaee5640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a40ce0d0 0x7f55a40b3730 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f5598000c00 tx=0x7f55980046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.631+0000 7f55aaee5640 1 -- 172.21.15.81:0/3442076889 shutdown_connections 2023-11-08T16:41:02.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.631+0000 7f55aaee5640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558806ae70 0x7f558806d330 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.631+0000 7f55aaee5640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55a415f0b0 0x7f55a40c51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.631+0000 7f55aaee5640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55a415bfc0 0x7f55a4157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.631+0000 7f55aaee5640 1 --2- 172.21.15.81:0/3442076889 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55a40ce0d0 0x7f55a40b3730 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.631+0000 7f55aaee5640 1 -- 172.21.15.81:0/3442076889 >> 172.21.15.81:0/3442076889 conn(0x7f55a40cb580 msgr2=0x7f55a40afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:02.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.632+0000 7f55aaee5640 1 -- 172.21.15.81:0/3442076889 shutdown_connections 2023-11-08T16:41:02.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.632+0000 7f55aaee5640 1 -- 172.21.15.81:0/3442076889 wait complete. 2023-11-08T16:41:02.651 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-08T16:41:02.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.855+0000 7f0988597640 1 Processor -- start 2023-11-08T16:41:02.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.855+0000 7f0988597640 1 -- start start 2023-11-08T16:41:02.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.856+0000 7f0988597640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f098015d310 0x7f09800d2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.856+0000 7f0988597640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09800d1300 0x7f09800cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.856+0000 7f0988597640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09800d0470 con 0x7f09800d0000 2023-11-08T16:41:02.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.857+0000 7f0988597640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09800b6480 con 0x7f098015d310 2023-11-08T16:41:02.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.857+0000 7f0988597640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09800b6100 con 0x7f09800d1300 2023-11-08T16:41:02.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.857+0000 7f0986b34640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f09800d0000 0x7f09800b4b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:33008/0 (socket says 172.21.15.81:33008) 2023-11-08T16:41:02.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.857+0000 7f0986b34640 1 -- 172.21.15.81:0/424300490 learned_addr learned my addr 172.21.15.81:0/424300490 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:02.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.857+0000 7f0986333640 1 --2- 172.21.15.81:0/424300490 >> v2:172.21.15.93:3300/0 conn(0x7f098015d310 0x7f09800d2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.857+0000 7f0985b32640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09800d1300 0x7f09800cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.857+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129797725 0 0) 0x7f09800d0470 con 0x7f09800d0000 2023-11-08T16:41:02.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.857+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f096c003520 con 0x7f09800d0000 2023-11-08T16:41:02.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3993002256 0 0) 0x7f096c003520 con 0x7f09800d0000 2023-11-08T16:41:02.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09800d1300 msgr2=0x7f09800cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:02.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09800d1300 0x7f09800cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 >> v2:172.21.15.93:3300/0 conn(0x7f098015d310 msgr2=0x7f09800d2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:02.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 --2- 172.21.15.81:0/424300490 >> v2:172.21.15.93:3300/0 conn(0x7f098015d310 0x7f09800d2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09800d1b50 con 0x7f09800d0000 2023-11-08T16:41:02.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f09780019f0 con 0x7f09800d0000 2023-11-08T16:41:02.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985b32640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09800d1300 0x7f09800cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:41:02.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 >> v1:172.21.15.81:6789/0 conn(0x7f09800d0000 legacy=0x7f09800b4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:02.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f096c003970 0x7f096c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09800d1300 0x7f096c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c004d00 0x7f096c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800b6480 con 0x7f096c003970 2023-11-08T16:41:02.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800b6100 con 0x7f096c004d00 2023-11-08T16:41:02.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c00baf0 con 0x7f09800d1300 2023-11-08T16:41:02.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0986333640 1 --2- 172.21.15.81:0/424300490 >> v2:172.21.15.93:3300/0 conn(0x7f098015d310 0x7f09800d2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:41:02.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0986333640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f096c003970 0x7f096c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.858+0000 7f0985b32640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09800d1300 0x7f096c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.859+0000 7f0986b34640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c004d00 0x7f096c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.859+0000 7f0986333640 1 -- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09800d1300 msgr2=0x7f096c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:02.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.859+0000 7f0986333640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f09800d1300 0x7f096c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.859+0000 7f0986333640 1 -- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c004d00 msgr2=0x7f096c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:02.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.859+0000 7f0986333640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c004d00 0x7f096c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.859+0000 7f0986333640 1 -- 172.21.15.81:0/424300490 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c003520 con 0x7f096c003970 2023-11-08T16:41:02.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.860+0000 7f0986333640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f096c003970 0x7f096c003d90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f097c0098a0 tx=0x7f096c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d42694fb1172343 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:02.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.860+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c037600 con 0x7f096c003970 2023-11-08T16:41:02.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.860+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097c037760 con 0x7f096c003970 2023-11-08T16:41:02.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.860+0000 7f0985331640 1 -- 172.21.15.81:0/424300490 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097c037ad0 con 0x7f096c003970 2023-11-08T16:41:02.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.861+0000 7f0988597640 1 -- 172.21.15.81:0/424300490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f096c003970 msgr2=0x7f096c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:02.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.861+0000 7f0988597640 1 --2- 172.21.15.81:0/424300490 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f096c003970 0x7f096c003d90 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f097c0098a0 tx=0x7f096c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.861+0000 7f0986b34640 1 -- 172.21.15.81:0/424300490 reap_dead start 2023-11-08T16:41:02.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.861+0000 7f0988597640 1 -- 172.21.15.81:0/424300490 shutdown_connections 2023-11-08T16:41:02.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.861+0000 7f0988597640 1 -- 172.21.15.81:0/424300490 >> 172.21.15.81:0/424300490 conn(0x7f098006aac0 msgr2=0x7f098006aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:02.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.861+0000 7f0988597640 1 -- 172.21.15.81:0/424300490 shutdown_connections 2023-11-08T16:41:02.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.861+0000 7f0988597640 1 -- 172.21.15.81:0/424300490 wait complete. 2023-11-08T16:41:02.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.863+0000 7f0988597640 1 Processor -- start 2023-11-08T16:41:02.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.863+0000 7f0988597640 1 -- start start 2023-11-08T16:41:02.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0988597640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f096c003970 0x7f098014e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0988597640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09800c88e0 0x7f09800b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0988597640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09800b7280 0x7f09800c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0988597640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09800b7710 con 0x7f09800c88e0 2023-11-08T16:41:02.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0988597640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098014eb90 con 0x7f09800b7280 2023-11-08T16:41:02.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0988597640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09800c7840 con 0x7f096c003970 2023-11-08T16:41:02.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0985b32640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09800c88e0 0x7f09800b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0985b32640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09800c88e0 0x7f09800b6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58698/0 (socket says 172.21.15.81:58698) 2023-11-08T16:41:02.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0985b32640 1 -- 172.21.15.81:0/1326241187 learned_addr learned my addr 172.21.15.81:0/1326241187 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:02.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0985b32640 1 -- 172.21.15.81:0/1326241187 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f096c003970 msgr2=0x7f098014e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:02.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0986333640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f096c003970 0x7f098014e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0986b34640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09800b7280 0x7f09800c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0985b32640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f096c003970 0x7f098014e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0985b32640 1 -- 172.21.15.81:0/1326241187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09800b7280 msgr2=0x7f09800c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:02.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0985b32640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09800b7280 0x7f09800c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.864+0000 7f0985b32640 1 -- 172.21.15.81:0/1326241187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09800c7b20 con 0x7f09800c88e0 2023-11-08T16:41:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.865+0000 7f0985b32640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09800c88e0 0x7f09800b6d40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f0970010860 tx=0x7f0970010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.865+0000 7f0976ffd640 1 -- 172.21.15.81:0/1326241187 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f097000cae0 con 0x7f09800c88e0 2023-11-08T16:41:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.865+0000 7f0976ffd640 1 -- 172.21.15.81:0/1326241187 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f097000cc40 con 0x7f09800c88e0 2023-11-08T16:41:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.865+0000 7f0976ffd640 1 -- 172.21.15.81:0/1326241187 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0970010e50 con 0x7f09800c88e0 2023-11-08T16:41:02.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.865+0000 7f0988597640 1 -- 172.21.15.81:0/1326241187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09800c7e00 con 0x7f09800c88e0 2023-11-08T16:41:02.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.865+0000 7f0988597640 1 -- 172.21.15.81:0/1326241187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09800c83c0 con 0x7f09800c88e0 2023-11-08T16:41:02.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.866+0000 7f0988597640 1 -- 172.21.15.81:0/1326241187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09800743f0 con 0x7f09800c88e0 2023-11-08T16:41:02.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.867+0000 7f0976ffd640 1 -- 172.21.15.81:0/1326241187 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0970015470 con 0x7f09800c88e0 2023-11-08T16:41:02.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.868+0000 7f0976ffd640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f096406aea0 0x7f096406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:02.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.868+0000 7f0976ffd640 1 -- 172.21.15.81:0/1326241187 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f09700899b0 con 0x7f09800c88e0 2023-11-08T16:41:02.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.868+0000 7f0986333640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f096406aea0 0x7f096406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:02.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.869+0000 7f0986333640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f096406aea0 0x7f096406d360 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f097c036000 tx=0x7f097c031000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:02.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:02.872+0000 7f0976ffd640 1 -- 172.21.15.81:0/1326241187 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0970054da0 con 0x7f09800c88e0 2023-11-08T16:41:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.056+0000 7f0988597640 1 -- 172.21.15.81:0/1326241187 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f09801e7af0 con 0x7f09800c88e0 2023-11-08T16:41:03.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.056+0000 7f0976ffd640 1 -- 172.21.15.81:0/1326241187 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v123) v1 ==== 84+0+65 (secure 0 0 0) 0x7f09700583c0 con 0x7f09800c88e0 2023-11-08T16:41:03.059 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:03.059 INFO:teuthology.orchestra.run.smithi081.stdout:[ 2023-11-08T16:41:03.059 INFO:teuthology.orchestra.run.smithi081.stdout: { 2023-11-08T16:41:03.059 INFO:teuthology.orchestra.run.smithi081.stdout: "poolnum": 1, 2023-11-08T16:41:03.060 INFO:teuthology.orchestra.run.smithi081.stdout: "poolname": ".mgr" 2023-11-08T16:41:03.060 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:03.060 INFO:teuthology.orchestra.run.smithi081.stdout:] 2023-11-08T16:41:03.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 -- 172.21.15.81:0/1326241187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f096406aea0 msgr2=0x7f096406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:03.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f096406aea0 0x7f096406d360 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f097c036000 tx=0x7f097c031000 comp rx=0 tx=0).stop 2023-11-08T16:41:03.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 -- 172.21.15.81:0/1326241187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09800c88e0 msgr2=0x7f09800b6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:03.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09800c88e0 0x7f09800b6d40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f0970010860 tx=0x7f0970010d20 comp rx=0 tx=0).stop 2023-11-08T16:41:03.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 -- 172.21.15.81:0/1326241187 shutdown_connections 2023-11-08T16:41:03.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f096406aea0 0x7f096406d360 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f09800b7280 0x7f09800c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f09800c88e0 0x7f09800b6d40 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 --2- 172.21.15.81:0/1326241187 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f096c003970 0x7f098014e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 -- 172.21.15.81:0/1326241187 >> 172.21.15.81:0/1326241187 conn(0x7f098006aac0 msgr2=0x7f09800cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:03.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 -- 172.21.15.81:0/1326241187 shutdown_connections 2023-11-08T16:41:03.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.059+0000 7f0988597640 1 -- 172.21.15.81:0/1326241187 wait complete. 2023-11-08T16:41:03.078 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-08T16:41:03.079 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-08T16:41:03.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.288+0000 7f515fb39640 1 Processor -- start 2023-11-08T16:41:03.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.288+0000 7f515fb39640 1 -- start start 2023-11-08T16:41:03.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.290+0000 7f515fb39640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f515815e170 0x7f515815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:03.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.290+0000 7f515fb39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f515815e550 0x7f51580b04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:03.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.290+0000 7f515fb39640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f51580b0a20 0x7f51580ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:03.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.290+0000 7f515fb39640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51580b0e70 con 0x7f51580b0a20 2023-11-08T16:41:03.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.290+0000 7f515fb39640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5158071460 con 0x7f515815e170 2023-11-08T16:41:03.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.290+0000 7f515fb39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51580703c0 con 0x7f515815e550 2023-11-08T16:41:03.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.290+0000 7f515e0d6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f51580b0a20 0x7f51580ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:03.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.290+0000 7f515e0d6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f51580b0a20 0x7f51580ad4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58706/0 (socket says 172.21.15.81:58706) 2023-11-08T16:41:03.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.290+0000 7f515e0d6640 1 -- 172.21.15.81:0/3557309149 learned_addr learned my addr 172.21.15.81:0/3557309149 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:03.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.290+0000 7f515d8d5640 1 --2- 172.21.15.81:0/3557309149 >> v2:172.21.15.93:3300/0 conn(0x7f515815e170 0x7f515815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:03.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.290+0000 7f515d0d4640 1 --2- 172.21.15.81:0/3557309149 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f515815e550 0x7f51580b04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:03.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.291+0000 7f515e0d6640 1 -- 172.21.15.81:0/3557309149 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f515815e550 msgr2=0x7f51580b04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:03.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.291+0000 7f515e0d6640 1 --2- 172.21.15.81:0/3557309149 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f515815e550 0x7f51580b04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.291+0000 7f515e0d6640 1 -- 172.21.15.81:0/3557309149 >> v2:172.21.15.93:3300/0 conn(0x7f515815e170 msgr2=0x7f515815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:03.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.291+0000 7f515e0d6640 1 --2- 172.21.15.81:0/3557309149 >> v2:172.21.15.93:3300/0 conn(0x7f515815e170 0x7f515815ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.291+0000 7f515e0d6640 1 -- 172.21.15.81:0/3557309149 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51580706a0 con 0x7f51580b0a20 2023-11-08T16:41:03.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.292+0000 7f515e0d6640 1 --2- 172.21.15.81:0/3557309149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51580b0a20 0x7f51580ad4f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f515806d3e0 tx=0x7f515002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9aab82b679b40e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:03.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.292+0000 7f515c8d3640 1 -- 172.21.15.81:0/3557309149 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515002ca70 con 0x7f51580b0a20 2023-11-08T16:41:03.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.292+0000 7f515c8d3640 1 -- 172.21.15.81:0/3557309149 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515002cbd0 con 0x7f51580b0a20 2023-11-08T16:41:03.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.292+0000 7f515c8d3640 1 -- 172.21.15.81:0/3557309149 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51500343f0 con 0x7f51580b0a20 2023-11-08T16:41:03.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.293+0000 7f515fb39640 1 -- 172.21.15.81:0/3557309149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51580b0a20 msgr2=0x7f51580ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:03.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.293+0000 7f515fb39640 1 --2- 172.21.15.81:0/3557309149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51580b0a20 0x7f51580ad4f0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f515806d3e0 tx=0x7f515002bed0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.293+0000 7f515fb39640 1 -- 172.21.15.81:0/3557309149 shutdown_connections 2023-11-08T16:41:03.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.293+0000 7f515fb39640 1 --2- 172.21.15.81:0/3557309149 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f515815e550 0x7f51580b04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.293+0000 7f515fb39640 1 --2- 172.21.15.81:0/3557309149 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f51580b0a20 0x7f51580ad4f0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.293+0000 7f515fb39640 1 --2- 172.21.15.81:0/3557309149 >> v2:172.21.15.93:3300/0 conn(0x7f515815e170 0x7f515815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.293+0000 7f515fb39640 1 -- 172.21.15.81:0/3557309149 >> 172.21.15.81:0/3557309149 conn(0x7f51580cb4e0 msgr2=0x7f51580cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:03.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.293+0000 7f515fb39640 1 -- 172.21.15.81:0/3557309149 shutdown_connections 2023-11-08T16:41:03.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.293+0000 7f515fb39640 1 -- 172.21.15.81:0/3557309149 wait complete. 2023-11-08T16:41:03.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.295+0000 7f515fb39640 1 Processor -- start 2023-11-08T16:41:03.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.295+0000 7f515fb39640 1 -- start start 2023-11-08T16:41:03.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515fb39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51580b0a20 0x7f51581f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:03.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515fb39640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f515815e170 0x7f51580c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:03.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515fb39640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f515815e550 0x7f51580c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:03.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515fb39640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5158145460 con 0x7f515815e550 2023-11-08T16:41:03.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515fb39640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5158151d20 con 0x7f515815e170 2023-11-08T16:41:03.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515fb39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51581f2f50 con 0x7f51580b0a20 2023-11-08T16:41:03.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515e0d6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f515815e550 0x7f51580c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:03.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515e0d6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f515815e550 0x7f51580c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58720/0 (socket says 172.21.15.81:58720) 2023-11-08T16:41:03.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515e0d6640 1 -- 172.21.15.81:0/357070312 learned_addr learned my addr 172.21.15.81:0/357070312 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:03.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515d8d5640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51580b0a20 0x7f51581f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:03.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515d0d4640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f515815e170 0x7f51580c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515e0d6640 1 -- 172.21.15.81:0/357070312 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51580b0a20 msgr2=0x7f51581f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.296+0000 7f515e0d6640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51580b0a20 0x7f51581f2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.297+0000 7f515e0d6640 1 -- 172.21.15.81:0/357070312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f515815e170 msgr2=0x7f51580c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.297+0000 7f515e0d6640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f515815e170 0x7f51580c44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.297+0000 7f515e0d6640 1 -- 172.21.15.81:0/357070312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5150009b30 con 0x7f515815e550 2023-11-08T16:41:03.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.297+0000 7f515e0d6640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f515815e550 0x7f51580c49f0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f5150036040 tx=0x7f5150004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.297+0000 7f514e7fc640 1 -- 172.21.15.81:0/357070312 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150009510 con 0x7f515815e550 2023-11-08T16:41:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.297+0000 7f514e7fc640 1 -- 172.21.15.81:0/357070312 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515003e450 con 0x7f515815e550 2023-11-08T16:41:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.297+0000 7f515fb39640 1 -- 172.21.15.81:0/357070312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5158151110 con 0x7f515815e550 2023-11-08T16:41:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.297+0000 7f514e7fc640 1 -- 172.21.15.81:0/357070312 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515003e800 con 0x7f515815e550 2023-11-08T16:41:03.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.297+0000 7f515fb39640 1 -- 172.21.15.81:0/357070312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51581516d0 con 0x7f515815e550 2023-11-08T16:41:03.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.298+0000 7f515fb39640 1 -- 172.21.15.81:0/357070312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51580743f0 con 0x7f515815e550 2023-11-08T16:41:03.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.299+0000 7f514e7fc640 1 -- 172.21.15.81:0/357070312 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f515003e960 con 0x7f515815e550 2023-11-08T16:41:03.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.300+0000 7f514e7fc640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513c06aea0 0x7f513c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:03.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.300+0000 7f514e7fc640 1 -- 172.21.15.81:0/357070312 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f51500ac0c0 con 0x7f515815e550 2023-11-08T16:41:03.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.300+0000 7f515d8d5640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513c06aea0 0x7f513c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:03.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.301+0000 7f515d8d5640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513c06aea0 0x7f513c06d360 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5154006fd0 tx=0x7f5154008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:03.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.304+0000 7f514e7fc640 1 -- 172.21.15.81:0/357070312 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5150077420 con 0x7f515815e550 2023-11-08T16:41:03.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:03.494+0000 7f515fb39640 1 -- 172.21.15.81:0/357070312 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f51580ccc60 con 0x7f515815e550 2023-11-08T16:41:04.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.232+0000 7f514e7fc640 1 -- 172.21.15.81:0/357070312 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v124) v1 ==== 138+0+0 (secure 0 0 0) 0x7f515007aa40 con 0x7f515815e550 2023-11-08T16:41:04.234 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' created 2023-11-08T16:41:04.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.236+0000 7f515fb39640 1 -- 172.21.15.81:0/357070312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513c06aea0 msgr2=0x7f513c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:04.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.236+0000 7f515fb39640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513c06aea0 0x7f513c06d360 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f5154006fd0 tx=0x7f5154008040 comp rx=0 tx=0).stop 2023-11-08T16:41:04.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.236+0000 7f515fb39640 1 -- 172.21.15.81:0/357070312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f515815e550 msgr2=0x7f51580c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:04.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.236+0000 7f515fb39640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f515815e550 0x7f51580c49f0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f5150036040 tx=0x7f5150004a40 comp rx=0 tx=0).stop 2023-11-08T16:41:04.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.236+0000 7f515fb39640 1 -- 172.21.15.81:0/357070312 shutdown_connections 2023-11-08T16:41:04.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.236+0000 7f515fb39640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f513c06aea0 0x7f513c06d360 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:04.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.236+0000 7f515fb39640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f515815e550 0x7f51580c49f0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:04.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.236+0000 7f515fb39640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f515815e170 0x7f51580c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:04.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.236+0000 7f515fb39640 1 --2- 172.21.15.81:0/357070312 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f51580b0a20 0x7f51581f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:04.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.236+0000 7f515fb39640 1 -- 172.21.15.81:0/357070312 >> 172.21.15.81:0/357070312 conn(0x7f51580cb4e0 msgr2=0x7f515806b250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:04.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.236+0000 7f515fb39640 1 -- 172.21.15.81:0/357070312 shutdown_connections 2023-11-08T16:41:04.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.237+0000 7f515fb39640 1 -- 172.21.15.81:0/357070312 wait complete. 2023-11-08T16:41:04.264 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-08T16:41:04.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.464+0000 7f605a3c9640 1 Processor -- start 2023-11-08T16:41:04.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.465+0000 7f605a3c9640 1 -- start start 2023-11-08T16:41:04.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.466+0000 7f605a3c9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f60540b0770 0x7f605406f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:04.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.466+0000 7f605a3c9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f60540ae000 0x7f60540cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:04.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.466+0000 7f605a3c9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f605406fe50 con 0x7f60540b0b50 2023-11-08T16:41:04.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.466+0000 7f605a3c9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60540cb3c0 con 0x7f60540b0770 2023-11-08T16:41:04.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.466+0000 7f605a3c9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60540ca580 con 0x7f60540ae000 2023-11-08T16:41:04.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.466+0000 7f6053fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f60540b0770 0x7f605406f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:04.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.466+0000 7f6053fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f60540b0770 0x7f605406f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58730/0 (socket says 172.21.15.81:58730) 2023-11-08T16:41:04.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.466+0000 7f6053fff640 1 -- 172.21.15.81:0/3715381290 learned_addr learned my addr 172.21.15.81:0/3715381290 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:04.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.467+0000 7f60537fe640 1 --2- 172.21.15.81:0/3715381290 >> v2:172.21.15.93:3300/0 conn(0x7f60540ae000 0x7f60540cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:04.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.467+0000 7f6053fff640 1 -- 172.21.15.81:0/3715381290 >> v2:172.21.15.93:3300/0 conn(0x7f60540ae000 msgr2=0x7f60540cc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:04.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.467+0000 7f6053fff640 1 --2- 172.21.15.81:0/3715381290 >> v2:172.21.15.93:3300/0 conn(0x7f60540ae000 0x7f60540cc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:04.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.467+0000 7f6053fff640 1 -- 172.21.15.81:0/3715381290 >> v1:172.21.15.93:6789/0 conn(0x7f60540b0b50 legacy=0x7f60540b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:04.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.467+0000 7f6053fff640 1 -- 172.21.15.81:0/3715381290 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60540ccec0 con 0x7f60540b0770 2023-11-08T16:41:04.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.468+0000 7f6053fff640 1 --2- 172.21.15.81:0/3715381290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60540b0770 0x7f605406f910 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f605415e9c0 tx=0x7f604402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d388162fc8cc7e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:04.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.469+0000 7f6052ffd640 1 -- 172.21.15.81:0/3715381290 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6044031870 con 0x7f60540b0770 2023-11-08T16:41:04.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.469+0000 7f6052ffd640 1 -- 172.21.15.81:0/3715381290 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60440319d0 con 0x7f60540b0770 2023-11-08T16:41:04.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.469+0000 7f6052ffd640 1 -- 172.21.15.81:0/3715381290 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6044031d60 con 0x7f60540b0770 2023-11-08T16:41:04.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.469+0000 7f605a3c9640 1 -- 172.21.15.81:0/3715381290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60540b0770 msgr2=0x7f605406f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:04.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.469+0000 7f605a3c9640 1 --2- 172.21.15.81:0/3715381290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60540b0770 0x7f605406f910 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f605415e9c0 tx=0x7f604402b910 comp rx=0 tx=0).stop 2023-11-08T16:41:04.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.469+0000 7f605a3c9640 1 -- 172.21.15.81:0/3715381290 shutdown_connections 2023-11-08T16:41:04.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.469+0000 7f605a3c9640 1 --2- 172.21.15.81:0/3715381290 >> v2:172.21.15.93:3300/0 conn(0x7f60540ae000 0x7f60540cc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:04.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.469+0000 7f605a3c9640 1 --2- 172.21.15.81:0/3715381290 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60540b0770 0x7f605406f910 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:04.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.469+0000 7f605a3c9640 1 -- 172.21.15.81:0/3715381290 >> 172.21.15.81:0/3715381290 conn(0x7f60540cb500 msgr2=0x7f60540cb910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:04.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.469+0000 7f605a3c9640 1 -- 172.21.15.81:0/3715381290 shutdown_connections 2023-11-08T16:41:04.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.470+0000 7f605a3c9640 1 -- 172.21.15.81:0/3715381290 wait complete. 2023-11-08T16:41:04.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.471+0000 7f605a3c9640 1 Processor -- start 2023-11-08T16:41:04.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.471+0000 7f605a3c9640 1 -- start start 2023-11-08T16:41:04.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.472+0000 7f605a3c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60540ae000 0x7f605414ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:04.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.472+0000 7f605a3c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f60540b0770 0x7f60541f6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:04.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.472+0000 7f605a3c9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60540b0b50 0x7f60540abb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:04.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.472+0000 7f605a3c9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60541f6fe0 con 0x7f60540ae000 2023-11-08T16:41:04.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.472+0000 7f605a3c9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6054154d40 con 0x7f60540b0b50 2023-11-08T16:41:04.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.472+0000 7f605a3c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6054154fe0 con 0x7f60540b0770 2023-11-08T16:41:04.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.473+0000 7f6053fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60540ae000 0x7f605414ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:04.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.473+0000 7f6053fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60540ae000 0x7f605414ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58732/0 (socket says 172.21.15.81:58732) 2023-11-08T16:41:04.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.473+0000 7f6053fff640 1 -- 172.21.15.81:0/3523128008 learned_addr learned my addr 172.21.15.81:0/3523128008 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:04.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.473+0000 7f60537fe640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f60540b0770 0x7f60541f6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:04.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.473+0000 7f6053fff640 1 -- 172.21.15.81:0/3523128008 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f60540b0770 msgr2=0x7f60541f6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:04.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.473+0000 7f6053fff640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f60540b0770 0x7f60541f6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:04.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.473+0000 7f6053fff640 1 -- 172.21.15.81:0/3523128008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60540b0b50 msgr2=0x7f60540abb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:04.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.473+0000 7f6053fff640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60540b0b50 0x7f60540abb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:04.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.473+0000 7f6053fff640 1 -- 172.21.15.81:0/3523128008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440094d0 con 0x7f60540ae000 2023-11-08T16:41:04.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.474+0000 7f6053fff640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60540ae000 0x7f605414ead0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f6044000c00 tx=0x7f60440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:04.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.474+0000 7f6050ff9640 1 -- 172.21.15.81:0/3523128008 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604403abe0 con 0x7f60540ae000 2023-11-08T16:41:04.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.474+0000 7f6050ff9640 1 -- 172.21.15.81:0/3523128008 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604403ad40 con 0x7f60540ae000 2023-11-08T16:41:04.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.474+0000 7f6050ff9640 1 -- 172.21.15.81:0/3523128008 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6044044610 con 0x7f60540ae000 2023-11-08T16:41:04.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.474+0000 7f605a3c9640 1 -- 172.21.15.81:0/3523128008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60541552c0 con 0x7f60540ae000 2023-11-08T16:41:04.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.474+0000 7f605a3c9640 1 -- 172.21.15.81:0/3523128008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60540c3bb0 con 0x7f60540ae000 2023-11-08T16:41:04.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.475+0000 7f605a3c9640 1 -- 172.21.15.81:0/3523128008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60540743f0 con 0x7f60540ae000 2023-11-08T16:41:04.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.476+0000 7f6050ff9640 1 -- 172.21.15.81:0/3523128008 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f604404d450 con 0x7f60540ae000 2023-11-08T16:41:04.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.477+0000 7f6050ff9640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f604006ae70 0x7f604006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:04.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.477+0000 7f6050ff9640 1 -- 172.21.15.81:0/3523128008 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f604402d040 con 0x7f60540ae000 2023-11-08T16:41:04.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.478+0000 7f60537fe640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f604006ae70 0x7f604006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:04.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.478+0000 7f60537fe640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f604006ae70 0x7f604006d330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6048006fd0 tx=0x7f6048008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:04.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.482+0000 7f6050ff9640 1 -- 172.21.15.81:0/3523128008 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6044076ba0 con 0x7f60540ae000 2023-11-08T16:41:04.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:04.674+0000 7f605a3c9640 1 -- 172.21.15.81:0/3523128008 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f605406fe50 con 0x7f60540ae000 2023-11-08T16:41:05.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.248+0000 7f6050ff9640 1 -- 172.21.15.81:0/3523128008 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v125) v1 ==== 170+0+0 (secure 0 0 0) 0x7f605406fe50 con 0x7f60540ae000 2023-11-08T16:41:05.250 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' created 2023-11-08T16:41:05.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.251+0000 7f605a3c9640 1 -- 172.21.15.81:0/3523128008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f604006ae70 msgr2=0x7f604006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:05.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.251+0000 7f605a3c9640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f604006ae70 0x7f604006d330 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6048006fd0 tx=0x7f6048008040 comp rx=0 tx=0).stop 2023-11-08T16:41:05.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.251+0000 7f605a3c9640 1 -- 172.21.15.81:0/3523128008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60540ae000 msgr2=0x7f605414ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:05.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.251+0000 7f605a3c9640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60540ae000 0x7f605414ead0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f6044000c00 tx=0x7f60440046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.251+0000 7f605a3c9640 1 -- 172.21.15.81:0/3523128008 shutdown_connections 2023-11-08T16:41:05.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.251+0000 7f605a3c9640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f604006ae70 0x7f604006d330 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.251+0000 7f605a3c9640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f60540b0b50 0x7f60540abb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.251+0000 7f605a3c9640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f60540b0770 0x7f60541f6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.251+0000 7f605a3c9640 1 --2- 172.21.15.81:0/3523128008 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f60540ae000 0x7f605414ead0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.251+0000 7f605a3c9640 1 -- 172.21.15.81:0/3523128008 >> 172.21.15.81:0/3523128008 conn(0x7f60540cb500 msgr2=0x7f60540cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:05.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.252+0000 7f605a3c9640 1 -- 172.21.15.81:0/3523128008 shutdown_connections 2023-11-08T16:41:05.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.252+0000 7f605a3c9640 1 -- 172.21.15.81:0/3523128008 wait complete. 2023-11-08T16:41:05.282 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-08T16:41:05.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.463+0000 7f11fdea3640 1 Processor -- start 2023-11-08T16:41:05.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.463+0000 7f11fdea3640 1 -- start start 2023-11-08T16:41:05.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.465+0000 7f11fdea3640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f11f815cf30 0x7f11f80d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:05.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.465+0000 7f11fdea3640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11f815d310 0x7f11f80cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:05.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.465+0000 7f11fdea3640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f11f80cf680 0x7f11f80cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:05.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.465+0000 7f11fdea3640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f80d0050 con 0x7f11f815cf30 2023-11-08T16:41:05.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.465+0000 7f11fdea3640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f80cdb70 con 0x7f11f80cf680 2023-11-08T16:41:05.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.465+0000 7f11fdea3640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80ce0b0 con 0x7f11f815d310 2023-11-08T16:41:05.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.465+0000 7f11fcea1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f11f815cf30 0x7f11f80d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:05.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.465+0000 7f11fcea1640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f11f815cf30 0x7f11f80d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58736/0 (socket says 172.21.15.81:58736) 2023-11-08T16:41:05.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.465+0000 7f11fcea1640 1 -- 172.21.15.81:0/270486299 learned_addr learned my addr 172.21.15.81:0/270486299 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:05.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.466+0000 7f11fd6a2640 1 --2- 172.21.15.81:0/270486299 >> v2:172.21.15.93:3300/0 conn(0x7f11f80cf680 0x7f11f80cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:05.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.466+0000 7f11f7fff640 1 --2- 172.21.15.81:0/270486299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11f815d310 0x7f11f80cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:05.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.466+0000 7f11fcea1640 1 -- 172.21.15.81:0/270486299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11f815d310 msgr2=0x7f11f80cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:05.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.466+0000 7f11fcea1640 1 --2- 172.21.15.81:0/270486299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11f815d310 0x7f11f80cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.466+0000 7f11fcea1640 1 -- 172.21.15.81:0/270486299 >> v2:172.21.15.93:3300/0 conn(0x7f11f80cf680 msgr2=0x7f11f80cfb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:05.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.466+0000 7f11fcea1640 1 --2- 172.21.15.81:0/270486299 >> v2:172.21.15.93:3300/0 conn(0x7f11f80cf680 0x7f11f80cfb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.466+0000 7f11fcea1640 1 -- 172.21.15.81:0/270486299 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80ce390 con 0x7f11f815cf30 2023-11-08T16:41:05.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.466+0000 7f11fd6a2640 1 --2- 172.21.15.81:0/270486299 >> v2:172.21.15.93:3300/0 conn(0x7f11f80cf680 0x7f11f80cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:05.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.466+0000 7f11f7fff640 1 --2- 172.21.15.81:0/270486299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11f815d310 0x7f11f80cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:05.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.467+0000 7f11fcea1640 1 --2- 172.21.15.81:0/270486299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11f815cf30 0x7f11f80d26c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f11f80b8770 tx=0x7f11e802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a57676897306517 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:05.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.467+0000 7f11f77fe640 1 -- 172.21.15.81:0/270486299 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e802ca80 con 0x7f11f815cf30 2023-11-08T16:41:05.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.467+0000 7f11f77fe640 1 -- 172.21.15.81:0/270486299 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e802cbe0 con 0x7f11f815cf30 2023-11-08T16:41:05.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.467+0000 7f11f77fe640 1 -- 172.21.15.81:0/270486299 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e80343f0 con 0x7f11f815cf30 2023-11-08T16:41:05.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.468+0000 7f11fdea3640 1 -- 172.21.15.81:0/270486299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11f815cf30 msgr2=0x7f11f80d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:05.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.468+0000 7f11fdea3640 1 --2- 172.21.15.81:0/270486299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11f815cf30 0x7f11f80d26c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f11f80b8770 tx=0x7f11e802bee0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.468+0000 7f11fdea3640 1 -- 172.21.15.81:0/270486299 shutdown_connections 2023-11-08T16:41:05.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.468+0000 7f11fdea3640 1 --2- 172.21.15.81:0/270486299 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11f815d310 0x7f11f80cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.468+0000 7f11fdea3640 1 --2- 172.21.15.81:0/270486299 >> v2:172.21.15.93:3300/0 conn(0x7f11f80cf680 0x7f11f80cfb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.468+0000 7f11fdea3640 1 --2- 172.21.15.81:0/270486299 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11f815cf30 0x7f11f80d26c0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.468+0000 7f11fdea3640 1 -- 172.21.15.81:0/270486299 >> 172.21.15.81:0/270486299 conn(0x7f11f806c040 msgr2=0x7f11f806c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:05.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.468+0000 7f11fdea3640 1 -- 172.21.15.81:0/270486299 shutdown_connections 2023-11-08T16:41:05.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.468+0000 7f11fdea3640 1 -- 172.21.15.81:0/270486299 wait complete. 2023-11-08T16:41:05.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.470+0000 7f11fdea3640 1 Processor -- start 2023-11-08T16:41:05.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.470+0000 7f11fdea3640 1 -- start start 2023-11-08T16:41:05.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.471+0000 7f11fdea3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11f80cf680 0x7f11f81f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:05.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.471+0000 7f11fdea3640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11f815cf30 0x7f11f80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:05.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.471+0000 7f11fdea3640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11f815bfc0 0x7f11f80cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:05.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.471+0000 7f11fdea3640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81f3420 con 0x7f11f80cf680 2023-11-08T16:41:05.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.471+0000 7f11fdea3640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f815e240 con 0x7f11f815bfc0 2023-11-08T16:41:05.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.471+0000 7f11fdea3640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f81f5e40 con 0x7f11f815cf30 2023-11-08T16:41:05.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.471+0000 7f11fcea1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11f80cf680 0x7f11f81f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:05.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.471+0000 7f11fcea1640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11f80cf680 0x7f11f81f5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58750/0 (socket says 172.21.15.81:58750) 2023-11-08T16:41:05.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.471+0000 7f11fcea1640 1 -- 172.21.15.81:0/1227512741 learned_addr learned my addr 172.21.15.81:0/1227512741 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:05.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.471+0000 7f11fcea1640 1 -- 172.21.15.81:0/1227512741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11f815cf30 msgr2=0x7f11f80b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:05.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.471+0000 7f11fcea1640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11f815cf30 0x7f11f80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.472+0000 7f11fcea1640 1 -- 172.21.15.81:0/1227512741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11f815bfc0 msgr2=0x7f11f80cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:05.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.472+0000 7f11fcea1640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11f815bfc0 0x7f11f80cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:05.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.472+0000 7f11fcea1640 1 -- 172.21.15.81:0/1227512741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e8009b30 con 0x7f11f80cf680 2023-11-08T16:41:05.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.472+0000 7f11fcea1640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11f80cf680 0x7f11f81f5900 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f11e8000f80 tx=0x7f11e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:05.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.472+0000 7f11f57fa640 1 -- 172.21.15.81:0/1227512741 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e803e450 con 0x7f11f80cf680 2023-11-08T16:41:05.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.472+0000 7f11f57fa640 1 -- 172.21.15.81:0/1227512741 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e803e5f0 con 0x7f11f80cf680 2023-11-08T16:41:05.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.472+0000 7f11f57fa640 1 -- 172.21.15.81:0/1227512741 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e803e9a0 con 0x7f11f80cf680 2023-11-08T16:41:05.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.472+0000 7f11fdea3640 1 -- 172.21.15.81:0/1227512741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f81f36e0 con 0x7f11f80cf680 2023-11-08T16:41:05.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.472+0000 7f11fdea3640 1 -- 172.21.15.81:0/1227512741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f81f6350 con 0x7f11f80cf680 2023-11-08T16:41:05.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.473+0000 7f11fdea3640 1 -- 172.21.15.81:0/1227512741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f804ece0 con 0x7f11f80cf680 2023-11-08T16:41:05.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.474+0000 7f11f57fa640 1 -- 172.21.15.81:0/1227512741 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11e802cd50 con 0x7f11f80cf680 2023-11-08T16:41:05.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.475+0000 7f11f57fa640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11d806ae70 0x7f11d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:05.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.475+0000 7f11f57fa640 1 -- 172.21.15.81:0/1227512741 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f11e80ac240 con 0x7f11f80cf680 2023-11-08T16:41:05.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.476+0000 7f11f7fff640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11d806ae70 0x7f11d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:05.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.476+0000 7f11f7fff640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11d806ae70 0x7f11d806d330 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f11e0006fd0 tx=0x7f11e0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:05.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.480+0000 7f11f57fa640 1 -- 172.21.15.81:0/1227512741 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f11e80773d0 con 0x7f11f80cf680 2023-11-08T16:41:05.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:05.698+0000 7f11fdea3640 1 -- 172.21.15.81:0/1227512741 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f11f80d1e90 con 0x7f11f80cf680 2023-11-08T16:41:06.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.252+0000 7f11f57fa640 1 -- 172.21.15.81:0/1227512741 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v104) v1 ==== 174+0+0 (secure 0 0 0) 0x7f11e807a9f0 con 0x7f11f80cf680 2023-11-08T16:41:06.254 INFO:teuthology.orchestra.run.smithi081.stderr:new fs with metadata pool 16 and data pool 17 2023-11-08T16:41:06.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.255+0000 7f11fdea3640 1 -- 172.21.15.81:0/1227512741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11d806ae70 msgr2=0x7f11d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:06.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.255+0000 7f11fdea3640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11d806ae70 0x7f11d806d330 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f11e0006fd0 tx=0x7f11e0008040 comp rx=0 tx=0).stop 2023-11-08T16:41:06.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.255+0000 7f11fdea3640 1 -- 172.21.15.81:0/1227512741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11f80cf680 msgr2=0x7f11f81f5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:06.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.255+0000 7f11fdea3640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11f80cf680 0x7f11f81f5900 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f11e8000f80 tx=0x7f11e8004a40 comp rx=0 tx=0).stop 2023-11-08T16:41:06.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.255+0000 7f11fdea3640 1 -- 172.21.15.81:0/1227512741 shutdown_connections 2023-11-08T16:41:06.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.255+0000 7f11fdea3640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f11d806ae70 0x7f11d806d330 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.255+0000 7f11fdea3640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f11f815bfc0 0x7f11f80cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.255+0000 7f11fdea3640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f11f815cf30 0x7f11f80b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.255+0000 7f11fdea3640 1 --2- 172.21.15.81:0/1227512741 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f11f80cf680 0x7f11f81f5900 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.255+0000 7f11fdea3640 1 -- 172.21.15.81:0/1227512741 >> 172.21.15.81:0/1227512741 conn(0x7f11f806c040 msgr2=0x7f11f806c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:06.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.256+0000 7f11fdea3640 1 -- 172.21.15.81:0/1227512741 shutdown_connections 2023-11-08T16:41:06.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.256+0000 7f11fdea3640 1 -- 172.21.15.81:0/1227512741 wait complete. 2023-11-08T16:41:06.297 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:41:06.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.478+0000 7f410f361640 1 Processor -- start 2023-11-08T16:41:06.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.478+0000 7f410f361640 1 -- start start 2023-11-08T16:41:06.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.479+0000 7f410f361640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080ade50 0x7f41080ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.479+0000 7f410f361640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f410815ea60 0x7f410815eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.479+0000 7f410f361640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f410806b630 con 0x7f410815a680 2023-11-08T16:41:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.479+0000 7f410f361640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41080b1860 con 0x7f410815ea60 2023-11-08T16:41:06.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.479+0000 7f410f361640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41080ca9c0 con 0x7f41080ade50 2023-11-08T16:41:06.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.480+0000 7f410d8fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f410815ea60 0x7f410815eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:06.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.480+0000 7f410d8fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f410815ea60 0x7f410815eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58758/0 (socket says 172.21.15.81:58758) 2023-11-08T16:41:06.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.480+0000 7f410d8fe640 1 -- 172.21.15.81:0/3528438751 learned_addr learned my addr 172.21.15.81:0/3528438751 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:06.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.480+0000 7f410d0fd640 1 --2- 172.21.15.81:0/3528438751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080ade50 0x7f41080ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:06.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.480+0000 7f410d8fe640 1 -- 172.21.15.81:0/3528438751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080ade50 msgr2=0x7f41080ca480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:06.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.480+0000 7f410d8fe640 1 --2- 172.21.15.81:0/3528438751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080ade50 0x7f41080ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.480+0000 7f410d8fe640 1 -- 172.21.15.81:0/3528438751 >> v1:172.21.15.93:6789/0 conn(0x7f410815a680 legacy=0x7f410815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:06.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.480+0000 7f410d8fe640 1 -- 172.21.15.81:0/3528438751 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080cabb0 con 0x7f410815ea60 2023-11-08T16:41:06.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.481+0000 7f410d8fe640 1 --2- 172.21.15.81:0/3528438751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f410815ea60 0x7f410815eef0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f4100009e50 tx=0x7f410002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c166793357be09b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:06.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.482+0000 7f40fffff640 1 -- 172.21.15.81:0/3528438751 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f410002ca70 con 0x7f410815ea60 2023-11-08T16:41:06.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.482+0000 7f40fffff640 1 -- 172.21.15.81:0/3528438751 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f410002cbd0 con 0x7f410815ea60 2023-11-08T16:41:06.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.482+0000 7f40fffff640 1 -- 172.21.15.81:0/3528438751 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41000343f0 con 0x7f410815ea60 2023-11-08T16:41:06.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.482+0000 7f410f361640 1 -- 172.21.15.81:0/3528438751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f410815ea60 msgr2=0x7f410815eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:06.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.482+0000 7f410f361640 1 --2- 172.21.15.81:0/3528438751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f410815ea60 0x7f410815eef0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f4100009e50 tx=0x7f410002bed0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.482+0000 7f410f361640 1 -- 172.21.15.81:0/3528438751 shutdown_connections 2023-11-08T16:41:06.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.482+0000 7f410f361640 1 --2- 172.21.15.81:0/3528438751 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f410815ea60 0x7f410815eef0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.482+0000 7f410f361640 1 --2- 172.21.15.81:0/3528438751 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080ade50 0x7f41080ca480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.482+0000 7f410f361640 1 -- 172.21.15.81:0/3528438751 >> 172.21.15.81:0/3528438751 conn(0x7f41080cb490 msgr2=0x7f41080cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:06.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.482+0000 7f410f361640 1 -- 172.21.15.81:0/3528438751 shutdown_connections 2023-11-08T16:41:06.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.483+0000 7f410f361640 1 -- 172.21.15.81:0/3528438751 wait complete. 2023-11-08T16:41:06.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.484+0000 7f410f361640 1 Processor -- start 2023-11-08T16:41:06.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.485+0000 7f410f361640 1 -- start start 2023-11-08T16:41:06.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.485+0000 7f410f361640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080ade50 0x7f4108149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:06.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410f361640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f410815a680 0x7f410815b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:06.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410f361640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f410815ea60 0x7f41080c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:06.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410f361640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41081f76a0 con 0x7f410815a680 2023-11-08T16:41:06.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410f361640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41081f73c0 con 0x7f410815ea60 2023-11-08T16:41:06.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410f361640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41080cd110 con 0x7f41080ade50 2023-11-08T16:41:06.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410c8fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f410815a680 0x7f410815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:06.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410c8fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f410815a680 0x7f410815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58766/0 (socket says 172.21.15.81:58766) 2023-11-08T16:41:06.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410c8fc640 1 -- 172.21.15.81:0/11936507 learned_addr learned my addr 172.21.15.81:0/11936507 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:06.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410c8fc640 1 -- 172.21.15.81:0/11936507 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080ade50 msgr2=0x7f4108149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:06.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410c8fc640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080ade50 0x7f4108149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410c8fc640 1 -- 172.21.15.81:0/11936507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f410815ea60 msgr2=0x7f41080c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:06.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410c8fc640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f410815ea60 0x7f41080c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.486+0000 7f410c8fc640 1 -- 172.21.15.81:0/11936507 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4100009b30 con 0x7f410815a680 2023-11-08T16:41:06.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.487+0000 7f410c8fc640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f410815a680 0x7f410815b280 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f40f800c430 tx=0x7f40f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:06.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.487+0000 7f40fdffb640 1 -- 172.21.15.81:0/11936507 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8011790 con 0x7f410815a680 2023-11-08T16:41:06.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.487+0000 7f40fdffb640 1 -- 172.21.15.81:0/11936507 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f80118f0 con 0x7f410815a680 2023-11-08T16:41:06.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.487+0000 7f40fdffb640 1 -- 172.21.15.81:0/11936507 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8011ce0 con 0x7f410815a680 2023-11-08T16:41:06.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.487+0000 7f410f361640 1 -- 172.21.15.81:0/11936507 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41080cd3f0 con 0x7f410815a680 2023-11-08T16:41:06.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.487+0000 7f410f361640 1 -- 172.21.15.81:0/11936507 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410814a270 con 0x7f410815a680 2023-11-08T16:41:06.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.488+0000 7f410f361640 1 -- 172.21.15.81:0/11936507 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41080cd910 con 0x7f410815a680 2023-11-08T16:41:06.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.489+0000 7f40fdffb640 1 -- 172.21.15.81:0/11936507 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f40f801c460 con 0x7f410815a680 2023-11-08T16:41:06.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.490+0000 7f40fdffb640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40ec06ae70 0x7f40ec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:06.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.490+0000 7f40fdffb640 1 -- 172.21.15.81:0/11936507 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40f80895e0 con 0x7f410815a680 2023-11-08T16:41:06.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.490+0000 7f410d0fd640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40ec06ae70 0x7f40ec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:06.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.491+0000 7f410d0fd640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40ec06ae70 0x7f40ec06d330 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4104006fd0 tx=0x7f4104008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:06.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.494+0000 7f40fdffb640 1 -- 172.21.15.81:0/11936507 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f41080cd910 con 0x7f410815a680 2023-11-08T16:41:06.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.685+0000 7f410f361640 1 -- 172.21.15.81:0/11936507 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41080cd910 con 0x7f410815a680 2023-11-08T16:41:06.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.687+0000 7f40fdffb640 1 -- 172.21.15.81:0/11936507 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16480 (secure 0 0 0) 0x7f41080cd910 con 0x7f410815a680 2023-11-08T16:41:06.689 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:06.689 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":126,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:05.699997+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:41:03.545565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T16:41:04.726492+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:41:06.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 -- 172.21.15.81:0/11936507 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40ec06ae70 msgr2=0x7f40ec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:06.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40ec06ae70 0x7f40ec06d330 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f4104006fd0 tx=0x7f4104008040 comp rx=0 tx=0).stop 2023-11-08T16:41:06.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 -- 172.21.15.81:0/11936507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f410815a680 msgr2=0x7f410815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:06.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f410815a680 0x7f410815b280 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f40f800c430 tx=0x7f40f800c8f0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 -- 172.21.15.81:0/11936507 shutdown_connections 2023-11-08T16:41:06.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f40ec06ae70 0x7f40ec06d330 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f410815ea60 0x7f41080c36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f410815a680 0x7f410815b280 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 --2- 172.21.15.81:0/11936507 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f41080ade50 0x7f4108149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 -- 172.21.15.81:0/11936507 >> 172.21.15.81:0/11936507 conn(0x7f41080cb490 msgr2=0x7f41080b2090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:06.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 -- 172.21.15.81:0/11936507 shutdown_connections 2023-11-08T16:41:06.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.690+0000 7f410f361640 1 -- 172.21.15.81:0/11936507 wait complete. 2023-11-08T16:41:06.710 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:41:06.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.926+0000 7f0ce7765640 1 Processor -- start 2023-11-08T16:41:06.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.927+0000 7f0ce7765640 1 -- start start 2023-11-08T16:41:06.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.928+0000 7f0ce7765640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0ce0160700 0x7f0ce00cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:06.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.928+0000 7f0ce7765640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce00cb3a0 con 0x7f0ce00cf050 2023-11-08T16:41:06.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.928+0000 7f0ce7765640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ce00cb680 con 0x7f0ce00cce90 2023-11-08T16:41:06.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.928+0000 7f0ce7765640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ce00cb960 con 0x7f0ce0160700 2023-11-08T16:41:06.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.928+0000 7f0ce5501640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0ce0160700 0x7f0ce00cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:06.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.928+0000 7f0ce5501640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f0ce0160700 0x7f0ce00cd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58776/0 (socket says 172.21.15.81:58776) 2023-11-08T16:41:06.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.928+0000 7f0ce4d00640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f0ce00cf050 0x7f0ce006b460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:33020/0 (socket says 172.21.15.81:33020) 2023-11-08T16:41:06.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.928+0000 7f0ce5501640 1 -- 172.21.15.81:0/2087058282 learned_addr learned my addr 172.21.15.81:0/2087058282 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:06.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.929+0000 7f0ce5501640 1 -- 172.21.15.81:0/2087058282 >> v1:172.21.15.93:6789/0 conn(0x7f0ce00cce90 legacy=0x7f0ce006bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:06.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.929+0000 7f0ce5501640 1 -- 172.21.15.81:0/2087058282 >> v1:172.21.15.81:6789/0 conn(0x7f0ce00cf050 legacy=0x7f0ce006b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:06.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.929+0000 7f0ce5501640 1 -- 172.21.15.81:0/2087058282 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce00cffc0 con 0x7f0ce0160700 2023-11-08T16:41:06.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.929+0000 7f0cd7fff640 1 -- 172.21.15.81:0/2087058282 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511214967 0 0) 0x7f0ce00cb3a0 con 0x7f0ce00cf050 2023-11-08T16:41:06.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.930+0000 7f0ce5501640 1 --2- 172.21.15.81:0/2087058282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ce0160700 0x7f0ce00cd530 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009e50 tx=0x7f0cdc02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09ddc57b56529bc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:06.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.930+0000 7f0cd7fff640 1 -- 172.21.15.81:0/2087058282 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc02cb10 con 0x7f0ce0160700 2023-11-08T16:41:06.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.930+0000 7f0cd7fff640 1 -- 172.21.15.81:0/2087058282 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc02cc70 con 0x7f0ce0160700 2023-11-08T16:41:06.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.930+0000 7f0cd7fff640 1 -- 172.21.15.81:0/2087058282 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc0343f0 con 0x7f0ce0160700 2023-11-08T16:41:06.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.930+0000 7f0ce7765640 1 -- 172.21.15.81:0/2087058282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ce0160700 msgr2=0x7f0ce00cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:06.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.930+0000 7f0ce7765640 1 --2- 172.21.15.81:0/2087058282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ce0160700 0x7f0ce00cd530 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009e50 tx=0x7f0cdc02bf70 comp rx=0 tx=0).stop 2023-11-08T16:41:06.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.931+0000 7f0ce7765640 1 -- 172.21.15.81:0/2087058282 shutdown_connections 2023-11-08T16:41:06.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.931+0000 7f0ce7765640 1 --2- 172.21.15.81:0/2087058282 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ce0160700 0x7f0ce00cd530 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.931+0000 7f0ce7765640 1 -- 172.21.15.81:0/2087058282 >> 172.21.15.81:0/2087058282 conn(0x7f0ce00d0cb0 msgr2=0x7f0ce00d10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:06.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.931+0000 7f0ce7765640 1 -- 172.21.15.81:0/2087058282 shutdown_connections 2023-11-08T16:41:06.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.931+0000 7f0ce7765640 1 -- 172.21.15.81:0/2087058282 wait complete. 2023-11-08T16:41:06.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.932+0000 7f0ce7765640 1 Processor -- start 2023-11-08T16:41:06.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.933+0000 7f0ce7765640 1 -- start start 2023-11-08T16:41:06.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.933+0000 7f0ce7765640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ce00cce90 0x7f0ce00c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:06.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.933+0000 7f0ce7765640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ce00cf050 0x7f0ce00cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:06.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.933+0000 7f0ce7765640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ce0160700 0x7f0ce015b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:06.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.933+0000 7f0ce7765640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00c6bc0 con 0x7f0ce00cf050 2023-11-08T16:41:06.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.933+0000 7f0ce7765640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00c5ff0 con 0x7f0ce00cce90 2023-11-08T16:41:06.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.933+0000 7f0ce7765640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce00c6290 con 0x7f0ce0160700 2023-11-08T16:41:06.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.934+0000 7f0ce4d00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ce00cf050 0x7f0ce00cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:06.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.934+0000 7f0ce4d00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ce00cf050 0x7f0ce00cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58778/0 (socket says 172.21.15.81:58778) 2023-11-08T16:41:06.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.934+0000 7f0ce4d00640 1 -- 172.21.15.81:0/2885373107 learned_addr learned my addr 172.21.15.81:0/2885373107 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:06.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.934+0000 7f0ce5501640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ce00cce90 0x7f0ce00c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:06.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.934+0000 7f0ce5d02640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ce0160700 0x7f0ce015b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:06.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.934+0000 7f0ce4d00640 1 -- 172.21.15.81:0/2885373107 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ce0160700 msgr2=0x7f0ce015b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:06.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.934+0000 7f0ce4d00640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ce0160700 0x7f0ce015b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.934+0000 7f0ce4d00640 1 -- 172.21.15.81:0/2885373107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ce00cce90 msgr2=0x7f0ce00c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:06.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.934+0000 7f0ce4d00640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ce00cce90 0x7f0ce00c7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:06.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.934+0000 7f0ce4d00640 1 -- 172.21.15.81:0/2885373107 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc009b30 con 0x7f0ce00cf050 2023-11-08T16:41:06.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.934+0000 7f0ce4d00640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ce00cf050 0x7f0ce00cc0f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000c4d0 tx=0x7f0cd000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:06.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.935+0000 7f0cd5ffb640 1 -- 172.21.15.81:0/2885373107 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd00074e0 con 0x7f0ce00cf050 2023-11-08T16:41:06.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.935+0000 7f0cd5ffb640 1 -- 172.21.15.81:0/2885373107 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cd0007660 con 0x7f0ce00cf050 2023-11-08T16:41:06.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.935+0000 7f0ce7765640 1 -- 172.21.15.81:0/2885373107 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce00c6570 con 0x7f0ce00cf050 2023-11-08T16:41:06.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.935+0000 7f0cd5ffb640 1 -- 172.21.15.81:0/2885373107 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0004a20 con 0x7f0ce00cf050 2023-11-08T16:41:06.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.935+0000 7f0ce7765640 1 -- 172.21.15.81:0/2885373107 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce00c7590 con 0x7f0ce00cf050 2023-11-08T16:41:06.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.936+0000 7f0ce7765640 1 -- 172.21.15.81:0/2885373107 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce004ece0 con 0x7f0ce00cf050 2023-11-08T16:41:06.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.937+0000 7f0cd5ffb640 1 -- 172.21.15.81:0/2885373107 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0cd0004b80 con 0x7f0ce00cf050 2023-11-08T16:41:06.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.938+0000 7f0cd5ffb640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0cc406aea0 0x7f0cc406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:06.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.938+0000 7f0cd5ffb640 1 -- 172.21.15.81:0/2885373107 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0cd0088ae0 con 0x7f0ce00cf050 2023-11-08T16:41:06.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.938+0000 7f0ce5501640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0cc406aea0 0x7f0cc406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:06.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.939+0000 7f0ce5501640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0cc406aea0 0x7f0cc406d360 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc035000 tx=0x7f0cdc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:06.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:06.943+0000 7f0cd5ffb640 1 -- 172.21.15.81:0/2885373107 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0cd0053b80 con 0x7f0ce00cf050 2023-11-08T16:41:07.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.122+0000 7f0ce7765640 1 -- 172.21.15.81:0/2885373107 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ce00d08d0 con 0x7f0ce00cf050 2023-11-08T16:41:07.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.125+0000 7f0cd5ffb640 1 -- 172.21.15.81:0/2885373107 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16480 (secure 0 0 0) 0x7f0cd0018020 con 0x7f0ce00cf050 2023-11-08T16:41:07.127 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:07.127 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":126,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:05.699997+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:41:03.545565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T16:41:04.726492+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:41:07.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.127+0000 7f0ce7765640 1 -- 172.21.15.81:0/2885373107 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0cc406aea0 msgr2=0x7f0cc406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:07.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.127+0000 7f0ce7765640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0cc406aea0 0x7f0cc406d360 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc035000 tx=0x7f0cdc030000 comp rx=0 tx=0).stop 2023-11-08T16:41:07.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.127+0000 7f0ce7765640 1 -- 172.21.15.81:0/2885373107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ce00cf050 msgr2=0x7f0ce00cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:07.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.127+0000 7f0ce7765640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ce00cf050 0x7f0ce00cc0f0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f0cd000c4d0 tx=0x7f0cd000c990 comp rx=0 tx=0).stop 2023-11-08T16:41:07.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.127+0000 7f0ce7765640 1 -- 172.21.15.81:0/2885373107 shutdown_connections 2023-11-08T16:41:07.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.127+0000 7f0ce7765640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0cc406aea0 0x7f0cc406d360 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:07.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.127+0000 7f0ce7765640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0ce0160700 0x7f0ce015b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:07.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.127+0000 7f0ce7765640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0ce00cf050 0x7f0ce00cc0f0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:07.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.127+0000 7f0ce7765640 1 --2- 172.21.15.81:0/2885373107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0ce00cce90 0x7f0ce00c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:07.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.127+0000 7f0ce7765640 1 -- 172.21.15.81:0/2885373107 >> 172.21.15.81:0/2885373107 conn(0x7f0ce00d0cb0 msgr2=0x7f0ce015bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:07.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.128+0000 7f0ce7765640 1 -- 172.21.15.81:0/2885373107 shutdown_connections 2023-11-08T16:41:07.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.128+0000 7f0ce7765640 1 -- 172.21.15.81:0/2885373107 wait complete. 2023-11-08T16:41:07.146 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-08T16:41:07.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.359+0000 7f7008a4a640 1 Processor -- start 2023-11-08T16:41:07.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.360+0000 7f7008a4a640 1 -- start start 2023-11-08T16:41:07.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.361+0000 7f7008a4a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f70040cfd90 0x7f70040d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:07.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.361+0000 7f7008a4a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70040ce0b0 con 0x7f700415cf30 2023-11-08T16:41:07.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.361+0000 7f7008a4a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70040ce390 con 0x7f700415d310 2023-11-08T16:41:07.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.361+0000 7f7008a4a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70040d1be0 con 0x7f70040cfd90 2023-11-08T16:41:07.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.361+0000 7f700359e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f70040cfd90 0x7f70040d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:07.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.362+0000 7f700359e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f70040cfd90 0x7f70040d26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58794/0 (socket says 172.21.15.81:58794) 2023-11-08T16:41:07.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.362+0000 7f700359e640 1 -- 172.21.15.81:0/1806508207 learned_addr learned my addr 172.21.15.81:0/1806508207 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:07.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.362+0000 7f7002d9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f700415cf30 0x7f70040cf680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:33024/0 (socket says 172.21.15.81:33024) 2023-11-08T16:41:07.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.362+0000 7f700359e640 1 -- 172.21.15.81:0/1806508207 >> v1:172.21.15.93:6789/0 conn(0x7f700415d310 legacy=0x7f70040b5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:07.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.362+0000 7f700359e640 1 -- 172.21.15.81:0/1806508207 >> v1:172.21.15.81:6789/0 conn(0x7f700415cf30 legacy=0x7f70040cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:07.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.362+0000 7f700359e640 1 -- 172.21.15.81:0/1806508207 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70040d1ec0 con 0x7f70040cfd90 2023-11-08T16:41:07.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.363+0000 7f700359e640 1 --2- 172.21.15.81:0/1806508207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70040cfd90 0x7f70040d26c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009e80 tx=0x7f6ffc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15b8caf2d7455b2d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:07.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.363+0000 7f700259c640 1 -- 172.21.15.81:0/1806508207 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc02cb00 con 0x7f70040cfd90 2023-11-08T16:41:07.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.364+0000 7f700259c640 1 -- 172.21.15.81:0/1806508207 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc02cc60 con 0x7f70040cfd90 2023-11-08T16:41:07.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.364+0000 7f700259c640 1 -- 172.21.15.81:0/1806508207 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc0343f0 con 0x7f70040cfd90 2023-11-08T16:41:07.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.364+0000 7f7008a4a640 1 -- 172.21.15.81:0/1806508207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70040cfd90 msgr2=0x7f70040d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:07.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.364+0000 7f7008a4a640 1 --2- 172.21.15.81:0/1806508207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70040cfd90 0x7f70040d26c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009e80 tx=0x7f6ffc02bf60 comp rx=0 tx=0).stop 2023-11-08T16:41:07.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.364+0000 7f7008a4a640 1 -- 172.21.15.81:0/1806508207 shutdown_connections 2023-11-08T16:41:07.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.364+0000 7f7008a4a640 1 --2- 172.21.15.81:0/1806508207 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70040cfd90 0x7f70040d26c0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:07.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.364+0000 7f7008a4a640 1 -- 172.21.15.81:0/1806508207 >> 172.21.15.81:0/1806508207 conn(0x7f700406c040 msgr2=0x7f700406c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:07.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.364+0000 7f7008a4a640 1 -- 172.21.15.81:0/1806508207 shutdown_connections 2023-11-08T16:41:07.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.364+0000 7f7008a4a640 1 -- 172.21.15.81:0/1806508207 wait complete. 2023-11-08T16:41:07.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.366+0000 7f7008a4a640 1 Processor -- start 2023-11-08T16:41:07.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.366+0000 7f7008a4a640 1 -- start start 2023-11-08T16:41:07.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.367+0000 7f7008a4a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70040cfd90 0x7f7004153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:07.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.367+0000 7f7008a4a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f700415cf30 0x7f70040ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:07.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.367+0000 7f7008a4a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70040cacd0 0x7f700415f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:07.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.367+0000 7f7008a4a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7004157180 con 0x7f70040cacd0 2023-11-08T16:41:07.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.367+0000 7f7008a4a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7004153790 con 0x7f70040cfd90 2023-11-08T16:41:07.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.367+0000 7f7008a4a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70040cb1e0 con 0x7f700415cf30 2023-11-08T16:41:07.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.368+0000 7f700359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70040cacd0 0x7f700415f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:07.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.368+0000 7f700359e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70040cacd0 0x7f700415f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58802/0 (socket says 172.21.15.81:58802) 2023-11-08T16:41:07.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.368+0000 7f700359e640 1 -- 172.21.15.81:0/616319512 learned_addr learned my addr 172.21.15.81:0/616319512 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:07.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.368+0000 7f7002d9d640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70040cfd90 0x7f7004153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:07.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.368+0000 7f6ffa59c640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f700415cf30 0x7f70040ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:07.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.368+0000 7f700359e640 1 -- 172.21.15.81:0/616319512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f700415cf30 msgr2=0x7f70040ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:07.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.368+0000 7f700359e640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f700415cf30 0x7f70040ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:07.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.368+0000 7f700359e640 1 -- 172.21.15.81:0/616319512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70040cfd90 msgr2=0x7f7004153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:07.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.368+0000 7f700359e640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70040cfd90 0x7f7004153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:07.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.368+0000 7f700359e640 1 -- 172.21.15.81:0/616319512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc009b30 con 0x7f70040cacd0 2023-11-08T16:41:07.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.369+0000 7f700359e640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70040cacd0 0x7f700415f350 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009fa0 tx=0x7f6ffc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:07.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.369+0000 7f6ffbfff640 1 -- 172.21.15.81:0/616319512 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc009510 con 0x7f70040cacd0 2023-11-08T16:41:07.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.369+0000 7f6ffbfff640 1 -- 172.21.15.81:0/616319512 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc03e450 con 0x7f70040cacd0 2023-11-08T16:41:07.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.369+0000 7f6ffbfff640 1 -- 172.21.15.81:0/616319512 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc03e7e0 con 0x7f70040cacd0 2023-11-08T16:41:07.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.369+0000 7f7008a4a640 1 -- 172.21.15.81:0/616319512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70040cb4c0 con 0x7f70040cacd0 2023-11-08T16:41:07.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.369+0000 7f7008a4a640 1 -- 172.21.15.81:0/616319512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70040cba80 con 0x7f70040cacd0 2023-11-08T16:41:07.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.370+0000 7f7008a4a640 1 -- 172.21.15.81:0/616319512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70041572f0 con 0x7f70040cacd0 2023-11-08T16:41:07.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.371+0000 7f6ffbfff640 1 -- 172.21.15.81:0/616319512 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ffc03e940 con 0x7f70040cacd0 2023-11-08T16:41:07.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.372+0000 7f6ffbfff640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6fe406ae70 0x7f6fe406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:07.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.372+0000 7f6ffbfff640 1 -- 172.21.15.81:0/616319512 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ffc0ac4c0 con 0x7f70040cacd0 2023-11-08T16:41:07.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.372+0000 7f7002d9d640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6fe406ae70 0x7f6fe406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:07.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.373+0000 7f7002d9d640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6fe406ae70 0x7f6fe406d330 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4000f60 tx=0x7f6ff4007c80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:07.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.376+0000 7f6ffbfff640 1 -- 172.21.15.81:0/616319512 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f70041572f0 con 0x7f70040cacd0 2023-11-08T16:41:07.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:07.589+0000 7f7008a4a640 1 -- 172.21.15.81:0/616319512 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f70041572f0 con 0x7f70040cacd0 2023-11-08T16:41:08.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.282+0000 7f6ffbfff640 1 -- 172.21.15.81:0/616319512 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v106) v1 ==== 118+0+0 (secure 0 0 0) 0x7f70041572f0 con 0x7f70040cacd0 2023-11-08T16:41:08.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.286+0000 7f7008a4a640 1 -- 172.21.15.81:0/616319512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6fe406ae70 msgr2=0x7f6fe406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:08.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.286+0000 7f7008a4a640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6fe406ae70 0x7f6fe406d330 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f6ff4000f60 tx=0x7f6ff4007c80 comp rx=0 tx=0).stop 2023-11-08T16:41:08.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.286+0000 7f7008a4a640 1 -- 172.21.15.81:0/616319512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70040cacd0 msgr2=0x7f700415f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:08.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.286+0000 7f7008a4a640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70040cacd0 0x7f700415f350 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc009fa0 tx=0x7f6ffc004a40 comp rx=0 tx=0).stop 2023-11-08T16:41:08.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.287+0000 7f7008a4a640 1 -- 172.21.15.81:0/616319512 shutdown_connections 2023-11-08T16:41:08.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.287+0000 7f7008a4a640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6fe406ae70 0x7f6fe406d330 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:08.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.287+0000 7f7008a4a640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f70040cacd0 0x7f700415f350 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:08.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.287+0000 7f7008a4a640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f700415cf30 0x7f70040ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:08.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.287+0000 7f7008a4a640 1 --2- 172.21.15.81:0/616319512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70040cfd90 0x7f7004153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:08.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.287+0000 7f7008a4a640 1 -- 172.21.15.81:0/616319512 >> 172.21.15.81:0/616319512 conn(0x7f700406c040 msgr2=0x7f70040b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:08.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.287+0000 7f7008a4a640 1 -- 172.21.15.81:0/616319512 shutdown_connections 2023-11-08T16:41:08.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.287+0000 7f7008a4a640 1 -- 172.21.15.81:0/616319512 wait complete. 2023-11-08T16:41:08.313 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-08T16:41:08.313 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-08T16:41:08.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.528+0000 7fd46151a640 1 Processor -- start 2023-11-08T16:41:08.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.528+0000 7fd46151a640 1 -- start start 2023-11-08T16:41:08.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd46151a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd45c15b250 0x7fd45c15a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:08.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd46151a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd45c0b11f0 0x7fd45c157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:08.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd46151a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd45c157a80 con 0x7fd45c1576a0 2023-11-08T16:41:08.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd46151a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd45c15c360 con 0x7fd45c0b11f0 2023-11-08T16:41:08.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd46151a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c15b6e0 con 0x7fd45c15b250 2023-11-08T16:41:08.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd460d19640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd45c0b11f0 0x7fd45c157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:08.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd460d19640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd45c0b11f0 0x7fd45c157cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58814/0 (socket says 172.21.15.81:58814) 2023-11-08T16:41:08.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd460d19640 1 -- 172.21.15.81:0/697773947 learned_addr learned my addr 172.21.15.81:0/697773947 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:08.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd453fff640 1 --2- 172.21.15.81:0/697773947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd45c15b250 0x7fd45c15a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:08.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd460d19640 1 -- 172.21.15.81:0/697773947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd45c15b250 msgr2=0x7fd45c15a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:08.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd460d19640 1 --2- 172.21.15.81:0/697773947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd45c15b250 0x7fd45c15a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:08.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd460d19640 1 -- 172.21.15.81:0/697773947 >> v1:172.21.15.93:6789/0 conn(0x7fd45c1576a0 legacy=0x7fd45c070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:08.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.530+0000 7fd460d19640 1 -- 172.21.15.81:0/697773947 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c15b9c0 con 0x7fd45c0b11f0 2023-11-08T16:41:08.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.531+0000 7fd453fff640 1 --2- 172.21.15.81:0/697773947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd45c15b250 0x7fd45c15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:08.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.532+0000 7fd460d19640 1 --2- 172.21.15.81:0/697773947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd45c0b11f0 0x7fd45c157cd0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd454009e50 tx=0x7fd45402e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf9dbb141a56632b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:08.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.532+0000 7fd45b7fe640 1 -- 172.21.15.81:0/697773947 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45402ebf0 con 0x7fd45c0b11f0 2023-11-08T16:41:08.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.532+0000 7fd45b7fe640 1 -- 172.21.15.81:0/697773947 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45402ed50 con 0x7fd45c0b11f0 2023-11-08T16:41:08.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.532+0000 7fd45b7fe640 1 -- 172.21.15.81:0/697773947 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454036610 con 0x7fd45c0b11f0 2023-11-08T16:41:08.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.532+0000 7fd46151a640 1 -- 172.21.15.81:0/697773947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd45c0b11f0 msgr2=0x7fd45c157cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:08.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.532+0000 7fd46151a640 1 --2- 172.21.15.81:0/697773947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd45c0b11f0 0x7fd45c157cd0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fd454009e50 tx=0x7fd45402e050 comp rx=0 tx=0).stop 2023-11-08T16:41:08.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.533+0000 7fd46151a640 1 -- 172.21.15.81:0/697773947 shutdown_connections 2023-11-08T16:41:08.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.533+0000 7fd46151a640 1 --2- 172.21.15.81:0/697773947 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd45c0b11f0 0x7fd45c157cd0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:08.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.533+0000 7fd46151a640 1 --2- 172.21.15.81:0/697773947 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd45c15b250 0x7fd45c15a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:08.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.533+0000 7fd46151a640 1 -- 172.21.15.81:0/697773947 >> 172.21.15.81:0/697773947 conn(0x7fd45c0ce8d0 msgr2=0x7fd45c0cece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:08.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.533+0000 7fd46151a640 1 -- 172.21.15.81:0/697773947 shutdown_connections 2023-11-08T16:41:08.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.533+0000 7fd46151a640 1 -- 172.21.15.81:0/697773947 wait complete. 2023-11-08T16:41:08.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.534+0000 7fd46151a640 1 Processor -- start 2023-11-08T16:41:08.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.534+0000 7fd46151a640 1 -- start start 2023-11-08T16:41:08.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.535+0000 7fd46151a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd45c0b11f0 0x7fd45c0cabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:08.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.535+0000 7fd46151a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd45c1576a0 0x7fd45c14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:08.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.535+0000 7fd46151a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd45c0c8bf0 0x7fd45c06bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:08.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.535+0000 7fd46151a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c14a9f0 con 0x7fd45c1576a0 2023-11-08T16:41:08.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.535+0000 7fd46151a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c0c9f70 con 0x7fd45c0c8bf0 2023-11-08T16:41:08.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.535+0000 7fd46151a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45c0c90a0 con 0x7fd45c0b11f0 2023-11-08T16:41:08.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.536+0000 7fd453fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd45c1576a0 0x7fd45c14a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:08.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.536+0000 7fd453fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd45c1576a0 0x7fd45c14a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58822/0 (socket says 172.21.15.81:58822) 2023-11-08T16:41:08.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.536+0000 7fd453fff640 1 -- 172.21.15.81:0/3287849861 learned_addr learned my addr 172.21.15.81:0/3287849861 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:08.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.536+0000 7fd453fff640 1 -- 172.21.15.81:0/3287849861 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd45c0b11f0 msgr2=0x7fd45c0cabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:08.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.536+0000 7fd453fff640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd45c0b11f0 0x7fd45c0cabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:08.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.536+0000 7fd453fff640 1 -- 172.21.15.81:0/3287849861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd45c0c8bf0 msgr2=0x7fd45c06bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:08.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.536+0000 7fd453fff640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd45c0c8bf0 0x7fd45c06bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:08.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.536+0000 7fd453fff640 1 -- 172.21.15.81:0/3287849861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454009b30 con 0x7fd45c1576a0 2023-11-08T16:41:08.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.536+0000 7fd453fff640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd45c1576a0 0x7fd45c14a4b0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd44400c7f0 tx=0x7fd44400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:08.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.536+0000 7fd4597fa640 1 -- 172.21.15.81:0/3287849861 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444004220 con 0x7fd45c1576a0 2023-11-08T16:41:08.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.537+0000 7fd4597fa640 1 -- 172.21.15.81:0/3287849861 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd444004380 con 0x7fd45c1576a0 2023-11-08T16:41:08.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.537+0000 7fd4597fa640 1 -- 172.21.15.81:0/3287849861 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd444005bb0 con 0x7fd45c1576a0 2023-11-08T16:41:08.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.537+0000 7fd46151a640 1 -- 172.21.15.81:0/3287849861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45c0c9360 con 0x7fd45c1576a0 2023-11-08T16:41:08.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.537+0000 7fd46151a640 1 -- 172.21.15.81:0/3287849861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd45c0c9920 con 0x7fd45c1576a0 2023-11-08T16:41:08.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.538+0000 7fd46151a640 1 -- 172.21.15.81:0/3287849861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45c0c7080 con 0x7fd45c1576a0 2023-11-08T16:41:08.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.539+0000 7fd4597fa640 1 -- 172.21.15.81:0/3287849861 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd444006030 con 0x7fd45c1576a0 2023-11-08T16:41:08.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.539+0000 7fd4597fa640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd43c06aea0 0x7fd43c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:08.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.540+0000 7fd4597fa640 1 -- 172.21.15.81:0/3287849861 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd444089ba0 con 0x7fd45c1576a0 2023-11-08T16:41:08.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.540+0000 7fd45bfff640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd43c06aea0 0x7fd43c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:08.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.540+0000 7fd45bfff640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd43c06aea0 0x7fd43c06d360 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0ca400 tx=0x7fd44c0079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:08.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.544+0000 7fd4597fa640 1 -- 172.21.15.81:0/3287849861 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd444054bb0 con 0x7fd45c1576a0 2023-11-08T16:41:08.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:08.772+0000 7fd46151a640 1 -- 172.21.15.81:0/3287849861 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd45c04ece0 con 0x7fd45c1576a0 2023-11-08T16:41:09.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.303+0000 7fd4597fa640 1 -- 172.21.15.81:0/3287849861 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v107) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd4440581d0 con 0x7fd45c1576a0 2023-11-08T16:41:09.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 -- 172.21.15.81:0/3287849861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd43c06aea0 msgr2=0x7fd43c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:09.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd43c06aea0 0x7fd43c06d360 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fd45c0ca400 tx=0x7fd44c0079b0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 -- 172.21.15.81:0/3287849861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd45c1576a0 msgr2=0x7fd45c14a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:09.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd45c1576a0 0x7fd45c14a4b0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fd44400c7f0 tx=0x7fd44400ccb0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 -- 172.21.15.81:0/3287849861 shutdown_connections 2023-11-08T16:41:09.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd43c06aea0 0x7fd43c06d360 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd45c0c8bf0 0x7fd45c06bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd45c1576a0 0x7fd45c14a4b0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 --2- 172.21.15.81:0/3287849861 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd45c0b11f0 0x7fd45c0cabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 -- 172.21.15.81:0/3287849861 >> 172.21.15.81:0/3287849861 conn(0x7fd45c0ce8d0 msgr2=0x7fd45c1525b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:09.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 -- 172.21.15.81:0/3287849861 shutdown_connections 2023-11-08T16:41:09.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.307+0000 7fd46151a640 1 -- 172.21.15.81:0/3287849861 wait complete. 2023-11-08T16:41:09.329 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-08T16:41:09.329 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:41:09.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.529+0000 7f5f108b3640 1 Processor -- start 2023-11-08T16:41:09.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.530+0000 7f5f108b3640 1 -- start start 2023-11-08T16:41:09.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.531+0000 7f5f108b3640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5f0c0cd870 0x7f5f0c0cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:09.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.531+0000 7f5f108b3640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c06c2f0 0x7f5f0c160a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:09.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.531+0000 7f5f108b3640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f0c0d13e0 con 0x7f5f0c0cdc50 2023-11-08T16:41:09.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.531+0000 7f5f108b3640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f0c0d1870 con 0x7f5f0c0cd870 2023-11-08T16:41:09.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.531+0000 7f5f108b3640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0d07d0 con 0x7f5f0c06c2f0 2023-11-08T16:41:09.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.531+0000 7f5f0b59e640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5f0c0cdc50 0x7f5f0c15c6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:33026/0 (socket says 172.21.15.81:33026) 2023-11-08T16:41:09.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.531+0000 7f5f0b59e640 1 -- 172.21.15.81:0/1596621665 learned_addr learned my addr 172.21.15.81:0/1596621665 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:09.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f03fff640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c06c2f0 0x7f5f0c160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:09.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0ad9d640 1 --2- 172.21.15.81:0/1596621665 >> v2:172.21.15.93:3300/0 conn(0x7f5f0c0cd870 0x7f5f0c0cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:09.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177910318 0 0) 0x7f5f0c0d13e0 con 0x7f5f0c0cdc50 2023-11-08T16:41:09.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ef8003520 con 0x7f5f0c0cdc50 2023-11-08T16:41:09.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4084181037 0 0) 0x7f5ef8003520 con 0x7f5f0c0cdc50 2023-11-08T16:41:09.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c06c2f0 msgr2=0x7f5f0c160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:09.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c06c2f0 0x7f5f0c160a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 >> v2:172.21.15.93:3300/0 conn(0x7f5f0c0cd870 msgr2=0x7f5f0c0cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:09.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 --2- 172.21.15.81:0/1596621665 >> v2:172.21.15.93:3300/0 conn(0x7f5f0c0cd870 0x7f5f0c0cfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c0d0ab0 con 0x7f5f0c0cdc50 2023-11-08T16:41:09.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f5f040019f0 con 0x7f5f0c0cdc50 2023-11-08T16:41:09.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 >> v1:172.21.15.81:6789/0 conn(0x7f5f0c0cdc50 legacy=0x7f5f0c15c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:09.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef8003a50 0x7f5ef8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:09.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.532+0000 7f5f0a59c640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ef80047c0 0x7f5ef8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:09.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f0a59c640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c06c2f0 0x7f5ef80051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:09.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0d1870 con 0x7f5ef80047c0 2023-11-08T16:41:09.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c0d07d0 con 0x7f5ef8003a50 2023-11-08T16:41:09.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef800bb60 con 0x7f5f0c06c2f0 2023-11-08T16:41:09.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f03fff640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ef80047c0 0x7f5ef8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:09.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f0b59e640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c06c2f0 0x7f5ef80051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:09.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f0ad9d640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef8003a50 0x7f5ef8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:09.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f03fff640 1 -- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c06c2f0 msgr2=0x7f5ef80051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:09.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f03fff640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c06c2f0 0x7f5ef80051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f03fff640 1 -- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef8003a50 msgr2=0x7f5ef8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:09.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f03fff640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef8003a50 0x7f5ef8003e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f03fff640 1 -- 172.21.15.81:0/1596621665 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef8003520 con 0x7f5ef80047c0 2023-11-08T16:41:09.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.533+0000 7f5f0b59e640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c06c2f0 0x7f5ef80051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:09.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.534+0000 7f5f03fff640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ef80047c0 0x7f5ef8004c40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400ba30 tx=0x7f5ef80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7e2ad77a64d5680 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:09.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.535+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4032620 con 0x7f5ef80047c0 2023-11-08T16:41:09.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.535+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef4032780 con 0x7f5ef80047c0 2023-11-08T16:41:09.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.535+0000 7f5f0a59c640 1 -- 172.21.15.81:0/1596621665 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef4032b10 con 0x7f5ef80047c0 2023-11-08T16:41:09.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.535+0000 7f5f108b3640 1 -- 172.21.15.81:0/1596621665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ef80047c0 msgr2=0x7f5ef8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:09.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.535+0000 7f5f108b3640 1 --2- 172.21.15.81:0/1596621665 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ef80047c0 0x7f5ef8004c40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f5ef400ba30 tx=0x7f5ef80057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.535+0000 7f5f0b59e640 1 -- 172.21.15.81:0/1596621665 reap_dead start 2023-11-08T16:41:09.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.535+0000 7f5f108b3640 1 -- 172.21.15.81:0/1596621665 shutdown_connections 2023-11-08T16:41:09.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.535+0000 7f5f108b3640 1 -- 172.21.15.81:0/1596621665 >> 172.21.15.81:0/1596621665 conn(0x7f5f0c06d940 msgr2=0x7f5f0c06dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:09.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.535+0000 7f5f108b3640 1 -- 172.21.15.81:0/1596621665 shutdown_connections 2023-11-08T16:41:09.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.535+0000 7f5f108b3640 1 -- 172.21.15.81:0/1596621665 wait complete. 2023-11-08T16:41:09.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.537+0000 7f5f108b3640 1 Processor -- start 2023-11-08T16:41:09.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.537+0000 7f5f108b3640 1 -- start start 2023-11-08T16:41:09.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.538+0000 7f5f108b3640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef80047c0 0x7f5f0c155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:09.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.538+0000 7f5f108b3640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c1597d0 0x7f5f0c1575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:09.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.538+0000 7f5f108b3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c159c50 0x7f5f0c157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:09.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.538+0000 7f5f108b3640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c15a220 con 0x7f5f0c159c50 2023-11-08T16:41:09.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.538+0000 7f5f108b3640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c15a500 con 0x7f5ef80047c0 2023-11-08T16:41:09.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.538+0000 7f5f108b3640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0c15a7e0 con 0x7f5f0c1597d0 2023-11-08T16:41:09.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.538+0000 7f5f0b59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c159c50 0x7f5f0c157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:09.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.538+0000 7f5f0b59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c159c50 0x7f5f0c157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58842/0 (socket says 172.21.15.81:58842) 2023-11-08T16:41:09.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.538+0000 7f5f0b59e640 1 -- 172.21.15.81:0/3130740161 learned_addr learned my addr 172.21.15.81:0/3130740161 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:09.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.539+0000 7f5f0ad9d640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef80047c0 0x7f5f0c155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:09.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.539+0000 7f5f03fff640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c1597d0 0x7f5f0c1575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:09.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.539+0000 7f5f0b59e640 1 -- 172.21.15.81:0/3130740161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c1597d0 msgr2=0x7f5f0c1575e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:09.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.539+0000 7f5f0b59e640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c1597d0 0x7f5f0c1575e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.539+0000 7f5f0b59e640 1 -- 172.21.15.81:0/3130740161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef80047c0 msgr2=0x7f5f0c155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:09.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.539+0000 7f5f0b59e640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef80047c0 0x7f5f0c155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.539+0000 7f5f0b59e640 1 -- 172.21.15.81:0/3130740161 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f0c1f2c60 con 0x7f5f0c159c50 2023-11-08T16:41:09.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.539+0000 7f5f0b59e640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c159c50 0x7f5f0c157b20 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f5f04001370 tx=0x7f5f0400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:09.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.539+0000 7f5f037fe640 1 -- 172.21.15.81:0/3130740161 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04004d60 con 0x7f5f0c159c50 2023-11-08T16:41:09.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.540+0000 7f5f037fe640 1 -- 172.21.15.81:0/3130740161 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f0400bb80 con 0x7f5f0c159c50 2023-11-08T16:41:09.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.540+0000 7f5f037fe640 1 -- 172.21.15.81:0/3130740161 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f04007070 con 0x7f5f0c159c50 2023-11-08T16:41:09.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.540+0000 7f5f108b3640 1 -- 172.21.15.81:0/3130740161 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0c15aac0 con 0x7f5f0c159c50 2023-11-08T16:41:09.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.540+0000 7f5f108b3640 1 -- 172.21.15.81:0/3130740161 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0c155840 con 0x7f5f0c159c50 2023-11-08T16:41:09.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.541+0000 7f5f108b3640 1 -- 172.21.15.81:0/3130740161 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0c067800 con 0x7f5f0c159c50 2023-11-08T16:41:09.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.542+0000 7f5f037fe640 1 -- 172.21.15.81:0/3130740161 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f040071d0 con 0x7f5f0c159c50 2023-11-08T16:41:09.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.543+0000 7f5f037fe640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5eec06ae70 0x7f5eec06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:09.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.543+0000 7f5f037fe640 1 -- 172.21.15.81:0/3130740161 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f04089a10 con 0x7f5f0c159c50 2023-11-08T16:41:09.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.543+0000 7f5f0ad9d640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5eec06ae70 0x7f5eec06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:09.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.544+0000 7f5f0ad9d640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5eec06ae70 0x7f5eec06d330 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5efc006fd0 tx=0x7f5efc008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:09.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.548+0000 7f5f037fe640 1 -- 172.21.15.81:0/3130740161 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5f04054a20 con 0x7f5f0c159c50 2023-11-08T16:41:09.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.774+0000 7f5f108b3640 1 -- 172.21.15.81:0/3130740161 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f0c158b80 con 0x7f5f0c159c50 2023-11-08T16:41:09.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.775+0000 7f5f037fe640 1 -- 172.21.15.81:0/3130740161 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4918 (secure 0 0 0) 0x7f5f04058040 con 0x7f5f0c159c50 2023-11-08T16:41:09.779 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:09.779 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/296230272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":296230272},{"type":"v1","addr":"172.21.15.81:6837","nonce":296230272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:08.773407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8574":{"gid":8574,"name":"d","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6835/4021510722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":4021510722},{"type":"v1","addr":"172.21.15.93:6835","nonce":4021510722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:41:09.780 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 107 2023-11-08T16:41:09.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.777+0000 7f5f108b3640 1 -- 172.21.15.81:0/3130740161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5eec06ae70 msgr2=0x7f5eec06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:09.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.778+0000 7f5f108b3640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5eec06ae70 0x7f5eec06d330 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f5efc006fd0 tx=0x7f5efc008040 comp rx=0 tx=0).stop 2023-11-08T16:41:09.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.778+0000 7f5f108b3640 1 -- 172.21.15.81:0/3130740161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c159c50 msgr2=0x7f5f0c157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:09.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.778+0000 7f5f108b3640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c159c50 0x7f5f0c157b20 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f5f04001370 tx=0x7f5f0400b4a0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.778+0000 7f5f108b3640 1 -- 172.21.15.81:0/3130740161 shutdown_connections 2023-11-08T16:41:09.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.778+0000 7f5f108b3640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5eec06ae70 0x7f5eec06d330 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.778+0000 7f5f108b3640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5f0c159c50 0x7f5f0c157b20 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.778+0000 7f5f108b3640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5f0c1597d0 0x7f5f0c1575e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.778+0000 7f5f108b3640 1 --2- 172.21.15.81:0/3130740161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef80047c0 0x7f5f0c155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:09.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.778+0000 7f5f108b3640 1 -- 172.21.15.81:0/3130740161 >> 172.21.15.81:0/3130740161 conn(0x7f5f0c06d940 msgr2=0x7f5f0c06dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:09.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.778+0000 7f5f108b3640 1 -- 172.21.15.81:0/3130740161 shutdown_connections 2023-11-08T16:41:09.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.778+0000 7f5f108b3640 1 -- 172.21.15.81:0/3130740161 wait complete. 2023-11-08T16:41:09.799 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:41:09.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.997+0000 7f8e9f244640 1 Processor -- start 2023-11-08T16:41:09.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.997+0000 7f8e9f244640 1 -- start start 2023-11-08T16:41:10.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.999+0000 7f8e9f244640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8e980ade50 0x7f8e980ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.999+0000 7f8e9f244640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8e9815a680 0x7f8e9815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.999+0000 7f8e9f244640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f8e9815b020 0x7f8e980c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.999+0000 7f8e9f244640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e980b1860 con 0x7f8e980ade50 2023-11-08T16:41:10.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.999+0000 7f8e9f244640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e980ca9c0 con 0x7f8e9815b020 2023-11-08T16:41:10.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.999+0000 7f8e9f244640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9806b630 con 0x7f8e9815a680 2023-11-08T16:41:10.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.999+0000 7f8e9cfe0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8e980ade50 0x7f8e980ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.999+0000 7f8e9cfe0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f8e980ade50 0x7f8e980ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58846/0 (socket says 172.21.15.81:58846) 2023-11-08T16:41:10.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:09.999+0000 7f8e9cfe0640 1 -- 172.21.15.81:0/1826720845 learned_addr learned my addr 172.21.15.81:0/1826720845 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:10.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.000+0000 7f8e9d7e1640 1 --2- 172.21.15.81:0/1826720845 >> v2:172.21.15.93:3300/0 conn(0x7f8e9815b020 0x7f8e980c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.000+0000 7f8e97fff640 1 --2- 172.21.15.81:0/1826720845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8e9815a680 0x7f8e9815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.000+0000 7f8e9cfe0640 1 -- 172.21.15.81:0/1826720845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8e9815a680 msgr2=0x7f8e9815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:10.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.000+0000 7f8e9cfe0640 1 --2- 172.21.15.81:0/1826720845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8e9815a680 0x7f8e9815aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.000+0000 7f8e9cfe0640 1 -- 172.21.15.81:0/1826720845 >> v2:172.21.15.93:3300/0 conn(0x7f8e9815b020 msgr2=0x7f8e980c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:10.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.000+0000 7f8e9cfe0640 1 --2- 172.21.15.81:0/1826720845 >> v2:172.21.15.93:3300/0 conn(0x7f8e9815b020 0x7f8e980c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.000+0000 7f8e9cfe0640 1 -- 172.21.15.81:0/1826720845 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e980cabb0 con 0x7f8e980ade50 2023-11-08T16:41:10.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.001+0000 7f8e9cfe0640 1 --2- 172.21.15.81:0/1826720845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e980ade50 0x7f8e980ca480 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f8e9806f840 tx=0x7f8e8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c26305b45c8e19e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:10.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.001+0000 7f8e977fe640 1 -- 172.21.15.81:0/1826720845 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c031870 con 0x7f8e980ade50 2023-11-08T16:41:10.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.001+0000 7f8e977fe640 1 -- 172.21.15.81:0/1826720845 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8c0319d0 con 0x7f8e980ade50 2023-11-08T16:41:10.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.001+0000 7f8e977fe640 1 -- 172.21.15.81:0/1826720845 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c031d60 con 0x7f8e980ade50 2023-11-08T16:41:10.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.002+0000 7f8e9f244640 1 -- 172.21.15.81:0/1826720845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e980ade50 msgr2=0x7f8e980ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.002+0000 7f8e9f244640 1 --2- 172.21.15.81:0/1826720845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e980ade50 0x7f8e980ca480 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f8e9806f840 tx=0x7f8e8c02b910 comp rx=0 tx=0).stop 2023-11-08T16:41:10.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.002+0000 7f8e9f244640 1 -- 172.21.15.81:0/1826720845 shutdown_connections 2023-11-08T16:41:10.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.002+0000 7f8e9f244640 1 --2- 172.21.15.81:0/1826720845 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8e9815a680 0x7f8e9815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.002+0000 7f8e9f244640 1 --2- 172.21.15.81:0/1826720845 >> v2:172.21.15.93:3300/0 conn(0x7f8e9815b020 0x7f8e980c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.002+0000 7f8e9f244640 1 --2- 172.21.15.81:0/1826720845 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e980ade50 0x7f8e980ca480 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.002+0000 7f8e9f244640 1 -- 172.21.15.81:0/1826720845 >> 172.21.15.81:0/1826720845 conn(0x7f8e980cb490 msgr2=0x7f8e980cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:10.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.002+0000 7f8e9f244640 1 -- 172.21.15.81:0/1826720845 shutdown_connections 2023-11-08T16:41:10.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.002+0000 7f8e9f244640 1 -- 172.21.15.81:0/1826720845 wait complete. 2023-11-08T16:41:10.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.004+0000 7f8e9f244640 1 Processor -- start 2023-11-08T16:41:10.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.004+0000 7f8e9f244640 1 -- start start 2023-11-08T16:41:10.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9f244640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e980ade50 0x7f8e9815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9f244640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8e9815a680 0x7f8e98157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9f244640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8e9815b020 0x7f8e980c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9f244640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9815cc50 con 0x7f8e980ade50 2023-11-08T16:41:10.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9f244640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9815cf30 con 0x7f8e9815a680 2023-11-08T16:41:10.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9f244640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9815d210 con 0x7f8e9815b020 2023-11-08T16:41:10.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9cfe0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e980ade50 0x7f8e9815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9cfe0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e980ade50 0x7f8e9815dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58860/0 (socket says 172.21.15.81:58860) 2023-11-08T16:41:10.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9cfe0640 1 -- 172.21.15.81:0/767448590 learned_addr learned my addr 172.21.15.81:0/767448590 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:10.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9cfe0640 1 -- 172.21.15.81:0/767448590 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8e9815b020 msgr2=0x7f8e980c4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9cfe0640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8e9815b020 0x7f8e980c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9cfe0640 1 -- 172.21.15.81:0/767448590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8e9815a680 msgr2=0x7f8e98157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9cfe0640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8e9815a680 0x7f8e98157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.005+0000 7f8e9cfe0640 1 -- 172.21.15.81:0/767448590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8c0094d0 con 0x7f8e980ade50 2023-11-08T16:41:10.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.006+0000 7f8e9cfe0640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e980ade50 0x7f8e9815dcf0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c000c00 tx=0x7f8e8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:10.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.006+0000 7f8e957fa640 1 -- 172.21.15.81:0/767448590 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c03abe0 con 0x7f8e980ade50 2023-11-08T16:41:10.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.006+0000 7f8e957fa640 1 -- 172.21.15.81:0/767448590 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e8c03ad40 con 0x7f8e980ade50 2023-11-08T16:41:10.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.006+0000 7f8e957fa640 1 -- 172.21.15.81:0/767448590 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e8c044630 con 0x7f8e980ade50 2023-11-08T16:41:10.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.006+0000 7f8e9f244640 1 -- 172.21.15.81:0/767448590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e9815d4f0 con 0x7f8e980ade50 2023-11-08T16:41:10.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.006+0000 7f8e9f244640 1 -- 172.21.15.81:0/767448590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e9815e230 con 0x7f8e980ade50 2023-11-08T16:41:10.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.007+0000 7f8e9f244640 1 -- 172.21.15.81:0/767448590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e9804ece0 con 0x7f8e980ade50 2023-11-08T16:41:10.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.008+0000 7f8e957fa640 1 -- 172.21.15.81:0/767448590 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8e8c04d450 con 0x7f8e980ade50 2023-11-08T16:41:10.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.009+0000 7f8e957fa640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8e7c06ae70 0x7f8e7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.009+0000 7f8e957fa640 1 -- 172.21.15.81:0/767448590 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e8c02d040 con 0x7f8e980ade50 2023-11-08T16:41:10.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.009+0000 7f8e97fff640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8e7c06ae70 0x7f8e7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.010+0000 7f8e97fff640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8e7c06ae70 0x7f8e7c06d330 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8e84006b10 tx=0x7f8e84008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:10.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.013+0000 7f8e957fa640 1 -- 172.21.15.81:0/767448590 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8e8c076d10 con 0x7f8e980ade50 2023-11-08T16:41:10.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.202+0000 7f8e9f244640 1 -- 172.21.15.81:0/767448590 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e980cc1d0 con 0x7f8e980ade50 2023-11-08T16:41:10.205 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.204+0000 7f8e957fa640 1 -- 172.21.15.81:0/767448590 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16467 (secure 0 0 0) 0x7f8e8c035090 con 0x7f8e980ade50 2023-11-08T16:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:10.206 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":127,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:07.268068+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:41:03.545565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T16:41:04.726492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:41:10.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 -- 172.21.15.81:0/767448590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8e7c06ae70 msgr2=0x7f8e7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8e7c06ae70 0x7f8e7c06d330 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8e84006b10 tx=0x7f8e84008040 comp rx=0 tx=0).stop 2023-11-08T16:41:10.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 -- 172.21.15.81:0/767448590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e980ade50 msgr2=0x7f8e9815dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e980ade50 0x7f8e9815dcf0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c000c00 tx=0x7f8e8c0046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 -- 172.21.15.81:0/767448590 shutdown_connections 2023-11-08T16:41:10.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8e7c06ae70 0x7f8e7c06d330 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8e9815b020 0x7f8e980c4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8e9815a680 0x7f8e98157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 --2- 172.21.15.81:0/767448590 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8e980ade50 0x7f8e9815dcf0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 -- 172.21.15.81:0/767448590 >> 172.21.15.81:0/767448590 conn(0x7f8e980cb490 msgr2=0x7f8e980b19b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:10.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 -- 172.21.15.81:0/767448590 shutdown_connections 2023-11-08T16:41:10.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.207+0000 7f8e9f244640 1 -- 172.21.15.81:0/767448590 wait complete. 2023-11-08T16:41:10.228 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-08T16:41:10.228 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:41:10.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.445+0000 7f24ccc7c640 1 Processor -- start 2023-11-08T16:41:10.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.445+0000 7f24ccc7c640 1 -- start start 2023-11-08T16:41:10.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24ccc7c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f24c80cc5e0 0x7f24c80cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24ccc7c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f24c80ccfb0 0x7f24c8070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24ccc7c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24c80c9a70 con 0x7f24c80adcd0 2023-11-08T16:41:10.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24ccc7c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c80cc3e0 con 0x7f24c80cc5e0 2023-11-08T16:41:10.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24ccc7c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24c806d070 con 0x7f24c80ccfb0 2023-11-08T16:41:10.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24c659c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f24c80adcd0 0x7f24c80ca1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:33038/0 (socket says 172.21.15.81:33038) 2023-11-08T16:41:10.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24bdd9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f24c80cc5e0 0x7f24c80cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24c659c640 1 -- 172.21.15.81:0/3517088383 learned_addr learned my addr 172.21.15.81:0/3517088383 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:10.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24bdd9b640 1 -- 172.21.15.81:0/3517088383 >> v2:172.21.15.93:3300/0 conn(0x7f24c80ccfb0 msgr2=0x7f24c8070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:10.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24bdd9b640 1 --2- 172.21.15.81:0/3517088383 >> v2:172.21.15.93:3300/0 conn(0x7f24c80ccfb0 0x7f24c8070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24bdd9b640 1 -- 172.21.15.81:0/3517088383 >> v1:172.21.15.81:6789/0 conn(0x7f24c80adcd0 legacy=0x7f24c80ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24bdd9b640 1 -- 172.21.15.81:0/3517088383 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c806d350 con 0x7f24c80cc5e0 2023-11-08T16:41:10.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.447+0000 7f24c5d9b640 1 -- 172.21.15.81:0/3517088383 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300548769 0 0) 0x7f24c80c9a70 con 0x7f24c80adcd0 2023-11-08T16:41:10.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.448+0000 7f24bdd9b640 1 --2- 172.21.15.81:0/3517088383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24c80cc5e0 0x7f24c80cca70 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f24c806d940 tx=0x7f24b002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c6364afb142a0a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:10.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.449+0000 7f24c5d9b640 1 -- 172.21.15.81:0/3517088383 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0031830 con 0x7f24c80cc5e0 2023-11-08T16:41:10.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.449+0000 7f24c5d9b640 1 -- 172.21.15.81:0/3517088383 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24b0031990 con 0x7f24c80cc5e0 2023-11-08T16:41:10.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.449+0000 7f24c5d9b640 1 -- 172.21.15.81:0/3517088383 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24b0031d20 con 0x7f24c80cc5e0 2023-11-08T16:41:10.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.449+0000 7f24ccc7c640 1 -- 172.21.15.81:0/3517088383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24c80cc5e0 msgr2=0x7f24c80cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.449+0000 7f24ccc7c640 1 --2- 172.21.15.81:0/3517088383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24c80cc5e0 0x7f24c80cca70 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f24c806d940 tx=0x7f24b002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.449+0000 7f24ccc7c640 1 -- 172.21.15.81:0/3517088383 shutdown_connections 2023-11-08T16:41:10.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.449+0000 7f24ccc7c640 1 --2- 172.21.15.81:0/3517088383 >> v2:172.21.15.93:3300/0 conn(0x7f24c80ccfb0 0x7f24c8070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.449+0000 7f24ccc7c640 1 --2- 172.21.15.81:0/3517088383 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24c80cc5e0 0x7f24c80cca70 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.449+0000 7f24ccc7c640 1 -- 172.21.15.81:0/3517088383 >> 172.21.15.81:0/3517088383 conn(0x7f24c80cb490 msgr2=0x7f24c80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:10.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.449+0000 7f24ccc7c640 1 -- 172.21.15.81:0/3517088383 shutdown_connections 2023-11-08T16:41:10.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.450+0000 7f24ccc7c640 1 -- 172.21.15.81:0/3517088383 wait complete. 2023-11-08T16:41:10.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.451+0000 7f24ccc7c640 1 Processor -- start 2023-11-08T16:41:10.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.451+0000 7f24ccc7c640 1 -- start start 2023-11-08T16:41:10.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.452+0000 7f24ccc7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24c80adcd0 0x7f24c80b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.452+0000 7f24ccc7c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c80cc5e0 0x7f24c81566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.452+0000 7f24ccc7c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24c80ccfb0 0x7f24c81525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.452+0000 7f24ccc7c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80b2430 con 0x7f24c80ccfb0 2023-11-08T16:41:10.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.452+0000 7f24ccc7c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80b2710 con 0x7f24c80cc5e0 2023-11-08T16:41:10.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.452+0000 7f24ccc7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24c80b29f0 con 0x7f24c80adcd0 2023-11-08T16:41:10.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.452+0000 7f24c6d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24c80ccfb0 0x7f24c81525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.453+0000 7f24c6d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24c80ccfb0 0x7f24c81525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58868/0 (socket says 172.21.15.81:58868) 2023-11-08T16:41:10.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.453+0000 7f24c6d9d640 1 -- 172.21.15.81:0/1102657446 learned_addr learned my addr 172.21.15.81:0/1102657446 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:10.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.453+0000 7f24bdd9b640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c80cc5e0 0x7f24c81566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.453+0000 7f24c659c640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24c80adcd0 0x7f24c80b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.453+0000 7f24c6d9d640 1 -- 172.21.15.81:0/1102657446 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24c80adcd0 msgr2=0x7f24c80b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.453+0000 7f24c6d9d640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24c80adcd0 0x7f24c80b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.453+0000 7f24c6d9d640 1 -- 172.21.15.81:0/1102657446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c80cc5e0 msgr2=0x7f24c81566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.453+0000 7f24c6d9d640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c80cc5e0 0x7f24c81566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.453+0000 7f24c6d9d640 1 -- 172.21.15.81:0/1102657446 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b0009400 con 0x7f24c80ccfb0 2023-11-08T16:41:10.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.453+0000 7f24c6d9d640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24c80ccfb0 0x7f24c81525e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f24c000b9e0 tx=0x7f24c000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:10.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.454+0000 7f24bf7fe640 1 -- 172.21.15.81:0/1102657446 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c000cb20 con 0x7f24c80ccfb0 2023-11-08T16:41:10.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.454+0000 7f24bf7fe640 1 -- 172.21.15.81:0/1102657446 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24c000cca0 con 0x7f24c80ccfb0 2023-11-08T16:41:10.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.454+0000 7f24ccc7c640 1 -- 172.21.15.81:0/1102657446 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24c80b2cd0 con 0x7f24c80ccfb0 2023-11-08T16:41:10.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.454+0000 7f24bf7fe640 1 -- 172.21.15.81:0/1102657446 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24c0014680 con 0x7f24c80ccfb0 2023-11-08T16:41:10.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.454+0000 7f24ccc7c640 1 -- 172.21.15.81:0/1102657446 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24c80b42c0 con 0x7f24c80ccfb0 2023-11-08T16:41:10.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.455+0000 7f24ccc7c640 1 -- 172.21.15.81:0/1102657446 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24c8072170 con 0x7f24c80ccfb0 2023-11-08T16:41:10.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.456+0000 7f24bf7fe640 1 -- 172.21.15.81:0/1102657446 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f24c001d3b0 con 0x7f24c80ccfb0 2023-11-08T16:41:10.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.457+0000 7f24bf7fe640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24a806ae70 0x7f24a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.457+0000 7f24bf7fe640 1 -- 172.21.15.81:0/1102657446 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f24c0089570 con 0x7f24c80ccfb0 2023-11-08T16:41:10.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.457+0000 7f24c659c640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24a806ae70 0x7f24a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.458+0000 7f24c659c640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24a806ae70 0x7f24a806d330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f24b8000f60 tx=0x7f24b8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:10.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.462+0000 7f24bf7fe640 1 -- 172.21.15.81:0/1102657446 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f24c0054620 con 0x7f24c80ccfb0 2023-11-08T16:41:10.641 INFO:tasks.ceph.mds.d.smithi093.stderr:2023-11-08T16:41:10.640+0000 7f958a393640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-08T16:41:10.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.646+0000 7f24ccc7c640 1 -- 172.21.15.81:0/1102657446 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24c80c9cc0 con 0x7f24a806ae70 2023-11-08T16:41:10.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.666+0000 7f24bf7fe640 1 -- 172.21.15.81:0/1102657446 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287096 (secure 0 0 0) 0x7f24b8001c70 con 0x7f24a806ae70 2023-11-08T16:41:10.669 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:10.673 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:41:10.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 -- 172.21.15.81:0/1102657446 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24a806ae70 msgr2=0x7f24a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24a806ae70 0x7f24a806d330 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f24b8000f60 tx=0x7f24b8009040 comp rx=0 tx=0).stop 2023-11-08T16:41:10.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 -- 172.21.15.81:0/1102657446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24c80ccfb0 msgr2=0x7f24c81525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24c80ccfb0 0x7f24c81525e0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f24c000b9e0 tx=0x7f24c000bea0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 -- 172.21.15.81:0/1102657446 shutdown_connections 2023-11-08T16:41:10.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f24a806ae70 0x7f24a806d330 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f24c80ccfb0 0x7f24c81525e0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24c80cc5e0 0x7f24c81566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 --2- 172.21.15.81:0/1102657446 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f24c80adcd0 0x7f24c80b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 -- 172.21.15.81:0/1102657446 >> 172.21.15.81:0/1102657446 conn(0x7f24c80cb490 msgr2=0x7f24c80c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:10.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 -- 172.21.15.81:0/1102657446 shutdown_connections 2023-11-08T16:41:10.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.673+0000 7f24ccc7c640 1 -- 172.21.15.81:0/1102657446 wait complete. 2023-11-08T16:41:10.695 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2023-11-08T16:41:08.807216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262820,"kb_used_data":13364,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711900,"statfs":{"total":773094113280,"available":772824985600,"internally_reserved":0,"allocated":13684736,"data_stored":6971620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.573119"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.305319+0000","last_change":"2023-11-08T16:41:06.301616+0000","last_active":"2023-11-08T16:41:06.305319+0000","last_peered":"2023-11-08T16:41:06.305319+0000","last_clean":"2023-11-08T16:41:06.305319+0000","last_became_active":"2023-11-08T16:41:06.301213+0000","last_became_peered":"2023-11-08T16:41:06.301213+0000","last_unstale":"2023-11-08T16:41:06.305319+0000","last_undegraded":"2023-11-08T16:41:06.305319+0000","last_fullsized":"2023-11-08T16:41:06.305319+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:05.385747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.288938+0000","last_change":"2023-11-08T16:41:06.284918+0000","last_active":"2023-11-08T16:41:06.288938+0000","last_peered":"2023-11-08T16:41:06.288938+0000","last_clean":"2023-11-08T16:41:06.288938+0000","last_became_active":"2023-11-08T16:41:06.284648+0000","last_became_peered":"2023-11-08T16:41:06.284648+0000","last_unstale":"2023-11-08T16:41:06.288938+0000","last_undegraded":"2023-11-08T16:41:06.288938+0000","last_fullsized":"2023-11-08T16:41:06.288938+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:58.442996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290919+0000","last_change":"2023-11-08T16:41:06.283004+0000","last_active":"2023-11-08T16:41:06.290919+0000","last_peered":"2023-11-08T16:41:06.290919+0000","last_clean":"2023-11-08T16:41:06.290919+0000","last_became_active":"2023-11-08T16:41:06.281991+0000","last_became_peered":"2023-11-08T16:41:06.281991+0000","last_unstale":"2023-11-08T16:41:06.290919+0000","last_undegraded":"2023-11-08T16:41:06.290919+0000","last_fullsized":"2023-11-08T16:41:06.290919+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:25:57.844111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.300975+0000","last_change":"2023-11-08T16:41:06.292131+0000","last_active":"2023-11-08T16:41:06.300975+0000","last_peered":"2023-11-08T16:41:06.300975+0000","last_clean":"2023-11-08T16:41:06.300975+0000","last_became_active":"2023-11-08T16:41:06.291843+0000","last_became_peered":"2023-11-08T16:41:06.291843+0000","last_unstale":"2023-11-08T16:41:06.300975+0000","last_undegraded":"2023-11-08T16:41:06.300975+0000","last_fullsized":"2023-11-08T16:41:06.300975+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:21.161754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.283831+0000","last_change":"2023-11-08T16:41:06.278719+0000","last_active":"2023-11-08T16:41:06.283831+0000","last_peered":"2023-11-08T16:41:06.283831+0000","last_clean":"2023-11-08T16:41:06.283831+0000","last_became_active":"2023-11-08T16:41:06.277896+0000","last_became_peered":"2023-11-08T16:41:06.277896+0000","last_unstale":"2023-11-08T16:41:06.283831+0000","last_undegraded":"2023-11-08T16:41:06.283831+0000","last_fullsized":"2023-11-08T16:41:06.283831+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:03.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.282121+0000","last_change":"2023-11-08T16:41:06.274978+0000","last_active":"2023-11-08T16:41:06.282121+0000","last_peered":"2023-11-08T16:41:06.282121+0000","last_clean":"2023-11-08T16:41:06.282121+0000","last_became_active":"2023-11-08T16:41:06.274665+0000","last_became_peered":"2023-11-08T16:41:06.274665+0000","last_unstale":"2023-11-08T16:41:06.282121+0000","last_undegraded":"2023-11-08T16:41:06.282121+0000","last_fullsized":"2023-11-08T16:41:06.282121+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:34.391636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.277524+0000","last_change":"2023-11-08T16:41:06.269506+0000","last_active":"2023-11-08T16:41:06.277524+0000","last_peered":"2023-11-08T16:41:06.277524+0000","last_clean":"2023-11-08T16:41:06.277524+0000","last_became_active":"2023-11-08T16:41:06.269083+0000","last_became_peered":"2023-11-08T16:41:06.269083+0000","last_unstale":"2023-11-08T16:41:06.277524+0000","last_undegraded":"2023-11-08T16:41:06.277524+0000","last_fullsized":"2023-11-08T16:41:06.277524+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:43.494464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291180+0000","last_change":"2023-11-08T16:41:06.286884+0000","last_active":"2023-11-08T16:41:06.291180+0000","last_peered":"2023-11-08T16:41:06.291180+0000","last_clean":"2023-11-08T16:41:06.291180+0000","last_became_active":"2023-11-08T16:41:06.282539+0000","last_became_peered":"2023-11-08T16:41:06.282539+0000","last_unstale":"2023-11-08T16:41:06.291180+0000","last_undegraded":"2023-11-08T16:41:06.291180+0000","last_fullsized":"2023-11-08T16:41:06.291180+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:34.391636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.303032+0000","last_change":"2023-11-08T16:41:06.299208+0000","last_active":"2023-11-08T16:41:06.303032+0000","last_peered":"2023-11-08T16:41:06.303032+0000","last_clean":"2023-11-08T16:41:06.303032+0000","last_became_active":"2023-11-08T16:41:06.297534+0000","last_became_peered":"2023-11-08T16:41:06.297534+0000","last_unstale":"2023-11-08T16:41:06.303032+0000","last_undegraded":"2023-11-08T16:41:06.303032+0000","last_fullsized":"2023-11-08T16:41:06.303032+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:27.335196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.300896+0000","last_change":"2023-11-08T16:41:06.291973+0000","last_active":"2023-11-08T16:41:06.300896+0000","last_peered":"2023-11-08T16:41:06.300896+0000","last_clean":"2023-11-08T16:41:06.300896+0000","last_became_active":"2023-11-08T16:41:06.291692+0000","last_became_peered":"2023-11-08T16:41:06.291692+0000","last_unstale":"2023-11-08T16:41:06.300896+0000","last_undegraded":"2023-11-08T16:41:06.300896+0000","last_fullsized":"2023-11-08T16:41:06.300896+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:19:21.405123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.285836+0000","last_change":"2023-11-08T16:41:06.282233+0000","last_active":"2023-11-08T16:41:06.285836+0000","last_peered":"2023-11-08T16:41:06.285836+0000","last_clean":"2023-11-08T16:41:06.285836+0000","last_became_active":"2023-11-08T16:41:06.281637+0000","last_became_peered":"2023-11-08T16:41:06.281637+0000","last_unstale":"2023-11-08T16:41:06.285836+0000","last_undegraded":"2023-11-08T16:41:06.285836+0000","last_fullsized":"2023-11-08T16:41:06.285836+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:52:57.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.300930+0000","last_change":"2023-11-08T16:41:06.288442+0000","last_active":"2023-11-08T16:41:06.300930+0000","last_peered":"2023-11-08T16:41:06.300930+0000","last_clean":"2023-11-08T16:41:06.300930+0000","last_became_active":"2023-11-08T16:41:06.288157+0000","last_became_peered":"2023-11-08T16:41:06.288157+0000","last_unstale":"2023-11-08T16:41:06.300930+0000","last_undegraded":"2023-11-08T16:41:06.300930+0000","last_fullsized":"2023-11-08T16:41:06.300930+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:58:34.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.292926+0000","last_change":"2023-11-08T16:41:06.290385+0000","last_active":"2023-11-08T16:41:06.292926+0000","last_peered":"2023-11-08T16:41:06.292926+0000","last_clean":"2023-11-08T16:41:06.292926+0000","last_became_active":"2023-11-08T16:41:06.286883+0000","last_became_peered":"2023-11-08T16:41:06.286883+0000","last_unstale":"2023-11-08T16:41:06.292926+0000","last_undegraded":"2023-11-08T16:41:06.292926+0000","last_fullsized":"2023-11-08T16:41:06.292926+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:29:25.784060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290475+0000","last_change":"2023-11-08T16:41:06.282234+0000","last_active":"2023-11-08T16:41:06.290475+0000","last_peered":"2023-11-08T16:41:06.290475+0000","last_clean":"2023-11-08T16:41:06.290475+0000","last_became_active":"2023-11-08T16:41:06.281818+0000","last_became_peered":"2023-11-08T16:41:06.281818+0000","last_unstale":"2023-11-08T16:41:06.290475+0000","last_undegraded":"2023-11-08T16:41:06.290475+0000","last_fullsized":"2023-11-08T16:41:06.290475+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:10:54.032689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.281244+0000","last_change":"2023-11-08T16:41:06.275909+0000","last_active":"2023-11-08T16:41:06.281244+0000","last_peered":"2023-11-08T16:41:06.281244+0000","last_clean":"2023-11-08T16:41:06.281244+0000","last_became_active":"2023-11-08T16:41:06.275560+0000","last_became_peered":"2023-11-08T16:41:06.275560+0000","last_unstale":"2023-11-08T16:41:06.281244+0000","last_undegraded":"2023-11-08T16:41:06.281244+0000","last_fullsized":"2023-11-08T16:41:06.281244+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:29:25.784060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.302702+0000","last_change":"2023-11-08T16:41:06.297676+0000","last_active":"2023-11-08T16:41:06.302702+0000","last_peered":"2023-11-08T16:41:06.302702+0000","last_clean":"2023-11-08T16:41:06.302702+0000","last_became_active":"2023-11-08T16:41:06.295552+0000","last_became_peered":"2023-11-08T16:41:06.295552+0000","last_unstale":"2023-11-08T16:41:06.302702+0000","last_undegraded":"2023-11-08T16:41:06.302702+0000","last_fullsized":"2023-11-08T16:41:06.302702+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:18.714608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.288921+0000","last_change":"2023-11-08T16:41:06.285617+0000","last_active":"2023-11-08T16:41:06.288921+0000","last_peered":"2023-11-08T16:41:06.288921+0000","last_clean":"2023-11-08T16:41:06.288921+0000","last_became_active":"2023-11-08T16:41:06.284944+0000","last_became_peered":"2023-11-08T16:41:06.284944+0000","last_unstale":"2023-11-08T16:41:06.288921+0000","last_undegraded":"2023-11-08T16:41:06.288921+0000","last_fullsized":"2023-11-08T16:41:06.288921+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:56.049065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.281893+0000","last_change":"2023-11-08T16:41:06.279410+0000","last_active":"2023-11-08T16:41:06.281893+0000","last_peered":"2023-11-08T16:41:06.281893+0000","last_clean":"2023-11-08T16:41:06.281893+0000","last_became_active":"2023-11-08T16:41:06.278826+0000","last_became_peered":"2023-11-08T16:41:06.278826+0000","last_unstale":"2023-11-08T16:41:06.281893+0000","last_undegraded":"2023-11-08T16:41:06.281893+0000","last_fullsized":"2023-11-08T16:41:06.281893+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:43.494464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291956+0000","last_change":"2023-11-08T16:41:06.289538+0000","last_active":"2023-11-08T16:41:06.291956+0000","last_peered":"2023-11-08T16:41:06.291956+0000","last_clean":"2023-11-08T16:41:06.291956+0000","last_became_active":"2023-11-08T16:41:06.282421+0000","last_became_peered":"2023-11-08T16:41:06.282421+0000","last_unstale":"2023-11-08T16:41:06.291956+0000","last_undegraded":"2023-11-08T16:41:06.291956+0000","last_fullsized":"2023-11-08T16:41:06.291956+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:52:57.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290574+0000","last_change":"2023-11-08T16:41:06.287860+0000","last_active":"2023-11-08T16:41:06.290574+0000","last_peered":"2023-11-08T16:41:06.290574+0000","last_clean":"2023-11-08T16:41:06.290574+0000","last_became_active":"2023-11-08T16:41:06.287608+0000","last_became_peered":"2023-11-08T16:41:06.287608+0000","last_unstale":"2023-11-08T16:41:06.290574+0000","last_undegraded":"2023-11-08T16:41:06.290574+0000","last_fullsized":"2023-11-08T16:41:06.290574+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:58.416407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291313+0000","last_change":"2023-11-08T16:41:06.289741+0000","last_active":"2023-11-08T16:41:06.291313+0000","last_peered":"2023-11-08T16:41:06.291313+0000","last_clean":"2023-11-08T16:41:06.291313+0000","last_became_active":"2023-11-08T16:41:06.288680+0000","last_became_peered":"2023-11-08T16:41:06.288680+0000","last_unstale":"2023-11-08T16:41:06.291313+0000","last_undegraded":"2023-11-08T16:41:06.291313+0000","last_fullsized":"2023-11-08T16:41:06.291313+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:08:21.221808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.275052+0000","last_change":"2023-11-08T16:41:06.272480+0000","last_active":"2023-11-08T16:41:06.275052+0000","last_peered":"2023-11-08T16:41:06.275052+0000","last_clean":"2023-11-08T16:41:06.275052+0000","last_became_active":"2023-11-08T16:41:06.271356+0000","last_became_peered":"2023-11-08T16:41:06.271356+0000","last_unstale":"2023-11-08T16:41:06.275052+0000","last_undegraded":"2023-11-08T16:41:06.275052+0000","last_fullsized":"2023-11-08T16:41:06.275052+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:08:21.221808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291065+0000","last_change":"2023-11-08T16:41:06.288411+0000","last_active":"2023-11-08T16:41:06.291065+0000","last_peered":"2023-11-08T16:41:06.291065+0000","last_clean":"2023-11-08T16:41:06.291065+0000","last_became_active":"2023-11-08T16:41:06.287939+0000","last_became_peered":"2023-11-08T16:41:06.287939+0000","last_unstale":"2023-11-08T16:41:06.291065+0000","last_undegraded":"2023-11-08T16:41:06.291065+0000","last_fullsized":"2023-11-08T16:41:06.291065+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:39:21.572152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.287809+0000","last_change":"2023-11-08T16:41:06.281996+0000","last_active":"2023-11-08T16:41:06.287809+0000","last_peered":"2023-11-08T16:41:06.287809+0000","last_clean":"2023-11-08T16:41:06.287809+0000","last_became_active":"2023-11-08T16:41:06.281666+0000","last_became_peered":"2023-11-08T16:41:06.281666+0000","last_unstale":"2023-11-08T16:41:06.287809+0000","last_undegraded":"2023-11-08T16:41:06.287809+0000","last_fullsized":"2023-11-08T16:41:06.287809+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:58.416407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.288272+0000","last_change":"2023-11-08T16:41:06.283649+0000","last_active":"2023-11-08T16:41:06.288272+0000","last_peered":"2023-11-08T16:41:06.288272+0000","last_clean":"2023-11-08T16:41:06.288272+0000","last_became_active":"2023-11-08T16:41:06.283344+0000","last_became_peered":"2023-11-08T16:41:06.283344+0000","last_unstale":"2023-11-08T16:41:06.288272+0000","last_undegraded":"2023-11-08T16:41:06.288272+0000","last_fullsized":"2023-11-08T16:41:06.288272+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:22:54.921237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.301074+0000","last_change":"2023-11-08T16:41:06.295380+0000","last_active":"2023-11-08T16:41:06.301074+0000","last_peered":"2023-11-08T16:41:06.301074+0000","last_clean":"2023-11-08T16:41:06.301074+0000","last_became_active":"2023-11-08T16:41:06.292522+0000","last_became_peered":"2023-11-08T16:41:06.292522+0000","last_unstale":"2023-11-08T16:41:06.301074+0000","last_undegraded":"2023-11-08T16:41:06.301074+0000","last_fullsized":"2023-11-08T16:41:06.301074+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:37:31.717459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.303481+0000","last_change":"2023-11-08T16:41:06.301848+0000","last_active":"2023-11-08T16:41:06.303481+0000","last_peered":"2023-11-08T16:41:06.303481+0000","last_clean":"2023-11-08T16:41:06.303481+0000","last_became_active":"2023-11-08T16:41:06.301111+0000","last_became_peered":"2023-11-08T16:41:06.301111+0000","last_unstale":"2023-11-08T16:41:06.303481+0000","last_undegraded":"2023-11-08T16:41:06.303481+0000","last_fullsized":"2023-11-08T16:41:06.303481+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:10:54.032689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.296304+0000","last_change":"2023-11-08T16:41:06.293306+0000","last_active":"2023-11-08T16:41:06.296304+0000","last_peered":"2023-11-08T16:41:06.296304+0000","last_clean":"2023-11-08T16:41:06.296304+0000","last_became_active":"2023-11-08T16:41:06.292834+0000","last_became_peered":"2023-11-08T16:41:06.292834+0000","last_unstale":"2023-11-08T16:41:06.296304+0000","last_undegraded":"2023-11-08T16:41:06.296304+0000","last_fullsized":"2023-11-08T16:41:06.296304+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:56.049065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.294138+0000","last_change":"2023-11-08T16:41:06.292936+0000","last_active":"2023-11-08T16:41:06.294138+0000","last_peered":"2023-11-08T16:41:06.294138+0000","last_clean":"2023-11-08T16:41:06.294138+0000","last_became_active":"2023-11-08T16:41:06.291914+0000","last_became_peered":"2023-11-08T16:41:06.291914+0000","last_unstale":"2023-11-08T16:41:06.294138+0000","last_undegraded":"2023-11-08T16:41:06.294138+0000","last_fullsized":"2023-11-08T16:41:06.294138+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.335700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.280246+0000","last_change":"2023-11-08T16:41:06.277501+0000","last_active":"2023-11-08T16:41:06.280246+0000","last_peered":"2023-11-08T16:41:06.280246+0000","last_clean":"2023-11-08T16:41:06.280246+0000","last_became_active":"2023-11-08T16:41:06.277217+0000","last_became_peered":"2023-11-08T16:41:06.277217+0000","last_unstale":"2023-11-08T16:41:06.280246+0000","last_undegraded":"2023-11-08T16:41:06.280246+0000","last_fullsized":"2023-11-08T16:41:06.280246+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:53.393145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.301170+0000","last_change":"2023-11-08T16:41:06.293906+0000","last_active":"2023-11-08T16:41:06.301170+0000","last_peered":"2023-11-08T16:41:06.301170+0000","last_clean":"2023-11-08T16:41:06.301170+0000","last_became_active":"2023-11-08T16:41:06.292421+0000","last_became_peered":"2023-11-08T16:41:06.292421+0000","last_unstale":"2023-11-08T16:41:06.301170+0000","last_undegraded":"2023-11-08T16:41:06.301170+0000","last_fullsized":"2023-11-08T16:41:06.301170+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:55:11.977247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.279176+0000","last_change":"2023-11-08T16:41:06.274274+0000","last_active":"2023-11-08T16:41:06.279176+0000","last_peered":"2023-11-08T16:41:06.279176+0000","last_clean":"2023-11-08T16:41:06.279176+0000","last_became_active":"2023-11-08T16:41:06.271427+0000","last_became_peered":"2023-11-08T16:41:06.271427+0000","last_unstale":"2023-11-08T16:41:06.279176+0000","last_undegraded":"2023-11-08T16:41:06.279176+0000","last_fullsized":"2023-11-08T16:41:06.279176+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:07:10.051108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.275214+0000","last_change":"2023-11-08T16:41:06.273090+0000","last_active":"2023-11-08T16:41:06.275214+0000","last_peered":"2023-11-08T16:41:06.275214+0000","last_clean":"2023-11-08T16:41:06.275214+0000","last_became_active":"2023-11-08T16:41:06.272643+0000","last_became_peered":"2023-11-08T16:41:06.272643+0000","last_unstale":"2023-11-08T16:41:06.275214+0000","last_undegraded":"2023-11-08T16:41:06.275214+0000","last_fullsized":"2023-11-08T16:41:06.275214+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:39:32.914205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"126'4","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.293326+0000","last_change":"2023-11-08T16:41:05.274000+0000","last_active":"2023-11-08T16:41:06.293326+0000","last_peered":"2023-11-08T16:41:06.293326+0000","last_clean":"2023-11-08T16:41:06.293326+0000","last_became_active":"2023-11-08T16:41:05.273705+0000","last_became_peered":"2023-11-08T16:41:05.273705+0000","last_unstale":"2023-11-08T16:41:06.293326+0000","last_undegraded":"2023-11-08T16:41:06.293326+0000","last_fullsized":"2023-11-08T16:41:06.293326+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:34:25.857434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.281934+0000","last_change":"2023-11-08T16:41:06.280252+0000","last_active":"2023-11-08T16:41:06.281934+0000","last_peered":"2023-11-08T16:41:06.281934+0000","last_clean":"2023-11-08T16:41:06.281934+0000","last_became_active":"2023-11-08T16:41:06.279960+0000","last_became_peered":"2023-11-08T16:41:06.279960+0000","last_unstale":"2023-11-08T16:41:06.281934+0000","last_undegraded":"2023-11-08T16:41:06.281934+0000","last_fullsized":"2023-11-08T16:41:06.281934+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:46:22.237288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"126'1","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.294383+0000","last_change":"2023-11-08T16:41:05.279755+0000","last_active":"2023-11-08T16:41:06.294383+0000","last_peered":"2023-11-08T16:41:06.294383+0000","last_clean":"2023-11-08T16:41:06.294383+0000","last_became_active":"2023-11-08T16:41:05.277987+0000","last_became_peered":"2023-11-08T16:41:05.277987+0000","last_unstale":"2023-11-08T16:41:06.294383+0000","last_undegraded":"2023-11-08T16:41:06.294383+0000","last_fullsized":"2023-11-08T16:41:06.294383+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:57.399931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.280296+0000","last_change":"2023-11-08T16:41:06.275763+0000","last_active":"2023-11-08T16:41:06.280296+0000","last_peered":"2023-11-08T16:41:06.280296+0000","last_clean":"2023-11-08T16:41:06.280296+0000","last_became_active":"2023-11-08T16:41:06.271606+0000","last_became_peered":"2023-11-08T16:41:06.271606+0000","last_unstale":"2023-11-08T16:41:06.280296+0000","last_undegraded":"2023-11-08T16:41:06.280296+0000","last_fullsized":"2023-11-08T16:41:06.280296+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:25:57.844111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291473+0000","last_change":"2023-11-08T16:41:05.267264+0000","last_active":"2023-11-08T16:41:06.291473+0000","last_peered":"2023-11-08T16:41:06.291473+0000","last_clean":"2023-11-08T16:41:06.291473+0000","last_became_active":"2023-11-08T16:41:05.266968+0000","last_became_peered":"2023-11-08T16:41:05.266968+0000","last_unstale":"2023-11-08T16:41:06.291473+0000","last_undegraded":"2023-11-08T16:41:06.291473+0000","last_fullsized":"2023-11-08T16:41:06.291473+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:02:16.702650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291175+0000","last_change":"2023-11-08T16:41:06.283362+0000","last_active":"2023-11-08T16:41:06.291175+0000","last_peered":"2023-11-08T16:41:06.291175+0000","last_clean":"2023-11-08T16:41:06.291175+0000","last_became_active":"2023-11-08T16:41:06.282682+0000","last_became_peered":"2023-11-08T16:41:06.282682+0000","last_unstale":"2023-11-08T16:41:06.291175+0000","last_undegraded":"2023-11-08T16:41:06.291175+0000","last_fullsized":"2023-11-08T16:41:06.291175+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:39:21.572152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291425+0000","last_change":"2023-11-08T16:41:05.273695+0000","last_active":"2023-11-08T16:41:06.291425+0000","last_peered":"2023-11-08T16:41:06.291425+0000","last_clean":"2023-11-08T16:41:06.291425+0000","last_became_active":"2023-11-08T16:41:05.273377+0000","last_became_peered":"2023-11-08T16:41:05.273377+0000","last_unstale":"2023-11-08T16:41:06.291425+0000","last_undegraded":"2023-11-08T16:41:06.291425+0000","last_fullsized":"2023-11-08T16:41:06.291425+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:32:16.371465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.301138+0000","last_change":"2023-11-08T16:41:06.296231+0000","last_active":"2023-11-08T16:41:06.301138+0000","last_peered":"2023-11-08T16:41:06.301138+0000","last_clean":"2023-11-08T16:41:06.301138+0000","last_became_active":"2023-11-08T16:41:06.292390+0000","last_became_peered":"2023-11-08T16:41:06.292390+0000","last_unstale":"2023-11-08T16:41:06.301138+0000","last_undegraded":"2023-11-08T16:41:06.301138+0000","last_fullsized":"2023-11-08T16:41:06.301138+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:29.424324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"126'1","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.278825+0000","last_change":"2023-11-08T16:41:05.270953+0000","last_active":"2023-11-08T16:41:06.278825+0000","last_peered":"2023-11-08T16:41:06.278825+0000","last_clean":"2023-11-08T16:41:06.278825+0000","last_became_active":"2023-11-08T16:41:05.270572+0000","last_became_peered":"2023-11-08T16:41:05.270572+0000","last_unstale":"2023-11-08T16:41:06.278825+0000","last_undegraded":"2023-11-08T16:41:06.278825+0000","last_fullsized":"2023-11-08T16:41:06.278825+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:59.029607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290737+0000","last_change":"2023-11-08T16:41:06.286455+0000","last_active":"2023-11-08T16:41:06.290737+0000","last_peered":"2023-11-08T16:41:06.290737+0000","last_clean":"2023-11-08T16:41:06.290737+0000","last_became_active":"2023-11-08T16:41:06.285809+0000","last_became_peered":"2023-11-08T16:41:06.285809+0000","last_unstale":"2023-11-08T16:41:06.290737+0000","last_undegraded":"2023-11-08T16:41:06.290737+0000","last_fullsized":"2023-11-08T16:41:06.290737+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:43:22.680629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"126'1","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290403+0000","last_change":"2023-11-08T16:41:05.273052+0000","last_active":"2023-11-08T16:41:06.290403+0000","last_peered":"2023-11-08T16:41:06.290403+0000","last_clean":"2023-11-08T16:41:06.290403+0000","last_became_active":"2023-11-08T16:41:05.272801+0000","last_became_peered":"2023-11-08T16:41:05.272801+0000","last_unstale":"2023-11-08T16:41:06.290403+0000","last_undegraded":"2023-11-08T16:41:06.290403+0000","last_fullsized":"2023-11-08T16:41:06.290403+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:37.319224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.302831+0000","last_change":"2023-11-08T16:41:06.299315+0000","last_active":"2023-11-08T16:41:06.302831+0000","last_peered":"2023-11-08T16:41:06.302831+0000","last_clean":"2023-11-08T16:41:06.302831+0000","last_became_active":"2023-11-08T16:41:06.298125+0000","last_became_peered":"2023-11-08T16:41:06.298125+0000","last_unstale":"2023-11-08T16:41:06.302831+0000","last_undegraded":"2023-11-08T16:41:06.302831+0000","last_fullsized":"2023-11-08T16:41:06.302831+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:38:59.249974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.274955+0000","last_change":"2023-11-08T16:41:05.279401+0000","last_active":"2023-11-08T16:41:06.274955+0000","last_peered":"2023-11-08T16:41:06.274955+0000","last_clean":"2023-11-08T16:41:06.274955+0000","last_became_active":"2023-11-08T16:41:05.278988+0000","last_became_peered":"2023-11-08T16:41:05.278988+0000","last_unstale":"2023-11-08T16:41:06.274955+0000","last_undegraded":"2023-11-08T16:41:06.274955+0000","last_fullsized":"2023-11-08T16:41:06.274955+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:16:54.418211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.270399+0000","last_change":"2023-11-08T16:41:06.268004+0000","last_active":"2023-11-08T16:41:06.270399+0000","last_peered":"2023-11-08T16:41:06.270399+0000","last_clean":"2023-11-08T16:41:06.270399+0000","last_became_active":"2023-11-08T16:41:06.267321+0000","last_became_peered":"2023-11-08T16:41:06.267321+0000","last_unstale":"2023-11-08T16:41:06.270399+0000","last_undegraded":"2023-11-08T16:41:06.270399+0000","last_fullsized":"2023-11-08T16:41:06.270399+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:45:16.031347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.285010+0000","last_change":"2023-11-08T16:41:05.275407+0000","last_active":"2023-11-08T16:41:06.285010+0000","last_peered":"2023-11-08T16:41:06.285010+0000","last_clean":"2023-11-08T16:41:06.285010+0000","last_became_active":"2023-11-08T16:41:05.274947+0000","last_became_peered":"2023-11-08T16:41:05.274947+0000","last_unstale":"2023-11-08T16:41:06.285010+0000","last_undegraded":"2023-11-08T16:41:06.285010+0000","last_fullsized":"2023-11-08T16:41:06.285010+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:34.700161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.279431+0000","last_change":"2023-11-08T16:41:06.272486+0000","last_active":"2023-11-08T16:41:06.279431+0000","last_peered":"2023-11-08T16:41:06.279431+0000","last_clean":"2023-11-08T16:41:06.279431+0000","last_became_active":"2023-11-08T16:41:06.272059+0000","last_became_peered":"2023-11-08T16:41:06.272059+0000","last_unstale":"2023-11-08T16:41:06.279431+0000","last_undegraded":"2023-11-08T16:41:06.279431+0000","last_fullsized":"2023-11-08T16:41:06.279431+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:45.675098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.265644+0000","last_change":"2023-11-08T16:41:05.270167+0000","last_active":"2023-11-08T16:41:06.265644+0000","last_peered":"2023-11-08T16:41:06.265644+0000","last_clean":"2023-11-08T16:41:06.265644+0000","last_became_active":"2023-11-08T16:41:05.269688+0000","last_became_peered":"2023-11-08T16:41:05.269688+0000","last_unstale":"2023-11-08T16:41:06.265644+0000","last_undegraded":"2023-11-08T16:41:06.265644+0000","last_fullsized":"2023-11-08T16:41:06.265644+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:52:59.580475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.299294+0000","last_change":"2023-11-08T16:41:06.298612+0000","last_active":"2023-11-08T16:41:06.299294+0000","last_peered":"2023-11-08T16:41:06.299294+0000","last_clean":"2023-11-08T16:41:06.299294+0000","last_became_active":"2023-11-08T16:41:06.298347+0000","last_became_peered":"2023-11-08T16:41:06.298347+0000","last_unstale":"2023-11-08T16:41:06.299294+0000","last_undegraded":"2023-11-08T16:41:06.299294+0000","last_fullsized":"2023-11-08T16:41:06.299294+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:25:57.844111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.267266+0000","last_change":"2023-11-08T16:41:05.279053+0000","last_active":"2023-11-08T16:41:06.267266+0000","last_peered":"2023-11-08T16:41:06.267266+0000","last_clean":"2023-11-08T16:41:06.267266+0000","last_became_active":"2023-11-08T16:41:05.278795+0000","last_became_peered":"2023-11-08T16:41:05.278795+0000","last_unstale":"2023-11-08T16:41:06.267266+0000","last_undegraded":"2023-11-08T16:41:06.267266+0000","last_fullsized":"2023-11-08T16:41:06.267266+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:32.302339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.286964+0000","last_change":"2023-11-08T16:41:06.280664+0000","last_active":"2023-11-08T16:41:06.286964+0000","last_peered":"2023-11-08T16:41:06.286964+0000","last_clean":"2023-11-08T16:41:06.286964+0000","last_became_active":"2023-11-08T16:41:06.280083+0000","last_became_peered":"2023-11-08T16:41:06.280083+0000","last_unstale":"2023-11-08T16:41:06.286964+0000","last_undegraded":"2023-11-08T16:41:06.286964+0000","last_fullsized":"2023-11-08T16:41:06.286964+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:32.971875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"126'3","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.305225+0000","last_change":"2023-11-08T16:41:05.281765+0000","last_active":"2023-11-08T16:41:06.305225+0000","last_peered":"2023-11-08T16:41:06.305225+0000","last_clean":"2023-11-08T16:41:06.305225+0000","last_became_active":"2023-11-08T16:41:05.281465+0000","last_became_peered":"2023-11-08T16:41:05.281465+0000","last_unstale":"2023-11-08T16:41:06.305225+0000","last_undegraded":"2023-11-08T16:41:06.305225+0000","last_fullsized":"2023-11-08T16:41:06.305225+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:31:00.273799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.285764+0000","last_change":"2023-11-08T16:41:06.285109+0000","last_active":"2023-11-08T16:41:06.285764+0000","last_peered":"2023-11-08T16:41:06.285764+0000","last_clean":"2023-11-08T16:41:06.285764+0000","last_became_active":"2023-11-08T16:41:06.284429+0000","last_became_peered":"2023-11-08T16:41:06.284429+0000","last_unstale":"2023-11-08T16:41:06.285764+0000","last_undegraded":"2023-11-08T16:41:06.285764+0000","last_fullsized":"2023-11-08T16:41:06.285764+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:23.747959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"126'6","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.305625+0000","last_change":"2023-11-08T16:41:05.282311+0000","last_active":"2023-11-08T16:41:06.305625+0000","last_peered":"2023-11-08T16:41:06.305625+0000","last_clean":"2023-11-08T16:41:06.305625+0000","last_became_active":"2023-11-08T16:41:05.281965+0000","last_became_peered":"2023-11-08T16:41:05.281965+0000","last_unstale":"2023-11-08T16:41:06.305625+0000","last_undegraded":"2023-11-08T16:41:06.305625+0000","last_fullsized":"2023-11-08T16:41:06.305625+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:38:27.762792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.287875+0000","last_change":"2023-11-08T16:41:06.282147+0000","last_active":"2023-11-08T16:41:06.287875+0000","last_peered":"2023-11-08T16:41:06.287875+0000","last_clean":"2023-11-08T16:41:06.287875+0000","last_became_active":"2023-11-08T16:41:06.281815+0000","last_became_peered":"2023-11-08T16:41:06.281815+0000","last_unstale":"2023-11-08T16:41:06.287875+0000","last_undegraded":"2023-11-08T16:41:06.287875+0000","last_fullsized":"2023-11-08T16:41:06.287875+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:43:22.680629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.293080+0000","last_change":"2023-11-08T16:41:05.274758+0000","last_active":"2023-11-08T16:41:06.293080+0000","last_peered":"2023-11-08T16:41:06.293080+0000","last_clean":"2023-11-08T16:41:06.293080+0000","last_became_active":"2023-11-08T16:41:05.273708+0000","last_became_peered":"2023-11-08T16:41:05.273708+0000","last_unstale":"2023-11-08T16:41:06.293080+0000","last_undegraded":"2023-11-08T16:41:06.293080+0000","last_fullsized":"2023-11-08T16:41:06.293080+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:23.456254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290771+0000","last_change":"2023-11-08T16:41:06.288502+0000","last_active":"2023-11-08T16:41:06.290771+0000","last_peered":"2023-11-08T16:41:06.290771+0000","last_clean":"2023-11-08T16:41:06.290771+0000","last_became_active":"2023-11-08T16:41:06.288165+0000","last_became_peered":"2023-11-08T16:41:06.288165+0000","last_unstale":"2023-11-08T16:41:06.290771+0000","last_undegraded":"2023-11-08T16:41:06.290771+0000","last_fullsized":"2023-11-08T16:41:06.290771+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:58.442996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.257711+0000","last_change":"2023-11-08T16:41:05.274096+0000","last_active":"2023-11-08T16:41:06.257711+0000","last_peered":"2023-11-08T16:41:06.257711+0000","last_clean":"2023-11-08T16:41:06.257711+0000","last_became_active":"2023-11-08T16:41:05.273591+0000","last_became_peered":"2023-11-08T16:41:05.273591+0000","last_unstale":"2023-11-08T16:41:06.257711+0000","last_undegraded":"2023-11-08T16:41:06.257711+0000","last_fullsized":"2023-11-08T16:41:06.257711+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:34:25.857434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.286786+0000","last_change":"2023-11-08T16:41:06.278021+0000","last_active":"2023-11-08T16:41:06.286786+0000","last_peered":"2023-11-08T16:41:06.286786+0000","last_clean":"2023-11-08T16:41:06.286786+0000","last_became_active":"2023-11-08T16:41:06.276696+0000","last_became_peered":"2023-11-08T16:41:06.276696+0000","last_unstale":"2023-11-08T16:41:06.286786+0000","last_undegraded":"2023-11-08T16:41:06.286786+0000","last_fullsized":"2023-11-08T16:41:06.286786+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:22:54.921237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"126'8","reported_seq":19,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.302490+0000","last_change":"2023-11-08T16:41:05.277630+0000","last_active":"2023-11-08T16:41:06.302490+0000","last_peered":"2023-11-08T16:41:06.302490+0000","last_clean":"2023-11-08T16:41:06.302490+0000","last_became_active":"2023-11-08T16:41:05.277385+0000","last_became_peered":"2023-11-08T16:41:05.277385+0000","last_unstale":"2023-11-08T16:41:06.302490+0000","last_undegraded":"2023-11-08T16:41:06.302490+0000","last_fullsized":"2023-11-08T16:41:06.302490+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:27.156278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.300059+0000","last_change":"2023-11-08T16:41:06.295988+0000","last_active":"2023-11-08T16:41:06.300059+0000","last_peered":"2023-11-08T16:41:06.300059+0000","last_clean":"2023-11-08T16:41:06.300059+0000","last_became_active":"2023-11-08T16:41:06.294174+0000","last_became_peered":"2023-11-08T16:41:06.294174+0000","last_unstale":"2023-11-08T16:41:06.300059+0000","last_undegraded":"2023-11-08T16:41:06.300059+0000","last_fullsized":"2023-11-08T16:41:06.300059+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:31:58.580635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"126'1","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.301494+0000","last_change":"2023-11-08T16:41:05.280216+0000","last_active":"2023-11-08T16:41:06.301494+0000","last_peered":"2023-11-08T16:41:06.301494+0000","last_clean":"2023-11-08T16:41:06.301494+0000","last_became_active":"2023-11-08T16:41:05.279862+0000","last_became_peered":"2023-11-08T16:41:05.279862+0000","last_unstale":"2023-11-08T16:41:06.301494+0000","last_undegraded":"2023-11-08T16:41:06.301494+0000","last_fullsized":"2023-11-08T16:41:06.301494+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:06:35.334785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.303375+0000","last_change":"2023-11-08T16:41:06.300698+0000","last_active":"2023-11-08T16:41:06.303375+0000","last_peered":"2023-11-08T16:41:06.303375+0000","last_clean":"2023-11-08T16:41:06.303375+0000","last_became_active":"2023-11-08T16:41:06.300426+0000","last_became_peered":"2023-11-08T16:41:06.300426+0000","last_unstale":"2023-11-08T16:41:06.303375+0000","last_undegraded":"2023-11-08T16:41:06.303375+0000","last_fullsized":"2023-11-08T16:41:06.303375+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:52:57.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.294997+0000","last_change":"2023-11-08T16:41:05.281131+0000","last_active":"2023-11-08T16:41:06.294997+0000","last_peered":"2023-11-08T16:41:06.294997+0000","last_clean":"2023-11-08T16:41:06.294997+0000","last_became_active":"2023-11-08T16:41:05.280861+0000","last_became_peered":"2023-11-08T16:41:05.280861+0000","last_unstale":"2023-11-08T16:41:06.294997+0000","last_undegraded":"2023-11-08T16:41:06.294997+0000","last_fullsized":"2023-11-08T16:41:06.294997+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:03:16.579687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.296115+0000","last_change":"2023-11-08T16:41:06.295497+0000","last_active":"2023-11-08T16:41:06.296115+0000","last_peered":"2023-11-08T16:41:06.296115+0000","last_clean":"2023-11-08T16:41:06.296115+0000","last_became_active":"2023-11-08T16:41:06.295216+0000","last_became_peered":"2023-11-08T16:41:06.295216+0000","last_unstale":"2023-11-08T16:41:06.296115+0000","last_undegraded":"2023-11-08T16:41:06.296115+0000","last_fullsized":"2023-11-08T16:41:06.296115+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:46:22.237288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"126'1","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.288099+0000","last_change":"2023-11-08T16:41:05.276756+0000","last_active":"2023-11-08T16:41:06.288099+0000","last_peered":"2023-11-08T16:41:06.288099+0000","last_clean":"2023-11-08T16:41:06.288099+0000","last_became_active":"2023-11-08T16:41:05.275622+0000","last_became_peered":"2023-11-08T16:41:05.275622+0000","last_unstale":"2023-11-08T16:41:06.288099+0000","last_undegraded":"2023-11-08T16:41:06.288099+0000","last_fullsized":"2023-11-08T16:41:06.288099+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:29:24.767585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.292899+0000","last_change":"2023-11-08T16:41:06.290815+0000","last_active":"2023-11-08T16:41:06.292899+0000","last_peered":"2023-11-08T16:41:06.292899+0000","last_clean":"2023-11-08T16:41:06.292899+0000","last_became_active":"2023-11-08T16:41:06.282718+0000","last_became_peered":"2023-11-08T16:41:06.282718+0000","last_unstale":"2023-11-08T16:41:06.292899+0000","last_undegraded":"2023-11-08T16:41:06.292899+0000","last_fullsized":"2023-11-08T16:41:06.292899+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:24.472729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.263176+0000","last_change":"2023-11-08T16:41:05.281320+0000","last_active":"2023-11-08T16:41:06.263176+0000","last_peered":"2023-11-08T16:41:06.263176+0000","last_clean":"2023-11-08T16:41:06.263176+0000","last_became_active":"2023-11-08T16:41:05.281003+0000","last_became_peered":"2023-11-08T16:41:05.281003+0000","last_unstale":"2023-11-08T16:41:06.263176+0000","last_undegraded":"2023-11-08T16:41:06.263176+0000","last_fullsized":"2023-11-08T16:41:06.263176+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:00:13.379826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.292886+0000","last_change":"2023-11-08T16:41:06.290606+0000","last_active":"2023-11-08T16:41:06.292886+0000","last_peered":"2023-11-08T16:41:06.292886+0000","last_clean":"2023-11-08T16:41:06.292886+0000","last_became_active":"2023-11-08T16:41:06.286415+0000","last_became_peered":"2023-11-08T16:41:06.286415+0000","last_unstale":"2023-11-08T16:41:06.292886+0000","last_undegraded":"2023-11-08T16:41:06.292886+0000","last_fullsized":"2023-11-08T16:41:06.292886+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:58:55.980017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290575+0000","last_change":"2023-11-08T16:41:05.276394+0000","last_active":"2023-11-08T16:41:06.290575+0000","last_peered":"2023-11-08T16:41:06.290575+0000","last_clean":"2023-11-08T16:41:06.290575+0000","last_became_active":"2023-11-08T16:41:05.275441+0000","last_became_peered":"2023-11-08T16:41:05.275441+0000","last_unstale":"2023-11-08T16:41:06.290575+0000","last_undegraded":"2023-11-08T16:41:06.290575+0000","last_fullsized":"2023-11-08T16:41:06.290575+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:57.399931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.303003+0000","last_change":"2023-11-08T16:41:06.300043+0000","last_active":"2023-11-08T16:41:06.303003+0000","last_peered":"2023-11-08T16:41:06.303003+0000","last_clean":"2023-11-08T16:41:06.303003+0000","last_became_active":"2023-11-08T16:41:06.298286+0000","last_became_peered":"2023-11-08T16:41:06.298286+0000","last_unstale":"2023-11-08T16:41:06.303003+0000","last_undegraded":"2023-11-08T16:41:06.303003+0000","last_fullsized":"2023-11-08T16:41:06.303003+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:02.726372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.264947+0000","last_change":"2023-11-08T16:41:05.267580+0000","last_active":"2023-11-08T16:41:06.264947+0000","last_peered":"2023-11-08T16:41:06.264947+0000","last_clean":"2023-11-08T16:41:06.264947+0000","last_became_active":"2023-11-08T16:41:05.267292+0000","last_became_peered":"2023-11-08T16:41:05.267292+0000","last_unstale":"2023-11-08T16:41:06.264947+0000","last_undegraded":"2023-11-08T16:41:06.264947+0000","last_fullsized":"2023-11-08T16:41:06.264947+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:07:24.245421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291198+0000","last_change":"2023-11-08T16:41:06.287414+0000","last_active":"2023-11-08T16:41:06.291198+0000","last_peered":"2023-11-08T16:41:06.291198+0000","last_clean":"2023-11-08T16:41:06.291198+0000","last_became_active":"2023-11-08T16:41:06.286596+0000","last_became_peered":"2023-11-08T16:41:06.286596+0000","last_unstale":"2023-11-08T16:41:06.291198+0000","last_undegraded":"2023-11-08T16:41:06.291198+0000","last_fullsized":"2023-11-08T16:41:06.291198+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:07:10.051108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.265043+0000","last_change":"2023-11-08T16:41:05.277292+0000","last_active":"2023-11-08T16:41:06.265043+0000","last_peered":"2023-11-08T16:41:06.265043+0000","last_clean":"2023-11-08T16:41:06.265043+0000","last_became_active":"2023-11-08T16:41:05.276983+0000","last_became_peered":"2023-11-08T16:41:05.276983+0000","last_unstale":"2023-11-08T16:41:06.265043+0000","last_undegraded":"2023-11-08T16:41:06.265043+0000","last_fullsized":"2023-11-08T16:41:06.265043+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:39:20.555676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.296799+0000","last_change":"2023-11-08T16:41:06.295080+0000","last_active":"2023-11-08T16:41:06.296799+0000","last_peered":"2023-11-08T16:41:06.296799+0000","last_clean":"2023-11-08T16:41:06.296799+0000","last_became_active":"2023-11-08T16:41:06.294796+0000","last_became_peered":"2023-11-08T16:41:06.294796+0000","last_unstale":"2023-11-08T16:41:06.296799+0000","last_undegraded":"2023-11-08T16:41:06.296799+0000","last_fullsized":"2023-11-08T16:41:06.296799+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:24.472729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.285130+0000","last_change":"2023-11-08T16:41:05.282283+0000","last_active":"2023-11-08T16:41:06.285130+0000","last_peered":"2023-11-08T16:41:06.285130+0000","last_clean":"2023-11-08T16:41:06.285130+0000","last_became_active":"2023-11-08T16:41:05.282009+0000","last_became_peered":"2023-11-08T16:41:05.282009+0000","last_unstale":"2023-11-08T16:41:06.285130+0000","last_undegraded":"2023-11-08T16:41:06.285130+0000","last_fullsized":"2023-11-08T16:41:06.285130+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:10:49.216473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.294430+0000","last_change":"2023-11-08T16:41:06.293933+0000","last_active":"2023-11-08T16:41:06.294430+0000","last_peered":"2023-11-08T16:41:06.294430+0000","last_clean":"2023-11-08T16:41:06.294430+0000","last_became_active":"2023-11-08T16:41:06.293031+0000","last_became_peered":"2023-11-08T16:41:06.293031+0000","last_unstale":"2023-11-08T16:41:06.294430+0000","last_undegraded":"2023-11-08T16:41:06.294430+0000","last_fullsized":"2023-11-08T16:41:06.294430+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:58:55.980017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.299821+0000","last_change":"2023-11-08T16:41:05.279231+0000","last_active":"2023-11-08T16:41:06.299821+0000","last_peered":"2023-11-08T16:41:06.299821+0000","last_clean":"2023-11-08T16:41:06.299821+0000","last_became_active":"2023-11-08T16:41:05.278993+0000","last_became_peered":"2023-11-08T16:41:05.278993+0000","last_unstale":"2023-11-08T16:41:06.299821+0000","last_undegraded":"2023-11-08T16:41:06.299821+0000","last_fullsized":"2023-11-08T16:41:06.299821+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:47:40.178540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.301426+0000","last_change":"2023-11-08T16:41:06.294046+0000","last_active":"2023-11-08T16:41:06.301426+0000","last_peered":"2023-11-08T16:41:06.301426+0000","last_clean":"2023-11-08T16:41:06.301426+0000","last_became_active":"2023-11-08T16:41:06.293365+0000","last_became_peered":"2023-11-08T16:41:06.293365+0000","last_unstale":"2023-11-08T16:41:06.301426+0000","last_undegraded":"2023-11-08T16:41:06.301426+0000","last_fullsized":"2023-11-08T16:41:06.301426+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:27:51.816735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.258583+0000","last_change":"2023-11-08T16:41:05.277465+0000","last_active":"2023-11-08T16:41:06.258583+0000","last_peered":"2023-11-08T16:41:06.258583+0000","last_clean":"2023-11-08T16:41:06.258583+0000","last_became_active":"2023-11-08T16:41:05.277059+0000","last_became_peered":"2023-11-08T16:41:05.277059+0000","last_unstale":"2023-11-08T16:41:06.258583+0000","last_undegraded":"2023-11-08T16:41:06.258583+0000","last_fullsized":"2023-11-08T16:41:06.258583+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:33.375161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.300954+0000","last_change":"2023-11-08T16:41:06.291247+0000","last_active":"2023-11-08T16:41:06.300954+0000","last_peered":"2023-11-08T16:41:06.300954+0000","last_clean":"2023-11-08T16:41:06.300954+0000","last_became_active":"2023-11-08T16:41:06.290304+0000","last_became_peered":"2023-11-08T16:41:06.290304+0000","last_unstale":"2023-11-08T16:41:06.300954+0000","last_undegraded":"2023-11-08T16:41:06.300954+0000","last_fullsized":"2023-11-08T16:41:06.300954+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:59:27.952350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.297070+0000","last_change":"2023-11-08T16:41:05.281703+0000","last_active":"2023-11-08T16:41:06.297070+0000","last_peered":"2023-11-08T16:41:06.297070+0000","last_clean":"2023-11-08T16:41:06.297070+0000","last_became_active":"2023-11-08T16:41:05.281137+0000","last_became_peered":"2023-11-08T16:41:05.281137+0000","last_unstale":"2023-11-08T16:41:06.297070+0000","last_undegraded":"2023-11-08T16:41:06.297070+0000","last_fullsized":"2023-11-08T16:41:06.297070+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:49:59.086294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.302778+0000","last_change":"2023-11-08T16:41:06.301967+0000","last_active":"2023-11-08T16:41:06.302778+0000","last_peered":"2023-11-08T16:41:06.302778+0000","last_clean":"2023-11-08T16:41:06.302778+0000","last_became_active":"2023-11-08T16:41:06.301744+0000","last_became_peered":"2023-11-08T16:41:06.301744+0000","last_unstale":"2023-11-08T16:41:06.302778+0000","last_undegraded":"2023-11-08T16:41:06.302778+0000","last_fullsized":"2023-11-08T16:41:06.302778+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:07:13.421490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.264954+0000","last_change":"2023-11-08T16:41:05.277639+0000","last_active":"2023-11-08T16:41:06.264954+0000","last_peered":"2023-11-08T16:41:06.264954+0000","last_clean":"2023-11-08T16:41:06.264954+0000","last_became_active":"2023-11-08T16:41:05.277362+0000","last_became_peered":"2023-11-08T16:41:05.277362+0000","last_unstale":"2023-11-08T16:41:06.264954+0000","last_undegraded":"2023-11-08T16:41:06.264954+0000","last_fullsized":"2023-11-08T16:41:06.264954+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:24:51.860834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.287649+0000","last_change":"2023-11-08T16:41:06.281767+0000","last_active":"2023-11-08T16:41:06.287649+0000","last_peered":"2023-11-08T16:41:06.287649+0000","last_clean":"2023-11-08T16:41:06.287649+0000","last_became_active":"2023-11-08T16:41:06.281422+0000","last_became_peered":"2023-11-08T16:41:06.281422+0000","last_unstale":"2023-11-08T16:41:06.287649+0000","last_undegraded":"2023-11-08T16:41:06.287649+0000","last_fullsized":"2023-11-08T16:41:06.287649+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:24.472729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.262980+0000","last_change":"2023-11-08T16:41:05.279443+0000","last_active":"2023-11-08T16:41:06.262980+0000","last_peered":"2023-11-08T16:41:06.262980+0000","last_clean":"2023-11-08T16:41:06.262980+0000","last_became_active":"2023-11-08T16:41:05.278965+0000","last_became_peered":"2023-11-08T16:41:05.278965+0000","last_unstale":"2023-11-08T16:41:06.262980+0000","last_undegraded":"2023-11-08T16:41:06.262980+0000","last_fullsized":"2023-11-08T16:41:06.262980+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:39:02.918253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290759+0000","last_change":"2023-11-08T16:41:06.286651+0000","last_active":"2023-11-08T16:41:06.290759+0000","last_peered":"2023-11-08T16:41:06.290759+0000","last_clean":"2023-11-08T16:41:06.290759+0000","last_became_active":"2023-11-08T16:41:06.286166+0000","last_became_peered":"2023-11-08T16:41:06.286166+0000","last_unstale":"2023-11-08T16:41:06.290759+0000","last_undegraded":"2023-11-08T16:41:06.290759+0000","last_fullsized":"2023-11-08T16:41:06.290759+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:24:52.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.274651+0000","last_change":"2023-11-08T16:41:05.276633+0000","last_active":"2023-11-08T16:41:06.274651+0000","last_peered":"2023-11-08T16:41:06.274651+0000","last_clean":"2023-11-08T16:41:06.274651+0000","last_became_active":"2023-11-08T16:41:05.276312+0000","last_became_peered":"2023-11-08T16:41:05.276312+0000","last_unstale":"2023-11-08T16:41:06.274651+0000","last_undegraded":"2023-11-08T16:41:06.274651+0000","last_fullsized":"2023-11-08T16:41:06.274651+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:31.955399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291215+0000","last_change":"2023-11-08T16:41:06.287582+0000","last_active":"2023-11-08T16:41:06.291215+0000","last_peered":"2023-11-08T16:41:06.291215+0000","last_clean":"2023-11-08T16:41:06.291215+0000","last_became_active":"2023-11-08T16:41:06.286681+0000","last_became_peered":"2023-11-08T16:41:06.286681+0000","last_unstale":"2023-11-08T16:41:06.291215+0000","last_undegraded":"2023-11-08T16:41:06.291215+0000","last_fullsized":"2023-11-08T16:41:06.291215+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:39:32.914205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.264852+0000","last_change":"2023-11-08T16:41:05.266581+0000","last_active":"2023-11-08T16:41:06.264852+0000","last_peered":"2023-11-08T16:41:06.264852+0000","last_clean":"2023-11-08T16:41:06.264852+0000","last_became_active":"2023-11-08T16:41:05.266299+0000","last_became_peered":"2023-11-08T16:41:05.266299+0000","last_unstale":"2023-11-08T16:41:06.264852+0000","last_undegraded":"2023-11-08T16:41:06.264852+0000","last_fullsized":"2023-11-08T16:41:06.264852+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:18:33.315259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.302652+0000","last_change":"2023-11-08T16:41:06.298027+0000","last_active":"2023-11-08T16:41:06.302652+0000","last_peered":"2023-11-08T16:41:06.302652+0000","last_clean":"2023-11-08T16:41:06.302652+0000","last_became_active":"2023-11-08T16:41:06.297229+0000","last_became_peered":"2023-11-08T16:41:06.297229+0000","last_unstale":"2023-11-08T16:41:06.302652+0000","last_undegraded":"2023-11-08T16:41:06.302652+0000","last_fullsized":"2023-11-08T16:41:06.302652+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:55:11.606257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.285725+0000","last_change":"2023-11-08T16:41:05.282519+0000","last_active":"2023-11-08T16:41:06.285725+0000","last_peered":"2023-11-08T16:41:06.285725+0000","last_clean":"2023-11-08T16:41:06.285725+0000","last_became_active":"2023-11-08T16:41:05.282239+0000","last_became_peered":"2023-11-08T16:41:05.282239+0000","last_unstale":"2023-11-08T16:41:06.285725+0000","last_undegraded":"2023-11-08T16:41:06.285725+0000","last_fullsized":"2023-11-08T16:41:06.285725+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:39.031870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290144+0000","last_change":"2023-11-08T16:41:06.283632+0000","last_active":"2023-11-08T16:41:06.290144+0000","last_peered":"2023-11-08T16:41:06.290144+0000","last_clean":"2023-11-08T16:41:06.290144+0000","last_became_active":"2023-11-08T16:41:06.283387+0000","last_became_peered":"2023-11-08T16:41:06.283387+0000","last_unstale":"2023-11-08T16:41:06.290144+0000","last_undegraded":"2023-11-08T16:41:06.290144+0000","last_fullsized":"2023-11-08T16:41:06.290144+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:16:55.434687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.264756+0000","last_change":"2023-11-08T16:41:05.274649+0000","last_active":"2023-11-08T16:41:06.264756+0000","last_peered":"2023-11-08T16:41:06.264756+0000","last_clean":"2023-11-08T16:41:06.264756+0000","last_became_active":"2023-11-08T16:41:05.274349+0000","last_became_peered":"2023-11-08T16:41:05.274349+0000","last_unstale":"2023-11-08T16:41:06.264756+0000","last_undegraded":"2023-11-08T16:41:06.264756+0000","last_fullsized":"2023-11-08T16:41:06.264756+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:07:12.405014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":276,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-08T16:41:05.266078+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:41:05.266078+0000","last_peered":"2023-11-08T16:41:05.266078+0000","last_clean":"2023-11-08T16:41:05.266078+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:41:05.266078+0000","last_undegraded":"2023-11-08T16:41:05.266078+0000","last_fullsized":"2023-11-08T16:41:05.266078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1310550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.0429999999999999,"15min":1.0740000000000001},"min":{"1min":0.60699999999999998,"5min":0.56399999999999995,"15min":0.49299999999999999},"max":{"1min":1.5600000000000001,"5min":1.6879999999999999,"15min":3.1899999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.032,"5min":1.0760000000000001,"15min":1.105},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.53600000000000003},"max":{"1min":1.6040000000000001,"5min":1.7410000000000001,"15min":4.2729999999999997},"last":0.874}]},{"osd":1,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.107,"15min":1.1160000000000001},"min":{"1min":0.81899999999999995,"5min":0.61599999999999999,"15min":0.57899999999999996},"max":{"1min":1.5580000000000001,"5min":1.891,"15min":1.891},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1879999999999999,"15min":1.204},"min":{"1min":0.61199999999999999,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6020000000000001,"5min":2.1419999999999999,"15min":3.3740000000000001},"last":1.0169999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.159,"15min":1.2190000000000001},"min":{"1min":0.79800000000000004,"5min":0.65900000000000003,"15min":0.58399999999999996},"max":{"1min":1.4159999999999999,"5min":1.8109999999999999,"15min":4.2249999999999996},"last":1.21},{"interface":"front","average":{"1min":1.246,"5min":1.2450000000000001,"15min":1.222},"min":{"1min":0.85599999999999998,"5min":0.66000000000000003,"15min":0.56299999999999994},"max":{"1min":1.6990000000000001,"5min":1.7010000000000001,"15min":3.4750000000000001},"last":1.0800000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1539999999999999,"15min":1.167},"min":{"1min":0.79800000000000004,"5min":0.70299999999999996,"15min":0.54900000000000004},"max":{"1min":1.631,"5min":1.631,"15min":2.964},"last":1.571},{"interface":"front","average":{"1min":1.351,"5min":1.365,"15min":1.4139999999999999},"min":{"1min":0.88700000000000001,"5min":0.86899999999999999,"15min":0.58599999999999997},"max":{"1min":1.698,"5min":2.25,"15min":4.181},"last":1.3260000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.206,"15min":1.2290000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.77200000000000002},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":3.6669999999999998},"last":1.778},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.325,"15min":1.3360000000000001},"min":{"1min":0.95499999999999996,"5min":0.77300000000000002,"15min":0.59099999999999997},"max":{"1min":1.865,"5min":2.0179999999999998,"15min":2.3730000000000002},"last":1.3600000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.3919999999999999,"15min":1.3759999999999999},"min":{"1min":0.96399999999999997,"5min":0.89200000000000002,"15min":0.61799999999999999},"max":{"1min":1.885,"5min":2.3559999999999999,"15min":3.1059999999999999},"last":1.8859999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4530000000000001,"15min":1.4550000000000001},"min":{"1min":1.1399999999999999,"5min":1.0589999999999999,"15min":0.878},"max":{"1min":1.6870000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.462}]},{"osd":6,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.526,"15min":1.5309999999999999},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":0.94799999999999995},"max":{"1min":1.8600000000000001,"5min":1.919,"15min":3.7530000000000001},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.4610000000000001,"15min":1.482},"min":{"1min":1.2310000000000001,"5min":0.89900000000000002,"15min":0.73999999999999999},"max":{"1min":1.786,"5min":2.0430000000000001,"15min":6.9560000000000004},"last":1.831}]}]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":720182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0609999999999999,"15min":1.0629999999999999},"min":{"1min":0.69799999999999995,"5min":0.58599999999999997,"15min":0.497},"max":{"1min":1.6100000000000001,"5min":1.7230000000000001,"15min":2.5609999999999999},"last":1.569},{"interface":"front","average":{"1min":1.218,"5min":1.133,"15min":1.095},"min":{"1min":0.75800000000000001,"5min":0.63100000000000001,"15min":0.56200000000000006},"max":{"1min":1.6279999999999999,"5min":1.6819999999999999,"15min":2.7519999999999998},"last":1.1479999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1259999999999999,"15min":1.079},"min":{"1min":0.72299999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5880000000000001,"5min":1.667,"15min":1.667},"last":1.097},{"interface":"front","average":{"1min":1.194,"5min":1.143,"15min":1.131},"min":{"1min":0.92200000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.714,"5min":1.7290000000000001,"15min":1.97},"last":1.5049999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.181,"15min":1.1830000000000001},"min":{"1min":0.77000000000000002,"5min":0.71599999999999997,"15min":0.56799999999999995},"max":{"1min":1.6040000000000001,"5min":1.698,"15min":1.804},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.2030000000000001,"15min":1.171},"min":{"1min":0.73399999999999999,"5min":0.72299999999999998,"15min":0.60499999999999998},"max":{"1min":1.6259999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.607}]},{"osd":3,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.224,"15min":1.2450000000000001},"min":{"1min":0.87,"5min":0.63200000000000001,"15min":0.58299999999999996},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":2.1120000000000001},"last":1.282},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2809999999999999,"15min":1.288},"min":{"1min":0.70899999999999996,"5min":0.68200000000000005,"15min":0.67300000000000004},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.95},"last":1.47}]},{"osd":4,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.321,"15min":1.3400000000000001},"min":{"1min":1.143,"5min":0.81799999999999995,"15min":0.65300000000000002},"max":{"1min":1.657,"5min":1.778,"15min":2.218},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.367,"15min":1.3979999999999999},"min":{"1min":0.878,"5min":0.86899999999999999,"15min":0.64900000000000002},"max":{"1min":1.681,"5min":1.9590000000000001,"15min":2.7080000000000002},"last":1.331}]},{"osd":5,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3380000000000001,"15min":1.3240000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.80500000000000005},"max":{"1min":1.8200000000000001,"5min":1.994,"15min":2.105},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.548,"5min":1.512,"15min":1.534},"min":{"1min":0.98599999999999999,"5min":0.91100000000000003,"15min":0.78500000000000003},"max":{"1min":1.833,"5min":2.1280000000000001,"15min":2.4700000000000002},"last":1.6599999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.51,"15min":1.5600000000000001},"min":{"1min":1.157,"5min":1.002,"15min":0.74299999999999999},"max":{"1min":1.95,"5min":2.1309999999999998,"15min":3.8959999999999999},"last":1.536},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.504,"15min":1.5049999999999999},"min":{"1min":1.105,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.843,"5min":2.0870000000000002,"15min":2.4889999999999999},"last":1.4319999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1572864,"data_stored":727561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.089,"15min":1.0720000000000001},"min":{"1min":0.63400000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.536,"5min":1.639,"15min":2.4609999999999999},"last":1.248},{"interface":"front","average":{"1min":1.125,"5min":1.127,"15min":1.1120000000000001},"min":{"1min":0.71499999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.4930000000000001,"5min":1.6000000000000001,"15min":2.3639999999999999},"last":1.474}]},{"osd":1,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.244,"15min":1.238},"min":{"1min":0.89900000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.609,"5min":1.9610000000000001,"15min":3.8690000000000002},"last":1.554},{"interface":"front","average":{"1min":1.375,"5min":1.238,"15min":1.2509999999999999},"min":{"1min":0.68899999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.8109999999999999,"5min":1.9370000000000001,"15min":3.2349999999999999},"last":1.423}]},{"osd":2,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1499999999999999,"15min":1.1930000000000001},"min":{"1min":0.753,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6499999999999999,"5min":1.764,"15min":2.6339999999999999},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.206,"15min":1.2},"min":{"1min":0.78300000000000003,"5min":0.70499999999999996,"15min":0.65800000000000003},"max":{"1min":1.752,"5min":1.752,"15min":2.2559999999999998},"last":0.89100000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2809999999999999,"15min":1.2789999999999999},"min":{"1min":0.97999999999999998,"5min":0.90900000000000003,"15min":0.66500000000000004},"max":{"1min":1.575,"5min":1.681,"15min":2.8330000000000002},"last":1.123},{"interface":"front","average":{"1min":1.306,"5min":1.3140000000000001,"15min":1.2969999999999999},"min":{"1min":1.0569999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.5549999999999999,"5min":2.0470000000000002,"15min":2.5760000000000001},"last":1.363}]},{"osd":4,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.2869999999999999,"15min":1.325},"min":{"1min":0.94899999999999995,"5min":0.872,"15min":0.61599999999999999},"max":{"1min":2.0019999999999998,"5min":2.0790000000000002,"15min":3.9790000000000001},"last":1.657},{"interface":"front","average":{"1min":1.325,"5min":1.401,"15min":1.4370000000000001},"min":{"1min":1.0680000000000001,"5min":1.0569999999999999,"15min":0.81499999999999995},"max":{"1min":1.696,"5min":2.0230000000000001,"15min":3.0209999999999999},"last":1.579}]},{"osd":6,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4339999999999999,"15min":1.415},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.752},"max":{"1min":1.9379999999999999,"5min":2.1560000000000001,"15min":2.5550000000000002},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.425,"15min":1.464},"min":{"1min":1.012,"5min":0.98499999999999999,"15min":0.93600000000000005},"max":{"1min":1.9419999999999999,"5min":2.1619999999999999,"15min":4.1189999999999998},"last":1.514}]},{"osd":7,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5229999999999999,"15min":1.546},"min":{"1min":1.21,"5min":1.21,"15min":1.0049999999999999},"max":{"1min":1.792,"5min":2.129,"15min":3.1160000000000001},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.5760000000000001,"15min":1.5549999999999999},"min":{"1min":1.075,"5min":1.075,"15min":0.96999999999999997},"max":{"1min":1.8580000000000001,"5min":2.3660000000000001,"15min":2.7050000000000001},"last":1.6910000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":726957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.73799999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4830000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.997},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.127,"15min":1.087},"min":{"1min":0.69599999999999995,"5min":0.58299999999999996,"15min":0.45400000000000001},"max":{"1min":1.591,"5min":1.772,"15min":1.772},"last":1.278}]},{"osd":1,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.107,"15min":1.123},"min":{"1min":0.624,"5min":0.60099999999999998,"15min":0.56999999999999995},"max":{"1min":1.3819999999999999,"5min":1.51,"15min":2.0459999999999998},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1559999999999999,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.577,"5min":1.577,"15min":1.742},"last":1.099}]},{"osd":2,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.236,"15min":1.212},"min":{"1min":0.85899999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.6930000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.226,"5min":1.214,"15min":1.246},"min":{"1min":0.877,"5min":0.78100000000000003,"15min":0.71699999999999997},"max":{"1min":1.6659999999999999,"5min":1.7749999999999999,"15min":1.9319999999999999},"last":1.3220000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.306,"15min":1.2829999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.911,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.2589999999999999,"15min":1.274},"min":{"1min":0.94899999999999995,"5min":0.72499999999999998,"15min":0.65100000000000002},"max":{"1min":1.611,"5min":1.8069999999999999,"15min":1.895},"last":1.2310000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3640000000000001,"15min":1.363},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.58799999999999997},"max":{"1min":1.698,"5min":2.286,"15min":2.286},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.472,"15min":1.4339999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.61899999999999999},"max":{"1min":1.821,"5min":2.1970000000000001,"15min":2.46},"last":1.538}]},{"osd":6,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.377,"15min":1.395},"min":{"1min":1.0740000000000001,"5min":0.88,"15min":0.74299999999999999},"max":{"1min":1.75,"5min":1.7769999999999999,"15min":2.5609999999999999},"last":1.593},{"interface":"front","average":{"1min":1.478,"5min":1.5029999999999999,"15min":1.53},"min":{"1min":1.038,"5min":1.032,"15min":0.81200000000000006},"max":{"1min":1.873,"5min":2.4780000000000002,"15min":2.6389999999999998},"last":1.833}]},{"osd":7,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.556,"15min":1.534},"min":{"1min":1.272,"5min":1.056,"15min":0.92300000000000004},"max":{"1min":1.9750000000000001,"5min":2.1040000000000001,"15min":2.2639999999999998},"last":1.649},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.542,"15min":1.5669999999999999},"min":{"1min":1.25,"5min":1.117,"15min":0.84799999999999998},"max":{"1min":1.847,"5min":2.1070000000000002,"15min":2.6859999999999999},"last":1.718}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":720182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82099999999999995,"15min":0.81399999999999995},"min":{"1min":0.45200000000000001,"5min":0.436,"15min":0.33800000000000002},"max":{"1min":1.304,"5min":1.4930000000000001,"15min":1.9339999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.48899999999999999,"5min":0.45000000000000001,"15min":0.316},"max":{"1min":1.486,"5min":1.647,"15min":1.647},"last":0.58999999999999997}]},{"osd":1,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90100000000000002,"15min":0.89100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36499999999999999},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.97699999999999998,"15min":0.97099999999999997},"min":{"1min":0.52500000000000002,"5min":0.48299999999999998,"15min":0.378},"max":{"1min":1.4610000000000001,"5min":1.5800000000000001,"15min":1.829},"last":0.749}]},{"osd":2,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.046,"15min":1.008},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.41699999999999998},"max":{"1min":1.6479999999999999,"5min":1.7490000000000001,"15min":2.0779999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.087,"15min":1.1120000000000001},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.595,"5min":1.7270000000000001,"15min":2.4660000000000002},"last":1.095}]},{"osd":4,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.161,"15min":1.1779999999999999},"min":{"1min":0.90700000000000003,"5min":0.73199999999999998,"15min":0.56399999999999995},"max":{"1min":1.399,"5min":1.5720000000000001,"15min":2.25},"last":1.129},{"interface":"front","average":{"1min":1.25,"5min":1.2829999999999999,"15min":1.27},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.823,"5min":2.0110000000000001,"15min":2.1829999999999998},"last":1.008}]},{"osd":5,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3440000000000001,"15min":1.329},"min":{"1min":0.94299999999999995,"5min":0.88700000000000001,"15min":0.68799999999999994},"max":{"1min":1.8109999999999999,"5min":2.2229999999999999,"15min":2.3999999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.278,"5min":1.3280000000000001,"15min":1.331},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.60999999999999999},"max":{"1min":1.887,"5min":1.887,"15min":2.593},"last":1.377}]},{"osd":6,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3839999999999999,"15min":1.3819999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.58799999999999997},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":2.9569999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.323,"15min":1.3500000000000001},"min":{"1min":0.86099999999999999,"5min":0.83599999999999997,"15min":0.73399999999999999},"max":{"1min":1.6910000000000001,"5min":1.8919999999999999,"15min":2.762},"last":1.226}]},{"osd":7,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.359,"15min":1.3919999999999999},"min":{"1min":1.0720000000000001,"5min":0.73699999999999999,"15min":0.70799999999999996},"max":{"1min":1.6799999999999999,"5min":1.764,"15min":2.8700000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.425,"15min":1.419},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.73399999999999999},"max":{"1min":2.0470000000000002,"5min":2.121,"15min":2.633},"last":0.98299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":726957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79800000000000004,"15min":0.91400000000000003},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.24099999999999999},"max":{"1min":1.2290000000000001,"5min":1.2929999999999999,"15min":4.5369999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.872,"15min":0.876},"min":{"1min":0.50900000000000001,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":1.4910000000000001,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.69199999999999995}]},{"osd":1,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.69699999999999995,"5min":0.44900000000000001,"15min":0.379},"max":{"1min":1.4950000000000001,"5min":1.7689999999999999,"15min":4.4980000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.091,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.40300000000000002},"max":{"1min":1.5740000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.91900000000000004}]},{"osd":3,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.878,"15min":0.96399999999999997},"min":{"1min":0.44800000000000001,"5min":0.437,"15min":0.35499999999999998},"max":{"1min":1.4630000000000001,"5min":1.4870000000000001,"15min":3.3050000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.004,"5min":1.0529999999999999,"15min":1.147},"min":{"1min":0.54700000000000004,"5min":0.44800000000000001,"15min":0.42999999999999999},"max":{"1min":1.3109999999999999,"5min":1.819,"15min":4.5899999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.2829999999999999,"15min":1.351},"min":{"1min":0.77200000000000002,"5min":0.74299999999999999,"15min":0.73499999999999999},"max":{"1min":1.6890000000000001,"5min":2.2160000000000002,"15min":4.9409999999999998},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.208,"15min":1.246},"min":{"1min":0.86499999999999999,"5min":0.73899999999999999,"15min":0.63100000000000001},"max":{"1min":1.508,"5min":2.0739999999999998,"15min":2.46},"last":1.2649999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2829999999999999,"15min":1.294},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.77200000000000002},"max":{"1min":1.6040000000000001,"5min":1.9510000000000001,"15min":2.8530000000000002},"last":1.228},{"interface":"front","average":{"1min":1.276,"5min":1.2709999999999999,"15min":1.3129999999999999},"min":{"1min":0.80500000000000005,"5min":0.70499999999999996,"15min":0.69399999999999995},"max":{"1min":1.8380000000000001,"5min":2.5390000000000001,"15min":3.5680000000000001},"last":1.1599999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.284,"15min":1.345},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.625},"max":{"1min":1.7230000000000001,"5min":1.9710000000000001,"15min":3.536},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.389,"15min":1.4510000000000001},"min":{"1min":0.95699999999999996,"5min":0.81200000000000006,"15min":0.72199999999999998},"max":{"1min":1.629,"5min":2.2719999999999998,"15min":4.7549999999999999},"last":1.2470000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3480000000000001,"15min":1.397},"min":{"1min":0.98199999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6439999999999999,"5min":2.2930000000000001,"15min":2.9700000000000002},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3779999999999999,"15min":1.4410000000000001},"min":{"1min":0.91400000000000003,"5min":0.79600000000000004,"15min":0.70499999999999996},"max":{"1min":1.7789999999999999,"5min":2.129,"15min":3.4950000000000001},"last":1.123}]}]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":720182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76900000000000002,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.34499999999999997,"15min":0.28899999999999998},"max":{"1min":1.246,"5min":1.246,"15min":1.542},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.877,"15min":0.95199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":4.0309999999999997},"last":0.45600000000000002}]},{"osd":2,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.999,"15min":1.042},"min":{"1min":0.57899999999999996,"5min":0.48899999999999999,"15min":0.45000000000000001},"max":{"1min":1.4159999999999999,"5min":1.7689999999999999,"15min":3.0339999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.94099999999999995,"15min":1.0129999999999999},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":1.413,"5min":1.542,"15min":2.6840000000000002},"last":0.72399999999999998}]},{"osd":3,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.98999999999999999,"15min":1.054},"min":{"1min":0.50600000000000001,"5min":0.47199999999999998,"15min":0.38900000000000001},"max":{"1min":1.488,"5min":1.5129999999999999,"15min":4.3179999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96699999999999997,"15min":1.0600000000000001},"min":{"1min":0.496,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.585,"5min":1.5900000000000001,"15min":8.5570000000000004},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.163,"15min":1.2450000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.63300000000000001},"max":{"1min":1.5549999999999999,"5min":1.907,"15min":3.1659999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.218,"15min":1.2869999999999999},"min":{"1min":0.72099999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.589,"5min":1.823,"15min":2.734},"last":1.1859999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2569999999999999,"15min":1.4690000000000001},"min":{"1min":0.85399999999999998,"5min":0.72399999999999998,"15min":0.67100000000000004},"max":{"1min":1.619,"5min":1.921,"15min":8.4730000000000008},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.246,"5min":1.2589999999999999,"15min":1.458},"min":{"1min":0.78600000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":8.8040000000000003},"last":1.2030000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2949999999999999,"15min":1.385},"min":{"1min":0.80100000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.508,"5min":1.9119999999999999,"15min":3.984},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3129999999999999,"15min":1.3919999999999999},"min":{"1min":0.74299999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0139999999999998,"5min":2.0449999999999999,"15min":3.2450000000000001},"last":1.3580000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3,"15min":1.4299999999999999},"min":{"1min":0.90000000000000002,"5min":0.80700000000000005,"15min":0.69199999999999995},"max":{"1min":1.7150000000000001,"5min":1.853,"15min":4.3609999999999998},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.373,"15min":1.5069999999999999},"min":{"1min":0.76400000000000001,"5min":0.72799999999999998,"15min":0.70499999999999996},"max":{"1min":1.8360000000000001,"5min":1.972,"15min":5.3840000000000003},"last":1.155}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2179072,"data_stored":1319049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.84099999999999997,"15min":0.81699999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.29399999999999998},"max":{"1min":1.248,"5min":1.3680000000000001,"15min":1.6220000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.96099999999999997,"15min":0.96299999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.492,"5min":1.722,"15min":2.3340000000000001},"last":0.96599999999999997}]},{"osd":2,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.91700000000000004,"15min":0.92000000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.37},"max":{"1min":1.087,"5min":1.6910000000000001,"15min":1.762},"last":1.05},{"interface":"front","average":{"1min":0.87,"5min":0.91300000000000003,"15min":0.96999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.36799999999999999},"max":{"1min":1.3029999999999999,"5min":1.613,"15min":1.879},"last":1.0700000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.91000000000000003,"15min":0.93200000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.34200000000000003},"max":{"1min":1.5129999999999999,"5min":1.72,"15min":1.728},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.079,"5min":1.165,"15min":1.1759999999999999},"min":{"1min":0.498,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.544,"5min":1.8740000000000001,"15min":2.7679999999999998},"last":1.1080000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1679999999999999,"15min":1.1579999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.65300000000000002},"max":{"1min":1.6359999999999999,"5min":2.0489999999999999,"15min":2.2559999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.2509999999999999,"15min":1.274},"min":{"1min":0.754,"5min":0.69999999999999996,"15min":0.59399999999999997},"max":{"1min":1.5669999999999999,"5min":1.8859999999999999,"15min":2.867},"last":1.1599999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2509999999999999,"15min":1.26},"min":{"1min":0.81699999999999995,"5min":0.79600000000000004,"15min":0.67900000000000005},"max":{"1min":1.6439999999999999,"5min":1.98,"15min":2.7280000000000002},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.214,"15min":1.2270000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.67600000000000005},"max":{"1min":1.5580000000000001,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.5349999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.353,"15min":1.3520000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.72499999999999998},"max":{"1min":1.6359999999999999,"5min":1.9399999999999999,"15min":2.6739999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.302,"15min":1.3460000000000001},"min":{"1min":0.89700000000000002,"5min":0.88200000000000001,"15min":0.64200000000000002},"max":{"1min":1.663,"5min":2.1280000000000001,"15min":2.8260000000000001},"last":1.7490000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.363,"15min":1.3979999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.6699999999999999,"5min":2.0150000000000001,"15min":4.5670000000000002},"last":1.8660000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.335,"15min":1.3520000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.748,"5min":2.6070000000000002,"15min":5.4450000000000003},"last":1.6080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:41:10.701 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:41:10.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.952+0000 7f6e68ec2640 1 Processor -- start 2023-11-08T16:41:10.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.952+0000 7f6e68ec2640 1 -- start start 2023-11-08T16:41:10.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.954+0000 7f6e68ec2640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6e640ca480 0x7f6e6415ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.954+0000 7f6e68ec2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6406b630 con 0x7f6e640ade50 2023-11-08T16:41:10.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.954+0000 7f6e68ec2640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e640ca910 con 0x7f6e6415ea60 2023-11-08T16:41:10.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.954+0000 7f6e68ec2640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e640b1860 con 0x7f6e640ca480 2023-11-08T16:41:10.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.954+0000 7f6e6259c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f6e640ade50 0x7f6e6415a680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39486/0 (socket says 172.21.15.81:39486) 2023-11-08T16:41:10.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.954+0000 7f6e6259c640 1 -- 172.21.15.81:0/3408727953 learned_addr learned my addr 172.21.15.81:0/3408727953 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:10.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.954+0000 7f6e61d9b640 1 --2- 172.21.15.81:0/3408727953 >> v2:172.21.15.93:3300/0 conn(0x7f6e640ca480 0x7f6e6415ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.954+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970759548 0 0) 0x7f6e6406b630 con 0x7f6e640ade50 2023-11-08T16:41:10.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e4c003520 con 0x7f6e640ade50 2023-11-08T16:41:10.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 216136934 0 0) 0x7f6e4c003520 con 0x7f6e640ade50 2023-11-08T16:41:10.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 >> v2:172.21.15.93:3300/0 conn(0x7f6e640ca480 msgr2=0x7f6e6415ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:10.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 --2- 172.21.15.81:0/3408727953 >> v2:172.21.15.93:3300/0 conn(0x7f6e640ca480 0x7f6e6415ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 >> v1:172.21.15.93:6789/0 conn(0x7f6e6415ea60 legacy=0x7f6e6415eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e640cabb0 con 0x7f6e640ade50 2023-11-08T16:41:10.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f6e54001670 con 0x7f6e640ade50 2023-11-08T16:41:10.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 >> v1:172.21.15.81:6789/0 conn(0x7f6e640ade50 legacy=0x7f6e6415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 --2- 172.21.15.81:0/3408727953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 0x7f6e4c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 --2- 172.21.15.81:0/3408727953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e4c0047c0 0x7f6e4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 --2- 172.21.15.81:0/3408727953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e4c0051d0 0x7f6e4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c00c0d0 con 0x7f6e4c003a50 2023-11-08T16:41:10.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c00c240 con 0x7f6e4c0047c0 2023-11-08T16:41:10.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c00c400 con 0x7f6e4c0051d0 2023-11-08T16:41:10.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.955+0000 7f6e6259c640 1 --2- 172.21.15.81:0/3408727953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 0x7f6e4c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.956+0000 7f6e61d9b640 1 --2- 172.21.15.81:0/3408727953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e4c0047c0 0x7f6e4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.956+0000 7f6e62d9d640 1 --2- 172.21.15.81:0/3408727953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e4c0051d0 0x7f6e4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.956+0000 7f6e6259c640 1 -- 172.21.15.81:0/3408727953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e4c0051d0 msgr2=0x7f6e4c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:10.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.956+0000 7f6e6259c640 1 --2- 172.21.15.81:0/3408727953 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e4c0051d0 0x7f6e4c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.956+0000 7f6e6259c640 1 -- 172.21.15.81:0/3408727953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e4c0047c0 msgr2=0x7f6e4c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:10.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.956+0000 7f6e6259c640 1 --2- 172.21.15.81:0/3408727953 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e4c0047c0 0x7f6e4c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.956+0000 7f6e6259c640 1 -- 172.21.15.81:0/3408727953 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c003520 con 0x7f6e4c003a50 2023-11-08T16:41:10.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.956+0000 7f6e62d9d640 1 -- 172.21.15.81:0/3408727953 reap_dead start 2023-11-08T16:41:10.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.957+0000 7f6e6259c640 1 --2- 172.21.15.81:0/3408727953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 0x7f6e4c003e70 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400c210 tx=0x7f6e4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acdf2b4cabad6820 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:10.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.957+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e54005ad0 con 0x7f6e4c003a50 2023-11-08T16:41:10.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.957+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e54005c30 con 0x7f6e4c003a50 2023-11-08T16:41:10.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.957+0000 7f6e6159a640 1 -- 172.21.15.81:0/3408727953 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5402cab0 con 0x7f6e4c003a50 2023-11-08T16:41:10.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.958+0000 7f6e68ec2640 1 -- 172.21.15.81:0/3408727953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 msgr2=0x7f6e4c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.958+0000 7f6e68ec2640 1 --2- 172.21.15.81:0/3408727953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 0x7f6e4c003e70 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f6e5400c210 tx=0x7f6e4c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.958+0000 7f6e68ec2640 1 -- 172.21.15.81:0/3408727953 shutdown_connections 2023-11-08T16:41:10.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.958+0000 7f6e68ec2640 1 --2- 172.21.15.81:0/3408727953 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 0x7f6e4c003e70 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.958+0000 7f6e68ec2640 1 -- 172.21.15.81:0/3408727953 >> 172.21.15.81:0/3408727953 conn(0x7f6e640cb490 msgr2=0x7f6e640cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:10.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.958+0000 7f6e68ec2640 1 -- 172.21.15.81:0/3408727953 shutdown_connections 2023-11-08T16:41:10.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.958+0000 7f6e68ec2640 1 -- 172.21.15.81:0/3408727953 wait complete. 2023-11-08T16:41:10.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.960+0000 7f6e68ec2640 1 Processor -- start 2023-11-08T16:41:10.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.960+0000 7f6e68ec2640 1 -- start start 2023-11-08T16:41:10.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e68ec2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 0x7f6e64144590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e68ec2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e640b24e0 0x7f6e6415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e68ec2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e640b15c0 0x7f6e640b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e68ec2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e64159ba0 con 0x7f6e4c003a50 2023-11-08T16:41:10.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e68ec2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e640b1f80 con 0x7f6e640b24e0 2023-11-08T16:41:10.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e68ec2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e641454c0 con 0x7f6e640b15c0 2023-11-08T16:41:10.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e6259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 0x7f6e64144590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e6259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 0x7f6e64144590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36882/0 (socket says 172.21.15.81:36882) 2023-11-08T16:41:10.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e6259c640 1 -- 172.21.15.81:0/23937775 learned_addr learned my addr 172.21.15.81:0/23937775 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:10.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e6259c640 1 -- 172.21.15.81:0/23937775 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e640b15c0 msgr2=0x7f6e640b1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e62d9d640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e640b15c0 0x7f6e640b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e61d9b640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e640b24e0 0x7f6e6415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e6259c640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e640b15c0 0x7f6e640b1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e6259c640 1 -- 172.21.15.81:0/23937775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e640b24e0 msgr2=0x7f6e6415bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:10.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e6259c640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e640b24e0 0x7f6e6415bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:10.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.961+0000 7f6e6259c640 1 -- 172.21.15.81:0/23937775 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64144c20 con 0x7f6e4c003a50 2023-11-08T16:41:10.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.962+0000 7f6e6259c640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 0x7f6e64144590 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f6e5402c410 tx=0x7f6e5402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:10.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.962+0000 7f6e52ffd640 1 -- 172.21.15.81:0/23937775 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5403c040 con 0x7f6e4c003a50 2023-11-08T16:41:10.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.962+0000 7f6e52ffd640 1 -- 172.21.15.81:0/23937775 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5402cd40 con 0x7f6e4c003a50 2023-11-08T16:41:10.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.962+0000 7f6e68ec2640 1 -- 172.21.15.81:0/23937775 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e64144f00 con 0x7f6e4c003a50 2023-11-08T16:41:10.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.962+0000 7f6e52ffd640 1 -- 172.21.15.81:0/23937775 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e540072b0 con 0x7f6e4c003a50 2023-11-08T16:41:10.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.962+0000 7f6e68ec2640 1 -- 172.21.15.81:0/23937775 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e640b2960 con 0x7f6e4c003a50 2023-11-08T16:41:10.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.963+0000 7f6e68ec2640 1 -- 172.21.15.81:0/23937775 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e64159e10 con 0x7f6e4c003a50 2023-11-08T16:41:10.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.964+0000 7f6e52ffd640 1 -- 172.21.15.81:0/23937775 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e54032070 con 0x7f6e4c003a50 2023-11-08T16:41:10.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.965+0000 7f6e52ffd640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e4406ae70 0x7f6e4406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:10.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.965+0000 7f6e52ffd640 1 -- 172.21.15.81:0/23937775 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e540aa330 con 0x7f6e4c003a50 2023-11-08T16:41:10.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.966+0000 7f6e61d9b640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e4406ae70 0x7f6e4406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:10.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.966+0000 7f6e61d9b640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e4406ae70 0x7f6e4406d330 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6e6415b840 tx=0x7f6e58008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:10.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:10.969+0000 7f6e52ffd640 1 -- 172.21.15.81:0/23937775 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6e54075470 con 0x7f6e4c003a50 2023-11-08T16:41:11.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.156+0000 7f6e68ec2640 1 -- 172.21.15.81:0/23937775 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e640cb170 con 0x7f6e4406ae70 2023-11-08T16:41:11.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.179+0000 7f6e52ffd640 1 -- 172.21.15.81:0/23937775 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287096 (secure 0 0 0) 0x7f6e58012950 con 0x7f6e4406ae70 2023-11-08T16:41:11.184 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:11.185 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:41:11.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 -- 172.21.15.81:0/23937775 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e4406ae70 msgr2=0x7f6e4406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e4406ae70 0x7f6e4406d330 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f6e6415b840 tx=0x7f6e58008040 comp rx=0 tx=0).stop 2023-11-08T16:41:11.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 -- 172.21.15.81:0/23937775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 msgr2=0x7f6e64144590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 0x7f6e64144590 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f6e5402c410 tx=0x7f6e5402e070 comp rx=0 tx=0).stop 2023-11-08T16:41:11.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 -- 172.21.15.81:0/23937775 shutdown_connections 2023-11-08T16:41:11.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6e4406ae70 0x7f6e4406d330 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6e640b15c0 0x7f6e640b1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e640b24e0 0x7f6e6415bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 --2- 172.21.15.81:0/23937775 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6e4c003a50 0x7f6e64144590 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 -- 172.21.15.81:0/23937775 >> 172.21.15.81:0/23937775 conn(0x7f6e640cb490 msgr2=0x7f6e6406bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:11.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 -- 172.21.15.81:0/23937775 shutdown_connections 2023-11-08T16:41:11.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.185+0000 7f6e68ec2640 1 -- 172.21.15.81:0/23937775 wait complete. 2023-11-08T16:41:11.207 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2023-11-08T16:41:10.807771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262820,"kb_used_data":13364,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711900,"statfs":{"total":773094113280,"available":772824985600,"internally_reserved":0,"allocated":13684736,"data_stored":6971620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001840"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.305319+0000","last_change":"2023-11-08T16:41:06.301616+0000","last_active":"2023-11-08T16:41:06.305319+0000","last_peered":"2023-11-08T16:41:06.305319+0000","last_clean":"2023-11-08T16:41:06.305319+0000","last_became_active":"2023-11-08T16:41:06.301213+0000","last_became_peered":"2023-11-08T16:41:06.301213+0000","last_unstale":"2023-11-08T16:41:06.305319+0000","last_undegraded":"2023-11-08T16:41:06.305319+0000","last_fullsized":"2023-11-08T16:41:06.305319+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:04:05.385747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.288938+0000","last_change":"2023-11-08T16:41:06.284918+0000","last_active":"2023-11-08T16:41:06.288938+0000","last_peered":"2023-11-08T16:41:06.288938+0000","last_clean":"2023-11-08T16:41:06.288938+0000","last_became_active":"2023-11-08T16:41:06.284648+0000","last_became_peered":"2023-11-08T16:41:06.284648+0000","last_unstale":"2023-11-08T16:41:06.288938+0000","last_undegraded":"2023-11-08T16:41:06.288938+0000","last_fullsized":"2023-11-08T16:41:06.288938+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:58.442996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290919+0000","last_change":"2023-11-08T16:41:06.283004+0000","last_active":"2023-11-08T16:41:06.290919+0000","last_peered":"2023-11-08T16:41:06.290919+0000","last_clean":"2023-11-08T16:41:06.290919+0000","last_became_active":"2023-11-08T16:41:06.281991+0000","last_became_peered":"2023-11-08T16:41:06.281991+0000","last_unstale":"2023-11-08T16:41:06.290919+0000","last_undegraded":"2023-11-08T16:41:06.290919+0000","last_fullsized":"2023-11-08T16:41:06.290919+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:25:57.844111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.300975+0000","last_change":"2023-11-08T16:41:06.292131+0000","last_active":"2023-11-08T16:41:06.300975+0000","last_peered":"2023-11-08T16:41:06.300975+0000","last_clean":"2023-11-08T16:41:06.300975+0000","last_became_active":"2023-11-08T16:41:06.291843+0000","last_became_peered":"2023-11-08T16:41:06.291843+0000","last_unstale":"2023-11-08T16:41:06.300975+0000","last_undegraded":"2023-11-08T16:41:06.300975+0000","last_fullsized":"2023-11-08T16:41:06.300975+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:29:21.161754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.283831+0000","last_change":"2023-11-08T16:41:06.278719+0000","last_active":"2023-11-08T16:41:06.283831+0000","last_peered":"2023-11-08T16:41:06.283831+0000","last_clean":"2023-11-08T16:41:06.283831+0000","last_became_active":"2023-11-08T16:41:06.277896+0000","last_became_peered":"2023-11-08T16:41:06.277896+0000","last_unstale":"2023-11-08T16:41:06.283831+0000","last_undegraded":"2023-11-08T16:41:06.283831+0000","last_fullsized":"2023-11-08T16:41:06.283831+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:56:03.598527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.282121+0000","last_change":"2023-11-08T16:41:06.274978+0000","last_active":"2023-11-08T16:41:06.282121+0000","last_peered":"2023-11-08T16:41:06.282121+0000","last_clean":"2023-11-08T16:41:06.282121+0000","last_became_active":"2023-11-08T16:41:06.274665+0000","last_became_peered":"2023-11-08T16:41:06.274665+0000","last_unstale":"2023-11-08T16:41:06.282121+0000","last_undegraded":"2023-11-08T16:41:06.282121+0000","last_fullsized":"2023-11-08T16:41:06.282121+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:34.391636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.277524+0000","last_change":"2023-11-08T16:41:06.269506+0000","last_active":"2023-11-08T16:41:06.277524+0000","last_peered":"2023-11-08T16:41:06.277524+0000","last_clean":"2023-11-08T16:41:06.277524+0000","last_became_active":"2023-11-08T16:41:06.269083+0000","last_became_peered":"2023-11-08T16:41:06.269083+0000","last_unstale":"2023-11-08T16:41:06.277524+0000","last_undegraded":"2023-11-08T16:41:06.277524+0000","last_fullsized":"2023-11-08T16:41:06.277524+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:43.494464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291180+0000","last_change":"2023-11-08T16:41:06.286884+0000","last_active":"2023-11-08T16:41:06.291180+0000","last_peered":"2023-11-08T16:41:06.291180+0000","last_clean":"2023-11-08T16:41:06.291180+0000","last_became_active":"2023-11-08T16:41:06.282539+0000","last_became_peered":"2023-11-08T16:41:06.282539+0000","last_unstale":"2023-11-08T16:41:06.291180+0000","last_undegraded":"2023-11-08T16:41:06.291180+0000","last_fullsized":"2023-11-08T16:41:06.291180+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:34.391636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.303032+0000","last_change":"2023-11-08T16:41:06.299208+0000","last_active":"2023-11-08T16:41:06.303032+0000","last_peered":"2023-11-08T16:41:06.303032+0000","last_clean":"2023-11-08T16:41:06.303032+0000","last_became_active":"2023-11-08T16:41:06.297534+0000","last_became_peered":"2023-11-08T16:41:06.297534+0000","last_unstale":"2023-11-08T16:41:06.303032+0000","last_undegraded":"2023-11-08T16:41:06.303032+0000","last_fullsized":"2023-11-08T16:41:06.303032+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:50:27.335196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.300896+0000","last_change":"2023-11-08T16:41:06.291973+0000","last_active":"2023-11-08T16:41:06.300896+0000","last_peered":"2023-11-08T16:41:06.300896+0000","last_clean":"2023-11-08T16:41:06.300896+0000","last_became_active":"2023-11-08T16:41:06.291692+0000","last_became_peered":"2023-11-08T16:41:06.291692+0000","last_unstale":"2023-11-08T16:41:06.300896+0000","last_undegraded":"2023-11-08T16:41:06.300896+0000","last_fullsized":"2023-11-08T16:41:06.300896+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:19:21.405123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.285836+0000","last_change":"2023-11-08T16:41:06.282233+0000","last_active":"2023-11-08T16:41:06.285836+0000","last_peered":"2023-11-08T16:41:06.285836+0000","last_clean":"2023-11-08T16:41:06.285836+0000","last_became_active":"2023-11-08T16:41:06.281637+0000","last_became_peered":"2023-11-08T16:41:06.281637+0000","last_unstale":"2023-11-08T16:41:06.285836+0000","last_undegraded":"2023-11-08T16:41:06.285836+0000","last_fullsized":"2023-11-08T16:41:06.285836+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:52:57.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.300930+0000","last_change":"2023-11-08T16:41:06.288442+0000","last_active":"2023-11-08T16:41:06.300930+0000","last_peered":"2023-11-08T16:41:06.300930+0000","last_clean":"2023-11-08T16:41:06.300930+0000","last_became_active":"2023-11-08T16:41:06.288157+0000","last_became_peered":"2023-11-08T16:41:06.288157+0000","last_unstale":"2023-11-08T16:41:06.300930+0000","last_undegraded":"2023-11-08T16:41:06.300930+0000","last_fullsized":"2023-11-08T16:41:06.300930+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:58:34.988070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.292926+0000","last_change":"2023-11-08T16:41:06.290385+0000","last_active":"2023-11-08T16:41:06.292926+0000","last_peered":"2023-11-08T16:41:06.292926+0000","last_clean":"2023-11-08T16:41:06.292926+0000","last_became_active":"2023-11-08T16:41:06.286883+0000","last_became_peered":"2023-11-08T16:41:06.286883+0000","last_unstale":"2023-11-08T16:41:06.292926+0000","last_undegraded":"2023-11-08T16:41:06.292926+0000","last_fullsized":"2023-11-08T16:41:06.292926+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:29:25.784060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290475+0000","last_change":"2023-11-08T16:41:06.282234+0000","last_active":"2023-11-08T16:41:06.290475+0000","last_peered":"2023-11-08T16:41:06.290475+0000","last_clean":"2023-11-08T16:41:06.290475+0000","last_became_active":"2023-11-08T16:41:06.281818+0000","last_became_peered":"2023-11-08T16:41:06.281818+0000","last_unstale":"2023-11-08T16:41:06.290475+0000","last_undegraded":"2023-11-08T16:41:06.290475+0000","last_fullsized":"2023-11-08T16:41:06.290475+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:10:54.032689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.281244+0000","last_change":"2023-11-08T16:41:06.275909+0000","last_active":"2023-11-08T16:41:06.281244+0000","last_peered":"2023-11-08T16:41:06.281244+0000","last_clean":"2023-11-08T16:41:06.281244+0000","last_became_active":"2023-11-08T16:41:06.275560+0000","last_became_peered":"2023-11-08T16:41:06.275560+0000","last_unstale":"2023-11-08T16:41:06.281244+0000","last_undegraded":"2023-11-08T16:41:06.281244+0000","last_fullsized":"2023-11-08T16:41:06.281244+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:29:25.784060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.302702+0000","last_change":"2023-11-08T16:41:06.297676+0000","last_active":"2023-11-08T16:41:06.302702+0000","last_peered":"2023-11-08T16:41:06.302702+0000","last_clean":"2023-11-08T16:41:06.302702+0000","last_became_active":"2023-11-08T16:41:06.295552+0000","last_became_peered":"2023-11-08T16:41:06.295552+0000","last_unstale":"2023-11-08T16:41:06.302702+0000","last_undegraded":"2023-11-08T16:41:06.302702+0000","last_fullsized":"2023-11-08T16:41:06.302702+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:18.714608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.288921+0000","last_change":"2023-11-08T16:41:06.285617+0000","last_active":"2023-11-08T16:41:06.288921+0000","last_peered":"2023-11-08T16:41:06.288921+0000","last_clean":"2023-11-08T16:41:06.288921+0000","last_became_active":"2023-11-08T16:41:06.284944+0000","last_became_peered":"2023-11-08T16:41:06.284944+0000","last_unstale":"2023-11-08T16:41:06.288921+0000","last_undegraded":"2023-11-08T16:41:06.288921+0000","last_fullsized":"2023-11-08T16:41:06.288921+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:56.049065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.281893+0000","last_change":"2023-11-08T16:41:06.279410+0000","last_active":"2023-11-08T16:41:06.281893+0000","last_peered":"2023-11-08T16:41:06.281893+0000","last_clean":"2023-11-08T16:41:06.281893+0000","last_became_active":"2023-11-08T16:41:06.278826+0000","last_became_peered":"2023-11-08T16:41:06.278826+0000","last_unstale":"2023-11-08T16:41:06.281893+0000","last_undegraded":"2023-11-08T16:41:06.281893+0000","last_fullsized":"2023-11-08T16:41:06.281893+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:43.494464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291956+0000","last_change":"2023-11-08T16:41:06.289538+0000","last_active":"2023-11-08T16:41:06.291956+0000","last_peered":"2023-11-08T16:41:06.291956+0000","last_clean":"2023-11-08T16:41:06.291956+0000","last_became_active":"2023-11-08T16:41:06.282421+0000","last_became_peered":"2023-11-08T16:41:06.282421+0000","last_unstale":"2023-11-08T16:41:06.291956+0000","last_undegraded":"2023-11-08T16:41:06.291956+0000","last_fullsized":"2023-11-08T16:41:06.291956+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:52:57.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290574+0000","last_change":"2023-11-08T16:41:06.287860+0000","last_active":"2023-11-08T16:41:06.290574+0000","last_peered":"2023-11-08T16:41:06.290574+0000","last_clean":"2023-11-08T16:41:06.290574+0000","last_became_active":"2023-11-08T16:41:06.287608+0000","last_became_peered":"2023-11-08T16:41:06.287608+0000","last_unstale":"2023-11-08T16:41:06.290574+0000","last_undegraded":"2023-11-08T16:41:06.290574+0000","last_fullsized":"2023-11-08T16:41:06.290574+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:58.416407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291313+0000","last_change":"2023-11-08T16:41:06.289741+0000","last_active":"2023-11-08T16:41:06.291313+0000","last_peered":"2023-11-08T16:41:06.291313+0000","last_clean":"2023-11-08T16:41:06.291313+0000","last_became_active":"2023-11-08T16:41:06.288680+0000","last_became_peered":"2023-11-08T16:41:06.288680+0000","last_unstale":"2023-11-08T16:41:06.291313+0000","last_undegraded":"2023-11-08T16:41:06.291313+0000","last_fullsized":"2023-11-08T16:41:06.291313+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:08:21.221808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.275052+0000","last_change":"2023-11-08T16:41:06.272480+0000","last_active":"2023-11-08T16:41:06.275052+0000","last_peered":"2023-11-08T16:41:06.275052+0000","last_clean":"2023-11-08T16:41:06.275052+0000","last_became_active":"2023-11-08T16:41:06.271356+0000","last_became_peered":"2023-11-08T16:41:06.271356+0000","last_unstale":"2023-11-08T16:41:06.275052+0000","last_undegraded":"2023-11-08T16:41:06.275052+0000","last_fullsized":"2023-11-08T16:41:06.275052+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:08:21.221808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291065+0000","last_change":"2023-11-08T16:41:06.288411+0000","last_active":"2023-11-08T16:41:06.291065+0000","last_peered":"2023-11-08T16:41:06.291065+0000","last_clean":"2023-11-08T16:41:06.291065+0000","last_became_active":"2023-11-08T16:41:06.287939+0000","last_became_peered":"2023-11-08T16:41:06.287939+0000","last_unstale":"2023-11-08T16:41:06.291065+0000","last_undegraded":"2023-11-08T16:41:06.291065+0000","last_fullsized":"2023-11-08T16:41:06.291065+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:39:21.572152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.287809+0000","last_change":"2023-11-08T16:41:06.281996+0000","last_active":"2023-11-08T16:41:06.287809+0000","last_peered":"2023-11-08T16:41:06.287809+0000","last_clean":"2023-11-08T16:41:06.287809+0000","last_became_active":"2023-11-08T16:41:06.281666+0000","last_became_peered":"2023-11-08T16:41:06.281666+0000","last_unstale":"2023-11-08T16:41:06.287809+0000","last_undegraded":"2023-11-08T16:41:06.287809+0000","last_fullsized":"2023-11-08T16:41:06.287809+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:58.416407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.288272+0000","last_change":"2023-11-08T16:41:06.283649+0000","last_active":"2023-11-08T16:41:06.288272+0000","last_peered":"2023-11-08T16:41:06.288272+0000","last_clean":"2023-11-08T16:41:06.288272+0000","last_became_active":"2023-11-08T16:41:06.283344+0000","last_became_peered":"2023-11-08T16:41:06.283344+0000","last_unstale":"2023-11-08T16:41:06.288272+0000","last_undegraded":"2023-11-08T16:41:06.288272+0000","last_fullsized":"2023-11-08T16:41:06.288272+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:22:54.921237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.301074+0000","last_change":"2023-11-08T16:41:06.295380+0000","last_active":"2023-11-08T16:41:06.301074+0000","last_peered":"2023-11-08T16:41:06.301074+0000","last_clean":"2023-11-08T16:41:06.301074+0000","last_became_active":"2023-11-08T16:41:06.292522+0000","last_became_peered":"2023-11-08T16:41:06.292522+0000","last_unstale":"2023-11-08T16:41:06.301074+0000","last_undegraded":"2023-11-08T16:41:06.301074+0000","last_fullsized":"2023-11-08T16:41:06.301074+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:37:31.717459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.303481+0000","last_change":"2023-11-08T16:41:06.301848+0000","last_active":"2023-11-08T16:41:06.303481+0000","last_peered":"2023-11-08T16:41:06.303481+0000","last_clean":"2023-11-08T16:41:06.303481+0000","last_became_active":"2023-11-08T16:41:06.301111+0000","last_became_peered":"2023-11-08T16:41:06.301111+0000","last_unstale":"2023-11-08T16:41:06.303481+0000","last_undegraded":"2023-11-08T16:41:06.303481+0000","last_fullsized":"2023-11-08T16:41:06.303481+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:10:54.032689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.296304+0000","last_change":"2023-11-08T16:41:06.293306+0000","last_active":"2023-11-08T16:41:06.296304+0000","last_peered":"2023-11-08T16:41:06.296304+0000","last_clean":"2023-11-08T16:41:06.296304+0000","last_became_active":"2023-11-08T16:41:06.292834+0000","last_became_peered":"2023-11-08T16:41:06.292834+0000","last_unstale":"2023-11-08T16:41:06.296304+0000","last_undegraded":"2023-11-08T16:41:06.296304+0000","last_fullsized":"2023-11-08T16:41:06.296304+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:06:56.049065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.294138+0000","last_change":"2023-11-08T16:41:06.292936+0000","last_active":"2023-11-08T16:41:06.294138+0000","last_peered":"2023-11-08T16:41:06.294138+0000","last_clean":"2023-11-08T16:41:06.294138+0000","last_became_active":"2023-11-08T16:41:06.291914+0000","last_became_peered":"2023-11-08T16:41:06.291914+0000","last_unstale":"2023-11-08T16:41:06.294138+0000","last_undegraded":"2023-11-08T16:41:06.294138+0000","last_fullsized":"2023-11-08T16:41:06.294138+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:38.335700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.280246+0000","last_change":"2023-11-08T16:41:06.277501+0000","last_active":"2023-11-08T16:41:06.280246+0000","last_peered":"2023-11-08T16:41:06.280246+0000","last_clean":"2023-11-08T16:41:06.280246+0000","last_became_active":"2023-11-08T16:41:06.277217+0000","last_became_peered":"2023-11-08T16:41:06.277217+0000","last_unstale":"2023-11-08T16:41:06.280246+0000","last_undegraded":"2023-11-08T16:41:06.280246+0000","last_fullsized":"2023-11-08T16:41:06.280246+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:40:53.393145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.301170+0000","last_change":"2023-11-08T16:41:06.293906+0000","last_active":"2023-11-08T16:41:06.301170+0000","last_peered":"2023-11-08T16:41:06.301170+0000","last_clean":"2023-11-08T16:41:06.301170+0000","last_became_active":"2023-11-08T16:41:06.292421+0000","last_became_peered":"2023-11-08T16:41:06.292421+0000","last_unstale":"2023-11-08T16:41:06.301170+0000","last_undegraded":"2023-11-08T16:41:06.301170+0000","last_fullsized":"2023-11-08T16:41:06.301170+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:55:11.977247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.279176+0000","last_change":"2023-11-08T16:41:06.274274+0000","last_active":"2023-11-08T16:41:06.279176+0000","last_peered":"2023-11-08T16:41:06.279176+0000","last_clean":"2023-11-08T16:41:06.279176+0000","last_became_active":"2023-11-08T16:41:06.271427+0000","last_became_peered":"2023-11-08T16:41:06.271427+0000","last_unstale":"2023-11-08T16:41:06.279176+0000","last_undegraded":"2023-11-08T16:41:06.279176+0000","last_fullsized":"2023-11-08T16:41:06.279176+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:07:10.051108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.275214+0000","last_change":"2023-11-08T16:41:06.273090+0000","last_active":"2023-11-08T16:41:06.275214+0000","last_peered":"2023-11-08T16:41:06.275214+0000","last_clean":"2023-11-08T16:41:06.275214+0000","last_became_active":"2023-11-08T16:41:06.272643+0000","last_became_peered":"2023-11-08T16:41:06.272643+0000","last_unstale":"2023-11-08T16:41:06.275214+0000","last_undegraded":"2023-11-08T16:41:06.275214+0000","last_fullsized":"2023-11-08T16:41:06.275214+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:39:32.914205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"126'4","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.293326+0000","last_change":"2023-11-08T16:41:05.274000+0000","last_active":"2023-11-08T16:41:06.293326+0000","last_peered":"2023-11-08T16:41:06.293326+0000","last_clean":"2023-11-08T16:41:06.293326+0000","last_became_active":"2023-11-08T16:41:05.273705+0000","last_became_peered":"2023-11-08T16:41:05.273705+0000","last_unstale":"2023-11-08T16:41:06.293326+0000","last_undegraded":"2023-11-08T16:41:06.293326+0000","last_fullsized":"2023-11-08T16:41:06.293326+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:34:25.857434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.281934+0000","last_change":"2023-11-08T16:41:06.280252+0000","last_active":"2023-11-08T16:41:06.281934+0000","last_peered":"2023-11-08T16:41:06.281934+0000","last_clean":"2023-11-08T16:41:06.281934+0000","last_became_active":"2023-11-08T16:41:06.279960+0000","last_became_peered":"2023-11-08T16:41:06.279960+0000","last_unstale":"2023-11-08T16:41:06.281934+0000","last_undegraded":"2023-11-08T16:41:06.281934+0000","last_fullsized":"2023-11-08T16:41:06.281934+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:46:22.237288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"126'1","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.294383+0000","last_change":"2023-11-08T16:41:05.279755+0000","last_active":"2023-11-08T16:41:06.294383+0000","last_peered":"2023-11-08T16:41:06.294383+0000","last_clean":"2023-11-08T16:41:06.294383+0000","last_became_active":"2023-11-08T16:41:05.277987+0000","last_became_peered":"2023-11-08T16:41:05.277987+0000","last_unstale":"2023-11-08T16:41:06.294383+0000","last_undegraded":"2023-11-08T16:41:06.294383+0000","last_fullsized":"2023-11-08T16:41:06.294383+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:57.399931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.280296+0000","last_change":"2023-11-08T16:41:06.275763+0000","last_active":"2023-11-08T16:41:06.280296+0000","last_peered":"2023-11-08T16:41:06.280296+0000","last_clean":"2023-11-08T16:41:06.280296+0000","last_became_active":"2023-11-08T16:41:06.271606+0000","last_became_peered":"2023-11-08T16:41:06.271606+0000","last_unstale":"2023-11-08T16:41:06.280296+0000","last_undegraded":"2023-11-08T16:41:06.280296+0000","last_fullsized":"2023-11-08T16:41:06.280296+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:25:57.844111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291473+0000","last_change":"2023-11-08T16:41:05.267264+0000","last_active":"2023-11-08T16:41:06.291473+0000","last_peered":"2023-11-08T16:41:06.291473+0000","last_clean":"2023-11-08T16:41:06.291473+0000","last_became_active":"2023-11-08T16:41:05.266968+0000","last_became_peered":"2023-11-08T16:41:05.266968+0000","last_unstale":"2023-11-08T16:41:06.291473+0000","last_undegraded":"2023-11-08T16:41:06.291473+0000","last_fullsized":"2023-11-08T16:41:06.291473+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:02:16.702650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291175+0000","last_change":"2023-11-08T16:41:06.283362+0000","last_active":"2023-11-08T16:41:06.291175+0000","last_peered":"2023-11-08T16:41:06.291175+0000","last_clean":"2023-11-08T16:41:06.291175+0000","last_became_active":"2023-11-08T16:41:06.282682+0000","last_became_peered":"2023-11-08T16:41:06.282682+0000","last_unstale":"2023-11-08T16:41:06.291175+0000","last_undegraded":"2023-11-08T16:41:06.291175+0000","last_fullsized":"2023-11-08T16:41:06.291175+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:39:21.572152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291425+0000","last_change":"2023-11-08T16:41:05.273695+0000","last_active":"2023-11-08T16:41:06.291425+0000","last_peered":"2023-11-08T16:41:06.291425+0000","last_clean":"2023-11-08T16:41:06.291425+0000","last_became_active":"2023-11-08T16:41:05.273377+0000","last_became_peered":"2023-11-08T16:41:05.273377+0000","last_unstale":"2023-11-08T16:41:06.291425+0000","last_undegraded":"2023-11-08T16:41:06.291425+0000","last_fullsized":"2023-11-08T16:41:06.291425+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:32:16.371465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.301138+0000","last_change":"2023-11-08T16:41:06.296231+0000","last_active":"2023-11-08T16:41:06.301138+0000","last_peered":"2023-11-08T16:41:06.301138+0000","last_clean":"2023-11-08T16:41:06.301138+0000","last_became_active":"2023-11-08T16:41:06.292390+0000","last_became_peered":"2023-11-08T16:41:06.292390+0000","last_unstale":"2023-11-08T16:41:06.301138+0000","last_undegraded":"2023-11-08T16:41:06.301138+0000","last_fullsized":"2023-11-08T16:41:06.301138+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:57:29.424324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"126'1","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.278825+0000","last_change":"2023-11-08T16:41:05.270953+0000","last_active":"2023-11-08T16:41:06.278825+0000","last_peered":"2023-11-08T16:41:06.278825+0000","last_clean":"2023-11-08T16:41:06.278825+0000","last_became_active":"2023-11-08T16:41:05.270572+0000","last_became_peered":"2023-11-08T16:41:05.270572+0000","last_unstale":"2023-11-08T16:41:06.278825+0000","last_undegraded":"2023-11-08T16:41:06.278825+0000","last_fullsized":"2023-11-08T16:41:06.278825+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:49:59.029607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290737+0000","last_change":"2023-11-08T16:41:06.286455+0000","last_active":"2023-11-08T16:41:06.290737+0000","last_peered":"2023-11-08T16:41:06.290737+0000","last_clean":"2023-11-08T16:41:06.290737+0000","last_became_active":"2023-11-08T16:41:06.285809+0000","last_became_peered":"2023-11-08T16:41:06.285809+0000","last_unstale":"2023-11-08T16:41:06.290737+0000","last_undegraded":"2023-11-08T16:41:06.290737+0000","last_fullsized":"2023-11-08T16:41:06.290737+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:43:22.680629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"126'1","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290403+0000","last_change":"2023-11-08T16:41:05.273052+0000","last_active":"2023-11-08T16:41:06.290403+0000","last_peered":"2023-11-08T16:41:06.290403+0000","last_clean":"2023-11-08T16:41:06.290403+0000","last_became_active":"2023-11-08T16:41:05.272801+0000","last_became_peered":"2023-11-08T16:41:05.272801+0000","last_unstale":"2023-11-08T16:41:06.290403+0000","last_undegraded":"2023-11-08T16:41:06.290403+0000","last_fullsized":"2023-11-08T16:41:06.290403+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:32:37.319224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.302831+0000","last_change":"2023-11-08T16:41:06.299315+0000","last_active":"2023-11-08T16:41:06.302831+0000","last_peered":"2023-11-08T16:41:06.302831+0000","last_clean":"2023-11-08T16:41:06.302831+0000","last_became_active":"2023-11-08T16:41:06.298125+0000","last_became_peered":"2023-11-08T16:41:06.298125+0000","last_unstale":"2023-11-08T16:41:06.302831+0000","last_undegraded":"2023-11-08T16:41:06.302831+0000","last_fullsized":"2023-11-08T16:41:06.302831+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:38:59.249974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.274955+0000","last_change":"2023-11-08T16:41:05.279401+0000","last_active":"2023-11-08T16:41:06.274955+0000","last_peered":"2023-11-08T16:41:06.274955+0000","last_clean":"2023-11-08T16:41:06.274955+0000","last_became_active":"2023-11-08T16:41:05.278988+0000","last_became_peered":"2023-11-08T16:41:05.278988+0000","last_unstale":"2023-11-08T16:41:06.274955+0000","last_undegraded":"2023-11-08T16:41:06.274955+0000","last_fullsized":"2023-11-08T16:41:06.274955+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:16:54.418211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.270399+0000","last_change":"2023-11-08T16:41:06.268004+0000","last_active":"2023-11-08T16:41:06.270399+0000","last_peered":"2023-11-08T16:41:06.270399+0000","last_clean":"2023-11-08T16:41:06.270399+0000","last_became_active":"2023-11-08T16:41:06.267321+0000","last_became_peered":"2023-11-08T16:41:06.267321+0000","last_unstale":"2023-11-08T16:41:06.270399+0000","last_undegraded":"2023-11-08T16:41:06.270399+0000","last_fullsized":"2023-11-08T16:41:06.270399+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:45:16.031347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.285010+0000","last_change":"2023-11-08T16:41:05.275407+0000","last_active":"2023-11-08T16:41:06.285010+0000","last_peered":"2023-11-08T16:41:06.285010+0000","last_clean":"2023-11-08T16:41:06.285010+0000","last_became_active":"2023-11-08T16:41:05.274947+0000","last_became_peered":"2023-11-08T16:41:05.274947+0000","last_unstale":"2023-11-08T16:41:06.285010+0000","last_undegraded":"2023-11-08T16:41:06.285010+0000","last_fullsized":"2023-11-08T16:41:06.285010+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:23:34.700161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.279431+0000","last_change":"2023-11-08T16:41:06.272486+0000","last_active":"2023-11-08T16:41:06.279431+0000","last_peered":"2023-11-08T16:41:06.279431+0000","last_clean":"2023-11-08T16:41:06.279431+0000","last_became_active":"2023-11-08T16:41:06.272059+0000","last_became_peered":"2023-11-08T16:41:06.272059+0000","last_unstale":"2023-11-08T16:41:06.279431+0000","last_undegraded":"2023-11-08T16:41:06.279431+0000","last_fullsized":"2023-11-08T16:41:06.279431+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:08:45.675098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.265644+0000","last_change":"2023-11-08T16:41:05.270167+0000","last_active":"2023-11-08T16:41:06.265644+0000","last_peered":"2023-11-08T16:41:06.265644+0000","last_clean":"2023-11-08T16:41:06.265644+0000","last_became_active":"2023-11-08T16:41:05.269688+0000","last_became_peered":"2023-11-08T16:41:05.269688+0000","last_unstale":"2023-11-08T16:41:06.265644+0000","last_undegraded":"2023-11-08T16:41:06.265644+0000","last_fullsized":"2023-11-08T16:41:06.265644+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:52:59.580475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.299294+0000","last_change":"2023-11-08T16:41:06.298612+0000","last_active":"2023-11-08T16:41:06.299294+0000","last_peered":"2023-11-08T16:41:06.299294+0000","last_clean":"2023-11-08T16:41:06.299294+0000","last_became_active":"2023-11-08T16:41:06.298347+0000","last_became_peered":"2023-11-08T16:41:06.298347+0000","last_unstale":"2023-11-08T16:41:06.299294+0000","last_undegraded":"2023-11-08T16:41:06.299294+0000","last_fullsized":"2023-11-08T16:41:06.299294+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:25:57.844111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.267266+0000","last_change":"2023-11-08T16:41:05.279053+0000","last_active":"2023-11-08T16:41:06.267266+0000","last_peered":"2023-11-08T16:41:06.267266+0000","last_clean":"2023-11-08T16:41:06.267266+0000","last_became_active":"2023-11-08T16:41:05.278795+0000","last_became_peered":"2023-11-08T16:41:05.278795+0000","last_unstale":"2023-11-08T16:41:06.267266+0000","last_undegraded":"2023-11-08T16:41:06.267266+0000","last_fullsized":"2023-11-08T16:41:06.267266+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:52:32.302339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.286964+0000","last_change":"2023-11-08T16:41:06.280664+0000","last_active":"2023-11-08T16:41:06.286964+0000","last_peered":"2023-11-08T16:41:06.286964+0000","last_clean":"2023-11-08T16:41:06.286964+0000","last_became_active":"2023-11-08T16:41:06.280083+0000","last_became_peered":"2023-11-08T16:41:06.280083+0000","last_unstale":"2023-11-08T16:41:06.286964+0000","last_undegraded":"2023-11-08T16:41:06.286964+0000","last_fullsized":"2023-11-08T16:41:06.286964+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:32.971875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"126'3","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.305225+0000","last_change":"2023-11-08T16:41:05.281765+0000","last_active":"2023-11-08T16:41:06.305225+0000","last_peered":"2023-11-08T16:41:06.305225+0000","last_clean":"2023-11-08T16:41:06.305225+0000","last_became_active":"2023-11-08T16:41:05.281465+0000","last_became_peered":"2023-11-08T16:41:05.281465+0000","last_unstale":"2023-11-08T16:41:06.305225+0000","last_undegraded":"2023-11-08T16:41:06.305225+0000","last_fullsized":"2023-11-08T16:41:06.305225+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:31:00.273799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.285764+0000","last_change":"2023-11-08T16:41:06.285109+0000","last_active":"2023-11-08T16:41:06.285764+0000","last_peered":"2023-11-08T16:41:06.285764+0000","last_clean":"2023-11-08T16:41:06.285764+0000","last_became_active":"2023-11-08T16:41:06.284429+0000","last_became_peered":"2023-11-08T16:41:06.284429+0000","last_unstale":"2023-11-08T16:41:06.285764+0000","last_undegraded":"2023-11-08T16:41:06.285764+0000","last_fullsized":"2023-11-08T16:41:06.285764+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:45:23.747959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"126'6","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.305625+0000","last_change":"2023-11-08T16:41:05.282311+0000","last_active":"2023-11-08T16:41:06.305625+0000","last_peered":"2023-11-08T16:41:06.305625+0000","last_clean":"2023-11-08T16:41:06.305625+0000","last_became_active":"2023-11-08T16:41:05.281965+0000","last_became_peered":"2023-11-08T16:41:05.281965+0000","last_unstale":"2023-11-08T16:41:06.305625+0000","last_undegraded":"2023-11-08T16:41:06.305625+0000","last_fullsized":"2023-11-08T16:41:06.305625+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:38:27.762792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.287875+0000","last_change":"2023-11-08T16:41:06.282147+0000","last_active":"2023-11-08T16:41:06.287875+0000","last_peered":"2023-11-08T16:41:06.287875+0000","last_clean":"2023-11-08T16:41:06.287875+0000","last_became_active":"2023-11-08T16:41:06.281815+0000","last_became_peered":"2023-11-08T16:41:06.281815+0000","last_unstale":"2023-11-08T16:41:06.287875+0000","last_undegraded":"2023-11-08T16:41:06.287875+0000","last_fullsized":"2023-11-08T16:41:06.287875+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:43:22.680629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.293080+0000","last_change":"2023-11-08T16:41:05.274758+0000","last_active":"2023-11-08T16:41:06.293080+0000","last_peered":"2023-11-08T16:41:06.293080+0000","last_clean":"2023-11-08T16:41:06.293080+0000","last_became_active":"2023-11-08T16:41:05.273708+0000","last_became_peered":"2023-11-08T16:41:05.273708+0000","last_unstale":"2023-11-08T16:41:06.293080+0000","last_undegraded":"2023-11-08T16:41:06.293080+0000","last_fullsized":"2023-11-08T16:41:06.293080+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:23.456254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290771+0000","last_change":"2023-11-08T16:41:06.288502+0000","last_active":"2023-11-08T16:41:06.290771+0000","last_peered":"2023-11-08T16:41:06.290771+0000","last_clean":"2023-11-08T16:41:06.290771+0000","last_became_active":"2023-11-08T16:41:06.288165+0000","last_became_peered":"2023-11-08T16:41:06.288165+0000","last_unstale":"2023-11-08T16:41:06.290771+0000","last_undegraded":"2023-11-08T16:41:06.290771+0000","last_fullsized":"2023-11-08T16:41:06.290771+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:10:58.442996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.257711+0000","last_change":"2023-11-08T16:41:05.274096+0000","last_active":"2023-11-08T16:41:06.257711+0000","last_peered":"2023-11-08T16:41:06.257711+0000","last_clean":"2023-11-08T16:41:06.257711+0000","last_became_active":"2023-11-08T16:41:05.273591+0000","last_became_peered":"2023-11-08T16:41:05.273591+0000","last_unstale":"2023-11-08T16:41:06.257711+0000","last_undegraded":"2023-11-08T16:41:06.257711+0000","last_fullsized":"2023-11-08T16:41:06.257711+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:34:25.857434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.286786+0000","last_change":"2023-11-08T16:41:06.278021+0000","last_active":"2023-11-08T16:41:06.286786+0000","last_peered":"2023-11-08T16:41:06.286786+0000","last_clean":"2023-11-08T16:41:06.286786+0000","last_became_active":"2023-11-08T16:41:06.276696+0000","last_became_peered":"2023-11-08T16:41:06.276696+0000","last_unstale":"2023-11-08T16:41:06.286786+0000","last_undegraded":"2023-11-08T16:41:06.286786+0000","last_fullsized":"2023-11-08T16:41:06.286786+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:22:54.921237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"126'8","reported_seq":19,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.302490+0000","last_change":"2023-11-08T16:41:05.277630+0000","last_active":"2023-11-08T16:41:06.302490+0000","last_peered":"2023-11-08T16:41:06.302490+0000","last_clean":"2023-11-08T16:41:06.302490+0000","last_became_active":"2023-11-08T16:41:05.277385+0000","last_became_peered":"2023-11-08T16:41:05.277385+0000","last_unstale":"2023-11-08T16:41:06.302490+0000","last_undegraded":"2023-11-08T16:41:06.302490+0000","last_fullsized":"2023-11-08T16:41:06.302490+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:36:27.156278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.300059+0000","last_change":"2023-11-08T16:41:06.295988+0000","last_active":"2023-11-08T16:41:06.300059+0000","last_peered":"2023-11-08T16:41:06.300059+0000","last_clean":"2023-11-08T16:41:06.300059+0000","last_became_active":"2023-11-08T16:41:06.294174+0000","last_became_peered":"2023-11-08T16:41:06.294174+0000","last_unstale":"2023-11-08T16:41:06.300059+0000","last_undegraded":"2023-11-08T16:41:06.300059+0000","last_fullsized":"2023-11-08T16:41:06.300059+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:31:58.580635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"126'1","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.301494+0000","last_change":"2023-11-08T16:41:05.280216+0000","last_active":"2023-11-08T16:41:06.301494+0000","last_peered":"2023-11-08T16:41:06.301494+0000","last_clean":"2023-11-08T16:41:06.301494+0000","last_became_active":"2023-11-08T16:41:05.279862+0000","last_became_peered":"2023-11-08T16:41:05.279862+0000","last_unstale":"2023-11-08T16:41:06.301494+0000","last_undegraded":"2023-11-08T16:41:06.301494+0000","last_fullsized":"2023-11-08T16:41:06.301494+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:06:35.334785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.303375+0000","last_change":"2023-11-08T16:41:06.300698+0000","last_active":"2023-11-08T16:41:06.303375+0000","last_peered":"2023-11-08T16:41:06.303375+0000","last_clean":"2023-11-08T16:41:06.303375+0000","last_became_active":"2023-11-08T16:41:06.300426+0000","last_became_peered":"2023-11-08T16:41:06.300426+0000","last_unstale":"2023-11-08T16:41:06.303375+0000","last_undegraded":"2023-11-08T16:41:06.303375+0000","last_fullsized":"2023-11-08T16:41:06.303375+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:52:57.192009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.294997+0000","last_change":"2023-11-08T16:41:05.281131+0000","last_active":"2023-11-08T16:41:06.294997+0000","last_peered":"2023-11-08T16:41:06.294997+0000","last_clean":"2023-11-08T16:41:06.294997+0000","last_became_active":"2023-11-08T16:41:05.280861+0000","last_became_peered":"2023-11-08T16:41:05.280861+0000","last_unstale":"2023-11-08T16:41:06.294997+0000","last_undegraded":"2023-11-08T16:41:06.294997+0000","last_fullsized":"2023-11-08T16:41:06.294997+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:03:16.579687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.296115+0000","last_change":"2023-11-08T16:41:06.295497+0000","last_active":"2023-11-08T16:41:06.296115+0000","last_peered":"2023-11-08T16:41:06.296115+0000","last_clean":"2023-11-08T16:41:06.296115+0000","last_became_active":"2023-11-08T16:41:06.295216+0000","last_became_peered":"2023-11-08T16:41:06.295216+0000","last_unstale":"2023-11-08T16:41:06.296115+0000","last_undegraded":"2023-11-08T16:41:06.296115+0000","last_fullsized":"2023-11-08T16:41:06.296115+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:46:22.237288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"126'1","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.288099+0000","last_change":"2023-11-08T16:41:05.276756+0000","last_active":"2023-11-08T16:41:06.288099+0000","last_peered":"2023-11-08T16:41:06.288099+0000","last_clean":"2023-11-08T16:41:06.288099+0000","last_became_active":"2023-11-08T16:41:05.275622+0000","last_became_peered":"2023-11-08T16:41:05.275622+0000","last_unstale":"2023-11-08T16:41:06.288099+0000","last_undegraded":"2023-11-08T16:41:06.288099+0000","last_fullsized":"2023-11-08T16:41:06.288099+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T02:29:24.767585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.292899+0000","last_change":"2023-11-08T16:41:06.290815+0000","last_active":"2023-11-08T16:41:06.292899+0000","last_peered":"2023-11-08T16:41:06.292899+0000","last_clean":"2023-11-08T16:41:06.292899+0000","last_became_active":"2023-11-08T16:41:06.282718+0000","last_became_peered":"2023-11-08T16:41:06.282718+0000","last_unstale":"2023-11-08T16:41:06.292899+0000","last_undegraded":"2023-11-08T16:41:06.292899+0000","last_fullsized":"2023-11-08T16:41:06.292899+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:24.472729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.263176+0000","last_change":"2023-11-08T16:41:05.281320+0000","last_active":"2023-11-08T16:41:06.263176+0000","last_peered":"2023-11-08T16:41:06.263176+0000","last_clean":"2023-11-08T16:41:06.263176+0000","last_became_active":"2023-11-08T16:41:05.281003+0000","last_became_peered":"2023-11-08T16:41:05.281003+0000","last_unstale":"2023-11-08T16:41:06.263176+0000","last_undegraded":"2023-11-08T16:41:06.263176+0000","last_fullsized":"2023-11-08T16:41:06.263176+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:00:13.379826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.292886+0000","last_change":"2023-11-08T16:41:06.290606+0000","last_active":"2023-11-08T16:41:06.292886+0000","last_peered":"2023-11-08T16:41:06.292886+0000","last_clean":"2023-11-08T16:41:06.292886+0000","last_became_active":"2023-11-08T16:41:06.286415+0000","last_became_peered":"2023-11-08T16:41:06.286415+0000","last_unstale":"2023-11-08T16:41:06.292886+0000","last_undegraded":"2023-11-08T16:41:06.292886+0000","last_fullsized":"2023-11-08T16:41:06.292886+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:58:55.980017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290575+0000","last_change":"2023-11-08T16:41:05.276394+0000","last_active":"2023-11-08T16:41:06.290575+0000","last_peered":"2023-11-08T16:41:06.290575+0000","last_clean":"2023-11-08T16:41:06.290575+0000","last_became_active":"2023-11-08T16:41:05.275441+0000","last_became_peered":"2023-11-08T16:41:05.275441+0000","last_unstale":"2023-11-08T16:41:06.290575+0000","last_undegraded":"2023-11-08T16:41:06.290575+0000","last_fullsized":"2023-11-08T16:41:06.290575+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:45:57.399931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.303003+0000","last_change":"2023-11-08T16:41:06.300043+0000","last_active":"2023-11-08T16:41:06.303003+0000","last_peered":"2023-11-08T16:41:06.303003+0000","last_clean":"2023-11-08T16:41:06.303003+0000","last_became_active":"2023-11-08T16:41:06.298286+0000","last_became_peered":"2023-11-08T16:41:06.298286+0000","last_unstale":"2023-11-08T16:41:06.303003+0000","last_undegraded":"2023-11-08T16:41:06.303003+0000","last_fullsized":"2023-11-08T16:41:06.303003+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:09:02.726372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.264947+0000","last_change":"2023-11-08T16:41:05.267580+0000","last_active":"2023-11-08T16:41:06.264947+0000","last_peered":"2023-11-08T16:41:06.264947+0000","last_clean":"2023-11-08T16:41:06.264947+0000","last_became_active":"2023-11-08T16:41:05.267292+0000","last_became_peered":"2023-11-08T16:41:05.267292+0000","last_unstale":"2023-11-08T16:41:06.264947+0000","last_undegraded":"2023-11-08T16:41:06.264947+0000","last_fullsized":"2023-11-08T16:41:06.264947+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:07:24.245421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291198+0000","last_change":"2023-11-08T16:41:06.287414+0000","last_active":"2023-11-08T16:41:06.291198+0000","last_peered":"2023-11-08T16:41:06.291198+0000","last_clean":"2023-11-08T16:41:06.291198+0000","last_became_active":"2023-11-08T16:41:06.286596+0000","last_became_peered":"2023-11-08T16:41:06.286596+0000","last_unstale":"2023-11-08T16:41:06.291198+0000","last_undegraded":"2023-11-08T16:41:06.291198+0000","last_fullsized":"2023-11-08T16:41:06.291198+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:07:10.051108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.265043+0000","last_change":"2023-11-08T16:41:05.277292+0000","last_active":"2023-11-08T16:41:06.265043+0000","last_peered":"2023-11-08T16:41:06.265043+0000","last_clean":"2023-11-08T16:41:06.265043+0000","last_became_active":"2023-11-08T16:41:05.276983+0000","last_became_peered":"2023-11-08T16:41:05.276983+0000","last_unstale":"2023-11-08T16:41:06.265043+0000","last_undegraded":"2023-11-08T16:41:06.265043+0000","last_fullsized":"2023-11-08T16:41:06.265043+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T18:39:20.555676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.296799+0000","last_change":"2023-11-08T16:41:06.295080+0000","last_active":"2023-11-08T16:41:06.296799+0000","last_peered":"2023-11-08T16:41:06.296799+0000","last_clean":"2023-11-08T16:41:06.296799+0000","last_became_active":"2023-11-08T16:41:06.294796+0000","last_became_peered":"2023-11-08T16:41:06.294796+0000","last_unstale":"2023-11-08T16:41:06.296799+0000","last_undegraded":"2023-11-08T16:41:06.296799+0000","last_fullsized":"2023-11-08T16:41:06.296799+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:24.472729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.285130+0000","last_change":"2023-11-08T16:41:05.282283+0000","last_active":"2023-11-08T16:41:06.285130+0000","last_peered":"2023-11-08T16:41:06.285130+0000","last_clean":"2023-11-08T16:41:06.285130+0000","last_became_active":"2023-11-08T16:41:05.282009+0000","last_became_peered":"2023-11-08T16:41:05.282009+0000","last_unstale":"2023-11-08T16:41:06.285130+0000","last_undegraded":"2023-11-08T16:41:06.285130+0000","last_fullsized":"2023-11-08T16:41:06.285130+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T20:10:49.216473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.294430+0000","last_change":"2023-11-08T16:41:06.293933+0000","last_active":"2023-11-08T16:41:06.294430+0000","last_peered":"2023-11-08T16:41:06.294430+0000","last_clean":"2023-11-08T16:41:06.294430+0000","last_became_active":"2023-11-08T16:41:06.293031+0000","last_became_peered":"2023-11-08T16:41:06.293031+0000","last_unstale":"2023-11-08T16:41:06.294430+0000","last_undegraded":"2023-11-08T16:41:06.294430+0000","last_fullsized":"2023-11-08T16:41:06.294430+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:58:55.980017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"126'2","reported_seq":13,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.299821+0000","last_change":"2023-11-08T16:41:05.279231+0000","last_active":"2023-11-08T16:41:06.299821+0000","last_peered":"2023-11-08T16:41:06.299821+0000","last_clean":"2023-11-08T16:41:06.299821+0000","last_became_active":"2023-11-08T16:41:05.278993+0000","last_became_peered":"2023-11-08T16:41:05.278993+0000","last_unstale":"2023-11-08T16:41:06.299821+0000","last_undegraded":"2023-11-08T16:41:06.299821+0000","last_fullsized":"2023-11-08T16:41:06.299821+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:47:40.178540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.301426+0000","last_change":"2023-11-08T16:41:06.294046+0000","last_active":"2023-11-08T16:41:06.301426+0000","last_peered":"2023-11-08T16:41:06.301426+0000","last_clean":"2023-11-08T16:41:06.301426+0000","last_became_active":"2023-11-08T16:41:06.293365+0000","last_became_peered":"2023-11-08T16:41:06.293365+0000","last_unstale":"2023-11-08T16:41:06.301426+0000","last_undegraded":"2023-11-08T16:41:06.301426+0000","last_fullsized":"2023-11-08T16:41:06.301426+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:27:51.816735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.258583+0000","last_change":"2023-11-08T16:41:05.277465+0000","last_active":"2023-11-08T16:41:06.258583+0000","last_peered":"2023-11-08T16:41:06.258583+0000","last_clean":"2023-11-08T16:41:06.258583+0000","last_became_active":"2023-11-08T16:41:05.277059+0000","last_became_peered":"2023-11-08T16:41:05.277059+0000","last_unstale":"2023-11-08T16:41:06.258583+0000","last_undegraded":"2023-11-08T16:41:06.258583+0000","last_fullsized":"2023-11-08T16:41:06.258583+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:37:33.375161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.300954+0000","last_change":"2023-11-08T16:41:06.291247+0000","last_active":"2023-11-08T16:41:06.300954+0000","last_peered":"2023-11-08T16:41:06.300954+0000","last_clean":"2023-11-08T16:41:06.300954+0000","last_became_active":"2023-11-08T16:41:06.290304+0000","last_became_peered":"2023-11-08T16:41:06.290304+0000","last_unstale":"2023-11-08T16:41:06.300954+0000","last_undegraded":"2023-11-08T16:41:06.300954+0000","last_fullsized":"2023-11-08T16:41:06.300954+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:59:27.952350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.297070+0000","last_change":"2023-11-08T16:41:05.281703+0000","last_active":"2023-11-08T16:41:06.297070+0000","last_peered":"2023-11-08T16:41:06.297070+0000","last_clean":"2023-11-08T16:41:06.297070+0000","last_became_active":"2023-11-08T16:41:05.281137+0000","last_became_peered":"2023-11-08T16:41:05.281137+0000","last_unstale":"2023-11-08T16:41:06.297070+0000","last_undegraded":"2023-11-08T16:41:06.297070+0000","last_fullsized":"2023-11-08T16:41:06.297070+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T16:49:59.086294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.302778+0000","last_change":"2023-11-08T16:41:06.301967+0000","last_active":"2023-11-08T16:41:06.302778+0000","last_peered":"2023-11-08T16:41:06.302778+0000","last_clean":"2023-11-08T16:41:06.302778+0000","last_became_active":"2023-11-08T16:41:06.301744+0000","last_became_peered":"2023-11-08T16:41:06.301744+0000","last_unstale":"2023-11-08T16:41:06.302778+0000","last_undegraded":"2023-11-08T16:41:06.302778+0000","last_fullsized":"2023-11-08T16:41:06.302778+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:07:13.421490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.264954+0000","last_change":"2023-11-08T16:41:05.277639+0000","last_active":"2023-11-08T16:41:06.264954+0000","last_peered":"2023-11-08T16:41:06.264954+0000","last_clean":"2023-11-08T16:41:06.264954+0000","last_became_active":"2023-11-08T16:41:05.277362+0000","last_became_peered":"2023-11-08T16:41:05.277362+0000","last_unstale":"2023-11-08T16:41:06.264954+0000","last_undegraded":"2023-11-08T16:41:06.264954+0000","last_fullsized":"2023-11-08T16:41:06.264954+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:24:51.860834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.287649+0000","last_change":"2023-11-08T16:41:06.281767+0000","last_active":"2023-11-08T16:41:06.287649+0000","last_peered":"2023-11-08T16:41:06.287649+0000","last_clean":"2023-11-08T16:41:06.287649+0000","last_became_active":"2023-11-08T16:41:06.281422+0000","last_became_peered":"2023-11-08T16:41:06.281422+0000","last_unstale":"2023-11-08T16:41:06.287649+0000","last_undegraded":"2023-11-08T16:41:06.287649+0000","last_fullsized":"2023-11-08T16:41:06.287649+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T21:15:24.472729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.262980+0000","last_change":"2023-11-08T16:41:05.279443+0000","last_active":"2023-11-08T16:41:06.262980+0000","last_peered":"2023-11-08T16:41:06.262980+0000","last_clean":"2023-11-08T16:41:06.262980+0000","last_became_active":"2023-11-08T16:41:05.278965+0000","last_became_peered":"2023-11-08T16:41:05.278965+0000","last_unstale":"2023-11-08T16:41:06.262980+0000","last_undegraded":"2023-11-08T16:41:06.262980+0000","last_fullsized":"2023-11-08T16:41:06.262980+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T01:39:02.918253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290759+0000","last_change":"2023-11-08T16:41:06.286651+0000","last_active":"2023-11-08T16:41:06.290759+0000","last_peered":"2023-11-08T16:41:06.290759+0000","last_clean":"2023-11-08T16:41:06.290759+0000","last_became_active":"2023-11-08T16:41:06.286166+0000","last_became_peered":"2023-11-08T16:41:06.286166+0000","last_unstale":"2023-11-08T16:41:06.290759+0000","last_undegraded":"2023-11-08T16:41:06.290759+0000","last_fullsized":"2023-11-08T16:41:06.290759+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:24:52.877309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.274651+0000","last_change":"2023-11-08T16:41:05.276633+0000","last_active":"2023-11-08T16:41:06.274651+0000","last_peered":"2023-11-08T16:41:06.274651+0000","last_clean":"2023-11-08T16:41:06.274651+0000","last_became_active":"2023-11-08T16:41:05.276312+0000","last_became_peered":"2023-11-08T16:41:05.276312+0000","last_unstale":"2023-11-08T16:41:06.274651+0000","last_undegraded":"2023-11-08T16:41:06.274651+0000","last_fullsized":"2023-11-08T16:41:06.274651+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T19:51:31.955399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.291215+0000","last_change":"2023-11-08T16:41:06.287582+0000","last_active":"2023-11-08T16:41:06.291215+0000","last_peered":"2023-11-08T16:41:06.291215+0000","last_clean":"2023-11-08T16:41:06.291215+0000","last_became_active":"2023-11-08T16:41:06.286681+0000","last_became_peered":"2023-11-08T16:41:06.286681+0000","last_unstale":"2023-11-08T16:41:06.291215+0000","last_undegraded":"2023-11-08T16:41:06.291215+0000","last_fullsized":"2023-11-08T16:41:06.291215+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:39:32.914205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.264852+0000","last_change":"2023-11-08T16:41:05.266581+0000","last_active":"2023-11-08T16:41:06.264852+0000","last_peered":"2023-11-08T16:41:06.264852+0000","last_clean":"2023-11-08T16:41:06.264852+0000","last_became_active":"2023-11-08T16:41:05.266299+0000","last_became_peered":"2023-11-08T16:41:05.266299+0000","last_unstale":"2023-11-08T16:41:06.264852+0000","last_undegraded":"2023-11-08T16:41:06.264852+0000","last_fullsized":"2023-11-08T16:41:06.264852+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T17:18:33.315259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.302652+0000","last_change":"2023-11-08T16:41:06.298027+0000","last_active":"2023-11-08T16:41:06.302652+0000","last_peered":"2023-11-08T16:41:06.302652+0000","last_clean":"2023-11-08T16:41:06.302652+0000","last_became_active":"2023-11-08T16:41:06.297229+0000","last_became_peered":"2023-11-08T16:41:06.297229+0000","last_unstale":"2023-11-08T16:41:06.302652+0000","last_undegraded":"2023-11-08T16:41:06.302652+0000","last_fullsized":"2023-11-08T16:41:06.302652+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T00:55:11.606257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.285725+0000","last_change":"2023-11-08T16:41:05.282519+0000","last_active":"2023-11-08T16:41:06.285725+0000","last_peered":"2023-11-08T16:41:06.285725+0000","last_clean":"2023-11-08T16:41:06.285725+0000","last_became_active":"2023-11-08T16:41:05.282239+0000","last_became_peered":"2023-11-08T16:41:05.282239+0000","last_unstale":"2023-11-08T16:41:06.285725+0000","last_undegraded":"2023-11-08T16:41:06.285725+0000","last_fullsized":"2023-11-08T16:41:06.285725+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T23:08:39.031870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.290144+0000","last_change":"2023-11-08T16:41:06.283632+0000","last_active":"2023-11-08T16:41:06.290144+0000","last_peered":"2023-11-08T16:41:06.290144+0000","last_clean":"2023-11-08T16:41:06.290144+0000","last_became_active":"2023-11-08T16:41:06.283387+0000","last_became_peered":"2023-11-08T16:41:06.283387+0000","last_unstale":"2023-11-08T16:41:06.290144+0000","last_undegraded":"2023-11-08T16:41:06.290144+0000","last_fullsized":"2023-11-08T16:41:06.290144+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":125,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:05.233573+0000","last_clean_scrub_stamp":"2023-11-08T16:41:05.233573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-09T22:16:55.434687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":126,"state":"active+clean","last_fresh":"2023-11-08T16:41:06.264756+0000","last_change":"2023-11-08T16:41:05.274649+0000","last_active":"2023-11-08T16:41:06.264756+0000","last_peered":"2023-11-08T16:41:06.264756+0000","last_clean":"2023-11-08T16:41:06.264756+0000","last_became_active":"2023-11-08T16:41:05.274349+0000","last_became_peered":"2023-11-08T16:41:05.274349+0000","last_unstale":"2023-11-08T16:41:06.264756+0000","last_undegraded":"2023-11-08T16:41:06.264756+0000","last_fullsized":"2023-11-08T16:41:06.264756+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:41:04.217097+0000","last_clean_scrub_stamp":"2023-11-08T16:41:04.217097+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:07:12.405014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":276,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-08T16:41:05.266078+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:41:05.266078+0000","last_peered":"2023-11-08T16:41:05.266078+0000","last_clean":"2023-11-08T16:41:05.266078+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:41:05.266078+0000","last_undegraded":"2023-11-08T16:41:05.266078+0000","last_fullsized":"2023-11-08T16:41:05.266078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2146304,"data_stored":1310550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":1.0429999999999999,"15min":1.0740000000000001},"min":{"1min":0.60699999999999998,"5min":0.56399999999999995,"15min":0.49299999999999999},"max":{"1min":1.5600000000000001,"5min":1.6879999999999999,"15min":3.1899999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.032,"5min":1.0760000000000001,"15min":1.105},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.53600000000000003},"max":{"1min":1.6040000000000001,"5min":1.7410000000000001,"15min":4.2729999999999997},"last":0.874}]},{"osd":1,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.107,"15min":1.1160000000000001},"min":{"1min":0.81899999999999995,"5min":0.61599999999999999,"15min":0.57899999999999996},"max":{"1min":1.5580000000000001,"5min":1.891,"15min":1.891},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.1879999999999999,"15min":1.204},"min":{"1min":0.61199999999999999,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.6020000000000001,"5min":2.1419999999999999,"15min":3.3740000000000001},"last":1.0169999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.159,"15min":1.2190000000000001},"min":{"1min":0.79800000000000004,"5min":0.65900000000000003,"15min":0.58399999999999996},"max":{"1min":1.4159999999999999,"5min":1.8109999999999999,"15min":4.2249999999999996},"last":1.21},{"interface":"front","average":{"1min":1.246,"5min":1.2450000000000001,"15min":1.222},"min":{"1min":0.85599999999999998,"5min":0.66000000000000003,"15min":0.56299999999999994},"max":{"1min":1.6990000000000001,"5min":1.7010000000000001,"15min":3.4750000000000001},"last":1.0800000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.1539999999999999,"15min":1.167},"min":{"1min":0.79800000000000004,"5min":0.70299999999999996,"15min":0.54900000000000004},"max":{"1min":1.631,"5min":1.631,"15min":2.964},"last":1.571},{"interface":"front","average":{"1min":1.351,"5min":1.365,"15min":1.4139999999999999},"min":{"1min":0.88700000000000001,"5min":0.86899999999999999,"15min":0.58599999999999997},"max":{"1min":1.698,"5min":2.25,"15min":4.181},"last":1.3260000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.206,"15min":1.2290000000000001},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.77200000000000002},"max":{"1min":1.7789999999999999,"5min":1.7789999999999999,"15min":3.6669999999999998},"last":1.778},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.325,"15min":1.3360000000000001},"min":{"1min":0.95499999999999996,"5min":0.77300000000000002,"15min":0.59099999999999997},"max":{"1min":1.865,"5min":2.0179999999999998,"15min":2.3730000000000002},"last":1.3600000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.3919999999999999,"15min":1.3759999999999999},"min":{"1min":0.96399999999999997,"5min":0.89200000000000002,"15min":0.61799999999999999},"max":{"1min":1.885,"5min":2.3559999999999999,"15min":3.1059999999999999},"last":1.8859999999999999},{"interface":"front","average":{"1min":1.4350000000000001,"5min":1.4530000000000001,"15min":1.4550000000000001},"min":{"1min":1.1399999999999999,"5min":1.0589999999999999,"15min":0.878},"max":{"1min":1.6870000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.462}]},{"osd":6,"last update":"Wed Nov 8 16:40:20 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.526,"15min":1.5309999999999999},"min":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":0.94799999999999995},"max":{"1min":1.8600000000000001,"5min":1.919,"15min":3.7530000000000001},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.4610000000000001,"15min":1.482},"min":{"1min":1.2310000000000001,"5min":0.89900000000000002,"15min":0.73999999999999999},"max":{"1min":1.786,"5min":2.0430000000000001,"15min":6.9560000000000004},"last":1.831}]}]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":720182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0609999999999999,"15min":1.0629999999999999},"min":{"1min":0.69799999999999995,"5min":0.58599999999999997,"15min":0.497},"max":{"1min":1.6100000000000001,"5min":1.7230000000000001,"15min":2.5609999999999999},"last":1.569},{"interface":"front","average":{"1min":1.218,"5min":1.133,"15min":1.095},"min":{"1min":0.75800000000000001,"5min":0.63100000000000001,"15min":0.56200000000000006},"max":{"1min":1.6279999999999999,"5min":1.6819999999999999,"15min":2.7519999999999998},"last":1.1479999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1259999999999999,"15min":1.079},"min":{"1min":0.72299999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.5880000000000001,"5min":1.667,"15min":1.667},"last":1.097},{"interface":"front","average":{"1min":1.194,"5min":1.143,"15min":1.131},"min":{"1min":0.92200000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.714,"5min":1.7290000000000001,"15min":1.97},"last":1.5049999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.181,"15min":1.1830000000000001},"min":{"1min":0.77000000000000002,"5min":0.71599999999999997,"15min":0.56799999999999995},"max":{"1min":1.6040000000000001,"5min":1.698,"15min":1.804},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.2030000000000001,"15min":1.171},"min":{"1min":0.73399999999999999,"5min":0.72299999999999998,"15min":0.60499999999999998},"max":{"1min":1.6259999999999999,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.607}]},{"osd":3,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.224,"15min":1.2450000000000001},"min":{"1min":0.87,"5min":0.63200000000000001,"15min":0.58299999999999996},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":2.1120000000000001},"last":1.282},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.2809999999999999,"15min":1.288},"min":{"1min":0.70899999999999996,"5min":0.68200000000000005,"15min":0.67300000000000004},"max":{"1min":1.7809999999999999,"5min":1.7809999999999999,"15min":1.95},"last":1.47}]},{"osd":4,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.321,"15min":1.3400000000000001},"min":{"1min":1.143,"5min":0.81799999999999995,"15min":0.65300000000000002},"max":{"1min":1.657,"5min":1.778,"15min":2.218},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.3380000000000001,"5min":1.367,"15min":1.3979999999999999},"min":{"1min":0.878,"5min":0.86899999999999999,"15min":0.64900000000000002},"max":{"1min":1.681,"5min":1.9590000000000001,"15min":2.7080000000000002},"last":1.331}]},{"osd":5,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3380000000000001,"15min":1.3240000000000001},"min":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.80500000000000005},"max":{"1min":1.8200000000000001,"5min":1.994,"15min":2.105},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.548,"5min":1.512,"15min":1.534},"min":{"1min":0.98599999999999999,"5min":0.91100000000000003,"15min":0.78500000000000003},"max":{"1min":1.833,"5min":2.1280000000000001,"15min":2.4700000000000002},"last":1.6599999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.51,"15min":1.5600000000000001},"min":{"1min":1.157,"5min":1.002,"15min":0.74299999999999999},"max":{"1min":1.95,"5min":2.1309999999999998,"15min":3.8959999999999999},"last":1.536},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.504,"15min":1.5049999999999999},"min":{"1min":1.105,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.843,"5min":2.0870000000000002,"15min":2.4889999999999999},"last":1.4319999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32716,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339124,"statfs":{"total":96636764160,"available":96603262976,"internally_reserved":0,"allocated":1572864,"data_stored":727561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.089,"15min":1.0720000000000001},"min":{"1min":0.63400000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.536,"5min":1.639,"15min":2.4609999999999999},"last":1.248},{"interface":"front","average":{"1min":1.125,"5min":1.127,"15min":1.1120000000000001},"min":{"1min":0.71499999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.4930000000000001,"5min":1.6000000000000001,"15min":2.3639999999999999},"last":1.474}]},{"osd":1,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.244,"15min":1.238},"min":{"1min":0.89900000000000002,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.609,"5min":1.9610000000000001,"15min":3.8690000000000002},"last":1.554},{"interface":"front","average":{"1min":1.375,"5min":1.238,"15min":1.2509999999999999},"min":{"1min":0.68899999999999995,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.8109999999999999,"5min":1.9370000000000001,"15min":3.2349999999999999},"last":1.423}]},{"osd":2,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.1499999999999999,"15min":1.1930000000000001},"min":{"1min":0.753,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6499999999999999,"5min":1.764,"15min":2.6339999999999999},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.1739999999999999,"5min":1.206,"15min":1.2},"min":{"1min":0.78300000000000003,"5min":0.70499999999999996,"15min":0.65800000000000003},"max":{"1min":1.752,"5min":1.752,"15min":2.2559999999999998},"last":0.89100000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2809999999999999,"15min":1.2789999999999999},"min":{"1min":0.97999999999999998,"5min":0.90900000000000003,"15min":0.66500000000000004},"max":{"1min":1.575,"5min":1.681,"15min":2.8330000000000002},"last":1.123},{"interface":"front","average":{"1min":1.306,"5min":1.3140000000000001,"15min":1.2969999999999999},"min":{"1min":1.0569999999999999,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.5549999999999999,"5min":2.0470000000000002,"15min":2.5760000000000001},"last":1.363}]},{"osd":4,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.2869999999999999,"15min":1.325},"min":{"1min":0.94899999999999995,"5min":0.872,"15min":0.61599999999999999},"max":{"1min":2.0019999999999998,"5min":2.0790000000000002,"15min":3.9790000000000001},"last":1.657},{"interface":"front","average":{"1min":1.325,"5min":1.401,"15min":1.4370000000000001},"min":{"1min":1.0680000000000001,"5min":1.0569999999999999,"15min":0.81499999999999995},"max":{"1min":1.696,"5min":2.0230000000000001,"15min":3.0209999999999999},"last":1.579}]},{"osd":6,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4339999999999999,"15min":1.415},"min":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.752},"max":{"1min":1.9379999999999999,"5min":2.1560000000000001,"15min":2.5550000000000002},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.425,"15min":1.464},"min":{"1min":1.012,"5min":0.98499999999999999,"15min":0.93600000000000005},"max":{"1min":1.9419999999999999,"5min":2.1619999999999999,"15min":4.1189999999999998},"last":1.514}]},{"osd":7,"last update":"Wed Nov 8 16:40:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5089999999999999,"5min":1.5229999999999999,"15min":1.546},"min":{"1min":1.21,"5min":1.21,"15min":1.0049999999999999},"max":{"1min":1.792,"5min":2.129,"15min":3.1160000000000001},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.526,"5min":1.5760000000000001,"15min":1.5549999999999999},"min":{"1min":1.075,"5min":1.075,"15min":0.96999999999999997},"max":{"1min":1.8580000000000001,"5min":2.3660000000000001,"15min":2.7050000000000001},"last":1.6910000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":726957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.73799999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4830000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.997},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.127,"15min":1.087},"min":{"1min":0.69599999999999995,"5min":0.58299999999999996,"15min":0.45400000000000001},"max":{"1min":1.591,"5min":1.772,"15min":1.772},"last":1.278}]},{"osd":1,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.107,"15min":1.123},"min":{"1min":0.624,"5min":0.60099999999999998,"15min":0.56999999999999995},"max":{"1min":1.3819999999999999,"5min":1.51,"15min":2.0459999999999998},"last":0.95099999999999996},{"interface":"front","average":{"1min":1.1140000000000001,"5min":1.1559999999999999,"15min":1.165},"min":{"1min":0.66800000000000004,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.577,"5min":1.577,"15min":1.742},"last":1.099}]},{"osd":2,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.236,"15min":1.212},"min":{"1min":0.85899999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.6930000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.226,"5min":1.214,"15min":1.246},"min":{"1min":0.877,"5min":0.78100000000000003,"15min":0.71699999999999997},"max":{"1min":1.6659999999999999,"5min":1.7749999999999999,"15min":1.9319999999999999},"last":1.3220000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.306,"15min":1.2829999999999999},"min":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.911,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.2589999999999999,"15min":1.274},"min":{"1min":0.94899999999999995,"5min":0.72499999999999998,"15min":0.65100000000000002},"max":{"1min":1.611,"5min":1.8069999999999999,"15min":1.895},"last":1.2310000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3100000000000001,"5min":1.3640000000000001,"15min":1.363},"min":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.58799999999999997},"max":{"1min":1.698,"5min":2.286,"15min":2.286},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.472,"15min":1.4339999999999999},"min":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.61899999999999999},"max":{"1min":1.821,"5min":2.1970000000000001,"15min":2.46},"last":1.538}]},{"osd":6,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.377,"15min":1.395},"min":{"1min":1.0740000000000001,"5min":0.88,"15min":0.74299999999999999},"max":{"1min":1.75,"5min":1.7769999999999999,"15min":2.5609999999999999},"last":1.593},{"interface":"front","average":{"1min":1.478,"5min":1.5029999999999999,"15min":1.53},"min":{"1min":1.038,"5min":1.032,"15min":0.81200000000000006},"max":{"1min":1.873,"5min":2.4780000000000002,"15min":2.6389999999999998},"last":1.833}]},{"osd":7,"last update":"Wed Nov 8 16:40:11 2023","interfaces":[{"interface":"back","average":{"1min":1.607,"5min":1.556,"15min":1.534},"min":{"1min":1.272,"5min":1.056,"15min":0.92300000000000004},"max":{"1min":1.9750000000000001,"5min":2.1040000000000001,"15min":2.2639999999999998},"last":1.649},{"interface":"front","average":{"1min":1.5980000000000001,"5min":1.542,"15min":1.5669999999999999},"min":{"1min":1.25,"5min":1.117,"15min":0.84799999999999998},"max":{"1min":1.847,"5min":2.1070000000000002,"15min":2.6859999999999999},"last":1.718}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":720182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82099999999999995,"15min":0.81399999999999995},"min":{"1min":0.45200000000000001,"5min":0.436,"15min":0.33800000000000002},"max":{"1min":1.304,"5min":1.4930000000000001,"15min":1.9339999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.98999999999999999,"5min":0.90400000000000003,"15min":0.90200000000000002},"min":{"1min":0.48899999999999999,"5min":0.45000000000000001,"15min":0.316},"max":{"1min":1.486,"5min":1.647,"15min":1.647},"last":0.58999999999999997}]},{"osd":1,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.90100000000000002,"15min":0.89100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.36499999999999999},"max":{"1min":1.6120000000000001,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.97699999999999998,"15min":0.97099999999999997},"min":{"1min":0.52500000000000002,"5min":0.48299999999999998,"15min":0.378},"max":{"1min":1.4610000000000001,"5min":1.5800000000000001,"15min":1.829},"last":0.749}]},{"osd":2,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.046,"15min":1.008},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.41699999999999998},"max":{"1min":1.6479999999999999,"5min":1.7490000000000001,"15min":2.0779999999999998},"last":0.63300000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.087,"15min":1.1120000000000001},"min":{"1min":0.55500000000000005,"5min":0.504,"15min":0.504},"max":{"1min":1.595,"5min":1.7270000000000001,"15min":2.4660000000000002},"last":1.095}]},{"osd":4,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2090000000000001,"5min":1.161,"15min":1.1779999999999999},"min":{"1min":0.90700000000000003,"5min":0.73199999999999998,"15min":0.56399999999999995},"max":{"1min":1.399,"5min":1.5720000000000001,"15min":2.25},"last":1.129},{"interface":"front","average":{"1min":1.25,"5min":1.2829999999999999,"15min":1.27},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.823,"5min":2.0110000000000001,"15min":2.1829999999999998},"last":1.008}]},{"osd":5,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3440000000000001,"15min":1.329},"min":{"1min":0.94299999999999995,"5min":0.88700000000000001,"15min":0.68799999999999994},"max":{"1min":1.8109999999999999,"5min":2.2229999999999999,"15min":2.3999999999999999},"last":0.94299999999999995},{"interface":"front","average":{"1min":1.278,"5min":1.3280000000000001,"15min":1.331},"min":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.60999999999999999},"max":{"1min":1.887,"5min":1.887,"15min":2.593},"last":1.377}]},{"osd":6,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.3839999999999999,"15min":1.3819999999999999},"min":{"1min":0.751,"5min":0.751,"15min":0.58799999999999997},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":2.9569999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.323,"15min":1.3500000000000001},"min":{"1min":0.86099999999999999,"5min":0.83599999999999997,"15min":0.73399999999999999},"max":{"1min":1.6910000000000001,"5min":1.8919999999999999,"15min":2.762},"last":1.226}]},{"osd":7,"last update":"Wed Nov 8 16:41:03 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.359,"15min":1.3919999999999999},"min":{"1min":1.0720000000000001,"5min":0.73699999999999999,"15min":0.70799999999999996},"max":{"1min":1.6799999999999999,"5min":1.764,"15min":2.8700000000000001},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.425,"15min":1.419},"min":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.73399999999999999},"max":{"1min":2.0470000000000002,"5min":2.121,"15min":2.633},"last":0.98299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32708,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339132,"statfs":{"total":96636764160,"available":96603271168,"internally_reserved":0,"allocated":1564672,"data_stored":726957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79800000000000004,"15min":0.91400000000000003},"min":{"1min":0.46300000000000002,"5min":0.40100000000000002,"15min":0.24099999999999999},"max":{"1min":1.2290000000000001,"5min":1.2929999999999999,"15min":4.5369999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.872,"15min":0.876},"min":{"1min":0.50900000000000001,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":1.4910000000000001,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.69199999999999995}]},{"osd":1,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.0449999999999999,"15min":1.069},"min":{"1min":0.69699999999999995,"5min":0.44900000000000001,"15min":0.379},"max":{"1min":1.4950000000000001,"5min":1.7689999999999999,"15min":4.4980000000000002},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.091,"15min":1.052},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.40300000000000002},"max":{"1min":1.5740000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.91900000000000004}]},{"osd":3,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.878,"15min":0.96399999999999997},"min":{"1min":0.44800000000000001,"5min":0.437,"15min":0.35499999999999998},"max":{"1min":1.4630000000000001,"5min":1.4870000000000001,"15min":3.3050000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":1.004,"5min":1.0529999999999999,"15min":1.147},"min":{"1min":0.54700000000000004,"5min":0.44800000000000001,"15min":0.42999999999999999},"max":{"1min":1.3109999999999999,"5min":1.819,"15min":4.5899999999999999},"last":0.63800000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.2829999999999999,"15min":1.351},"min":{"1min":0.77200000000000002,"5min":0.74299999999999999,"15min":0.73499999999999999},"max":{"1min":1.6890000000000001,"5min":2.2160000000000002,"15min":4.9409999999999998},"last":0.89600000000000002},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.208,"15min":1.246},"min":{"1min":0.86499999999999999,"5min":0.73899999999999999,"15min":0.63100000000000001},"max":{"1min":1.508,"5min":2.0739999999999998,"15min":2.46},"last":1.2649999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2829999999999999,"15min":1.294},"min":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.77200000000000002},"max":{"1min":1.6040000000000001,"5min":1.9510000000000001,"15min":2.8530000000000002},"last":1.228},{"interface":"front","average":{"1min":1.276,"5min":1.2709999999999999,"15min":1.3129999999999999},"min":{"1min":0.80500000000000005,"5min":0.70499999999999996,"15min":0.69399999999999995},"max":{"1min":1.8380000000000001,"5min":2.5390000000000001,"15min":3.5680000000000001},"last":1.1599999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.284,"15min":1.345},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.625},"max":{"1min":1.7230000000000001,"5min":1.9710000000000001,"15min":3.536},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.347,"5min":1.389,"15min":1.4510000000000001},"min":{"1min":0.95699999999999996,"5min":0.81200000000000006,"15min":0.72199999999999998},"max":{"1min":1.629,"5min":2.2719999999999998,"15min":4.7549999999999999},"last":1.2470000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:40:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.3480000000000001,"15min":1.397},"min":{"1min":0.98199999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6439999999999999,"5min":2.2930000000000001,"15min":2.9700000000000002},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3779999999999999,"15min":1.4410000000000001},"min":{"1min":0.91400000000000003,"5min":0.79600000000000004,"15min":0.70499999999999996},"max":{"1min":1.7789999999999999,"5min":2.129,"15min":3.4950000000000001},"last":1.123}]}]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32700,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339140,"statfs":{"total":96636764160,"available":96603279360,"internally_reserved":0,"allocated":1552384,"data_stored":720182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.76900000000000002,"15min":0.76600000000000001},"min":{"1min":0.438,"5min":0.34499999999999997,"15min":0.28899999999999998},"max":{"1min":1.246,"5min":1.246,"15min":1.542},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.877,"15min":0.95199999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":1.5429999999999999,"5min":1.5429999999999999,"15min":4.0309999999999997},"last":0.45600000000000002}]},{"osd":2,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.999,"15min":1.042},"min":{"1min":0.57899999999999996,"5min":0.48899999999999999,"15min":0.45000000000000001},"max":{"1min":1.4159999999999999,"5min":1.7689999999999999,"15min":3.0339999999999998},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.94099999999999995,"15min":1.0129999999999999},"min":{"1min":0.51300000000000001,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":1.413,"5min":1.542,"15min":2.6840000000000002},"last":0.72399999999999998}]},{"osd":3,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":0.98999999999999999,"15min":1.054},"min":{"1min":0.50600000000000001,"5min":0.47199999999999998,"15min":0.38900000000000001},"max":{"1min":1.488,"5min":1.5129999999999999,"15min":4.3179999999999996},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.96699999999999997,"15min":1.0600000000000001},"min":{"1min":0.496,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.585,"5min":1.5900000000000001,"15min":8.5570000000000004},"last":0.61799999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.163,"15min":1.2450000000000001},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.63300000000000001},"max":{"1min":1.5549999999999999,"5min":1.907,"15min":3.1659999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.218,"15min":1.2869999999999999},"min":{"1min":0.72099999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.589,"5min":1.823,"15min":2.734},"last":1.1859999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.2569999999999999,"15min":1.4690000000000001},"min":{"1min":0.85399999999999998,"5min":0.72399999999999998,"15min":0.67100000000000004},"max":{"1min":1.619,"5min":1.921,"15min":8.4730000000000008},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.246,"5min":1.2589999999999999,"15min":1.458},"min":{"1min":0.78600000000000003,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.9470000000000001,"5min":1.9470000000000001,"15min":8.8040000000000003},"last":1.2030000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2949999999999999,"15min":1.385},"min":{"1min":0.80100000000000005,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.508,"5min":1.9119999999999999,"15min":3.984},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.2769999999999999,"5min":1.3129999999999999,"15min":1.3919999999999999},"min":{"1min":0.74299999999999999,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":2.0139999999999998,"5min":2.0449999999999999,"15min":3.2450000000000001},"last":1.3580000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:40:13 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.3,"15min":1.4299999999999999},"min":{"1min":0.90000000000000002,"5min":0.80700000000000005,"15min":0.69199999999999995},"max":{"1min":1.7150000000000001,"5min":1.853,"15min":4.3609999999999998},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.353,"5min":1.373,"15min":1.5069999999999999},"min":{"1min":0.76400000000000001,"5min":0.72799999999999998,"15min":0.70499999999999996},"max":{"1min":1.8360000000000001,"5min":1.972,"15min":5.3840000000000003},"last":1.155}]}]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33312,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338528,"statfs":{"total":96636764160,"available":96602652672,"internally_reserved":0,"allocated":2179072,"data_stored":1319049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.84099999999999997,"15min":0.81699999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.29399999999999998},"max":{"1min":1.248,"5min":1.3680000000000001,"15min":1.6220000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.96099999999999997,"15min":0.96299999999999997},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.492,"5min":1.722,"15min":2.3340000000000001},"last":0.96599999999999997}]},{"osd":2,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.81299999999999994,"5min":0.91700000000000004,"15min":0.92000000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.37},"max":{"1min":1.087,"5min":1.6910000000000001,"15min":1.762},"last":1.05},{"interface":"front","average":{"1min":0.87,"5min":0.91300000000000003,"15min":0.96999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.36799999999999999},"max":{"1min":1.3029999999999999,"5min":1.613,"15min":1.879},"last":1.0700000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.91000000000000003,"15min":0.93200000000000005},"min":{"1min":0.442,"5min":0.442,"15min":0.34200000000000003},"max":{"1min":1.5129999999999999,"5min":1.72,"15min":1.728},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.079,"5min":1.165,"15min":1.1759999999999999},"min":{"1min":0.498,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.544,"5min":1.8740000000000001,"15min":2.7679999999999998},"last":1.1080000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1679999999999999,"15min":1.1579999999999999},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.65300000000000002},"max":{"1min":1.6359999999999999,"5min":2.0489999999999999,"15min":2.2559999999999998},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.2509999999999999,"15min":1.274},"min":{"1min":0.754,"5min":0.69999999999999996,"15min":0.59399999999999997},"max":{"1min":1.5669999999999999,"5min":1.8859999999999999,"15min":2.867},"last":1.1599999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.2509999999999999,"15min":1.26},"min":{"1min":0.81699999999999995,"5min":0.79600000000000004,"15min":0.67900000000000005},"max":{"1min":1.6439999999999999,"5min":1.98,"15min":2.7280000000000002},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.214,"15min":1.2270000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.67600000000000005},"max":{"1min":1.5580000000000001,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.5349999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.353,"15min":1.3520000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.72499999999999998},"max":{"1min":1.6359999999999999,"5min":1.9399999999999999,"15min":2.6739999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.302,"15min":1.3460000000000001},"min":{"1min":0.89700000000000002,"5min":0.88200000000000001,"15min":0.64200000000000002},"max":{"1min":1.663,"5min":2.1280000000000001,"15min":2.8260000000000001},"last":1.7490000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:40:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.363,"15min":1.3979999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.6699999999999999,"5min":2.0150000000000001,"15min":4.5670000000000002},"last":1.8660000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.335,"15min":1.3520000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.748,"5min":2.6070000000000002,"15min":5.4450000000000003},"last":1.6080000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:41:11.215 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-08T16:41:11.215 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-08T16:41:11.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.476+0000 7ff37ebb6640 1 Processor -- start 2023-11-08T16:41:11.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.477+0000 7ff37ebb6640 1 -- start start 2023-11-08T16:41:11.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.478+0000 7ff37ebb6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780b0020 0x7ff3780cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.478+0000 7ff37ebb6640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff3780cdcf0 0x7ff37815c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.478+0000 7ff37ebb6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3780b0400 con 0x7ff3780cbbc0 2023-11-08T16:41:11.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.478+0000 7ff37ebb6640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3780cc350 con 0x7ff3780cdcf0 2023-11-08T16:41:11.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.478+0000 7ff37ebb6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780cc630 con 0x7ff3780b0020 2023-11-08T16:41:11.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.479+0000 7ff377fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff3780cbbc0 0x7ff3780cd5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39498/0 (socket says 172.21.15.81:39498) 2023-11-08T16:41:11.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.479+0000 7ff377fff640 1 -- 172.21.15.81:0/168262723 learned_addr learned my addr 172.21.15.81:0/168262723 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:11.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.479+0000 7ff37d153640 1 --2- 172.21.15.81:0/168262723 >> v2:172.21.15.93:3300/0 conn(0x7ff3780cdcf0 0x7ff37815c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.479+0000 7ff37c952640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780b0020 0x7ff3780cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.479+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3254381285 0 0) 0x7ff3780b0400 con 0x7ff3780cbbc0 2023-11-08T16:41:11.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.479+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff368003520 con 0x7ff3780cbbc0 2023-11-08T16:41:11.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3942618186 0 0) 0x7ff368003520 con 0x7ff3780cbbc0 2023-11-08T16:41:11.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780b0020 msgr2=0x7ff3780cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:11.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780b0020 0x7ff3780cfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 >> v2:172.21.15.93:3300/0 conn(0x7ff3780cdcf0 msgr2=0x7ff37815c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:11.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 --2- 172.21.15.81:0/168262723 >> v2:172.21.15.93:3300/0 conn(0x7ff3780cdcf0 0x7ff37815c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3780cc910 con 0x7ff3780cbbc0 2023-11-08T16:41:11.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ff364001670 con 0x7ff3780cbbc0 2023-11-08T16:41:11.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 >> v1:172.21.15.81:6789/0 conn(0x7ff3780cbbc0 legacy=0x7ff3780cd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780b0020 0x7ff368003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 0x7ff3680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff368004d00 0x7ff368005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780cc350 con 0x7ff368003ef0 2023-11-08T16:41:11.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780cc630 con 0x7ff368004d00 2023-11-08T16:41:11.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff36800b9d0 con 0x7ff3780b0020 2023-11-08T16:41:11.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.480+0000 7ff377fff640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 0x7ff3680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.481+0000 7ff37c952640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780b0020 0x7ff368003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.481+0000 7ff37d153640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff368004d00 0x7ff368005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.481+0000 7ff377fff640 1 -- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780b0020 msgr2=0x7ff368003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:11.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.481+0000 7ff377fff640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780b0020 0x7ff368003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.481+0000 7ff377fff640 1 -- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff368004d00 msgr2=0x7ff368005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:11.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.481+0000 7ff377fff640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff368004d00 0x7ff368005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.481+0000 7ff377fff640 1 -- 172.21.15.81:0/168262723 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff368003520 con 0x7ff368003ef0 2023-11-08T16:41:11.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.481+0000 7ff37d153640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff368004d00 0x7ff368005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:11.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.482+0000 7ff377fff640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 0x7ff3680047c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c2c0 tx=0x7ff3680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4e44d527db900a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:11.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.482+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364005ad0 con 0x7ff368003ef0 2023-11-08T16:41:11.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.482+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff364005c30 con 0x7ff368003ef0 2023-11-08T16:41:11.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.482+0000 7ff3777fe640 1 -- 172.21.15.81:0/168262723 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36402c8b0 con 0x7ff368003ef0 2023-11-08T16:41:11.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.483+0000 7ff37ebb6640 1 -- 172.21.15.81:0/168262723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 msgr2=0x7ff3680047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.483+0000 7ff37ebb6640 1 --2- 172.21.15.81:0/168262723 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 0x7ff3680047c0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c2c0 tx=0x7ff3680057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.483+0000 7ff37d153640 1 -- 172.21.15.81:0/168262723 reap_dead start 2023-11-08T16:41:11.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.483+0000 7ff37ebb6640 1 -- 172.21.15.81:0/168262723 shutdown_connections 2023-11-08T16:41:11.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.483+0000 7ff37ebb6640 1 -- 172.21.15.81:0/168262723 >> 172.21.15.81:0/168262723 conn(0x7ff3780d0d10 msgr2=0x7ff3780d1120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:11.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.483+0000 7ff37ebb6640 1 -- 172.21.15.81:0/168262723 shutdown_connections 2023-11-08T16:41:11.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.483+0000 7ff37ebb6640 1 -- 172.21.15.81:0/168262723 wait complete. 2023-11-08T16:41:11.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.485+0000 7ff37ebb6640 1 Processor -- start 2023-11-08T16:41:11.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.485+0000 7ff37ebb6640 1 -- start start 2023-11-08T16:41:11.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37ebb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 0x7ff378151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37ebb6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff37815d5a0 0x7ff3780c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37ebb6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780ca070 0x7ff3780c6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37ebb6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780c5e20 con 0x7ff368003ef0 2023-11-08T16:41:11.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37ebb6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780c4840 con 0x7ff37815d5a0 2023-11-08T16:41:11.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37ebb6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3780c4b20 con 0x7ff3780ca070 2023-11-08T16:41:11.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37c952640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 0x7ff378151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37c952640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 0x7ff378151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36902/0 (socket says 172.21.15.81:36902) 2023-11-08T16:41:11.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37c952640 1 -- 172.21.15.81:0/3204468007 learned_addr learned my addr 172.21.15.81:0/3204468007 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:11.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff377fff640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff37815d5a0 0x7ff3780c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37c952640 1 -- 172.21.15.81:0/3204468007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780ca070 msgr2=0x7ff3780c6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37c952640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780ca070 0x7ff3780c6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37c952640 1 -- 172.21.15.81:0/3204468007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff37815d5a0 msgr2=0x7ff3780c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37c952640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff37815d5a0 0x7ff3780c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.486+0000 7ff37c952640 1 -- 172.21.15.81:0/3204468007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3780c4e00 con 0x7ff368003ef0 2023-11-08T16:41:11.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.487+0000 7ff37c952640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 0x7ff378151340 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0107e0 tx=0x7ff36c010ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:11.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.487+0000 7ff3757fa640 1 -- 172.21.15.81:0/3204468007 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c009610 con 0x7ff368003ef0 2023-11-08T16:41:11.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.487+0000 7ff3757fa640 1 -- 172.21.15.81:0/3204468007 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c00eb60 con 0x7ff368003ef0 2023-11-08T16:41:11.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.487+0000 7ff3757fa640 1 -- 172.21.15.81:0/3204468007 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c01c400 con 0x7ff368003ef0 2023-11-08T16:41:11.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.487+0000 7ff37ebb6640 1 -- 172.21.15.81:0/3204468007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3780c50e0 con 0x7ff368003ef0 2023-11-08T16:41:11.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.487+0000 7ff37ebb6640 1 -- 172.21.15.81:0/3204468007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3780ca4f0 con 0x7ff368003ef0 2023-11-08T16:41:11.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.488+0000 7ff37ebb6640 1 -- 172.21.15.81:0/3204468007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3780743f0 con 0x7ff368003ef0 2023-11-08T16:41:11.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.489+0000 7ff3757fa640 1 -- 172.21.15.81:0/3204468007 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff36c01c560 con 0x7ff368003ef0 2023-11-08T16:41:11.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.490+0000 7ff3757fa640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff35c06ae70 0x7ff35c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.490+0000 7ff3757fa640 1 -- 172.21.15.81:0/3204468007 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff36c089a20 con 0x7ff368003ef0 2023-11-08T16:41:11.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.490+0000 7ff377fff640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff35c06ae70 0x7ff35c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.491+0000 7ff377fff640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff35c06ae70 0x7ff35c06d330 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff364034000 tx=0x7ff364007000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:11.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.494+0000 7ff3757fa640 1 -- 172.21.15.81:0/3204468007 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff36c054b80 con 0x7ff368003ef0 2023-11-08T16:41:11.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.743+0000 7ff37ebb6640 1 -- 172.21.15.81:0/3204468007 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff37806d150 con 0x7ff368003ef0 2023-11-08T16:41:11.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.749+0000 7ff3757fa640 1 -- 172.21.15.81:0/3204468007 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7ff36c0581a0 con 0x7ff368003ef0 2023-11-08T16:41:11.750 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2023-11-08T16:41:11.751 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQAHt0tlBqISAhAALQwJxb2CflpW7la8xhZ4gQ== 2023-11-08T16:41:11.751 INFO:teuthology.orchestra.run.smithi081.stdout: caps mds = "allow" 2023-11-08T16:41:11.751 INFO:teuthology.orchestra.run.smithi081.stdout: caps mon = "allow r" 2023-11-08T16:41:11.751 INFO:teuthology.orchestra.run.smithi081.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-08T16:41:11.752 INFO:teuthology.orchestra.run.smithi081.stderr:updated caps for client.0 2023-11-08T16:41:11.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.751+0000 7ff37ebb6640 1 -- 172.21.15.81:0/3204468007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff35c06ae70 msgr2=0x7ff35c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.751+0000 7ff37ebb6640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff35c06ae70 0x7ff35c06d330 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7ff364034000 tx=0x7ff364007000 comp rx=0 tx=0).stop 2023-11-08T16:41:11.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.751+0000 7ff37ebb6640 1 -- 172.21.15.81:0/3204468007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 msgr2=0x7ff378151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.751+0000 7ff37ebb6640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 0x7ff378151340 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0107e0 tx=0x7ff36c010ca0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.751+0000 7ff37ebb6640 1 -- 172.21.15.81:0/3204468007 shutdown_connections 2023-11-08T16:41:11.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.751+0000 7ff37ebb6640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff35c06ae70 0x7ff35c06d330 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.751+0000 7ff37ebb6640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff3780ca070 0x7ff3780c6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.751+0000 7ff37ebb6640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff37815d5a0 0x7ff3780c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.751+0000 7ff37ebb6640 1 --2- 172.21.15.81:0/3204468007 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff368003ef0 0x7ff378151340 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.751+0000 7ff37ebb6640 1 -- 172.21.15.81:0/3204468007 >> 172.21.15.81:0/3204468007 conn(0x7ff3780d0d10 msgr2=0x7ff37806f120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:11.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.752+0000 7ff37ebb6640 1 -- 172.21.15.81:0/3204468007 shutdown_connections 2023-11-08T16:41:11.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.752+0000 7ff37ebb6640 1 -- 172.21.15.81:0/3204468007 wait complete. 2023-11-08T16:41:11.774 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:41:11.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.977+0000 7fb66813e640 1 Processor -- start 2023-11-08T16:41:11.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.977+0000 7fb66813e640 1 -- start start 2023-11-08T16:41:11.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.978+0000 7fb66813e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6600adcd0 0x7fb6600ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.978+0000 7fb66813e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb660070050 0x7fb6600704e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.978+0000 7fb66813e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6600c9a70 con 0x7fb6600cc5e0 2023-11-08T16:41:11.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.979+0000 7fb66813e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6600cc3e0 con 0x7fb660070050 2023-11-08T16:41:11.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.979+0000 7fb66813e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb66006d070 con 0x7fb6600adcd0 2023-11-08T16:41:11.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.979+0000 7fb6666db640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb660070050 0x7fb6600704e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.979+0000 7fb665eda640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6600adcd0 0x7fb6600ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.979+0000 7fb6666db640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb660070050 0x7fb6600704e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:51330/0 (socket says 172.21.15.81:51330) 2023-11-08T16:41:11.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.979+0000 7fb6666db640 1 -- 172.21.15.81:0/883158824 learned_addr learned my addr 172.21.15.81:0/883158824 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:11.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.980+0000 7fb6666db640 1 -- 172.21.15.81:0/883158824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6600adcd0 msgr2=0x7fb6600ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:11.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.980+0000 7fb6666db640 1 --2- 172.21.15.81:0/883158824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6600adcd0 0x7fb6600ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.980+0000 7fb6666db640 1 -- 172.21.15.81:0/883158824 >> v1:172.21.15.93:6789/0 conn(0x7fb6600cc5e0 legacy=0x7fb6600cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.980+0000 7fb6666db640 1 -- 172.21.15.81:0/883158824 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb66006d350 con 0x7fb660070050 2023-11-08T16:41:11.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.981+0000 7fb6666db640 1 --2- 172.21.15.81:0/883158824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb660070050 0x7fb6600704e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb658009e50 tx=0x7fb65802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=381e52dabd2ac12b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:11.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.982+0000 7fb664ed8640 1 -- 172.21.15.81:0/883158824 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65802caf0 con 0x7fb660070050 2023-11-08T16:41:11.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.982+0000 7fb664ed8640 1 -- 172.21.15.81:0/883158824 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65802cc50 con 0x7fb660070050 2023-11-08T16:41:11.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.982+0000 7fb664ed8640 1 -- 172.21.15.81:0/883158824 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6580343f0 con 0x7fb660070050 2023-11-08T16:41:11.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.982+0000 7fb66813e640 1 -- 172.21.15.81:0/883158824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb660070050 msgr2=0x7fb6600704e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.982+0000 7fb66813e640 1 --2- 172.21.15.81:0/883158824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb660070050 0x7fb6600704e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb658009e50 tx=0x7fb65802bf50 comp rx=0 tx=0).stop 2023-11-08T16:41:11.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.982+0000 7fb66813e640 1 -- 172.21.15.81:0/883158824 shutdown_connections 2023-11-08T16:41:11.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.982+0000 7fb66813e640 1 --2- 172.21.15.81:0/883158824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb660070050 0x7fb6600704e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.982+0000 7fb66813e640 1 --2- 172.21.15.81:0/883158824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6600adcd0 0x7fb6600ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.982+0000 7fb66813e640 1 -- 172.21.15.81:0/883158824 >> 172.21.15.81:0/883158824 conn(0x7fb6600cb490 msgr2=0x7fb6600cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:11.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.982+0000 7fb66813e640 1 -- 172.21.15.81:0/883158824 shutdown_connections 2023-11-08T16:41:11.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.982+0000 7fb66813e640 1 -- 172.21.15.81:0/883158824 wait complete. 2023-11-08T16:41:11.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.984+0000 7fb66813e640 1 Processor -- start 2023-11-08T16:41:11.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.984+0000 7fb66813e640 1 -- start start 2023-11-08T16:41:11.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.985+0000 7fb66813e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb660070050 0x7fb6600b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.985+0000 7fb66813e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6600adcd0 0x7fb6600c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.985+0000 7fb66813e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6600cc5e0 0x7fb6600c46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.985+0000 7fb66813e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600b27f0 con 0x7fb6600cc5e0 2023-11-08T16:41:11.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.985+0000 7fb66813e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600b2ad0 con 0x7fb660070050 2023-11-08T16:41:11.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.985+0000 7fb66813e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600b2db0 con 0x7fb6600adcd0 2023-11-08T16:41:11.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb6666db640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6600cc5e0 0x7fb6600c46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb6666db640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6600cc5e0 0x7fb6600c46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36904/0 (socket says 172.21.15.81:36904) 2023-11-08T16:41:11.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb6666db640 1 -- 172.21.15.81:0/2706599188 learned_addr learned my addr 172.21.15.81:0/2706599188 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:11.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb6656d9640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6600adcd0 0x7fb6600c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb665eda640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb660070050 0x7fb6600b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb6666db640 1 -- 172.21.15.81:0/2706599188 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6600adcd0 msgr2=0x7fb6600c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb6666db640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6600adcd0 0x7fb6600c4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb6666db640 1 -- 172.21.15.81:0/2706599188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb660070050 msgr2=0x7fb6600b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:11.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb6666db640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb660070050 0x7fb6600b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:11.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb6666db640 1 -- 172.21.15.81:0/2706599188 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb658009b30 con 0x7fb6600cc5e0 2023-11-08T16:41:11.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb6656d9640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6600adcd0 0x7fb6600c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:11.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.986+0000 7fb6666db640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6600cc5e0 0x7fb6600c46a0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fb658036040 tx=0x7fb658004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:11.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.987+0000 7fb6567fc640 1 -- 172.21.15.81:0/2706599188 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658009510 con 0x7fb6600cc5e0 2023-11-08T16:41:11.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.987+0000 7fb6567fc640 1 -- 172.21.15.81:0/2706599188 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65803e450 con 0x7fb6600cc5e0 2023-11-08T16:41:11.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.987+0000 7fb6567fc640 1 -- 172.21.15.81:0/2706599188 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65803e800 con 0x7fb6600cc5e0 2023-11-08T16:41:11.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.987+0000 7fb66813e640 1 -- 172.21.15.81:0/2706599188 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6600b3090 con 0x7fb6600cc5e0 2023-11-08T16:41:11.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.987+0000 7fb66813e640 1 -- 172.21.15.81:0/2706599188 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb660153d40 con 0x7fb6600cc5e0 2023-11-08T16:41:11.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.988+0000 7fb66813e640 1 -- 172.21.15.81:0/2706599188 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb660153ed0 con 0x7fb6600cc5e0 2023-11-08T16:41:11.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.989+0000 7fb6567fc640 1 -- 172.21.15.81:0/2706599188 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb65803e960 con 0x7fb6600cc5e0 2023-11-08T16:41:11.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.990+0000 7fb6567fc640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb64406aea0 0x7fb64406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:11.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.990+0000 7fb6567fc640 1 -- 172.21.15.81:0/2706599188 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6580ac130 con 0x7fb6600cc5e0 2023-11-08T16:41:11.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.990+0000 7fb665eda640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb64406aea0 0x7fb64406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:11.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.991+0000 7fb665eda640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb64406aea0 0x7fb64406d360 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009970 tx=0x7fb65c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:11.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:11.994+0000 7fb6567fc640 1 -- 172.21.15.81:0/2706599188 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb658077140 con 0x7fb6600cc5e0 2023-11-08T16:41:12.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.222+0000 7fb66813e640 1 -- 172.21.15.81:0/2706599188 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb660072170 con 0x7fb6600cc5e0 2023-11-08T16:41:12.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.223+0000 7fb6567fc640 1 -- 172.21.15.81:0/2706599188 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4918 (secure 0 0 0) 0x7fb65807a760 con 0x7fb6600cc5e0 2023-11-08T16:41:12.227 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:12.227 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/296230272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":296230272},{"type":"v1","addr":"172.21.15.81:6837","nonce":296230272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:08.773407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8574":{"gid":8574,"name":"d","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6835/4021510722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":4021510722},{"type":"v1","addr":"172.21.15.93:6835","nonce":4021510722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:41:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 107 2023-11-08T16:41:12.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 -- 172.21.15.81:0/2706599188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb64406aea0 msgr2=0x7fb64406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb64406aea0 0x7fb64406d360 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb65c009970 tx=0x7fb65c008040 comp rx=0 tx=0).stop 2023-11-08T16:41:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 -- 172.21.15.81:0/2706599188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6600cc5e0 msgr2=0x7fb6600c46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6600cc5e0 0x7fb6600c46a0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fb658036040 tx=0x7fb658004a40 comp rx=0 tx=0).stop 2023-11-08T16:41:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 -- 172.21.15.81:0/2706599188 shutdown_connections 2023-11-08T16:41:12.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb64406aea0 0x7fb64406d360 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb6600cc5e0 0x7fb6600c46a0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb6600adcd0 0x7fb6600c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 --2- 172.21.15.81:0/2706599188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb660070050 0x7fb6600b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 -- 172.21.15.81:0/2706599188 >> 172.21.15.81:0/2706599188 conn(0x7fb6600cb490 msgr2=0x7fb6600cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:12.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 -- 172.21.15.81:0/2706599188 shutdown_connections 2023-11-08T16:41:12.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.226+0000 7fb66813e640 1 -- 172.21.15.81:0/2706599188 wait complete. 2023-11-08T16:41:12.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:41:05.700435+0000', 'modified': '2023-11-08T16:41:08.773407+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8574}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8574': {'gid': 8574, 'name': 'd', 'rank': 0, 'incarnation': 104, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.93:6835/4021510722', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.93:6833', 'nonce': 4021510722}, {'type': 'v1', 'addr': '172.21.15.93:6835', 'nonce': 4021510722}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:41:12.648 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:41:12.649 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-08T16:41:12.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.871+0000 7f6cd3fff640 1 Processor -- start 2023-11-08T16:41:12.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.871+0000 7f6cd3fff640 1 -- start start 2023-11-08T16:41:12.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.873+0000 7f6cd3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6cd40cb930 0x7f6cd40cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.873+0000 7f6cd3fff640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd406f850 con 0x7f6cd40ac8b0 2023-11-08T16:41:12.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.873+0000 7f6cd3fff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cd415d410 con 0x7f6cd4160af0 2023-11-08T16:41:12.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.873+0000 7f6cd3fff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cd4161040 con 0x7f6cd40cb930 2023-11-08T16:41:12.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.873+0000 7f6cd2ffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6cd40cb930 0x7f6cd40cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.873+0000 7f6cd37fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f6cd40ac8b0 0x7f6cd415cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39508/0 (socket says 172.21.15.81:39508) 2023-11-08T16:41:12.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.873+0000 7f6cd37fe640 1 -- 172.21.15.81:0/3682913576 learned_addr learned my addr 172.21.15.81:0/3682913576 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:12.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.874+0000 7f6cd2ffd640 1 -- 172.21.15.81:0/3682913576 >> v1:172.21.15.93:6789/0 conn(0x7f6cd4160af0 legacy=0x7f6cd415c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:12.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.874+0000 7f6cd2ffd640 1 -- 172.21.15.81:0/3682913576 >> v1:172.21.15.81:6789/0 conn(0x7f6cd40ac8b0 legacy=0x7f6cd415cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.874+0000 7f6cd2ffd640 1 -- 172.21.15.81:0/3682913576 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4161320 con 0x7f6cd40cb930 2023-11-08T16:41:12.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.875+0000 7f6cd2ffd640 1 --2- 172.21.15.81:0/3682913576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cd40cb930 0x7f6cd40cc530 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40097a0 tx=0x7f6cc402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be600b862dc55c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:12.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.875+0000 7f6cd1ffb640 1 -- 172.21.15.81:0/3682913576 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4031830 con 0x7f6cd40cb930 2023-11-08T16:41:12.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.875+0000 7f6cd1ffb640 1 -- 172.21.15.81:0/3682913576 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc4031990 con 0x7f6cd40cb930 2023-11-08T16:41:12.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.875+0000 7f6cd1ffb640 1 -- 172.21.15.81:0/3682913576 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4031d20 con 0x7f6cd40cb930 2023-11-08T16:41:12.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.876+0000 7f6cd3fff640 1 -- 172.21.15.81:0/3682913576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cd40cb930 msgr2=0x7f6cd40cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.876+0000 7f6cd3fff640 1 --2- 172.21.15.81:0/3682913576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cd40cb930 0x7f6cd40cc530 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f6cc40097a0 tx=0x7f6cc402b8d0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.876+0000 7f6cd3fff640 1 -- 172.21.15.81:0/3682913576 shutdown_connections 2023-11-08T16:41:12.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.876+0000 7f6cd3fff640 1 --2- 172.21.15.81:0/3682913576 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cd40cb930 0x7f6cd40cc530 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.876+0000 7f6cd3fff640 1 -- 172.21.15.81:0/3682913576 >> 172.21.15.81:0/3682913576 conn(0x7f6cd40b04d0 msgr2=0x7f6cd40b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:12.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.876+0000 7f6cd3fff640 1 -- 172.21.15.81:0/3682913576 shutdown_connections 2023-11-08T16:41:12.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.876+0000 7f6cd3fff640 1 -- 172.21.15.81:0/3682913576 wait complete. 2023-11-08T16:41:12.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.878+0000 7f6cd3fff640 1 Processor -- start 2023-11-08T16:41:12.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.878+0000 7f6cd3fff640 1 -- start start 2023-11-08T16:41:12.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd3fff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cd40ac8b0 0x7f6cd4157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd3fff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6cd40cb930 0x7f6cd414abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cd4160af0 0x7f6cd40c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd3fff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd415e620 con 0x7f6cd4160af0 2023-11-08T16:41:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd3fff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd414b130 con 0x7f6cd40ac8b0 2023-11-08T16:41:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd3fff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd4155e50 con 0x7f6cd40cb930 2023-11-08T16:41:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cd4160af0 0x7f6cd40c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cd4160af0 0x7f6cd40c44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36914/0 (socket says 172.21.15.81:36914) 2023-11-08T16:41:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd37fe640 1 -- 172.21.15.81:0/4112412765 learned_addr learned my addr 172.21.15.81:0/4112412765 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd2ffd640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cd40ac8b0 0x7f6cd4157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd27fc640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6cd40cb930 0x7f6cd414abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd37fe640 1 -- 172.21.15.81:0/4112412765 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6cd40cb930 msgr2=0x7f6cd414abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd37fe640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6cd40cb930 0x7f6cd414abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd37fe640 1 -- 172.21.15.81:0/4112412765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cd40ac8b0 msgr2=0x7f6cd4157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd37fe640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cd40ac8b0 0x7f6cd4157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.879+0000 7f6cd37fe640 1 -- 172.21.15.81:0/4112412765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc4009400 con 0x7f6cd4160af0 2023-11-08T16:41:12.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.880+0000 7f6cd2ffd640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cd40ac8b0 0x7f6cd4157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:12.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.880+0000 7f6cd37fe640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cd4160af0 0x7f6cd40c44b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00caf0 tx=0x7f6ccc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:12.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.880+0000 7f6ccb7fe640 1 -- 172.21.15.81:0/4112412765 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc00aca0 con 0x7f6cd4160af0 2023-11-08T16:41:12.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.880+0000 7f6ccb7fe640 1 -- 172.21.15.81:0/4112412765 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ccc00ae00 con 0x7f6cd4160af0 2023-11-08T16:41:12.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.880+0000 7f6ccb7fe640 1 -- 172.21.15.81:0/4112412765 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc012600 con 0x7f6cd4160af0 2023-11-08T16:41:12.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.880+0000 7f6cd3fff640 1 -- 172.21.15.81:0/4112412765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd4158410 con 0x7f6cd4160af0 2023-11-08T16:41:12.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.880+0000 7f6cd3fff640 1 -- 172.21.15.81:0/4112412765 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd415e800 con 0x7f6cd4160af0 2023-11-08T16:41:12.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.882+0000 7f6ccb7fe640 1 -- 172.21.15.81:0/4112412765 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ccc007070 con 0x7f6cd4160af0 2023-11-08T16:41:12.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.883+0000 7f6ccb7fe640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6cb406ae70 0x7f6cb406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.883+0000 7f6ccb7fe640 1 -- 172.21.15.81:0/4112412765 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ccc089450 con 0x7f6cd4160af0 2023-11-08T16:41:12.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.884+0000 7f6cd2ffd640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6cb406ae70 0x7f6cb406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.884+0000 7f6cdc22f740 1 Processor -- start 2023-11-08T16:41:12.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.884+0000 7f6cdc22f740 1 -- start start 2023-11-08T16:41:12.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.884+0000 7f6cd2ffd640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6cb406ae70 0x7f6cb406d330 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4036000 tx=0x7f6cc4008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:12.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 0x55d86e202f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e232e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e2034c0 0x55d86e233350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e1757e0 con 0x55d86e230460 2023-11-08T16:41:12.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e172ed0 con 0x55d86e230080 2023-11-08T16:41:12.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d86e16e7c0 con 0x55d86e2034c0 2023-11-08T16:41:12.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cd27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e232e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cd27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e232e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36926/0 (socket says 172.21.15.81:36926) 2023-11-08T16:41:12.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cd27fc640 1 -- 172.21.15.81:0/1114385525 learned_addr learned my addr 172.21.15.81:0/1114385525 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:12.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cd27fc640 1 -- 172.21.15.81:0/1114385525 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e2034c0 msgr2=0x55d86e233350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:12.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cd27fc640 1 --2- 172.21.15.81:0/1114385525 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e2034c0 0x55d86e233350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cd27fc640 1 -- 172.21.15.81:0/1114385525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 msgr2=0x55d86e202f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:12.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.885+0000 7f6cd2ffd640 1 --2- 172.21.15.81:0/1114385525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 0x55d86e202f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.886+0000 7f6cd27fc640 1 --2- 172.21.15.81:0/1114385525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 0x55d86e202f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.886+0000 7f6cd27fc640 1 -- 172.21.15.81:0/1114385525 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86e141b10 con 0x55d86e230460 2023-11-08T16:41:12.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.886+0000 7f6cd27fc640 1 --2- 172.21.15.81:0/1114385525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e232e10 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00a850 tx=0x7f6cbc00ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f9278fa7459a1b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:12.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.886+0000 7f6cc8ff9640 1 -- 172.21.15.81:0/1114385525 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00cb40 con 0x55d86e230460 2023-11-08T16:41:12.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.886+0000 7f6cc8ff9640 1 -- 172.21.15.81:0/1114385525 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cbc00cca0 con 0x55d86e230460 2023-11-08T16:41:12.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.886+0000 7f6cc8ff9640 1 -- 172.21.15.81:0/1114385525 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc011630 con 0x55d86e230460 2023-11-08T16:41:12.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.887+0000 7f6cdc22f740 1 -- 172.21.15.81:0/1114385525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 msgr2=0x55d86e232e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.887+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/1114385525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e232e10 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc00a850 tx=0x7f6cbc00ad10 comp rx=0 tx=0).stop 2023-11-08T16:41:12.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.887+0000 7f6cdc22f740 1 -- 172.21.15.81:0/1114385525 shutdown_connections 2023-11-08T16:41:12.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.887+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/1114385525 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e2034c0 0x55d86e233350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.887+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/1114385525 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e232e10 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.887+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/1114385525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 0x55d86e202f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.887+0000 7f6cdc22f740 1 -- 172.21.15.81:0/1114385525 >> 172.21.15.81:0/1114385525 conn(0x55d86e205290 msgr2=0x55d86e24de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:12.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.887+0000 7f6cdc22f740 1 -- 172.21.15.81:0/1114385525 shutdown_connections 2023-11-08T16:41:12.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.887+0000 7f6cdc22f740 1 -- 172.21.15.81:0/1114385525 wait complete. 2023-11-08T16:41:12.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.888+0000 7f6cdc22f740 1 -- start start 2023-11-08T16:41:12.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.888+0000 7f6cdc22f740 1 Processor -- start 2023-11-08T16:41:12.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.889+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 0x55d86e221030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.889+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2479a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.889+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e251320 0x55d86e2517a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.889+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e10bc40 con 0x55d86e230460 2023-11-08T16:41:12.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.889+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e10b8e0 con 0x55d86e230080 2023-11-08T16:41:12.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.889+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d86ded1f20 con 0x55d86e251320 2023-11-08T16:41:12.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.889+0000 7f6cdc22f740 10 client.0 _ll_register_callbacks cb 0x55d86e12dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:41:12.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.889+0000 7f6cd27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.889+0000 7f6cd27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2479a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36928/0 (socket says 172.21.15.81:36928) 2023-11-08T16:41:12.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.889+0000 7f6cd27fc640 1 -- 172.21.15.81:0/2385968231 learned_addr learned my addr 172.21.15.81:0/2385968231 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:12.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cd37fe640 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e251320 0x55d86e2517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cd27fc640 1 -- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e251320 msgr2=0x55d86e2517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:12.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cd2ffd640 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 0x55d86e221030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cd27fc640 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e251320 0x55d86e2517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cd27fc640 1 -- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 msgr2=0x55d86e221030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:12.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cd27fc640 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 0x55d86e221030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cd27fc640 1 -- 172.21.15.81:0/2385968231 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc00a450 con 0x55d86e230460 2023-11-08T16:41:12.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cd27fc640 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2479a0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc008010 tx=0x7f6cbc004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c81cb7bc8a6bbbe server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:12.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:41:12.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/2385968231 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc020070 con 0x55d86e230460 2023-11-08T16:41:12.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/2385968231 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cbc0042d0 con 0x55d86e230460 2023-11-08T16:41:12.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cdc22f740 1 -- 172.21.15.81:0/2385968231 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d86e142c80 con 0x55d86e230460 2023-11-08T16:41:12.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.890+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/2385968231 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cbc004490 con 0x55d86e230460 2023-11-08T16:41:12.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.891+0000 7f6cdc22f740 10 client.8703 fetch_fsmap learned FSMap version 107 2023-11-08T16:41:12.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.891+0000 7f6cdc22f740 1 -- 172.21.15.81:0/2385968231 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x55d86e16ce40 con 0x55d86e230460 2023-11-08T16:41:12.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.891+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/2385968231 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d86e142c80 con 0x55d86e230460 2023-11-08T16:41:12.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.891+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/2385968231 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 107) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6cbc0035f0 con 0x55d86e230460 2023-11-08T16:41:12.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.891+0000 7f6cdc22f740 10 client.8703 fetch_fsmap finished waiting for FSMap version 107 2023-11-08T16:41:12.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.891+0000 7f6cdc22f740 10 client.8703 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T16:41:12.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.891+0000 7f6cdc22f740 20 client.8703 populate_metadata read hostname 'smithi081' 2023-11-08T16:41:12.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.891+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:6833/4021510722,v1:172.21.15.93:6835/4021510722] conn(0x55d86e234e50 0x55d86e235230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.891+0000 7f6cdc22f740 4 client.8703 mds_command: new command op to 8574 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:41:12.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.891+0000 7f6cdc22f740 1 -- 172.21.15.81:0/2385968231 --> [v2:172.21.15.93:6833/4021510722,v1:172.21.15.93:6835/4021510722] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d86e1677a0 con 0x55d86e234e50 2023-11-08T16:41:12.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.892+0000 7f6cd2ffd640 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:6833/4021510722,v1:172.21.15.93:6835/4021510722] conn(0x55d86e234e50 0x55d86e235230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.892+0000 7f6cd2ffd640 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:6833/4021510722,v1:172.21.15.93:6835/4021510722] conn(0x55d86e234e50 0x55d86e235230 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a89797484f12a2 server_cookie=3d6e77f6c952d4b6 in_seq=0 out_seq=0 2023-11-08T16:41:12.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.892+0000 7f6cb1ffb640 10 client.8703 ms_handle_connect on v2:172.21.15.93:6833/4021510722 2023-11-08T16:41:12.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.894+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/2385968231 <== mds.0 v2:172.21.15.93:6833/4021510722 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d86e1677a0 con 0x55d86e234e50 2023-11-08T16:41:12.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.894+0000 7f6cb1ffb640 10 client.8703 handle_command_reply: tid=0 2023-11-08T16:41:12.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 1 client.8703 shutdown 2023-11-08T16:41:12.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 1 -- 172.21.15.81:0/2385968231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 msgr2=0x55d86e2479a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2479a0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc008010 tx=0x7f6cbc004680 comp rx=0 tx=0).stop 2023-11-08T16:41:12.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 1 -- 172.21.15.81:0/2385968231 shutdown_connections 2023-11-08T16:41:12.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:6833/4021510722,v1:172.21.15.93:6835/4021510722] conn(0x55d86e234e50 0x55d86e235230 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e251320 0x55d86e2517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2479a0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cb1ffb640 0 client.8703 ms_handle_reset on v2:172.21.15.93:6833/4021510722 2023-11-08T16:41:12.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/2385968231 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 0x55d86e221030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 1 -- 172.21.15.81:0/2385968231 >> 172.21.15.81:0/2385968231 conn(0x55d86e2034c0 msgr2=0x55d86e2026d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:12.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 1 -- 172.21.15.81:0/2385968231 shutdown_connections 2023-11-08T16:41:12.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 1 -- 172.21.15.81:0/2385968231 wait complete. 2023-11-08T16:41:12.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.895+0000 7f6cdc22f740 20 client.8703 trim_cache size 0 max 16384 2023-11-08T16:41:12.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.912+0000 7f6cdc22f740 1 Processor -- start 2023-11-08T16:41:12.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.912+0000 7f6cdc22f740 1 -- start start 2023-11-08T16:41:12.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e230080 0x55d86e248e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e251320 0x55d86e2477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e2373f0 con 0x55d86e230460 2023-11-08T16:41:12.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e235de0 con 0x55d86e251320 2023-11-08T16:41:12.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d86e21d740 con 0x55d86e230080 2023-11-08T16:41:12.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cd27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cd27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36944/0 (socket says 172.21.15.81:36944) 2023-11-08T16:41:12.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cd27fc640 1 -- 172.21.15.81:0/3479374323 learned_addr learned my addr 172.21.15.81:0/3479374323 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:12.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cd27fc640 1 -- 172.21.15.81:0/3479374323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e230080 msgr2=0x55d86e248e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:12.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cd27fc640 1 --2- 172.21.15.81:0/3479374323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e230080 0x55d86e248e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cd27fc640 1 -- 172.21.15.81:0/3479374323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e251320 msgr2=0x55d86e2477f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:12.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cd27fc640 1 --2- 172.21.15.81:0/3479374323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e251320 0x55d86e2477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.913+0000 7f6cd27fc640 1 -- 172.21.15.81:0/3479374323 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d86e10ab90 con 0x55d86e230460 2023-11-08T16:41:12.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cd27fc640 1 --2- 172.21.15.81:0/3479374323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2b2b10 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc006d50 tx=0x7f6cbc01d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31667a7eb84eda09 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:12.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cc97fa640 1 -- 172.21.15.81:0/3479374323 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc0069a0 con 0x55d86e230460 2023-11-08T16:41:12.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cc97fa640 1 -- 172.21.15.81:0/3479374323 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cbc0035f0 con 0x55d86e230460 2023-11-08T16:41:12.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cc97fa640 1 -- 172.21.15.81:0/3479374323 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc003760 con 0x55d86e230460 2023-11-08T16:41:12.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cdc22f740 1 -- 172.21.15.81:0/3479374323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 msgr2=0x55d86e2b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/3479374323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2b2b10 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc006d50 tx=0x7f6cbc01d070 comp rx=0 tx=0).stop 2023-11-08T16:41:12.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cdc22f740 1 -- 172.21.15.81:0/3479374323 shutdown_connections 2023-11-08T16:41:12.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/3479374323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e251320 0x55d86e2477f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/3479374323 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e230460 0x55d86e2b2b10 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/3479374323 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e230080 0x55d86e248e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cdc22f740 1 -- 172.21.15.81:0/3479374323 >> 172.21.15.81:0/3479374323 conn(0x55d86e2034c0 msgr2=0x55d86e270650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:12.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.914+0000 7f6cdc22f740 1 -- 172.21.15.81:0/3479374323 shutdown_connections 2023-11-08T16:41:12.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.915+0000 7f6cdc22f740 1 -- 172.21.15.81:0/3479374323 wait complete. 2023-11-08T16:41:12.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.915+0000 7f6cdc22f740 1 -- start start 2023-11-08T16:41:12.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.915+0000 7f6cdc22f740 1 Processor -- start 2023-11-08T16:41:12.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.916+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 0x55d86e25fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.916+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e230460 0x55d86e235230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.916+0000 7f6cdc22f740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e251320 0x55d86e235770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.916+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e2ade30 con 0x55d86e251320 2023-11-08T16:41:12.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.916+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d86e22f190 con 0x55d86e230080 2023-11-08T16:41:12.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.916+0000 7f6cdc22f740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d86e2ae350 con 0x55d86e230460 2023-11-08T16:41:12.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.916+0000 7f6cdc22f740 10 client.0 _ll_register_callbacks cb 0x7f6cbc006ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:41:12.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.917+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e251320 0x55d86e235770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.917+0000 7f6cd37fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e251320 0x55d86e235770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36948/0 (socket says 172.21.15.81:36948) 2023-11-08T16:41:12.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.917+0000 7f6cd37fe640 1 -- 172.21.15.81:0/120005834 learned_addr learned my addr 172.21.15.81:0/120005834 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:12.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.917+0000 7f6cd37fe640 1 -- 172.21.15.81:0/120005834 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e230460 msgr2=0x55d86e235230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:12.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.917+0000 7f6cd37fe640 1 --2- 172.21.15.81:0/120005834 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e230460 0x55d86e235230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.917+0000 7f6cd37fe640 1 -- 172.21.15.81:0/120005834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 msgr2=0x55d86e25fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:12.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.917+0000 7f6cd37fe640 1 --2- 172.21.15.81:0/120005834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 0x55d86e25fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.917+0000 7f6cd37fe640 1 -- 172.21.15.81:0/120005834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d86e16ce40 con 0x55d86e251320 2023-11-08T16:41:12.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.917+0000 7f6cd37fe640 1 --2- 172.21.15.81:0/120005834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e251320 0x55d86e235770 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00afd0 tx=0x7f6ccc05c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2570867b678bc40 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:12.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.917+0000 7f6cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:41:12.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.917+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/120005834 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ccc057950 con 0x55d86e251320 2023-11-08T16:41:12.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/120005834 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ccc0566c0 con 0x55d86e251320 2023-11-08T16:41:12.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cdc22f740 1 -- 172.21.15.81:0/120005834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6cbc00cca0 con 0x55d86e251320 2023-11-08T16:41:12.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/120005834 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=107) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ccc089280 con 0x55d86e251320 2023-11-08T16:41:12.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cdc22f740 10 client.8709 fetch_fsmap learned FSMap version 107 2023-11-08T16:41:12.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cdc22f740 1 -- 172.21.15.81:0/120005834 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=107,monmap=2+}) v3 -- 0x55d86e130180 con 0x55d86e251320 2023-11-08T16:41:12.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/120005834 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cbc00cca0 con 0x55d86e251320 2023-11-08T16:41:12.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/120005834 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 107) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6ccc060020 con 0x55d86e251320 2023-11-08T16:41:12.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cdc22f740 10 client.8709 fetch_fsmap finished waiting for FSMap version 107 2023-11-08T16:41:12.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cdc22f740 10 client.8709 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-08T16:41:12.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cdc22f740 20 client.8709 populate_metadata read hostname 'smithi081' 2023-11-08T16:41:12.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/120005834 >> [v2:172.21.15.93:6833/4021510722,v1:172.21.15.93:6835/4021510722] conn(0x55d86e22e5f0 0x55d86e22e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:12.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cdc22f740 4 client.8709 mds_command: new command op to 8574 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:41:12.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.918+0000 7f6cdc22f740 1 -- 172.21.15.81:0/120005834 --> [v2:172.21.15.93:6833/4021510722,v1:172.21.15.93:6835/4021510722] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d86e1674c0 con 0x55d86e22e5f0 2023-11-08T16:41:12.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.919+0000 7f6cd2ffd640 1 --2- 172.21.15.81:0/120005834 >> [v2:172.21.15.93:6833/4021510722,v1:172.21.15.93:6835/4021510722] conn(0x55d86e22e5f0 0x55d86e22e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:12.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.920+0000 7f6cb1ffb640 10 client.8709 ms_handle_connect on v2:172.21.15.93:6833/4021510722 2023-11-08T16:41:12.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.920+0000 7f6cd2ffd640 1 --2- 172.21.15.81:0/120005834 >> [v2:172.21.15.93:6833/4021510722,v1:172.21.15.93:6835/4021510722] conn(0x55d86e22e5f0 0x55d86e22e9d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26c1e6f06d3a7e13 server_cookie=fbf1520ed0c339e7 in_seq=0 out_seq=0 2023-11-08T16:41:12.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.920+0000 7f6cb1ffb640 1 -- 172.21.15.81:0/120005834 <== mds.0 v2:172.21.15.93:6833/4021510722 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x55d86e1674c0 con 0x55d86e22e5f0 2023-11-08T16:41:12.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.920+0000 7f6cb1ffb640 10 client.8709 handle_command_reply: tid=0 2023-11-08T16:41:12.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.920+0000 7f6cdc22f740 1 client.8709 shutdown 2023-11-08T16:41:12.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cdc22f740 1 -- 172.21.15.81:0/120005834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e251320 msgr2=0x55d86e235770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/120005834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e251320 0x55d86e235770 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00afd0 tx=0x7f6ccc05c440 comp rx=0 tx=0).stop 2023-11-08T16:41:12.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cdc22f740 1 -- 172.21.15.81:0/120005834 shutdown_connections 2023-11-08T16:41:12.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/120005834 >> [v2:172.21.15.93:6833/4021510722,v1:172.21.15.93:6835/4021510722] conn(0x55d86e22e5f0 0x55d86e22e9d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/120005834 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55d86e251320 0x55d86e235770 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/120005834 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55d86e230460 0x55d86e235230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cdc22f740 1 --2- 172.21.15.81:0/120005834 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55d86e230080 0x55d86e25fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cb1ffb640 0 client.8709 ms_handle_reset on v2:172.21.15.93:6833/4021510722 2023-11-08T16:41:12.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cdc22f740 1 -- 172.21.15.81:0/120005834 >> 172.21.15.81:0/120005834 conn(0x55d86e2034c0 msgr2=0x55d86e234320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:12.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cdc22f740 1 -- 172.21.15.81:0/120005834 shutdown_connections 2023-11-08T16:41:12.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cdc22f740 1 -- 172.21.15.81:0/120005834 wait complete. 2023-11-08T16:41:12.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.921+0000 7f6cdc22f740 20 client.8709 trim_cache size 0 max 16384 2023-11-08T16:41:12.935 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:41:12.935 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:41:12.936 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:41:12.936 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 8574, 2023-11-08T16:41:12.936 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:41:12.936 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:41:12.936 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:41:12.937 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 6.6660161200000001, 2023-11-08T16:41:12.937 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 107, 2023-11-08T16:41:12.937 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 127, 2023-11-08T16:41:12.937 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 127, 2023-11-08T16:41:12.937 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 13.047685929 2023-11-08T16:41:12.938 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:41:12.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 -- 172.21.15.81:0/4112412765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6cb406ae70 msgr2=0x7f6cb406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6cb406ae70 0x7f6cb406d330 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f6cc4036000 tx=0x7f6cc4008030 comp rx=0 tx=0).stop 2023-11-08T16:41:12.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 -- 172.21.15.81:0/4112412765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cd4160af0 msgr2=0x7f6cd40c44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:12.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cd4160af0 0x7f6cd40c44b0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc00caf0 tx=0x7f6ccc00a060 comp rx=0 tx=0).stop 2023-11-08T16:41:12.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 -- 172.21.15.81:0/4112412765 shutdown_connections 2023-11-08T16:41:12.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6cb406ae70 0x7f6cb406d330 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6cd4160af0 0x7f6cd40c44b0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6cd40cb930 0x7f6cd414abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 --2- 172.21.15.81:0/4112412765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cd40ac8b0 0x7f6cd4157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:12.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 -- 172.21.15.81:0/4112412765 >> 172.21.15.81:0/4112412765 conn(0x7f6cd40b04d0 msgr2=0x7f6cd40cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:12.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 -- 172.21.15.81:0/4112412765 shutdown_connections 2023-11-08T16:41:12.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:12.922+0000 7f6cd3fff640 1 -- 172.21.15.81:0/4112412765 wait complete. 2023-11-08T16:41:12.942 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:41:13.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.152+0000 7fbb36064640 1 Processor -- start 2023-11-08T16:41:13.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.152+0000 7fbb36064640 1 -- start start 2023-11-08T16:41:13.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.153+0000 7fbb36064640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb300ade50 0x7fbb300ca480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:13.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.153+0000 7fbb36064640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbb3015a680 0x7fbb3015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:13.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.153+0000 7fbb36064640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb3015b050 0x7fbb300c9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:13.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.153+0000 7fbb36064640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb300b1860 con 0x7fbb300ade50 2023-11-08T16:41:13.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb36064640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb300ca9c0 con 0x7fbb3015a680 2023-11-08T16:41:13.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb36064640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3006b630 con 0x7fbb3015b050 2023-11-08T16:41:13.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb2f7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb300ade50 0x7fbb300ca480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:13.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb2f7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbb300ade50 0x7fbb300ca480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36960/0 (socket says 172.21.15.81:36960) 2023-11-08T16:41:13.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb2f7fe640 1 -- 172.21.15.81:0/1389883403 learned_addr learned my addr 172.21.15.81:0/1389883403 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:13.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb2effd640 1 --2- 172.21.15.81:0/1389883403 >> v2:172.21.15.93:3300/0 conn(0x7fbb3015a680 0x7fbb3015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:13.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb2f7fe640 1 -- 172.21.15.81:0/1389883403 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb3015b050 msgr2=0x7fbb300c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:13.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb2ffff640 1 --2- 172.21.15.81:0/1389883403 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb3015b050 0x7fbb300c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:13.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb2f7fe640 1 --2- 172.21.15.81:0/1389883403 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb3015b050 0x7fbb300c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb2f7fe640 1 -- 172.21.15.81:0/1389883403 >> v2:172.21.15.93:3300/0 conn(0x7fbb3015a680 msgr2=0x7fbb3015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:13.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb2f7fe640 1 --2- 172.21.15.81:0/1389883403 >> v2:172.21.15.93:3300/0 conn(0x7fbb3015a680 0x7fbb3015ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb2f7fe640 1 -- 172.21.15.81:0/1389883403 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb300cabb0 con 0x7fbb300ade50 2023-11-08T16:41:13.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.154+0000 7fbb2effd640 1 --2- 172.21.15.81:0/1389883403 >> v2:172.21.15.93:3300/0 conn(0x7fbb3015a680 0x7fbb3015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:13.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.155+0000 7fbb2f7fe640 1 --2- 172.21.15.81:0/1389883403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb300ade50 0x7fbb300ca480 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fbb30073d00 tx=0x7fbb2002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a485a37df1681dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:13.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.156+0000 7fbb2e7fc640 1 -- 172.21.15.81:0/1389883403 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20031860 con 0x7fbb300ade50 2023-11-08T16:41:13.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.156+0000 7fbb2e7fc640 1 -- 172.21.15.81:0/1389883403 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb200319c0 con 0x7fbb300ade50 2023-11-08T16:41:13.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.156+0000 7fbb2e7fc640 1 -- 172.21.15.81:0/1389883403 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20031d50 con 0x7fbb300ade50 2023-11-08T16:41:13.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.156+0000 7fbb36064640 1 -- 172.21.15.81:0/1389883403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb300ade50 msgr2=0x7fbb300ca480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:13.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.156+0000 7fbb36064640 1 --2- 172.21.15.81:0/1389883403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb300ade50 0x7fbb300ca480 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fbb30073d00 tx=0x7fbb2002b900 comp rx=0 tx=0).stop 2023-11-08T16:41:13.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.157+0000 7fbb36064640 1 -- 172.21.15.81:0/1389883403 shutdown_connections 2023-11-08T16:41:13.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.157+0000 7fbb36064640 1 --2- 172.21.15.81:0/1389883403 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb3015b050 0x7fbb300c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.157+0000 7fbb36064640 1 --2- 172.21.15.81:0/1389883403 >> v2:172.21.15.93:3300/0 conn(0x7fbb3015a680 0x7fbb3015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.157+0000 7fbb36064640 1 --2- 172.21.15.81:0/1389883403 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb300ade50 0x7fbb300ca480 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.157+0000 7fbb36064640 1 -- 172.21.15.81:0/1389883403 >> 172.21.15.81:0/1389883403 conn(0x7fbb300cb490 msgr2=0x7fbb300cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:13.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.157+0000 7fbb36064640 1 -- 172.21.15.81:0/1389883403 shutdown_connections 2023-11-08T16:41:13.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.157+0000 7fbb36064640 1 -- 172.21.15.81:0/1389883403 wait complete. 2023-11-08T16:41:13.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.158+0000 7fbb36064640 1 Processor -- start 2023-11-08T16:41:13.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.159+0000 7fbb36064640 1 -- start start 2023-11-08T16:41:13.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.159+0000 7fbb36064640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb300ade50 0x7fbb3015d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:13.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.159+0000 7fbb36064640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb3015a680 0x7fbb300c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:13.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb36064640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb3015b050 0x7fbb300c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:13.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb36064640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb300c55a0 con 0x7fbb300ade50 2023-11-08T16:41:13.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb36064640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb300a9440 con 0x7fbb3015b050 2023-11-08T16:41:13.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb36064640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30145de0 con 0x7fbb3015a680 2023-11-08T16:41:13.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb2f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb300ade50 0x7fbb3015d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:13.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb2f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb300ade50 0x7fbb3015d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36964/0 (socket says 172.21.15.81:36964) 2023-11-08T16:41:13.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb2f7fe640 1 -- 172.21.15.81:0/933693152 learned_addr learned my addr 172.21.15.81:0/933693152 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:13.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb2f7fe640 1 -- 172.21.15.81:0/933693152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb3015a680 msgr2=0x7fbb300c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:13.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb2f7fe640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb3015a680 0x7fbb300c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb2f7fe640 1 -- 172.21.15.81:0/933693152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb3015b050 msgr2=0x7fbb300c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:13.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb2f7fe640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb3015b050 0x7fbb300c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb2f7fe640 1 -- 172.21.15.81:0/933693152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb200094d0 con 0x7fbb300ade50 2023-11-08T16:41:13.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.160+0000 7fbb2f7fe640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb300ade50 0x7fbb3015d150 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fbb2002b940 tx=0x7fbb200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:13.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.161+0000 7fbb17fff640 1 -- 172.21.15.81:0/933693152 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20031850 con 0x7fbb300ade50 2023-11-08T16:41:13.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.161+0000 7fbb17fff640 1 -- 172.21.15.81:0/933693152 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb200443f0 con 0x7fbb300ade50 2023-11-08T16:41:13.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.161+0000 7fbb36064640 1 -- 172.21.15.81:0/933693152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb301460c0 con 0x7fbb300ade50 2023-11-08T16:41:13.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.161+0000 7fbb17fff640 1 -- 172.21.15.81:0/933693152 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb200447a0 con 0x7fbb300ade50 2023-11-08T16:41:13.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.161+0000 7fbb36064640 1 -- 172.21.15.81:0/933693152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb30146680 con 0x7fbb300ade50 2023-11-08T16:41:13.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.162+0000 7fbb36064640 1 -- 172.21.15.81:0/933693152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb30146870 con 0x7fbb300ade50 2023-11-08T16:41:13.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.163+0000 7fbb17fff640 1 -- 172.21.15.81:0/933693152 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbb2004d450 con 0x7fbb300ade50 2023-11-08T16:41:13.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.164+0000 7fbb17fff640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb1c06ae70 0x7fbb1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:13.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.164+0000 7fbb17fff640 1 -- 172.21.15.81:0/933693152 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb200abd00 con 0x7fbb300ade50 2023-11-08T16:41:13.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.164+0000 7fbb2effd640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb1c06ae70 0x7fbb1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:13.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.165+0000 7fbb2effd640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb1c06ae70 0x7fbb1c06d330 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbb30158df0 tx=0x7fbb24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:13.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.169+0000 7fbb17fff640 1 -- 172.21.15.81:0/933693152 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbb20076da0 con 0x7fbb300ade50 2023-11-08T16:41:13.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.403+0000 7fbb36064640 1 -- 172.21.15.81:0/933693152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbb3004ece0 con 0x7fbb300ade50 2023-11-08T16:41:13.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.404+0000 7fbb17fff640 1 -- 172.21.15.81:0/933693152 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v107) v1 ==== 53+0+73 (secure 0 0 0) 0x7fbb20035020 con 0x7fbb300ade50 2023-11-08T16:41:13.406 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T16:41:13.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.406+0000 7fbb36064640 1 -- 172.21.15.81:0/933693152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb1c06ae70 msgr2=0x7fbb1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:13.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.406+0000 7fbb36064640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb1c06ae70 0x7fbb1c06d330 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbb30158df0 tx=0x7fbb24008040 comp rx=0 tx=0).stop 2023-11-08T16:41:13.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.406+0000 7fbb36064640 1 -- 172.21.15.81:0/933693152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb300ade50 msgr2=0x7fbb3015d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:13.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.406+0000 7fbb36064640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb300ade50 0x7fbb3015d150 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fbb2002b940 tx=0x7fbb200046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.406+0000 7fbb36064640 1 -- 172.21.15.81:0/933693152 shutdown_connections 2023-11-08T16:41:13.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.406+0000 7fbb36064640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbb1c06ae70 0x7fbb1c06d330 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.406+0000 7fbb36064640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb3015b050 0x7fbb300c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.406+0000 7fbb36064640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbb3015a680 0x7fbb300c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.406+0000 7fbb36064640 1 --2- 172.21.15.81:0/933693152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbb300ade50 0x7fbb3015d150 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:13.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.406+0000 7fbb36064640 1 -- 172.21.15.81:0/933693152 >> 172.21.15.81:0/933693152 conn(0x7fbb300cb490 msgr2=0x7fbb300b32b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:13.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.407+0000 7fbb36064640 1 -- 172.21.15.81:0/933693152 shutdown_connections 2023-11-08T16:41:13.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:13.407+0000 7fbb36064640 1 -- 172.21.15.81:0/933693152 wait complete. 2023-11-08T16:41:13.427 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:41:13.428 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:41:13.428 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:41:13.428 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:41:13.429 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:13.429 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:41:13.429 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:41:13.430 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:41:13.430 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:41:13.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:13.431 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:41:13.485 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:41:13.486 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:41:13.486 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:41:13.486 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:13.486 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:41:13.486 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:13.487 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:41:13.487 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:13.487 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:41:13.487 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:41:13.487 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:13.488 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:41:13.488 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:41:13.488 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:13.488 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:41:13.488 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:13.489 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:41:13.489 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 41636sec preferred_lft 41636sec 2023-11-08T16:41:13.489 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:41:13.489 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:13.489 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:41:13.490 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:13.490 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:41:13.490 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:13.490 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:41:13.490 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:13.490 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:41:13.491 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:13.491 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:41:13.491 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:13.491 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:41:13.491 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:13.492 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::2cac:17ff:fe64:67da/64 scope link 2023-11-08T16:41:13.492 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:13.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:13.493 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:41:13.581 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:41:13.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:13.585 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:41:13.640 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:41:13.641 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:41:13.641 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:41:13.641 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:41:13.642 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:41:13.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:13.643 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:41:13.643 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:41:13.644 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:41:13.644 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:41:13.645 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:41:13.645 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:41:13.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:13.831 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:41:13.886 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:41:13.887 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:41:13.887 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:41:13.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:13.888 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:41:13.983 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:41:13.984 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:41:13.984 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:14.040 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:41:14.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:41:14.041 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:14.091 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:41:14.188 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:41:14.243 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:41:14.243 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:41:14.244 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:41:14.244 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:41:14.244 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:41:14.244 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:41:14.244 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:41:14.245 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:41:14.245 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:41:14.245 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:41:14.245 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:41:14.245 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:41:14.246 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:41:14.246 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:41:14.246 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:41:14.246 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:41:14.246 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:41:14.247 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:41:14.247 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:41:14.247 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:41:14.247 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:41:14.248 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:41:14.248 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:41:14.248 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:41:14.248 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:41:14.248 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:41:14.248 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:41:14.249 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:41:14.249 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:41:14.249 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:14.249 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:14.249 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:14.250 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:14.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:14.251 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:41:14.303 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:41:14.304 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:41:14.310 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:41:14.387 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:41:14.430 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:41:14.430 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:41:14.430 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:41:14.430 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:41:14.431 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:41:14.431 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:41:14.431 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:41:14.431 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:41:14.431 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:41:14.432 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:41:14.432 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:41:14.432 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:41:14.432 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:41:14.433 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:41:14.433 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:41:14.433 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:41:14.433 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:41:14.433 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:41:14.433 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:41:14.434 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:41:14.434 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:41:14.434 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:41:14.434 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:41:14.434 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:41:14.435 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:41:14.435 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:41:14.435 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:41:14.435 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:41:14.435 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:41:14.436 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:14.436 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:14.436 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:14.436 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:14.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:14.438 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:41:14.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:41:14.494+0000 7fdde8e3f480 -1 init, newargv = 0x5563167ef9d0 newargc=15 2023-11-08T16:41:14.496 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[125091]: starting ceph client 2023-11-08T16:41:14.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[125091]: starting fuse 2023-11-08T16:41:14.524 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:41:14.524 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:41:14.525 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:41:14.525 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:41:14.526 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:41:14.526 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:41:14.527 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:41:14.527 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:14.527 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:41:14.528 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:41:14.528 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:41:14.528 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:41:14.529 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:14.529 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:41:14.529 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:41:14.530 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:41:14.530 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:41:14.531 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:14.531 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:41:14.531 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:41:14.532 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:41:14.532 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:41:14.533 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:41:14.533 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:41:14.533 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:41:14.534 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:41:14.534 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:14.534 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:41:14.535 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:41:14.725 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.125091.asok 2023-11-08T16:41:14.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:41:14.741 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.125091.asok 2023-11-08T16:41:14.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:14.742 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.125091.asok status 2023-11-08T16:41:14.981 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:41:14.982 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:41:14.982 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:41:14.982 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:41:14.982 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:41:14.983 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:41:14.983 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:41:14.983 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "125091", 2023-11-08T16:41:14.983 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:41:14.983 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:14.984 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:41:14.984 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:41:14.984 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 8724, 2023-11-08T16:41:14.984 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:41:14.984 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:41:14.985 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:41:14.985 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 8724 2023-11-08T16:41:14.985 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:14.985 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:41:14.985 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:41:14.985 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:41:14.986 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1959407895 2023-11-08T16:41:14.986 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:14.986 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:14.986 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:41:14.987 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:41:14.987 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:41:14.987 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1959407895 2023-11-08T16:41:14.987 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:14.987 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.8724 192.168.0.1:0/1959407895", 2023-11-08T16:41:14.988 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/1959407895", 2023-11-08T16:41:14.988 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:41:14.988 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 107, 2023-11-08T16:41:14.988 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 127, 2023-11-08T16:41:14.988 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:41:14.988 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:41:14.989 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:41:14.989 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:41:14.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:14.996 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:41:14.997 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:15.051 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:41:15.051 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:15.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:15.052 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:15.148 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:41:15.155 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:41:15.322 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2023-11-08T16:41:15.322 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2023-11-08T16:41:15.323 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:41:15.323 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2023-11-08T16:41:15.323 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.323 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2023-11-08T16:41:15.324 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.324 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2023-11-08T16:41:15.324 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.324 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2023-11-08T16:41:15.324 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2023-11-08T16:41:15.325 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.325 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2023-11-08T16:41:15.325 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2023-11-08T16:41:15.325 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.326 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:41:15.326 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2023-11-08T16:41:15.326 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.326 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:41:15.326 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:41:15.327 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.327 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2023-11-08T16:41:15.327 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.328 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2023-11-08T16:41:15.328 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2023-11-08T16:41:15.328 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:41:15.328 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:41:15.329 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2023-11-08T16:41:15.329 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:41:15.329 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2023-11-08T16:41:15.329 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:41:15.330 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:41:15.330 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2023-11-08T16:41:15.330 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.330 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:41:15.330 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.331 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2023-11-08T16:41:15.331 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.331 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2023-11-08T16:41:15.331 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.331 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2023-11-08T16:41:15.332 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2023-11-08T16:41:15.332 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2023-11-08T16:41:15.332 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:41:15.332 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:41:15.332 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.333 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2023-11-08T16:41:15.333 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2023-11-08T16:41:15.333 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.333 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2023-11-08T16:41:15.334 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2023-11-08T16:41:15.334 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:41:15.334 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.334 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2023-11-08T16:41:15.334 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2023-11-08T16:41:15.335 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.335 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2023-11-08T16:41:15.335 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2023-11-08T16:41:15.335 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2023-11-08T16:41:15.335 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2023-11-08T16:41:15.336 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.336 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2023-11-08T16:41:15.336 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2023-11-08T16:41:15.336 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.336 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:41:15.337 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.337 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2023-11-08T16:41:15.337 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.337 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2023-11-08T16:41:15.338 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.338 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:41:15.338 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2023-11-08T16:41:15.338 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.338 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2023-11-08T16:41:15.339 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2023-11-08T16:41:15.339 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2023-11-08T16:41:15.339 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2023-11-08T16:41:15.339 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2023-11-08T16:41:15.339 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2023-11-08T16:41:15.340 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2023-11-08T16:41:15.340 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2023-11-08T16:41:15.340 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2023-11-08T16:41:15.340 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:41:15.340 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2023-11-08T16:41:15.341 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2023-11-08T16:41:15.341 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:41:15.341 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2023-11-08T16:41:15.341 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:41:15.341 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:41:15.342 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2023-11-08T16:41:15.342 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:41:15.342 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.342 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2023-11-08T16:41:15.343 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:41:15.343 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2023-11-08T16:41:15.343 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:41:15.343 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:41:15.343 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:41:15.344 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:41:15.344 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2023-11-08T16:41:15.344 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.344 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2023-11-08T16:41:15.344 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:41:15.345 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:41:15.345 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2023-11-08T16:41:15.345 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:41:15.345 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2023-11-08T16:41:15.345 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2023-11-08T16:41:15.346 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:41:15.346 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2023-11-08T16:41:15.346 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2023-11-08T16:41:15.346 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:41:15.347 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.347 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:41:15.347 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:41:15.347 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:41:15.347 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.348 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:41:15.348 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:41:15.348 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:41:15.348 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.348 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2023-11-08T16:41:15.349 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:41:15.349 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.349 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:41:15.350 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2023-11-08T16:41:15.350 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.350 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2023-11-08T16:41:15.350 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2023-11-08T16:41:15.350 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2023-11-08T16:41:15.351 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2023-11-08T16:41:15.351 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:41:15.351 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:41:15.351 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2023-11-08T16:41:15.351 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2023-11-08T16:41:15.352 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:41:15.352 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2023-11-08T16:41:15.352 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:41:15.352 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:41:15.352 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2023-11-08T16:41:15.353 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2023-11-08T16:41:15.353 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2023-11-08T16:41:15.353 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:41:15.353 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:41:15.353 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:41:15.354 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:41:15.354 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2023-11-08T16:41:15.354 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2023-11-08T16:41:15.354 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2023-11-08T16:41:15.354 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:41:15.355 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2023-11-08T16:41:15.355 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = False 2023-11-08T16:41:15.355 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2023-11-08T16:41:15.355 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = False 2023-11-08T16:41:15.355 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:41:15.356 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:41:15.356 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:41:15.356 INFO:teuthology.orchestra.run.smithi081.stdout: mds bal merge size = 0 2023-11-08T16:41:15.357 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-08T16:41:15.357 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2023-11-08T16:41:15.357 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:41:15.358 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2023-11-08T16:41:15.358 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.358 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2023-11-08T16:41:15.358 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.358 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2023-11-08T16:41:15.359 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.359 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2023-11-08T16:41:15.359 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2023-11-08T16:41:15.359 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.359 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2023-11-08T16:41:15.360 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2023-11-08T16:41:15.360 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.360 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:41:15.360 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2023-11-08T16:41:15.360 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.361 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:41:15.361 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:41:15.361 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.361 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2023-11-08T16:41:15.362 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.362 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2023-11-08T16:41:15.362 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2023-11-08T16:41:15.362 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:41:15.362 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:41:15.363 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2023-11-08T16:41:15.363 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:41:15.363 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2023-11-08T16:41:15.363 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:41:15.364 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:41:15.364 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2023-11-08T16:41:15.364 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.364 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:41:15.364 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.365 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2023-11-08T16:41:15.365 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.365 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2023-11-08T16:41:15.365 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.365 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2023-11-08T16:41:15.366 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2023-11-08T16:41:15.366 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2023-11-08T16:41:15.366 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:41:15.366 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:41:15.366 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.367 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2023-11-08T16:41:15.367 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2023-11-08T16:41:15.367 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.367 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2023-11-08T16:41:15.368 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2023-11-08T16:41:15.368 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:41:15.368 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.368 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2023-11-08T16:41:15.368 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2023-11-08T16:41:15.369 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.369 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2023-11-08T16:41:15.369 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2023-11-08T16:41:15.369 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2023-11-08T16:41:15.370 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2023-11-08T16:41:15.370 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.370 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2023-11-08T16:41:15.370 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2023-11-08T16:41:15.371 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.371 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:41:15.371 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.371 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2023-11-08T16:41:15.371 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.372 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2023-11-08T16:41:15.372 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.372 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:41:15.372 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2023-11-08T16:41:15.373 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.373 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2023-11-08T16:41:15.373 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2023-11-08T16:41:15.373 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2023-11-08T16:41:15.373 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2023-11-08T16:41:15.374 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2023-11-08T16:41:15.374 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2023-11-08T16:41:15.374 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2023-11-08T16:41:15.374 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2023-11-08T16:41:15.374 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2023-11-08T16:41:15.375 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:41:15.375 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2023-11-08T16:41:15.375 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2023-11-08T16:41:15.375 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:41:15.376 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2023-11-08T16:41:15.376 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:41:15.376 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:41:15.376 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2023-11-08T16:41:15.376 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:41:15.377 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.377 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2023-11-08T16:41:15.377 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:41:15.377 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2023-11-08T16:41:15.377 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:41:15.378 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:41:15.378 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:41:15.378 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:41:15.378 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2023-11-08T16:41:15.379 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.379 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2023-11-08T16:41:15.379 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:41:15.379 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:41:15.379 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2023-11-08T16:41:15.380 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:41:15.380 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2023-11-08T16:41:15.380 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2023-11-08T16:41:15.380 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:41:15.380 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2023-11-08T16:41:15.381 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2023-11-08T16:41:15.381 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:41:15.381 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.381 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:41:15.381 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:41:15.382 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:41:15.382 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.382 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:41:15.382 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:41:15.382 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:41:15.383 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.383 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2023-11-08T16:41:15.383 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:41:15.383 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.384 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:41:15.384 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2023-11-08T16:41:15.384 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.384 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2023-11-08T16:41:15.384 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2023-11-08T16:41:15.385 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2023-11-08T16:41:15.385 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2023-11-08T16:41:15.385 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:41:15.385 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:41:15.386 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2023-11-08T16:41:15.386 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2023-11-08T16:41:15.386 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:41:15.386 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2023-11-08T16:41:15.386 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:41:15.387 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:41:15.387 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2023-11-08T16:41:15.387 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2023-11-08T16:41:15.387 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2023-11-08T16:41:15.388 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:41:15.388 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:41:15.388 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:41:15.388 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:41:15.388 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2023-11-08T16:41:15.389 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2023-11-08T16:41:15.389 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2023-11-08T16:41:15.389 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:41:15.389 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2023-11-08T16:41:15.389 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = False 2023-11-08T16:41:15.390 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2023-11-08T16:41:15.390 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = False 2023-11-08T16:41:15.390 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:41:15.390 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:41:15.391 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:41:15.391 INFO:teuthology.orchestra.run.smithi093.stdout: mds bal merge size = 0 2023-11-08T16:41:15.393 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:41:15.402 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-08T16:41:15.568 INFO:teuthology.orchestra.run.smithi081.stdout:[global] 2023-11-08T16:41:15.568 INFO:teuthology.orchestra.run.smithi081.stdout: chdir = "" 2023-11-08T16:41:15.568 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:41:15.569 INFO:teuthology.orchestra.run.smithi081.stdout: auth supported = cephx 2023-11-08T16:41:15.569 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.569 INFO:teuthology.orchestra.run.smithi081.stdout: filestore xattr use omap = true 2023-11-08T16:41:15.569 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.569 INFO:teuthology.orchestra.run.smithi081.stdout: mon clock drift allowed = 1.000 2023-11-08T16:41:15.570 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.570 INFO:teuthology.orchestra.run.smithi081.stdout: osd crush chooseleaf type = 0 2023-11-08T16:41:15.570 INFO:teuthology.orchestra.run.smithi081.stdout: auth debug = true 2023-11-08T16:41:15.570 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.571 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on old message = true 2023-11-08T16:41:15.571 INFO:teuthology.orchestra.run.smithi081.stdout: ms die on bug = true 2023-11-08T16:41:15.571 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.571 INFO:teuthology.orchestra.run.smithi081.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:41:15.572 INFO:teuthology.orchestra.run.smithi081.stdout: mon pg warn max object skew = 0 2023-11-08T16:41:15.572 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.572 INFO:teuthology.orchestra.run.smithi081.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:41:15.572 INFO:teuthology.orchestra.run.smithi081.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:41:15.573 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.573 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default size = 2 2023-11-08T16:41:15.573 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.573 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow primary affinity = true 2023-11-08T16:41:15.573 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd allow pg remap = true 2023-11-08T16:41:15.574 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:41:15.574 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:41:15.574 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on no sortbitwise = false 2023-11-08T16:41:15.574 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:41:15.574 INFO:teuthology.orchestra.run.smithi081.stdout: mon warn on too few osds = false 2023-11-08T16:41:15.575 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:41:15.575 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:41:15.575 INFO:teuthology.orchestra.run.smithi081.stdout: mon_allow_pool_size_one = true 2023-11-08T16:41:15.575 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.576 INFO:teuthology.orchestra.run.smithi081.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:41:15.576 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.576 INFO:teuthology.orchestra.run.smithi081.stdout: osd default data pool replay window = 5 2023-11-08T16:41:15.576 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.577 INFO:teuthology.orchestra.run.smithi081.stdout: mon allow pool delete = true 2023-11-08T16:41:15.577 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.577 INFO:teuthology.orchestra.run.smithi081.stdout: mon cluster log file level = debug 2023-11-08T16:41:15.577 INFO:teuthology.orchestra.run.smithi081.stdout: debug asserts on shutdown = true 2023-11-08T16:41:15.577 INFO:teuthology.orchestra.run.smithi081.stdout: mon health detail to clog = false 2023-11-08T16:41:15.578 INFO:teuthology.orchestra.run.smithi081.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:41:15.578 INFO:teuthology.orchestra.run.smithi081.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:41:15.578 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.578 INFO:teuthology.orchestra.run.smithi081.stdout:[osd] 2023-11-08T16:41:15.579 INFO:teuthology.orchestra.run.smithi081.stdout: osd journal size = 100 2023-11-08T16:41:15.579 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.579 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub load threshold = 5.0 2023-11-08T16:41:15.579 INFO:teuthology.orchestra.run.smithi081.stdout: osd scrub max interval = 600 2023-11-08T16:41:15.579 INFO:teuthology.orchestra.run.smithi081.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:41:15.580 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.580 INFO:teuthology.orchestra.run.smithi081.stdout: osd recover clone overlap = true 2023-11-08T16:41:15.580 INFO:teuthology.orchestra.run.smithi081.stdout: osd recovery max chunk = 1048576 2023-11-08T16:41:15.580 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.581 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug shutdown = true 2023-11-08T16:41:15.581 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug op order = true 2023-11-08T16:41:15.581 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug verify stray on activate = true 2023-11-08T16:41:15.581 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug trim objects = true 2023-11-08T16:41:15.581 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.582 INFO:teuthology.orchestra.run.smithi081.stdout: osd open classes on start = true 2023-11-08T16:41:15.582 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug pg log writeout = true 2023-11-08T16:41:15.582 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.582 INFO:teuthology.orchestra.run.smithi081.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:41:15.583 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.583 INFO:teuthology.orchestra.run.smithi081.stdout: osd map max advance = 10 2023-11-08T16:41:15.583 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.583 INFO:teuthology.orchestra.run.smithi081.stdout: journal zero on create = true 2023-11-08T16:41:15.583 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.584 INFO:teuthology.orchestra.run.smithi081.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:41:15.584 INFO:teuthology.orchestra.run.smithi081.stdout: filestore apply finisher threads = 3 2023-11-08T16:41:15.584 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.584 INFO:teuthology.orchestra.run.smithi081.stdout: bdev debug aio = true 2023-11-08T16:41:15.584 INFO:teuthology.orchestra.run.smithi081.stdout: osd debug misdirected ops = true 2023-11-08T16:41:15.585 INFO:teuthology.orchestra.run.smithi081.stdout: bdev async discard = True 2023-11-08T16:41:15.585 INFO:teuthology.orchestra.run.smithi081.stdout: bdev enable discard = True 2023-11-08T16:41:15.585 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore allocator = bitmap 2023-11-08T16:41:15.585 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore block size = 96636764160 2023-11-08T16:41:15.585 INFO:teuthology.orchestra.run.smithi081.stdout: bluestore fsck on mount = True 2023-11-08T16:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluefs = 1/20 2023-11-08T16:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: debug bluestore = 1/20 2023-11-08T16:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: debug osd = 20 2023-11-08T16:41:15.586 INFO:teuthology.orchestra.run.smithi081.stdout: debug rocksdb = 4/10 2023-11-08T16:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd full ratio = 0.9 2023-11-08T16:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:41:15.587 INFO:teuthology.orchestra.run.smithi081.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: osd objectstore = bluestore 2023-11-08T16:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.588 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr] 2023-11-08T16:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: debug mgr = 20 2023-11-08T16:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:41:15.589 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: mgr/telemetry/nag = false 2023-11-08T16:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.590 INFO:teuthology.orchestra.run.smithi081.stdout:[mon] 2023-11-08T16:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: debug mon = 20 2023-11-08T16:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: debug paxos = 20 2023-11-08T16:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: debug auth = 20 2023-11-08T16:41:15.591 INFO:teuthology.orchestra.run.smithi081.stdout: mon data avail warn = 5 2023-11-08T16:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: mon mgr mkfs grace = 240 2023-11-08T16:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd reporter subtree level = osd 2023-11-08T16:41:15.592 INFO:teuthology.orchestra.run.smithi081.stdout: mon osd prime pg temp = true 2023-11-08T16:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:41:15.593 INFO:teuthology.orchestra.run.smithi081.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:41:15.594 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: # 1m isn't quite enough 2023-11-08T16:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.595 INFO:teuthology.orchestra.run.smithi081.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: mon op complaint time = 120 2023-11-08T16:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout:[client] 2023-11-08T16:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: rgw cache enabled = true 2023-11-08T16:41:15.596 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable ops log = true 2023-11-08T16:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: rgw enable usage log = true 2023-11-08T16:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: client mount timeout = 600 2023-11-08T16:41:15.597 INFO:teuthology.orchestra.run.smithi081.stdout: debug client = 20 2023-11-08T16:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: pid file = "" 2023-11-08T16:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:41:15.598 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.a] 2023-11-08T16:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.b] 2023-11-08T16:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout:[mon.c] 2023-11-08T16:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.a] 2023-11-08T16:41:15.599 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.c] 2023-11-08T16:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.b] 2023-11-08T16:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout:[mds.d] 2023-11-08T16:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout:[mds] 2023-11-08T16:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds = 20 2023-11-08T16:41:15.600 INFO:teuthology.orchestra.run.smithi081.stdout: debug mds balancer = 20 2023-11-08T16:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: debug ms = 1 2023-11-08T16:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug frag = True 2023-11-08T16:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: mds debug scatterstat = False 2023-11-08T16:41:15.601 INFO:teuthology.orchestra.run.smithi081.stdout: mds op complaint time = 180 2023-11-08T16:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: mds verify scatter = False 2023-11-08T16:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: osd op complaint time = 180 2023-11-08T16:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: rados mon op timeout = 900 2023-11-08T16:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: rados osd op timeout = 900 2023-11-08T16:41:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: mds bal merge size = 0 2023-11-08T16:41:15.603 INFO:teuthology.orchestra.run.smithi081.stdout: mds bal split size = 10000 2023-11-08T16:41:15.603 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-11-08T16:41:15.604 INFO:teuthology.orchestra.run.smithi093.stdout: chdir = "" 2023-11-08T16:41:15.604 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-08T16:41:15.604 INFO:teuthology.orchestra.run.smithi093.stdout: auth supported = cephx 2023-11-08T16:41:15.604 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.604 INFO:teuthology.orchestra.run.smithi093.stdout: filestore xattr use omap = true 2023-11-08T16:41:15.605 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.605 INFO:teuthology.orchestra.run.smithi093.stdout: mon clock drift allowed = 1.000 2023-11-08T16:41:15.605 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.605 INFO:teuthology.orchestra.run.smithi093.stdout: osd crush chooseleaf type = 0 2023-11-08T16:41:15.605 INFO:teuthology.orchestra.run.smithi093.stdout: auth debug = true 2023-11-08T16:41:15.606 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.606 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on old message = true 2023-11-08T16:41:15.606 INFO:teuthology.orchestra.run.smithi093.stdout: ms die on bug = true 2023-11-08T16:41:15.606 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.606 INFO:teuthology.orchestra.run.smithi093.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-08T16:41:15.607 INFO:teuthology.orchestra.run.smithi093.stdout: mon pg warn max object skew = 0 2023-11-08T16:41:15.607 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.607 INFO:teuthology.orchestra.run.smithi093.stdout: # disable pg_autoscaler by default for new pools 2023-11-08T16:41:15.607 INFO:teuthology.orchestra.run.smithi093.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-08T16:41:15.608 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.608 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default size = 2 2023-11-08T16:41:15.608 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.608 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow primary affinity = true 2023-11-08T16:41:15.608 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd allow pg remap = true 2023-11-08T16:41:15.609 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on legacy crush tunables = false 2023-11-08T16:41:15.609 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on crush straw calc version zero = false 2023-11-08T16:41:15.609 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on no sortbitwise = false 2023-11-08T16:41:15.609 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on osd down out interval zero = false 2023-11-08T16:41:15.610 INFO:teuthology.orchestra.run.smithi093.stdout: mon warn on too few osds = false 2023-11-08T16:41:15.610 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-08T16:41:15.610 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-08T16:41:15.610 INFO:teuthology.orchestra.run.smithi093.stdout: mon_allow_pool_size_one = true 2023-11-08T16:41:15.610 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.611 INFO:teuthology.orchestra.run.smithi093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-08T16:41:15.611 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.611 INFO:teuthology.orchestra.run.smithi093.stdout: osd default data pool replay window = 5 2023-11-08T16:41:15.611 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.611 INFO:teuthology.orchestra.run.smithi093.stdout: mon allow pool delete = true 2023-11-08T16:41:15.612 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.612 INFO:teuthology.orchestra.run.smithi093.stdout: mon cluster log file level = debug 2023-11-08T16:41:15.612 INFO:teuthology.orchestra.run.smithi093.stdout: debug asserts on shutdown = true 2023-11-08T16:41:15.612 INFO:teuthology.orchestra.run.smithi093.stdout: mon health detail to clog = false 2023-11-08T16:41:15.613 INFO:teuthology.orchestra.run.smithi093.stdout: mon host = "172.21.15.81,172.21.15.93,[v2:172.21.15.93:3301,v1:172.21.15.93:6790]" 2023-11-08T16:41:15.613 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 49b142e1-3087-4ba1-853b-235f756dae94 2023-11-08T16:41:15.613 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.613 INFO:teuthology.orchestra.run.smithi093.stdout:[osd] 2023-11-08T16:41:15.613 INFO:teuthology.orchestra.run.smithi093.stdout: osd journal size = 100 2023-11-08T16:41:15.614 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.614 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub load threshold = 5.0 2023-11-08T16:41:15.614 INFO:teuthology.orchestra.run.smithi093.stdout: osd scrub max interval = 600 2023-11-08T16:41:15.614 INFO:teuthology.orchestra.run.smithi093.stdout: osd mclock profile = high_recovery_ops 2023-11-08T16:41:15.615 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.615 INFO:teuthology.orchestra.run.smithi093.stdout: osd recover clone overlap = true 2023-11-08T16:41:15.615 INFO:teuthology.orchestra.run.smithi093.stdout: osd recovery max chunk = 1048576 2023-11-08T16:41:15.615 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.615 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug shutdown = true 2023-11-08T16:41:15.616 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug op order = true 2023-11-08T16:41:15.616 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug verify stray on activate = true 2023-11-08T16:41:15.616 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug trim objects = true 2023-11-08T16:41:15.616 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.617 INFO:teuthology.orchestra.run.smithi093.stdout: osd open classes on start = true 2023-11-08T16:41:15.617 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug pg log writeout = true 2023-11-08T16:41:15.617 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.617 INFO:teuthology.orchestra.run.smithi093.stdout: osd deep scrub update digest min age = 30 2023-11-08T16:41:15.617 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.618 INFO:teuthology.orchestra.run.smithi093.stdout: osd map max advance = 10 2023-11-08T16:41:15.618 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.618 INFO:teuthology.orchestra.run.smithi093.stdout: journal zero on create = true 2023-11-08T16:41:15.618 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.619 INFO:teuthology.orchestra.run.smithi093.stdout: filestore ondisk finisher threads = 3 2023-11-08T16:41:15.619 INFO:teuthology.orchestra.run.smithi093.stdout: filestore apply finisher threads = 3 2023-11-08T16:41:15.619 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.619 INFO:teuthology.orchestra.run.smithi093.stdout: bdev debug aio = true 2023-11-08T16:41:15.619 INFO:teuthology.orchestra.run.smithi093.stdout: osd debug misdirected ops = true 2023-11-08T16:41:15.620 INFO:teuthology.orchestra.run.smithi093.stdout: bdev async discard = True 2023-11-08T16:41:15.620 INFO:teuthology.orchestra.run.smithi093.stdout: bdev enable discard = True 2023-11-08T16:41:15.620 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore allocator = bitmap 2023-11-08T16:41:15.620 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore block size = 96636764160 2023-11-08T16:41:15.620 INFO:teuthology.orchestra.run.smithi093.stdout: bluestore fsck on mount = True 2023-11-08T16:41:15.621 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluefs = 1/20 2023-11-08T16:41:15.621 INFO:teuthology.orchestra.run.smithi093.stdout: debug bluestore = 1/20 2023-11-08T16:41:15.621 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:41:15.621 INFO:teuthology.orchestra.run.smithi093.stdout: debug osd = 20 2023-11-08T16:41:15.621 INFO:teuthology.orchestra.run.smithi093.stdout: debug rocksdb = 4/10 2023-11-08T16:41:15.622 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd backfillfull_ratio = 0.85 2023-11-08T16:41:15.622 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd full ratio = 0.9 2023-11-08T16:41:15.622 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd nearfull ratio = 0.8 2023-11-08T16:41:15.622 INFO:teuthology.orchestra.run.smithi093.stdout: osd failsafe full ratio = 0.95 2023-11-08T16:41:15.622 INFO:teuthology.orchestra.run.smithi093.stdout: osd objectstore = bluestore 2023-11-08T16:41:15.623 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:41:15.623 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.623 INFO:teuthology.orchestra.run.smithi093.stdout:[mgr] 2023-11-08T16:41:15.623 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:41:15.624 INFO:teuthology.orchestra.run.smithi093.stdout: debug mgr = 20 2023-11-08T16:41:15.624 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:41:15.624 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:41:15.624 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:41:15.624 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:41:15.625 INFO:teuthology.orchestra.run.smithi093.stdout: mgr/telemetry/nag = false 2023-11-08T16:41:15.625 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.625 INFO:teuthology.orchestra.run.smithi093.stdout:[mon] 2023-11-08T16:41:15.625 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:41:15.626 INFO:teuthology.orchestra.run.smithi093.stdout: debug mon = 20 2023-11-08T16:41:15.626 INFO:teuthology.orchestra.run.smithi093.stdout: debug paxos = 20 2023-11-08T16:41:15.626 INFO:teuthology.orchestra.run.smithi093.stdout: debug auth = 20 2023-11-08T16:41:15.626 INFO:teuthology.orchestra.run.smithi093.stdout: mon data avail warn = 5 2023-11-08T16:41:15.627 INFO:teuthology.orchestra.run.smithi093.stdout: mon mgr mkfs grace = 240 2023-11-08T16:41:15.627 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min pgs per osd = 4 2023-11-08T16:41:15.627 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd reporter subtree level = osd 2023-11-08T16:41:15.627 INFO:teuthology.orchestra.run.smithi093.stdout: mon osd prime pg temp = true 2023-11-08T16:41:15.627 INFO:teuthology.orchestra.run.smithi093.stdout: mon reweight min bytes per osd = 10 2023-11-08T16:41:15.628 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.628 INFO:teuthology.orchestra.run.smithi093.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-08T16:41:15.628 INFO:teuthology.orchestra.run.smithi093.stdout: auth mon ticket ttl = 660 # 11m 2023-11-08T16:41:15.628 INFO:teuthology.orchestra.run.smithi093.stdout: auth service ticket ttl = 240 # 4m 2023-11-08T16:41:15.629 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.629 INFO:teuthology.orchestra.run.smithi093.stdout: # don't complain about insecure global_id in the test suite 2023-11-08T16:41:15.629 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-08T16:41:15.629 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-08T16:41:15.629 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.630 INFO:teuthology.orchestra.run.smithi093.stdout: # 1m isn't quite enough 2023-11-08T16:41:15.630 INFO:teuthology.orchestra.run.smithi093.stdout: mon_down_mkfs_grace = 2m 2023-11-08T16:41:15.630 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.630 INFO:teuthology.orchestra.run.smithi093.stdout: mon_warn_on_filestore_osds = false 2023-11-08T16:41:15.631 INFO:teuthology.orchestra.run.smithi093.stdout: mon op complaint time = 120 2023-11-08T16:41:15.631 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-11-08T16:41:15.631 INFO:teuthology.orchestra.run.smithi093.stdout:[client] 2023-11-08T16:41:15.631 INFO:teuthology.orchestra.run.smithi093.stdout: rgw cache enabled = true 2023-11-08T16:41:15.632 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable ops log = true 2023-11-08T16:41:15.632 INFO:teuthology.orchestra.run.smithi093.stdout: rgw enable usage log = true 2023-11-08T16:41:15.632 INFO:teuthology.orchestra.run.smithi093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-08T16:41:15.632 INFO:teuthology.orchestra.run.smithi093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-08T16:41:15.632 INFO:teuthology.orchestra.run.smithi093.stdout: client mount timeout = 600 2023-11-08T16:41:15.633 INFO:teuthology.orchestra.run.smithi093.stdout: debug client = 20 2023-11-08T16:41:15.633 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:41:15.633 INFO:teuthology.orchestra.run.smithi093.stdout: pid file = "" 2023-11-08T16:41:15.633 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:41:15.633 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:41:15.634 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.a] 2023-11-08T16:41:15.634 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.b] 2023-11-08T16:41:15.634 INFO:teuthology.orchestra.run.smithi093.stdout:[mon.c] 2023-11-08T16:41:15.634 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a] 2023-11-08T16:41:15.635 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c] 2023-11-08T16:41:15.635 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b] 2023-11-08T16:41:15.635 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d] 2023-11-08T16:41:15.635 INFO:teuthology.orchestra.run.smithi093.stdout:[mds] 2023-11-08T16:41:15.635 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds = 20 2023-11-08T16:41:15.636 INFO:teuthology.orchestra.run.smithi093.stdout: debug mds balancer = 20 2023-11-08T16:41:15.636 INFO:teuthology.orchestra.run.smithi093.stdout: debug ms = 1 2023-11-08T16:41:15.636 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug frag = True 2023-11-08T16:41:15.636 INFO:teuthology.orchestra.run.smithi093.stdout: mds debug scatterstat = False 2023-11-08T16:41:15.636 INFO:teuthology.orchestra.run.smithi093.stdout: mds op complaint time = 180 2023-11-08T16:41:15.637 INFO:teuthology.orchestra.run.smithi093.stdout: mds verify scatter = False 2023-11-08T16:41:15.637 INFO:teuthology.orchestra.run.smithi093.stdout: osd op complaint time = 180 2023-11-08T16:41:15.637 INFO:teuthology.orchestra.run.smithi093.stdout: rados mon op timeout = 900 2023-11-08T16:41:15.637 INFO:teuthology.orchestra.run.smithi093.stdout: rados osd op timeout = 900 2023-11-08T16:41:15.638 INFO:teuthology.orchestra.run.smithi093.stdout: mds bal merge size = 0 2023-11-08T16:41:15.638 INFO:teuthology.orchestra.run.smithi093.stdout: mds bal split size = 10000 2023-11-08T16:41:15.640 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:41:15.640 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:41:15.640 DEBUG:teuthology.orchestra.run.smithi081:> path = os.path.join("/home/ubuntu/cephtest/mnt.0", "subdir") 2023-11-08T16:41:15.641 DEBUG:teuthology.orchestra.run.smithi081:> os.mkdir(path) 2023-11-08T16:41:15.641 DEBUG:teuthology.orchestra.run.smithi081:> for n in range(0, 100): 2023-11-08T16:41:15.641 DEBUG:teuthology.orchestra.run.smithi081:> open(os.path.join(path, "%s" % n), '"'"'w'"'"').write("%s" % n) 2023-11-08T16:41:15.642 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:41:16.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:41:16.430 DEBUG:teuthology.orchestra.run.smithi081:> stdin-killer -- python3 -c ' 2023-11-08T16:41:16.431 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:41:16.432 DEBUG:teuthology.orchestra.run.smithi081:> import stat 2023-11-08T16:41:16.432 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:16.433 DEBUG:teuthology.orchestra.run.smithi081:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2023-11-08T16:41:16.433 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:41:16.617 INFO:teuthology.orchestra.run.smithi081.stdout:1099511627776 2023-11-08T16:41:16.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:41:16.631 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:41:16.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.890+0000 7f8f26de9640 1 Processor -- start 2023-11-08T16:41:16.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.890+0000 7f8f26de9640 1 -- start start 2023-11-08T16:41:16.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.891+0000 7f8f26de9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f2015a160 0x7f8f200ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:16.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.891+0000 7f8f26de9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f200706a0 con 0x7f8f20159d80 2023-11-08T16:41:16.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.891+0000 7f8f26de9640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f200cf230 con 0x7f8f200cee20 2023-11-08T16:41:16.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.891+0000 7f8f26de9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f200ca510 con 0x7f8f2015a160 2023-11-08T16:41:16.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.892+0000 7f8f25de7640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f8f20159d80 0x7f8f2006ff90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39516/0 (socket says 172.21.15.81:39516) 2023-11-08T16:41:16.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.892+0000 7f8f25de7640 1 -- 172.21.15.81:0/4179769647 learned_addr learned my addr 172.21.15.81:0/4179769647 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:16.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.892+0000 7f8f255e6640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f2015a160 0x7f8f200ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:16.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.892+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171784507 0 0) 0x7f8f200706a0 con 0x7f8f20159d80 2023-11-08T16:41:16.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.892+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f0c003520 con 0x7f8f20159d80 2023-11-08T16:41:16.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 174335348 0 0) 0x7f8f0c003520 con 0x7f8f20159d80 2023-11-08T16:41:16.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f2015a160 msgr2=0x7f8f200ce8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:16.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f2015a160 0x7f8f200ce8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:16.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 >> v1:172.21.15.93:6789/0 conn(0x7f8f200cee20 legacy=0x7f8f200c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:16.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f200c9f50 con 0x7f8f20159d80 2023-11-08T16:41:16.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f8f1c001670 con 0x7f8f20159d80 2023-11-08T16:41:16.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 >> v1:172.21.15.81:6789/0 conn(0x7f8f20159d80 legacy=0x7f8f2006ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:16.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 0x7f8f0c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:16.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0c0047c0 0x7f8f0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:16.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f0c0051d0 0x7f8f0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:16.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f200ca510 con 0x7f8f0c0039a0 2023-11-08T16:41:16.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c00bef0 con 0x7f8f0c0047c0 2023-11-08T16:41:16.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c00c0b0 con 0x7f8f0c0051d0 2023-11-08T16:41:16.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.893+0000 7f8f25de7640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 0x7f8f0c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:16.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.894+0000 7f8f255e6640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0c0047c0 0x7f8f0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:16.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.894+0000 7f8f265e8640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f0c0051d0 0x7f8f0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:16.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.894+0000 7f8f25de7640 1 -- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f0c0051d0 msgr2=0x7f8f0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:16.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.894+0000 7f8f25de7640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f0c0051d0 0x7f8f0c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:16.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.894+0000 7f8f25de7640 1 -- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0c0047c0 msgr2=0x7f8f0c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:16.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.894+0000 7f8f25de7640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0c0047c0 0x7f8f0c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:16.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.894+0000 7f8f25de7640 1 -- 172.21.15.81:0/4179769647 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c003520 con 0x7f8f0c0039a0 2023-11-08T16:41:16.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.894+0000 7f8f255e6640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f0c0047c0 0x7f8f0c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:16.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.894+0000 7f8f265e8640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f0c0051d0 0x7f8f0c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:16.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.895+0000 7f8f25de7640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 0x7f8f0c003dc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c001eb0 tx=0x7f8f0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179084f543d0694a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:16.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.895+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c005ad0 con 0x7f8f0c0039a0 2023-11-08T16:41:16.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.895+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c005c30 con 0x7f8f0c0039a0 2023-11-08T16:41:16.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.895+0000 7f8f24de5640 1 -- 172.21.15.81:0/4179769647 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c02c7a0 con 0x7f8f0c0039a0 2023-11-08T16:41:16.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.896+0000 7f8f26de9640 1 -- 172.21.15.81:0/4179769647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 msgr2=0x7f8f0c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:16.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.896+0000 7f8f26de9640 1 --2- 172.21.15.81:0/4179769647 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 0x7f8f0c003dc0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c001eb0 tx=0x7f8f0c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:16.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.896+0000 7f8f265e8640 1 -- 172.21.15.81:0/4179769647 reap_dead start 2023-11-08T16:41:16.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.896+0000 7f8f26de9640 1 -- 172.21.15.81:0/4179769647 shutdown_connections 2023-11-08T16:41:16.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.896+0000 7f8f26de9640 1 -- 172.21.15.81:0/4179769647 >> 172.21.15.81:0/4179769647 conn(0x7f8f200b06e0 msgr2=0x7f8f200b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:16.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.896+0000 7f8f26de9640 1 -- 172.21.15.81:0/4179769647 shutdown_connections 2023-11-08T16:41:16.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.896+0000 7f8f26de9640 1 -- 172.21.15.81:0/4179769647 wait complete. 2023-11-08T16:41:16.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.897+0000 7f8f26de9640 1 Processor -- start 2023-11-08T16:41:16.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.898+0000 7f8f26de9640 1 -- start start 2023-11-08T16:41:16.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.898+0000 7f8f26de9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 0x7f8f20157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:16.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f26de9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f20151e00 0x7f8f20152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:16.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f26de9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f2014db60 0x7f8f2014dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:16.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f26de9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f200cd590 con 0x7f8f0c0039a0 2023-11-08T16:41:16.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f26de9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2014e520 con 0x7f8f2014db60 2023-11-08T16:41:16.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f26de9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f201e3830 con 0x7f8f20151e00 2023-11-08T16:41:16.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f25de7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 0x7f8f20157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:16.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f25de7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 0x7f8f20157960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36988/0 (socket says 172.21.15.81:36988) 2023-11-08T16:41:16.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f25de7640 1 -- 172.21.15.81:0/3817394257 learned_addr learned my addr 172.21.15.81:0/3817394257 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:16.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f255e6640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f20151e00 0x7f8f20152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:16.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f265e8640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f2014db60 0x7f8f2014dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:16.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f25de7640 1 -- 172.21.15.81:0/3817394257 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f20151e00 msgr2=0x7f8f20152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:16.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f25de7640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f20151e00 0x7f8f20152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:16.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f25de7640 1 -- 172.21.15.81:0/3817394257 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f2014db60 msgr2=0x7f8f2014dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:16.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f25de7640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f2014db60 0x7f8f2014dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:16.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f25de7640 1 -- 172.21.15.81:0/3817394257 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f200b15f0 con 0x7f8f0c0039a0 2023-11-08T16:41:16.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f255e6640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f20151e00 0x7f8f20152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:16.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.899+0000 7f8f265e8640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f2014db60 0x7f8f2014dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:16.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.900+0000 7f8f25de7640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 0x7f8f20157960 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c001eb0 tx=0x7f8f1c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:16.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.900+0000 7f8f127fc640 1 -- 172.21.15.81:0/3817394257 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c03c040 con 0x7f8f0c0039a0 2023-11-08T16:41:16.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.900+0000 7f8f127fc640 1 -- 172.21.15.81:0/3817394257 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f1c02dc40 con 0x7f8f0c0039a0 2023-11-08T16:41:16.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.900+0000 7f8f127fc640 1 -- 172.21.15.81:0/3817394257 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f1c007070 con 0x7f8f0c0039a0 2023-11-08T16:41:16.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.900+0000 7f8f26de9640 1 -- 172.21.15.81:0/3817394257 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2014f8b0 con 0x7f8f0c0039a0 2023-11-08T16:41:16.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.900+0000 7f8f26de9640 1 -- 172.21.15.81:0/3817394257 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2014fe30 con 0x7f8f0c0039a0 2023-11-08T16:41:16.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.901+0000 7f8f26de9640 1 -- 172.21.15.81:0/3817394257 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f200ce860 con 0x7f8f0c0039a0 2023-11-08T16:41:16.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.902+0000 7f8f127fc640 1 -- 172.21.15.81:0/3817394257 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f1c032070 con 0x7f8f0c0039a0 2023-11-08T16:41:16.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.903+0000 7f8f127fc640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f0406ae70 0x7f8f0406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:16.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.903+0000 7f8f127fc640 1 -- 172.21.15.81:0/3817394257 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f1c0ab060 con 0x7f8f0c0039a0 2023-11-08T16:41:16.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.903+0000 7f8f255e6640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f0406ae70 0x7f8f0406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:16.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.904+0000 7f8f255e6640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f0406ae70 0x7f8f0406d330 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8f2014e940 tx=0x7f8f14008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:16.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:16.908+0000 7f8f127fc640 1 -- 172.21.15.81:0/3817394257 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f1c0ddca0 con 0x7f8f0c0039a0 2023-11-08T16:41:17.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.135+0000 7f8f26de9640 1 -- 172.21.15.81:0/3817394257 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f2004ece0 con 0x7f8f0c0039a0 2023-11-08T16:41:17.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.136+0000 7f8f127fc640 1 -- 172.21.15.81:0/3817394257 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4918 (secure 0 0 0) 0x7f8f1c076100 con 0x7f8f0c0039a0 2023-11-08T16:41:17.138 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:17.139 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/296230272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":296230272},{"type":"v1","addr":"172.21.15.81:6837","nonce":296230272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:08.773407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8574":{"gid":8574,"name":"d","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6835/4021510722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":4021510722},{"type":"v1","addr":"172.21.15.93:6835","nonce":4021510722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:41:17.139 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 107 2023-11-08T16:41:17.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.139+0000 7f8f26de9640 1 -- 172.21.15.81:0/3817394257 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f0406ae70 msgr2=0x7f8f0406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:17.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.139+0000 7f8f26de9640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f0406ae70 0x7f8f0406d330 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8f2014e940 tx=0x7f8f14008040 comp rx=0 tx=0).stop 2023-11-08T16:41:17.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.139+0000 7f8f26de9640 1 -- 172.21.15.81:0/3817394257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 msgr2=0x7f8f20157960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:17.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.139+0000 7f8f26de9640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 0x7f8f20157960 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c001eb0 tx=0x7f8f1c001930 comp rx=0 tx=0).stop 2023-11-08T16:41:17.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.139+0000 7f8f26de9640 1 -- 172.21.15.81:0/3817394257 shutdown_connections 2023-11-08T16:41:17.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.139+0000 7f8f26de9640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f0406ae70 0x7f8f0406d330 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.139+0000 7f8f26de9640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f2014db60 0x7f8f2014dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.139+0000 7f8f26de9640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f20151e00 0x7f8f20152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.139+0000 7f8f26de9640 1 --2- 172.21.15.81:0/3817394257 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f0c0039a0 0x7f8f20157960 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.139+0000 7f8f26de9640 1 -- 172.21.15.81:0/3817394257 >> 172.21.15.81:0/3817394257 conn(0x7f8f200b06e0 msgr2=0x7f8f200c4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:17.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.140+0000 7f8f26de9640 1 -- 172.21.15.81:0/3817394257 shutdown_connections 2023-11-08T16:41:17.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.140+0000 7f8f26de9640 1 -- 172.21.15.81:0/3817394257 wait complete. 2023-11-08T16:41:17.159 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:41:17.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.364+0000 7ff7662d7640 1 Processor -- start 2023-11-08T16:41:17.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.364+0000 7ff7662d7640 1 -- start start 2023-11-08T16:41:17.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.365+0000 7ff7662d7640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff760143800 0x7ff7600b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:17.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.365+0000 7ff7662d7640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ff7600cca70 0x7ff76015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:17.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.365+0000 7ff7662d7640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7600714f0 con 0x7ff76015df20 2023-11-08T16:41:17.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.365+0000 7ff7662d7640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7600b2df0 con 0x7ff760143800 2023-11-08T16:41:17.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.365+0000 7ff7662d7640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff76015e330 con 0x7ff7600cca70 2023-11-08T16:41:17.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.366+0000 7ff75ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff760143800 0x7ff7600b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:17.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.366+0000 7ff75ffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7ff760143800 0x7ff7600b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:36994/0 (socket says 172.21.15.81:36994) 2023-11-08T16:41:17.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.366+0000 7ff75ffff640 1 -- 172.21.15.81:0/368040595 learned_addr learned my addr 172.21.15.81:0/368040595 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:17.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.366+0000 7ff757fff640 1 --2- 172.21.15.81:0/368040595 >> v2:172.21.15.93:3300/0 conn(0x7ff7600cca70 0x7ff76015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:17.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.366+0000 7ff75ffff640 1 -- 172.21.15.81:0/368040595 >> v2:172.21.15.93:3300/0 conn(0x7ff7600cca70 msgr2=0x7ff76015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:17.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.366+0000 7ff75ffff640 1 --2- 172.21.15.81:0/368040595 >> v2:172.21.15.93:3300/0 conn(0x7ff7600cca70 0x7ff76015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.366+0000 7ff75ffff640 1 -- 172.21.15.81:0/368040595 >> v1:172.21.15.93:6789/0 conn(0x7ff76015df20 legacy=0x7ff76015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:17.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.366+0000 7ff75ffff640 1 -- 172.21.15.81:0/368040595 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7600cd1e0 con 0x7ff760143800 2023-11-08T16:41:17.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.366+0000 7ff757fff640 1 --2- 172.21.15.81:0/368040595 >> v2:172.21.15.93:3300/0 conn(0x7ff7600cca70 0x7ff76015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:17.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.367+0000 7ff75ffff640 1 --2- 172.21.15.81:0/368040595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760143800 0x7ff7600b28b0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ff7600cc8c0 tx=0x7ff75002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe43f557c65ccb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:17.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.368+0000 7ff75f7fe640 1 -- 172.21.15.81:0/368040595 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750031870 con 0x7ff760143800 2023-11-08T16:41:17.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.368+0000 7ff75f7fe640 1 -- 172.21.15.81:0/368040595 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7500319d0 con 0x7ff760143800 2023-11-08T16:41:17.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.368+0000 7ff75f7fe640 1 -- 172.21.15.81:0/368040595 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff750031d60 con 0x7ff760143800 2023-11-08T16:41:17.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.368+0000 7ff7662d7640 1 -- 172.21.15.81:0/368040595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760143800 msgr2=0x7ff7600b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:17.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.368+0000 7ff7662d7640 1 --2- 172.21.15.81:0/368040595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760143800 0x7ff7600b28b0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ff7600cc8c0 tx=0x7ff75002b910 comp rx=0 tx=0).stop 2023-11-08T16:41:17.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.368+0000 7ff7662d7640 1 -- 172.21.15.81:0/368040595 shutdown_connections 2023-11-08T16:41:17.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.368+0000 7ff7662d7640 1 --2- 172.21.15.81:0/368040595 >> v2:172.21.15.93:3300/0 conn(0x7ff7600cca70 0x7ff76015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.368+0000 7ff7662d7640 1 --2- 172.21.15.81:0/368040595 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760143800 0x7ff7600b28b0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.368+0000 7ff7662d7640 1 -- 172.21.15.81:0/368040595 >> 172.21.15.81:0/368040595 conn(0x7ff76006a6a0 msgr2=0x7ff7600b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:17.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.369+0000 7ff7662d7640 1 -- 172.21.15.81:0/368040595 shutdown_connections 2023-11-08T16:41:17.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.369+0000 7ff7662d7640 1 -- 172.21.15.81:0/368040595 wait complete. 2023-11-08T16:41:17.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.371+0000 7ff7662d7640 1 Processor -- start 2023-11-08T16:41:17.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.371+0000 7ff7662d7640 1 -- start start 2023-11-08T16:41:17.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.372+0000 7ff7662d7640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7600cca70 0x7ff7601599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:17.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.372+0000 7ff7662d7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760143800 0x7ff760138400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:17.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.372+0000 7ff7662d7640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff76015df20 0x7ff7600702a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:17.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.372+0000 7ff7662d7640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff760159f20 con 0x7ff760143800 2023-11-08T16:41:17.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.372+0000 7ff7662d7640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76006ccc0 con 0x7ff7600cca70 2023-11-08T16:41:17.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.372+0000 7ff7662d7640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff760158350 con 0x7ff76015df20 2023-11-08T16:41:17.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.372+0000 7ff757fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760143800 0x7ff760138400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:17.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.372+0000 7ff757fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760143800 0x7ff760138400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37008/0 (socket says 172.21.15.81:37008) 2023-11-08T16:41:17.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.372+0000 7ff757fff640 1 -- 172.21.15.81:0/3409774804 learned_addr learned my addr 172.21.15.81:0/3409774804 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:17.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.373+0000 7ff75ffff640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7600cca70 0x7ff7601599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:17.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.373+0000 7ff764874640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff76015df20 0x7ff7600702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:17.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.373+0000 7ff757fff640 1 -- 172.21.15.81:0/3409774804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff76015df20 msgr2=0x7ff7600702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:17.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.373+0000 7ff757fff640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff76015df20 0x7ff7600702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.373+0000 7ff757fff640 1 -- 172.21.15.81:0/3409774804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7600cca70 msgr2=0x7ff7601599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:17.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.373+0000 7ff757fff640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7600cca70 0x7ff7601599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.373+0000 7ff757fff640 1 -- 172.21.15.81:0/3409774804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500094d0 con 0x7ff760143800 2023-11-08T16:41:17.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.373+0000 7ff757fff640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760143800 0x7ff760138400 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7ff74800c7f0 tx=0x7ff74800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:17.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.374+0000 7ff75d7fa640 1 -- 172.21.15.81:0/3409774804 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748007a30 con 0x7ff760143800 2023-11-08T16:41:17.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.374+0000 7ff75d7fa640 1 -- 172.21.15.81:0/3409774804 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff748007b90 con 0x7ff760143800 2023-11-08T16:41:17.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.374+0000 7ff75d7fa640 1 -- 172.21.15.81:0/3409774804 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74800ce40 con 0x7ff760143800 2023-11-08T16:41:17.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.374+0000 7ff7662d7640 1 -- 172.21.15.81:0/3409774804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff76006cf10 con 0x7ff760143800 2023-11-08T16:41:17.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.374+0000 7ff7662d7640 1 -- 172.21.15.81:0/3409774804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff76006d4d0 con 0x7ff760143800 2023-11-08T16:41:17.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.375+0000 7ff7662d7640 1 -- 172.21.15.81:0/3409774804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff760074440 con 0x7ff760143800 2023-11-08T16:41:17.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.376+0000 7ff75d7fa640 1 -- 172.21.15.81:0/3409774804 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff7480113f0 con 0x7ff760143800 2023-11-08T16:41:17.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.377+0000 7ff75d7fa640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74006ae70 0x7ff74006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:17.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.377+0000 7ff75d7fa640 1 -- 172.21.15.81:0/3409774804 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff748089850 con 0x7ff760143800 2023-11-08T16:41:17.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.377+0000 7ff75ffff640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74006ae70 0x7ff74006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:17.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.378+0000 7ff75ffff640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74006ae70 0x7ff74006d330 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff750036000 tx=0x7ff75003c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:17.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.382+0000 7ff75d7fa640 1 -- 172.21.15.81:0/3409774804 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff748054860 con 0x7ff760143800 2023-11-08T16:41:17.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.608+0000 7ff7662d7640 1 -- 172.21.15.81:0/3409774804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff76006fde0 con 0x7ff760143800 2023-11-08T16:41:17.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.610+0000 7ff75d7fa640 1 -- 172.21.15.81:0/3409774804 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) v1 ==== 95+0+4918 (secure 0 0 0) 0x7ff748057e80 con 0x7ff760143800 2023-11-08T16:41:17.612 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:17.612 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":107,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":8556,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/296230272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":296230272},{"type":"v1","addr":"172.21.15.81:6837","nonce":296230272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:08.773407+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8574},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8574":{"gid":8574,"name":"d","rank":0,"incarnation":104,"state":"up:active","state_seq":3,"addr":"172.21.15.93:6835/4021510722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":4021510722},{"type":"v1","addr":"172.21.15.93:6835","nonce":4021510722}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:41:17.612 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 107 2023-11-08T16:41:17.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 -- 172.21.15.81:0/3409774804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74006ae70 msgr2=0x7ff74006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:17.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74006ae70 0x7ff74006d330 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7ff750036000 tx=0x7ff75003c040 comp rx=0 tx=0).stop 2023-11-08T16:41:17.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 -- 172.21.15.81:0/3409774804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760143800 msgr2=0x7ff760138400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:17.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760143800 0x7ff760138400 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7ff74800c7f0 tx=0x7ff74800cbf0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 -- 172.21.15.81:0/3409774804 shutdown_connections 2023-11-08T16:41:17.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff74006ae70 0x7ff74006d330 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff76015df20 0x7ff7600702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff760143800 0x7ff760138400 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 --2- 172.21.15.81:0/3409774804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7600cca70 0x7ff7601599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:17.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 -- 172.21.15.81:0/3409774804 >> 172.21.15.81:0/3409774804 conn(0x7ff76006a6a0 msgr2=0x7ff7600b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:17.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 -- 172.21.15.81:0/3409774804 shutdown_connections 2023-11-08T16:41:17.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:17.613+0000 7ff7662d7640 1 -- 172.21.15.81:0/3409774804 wait complete. 2023-11-08T16:41:17.634 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json dirfrag split /subdir 0/0 1 2023-11-08T16:41:17.864 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-11-08T16:41:17.865 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:41:17.865 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:41:17.928 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:41:18.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.137+0000 7f16240b4640 1 Processor -- start 2023-11-08T16:41:18.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.137+0000 7f16240b4640 1 -- start start 2023-11-08T16:41:18.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.139+0000 7f16240b4640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f161c0b28b0 0x7f161c0cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.139+0000 7f16240b4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161c0ccfb0 con 0x7f161c143800 2023-11-08T16:41:18.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.139+0000 7f16240b4640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161c0cd1e0 con 0x7f161c15e0f0 2023-11-08T16:41:18.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.139+0000 7f16240b4640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161c0cd4c0 con 0x7f161c0b28b0 2023-11-08T16:41:18.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.139+0000 7f1621e50640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f161c143800 0x7f161c15d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39524/0 (socket says 172.21.15.81:39524) 2023-11-08T16:41:18.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.139+0000 7f1621e50640 1 -- 172.21.15.81:0/1571172917 learned_addr learned my addr 172.21.15.81:0/1571172917 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:18.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.139+0000 7f1622651640 1 --2- 172.21.15.81:0/1571172917 >> v2:172.21.15.93:3300/0 conn(0x7f161c0b28b0 0x7f161c0cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431195059 0 0) 0x7f161c0ccfb0 con 0x7f161c143800 2023-11-08T16:41:18.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1608003520 con 0x7f161c143800 2023-11-08T16:41:18.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3458172353 0 0) 0x7f1608003520 con 0x7f161c143800 2023-11-08T16:41:18.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 >> v2:172.21.15.93:3300/0 conn(0x7f161c0b28b0 msgr2=0x7f161c0cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:18.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 --2- 172.21.15.81:0/1571172917 >> v2:172.21.15.93:3300/0 conn(0x7f161c0b28b0 0x7f161c0cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 >> v1:172.21.15.93:6789/0 conn(0x7f161c15e0f0 legacy=0x7f161c0704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c0cd7a0 con 0x7f161c143800 2023-11-08T16:41:18.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f1618001670 con 0x7f161c143800 2023-11-08T16:41:18.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1622651640 1 --2- 172.21.15.81:0/1571172917 >> v2:172.21.15.93:3300/0 conn(0x7f161c0b28b0 0x7f161c0cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:41:18.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 >> v1:172.21.15.81:6789/0 conn(0x7f161c143800 legacy=0x7f161c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 --2- 172.21.15.81:0/1571172917 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1608003970 0x7f1608003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 --2- 172.21.15.81:0/1571172917 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16080047c0 0x7f1608004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 --2- 172.21.15.81:0/1571172917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1608005180 0x7f160800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c0cd4c0 con 0x7f1608005180 2023-11-08T16:41:18.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f160800c020 con 0x7f16080047c0 2023-11-08T16:41:18.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.140+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f160800e3f0 con 0x7f1608003970 2023-11-08T16:41:18.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.141+0000 7f1622651640 1 --2- 172.21.15.81:0/1571172917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1608005180 0x7f160800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.141+0000 7f162164f640 1 --2- 172.21.15.81:0/1571172917 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16080047c0 0x7f1608004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.141+0000 7f1621e50640 1 --2- 172.21.15.81:0/1571172917 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1608003970 0x7f1608003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.141+0000 7f1622651640 1 -- 172.21.15.81:0/1571172917 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1608003970 msgr2=0x7f1608003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:18.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.141+0000 7f1622651640 1 --2- 172.21.15.81:0/1571172917 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1608003970 0x7f1608003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.141+0000 7f1622651640 1 -- 172.21.15.81:0/1571172917 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16080047c0 msgr2=0x7f1608004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:18.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.141+0000 7f1622651640 1 --2- 172.21.15.81:0/1571172917 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16080047c0 0x7f1608004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.141+0000 7f1622651640 1 -- 172.21.15.81:0/1571172917 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1608003520 con 0x7f1608005180 2023-11-08T16:41:18.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.142+0000 7f1622651640 1 -- 172.21.15.81:0/1571172917 reap_dead start 2023-11-08T16:41:18.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.142+0000 7f1622651640 1 --2- 172.21.15.81:0/1571172917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1608005180 0x7f160800b980 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f161400e350 tx=0x7f16080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71302d40df171423 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:18.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.142+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161402ce00 con 0x7f1608005180 2023-11-08T16:41:18.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.142+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161400f040 con 0x7f1608005180 2023-11-08T16:41:18.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.142+0000 7f1620e4e640 1 -- 172.21.15.81:0/1571172917 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614003bd0 con 0x7f1608005180 2023-11-08T16:41:18.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.143+0000 7f16240b4640 1 -- 172.21.15.81:0/1571172917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1608005180 msgr2=0x7f160800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.143+0000 7f16240b4640 1 --2- 172.21.15.81:0/1571172917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1608005180 0x7f160800b980 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f161400e350 tx=0x7f16080057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.143+0000 7f16240b4640 1 -- 172.21.15.81:0/1571172917 shutdown_connections 2023-11-08T16:41:18.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.143+0000 7f16240b4640 1 --2- 172.21.15.81:0/1571172917 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1608005180 0x7f160800b980 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.143+0000 7f16240b4640 1 -- 172.21.15.81:0/1571172917 >> 172.21.15.81:0/1571172917 conn(0x7f161c06a6a0 msgr2=0x7f161c0b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:18.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.143+0000 7f16240b4640 1 -- 172.21.15.81:0/1571172917 shutdown_connections 2023-11-08T16:41:18.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.143+0000 7f16240b4640 1 -- 172.21.15.81:0/1571172917 wait complete. 2023-11-08T16:41:18.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.145+0000 7f16240b4640 1 Processor -- start 2023-11-08T16:41:18.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.145+0000 7f16240b4640 1 -- start start 2023-11-08T16:41:18.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.146+0000 7f16240b4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1608005180 0x7f161c1f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.146+0000 7f16240b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f161c06ba10 0x7f161c0c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.146+0000 7f16240b4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f161c13d510 0x7f161c13ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.146+0000 7f16240b4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c06cac0 con 0x7f161c06ba10 2023-11-08T16:41:18.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.146+0000 7f16240b4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f161c06c220 con 0x7f161c13d510 2023-11-08T16:41:18.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.146+0000 7f16240b4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f161c06c500 con 0x7f1608005180 2023-11-08T16:41:18.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.146+0000 7f162164f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f161c06ba10 0x7f161c0c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.146+0000 7f162164f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f161c06ba10 0x7f161c0c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37038/0 (socket says 172.21.15.81:37038) 2023-11-08T16:41:18.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.146+0000 7f162164f640 1 -- 172.21.15.81:0/753822621 learned_addr learned my addr 172.21.15.81:0/753822621 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:18.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f1621e50640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1608005180 0x7f161c1f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f1622651640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f161c13d510 0x7f161c13ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f162164f640 1 -- 172.21.15.81:0/753822621 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1608005180 msgr2=0x7f161c1f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f162164f640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1608005180 0x7f161c1f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f162164f640 1 -- 172.21.15.81:0/753822621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f161c13d510 msgr2=0x7f161c13ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f162164f640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f161c13d510 0x7f161c13ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f162164f640 1 -- 172.21.15.81:0/753822621 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c06c7e0 con 0x7f161c06ba10 2023-11-08T16:41:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f1621e50640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1608005180 0x7f161c1f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f1622651640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f161c13d510 0x7f161c13ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f162164f640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f161c06ba10 0x7f161c0c4e50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f160c00e4d0 tx=0x7f160c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:18.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f16127fc640 1 -- 172.21.15.81:0/753822621 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c0052e0 con 0x7f161c06ba10 2023-11-08T16:41:18.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f16127fc640 1 -- 172.21.15.81:0/753822621 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f160c005440 con 0x7f161c06ba10 2023-11-08T16:41:18.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f16127fc640 1 -- 172.21.15.81:0/753822621 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f160c003810 con 0x7f161c06ba10 2023-11-08T16:41:18.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.147+0000 7f16240b4640 1 -- 172.21.15.81:0/753822621 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f161c0ac9b0 con 0x7f161c06ba10 2023-11-08T16:41:18.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.148+0000 7f16240b4640 1 -- 172.21.15.81:0/753822621 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f161c06be90 con 0x7f161c06ba10 2023-11-08T16:41:18.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.149+0000 7f16240b4640 1 -- 172.21.15.81:0/753822621 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f161c0ac810 con 0x7f161c06ba10 2023-11-08T16:41:18.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.150+0000 7f16127fc640 1 -- 172.21.15.81:0/753822621 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f160c003970 con 0x7f161c06ba10 2023-11-08T16:41:18.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.150+0000 7f16127fc640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f160006aea0 0x7f160006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.150+0000 7f16127fc640 1 -- 172.21.15.81:0/753822621 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f160c0893c0 con 0x7f161c06ba10 2023-11-08T16:41:18.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.151+0000 7f1621e50640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f160006aea0 0x7f160006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.151+0000 7f1621e50640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f160006aea0 0x7f160006d360 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f16180018e0 tx=0x7f1618009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:18.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.155+0000 7f16127fc640 1 -- 172.21.15.81:0/753822621 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f161c0ac810 con 0x7f161c06ba10 2023-11-08T16:41:18.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.348+0000 7f16240b4640 1 -- 172.21.15.81:0/753822621 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f161c0ac810 con 0x7f161c06ba10 2023-11-08T16:41:18.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.351+0000 7f16127fc640 1 -- 172.21.15.81:0/753822621 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16467 (secure 0 0 0) 0x7f161c0ac810 con 0x7f161c06ba10 2023-11-08T16:41:18.353 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:18.354 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":127,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:07.268068+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:41:03.545565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T16:41:04.726492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:41:18.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 -- 172.21.15.81:0/753822621 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f160006aea0 msgr2=0x7f160006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f160006aea0 0x7f160006d360 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f16180018e0 tx=0x7f1618009040 comp rx=0 tx=0).stop 2023-11-08T16:41:18.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 -- 172.21.15.81:0/753822621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f161c06ba10 msgr2=0x7f161c0c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f161c06ba10 0x7f161c0c4e50 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f160c00e4d0 tx=0x7f160c00e990 comp rx=0 tx=0).stop 2023-11-08T16:41:18.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 -- 172.21.15.81:0/753822621 shutdown_connections 2023-11-08T16:41:18.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f160006aea0 0x7f160006d360 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f161c13d510 0x7f161c13ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f161c06ba10 0x7f161c0c4e50 secure :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f160c00e4d0 tx=0x7f160c00e990 comp rx=0 tx=0).stop 2023-11-08T16:41:18.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 --2- 172.21.15.81:0/753822621 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1608005180 0x7f161c1f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 -- 172.21.15.81:0/753822621 >> 172.21.15.81:0/753822621 conn(0x7f161c06a6a0 msgr2=0x7f161c15cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:18.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 -- 172.21.15.81:0/753822621 shutdown_connections 2023-11-08T16:41:18.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.354+0000 7f16240b4640 1 -- 172.21.15.81:0/753822621 wait complete. 2023-11-08T16:41:18.374 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:41:03.545565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:41:04.726492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:41:18.375 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:41:18.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.588+0000 7fd0d94ec640 1 Processor -- start 2023-11-08T16:41:18.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.588+0000 7fd0d94ec640 1 -- start start 2023-11-08T16:41:18.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.590+0000 7fd0d94ec640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd0d4071ad0 0x7fd0d415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.590+0000 7fd0d94ec640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d415fad0 con 0x7fd0d4158c60 2023-11-08T16:41:18.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.590+0000 7fd0d94ec640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0d406d760 con 0x7fd0d415f1b0 2023-11-08T16:41:18.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.590+0000 7fd0d94ec640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0d406df80 con 0x7fd0d4071ad0 2023-11-08T16:41:18.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.590+0000 7fd0d27fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd0d4071ad0 0x7fd0d415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.590+0000 7fd0d27fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd0d4071ad0 0x7fd0d415ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37044/0 (socket says 172.21.15.81:37044) 2023-11-08T16:41:18.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.590+0000 7fd0d2ffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd0d4158c60 0x7fd0d40705d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39536/0 (socket says 172.21.15.81:39536) 2023-11-08T16:41:18.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.590+0000 7fd0d27fc640 1 -- 172.21.15.81:0/4121094901 learned_addr learned my addr 172.21.15.81:0/4121094901 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:18.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.591+0000 7fd0d27fc640 1 -- 172.21.15.81:0/4121094901 >> v1:172.21.15.93:6789/0 conn(0x7fd0d415f1b0 legacy=0x7fd0d40c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:18.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.591+0000 7fd0d27fc640 1 -- 172.21.15.81:0/4121094901 >> v1:172.21.15.81:6789/0 conn(0x7fd0d4158c60 legacy=0x7fd0d40705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.591+0000 7fd0d27fc640 1 -- 172.21.15.81:0/4121094901 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d406da00 con 0x7fd0d4071ad0 2023-11-08T16:41:18.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.592+0000 7fd0d27fc640 1 --2- 172.21.15.81:0/4121094901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d4071ad0 0x7fd0d415ec70 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009870 tx=0x7fd0c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980da2ceee436459 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:18.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.592+0000 7fd0d1ffb640 1 -- 172.21.15.81:0/4121094901 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8031870 con 0x7fd0d4071ad0 2023-11-08T16:41:18.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.592+0000 7fd0d1ffb640 1 -- 172.21.15.81:0/4121094901 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c80319d0 con 0x7fd0d4071ad0 2023-11-08T16:41:18.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.592+0000 7fd0d1ffb640 1 -- 172.21.15.81:0/4121094901 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8031d60 con 0x7fd0d4071ad0 2023-11-08T16:41:18.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.592+0000 7fd0d94ec640 1 -- 172.21.15.81:0/4121094901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d4071ad0 msgr2=0x7fd0d415ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.592+0000 7fd0d94ec640 1 --2- 172.21.15.81:0/4121094901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d4071ad0 0x7fd0d415ec70 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009870 tx=0x7fd0c802b910 comp rx=0 tx=0).stop 2023-11-08T16:41:18.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.593+0000 7fd0d94ec640 1 -- 172.21.15.81:0/4121094901 shutdown_connections 2023-11-08T16:41:18.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.593+0000 7fd0d94ec640 1 --2- 172.21.15.81:0/4121094901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d4071ad0 0x7fd0d415ec70 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.593+0000 7fd0d94ec640 1 -- 172.21.15.81:0/4121094901 >> 172.21.15.81:0/4121094901 conn(0x7fd0d40ceb70 msgr2=0x7fd0d40cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:18.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.593+0000 7fd0d94ec640 1 -- 172.21.15.81:0/4121094901 shutdown_connections 2023-11-08T16:41:18.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.593+0000 7fd0d94ec640 1 -- 172.21.15.81:0/4121094901 wait complete. 2023-11-08T16:41:18.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.594+0000 7fd0d94ec640 1 Processor -- start 2023-11-08T16:41:18.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.595+0000 7fd0d94ec640 1 -- start start 2023-11-08T16:41:18.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.595+0000 7fd0d94ec640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0d4071ad0 0x7fd0d4150740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.595+0000 7fd0d94ec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d4158c60 0x7fd0d41f0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.595+0000 7fd0d94ec640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0d415f1b0 0x7fd0d415d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.595+0000 7fd0d94ec640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d4151f50 con 0x7fd0d4158c60 2023-11-08T16:41:18.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.595+0000 7fd0d94ec640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d4153a40 con 0x7fd0d415f1b0 2023-11-08T16:41:18.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.595+0000 7fd0d94ec640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0d4150c80 con 0x7fd0d4071ad0 2023-11-08T16:41:18.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.596+0000 7fd0d27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d4158c60 0x7fd0d41f0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.596+0000 7fd0d27fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d4158c60 0x7fd0d41f0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37052/0 (socket says 172.21.15.81:37052) 2023-11-08T16:41:18.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.596+0000 7fd0d27fc640 1 -- 172.21.15.81:0/905316804 learned_addr learned my addr 172.21.15.81:0/905316804 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:18.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.596+0000 7fd0d2ffd640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0d4071ad0 0x7fd0d4150740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.596+0000 7fd0d37fe640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0d415f1b0 0x7fd0d415d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.596+0000 7fd0d27fc640 1 -- 172.21.15.81:0/905316804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0d4071ad0 msgr2=0x7fd0d4150740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.596+0000 7fd0d27fc640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0d4071ad0 0x7fd0d4150740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.596+0000 7fd0d27fc640 1 -- 172.21.15.81:0/905316804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0d415f1b0 msgr2=0x7fd0d415d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.596+0000 7fd0d27fc640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0d415f1b0 0x7fd0d415d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.596+0000 7fd0d27fc640 1 -- 172.21.15.81:0/905316804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c80094d0 con 0x7fd0d4158c60 2023-11-08T16:41:18.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.596+0000 7fd0d37fe640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0d415f1b0 0x7fd0d415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:18.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.597+0000 7fd0d27fc640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d4158c60 0x7fd0d41f0450 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8000c00 tx=0x7fd0c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:18.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.597+0000 7fd0bb7fe640 1 -- 172.21.15.81:0/905316804 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c8031820 con 0x7fd0d4158c60 2023-11-08T16:41:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.597+0000 7fd0bb7fe640 1 -- 172.21.15.81:0/905316804 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0c80443f0 con 0x7fd0d4158c60 2023-11-08T16:41:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.597+0000 7fd0bb7fe640 1 -- 172.21.15.81:0/905316804 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0c80447a0 con 0x7fd0d4158c60 2023-11-08T16:41:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.597+0000 7fd0d94ec640 1 -- 172.21.15.81:0/905316804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0d41f0990 con 0x7fd0d4158c60 2023-11-08T16:41:18.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.597+0000 7fd0d94ec640 1 -- 172.21.15.81:0/905316804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0d4151190 con 0x7fd0d4158c60 2023-11-08T16:41:18.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.598+0000 7fd0d94ec640 1 -- 172.21.15.81:0/905316804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0d4074440 con 0x7fd0d4158c60 2023-11-08T16:41:18.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.599+0000 7fd0bb7fe640 1 -- 172.21.15.81:0/905316804 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd0c8044900 con 0x7fd0d4158c60 2023-11-08T16:41:18.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.601+0000 7fd0bb7fe640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0c006ae70 0x7fd0c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:18.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.601+0000 7fd0bb7fe640 1 -- 172.21.15.81:0/905316804 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0c802d040 con 0x7fd0d4158c60 2023-11-08T16:41:18.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.601+0000 7fd0d2ffd640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0c006ae70 0x7fd0c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:18.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.602+0000 7fd0d2ffd640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0c006ae70 0x7fd0c006d330 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4007fd0 tx=0x7fd0c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:18.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.605+0000 7fd0bb7fe640 1 -- 172.21.15.81:0/905316804 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd0c8076f10 con 0x7fd0d4158c60 2023-11-08T16:41:18.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.793+0000 7fd0d94ec640 1 -- 172.21.15.81:0/905316804 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd0d406bd00 con 0x7fd0d4158c60 2023-11-08T16:41:18.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.794+0000 7fd0bb7fe640 1 -- 172.21.15.81:0/905316804 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd0c807a530 con 0x7fd0d4158c60 2023-11-08T16:41:18.796 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:41:18.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 -- 172.21.15.81:0/905316804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0c006ae70 msgr2=0x7fd0c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0c006ae70 0x7fd0c006d330 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4007fd0 tx=0x7fd0c4009040 comp rx=0 tx=0).stop 2023-11-08T16:41:18.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 -- 172.21.15.81:0/905316804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d4158c60 msgr2=0x7fd0d41f0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:18.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d4158c60 0x7fd0d41f0450 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8000c00 tx=0x7fd0c80046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 -- 172.21.15.81:0/905316804 shutdown_connections 2023-11-08T16:41:18.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd0c006ae70 0x7fd0c006d330 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0d415f1b0 0x7fd0d415d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0d4158c60 0x7fd0d41f0450 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 --2- 172.21.15.81:0/905316804 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0d4071ad0 0x7fd0d4150740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:18.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 -- 172.21.15.81:0/905316804 >> 172.21.15.81:0/905316804 conn(0x7fd0d40ceb70 msgr2=0x7fd0d4070d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:18.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 -- 172.21.15.81:0/905316804 shutdown_connections 2023-11-08T16:41:18.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:18.797+0000 7fd0d94ec640 1 -- 172.21.15.81:0/905316804 wait complete. 2023-11-08T16:41:18.820 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:41:19.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.039+0000 7f033c207640 1 Processor -- start 2023-11-08T16:41:19.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.039+0000 7f033c207640 1 -- start start 2023-11-08T16:41:19.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f033c207640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03341579e0 0x7f03340c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f033c207640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f033415a460 0x7f033406d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f033c207640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f033406dae0 0x7f033415da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f033c207640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033406df50 con 0x7f033415a460 2023-11-08T16:41:19.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f033c207640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0334070640 con 0x7f033406dae0 2023-11-08T16:41:19.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f033c207640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033415b400 con 0x7f03341579e0 2023-11-08T16:41:19.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f03397a2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f033415a460 0x7f033406d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f03397a2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f033415a460 0x7f033406d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37066/0 (socket says 172.21.15.81:37066) 2023-11-08T16:41:19.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f03397a2640 1 -- 172.21.15.81:0/4287799853 learned_addr learned my addr 172.21.15.81:0/4287799853 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:19.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f0339fa3640 1 --2- 172.21.15.81:0/4287799853 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03341579e0 0x7f03340c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f033a7a4640 1 --2- 172.21.15.81:0/4287799853 >> v2:172.21.15.93:3300/0 conn(0x7f033406dae0 0x7f033415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f03397a2640 1 -- 172.21.15.81:0/4287799853 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03341579e0 msgr2=0x7f03340c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:19.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f03397a2640 1 --2- 172.21.15.81:0/4287799853 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03341579e0 0x7f03340c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f03397a2640 1 -- 172.21.15.81:0/4287799853 >> v2:172.21.15.93:3300/0 conn(0x7f033406dae0 msgr2=0x7f033415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:19.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f03397a2640 1 --2- 172.21.15.81:0/4287799853 >> v2:172.21.15.93:3300/0 conn(0x7f033406dae0 0x7f033415da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.041+0000 7f03397a2640 1 -- 172.21.15.81:0/4287799853 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033406c2b0 con 0x7f033415a460 2023-11-08T16:41:19.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.042+0000 7f033a7a4640 1 --2- 172.21.15.81:0/4287799853 >> v2:172.21.15.93:3300/0 conn(0x7f033406dae0 0x7f033415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:19.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.043+0000 7f03397a2640 1 --2- 172.21.15.81:0/4287799853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f033415a460 0x7f033406d5a0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f033406f6a0 tx=0x7f032402e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c8ee8d7629be7c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:19.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.043+0000 7f0338fa1640 1 -- 172.21.15.81:0/4287799853 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032402ee80 con 0x7f033415a460 2023-11-08T16:41:19.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.043+0000 7f0338fa1640 1 -- 172.21.15.81:0/4287799853 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0324036440 con 0x7f033415a460 2023-11-08T16:41:19.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.043+0000 7f0338fa1640 1 -- 172.21.15.81:0/4287799853 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03240367b0 con 0x7f033415a460 2023-11-08T16:41:19.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.044+0000 7f033c207640 1 -- 172.21.15.81:0/4287799853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f033415a460 msgr2=0x7f033406d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.044+0000 7f033c207640 1 --2- 172.21.15.81:0/4287799853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f033415a460 0x7f033406d5a0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f033406f6a0 tx=0x7f032402e0d0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.044+0000 7f033c207640 1 -- 172.21.15.81:0/4287799853 shutdown_connections 2023-11-08T16:41:19.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.044+0000 7f033c207640 1 --2- 172.21.15.81:0/4287799853 >> v2:172.21.15.93:3300/0 conn(0x7f033406dae0 0x7f033415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.044+0000 7f033c207640 1 --2- 172.21.15.81:0/4287799853 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f033415a460 0x7f033406d5a0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.044+0000 7f033c207640 1 --2- 172.21.15.81:0/4287799853 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f03341579e0 0x7f03340c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.044+0000 7f033c207640 1 -- 172.21.15.81:0/4287799853 >> 172.21.15.81:0/4287799853 conn(0x7f03340ceb70 msgr2=0x7f03340cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:19.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.044+0000 7f033c207640 1 -- 172.21.15.81:0/4287799853 shutdown_connections 2023-11-08T16:41:19.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.044+0000 7f033c207640 1 -- 172.21.15.81:0/4287799853 wait complete. 2023-11-08T16:41:19.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.046+0000 7f033c207640 1 Processor -- start 2023-11-08T16:41:19.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.046+0000 7f033c207640 1 -- start start 2023-11-08T16:41:19.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.047+0000 7f033c207640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f033406dae0 0x7f033414ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.047+0000 7f033c207640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03341579e0 0x7f03340b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.047+0000 7f033c207640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f033415a460 0x7f03340cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.047+0000 7f033c207640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03340b3020 con 0x7f03341579e0 2023-11-08T16:41:19.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.047+0000 7f033c207640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03340c33b0 con 0x7f033406dae0 2023-11-08T16:41:19.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.047+0000 7f033c207640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03340b2410 con 0x7f033415a460 2023-11-08T16:41:19.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.047+0000 7f03397a2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03341579e0 0x7f03340b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.047+0000 7f03397a2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03341579e0 0x7f03340b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37078/0 (socket says 172.21.15.81:37078) 2023-11-08T16:41:19.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.047+0000 7f03397a2640 1 -- 172.21.15.81:0/2361805269 learned_addr learned my addr 172.21.15.81:0/2361805269 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:19.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.048+0000 7f033a7a4640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f033415a460 0x7f03340cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.048+0000 7f0339fa3640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f033406dae0 0x7f033414ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.048+0000 7f03397a2640 1 -- 172.21.15.81:0/2361805269 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f033415a460 msgr2=0x7f03340cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.048+0000 7f03397a2640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f033415a460 0x7f03340cd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.048+0000 7f03397a2640 1 -- 172.21.15.81:0/2361805269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f033406dae0 msgr2=0x7f033414ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.048+0000 7f03397a2640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f033406dae0 0x7f033414ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.048+0000 7f03397a2640 1 -- 172.21.15.81:0/2361805269 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032400b770 con 0x7f03341579e0 2023-11-08T16:41:19.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.048+0000 7f03397a2640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03341579e0 0x7f03340b19c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f032402e110 tx=0x7f03240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:19.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.049+0000 7f032a7fc640 1 -- 172.21.15.81:0/2361805269 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324034b80 con 0x7f03341579e0 2023-11-08T16:41:19.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.049+0000 7f032a7fc640 1 -- 172.21.15.81:0/2361805269 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0324036e90 con 0x7f03341579e0 2023-11-08T16:41:19.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.049+0000 7f032a7fc640 1 -- 172.21.15.81:0/2361805269 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324041440 con 0x7f03341579e0 2023-11-08T16:41:19.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.049+0000 7f033c207640 1 -- 172.21.15.81:0/2361805269 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03340b26f0 con 0x7f03341579e0 2023-11-08T16:41:19.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.049+0000 7f033c207640 1 -- 172.21.15.81:0/2361805269 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03340b2cb0 con 0x7f03341579e0 2023-11-08T16:41:19.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.050+0000 7f033c207640 1 -- 172.21.15.81:0/2361805269 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03340c3540 con 0x7f03341579e0 2023-11-08T16:41:19.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.051+0000 7f032a7fc640 1 -- 172.21.15.81:0/2361805269 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f032403d040 con 0x7f03341579e0 2023-11-08T16:41:19.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.052+0000 7f032a7fc640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031806ae70 0x7f031806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.052+0000 7f032a7fc640 1 -- 172.21.15.81:0/2361805269 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03240ae4a0 con 0x7f03341579e0 2023-11-08T16:41:19.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.052+0000 7f0339fa3640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031806ae70 0x7f031806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.053+0000 7f0339fa3640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031806ae70 0x7f031806d330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f033000b7e0 tx=0x7f0330008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:19.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.056+0000 7f032a7fc640 1 -- 172.21.15.81:0/2361805269 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f03340c3540 con 0x7f03341579e0 2023-11-08T16:41:19.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.244+0000 7f033c207640 1 -- 172.21.15.81:0/2361805269 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03340c3540 con 0x7f03341579e0 2023-11-08T16:41:19.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.245+0000 7f032a7fc640 1 -- 172.21.15.81:0/2361805269 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03340c3540 con 0x7f03341579e0 2023-11-08T16:41:19.247 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:41:19.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.247+0000 7f033c207640 1 -- 172.21.15.81:0/2361805269 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031806ae70 msgr2=0x7f031806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.247+0000 7f033c207640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031806ae70 0x7f031806d330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f033000b7e0 tx=0x7f0330008040 comp rx=0 tx=0).stop 2023-11-08T16:41:19.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.247+0000 7f033c207640 1 -- 172.21.15.81:0/2361805269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03341579e0 msgr2=0x7f03340b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.247+0000 7f033c207640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03341579e0 0x7f03340b19c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f032402e110 tx=0x7f03240046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.247+0000 7f033c207640 1 -- 172.21.15.81:0/2361805269 shutdown_connections 2023-11-08T16:41:19.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.247+0000 7f033c207640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f031806ae70 0x7f031806d330 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.247+0000 7f033c207640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f033415a460 0x7f03340cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.247+0000 7f033c207640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f03341579e0 0x7f03340b19c0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.247+0000 7f033c207640 1 --2- 172.21.15.81:0/2361805269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f033406dae0 0x7f033414ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.247+0000 7f033c207640 1 -- 172.21.15.81:0/2361805269 >> 172.21.15.81:0/2361805269 conn(0x7f03340ceb70 msgr2=0x7f033415d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:19.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.247+0000 7f033c207640 1 -- 172.21.15.81:0/2361805269 shutdown_connections 2023-11-08T16:41:19.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.248+0000 7f033c207640 1 -- 172.21.15.81:0/2361805269 wait complete. 2023-11-08T16:41:19.268 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:41:19.478 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:41:19.484 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:41:19.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.480+0000 7fcec2f55640 1 Processor -- start 2023-11-08T16:41:19.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.480+0000 7fcec2f55640 1 -- start start 2023-11-08T16:41:19.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.482+0000 7fcec2f55640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcebc070310 0x7fcebc06b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.482+0000 7fcec2f55640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcebc0725c0 con 0x7fcebc0b2950 2023-11-08T16:41:19.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.482+0000 7fcec2f55640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcebc0722e0 con 0x7fcebc15e510 2023-11-08T16:41:19.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.482+0000 7fcec2f55640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcebc06abb0 con 0x7fcebc070310 2023-11-08T16:41:19.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.482+0000 7fcebbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcebc070310 0x7fcebc06b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.482+0000 7fcebbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcebc070310 0x7fcebc06b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37088/0 (socket says 172.21.15.81:37088) 2023-11-08T16:41:19.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.482+0000 7fcebbfff640 1 -- 172.21.15.81:0/2903881798 learned_addr learned my addr 172.21.15.81:0/2903881798 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:19.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.483+0000 7fcebbfff640 1 -- 172.21.15.81:0/2903881798 >> v1:172.21.15.93:6789/0 conn(0x7fcebc15e510 legacy=0x7fcebc0b2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:19.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.483+0000 7fcebbfff640 1 -- 172.21.15.81:0/2903881798 >> v1:172.21.15.81:6789/0 conn(0x7fcebc0b2950 legacy=0x7fcebc0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:19.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.483+0000 7fcebbfff640 1 -- 172.21.15.81:0/2903881798 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc06ae90 con 0x7fcebc070310 2023-11-08T16:41:19.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.484+0000 7fcebbfff640 1 --2- 172.21.15.81:0/2903881798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc070310 0x7fcebc06b690 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009870 tx=0x7fcea802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6aa583dd87cb9a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:19.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.484+0000 7fcebb7fe640 1 -- 172.21.15.81:0/2903881798 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8031870 con 0x7fcebc070310 2023-11-08T16:41:19.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.484+0000 7fcebb7fe640 1 -- 172.21.15.81:0/2903881798 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea80319d0 con 0x7fcebc070310 2023-11-08T16:41:19.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.484+0000 7fcebb7fe640 1 -- 172.21.15.81:0/2903881798 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8031d60 con 0x7fcebc070310 2023-11-08T16:41:19.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.485+0000 7fcec2f55640 1 -- 172.21.15.81:0/2903881798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc070310 msgr2=0x7fcebc06b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.485+0000 7fcec2f55640 1 --2- 172.21.15.81:0/2903881798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc070310 0x7fcebc06b690 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fcea8009870 tx=0x7fcea802b910 comp rx=0 tx=0).stop 2023-11-08T16:41:19.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.485+0000 7fcec2f55640 1 -- 172.21.15.81:0/2903881798 shutdown_connections 2023-11-08T16:41:19.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.485+0000 7fcec2f55640 1 --2- 172.21.15.81:0/2903881798 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc070310 0x7fcebc06b690 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.485+0000 7fcec2f55640 1 -- 172.21.15.81:0/2903881798 >> 172.21.15.81:0/2903881798 conn(0x7fcebc0cf3f0 msgr2=0x7fcebc0cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:19.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.485+0000 7fcec2f55640 1 -- 172.21.15.81:0/2903881798 shutdown_connections 2023-11-08T16:41:19.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.485+0000 7fcec2f55640 1 -- 172.21.15.81:0/2903881798 wait complete. 2023-11-08T16:41:19.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.487+0000 7fcec2f55640 1 Processor -- start 2023-11-08T16:41:19.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.487+0000 7fcec2f55640 1 -- start start 2023-11-08T16:41:19.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.488+0000 7fcec2f55640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcebc070310 0x7fcebc0aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.488+0000 7fcec2f55640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0b2950 0x7fcebc14f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.488+0000 7fcec2f55640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcebc15e510 0x7fcebc0ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.488+0000 7fcec2f55640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc0aa6e0 con 0x7fcebc0b2950 2023-11-08T16:41:19.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.488+0000 7fcec2f55640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc0a9d10 con 0x7fcebc15e510 2023-11-08T16:41:19.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.488+0000 7fcec2f55640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc0a9100 con 0x7fcebc070310 2023-11-08T16:41:19.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.489+0000 7fcebbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0b2950 0x7fcebc14f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.489+0000 7fcebbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0b2950 0x7fcebc14f990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37092/0 (socket says 172.21.15.81:37092) 2023-11-08T16:41:19.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.489+0000 7fcebbfff640 1 -- 172.21.15.81:0/2802133612 learned_addr learned my addr 172.21.15.81:0/2802133612 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:19.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.489+0000 7fcec0cf1640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcebc070310 0x7fcebc0aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.489+0000 7fcebbfff640 1 -- 172.21.15.81:0/2802133612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcebc070310 msgr2=0x7fcebc0aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.489+0000 7fcebbfff640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcebc070310 0x7fcebc0aa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.489+0000 7fcebbfff640 1 -- 172.21.15.81:0/2802133612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcebc15e510 msgr2=0x7fcebc0ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.489+0000 7fcebbfff640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcebc15e510 0x7fcebc0ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.489+0000 7fcebbfff640 1 -- 172.21.15.81:0/2802133612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcea80094d0 con 0x7fcebc0b2950 2023-11-08T16:41:19.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.489+0000 7fcec0cf1640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcebc070310 0x7fcebc0aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:19.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.489+0000 7fcebbfff640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0b2950 0x7fcebc14f990 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fcea8000c00 tx=0x7fcea80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:19.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.490+0000 7fceb97fa640 1 -- 172.21.15.81:0/2802133612 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8031820 con 0x7fcebc0b2950 2023-11-08T16:41:19.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.490+0000 7fceb97fa640 1 -- 172.21.15.81:0/2802133612 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcea80443f0 con 0x7fcebc0b2950 2023-11-08T16:41:19.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.490+0000 7fceb97fa640 1 -- 172.21.15.81:0/2802133612 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea80447a0 con 0x7fcebc0b2950 2023-11-08T16:41:19.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.490+0000 7fcec2f55640 1 -- 172.21.15.81:0/2802133612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcebc0a93e0 con 0x7fcebc0b2950 2023-11-08T16:41:19.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.490+0000 7fcec2f55640 1 -- 172.21.15.81:0/2802133612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcebc0a99a0 con 0x7fcebc0b2950 2023-11-08T16:41:19.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.491+0000 7fcec2f55640 1 -- 172.21.15.81:0/2802133612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcebc074440 con 0x7fcebc0b2950 2023-11-08T16:41:19.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.492+0000 7fceb97fa640 1 -- 172.21.15.81:0/2802133612 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcea8044900 con 0x7fcebc0b2950 2023-11-08T16:41:19.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.493+0000 7fceb97fa640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcea006ae70 0x7fcea006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.493+0000 7fceb97fa640 1 -- 172.21.15.81:0/2802133612 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcea802d040 con 0x7fcebc0b2950 2023-11-08T16:41:19.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.493+0000 7fcec0cf1640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcea006ae70 0x7fcea006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.494+0000 7fcec0cf1640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcea006ae70 0x7fcea006d330 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fceb000a830 tx=0x7fceb0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:19.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.497+0000 7fceb97fa640 1 -- 172.21.15.81:0/2802133612 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcea8076ed0 con 0x7fcebc0b2950 2023-11-08T16:41:19.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.688+0000 7fcec2f55640 1 -- 172.21.15.81:0/2802133612 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcebc074440 con 0x7fcebc0b2950 2023-11-08T16:41:19.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.689+0000 7fceb97fa640 1 -- 172.21.15.81:0/2802133612 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcea807a4f0 con 0x7fcebc0b2950 2023-11-08T16:41:19.691 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:41:19.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.691+0000 7fcec2f55640 1 -- 172.21.15.81:0/2802133612 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcea006ae70 msgr2=0x7fcea006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.691+0000 7fcec2f55640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcea006ae70 0x7fcea006d330 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fceb000a830 tx=0x7fceb0008040 comp rx=0 tx=0).stop 2023-11-08T16:41:19.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.691+0000 7fcec2f55640 1 -- 172.21.15.81:0/2802133612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0b2950 msgr2=0x7fcebc14f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.691+0000 7fcec2f55640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0b2950 0x7fcebc14f990 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fcea8000c00 tx=0x7fcea80046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.691+0000 7fcec2f55640 1 -- 172.21.15.81:0/2802133612 shutdown_connections 2023-11-08T16:41:19.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.691+0000 7fcec2f55640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcea006ae70 0x7fcea006d330 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.691+0000 7fcec2f55640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcebc15e510 0x7fcebc0ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.691+0000 7fcec2f55640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcebc0b2950 0x7fcebc14f990 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.691+0000 7fcec2f55640 1 --2- 172.21.15.81:0/2802133612 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcebc070310 0x7fcebc0aa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.691+0000 7fcec2f55640 1 -- 172.21.15.81:0/2802133612 >> 172.21.15.81:0/2802133612 conn(0x7fcebc0cf3f0 msgr2=0x7fcebc0b2d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:19.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.691+0000 7fcec2f55640 1 -- 172.21.15.81:0/2802133612 shutdown_connections 2023-11-08T16:41:19.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.692+0000 7fcec2f55640 1 -- 172.21.15.81:0/2802133612 wait complete. 2023-11-08T16:41:19.712 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:41:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.976+0000 7f13729c9640 1 Processor -- start 2023-11-08T16:41:19.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.977+0000 7f13729c9640 1 -- start start 2023-11-08T16:41:19.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.978+0000 7f13729c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f136c158aa0 0x7f136c06fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.978+0000 7f13729c9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f136c0cd3c0 0x7f136c0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.978+0000 7f13729c9640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f136c071620 con 0x7f136c0cbb10 2023-11-08T16:41:19.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.978+0000 7f13729c9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f136c0cbfa0 con 0x7f136c0cd3c0 2023-11-08T16:41:19.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.978+0000 7f13729c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c158e80 con 0x7f136c158aa0 2023-11-08T16:41:19.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.978+0000 7f1370f66640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f136c0cbb10 0x7f136c06bc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39548/0 (socket says 172.21.15.81:39548) 2023-11-08T16:41:19.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.978+0000 7f1370f66640 1 -- 172.21.15.81:0/3326511337 learned_addr learned my addr 172.21.15.81:0/3326511337 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:19.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136bfff640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f136c158aa0 0x7f136c06fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f1363fff640 1 --2- 172.21.15.81:0/3326511337 >> v2:172.21.15.93:3300/0 conn(0x7f136c0cd3c0 0x7f136c0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3536502673 0 0) 0x7f136c071620 con 0x7f136c0cbb10 2023-11-08T16:41:19.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1358003520 con 0x7f136c0cbb10 2023-11-08T16:41:19.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3892411857 0 0) 0x7f1358003520 con 0x7f136c0cbb10 2023-11-08T16:41:19.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f136c158aa0 msgr2=0x7f136c06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:19.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f136c158aa0 0x7f136c06fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 >> v2:172.21.15.93:3300/0 conn(0x7f136c0cd3c0 msgr2=0x7f136c0cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:19.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 --2- 172.21.15.81:0/3326511337 >> v2:172.21.15.93:3300/0 conn(0x7f136c0cd3c0 0x7f136c0cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c071850 con 0x7f136c0cbb10 2023-11-08T16:41:19.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f13640019f0 con 0x7f136c0cbb10 2023-11-08T16:41:19.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 >> v1:172.21.15.81:6789/0 conn(0x7f136c0cbb10 legacy=0x7f136c06bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f136c158aa0 0x7f1358003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1358003ef0 0x7f13580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1358004d00 0x7f1358005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0cbfa0 con 0x7f1358004d00 2023-11-08T16:41:19.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c158e80 con 0x7f1358003ef0 2023-11-08T16:41:19.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.979+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f135800b980 con 0x7f136c158aa0 2023-11-08T16:41:19.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.980+0000 7f1370f66640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1358004d00 0x7f1358005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.980+0000 7f136bfff640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f136c158aa0 0x7f1358003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.980+0000 7f1363fff640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1358003ef0 0x7f13580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.980+0000 7f1370f66640 1 -- 172.21.15.81:0/3326511337 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f136c158aa0 msgr2=0x7f1358003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:19.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.980+0000 7f1370f66640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f136c158aa0 0x7f1358003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.980+0000 7f1370f66640 1 -- 172.21.15.81:0/3326511337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1358003ef0 msgr2=0x7f13580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:19.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.980+0000 7f1370f66640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1358003ef0 0x7f13580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.980+0000 7f1370f66640 1 -- 172.21.15.81:0/3326511337 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1358003520 con 0x7f1358004d00 2023-11-08T16:41:19.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.981+0000 7f1370f66640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1358004d00 0x7f1358005180 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f136400a860 tx=0x7f13580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb985416cd133354 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:19.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.982+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364034630 con 0x7f1358004d00 2023-11-08T16:41:19.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.982+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f136402f070 con 0x7f1358004d00 2023-11-08T16:41:19.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.982+0000 7f136b7fe640 1 -- 172.21.15.81:0/3326511337 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364034a30 con 0x7f1358004d00 2023-11-08T16:41:19.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.982+0000 7f13729c9640 1 -- 172.21.15.81:0/3326511337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1358004d00 msgr2=0x7f1358005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.982+0000 7f13729c9640 1 --2- 172.21.15.81:0/3326511337 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1358004d00 0x7f1358005180 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f136400a860 tx=0x7f13580057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.982+0000 7f1370f66640 1 -- 172.21.15.81:0/3326511337 reap_dead start 2023-11-08T16:41:19.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.982+0000 7f13729c9640 1 -- 172.21.15.81:0/3326511337 shutdown_connections 2023-11-08T16:41:19.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.982+0000 7f13729c9640 1 -- 172.21.15.81:0/3326511337 >> 172.21.15.81:0/3326511337 conn(0x7f136c0b0770 msgr2=0x7f136c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:19.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.982+0000 7f13729c9640 1 -- 172.21.15.81:0/3326511337 shutdown_connections 2023-11-08T16:41:19.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.982+0000 7f13729c9640 1 -- 172.21.15.81:0/3326511337 wait complete. 2023-11-08T16:41:19.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.984+0000 7f13729c9640 1 Processor -- start 2023-11-08T16:41:19.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.984+0000 7f13729c9640 1 -- start start 2023-11-08T16:41:19.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.985+0000 7f13729c9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1358004d00 0x7f136c0c3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.985+0000 7f13729c9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f136c0c5290 0x7f136c0aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.985+0000 7f13729c9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f136c0b2060 0x7f136c0b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.985+0000 7f13729c9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0b1bd0 con 0x7f136c0b2060 2023-11-08T16:41:19.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.985+0000 7f13729c9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0c4020 con 0x7f136c0c5290 2023-11-08T16:41:19.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.985+0000 7f13729c9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f136c0c4300 con 0x7f1358004d00 2023-11-08T16:41:19.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.985+0000 7f1370f66640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f136c0b2060 0x7f136c0b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.986+0000 7f1370f66640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f136c0b2060 0x7f136c0b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37118/0 (socket says 172.21.15.81:37118) 2023-11-08T16:41:19.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.986+0000 7f1370f66640 1 -- 172.21.15.81:0/4093273657 learned_addr learned my addr 172.21.15.81:0/4093273657 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:19.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.986+0000 7f136bfff640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1358004d00 0x7f136c0c3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.986+0000 7f1363fff640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f136c0c5290 0x7f136c0aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.986+0000 7f1370f66640 1 -- 172.21.15.81:0/4093273657 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1358004d00 msgr2=0x7f136c0c3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.986+0000 7f1370f66640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1358004d00 0x7f136c0c3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.986+0000 7f1370f66640 1 -- 172.21.15.81:0/4093273657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f136c0c5290 msgr2=0x7f136c0aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:19.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.986+0000 7f1370f66640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f136c0c5290 0x7f136c0aeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:19.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.986+0000 7f1370f66640 1 -- 172.21.15.81:0/4093273657 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f136c0c45e0 con 0x7f136c0b2060 2023-11-08T16:41:19.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.986+0000 7f1370f66640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f136c0b2060 0x7f136c0b2510 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1364001cb0 tx=0x7f1364001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:19.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.987+0000 7f13697fa640 1 -- 172.21.15.81:0/4093273657 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1364001fd0 con 0x7f136c0b2060 2023-11-08T16:41:19.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.987+0000 7f13697fa640 1 -- 172.21.15.81:0/4093273657 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1364034640 con 0x7f136c0b2060 2023-11-08T16:41:19.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.987+0000 7f13697fa640 1 -- 172.21.15.81:0/4093273657 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13640349b0 con 0x7f136c0b2060 2023-11-08T16:41:19.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.987+0000 7f13729c9640 1 -- 172.21.15.81:0/4093273657 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c0b1860 con 0x7f136c0b2060 2023-11-08T16:41:19.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.987+0000 7f13729c9640 1 -- 172.21.15.81:0/4093273657 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c0aefe0 con 0x7f136c0b2060 2023-11-08T16:41:19.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.988+0000 7f13729c9640 1 -- 172.21.15.81:0/4093273657 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136c0b19f0 con 0x7f136c0b2060 2023-11-08T16:41:19.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.989+0000 7f13697fa640 1 -- 172.21.15.81:0/4093273657 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1364031030 con 0x7f136c0b2060 2023-11-08T16:41:19.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.990+0000 7f13697fa640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f134c06ae70 0x7f134c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:19.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.990+0000 7f13697fa640 1 -- 172.21.15.81:0/4093273657 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13640ab7e0 con 0x7f136c0b2060 2023-11-08T16:41:19.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.990+0000 7f136bfff640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f134c06ae70 0x7f134c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:19.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.991+0000 7f136bfff640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f134c06ae70 0x7f134c06d330 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f135c009a10 tx=0x7f135c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:20.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:19.994+0000 7f13697fa640 1 -- 172.21.15.81:0/4093273657 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1364076890 con 0x7f136c0b2060 2023-11-08T16:41:20.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.194+0000 7f13729c9640 1 -- 172.21.15.81:0/4093273657 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f136c0b19f0 con 0x7f136c0b2060 2023-11-08T16:41:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.195+0000 7f13697fa640 1 -- 172.21.15.81:0/4093273657 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1364079eb0 con 0x7f136c0b2060 2023-11-08T16:41:20.197 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:41:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 -- 172.21.15.81:0/4093273657 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f134c06ae70 msgr2=0x7f134c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f134c06ae70 0x7f134c06d330 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f135c009a10 tx=0x7f135c008040 comp rx=0 tx=0).stop 2023-11-08T16:41:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 -- 172.21.15.81:0/4093273657 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f136c0b2060 msgr2=0x7f136c0b2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f136c0b2060 0x7f136c0b2510 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f1364001cb0 tx=0x7f1364001d80 comp rx=0 tx=0).stop 2023-11-08T16:41:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 -- 172.21.15.81:0/4093273657 shutdown_connections 2023-11-08T16:41:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f134c06ae70 0x7f134c06d330 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f136c0b2060 0x7f136c0b2510 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f136c0c5290 0x7f136c0aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 --2- 172.21.15.81:0/4093273657 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1358004d00 0x7f136c0c3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 -- 172.21.15.81:0/4093273657 >> 172.21.15.81:0/4093273657 conn(0x7f136c0b0770 msgr2=0x7f136c15c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:20.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 -- 172.21.15.81:0/4093273657 shutdown_connections 2023-11-08T16:41:20.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.198+0000 7f13729c9640 1 -- 172.21.15.81:0/4093273657 wait complete. 2023-11-08T16:41:20.217 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:41:20.218 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:41:20.277 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:41:20.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.488+0000 7fa6b0e07640 1 Processor -- start 2023-11-08T16:41:20.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.488+0000 7fa6b0e07640 1 -- start start 2023-11-08T16:41:20.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.489+0000 7fa6b0e07640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa6ac15b500 0x7fa6ac144230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.489+0000 7fa6b0e07640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ac0b2310 con 0x7fa6ac0b3170 2023-11-08T16:41:20.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.489+0000 7fa6b0e07640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6ac0b25f0 con 0x7fa6ac0b3550 2023-11-08T16:41:20.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.489+0000 7fa6b0e07640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6ac0b2bb0 con 0x7fa6ac15b500 2023-11-08T16:41:20.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.489+0000 7fa6aa59c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa6ac0b3170 0x7fa6ac0b1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:39554/0 (socket says 172.21.15.81:39554) 2023-11-08T16:41:20.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.489+0000 7fa6aa59c640 1 -- 172.21.15.81:0/1597927846 learned_addr learned my addr 172.21.15.81:0/1597927846 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:20.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6aad9d640 1 --2- 172.21.15.81:0/1597927846 >> v2:172.21.15.93:3300/0 conn(0x7fa6ac15b500 0x7fa6ac144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2506444602 0 0) 0x7fa6ac0b2310 con 0x7fa6ac0b3170 2023-11-08T16:41:20.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa698003520 con 0x7fa6ac0b3170 2023-11-08T16:41:20.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3961614516 0 0) 0x7fa698003520 con 0x7fa6ac0b3170 2023-11-08T16:41:20.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 >> v2:172.21.15.93:3300/0 conn(0x7fa6ac15b500 msgr2=0x7fa6ac144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:20.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 --2- 172.21.15.81:0/1597927846 >> v2:172.21.15.93:3300/0 conn(0x7fa6ac15b500 0x7fa6ac144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 >> v1:172.21.15.93:6789/0 conn(0x7fa6ac0b3550 legacy=0x7fa6ac06c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac0b28d0 con 0x7fa6ac0b3170 2023-11-08T16:41:20.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fa69c001670 con 0x7fa6ac0b3170 2023-11-08T16:41:20.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 >> v1:172.21.15.81:6789/0 conn(0x7fa6ac0b3170 legacy=0x7fa6ac0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa6980039c0 0x7fa698003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6980047c0 0x7fa698004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6980051d0 0x7fa69800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac0b2bb0 con 0x7fa6980051d0 2023-11-08T16:41:20.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa69800c120 con 0x7fa6980047c0 2023-11-08T16:41:20.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.490+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa69800c2e0 con 0x7fa6980039c0 2023-11-08T16:41:20.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.491+0000 7fa6aad9d640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6980051d0 0x7fa69800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.491+0000 7fa6aa59c640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa6980039c0 0x7fa698003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.491+0000 7fa6a3fff640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6980047c0 0x7fa698004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.491+0000 7fa6aad9d640 1 -- 172.21.15.81:0/1597927846 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa6980039c0 msgr2=0x7fa698003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:20.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.491+0000 7fa6aad9d640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa6980039c0 0x7fa698003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.491+0000 7fa6aad9d640 1 -- 172.21.15.81:0/1597927846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6980047c0 msgr2=0x7fa698004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:20.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.491+0000 7fa6aad9d640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6980047c0 0x7fa698004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.491+0000 7fa6aad9d640 1 -- 172.21.15.81:0/1597927846 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa698003520 con 0x7fa6980051d0 2023-11-08T16:41:20.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.491+0000 7fa6aa59c640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa6980039c0 0x7fa698003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:20.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.491+0000 7fa6a3fff640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6980047c0 0x7fa698004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:20.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.492+0000 7fa6aad9d640 1 -- 172.21.15.81:0/1597927846 reap_dead start 2023-11-08T16:41:20.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.492+0000 7fa6aad9d640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6980051d0 0x7fa69800b980 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009f90 tx=0x7fa6980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974a69b74f9ddc46 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:20.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.492+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a402cc20 con 0x7fa6980051d0 2023-11-08T16:41:20.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.492+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6a400f040 con 0x7fa6980051d0 2023-11-08T16:41:20.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.492+0000 7fa6a9d9b640 1 -- 172.21.15.81:0/1597927846 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4003990 con 0x7fa6980051d0 2023-11-08T16:41:20.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.493+0000 7fa6b0e07640 1 -- 172.21.15.81:0/1597927846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6980051d0 msgr2=0x7fa69800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.493+0000 7fa6b0e07640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6980051d0 0x7fa69800b980 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fa6a4009f90 tx=0x7fa6980057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.493+0000 7fa6b0e07640 1 -- 172.21.15.81:0/1597927846 shutdown_connections 2023-11-08T16:41:20.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.493+0000 7fa6b0e07640 1 --2- 172.21.15.81:0/1597927846 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6980051d0 0x7fa69800b980 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.493+0000 7fa6b0e07640 1 -- 172.21.15.81:0/1597927846 >> 172.21.15.81:0/1597927846 conn(0x7fa6ac06a6a0 msgr2=0x7fa6ac0b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:20.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.493+0000 7fa6b0e07640 1 -- 172.21.15.81:0/1597927846 shutdown_connections 2023-11-08T16:41:20.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.493+0000 7fa6b0e07640 1 -- 172.21.15.81:0/1597927846 wait complete. 2023-11-08T16:41:20.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.495+0000 7fa6b0e07640 1 Processor -- start 2023-11-08T16:41:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.495+0000 7fa6b0e07640 1 -- start start 2023-11-08T16:41:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.496+0000 7fa6b0e07640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6980051d0 0x7fa6ac1e6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.496+0000 7fa6b0e07640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ac0c3a50 0x7fa6ac0c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.496+0000 7fa6b0e07640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa6ac0ae9d0 0x7fa6ac0aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.496+0000 7fa6b0e07640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac0ad970 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.496+0000 7fa6b0e07640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac0adc10 con 0x7fa6980051d0 2023-11-08T16:41:20.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.496+0000 7fa6b0e07640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ac0adef0 con 0x7fa6ac0ae9d0 2023-11-08T16:41:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.496+0000 7fa6a3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ac0c3a50 0x7fa6ac0c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.496+0000 7fa6a3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ac0c3a50 0x7fa6ac0c3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37128/0 (socket says 172.21.15.81:37128) 2023-11-08T16:41:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.496+0000 7fa6a3fff640 1 -- 172.21.15.81:0/3408412182 learned_addr learned my addr 172.21.15.81:0/3408412182 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6aa59c640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6980051d0 0x7fa6ac1e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6aad9d640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa6ac0ae9d0 0x7fa6ac0aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6a3fff640 1 -- 172.21.15.81:0/3408412182 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa6ac0ae9d0 msgr2=0x7fa6ac0aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6a3fff640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa6ac0ae9d0 0x7fa6ac0aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6a3fff640 1 -- 172.21.15.81:0/3408412182 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6980051d0 msgr2=0x7fa6ac1e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6a3fff640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6980051d0 0x7fa6ac1e6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6a3fff640 1 -- 172.21.15.81:0/3408412182 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac0ae1d0 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6aa59c640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6980051d0 0x7fa6ac1e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6a3fff640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ac0c3a50 0x7fa6ac0c3ed0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d770 tx=0x7fa69400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6a2ffd640 1 -- 172.21.15.81:0/3408412182 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694005270 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6a2ffd640 1 -- 172.21.15.81:0/3408412182 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6940053d0 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6a2ffd640 1 -- 172.21.15.81:0/3408412182 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69400b8d0 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6b0e07640 1 -- 172.21.15.81:0/3408412182 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ac0ae4b0 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.497+0000 7fa6b0e07640 1 -- 172.21.15.81:0/3408412182 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ac1e6840 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.498+0000 7fa6b0e07640 1 -- 172.21.15.81:0/3408412182 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ac074440 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.499+0000 7fa6a2ffd640 1 -- 172.21.15.81:0/3408412182 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa69400ba30 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.500+0000 7fa6a2ffd640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa68c06aea0 0x7fa68c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.500+0000 7fa6a2ffd640 1 -- 172.21.15.81:0/3408412182 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6940893a0 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.501+0000 7fa6aa59c640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa68c06aea0 0x7fa68c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.501+0000 7fa6aa59c640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa68c06aea0 0x7fa68c06d360 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0078d0 tx=0x7fa69c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:20.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.505+0000 7fa6a2ffd640 1 -- 172.21.15.81:0/3408412182 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa694054440 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.694+0000 7fa6b0e07640 1 -- 172.21.15.81:0/3408412182 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6ac1e6840 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.697+0000 7fa6a2ffd640 1 -- 172.21.15.81:0/3408412182 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+16467 (secure 0 0 0) 0x7fa6ac1e6840 con 0x7fa6ac0c3a50 2023-11-08T16:41:20.699 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:20.699 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":127,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:07.268068+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:41:03.545565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T16:41:04.726492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:41:20.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 -- 172.21.15.81:0/3408412182 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa68c06aea0 msgr2=0x7fa68c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa68c06aea0 0x7fa68c06d360 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa69c0078d0 tx=0x7fa69c009040 comp rx=0 tx=0).stop 2023-11-08T16:41:20.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 -- 172.21.15.81:0/3408412182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ac0c3a50 msgr2=0x7fa6ac0c3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ac0c3a50 0x7fa6ac0c3ed0 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fa69400d770 tx=0x7fa69400dc30 comp rx=0 tx=0).stop 2023-11-08T16:41:20.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 -- 172.21.15.81:0/3408412182 shutdown_connections 2023-11-08T16:41:20.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa68c06aea0 0x7fa68c06d360 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa6ac0ae9d0 0x7fa6ac0aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa6ac0c3a50 0x7fa6ac0c3ed0 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 --2- 172.21.15.81:0/3408412182 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6980051d0 0x7fa6ac1e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 -- 172.21.15.81:0/3408412182 >> 172.21.15.81:0/3408412182 conn(0x7fa6ac06a6a0 msgr2=0x7fa6ac0ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:20.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 -- 172.21.15.81:0/3408412182 shutdown_connections 2023-11-08T16:41:20.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.700+0000 7fa6b0e07640 1 -- 172.21.15.81:0/3408412182 wait complete. 2023-11-08T16:41:20.721 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:41:03.545565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:41:04.726492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:41:20.722 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:41:20.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.937+0000 7f36cba56640 1 Processor -- start 2023-11-08T16:41:20.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.937+0000 7f36cba56640 1 -- start start 2023-11-08T16:41:20.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.938+0000 7f36cba56640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c4159790 0x7f36c40ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.939+0000 7f36cba56640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c4159b70 con 0x7f36c40c9b40 2023-11-08T16:41:20.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.939+0000 7f36cba56640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c40cd1e0 con 0x7f36c415d9d0 2023-11-08T16:41:20.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.939+0000 7f36cba56640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c415e120 con 0x7f36c4159790 2023-11-08T16:41:20.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.939+0000 7f36c8ff1640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f36c40c9b40 0x7f36c415d2c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60970/0 (socket says 172.21.15.81:60970) 2023-11-08T16:41:20.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.939+0000 7f36c8ff1640 1 -- 172.21.15.81:0/1471431321 learned_addr learned my addr 172.21.15.81:0/1471431321 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:20.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.939+0000 7f36c97f2640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c4159790 0x7f36c40ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.939+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763448158 0 0) 0x7f36c4159b70 con 0x7f36c40c9b40 2023-11-08T16:41:20.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.939+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36b0003520 con 0x7f36c40c9b40 2023-11-08T16:41:20.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 333248301 0 0) 0x7f36b0003520 con 0x7f36c40c9b40 2023-11-08T16:41:20.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c4159790 msgr2=0x7f36c40ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:20.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c4159790 0x7f36c40ce280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 >> v1:172.21.15.93:6789/0 conn(0x7f36c415d9d0 legacy=0x7f36c406cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:20.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c40cd4c0 con 0x7f36c40c9b40 2023-11-08T16:41:20.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f36b4001670 con 0x7f36c40c9b40 2023-11-08T16:41:20.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 >> v1:172.21.15.81:6789/0 conn(0x7f36c40c9b40 legacy=0x7f36c415d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36b00039a0 0x7f36b0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 0x7f36b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c4159790 0x7f36b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c415e120 con 0x7f36b00047c0 2023-11-08T16:41:20.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b00054e0 con 0x7f36b00039a0 2023-11-08T16:41:20.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0005620 con 0x7f36c4159790 2023-11-08T16:41:20.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36c8ff1640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 0x7f36b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36c97f2640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36b00039a0 0x7f36b0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.940+0000 7f36c9ff3640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c4159790 0x7f36b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.941+0000 7f36c8ff1640 1 -- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c4159790 msgr2=0x7f36b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:20.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.941+0000 7f36c8ff1640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c4159790 0x7f36b000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.941+0000 7f36c8ff1640 1 -- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36b00039a0 msgr2=0x7f36b0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:20.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.941+0000 7f36c8ff1640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36b00039a0 0x7f36b0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.941+0000 7f36c8ff1640 1 -- 172.21.15.81:0/1471431321 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b0003520 con 0x7f36b00047c0 2023-11-08T16:41:20.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.941+0000 7f36c97f2640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36b00039a0 0x7f36b0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:20.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.942+0000 7f36c8ff1640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 0x7f36b0004c40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c240 tx=0x7f36b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44f1e613b6dc4e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:20.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.942+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b4005ad0 con 0x7f36b00047c0 2023-11-08T16:41:20.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.942+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b4005c30 con 0x7f36b00047c0 2023-11-08T16:41:20.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.942+0000 7f36bbfff640 1 -- 172.21.15.81:0/1471431321 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b402c8b0 con 0x7f36b00047c0 2023-11-08T16:41:20.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.942+0000 7f36cba56640 1 -- 172.21.15.81:0/1471431321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 msgr2=0x7f36b0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.942+0000 7f36cba56640 1 --2- 172.21.15.81:0/1471431321 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 0x7f36b0004c40 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f36b400c240 tx=0x7f36b00057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.942+0000 7f36c9ff3640 1 -- 172.21.15.81:0/1471431321 reap_dead start 2023-11-08T16:41:20.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.942+0000 7f36cba56640 1 -- 172.21.15.81:0/1471431321 shutdown_connections 2023-11-08T16:41:20.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.942+0000 7f36cba56640 1 -- 172.21.15.81:0/1471431321 >> 172.21.15.81:0/1471431321 conn(0x7f36c40ceb70 msgr2=0x7f36c40cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:20.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.943+0000 7f36cba56640 1 -- 172.21.15.81:0/1471431321 shutdown_connections 2023-11-08T16:41:20.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.943+0000 7f36cba56640 1 -- 172.21.15.81:0/1471431321 wait complete. 2023-11-08T16:41:20.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.944+0000 7f36cba56640 1 Processor -- start 2023-11-08T16:41:20.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.945+0000 7f36cba56640 1 -- start start 2023-11-08T16:41:20.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.945+0000 7f36cba56640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 0x7f36c41549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36cba56640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36c40b3e50 0x7f36c40b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36cba56640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c40ad870 0x7f36c40add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36cba56640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40ae240 con 0x7f36b00047c0 2023-11-08T16:41:20.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36cba56640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c41e56a0 con 0x7f36c40b3e50 2023-11-08T16:41:20.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36cba56640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c40c6950 con 0x7f36c40ad870 2023-11-08T16:41:20.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36c97f2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 0x7f36c41549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36c97f2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 0x7f36c41549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43846/0 (socket says 172.21.15.81:43846) 2023-11-08T16:41:20.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36c97f2640 1 -- 172.21.15.81:0/1843478194 learned_addr learned my addr 172.21.15.81:0/1843478194 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:20.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36c8ff1640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36c40b3e50 0x7f36c40b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36c97f2640 1 -- 172.21.15.81:0/1843478194 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c40ad870 msgr2=0x7f36c40add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36c9ff3640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c40ad870 0x7f36c40add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36c97f2640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c40ad870 0x7f36c40add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36c97f2640 1 -- 172.21.15.81:0/1843478194 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36c40b3e50 msgr2=0x7f36c40b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:20.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36c97f2640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36c40b3e50 0x7f36c40b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:20.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36c97f2640 1 -- 172.21.15.81:0/1843478194 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c40c6c30 con 0x7f36b00047c0 2023-11-08T16:41:20.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.946+0000 7f36c8ff1640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36c40b3e50 0x7f36c40b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:20.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.947+0000 7f36c97f2640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 0x7f36c41549f0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f36c000e7e0 tx=0x7f36c000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:20.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.947+0000 7f36b9ffb640 1 -- 172.21.15.81:0/1843478194 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c000c670 con 0x7f36b00047c0 2023-11-08T16:41:20.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.947+0000 7f36b9ffb640 1 -- 172.21.15.81:0/1843478194 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36c000c7d0 con 0x7f36b00047c0 2023-11-08T16:41:20.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.947+0000 7f36b9ffb640 1 -- 172.21.15.81:0/1843478194 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36c000cb40 con 0x7f36b00047c0 2023-11-08T16:41:20.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.947+0000 7f36cba56640 1 -- 172.21.15.81:0/1843478194 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c40b3390 con 0x7f36b00047c0 2023-11-08T16:41:20.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.947+0000 7f36cba56640 1 -- 172.21.15.81:0/1843478194 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c40b3950 con 0x7f36b00047c0 2023-11-08T16:41:20.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.948+0000 7f36cba56640 1 -- 172.21.15.81:0/1843478194 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c4074440 con 0x7f36b00047c0 2023-11-08T16:41:20.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.949+0000 7f36b9ffb640 1 -- 172.21.15.81:0/1843478194 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f36c000cca0 con 0x7f36b00047c0 2023-11-08T16:41:20.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.950+0000 7f36b9ffb640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f36a806aea0 0x7f36a806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:20.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.950+0000 7f36b9ffb640 1 -- 172.21.15.81:0/1843478194 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36c0089460 con 0x7f36b00047c0 2023-11-08T16:41:20.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.950+0000 7f36c8ff1640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f36a806aea0 0x7f36a806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:20.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.951+0000 7f36c8ff1640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f36a806aea0 0x7f36a806d360 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f36b4034000 tx=0x7f36b4009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:20.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:20.954+0000 7f36b9ffb640 1 -- 172.21.15.81:0/1843478194 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f36c0054470 con 0x7f36b00047c0 2023-11-08T16:41:21.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.143+0000 7f36cba56640 1 -- 172.21.15.81:0/1843478194 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36c40cf440 con 0x7f36b00047c0 2023-11-08T16:41:21.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.144+0000 7f36b9ffb640 1 -- 172.21.15.81:0/1843478194 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7f36c0057a90 con 0x7f36b00047c0 2023-11-08T16:41:21.146 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:41:21.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 -- 172.21.15.81:0/1843478194 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f36a806aea0 msgr2=0x7f36a806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f36a806aea0 0x7f36a806d360 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f36b4034000 tx=0x7f36b4009030 comp rx=0 tx=0).stop 2023-11-08T16:41:21.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 -- 172.21.15.81:0/1843478194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 msgr2=0x7f36c41549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 0x7f36c41549f0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f36c000e7e0 tx=0x7f36c000eca0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 -- 172.21.15.81:0/1843478194 shutdown_connections 2023-11-08T16:41:21.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f36a806aea0 0x7f36a806d360 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f36c40ad870 0x7f36c40add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f36c40b3e50 0x7f36c40b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 --2- 172.21.15.81:0/1843478194 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f36b00047c0 0x7f36c41549f0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 -- 172.21.15.81:0/1843478194 >> 172.21.15.81:0/1843478194 conn(0x7f36c40ceb70 msgr2=0x7f36c406fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:21.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 -- 172.21.15.81:0/1843478194 shutdown_connections 2023-11-08T16:41:21.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.147+0000 7f36cba56640 1 -- 172.21.15.81:0/1843478194 wait complete. 2023-11-08T16:41:21.168 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:41:21.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.383+0000 7f2bc58da640 1 Processor -- start 2023-11-08T16:41:21.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.383+0000 7f2bc58da640 1 -- start start 2023-11-08T16:41:21.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.385+0000 7f2bc58da640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2bc015d210 0x7f2bc00d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.385+0000 7f2bc58da640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2bc00d15a0 0x7f2bc00cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.385+0000 7f2bc58da640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc00d1a90 con 0x7f2bc00ce280 2023-11-08T16:41:21.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.385+0000 7f2bc58da640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc00ce730 con 0x7f2bc015d210 2023-11-08T16:41:21.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.385+0000 7f2bc58da640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00d1d70 con 0x7f2bc00d15a0 2023-11-08T16:41:21.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.385+0000 7f2bbeffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2bc015d210 0x7f2bc00d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:21.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.385+0000 7f2bbeffd640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2bc015d210 0x7f2bc00d1060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43848/0 (socket says 172.21.15.81:43848) 2023-11-08T16:41:21.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.385+0000 7f2bbeffd640 1 -- 172.21.15.81:0/579060262 learned_addr learned my addr 172.21.15.81:0/579060262 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:21.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.385+0000 7f2bbf7fe640 1 --1- 172.21.15.81:0/579060262 >> v1:172.21.15.81:6789/0 conn(0x7f2bc00ce280 0x7f2bc00b49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60986/0 (socket says 172.21.15.81:60986) 2023-11-08T16:41:21.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.386+0000 7f2bbe7fc640 1 --2- 172.21.15.81:0/579060262 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2bc00d15a0 0x7f2bc00cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:21.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.386+0000 7f2bbeffd640 1 -- 172.21.15.81:0/579060262 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2bc00d15a0 msgr2=0x7f2bc00cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:21.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.386+0000 7f2bbeffd640 1 --2- 172.21.15.81:0/579060262 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2bc00d15a0 0x7f2bc00cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.386+0000 7f2bbeffd640 1 -- 172.21.15.81:0/579060262 >> v1:172.21.15.81:6789/0 conn(0x7f2bc00ce280 legacy=0x7f2bc00b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.386+0000 7f2bbeffd640 1 -- 172.21.15.81:0/579060262 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc00d2050 con 0x7f2bc015d210 2023-11-08T16:41:21.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.387+0000 7f2bbeffd640 1 --2- 172.21.15.81:0/579060262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bc015d210 0x7f2bc00d1060 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009e50 tx=0x7f2bb002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fe7bb6fade7954a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:21.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.387+0000 7f2bbdffb640 1 -- 172.21.15.81:0/579060262 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb002cb00 con 0x7f2bc015d210 2023-11-08T16:41:21.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.387+0000 7f2bbdffb640 1 -- 172.21.15.81:0/579060262 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb002cc60 con 0x7f2bc015d210 2023-11-08T16:41:21.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.387+0000 7f2bbdffb640 1 -- 172.21.15.81:0/579060262 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00343f0 con 0x7f2bc015d210 2023-11-08T16:41:21.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.388+0000 7f2bc58da640 1 -- 172.21.15.81:0/579060262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bc015d210 msgr2=0x7f2bc00d1060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.388+0000 7f2bc58da640 1 --2- 172.21.15.81:0/579060262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bc015d210 0x7f2bc00d1060 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0009e50 tx=0x7f2bb002bf60 comp rx=0 tx=0).stop 2023-11-08T16:41:21.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.388+0000 7f2bc58da640 1 -- 172.21.15.81:0/579060262 shutdown_connections 2023-11-08T16:41:21.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.388+0000 7f2bc58da640 1 --2- 172.21.15.81:0/579060262 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2bc00d15a0 0x7f2bc00cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.388+0000 7f2bc58da640 1 --2- 172.21.15.81:0/579060262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bc015d210 0x7f2bc00d1060 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.388+0000 7f2bc58da640 1 -- 172.21.15.81:0/579060262 >> 172.21.15.81:0/579060262 conn(0x7f2bc00d2f70 msgr2=0x7f2bc00d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:21.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.388+0000 7f2bc58da640 1 -- 172.21.15.81:0/579060262 shutdown_connections 2023-11-08T16:41:21.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.388+0000 7f2bc58da640 1 -- 172.21.15.81:0/579060262 wait complete. 2023-11-08T16:41:21.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.390+0000 7f2bc58da640 1 Processor -- start 2023-11-08T16:41:21.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.390+0000 7f2bc58da640 1 -- start start 2023-11-08T16:41:21.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.391+0000 7f2bc58da640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2bc00ce280 0x7f2bc00cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.391+0000 7f2bc58da640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bc00d15a0 0x7f2bc015f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.391+0000 7f2bc58da640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2bc015d210 0x7f2bc00ca8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.391+0000 7f2bc58da640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc0160280 con 0x7f2bc00d15a0 2023-11-08T16:41:21.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.391+0000 7f2bc58da640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc00b38e0 con 0x7f2bc015d210 2023-11-08T16:41:21.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.391+0000 7f2bc58da640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc015f7d0 con 0x7f2bc00ce280 2023-11-08T16:41:21.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.391+0000 7f2bbe7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bc00d15a0 0x7f2bc015f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:21.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.391+0000 7f2bbe7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bc00d15a0 0x7f2bc015f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43860/0 (socket says 172.21.15.81:43860) 2023-11-08T16:41:21.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.391+0000 7f2bbe7fc640 1 -- 172.21.15.81:0/2802935228 learned_addr learned my addr 172.21.15.81:0/2802935228 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:21.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.392+0000 7f2bbe7fc640 1 -- 172.21.15.81:0/2802935228 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2bc00ce280 msgr2=0x7f2bc00cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.392+0000 7f2bbe7fc640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2bc00ce280 0x7f2bc00cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.392+0000 7f2bbe7fc640 1 -- 172.21.15.81:0/2802935228 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2bc015d210 msgr2=0x7f2bc00ca8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.392+0000 7f2bbe7fc640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2bc015d210 0x7f2bc00ca8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.392+0000 7f2bbe7fc640 1 -- 172.21.15.81:0/2802935228 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0009b30 con 0x7f2bc00d15a0 2023-11-08T16:41:21.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.392+0000 7f2bbe7fc640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bc00d15a0 0x7f2bc015f290 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800d800 tx=0x7f2ba800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:21.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.392+0000 7f2bb7fff640 1 -- 172.21.15.81:0/2802935228 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba800b6c0 con 0x7f2bc00d15a0 2023-11-08T16:41:21.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.392+0000 7f2bb7fff640 1 -- 172.21.15.81:0/2802935228 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba800b820 con 0x7f2bc00d15a0 2023-11-08T16:41:21.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.392+0000 7f2bb7fff640 1 -- 172.21.15.81:0/2802935228 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba800bbb0 con 0x7f2bc00d15a0 2023-11-08T16:41:21.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.392+0000 7f2bc58da640 1 -- 172.21.15.81:0/2802935228 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc015f9e0 con 0x7f2bc00d15a0 2023-11-08T16:41:21.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.392+0000 7f2bc58da640 1 -- 172.21.15.81:0/2802935228 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc015ffa0 con 0x7f2bc00d15a0 2023-11-08T16:41:21.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.393+0000 7f2bc58da640 1 -- 172.21.15.81:0/2802935228 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc00b3640 con 0x7f2bc00d15a0 2023-11-08T16:41:21.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.394+0000 7f2bb7fff640 1 -- 172.21.15.81:0/2802935228 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2ba800bd10 con 0x7f2bc00d15a0 2023-11-08T16:41:21.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.395+0000 7f2bb7fff640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ba006ae70 0x7f2ba006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.395+0000 7f2bb7fff640 1 -- 172.21.15.81:0/2802935228 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ba8089b00 con 0x7f2bc00d15a0 2023-11-08T16:41:21.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.395+0000 7f2bbeffd640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ba006ae70 0x7f2ba006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:21.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.396+0000 7f2bbeffd640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ba006ae70 0x7f2ba006d330 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0035000 tx=0x7f2bb002e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:21.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.399+0000 7f2bb7fff640 1 -- 172.21.15.81:0/2802935228 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2bc00b3640 con 0x7f2bc00d15a0 2023-11-08T16:41:21.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.576+0000 7f2bc58da640 1 -- 172.21.15.81:0/2802935228 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2bc00b3640 con 0x7f2bc00d15a0 2023-11-08T16:41:21.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.577+0000 7f2bb7fff640 1 -- 172.21.15.81:0/2802935228 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2bc00b3640 con 0x7f2bc00d15a0 2023-11-08T16:41:21.579 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:41:21.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.579+0000 7f2bc58da640 1 -- 172.21.15.81:0/2802935228 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ba006ae70 msgr2=0x7f2ba006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.579+0000 7f2bc58da640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ba006ae70 0x7f2ba006d330 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2bb0035000 tx=0x7f2bb002e000 comp rx=0 tx=0).stop 2023-11-08T16:41:21.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.579+0000 7f2bc58da640 1 -- 172.21.15.81:0/2802935228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bc00d15a0 msgr2=0x7f2bc015f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.579+0000 7f2bc58da640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bc00d15a0 0x7f2bc015f290 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800d800 tx=0x7f2ba800dcc0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.579+0000 7f2bc58da640 1 -- 172.21.15.81:0/2802935228 shutdown_connections 2023-11-08T16:41:21.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.579+0000 7f2bc58da640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2ba006ae70 0x7f2ba006d330 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.579+0000 7f2bc58da640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2bc015d210 0x7f2bc00ca8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.579+0000 7f2bc58da640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2bc00d15a0 0x7f2bc015f290 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.579+0000 7f2bc58da640 1 --2- 172.21.15.81:0/2802935228 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2bc00ce280 0x7f2bc00cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.579+0000 7f2bc58da640 1 -- 172.21.15.81:0/2802935228 >> 172.21.15.81:0/2802935228 conn(0x7f2bc00d2f70 msgr2=0x7f2bc015c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:21.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.580+0000 7f2bc58da640 1 -- 172.21.15.81:0/2802935228 shutdown_connections 2023-11-08T16:41:21.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.580+0000 7f2bc58da640 1 -- 172.21.15.81:0/2802935228 wait complete. 2023-11-08T16:41:21.598 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:41:21.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.803+0000 7ff6f3bb6640 1 Processor -- start 2023-11-08T16:41:21.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.803+0000 7ff6f3bb6640 1 -- start start 2023-11-08T16:41:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.805+0000 7ff6f3bb6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec15f380 0x7ff6ec0c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.805+0000 7ff6f3bb6640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec15fad0 con 0x7ff6ec071ad0 2023-11-08T16:41:21.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.805+0000 7ff6f3bb6640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec06d760 con 0x7ff6ec158c60 2023-11-08T16:41:21.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.805+0000 7ff6f3bb6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec06df80 con 0x7ff6ec15f380 2023-11-08T16:41:21.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.805+0000 7ff6f1151640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7ff6ec071ad0 0x7ff6ec15ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:60990/0 (socket says 172.21.15.81:60990) 2023-11-08T16:41:21.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.805+0000 7ff6f1151640 1 -- 172.21.15.81:0/2622451581 learned_addr learned my addr 172.21.15.81:0/2622451581 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:21.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.805+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455471614 0 0) 0x7ff6ec15fad0 con 0x7ff6ec071ad0 2023-11-08T16:41:21.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.805+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6d8003520 con 0x7ff6ec071ad0 2023-11-08T16:41:21.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.805+0000 7ff6f2153640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec15f380 0x7ff6ec0c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:21.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1796450052 0 0) 0x7ff6d8003520 con 0x7ff6ec071ad0 2023-11-08T16:41:21.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec15f380 msgr2=0x7ff6ec0c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:21.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec15f380 0x7ff6ec0c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 >> v1:172.21.15.93:6789/0 conn(0x7ff6ec158c60 legacy=0x7ff6ec0705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec06da00 con 0x7ff6ec071ad0 2023-11-08T16:41:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7ff6dc001670 con 0x7ff6ec071ad0 2023-11-08T16:41:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 >> v1:172.21.15.81:6789/0 conn(0x7ff6ec071ad0 legacy=0x7ff6ec15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d80039a0 0x7ff6d8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec15f380 0x7ff6d80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6d8004d00 0x7ff6d8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec06df80 con 0x7ff6d8004d00 2023-11-08T16:41:21.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d800bb10 con 0x7ff6d80039a0 2023-11-08T16:41:21.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d800bcd0 con 0x7ff6ec15f380 2023-11-08T16:41:21.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.806+0000 7ff6f2153640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6d8004d00 0x7ff6d8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:21.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.807+0000 7ff6f1151640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec15f380 0x7ff6d80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:21.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.807+0000 7ff6f1952640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d80039a0 0x7ff6d8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:21.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.807+0000 7ff6f2153640 1 -- 172.21.15.81:0/2622451581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec15f380 msgr2=0x7ff6d80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:21.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.807+0000 7ff6f2153640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec15f380 0x7ff6d80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.807+0000 7ff6f2153640 1 -- 172.21.15.81:0/2622451581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d80039a0 msgr2=0x7ff6d8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:21.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.807+0000 7ff6f2153640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6d80039a0 0x7ff6d8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.807+0000 7ff6f2153640 1 -- 172.21.15.81:0/2622451581 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d8003520 con 0x7ff6d8004d00 2023-11-08T16:41:21.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.808+0000 7ff6f2153640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6d8004d00 0x7ff6d8005180 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4007360 tx=0x7ff6d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5032d2c5321da4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:21.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.808+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e402ccd0 con 0x7ff6d8004d00 2023-11-08T16:41:21.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.808+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e402ce30 con 0x7ff6d8004d00 2023-11-08T16:41:21.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.808+0000 7ff6f0950640 1 -- 172.21.15.81:0/2622451581 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4034660 con 0x7ff6d8004d00 2023-11-08T16:41:21.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.808+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2622451581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6d8004d00 msgr2=0x7ff6d8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.816 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.808+0000 7ff6f3bb6640 1 --2- 172.21.15.81:0/2622451581 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6d8004d00 0x7ff6d8005180 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4007360 tx=0x7ff6d80057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.808+0000 7ff6f2153640 1 -- 172.21.15.81:0/2622451581 reap_dead start 2023-11-08T16:41:21.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.809+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2622451581 shutdown_connections 2023-11-08T16:41:21.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.809+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2622451581 >> 172.21.15.81:0/2622451581 conn(0x7ff6ec0ceb70 msgr2=0x7ff6ec0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:21.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.809+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2622451581 shutdown_connections 2023-11-08T16:41:21.817 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.809+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2622451581 wait complete. 2023-11-08T16:41:21.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.810+0000 7ff6f3bb6640 1 Processor -- start 2023-11-08T16:41:21.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.810+0000 7ff6f3bb6640 1 -- start start 2023-11-08T16:41:21.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.811+0000 7ff6f3bb6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6ec1f22b0 0x7ff6ec0ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.811+0000 7ff6f3bb6640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6ec1f2690 0x7ff6ec1f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.811+0000 7ff6f3bb6640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec1f3050 0x7ff6ec1f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.811+0000 7ff6f3bb6640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec150de0 con 0x7ff6ec1f22b0 2023-11-08T16:41:21.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.811+0000 7ff6f3bb6640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec1f34d0 con 0x7ff6ec1f2690 2023-11-08T16:41:21.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.811+0000 7ff6f3bb6640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec0b0fc0 con 0x7ff6ec1f3050 2023-11-08T16:41:21.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.812+0000 7ff6f1952640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6ec1f22b0 0x7ff6ec0ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:21.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.812+0000 7ff6f1952640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6ec1f22b0 0x7ff6ec0ae5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43876/0 (socket says 172.21.15.81:43876) 2023-11-08T16:41:21.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.812+0000 7ff6f1952640 1 -- 172.21.15.81:0/2724614184 learned_addr learned my addr 172.21.15.81:0/2724614184 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:21.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.812+0000 7ff6f1952640 1 -- 172.21.15.81:0/2724614184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec1f3050 msgr2=0x7ff6ec1f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.812+0000 7ff6f1952640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec1f3050 0x7ff6ec1f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.812+0000 7ff6f1952640 1 -- 172.21.15.81:0/2724614184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6ec1f2690 msgr2=0x7ff6ec1f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:21.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.812+0000 7ff6f1952640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6ec1f2690 0x7ff6ec1f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:21.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.812+0000 7ff6f1952640 1 -- 172.21.15.81:0/2724614184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0b12a0 con 0x7ff6ec1f22b0 2023-11-08T16:41:21.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.812+0000 7ff6f1952640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6ec1f22b0 0x7ff6ec0ae5d0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800e4d0 tx=0x7ff6e800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:21.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.812+0000 7ff6e27fc640 1 -- 172.21.15.81:0/2724614184 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80053a0 con 0x7ff6ec1f22b0 2023-11-08T16:41:21.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.813+0000 7ff6e27fc640 1 -- 172.21.15.81:0/2724614184 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e80035f0 con 0x7ff6ec1f22b0 2023-11-08T16:41:21.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.813+0000 7ff6e27fc640 1 -- 172.21.15.81:0/2724614184 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e80039a0 con 0x7ff6ec1f22b0 2023-11-08T16:41:21.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.813+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2724614184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec150ff0 con 0x7ff6ec1f22b0 2023-11-08T16:41:21.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.813+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2724614184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec1f3950 con 0x7ff6ec1f22b0 2023-11-08T16:41:21.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.814+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2724614184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ec074440 con 0x7ff6ec1f22b0 2023-11-08T16:41:21.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.815+0000 7ff6e27fc640 1 -- 172.21.15.81:0/2724614184 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff6e8003b00 con 0x7ff6ec1f22b0 2023-11-08T16:41:21.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.816+0000 7ff6e27fc640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff6d006ae70 0x7ff6d006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:21.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.816+0000 7ff6e27fc640 1 -- 172.21.15.81:0/2724614184 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6e8089380 con 0x7ff6ec1f22b0 2023-11-08T16:41:21.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.816+0000 7ff6f1151640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff6d006ae70 0x7ff6d006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:21.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.816+0000 7ff6f1151640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff6d006ae70 0x7ff6d006d330 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0017a0 tx=0x7ff6dc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:21.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:21.820+0000 7ff6e27fc640 1 -- 172.21.15.81:0/2724614184 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ff6e8054390 con 0x7ff6ec1f22b0 2023-11-08T16:41:22.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.010+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2724614184 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6ec156ef0 con 0x7ff6ec1f22b0 2023-11-08T16:41:22.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.011+0000 7ff6e27fc640 1 -- 172.21.15.81:0/2724614184 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6e8003db0 con 0x7ff6ec1f22b0 2023-11-08T16:41:22.013 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:41:22.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2724614184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff6d006ae70 msgr2=0x7ff6d006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:22.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff6d006ae70 0x7ff6d006d330 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ff6dc0017a0 tx=0x7ff6dc009040 comp rx=0 tx=0).stop 2023-11-08T16:41:22.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2724614184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6ec1f22b0 msgr2=0x7ff6ec0ae5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:22.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6ec1f22b0 0x7ff6ec0ae5d0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7ff6e800e4d0 tx=0x7ff6e800e990 comp rx=0 tx=0).stop 2023-11-08T16:41:22.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2724614184 shutdown_connections 2023-11-08T16:41:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ff6d006ae70 0x7ff6d006d330 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ff6ec1f3050 0x7ff6ec1f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff6ec1f2690 0x7ff6ec1f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 --2- 172.21.15.81:0/2724614184 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ff6ec1f22b0 0x7ff6ec0ae5d0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2724614184 >> 172.21.15.81:0/2724614184 conn(0x7ff6ec0ceb70 msgr2=0x7ff6ec0ca090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2724614184 shutdown_connections 2023-11-08T16:41:22.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.014+0000 7ff6f3bb6640 1 -- 172.21.15.81:0/2724614184 wait complete. 2023-11-08T16:41:22.035 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:41:22.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.244+0000 7f1825cf2640 1 Processor -- start 2023-11-08T16:41:22.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.245+0000 7f1825cf2640 1 -- start start 2023-11-08T16:41:22.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.246+0000 7f1825cf2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1820158aa0 0x7f182006fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:22.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.246+0000 7f1825cf2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18200cd3c0 0x7f18200cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:22.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.246+0000 7f1825cf2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1820071620 con 0x7f18200cbb10 2023-11-08T16:41:22.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.246+0000 7f1825cf2640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18200cbfa0 con 0x7f1820158aa0 2023-11-08T16:41:22.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.246+0000 7f1825cf2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1820158e80 con 0x7f18200cd3c0 2023-11-08T16:41:22.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.246+0000 7f181f7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1820158aa0 0x7f182006fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:22.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.246+0000 7f181ffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f18200cbb10 0x7f182006bc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32772/0 (socket says 172.21.15.81:32772) 2023-11-08T16:41:22.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.246+0000 7f181ffff640 1 -- 172.21.15.81:0/1991126507 learned_addr learned my addr 172.21.15.81:0/1991126507 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:22.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.247+0000 7f181effd640 1 --2- 172.21.15.81:0/1991126507 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18200cd3c0 0x7f18200cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:22.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.247+0000 7f181e7fc640 1 -- 172.21.15.81:0/1991126507 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186464782 0 0) 0x7f1820071620 con 0x7f18200cbb10 2023-11-08T16:41:22.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.247+0000 7f181f7fe640 1 -- 172.21.15.81:0/1991126507 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18200cd3c0 msgr2=0x7f18200cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:22.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.247+0000 7f181f7fe640 1 --2- 172.21.15.81:0/1991126507 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18200cd3c0 0x7f18200cb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.247+0000 7f181f7fe640 1 -- 172.21.15.81:0/1991126507 >> v1:172.21.15.81:6789/0 conn(0x7f18200cbb10 legacy=0x7f182006bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:22.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.247+0000 7f181f7fe640 1 -- 172.21.15.81:0/1991126507 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1820071850 con 0x7f1820158aa0 2023-11-08T16:41:22.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.248+0000 7f181f7fe640 1 --2- 172.21.15.81:0/1991126507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1820158aa0 0x7f182006fba0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f18100097a0 tx=0x7f181002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e395e588fdcd302f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:22.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.248+0000 7f181e7fc640 1 -- 172.21.15.81:0/1991126507 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181002d040 con 0x7f1820158aa0 2023-11-08T16:41:22.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.248+0000 7f181e7fc640 1 -- 172.21.15.81:0/1991126507 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1810031810 con 0x7f1820158aa0 2023-11-08T16:41:22.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.248+0000 7f181e7fc640 1 -- 172.21.15.81:0/1991126507 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1810031ba0 con 0x7f1820158aa0 2023-11-08T16:41:22.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.249+0000 7f1825cf2640 1 -- 172.21.15.81:0/1991126507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1820158aa0 msgr2=0x7f182006fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:22.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.249+0000 7f1825cf2640 1 --2- 172.21.15.81:0/1991126507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1820158aa0 0x7f182006fba0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f18100097a0 tx=0x7f181002b8b0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.249+0000 7f1825cf2640 1 -- 172.21.15.81:0/1991126507 shutdown_connections 2023-11-08T16:41:22.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.249+0000 7f1825cf2640 1 --2- 172.21.15.81:0/1991126507 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18200cd3c0 0x7f18200cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.249+0000 7f1825cf2640 1 --2- 172.21.15.81:0/1991126507 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1820158aa0 0x7f182006fba0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.249+0000 7f1825cf2640 1 -- 172.21.15.81:0/1991126507 >> 172.21.15.81:0/1991126507 conn(0x7f18200b0770 msgr2=0x7f18200b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:22.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.249+0000 7f1825cf2640 1 -- 172.21.15.81:0/1991126507 shutdown_connections 2023-11-08T16:41:22.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.249+0000 7f1825cf2640 1 -- 172.21.15.81:0/1991126507 wait complete. 2023-11-08T16:41:22.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.251+0000 7f1825cf2640 1 Processor -- start 2023-11-08T16:41:22.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.251+0000 7f1825cf2640 1 -- start start 2023-11-08T16:41:22.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.252+0000 7f1825cf2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18200cbb10 0x7f18201f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:22.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.252+0000 7f1825cf2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18200cd3c0 0x7f18200b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:22.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.252+0000 7f1825cf2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1820158aa0 0x7f182015da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:22.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.252+0000 7f1825cf2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18201f1690 con 0x7f18200cbb10 2023-11-08T16:41:22.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.252+0000 7f1825cf2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18201f0d60 con 0x7f1820158aa0 2023-11-08T16:41:22.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.252+0000 7f1825cf2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18201f1040 con 0x7f18200cd3c0 2023-11-08T16:41:22.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.252+0000 7f181f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18200cbb10 0x7f18201f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:22.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.252+0000 7f181f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18200cbb10 0x7f18201f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43894/0 (socket says 172.21.15.81:43894) 2023-11-08T16:41:22.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.252+0000 7f181f7fe640 1 -- 172.21.15.81:0/1258404200 learned_addr learned my addr 172.21.15.81:0/1258404200 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:22.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.253+0000 7f181f7fe640 1 -- 172.21.15.81:0/1258404200 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18200cd3c0 msgr2=0x7f18200b41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:22.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.253+0000 7f181f7fe640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18200cd3c0 0x7f18200b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.253+0000 7f181ffff640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1820158aa0 0x7f182015da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:22.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.253+0000 7f181f7fe640 1 -- 172.21.15.81:0/1258404200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1820158aa0 msgr2=0x7f182015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:22.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.253+0000 7f181f7fe640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1820158aa0 0x7f182015da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.253+0000 7f181f7fe640 1 -- 172.21.15.81:0/1258404200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1810009400 con 0x7f18200cbb10 2023-11-08T16:41:22.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.253+0000 7f181f7fe640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18200cbb10 0x7f18201f3800 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f18100097a0 tx=0x7f18100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:22.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.253+0000 7f1807fff640 1 -- 172.21.15.81:0/1258404200 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181003ad60 con 0x7f18200cbb10 2023-11-08T16:41:22.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.254+0000 7f1807fff640 1 -- 172.21.15.81:0/1258404200 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18100443f0 con 0x7f18200cbb10 2023-11-08T16:41:22.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.254+0000 7f1807fff640 1 -- 172.21.15.81:0/1258404200 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18100447a0 con 0x7f18200cbb10 2023-11-08T16:41:22.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.254+0000 7f1825cf2640 1 -- 172.21.15.81:0/1258404200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18201f1320 con 0x7f18200cbb10 2023-11-08T16:41:22.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.254+0000 7f1825cf2640 1 -- 172.21.15.81:0/1258404200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18201f0a10 con 0x7f18200cbb10 2023-11-08T16:41:22.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.255+0000 7f1825cf2640 1 -- 172.21.15.81:0/1258404200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1820074440 con 0x7f18200cbb10 2023-11-08T16:41:22.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.256+0000 7f1807fff640 1 -- 172.21.15.81:0/1258404200 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f181004d450 con 0x7f18200cbb10 2023-11-08T16:41:22.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.256+0000 7f1807fff640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f180c06ae70 0x7f180c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:22.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.257+0000 7f1807fff640 1 -- 172.21.15.81:0/1258404200 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18100ab8f0 con 0x7f18200cbb10 2023-11-08T16:41:22.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.257+0000 7f181effd640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f180c06ae70 0x7f180c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:22.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.257+0000 7f181effd640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f180c06ae70 0x7f180c06d330 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1814006fd0 tx=0x7f1814008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:22.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.261+0000 7f1807fff640 1 -- 172.21.15.81:0/1258404200 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1810076990 con 0x7f18200cbb10 2023-11-08T16:41:22.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.467+0000 7f1825cf2640 1 -- 172.21.15.81:0/1258404200 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f18200ce1f0 con 0x7f18200cbb10 2023-11-08T16:41:22.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.468+0000 7f1807fff640 1 -- 172.21.15.81:0/1258404200 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v127) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1810079fb0 con 0x7f18200cbb10 2023-11-08T16:41:22.470 INFO:teuthology.orchestra.run.smithi081.stderr:listed 0 entries 2023-11-08T16:41:22.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 -- 172.21.15.81:0/1258404200 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f180c06ae70 msgr2=0x7f180c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:22.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f180c06ae70 0x7f180c06d330 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1814006fd0 tx=0x7f1814008040 comp rx=0 tx=0).stop 2023-11-08T16:41:22.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 -- 172.21.15.81:0/1258404200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18200cbb10 msgr2=0x7f18201f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:22.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18200cbb10 0x7f18201f3800 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f18100097a0 tx=0x7f18100046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 -- 172.21.15.81:0/1258404200 shutdown_connections 2023-11-08T16:41:22.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f180c06ae70 0x7f180c06d330 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1820158aa0 0x7f182015da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18200cd3c0 0x7f18200b41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 --2- 172.21.15.81:0/1258404200 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f18200cbb10 0x7f18201f3800 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:22.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 -- 172.21.15.81:0/1258404200 >> 172.21.15.81:0/1258404200 conn(0x7f18200b0770 msgr2=0x7f18200b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:22.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 -- 172.21.15.81:0/1258404200 shutdown_connections 2023-11-08T16:41:22.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:22.471+0000 7f1825cf2640 1 -- 172.21.15.81:0/1258404200 wait complete. 2023-11-08T16:41:22.492 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:41:22.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:22.493 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:22.578 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[125091]: fuse finished with error 0 and tester_r 0 2023-11-08T16:41:22.585 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:41:28.686 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:41:28.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:28.687 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:41:28.749 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-08T16:41:29.028 INFO:teuthology.orchestra.run.smithi093.stdout:{"message":"","return_code":0} 2023-11-08T16:41:29.029 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-08T16:41:29.030 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:41:29.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.220+0000 7fa762d97640 1 Processor -- start 2023-11-08T16:41:29.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.221+0000 7fa762d97640 1 -- start start 2023-11-08T16:41:29.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.222+0000 7fa762d97640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa75c161450 0x7fa75c165080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.222+0000 7fa762d97640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa75c0cb950 0x7fa75c0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.222+0000 7fa762d97640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75c06ddd0 con 0x7fa75c1655c0 2023-11-08T16:41:29.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.222+0000 7fa762d97640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa75c0cc7b0 con 0x7fa75c161450 2023-11-08T16:41:29.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.222+0000 7fa762d97640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c165bd0 con 0x7fa75c0cb950 2023-11-08T16:41:29.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.223+0000 7fa760b33640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa75c161450 0x7fa75c165080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.223+0000 7fa761334640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa75c0cb950 0x7fa75c0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.223+0000 7fa760b33640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa75c161450 0x7fa75c165080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:37232/0 (socket says 172.21.15.81:37232) 2023-11-08T16:41:29.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.223+0000 7fa760b33640 1 -- 172.21.15.81:0/2641637949 learned_addr learned my addr 172.21.15.81:0/2641637949 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:29.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.224+0000 7fa760b33640 1 -- 172.21.15.81:0/2641637949 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa75c0cb950 msgr2=0x7fa75c0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:29.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.224+0000 7fa760b33640 1 --2- 172.21.15.81:0/2641637949 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa75c0cb950 0x7fa75c0cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.224+0000 7fa760b33640 1 -- 172.21.15.81:0/2641637949 >> v1:172.21.15.93:6789/0 conn(0x7fa75c1655c0 legacy=0x7fa75c0cb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.224+0000 7fa760b33640 1 -- 172.21.15.81:0/2641637949 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75c165eb0 con 0x7fa75c161450 2023-11-08T16:41:29.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.224+0000 7fa761334640 1 --2- 172.21.15.81:0/2641637949 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa75c0cb950 0x7fa75c0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:41:29.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.225+0000 7fa760b33640 1 --2- 172.21.15.81:0/2641637949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa75c161450 0x7fa75c165080 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009870 tx=0x7fa74c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=347dd2428ef787dd server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.225+0000 7fa75b7fe640 1 -- 172.21.15.81:0/2641637949 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c031640 con 0x7fa75c161450 2023-11-08T16:41:29.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.226+0000 7fa75b7fe640 1 -- 172.21.15.81:0/2641637949 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74c0317a0 con 0x7fa75c161450 2023-11-08T16:41:29.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.226+0000 7fa75b7fe640 1 -- 172.21.15.81:0/2641637949 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74c031b30 con 0x7fa75c161450 2023-11-08T16:41:29.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.226+0000 7fa762d97640 1 -- 172.21.15.81:0/2641637949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa75c161450 msgr2=0x7fa75c165080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.229 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.226+0000 7fa762d97640 1 --2- 172.21.15.81:0/2641637949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa75c161450 0x7fa75c165080 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa74c009870 tx=0x7fa74c02b8f0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.226+0000 7fa762d97640 1 -- 172.21.15.81:0/2641637949 shutdown_connections 2023-11-08T16:41:29.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.226+0000 7fa762d97640 1 --2- 172.21.15.81:0/2641637949 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa75c0cb950 0x7fa75c0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.226+0000 7fa762d97640 1 --2- 172.21.15.81:0/2641637949 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa75c161450 0x7fa75c165080 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.226+0000 7fa762d97640 1 -- 172.21.15.81:0/2641637949 >> 172.21.15.81:0/2641637949 conn(0x7fa75c0b0770 msgr2=0x7fa75c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:29.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.226+0000 7fa762d97640 1 -- 172.21.15.81:0/2641637949 shutdown_connections 2023-11-08T16:41:29.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.226+0000 7fa762d97640 1 -- 172.21.15.81:0/2641637949 wait complete. 2023-11-08T16:41:29.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.228+0000 7fa762d97640 1 Processor -- start 2023-11-08T16:41:29.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.228+0000 7fa762d97640 1 -- start start 2023-11-08T16:41:29.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.229+0000 7fa762d97640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa75c0cb950 0x7fa75c0bd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.229+0000 7fa762d97640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa75c161450 0x7fa75c14ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.229+0000 7fa762d97640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c1655c0 0x7fa75c0af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.229+0000 7fa762d97640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c0bdce0 con 0x7fa75c1655c0 2023-11-08T16:41:29.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.229+0000 7fa762d97640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c152590 con 0x7fa75c0cb950 2023-11-08T16:41:29.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.229+0000 7fa762d97640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75c1fa5d0 con 0x7fa75c161450 2023-11-08T16:41:29.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.230+0000 7fa761334640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c1655c0 0x7fa75c0af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.230+0000 7fa761334640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c1655c0 0x7fa75c0af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43900/0 (socket says 172.21.15.81:43900) 2023-11-08T16:41:29.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.230+0000 7fa761334640 1 -- 172.21.15.81:0/140791417 learned_addr learned my addr 172.21.15.81:0/140791417 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:29.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.230+0000 7fa760b33640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa75c0cb950 0x7fa75c0bd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.230+0000 7fa75bfff640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa75c161450 0x7fa75c14ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.230+0000 7fa761334640 1 -- 172.21.15.81:0/140791417 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa75c161450 msgr2=0x7fa75c14ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.230+0000 7fa761334640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa75c161450 0x7fa75c14ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.230+0000 7fa761334640 1 -- 172.21.15.81:0/140791417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa75c0cb950 msgr2=0x7fa75c0bd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.230+0000 7fa761334640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa75c0cb950 0x7fa75c0bd7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.230+0000 7fa761334640 1 -- 172.21.15.81:0/140791417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa754009a70 con 0x7fa75c1655c0 2023-11-08T16:41:29.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.230+0000 7fa760b33640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa75c0cb950 0x7fa75c0bd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:29.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.231+0000 7fa761334640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c1655c0 0x7fa75c0af400 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fa75400efb0 tx=0x7fa75400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.231+0000 7fa7597fa640 1 -- 172.21.15.81:0/140791417 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa754009d50 con 0x7fa75c1655c0 2023-11-08T16:41:29.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.231+0000 7fa7597fa640 1 -- 172.21.15.81:0/140791417 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75400f040 con 0x7fa75c1655c0 2023-11-08T16:41:29.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.231+0000 7fa7597fa640 1 -- 172.21.15.81:0/140791417 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7540146a0 con 0x7fa75c1655c0 2023-11-08T16:41:29.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.231+0000 7fa762d97640 1 -- 172.21.15.81:0/140791417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74c0094d0 con 0x7fa75c1655c0 2023-11-08T16:41:29.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.231+0000 7fa762d97640 1 -- 172.21.15.81:0/140791417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75c1fab90 con 0x7fa75c1655c0 2023-11-08T16:41:29.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.232+0000 7fa762d97640 1 -- 172.21.15.81:0/140791417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75c1fa430 con 0x7fa75c1655c0 2023-11-08T16:41:29.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.233+0000 7fa7597fa640 1 -- 172.21.15.81:0/140791417 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa754014800 con 0x7fa75c1655c0 2023-11-08T16:41:29.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.234+0000 7fa7597fa640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa73c06ae70 0x7fa73c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.234+0000 7fa7597fa640 1 -- 172.21.15.81:0/140791417 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa75408a680 con 0x7fa75c1655c0 2023-11-08T16:41:29.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.234+0000 7fa760b33640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa73c06ae70 0x7fa73c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.235+0000 7fa760b33640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa73c06ae70 0x7fa73c06d330 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa74c036000 tx=0x7fa74c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.239+0000 7fa7597fa640 1 -- 172.21.15.81:0/140791417 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa75c1fa430 con 0x7fa75c1655c0 2023-11-08T16:41:29.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.468+0000 7fa762d97640 1 -- 172.21.15.81:0/140791417 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa75c1fa430 con 0x7fa75c1655c0 2023-11-08T16:41:29.471 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:41:29.469+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:41:29.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.472+0000 7fa7597fa640 1 -- 172.21.15.81:0/140791417 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v108) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa75c1fa430 con 0x7fa75c1655c0 2023-11-08T16:41:29.474 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:41:29.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.475+0000 7fa762d97640 1 -- 172.21.15.81:0/140791417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa73c06ae70 msgr2=0x7fa73c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.475+0000 7fa762d97640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa73c06ae70 0x7fa73c06d330 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa74c036000 tx=0x7fa74c03c040 comp rx=0 tx=0).stop 2023-11-08T16:41:29.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.476+0000 7fa762d97640 1 -- 172.21.15.81:0/140791417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c1655c0 msgr2=0x7fa75c0af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.476+0000 7fa762d97640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c1655c0 0x7fa75c0af400 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fa75400efb0 tx=0x7fa75400c490 comp rx=0 tx=0).stop 2023-11-08T16:41:29.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.476+0000 7fa762d97640 1 -- 172.21.15.81:0/140791417 shutdown_connections 2023-11-08T16:41:29.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.476+0000 7fa762d97640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa73c06ae70 0x7fa73c06d330 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.476+0000 7fa762d97640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa75c1655c0 0x7fa75c0af400 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.476+0000 7fa762d97640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa75c161450 0x7fa75c14ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.476+0000 7fa762d97640 1 --2- 172.21.15.81:0/140791417 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa75c0cb950 0x7fa75c0bd7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.476+0000 7fa762d97640 1 -- 172.21.15.81:0/140791417 >> 172.21.15.81:0/140791417 conn(0x7fa75c0b0770 msgr2=0x7fa75c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:29.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.476+0000 7fa762d97640 1 -- 172.21.15.81:0/140791417 shutdown_connections 2023-11-08T16:41:29.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.476+0000 7fa762d97640 1 -- 172.21.15.81:0/140791417 wait complete. 2023-11-08T16:41:29.498 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.01000000 2023-11-08T16:41:29.593 INFO:tasks.ceph.mds.d.smithi093.stdout:starting mds.d at 2023-11-08T16:41:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.600+0000 7f958bc51f40 1 Processor -- start 2023-11-08T16:41:29.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.601+0000 7f958bc51f40 1 -- start start 2023-11-08T16:41:29.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.601+0000 7f958bc51f40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561ff2749650 0x561ff2779a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.601+0000 7f958bc51f40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ff281d3d0 0x561ff281aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.601+0000 7f958bc51f40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff2823320 0x561ff281d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.601+0000 7f958bc51f40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff26fa340 con 0x561ff2823320 2023-11-08T16:41:29.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.601+0000 7f958bc51f40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff2694a70 con 0x561ff281d3d0 2023-11-08T16:41:29.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.601+0000 7f958bc51f40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ff2694710 con 0x561ff2749650 2023-11-08T16:41:29.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.602+0000 7f958981c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff2823320 0x561ff281d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.602+0000 7f958981c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff2823320 0x561ff281d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43926/0 (socket says 172.21.15.81:43926) 2023-11-08T16:41:29.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.602+0000 7f958981c640 1 -- 172.21.15.81:0/3391743541 learned_addr learned my addr 172.21.15.81:0/3391743541 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:29.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.602+0000 7f958081a640 1 --2- 172.21.15.81:0/3391743541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ff281d3d0 0x561ff281aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.602+0000 7f958901b640 1 --2- 172.21.15.81:0/3391743541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561ff2749650 0x561ff2779a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.602+0000 7f958981c640 1 -- 172.21.15.81:0/3391743541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561ff2749650 msgr2=0x561ff2779a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:29.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.602+0000 7f958981c640 1 --2- 172.21.15.81:0/3391743541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561ff2749650 0x561ff2779a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.602+0000 7f958981c640 1 -- 172.21.15.81:0/3391743541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ff281d3d0 msgr2=0x561ff281aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:29.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.602+0000 7f958981c640 1 --2- 172.21.15.81:0/3391743541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ff281d3d0 0x561ff281aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.602+0000 7f958981c640 1 -- 172.21.15.81:0/3391743541 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c00a1c0 con 0x561ff2823320 2023-11-08T16:41:29.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.602+0000 7f958981c640 1 --2- 172.21.15.81:0/3391743541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff2823320 0x561ff281d9c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f9584001e60 tx=0x7f958400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954f49cfbbb32ad3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f9583fff640 1 -- 172.21.15.81:0/3391743541 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958400f5a0 con 0x561ff2823320 2023-11-08T16:41:29.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f9583fff640 1 -- 172.21.15.81:0/3391743541 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958400a040 con 0x561ff2823320 2023-11-08T16:41:29.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f9583fff640 1 -- 172.21.15.81:0/3391743541 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958400f960 con 0x561ff2823320 2023-11-08T16:41:29.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f958bc51f40 1 -- 172.21.15.81:0/3391743541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff2823320 msgr2=0x561ff281d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f958bc51f40 1 --2- 172.21.15.81:0/3391743541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff2823320 0x561ff281d9c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f9584001e60 tx=0x7f958400b650 comp rx=0 tx=0).stop 2023-11-08T16:41:29.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f958bc51f40 1 -- 172.21.15.81:0/3391743541 shutdown_connections 2023-11-08T16:41:29.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f958bc51f40 1 --2- 172.21.15.81:0/3391743541 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff2823320 0x561ff281d9c0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f958bc51f40 1 --2- 172.21.15.81:0/3391743541 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ff281d3d0 0x561ff281aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f958bc51f40 1 --2- 172.21.15.81:0/3391743541 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561ff2749650 0x561ff2779a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f958bc51f40 1 -- 172.21.15.81:0/3391743541 >> 172.21.15.81:0/3391743541 conn(0x561ff2795e00 msgr2=0x561ff2819b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:29.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f958bc51f40 1 -- 172.21.15.81:0/3391743541 shutdown_connections 2023-11-08T16:41:29.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.603+0000 7f958bc51f40 1 -- 172.21.15.81:0/3391743541 wait complete. 2023-11-08T16:41:29.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.604+0000 7f958bc51f40 1 Processor -- start 2023-11-08T16:41:29.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.604+0000 7f958bc51f40 1 -- start start 2023-11-08T16:41:29.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.605+0000 7f958bc51f40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ff2749650 0x561ff277ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.605+0000 7f958bc51f40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff281d3d0 0x561ff280c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.605+0000 7f958bc51f40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561ff2823320 0x561ff277a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.605+0000 7f958bc51f40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff2688520 con 0x561ff281d3d0 2023-11-08T16:41:29.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.605+0000 7f958bc51f40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561ff280cf70 con 0x561ff2749650 2023-11-08T16:41:29.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.605+0000 7f958bc51f40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x561ff277e1a0 con 0x561ff2823320 2023-11-08T16:41:29.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.605+0000 7f958081a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff281d3d0 0x561ff280c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.605+0000 7f958901b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ff2749650 0x561ff277ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.605+0000 7f958981c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561ff2823320 0x561ff277a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.605+0000 7f958901b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ff2749650 0x561ff277ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:37244/0 (socket says 172.21.15.81:37244) 2023-11-08T16:41:29.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.605+0000 7f958901b640 1 -- 172.21.15.81:0/2170124001 learned_addr learned my addr 172.21.15.81:0/2170124001 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:29.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.606+0000 7f958901b640 1 -- 172.21.15.81:0/2170124001 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561ff2823320 msgr2=0x561ff277a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.606+0000 7f958901b640 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x561ff2823320 0x561ff277a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.606+0000 7f958901b640 1 -- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff281d3d0 msgr2=0x561ff280c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.606+0000 7f958901b640 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff281d3d0 0x561ff280c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.606+0000 7f958901b640 1 -- 172.21.15.81:0/2170124001 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9584001a60 con 0x561ff2749650 2023-11-08T16:41:29.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.606+0000 7f958081a640 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x561ff281d3d0 0x561ff280c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:41:29.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.606+0000 7f958901b640 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ff2749650 0x561ff277ef60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f957c007000 tx=0x7f957c003330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.607+0000 7f9582ffd640 1 -- 172.21.15.81:0/2170124001 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c05d5f0 con 0x561ff2749650 2023-11-08T16:41:29.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.607+0000 7f9582ffd640 1 -- 172.21.15.81:0/2170124001 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957c05d750 con 0x561ff2749650 2023-11-08T16:41:29.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.607+0000 7f958bc51f40 1 -- 172.21.15.81:0/2170124001 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561ff26a0680 con 0x561ff2749650 2023-11-08T16:41:29.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.607+0000 7f9582ffd640 1 -- 172.21.15.81:0/2170124001 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957c05dac0 con 0x561ff2749650 2023-11-08T16:41:29.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.607+0000 7f958bc51f40 1 -- 172.21.15.81:0/2170124001 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561ff2693830 con 0x561ff2749650 2023-11-08T16:41:29.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.609+0000 7f9582ffd640 1 -- 172.21.15.81:0/2170124001 <== mon.1 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f957c05dc20 con 0x561ff2749650 2023-11-08T16:41:29.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.610+0000 7f9582ffd640 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f956c069820 0x7f956c06bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.610+0000 7f9582ffd640 1 -- 172.21.15.81:0/2170124001 <== mon.1 v2:172.21.15.93:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f957c0ca530 con 0x561ff2749650 2023-11-08T16:41:29.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.610+0000 7f958081a640 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f956c069820 0x7f956c06bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.610+0000 7f958bc51f40 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561ff2817bd0 0x561ff2815560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.610+0000 7f958bc51f40 1 -- 172.21.15.81:0/2170124001 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:1 16.5 16:a40620de:::10000000000.01000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e128) v8 -- 0x561ff2815aa0 con 0x561ff2817bd0 2023-11-08T16:41:29.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.610+0000 7f958981c640 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561ff2817bd0 0x561ff2815560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.611+0000 7f958081a640 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f956c069820 0x7f956c06bce0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x561ff2814350 tx=0x7f957400c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.611+0000 7f958981c640 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561ff2817bd0 0x561ff2815560 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.611+0000 7f958981c640 1 -- 172.21.15.81:0/2170124001 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(1 10000000000.01000000 [omap-get-keys out=560b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+560 (crc 0 0 0) 0x7f9584018040 con 0x561ff2817bd0 2023-11-08T16:41:29.621 INFO:teuthology.orchestra.run.smithi081.stdout:10_head 2023-11-08T16:41:29.621 INFO:teuthology.orchestra.run.smithi081.stdout:11_head 2023-11-08T16:41:29.621 INFO:teuthology.orchestra.run.smithi081.stdout:12_head 2023-11-08T16:41:29.621 INFO:teuthology.orchestra.run.smithi081.stdout:15_head 2023-11-08T16:41:29.622 INFO:teuthology.orchestra.run.smithi081.stdout:19_head 2023-11-08T16:41:29.622 INFO:teuthology.orchestra.run.smithi081.stdout:1_head 2023-11-08T16:41:29.622 INFO:teuthology.orchestra.run.smithi081.stdout:20_head 2023-11-08T16:41:29.622 INFO:teuthology.orchestra.run.smithi081.stdout:21_head 2023-11-08T16:41:29.622 INFO:teuthology.orchestra.run.smithi081.stdout:27_head 2023-11-08T16:41:29.623 INFO:teuthology.orchestra.run.smithi081.stdout:28_head 2023-11-08T16:41:29.623 INFO:teuthology.orchestra.run.smithi081.stdout:2_head 2023-11-08T16:41:29.623 INFO:teuthology.orchestra.run.smithi081.stdout:31_head 2023-11-08T16:41:29.623 INFO:teuthology.orchestra.run.smithi081.stdout:34_head 2023-11-08T16:41:29.623 INFO:teuthology.orchestra.run.smithi081.stdout:36_head 2023-11-08T16:41:29.623 INFO:teuthology.orchestra.run.smithi081.stdout:38_head 2023-11-08T16:41:29.624 INFO:teuthology.orchestra.run.smithi081.stdout:39_head 2023-11-08T16:41:29.624 INFO:teuthology.orchestra.run.smithi081.stdout:3_head 2023-11-08T16:41:29.624 INFO:teuthology.orchestra.run.smithi081.stdout:42_head 2023-11-08T16:41:29.624 INFO:teuthology.orchestra.run.smithi081.stdout:43_head 2023-11-08T16:41:29.624 INFO:teuthology.orchestra.run.smithi081.stdout:46_head 2023-11-08T16:41:29.625 INFO:teuthology.orchestra.run.smithi081.stdout:4_head 2023-11-08T16:41:29.625 INFO:teuthology.orchestra.run.smithi081.stdout:50_head 2023-11-08T16:41:29.625 INFO:teuthology.orchestra.run.smithi081.stdout:52_head 2023-11-08T16:41:29.625 INFO:teuthology.orchestra.run.smithi081.stdout:53_head 2023-11-08T16:41:29.625 INFO:teuthology.orchestra.run.smithi081.stdout:55_head 2023-11-08T16:41:29.626 INFO:teuthology.orchestra.run.smithi081.stdout:59_head 2023-11-08T16:41:29.626 INFO:teuthology.orchestra.run.smithi081.stdout:61_head 2023-11-08T16:41:29.626 INFO:teuthology.orchestra.run.smithi081.stdout:63_head 2023-11-08T16:41:29.626 INFO:teuthology.orchestra.run.smithi081.stdout:64_head 2023-11-08T16:41:29.626 INFO:teuthology.orchestra.run.smithi081.stdout:65_head 2023-11-08T16:41:29.627 INFO:teuthology.orchestra.run.smithi081.stdout:66_head 2023-11-08T16:41:29.627 INFO:teuthology.orchestra.run.smithi081.stdout:69_head 2023-11-08T16:41:29.627 INFO:teuthology.orchestra.run.smithi081.stdout:70_head 2023-11-08T16:41:29.627 INFO:teuthology.orchestra.run.smithi081.stdout:73_head 2023-11-08T16:41:29.627 INFO:teuthology.orchestra.run.smithi081.stdout:74_head 2023-11-08T16:41:29.627 INFO:teuthology.orchestra.run.smithi081.stdout:75_head 2023-11-08T16:41:29.628 INFO:teuthology.orchestra.run.smithi081.stdout:77_head 2023-11-08T16:41:29.628 INFO:teuthology.orchestra.run.smithi081.stdout:80_head 2023-11-08T16:41:29.628 INFO:teuthology.orchestra.run.smithi081.stdout:83_head 2023-11-08T16:41:29.628 INFO:teuthology.orchestra.run.smithi081.stdout:84_head 2023-11-08T16:41:29.628 INFO:teuthology.orchestra.run.smithi081.stdout:85_head 2023-11-08T16:41:29.629 INFO:teuthology.orchestra.run.smithi081.stdout:86_head 2023-11-08T16:41:29.629 INFO:teuthology.orchestra.run.smithi081.stdout:87_head 2023-11-08T16:41:29.629 INFO:teuthology.orchestra.run.smithi081.stdout:88_head 2023-11-08T16:41:29.629 INFO:teuthology.orchestra.run.smithi081.stdout:8_head 2023-11-08T16:41:29.629 INFO:teuthology.orchestra.run.smithi081.stdout:90_head 2023-11-08T16:41:29.630 INFO:teuthology.orchestra.run.smithi081.stdout:92_head 2023-11-08T16:41:29.630 INFO:teuthology.orchestra.run.smithi081.stdout:95_head 2023-11-08T16:41:29.630 INFO:teuthology.orchestra.run.smithi081.stdout:96_head 2023-11-08T16:41:29.630 INFO:teuthology.orchestra.run.smithi081.stdout:97_head 2023-11-08T16:41:29.630 INFO:teuthology.orchestra.run.smithi081.stdout:9_head 2023-11-08T16:41:29.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.612+0000 7f958bc51f40 1 -- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561ff2817bd0 msgr2=0x561ff2815560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.612+0000 7f958bc51f40 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x561ff2817bd0 0x561ff2815560 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.612+0000 7f958bc51f40 1 -- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f956c069820 msgr2=0x7f956c06bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.612+0000 7f958bc51f40 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f956c069820 0x7f956c06bce0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x561ff2814350 tx=0x7f957400c040 comp rx=0 tx=0).stop 2023-11-08T16:41:29.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.612+0000 7f958bc51f40 1 -- 172.21.15.81:0/2170124001 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ff2749650 msgr2=0x561ff277ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.612+0000 7f958bc51f40 1 --2- 172.21.15.81:0/2170124001 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561ff2749650 0x561ff277ef60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f957c007000 tx=0x7f957c003330 comp rx=0 tx=0).stop 2023-11-08T16:41:29.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.612+0000 7f958981c640 1 -- 172.21.15.81:0/2170124001 reap_dead start 2023-11-08T16:41:29.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.612+0000 7f958bc51f40 1 -- 172.21.15.81:0/2170124001 shutdown_connections 2023-11-08T16:41:29.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.612+0000 7f958bc51f40 1 -- 172.21.15.81:0/2170124001 >> 172.21.15.81:0/2170124001 conn(0x561ff2795e00 msgr2=0x561ff2738a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:29.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.612+0000 7f958bc51f40 1 -- 172.21.15.81:0/2170124001 shutdown_connections 2023-11-08T16:41:29.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.612+0000 7f958bc51f40 1 -- 172.21.15.81:0/2170124001 wait complete. 2023-11-08T16:41:29.634 INFO:tasks.cephfs.test_data_scan:victim_key=21_head 2023-11-08T16:41:29.635 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.01000000 21_head 2023-11-08T16:41:29.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.733+0000 7fe172234f40 1 Processor -- start 2023-11-08T16:41:29.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.733+0000 7fe172234f40 1 -- start start 2023-11-08T16:41:29.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.734+0000 7fe172234f40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c521eb0740 0x55c521f5ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.734+0000 7fe172234f40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c521ecbd00 0x55c521eac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.734+0000 7fe172234f40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521f57db0 0x55c521f55940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.734+0000 7fe172234f40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c521dc9a70 con 0x55c521f57db0 2023-11-08T16:41:29.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.734+0000 7fe172234f40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c521dc9710 con 0x55c521eb0740 2023-11-08T16:41:29.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.734+0000 7fe172234f40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c521dbd520 con 0x55c521ecbd00 2023-11-08T16:41:29.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.735+0000 7fe16fcf7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521f57db0 0x55c521f55940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.735+0000 7fe16fcf7640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521f57db0 0x55c521f55940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43954/0 (socket says 172.21.15.81:43954) 2023-11-08T16:41:29.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.735+0000 7fe16fcf7640 1 -- 172.21.15.81:0/321267134 learned_addr learned my addr 172.21.15.81:0/321267134 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:29.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.735+0000 7fe16fcf7640 1 -- 172.21.15.81:0/321267134 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c521ecbd00 msgr2=0x55c521eac5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:29.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.735+0000 7fe16fcf7640 1 --2- 172.21.15.81:0/321267134 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c521ecbd00 0x55c521eac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.735+0000 7fe16fcf7640 1 -- 172.21.15.81:0/321267134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c521eb0740 msgr2=0x55c521f5ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:29.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.735+0000 7fe16fcf7640 1 --2- 172.21.15.81:0/321267134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c521eb0740 0x55c521f5ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.735+0000 7fe16fcf7640 1 -- 172.21.15.81:0/321267134 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe158005de0 con 0x55c521f57db0 2023-11-08T16:41:29.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.735+0000 7fe16fcf7640 1 --2- 172.21.15.81:0/321267134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521f57db0 0x55c521f55940 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fe1680095a0 tx=0x7fe168009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d34c29cd0ee3da server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe16dcf3640 1 -- 172.21.15.81:0/321267134 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe16800e070 con 0x55c521f57db0 2023-11-08T16:41:29.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe16dcf3640 1 -- 172.21.15.81:0/321267134 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1680065c0 con 0x55c521f57db0 2023-11-08T16:41:29.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe16dcf3640 1 -- 172.21.15.81:0/321267134 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168006970 con 0x55c521f57db0 2023-11-08T16:41:29.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe172234f40 1 -- 172.21.15.81:0/321267134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521f57db0 msgr2=0x55c521f55940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe172234f40 1 --2- 172.21.15.81:0/321267134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521f57db0 0x55c521f55940 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fe1680095a0 tx=0x7fe168009a60 comp rx=0 tx=0).stop 2023-11-08T16:41:29.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe172234f40 1 -- 172.21.15.81:0/321267134 shutdown_connections 2023-11-08T16:41:29.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe172234f40 1 --2- 172.21.15.81:0/321267134 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521f57db0 0x55c521f55940 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe172234f40 1 --2- 172.21.15.81:0/321267134 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c521ecbd00 0x55c521eac5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe172234f40 1 --2- 172.21.15.81:0/321267134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c521eb0740 0x55c521f5ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe172234f40 1 -- 172.21.15.81:0/321267134 >> 172.21.15.81:0/321267134 conn(0x55c521ecc170 msgr2=0x55c521ec7ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:29.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe172234f40 1 -- 172.21.15.81:0/321267134 shutdown_connections 2023-11-08T16:41:29.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.736+0000 7fe172234f40 1 -- 172.21.15.81:0/321267134 wait complete. 2023-11-08T16:41:29.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.737+0000 7fe172234f40 1 Processor -- start 2023-11-08T16:41:29.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.737+0000 7fe172234f40 1 -- start start 2023-11-08T16:41:29.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.738+0000 7fe172234f40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c521eb0740 0x55c521f4b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.738+0000 7fe172234f40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521ecbd00 0x55c521eac680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.738+0000 7fe172234f40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c521f57db0 0x55c521f49b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.738+0000 7fe172234f40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c521f4aba0 con 0x55c521ecbd00 2023-11-08T16:41:29.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.738+0000 7fe172234f40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c521e839e0 con 0x55c521f57db0 2023-11-08T16:41:29.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.738+0000 7fe172234f40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c521e83e70 con 0x55c521eb0740 2023-11-08T16:41:29.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.738+0000 7fe16ecf5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521ecbd00 0x55c521eac680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.738+0000 7fe16ecf5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521ecbd00 0x55c521eac680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43956/0 (socket says 172.21.15.81:43956) 2023-11-08T16:41:29.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.738+0000 7fe16ecf5640 1 -- 172.21.15.81:0/1065789724 learned_addr learned my addr 172.21.15.81:0/1065789724 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:29.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.739+0000 7fe16fcf7640 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c521f57db0 0x55c521f49b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.739+0000 7fe16f4f6640 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c521eb0740 0x55c521f4b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.739+0000 7fe16ecf5640 1 -- 172.21.15.81:0/1065789724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c521eb0740 msgr2=0x55c521f4b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.739+0000 7fe16ecf5640 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55c521eb0740 0x55c521f4b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.739+0000 7fe16ecf5640 1 -- 172.21.15.81:0/1065789724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c521f57db0 msgr2=0x55c521f49b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.739+0000 7fe16ecf5640 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c521f57db0 0x55c521f49b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.739+0000 7fe16ecf5640 1 -- 172.21.15.81:0/1065789724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe168005dd0 con 0x55c521ecbd00 2023-11-08T16:41:29.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.739+0000 7fe16ecf5640 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521ecbd00 0x55c521eac680 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fe15800a4f0 tx=0x7fe15804adc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.740+0000 7fe16ccf1640 1 -- 172.21.15.81:0/1065789724 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1580529c0 con 0x55c521ecbd00 2023-11-08T16:41:29.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.740+0000 7fe16ccf1640 1 -- 172.21.15.81:0/1065789724 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158052b20 con 0x55c521ecbd00 2023-11-08T16:41:29.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.740+0000 7fe16ccf1640 1 -- 172.21.15.81:0/1065789724 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15805c400 con 0x55c521ecbd00 2023-11-08T16:41:29.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.740+0000 7fe172234f40 1 -- 172.21.15.81:0/1065789724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c521def670 con 0x55c521ecbd00 2023-11-08T16:41:29.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.740+0000 7fe172234f40 1 -- 172.21.15.81:0/1065789724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c521dd5680 con 0x55c521ecbd00 2023-11-08T16:41:29.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.742+0000 7fe16ccf1640 1 -- 172.21.15.81:0/1065789724 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe15805c560 con 0x55c521ecbd00 2023-11-08T16:41:29.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.742+0000 7fe16ccf1640 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe150069820 0x7fe15006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.743+0000 7fe16ccf1640 1 -- 172.21.15.81:0/1065789724 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fe1580c9b90 con 0x55c521ecbd00 2023-11-08T16:41:29.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.743+0000 7fe16f4f6640 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe150069820 0x7fe15006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.743+0000 7fe172234f40 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55c521ec54e0 0x55c521f4dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.743+0000 7fe172234f40 1 -- 172.21.15.81:0/1065789724 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:1 16.5 16:a40620de:::10000000000.01000000:head [omap-rm-keys in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e128) v8 -- 0x55c521eacbc0 con 0x55c521ec54e0 2023-11-08T16:41:29.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.743+0000 7fe16fcf7640 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55c521ec54e0 0x55c521f4dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.744+0000 7fe16f4f6640 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe150069820 0x7fe15006bce0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe16000b560 tx=0x7fe160007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.744+0000 7fe16fcf7640 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55c521ec54e0 0x55c521f4dc30 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe16fcf7640 1 -- 172.21.15.81:0/1065789724 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(1 10000000000.01000000 [omap-rm-keys] v128'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7fe168006b00 con 0x55c521ec54e0 2023-11-08T16:41:29.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe172234f40 1 -- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55c521ec54e0 msgr2=0x55c521f4dc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe172234f40 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55c521ec54e0 0x55c521f4dc30 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe172234f40 1 -- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe150069820 msgr2=0x7fe15006bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe172234f40 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe150069820 0x7fe15006bce0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fe16000b560 tx=0x7fe160007040 comp rx=0 tx=0).stop 2023-11-08T16:41:29.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe172234f40 1 -- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521ecbd00 msgr2=0x55c521eac680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe172234f40 1 --2- 172.21.15.81:0/1065789724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55c521ecbd00 0x55c521eac680 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fe15800a4f0 tx=0x7fe15804adc0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe16fcf7640 1 -- 172.21.15.81:0/1065789724 reap_dead start 2023-11-08T16:41:29.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe172234f40 1 -- 172.21.15.81:0/1065789724 shutdown_connections 2023-11-08T16:41:29.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe172234f40 1 -- 172.21.15.81:0/1065789724 >> 172.21.15.81:0/1065789724 conn(0x55c521ecc170 msgr2=0x55c521f505d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:29.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe172234f40 1 -- 172.21.15.81:0/1065789724 shutdown_connections 2023-11-08T16:41:29.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.747+0000 7fe172234f40 1 -- 172.21.15.81:0/1065789724 wait complete. 2023-11-08T16:41:29.757 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-08T16:41:29.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.955+0000 7fd8cfc7c640 1 Processor -- start 2023-11-08T16:41:29.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.955+0000 7fd8cfc7c640 1 -- start start 2023-11-08T16:41:29.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.956+0000 7fd8cfc7c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd8c80ce550 0x7fd8c80c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.956+0000 7fd8cfc7c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c80c9d40 con 0x7fd8c80c9910 2023-11-08T16:41:29.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.956+0000 7fd8cfc7c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8c8160f60 con 0x7fd8c815d280 2023-11-08T16:41:29.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.956+0000 7fd8cfc7c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8c8161be0 con 0x7fd8c80ce550 2023-11-08T16:41:29.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.957+0000 7fd8ce219640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd8c80c9910 0x7fd8c80ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:32784/0 (socket says 172.21.15.81:32784) 2023-11-08T16:41:29.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.957+0000 7fd8ce219640 1 -- 172.21.15.81:0/1486011861 learned_addr learned my addr 172.21.15.81:0/1486011861 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:29.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.957+0000 7fd8cd217640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fd8c80ce550 0x7fd8c80c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.957+0000 7fd8cca16640 1 -- 172.21.15.81:0/1486011861 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935624520 0 0) 0x7fd8c80c9d40 con 0x7fd8c80c9910 2023-11-08T16:41:29.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.957+0000 7fd8cca16640 1 -- 172.21.15.81:0/1486011861 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b4003520 con 0x7fd8c80c9910 2023-11-08T16:41:29.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.958+0000 7fd8cd217640 1 -- 172.21.15.81:0/1486011861 >> v1:172.21.15.93:6789/0 conn(0x7fd8c815d280 legacy=0x7fd8c815e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.958+0000 7fd8cd217640 1 -- 172.21.15.81:0/1486011861 >> v1:172.21.15.81:6789/0 conn(0x7fd8c80c9910 legacy=0x7fd8c80ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.958+0000 7fd8cd217640 1 -- 172.21.15.81:0/1486011861 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8c8161240 con 0x7fd8c80ce550 2023-11-08T16:41:29.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.959+0000 7fd8cd217640 1 --2- 172.21.15.81:0/1486011861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c80ce550 0x7fd8c80c93d0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0097a0 tx=0x7fd8bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c3ae4db4c5ccd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.959+0000 7fd8cca16640 1 -- 172.21.15.81:0/1486011861 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc02e070 con 0x7fd8c80ce550 2023-11-08T16:41:29.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.959+0000 7fd8cca16640 1 -- 172.21.15.81:0/1486011861 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8bc0315a0 con 0x7fd8c80ce550 2023-11-08T16:41:29.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.959+0000 7fd8cca16640 1 -- 172.21.15.81:0/1486011861 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc031930 con 0x7fd8c80ce550 2023-11-08T16:41:29.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.959+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/1486011861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c80ce550 msgr2=0x7fd8c80c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.959+0000 7fd8cfc7c640 1 --2- 172.21.15.81:0/1486011861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c80ce550 0x7fd8c80c93d0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc0097a0 tx=0x7fd8bc02b840 comp rx=0 tx=0).stop 2023-11-08T16:41:29.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.960+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/1486011861 shutdown_connections 2023-11-08T16:41:29.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.960+0000 7fd8cfc7c640 1 --2- 172.21.15.81:0/1486011861 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c80ce550 0x7fd8c80c93d0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.960+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/1486011861 >> 172.21.15.81:0/1486011861 conn(0x7fd8c80ceb70 msgr2=0x7fd8c80cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:29.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.960+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/1486011861 shutdown_connections 2023-11-08T16:41:29.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.960+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/1486011861 wait complete. 2023-11-08T16:41:29.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.961+0000 7fd8cfc7c640 1 Processor -- start 2023-11-08T16:41:29.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.962+0000 7fd8cfc7c640 1 -- start start 2023-11-08T16:41:29.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.962+0000 7fd8cfc7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8c80c9910 0x7fd8c8152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.962+0000 7fd8cfc7c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c80ce550 0x7fd8c80c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8cfc7c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c815d280 0x7fd8c8156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8cfc7c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80c31a0 con 0x7fd8c815d280 2023-11-08T16:41:29.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8cfc7c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80c3440 con 0x7fd8c80ce550 2023-11-08T16:41:29.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8cfc7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8c80c3720 con 0x7fd8c80c9910 2023-11-08T16:41:29.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8ce219640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c815d280 0x7fd8c8156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8ce219640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c815d280 0x7fd8c8156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43980/0 (socket says 172.21.15.81:43980) 2023-11-08T16:41:29.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8ce219640 1 -- 172.21.15.81:0/3661217785 learned_addr learned my addr 172.21.15.81:0/3661217785 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:29.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8ce219640 1 -- 172.21.15.81:0/3661217785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8c80c9910 msgr2=0x7fd8c8152260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8ce219640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8c80c9910 0x7fd8c8152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8ce219640 1 -- 172.21.15.81:0/3661217785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c80ce550 msgr2=0x7fd8c80c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:29.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8ce219640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c80ce550 0x7fd8c80c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:29.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.963+0000 7fd8ce219640 1 -- 172.21.15.81:0/3661217785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8bc009400 con 0x7fd8c815d280 2023-11-08T16:41:29.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.964+0000 7fd8ce219640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c815d280 0x7fd8c8156e10 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000cae0 tx=0x7fd8c000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.964+0000 7fd8ba7fc640 1 -- 172.21.15.81:0/3661217785 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c000ac40 con 0x7fd8c815d280 2023-11-08T16:41:29.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.964+0000 7fd8ba7fc640 1 -- 172.21.15.81:0/3661217785 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8c000ada0 con 0x7fd8c815d280 2023-11-08T16:41:29.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.964+0000 7fd8ba7fc640 1 -- 172.21.15.81:0/3661217785 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0012600 con 0x7fd8c815d280 2023-11-08T16:41:29.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.964+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/3661217785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8c80c3a00 con 0x7fd8c815d280 2023-11-08T16:41:29.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.964+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/3661217785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8c81527a0 con 0x7fd8c815d280 2023-11-08T16:41:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.965+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/3661217785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8c80c3f20 con 0x7fd8c815d280 2023-11-08T16:41:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.966+0000 7fd8ba7fc640 1 -- 172.21.15.81:0/3661217785 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8c0007070 con 0x7fd8c815d280 2023-11-08T16:41:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.967+0000 7fd8ba7fc640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd8ac06aea0 0x7fd8ac06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:29.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.967+0000 7fd8ba7fc640 1 -- 172.21.15.81:0/3661217785 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd8c00894c0 con 0x7fd8c815d280 2023-11-08T16:41:29.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.967+0000 7fd8cda18640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd8ac06aea0 0x7fd8ac06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:29.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.968+0000 7fd8cda18640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd8ac06aea0 0x7fd8ac06d360 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4000f60 tx=0x7fd8c40079b0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:29.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:29.971+0000 7fd8ba7fc640 1 -- 172.21.15.81:0/3661217785 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd8c80c3f20 con 0x7fd8c815d280 2023-11-08T16:41:30.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.193+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/3661217785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fd8c80c3f20 con 0x7fd8c815d280 2023-11-08T16:41:30.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.476+0000 7fd8ba7fc640 1 -- 172.21.15.81:0/3661217785 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v109) v1 ==== 162+0+0 (secure 0 0 0) 0x7fd8c80c3f20 con 0x7fd8c815d280 2023-11-08T16:41:30.478 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-08T16:41:30.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.480+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/3661217785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd8ac06aea0 msgr2=0x7fd8ac06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:30.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.480+0000 7fd8cfc7c640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd8ac06aea0 0x7fd8ac06d360 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fd8c4000f60 tx=0x7fd8c40079b0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.480+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/3661217785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c815d280 msgr2=0x7fd8c8156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:30.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.480+0000 7fd8cfc7c640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c815d280 0x7fd8c8156e10 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000cae0 tx=0x7fd8c000a060 comp rx=0 tx=0).stop 2023-11-08T16:41:30.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.480+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/3661217785 shutdown_connections 2023-11-08T16:41:30.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.480+0000 7fd8cfc7c640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd8ac06aea0 0x7fd8ac06d360 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.480+0000 7fd8cfc7c640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd8c815d280 0x7fd8c8156e10 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.480+0000 7fd8cfc7c640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8c80ce550 0x7fd8c80c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.481+0000 7fd8cfc7c640 1 --2- 172.21.15.81:0/3661217785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd8c80c9910 0x7fd8c8152260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.481+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/3661217785 >> 172.21.15.81:0/3661217785 conn(0x7fd8c80ceb70 msgr2=0x7fd8c8158af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:30.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.481+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/3661217785 shutdown_connections 2023-11-08T16:41:30.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.481+0000 7fd8cfc7c640 1 -- 172.21.15.81:0/3661217785 wait complete. 2023-11-08T16:41:30.505 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:41:30.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.690+0000 7f235e1ee640 1 Processor -- start 2023-11-08T16:41:30.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.690+0000 7f235e1ee640 1 -- start start 2023-11-08T16:41:30.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.691+0000 7f235e1ee640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f23580b3170 0x7f235815b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:30.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.691+0000 7f235e1ee640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23580b3550 0x7f23580b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:30.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.691+0000 7f235e1ee640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23580b2310 con 0x7f235806c9f0 2023-11-08T16:41:30.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.691+0000 7f235e1ee640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23580b2bb0 con 0x7f23580b3170 2023-11-08T16:41:30.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.691+0000 7f235e1ee640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23580b25f0 con 0x7f23580b3550 2023-11-08T16:41:30.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.692+0000 7f234effd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23580b3550 0x7f23580b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:30.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.692+0000 7f23577fe640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f23580b3170 0x7f235815b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:30.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.692+0000 7f2357fff640 1 --1- >> v1:172.21.15.93:6789/0 conn(0x7f235806c9f0 0x7f2358144230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:52838/0 (socket says 172.21.15.81:52838) 2023-11-08T16:41:30.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.692+0000 7f2357fff640 1 -- 172.21.15.81:0/4113676276 learned_addr learned my addr 172.21.15.81:0/4113676276 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:30.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.693+0000 7f23577fe640 1 -- 172.21.15.81:0/4113676276 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23580b3550 msgr2=0x7f23580b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:30.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.693+0000 7f23577fe640 1 --2- 172.21.15.81:0/4113676276 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23580b3550 0x7f23580b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.693+0000 7f23577fe640 1 -- 172.21.15.81:0/4113676276 >> v1:172.21.15.93:6789/0 conn(0x7f235806c9f0 legacy=0x7f2358144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:30.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.693+0000 7f23577fe640 1 -- 172.21.15.81:0/4113676276 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23580b28d0 con 0x7f23580b3170 2023-11-08T16:41:30.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.694+0000 7f23577fe640 1 --2- 172.21.15.81:0/4113676276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23580b3170 0x7f235815b500 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f23480097a0 tx=0x7f234802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53319d60efe17cf9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:30.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.695+0000 7f2356ffd640 1 -- 172.21.15.81:0/4113676276 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234802d040 con 0x7f23580b3170 2023-11-08T16:41:30.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.695+0000 7f2356ffd640 1 -- 172.21.15.81:0/4113676276 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2348031810 con 0x7f23580b3170 2023-11-08T16:41:30.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.695+0000 7f2356ffd640 1 -- 172.21.15.81:0/4113676276 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2348031ba0 con 0x7f23580b3170 2023-11-08T16:41:30.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.695+0000 7f235e1ee640 1 -- 172.21.15.81:0/4113676276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23580b3170 msgr2=0x7f235815b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:30.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.695+0000 7f235e1ee640 1 --2- 172.21.15.81:0/4113676276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23580b3170 0x7f235815b500 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f23480097a0 tx=0x7f234802b8b0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.696+0000 7f235e1ee640 1 -- 172.21.15.81:0/4113676276 shutdown_connections 2023-11-08T16:41:30.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.696+0000 7f235e1ee640 1 --2- 172.21.15.81:0/4113676276 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23580b3550 0x7f23580b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.696+0000 7f235e1ee640 1 --2- 172.21.15.81:0/4113676276 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23580b3170 0x7f235815b500 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.696+0000 7f235e1ee640 1 -- 172.21.15.81:0/4113676276 >> 172.21.15.81:0/4113676276 conn(0x7f235806a6a0 msgr2=0x7f23580b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:30.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.696+0000 7f235e1ee640 1 -- 172.21.15.81:0/4113676276 shutdown_connections 2023-11-08T16:41:30.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.696+0000 7f235e1ee640 1 -- 172.21.15.81:0/4113676276 wait complete. 2023-11-08T16:41:30.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.697+0000 7f235e1ee640 1 Processor -- start 2023-11-08T16:41:30.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.698+0000 7f235e1ee640 1 -- start start 2023-11-08T16:41:30.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.698+0000 7f235e1ee640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235806c9f0 0x7f2358138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:30.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.698+0000 7f235e1ee640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23580b3170 0x7f2358140e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:30.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f235e1ee640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23580b3550 0x7f23581413c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:30.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f235e1ee640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23581f2450 con 0x7f235806c9f0 2023-11-08T16:41:30.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f235e1ee640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2358159fc0 con 0x7f23580b3550 2023-11-08T16:41:30.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f235e1ee640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23581f2ba0 con 0x7f23580b3170 2023-11-08T16:41:30.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f23577fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235806c9f0 0x7f2358138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:30.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f23577fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235806c9f0 0x7f2358138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:43984/0 (socket says 172.21.15.81:43984) 2023-11-08T16:41:30.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f23577fe640 1 -- 172.21.15.81:0/3433750344 learned_addr learned my addr 172.21.15.81:0/3433750344 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:30.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f234effd640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23580b3170 0x7f2358140e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:30.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f23577fe640 1 -- 172.21.15.81:0/3433750344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23580b3170 msgr2=0x7f2358140e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:30.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f23577fe640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23580b3170 0x7f2358140e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f2357fff640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23580b3550 0x7f23581413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f23577fe640 1 -- 172.21.15.81:0/3433750344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23580b3550 msgr2=0x7f23581413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f23577fe640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23580b3550 0x7f23581413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f23577fe640 1 -- 172.21.15.81:0/3433750344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2348009400 con 0x7f235806c9f0 2023-11-08T16:41:30.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.699+0000 7f2357fff640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23580b3550 0x7f23581413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.700+0000 7f23577fe640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235806c9f0 0x7f2358138d10 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f234802bec0 tx=0x7f23480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.700+0000 7f2354ff9640 1 -- 172.21.15.81:0/3433750344 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234803ad60 con 0x7f235806c9f0 2023-11-08T16:41:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.700+0000 7f2354ff9640 1 -- 172.21.15.81:0/3433750344 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23480423f0 con 0x7f235806c9f0 2023-11-08T16:41:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.700+0000 7f235e1ee640 1 -- 172.21.15.81:0/3433750344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23581f1e90 con 0x7f235806c9f0 2023-11-08T16:41:30.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.700+0000 7f2354ff9640 1 -- 172.21.15.81:0/3433750344 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23480427a0 con 0x7f235806c9f0 2023-11-08T16:41:30.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.700+0000 7f235e1ee640 1 -- 172.21.15.81:0/3433750344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23581f2830 con 0x7f235806c9f0 2023-11-08T16:41:30.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.701+0000 7f235e1ee640 1 -- 172.21.15.81:0/3433750344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2358074440 con 0x7f235806c9f0 2023-11-08T16:41:30.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.702+0000 7f2354ff9640 1 -- 172.21.15.81:0/3433750344 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2348042900 con 0x7f235806c9f0 2023-11-08T16:41:30.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.703+0000 7f2354ff9640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 0x7f233806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:30.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.703+0000 7f2354ff9640 1 -- 172.21.15.81:0/3433750344 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f23480ab6a0 con 0x7f235806c9f0 2023-11-08T16:41:30.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.703+0000 7f234effd640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 0x7f233806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:30.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.704+0000 7f234effd640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 0x7f233806d330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f235815b010 tx=0x7f2340008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:30.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.707+0000 7f2354ff9640 1 -- 172.21.15.81:0/3433750344 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2348076660 con 0x7f235806c9f0 2023-11-08T16:41:30.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.938+0000 7f235e1ee640 1 -- 172.21.15.81:0/3433750344 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f235806df50 con 0x7f235806c9f0 2023-11-08T16:41:30.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.939+0000 7f2354ff9640 1 -- 172.21.15.81:0/3433750344 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 110 v110) v1 ==== 95+0+4917 (secure 0 0 0) 0x7f2348079c80 con 0x7f235806c9f0 2023-11-08T16:41:30.941 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:30.941 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":110,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/329260749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":329260749},{"type":"v1","addr":"172.21.15.93:6835","nonce":329260749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":110,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:30.481884+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8556":{"gid":8556,"name":"c","rank":0,"incarnation":110,"state":"up:replay","state_seq":1,"addr":"172.21.15.81:6837/296230272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":296230272},{"type":"v1","addr":"172.21.15.81:6837","nonce":296230272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:41:30.942 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 110 2023-11-08T16:41:30.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.941+0000 7f235e1ee640 1 -- 172.21.15.81:0/3433750344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 msgr2=0x7f233806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:30.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.941+0000 7f235e1ee640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 0x7f233806d330 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f235815b010 tx=0x7f2340008040 comp rx=0 tx=0).stop 2023-11-08T16:41:30.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.941+0000 7f235e1ee640 1 -- 172.21.15.81:0/3433750344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235806c9f0 msgr2=0x7f2358138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:30.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.941+0000 7f235e1ee640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235806c9f0 0x7f2358138d10 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f234802bec0 tx=0x7f23480046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.941+0000 7f235e1ee640 1 -- 172.21.15.81:0/3433750344 shutdown_connections 2023-11-08T16:41:30.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.941+0000 7f235e1ee640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f233806ae70 0x7f233806d330 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.941+0000 7f235e1ee640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23580b3550 0x7f23581413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.941+0000 7f235e1ee640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23580b3170 0x7f2358140e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.941+0000 7f235e1ee640 1 --2- 172.21.15.81:0/3433750344 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f235806c9f0 0x7f2358138d10 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:30.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.941+0000 7f235e1ee640 1 -- 172.21.15.81:0/3433750344 >> 172.21.15.81:0/3433750344 conn(0x7f235806a6a0 msgr2=0x7f23580cd030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:30.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.942+0000 7f235e1ee640 1 -- 172.21.15.81:0/3433750344 shutdown_connections 2023-11-08T16:41:30.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:30.942+0000 7f235e1ee640 1 -- 172.21.15.81:0/3433750344 wait complete. 2023-11-08T16:41:31.363 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 110, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:41:05.700435+0000', 'modified': '2023-11-08T16:41:30.481884+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8556': {'gid': 8556, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.81:6837/296230272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 296230272}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 296230272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:41:31.363 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8556:up:replay 2023-11-08T16:41:32.364 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:41:32.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.586+0000 7fd6fef6c640 1 Processor -- start 2023-11-08T16:41:32.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.586+0000 7fd6fef6c640 1 -- start start 2023-11-08T16:41:32.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.588+0000 7fd6fef6c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6f8143800 0x7fd6f80b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:32.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.588+0000 7fd6fef6c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fd6f80cca70 0x7fd6f815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:32.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.588+0000 7fd6fef6c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f80714f0 con 0x7fd6f815df20 2023-11-08T16:41:32.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.588+0000 7fd6fef6c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f80b2df0 con 0x7fd6f80cca70 2023-11-08T16:41:32.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.588+0000 7fd6fef6c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f815e390 con 0x7fd6f8143800 2023-11-08T16:41:32.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.588+0000 7fd6fd509640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd6f815df20 0x7fd6f815a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56158/0 (socket says 172.21.15.81:56158) 2023-11-08T16:41:32.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.588+0000 7fd6fd509640 1 -- 172.21.15.81:0/2718780152 learned_addr learned my addr 172.21.15.81:0/2718780152 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:32.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.588+0000 7fd6fcd08640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6f8143800 0x7fd6f80b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:32.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.588+0000 7fd6f7fff640 1 --2- 172.21.15.81:0/2718780152 >> v2:172.21.15.93:3300/0 conn(0x7fd6f80cca70 0x7fd6f815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:32.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080421456 0 0) 0x7fd6f80714f0 con 0x7fd6f815df20 2023-11-08T16:41:32.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e8003520 con 0x7fd6f815df20 2023-11-08T16:41:32.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 872419754 0 0) 0x7fd6e8003520 con 0x7fd6f815df20 2023-11-08T16:41:32.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6f8143800 msgr2=0x7fd6f80b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:32.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6f8143800 0x7fd6f80b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:32.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 >> v2:172.21.15.93:3300/0 conn(0x7fd6f80cca70 msgr2=0x7fd6f815d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:32.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 --2- 172.21.15.81:0/2718780152 >> v2:172.21.15.93:3300/0 conn(0x7fd6f80cca70 0x7fd6f815d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:32.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f80cd1e0 con 0x7fd6f815df20 2023-11-08T16:41:32.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fd6f00019f0 con 0x7fd6f815df20 2023-11-08T16:41:32.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 >> v1:172.21.15.81:6789/0 conn(0x7fd6f815df20 legacy=0x7fd6f815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:32.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6e8003970 0x7fd6e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:32.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6f8143800 0x7fd6e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:32.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6e8004d00 0x7fd6e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:32.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80b2df0 con 0x7fd6e8004d00 2023-11-08T16:41:32.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f815e390 con 0x7fd6e8003970 2023-11-08T16:41:32.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e800b980 con 0x7fd6f8143800 2023-11-08T16:41:32.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.589+0000 7fd6fd509640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6e8004d00 0x7fd6e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:32.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.590+0000 7fd6f7fff640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6f8143800 0x7fd6e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:32.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.590+0000 7fd6fcd08640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6e8003970 0x7fd6e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:32.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.590+0000 7fd6fd509640 1 -- 172.21.15.81:0/2718780152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6f8143800 msgr2=0x7fd6e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:32.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.590+0000 7fd6fd509640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6f8143800 0x7fd6e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:32.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.590+0000 7fd6fd509640 1 -- 172.21.15.81:0/2718780152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6e8003970 msgr2=0x7fd6e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:32.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.590+0000 7fd6fd509640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6e8003970 0x7fd6e8003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:32.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.590+0000 7fd6fd509640 1 -- 172.21.15.81:0/2718780152 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e8003520 con 0x7fd6e8004d00 2023-11-08T16:41:32.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.591+0000 7fd6fd509640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6e8004d00 0x7fd6e8005180 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000a860 tx=0x7fd6e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d972d16a9c69020 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:32.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.591+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0034630 con 0x7fd6e8004d00 2023-11-08T16:41:32.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.591+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f002f070 con 0x7fd6e8004d00 2023-11-08T16:41:32.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.591+0000 7fd6f77fe640 1 -- 172.21.15.81:0/2718780152 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f00349f0 con 0x7fd6e8004d00 2023-11-08T16:41:32.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.592+0000 7fd6fef6c640 1 -- 172.21.15.81:0/2718780152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6e8004d00 msgr2=0x7fd6e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:32.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.592+0000 7fd6fef6c640 1 --2- 172.21.15.81:0/2718780152 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6e8004d00 0x7fd6e8005180 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000a860 tx=0x7fd6e80057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:32.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.592+0000 7fd6fd509640 1 -- 172.21.15.81:0/2718780152 reap_dead start 2023-11-08T16:41:32.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.592+0000 7fd6fef6c640 1 -- 172.21.15.81:0/2718780152 shutdown_connections 2023-11-08T16:41:32.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.592+0000 7fd6fef6c640 1 -- 172.21.15.81:0/2718780152 >> 172.21.15.81:0/2718780152 conn(0x7fd6f806a6a0 msgr2=0x7fd6f80b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:32.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.592+0000 7fd6fef6c640 1 -- 172.21.15.81:0/2718780152 shutdown_connections 2023-11-08T16:41:32.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.592+0000 7fd6fef6c640 1 -- 172.21.15.81:0/2718780152 wait complete. 2023-11-08T16:41:32.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.594+0000 7fd6fef6c640 1 Processor -- start 2023-11-08T16:41:32.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.594+0000 7fd6fef6c640 1 -- start start 2023-11-08T16:41:32.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.595+0000 7fd6fef6c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e8004d00 0x7fd6f81599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:32.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.595+0000 7fd6fef6c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6f813b830 0x7fd6f813bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:32.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.595+0000 7fd6fef6c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f80af440 0x7fd6f80af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:32.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.595+0000 7fd6fef6c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f80afe00 con 0x7fd6f80af440 2023-11-08T16:41:32.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.595+0000 7fd6fef6c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f8159ef0 con 0x7fd6f813b830 2023-11-08T16:41:32.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.595+0000 7fd6fef6c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f815ccb0 con 0x7fd6e8004d00 2023-11-08T16:41:32.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.595+0000 7fd6fd509640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f80af440 0x7fd6f80af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:32.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.595+0000 7fd6fd509640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f80af440 0x7fd6f80af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37470/0 (socket says 172.21.15.81:37470) 2023-11-08T16:41:32.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.595+0000 7fd6fd509640 1 -- 172.21.15.81:0/324156003 learned_addr learned my addr 172.21.15.81:0/324156003 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:32.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6fcd08640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e8004d00 0x7fd6f81599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:32.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6fd509640 1 -- 172.21.15.81:0/324156003 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e8004d00 msgr2=0x7fd6f81599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:32.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6f7fff640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6f813b830 0x7fd6f813bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:32.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6fd509640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e8004d00 0x7fd6f81599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:32.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6fd509640 1 -- 172.21.15.81:0/324156003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6f813b830 msgr2=0x7fd6f813bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:32.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6fd509640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6f813b830 0x7fd6f813bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:32.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6fd509640 1 -- 172.21.15.81:0/324156003 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f813c380 con 0x7fd6f80af440 2023-11-08T16:41:32.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6fcd08640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e8004d00 0x7fd6f81599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:32.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6fd509640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f80af440 0x7fd6f80af8c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000a980 tx=0x7fd6f0001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:32.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6f57fa640 1 -- 172.21.15.81:0/324156003 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0034840 con 0x7fd6f80af440 2023-11-08T16:41:32.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6f57fa640 1 -- 172.21.15.81:0/324156003 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f00349a0 con 0x7fd6f80af440 2023-11-08T16:41:32.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6fef6c640 1 -- 172.21.15.81:0/324156003 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f813c660 con 0x7fd6f80af440 2023-11-08T16:41:32.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6f57fa640 1 -- 172.21.15.81:0/324156003 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0034d10 con 0x7fd6f80af440 2023-11-08T16:41:32.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.596+0000 7fd6fef6c640 1 -- 172.21.15.81:0/324156003 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f813cc20 con 0x7fd6f80af440 2023-11-08T16:41:32.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.597+0000 7fd6fef6c640 1 -- 172.21.15.81:0/324156003 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f8074440 con 0x7fd6f80af440 2023-11-08T16:41:32.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.598+0000 7fd6f57fa640 1 -- 172.21.15.81:0/324156003 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd6f0031030 con 0x7fd6f80af440 2023-11-08T16:41:32.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.599+0000 7fd6f57fa640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae00 0x7fd6dc06d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:32.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.599+0000 7fd6f57fa640 1 -- 172.21.15.81:0/324156003 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd6f00aae50 con 0x7fd6f80af440 2023-11-08T16:41:32.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.599+0000 7fd6fcd08640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae00 0x7fd6dc06d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:32.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.600+0000 7fd6fcd08640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae00 0x7fd6dc06d2c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0099e0 tx=0x7fd6ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:32.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.604+0000 7fd6f57fa640 1 -- 172.21.15.81:0/324156003 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd6f0075ea0 con 0x7fd6f80af440 2023-11-08T16:41:32.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.827+0000 7fd6fef6c640 1 -- 172.21.15.81:0/324156003 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd6f813d6f0 con 0x7fd6f80af440 2023-11-08T16:41:32.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.828+0000 7fd6f57fa640 1 -- 172.21.15.81:0/324156003 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4918 (secure 0 0 0) 0x7fd6f0044020 con 0x7fd6f80af440 2023-11-08T16:41:32.831 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:32.832 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":112,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/329260749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":329260749},{"type":"v1","addr":"172.21.15.93:6835","nonce":329260749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:31.499397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8556":{"gid":8556,"name":"c","rank":0,"incarnation":110,"state":"up:rejoin","state_seq":10,"addr":"172.21.15.81:6837/296230272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":296230272},{"type":"v1","addr":"172.21.15.81:6837","nonce":296230272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:41:32.832 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 112 2023-11-08T16:41:32.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.831+0000 7fd6fef6c640 1 -- 172.21.15.81:0/324156003 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae00 msgr2=0x7fd6dc06d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:32.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.831+0000 7fd6fef6c640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae00 0x7fd6dc06d2c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec0099e0 tx=0x7fd6ec008040 comp rx=0 tx=0).stop 2023-11-08T16:41:32.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.831+0000 7fd6fef6c640 1 -- 172.21.15.81:0/324156003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f80af440 msgr2=0x7fd6f80af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:32.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.831+0000 7fd6fef6c640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f80af440 0x7fd6f80af8c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd6f000a980 tx=0x7fd6f0001d80 comp rx=0 tx=0).stop 2023-11-08T16:41:32.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.831+0000 7fd6fef6c640 1 -- 172.21.15.81:0/324156003 shutdown_connections 2023-11-08T16:41:32.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.831+0000 7fd6fef6c640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd6dc06ae00 0x7fd6dc06d2c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:32.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.831+0000 7fd6fef6c640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd6f80af440 0x7fd6f80af8c0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:32.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.831+0000 7fd6fef6c640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6f813b830 0x7fd6f813bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:32.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.831+0000 7fd6fef6c640 1 --2- 172.21.15.81:0/324156003 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd6e8004d00 0x7fd6f81599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:32.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.831+0000 7fd6fef6c640 1 -- 172.21.15.81:0/324156003 >> 172.21.15.81:0/324156003 conn(0x7fd6f806a6a0 msgr2=0x7fd6f80ae750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:32.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.831+0000 7fd6fef6c640 1 -- 172.21.15.81:0/324156003 shutdown_connections 2023-11-08T16:41:32.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:32.832+0000 7fd6fef6c640 1 -- 172.21.15.81:0/324156003 wait complete. 2023-11-08T16:41:33.254 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:41:05.700435+0000', 'modified': '2023-11-08T16:41:31.499397+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8556': {'gid': 8556, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:rejoin', 'state_seq': 10, 'addr': '172.21.15.81:6837/296230272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 296230272}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 296230272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:41:33.254 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8556:up:rejoin 2023-11-08T16:41:34.255 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:41:34.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.475+0000 7f657d35a640 1 Processor -- start 2023-11-08T16:41:34.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.475+0000 7f657d35a640 1 -- start start 2023-11-08T16:41:34.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.477+0000 7f657d35a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f657815b500 0x7f6578144230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:34.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.477+0000 7f657d35a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65780b2310 con 0x7f65780b3170 2023-11-08T16:41:34.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.477+0000 7f657d35a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65780b25f0 con 0x7f65780b3550 2023-11-08T16:41:34.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.477+0000 7f657d35a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65780b2bb0 con 0x7f657815b500 2023-11-08T16:41:34.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.477+0000 7f6576ffd640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f65780b3170 0x7f65780b1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56170/0 (socket says 172.21.15.81:56170) 2023-11-08T16:41:34.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.477+0000 7f6576ffd640 1 -- 172.21.15.81:0/1424607995 learned_addr learned my addr 172.21.15.81:0/1424607995 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:34.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.477+0000 7f65777fe640 1 --2- 172.21.15.81:0/1424607995 >> v2:172.21.15.81:3300/0 conn(0x7f657815b500 0x7f6578144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:34.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.478+0000 7f6575ffb640 1 -- 172.21.15.81:0/1424607995 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67591620 0 0) 0x7f65780b2310 con 0x7f65780b3170 2023-11-08T16:41:34.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.478+0000 7f6575ffb640 1 -- 172.21.15.81:0/1424607995 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6560003520 con 0x7f65780b3170 2023-11-08T16:41:34.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.478+0000 7f65777fe640 1 -- 172.21.15.81:0/1424607995 >> v1:172.21.15.93:6789/0 conn(0x7f65780b3550 legacy=0x7f657806c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:34.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.478+0000 7f65777fe640 1 -- 172.21.15.81:0/1424607995 >> v1:172.21.15.81:6789/0 conn(0x7f65780b3170 legacy=0x7f65780b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:34.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.478+0000 7f65777fe640 1 -- 172.21.15.81:0/1424607995 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65780b28d0 con 0x7f657815b500 2023-11-08T16:41:34.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.479+0000 7f65777fe640 1 --2- 172.21.15.81:0/1424607995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f657815b500 0x7f6578144230 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f6570009d90 tx=0x7f657002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68668e42a8a5a08 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:34.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.479+0000 7f6575ffb640 1 -- 172.21.15.81:0/1424607995 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657002cbd0 con 0x7f657815b500 2023-11-08T16:41:34.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.479+0000 7f6575ffb640 1 -- 172.21.15.81:0/1424607995 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f657002cd30 con 0x7f657815b500 2023-11-08T16:41:34.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.479+0000 7f6575ffb640 1 -- 172.21.15.81:0/1424607995 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657002cea0 con 0x7f657815b500 2023-11-08T16:41:34.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.480+0000 7f657d35a640 1 -- 172.21.15.81:0/1424607995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f657815b500 msgr2=0x7f6578144230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:34.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.480+0000 7f657d35a640 1 --2- 172.21.15.81:0/1424607995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f657815b500 0x7f6578144230 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f6570009d90 tx=0x7f657002be20 comp rx=0 tx=0).stop 2023-11-08T16:41:34.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.480+0000 7f657d35a640 1 -- 172.21.15.81:0/1424607995 shutdown_connections 2023-11-08T16:41:34.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.480+0000 7f657d35a640 1 --2- 172.21.15.81:0/1424607995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f657815b500 0x7f6578144230 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:34.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.480+0000 7f657d35a640 1 -- 172.21.15.81:0/1424607995 >> 172.21.15.81:0/1424607995 conn(0x7f657806a6a0 msgr2=0x7f65780b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:34.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.480+0000 7f657d35a640 1 -- 172.21.15.81:0/1424607995 shutdown_connections 2023-11-08T16:41:34.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.480+0000 7f657d35a640 1 -- 172.21.15.81:0/1424607995 wait complete. 2023-11-08T16:41:34.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.482+0000 7f657d35a640 1 Processor -- start 2023-11-08T16:41:34.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.482+0000 7f657d35a640 1 -- start start 2023-11-08T16:41:34.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.483+0000 7f657d35a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f65780b3170 0x7f657813b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:34.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.483+0000 7f657d35a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65780b3550 0x7f65781f0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:34.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.483+0000 7f657d35a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f657815b500 0x7f65781f0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:34.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.483+0000 7f657d35a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65781f1010 con 0x7f65780b3550 2023-11-08T16:41:34.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.483+0000 7f657d35a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65781f1890 con 0x7f657815b500 2023-11-08T16:41:34.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.483+0000 7f657d35a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65781f1b70 con 0x7f65780b3170 2023-11-08T16:41:34.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.483+0000 7f65767fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65780b3550 0x7f65781f0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:34.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.483+0000 7f65767fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65780b3550 0x7f65781f0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37496/0 (socket says 172.21.15.81:37496) 2023-11-08T16:41:34.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.483+0000 7f65767fc640 1 -- 172.21.15.81:0/1492148865 learned_addr learned my addr 172.21.15.81:0/1492148865 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:34.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.483+0000 7f65777fe640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f657815b500 0x7f65781f0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:34.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.483+0000 7f6576ffd640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f65780b3170 0x7f657813b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:34.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.484+0000 7f65767fc640 1 -- 172.21.15.81:0/1492148865 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f65780b3170 msgr2=0x7f657813b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:34.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.484+0000 7f65767fc640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f65780b3170 0x7f657813b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:34.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.484+0000 7f65767fc640 1 -- 172.21.15.81:0/1492148865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f657815b500 msgr2=0x7f65781f0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:34.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.484+0000 7f65767fc640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f657815b500 0x7f65781f0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:34.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.484+0000 7f65767fc640 1 -- 172.21.15.81:0/1492148865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6570009a70 con 0x7f65780b3550 2023-11-08T16:41:34.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.484+0000 7f65767fc640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65780b3550 0x7f65781f0590 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f656c00c430 tx=0x7f656c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:34.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.484+0000 7f655f7fe640 1 -- 172.21.15.81:0/1492148865 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c011790 con 0x7f65780b3550 2023-11-08T16:41:34.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.484+0000 7f655f7fe640 1 -- 172.21.15.81:0/1492148865 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656c0118f0 con 0x7f65780b3550 2023-11-08T16:41:34.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.484+0000 7f655f7fe640 1 -- 172.21.15.81:0/1492148865 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656c011ce0 con 0x7f65780b3550 2023-11-08T16:41:34.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.484+0000 7f657d35a640 1 -- 172.21.15.81:0/1492148865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65781f1e50 con 0x7f65780b3550 2023-11-08T16:41:34.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.485+0000 7f657d35a640 1 -- 172.21.15.81:0/1492148865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65781f2410 con 0x7f65780b3550 2023-11-08T16:41:34.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.486+0000 7f657d35a640 1 -- 172.21.15.81:0/1492148865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6578074440 con 0x7f65780b3550 2023-11-08T16:41:34.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.486+0000 7f655f7fe640 1 -- 172.21.15.81:0/1492148865 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f656c01c460 con 0x7f65780b3550 2023-11-08T16:41:34.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.487+0000 7f655f7fe640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f656406ae70 0x7f656406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:34.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.487+0000 7f655f7fe640 1 -- 172.21.15.81:0/1492148865 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f656c089580 con 0x7f65780b3550 2023-11-08T16:41:34.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.488+0000 7f6576ffd640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f656406ae70 0x7f656406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:34.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.488+0000 7f6576ffd640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f656406ae70 0x7f656406d330 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f65680011e0 tx=0x7f6568009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:34.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.492+0000 7f655f7fe640 1 -- 172.21.15.81:0/1492148865 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f656c054660 con 0x7f65780b3550 2023-11-08T16:41:34.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.721+0000 7f657d35a640 1 -- 172.21.15.81:0/1492148865 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f657806df50 con 0x7f65780b3550 2023-11-08T16:41:34.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.723+0000 7f655f7fe640 1 -- 172.21.15.81:0/1492148865 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4918 (secure 0 0 0) 0x7f656c057c80 con 0x7f65780b3550 2023-11-08T16:41:34.725 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:34.726 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":113,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/329260749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":329260749},{"type":"v1","addr":"172.21.15.93:6835","nonce":329260749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:32.502912+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":128,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8556},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8556":{"gid":8556,"name":"c","rank":0,"incarnation":110,"state":"up:active","state_seq":11,"addr":"172.21.15.81:6837/296230272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":296230272},{"type":"v1","addr":"172.21.15.81:6837","nonce":296230272}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:41:34.726 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 113 2023-11-08T16:41:34.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.725+0000 7f657d35a640 1 -- 172.21.15.81:0/1492148865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f656406ae70 msgr2=0x7f656406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:34.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.725+0000 7f657d35a640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f656406ae70 0x7f656406d330 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f65680011e0 tx=0x7f6568009040 comp rx=0 tx=0).stop 2023-11-08T16:41:34.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.725+0000 7f657d35a640 1 -- 172.21.15.81:0/1492148865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65780b3550 msgr2=0x7f65781f0590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:34.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.725+0000 7f657d35a640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65780b3550 0x7f65781f0590 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f656c00c430 tx=0x7f656c00c8f0 comp rx=0 tx=0).stop 2023-11-08T16:41:34.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.726+0000 7f657d35a640 1 -- 172.21.15.81:0/1492148865 shutdown_connections 2023-11-08T16:41:34.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.726+0000 7f657d35a640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f656406ae70 0x7f656406d330 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:34.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.726+0000 7f657d35a640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f657815b500 0x7f65781f0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:34.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.726+0000 7f657d35a640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f65780b3550 0x7f65781f0590 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:34.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.726+0000 7f657d35a640 1 --2- 172.21.15.81:0/1492148865 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f65780b3170 0x7f657813b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:34.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.726+0000 7f657d35a640 1 -- 172.21.15.81:0/1492148865 >> 172.21.15.81:0/1492148865 conn(0x7f657806a6a0 msgr2=0x7f65780cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:34.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.726+0000 7f657d35a640 1 -- 172.21.15.81:0/1492148865 shutdown_connections 2023-11-08T16:41:34.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:34.726+0000 7f657d35a640 1 -- 172.21.15.81:0/1492148865 wait complete. 2023-11-08T16:41:35.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:41:05.700435+0000', 'modified': '2023-11-08T16:41:32.502912+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 128, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8556}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8556': {'gid': 8556, 'name': 'c', 'rank': 0, 'incarnation': 110, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.81:6837/296230272', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 296230272}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 296230272}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:41:35.152 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:41:35.153 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T16:41:35.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.375+0000 7f2b948fc640 1 Processor -- start 2023-11-08T16:41:35.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.376+0000 7f2b948fc640 1 -- start start 2023-11-08T16:41:35.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.377+0000 7f2b948fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2b900cd3c0 0x7f2b900cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.377+0000 7f2b948fc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b900cbb10 0x7f2b9015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.377+0000 7f2b948fc640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b90071620 con 0x7f2b90158aa0 2023-11-08T16:41:35.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.377+0000 7f2b948fc640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b900cbfa0 con 0x7f2b900cd3c0 2023-11-08T16:41:35.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.377+0000 7f2b948fc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b90158e80 con 0x7f2b900cbb10 2023-11-08T16:41:35.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.378+0000 7f2b8e59c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f2b90158aa0 0x7f2b9006fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56178/0 (socket says 172.21.15.81:56178) 2023-11-08T16:41:35.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.378+0000 7f2b8e59c640 1 -- 172.21.15.81:0/4223907178 learned_addr learned my addr 172.21.15.81:0/4223907178 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:35.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.378+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/4223907178 >> v2:172.21.15.81:3300/0 conn(0x7f2b900cd3c0 0x7f2b900cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.378+0000 7f2b8ed9d640 1 --2- 172.21.15.81:0/4223907178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b900cbb10 0x7f2b9015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.378+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/4223907178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b900cbb10 msgr2=0x7f2b9015ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:35.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.378+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/4223907178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b900cbb10 0x7f2b9015ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.378+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/4223907178 >> v1:172.21.15.81:6789/0 conn(0x7f2b90158aa0 legacy=0x7f2b9006fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.378+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/4223907178 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b90071850 con 0x7f2b900cd3c0 2023-11-08T16:41:35.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.378+0000 7f2b8ed9d640 1 --2- 172.21.15.81:0/4223907178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b900cbb10 0x7f2b9015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:35.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.379+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/4223907178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b900cd3c0 0x7f2b900cb5d0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f2b780097a0 tx=0x7f2b7802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a7da32132f0b74 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:35.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.380+0000 7f2b8d59a640 1 -- 172.21.15.81:0/4223907178 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7802d040 con 0x7f2b900cd3c0 2023-11-08T16:41:35.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.380+0000 7f2b8d59a640 1 -- 172.21.15.81:0/4223907178 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78031810 con 0x7f2b900cd3c0 2023-11-08T16:41:35.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.380+0000 7f2b8d59a640 1 -- 172.21.15.81:0/4223907178 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78031ba0 con 0x7f2b900cd3c0 2023-11-08T16:41:35.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.380+0000 7f2b948fc640 1 -- 172.21.15.81:0/4223907178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b900cd3c0 msgr2=0x7f2b900cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.380+0000 7f2b948fc640 1 --2- 172.21.15.81:0/4223907178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b900cd3c0 0x7f2b900cb5d0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f2b780097a0 tx=0x7f2b7802b8b0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.380+0000 7f2b948fc640 1 -- 172.21.15.81:0/4223907178 shutdown_connections 2023-11-08T16:41:35.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.380+0000 7f2b948fc640 1 --2- 172.21.15.81:0/4223907178 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b900cbb10 0x7f2b9015ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.380+0000 7f2b948fc640 1 --2- 172.21.15.81:0/4223907178 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b900cd3c0 0x7f2b900cb5d0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.380+0000 7f2b948fc640 1 -- 172.21.15.81:0/4223907178 >> 172.21.15.81:0/4223907178 conn(0x7f2b900b0770 msgr2=0x7f2b900b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:35.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.380+0000 7f2b948fc640 1 -- 172.21.15.81:0/4223907178 shutdown_connections 2023-11-08T16:41:35.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.380+0000 7f2b948fc640 1 -- 172.21.15.81:0/4223907178 wait complete. 2023-11-08T16:41:35.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.382+0000 7f2b948fc640 1 Processor -- start 2023-11-08T16:41:35.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.382+0000 7f2b948fc640 1 -- start start 2023-11-08T16:41:35.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.383+0000 7f2b948fc640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b900cbb10 0x7f2b90149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.383+0000 7f2b948fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b900cd3c0 0x7f2b900c5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.383+0000 7f2b948fc640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b90158aa0 0x7f2b900c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.383+0000 7f2b948fc640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900c9730 con 0x7f2b900cd3c0 2023-11-08T16:41:35.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.383+0000 7f2b948fc640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900c9a10 con 0x7f2b90158aa0 2023-11-08T16:41:35.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.383+0000 7f2b948fc640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b900c9cf0 con 0x7f2b900cbb10 2023-11-08T16:41:35.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b900cd3c0 0x7f2b900c5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b900cd3c0 0x7f2b900c5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37524/0 (socket says 172.21.15.81:37524) 2023-11-08T16:41:35.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/4005843659 learned_addr learned my addr 172.21.15.81:0/4005843659 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:35.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8e59c640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b900cbb10 0x7f2b90149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8ed9d640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b90158aa0 0x7f2b900c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/4005843659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b900cbb10 msgr2=0x7f2b90149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b900cbb10 0x7f2b90149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/4005843659 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b90158aa0 msgr2=0x7f2b900c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b90158aa0 0x7f2b900c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/4005843659 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b78009400 con 0x7f2b900cd3c0 2023-11-08T16:41:35.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8ed9d640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b90158aa0 0x7f2b900c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:35.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.384+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b900cd3c0 0x7f2b900c5cb0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f2b780098c0 tx=0x7f2b780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:35.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.385+0000 7f2b86ffd640 1 -- 172.21.15.81:0/4005843659 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7803ad60 con 0x7f2b900cd3c0 2023-11-08T16:41:35.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.385+0000 7f2b86ffd640 1 -- 172.21.15.81:0/4005843659 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78044400 con 0x7f2b900cd3c0 2023-11-08T16:41:35.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.385+0000 7f2b86ffd640 1 -- 172.21.15.81:0/4005843659 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b780447b0 con 0x7f2b900cd3c0 2023-11-08T16:41:35.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.385+0000 7f2b948fc640 1 -- 172.21.15.81:0/4005843659 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b900c9fd0 con 0x7f2b900cd3c0 2023-11-08T16:41:35.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.385+0000 7f2b948fc640 1 -- 172.21.15.81:0/4005843659 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b900c9400 con 0x7f2b900cd3c0 2023-11-08T16:41:35.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.388+0000 7f2b86ffd640 1 -- 172.21.15.81:0/4005843659 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b7804d440 con 0x7f2b900cd3c0 2023-11-08T16:41:35.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.389+0000 7f2b86ffd640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b7006aea0 0x7f2b7006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.389+0000 7f2b86ffd640 1 -- 172.21.15.81:0/4005843659 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2b780abb30 con 0x7f2b900cd3c0 2023-11-08T16:41:35.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.389+0000 7f2b8e59c640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b7006aea0 0x7f2b7006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.389+0000 7f2b97566740 1 Processor -- start 2023-11-08T16:41:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.389+0000 7f2b8e59c640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b7006aea0 0x7f2b7006d360 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2b80005880 tx=0x7f2b80009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.390+0000 7f2b97566740 1 -- start start 2023-11-08T16:41:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.390+0000 7f2b97566740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce86ec0 0x55f7fce59dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.390+0000 7f2b97566740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce89c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.390+0000 7f2b97566740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce5a300 0x55f7fce8a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.390+0000 7f2b97566740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcdcc7e0 con 0x55f7fce872a0 2023-11-08T16:41:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.390+0000 7f2b97566740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcdc9ed0 con 0x55f7fce86ec0 2023-11-08T16:41:35.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.390+0000 7f2b97566740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcdc57c0 con 0x55f7fce5a300 2023-11-08T16:41:35.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.391+0000 7f2b8dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce89c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.391+0000 7f2b8dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce89c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37526/0 (socket says 172.21.15.81:37526) 2023-11-08T16:41:35.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.391+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/3790214120 learned_addr learned my addr 172.21.15.81:0/3790214120 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:35.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.391+0000 7f2b8e59c640 1 --2- 172.21.15.81:0/3790214120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce86ec0 0x55f7fce59dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.391+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/3790214120 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce5a300 msgr2=0x55f7fce8a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:35.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.391+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/3790214120 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce5a300 0x55f7fce8a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.391+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/3790214120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce86ec0 msgr2=0x55f7fce59dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:35.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.391+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/3790214120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce86ec0 0x55f7fce59dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.391+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/3790214120 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7fcd98b10 con 0x55f7fce872a0 2023-11-08T16:41:35.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.391+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/3790214120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce89c50 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f2b78076d40 tx=0x7f2b7807a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f444f64965fa9a71 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:35.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.392+0000 7f2b6ffff640 1 -- 172.21.15.81:0/3790214120 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78082b40 con 0x55f7fce872a0 2023-11-08T16:41:35.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.392+0000 7f2b6ffff640 1 -- 172.21.15.81:0/3790214120 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78078be0 con 0x55f7fce872a0 2023-11-08T16:41:35.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.392+0000 7f2b6ffff640 1 -- 172.21.15.81:0/3790214120 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78076f50 con 0x55f7fce872a0 2023-11-08T16:41:35.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.392+0000 7f2b97566740 1 -- 172.21.15.81:0/3790214120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 msgr2=0x55f7fce89c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.392+0000 7f2b97566740 1 --2- 172.21.15.81:0/3790214120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce89c50 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f2b78076d40 tx=0x7f2b7807a4b0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.392+0000 7f2b97566740 1 -- 172.21.15.81:0/3790214120 shutdown_connections 2023-11-08T16:41:35.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.392+0000 7f2b97566740 1 --2- 172.21.15.81:0/3790214120 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce5a300 0x55f7fce8a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.392+0000 7f2b97566740 1 --2- 172.21.15.81:0/3790214120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce89c50 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.392+0000 7f2b97566740 1 --2- 172.21.15.81:0/3790214120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce86ec0 0x55f7fce59dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.392+0000 7f2b97566740 1 -- 172.21.15.81:0/3790214120 >> 172.21.15.81:0/3790214120 conn(0x55f7fce5c0d0 msgr2=0x55f7fcea4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:35.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.393+0000 7f2b97566740 1 -- 172.21.15.81:0/3790214120 shutdown_connections 2023-11-08T16:41:35.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.393+0000 7f2b97566740 1 -- 172.21.15.81:0/3790214120 wait complete. 2023-11-08T16:41:35.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.393+0000 7f2b97566740 1 -- start start 2023-11-08T16:41:35.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.393+0000 7f2b97566740 1 Processor -- start 2023-11-08T16:41:35.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.394+0000 7f2b97566740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce86ec0 0x55f7fce7b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.394+0000 7f2b97566740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce9e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.394+0000 7f2b97566740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fceb49e0 0x55f7fceb4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.394+0000 7f2b97566740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcd62c40 con 0x55f7fce872a0 2023-11-08T16:41:35.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.394+0000 7f2b97566740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcd628e0 con 0x55f7fceb49e0 2023-11-08T16:41:35.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.394+0000 7f2b97566740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcb28f20 con 0x55f7fce86ec0 2023-11-08T16:41:35.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b97566740 10 client.0 _ll_register_callbacks cb 0x55f7fcd84cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:41:35.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b8dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce9e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b8dd9b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce9e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37542/0 (socket says 172.21.15.81:37542) 2023-11-08T16:41:35.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/2823284580 learned_addr learned my addr 172.21.15.81:0/2823284580 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:35.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b8e59c640 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce86ec0 0x55f7fce7b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/2823284580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce86ec0 msgr2=0x55f7fce7b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:35.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce86ec0 0x55f7fce7b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/2823284580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fceb49e0 msgr2=0x55f7fceb4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:35.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fceb49e0 0x55f7fceb4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b8dd9b640 1 -- 172.21.15.81:0/2823284580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b900c9400 con 0x55f7fce872a0 2023-11-08T16:41:35.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce9e050 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f2b78031fd0 tx=0x7f2b78045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d93954ff2126f4c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:35.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.395+0000 7f2b6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:41:35.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2823284580 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b780a7070 con 0x55f7fce872a0 2023-11-08T16:41:35.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2823284580 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2b7802f4c0 con 0x55f7fce872a0 2023-11-08T16:41:35.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b97566740 1 -- 172.21.15.81:0/2823284580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7fcd99c80 con 0x55f7fce872a0 2023-11-08T16:41:35.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2823284580 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b780991f0 con 0x55f7fce872a0 2023-11-08T16:41:35.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b97566740 10 client.8874 fetch_fsmap learned FSMap version 113 2023-11-08T16:41:35.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b97566740 1 -- 172.21.15.81:0/2823284580 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55f7fcdc3e40 con 0x55f7fce872a0 2023-11-08T16:41:35.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2823284580 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f7fcd99c80 con 0x55f7fce872a0 2023-11-08T16:41:35.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2823284580 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b7804b060 con 0x55f7fce872a0 2023-11-08T16:41:35.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b97566740 10 client.8874 fetch_fsmap finished waiting for FSMap version 113 2023-11-08T16:41:35.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b97566740 10 client.8874 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:41:35.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b97566740 20 client.8874 populate_metadata read hostname 'smithi081' 2023-11-08T16:41:35.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b97566740 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.81:6836/296230272,v1:172.21.15.81:6837/296230272] conn(0x55f7fce8c350 0x55f7fcec2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b97566740 4 client.8874 mds_command: new command op to 8556 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:41:35.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.396+0000 7f2b97566740 1 -- 172.21.15.81:0/2823284580 --> [v2:172.21.15.81:6836/296230272,v1:172.21.15.81:6837/296230272] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7fcdbe7a0 con 0x55f7fce8c350 2023-11-08T16:41:35.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.397+0000 7f2b8e59c640 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.81:6836/296230272,v1:172.21.15.81:6837/296230272] conn(0x55f7fce8c350 0x55f7fcec2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.397+0000 7f2b8e59c640 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.81:6836/296230272,v1:172.21.15.81:6837/296230272] conn(0x55f7fce8c350 0x55f7fcec2150 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=825a73d1776c0050 server_cookie=b144f4cc0735e9fe in_seq=0 out_seq=0 2023-11-08T16:41:35.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.397+0000 7f2b6d7fa640 10 client.8874 ms_handle_connect on v2:172.21.15.81:6836/296230272 2023-11-08T16:41:35.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2823284580 <== mds.0 v2:172.21.15.81:6836/296230272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f7fcdbe7a0 con 0x55f7fce8c350 2023-11-08T16:41:35.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b6d7fa640 10 client.8874 handle_command_reply: tid=0 2023-11-08T16:41:35.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b97566740 1 client.8874 shutdown 2023-11-08T16:41:35.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b97566740 1 -- 172.21.15.81:0/2823284580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 msgr2=0x55f7fce9e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b97566740 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce9e050 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f2b78031fd0 tx=0x7f2b78045000 comp rx=0 tx=0).stop 2023-11-08T16:41:35.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b97566740 1 -- 172.21.15.81:0/2823284580 shutdown_connections 2023-11-08T16:41:35.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b97566740 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.81:6836/296230272,v1:172.21.15.81:6837/296230272] conn(0x55f7fce8c350 0x55f7fcec2150 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b97566740 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fceb49e0 0x55f7fceb4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b97566740 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fce872a0 0x55f7fce9e050 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b97566740 1 --2- 172.21.15.81:0/2823284580 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce86ec0 0x55f7fce7b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b97566740 1 -- 172.21.15.81:0/2823284580 >> 172.21.15.81:0/2823284580 conn(0x55f7fce5a300 msgr2=0x55f7fce5de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:35.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b6d7fa640 0 client.8874 ms_handle_reset on v2:172.21.15.81:6836/296230272 2023-11-08T16:41:35.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.398+0000 7f2b97566740 1 -- 172.21.15.81:0/2823284580 shutdown_connections 2023-11-08T16:41:35.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.399+0000 7f2b97566740 1 -- 172.21.15.81:0/2823284580 wait complete. 2023-11-08T16:41:35.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.399+0000 7f2b97566740 20 client.8874 trim_cache size 0 max 16384 2023-11-08T16:41:35.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.416+0000 7f2b97566740 1 Processor -- start 2023-11-08T16:41:35.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.416+0000 7f2b97566740 1 -- start start 2023-11-08T16:41:35.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.416+0000 7f2b97566740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce86ec0 0x55f7fce5ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b97566740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce872a0 0x55f7fce85900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b97566740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcec64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b97566740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcea06d0 con 0x55f7fceb49e0 2023-11-08T16:41:35.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b97566740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fce8e200 con 0x55f7fce872a0 2023-11-08T16:41:35.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b97566740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcdc3e40 con 0x55f7fce86ec0 2023-11-08T16:41:35.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b8ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcec64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b8ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcec64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37554/0 (socket says 172.21.15.81:37554) 2023-11-08T16:41:35.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b8ed9d640 1 -- 172.21.15.81:0/875122177 learned_addr learned my addr 172.21.15.81:0/875122177 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:35.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b8e59c640 1 --2- 172.21.15.81:0/875122177 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce86ec0 0x55f7fce5ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b8ed9d640 1 -- 172.21.15.81:0/875122177 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce86ec0 msgr2=0x55f7fce5ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:35.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b8ed9d640 1 --2- 172.21.15.81:0/875122177 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce86ec0 0x55f7fce5ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b8ed9d640 1 -- 172.21.15.81:0/875122177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce872a0 msgr2=0x55f7fce85900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:35.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b8dd9b640 1 --2- 172.21.15.81:0/875122177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce872a0 0x55f7fce85900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b8ed9d640 1 --2- 172.21.15.81:0/875122177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce872a0 0x55f7fce85900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.417+0000 7f2b8ed9d640 1 -- 172.21.15.81:0/875122177 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7fcd61b90 con 0x55f7fceb49e0 2023-11-08T16:41:35.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.418+0000 7f2b8ed9d640 1 --2- 172.21.15.81:0/875122177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcec64c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f2b88007db0 tx=0x7f2b8800d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d26a38712f5de8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:35.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.418+0000 7f2b84ff9640 1 -- 172.21.15.81:0/875122177 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8800f860 con 0x55f7fceb49e0 2023-11-08T16:41:35.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.418+0000 7f2b84ff9640 1 -- 172.21.15.81:0/875122177 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8800f9c0 con 0x55f7fceb49e0 2023-11-08T16:41:35.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.418+0000 7f2b84ff9640 1 -- 172.21.15.81:0/875122177 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b8800fd70 con 0x55f7fceb49e0 2023-11-08T16:41:35.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.419+0000 7f2b97566740 1 -- 172.21.15.81:0/875122177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 msgr2=0x55f7fcec64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.419+0000 7f2b97566740 1 --2- 172.21.15.81:0/875122177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcec64c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f2b88007db0 tx=0x7f2b8800d740 comp rx=0 tx=0).stop 2023-11-08T16:41:35.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.419+0000 7f2b97566740 1 -- 172.21.15.81:0/875122177 shutdown_connections 2023-11-08T16:41:35.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.419+0000 7f2b97566740 1 --2- 172.21.15.81:0/875122177 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcec64c0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.419+0000 7f2b97566740 1 --2- 172.21.15.81:0/875122177 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce872a0 0x55f7fce85900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.419+0000 7f2b97566740 1 --2- 172.21.15.81:0/875122177 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce86ec0 0x55f7fce5ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.419+0000 7f2b97566740 1 -- 172.21.15.81:0/875122177 >> 172.21.15.81:0/875122177 conn(0x55f7fce5a300 msgr2=0x55f7fceb41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:35.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.419+0000 7f2b97566740 1 -- 172.21.15.81:0/875122177 shutdown_connections 2023-11-08T16:41:35.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.419+0000 7f2b97566740 1 -- 172.21.15.81:0/875122177 wait complete. 2023-11-08T16:41:35.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.419+0000 7f2b97566740 1 -- start start 2023-11-08T16:41:35.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.420+0000 7f2b97566740 1 Processor -- start 2023-11-08T16:41:35.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b97566740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce86ec0 0x55f7fce86310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b97566740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce872a0 0x55f7fcf0e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b97566740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcea5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b97566740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fce9e420 con 0x55f7fceb49e0 2023-11-08T16:41:35.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b97566740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7fcf1b8d0 con 0x55f7fce86ec0 2023-11-08T16:41:35.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b97566740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7fce7ab60 con 0x55f7fce872a0 2023-11-08T16:41:35.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b97566740 10 client.0 _ll_register_callbacks cb 0x55f7fcea7940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:41:35.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b8ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcea5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b8ed9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcea5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37566/0 (socket says 172.21.15.81:37566) 2023-11-08T16:41:35.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b8ed9d640 1 -- 172.21.15.81:0/2149401906 learned_addr learned my addr 172.21.15.81:0/2149401906 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:35.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b8ed9d640 1 -- 172.21.15.81:0/2149401906 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce872a0 msgr2=0x55f7fcf0e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:35.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b8ed9d640 1 --2- 172.21.15.81:0/2149401906 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce872a0 0x55f7fcf0e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b8ed9d640 1 -- 172.21.15.81:0/2149401906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce86ec0 msgr2=0x55f7fce86310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:35.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b8ed9d640 1 --2- 172.21.15.81:0/2149401906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce86ec0 0x55f7fce86310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.421+0000 7f2b8ed9d640 1 -- 172.21.15.81:0/2149401906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b88007a30 con 0x55f7fceb49e0 2023-11-08T16:41:35.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b8ed9d640 1 --2- 172.21.15.81:0/2149401906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcea5020 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f2b88004af0 tx=0x7f2b88010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3696e2797d825eea server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:35.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b6d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:41:35.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2149401906 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88018680 con 0x55f7fceb49e0 2023-11-08T16:41:35.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2149401906 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f2b88013070 con 0x55f7fceb49e0 2023-11-08T16:41:35.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b97566740 1 -- 172.21.15.81:0/2149401906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b78078be0 con 0x55f7fceb49e0 2023-11-08T16:41:35.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2149401906 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b8800de50 con 0x55f7fceb49e0 2023-11-08T16:41:35.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b97566740 10 client.8880 fetch_fsmap learned FSMap version 113 2023-11-08T16:41:35.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b97566740 1 -- 172.21.15.81:0/2149401906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55f7fcd87180 con 0x55f7fceb49e0 2023-11-08T16:41:35.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2149401906 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b78078be0 con 0x55f7fceb49e0 2023-11-08T16:41:35.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2149401906 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b88021070 con 0x55f7fceb49e0 2023-11-08T16:41:35.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b97566740 10 client.8880 fetch_fsmap finished waiting for FSMap version 113 2023-11-08T16:41:35.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b97566740 10 client.8880 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:41:35.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b97566740 20 client.8880 populate_metadata read hostname 'smithi081' 2023-11-08T16:41:35.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.422+0000 7f2b97566740 1 --2- 172.21.15.81:0/2149401906 >> [v2:172.21.15.81:6836/296230272,v1:172.21.15.81:6837/296230272] conn(0x55f7fcf18130 0x55f7fcf18510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.423+0000 7f2b97566740 4 client.8880 mds_command: new command op to 8556 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:41:35.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.423+0000 7f2b97566740 1 -- 172.21.15.81:0/2149401906 --> [v2:172.21.15.81:6836/296230272,v1:172.21.15.81:6837/296230272] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f7fcdbe4c0 con 0x55f7fcf18130 2023-11-08T16:41:35.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.423+0000 7f2b8e59c640 1 --2- 172.21.15.81:0/2149401906 >> [v2:172.21.15.81:6836/296230272,v1:172.21.15.81:6837/296230272] conn(0x55f7fcf18130 0x55f7fcf18510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.423+0000 7f2b8e59c640 1 --2- 172.21.15.81:0/2149401906 >> [v2:172.21.15.81:6836/296230272,v1:172.21.15.81:6837/296230272] conn(0x55f7fcf18130 0x55f7fcf18510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4412fbeb649f46c server_cookie=4a467e370af7f3af in_seq=0 out_seq=0 2023-11-08T16:41:35.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.423+0000 7f2b6d7fa640 10 client.8880 ms_handle_connect on v2:172.21.15.81:6836/296230272 2023-11-08T16:41:35.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b6d7fa640 1 -- 172.21.15.81:0/2149401906 <== mds.0 v2:172.21.15.81:6836/296230272 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55f7fcdbe4c0 con 0x55f7fcf18130 2023-11-08T16:41:35.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b6d7fa640 10 client.8880 handle_command_reply: tid=0 2023-11-08T16:41:35.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 1 client.8880 shutdown 2023-11-08T16:41:35.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 1 -- 172.21.15.81:0/2149401906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 msgr2=0x55f7fcea5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 1 --2- 172.21.15.81:0/2149401906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcea5020 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f2b88004af0 tx=0x7f2b88010000 comp rx=0 tx=0).stop 2023-11-08T16:41:35.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 1 -- 172.21.15.81:0/2149401906 shutdown_connections 2023-11-08T16:41:35.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 1 --2- 172.21.15.81:0/2149401906 >> [v2:172.21.15.81:6836/296230272,v1:172.21.15.81:6837/296230272] conn(0x55f7fcf18130 0x55f7fcf18510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 1 --2- 172.21.15.81:0/2149401906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55f7fceb49e0 0x55f7fcea5020 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 1 --2- 172.21.15.81:0/2149401906 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55f7fce872a0 0x55f7fcf0e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 1 --2- 172.21.15.81:0/2149401906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f7fce86ec0 0x55f7fce86310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b6d7fa640 0 client.8880 ms_handle_reset on v2:172.21.15.81:6836/296230272 2023-11-08T16:41:35.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 1 -- 172.21.15.81:0/2149401906 >> 172.21.15.81:0/2149401906 conn(0x55f7fce5a300 msgr2=0x55f7fce9f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:35.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 1 -- 172.21.15.81:0/2149401906 shutdown_connections 2023-11-08T16:41:35.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 1 -- 172.21.15.81:0/2149401906 wait complete. 2023-11-08T16:41:35.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.424+0000 7f2b97566740 20 client.8880 trim_cache size 0 max 16384 2023-11-08T16:41:35.440 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:41:35.440 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:41:35.441 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:41:35.441 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 8556, 2023-11-08T16:41:35.441 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:41:35.441 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:41:35.441 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:41:35.442 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 4.939224726, 2023-11-08T16:41:35.442 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 113, 2023-11-08T16:41:35.442 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 128, 2023-11-08T16:41:35.442 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 128, 2023-11-08T16:41:35.442 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 35.568350410999997 2023-11-08T16:41:35.443 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:41:35.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 -- 172.21.15.81:0/4005843659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b7006aea0 msgr2=0x7f2b7006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b7006aea0 0x7f2b7006d360 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f2b80005880 tx=0x7f2b80009040 comp rx=0 tx=0).stop 2023-11-08T16:41:35.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 -- 172.21.15.81:0/4005843659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b900cd3c0 msgr2=0x7f2b900c5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b900cd3c0 0x7f2b900c5cb0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f2b780098c0 tx=0x7f2b780046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 -- 172.21.15.81:0/4005843659 shutdown_connections 2023-11-08T16:41:35.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b7006aea0 0x7f2b7006d360 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b90158aa0 0x7f2b900c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b900cd3c0 0x7f2b900c5cb0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 --2- 172.21.15.81:0/4005843659 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b900cbb10 0x7f2b90149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 -- 172.21.15.81:0/4005843659 >> 172.21.15.81:0/4005843659 conn(0x7f2b900b0770 msgr2=0x7f2b9015d840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:35.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 -- 172.21.15.81:0/4005843659 shutdown_connections 2023-11-08T16:41:35.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.425+0000 7f2b948fc640 1 -- 172.21.15.81:0/4005843659 wait complete. 2023-11-08T16:41:35.447 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:41:35.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.654+0000 7f330590f640 1 Processor -- start 2023-11-08T16:41:35.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.654+0000 7f330590f640 1 -- start start 2023-11-08T16:41:35.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.656+0000 7f330590f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33000b3170 0x7f330015b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.656+0000 7f330590f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f33000b3550 0x7f33000b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.656+0000 7f330590f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33000b2310 con 0x7f330006c9f0 2023-11-08T16:41:35.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.656+0000 7f330590f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33000b2bb0 con 0x7f33000b3550 2023-11-08T16:41:35.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.656+0000 7f330590f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33000b25f0 con 0x7f33000b3170 2023-11-08T16:41:35.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.656+0000 7f32fe7fc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f33000b3550 0x7f33000b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.656+0000 7f32feffd640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33000b3170 0x7f330015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.657+0000 7f32fe7fc640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f33000b3550 0x7f33000b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:57102/0 (socket says 172.21.15.81:57102) 2023-11-08T16:41:35.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.657+0000 7f32fe7fc640 1 -- 172.21.15.81:0/1335881803 learned_addr learned my addr 172.21.15.81:0/1335881803 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:35.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.657+0000 7f32fe7fc640 1 -- 172.21.15.81:0/1335881803 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33000b3170 msgr2=0x7f330015b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:35.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.657+0000 7f32fe7fc640 1 --2- 172.21.15.81:0/1335881803 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33000b3170 0x7f330015b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.657+0000 7f32fe7fc640 1 -- 172.21.15.81:0/1335881803 >> v1:172.21.15.93:6789/0 conn(0x7f330006c9f0 legacy=0x7f3300144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.657+0000 7f32fe7fc640 1 -- 172.21.15.81:0/1335881803 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33000b28d0 con 0x7f33000b3550 2023-11-08T16:41:35.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.657+0000 7f32feffd640 1 --2- 172.21.15.81:0/1335881803 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33000b3170 0x7f330015b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:41:35.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.659+0000 7f32fe7fc640 1 --2- 172.21.15.81:0/1335881803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33000b3550 0x7f33000b1ae0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009870 tx=0x7f32f402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3417744f8f4d0636 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:35.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.659+0000 7f32fdffb640 1 -- 172.21.15.81:0/1335881803 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4030840 con 0x7f33000b3550 2023-11-08T16:41:35.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.659+0000 7f32fdffb640 1 -- 172.21.15.81:0/1335881803 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f40309a0 con 0x7f33000b3550 2023-11-08T16:41:35.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.659+0000 7f32fdffb640 1 -- 172.21.15.81:0/1335881803 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4030d30 con 0x7f33000b3550 2023-11-08T16:41:35.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.660+0000 7f330590f640 1 -- 172.21.15.81:0/1335881803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33000b3550 msgr2=0x7f33000b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.660+0000 7f330590f640 1 --2- 172.21.15.81:0/1335881803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33000b3550 0x7f33000b1ae0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009870 tx=0x7f32f402b980 comp rx=0 tx=0).stop 2023-11-08T16:41:35.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.660+0000 7f330590f640 1 -- 172.21.15.81:0/1335881803 shutdown_connections 2023-11-08T16:41:35.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.660+0000 7f330590f640 1 --2- 172.21.15.81:0/1335881803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33000b3550 0x7f33000b1ae0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.660+0000 7f330590f640 1 --2- 172.21.15.81:0/1335881803 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f33000b3170 0x7f330015b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.660+0000 7f330590f640 1 -- 172.21.15.81:0/1335881803 >> 172.21.15.81:0/1335881803 conn(0x7f330006a6a0 msgr2=0x7f33000b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:35.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.660+0000 7f330590f640 1 -- 172.21.15.81:0/1335881803 shutdown_connections 2023-11-08T16:41:35.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.660+0000 7f330590f640 1 -- 172.21.15.81:0/1335881803 wait complete. 2023-11-08T16:41:35.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.662+0000 7f330590f640 1 Processor -- start 2023-11-08T16:41:35.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.662+0000 7f330590f640 1 -- start start 2023-11-08T16:41:35.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.664+0000 7f330590f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f330006c9f0 0x7f33001f3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.664+0000 7f330590f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33000b3170 0x7f33001e5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.664+0000 7f330590f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33000b3550 0x7f33001f3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.664+0000 7f330590f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33001e6340 con 0x7f33000b3170 2023-11-08T16:41:35.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.664+0000 7f330590f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33001e6620 con 0x7f33000b3550 2023-11-08T16:41:35.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.664+0000 7f330590f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33001e6900 con 0x7f330006c9f0 2023-11-08T16:41:35.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.664+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33000b3170 0x7f33001e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.664+0000 7f32fe7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33000b3170 0x7f33001e5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37580/0 (socket says 172.21.15.81:37580) 2023-11-08T16:41:35.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.664+0000 7f32fe7fc640 1 -- 172.21.15.81:0/794244706 learned_addr learned my addr 172.21.15.81:0/794244706 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:35.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.664+0000 7f32feffd640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f330006c9f0 0x7f33001f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.664+0000 7f32ff7fe640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33000b3550 0x7f33001f3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.665+0000 7f32fe7fc640 1 -- 172.21.15.81:0/794244706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f330006c9f0 msgr2=0x7f33001f3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.665+0000 7f32fe7fc640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f330006c9f0 0x7f33001f3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.665+0000 7f32fe7fc640 1 -- 172.21.15.81:0/794244706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33000b3550 msgr2=0x7f33001f3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.665+0000 7f32fe7fc640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33000b3550 0x7f33001f3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.665+0000 7f32fe7fc640 1 -- 172.21.15.81:0/794244706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f0009400 con 0x7f33000b3170 2023-11-08T16:41:35.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.665+0000 7f32fe7fc640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33000b3170 0x7f33001e5980 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f32f402b9c0 tx=0x7f32f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:35.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.665+0000 7f32e7fff640 1 -- 172.21.15.81:0/794244706 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f403e070 con 0x7f33000b3170 2023-11-08T16:41:35.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.665+0000 7f32e7fff640 1 -- 172.21.15.81:0/794244706 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f4039a10 con 0x7f33000b3170 2023-11-08T16:41:35.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.665+0000 7f32e7fff640 1 -- 172.21.15.81:0/794244706 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4039de0 con 0x7f33000b3170 2023-11-08T16:41:35.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.665+0000 7f330590f640 1 -- 172.21.15.81:0/794244706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f40094d0 con 0x7f33000b3170 2023-11-08T16:41:35.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.666+0000 7f330590f640 1 -- 172.21.15.81:0/794244706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33001e6ec0 con 0x7f33000b3170 2023-11-08T16:41:35.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.667+0000 7f330590f640 1 -- 172.21.15.81:0/794244706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3300074440 con 0x7f33000b3170 2023-11-08T16:41:35.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.668+0000 7f32e7fff640 1 -- 172.21.15.81:0/794244706 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32f404b400 con 0x7f33000b3170 2023-11-08T16:41:35.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.668+0000 7f32e7fff640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ec06aea0 0x7f32ec06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:35.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.669+0000 7f32e7fff640 1 -- 172.21.15.81:0/794244706 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f32f40abc40 con 0x7f33000b3170 2023-11-08T16:41:35.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.669+0000 7f32feffd640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ec06aea0 0x7f32ec06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:35.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.669+0000 7f32feffd640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ec06aea0 0x7f32ec06d360 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009c60 tx=0x7f32f0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:35.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.675+0000 7f32e7fff640 1 -- 172.21.15.81:0/794244706 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f32f4076bf0 con 0x7f33000b3170 2023-11-08T16:41:35.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.903+0000 7f330590f640 1 -- 172.21.15.81:0/794244706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f330006df50 con 0x7f33000b3170 2023-11-08T16:41:35.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.904+0000 7f32e7fff640 1 -- 172.21.15.81:0/794244706 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) v1 ==== 53+0+73 (secure 0 0 0) 0x7f32f4034020 con 0x7f33000b3170 2023-11-08T16:41:35.907 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T16:41:35.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.906+0000 7f330590f640 1 -- 172.21.15.81:0/794244706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ec06aea0 msgr2=0x7f32ec06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.906+0000 7f330590f640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ec06aea0 0x7f32ec06d360 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009c60 tx=0x7f32f0008040 comp rx=0 tx=0).stop 2023-11-08T16:41:35.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.906+0000 7f330590f640 1 -- 172.21.15.81:0/794244706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33000b3170 msgr2=0x7f33001e5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:35.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.906+0000 7f330590f640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33000b3170 0x7f33001e5980 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f32f402b9c0 tx=0x7f32f40046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.906+0000 7f330590f640 1 -- 172.21.15.81:0/794244706 shutdown_connections 2023-11-08T16:41:35.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.906+0000 7f330590f640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f32ec06aea0 0x7f32ec06d360 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.907+0000 7f330590f640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f33000b3550 0x7f33001f3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.907+0000 7f330590f640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f33000b3170 0x7f33001e5980 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.907+0000 7f330590f640 1 --2- 172.21.15.81:0/794244706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f330006c9f0 0x7f33001f3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:35.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.907+0000 7f330590f640 1 -- 172.21.15.81:0/794244706 >> 172.21.15.81:0/794244706 conn(0x7f330006a6a0 msgr2=0x7f33000ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:35.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.907+0000 7f330590f640 1 -- 172.21.15.81:0/794244706 shutdown_connections 2023-11-08T16:41:35.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:35.907+0000 7f330590f640 1 -- 172.21.15.81:0/794244706 wait complete. 2023-11-08T16:41:35.928 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:41:35.928 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:41:35.929 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:41:35.929 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:41:35.929 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:35.930 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:41:35.930 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:41:35.930 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:41:35.931 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:41:35.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:35.932 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:41:35.986 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:41:35.987 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:41:35.987 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:41:35.987 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:35.987 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:41:35.987 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:35.988 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:41:35.988 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:35.988 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:41:35.988 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:41:35.988 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:35.989 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:41:35.989 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:41:35.989 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:35.989 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:41:35.989 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:35.990 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:41:35.990 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 41614sec preferred_lft 41614sec 2023-11-08T16:41:35.990 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:41:35.990 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:35.990 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:41:35.991 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:35.991 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:41:35.991 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:35.993 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:41:35.993 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:35.994 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:41:35.994 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:35.995 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:41:35.995 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:35.995 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:41:35.995 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:35.996 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::2cac:17ff:fe64:67da/64 scope link 2023-11-08T16:41:35.996 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:35.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:35.997 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:41:36.090 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:41:36.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:36.095 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:41:36.152 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:41:36.153 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:41:36.153 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:41:36.153 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:41:36.153 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:41:36.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:36.154 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:41:36.155 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:41:36.155 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:41:36.155 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:41:36.156 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:41:36.156 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:41:36.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:36.336 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:41:36.390 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:41:36.416 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:41:36.417 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:41:36.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:36.418 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:41:36.518 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:41:36.518 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:41:36.519 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:36.576 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:41:36.576 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:41:36.577 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:36.627 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:41:36.717 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:41:36.777 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:41:36.778 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:41:36.778 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:41:36.778 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:41:36.778 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:41:36.779 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:41:36.779 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:41:36.779 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:41:36.779 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:41:36.779 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:41:36.780 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:41:36.780 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:41:36.780 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:41:36.780 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:41:36.780 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:41:36.781 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:41:36.781 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:41:36.781 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:41:36.781 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:41:36.781 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:41:36.781 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:41:36.782 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:41:36.782 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:41:36.782 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:41:36.782 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:41:36.783 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:41:36.783 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:41:36.783 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:41:36.783 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:41:36.783 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:36.784 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:36.784 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:36.784 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:36.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:36.786 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:41:36.838 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:41:36.839 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:41:36.845 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:41:36.921 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:41:36.964 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:41:36.964 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:41:36.965 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:41:36.965 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:41:36.965 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:41:36.965 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:41:36.966 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:41:36.966 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:41:36.966 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:41:36.966 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:41:36.966 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:41:36.967 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:41:36.967 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:41:36.967 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:41:36.967 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:41:36.967 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:41:36.968 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:41:36.968 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:41:36.968 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:41:36.968 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:41:36.968 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:41:36.968 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:41:36.969 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:41:36.969 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:41:36.969 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:41:36.969 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:41:36.969 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:41:36.970 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:41:36.970 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:41:36.970 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:36.970 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:36.970 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:36.971 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:36.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:36.972 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:41:37.030 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:41:37.027+0000 7f792846f480 -1 init, newargv = 0x55786839e1c0 newargc=15 2023-11-08T16:41:37.030 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[126550]: starting ceph client 2023-11-08T16:41:37.038 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[126550]: starting fuse 2023-11-08T16:41:37.058 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:41:37.058 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:41:37.059 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:41:37.060 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:41:37.060 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:41:37.060 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:41:37.061 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:41:37.061 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:37.062 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:41:37.062 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:41:37.063 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:41:37.063 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:41:37.064 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:37.065 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:41:37.065 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:41:37.066 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:41:37.066 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:41:37.066 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:37.067 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:41:37.067 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:41:37.068 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:41:37.068 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:41:37.068 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:41:37.069 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:41:37.069 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:41:37.069 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:41:37.070 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:37.070 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:41:37.070 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:41:37.263 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.126550.asok 2023-11-08T16:41:37.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:41:37.280 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.126550.asok 2023-11-08T16:41:37.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:37.281 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.126550.asok status 2023-11-08T16:41:37.522 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:41:37.522 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:41:37.523 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:41:37.523 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:41:37.523 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:41:37.523 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:41:37.523 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:41:37.524 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "126550", 2023-11-08T16:41:37.524 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:41:37.524 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:37.524 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:41:37.525 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:41:37.525 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 8892, 2023-11-08T16:41:37.525 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:41:37.525 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:41:37.525 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:41:37.526 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 8892 2023-11-08T16:41:37.526 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:37.526 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:41:37.526 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:41:37.526 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:41:37.527 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2724252863 2023-11-08T16:41:37.527 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:41:37.527 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:37.527 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:41:37.528 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:41:37.528 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:41:37.528 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 2724252863 2023-11-08T16:41:37.528 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:41:37.528 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.8892 192.168.0.1:0/2724252863", 2023-11-08T16:41:37.529 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/2724252863", 2023-11-08T16:41:37.529 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:41:37.529 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 113, 2023-11-08T16:41:37.529 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 128, 2023-11-08T16:41:37.529 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:41:37.530 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:41:37.530 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:41:37.530 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:41:37.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:37.537 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:41:37.538 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:37.590 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:41:37.591 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:37.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:37.592 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:37.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:37.687 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2023-11-08T16:41:37.765 INFO:teuthology.orchestra.run.smithi081.stdout:0 2023-11-08T16:41:37.765 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:41:37.766 INFO:teuthology.orchestra.run.smithi081.stdout:10 2023-11-08T16:41:37.766 INFO:teuthology.orchestra.run.smithi081.stdout:11 2023-11-08T16:41:37.767 INFO:teuthology.orchestra.run.smithi081.stdout:12 2023-11-08T16:41:37.767 INFO:teuthology.orchestra.run.smithi081.stdout:13 2023-11-08T16:41:37.767 INFO:teuthology.orchestra.run.smithi081.stdout:14 2023-11-08T16:41:37.767 INFO:teuthology.orchestra.run.smithi081.stdout:15 2023-11-08T16:41:37.768 INFO:teuthology.orchestra.run.smithi081.stdout:16 2023-11-08T16:41:37.768 INFO:teuthology.orchestra.run.smithi081.stdout:17 2023-11-08T16:41:37.768 INFO:teuthology.orchestra.run.smithi081.stdout:18 2023-11-08T16:41:37.768 INFO:teuthology.orchestra.run.smithi081.stdout:19 2023-11-08T16:41:37.769 INFO:teuthology.orchestra.run.smithi081.stdout:2 2023-11-08T16:41:37.769 INFO:teuthology.orchestra.run.smithi081.stdout:20 2023-11-08T16:41:37.769 INFO:teuthology.orchestra.run.smithi081.stdout:22 2023-11-08T16:41:37.769 INFO:teuthology.orchestra.run.smithi081.stdout:23 2023-11-08T16:41:37.770 INFO:teuthology.orchestra.run.smithi081.stdout:24 2023-11-08T16:41:37.770 INFO:teuthology.orchestra.run.smithi081.stdout:25 2023-11-08T16:41:37.770 INFO:teuthology.orchestra.run.smithi081.stdout:26 2023-11-08T16:41:37.770 INFO:teuthology.orchestra.run.smithi081.stdout:27 2023-11-08T16:41:37.771 INFO:teuthology.orchestra.run.smithi081.stdout:28 2023-11-08T16:41:37.771 INFO:teuthology.orchestra.run.smithi081.stdout:29 2023-11-08T16:41:37.771 INFO:teuthology.orchestra.run.smithi081.stdout:3 2023-11-08T16:41:37.772 INFO:teuthology.orchestra.run.smithi081.stdout:30 2023-11-08T16:41:37.772 INFO:teuthology.orchestra.run.smithi081.stdout:31 2023-11-08T16:41:37.772 INFO:teuthology.orchestra.run.smithi081.stdout:32 2023-11-08T16:41:37.772 INFO:teuthology.orchestra.run.smithi081.stdout:33 2023-11-08T16:41:37.772 INFO:teuthology.orchestra.run.smithi081.stdout:34 2023-11-08T16:41:37.773 INFO:teuthology.orchestra.run.smithi081.stdout:35 2023-11-08T16:41:37.773 INFO:teuthology.orchestra.run.smithi081.stdout:36 2023-11-08T16:41:37.773 INFO:teuthology.orchestra.run.smithi081.stdout:37 2023-11-08T16:41:37.773 INFO:teuthology.orchestra.run.smithi081.stdout:38 2023-11-08T16:41:37.774 INFO:teuthology.orchestra.run.smithi081.stdout:39 2023-11-08T16:41:37.774 INFO:teuthology.orchestra.run.smithi081.stdout:4 2023-11-08T16:41:37.774 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:41:37.774 INFO:teuthology.orchestra.run.smithi081.stdout:41 2023-11-08T16:41:37.775 INFO:teuthology.orchestra.run.smithi081.stdout:42 2023-11-08T16:41:37.775 INFO:teuthology.orchestra.run.smithi081.stdout:43 2023-11-08T16:41:37.775 INFO:teuthology.orchestra.run.smithi081.stdout:44 2023-11-08T16:41:37.775 INFO:teuthology.orchestra.run.smithi081.stdout:45 2023-11-08T16:41:37.776 INFO:teuthology.orchestra.run.smithi081.stdout:46 2023-11-08T16:41:37.776 INFO:teuthology.orchestra.run.smithi081.stdout:47 2023-11-08T16:41:37.776 INFO:teuthology.orchestra.run.smithi081.stdout:48 2023-11-08T16:41:37.776 INFO:teuthology.orchestra.run.smithi081.stdout:49 2023-11-08T16:41:37.776 INFO:teuthology.orchestra.run.smithi081.stdout:5 2023-11-08T16:41:37.777 INFO:teuthology.orchestra.run.smithi081.stdout:50 2023-11-08T16:41:37.777 INFO:teuthology.orchestra.run.smithi081.stdout:51 2023-11-08T16:41:37.777 INFO:teuthology.orchestra.run.smithi081.stdout:52 2023-11-08T16:41:37.777 INFO:teuthology.orchestra.run.smithi081.stdout:53 2023-11-08T16:41:37.778 INFO:teuthology.orchestra.run.smithi081.stdout:54 2023-11-08T16:41:37.778 INFO:teuthology.orchestra.run.smithi081.stdout:55 2023-11-08T16:41:37.778 INFO:teuthology.orchestra.run.smithi081.stdout:56 2023-11-08T16:41:37.778 INFO:teuthology.orchestra.run.smithi081.stdout:57 2023-11-08T16:41:37.778 INFO:teuthology.orchestra.run.smithi081.stdout:58 2023-11-08T16:41:37.779 INFO:teuthology.orchestra.run.smithi081.stdout:59 2023-11-08T16:41:37.779 INFO:teuthology.orchestra.run.smithi081.stdout:6 2023-11-08T16:41:37.779 INFO:teuthology.orchestra.run.smithi081.stdout:60 2023-11-08T16:41:37.779 INFO:teuthology.orchestra.run.smithi081.stdout:61 2023-11-08T16:41:37.780 INFO:teuthology.orchestra.run.smithi081.stdout:62 2023-11-08T16:41:37.780 INFO:teuthology.orchestra.run.smithi081.stdout:63 2023-11-08T16:41:37.780 INFO:teuthology.orchestra.run.smithi081.stdout:64 2023-11-08T16:41:37.780 INFO:teuthology.orchestra.run.smithi081.stdout:65 2023-11-08T16:41:37.780 INFO:teuthology.orchestra.run.smithi081.stdout:66 2023-11-08T16:41:37.781 INFO:teuthology.orchestra.run.smithi081.stdout:67 2023-11-08T16:41:37.781 INFO:teuthology.orchestra.run.smithi081.stdout:68 2023-11-08T16:41:37.781 INFO:teuthology.orchestra.run.smithi081.stdout:69 2023-11-08T16:41:37.781 INFO:teuthology.orchestra.run.smithi081.stdout:7 2023-11-08T16:41:37.781 INFO:teuthology.orchestra.run.smithi081.stdout:70 2023-11-08T16:41:37.782 INFO:teuthology.orchestra.run.smithi081.stdout:71 2023-11-08T16:41:37.782 INFO:teuthology.orchestra.run.smithi081.stdout:72 2023-11-08T16:41:37.782 INFO:teuthology.orchestra.run.smithi081.stdout:73 2023-11-08T16:41:37.782 INFO:teuthology.orchestra.run.smithi081.stdout:74 2023-11-08T16:41:37.783 INFO:teuthology.orchestra.run.smithi081.stdout:75 2023-11-08T16:41:37.783 INFO:teuthology.orchestra.run.smithi081.stdout:76 2023-11-08T16:41:37.783 INFO:teuthology.orchestra.run.smithi081.stdout:77 2023-11-08T16:41:37.783 INFO:teuthology.orchestra.run.smithi081.stdout:78 2023-11-08T16:41:37.783 INFO:teuthology.orchestra.run.smithi081.stdout:79 2023-11-08T16:41:37.784 INFO:teuthology.orchestra.run.smithi081.stdout:8 2023-11-08T16:41:37.784 INFO:teuthology.orchestra.run.smithi081.stdout:80 2023-11-08T16:41:37.784 INFO:teuthology.orchestra.run.smithi081.stdout:81 2023-11-08T16:41:37.784 INFO:teuthology.orchestra.run.smithi081.stdout:82 2023-11-08T16:41:37.785 INFO:teuthology.orchestra.run.smithi081.stdout:83 2023-11-08T16:41:37.785 INFO:teuthology.orchestra.run.smithi081.stdout:84 2023-11-08T16:41:37.785 INFO:teuthology.orchestra.run.smithi081.stdout:85 2023-11-08T16:41:37.785 INFO:teuthology.orchestra.run.smithi081.stdout:86 2023-11-08T16:41:37.785 INFO:teuthology.orchestra.run.smithi081.stdout:87 2023-11-08T16:41:37.786 INFO:teuthology.orchestra.run.smithi081.stdout:88 2023-11-08T16:41:37.786 INFO:teuthology.orchestra.run.smithi081.stdout:89 2023-11-08T16:41:37.786 INFO:teuthology.orchestra.run.smithi081.stdout:9 2023-11-08T16:41:37.786 INFO:teuthology.orchestra.run.smithi081.stdout:90 2023-11-08T16:41:37.787 INFO:teuthology.orchestra.run.smithi081.stdout:91 2023-11-08T16:41:37.787 INFO:teuthology.orchestra.run.smithi081.stdout:92 2023-11-08T16:41:37.787 INFO:teuthology.orchestra.run.smithi081.stdout:93 2023-11-08T16:41:37.787 INFO:teuthology.orchestra.run.smithi081.stdout:94 2023-11-08T16:41:37.788 INFO:teuthology.orchestra.run.smithi081.stdout:95 2023-11-08T16:41:37.788 INFO:teuthology.orchestra.run.smithi081.stdout:96 2023-11-08T16:41:37.788 INFO:teuthology.orchestra.run.smithi081.stdout:97 2023-11-08T16:41:37.788 INFO:teuthology.orchestra.run.smithi081.stdout:98 2023-11-08T16:41:37.788 INFO:teuthology.orchestra.run.smithi081.stdout:99 2023-11-08T16:41:37.790 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:41:37.791 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:41:37.848 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:41:38.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.055+0000 7fc91759e640 1 Processor -- start 2023-11-08T16:41:38.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.055+0000 7fc91759e640 1 -- start start 2023-11-08T16:41:38.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.056+0000 7fc91759e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc91815a160 0x7fc9180ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.056+0000 7fc91759e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9180cee20 0x7fc9180c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.056+0000 7fc91759e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9180cf260 con 0x7fc918159d80 2023-11-08T16:41:38.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.056+0000 7fc91759e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9180ca510 con 0x7fc91815a160 2023-11-08T16:41:38.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.056+0000 7fc91759e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180cd590 con 0x7fc9180cee20 2023-11-08T16:41:38.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.057+0000 7fc915d9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc91815a160 0x7fc9180ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.057+0000 7fc915d9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc91815a160 0x7fc9180ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37594/0 (socket says 172.21.15.81:37594) 2023-11-08T16:41:38.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.057+0000 7fc915d9b640 1 -- 172.21.15.81:0/879980820 learned_addr learned my addr 172.21.15.81:0/879980820 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:38.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.057+0000 7fc915d9b640 1 -- 172.21.15.81:0/879980820 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9180cee20 msgr2=0x7fc9180c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:38.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.057+0000 7fc916d9d640 1 --2- 172.21.15.81:0/879980820 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9180cee20 0x7fc9180c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.057+0000 7fc915d9b640 1 --2- 172.21.15.81:0/879980820 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9180cee20 0x7fc9180c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.057+0000 7fc915d9b640 1 -- 172.21.15.81:0/879980820 >> v1:172.21.15.81:6789/0 conn(0x7fc918159d80 legacy=0x7fc91806ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.057+0000 7fc915d9b640 1 -- 172.21.15.81:0/879980820 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9180706a0 con 0x7fc91815a160 2023-11-08T16:41:38.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.057+0000 7fc91559a640 1 -- 172.21.15.81:0/879980820 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1824057287 0 0) 0x7fc9180cf260 con 0x7fc918159d80 2023-11-08T16:41:38.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.058+0000 7fc915d9b640 1 --2- 172.21.15.81:0/879980820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc91815a160 0x7fc9180ce8e0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fc908009870 tx=0x7fc90802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6604c159ec85ab0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:38.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.059+0000 7fc91559a640 1 -- 172.21.15.81:0/879980820 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908031860 con 0x7fc91815a160 2023-11-08T16:41:38.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.059+0000 7fc91559a640 1 -- 172.21.15.81:0/879980820 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9080319c0 con 0x7fc91815a160 2023-11-08T16:41:38.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.059+0000 7fc91559a640 1 -- 172.21.15.81:0/879980820 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc908031d50 con 0x7fc91815a160 2023-11-08T16:41:38.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.059+0000 7fc91759e640 1 -- 172.21.15.81:0/879980820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc91815a160 msgr2=0x7fc9180ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.059+0000 7fc91759e640 1 --2- 172.21.15.81:0/879980820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc91815a160 0x7fc9180ce8e0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fc908009870 tx=0x7fc90802b900 comp rx=0 tx=0).stop 2023-11-08T16:41:38.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.059+0000 7fc91759e640 1 -- 172.21.15.81:0/879980820 shutdown_connections 2023-11-08T16:41:38.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.059+0000 7fc91759e640 1 --2- 172.21.15.81:0/879980820 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9180cee20 0x7fc9180c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.059+0000 7fc91759e640 1 --2- 172.21.15.81:0/879980820 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc91815a160 0x7fc9180ce8e0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.059+0000 7fc91759e640 1 -- 172.21.15.81:0/879980820 >> 172.21.15.81:0/879980820 conn(0x7fc9180b06e0 msgr2=0x7fc9180b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:38.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.060+0000 7fc91759e640 1 -- 172.21.15.81:0/879980820 shutdown_connections 2023-11-08T16:41:38.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.060+0000 7fc91759e640 1 -- 172.21.15.81:0/879980820 wait complete. 2023-11-08T16:41:38.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.061+0000 7fc91759e640 1 Processor -- start 2023-11-08T16:41:38.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.061+0000 7fc91759e640 1 -- start start 2023-11-08T16:41:38.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.062+0000 7fc91759e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9180cee20 0x7fc918151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.062+0000 7fc91759e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc918159d80 0x7fc9181f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.062+0000 7fc91759e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9181f6720 0x7fc9180b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.062+0000 7fc91759e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9181f6bb0 con 0x7fc9181f6720 2023-11-08T16:41:38.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.062+0000 7fc91759e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180b2e80 con 0x7fc918159d80 2023-11-08T16:41:38.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.062+0000 7fc91759e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc918153f60 con 0x7fc9180cee20 2023-11-08T16:41:38.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.063+0000 7fc916d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9181f6720 0x7fc9180b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.063+0000 7fc916d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9181f6720 0x7fc9180b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37602/0 (socket says 172.21.15.81:37602) 2023-11-08T16:41:38.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.063+0000 7fc916d9d640 1 -- 172.21.15.81:0/3768261668 learned_addr learned my addr 172.21.15.81:0/3768261668 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:38.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.063+0000 7fc91659c640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9180cee20 0x7fc918151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.063+0000 7fc915d9b640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc918159d80 0x7fc9181f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.063+0000 7fc916d9d640 1 -- 172.21.15.81:0/3768261668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9180cee20 msgr2=0x7fc918151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.063+0000 7fc916d9d640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9180cee20 0x7fc918151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.063+0000 7fc916d9d640 1 -- 172.21.15.81:0/3768261668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc918159d80 msgr2=0x7fc9181f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.063+0000 7fc916d9d640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc918159d80 0x7fc9181f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.063+0000 7fc916d9d640 1 -- 172.21.15.81:0/3768261668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9080094d0 con 0x7fc9181f6720 2023-11-08T16:41:38.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.063+0000 7fc91659c640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9180cee20 0x7fc918151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:38.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.064+0000 7fc916d9d640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9181f6720 0x7fc9180b35b0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fc91000cd70 tx=0x7fc91000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:38.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.064+0000 7fc90effd640 1 -- 172.21.15.81:0/3768261668 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910003b90 con 0x7fc9181f6720 2023-11-08T16:41:38.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.064+0000 7fc90effd640 1 -- 172.21.15.81:0/3768261668 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc910003cf0 con 0x7fc9181f6720 2023-11-08T16:41:38.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.064+0000 7fc90effd640 1 -- 172.21.15.81:0/3768261668 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc910004240 con 0x7fc9181f6720 2023-11-08T16:41:38.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.064+0000 7fc91759e640 1 -- 172.21.15.81:0/3768261668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc918154240 con 0x7fc9181f6720 2023-11-08T16:41:38.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.064+0000 7fc91759e640 1 -- 172.21.15.81:0/3768261668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9180b3090 con 0x7fc9181f6720 2023-11-08T16:41:38.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.065+0000 7fc91759e640 1 -- 172.21.15.81:0/3768261668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9180ca7a0 con 0x7fc9181f6720 2023-11-08T16:41:38.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.066+0000 7fc90effd640 1 -- 172.21.15.81:0/3768261668 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc9100043a0 con 0x7fc9181f6720 2023-11-08T16:41:38.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.067+0000 7fc90effd640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc8f806ae70 0x7fc8f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.067+0000 7fc90effd640 1 -- 172.21.15.81:0/3768261668 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc910012070 con 0x7fc9181f6720 2023-11-08T16:41:38.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.067+0000 7fc91659c640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc8f806ae70 0x7fc8f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.068+0000 7fc91659c640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc8f806ae70 0x7fc8f806d330 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc900000f60 tx=0x7fc900009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:38.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.072+0000 7fc90effd640 1 -- 172.21.15.81:0/3768261668 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc9100550b0 con 0x7fc9181f6720 2023-11-08T16:41:38.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.263+0000 7fc91759e640 1 -- 172.21.15.81:0/3768261668 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9180cfaa0 con 0x7fc9181f6720 2023-11-08T16:41:38.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.265+0000 7fc90effd640 1 -- 172.21.15.81:0/3768261668 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16596 (secure 0 0 0) 0x7fc9100586d0 con 0x7fc9181f6720 2023-11-08T16:41:38.267 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:38.268 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":128,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:29.470231+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:41:03.545565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T16:41:04.726492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6835/4021510722":"2023-11-09T16:41:29.470203+0000","172.21.15.93:6833/4021510722":"2023-11-09T16:41:29.470203+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:41:38.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.267+0000 7fc91759e640 1 -- 172.21.15.81:0/3768261668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc8f806ae70 msgr2=0x7fc8f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.267+0000 7fc91759e640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc8f806ae70 0x7fc8f806d330 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc900000f60 tx=0x7fc900009040 comp rx=0 tx=0).stop 2023-11-08T16:41:38.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.267+0000 7fc91759e640 1 -- 172.21.15.81:0/3768261668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9181f6720 msgr2=0x7fc9180b35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.267+0000 7fc91759e640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9181f6720 0x7fc9180b35b0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fc91000cd70 tx=0x7fc91000a660 comp rx=0 tx=0).stop 2023-11-08T16:41:38.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.267+0000 7fc91759e640 1 -- 172.21.15.81:0/3768261668 shutdown_connections 2023-11-08T16:41:38.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.267+0000 7fc91759e640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc8f806ae70 0x7fc8f806d330 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.267+0000 7fc91759e640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc9181f6720 0x7fc9180b35b0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.267+0000 7fc91759e640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc918159d80 0x7fc9181f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.267+0000 7fc91759e640 1 --2- 172.21.15.81:0/3768261668 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc9180cee20 0x7fc918151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.267+0000 7fc91759e640 1 -- 172.21.15.81:0/3768261668 >> 172.21.15.81:0/3768261668 conn(0x7fc9180b06e0 msgr2=0x7fc91806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:38.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.268+0000 7fc91759e640 1 -- 172.21.15.81:0/3768261668 shutdown_connections 2023-11-08T16:41:38.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.268+0000 7fc91759e640 1 -- 172.21.15.81:0/3768261668 wait complete. 2023-11-08T16:41:38.289 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:41:03.545565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:41:04.726492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:41:38.290 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:41:38.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.494+0000 7fc1b9954640 1 Processor -- start 2023-11-08T16:41:38.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.495+0000 7fc1b9954640 1 -- start start 2023-11-08T16:41:38.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.497+0000 7fc1b9954640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc1b40b3380 0x7fc1b40b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.497+0000 7fc1b9954640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1b40b37e0 0x7fc1b40c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.497+0000 7fc1b9954640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1b40cdb20 con 0x7fc1b40d12c0 2023-11-08T16:41:38.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.497+0000 7fc1b9954640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1b40d38a0 con 0x7fc1b40b3380 2023-11-08T16:41:38.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.497+0000 7fc1b9954640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40d0b60 con 0x7fc1b40b37e0 2023-11-08T16:41:38.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.497+0000 7fc1b8952640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc1b40b3380 0x7fc1b40b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.497+0000 7fc1b8952640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fc1b40b3380 0x7fc1b40b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37618/0 (socket says 172.21.15.81:37618) 2023-11-08T16:41:38.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.497+0000 7fc1b8952640 1 -- 172.21.15.81:0/306445245 learned_addr learned my addr 172.21.15.81:0/306445245 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:38.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.497+0000 7fc1b3fff640 1 --2- 172.21.15.81:0/306445245 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1b40b37e0 0x7fc1b40c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.498+0000 7fc1b8952640 1 -- 172.21.15.81:0/306445245 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1b40b37e0 msgr2=0x7fc1b40c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:38.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.498+0000 7fc1b8952640 1 --2- 172.21.15.81:0/306445245 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1b40b37e0 0x7fc1b40c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.498+0000 7fc1b8952640 1 -- 172.21.15.81:0/306445245 >> v1:172.21.15.93:6789/0 conn(0x7fc1b40d12c0 legacy=0x7fc1b40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:38.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.498+0000 7fc1b8952640 1 -- 172.21.15.81:0/306445245 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1b40d3b40 con 0x7fc1b40b3380 2023-11-08T16:41:38.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.499+0000 7fc1b8952640 1 --2- 172.21.15.81:0/306445245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1b40b3380 0x7fc1b40b3d10 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0098a0 tx=0x7fc19c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccbe561f951a661c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:38.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.499+0000 7fc1b37fe640 1 -- 172.21.15.81:0/306445245 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c031640 con 0x7fc1b40b3380 2023-11-08T16:41:38.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.499+0000 7fc1b37fe640 1 -- 172.21.15.81:0/306445245 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc19c0317a0 con 0x7fc1b40b3380 2023-11-08T16:41:38.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.500+0000 7fc1b37fe640 1 -- 172.21.15.81:0/306445245 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19c031b30 con 0x7fc1b40b3380 2023-11-08T16:41:38.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.500+0000 7fc1b9954640 1 -- 172.21.15.81:0/306445245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1b40b3380 msgr2=0x7fc1b40b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.500+0000 7fc1b9954640 1 --2- 172.21.15.81:0/306445245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1b40b3380 0x7fc1b40b3d10 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7fc19c0098a0 tx=0x7fc19c02b8f0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.500+0000 7fc1b9954640 1 -- 172.21.15.81:0/306445245 shutdown_connections 2023-11-08T16:41:38.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.500+0000 7fc1b9954640 1 --2- 172.21.15.81:0/306445245 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1b40b37e0 0x7fc1b40c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.500+0000 7fc1b9954640 1 --2- 172.21.15.81:0/306445245 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1b40b3380 0x7fc1b40b3d10 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.500+0000 7fc1b9954640 1 -- 172.21.15.81:0/306445245 >> 172.21.15.81:0/306445245 conn(0x7fc1b40d2ce0 msgr2=0x7fc1b40d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:38.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.500+0000 7fc1b9954640 1 -- 172.21.15.81:0/306445245 shutdown_connections 2023-11-08T16:41:38.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.501+0000 7fc1b9954640 1 -- 172.21.15.81:0/306445245 wait complete. 2023-11-08T16:41:38.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.502+0000 7fc1b9954640 1 Processor -- start 2023-11-08T16:41:38.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.502+0000 7fc1b9954640 1 -- start start 2023-11-08T16:41:38.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b9954640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1b40b3380 0x7fc1b415bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b9954640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1b40b37e0 0x7fc1b415a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b9954640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1b40d12c0 0x7fc1b415ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b9954640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b4155ea0 con 0x7fc1b40b37e0 2023-11-08T16:41:38.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b9954640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40cbd80 con 0x7fc1b40d12c0 2023-11-08T16:41:38.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b9954640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1b40cc060 con 0x7fc1b40b3380 2023-11-08T16:41:38.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1b40b37e0 0x7fc1b415a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1b40b37e0 0x7fc1b415a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37622/0 (socket says 172.21.15.81:37622) 2023-11-08T16:41:38.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b3fff640 1 -- 172.21.15.81:0/2755034693 learned_addr learned my addr 172.21.15.81:0/2755034693 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:38.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b8952640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1b40b3380 0x7fc1b415bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b3fff640 1 -- 172.21.15.81:0/2755034693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1b40b3380 msgr2=0x7fc1b415bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b3fff640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1b40b3380 0x7fc1b415bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.504+0000 7fc1b3fff640 1 -- 172.21.15.81:0/2755034693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1b40d12c0 msgr2=0x7fc1b415ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.505+0000 7fc1b3fff640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1b40d12c0 0x7fc1b415ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.505+0000 7fc1b3fff640 1 -- 172.21.15.81:0/2755034693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c0094d0 con 0x7fc1b40b37e0 2023-11-08T16:41:38.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.505+0000 7fc1b8952640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1b40b3380 0x7fc1b415bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:38.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.505+0000 7fc1b3fff640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1b40b37e0 0x7fc1b415a8f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4007e60 tx=0x7fc1a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:38.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.505+0000 7fc1b17fa640 1 -- 172.21.15.81:0/2755034693 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4003b90 con 0x7fc1b40b37e0 2023-11-08T16:41:38.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.505+0000 7fc1b17fa640 1 -- 172.21.15.81:0/2755034693 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1a400ee90 con 0x7fc1b40b37e0 2023-11-08T16:41:38.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.505+0000 7fc1b9954640 1 -- 172.21.15.81:0/2755034693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1b40cc340 con 0x7fc1b40b37e0 2023-11-08T16:41:38.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.505+0000 7fc1b17fa640 1 -- 172.21.15.81:0/2755034693 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4004030 con 0x7fc1b40b37e0 2023-11-08T16:41:38.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.505+0000 7fc1b9954640 1 -- 172.21.15.81:0/2755034693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1b415c020 con 0x7fc1b40b37e0 2023-11-08T16:41:38.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.507+0000 7fc1b9954640 1 -- 172.21.15.81:0/2755034693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1b404ece0 con 0x7fc1b40b37e0 2023-11-08T16:41:38.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.507+0000 7fc1b17fa640 1 -- 172.21.15.81:0/2755034693 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc1a4004190 con 0x7fc1b40b37e0 2023-11-08T16:41:38.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.509+0000 7fc1b17fa640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc19406ae70 0x7fc19406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.509+0000 7fc1b17fa640 1 -- 172.21.15.81:0/2755034693 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc1a4089950 con 0x7fc1b40b37e0 2023-11-08T16:41:38.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.509+0000 7fc1b8952640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc19406ae70 0x7fc19406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.510+0000 7fc1b8952640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc19406ae70 0x7fc19406d330 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc19c036000 tx=0x7fc19c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:38.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.514+0000 7fc1b17fa640 1 -- 172.21.15.81:0/2755034693 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fc1a4054a50 con 0x7fc1b40b37e0 2023-11-08T16:41:38.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.704+0000 7fc1b9954640 1 -- 172.21.15.81:0/2755034693 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1b4155fe0 con 0x7fc1b40b37e0 2023-11-08T16:41:38.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.705+0000 7fc1b17fa640 1 -- 172.21.15.81:0/2755034693 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1a4058070 con 0x7fc1b40b37e0 2023-11-08T16:41:38.707 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:41:38.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.707+0000 7fc1b9954640 1 -- 172.21.15.81:0/2755034693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc19406ae70 msgr2=0x7fc19406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.707+0000 7fc1b9954640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc19406ae70 0x7fc19406d330 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc19c036000 tx=0x7fc19c03c040 comp rx=0 tx=0).stop 2023-11-08T16:41:38.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.707+0000 7fc1b9954640 1 -- 172.21.15.81:0/2755034693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1b40b37e0 msgr2=0x7fc1b415a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.707+0000 7fc1b9954640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1b40b37e0 0x7fc1b415a8f0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fc1a4007e60 tx=0x7fc1a400e4b0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.708+0000 7fc1b9954640 1 -- 172.21.15.81:0/2755034693 shutdown_connections 2023-11-08T16:41:38.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.708+0000 7fc1b9954640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fc19406ae70 0x7fc19406d330 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.708+0000 7fc1b9954640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1b40d12c0 0x7fc1b415ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.708+0000 7fc1b9954640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fc1b40b37e0 0x7fc1b415a8f0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.708+0000 7fc1b9954640 1 --2- 172.21.15.81:0/2755034693 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fc1b40b3380 0x7fc1b415bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.708+0000 7fc1b9954640 1 -- 172.21.15.81:0/2755034693 >> 172.21.15.81:0/2755034693 conn(0x7fc1b40d2ce0 msgr2=0x7fc1b40d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:38.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.708+0000 7fc1b9954640 1 -- 172.21.15.81:0/2755034693 shutdown_connections 2023-11-08T16:41:38.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.708+0000 7fc1b9954640 1 -- 172.21.15.81:0/2755034693 wait complete. 2023-11-08T16:41:38.729 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:41:38.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.948+0000 7fcf6e094640 1 Processor -- start 2023-11-08T16:41:38.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.948+0000 7fcf6e094640 1 -- start start 2023-11-08T16:41:38.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.949+0000 7fcf6e094640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf68158c60 0x7fcf680705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.949+0000 7fcf6e094640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6815fad0 con 0x7fcf6815f380 2023-11-08T16:41:38.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.949+0000 7fcf6e094640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6806d760 con 0x7fcf68071ad0 2023-11-08T16:41:38.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.949+0000 7fcf6e094640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6806df80 con 0x7fcf68158c60 2023-11-08T16:41:38.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.950+0000 7fcf67fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcf6815f380 0x7fcf680c91c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56194/0 (socket says 172.21.15.81:56194) 2023-11-08T16:41:38.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.950+0000 7fcf67fff640 1 -- 172.21.15.81:0/2824317766 learned_addr learned my addr 172.21.15.81:0/2824317766 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:38.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.950+0000 7fcf677fe640 1 --2- 172.21.15.81:0/2824317766 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf68158c60 0x7fcf680705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.950+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 989703362 0 0) 0x7fcf6815fad0 con 0x7fcf6815f380 2023-11-08T16:41:38.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.950+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf54003520 con 0x7fcf6815f380 2023-11-08T16:41:38.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2656405993 0 0) 0x7fcf54003520 con 0x7fcf6815f380 2023-11-08T16:41:38.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf68158c60 msgr2=0x7fcf680705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:38.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 --2- 172.21.15.81:0/2824317766 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf68158c60 0x7fcf680705d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 >> v1:172.21.15.93:6789/0 conn(0x7fcf68071ad0 legacy=0x7fcf6815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6806da00 con 0x7fcf6815f380 2023-11-08T16:41:38.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fcf600019f0 con 0x7fcf6815f380 2023-11-08T16:41:38.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 >> v1:172.21.15.81:6789/0 conn(0x7fcf6815f380 legacy=0x7fcf680c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 --2- 172.21.15.81:0/2824317766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf540039a0 0x7fcf54003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 --2- 172.21.15.81:0/2824317766 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf68158c60 0x7fcf540047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 --2- 172.21.15.81:0/2824317766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 0x7fcf54005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6806df80 con 0x7fcf54004d00 2023-11-08T16:41:38.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5400b980 con 0x7fcf540039a0 2023-11-08T16:41:38.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5400bb40 con 0x7fcf68158c60 2023-11-08T16:41:38.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.951+0000 7fcf67fff640 1 --2- 172.21.15.81:0/2824317766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 0x7fcf54005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.952+0000 7fcf677fe640 1 --2- 172.21.15.81:0/2824317766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf540039a0 0x7fcf54003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.952+0000 7fcf67fff640 1 -- 172.21.15.81:0/2824317766 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf68158c60 msgr2=0x7fcf540047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:38.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.952+0000 7fcf67fff640 1 --2- 172.21.15.81:0/2824317766 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf68158c60 0x7fcf540047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.952+0000 7fcf67fff640 1 -- 172.21.15.81:0/2824317766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf540039a0 msgr2=0x7fcf54003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:38.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.952+0000 7fcf67fff640 1 --2- 172.21.15.81:0/2824317766 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf540039a0 0x7fcf54003d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.952+0000 7fcf67fff640 1 -- 172.21.15.81:0/2824317766 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54003520 con 0x7fcf54004d00 2023-11-08T16:41:38.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.953+0000 7fcf67fff640 1 --2- 172.21.15.81:0/2824317766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 0x7fcf54005180 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000a840 tx=0x7fcf540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=893511d5d1793d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:38.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.953+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6002cea0 con 0x7fcf54004d00 2023-11-08T16:41:38.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.953+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6002f070 con 0x7fcf54004d00 2023-11-08T16:41:38.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.953+0000 7fcf66ffd640 1 -- 172.21.15.81:0/2824317766 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60034870 con 0x7fcf54004d00 2023-11-08T16:41:38.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.953+0000 7fcf6e094640 1 -- 172.21.15.81:0/2824317766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 msgr2=0x7fcf54005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.953+0000 7fcf6e094640 1 --2- 172.21.15.81:0/2824317766 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 0x7fcf54005180 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000a840 tx=0x7fcf540057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.953+0000 7fcf67fff640 1 -- 172.21.15.81:0/2824317766 reap_dead start 2023-11-08T16:41:38.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.954+0000 7fcf6e094640 1 -- 172.21.15.81:0/2824317766 shutdown_connections 2023-11-08T16:41:38.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.954+0000 7fcf6e094640 1 -- 172.21.15.81:0/2824317766 >> 172.21.15.81:0/2824317766 conn(0x7fcf680ceb70 msgr2=0x7fcf680cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:38.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.954+0000 7fcf6e094640 1 -- 172.21.15.81:0/2824317766 shutdown_connections 2023-11-08T16:41:38.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.954+0000 7fcf6e094640 1 -- 172.21.15.81:0/2824317766 wait complete. 2023-11-08T16:41:38.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.955+0000 7fcf6e094640 1 Processor -- start 2023-11-08T16:41:38.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.956+0000 7fcf6e094640 1 -- start start 2023-11-08T16:41:38.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf6e094640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 0x7fcf681f1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf6e094640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf680c3a90 0x7fcf6814b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf6e094640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf680c7ea0 0x7fcf680c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf6e094640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680c8860 con 0x7fcf54004d00 2023-11-08T16:41:38.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf6e094640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf680c3600 con 0x7fcf680c7ea0 2023-11-08T16:41:38.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf6e094640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf681f19b0 con 0x7fcf680c3a90 2023-11-08T16:41:38.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 0x7fcf681f1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf677fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 0x7fcf681f1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37650/0 (socket says 172.21.15.81:37650) 2023-11-08T16:41:38.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf677fe640 1 -- 172.21.15.81:0/1009105378 learned_addr learned my addr 172.21.15.81:0/1009105378 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:38.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf5ffff640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf680c3a90 0x7fcf6814b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf677fe640 1 -- 172.21.15.81:0/1009105378 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf680c3a90 msgr2=0x7fcf6814b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf677fe640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf680c3a90 0x7fcf6814b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf67fff640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf680c7ea0 0x7fcf680c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf677fe640 1 -- 172.21.15.81:0/1009105378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf680c7ea0 msgr2=0x7fcf680c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:38.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf677fe640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf680c7ea0 0x7fcf680c8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:38.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf677fe640 1 -- 172.21.15.81:0/1009105378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6814bea0 con 0x7fcf54004d00 2023-11-08T16:41:38.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.957+0000 7fcf5ffff640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf680c3a90 0x7fcf6814b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:38.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.958+0000 7fcf677fe640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 0x7fcf681f1470 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fcf580108f0 tx=0x7fcf58010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:38.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.958+0000 7fcf64ff9640 1 -- 172.21.15.81:0/1009105378 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800eb00 con 0x7fcf54004d00 2023-11-08T16:41:38.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.958+0000 7fcf64ff9640 1 -- 172.21.15.81:0/1009105378 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5800ec60 con 0x7fcf54004d00 2023-11-08T16:41:38.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.958+0000 7fcf64ff9640 1 -- 172.21.15.81:0/1009105378 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5801c400 con 0x7fcf54004d00 2023-11-08T16:41:38.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.958+0000 7fcf6e094640 1 -- 172.21.15.81:0/1009105378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf680c29f0 con 0x7fcf54004d00 2023-11-08T16:41:38.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.958+0000 7fcf6e094640 1 -- 172.21.15.81:0/1009105378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf680c2fb0 con 0x7fcf54004d00 2023-11-08T16:41:38.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.959+0000 7fcf6e094640 1 -- 172.21.15.81:0/1009105378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf68074440 con 0x7fcf54004d00 2023-11-08T16:41:38.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.960+0000 7fcf64ff9640 1 -- 172.21.15.81:0/1009105378 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcf5801c560 con 0x7fcf54004d00 2023-11-08T16:41:38.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.961+0000 7fcf64ff9640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf4806aea0 0x7fcf4806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:38.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.961+0000 7fcf64ff9640 1 -- 172.21.15.81:0/1009105378 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcf58089b80 con 0x7fcf54004d00 2023-11-08T16:41:38.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.962+0000 7fcf5ffff640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf4806aea0 0x7fcf4806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:38.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.962+0000 7fcf5ffff640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf4806aea0 0x7fcf4806d360 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcf6814c6d0 tx=0x7fcf50009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:38.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:38.966+0000 7fcf64ff9640 1 -- 172.21.15.81:0/1009105378 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fcf58054be0 con 0x7fcf54004d00 2023-11-08T16:41:39.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.157+0000 7fcf6e094640 1 -- 172.21.15.81:0/1009105378 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcf6815d3a0 con 0x7fcf54004d00 2023-11-08T16:41:39.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.158+0000 7fcf64ff9640 1 -- 172.21.15.81:0/1009105378 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcf58058200 con 0x7fcf54004d00 2023-11-08T16:41:39.161 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:41:39.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 -- 172.21.15.81:0/1009105378 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf4806aea0 msgr2=0x7fcf4806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf4806aea0 0x7fcf4806d360 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcf6814c6d0 tx=0x7fcf50009040 comp rx=0 tx=0).stop 2023-11-08T16:41:39.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 -- 172.21.15.81:0/1009105378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 msgr2=0x7fcf681f1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 0x7fcf681f1470 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fcf580108f0 tx=0x7fcf58010db0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 -- 172.21.15.81:0/1009105378 shutdown_connections 2023-11-08T16:41:39.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcf4806aea0 0x7fcf4806d360 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf680c7ea0 0x7fcf680c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf680c3a90 0x7fcf6814b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 --2- 172.21.15.81:0/1009105378 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf54004d00 0x7fcf681f1470 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 -- 172.21.15.81:0/1009105378 >> 172.21.15.81:0/1009105378 conn(0x7fcf680ceb70 msgr2=0x7fcf680c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:39.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 -- 172.21.15.81:0/1009105378 shutdown_connections 2023-11-08T16:41:39.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.161+0000 7fcf6e094640 1 -- 172.21.15.81:0/1009105378 wait complete. 2023-11-08T16:41:39.183 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:41:39.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.390+0000 7fdccbb97640 1 Processor -- start 2023-11-08T16:41:39.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.390+0000 7fdccbb97640 1 -- start start 2023-11-08T16:41:39.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.391+0000 7fdccbb97640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcc415d280 0x7fdcc415e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.391+0000 7fdccbb97640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fdcc40c9ae0 0x7fdcc40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.391+0000 7fdccbb97640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcc415e800 con 0x7fdcc40ce550 2023-11-08T16:41:39.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.391+0000 7fdccbb97640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcc4161be0 con 0x7fdcc40c9ae0 2023-11-08T16:41:39.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.391+0000 7fdccbb97640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc4160f60 con 0x7fdcc415d280 2023-11-08T16:41:39.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.392+0000 7fdcc9132640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fdcc40ce550 0x7fdcc40c93d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56208/0 (socket says 172.21.15.81:56208) 2023-11-08T16:41:39.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.392+0000 7fdcc9132640 1 -- 172.21.15.81:0/2274369076 learned_addr learned my addr 172.21.15.81:0/2274369076 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:39.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.392+0000 7fdcca134640 1 --2- 172.21.15.81:0/2274369076 >> v2:172.21.15.81:3300/0 conn(0x7fdcc40c9ae0 0x7fdcc40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.392+0000 7fdcc9933640 1 --2- 172.21.15.81:0/2274369076 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcc415d280 0x7fdcc415e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.392+0000 7fdcc8931640 1 -- 172.21.15.81:0/2274369076 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919981194 0 0) 0x7fdcc415e800 con 0x7fdcc40ce550 2023-11-08T16:41:39.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.392+0000 7fdcc8931640 1 -- 172.21.15.81:0/2274369076 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb0003520 con 0x7fdcc40ce550 2023-11-08T16:41:39.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.392+0000 7fdcca134640 1 -- 172.21.15.81:0/2274369076 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcc415d280 msgr2=0x7fdcc415e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:39.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.392+0000 7fdcca134640 1 --2- 172.21.15.81:0/2274369076 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcc415d280 0x7fdcc415e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.392+0000 7fdcca134640 1 -- 172.21.15.81:0/2274369076 >> v1:172.21.15.81:6789/0 conn(0x7fdcc40ce550 legacy=0x7fdcc40c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.392+0000 7fdcca134640 1 -- 172.21.15.81:0/2274369076 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc4161240 con 0x7fdcc40c9ae0 2023-11-08T16:41:39.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.394+0000 7fdcca134640 1 --2- 172.21.15.81:0/2274369076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcc40c9ae0 0x7fdcc40ac8b0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009d90 tx=0x7fdcbc02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67c30c50cdb07372 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:39.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.394+0000 7fdcc8931640 1 -- 172.21.15.81:0/2274369076 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc02cb30 con 0x7fdcc40c9ae0 2023-11-08T16:41:39.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.394+0000 7fdcc8931640 1 -- 172.21.15.81:0/2274369076 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcbc02cc90 con 0x7fdcc40c9ae0 2023-11-08T16:41:39.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.394+0000 7fdcc8931640 1 -- 172.21.15.81:0/2274369076 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc034440 con 0x7fdcc40c9ae0 2023-11-08T16:41:39.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.394+0000 7fdccbb97640 1 -- 172.21.15.81:0/2274369076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcc40c9ae0 msgr2=0x7fdcc40ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.394+0000 7fdccbb97640 1 --2- 172.21.15.81:0/2274369076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcc40c9ae0 0x7fdcc40ac8b0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009d90 tx=0x7fdcbc02bd80 comp rx=0 tx=0).stop 2023-11-08T16:41:39.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.394+0000 7fdccbb97640 1 -- 172.21.15.81:0/2274369076 shutdown_connections 2023-11-08T16:41:39.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.394+0000 7fdccbb97640 1 --2- 172.21.15.81:0/2274369076 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcc40c9ae0 0x7fdcc40ac8b0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.394+0000 7fdccbb97640 1 --2- 172.21.15.81:0/2274369076 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcc415d280 0x7fdcc415e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.394+0000 7fdccbb97640 1 -- 172.21.15.81:0/2274369076 >> 172.21.15.81:0/2274369076 conn(0x7fdcc40ceb70 msgr2=0x7fdcc40cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:39.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.395+0000 7fdccbb97640 1 -- 172.21.15.81:0/2274369076 shutdown_connections 2023-11-08T16:41:39.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.395+0000 7fdccbb97640 1 -- 172.21.15.81:0/2274369076 wait complete. 2023-11-08T16:41:39.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.396+0000 7fdccbb97640 1 Processor -- start 2023-11-08T16:41:39.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.396+0000 7fdccbb97640 1 -- start start 2023-11-08T16:41:39.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.397+0000 7fdccbb97640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdcc40c9ae0 0x7fdcc41f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.397+0000 7fdccbb97640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcc40ce550 0x7fdcc406cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.397+0000 7fdccbb97640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcc415d280 0x7fdcc4148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.397+0000 7fdccbb97640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc41f71e0 con 0x7fdcc415d280 2023-11-08T16:41:39.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.397+0000 7fdccbb97640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc414ba90 con 0x7fdcc40c9ae0 2023-11-08T16:41:39.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.397+0000 7fdccbb97640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc414bd70 con 0x7fdcc40ce550 2023-11-08T16:41:39.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.398+0000 7fdcca134640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcc415d280 0x7fdcc4148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.398+0000 7fdcca134640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcc415d280 0x7fdcc4148dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37668/0 (socket says 172.21.15.81:37668) 2023-11-08T16:41:39.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.398+0000 7fdcca134640 1 -- 172.21.15.81:0/2565349706 learned_addr learned my addr 172.21.15.81:0/2565349706 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:39.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.398+0000 7fdcc9933640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdcc40c9ae0 0x7fdcc41f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.398+0000 7fdcc9132640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcc40ce550 0x7fdcc406cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.398+0000 7fdcca134640 1 -- 172.21.15.81:0/2565349706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcc40ce550 msgr2=0x7fdcc406cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.398+0000 7fdcca134640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcc40ce550 0x7fdcc406cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.398+0000 7fdcca134640 1 -- 172.21.15.81:0/2565349706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdcc40c9ae0 msgr2=0x7fdcc41f6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.398+0000 7fdcca134640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdcc40c9ae0 0x7fdcc41f6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.398+0000 7fdcca134640 1 -- 172.21.15.81:0/2565349706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc009a70 con 0x7fdcc415d280 2023-11-08T16:41:39.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.398+0000 7fdcca134640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcc415d280 0x7fdcc4148dd0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009d90 tx=0x7fdcbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:39.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.399+0000 7fdcba7fc640 1 -- 172.21.15.81:0/2565349706 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc009450 con 0x7fdcc415d280 2023-11-08T16:41:39.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.399+0000 7fdcba7fc640 1 -- 172.21.15.81:0/2565349706 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcbc040450 con 0x7fdcc415d280 2023-11-08T16:41:39.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.399+0000 7fdcba7fc640 1 -- 172.21.15.81:0/2565349706 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc040800 con 0x7fdcc415d280 2023-11-08T16:41:39.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.399+0000 7fdccbb97640 1 -- 172.21.15.81:0/2565349706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc414c050 con 0x7fdcc415d280 2023-11-08T16:41:39.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.399+0000 7fdccbb97640 1 -- 172.21.15.81:0/2565349706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc414c610 con 0x7fdcc415d280 2023-11-08T16:41:39.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.400+0000 7fdccbb97640 1 -- 172.21.15.81:0/2565349706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc4074440 con 0x7fdcc415d280 2023-11-08T16:41:39.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.401+0000 7fdcba7fc640 1 -- 172.21.15.81:0/2565349706 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdcbc040960 con 0x7fdcc415d280 2023-11-08T16:41:39.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.402+0000 7fdcba7fc640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdca806aea0 0x7fdca806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.402+0000 7fdcba7fc640 1 -- 172.21.15.81:0/2565349706 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fdcbc0abf30 con 0x7fdcc415d280 2023-11-08T16:41:39.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.402+0000 7fdcc9933640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdca806aea0 0x7fdca806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.403+0000 7fdcc9933640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdca806aea0 0x7fdca806d360 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009a30 tx=0x7fdcc0008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:39.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.406+0000 7fdcba7fc640 1 -- 172.21.15.81:0/2565349706 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fdcbc076f70 con 0x7fdcc415d280 2023-11-08T16:41:39.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.595+0000 7fdccbb97640 1 -- 172.21.15.81:0/2565349706 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdcc40b4af0 con 0x7fdcc415d280 2023-11-08T16:41:39.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.596+0000 7fdcba7fc640 1 -- 172.21.15.81:0/2565349706 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcbc07a590 con 0x7fdcc415d280 2023-11-08T16:41:39.599 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:41:39.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 -- 172.21.15.81:0/2565349706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdca806aea0 msgr2=0x7fdca806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdca806aea0 0x7fdca806d360 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0009a30 tx=0x7fdcc0008040 comp rx=0 tx=0).stop 2023-11-08T16:41:39.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 -- 172.21.15.81:0/2565349706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcc415d280 msgr2=0x7fdcc4148dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcc415d280 0x7fdcc4148dd0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc009d90 tx=0x7fdcbc004a40 comp rx=0 tx=0).stop 2023-11-08T16:41:39.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 -- 172.21.15.81:0/2565349706 shutdown_connections 2023-11-08T16:41:39.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdca806aea0 0x7fdca806d360 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fdcc415d280 0x7fdcc4148dd0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fdcc40ce550 0x7fdcc406cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 --2- 172.21.15.81:0/2565349706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdcc40c9ae0 0x7fdcc41f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 -- 172.21.15.81:0/2565349706 >> 172.21.15.81:0/2565349706 conn(0x7fdcc40ceb70 msgr2=0x7fdcc40ad270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:39.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 -- 172.21.15.81:0/2565349706 shutdown_connections 2023-11-08T16:41:39.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.599+0000 7fdccbb97640 1 -- 172.21.15.81:0/2565349706 wait complete. 2023-11-08T16:41:39.620 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:41:39.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.831+0000 7fea23c1a640 1 Processor -- start 2023-11-08T16:41:39.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.831+0000 7fea23c1a640 1 -- start start 2023-11-08T16:41:39.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.833+0000 7fea23c1a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fea1c0ae220 0x7fea1c160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.833+0000 7fea23c1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea1c15d3a0 0x7fea1c0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.833+0000 7fea23c1a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1c161650 con 0x7fea1c15cfc0 2023-11-08T16:41:39.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.833+0000 7fea23c1a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea1c161930 con 0x7fea1c0ae220 2023-11-08T16:41:39.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.833+0000 7fea23c1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c161c10 con 0x7fea1c15d3a0 2023-11-08T16:41:39.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.833+0000 7fea219b6640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fea1c15cfc0 0x7fea1c06fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56220/0 (socket says 172.21.15.81:56220) 2023-11-08T16:41:39.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.833+0000 7fea219b6640 1 -- 172.21.15.81:0/3243342948 learned_addr learned my addr 172.21.15.81:0/3243342948 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:39.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.833+0000 7fea221b7640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea1c15d3a0 0x7fea1c0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.833+0000 7fea211b5640 1 --2- 172.21.15.81:0/3243342948 >> v2:172.21.15.93:3300/0 conn(0x7fea1c0ae220 0x7fea1c160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128429465 0 0) 0x7fea1c161650 con 0x7fea1c15cfc0 2023-11-08T16:41:39.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea08003520 con 0x7fea1c15cfc0 2023-11-08T16:41:39.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3261845362 0 0) 0x7fea08003520 con 0x7fea1c15cfc0 2023-11-08T16:41:39.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea1c15d3a0 msgr2=0x7fea1c0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:39.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea1c15d3a0 0x7fea1c0cb5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 >> v2:172.21.15.93:3300/0 conn(0x7fea1c0ae220 msgr2=0x7fea1c160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:39.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 --2- 172.21.15.81:0/3243342948 >> v2:172.21.15.93:3300/0 conn(0x7fea1c0ae220 0x7fea1c160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c0c92d0 con 0x7fea1c15cfc0 2023-11-08T16:41:39.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fea18001670 con 0x7fea1c15cfc0 2023-11-08T16:41:39.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 >> v1:172.21.15.81:6789/0 conn(0x7fea1c15cfc0 legacy=0x7fea1c06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea1c15d3a0 0x7fea08003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08003ef0 0x7fea080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea08004d00 0x7fea08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c161930 con 0x7fea08004d00 2023-11-08T16:41:39.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c161c10 con 0x7fea08003ef0 2023-11-08T16:41:39.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.834+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0800b9b0 con 0x7fea1c15d3a0 2023-11-08T16:41:39.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.835+0000 7fea221b7640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea08004d00 0x7fea08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.835+0000 7fea211b5640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08003ef0 0x7fea080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.835+0000 7fea219b6640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea1c15d3a0 0x7fea08003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.835+0000 7fea221b7640 1 -- 172.21.15.81:0/3243342948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea1c15d3a0 msgr2=0x7fea08003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:39.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.835+0000 7fea221b7640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea1c15d3a0 0x7fea08003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.835+0000 7fea221b7640 1 -- 172.21.15.81:0/3243342948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08003ef0 msgr2=0x7fea080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:39.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.835+0000 7fea221b7640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea08003ef0 0x7fea080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.835+0000 7fea221b7640 1 -- 172.21.15.81:0/3243342948 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea08003520 con 0x7fea08004d00 2023-11-08T16:41:39.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.836+0000 7fea221b7640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea08004d00 0x7fea08005180 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fea14009fc0 tx=0x7fea080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0eb6eb63de30aa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:39.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.836+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1402cc20 con 0x7fea08004d00 2023-11-08T16:41:39.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.836+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1400f040 con 0x7fea08004d00 2023-11-08T16:41:39.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.836+0000 7fea209b4640 1 -- 172.21.15.81:0/3243342948 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea14034440 con 0x7fea08004d00 2023-11-08T16:41:39.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.837+0000 7fea23c1a640 1 -- 172.21.15.81:0/3243342948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea08004d00 msgr2=0x7fea08005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.837+0000 7fea23c1a640 1 --2- 172.21.15.81:0/3243342948 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea08004d00 0x7fea08005180 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fea14009fc0 tx=0x7fea080057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.837+0000 7fea221b7640 1 -- 172.21.15.81:0/3243342948 reap_dead start 2023-11-08T16:41:39.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.837+0000 7fea23c1a640 1 -- 172.21.15.81:0/3243342948 shutdown_connections 2023-11-08T16:41:39.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.837+0000 7fea23c1a640 1 -- 172.21.15.81:0/3243342948 >> 172.21.15.81:0/3243342948 conn(0x7fea1c0b0770 msgr2=0x7fea1c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:39.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.837+0000 7fea23c1a640 1 -- 172.21.15.81:0/3243342948 shutdown_connections 2023-11-08T16:41:39.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.837+0000 7fea23c1a640 1 -- 172.21.15.81:0/3243342948 wait complete. 2023-11-08T16:41:39.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.839+0000 7fea23c1a640 1 Processor -- start 2023-11-08T16:41:39.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.839+0000 7fea23c1a640 1 -- start start 2023-11-08T16:41:39.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.840+0000 7fea23c1a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea08004d00 0x7fea1c070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.840+0000 7fea23c1a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea1c15f2b0 0x7fea1c0c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.840+0000 7fea23c1a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea1c1f5860 0x7fea1c1f5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.840+0000 7fea23c1a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0c38c0 con 0x7fea1c15f2b0 2023-11-08T16:41:39.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.840+0000 7fea23c1a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c0c3a30 con 0x7fea1c1f5860 2023-11-08T16:41:39.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.840+0000 7fea23c1a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1c1f61e0 con 0x7fea08004d00 2023-11-08T16:41:39.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.840+0000 7fea211b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea1c15f2b0 0x7fea1c0c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.840+0000 7fea211b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea1c15f2b0 0x7fea1c0c83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37686/0 (socket says 172.21.15.81:37686) 2023-11-08T16:41:39.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.840+0000 7fea211b5640 1 -- 172.21.15.81:0/3557254441 learned_addr learned my addr 172.21.15.81:0/3557254441 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:39.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea221b7640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea1c1f5860 0x7fea1c1f5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea219b6640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea08004d00 0x7fea1c070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea211b5640 1 -- 172.21.15.81:0/3557254441 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea08004d00 msgr2=0x7fea1c070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea211b5640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea08004d00 0x7fea1c070ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea211b5640 1 -- 172.21.15.81:0/3557254441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea1c1f5860 msgr2=0x7fea1c1f5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:39.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea211b5640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea1c1f5860 0x7fea1c1f5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:39.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea211b5640 1 -- 172.21.15.81:0/3557254441 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c15f820 con 0x7fea1c15f2b0 2023-11-08T16:41:39.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea219b6640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea08004d00 0x7fea1c070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:39.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea221b7640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea1c1f5860 0x7fea1c1f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:39.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea211b5640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea1c15f2b0 0x7fea1c0c83b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fea0c011630 tx=0x7fea0c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:39.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea127fc640 1 -- 172.21.15.81:0/3557254441 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c00f920 con 0x7fea1c15f2b0 2023-11-08T16:41:39.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea127fc640 1 -- 172.21.15.81:0/3557254441 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0c00fa80 con 0x7fea1c15f2b0 2023-11-08T16:41:39.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.841+0000 7fea127fc640 1 -- 172.21.15.81:0/3557254441 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0c00fdf0 con 0x7fea1c15f2b0 2023-11-08T16:41:39.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.842+0000 7fea23c1a640 1 -- 172.21.15.81:0/3557254441 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1c15fb60 con 0x7fea1c15f2b0 2023-11-08T16:41:39.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.842+0000 7fea23c1a640 1 -- 172.21.15.81:0/3557254441 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1c0713e0 con 0x7fea1c15f2b0 2023-11-08T16:41:39.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.843+0000 7fea23c1a640 1 -- 172.21.15.81:0/3557254441 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1c074440 con 0x7fea1c15f2b0 2023-11-08T16:41:39.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.844+0000 7fea127fc640 1 -- 172.21.15.81:0/3557254441 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea0c015460 con 0x7fea1c15f2b0 2023-11-08T16:41:39.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.845+0000 7fea127fc640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea0006ae70 0x7fea0006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:39.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.845+0000 7fea127fc640 1 -- 172.21.15.81:0/3557254441 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fea0c089bf0 con 0x7fea1c15f2b0 2023-11-08T16:41:39.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.845+0000 7fea219b6640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea0006ae70 0x7fea0006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:39.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.846+0000 7fea219b6640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea0006ae70 0x7fea0006d330 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fea18000f60 tx=0x7fea18000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:39.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:39.849+0000 7fea127fc640 1 -- 172.21.15.81:0/3557254441 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fea0c054ba0 con 0x7fea1c15f2b0 2023-11-08T16:41:40.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.051+0000 7fea23c1a640 1 -- 172.21.15.81:0/3557254441 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fea1c0c5500 con 0x7fea1c15f2b0 2023-11-08T16:41:40.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.052+0000 7fea127fc640 1 -- 172.21.15.81:0/3557254441 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v128) v1 ==== 80+0+122 (secure 0 0 0) 0x7fea0c022080 con 0x7fea1c15f2b0 2023-11-08T16:41:40.055 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.93:6835/4021510722 2023-11-09T16:41:29.470203+0000 2023-11-08T16:41:40.055 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.93:6833/4021510722 2023-11-09T16:41:29.470203+0000 2023-11-08T16:41:40.056 INFO:teuthology.orchestra.run.smithi081.stderr:listed 2 entries 2023-11-08T16:41:40.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 -- 172.21.15.81:0/3557254441 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea0006ae70 msgr2=0x7fea0006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea0006ae70 0x7fea0006d330 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fea18000f60 tx=0x7fea18000ed0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 -- 172.21.15.81:0/3557254441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea1c15f2b0 msgr2=0x7fea1c0c83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea1c15f2b0 0x7fea1c0c83b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fea0c011630 tx=0x7fea0c011af0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 -- 172.21.15.81:0/3557254441 shutdown_connections 2023-11-08T16:41:40.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fea0006ae70 0x7fea0006d330 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fea1c1f5860 0x7fea1c1f5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fea1c15f2b0 0x7fea1c0c83b0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 --2- 172.21.15.81:0/3557254441 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fea08004d00 0x7fea1c070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 -- 172.21.15.81:0/3557254441 >> 172.21.15.81:0/3557254441 conn(0x7fea1c0b0770 msgr2=0x7fea1c0ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:40.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 -- 172.21.15.81:0/3557254441 shutdown_connections 2023-11-08T16:41:40.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.055+0000 7fea23c1a640 1 -- 172.21.15.81:0/3557254441 wait complete. 2023-11-08T16:41:40.076 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:41:40.077 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:41:40.126 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:41:40.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.382+0000 7fabf407c640 1 Processor -- start 2023-11-08T16:41:40.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.383+0000 7fabf407c640 1 -- start start 2023-11-08T16:41:40.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.384+0000 7fabf407c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0b3550 0x7fabec15b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.384+0000 7fabf407c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fabec06c9f0 0x7fabec144230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.384+0000 7fabf407c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabec0b2310 con 0x7fabec0b3170 2023-11-08T16:41:40.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.384+0000 7fabf407c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabec0b2bb0 con 0x7fabec06c9f0 2023-11-08T16:41:40.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.384+0000 7fabf407c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0b25f0 con 0x7fabec0b3550 2023-11-08T16:41:40.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.385+0000 7fabf1e18640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fabec0b3170 0x7fabec0b1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56232/0 (socket says 172.21.15.81:56232) 2023-11-08T16:41:40.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.385+0000 7fabf1e18640 1 -- 172.21.15.81:0/80593433 learned_addr learned my addr 172.21.15.81:0/80593433 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:40.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.385+0000 7fabf1617640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0b3550 0x7fabec15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.385+0000 7fabf2619640 1 --2- 172.21.15.81:0/80593433 >> v2:172.21.15.93:3300/0 conn(0x7fabec06c9f0 0x7fabec144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.385+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767137080 0 0) 0x7fabec0b2310 con 0x7fabec0b3170 2023-11-08T16:41:40.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.385+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd8003520 con 0x7fabec0b3170 2023-11-08T16:41:40.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.385+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2495633321 0 0) 0x7fabd8003520 con 0x7fabec0b3170 2023-11-08T16:41:40.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0b3550 msgr2=0x7fabec15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:40.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0b3550 0x7fabec15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 >> v2:172.21.15.93:3300/0 conn(0x7fabec06c9f0 msgr2=0x7fabec144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:40.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 --2- 172.21.15.81:0/80593433 >> v2:172.21.15.93:3300/0 conn(0x7fabec06c9f0 0x7fabec144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec0b28d0 con 0x7fabec0b3170 2023-11-08T16:41:40.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fabe8001670 con 0x7fabec0b3170 2023-11-08T16:41:40.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 >> v1:172.21.15.81:6789/0 conn(0x7fabec0b3170 legacy=0x7fabec0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0b3550 0x7fabd8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8003f40 0x7fabd80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd8004d00 0x7fabd8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0b2bb0 con 0x7fabd8003f40 2023-11-08T16:41:40.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec0b25f0 con 0x7fabd8004d00 2023-11-08T16:41:40.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd800ba30 con 0x7fabec0b3550 2023-11-08T16:41:40.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf1617640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8003f40 0x7fabd80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf1617640 1 -- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0b3550 msgr2=0x7fabd8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:40.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf1e18640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0b3550 0x7fabd8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf1617640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0b3550 0x7fabd8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf2619640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd8004d00 0x7fabd8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf1617640 1 -- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd8004d00 msgr2=0x7fabd8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:40.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf1617640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd8004d00 0x7fabd8005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf1617640 1 -- 172.21.15.81:0/80593433 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8003520 con 0x7fabd8003f40 2023-11-08T16:41:40.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.386+0000 7fabf1e18640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec0b3550 0x7fabd8003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:40.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.387+0000 7fabf1617640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8003f40 0x7fabd80047c0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009ac0 tx=0x7fabd80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d9ecc2bd27fe9a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:40.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.388+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc02c6b0 con 0x7fabd8003f40 2023-11-08T16:41:40.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.388+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc02c810 con 0x7fabd8003f40 2023-11-08T16:41:40.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.388+0000 7fabf0e16640 1 -- 172.21.15.81:0/80593433 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc02cba0 con 0x7fabd8003f40 2023-11-08T16:41:40.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.388+0000 7fabf407c640 1 -- 172.21.15.81:0/80593433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8003f40 msgr2=0x7fabd80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.388+0000 7fabf407c640 1 --2- 172.21.15.81:0/80593433 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabd8003f40 0x7fabd80047c0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fabdc009ac0 tx=0x7fabd80057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.388+0000 7fabf2619640 1 -- 172.21.15.81:0/80593433 reap_dead start 2023-11-08T16:41:40.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.388+0000 7fabf407c640 1 -- 172.21.15.81:0/80593433 shutdown_connections 2023-11-08T16:41:40.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.388+0000 7fabf407c640 1 -- 172.21.15.81:0/80593433 >> 172.21.15.81:0/80593433 conn(0x7fabec06a6a0 msgr2=0x7fabec0b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:40.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.388+0000 7fabf407c640 1 -- 172.21.15.81:0/80593433 shutdown_connections 2023-11-08T16:41:40.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.388+0000 7fabf407c640 1 -- 172.21.15.81:0/80593433 wait complete. 2023-11-08T16:41:40.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.390+0000 7fabf407c640 1 Processor -- start 2023-11-08T16:41:40.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.390+0000 7fabf407c640 1 -- start start 2023-11-08T16:41:40.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.391+0000 7fabf407c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd8003f40 0x7fabec159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.391+0000 7fabf407c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabec13f960 0x7fabec13fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.391+0000 7fabf407c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec06fee0 0x7fabec070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.391+0000 7fabf407c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec13bf60 con 0x7fabec13f960 2023-11-08T16:41:40.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.391+0000 7fabf407c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabec140320 con 0x7fabd8003f40 2023-11-08T16:41:40.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.391+0000 7fabf407c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabec13c380 con 0x7fabec06fee0 2023-11-08T16:41:40.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.391+0000 7fabf1617640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabec13f960 0x7fabec13fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.392+0000 7fabf1617640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabec13f960 0x7fabec13fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37706/0 (socket says 172.21.15.81:37706) 2023-11-08T16:41:40.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.392+0000 7fabf1617640 1 -- 172.21.15.81:0/2380101070 learned_addr learned my addr 172.21.15.81:0/2380101070 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:40.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.392+0000 7fabf1617640 1 -- 172.21.15.81:0/2380101070 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec06fee0 msgr2=0x7fabec070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.392+0000 7fabf1617640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec06fee0 0x7fabec070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.392+0000 7fabf1617640 1 -- 172.21.15.81:0/2380101070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd8003f40 msgr2=0x7fabec159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.392+0000 7fabf1617640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd8003f40 0x7fabec159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.392+0000 7fabf1617640 1 -- 172.21.15.81:0/2380101070 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabec06d000 con 0x7fabec13f960 2023-11-08T16:41:40.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.392+0000 7fabf1617640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabec13f960 0x7fabec13fde0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fabdc02c250 tx=0x7fabdc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:40.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.392+0000 7fabe27fc640 1 -- 172.21.15.81:0/2380101070 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc003810 con 0x7fabec13f960 2023-11-08T16:41:40.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.392+0000 7fabe27fc640 1 -- 172.21.15.81:0/2380101070 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabdc003970 con 0x7fabec13f960 2023-11-08T16:41:40.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.393+0000 7fabf407c640 1 -- 172.21.15.81:0/2380101070 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabec13b5c0 con 0x7fabec13f960 2023-11-08T16:41:40.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.393+0000 7fabe27fc640 1 -- 172.21.15.81:0/2380101070 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabdc003ce0 con 0x7fabec13f960 2023-11-08T16:41:40.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.393+0000 7fabf407c640 1 -- 172.21.15.81:0/2380101070 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabec13bb80 con 0x7fabec13f960 2023-11-08T16:41:40.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.394+0000 7fabf407c640 1 -- 172.21.15.81:0/2380101070 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabec074440 con 0x7fabec13f960 2023-11-08T16:41:40.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.395+0000 7fabe27fc640 1 -- 172.21.15.81:0/2380101070 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fabdc047450 con 0x7fabec13f960 2023-11-08T16:41:40.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.396+0000 7fabe27fc640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabd006aea0 0x7fabd006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.396+0000 7fabe27fc640 1 -- 172.21.15.81:0/2380101070 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fabdc0ab8c0 con 0x7fabec13f960 2023-11-08T16:41:40.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.396+0000 7fabf1e18640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabd006aea0 0x7fabd006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.397+0000 7fabf1e18640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabd006aea0 0x7fabd006d360 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fabe8001820 tx=0x7fabe8009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:40.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.400+0000 7fabe27fc640 1 -- 172.21.15.81:0/2380101070 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fabdc076a40 con 0x7fabec13f960 2023-11-08T16:41:40.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.591+0000 7fabf407c640 1 -- 172.21.15.81:0/2380101070 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabec13c700 con 0x7fabec13f960 2023-11-08T16:41:40.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.593+0000 7fabe27fc640 1 -- 172.21.15.81:0/2380101070 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16596 (secure 0 0 0) 0x7fabdc0044e0 con 0x7fabec13f960 2023-11-08T16:41:40.596 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:40.596 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":128,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:29.470231+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:41:03.545565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T16:41:04.726492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6835/4021510722":"2023-11-09T16:41:29.470203+0000","172.21.15.93:6833/4021510722":"2023-11-09T16:41:29.470203+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:41:40.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 -- 172.21.15.81:0/2380101070 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabd006aea0 msgr2=0x7fabd006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabd006aea0 0x7fabd006d360 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fabe8001820 tx=0x7fabe8009040 comp rx=0 tx=0).stop 2023-11-08T16:41:40.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 -- 172.21.15.81:0/2380101070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabec13f960 msgr2=0x7fabec13fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabec13f960 0x7fabec13fde0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fabdc02c250 tx=0x7fabdc004730 comp rx=0 tx=0).stop 2023-11-08T16:41:40.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 -- 172.21.15.81:0/2380101070 shutdown_connections 2023-11-08T16:41:40.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fabd006aea0 0x7fabd006d360 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fabec06fee0 0x7fabec070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fabec13f960 0x7fabec13fde0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 --2- 172.21.15.81:0/2380101070 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fabd8003f40 0x7fabec159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 -- 172.21.15.81:0/2380101070 >> 172.21.15.81:0/2380101070 conn(0x7fabec06a6a0 msgr2=0x7fabec0cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:40.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 -- 172.21.15.81:0/2380101070 shutdown_connections 2023-11-08T16:41:40.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.596+0000 7fabf407c640 1 -- 172.21.15.81:0/2380101070 wait complete. 2023-11-08T16:41:40.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:41:03.545565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:41:04.726492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:41:40.619 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:41:40.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.836+0000 7f6d91b3c640 1 Processor -- start 2023-11-08T16:41:40.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.836+0000 7f6d91b3c640 1 -- start start 2023-11-08T16:41:40.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.838+0000 7f6d91b3c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f6d8c161b00 0x7f6d8c0af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.838+0000 7f6d91b3c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c06a280 0x7f6d8c06a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.838+0000 7f6d91b3c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8c0aed20 con 0x7f6d8c0ae940 2023-11-08T16:41:40.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.838+0000 7f6d91b3c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8c161f80 con 0x7f6d8c161b00 2023-11-08T16:41:40.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.838+0000 7f6d91b3c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c06ac50 con 0x7f6d8c06a280 2023-11-08T16:41:40.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.838+0000 7f6d8b7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f6d8c0ae940 0x7f6d8c0cf2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56240/0 (socket says 172.21.15.81:56240) 2023-11-08T16:41:40.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.838+0000 7f6d8b7fe640 1 -- 172.21.15.81:0/4288956105 learned_addr learned my addr 172.21.15.81:0/4288956105 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:40.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.838+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2318705661 0 0) 0x7f6d8c0aed20 con 0x7f6d8c0ae940 2023-11-08T16:41:40.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.838+0000 7f6d8affd640 1 --2- 172.21.15.81:0/4288956105 >> v2:172.21.15.93:3300/0 conn(0x7f6d8c161b00 0x7f6d8c0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.838+0000 7f6d8bfff640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c06a280 0x7f6d8c06a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.838+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d74003520 con 0x7f6d8c0ae940 2023-11-08T16:41:40.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1194014885 0 0) 0x7f6d74003520 con 0x7f6d8c0ae940 2023-11-08T16:41:40.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c06a280 msgr2=0x7f6d8c06a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:40.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c06a280 0x7f6d8c06a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 >> v2:172.21.15.93:3300/0 conn(0x7f6d8c161b00 msgr2=0x7f6d8c0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:40.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 --2- 172.21.15.81:0/4288956105 >> v2:172.21.15.93:3300/0 conn(0x7f6d8c161b00 0x7f6d8c0af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0cfab0 con 0x7f6d8c0ae940 2023-11-08T16:41:40.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f6d7c001670 con 0x7f6d8c0ae940 2023-11-08T16:41:40.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 >> v1:172.21.15.81:6789/0 conn(0x7f6d8c0ae940 legacy=0x7f6d8c0cf2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d74003970 0x7f6d74003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d740047c0 0x7f6d74004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c06a280 0x7f6d74005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c161f80 con 0x7f6d740047c0 2023-11-08T16:41:40.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c06ac50 con 0x7f6d74003970 2023-11-08T16:41:40.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d7400b980 con 0x7f6d8c06a280 2023-11-08T16:41:40.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.839+0000 7f6d8affd640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d740047c0 0x7f6d74004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.840+0000 7f6d8bfff640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c06a280 0x7f6d74005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.840+0000 7f6d8b7fe640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d74003970 0x7f6d74003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.840+0000 7f6d8affd640 1 -- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c06a280 msgr2=0x7f6d74005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:40.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.840+0000 7f6d8affd640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c06a280 0x7f6d74005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.840+0000 7f6d8affd640 1 -- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d74003970 msgr2=0x7f6d74003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:40.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.840+0000 7f6d8affd640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d74003970 0x7f6d74003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.840+0000 7f6d8affd640 1 -- 172.21.15.81:0/4288956105 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d74003520 con 0x7f6d740047c0 2023-11-08T16:41:40.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.840+0000 7f6d8b7fe640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d74003970 0x7f6d74003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:40.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.841+0000 7f6d8affd640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d740047c0 0x7f6d74004c40 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009ac0 tx=0x7f6d740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e4dae8302c1db9a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:40.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.841+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8002c710 con 0x7f6d740047c0 2023-11-08T16:41:40.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.841+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8002c870 con 0x7f6d740047c0 2023-11-08T16:41:40.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.841+0000 7f6d8a7fc640 1 -- 172.21.15.81:0/4288956105 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d8002cc00 con 0x7f6d740047c0 2023-11-08T16:41:40.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.841+0000 7f6d91b3c640 1 -- 172.21.15.81:0/4288956105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d740047c0 msgr2=0x7f6d74004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.841+0000 7f6d91b3c640 1 --2- 172.21.15.81:0/4288956105 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d740047c0 0x7f6d74004c40 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009ac0 tx=0x7f6d740057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.841+0000 7f6d8bfff640 1 -- 172.21.15.81:0/4288956105 reap_dead start 2023-11-08T16:41:40.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.842+0000 7f6d91b3c640 1 -- 172.21.15.81:0/4288956105 shutdown_connections 2023-11-08T16:41:40.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.842+0000 7f6d91b3c640 1 -- 172.21.15.81:0/4288956105 >> 172.21.15.81:0/4288956105 conn(0x7f6d8c071770 msgr2=0x7f6d8c071b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.842+0000 7f6d91b3c640 1 -- 172.21.15.81:0/4288956105 shutdown_connections 2023-11-08T16:41:40.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.842+0000 7f6d91b3c640 1 -- 172.21.15.81:0/4288956105 wait complete. 2023-11-08T16:41:40.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.843+0000 7f6d91b3c640 1 Processor -- start 2023-11-08T16:41:40.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.844+0000 7f6d91b3c640 1 -- start start 2023-11-08T16:41:40.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.844+0000 7f6d91b3c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d740047c0 0x7f6d8c098a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.844+0000 7f6d91b3c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d8c0adda0 0x7f6d8c0c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.844+0000 7f6d91b3c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c0c36e0 0x7f6d8c0c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.844+0000 7f6d91b3c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0acab0 con 0x7f6d8c0adda0 2023-11-08T16:41:40.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.844+0000 7f6d91b3c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0ad910 con 0x7f6d740047c0 2023-11-08T16:41:40.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.844+0000 7f6d91b3c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8c0acd00 con 0x7f6d8c0c36e0 2023-11-08T16:41:40.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.845+0000 7f6d8affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d8c0adda0 0x7f6d8c0c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.845+0000 7f6d8affd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d8c0adda0 0x7f6d8c0c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:37712/0 (socket says 172.21.15.81:37712) 2023-11-08T16:41:40.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.845+0000 7f6d8affd640 1 -- 172.21.15.81:0/2764286527 learned_addr learned my addr 172.21.15.81:0/2764286527 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:40.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.845+0000 7f6d8affd640 1 -- 172.21.15.81:0/2764286527 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c0c36e0 msgr2=0x7f6d8c0c3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.845+0000 7f6d8affd640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c0c36e0 0x7f6d8c0c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.845+0000 7f6d8affd640 1 -- 172.21.15.81:0/2764286527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d740047c0 msgr2=0x7f6d8c098a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:40.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.845+0000 7f6d8affd640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d740047c0 0x7f6d8c098a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:40.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.845+0000 7f6d8affd640 1 -- 172.21.15.81:0/2764286527 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d8c0acfe0 con 0x7f6d8c0adda0 2023-11-08T16:41:40.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.845+0000 7f6d8affd640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d8c0adda0 0x7f6d8c0c31a0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f6d8002c2b0 tx=0x7f6d80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:40.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.846+0000 7f6d73fff640 1 -- 172.21.15.81:0/2764286527 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80003810 con 0x7f6d8c0adda0 2023-11-08T16:41:40.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.846+0000 7f6d73fff640 1 -- 172.21.15.81:0/2764286527 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d8002de90 con 0x7f6d8c0adda0 2023-11-08T16:41:40.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.846+0000 7f6d91b3c640 1 -- 172.21.15.81:0/2764286527 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d8c0ad2c0 con 0x7f6d8c0adda0 2023-11-08T16:41:40.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.846+0000 7f6d73fff640 1 -- 172.21.15.81:0/2764286527 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d80003b70 con 0x7f6d8c0adda0 2023-11-08T16:41:40.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.846+0000 7f6d91b3c640 1 -- 172.21.15.81:0/2764286527 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d8c0ae220 con 0x7f6d8c0adda0 2023-11-08T16:41:40.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.847+0000 7f6d91b3c640 1 -- 172.21.15.81:0/2764286527 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8c074440 con 0x7f6d8c0adda0 2023-11-08T16:41:40.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.848+0000 7f6d73fff640 1 -- 172.21.15.81:0/2764286527 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6d80003cd0 con 0x7f6d8c0adda0 2023-11-08T16:41:40.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.849+0000 7f6d73fff640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d7806aea0 0x7f6d7806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:40.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.849+0000 7f6d73fff640 1 -- 172.21.15.81:0/2764286527 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6d800ab6c0 con 0x7f6d8c0adda0 2023-11-08T16:41:40.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.849+0000 7f6d8b7fe640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d7806aea0 0x7f6d7806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:40.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.850+0000 7f6d8b7fe640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d7806aea0 0x7f6d7806d360 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0018e0 tx=0x7f6d7c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:40.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:40.853+0000 7f6d73fff640 1 -- 172.21.15.81:0/2764286527 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6d80076790 con 0x7f6d8c0adda0 2023-11-08T16:41:41.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.040+0000 7f6d91b3c640 1 -- 172.21.15.81:0/2764286527 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d8c0ada80 con 0x7f6d8c0adda0 2023-11-08T16:41:41.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.041+0000 7f6d73fff640 1 -- 172.21.15.81:0/2764286527 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d80079db0 con 0x7f6d8c0adda0 2023-11-08T16:41:41.043 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:41:41.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 -- 172.21.15.81:0/2764286527 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d7806aea0 msgr2=0x7f6d7806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d7806aea0 0x7f6d7806d360 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c0018e0 tx=0x7f6d7c009040 comp rx=0 tx=0).stop 2023-11-08T16:41:41.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 -- 172.21.15.81:0/2764286527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d8c0adda0 msgr2=0x7f6d8c0c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d8c0adda0 0x7f6d8c0c31a0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f6d8002c2b0 tx=0x7f6d80004730 comp rx=0 tx=0).stop 2023-11-08T16:41:41.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 -- 172.21.15.81:0/2764286527 shutdown_connections 2023-11-08T16:41:41.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6d7806aea0 0x7f6d7806d360 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6d8c0c36e0 0x7f6d8c0c3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6d8c0adda0 0x7f6d8c0c31a0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 --2- 172.21.15.81:0/2764286527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6d740047c0 0x7f6d8c098a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 -- 172.21.15.81:0/2764286527 >> 172.21.15.81:0/2764286527 conn(0x7f6d8c071770 msgr2=0x7f6d8c0ce790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:41.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 -- 172.21.15.81:0/2764286527 shutdown_connections 2023-11-08T16:41:41.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.043+0000 7f6d91b3c640 1 -- 172.21.15.81:0/2764286527 wait complete. 2023-11-08T16:41:41.064 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:41:41.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.268+0000 7ffac320d640 1 Processor -- start 2023-11-08T16:41:41.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.268+0000 7ffac320d640 1 -- start start 2023-11-08T16:41:41.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.270+0000 7ffac320d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ffabc0b3380 0x7ffabc0b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.270+0000 7ffac320d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0d19d0 0x7ffabc0c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.270+0000 7ffac320d640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffabc0cdb20 con 0x7ffabc0b37e0 2023-11-08T16:41:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.270+0000 7ffac320d640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffabc0d38a0 con 0x7ffabc0b3380 2023-11-08T16:41:41.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.270+0000 7ffac320d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0d0b60 con 0x7ffabc0d19d0 2023-11-08T16:41:41.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.271+0000 7ffac220b640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ffabc0b3380 0x7ffabc0b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.271+0000 7ffac2a0c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0d19d0 0x7ffabc0c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.271+0000 7ffac220b640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7ffabc0b3380 0x7ffabc0b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:34284/0 (socket says 172.21.15.81:34284) 2023-11-08T16:41:41.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.271+0000 7ffac220b640 1 -- 172.21.15.81:0/2420120876 learned_addr learned my addr 172.21.15.81:0/2420120876 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:41.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.271+0000 7ffac1a0a640 1 --1- 172.21.15.81:0/2420120876 >> v1:172.21.15.93:6789/0 conn(0x7ffabc0b37e0 0x7ffabc0d12c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.93:6789/0 says I am v1:172.21.15.81:54598/0 (socket says 172.21.15.81:54598) 2023-11-08T16:41:41.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.271+0000 7ffac2a0c640 1 -- 172.21.15.81:0/2420120876 >> v2:172.21.15.93:3300/0 conn(0x7ffabc0b3380 msgr2=0x7ffabc0b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:41.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.271+0000 7ffac2a0c640 1 --2- 172.21.15.81:0/2420120876 >> v2:172.21.15.93:3300/0 conn(0x7ffabc0b3380 0x7ffabc0b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.271+0000 7ffac2a0c640 1 -- 172.21.15.81:0/2420120876 >> v1:172.21.15.93:6789/0 conn(0x7ffabc0b37e0 legacy=0x7ffabc0d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.271+0000 7ffac2a0c640 1 -- 172.21.15.81:0/2420120876 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc0d3b40 con 0x7ffabc0d19d0 2023-11-08T16:41:41.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.273+0000 7ffac2a0c640 1 --2- 172.21.15.81:0/2420120876 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0d19d0 0x7ffabc0c6830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009d90 tx=0x7ffab402bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83c1d414c5901e1e server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:41.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.273+0000 7ffac1209640 1 -- 172.21.15.81:0/2420120876 <== mon.2 v2:172.21.15.93:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab402cc50 con 0x7ffabc0d19d0 2023-11-08T16:41:41.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.273+0000 7ffac1209640 1 -- 172.21.15.81:0/2420120876 <== mon.2 v2:172.21.15.93:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab402cdb0 con 0x7ffabc0d19d0 2023-11-08T16:41:41.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.273+0000 7ffac1209640 1 -- 172.21.15.81:0/2420120876 <== mon.2 v2:172.21.15.93:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4034660 con 0x7ffabc0d19d0 2023-11-08T16:41:41.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.274+0000 7ffac320d640 1 -- 172.21.15.81:0/2420120876 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0d19d0 msgr2=0x7ffabc0c6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.274+0000 7ffac320d640 1 --2- 172.21.15.81:0/2420120876 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0d19d0 0x7ffabc0c6830 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009d90 tx=0x7ffab402bea0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.274+0000 7ffac320d640 1 -- 172.21.15.81:0/2420120876 shutdown_connections 2023-11-08T16:41:41.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.274+0000 7ffac320d640 1 --2- 172.21.15.81:0/2420120876 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0d19d0 0x7ffabc0c6830 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.274+0000 7ffac320d640 1 --2- 172.21.15.81:0/2420120876 >> v2:172.21.15.93:3300/0 conn(0x7ffabc0b3380 0x7ffabc0b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.274+0000 7ffac320d640 1 -- 172.21.15.81:0/2420120876 >> 172.21.15.81:0/2420120876 conn(0x7ffabc0d2ce0 msgr2=0x7ffabc0d30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:41.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.274+0000 7ffac320d640 1 -- 172.21.15.81:0/2420120876 shutdown_connections 2023-11-08T16:41:41.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.274+0000 7ffac320d640 1 -- 172.21.15.81:0/2420120876 wait complete. 2023-11-08T16:41:41.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.276+0000 7ffac320d640 1 Processor -- start 2023-11-08T16:41:41.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.276+0000 7ffac320d640 1 -- start start 2023-11-08T16:41:41.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.277+0000 7ffac320d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffabc0b3380 0x7ffabc0a0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.277+0000 7ffac320d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0b37e0 0x7ffabc06b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.277+0000 7ffac320d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffabc0d19d0 0x7ffabc06bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.277+0000 7ffac320d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0a08b0 con 0x7ffabc0d19d0 2023-11-08T16:41:41.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.277+0000 7ffac320d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1f3bd0 con 0x7ffabc0b3380 2023-11-08T16:41:41.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.277+0000 7ffac320d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0b5910 con 0x7ffabc0b37e0 2023-11-08T16:41:41.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.277+0000 7ffac2a0c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffabc0d19d0 0x7ffabc06bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.277+0000 7ffac2a0c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffabc0d19d0 0x7ffabc06bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:47970/0 (socket says 172.21.15.81:47970) 2023-11-08T16:41:41.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.277+0000 7ffac2a0c640 1 -- 172.21.15.81:0/1990344161 learned_addr learned my addr 172.21.15.81:0/1990344161 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:41.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.277+0000 7ffac220b640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffabc0b3380 0x7ffabc0a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.277+0000 7ffac1a0a640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0b37e0 0x7ffabc06b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffac2a0c640 1 -- 172.21.15.81:0/1990344161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0b37e0 msgr2=0x7ffabc06b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffac2a0c640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0b37e0 0x7ffabc06b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffac2a0c640 1 -- 172.21.15.81:0/1990344161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffabc0b3380 msgr2=0x7ffabc0a0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffac2a0c640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffabc0b3380 0x7ffabc0a0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffac2a0c640 1 -- 172.21.15.81:0/1990344161 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab4009a70 con 0x7ffabc0d19d0 2023-11-08T16:41:41.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffac1a0a640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0b37e0 0x7ffabc06b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:41.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffac220b640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffabc0b3380 0x7ffabc0a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:41.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffac2a0c640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffabc0d19d0 0x7ffabc06bea0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ffab402c4b0 tx=0x7ffab4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:41.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffab2ffd640 1 -- 172.21.15.81:0/1990344161 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab403e610 con 0x7ffabc0d19d0 2023-11-08T16:41:41.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffab2ffd640 1 -- 172.21.15.81:0/1990344161 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab403e770 con 0x7ffabc0d19d0 2023-11-08T16:41:41.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffab2ffd640 1 -- 172.21.15.81:0/1990344161 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab403eb20 con 0x7ffabc0d19d0 2023-11-08T16:41:41.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffac320d640 1 -- 172.21.15.81:0/1990344161 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc0b4fe0 con 0x7ffabc0d19d0 2023-11-08T16:41:41.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.278+0000 7ffac320d640 1 -- 172.21.15.81:0/1990344161 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc0b55a0 con 0x7ffabc0d19d0 2023-11-08T16:41:41.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.279+0000 7ffac320d640 1 -- 172.21.15.81:0/1990344161 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffabc04ece0 con 0x7ffabc0d19d0 2023-11-08T16:41:41.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.280+0000 7ffab2ffd640 1 -- 172.21.15.81:0/1990344161 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffab403b050 con 0x7ffabc0d19d0 2023-11-08T16:41:41.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.281+0000 7ffab2ffd640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffaa006ae70 0x7ffaa006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.281+0000 7ffab2ffd640 1 -- 172.21.15.81:0/1990344161 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffab40abf30 con 0x7ffabc0d19d0 2023-11-08T16:41:41.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.281+0000 7ffac220b640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffaa006ae70 0x7ffaa006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.282+0000 7ffac220b640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffaa006ae70 0x7ffaa006d330 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0b84a0 tx=0x7ffab8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:41.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.285+0000 7ffab2ffd640 1 -- 172.21.15.81:0/1990344161 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7ffab4077090 con 0x7ffabc0d19d0 2023-11-08T16:41:41.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.475+0000 7ffac320d640 1 -- 172.21.15.81:0/1990344161 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffabc0d10a0 con 0x7ffabc0d19d0 2023-11-08T16:41:41.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.476+0000 7ffab2ffd640 1 -- 172.21.15.81:0/1990344161 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v128) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffab407a6b0 con 0x7ffabc0d19d0 2023-11-08T16:41:41.478 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:41:41.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.478+0000 7ffac320d640 1 -- 172.21.15.81:0/1990344161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffaa006ae70 msgr2=0x7ffaa006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.478+0000 7ffac320d640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffaa006ae70 0x7ffaa006d330 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ffabc0b84a0 tx=0x7ffab8008040 comp rx=0 tx=0).stop 2023-11-08T16:41:41.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.478+0000 7ffac320d640 1 -- 172.21.15.81:0/1990344161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffabc0d19d0 msgr2=0x7ffabc06bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.478+0000 7ffac320d640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffabc0d19d0 0x7ffabc06bea0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ffab402c4b0 tx=0x7ffab4004a40 comp rx=0 tx=0).stop 2023-11-08T16:41:41.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.479+0000 7ffac320d640 1 -- 172.21.15.81:0/1990344161 shutdown_connections 2023-11-08T16:41:41.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.479+0000 7ffac320d640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7ffaa006ae70 0x7ffaa006d330 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.479+0000 7ffac320d640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7ffabc0d19d0 0x7ffabc06bea0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.479+0000 7ffac320d640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7ffabc0b37e0 0x7ffabc06b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.479+0000 7ffac320d640 1 --2- 172.21.15.81:0/1990344161 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffabc0b3380 0x7ffabc0a0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.479+0000 7ffac320d640 1 -- 172.21.15.81:0/1990344161 >> 172.21.15.81:0/1990344161 conn(0x7ffabc0d2ce0 msgr2=0x7ffabc0cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:41.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.479+0000 7ffac320d640 1 -- 172.21.15.81:0/1990344161 shutdown_connections 2023-11-08T16:41:41.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.479+0000 7ffac320d640 1 -- 172.21.15.81:0/1990344161 wait complete. 2023-11-08T16:41:41.501 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:41:41.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.717+0000 7f18b81e2640 1 Processor -- start 2023-11-08T16:41:41.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.718+0000 7f18b81e2640 1 -- start start 2023-11-08T16:41:41.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.719+0000 7f18b81e2640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f18b0158c60 0x7f18b00705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.719+0000 7f18b81e2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b015fad0 con 0x7f18b0071ad0 2023-11-08T16:41:41.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.719+0000 7f18b81e2640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b006d760 con 0x7f18b015f380 2023-11-08T16:41:41.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.719+0000 7f18b81e2640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b006df80 con 0x7f18b0158c60 2023-11-08T16:41:41.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b577d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f18b0071ad0 0x7f18b015ec70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56292/0 (socket says 172.21.15.81:56292) 2023-11-08T16:41:41.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b577d640 1 -- 172.21.15.81:0/1777586247 learned_addr learned my addr 172.21.15.81:0/1777586247 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:41.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b5f7e640 1 --2- 172.21.15.81:0/1777586247 >> v2:172.21.15.93:3300/0 conn(0x7f18b0158c60 0x7f18b00705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195674186 0 0) 0x7f18b015fad0 con 0x7f18b0071ad0 2023-11-08T16:41:41.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f189c003520 con 0x7f18b0071ad0 2023-11-08T16:41:41.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3968361919 0 0) 0x7f189c003520 con 0x7f18b0071ad0 2023-11-08T16:41:41.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 >> v2:172.21.15.93:3300/0 conn(0x7f18b0158c60 msgr2=0x7f18b00705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:41.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b4f7c640 1 --2- 172.21.15.81:0/1777586247 >> v2:172.21.15.93:3300/0 conn(0x7f18b0158c60 0x7f18b00705d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 >> v1:172.21.15.93:6789/0 conn(0x7f18b015f380 legacy=0x7f18b00c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b006da00 con 0x7f18b0071ad0 2023-11-08T16:41:41.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f18a0001670 con 0x7f18b0071ad0 2023-11-08T16:41:41.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.720+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 >> v1:172.21.15.81:6789/0 conn(0x7f18b0071ad0 legacy=0x7f18b015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b4f7c640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 0x7f189c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b4f7c640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f189c0047c0 0x7f189c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b4f7c640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f189c0051d0 0x7f189c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b006df80 con 0x7f189c0039c0 2023-11-08T16:41:41.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c00c120 con 0x7f189c0051d0 2023-11-08T16:41:41.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c00c2e0 con 0x7f189c0047c0 2023-11-08T16:41:41.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b5f7e640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 0x7f189c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b5f7e640 1 -- 172.21.15.81:0/1777586247 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f189c0047c0 msgr2=0x7f189c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:41.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b677f640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f189c0051d0 0x7f189c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b577d640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f189c0047c0 0x7f189c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b5f7e640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f189c0047c0 0x7f189c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b5f7e640 1 -- 172.21.15.81:0/1777586247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f189c0051d0 msgr2=0x7f189c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:41.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b5f7e640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f189c0051d0 0x7f189c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b5f7e640 1 -- 172.21.15.81:0/1777586247 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c003520 con 0x7f189c0039c0 2023-11-08T16:41:41.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b677f640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f189c0051d0 0x7f189c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:41.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.721+0000 7f18b677f640 1 -- 172.21.15.81:0/1777586247 reap_dead start 2023-11-08T16:41:41.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.723+0000 7f18b5f7e640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 0x7f189c003de0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009a90 tx=0x7f189c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c148cbef1f6996d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:41.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.723+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac02c680 con 0x7f189c0039c0 2023-11-08T16:41:41.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.723+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ac02c7e0 con 0x7f189c0039c0 2023-11-08T16:41:41.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.723+0000 7f18b4f7c640 1 -- 172.21.15.81:0/1777586247 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac02cb70 con 0x7f189c0039c0 2023-11-08T16:41:41.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.723+0000 7f18b81e2640 1 -- 172.21.15.81:0/1777586247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 msgr2=0x7f189c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.723+0000 7f18b81e2640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 0x7f189c003de0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009a90 tx=0x7f189c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.724+0000 7f18b81e2640 1 -- 172.21.15.81:0/1777586247 shutdown_connections 2023-11-08T16:41:41.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.724+0000 7f18b81e2640 1 --2- 172.21.15.81:0/1777586247 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 0x7f189c003de0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.724+0000 7f18b81e2640 1 -- 172.21.15.81:0/1777586247 >> 172.21.15.81:0/1777586247 conn(0x7f18b00ceb70 msgr2=0x7f18b00cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:41.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.724+0000 7f18b81e2640 1 -- 172.21.15.81:0/1777586247 shutdown_connections 2023-11-08T16:41:41.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.724+0000 7f18b81e2640 1 -- 172.21.15.81:0/1777586247 wait complete. 2023-11-08T16:41:41.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.725+0000 7f18b81e2640 1 Processor -- start 2023-11-08T16:41:41.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.726+0000 7f18b81e2640 1 -- start start 2023-11-08T16:41:41.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.726+0000 7f18b81e2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 0x7f18b0159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.726+0000 7f18b81e2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18b0153e40 0x7f18b0153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.726+0000 7f18b81e2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18b01eefb0 0x7f18b01ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.726+0000 7f18b81e2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0159f90 con 0x7f189c0039c0 2023-11-08T16:41:41.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.726+0000 7f18b81e2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b0154640 con 0x7f18b0153e40 2023-11-08T16:41:41.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.726+0000 7f18b81e2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b015a1c0 con 0x7f18b01eefb0 2023-11-08T16:41:41.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.727+0000 7f18b5f7e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 0x7f18b0159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.727+0000 7f18b5f7e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 0x7f18b0159a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48002/0 (socket says 172.21.15.81:48002) 2023-11-08T16:41:41.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.727+0000 7f18b5f7e640 1 -- 172.21.15.81:0/821808229 learned_addr learned my addr 172.21.15.81:0/821808229 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:41.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.727+0000 7f18b5f7e640 1 -- 172.21.15.81:0/821808229 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18b01eefb0 msgr2=0x7f18b01ef440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.727+0000 7f18b677f640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18b01eefb0 0x7f18b01ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.727+0000 7f18b577d640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18b0153e40 0x7f18b0153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.727+0000 7f18b5f7e640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18b01eefb0 0x7f18b01ef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.727+0000 7f18b5f7e640 1 -- 172.21.15.81:0/821808229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18b0153e40 msgr2=0x7f18b0153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.727+0000 7f18b5f7e640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18b0153e40 0x7f18b0153580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.727+0000 7f18b5f7e640 1 -- 172.21.15.81:0/821808229 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b015a4a0 con 0x7f189c0039c0 2023-11-08T16:41:41.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.727+0000 7f18b5f7e640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 0x7f18b0159a50 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009a90 tx=0x7f18ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:41.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.728+0000 7f18a67fc640 1 -- 172.21.15.81:0/821808229 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac003810 con 0x7f189c0039c0 2023-11-08T16:41:41.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.728+0000 7f18a67fc640 1 -- 172.21.15.81:0/821808229 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ac003970 con 0x7f189c0039c0 2023-11-08T16:41:41.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.728+0000 7f18a67fc640 1 -- 172.21.15.81:0/821808229 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac003d00 con 0x7f189c0039c0 2023-11-08T16:41:41.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.728+0000 7f18b81e2640 1 -- 172.21.15.81:0/821808229 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b015a780 con 0x7f189c0039c0 2023-11-08T16:41:41.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.728+0000 7f18b81e2640 1 -- 172.21.15.81:0/821808229 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b015ad40 con 0x7f189c0039c0 2023-11-08T16:41:41.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.729+0000 7f18b81e2640 1 -- 172.21.15.81:0/821808229 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b015aed0 con 0x7f189c0039c0 2023-11-08T16:41:41.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.730+0000 7f18a67fc640 1 -- 172.21.15.81:0/821808229 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f18ac040450 con 0x7f189c0039c0 2023-11-08T16:41:41.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.731+0000 7f18a67fc640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f189406aec0 0x7f189406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:41.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.731+0000 7f18a67fc640 1 -- 172.21.15.81:0/821808229 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f18ac0ab590 con 0x7f189c0039c0 2023-11-08T16:41:41.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.731+0000 7f18b577d640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f189406aec0 0x7f189406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:41.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.732+0000 7f18b577d640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f189406aec0 0x7f189406d380 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f18a0000ff0 tx=0x7f18a0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:41.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.735+0000 7f18a67fc640 1 -- 172.21.15.81:0/821808229 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f18ac0765c0 con 0x7f189c0039c0 2023-11-08T16:41:41.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.924+0000 7f18b81e2640 1 -- 172.21.15.81:0/821808229 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f18b0074440 con 0x7f189c0039c0 2023-11-08T16:41:41.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.925+0000 7f18a67fc640 1 -- 172.21.15.81:0/821808229 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v128) v1 ==== 100+0+11 (secure 0 0 0) 0x7f18ac003e60 con 0x7f189c0039c0 2023-11-08T16:41:41.928 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:41:41.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.927+0000 7f18b81e2640 1 -- 172.21.15.81:0/821808229 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f189406aec0 msgr2=0x7f189406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.927+0000 7f18b81e2640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f189406aec0 0x7f189406d380 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f18a0000ff0 tx=0x7f18a0009040 comp rx=0 tx=0).stop 2023-11-08T16:41:41.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.927+0000 7f18b81e2640 1 -- 172.21.15.81:0/821808229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 msgr2=0x7f18b0159a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:41.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.927+0000 7f18b81e2640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 0x7f18b0159a50 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f18ac009a90 tx=0x7f18ac004730 comp rx=0 tx=0).stop 2023-11-08T16:41:41.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.927+0000 7f18b81e2640 1 -- 172.21.15.81:0/821808229 shutdown_connections 2023-11-08T16:41:41.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.927+0000 7f18b81e2640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f189406aec0 0x7f189406d380 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.927+0000 7f18b81e2640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f18b01eefb0 0x7f18b01ef440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.927+0000 7f18b81e2640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f18b0153e40 0x7f18b0153580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.927+0000 7f18b81e2640 1 --2- 172.21.15.81:0/821808229 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f189c0039c0 0x7f18b0159a50 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:41.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.927+0000 7f18b81e2640 1 -- 172.21.15.81:0/821808229 >> 172.21.15.81:0/821808229 conn(0x7f18b00ceb70 msgr2=0x7f18b0154ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:41.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.928+0000 7f18b81e2640 1 -- 172.21.15.81:0/821808229 shutdown_connections 2023-11-08T16:41:41.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:41.928+0000 7f18b81e2640 1 -- 172.21.15.81:0/821808229 wait complete. 2023-11-08T16:41:41.948 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:41:42.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.160+0000 7f4543790640 1 Processor -- start 2023-11-08T16:41:42.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.160+0000 7f4543790640 1 -- start start 2023-11-08T16:41:42.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.162+0000 7f4543790640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f453c159f90 0x7f453c15a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:42.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.162+0000 7f4543790640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f453c0cb5f0 0x7f453c0cba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:42.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.162+0000 7f4543790640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c15dd30 con 0x7f453c06c360 2023-11-08T16:41:42.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.162+0000 7f4543790640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f453c0cbfc0 con 0x7f453c159f90 2023-11-08T16:41:42.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.162+0000 7f4543790640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c15da00 con 0x7f453c0cb5f0 2023-11-08T16:41:42.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.162+0000 7f454152c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f453c159f90 0x7f453c15a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:42.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.162+0000 7f4540d2b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f453c06c360 0x7f453c15e1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:56308/0 (socket says 172.21.15.81:56308) 2023-11-08T16:41:42.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.162+0000 7f4540d2b640 1 -- 172.21.15.81:0/294944824 learned_addr learned my addr 172.21.15.81:0/294944824 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:42.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.162+0000 7f454152c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f453c159f90 0x7f453c15a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48014/0 (socket says 172.21.15.81:48014) 2023-11-08T16:41:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.163+0000 7f4541d2d640 1 --2- 172.21.15.81:0/294944824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f453c0cb5f0 0x7f453c0cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.163+0000 7f454152c640 1 -- 172.21.15.81:0/294944824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f453c0cb5f0 msgr2=0x7f453c0cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.163+0000 7f454152c640 1 --2- 172.21.15.81:0/294944824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f453c0cb5f0 0x7f453c0cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.163+0000 7f454152c640 1 -- 172.21.15.81:0/294944824 >> v1:172.21.15.81:6789/0 conn(0x7f453c06c360 legacy=0x7f453c15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:42.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.163+0000 7f454152c640 1 -- 172.21.15.81:0/294944824 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c15acd0 con 0x7f453c159f90 2023-11-08T16:41:42.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.163+0000 7f4533fff640 1 -- 172.21.15.81:0/294944824 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418641114 0 0) 0x7f453c15dd30 con 0x7f453c06c360 2023-11-08T16:41:42.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.164+0000 7f454152c640 1 --2- 172.21.15.81:0/294944824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f453c159f90 0x7f453c15a370 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f45380097a0 tx=0x7f453802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8b0ad506bfe557 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:42.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.164+0000 7f4533fff640 1 -- 172.21.15.81:0/294944824 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538031830 con 0x7f453c159f90 2023-11-08T16:41:42.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.164+0000 7f4533fff640 1 -- 172.21.15.81:0/294944824 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4538031990 con 0x7f453c159f90 2023-11-08T16:41:42.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.164+0000 7f4533fff640 1 -- 172.21.15.81:0/294944824 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538031d20 con 0x7f453c159f90 2023-11-08T16:41:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.165+0000 7f4543790640 1 -- 172.21.15.81:0/294944824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f453c159f90 msgr2=0x7f453c15a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.165+0000 7f4543790640 1 --2- 172.21.15.81:0/294944824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f453c159f90 0x7f453c15a370 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f45380097a0 tx=0x7f453802b8d0 comp rx=0 tx=0).stop 2023-11-08T16:41:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.165+0000 7f4543790640 1 -- 172.21.15.81:0/294944824 shutdown_connections 2023-11-08T16:41:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.165+0000 7f4543790640 1 --2- 172.21.15.81:0/294944824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f453c0cb5f0 0x7f453c0cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:42.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.165+0000 7f4543790640 1 --2- 172.21.15.81:0/294944824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f453c159f90 0x7f453c15a370 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:42.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.165+0000 7f4543790640 1 -- 172.21.15.81:0/294944824 >> 172.21.15.81:0/294944824 conn(0x7f453c0b0770 msgr2=0x7f453c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:42.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.165+0000 7f4543790640 1 -- 172.21.15.81:0/294944824 shutdown_connections 2023-11-08T16:41:42.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.165+0000 7f4543790640 1 -- 172.21.15.81:0/294944824 wait complete. 2023-11-08T16:41:42.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.167+0000 7f4543790640 1 Processor -- start 2023-11-08T16:41:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.167+0000 7f4543790640 1 -- start start 2023-11-08T16:41:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.168+0000 7f4543790640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f453c06c360 0x7f453c0c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.168+0000 7f4543790640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f453c0cb5f0 0x7f453c1f21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:42.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.168+0000 7f4543790640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f453c159f90 0x7f453c0c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.168+0000 7f4543790640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0c64c0 con 0x7f453c0cb5f0 2023-11-08T16:41:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.168+0000 7f4543790640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0c5f00 con 0x7f453c159f90 2023-11-08T16:41:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.168+0000 7f4543790640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0c61e0 con 0x7f453c06c360 2023-11-08T16:41:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.168+0000 7f4540d2b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f453c0cb5f0 0x7f453c1f21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:42.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.168+0000 7f4540d2b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f453c0cb5f0 0x7f453c1f21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48022/0 (socket says 172.21.15.81:48022) 2023-11-08T16:41:42.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.168+0000 7f4540d2b640 1 -- 172.21.15.81:0/2699530250 learned_addr learned my addr 172.21.15.81:0/2699530250 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:42.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f454152c640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f453c06c360 0x7f453c0c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:42.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4541d2d640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f453c159f90 0x7f453c0c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:42.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4540d2b640 1 -- 172.21.15.81:0/2699530250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f453c06c360 msgr2=0x7f453c0c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:42.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4540d2b640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f453c06c360 0x7f453c0c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:42.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4540d2b640 1 -- 172.21.15.81:0/2699530250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f453c159f90 msgr2=0x7f453c0c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:42.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4540d2b640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f453c159f90 0x7f453c0c2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:42.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4540d2b640 1 -- 172.21.15.81:0/2699530250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4538009400 con 0x7f453c0cb5f0 2023-11-08T16:41:42.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f454152c640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f453c06c360 0x7f453c0c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:42.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4541d2d640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f453c159f90 0x7f453c0c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:42.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4540d2b640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f453c0cb5f0 0x7f453c1f21c0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c4d0 tx=0x7f452c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:42.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4531ffb640 1 -- 172.21.15.81:0/2699530250 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c0074e0 con 0x7f453c0cb5f0 2023-11-08T16:41:42.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4531ffb640 1 -- 172.21.15.81:0/2699530250 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c007660 con 0x7f453c0cb5f0 2023-11-08T16:41:42.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4531ffb640 1 -- 172.21.15.81:0/2699530250 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c004a20 con 0x7f453c0cb5f0 2023-11-08T16:41:42.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.169+0000 7f4543790640 1 -- 172.21.15.81:0/2699530250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453c0c4f00 con 0x7f453c0cb5f0 2023-11-08T16:41:42.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.170+0000 7f4543790640 1 -- 172.21.15.81:0/2699530250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c0c2c50 con 0x7f453c0cb5f0 2023-11-08T16:41:42.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.171+0000 7f4543790640 1 -- 172.21.15.81:0/2699530250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453c0c5420 con 0x7f453c0cb5f0 2023-11-08T16:41:42.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.172+0000 7f4531ffb640 1 -- 172.21.15.81:0/2699530250 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f452c004b80 con 0x7f453c0cb5f0 2023-11-08T16:41:42.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.173+0000 7f4531ffb640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f452006ae70 0x7f452006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:42.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.174+0000 7f4531ffb640 1 -- 172.21.15.81:0/2699530250 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f452c088900 con 0x7f453c0cb5f0 2023-11-08T16:41:42.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.174+0000 7f454152c640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f452006ae70 0x7f452006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:42.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.174+0000 7f454152c640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f452006ae70 0x7f452006d330 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4538036000 tx=0x7f4538008030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:42.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.178+0000 7f4531ffb640 1 -- 172.21.15.81:0/2699530250 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f453c0c5420 con 0x7f453c0cb5f0 2023-11-08T16:41:42.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.385+0000 7f4543790640 1 -- 172.21.15.81:0/2699530250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f453c0c5420 con 0x7f453c0cb5f0 2023-11-08T16:41:42.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.386+0000 7f4531ffb640 1 -- 172.21.15.81:0/2699530250 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v128) v1 ==== 80+0+122 (secure 0 0 0) 0x7f453c0c5420 con 0x7f453c0cb5f0 2023-11-08T16:41:42.389 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.93:6835/4021510722 2023-11-09T16:41:29.470203+0000 2023-11-08T16:41:42.389 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.93:6833/4021510722 2023-11-09T16:41:29.470203+0000 2023-11-08T16:41:42.389 INFO:teuthology.orchestra.run.smithi081.stderr:listed 2 entries 2023-11-08T16:41:42.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.388+0000 7f4543790640 1 -- 172.21.15.81:0/2699530250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f452006ae70 msgr2=0x7f452006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:42.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.388+0000 7f4543790640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f452006ae70 0x7f452006d330 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f4538036000 tx=0x7f4538008030 comp rx=0 tx=0).stop 2023-11-08T16:41:42.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.389+0000 7f4543790640 1 -- 172.21.15.81:0/2699530250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f453c0cb5f0 msgr2=0x7f453c1f21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:42.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.389+0000 7f4543790640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f453c0cb5f0 0x7f453c1f21c0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f452c00c4d0 tx=0x7f452c00c990 comp rx=0 tx=0).stop 2023-11-08T16:41:42.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.389+0000 7f4543790640 1 -- 172.21.15.81:0/2699530250 shutdown_connections 2023-11-08T16:41:42.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.389+0000 7f4543790640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f452006ae70 0x7f452006d330 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:42.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.389+0000 7f4543790640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f453c159f90 0x7f453c0c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:42.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.389+0000 7f4543790640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f453c0cb5f0 0x7f453c1f21c0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:42.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.389+0000 7f4543790640 1 --2- 172.21.15.81:0/2699530250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f453c06c360 0x7f453c0c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:42.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.389+0000 7f4543790640 1 -- 172.21.15.81:0/2699530250 >> 172.21.15.81:0/2699530250 conn(0x7f453c0b0770 msgr2=0x7f453c15e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:42.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.389+0000 7f4543790640 1 -- 172.21.15.81:0/2699530250 shutdown_connections 2023-11-08T16:41:42.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:42.389+0000 7f4543790640 1 -- 172.21.15.81:0/2699530250 wait complete. 2023-11-08T16:41:42.410 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:41:42.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:42.411 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:42.496 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[126550]: fuse finished with error 0 and tester_r 0 2023-11-08T16:41:42.503 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:41:48.605 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:41:48.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:48.606 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:41:48.667 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:41:48.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.883+0000 7f8991e7c640 1 Processor -- start 2023-11-08T16:41:48.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.883+0000 7f8991e7c640 1 -- start start 2023-11-08T16:41:48.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.885+0000 7f8991e7c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f898c160820 0x7f898c0cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:48.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.885+0000 7f8991e7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f898c160f00 0x7f898c0cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:48.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.885+0000 7f8991e7c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f898c0cfde0 0x7f898c0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:48.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.885+0000 7f8991e7c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c0d1870 con 0x7f898c0cfde0 2023-11-08T16:41:48.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.885+0000 7f8991e7c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f898c0d0270 con 0x7f898c160820 2023-11-08T16:41:48.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.885+0000 7f8991e7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c0ce140 con 0x7f898c160f00 2023-11-08T16:41:48.888 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.885+0000 7f898bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f898c0cfde0 0x7f898c0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:48.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.885+0000 7f898bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f898c0cfde0 0x7f898c0b4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48032/0 (socket says 172.21.15.81:48032) 2023-11-08T16:41:48.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.885+0000 7f898bfff640 1 -- 172.21.15.81:0/417388358 learned_addr learned my addr 172.21.15.81:0/417388358 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:48.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.885+0000 7f898b7fe640 1 --2- 172.21.15.81:0/417388358 >> v2:172.21.15.93:3300/0 conn(0x7f898c160820 0x7f898c0cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:48.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.885+0000 7f8982ffd640 1 --2- 172.21.15.81:0/417388358 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f898c160f00 0x7f898c0cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:48.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.886+0000 7f898bfff640 1 -- 172.21.15.81:0/417388358 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f898c160f00 msgr2=0x7f898c0cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:48.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.886+0000 7f898bfff640 1 --2- 172.21.15.81:0/417388358 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f898c160f00 0x7f898c0cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:48.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.886+0000 7f898bfff640 1 -- 172.21.15.81:0/417388358 >> v2:172.21.15.93:3300/0 conn(0x7f898c160820 msgr2=0x7f898c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:48.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.886+0000 7f898bfff640 1 --2- 172.21.15.81:0/417388358 >> v2:172.21.15.93:3300/0 conn(0x7f898c160820 0x7f898c0cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:48.890 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.886+0000 7f898bfff640 1 -- 172.21.15.81:0/417388358 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f898c0ce420 con 0x7f898c0cfde0 2023-11-08T16:41:48.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.887+0000 7f898bfff640 1 --2- 172.21.15.81:0/417388358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c0cfde0 0x7f898c0b4b10 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f898c0686a0 tx=0x7f898402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e79d72a887dd3f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:48.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.887+0000 7f898affd640 1 -- 172.21.15.81:0/417388358 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984031870 con 0x7f898c0cfde0 2023-11-08T16:41:48.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.887+0000 7f898affd640 1 -- 172.21.15.81:0/417388358 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89840319d0 con 0x7f898c0cfde0 2023-11-08T16:41:48.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.887+0000 7f898affd640 1 -- 172.21.15.81:0/417388358 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984031d60 con 0x7f898c0cfde0 2023-11-08T16:41:48.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.888+0000 7f8991e7c640 1 -- 172.21.15.81:0/417388358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c0cfde0 msgr2=0x7f898c0b4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:48.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.888+0000 7f8991e7c640 1 --2- 172.21.15.81:0/417388358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c0cfde0 0x7f898c0b4b10 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f898c0686a0 tx=0x7f898402b910 comp rx=0 tx=0).stop 2023-11-08T16:41:48.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.888+0000 7f8991e7c640 1 -- 172.21.15.81:0/417388358 shutdown_connections 2023-11-08T16:41:48.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.888+0000 7f8991e7c640 1 --2- 172.21.15.81:0/417388358 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f898c160f00 0x7f898c0cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:48.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.888+0000 7f8991e7c640 1 --2- 172.21.15.81:0/417388358 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c0cfde0 0x7f898c0b4b10 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:48.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.888+0000 7f8991e7c640 1 --2- 172.21.15.81:0/417388358 >> v2:172.21.15.93:3300/0 conn(0x7f898c160820 0x7f898c0cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:48.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.888+0000 7f8991e7c640 1 -- 172.21.15.81:0/417388358 >> 172.21.15.81:0/417388358 conn(0x7f898c06aa00 msgr2=0x7f898c06ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:48.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.888+0000 7f8991e7c640 1 -- 172.21.15.81:0/417388358 shutdown_connections 2023-11-08T16:41:48.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.888+0000 7f8991e7c640 1 -- 172.21.15.81:0/417388358 wait complete. 2023-11-08T16:41:48.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.890+0000 7f8991e7c640 1 Processor -- start 2023-11-08T16:41:48.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.890+0000 7f8991e7c640 1 -- start start 2023-11-08T16:41:48.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.891+0000 7f8991e7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f898c0cfde0 0x7f898c153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:48.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.891+0000 7f8991e7c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f898c160820 0x7f898c156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:48.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.891+0000 7f8991e7c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c160f00 0x7f898c0d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:48.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.891+0000 7f8991e7c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c157b60 con 0x7f898c160f00 2023-11-08T16:41:48.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.891+0000 7f8991e7c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f898c155200 con 0x7f898c160820 2023-11-08T16:41:48.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.891+0000 7f8991e7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f898c155aa0 con 0x7f898c0cfde0 2023-11-08T16:41:48.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.891+0000 7f898bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c160f00 0x7f898c0d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:48.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.891+0000 7f898bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c160f00 0x7f898c0d2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48046/0 (socket says 172.21.15.81:48046) 2023-11-08T16:41:48.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.891+0000 7f898bfff640 1 -- 172.21.15.81:0/2514687574 learned_addr learned my addr 172.21.15.81:0/2514687574 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:48.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f8982ffd640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f898c160820 0x7f898c156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:48.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f898b7fe640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f898c0cfde0 0x7f898c153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:48.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f898bfff640 1 -- 172.21.15.81:0/2514687574 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f898c0cfde0 msgr2=0x7f898c153b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:48.900 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f898bfff640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f898c0cfde0 0x7f898c153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:48.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f898bfff640 1 -- 172.21.15.81:0/2514687574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f898c160820 msgr2=0x7f898c156e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:48.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f898bfff640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f898c160820 0x7f898c156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:48.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f898bfff640 1 -- 172.21.15.81:0/2514687574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89840094d0 con 0x7f898c160f00 2023-11-08T16:41:48.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f898bfff640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c160f00 0x7f898c0d2b00 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f89840090b0 tx=0x7f89840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:48.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f8988ff9640 1 -- 172.21.15.81:0/2514687574 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f898403ac60 con 0x7f898c160f00 2023-11-08T16:41:48.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f8988ff9640 1 -- 172.21.15.81:0/2514687574 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f898403adc0 con 0x7f898c160f00 2023-11-08T16:41:48.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f8988ff9640 1 -- 172.21.15.81:0/2514687574 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8984044630 con 0x7f898c160f00 2023-11-08T16:41:48.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.892+0000 7f8991e7c640 1 -- 172.21.15.81:0/2514687574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f898c1540c0 con 0x7f898c160f00 2023-11-08T16:41:48.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.893+0000 7f8991e7c640 1 -- 172.21.15.81:0/2514687574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f898c155450 con 0x7f898c160f00 2023-11-08T16:41:48.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.894+0000 7f8991e7c640 1 -- 172.21.15.81:0/2514687574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898c074440 con 0x7f898c160f00 2023-11-08T16:41:48.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.895+0000 7f8988ff9640 1 -- 172.21.15.81:0/2514687574 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8984044790 con 0x7f898c160f00 2023-11-08T16:41:48.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.896+0000 7f8988ff9640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896c06ae70 0x7f896c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:48.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.896+0000 7f8988ff9640 1 -- 172.21.15.81:0/2514687574 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f898402d040 con 0x7f898c160f00 2023-11-08T16:41:48.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.896+0000 7f898b7fe640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896c06ae70 0x7f896c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:48.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.897+0000 7f898b7fe640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896c06ae70 0x7f896c06d330 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f897c009e10 tx=0x7f897c00b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:48.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:48.901+0000 7f8988ff9640 1 -- 172.21.15.81:0/2514687574 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8984076c30 con 0x7f898c160f00 2023-11-08T16:41:49.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.126+0000 7f8991e7c640 1 -- 172.21.15.81:0/2514687574 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f898c155be0 con 0x7f898c160f00 2023-11-08T16:41:49.129 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:41:49.127+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:41:49.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.130+0000 7f8988ff9640 1 -- 172.21.15.81:0/2514687574 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) v1 ==== 161+0+0 (secure 0 0 0) 0x7f898407a250 con 0x7f898c160f00 2023-11-08T16:41:49.132 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:41:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.133+0000 7f8991e7c640 1 -- 172.21.15.81:0/2514687574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896c06ae70 msgr2=0x7f896c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.133+0000 7f8991e7c640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896c06ae70 0x7f896c06d330 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f897c009e10 tx=0x7f897c00b040 comp rx=0 tx=0).stop 2023-11-08T16:41:49.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.133+0000 7f8991e7c640 1 -- 172.21.15.81:0/2514687574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c160f00 msgr2=0x7f898c0d2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.133+0000 7f8991e7c640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c160f00 0x7f898c0d2b00 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f89840090b0 tx=0x7f89840046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.134+0000 7f8991e7c640 1 -- 172.21.15.81:0/2514687574 shutdown_connections 2023-11-08T16:41:49.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.134+0000 7f8991e7c640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f896c06ae70 0x7f896c06d330 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.134+0000 7f8991e7c640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f898c160f00 0x7f898c0d2b00 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.134+0000 7f8991e7c640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f898c160820 0x7f898c156e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.134+0000 7f8991e7c640 1 --2- 172.21.15.81:0/2514687574 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f898c0cfde0 0x7f898c153b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.134+0000 7f8991e7c640 1 -- 172.21.15.81:0/2514687574 >> 172.21.15.81:0/2514687574 conn(0x7f898c06aa00 msgr2=0x7f898c15e950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.135+0000 7f8991e7c640 1 -- 172.21.15.81:0/2514687574 shutdown_connections 2023-11-08T16:41:49.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.135+0000 7f8991e7c640 1 -- 172.21.15.81:0/2514687574 wait complete. 2023-11-08T16:41:49.155 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents 2023-11-08T16:41:49.211 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:41:49.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.240+0000 7ff90523d5c0 1 -- start start 2023-11-08T16:41:49.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.240+0000 7ff90523d5c0 1 Processor -- start 2023-11-08T16:41:49.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.241+0000 7ff90523d5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66ee06c00 0x55b66ee70580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.241+0000 7ff90523d5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66ee07800 0x55b66ee70b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.241+0000 7ff90523d5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66ee07c00 0x55b66ee71080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.241+0000 7ff90523d5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66ee4f180 con 0x55b66ee07c00 2023-11-08T16:41:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.241+0000 7ff90523d5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66ee4e140 con 0x55b66ee07800 2023-11-08T16:41:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.241+0000 7ff90523d5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b66ee4e000 con 0x55b66ee06c00 2023-11-08T16:41:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.241+0000 7ff903a06640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66ee07c00 0x55b66ee71080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.241+0000 7ff903a06640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66ee07c00 0x55b66ee71080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48080/0 (socket says 172.21.15.81:48080) 2023-11-08T16:41:49.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.241+0000 7ff903a06640 1 -- 172.21.15.81:0/855502515 learned_addr learned my addr 172.21.15.81:0/855502515 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:49.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff902a04640 1 --2- 172.21.15.81:0/855502515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66ee07800 0x55b66ee70b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff903a06640 1 -- 172.21.15.81:0/855502515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66ee06c00 msgr2=0x55b66ee70580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:49.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff903205640 1 --2- 172.21.15.81:0/855502515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66ee06c00 0x55b66ee70580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff903a06640 1 --2- 172.21.15.81:0/855502515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66ee06c00 0x55b66ee70580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff903a06640 1 -- 172.21.15.81:0/855502515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66ee07800 msgr2=0x55b66ee70b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:49.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff903a06640 1 --2- 172.21.15.81:0/855502515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66ee07800 0x55b66ee70b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff903a06640 1 -- 172.21.15.81:0/855502515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b66ee2c9c0 con 0x55b66ee07c00 2023-11-08T16:41:49.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff902a04640 1 --2- 172.21.15.81:0/855502515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66ee07800 0x55b66ee70b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:49.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff903a06640 1 --2- 172.21.15.81:0/855502515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66ee07c00 0x55b66ee71080 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x55b66fc18b40 tx=0x55b66fbda3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9a17e28ac63919 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff900a00640 1 -- 172.21.15.81:0/855502515 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b66ee4cf20 con 0x55b66ee07c00 2023-11-08T16:41:49.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff900a00640 1 -- 172.21.15.81:0/855502515 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b66ee60180 con 0x55b66ee07c00 2023-11-08T16:41:49.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff900a00640 1 -- 172.21.15.81:0/855502515 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b66ee4d1e0 con 0x55b66ee07c00 2023-11-08T16:41:49.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff90523d5c0 1 -- 172.21.15.81:0/855502515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b66ee2cd00 con 0x55b66ee07c00 2023-11-08T16:41:49.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff90523d5c0 1 -- 172.21.15.81:0/855502515 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b66ee2c9c0 con 0x55b66ee07c00 2023-11-08T16:41:49.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.242+0000 7ff90523d5c0 4 waiting for MDS map... 2023-11-08T16:41:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.243+0000 7ff900a00640 1 -- 172.21.15.81:0/855502515 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x55b66eec6c40 con 0x55b66ee07c00 2023-11-08T16:41:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.243+0000 7ff900a00640 1 -- 172.21.15.81:0/855502515 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 114) v1 ==== 2723+0+0 (secure 0 0 0) 0x55b66ee56f00 con 0x55b66ee07c00 2023-11-08T16:41:49.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.243+0000 7ff90523d5c0 4 Got MDS map 114 2023-11-08T16:41:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.243+0000 7ff90523d5c0 4 datascan.main: Using metadata pool output 2023-11-08T16:41:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.243+0000 7ff90523d5c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:41:49.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.244+0000 7ff90523d5c0 1 Processor -- start 2023-11-08T16:41:49.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.244+0000 7ff90523d5c0 1 -- start start 2023-11-08T16:41:49.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff90523d5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66fbd7c00 0x55b66ee71600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff90523d5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2e000 0x55b66ee71b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff90523d5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66fd2e400 0x55b66ee72100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff90523d5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66ee4ea00 con 0x55b66fd2e000 2023-11-08T16:41:49.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff90523d5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66eed5180 con 0x55b66fbd7c00 2023-11-08T16:41:49.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff90523d5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b66eed4dc0 con 0x55b66fd2e400 2023-11-08T16:41:49.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff903205640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2e000 0x55b66ee71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff903205640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2e000 0x55b66ee71b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48082/0 (socket says 172.21.15.81:48082) 2023-11-08T16:41:49.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff903205640 1 -- 172.21.15.81:0/4191038352 learned_addr learned my addr 172.21.15.81:0/4191038352 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:49.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff902a04640 1 --2- 172.21.15.81:0/4191038352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66fd2e400 0x55b66ee72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff902a04640 1 --2- 172.21.15.81:0/4191038352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66fbd7c00 0x55b66ee71600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff903205640 1 -- 172.21.15.81:0/4191038352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66fd2e400 msgr2=0x55b66ee72100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:49.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff903205640 1 --2- 172.21.15.81:0/4191038352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66fd2e400 0x55b66ee72100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff903205640 1 -- 172.21.15.81:0/4191038352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66fbd7c00 msgr2=0x55b66ee71600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:49.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff903205640 1 --2- 172.21.15.81:0/4191038352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66fbd7c00 0x55b66ee71600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.245+0000 7ff903205640 1 -- 172.21.15.81:0/4191038352 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b66ee2cb60 con 0x55b66fd2e000 2023-11-08T16:41:49.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff903205640 1 --2- 172.21.15.81:0/4191038352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2e000 0x55b66ee71b80 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x55b66fcd4ae0 tx=0x55b66fcdccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c52df8118993b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/4191038352 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b66ee4cf20 con 0x55b66fd2e000 2023-11-08T16:41:49.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/4191038352 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b66fbe8300 con 0x55b66fd2e000 2023-11-08T16:41:49.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/4191038352 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b66ee4d1e0 con 0x55b66fd2e000 2023-11-08T16:41:49.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff90523d5c0 1 -- 172.21.15.81:0/4191038352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2e000 msgr2=0x55b66ee71b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/4191038352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2e000 0x55b66ee71b80 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x55b66fcd4ae0 tx=0x55b66fcdccc0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff90523d5c0 1 -- 172.21.15.81:0/4191038352 shutdown_connections 2023-11-08T16:41:49.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/4191038352 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66fd2e400 0x55b66ee72100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/4191038352 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2e000 0x55b66ee71b80 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/4191038352 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66fbd7c00 0x55b66ee71600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff90523d5c0 1 -- 172.21.15.81:0/4191038352 >> 172.21.15.81:0/4191038352 conn(0x55b66fbd7800 msgr2=0x55b66fb81800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:49.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff90523d5c0 1 -- 172.21.15.81:0/4191038352 shutdown_connections 2023-11-08T16:41:49.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.246+0000 7ff90523d5c0 1 -- 172.21.15.81:0/4191038352 wait complete. 2023-11-08T16:41:49.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.247+0000 7ff90523d5c0 1 Processor -- start 2023-11-08T16:41:49.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.247+0000 7ff90523d5c0 1 -- start start 2023-11-08T16:41:49.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff90523d5c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66fbd7800 0x55b66ee73180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff90523d5c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66fd2f000 0x55b66ee72c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff90523d5c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2ec00 0x55b66ee72680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff90523d5c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66ee4e140 con 0x55b66fd2ec00 2023-11-08T16:41:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff90523d5c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66ee4f180 con 0x55b66fd2f000 2023-11-08T16:41:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff90523d5c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b66ee4eb40 con 0x55b66fbd7800 2023-11-08T16:41:49.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff902a04640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2ec00 0x55b66ee72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff902a04640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2ec00 0x55b66ee72680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48096/0 (socket says 172.21.15.81:48096) 2023-11-08T16:41:49.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 learned_addr learned my addr 172.21.15.81:0/1186316120 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:49.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66fbd7800 msgr2=0x55b66ee73180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff902a04640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66fbd7800 0x55b66ee73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66fd2f000 msgr2=0x55b66ee72c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.248+0000 7ff903205640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66fd2f000 0x55b66ee72c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.249+0000 7ff902a04640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66fd2f000 0x55b66ee72c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.249+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b66ee2c820 con 0x55b66fd2ec00 2023-11-08T16:41:49.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.249+0000 7ff902a04640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2ec00 0x55b66ee72680 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x55b66fcd57d0 tx=0x55b66fcdcde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.249+0000 7ff8fd1f9640 1 -- 172.21.15.81:0/1186316120 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b66ee4cf20 con 0x55b66fd2ec00 2023-11-08T16:41:49.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.249+0000 7ff8fd1f9640 1 -- 172.21.15.81:0/1186316120 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b66fbe9500 con 0x55b66fd2ec00 2023-11-08T16:41:49.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.249+0000 7ff8fd1f9640 1 -- 172.21.15.81:0/1186316120 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b66ee4d1e0 con 0x55b66fd2ec00 2023-11-08T16:41:49.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.249+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b66ee2c4e0 con 0x55b66fd2ec00 2023-11-08T16:41:49.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.249+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b66ee2c1a0 con 0x55b66fd2ec00 2023-11-08T16:41:49.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.249+0000 7ff90523d5c0 4 datascan.init: resolving metadata pool 16 2023-11-08T16:41:49.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.251+0000 7ff8fd1f9640 1 -- 172.21.15.81:0/1186316120 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x55b66ee63340 con 0x55b66fd2ec00 2023-11-08T16:41:49.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.251+0000 7ff8fd1f9640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b66fd3ac00 0x55b66ee73700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.251+0000 7ff8fd1f9640 1 -- 172.21.15.81:0/1186316120 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x55b66eec6a80 con 0x55b66fd2ec00 2023-11-08T16:41:49.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.251+0000 7ff90523d5c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:41:49.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.251+0000 7ff90523d5c0 4 datascan.main: data pool 'cephfs_data' has ID 17 2023-11-08T16:41:49.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.251+0000 7ff90523d5c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:41:49.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.251+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b66fd2e800 0x55b66ee72100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.251+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5f880 con 0x55b66fd2e800 2023-11-08T16:41:49.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.252+0000 7ff903205640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b66fd3ac00 0x55b66ee73700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.252+0000 7ff903205640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b66fd3ac00 0x55b66ee73700 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55b66fcd4ae0 tx=0x55b66fcdcc00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.252+0000 7ff902a04640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b66fd2e800 0x55b66ee72100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.253+0000 7ff902a04640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b66fd2e800 0x55b66ee72100 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.254+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(1 [pgnls start_epoch 131 out=81b] v127'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.254+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:2 17.0 17:012cdbc7:::10000000206.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66fd2e800 2023-11-08T16:41:49.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.255+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(2 10000000206.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.255+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628294.0 2023-11-08T16:41:49.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.255+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:3 17.0 17:012cdbc7:::10000000206.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5f880 con 0x55b66fd2e800 2023-11-08T16:41:49.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.258+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(3 10000000206.00000000 [call] v131'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.258+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b66fd2e400 0x55b66ee71b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.258+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66fd2e400 2023-11-08T16:41:49.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.259+0000 7ff903a06640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b66fd2e400 0x55b66ee71b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.259+0000 7ff903a06640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b66fd2e400 0x55b66ee71b80 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.260+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(4 [pgnls start_epoch 131 out=81b] v127'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.260+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:5 17.20 17:05552577:::1000000020b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66fd2e400 2023-11-08T16:41:49.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.260+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(5 1000000020b.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.260+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628299.0 2023-11-08T16:41:49.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.260+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:6 17.20 17:05552577:::1000000020b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174380 con 0x55b66fd2e400 2023-11-08T16:41:49.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.262+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(6 1000000020b.00000000 [call] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.262+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b66fd2e000 0x55b66ee71600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.262+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:7 17.10 17:08000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd2e000 2023-11-08T16:41:49.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.262+0000 7ff903205640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b66fd2e000 0x55b66ee71600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.263+0000 7ff903205640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b66fd2e000 0x55b66ee71600 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.263+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(7 [pgnls start_epoch 131 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.263+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:8 17.10 17:083d5826:::1000000022a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd2e000 2023-11-08T16:41:49.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.263+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(8 1000000022a.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.263+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628330.0 2023-11-08T16:41:49.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.263+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:9 17.10 17:083d5826:::1000000022a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd2e000 2023-11-08T16:41:49.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.266+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(9 1000000022a.00000000 [call] v131'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.266+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:10 17.10 17:08500202::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd2e000 2023-11-08T16:41:49.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.266+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(10 [pgnls start_epoch 131 out=101b] v131'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.266+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:11 17.10 17:08500202:::1000000023e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd2e000 2023-11-08T16:41:49.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.266+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(11 1000000023e.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.266+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628350.0 2023-11-08T16:41:49.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.266+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:12 17.10 17:08500202:::1000000023e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66fd2e000 2023-11-08T16:41:49.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.269+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(12 1000000023e.00000000 [call] v131'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.269+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:13 17.10 17:08b99fea::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174380 con 0x55b66fd2e000 2023-11-08T16:41:49.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.269+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(13 [pgnls start_epoch 131 out=101b] v131'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.269+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:14 17.10 17:08b99fea:::100000001fd.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66fd2e000 2023-11-08T16:41:49.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.269+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(14 100000001fd.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.269+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628285.0 2023-11-08T16:41:49.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.269+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:15 17.10 17:08b99fea:::100000001fd.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66fd2e000 2023-11-08T16:41:49.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.272+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(15 100000001fd.00000000 [call] v131'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.272+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:16 17.10 17:08e6f8ca::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66fd2e000 2023-11-08T16:41:49.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.273+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(16 [pgnls start_epoch 131 out=81b] v131'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.273+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:17 17.10 17:08e6f8ca:::10000000257.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66fd2e000 2023-11-08T16:41:49.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.273+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(17 10000000257.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.273+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628375.0 2023-11-08T16:41:49.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.273+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:18 17.10 17:08e6f8ca:::10000000257.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174380 con 0x55b66fd2e000 2023-11-08T16:41:49.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.275+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(18 10000000257.00000000 [call] v131'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.275+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b66fd3b000 0x55b66f184000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.275+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:19 17.30 17:0c000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd3b000 2023-11-08T16:41:49.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.276+0000 7ff902a04640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b66fd3b000 0x55b66f184000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.276+0000 7ff902a04640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b66fd3b000 0x55b66f184000 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.279+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(19 [pgnls start_epoch 131 out=101b] v127'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.279+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:20 17.30 17:0c00cb85:::1000000021d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd3b000 2023-11-08T16:41:49.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.279+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(20 1000000021d.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.279+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628317.0 2023-11-08T16:41:49.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.279+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:21 17.30 17:0c00cb85:::1000000021d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd3b000 2023-11-08T16:41:49.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.282+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(21 1000000021d.00000000 [call] v131'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.282+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:22 17.30 17:0c28cce1::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66fd3b000 2023-11-08T16:41:49.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.282+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(22 [pgnls start_epoch 131 out=101b] v131'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.282+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:23 17.30 17:0c28cce1:::10000000244.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd3b000 2023-11-08T16:41:49.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.283+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(23 10000000244.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.283+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628356.0 2023-11-08T16:41:49.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.283+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:24 17.30 17:0c28cce1:::10000000244.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175880 con 0x55b66fd3b000 2023-11-08T16:41:49.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.285+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(24 10000000244.00000000 [call] v131'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.285+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:25 17.30 17:0cc9e906::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd3b000 2023-11-08T16:41:49.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.285+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(25 [pgnls start_epoch 131 out=101b] v131'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.285+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:26 17.30 17:0cc9e906:::10000000227.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66fd3b000 2023-11-08T16:41:49.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.286+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(26 10000000227.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.286+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628327.0 2023-11-08T16:41:49.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.286+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:27 17.30 17:0cc9e906:::10000000227.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd3b000 2023-11-08T16:41:49.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.288+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(27 10000000227.00000000 [call] v131'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.288+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:28 17.30 17:0e3d5a27::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175880 con 0x55b66fd3b000 2023-11-08T16:41:49.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.289+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(28 [pgnls start_epoch 131 out=81b] v131'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.289+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:29 17.30 17:0e3d5a27:::10000000250.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd3b000 2023-11-08T16:41:49.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.289+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 11 ==== osd_op_reply(29 10000000250.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.289+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628368.0 2023-11-08T16:41:49.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.289+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:30 17.30 17:0e3d5a27:::10000000250.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd3b000 2023-11-08T16:41:49.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.291+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 12 ==== osd_op_reply(30 10000000250.00000000 [call] v131'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.291+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:31 17.8 17:10000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175c00 con 0x55b66fd2e800 2023-11-08T16:41:49.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.292+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(31 [pgnls start_epoch 131 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.292+0000 7ff8fe1fb640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:32 17.28 17:14000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd2e000 2023-11-08T16:41:49.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.293+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(32 [pgnls start_epoch 131 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.293+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:33 17.28 17:14ce3b86:::10000000233.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5f880 con 0x55b66fd2e000 2023-11-08T16:41:49.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.293+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(33 10000000233.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.293+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628339.0 2023-11-08T16:41:49.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.293+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:34 17.28 17:14ce3b86:::10000000233.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66fd2e000 2023-11-08T16:41:49.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.296+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(34 10000000233.00000000 [call] v131'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.296+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:35 17.28 17:15246602::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174380 con 0x55b66fd2e000 2023-11-08T16:41:49.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.297+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(35 [pgnls start_epoch 131 out=101b] v131'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.297+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:36 17.28 17:15246602:::10000000202.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd2e000 2023-11-08T16:41:49.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.297+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(36 10000000202.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.297+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628290.0 2023-11-08T16:41:49.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.297+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:37 17.28 17:15246602:::10000000202.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66fd2e000 2023-11-08T16:41:49.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.300+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 18 ==== osd_op_reply(37 10000000202.00000000 [call] v131'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.300+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:38 17.28 17:1537fa38::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd2e000 2023-11-08T16:41:49.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.300+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 19 ==== osd_op_reply(38 [pgnls start_epoch 131 out=101b] v131'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.300+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:39 17.28 17:1537fa38:::10000000236.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175880 con 0x55b66fd2e000 2023-11-08T16:41:49.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.301+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 20 ==== osd_op_reply(39 10000000236.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.301+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628342.0 2023-11-08T16:41:49.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.301+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:40 17.28 17:1537fa38:::10000000236.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd2e000 2023-11-08T16:41:49.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.303+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 21 ==== osd_op_reply(40 10000000236.00000000 [call] v131'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.303+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:41 17.28 17:177a762d::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66fd2e000 2023-11-08T16:41:49.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.303+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 22 ==== osd_op_reply(41 [pgnls start_epoch 131 out=81b] v131'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.304+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:42 17.28 17:177a762d:::10000000201.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5f880 con 0x55b66fd2e000 2023-11-08T16:41:49.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.304+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 23 ==== osd_op_reply(42 10000000201.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.304+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628289.0 2023-11-08T16:41:49.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.304+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:43 17.28 17:177a762d:::10000000201.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66fd2e000 2023-11-08T16:41:49.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.307+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 24 ==== osd_op_reply(43 10000000201.00000000 [call] v131'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.307+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b66fd3b400 0x55b66f184580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.307+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:44 17.18 17:18000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174380 con 0x55b66fd3b400 2023-11-08T16:41:49.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.307+0000 7ff903a06640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b66fd3b400 0x55b66f184580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.308+0000 7ff903a06640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b66fd3b400 0x55b66f184580 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.310+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(44 [pgnls start_epoch 131 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.310+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:45 17.38 17:1c000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4000 con 0x55b66fd2e800 2023-11-08T16:41:49.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.311+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(45 [pgnls start_epoch 131 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.311+0000 7ff8fe1fb640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:46 17.4 17:20000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66fd3b000 2023-11-08T16:41:49.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.312+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 13 ==== osd_op_reply(46 [pgnls start_epoch 131 out=81b] v127'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.312+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:47 17.4 17:2231e764:::10000000220.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd3b000 2023-11-08T16:41:49.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.312+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 14 ==== osd_op_reply(47 10000000220.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.312+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628320.0 2023-11-08T16:41:49.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.313+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:48 17.4 17:2231e764:::10000000220.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66fd3b000 2023-11-08T16:41:49.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.314+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 15 ==== osd_op_reply(48 10000000220.00000000 [call] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.314+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:49 17.24 17:24000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd2e000 2023-11-08T16:41:49.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.315+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 25 ==== osd_op_reply(49 [pgnls start_epoch 131 out=101b] v127'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.315+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:50 17.24 17:242ba651:::10000000234.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175880 con 0x55b66fd2e000 2023-11-08T16:41:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.315+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 26 ==== osd_op_reply(50 10000000234.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.315+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628340.0 2023-11-08T16:41:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.315+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:51 17.24 17:242ba651:::10000000234.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4000 con 0x55b66fd2e000 2023-11-08T16:41:49.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.318+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 27 ==== osd_op_reply(51 10000000234.00000000 [call] v131'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.318+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:52 17.24 17:25b5fce8::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66fd2e000 2023-11-08T16:41:49.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.318+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 28 ==== osd_op_reply(52 [pgnls start_epoch 131 out=81b] v131'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.319+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:53 17.24 17:25b5fce8:::10000000231.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4a80 con 0x55b66fd2e000 2023-11-08T16:41:49.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.319+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 29 ==== osd_op_reply(53 10000000231.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.319+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628337.0 2023-11-08T16:41:49.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.319+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:54 17.24 17:25b5fce8:::10000000231.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4e00 con 0x55b66fd2e000 2023-11-08T16:41:49.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.322+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 30 ==== osd_op_reply(54 10000000231.00000000 [call] v131'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.322+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:55 17.14 17:28000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5180 con 0x55b66fd2e000 2023-11-08T16:41:49.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.322+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 31 ==== osd_op_reply(55 [pgnls start_epoch 131 out=101b] v127'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.322+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:56 17.14 17:29971f97:::1000000020e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5500 con 0x55b66fd2e000 2023-11-08T16:41:49.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.323+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 32 ==== osd_op_reply(56 1000000020e.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.323+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628302.0 2023-11-08T16:41:49.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.323+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:57 17.14 17:29971f97:::1000000020e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5880 con 0x55b66fd2e000 2023-11-08T16:41:49.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.326+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 33 ==== osd_op_reply(57 1000000020e.00000000 [call] v131'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.326+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:58 17.14 17:2ab28be7::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5c00 con 0x55b66fd2e000 2023-11-08T16:41:49.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.327+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 34 ==== osd_op_reply(58 [pgnls start_epoch 131 out=101b] v131'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.327+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:59 17.14 17:2ab28be7:::10000000211.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd2e000 2023-11-08T16:41:49.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.328+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 35 ==== osd_op_reply(59 10000000211.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.328+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628305.0 2023-11-08T16:41:49.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.328+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:60 17.14 17:2ab28be7:::10000000211.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175c00 con 0x55b66fd2e000 2023-11-08T16:41:49.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.330+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 36 ==== osd_op_reply(60 10000000211.00000000 [call] v131'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.331+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:61 17.14 17:2b89f3ba::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd2e000 2023-11-08T16:41:49.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.331+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 37 ==== osd_op_reply(61 [pgnls start_epoch 131 out=81b] v131'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.331+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:62 17.14 17:2b89f3ba:::10000000221.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66fd2e000 2023-11-08T16:41:49.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.332+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 38 ==== osd_op_reply(62 10000000221.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.332+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628321.0 2023-11-08T16:41:49.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.332+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:63 17.14 17:2b89f3ba:::10000000221.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66fd2e000 2023-11-08T16:41:49.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.334+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 39 ==== osd_op_reply(63 10000000221.00000000 [call] v131'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.334+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:64 17.34 17:2c000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4a80 con 0x55b66fd3b000 2023-11-08T16:41:49.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.335+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 16 ==== osd_op_reply(64 [pgnls start_epoch 131 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.335+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:65 17.c 17:30000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5c00 con 0x55b66fd2e800 2023-11-08T16:41:49.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.336+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(65 [pgnls start_epoch 131 out=81b] v127'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.336+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:66 17.c 17:30604137:::100000001fc.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4e00 con 0x55b66fd2e800 2023-11-08T16:41:49.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.337+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(66 100000001fc.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.337+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628284.0 2023-11-08T16:41:49.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.337+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:67 17.c 17:30604137:::100000001fc.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5180 con 0x55b66fd2e800 2023-11-08T16:41:49.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.341+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(67 100000001fc.00000000 [call] v131'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.341+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b66fd3b800 0x55b66f184b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.341+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:68 17.2c 17:34000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5500 con 0x55b66fd3b800 2023-11-08T16:41:49.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.342+0000 7ff903205640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b66fd3b800 0x55b66f184b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.343+0000 7ff903205640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b66fd3b800 0x55b66f184b00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.344+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(68 [pgnls start_epoch 131 out=101b] v127'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.344+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:69 17.2c 17:35964134:::10000000223.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5880 con 0x55b66fd3b800 2023-11-08T16:41:49.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.345+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(69 10000000223.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.345+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628323.0 2023-11-08T16:41:49.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.345+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:70 17.2c 17:35964134:::10000000223.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd3b800 2023-11-08T16:41:49.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.349+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(70 10000000223.00000000 [call] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.349+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:71 17.2c 17:37389ade::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd3b800 2023-11-08T16:41:49.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.350+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(71 [pgnls start_epoch 131 out=81b] v131'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.350+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:72 17.2c 17:37389ade:::10000000209.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66fd3b800 2023-11-08T16:41:49.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.351+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(72 10000000209.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.351+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628297.0 2023-11-08T16:41:49.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.351+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:73 17.2c 17:37389ade:::10000000209.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd3b800 2023-11-08T16:41:49.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.355+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(73 10000000209.00000000 [call] v131'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.355+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:74 17.1c 17:38000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175880 con 0x55b66fd2e400 2023-11-08T16:41:49.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.355+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(74 [pgnls start_epoch 131 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.355+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:75 17.3c 17:3c000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4000 con 0x55b66fd3b000 2023-11-08T16:41:49.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.356+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 17 ==== osd_op_reply(75 [pgnls start_epoch 131 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.356+0000 7ff8fe1fb640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:76 17.2 17:40000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4700 con 0x55b66fd3b800 2023-11-08T16:41:49.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.357+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(76 [pgnls start_epoch 131 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.357+0000 7ff8fe9fc640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b66f1b0000 0x55b66f186680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.357+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:77 17.22 17:44000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66f1b0000 2023-11-08T16:41:49.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.358+0000 7ff902a04640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b66f1b0000 0x55b66f186680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.359+0000 7ff902a04640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b66f1b0000 0x55b66f186680 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.360+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(77 [pgnls start_epoch 131 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.360+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:78 17.22 17:450d4ca5:::1000000024c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66f1b0000 2023-11-08T16:41:49.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.361+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(78 1000000024c.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.361+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628364.0 2023-11-08T16:41:49.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.361+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:79 17.22 17:450d4ca5:::1000000024c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5f880 con 0x55b66f1b0000 2023-11-08T16:41:49.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.365+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(79 1000000024c.00000000 [call] v131'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.365+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:80 17.22 17:45b8cac8::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175c00 con 0x55b66f1b0000 2023-11-08T16:41:49.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.366+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(80 [pgnls start_epoch 131 out=101b] v131'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.367+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:81 17.22 17:45b8cac8:::1000000023d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66f1b0000 2023-11-08T16:41:49.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.368+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(81 1000000023d.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.368+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628349.0 2023-11-08T16:41:49.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.368+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:82 17.22 17:45b8cac8:::1000000023d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66f1b0000 2023-11-08T16:41:49.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.371+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(82 1000000023d.00000000 [call] v131'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.372+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:83 17.22 17:47efb22f::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66f1b0000 2023-11-08T16:41:49.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.373+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(83 [pgnls start_epoch 131 out=81b] v131'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.373+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:84 17.22 17:47efb22f:::10000000235.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66f1b0000 2023-11-08T16:41:49.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.374+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(84 10000000235.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.374+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628341.0 2023-11-08T16:41:49.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.374+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:85 17.22 17:47efb22f:::10000000235.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66f1b0000 2023-11-08T16:41:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.377+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 9 ==== osd_op_reply(85 10000000235.00000000 [call] v131'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.377+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:86 17.12 17:48000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66fd2e000 2023-11-08T16:41:49.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.378+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 40 ==== osd_op_reply(86 [pgnls start_epoch 131 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.378+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:87 17.32 17:4c000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5500 con 0x55b66fd2e800 2023-11-08T16:41:49.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.379+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 9 ==== osd_op_reply(87 [pgnls start_epoch 131 out=81b] v127'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.379+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:88 17.32 17:4e8c6251:::10000000252.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4a80 con 0x55b66fd2e800 2023-11-08T16:41:49.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.380+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 10 ==== osd_op_reply(88 10000000252.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.381+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628370.0 2023-11-08T16:41:49.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.381+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:89 17.32 17:4e8c6251:::10000000252.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5c00 con 0x55b66fd2e800 2023-11-08T16:41:49.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.384+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 11 ==== osd_op_reply(89 10000000252.00000000 [call] v131'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.384+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:90 17.a 17:50000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4e00 con 0x55b66fd2e800 2023-11-08T16:41:49.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.386+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 12 ==== osd_op_reply(90 [pgnls start_epoch 131 out=101b] v127'9 uv9 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.386+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:91 17.a 17:52dcfedc:::1000000023f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5180 con 0x55b66fd2e800 2023-11-08T16:41:49.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.387+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 13 ==== osd_op_reply(91 1000000023f.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.387+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628351.0 2023-11-08T16:41:49.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.387+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:92 17.a 17:52dcfedc:::1000000023f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4000 con 0x55b66fd2e800 2023-11-08T16:41:49.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.391+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 14 ==== osd_op_reply(92 1000000023f.00000000 [call] v131'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.391+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:93 17.a 17:53d16f76::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66fd2e800 2023-11-08T16:41:49.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.392+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 15 ==== osd_op_reply(93 [pgnls start_epoch 131 out=81b] v131'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.392+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:94 17.a 17:53d16f76:::1000000023b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5500 con 0x55b66fd2e800 2023-11-08T16:41:49.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.393+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 16 ==== osd_op_reply(94 1000000023b.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.393+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628347.0 2023-11-08T16:41:49.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.393+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:95 17.a 17:53d16f76:::1000000023b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175c00 con 0x55b66fd2e800 2023-11-08T16:41:49.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.398+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 17 ==== osd_op_reply(95 1000000023b.00000000 [call] v131'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.398+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:96 17.2a 17:54000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd2e000 2023-11-08T16:41:49.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.399+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 41 ==== osd_op_reply(96 [pgnls start_epoch 131 out=101b] v127'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.399+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:97 17.2a 17:55a6cd83:::1000000020a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66fd2e000 2023-11-08T16:41:49.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.399+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 42 ==== osd_op_reply(97 1000000020a.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.400+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628298.0 2023-11-08T16:41:49.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.400+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:98 17.2a 17:55a6cd83:::1000000020a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd2e000 2023-11-08T16:41:49.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.403+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 43 ==== osd_op_reply(98 1000000020a.00000000 [call] v131'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.403+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:99 17.2a 17:56889a45::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd2e000 2023-11-08T16:41:49.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.404+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 44 ==== osd_op_reply(99 [pgnls start_epoch 131 out=101b] v131'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.404+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:100 17.2a 17:56889a45:::10000000213.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66fd2e000 2023-11-08T16:41:49.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.405+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 45 ==== osd_op_reply(100 10000000213.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.405+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628307.0 2023-11-08T16:41:49.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.405+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:101 17.2a 17:56889a45:::10000000213.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5f880 con 0x55b66fd2e000 2023-11-08T16:41:49.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.408+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 46 ==== osd_op_reply(101 10000000213.00000000 [call] v131'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.409+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:102 17.2a 17:576ad4f4::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4000 con 0x55b66fd2e000 2023-11-08T16:41:49.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.409+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 47 ==== osd_op_reply(102 [pgnls start_epoch 131 out=101b] v131'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.410+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:103 17.2a 17:576ad4f4:::10000000245.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4380 con 0x55b66fd2e000 2023-11-08T16:41:49.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.410+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 48 ==== osd_op_reply(103 10000000245.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.410+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628357.0 2023-11-08T16:41:49.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.411+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:104 17.2a 17:576ad4f4:::10000000245.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4700 con 0x55b66fd2e000 2023-11-08T16:41:49.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.414+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 49 ==== osd_op_reply(104 10000000245.00000000 [call] v131'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.414+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:105 17.2a 17:57fe7535::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4a80 con 0x55b66fd2e000 2023-11-08T16:41:49.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.415+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 50 ==== osd_op_reply(105 [pgnls start_epoch 131 out=81b] v131'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.415+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:106 17.2a 17:57fe7535:::100000001f6.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4e00 con 0x55b66fd2e000 2023-11-08T16:41:49.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.415+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 51 ==== osd_op_reply(106 100000001f6.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.416+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628278.0 2023-11-08T16:41:49.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.416+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:107 17.2a 17:57fe7535:::100000001f6.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5180 con 0x55b66fd2e000 2023-11-08T16:41:49.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.419+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 52 ==== osd_op_reply(107 100000001f6.00000000 [call] v131'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.419+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:108 17.1a 17:58000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5500 con 0x55b66fd3b000 2023-11-08T16:41:49.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.420+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 18 ==== osd_op_reply(108 [pgnls start_epoch 131 out=101b] v127'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.420+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:109 17.1a 17:582f7b2d:::10000000226.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5880 con 0x55b66fd3b000 2023-11-08T16:41:49.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.421+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 19 ==== osd_op_reply(109 10000000226.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.421+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628326.0 2023-11-08T16:41:49.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.421+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:110 17.1a 17:582f7b2d:::10000000226.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5c00 con 0x55b66fd3b000 2023-11-08T16:41:49.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.424+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 20 ==== osd_op_reply(110 10000000226.00000000 [call] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.424+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:111 17.1a 17:5a2a1294::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66fd3b000 2023-11-08T16:41:49.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.425+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 21 ==== osd_op_reply(111 [pgnls start_epoch 131 out=81b] v131'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.425+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:112 17.1a 17:5a2a1294:::1000000023a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5f880 con 0x55b66fd3b000 2023-11-08T16:41:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.426+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 22 ==== osd_op_reply(112 1000000023a.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.426+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628346.0 2023-11-08T16:41:49.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.426+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:113 17.1a 17:5a2a1294:::1000000023a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175c00 con 0x55b66fd3b000 2023-11-08T16:41:49.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.430+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 23 ==== osd_op_reply(113 1000000023a.00000000 [call] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.430+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:114 17.3a 17:5c000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd2e400 2023-11-08T16:41:49.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.431+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(114 [pgnls start_epoch 131 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.431+0000 7ff8fe9fc640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b66f1b0400 0x55b66f186c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.431+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:115 17.6 17:60000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5880 con 0x55b66f1b0400 2023-11-08T16:41:49.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.432+0000 7ff903a06640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b66f1b0400 0x55b66f186c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.433+0000 7ff903a06640 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b66f1b0400 0x55b66f186c00 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.434+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(115 [pgnls start_epoch 131 out=81b] v127'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.434+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:116 17.6 17:614797eb:::10000000228.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66f1b0400 2023-11-08T16:41:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.435+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(116 10000000228.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.435+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628328.0 2023-11-08T16:41:49.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.435+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:117 17.6 17:614797eb:::10000000228.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66f1b0400 2023-11-08T16:41:49.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.439+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(117 10000000228.00000000 [call] v131'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.439+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:118 17.26 17:64000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd3b000 2023-11-08T16:41:49.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.440+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 24 ==== osd_op_reply(118 [pgnls start_epoch 131 out=101b] v127'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.441+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:119 17.26 17:658f2455:::100000001ff.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd3b000 2023-11-08T16:41:49.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.441+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 25 ==== osd_op_reply(119 100000001ff.00000000 [stat out=16b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.441+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628287.0 2023-11-08T16:41:49.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.442+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:120 17.26 17:658f2455:::100000001ff.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66fd3b000 2023-11-08T16:41:49.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.445+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 26 ==== osd_op_reply(120 100000001ff.00000000 [call] v131'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.445+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:121 17.26 17:65d05a5c::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4a80 con 0x55b66fd3b000 2023-11-08T16:41:49.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.446+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 27 ==== osd_op_reply(121 [pgnls start_epoch 131 out=101b] v131'22 uv22 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.447+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:122 17.26 17:65d05a5c:::10000000258.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5c00 con 0x55b66fd3b000 2023-11-08T16:41:49.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.448+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 28 ==== osd_op_reply(122 10000000258.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.448+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628376.0 2023-11-08T16:41:49.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.448+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:123 17.26 17:65d05a5c:::10000000258.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4e00 con 0x55b66fd3b000 2023-11-08T16:41:49.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.451+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 29 ==== osd_op_reply(123 10000000258.00000000 [call] v131'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.451+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:124 17.26 17:65e03fee::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5180 con 0x55b66fd3b000 2023-11-08T16:41:49.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.452+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 30 ==== osd_op_reply(124 [pgnls start_epoch 131 out=81b] v131'23 uv23 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.452+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:125 17.26 17:65e03fee:::10000000240.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4000 con 0x55b66fd3b000 2023-11-08T16:41:49.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.453+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 31 ==== osd_op_reply(125 10000000240.00000000 [stat out=16b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.453+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628352.0 2023-11-08T16:41:49.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.453+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:126 17.26 17:65e03fee:::10000000240.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66fd3b000 2023-11-08T16:41:49.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.458+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 32 ==== osd_op_reply(126 10000000240.00000000 [call] v131'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.458+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:127 17.16 17:68000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5500 con 0x55b66f1b0000 2023-11-08T16:41:49.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.459+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 10 ==== osd_op_reply(127 [pgnls start_epoch 131 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.459+0000 7ff8fe1fb640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:128 17.36 17:6c000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175c00 con 0x55b66fd3b000 2023-11-08T16:41:49.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.460+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 33 ==== osd_op_reply(128 [pgnls start_epoch 131 out=101b] v127'23 uv23 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.460+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:129 17.36 17:6cdfec98:::10000000255.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4700 con 0x55b66fd3b000 2023-11-08T16:41:49.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.461+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 34 ==== osd_op_reply(129 10000000255.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.461+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628373.0 2023-11-08T16:41:49.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.461+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:130 17.36 17:6cdfec98:::10000000255.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5880 con 0x55b66fd3b000 2023-11-08T16:41:49.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.465+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 35 ==== osd_op_reply(130 10000000255.00000000 [call] v131'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.465+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:131 17.36 17:6ee5e2c4::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5c00 con 0x55b66fd3b000 2023-11-08T16:41:49.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.466+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 36 ==== osd_op_reply(131 [pgnls start_epoch 131 out=101b] v131'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.466+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:132 17.36 17:6ee5e2c4:::100000001f9.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5500 con 0x55b66fd3b000 2023-11-08T16:41:49.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.467+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 37 ==== osd_op_reply(132 100000001f9.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.467+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628281.0 2023-11-08T16:41:49.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.467+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:133 17.36 17:6ee5e2c4:::100000001f9.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4000 con 0x55b66fd3b000 2023-11-08T16:41:49.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.470+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 38 ==== osd_op_reply(133 100000001f9.00000000 [call] v131'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.470+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:134 17.36 17:6f746908::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4a80 con 0x55b66fd3b000 2023-11-08T16:41:49.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.471+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 39 ==== osd_op_reply(134 [pgnls start_epoch 131 out=101b] v131'25 uv25 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.471+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:135 17.36 17:6f746908:::1000000022c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5c00 con 0x55b66fd3b000 2023-11-08T16:41:49.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.472+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 40 ==== osd_op_reply(135 1000000022c.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.472+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628332.0 2023-11-08T16:41:49.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.472+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:136 17.36 17:6f746908:::1000000022c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5880 con 0x55b66fd3b000 2023-11-08T16:41:49.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.476+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 41 ==== osd_op_reply(136 1000000022c.00000000 [call] v131'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.476+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:137 17.36 17:6fe38dc2::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd3b000 2023-11-08T16:41:49.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.476+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 42 ==== osd_op_reply(137 [pgnls start_epoch 131 out=81b] v131'26 uv26 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.477+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:138 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd3b000 2023-11-08T16:41:49.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.477+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 43 ==== osd_op_reply(138 100000001f7.00000000 [stat out=16b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.477+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628279.0 2023-11-08T16:41:49.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.477+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:139 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66fd3b000 2023-11-08T16:41:49.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.481+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 44 ==== osd_op_reply(139 100000001f7.00000000 [call] v131'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.481+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:140 17.e 17:70000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174380 con 0x55b66fd3b400 2023-11-08T16:41:49.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.482+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(140 [pgnls start_epoch 131 out=81b] v127'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.482+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:141 17.e 17:71c13145:::1000000022f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175880 con 0x55b66fd3b400 2023-11-08T16:41:49.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.483+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(141 1000000022f.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.483+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628335.0 2023-11-08T16:41:49.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.483+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:142 17.e 17:71c13145:::1000000022f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd3b400 2023-11-08T16:41:49.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.487+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(142 1000000022f.00000000 [call] v131'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.487+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:143 17.2e 17:74000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5880 con 0x55b66f1b0400 2023-11-08T16:41:49.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.488+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(143 [pgnls start_epoch 131 out=101b] v127'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.488+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:144 17.2e 17:74d13cfc:::1000000021c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5c00 con 0x55b66f1b0400 2023-11-08T16:41:49.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.489+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(144 1000000021c.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.489+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628316.0 2023-11-08T16:41:49.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.489+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:145 17.2e 17:74d13cfc:::1000000021c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5500 con 0x55b66f1b0400 2023-11-08T16:41:49.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.493+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(145 1000000021c.00000000 [call] v131'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.494+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:146 17.2e 17:7528f44f::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4000 con 0x55b66f1b0400 2023-11-08T16:41:49.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.495+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(146 [pgnls start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.495+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:147 17.2e 17:7528f44f:::10000000242.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4e00 con 0x55b66f1b0400 2023-11-08T16:41:49.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.496+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(147 10000000242.00000000 [stat out=16b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.496+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628354.0 2023-11-08T16:41:49.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.496+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:148 17.2e 17:7528f44f:::10000000242.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5180 con 0x55b66f1b0400 2023-11-08T16:41:49.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.500+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(148 10000000242.00000000 [call] v131'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.500+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:149 17.2e 17:7611a124::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4000 con 0x55b66f1b0400 2023-11-08T16:41:49.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.501+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(149 [pgnls start_epoch 131 out=81b] v131'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.501+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:150 17.2e 17:7611a124:::1000000021e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66f1b0400 2023-11-08T16:41:49.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.503+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 11 ==== osd_op_reply(150 1000000021e.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.503+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628318.0 2023-11-08T16:41:49.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.503+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:151 17.2e 17:7611a124:::1000000021e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5500 con 0x55b66f1b0400 2023-11-08T16:41:49.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.507+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 12 ==== osd_op_reply(151 1000000021e.00000000 [call] v131'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.507+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:152 17.1e 17:78000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4700 con 0x55b66fd2e400 2023-11-08T16:41:49.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.508+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(152 [pgnls start_epoch 131 out=101b] v127'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.509+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:153 17.1e 17:789b801b:::1000000020c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd2e400 2023-11-08T16:41:49.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.509+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(153 1000000020c.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.509+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628300.0 2023-11-08T16:41:49.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.509+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:154 17.1e 17:789b801b:::1000000020c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66fd2e400 2023-11-08T16:41:49.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.513+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(154 1000000020c.00000000 [call] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.513+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:155 17.1e 17:7a9d9b47::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174380 con 0x55b66fd2e400 2023-11-08T16:41:49.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.514+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(155 [pgnls start_epoch 131 out=101b] v131'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.514+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:156 17.1e 17:7a9d9b47:::1000000022b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175880 con 0x55b66fd2e400 2023-11-08T16:41:49.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.515+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(156 1000000022b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.515+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628331.0 2023-11-08T16:41:49.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.515+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:157 17.1e 17:7a9d9b47:::1000000022b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd2e400 2023-11-08T16:41:49.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.518+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(157 1000000022b.00000000 [call] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.518+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:158 17.1e 17:7ade2fd8::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175c00 con 0x55b66fd2e400 2023-11-08T16:41:49.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.519+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 12 ==== osd_op_reply(158 [pgnls start_epoch 131 out=81b] v131'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.519+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:159 17.1e 17:7ade2fd8:::1000000024e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4000 con 0x55b66fd2e400 2023-11-08T16:41:49.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.520+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 13 ==== osd_op_reply(159 1000000024e.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.520+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628366.0 2023-11-08T16:41:49.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.520+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:160 17.1e 17:7ade2fd8:::1000000024e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66fd2e400 2023-11-08T16:41:49.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.524+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 14 ==== osd_op_reply(160 1000000024e.00000000 [call] v131'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.524+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:161 17.3e 17:7c000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5500 con 0x55b66fd3b800 2023-11-08T16:41:49.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.525+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(161 [pgnls start_epoch 131 out=81b] v127'6 uv6 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.525+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:162 17.3e 17:7f918703:::10000000229.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4700 con 0x55b66fd3b800 2023-11-08T16:41:49.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.526+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(162 10000000229.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.526+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628329.0 2023-11-08T16:41:49.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.526+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:163 17.3e 17:7f918703:::10000000229.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4e00 con 0x55b66fd3b800 2023-11-08T16:41:49.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.530+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(163 10000000229.00000000 [call] v131'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.530+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:164 17.1 17:80000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5180 con 0x55b66fd3b000 2023-11-08T16:41:49.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.531+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 45 ==== osd_op_reply(164 [pgnls start_epoch 131 out=81b] v127'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.531+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:165 17.1 17:805bc955:::10000000224.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5880 con 0x55b66fd3b000 2023-11-08T16:41:49.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.532+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 46 ==== osd_op_reply(165 10000000224.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.532+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628324.0 2023-11-08T16:41:49.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.532+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:166 17.1 17:805bc955:::10000000224.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5c00 con 0x55b66fd3b000 2023-11-08T16:41:49.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.536+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 47 ==== osd_op_reply(166 10000000224.00000000 [call] v131'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.536+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:167 17.21 17:84000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5500 con 0x55b66fd3b000 2023-11-08T16:41:49.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.537+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 48 ==== osd_op_reply(167 [pgnls start_epoch 131 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.537+0000 7ff8fe1fb640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:168 17.11 17:88000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66fd2e000 2023-11-08T16:41:49.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.538+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 53 ==== osd_op_reply(168 [pgnls start_epoch 131 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.538+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:169 17.31 17:8c000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc000 con 0x55b66fd2e400 2023-11-08T16:41:49.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.539+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 15 ==== osd_op_reply(169 [pgnls start_epoch 131 out=101b] v127'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.540+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:170 17.31 17:8c51d9b0:::10000000216.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4000 con 0x55b66fd2e400 2023-11-08T16:41:49.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.540+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 16 ==== osd_op_reply(170 10000000216.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.540+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628310.0 2023-11-08T16:41:49.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.540+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:171 17.31 17:8c51d9b0:::10000000216.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd2e400 2023-11-08T16:41:49.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.544+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 17 ==== osd_op_reply(171 10000000216.00000000 [call] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.544+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:172 17.31 17:8fb7f7b7::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66fd2e400 2023-11-08T16:41:49.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.545+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 18 ==== osd_op_reply(172 [pgnls start_epoch 131 out=81b] v131'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.545+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:173 17.31 17:8fb7f7b7:::10000000251.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174380 con 0x55b66fd2e400 2023-11-08T16:41:49.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.546+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 19 ==== osd_op_reply(173 10000000251.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.546+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628369.0 2023-11-08T16:41:49.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.546+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:174 17.31 17:8fb7f7b7:::10000000251.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc000 con 0x55b66fd2e400 2023-11-08T16:41:49.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.549+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 20 ==== osd_op_reply(174 10000000251.00000000 [call] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.549+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:175 17.9 17:90000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dce00 con 0x55b66fd2e000 2023-11-08T16:41:49.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.550+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 54 ==== osd_op_reply(175 [pgnls start_epoch 131 out=101b] v127'6 uv6 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.550+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:176 17.9 17:915bff82:::1000000024d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd180 con 0x55b66fd2e000 2023-11-08T16:41:49.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.551+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 55 ==== osd_op_reply(176 1000000024d.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.551+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628365.0 2023-11-08T16:41:49.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.551+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:177 17.9 17:915bff82:::1000000024d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd500 con 0x55b66fd2e000 2023-11-08T16:41:49.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.555+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 56 ==== osd_op_reply(177 1000000024d.00000000 [call] v131'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.555+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:178 17.9 17:91e366d7::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd880 con 0x55b66fd2e000 2023-11-08T16:41:49.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.556+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 57 ==== osd_op_reply(178 [pgnls start_epoch 131 out=81b] v131'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.556+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:179 17.9 17:91e366d7:::10000000230.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1ddc00 con 0x55b66fd2e000 2023-11-08T16:41:49.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.557+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 58 ==== osd_op_reply(179 10000000230.00000000 [stat out=16b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.557+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628336.0 2023-11-08T16:41:49.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.557+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:180 17.9 17:91e366d7:::10000000230.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175880 con 0x55b66fd2e000 2023-11-08T16:41:49.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.560+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 59 ==== osd_op_reply(180 10000000230.00000000 [call] v131'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.560+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:181 17.29 17:94000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd3b800 2023-11-08T16:41:49.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.562+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(181 [pgnls start_epoch 131 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.562+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:182 17.29 17:94c02207:::1000000024a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175c00 con 0x55b66fd3b800 2023-11-08T16:41:49.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.563+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(182 1000000024a.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.563+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628362.0 2023-11-08T16:41:49.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.563+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:183 17.29 17:94c02207:::1000000024a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd3b800 2023-11-08T16:41:49.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.567+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 13 ==== osd_op_reply(183 1000000024a.00000000 [call] v131'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.567+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:184 17.29 17:95e8e66d::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4000 con 0x55b66fd3b800 2023-11-08T16:41:49.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.568+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 14 ==== osd_op_reply(184 [pgnls start_epoch 131 out=101b] v131'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.568+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:185 17.29 17:95e8e66d:::10000000256.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4000 con 0x55b66fd3b800 2023-11-08T16:41:49.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.569+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 15 ==== osd_op_reply(185 10000000256.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.570+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628374.0 2023-11-08T16:41:49.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.570+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:186 17.29 17:95e8e66d:::10000000256.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66fd3b800 2023-11-08T16:41:49.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.573+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 16 ==== osd_op_reply(186 10000000256.00000000 [call] v131'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.573+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:187 17.29 17:97401066::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dce00 con 0x55b66fd3b800 2023-11-08T16:41:49.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.574+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 17 ==== osd_op_reply(187 [pgnls start_epoch 131 out=81b] v131'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.575+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:188 17.29 17:97401066:::10000000247.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd180 con 0x55b66fd3b800 2023-11-08T16:41:49.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.575+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 18 ==== osd_op_reply(188 10000000247.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.576+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628359.0 2023-11-08T16:41:49.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.576+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:189 17.29 17:97401066:::10000000247.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66fd3b800 2023-11-08T16:41:49.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.579+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 19 ==== osd_op_reply(189 10000000247.00000000 [call] v131'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.579+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:190 17.19 17:98000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5500 con 0x55b66f1b0000 2023-11-08T16:41:49.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.580+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 11 ==== osd_op_reply(190 [pgnls start_epoch 131 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.581+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:191 17.19 17:992a6246:::10000000200.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4700 con 0x55b66f1b0000 2023-11-08T16:41:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.582+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 12 ==== osd_op_reply(191 10000000200.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.582+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628288.0 2023-11-08T16:41:49.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.582+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:192 17.19 17:992a6246:::10000000200.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4e00 con 0x55b66f1b0000 2023-11-08T16:41:49.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.586+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 13 ==== osd_op_reply(192 10000000200.00000000 [call] v131'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.586+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:193 17.19 17:997b7129::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4380 con 0x55b66f1b0000 2023-11-08T16:41:49.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.587+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 14 ==== osd_op_reply(193 [pgnls start_epoch 131 out=81b] v131'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.587+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:194 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4700 con 0x55b66f1b0000 2023-11-08T16:41:49.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.589+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 15 ==== osd_op_reply(194 100000001f8.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.589+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628280.0 2023-11-08T16:41:49.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.589+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:195 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4a80 con 0x55b66f1b0000 2023-11-08T16:41:49.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.592+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 16 ==== osd_op_reply(195 100000001f8.00000000 [call] v131'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.592+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:196 17.39 17:9c000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4e00 con 0x55b66f1b0000 2023-11-08T16:41:49.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.594+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 17 ==== osd_op_reply(196 [pgnls start_epoch 131 out=81b] v127'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.594+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:197 17.39 17:9f1351c3:::10000000222.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5180 con 0x55b66f1b0000 2023-11-08T16:41:49.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.595+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 18 ==== osd_op_reply(197 10000000222.00000000 [stat out=16b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.595+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628322.0 2023-11-08T16:41:49.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.595+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:198 17.39 17:9f1351c3:::10000000222.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd500 con 0x55b66f1b0000 2023-11-08T16:41:49.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.599+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 19 ==== osd_op_reply(198 10000000222.00000000 [call] v131'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.599+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:199 17.5 17:a0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd880 con 0x55b66f1b0400 2023-11-08T16:41:49.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.600+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 13 ==== osd_op_reply(199 [pgnls start_epoch 131 out=101b] v127'9 uv9 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.601+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:200 17.5 17:a0beda95:::10000000204.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1ddc00 con 0x55b66f1b0400 2023-11-08T16:41:49.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.602+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 14 ==== osd_op_reply(200 10000000204.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.602+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628292.0 2023-11-08T16:41:49.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.602+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:201 17.5 17:a0beda95:::10000000204.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dce00 con 0x55b66f1b0400 2023-11-08T16:41:49.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.606+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 15 ==== osd_op_reply(201 10000000204.00000000 [call] v131'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.606+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:202 17.5 17:a3a50212::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4a80 con 0x55b66f1b0400 2023-11-08T16:41:49.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.607+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 16 ==== osd_op_reply(202 [pgnls start_epoch 131 out=81b] v131'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.608+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:203 17.5 17:a3a50212:::10000000254.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5c00 con 0x55b66f1b0400 2023-11-08T16:41:49.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.609+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 17 ==== osd_op_reply(203 10000000254.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.609+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628372.0 2023-11-08T16:41:49.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.609+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:204 17.5 17:a3a50212:::10000000254.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5880 con 0x55b66f1b0400 2023-11-08T16:41:49.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.613+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 18 ==== osd_op_reply(204 10000000254.00000000 [call] v131'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.613+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:205 17.25 17:a4000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5180 con 0x55b66fd2e400 2023-11-08T16:41:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.614+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 21 ==== osd_op_reply(205 [pgnls start_epoch 131 out=101b] v127'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.614+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:206 17.25 17:a53878a9:::10000000237.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5500 con 0x55b66fd2e400 2023-11-08T16:41:49.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.615+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 22 ==== osd_op_reply(206 10000000237.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.615+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628343.0 2023-11-08T16:41:49.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.615+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:207 17.25 17:a53878a9:::10000000237.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4700 con 0x55b66fd2e400 2023-11-08T16:41:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.618+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 23 ==== osd_op_reply(207 10000000237.00000000 [call] v131'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.619+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:208 17.25 17:a7ae8a0f::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4000 con 0x55b66fd2e400 2023-11-08T16:41:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.619+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 24 ==== osd_op_reply(208 [pgnls start_epoch 131 out=81b] v131'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.620+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:209 17.25 17:a7ae8a0f:::10000000243.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66fd2e400 2023-11-08T16:41:49.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.620+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 25 ==== osd_op_reply(209 10000000243.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.620+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628355.0 2023-11-08T16:41:49.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.620+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:210 17.25 17:a7ae8a0f:::10000000243.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5880 con 0x55b66fd2e400 2023-11-08T16:41:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.624+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 26 ==== osd_op_reply(210 10000000243.00000000 [call] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.624+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:211 17.15 17:a8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5c00 con 0x55b66f1b0000 2023-11-08T16:41:49.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.626+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 20 ==== osd_op_reply(211 [pgnls start_epoch 131 out=81b] v127'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.626+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:212 17.15 17:a8e69ea8:::1000000020d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5500 con 0x55b66f1b0000 2023-11-08T16:41:49.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.627+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 21 ==== osd_op_reply(212 1000000020d.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.627+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628301.0 2023-11-08T16:41:49.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.627+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:213 17.15 17:a8e69ea8:::1000000020d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4000 con 0x55b66f1b0000 2023-11-08T16:41:49.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.632+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.5 v2:172.21.15.93:6802/2406579932 22 ==== osd_op_reply(213 1000000020d.00000000 [call] v131'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66f1b0000 2023-11-08T16:41:49.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.632+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:214 17.35 17:ac000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd3b800 2023-11-08T16:41:49.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.633+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 20 ==== osd_op_reply(214 [pgnls start_epoch 131 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.633+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:215 17.d 17:b0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc380 con 0x55b66fd2e800 2023-11-08T16:41:49.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.634+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 18 ==== osd_op_reply(215 [pgnls start_epoch 131 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.634+0000 7ff8fe1fb640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:216 17.2d 17:b4000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5f880 con 0x55b66fd2e800 2023-11-08T16:41:49.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.635+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 19 ==== osd_op_reply(216 [pgnls start_epoch 131 out=81b] v127'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.635+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:217 17.2d 17:b79a54f6:::10000000218.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd2e800 2023-11-08T16:41:49.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.636+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 20 ==== osd_op_reply(217 10000000218.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.636+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628312.0 2023-11-08T16:41:49.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.636+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:218 17.2d 17:b79a54f6:::10000000218.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66fd2e800 2023-11-08T16:41:49.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.640+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 21 ==== osd_op_reply(218 10000000218.00000000 [call] v131'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.642 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.640+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:219 17.1d 17:b8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175880 con 0x55b66fd2e800 2023-11-08T16:41:49.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.641+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 22 ==== osd_op_reply(219 [pgnls start_epoch 131 out=81b] v127'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.642+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:220 17.1d 17:bab97ecf:::10000000246.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd2e800 2023-11-08T16:41:49.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.643+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 23 ==== osd_op_reply(220 10000000246.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.643+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628358.0 2023-11-08T16:41:49.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.643+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:221 17.1d 17:bab97ecf:::10000000246.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175c00 con 0x55b66fd2e800 2023-11-08T16:41:49.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.647+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 24 ==== osd_op_reply(221 10000000246.00000000 [call] v131'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.647+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:222 17.3d 17:bc000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd2e800 2023-11-08T16:41:49.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.648+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 25 ==== osd_op_reply(222 [pgnls start_epoch 131 out=101b] v127'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.648+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:223 17.3d 17:bc0c3701:::10000000219.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd500 con 0x55b66fd2e800 2023-11-08T16:41:49.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.649+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 26 ==== osd_op_reply(223 10000000219.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.649+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628313.0 2023-11-08T16:41:49.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.649+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:224 17.3d 17:bc0c3701:::10000000219.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc380 con 0x55b66fd2e800 2023-11-08T16:41:49.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.653+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 27 ==== osd_op_reply(224 10000000219.00000000 [call] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.653+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:225 17.3d 17:bfcd3380::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc000 con 0x55b66fd2e800 2023-11-08T16:41:49.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.654+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 28 ==== osd_op_reply(225 [pgnls start_epoch 131 out=81b] v131'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.654+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:226 17.3d 17:bfcd3380:::10000000205.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd880 con 0x55b66fd2e800 2023-11-08T16:41:49.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.655+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 29 ==== osd_op_reply(226 10000000205.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.655+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628293.0 2023-11-08T16:41:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.655+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:227 17.3d 17:bfcd3380:::10000000205.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1ddc00 con 0x55b66fd2e800 2023-11-08T16:41:49.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.658+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 30 ==== osd_op_reply(227 10000000205.00000000 [call] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.659+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:228 17.3 17:c0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dce00 con 0x55b66fd3b800 2023-11-08T16:41:49.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.659+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 21 ==== osd_op_reply(228 [pgnls start_epoch 131 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.660+0000 7ff8fe9fc640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:229 17.23 17:c4000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc700 con 0x55b66fd3b800 2023-11-08T16:41:49.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.661+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 22 ==== osd_op_reply(229 [pgnls start_epoch 131 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.661+0000 7ff8fe1fb640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:230 17.13 17:c8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f8000 con 0x55b66fd2e000 2023-11-08T16:41:49.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.662+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 60 ==== osd_op_reply(230 [pgnls start_epoch 131 out=101b] v127'25 uv25 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.662+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:231 17.13 17:c8893608:::1000000023c.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd180 con 0x55b66fd2e000 2023-11-08T16:41:49.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.663+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 61 ==== osd_op_reply(231 1000000023c.00000000 [stat out=16b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.663+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628348.0 2023-11-08T16:41:49.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.663+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:232 17.13 17:c8893608:::1000000023c.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4e00 con 0x55b66fd2e000 2023-11-08T16:41:49.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.667+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 62 ==== osd_op_reply(232 1000000023c.00000000 [call] v131'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.667+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:233 17.13 17:c933d8fa::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4a80 con 0x55b66fd2e000 2023-11-08T16:41:49.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.668+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 63 ==== osd_op_reply(233 [pgnls start_epoch 131 out=101b] v131'26 uv26 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.668+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:234 17.13 17:c933d8fa:::10000000241.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5c00 con 0x55b66fd2e000 2023-11-08T16:41:49.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.669+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 64 ==== osd_op_reply(234 10000000241.00000000 [stat out=16b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.669+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628353.0 2023-11-08T16:41:49.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.669+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:235 17.13 17:c933d8fa:::10000000241.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5880 con 0x55b66fd2e000 2023-11-08T16:41:49.673 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:41:49.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.672+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 65 ==== osd_op_reply(235 10000000241.00000000 [call] v131'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.672+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:236 17.13 17:c9855970::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd2e000 2023-11-08T16:41:49.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.673+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 66 ==== osd_op_reply(236 [pgnls start_epoch 131 out=101b] v131'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.673+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:237 17.13 17:c9855970:::10000000239.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66fd2e000 2023-11-08T16:41:49.683 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:41:49.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.674+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 67 ==== osd_op_reply(237 10000000239.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.674+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628345.0 2023-11-08T16:41:49.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.674+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:238 17.13 17:c9855970:::10000000239.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174380 con 0x55b66fd2e000 2023-11-08T16:41:49.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.678+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 68 ==== osd_op_reply(238 10000000239.00000000 [call] v131'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.678+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:239 17.13 17:ca8fdbba::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5f880 con 0x55b66fd2e000 2023-11-08T16:41:49.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.679+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 69 ==== osd_op_reply(239 [pgnls start_epoch 131 out=101b] v131'28 uv28 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.679+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:240 17.13 17:ca8fdbba:::100000001fb.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4380 con 0x55b66fd2e000 2023-11-08T16:41:49.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.680+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 70 ==== osd_op_reply(240 100000001fb.00000000 [stat out=16b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.680+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628283.0 2023-11-08T16:41:49.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.680+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:241 17.13 17:ca8fdbba:::100000001fb.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4700 con 0x55b66fd2e000 2023-11-08T16:41:49.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.684+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 71 ==== osd_op_reply(241 100000001fb.00000000 [call] v131'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.684+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:242 17.13 17:cbcb6224::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4a80 con 0x55b66fd2e000 2023-11-08T16:41:49.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.684+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 72 ==== osd_op_reply(242 [pgnls start_epoch 131 out=81b] v131'29 uv29 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.685+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:243 17.13 17:cbcb6224:::10000000212.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4e00 con 0x55b66fd2e000 2023-11-08T16:41:49.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.685+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 73 ==== osd_op_reply(243 10000000212.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.685+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628306.0 2023-11-08T16:41:49.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.685+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:244 17.13 17:cbcb6224:::10000000212.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5180 con 0x55b66fd2e000 2023-11-08T16:41:49.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.688+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.3 v2:172.21.15.81:6816/2756229107 74 ==== osd_op_reply(244 10000000212.00000000 [call] v131'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd2e000 2023-11-08T16:41:49.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.689+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:245 17.33 17:cc000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5c00 con 0x55b66fd2e800 2023-11-08T16:41:49.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.690+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 31 ==== osd_op_reply(245 [pgnls start_epoch 131 out=81b] v127'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.690+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:246 17.33 17:cf9e7a4f:::10000000248.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5500 con 0x55b66fd2e800 2023-11-08T16:41:49.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.691+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 32 ==== osd_op_reply(246 10000000248.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.691+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628360.0 2023-11-08T16:41:49.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.691+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:247 17.33 17:cf9e7a4f:::10000000248.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4000 con 0x55b66fd2e800 2023-11-08T16:41:49.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.694+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.7 v2:172.21.15.93:6818/433691658 33 ==== osd_op_reply(247 10000000248.00000000 [call] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd2e800 2023-11-08T16:41:49.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.694+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:248 17.b 17:d0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f8000 con 0x55b66fd3b000 2023-11-08T16:41:49.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.695+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 49 ==== osd_op_reply(248 [pgnls start_epoch 131 out=101b] v127'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.695+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:249 17.b 17:d0d26e50:::10000000225.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f8e00 con 0x55b66fd3b000 2023-11-08T16:41:49.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.695+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 50 ==== osd_op_reply(249 10000000225.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.695+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628325.0 2023-11-08T16:41:49.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.695+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:250 17.b 17:d0d26e50:::10000000225.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f9180 con 0x55b66fd3b000 2023-11-08T16:41:49.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.698+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 51 ==== osd_op_reply(250 10000000225.00000000 [call] v131'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.698+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:251 17.b 17:d2dbd4ac::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f9500 con 0x55b66fd3b000 2023-11-08T16:41:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.699+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 52 ==== osd_op_reply(251 [pgnls start_epoch 131 out=101b] v131'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.699+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:252 17.b 17:d2dbd4ac:::10000000217.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f9880 con 0x55b66fd3b000 2023-11-08T16:41:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.700+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 53 ==== osd_op_reply(252 10000000217.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.700+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628311.0 2023-11-08T16:41:49.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.700+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:253 17.b 17:d2dbd4ac:::10000000217.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f9c00 con 0x55b66fd3b000 2023-11-08T16:41:49.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.703+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 54 ==== osd_op_reply(253 10000000217.00000000 [call] v131'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.703+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:254 17.b 17:d3eda40d::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd3b000 2023-11-08T16:41:49.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.703+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 55 ==== osd_op_reply(254 [pgnls start_epoch 131 out=81b] v131'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.704+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:255 17.b 17:d3eda40d:::1000000020f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66fd3b000 2023-11-08T16:41:49.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.704+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 56 ==== osd_op_reply(255 1000000020f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.704+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628303.0 2023-11-08T16:41:49.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.704+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:256 17.b 17:d3eda40d:::1000000020f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66fd3b000 2023-11-08T16:41:49.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.707+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 57 ==== osd_op_reply(256 1000000020f.00000000 [call] v131'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.707+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:257 17.2b 17:d4000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174380 con 0x55b66fd3b800 2023-11-08T16:41:49.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.708+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 23 ==== osd_op_reply(257 [pgnls start_epoch 131 out=101b] v127'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.708+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:258 17.2b 17:d4a58307:::10000000203.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4e00 con 0x55b66fd3b800 2023-11-08T16:41:49.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.709+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 24 ==== osd_op_reply(258 10000000203.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.709+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628291.0 2023-11-08T16:41:49.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.709+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:259 17.2b 17:d4a58307:::10000000203.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4a80 con 0x55b66fd3b800 2023-11-08T16:41:49.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.711+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 25 ==== osd_op_reply(259 10000000203.00000000 [call] v131'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.711+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:260 17.2b 17:d5ee56ee::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5c00 con 0x55b66fd3b800 2023-11-08T16:41:49.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.712+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 26 ==== osd_op_reply(260 [pgnls start_epoch 131 out=81b] v131'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.712+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:261 17.2b 17:d5ee56ee:::1000000022d.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5880 con 0x55b66fd3b800 2023-11-08T16:41:49.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.713+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 27 ==== osd_op_reply(261 1000000022d.00000000 [stat out=16b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.713+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628333.0 2023-11-08T16:41:49.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.713+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:262 17.2b 17:d5ee56ee:::1000000022d.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dce00 con 0x55b66fd3b800 2023-11-08T16:41:49.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.715+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 28 ==== osd_op_reply(262 1000000022d.00000000 [call] v131'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.715+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:263 17.1b 17:d8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc700 con 0x55b66fd3b800 2023-11-08T16:41:49.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.716+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 29 ==== osd_op_reply(263 [pgnls start_epoch 131 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.716+0000 7ff8fe1fb640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:264 17.3b 17:dc000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f8380 con 0x55b66fd3b400 2023-11-08T16:41:49.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.716+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 5 ==== osd_op_reply(264 [pgnls start_epoch 131 out=101b] v127'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.716+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:265 17.3b 17:dd50df43:::10000000249.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd180 con 0x55b66fd3b400 2023-11-08T16:41:49.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.717+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 6 ==== osd_op_reply(265 10000000249.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.717+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628361.0 2023-11-08T16:41:49.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.717+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:266 17.3b 17:dd50df43:::10000000249.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66fd3b400 2023-11-08T16:41:49.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.719+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 7 ==== osd_op_reply(266 10000000249.00000000 [call] v131'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.719+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:267 17.3b 17:deccbd37::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f8000 con 0x55b66fd3b400 2023-11-08T16:41:49.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.719+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 8 ==== osd_op_reply(267 [pgnls start_epoch 131 out=101b] v131'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.719+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:268 17.3b 17:deccbd37:::100000001fe.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5c00 con 0x55b66fd3b400 2023-11-08T16:41:49.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.720+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 9 ==== osd_op_reply(268 100000001fe.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.720+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628286.0 2023-11-08T16:41:49.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.720+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:269 17.3b 17:deccbd37:::100000001fe.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5500 con 0x55b66fd3b400 2023-11-08T16:41:49.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.721+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 10 ==== osd_op_reply(269 100000001fe.00000000 [call] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.721+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:270 17.3b 17:ded14b84::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c4000 con 0x55b66fd3b400 2023-11-08T16:41:49.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.722+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 11 ==== osd_op_reply(270 [pgnls start_epoch 131 out=81b] v131'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.722+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:271 17.3b 17:ded14b84:::1000000024b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd500 con 0x55b66fd3b400 2023-11-08T16:41:49.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.722+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 12 ==== osd_op_reply(271 1000000024b.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.722+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628363.0 2023-11-08T16:41:49.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.722+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:272 17.3b 17:ded14b84:::1000000024b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc380 con 0x55b66fd3b400 2023-11-08T16:41:49.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.724+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.2 v2:172.21.15.81:6824/3759311428 13 ==== osd_op_reply(272 1000000024b.00000000 [call] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd3b400 2023-11-08T16:41:49.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.724+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:273 17.7 17:e0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc000 con 0x55b66fd3b000 2023-11-08T16:41:49.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.725+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 58 ==== osd_op_reply(273 [pgnls start_epoch 131 out=101b] v127'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.725+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:274 17.7 17:e15a914f:::10000000207.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd880 con 0x55b66fd3b000 2023-11-08T16:41:49.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.726+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 59 ==== osd_op_reply(274 10000000207.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.726+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628295.0 2023-11-08T16:41:49.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.726+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:275 17.7 17:e15a914f:::10000000207.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1ddc00 con 0x55b66fd3b000 2023-11-08T16:41:49.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.728+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 60 ==== osd_op_reply(275 10000000207.00000000 [call] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.728+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:276 17.7 17:e2e07930::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66fd3b000 2023-11-08T16:41:49.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.728+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 61 ==== osd_op_reply(276 [pgnls start_epoch 131 out=101b] v131'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.728+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:277 17.7 17:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175880 con 0x55b66fd3b000 2023-11-08T16:41:49.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.729+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 62 ==== osd_op_reply(277 10000000001.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.729+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511627777.0 2023-11-08T16:41:49.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.729+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:278 17.7 17:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd3b000 2023-11-08T16:41:49.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.731+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 63 ==== osd_op_reply(278 10000000001.00000000 [call] v131'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.731+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:279 17.7 17:e37752f2::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175c00 con 0x55b66fd3b000 2023-11-08T16:41:49.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.731+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 64 ==== osd_op_reply(279 [pgnls start_epoch 131 out=81b] v131'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.731+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:280 17.7 17:e37752f2:::1000000021f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174e00 con 0x55b66fd3b000 2023-11-08T16:41:49.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.732+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 65 ==== osd_op_reply(280 1000000021f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.732+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628319.0 2023-11-08T16:41:49.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.732+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:281 17.7 17:e37752f2:::1000000021f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f8e00 con 0x55b66fd3b000 2023-11-08T16:41:49.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.734+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 66 ==== osd_op_reply(281 1000000021f.00000000 [call] v131'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.734+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:282 17.27 17:e4000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f9180 con 0x55b66fd3b800 2023-11-08T16:41:49.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.735+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 30 ==== osd_op_reply(282 [pgnls start_epoch 131 out=101b] v127'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.735+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:283 17.27 17:e40c9c52:::10000000210.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f9500 con 0x55b66fd3b800 2023-11-08T16:41:49.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.735+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 31 ==== osd_op_reply(283 10000000210.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.735+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628304.0 2023-11-08T16:41:49.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.735+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:284 17.27 17:e40c9c52:::10000000210.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f9880 con 0x55b66fd3b800 2023-11-08T16:41:49.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.737+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 32 ==== osd_op_reply(284 10000000210.00000000 [call] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.737+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:285 17.27 17:e5671758::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f9c00 con 0x55b66fd3b800 2023-11-08T16:41:49.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.738+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 33 ==== osd_op_reply(285 [pgnls start_epoch 131 out=101b] v131'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.738+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:286 17.27 17:e5671758:::1000000024f.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f8380 con 0x55b66fd3b800 2023-11-08T16:41:49.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.739+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 34 ==== osd_op_reply(286 1000000024f.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.739+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628367.0 2023-11-08T16:41:49.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.739+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:287 17.27 17:e5671758:::1000000024f.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f8000 con 0x55b66fd3b800 2023-11-08T16:41:49.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.740+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 35 ==== osd_op_reply(287 1000000024f.00000000 [call] v131'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.740+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:288 17.27 17:e5993619::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66ee5fc00 con 0x55b66fd3b800 2023-11-08T16:41:49.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.741+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 36 ==== osd_op_reply(288 [pgnls start_epoch 131 out=101b] v131'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.741+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:289 17.27 17:e5993619:::10000000208.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc000 con 0x55b66fd3b800 2023-11-08T16:41:49.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.741+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 37 ==== osd_op_reply(289 10000000208.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.741+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628296.0 2023-11-08T16:41:49.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.741+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:290 17.27 17:e5993619:::10000000208.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd880 con 0x55b66fd3b800 2023-11-08T16:41:49.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.743+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 38 ==== osd_op_reply(290 10000000208.00000000 [call] v131'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.743+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:291 17.27 17:e5a5749e::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1ddc00 con 0x55b66fd3b800 2023-11-08T16:41:49.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.744+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 39 ==== osd_op_reply(291 [pgnls start_epoch 131 out=101b] v131'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.744+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:292 17.27 17:e5a5749e:::1000000021a.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd180 con 0x55b66fd3b800 2023-11-08T16:41:49.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.744+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 40 ==== osd_op_reply(292 1000000021a.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.744+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628314.0 2023-11-08T16:41:49.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.744+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:293 17.27 17:e5a5749e:::1000000021a.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1c5880 con 0x55b66fd3b800 2023-11-08T16:41:49.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.747+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 41 ==== osd_op_reply(293 1000000021a.00000000 [call] v131'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.747+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:294 17.27 17:e692fc64::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5180 con 0x55b66fd3b800 2023-11-08T16:41:49.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.747+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 42 ==== osd_op_reply(294 [pgnls start_epoch 131 out=81b] v131'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.747+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:295 17.27 17:e692fc64:::1000000021b.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5500 con 0x55b66fd3b800 2023-11-08T16:41:49.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.748+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 43 ==== osd_op_reply(295 1000000021b.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.748+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628315.0 2023-11-08T16:41:49.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.748+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:296 17.27 17:e692fc64:::1000000021b.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4700 con 0x55b66fd3b800 2023-11-08T16:41:49.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.749+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 44 ==== osd_op_reply(296 1000000021b.00000000 [call] v131'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.749+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:297 17.17 17:e8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4000 con 0x55b66f1b0400 2023-11-08T16:41:49.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.750+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 19 ==== osd_op_reply(297 [pgnls start_epoch 131 out=101b] v127'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.750+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:298 17.17 17:eace0b58:::10000000215.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66f1b0400 2023-11-08T16:41:49.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.751+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 20 ==== osd_op_reply(298 10000000215.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.751+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628309.0 2023-11-08T16:41:49.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.751+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:299 17.17 17:eace0b58:::10000000215.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66f1b0400 2023-11-08T16:41:49.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.753+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 21 ==== osd_op_reply(299 10000000215.00000000 [call] v131'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.753+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:300 17.17 17:eb1b33a4::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174700 con 0x55b66f1b0400 2023-11-08T16:41:49.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.754+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 22 ==== osd_op_reply(300 [pgnls start_epoch 131 out=81b] v131'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.754+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:301 17.17 17:eb1b33a4:::10000000253.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175180 con 0x55b66f1b0400 2023-11-08T16:41:49.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.755+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 23 ==== osd_op_reply(301 10000000253.00000000 [stat out=16b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.755+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628371.0 2023-11-08T16:41:49.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.755+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:302 17.17 17:eb1b33a4:::10000000253.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174000 con 0x55b66f1b0400 2023-11-08T16:41:49.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.758+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 24 ==== osd_op_reply(302 10000000253.00000000 [call] v131'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.758+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:303 17.37 17:ec000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175880 con 0x55b66fd3b000 2023-11-08T16:41:49.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.758+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 67 ==== osd_op_reply(303 [pgnls start_epoch 131 out=49b] v127'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.758+0000 7ff8fe1fb640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:304 17.f 17:f0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f8700 con 0x55b66fd2e400 2023-11-08T16:41:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.759+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 27 ==== osd_op_reply(304 [pgnls start_epoch 131 out=81b] v127'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.759+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:305 17.f 17:f25dd5df:::1000000022e.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175500 con 0x55b66fd2e400 2023-11-08T16:41:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.760+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 28 ==== osd_op_reply(305 1000000022e.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.760+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628334.0 2023-11-08T16:41:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.760+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:306 17.f 17:f25dd5df:::1000000022e.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f175c00 con 0x55b66fd2e400 2023-11-08T16:41:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.762+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.1 v2:172.21.15.81:6800/2060798739 29 ==== osd_op_reply(306 1000000022e.00000000 [call] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66fd2e400 2023-11-08T16:41:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.762+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:307 17.2f 17:f4000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f9180 con 0x55b66fd3b800 2023-11-08T16:41:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.763+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 <== osd.4 v2:172.21.15.93:6800/3259756279 45 ==== osd_op_reply(307 [pgnls start_epoch 131 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55b66fb62fc0 con 0x55b66fd3b800 2023-11-08T16:41:49.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.763+0000 7ff8fe1fb640 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:308 17.1f 17:f8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1f8a80 con 0x55b66f1b0400 2023-11-08T16:41:49.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.764+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 25 ==== osd_op_reply(308 [pgnls start_epoch 131 out=101b] v127'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.764+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:309 17.1f 17:f87ea8f1:::10000000238.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dd500 con 0x55b66f1b0400 2023-11-08T16:41:49.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.765+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 26 ==== osd_op_reply(309 10000000238.00000000 [stat out=16b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.765+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628344.0 2023-11-08T16:41:49.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.765+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:310 17.1f 17:f87ea8f1:::10000000238.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc380 con 0x55b66f1b0400 2023-11-08T16:41:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.767+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 27 ==== osd_op_reply(310 10000000238.00000000 [call] v131'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.767+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:311 17.1f 17:f8e2d5cc::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dce00 con 0x55b66f1b0400 2023-11-08T16:41:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.768+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 28 ==== osd_op_reply(311 [pgnls start_epoch 131 out=81b] v131'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.768+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:312 17.1f 17:f8e2d5cc:::10000000214.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1dc700 con 0x55b66f1b0400 2023-11-08T16:41:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.768+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 29 ==== osd_op_reply(312 10000000214.00000000 [stat out=16b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.768+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628308.0 2023-11-08T16:41:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.768+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:313 17.1f 17:f8e2d5cc:::10000000214.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4000 con 0x55b66f1b0400 2023-11-08T16:41:49.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.772+0000 7ff903a06640 1 -- 172.21.15.81:0/1186316120 <== osd.6 v2:172.21.15.93:6816/607960134 30 ==== osd_op_reply(313 10000000214.00000000 [call] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62d80 con 0x55b66f1b0400 2023-11-08T16:41:49.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.772+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:314 17.3f 17:fc000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4380 con 0x55b66fd3b000 2023-11-08T16:41:49.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.772+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 68 ==== osd_op_reply(314 [pgnls start_epoch 131 out=101b] v127'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.772+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:315 17.3f 17:fd83cc12:::10000000232.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4e00 con 0x55b66fd3b000 2023-11-08T16:41:49.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.773+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 69 ==== osd_op_reply(315 10000000232.00000000 [stat out=16b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.773+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628338.0 2023-11-08T16:41:49.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.773+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:316 17.3f 17:fd83cc12:::10000000232.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a4a80 con 0x55b66fd3b000 2023-11-08T16:41:49.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.775+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 70 ==== osd_op_reply(316 10000000232.00000000 [call] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.775+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:317 17.3f 17:feb1c803::::head [pgnls start_epoch 131 in=59b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5c00 con 0x55b66fd3b000 2023-11-08T16:41:49.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.775+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 71 ==== osd_op_reply(317 [pgnls start_epoch 131 out=81b] v131'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.775+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:318 17.3f 17:feb1c803:::100000001fa.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f1a5880 con 0x55b66fd3b000 2023-11-08T16:41:49.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.776+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 72 ==== osd_op_reply(318 100000001fa.00000000 [stat out=16b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.776+0000 7ff90523d5c0 10 datascan.operator(): handling object 1099511628282.0 2023-11-08T16:41:49.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.776+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:319 17.3f 17:feb1c803:::100000001fa.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x55b66f174a80 con 0x55b66fd3b000 2023-11-08T16:41:49.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff902a04640 1 -- 172.21.15.81:0/1186316120 <== osd.0 v2:172.21.15.81:6808/1666604866 73 ==== osd_op_reply(319 100000001fa.00000000 [call] v131'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55b66fb62b40 con 0x55b66fd3b000 2023-11-08T16:41:49.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b66fd3b000 msgr2=0x55b66f184000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x55b66fd3b000 0x55b66f184000 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b66fd2e400 msgr2=0x55b66ee71b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x55b66fd2e400 0x55b66ee71b80 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b66fd3b400 msgr2=0x55b66f184580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x55b66fd3b400 0x55b66f184580 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b66fd2e000 msgr2=0x55b66ee71600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x55b66fd2e000 0x55b66ee71600 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b66fd3b800 msgr2=0x55b66f184b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x55b66fd3b800 0x55b66f184b00 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b66f1b0000 msgr2=0x55b66f186680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x55b66f1b0000 0x55b66f186680 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 reap_dead start 2023-11-08T16:41:49.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b66f1b0400 msgr2=0x55b66f186c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x55b66f1b0400 0x55b66f186c00 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 reap_dead start 2023-11-08T16:41:49.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b66fd2e800 msgr2=0x55b66ee72100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x55b66fd2e800 0x55b66ee72100 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 reap_dead start 2023-11-08T16:41:49.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff903205640 1 -- 172.21.15.81:0/1186316120 reap_dead start 2023-11-08T16:41:49.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b66fd3ac00 msgr2=0x55b66ee73700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b66fd3ac00 0x55b66ee73700 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x55b66fcd4ae0 tx=0x55b66fcdcc00 comp rx=0 tx=0).stop 2023-11-08T16:41:49.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2ec00 msgr2=0x55b66ee72680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2ec00 0x55b66ee72680 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x55b66fcd57d0 tx=0x55b66fcdcde0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 shutdown_connections 2023-11-08T16:41:49.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x55b66fd3ac00 0x55b66ee73700 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/1186316120 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66fd2ec00 0x55b66ee72680 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.778+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 >> 172.21.15.81:0/1186316120 conn(0x55b66fbd7c00 msgr2=0x55b66fb81800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.779+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 shutdown_connections 2023-11-08T16:41:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.779+0000 7ff90523d5c0 1 -- 172.21.15.81:0/1186316120 wait complete. 2023-11-08T16:41:49.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.779+0000 7ff90523d5c0 1 -- 172.21.15.81:0/855502515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66ee07c00 msgr2=0x55b66ee71080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.779+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/855502515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66ee07c00 0x55b66ee71080 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x55b66fc18b40 tx=0x55b66fbda3c0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.779+0000 7ff90523d5c0 1 -- 172.21.15.81:0/855502515 shutdown_connections 2023-11-08T16:41:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.779+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/855502515 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x55b66ee07c00 0x55b66ee71080 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.779+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/855502515 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b66ee07800 0x55b66ee70b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.780+0000 7ff90523d5c0 1 --2- 172.21.15.81:0/855502515 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x55b66ee06c00 0x55b66ee70580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.780+0000 7ff90523d5c0 1 -- 172.21.15.81:0/855502515 >> 172.21.15.81:0/855502515 conn(0x55b66ee07000 msgr2=0x55b66fb81000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.780+0000 7ff90523d5c0 1 -- 172.21.15.81:0/855502515 shutdown_connections 2023-11-08T16:41:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.780+0000 7ff90523d5c0 1 -- 172.21.15.81:0/855502515 wait complete. 2023-11-08T16:41:49.790 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents'] in time 0:00:00.634871, result: 2023-11-08T16:41:49.791 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes 2023-11-08T16:41:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.961+0000 7f9189c365c0 1 -- start start 2023-11-08T16:41:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.961+0000 7f9189c365c0 1 Processor -- start 2023-11-08T16:41:49.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.962+0000 7f9189c365c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eede60800 0x564eed0fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.962+0000 7f9189c365c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091c00 0x564eed0fab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.962+0000 7f9189c365c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede61c00 0x564eed0fb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.962+0000 7f9189c365c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eed0d8140 con 0x564eede61c00 2023-11-08T16:41:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.962+0000 7f9189c365c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eed0d9180 con 0x564eede60800 2023-11-08T16:41:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.962+0000 7f9189c365c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x564eed15edc0 con 0x564eed091c00 2023-11-08T16:41:49.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.962+0000 7f9188344640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede61c00 0x564eed0fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.962+0000 7f9188344640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede61c00 0x564eed0fb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48116/0 (socket says 172.21.15.81:48116) 2023-11-08T16:41:49.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.962+0000 7f9188344640 1 -- 172.21.15.81:0/1804835141 learned_addr learned my addr 172.21.15.81:0/1804835141 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:49.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f9187342640 1 --2- 172.21.15.81:0/1804835141 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091c00 0x564eed0fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f9188344640 1 -- 172.21.15.81:0/1804835141 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091c00 msgr2=0x564eed0fab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:49.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f9187b43640 1 --2- 172.21.15.81:0/1804835141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eede60800 0x564eed0fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f9188344640 1 --2- 172.21.15.81:0/1804835141 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091c00 0x564eed0fab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f9188344640 1 -- 172.21.15.81:0/1804835141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eede60800 msgr2=0x564eed0fbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:49.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f9188344640 1 --2- 172.21.15.81:0/1804835141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eede60800 0x564eed0fbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f9188344640 1 -- 172.21.15.81:0/1804835141 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eed0b6820 con 0x564eede61c00 2023-11-08T16:41:49.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f9188344640 1 --2- 172.21.15.81:0/1804835141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede61c00 0x564eed0fb600 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x564eedeab110 tx=0x564eede6aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=509dc7ad0aca984c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f918533e640 1 -- 172.21.15.81:0/1804835141 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564eed0d6f20 con 0x564eede61c00 2023-11-08T16:41:49.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f918533e640 1 -- 172.21.15.81:0/1804835141 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564eedf56000 con 0x564eede61c00 2023-11-08T16:41:49.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f918533e640 1 -- 172.21.15.81:0/1804835141 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564eed0d71e0 con 0x564eede61c00 2023-11-08T16:41:49.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f9189c365c0 1 -- 172.21.15.81:0/1804835141 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564eed0b6b60 con 0x564eede61c00 2023-11-08T16:41:49.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f9189c365c0 1 -- 172.21.15.81:0/1804835141 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x564eed0b6d00 con 0x564eede61c00 2023-11-08T16:41:49.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.963+0000 7f9189c365c0 4 waiting for MDS map... 2023-11-08T16:41:49.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.964+0000 7f918533e640 1 -- 172.21.15.81:0/1804835141 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x564eed150c40 con 0x564eede61c00 2023-11-08T16:41:49.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.964+0000 7f918533e640 1 -- 172.21.15.81:0/1804835141 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 114) v1 ==== 2723+0+0 (secure 0 0 0) 0x564eed0e0f00 con 0x564eede61c00 2023-11-08T16:41:49.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.964+0000 7f9189c365c0 4 Got MDS map 114 2023-11-08T16:41:49.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.964+0000 7f9189c365c0 4 datascan.main: Using metadata pool output 2023-11-08T16:41:49.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.964+0000 7f9189c365c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:41:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.965+0000 7f9189c365c0 1 Processor -- start 2023-11-08T16:41:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.965+0000 7f9189c365c0 1 -- start start 2023-11-08T16:41:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9189c365c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede60000 0x564eed0fb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9189c365c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091800 0x564eed0fc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9189c365c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eeddf6000 0x564eed0fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9189c365c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eed15f180 con 0x564eede60000 2023-11-08T16:41:49.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9189c365c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eed0d8b40 con 0x564eeddf6000 2023-11-08T16:41:49.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9189c365c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x564eed0d8a00 con 0x564eed091800 2023-11-08T16:41:49.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9187342640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede60000 0x564eed0fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9187342640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede60000 0x564eed0fb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48130/0 (socket says 172.21.15.81:48130) 2023-11-08T16:41:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9187342640 1 -- 172.21.15.81:0/1434884155 learned_addr learned my addr 172.21.15.81:0/1434884155 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9187342640 1 -- 172.21.15.81:0/1434884155 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091800 msgr2=0x564eed0fc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9187342640 1 --2- 172.21.15.81:0/1434884155 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091800 0x564eed0fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9187342640 1 -- 172.21.15.81:0/1434884155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eeddf6000 msgr2=0x564eed0fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:49.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9187342640 1 --2- 172.21.15.81:0/1434884155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eeddf6000 0x564eed0fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.966+0000 7f9187342640 1 -- 172.21.15.81:0/1434884155 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eed0b6680 con 0x564eede60000 2023-11-08T16:41:49.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.967+0000 7f9187342640 1 --2- 172.21.15.81:0/1434884155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede60000 0x564eed0fb080 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x564eedf686c0 tx=0x564eedf6acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4b134b9695f76 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.967+0000 7f918333a640 1 -- 172.21.15.81:0/1434884155 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564eed0d6f20 con 0x564eede60000 2023-11-08T16:41:49.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.967+0000 7f918333a640 1 -- 172.21.15.81:0/1434884155 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564eedf56000 con 0x564eede60000 2023-11-08T16:41:49.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.967+0000 7f918333a640 1 -- 172.21.15.81:0/1434884155 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564eed0d71e0 con 0x564eede60000 2023-11-08T16:41:49.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.967+0000 7f9189c365c0 1 -- 172.21.15.81:0/1434884155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede60000 msgr2=0x564eed0fb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.967+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1434884155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede60000 0x564eed0fb080 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x564eedf686c0 tx=0x564eedf6acc0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.968+0000 7f9189c365c0 1 -- 172.21.15.81:0/1434884155 shutdown_connections 2023-11-08T16:41:49.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.968+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1434884155 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eeddf6000 0x564eed0fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.968+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1434884155 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091800 0x564eed0fc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.968+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1434884155 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede60000 0x564eed0fb080 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.968+0000 7f9189c365c0 1 -- 172.21.15.81:0/1434884155 >> 172.21.15.81:0/1434884155 conn(0x564eede60400 msgr2=0x564eede1b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:49.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.968+0000 7f9189c365c0 1 -- 172.21.15.81:0/1434884155 shutdown_connections 2023-11-08T16:41:49.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.968+0000 7f9189c365c0 1 -- 172.21.15.81:0/1434884155 wait complete. 2023-11-08T16:41:49.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.968+0000 7f9189c365c0 1 Processor -- start 2023-11-08T16:41:49.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.969+0000 7f9189c365c0 1 -- start start 2023-11-08T16:41:49.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.969+0000 7f9189c365c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091800 0x564eed0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.969+0000 7f9189c365c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eeddf7400 0x564eed0fd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.969+0000 7f9189c365c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eeddf7000 0x564eed0fcc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.969+0000 7f9189c365c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eed15edc0 con 0x564eeddf7400 2023-11-08T16:41:49.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.969+0000 7f9189c365c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eed0d8a00 con 0x564eeddf7000 2023-11-08T16:41:49.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.969+0000 7f9189c365c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x564eed0d8b40 con 0x564eed091800 2023-11-08T16:41:49.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.970+0000 7f9187b43640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eeddf7400 0x564eed0fd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.970+0000 7f9187b43640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eeddf7400 0x564eed0fd180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:48132/0 (socket says 172.21.15.81:48132) 2023-11-08T16:41:49.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.970+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 learned_addr learned my addr 172.21.15.81:0/1589379186 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:49.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.970+0000 7f9187342640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091800 0x564eed0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.970+0000 7f9187342640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eeddf7000 0x564eed0fcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.970+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091800 msgr2=0x564eed0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.970+0000 7f9187b43640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091800 0x564eed0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.970+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eeddf7000 msgr2=0x564eed0fcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:49.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.970+0000 7f9187b43640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eeddf7000 0x564eed0fcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:49.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.970+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eed0b6820 con 0x564eeddf7400 2023-11-08T16:41:49.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.970+0000 7f9187b43640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eeddf7400 0x564eed0fd180 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x564eedf69950 tx=0x564eedf6ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.971+0000 7f9181b37640 1 -- 172.21.15.81:0/1589379186 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564eed0d6f20 con 0x564eeddf7400 2023-11-08T16:41:49.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.971+0000 7f9181b37640 1 -- 172.21.15.81:0/1589379186 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564eedf57b00 con 0x564eeddf7400 2023-11-08T16:41:49.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.971+0000 7f9181b37640 1 -- 172.21.15.81:0/1589379186 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x564eed0d71e0 con 0x564eeddf7400 2023-11-08T16:41:49.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.971+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x564eed0b64e0 con 0x564eeddf7400 2023-11-08T16:41:49.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.971+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564eed0b61a0 con 0x564eeddf7400 2023-11-08T16:41:49.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.971+0000 7f9189c365c0 4 datascan.init: resolving metadata pool 16 2023-11-08T16:41:49.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.973+0000 7f9181b37640 1 -- 172.21.15.81:0/1589379186 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x564eed0ecb00 con 0x564eeddf7400 2023-11-08T16:41:49.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.973+0000 7f9181b37640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x564eedfcac00 0x564eed3e6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.973+0000 7f9181b37640 1 -- 172.21.15.81:0/1589379186 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x564eed150a80 con 0x564eeddf7400 2023-11-08T16:41:49.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.973+0000 7f9189c365c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:41:49.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.973+0000 7f9189c365c0 4 datascan.main: data pool 'cephfs_data' has ID 17 2023-11-08T16:41:49.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.973+0000 7f9189c365c0 4 datascan.main: opening data pool 'cephfs_data' 2023-11-08T16:41:49.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.973+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x564eeddf6c00 0x564eed0fb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.973+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eeddf6c00 2023-11-08T16:41:49.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.974+0000 7f9187342640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x564eedfcac00 0x564eed3e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.974+0000 7f9187342640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x564eeddf6c00 0x564eed0fb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.974+0000 7f9187342640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x564eedfcac00 0x564eed3e6000 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x564eedf68810 tx=0x564eedf6bf80 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.974+0000 7f9187342640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x564eeddf6c00 0x564eed0fb080 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.975+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv18 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:49.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.975+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x564eeddf6800 0x564eed0fc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.975+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eeddf6800 2023-11-08T16:41:49.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.976+0000 7f9188344640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x564eeddf6800 0x564eed0fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.976+0000 7f9188344640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x564eeddf6800 0x564eed0fc680 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.977+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:49.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.977+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x564eeddf6400 0x564eed0fc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:49.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.977+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 131 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6400 2023-11-08T16:41:49.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.978+0000 7f9187b43640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x564eeddf6400 0x564eed0fc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:49.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.979+0000 7f9187b43640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x564eeddf6400 0x564eed0fc100 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:49.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.980+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 131 out=81b] v131'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:49.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.980+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:4 17.0 17:00000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6400 2023-11-08T16:41:49.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.980+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 131 out=81b] v131'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:49.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.981+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000206.00000000 2023-11-08T16:41:49.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.981+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000206.0 2023-11-08T16:41:49.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.981+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:5 17.0 17:012cdbc7:::10000000206.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6400 2023-11-08T16:41:49.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.982+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 3 ==== osd_op_reply(5 10000000206.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:49.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.982+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000206 2023-11-08T16:41:49.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.982+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/17 2023-11-08T16:41:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.982+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=17 2023-11-08T16:41:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.982+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x564eede60000 0x564eed3e6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.982+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:6 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eede60000 2023-11-08T16:41:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.982+0000 7f9187342640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x564eede60000 0x564eed3e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.983+0000 7f9187342640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x564eede60000 0x564eed3e6580 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:50.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.983+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(6 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.983+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.983+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:7 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6800 2023-11-08T16:41:50.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.984+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(7 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.984+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.984+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.984+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.984+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:8 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eeddf6800 2023-11-08T16:41:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.985+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(8 10000000000.01800000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.985+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:41:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.985+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:9 16.1b 16:dcc776e0:::10000000000.01800000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6800 2023-11-08T16:41:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.987+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(9 10000000000.01800000 [create,omap-set-header] v131'6 uv6 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.987+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-11-08T16:41:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.987+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:10 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405500 con 0x564eeddf6800 2023-11-08T16:41:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.988+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(10 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.988+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=17 frag=1*, r=0 2023-11-08T16:41:50.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.988+0000 7f9189c365c0 20 datascan.read_dentry: 17_head not found in result 2023-11-08T16:41:50.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.988+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000206 at 0x0x10000000000/17 with size=2 bytes 2023-11-08T16:41:50.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.988+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:11 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6800 2023-11-08T16:41:50.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.991+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(11 10000000000.01800000 [omap-set-vals] v131'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.991+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-11-08T16:41:50.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.991+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-11-08T16:41:50.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.991+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-11-08T16:41:50.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.991+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:12 16.1f 16:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eeddf6800 2023-11-08T16:41:50.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.992+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(12 1.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+30 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.992+0000 7f9189c365c0 4 datascan.get_frag_of: No backtrace on '1.00000000': cannot determine fragtree 2023-11-08T16:41:50.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.992+0000 7f9189c365c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-11-08T16:41:50.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.992+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:13 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6800 2023-11-08T16:41:50.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.993+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(13 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.993+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 6 2023-11-08T16:41:50.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.993+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x1 * 2023-11-08T16:41:50.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.993+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:14 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eeddf6800 2023-11-08T16:41:50.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.995+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 9 ==== osd_op_reply(14 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.995+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.995+0000 7f9189c365c0 20 datascan.inject_with_backtrace: Dentry 0x0x1/subdir already exists and points to me 2023-11-08T16:41:50.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.995+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x564eedfcb000 0x564eed3e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:50.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.995+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:15 17.20 17:04000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eedfcb000 2023-11-08T16:41:50.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.995+0000 7f9188344640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x564eedfcb000 0x564eed3e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:50.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.996+0000 7f9188344640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x564eedfcb000 0x564eed3e6b00 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:50.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.996+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(15 [pgnls-filter start_epoch 131 out=81b] v131'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.996+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000020b.00000000 2023-11-08T16:41:50.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.996+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000020b.0 2023-11-08T16:41:50.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.996+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:16 17.20 17:05552577:::1000000020b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eedfcb000 2023-11-08T16:41:50.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.997+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(16 1000000020b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.997+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020b 2023-11-08T16:41:50.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.997+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/22 2023-11-08T16:41:50.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.997+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=22 2023-11-08T16:41:50.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.997+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:17 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405500 con 0x564eede60000 2023-11-08T16:41:50.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.997+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(17 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.997+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.997+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:18 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6800 2023-11-08T16:41:50.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.998+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 10 ==== osd_op_reply(18 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.998+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.998+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.998+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.998+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:19 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eeddf6800 2023-11-08T16:41:50.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.999+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 11 ==== osd_op_reply(19 10000000000.01800000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.999+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 7 2023-11-08T16:41:50.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.999+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:49.999+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:20 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eeddf6800 2023-11-08T16:41:50.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.000+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 12 ==== osd_op_reply(20 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.000+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=22 frag=1*, r=0 2023-11-08T16:41:50.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.000+0000 7f9189c365c0 20 datascan.read_dentry: 22_head not found in result 2023-11-08T16:41:50.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.000+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020b at 0x0x10000000000/22 with size=2 bytes 2023-11-08T16:41:50.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.000+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:21 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6800 2023-11-08T16:41:50.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.003+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 13 ==== osd_op_reply(21 10000000000.01800000 [omap-set-vals] v131'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.003+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-11-08T16:41:50.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.003+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x564eedfcb800 0x564eed3e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:50.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.003+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:22 17.10 17:08000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eedfcb800 2023-11-08T16:41:50.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.004+0000 7f9187b43640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x564eedfcb800 0x564eed3e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:50.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.004+0000 7f9187b43640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x564eedfcb800 0x564eed3e7080 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:50.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.005+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(22 [pgnls-filter start_epoch 131 out=101b] v131'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.005+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000022a.00000000 2023-11-08T16:41:50.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.005+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000022a.0 2023-11-08T16:41:50.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.005+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:23 17.10 17:083d5826:::1000000022a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eedfcb800 2023-11-08T16:41:50.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.005+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(23 1000000022a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.005+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022a 2023-11-08T16:41:50.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.005+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/53 2023-11-08T16:41:50.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.005+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=53 2023-11-08T16:41:50.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.005+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:24 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eede60000 2023-11-08T16:41:50.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.006+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(24 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.006+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.006+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:25 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eeddf6800 2023-11-08T16:41:50.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.007+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 14 ==== osd_op_reply(25 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.007+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.007+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.007+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.007+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:26 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eeddf6c00 2023-11-08T16:41:50.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.007+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(26 10000000000.01000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.007+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 0 2023-11-08T16:41:50.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.007+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:27 16.5 16:a40620de:::10000000000.01000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405500 con 0x564eeddf6c00 2023-11-08T16:41:50.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.011+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(27 10000000000.01000000 [create,omap-set-header] v131'5 uv5 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.011+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-11-08T16:41:50.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.011+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:28 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eeddf6c00 2023-11-08T16:41:50.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.011+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(28 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.011+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=53 frag=0*, r=0 2023-11-08T16:41:50.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.011+0000 7f9189c365c0 20 datascan.read_dentry: 53_head not found in result 2023-11-08T16:41:50.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.011+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022a at 0x0x10000000000/53 with size=2 bytes 2023-11-08T16:41:50.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.011+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:29 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6c00 2023-11-08T16:41:50.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.014+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(29 10000000000.01000000 [omap-set-vals] v131'6 uv6 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.014+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/53 pointing to 0x0x1000000022a 2023-11-08T16:41:50.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.014+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-11-08T16:41:50.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.014+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-11-08T16:41:50.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.014+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:30 16.1f 16:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6800 2023-11-08T16:41:50.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.015+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 15 ==== osd_op_reply(30 1.00000000 [getxattr (30) out=30b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+30 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.015+0000 7f9189c365c0 4 datascan.get_frag_of: No backtrace on '1.00000000': cannot determine fragtree 2023-11-08T16:41:50.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.015+0000 7f9189c365c0 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-11-08T16:41:50.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.015+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:31 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eeddf6800 2023-11-08T16:41:50.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.016+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 16 ==== osd_op_reply(31 1.00000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.016+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 6 2023-11-08T16:41:50.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.016+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x1 * 2023-11-08T16:41:50.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.016+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:32 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eeddf6800 2023-11-08T16:41:50.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.017+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 17 ==== osd_op_reply(32 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.017+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.017+0000 7f9189c365c0 20 datascan.inject_with_backtrace: Dentry 0x0x1/subdir already exists and points to me 2023-11-08T16:41:50.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.017+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:33 17.10 17:08500202::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420000 con 0x564eedfcb800 2023-11-08T16:41:50.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.018+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 131 out=101b] v131'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.018+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000023e.00000000 2023-11-08T16:41:50.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.018+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000023e.0 2023-11-08T16:41:50.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.018+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:34 17.10 17:08500202:::1000000023e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420380 con 0x564eedfcb800 2023-11-08T16:41:50.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.019+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(34 1000000023e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.019+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023e 2023-11-08T16:41:50.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.019+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/73 2023-11-08T16:41:50.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.019+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=73 2023-11-08T16:41:50.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.019+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420a80 con 0x564eede60000 2023-11-08T16:41:50.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.020+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(35 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.020+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.020+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:36 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420e00 con 0x564eeddf6800 2023-11-08T16:41:50.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.021+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 18 ==== osd_op_reply(36 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.021+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.021+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.021+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.021+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:37 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421180 con 0x564eeddf6c00 2023-11-08T16:41:50.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.021+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(37 10000000000.01000000 [omap-get-header out=274b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.021+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 6 2023-11-08T16:41:50.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.021+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.022+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:38 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421500 con 0x564eeddf6c00 2023-11-08T16:41:50.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.022+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(38 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv6 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.022+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=73 frag=0*, r=0 2023-11-08T16:41:50.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.022+0000 7f9189c365c0 20 datascan.read_dentry: 73_head not found in result 2023-11-08T16:41:50.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.022+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023e at 0x0x10000000000/73 with size=2 bytes 2023-11-08T16:41:50.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.022+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:39 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421880 con 0x564eeddf6c00 2023-11-08T16:41:50.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.025+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(39 10000000000.01000000 [omap-set-vals] v131'7 uv7 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.025+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/73 pointing to 0x0x1000000023e 2023-11-08T16:41:50.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.025+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:40 17.10 17:08b99fea::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421c00 con 0x564eedfcb800 2023-11-08T16:41:50.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.026+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(40 [pgnls-filter start_epoch 131 out=101b] v131'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.026+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 100000001fd.00000000 2023-11-08T16:41:50.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.026+0000 7f9189c365c0 10 datascan.operator(): handling object 100000001fd.0 2023-11-08T16:41:50.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.026+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:41 17.10 17:08b99fea:::100000001fd.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420a80 con 0x564eedfcb800 2023-11-08T16:41:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.027+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(41 100000001fd.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.027+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fd 2023-11-08T16:41:50.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.027+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/8 2023-11-08T16:41:50.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.027+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=8 2023-11-08T16:41:50.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.027+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:42 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420700 con 0x564eede60000 2023-11-08T16:41:50.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.028+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 5 ==== osd_op_reply(42 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.028+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.028+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:43 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405500 con 0x564eeddf6800 2023-11-08T16:41:50.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.029+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 19 ==== osd_op_reply(43 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.029+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.029+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.029+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.029+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:44 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eeddf6c00 2023-11-08T16:41:50.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.030+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(44 10000000000.01000000 [omap-get-header out=274b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.030+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 7 2023-11-08T16:41:50.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.030+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.030+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:45 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6c00 2023-11-08T16:41:50.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.030+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(45 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv7 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.030+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=8 frag=0*, r=0 2023-11-08T16:41:50.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.030+0000 7f9189c365c0 20 datascan.read_dentry: 8_head not found in result 2023-11-08T16:41:50.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.030+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fd at 0x0x10000000000/8 with size=1 bytes 2023-11-08T16:41:50.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.031+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:46 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6c00 2023-11-08T16:41:50.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.034+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 11 ==== osd_op_reply(46 10000000000.01000000 [omap-set-vals] v131'8 uv8 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.034+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-11-08T16:41:50.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.034+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:47 17.10 17:08e6f8ca::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eedfcb800 2023-11-08T16:41:50.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.035+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(47 [pgnls-filter start_epoch 131 out=81b] v131'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.035+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000257.00000000 2023-11-08T16:41:50.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.035+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000257.0 2023-11-08T16:41:50.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.035+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:48 17.10 17:08e6f8ca:::10000000257.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eedfcb800 2023-11-08T16:41:50.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.036+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(48 10000000257.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.036+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000257 2023-11-08T16:41:50.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.036+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/98 2023-11-08T16:41:50.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.036+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=98 2023-11-08T16:41:50.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.036+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:49 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eede60000 2023-11-08T16:41:50.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.037+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 6 ==== osd_op_reply(49 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.037+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.037+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:50 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6800 2023-11-08T16:41:50.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.038+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 20 ==== osd_op_reply(50 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.038+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.038+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.038+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.038+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:51 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6800 2023-11-08T16:41:50.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.040+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 21 ==== osd_op_reply(51 10000000000.01800000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.040+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 8 2023-11-08T16:41:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.040+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.040+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:52 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eeddf6800 2023-11-08T16:41:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.041+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 22 ==== osd_op_reply(52 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.041+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=98 frag=1*, r=0 2023-11-08T16:41:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.041+0000 7f9189c365c0 20 datascan.read_dentry: 98_head not found in result 2023-11-08T16:41:50.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.041+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000257 at 0x0x10000000000/98 with size=2 bytes 2023-11-08T16:41:50.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.041+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:53 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405500 con 0x564eeddf6800 2023-11-08T16:41:50.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.044+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 23 ==== osd_op_reply(53 10000000000.01800000 [omap-set-vals] v131'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.045+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/98 pointing to 0x0x10000000257 2023-11-08T16:41:50.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.045+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:54 17.30 17:0c000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eeddf6c00 2023-11-08T16:41:50.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.045+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 12 ==== osd_op_reply(54 [pgnls-filter start_epoch 131 out=101b] v131'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.046+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000021d.00000000 2023-11-08T16:41:50.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.046+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000021d.0 2023-11-08T16:41:50.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.046+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:55 17.30 17:0c00cb85:::1000000021d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eeddf6c00 2023-11-08T16:41:50.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.047+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 13 ==== osd_op_reply(55 1000000021d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.047+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021d 2023-11-08T16:41:50.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.047+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/40 2023-11-08T16:41:50.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.047+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=40 2023-11-08T16:41:50.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.047+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:56 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420380 con 0x564eede60000 2023-11-08T16:41:50.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.048+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 7 ==== osd_op_reply(56 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.048+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.048+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:57 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420700 con 0x564eeddf6800 2023-11-08T16:41:50.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.049+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 24 ==== osd_op_reply(57 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.049+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.049+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.049+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.049+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:58 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420a80 con 0x564eeddf6800 2023-11-08T16:41:50.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.050+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 25 ==== osd_op_reply(58 10000000000.01800000 [omap-get-header out=274b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.050+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 9 2023-11-08T16:41:50.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.050+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.050+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:59 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420e00 con 0x564eeddf6800 2023-11-08T16:41:50.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.051+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 26 ==== osd_op_reply(59 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.051+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=40 frag=1*, r=0 2023-11-08T16:41:50.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.051+0000 7f9189c365c0 20 datascan.read_dentry: 40_head not found in result 2023-11-08T16:41:50.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.051+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021d at 0x0x10000000000/40 with size=2 bytes 2023-11-08T16:41:50.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.051+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:60 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421180 con 0x564eeddf6800 2023-11-08T16:41:50.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.056+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 27 ==== osd_op_reply(60 10000000000.01800000 [omap-set-vals] v131'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.056+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/40 pointing to 0x0x1000000021d 2023-11-08T16:41:50.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.056+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:61 17.30 17:0c28cce1::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421500 con 0x564eeddf6c00 2023-11-08T16:41:50.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.057+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 14 ==== osd_op_reply(61 [pgnls-filter start_epoch 131 out=101b] v131'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.057+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000244.00000000 2023-11-08T16:41:50.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.057+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000244.0 2023-11-08T16:41:50.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.057+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:62 17.30 17:0c28cce1:::10000000244.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421880 con 0x564eeddf6c00 2023-11-08T16:41:50.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.058+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 15 ==== osd_op_reply(62 10000000244.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.058+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000244 2023-11-08T16:41:50.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.058+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/79 2023-11-08T16:41:50.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.058+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=79 2023-11-08T16:41:50.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.058+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:63 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420000 con 0x564eede60000 2023-11-08T16:41:50.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.059+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 8 ==== osd_op_reply(63 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.059+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.059+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:64 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420380 con 0x564eeddf6800 2023-11-08T16:41:50.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.060+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 28 ==== osd_op_reply(64 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.060+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.060+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.060+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.060+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:65 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421500 con 0x564eeddf6800 2023-11-08T16:41:50.069 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.061+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 29 ==== osd_op_reply(65 10000000000.01800000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.061+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 10 2023-11-08T16:41:50.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.061+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.061+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:66 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eeddf6800 2023-11-08T16:41:50.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.062+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 30 ==== osd_op_reply(66 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.062+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=79 frag=1*, r=0 2023-11-08T16:41:50.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.062+0000 7f9189c365c0 20 datascan.read_dentry: 79_head not found in result 2023-11-08T16:41:50.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.062+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000244 at 0x0x10000000000/79 with size=2 bytes 2023-11-08T16:41:50.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.062+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:67 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eeddf6800 2023-11-08T16:41:50.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.066+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 31 ==== osd_op_reply(67 10000000000.01800000 [omap-set-vals] v131'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.066+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/79 pointing to 0x0x10000000244 2023-11-08T16:41:50.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.066+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:68 17.30 17:0cc9e906::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eeddf6c00 2023-11-08T16:41:50.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.067+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 16 ==== osd_op_reply(68 [pgnls-filter start_epoch 131 out=101b] v131'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.067+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000227.00000000 2023-11-08T16:41:50.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.067+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000227.0 2023-11-08T16:41:50.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.067+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:69 17.30 17:0cc9e906:::10000000227.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6c00 2023-11-08T16:41:50.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.068+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 17 ==== osd_op_reply(69 10000000227.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.068+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000227 2023-11-08T16:41:50.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.068+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/50 2023-11-08T16:41:50.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.068+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=50 2023-11-08T16:41:50.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.069+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:70 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420700 con 0x564eede60000 2023-11-08T16:41:50.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.069+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 9 ==== osd_op_reply(70 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.069+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.070+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:71 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420a80 con 0x564eeddf6800 2023-11-08T16:41:50.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.071+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 32 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.071+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.071+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.071+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.071+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:72 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420e00 con 0x564eeddf6c00 2023-11-08T16:41:50.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.071+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 18 ==== osd_op_reply(72 10000000000.01000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.072+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 8 2023-11-08T16:41:50.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.072+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.072+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:73 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421180 con 0x564eeddf6c00 2023-11-08T16:41:50.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.072+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 19 ==== osd_op_reply(73 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.072+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=50 frag=0*, r=0 2023-11-08T16:41:50.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.072+0000 7f9189c365c0 20 datascan.read_dentry: 50_head not found in result 2023-11-08T16:41:50.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.072+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000227 at 0x0x10000000000/50 with size=2 bytes 2023-11-08T16:41:50.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.072+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:74 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420380 con 0x564eeddf6c00 2023-11-08T16:41:50.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.076+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 20 ==== osd_op_reply(74 10000000000.01000000 [omap-set-vals] v131'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.076+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/50 pointing to 0x0x10000000227 2023-11-08T16:41:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.076+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:75 17.30 17:0e3d5a27::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6c00 2023-11-08T16:41:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.076+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 21 ==== osd_op_reply(75 [pgnls-filter start_epoch 131 out=81b] v131'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.077+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000250.00000000 2023-11-08T16:41:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.077+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000250.0 2023-11-08T16:41:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.077+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:76 17.30 17:0e3d5a27:::10000000250.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6c00 2023-11-08T16:41:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.078+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 22 ==== osd_op_reply(76 10000000250.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.078+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000250 2023-11-08T16:41:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.078+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/91 2023-11-08T16:41:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.078+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=91 2023-11-08T16:41:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.078+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:77 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405500 con 0x564eede60000 2023-11-08T16:41:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.078+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 10 ==== osd_op_reply(77 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.078+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.079+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:78 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421c00 con 0x564eeddf6800 2023-11-08T16:41:50.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.080+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 33 ==== osd_op_reply(78 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.080+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.080+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.080+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.080+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:79 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421880 con 0x564eeddf6800 2023-11-08T16:41:50.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.081+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 34 ==== osd_op_reply(79 10000000000.01800000 [omap-get-header out=274b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.081+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 11 2023-11-08T16:41:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.081+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.081+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:80 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420000 con 0x564eeddf6800 2023-11-08T16:41:50.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.083+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 35 ==== osd_op_reply(80 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.083+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=91 frag=1*, r=0 2023-11-08T16:41:50.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.083+0000 7f9189c365c0 20 datascan.read_dentry: 91_head not found in result 2023-11-08T16:41:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.083+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000250 at 0x0x10000000000/91 with size=2 bytes 2023-11-08T16:41:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.083+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:81 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420700 con 0x564eeddf6800 2023-11-08T16:41:50.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.087+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 36 ==== osd_op_reply(81 10000000000.01800000 [omap-set-vals] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.087+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/91 pointing to 0x0x10000000250 2023-11-08T16:41:50.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.087+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:82 17.8 17:10000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420e00 con 0x564eeddf6400 2023-11-08T16:41:50.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.088+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 4 ==== osd_op_reply(82 [pgnls-filter start_epoch 131 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.088+0000 7f918333a640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:83 17.28 17:14000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed436000 con 0x564eedfcb800 2023-11-08T16:41:50.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.089+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 131 out=101b] v131'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.089+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000233.00000000 2023-11-08T16:41:50.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.089+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000233.0 2023-11-08T16:41:50.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.089+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:84 17.28 17:14ce3b86:::10000000233.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421180 con 0x564eedfcb800 2023-11-08T16:41:50.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.090+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(84 10000000233.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.090+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000233 2023-11-08T16:41:50.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.090+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/62 2023-11-08T16:41:50.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.090+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=62 2023-11-08T16:41:50.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.090+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:85 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eede60000 2023-11-08T16:41:50.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.091+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 11 ==== osd_op_reply(85 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.091+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.091+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:86 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6800 2023-11-08T16:41:50.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.092+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 37 ==== osd_op_reply(86 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.092+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.092+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.092+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.092+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:87 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2000 con 0x564eeddf6800 2023-11-08T16:41:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.094+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 38 ==== osd_op_reply(87 10000000000.01800000 [omap-get-header out=274b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.094+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 12 2023-11-08T16:41:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.094+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.094+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:88 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eeddf6800 2023-11-08T16:41:50.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.095+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 39 ==== osd_op_reply(88 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.095+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=62 frag=1*, r=0 2023-11-08T16:41:50.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.095+0000 7f9189c365c0 20 datascan.read_dentry: 62_head not found in result 2023-11-08T16:41:50.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.095+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000233 at 0x0x10000000000/62 with size=2 bytes 2023-11-08T16:41:50.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.095+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:89 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2700 con 0x564eeddf6800 2023-11-08T16:41:50.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.100+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 40 ==== osd_op_reply(89 10000000000.01800000 [omap-set-vals] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.100+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/62 pointing to 0x0x10000000233 2023-11-08T16:41:50.101 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.100+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:90 17.28 17:15246602::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2a80 con 0x564eedfcb800 2023-11-08T16:41:50.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.101+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 11 ==== osd_op_reply(90 [pgnls-filter start_epoch 131 out=101b] v131'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.102 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.101+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000202.00000000 2023-11-08T16:41:50.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.101+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000202.0 2023-11-08T16:41:50.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.101+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:91 17.28 17:15246602:::10000000202.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2e00 con 0x564eedfcb800 2023-11-08T16:41:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.102+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 12 ==== osd_op_reply(91 10000000202.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.102+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000202 2023-11-08T16:41:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.102+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/13 2023-11-08T16:41:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.102+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=13 2023-11-08T16:41:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.102+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:92 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eede60000 2023-11-08T16:41:50.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.103+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 12 ==== osd_op_reply(92 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.103+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.103+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:93 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3880 con 0x564eeddf6800 2023-11-08T16:41:50.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.104+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 41 ==== osd_op_reply(93 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.104+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.104+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.104+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.104+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:94 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3c00 con 0x564eeddf6800 2023-11-08T16:41:50.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.105+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 42 ==== osd_op_reply(94 10000000000.01800000 [omap-get-header out=274b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.105+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 13 2023-11-08T16:41:50.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.105+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.105+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:95 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2a80 con 0x564eeddf6800 2023-11-08T16:41:50.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.106+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 43 ==== osd_op_reply(95 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.106+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=13 frag=1*, r=0 2023-11-08T16:41:50.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.106+0000 7f9189c365c0 20 datascan.read_dentry: 13_head not found in result 2023-11-08T16:41:50.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.106+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000202 at 0x0x10000000000/13 with size=2 bytes 2023-11-08T16:41:50.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.106+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:96 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6800 2023-11-08T16:41:50.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.109+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 44 ==== osd_op_reply(96 10000000000.01800000 [omap-set-vals] v131'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.109+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-11-08T16:41:50.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.109+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:97 17.28 17:1537fa38::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2e00 con 0x564eedfcb800 2023-11-08T16:41:50.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.110+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 13 ==== osd_op_reply(97 [pgnls-filter start_epoch 131 out=101b] v131'18 uv18 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.110+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000236.00000000 2023-11-08T16:41:50.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.110+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000236.0 2023-11-08T16:41:50.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.110+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:98 17.28 17:1537fa38:::10000000236.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2000 con 0x564eedfcb800 2023-11-08T16:41:50.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.111+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 14 ==== osd_op_reply(98 10000000236.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.111+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000236 2023-11-08T16:41:50.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.111+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/65 2023-11-08T16:41:50.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.111+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=65 2023-11-08T16:41:50.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.111+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:99 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eede60000 2023-11-08T16:41:50.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.112+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 13 ==== osd_op_reply(99 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.112+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.112+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:100 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eeddf6800 2023-11-08T16:41:50.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.113+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 45 ==== osd_op_reply(100 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.113+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.113+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.113+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.113+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:101 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6c00 2023-11-08T16:41:50.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.114+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 23 ==== osd_op_reply(101 10000000000.01000000 [omap-get-header out=274b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.114+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 9 2023-11-08T16:41:50.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.114+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.114+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:102 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420000 con 0x564eeddf6c00 2023-11-08T16:41:50.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.115+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 24 ==== osd_op_reply(102 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv9 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.115+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=65 frag=0*, r=0 2023-11-08T16:41:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.115+0000 7f9189c365c0 20 datascan.read_dentry: 65_head not found in result 2023-11-08T16:41:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.115+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000236 at 0x0x10000000000/65 with size=2 bytes 2023-11-08T16:41:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.115+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:103 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420700 con 0x564eeddf6c00 2023-11-08T16:41:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.119+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 25 ==== osd_op_reply(103 10000000000.01000000 [omap-set-vals] v131'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.119+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/65 pointing to 0x0x10000000236 2023-11-08T16:41:50.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.119+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:104 17.28 17:177a762d::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eedfcb800 2023-11-08T16:41:50.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.120+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 15 ==== osd_op_reply(104 [pgnls-filter start_epoch 131 out=81b] v131'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.120+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000201.00000000 2023-11-08T16:41:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.120+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000201.0 2023-11-08T16:41:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.120+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:105 17.28 17:177a762d:::10000000201.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eedfcb800 2023-11-08T16:41:50.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.121+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 16 ==== osd_op_reply(105 10000000201.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.121+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000201 2023-11-08T16:41:50.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.121+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/12 2023-11-08T16:41:50.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.121+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=12 2023-11-08T16:41:50.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.121+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:106 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405500 con 0x564eede60000 2023-11-08T16:41:50.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.121+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 14 ==== osd_op_reply(106 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.122+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.122+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:107 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6800 2023-11-08T16:41:50.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.123+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 46 ==== osd_op_reply(107 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.123+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.123+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.123+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.123+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:108 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6c00 2023-11-08T16:41:50.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.124+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 26 ==== osd_op_reply(108 10000000000.01000000 [omap-get-header out=274b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.129 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.124+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 10 2023-11-08T16:41:50.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.124+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.124+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:109 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6c00 2023-11-08T16:41:50.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.125+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 27 ==== osd_op_reply(109 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.130 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.125+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=12 frag=0*, r=0 2023-11-08T16:41:50.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.125+0000 7f9189c365c0 20 datascan.read_dentry: 12_head not found in result 2023-11-08T16:41:50.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.125+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000201 at 0x0x10000000000/12 with size=2 bytes 2023-11-08T16:41:50.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.125+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:110 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eeddf6c00 2023-11-08T16:41:50.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.128+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 28 ==== osd_op_reply(110 10000000000.01000000 [omap-set-vals] v131'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.128+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-11-08T16:41:50.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.128+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:111 17.18 17:18000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2e00 con 0x564eede60000 2023-11-08T16:41:50.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.129+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 15 ==== osd_op_reply(111 [pgnls-filter start_epoch 131 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.129+0000 7f9182b39640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:112 17.38 17:1c000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eeddf6400 2023-11-08T16:41:50.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.130+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 5 ==== osd_op_reply(112 [pgnls-filter start_epoch 131 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.133 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.131+0000 7f918333a640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:113 17.4 17:20000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eeddf6c00 2023-11-08T16:41:50.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.132+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 29 ==== osd_op_reply(113 [pgnls-filter start_epoch 131 out=81b] v131'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.132+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000220.00000000 2023-11-08T16:41:50.134 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.132+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000220.0 2023-11-08T16:41:50.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.132+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:114 17.4 17:2231e764:::10000000220.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420380 con 0x564eeddf6c00 2023-11-08T16:41:50.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.133+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 30 ==== osd_op_reply(114 10000000220.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.133+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000220 2023-11-08T16:41:50.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.133+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/43 2023-11-08T16:41:50.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.133+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=43 2023-11-08T16:41:50.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.133+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:115 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421880 con 0x564eede60000 2023-11-08T16:41:50.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.134+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 16 ==== osd_op_reply(115 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.134+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.134+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:116 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421c00 con 0x564eeddf6800 2023-11-08T16:41:50.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.136+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 47 ==== osd_op_reply(116 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.136+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.136+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.136+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.136+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:117 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420a80 con 0x564eeddf6c00 2023-11-08T16:41:50.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.136+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 31 ==== osd_op_reply(117 10000000000.01000000 [omap-get-header out=274b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.137+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 11 2023-11-08T16:41:50.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.137+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.137+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:118 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421500 con 0x564eeddf6c00 2023-11-08T16:41:50.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.137+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 32 ==== osd_op_reply(118 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.137+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=43 frag=0*, r=0 2023-11-08T16:41:50.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.137+0000 7f9189c365c0 20 datascan.read_dentry: 43_head not found in result 2023-11-08T16:41:50.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.137+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000220 at 0x0x10000000000/43 with size=2 bytes 2023-11-08T16:41:50.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.137+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:119 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420e00 con 0x564eeddf6c00 2023-11-08T16:41:50.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.141+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 33 ==== osd_op_reply(119 10000000000.01000000 [omap-set-vals] v131'12 uv12 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.141+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/43 pointing to 0x0x10000000220 2023-11-08T16:41:50.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.141+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:120 17.24 17:24000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421180 con 0x564eedfcb800 2023-11-08T16:41:50.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.142+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 17 ==== osd_op_reply(120 [pgnls-filter start_epoch 131 out=101b] v131'10 uv10 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.142+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000234.00000000 2023-11-08T16:41:50.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.142+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000234.0 2023-11-08T16:41:50.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.142+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:121 17.24 17:242ba651:::10000000234.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eedfcb800 2023-11-08T16:41:50.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.143+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 18 ==== osd_op_reply(121 10000000234.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.143+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000234 2023-11-08T16:41:50.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.143+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/63 2023-11-08T16:41:50.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.143+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=63 2023-11-08T16:41:50.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.143+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:122 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eede60000 2023-11-08T16:41:50.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.144+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 17 ==== osd_op_reply(122 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.144+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.144+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:123 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4700 con 0x564eeddf6800 2023-11-08T16:41:50.147 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.145+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 48 ==== osd_op_reply(123 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.145+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.145+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.145+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.145+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:124 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4a80 con 0x564eeddf6c00 2023-11-08T16:41:50.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.146+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 34 ==== osd_op_reply(124 10000000000.01000000 [omap-get-header out=274b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.148 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.146+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 12 2023-11-08T16:41:50.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.146+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.146+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:125 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eeddf6c00 2023-11-08T16:41:50.149 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.147+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 35 ==== osd_op_reply(125 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.147+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=63 frag=0*, r=0 2023-11-08T16:41:50.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.147+0000 7f9189c365c0 20 datascan.read_dentry: 63_head not found in result 2023-11-08T16:41:50.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.147+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000234 at 0x0x10000000000/63 with size=2 bytes 2023-11-08T16:41:50.150 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.147+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:126 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eeddf6c00 2023-11-08T16:41:50.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.150+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 36 ==== osd_op_reply(126 10000000000.01000000 [omap-set-vals] v131'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.150+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/63 pointing to 0x0x10000000234 2023-11-08T16:41:50.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.150+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:127 17.24 17:25b5fce8::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5500 con 0x564eedfcb800 2023-11-08T16:41:50.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.151+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 19 ==== osd_op_reply(127 [pgnls-filter start_epoch 131 out=81b] v131'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.151+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000231.00000000 2023-11-08T16:41:50.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.151+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000231.0 2023-11-08T16:41:50.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.151+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:128 17.24 17:25b5fce8:::10000000231.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5880 con 0x564eedfcb800 2023-11-08T16:41:50.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.152+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 20 ==== osd_op_reply(128 10000000231.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.152+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000231 2023-11-08T16:41:50.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.152+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/60 2023-11-08T16:41:50.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.152+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=60 2023-11-08T16:41:50.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.152+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:129 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420700 con 0x564eede60000 2023-11-08T16:41:50.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.153+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 18 ==== osd_op_reply(129 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.153+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.153+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:130 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420000 con 0x564eeddf6800 2023-11-08T16:41:50.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.154+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 49 ==== osd_op_reply(130 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.154+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.154+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.154+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.154+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:131 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420380 con 0x564eeddf6800 2023-11-08T16:41:50.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.155+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 50 ==== osd_op_reply(131 10000000000.01800000 [omap-get-header out=274b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.155+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 14 2023-11-08T16:41:50.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.155+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.155+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:132 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed421880 con 0x564eeddf6800 2023-11-08T16:41:50.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.156+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 51 ==== osd_op_reply(132 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.156+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=60 frag=1*, r=0 2023-11-08T16:41:50.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.156+0000 7f9189c365c0 20 datascan.read_dentry: 60_head not found in result 2023-11-08T16:41:50.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.156+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000231 at 0x0x10000000000/60 with size=2 bytes 2023-11-08T16:41:50.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.156+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:133 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed420a80 con 0x564eeddf6800 2023-11-08T16:41:50.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.160+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 52 ==== osd_op_reply(133 10000000000.01800000 [omap-set-vals] v131'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.160+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/60 pointing to 0x0x10000000231 2023-11-08T16:41:50.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.160+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:134 17.14 17:28000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eedfcb800 2023-11-08T16:41:50.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.161+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 21 ==== osd_op_reply(134 [pgnls-filter start_epoch 131 out=101b] v131'20 uv20 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.161+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000020e.00000000 2023-11-08T16:41:50.162 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.161+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000020e.0 2023-11-08T16:41:50.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.161+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:135 17.14 17:29971f97:::1000000020e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2e00 con 0x564eedfcb800 2023-11-08T16:41:50.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.162+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 22 ==== osd_op_reply(135 1000000020e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.162+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020e 2023-11-08T16:41:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.162+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/25 2023-11-08T16:41:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.162+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=25 2023-11-08T16:41:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.162+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:136 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eede60000 2023-11-08T16:41:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.163+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 19 ==== osd_op_reply(136 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.163+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.163+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:137 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6800 2023-11-08T16:41:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.164+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 53 ==== osd_op_reply(137 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.164+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.164+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.164+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.164+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:138 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5880 con 0x564eeddf6800 2023-11-08T16:41:50.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.166+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 54 ==== osd_op_reply(138 10000000000.01800000 [omap-get-header out=274b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.166+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 15 2023-11-08T16:41:50.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.166+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.166+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:139 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5c00 con 0x564eeddf6800 2023-11-08T16:41:50.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.167+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 55 ==== osd_op_reply(139 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.167+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=25 frag=1*, r=0 2023-11-08T16:41:50.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.167+0000 7f9189c365c0 20 datascan.read_dentry: 25_head not found in result 2023-11-08T16:41:50.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.167+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020e at 0x0x10000000000/25 with size=2 bytes 2023-11-08T16:41:50.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.167+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:140 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5500 con 0x564eeddf6800 2023-11-08T16:41:50.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.172+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 56 ==== osd_op_reply(140 10000000000.01800000 [omap-set-vals] v131'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.172+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/25 pointing to 0x0x1000000020e 2023-11-08T16:41:50.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.172+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:141 17.14 17:2ab28be7::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4700 con 0x564eedfcb800 2023-11-08T16:41:50.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.173+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 23 ==== osd_op_reply(141 [pgnls-filter start_epoch 131 out=101b] v131'20 uv20 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.173+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000211.00000000 2023-11-08T16:41:50.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.173+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000211.0 2023-11-08T16:41:50.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.173+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:142 17.14 17:2ab28be7:::10000000211.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eedfcb800 2023-11-08T16:41:50.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.174+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 24 ==== osd_op_reply(142 10000000211.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.174+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000211 2023-11-08T16:41:50.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.174+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/28 2023-11-08T16:41:50.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.174+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=28 2023-11-08T16:41:50.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.174+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:143 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eede60000 2023-11-08T16:41:50.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.175+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 20 ==== osd_op_reply(143 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.177 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.175+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.175+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:144 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6800 2023-11-08T16:41:50.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.176+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 57 ==== osd_op_reply(144 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.176+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.176+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.178 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.176+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.176+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:145 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405500 con 0x564eeddf6c00 2023-11-08T16:41:50.179 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.177+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 37 ==== osd_op_reply(145 10000000000.01000000 [omap-get-header out=274b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.177+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 13 2023-11-08T16:41:50.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.177+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.177+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:146 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eeddf6c00 2023-11-08T16:41:50.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.178+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 38 ==== osd_op_reply(146 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv13 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.180 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.178+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=28 frag=0*, r=0 2023-11-08T16:41:50.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.178+0000 7f9189c365c0 20 datascan.read_dentry: 28_head not found in result 2023-11-08T16:41:50.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.178+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000211 at 0x0x10000000000/28 with size=2 bytes 2023-11-08T16:41:50.181 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.178+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:147 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6c00 2023-11-08T16:41:50.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.181+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 39 ==== osd_op_reply(147 10000000000.01000000 [omap-set-vals] v131'14 uv14 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.181+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/28 pointing to 0x0x10000000211 2023-11-08T16:41:50.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.181+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:148 17.14 17:2b89f3ba::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2000 con 0x564eedfcb800 2023-11-08T16:41:50.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.182+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 25 ==== osd_op_reply(148 [pgnls-filter start_epoch 131 out=81b] v131'20 uv20 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.182+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000221.00000000 2023-11-08T16:41:50.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.182+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000221.0 2023-11-08T16:41:50.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.183+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:149 17.14 17:2b89f3ba:::10000000221.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eedfcb800 2023-11-08T16:41:50.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.184+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 26 ==== osd_op_reply(149 10000000221.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.184+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000221 2023-11-08T16:41:50.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.184+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/44 2023-11-08T16:41:50.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.184+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=44 2023-11-08T16:41:50.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.184+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:150 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2700 con 0x564eede60000 2023-11-08T16:41:50.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.185+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 21 ==== osd_op_reply(150 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.185+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.185+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:151 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3880 con 0x564eeddf6800 2023-11-08T16:41:50.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.186+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 58 ==== osd_op_reply(151 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.186+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.188 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.186+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.186+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.186+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:152 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3c00 con 0x564eeddf6800 2023-11-08T16:41:50.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.187+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 59 ==== osd_op_reply(152 10000000000.01800000 [omap-get-header out=274b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.187+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 16 2023-11-08T16:41:50.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.187+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.187+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:153 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2a80 con 0x564eeddf6800 2023-11-08T16:41:50.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.188+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 60 ==== osd_op_reply(153 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.188+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=44 frag=1*, r=0 2023-11-08T16:41:50.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.188+0000 7f9189c365c0 20 datascan.read_dentry: 44_head not found in result 2023-11-08T16:41:50.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.188+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000221 at 0x0x10000000000/44 with size=2 bytes 2023-11-08T16:41:50.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.188+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:154 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6800 2023-11-08T16:41:50.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.192+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 61 ==== osd_op_reply(154 10000000000.01800000 [omap-set-vals] v131'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.192+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/44 pointing to 0x0x10000000221 2023-11-08T16:41:50.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.192+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:155 17.34 17:2c000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eeddf6c00 2023-11-08T16:41:50.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.193+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 40 ==== osd_op_reply(155 [pgnls-filter start_epoch 131 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.193+0000 7f9182b39640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:156 17.c 17:30000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3880 con 0x564eeddf6400 2023-11-08T16:41:50.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.195+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 6 ==== osd_op_reply(156 [pgnls-filter start_epoch 131 out=81b] v131'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.195+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 100000001fc.00000000 2023-11-08T16:41:50.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.195+0000 7f9189c365c0 10 datascan.operator(): handling object 100000001fc.0 2023-11-08T16:41:50.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.195+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:157 17.c 17:30604137:::100000001fc.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eeddf6400 2023-11-08T16:41:50.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.196+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 7 ==== osd_op_reply(157 100000001fc.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.196+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fc 2023-11-08T16:41:50.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.196+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/7 2023-11-08T16:41:50.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.196+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=7 2023-11-08T16:41:50.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.196+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:158 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eede60000 2023-11-08T16:41:50.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.197+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 22 ==== osd_op_reply(158 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.197+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.197+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:159 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6800 2023-11-08T16:41:50.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.199+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 62 ==== osd_op_reply(159 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.199+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.199+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.199+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.199+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:160 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6800 2023-11-08T16:41:50.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.200+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 63 ==== osd_op_reply(160 10000000000.01800000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.200+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-11-08T16:41:50.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.200+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.200+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:161 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405500 con 0x564eeddf6800 2023-11-08T16:41:50.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.201+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 64 ==== osd_op_reply(161 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.201+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=7 frag=1*, r=0 2023-11-08T16:41:50.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.201+0000 7f9189c365c0 20 datascan.read_dentry: 7_head not found in result 2023-11-08T16:41:50.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.201+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fc at 0x0x10000000000/7 with size=1 bytes 2023-11-08T16:41:50.204 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.201+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:162 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eeddf6800 2023-11-08T16:41:50.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.205+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 65 ==== osd_op_reply(162 10000000000.01800000 [omap-set-vals] v131'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.205+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-11-08T16:41:50.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.205+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x564eed426c00 0x564eed3e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:50.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.205+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:163 17.2c 17:34000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eed426c00 2023-11-08T16:41:50.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.206+0000 7f9187342640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x564eed426c00 0x564eed3e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:50.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.207+0000 7f9187342640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x564eed426c00 0x564eed3e7600 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:50.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.208+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(163 [pgnls-filter start_epoch 131 out=101b] v131'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.208+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000223.00000000 2023-11-08T16:41:50.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.208+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000223.0 2023-11-08T16:41:50.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.208+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:164 17.2c 17:35964134:::10000000223.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5880 con 0x564eed426c00 2023-11-08T16:41:50.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.210+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(164 10000000223.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.210+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000223 2023-11-08T16:41:50.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.210+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/46 2023-11-08T16:41:50.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.210+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=46 2023-11-08T16:41:50.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.210+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:165 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5500 con 0x564eede60000 2023-11-08T16:41:50.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.211+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 23 ==== osd_op_reply(165 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.211+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.213 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.211+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:166 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eeddf6800 2023-11-08T16:41:50.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.212+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 66 ==== osd_op_reply(166 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.212+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.212+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.214 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.212+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.212+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:167 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4a80 con 0x564eeddf6c00 2023-11-08T16:41:50.215 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.213+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 41 ==== osd_op_reply(167 10000000000.01000000 [omap-get-header out=274b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.213+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 14 2023-11-08T16:41:50.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.213+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.213+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:168 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eeddf6c00 2023-11-08T16:41:50.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.214+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 42 ==== osd_op_reply(168 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv14 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.214+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=46 frag=0*, r=0 2023-11-08T16:41:50.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.214+0000 7f9189c365c0 20 datascan.read_dentry: 46_head not found in result 2023-11-08T16:41:50.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.214+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000223 at 0x0x10000000000/46 with size=2 bytes 2023-11-08T16:41:50.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.214+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:169 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eeddf6c00 2023-11-08T16:41:50.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.218+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 43 ==== osd_op_reply(169 10000000000.01000000 [omap-set-vals] v131'15 uv15 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.218+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/46 pointing to 0x0x10000000223 2023-11-08T16:41:50.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.218+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:170 17.2c 17:37389ade::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2a80 con 0x564eed426c00 2023-11-08T16:41:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.219+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(170 [pgnls-filter start_epoch 131 out=81b] v131'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.219+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000209.00000000 2023-11-08T16:41:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.219+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000209.0 2023-11-08T16:41:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.219+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:171 17.2c 17:37389ade:::10000000209.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eed426c00 2023-11-08T16:41:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.220+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(171 10000000209.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.220+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000209 2023-11-08T16:41:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.220+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/20 2023-11-08T16:41:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.220+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=20 2023-11-08T16:41:50.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.220+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:172 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454380 con 0x564eede60000 2023-11-08T16:41:50.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.221+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 24 ==== osd_op_reply(172 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.221+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.222+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:173 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454700 con 0x564eeddf6800 2023-11-08T16:41:50.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.223+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 67 ==== osd_op_reply(173 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.223+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.224 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.223+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.223+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.223+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:174 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454a80 con 0x564eeddf6c00 2023-11-08T16:41:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.224+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 44 ==== osd_op_reply(174 10000000000.01000000 [omap-get-header out=274b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.224+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 15 2023-11-08T16:41:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.224+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.226 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.224+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:175 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454e00 con 0x564eeddf6c00 2023-11-08T16:41:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.225+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 45 ==== osd_op_reply(175 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv15 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.225+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=20 frag=0*, r=0 2023-11-08T16:41:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.225+0000 7f9189c365c0 20 datascan.read_dentry: 20_head not found in result 2023-11-08T16:41:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.225+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000209 at 0x0x10000000000/20 with size=2 bytes 2023-11-08T16:41:50.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.225+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:176 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455180 con 0x564eeddf6c00 2023-11-08T16:41:50.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.229+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 46 ==== osd_op_reply(176 10000000000.01000000 [omap-set-vals] v131'16 uv16 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.230 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.229+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-11-08T16:41:50.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.229+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:177 17.1c 17:38000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455500 con 0x564eedfcb000 2023-11-08T16:41:50.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.230+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(177 [pgnls-filter start_epoch 131 out=49b] v127'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.230+0000 7f9182b39640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:178 17.3c 17:3c000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3c00 con 0x564eeddf6c00 2023-11-08T16:41:50.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.231+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 47 ==== osd_op_reply(178 [pgnls-filter start_epoch 131 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.231+0000 7f918333a640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:179 17.2 17:40000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4380 con 0x564eed426c00 2023-11-08T16:41:50.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.232+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 5 ==== osd_op_reply(179 [pgnls-filter start_epoch 131 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.233+0000 7f9182b39640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:180 17.22 17:44000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6800 2023-11-08T16:41:50.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.234+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 68 ==== osd_op_reply(180 [pgnls-filter start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.234+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000024c.00000000 2023-11-08T16:41:50.235 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.234+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000024c.0 2023-11-08T16:41:50.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.234+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:181 17.22 17:450d4ca5:::1000000024c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455880 con 0x564eeddf6800 2023-11-08T16:41:50.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.235+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 69 ==== osd_op_reply(181 1000000024c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.235+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024c 2023-11-08T16:41:50.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.235+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/87 2023-11-08T16:41:50.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.235+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=87 2023-11-08T16:41:50.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.235+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:182 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454000 con 0x564eede60000 2023-11-08T16:41:50.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.236+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 25 ==== osd_op_reply(182 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.236+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.236+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:183 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454380 con 0x564eeddf6800 2023-11-08T16:41:50.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.237+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 70 ==== osd_op_reply(183 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.237+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.237+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.237+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.237+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:184 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454a80 con 0x564eeddf6c00 2023-11-08T16:41:50.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.238+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 48 ==== osd_op_reply(184 10000000000.01000000 [omap-get-header out=274b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.238+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 16 2023-11-08T16:41:50.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.238+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.238+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:185 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6c00 2023-11-08T16:41:50.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.239+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 49 ==== osd_op_reply(185 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv16 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.239+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=87 frag=0*, r=0 2023-11-08T16:41:50.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.239+0000 7f9189c365c0 20 datascan.read_dentry: 87_head not found in result 2023-11-08T16:41:50.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.239+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024c at 0x0x10000000000/87 with size=2 bytes 2023-11-08T16:41:50.242 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.239+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:186 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2a80 con 0x564eeddf6c00 2023-11-08T16:41:50.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.243+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 50 ==== osd_op_reply(186 10000000000.01000000 [omap-set-vals] v131'17 uv17 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.243+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/87 pointing to 0x0x1000000024c 2023-11-08T16:41:50.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.243+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:187 17.22 17:45b8cac8::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2700 con 0x564eeddf6800 2023-11-08T16:41:50.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.244+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 71 ==== osd_op_reply(187 [pgnls-filter start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.244+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000023d.00000000 2023-11-08T16:41:50.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.244+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000023d.0 2023-11-08T16:41:50.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.244+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:188 17.22 17:45b8cac8:::1000000023d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eeddf6800 2023-11-08T16:41:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.245+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 72 ==== osd_op_reply(188 1000000023d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.245+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023d 2023-11-08T16:41:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.245+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/72 2023-11-08T16:41:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.245+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=72 2023-11-08T16:41:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.245+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:189 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4a80 con 0x564eede60000 2023-11-08T16:41:50.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.246+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 26 ==== osd_op_reply(189 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.246+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.247+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:190 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5500 con 0x564eeddf6800 2023-11-08T16:41:50.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.248+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 73 ==== osd_op_reply(190 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.248+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.248+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.248+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.248+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:191 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5880 con 0x564eeddf6800 2023-11-08T16:41:50.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.249+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 74 ==== osd_op_reply(191 10000000000.01800000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.249+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-11-08T16:41:50.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.249+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.249+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:192 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5c00 con 0x564eeddf6800 2023-11-08T16:41:50.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.251+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 75 ==== osd_op_reply(192 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.251+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=72 frag=1*, r=0 2023-11-08T16:41:50.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.251+0000 7f9189c365c0 20 datascan.read_dentry: 72_head not found in result 2023-11-08T16:41:50.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.251+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023d at 0x0x10000000000/72 with size=2 bytes 2023-11-08T16:41:50.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.251+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:193 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eeddf6800 2023-11-08T16:41:50.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.255+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 76 ==== osd_op_reply(193 10000000000.01800000 [omap-set-vals] v131'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.256 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.255+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/72 pointing to 0x0x1000000023d 2023-11-08T16:41:50.257 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.255+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:194 17.22 17:47efb22f::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eeddf6800 2023-11-08T16:41:50.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.256+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 77 ==== osd_op_reply(194 [pgnls-filter start_epoch 131 out=81b] v131'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.256+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000235.00000000 2023-11-08T16:41:50.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.256+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000235.0 2023-11-08T16:41:50.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.256+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:195 17.22 17:47efb22f:::10000000235.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2700 con 0x564eeddf6800 2023-11-08T16:41:50.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.258+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 78 ==== osd_op_reply(195 10000000235.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.258+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000235 2023-11-08T16:41:50.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.258+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/64 2023-11-08T16:41:50.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.258+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=64 2023-11-08T16:41:50.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.258+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:196 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455500 con 0x564eede60000 2023-11-08T16:41:50.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.259+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 27 ==== osd_op_reply(196 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.259+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.259+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:197 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455c00 con 0x564eeddf6800 2023-11-08T16:41:50.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.260+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 79 ==== osd_op_reply(197 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.260+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.260+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.260+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.260+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:198 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455880 con 0x564eeddf6c00 2023-11-08T16:41:50.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.261+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 51 ==== osd_op_reply(198 10000000000.01000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.261+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 17 2023-11-08T16:41:50.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.261+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.261+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:199 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454380 con 0x564eeddf6c00 2023-11-08T16:41:50.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.262+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 52 ==== osd_op_reply(199 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.262+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=64 frag=0*, r=0 2023-11-08T16:41:50.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.262+0000 7f9189c365c0 20 datascan.read_dentry: 64_head not found in result 2023-11-08T16:41:50.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.262+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000235 at 0x0x10000000000/64 with size=2 bytes 2023-11-08T16:41:50.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.262+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:200 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6c00 2023-11-08T16:41:50.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.265+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 53 ==== osd_op_reply(200 10000000000.01000000 [omap-set-vals] v131'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.265+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/64 pointing to 0x0x10000000235 2023-11-08T16:41:50.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.265+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:201 17.12 17:48000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eedfcb800 2023-11-08T16:41:50.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.266+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 27 ==== osd_op_reply(201 [pgnls-filter start_epoch 131 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.266+0000 7f9182b39640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:202 17.32 17:4c000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6400 2023-11-08T16:41:50.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.268+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 8 ==== osd_op_reply(202 [pgnls-filter start_epoch 131 out=81b] v131'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.268+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000252.00000000 2023-11-08T16:41:50.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.268+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000252.0 2023-11-08T16:41:50.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.268+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:203 17.32 17:4e8c6251:::10000000252.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405500 con 0x564eeddf6400 2023-11-08T16:41:50.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.269+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 9 ==== osd_op_reply(203 10000000252.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.269+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000252 2023-11-08T16:41:50.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.269+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/93 2023-11-08T16:41:50.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.269+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=93 2023-11-08T16:41:50.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.269+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:204 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eede60000 2023-11-08T16:41:50.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.270+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 28 ==== osd_op_reply(204 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.273 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.270+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.271+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:205 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464000 con 0x564eeddf6800 2023-11-08T16:41:50.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.272+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 80 ==== osd_op_reply(205 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.272+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.272+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.272+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.272+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:206 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464380 con 0x564eeddf6800 2023-11-08T16:41:50.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.273+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 81 ==== osd_op_reply(206 10000000000.01800000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.273+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-11-08T16:41:50.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.273+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.273+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:207 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464700 con 0x564eeddf6800 2023-11-08T16:41:50.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.275+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 82 ==== osd_op_reply(207 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.275+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=93 frag=1*, r=0 2023-11-08T16:41:50.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.275+0000 7f9189c365c0 20 datascan.read_dentry: 93_head not found in result 2023-11-08T16:41:50.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.275+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000252 at 0x0x10000000000/93 with size=2 bytes 2023-11-08T16:41:50.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.275+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:208 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464a80 con 0x564eeddf6800 2023-11-08T16:41:50.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.278+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 83 ==== osd_op_reply(208 10000000000.01800000 [omap-set-vals] v131'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.278+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/93 pointing to 0x0x10000000252 2023-11-08T16:41:50.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.278+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:209 17.a 17:50000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464e00 con 0x564eeddf6400 2023-11-08T16:41:50.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.280+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 10 ==== osd_op_reply(209 [pgnls-filter start_epoch 131 out=101b] v131'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.280+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000023f.00000000 2023-11-08T16:41:50.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.280+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000023f.0 2023-11-08T16:41:50.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.280+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:210 17.a 17:52dcfedc:::1000000023f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465180 con 0x564eeddf6400 2023-11-08T16:41:50.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.281+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 11 ==== osd_op_reply(210 1000000023f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.281+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023f 2023-11-08T16:41:50.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.281+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/74 2023-11-08T16:41:50.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.281+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=74 2023-11-08T16:41:50.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.281+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:211 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465880 con 0x564eede60000 2023-11-08T16:41:50.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.282+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 29 ==== osd_op_reply(211 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.282+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.282+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:212 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465c00 con 0x564eeddf6800 2023-11-08T16:41:50.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.283+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 84 ==== osd_op_reply(212 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.283+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.283+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.283+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.283+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:213 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464000 con 0x564eeddf6c00 2023-11-08T16:41:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.284+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 54 ==== osd_op_reply(213 10000000000.01000000 [omap-get-header out=274b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.284+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 18 2023-11-08T16:41:50.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.284+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.284+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:214 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464380 con 0x564eeddf6c00 2023-11-08T16:41:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.285+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 55 ==== osd_op_reply(214 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv18 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.285+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=74 frag=0*, r=0 2023-11-08T16:41:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.285+0000 7f9189c365c0 20 datascan.read_dentry: 74_head not found in result 2023-11-08T16:41:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.285+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023f at 0x0x10000000000/74 with size=2 bytes 2023-11-08T16:41:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.285+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:215 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464700 con 0x564eeddf6c00 2023-11-08T16:41:50.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.289+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 56 ==== osd_op_reply(215 10000000000.01000000 [omap-set-vals] v131'19 uv19 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.289+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/74 pointing to 0x0x1000000023f 2023-11-08T16:41:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.289+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:216 17.a 17:53d16f76::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464a80 con 0x564eeddf6400 2023-11-08T16:41:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.290+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 12 ==== osd_op_reply(216 [pgnls-filter start_epoch 131 out=81b] v131'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.290+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000023b.00000000 2023-11-08T16:41:50.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.290+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000023b.0 2023-11-08T16:41:50.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.290+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:217 17.a 17:53d16f76:::1000000023b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405500 con 0x564eeddf6400 2023-11-08T16:41:50.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.291+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 13 ==== osd_op_reply(217 1000000023b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.292+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023b 2023-11-08T16:41:50.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.292+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/70 2023-11-08T16:41:50.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.292+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=70 2023-11-08T16:41:50.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.292+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:218 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eede60000 2023-11-08T16:41:50.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.292+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 30 ==== osd_op_reply(218 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.293+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.293+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:219 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6800 2023-11-08T16:41:50.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.294+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 85 ==== osd_op_reply(219 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.294+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.294+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.294+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.294+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:220 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6c00 2023-11-08T16:41:50.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.295+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 57 ==== osd_op_reply(220 10000000000.01000000 [omap-get-header out=274b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.295+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 19 2023-11-08T16:41:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.295+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.295+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:221 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eeddf6c00 2023-11-08T16:41:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.296+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 58 ==== osd_op_reply(221 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv19 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.296+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=70 frag=0*, r=0 2023-11-08T16:41:50.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.296+0000 7f9189c365c0 20 datascan.read_dentry: 70_head not found in result 2023-11-08T16:41:50.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.296+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023b at 0x0x10000000000/70 with size=2 bytes 2023-11-08T16:41:50.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.296+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:222 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eeddf6c00 2023-11-08T16:41:50.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.299+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 59 ==== osd_op_reply(222 10000000000.01000000 [omap-set-vals] v131'20 uv20 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.299+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/70 pointing to 0x0x1000000023b 2023-11-08T16:41:50.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.299+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:223 17.2a 17:54000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eedfcb800 2023-11-08T16:41:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.301+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 28 ==== osd_op_reply(223 [pgnls-filter start_epoch 131 out=101b] v131'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.301+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000020a.00000000 2023-11-08T16:41:50.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.301+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000020a.0 2023-11-08T16:41:50.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.301+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:224 17.2a 17:55a6cd83:::1000000020a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2700 con 0x564eedfcb800 2023-11-08T16:41:50.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.302+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 29 ==== osd_op_reply(224 1000000020a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.302+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020a 2023-11-08T16:41:50.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.302+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/21 2023-11-08T16:41:50.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.302+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=21 2023-11-08T16:41:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.302+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:225 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eede60000 2023-11-08T16:41:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.304+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 31 ==== osd_op_reply(225 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.304+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.304+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:226 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eeddf6800 2023-11-08T16:41:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.305+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 86 ==== osd_op_reply(226 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.305+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.305+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.305+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.305+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:227 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2000 con 0x564eeddf6c00 2023-11-08T16:41:50.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.306+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 60 ==== osd_op_reply(227 10000000000.01000000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.306+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-11-08T16:41:50.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.306+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.306+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:228 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2e00 con 0x564eeddf6c00 2023-11-08T16:41:50.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.307+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 61 ==== osd_op_reply(228 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.307+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=21 frag=0*, r=0 2023-11-08T16:41:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.307+0000 7f9189c365c0 20 datascan.read_dentry: 21_head not found in result 2023-11-08T16:41:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.307+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020a at 0x0x10000000000/21 with size=2 bytes 2023-11-08T16:41:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.307+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:229 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eeddf6c00 2023-11-08T16:41:50.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.311+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 62 ==== osd_op_reply(229 10000000000.01000000 [omap-set-vals] v131'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.311+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-11-08T16:41:50.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.311+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:230 17.2a 17:56889a45::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5500 con 0x564eedfcb800 2023-11-08T16:41:50.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.312+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 30 ==== osd_op_reply(230 [pgnls-filter start_epoch 131 out=101b] v131'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.312+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000213.00000000 2023-11-08T16:41:50.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.312+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000213.0 2023-11-08T16:41:50.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.312+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:231 17.2a 17:56889a45:::10000000213.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5880 con 0x564eedfcb800 2023-11-08T16:41:50.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.314+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 31 ==== osd_op_reply(231 10000000213.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.314+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000213 2023-11-08T16:41:50.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.314+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/30 2023-11-08T16:41:50.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.314+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=30 2023-11-08T16:41:50.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.314+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:232 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eede60000 2023-11-08T16:41:50.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.315+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 32 ==== osd_op_reply(232 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.315+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.315+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:233 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eeddf6800 2023-11-08T16:41:50.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.316+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 87 ==== osd_op_reply(233 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.316+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.316+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.316+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.316+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:234 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4700 con 0x564eeddf6800 2023-11-08T16:41:50.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.317+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 88 ==== osd_op_reply(234 10000000000.01800000 [omap-get-header out=274b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.317+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 20 2023-11-08T16:41:50.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.317+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.317+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:235 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454700 con 0x564eeddf6800 2023-11-08T16:41:50.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.318+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 89 ==== osd_op_reply(235 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv20 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.318+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=30 frag=1*, r=0 2023-11-08T16:41:50.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.318+0000 7f9189c365c0 20 datascan.read_dentry: 30_head not found in result 2023-11-08T16:41:50.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.318+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000213 at 0x0x10000000000/30 with size=2 bytes 2023-11-08T16:41:50.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.318+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:236 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455c00 con 0x564eeddf6800 2023-11-08T16:41:50.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.322+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 90 ==== osd_op_reply(236 10000000000.01800000 [omap-set-vals] v131'21 uv21 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.322+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/30 pointing to 0x0x10000000213 2023-11-08T16:41:50.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.323+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:237 17.2a 17:576ad4f4::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465880 con 0x564eedfcb800 2023-11-08T16:41:50.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.324+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 32 ==== osd_op_reply(237 [pgnls-filter start_epoch 131 out=101b] v131'21 uv21 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.324+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000245.00000000 2023-11-08T16:41:50.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.324+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000245.0 2023-11-08T16:41:50.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.324+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:238 17.2a 17:576ad4f4:::10000000245.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464000 con 0x564eedfcb800 2023-11-08T16:41:50.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.325+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 33 ==== osd_op_reply(238 10000000245.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv20 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.325+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000245 2023-11-08T16:41:50.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.325+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/80 2023-11-08T16:41:50.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.325+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=80 2023-11-08T16:41:50.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.325+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:239 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464e00 con 0x564eede60000 2023-11-08T16:41:50.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.326+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 33 ==== osd_op_reply(239 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.326+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.326+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:240 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465500 con 0x564eeddf6800 2023-11-08T16:41:50.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.328+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 91 ==== osd_op_reply(240 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.328+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.328+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.328+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.328+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:241 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6c00 2023-11-08T16:41:50.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.329+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 63 ==== osd_op_reply(241 10000000000.01000000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.329+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 21 2023-11-08T16:41:50.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.329+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.329+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:242 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6c00 2023-11-08T16:41:50.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.330+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 64 ==== osd_op_reply(242 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.330+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=80 frag=0*, r=0 2023-11-08T16:41:50.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.330+0000 7f9189c365c0 20 datascan.read_dentry: 80_head not found in result 2023-11-08T16:41:50.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.330+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000245 at 0x0x10000000000/80 with size=2 bytes 2023-11-08T16:41:50.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.330+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:243 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454e00 con 0x564eeddf6c00 2023-11-08T16:41:50.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.334+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 65 ==== osd_op_reply(243 10000000000.01000000 [omap-set-vals] v131'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.334+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/80 pointing to 0x0x10000000245 2023-11-08T16:41:50.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.334+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:244 17.2a 17:57fe7535::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455180 con 0x564eedfcb800 2023-11-08T16:41:50.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.335+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 34 ==== osd_op_reply(244 [pgnls-filter start_epoch 131 out=81b] v131'21 uv21 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.335+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 100000001f6.00000000 2023-11-08T16:41:50.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.335+0000 7f9189c365c0 10 datascan.operator(): handling object 100000001f6.0 2023-11-08T16:41:50.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.335+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:245 17.2a 17:57fe7535:::100000001f6.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454000 con 0x564eedfcb800 2023-11-08T16:41:50.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.337+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 35 ==== osd_op_reply(245 100000001f6.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv21 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.337+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f6 2023-11-08T16:41:50.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.337+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/1 2023-11-08T16:41:50.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.337+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=1 2023-11-08T16:41:50.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.337+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:246 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455500 con 0x564eede60000 2023-11-08T16:41:50.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.338+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 34 ==== osd_op_reply(246 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.338+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.338+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:247 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455880 con 0x564eeddf6800 2023-11-08T16:41:50.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.339+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 92 ==== osd_op_reply(247 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.339+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.339+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.339+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.339+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:248 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454380 con 0x564eeddf6c00 2023-11-08T16:41:50.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.340+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 66 ==== osd_op_reply(248 10000000000.01000000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.340+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 22 2023-11-08T16:41:50.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.340+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.340+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:249 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4380 con 0x564eeddf6c00 2023-11-08T16:41:50.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.340+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 67 ==== osd_op_reply(249 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.341+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=1 frag=0*, r=0 2023-11-08T16:41:50.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.341+0000 7f9189c365c0 20 datascan.read_dentry: 1_head not found in result 2023-11-08T16:41:50.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.341+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f6 at 0x0x10000000000/1 with size=1 bytes 2023-11-08T16:41:50.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.341+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:250 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4a80 con 0x564eeddf6c00 2023-11-08T16:41:50.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.344+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 68 ==== osd_op_reply(250 10000000000.01000000 [omap-set-vals] v131'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.344+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-11-08T16:41:50.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.344+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:251 17.1a 17:58000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3c00 con 0x564eeddf6c00 2023-11-08T16:41:50.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.345+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 69 ==== osd_op_reply(251 [pgnls-filter start_epoch 131 out=101b] v131'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.345+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000226.00000000 2023-11-08T16:41:50.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.345+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000226.0 2023-11-08T16:41:50.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.345+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:252 17.1a 17:582f7b2d:::10000000226.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6c00 2023-11-08T16:41:50.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.346+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 70 ==== osd_op_reply(252 10000000226.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.346+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000226 2023-11-08T16:41:50.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.346+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/49 2023-11-08T16:41:50.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.346+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=49 2023-11-08T16:41:50.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.346+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:253 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454e00 con 0x564eede60000 2023-11-08T16:41:50.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.347+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 35 ==== osd_op_reply(253 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.347+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.347+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:254 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455500 con 0x564eeddf6800 2023-11-08T16:41:50.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.348+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 93 ==== osd_op_reply(254 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.348+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.349+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.349+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.349+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:255 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454380 con 0x564eeddf6800 2023-11-08T16:41:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.350+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 94 ==== osd_op_reply(255 10000000000.01800000 [omap-get-header out=274b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.350+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 21 2023-11-08T16:41:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.350+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.350+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:256 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6800 2023-11-08T16:41:50.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.351+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 95 ==== osd_op_reply(256 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv21 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.351+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=49 frag=1*, r=0 2023-11-08T16:41:50.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.351+0000 7f9189c365c0 20 datascan.read_dentry: 49_head not found in result 2023-11-08T16:41:50.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.351+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000226 at 0x0x10000000000/49 with size=2 bytes 2023-11-08T16:41:50.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.351+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:257 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6800 2023-11-08T16:41:50.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.355+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 96 ==== osd_op_reply(257 10000000000.01800000 [omap-set-vals] v131'22 uv22 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.355+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/49 pointing to 0x0x10000000226 2023-11-08T16:41:50.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.355+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:258 17.1a 17:5a2a1294::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eeddf6c00 2023-11-08T16:41:50.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.356+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 71 ==== osd_op_reply(258 [pgnls-filter start_epoch 131 out=81b] v131'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.356+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000023a.00000000 2023-11-08T16:41:50.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.356+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000023a.0 2023-11-08T16:41:50.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.356+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:259 17.1a 17:5a2a1294:::1000000023a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eeddf6c00 2023-11-08T16:41:50.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.357+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 72 ==== osd_op_reply(259 1000000023a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.357+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023a 2023-11-08T16:41:50.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.357+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/69 2023-11-08T16:41:50.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.357+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=69 2023-11-08T16:41:50.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.357+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:260 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eede60000 2023-11-08T16:41:50.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.359+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 36 ==== osd_op_reply(260 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.359+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.359+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:261 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464380 con 0x564eeddf6800 2023-11-08T16:41:50.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.360+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 97 ==== osd_op_reply(261 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.360+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.360+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.360+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.360+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:262 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464700 con 0x564eeddf6c00 2023-11-08T16:41:50.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.361+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 73 ==== osd_op_reply(262 10000000000.01000000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.361+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 23 2023-11-08T16:41:50.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.361+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.361+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:263 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464e00 con 0x564eeddf6c00 2023-11-08T16:41:50.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.362+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 74 ==== osd_op_reply(263 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.362+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=69 frag=0*, r=0 2023-11-08T16:41:50.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.362+0000 7f9189c365c0 20 datascan.read_dentry: 69_head not found in result 2023-11-08T16:41:50.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.362+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023a at 0x0x10000000000/69 with size=2 bytes 2023-11-08T16:41:50.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.362+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:264 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eeddf6c00 2023-11-08T16:41:50.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.366+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 75 ==== osd_op_reply(264 10000000000.01000000 [omap-set-vals] v131'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.366+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/69 pointing to 0x0x1000000023a 2023-11-08T16:41:50.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.366+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:265 17.3a 17:5c000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eedfcb000 2023-11-08T16:41:50.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.367+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(265 [pgnls-filter start_epoch 131 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.367+0000 7f9182b39640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x564eed427800 0x564eed3e8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:50.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.367+0000 7f9182b39640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:266 17.6 17:60000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eed427800 2023-11-08T16:41:50.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.368+0000 7f9188344640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x564eed427800 0x564eed3e8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:50.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.369+0000 7f9188344640 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x564eed427800 0x564eed3e8c00 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:50.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.370+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(266 [pgnls-filter start_epoch 131 out=81b] v131'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72d80 con 0x564eed427800 2023-11-08T16:41:50.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.371+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000228.00000000 2023-11-08T16:41:50.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.371+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000228.0 2023-11-08T16:41:50.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.371+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:267 17.6 17:614797eb:::10000000228.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eed427800 2023-11-08T16:41:50.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.372+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(267 10000000228.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eed427800 2023-11-08T16:41:50.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.372+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000228 2023-11-08T16:41:50.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.372+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/51 2023-11-08T16:41:50.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.372+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=51 2023-11-08T16:41:50.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.372+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:268 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2e00 con 0x564eede60000 2023-11-08T16:41:50.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.373+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 37 ==== osd_op_reply(268 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.373+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.373+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:269 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eeddf6800 2023-11-08T16:41:50.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.374+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 98 ==== osd_op_reply(269 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.374+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.374+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.374+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.374+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:270 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eeddf6800 2023-11-08T16:41:50.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.375+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 99 ==== osd_op_reply(270 10000000000.01800000 [omap-get-header out=274b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.375+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 22 2023-11-08T16:41:50.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.375+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.375+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:271 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4700 con 0x564eeddf6800 2023-11-08T16:41:50.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.376+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 100 ==== osd_op_reply(271 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv22 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.376+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=51 frag=1*, r=0 2023-11-08T16:41:50.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.376+0000 7f9189c365c0 20 datascan.read_dentry: 51_head not found in result 2023-11-08T16:41:50.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.376+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000228 at 0x0x10000000000/51 with size=2 bytes 2023-11-08T16:41:50.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.376+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:272 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eeddf6800 2023-11-08T16:41:50.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.380+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 101 ==== osd_op_reply(272 10000000000.01800000 [omap-set-vals] v131'23 uv23 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.380+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/51 pointing to 0x0x10000000228 2023-11-08T16:41:50.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.380+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:273 17.26 17:64000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465500 con 0x564eeddf6c00 2023-11-08T16:41:50.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.381+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 76 ==== osd_op_reply(273 [pgnls-filter start_epoch 131 out=101b] v131'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.381+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 100000001ff.00000000 2023-11-08T16:41:50.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.381+0000 7f9189c365c0 10 datascan.operator(): handling object 100000001ff.0 2023-11-08T16:41:50.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.381+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:274 17.26 17:658f2455:::100000001ff.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464380 con 0x564eeddf6c00 2023-11-08T16:41:50.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.382+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 77 ==== osd_op_reply(274 100000001ff.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv22 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.382+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001ff 2023-11-08T16:41:50.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.382+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/10 2023-11-08T16:41:50.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.382+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=10 2023-11-08T16:41:50.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.382+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:275 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eede60000 2023-11-08T16:41:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.383+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 38 ==== osd_op_reply(275 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.383+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.384+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:276 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6800 2023-11-08T16:41:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.385+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 102 ==== osd_op_reply(276 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.385+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.385+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.385+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.385+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:277 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454700 con 0x564eeddf6c00 2023-11-08T16:41:50.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.386+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 78 ==== osd_op_reply(277 10000000000.01000000 [omap-get-header out=274b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.386+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 24 2023-11-08T16:41:50.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.386+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.386+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:278 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455c00 con 0x564eeddf6c00 2023-11-08T16:41:50.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.387+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 79 ==== osd_op_reply(278 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.387+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=10 frag=0*, r=0 2023-11-08T16:41:50.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.387+0000 7f9189c365c0 20 datascan.read_dentry: 10_head not found in result 2023-11-08T16:41:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.387+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001ff at 0x0x10000000000/10 with size=2 bytes 2023-11-08T16:41:50.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.387+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:279 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455880 con 0x564eeddf6c00 2023-11-08T16:41:50.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.391+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 80 ==== osd_op_reply(279 10000000000.01000000 [omap-set-vals] v131'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.391+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-11-08T16:41:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.391+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:280 17.26 17:65d05a5c::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455500 con 0x564eeddf6c00 2023-11-08T16:41:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.392+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 81 ==== osd_op_reply(280 [pgnls-filter start_epoch 131 out=101b] v131'24 uv24 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.392+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000258.00000000 2023-11-08T16:41:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.392+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000258.0 2023-11-08T16:41:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.392+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:281 17.26 17:65d05a5c:::10000000258.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454380 con 0x564eeddf6c00 2023-11-08T16:41:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.393+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 82 ==== osd_op_reply(281 10000000258.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv23 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.393+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000258 2023-11-08T16:41:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.393+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/99 2023-11-08T16:41:50.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.393+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=99 2023-11-08T16:41:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.393+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:282 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eede60000 2023-11-08T16:41:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.394+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 39 ==== osd_op_reply(282 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.394+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.394+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:283 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6800 2023-11-08T16:41:50.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.396+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 103 ==== osd_op_reply(283 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.396+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.396+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.396+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.396+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:284 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eeddf6800 2023-11-08T16:41:50.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.397+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 104 ==== osd_op_reply(284 10000000000.01800000 [omap-get-header out=274b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.397+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 23 2023-11-08T16:41:50.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.397+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.397+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:285 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6800 2023-11-08T16:41:50.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.398+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 105 ==== osd_op_reply(285 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv23 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.398+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=99 frag=1*, r=0 2023-11-08T16:41:50.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.398+0000 7f9189c365c0 20 datascan.read_dentry: 99_head not found in result 2023-11-08T16:41:50.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.398+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000258 at 0x0x10000000000/99 with size=2 bytes 2023-11-08T16:41:50.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.398+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:286 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6800 2023-11-08T16:41:50.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.402+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 106 ==== osd_op_reply(286 10000000000.01800000 [omap-set-vals] v131'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.402+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/99 pointing to 0x0x10000000258 2023-11-08T16:41:50.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.402+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:287 17.26 17:65e03fee::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6c00 2023-11-08T16:41:50.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.403+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 83 ==== osd_op_reply(287 [pgnls-filter start_epoch 131 out=81b] v131'24 uv24 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.403+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000240.00000000 2023-11-08T16:41:50.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.403+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000240.0 2023-11-08T16:41:50.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.403+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:288 17.26 17:65e03fee:::10000000240.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464700 con 0x564eeddf6c00 2023-11-08T16:41:50.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.404+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 84 ==== osd_op_reply(288 10000000240.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv24 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.404+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000240 2023-11-08T16:41:50.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.404+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/75 2023-11-08T16:41:50.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.404+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=75 2023-11-08T16:41:50.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.404+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:289 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465500 con 0x564eede60000 2023-11-08T16:41:50.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.405+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 40 ==== osd_op_reply(289 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.405+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.405+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:290 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464380 con 0x564eeddf6800 2023-11-08T16:41:50.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.406+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 107 ==== osd_op_reply(290 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.406+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.406+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.406+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.407+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:291 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3c00 con 0x564eeddf6c00 2023-11-08T16:41:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.407+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 85 ==== osd_op_reply(291 10000000000.01000000 [omap-get-header out=274b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.408+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 25 2023-11-08T16:41:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.408+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.408+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:292 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2a80 con 0x564eeddf6c00 2023-11-08T16:41:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.408+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 86 ==== osd_op_reply(292 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.409+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=75 frag=0*, r=0 2023-11-08T16:41:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.409+0000 7f9189c365c0 20 datascan.read_dentry: 75_head not found in result 2023-11-08T16:41:50.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.409+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000240 at 0x0x10000000000/75 with size=2 bytes 2023-11-08T16:41:50.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.409+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:293 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6c00 2023-11-08T16:41:50.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.412+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 87 ==== osd_op_reply(293 10000000000.01000000 [omap-set-vals] v131'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.412+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/75 pointing to 0x0x10000000240 2023-11-08T16:41:50.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.412+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:294 17.16 17:68000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2e00 con 0x564eeddf6800 2023-11-08T16:41:50.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.414+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 108 ==== osd_op_reply(294 [pgnls-filter start_epoch 131 out=49b] v127'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.414+0000 7f918333a640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:295 17.36 17:6c000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c000 con 0x564eeddf6c00 2023-11-08T16:41:50.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.415+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 88 ==== osd_op_reply(295 [pgnls-filter start_epoch 131 out=101b] v131'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.415+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000255.00000000 2023-11-08T16:41:50.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.415+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000255.0 2023-11-08T16:41:50.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.415+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:296 17.36 17:6cdfec98:::10000000255.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4380 con 0x564eeddf6c00 2023-11-08T16:41:50.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.416+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 89 ==== osd_op_reply(296 10000000255.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv24 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.416+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000255 2023-11-08T16:41:50.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.416+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/96 2023-11-08T16:41:50.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.416+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=96 2023-11-08T16:41:50.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.416+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:297 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eede60000 2023-11-08T16:41:50.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.417+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 41 ==== osd_op_reply(297 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.417+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.417+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:298 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454e00 con 0x564eeddf6800 2023-11-08T16:41:50.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.418+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 109 ==== osd_op_reply(298 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.419+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.419+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.419+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.419+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:299 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ca80 con 0x564eeddf6c00 2023-11-08T16:41:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.419+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 90 ==== osd_op_reply(299 10000000000.01000000 [omap-get-header out=274b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.420+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 26 2023-11-08T16:41:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.420+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.420+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:300 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ce00 con 0x564eeddf6c00 2023-11-08T16:41:50.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.420+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 91 ==== osd_op_reply(300 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.421+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=96 frag=0*, r=0 2023-11-08T16:41:50.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.421+0000 7f9189c365c0 20 datascan.read_dentry: 96_head not found in result 2023-11-08T16:41:50.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.421+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000255 at 0x0x10000000000/96 with size=2 bytes 2023-11-08T16:41:50.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.421+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:301 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d180 con 0x564eeddf6c00 2023-11-08T16:41:50.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.424+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 92 ==== osd_op_reply(301 10000000000.01000000 [omap-set-vals] v131'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.424+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/96 pointing to 0x0x10000000255 2023-11-08T16:41:50.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.424+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:302 17.36 17:6ee5e2c4::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d500 con 0x564eeddf6c00 2023-11-08T16:41:50.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.425+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 93 ==== osd_op_reply(302 [pgnls-filter start_epoch 131 out=101b] v131'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.425+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 100000001f9.00000000 2023-11-08T16:41:50.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.425+0000 7f9189c365c0 10 datascan.operator(): handling object 100000001f9.0 2023-11-08T16:41:50.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.426+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:303 17.36 17:6ee5e2c4:::100000001f9.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d880 con 0x564eeddf6c00 2023-11-08T16:41:50.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.427+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 94 ==== osd_op_reply(303 100000001f9.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv25 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.427+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f9 2023-11-08T16:41:50.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.427+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/4 2023-11-08T16:41:50.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.427+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=4 2023-11-08T16:41:50.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.427+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:304 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c000 con 0x564eede60000 2023-11-08T16:41:50.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.427+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 42 ==== osd_op_reply(304 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.428+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.428+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:305 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ca80 con 0x564eeddf6800 2023-11-08T16:41:50.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.429+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 110 ==== osd_op_reply(305 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.429+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.429+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.429+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.429+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:306 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4a80 con 0x564eeddf6c00 2023-11-08T16:41:50.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.430+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 95 ==== osd_op_reply(306 10000000000.01000000 [omap-get-header out=274b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.430+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 27 2023-11-08T16:41:50.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.430+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.430+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:307 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4380 con 0x564eeddf6c00 2023-11-08T16:41:50.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.431+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 96 ==== osd_op_reply(307 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.431+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=4 frag=0*, r=0 2023-11-08T16:41:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.431+0000 7f9189c365c0 20 datascan.read_dentry: 4_head not found in result 2023-11-08T16:41:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.431+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f9 at 0x0x10000000000/4 with size=1 bytes 2023-11-08T16:41:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.431+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:308 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eeddf6c00 2023-11-08T16:41:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.435+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 97 ==== osd_op_reply(308 10000000000.01000000 [omap-set-vals] v131'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.435+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-11-08T16:41:50.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.435+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:309 17.36 17:6f746908::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eeddf6c00 2023-11-08T16:41:50.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.436+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 98 ==== osd_op_reply(309 [pgnls-filter start_epoch 131 out=101b] v131'27 uv27 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.436+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000022c.00000000 2023-11-08T16:41:50.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.436+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000022c.0 2023-11-08T16:41:50.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.436+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:310 17.36 17:6f746908:::1000000022c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3c00 con 0x564eeddf6c00 2023-11-08T16:41:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.437+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 99 ==== osd_op_reply(310 1000000022c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv26 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.437+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022c 2023-11-08T16:41:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.437+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/55 2023-11-08T16:41:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.437+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=55 2023-11-08T16:41:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.437+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:311 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eede60000 2023-11-08T16:41:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.438+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 43 ==== osd_op_reply(311 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.438+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.438+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:312 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464e00 con 0x564eeddf6800 2023-11-08T16:41:50.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.439+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 111 ==== osd_op_reply(312 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.439+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.439+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.439+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.439+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:313 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464700 con 0x564eeddf6c00 2023-11-08T16:41:50.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.440+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 100 ==== osd_op_reply(313 10000000000.01000000 [omap-get-header out=274b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.441+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 28 2023-11-08T16:41:50.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.441+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.441+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:314 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465500 con 0x564eeddf6c00 2023-11-08T16:41:50.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.442+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 101 ==== osd_op_reply(314 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.442+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=55 frag=0*, r=0 2023-11-08T16:41:50.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.442+0000 7f9189c365c0 20 datascan.read_dentry: 55_head not found in result 2023-11-08T16:41:50.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.442+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022c at 0x0x10000000000/55 with size=2 bytes 2023-11-08T16:41:50.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.442+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:315 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eeddf6c00 2023-11-08T16:41:50.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.445+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 102 ==== osd_op_reply(315 10000000000.01000000 [omap-set-vals] v131'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.445+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/55 pointing to 0x0x1000000022c 2023-11-08T16:41:50.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.445+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:316 17.36 17:6fe38dc2::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6c00 2023-11-08T16:41:50.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.446+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 103 ==== osd_op_reply(316 [pgnls-filter start_epoch 131 out=81b] v131'27 uv27 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.446+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2023-11-08T16:41:50.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.446+0000 7f9189c365c0 10 datascan.operator(): handling object 100000001f7.0 2023-11-08T16:41:50.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.446+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:317 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454700 con 0x564eeddf6c00 2023-11-08T16:41:50.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.448+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 104 ==== osd_op_reply(317 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.448+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2023-11-08T16:41:50.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.448+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/2 2023-11-08T16:41:50.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.448+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=2 2023-11-08T16:41:50.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.448+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:318 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455880 con 0x564eede60000 2023-11-08T16:41:50.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.449+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 44 ==== osd_op_reply(318 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.449+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.449+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:319 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455500 con 0x564eeddf6800 2023-11-08T16:41:50.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.450+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 112 ==== osd_op_reply(319 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.450+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.450+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.450+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.450+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:320 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454380 con 0x564eeddf6c00 2023-11-08T16:41:50.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.451+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 105 ==== osd_op_reply(320 10000000000.01000000 [omap-get-header out=274b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.451+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 29 2023-11-08T16:41:50.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.451+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.451+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:321 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ce00 con 0x564eeddf6c00 2023-11-08T16:41:50.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.452+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 106 ==== osd_op_reply(321 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.452+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=2 frag=0*, r=0 2023-11-08T16:41:50.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.452+0000 7f9189c365c0 20 datascan.read_dentry: 2_head not found in result 2023-11-08T16:41:50.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.452+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000000/2 with size=1 bytes 2023-11-08T16:41:50.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.452+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:322 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d180 con 0x564eeddf6c00 2023-11-08T16:41:50.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.456+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 107 ==== osd_op_reply(322 10000000000.01000000 [omap-set-vals] v131'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.456+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-11-08T16:41:50.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.456+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:323 17.e 17:70000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d500 con 0x564eede60000 2023-11-08T16:41:50.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.457+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 45 ==== osd_op_reply(323 [pgnls-filter start_epoch 131 out=81b] v131'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.457+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000022f.00000000 2023-11-08T16:41:50.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.457+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000022f.0 2023-11-08T16:41:50.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.457+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:324 17.e 17:71c13145:::1000000022f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47dc00 con 0x564eede60000 2023-11-08T16:41:50.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.458+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 46 ==== osd_op_reply(324 1000000022f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.458+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022f 2023-11-08T16:41:50.460 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.458+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/58 2023-11-08T16:41:50.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.458+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=58 2023-11-08T16:41:50.461 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.458+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:325 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c000 con 0x564eede60000 2023-11-08T16:41:50.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.459+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 47 ==== osd_op_reply(325 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.459+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.459+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:326 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ca80 con 0x564eeddf6800 2023-11-08T16:41:50.462 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.460+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 113 ==== osd_op_reply(326 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.460+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.460+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.460+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.463 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.460+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:327 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455c00 con 0x564eeddf6800 2023-11-08T16:41:50.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.461+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 114 ==== osd_op_reply(327 10000000000.01800000 [omap-get-header out=274b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.461+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 24 2023-11-08T16:41:50.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.461+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.464 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.461+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:328 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454700 con 0x564eeddf6800 2023-11-08T16:41:50.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.462+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 115 ==== osd_op_reply(328 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv24 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.462+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=58 frag=1*, r=0 2023-11-08T16:41:50.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.462+0000 7f9189c365c0 20 datascan.read_dentry: 58_head not found in result 2023-11-08T16:41:50.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.462+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022f at 0x0x10000000000/58 with size=2 bytes 2023-11-08T16:41:50.465 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.462+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:329 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455880 con 0x564eeddf6800 2023-11-08T16:41:50.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.466+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 116 ==== osd_op_reply(329 10000000000.01800000 [omap-set-vals] v131'25 uv25 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.466+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/58 pointing to 0x0x1000000022f 2023-11-08T16:41:50.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.466+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:330 17.2e 17:74000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454e00 con 0x564eed427800 2023-11-08T16:41:50.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.467+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(330 [pgnls-filter start_epoch 131 out=101b] v131'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72d80 con 0x564eed427800 2023-11-08T16:41:50.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.467+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000021c.00000000 2023-11-08T16:41:50.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.467+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000021c.0 2023-11-08T16:41:50.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.467+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:331 17.2e 17:74d13cfc:::1000000021c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eed427800 2023-11-08T16:41:50.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.468+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(331 1000000021c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eed427800 2023-11-08T16:41:50.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.468+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021c 2023-11-08T16:41:50.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.468+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/39 2023-11-08T16:41:50.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.468+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=39 2023-11-08T16:41:50.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.468+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:332 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3c00 con 0x564eede60000 2023-11-08T16:41:50.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.469+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 48 ==== osd_op_reply(332 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.469+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.469+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:333 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6800 2023-11-08T16:41:50.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.470+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 117 ==== osd_op_reply(333 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.470+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.471+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.471+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.471+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:334 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2000 con 0x564eeddf6c00 2023-11-08T16:41:50.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.471+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 108 ==== osd_op_reply(334 10000000000.01000000 [omap-get-header out=274b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.472+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 30 2023-11-08T16:41:50.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.472+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.472+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:335 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eeddf6c00 2023-11-08T16:41:50.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.472+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 109 ==== osd_op_reply(335 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.473+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=39 frag=0*, r=0 2023-11-08T16:41:50.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.473+0000 7f9189c365c0 20 datascan.read_dentry: 39_head not found in result 2023-11-08T16:41:50.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.473+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021c at 0x0x10000000000/39 with size=2 bytes 2023-11-08T16:41:50.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.473+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:336 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2e00 con 0x564eeddf6c00 2023-11-08T16:41:50.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.476+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 110 ==== osd_op_reply(336 10000000000.01000000 [omap-set-vals] v131'31 uv31 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.476+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/39 pointing to 0x0x1000000021c 2023-11-08T16:41:50.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.476+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:337 17.2e 17:7528f44f::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eed427800 2023-11-08T16:41:50.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.477+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(337 [pgnls-filter start_epoch 131 out=101b] v131'19 uv19 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72d80 con 0x564eed427800 2023-11-08T16:41:50.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.477+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000242.00000000 2023-11-08T16:41:50.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.477+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000242.0 2023-11-08T16:41:50.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.477+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:338 17.2e 17:7528f44f:::10000000242.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eed427800 2023-11-08T16:41:50.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.478+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(338 10000000242.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eed427800 2023-11-08T16:41:50.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.479+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000242 2023-11-08T16:41:50.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.479+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/77 2023-11-08T16:41:50.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.479+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=77 2023-11-08T16:41:50.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.479+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:339 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eede60000 2023-11-08T16:41:50.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.479+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 49 ==== osd_op_reply(339 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.480+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.480+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:340 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eeddf6800 2023-11-08T16:41:50.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.481+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 118 ==== osd_op_reply(340 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.481+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.481+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.481+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.481+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:341 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6c00 2023-11-08T16:41:50.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.482+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 111 ==== osd_op_reply(341 10000000000.01000000 [omap-get-header out=274b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.482+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 31 2023-11-08T16:41:50.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.482+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.482+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:342 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6c00 2023-11-08T16:41:50.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.483+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 112 ==== osd_op_reply(342 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.483+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=77 frag=0*, r=0 2023-11-08T16:41:50.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.483+0000 7f9189c365c0 20 datascan.read_dentry: 77_head not found in result 2023-11-08T16:41:50.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.483+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000242 at 0x0x10000000000/77 with size=2 bytes 2023-11-08T16:41:50.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.483+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:343 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4a80 con 0x564eeddf6c00 2023-11-08T16:41:50.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.486+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 113 ==== osd_op_reply(343 10000000000.01000000 [omap-set-vals] v131'32 uv32 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.486+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/77 pointing to 0x0x10000000242 2023-11-08T16:41:50.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.486+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:344 17.2e 17:7611a124::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4380 con 0x564eed427800 2023-11-08T16:41:50.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.487+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(344 [pgnls-filter start_epoch 131 out=81b] v131'19 uv19 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72d80 con 0x564eed427800 2023-11-08T16:41:50.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.487+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000021e.00000000 2023-11-08T16:41:50.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.487+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000021e.0 2023-11-08T16:41:50.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.487+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:345 17.2e 17:7611a124:::1000000021e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eed427800 2023-11-08T16:41:50.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.489+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 8 ==== osd_op_reply(345 1000000021e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv19 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eed427800 2023-11-08T16:41:50.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.489+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021e 2023-11-08T16:41:50.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.489+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/41 2023-11-08T16:41:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.489+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=41 2023-11-08T16:41:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.489+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:346 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4700 con 0x564eede60000 2023-11-08T16:41:50.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.490+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 50 ==== osd_op_reply(346 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.490+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.490+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:347 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eeddf6800 2023-11-08T16:41:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.491+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 119 ==== osd_op_reply(347 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.491+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.491+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.491+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.491+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:348 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eeddf6800 2023-11-08T16:41:50.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.492+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 120 ==== osd_op_reply(348 10000000000.01800000 [omap-get-header out=274b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.492+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 25 2023-11-08T16:41:50.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.492+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.492+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:349 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464700 con 0x564eeddf6800 2023-11-08T16:41:50.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.493+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 121 ==== osd_op_reply(349 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv25 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.493+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=41 frag=1*, r=0 2023-11-08T16:41:50.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.493+0000 7f9189c365c0 20 datascan.read_dentry: 41_head not found in result 2023-11-08T16:41:50.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.493+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021e at 0x0x10000000000/41 with size=2 bytes 2023-11-08T16:41:50.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.493+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:350 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465500 con 0x564eeddf6800 2023-11-08T16:41:50.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.496+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 122 ==== osd_op_reply(350 10000000000.01800000 [omap-set-vals] v131'26 uv26 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.496+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/41 pointing to 0x0x1000000021e 2023-11-08T16:41:50.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.496+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:351 17.1e 17:78000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464380 con 0x564eedfcb000 2023-11-08T16:41:50.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.497+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(351 [pgnls-filter start_epoch 131 out=101b] v131'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.497+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000020c.00000000 2023-11-08T16:41:50.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.497+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000020c.0 2023-11-08T16:41:50.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.498+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:352 17.1e 17:789b801b:::1000000020c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464e00 con 0x564eedfcb000 2023-11-08T16:41:50.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.499+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(352 1000000020c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.499+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020c 2023-11-08T16:41:50.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.499+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/23 2023-11-08T16:41:50.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.499+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=23 2023-11-08T16:41:50.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.499+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:353 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eede60000 2023-11-08T16:41:50.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.500+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 51 ==== osd_op_reply(353 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.500+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.500+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:354 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eeddf6800 2023-11-08T16:41:50.502 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.501+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 123 ==== osd_op_reply(354 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.501+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.501+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.501+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.501+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:355 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eeddf6800 2023-11-08T16:41:50.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.502+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 124 ==== osd_op_reply(355 10000000000.01800000 [omap-get-header out=274b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.502+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 26 2023-11-08T16:41:50.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.502+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.502+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:356 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eeddf6800 2023-11-08T16:41:50.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.503+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 125 ==== osd_op_reply(356 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv26 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.503+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=23 frag=1*, r=0 2023-11-08T16:41:50.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.503+0000 7f9189c365c0 20 datascan.read_dentry: 23_head not found in result 2023-11-08T16:41:50.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.503+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020c at 0x0x10000000000/23 with size=2 bytes 2023-11-08T16:41:50.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.503+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:357 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6800 2023-11-08T16:41:50.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.507+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 126 ==== osd_op_reply(357 10000000000.01800000 [omap-set-vals] v131'27 uv27 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.507+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-11-08T16:41:50.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.507+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:358 17.1e 17:7a9d9b47::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eedfcb000 2023-11-08T16:41:50.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.508+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(358 [pgnls-filter start_epoch 131 out=101b] v131'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.508+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000022b.00000000 2023-11-08T16:41:50.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.508+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000022b.0 2023-11-08T16:41:50.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.508+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:359 17.1e 17:7a9d9b47:::1000000022b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eedfcb000 2023-11-08T16:41:50.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.509+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(359 1000000022b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.509+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022b 2023-11-08T16:41:50.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.509+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/54 2023-11-08T16:41:50.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.509+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=54 2023-11-08T16:41:50.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.509+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:360 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eede60000 2023-11-08T16:41:50.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.510+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 52 ==== osd_op_reply(360 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.510+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.510+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:361 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6800 2023-11-08T16:41:50.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.511+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 127 ==== osd_op_reply(361 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.512+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.512+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.512+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.512+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:362 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455500 con 0x564eeddf6800 2023-11-08T16:41:50.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.513+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 128 ==== osd_op_reply(362 10000000000.01800000 [omap-get-header out=274b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.513+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 27 2023-11-08T16:41:50.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.513+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.513+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:363 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455c00 con 0x564eeddf6800 2023-11-08T16:41:50.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.514+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 129 ==== osd_op_reply(363 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv27 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.514+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=54 frag=1*, r=0 2023-11-08T16:41:50.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.514+0000 7f9189c365c0 20 datascan.read_dentry: 54_head not found in result 2023-11-08T16:41:50.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.514+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022b at 0x0x10000000000/54 with size=2 bytes 2023-11-08T16:41:50.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.514+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:364 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454700 con 0x564eeddf6800 2023-11-08T16:41:50.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.518+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 130 ==== osd_op_reply(364 10000000000.01800000 [omap-set-vals] v131'28 uv28 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.518+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/54 pointing to 0x0x1000000022b 2023-11-08T16:41:50.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.518+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:365 17.1e 17:7ade2fd8::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455880 con 0x564eedfcb000 2023-11-08T16:41:50.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.519+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(365 [pgnls-filter start_epoch 131 out=81b] v131'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.519+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000024e.00000000 2023-11-08T16:41:50.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.519+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000024e.0 2023-11-08T16:41:50.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.520+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:366 17.1e 17:7ade2fd8:::1000000024e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454e00 con 0x564eedfcb000 2023-11-08T16:41:50.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.521+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(366 1000000024e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.521+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024e 2023-11-08T16:41:50.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.521+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/89 2023-11-08T16:41:50.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.521+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=89 2023-11-08T16:41:50.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.521+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:367 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455500 con 0x564eede60000 2023-11-08T16:41:50.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.522+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 53 ==== osd_op_reply(367 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.522+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.522+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:368 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455c00 con 0x564eeddf6800 2023-11-08T16:41:50.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.523+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 131 ==== osd_op_reply(368 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.523+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.523+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.523+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.523+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:369 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454700 con 0x564eeddf6800 2023-11-08T16:41:50.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.524+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 132 ==== osd_op_reply(369 10000000000.01800000 [omap-get-header out=274b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.525+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 28 2023-11-08T16:41:50.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.525+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.525+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:370 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455880 con 0x564eeddf6800 2023-11-08T16:41:50.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.526+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 133 ==== osd_op_reply(370 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv28 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.526+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=89 frag=1*, r=0 2023-11-08T16:41:50.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.526+0000 7f9189c365c0 20 datascan.read_dentry: 89_head not found in result 2023-11-08T16:41:50.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.526+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024e at 0x0x10000000000/89 with size=2 bytes 2023-11-08T16:41:50.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.526+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:371 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2a80 con 0x564eeddf6800 2023-11-08T16:41:50.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.529+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 134 ==== osd_op_reply(371 10000000000.01800000 [omap-set-vals] v131'29 uv29 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.530+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/89 pointing to 0x0x1000000024e 2023-11-08T16:41:50.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.530+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:372 17.3e 17:7c000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eed426c00 2023-11-08T16:41:50.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.531+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 6 ==== osd_op_reply(372 [pgnls-filter start_epoch 131 out=81b] v131'7 uv7 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.531+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000229.00000000 2023-11-08T16:41:50.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.531+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000229.0 2023-11-08T16:41:50.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.531+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:373 17.3e 17:7f918703:::10000000229.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eed426c00 2023-11-08T16:41:50.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.532+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 7 ==== osd_op_reply(373 10000000229.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.533+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000229 2023-11-08T16:41:50.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.533+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/52 2023-11-08T16:41:50.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.533+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=52 2023-11-08T16:41:50.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.533+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:374 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eede60000 2023-11-08T16:41:50.539 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.533+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 54 ==== osd_op_reply(374 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.533+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.534+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:375 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eeddf6800 2023-11-08T16:41:50.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.535+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 135 ==== osd_op_reply(375 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.535+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.535+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.535+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.535+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:376 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4380 con 0x564eeddf6c00 2023-11-08T16:41:50.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.536+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 114 ==== osd_op_reply(376 10000000000.01000000 [omap-get-header out=274b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.536+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 32 2023-11-08T16:41:50.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.536+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.536+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:377 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eeddf6c00 2023-11-08T16:41:50.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.537+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 115 ==== osd_op_reply(377 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.537+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=52 frag=0*, r=0 2023-11-08T16:41:50.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.537+0000 7f9189c365c0 20 datascan.read_dentry: 52_head not found in result 2023-11-08T16:41:50.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.537+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000229 at 0x0x10000000000/52 with size=2 bytes 2023-11-08T16:41:50.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.537+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:378 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eeddf6c00 2023-11-08T16:41:50.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.541+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 116 ==== osd_op_reply(378 10000000000.01000000 [omap-set-vals] v131'33 uv33 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.541+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/52 pointing to 0x0x10000000229 2023-11-08T16:41:50.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.541+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:379 17.1 17:80000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464700 con 0x564eeddf6c00 2023-11-08T16:41:50.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.542+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 117 ==== osd_op_reply(379 [pgnls-filter start_epoch 131 out=81b] v131'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.542+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000224.00000000 2023-11-08T16:41:50.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.542+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000224.0 2023-11-08T16:41:50.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.542+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:380 17.1 17:805bc955:::10000000224.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465500 con 0x564eeddf6c00 2023-11-08T16:41:50.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.543+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 118 ==== osd_op_reply(380 10000000224.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.543+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000224 2023-11-08T16:41:50.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.543+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/47 2023-11-08T16:41:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.543+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=47 2023-11-08T16:41:50.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.543+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:381 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464e00 con 0x564eede60000 2023-11-08T16:41:50.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.544+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 55 ==== osd_op_reply(381 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.544+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.544+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:382 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eeddf6800 2023-11-08T16:41:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.545+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 136 ==== osd_op_reply(382 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.545+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.545+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.545+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.545+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:383 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6800 2023-11-08T16:41:50.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.547+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 137 ==== osd_op_reply(383 10000000000.01800000 [omap-get-header out=274b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.547+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 29 2023-11-08T16:41:50.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.547+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.547+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:384 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6800 2023-11-08T16:41:50.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.548+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 138 ==== osd_op_reply(384 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv29 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.548+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=47 frag=1*, r=0 2023-11-08T16:41:50.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.548+0000 7f9189c365c0 20 datascan.read_dentry: 47_head not found in result 2023-11-08T16:41:50.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.548+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000224 at 0x0x10000000000/47 with size=2 bytes 2023-11-08T16:41:50.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.548+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:385 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6800 2023-11-08T16:41:50.553 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.552+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 139 ==== osd_op_reply(385 10000000000.01800000 [omap-set-vals] v131'30 uv30 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.552+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/47 pointing to 0x0x10000000224 2023-11-08T16:41:50.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.552+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:386 17.21 17:84000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6c00 2023-11-08T16:41:50.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.553+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 119 ==== osd_op_reply(386 [pgnls-filter start_epoch 131 out=49b] v127'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.553+0000 7f918333a640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:387 17.11 17:88000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c380 con 0x564eedfcb800 2023-11-08T16:41:50.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.554+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 36 ==== osd_op_reply(387 [pgnls-filter start_epoch 131 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.554+0000 7f9182b39640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:388 17.31 17:8c000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3880 con 0x564eedfcb000 2023-11-08T16:41:50.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.555+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 11 ==== osd_op_reply(388 [pgnls-filter start_epoch 131 out=101b] v131'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.555+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000216.00000000 2023-11-08T16:41:50.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.555+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000216.0 2023-11-08T16:41:50.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.555+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:389 17.31 17:8c51d9b0:::10000000216.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3c00 con 0x564eedfcb000 2023-11-08T16:41:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.556+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 12 ==== osd_op_reply(389 10000000216.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.557+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000216 2023-11-08T16:41:50.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.557+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/33 2023-11-08T16:41:50.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.557+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=33 2023-11-08T16:41:50.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.557+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:390 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eede60000 2023-11-08T16:41:50.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.557+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 56 ==== osd_op_reply(390 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.558+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.558+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:391 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2e00 con 0x564eeddf6800 2023-11-08T16:41:50.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.559+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 140 ==== osd_op_reply(391 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.559+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.559+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.559+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.559+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:392 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eeddf6800 2023-11-08T16:41:50.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.560+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 141 ==== osd_op_reply(392 10000000000.01800000 [omap-get-header out=274b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.560+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 30 2023-11-08T16:41:50.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.560+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.560+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:393 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6800 2023-11-08T16:41:50.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.561+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 142 ==== osd_op_reply(393 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv30 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.561+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=33 frag=1*, r=0 2023-11-08T16:41:50.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.561+0000 7f9189c365c0 20 datascan.read_dentry: 33_head not found in result 2023-11-08T16:41:50.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.561+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000216 at 0x0x10000000000/33 with size=2 bytes 2023-11-08T16:41:50.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.561+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:394 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed454380 con 0x564eeddf6800 2023-11-08T16:41:50.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.565+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 143 ==== osd_op_reply(394 10000000000.01800000 [omap-set-vals] v131'31 uv31 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.565+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/33 pointing to 0x0x10000000216 2023-11-08T16:41:50.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.565+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:395 17.31 17:8fb7f7b7::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed455500 con 0x564eedfcb000 2023-11-08T16:41:50.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.566+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 13 ==== osd_op_reply(395 [pgnls-filter start_epoch 131 out=81b] v131'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.567 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.566+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000251.00000000 2023-11-08T16:41:50.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.566+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000251.0 2023-11-08T16:41:50.568 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.566+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:396 17.31 17:8fb7f7b7:::10000000251.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4a80 con 0x564eedfcb000 2023-11-08T16:41:50.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.567+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 14 ==== osd_op_reply(396 10000000251.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eedfcb000 2023-11-08T16:41:50.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.567+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000251 2023-11-08T16:41:50.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.567+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/92 2023-11-08T16:41:50.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.567+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=92 2023-11-08T16:41:50.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.567+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:397 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eede60000 2023-11-08T16:41:50.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.568+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 57 ==== osd_op_reply(397 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.568+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.568+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:398 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ce00 con 0x564eeddf6800 2023-11-08T16:41:50.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.569+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 144 ==== osd_op_reply(398 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.569+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.569+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.569+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.569+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:399 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d180 con 0x564eeddf6c00 2023-11-08T16:41:50.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.570+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 120 ==== osd_op_reply(399 10000000000.01000000 [omap-get-header out=274b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.570+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 33 2023-11-08T16:41:50.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.570+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.570+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:400 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d500 con 0x564eeddf6c00 2023-11-08T16:41:50.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.571+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 121 ==== osd_op_reply(400 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.571+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=92 frag=0*, r=0 2023-11-08T16:41:50.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.571+0000 7f9189c365c0 20 datascan.read_dentry: 92_head not found in result 2023-11-08T16:41:50.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.571+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000251 at 0x0x10000000000/92 with size=2 bytes 2023-11-08T16:41:50.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.571+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:401 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d880 con 0x564eeddf6c00 2023-11-08T16:41:50.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.575+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 122 ==== osd_op_reply(401 10000000000.01000000 [omap-set-vals] v131'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.575+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/92 pointing to 0x0x10000000251 2023-11-08T16:41:50.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.575+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:402 17.9 17:90000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47dc00 con 0x564eedfcb800 2023-11-08T16:41:50.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.576+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 37 ==== osd_op_reply(402 [pgnls-filter start_epoch 131 out=101b] v131'8 uv8 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.576+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000024d.00000000 2023-11-08T16:41:50.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.576+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000024d.0 2023-11-08T16:41:50.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.576+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:403 17.9 17:915bff82:::1000000024d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c000 con 0x564eedfcb800 2023-11-08T16:41:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.577+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 38 ==== osd_op_reply(403 1000000024d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.577+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024d 2023-11-08T16:41:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.577+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/88 2023-11-08T16:41:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.577+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=88 2023-11-08T16:41:50.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.577+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:404 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c380 con 0x564eede60000 2023-11-08T16:41:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.578+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 58 ==== osd_op_reply(404 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.578+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.578+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:405 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465180 con 0x564eeddf6800 2023-11-08T16:41:50.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.579+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 145 ==== osd_op_reply(405 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.579+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.579+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.579+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.580+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:406 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464a80 con 0x564eeddf6c00 2023-11-08T16:41:50.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.580+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 123 ==== osd_op_reply(406 10000000000.01000000 [omap-get-header out=274b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.581+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 34 2023-11-08T16:41:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.581+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.581+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:407 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465880 con 0x564eeddf6c00 2023-11-08T16:41:50.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.581+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 124 ==== osd_op_reply(407 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.582+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=88 frag=0*, r=0 2023-11-08T16:41:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.582+0000 7f9189c365c0 20 datascan.read_dentry: 88_head not found in result 2023-11-08T16:41:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.582+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024d at 0x0x10000000000/88 with size=2 bytes 2023-11-08T16:41:50.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.582+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:408 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465c00 con 0x564eeddf6c00 2023-11-08T16:41:50.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.585+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 125 ==== osd_op_reply(408 10000000000.01000000 [omap-set-vals] v131'35 uv35 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.586+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/88 pointing to 0x0x1000000024d 2023-11-08T16:41:50.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.586+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:409 17.9 17:91e366d7::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464000 con 0x564eedfcb800 2023-11-08T16:41:50.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.586+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 39 ==== osd_op_reply(409 [pgnls-filter start_epoch 131 out=81b] v131'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.587+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000230.00000000 2023-11-08T16:41:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.587+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000230.0 2023-11-08T16:41:50.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.587+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:410 17.9 17:91e366d7:::10000000230.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5500 con 0x564eedfcb800 2023-11-08T16:41:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.588+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 40 ==== osd_op_reply(410 10000000230.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.588+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000230 2023-11-08T16:41:50.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.588+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/59 2023-11-08T16:41:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.588+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=59 2023-11-08T16:41:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.588+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:411 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5880 con 0x564eede60000 2023-11-08T16:41:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.589+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 59 ==== osd_op_reply(411 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.589+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.589+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:412 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2a80 con 0x564eeddf6800 2023-11-08T16:41:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.590+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 146 ==== osd_op_reply(412 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.590+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.590+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.590+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.590+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:413 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3880 con 0x564eeddf6c00 2023-11-08T16:41:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.591+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 126 ==== osd_op_reply(413 10000000000.01000000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.591+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 35 2023-11-08T16:41:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.591+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.591+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:414 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2000 con 0x564eeddf6c00 2023-11-08T16:41:50.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.592+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 127 ==== osd_op_reply(414 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.592+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=59 frag=0*, r=0 2023-11-08T16:41:50.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.592+0000 7f9189c365c0 20 datascan.read_dentry: 59_head not found in result 2023-11-08T16:41:50.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.592+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000230 at 0x0x10000000000/59 with size=2 bytes 2023-11-08T16:41:50.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.592+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:415 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3c00 con 0x564eeddf6c00 2023-11-08T16:41:50.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.596+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 128 ==== osd_op_reply(415 10000000000.01000000 [omap-set-vals] v131'36 uv36 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.596+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/59 pointing to 0x0x10000000230 2023-11-08T16:41:50.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.596+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:416 17.29 17:94000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2e00 con 0x564eed426c00 2023-11-08T16:41:50.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.597+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 8 ==== osd_op_reply(416 [pgnls-filter start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.598+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000024a.00000000 2023-11-08T16:41:50.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.598+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000024a.0 2023-11-08T16:41:50.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.598+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:417 17.29 17:94c02207:::1000000024a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eed426c00 2023-11-08T16:41:50.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.599+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 9 ==== osd_op_reply(417 1000000024a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.599+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024a 2023-11-08T16:41:50.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.599+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/85 2023-11-08T16:41:50.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.599+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=85 2023-11-08T16:41:50.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.599+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:418 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2700 con 0x564eede60000 2023-11-08T16:41:50.602 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.600+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 60 ==== osd_op_reply(418 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.600+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.600+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:419 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eeddf6800 2023-11-08T16:41:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.602+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 147 ==== osd_op_reply(419 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.602+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.602+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.602+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.602+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:420 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6c00 2023-11-08T16:41:50.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.603+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 129 ==== osd_op_reply(420 10000000000.01000000 [omap-get-header out=274b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.603+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 36 2023-11-08T16:41:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.603+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.603+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:421 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6c00 2023-11-08T16:41:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.604+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 130 ==== osd_op_reply(421 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.604+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=85 frag=0*, r=0 2023-11-08T16:41:50.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.604+0000 7f9189c365c0 20 datascan.read_dentry: 85_head not found in result 2023-11-08T16:41:50.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.604+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024a at 0x0x10000000000/85 with size=2 bytes 2023-11-08T16:41:50.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.604+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:422 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6c00 2023-11-08T16:41:50.608 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.607+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 131 ==== osd_op_reply(422 10000000000.01000000 [omap-set-vals] v131'37 uv37 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.607+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/85 pointing to 0x0x1000000024a 2023-11-08T16:41:50.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.607+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:423 17.29 17:95e8e66d::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eed426c00 2023-11-08T16:41:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.608+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 10 ==== osd_op_reply(423 [pgnls-filter start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.608+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000256.00000000 2023-11-08T16:41:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.608+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000256.0 2023-11-08T16:41:50.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.609+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:424 17.29 17:95e8e66d:::10000000256.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eed426c00 2023-11-08T16:41:50.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.610+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 11 ==== osd_op_reply(424 10000000256.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.610+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000256 2023-11-08T16:41:50.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.610+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/97 2023-11-08T16:41:50.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.610+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=97 2023-11-08T16:41:50.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.610+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:425 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2000 con 0x564eede60000 2023-11-08T16:41:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.611+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 61 ==== osd_op_reply(425 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.611+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.611+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:426 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4380 con 0x564eeddf6800 2023-11-08T16:41:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.612+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 148 ==== osd_op_reply(426 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.612+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.612+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.612+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.612+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:427 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eeddf6c00 2023-11-08T16:41:50.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.613+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 132 ==== osd_op_reply(427 10000000000.01000000 [omap-get-header out=274b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.613+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 37 2023-11-08T16:41:50.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.613+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.613+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:428 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eeddf6c00 2023-11-08T16:41:50.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.614+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 133 ==== osd_op_reply(428 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.614+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=97 frag=0*, r=0 2023-11-08T16:41:50.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.614+0000 7f9189c365c0 20 datascan.read_dentry: 97_head not found in result 2023-11-08T16:41:50.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.614+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000256 at 0x0x10000000000/97 with size=2 bytes 2023-11-08T16:41:50.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.614+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:429 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eeddf6c00 2023-11-08T16:41:50.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.617+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 134 ==== osd_op_reply(429 10000000000.01000000 [omap-set-vals] v131'38 uv38 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.618+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/97 pointing to 0x0x10000000256 2023-11-08T16:41:50.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.618+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:430 17.29 17:97401066::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5880 con 0x564eed426c00 2023-11-08T16:41:50.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.619+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 12 ==== osd_op_reply(430 [pgnls-filter start_epoch 131 out=81b] v131'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.619+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000247.00000000 2023-11-08T16:41:50.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.619+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000247.0 2023-11-08T16:41:50.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.619+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:431 17.29 17:97401066:::10000000247.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464700 con 0x564eed426c00 2023-11-08T16:41:50.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.621+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 13 ==== osd_op_reply(431 10000000247.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.621+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000247 2023-11-08T16:41:50.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.621+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/82 2023-11-08T16:41:50.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.621+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=82 2023-11-08T16:41:50.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.621+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:432 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465500 con 0x564eede60000 2023-11-08T16:41:50.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.622+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 62 ==== osd_op_reply(432 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.622+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.622+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:433 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464e00 con 0x564eeddf6800 2023-11-08T16:41:50.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.623+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 149 ==== osd_op_reply(433 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.623+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.623+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.623+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.623+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:434 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464a80 con 0x564eeddf6800 2023-11-08T16:41:50.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.625+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 150 ==== osd_op_reply(434 10000000000.01800000 [omap-get-header out=274b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.625+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 31 2023-11-08T16:41:50.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.625+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.625+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:435 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465880 con 0x564eeddf6800 2023-11-08T16:41:50.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.626+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 151 ==== osd_op_reply(435 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv31 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.626+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=82 frag=1*, r=0 2023-11-08T16:41:50.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.626+0000 7f9189c365c0 20 datascan.read_dentry: 82_head not found in result 2023-11-08T16:41:50.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.626+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000247 at 0x0x10000000000/82 with size=2 bytes 2023-11-08T16:41:50.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.626+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:436 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465c00 con 0x564eeddf6800 2023-11-08T16:41:50.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.630+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 152 ==== osd_op_reply(436 10000000000.01800000 [omap-set-vals] v131'32 uv32 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.630+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/82 pointing to 0x0x10000000247 2023-11-08T16:41:50.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.630+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:437 17.19 17:98000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d180 con 0x564eeddf6800 2023-11-08T16:41:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.631+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 153 ==== osd_op_reply(437 [pgnls-filter start_epoch 131 out=101b] v131'16 uv16 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.631+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000200.00000000 2023-11-08T16:41:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.631+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000200.0 2023-11-08T16:41:50.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.631+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:438 17.19 17:992a6246:::10000000200.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d500 con 0x564eeddf6800 2023-11-08T16:41:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.633+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 154 ==== osd_op_reply(438 10000000200.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.633+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000200 2023-11-08T16:41:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.633+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/11 2023-11-08T16:41:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.633+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=11 2023-11-08T16:41:50.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.633+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:439 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c380 con 0x564eede60000 2023-11-08T16:41:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.634+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 63 ==== osd_op_reply(439 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.634+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.634+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:440 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eeddf6800 2023-11-08T16:41:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.635+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 155 ==== osd_op_reply(440 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.635+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.637 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.635+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.635+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.635+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:441 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6c00 2023-11-08T16:41:50.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.636+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 135 ==== osd_op_reply(441 10000000000.01000000 [omap-get-header out=274b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.636+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 38 2023-11-08T16:41:50.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.636+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.636+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:442 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6c00 2023-11-08T16:41:50.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.637+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 136 ==== osd_op_reply(442 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.637+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=11 frag=0*, r=0 2023-11-08T16:41:50.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.637+0000 7f9189c365c0 20 datascan.read_dentry: 11_head not found in result 2023-11-08T16:41:50.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.637+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000200 at 0x0x10000000000/11 with size=2 bytes 2023-11-08T16:41:50.641 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.637+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:443 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed494000 con 0x564eeddf6c00 2023-11-08T16:41:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.641+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 137 ==== osd_op_reply(443 10000000000.01000000 [omap-set-vals] v131'39 uv39 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.641+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-11-08T16:41:50.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.641+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:444 17.19 17:997b7129::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed494380 con 0x564eeddf6800 2023-11-08T16:41:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.642+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 156 ==== osd_op_reply(444 [pgnls-filter start_epoch 131 out=81b] v131'16 uv16 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.643+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 100000001f8.00000000 2023-11-08T16:41:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.643+0000 7f9189c365c0 10 datascan.operator(): handling object 100000001f8.0 2023-11-08T16:41:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.643+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:445 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed494700 con 0x564eeddf6800 2023-11-08T16:41:50.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.644+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 157 ==== osd_op_reply(445 100000001f8.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.644+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001f8 2023-11-08T16:41:50.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.644+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/3 2023-11-08T16:41:50.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.644+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=3 2023-11-08T16:41:50.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.644+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:446 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed494e00 con 0x564eede60000 2023-11-08T16:41:50.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.645+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 64 ==== osd_op_reply(446 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.645+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.645+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:447 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed495180 con 0x564eeddf6800 2023-11-08T16:41:50.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.646+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 158 ==== osd_op_reply(447 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.646+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.646+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.646+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.646+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:448 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed495500 con 0x564eeddf6c00 2023-11-08T16:41:50.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.647+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 138 ==== osd_op_reply(448 10000000000.01000000 [omap-get-header out=274b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.647+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 39 2023-11-08T16:41:50.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.647+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.647+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:449 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed495880 con 0x564eeddf6c00 2023-11-08T16:41:50.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.648+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 139 ==== osd_op_reply(449 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.648+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=3 frag=0*, r=0 2023-11-08T16:41:50.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.648+0000 7f9189c365c0 20 datascan.read_dentry: 3_head not found in result 2023-11-08T16:41:50.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.648+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f8 at 0x0x10000000000/3 with size=1 bytes 2023-11-08T16:41:50.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.648+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:450 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed495c00 con 0x564eeddf6c00 2023-11-08T16:41:50.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.652+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 140 ==== osd_op_reply(450 10000000000.01000000 [omap-set-vals] v131'40 uv40 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.652+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-11-08T16:41:50.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.652+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:451 17.39 17:9c000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed494000 con 0x564eeddf6800 2023-11-08T16:41:50.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.653+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 159 ==== osd_op_reply(451 [pgnls-filter start_epoch 131 out=81b] v131'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.653+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000222.00000000 2023-11-08T16:41:50.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.653+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000222.0 2023-11-08T16:41:50.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.653+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:452 17.39 17:9f1351c3:::10000000222.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6800 2023-11-08T16:41:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.655+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 160 ==== osd_op_reply(452 10000000222.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv8 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72d80 con 0x564eeddf6800 2023-11-08T16:41:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.655+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000222 2023-11-08T16:41:50.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.655+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/45 2023-11-08T16:41:50.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.655+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=45 2023-11-08T16:41:50.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.655+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:453 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eede60000 2023-11-08T16:41:50.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.656+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 65 ==== osd_op_reply(453 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.656+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.656+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:454 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6800 2023-11-08T16:41:50.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.657+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 161 ==== osd_op_reply(454 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.657+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.657+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.657+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.657+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:455 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6800 2023-11-08T16:41:50.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.658+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 162 ==== osd_op_reply(455 10000000000.01800000 [omap-get-header out=274b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.658+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 32 2023-11-08T16:41:50.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.658+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.658+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:456 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c380 con 0x564eeddf6800 2023-11-08T16:41:50.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.659+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 163 ==== osd_op_reply(456 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv32 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.659+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=45 frag=1*, r=0 2023-11-08T16:41:50.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.659+0000 7f9189c365c0 20 datascan.read_dentry: 45_head not found in result 2023-11-08T16:41:50.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.659+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000222 at 0x0x10000000000/45 with size=2 bytes 2023-11-08T16:41:50.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.660+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:457 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464380 con 0x564eeddf6800 2023-11-08T16:41:50.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.664+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 164 ==== osd_op_reply(457 10000000000.01800000 [omap-set-vals] v131'33 uv33 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.664+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/45 pointing to 0x0x10000000222 2023-11-08T16:41:50.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.664+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:458 17.5 17:a0000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464700 con 0x564eed427800 2023-11-08T16:41:50.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.665+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 9 ==== osd_op_reply(458 [pgnls-filter start_epoch 131 out=101b] v131'11 uv11 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:50.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.665+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000204.00000000 2023-11-08T16:41:50.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.665+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000204.0 2023-11-08T16:41:50.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.665+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:459 17.5 17:a0beda95:::10000000204.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465500 con 0x564eed427800 2023-11-08T16:41:50.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.667+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 10 ==== osd_op_reply(459 10000000204.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:50.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.667+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000204 2023-11-08T16:41:50.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.667+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/15 2023-11-08T16:41:50.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.667+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=15 2023-11-08T16:41:50.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.667+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:460 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eede60000 2023-11-08T16:41:50.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.668+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 66 ==== osd_op_reply(460 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.668+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.668+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:461 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eeddf6800 2023-11-08T16:41:50.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.669+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 165 ==== osd_op_reply(461 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.669+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.669+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.669+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.669+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:462 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5880 con 0x564eeddf6c00 2023-11-08T16:41:50.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.670+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 141 ==== osd_op_reply(462 10000000000.01000000 [omap-get-header out=274b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.670+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 40 2023-11-08T16:41:50.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.670+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.670+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:463 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3c00 con 0x564eeddf6c00 2023-11-08T16:41:50.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.671+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 142 ==== osd_op_reply(463 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.671+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=15 frag=0*, r=0 2023-11-08T16:41:50.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.671+0000 7f9189c365c0 20 datascan.read_dentry: 15_head not found in result 2023-11-08T16:41:50.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.671+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000204 at 0x0x10000000000/15 with size=2 bytes 2023-11-08T16:41:50.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.671+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:464 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2e00 con 0x564eeddf6c00 2023-11-08T16:41:50.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.674+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 143 ==== osd_op_reply(464 10000000000.01000000 [omap-set-vals] v131'41 uv41 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.674+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-11-08T16:41:50.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.674+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:465 17.5 17:a3a50212::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eed427800 2023-11-08T16:41:50.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.676+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 11 ==== osd_op_reply(465 [pgnls-filter start_epoch 131 out=81b] v131'11 uv11 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:50.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.676+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000254.00000000 2023-11-08T16:41:50.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.676+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000254.0 2023-11-08T16:41:50.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.676+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:466 17.5 17:a3a50212:::10000000254.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eed427800 2023-11-08T16:41:50.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.677+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 12 ==== osd_op_reply(466 10000000254.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:50.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.677+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000254 2023-11-08T16:41:50.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.677+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/95 2023-11-08T16:41:50.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.677+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=95 2023-11-08T16:41:50.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.677+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:467 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3880 con 0x564eede60000 2023-11-08T16:41:50.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.678+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 67 ==== osd_op_reply(467 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.678+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.678+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:468 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eeddf6800 2023-11-08T16:41:50.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.680+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 166 ==== osd_op_reply(468 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.680+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.680+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.680+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.680+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:469 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2000 con 0x564eeddf6c00 2023-11-08T16:41:50.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.681+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 144 ==== osd_op_reply(469 10000000000.01000000 [omap-get-header out=274b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.681+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 41 2023-11-08T16:41:50.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.681+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.681+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:470 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eeddf6c00 2023-11-08T16:41:50.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.682+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 145 ==== osd_op_reply(470 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.682+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=95 frag=0*, r=0 2023-11-08T16:41:50.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.682+0000 7f9189c365c0 20 datascan.read_dentry: 95_head not found in result 2023-11-08T16:41:50.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.682+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000254 at 0x0x10000000000/95 with size=2 bytes 2023-11-08T16:41:50.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.682+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:471 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed494380 con 0x564eeddf6c00 2023-11-08T16:41:50.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.686+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 146 ==== osd_op_reply(471 10000000000.01000000 [omap-set-vals] v131'42 uv42 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.686+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/95 pointing to 0x0x10000000254 2023-11-08T16:41:50.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.686+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:472 17.25 17:a4000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed494a80 con 0x564eedfcb000 2023-11-08T16:41:50.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.687+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 15 ==== osd_op_reply(472 [pgnls-filter start_epoch 131 out=101b] v131'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede73440 con 0x564eedfcb000 2023-11-08T16:41:50.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.687+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000237.00000000 2023-11-08T16:41:50.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.687+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000237.0 2023-11-08T16:41:50.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.687+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:473 17.25 17:a53878a9:::10000000237.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed494700 con 0x564eedfcb000 2023-11-08T16:41:50.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.688+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 16 ==== osd_op_reply(473 10000000237.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede73440 con 0x564eedfcb000 2023-11-08T16:41:50.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.688+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000237 2023-11-08T16:41:50.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.688+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/66 2023-11-08T16:41:50.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.688+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=66 2023-11-08T16:41:50.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.688+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:474 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed494000 con 0x564eede60000 2023-11-08T16:41:50.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.689+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 68 ==== osd_op_reply(474 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.689+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.689+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:475 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eeddf6800 2023-11-08T16:41:50.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.690+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 167 ==== osd_op_reply(475 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.690+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.690+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.690+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.690+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:476 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eeddf6c00 2023-11-08T16:41:50.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.692+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 147 ==== osd_op_reply(476 10000000000.01000000 [omap-get-header out=274b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.692+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 42 2023-11-08T16:41:50.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.692+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.692+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:477 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ce00 con 0x564eeddf6c00 2023-11-08T16:41:50.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.693+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 148 ==== osd_op_reply(477 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.693+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=66 frag=0*, r=0 2023-11-08T16:41:50.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.693+0000 7f9189c365c0 20 datascan.read_dentry: 66_head not found in result 2023-11-08T16:41:50.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.693+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000237 at 0x0x10000000000/66 with size=2 bytes 2023-11-08T16:41:50.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.693+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:478 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d180 con 0x564eeddf6c00 2023-11-08T16:41:50.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.696+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 149 ==== osd_op_reply(478 10000000000.01000000 [omap-set-vals] v131'43 uv43 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.696+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/66 pointing to 0x0x10000000237 2023-11-08T16:41:50.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.696+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:479 17.25 17:a7ae8a0f::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d880 con 0x564eedfcb000 2023-11-08T16:41:50.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.697+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 17 ==== osd_op_reply(479 [pgnls-filter start_epoch 131 out=81b] v131'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede73440 con 0x564eedfcb000 2023-11-08T16:41:50.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.698+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000243.00000000 2023-11-08T16:41:50.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.698+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000243.0 2023-11-08T16:41:50.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.698+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:480 17.25 17:a7ae8a0f:::10000000243.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d500 con 0x564eedfcb000 2023-11-08T16:41:50.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.699+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 18 ==== osd_op_reply(480 10000000243.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede73440 con 0x564eedfcb000 2023-11-08T16:41:50.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.699+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000243 2023-11-08T16:41:50.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.699+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/78 2023-11-08T16:41:50.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.699+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=78 2023-11-08T16:41:50.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.699+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:481 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464e00 con 0x564eede60000 2023-11-08T16:41:50.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.700+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 69 ==== osd_op_reply(481 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.700+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.700+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:482 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464a80 con 0x564eeddf6800 2023-11-08T16:41:50.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.701+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 168 ==== osd_op_reply(482 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.701+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.701+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.701+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.701+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:483 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465880 con 0x564eeddf6800 2023-11-08T16:41:50.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.702+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 169 ==== osd_op_reply(483 10000000000.01800000 [omap-get-header out=274b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.702+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 33 2023-11-08T16:41:50.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.702+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.702+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:484 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465c00 con 0x564eeddf6800 2023-11-08T16:41:50.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.703+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 170 ==== osd_op_reply(484 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv33 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.703+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=78 frag=1*, r=0 2023-11-08T16:41:50.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.703+0000 7f9189c365c0 20 datascan.read_dentry: 78_head not found in result 2023-11-08T16:41:50.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.703+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000243 at 0x0x10000000000/78 with size=2 bytes 2023-11-08T16:41:50.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.703+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:485 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4700 con 0x564eeddf6800 2023-11-08T16:41:50.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.708+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 171 ==== osd_op_reply(485 10000000000.01800000 [omap-set-vals] v131'34 uv34 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.708+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/78 pointing to 0x0x10000000243 2023-11-08T16:41:50.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.708+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:486 17.15 17:a8000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4a80 con 0x564eeddf6800 2023-11-08T16:41:50.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.709+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 172 ==== osd_op_reply(486 [pgnls-filter start_epoch 131 out=81b] v131'9 uv9 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.709+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000020d.00000000 2023-11-08T16:41:50.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.709+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000020d.0 2023-11-08T16:41:50.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.709+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:487 17.15 17:a8e69ea8:::1000000020d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4380 con 0x564eeddf6800 2023-11-08T16:41:50.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.710+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 173 ==== osd_op_reply(487 1000000020d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv9 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.711+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020d 2023-11-08T16:41:50.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.711+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/24 2023-11-08T16:41:50.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.711+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=24 2023-11-08T16:41:50.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.711+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:488 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465180 con 0x564eede60000 2023-11-08T16:41:50.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.712+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 70 ==== osd_op_reply(488 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.712+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.712+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:489 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465500 con 0x564eeddf6800 2023-11-08T16:41:50.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.713+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 174 ==== osd_op_reply(489 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.713+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.713+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.713+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.713+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:490 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464700 con 0x564eeddf6800 2023-11-08T16:41:50.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.714+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 175 ==== osd_op_reply(490 10000000000.01800000 [omap-get-header out=274b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.714+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 34 2023-11-08T16:41:50.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.714+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.714+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:491 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464380 con 0x564eeddf6800 2023-11-08T16:41:50.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.715+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 176 ==== osd_op_reply(491 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv34 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.715+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=24 frag=1*, r=0 2023-11-08T16:41:50.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.715+0000 7f9189c365c0 20 datascan.read_dentry: 24_head not found in result 2023-11-08T16:41:50.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.715+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020d at 0x0x10000000000/24 with size=2 bytes 2023-11-08T16:41:50.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.716+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:492 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464a80 con 0x564eeddf6800 2023-11-08T16:41:50.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.719+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 177 ==== osd_op_reply(492 10000000000.01800000 [omap-set-vals] v131'35 uv35 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.719+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-11-08T16:41:50.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.719+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:493 17.35 17:ac000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465880 con 0x564eed426c00 2023-11-08T16:41:50.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.721+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 14 ==== osd_op_reply(493 [pgnls-filter start_epoch 131 out=49b] v127'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.721+0000 7f9182b39640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:494 17.d 17:b0000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eeddf6400 2023-11-08T16:41:50.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.722+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 14 ==== osd_op_reply(494 [pgnls-filter start_epoch 131 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.722+0000 7f918333a640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:495 17.2d 17:b4000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c700 con 0x564eeddf6400 2023-11-08T16:41:50.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.723+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 15 ==== osd_op_reply(495 [pgnls-filter start_epoch 131 out=81b] v131'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.724+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000218.00000000 2023-11-08T16:41:50.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.724+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000218.0 2023-11-08T16:41:50.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.724+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:496 17.2d 17:b79a54f6:::10000000218.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d500 con 0x564eeddf6400 2023-11-08T16:41:50.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.725+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 16 ==== osd_op_reply(496 10000000218.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.725+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000218 2023-11-08T16:41:50.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.725+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/35 2023-11-08T16:41:50.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.725+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=35 2023-11-08T16:41:50.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.725+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:497 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c380 con 0x564eede60000 2023-11-08T16:41:50.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.726+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 71 ==== osd_op_reply(497 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.726+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.726+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:498 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eeddf6800 2023-11-08T16:41:50.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.728+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 178 ==== osd_op_reply(498 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.728+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.728+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.728+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.728+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:499 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eeddf6800 2023-11-08T16:41:50.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.729+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 179 ==== osd_op_reply(499 10000000000.01800000 [omap-get-header out=274b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.729+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 35 2023-11-08T16:41:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.729+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.729+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:500 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405180 con 0x564eeddf6800 2023-11-08T16:41:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.730+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 180 ==== osd_op_reply(500 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv35 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.731+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=35 frag=1*, r=0 2023-11-08T16:41:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.731+0000 7f9189c365c0 20 datascan.read_dentry: 35_head not found in result 2023-11-08T16:41:50.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.731+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000218 at 0x0x10000000000/35 with size=2 bytes 2023-11-08T16:41:50.733 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.731+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:501 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6800 2023-11-08T16:41:50.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.735+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 181 ==== osd_op_reply(501 10000000000.01800000 [omap-set-vals] v131'36 uv36 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.735+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/35 pointing to 0x0x10000000218 2023-11-08T16:41:50.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.735+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:502 17.1d 17:b8000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6400 2023-11-08T16:41:50.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.736+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 17 ==== osd_op_reply(502 [pgnls-filter start_epoch 131 out=81b] v131'18 uv18 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.737+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000246.00000000 2023-11-08T16:41:50.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.737+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000246.0 2023-11-08T16:41:50.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.737+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:503 17.1d 17:bab97ecf:::10000000246.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed494700 con 0x564eeddf6400 2023-11-08T16:41:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.738+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 18 ==== osd_op_reply(503 10000000246.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv18 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.738+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000246 2023-11-08T16:41:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.738+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/81 2023-11-08T16:41:50.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.738+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=81 2023-11-08T16:41:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.738+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:504 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed494a80 con 0x564eede60000 2023-11-08T16:41:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.739+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 72 ==== osd_op_reply(504 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.739+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.739+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:505 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eeddf6800 2023-11-08T16:41:50.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.741+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 182 ==== osd_op_reply(505 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.741+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.741+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.741+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.741+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:506 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eeddf6800 2023-11-08T16:41:50.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.742+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 183 ==== osd_op_reply(506 10000000000.01800000 [omap-get-header out=274b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.742+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 36 2023-11-08T16:41:50.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.742+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.742+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:507 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2700 con 0x564eeddf6800 2023-11-08T16:41:50.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.743+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 184 ==== osd_op_reply(507 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv36 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.744+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=81 frag=1*, r=0 2023-11-08T16:41:50.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.744+0000 7f9189c365c0 20 datascan.read_dentry: 81_head not found in result 2023-11-08T16:41:50.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.744+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000246 at 0x0x10000000000/81 with size=2 bytes 2023-11-08T16:41:50.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.744+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:508 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6800 2023-11-08T16:41:50.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.748+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 185 ==== osd_op_reply(508 10000000000.01800000 [omap-set-vals] v131'37 uv37 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.748+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/81 pointing to 0x0x10000000246 2023-11-08T16:41:50.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.748+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:509 17.3d 17:bc000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eeddf6400 2023-11-08T16:41:50.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.749+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 19 ==== osd_op_reply(509 [pgnls-filter start_epoch 131 out=101b] v131'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.749+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000219.00000000 2023-11-08T16:41:50.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.749+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000219.0 2023-11-08T16:41:50.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.749+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:510 17.3d 17:bc0c3701:::10000000219.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eeddf6400 2023-11-08T16:41:50.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.751+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 20 ==== osd_op_reply(510 10000000219.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.751+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000219 2023-11-08T16:41:50.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.751+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/36 2023-11-08T16:41:50.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.751+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=36 2023-11-08T16:41:50.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.751+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:511 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c380 con 0x564eede60000 2023-11-08T16:41:50.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.752+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 73 ==== osd_op_reply(511 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.752+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.752+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:512 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c700 con 0x564eeddf6800 2023-11-08T16:41:50.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.753+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 186 ==== osd_op_reply(512 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.753+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.753+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.753+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.753+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:513 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ca80 con 0x564eeddf6c00 2023-11-08T16:41:50.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.754+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 150 ==== osd_op_reply(513 10000000000.01000000 [omap-get-header out=274b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.754+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 43 2023-11-08T16:41:50.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.754+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.754+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:514 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ce00 con 0x564eeddf6c00 2023-11-08T16:41:50.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.755+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 151 ==== osd_op_reply(514 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.755+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=36 frag=0*, r=0 2023-11-08T16:41:50.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.755+0000 7f9189c365c0 20 datascan.read_dentry: 36_head not found in result 2023-11-08T16:41:50.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.755+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000219 at 0x0x10000000000/36 with size=2 bytes 2023-11-08T16:41:50.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.755+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:515 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49d180 con 0x564eeddf6c00 2023-11-08T16:41:50.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.758+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 152 ==== osd_op_reply(515 10000000000.01000000 [omap-set-vals] v131'44 uv44 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.758+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/36 pointing to 0x0x10000000219 2023-11-08T16:41:50.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.758+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:516 17.3d 17:bfcd3380::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49d500 con 0x564eeddf6400 2023-11-08T16:41:50.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.760+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 21 ==== osd_op_reply(516 [pgnls-filter start_epoch 131 out=81b] v131'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.760+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000205.00000000 2023-11-08T16:41:50.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.760+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000205.0 2023-11-08T16:41:50.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.760+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:517 17.3d 17:bfcd3380:::10000000205.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49d880 con 0x564eeddf6400 2023-11-08T16:41:50.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.761+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 22 ==== osd_op_reply(517 10000000205.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.761+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000205 2023-11-08T16:41:50.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.762+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/16 2023-11-08T16:41:50.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.762+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=16 2023-11-08T16:41:50.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.762+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:518 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c000 con 0x564eede60000 2023-11-08T16:41:50.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.763+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 74 ==== osd_op_reply(518 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.763+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.763+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:519 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49d500 con 0x564eeddf6800 2023-11-08T16:41:50.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.764+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 187 ==== osd_op_reply(519 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.764+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.764+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.764+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.764+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:520 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49dc00 con 0x564eeddf6800 2023-11-08T16:41:50.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.765+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 188 ==== osd_op_reply(520 10000000000.01800000 [omap-get-header out=274b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.765+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 37 2023-11-08T16:41:50.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.765+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.765+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:521 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c380 con 0x564eeddf6800 2023-11-08T16:41:50.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.766+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 189 ==== osd_op_reply(521 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv37 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.766+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=16 frag=1*, r=0 2023-11-08T16:41:50.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.766+0000 7f9189c365c0 20 datascan.read_dentry: 16_head not found in result 2023-11-08T16:41:50.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.766+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000205 at 0x0x10000000000/16 with size=2 bytes 2023-11-08T16:41:50.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.766+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:522 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ca80 con 0x564eeddf6800 2023-11-08T16:41:50.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.770+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 190 ==== osd_op_reply(522 10000000000.01800000 [omap-set-vals] v131'38 uv38 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.770+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-11-08T16:41:50.772 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.770+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:523 17.3 17:c0000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464000 con 0x564eed426c00 2023-11-08T16:41:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.772+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 15 ==== osd_op_reply(523 [pgnls-filter start_epoch 131 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.773 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.772+0000 7f9182b39640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:524 17.23 17:c4000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eed426c00 2023-11-08T16:41:50.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.773+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 16 ==== osd_op_reply(524 [pgnls-filter start_epoch 131 out=49b] v127'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.773+0000 7f918333a640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:525 17.13 17:c8000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c000 con 0x564eedfcb800 2023-11-08T16:41:50.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.774+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 41 ==== osd_op_reply(525 [pgnls-filter start_epoch 131 out=101b] v131'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.774+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000023c.00000000 2023-11-08T16:41:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.774+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000023c.0 2023-11-08T16:41:50.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.774+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:526 17.13 17:c8893608:::1000000023c.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464e00 con 0x564eedfcb800 2023-11-08T16:41:50.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.775+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 42 ==== osd_op_reply(526 1000000023c.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv26 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.775+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000023c 2023-11-08T16:41:50.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.775+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/71 2023-11-08T16:41:50.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.775+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=71 2023-11-08T16:41:50.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.775+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:527 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465880 con 0x564eede60000 2023-11-08T16:41:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.776+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 75 ==== osd_op_reply(527 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.776+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.776+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:528 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6800 2023-11-08T16:41:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.777+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 191 ==== osd_op_reply(528 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.778+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.778+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.778+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.778+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:529 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6800 2023-11-08T16:41:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.779+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 192 ==== osd_op_reply(529 10000000000.01800000 [omap-get-header out=274b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.779+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 38 2023-11-08T16:41:50.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.779+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.779+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:530 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eeddf6800 2023-11-08T16:41:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.780+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 193 ==== osd_op_reply(530 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv38 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.781+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=71 frag=1*, r=0 2023-11-08T16:41:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.781+0000 7f9189c365c0 20 datascan.read_dentry: 71_head not found in result 2023-11-08T16:41:50.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.781+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000023c at 0x0x10000000000/71 with size=2 bytes 2023-11-08T16:41:50.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.781+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:531 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eeddf6800 2023-11-08T16:41:50.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.785+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 194 ==== osd_op_reply(531 10000000000.01800000 [omap-set-vals] v131'39 uv39 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.785+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/71 pointing to 0x0x1000000023c 2023-11-08T16:41:50.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.785+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:532 17.13 17:c933d8fa::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9880 con 0x564eedfcb800 2023-11-08T16:41:50.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.786+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 43 ==== osd_op_reply(532 [pgnls-filter start_epoch 131 out=101b] v131'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.786+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000241.00000000 2023-11-08T16:41:50.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.786+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000241.0 2023-11-08T16:41:50.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.786+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:533 17.13 17:c933d8fa:::10000000241.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3c00 con 0x564eedfcb800 2023-11-08T16:41:50.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.787+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 44 ==== osd_op_reply(533 10000000241.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv27 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.787+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000241 2023-11-08T16:41:50.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.787+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/76 2023-11-08T16:41:50.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.787+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=76 2023-11-08T16:41:50.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.787+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:534 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3880 con 0x564eede60000 2023-11-08T16:41:50.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.788+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 76 ==== osd_op_reply(534 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.788+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.788+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:535 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2000 con 0x564eeddf6800 2023-11-08T16:41:50.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.789+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 195 ==== osd_op_reply(535 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.789+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.789+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.789+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.789+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:536 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5c00 con 0x564eeddf6800 2023-11-08T16:41:50.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.791+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 196 ==== osd_op_reply(536 10000000000.01800000 [omap-get-header out=274b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.791+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 39 2023-11-08T16:41:50.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.791+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.791+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:537 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5500 con 0x564eeddf6800 2023-11-08T16:41:50.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.792+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 197 ==== osd_op_reply(537 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv39 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.792+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=76 frag=1*, r=0 2023-11-08T16:41:50.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.792+0000 7f9189c365c0 20 datascan.read_dentry: 76_head not found in result 2023-11-08T16:41:50.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.792+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000241 at 0x0x10000000000/76 with size=2 bytes 2023-11-08T16:41:50.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.792+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:538 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eeddf6800 2023-11-08T16:41:50.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.796+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 198 ==== osd_op_reply(538 10000000000.01800000 [omap-set-vals] v131'40 uv40 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.796+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/76 pointing to 0x0x10000000241 2023-11-08T16:41:50.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.796+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:539 17.13 17:c9855970::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5180 con 0x564eedfcb800 2023-11-08T16:41:50.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.797+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 45 ==== osd_op_reply(539 [pgnls-filter start_epoch 131 out=101b] v131'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.797+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000239.00000000 2023-11-08T16:41:50.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.797+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000239.0 2023-11-08T16:41:50.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.797+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:540 17.13 17:c9855970:::10000000239.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4000 con 0x564eedfcb800 2023-11-08T16:41:50.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.798+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 46 ==== osd_op_reply(540 10000000239.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv28 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.798+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000239 2023-11-08T16:41:50.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.798+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/68 2023-11-08T16:41:50.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.798+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=68 2023-11-08T16:41:50.801 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.798+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:541 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ce00 con 0x564eede60000 2023-11-08T16:41:50.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.799+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 77 ==== osd_op_reply(541 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.799+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.799+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:542 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d180 con 0x564eeddf6800 2023-11-08T16:41:50.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.800+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 199 ==== osd_op_reply(542 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.801+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.801+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.801+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.801+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:543 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d500 con 0x564eeddf6800 2023-11-08T16:41:50.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.802+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 200 ==== osd_op_reply(543 10000000000.01800000 [omap-get-header out=274b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.802+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 40 2023-11-08T16:41:50.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.802+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.802+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:544 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d880 con 0x564eeddf6800 2023-11-08T16:41:50.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.803+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 201 ==== osd_op_reply(544 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv40 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.803+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=68 frag=1*, r=0 2023-11-08T16:41:50.806 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.803+0000 7f9189c365c0 20 datascan.read_dentry: 68_head not found in result 2023-11-08T16:41:50.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.803+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000239 at 0x0x10000000000/68 with size=2 bytes 2023-11-08T16:41:50.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.803+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:545 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47dc00 con 0x564eeddf6800 2023-11-08T16:41:50.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.807+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 202 ==== osd_op_reply(545 10000000000.01800000 [omap-set-vals] v131'41 uv41 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.808+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/68 pointing to 0x0x10000000239 2023-11-08T16:41:50.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.808+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:546 17.13 17:ca8fdbba::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405880 con 0x564eedfcb800 2023-11-08T16:41:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.808+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 47 ==== osd_op_reply(546 [pgnls-filter start_epoch 131 out=101b] v131'30 uv30 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.809+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 100000001fb.00000000 2023-11-08T16:41:50.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.809+0000 7f9189c365c0 10 datascan.operator(): handling object 100000001fb.0 2023-11-08T16:41:50.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.809+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:547 17.13 17:ca8fdbba:::100000001fb.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404380 con 0x564eedfcb800 2023-11-08T16:41:50.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.810+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 48 ==== osd_op_reply(547 100000001fb.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv29 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.810+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fb 2023-11-08T16:41:50.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.810+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/6 2023-11-08T16:41:50.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.810+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=6 2023-11-08T16:41:50.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.810+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:548 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eede60000 2023-11-08T16:41:50.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.811+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 78 ==== osd_op_reply(548 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.811+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.811+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:549 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6800 2023-11-08T16:41:50.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.812+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 203 ==== osd_op_reply(549 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.812+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.812+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.812+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.812+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:550 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6800 2023-11-08T16:41:50.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.813+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 204 ==== osd_op_reply(550 10000000000.01800000 [omap-get-header out=274b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.813+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 41 2023-11-08T16:41:50.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.813+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.813+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:551 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c700 con 0x564eeddf6800 2023-11-08T16:41:50.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.815+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 205 ==== osd_op_reply(551 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv41 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.815+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=6 frag=1*, r=0 2023-11-08T16:41:50.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.815+0000 7f9189c365c0 20 datascan.read_dentry: 6_head not found in result 2023-11-08T16:41:50.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.815+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fb at 0x0x10000000000/6 with size=1 bytes 2023-11-08T16:41:50.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.815+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:552 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49d500 con 0x564eeddf6800 2023-11-08T16:41:50.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.819+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 206 ==== osd_op_reply(552 10000000000.01800000 [omap-set-vals] v131'42 uv42 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.819+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-11-08T16:41:50.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.819+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:553 17.13 17:cbcb6224::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49dc00 con 0x564eedfcb800 2023-11-08T16:41:50.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.820+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 49 ==== osd_op_reply(553 [pgnls-filter start_epoch 131 out=81b] v131'30 uv30 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.820+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000212.00000000 2023-11-08T16:41:50.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.820+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000212.0 2023-11-08T16:41:50.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.820+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:554 17.13 17:cbcb6224:::10000000212.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c380 con 0x564eedfcb800 2023-11-08T16:41:50.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.822+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.3 v2:172.21.15.81:6816/2756229107 50 ==== osd_op_reply(554 10000000212.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv30 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eedfcb800 2023-11-08T16:41:50.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.822+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000212 2023-11-08T16:41:50.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.822+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/29 2023-11-08T16:41:50.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.822+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=29 2023-11-08T16:41:50.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.822+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:555 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2a80 con 0x564eede60000 2023-11-08T16:41:50.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.823+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 79 ==== osd_op_reply(555 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.823+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.823+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:556 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eeddf6800 2023-11-08T16:41:50.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.824+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 207 ==== osd_op_reply(556 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.824+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.824+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.824+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.824+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:557 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eeddf6800 2023-11-08T16:41:50.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.826+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 208 ==== osd_op_reply(557 10000000000.01800000 [omap-get-header out=274b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.826+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 42 2023-11-08T16:41:50.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.826+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.826+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:558 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2700 con 0x564eeddf6800 2023-11-08T16:41:50.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.827+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 209 ==== osd_op_reply(558 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv42 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.827+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=29 frag=1*, r=0 2023-11-08T16:41:50.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.827+0000 7f9189c365c0 20 datascan.read_dentry: 29_head not found in result 2023-11-08T16:41:50.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.827+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000212 at 0x0x10000000000/29 with size=2 bytes 2023-11-08T16:41:50.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.827+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:559 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6800 2023-11-08T16:41:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.831+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 210 ==== osd_op_reply(559 10000000000.01800000 [omap-set-vals] v131'43 uv43 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.831+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/29 pointing to 0x0x10000000212 2023-11-08T16:41:50.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.831+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:560 17.33 17:cc000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465c00 con 0x564eeddf6400 2023-11-08T16:41:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.832+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 23 ==== osd_op_reply(560 [pgnls-filter start_epoch 131 out=81b] v131'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.832+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000248.00000000 2023-11-08T16:41:50.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.832+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000248.0 2023-11-08T16:41:50.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.832+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:561 17.33 17:cf9e7a4f:::10000000248.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465500 con 0x564eeddf6400 2023-11-08T16:41:50.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.834+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 <== osd.7 v2:172.21.15.93:6818/433691658 24 ==== osd_op_reply(561 10000000248.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72fc0 con 0x564eeddf6400 2023-11-08T16:41:50.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.834+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000248 2023-11-08T16:41:50.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.834+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/83 2023-11-08T16:41:50.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.834+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=83 2023-11-08T16:41:50.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.834+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:562 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464380 con 0x564eede60000 2023-11-08T16:41:50.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.835+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 80 ==== osd_op_reply(562 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.835+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.835+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:563 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464a80 con 0x564eeddf6800 2023-11-08T16:41:50.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.836+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 211 ==== osd_op_reply(563 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.836+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.836+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.836+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.836+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:564 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4700 con 0x564eeddf6c00 2023-11-08T16:41:50.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.837+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 153 ==== osd_op_reply(564 10000000000.01000000 [omap-get-header out=274b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.837+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 44 2023-11-08T16:41:50.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.837+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.837+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:565 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4a80 con 0x564eeddf6c00 2023-11-08T16:41:50.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.838+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 154 ==== osd_op_reply(565 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.838+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=83 frag=0*, r=0 2023-11-08T16:41:50.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.838+0000 7f9189c365c0 20 datascan.read_dentry: 83_head not found in result 2023-11-08T16:41:50.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.838+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000248 at 0x0x10000000000/83 with size=2 bytes 2023-11-08T16:41:50.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.838+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:566 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4380 con 0x564eeddf6c00 2023-11-08T16:41:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.842+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 155 ==== osd_op_reply(566 10000000000.01000000 [omap-set-vals] v131'45 uv45 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.842+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/83 pointing to 0x0x10000000248 2023-11-08T16:41:50.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.842+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:567 17.b 17:d0000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49e000 con 0x564eeddf6c00 2023-11-08T16:41:50.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.844+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 156 ==== osd_op_reply(567 [pgnls-filter start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.844+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000225.00000000 2023-11-08T16:41:50.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.844+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000225.0 2023-11-08T16:41:50.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.844+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:568 17.b 17:d0d26e50:::10000000225.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49e380 con 0x564eeddf6c00 2023-11-08T16:41:50.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.845+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 157 ==== osd_op_reply(568 10000000225.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.845+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000225 2023-11-08T16:41:50.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.845+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/48 2023-11-08T16:41:50.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.845+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=48 2023-11-08T16:41:50.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.845+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:569 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ea80 con 0x564eede60000 2023-11-08T16:41:50.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.846+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 81 ==== osd_op_reply(569 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.846+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.846+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:570 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ee00 con 0x564eeddf6800 2023-11-08T16:41:50.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.847+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 212 ==== osd_op_reply(570 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.848+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.848+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.848+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.848+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:571 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f180 con 0x564eeddf6800 2023-11-08T16:41:50.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.849+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 213 ==== osd_op_reply(571 10000000000.01800000 [omap-get-header out=274b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.849+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 43 2023-11-08T16:41:50.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.849+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.849+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:572 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f500 con 0x564eeddf6800 2023-11-08T16:41:50.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.850+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 214 ==== osd_op_reply(572 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv43 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.850+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=48 frag=1*, r=0 2023-11-08T16:41:50.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.850+0000 7f9189c365c0 20 datascan.read_dentry: 48_head not found in result 2023-11-08T16:41:50.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.850+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000225 at 0x0x10000000000/48 with size=2 bytes 2023-11-08T16:41:50.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.850+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:573 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f880 con 0x564eeddf6800 2023-11-08T16:41:50.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.854+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 215 ==== osd_op_reply(573 10000000000.01800000 [omap-set-vals] v131'44 uv44 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.854+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/48 pointing to 0x0x10000000225 2023-11-08T16:41:50.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.854+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:574 17.b 17:d2dbd4ac::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49fc00 con 0x564eeddf6c00 2023-11-08T16:41:50.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.855+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 158 ==== osd_op_reply(574 [pgnls-filter start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.855+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000217.00000000 2023-11-08T16:41:50.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.855+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000217.0 2023-11-08T16:41:50.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.855+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:575 17.b 17:d2dbd4ac:::10000000217.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0000 con 0x564eeddf6c00 2023-11-08T16:41:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.856+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 159 ==== osd_op_reply(575 10000000217.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.856+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000217 2023-11-08T16:41:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.856+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/34 2023-11-08T16:41:50.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.856+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=34 2023-11-08T16:41:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.856+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:576 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0700 con 0x564eede60000 2023-11-08T16:41:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.857+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 82 ==== osd_op_reply(576 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.857+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.857+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:577 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0a80 con 0x564eeddf6800 2023-11-08T16:41:50.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.859+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 216 ==== osd_op_reply(577 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.859+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.859+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.859+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.859+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:578 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0e00 con 0x564eeddf6c00 2023-11-08T16:41:50.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.860+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 160 ==== osd_op_reply(578 10000000000.01000000 [omap-get-header out=274b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.860+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 45 2023-11-08T16:41:50.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.860+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.860+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:579 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1180 con 0x564eeddf6c00 2023-11-08T16:41:50.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.861+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 161 ==== osd_op_reply(579 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.861+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=34 frag=0*, r=0 2023-11-08T16:41:50.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.861+0000 7f9189c365c0 20 datascan.read_dentry: 34_head not found in result 2023-11-08T16:41:50.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.861+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000217 at 0x0x10000000000/34 with size=2 bytes 2023-11-08T16:41:50.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.861+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:580 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1500 con 0x564eeddf6c00 2023-11-08T16:41:50.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.864+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 162 ==== osd_op_reply(580 10000000000.01000000 [omap-set-vals] v131'46 uv46 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.864+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/34 pointing to 0x0x10000000217 2023-11-08T16:41:50.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.864+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:581 17.b 17:d3eda40d::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1880 con 0x564eeddf6c00 2023-11-08T16:41:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.865+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 163 ==== osd_op_reply(581 [pgnls-filter start_epoch 131 out=81b] v131'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.865+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000020f.00000000 2023-11-08T16:41:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.865+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000020f.0 2023-11-08T16:41:50.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.865+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:582 17.b 17:d3eda40d:::1000000020f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1c00 con 0x564eeddf6c00 2023-11-08T16:41:50.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.866+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 164 ==== osd_op_reply(582 1000000020f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.866+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000020f 2023-11-08T16:41:50.868 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.866+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/26 2023-11-08T16:41:50.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.866+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=26 2023-11-08T16:41:50.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.866+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:583 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0000 con 0x564eede60000 2023-11-08T16:41:50.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.867+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 83 ==== osd_op_reply(583 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.867+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.867+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:584 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0700 con 0x564eeddf6800 2023-11-08T16:41:50.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.868+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 217 ==== osd_op_reply(584 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.869+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.869+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.869+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.869+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:585 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0e00 con 0x564eeddf6800 2023-11-08T16:41:50.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.870+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 218 ==== osd_op_reply(585 10000000000.01800000 [omap-get-header out=274b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.870+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 44 2023-11-08T16:41:50.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.870+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.870+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:586 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1180 con 0x564eeddf6800 2023-11-08T16:41:50.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.871+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 219 ==== osd_op_reply(586 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv44 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.871+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=26 frag=1*, r=0 2023-11-08T16:41:50.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.871+0000 7f9189c365c0 20 datascan.read_dentry: 26_head not found in result 2023-11-08T16:41:50.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.871+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000020f at 0x0x10000000000/26 with size=2 bytes 2023-11-08T16:41:50.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.871+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:587 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1500 con 0x564eeddf6800 2023-11-08T16:41:50.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.876+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 220 ==== osd_op_reply(587 10000000000.01800000 [omap-set-vals] v131'45 uv45 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.876+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/26 pointing to 0x0x1000000020f 2023-11-08T16:41:50.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.876+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:588 17.2b 17:d4000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49e000 con 0x564eed426c00 2023-11-08T16:41:50.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.877+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 17 ==== osd_op_reply(588 [pgnls-filter start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.877+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000203.00000000 2023-11-08T16:41:50.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.877+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000203.0 2023-11-08T16:41:50.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.877+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:589 17.2b 17:d4a58307:::10000000203.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49e700 con 0x564eed426c00 2023-11-08T16:41:50.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.878+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 18 ==== osd_op_reply(589 10000000203.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.878+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000203 2023-11-08T16:41:50.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.878+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/14 2023-11-08T16:41:50.880 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.878+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=14 2023-11-08T16:41:50.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.878+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:590 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ea80 con 0x564eede60000 2023-11-08T16:41:50.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.879+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 84 ==== osd_op_reply(590 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.881 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.879+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.879+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:591 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49fc00 con 0x564eeddf6800 2023-11-08T16:41:50.882 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.881+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 221 ==== osd_op_reply(591 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.881+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.881+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.883 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.881+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.881+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:592 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4700 con 0x564eeddf6800 2023-11-08T16:41:50.884 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.882+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 222 ==== osd_op_reply(592 10000000000.01800000 [omap-get-header out=274b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.882+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 45 2023-11-08T16:41:50.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.882+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.885 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.882+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:593 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4a80 con 0x564eeddf6800 2023-11-08T16:41:50.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.883+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 223 ==== osd_op_reply(593 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv45 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.886 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.884+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=14 frag=1*, r=0 2023-11-08T16:41:50.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.884+0000 7f9189c365c0 20 datascan.read_dentry: 14_head not found in result 2023-11-08T16:41:50.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.884+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000203 at 0x0x10000000000/14 with size=2 bytes 2023-11-08T16:41:50.887 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.884+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:594 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4380 con 0x564eeddf6800 2023-11-08T16:41:50.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.888+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 224 ==== osd_op_reply(594 10000000000.01800000 [omap-set-vals] v131'46 uv46 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.888+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-11-08T16:41:50.889 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.888+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:595 17.2b 17:d5ee56ee::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464000 con 0x564eed426c00 2023-11-08T16:41:50.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.889+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 19 ==== osd_op_reply(595 [pgnls-filter start_epoch 131 out=81b] v131'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.889+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000022d.00000000 2023-11-08T16:41:50.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.889+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000022d.0 2023-11-08T16:41:50.891 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.889+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:596 17.2b 17:d5ee56ee:::1000000022d.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465880 con 0x564eed426c00 2023-11-08T16:41:50.892 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.890+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 20 ==== osd_op_reply(596 1000000022d.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.890+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022d 2023-11-08T16:41:50.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.890+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/56 2023-11-08T16:41:50.893 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.890+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=56 2023-11-08T16:41:50.894 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.890+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:597 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3880 con 0x564eede60000 2023-11-08T16:41:50.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.891+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 85 ==== osd_op_reply(597 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.891+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.891+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:598 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2a80 con 0x564eeddf6800 2023-11-08T16:41:50.895 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.893+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 225 ==== osd_op_reply(598 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.893+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.893+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.893+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.896 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.893+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:599 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eeddf6800 2023-11-08T16:41:50.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.894+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 226 ==== osd_op_reply(599 10000000000.01800000 [omap-get-header out=274b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.897 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.894+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 46 2023-11-08T16:41:50.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.894+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.894+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:600 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6800 2023-11-08T16:41:50.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.895+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 227 ==== osd_op_reply(600 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.898 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.896+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=56 frag=1*, r=0 2023-11-08T16:41:50.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.896+0000 7f9189c365c0 20 datascan.read_dentry: 56_head not found in result 2023-11-08T16:41:50.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.896+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022d at 0x0x10000000000/56 with size=2 bytes 2023-11-08T16:41:50.899 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.896+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:601 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ce00 con 0x564eeddf6800 2023-11-08T16:41:50.901 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.900+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 228 ==== osd_op_reply(601 10000000000.01800000 [omap-set-vals] v131'47 uv47 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.900+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/56 pointing to 0x0x1000000022d 2023-11-08T16:41:50.902 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.900+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:602 17.1b 17:d8000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ce00 con 0x564eed426c00 2023-11-08T16:41:50.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.901+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 21 ==== osd_op_reply(602 [pgnls-filter start_epoch 131 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.903 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.901+0000 7f918333a640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:603 17.3b 17:dc000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c380 con 0x564eede60000 2023-11-08T16:41:50.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.902+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 86 ==== osd_op_reply(603 [pgnls-filter start_epoch 131 out=101b] v131'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.902+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000249.00000000 2023-11-08T16:41:50.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.902+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000249.0 2023-11-08T16:41:50.904 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.902+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:604 17.3b 17:dd50df43:::10000000249.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49d180 con 0x564eede60000 2023-11-08T16:41:50.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.903+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 87 ==== osd_op_reply(604 10000000249.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.904+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000249 2023-11-08T16:41:50.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.904+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/84 2023-11-08T16:41:50.905 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.904+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=84 2023-11-08T16:41:50.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.904+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:605 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0a80 con 0x564eede60000 2023-11-08T16:41:50.906 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.904+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 88 ==== osd_op_reply(605 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.905+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.905+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:606 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0700 con 0x564eeddf6800 2023-11-08T16:41:50.907 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.906+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 229 ==== osd_op_reply(606 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.906+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.906+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.908 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.906+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.909 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.906+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:607 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ee00 con 0x564eeddf6c00 2023-11-08T16:41:50.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.907+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 165 ==== osd_op_reply(607 10000000000.01000000 [omap-get-header out=274b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.907+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 46 2023-11-08T16:41:50.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.907+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.907+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:608 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f180 con 0x564eeddf6c00 2023-11-08T16:41:50.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.908+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 166 ==== osd_op_reply(608 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv46 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.908+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=84 frag=0*, r=0 2023-11-08T16:41:50.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.908+0000 7f9189c365c0 20 datascan.read_dentry: 84_head not found in result 2023-11-08T16:41:50.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.908+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000249 at 0x0x10000000000/84 with size=2 bytes 2023-11-08T16:41:50.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.908+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:609 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f500 con 0x564eeddf6c00 2023-11-08T16:41:50.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.911+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 167 ==== osd_op_reply(609 10000000000.01000000 [omap-set-vals] v131'47 uv47 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.912+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/84 pointing to 0x0x10000000249 2023-11-08T16:41:50.914 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.912+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:610 17.3b 17:deccbd37::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f880 con 0x564eede60000 2023-11-08T16:41:50.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.913+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 89 ==== osd_op_reply(610 [pgnls-filter start_epoch 131 out=101b] v131'13 uv13 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.915 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.913+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 100000001fe.00000000 2023-11-08T16:41:50.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.913+0000 7f9189c365c0 10 datascan.operator(): handling object 100000001fe.0 2023-11-08T16:41:50.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.913+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:611 17.3b 17:deccbd37:::100000001fe.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464a80 con 0x564eede60000 2023-11-08T16:41:50.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.914+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 90 ==== osd_op_reply(611 100000001fe.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.914+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fe 2023-11-08T16:41:50.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.914+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/9 2023-11-08T16:41:50.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.914+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=9 2023-11-08T16:41:50.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.914+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:612 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eede60000 2023-11-08T16:41:50.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.915+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 91 ==== osd_op_reply(612 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.915+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.915+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:613 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eeddf6800 2023-11-08T16:41:50.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.916+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 230 ==== osd_op_reply(613 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.917+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.917+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.917+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.917+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:614 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2700 con 0x564eeddf6c00 2023-11-08T16:41:50.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.918+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 168 ==== osd_op_reply(614 10000000000.01000000 [omap-get-header out=274b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.918+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 47 2023-11-08T16:41:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.918+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.918+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:615 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6c00 2023-11-08T16:41:50.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.919+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 169 ==== osd_op_reply(615 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.919+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=9 frag=0*, r=0 2023-11-08T16:41:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.919+0000 7f9189c365c0 20 datascan.read_dentry: 9_head not found in result 2023-11-08T16:41:50.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.919+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fe at 0x0x10000000000/9 with size=1 bytes 2023-11-08T16:41:50.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.919+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:616 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c700 con 0x564eeddf6c00 2023-11-08T16:41:50.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.923+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 170 ==== osd_op_reply(616 10000000000.01000000 [omap-set-vals] v131'48 uv48 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.923+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-11-08T16:41:50.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.923+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:617 17.3b 17:ded14b84::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49d500 con 0x564eede60000 2023-11-08T16:41:50.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.924+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 92 ==== osd_op_reply(617 [pgnls-filter start_epoch 131 out=81b] v131'13 uv13 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.924+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000024b.00000000 2023-11-08T16:41:50.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.924+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000024b.0 2023-11-08T16:41:50.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.924+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:618 17.3b 17:ded14b84:::1000000024b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eede60000 2023-11-08T16:41:50.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.925+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 93 ==== osd_op_reply(618 1000000024b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.926+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024b 2023-11-08T16:41:50.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.926+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/86 2023-11-08T16:41:50.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.926+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=86 2023-11-08T16:41:50.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.926+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:619 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eede60000 2023-11-08T16:41:50.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.926+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 94 ==== osd_op_reply(619 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.927+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.927+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:620 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d180 con 0x564eeddf6800 2023-11-08T16:41:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.928+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 231 ==== osd_op_reply(620 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.928+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.928+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.928+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.928+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:621 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d500 con 0x564eeddf6c00 2023-11-08T16:41:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.929+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 171 ==== osd_op_reply(621 10000000000.01000000 [omap-get-header out=274b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.929+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 48 2023-11-08T16:41:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.929+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.929+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:622 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d880 con 0x564eeddf6c00 2023-11-08T16:41:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.930+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 172 ==== osd_op_reply(622 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.930+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=86 frag=0*, r=0 2023-11-08T16:41:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.930+0000 7f9189c365c0 20 datascan.read_dentry: 86_head not found in result 2023-11-08T16:41:50.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.930+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024b at 0x0x10000000000/86 with size=2 bytes 2023-11-08T16:41:50.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.930+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:623 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47dc00 con 0x564eeddf6c00 2023-11-08T16:41:50.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.934+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 173 ==== osd_op_reply(623 10000000000.01000000 [omap-set-vals] v131'49 uv49 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.934+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/86 pointing to 0x0x1000000024b 2023-11-08T16:41:50.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.934+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:624 17.7 17:e0000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5c00 con 0x564eeddf6c00 2023-11-08T16:41:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.935+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 174 ==== osd_op_reply(624 [pgnls-filter start_epoch 131 out=101b] v131'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.935+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000207.00000000 2023-11-08T16:41:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.935+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000207.0 2023-11-08T16:41:50.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.935+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:625 17.7 17:e15a914f:::10000000207.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5500 con 0x564eeddf6c00 2023-11-08T16:41:50.939 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.936+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 175 ==== osd_op_reply(625 10000000207.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.936+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000207 2023-11-08T16:41:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.937+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/18 2023-11-08T16:41:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.937+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=18 2023-11-08T16:41:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.937+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:626 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eede60000 2023-11-08T16:41:50.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.938+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 95 ==== osd_op_reply(626 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.938+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.938+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:627 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3880 con 0x564eeddf6800 2023-11-08T16:41:50.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.939+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 232 ==== osd_op_reply(627 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.939+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.939+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.939+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.939+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:628 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2000 con 0x564eeddf6800 2023-11-08T16:41:50.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.940+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 233 ==== osd_op_reply(628 10000000000.01800000 [omap-get-header out=274b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.940+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 47 2023-11-08T16:41:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.940+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.940+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:629 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3500 con 0x564eeddf6800 2023-11-08T16:41:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.942+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 234 ==== osd_op_reply(629 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv47 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.942+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=18 frag=1*, r=0 2023-11-08T16:41:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.942+0000 7f9189c365c0 20 datascan.read_dentry: 18_head not found in result 2023-11-08T16:41:50.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.942+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000207 at 0x0x10000000000/18 with size=2 bytes 2023-11-08T16:41:50.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.942+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:630 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2700 con 0x564eeddf6800 2023-11-08T16:41:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.946+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 235 ==== osd_op_reply(630 10000000000.01800000 [omap-set-vals] v131'48 uv48 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.947 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.946+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-11-08T16:41:50.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.946+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:631 17.7 17:e2e07930::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6c00 2023-11-08T16:41:50.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.947+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 176 ==== osd_op_reply(631 [pgnls-filter start_epoch 131 out=101b] v131'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.947+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-11-08T16:41:50.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.947+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000001.0 2023-11-08T16:41:50.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.947+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:632 17.7 17:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464000 con 0x564eeddf6c00 2023-11-08T16:41:50.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.948+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 177 ==== osd_op_reply(632 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.948+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-11-08T16:41:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.948+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/0 2023-11-08T16:41:50.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.948+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=0 2023-11-08T16:41:50.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.948+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:633 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465880 con 0x564eede60000 2023-11-08T16:41:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.949+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 96 ==== osd_op_reply(633 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.949+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.949+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:634 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464a80 con 0x564eeddf6800 2023-11-08T16:41:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.950+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 236 ==== osd_op_reply(634 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.951+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.951+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.951+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:50.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.951+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:635 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49e000 con 0x564eeddf6800 2023-11-08T16:41:50.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.952+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 237 ==== osd_op_reply(635 10000000000.01800000 [omap-get-header out=274b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.952+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 48 2023-11-08T16:41:50.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.952+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:50.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.952+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:636 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49e380 con 0x564eeddf6800 2023-11-08T16:41:50.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.953+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 238 ==== osd_op_reply(636 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv48 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.953+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=0 frag=1*, r=0 2023-11-08T16:41:50.958 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.953+0000 7f9189c365c0 20 datascan.read_dentry: 0_head not found in result 2023-11-08T16:41:50.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.953+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/0 with size=1 bytes 2023-11-08T16:41:50.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.953+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:637 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49e700 con 0x564eeddf6800 2023-11-08T16:41:50.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.957+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 239 ==== osd_op_reply(637 10000000000.01800000 [omap-set-vals] v131'49 uv49 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.957+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-11-08T16:41:50.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.957+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:638 17.7 17:e37752f2::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ea80 con 0x564eeddf6c00 2023-11-08T16:41:50.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.958+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 178 ==== osd_op_reply(638 [pgnls-filter start_epoch 131 out=81b] v131'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.958+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000021f.00000000 2023-11-08T16:41:50.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.958+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000021f.0 2023-11-08T16:41:50.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.958+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:639 17.7 17:e37752f2:::1000000021f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ee00 con 0x564eeddf6c00 2023-11-08T16:41:50.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.959+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 179 ==== osd_op_reply(639 1000000021f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.959+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021f 2023-11-08T16:41:50.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.959+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/42 2023-11-08T16:41:50.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.959+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=42 2023-11-08T16:41:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.959+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:640 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f500 con 0x564eede60000 2023-11-08T16:41:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.960+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 97 ==== osd_op_reply(640 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.960+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.960+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:641 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f880 con 0x564eeddf6800 2023-11-08T16:41:50.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.961+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 240 ==== osd_op_reply(641 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.962+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.962+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.962+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.962+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:642 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1880 con 0x564eeddf6c00 2023-11-08T16:41:50.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.962+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 180 ==== osd_op_reply(642 10000000000.01000000 [omap-get-header out=274b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.963+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 49 2023-11-08T16:41:50.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.963+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.963+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:643 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0380 con 0x564eeddf6c00 2023-11-08T16:41:50.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.964+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 181 ==== osd_op_reply(643 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.964+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=42 frag=0*, r=0 2023-11-08T16:41:50.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.964+0000 7f9189c365c0 20 datascan.read_dentry: 42_head not found in result 2023-11-08T16:41:50.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.964+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021f at 0x0x10000000000/42 with size=2 bytes 2023-11-08T16:41:50.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.964+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:644 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1c00 con 0x564eeddf6c00 2023-11-08T16:41:50.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.968+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 182 ==== osd_op_reply(644 10000000000.01000000 [omap-set-vals] v131'50 uv50 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.968+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/42 pointing to 0x0x1000000021f 2023-11-08T16:41:50.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.968+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:645 17.27 17:e4000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0000 con 0x564eed426c00 2023-11-08T16:41:50.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.969+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 22 ==== osd_op_reply(645 [pgnls-filter start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.969+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000210.00000000 2023-11-08T16:41:50.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.969+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000210.0 2023-11-08T16:41:50.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.969+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:646 17.27 17:e40c9c52:::10000000210.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0a80 con 0x564eed426c00 2023-11-08T16:41:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.970+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 23 ==== osd_op_reply(646 10000000210.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.970+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000210 2023-11-08T16:41:50.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.970+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/27 2023-11-08T16:41:50.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.970+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=27 2023-11-08T16:41:50.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.970+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:647 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c000 con 0x564eede60000 2023-11-08T16:41:50.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.971+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 98 ==== osd_op_reply(647 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.971+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.971+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:648 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49d180 con 0x564eeddf6800 2023-11-08T16:41:50.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.972+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 241 ==== osd_op_reply(648 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.973+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.973+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.973+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.973+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:649 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c380 con 0x564eeddf6c00 2023-11-08T16:41:50.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.974+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 183 ==== osd_op_reply(649 10000000000.01000000 [omap-get-header out=274b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.974+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 50 2023-11-08T16:41:50.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.974+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.974+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:650 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c700 con 0x564eeddf6c00 2023-11-08T16:41:50.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.975+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 184 ==== osd_op_reply(650 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.975+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=27 frag=0*, r=0 2023-11-08T16:41:50.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.975+0000 7f9189c365c0 20 datascan.read_dentry: 27_head not found in result 2023-11-08T16:41:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.975+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000210 at 0x0x10000000000/27 with size=2 bytes 2023-11-08T16:41:50.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.975+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:651 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49d500 con 0x564eeddf6c00 2023-11-08T16:41:50.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.979+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 185 ==== osd_op_reply(651 10000000000.01000000 [omap-set-vals] v131'51 uv51 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.979+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/27 pointing to 0x0x10000000210 2023-11-08T16:41:50.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.979+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:652 17.27 17:e5671758::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ce00 con 0x564eed426c00 2023-11-08T16:41:50.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.980+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 24 ==== osd_op_reply(652 [pgnls-filter start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.980+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000024f.00000000 2023-11-08T16:41:50.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.980+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000024f.0 2023-11-08T16:41:50.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.980+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:653 17.27 17:e5671758:::1000000024f.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c000 con 0x564eed426c00 2023-11-08T16:41:50.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.981+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 25 ==== osd_op_reply(653 1000000024f.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.981+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000024f 2023-11-08T16:41:50.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.981+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/90 2023-11-08T16:41:50.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.981+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=90 2023-11-08T16:41:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.981+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:654 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0380 con 0x564eede60000 2023-11-08T16:41:50.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.982+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 99 ==== osd_op_reply(654 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:50.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.982+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:50.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.982+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:655 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1c00 con 0x564eeddf6800 2023-11-08T16:41:50.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.984+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 242 ==== osd_op_reply(655 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:50.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.984+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:50.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.984+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:50.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.984+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:50.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.984+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:656 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0000 con 0x564eeddf6c00 2023-11-08T16:41:50.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.985+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 186 ==== osd_op_reply(656 10000000000.01000000 [omap-get-header out=274b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.985+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 51 2023-11-08T16:41:50.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.985+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:50.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.985+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:657 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0a80 con 0x564eeddf6c00 2023-11-08T16:41:50.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.986+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 187 ==== osd_op_reply(657 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.986+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=90 frag=0*, r=0 2023-11-08T16:41:50.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.986+0000 7f9189c365c0 20 datascan.read_dentry: 90_head not found in result 2023-11-08T16:41:50.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.986+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000024f at 0x0x10000000000/90 with size=2 bytes 2023-11-08T16:41:50.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.986+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:658 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ea80 con 0x564eeddf6c00 2023-11-08T16:41:50.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.989+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 188 ==== osd_op_reply(658 10000000000.01000000 [omap-set-vals] v131'52 uv52 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:50.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.990+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/90 pointing to 0x0x1000000024f 2023-11-08T16:41:50.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.990+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:659 17.27 17:e5993619::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f180 con 0x564eed426c00 2023-11-08T16:41:50.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.991+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 26 ==== osd_op_reply(659 [pgnls-filter start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.991+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000208.00000000 2023-11-08T16:41:50.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.991+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000208.0 2023-11-08T16:41:50.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.991+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:660 17.27 17:e5993619:::10000000208.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ee00 con 0x564eed426c00 2023-11-08T16:41:50.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.992+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 27 ==== osd_op_reply(660 10000000208.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:50.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.992+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000208 2023-11-08T16:41:50.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.992+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/19 2023-11-08T16:41:50.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.992+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=19 2023-11-08T16:41:50.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.992+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:661 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464380 con 0x564eede60000 2023-11-08T16:41:50.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.993+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 100 ==== osd_op_reply(661 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:51.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.993+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:51.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.993+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:662 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464000 con 0x564eeddf6800 2023-11-08T16:41:51.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.994+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 243 ==== osd_op_reply(662 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.994+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:51.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.995+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:51.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.995+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:51.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.995+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:663 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465880 con 0x564eeddf6c00 2023-11-08T16:41:51.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.995+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 189 ==== osd_op_reply(663 10000000000.01000000 [omap-get-header out=274b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.996+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 52 2023-11-08T16:41:51.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.996+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:51.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.996+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:664 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6c00 2023-11-08T16:41:51.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.997+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 190 ==== osd_op_reply(664 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.997+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=19 frag=0*, r=0 2023-11-08T16:41:51.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.997+0000 7f9189c365c0 20 datascan.read_dentry: 19_head not found in result 2023-11-08T16:41:51.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.997+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000208 at 0x0x10000000000/19 with size=2 bytes 2023-11-08T16:41:51.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:50.997+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:665 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eeddf6c00 2023-11-08T16:41:51.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.001+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 191 ==== osd_op_reply(665 10000000000.01000000 [omap-set-vals] v131'53 uv53 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.001+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-11-08T16:41:51.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.001+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:666 17.27 17:e5a5749e::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5c00 con 0x564eed426c00 2023-11-08T16:41:51.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.002+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 28 ==== osd_op_reply(666 [pgnls-filter start_epoch 131 out=101b] v131'17 uv17 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:51.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.002+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000021a.00000000 2023-11-08T16:41:51.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.002+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000021a.0 2023-11-08T16:41:51.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.002+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:667 17.27 17:e5a5749e:::1000000021a.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eed426c00 2023-11-08T16:41:51.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.003+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 29 ==== osd_op_reply(667 1000000021a.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv16 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:51.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.003+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021a 2023-11-08T16:41:51.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.003+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/37 2023-11-08T16:41:51.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.003+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=37 2023-11-08T16:41:51.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.003+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:668 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d500 con 0x564eede60000 2023-11-08T16:41:51.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.004+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 101 ==== osd_op_reply(668 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:51.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.004+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:51.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.004+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:669 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d880 con 0x564eeddf6800 2023-11-08T16:41:51.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.005+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 244 ==== osd_op_reply(669 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.005+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:51.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.006+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:51.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.006+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:51.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.006+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:670 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47dc00 con 0x564eeddf6800 2023-11-08T16:41:51.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.007+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 245 ==== osd_op_reply(670 10000000000.01800000 [omap-get-header out=274b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.007+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 49 2023-11-08T16:41:51.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.007+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:51.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.007+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:671 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404a80 con 0x564eeddf6800 2023-11-08T16:41:51.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.008+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 246 ==== osd_op_reply(671 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv49 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.008+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=37 frag=1*, r=0 2023-11-08T16:41:51.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.008+0000 7f9189c365c0 20 datascan.read_dentry: 37_head not found in result 2023-11-08T16:41:51.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.008+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021a at 0x0x10000000000/37 with size=2 bytes 2023-11-08T16:41:51.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.008+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:672 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed405c00 con 0x564eeddf6800 2023-11-08T16:41:51.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.012+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 247 ==== osd_op_reply(672 10000000000.01800000 [omap-set-vals] v131'50 uv50 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.012+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/37 pointing to 0x0x1000000021a 2023-11-08T16:41:51.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.012+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:673 17.27 17:e692fc64::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eed426c00 2023-11-08T16:41:51.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.013+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 30 ==== osd_op_reply(673 [pgnls-filter start_epoch 131 out=81b] v131'17 uv17 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:51.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.013+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000021b.00000000 2023-11-08T16:41:51.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.013+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000021b.0 2023-11-08T16:41:51.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.013+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:674 17.27 17:e692fc64:::1000000021b.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464a80 con 0x564eed426c00 2023-11-08T16:41:51.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.015+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 31 ==== osd_op_reply(674 1000000021b.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv17 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:51.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.015+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000021b 2023-11-08T16:41:51.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.015+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/38 2023-11-08T16:41:51.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.015+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=38 2023-11-08T16:41:51.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.015+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:675 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0e00 con 0x564eede60000 2023-11-08T16:41:51.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.016+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 102 ==== osd_op_reply(675 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:51.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.016+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:51.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.016+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:676 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1180 con 0x564eeddf6800 2023-11-08T16:41:51.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.017+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 248 ==== osd_op_reply(676 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.017+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:51.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.017+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:51.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.017+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:51.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.017+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:677 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1500 con 0x564eeddf6c00 2023-11-08T16:41:51.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.018+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 192 ==== osd_op_reply(677 10000000000.01000000 [omap-get-header out=274b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.018+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 53 2023-11-08T16:41:51.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.018+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:51.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.018+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:678 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0700 con 0x564eeddf6c00 2023-11-08T16:41:51.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.019+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 193 ==== osd_op_reply(678 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.019+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=38 frag=0*, r=0 2023-11-08T16:41:51.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.019+0000 7f9189c365c0 20 datascan.read_dentry: 38_head not found in result 2023-11-08T16:41:51.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.019+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000021b at 0x0x10000000000/38 with size=2 bytes 2023-11-08T16:41:51.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.019+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:679 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1c00 con 0x564eeddf6c00 2023-11-08T16:41:51.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.023+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 194 ==== osd_op_reply(679 10000000000.01000000 [omap-set-vals] v131'54 uv54 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.023+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/38 pointing to 0x0x1000000021b 2023-11-08T16:41:51.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.023+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:680 17.17 17:e8000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49d180 con 0x564eed427800 2023-11-08T16:41:51.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.024+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 13 ==== osd_op_reply(680 [pgnls-filter start_epoch 131 out=101b] v131'15 uv15 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:51.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.024+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000215.00000000 2023-11-08T16:41:51.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.024+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000215.0 2023-11-08T16:41:51.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.024+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:681 17.17 17:eace0b58:::10000000215.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49fc00 con 0x564eed427800 2023-11-08T16:41:51.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.026+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 14 ==== osd_op_reply(681 10000000215.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:51.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.026+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000215 2023-11-08T16:41:51.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.026+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/32 2023-11-08T16:41:51.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.026+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=32 2023-11-08T16:41:51.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.026+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:682 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49e380 con 0x564eede60000 2023-11-08T16:41:51.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.027+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 103 ==== osd_op_reply(682 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:51.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.027+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:51.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.027+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:683 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49e700 con 0x564eeddf6800 2023-11-08T16:41:51.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.028+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 249 ==== osd_op_reply(683 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.028+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:51.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.028+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:51.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.028+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:51.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.028+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:684 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f880 con 0x564eeddf6800 2023-11-08T16:41:51.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.029+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 250 ==== osd_op_reply(684 10000000000.01800000 [omap-get-header out=274b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.029+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 50 2023-11-08T16:41:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.029+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:51.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.029+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:685 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2a80 con 0x564eeddf6800 2023-11-08T16:41:51.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.031+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 251 ==== osd_op_reply(685 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv50 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.031+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=32 frag=1*, r=0 2023-11-08T16:41:51.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.031+0000 7f9189c365c0 20 datascan.read_dentry: 32_head not found in result 2023-11-08T16:41:51.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.031+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000215 at 0x0x10000000000/32 with size=2 bytes 2023-11-08T16:41:51.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.031+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:686 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2380 con 0x564eeddf6800 2023-11-08T16:41:51.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.034+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 252 ==== osd_op_reply(686 10000000000.01800000 [omap-set-vals] v131'51 uv51 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.035+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/32 pointing to 0x0x10000000215 2023-11-08T16:41:51.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.035+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:687 17.17 17:eb1b33a4::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3880 con 0x564eed427800 2023-11-08T16:41:51.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.036+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 15 ==== osd_op_reply(687 [pgnls-filter start_epoch 131 out=81b] v131'15 uv15 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:51.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.036+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000253.00000000 2023-11-08T16:41:51.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.036+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000253.0 2023-11-08T16:41:51.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.036+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:688 17.17 17:eb1b33a4:::10000000253.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2000 con 0x564eed427800 2023-11-08T16:41:51.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.037+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 16 ==== osd_op_reply(688 10000000253.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv15 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:51.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.037+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000253 2023-11-08T16:41:51.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.037+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/94 2023-11-08T16:41:51.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.037+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=94 2023-11-08T16:41:51.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.037+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:689 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb2700 con 0x564eede60000 2023-11-08T16:41:51.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.038+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 104 ==== osd_op_reply(689 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:51.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.038+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:51.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.038+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:690 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ce00 con 0x564eeddf6800 2023-11-08T16:41:51.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.039+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 253 ==== osd_op_reply(690 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.039+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:51.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.039+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:51.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.039+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:51.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.039+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:691 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d180 con 0x564eeddf6800 2023-11-08T16:41:51.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.040+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 254 ==== osd_op_reply(691 10000000000.01800000 [omap-get-header out=274b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.040+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 51 2023-11-08T16:41:51.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.040+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:51.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.040+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:692 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404e00 con 0x564eeddf6800 2023-11-08T16:41:51.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.041+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 255 ==== osd_op_reply(692 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv51 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.042+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=94 frag=1*, r=0 2023-11-08T16:41:51.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.042+0000 7f9189c365c0 20 datascan.read_dentry: 94_head not found in result 2023-11-08T16:41:51.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.042+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000253 at 0x0x10000000000/94 with size=2 bytes 2023-11-08T16:41:51.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.042+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:693 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404000 con 0x564eeddf6800 2023-11-08T16:41:51.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.045+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 256 ==== osd_op_reply(693 10000000000.01800000 [omap-set-vals] v131'52 uv52 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.045+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/94 pointing to 0x0x10000000253 2023-11-08T16:41:51.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.045+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:694 17.37 17:ec000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4700 con 0x564eeddf6c00 2023-11-08T16:41:51.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.047+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 195 ==== osd_op_reply(694 [pgnls-filter start_epoch 131 out=49b] v127'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.047+0000 7f918333a640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:695 17.f 17:f0000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c700 con 0x564eedfcb000 2023-11-08T16:41:51.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.048+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 19 ==== osd_op_reply(695 [pgnls-filter start_epoch 131 out=81b] v131'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede73440 con 0x564eedfcb000 2023-11-08T16:41:51.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.048+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 1000000022e.00000000 2023-11-08T16:41:51.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.048+0000 7f9189c365c0 10 datascan.operator(): handling object 1000000022e.0 2023-11-08T16:41:51.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.048+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:696 17.f 17:f25dd5df:::1000000022e.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4a80 con 0x564eedfcb000 2023-11-08T16:41:51.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.049+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.1 v2:172.21.15.81:6800/2060798739 20 ==== osd_op_reply(696 1000000022e.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede73440 con 0x564eedfcb000 2023-11-08T16:41:51.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.049+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x1000000022e 2023-11-08T16:41:51.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.049+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/57 2023-11-08T16:41:51.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.049+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=57 2023-11-08T16:41:51.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.049+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:697 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1880 con 0x564eede60000 2023-11-08T16:41:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.050+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 105 ==== osd_op_reply(697 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.050+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:51.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.050+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:698 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0380 con 0x564eeddf6800 2023-11-08T16:41:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.051+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 257 ==== osd_op_reply(698 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.051+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.051+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.051+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:51.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.051+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:699 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0000 con 0x564eeddf6800 2023-11-08T16:41:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.052+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 258 ==== osd_op_reply(699 10000000000.01800000 [omap-get-header out=274b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.052+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 52 2023-11-08T16:41:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.052+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:51.055 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.052+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:700 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0a80 con 0x564eeddf6800 2023-11-08T16:41:51.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.053+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 259 ==== osd_op_reply(700 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv52 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.053+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=57 frag=1*, r=0 2023-11-08T16:41:51.056 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.053+0000 7f9189c365c0 20 datascan.read_dentry: 57_head not found in result 2023-11-08T16:41:51.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.053+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x1000000022e at 0x0x10000000000/57 with size=2 bytes 2023-11-08T16:41:51.057 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.053+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:701 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0e00 con 0x564eeddf6800 2023-11-08T16:41:51.058 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.057+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 260 ==== osd_op_reply(701 10000000000.01800000 [omap-set-vals] v131'53 uv53 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.057+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/57 pointing to 0x0x1000000022e 2023-11-08T16:41:51.059 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.057+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:702 17.2f 17:f4000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1500 con 0x564eed426c00 2023-11-08T16:41:51.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.058+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.4 v2:172.21.15.93:6800/3259756279 32 ==== osd_op_reply(702 [pgnls-filter start_epoch 131 out=49b] v127'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x564eede72b40 con 0x564eed426c00 2023-11-08T16:41:51.060 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.058+0000 7f918333a640 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:703 17.1f 17:f8000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ca80 con 0x564eed427800 2023-11-08T16:41:51.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.060+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 17 ==== osd_op_reply(703 [pgnls-filter start_epoch 131 out=101b] v131'12 uv12 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:51.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.060+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000238.00000000 2023-11-08T16:41:51.061 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.060+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000238.0 2023-11-08T16:41:51.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.060+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:704 17.1f 17:f87ea8f1:::10000000238.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0700 con 0x564eed427800 2023-11-08T16:41:51.062 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.061+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 18 ==== osd_op_reply(704 10000000238.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv11 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:51.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.061+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000238 2023-11-08T16:41:51.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.061+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/67 2023-11-08T16:41:51.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.061+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=67 2023-11-08T16:41:51.063 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.061+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:705 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464380 con 0x564eede60000 2023-11-08T16:41:51.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.062+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 106 ==== osd_op_reply(705 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:51.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.062+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:51.064 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.062+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:706 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed465880 con 0x564eeddf6800 2023-11-08T16:41:51.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.063+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 261 ==== osd_op_reply(706 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.063+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:51.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.063+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:51.065 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.063+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:51.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.063+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:707 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464000 con 0x564eeddf6800 2023-11-08T16:41:51.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.064+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 262 ==== osd_op_reply(707 10000000000.01800000 [omap-get-header out=274b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.066 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.065+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 53 2023-11-08T16:41:51.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.065+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:51.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.065+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:708 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed464a80 con 0x564eeddf6800 2023-11-08T16:41:51.067 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.066+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 263 ==== osd_op_reply(708 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv53 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.066+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=67 frag=1*, r=0 2023-11-08T16:41:51.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.066+0000 7f9189c365c0 20 datascan.read_dentry: 67_head not found in result 2023-11-08T16:41:51.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.066+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000238 at 0x0x10000000000/67 with size=2 bytes 2023-11-08T16:41:51.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.066+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:709 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed404700 con 0x564eeddf6800 2023-11-08T16:41:51.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.070+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 264 ==== osd_op_reply(709 10000000000.01800000 [omap-set-vals] v131'54 uv54 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.070+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/67 pointing to 0x0x10000000238 2023-11-08T16:41:51.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.070+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:710 17.1f 17:f8e2d5cc::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c380 con 0x564eed427800 2023-11-08T16:41:51.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.071+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 19 ==== osd_op_reply(710 [pgnls-filter start_epoch 131 out=81b] v131'12 uv12 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:51.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.071+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000214.00000000 2023-11-08T16:41:51.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.071+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000214.0 2023-11-08T16:41:51.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.071+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:711 17.1f 17:f8e2d5cc:::10000000214.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d500 con 0x564eed427800 2023-11-08T16:41:51.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.072+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.6 v2:172.21.15.93:6816/607960134 20 ==== osd_op_reply(711 10000000214.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv12 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede73440 con 0x564eed427800 2023-11-08T16:41:51.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.073+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000214 2023-11-08T16:41:51.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.073+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/31 2023-11-08T16:41:51.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.073+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=31 2023-11-08T16:41:51.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.073+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:712 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb5500 con 0x564eede60000 2023-11-08T16:41:51.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.074+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 107 ==== osd_op_reply(712 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:51.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.074+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:51.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.074+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:713 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb4e00 con 0x564eeddf6800 2023-11-08T16:41:51.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.075+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 265 ==== osd_op_reply(713 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.075+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:51.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.075+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:51.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.075+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:51.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.075+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:714 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed0e9c00 con 0x564eeddf6c00 2023-11-08T16:41:51.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.076+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 196 ==== osd_op_reply(714 10000000000.01000000 [omap-get-header out=274b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.076+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 54 2023-11-08T16:41:51.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.076+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:51.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.076+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:715 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eedeb3180 con 0x564eeddf6c00 2023-11-08T16:41:51.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.077+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 197 ==== osd_op_reply(715 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.077+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=31 frag=0*, r=0 2023-11-08T16:41:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.077+0000 7f9189c365c0 20 datascan.read_dentry: 31_head not found in result 2023-11-08T16:41:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.077+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000214 at 0x0x10000000000/31 with size=2 bytes 2023-11-08T16:41:51.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.077+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:716 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ea80 con 0x564eeddf6c00 2023-11-08T16:41:51.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.080+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 198 ==== osd_op_reply(716 10000000000.01000000 [omap-set-vals] v131'55 uv55 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.080+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/31 pointing to 0x0x10000000214 2023-11-08T16:41:51.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.080+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:717 17.3f 17:fc000000::::head [pgnls-filter start_epoch 131 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f180 con 0x564eeddf6c00 2023-11-08T16:41:51.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.081+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 199 ==== osd_op_reply(717 [pgnls-filter start_epoch 131 out=101b] v131'14 uv14 ondisk = 0) v8 ==== 144+0+101 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.081+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 10000000232.00000000 2023-11-08T16:41:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.081+0000 7f9189c365c0 10 datascan.operator(): handling object 10000000232.0 2023-11-08T16:41:51.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.081+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:718 17.3f 17:fd83cc12:::10000000232.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49f500 con 0x564eeddf6c00 2023-11-08T16:41:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.082+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 200 ==== osd_op_reply(718 10000000232.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (90) out=90b,getxattr (30) out=30b,getxattr] v0'0 uv13 ondisk = 0) v8 ==== 416+0+158 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.082+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x10000000232 2023-11-08T16:41:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.082+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/61 2023-11-08T16:41:51.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.082+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=61 2023-11-08T16:41:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.082+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:719 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c700 con 0x564eede60000 2023-11-08T16:41:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.083+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 108 ==== osd_op_reply(719 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:51.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.083+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:51.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.083+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:720 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49d500 con 0x564eeddf6800 2023-11-08T16:41:51.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.085+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 266 ==== osd_op_reply(720 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.085+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:51.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.085+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:51.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.085+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 0* 2023-11-08T16:41:51.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.085+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:721 16.5 16:a40620de:::10000000000.01000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49ce00 con 0x564eeddf6c00 2023-11-08T16:41:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.086+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 201 ==== osd_op_reply(721 10000000000.01000000 [omap-get-header out=274b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.086+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 55 2023-11-08T16:41:51.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.086+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 0* 2023-11-08T16:41:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.086+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:722 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed49c000 con 0x564eeddf6c00 2023-11-08T16:41:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.086+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 202 ==== osd_op_reply(722 10000000000.01000000 [omap-get-vals-by-keys out=4b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.087+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=61 frag=0*, r=0 2023-11-08T16:41:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.087+0000 7f9189c365c0 20 datascan.read_dentry: 61_head not found in result 2023-11-08T16:41:51.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.087+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000232 at 0x0x10000000000/61 with size=2 bytes 2023-11-08T16:41:51.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.087+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:723 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47d880 con 0x564eeddf6c00 2023-11-08T16:41:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.091+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 203 ==== osd_op_reply(723 10000000000.01000000 [omap-set-vals] v131'56 uv56 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.091+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/61 pointing to 0x0x10000000232 2023-11-08T16:41:51.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.091+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:724 17.3f 17:feb1c803::::head [pgnls-filter start_epoch 131 in=105b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47dc00 con 0x564eeddf6c00 2023-11-08T16:41:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.092+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 204 ==== osd_op_reply(724 [pgnls-filter start_epoch 131 out=81b] v131'14 uv14 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.092+0000 7f9189c365c0 20 datascan.forall_objects: OSD matched oid 100000001fa.00000000 2023-11-08T16:41:51.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.092+0000 7f9189c365c0 10 datascan.operator(): handling object 100000001fa.0 2023-11-08T16:41:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.092+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:725 17.3f 17:feb1c803:::100000001fa.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ce00 con 0x564eeddf6c00 2023-11-08T16:41:51.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.093+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.0 v2:172.21.15.81:6808/1666604866 205 ==== osd_op_reply(725 100000001fa.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv14 ondisk = 0) v8 ==== 416+0+157 (crc 0 0 0) 0x564eede72b40 con 0x564eeddf6c00 2023-11-08T16:41:51.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.093+0000 7f9189c365c0 10 datascan.inject_with_backtrace: inode: 0x0x100000001fa 2023-11-08T16:41:51.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.093+0000 7f9189c365c0 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/5 2023-11-08T16:41:51.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.093+0000 7f9189c365c0 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=5 2023-11-08T16:41:51.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.093+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:726 16.10 16:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47c700 con 0x564eede60000 2023-11-08T16:41:51.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.094+0000 7f9187342640 1 -- 172.21.15.81:0/1589379186 <== osd.2 v2:172.21.15.81:6824/3759311428 109 ==== osd_op_reply(726 10000000000.00000000 [getxattr (62) out=62b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+62 (crc 0 0 0) 0x564eede72b40 con 0x564eede60000 2023-11-08T16:41:51.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.094+0000 7f9189c365c0 20 datascan.get_frag_of: got backtrace parent 0x1/subdir 2023-11-08T16:41:51.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.094+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:727 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed47ca80 con 0x564eeddf6800 2023-11-08T16:41:51.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.095+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 267 ==== osd_op_reply(727 1.00000000 [omap-get-vals-by-keys out=531b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+531 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.095+0000 7f9189c365c0 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-11-08T16:41:51.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.095+0000 7f9189c365c0 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*^1) 2023-11-08T16:41:51.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.095+0000 7f9189c365c0 20 datascan.get_frag_of: frag is 1* 2023-11-08T16:41:51.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.095+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:728 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a1180 con 0x564eeddf6800 2023-11-08T16:41:51.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.096+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 268 ==== osd_op_reply(728 10000000000.01800000 [omap-get-header out=274b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+274 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.096+0000 7f9189c365c0 10 datascan.find_or_create_dirfrag: read_version = 54 2023-11-08T16:41:51.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.096+0000 7f9189c365c0 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 1* 2023-11-08T16:41:51.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.096+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:729 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0380 con 0x564eeddf6800 2023-11-08T16:41:51.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.098+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 269 ==== osd_op_reply(729 10000000000.01800000 [omap-get-vals-by-keys out=4b] v0'0 uv54 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.098+0000 7f9189c365c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=5 frag=1*, r=0 2023-11-08T16:41:51.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.098+0000 7f9189c365c0 20 datascan.read_dentry: 5_head not found in result 2023-11-08T16:41:51.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.098+0000 7f9189c365c0 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001fa at 0x0x10000000000/5 with size=1 bytes 2023-11-08T16:41:51.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.098+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:730 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x564eed4a0000 con 0x564eeddf6800 2023-11-08T16:41:51.103 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9188344640 1 -- 172.21.15.81:0/1589379186 <== osd.5 v2:172.21.15.93:6802/2406579932 270 ==== osd_op_reply(730 10000000000.01800000 [omap-set-vals] v131'55 uv55 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x564eede73440 con 0x564eeddf6800 2023-11-08T16:41:51.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-11-08T16:41:51.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x564eeddf6c00 msgr2=0x564eed0fb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x564eeddf6c00 0x564eed0fb080 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x564eedfcb000 msgr2=0x564eed3e6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.104 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x564eedfcb000 0x564eed3e6b00 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x564eede60000 msgr2=0x564eed3e6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x564eede60000 0x564eed3e6580 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x564eedfcb800 msgr2=0x564eed3e7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x564eedfcb800 0x564eed3e7080 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x564eed426c00 msgr2=0x564eed3e7600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x564eed426c00 0x564eed3e7600 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 reap_dead start 2023-11-08T16:41:51.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x564eeddf6800 msgr2=0x564eed0fc680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x564eeddf6800 0x564eed0fc680 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 reap_dead start 2023-11-08T16:41:51.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9187b43640 1 -- 172.21.15.81:0/1589379186 reap_dead start 2023-11-08T16:41:51.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x564eed427800 msgr2=0x564eed3e8c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x564eed427800 0x564eed3e8c00 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x564eeddf6400 msgr2=0x564eed0fc100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.102+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x564eeddf6400 0x564eed0fc100 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.103+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x564eedfcac00 msgr2=0x564eed3e6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.103+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x564eedfcac00 0x564eed3e6000 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x564eedf68810 tx=0x564eedf6bf80 comp rx=0 tx=0).stop 2023-11-08T16:41:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.103+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eeddf7400 msgr2=0x564eed0fd180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.103+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eeddf7400 0x564eed0fd180 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x564eedf69950 tx=0x564eedf6ade0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.103+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 shutdown_connections 2023-11-08T16:41:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.103+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x564eeddf6400 0x564eed0fc100 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.103+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x564eed427800 0x564eed3e8c00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.103+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x564eedfcac00 0x564eed3e6000 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.103+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1589379186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eeddf7400 0x564eed0fd180 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.103+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 >> 172.21.15.81:0/1589379186 conn(0x564eede60400 msgr2=0x564eede1b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.104+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 shutdown_connections 2023-11-08T16:41:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.104+0000 7f9189c365c0 1 -- 172.21.15.81:0/1589379186 wait complete. 2023-11-08T16:41:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.105+0000 7f9189c365c0 1 -- 172.21.15.81:0/1804835141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede61c00 msgr2=0x564eed0fb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.105+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1804835141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede61c00 0x564eed0fb600 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x564eedeab110 tx=0x564eede6aae0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.105+0000 7f9189c365c0 1 -- 172.21.15.81:0/1804835141 shutdown_connections 2023-11-08T16:41:51.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.105+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1804835141 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x564eede61c00 0x564eed0fb600 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.105+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1804835141 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x564eed091c00 0x564eed0fab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.105+0000 7f9189c365c0 1 --2- 172.21.15.81:0/1804835141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x564eede60800 0x564eed0fbb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.105+0000 7f9189c365c0 1 -- 172.21.15.81:0/1804835141 >> 172.21.15.81:0/1804835141 conn(0x564eed091000 msgr2=0x564eede1a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:51.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.105+0000 7f9189c365c0 1 -- 172.21.15.81:0/1804835141 shutdown_connections 2023-11-08T16:41:51.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.105+0000 7f9189c365c0 1 -- 172.21.15.81:0/1804835141 wait complete. 2023-11-08T16:41:51.114 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes'] in time 0:00:01.323829, result: 2023-11-08T16:41:51.115 DEBUG:teuthology.orchestra.run.smithi081:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2023-11-08T16:41:51.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.261+0000 7f5f9aa385c0 1 -- start start 2023-11-08T16:41:51.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.261+0000 7f5f9aa385c0 1 Processor -- start 2023-11-08T16:41:51.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.262+0000 7f5f9aa385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdb800 0x562385c44b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.262+0000 7f5f9aa385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562385bdbc00 0x562385c45080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.262+0000 7f5f9aa385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5623869b7400 0x562385c45600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.262+0000 7f5f9aa385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562385c23180 con 0x562385bdbc00 2023-11-08T16:41:51.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.262+0000 7f5f9aa385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562385c22140 con 0x562385bdb800 2023-11-08T16:41:51.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.262+0000 7f5f9aa385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562385c22000 con 0x5623869b7400 2023-11-08T16:41:51.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.262+0000 7f5f98267640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562385bdbc00 0x562385c45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.262+0000 7f5f98267640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562385bdbc00 0x562385c45080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40578/0 (socket says 172.21.15.81:40578) 2023-11-08T16:41:51.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.262+0000 7f5f98267640 1 -- 172.21.15.81:0/2672302937 learned_addr learned my addr 172.21.15.81:0/2672302937 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:51.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f98a68640 1 --2- 172.21.15.81:0/2672302937 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdb800 0x562385c44b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f98267640 1 -- 172.21.15.81:0/2672302937 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5623869b7400 msgr2=0x562385c45600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:51.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f99269640 1 --2- 172.21.15.81:0/2672302937 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5623869b7400 0x562385c45600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f98267640 1 --2- 172.21.15.81:0/2672302937 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5623869b7400 0x562385c45600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f98267640 1 -- 172.21.15.81:0/2672302937 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdb800 msgr2=0x562385c44b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:51.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f98267640 1 --2- 172.21.15.81:0/2672302937 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdb800 0x562385c44b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f98267640 1 -- 172.21.15.81:0/2672302937 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562385c009c0 con 0x562385bdbc00 2023-11-08T16:41:51.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f98a68640 1 --2- 172.21.15.81:0/2672302937 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdb800 0x562385c44b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:51.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f99269640 1 --2- 172.21.15.81:0/2672302937 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5623869b7400 0x562385c45600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:51.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f98267640 1 --2- 172.21.15.81:0/2672302937 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562385bdbc00 0x562385c45080 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x5623869e7530 tx=0x5623869aac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7491cabb4ce44c84 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f96263640 1 -- 172.21.15.81:0/2672302937 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562385c211e0 con 0x562385bdbc00 2023-11-08T16:41:51.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f96263640 1 -- 172.21.15.81:0/2672302937 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562385c34480 con 0x562385bdbc00 2023-11-08T16:41:51.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f96263640 1 -- 172.21.15.81:0/2672302937 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562385c21080 con 0x562385bdbc00 2023-11-08T16:41:51.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/2672302937 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562385c00b60 con 0x562385bdbc00 2023-11-08T16:41:51.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/2672302937 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x562385c00820 con 0x562385bdbc00 2023-11-08T16:41:51.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.263+0000 7f5f9aa385c0 4 waiting for MDS map... 2023-11-08T16:41:51.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.264+0000 7f5f96263640 1 -- 172.21.15.81:0/2672302937 <== mon.0 v2:172.21.15.81:3300/0 4 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x562385c9ac40 con 0x562385bdbc00 2023-11-08T16:41:51.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.264+0000 7f5f96263640 1 -- 172.21.15.81:0/2672302937 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 115) v1 ==== 3298+0+0 (secure 0 0 0) 0x562385c2af00 con 0x562385bdbc00 2023-11-08T16:41:51.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.264+0000 7f5f9aa385c0 4 Got MDS map 115 2023-11-08T16:41:51.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.264+0000 7f5f9aa385c0 4 datascan.main: Using metadata pool output 2023-11-08T16:41:51.272 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.264+0000 7f5f9aa385c0 4 datascan.main: connecting to RADOS... 2023-11-08T16:41:51.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.265+0000 7f5f9aa385c0 1 Processor -- start 2023-11-08T16:41:51.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.265+0000 7f5f9aa385c0 1 -- start start 2023-11-08T16:41:51.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f9aa385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623869b6800 0x562385c45b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f9aa385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdac00 0x562385c46100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f9aa385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562386af6000 0x562385c46680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f9aa385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562385c22a00 con 0x5623869b6800 2023-11-08T16:41:51.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f9aa385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562385c22b40 con 0x562385bdac00 2023-11-08T16:41:51.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f9aa385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562385ca9180 con 0x562386af6000 2023-11-08T16:41:51.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f99269640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623869b6800 0x562385c45b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f99269640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623869b6800 0x562385c45b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40586/0 (socket says 172.21.15.81:40586) 2023-11-08T16:41:51.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f99269640 1 -- 172.21.15.81:0/3060983096 learned_addr learned my addr 172.21.15.81:0/3060983096 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:51.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f99269640 1 -- 172.21.15.81:0/3060983096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562386af6000 msgr2=0x562385c46680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:51.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f99269640 1 --2- 172.21.15.81:0/3060983096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562386af6000 0x562385c46680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f99269640 1 -- 172.21.15.81:0/3060983096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdac00 msgr2=0x562385c46100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:51.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f99269640 1 --2- 172.21.15.81:0/3060983096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdac00 0x562385c46100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.266+0000 7f5f99269640 1 -- 172.21.15.81:0/3060983096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562385c004e0 con 0x5623869b6800 2023-11-08T16:41:51.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.267+0000 7f5f99269640 1 --2- 172.21.15.81:0/3060983096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623869b6800 0x562385c45b80 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x562386aa4a20 tx=0x562386aaccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f039dd730d1fccb server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.267+0000 7f5f9425f640 1 -- 172.21.15.81:0/3060983096 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562385c211e0 con 0x5623869b6800 2023-11-08T16:41:51.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.267+0000 7f5f9425f640 1 -- 172.21.15.81:0/3060983096 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562385cba600 con 0x5623869b6800 2023-11-08T16:41:51.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.267+0000 7f5f9425f640 1 -- 172.21.15.81:0/3060983096 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562385c21080 con 0x5623869b6800 2023-11-08T16:41:51.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.267+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/3060983096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623869b6800 msgr2=0x562385c45b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.267+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/3060983096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623869b6800 0x562385c45b80 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x562386aa4a20 tx=0x562386aaccc0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.268+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/3060983096 shutdown_connections 2023-11-08T16:41:51.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.268+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/3060983096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x562386af6000 0x562385c46680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.268+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/3060983096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdac00 0x562385c46100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.268+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/3060983096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5623869b6800 0x562385c45b80 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.268+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/3060983096 >> 172.21.15.81:0/3060983096 conn(0x5623869b7000 msgr2=0x562386965000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:51.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.268+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/3060983096 shutdown_connections 2023-11-08T16:41:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.268+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/3060983096 wait complete. 2023-11-08T16:41:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.268+0000 7f5f9aa385c0 1 Processor -- start 2023-11-08T16:41:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.269+0000 7f5f9aa385c0 1 -- start start 2023-11-08T16:41:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.269+0000 7f5f9aa385c0 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdac00 0x562385c46100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.269+0000 7f5f9aa385c0 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5623869b6800 0x562385c45b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.269+0000 7f5f9aa385c0 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562386af7000 0x562385c46680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.269+0000 7f5f9aa385c0 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x562385ca8dc0 con 0x562386af7000 2023-11-08T16:41:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.269+0000 7f5f9aa385c0 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x562385ca8c80 con 0x562385bdac00 2023-11-08T16:41:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.269+0000 7f5f9aa385c0 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x562385ca8b40 con 0x5623869b6800 2023-11-08T16:41:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.270+0000 7f5f98267640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562386af7000 0x562385c46680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.270+0000 7f5f98267640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562386af7000 0x562385c46680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40598/0 (socket says 172.21.15.81:40598) 2023-11-08T16:41:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.270+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 learned_addr learned my addr 172.21.15.81:0/4210848566 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.270+0000 7f5f98a68640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5623869b6800 0x562385c45b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.270+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5623869b6800 msgr2=0x562385c45b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.270+0000 7f5f98267640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5623869b6800 0x562385c45b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.270+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdac00 msgr2=0x562385c46100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.270+0000 7f5f98267640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdac00 0x562385c46100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.270+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562385c009c0 con 0x562386af7000 2023-11-08T16:41:51.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.270+0000 7f5f98267640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562386af7000 0x562385c46680 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x562386aa48d0 tx=0x56238696a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.271+0000 7f5f92a5c640 1 -- 172.21.15.81:0/4210848566 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562385c20dc0 con 0x562386af7000 2023-11-08T16:41:51.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.271+0000 7f5f92a5c640 1 -- 172.21.15.81:0/4210848566 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562385cbaa80 con 0x562386af7000 2023-11-08T16:41:51.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.271+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562385c00b60 con 0x562386af7000 2023-11-08T16:41:51.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.271+0000 7f5f92a5c640 1 -- 172.21.15.81:0/4210848566 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x562385c211e0 con 0x562386af7000 2023-11-08T16:41:51.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.271+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562385c001a0 con 0x562386af7000 2023-11-08T16:41:51.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.271+0000 7f5f9aa385c0 4 datascan.init: resolving metadata pool 16 2023-11-08T16:41:51.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.273+0000 7f5f92a5c640 1 -- 172.21.15.81:0/4210848566 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x562385c36b00 con 0x562386af7000 2023-11-08T16:41:51.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.273+0000 7f5f92a5c640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562386b10c00 0x562385c47700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.273+0000 7f5f92a5c640 1 -- 172.21.15.81:0/4210848566 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x562385c9aa80 con 0x562386af7000 2023-11-08T16:41:51.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.273+0000 7f5f9aa385c0 4 datascan.init: found metadata pool 'cephfs_metadata' 2023-11-08T16:41:51.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.273+0000 7f5f9aa385c0 4 datascan.main: resolving metadata pool 16 2023-11-08T16:41:51.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.273+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562386af6c00 0x562385c46c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.273+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.273+0000 7f5f99269640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562386b10c00 0x562385c47700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.274+0000 7f5f98a68640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562386af6c00 0x562385c46c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.274+0000 7f5f99269640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562386b10c00 0x562385c47700 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x562386aa4a20 tx=0x562386aacc00 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.274+0000 7f5f98a68640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562386af6c00 0x562385c46c00 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.275+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(1 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.275+0000 7f5f98a68640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562386b11000 0x562385c44580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.275+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11000 2023-11-08T16:41:51.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.275+0000 7f5f99269640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562386b11000 0x562385c44580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.276+0000 7f5f99269640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562386b11000 0x562385c44580 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.276+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== osd_op_reply(2 [pgnls start_epoch 131 out=177b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.276+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.276+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== osd_op_reply(3 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.276+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.277+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 3 ==== osd_op_reply(4 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.277+0000 7f5f98a68640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562386b11400 0x562385f46000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.277+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11400 2023-11-08T16:41:51.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.277+0000 7f5f98a68640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562386b11400 0x562385f46000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.278+0000 7f5f98a68640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562386b11400 0x562385f46000 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.278+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== osd_op_reply(5 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386b11400 2023-11-08T16:41:51.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.278+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.278+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 4 ==== osd_op_reply(6 [pgnls start_epoch 131 out=149b] v130'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.278+0000 7f5f98a68640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562386b11800 0x562385f46580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.278+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11800 2023-11-08T16:41:51.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.279+0000 7f5f98267640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562386b11800 0x562385f46580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.280+0000 7f5f98267640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562386b11800 0x562385f46580 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.281+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== osd_op_reply(7 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.281+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11000 2023-11-08T16:41:51.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.281+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== osd_op_reply(8 [pgnls start_epoch 131 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.282+0000 7f5f99269640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562386af7c00 0x562385f47080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.282+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af7c00 2023-11-08T16:41:51.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.282+0000 7f5f99269640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562386af7c00 0x562385f47080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.283+0000 7f5f99269640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562386af7c00 0x562385f47080 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.284+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== osd_op_reply(9 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562386af7c00 2023-11-08T16:41:51.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.284+0000 7f5f99269640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562385f5e000 0x562385f47600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.284+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562385f5e000 2023-11-08T16:41:51.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.285+0000 7f5f98a68640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562385f5e000 0x562385f47600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.286+0000 7f5f98a68640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562385f5e000 0x562385f47600 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.286+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== osd_op_reply(10 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562385f5e000 2023-11-08T16:41:51.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.287+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11000 2023-11-08T16:41:51.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.287+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 3 ==== osd_op_reply(11 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.287+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11400 2023-11-08T16:41:51.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.287+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== osd_op_reply(12 [pgnls start_epoch 131 out=49b] v128'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386b11400 2023-11-08T16:41:51.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.287+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562385f5e000 2023-11-08T16:41:51.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.288+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== osd_op_reply(13 [pgnls start_epoch 131 out=77b] v130'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x562386a82b40 con 0x562385f5e000 2023-11-08T16:41:51.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.288+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11400 2023-11-08T16:41:51.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.289+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.3 v2:172.21.15.81:6816/2756229107 3 ==== osd_op_reply(14 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386b11400 2023-11-08T16:41:51.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.289+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11000 2023-11-08T16:41:51.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.289+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 4 ==== osd_op_reply(15 [pgnls start_epoch 131 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.289+0000 7f5f99269640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562385f5e400 0x562385f47b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.289+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562385f5e400 2023-11-08T16:41:51.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.290+0000 7f5f98267640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562385f5e400 0x562385f47b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.291+0000 7f5f98267640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562385f5e400 0x562385f47b80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.292+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== osd_op_reply(16 [pgnls start_epoch 131 out=49b] v128'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82fc0 con 0x562385f5e400 2023-11-08T16:41:51.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.292+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11000 2023-11-08T16:41:51.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.292+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 5 ==== osd_op_reply(17 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.292+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11800 2023-11-08T16:41:51.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.293+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== osd_op_reply(18 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.293+0000 7f5f98267640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562385f5ec00 0x562385f48100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:51.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.293+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562385f5ec00 2023-11-08T16:41:51.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.294+0000 7f5f99269640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562385f5ec00 0x562385f48100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:51.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.294+0000 7f5f99269640 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562385f5ec00 0x562385f48100 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:51.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.295+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== osd_op_reply(19 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562385f5ec00 2023-11-08T16:41:51.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.295+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af7c00 2023-11-08T16:41:51.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.296+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== osd_op_reply(20 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562386af7c00 2023-11-08T16:41:51.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.296+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.296+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 5 ==== osd_op_reply(21 [pgnls start_epoch 131 out=81b] v131'56 uv56 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.296+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.297+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 6 ==== osd_op_reply(22 [pgnls start_epoch 131 out=97b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.297+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562385f5e000 2023-11-08T16:41:51.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.298+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.6 v2:172.21.15.93:6816/607960134 3 ==== osd_op_reply(23 [pgnls start_epoch 131 out=74b] v126'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x562386a82b40 con 0x562385f5e000 2023-11-08T16:41:51.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.298+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11000 2023-11-08T16:41:51.313 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.298+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 6 ==== osd_op_reply(24 [pgnls start_epoch 131 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.298+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11800 2023-11-08T16:41:51.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.299+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 3 ==== osd_op_reply(25 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.299+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562385f5ec00 2023-11-08T16:41:51.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.300+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== osd_op_reply(26 [pgnls start_epoch 131 out=98b] v127'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x562386a82d80 con 0x562385f5ec00 2023-11-08T16:41:51.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.300+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562385f5ec00 2023-11-08T16:41:51.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.300+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.1 v2:172.21.15.81:6800/2060798739 3 ==== osd_op_reply(27 [pgnls start_epoch 131 out=76b] v128'4 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x562386a82d80 con 0x562385f5ec00 2023-11-08T16:41:51.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.300+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11800 2023-11-08T16:41:51.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.302+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 4 ==== osd_op_reply(28 [pgnls start_epoch 131 out=81b] v131'55 uv55 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.302+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11400 2023-11-08T16:41:51.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.302+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.3 v2:172.21.15.81:6816/2756229107 4 ==== osd_op_reply(29 [pgnls start_epoch 131 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562386a82b40 con 0x562386b11400 2023-11-08T16:41:51.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.302+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11000 2023-11-08T16:41:51.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.303+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 7 ==== osd_op_reply(30 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.303+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.304+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 7 ==== osd_op_reply(31 [pgnls start_epoch 131 out=73b] v126'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.304+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11800 2023-11-08T16:41:51.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.305+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 5 ==== osd_op_reply(32 [pgnls start_epoch 131 out=101b] v128'7 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.305+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 601.00000000 2023-11-08T16:41:51.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.305+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386b11000 2023-11-08T16:41:51.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.305+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.305+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 602.00000000 2023-11-08T16:41:51.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.305+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a000 con 0x562386b11000 2023-11-08T16:41:51.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.306+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.306+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-11-08T16:41:51.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.306+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386b11000 2023-11-08T16:41:51.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.308+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=50980b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+50980 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.308+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 600.00000000 2023-11-08T16:41:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.308+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386b11000 2023-11-08T16:41:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.309+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.309+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 603.00000000 2023-11-08T16:41:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.309+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11000 2023-11-08T16:41:51.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-11-08T16:41:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-08T16:41:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 200.00000000 2023-11-08T16:41:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-08T16:41:51.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 200.00000001 2023-11-08T16:41:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-08T16:41:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 606.00000000 2023-11-08T16:41:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386af6c00 2023-11-08T16:41:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 607.00000000 2023-11-08T16:41:51.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.310+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a000 con 0x562386b11000 2023-11-08T16:41:51.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.311+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.311+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2023-11-08T16:41:51.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.311+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2023-11-08T16:41:51.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.311+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 608.00000000 2023-11-08T16:41:51.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.311+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:40 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386b11000 2023-11-08T16:41:51.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.311+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 14 ==== osd_op_reply(40 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.311+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 10000000000.01000000 2023-11-08T16:41:51.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.311+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:41 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386af6c00 2023-11-08T16:41:51.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 9 ==== osd_op_reply(41 10000000000.01000000 [omap-get-vals out=25448b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+25448 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.335 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.336 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.312+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 604.00000000 2023-11-08T16:41:51.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 10 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 500.00000000 2023-11-08T16:41:51.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-08T16:41:51.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-11-08T16:41:51.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-08T16:41:51.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 605.00000000 2023-11-08T16:41:51.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.313+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386b11000 2023-11-08T16:41:51.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.314+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.314+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-11-08T16:41:51.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.314+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-08T16:41:51.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.314+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 100.00000000 2023-11-08T16:41:51.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.314+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b180 con 0x562385f5ec00 2023-11-08T16:41:51.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.314+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.1 v2:172.21.15.81:6800/2060798739 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x562386a82d80 con 0x562385f5ec00 2023-11-08T16:41:51.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.315+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-11-08T16:41:51.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.315+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-08T16:41:51.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.315+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 10000000000.01800000 2023-11-08T16:41:51.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.315+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:45 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a000 con 0x562386b11800 2023-11-08T16:41:51.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 6 ==== osd_op_reply(45 10000000000.01800000 [omap-get-vals out=24452b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+24452 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.317+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 609.00000000 2023-11-08T16:41:51.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:46 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386b11400 2023-11-08T16:41:51.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.3 v2:172.21.15.81:6816/2756229107 5 ==== osd_op_reply(46 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82b40 con 0x562386b11400 2023-11-08T16:41:51.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 400.00000000 2023-11-08T16:41:51.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-08T16:41:51.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-11-08T16:41:51.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-08T16:41:51.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 10 datascan.scan_links: step 1: handling object 1.00000000 2023-11-08T16:41:51.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.318+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386b11800 2023-11-08T16:41:51.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.319+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 7 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=532b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+532 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.319+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b180 con 0x562386af6c00 2023-11-08T16:41:51.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.320+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 11 ==== osd_op_reply(48 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.320+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11000 2023-11-08T16:41:51.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.321+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 16 ==== osd_op_reply(49 [pgnls start_epoch 131 out=177b] v128'12 uv12 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.321+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.322+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 12 ==== osd_op_reply(50 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.322+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.322+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 13 ==== osd_op_reply(51 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.322+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11400 2023-11-08T16:41:51.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.323+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.3 v2:172.21.15.81:6816/2756229107 6 ==== osd_op_reply(52 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386b11400 2023-11-08T16:41:51.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.323+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.323+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 14 ==== osd_op_reply(53 [pgnls start_epoch 131 out=149b] v130'27 uv27 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.323+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11800 2023-11-08T16:41:51.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.324+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 8 ==== osd_op_reply(54 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.324+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11000 2023-11-08T16:41:51.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.325+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 17 ==== osd_op_reply(55 [pgnls start_epoch 131 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.325+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af7c00 2023-11-08T16:41:51.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.326+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.4 v2:172.21.15.93:6800/3259756279 3 ==== osd_op_reply(56 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562386af7c00 2023-11-08T16:41:51.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.326+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562385f5e000 2023-11-08T16:41:51.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.328+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.6 v2:172.21.15.93:6816/607960134 4 ==== osd_op_reply(57 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562385f5e000 2023-11-08T16:41:51.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.328+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11000 2023-11-08T16:41:51.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.328+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 18 ==== osd_op_reply(58 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.328+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11400 2023-11-08T16:41:51.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.329+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.3 v2:172.21.15.81:6816/2756229107 7 ==== osd_op_reply(59 [pgnls start_epoch 131 out=49b] v128'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386b11400 2023-11-08T16:41:51.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.329+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562385f5e000 2023-11-08T16:41:51.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.330+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.6 v2:172.21.15.93:6816/607960134 5 ==== osd_op_reply(60 [pgnls start_epoch 131 out=77b] v130'7 uv7 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x562386a82b40 con 0x562385f5e000 2023-11-08T16:41:51.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.330+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11400 2023-11-08T16:41:51.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.331+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.3 v2:172.21.15.81:6816/2756229107 8 ==== osd_op_reply(61 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82b40 con 0x562386b11400 2023-11-08T16:41:51.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.331+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11000 2023-11-08T16:41:51.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.332+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 19 ==== osd_op_reply(62 [pgnls start_epoch 131 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.332+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562385f5e400 2023-11-08T16:41:51.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.333+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== osd_op_reply(63 [pgnls start_epoch 131 out=49b] v128'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82fc0 con 0x562385f5e400 2023-11-08T16:41:51.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.333+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11000 2023-11-08T16:41:51.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.334+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 20 ==== osd_op_reply(64 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.334+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11800 2023-11-08T16:41:51.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.335+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 9 ==== osd_op_reply(65 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.335+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562385f5ec00 2023-11-08T16:41:51.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.335+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.1 v2:172.21.15.81:6800/2060798739 5 ==== osd_op_reply(66 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562385f5ec00 2023-11-08T16:41:51.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.335+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af7c00 2023-11-08T16:41:51.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.336+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.4 v2:172.21.15.93:6800/3259756279 4 ==== osd_op_reply(67 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562386af7c00 2023-11-08T16:41:51.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.336+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.337+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 15 ==== osd_op_reply(68 [pgnls start_epoch 131 out=81b] v131'56 uv56 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.337+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.338+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 16 ==== osd_op_reply(69 [pgnls start_epoch 131 out=97b] v126'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.338+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562385f5e000 2023-11-08T16:41:51.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.339+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.6 v2:172.21.15.93:6816/607960134 6 ==== osd_op_reply(70 [pgnls start_epoch 131 out=74b] v126'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x562386a82b40 con 0x562385f5e000 2023-11-08T16:41:51.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.339+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11000 2023-11-08T16:41:51.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.340+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 21 ==== osd_op_reply(71 [pgnls start_epoch 131 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.340+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11800 2023-11-08T16:41:51.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.341+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 10 ==== osd_op_reply(72 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.341+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562385f5ec00 2023-11-08T16:41:51.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.341+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.1 v2:172.21.15.81:6800/2060798739 6 ==== osd_op_reply(73 [pgnls start_epoch 131 out=98b] v127'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x562386a82d80 con 0x562385f5ec00 2023-11-08T16:41:51.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.341+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562385f5ec00 2023-11-08T16:41:51.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.342+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.1 v2:172.21.15.81:6800/2060798739 7 ==== osd_op_reply(74 [pgnls start_epoch 131 out=76b] v128'4 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x562386a82d80 con 0x562385f5ec00 2023-11-08T16:41:51.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.342+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11800 2023-11-08T16:41:51.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.343+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 11 ==== osd_op_reply(75 [pgnls start_epoch 131 out=81b] v131'55 uv55 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.343+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11400 2023-11-08T16:41:51.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.344+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.3 v2:172.21.15.81:6816/2756229107 9 ==== osd_op_reply(76 [pgnls start_epoch 131 out=73b] v126'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562386a82b40 con 0x562386b11400 2023-11-08T16:41:51.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.344+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11000 2023-11-08T16:41:51.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.345+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 22 ==== osd_op_reply(77 [pgnls start_epoch 131 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.345+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.346+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 17 ==== osd_op_reply(78 [pgnls start_epoch 131 out=73b] v126'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.346+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 131 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11800 2023-11-08T16:41:51.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.347+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 12 ==== osd_op_reply(79 [pgnls start_epoch 131 out=101b] v128'7 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.347+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 601.00000000 2023-11-08T16:41:51.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.347+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386b11000 2023-11-08T16:41:51.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.348+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv6 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.348+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 602.00000000 2023-11-08T16:41:51.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.348+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386b11000 2023-11-08T16:41:51.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.349+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.349+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-11-08T16:41:51.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.349+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386b11000 2023-11-08T16:41:51.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.351+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=50980b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+50980 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.352+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 600.00000000 2023-11-08T16:41:51.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.352+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b180 con 0x562386b11000 2023-11-08T16:41:51.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.353+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.353+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 603.00000000 2023-11-08T16:41:51.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.353+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a000 con 0x562386b11000 2023-11-08T16:41:51.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.354+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv8 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.354+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-11-08T16:41:51.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.354+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-11-08T16:41:51.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.354+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 200.00000000 2023-11-08T16:41:51.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.354+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2023-11-08T16:41:51.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.354+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 200.00000001 2023-11-08T16:41:51.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.354+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2023-11-08T16:41:51.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.354+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 606.00000000 2023-11-08T16:41:51.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.354+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386af6c00 2023-11-08T16:41:51.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.355+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 18 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.355+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 607.00000000 2023-11-08T16:41:51.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.355+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b880 con 0x562386b11000 2023-11-08T16:41:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.356+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.356+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2023-11-08T16:41:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.356+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2023-11-08T16:41:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.356+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 608.00000000 2023-11-08T16:41:51.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.356+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:87 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6bc00 con 0x562386b11000 2023-11-08T16:41:51.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.356+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.356+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 10000000000.01000000 2023-11-08T16:41:51.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.356+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:88 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386af6c00 2023-11-08T16:41:51.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 19 ==== osd_op_reply(88 10000000000.01000000 [omap-get-vals out=25448b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+25448 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x100000001ff for future processing to fix dnfirst 2023-11-08T16:41:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000200 for future processing to fix dnfirst 2023-11-08T16:41:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000201 for future processing to fix dnfirst 2023-11-08T16:41:51.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000204 for future processing to fix dnfirst 2023-11-08T16:41:51.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000208 for future processing to fix dnfirst 2023-11-08T16:41:51.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x100000001f6 for future processing to fix dnfirst 2023-11-08T16:41:51.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000209 for future processing to fix dnfirst 2023-11-08T16:41:51.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000020a for future processing to fix dnfirst 2023-11-08T16:41:51.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000210 for future processing to fix dnfirst 2023-11-08T16:41:51.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000211 for future processing to fix dnfirst 2023-11-08T16:41:51.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2023-11-08T16:41:51.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000214 for future processing to fix dnfirst 2023-11-08T16:41:51.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000217 for future processing to fix dnfirst 2023-11-08T16:41:51.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000219 for future processing to fix dnfirst 2023-11-08T16:41:51.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000021b for future processing to fix dnfirst 2023-11-08T16:41:51.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000021c for future processing to fix dnfirst 2023-11-08T16:41:51.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x100000001f8 for future processing to fix dnfirst 2023-11-08T16:41:51.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000021f for future processing to fix dnfirst 2023-11-08T16:41:51.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000220 for future processing to fix dnfirst 2023-11-08T16:41:51.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000223 for future processing to fix dnfirst 2023-11-08T16:41:51.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x100000001f9 for future processing to fix dnfirst 2023-11-08T16:41:51.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000227 for future processing to fix dnfirst 2023-11-08T16:41:51.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000229 for future processing to fix dnfirst 2023-11-08T16:41:51.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000022a for future processing to fix dnfirst 2023-11-08T16:41:51.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000022c for future processing to fix dnfirst 2023-11-08T16:41:51.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000230 for future processing to fix dnfirst 2023-11-08T16:41:51.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000232 for future processing to fix dnfirst 2023-11-08T16:41:51.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000234 for future processing to fix dnfirst 2023-11-08T16:41:51.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000235 for future processing to fix dnfirst 2023-11-08T16:41:51.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000236 for future processing to fix dnfirst 2023-11-08T16:41:51.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000237 for future processing to fix dnfirst 2023-11-08T16:41:51.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000023a for future processing to fix dnfirst 2023-11-08T16:41:51.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000023b for future processing to fix dnfirst 2023-11-08T16:41:51.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000023e for future processing to fix dnfirst 2023-11-08T16:41:51.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000023f for future processing to fix dnfirst 2023-11-08T16:41:51.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.358+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000240 for future processing to fix dnfirst 2023-11-08T16:41:51.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000242 for future processing to fix dnfirst 2023-11-08T16:41:51.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000245 for future processing to fix dnfirst 2023-11-08T16:41:51.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000248 for future processing to fix dnfirst 2023-11-08T16:41:51.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000249 for future processing to fix dnfirst 2023-11-08T16:41:51.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000024a for future processing to fix dnfirst 2023-11-08T16:41:51.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000024b for future processing to fix dnfirst 2023-11-08T16:41:51.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000024c for future processing to fix dnfirst 2023-11-08T16:41:51.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000024d for future processing to fix dnfirst 2023-11-08T16:41:51.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x100000001fd for future processing to fix dnfirst 2023-11-08T16:41:51.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000024f for future processing to fix dnfirst 2023-11-08T16:41:51.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000251 for future processing to fix dnfirst 2023-11-08T16:41:51.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000254 for future processing to fix dnfirst 2023-11-08T16:41:51.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000255 for future processing to fix dnfirst 2023-11-08T16:41:51.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000256 for future processing to fix dnfirst 2023-11-08T16:41:51.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x100000001fe for future processing to fix dnfirst 2023-11-08T16:41:51.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 604.00000000 2023-11-08T16:41:51.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386af6c00 2023-11-08T16:41:51.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.359+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 20 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.360+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 500.00000000 2023-11-08T16:41:51.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.360+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2023-11-08T16:41:51.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.360+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-11-08T16:41:51.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.360+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-11-08T16:41:51.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.360+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 605.00000000 2023-11-08T16:41:51.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.360+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b180 con 0x562386b11000 2023-11-08T16:41:51.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.360+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.2 v2:172.21.15.81:6824/3759311428 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82d80 con 0x562386b11000 2023-11-08T16:41:51.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.360+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-11-08T16:41:51.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.360+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-11-08T16:41:51.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.360+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 100.00000000 2023-11-08T16:41:51.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.360+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a000 con 0x562385f5ec00 2023-11-08T16:41:51.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.361+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.1 v2:172.21.15.81:6800/2060798739 8 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5195 (crc 0 0 0) 0x562386a82d80 con 0x562385f5ec00 2023-11-08T16:41:51.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.362+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-11-08T16:41:51.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.362+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-11-08T16:41:51.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.362+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 10000000000.01800000 2023-11-08T16:41:51.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.362+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:92 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386b11800 2023-11-08T16:41:51.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.364+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 13 ==== osd_op_reply(92 10000000000.01800000 [omap-get-vals out=24452b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+24452 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-11-08T16:41:51.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000202 for future processing to fix dnfirst 2023-11-08T16:41:51.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000203 for future processing to fix dnfirst 2023-11-08T16:41:51.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000205 for future processing to fix dnfirst 2023-11-08T16:41:51.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000206 for future processing to fix dnfirst 2023-11-08T16:41:51.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000207 for future processing to fix dnfirst 2023-11-08T16:41:51.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000020b for future processing to fix dnfirst 2023-11-08T16:41:51.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000020c for future processing to fix dnfirst 2023-11-08T16:41:51.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000020d for future processing to fix dnfirst 2023-11-08T16:41:51.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000020e for future processing to fix dnfirst 2023-11-08T16:41:51.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000020f for future processing to fix dnfirst 2023-11-08T16:41:51.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000212 for future processing to fix dnfirst 2023-11-08T16:41:51.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000213 for future processing to fix dnfirst 2023-11-08T16:41:51.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000215 for future processing to fix dnfirst 2023-11-08T16:41:51.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000216 for future processing to fix dnfirst 2023-11-08T16:41:51.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000218 for future processing to fix dnfirst 2023-11-08T16:41:51.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000021a for future processing to fix dnfirst 2023-11-08T16:41:51.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000021d for future processing to fix dnfirst 2023-11-08T16:41:51.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000021e for future processing to fix dnfirst 2023-11-08T16:41:51.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000221 for future processing to fix dnfirst 2023-11-08T16:41:51.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000222 for future processing to fix dnfirst 2023-11-08T16:41:51.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000224 for future processing to fix dnfirst 2023-11-08T16:41:51.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000225 for future processing to fix dnfirst 2023-11-08T16:41:51.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000226 for future processing to fix dnfirst 2023-11-08T16:41:51.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000228 for future processing to fix dnfirst 2023-11-08T16:41:51.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000022b for future processing to fix dnfirst 2023-11-08T16:41:51.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000022d for future processing to fix dnfirst 2023-11-08T16:41:51.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000022e for future processing to fix dnfirst 2023-11-08T16:41:51.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000022f for future processing to fix dnfirst 2023-11-08T16:41:51.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x100000001fa for future processing to fix dnfirst 2023-11-08T16:41:51.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000231 for future processing to fix dnfirst 2023-11-08T16:41:51.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000233 for future processing to fix dnfirst 2023-11-08T16:41:51.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000238 for future processing to fix dnfirst 2023-11-08T16:41:51.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000239 for future processing to fix dnfirst 2023-11-08T16:41:51.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.428 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x100000001fb for future processing to fix dnfirst 2023-11-08T16:41:51.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000023c for future processing to fix dnfirst 2023-11-08T16:41:51.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000023d for future processing to fix dnfirst 2023-11-08T16:41:51.429 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000241 for future processing to fix dnfirst 2023-11-08T16:41:51.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000243 for future processing to fix dnfirst 2023-11-08T16:41:51.430 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000244 for future processing to fix dnfirst 2023-11-08T16:41:51.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x100000001fc for future processing to fix dnfirst 2023-11-08T16:41:51.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.431 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000246 for future processing to fix dnfirst 2023-11-08T16:41:51.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000247 for future processing to fix dnfirst 2023-11-08T16:41:51.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.432 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x1000000024e for future processing to fix dnfirst 2023-11-08T16:41:51.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000250 for future processing to fix dnfirst 2023-11-08T16:41:51.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000252 for future processing to fix dnfirst 2023-11-08T16:41:51.433 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000253 for future processing to fix dnfirst 2023-11-08T16:41:51.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000257 for future processing to fix dnfirst 2023-11-08T16:41:51.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: injected ino detected 2023-11-08T16:41:51.434 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 20 datascan.scan_links: adding 0x10000000258 for future processing to fix dnfirst 2023-11-08T16:41:51.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 609.00000000 2023-11-08T16:41:51.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.365+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- osd_op(unknown.0.0:93 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386b11400 2023-11-08T16:41:51.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.366+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.3 v2:172.21.15.81:6816/2756229107 10 ==== osd_op_reply(93 609.00000000 [omap-get-vals out=5b] v0'0 uv2 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x562386a82b40 con 0x562386b11400 2023-11-08T16:41:51.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.366+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 400.00000000 2023-11-08T16:41:51.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.366+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2023-11-08T16:41:51.435 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.366+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-11-08T16:41:51.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.366+0000 7f5f9aa385c0 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-11-08T16:41:51.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.366+0000 7f5f9aa385c0 10 datascan.scan_links: step 2: handling object 1.00000000 2023-11-08T16:41:51.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.366+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386b11800 2023-11-08T16:41:51.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 14 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=532b] v0'0 uv6 ondisk = 0) v8 ==== 154+0+532 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.436 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: processing 99 dup_primaries, 0 remote_links 2023-11-08T16:41:51.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000001 2023-11-08T16:41:51.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x100000001f6 2023-11-08T16:41:51.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x100000001f7 2023-11-08T16:41:51.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x100000001f8 2023-11-08T16:41:51.437 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x100000001f9 2023-11-08T16:41:51.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x100000001fa 2023-11-08T16:41:51.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x100000001fb 2023-11-08T16:41:51.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x100000001fc 2023-11-08T16:41:51.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x100000001fd 2023-11-08T16:41:51.438 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x100000001fe 2023-11-08T16:41:51.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x100000001ff 2023-11-08T16:41:51.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000200 2023-11-08T16:41:51.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000201 2023-11-08T16:41:51.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000202 2023-11-08T16:41:51.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000203 2023-11-08T16:41:51.439 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000204 2023-11-08T16:41:51.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000205 2023-11-08T16:41:51.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000206 2023-11-08T16:41:51.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000207 2023-11-08T16:41:51.440 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000208 2023-11-08T16:41:51.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000209 2023-11-08T16:41:51.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000020b 2023-11-08T16:41:51.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000020c 2023-11-08T16:41:51.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000020d 2023-11-08T16:41:51.441 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000020e 2023-11-08T16:41:51.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000020f 2023-11-08T16:41:51.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000210 2023-11-08T16:41:51.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000211 2023-11-08T16:41:51.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000212 2023-11-08T16:41:51.442 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000213 2023-11-08T16:41:51.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000214 2023-11-08T16:41:51.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000215 2023-11-08T16:41:51.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000216 2023-11-08T16:41:51.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000217 2023-11-08T16:41:51.443 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000218 2023-11-08T16:41:51.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000219 2023-11-08T16:41:51.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000021a 2023-11-08T16:41:51.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000021b 2023-11-08T16:41:51.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000021c 2023-11-08T16:41:51.444 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000021d 2023-11-08T16:41:51.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000021e 2023-11-08T16:41:51.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000021f 2023-11-08T16:41:51.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000220 2023-11-08T16:41:51.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000221 2023-11-08T16:41:51.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000222 2023-11-08T16:41:51.445 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000223 2023-11-08T16:41:51.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000224 2023-11-08T16:41:51.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000225 2023-11-08T16:41:51.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000226 2023-11-08T16:41:51.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000227 2023-11-08T16:41:51.446 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000228 2023-11-08T16:41:51.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000229 2023-11-08T16:41:51.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000022a 2023-11-08T16:41:51.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000022b 2023-11-08T16:41:51.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000022c 2023-11-08T16:41:51.447 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000022d 2023-11-08T16:41:51.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000022e 2023-11-08T16:41:51.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000022f 2023-11-08T16:41:51.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000230 2023-11-08T16:41:51.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000231 2023-11-08T16:41:51.448 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000232 2023-11-08T16:41:51.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000233 2023-11-08T16:41:51.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000234 2023-11-08T16:41:51.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000235 2023-11-08T16:41:51.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000236 2023-11-08T16:41:51.449 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000237 2023-11-08T16:41:51.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000238 2023-11-08T16:41:51.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000239 2023-11-08T16:41:51.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000023a 2023-11-08T16:41:51.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000023b 2023-11-08T16:41:51.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000023c 2023-11-08T16:41:51.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000023d 2023-11-08T16:41:51.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000023e 2023-11-08T16:41:51.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000023f 2023-11-08T16:41:51.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000240 2023-11-08T16:41:51.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000241 2023-11-08T16:41:51.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000242 2023-11-08T16:41:51.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000243 2023-11-08T16:41:51.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000244 2023-11-08T16:41:51.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000245 2023-11-08T16:41:51.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000246 2023-11-08T16:41:51.452 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000247 2023-11-08T16:41:51.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000248 2023-11-08T16:41:51.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000249 2023-11-08T16:41:51.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000024a 2023-11-08T16:41:51.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000024b 2023-11-08T16:41:51.453 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000024c 2023-11-08T16:41:51.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000024d 2023-11-08T16:41:51.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000024e 2023-11-08T16:41:51.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x1000000024f 2023-11-08T16:41:51.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000250 2023-11-08T16:41:51.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000251 2023-11-08T16:41:51.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000252 2023-11-08T16:41:51.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000253 2023-11-08T16:41:51.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000254 2023-11-08T16:41:51.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000255 2023-11-08T16:41:51.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000256 2023-11-08T16:41:51.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000257 2023-11-08T16:41:51.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling dup 0x10000000258 2023-11-08T16:41:51.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: removing dup dentries from 0 objects 2023-11-08T16:41:51.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-11-08T16:41:51.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: processing 100 injected_inos 2023-11-08T16:41:51.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-11-08T16:41:51.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:0 2023-11-08T16:41:51.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.367+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:95 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386b11800 2023-11-08T16:41:51.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.368+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 15 ==== osd_op_reply(95 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv55 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.369+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=0 frag=1*, r=0 2023-11-08T16:41:51.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.369+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.369+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:96 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386b11800 2023-11-08T16:41:51.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.373+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 16 ==== osd_op_reply(96 10000000000.01800000 [omap-set-vals] v131'56 uv56 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.373+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/0 pointing to 0x0x10000000001 2023-11-08T16:41:51.458 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.373+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x100000001f6 2023-11-08T16:41:51.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.373+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:1 2023-11-08T16:41:51.459 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.373+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:97 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.374+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 21 ==== osd_op_reply(97 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.466 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.374+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=1 frag=0*, r=0 2023-11-08T16:41:51.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.374+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.374+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:98 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.377+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 22 ==== osd_op_reply(98 10000000000.01000000 [omap-set-vals] v131'57 uv57 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.377+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/1 pointing to 0x0x100000001f6 2023-11-08T16:41:51.467 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.377+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x100000001f7 2023-11-08T16:41:51.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.377+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:2 2023-11-08T16:41:51.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.377+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:99 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386af6c00 2023-11-08T16:41:51.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.378+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 23 ==== osd_op_reply(99 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv57 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.378+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=2 frag=0*, r=0 2023-11-08T16:41:51.468 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.378+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.378+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:100 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386af6c00 2023-11-08T16:41:51.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.381+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 24 ==== osd_op_reply(100 10000000000.01000000 [omap-set-vals] v131'58 uv58 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.381+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/2 pointing to 0x0x100000001f7 2023-11-08T16:41:51.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.381+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x100000001f8 2023-11-08T16:41:51.469 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.381+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:3 2023-11-08T16:41:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.381+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:101 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386af6c00 2023-11-08T16:41:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.382+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 25 ==== osd_op_reply(101 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv58 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.382+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=3 frag=0*, r=0 2023-11-08T16:41:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.382+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.470 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.382+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:102 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.386+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 26 ==== osd_op_reply(102 10000000000.01000000 [omap-set-vals] v131'59 uv59 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.386+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/3 pointing to 0x0x100000001f8 2023-11-08T16:41:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.386+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x100000001f9 2023-11-08T16:41:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.386+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:4 2023-11-08T16:41:51.471 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.386+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:103 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386af6c00 2023-11-08T16:41:51.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.387+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 27 ==== osd_op_reply(103 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv59 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.387+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=4 frag=0*, r=0 2023-11-08T16:41:51.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.387+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.387+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:104 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386af6c00 2023-11-08T16:41:51.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.391+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 28 ==== osd_op_reply(104 10000000000.01000000 [omap-set-vals] v131'60 uv60 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.391+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/4 pointing to 0x0x100000001f9 2023-11-08T16:41:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.391+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x100000001fa 2023-11-08T16:41:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.391+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:5 2023-11-08T16:41:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.391+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:105 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11800 2023-11-08T16:41:51.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.392+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 17 ==== osd_op_reply(105 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv56 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.392+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=5 frag=1*, r=0 2023-11-08T16:41:51.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.392+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.392+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:106 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386b11800 2023-11-08T16:41:51.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.396+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 18 ==== osd_op_reply(106 10000000000.01800000 [omap-set-vals] v131'57 uv57 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.396+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/5 pointing to 0x0x100000001fa 2023-11-08T16:41:51.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.396+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x100000001fb 2023-11-08T16:41:51.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.396+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:6 2023-11-08T16:41:51.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.396+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:107 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386b11800 2023-11-08T16:41:51.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.397+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 19 ==== osd_op_reply(107 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv57 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.397+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=6 frag=1*, r=0 2023-11-08T16:41:51.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.397+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.397+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:108 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016000 con 0x562386b11800 2023-11-08T16:41:51.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.401+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 20 ==== osd_op_reply(108 10000000000.01800000 [omap-set-vals] v131'58 uv58 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.401+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/6 pointing to 0x0x100000001fb 2023-11-08T16:41:51.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.401+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x100000001fc 2023-11-08T16:41:51.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.401+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:7 2023-11-08T16:41:51.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.401+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:109 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016380 con 0x562386b11800 2023-11-08T16:41:51.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.402+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 21 ==== osd_op_reply(109 10000000000.01800000 [omap-get-vals-by-keys out=502b] v0'0 uv58 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.402+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=7 frag=1*, r=0 2023-11-08T16:41:51.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.402+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.402+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:110 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016700 con 0x562386b11800 2023-11-08T16:41:51.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.406+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 22 ==== osd_op_reply(110 10000000000.01800000 [omap-set-vals] v131'59 uv59 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.407+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/7 pointing to 0x0x100000001fc 2023-11-08T16:41:51.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.407+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x100000001fd 2023-11-08T16:41:51.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.407+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:8 2023-11-08T16:41:51.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.407+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:111 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016a80 con 0x562386af6c00 2023-11-08T16:41:51.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.407+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 29 ==== osd_op_reply(111 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv60 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.408+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=8 frag=0*, r=0 2023-11-08T16:41:51.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.408+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.408+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:112 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016e00 con 0x562386af6c00 2023-11-08T16:41:51.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.411+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 30 ==== osd_op_reply(112 10000000000.01000000 [omap-set-vals] v131'61 uv61 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.411+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/8 pointing to 0x0x100000001fd 2023-11-08T16:41:51.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.411+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x100000001fe 2023-11-08T16:41:51.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.411+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:9 2023-11-08T16:41:51.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.411+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:113 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=14b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017180 con 0x562386af6c00 2023-11-08T16:41:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.412+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 31 ==== osd_op_reply(113 10000000000.01000000 [omap-get-vals-by-keys out=502b] v0'0 uv61 ondisk = 0) v8 ==== 164+0+502 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.412+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=9 frag=0*, r=0 2023-11-08T16:41:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.412+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.412+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:114 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017500 con 0x562386af6c00 2023-11-08T16:41:51.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.416+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 32 ==== osd_op_reply(114 10000000000.01000000 [omap-set-vals] v131'62 uv62 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.416+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/9 pointing to 0x0x100000001fe 2023-11-08T16:41:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.416+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x100000001ff 2023-11-08T16:41:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.416+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:10 2023-11-08T16:41:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.416+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:115 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017880 con 0x562386af6c00 2023-11-08T16:41:51.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.417+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 33 ==== osd_op_reply(115 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv62 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.417+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=10 frag=0*, r=0 2023-11-08T16:41:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.417+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.417+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:116 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017c00 con 0x562386af6c00 2023-11-08T16:41:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.420+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 34 ==== osd_op_reply(116 10000000000.01000000 [omap-set-vals] v131'63 uv63 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.420+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/10 pointing to 0x0x100000001ff 2023-11-08T16:41:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.420+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000200 2023-11-08T16:41:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.420+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:11 2023-11-08T16:41:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.420+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:117 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016a80 con 0x562386af6c00 2023-11-08T16:41:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.421+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 35 ==== osd_op_reply(117 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv63 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.421+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=11 frag=0*, r=0 2023-11-08T16:41:51.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.421+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.421+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:118 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016e00 con 0x562386af6c00 2023-11-08T16:41:51.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.425+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 36 ==== osd_op_reply(118 10000000000.01000000 [omap-set-vals] v131'64 uv64 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.425+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/11 pointing to 0x0x10000000200 2023-11-08T16:41:51.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.425+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000201 2023-11-08T16:41:51.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.425+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:12 2023-11-08T16:41:51.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.425+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:119 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017180 con 0x562386af6c00 2023-11-08T16:41:51.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.426+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 37 ==== osd_op_reply(119 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv64 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.426+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=12 frag=0*, r=0 2023-11-08T16:41:51.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.426+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.426+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:120 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017500 con 0x562386af6c00 2023-11-08T16:41:51.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.430+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 38 ==== osd_op_reply(120 10000000000.01000000 [omap-set-vals] v131'65 uv65 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.430+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/12 pointing to 0x0x10000000201 2023-11-08T16:41:51.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.430+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000202 2023-11-08T16:41:51.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.430+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:13 2023-11-08T16:41:51.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.430+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:121 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016000 con 0x562386b11800 2023-11-08T16:41:51.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.431+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 23 ==== osd_op_reply(121 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv59 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.431+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=13 frag=1*, r=0 2023-11-08T16:41:51.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.431+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.431+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:122 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386b11800 2023-11-08T16:41:51.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.435+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 24 ==== osd_op_reply(122 10000000000.01800000 [omap-set-vals] v131'60 uv60 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.435+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/13 pointing to 0x0x10000000202 2023-11-08T16:41:51.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.435+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000203 2023-11-08T16:41:51.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.435+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:14 2023-11-08T16:41:51.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.435+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:123 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386b11800 2023-11-08T16:41:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.436+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 25 ==== osd_op_reply(123 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv60 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.436+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=14 frag=1*, r=0 2023-11-08T16:41:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.436+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.436+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:124 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11800 2023-11-08T16:41:51.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.440+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 26 ==== osd_op_reply(124 10000000000.01800000 [omap-set-vals] v131'61 uv61 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.440+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/14 pointing to 0x0x10000000203 2023-11-08T16:41:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.440+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000204 2023-11-08T16:41:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.440+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:15 2023-11-08T16:41:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.440+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:125 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386af6c00 2023-11-08T16:41:51.491 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.441+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 39 ==== osd_op_reply(125 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv65 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.441+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=15 frag=0*, r=0 2023-11-08T16:41:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.441+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.441+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:126 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386af6c00 2023-11-08T16:41:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.444+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 40 ==== osd_op_reply(126 10000000000.01000000 [omap-set-vals] v131'66 uv66 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.492 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.444+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/15 pointing to 0x0x10000000204 2023-11-08T16:41:51.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.444+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000205 2023-11-08T16:41:51.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.444+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:16 2023-11-08T16:41:51.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.444+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:127 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11800 2023-11-08T16:41:51.493 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.446+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 27 ==== osd_op_reply(127 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv61 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.446+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=16 frag=1*, r=0 2023-11-08T16:41:51.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.446+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.446+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:128 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386b11800 2023-11-08T16:41:51.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.450+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 28 ==== osd_op_reply(128 10000000000.01800000 [omap-set-vals] v131'62 uv62 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.494 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.450+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/16 pointing to 0x0x10000000205 2023-11-08T16:41:51.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.450+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000206 2023-11-08T16:41:51.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.450+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:17 2023-11-08T16:41:51.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.450+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:129 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386b11800 2023-11-08T16:41:51.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.451+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 29 ==== osd_op_reply(129 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv62 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.495 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.452+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=17 frag=1*, r=0 2023-11-08T16:41:51.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.452+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.452+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:130 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386b11800 2023-11-08T16:41:51.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.455+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 30 ==== osd_op_reply(130 10000000000.01800000 [omap-set-vals] v131'63 uv63 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.456+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/17 pointing to 0x0x10000000206 2023-11-08T16:41:51.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.456+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000207 2023-11-08T16:41:51.496 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.456+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:18 2023-11-08T16:41:51.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.456+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:131 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386b11800 2023-11-08T16:41:51.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.457+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 31 ==== osd_op_reply(131 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv63 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.457+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=18 frag=1*, r=0 2023-11-08T16:41:51.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.457+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.497 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.457+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:132 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386b11800 2023-11-08T16:41:51.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.461+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 32 ==== osd_op_reply(132 10000000000.01800000 [omap-set-vals] v131'64 uv64 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.461+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/18 pointing to 0x0x10000000207 2023-11-08T16:41:51.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.461+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000208 2023-11-08T16:41:51.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.461+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:19 2023-11-08T16:41:51.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.461+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:133 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.462+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 41 ==== osd_op_reply(133 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv66 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.462+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=19 frag=0*, r=0 2023-11-08T16:41:51.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.462+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.462+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:134 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016000 con 0x562386af6c00 2023-11-08T16:41:51.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.465+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 42 ==== osd_op_reply(134 10000000000.01000000 [omap-set-vals] v131'67 uv67 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.466+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/19 pointing to 0x0x10000000208 2023-11-08T16:41:51.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.466+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000209 2023-11-08T16:41:51.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.466+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:20 2023-11-08T16:41:51.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.466+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:135 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016380 con 0x562386af6c00 2023-11-08T16:41:51.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.467+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 43 ==== osd_op_reply(135 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv67 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.467+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=20 frag=0*, r=0 2023-11-08T16:41:51.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.467+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.467+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:136 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016700 con 0x562386af6c00 2023-11-08T16:41:51.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.471+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 44 ==== osd_op_reply(136 10000000000.01000000 [omap-set-vals] v131'68 uv68 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.471+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/20 pointing to 0x0x10000000209 2023-11-08T16:41:51.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.471+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000020a 2023-11-08T16:41:51.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.471+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:21 2023-11-08T16:41:51.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.471+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:137 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016a80 con 0x562386af6c00 2023-11-08T16:41:51.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.472+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 45 ==== osd_op_reply(137 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv68 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.472+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=21 frag=0*, r=0 2023-11-08T16:41:51.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.472+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.472+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:138 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016e00 con 0x562386af6c00 2023-11-08T16:41:51.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.475+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 46 ==== osd_op_reply(138 10000000000.01000000 [omap-set-vals] v131'69 uv69 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.476+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/21 pointing to 0x0x1000000020a 2023-11-08T16:41:51.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.476+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000020b 2023-11-08T16:41:51.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.476+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:22 2023-11-08T16:41:51.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.476+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:139 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017180 con 0x562386b11800 2023-11-08T16:41:51.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.477+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 33 ==== osd_op_reply(139 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv64 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.477+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=22 frag=1*, r=0 2023-11-08T16:41:51.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.477+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.508 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.477+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:140 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017500 con 0x562386b11800 2023-11-08T16:41:51.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.481+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 34 ==== osd_op_reply(140 10000000000.01800000 [omap-set-vals] v131'65 uv65 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.481+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/22 pointing to 0x0x1000000020b 2023-11-08T16:41:51.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.481+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000020c 2023-11-08T16:41:51.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.481+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:23 2023-11-08T16:41:51.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.481+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:141 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017880 con 0x562386b11800 2023-11-08T16:41:51.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.482+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 35 ==== osd_op_reply(141 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv65 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.482+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=23 frag=1*, r=0 2023-11-08T16:41:51.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.482+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.482+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:142 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017c00 con 0x562386b11800 2023-11-08T16:41:51.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.486+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 36 ==== osd_op_reply(142 10000000000.01800000 [omap-set-vals] v131'66 uv66 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.486+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/23 pointing to 0x0x1000000020c 2023-11-08T16:41:51.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.486+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000020d 2023-11-08T16:41:51.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.486+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:24 2023-11-08T16:41:51.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.486+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:143 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017180 con 0x562386b11800 2023-11-08T16:41:51.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.487+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 37 ==== osd_op_reply(143 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv66 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.487+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=24 frag=1*, r=0 2023-11-08T16:41:51.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.487+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.487+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:144 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386b11800 2023-11-08T16:41:51.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.490+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 38 ==== osd_op_reply(144 10000000000.01800000 [omap-set-vals] v131'67 uv67 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.491+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/24 pointing to 0x0x1000000020d 2023-11-08T16:41:51.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.491+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000020e 2023-11-08T16:41:51.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.491+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:25 2023-11-08T16:41:51.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.491+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:145 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386b11800 2023-11-08T16:41:51.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.492+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 39 ==== osd_op_reply(145 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv67 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.492+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=25 frag=1*, r=0 2023-11-08T16:41:51.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.492+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.492+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:146 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386b11800 2023-11-08T16:41:51.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.495+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 40 ==== osd_op_reply(146 10000000000.01800000 [omap-set-vals] v131'68 uv68 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.496+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/25 pointing to 0x0x1000000020e 2023-11-08T16:41:51.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.496+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000020f 2023-11-08T16:41:51.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.496+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:26 2023-11-08T16:41:51.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.496+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:147 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386b11800 2023-11-08T16:41:51.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.496+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 41 ==== osd_op_reply(147 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv68 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.497+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=26 frag=1*, r=0 2023-11-08T16:41:51.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.497+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.497+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:148 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11800 2023-11-08T16:41:51.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.500+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 42 ==== osd_op_reply(148 10000000000.01800000 [omap-set-vals] v131'69 uv69 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.500+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/26 pointing to 0x0x1000000020f 2023-11-08T16:41:51.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.500+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000210 2023-11-08T16:41:51.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.500+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:27 2023-11-08T16:41:51.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.500+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:149 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386af6c00 2023-11-08T16:41:51.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.501+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 47 ==== osd_op_reply(149 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv69 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.501+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=27 frag=0*, r=0 2023-11-08T16:41:51.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.501+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.501+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:150 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386af6c00 2023-11-08T16:41:51.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.505+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 48 ==== osd_op_reply(150 10000000000.01000000 [omap-set-vals] v131'70 uv70 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.505+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/27 pointing to 0x0x10000000210 2023-11-08T16:41:51.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.505+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000211 2023-11-08T16:41:51.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.505+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:28 2023-11-08T16:41:51.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.505+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:151 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386af6c00 2023-11-08T16:41:51.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.506+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 49 ==== osd_op_reply(151 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv70 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.506+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=28 frag=0*, r=0 2023-11-08T16:41:51.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.506+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.520 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.506+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:152 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386af6c00 2023-11-08T16:41:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.509+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 50 ==== osd_op_reply(152 10000000000.01000000 [omap-set-vals] v131'71 uv71 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.510+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/28 pointing to 0x0x10000000211 2023-11-08T16:41:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.510+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000212 2023-11-08T16:41:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.510+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:29 2023-11-08T16:41:51.521 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.510+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:153 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386b11800 2023-11-08T16:41:51.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.511+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 43 ==== osd_op_reply(153 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv69 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.511+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=29 frag=1*, r=0 2023-11-08T16:41:51.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.511+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.511+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:154 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017500 con 0x562386b11800 2023-11-08T16:41:51.522 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.515+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 44 ==== osd_op_reply(154 10000000000.01800000 [omap-set-vals] v131'70 uv70 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.515+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/29 pointing to 0x0x10000000212 2023-11-08T16:41:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.515+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000213 2023-11-08T16:41:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.515+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:30 2023-11-08T16:41:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.515+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:155 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017880 con 0x562386b11800 2023-11-08T16:41:51.523 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.516+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 45 ==== osd_op_reply(155 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv70 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.516+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=30 frag=1*, r=0 2023-11-08T16:41:51.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.516+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.524 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.516+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:156 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017c00 con 0x562386b11800 2023-11-08T16:41:51.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.520+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 46 ==== osd_op_reply(156 10000000000.01800000 [omap-set-vals] v131'71 uv71 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.520+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/30 pointing to 0x0x10000000213 2023-11-08T16:41:51.525 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.520+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000214 2023-11-08T16:41:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.520+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:31 2023-11-08T16:41:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.520+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:157 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017180 con 0x562386af6c00 2023-11-08T16:41:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.521+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 51 ==== osd_op_reply(157 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv71 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.521+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=31 frag=0*, r=0 2023-11-08T16:41:51.526 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.521+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.527 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.521+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:158 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.525+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 52 ==== osd_op_reply(158 10000000000.01000000 [omap-set-vals] v131'72 uv72 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.525+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/31 pointing to 0x0x10000000214 2023-11-08T16:41:51.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.525+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000215 2023-11-08T16:41:51.528 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.525+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:32 2023-11-08T16:41:51.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.525+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:159 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386b11800 2023-11-08T16:41:51.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.526+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 47 ==== osd_op_reply(159 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv71 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.526+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=32 frag=1*, r=0 2023-11-08T16:41:51.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.526+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.529 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.526+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:160 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386b11800 2023-11-08T16:41:51.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.529+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 48 ==== osd_op_reply(160 10000000000.01800000 [omap-set-vals] v131'72 uv72 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.529+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/32 pointing to 0x0x10000000215 2023-11-08T16:41:51.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.529+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000216 2023-11-08T16:41:51.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.529+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:33 2023-11-08T16:41:51.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.529+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:161 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386b11800 2023-11-08T16:41:51.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.531+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 49 ==== osd_op_reply(161 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv72 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.532 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.531+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=33 frag=1*, r=0 2023-11-08T16:41:51.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.531+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.531+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:162 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016000 con 0x562386b11800 2023-11-08T16:41:51.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.534+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 50 ==== osd_op_reply(162 10000000000.01800000 [omap-set-vals] v131'73 uv73 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.534+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/33 pointing to 0x0x10000000216 2023-11-08T16:41:51.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.534+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000217 2023-11-08T16:41:51.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.534+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:34 2023-11-08T16:41:51.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.534+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:163 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016380 con 0x562386af6c00 2023-11-08T16:41:51.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.535+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 53 ==== osd_op_reply(163 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv72 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.535+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=34 frag=0*, r=0 2023-11-08T16:41:51.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.535+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.536+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:164 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016700 con 0x562386af6c00 2023-11-08T16:41:51.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.540+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 54 ==== osd_op_reply(164 10000000000.01000000 [omap-set-vals] v131'73 uv73 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.540+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/34 pointing to 0x0x10000000217 2023-11-08T16:41:51.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.540+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000218 2023-11-08T16:41:51.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.540+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:35 2023-11-08T16:41:51.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.540+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:165 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016a80 con 0x562386b11800 2023-11-08T16:41:51.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.541+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 51 ==== osd_op_reply(165 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv73 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.541+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=35 frag=1*, r=0 2023-11-08T16:41:51.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.541+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.541+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:166 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016e00 con 0x562386b11800 2023-11-08T16:41:51.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.544+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 52 ==== osd_op_reply(166 10000000000.01800000 [omap-set-vals] v131'74 uv74 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.544+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/35 pointing to 0x0x10000000218 2023-11-08T16:41:51.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.544+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000219 2023-11-08T16:41:51.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.544+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:36 2023-11-08T16:41:51.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.544+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:167 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017500 con 0x562386af6c00 2023-11-08T16:41:51.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.545+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 55 ==== osd_op_reply(167 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv73 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.547 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.545+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=36 frag=0*, r=0 2023-11-08T16:41:51.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.545+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.545+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:168 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017880 con 0x562386af6c00 2023-11-08T16:41:51.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.549+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 56 ==== osd_op_reply(168 10000000000.01000000 [omap-set-vals] v131'74 uv74 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.549+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/36 pointing to 0x0x10000000219 2023-11-08T16:41:51.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.549+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000021a 2023-11-08T16:41:51.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.549+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:37 2023-11-08T16:41:51.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.549+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:169 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017c00 con 0x562386b11800 2023-11-08T16:41:51.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.550+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 53 ==== osd_op_reply(169 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv74 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.550+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=37 frag=1*, r=0 2023-11-08T16:41:51.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.550+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.550+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:170 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016000 con 0x562386b11800 2023-11-08T16:41:51.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.553+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 54 ==== osd_op_reply(170 10000000000.01800000 [omap-set-vals] v131'75 uv75 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.553+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/37 pointing to 0x0x1000000021a 2023-11-08T16:41:51.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.553+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000021b 2023-11-08T16:41:51.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.553+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:38 2023-11-08T16:41:51.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.553+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:171 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.554+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 57 ==== osd_op_reply(171 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv74 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.554+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=38 frag=0*, r=0 2023-11-08T16:41:51.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.554+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.554+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:172 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386af6c00 2023-11-08T16:41:51.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.558+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 58 ==== osd_op_reply(172 10000000000.01000000 [omap-set-vals] v131'75 uv75 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.558+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/38 pointing to 0x0x1000000021b 2023-11-08T16:41:51.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.558+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000021c 2023-11-08T16:41:51.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.558+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:39 2023-11-08T16:41:51.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.558+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:173 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386af6c00 2023-11-08T16:41:51.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.559+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 59 ==== osd_op_reply(173 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv75 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.559+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=39 frag=0*, r=0 2023-11-08T16:41:51.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.559+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.559+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:174 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386af6c00 2023-11-08T16:41:51.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.562+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 60 ==== osd_op_reply(174 10000000000.01000000 [omap-set-vals] v131'76 uv76 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.562+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/39 pointing to 0x0x1000000021c 2023-11-08T16:41:51.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.562+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000021d 2023-11-08T16:41:51.564 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.562+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:40 2023-11-08T16:41:51.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.562+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:175 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386b11800 2023-11-08T16:41:51.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.564+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 55 ==== osd_op_reply(175 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv75 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.565 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.564+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=40 frag=1*, r=0 2023-11-08T16:41:51.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.564+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.566 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.564+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:176 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386b11800 2023-11-08T16:41:51.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.568+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 56 ==== osd_op_reply(176 10000000000.01800000 [omap-set-vals] v131'76 uv76 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.569 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.568+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/40 pointing to 0x0x1000000021d 2023-11-08T16:41:51.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.568+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000021e 2023-11-08T16:41:51.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.568+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:41 2023-11-08T16:41:51.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.568+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:177 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11800 2023-11-08T16:41:51.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.569+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 57 ==== osd_op_reply(177 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv76 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.569+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=41 frag=1*, r=0 2023-11-08T16:41:51.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.569+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.569+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:178 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386b11800 2023-11-08T16:41:51.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.573+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 58 ==== osd_op_reply(178 10000000000.01800000 [omap-set-vals] v131'77 uv77 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.573+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/41 pointing to 0x0x1000000021e 2023-11-08T16:41:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.573+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000021f 2023-11-08T16:41:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.573+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:42 2023-11-08T16:41:51.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.573+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:179 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.574+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 61 ==== osd_op_reply(179 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv76 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.574+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=42 frag=0*, r=0 2023-11-08T16:41:51.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.574+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.574+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:180 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386af6c00 2023-11-08T16:41:51.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.577+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 62 ==== osd_op_reply(180 10000000000.01000000 [omap-set-vals] v131'77 uv77 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.577+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/42 pointing to 0x0x1000000021f 2023-11-08T16:41:51.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.577+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000220 2023-11-08T16:41:51.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.577+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:43 2023-11-08T16:41:51.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.577+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:181 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017180 con 0x562386af6c00 2023-11-08T16:41:51.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.578+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 63 ==== osd_op_reply(181 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv77 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.578+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=43 frag=0*, r=0 2023-11-08T16:41:51.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.578+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.578+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:182 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016380 con 0x562386af6c00 2023-11-08T16:41:51.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.582+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 64 ==== osd_op_reply(182 10000000000.01000000 [omap-set-vals] v131'78 uv78 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.582+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/43 pointing to 0x0x10000000220 2023-11-08T16:41:51.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.582+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000221 2023-11-08T16:41:51.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.582+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:44 2023-11-08T16:41:51.584 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.582+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:183 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016700 con 0x562386b11800 2023-11-08T16:41:51.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.583+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 59 ==== osd_op_reply(183 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv77 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.583+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=44 frag=1*, r=0 2023-11-08T16:41:51.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.583+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.583+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:184 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017500 con 0x562386b11800 2023-11-08T16:41:51.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.586+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 60 ==== osd_op_reply(184 10000000000.01800000 [omap-set-vals] v131'78 uv78 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.586+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/44 pointing to 0x0x10000000221 2023-11-08T16:41:51.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.586+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000222 2023-11-08T16:41:51.588 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.586+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:45 2023-11-08T16:41:51.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.586+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:185 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017880 con 0x562386b11800 2023-11-08T16:41:51.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.588+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 61 ==== osd_op_reply(185 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv78 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.588+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=45 frag=1*, r=0 2023-11-08T16:41:51.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.588+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.588+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:186 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016a80 con 0x562386b11800 2023-11-08T16:41:51.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.592+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 62 ==== osd_op_reply(186 10000000000.01800000 [omap-set-vals] v131'79 uv79 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.592+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/45 pointing to 0x0x10000000222 2023-11-08T16:41:51.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.592+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000223 2023-11-08T16:41:51.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.592+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:46 2023-11-08T16:41:51.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.592+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:187 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016e00 con 0x562386af6c00 2023-11-08T16:41:51.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.593+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 65 ==== osd_op_reply(187 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv78 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.593+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=46 frag=0*, r=0 2023-11-08T16:41:51.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.593+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.593+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:188 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017c00 con 0x562386af6c00 2023-11-08T16:41:51.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.597+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 66 ==== osd_op_reply(188 10000000000.01000000 [omap-set-vals] v131'79 uv79 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.597+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/46 pointing to 0x0x10000000223 2023-11-08T16:41:51.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.597+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000224 2023-11-08T16:41:51.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.597+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:47 2023-11-08T16:41:51.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.597+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:189 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016000 con 0x562386b11800 2023-11-08T16:41:51.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.598+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 63 ==== osd_op_reply(189 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv79 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.598+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=47 frag=1*, r=0 2023-11-08T16:41:51.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.598+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.598+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:190 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016700 con 0x562386b11800 2023-11-08T16:41:51.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.602+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 64 ==== osd_op_reply(190 10000000000.01800000 [omap-set-vals] v131'80 uv80 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.602+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/47 pointing to 0x0x10000000224 2023-11-08T16:41:51.603 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.602+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000225 2023-11-08T16:41:51.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.602+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:48 2023-11-08T16:41:51.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.602+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:191 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017500 con 0x562386b11800 2023-11-08T16:41:51.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.603+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 65 ==== osd_op_reply(191 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv80 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.603+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=48 frag=1*, r=0 2023-11-08T16:41:51.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.603+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.603+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:192 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386b11800 2023-11-08T16:41:51.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.606+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 66 ==== osd_op_reply(192 10000000000.01800000 [omap-set-vals] v131'81 uv81 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.607+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/48 pointing to 0x0x10000000225 2023-11-08T16:41:51.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.607+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000226 2023-11-08T16:41:51.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.607+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:49 2023-11-08T16:41:51.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.607+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:193 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386b11800 2023-11-08T16:41:51.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.608+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 67 ==== osd_op_reply(193 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv81 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.608+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=49 frag=1*, r=0 2023-11-08T16:41:51.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.608+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.608+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:194 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11800 2023-11-08T16:41:51.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.612+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 68 ==== osd_op_reply(194 10000000000.01800000 [omap-set-vals] v131'82 uv82 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.612+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/49 pointing to 0x0x10000000226 2023-11-08T16:41:51.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.612+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000227 2023-11-08T16:41:51.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.612+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:50 2023-11-08T16:41:51.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.612+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:195 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386af6c00 2023-11-08T16:41:51.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.613+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 67 ==== osd_op_reply(195 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv79 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.613+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=50 frag=0*, r=0 2023-11-08T16:41:51.615 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.613+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.613+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:196 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046000 con 0x562386af6c00 2023-11-08T16:41:51.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.617+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 68 ==== osd_op_reply(196 10000000000.01000000 [omap-set-vals] v131'80 uv80 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.617+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/50 pointing to 0x0x10000000227 2023-11-08T16:41:51.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.617+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000228 2023-11-08T16:41:51.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.617+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:51 2023-11-08T16:41:51.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.617+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:197 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046380 con 0x562386b11800 2023-11-08T16:41:51.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.618+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 69 ==== osd_op_reply(197 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv82 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.618+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=51 frag=1*, r=0 2023-11-08T16:41:51.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.618+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.618+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:198 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046700 con 0x562386b11800 2023-11-08T16:41:51.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.622+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 70 ==== osd_op_reply(198 10000000000.01800000 [omap-set-vals] v131'83 uv83 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.622+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/51 pointing to 0x0x10000000228 2023-11-08T16:41:51.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.622+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000229 2023-11-08T16:41:51.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.622+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:52 2023-11-08T16:41:51.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.622+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:199 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046a80 con 0x562386af6c00 2023-11-08T16:41:51.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.623+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 69 ==== osd_op_reply(199 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv80 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.623+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=52 frag=0*, r=0 2023-11-08T16:41:51.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.623+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.625 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.623+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:200 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046e00 con 0x562386af6c00 2023-11-08T16:41:51.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.626+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 70 ==== osd_op_reply(200 10000000000.01000000 [omap-set-vals] v131'81 uv81 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.626+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/52 pointing to 0x0x10000000229 2023-11-08T16:41:51.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.626+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000022a 2023-11-08T16:41:51.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.626+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:53 2023-11-08T16:41:51.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.626+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:201 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047180 con 0x562386af6c00 2023-11-08T16:41:51.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.627+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 71 ==== osd_op_reply(201 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv81 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.627+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=53 frag=0*, r=0 2023-11-08T16:41:51.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.627+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.628+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:202 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047500 con 0x562386af6c00 2023-11-08T16:41:51.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.631+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 72 ==== osd_op_reply(202 10000000000.01000000 [omap-set-vals] v131'82 uv82 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.631+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/53 pointing to 0x0x1000000022a 2023-11-08T16:41:51.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.631+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000022b 2023-11-08T16:41:51.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.631+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:54 2023-11-08T16:41:51.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.632+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:203 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047880 con 0x562386b11800 2023-11-08T16:41:51.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.633+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 71 ==== osd_op_reply(203 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv83 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.633+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=54 frag=1*, r=0 2023-11-08T16:41:51.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.633+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.633+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:204 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047c00 con 0x562386b11800 2023-11-08T16:41:51.638 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.637+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 72 ==== osd_op_reply(204 10000000000.01800000 [omap-set-vals] v131'84 uv84 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.637+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/54 pointing to 0x0x1000000022b 2023-11-08T16:41:51.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.637+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000022c 2023-11-08T16:41:51.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.637+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:55 2023-11-08T16:41:51.639 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.637+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:205 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046380 con 0x562386af6c00 2023-11-08T16:41:51.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.638+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 73 ==== osd_op_reply(205 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv82 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.638+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=55 frag=0*, r=0 2023-11-08T16:41:51.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.638+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.640 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.638+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:206 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046700 con 0x562386af6c00 2023-11-08T16:41:51.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.642+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 74 ==== osd_op_reply(206 10000000000.01000000 [omap-set-vals] v131'83 uv83 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.642+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/55 pointing to 0x0x1000000022c 2023-11-08T16:41:51.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.642+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000022d 2023-11-08T16:41:51.643 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.642+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:56 2023-11-08T16:41:51.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.642+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:207 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047880 con 0x562386b11800 2023-11-08T16:41:51.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.643+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 73 ==== osd_op_reply(207 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv84 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.643+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=56 frag=1*, r=0 2023-11-08T16:41:51.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.643+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.643+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:208 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046000 con 0x562386b11800 2023-11-08T16:41:51.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.647+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 74 ==== osd_op_reply(208 10000000000.01800000 [omap-set-vals] v131'85 uv85 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.647+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/56 pointing to 0x0x1000000022d 2023-11-08T16:41:51.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.647+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000022e 2023-11-08T16:41:51.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.647+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:57 2023-11-08T16:41:51.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.647+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:209 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046a80 con 0x562386b11800 2023-11-08T16:41:51.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.648+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 75 ==== osd_op_reply(209 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv85 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.648+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=57 frag=1*, r=0 2023-11-08T16:41:51.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.648+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.648+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:210 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046e00 con 0x562386b11800 2023-11-08T16:41:51.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.651+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 76 ==== osd_op_reply(210 10000000000.01800000 [omap-set-vals] v131'86 uv86 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.651+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/57 pointing to 0x0x1000000022e 2023-11-08T16:41:51.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.651+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000022f 2023-11-08T16:41:51.653 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.651+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:58 2023-11-08T16:41:51.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.651+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:211 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386b11800 2023-11-08T16:41:51.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.653+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 77 ==== osd_op_reply(211 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv86 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.653+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=58 frag=1*, r=0 2023-11-08T16:41:51.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.653+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.653+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:212 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386b11800 2023-11-08T16:41:51.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.657+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 78 ==== osd_op_reply(212 10000000000.01800000 [omap-set-vals] v131'87 uv87 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.657+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/58 pointing to 0x0x1000000022f 2023-11-08T16:41:51.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.657+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000230 2023-11-08T16:41:51.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.657+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:59 2023-11-08T16:41:51.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.657+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:213 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.658+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 75 ==== osd_op_reply(213 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv83 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.658+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=59 frag=0*, r=0 2023-11-08T16:41:51.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.658+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.658+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:214 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386af6c00 2023-11-08T16:41:51.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.661+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 76 ==== osd_op_reply(214 10000000000.01000000 [omap-set-vals] v131'84 uv84 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.661+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/59 pointing to 0x0x10000000230 2023-11-08T16:41:51.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.661+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000231 2023-11-08T16:41:51.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.661+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:60 2023-11-08T16:41:51.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.661+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:215 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386b11800 2023-11-08T16:41:51.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.663+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 79 ==== osd_op_reply(215 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv87 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.663+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=60 frag=1*, r=0 2023-11-08T16:41:51.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.663+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.663+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:216 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386b11800 2023-11-08T16:41:51.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.667+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 80 ==== osd_op_reply(216 10000000000.01800000 [omap-set-vals] v131'88 uv88 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.667+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/60 pointing to 0x0x10000000231 2023-11-08T16:41:51.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.667+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000232 2023-11-08T16:41:51.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.667+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:61 2023-11-08T16:41:51.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.667+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:217 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386af6c00 2023-11-08T16:41:51.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.668+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 77 ==== osd_op_reply(217 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv84 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.668+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=61 frag=0*, r=0 2023-11-08T16:41:51.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.668+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.668+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:218 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016000 con 0x562386af6c00 2023-11-08T16:41:51.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.672+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 78 ==== osd_op_reply(218 10000000000.01000000 [omap-set-vals] v131'85 uv85 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.672+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/61 pointing to 0x0x10000000232 2023-11-08T16:41:51.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.672+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000233 2023-11-08T16:41:51.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.672+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:62 2023-11-08T16:41:51.674 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.672+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:219 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016380 con 0x562386b11800 2023-11-08T16:41:51.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.673+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 81 ==== osd_op_reply(219 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv88 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.675 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.673+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=62 frag=1*, r=0 2023-11-08T16:41:51.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.673+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.673+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:220 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016700 con 0x562386b11800 2023-11-08T16:41:51.678 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.677+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 82 ==== osd_op_reply(220 10000000000.01800000 [omap-set-vals] v131'89 uv89 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.677+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/62 pointing to 0x0x10000000233 2023-11-08T16:41:51.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.677+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000234 2023-11-08T16:41:51.679 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.677+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:63 2023-11-08T16:41:51.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.677+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:221 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016a80 con 0x562386af6c00 2023-11-08T16:41:51.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.678+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 79 ==== osd_op_reply(221 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv85 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.678+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=63 frag=0*, r=0 2023-11-08T16:41:51.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.678+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.678+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:222 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016e00 con 0x562386af6c00 2023-11-08T16:41:51.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.682+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 80 ==== osd_op_reply(222 10000000000.01000000 [omap-set-vals] v131'86 uv86 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.683 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.682+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/63 pointing to 0x0x10000000234 2023-11-08T16:41:51.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.682+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000235 2023-11-08T16:41:51.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.682+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:64 2023-11-08T16:41:51.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.682+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:223 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017180 con 0x562386af6c00 2023-11-08T16:41:51.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.683+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 81 ==== osd_op_reply(223 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv86 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.683+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=64 frag=0*, r=0 2023-11-08T16:41:51.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.683+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.683+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:224 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017500 con 0x562386af6c00 2023-11-08T16:41:51.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.687+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 82 ==== osd_op_reply(224 10000000000.01000000 [omap-set-vals] v131'87 uv87 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.687+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/64 pointing to 0x0x10000000235 2023-11-08T16:41:51.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.687+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000236 2023-11-08T16:41:51.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.687+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:65 2023-11-08T16:41:51.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.687+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:225 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017880 con 0x562386af6c00 2023-11-08T16:41:51.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.688+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 83 ==== osd_op_reply(225 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv87 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.688+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=65 frag=0*, r=0 2023-11-08T16:41:51.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.688+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.688+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:226 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017c00 con 0x562386af6c00 2023-11-08T16:41:51.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.691+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 84 ==== osd_op_reply(226 10000000000.01000000 [omap-set-vals] v131'88 uv88 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.691+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/65 pointing to 0x0x10000000236 2023-11-08T16:41:51.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.691+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000237 2023-11-08T16:41:51.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.691+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:66 2023-11-08T16:41:51.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.691+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:227 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016000 con 0x562386af6c00 2023-11-08T16:41:51.694 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.692+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 85 ==== osd_op_reply(227 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv88 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.693+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=66 frag=0*, r=0 2023-11-08T16:41:51.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.693+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.695 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.693+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:228 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016a80 con 0x562386af6c00 2023-11-08T16:41:51.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.696+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 86 ==== osd_op_reply(228 10000000000.01000000 [omap-set-vals] v131'89 uv89 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.696+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/66 pointing to 0x0x10000000237 2023-11-08T16:41:51.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.696+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000238 2023-11-08T16:41:51.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.696+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:67 2023-11-08T16:41:51.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.696+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:229 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016e00 con 0x562386b11800 2023-11-08T16:41:51.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.697+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 83 ==== osd_op_reply(229 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv89 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.698+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=67 frag=1*, r=0 2023-11-08T16:41:51.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.698+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.698+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:230 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017180 con 0x562386b11800 2023-11-08T16:41:51.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.702+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 84 ==== osd_op_reply(230 10000000000.01800000 [omap-set-vals] v131'90 uv90 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.702+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/67 pointing to 0x0x10000000238 2023-11-08T16:41:51.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.702+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000239 2023-11-08T16:41:51.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.702+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:68 2023-11-08T16:41:51.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.702+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:231 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017500 con 0x562386b11800 2023-11-08T16:41:51.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.703+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 85 ==== osd_op_reply(231 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv90 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.703+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=68 frag=1*, r=0 2023-11-08T16:41:51.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.703+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.703+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:232 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386b11800 2023-11-08T16:41:51.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.707+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 86 ==== osd_op_reply(232 10000000000.01800000 [omap-set-vals] v131'91 uv91 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.707+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/68 pointing to 0x0x10000000239 2023-11-08T16:41:51.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.707+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000023a 2023-11-08T16:41:51.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.707+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:69 2023-11-08T16:41:51.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.707+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:233 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.708+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 87 ==== osd_op_reply(233 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv89 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.708+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=69 frag=0*, r=0 2023-11-08T16:41:51.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.708+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.708+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:234 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386af6c00 2023-11-08T16:41:51.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.712+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 88 ==== osd_op_reply(234 10000000000.01000000 [omap-set-vals] v131'90 uv90 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.712+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/69 pointing to 0x0x1000000023a 2023-11-08T16:41:51.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.712+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000023b 2023-11-08T16:41:51.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.712+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:70 2023-11-08T16:41:51.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.712+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:235 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047180 con 0x562386af6c00 2023-11-08T16:41:51.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.713+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 89 ==== osd_op_reply(235 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv90 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.713+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=70 frag=0*, r=0 2023-11-08T16:41:51.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.713+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.713+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:236 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047500 con 0x562386af6c00 2023-11-08T16:41:51.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.717+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 90 ==== osd_op_reply(236 10000000000.01000000 [omap-set-vals] v131'91 uv91 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.717+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/70 pointing to 0x0x1000000023b 2023-11-08T16:41:51.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.717+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000023c 2023-11-08T16:41:51.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.717+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:71 2023-11-08T16:41:51.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.717+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:237 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046380 con 0x562386b11800 2023-11-08T16:41:51.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.718+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 87 ==== osd_op_reply(237 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv91 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.718+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=71 frag=1*, r=0 2023-11-08T16:41:51.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.718+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.718+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:238 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046700 con 0x562386b11800 2023-11-08T16:41:51.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.722+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 88 ==== osd_op_reply(238 10000000000.01800000 [omap-set-vals] v131'92 uv92 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.722+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/71 pointing to 0x0x1000000023c 2023-11-08T16:41:51.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.722+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000023d 2023-11-08T16:41:51.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.722+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:72 2023-11-08T16:41:51.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.722+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:239 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016380 con 0x562386b11800 2023-11-08T16:41:51.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.724+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 89 ==== osd_op_reply(239 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv92 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.724+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=72 frag=1*, r=0 2023-11-08T16:41:51.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.724+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.724+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:240 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016700 con 0x562386b11800 2023-11-08T16:41:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.728+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 90 ==== osd_op_reply(240 10000000000.01800000 [omap-set-vals] v131'93 uv93 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.728+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/72 pointing to 0x0x1000000023d 2023-11-08T16:41:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.728+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000023e 2023-11-08T16:41:51.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.728+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:73 2023-11-08T16:41:51.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.728+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:241 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016e00 con 0x562386af6c00 2023-11-08T16:41:51.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.729+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 91 ==== osd_op_reply(241 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv91 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.729+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=73 frag=0*, r=0 2023-11-08T16:41:51.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.729+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.732 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.729+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:242 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.733+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 92 ==== osd_op_reply(242 10000000000.01000000 [omap-set-vals] v131'92 uv92 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.733+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/73 pointing to 0x0x1000000023e 2023-11-08T16:41:51.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.733+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000023f 2023-11-08T16:41:51.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.733+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:74 2023-11-08T16:41:51.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.733+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:243 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386af6c00 2023-11-08T16:41:51.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.734+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 93 ==== osd_op_reply(243 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv92 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.734+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=74 frag=0*, r=0 2023-11-08T16:41:51.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.734+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.734+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:244 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386af6c00 2023-11-08T16:41:51.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.738+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 94 ==== osd_op_reply(244 10000000000.01000000 [omap-set-vals] v131'93 uv93 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.739+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/74 pointing to 0x0x1000000023f 2023-11-08T16:41:51.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.739+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000240 2023-11-08T16:41:51.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.739+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:75 2023-11-08T16:41:51.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.739+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:245 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386af6c00 2023-11-08T16:41:51.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.739+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 95 ==== osd_op_reply(245 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv93 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.740+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=75 frag=0*, r=0 2023-11-08T16:41:51.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.740+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.740+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:246 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047c00 con 0x562386af6c00 2023-11-08T16:41:51.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.743+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 96 ==== osd_op_reply(246 10000000000.01000000 [omap-set-vals] v131'94 uv94 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.743+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/75 pointing to 0x0x10000000240 2023-11-08T16:41:51.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.744+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000241 2023-11-08T16:41:51.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.744+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:76 2023-11-08T16:41:51.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.744+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:247 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047880 con 0x562386b11800 2023-11-08T16:41:51.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.745+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 91 ==== osd_op_reply(247 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv93 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.745+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=76 frag=1*, r=0 2023-11-08T16:41:51.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.745+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.745+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:248 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046000 con 0x562386b11800 2023-11-08T16:41:51.750 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.749+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 92 ==== osd_op_reply(248 10000000000.01800000 [omap-set-vals] v131'94 uv94 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.749+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/76 pointing to 0x0x10000000241 2023-11-08T16:41:51.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.749+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000242 2023-11-08T16:41:51.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.749+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:77 2023-11-08T16:41:51.751 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.749+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:249 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046a80 con 0x562386af6c00 2023-11-08T16:41:51.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.750+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 97 ==== osd_op_reply(249 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv94 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.750+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=77 frag=0*, r=0 2023-11-08T16:41:51.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.750+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.750+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:250 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046e00 con 0x562386af6c00 2023-11-08T16:41:51.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.753+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 98 ==== osd_op_reply(250 10000000000.01000000 [omap-set-vals] v131'95 uv95 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.753+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/77 pointing to 0x0x10000000242 2023-11-08T16:41:51.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.753+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000243 2023-11-08T16:41:51.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.754+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:78 2023-11-08T16:41:51.756 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.754+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:251 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386b11800 2023-11-08T16:41:51.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.755+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 93 ==== osd_op_reply(251 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv94 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.755+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=78 frag=1*, r=0 2023-11-08T16:41:51.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.755+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.755+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:252 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386b11800 2023-11-08T16:41:51.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.758+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 94 ==== osd_op_reply(252 10000000000.01800000 [omap-set-vals] v131'95 uv95 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.759+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/78 pointing to 0x0x10000000243 2023-11-08T16:41:51.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.759+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000244 2023-11-08T16:41:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.759+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:79 2023-11-08T16:41:51.761 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.759+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:253 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386b11800 2023-11-08T16:41:51.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.760+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 95 ==== osd_op_reply(253 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv95 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.760+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=79 frag=1*, r=0 2023-11-08T16:41:51.762 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.760+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.760+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:254 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386b11800 2023-11-08T16:41:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.764+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 96 ==== osd_op_reply(254 10000000000.01800000 [omap-set-vals] v131'96 uv96 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.764+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/79 pointing to 0x0x10000000244 2023-11-08T16:41:51.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.764+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000245 2023-11-08T16:41:51.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.764+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:80 2023-11-08T16:41:51.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.764+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:255 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386af6c00 2023-11-08T16:41:51.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.765+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 99 ==== osd_op_reply(255 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv95 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.765+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=80 frag=0*, r=0 2023-11-08T16:41:51.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.765+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.765+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:256 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.768+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 100 ==== osd_op_reply(256 10000000000.01000000 [omap-set-vals] v131'96 uv96 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.769+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/80 pointing to 0x0x10000000245 2023-11-08T16:41:51.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.769+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000246 2023-11-08T16:41:51.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.769+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:81 2023-11-08T16:41:51.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.769+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:257 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017880 con 0x562386b11800 2023-11-08T16:41:51.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.770+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 97 ==== osd_op_reply(257 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv96 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.774 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.770+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=81 frag=1*, r=0 2023-11-08T16:41:51.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.770+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.770+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:258 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017c00 con 0x562386b11800 2023-11-08T16:41:51.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.773+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 98 ==== osd_op_reply(258 10000000000.01800000 [omap-set-vals] v131'97 uv97 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.773+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/81 pointing to 0x0x10000000246 2023-11-08T16:41:51.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.773+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000247 2023-11-08T16:41:51.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.773+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:82 2023-11-08T16:41:51.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.773+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:259 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016000 con 0x562386b11800 2023-11-08T16:41:51.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.774+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 99 ==== osd_op_reply(259 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv97 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.774+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=82 frag=1*, r=0 2023-11-08T16:41:51.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.774+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.774+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:260 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016a80 con 0x562386b11800 2023-11-08T16:41:51.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.778+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 100 ==== osd_op_reply(260 10000000000.01800000 [omap-set-vals] v131'98 uv98 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.778+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/82 pointing to 0x0x10000000247 2023-11-08T16:41:51.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.778+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000248 2023-11-08T16:41:51.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.778+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:83 2023-11-08T16:41:51.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.778+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:261 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016e00 con 0x562386af6c00 2023-11-08T16:41:51.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.779+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 101 ==== osd_op_reply(261 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv96 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.779+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=83 frag=0*, r=0 2023-11-08T16:41:51.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.779+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.779+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:262 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047180 con 0x562386af6c00 2023-11-08T16:41:51.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.782+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 102 ==== osd_op_reply(262 10000000000.01000000 [omap-set-vals] v131'97 uv97 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.782+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/83 pointing to 0x0x10000000248 2023-11-08T16:41:51.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.782+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000249 2023-11-08T16:41:51.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.782+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:84 2023-11-08T16:41:51.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.782+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:263 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047500 con 0x562386af6c00 2023-11-08T16:41:51.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.783+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 103 ==== osd_op_reply(263 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv97 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.783+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=84 frag=0*, r=0 2023-11-08T16:41:51.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.783+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.783+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:264 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.787+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 104 ==== osd_op_reply(264 10000000000.01000000 [omap-set-vals] v131'98 uv98 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.787+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/84 pointing to 0x0x10000000249 2023-11-08T16:41:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.787+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000024a 2023-11-08T16:41:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.787+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:85 2023-11-08T16:41:51.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.787+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:265 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386af6c00 2023-11-08T16:41:51.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.788+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 105 ==== osd_op_reply(265 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv98 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.788+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=85 frag=0*, r=0 2023-11-08T16:41:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.788+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.788+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:266 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386af6c00 2023-11-08T16:41:51.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.792+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 106 ==== osd_op_reply(266 10000000000.01000000 [omap-set-vals] v131'99 uv99 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.792+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/85 pointing to 0x0x1000000024a 2023-11-08T16:41:51.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.792+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000024b 2023-11-08T16:41:51.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.792+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:86 2023-11-08T16:41:51.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.792+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:267 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386af6c00 2023-11-08T16:41:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.793+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 107 ==== osd_op_reply(267 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv99 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.793+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=86 frag=0*, r=0 2023-11-08T16:41:51.795 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.793+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.793+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:268 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046380 con 0x562386af6c00 2023-11-08T16:41:51.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.796+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 108 ==== osd_op_reply(268 10000000000.01000000 [omap-set-vals] v131'100 uv100 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.797+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/86 pointing to 0x0x1000000024b 2023-11-08T16:41:51.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.797+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000024c 2023-11-08T16:41:51.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.797+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:87 2023-11-08T16:41:51.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.797+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:269 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046700 con 0x562386af6c00 2023-11-08T16:41:51.799 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.798+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 109 ==== osd_op_reply(269 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv100 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.798+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=87 frag=0*, r=0 2023-11-08T16:41:51.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.798+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.800 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.798+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:270 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047880 con 0x562386af6c00 2023-11-08T16:41:51.802 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.801+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 110 ==== osd_op_reply(270 10000000000.01000000 [omap-set-vals] v131'101 uv101 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.801+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/87 pointing to 0x0x1000000024c 2023-11-08T16:41:51.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.801+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000024d 2023-11-08T16:41:51.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.801+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:88 2023-11-08T16:41:51.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.801+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:271 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046000 con 0x562386af6c00 2023-11-08T16:41:51.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.802+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 111 ==== osd_op_reply(271 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv101 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.802+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=88 frag=0*, r=0 2023-11-08T16:41:51.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.802+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.802+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:272 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017180 con 0x562386af6c00 2023-11-08T16:41:51.807 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.806+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 112 ==== osd_op_reply(272 10000000000.01000000 [omap-set-vals] v131'102 uv102 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.806+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/88 pointing to 0x0x1000000024d 2023-11-08T16:41:51.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.806+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000024e 2023-11-08T16:41:51.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.806+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:89 2023-11-08T16:41:51.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.806+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:273 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386017500 con 0x562386b11800 2023-11-08T16:41:51.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.807+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 101 ==== osd_op_reply(273 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv98 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.807+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=89 frag=1*, r=0 2023-11-08T16:41:51.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.807+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.807+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:274 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016380 con 0x562386b11800 2023-11-08T16:41:51.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.811+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 102 ==== osd_op_reply(274 10000000000.01800000 [omap-set-vals] v131'99 uv99 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.811+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/89 pointing to 0x0x1000000024e 2023-11-08T16:41:51.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.812+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x1000000024f 2023-11-08T16:41:51.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.812+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:90 2023-11-08T16:41:51.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.812+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:275 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016700 con 0x562386af6c00 2023-11-08T16:41:51.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.812+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 113 ==== osd_op_reply(275 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv102 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.812+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=90 frag=0*, r=0 2023-11-08T16:41:51.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.813+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.813+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:276 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33c00 con 0x562386af6c00 2023-11-08T16:41:51.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.816+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 114 ==== osd_op_reply(276 10000000000.01000000 [omap-set-vals] v131'103 uv103 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.816+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/90 pointing to 0x0x1000000024f 2023-11-08T16:41:51.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.816+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000250 2023-11-08T16:41:51.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.816+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:91 2023-11-08T16:41:51.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.816+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:277 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047c00 con 0x562386b11800 2023-11-08T16:41:51.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.817+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 103 ==== osd_op_reply(277 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv99 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.817+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=91 frag=1*, r=0 2023-11-08T16:41:51.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.817+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.817+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:278 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046a80 con 0x562386b11800 2023-11-08T16:41:51.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.821+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 104 ==== osd_op_reply(278 10000000000.01800000 [omap-set-vals] v131'100 uv100 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.821+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/91 pointing to 0x0x10000000250 2023-11-08T16:41:51.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.821+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000251 2023-11-08T16:41:51.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.821+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:92 2023-11-08T16:41:51.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.821+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:279 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046e00 con 0x562386af6c00 2023-11-08T16:41:51.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.822+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 115 ==== osd_op_reply(279 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv103 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.822+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=92 frag=0*, r=0 2023-11-08T16:41:51.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.822+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.822+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:280 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047180 con 0x562386af6c00 2023-11-08T16:41:51.827 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.826+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 116 ==== osd_op_reply(280 10000000000.01000000 [omap-set-vals] v131'104 uv104 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.826+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/92 pointing to 0x0x10000000251 2023-11-08T16:41:51.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.826+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000252 2023-11-08T16:41:51.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.826+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:93 2023-11-08T16:41:51.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.826+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:281 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386047500 con 0x562386b11800 2023-11-08T16:41:51.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.827+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 105 ==== osd_op_reply(281 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv100 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.827+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=93 frag=1*, r=0 2023-11-08T16:41:51.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.827+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.828+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:282 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046380 con 0x562386b11800 2023-11-08T16:41:51.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.831+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 106 ==== osd_op_reply(282 10000000000.01800000 [omap-set-vals] v131'101 uv101 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.832 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.831+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/93 pointing to 0x0x10000000252 2023-11-08T16:41:51.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.831+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000253 2023-11-08T16:41:51.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.831+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:94 2023-11-08T16:41:51.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.831+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:283 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386046700 con 0x562386b11800 2023-11-08T16:41:51.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.832+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 107 ==== osd_op_reply(283 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv101 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.832+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=94 frag=1*, r=0 2023-11-08T16:41:51.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.832+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.832+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:284 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6b500 con 0x562386b11800 2023-11-08T16:41:51.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.836+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 108 ==== osd_op_reply(284 10000000000.01800000 [omap-set-vals] v131'102 uv102 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.836+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/94 pointing to 0x0x10000000253 2023-11-08T16:41:51.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.836+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000254 2023-11-08T16:41:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.836+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:95 2023-11-08T16:41:51.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.836+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:285 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6aa80 con 0x562386af6c00 2023-11-08T16:41:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.837+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 117 ==== osd_op_reply(285 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv104 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.837+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=95 frag=0*, r=0 2023-11-08T16:41:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.837+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.837+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:286 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6ae00 con 0x562386af6c00 2023-11-08T16:41:51.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.841+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 118 ==== osd_op_reply(286 10000000000.01000000 [omap-set-vals] v131'105 uv105 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.841+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/95 pointing to 0x0x10000000254 2023-11-08T16:41:51.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.841+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000255 2023-11-08T16:41:51.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.841+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:96 2023-11-08T16:41:51.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.841+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:287 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a700 con 0x562386af6c00 2023-11-08T16:41:51.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.842+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 119 ==== osd_op_reply(287 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv105 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.842+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=96 frag=0*, r=0 2023-11-08T16:41:51.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.842+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.842+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:288 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385f6a380 con 0x562386af6c00 2023-11-08T16:41:51.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.845+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 120 ==== osd_op_reply(288 10000000000.01000000 [omap-set-vals] v131'106 uv106 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.845+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/96 pointing to 0x0x10000000255 2023-11-08T16:41:51.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.845+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000256 2023-11-08T16:41:51.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.845+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:97 2023-11-08T16:41:51.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.845+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:289 16.5 16:a40620de:::10000000000.01000000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386016e00 con 0x562386af6c00 2023-11-08T16:41:51.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.846+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 121 ==== osd_op_reply(289 10000000000.01000000 [omap-get-vals-by-keys out=503b] v0'0 uv106 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.846+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01000000 dname=97 frag=0*, r=0 2023-11-08T16:41:51.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.846+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.846+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:290 16.5 16:a40620de:::10000000000.01000000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562385c33880 con 0x562386af6c00 2023-11-08T16:41:51.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.850+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.0 v2:172.21.15.81:6808/1666604866 122 ==== osd_op_reply(290 10000000000.01000000 [omap-set-vals] v131'107 uv107 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82b40 con 0x562386af6c00 2023-11-08T16:41:51.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.850+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/97 pointing to 0x0x10000000256 2023-11-08T16:41:51.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.850+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000257 2023-11-08T16:41:51.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.850+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:98 2023-11-08T16:41:51.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.850+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:291 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386944000 con 0x562386b11800 2023-11-08T16:41:51.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.851+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 109 ==== osd_op_reply(291 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv102 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.851+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=98 frag=1*, r=0 2023-11-08T16:41:51.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.851+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.851+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:292 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386944380 con 0x562386b11800 2023-11-08T16:41:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.855+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 110 ==== osd_op_reply(292 10000000000.01800000 [omap-set-vals] v131'103 uv103 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.855+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/98 pointing to 0x0x10000000257 2023-11-08T16:41:51.856 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.855+0000 7f5f9aa385c0 10 datascan.scan_links: handling injected_ino 0x10000000258 2023-11-08T16:41:51.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.855+0000 7f5f9aa385c0 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:99 2023-11-08T16:41:51.857 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.855+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:293 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-get-vals-by-keys in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386944700 con 0x562386b11800 2023-11-08T16:41:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.856+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 111 ==== osd_op_reply(293 10000000000.01800000 [omap-get-vals-by-keys out=503b] v0'0 uv103 ondisk = 0) v8 ==== 164+0+503 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.856+0000 7f5f9aa385c0 20 datascan.read_dentry: oid=10000000000.01800000 dname=99 frag=1*, r=0 2023-11-08T16:41:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.856+0000 7f5f9aa385c0 20 datascan.scan_links: first is now 2 2023-11-08T16:41:51.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.856+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- osd_op(unknown.0.0:294 16.1b 16:dcc776e0:::10000000000.01800000:head [omap-set-vals in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386944a80 con 0x562386b11800 2023-11-08T16:41:51.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.860+0000 7f5f98267640 1 -- 172.21.15.81:0/4210848566 <== osd.5 v2:172.21.15.93:6802/2406579932 112 ==== osd_op_reply(294 10000000000.01800000 [omap-set-vals] v131'104 uv104 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x562386a82fc0 con 0x562386b11800 2023-11-08T16:41:51.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.860+0000 7f5f9aa385c0 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/99 pointing to 0x0x10000000258 2023-11-08T16:41:51.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.860+0000 7f5f9aa385c0 10 datascan.scan_links: updating inotable 2023-11-08T16:41:51.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.860+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:295 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386944e00 con 0x562385f5ec00 2023-11-08T16:41:51.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.861+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.1 v2:172.21.15.81:6800/2060798739 9 ==== osd_op_reply(295 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) v8 ==== 157+0+50 (crc 0 0 0) 0x562386a82d80 con 0x562385f5ec00 2023-11-08T16:41:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.861+0000 7f5f9aa385c0 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x10000000259~0xfffffffda7] 2023-11-08T16:41:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.861+0000 7f5f9aa385c0 10 mds.0.inotable: skip_inos now [0x10000000259~0xfffffffda7] 2023-11-08T16:41:51.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.861+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- osd_op(unknown.0.0:296 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386945180 con 0x562385f5ec00 2023-11-08T16:41:51.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.865+0000 7f5f99269640 1 -- 172.21.15.81:0/4210848566 <== osd.1 v2:172.21.15.81:6800/2060798739 10 ==== osd_op_reply(296 mds0_inotable [writefull 0~34] v131'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x562386a82d80 con 0x562385f5ec00 2023-11-08T16:41:51.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.865+0000 7f5f9aa385c0 10 datascan.scan_links: updating snaptable 2023-11-08T16:41:51.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.865+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- osd_op(unknown.0.0:297 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x562386945500 con 0x562385f5e000 2023-11-08T16:41:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.866+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 <== osd.6 v2:172.21.15.93:6816/607960134 7 ==== osd_op_reply(297 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x562386a82b40 con 0x562385f5e000 2023-11-08T16:41:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562386af6c00 msgr2=0x562385c46c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x562386af6c00 0x562385c46c00 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562385f5ec00 msgr2=0x562385f48100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x562385f5ec00 0x562385f48100 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562386b11000 msgr2=0x562385c44580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x562386b11000 0x562385c44580 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562386b11400 msgr2=0x562385f46000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x562386b11400 0x562385f46000 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562386af7c00 msgr2=0x562385f47080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x562386af7c00 0x562385f47080 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562386b11800 msgr2=0x562385f46580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x562386b11800 0x562385f46580 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562385f5e000 msgr2=0x562385f47600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x562385f5e000 0x562385f47600 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 reap_dead start 2023-11-08T16:41:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562385f5e400 msgr2=0x562385f47b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x562385f5e400 0x562385f47b80 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 reap_dead start 2023-11-08T16:41:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 reap_dead start 2023-11-08T16:41:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f98a68640 1 -- 172.21.15.81:0/4210848566 reap_dead start 2023-11-08T16:41:51.873 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562386b10c00 msgr2=0x562385c47700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562386b10c00 0x562385c47700 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x562386aa4a20 tx=0x562386aacc00 comp rx=0 tx=0).stop 2023-11-08T16:41:51.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562386af7000 msgr2=0x562385c46680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.867+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562386af7000 0x562385c46680 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x562386aa48d0 tx=0x56238696a120 comp rx=0 tx=0).stop 2023-11-08T16:41:51.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.868+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 shutdown_connections 2023-11-08T16:41:51.874 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.868+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x562386b10c00 0x562385c47700 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.868+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/4210848566 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562386af7000 0x562385c46680 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.875 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.868+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 >> 172.21.15.81:0/4210848566 conn(0x5623869b7000 msgr2=0x562386965000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:51.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.868+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 shutdown_connections 2023-11-08T16:41:51.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.868+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/4210848566 wait complete. 2023-11-08T16:41:51.876 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.869+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/2672302937 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562385bdbc00 msgr2=0x562385c45080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:51.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.869+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/2672302937 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562385bdbc00 0x562385c45080 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x5623869e7530 tx=0x5623869aac60 comp rx=0 tx=0).stop 2023-11-08T16:41:51.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.869+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/2672302937 shutdown_connections 2023-11-08T16:41:51.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.869+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/2672302937 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5623869b7400 0x562385c45600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.869+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/2672302937 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x562385bdbc00 0x562385c45080 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.877 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.869+0000 7f5f9aa385c0 1 --2- 172.21.15.81:0/2672302937 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x562385bdb800 0x562385c44b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:51.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.870+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/2672302937 >> 172.21.15.81:0/2672302937 conn(0x562385bdb000 msgr2=0x562386964800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:51.878 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.870+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/2672302937 shutdown_connections 2023-11-08T16:41:51.879 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:51.870+0000 7f5f9aa385c0 1 -- 172.21.15.81:0/2672302937 wait complete. 2023-11-08T16:41:51.883 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.768583, result: 2023-11-08T16:41:51.884 DEBUG:teuthology.orchestra.run.smithi081:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata listomapkeys 10000000000.01000000 2023-11-08T16:41:52.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.034+0000 7f1baaf69f40 1 Processor -- start 2023-11-08T16:41:52.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.034+0000 7f1baaf69f40 1 -- start start 2023-11-08T16:41:52.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.035+0000 7f1baaf69f40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x560895f81650 0x560896052af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.035+0000 7f1baaf69f40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608960553d0 0x560896050670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.035+0000 7f1baaf69f40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56089605b320 0x560896050bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.035+0000 7f1baaf69f40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560895f32340 con 0x5608960553d0 2023-11-08T16:41:52.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.035+0000 7f1baaf69f40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560895ecca70 con 0x56089605b320 2023-11-08T16:41:52.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.035+0000 7f1baaf69f40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x560895ecc710 con 0x560895f81650 2023-11-08T16:41:52.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.035+0000 7f1ba3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608960553d0 0x560896050670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.035+0000 7f1ba3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608960553d0 0x560896050670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40606/0 (socket says 172.21.15.81:40606) 2023-11-08T16:41:52.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.035+0000 7f1ba3fff640 1 -- 172.21.15.81:0/3903429069 learned_addr learned my addr 172.21.15.81:0/3903429069 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:52.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.036+0000 7f1ba8a6b640 1 --2- 172.21.15.81:0/3903429069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x560895f81650 0x560896052af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.036+0000 7f1ba3fff640 1 -- 172.21.15.81:0/3903429069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x560895f81650 msgr2=0x560896052af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:52.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.036+0000 7f1ba3fff640 1 --2- 172.21.15.81:0/3903429069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x560895f81650 0x560896052af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.036+0000 7f1ba3fff640 1 -- 172.21.15.81:0/3903429069 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56089605b320 msgr2=0x560896050bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:52.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.036+0000 7f1ba3fff640 1 --2- 172.21.15.81:0/3903429069 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56089605b320 0x560896050bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.036+0000 7f1ba3fff640 1 -- 172.21.15.81:0/3903429069 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b98005d20 con 0x5608960553d0 2023-11-08T16:41:52.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.036+0000 7f1ba3fff640 1 --2- 172.21.15.81:0/3903429069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608960553d0 0x560896050670 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800a440 tx=0x7f1b9804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b95294ecd43c8f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:52.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.036+0000 7f1ba2ffd640 1 -- 172.21.15.81:0/3903429069 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b98052640 con 0x5608960553d0 2023-11-08T16:41:52.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.036+0000 7f1ba2ffd640 1 -- 172.21.15.81:0/3903429069 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9804ae90 con 0x5608960553d0 2023-11-08T16:41:52.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.037+0000 7f1ba2ffd640 1 -- 172.21.15.81:0/3903429069 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b980529e0 con 0x5608960553d0 2023-11-08T16:41:52.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.037+0000 7f1baaf69f40 1 -- 172.21.15.81:0/3903429069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608960553d0 msgr2=0x560896050670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:52.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.037+0000 7f1baaf69f40 1 --2- 172.21.15.81:0/3903429069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608960553d0 0x560896050670 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f1b9800a440 tx=0x7f1b9804c000 comp rx=0 tx=0).stop 2023-11-08T16:41:52.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.037+0000 7f1baaf69f40 1 -- 172.21.15.81:0/3903429069 shutdown_connections 2023-11-08T16:41:52.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.037+0000 7f1baaf69f40 1 --2- 172.21.15.81:0/3903429069 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56089605b320 0x560896050bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.037+0000 7f1baaf69f40 1 --2- 172.21.15.81:0/3903429069 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5608960553d0 0x560896050670 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.037+0000 7f1baaf69f40 1 --2- 172.21.15.81:0/3903429069 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x560895f81650 0x560896052af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.037+0000 7f1baaf69f40 1 -- 172.21.15.81:0/3903429069 >> 172.21.15.81:0/3903429069 conn(0x560895fcde00 msgr2=0x560895fb0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:52.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.037+0000 7f1baaf69f40 1 -- 172.21.15.81:0/3903429069 shutdown_connections 2023-11-08T16:41:52.045 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.037+0000 7f1baaf69f40 1 -- 172.21.15.81:0/3903429069 wait complete. 2023-11-08T16:41:52.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.038+0000 7f1baaf69f40 1 Processor -- start 2023-11-08T16:41:52.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.038+0000 7f1baaf69f40 1 -- start start 2023-11-08T16:41:52.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.039+0000 7f1baaf69f40 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x560895f81650 0x560895fb67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.039+0000 7f1baaf69f40 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5608960553d0 0x56089604cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.039+0000 7f1baaf69f40 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56089605b320 0x560896057490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.039+0000 7f1baaf69f40 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560895ec0520 con 0x56089605b320 2023-11-08T16:41:52.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.039+0000 7f1baaf69f40 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x560896057070 con 0x5608960553d0 2023-11-08T16:41:52.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.039+0000 7f1baaf69f40 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x560896050f30 con 0x560895f81650 2023-11-08T16:41:52.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.039+0000 7f1ba926c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56089605b320 0x560896057490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.047 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.039+0000 7f1ba926c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56089605b320 0x560896057490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40612/0 (socket says 172.21.15.81:40612) 2023-11-08T16:41:52.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.039+0000 7f1ba926c640 1 -- 172.21.15.81:0/2598185 learned_addr learned my addr 172.21.15.81:0/2598185 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:52.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1ba3fff640 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5608960553d0 0x56089604cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1ba8a6b640 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x560895f81650 0x560895fb67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1ba926c640 1 -- 172.21.15.81:0/2598185 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x560895f81650 msgr2=0x560895fb67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:52.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1ba926c640 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x560895f81650 0x560895fb67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1ba926c640 1 -- 172.21.15.81:0/2598185 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5608960553d0 msgr2=0x56089604cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:52.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1ba926c640 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5608960553d0 0x56089604cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1ba926c640 1 -- 172.21.15.81:0/2598185 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560895ef2670 con 0x56089605b320 2023-11-08T16:41:52.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1ba926c640 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56089605b320 0x560896057490 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400bdd0 tx=0x7f1ba4009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:52.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1ba1ffb640 1 -- 172.21.15.81:0/2598185 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4014030 con 0x56089605b320 2023-11-08T16:41:52.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1ba1ffb640 1 -- 172.21.15.81:0/2598185 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba400f650 con 0x56089605b320 2023-11-08T16:41:52.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1ba1ffb640 1 -- 172.21.15.81:0/2598185 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba400fa40 con 0x56089605b320 2023-11-08T16:41:52.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.040+0000 7f1baaf69f40 1 -- 172.21.15.81:0/2598185 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560895ed8680 con 0x56089605b320 2023-11-08T16:41:52.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.041+0000 7f1baaf69f40 1 -- 172.21.15.81:0/2598185 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560895ecb830 con 0x56089605b320 2023-11-08T16:41:52.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.043+0000 7f1ba1ffb640 1 -- 172.21.15.81:0/2598185 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1ba4017070 con 0x56089605b320 2023-11-08T16:41:52.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.043+0000 7f1ba1ffb640 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1b94069820 0x7f1b9406bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.044+0000 7f1ba1ffb640 1 -- 172.21.15.81:0/2598185 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f1ba4015070 con 0x56089605b320 2023-11-08T16:41:52.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.044+0000 7f1ba8a6b640 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1b94069820 0x7f1b9406bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.044+0000 7f1baaf69f40 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56089604bbb0 0x5608960579d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.044+0000 7f1baaf69f40 1 -- 172.21.15.81:0/2598185 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- osd_op(unknown.0.0:1 16.5 16:a40620de:::10000000000.01000000:head [omap-get-keys in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e131) v8 -- 0x560895fb2660 con 0x56089604bbb0 2023-11-08T16:41:52.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.044+0000 7f1ba3fff640 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56089604bbb0 0x5608960579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.053 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.044+0000 7f1ba8a6b640 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1b94069820 0x7f1b9406bce0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x560895fb7140 tx=0x7f1b9c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:52.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.045+0000 7f1ba3fff640 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56089604bbb0 0x5608960579d0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:52.054 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.045+0000 7f1ba3fff640 1 -- 172.21.15.81:0/2598185 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== osd_op_reply(1 10000000000.01000000 [omap-get-keys out=560b] v0'0 uv107 ondisk = 0) v8 ==== 164+0+560 (crc 0 0 0) 0x7f1b98059040 con 0x56089604bbb0 2023-11-08T16:41:52.055 INFO:teuthology.orchestra.run.smithi081.stdout:10_head 2023-11-08T16:41:52.055 INFO:teuthology.orchestra.run.smithi081.stdout:11_head 2023-11-08T16:41:52.055 INFO:teuthology.orchestra.run.smithi081.stdout:12_head 2023-11-08T16:41:52.056 INFO:teuthology.orchestra.run.smithi081.stdout:15_head 2023-11-08T16:41:52.057 INFO:teuthology.orchestra.run.smithi081.stdout:19_head 2023-11-08T16:41:52.057 INFO:teuthology.orchestra.run.smithi081.stdout:1_head 2023-11-08T16:41:52.057 INFO:teuthology.orchestra.run.smithi081.stdout:20_head 2023-11-08T16:41:52.057 INFO:teuthology.orchestra.run.smithi081.stdout:21_head 2023-11-08T16:41:52.058 INFO:teuthology.orchestra.run.smithi081.stdout:27_head 2023-11-08T16:41:52.058 INFO:teuthology.orchestra.run.smithi081.stdout:28_head 2023-11-08T16:41:52.059 INFO:teuthology.orchestra.run.smithi081.stdout:2_head 2023-11-08T16:41:52.059 INFO:teuthology.orchestra.run.smithi081.stdout:31_head 2023-11-08T16:41:52.060 INFO:teuthology.orchestra.run.smithi081.stdout:34_head 2023-11-08T16:41:52.060 INFO:teuthology.orchestra.run.smithi081.stdout:36_head 2023-11-08T16:41:52.060 INFO:teuthology.orchestra.run.smithi081.stdout:38_head 2023-11-08T16:41:52.061 INFO:teuthology.orchestra.run.smithi081.stdout:39_head 2023-11-08T16:41:52.061 INFO:teuthology.orchestra.run.smithi081.stdout:3_head 2023-11-08T16:41:52.061 INFO:teuthology.orchestra.run.smithi081.stdout:42_head 2023-11-08T16:41:52.062 INFO:teuthology.orchestra.run.smithi081.stdout:43_head 2023-11-08T16:41:52.062 INFO:teuthology.orchestra.run.smithi081.stdout:46_head 2023-11-08T16:41:52.062 INFO:teuthology.orchestra.run.smithi081.stdout:4_head 2023-11-08T16:41:52.062 INFO:teuthology.orchestra.run.smithi081.stdout:50_head 2023-11-08T16:41:52.063 INFO:teuthology.orchestra.run.smithi081.stdout:52_head 2023-11-08T16:41:52.063 INFO:teuthology.orchestra.run.smithi081.stdout:53_head 2023-11-08T16:41:52.063 INFO:teuthology.orchestra.run.smithi081.stdout:55_head 2023-11-08T16:41:52.063 INFO:teuthology.orchestra.run.smithi081.stdout:59_head 2023-11-08T16:41:52.064 INFO:teuthology.orchestra.run.smithi081.stdout:61_head 2023-11-08T16:41:52.064 INFO:teuthology.orchestra.run.smithi081.stdout:63_head 2023-11-08T16:41:52.064 INFO:teuthology.orchestra.run.smithi081.stdout:64_head 2023-11-08T16:41:52.064 INFO:teuthology.orchestra.run.smithi081.stdout:65_head 2023-11-08T16:41:52.065 INFO:teuthology.orchestra.run.smithi081.stdout:66_head 2023-11-08T16:41:52.065 INFO:teuthology.orchestra.run.smithi081.stdout:69_head 2023-11-08T16:41:52.065 INFO:teuthology.orchestra.run.smithi081.stdout:70_head 2023-11-08T16:41:52.066 INFO:teuthology.orchestra.run.smithi081.stdout:73_head 2023-11-08T16:41:52.066 INFO:teuthology.orchestra.run.smithi081.stdout:74_head 2023-11-08T16:41:52.066 INFO:teuthology.orchestra.run.smithi081.stdout:75_head 2023-11-08T16:41:52.066 INFO:teuthology.orchestra.run.smithi081.stdout:77_head 2023-11-08T16:41:52.067 INFO:teuthology.orchestra.run.smithi081.stdout:80_head 2023-11-08T16:41:52.067 INFO:teuthology.orchestra.run.smithi081.stdout:83_head 2023-11-08T16:41:52.068 INFO:teuthology.orchestra.run.smithi081.stdout:84_head 2023-11-08T16:41:52.068 INFO:teuthology.orchestra.run.smithi081.stdout:85_head 2023-11-08T16:41:52.068 INFO:teuthology.orchestra.run.smithi081.stdout:86_head 2023-11-08T16:41:52.068 INFO:teuthology.orchestra.run.smithi081.stdout:87_head 2023-11-08T16:41:52.069 INFO:teuthology.orchestra.run.smithi081.stdout:88_head 2023-11-08T16:41:52.069 INFO:teuthology.orchestra.run.smithi081.stdout:8_head 2023-11-08T16:41:52.069 INFO:teuthology.orchestra.run.smithi081.stdout:90_head 2023-11-08T16:41:52.069 INFO:teuthology.orchestra.run.smithi081.stdout:92_head 2023-11-08T16:41:52.070 INFO:teuthology.orchestra.run.smithi081.stdout:95_head 2023-11-08T16:41:52.070 INFO:teuthology.orchestra.run.smithi081.stdout:96_head 2023-11-08T16:41:52.070 INFO:teuthology.orchestra.run.smithi081.stdout:97_head 2023-11-08T16:41:52.071 INFO:teuthology.orchestra.run.smithi081.stdout:9_head 2023-11-08T16:41:52.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.046+0000 7f1baaf69f40 1 -- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56089604bbb0 msgr2=0x5608960579d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:52.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.046+0000 7f1baaf69f40 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x56089604bbb0 0x5608960579d0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.046+0000 7f1baaf69f40 1 -- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1b94069820 msgr2=0x7f1b9406bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:52.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.046+0000 7f1baaf69f40 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1b94069820 0x7f1b9406bce0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x560895fb7140 tx=0x7f1b9c007040 comp rx=0 tx=0).stop 2023-11-08T16:41:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.046+0000 7f1baaf69f40 1 -- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56089605b320 msgr2=0x560896057490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.046+0000 7f1baaf69f40 1 --2- 172.21.15.81:0/2598185 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56089605b320 0x560896057490 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f1ba400bdd0 tx=0x7f1ba4009560 comp rx=0 tx=0).stop 2023-11-08T16:41:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.046+0000 7f1ba926c640 1 -- 172.21.15.81:0/2598185 reap_dead start 2023-11-08T16:41:52.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.046+0000 7f1baaf69f40 1 -- 172.21.15.81:0/2598185 shutdown_connections 2023-11-08T16:41:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.046+0000 7f1baaf69f40 1 -- 172.21.15.81:0/2598185 >> 172.21.15.81:0/2598185 conn(0x560895fcde00 msgr2=0x560895fb1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.046+0000 7f1baaf69f40 1 -- 172.21.15.81:0/2598185 shutdown_connections 2023-11-08T16:41:52.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.046+0000 7f1baaf69f40 1 -- 172.21.15.81:0/2598185 wait complete. 2023-11-08T16:41:52.076 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2023-11-08T16:41:52.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.283+0000 7f5ef5439640 1 Processor -- start 2023-11-08T16:41:52.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.284+0000 7f5ef5439640 1 -- start start 2023-11-08T16:41:52.286 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.285+0000 7f5ef5439640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f5ef0158c60 0x7f5ef00705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.285+0000 7f5ef5439640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef015fad0 con 0x7f5ef015f380 2023-11-08T16:41:52.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.285+0000 7f5ef5439640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef006d760 con 0x7f5ef0071ad0 2023-11-08T16:41:52.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.285+0000 7f5ef5439640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef006df80 con 0x7f5ef0158c60 2023-11-08T16:41:52.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.285+0000 7f5eef7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5ef015f380 0x7f5ef00c91c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44898/0 (socket says 172.21.15.81:44898) 2023-11-08T16:41:52.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.285+0000 7f5eef7fe640 1 -- 172.21.15.81:0/3305777293 learned_addr learned my addr 172.21.15.81:0/3305777293 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:52.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eeeffd640 1 --2- 172.21.15.81:0/3305777293 >> v2:172.21.15.93:3300/0 conn(0x7f5ef0158c60 0x7f5ef00705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003777545 0 0) 0x7f5ef015fad0 con 0x7f5ef015f380 2023-11-08T16:41:52.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed8003520 con 0x7f5ef015f380 2023-11-08T16:41:52.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2198922557 0 0) 0x7f5ed8003520 con 0x7f5ef015f380 2023-11-08T16:41:52.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 >> v2:172.21.15.93:3300/0 conn(0x7f5ef0158c60 msgr2=0x7f5ef00705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:52.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eedffb640 1 --2- 172.21.15.81:0/3305777293 >> v2:172.21.15.93:3300/0 conn(0x7f5ef0158c60 0x7f5ef00705d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 >> v1:172.21.15.93:6789/0 conn(0x7f5ef0071ad0 legacy=0x7f5ef015ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:52.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef006da00 con 0x7f5ef015f380 2023-11-08T16:41:52.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f5ee80019f0 con 0x7f5ef015f380 2023-11-08T16:41:52.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 >> v1:172.21.15.81:6789/0 conn(0x7f5ef015f380 legacy=0x7f5ef00c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:52.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eedffb640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ed8003970 0x7f5ed8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.286+0000 7f5eedffb640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ed80047c0 0x7f5ed8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eedffb640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 0x7f5ed800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef006df80 con 0x7f5ed8005180 2023-11-08T16:41:52.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed800c120 con 0x7f5ed80047c0 2023-11-08T16:41:52.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed800c2e0 con 0x7f5ed8003970 2023-11-08T16:41:52.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eef7fe640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 0x7f5ed800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eee7fc640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ed80047c0 0x7f5ed8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eeeffd640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ed8003970 0x7f5ed8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eef7fe640 1 -- 172.21.15.81:0/3305777293 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ed8003970 msgr2=0x7f5ed8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:52.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eef7fe640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ed8003970 0x7f5ed8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eef7fe640 1 -- 172.21.15.81:0/3305777293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ed80047c0 msgr2=0x7f5ed8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:52.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eef7fe640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ed80047c0 0x7f5ed8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eef7fe640 1 -- 172.21.15.81:0/3305777293 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed8003520 con 0x7f5ed8005180 2023-11-08T16:41:52.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.287+0000 7f5eeeffd640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ed8003970 0x7f5ed8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:52.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.288+0000 7f5eef7fe640 1 -- 172.21.15.81:0/3305777293 reap_dead start 2023-11-08T16:41:52.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.288+0000 7f5eef7fe640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 0x7f5ed800b980 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800a7f0 tx=0x7f5ed80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2cc6fa0ec6e499 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:52.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.289+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80072d0 con 0x7f5ed8005180 2023-11-08T16:41:52.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.289+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee802f070 con 0x7f5ed8005180 2023-11-08T16:41:52.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.289+0000 7f5eedffb640 1 -- 172.21.15.81:0/3305777293 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8007670 con 0x7f5ed8005180 2023-11-08T16:41:52.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.289+0000 7f5ef5439640 1 -- 172.21.15.81:0/3305777293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 msgr2=0x7f5ed800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:52.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.289+0000 7f5ef5439640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 0x7f5ed800b980 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800a7f0 tx=0x7f5ed80057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.289+0000 7f5ef5439640 1 -- 172.21.15.81:0/3305777293 shutdown_connections 2023-11-08T16:41:52.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.289+0000 7f5ef5439640 1 --2- 172.21.15.81:0/3305777293 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 0x7f5ed800b980 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.289+0000 7f5ef5439640 1 -- 172.21.15.81:0/3305777293 >> 172.21.15.81:0/3305777293 conn(0x7f5ef00ceb70 msgr2=0x7f5ef00cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:52.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.289+0000 7f5ef5439640 1 -- 172.21.15.81:0/3305777293 shutdown_connections 2023-11-08T16:41:52.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.289+0000 7f5ef5439640 1 -- 172.21.15.81:0/3305777293 wait complete. 2023-11-08T16:41:52.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.291+0000 7f5ef5439640 1 Processor -- start 2023-11-08T16:41:52.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.291+0000 7f5ef5439640 1 -- start start 2023-11-08T16:41:52.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.292+0000 7f5ef5439640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 0x7f5ef015a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.292+0000 7f5ef5439640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ef01e7420 0x7f5ef00c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.292+0000 7f5ef5439640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef01e4ab0 0x7f5ef01e4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.292+0000 7f5ef5439640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0147ce0 con 0x7f5ed8005180 2023-11-08T16:41:52.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.292+0000 7f5ef5439640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef01e5480 con 0x7f5ef01e4ab0 2023-11-08T16:41:52.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.292+0000 7f5ef5439640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef014afe0 con 0x7f5ef01e7420 2023-11-08T16:41:52.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.293+0000 7f5eeeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 0x7f5ef015a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.293+0000 7f5eeeffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 0x7f5ef015a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40626/0 (socket says 172.21.15.81:40626) 2023-11-08T16:41:52.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.293+0000 7f5eeeffd640 1 -- 172.21.15.81:0/104491901 learned_addr learned my addr 172.21.15.81:0/104491901 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:52.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.293+0000 7f5eee7fc640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ef01e7420 0x7f5ef00c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.293+0000 7f5eef7fe640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef01e4ab0 0x7f5ef01e4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.293+0000 7f5eeeffd640 1 -- 172.21.15.81:0/104491901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ef01e7420 msgr2=0x7f5ef00c2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:52.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.293+0000 7f5eeeffd640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ef01e7420 0x7f5ef00c2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.293+0000 7f5eeeffd640 1 -- 172.21.15.81:0/104491901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef01e4ab0 msgr2=0x7f5ef01e4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:52.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.293+0000 7f5eeeffd640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef01e4ab0 0x7f5ef01e4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.293+0000 7f5eeeffd640 1 -- 172.21.15.81:0/104491901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef0147060 con 0x7f5ed8005180 2023-11-08T16:41:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.293+0000 7f5eeeffd640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 0x7f5ef015a090 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00108b0 tx=0x7f5ee0010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.294+0000 7f5ed77fe640 1 -- 172.21.15.81:0/104491901 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee000eb60 con 0x7f5ed8005180 2023-11-08T16:41:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.294+0000 7f5ed77fe640 1 -- 172.21.15.81:0/104491901 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee000ecc0 con 0x7f5ed8005180 2023-11-08T16:41:52.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.294+0000 7f5ed77fe640 1 -- 172.21.15.81:0/104491901 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee001c400 con 0x7f5ed8005180 2023-11-08T16:41:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.294+0000 7f5ef5439640 1 -- 172.21.15.81:0/104491901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef0147340 con 0x7f5ed8005180 2023-11-08T16:41:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.294+0000 7f5ef5439640 1 -- 172.21.15.81:0/104491901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef0147900 con 0x7f5ed8005180 2023-11-08T16:41:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.295+0000 7f5ef5439640 1 -- 172.21.15.81:0/104491901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef0074440 con 0x7f5ed8005180 2023-11-08T16:41:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.296+0000 7f5ed77fe640 1 -- 172.21.15.81:0/104491901 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ee001c560 con 0x7f5ed8005180 2023-11-08T16:41:52.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.297+0000 7f5ed77fe640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5edc06ae70 0x7f5edc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.297+0000 7f5ed77fe640 1 -- 172.21.15.81:0/104491901 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f5ee0089d10 con 0x7f5ed8005180 2023-11-08T16:41:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.297+0000 7f5eee7fc640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5edc06ae70 0x7f5edc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:52.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.298+0000 7f5eee7fc640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5edc06ae70 0x7f5edc06d330 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4007fd0 tx=0x7f5ee4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:52.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.301+0000 7f5ed77fe640 1 -- 172.21.15.81:0/104491901 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5ee008f020 con 0x7f5ed8005180 2023-11-08T16:41:52.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:52.532+0000 7f5ef5439640 1 -- 172.21.15.81:0/104491901 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7f5ef00b16d0 con 0x7f5ed8005180 2023-11-08T16:41:53.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.068+0000 7f5ed77fe640 1 -- 172.21.15.81:0/104491901 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v116) v1 ==== 162+0+0 (secure 0 0 0) 0x7f5ee0054d70 con 0x7f5ed8005180 2023-11-08T16:41:53.070 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked joinable; MDS may join as newly active. 2023-11-08T16:41:53.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 -- 172.21.15.81:0/104491901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5edc06ae70 msgr2=0x7f5edc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:53.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5edc06ae70 0x7f5edc06d330 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f5ee4007fd0 tx=0x7f5ee4009040 comp rx=0 tx=0).stop 2023-11-08T16:41:53.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 -- 172.21.15.81:0/104491901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 msgr2=0x7f5ef015a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:53.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 0x7f5ef015a090 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00108b0 tx=0x7f5ee0010d70 comp rx=0 tx=0).stop 2023-11-08T16:41:53.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 -- 172.21.15.81:0/104491901 shutdown_connections 2023-11-08T16:41:53.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5edc06ae70 0x7f5edc06d330 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ef01e4ab0 0x7f5ef01e4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ef01e7420 0x7f5ef00c2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 --2- 172.21.15.81:0/104491901 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ed8005180 0x7f5ef015a090 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 -- 172.21.15.81:0/104491901 >> 172.21.15.81:0/104491901 conn(0x7f5ef00ceb70 msgr2=0x7f5ef015e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:53.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 -- 172.21.15.81:0/104491901 shutdown_connections 2023-11-08T16:41:53.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.072+0000 7f5ef5439640 1 -- 172.21.15.81:0/104491901 wait complete. 2023-11-08T16:41:53.096 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:41:53.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.289+0000 7f805bfff640 1 Processor -- start 2023-11-08T16:41:53.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.289+0000 7f805bfff640 1 -- start start 2023-11-08T16:41:53.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.290+0000 7f805bfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f805c159d80 0x7f805c06ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:53.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.290+0000 7f805bfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f805c15a160 0x7f805c0ce8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:53.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.290+0000 7f805bfff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f805c0cee20 0x7f805c0c9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:53.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.290+0000 7f805bfff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805c0ca510 con 0x7f805c15a160 2023-11-08T16:41:53.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.290+0000 7f805bfff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805c0cd590 con 0x7f805c0cee20 2023-11-08T16:41:53.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.290+0000 7f805bfff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805c0cf230 con 0x7f805c159d80 2023-11-08T16:41:53.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.291+0000 7f805a7fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f805c15a160 0x7f805c0ce8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:53.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.291+0000 7f805a7fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f805c15a160 0x7f805c0ce8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40636/0 (socket says 172.21.15.81:40636) 2023-11-08T16:41:53.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.291+0000 7f805a7fc640 1 -- 172.21.15.81:0/960218180 learned_addr learned my addr 172.21.15.81:0/960218180 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:53.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.291+0000 7f805affd640 1 --2- 172.21.15.81:0/960218180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f805c159d80 0x7f805c06ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:53.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.291+0000 7f805b7fe640 1 --2- 172.21.15.81:0/960218180 >> v2:172.21.15.93:3300/0 conn(0x7f805c0cee20 0x7f805c0c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:53.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.291+0000 7f805a7fc640 1 -- 172.21.15.81:0/960218180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f805c159d80 msgr2=0x7f805c06ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:53.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.291+0000 7f805a7fc640 1 --2- 172.21.15.81:0/960218180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f805c159d80 0x7f805c06ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.291+0000 7f805a7fc640 1 -- 172.21.15.81:0/960218180 >> v2:172.21.15.93:3300/0 conn(0x7f805c0cee20 msgr2=0x7f805c0c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:53.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.291+0000 7f805a7fc640 1 --2- 172.21.15.81:0/960218180 >> v2:172.21.15.93:3300/0 conn(0x7f805c0cee20 0x7f805c0c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.291+0000 7f805a7fc640 1 -- 172.21.15.81:0/960218180 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805c0c9f50 con 0x7f805c15a160 2023-11-08T16:41:53.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.291+0000 7f805affd640 1 --2- 172.21.15.81:0/960218180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f805c159d80 0x7f805c06ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:53.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.292+0000 7f805a7fc640 1 --2- 172.21.15.81:0/960218180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805c15a160 0x7f805c0ce8e0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f805c0b4aa0 tx=0x7f805002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a362df14d3fb63 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:53.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.293+0000 7f8059ffb640 1 -- 172.21.15.81:0/960218180 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80500317d0 con 0x7f805c15a160 2023-11-08T16:41:53.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.293+0000 7f8059ffb640 1 -- 172.21.15.81:0/960218180 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8050031930 con 0x7f805c15a160 2023-11-08T16:41:53.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.293+0000 7f8059ffb640 1 -- 172.21.15.81:0/960218180 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050031cc0 con 0x7f805c15a160 2023-11-08T16:41:53.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.293+0000 7f805bfff640 1 -- 172.21.15.81:0/960218180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805c15a160 msgr2=0x7f805c0ce8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:53.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.293+0000 7f805bfff640 1 --2- 172.21.15.81:0/960218180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805c15a160 0x7f805c0ce8e0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f805c0b4aa0 tx=0x7f805002b870 comp rx=0 tx=0).stop 2023-11-08T16:41:53.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.293+0000 7f805bfff640 1 -- 172.21.15.81:0/960218180 shutdown_connections 2023-11-08T16:41:53.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.293+0000 7f805bfff640 1 --2- 172.21.15.81:0/960218180 >> v2:172.21.15.93:3300/0 conn(0x7f805c0cee20 0x7f805c0c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.293+0000 7f805bfff640 1 --2- 172.21.15.81:0/960218180 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805c15a160 0x7f805c0ce8e0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.293+0000 7f805bfff640 1 --2- 172.21.15.81:0/960218180 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f805c159d80 0x7f805c06ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.293+0000 7f805bfff640 1 -- 172.21.15.81:0/960218180 >> 172.21.15.81:0/960218180 conn(0x7f805c0b06e0 msgr2=0x7f805c0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:53.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.294+0000 7f805bfff640 1 -- 172.21.15.81:0/960218180 shutdown_connections 2023-11-08T16:41:53.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.294+0000 7f805bfff640 1 -- 172.21.15.81:0/960218180 wait complete. 2023-11-08T16:41:53.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.295+0000 7f805bfff640 1 Processor -- start 2023-11-08T16:41:53.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.295+0000 7f805bfff640 1 -- start start 2023-11-08T16:41:53.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.296+0000 7f805bfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f805c0cee20 0x7f805c155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:53.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.296+0000 7f805bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805c159d80 0x7f805c152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:53.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.296+0000 7f805bfff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f805c0b2000 0x7f805c154b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:53.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.296+0000 7f805bfff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c0b15f0 con 0x7f805c159d80 2023-11-08T16:41:53.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.296+0000 7f805bfff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f805c0b24b0 con 0x7f805c0b2000 2023-11-08T16:41:53.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.296+0000 7f805bfff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f805c0c4e40 con 0x7f805c0cee20 2023-11-08T16:41:53.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805c159d80 0x7f805c152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:53.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805a7fc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805c159d80 0x7f805c152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40646/0 (socket says 172.21.15.81:40646) 2023-11-08T16:41:53.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805a7fc640 1 -- 172.21.15.81:0/1546993946 learned_addr learned my addr 172.21.15.81:0/1546993946 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:53.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805affd640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f805c0cee20 0x7f805c155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:53.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805b7fe640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f805c0b2000 0x7f805c154b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:53.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805a7fc640 1 -- 172.21.15.81:0/1546993946 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f805c0cee20 msgr2=0x7f805c155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:53.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805a7fc640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f805c0cee20 0x7f805c155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805a7fc640 1 -- 172.21.15.81:0/1546993946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f805c0b2000 msgr2=0x7f805c154b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:53.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805a7fc640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f805c0b2000 0x7f805c154b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805a7fc640 1 -- 172.21.15.81:0/1546993946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80500094d0 con 0x7f805c159d80 2023-11-08T16:41:53.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805b7fe640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f805c0b2000 0x7f805c154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:53.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.297+0000 7f805a7fc640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805c159d80 0x7f805c152c60 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f805002b8b0 tx=0x7f80500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:53.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.298+0000 7f80437fe640 1 -- 172.21.15.81:0/1546993946 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050032970 con 0x7f805c159d80 2023-11-08T16:41:53.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.298+0000 7f80437fe640 1 -- 172.21.15.81:0/1546993946 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8050032af0 con 0x7f805c159d80 2023-11-08T16:41:53.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.298+0000 7f80437fe640 1 -- 172.21.15.81:0/1546993946 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8050032ea0 con 0x7f805c159d80 2023-11-08T16:41:53.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.298+0000 7f805bfff640 1 -- 172.21.15.81:0/1546993946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f805c0c5120 con 0x7f805c159d80 2023-11-08T16:41:53.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.298+0000 7f805bfff640 1 -- 172.21.15.81:0/1546993946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805c0b1ae0 con 0x7f805c159d80 2023-11-08T16:41:53.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.299+0000 7f805bfff640 1 -- 172.21.15.81:0/1546993946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f805c0cd190 con 0x7f805c159d80 2023-11-08T16:41:53.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.300+0000 7f80437fe640 1 -- 172.21.15.81:0/1546993946 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8050030070 con 0x7f805c159d80 2023-11-08T16:41:53.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.301+0000 7f80437fe640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f804806aea0 0x7f804806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:53.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.301+0000 7f80437fe640 1 -- 172.21.15.81:0/1546993946 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f80500ab470 con 0x7f805c159d80 2023-11-08T16:41:53.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.301+0000 7f805affd640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f804806aea0 0x7f804806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:53.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.302+0000 7f805affd640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f804806aea0 0x7f804806d360 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f805c0c4b60 tx=0x7f804c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:53.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.305+0000 7f80437fe640 1 -- 172.21.15.81:0/1546993946 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f805c0cd190 con 0x7f805c159d80 2023-11-08T16:41:53.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.530+0000 7f805bfff640 1 -- 172.21.15.81:0/1546993946 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f805c0cd190 con 0x7f805c159d80 2023-11-08T16:41:53.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.531+0000 7f80437fe640 1 -- 172.21.15.81:0/1546993946 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) v1 ==== 95+0+4920 (secure 0 0 0) 0x7f805c0cd190 con 0x7f805c159d80 2023-11-08T16:41:53.533 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:53.533 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":117,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/329260749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":329260749},{"type":"v1","addr":"172.21.15.93:6835","nonce":329260749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":117,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:53.073633+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":117,"state":"up:replay","state_seq":1,"addr":"172.21.15.81:6837/1432485913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1432485913},{"type":"v1","addr":"172.21.15.81:6837","nonce":1432485913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:41:53.534 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 117 2023-11-08T16:41:53.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.534+0000 7f805bfff640 1 -- 172.21.15.81:0/1546993946 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f804806aea0 msgr2=0x7f804806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:53.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.534+0000 7f805bfff640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f804806aea0 0x7f804806d360 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f805c0c4b60 tx=0x7f804c008040 comp rx=0 tx=0).stop 2023-11-08T16:41:53.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.534+0000 7f805bfff640 1 -- 172.21.15.81:0/1546993946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805c159d80 msgr2=0x7f805c152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:53.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.534+0000 7f805bfff640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805c159d80 0x7f805c152c60 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f805002b8b0 tx=0x7f80500046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.534+0000 7f805bfff640 1 -- 172.21.15.81:0/1546993946 shutdown_connections 2023-11-08T16:41:53.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.534+0000 7f805bfff640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f804806aea0 0x7f804806d360 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.534+0000 7f805bfff640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f805c0b2000 0x7f805c154b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.534+0000 7f805bfff640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f805c159d80 0x7f805c152c60 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.534+0000 7f805bfff640 1 --2- 172.21.15.81:0/1546993946 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f805c0cee20 0x7f805c155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:53.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.534+0000 7f805bfff640 1 -- 172.21.15.81:0/1546993946 >> 172.21.15.81:0/1546993946 conn(0x7f805c0b06e0 msgr2=0x7f805c0cafa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:53.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.534+0000 7f805bfff640 1 -- 172.21.15.81:0/1546993946 shutdown_connections 2023-11-08T16:41:53.538 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:53.535+0000 7f805bfff640 1 -- 172.21.15.81:0/1546993946 wait complete. 2023-11-08T16:41:53.956 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 117, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:41:05.700435+0000', 'modified': '2023-11-08T16:41:53.073633+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 131, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8970': {'gid': 8970, 'name': 'c', 'rank': 0, 'incarnation': 117, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.81:6837/1432485913', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 1432485913}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 1432485913}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:41:53.956 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8970:up:replay 2023-11-08T16:41:54.957 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:41:55.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.180+0000 7f04d976e640 1 Processor -- start 2023-11-08T16:41:55.182 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.180+0000 7f04d976e640 1 -- start start 2023-11-08T16:41:55.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.182+0000 7f04d976e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f04d415b050 0x7f04d40cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:55.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.182+0000 7f04d976e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f04d406f920 0x7f04d4071050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:55.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.182+0000 7f04d976e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d40cc850 con 0x7f04d415ac70 2023-11-08T16:41:55.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.182+0000 7f04d976e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d40c8e20 con 0x7f04d415b050 2023-11-08T16:41:55.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.182+0000 7f04d976e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d40ccb30 con 0x7f04d406f920 2023-11-08T16:41:55.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.182+0000 7f04cbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f04d415b050 0x7f04d40cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:55.185 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.182+0000 7f04d3fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f04d415ac70 0x7f04d406ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44902/0 (socket says 172.21.15.81:44902) 2023-11-08T16:41:55.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.182+0000 7f04d3fff640 1 -- 172.21.15.81:0/3133777442 learned_addr learned my addr 172.21.15.81:0/3133777442 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:55.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.183+0000 7f04d8f6d640 1 --2- 172.21.15.81:0/3133777442 >> v2:172.21.15.93:3300/0 conn(0x7f04d406f920 0x7f04d4071050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:55.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.183+0000 7f04d37fe640 1 -- 172.21.15.81:0/3133777442 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3016313133 0 0) 0x7f04d40cc850 con 0x7f04d415ac70 2023-11-08T16:41:55.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.183+0000 7f04d37fe640 1 -- 172.21.15.81:0/3133777442 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04c0003520 con 0x7f04d415ac70 2023-11-08T16:41:55.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.183+0000 7f04cbfff640 1 -- 172.21.15.81:0/3133777442 >> v2:172.21.15.93:3300/0 conn(0x7f04d406f920 msgr2=0x7f04d4071050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:55.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.183+0000 7f04cbfff640 1 --2- 172.21.15.81:0/3133777442 >> v2:172.21.15.93:3300/0 conn(0x7f04d406f920 0x7f04d4071050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:55.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.183+0000 7f04cbfff640 1 -- 172.21.15.81:0/3133777442 >> v1:172.21.15.81:6789/0 conn(0x7f04d415ac70 legacy=0x7f04d406ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:55.187 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.183+0000 7f04cbfff640 1 -- 172.21.15.81:0/3133777442 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d40cce10 con 0x7f04d415b050 2023-11-08T16:41:55.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.184+0000 7f04cbfff640 1 --2- 172.21.15.81:0/3133777442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d415b050 0x7f04d40cd8f0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f04d40c2430 tx=0x7f04bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2199288096001c68 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:55.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.185+0000 7f04d37fe640 1 -- 172.21.15.81:0/3133777442 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc031830 con 0x7f04d415b050 2023-11-08T16:41:55.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.185+0000 7f04d37fe640 1 -- 172.21.15.81:0/3133777442 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc031990 con 0x7f04d415b050 2023-11-08T16:41:55.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.185+0000 7f04d37fe640 1 -- 172.21.15.81:0/3133777442 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc031d20 con 0x7f04d415b050 2023-11-08T16:41:55.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.185+0000 7f04d976e640 1 -- 172.21.15.81:0/3133777442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d415b050 msgr2=0x7f04d40cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:55.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.185+0000 7f04d976e640 1 --2- 172.21.15.81:0/3133777442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d415b050 0x7f04d40cd8f0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f04d40c2430 tx=0x7f04bc02b8d0 comp rx=0 tx=0).stop 2023-11-08T16:41:55.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.185+0000 7f04d976e640 1 -- 172.21.15.81:0/3133777442 shutdown_connections 2023-11-08T16:41:55.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.185+0000 7f04d976e640 1 --2- 172.21.15.81:0/3133777442 >> v2:172.21.15.93:3300/0 conn(0x7f04d406f920 0x7f04d4071050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:55.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.185+0000 7f04d976e640 1 --2- 172.21.15.81:0/3133777442 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d415b050 0x7f04d40cd8f0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:55.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.185+0000 7f04d976e640 1 -- 172.21.15.81:0/3133777442 >> 172.21.15.81:0/3133777442 conn(0x7f04d40b06e0 msgr2=0x7f04d40b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:55.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.186+0000 7f04d976e640 1 -- 172.21.15.81:0/3133777442 shutdown_connections 2023-11-08T16:41:55.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.186+0000 7f04d976e640 1 -- 172.21.15.81:0/3133777442 wait complete. 2023-11-08T16:41:55.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.187+0000 7f04d976e640 1 Processor -- start 2023-11-08T16:41:55.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.187+0000 7f04d976e640 1 -- start start 2023-11-08T16:41:55.193 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.188+0000 7f04d976e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04d406f920 0x7f04d40b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:55.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.188+0000 7f04d976e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d415ac70 0x7f04d415dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:55.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.188+0000 7f04d976e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04d415b050 0x7f04d415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:55.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.188+0000 7f04d976e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d40b3b30 con 0x7f04d415ac70 2023-11-08T16:41:55.194 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.188+0000 7f04d976e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d415c350 con 0x7f04d415b050 2023-11-08T16:41:55.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.188+0000 7f04d976e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d415c630 con 0x7f04d406f920 2023-11-08T16:41:55.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04cbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d415ac70 0x7f04d415dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:55.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04cbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d415ac70 0x7f04d415dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40660/0 (socket says 172.21.15.81:40660) 2023-11-08T16:41:55.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04cbfff640 1 -- 172.21.15.81:0/989345286 learned_addr learned my addr 172.21.15.81:0/989345286 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:55.195 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04d3fff640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04d406f920 0x7f04d40b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:55.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04d8f6d640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04d415b050 0x7f04d415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:55.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04cbfff640 1 -- 172.21.15.81:0/989345286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04d406f920 msgr2=0x7f04d40b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:55.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04cbfff640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04d406f920 0x7f04d40b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:55.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04cbfff640 1 -- 172.21.15.81:0/989345286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04d415b050 msgr2=0x7f04d415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:55.196 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04cbfff640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04d415b050 0x7f04d415ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:55.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04cbfff640 1 -- 172.21.15.81:0/989345286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04bc009400 con 0x7f04d415ac70 2023-11-08T16:41:55.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04d3fff640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04d406f920 0x7f04d40b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:55.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.189+0000 7f04d8f6d640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04d415b050 0x7f04d415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:55.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.190+0000 7f04cbfff640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d415ac70 0x7f04d415dc00 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f04bc000c00 tx=0x7f04bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:55.197 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.190+0000 7f04d17fa640 1 -- 172.21.15.81:0/989345286 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc03a830 con 0x7f04d415ac70 2023-11-08T16:41:55.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.190+0000 7f04d17fa640 1 -- 172.21.15.81:0/989345286 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04bc03a9b0 con 0x7f04d415ac70 2023-11-08T16:41:55.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.190+0000 7f04d17fa640 1 -- 172.21.15.81:0/989345286 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04bc03ad60 con 0x7f04d415ac70 2023-11-08T16:41:55.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.190+0000 7f04d976e640 1 -- 172.21.15.81:0/989345286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d415c910 con 0x7f04d415ac70 2023-11-08T16:41:55.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.190+0000 7f04d976e640 1 -- 172.21.15.81:0/989345286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d40b3e10 con 0x7f04d415ac70 2023-11-08T16:41:55.198 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.191+0000 7f04d976e640 1 -- 172.21.15.81:0/989345286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04d415c1b0 con 0x7f04d415ac70 2023-11-08T16:41:55.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.192+0000 7f04d17fa640 1 -- 172.21.15.81:0/989345286 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f04bc04d450 con 0x7f04d415ac70 2023-11-08T16:41:55.199 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.193+0000 7f04d17fa640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04b406ae70 0x7f04b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:55.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.193+0000 7f04d17fa640 1 -- 172.21.15.81:0/989345286 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f04bc0ab880 con 0x7f04d415ac70 2023-11-08T16:41:55.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.193+0000 7f04d3fff640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04b406ae70 0x7f04b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:55.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.194+0000 7f04d3fff640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04b406ae70 0x7f04b406d330 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f04c40011e0 tx=0x7f04c4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:55.200 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.197+0000 7f04d17fa640 1 -- 172.21.15.81:0/989345286 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f04d415c1b0 con 0x7f04d415ac70 2023-11-08T16:41:55.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.417+0000 7f04d976e640 1 -- 172.21.15.81:0/989345286 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04d415c1b0 con 0x7f04d415ac70 2023-11-08T16:41:55.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.418+0000 7f04d17fa640 1 -- 172.21.15.81:0/989345286 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) v1 ==== 95+0+4920 (secure 0 0 0) 0x7f04d415c1b0 con 0x7f04d415ac70 2023-11-08T16:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:55.420 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":119,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/329260749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":329260749},{"type":"v1","addr":"172.21.15.93:6835","nonce":329260749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":119,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:54.085982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":117,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.81:6837/1432485913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1432485913},{"type":"v1","addr":"172.21.15.81:6837","nonce":1432485913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:41:55.421 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 119 2023-11-08T16:41:55.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.420+0000 7f04d976e640 1 -- 172.21.15.81:0/989345286 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04b406ae70 msgr2=0x7f04b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:55.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.420+0000 7f04d976e640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04b406ae70 0x7f04b406d330 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f04c40011e0 tx=0x7f04c4009040 comp rx=0 tx=0).stop 2023-11-08T16:41:55.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.420+0000 7f04d976e640 1 -- 172.21.15.81:0/989345286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d415ac70 msgr2=0x7f04d415dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:55.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.420+0000 7f04d976e640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d415ac70 0x7f04d415dc00 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f04bc000c00 tx=0x7f04bc0046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:55.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.421+0000 7f04d976e640 1 -- 172.21.15.81:0/989345286 shutdown_connections 2023-11-08T16:41:55.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.421+0000 7f04d976e640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f04b406ae70 0x7f04b406d330 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:55.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.421+0000 7f04d976e640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f04d415b050 0x7f04d415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:55.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.421+0000 7f04d976e640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f04d415ac70 0x7f04d415dc00 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:55.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.421+0000 7f04d976e640 1 --2- 172.21.15.81:0/989345286 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f04d406f920 0x7f04d40b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:55.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.421+0000 7f04d976e640 1 -- 172.21.15.81:0/989345286 >> 172.21.15.81:0/989345286 conn(0x7f04d40b06e0 msgr2=0x7f04d40ae930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:55.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.421+0000 7f04d976e640 1 -- 172.21.15.81:0/989345286 shutdown_connections 2023-11-08T16:41:55.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:55.421+0000 7f04d976e640 1 -- 172.21.15.81:0/989345286 wait complete. 2023-11-08T16:41:55.841 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 119, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:41:05.700435+0000', 'modified': '2023-11-08T16:41:54.085982+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 131, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8970': {'gid': 8970, 'name': 'c', 'rank': 0, 'incarnation': 117, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.81:6837/1432485913', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 1432485913}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 1432485913}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:41:55.842 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_8970:up:rejoin 2023-11-08T16:41:56.843 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:41:57.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.066+0000 7f26be8e0640 1 Processor -- start 2023-11-08T16:41:57.068 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.067+0000 7f26be8e0640 1 -- start start 2023-11-08T16:41:57.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.068+0000 7f26be8e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26b806a2e0 0x7f26b80cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.070 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.068+0000 7f26be8e0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f26b806a6c0 0x7f26b80cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.068+0000 7f26be8e0640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b80ca2a0 con 0x7f26b806ee90 2023-11-08T16:41:57.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.068+0000 7f26be8e0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b806f420 con 0x7f26b806a6c0 2023-11-08T16:41:57.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.068+0000 7f26be8e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80ca4f0 con 0x7f26b806a2e0 2023-11-08T16:41:57.071 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.068+0000 7f26affff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f26b806a6c0 0x7f26b80cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.068+0000 7f26bce7d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f26b806ee90 0x7f26b80cba50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44908/0 (socket says 172.21.15.81:44908) 2023-11-08T16:41:57.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.068+0000 7f26bce7d640 1 -- 172.21.15.81:0/3401017270 learned_addr learned my addr 172.21.15.81:0/3401017270 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:57.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.069+0000 7f26b7fff640 1 --2- 172.21.15.81:0/3401017270 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26b806a2e0 0x7f26b80cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.072 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.069+0000 7f26affff640 1 -- 172.21.15.81:0/3401017270 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26b806a2e0 msgr2=0x7f26b80cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:57.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.069+0000 7f26affff640 1 --2- 172.21.15.81:0/3401017270 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26b806a2e0 0x7f26b80cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.069+0000 7f26affff640 1 -- 172.21.15.81:0/3401017270 >> v1:172.21.15.81:6789/0 conn(0x7f26b806ee90 legacy=0x7f26b80cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.069+0000 7f26affff640 1 -- 172.21.15.81:0/3401017270 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b80ca7d0 con 0x7f26b806a6c0 2023-11-08T16:41:57.073 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.069+0000 7f26b77fe640 1 -- 172.21.15.81:0/3401017270 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898465343 0 0) 0x7f26b80ca2a0 con 0x7f26b806ee90 2023-11-08T16:41:57.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.070+0000 7f26affff640 1 --2- 172.21.15.81:0/3401017270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26b806a6c0 0x7f26b80cb510 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f26a00097a0 tx=0x7f26a002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b57b8d04b13274 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:57.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.070+0000 7f26b77fe640 1 -- 172.21.15.81:0/3401017270 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a0031830 con 0x7f26b806a6c0 2023-11-08T16:41:57.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.070+0000 7f26b77fe640 1 -- 172.21.15.81:0/3401017270 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a0031990 con 0x7f26b806a6c0 2023-11-08T16:41:57.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.070+0000 7f26b77fe640 1 -- 172.21.15.81:0/3401017270 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a0031d20 con 0x7f26b806a6c0 2023-11-08T16:41:57.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.071+0000 7f26be8e0640 1 -- 172.21.15.81:0/3401017270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26b806a6c0 msgr2=0x7f26b80cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.071+0000 7f26be8e0640 1 --2- 172.21.15.81:0/3401017270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26b806a6c0 0x7f26b80cb510 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f26a00097a0 tx=0x7f26a002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.071+0000 7f26be8e0640 1 -- 172.21.15.81:0/3401017270 shutdown_connections 2023-11-08T16:41:57.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.071+0000 7f26be8e0640 1 --2- 172.21.15.81:0/3401017270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26b806a6c0 0x7f26b80cb510 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.071+0000 7f26be8e0640 1 --2- 172.21.15.81:0/3401017270 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26b806a2e0 0x7f26b80cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.071+0000 7f26be8e0640 1 -- 172.21.15.81:0/3401017270 >> 172.21.15.81:0/3401017270 conn(0x7f26b80cc6d0 msgr2=0x7f26b80ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:57.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.071+0000 7f26be8e0640 1 -- 172.21.15.81:0/3401017270 shutdown_connections 2023-11-08T16:41:57.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.071+0000 7f26be8e0640 1 -- 172.21.15.81:0/3401017270 wait complete. 2023-11-08T16:41:57.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.073+0000 7f26be8e0640 1 Processor -- start 2023-11-08T16:41:57.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.073+0000 7f26be8e0640 1 -- start start 2023-11-08T16:41:57.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.074+0000 7f26be8e0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26b806a2e0 0x7f26b80ce230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.074+0000 7f26be8e0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26b806a6c0 0x7f26b80c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.074+0000 7f26be8e0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b806ee90 0x7f26b8157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.074+0000 7f26be8e0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80c44b0 con 0x7f26b806a6c0 2023-11-08T16:41:57.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.074+0000 7f26be8e0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8158510 con 0x7f26b806ee90 2023-11-08T16:41:57.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.074+0000 7f26be8e0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b81f65b0 con 0x7f26b806a2e0 2023-11-08T16:41:57.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.074+0000 7f26affff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26b806a6c0 0x7f26b80c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.074+0000 7f26affff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26b806a6c0 0x7f26b80c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40676/0 (socket says 172.21.15.81:40676) 2023-11-08T16:41:57.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.074+0000 7f26affff640 1 -- 172.21.15.81:0/3292455565 learned_addr learned my addr 172.21.15.81:0/3292455565 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:57.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26b7fff640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26b806a2e0 0x7f26b80ce230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26affff640 1 -- 172.21.15.81:0/3292455565 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26b806a2e0 msgr2=0x7f26b80ce230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26affff640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26b806a2e0 0x7f26b80ce230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26affff640 1 -- 172.21.15.81:0/3292455565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b806ee90 msgr2=0x7f26b8157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26bce7d640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b806ee90 0x7f26b8157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26affff640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b806ee90 0x7f26b8157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26affff640 1 -- 172.21.15.81:0/3292455565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a0009400 con 0x7f26b806a6c0 2023-11-08T16:41:57.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26affff640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26b806a6c0 0x7f26b80c3f70 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f26a0000c00 tx=0x7f26a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:57.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26b57fa640 1 -- 172.21.15.81:0/3292455565 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a003aa60 con 0x7f26b806a6c0 2023-11-08T16:41:57.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26b57fa640 1 -- 172.21.15.81:0/3292455565 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a003abc0 con 0x7f26b806a6c0 2023-11-08T16:41:57.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26be8e0640 1 -- 172.21.15.81:0/3292455565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b81f6890 con 0x7f26b806a6c0 2023-11-08T16:41:57.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26b57fa640 1 -- 172.21.15.81:0/3292455565 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a00443f0 con 0x7f26b806a6c0 2023-11-08T16:41:57.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.075+0000 7f26be8e0640 1 -- 172.21.15.81:0/3292455565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b81f6e50 con 0x7f26b806a6c0 2023-11-08T16:41:57.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.077+0000 7f26be8e0640 1 -- 172.21.15.81:0/3292455565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b815bef0 con 0x7f26b806a6c0 2023-11-08T16:41:57.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.077+0000 7f26b57fa640 1 -- 172.21.15.81:0/3292455565 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f26a004d450 con 0x7f26b806a6c0 2023-11-08T16:41:57.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.078+0000 7f26b57fa640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f269806ae70 0x7f269806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.078+0000 7f26b57fa640 1 -- 172.21.15.81:0/3292455565 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f26a00ab7f0 con 0x7f26b806a6c0 2023-11-08T16:41:57.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.079+0000 7f26b7fff640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f269806ae70 0x7f269806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.079+0000 7f26b7fff640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f269806ae70 0x7f269806d330 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f26a8006fd0 tx=0x7f26a8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:57.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.083+0000 7f26b57fa640 1 -- 172.21.15.81:0/3292455565 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f26b815bef0 con 0x7f26b806a6c0 2023-11-08T16:41:57.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.296+0000 7f26be8e0640 1 -- 172.21.15.81:0/3292455565 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26b815bef0 con 0x7f26b806a6c0 2023-11-08T16:41:57.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.297+0000 7f26b57fa640 1 -- 172.21.15.81:0/3292455565 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4920 (secure 0 0 0) 0x7f26b815bef0 con 0x7f26b806a6c0 2023-11-08T16:41:57.299 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:41:57.299 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":120,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/329260749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":329260749},{"type":"v1","addr":"172.21.15.93:6835","nonce":329260749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:55.094753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":117,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/1432485913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1432485913},{"type":"v1","addr":"172.21.15.81:6837","nonce":1432485913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:41:57.300 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 120 2023-11-08T16:41:57.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.299+0000 7f26be8e0640 1 -- 172.21.15.81:0/3292455565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f269806ae70 msgr2=0x7f269806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.299+0000 7f26be8e0640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f269806ae70 0x7f269806d330 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f26a8006fd0 tx=0x7f26a8008040 comp rx=0 tx=0).stop 2023-11-08T16:41:57.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.299+0000 7f26be8e0640 1 -- 172.21.15.81:0/3292455565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26b806a6c0 msgr2=0x7f26b80c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.299+0000 7f26be8e0640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26b806a6c0 0x7f26b80c3f70 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f26a0000c00 tx=0x7f26a00046c0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.302 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.299+0000 7f26be8e0640 1 -- 172.21.15.81:0/3292455565 shutdown_connections 2023-11-08T16:41:57.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.299+0000 7f26be8e0640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f269806ae70 0x7f269806d330 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.299+0000 7f26be8e0640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b806ee90 0x7f26b8157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.299+0000 7f26be8e0640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f26b806a6c0 0x7f26b80c3f70 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.299+0000 7f26be8e0640 1 --2- 172.21.15.81:0/3292455565 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f26b806a2e0 0x7f26b80ce230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.299+0000 7f26be8e0640 1 -- 172.21.15.81:0/3292455565 >> 172.21.15.81:0/3292455565 conn(0x7f26b80cc6d0 msgr2=0x7f26b8159d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:57.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.300+0000 7f26be8e0640 1 -- 172.21.15.81:0/3292455565 shutdown_connections 2023-11-08T16:41:57.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.300+0000 7f26be8e0640 1 -- 172.21.15.81:0/3292455565 wait complete. 2023-11-08T16:41:57.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 120, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-08T16:41:05.700435+0000', 'modified': '2023-11-08T16:41:55.094753+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 131, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8970': {'gid': 8970, 'name': 'c', 'rank': 0, 'incarnation': 117, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.81:6837/1432485913', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.81:6836', 'nonce': 1432485913}, {'type': 'v1', 'addr': '172.21.15.81:6837', 'nonce': 1432485913}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-08T16:41:57.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-08T16:41:57.724 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-08T16:41:57.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.948+0000 7fa8425f5640 1 Processor -- start 2023-11-08T16:41:57.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.948+0000 7fa8425f5640 1 -- start start 2023-11-08T16:41:57.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.950+0000 7fa8425f5640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa83c15d200 0x7fa83c0b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.950+0000 7fa8425f5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83c15e660 con 0x7fa83c0b8e50 2023-11-08T16:41:57.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.950+0000 7fa8425f5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83c15dae0 con 0x7fa83c0d0230 2023-11-08T16:41:57.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.950+0000 7fa8425f5640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83c15ddc0 con 0x7fa83c15d200 2023-11-08T16:41:57.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.950+0000 7fa83b7fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa83c0b8e50 0x7fa83c0cfb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44920/0 (socket says 172.21.15.81:44920) 2023-11-08T16:41:57.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.950+0000 7fa83b7fe640 1 -- 172.21.15.81:0/2439868556 learned_addr learned my addr 172.21.15.81:0/2439868556 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:57.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.950+0000 7fa83bfff640 1 --2- 172.21.15.81:0/2439868556 >> v2:172.21.15.81:3300/0 conn(0x7fa83c15d200 0x7fa83c0b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.951+0000 7fa83bfff640 1 -- 172.21.15.81:0/2439868556 >> v1:172.21.15.93:6789/0 conn(0x7fa83c0d0230 legacy=0x7fa83c0d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:57.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.951+0000 7fa83bfff640 1 -- 172.21.15.81:0/2439868556 >> v1:172.21.15.81:6789/0 conn(0x7fa83c0b8e50 legacy=0x7fa83c0cfb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.951+0000 7fa83affd640 1 -- 172.21.15.81:0/2439868556 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 533590617 0 0) 0x7fa83c15e660 con 0x7fa83c0b8e50 2023-11-08T16:41:57.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.951+0000 7fa83bfff640 1 -- 172.21.15.81:0/2439868556 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c15e0a0 con 0x7fa83c15d200 2023-11-08T16:41:57.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.952+0000 7fa83bfff640 1 --2- 172.21.15.81:0/2439868556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa83c15d200 0x7fa83c0b8910 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009d90 tx=0x7fa82c02bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87cae23092668a5a server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:57.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.952+0000 7fa83affd640 1 -- 172.21.15.81:0/2439868556 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c02cb40 con 0x7fa83c15d200 2023-11-08T16:41:57.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.952+0000 7fa83affd640 1 -- 172.21.15.81:0/2439868556 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c02cca0 con 0x7fa83c15d200 2023-11-08T16:41:57.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.952+0000 7fa83affd640 1 -- 172.21.15.81:0/2439868556 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c034660 con 0x7fa83c15d200 2023-11-08T16:41:57.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.953+0000 7fa8425f5640 1 -- 172.21.15.81:0/2439868556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa83c15d200 msgr2=0x7fa83c0b8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.953+0000 7fa8425f5640 1 --2- 172.21.15.81:0/2439868556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa83c15d200 0x7fa83c0b8910 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009d90 tx=0x7fa82c02bd90 comp rx=0 tx=0).stop 2023-11-08T16:41:57.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.953+0000 7fa8425f5640 1 -- 172.21.15.81:0/2439868556 shutdown_connections 2023-11-08T16:41:57.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.953+0000 7fa8425f5640 1 --2- 172.21.15.81:0/2439868556 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa83c15d200 0x7fa83c0b8910 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.953+0000 7fa8425f5640 1 -- 172.21.15.81:0/2439868556 >> 172.21.15.81:0/2439868556 conn(0x7fa83c06aae0 msgr2=0x7fa83c06aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:57.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.953+0000 7fa8425f5640 1 -- 172.21.15.81:0/2439868556 shutdown_connections 2023-11-08T16:41:57.957 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.953+0000 7fa8425f5640 1 -- 172.21.15.81:0/2439868556 wait complete. 2023-11-08T16:41:57.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.955+0000 7fa8425f5640 1 Processor -- start 2023-11-08T16:41:57.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.955+0000 7fa8425f5640 1 -- start start 2023-11-08T16:41:57.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.956+0000 7fa8425f5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa83c0b8e50 0x7fa83c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.956+0000 7fa8425f5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa83c0d0230 0x7fa83c159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.956+0000 7fa8425f5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa83c15d200 0x7fa83c14d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.956+0000 7fa8425f5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c1588e0 con 0x7fa83c0b8e50 2023-11-08T16:41:57.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.956+0000 7fa8425f5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c158bc0 con 0x7fa83c15d200 2023-11-08T16:41:57.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.956+0000 7fa8425f5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c158ea0 con 0x7fa83c0d0230 2023-11-08T16:41:57.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.956+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa83c0b8e50 0x7fa83c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.956+0000 7fa83bfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa83c0b8e50 0x7fa83c15b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40696/0 (socket says 172.21.15.81:40696) 2023-11-08T16:41:57.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.956+0000 7fa83bfff640 1 -- 172.21.15.81:0/2938828971 learned_addr learned my addr 172.21.15.81:0/2938828971 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:57.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa840b92640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa83c15d200 0x7fa83c14d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa83bfff640 1 -- 172.21.15.81:0/2938828971 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa83c0d0230 msgr2=0x7fa83c159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa83b7fe640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa83c0d0230 0x7fa83c159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa83bfff640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa83c0d0230 0x7fa83c159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa83bfff640 1 -- 172.21.15.81:0/2938828971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa83c15d200 msgr2=0x7fa83c14d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa83bfff640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa83c15d200 0x7fa83c14d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa83bfff640 1 -- 172.21.15.81:0/2938828971 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c009a70 con 0x7fa83c0b8e50 2023-11-08T16:41:57.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa840b92640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa83c15d200 0x7fa83c14d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:57.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa83bfff640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa83c0b8e50 0x7fa83c15b280 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fa82c037040 tx=0x7fa82c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:57.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa838ff9640 1 -- 172.21.15.81:0/2938828971 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c03e610 con 0x7fa83c0b8e50 2023-11-08T16:41:57.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa838ff9640 1 -- 172.21.15.81:0/2938828971 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c03e770 con 0x7fa83c0b8e50 2023-11-08T16:41:57.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa838ff9640 1 -- 172.21.15.81:0/2938828971 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c03eb20 con 0x7fa83c0b8e50 2023-11-08T16:41:57.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.957+0000 7fa8425f5640 1 -- 172.21.15.81:0/2938828971 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c159180 con 0x7fa83c0b8e50 2023-11-08T16:41:57.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.958+0000 7fa8425f5640 1 -- 172.21.15.81:0/2938828971 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c0cb410 con 0x7fa83c0b8e50 2023-11-08T16:41:57.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.960+0000 7fa838ff9640 1 -- 172.21.15.81:0/2938828971 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa82c03b050 con 0x7fa83c0b8e50 2023-11-08T16:41:57.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.961+0000 7fa838ff9640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa81c06ae70 0x7fa81c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.961+0000 7fa838ff9640 1 -- 172.21.15.81:0/2938828971 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa82c0ac2e0 con 0x7fa83c0b8e50 2023-11-08T16:41:57.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.962+0000 7fa83b7fe640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa81c06ae70 0x7fa81c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.962+0000 7fa845187740 1 Processor -- start 2023-11-08T16:41:57.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.962+0000 7fa845187740 1 -- start start 2023-11-08T16:41:57.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.962+0000 7fa83b7fe640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa81c06ae70 0x7fa81c06d330 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa824007fd0 tx=0x7fa824009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:57.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.963+0000 7fa845187740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956abdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.963+0000 7fa845187740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606956d92a0 0x5606956dbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.963+0000 7fa845187740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956ac300 0x5606956dc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.963+0000 7fa845187740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069561e7e0 con 0x5606956d92a0 2023-11-08T16:41:57.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.963+0000 7fa845187740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56069561bed0 con 0x5606956ac300 2023-11-08T16:41:57.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.963+0000 7fa845187740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606956177c0 con 0x5606956d8ec0 2023-11-08T16:41:57.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.963+0000 7fa83b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606956d92a0 0x5606956dbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.963+0000 7fa83b7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606956d92a0 0x5606956dbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40710/0 (socket says 172.21.15.81:40710) 2023-11-08T16:41:57.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.963+0000 7fa83b7fe640 1 -- 172.21.15.81:0/2346279869 learned_addr learned my addr 172.21.15.81:0/2346279869 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:57.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa83bfff640 1 --2- 172.21.15.81:0/2346279869 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956abdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa83b7fe640 1 -- 172.21.15.81:0/2346279869 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 msgr2=0x5606956abdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:57.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa840b92640 1 --2- 172.21.15.81:0/2346279869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956ac300 0x5606956dc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa83b7fe640 1 --2- 172.21.15.81:0/2346279869 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956abdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa83b7fe640 1 -- 172.21.15.81:0/2346279869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956ac300 msgr2=0x5606956dc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:57.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa83b7fe640 1 --2- 172.21.15.81:0/2346279869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956ac300 0x5606956dc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa83b7fe640 1 -- 172.21.15.81:0/2346279869 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606955eab10 con 0x5606956d92a0 2023-11-08T16:41:57.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa83bfff640 1 --2- 172.21.15.81:0/2346279869 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:57.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa83b7fe640 1 --2- 172.21.15.81:0/2346279869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606956d92a0 0x5606956dbc50 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fa824007230 tx=0x7fa824010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866d7920e0a2fa53 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:57.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa831ffb640 1 -- 172.21.15.81:0/2346279869 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400d9a0 con 0x5606956d92a0 2023-11-08T16:41:57.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa831ffb640 1 -- 172.21.15.81:0/2346279869 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82400db00 con 0x5606956d92a0 2023-11-08T16:41:57.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.964+0000 7fa831ffb640 1 -- 172.21.15.81:0/2346279869 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82400de70 con 0x5606956d92a0 2023-11-08T16:41:57.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.965+0000 7fa845187740 1 -- 172.21.15.81:0/2346279869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606956d92a0 msgr2=0x5606956dbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.965+0000 7fa845187740 1 --2- 172.21.15.81:0/2346279869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606956d92a0 0x5606956dbc50 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fa824007230 tx=0x7fa824010000 comp rx=0 tx=0).stop 2023-11-08T16:41:57.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.965+0000 7fa845187740 1 -- 172.21.15.81:0/2346279869 shutdown_connections 2023-11-08T16:41:57.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.965+0000 7fa845187740 1 --2- 172.21.15.81:0/2346279869 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956ac300 0x5606956dc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.965+0000 7fa845187740 1 --2- 172.21.15.81:0/2346279869 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x5606956d92a0 0x5606956dbc50 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.965+0000 7fa845187740 1 --2- 172.21.15.81:0/2346279869 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956abdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.965+0000 7fa845187740 1 -- 172.21.15.81:0/2346279869 >> 172.21.15.81:0/2346279869 conn(0x5606956ae0d0 msgr2=0x5606956f6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:57.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.965+0000 7fa845187740 1 -- 172.21.15.81:0/2346279869 shutdown_connections 2023-11-08T16:41:57.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.965+0000 7fa845187740 1 -- 172.21.15.81:0/2346279869 wait complete. 2023-11-08T16:41:57.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.966+0000 7fa845187740 1 -- start start 2023-11-08T16:41:57.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.966+0000 7fa845187740 1 Processor -- start 2023-11-08T16:41:57.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.967+0000 7fa845187740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d8ec0 0x5606956c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.967+0000 7fa845187740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d92a0 0x5606956df810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.967+0000 7fa845187740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956f00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.967+0000 7fa845187740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606955b4c40 con 0x56069570ac90 2023-11-08T16:41:57.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.967+0000 7fa845187740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606955b48e0 con 0x5606956d8ec0 2023-11-08T16:41:57.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.967+0000 7fa845187740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x56069537af20 con 0x5606956d92a0 2023-11-08T16:41:57.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.967+0000 7fa845187740 10 client.0 _ll_register_callbacks cb 0x5606955d6cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:41:57.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.967+0000 7fa840b92640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956f00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.967+0000 7fa840b92640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956f00a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40714/0 (socket says 172.21.15.81:40714) 2023-11-08T16:41:57.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.967+0000 7fa840b92640 1 -- 172.21.15.81:0/3955307153 learned_addr learned my addr 172.21.15.81:0/3955307153 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:57.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa83b7fe640 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d92a0 0x5606956df810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa83bfff640 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d8ec0 0x5606956c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa840b92640 1 -- 172.21.15.81:0/3955307153 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d92a0 msgr2=0x5606956df810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:57.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa840b92640 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d92a0 0x5606956df810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa840b92640 1 -- 172.21.15.81:0/3955307153 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d8ec0 msgr2=0x5606956c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:57.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa840b92640 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d8ec0 0x5606956c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa840b92640 1 -- 172.21.15.81:0/3955307153 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa824001d10 con 0x56069570ac90 2023-11-08T16:41:57.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa840b92640 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956f00a0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fa8340054a0 tx=0x7fa83400e590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d468ea2216267e32 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:57.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa81affd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:41:57.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa81affd640 1 -- 172.21.15.81:0/3955307153 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834003a60 con 0x56069570ac90 2023-11-08T16:41:57.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa81affd640 1 -- 172.21.15.81:0/3955307153 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa834003bc0 con 0x56069570ac90 2023-11-08T16:41:57.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.968+0000 7fa845187740 1 -- 172.21.15.81:0/3955307153 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5606955ebc80 con 0x56069570ac90 2023-11-08T16:41:57.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa81affd640 1 -- 172.21.15.81:0/3955307153 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa834003d80 con 0x56069570ac90 2023-11-08T16:41:57.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa845187740 10 client.9066 fetch_fsmap learned FSMap version 120 2023-11-08T16:41:57.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa845187740 1 -- 172.21.15.81:0/3955307153 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) v3 -- 0x560695615e40 con 0x56069570ac90 2023-11-08T16:41:57.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa81affd640 1 -- 172.21.15.81:0/3955307153 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5606955ebc80 con 0x56069570ac90 2023-11-08T16:41:57.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa81affd640 1 -- 172.21.15.81:0/3955307153 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 120) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa8340067d0 con 0x56069570ac90 2023-11-08T16:41:57.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa845187740 10 client.9066 fetch_fsmap finished waiting for FSMap version 120 2023-11-08T16:41:57.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa845187740 10 client.9066 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:41:57.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa845187740 20 client.9066 populate_metadata read hostname 'smithi081' 2023-11-08T16:41:57.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa845187740 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.81:6836/1432485913,v1:172.21.15.81:6837/1432485913] conn(0x5606956cb9c0 0x5606956f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:57.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa845187740 4 client.9066 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-08T16:41:57.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa845187740 1 -- 172.21.15.81:0/3955307153 --> [v2:172.21.15.81:6836/1432485913,v1:172.21.15.81:6837/1432485913] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5606956107a0 con 0x5606956cb9c0 2023-11-08T16:41:57.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.969+0000 7fa83bfff640 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.81:6836/1432485913,v1:172.21.15.81:6837/1432485913] conn(0x5606956cb9c0 0x5606956f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:57.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.970+0000 7fa83bfff640 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.81:6836/1432485913,v1:172.21.15.81:6837/1432485913] conn(0x5606956cb9c0 0x5606956f6e80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e319b46cccc567f2 server_cookie=5ef3b419a79cc583 in_seq=0 out_seq=0 2023-11-08T16:41:57.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.970+0000 7fa81affd640 10 client.9066 ms_handle_connect on v2:172.21.15.81:6836/1432485913 2023-11-08T16:41:57.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa81affd640 1 -- 172.21.15.81:0/3955307153 <== mds.0 v2:172.21.15.81:6836/1432485913 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5606956107a0 con 0x5606956cb9c0 2023-11-08T16:41:57.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa81affd640 10 client.9066 handle_command_reply: tid=0 2023-11-08T16:41:57.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 1 client.9066 shutdown 2023-11-08T16:41:57.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 1 -- 172.21.15.81:0/3955307153 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 msgr2=0x5606956f00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:57.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956f00a0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fa8340054a0 tx=0x7fa83400e590 comp rx=0 tx=0).stop 2023-11-08T16:41:57.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 1 -- 172.21.15.81:0/3955307153 shutdown_connections 2023-11-08T16:41:57.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.81:6836/1432485913,v1:172.21.15.81:6837/1432485913] conn(0x5606956cb9c0 0x5606956f6e80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956f00a0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d92a0 0x5606956df810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa81affd640 0 client.9066 ms_handle_reset on v2:172.21.15.81:6836/1432485913 2023-11-08T16:41:57.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 1 --2- 172.21.15.81:0/3955307153 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d8ec0 0x5606956c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:57.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 1 -- 172.21.15.81:0/3955307153 >> 172.21.15.81:0/3955307153 conn(0x5606956ac300 msgr2=0x5606956afc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:57.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 1 -- 172.21.15.81:0/3955307153 shutdown_connections 2023-11-08T16:41:57.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 1 -- 172.21.15.81:0/3955307153 wait complete. 2023-11-08T16:41:57.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.971+0000 7fa845187740 20 client.9066 trim_cache size 0 max 16384 2023-11-08T16:41:58.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.988+0000 7fa845187740 1 Processor -- start 2023-11-08T16:41:58.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.988+0000 7fa845187740 1 -- start start 2023-11-08T16:41:58.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.989+0000 7fa845187740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956d6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.989+0000 7fa845187740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d92a0 0x5606956c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.989+0000 7fa845187740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956c5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.989+0000 7fa845187740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606956ab610 con 0x56069570ac90 2023-11-08T16:41:58.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.989+0000 7fa845187740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606956ab1a0 con 0x5606956d92a0 2023-11-08T16:41:58.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.989+0000 7fa845187740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x5606956f08d0 con 0x5606956d8ec0 2023-11-08T16:41:58.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.990+0000 7fa840b92640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.990+0000 7fa840b92640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40730/0 (socket says 172.21.15.81:40730) 2023-11-08T16:41:58.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.990+0000 7fa840b92640 1 -- 172.21.15.81:0/1504233487 learned_addr learned my addr 172.21.15.81:0/1504233487 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:58.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.990+0000 7fa83bfff640 1 --2- 172.21.15.81:0/1504233487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956d6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.990+0000 7fa83b7fe640 1 --2- 172.21.15.81:0/1504233487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d92a0 0x5606956c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.990+0000 7fa840b92640 1 -- 172.21.15.81:0/1504233487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 msgr2=0x5606956d6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:58.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.990+0000 7fa840b92640 1 --2- 172.21.15.81:0/1504233487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956d6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.990+0000 7fa840b92640 1 -- 172.21.15.81:0/1504233487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d92a0 msgr2=0x5606956c4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:58.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.990+0000 7fa840b92640 1 --2- 172.21.15.81:0/1504233487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d92a0 0x5606956c4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.990+0000 7fa840b92640 1 -- 172.21.15.81:0/1504233487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5606955b3b90 con 0x56069570ac90 2023-11-08T16:41:58.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.991+0000 7fa840b92640 1 --2- 172.21.15.81:0/1504233487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956c5450 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa8340050f0 tx=0x7fa834012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf69434651ddeb1f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:58.007 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.991+0000 7fa8327fc640 1 -- 172.21.15.81:0/1504233487 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834007ea0 con 0x56069570ac90 2023-11-08T16:41:58.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.991+0000 7fa8327fc640 1 -- 172.21.15.81:0/1504233487 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83400d040 con 0x56069570ac90 2023-11-08T16:41:58.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.991+0000 7fa8327fc640 1 -- 172.21.15.81:0/1504233487 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834003aa0 con 0x56069570ac90 2023-11-08T16:41:58.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.992+0000 7fa845187740 1 -- 172.21.15.81:0/1504233487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 msgr2=0x5606956c5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:58.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.992+0000 7fa845187740 1 --2- 172.21.15.81:0/1504233487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956c5450 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fa8340050f0 tx=0x7fa834012000 comp rx=0 tx=0).stop 2023-11-08T16:41:58.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.992+0000 7fa845187740 1 -- 172.21.15.81:0/1504233487 shutdown_connections 2023-11-08T16:41:58.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.992+0000 7fa845187740 1 --2- 172.21.15.81:0/1504233487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606956c5450 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.992+0000 7fa845187740 1 --2- 172.21.15.81:0/1504233487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d92a0 0x5606956c4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.992+0000 7fa845187740 1 --2- 172.21.15.81:0/1504233487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956d6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.992+0000 7fa845187740 1 -- 172.21.15.81:0/1504233487 >> 172.21.15.81:0/1504233487 conn(0x5606956ac300 msgr2=0x5606956fa160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:58.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.992+0000 7fa845187740 1 -- 172.21.15.81:0/1504233487 shutdown_connections 2023-11-08T16:41:58.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.992+0000 7fa845187740 1 -- 172.21.15.81:0/1504233487 wait complete. 2023-11-08T16:41:58.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.992+0000 7fa845187740 1 -- start start 2023-11-08T16:41:58.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.993+0000 7fa845187740 1 Processor -- start 2023-11-08T16:41:58.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa845187740 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956f6f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa845187740 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d92a0 0x5606956f7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa845187740 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606957113e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa845187740 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x560695762690 con 0x56069570ac90 2023-11-08T16:41:58.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa845187740 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5606956dec10 con 0x5606956d92a0 2023-11-08T16:41:58.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa845187740 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x560695761a10 con 0x5606956d8ec0 2023-11-08T16:41:58.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa845187740 10 client.0 _ll_register_callbacks cb 0x7fa834007c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-08T16:41:58.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa840b92640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606957113e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa840b92640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606957113e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40742/0 (socket says 172.21.15.81:40742) 2023-11-08T16:41:58.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa840b92640 1 -- 172.21.15.81:0/2196271944 learned_addr learned my addr 172.21.15.81:0/2196271944 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:58.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa83b7fe640 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d92a0 0x5606956f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa83bfff640 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956f6f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa840b92640 1 -- 172.21.15.81:0/2196271944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 msgr2=0x5606956f6f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:58.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa840b92640 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956f6f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa840b92640 1 -- 172.21.15.81:0/2196271944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d92a0 msgr2=0x5606956f7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:58.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa840b92640 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d92a0 0x5606956f7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa840b92640 1 -- 172.21.15.81:0/2196271944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560695615e40 con 0x56069570ac90 2023-11-08T16:41:58.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.994+0000 7fa83b7fe640 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d92a0 0x5606956f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:58.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.995+0000 7fa840b92640 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606957113e0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa834000c00 tx=0x7fa834003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ba001fc3d83250 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:58.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.995+0000 7fa81affd640 10 client.0 ms_handle_connect on v2:172.21.15.81:3300/0 2023-11-08T16:41:58.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.995+0000 7fa81affd640 1 -- 172.21.15.81:0/2196271944 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834033070 con 0x56069570ac90 2023-11-08T16:41:58.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.995+0000 7fa81affd640 1 -- 172.21.15.81:0/2196271944 <== mon.0 v2:172.21.15.81:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa834000e40 con 0x56069570ac90 2023-11-08T16:41:58.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.995+0000 7fa845187740 1 -- 172.21.15.81:0/2196271944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa82400db00 con 0x56069570ac90 2023-11-08T16:41:58.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.995+0000 7fa81affd640 1 -- 172.21.15.81:0/2196271944 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_get_version_reply(handle=1 version=120) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa834003bf0 con 0x56069570ac90 2023-11-08T16:41:58.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.995+0000 7fa845187740 10 client.9072 fetch_fsmap learned FSMap version 120 2023-11-08T16:41:58.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.995+0000 7fa845187740 1 -- 172.21.15.81:0/2196271944 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,fsmap=120,monmap=2+}) v3 -- 0x5606955d9180 con 0x56069570ac90 2023-11-08T16:41:58.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.996+0000 7fa81affd640 1 -- 172.21.15.81:0/2196271944 <== mon.0 v2:172.21.15.81:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82400db00 con 0x56069570ac90 2023-11-08T16:41:58.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.996+0000 7fa81affd640 1 -- 172.21.15.81:0/2196271944 <== mon.0 v2:172.21.15.81:3300/0 5 ==== fsmap(e 120) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa834004040 con 0x56069570ac90 2023-11-08T16:41:58.017 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.996+0000 7fa845187740 10 client.9072 fetch_fsmap finished waiting for FSMap version 120 2023-11-08T16:41:58.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.996+0000 7fa845187740 10 client.9072 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-08T16:41:58.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.996+0000 7fa845187740 20 client.9072 populate_metadata read hostname 'smithi081' 2023-11-08T16:41:58.018 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.996+0000 7fa845187740 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.81:6836/1432485913,v1:172.21.15.81:6837/1432485913] conn(0x560695711920 0x560695711d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.996+0000 7fa845187740 4 client.9072 mds_command: new command op to 8970 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-08T16:41:58.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.996+0000 7fa845187740 1 -- 172.21.15.81:0/2196271944 --> [v2:172.21.15.81:6836/1432485913,v1:172.21.15.81:6837/1432485913] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5606956104c0 con 0x560695711920 2023-11-08T16:41:58.019 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.996+0000 7fa83bfff640 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.81:6836/1432485913,v1:172.21.15.81:6837/1432485913] conn(0x560695711920 0x560695711d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa83bfff640 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.81:6836/1432485913,v1:172.21.15.81:6837/1432485913] conn(0x560695711920 0x560695711d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ddafd2c70dbb71c server_cookie=8973d34e809e98ba in_seq=0 out_seq=0 2023-11-08T16:41:58.020 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa81affd640 10 client.9072 ms_handle_connect on v2:172.21.15.81:6836/1432485913 2023-11-08T16:41:58.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa81affd640 1 -- 172.21.15.81:0/2196271944 <== mds.0 v2:172.21.15.81:6836/1432485913 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x5606956104c0 con 0x560695711920 2023-11-08T16:41:58.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa81affd640 10 client.9072 handle_command_reply: tid=0 2023-11-08T16:41:58.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 1 client.9072 shutdown 2023-11-08T16:41:58.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 1 -- 172.21.15.81:0/2196271944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 msgr2=0x5606957113e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:58.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606957113e0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa834000c00 tx=0x7fa834003760 comp rx=0 tx=0).stop 2023-11-08T16:41:58.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 1 -- 172.21.15.81:0/2196271944 shutdown_connections 2023-11-08T16:41:58.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.81:6836/1432485913,v1:172.21.15.81:6837/1432485913] conn(0x560695711920 0x560695711d00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x56069570ac90 0x5606957113e0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5606956d92a0 0x5606956f7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 1 --2- 172.21.15.81:0/2196271944 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x5606956d8ec0 0x5606956f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa81affd640 0 client.9072 ms_handle_reset on v2:172.21.15.81:6836/1432485913 2023-11-08T16:41:58.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 1 -- 172.21.15.81:0/2196271944 >> 172.21.15.81:0/2196271944 conn(0x5606956ac300 msgr2=0x5606956d6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:58.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 1 -- 172.21.15.81:0/2196271944 shutdown_connections 2023-11-08T16:41:58.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 1 -- 172.21.15.81:0/2196271944 wait complete. 2023-11-08T16:41:58.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.997+0000 7fa845187740 20 client.9072 trim_cache size 0 max 16384 2023-11-08T16:41:58.026 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:41:58.026 INFO:teuthology.orchestra.run.smithi081.stdout: "cluster_fsid": "49b142e1-3087-4ba1-853b-235f756dae94", 2023-11-08T16:41:58.027 INFO:teuthology.orchestra.run.smithi081.stdout: "whoami": 0, 2023-11-08T16:41:58.027 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 8970, 2023-11-08T16:41:58.027 INFO:teuthology.orchestra.run.smithi081.stdout: "want_state": "up:active", 2023-11-08T16:41:58.027 INFO:teuthology.orchestra.run.smithi081.stdout: "state": "up:active", 2023-11-08T16:41:58.027 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs", 2023-11-08T16:41:58.028 INFO:teuthology.orchestra.run.smithi081.stdout: "rank_uptime": 4.9193323400000004, 2023-11-08T16:41:58.028 INFO:teuthology.orchestra.run.smithi081.stdout: "mdsmap_epoch": 120, 2023-11-08T16:41:58.028 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch": 131, 2023-11-08T16:41:58.028 INFO:teuthology.orchestra.run.smithi081.stdout: "osdmap_epoch_barrier": 131, 2023-11-08T16:41:58.028 INFO:teuthology.orchestra.run.smithi081.stdout: "uptime": 8.7832564620000007 2023-11-08T16:41:58.029 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:41:58.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 -- 172.21.15.81:0/2938828971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa81c06ae70 msgr2=0x7fa81c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:58.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa81c06ae70 0x7fa81c06d330 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fa824007fd0 tx=0x7fa824009040 comp rx=0 tx=0).stop 2023-11-08T16:41:58.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 -- 172.21.15.81:0/2938828971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa83c0b8e50 msgr2=0x7fa83c15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:58.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa83c0b8e50 0x7fa83c15b280 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7fa82c037040 tx=0x7fa82c004a40 comp rx=0 tx=0).stop 2023-11-08T16:41:58.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 -- 172.21.15.81:0/2938828971 shutdown_connections 2023-11-08T16:41:58.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa81c06ae70 0x7fa81c06d330 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa83c15d200 0x7fa83c14d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa83c0d0230 0x7fa83c159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 --2- 172.21.15.81:0/2938828971 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa83c0b8e50 0x7fa83c15b280 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 -- 172.21.15.81:0/2938828971 >> 172.21.15.81:0/2938828971 conn(0x7fa83c06aae0 msgr2=0x7fa83c0b3160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:58.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 -- 172.21.15.81:0/2938828971 shutdown_connections 2023-11-08T16:41:58.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:57.998+0000 7fa8425f5640 1 -- 172.21.15.81:0/2938828971 wait complete. 2023-11-08T16:41:58.033 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-08T16:41:58.236 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.235+0000 7fe65a8bb640 1 Processor -- start 2023-11-08T16:41:58.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.235+0000 7fe65a8bb640 1 -- start start 2023-11-08T16:41:58.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.237+0000 7fe65a8bb640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe65415d580 0x7fe6540cf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.237+0000 7fe65a8bb640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe6540b36b0 0x7fe6540cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.237+0000 7fe65a8bb640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6540d38b0 con 0x7fe6540b43b0 2023-11-08T16:41:58.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.237+0000 7fe65a8bb640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6540cee50 con 0x7fe6540b36b0 2023-11-08T16:41:58.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.237+0000 7fe65a8bb640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65415d960 con 0x7fe65415d580 2023-11-08T16:41:58.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.237+0000 7fe6590b8640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe6540b43b0 0x7fe6540d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:44922/0 (socket says 172.21.15.81:44922) 2023-11-08T16:41:58.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.237+0000 7fe6590b8640 1 -- 172.21.15.81:0/2699538628 learned_addr learned my addr 172.21.15.81:0/2699538628 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:41:58.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.237+0000 7fe65a0ba640 1 --2- 172.21.15.81:0/2699538628 >> v2:172.21.15.93:3300/0 conn(0x7fe6540b36b0 0x7fe6540cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.237+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1041579181 0 0) 0x7fe6540d38b0 con 0x7fe6540b43b0 2023-11-08T16:41:58.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.237+0000 7fe6598b9640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe65415d580 0x7fe6540cf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe640003520 con 0x7fe6540b43b0 2023-11-08T16:41:58.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 479839723 0 0) 0x7fe640003520 con 0x7fe6540b43b0 2023-11-08T16:41:58.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe65415d580 msgr2=0x7fe6540cf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:58.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe65415d580 0x7fe6540cf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 >> v2:172.21.15.93:3300/0 conn(0x7fe6540b36b0 msgr2=0x7fe6540cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:58.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 --2- 172.21.15.81:0/2699538628 >> v2:172.21.15.93:3300/0 conn(0x7fe6540b36b0 0x7fe6540cfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6540ce560 con 0x7fe6540b43b0 2023-11-08T16:41:58.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fe644001670 con 0x7fe6540b43b0 2023-11-08T16:41:58.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 >> v1:172.21.15.81:6789/0 conn(0x7fe6540b43b0 legacy=0x7fe6540d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:58.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe65415d580 0x7fe640003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe640003ef0 0x7fe6400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe640004d00 0x7fe640005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6540cee50 con 0x7fe640004d00 2023-11-08T16:41:58.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65415d960 con 0x7fe640003ef0 2023-11-08T16:41:58.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64000b980 con 0x7fe65415d580 2023-11-08T16:41:58.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.238+0000 7fe65a0ba640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe640004d00 0x7fe640005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.239+0000 7fe6590b8640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe640003ef0 0x7fe6400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.239+0000 7fe6598b9640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe65415d580 0x7fe640003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.239+0000 7fe65a0ba640 1 -- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe65415d580 msgr2=0x7fe640003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:58.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.239+0000 7fe65a0ba640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe65415d580 0x7fe640003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.239+0000 7fe65a0ba640 1 -- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe640003ef0 msgr2=0x7fe6400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:41:58.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.239+0000 7fe65a0ba640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe640003ef0 0x7fe6400047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.239+0000 7fe65a0ba640 1 -- 172.21.15.81:0/2699538628 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe640003520 con 0x7fe640004d00 2023-11-08T16:41:58.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.239+0000 7fe6598b9640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe65415d580 0x7fe640003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:58.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.240+0000 7fe65a0ba640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe640004d00 0x7fe640005180 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009f90 tx=0x7fe6400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a39a1c47271a309 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:58.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.240+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c02cc00 con 0x7fe640004d00 2023-11-08T16:41:58.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.241+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c00f040 con 0x7fe640004d00 2023-11-08T16:41:58.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.241+0000 7fe6588b7640 1 -- 172.21.15.81:0/2699538628 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c034440 con 0x7fe640004d00 2023-11-08T16:41:58.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.241+0000 7fe65a8bb640 1 -- 172.21.15.81:0/2699538628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe640004d00 msgr2=0x7fe640005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:58.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.241+0000 7fe65a8bb640 1 --2- 172.21.15.81:0/2699538628 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe640004d00 0x7fe640005180 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009f90 tx=0x7fe6400057f0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.241+0000 7fe65a0ba640 1 -- 172.21.15.81:0/2699538628 reap_dead start 2023-11-08T16:41:58.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.241+0000 7fe65a8bb640 1 -- 172.21.15.81:0/2699538628 shutdown_connections 2023-11-08T16:41:58.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.241+0000 7fe65a8bb640 1 -- 172.21.15.81:0/2699538628 >> 172.21.15.81:0/2699538628 conn(0x7fe6540d2d90 msgr2=0x7fe6540d31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:58.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.241+0000 7fe65a8bb640 1 -- 172.21.15.81:0/2699538628 shutdown_connections 2023-11-08T16:41:58.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.241+0000 7fe65a8bb640 1 -- 172.21.15.81:0/2699538628 wait complete. 2023-11-08T16:41:58.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.243+0000 7fe65a8bb640 1 Processor -- start 2023-11-08T16:41:58.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.243+0000 7fe65a8bb640 1 -- start start 2023-11-08T16:41:58.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.244+0000 7fe65a8bb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6541f6bb0 0x7fe6541f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.244+0000 7fe65a8bb640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6540b0e30 0x7fe6541e4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.244+0000 7fe65a8bb640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe65415a450 0x7fe65415a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.244+0000 7fe65a8bb640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6541e49b0 con 0x7fe6541f6bb0 2023-11-08T16:41:58.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.244+0000 7fe65a8bb640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65415ae10 con 0x7fe65415a450 2023-11-08T16:41:58.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.244+0000 7fe65a8bb640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6540afa40 con 0x7fe6540b0e30 2023-11-08T16:41:58.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.244+0000 7fe6598b9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6541f6bb0 0x7fe6541f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe6598b9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6541f6bb0 0x7fe6541f6140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:40756/0 (socket says 172.21.15.81:40756) 2023-11-08T16:41:58.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe6598b9640 1 -- 172.21.15.81:0/1070822736 learned_addr learned my addr 172.21.15.81:0/1070822736 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:41:58.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe6590b8640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6540b0e30 0x7fe6541e4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe65a0ba640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe65415a450 0x7fe65415a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe6598b9640 1 -- 172.21.15.81:0/1070822736 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6540b0e30 msgr2=0x7fe6541e4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:58.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe6598b9640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6540b0e30 0x7fe6541e4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe6598b9640 1 -- 172.21.15.81:0/1070822736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe65415a450 msgr2=0x7fe65415a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:58.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe6598b9640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe65415a450 0x7fe65415a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe6598b9640 1 -- 172.21.15.81:0/1070822736 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6540afc90 con 0x7fe6541f6bb0 2023-11-08T16:41:58.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe6590b8640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6540b0e30 0x7fe6541e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:58.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe65a0ba640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe65415a450 0x7fe65415a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:41:58.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.245+0000 7fe6598b9640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6541f6bb0 0x7fe6541f6140 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe6500108b0 tx=0x7fe650010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:58.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.246+0000 7fe64a7fc640 1 -- 172.21.15.81:0/1070822736 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65000eb60 con 0x7fe6541f6bb0 2023-11-08T16:41:58.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.246+0000 7fe64a7fc640 1 -- 172.21.15.81:0/1070822736 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe65000ecc0 con 0x7fe6541f6bb0 2023-11-08T16:41:58.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.246+0000 7fe64a7fc640 1 -- 172.21.15.81:0/1070822736 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65001c400 con 0x7fe6541f6bb0 2023-11-08T16:41:58.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.246+0000 7fe65a8bb640 1 -- 172.21.15.81:0/1070822736 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6540aff70 con 0x7fe6541f6bb0 2023-11-08T16:41:58.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.246+0000 7fe65a8bb640 1 -- 172.21.15.81:0/1070822736 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6540b0530 con 0x7fe6541f6bb0 2023-11-08T16:41:58.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.247+0000 7fe65a8bb640 1 -- 172.21.15.81:0/1070822736 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65404ece0 con 0x7fe6541f6bb0 2023-11-08T16:41:58.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.248+0000 7fe64a7fc640 1 -- 172.21.15.81:0/1070822736 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe65001c560 con 0x7fe6541f6bb0 2023-11-08T16:41:58.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.249+0000 7fe64a7fc640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe63806aea0 0x7fe63806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:41:58.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.249+0000 7fe64a7fc640 1 -- 172.21.15.81:0/1070822736 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fe650089cb0 con 0x7fe6541f6bb0 2023-11-08T16:41:58.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.249+0000 7fe6590b8640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe63806aea0 0x7fe63806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:41:58.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.250+0000 7fe6590b8640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe63806aea0 0x7fe63806d360 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe6440017a0 tx=0x7fe644000ed0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:41:58.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.253+0000 7fe64a7fc640 1 -- 172.21.15.81:0/1070822736 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe650054d20 con 0x7fe6541f6bb0 2023-11-08T16:41:58.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.479+0000 7fe65a8bb640 1 -- 172.21.15.81:0/1070822736 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe6540b12e0 con 0x7fe6541f6bb0 2023-11-08T16:41:58.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.480+0000 7fe64a7fc640 1 -- 172.21.15.81:0/1070822736 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v120) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe650058340 con 0x7fe6541f6bb0 2023-11-08T16:41:58.482 INFO:teuthology.orchestra.run.smithi081.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-08T16:41:58.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.482+0000 7fe65a8bb640 1 -- 172.21.15.81:0/1070822736 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe63806aea0 msgr2=0x7fe63806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:58.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.482+0000 7fe65a8bb640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe63806aea0 0x7fe63806d360 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fe6440017a0 tx=0x7fe644000ed0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.482+0000 7fe65a8bb640 1 -- 172.21.15.81:0/1070822736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6541f6bb0 msgr2=0x7fe6541f6140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:41:58.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.482+0000 7fe65a8bb640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6541f6bb0 0x7fe6541f6140 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe6500108b0 tx=0x7fe650010d70 comp rx=0 tx=0).stop 2023-11-08T16:41:58.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.483+0000 7fe65a8bb640 1 -- 172.21.15.81:0/1070822736 shutdown_connections 2023-11-08T16:41:58.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.483+0000 7fe65a8bb640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe63806aea0 0x7fe63806d360 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.483+0000 7fe65a8bb640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe65415a450 0x7fe65415a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.483+0000 7fe65a8bb640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe6540b0e30 0x7fe6541e4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.483+0000 7fe65a8bb640 1 --2- 172.21.15.81:0/1070822736 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe6541f6bb0 0x7fe6541f6140 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:41:58.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.483+0000 7fe65a8bb640 1 -- 172.21.15.81:0/1070822736 >> 172.21.15.81:0/1070822736 conn(0x7fe6540d2d90 msgr2=0x7fe65406b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:41:58.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.483+0000 7fe65a8bb640 1 -- 172.21.15.81:0/1070822736 shutdown_connections 2023-11-08T16:41:58.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:58.483+0000 7fe65a8bb640 1 -- 172.21.15.81:0/1070822736 wait complete. 2023-11-08T16:41:58.503 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-08T16:41:58.504 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-08T16:41:58.504 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi081.front.sepia.ceph.com 2023-11-08T16:41:58.505 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-08T16:41:58.505 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:58.505 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-08T16:41:58.506 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-08T16:41:58.506 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-08T16:41:58.506 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:41:58.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:58.507 DEBUG:teuthology.orchestra.run.smithi081:> ip addr 2023-11-08T16:41:58.562 INFO:teuthology.orchestra.run.smithi081.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-08T16:41:58.563 INFO:teuthology.orchestra.run.smithi081.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-08T16:41:58.563 INFO:teuthology.orchestra.run.smithi081.stdout: inet 127.0.0.1/8 scope host lo 2023-11-08T16:41:58.563 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:58.563 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 ::1/128 scope host 2023-11-08T16:41:58.564 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:58.564 INFO:teuthology.orchestra.run.smithi081.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:41:58.564 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:30 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:58.564 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f0 2023-11-08T16:41:58.564 INFO:teuthology.orchestra.run.smithi081.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:41:58.565 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:d8:a1:31 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:58.565 INFO:teuthology.orchestra.run.smithi081.stdout: altname enp7s0f1 2023-11-08T16:41:58.565 INFO:teuthology.orchestra.run.smithi081.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-08T16:41:58.565 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:52 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:58.565 INFO:teuthology.orchestra.run.smithi081.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-08T16:41:58.566 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 0c:c4:7a:8f:d7:53 brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:58.566 INFO:teuthology.orchestra.run.smithi081.stdout: inet 172.21.15.81/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-08T16:41:58.566 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft 41591sec preferred_lft 41591sec 2023-11-08T16:41:58.566 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::ec4:7aff:fe8f:d753/64 scope link 2023-11-08T16:41:58.566 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:58.567 INFO:teuthology.orchestra.run.smithi081.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-08T16:41:58.567 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff 2023-11-08T16:41:58.567 INFO:teuthology.orchestra.run.smithi081.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-08T16:41:58.567 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:58.567 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::80aa:8eff:fe0f:5475/64 scope link 2023-11-08T16:41:58.568 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:58.568 INFO:teuthology.orchestra.run.smithi081.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:41:58.568 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 8e:6c:a7:b8:3e:3c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:58.568 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::8c6c:a7ff:feb8:3e3c/64 scope link 2023-11-08T16:41:58.568 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:58.569 INFO:teuthology.orchestra.run.smithi081.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-08T16:41:58.569 INFO:teuthology.orchestra.run.smithi081.stdout: link/ether 2e:ac:17:64:67:da brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:58.569 INFO:teuthology.orchestra.run.smithi081.stdout: inet6 fe80::2cac:17ff:fe64:67da/64 scope link 2023-11-08T16:41:58.569 INFO:teuthology.orchestra.run.smithi081.stdout: valid_lft forever preferred_lft forever 2023-11-08T16:41:58.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:58.570 DEBUG:teuthology.orchestra.run.smithi081:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-08T16:41:58.658 INFO:teuthology.orchestra.run.smithi081.stdout:1 2023-11-08T16:41:58.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:58.662 DEBUG:teuthology.orchestra.run.smithi081:> route 2023-11-08T16:41:58.717 INFO:teuthology.orchestra.run.smithi081.stdout:Kernel IP routing table 2023-11-08T16:41:58.718 INFO:teuthology.orchestra.run.smithi081.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-08T16:41:58.718 INFO:teuthology.orchestra.run.smithi081.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-08T16:41:58.718 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-08T16:41:58.718 INFO:teuthology.orchestra.run.smithi081.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-08T16:41:58.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:58.719 DEBUG:teuthology.orchestra.run.smithi081:> (cd / && exec bash -c ' 2023-11-08T16:41:58.720 DEBUG:teuthology.orchestra.run.smithi081:> set -e 2023-11-08T16:41:58.720 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-08T16:41:58.720 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-08T16:41:58.721 DEBUG:teuthology.orchestra.run.smithi081:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-08T16:41:58.721 DEBUG:teuthology.orchestra.run.smithi081:> ') 2023-11-08T16:41:58.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:58.903 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:41:58.954 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:41:58.955 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:41:58.956 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:41:58.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:58.957 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link set brx.0 up 2023-11-08T16:41:59.049 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-08T16:41:59.049 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:41:59.050 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:59.105 INFO:teuthology.orchestra.run.smithi081.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-08T16:41:59.106 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-08T16:41:59.106 DEBUG:teuthology.orchestra.run.smithi081:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:41:59.156 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:41:59.249 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:41:59.305 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:41:59.306 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:41:59.306 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:41:59.306 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:41:59.306 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:41:59.307 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:41:59.307 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:41:59.307 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:41:59.307 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:41:59.307 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:41:59.308 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:41:59.308 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:41:59.308 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:41:59.308 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:41:59.308 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:41:59.309 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:41:59.309 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:41:59.309 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:41:59.309 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:41:59.309 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:41:59.310 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:41:59.310 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:41:59.310 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:41:59.310 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:41:59.310 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:41:59.311 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:41:59.311 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:41:59.311 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:41:59.311 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:41:59.311 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:59.312 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:59.312 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:59.312 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:59.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:59.314 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:41:59.369 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-08T16:41:59.370 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-11-08T16:41:59.376 DEBUG:teuthology.orchestra.run.smithi081:> sudo modprobe fuse 2023-11-08T16:41:59.456 DEBUG:teuthology.orchestra.run.smithi081:> cat /proc/self/mounts | awk '{print $2}' 2023-11-08T16:41:59.500 INFO:teuthology.orchestra.run.smithi081.stdout:/proc 2023-11-08T16:41:59.500 INFO:teuthology.orchestra.run.smithi081.stdout:/sys 2023-11-08T16:41:59.501 INFO:teuthology.orchestra.run.smithi081.stdout:/dev 2023-11-08T16:41:59.501 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/security 2023-11-08T16:41:59.501 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/shm 2023-11-08T16:41:59.501 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/pts 2023-11-08T16:41:59.501 INFO:teuthology.orchestra.run.smithi081.stdout:/run 2023-11-08T16:41:59.502 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/cgroup 2023-11-08T16:41:59.502 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/pstore 2023-11-08T16:41:59.502 INFO:teuthology.orchestra.run.smithi081.stdout:/ 2023-11-08T16:41:59.502 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/selinux 2023-11-08T16:41:59.502 INFO:teuthology.orchestra.run.smithi081.stdout:/proc/sys/fs/binfmt_misc 2023-11-08T16:41:59.503 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/hugepages 2023-11-08T16:41:59.503 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/mqueue 2023-11-08T16:41:59.503 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/debug 2023-11-08T16:41:59.503 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/tracing 2023-11-08T16:41:59.503 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2023-11-08T16:41:59.504 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/fs/fuse/connections 2023-11-08T16:41:59.504 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-sysctl.service 2023-11-08T16:41:59.504 INFO:teuthology.orchestra.run.smithi081.stdout:/sys/kernel/config 2023-11-08T16:41:59.504 INFO:teuthology.orchestra.run.smithi081.stdout:/run/credentials/systemd-tmpfiles-setup.service 2023-11-08T16:41:59.504 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/nfs/rpc_pipefs 2023-11-08T16:41:59.505 INFO:teuthology.orchestra.run.smithi081.stdout:/run/user/1000 2023-11-08T16:41:59.505 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph 2023-11-08T16:41:59.505 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-0 2023-11-08T16:41:59.505 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-1 2023-11-08T16:41:59.505 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-2 2023-11-08T16:41:59.505 INFO:teuthology.orchestra.run.smithi081.stdout:/var/lib/ceph/osd/ceph-3 2023-11-08T16:41:59.506 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns 2023-11-08T16:41:59.506 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:59.506 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:41:59.506 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:59.506 INFO:teuthology.orchestra.run.smithi081.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:41:59.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:59.508 DEBUG:teuthology.orchestra.run.smithi081:> ls /sys/fs/fuse/connections 2023-11-08T16:41:59.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:2023-11-08T16:41:59.566+0000 7ff9cf3f6480 -1 init, newargv = 0x56363c9b97c0 newargc=15 2023-11-08T16:41:59.568 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[127959]: starting ceph client 2023-11-08T16:41:59.575 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[127959]: starting fuse 2023-11-08T16:41:59.594 INFO:teuthology.orchestra.run.smithi081.stdout:40 2023-11-08T16:41:59.595 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-11-08T16:41:59.595 DEBUG:teuthology.orchestra.run.smithi081:> sudo stdin-killer -- python3 -c ' 2023-11-08T16:41:59.596 DEBUG:teuthology.orchestra.run.smithi081:> import glob 2023-11-08T16:41:59.596 DEBUG:teuthology.orchestra.run.smithi081:> import re 2023-11-08T16:41:59.597 DEBUG:teuthology.orchestra.run.smithi081:> import os 2023-11-08T16:41:59.597 DEBUG:teuthology.orchestra.run.smithi081:> import subprocess 2023-11-08T16:41:59.597 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:59.598 DEBUG:teuthology.orchestra.run.smithi081:> def _find_admin_socket(client_name): 2023-11-08T16:41:59.598 DEBUG:teuthology.orchestra.run.smithi081:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-08T16:41:59.598 DEBUG:teuthology.orchestra.run.smithi081:> files = glob.glob(asok_path) 2023-11-08T16:41:59.599 DEBUG:teuthology.orchestra.run.smithi081:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-08T16:41:59.599 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:59.600 DEBUG:teuthology.orchestra.run.smithi081:> # Given a non-glob path, it better be there 2023-11-08T16:41:59.600 DEBUG:teuthology.orchestra.run.smithi081:> if "*" not in asok_path: 2023-11-08T16:41:59.600 DEBUG:teuthology.orchestra.run.smithi081:> assert(len(files) == 1) 2023-11-08T16:41:59.601 DEBUG:teuthology.orchestra.run.smithi081:> return files[0] 2023-11-08T16:41:59.601 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:59.601 DEBUG:teuthology.orchestra.run.smithi081:> for f in files: 2023-11-08T16:41:59.602 DEBUG:teuthology.orchestra.run.smithi081:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-08T16:41:59.602 DEBUG:teuthology.orchestra.run.smithi081:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-08T16:41:59.603 DEBUG:teuthology.orchestra.run.smithi081:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-08T16:41:59.603 DEBUG:teuthology.orchestra.run.smithi081:> contents = proc_f.read() 2023-11-08T16:41:59.603 DEBUG:teuthology.orchestra.run.smithi081:> if mountpoint in contents: 2023-11-08T16:41:59.604 DEBUG:teuthology.orchestra.run.smithi081:> return f 2023-11-08T16:41:59.604 DEBUG:teuthology.orchestra.run.smithi081:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-08T16:41:59.604 DEBUG:teuthology.orchestra.run.smithi081:> 2023-11-08T16:41:59.605 DEBUG:teuthology.orchestra.run.smithi081:> print(_find_admin_socket("client.0")) 2023-11-08T16:41:59.605 DEBUG:teuthology.orchestra.run.smithi081:> ' 2023-11-08T16:41:59.799 INFO:teuthology.orchestra.run.smithi081.stdout:/var/run/ceph/ceph-client.0.127959.asok 2023-11-08T16:41:59.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:41:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-11-08T16:41:59.816 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.127959.asok 2023-11-08T16:41:59.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:41:59.817 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.127959.asok status 2023-11-08T16:42:00.057 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:42:00.057 INFO:teuthology.orchestra.run.smithi081.stdout: "metadata": { 2023-11-08T16:42:00.057 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_sha1": "85bdb240f5cf2a758acc9b247951fbc8f5799bc1", 2023-11-08T16:42:00.058 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 18.0.0-7083-g85bdb240 (85bdb240f5cf2a758acc9b247951fbc8f5799bc1) reef (dev)", 2023-11-08T16:42:00.058 INFO:teuthology.orchestra.run.smithi081.stdout: "entity_id": "0", 2023-11-08T16:42:00.058 INFO:teuthology.orchestra.run.smithi081.stdout: "hostname": "smithi081", 2023-11-08T16:42:00.058 INFO:teuthology.orchestra.run.smithi081.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-08T16:42:00.058 INFO:teuthology.orchestra.run.smithi081.stdout: "pid": "127959", 2023-11-08T16:42:00.059 INFO:teuthology.orchestra.run.smithi081.stdout: "root": "/" 2023-11-08T16:42:00.059 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:42:00.059 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_count": 0, 2023-11-08T16:42:00.059 INFO:teuthology.orchestra.run.smithi081.stdout: "dentry_pinned_count": 0, 2023-11-08T16:42:00.059 INFO:teuthology.orchestra.run.smithi081.stdout: "id": 9084, 2023-11-08T16:42:00.060 INFO:teuthology.orchestra.run.smithi081.stdout: "inst": { 2023-11-08T16:42:00.060 INFO:teuthology.orchestra.run.smithi081.stdout: "name": { 2023-11-08T16:42:00.060 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "client", 2023-11-08T16:42:00.060 INFO:teuthology.orchestra.run.smithi081.stdout: "num": 9084 2023-11-08T16:42:00.060 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:42:00.061 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:42:00.061 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:42:00.061 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:42:00.061 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1228083380 2023-11-08T16:42:00.061 INFO:teuthology.orchestra.run.smithi081.stdout: } 2023-11-08T16:42:00.062 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:42:00.062 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": { 2023-11-08T16:42:00.062 INFO:teuthology.orchestra.run.smithi081.stdout: "type": "v1", 2023-11-08T16:42:00.062 INFO:teuthology.orchestra.run.smithi081.stdout: "addr": "192.168.0.1:0", 2023-11-08T16:42:00.062 INFO:teuthology.orchestra.run.smithi081.stdout: "nonce": 1228083380 2023-11-08T16:42:00.063 INFO:teuthology.orchestra.run.smithi081.stdout: }, 2023-11-08T16:42:00.063 INFO:teuthology.orchestra.run.smithi081.stdout: "inst_str": "client.9084 192.168.0.1:0/1228083380", 2023-11-08T16:42:00.063 INFO:teuthology.orchestra.run.smithi081.stdout: "addr_str": "192.168.0.1:0/1228083380", 2023-11-08T16:42:00.063 INFO:teuthology.orchestra.run.smithi081.stdout: "inode_count": 1, 2023-11-08T16:42:00.063 INFO:teuthology.orchestra.run.smithi081.stdout: "mds_epoch": 120, 2023-11-08T16:42:00.064 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch": 131, 2023-11-08T16:42:00.064 INFO:teuthology.orchestra.run.smithi081.stdout: "osd_epoch_barrier": 0, 2023-11-08T16:42:00.064 INFO:teuthology.orchestra.run.smithi081.stdout: "blocklisted": false, 2023-11-08T16:42:00.064 INFO:teuthology.orchestra.run.smithi081.stdout: "fs_name": "cephfs" 2023-11-08T16:42:00.064 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:42:00.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:00.072 DEBUG:teuthology.orchestra.run.smithi081:> stat --file-system '--printf=%T 2023-11-08T16:42:00.072 DEBUG:teuthology.orchestra.run.smithi081:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-08T16:42:00.125 INFO:teuthology.orchestra.run.smithi081.stdout:fuseblk 2023-11-08T16:42:00.126 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-08T16:42:00.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:00.127 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-08T16:42:00.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:00.221 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls -l subdir/) 2023-11-08T16:42:00.554 INFO:teuthology.orchestra.run.smithi081.stdout:total 50 2023-11-08T16:42:00.555 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Nov 8 16:41 0 2023-11-08T16:42:00.555 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Nov 8 16:41 1 2023-11-08T16:42:00.555 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 10 2023-11-08T16:42:00.556 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 11 2023-11-08T16:42:00.556 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 12 2023-11-08T16:42:00.556 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 13 2023-11-08T16:42:00.556 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 14 2023-11-08T16:42:00.556 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 15 2023-11-08T16:42:00.557 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 16 2023-11-08T16:42:00.557 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 17 2023-11-08T16:42:00.557 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 18 2023-11-08T16:42:00.557 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 19 2023-11-08T16:42:00.557 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Nov 8 16:41 2 2023-11-08T16:42:00.557 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 20 2023-11-08T16:42:00.558 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 22 2023-11-08T16:42:00.558 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 23 2023-11-08T16:42:00.558 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 24 2023-11-08T16:42:00.558 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 25 2023-11-08T16:42:00.559 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 26 2023-11-08T16:42:00.559 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 27 2023-11-08T16:42:00.559 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 28 2023-11-08T16:42:00.559 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 29 2023-11-08T16:42:00.559 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Nov 8 16:41 3 2023-11-08T16:42:00.560 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 30 2023-11-08T16:42:00.560 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 31 2023-11-08T16:42:00.560 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 32 2023-11-08T16:42:00.560 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 33 2023-11-08T16:42:00.561 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 34 2023-11-08T16:42:00.561 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 35 2023-11-08T16:42:00.561 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 36 2023-11-08T16:42:00.561 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 37 2023-11-08T16:42:00.561 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 38 2023-11-08T16:42:00.561 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 39 2023-11-08T16:42:00.562 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Nov 8 16:41 4 2023-11-08T16:42:00.562 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 40 2023-11-08T16:42:00.562 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 41 2023-11-08T16:42:00.563 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 42 2023-11-08T16:42:00.563 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 43 2023-11-08T16:42:00.563 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 44 2023-11-08T16:42:00.564 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 45 2023-11-08T16:42:00.564 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 46 2023-11-08T16:42:00.564 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 47 2023-11-08T16:42:00.564 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 48 2023-11-08T16:42:00.564 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 49 2023-11-08T16:42:00.565 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Nov 8 16:41 5 2023-11-08T16:42:00.565 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 50 2023-11-08T16:42:00.565 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 51 2023-11-08T16:42:00.565 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 52 2023-11-08T16:42:00.566 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 53 2023-11-08T16:42:00.566 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 54 2023-11-08T16:42:00.566 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 55 2023-11-08T16:42:00.566 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 56 2023-11-08T16:42:00.566 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 57 2023-11-08T16:42:00.567 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 58 2023-11-08T16:42:00.567 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 59 2023-11-08T16:42:00.567 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Nov 8 16:41 6 2023-11-08T16:42:00.567 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 60 2023-11-08T16:42:00.568 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 61 2023-11-08T16:42:00.568 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 62 2023-11-08T16:42:00.568 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 63 2023-11-08T16:42:00.568 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 64 2023-11-08T16:42:00.568 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 65 2023-11-08T16:42:00.569 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 66 2023-11-08T16:42:00.569 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 67 2023-11-08T16:42:00.569 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 68 2023-11-08T16:42:00.569 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 69 2023-11-08T16:42:00.569 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Nov 8 16:41 7 2023-11-08T16:42:00.569 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 70 2023-11-08T16:42:00.570 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 71 2023-11-08T16:42:00.570 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 72 2023-11-08T16:42:00.570 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 73 2023-11-08T16:42:00.570 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 74 2023-11-08T16:42:00.570 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 75 2023-11-08T16:42:00.571 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 76 2023-11-08T16:42:00.571 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 77 2023-11-08T16:42:00.571 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 78 2023-11-08T16:42:00.571 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 79 2023-11-08T16:42:00.571 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Nov 8 16:41 8 2023-11-08T16:42:00.572 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 80 2023-11-08T16:42:00.572 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 81 2023-11-08T16:42:00.572 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 82 2023-11-08T16:42:00.572 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 83 2023-11-08T16:42:00.573 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 84 2023-11-08T16:42:00.574 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 85 2023-11-08T16:42:00.574 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 86 2023-11-08T16:42:00.575 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 87 2023-11-08T16:42:00.575 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 88 2023-11-08T16:42:00.575 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 89 2023-11-08T16:42:00.575 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 1 Nov 8 16:41 9 2023-11-08T16:42:00.575 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 90 2023-11-08T16:42:00.576 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 91 2023-11-08T16:42:00.576 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 92 2023-11-08T16:42:00.576 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 93 2023-11-08T16:42:00.576 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 94 2023-11-08T16:42:00.576 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 95 2023-11-08T16:42:00.577 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 96 2023-11-08T16:42:00.577 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 97 2023-11-08T16:42:00.577 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 98 2023-11-08T16:42:00.577 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 2 Nov 8 16:41 99 2023-11-08T16:42:00.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:00.579 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo cat subdir/21') 2023-11-08T16:42:00.680 INFO:teuthology.orchestra.run.smithi081.stderr:cat: subdir/21: No such file or directory 2023-11-08T16:42:00.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:42:00.685 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-08T16:42:00.686 DEBUG:teuthology.orchestra.run.smithi081:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:42:00.798 DEBUG:teuthology.orchestra.run.smithi093:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-08T16:42:00.926 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-08T16:42:00.927 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:42:00.927 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:42:00.986 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:42:01.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.200+0000 7f0d3b01a640 1 Processor -- start 2023-11-08T16:42:01.201 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.200+0000 7f0d3b01a640 1 -- start start 2023-11-08T16:42:01.202 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.201+0000 7f0d3b01a640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f0d340c8fa0 0x7f0d3415d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.201+0000 7f0d3b01a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3415d050 con 0x7f0d3415dce0 2023-11-08T16:42:01.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.201+0000 7f0d3b01a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3415a9e0 con 0x7f0d34159ef0 2023-11-08T16:42:01.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.201+0000 7f0d3b01a640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3415d380 con 0x7f0d340c8fa0 2023-11-08T16:42:01.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.201+0000 7f0d3a819640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f0d3415dce0 0x7f0d3406b9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45112/0 (socket says 172.21.15.81:45112) 2023-11-08T16:42:01.203 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.201+0000 7f0d3a819640 1 -- 172.21.15.81:0/1071285813 learned_addr learned my addr 172.21.15.81:0/1071285813 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:01.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39817640 1 --2- 172.21.15.81:0/1071285813 >> v2:172.21.15.93:3300/0 conn(0x7f0d340c8fa0 0x7f0d3415d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1935088346 0 0) 0x7f0d3415d050 con 0x7f0d3415dce0 2023-11-08T16:42:01.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d20003520 con 0x7f0d3415dce0 2023-11-08T16:42:01.206 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2204098375 0 0) 0x7f0d20003520 con 0x7f0d3415dce0 2023-11-08T16:42:01.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 >> v2:172.21.15.93:3300/0 conn(0x7f0d340c8fa0 msgr2=0x7f0d3415d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:01.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 --2- 172.21.15.81:0/1071285813 >> v2:172.21.15.93:3300/0 conn(0x7f0d340c8fa0 0x7f0d3415d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 >> v1:172.21.15.93:6789/0 conn(0x7f0d34159ef0 legacy=0x7f0d3415a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:01.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340ccae0 con 0x7f0d3415dce0 2023-11-08T16:42:01.207 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f0d2c0019f0 con 0x7f0d3415dce0 2023-11-08T16:42:01.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 >> v1:172.21.15.81:6789/0 conn(0x7f0d3415dce0 legacy=0x7f0d3406b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:01.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d20003970 0x7f0d20003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d200047c0 0x7f0d20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 0x7f0d2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.208 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3415d380 con 0x7f0d20005180 2023-11-08T16:42:01.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2000c000 con 0x7f0d200047c0 2023-11-08T16:42:01.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.202+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d2000c1c0 con 0x7f0d20003970 2023-11-08T16:42:01.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.203+0000 7f0d3a819640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 0x7f0d2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.203+0000 7f0d39817640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d200047c0 0x7f0d20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.209 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.203+0000 7f0d3a018640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d20003970 0x7f0d20003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.203+0000 7f0d3a819640 1 -- 172.21.15.81:0/1071285813 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d20003970 msgr2=0x7f0d20003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.203+0000 7f0d3a819640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d20003970 0x7f0d20003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.203+0000 7f0d3a819640 1 -- 172.21.15.81:0/1071285813 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d200047c0 msgr2=0x7f0d20004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:01.210 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.203+0000 7f0d3a819640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d200047c0 0x7f0d20004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.203+0000 7f0d3a819640 1 -- 172.21.15.81:0/1071285813 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20003520 con 0x7f0d20005180 2023-11-08T16:42:01.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.203+0000 7f0d39817640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d200047c0 0x7f0d20004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:01.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.204+0000 7f0d3a819640 1 -- 172.21.15.81:0/1071285813 reap_dead start 2023-11-08T16:42:01.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.204+0000 7f0d3a819640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 0x7f0d2000b980 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f0d340c8f50 tx=0x7f0d200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95d861b30ac45e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:01.211 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.204+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c034630 con 0x7f0d20005180 2023-11-08T16:42:01.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.204+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d2c02f070 con 0x7f0d20005180 2023-11-08T16:42:01.212 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.204+0000 7f0d39016640 1 -- 172.21.15.81:0/1071285813 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d2c0349d0 con 0x7f0d20005180 2023-11-08T16:42:01.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.205+0000 7f0d3b01a640 1 -- 172.21.15.81:0/1071285813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 msgr2=0x7f0d2000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:01.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.205+0000 7f0d3b01a640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 0x7f0d2000b980 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f0d340c8f50 tx=0x7f0d200057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.205+0000 7f0d3b01a640 1 -- 172.21.15.81:0/1071285813 shutdown_connections 2023-11-08T16:42:01.216 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.205+0000 7f0d3b01a640 1 --2- 172.21.15.81:0/1071285813 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 0x7f0d2000b980 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.205+0000 7f0d3b01a640 1 -- 172.21.15.81:0/1071285813 >> 172.21.15.81:0/1071285813 conn(0x7f0d340b06e0 msgr2=0x7f0d340b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:01.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.205+0000 7f0d3b01a640 1 -- 172.21.15.81:0/1071285813 shutdown_connections 2023-11-08T16:42:01.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.205+0000 7f0d3b01a640 1 -- 172.21.15.81:0/1071285813 wait complete. 2023-11-08T16:42:01.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.207+0000 7f0d3b01a640 1 Processor -- start 2023-11-08T16:42:01.217 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.207+0000 7f0d3b01a640 1 -- start start 2023-11-08T16:42:01.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.208+0000 7f0d3b01a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 0x7f0d340c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.208+0000 7f0d3b01a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d340af280 0x7f0d340af700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.208+0000 7f0d3b01a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d340c63f0 0x7f0d340c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.208+0000 7f0d3b01a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340afc40 con 0x7f0d20005180 2023-11-08T16:42:01.218 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.208+0000 7f0d3b01a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340c7120 con 0x7f0d340af280 2023-11-08T16:42:01.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.208+0000 7f0d3b01a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340c7400 con 0x7f0d340c63f0 2023-11-08T16:42:01.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.208+0000 7f0d3a018640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 0x7f0d340c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.208+0000 7f0d3a018640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 0x7f0d340c4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58536/0 (socket says 172.21.15.81:58536) 2023-11-08T16:42:01.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.208+0000 7f0d3a018640 1 -- 172.21.15.81:0/3496140219 learned_addr learned my addr 172.21.15.81:0/3496140219 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:01.219 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.209+0000 7f0d39817640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d340af280 0x7f0d340af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.209+0000 7f0d3a018640 1 -- 172.21.15.81:0/3496140219 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d340c63f0 msgr2=0x7f0d340c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:01.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.209+0000 7f0d3a018640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d340c63f0 0x7f0d340c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.209+0000 7f0d3a018640 1 -- 172.21.15.81:0/3496140219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d340af280 msgr2=0x7f0d340af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:01.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.209+0000 7f0d3a018640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d340af280 0x7f0d340af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.220 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.209+0000 7f0d3a018640 1 -- 172.21.15.81:0/3496140219 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340c76e0 con 0x7f0d20005180 2023-11-08T16:42:01.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.209+0000 7f0d39817640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d340af280 0x7f0d340af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:01.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.209+0000 7f0d3a018640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 0x7f0d340c4340 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000e460 tx=0x7f0d3000e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:01.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.209+0000 7f0d2affd640 1 -- 172.21.15.81:0/3496140219 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d300135f0 con 0x7f0d20005180 2023-11-08T16:42:01.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.209+0000 7f0d2affd640 1 -- 172.21.15.81:0/3496140219 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d30013750 con 0x7f0d20005180 2023-11-08T16:42:01.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.210+0000 7f0d2affd640 1 -- 172.21.15.81:0/3496140219 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d30013b00 con 0x7f0d20005180 2023-11-08T16:42:01.221 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.210+0000 7f0d3b01a640 1 -- 172.21.15.81:0/3496140219 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d340c79c0 con 0x7f0d20005180 2023-11-08T16:42:01.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.210+0000 7f0d3b01a640 1 -- 172.21.15.81:0/3496140219 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d340c4880 con 0x7f0d20005180 2023-11-08T16:42:01.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.211+0000 7f0d3b01a640 1 -- 172.21.15.81:0/3496140219 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d3404ece0 con 0x7f0d20005180 2023-11-08T16:42:01.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.212+0000 7f0d2affd640 1 -- 172.21.15.81:0/3496140219 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0d30013c60 con 0x7f0d20005180 2023-11-08T16:42:01.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.213+0000 7f0d2affd640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0d1806aea0 0x7f0d1806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.222 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.213+0000 7f0d2affd640 1 -- 172.21.15.81:0/3496140219 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f0d30089550 con 0x7f0d20005180 2023-11-08T16:42:01.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.213+0000 7f0d39817640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0d1806aea0 0x7f0d1806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.214+0000 7f0d39817640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0d1806aea0 0x7f0d1806d360 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0d341f5c30 tx=0x7f0d24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:01.223 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.217+0000 7f0d2affd640 1 -- 172.21.15.81:0/3496140219 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0d30054470 con 0x7f0d20005180 2023-11-08T16:42:01.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.411+0000 7f0d3b01a640 1 -- 172.21.15.81:0/3496140219 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d340ce2c0 con 0x7f0d20005180 2023-11-08T16:42:01.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.414+0000 7f0d2affd640 1 -- 172.21.15.81:0/3496140219 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16878 (secure 0 0 0) 0x7f0d30057a90 con 0x7f0d20005180 2023-11-08T16:42:01.418 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:01.418 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":131,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:49.127728+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:41:03.545565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T16:41:04.726492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":2}]},{"pgid":"17.24","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/296230272":"2023-11-09T16:41:49.127674+0000","172.21.15.81:6836/296230272":"2023-11-09T16:41:49.127674+0000","172.21.15.93:6835/4021510722":"2023-11-09T16:41:29.470203+0000","172.21.15.93:6833/4021510722":"2023-11-09T16:41:29.470203+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:42:01.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 -- 172.21.15.81:0/3496140219 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0d1806aea0 msgr2=0x7f0d1806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:01.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0d1806aea0 0x7f0d1806d360 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f0d341f5c30 tx=0x7f0d24008040 comp rx=0 tx=0).stop 2023-11-08T16:42:01.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 -- 172.21.15.81:0/3496140219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 msgr2=0x7f0d340c4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:01.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 0x7f0d340c4340 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000e460 tx=0x7f0d3000e920 comp rx=0 tx=0).stop 2023-11-08T16:42:01.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 -- 172.21.15.81:0/3496140219 shutdown_connections 2023-11-08T16:42:01.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0d1806aea0 0x7f0d1806d360 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0d340c63f0 0x7f0d340c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0d340af280 0x7f0d340af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 --2- 172.21.15.81:0/3496140219 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0d20005180 0x7f0d340c4340 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 -- 172.21.15.81:0/3496140219 >> 172.21.15.81:0/3496140219 conn(0x7f0d340b06e0 msgr2=0x7f0d340ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:01.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 -- 172.21.15.81:0/3496140219 shutdown_connections 2023-11-08T16:42:01.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.417+0000 7f0d3b01a640 1 -- 172.21.15.81:0/3496140219 wait complete. 2023-11-08T16:42:01.438 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:41:03.545565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:41:04.726492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:42:01.439 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:42:01.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.654+0000 7fb2ac941640 1 Processor -- start 2023-11-08T16:42:01.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.654+0000 7fb2ac941640 1 -- start start 2023-11-08T16:42:01.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2ac941640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb2a8158c60 0x7fb2a80705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2ac941640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb2a8071ad0 0x7fb2a815ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2ac941640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a815f5e0 con 0x7fb2a815f1b0 2023-11-08T16:42:01.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2ac941640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a806df80 con 0x7fb2a8158c60 2023-11-08T16:42:01.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2ac941640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a815fad0 con 0x7fb2a8071ad0 2023-11-08T16:42:01.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2a659c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb2a8158c60 0x7fb2a80705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2a659c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb2a8158c60 0x7fb2a80705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58542/0 (socket says 172.21.15.81:58542) 2023-11-08T16:42:01.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2a659c640 1 -- 172.21.15.81:0/2873964270 learned_addr learned my addr 172.21.15.81:0/2873964270 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:01.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2a659c640 1 -- 172.21.15.81:0/2873964270 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb2a8071ad0 msgr2=0x7fb2a815ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:01.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2a659c640 1 --2- 172.21.15.81:0/2873964270 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb2a8071ad0 0x7fb2a815ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2a659c640 1 -- 172.21.15.81:0/2873964270 >> v1:172.21.15.93:6789/0 conn(0x7fb2a815f1b0 legacy=0x7fb2a80c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:01.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.656+0000 7fb2a659c640 1 -- 172.21.15.81:0/2873964270 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a806d760 con 0x7fb2a8158c60 2023-11-08T16:42:01.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.658+0000 7fb2a659c640 1 --2- 172.21.15.81:0/2873964270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a8158c60 0x7fb2a80705d0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80cfc70 tx=0x7fb29802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b45ac326955270 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:01.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.658+0000 7fb2a559a640 1 -- 172.21.15.81:0/2873964270 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298031640 con 0x7fb2a8158c60 2023-11-08T16:42:01.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.658+0000 7fb2a559a640 1 -- 172.21.15.81:0/2873964270 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2980317a0 con 0x7fb2a8158c60 2023-11-08T16:42:01.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.658+0000 7fb2a559a640 1 -- 172.21.15.81:0/2873964270 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298031b30 con 0x7fb2a8158c60 2023-11-08T16:42:01.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.659+0000 7fb2ac941640 1 -- 172.21.15.81:0/2873964270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a8158c60 msgr2=0x7fb2a80705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:01.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.659+0000 7fb2ac941640 1 --2- 172.21.15.81:0/2873964270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a8158c60 0x7fb2a80705d0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fb2a80cfc70 tx=0x7fb29802b8f0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.659+0000 7fb2ac941640 1 -- 172.21.15.81:0/2873964270 shutdown_connections 2023-11-08T16:42:01.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.659+0000 7fb2ac941640 1 --2- 172.21.15.81:0/2873964270 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb2a8071ad0 0x7fb2a815ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.659+0000 7fb2ac941640 1 --2- 172.21.15.81:0/2873964270 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a8158c60 0x7fb2a80705d0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.659+0000 7fb2ac941640 1 -- 172.21.15.81:0/2873964270 >> 172.21.15.81:0/2873964270 conn(0x7fb2a80ceb70 msgr2=0x7fb2a80cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:01.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.659+0000 7fb2ac941640 1 -- 172.21.15.81:0/2873964270 shutdown_connections 2023-11-08T16:42:01.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.659+0000 7fb2ac941640 1 -- 172.21.15.81:0/2873964270 wait complete. 2023-11-08T16:42:01.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.660+0000 7fb2ac941640 1 Processor -- start 2023-11-08T16:42:01.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.661+0000 7fb2ac941640 1 -- start start 2023-11-08T16:42:01.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2ac941640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a8071ad0 0x7fb2a814da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2ac941640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb2a8158c60 0x7fb2a81551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2ac941640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2a815f1b0 0x7fb2a815d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2ac941640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a8159df0 con 0x7fb2a8071ad0 2023-11-08T16:42:01.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2ac941640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a8159170 con 0x7fb2a815f1b0 2023-11-08T16:42:01.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2ac941640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a8159450 con 0x7fb2a8158c60 2023-11-08T16:42:01.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2a659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a8071ad0 0x7fb2a814da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2a659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a8071ad0 0x7fb2a814da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58544/0 (socket says 172.21.15.81:58544) 2023-11-08T16:42:01.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2a659c640 1 -- 172.21.15.81:0/3648902906 learned_addr learned my addr 172.21.15.81:0/3648902906 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:01.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2a5d9b640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb2a8158c60 0x7fb2a81551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2a6d9d640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2a815f1b0 0x7fb2a815d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2a659c640 1 -- 172.21.15.81:0/3648902906 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb2a8158c60 msgr2=0x7fb2a81551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:01.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2a659c640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb2a8158c60 0x7fb2a81551b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2a659c640 1 -- 172.21.15.81:0/3648902906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2a815f1b0 msgr2=0x7fb2a815d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:01.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2a659c640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2a815f1b0 0x7fb2a815d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.662+0000 7fb2a659c640 1 -- 172.21.15.81:0/3648902906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2980094d0 con 0x7fb2a8071ad0 2023-11-08T16:42:01.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.663+0000 7fb2a659c640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a8071ad0 0x7fb2a814da90 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb298000c00 tx=0x7fb2980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:01.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.663+0000 7fb296ffd640 1 -- 172.21.15.81:0/3648902906 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29803ac60 con 0x7fb2a8071ad0 2023-11-08T16:42:01.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.663+0000 7fb296ffd640 1 -- 172.21.15.81:0/3648902906 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29803ade0 con 0x7fb2a8071ad0 2023-11-08T16:42:01.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.663+0000 7fb296ffd640 1 -- 172.21.15.81:0/3648902906 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298044630 con 0x7fb2a8071ad0 2023-11-08T16:42:01.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.663+0000 7fb2ac941640 1 -- 172.21.15.81:0/3648902906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a8159730 con 0x7fb2a8071ad0 2023-11-08T16:42:01.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.663+0000 7fb2ac941640 1 -- 172.21.15.81:0/3648902906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a81556f0 con 0x7fb2a8071ad0 2023-11-08T16:42:01.670 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.664+0000 7fb2ac941640 1 -- 172.21.15.81:0/3648902906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a8074440 con 0x7fb2a8071ad0 2023-11-08T16:42:01.671 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.665+0000 7fb296ffd640 1 -- 172.21.15.81:0/3648902906 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb29802f050 con 0x7fb2a8071ad0 2023-11-08T16:42:01.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.666+0000 7fb296ffd640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb28806ae70 0x7fb28806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:01.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.667+0000 7fb296ffd640 1 -- 172.21.15.81:0/3648902906 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fb2980ab6e0 con 0x7fb2a8071ad0 2023-11-08T16:42:01.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.667+0000 7fb2a5d9b640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb28806ae70 0x7fb28806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:01.672 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.667+0000 7fb2a5d9b640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb28806ae70 0x7fb28806d330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb2a81f4610 tx=0x7fb29c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:01.673 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.671+0000 7fb296ffd640 1 -- 172.21.15.81:0/3648902906 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb298076690 con 0x7fb2a8071ad0 2023-11-08T16:42:01.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.864+0000 7fb2ac941640 1 -- 172.21.15.81:0/3648902906 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2a815c420 con 0x7fb2a8071ad0 2023-11-08T16:42:01.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.865+0000 7fb296ffd640 1 -- 172.21.15.81:0/3648902906 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb298031e60 con 0x7fb2a8071ad0 2023-11-08T16:42:01.867 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:42:01.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.867+0000 7fb2ac941640 1 -- 172.21.15.81:0/3648902906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb28806ae70 msgr2=0x7fb28806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:01.869 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.867+0000 7fb2ac941640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb28806ae70 0x7fb28806d330 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb2a81f4610 tx=0x7fb29c008040 comp rx=0 tx=0).stop 2023-11-08T16:42:01.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.867+0000 7fb2ac941640 1 -- 172.21.15.81:0/3648902906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a8071ad0 msgr2=0x7fb2a814da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:01.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.867+0000 7fb2ac941640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a8071ad0 0x7fb2a814da90 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fb298000c00 tx=0x7fb2980046c0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.868+0000 7fb2ac941640 1 -- 172.21.15.81:0/3648902906 shutdown_connections 2023-11-08T16:42:01.870 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.868+0000 7fb2ac941640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb28806ae70 0x7fb28806d330 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.868+0000 7fb2ac941640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2a815f1b0 0x7fb2a815d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.868+0000 7fb2ac941640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb2a8158c60 0x7fb2a81551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.868+0000 7fb2ac941640 1 --2- 172.21.15.81:0/3648902906 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb2a8071ad0 0x7fb2a814da90 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:01.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.868+0000 7fb2ac941640 1 -- 172.21.15.81:0/3648902906 >> 172.21.15.81:0/3648902906 conn(0x7fb2a80ceb70 msgr2=0x7fb2a80ca690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:01.871 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.868+0000 7fb2ac941640 1 -- 172.21.15.81:0/3648902906 shutdown_connections 2023-11-08T16:42:01.872 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:01.868+0000 7fb2ac941640 1 -- 172.21.15.81:0/3648902906 wait complete. 2023-11-08T16:42:01.888 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:42:02.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.104+0000 7fa078a00640 1 Processor -- start 2023-11-08T16:42:02.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.105+0000 7fa078a00640 1 -- start start 2023-11-08T16:42:02.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.106+0000 7fa078a00640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa074071ad0 0x7fa07415ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.106+0000 7fa078a00640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa07415f1b0 0x7fa0740c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.106+0000 7fa078a00640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07415fad0 con 0x7fa074158c60 2023-11-08T16:42:02.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.106+0000 7fa078a00640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07406df80 con 0x7fa074071ad0 2023-11-08T16:42:02.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.106+0000 7fa078a00640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07415f620 con 0x7fa07415f1b0 2023-11-08T16:42:02.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.106+0000 7fa071d9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa074071ad0 0x7fa07415ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.107+0000 7fa071d9b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa074071ad0 0x7fa07415ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58560/0 (socket says 172.21.15.81:58560) 2023-11-08T16:42:02.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.107+0000 7fa071d9b640 1 -- 172.21.15.81:0/1817952186 learned_addr learned my addr 172.21.15.81:0/1817952186 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:02.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.107+0000 7fa07259c640 1 --1- 172.21.15.81:0/1817952186 >> v1:172.21.15.81:6789/0 conn(0x7fa074158c60 0x7fa0740705d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45122/0 (socket says 172.21.15.81:45122) 2023-11-08T16:42:02.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.107+0000 7fa071d9b640 1 -- 172.21.15.81:0/1817952186 >> v2:172.21.15.93:3300/0 conn(0x7fa07415f1b0 msgr2=0x7fa0740c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:02.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.107+0000 7fa072d9d640 1 --2- 172.21.15.81:0/1817952186 >> v2:172.21.15.93:3300/0 conn(0x7fa07415f1b0 0x7fa0740c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.107+0000 7fa071d9b640 1 --2- 172.21.15.81:0/1817952186 >> v2:172.21.15.93:3300/0 conn(0x7fa07415f1b0 0x7fa0740c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.107+0000 7fa071d9b640 1 -- 172.21.15.81:0/1817952186 >> v1:172.21.15.81:6789/0 conn(0x7fa074158c60 legacy=0x7fa0740705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:02.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.107+0000 7fa071d9b640 1 -- 172.21.15.81:0/1817952186 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07406d760 con 0x7fa074071ad0 2023-11-08T16:42:02.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.107+0000 7fa072d9d640 1 --2- 172.21.15.81:0/1817952186 >> v2:172.21.15.93:3300/0 conn(0x7fa07415f1b0 0x7fa0740c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:02.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.108+0000 7fa071d9b640 1 --2- 172.21.15.81:0/1817952186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa074071ad0 0x7fa07415ec70 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fa0740b4ce0 tx=0x7fa05c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29158f17e8b1542 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:02.115 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.108+0000 7fa07159a640 1 -- 172.21.15.81:0/1817952186 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c02ca80 con 0x7fa074071ad0 2023-11-08T16:42:02.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.108+0000 7fa07159a640 1 -- 172.21.15.81:0/1817952186 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05c02cbe0 con 0x7fa074071ad0 2023-11-08T16:42:02.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.108+0000 7fa07159a640 1 -- 172.21.15.81:0/1817952186 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05c0343f0 con 0x7fa074071ad0 2023-11-08T16:42:02.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.109+0000 7fa078a00640 1 -- 172.21.15.81:0/1817952186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa074071ad0 msgr2=0x7fa07415ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:02.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.109+0000 7fa078a00640 1 --2- 172.21.15.81:0/1817952186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa074071ad0 0x7fa07415ec70 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fa0740b4ce0 tx=0x7fa05c02bee0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.109+0000 7fa078a00640 1 -- 172.21.15.81:0/1817952186 shutdown_connections 2023-11-08T16:42:02.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.109+0000 7fa078a00640 1 --2- 172.21.15.81:0/1817952186 >> v2:172.21.15.93:3300/0 conn(0x7fa07415f1b0 0x7fa0740c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.109+0000 7fa078a00640 1 --2- 172.21.15.81:0/1817952186 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa074071ad0 0x7fa07415ec70 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.109+0000 7fa078a00640 1 -- 172.21.15.81:0/1817952186 >> 172.21.15.81:0/1817952186 conn(0x7fa0740ceb70 msgr2=0x7fa0740cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:02.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.109+0000 7fa078a00640 1 -- 172.21.15.81:0/1817952186 shutdown_connections 2023-11-08T16:42:02.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.109+0000 7fa078a00640 1 -- 172.21.15.81:0/1817952186 wait complete. 2023-11-08T16:42:02.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.111+0000 7fa078a00640 1 Processor -- start 2023-11-08T16:42:02.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.111+0000 7fa078a00640 1 -- start start 2023-11-08T16:42:02.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.112+0000 7fa078a00640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa074071ad0 0x7fa0740b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.112+0000 7fa078a00640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa074158c60 0x7fa0740c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.112+0000 7fa078a00640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa07415f1b0 0x7fa07414b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.112+0000 7fa078a00640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07414b580 con 0x7fa074071ad0 2023-11-08T16:42:02.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.112+0000 7fa078a00640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740c72a0 con 0x7fa07415f1b0 2023-11-08T16:42:02.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.112+0000 7fa078a00640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0740c6200 con 0x7fa074158c60 2023-11-08T16:42:02.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.112+0000 7fa07259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa074071ad0 0x7fa0740b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.112+0000 7fa07259c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa074071ad0 0x7fa0740b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58576/0 (socket says 172.21.15.81:58576) 2023-11-08T16:42:02.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.112+0000 7fa07259c640 1 -- 172.21.15.81:0/4184281099 learned_addr learned my addr 172.21.15.81:0/4184281099 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:02.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa072d9d640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa07415f1b0 0x7fa07414b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa07259c640 1 -- 172.21.15.81:0/4184281099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa074158c60 msgr2=0x7fa0740c8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:02.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa071d9b640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa074158c60 0x7fa0740c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa07259c640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa074158c60 0x7fa0740c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa07259c640 1 -- 172.21.15.81:0/4184281099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa07415f1b0 msgr2=0x7fa07414b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:02.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa07259c640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa07415f1b0 0x7fa07414b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa07259c640 1 -- 172.21.15.81:0/4184281099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c009b30 con 0x7fa074071ad0 2023-11-08T16:42:02.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa07259c640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa074071ad0 0x7fa0740b1010 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa06400c4d0 tx=0x7fa06400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:02.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa06affd640 1 -- 172.21.15.81:0/4184281099 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa064007480 con 0x7fa074071ad0 2023-11-08T16:42:02.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa06affd640 1 -- 172.21.15.81:0/4184281099 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0640075e0 con 0x7fa074071ad0 2023-11-08T16:42:02.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa06affd640 1 -- 172.21.15.81:0/4184281099 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0640049e0 con 0x7fa074071ad0 2023-11-08T16:42:02.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.113+0000 7fa078a00640 1 -- 172.21.15.81:0/4184281099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0740c64e0 con 0x7fa074071ad0 2023-11-08T16:42:02.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.114+0000 7fa078a00640 1 -- 172.21.15.81:0/4184281099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0740c6aa0 con 0x7fa074071ad0 2023-11-08T16:42:02.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.115+0000 7fa078a00640 1 -- 172.21.15.81:0/4184281099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa074074440 con 0x7fa074071ad0 2023-11-08T16:42:02.126 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.116+0000 7fa06affd640 1 -- 172.21.15.81:0/4184281099 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa064004b40 con 0x7fa074071ad0 2023-11-08T16:42:02.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.117+0000 7fa06affd640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 0x7fa05806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.117+0000 7fa06affd640 1 -- 172.21.15.81:0/4184281099 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa064088820 con 0x7fa074071ad0 2023-11-08T16:42:02.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.117+0000 7fa071d9b640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 0x7fa05806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.127 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.118+0000 7fa071d9b640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 0x7fa05806d330 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa05c035000 tx=0x7fa05c030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:02.128 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.121+0000 7fa06affd640 1 -- 172.21.15.81:0/4184281099 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa0640537f0 con 0x7fa074071ad0 2023-11-08T16:42:02.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.317+0000 7fa078a00640 1 -- 172.21.15.81:0/4184281099 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa07415e840 con 0x7fa074071ad0 2023-11-08T16:42:02.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.318+0000 7fa06affd640 1 -- 172.21.15.81:0/4184281099 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa064056e10 con 0x7fa074071ad0 2023-11-08T16:42:02.320 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:42:02.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 -- 172.21.15.81:0/4184281099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 msgr2=0x7fa05806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:02.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 0x7fa05806d330 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fa05c035000 tx=0x7fa05c030000 comp rx=0 tx=0).stop 2023-11-08T16:42:02.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 -- 172.21.15.81:0/4184281099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa074071ad0 msgr2=0x7fa0740b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:02.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa074071ad0 0x7fa0740b1010 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fa06400c4d0 tx=0x7fa06400c990 comp rx=0 tx=0).stop 2023-11-08T16:42:02.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 -- 172.21.15.81:0/4184281099 shutdown_connections 2023-11-08T16:42:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa05806ae70 0x7fa05806d330 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa07415f1b0 0x7fa07414b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa074158c60 0x7fa0740c8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 --2- 172.21.15.81:0/4184281099 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa074071ad0 0x7fa0740b1010 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 -- 172.21.15.81:0/4184281099 >> 172.21.15.81:0/4184281099 conn(0x7fa0740ceb70 msgr2=0x7fa0740b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 -- 172.21.15.81:0/4184281099 shutdown_connections 2023-11-08T16:42:02.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.321+0000 7fa078a00640 1 -- 172.21.15.81:0/4184281099 wait complete. 2023-11-08T16:42:02.341 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:42:02.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.558+0000 7f1fa212c640 1 Processor -- start 2023-11-08T16:42:02.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.558+0000 7f1fa212c640 1 -- start start 2023-11-08T16:42:02.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.559+0000 7f1fa212c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1f9c158c60 0x7f1f9c0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.559+0000 7f1fa212c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f9c071ad0 0x7f1f9c15ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.559+0000 7f1fa212c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f1f9c15f1b0 0x7f1f9c0c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.559+0000 7f1fa212c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9c06df80 con 0x7f1f9c158c60 2023-11-08T16:42:02.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.560+0000 7f1fa212c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f9c15f5e0 con 0x7f1f9c15f1b0 2023-11-08T16:42:02.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.560+0000 7f1fa212c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c15fad0 con 0x7f1f9c071ad0 2023-11-08T16:42:02.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.560+0000 7f1f9b7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1f9c158c60 0x7f1f9c0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.570 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.560+0000 7f1f9b7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f1f9c158c60 0x7f1f9c0705d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58586/0 (socket says 172.21.15.81:58586) 2023-11-08T16:42:02.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.560+0000 7f1f9b7fe640 1 -- 172.21.15.81:0/459672094 learned_addr learned my addr 172.21.15.81:0/459672094 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:02.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.560+0000 7f1f93fff640 1 --2- 172.21.15.81:0/459672094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f9c071ad0 0x7f1f9c15ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.560+0000 7f1f9b7fe640 1 -- 172.21.15.81:0/459672094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f9c071ad0 msgr2=0x7f1f9c15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:02.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.560+0000 7f1f9b7fe640 1 --2- 172.21.15.81:0/459672094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f9c071ad0 0x7f1f9c15ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.571 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.560+0000 7f1f9b7fe640 1 -- 172.21.15.81:0/459672094 >> v2:172.21.15.93:3300/0 conn(0x7f1f9c15f1b0 msgr2=0x7f1f9c0c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:02.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.560+0000 7f1f9bfff640 1 --2- 172.21.15.81:0/459672094 >> v2:172.21.15.93:3300/0 conn(0x7f1f9c15f1b0 0x7f1f9c0c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.561+0000 7f1f9b7fe640 1 --2- 172.21.15.81:0/459672094 >> v2:172.21.15.93:3300/0 conn(0x7f1f9c15f1b0 0x7f1f9c0c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.561+0000 7f1f9b7fe640 1 -- 172.21.15.81:0/459672094 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9c06d760 con 0x7f1f9c158c60 2023-11-08T16:42:02.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.561+0000 7f1f93fff640 1 --2- 172.21.15.81:0/459672094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f9c071ad0 0x7f1f9c15ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:02.572 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.561+0000 7f1f9bfff640 1 --2- 172.21.15.81:0/459672094 >> v2:172.21.15.93:3300/0 conn(0x7f1f9c15f1b0 0x7f1f9c0c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:02.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.562+0000 7f1f9b7fe640 1 --2- 172.21.15.81:0/459672094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f9c158c60 0x7f1f9c0705d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0caed0 tx=0x7f1f8c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98e4b254b95f73f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:02.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.562+0000 7f1f9affd640 1 -- 172.21.15.81:0/459672094 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c031880 con 0x7f1f9c158c60 2023-11-08T16:42:02.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.562+0000 7f1f9affd640 1 -- 172.21.15.81:0/459672094 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c0319e0 con 0x7f1f9c158c60 2023-11-08T16:42:02.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.562+0000 7f1f9affd640 1 -- 172.21.15.81:0/459672094 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c031d70 con 0x7f1f9c158c60 2023-11-08T16:42:02.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.563+0000 7f1fa212c640 1 -- 172.21.15.81:0/459672094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f9c158c60 msgr2=0x7f1f9c0705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:02.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.563+0000 7f1fa212c640 1 --2- 172.21.15.81:0/459672094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f9c158c60 0x7f1f9c0705d0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f1f9c0caed0 tx=0x7f1f8c02b920 comp rx=0 tx=0).stop 2023-11-08T16:42:02.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.563+0000 7f1fa212c640 1 -- 172.21.15.81:0/459672094 shutdown_connections 2023-11-08T16:42:02.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.563+0000 7f1fa212c640 1 --2- 172.21.15.81:0/459672094 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f9c071ad0 0x7f1f9c15ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.574 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.563+0000 7f1fa212c640 1 --2- 172.21.15.81:0/459672094 >> v2:172.21.15.93:3300/0 conn(0x7f1f9c15f1b0 0x7f1f9c0c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.563+0000 7f1fa212c640 1 --2- 172.21.15.81:0/459672094 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f9c158c60 0x7f1f9c0705d0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.563+0000 7f1fa212c640 1 -- 172.21.15.81:0/459672094 >> 172.21.15.81:0/459672094 conn(0x7f1f9c0ceb70 msgr2=0x7f1f9c0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:02.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.563+0000 7f1fa212c640 1 -- 172.21.15.81:0/459672094 shutdown_connections 2023-11-08T16:42:02.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.563+0000 7f1fa212c640 1 -- 172.21.15.81:0/459672094 wait complete. 2023-11-08T16:42:02.575 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.565+0000 7f1fa212c640 1 Processor -- start 2023-11-08T16:42:02.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.565+0000 7f1fa212c640 1 -- start start 2023-11-08T16:42:02.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1fa212c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f9c071ad0 0x7f1f9c155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1fa212c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f9c158c60 0x7f1f9c0aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.576 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1fa212c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f9c15f1b0 0x7f1f9c15d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1fa212c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c159440 con 0x7f1f9c158c60 2023-11-08T16:42:02.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1fa212c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0af370 con 0x7f1f9c15f1b0 2023-11-08T16:42:02.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1fa212c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0af5f0 con 0x7f1f9c071ad0 2023-11-08T16:42:02.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1f93fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f9c158c60 0x7f1f9c0aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1f93fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f9c158c60 0x7f1f9c0aee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58594/0 (socket says 172.21.15.81:58594) 2023-11-08T16:42:02.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1f93fff640 1 -- 172.21.15.81:0/385956412 learned_addr learned my addr 172.21.15.81:0/385956412 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:02.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1f9b7fe640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f9c071ad0 0x7f1f9c155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1f93fff640 1 -- 172.21.15.81:0/385956412 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f9c071ad0 msgr2=0x7f1f9c155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:02.578 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1f93fff640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f9c071ad0 0x7f1f9c155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1f9bfff640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f9c15f1b0 0x7f1f9c15d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.566+0000 7f1f93fff640 1 -- 172.21.15.81:0/385956412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f9c15f1b0 msgr2=0x7f1f9c15d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:02.579 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.567+0000 7f1f93fff640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f9c15f1b0 0x7f1f9c15d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.567+0000 7f1f93fff640 1 -- 172.21.15.81:0/385956412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c0094d0 con 0x7f1f9c158c60 2023-11-08T16:42:02.580 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.567+0000 7f1f9bfff640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f9c15f1b0 0x7f1f9c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:02.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.567+0000 7f1f93fff640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f9c158c60 0x7f1f9c0aee30 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400c7f0 tx=0x7f1f8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:02.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.567+0000 7f1f98ff9640 1 -- 172.21.15.81:0/385956412 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84007820 con 0x7f1f9c158c60 2023-11-08T16:42:02.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.567+0000 7f1f98ff9640 1 -- 172.21.15.81:0/385956412 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f84007980 con 0x7f1f9c158c60 2023-11-08T16:42:02.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.567+0000 7f1f98ff9640 1 -- 172.21.15.81:0/385956412 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f84007d30 con 0x7f1f9c158c60 2023-11-08T16:42:02.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.567+0000 7f1fa212c640 1 -- 172.21.15.81:0/385956412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c0af8d0 con 0x7f1f9c158c60 2023-11-08T16:42:02.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.567+0000 7f1fa212c640 1 -- 172.21.15.81:0/385956412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9c0afe90 con 0x7f1f9c158c60 2023-11-08T16:42:02.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.569+0000 7f1fa212c640 1 -- 172.21.15.81:0/385956412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c074440 con 0x7f1f9c158c60 2023-11-08T16:42:02.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.569+0000 7f1f98ff9640 1 -- 172.21.15.81:0/385956412 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1f840113f0 con 0x7f1f9c158c60 2023-11-08T16:42:02.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.571+0000 7f1f98ff9640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f7c06ae70 0x7f1f7c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:02.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.571+0000 7f1f98ff9640 1 -- 172.21.15.81:0/385956412 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f1f84089990 con 0x7f1f9c158c60 2023-11-08T16:42:02.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.571+0000 7f1f9b7fe640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f7c06ae70 0x7f1f7c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:02.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.572+0000 7f1f9b7fe640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f7c06ae70 0x7f1f7c06d330 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c036000 tx=0x7f1f8c03c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:02.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.575+0000 7f1f98ff9640 1 -- 172.21.15.81:0/385956412 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1f84054960 con 0x7f1f9c158c60 2023-11-08T16:42:02.765 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.764+0000 7f1fa212c640 1 -- 172.21.15.81:0/385956412 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f9c0aec10 con 0x7f1f9c158c60 2023-11-08T16:42:02.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.765+0000 7f1f98ff9640 1 -- 172.21.15.81:0/385956412 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f84057f80 con 0x7f1f9c158c60 2023-11-08T16:42:02.767 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:42:02.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.767+0000 7f1fa212c640 1 -- 172.21.15.81:0/385956412 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f7c06ae70 msgr2=0x7f1f7c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:02.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.767+0000 7f1fa212c640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f7c06ae70 0x7f1f7c06d330 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c036000 tx=0x7f1f8c03c040 comp rx=0 tx=0).stop 2023-11-08T16:42:02.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.767+0000 7f1fa212c640 1 -- 172.21.15.81:0/385956412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f9c158c60 msgr2=0x7f1f9c0aee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:02.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.767+0000 7f1fa212c640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f9c158c60 0x7f1f9c0aee30 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f1f8400c7f0 tx=0x7f1f8400ccb0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.767+0000 7f1fa212c640 1 -- 172.21.15.81:0/385956412 shutdown_connections 2023-11-08T16:42:02.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.767+0000 7f1fa212c640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f1f7c06ae70 0x7f1f7c06d330 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.767+0000 7f1fa212c640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f9c15f1b0 0x7f1f9c15d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.767+0000 7f1fa212c640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1f9c158c60 0x7f1f9c0aee30 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.767+0000 7f1fa212c640 1 --2- 172.21.15.81:0/385956412 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f1f9c071ad0 0x7f1f9c155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:02.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.767+0000 7f1fa212c640 1 -- 172.21.15.81:0/385956412 >> 172.21.15.81:0/385956412 conn(0x7f1f9c0ceb70 msgr2=0x7f1f9c0b3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:02.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.767+0000 7f1fa212c640 1 -- 172.21.15.81:0/385956412 shutdown_connections 2023-11-08T16:42:02.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:02.768+0000 7f1fa212c640 1 -- 172.21.15.81:0/385956412 wait complete. 2023-11-08T16:42:02.787 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-08T16:42:03.005 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.004+0000 7f227f821640 1 Processor -- start 2023-11-08T16:42:03.006 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.004+0000 7f227f821640 1 -- start start 2023-11-08T16:42:03.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.005+0000 7f227f821640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f22780cd3c0 0x7f22780cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:03.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.005+0000 7f227f821640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f22780cbb10 0x7f227815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:03.010 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.005+0000 7f227f821640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2278071620 con 0x7f2278158aa0 2023-11-08T16:42:03.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.005+0000 7f227f821640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22780cbfa0 con 0x7f22780cd3c0 2023-11-08T16:42:03.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.005+0000 7f227f821640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2278158e80 con 0x7f22780cbb10 2023-11-08T16:42:03.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.006+0000 7f227cdbc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f22780cd3c0 0x7f22780cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:03.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.006+0000 7f227cdbc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f22780cd3c0 0x7f22780cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58606/0 (socket says 172.21.15.81:58606) 2023-11-08T16:42:03.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.006+0000 7f227cdbc640 1 -- 172.21.15.81:0/1850788603 learned_addr learned my addr 172.21.15.81:0/1850788603 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:03.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.006+0000 7f227ddbe640 1 --2- 172.21.15.81:0/1850788603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f22780cbb10 0x7f227815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:03.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.006+0000 7f227cdbc640 1 -- 172.21.15.81:0/1850788603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f22780cbb10 msgr2=0x7f227815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:03.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.006+0000 7f227cdbc640 1 --2- 172.21.15.81:0/1850788603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f22780cbb10 0x7f227815ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:03.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.006+0000 7f227cdbc640 1 -- 172.21.15.81:0/1850788603 >> v1:172.21.15.93:6789/0 conn(0x7f2278158aa0 legacy=0x7f227806fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:03.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.006+0000 7f227cdbc640 1 -- 172.21.15.81:0/1850788603 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2278071850 con 0x7f22780cd3c0 2023-11-08T16:42:03.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.006+0000 7f227ddbe640 1 --2- 172.21.15.81:0/1850788603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f22780cbb10 0x7f227815ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:03.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.007+0000 7f227cdbc640 1 --2- 172.21.15.81:0/1850788603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22780cd3c0 0x7f22780cb5d0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2268009870 tx=0x7f226802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3154090d42ed3c4d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:03.015 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.008+0000 7f226ffff640 1 -- 172.21.15.81:0/1850788603 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268031640 con 0x7f22780cd3c0 2023-11-08T16:42:03.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.008+0000 7f226ffff640 1 -- 172.21.15.81:0/1850788603 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22680317a0 con 0x7f22780cd3c0 2023-11-08T16:42:03.016 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.008+0000 7f226ffff640 1 -- 172.21.15.81:0/1850788603 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268031b30 con 0x7f22780cd3c0 2023-11-08T16:42:03.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.008+0000 7f227f821640 1 -- 172.21.15.81:0/1850788603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22780cd3c0 msgr2=0x7f22780cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:03.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.008+0000 7f227f821640 1 --2- 172.21.15.81:0/1850788603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22780cd3c0 0x7f22780cb5d0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2268009870 tx=0x7f226802b8f0 comp rx=0 tx=0).stop 2023-11-08T16:42:03.021 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.008+0000 7f227f821640 1 -- 172.21.15.81:0/1850788603 shutdown_connections 2023-11-08T16:42:03.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.008+0000 7f227f821640 1 --2- 172.21.15.81:0/1850788603 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f22780cbb10 0x7f227815ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:03.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.008+0000 7f227f821640 1 --2- 172.21.15.81:0/1850788603 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22780cd3c0 0x7f22780cb5d0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:03.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.008+0000 7f227f821640 1 -- 172.21.15.81:0/1850788603 >> 172.21.15.81:0/1850788603 conn(0x7f22780b0770 msgr2=0x7f22780b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:03.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.008+0000 7f227f821640 1 -- 172.21.15.81:0/1850788603 shutdown_connections 2023-11-08T16:42:03.022 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.008+0000 7f227f821640 1 -- 172.21.15.81:0/1850788603 wait complete. 2023-11-08T16:42:03.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.010+0000 7f227f821640 1 Processor -- start 2023-11-08T16:42:03.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.010+0000 7f227f821640 1 -- start start 2023-11-08T16:42:03.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.011+0000 7f227f821640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22780cbb10 0x7f22781512c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:03.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.011+0000 7f227f821640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22780cd3c0 0x7f22780c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:03.023 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.011+0000 7f227f821640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2278158aa0 0x7f22780c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:03.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.011+0000 7f227f821640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2278155160 con 0x7f22780cd3c0 2023-11-08T16:42:03.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.011+0000 7f227f821640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2278155440 con 0x7f22780cbb10 2023-11-08T16:42:03.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.011+0000 7f227f821640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2278155720 con 0x7f2278158aa0 2023-11-08T16:42:03.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.011+0000 7f227cdbc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22780cd3c0 0x7f22780c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:03.024 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.012+0000 7f227cdbc640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22780cd3c0 0x7f22780c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58618/0 (socket says 172.21.15.81:58618) 2023-11-08T16:42:03.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.012+0000 7f227cdbc640 1 -- 172.21.15.81:0/276289274 learned_addr learned my addr 172.21.15.81:0/276289274 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:03.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.012+0000 7f227d5bd640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22780cbb10 0x7f22781512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:03.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.012+0000 7f227cdbc640 1 -- 172.21.15.81:0/276289274 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2278158aa0 msgr2=0x7f22780c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:03.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.012+0000 7f227cdbc640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2278158aa0 0x7f22780c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:03.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.012+0000 7f227cdbc640 1 -- 172.21.15.81:0/276289274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22780cbb10 msgr2=0x7f22781512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:03.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.012+0000 7f227cdbc640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22780cbb10 0x7f22781512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:03.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.012+0000 7f227cdbc640 1 -- 172.21.15.81:0/276289274 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22680094d0 con 0x7f22780cd3c0 2023-11-08T16:42:03.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.012+0000 7f227d5bd640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22780cbb10 0x7f22781512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:03.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.012+0000 7f227cdbc640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22780cd3c0 0x7f22780c81f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f2268009870 tx=0x7f22680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:03.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.013+0000 7f226dffb640 1 -- 172.21.15.81:0/276289274 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226803ac60 con 0x7f22780cd3c0 2023-11-08T16:42:03.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.013+0000 7f226dffb640 1 -- 172.21.15.81:0/276289274 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226803ade0 con 0x7f22780cd3c0 2023-11-08T16:42:03.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.013+0000 7f226dffb640 1 -- 172.21.15.81:0/276289274 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268044630 con 0x7f22780cd3c0 2023-11-08T16:42:03.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.013+0000 7f227f821640 1 -- 172.21.15.81:0/276289274 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2278155a00 con 0x7f22780cd3c0 2023-11-08T16:42:03.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.013+0000 7f227f821640 1 -- 172.21.15.81:0/276289274 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2278155fc0 con 0x7f22780cd3c0 2023-11-08T16:42:03.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.014+0000 7f227f821640 1 -- 172.21.15.81:0/276289274 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22781561b0 con 0x7f22780cd3c0 2023-11-08T16:42:03.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.015+0000 7f226dffb640 1 -- 172.21.15.81:0/276289274 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f226802f050 con 0x7f22780cd3c0 2023-11-08T16:42:03.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.016+0000 7f226dffb640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f225c06ae70 0x7f225c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:03.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.016+0000 7f226dffb640 1 -- 172.21.15.81:0/276289274 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f22680ab880 con 0x7f22780cd3c0 2023-11-08T16:42:03.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.016+0000 7f227d5bd640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f225c06ae70 0x7f225c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:03.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.017+0000 7f227d5bd640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f225c06ae70 0x7f225c06d330 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2274003eb0 tx=0x7f2274009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:03.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.021+0000 7f226dffb640 1 -- 172.21.15.81:0/276289274 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2268076840 con 0x7f22780cd3c0 2023-11-08T16:42:03.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.226+0000 7f227f821640 1 -- 172.21.15.81:0/276289274 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2278074440 con 0x7f22780cd3c0 2023-11-08T16:42:03.228 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.227+0000 7f226dffb640 1 -- 172.21.15.81:0/276289274 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v131) v1 ==== 80+0+242 (secure 0 0 0) 0x7f2268035110 con 0x7f22780cd3c0 2023-11-08T16:42:03.229 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6837/296230272 2023-11-09T16:41:49.127674+0000 2023-11-08T16:42:03.229 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.81:6836/296230272 2023-11-09T16:41:49.127674+0000 2023-11-08T16:42:03.229 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.93:6835/4021510722 2023-11-09T16:41:29.470203+0000 2023-11-08T16:42:03.230 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.93:6833/4021510722 2023-11-09T16:41:29.470203+0000 2023-11-08T16:42:03.230 INFO:teuthology.orchestra.run.smithi081.stderr:listed 4 entries 2023-11-08T16:42:03.231 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 -- 172.21.15.81:0/276289274 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f225c06ae70 msgr2=0x7f225c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f225c06ae70 0x7f225c06d330 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2274003eb0 tx=0x7f2274009040 comp rx=0 tx=0).stop 2023-11-08T16:42:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 -- 172.21.15.81:0/276289274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22780cd3c0 msgr2=0x7f22780c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22780cd3c0 0x7f22780c81f0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f2268009870 tx=0x7f22680046c0 comp rx=0 tx=0).stop 2023-11-08T16:42:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 -- 172.21.15.81:0/276289274 shutdown_connections 2023-11-08T16:42:03.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f225c06ae70 0x7f225c06d330 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:03.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2278158aa0 0x7f22780c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:03.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f22780cd3c0 0x7f22780c81f0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:03.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 --2- 172.21.15.81:0/276289274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f22780cbb10 0x7f22781512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:03.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 -- 172.21.15.81:0/276289274 >> 172.21.15.81:0/276289274 conn(0x7f22780b0770 msgr2=0x7f22780ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:03.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 -- 172.21.15.81:0/276289274 shutdown_connections 2023-11-08T16:42:03.234 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:03.230+0000 7f227f821640 1 -- 172.21.15.81:0/276289274 wait complete. 2023-11-08T16:42:03.250 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi081.front.sepia.ceph.com... 2023-11-08T16:42:03.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:03.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-08T16:42:03.339 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi081.stderr:ceph-fuse[127959]: fuse finished with error 0 and tester_r 0 2023-11-08T16:42:03.345 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:42:03.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:03.346 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-08T16:42:03.525 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:42:03.526 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:42:03.586 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:42:03.587 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:42:03.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:03.588 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-08T16:42:03.642 INFO:teuthology.orchestra.run.smithi093.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-08T16:42:03.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:42:03.644 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:42:03.644 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:42:03.703 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:42:03.703 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:42:03.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:03.704 DEBUG:teuthology.orchestra.run.smithi081:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-08T16:42:03.758 INFO:teuthology.orchestra.run.smithi081.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-08T16:42:03.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:42:03.760 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:42:03.760 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:42:03.819 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:42:03.820 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:42:03.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:03.821 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-08T16:42:03.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:42:03.880 INFO:teuthology.orchestra.run.smithi093.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-08T16:42:03.880 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:42:04.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.104+0000 7fda3fe90640 1 Processor -- start 2023-11-08T16:42:04.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.104+0000 7fda3fe90640 1 -- start start 2023-11-08T16:42:04.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.105+0000 7fda3fe90640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fda38143800 0x7fda380b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.105+0000 7fda3fe90640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fda380cca70 0x7fda3815d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.106+0000 7fda3fe90640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda380714f0 con 0x7fda3815df20 2023-11-08T16:42:04.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.106+0000 7fda3fe90640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda380b2df0 con 0x7fda380cca70 2023-11-08T16:42:04.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.106+0000 7fda3fe90640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3815e330 con 0x7fda38143800 2023-11-08T16:42:04.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.106+0000 7fda3d42b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fda380cca70 0x7fda3815d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:04.108 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.106+0000 7fda3e42d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fda3815df20 0x7fda3815a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45124/0 (socket says 172.21.15.81:45124) 2023-11-08T16:42:04.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.106+0000 7fda3e42d640 1 -- 172.21.15.81:0/1687220143 learned_addr learned my addr 172.21.15.81:0/1687220143 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:04.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.106+0000 7fda3dc2c640 1 --2- 172.21.15.81:0/1687220143 >> v2:172.21.15.93:3300/0 conn(0x7fda38143800 0x7fda380b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:04.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.106+0000 7fda3d42b640 1 -- 172.21.15.81:0/1687220143 >> v2:172.21.15.93:3300/0 conn(0x7fda38143800 msgr2=0x7fda380b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:04.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.106+0000 7fda3d42b640 1 --2- 172.21.15.81:0/1687220143 >> v2:172.21.15.93:3300/0 conn(0x7fda38143800 0x7fda380b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.106+0000 7fda3d42b640 1 -- 172.21.15.81:0/1687220143 >> v1:172.21.15.81:6789/0 conn(0x7fda3815df20 legacy=0x7fda3815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:04.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.106+0000 7fda3d42b640 1 -- 172.21.15.81:0/1687220143 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda380cd1e0 con 0x7fda380cca70 2023-11-08T16:42:04.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.107+0000 7fda3d42b640 1 --2- 172.21.15.81:0/1687220143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda380cca70 0x7fda3815d9e0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fda380cc720 tx=0x7fda2802dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944c4a3ee80e16df server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:04.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.108+0000 7fda3cc2a640 1 -- 172.21.15.81:0/1687220143 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2802f8a0 con 0x7fda380cca70 2023-11-08T16:42:04.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.108+0000 7fda3cc2a640 1 -- 172.21.15.81:0/1687220143 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2802fa00 con 0x7fda380cca70 2023-11-08T16:42:04.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.108+0000 7fda3cc2a640 1 -- 172.21.15.81:0/1687220143 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2802fd90 con 0x7fda380cca70 2023-11-08T16:42:04.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.108+0000 7fda3fe90640 1 -- 172.21.15.81:0/1687220143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda380cca70 msgr2=0x7fda3815d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:04.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.108+0000 7fda3fe90640 1 --2- 172.21.15.81:0/1687220143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda380cca70 0x7fda3815d9e0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fda380cc720 tx=0x7fda2802dae0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.108+0000 7fda3fe90640 1 -- 172.21.15.81:0/1687220143 shutdown_connections 2023-11-08T16:42:04.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.108+0000 7fda3fe90640 1 --2- 172.21.15.81:0/1687220143 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda380cca70 0x7fda3815d9e0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.108+0000 7fda3fe90640 1 --2- 172.21.15.81:0/1687220143 >> v2:172.21.15.93:3300/0 conn(0x7fda38143800 0x7fda380b28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.108+0000 7fda3fe90640 1 -- 172.21.15.81:0/1687220143 >> 172.21.15.81:0/1687220143 conn(0x7fda3806a6a0 msgr2=0x7fda380b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:04.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.108+0000 7fda3fe90640 1 -- 172.21.15.81:0/1687220143 shutdown_connections 2023-11-08T16:42:04.113 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.108+0000 7fda3fe90640 1 -- 172.21.15.81:0/1687220143 wait complete. 2023-11-08T16:42:04.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.110+0000 7fda3fe90640 1 Processor -- start 2023-11-08T16:42:04.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.110+0000 7fda3fe90640 1 -- start start 2023-11-08T16:42:04.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.111+0000 7fda3fe90640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda380cca70 0x7fda38132370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.116 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.111+0000 7fda3fe90640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda38143800 0x7fda3815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.111+0000 7fda3fe90640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fda3815df20 0x7fda381e3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.111+0000 7fda3fe90640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3815bc90 con 0x7fda38143800 2023-11-08T16:42:04.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.111+0000 7fda3fe90640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3815bf70 con 0x7fda380cca70 2023-11-08T16:42:04.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.111+0000 7fda3fe90640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3815c250 con 0x7fda3815df20 2023-11-08T16:42:04.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.111+0000 7fda3d42b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda38143800 0x7fda3815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:04.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda3d42b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda38143800 0x7fda3815cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58636/0 (socket says 172.21.15.81:58636) 2023-11-08T16:42:04.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda3d42b640 1 -- 172.21.15.81:0/1075387967 learned_addr learned my addr 172.21.15.81:0/1075387967 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:04.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda3dc2c640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda380cca70 0x7fda38132370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:04.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda3d42b640 1 -- 172.21.15.81:0/1075387967 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fda3815df20 msgr2=0x7fda381e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:04.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda3d42b640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fda3815df20 0x7fda381e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda3d42b640 1 -- 172.21.15.81:0/1075387967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda380cca70 msgr2=0x7fda38132370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:04.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda3d42b640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda380cca70 0x7fda38132370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda3d42b640 1 -- 172.21.15.81:0/1075387967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28009400 con 0x7fda38143800 2023-11-08T16:42:04.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda3dc2c640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda380cca70 0x7fda38132370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:04.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda3d42b640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda38143800 0x7fda3815cd30 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fda28009890 tx=0x7fda280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:04.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda2e7fc640 1 -- 172.21.15.81:0/1075387967 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28030ab0 con 0x7fda38143800 2023-11-08T16:42:04.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.112+0000 7fda2e7fc640 1 -- 172.21.15.81:0/1075387967 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda28030c30 con 0x7fda38143800 2023-11-08T16:42:04.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.113+0000 7fda3fe90640 1 -- 172.21.15.81:0/1075387967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3815c530 con 0x7fda38143800 2023-11-08T16:42:04.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.113+0000 7fda2e7fc640 1 -- 172.21.15.81:0/1075387967 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28042410 con 0x7fda38143800 2023-11-08T16:42:04.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.113+0000 7fda3fe90640 1 -- 172.21.15.81:0/1075387967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda38132b40 con 0x7fda38143800 2023-11-08T16:42:04.121 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.114+0000 7fda3fe90640 1 -- 172.21.15.81:0/1075387967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3815ca50 con 0x7fda38143800 2023-11-08T16:42:04.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.115+0000 7fda2e7fc640 1 -- 172.21.15.81:0/1075387967 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fda28042570 con 0x7fda38143800 2023-11-08T16:42:04.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.116+0000 7fda2e7fc640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda1c06ae70 0x7fda1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.116+0000 7fda2e7fc640 1 -- 172.21.15.81:0/1075387967 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fda280ab580 con 0x7fda38143800 2023-11-08T16:42:04.122 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.116+0000 7fda3dc2c640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda1c06ae70 0x7fda1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:04.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.117+0000 7fda3dc2c640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda1c06ae70 0x7fda1c06d330 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fda34006fd0 tx=0x7fda34008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:04.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.120+0000 7fda2e7fc640 1 -- 172.21.15.81:0/1075387967 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fda3815ca50 con 0x7fda38143800 2023-11-08T16:42:04.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.350+0000 7fda3fe90640 1 -- 172.21.15.81:0/1075387967 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda3815ca50 con 0x7fda38143800 2023-11-08T16:42:04.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.351+0000 7fda2e7fc640 1 -- 172.21.15.81:0/1075387967 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4920 (secure 0 0 0) 0x7fda3815ca50 con 0x7fda38143800 2023-11-08T16:42:04.353 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:04.353 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":120,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/329260749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":329260749},{"type":"v1","addr":"172.21.15.93:6835","nonce":329260749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:55.094753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":117,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/1432485913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1432485913},{"type":"v1","addr":"172.21.15.81:6837","nonce":1432485913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:42:04.354 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 120 2023-11-08T16:42:04.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.354+0000 7fda3fe90640 1 -- 172.21.15.81:0/1075387967 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda1c06ae70 msgr2=0x7fda1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:04.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.354+0000 7fda3fe90640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda1c06ae70 0x7fda1c06d330 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fda34006fd0 tx=0x7fda34008040 comp rx=0 tx=0).stop 2023-11-08T16:42:04.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.354+0000 7fda3fe90640 1 -- 172.21.15.81:0/1075387967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda38143800 msgr2=0x7fda3815cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:04.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.354+0000 7fda3fe90640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda38143800 0x7fda3815cd30 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fda28009890 tx=0x7fda280046c0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.354+0000 7fda3fe90640 1 -- 172.21.15.81:0/1075387967 shutdown_connections 2023-11-08T16:42:04.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.354+0000 7fda3fe90640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fda1c06ae70 0x7fda1c06d330 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.354+0000 7fda3fe90640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fda3815df20 0x7fda381e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.354+0000 7fda3fe90640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fda38143800 0x7fda3815cd30 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.354+0000 7fda3fe90640 1 --2- 172.21.15.81:0/1075387967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda380cca70 0x7fda38132370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.354+0000 7fda3fe90640 1 -- 172.21.15.81:0/1075387967 >> 172.21.15.81:0/1075387967 conn(0x7fda3806a6a0 msgr2=0x7fda38070330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:04.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.355+0000 7fda3fe90640 1 -- 172.21.15.81:0/1075387967 shutdown_connections 2023-11-08T16:42:04.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.355+0000 7fda3fe90640 1 -- 172.21.15.81:0/1075387967 wait complete. 2023-11-08T16:42:04.375 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:42:04.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.581+0000 7fbce667c640 1 Processor -- start 2023-11-08T16:42:04.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.581+0000 7fbce667c640 1 -- start start 2023-11-08T16:42:04.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.583+0000 7fbce667c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbce00b3170 0x7fbce015b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.583+0000 7fbce667c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fbce00b3550 0x7fbce00b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.583+0000 7fbce667c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbce00b2310 con 0x7fbce006c9f0 2023-11-08T16:42:04.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.583+0000 7fbce667c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce00b2bb0 con 0x7fbce00b3170 2023-11-08T16:42:04.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.583+0000 7fbce667c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbce00b25f0 con 0x7fbce00b3550 2023-11-08T16:42:04.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.583+0000 7fbcdffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbce00b3170 0x7fbce015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:04.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.583+0000 7fbcdffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbce00b3170 0x7fbce015b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58648/0 (socket says 172.21.15.81:58648) 2023-11-08T16:42:04.586 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.583+0000 7fbcdffff640 1 -- 172.21.15.81:0/1647531168 learned_addr learned my addr 172.21.15.81:0/1647531168 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:04.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.584+0000 7fbcdffff640 1 -- 172.21.15.81:0/1647531168 >> v2:172.21.15.93:3300/0 conn(0x7fbce00b3550 msgr2=0x7fbce00b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:04.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.584+0000 7fbcdffff640 1 --2- 172.21.15.81:0/1647531168 >> v2:172.21.15.93:3300/0 conn(0x7fbce00b3550 0x7fbce00b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.584+0000 7fbcdffff640 1 -- 172.21.15.81:0/1647531168 >> v1:172.21.15.81:6789/0 conn(0x7fbce006c9f0 legacy=0x7fbce0144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:04.587 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.584+0000 7fbcdffff640 1 -- 172.21.15.81:0/1647531168 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce00b28d0 con 0x7fbce00b3170 2023-11-08T16:42:04.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.585+0000 7fbcdffff640 1 --2- 172.21.15.81:0/1647531168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbce00b3170 0x7fbce015b500 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fbce00b39c0 tx=0x7fbcd002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda395f16707b234 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:04.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.585+0000 7fbcdf7fe640 1 -- 172.21.15.81:0/1647531168 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0030860 con 0x7fbce00b3170 2023-11-08T16:42:04.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.585+0000 7fbcdf7fe640 1 -- 172.21.15.81:0/1647531168 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd00309c0 con 0x7fbce00b3170 2023-11-08T16:42:04.589 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.585+0000 7fbcdf7fe640 1 -- 172.21.15.81:0/1647531168 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0030d50 con 0x7fbce00b3170 2023-11-08T16:42:04.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.586+0000 7fbce667c640 1 -- 172.21.15.81:0/1647531168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbce00b3170 msgr2=0x7fbce015b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:04.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.586+0000 7fbce667c640 1 --2- 172.21.15.81:0/1647531168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbce00b3170 0x7fbce015b500 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fbce00b39c0 tx=0x7fbcd002b9a0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.586+0000 7fbce667c640 1 -- 172.21.15.81:0/1647531168 shutdown_connections 2023-11-08T16:42:04.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.586+0000 7fbce667c640 1 --2- 172.21.15.81:0/1647531168 >> v2:172.21.15.93:3300/0 conn(0x7fbce00b3550 0x7fbce00b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.590 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.586+0000 7fbce667c640 1 --2- 172.21.15.81:0/1647531168 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbce00b3170 0x7fbce015b500 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.586+0000 7fbce667c640 1 -- 172.21.15.81:0/1647531168 >> 172.21.15.81:0/1647531168 conn(0x7fbce006a6a0 msgr2=0x7fbce00b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:04.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.586+0000 7fbce667c640 1 -- 172.21.15.81:0/1647531168 shutdown_connections 2023-11-08T16:42:04.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.586+0000 7fbce667c640 1 -- 172.21.15.81:0/1647531168 wait complete. 2023-11-08T16:42:04.591 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.588+0000 7fbce667c640 1 Processor -- start 2023-11-08T16:42:04.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.588+0000 7fbce667c640 1 -- start start 2023-11-08T16:42:04.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce667c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbce006c9f0 0x7fbce0140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce667c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbce00b3170 0x7fbce00c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce667c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbce00b3550 0x7fbce015ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.594 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce667c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c3cf0 con 0x7fbce00b3550 2023-11-08T16:42:04.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce667c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c3f40 con 0x7fbce00b3170 2023-11-08T16:42:04.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce667c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce00c4220 con 0x7fbce006c9f0 2023-11-08T16:42:04.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce4c19640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbce00b3550 0x7fbce015ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:04.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce4c19640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbce00b3550 0x7fbce015ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58660/0 (socket says 172.21.15.81:58660) 2023-11-08T16:42:04.595 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce4c19640 1 -- 172.21.15.81:0/1284478860 learned_addr learned my addr 172.21.15.81:0/1284478860 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:04.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbcdffff640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbce006c9f0 0x7fbce0140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:04.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce4c19640 1 -- 172.21.15.81:0/1284478860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbce006c9f0 msgr2=0x7fbce0140b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:04.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce4c19640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbce006c9f0 0x7fbce0140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce4c19640 1 -- 172.21.15.81:0/1284478860 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbce00b3170 msgr2=0x7fbce00c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:04.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce4c19640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbce00b3170 0x7fbce00c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.589+0000 7fbce4c19640 1 -- 172.21.15.81:0/1284478860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd00094d0 con 0x7fbce00b3550 2023-11-08T16:42:04.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.590+0000 7fbce4c19640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbce00b3550 0x7fbce015ea80 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c6b0 tx=0x7fbcd800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:04.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.590+0000 7fbcdd7fa640 1 -- 172.21.15.81:0/1284478860 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8003b70 con 0x7fbce00b3550 2023-11-08T16:42:04.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.590+0000 7fbcdd7fa640 1 -- 172.21.15.81:0/1284478860 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcd8003cf0 con 0x7fbce00b3550 2023-11-08T16:42:04.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.590+0000 7fbcdd7fa640 1 -- 172.21.15.81:0/1284478860 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcd8003ea0 con 0x7fbce00b3550 2023-11-08T16:42:04.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.590+0000 7fbce667c640 1 -- 172.21.15.81:0/1284478860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbce00c4500 con 0x7fbce00b3550 2023-11-08T16:42:04.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.590+0000 7fbce667c640 1 -- 172.21.15.81:0/1284478860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbce0141040 con 0x7fbce00b3550 2023-11-08T16:42:04.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.591+0000 7fbce667c640 1 -- 172.21.15.81:0/1284478860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbce00c4a20 con 0x7fbce00b3550 2023-11-08T16:42:04.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.592+0000 7fbcdd7fa640 1 -- 172.21.15.81:0/1284478860 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbcd8005310 con 0x7fbce00b3550 2023-11-08T16:42:04.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.593+0000 7fbcdd7fa640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcc006ae70 0x7fbcc006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:04.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.593+0000 7fbcdd7fa640 1 -- 172.21.15.81:0/1284478860 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fbcd80894b0 con 0x7fbce00b3550 2023-11-08T16:42:04.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.593+0000 7fbcdffff640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcc006ae70 0x7fbcc006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:04.599 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.594+0000 7fbcdffff640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcc006ae70 0x7fbcc006d330 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0036000 tx=0x7fbcd003b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:04.600 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.598+0000 7fbcdd7fa640 1 -- 172.21.15.81:0/1284478860 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbce00c4a20 con 0x7fbce00b3550 2023-11-08T16:42:04.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.788+0000 7fbce667c640 1 -- 172.21.15.81:0/1284478860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbce00c4a20 con 0x7fbce00b3550 2023-11-08T16:42:04.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.791+0000 7fbcdd7fa640 1 -- 172.21.15.81:0/1284478860 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16878 (secure 0 0 0) 0x7fbce00c4a20 con 0x7fbce00b3550 2023-11-08T16:42:04.793 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:04.794 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":131,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:49.127728+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:41:03.545565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T16:41:04.726492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":2}]},{"pgid":"17.24","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/296230272":"2023-11-09T16:41:49.127674+0000","172.21.15.81:6836/296230272":"2023-11-09T16:41:49.127674+0000","172.21.15.93:6835/4021510722":"2023-11-09T16:41:29.470203+0000","172.21.15.93:6833/4021510722":"2023-11-09T16:41:29.470203+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:42:04.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 -- 172.21.15.81:0/1284478860 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcc006ae70 msgr2=0x7fbcc006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:04.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcc006ae70 0x7fbcc006d330 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0036000 tx=0x7fbcd003b040 comp rx=0 tx=0).stop 2023-11-08T16:42:04.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 -- 172.21.15.81:0/1284478860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbce00b3550 msgr2=0x7fbce015ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:04.796 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbce00b3550 0x7fbce015ea80 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fbcd800c6b0 tx=0x7fbcd800cb70 comp rx=0 tx=0).stop 2023-11-08T16:42:04.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 -- 172.21.15.81:0/1284478860 shutdown_connections 2023-11-08T16:42:04.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbcc006ae70 0x7fbcc006d330 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbce00b3550 0x7fbce015ea80 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbce00b3170 0x7fbce00c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.797 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 --2- 172.21.15.81:0/1284478860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbce006c9f0 0x7fbce0140b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:04.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 -- 172.21.15.81:0/1284478860 >> 172.21.15.81:0/1284478860 conn(0x7fbce006a6a0 msgr2=0x7fbce00ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:04.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 -- 172.21.15.81:0/1284478860 shutdown_connections 2023-11-08T16:42:04.798 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:04.794+0000 7fbce667c640 1 -- 172.21.15.81:0/1284478860 wait complete. 2023-11-08T16:42:04.813 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-08T16:28:52.620854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-08T16:41:03.545565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-11-08T16:41:04.726492+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '127', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-08T16:42:04.814 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-08T16:42:05.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.023+0000 7fa896e32640 1 Processor -- start 2023-11-08T16:42:05.025 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.023+0000 7fa896e32640 1 -- start start 2023-11-08T16:42:05.026 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.024+0000 7fa896e32640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa890071ad0 0x7fa89015ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.025+0000 7fa896e32640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa89015f1b0 0x7fa8900c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.025+0000 7fa896e32640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89015fad0 con 0x7fa890158c60 2023-11-08T16:42:05.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.025+0000 7fa896e32640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89006df80 con 0x7fa890071ad0 2023-11-08T16:42:05.027 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.025+0000 7fa896e32640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89015f620 con 0x7fa89015f1b0 2023-11-08T16:42:05.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.025+0000 7fa887fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fa890071ad0 0x7fa89015ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.025+0000 7fa894bce640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa890158c60 0x7fa8900705d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45136/0 (socket says 172.21.15.81:45136) 2023-11-08T16:42:05.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.025+0000 7fa894bce640 1 -- 172.21.15.81:0/3697033131 learned_addr learned my addr 172.21.15.81:0/3697033131 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:05.028 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.025+0000 7fa8953cf640 1 --2- 172.21.15.81:0/3697033131 >> v2:172.21.15.93:3300/0 conn(0x7fa89015f1b0 0x7fa8900c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.026+0000 7fa887fff640 1 -- 172.21.15.81:0/3697033131 >> v2:172.21.15.93:3300/0 conn(0x7fa89015f1b0 msgr2=0x7fa8900c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:05.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.026+0000 7fa887fff640 1 --2- 172.21.15.81:0/3697033131 >> v2:172.21.15.93:3300/0 conn(0x7fa89015f1b0 0x7fa8900c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.030 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.026+0000 7fa887fff640 1 -- 172.21.15.81:0/3697033131 >> v1:172.21.15.81:6789/0 conn(0x7fa890158c60 legacy=0x7fa8900705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.026+0000 7fa8877fe640 1 -- 172.21.15.81:0/3697033131 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 606659942 0 0) 0x7fa89015fad0 con 0x7fa890158c60 2023-11-08T16:42:05.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.026+0000 7fa887fff640 1 -- 172.21.15.81:0/3697033131 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89006d760 con 0x7fa890071ad0 2023-11-08T16:42:05.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.027+0000 7fa887fff640 1 --2- 172.21.15.81:0/3697033131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa890071ad0 0x7fa89015ec70 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa8900b4ce0 tx=0x7fa88002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e1776ba149d0b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:05.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.027+0000 7fa8877fe640 1 -- 172.21.15.81:0/3697033131 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88002e070 con 0x7fa890071ad0 2023-11-08T16:42:05.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.027+0000 7fa8877fe640 1 -- 172.21.15.81:0/3697033131 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8800315a0 con 0x7fa890071ad0 2023-11-08T16:42:05.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.027+0000 7fa8877fe640 1 -- 172.21.15.81:0/3697033131 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880031930 con 0x7fa890071ad0 2023-11-08T16:42:05.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.028+0000 7fa896e32640 1 -- 172.21.15.81:0/3697033131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa890071ad0 msgr2=0x7fa89015ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.028+0000 7fa896e32640 1 --2- 172.21.15.81:0/3697033131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa890071ad0 0x7fa89015ec70 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa8900b4ce0 tx=0x7fa88002b840 comp rx=0 tx=0).stop 2023-11-08T16:42:05.032 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.028+0000 7fa896e32640 1 -- 172.21.15.81:0/3697033131 shutdown_connections 2023-11-08T16:42:05.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.028+0000 7fa896e32640 1 --2- 172.21.15.81:0/3697033131 >> v2:172.21.15.93:3300/0 conn(0x7fa89015f1b0 0x7fa8900c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.028+0000 7fa896e32640 1 --2- 172.21.15.81:0/3697033131 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa890071ad0 0x7fa89015ec70 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.028+0000 7fa896e32640 1 -- 172.21.15.81:0/3697033131 >> 172.21.15.81:0/3697033131 conn(0x7fa8900ceb70 msgr2=0x7fa8900cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:05.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.028+0000 7fa896e32640 1 -- 172.21.15.81:0/3697033131 shutdown_connections 2023-11-08T16:42:05.033 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.028+0000 7fa896e32640 1 -- 172.21.15.81:0/3697033131 wait complete. 2023-11-08T16:42:05.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.029+0000 7fa896e32640 1 Processor -- start 2023-11-08T16:42:05.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.030+0000 7fa896e32640 1 -- start start 2023-11-08T16:42:05.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa896e32640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa890071ad0 0x7fa8900adeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa896e32640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa890158c60 0x7fa89015c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa896e32640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89015f1b0 0x7fa8900cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa896e32640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900ad230 con 0x7fa89015f1b0 2023-11-08T16:42:05.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa896e32640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89015a740 con 0x7fa890071ad0 2023-11-08T16:42:05.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa896e32640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900ad480 con 0x7fa890158c60 2023-11-08T16:42:05.037 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa8953cf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89015f1b0 0x7fa8900cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa8953cf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89015f1b0 0x7fa8900cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58682/0 (socket says 172.21.15.81:58682) 2023-11-08T16:42:05.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa8953cf640 1 -- 172.21.15.81:0/1821391530 learned_addr learned my addr 172.21.15.81:0/1821391530 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:05.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa894bce640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa890071ad0 0x7fa8900adeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.038 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa887fff640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa890158c60 0x7fa89015c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa8953cf640 1 -- 172.21.15.81:0/1821391530 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa890158c60 msgr2=0x7fa89015c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa8953cf640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa890158c60 0x7fa89015c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa8953cf640 1 -- 172.21.15.81:0/1821391530 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa890071ad0 msgr2=0x7fa8900adeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa8953cf640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa890071ad0 0x7fa8900adeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.039 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.031+0000 7fa8953cf640 1 -- 172.21.15.81:0/1821391530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa880009400 con 0x7fa89015f1b0 2023-11-08T16:42:05.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.032+0000 7fa8953cf640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89015f1b0 0x7fa8900cc740 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa888007e40 tx=0x7fa88800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:05.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.032+0000 7fa8857fa640 1 -- 172.21.15.81:0/1821391530 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888014620 con 0x7fa89015f1b0 2023-11-08T16:42:05.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.032+0000 7fa8857fa640 1 -- 172.21.15.81:0/1821391530 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88800ee90 con 0x7fa89015f1b0 2023-11-08T16:42:05.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.032+0000 7fa8857fa640 1 -- 172.21.15.81:0/1821391530 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8880149e0 con 0x7fa89015f1b0 2023-11-08T16:42:05.040 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.032+0000 7fa896e32640 1 -- 172.21.15.81:0/1821391530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8900ad760 con 0x7fa89015f1b0 2023-11-08T16:42:05.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.032+0000 7fa896e32640 1 -- 172.21.15.81:0/1821391530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89015bba0 con 0x7fa89015f1b0 2023-11-08T16:42:05.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.033+0000 7fa896e32640 1 -- 172.21.15.81:0/1821391530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89015a230 con 0x7fa89015f1b0 2023-11-08T16:42:05.041 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.034+0000 7fa8857fa640 1 -- 172.21.15.81:0/1821391530 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa888014b40 con 0x7fa89015f1b0 2023-11-08T16:42:05.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.036+0000 7fa8857fa640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa87c06ae70 0x7fa87c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.036+0000 7fa8857fa640 1 -- 172.21.15.81:0/1821391530 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fa88808a130 con 0x7fa89015f1b0 2023-11-08T16:42:05.042 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.036+0000 7fa894bce640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa87c06ae70 0x7fa87c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.037+0000 7fa894bce640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa87c06ae70 0x7fa87c06d330 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa88c000f60 tx=0x7fa88c007b70 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:05.043 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.040+0000 7fa8857fa640 1 -- 172.21.15.81:0/1821391530 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa89015a230 con 0x7fa89015f1b0 2023-11-08T16:42:05.232 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.231+0000 7fa896e32640 1 -- 172.21.15.81:0/1821391530 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa89015a230 con 0x7fa89015f1b0 2023-11-08T16:42:05.233 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.232+0000 7fa8857fa640 1 -- 172.21.15.81:0/1821391530 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa89015a230 con 0x7fa89015f1b0 2023-11-08T16:42:05.233 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 1 2023-11-08T16:42:05.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 -- 172.21.15.81:0/1821391530 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa87c06ae70 msgr2=0x7fa87c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa87c06ae70 0x7fa87c06d330 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fa88c000f60 tx=0x7fa88c007b70 comp rx=0 tx=0).stop 2023-11-08T16:42:05.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 -- 172.21.15.81:0/1821391530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89015f1b0 msgr2=0x7fa8900cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.237 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89015f1b0 0x7fa8900cc740 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa888007e40 tx=0x7fa88800e4b0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 -- 172.21.15.81:0/1821391530 shutdown_connections 2023-11-08T16:42:05.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa87c06ae70 0x7fa87c06d330 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa89015f1b0 0x7fa8900cc740 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa890158c60 0x7fa89015c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 --2- 172.21.15.81:0/1821391530 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa890071ad0 0x7fa8900adeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 -- 172.21.15.81:0/1821391530 >> 172.21.15.81:0/1821391530 conn(0x7fa8900ceb70 msgr2=0x7fa8900b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:05.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 -- 172.21.15.81:0/1821391530 shutdown_connections 2023-11-08T16:42:05.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.235+0000 7fa896e32640 1 -- 172.21.15.81:0/1821391530 wait complete. 2023-11-08T16:42:05.255 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-08T16:42:05.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.465+0000 7f7901ce5640 1 Processor -- start 2023-11-08T16:42:05.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.466+0000 7f7901ce5640 1 -- start start 2023-11-08T16:42:05.472 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.467+0000 7f7901ce5640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f78fc071ad0 0x7f78fc15ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.467+0000 7f7901ce5640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc15fad0 con 0x7f78fc15f1b0 2023-11-08T16:42:05.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.467+0000 7f7901ce5640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc06d760 con 0x7f78fc158c60 2023-11-08T16:42:05.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.467+0000 7f7901ce5640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc06df80 con 0x7f78fc071ad0 2023-11-08T16:42:05.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.467+0000 7f78fbfff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f78fc15f1b0 0x7f78fc0c91c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45144/0 (socket says 172.21.15.81:45144) 2023-11-08T16:42:05.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.467+0000 7f78fbfff640 1 -- 172.21.15.81:0/2976744118 learned_addr learned my addr 172.21.15.81:0/2976744118 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:05.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.468+0000 7f78faffd640 1 --2- 172.21.15.81:0/2976744118 >> v2:172.21.15.93:3300/0 conn(0x7f78fc071ad0 0x7f78fc15ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.468+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168921118 0 0) 0x7f78fc15fad0 con 0x7f78fc15f1b0 2023-11-08T16:42:05.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.468+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e8003520 con 0x7f78fc15f1b0 2023-11-08T16:42:05.474 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.468+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3225634647 0 0) 0x7f78e8003520 con 0x7f78fc15f1b0 2023-11-08T16:42:05.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.468+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 >> v2:172.21.15.93:3300/0 conn(0x7f78fc071ad0 msgr2=0x7f78fc15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:05.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.468+0000 7f78fa7fc640 1 --2- 172.21.15.81:0/2976744118 >> v2:172.21.15.93:3300/0 conn(0x7f78fc071ad0 0x7f78fc15ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.468+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 >> v1:172.21.15.93:6789/0 conn(0x7f78fc158c60 legacy=0x7f78fc0705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.475 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.468+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc06da00 con 0x7f78fc15f1b0 2023-11-08T16:42:05.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.468+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f78f4001670 con 0x7f78fc15f1b0 2023-11-08T16:42:05.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.468+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 >> v1:172.21.15.81:6789/0 conn(0x7f78fc15f1b0 legacy=0x7f78fc0c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.468+0000 7f78fa7fc640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78e8003970 0x7f78e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fa7fc640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78e80047c0 0x7f78e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fa7fc640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e8005180 0x7f78e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.476 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc06df80 con 0x7f78e8005180 2023-11-08T16:42:05.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e800c120 con 0x7f78e80047c0 2023-11-08T16:42:05.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e800c2e0 con 0x7f78e8003970 2023-11-08T16:42:05.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fbfff640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e8005180 0x7f78e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78faffd640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78e80047c0 0x7f78e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fbfff640 1 -- 172.21.15.81:0/2976744118 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78e8003970 msgr2=0x7f78e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:05.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fb7fe640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78e8003970 0x7f78e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fbfff640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78e8003970 0x7f78e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fbfff640 1 -- 172.21.15.81:0/2976744118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78e80047c0 msgr2=0x7f78e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:05.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fbfff640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78e80047c0 0x7f78e8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.478 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fbfff640 1 -- 172.21.15.81:0/2976744118 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e8003520 con 0x7f78e8005180 2023-11-08T16:42:05.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78fb7fe640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78e8003970 0x7f78e8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:05.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.469+0000 7f78faffd640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78e80047c0 0x7f78e8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:42:05.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.470+0000 7f78fbfff640 1 -- 172.21.15.81:0/2976744118 reap_dead start 2023-11-08T16:42:05.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.471+0000 7f78fbfff640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e8005180 0x7f78e800b980 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f78f400c190 tx=0x7f78e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2497d90847f47d25 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:05.479 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.471+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f4005ad0 con 0x7f78e8005180 2023-11-08T16:42:05.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.471+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f4005c70 con 0x7f78e8005180 2023-11-08T16:42:05.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.471+0000 7f78fa7fc640 1 -- 172.21.15.81:0/2976744118 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f402c8b0 con 0x7f78e8005180 2023-11-08T16:42:05.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.471+0000 7f7901ce5640 1 -- 172.21.15.81:0/2976744118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e8005180 msgr2=0x7f78e800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.471+0000 7f7901ce5640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e8005180 0x7f78e800b980 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f78f400c190 tx=0x7f78e80057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.472+0000 7f7901ce5640 1 -- 172.21.15.81:0/2976744118 shutdown_connections 2023-11-08T16:42:05.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.472+0000 7f7901ce5640 1 --2- 172.21.15.81:0/2976744118 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78e8005180 0x7f78e800b980 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.472+0000 7f7901ce5640 1 -- 172.21.15.81:0/2976744118 >> 172.21.15.81:0/2976744118 conn(0x7f78fc0ceb70 msgr2=0x7f78fc0cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:05.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.472+0000 7f7901ce5640 1 -- 172.21.15.81:0/2976744118 shutdown_connections 2023-11-08T16:42:05.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.472+0000 7f7901ce5640 1 -- 172.21.15.81:0/2976744118 wait complete. 2023-11-08T16:42:05.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.473+0000 7f7901ce5640 1 Processor -- start 2023-11-08T16:42:05.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.474+0000 7f7901ce5640 1 -- start start 2023-11-08T16:42:05.484 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.474+0000 7f7901ce5640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78e8005180 0x7f78fc14de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.474+0000 7f7901ce5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78fc14e5d0 0x7f78fc1f4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f7901ce5640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78fc0b3300 0x7f78fc0b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f7901ce5640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0b3cc0 con 0x7f78fc14e5d0 2023-11-08T16:42:05.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f7901ce5640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0b0030 con 0x7f78fc0b3300 2023-11-08T16:42:05.485 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f7901ce5640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc14e3a0 con 0x7f78e8005180 2023-11-08T16:42:05.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f78faffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78fc14e5d0 0x7f78fc1f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f78faffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78fc14e5d0 0x7f78fc1f4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58712/0 (socket says 172.21.15.81:58712) 2023-11-08T16:42:05.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f78faffd640 1 -- 172.21.15.81:0/609616615 learned_addr learned my addr 172.21.15.81:0/609616615 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:05.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f78fb7fe640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78e8005180 0x7f78fc14de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.486 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f78faffd640 1 -- 172.21.15.81:0/609616615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78e8005180 msgr2=0x7f78fc14de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f78faffd640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78e8005180 0x7f78fc14de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f78faffd640 1 -- 172.21.15.81:0/609616615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78fc0b3300 msgr2=0x7f78fc0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f78faffd640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78fc0b3300 0x7f78fc0b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f78faffd640 1 -- 172.21.15.81:0/609616615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc1f4cc0 con 0x7f78fc14e5d0 2023-11-08T16:42:05.487 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.475+0000 7f78fb7fe640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78e8005180 0x7f78fc14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:05.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.476+0000 7f78faffd640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78fc14e5d0 0x7f78fc1f4780 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f78e4011540 tx=0x7f78e4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:05.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.476+0000 7f78f3fff640 1 -- 172.21.15.81:0/609616615 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4015870 con 0x7f78fc14e5d0 2023-11-08T16:42:05.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.476+0000 7f78f3fff640 1 -- 172.21.15.81:0/609616615 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e40159d0 con 0x7f78fc14e5d0 2023-11-08T16:42:05.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.476+0000 7f7901ce5640 1 -- 172.21.15.81:0/609616615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc0b02d0 con 0x7f78fc14e5d0 2023-11-08T16:42:05.488 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.476+0000 7f78f3fff640 1 -- 172.21.15.81:0/609616615 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4015d40 con 0x7f78fc14e5d0 2023-11-08T16:42:05.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.476+0000 7f7901ce5640 1 -- 172.21.15.81:0/609616615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc152160 con 0x7f78fc14e5d0 2023-11-08T16:42:05.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.477+0000 7f7901ce5640 1 -- 172.21.15.81:0/609616615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc074440 con 0x7f78fc14e5d0 2023-11-08T16:42:05.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.478+0000 7f78f3fff640 1 -- 172.21.15.81:0/609616615 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78e401d400 con 0x7f78fc14e5d0 2023-11-08T16:42:05.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.479+0000 7f78f3fff640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78dc06aea0 0x7f78dc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.479+0000 7f78f3fff640 1 -- 172.21.15.81:0/609616615 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f78e4089e10 con 0x7f78fc14e5d0 2023-11-08T16:42:05.489 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.479+0000 7f78fb7fe640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78dc06aea0 0x7f78dc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.480+0000 7f78fb7fe640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78dc06aea0 0x7f78dc06d360 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f78ec001d40 tx=0x7f78ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:05.490 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.483+0000 7f78f3fff640 1 -- 172.21.15.81:0/609616615 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f78e4054dd0 con 0x7f78fc14e5d0 2023-11-08T16:42:05.676 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.675+0000 7f7901ce5640 1 -- 172.21.15.81:0/609616615 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78fc15dcb0 con 0x7f78fc14e5d0 2023-11-08T16:42:05.677 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.676+0000 7f78f3fff640 1 -- 172.21.15.81:0/609616615 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v131) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78e40583f0 con 0x7f78fc14e5d0 2023-11-08T16:42:05.677 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 32 2023-11-08T16:42:05.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.678+0000 7f7901ce5640 1 -- 172.21.15.81:0/609616615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78dc06aea0 msgr2=0x7f78dc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.678+0000 7f7901ce5640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78dc06aea0 0x7f78dc06d360 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f78ec001d40 tx=0x7f78ec009040 comp rx=0 tx=0).stop 2023-11-08T16:42:05.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.678+0000 7f7901ce5640 1 -- 172.21.15.81:0/609616615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78fc14e5d0 msgr2=0x7f78fc1f4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.678+0000 7f7901ce5640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78fc14e5d0 0x7f78fc1f4780 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f78e4011540 tx=0x7f78e4011a00 comp rx=0 tx=0).stop 2023-11-08T16:42:05.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.678+0000 7f7901ce5640 1 -- 172.21.15.81:0/609616615 shutdown_connections 2023-11-08T16:42:05.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.678+0000 7f7901ce5640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f78dc06aea0 0x7f78dc06d360 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.678+0000 7f7901ce5640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78fc0b3300 0x7f78fc0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.678+0000 7f7901ce5640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f78fc14e5d0 0x7f78fc1f4780 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.678+0000 7f7901ce5640 1 --2- 172.21.15.81:0/609616615 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f78e8005180 0x7f78fc14de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.678+0000 7f7901ce5640 1 -- 172.21.15.81:0/609616615 >> 172.21.15.81:0/609616615 conn(0x7f78fc0ceb70 msgr2=0x7f78fc0c9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.679+0000 7f7901ce5640 1 -- 172.21.15.81:0/609616615 shutdown_connections 2023-11-08T16:42:05.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.679+0000 7f7901ce5640 1 -- 172.21.15.81:0/609616615 wait complete. 2023-11-08T16:42:05.698 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-08T16:42:05.910 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.909+0000 7f5952a4f640 1 Processor -- start 2023-11-08T16:42:05.911 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.909+0000 7f5952a4f640 1 -- start start 2023-11-08T16:42:05.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.911+0000 7f5952a4f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f594c0b6340 0x7f594c0cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.911+0000 7f5952a4f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f594c155b00 con 0x7f594c0b2f00 2023-11-08T16:42:05.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.911+0000 7f5952a4f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f594c15b930 con 0x7f594c0ccc90 2023-11-08T16:42:05.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.911+0000 7f5952a4f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f594c15bc10 con 0x7f594c0b6340 2023-11-08T16:42:05.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.911+0000 7f5950fec640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f594c0b2f00 0x7f594c15ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45158/0 (socket says 172.21.15.81:45158) 2023-11-08T16:42:05.913 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.911+0000 7f5950fec640 1 -- 172.21.15.81:0/2908419676 learned_addr learned my addr 172.21.15.81:0/2908419676 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:05.916 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.911+0000 7f594bfff640 1 --2- 172.21.15.81:0/2908419676 >> v2:172.21.15.93:3300/0 conn(0x7f594c0b6340 0x7f594c0cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.911+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346896753 0 0) 0x7f5944001490 con 0x7f594c0b2f00 2023-11-08T16:42:05.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5934003520 con 0x7f594c0b2f00 2023-11-08T16:42:05.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1184139463 0 0) 0x7f5934003520 con 0x7f594c0b2f00 2023-11-08T16:42:05.917 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 >> v2:172.21.15.93:3300/0 conn(0x7f594c0b6340 msgr2=0x7f594c0cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:05.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 --2- 172.21.15.81:0/2908419676 >> v2:172.21.15.93:3300/0 conn(0x7f594c0b6340 0x7f594c0cb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 >> v1:172.21.15.93:6789/0 conn(0x7f594c0ccc90 legacy=0x7f594c0b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.918 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c15bef0 con 0x7f594c0b2f00 2023-11-08T16:42:05.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f5944001b80 con 0x7f594c0b2f00 2023-11-08T16:42:05.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 >> v1:172.21.15.81:6789/0 conn(0x7f594c0b2f00 legacy=0x7f594c15ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59340039c0 0x7f5934003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59340047c0 0x7f5934004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59340051d0 0x7f593400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c15bc10 con 0x7f59340051d0 2023-11-08T16:42:05.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f593400c120 con 0x7f59340039c0 2023-11-08T16:42:05.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f593400c2e0 con 0x7f59340047c0 2023-11-08T16:42:05.920 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.912+0000 7f5950fec640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59340051d0 0x7f593400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.913+0000 7f594bfff640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59340039c0 0x7f5934003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.913+0000 7f594b7fe640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59340047c0 0x7f5934004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.913+0000 7f5950fec640 1 -- 172.21.15.81:0/2908419676 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59340047c0 msgr2=0x7f5934004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:05.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.913+0000 7f5950fec640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59340047c0 0x7f5934004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.921 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.913+0000 7f5950fec640 1 -- 172.21.15.81:0/2908419676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59340039c0 msgr2=0x7f5934003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:05.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.913+0000 7f5950fec640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59340039c0 0x7f5934003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.913+0000 7f5950fec640 1 -- 172.21.15.81:0/2908419676 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5934003520 con 0x7f59340051d0 2023-11-08T16:42:05.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.913+0000 7f594b7fe640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59340047c0 0x7f5934004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:05.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.914+0000 7f5950fec640 1 -- 172.21.15.81:0/2908419676 reap_dead start 2023-11-08T16:42:05.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.914+0000 7f5950fec640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59340051d0 0x7f593400b980 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5944002230 tx=0x7f59340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c274785fd0b4f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:05.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.914+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440072d0 con 0x7f59340051d0 2023-11-08T16:42:05.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.914+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594402f070 con 0x7f59340051d0 2023-11-08T16:42:05.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.914+0000 7f594affd640 1 -- 172.21.15.81:0/2908419676 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944007670 con 0x7f59340051d0 2023-11-08T16:42:05.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.915+0000 7f5952a4f640 1 -- 172.21.15.81:0/2908419676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59340051d0 msgr2=0x7f593400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.915+0000 7f5952a4f640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59340051d0 0x7f593400b980 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5944002230 tx=0x7f59340057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.915+0000 7f5952a4f640 1 -- 172.21.15.81:0/2908419676 shutdown_connections 2023-11-08T16:42:05.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.915+0000 7f5952a4f640 1 --2- 172.21.15.81:0/2908419676 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f59340051d0 0x7f593400b980 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.915+0000 7f5952a4f640 1 -- 172.21.15.81:0/2908419676 >> 172.21.15.81:0/2908419676 conn(0x7f594c06dde0 msgr2=0x7f594c06e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:05.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.915+0000 7f5952a4f640 1 -- 172.21.15.81:0/2908419676 shutdown_connections 2023-11-08T16:42:05.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.915+0000 7f5952a4f640 1 -- 172.21.15.81:0/2908419676 wait complete. 2023-11-08T16:42:05.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.917+0000 7f5952a4f640 1 Processor -- start 2023-11-08T16:42:05.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.917+0000 7f5952a4f640 1 -- start start 2023-11-08T16:42:05.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.918+0000 7f5952a4f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59340051d0 0x7f594c0c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.918+0000 7f5952a4f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f594c0ca640 0x7f594c153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.918+0000 7f5952a4f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f594c1f2a70 0x7f594c15a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.918+0000 7f5952a4f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c06cf20 con 0x7f594c1f2a70 2023-11-08T16:42:05.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.918+0000 7f5952a4f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f594c0c5210 con 0x7f594c0ca640 2023-11-08T16:42:05.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.918+0000 7f5952a4f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f594c06c960 con 0x7f59340051d0 2023-11-08T16:42:05.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.918+0000 7f5950fec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f594c1f2a70 0x7f594c15a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.918+0000 7f5950fec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f594c1f2a70 0x7f594c15a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58728/0 (socket says 172.21.15.81:58728) 2023-11-08T16:42:05.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.918+0000 7f5950fec640 1 -- 172.21.15.81:0/3927151058 learned_addr learned my addr 172.21.15.81:0/3927151058 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:05.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f594bfff640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59340051d0 0x7f594c0c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f594b7fe640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f594c0ca640 0x7f594c153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f5950fec640 1 -- 172.21.15.81:0/3927151058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59340051d0 msgr2=0x7f594c0c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f5950fec640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59340051d0 0x7f594c0c5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f5950fec640 1 -- 172.21.15.81:0/3927151058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f594c0ca640 msgr2=0x7f594c153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:05.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f5950fec640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f594c0ca640 0x7f594c153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:05.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f5950fec640 1 -- 172.21.15.81:0/3927151058 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f594c06cc40 con 0x7f594c1f2a70 2023-11-08T16:42:05.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f594b7fe640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f594c0ca640 0x7f594c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:05.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f5950fec640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f594c1f2a70 0x7f594c15a0b0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5944000f80 tx=0x7f5944001e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:05.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f5948ff9640 1 -- 172.21.15.81:0/3927151058 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5944004b60 con 0x7f594c1f2a70 2023-11-08T16:42:05.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f5948ff9640 1 -- 172.21.15.81:0/3927151058 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5944004ce0 con 0x7f594c1f2a70 2023-11-08T16:42:05.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f5952a4f640 1 -- 172.21.15.81:0/3927151058 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f594c06b290 con 0x7f594c1f2a70 2023-11-08T16:42:05.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.919+0000 7f5948ff9640 1 -- 172.21.15.81:0/3927151058 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594403e6a0 con 0x7f594c1f2a70 2023-11-08T16:42:05.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.920+0000 7f5952a4f640 1 -- 172.21.15.81:0/3927151058 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f594c1544a0 con 0x7f594c1f2a70 2023-11-08T16:42:05.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.921+0000 7f5952a4f640 1 -- 172.21.15.81:0/3927151058 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f594c074440 con 0x7f594c1f2a70 2023-11-08T16:42:05.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.922+0000 7f5948ff9640 1 -- 172.21.15.81:0/3927151058 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5944031030 con 0x7f594c1f2a70 2023-11-08T16:42:05.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.923+0000 7f5948ff9640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f593806ae70 0x7f593806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:05.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.923+0000 7f5948ff9640 1 -- 172.21.15.81:0/3927151058 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f59440aa780 con 0x7f594c1f2a70 2023-11-08T16:42:05.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.923+0000 7f594bfff640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f593806ae70 0x7f593806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:05.933 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.924+0000 7f594bfff640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f593806ae70 0x7f593806d330 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f593c0098d0 tx=0x7f593c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:05.934 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:05.927+0000 7f5948ff9640 1 -- 172.21.15.81:0/3927151058 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5944075730 con 0x7f594c1f2a70 2023-11-08T16:42:06.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.118+0000 7f5952a4f640 1 -- 172.21.15.81:0/3927151058 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f594c155b00 con 0x7f594c1f2a70 2023-11-08T16:42:06.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.118+0000 7f5948ff9640 1 -- 172.21.15.81:0/3927151058 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v131) v1 ==== 100+0+11 (secure 0 0 0) 0x7f594c155b00 con 0x7f594c1f2a70 2023-11-08T16:42:06.120 INFO:teuthology.orchestra.run.smithi081.stdout:pg_num: 64 2023-11-08T16:42:06.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.121+0000 7f5952a4f640 1 -- 172.21.15.81:0/3927151058 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f593806ae70 msgr2=0x7f593806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.123 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.121+0000 7f5952a4f640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f593806ae70 0x7f593806d330 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f593c0098d0 tx=0x7f593c008040 comp rx=0 tx=0).stop 2023-11-08T16:42:06.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.121+0000 7f5952a4f640 1 -- 172.21.15.81:0/3927151058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f594c1f2a70 msgr2=0x7f594c15a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.121+0000 7f5952a4f640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f594c1f2a70 0x7f594c15a0b0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5944000f80 tx=0x7f5944001e40 comp rx=0 tx=0).stop 2023-11-08T16:42:06.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.121+0000 7f5952a4f640 1 -- 172.21.15.81:0/3927151058 shutdown_connections 2023-11-08T16:42:06.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.121+0000 7f5952a4f640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f593806ae70 0x7f593806d330 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.124 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.121+0000 7f5952a4f640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f594c1f2a70 0x7f594c15a0b0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.121+0000 7f5952a4f640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f594c0ca640 0x7f594c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.122+0000 7f5952a4f640 1 --2- 172.21.15.81:0/3927151058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f59340051d0 0x7f594c0c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.122+0000 7f5952a4f640 1 -- 172.21.15.81:0/3927151058 >> 172.21.15.81:0/3927151058 conn(0x7f594c06dde0 msgr2=0x7f594c0b5630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:06.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.122+0000 7f5952a4f640 1 -- 172.21.15.81:0/3927151058 shutdown_connections 2023-11-08T16:42:06.125 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.122+0000 7f5952a4f640 1 -- 172.21.15.81:0/3927151058 wait complete. 2023-11-08T16:42:06.141 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:42:06.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.341+0000 7f7b3620e640 1 Processor -- start 2023-11-08T16:42:06.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.341+0000 7f7b3620e640 1 -- start start 2023-11-08T16:42:06.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.343+0000 7f7b3620e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b30143800 0x7f7b300b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:06.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.343+0000 7f7b3620e640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f7b300cca70 0x7f7b3015d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:06.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.343+0000 7f7b3620e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b300714f0 con 0x7f7b3015df20 2023-11-08T16:42:06.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.343+0000 7f7b3620e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b300b2df0 con 0x7f7b30143800 2023-11-08T16:42:06.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.343+0000 7f7b3620e640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b3015e330 con 0x7f7b300cca70 2023-11-08T16:42:06.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.343+0000 7f7b2f7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b30143800 0x7f7b300b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:06.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.343+0000 7f7b2f7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7b30143800 0x7f7b300b28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58734/0 (socket says 172.21.15.81:58734) 2023-11-08T16:42:06.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.343+0000 7f7b2f7fe640 1 -- 172.21.15.81:0/2590194388 learned_addr learned my addr 172.21.15.81:0/2590194388 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:06.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.343+0000 7f7b2effd640 1 --2- 172.21.15.81:0/2590194388 >> v2:172.21.15.93:3300/0 conn(0x7f7b300cca70 0x7f7b3015d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:06.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.344+0000 7f7b2f7fe640 1 -- 172.21.15.81:0/2590194388 >> v2:172.21.15.93:3300/0 conn(0x7f7b300cca70 msgr2=0x7f7b3015d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:06.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.344+0000 7f7b2f7fe640 1 --2- 172.21.15.81:0/2590194388 >> v2:172.21.15.93:3300/0 conn(0x7f7b300cca70 0x7f7b3015d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.344+0000 7f7b2f7fe640 1 -- 172.21.15.81:0/2590194388 >> v1:172.21.15.81:6789/0 conn(0x7f7b3015df20 legacy=0x7f7b3015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.344+0000 7f7b2f7fe640 1 -- 172.21.15.81:0/2590194388 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b300cd1e0 con 0x7f7b30143800 2023-11-08T16:42:06.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.345+0000 7f7b2f7fe640 1 --2- 172.21.15.81:0/2590194388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b30143800 0x7f7b300b28b0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f7b300cc8c0 tx=0x7f7b2002dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c9ad201fe72917 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:06.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.345+0000 7f7b2e7fc640 1 -- 172.21.15.81:0/2590194388 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2002e960 con 0x7f7b30143800 2023-11-08T16:42:06.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.345+0000 7f7b2e7fc640 1 -- 172.21.15.81:0/2590194388 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2002eac0 con 0x7f7b30143800 2023-11-08T16:42:06.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.345+0000 7f7b2e7fc640 1 -- 172.21.15.81:0/2590194388 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2002ee50 con 0x7f7b30143800 2023-11-08T16:42:06.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.346+0000 7f7b3620e640 1 -- 172.21.15.81:0/2590194388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b30143800 msgr2=0x7f7b300b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.346+0000 7f7b3620e640 1 --2- 172.21.15.81:0/2590194388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b30143800 0x7f7b300b28b0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f7b300cc8c0 tx=0x7f7b2002dbb0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.346+0000 7f7b3620e640 1 -- 172.21.15.81:0/2590194388 shutdown_connections 2023-11-08T16:42:06.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.346+0000 7f7b3620e640 1 --2- 172.21.15.81:0/2590194388 >> v2:172.21.15.93:3300/0 conn(0x7f7b300cca70 0x7f7b3015d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.346+0000 7f7b3620e640 1 --2- 172.21.15.81:0/2590194388 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b30143800 0x7f7b300b28b0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.346+0000 7f7b3620e640 1 -- 172.21.15.81:0/2590194388 >> 172.21.15.81:0/2590194388 conn(0x7f7b3006a6a0 msgr2=0x7f7b300b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:06.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.346+0000 7f7b3620e640 1 -- 172.21.15.81:0/2590194388 shutdown_connections 2023-11-08T16:42:06.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.346+0000 7f7b3620e640 1 -- 172.21.15.81:0/2590194388 wait complete. 2023-11-08T16:42:06.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.348+0000 7f7b3620e640 1 Processor -- start 2023-11-08T16:42:06.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.348+0000 7f7b3620e640 1 -- start start 2023-11-08T16:42:06.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b3620e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b300cca70 0x7f7b300ca6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:06.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b3620e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b30143800 0x7f7b3015ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:06.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b3620e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b3015df20 0x7f7b3015cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:06.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b3620e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3015be80 con 0x7f7b300cca70 2023-11-08T16:42:06.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b3620e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3015b200 con 0x7f7b3015df20 2023-11-08T16:42:06.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b3620e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3015b4e0 con 0x7f7b30143800 2023-11-08T16:42:06.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b2f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b300cca70 0x7f7b300ca6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:06.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b2f7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b300cca70 0x7f7b300ca6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58740/0 (socket says 172.21.15.81:58740) 2023-11-08T16:42:06.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b2f7fe640 1 -- 172.21.15.81:0/3706328503 learned_addr learned my addr 172.21.15.81:0/3706328503 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:06.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b2f7fe640 1 -- 172.21.15.81:0/3706328503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b30143800 msgr2=0x7f7b3015ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b2f7fe640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b30143800 0x7f7b3015ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b2f7fe640 1 -- 172.21.15.81:0/3706328503 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b3015df20 msgr2=0x7f7b3015cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.349+0000 7f7b2f7fe640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b3015df20 0x7f7b3015cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.350+0000 7f7b2f7fe640 1 -- 172.21.15.81:0/3706328503 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b200094d0 con 0x7f7b300cca70 2023-11-08T16:42:06.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.350+0000 7f7b2f7fe640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b300cca70 0x7f7b300ca6f0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f7b2002dbf0 tx=0x7f7b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:06.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.350+0000 7f7b17fff640 1 -- 172.21.15.81:0/3706328503 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20041640 con 0x7f7b300cca70 2023-11-08T16:42:06.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.350+0000 7f7b17fff640 1 -- 172.21.15.81:0/3706328503 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b200417c0 con 0x7f7b300cca70 2023-11-08T16:42:06.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.350+0000 7f7b17fff640 1 -- 172.21.15.81:0/3706328503 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b20041b70 con 0x7f7b300cca70 2023-11-08T16:42:06.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.350+0000 7f7b3620e640 1 -- 172.21.15.81:0/3706328503 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3015b7c0 con 0x7f7b300cca70 2023-11-08T16:42:06.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.350+0000 7f7b3620e640 1 -- 172.21.15.81:0/3706328503 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3015a4d0 con 0x7f7b300cca70 2023-11-08T16:42:06.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.351+0000 7f7b3620e640 1 -- 172.21.15.81:0/3706328503 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3013fe90 con 0x7f7b300cca70 2023-11-08T16:42:06.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.352+0000 7f7b17fff640 1 -- 172.21.15.81:0/3706328503 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7b2003e3f0 con 0x7f7b300cca70 2023-11-08T16:42:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.353+0000 7f7b17fff640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b1c06ae70 0x7f7b1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.353+0000 7f7b17fff640 1 -- 172.21.15.81:0/3706328503 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f7b200aba10 con 0x7f7b300cca70 2023-11-08T16:42:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.354+0000 7f7b2effd640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b1c06ae70 0x7f7b1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:06.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.354+0000 7f7b2effd640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b1c06ae70 0x7f7b1c06d330 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7b30071090 tx=0x7f7b24008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:06.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.357+0000 7f7b17fff640 1 -- 172.21.15.81:0/3706328503 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7b3013fe90 con 0x7f7b300cca70 2023-11-08T16:42:06.592 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.590+0000 7f7b3620e640 1 -- 172.21.15.81:0/3706328503 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b3013fe90 con 0x7f7b300cca70 2023-11-08T16:42:06.593 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.592+0000 7f7b17fff640 1 -- 172.21.15.81:0/3706328503 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4920 (secure 0 0 0) 0x7f7b3013fe90 con 0x7f7b300cca70 2023-11-08T16:42:06.593 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:06.594 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":120,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/329260749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":329260749},{"type":"v1","addr":"172.21.15.93:6835","nonce":329260749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:55.094753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":117,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/1432485913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1432485913},{"type":"v1","addr":"172.21.15.81:6837","nonce":1432485913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:42:06.594 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 120 2023-11-08T16:42:06.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.594+0000 7f7b3620e640 1 -- 172.21.15.81:0/3706328503 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b1c06ae70 msgr2=0x7f7b1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.594+0000 7f7b3620e640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b1c06ae70 0x7f7b1c06d330 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7b30071090 tx=0x7f7b24008040 comp rx=0 tx=0).stop 2023-11-08T16:42:06.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.594+0000 7f7b3620e640 1 -- 172.21.15.81:0/3706328503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b300cca70 msgr2=0x7f7b300ca6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.594+0000 7f7b3620e640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b300cca70 0x7f7b300ca6f0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f7b2002dbf0 tx=0x7f7b200046c0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.594+0000 7f7b3620e640 1 -- 172.21.15.81:0/3706328503 shutdown_connections 2023-11-08T16:42:06.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.594+0000 7f7b3620e640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7b1c06ae70 0x7f7b1c06d330 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.594+0000 7f7b3620e640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b3015df20 0x7f7b3015cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.597 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.594+0000 7f7b3620e640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7b30143800 0x7f7b3015ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.594+0000 7f7b3620e640 1 --2- 172.21.15.81:0/3706328503 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7b300cca70 0x7f7b300ca6f0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.594+0000 7f7b3620e640 1 -- 172.21.15.81:0/3706328503 >> 172.21.15.81:0/3706328503 conn(0x7f7b3006a6a0 msgr2=0x7f7b300ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:06.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.595+0000 7f7b3620e640 1 -- 172.21.15.81:0/3706328503 shutdown_connections 2023-11-08T16:42:06.598 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.595+0000 7f7b3620e640 1 -- 172.21.15.81:0/3706328503 wait complete. 2023-11-08T16:42:06.615 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:42:06.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.822+0000 7f795ac34640 1 Processor -- start 2023-11-08T16:42:06.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.822+0000 7f795ac34640 1 -- start start 2023-11-08T16:42:06.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.823+0000 7f795ac34640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f795415e0f0 0x7f79540b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:06.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.823+0000 7f795ac34640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79540cd1e0 con 0x7f7954143800 2023-11-08T16:42:06.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.824+0000 7f795ac34640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79540cd4c0 con 0x7f79540cca70 2023-11-08T16:42:06.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.824+0000 7f795ac34640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79540b2df0 con 0x7f795415e0f0 2023-11-08T16:42:06.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.824+0000 7f79589d0640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f7954143800 0x7f795415d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45178/0 (socket says 172.21.15.81:45178) 2023-11-08T16:42:06.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.824+0000 7f79589d0640 1 -- 172.21.15.81:0/2394172262 learned_addr learned my addr 172.21.15.81:0/2394172262 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:06.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.824+0000 7f7953fff640 1 --2- 172.21.15.81:0/2394172262 >> v2:172.21.15.81:3300/0 conn(0x7f795415e0f0 0x7f79540b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:06.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.824+0000 7f79537fe640 1 -- 172.21.15.81:0/2394172262 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 577857441 0 0) 0x7f79540cd1e0 con 0x7f7954143800 2023-11-08T16:42:06.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.824+0000 7f79537fe640 1 -- 172.21.15.81:0/2394172262 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7944003520 con 0x7f7954143800 2023-11-08T16:42:06.828 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.824+0000 7f7953fff640 1 -- 172.21.15.81:0/2394172262 >> v1:172.21.15.93:6789/0 conn(0x7f79540cca70 legacy=0x7f79540704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:06.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.824+0000 7f7953fff640 1 -- 172.21.15.81:0/2394172262 >> v1:172.21.15.81:6789/0 conn(0x7f7954143800 legacy=0x7f795415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.824+0000 7f7953fff640 1 -- 172.21.15.81:0/2394172262 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540cd7a0 con 0x7f795415e0f0 2023-11-08T16:42:06.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.826+0000 7f7953fff640 1 --2- 172.21.15.81:0/2394172262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f795415e0f0 0x7f79540b28b0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f79400097a0 tx=0x7f794002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec836aa19dad714 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:06.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.826+0000 7f79537fe640 1 -- 172.21.15.81:0/2394172262 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940031830 con 0x7f795415e0f0 2023-11-08T16:42:06.829 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.826+0000 7f79537fe640 1 -- 172.21.15.81:0/2394172262 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7940031990 con 0x7f795415e0f0 2023-11-08T16:42:06.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.826+0000 7f79537fe640 1 -- 172.21.15.81:0/2394172262 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7940031d20 con 0x7f795415e0f0 2023-11-08T16:42:06.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.826+0000 7f795ac34640 1 -- 172.21.15.81:0/2394172262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f795415e0f0 msgr2=0x7f79540b28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.826+0000 7f795ac34640 1 --2- 172.21.15.81:0/2394172262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f795415e0f0 0x7f79540b28b0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f79400097a0 tx=0x7f794002b8d0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.826+0000 7f795ac34640 1 -- 172.21.15.81:0/2394172262 shutdown_connections 2023-11-08T16:42:06.830 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.826+0000 7f795ac34640 1 --2- 172.21.15.81:0/2394172262 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f795415e0f0 0x7f79540b28b0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.827+0000 7f795ac34640 1 -- 172.21.15.81:0/2394172262 >> 172.21.15.81:0/2394172262 conn(0x7f795406a6a0 msgr2=0x7f79540b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:06.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.827+0000 7f795ac34640 1 -- 172.21.15.81:0/2394172262 shutdown_connections 2023-11-08T16:42:06.831 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.827+0000 7f795ac34640 1 -- 172.21.15.81:0/2394172262 wait complete. 2023-11-08T16:42:06.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.828+0000 7f795ac34640 1 Processor -- start 2023-11-08T16:42:06.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.829+0000 7f795ac34640 1 -- start start 2023-11-08T16:42:06.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f795ac34640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79540cca70 0x7f79541597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:06.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f795ac34640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7954143800 0x7f795415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:06.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f795ac34640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f795415e0f0 0x7f795415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:06.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f795ac34640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79541f2d70 con 0x7f79540cca70 2023-11-08T16:42:06.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f795ac34640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79541ee730 con 0x7f795415e0f0 2023-11-08T16:42:06.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f795ac34640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79541f2a90 con 0x7f7954143800 2023-11-08T16:42:06.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f79589d0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79540cca70 0x7f79541597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:06.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f79589d0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79540cca70 0x7f79541597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58752/0 (socket says 172.21.15.81:58752) 2023-11-08T16:42:06.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f79589d0640 1 -- 172.21.15.81:0/2263758296 learned_addr learned my addr 172.21.15.81:0/2263758296 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:06.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f79589d0640 1 -- 172.21.15.81:0/2263758296 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7954143800 msgr2=0x7f795415b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f79589d0640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7954143800 0x7f795415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f79589d0640 1 -- 172.21.15.81:0/2263758296 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f795415e0f0 msgr2=0x7f795415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:06.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f79589d0640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f795415e0f0 0x7f795415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:06.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.830+0000 7f79589d0640 1 -- 172.21.15.81:0/2263758296 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7940009400 con 0x7f79540cca70 2023-11-08T16:42:06.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.831+0000 7f79589d0640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79540cca70 0x7f79541597b0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f794800c530 tx=0x7f794800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:06.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.831+0000 7f79517fa640 1 -- 172.21.15.81:0/2263758296 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7948005ce0 con 0x7f79540cca70 2023-11-08T16:42:06.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.831+0000 7f79517fa640 1 -- 172.21.15.81:0/2263758296 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7948005e40 con 0x7f79540cca70 2023-11-08T16:42:06.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.831+0000 7f79517fa640 1 -- 172.21.15.81:0/2263758296 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79480072b0 con 0x7f79540cca70 2023-11-08T16:42:06.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.831+0000 7f795ac34640 1 -- 172.21.15.81:0/2263758296 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79541ee9a0 con 0x7f79540cca70 2023-11-08T16:42:06.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.831+0000 7f795ac34640 1 -- 172.21.15.81:0/2263758296 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79541eef60 con 0x7f79540cca70 2023-11-08T16:42:06.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.832+0000 7f795ac34640 1 -- 172.21.15.81:0/2263758296 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79541ef150 con 0x7f79540cca70 2023-11-08T16:42:06.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.833+0000 7f79517fa640 1 -- 172.21.15.81:0/2263758296 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7948007410 con 0x7f79540cca70 2023-11-08T16:42:06.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.834+0000 7f79517fa640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f793806ae70 0x7f793806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:06.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.834+0000 7f79517fa640 1 -- 172.21.15.81:0/2263758296 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f7948088d50 con 0x7f79540cca70 2023-11-08T16:42:06.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.834+0000 7f7953fff640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f793806ae70 0x7f793806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:06.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.835+0000 7f7953fff640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f793806ae70 0x7f793806d330 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7940036000 tx=0x7f7940030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:06.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:06.838+0000 7f79517fa640 1 -- 172.21.15.81:0/2263758296 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7948057850 con 0x7f79540cca70 2023-11-08T16:42:07.029 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.028+0000 7f795ac34640 1 -- 172.21.15.81:0/2263758296 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7954074440 con 0x7f79540cca70 2023-11-08T16:42:07.031 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.030+0000 7f79517fa640 1 -- 172.21.15.81:0/2263758296 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16878 (secure 0 0 0) 0x7f7948017020 con 0x7f79540cca70 2023-11-08T16:42:07.032 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:07.032 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":131,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:49.127728+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:41:03.545565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T16:41:04.726492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":2}]},{"pgid":"17.24","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/296230272":"2023-11-09T16:41:49.127674+0000","172.21.15.81:6836/296230272":"2023-11-09T16:41:49.127674+0000","172.21.15.93:6835/4021510722":"2023-11-09T16:41:29.470203+0000","172.21.15.93:6833/4021510722":"2023-11-09T16:41:29.470203+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:42:07.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.032+0000 7f795ac34640 1 -- 172.21.15.81:0/2263758296 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f793806ae70 msgr2=0x7f793806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.034 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.032+0000 7f795ac34640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f793806ae70 0x7f793806d330 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7940036000 tx=0x7f7940030000 comp rx=0 tx=0).stop 2023-11-08T16:42:07.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.032+0000 7f795ac34640 1 -- 172.21.15.81:0/2263758296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79540cca70 msgr2=0x7f79541597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.032+0000 7f795ac34640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79540cca70 0x7f79541597b0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7f794800c530 tx=0x7f794800c9f0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.033+0000 7f795ac34640 1 -- 172.21.15.81:0/2263758296 shutdown_connections 2023-11-08T16:42:07.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.033+0000 7f795ac34640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f793806ae70 0x7f793806d330 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.035 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.033+0000 7f795ac34640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f795415e0f0 0x7f795415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.033+0000 7f795ac34640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7954143800 0x7f795415b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.033+0000 7f795ac34640 1 --2- 172.21.15.81:0/2263758296 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f79540cca70 0x7f79541597b0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.033+0000 7f795ac34640 1 -- 172.21.15.81:0/2263758296 >> 172.21.15.81:0/2263758296 conn(0x7f795406a6a0 msgr2=0x7f795413de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:07.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.033+0000 7f795ac34640 1 -- 172.21.15.81:0/2263758296 shutdown_connections 2023-11-08T16:42:07.036 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.033+0000 7f795ac34640 1 -- 172.21.15.81:0/2263758296 wait complete. 2023-11-08T16:42:07.052 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T16:42:07.053 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:42:07.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.266+0000 7f7f76b4c640 1 Processor -- start 2023-11-08T16:42:07.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.266+0000 7f7f76b4c640 1 -- start start 2023-11-08T16:42:07.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.267+0000 7f7f76b4c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7f700d0850 0x7f7f7015f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.267+0000 7f7f76b4c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f700d0450 con 0x7f7f700b3010 2023-11-08T16:42:07.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.267+0000 7f7f76b4c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f700cb440 con 0x7f7f7015b220 2023-11-08T16:42:07.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.267+0000 7f7f76b4c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f700cb6e0 con 0x7f7f700d0850 2023-11-08T16:42:07.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.268+0000 7f7f750e9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f7f700d0850 0x7f7f7015f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.268+0000 7f7f6ffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f7f700b3010 0x7f7f7006f480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45182/0 (socket says 172.21.15.81:45182) 2023-11-08T16:42:07.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.268+0000 7f7f6ffff640 1 -- 172.21.15.81:0/768189276 learned_addr learned my addr 172.21.15.81:0/768189276 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:07.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.268+0000 7f7f750e9640 1 -- 172.21.15.81:0/768189276 >> v1:172.21.15.93:6789/0 conn(0x7f7f7015b220 legacy=0x7f7f700b2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.268+0000 7f7f750e9640 1 -- 172.21.15.81:0/768189276 >> v1:172.21.15.81:6789/0 conn(0x7f7f700b3010 legacy=0x7f7f7006f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.268+0000 7f7f750e9640 1 -- 172.21.15.81:0/768189276 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700cd0b0 con 0x7f7f700d0850 2023-11-08T16:42:07.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.268+0000 7f7f6f7fe640 1 -- 172.21.15.81:0/768189276 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4151182639 0 0) 0x7f7f700d0450 con 0x7f7f700b3010 2023-11-08T16:42:07.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.269+0000 7f7f750e9640 1 --2- 172.21.15.81:0/768189276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f700d0850 0x7f7f7015f680 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f7f680097a0 tx=0x7f7f6802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f12f901f8e839123 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:07.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.269+0000 7f7f6f7fe640 1 -- 172.21.15.81:0/768189276 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68030610 con 0x7f7f700d0850 2023-11-08T16:42:07.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.270+0000 7f7f6f7fe640 1 -- 172.21.15.81:0/768189276 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f68030770 con 0x7f7f700d0850 2023-11-08T16:42:07.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.270+0000 7f7f6f7fe640 1 -- 172.21.15.81:0/768189276 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f68030b00 con 0x7f7f700d0850 2023-11-08T16:42:07.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.270+0000 7f7f76b4c640 1 -- 172.21.15.81:0/768189276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f700d0850 msgr2=0x7f7f7015f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.270+0000 7f7f76b4c640 1 --2- 172.21.15.81:0/768189276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f700d0850 0x7f7f7015f680 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f7f680097a0 tx=0x7f7f6802b960 comp rx=0 tx=0).stop 2023-11-08T16:42:07.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.270+0000 7f7f76b4c640 1 -- 172.21.15.81:0/768189276 shutdown_connections 2023-11-08T16:42:07.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.270+0000 7f7f76b4c640 1 --2- 172.21.15.81:0/768189276 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f700d0850 0x7f7f7015f680 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.270+0000 7f7f76b4c640 1 -- 172.21.15.81:0/768189276 >> 172.21.15.81:0/768189276 conn(0x7f7f700cd900 msgr2=0x7f7f700cdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:07.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.270+0000 7f7f76b4c640 1 -- 172.21.15.81:0/768189276 shutdown_connections 2023-11-08T16:42:07.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.270+0000 7f7f76b4c640 1 -- 172.21.15.81:0/768189276 wait complete. 2023-11-08T16:42:07.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.273+0000 7f7f76b4c640 1 Processor -- start 2023-11-08T16:42:07.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.273+0000 7f7f76b4c640 1 -- start start 2023-11-08T16:42:07.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.274+0000 7f7f76b4c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7f700b3010 0x7f7f70159750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.274+0000 7f7f76b4c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f700d0850 0x7f7f700ca4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.274+0000 7f7f76b4c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f7015b220 0x7f7f700c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.274+0000 7f7f76b4c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70159c90 con 0x7f7f7015b220 2023-11-08T16:42:07.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.274+0000 7f7f76b4c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f701f2c90 con 0x7f7f700d0850 2023-11-08T16:42:07.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.274+0000 7f7f76b4c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f701f2f70 con 0x7f7f700b3010 2023-11-08T16:42:07.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.274+0000 7f7f750e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f7015b220 0x7f7f700c62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.274+0000 7f7f750e9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f7015b220 0x7f7f700c62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58772/0 (socket says 172.21.15.81:58772) 2023-11-08T16:42:07.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.274+0000 7f7f750e9640 1 -- 172.21.15.81:0/3480475487 learned_addr learned my addr 172.21.15.81:0/3480475487 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:07.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.275+0000 7f7f6ffff640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f700d0850 0x7f7f700ca4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.275+0000 7f7f748e8640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7f700b3010 0x7f7f70159750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.275+0000 7f7f750e9640 1 -- 172.21.15.81:0/3480475487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7f700b3010 msgr2=0x7f7f70159750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.275+0000 7f7f750e9640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7f700b3010 0x7f7f70159750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.275+0000 7f7f750e9640 1 -- 172.21.15.81:0/3480475487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f700d0850 msgr2=0x7f7f700ca4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.275+0000 7f7f750e9640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f700d0850 0x7f7f700ca4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.275+0000 7f7f750e9640 1 -- 172.21.15.81:0/3480475487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f68009400 con 0x7f7f7015b220 2023-11-08T16:42:07.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.275+0000 7f7f6ffff640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f700d0850 0x7f7f700ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:07.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.275+0000 7f7f750e9640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f7015b220 0x7f7f700c62b0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f7f680098c0 tx=0x7f7f68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:07.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.275+0000 7f7f6d7fa640 1 -- 172.21.15.81:0/3480475487 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6803e040 con 0x7f7f7015b220 2023-11-08T16:42:07.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.276+0000 7f7f6d7fa640 1 -- 172.21.15.81:0/3480475487 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f68039c10 con 0x7f7f7015b220 2023-11-08T16:42:07.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.276+0000 7f7f6d7fa640 1 -- 172.21.15.81:0/3480475487 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f680443f0 con 0x7f7f7015b220 2023-11-08T16:42:07.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.276+0000 7f7f76b4c640 1 -- 172.21.15.81:0/3480475487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f701f3250 con 0x7f7f7015b220 2023-11-08T16:42:07.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.276+0000 7f7f76b4c640 1 -- 172.21.15.81:0/3480475487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f700c9fc0 con 0x7f7f7015b220 2023-11-08T16:42:07.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.277+0000 7f7f76b4c640 1 -- 172.21.15.81:0/3480475487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f701f3a50 con 0x7f7f7015b220 2023-11-08T16:42:07.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.278+0000 7f7f6d7fa640 1 -- 172.21.15.81:0/3480475487 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7f6804d3b0 con 0x7f7f7015b220 2023-11-08T16:42:07.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.279+0000 7f7f6d7fa640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7f5006ae70 0x7f7f5006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.279+0000 7f7f6d7fa640 1 -- 172.21.15.81:0/3480475487 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f7f680abe10 con 0x7f7f7015b220 2023-11-08T16:42:07.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.279+0000 7f7f748e8640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7f5006ae70 0x7f7f5006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.279+0000 7f7f748e8640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7f5006ae70 0x7f7f5006d330 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7f58000f60 tx=0x7f7f58009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:07.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.283+0000 7f7f6d7fa640 1 -- 172.21.15.81:0/3480475487 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f7f701f3a50 con 0x7f7f7015b220 2023-11-08T16:42:07.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.512+0000 7f7f76b4c640 1 -- 172.21.15.81:0/3480475487 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f701f3a50 con 0x7f7f7015b220 2023-11-08T16:42:07.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.513+0000 7f7f6d7fa640 1 -- 172.21.15.81:0/3480475487 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4920 (secure 0 0 0) 0x7f7f701f3a50 con 0x7f7f7015b220 2023-11-08T16:42:07.719 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:07.719 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":120,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/329260749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":329260749},{"type":"v1","addr":"172.21.15.93:6835","nonce":329260749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:55.094753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":117,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/1432485913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1432485913},{"type":"v1","addr":"172.21.15.81:6837","nonce":1432485913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:42:07.721 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 120 2023-11-08T16:42:07.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 -- 172.21.15.81:0/3480475487 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7f5006ae70 msgr2=0x7f7f5006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7f5006ae70 0x7f7f5006d330 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7f58000f60 tx=0x7f7f58009040 comp rx=0 tx=0).stop 2023-11-08T16:42:07.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 -- 172.21.15.81:0/3480475487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f7015b220 msgr2=0x7f7f700c62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f7015b220 0x7f7f700c62b0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f7f680098c0 tx=0x7f7f68004730 comp rx=0 tx=0).stop 2023-11-08T16:42:07.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 -- 172.21.15.81:0/3480475487 shutdown_connections 2023-11-08T16:42:07.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f7f5006ae70 0x7f7f5006d330 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7f7015b220 0x7f7f700c62b0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f700d0850 0x7f7f700ca4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 --2- 172.21.15.81:0/3480475487 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f7f700b3010 0x7f7f70159750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 -- 172.21.15.81:0/3480475487 >> 172.21.15.81:0/3480475487 conn(0x7f7f700cd900 msgr2=0x7f7f700b5060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:07.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 -- 172.21.15.81:0/3480475487 shutdown_connections 2023-11-08T16:42:07.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.516+0000 7f7f76b4c640 1 -- 172.21.15.81:0/3480475487 wait complete. 2023-11-08T16:42:07.723 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:42:07.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.942+0000 7f1643d39640 1 Processor -- start 2023-11-08T16:42:07.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.942+0000 7f1643d39640 1 -- start start 2023-11-08T16:42:07.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.943+0000 7f1643d39640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f163c0b3550 0x7f163c15b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.943+0000 7f1643d39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c06c9f0 0x7f163c144230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.943+0000 7f1643d39640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163c0b2310 con 0x7f163c0b3170 2023-11-08T16:42:07.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.943+0000 7f1643d39640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163c0b2bb0 con 0x7f163c0b3550 2023-11-08T16:42:07.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.943+0000 7f1643d39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0b25f0 con 0x7f163c06c9f0 2023-11-08T16:42:07.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.944+0000 7f1641ad5640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f163c0b3170 0x7f163c0b1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45198/0 (socket says 172.21.15.81:45198) 2023-11-08T16:42:07.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.944+0000 7f1641ad5640 1 -- 172.21.15.81:0/2977659000 learned_addr learned my addr 172.21.15.81:0/2977659000 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:07.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.944+0000 7f16412d4640 1 --2- 172.21.15.81:0/2977659000 >> v2:172.21.15.93:3300/0 conn(0x7f163c0b3550 0x7f163c15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.944+0000 7f16422d6640 1 --2- 172.21.15.81:0/2977659000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c06c9f0 0x7f163c144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.944+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3029838453 0 0) 0x7f163c0b2310 con 0x7f163c0b3170 2023-11-08T16:42:07.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.944+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1628003520 con 0x7f163c0b3170 2023-11-08T16:42:07.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2739604219 0 0) 0x7f1628003520 con 0x7f163c0b3170 2023-11-08T16:42:07.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c06c9f0 msgr2=0x7f163c144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:07.950 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 --2- 172.21.15.81:0/2977659000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c06c9f0 0x7f163c144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 >> v2:172.21.15.93:3300/0 conn(0x7f163c0b3550 msgr2=0x7f163c15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:07.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 --2- 172.21.15.81:0/2977659000 >> v2:172.21.15.93:3300/0 conn(0x7f163c0b3550 0x7f163c15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c0b28d0 con 0x7f163c0b3170 2023-11-08T16:42:07.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f1638001670 con 0x7f163c0b3170 2023-11-08T16:42:07.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 >> v1:172.21.15.81:6789/0 conn(0x7f163c0b3170 legacy=0x7f163c0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 --2- 172.21.15.81:0/2977659000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c06c9f0 0x7f1628003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 --2- 172.21.15.81:0/2977659000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1628003f40 0x7f16280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 --2- 172.21.15.81:0/2977659000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1628004d00 0x7f1628005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0b2bb0 con 0x7f1628003f40 2023-11-08T16:42:07.952 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c0b25f0 con 0x7f1628004d00 2023-11-08T16:42:07.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162800b980 con 0x7f163c06c9f0 2023-11-08T16:42:07.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.945+0000 7f16412d4640 1 --2- 172.21.15.81:0/2977659000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1628003f40 0x7f16280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.946+0000 7f16412d4640 1 -- 172.21.15.81:0/2977659000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c06c9f0 msgr2=0x7f1628003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:07.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.946+0000 7f16412d4640 1 --2- 172.21.15.81:0/2977659000 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c06c9f0 0x7f1628003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.946+0000 7f16412d4640 1 -- 172.21.15.81:0/2977659000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1628004d00 msgr2=0x7f1628005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:07.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.946+0000 7f16412d4640 1 --2- 172.21.15.81:0/2977659000 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1628004d00 0x7f1628005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.946+0000 7f16412d4640 1 -- 172.21.15.81:0/2977659000 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1628003520 con 0x7f1628003f40 2023-11-08T16:42:07.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.947+0000 7f16412d4640 1 --2- 172.21.15.81:0/2977659000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1628003f40 0x7f16280047c0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f162c009af0 tx=0x7f16280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9efd7d65ac73e01d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:07.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.947+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c02c710 con 0x7f1628003f40 2023-11-08T16:42:07.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.947+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c02c870 con 0x7f1628003f40 2023-11-08T16:42:07.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.947+0000 7f1640ad3640 1 -- 172.21.15.81:0/2977659000 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c02cc00 con 0x7f1628003f40 2023-11-08T16:42:07.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.947+0000 7f1643d39640 1 -- 172.21.15.81:0/2977659000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1628003f40 msgr2=0x7f16280047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.947+0000 7f1643d39640 1 --2- 172.21.15.81:0/2977659000 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f1628003f40 0x7f16280047c0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f162c009af0 tx=0x7f16280057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.947+0000 7f16422d6640 1 -- 172.21.15.81:0/2977659000 reap_dead start 2023-11-08T16:42:07.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.948+0000 7f1643d39640 1 -- 172.21.15.81:0/2977659000 shutdown_connections 2023-11-08T16:42:07.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.948+0000 7f1643d39640 1 -- 172.21.15.81:0/2977659000 >> 172.21.15.81:0/2977659000 conn(0x7f163c06a6a0 msgr2=0x7f163c0b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:07.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.948+0000 7f1643d39640 1 -- 172.21.15.81:0/2977659000 shutdown_connections 2023-11-08T16:42:07.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.948+0000 7f1643d39640 1 -- 172.21.15.81:0/2977659000 wait complete. 2023-11-08T16:42:07.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.949+0000 7f1643d39640 1 Processor -- start 2023-11-08T16:42:07.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.950+0000 7f1643d39640 1 -- start start 2023-11-08T16:42:07.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.950+0000 7f1643d39640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1628003f40 0x7f163c138200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.950+0000 7f1643d39640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c13bc20 0x7f163c13a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.950+0000 7f1643d39640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f163c13a990 0x7f163c155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.950+0000 7f1643d39640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c1393f0 con 0x7f163c13a990 2023-11-08T16:42:07.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.950+0000 7f1643d39640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f163c138740 con 0x7f1628003f40 2023-11-08T16:42:07.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.950+0000 7f1643d39640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163c139690 con 0x7f163c13bc20 2023-11-08T16:42:07.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.951+0000 7f16422d6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f163c13a990 0x7f163c155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.951+0000 7f16422d6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f163c13a990 0x7f163c155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58782/0 (socket says 172.21.15.81:58782) 2023-11-08T16:42:07.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.951+0000 7f16422d6640 1 -- 172.21.15.81:0/3849800789 learned_addr learned my addr 172.21.15.81:0/3849800789 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:07.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.951+0000 7f16412d4640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c13bc20 0x7f163c13a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.951+0000 7f1641ad5640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1628003f40 0x7f163c138200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.951+0000 7f16422d6640 1 -- 172.21.15.81:0/3849800789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c13bc20 msgr2=0x7f163c13a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.951+0000 7f16422d6640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c13bc20 0x7f163c13a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.951+0000 7f16422d6640 1 -- 172.21.15.81:0/3849800789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1628003f40 msgr2=0x7f163c138200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:07.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.951+0000 7f16422d6640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1628003f40 0x7f163c138200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:07.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.951+0000 7f16422d6640 1 -- 172.21.15.81:0/3849800789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163c06d000 con 0x7f163c13a990 2023-11-08T16:42:07.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.951+0000 7f16422d6640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f163c13a990 0x7f163c155ad0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f163c0b1680 tx=0x7f163400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:07.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.952+0000 7f16327fc640 1 -- 172.21.15.81:0/3849800789 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634010040 con 0x7f163c13a990 2023-11-08T16:42:07.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.952+0000 7f16327fc640 1 -- 172.21.15.81:0/3849800789 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163400bae0 con 0x7f163c13a990 2023-11-08T16:42:07.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.952+0000 7f16327fc640 1 -- 172.21.15.81:0/3849800789 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1634014400 con 0x7f163c13a990 2023-11-08T16:42:07.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.952+0000 7f1643d39640 1 -- 172.21.15.81:0/3849800789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163c139970 con 0x7f163c13a990 2023-11-08T16:42:07.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.952+0000 7f1643d39640 1 -- 172.21.15.81:0/3849800789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163c139f30 con 0x7f163c13a990 2023-11-08T16:42:07.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.953+0000 7f1643d39640 1 -- 172.21.15.81:0/3849800789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f163c13a120 con 0x7f163c13a990 2023-11-08T16:42:07.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.954+0000 7f16327fc640 1 -- 172.21.15.81:0/3849800789 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1634014560 con 0x7f163c13a990 2023-11-08T16:42:07.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.955+0000 7f16327fc640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f162006aea0 0x7f162006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:07.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.956+0000 7f16327fc640 1 -- 172.21.15.81:0/3849800789 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f16340899c0 con 0x7f163c13a990 2023-11-08T16:42:07.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.956+0000 7f1641ad5640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f162006aea0 0x7f162006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:07.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.956+0000 7f1641ad5640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f162006aea0 0x7f162006d360 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f16380079b0 tx=0x7f1638009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:07.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:07.960+0000 7f16327fc640 1 -- 172.21.15.81:0/3849800789 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f1634054970 con 0x7f163c13a990 2023-11-08T16:42:08.183 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.181+0000 7f1643d39640 1 -- 172.21.15.81:0/3849800789 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f163c074440 con 0x7f163c13a990 2023-11-08T16:42:08.184 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.183+0000 7f16327fc640 1 -- 172.21.15.81:0/3849800789 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4920 (secure 0 0 0) 0x7f1634018090 con 0x7f163c13a990 2023-11-08T16:42:08.184 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:08.184 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":120,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/329260749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":329260749},{"type":"v1","addr":"172.21.15.93:6835","nonce":329260749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":120,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-08T16:41:05.700435+0000","modified":"2023-11-08T16:41:55.094753+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":131,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8970":{"gid":8970,"name":"c","rank":0,"incarnation":117,"state":"up:active","state_seq":4,"addr":"172.21.15.81:6837/1432485913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":1432485913},{"type":"v1","addr":"172.21.15.81:6837","nonce":1432485913}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-08T16:42:08.185 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 120 2023-11-08T16:42:08.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.185+0000 7f1643d39640 1 -- 172.21.15.81:0/3849800789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f162006aea0 msgr2=0x7f162006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.186 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.185+0000 7f1643d39640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f162006aea0 0x7f162006d360 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f16380079b0 tx=0x7f1638009040 comp rx=0 tx=0).stop 2023-11-08T16:42:08.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.185+0000 7f1643d39640 1 -- 172.21.15.81:0/3849800789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f163c13a990 msgr2=0x7f163c155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.189 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.185+0000 7f1643d39640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f163c13a990 0x7f163c155ad0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f163c0b1680 tx=0x7f163400b480 comp rx=0 tx=0).stop 2023-11-08T16:42:08.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.185+0000 7f1643d39640 1 -- 172.21.15.81:0/3849800789 shutdown_connections 2023-11-08T16:42:08.190 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.185+0000 7f1643d39640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f162006aea0 0x7f162006d360 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.185+0000 7f1643d39640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f163c13a990 0x7f163c155ad0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.185+0000 7f1643d39640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f163c13bc20 0x7f163c13a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.185+0000 7f1643d39640 1 --2- 172.21.15.81:0/3849800789 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1628003f40 0x7f163c138200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.185+0000 7f1643d39640 1 -- 172.21.15.81:0/3849800789 >> 172.21.15.81:0/3849800789 conn(0x7f163c06a6a0 msgr2=0x7f163c15b500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:08.191 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.185+0000 7f1643d39640 1 -- 172.21.15.81:0/3849800789 shutdown_connections 2023-11-08T16:42:08.192 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.186+0000 7f1643d39640 1 -- 172.21.15.81:0/3849800789 wait complete. 2023-11-08T16:42:08.205 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:42:08.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.400+0000 7f44eae4f640 1 Processor -- start 2023-11-08T16:42:08.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.400+0000 7f44eae4f640 1 -- start start 2023-11-08T16:42:08.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.401+0000 7f44eae4f640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f44e4143800 0x7f44e40b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.401+0000 7f44eae4f640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e40cd1e0 con 0x7f44e415e0f0 2023-11-08T16:42:08.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.401+0000 7f44eae4f640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e40cd4c0 con 0x7f44e40cca70 2023-11-08T16:42:08.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.401+0000 7f44eae4f640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e40b2df0 con 0x7f44e4143800 2023-11-08T16:42:08.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.402+0000 7f44e93ec640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f44e415e0f0 0x7f44e40704e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45204/0 (socket says 172.21.15.81:45204) 2023-11-08T16:42:08.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.402+0000 7f44e93ec640 1 -- 172.21.15.81:0/1858336995 learned_addr learned my addr 172.21.15.81:0/1858336995 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:08.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.402+0000 7f44e8beb640 1 --2- 172.21.15.81:0/1858336995 >> v2:172.21.15.93:3300/0 conn(0x7f44e4143800 0x7f44e40b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.402+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253413270 0 0) 0x7f44e40cd1e0 con 0x7f44e415e0f0 2023-11-08T16:42:08.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.402+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44d4003520 con 0x7f44e415e0f0 2023-11-08T16:42:08.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 755572397 0 0) 0x7f44d4003520 con 0x7f44e415e0f0 2023-11-08T16:42:08.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 >> v2:172.21.15.93:3300/0 conn(0x7f44e4143800 msgr2=0x7f44e40b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:08.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 --2- 172.21.15.81:0/1858336995 >> v2:172.21.15.93:3300/0 conn(0x7f44e4143800 0x7f44e40b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 >> v1:172.21.15.93:6789/0 conn(0x7f44e40cca70 legacy=0x7f44e415d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40cd7a0 con 0x7f44e415e0f0 2023-11-08T16:42:08.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f44dc0019f0 con 0x7f44e415e0f0 2023-11-08T16:42:08.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 >> v1:172.21.15.81:6789/0 conn(0x7f44e415e0f0 legacy=0x7f44e40704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44d4003970 0x7f44d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 0x7f44d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44d4005180 0x7f44d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40b2df0 con 0x7f44d40047c0 2023-11-08T16:42:08.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d400c020 con 0x7f44d4005180 2023-11-08T16:42:08.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d400c1e0 con 0x7f44d4003970 2023-11-08T16:42:08.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e3fff640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 0x7f44d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.403+0000 7f44e93ec640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44d4005180 0x7f44d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.404+0000 7f44e8beb640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44d4003970 0x7f44d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.404+0000 7f44e3fff640 1 -- 172.21.15.81:0/1858336995 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44d4003970 msgr2=0x7f44d4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:08.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.404+0000 7f44e3fff640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44d4003970 0x7f44d4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.404+0000 7f44e3fff640 1 -- 172.21.15.81:0/1858336995 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44d4005180 msgr2=0x7f44d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:08.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.404+0000 7f44e3fff640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44d4005180 0x7f44d400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.404+0000 7f44e3fff640 1 -- 172.21.15.81:0/1858336995 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d4003520 con 0x7f44d40047c0 2023-11-08T16:42:08.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.404+0000 7f44e93ec640 1 -- 172.21.15.81:0/1858336995 reap_dead start 2023-11-08T16:42:08.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.404+0000 7f44e8beb640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44d4003970 0x7f44d4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:08.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.405+0000 7f44e3fff640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 0x7f44d4004c40 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f44d0001750 tx=0x7f44d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f7db3e4ed6597be server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:08.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.405+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d00052d0 con 0x7f44d40047c0 2023-11-08T16:42:08.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.405+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d002e050 con 0x7f44d40047c0 2023-11-08T16:42:08.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.405+0000 7f44e37fe640 1 -- 172.21.15.81:0/1858336995 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d0003830 con 0x7f44d40047c0 2023-11-08T16:42:08.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.406+0000 7f44eae4f640 1 -- 172.21.15.81:0/1858336995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 msgr2=0x7f44d4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.406+0000 7f44eae4f640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 0x7f44d4004c40 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f44d0001750 tx=0x7f44d40057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.406+0000 7f44eae4f640 1 -- 172.21.15.81:0/1858336995 shutdown_connections 2023-11-08T16:42:08.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.406+0000 7f44eae4f640 1 --2- 172.21.15.81:0/1858336995 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 0x7f44d4004c40 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.406+0000 7f44eae4f640 1 -- 172.21.15.81:0/1858336995 >> 172.21.15.81:0/1858336995 conn(0x7f44e406a6a0 msgr2=0x7f44e40b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:08.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.406+0000 7f44eae4f640 1 -- 172.21.15.81:0/1858336995 shutdown_connections 2023-11-08T16:42:08.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.406+0000 7f44eae4f640 1 -- 172.21.15.81:0/1858336995 wait complete. 2023-11-08T16:42:08.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.407+0000 7f44eae4f640 1 Processor -- start 2023-11-08T16:42:08.417 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.408+0000 7f44eae4f640 1 -- start start 2023-11-08T16:42:08.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44eae4f640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 0x7f44e40ac810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44eae4f640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44e40ad8d0 0x7f44e41596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44eae4f640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44e406ca00 0x7f44e406ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44eae4f640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e406d3c0 con 0x7f44d40047c0 2023-11-08T16:42:08.418 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44eae4f640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e4159bf0 con 0x7f44e406ca00 2023-11-08T16:42:08.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44eae4f640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e40ad690 con 0x7f44e40ad8d0 2023-11-08T16:42:08.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44e8beb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 0x7f44e40ac810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44e8beb640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 0x7f44e40ac810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58808/0 (socket says 172.21.15.81:58808) 2023-11-08T16:42:08.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44e8beb640 1 -- 172.21.15.81:0/639674466 learned_addr learned my addr 172.21.15.81:0/639674466 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:08.419 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44e8beb640 1 -- 172.21.15.81:0/639674466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44e40ad8d0 msgr2=0x7f44e41596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44e8beb640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44e40ad8d0 0x7f44e41596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44e8beb640 1 -- 172.21.15.81:0/639674466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44e406ca00 msgr2=0x7f44e406ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44e8beb640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44e406ca00 0x7f44e406ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.409+0000 7f44e8beb640 1 -- 172.21.15.81:0/639674466 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40acdf0 con 0x7f44d40047c0 2023-11-08T16:42:08.420 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.410+0000 7f44e8beb640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 0x7f44e40ac810 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f44d80107f0 tx=0x7f44d8010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:08.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.410+0000 7f44e17fa640 1 -- 172.21.15.81:0/639674466 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d800eb60 con 0x7f44d40047c0 2023-11-08T16:42:08.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.410+0000 7f44e17fa640 1 -- 172.21.15.81:0/639674466 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d800ecc0 con 0x7f44d40047c0 2023-11-08T16:42:08.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.410+0000 7f44e17fa640 1 -- 172.21.15.81:0/639674466 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8010ea0 con 0x7f44d40047c0 2023-11-08T16:42:08.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.410+0000 7f44eae4f640 1 -- 172.21.15.81:0/639674466 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e40ad0d0 con 0x7f44d40047c0 2023-11-08T16:42:08.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.410+0000 7f44eae4f640 1 -- 172.21.15.81:0/639674466 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e413ada0 con 0x7f44d40047c0 2023-11-08T16:42:08.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.411+0000 7f44eae4f640 1 -- 172.21.15.81:0/639674466 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e413af60 con 0x7f44d40047c0 2023-11-08T16:42:08.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.412+0000 7f44e17fa640 1 -- 172.21.15.81:0/639674466 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f44d801c400 con 0x7f44d40047c0 2023-11-08T16:42:08.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.413+0000 7f44e17fa640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44c806ae70 0x7f44c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.413+0000 7f44e17fa640 1 -- 172.21.15.81:0/639674466 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f44d8089bf0 con 0x7f44d40047c0 2023-11-08T16:42:08.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.414+0000 7f44e3fff640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44c806ae70 0x7f44c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.414+0000 7f44e3fff640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44c806ae70 0x7f44c806d330 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f44d0033000 tx=0x7f44d002f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:08.423 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.418+0000 7f44e17fa640 1 -- 172.21.15.81:0/639674466 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f44d8054bc0 con 0x7f44d40047c0 2023-11-08T16:42:08.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.603+0000 7f44eae4f640 1 -- 172.21.15.81:0/639674466 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44e413ac00 con 0x7f44d40047c0 2023-11-08T16:42:08.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.605+0000 7f44e17fa640 1 -- 172.21.15.81:0/639674466 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+16878 (secure 0 0 0) 0x7f44e413ac00 con 0x7f44d40047c0 2023-11-08T16:42:08.607 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:08.607 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":131,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:41:49.127728+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-11-08T16:41:03.545565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-11-08T16:41:04.726492+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"127","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.18","mappings":[{"from":0,"to":2}]},{"pgid":"17.24","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/296230272":"2023-11-09T16:41:49.127674+0000","172.21.15.81:6836/296230272":"2023-11-09T16:41:49.127674+0000","172.21.15.93:6835/4021510722":"2023-11-09T16:41:29.470203+0000","172.21.15.93:6833/4021510722":"2023-11-09T16:41:29.470203+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:42:08.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.607+0000 7f44eae4f640 1 -- 172.21.15.81:0/639674466 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44c806ae70 msgr2=0x7f44c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.607+0000 7f44eae4f640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44c806ae70 0x7f44c806d330 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f44d0033000 tx=0x7f44d002f000 comp rx=0 tx=0).stop 2023-11-08T16:42:08.609 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.607+0000 7f44eae4f640 1 -- 172.21.15.81:0/639674466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 msgr2=0x7f44e40ac810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.607+0000 7f44eae4f640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 0x7f44e40ac810 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f44d80107f0 tx=0x7f44d8010cb0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.608+0000 7f44eae4f640 1 -- 172.21.15.81:0/639674466 shutdown_connections 2023-11-08T16:42:08.610 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.608+0000 7f44eae4f640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f44c806ae70 0x7f44c806d330 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.608+0000 7f44eae4f640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f44e406ca00 0x7f44e406ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.608+0000 7f44eae4f640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f44e40ad8d0 0x7f44e41596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.608+0000 7f44eae4f640 1 --2- 172.21.15.81:0/639674466 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f44d40047c0 0x7f44e40ac810 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.611 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.608+0000 7f44eae4f640 1 -- 172.21.15.81:0/639674466 >> 172.21.15.81:0/639674466 conn(0x7f44e406a6a0 msgr2=0x7f44e4142080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:08.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.608+0000 7f44eae4f640 1 -- 172.21.15.81:0/639674466 shutdown_connections 2023-11-08T16:42:08.612 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.608+0000 7f44eae4f640 1 -- 172.21.15.81:0/639674466 wait complete. 2023-11-08T16:42:08.627 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-08T16:42:08.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.831+0000 7fd0a517c640 1 Processor -- start 2023-11-08T16:42:08.833 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.832+0000 7fd0a517c640 1 -- start start 2023-11-08T16:42:08.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.833+0000 7fd0a517c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0a00cb410 0x7fd0a00ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.833+0000 7fd0a517c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a00cb9b0 con 0x7fd0a00b3c10 2023-11-08T16:42:08.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.833+0000 7fd0a517c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a00d0610 con 0x7fd0a015e750 2023-11-08T16:42:08.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.833+0000 7fd0a517c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a00cbc90 con 0x7fd0a00cb410 2023-11-08T16:42:08.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09f59e640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fd0a00b3c10 0x7fd0a00cd490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:45210/0 (socket says 172.21.15.81:45210) 2023-11-08T16:42:08.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09f59e640 1 -- 172.21.15.81:0/1685677993 learned_addr learned my addr 172.21.15.81:0/1685677993 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:08.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09e59c640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0a00cb410 0x7fd0a00ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3252483175 0 0) 0x7fd0a00cb9b0 con 0x7fd0a00b3c10 2023-11-08T16:42:08.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd08c003520 con 0x7fd0a00b3c10 2023-11-08T16:42:08.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2540905318 0 0) 0x7fd08c003520 con 0x7fd0a00b3c10 2023-11-08T16:42:08.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0a00cb410 msgr2=0x7fd0a00ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:08.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09dd9b640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0a00cb410 0x7fd0a00ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 >> v1:172.21.15.93:6789/0 conn(0x7fd0a015e750 legacy=0x7fd0a015f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a00d08f0 con 0x7fd0a00b3c10 2023-11-08T16:42:08.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fd098001670 con 0x7fd0a00b3c10 2023-11-08T16:42:08.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 >> v1:172.21.15.81:6789/0 conn(0x7fd0a00b3c10 legacy=0x7fd0a00cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.834+0000 7fd09dd9b640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c0039a0 0x7fd08c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09dd9b640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0a00cb410 0x7fd08c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09dd9b640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd08c004d00 0x7fd08c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c00ba30 con 0x7fd08c0039a0 2023-11-08T16:42:08.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c00bb70 con 0x7fd08c004d00 2023-11-08T16:42:08.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c00bd30 con 0x7fd0a00cb410 2023-11-08T16:42:08.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09ed9d640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c0039a0 0x7fd08c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09e59c640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0a00cb410 0x7fd08c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09ed9d640 1 -- 172.21.15.81:0/1685677993 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0a00cb410 msgr2=0x7fd08c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:08.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09ed9d640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd0a00cb410 0x7fd08c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09f59e640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd08c004d00 0x7fd08c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09ed9d640 1 -- 172.21.15.81:0/1685677993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd08c004d00 msgr2=0x7fd08c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:08.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09ed9d640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd08c004d00 0x7fd08c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.835+0000 7fd09ed9d640 1 -- 172.21.15.81:0/1685677993 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd08c003520 con 0x7fd08c0039a0 2023-11-08T16:42:08.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.836+0000 7fd09ed9d640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c0039a0 0x7fd08c003dc0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fd090001e20 tx=0x7fd08c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db4cc5262d172211 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:08.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.837+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090003b70 con 0x7fd08c0039a0 2023-11-08T16:42:08.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.837+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd090003cd0 con 0x7fd08c0039a0 2023-11-08T16:42:08.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.837+0000 7fd09dd9b640 1 -- 172.21.15.81:0/1685677993 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090005310 con 0x7fd08c0039a0 2023-11-08T16:42:08.844 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.837+0000 7fd0a517c640 1 -- 172.21.15.81:0/1685677993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c0039a0 msgr2=0x7fd08c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.837+0000 7fd0a517c640 1 --2- 172.21.15.81:0/1685677993 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd08c0039a0 0x7fd08c003dc0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fd090001e20 tx=0x7fd08c0057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.837+0000 7fd09f59e640 1 -- 172.21.15.81:0/1685677993 reap_dead start 2023-11-08T16:42:08.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.837+0000 7fd0a517c640 1 -- 172.21.15.81:0/1685677993 shutdown_connections 2023-11-08T16:42:08.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.837+0000 7fd0a517c640 1 -- 172.21.15.81:0/1685677993 >> 172.21.15.81:0/1685677993 conn(0x7fd0a006f830 msgr2=0x7fd0a006fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:08.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.837+0000 7fd0a517c640 1 -- 172.21.15.81:0/1685677993 shutdown_connections 2023-11-08T16:42:08.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.837+0000 7fd0a517c640 1 -- 172.21.15.81:0/1685677993 wait complete. 2023-11-08T16:42:08.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.839+0000 7fd0a517c640 1 Processor -- start 2023-11-08T16:42:08.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.839+0000 7fd0a517c640 1 -- start start 2023-11-08T16:42:08.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.840+0000 7fd0a517c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd08c0039a0 0x7fd0a00ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.840+0000 7fd0a517c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0a014d5b0 0x7fd0a0150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.840+0000 7fd0a517c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0a006b700 0x7fd0a006bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.840+0000 7fd0a517c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a006c0c0 con 0x7fd0a006b700 2023-11-08T16:42:08.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.840+0000 7fd0a517c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a01510f0 con 0x7fd0a014d5b0 2023-11-08T16:42:08.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.840+0000 7fd0a517c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a014aab0 con 0x7fd08c0039a0 2023-11-08T16:42:08.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.840+0000 7fd09f59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0a006b700 0x7fd0a006bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.840+0000 7fd09f59e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0a006b700 0x7fd0a006bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58828/0 (socket says 172.21.15.81:58828) 2023-11-08T16:42:08.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.840+0000 7fd09f59e640 1 -- 172.21.15.81:0/1039833136 learned_addr learned my addr 172.21.15.81:0/1039833136 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:08.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd09e59c640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0a014d5b0 0x7fd0a0150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd09ed9d640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd08c0039a0 0x7fd0a00ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd09f59e640 1 -- 172.21.15.81:0/1039833136 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd08c0039a0 msgr2=0x7fd0a00ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd09f59e640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd08c0039a0 0x7fd0a00ca3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd09f59e640 1 -- 172.21.15.81:0/1039833136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0a014d5b0 msgr2=0x7fd0a0150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:08.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd09f59e640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0a014d5b0 0x7fd0a0150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:08.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd09f59e640 1 -- 172.21.15.81:0/1039833136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a014ad90 con 0x7fd0a006b700 2023-11-08T16:42:08.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd09f59e640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0a006b700 0x7fd0a006bb80 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00740d0 tx=0x7fd09800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:08.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd0977fe640 1 -- 172.21.15.81:0/1039833136 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0980074e0 con 0x7fd0a006b700 2023-11-08T16:42:08.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd0977fe640 1 -- 172.21.15.81:0/1039833136 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd098007640 con 0x7fd0a006b700 2023-11-08T16:42:08.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd0977fe640 1 -- 172.21.15.81:0/1039833136 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098004a20 con 0x7fd0a006b700 2023-11-08T16:42:08.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.841+0000 7fd0a517c640 1 -- 172.21.15.81:0/1039833136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a014b070 con 0x7fd0a006b700 2023-11-08T16:42:08.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.842+0000 7fd0a517c640 1 -- 172.21.15.81:0/1039833136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a01513d0 con 0x7fd0a006b700 2023-11-08T16:42:08.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.843+0000 7fd0a517c640 1 -- 172.21.15.81:0/1039833136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a0074440 con 0x7fd0a006b700 2023-11-08T16:42:08.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.843+0000 7fd0977fe640 1 -- 172.21.15.81:0/1039833136 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd098004b80 con 0x7fd0a006b700 2023-11-08T16:42:08.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.844+0000 7fd0977fe640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd08006aea0 0x7fd08006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:08.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.844+0000 7fd0977fe640 1 -- 172.21.15.81:0/1039833136 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5939+0+0 (secure 0 0 0) 0x7fd098089360 con 0x7fd0a006b700 2023-11-08T16:42:08.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.845+0000 7fd09ed9d640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd08006aea0 0x7fd08006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:08.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.845+0000 7fd09ed9d640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd08006aea0 0x7fd08006d360 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd090034000 tx=0x7fd090039040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:08.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:08.849+0000 7fd0977fe640 1 -- 172.21.15.81:0/1039833136 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fd098057e40 con 0x7fd0a006b700 2023-11-08T16:42:09.074 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.073+0000 7fd0a517c640 1 -- 172.21.15.81:0/1039833136 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd0a00b6ae0 con 0x7fd0a006b700 2023-11-08T16:42:09.075 INFO:tasks.ceph.mon.a.smithi081.stderr:2023-11-08T16:42:09.074+0000 7f5cc31ce640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-08T16:42:09.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.077+0000 7fd0977fe640 1 -- 172.21.15.81:0/1039833136 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v121) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd0980579d0 con 0x7fd0a006b700 2023-11-08T16:42:09.079 INFO:teuthology.orchestra.run.smithi081.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-08T16:42:09.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 -- 172.21.15.81:0/1039833136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd08006aea0 msgr2=0x7fd08006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:09.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd08006aea0 0x7fd08006d360 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fd090034000 tx=0x7fd090039040 comp rx=0 tx=0).stop 2023-11-08T16:42:09.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 -- 172.21.15.81:0/1039833136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0a006b700 msgr2=0x7fd0a006bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:09.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0a006b700 0x7fd0a006bb80 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fd0a00740d0 tx=0x7fd09800d9d0 comp rx=0 tx=0).stop 2023-11-08T16:42:09.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 -- 172.21.15.81:0/1039833136 shutdown_connections 2023-11-08T16:42:09.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fd08006aea0 0x7fd08006d360 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:09.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fd0a006b700 0x7fd0a006bb80 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:09.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0a014d5b0 0x7fd0a0150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:09.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 --2- 172.21.15.81:0/1039833136 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fd08c0039a0 0x7fd0a00ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:09.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 -- 172.21.15.81:0/1039833136 >> 172.21.15.81:0/1039833136 conn(0x7fd0a006f830 msgr2=0x7fd0a00b49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:09.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 -- 172.21.15.81:0/1039833136 shutdown_connections 2023-11-08T16:42:09.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.080+0000 7fd0a517c640 1 -- 172.21.15.81:0/1039833136 wait complete. 2023-11-08T16:42:09.100 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-08T16:42:09.152 INFO:tasks.ceph.mds.c.smithi081.stdout:starting mds.c at 2023-11-08T16:42:09.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.290+0000 7f502673c640 1 Processor -- start 2023-11-08T16:42:09.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.290+0000 7f502673c640 1 -- start start 2023-11-08T16:42:09.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.292+0000 7f502673c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f50200add30 0x7f50200cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:09.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.292+0000 7f502673c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f50200b3da0 0x7f50200cc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:09.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.292+0000 7f502673c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50200ce130 con 0x7f50201444d0 2023-11-08T16:42:09.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.292+0000 7f502673c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5020143e20 con 0x7f50200b3da0 2023-11-08T16:42:09.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.292+0000 7f502673c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50201442d0 con 0x7f50200add30 2023-11-08T16:42:09.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.292+0000 7f5024f39640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f50200b3da0 0x7f50200cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:09.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.292+0000 7f5024f39640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f50200b3da0 0x7f50200cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58854/0 (socket says 172.21.15.81:58854) 2023-11-08T16:42:09.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.292+0000 7f5024f39640 1 -- 172.21.15.81:0/2173931198 learned_addr learned my addr 172.21.15.81:0/2173931198 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:09.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.293+0000 7f502573a640 1 --2- 172.21.15.81:0/2173931198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f50200add30 0x7f50200cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:09.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.293+0000 7f5024f39640 1 -- 172.21.15.81:0/2173931198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f50200add30 msgr2=0x7f50200cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:09.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.293+0000 7f5024f39640 1 --2- 172.21.15.81:0/2173931198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f50200add30 0x7f50200cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:09.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.293+0000 7f5024f39640 1 -- 172.21.15.81:0/2173931198 >> v1:172.21.15.93:6789/0 conn(0x7f50201444d0 legacy=0x7f502015e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:09.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.293+0000 7f5024f39640 1 -- 172.21.15.81:0/2173931198 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50200b3910 con 0x7f50200b3da0 2023-11-08T16:42:09.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.294+0000 7f5024f39640 1 --2- 172.21.15.81:0/2173931198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200b3da0 0x7f50200cc410 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f5010009870 tx=0x7f501002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab63da5a239be902 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:09.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.294+0000 7f5017fff640 1 -- 172.21.15.81:0/2173931198 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010031640 con 0x7f50200b3da0 2023-11-08T16:42:09.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.294+0000 7f5017fff640 1 -- 172.21.15.81:0/2173931198 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50100317a0 con 0x7f50200b3da0 2023-11-08T16:42:09.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.294+0000 7f5017fff640 1 -- 172.21.15.81:0/2173931198 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010031b30 con 0x7f50200b3da0 2023-11-08T16:42:09.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.295+0000 7f502673c640 1 -- 172.21.15.81:0/2173931198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200b3da0 msgr2=0x7f50200cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:09.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.295+0000 7f502673c640 1 --2- 172.21.15.81:0/2173931198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200b3da0 0x7f50200cc410 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f5010009870 tx=0x7f501002b8f0 comp rx=0 tx=0).stop 2023-11-08T16:42:09.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.295+0000 7f502673c640 1 -- 172.21.15.81:0/2173931198 shutdown_connections 2023-11-08T16:42:09.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.295+0000 7f502673c640 1 --2- 172.21.15.81:0/2173931198 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200b3da0 0x7f50200cc410 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:09.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.295+0000 7f502673c640 1 --2- 172.21.15.81:0/2173931198 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f50200add30 0x7f50200cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:09.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.295+0000 7f502673c640 1 -- 172.21.15.81:0/2173931198 >> 172.21.15.81:0/2173931198 conn(0x7f50200b07c0 msgr2=0x7f50200b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:09.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.295+0000 7f502673c640 1 -- 172.21.15.81:0/2173931198 shutdown_connections 2023-11-08T16:42:09.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.295+0000 7f502673c640 1 -- 172.21.15.81:0/2173931198 wait complete. 2023-11-08T16:42:09.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.296+0000 7f502673c640 1 Processor -- start 2023-11-08T16:42:09.301 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.297+0000 7f502673c640 1 -- start start 2023-11-08T16:42:09.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.297+0000 7f502673c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f50200add30 0x7f50200c7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:09.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.297+0000 7f502673c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200b3da0 0x7f5020140680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:09.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f502673c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f50201444d0 0x7f5020070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:09.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f502673c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200c4390 con 0x7f50200b3da0 2023-11-08T16:42:09.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f502673c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200c3ad0 con 0x7f50200add30 2023-11-08T16:42:09.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f502673c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50200c3d40 con 0x7f50201444d0 2023-11-08T16:42:09.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f5024f39640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200b3da0 0x7f5020140680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:09.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f5024f39640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200b3da0 0x7f5020140680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58864/0 (socket says 172.21.15.81:58864) 2023-11-08T16:42:09.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f5024f39640 1 -- 172.21.15.81:0/1537920558 learned_addr learned my addr 172.21.15.81:0/1537920558 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:09.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f5024f39640 1 -- 172.21.15.81:0/1537920558 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f50201444d0 msgr2=0x7f5020070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:09.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f5024f39640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f50201444d0 0x7f5020070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:09.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f5024f39640 1 -- 172.21.15.81:0/1537920558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f50200add30 msgr2=0x7f50200c7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:09.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f5024f39640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f50200add30 0x7f50200c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:09.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f5024f39640 1 -- 172.21.15.81:0/1537920558 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50100094d0 con 0x7f50200b3da0 2023-11-08T16:42:09.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.298+0000 7f5024f39640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200b3da0 0x7f5020140680 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f501002b930 tx=0x7f50100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:09.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.299+0000 7f5015ffb640 1 -- 172.21.15.81:0/1537920558 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f501003ac60 con 0x7f50200b3da0 2023-11-08T16:42:09.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.299+0000 7f5015ffb640 1 -- 172.21.15.81:0/1537920558 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501003ade0 con 0x7f50200b3da0 2023-11-08T16:42:09.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.299+0000 7f5015ffb640 1 -- 172.21.15.81:0/1537920558 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010042630 con 0x7f50200b3da0 2023-11-08T16:42:09.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.299+0000 7f502673c640 1 -- 172.21.15.81:0/1537920558 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50200c4020 con 0x7f50200b3da0 2023-11-08T16:42:09.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.299+0000 7f502673c640 1 -- 172.21.15.81:0/1537920558 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50201410f0 con 0x7f50200b3da0 2023-11-08T16:42:09.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.300+0000 7f502673c640 1 -- 172.21.15.81:0/1537920558 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f502004ece0 con 0x7f50200b3da0 2023-11-08T16:42:09.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.301+0000 7f5015ffb640 1 -- 172.21.15.81:0/1537920558 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f501002f050 con 0x7f50200b3da0 2023-11-08T16:42:09.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.302+0000 7f5015ffb640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f500406aea0 0x7f500406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:09.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.302+0000 7f5015ffb640 1 -- 172.21.15.81:0/1537920558 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f50100ab370 con 0x7f50200b3da0 2023-11-08T16:42:09.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.302+0000 7f502573a640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f500406aea0 0x7f500406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:09.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.302+0000 7f502573a640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f500406aea0 0x7f500406d360 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f501c006fd0 tx=0x7f501c008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:09.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.306+0000 7f5015ffb640 1 -- 172.21.15.81:0/1537920558 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f50100761b0 con 0x7f50200b3da0 2023-11-08T16:42:09.519 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:09.518+0000 7f502673c640 1 -- 172.21.15.81:0/1537920558 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f50200c4540 con 0x7f50200b3da0 2023-11-08T16:42:10.114 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.112+0000 7f5015ffb640 1 -- 172.21.15.81:0/1537920558 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v122) v1 ==== 104+0+0 (secure 0 0 0) 0x7f50200c4540 con 0x7f50200b3da0 2023-11-08T16:42:10.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.115+0000 7f502673c640 1 -- 172.21.15.81:0/1537920558 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f500406aea0 msgr2=0x7f500406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:10.117 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.115+0000 7f502673c640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f500406aea0 0x7f500406d360 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f501c006fd0 tx=0x7f501c008040 comp rx=0 tx=0).stop 2023-11-08T16:42:10.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.115+0000 7f502673c640 1 -- 172.21.15.81:0/1537920558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200b3da0 msgr2=0x7f5020140680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:10.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.115+0000 7f502673c640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200b3da0 0x7f5020140680 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f501002b930 tx=0x7f50100046c0 comp rx=0 tx=0).stop 2023-11-08T16:42:10.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.115+0000 7f502673c640 1 -- 172.21.15.81:0/1537920558 shutdown_connections 2023-11-08T16:42:10.118 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.116+0000 7f502673c640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f500406aea0 0x7f500406d360 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:10.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.116+0000 7f502673c640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f50201444d0 0x7f5020070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:10.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.116+0000 7f502673c640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f50200b3da0 0x7f5020140680 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:10.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.116+0000 7f502673c640 1 --2- 172.21.15.81:0/1537920558 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f50200add30 0x7f50200c7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:10.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.116+0000 7f502673c640 1 -- 172.21.15.81:0/1537920558 >> 172.21.15.81:0/1537920558 conn(0x7f50200b07c0 msgr2=0x7f502015d160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:10.119 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.116+0000 7f502673c640 1 -- 172.21.15.81:0/1537920558 shutdown_connections 2023-11-08T16:42:10.120 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.116+0000 7f502673c640 1 -- 172.21.15.81:0/1537920558 wait complete. 2023-11-08T16:42:10.139 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-08T16:42:10.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.348+0000 7f6c38434640 1 Processor -- start 2023-11-08T16:42:10.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.349+0000 7f6c38434640 1 -- start start 2023-11-08T16:42:10.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.350+0000 7f6c38434640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6c3015a3b0 0x7f6c300ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:10.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.350+0000 7f6c38434640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c300cf060 con 0x7f6c300c9a90 2023-11-08T16:42:10.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.350+0000 7f6c38434640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c300cde00 con 0x7f6c3006b180 2023-11-08T16:42:10.352 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.350+0000 7f6c38434640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c300ce0e0 con 0x7f6c3015a3b0 2023-11-08T16:42:10.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.350+0000 7f6c359cf640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6c3015a3b0 0x7f6c300ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:10.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.350+0000 7f6c359cf640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f6c3015a3b0 0x7f6c300ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58874/0 (socket says 172.21.15.81:58874) 2023-11-08T16:42:10.353 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.350+0000 7f6c359cf640 1 -- 172.21.15.81:0/2303336201 learned_addr learned my addr 172.21.15.81:0/2303336201 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:10.354 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.351+0000 7f6c359cf640 1 -- 172.21.15.81:0/2303336201 >> v1:172.21.15.93:6789/0 conn(0x7f6c3006b180 legacy=0x7f6c3006b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:10.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.351+0000 7f6c359cf640 1 -- 172.21.15.81:0/2303336201 >> v1:172.21.15.81:6789/0 conn(0x7f6c300c9a90 legacy=0x7f6c30071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:10.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.351+0000 7f6c359cf640 1 -- 172.21.15.81:0/2303336201 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30071230 con 0x7f6c3015a3b0 2023-11-08T16:42:10.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.352+0000 7f6c359cf640 1 --2- 172.21.15.81:0/2303336201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c3015a3b0 0x7f6c300ce4f0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009870 tx=0x7f6c2402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8b7372fa64a0d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:10.355 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.352+0000 7f6c351ce640 1 -- 172.21.15.81:0/2303336201 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24030860 con 0x7f6c3015a3b0 2023-11-08T16:42:10.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.352+0000 7f6c351ce640 1 -- 172.21.15.81:0/2303336201 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c240309c0 con 0x7f6c3015a3b0 2023-11-08T16:42:10.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.352+0000 7f6c351ce640 1 -- 172.21.15.81:0/2303336201 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24030d50 con 0x7f6c3015a3b0 2023-11-08T16:42:10.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.353+0000 7f6c38434640 1 -- 172.21.15.81:0/2303336201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c3015a3b0 msgr2=0x7f6c300ce4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:10.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.353+0000 7f6c38434640 1 --2- 172.21.15.81:0/2303336201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c3015a3b0 0x7f6c300ce4f0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009870 tx=0x7f6c2402b9a0 comp rx=0 tx=0).stop 2023-11-08T16:42:10.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.353+0000 7f6c38434640 1 -- 172.21.15.81:0/2303336201 shutdown_connections 2023-11-08T16:42:10.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.353+0000 7f6c38434640 1 --2- 172.21.15.81:0/2303336201 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c3015a3b0 0x7f6c300ce4f0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:10.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.353+0000 7f6c38434640 1 -- 172.21.15.81:0/2303336201 >> 172.21.15.81:0/2303336201 conn(0x7f6c300cf440 msgr2=0x7f6c300cf820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:10.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.353+0000 7f6c38434640 1 -- 172.21.15.81:0/2303336201 shutdown_connections 2023-11-08T16:42:10.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.353+0000 7f6c38434640 1 -- 172.21.15.81:0/2303336201 wait complete. 2023-11-08T16:42:10.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.355+0000 7f6c38434640 1 Processor -- start 2023-11-08T16:42:10.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.355+0000 7f6c38434640 1 -- start start 2023-11-08T16:42:10.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c38434640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3006b180 0x7f6c30153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:10.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c38434640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c300c9a90 0x7f6c300c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:10.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c38434640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c3015a3b0 0x7f6c300b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:10.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c38434640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c301535c0 con 0x7f6c300c9a90 2023-11-08T16:42:10.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c38434640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300b3450 con 0x7f6c3015a3b0 2023-11-08T16:42:10.361 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c38434640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300b3de0 con 0x7f6c3006b180 2023-11-08T16:42:10.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c359cf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c300c9a90 0x7f6c300c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:10.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c359cf640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c300c9a90 0x7f6c300c3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:58886/0 (socket says 172.21.15.81:58886) 2023-11-08T16:42:10.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c359cf640 1 -- 172.21.15.81:0/3817734824 learned_addr learned my addr 172.21.15.81:0/3817734824 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:10.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c359cf640 1 -- 172.21.15.81:0/3817734824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3006b180 msgr2=0x7f6c30153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:10.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c361d0640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3006b180 0x7f6c30153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:10.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c359cf640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3006b180 0x7f6c30153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:10.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c359cf640 1 -- 172.21.15.81:0/3817734824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c3015a3b0 msgr2=0x7f6c300b4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:10.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c369d1640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c3015a3b0 0x7f6c300b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:10.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c359cf640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c3015a3b0 0x7f6c300b4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:10.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c359cf640 1 -- 172.21.15.81:0/3817734824 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c240094d0 con 0x7f6c300c9a90 2023-11-08T16:42:10.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.356+0000 7f6c369d1640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c3015a3b0 0x7f6c300b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:10.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.357+0000 7f6c359cf640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c300c9a90 0x7f6c300c3fc0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009990 tx=0x7f6c24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:10.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.357+0000 7f6c2effd640 1 -- 172.21.15.81:0/3817734824 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24039e20 con 0x7f6c300c9a90 2023-11-08T16:42:10.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.357+0000 7f6c2effd640 1 -- 172.21.15.81:0/3817734824 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c24043400 con 0x7f6c300c9a90 2023-11-08T16:42:10.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.357+0000 7f6c38434640 1 -- 172.21.15.81:0/3817734824 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c300b36a0 con 0x7f6c300c9a90 2023-11-08T16:42:10.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.357+0000 7f6c2effd640 1 -- 172.21.15.81:0/3817734824 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240437b0 con 0x7f6c300c9a90 2023-11-08T16:42:10.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.357+0000 7f6c38434640 1 -- 172.21.15.81:0/3817734824 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c300b4030 con 0x7f6c300c9a90 2023-11-08T16:42:10.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.358+0000 7f6c38434640 1 -- 172.21.15.81:0/3817734824 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c30074440 con 0x7f6c300c9a90 2023-11-08T16:42:10.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.359+0000 7f6c2effd640 1 -- 172.21.15.81:0/3817734824 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6c24043910 con 0x7f6c300c9a90 2023-11-08T16:42:10.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.360+0000 7f6c2effd640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1c06ae70 0x7f6c1c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:10.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.360+0000 7f6c2effd640 1 -- 172.21.15.81:0/3817734824 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 6025+0+0 (secure 0 0 0) 0x7f6c2403e070 con 0x7f6c300c9a90 2023-11-08T16:42:10.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.361+0000 7f6c361d0640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1c06ae70 0x7f6c1c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:10.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.361+0000 7f6c361d0640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1c06ae70 0x7f6c1c06d330 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6c300c3370 tx=0x7f6c280072f0 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:10.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.365+0000 7f6c2effd640 1 -- 172.21.15.81:0/3817734824 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f6c24077a50 con 0x7f6c300c9a90 2023-11-08T16:42:10.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:10.540+0000 7f6c38434640 1 -- 172.21.15.81:0/3817734824 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6c300660e0 con 0x7f6c300c9a90 2023-11-08T16:42:11.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.133+0000 7f6c2effd640 1 -- 172.21.15.81:0/3817734824 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v133) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6c2407b070 con 0x7f6c300c9a90 2023-11-08T16:42:11.135 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_metadata' removed 2023-11-08T16:42:11.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.136+0000 7f6c38434640 1 -- 172.21.15.81:0/3817734824 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1c06ae70 msgr2=0x7f6c1c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:11.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.136+0000 7f6c38434640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1c06ae70 0x7f6c1c06d330 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f6c300c3370 tx=0x7f6c280072f0 comp rx=0 tx=0).stop 2023-11-08T16:42:11.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.137+0000 7f6c38434640 1 -- 172.21.15.81:0/3817734824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c300c9a90 msgr2=0x7f6c300c3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:11.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.137+0000 7f6c38434640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c300c9a90 0x7f6c300c3fc0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f6c24009990 tx=0x7f6c24004730 comp rx=0 tx=0).stop 2023-11-08T16:42:11.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.137+0000 7f6c38434640 1 -- 172.21.15.81:0/3817734824 shutdown_connections 2023-11-08T16:42:11.139 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.137+0000 7f6c38434640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f6c1c06ae70 0x7f6c1c06d330 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:11.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.137+0000 7f6c38434640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6c3015a3b0 0x7f6c300b4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:11.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.137+0000 7f6c38434640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f6c300c9a90 0x7f6c300c3fc0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:11.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.137+0000 7f6c38434640 1 --2- 172.21.15.81:0/3817734824 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f6c3006b180 0x7f6c30153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:11.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.137+0000 7f6c38434640 1 -- 172.21.15.81:0/3817734824 >> 172.21.15.81:0/3817734824 conn(0x7f6c300cf440 msgr2=0x7f6c300aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:11.140 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.137+0000 7f6c38434640 1 -- 172.21.15.81:0/3817734824 shutdown_connections 2023-11-08T16:42:11.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.138+0000 7f6c38434640 1 -- 172.21.15.81:0/3817734824 wait complete. 2023-11-08T16:42:11.164 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-08T16:42:11.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.375+0000 7faac68b4640 1 Processor -- start 2023-11-08T16:42:11.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.375+0000 7faac68b4640 1 -- start start 2023-11-08T16:42:11.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.377+0000 7faac68b4640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faac0154400 0x7faac006a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:11.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.377+0000 7faac68b4640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac015d940 con 0x7faac00b3850 2023-11-08T16:42:11.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.377+0000 7faac68b4640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac00af240 con 0x7faac0154020 2023-11-08T16:42:11.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.377+0000 7faac68b4640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac015d730 con 0x7faac0154400 2023-11-08T16:42:11.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.377+0000 7faabf7fe640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7faac0154400 0x7faac006a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:11.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.377+0000 7faac4e51640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7faac00b3850 0x7faac00cfc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54316/0 (socket says 172.21.15.81:54316) 2023-11-08T16:42:11.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.377+0000 7faac4e51640 1 -- 172.21.15.81:0/3045190307 learned_addr learned my addr 172.21.15.81:0/3045190307 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:11.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.378+0000 7faabeffd640 1 -- 172.21.15.81:0/3045190307 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2930798833 0 0) 0x7faac015d940 con 0x7faac00b3850 2023-11-08T16:42:11.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.378+0000 7faabf7fe640 1 -- 172.21.15.81:0/3045190307 >> v1:172.21.15.93:6789/0 conn(0x7faac0154020 legacy=0x7faac00aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:11.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.378+0000 7faabf7fe640 1 -- 172.21.15.81:0/3045190307 >> v1:172.21.15.81:6789/0 conn(0x7faac00b3850 legacy=0x7faac00cfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:11.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.378+0000 7faabf7fe640 1 -- 172.21.15.81:0/3045190307 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac00af4f0 con 0x7faac0154400 2023-11-08T16:42:11.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.379+0000 7faabf7fe640 1 --2- 172.21.15.81:0/3045190307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faac0154400 0x7faac006a070 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7faaa80097d0 tx=0x7faaa802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6466aea2910d51b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:11.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.379+0000 7faabeffd640 1 -- 172.21.15.81:0/3045190307 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8031830 con 0x7faac0154400 2023-11-08T16:42:11.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.379+0000 7faabeffd640 1 -- 172.21.15.81:0/3045190307 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa8031990 con 0x7faac0154400 2023-11-08T16:42:11.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.379+0000 7faabeffd640 1 -- 172.21.15.81:0/3045190307 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa8031d20 con 0x7faac0154400 2023-11-08T16:42:11.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.380+0000 7faac68b4640 1 -- 172.21.15.81:0/3045190307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faac0154400 msgr2=0x7faac006a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:11.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.380+0000 7faac68b4640 1 --2- 172.21.15.81:0/3045190307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faac0154400 0x7faac006a070 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7faaa80097d0 tx=0x7faaa802b8d0 comp rx=0 tx=0).stop 2023-11-08T16:42:11.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.380+0000 7faac68b4640 1 -- 172.21.15.81:0/3045190307 shutdown_connections 2023-11-08T16:42:11.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.380+0000 7faac68b4640 1 --2- 172.21.15.81:0/3045190307 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faac0154400 0x7faac006a070 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:11.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.380+0000 7faac68b4640 1 -- 172.21.15.81:0/3045190307 >> 172.21.15.81:0/3045190307 conn(0x7faac006b180 msgr2=0x7faac006b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:11.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.380+0000 7faac68b4640 1 -- 172.21.15.81:0/3045190307 shutdown_connections 2023-11-08T16:42:11.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.380+0000 7faac68b4640 1 -- 172.21.15.81:0/3045190307 wait complete. 2023-11-08T16:42:11.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.382+0000 7faac68b4640 1 Processor -- start 2023-11-08T16:42:11.387 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.382+0000 7faac68b4640 1 -- start start 2023-11-08T16:42:11.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.383+0000 7faac68b4640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faac00b3850 0x7faac01521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:11.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.383+0000 7faac68b4640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faac0154020 0x7faac0152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:11.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.383+0000 7faac68b4640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faac0154400 0x7faac013a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:11.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.383+0000 7faac68b4640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac0148f00 con 0x7faac0154400 2023-11-08T16:42:11.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.383+0000 7faac68b4640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac01387c0 con 0x7faac00b3850 2023-11-08T16:42:11.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.383+0000 7faac68b4640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac01566c0 con 0x7faac0154020 2023-11-08T16:42:11.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.383+0000 7faac4e51640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faac0154400 0x7faac013a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:11.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.383+0000 7faac4e51640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faac0154400 0x7faac013a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51346/0 (socket says 172.21.15.81:51346) 2023-11-08T16:42:11.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.383+0000 7faac4e51640 1 -- 172.21.15.81:0/2712694724 learned_addr learned my addr 172.21.15.81:0/2712694724 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:11.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faabffff640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faac00b3850 0x7faac01521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:11.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faabf7fe640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faac0154020 0x7faac0152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:11.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faac4e51640 1 -- 172.21.15.81:0/2712694724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faac0154020 msgr2=0x7faac0152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:11.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faac4e51640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faac0154020 0x7faac0152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:11.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faac4e51640 1 -- 172.21.15.81:0/2712694724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faac00b3850 msgr2=0x7faac01521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:11.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faac4e51640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faac00b3850 0x7faac01521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:11.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faac4e51640 1 -- 172.21.15.81:0/2712694724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa8009400 con 0x7faac0154400 2023-11-08T16:42:11.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faac4e51640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faac0154400 0x7faac013a5d0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7faab800c390 tx=0x7faab800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:11.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faabcff9640 1 -- 172.21.15.81:0/2712694724 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab80074e0 con 0x7faac0154400 2023-11-08T16:42:11.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faabcff9640 1 -- 172.21.15.81:0/2712694724 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab8007660 con 0x7faac0154400 2023-11-08T16:42:11.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faac68b4640 1 -- 172.21.15.81:0/2712694724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac0155e20 con 0x7faac0154400 2023-11-08T16:42:11.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.384+0000 7faabcff9640 1 -- 172.21.15.81:0/2712694724 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8004a70 con 0x7faac0154400 2023-11-08T16:42:11.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.385+0000 7faac68b4640 1 -- 172.21.15.81:0/2712694724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac01563e0 con 0x7faac0154400 2023-11-08T16:42:11.393 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.386+0000 7faac68b4640 1 -- 172.21.15.81:0/2712694724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac0074440 con 0x7faac0154400 2023-11-08T16:42:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.387+0000 7faabcff9640 1 -- 172.21.15.81:0/2712694724 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faab8004bd0 con 0x7faac0154400 2023-11-08T16:42:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.388+0000 7faabcff9640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faaa006ae70 0x7faaa006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.388+0000 7faabcff9640 1 -- 172.21.15.81:0/2712694724 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5587+0+0 (secure 0 0 0) 0x7faab80893b0 con 0x7faac0154400 2023-11-08T16:42:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.388+0000 7faabffff640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faaa006ae70 0x7faaa006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:11.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.389+0000 7faabffff640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faaa006ae70 0x7faaa006d330 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7faac01f6400 tx=0x7faab0009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:11.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.393+0000 7faabcff9640 1 -- 172.21.15.81:0/2712694724 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7faab8054430 con 0x7faac0154400 2023-11-08T16:42:11.585 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:11.584+0000 7faac68b4640 1 -- 172.21.15.81:0/2712694724 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faac0138900 con 0x7faac0154400 2023-11-08T16:42:12.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.136+0000 7faabcff9640 1 -- 172.21.15.81:0/2712694724 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v134) v1 ==== 169+0+0 (secure 0 0 0) 0x7faab8057a50 con 0x7faac0154400 2023-11-08T16:42:12.138 INFO:teuthology.orchestra.run.smithi081.stderr:pool 'cephfs_data' removed 2023-11-08T16:42:12.141 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.140+0000 7faac68b4640 1 -- 172.21.15.81:0/2712694724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faaa006ae70 msgr2=0x7faaa006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.140+0000 7faac68b4640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faaa006ae70 0x7faaa006d330 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7faac01f6400 tx=0x7faab0009040 comp rx=0 tx=0).stop 2023-11-08T16:42:12.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.140+0000 7faac68b4640 1 -- 172.21.15.81:0/2712694724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faac0154400 msgr2=0x7faac013a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.142 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.140+0000 7faac68b4640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faac0154400 0x7faac013a5d0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7faab800c390 tx=0x7faab800c850 comp rx=0 tx=0).stop 2023-11-08T16:42:12.143 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.141+0000 7faac68b4640 1 -- 172.21.15.81:0/2712694724 shutdown_connections 2023-11-08T16:42:12.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.141+0000 7faac68b4640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7faaa006ae70 0x7faaa006d330 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.141+0000 7faac68b4640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7faac0154400 0x7faac013a5d0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.141+0000 7faac68b4640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7faac0154020 0x7faac0152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.144 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.141+0000 7faac68b4640 1 --2- 172.21.15.81:0/2712694724 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faac00b3850 0x7faac01521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.141+0000 7faac68b4640 1 -- 172.21.15.81:0/2712694724 >> 172.21.15.81:0/2712694724 conn(0x7faac006b180 msgr2=0x7faac006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:12.145 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.143+0000 7faac68b4640 1 -- 172.21.15.81:0/2712694724 shutdown_connections 2023-11-08T16:42:12.146 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.143+0000 7faac68b4640 1 -- 172.21.15.81:0/2712694724 wait complete. 2023-11-08T16:42:12.169 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-08T16:42:12.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.365+0000 7fde08d7d640 1 Processor -- start 2023-11-08T16:42:12.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.365+0000 7fde08d7d640 1 -- start start 2023-11-08T16:42:12.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.366+0000 7fde08d7d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 0x7fde040c9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.366+0000 7fde08d7d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fde0406dcb0 0x7fde04070d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.366+0000 7fde08d7d640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde04070640 con 0x7fde0415a460 2023-11-08T16:42:12.368 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.366+0000 7fde08d7d640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde0415b400 con 0x7fde0406dcb0 2023-11-08T16:42:12.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.366+0000 7fde08d7d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde0406c2b0 con 0x7fde041579e0 2023-11-08T16:42:12.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.367+0000 7fde02d9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fde0406dcb0 0x7fde04070d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.367+0000 7fde0259c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 0x7fde040c9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.367+0000 7fde02d9d640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fde0406dcb0 0x7fde04070d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:34334/0 (socket says 172.21.15.81:34334) 2023-11-08T16:42:12.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.367+0000 7fde02d9d640 1 -- 172.21.15.81:0/2557136790 learned_addr learned my addr 172.21.15.81:0/2557136790 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:12.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.367+0000 7fde0259c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 0x7fde040c9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3301/0 says I am v2:172.21.15.81:58572/0 (socket says 172.21.15.81:58572) 2023-11-08T16:42:12.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.368+0000 7fde02d9d640 1 -- 172.21.15.81:0/2557136790 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 msgr2=0x7fde040c9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:12.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.368+0000 7fde02d9d640 1 --2- 172.21.15.81:0/2557136790 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 0x7fde040c9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.368+0000 7fde02d9d640 1 -- 172.21.15.81:0/2557136790 >> v1:172.21.15.93:6789/0 conn(0x7fde0415a460 legacy=0x7fde0406d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.368+0000 7fde02d9d640 1 -- 172.21.15.81:0/2557136790 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde0415a8e0 con 0x7fde0406dcb0 2023-11-08T16:42:12.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.368+0000 7fde0259c640 1 --2- 172.21.15.81:0/2557136790 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 0x7fde040c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-08T16:42:12.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.369+0000 7fde02d9d640 1 --2- 172.21.15.81:0/2557136790 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde0406dcb0 0x7fde04070d90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d150 tx=0x7fddfc02e160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0af593e2d5d37ef server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:12.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.370+0000 7fde01d9b640 1 -- 172.21.15.81:0/2557136790 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00b0d0 con 0x7fde0406dcb0 2023-11-08T16:42:12.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.370+0000 7fde01d9b640 1 -- 172.21.15.81:0/2557136790 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc00b230 con 0x7fde0406dcb0 2023-11-08T16:42:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.370+0000 7fde01d9b640 1 -- 172.21.15.81:0/2557136790 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc009600 con 0x7fde0406dcb0 2023-11-08T16:42:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.370+0000 7fde08d7d640 1 -- 172.21.15.81:0/2557136790 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde0406dcb0 msgr2=0x7fde04070d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.370+0000 7fde08d7d640 1 --2- 172.21.15.81:0/2557136790 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde0406dcb0 0x7fde04070d90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00d150 tx=0x7fddfc02e160 comp rx=0 tx=0).stop 2023-11-08T16:42:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.371+0000 7fde08d7d640 1 -- 172.21.15.81:0/2557136790 shutdown_connections 2023-11-08T16:42:12.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.371+0000 7fde08d7d640 1 --2- 172.21.15.81:0/2557136790 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde0406dcb0 0x7fde04070d90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.371+0000 7fde08d7d640 1 --2- 172.21.15.81:0/2557136790 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 0x7fde040c9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.371+0000 7fde08d7d640 1 -- 172.21.15.81:0/2557136790 >> 172.21.15.81:0/2557136790 conn(0x7fde040ceb70 msgr2=0x7fde040cef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.371+0000 7fde08d7d640 1 -- 172.21.15.81:0/2557136790 shutdown_connections 2023-11-08T16:42:12.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.371+0000 7fde08d7d640 1 -- 172.21.15.81:0/2557136790 wait complete. 2023-11-08T16:42:12.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.372+0000 7fde08d7d640 1 Processor -- start 2023-11-08T16:42:12.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.373+0000 7fde08d7d640 1 -- start start 2023-11-08T16:42:12.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.373+0000 7fde08d7d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde0406dcb0 0x7fde041ee160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.373+0000 7fde08d7d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 0x7fde040b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.373+0000 7fde08d7d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde0415a460 0x7fde040cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde08d7d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde040b4550 con 0x7fde0415a460 2023-11-08T16:42:12.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde08d7d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde041ee6a0 con 0x7fde0406dcb0 2023-11-08T16:42:12.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde08d7d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde040b36a0 con 0x7fde041579e0 2023-11-08T16:42:12.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde02d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde0415a460 0x7fde040cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde02d9d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde0415a460 0x7fde040cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51354/0 (socket says 172.21.15.81:51354) 2023-11-08T16:42:12.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde02d9d640 1 -- 172.21.15.81:0/2324225277 learned_addr learned my addr 172.21.15.81:0/2324225277 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde0259c640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde0406dcb0 0x7fde041ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fddfbfff640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 0x7fde040b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde02d9d640 1 -- 172.21.15.81:0/2324225277 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 msgr2=0x7fde040b19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde02d9d640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 0x7fde040b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.381 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde02d9d640 1 -- 172.21.15.81:0/2324225277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde0406dcb0 msgr2=0x7fde041ee160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde02d9d640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde0406dcb0 0x7fde041ee160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde02d9d640 1 -- 172.21.15.81:0/2324225277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf4009400 con 0x7fde0415a460 2023-11-08T16:42:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.374+0000 7fde0259c640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde0406dcb0 0x7fde041ee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.375+0000 7fde02d9d640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde0415a460 0x7fde040cd6f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fddfc038040 tx=0x7fddfc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:12.382 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.375+0000 7fddfaffd640 1 -- 172.21.15.81:0/2324225277 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc00b720 con 0x7fde0415a460 2023-11-08T16:42:12.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.375+0000 7fddfaffd640 1 -- 172.21.15.81:0/2324225277 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddfc009e30 con 0x7fde0415a460 2023-11-08T16:42:12.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.375+0000 7fddfaffd640 1 -- 172.21.15.81:0/2324225277 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddfc040880 con 0x7fde0415a460 2023-11-08T16:42:12.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.375+0000 7fde08d7d640 1 -- 172.21.15.81:0/2324225277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddfc00bd40 con 0x7fde0415a460 2023-11-08T16:42:12.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.375+0000 7fde08d7d640 1 -- 172.21.15.81:0/2324225277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde040b3c20 con 0x7fde0415a460 2023-11-08T16:42:12.383 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.376+0000 7fde08d7d640 1 -- 172.21.15.81:0/2324225277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde040b46c0 con 0x7fde0415a460 2023-11-08T16:42:12.384 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.377+0000 7fddfaffd640 1 -- 172.21.15.81:0/2324225277 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fddfc0409e0 con 0x7fde0415a460 2023-11-08T16:42:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.378+0000 7fddfaffd640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdde406ae70 0x7fdde406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.378+0000 7fddfaffd640 1 -- 172.21.15.81:0/2324225277 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fddfc0ae440 con 0x7fde0415a460 2023-11-08T16:42:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.378+0000 7fde0259c640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdde406ae70 0x7fdde406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.379+0000 7fde0259c640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdde406ae70 0x7fdde406d330 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009bb0 tx=0x7fddf4008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:12.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.382+0000 7fddfaffd640 1 -- 172.21.15.81:0/2324225277 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fddfc0796a0 con 0x7fde0415a460 2023-11-08T16:42:12.573 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.571+0000 7fde08d7d640 1 -- 172.21.15.81:0/2324225277 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fde04074440 con 0x7fde0415a460 2023-11-08T16:42:12.577 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.576+0000 7fddfaffd640 1 -- 172.21.15.81:0/2324225277 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fddfc07ccc0 con 0x7fde0415a460 2023-11-08T16:42:12.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 -- 172.21.15.81:0/2324225277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdde406ae70 msgr2=0x7fdde406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdde406ae70 0x7fdde406d330 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fddf4009bb0 tx=0x7fddf4008040 comp rx=0 tx=0).stop 2023-11-08T16:42:12.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 -- 172.21.15.81:0/2324225277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde0415a460 msgr2=0x7fde040cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.581 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde0415a460 0x7fde040cd6f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fddfc038040 tx=0x7fddfc004a40 comp rx=0 tx=0).stop 2023-11-08T16:42:12.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 -- 172.21.15.81:0/2324225277 shutdown_connections 2023-11-08T16:42:12.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fdde406ae70 0x7fdde406d330 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fde0415a460 0x7fde040cd6f0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.582 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fde041579e0 0x7fde040b19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 --2- 172.21.15.81:0/2324225277 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fde0406dcb0 0x7fde041ee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 -- 172.21.15.81:0/2324225277 >> 172.21.15.81:0/2324225277 conn(0x7fde040ceb70 msgr2=0x7fde040cc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:12.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 -- 172.21.15.81:0/2324225277 shutdown_connections 2023-11-08T16:42:12.583 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.579+0000 7fde08d7d640 1 -- 172.21.15.81:0/2324225277 wait complete. 2023-11-08T16:42:12.598 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection' 2023-11-08T16:42:12.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.801+0000 7f8fa4705640 1 Processor -- start 2023-11-08T16:42:12.803 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.802+0000 7f8fa4705640 1 -- start start 2023-11-08T16:42:12.804 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.803+0000 7f8fa4705640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f9c0b3170 0x7f8f9c15b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.803+0000 7f8fa4705640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9c0b2310 con 0x7f8f9c0b3550 2023-11-08T16:42:12.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.803+0000 7f8fa4705640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f9c0b25f0 con 0x7f8f9c06c9f0 2023-11-08T16:42:12.805 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.803+0000 7f8fa4705640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c0b2bb0 con 0x7f8f9c0b3170 2023-11-08T16:42:12.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.804+0000 7f8fa1ca0640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f8f9c0b3550 0x7f8f9c0b1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54332/0 (socket says 172.21.15.81:54332) 2023-11-08T16:42:12.808 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.804+0000 7f8fa1ca0640 1 -- 172.21.15.81:0/616698096 learned_addr learned my addr 172.21.15.81:0/616698096 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:12.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.804+0000 7f8fa24a1640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f9c0b3170 0x7f8f9c15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.804+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401706914 0 0) 0x7f8f9c0b2310 con 0x7f8f9c0b3550 2023-11-08T16:42:12.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.804+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f88003520 con 0x7f8f9c0b3550 2023-11-08T16:42:12.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.804+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2623184812 0 0) 0x7f8f88003520 con 0x7f8f9c0b3550 2023-11-08T16:42:12.809 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.804+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f9c0b3170 msgr2=0x7f8f9c15b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:12.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.804+0000 7f8fa149f640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f9c0b3170 0x7f8f9c15b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.804+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 >> v1:172.21.15.93:6789/0 conn(0x7f8f9c06c9f0 legacy=0x7f8f9c144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c0b28d0 con 0x7f8f9c0b3550 2023-11-08T16:42:12.810 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f8f900019f0 con 0x7f8f9c0b3550 2023-11-08T16:42:12.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 >> v1:172.21.15.81:6789/0 conn(0x7f8f9c0b3550 legacy=0x7f8f9c0b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa149f640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f880039a0 0x7f8f88003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa149f640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f9c0b3170 0x7f8f880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa149f640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f88004d00 0x7f8f88005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.811 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c0b2bb0 con 0x7f8f88004d00 2023-11-08T16:42:12.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8800ba60 con 0x7f8f880039a0 2023-11-08T16:42:12.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8800bc20 con 0x7f8f9c0b3170 2023-11-08T16:42:12.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa2ca2640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f88004d00 0x7f8f88005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.812 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa24a1640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f880039a0 0x7f8f88003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa1ca0640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f9c0b3170 0x7f8f880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa2ca2640 1 -- 172.21.15.81:0/616698096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f9c0b3170 msgr2=0x7f8f880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:12.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa2ca2640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f9c0b3170 0x7f8f880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.813 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa2ca2640 1 -- 172.21.15.81:0/616698096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f880039a0 msgr2=0x7f8f88003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:12.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.805+0000 7f8fa2ca2640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f880039a0 0x7f8f88003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.806+0000 7f8fa2ca2640 1 -- 172.21.15.81:0/616698096 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f88003520 con 0x7f8f88004d00 2023-11-08T16:42:12.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.807+0000 7f8fa2ca2640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f88004d00 0x7f8f88005180 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f8f94001720 tx=0x7f8f880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a90d77aec2259e89 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:12.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.807+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f940052d0 con 0x7f8f88004d00 2023-11-08T16:42:12.814 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.807+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9402e050 con 0x7f8f88004d00 2023-11-08T16:42:12.815 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.807+0000 7f8fa149f640 1 -- 172.21.15.81:0/616698096 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f940035f0 con 0x7f8f88004d00 2023-11-08T16:42:12.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.807+0000 7f8fa4705640 1 -- 172.21.15.81:0/616698096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f88004d00 msgr2=0x7f8f88005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.807+0000 7f8fa4705640 1 --2- 172.21.15.81:0/616698096 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f88004d00 0x7f8f88005180 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f8f94001720 tx=0x7f8f880057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.818 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.807+0000 7f8fa2ca2640 1 -- 172.21.15.81:0/616698096 reap_dead start 2023-11-08T16:42:12.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.808+0000 7f8fa4705640 1 -- 172.21.15.81:0/616698096 shutdown_connections 2023-11-08T16:42:12.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.808+0000 7f8fa4705640 1 -- 172.21.15.81:0/616698096 >> 172.21.15.81:0/616698096 conn(0x7f8f9c06a6a0 msgr2=0x7f8f9c0b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:12.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.808+0000 7f8fa4705640 1 -- 172.21.15.81:0/616698096 shutdown_connections 2023-11-08T16:42:12.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.808+0000 7f8fa4705640 1 -- 172.21.15.81:0/616698096 wait complete. 2023-11-08T16:42:12.819 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.809+0000 7f8fa4705640 1 Processor -- start 2023-11-08T16:42:12.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.810+0000 7f8fa4705640 1 -- start start 2023-11-08T16:42:12.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.810+0000 7f8fa4705640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f88004d00 0x7f8f9c071640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa4705640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f9c06fc30 0x7f8f9c15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.820 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa4705640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f9c15a6a0 0x7f8f9c13a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa4705640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c070c40 con 0x7f8f9c15a6a0 2023-11-08T16:42:12.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa4705640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c1f3260 con 0x7f8f9c06fc30 2023-11-08T16:42:12.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa4705640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9c071b80 con 0x7f8f88004d00 2023-11-08T16:42:12.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa2ca2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f9c15a6a0 0x7f8f9c13a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.821 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa2ca2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f9c15a6a0 0x7f8f9c13a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51380/0 (socket says 172.21.15.81:51380) 2023-11-08T16:42:12.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa2ca2640 1 -- 172.21.15.81:0/2129254955 learned_addr learned my addr 172.21.15.81:0/2129254955 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:12.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa1ca0640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f9c06fc30 0x7f8f9c15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa24a1640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f88004d00 0x7f8f9c071640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa2ca2640 1 -- 172.21.15.81:0/2129254955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f88004d00 msgr2=0x7f8f9c071640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.822 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa2ca2640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f88004d00 0x7f8f9c071640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa2ca2640 1 -- 172.21.15.81:0/2129254955 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f9c06fc30 msgr2=0x7f8f9c15a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:12.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa2ca2640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f9c06fc30 0x7f8f9c15a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:12.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa2ca2640 1 -- 172.21.15.81:0/2129254955 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f9c06d000 con 0x7f8f9c15a6a0 2023-11-08T16:42:12.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa1ca0640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f9c06fc30 0x7f8f9c15a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:12.823 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.811+0000 7f8fa24a1640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f88004d00 0x7f8f9c071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:12.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.812+0000 7f8fa2ca2640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f9c15a6a0 0x7f8f9c13a710 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f8f94034040 tx=0x7f8f94001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:12.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.812+0000 7f8f8effd640 1 -- 172.21.15.81:0/2129254955 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94043070 con 0x7f8f9c15a6a0 2023-11-08T16:42:12.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.812+0000 7f8f8effd640 1 -- 172.21.15.81:0/2129254955 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f94007a30 con 0x7f8f9c15a6a0 2023-11-08T16:42:12.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.812+0000 7f8f8effd640 1 -- 172.21.15.81:0/2129254955 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f940067d0 con 0x7f8f9c15a6a0 2023-11-08T16:42:12.824 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.812+0000 7f8fa4705640 1 -- 172.21.15.81:0/2129254955 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f9c0700c0 con 0x7f8f9c15a6a0 2023-11-08T16:42:12.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.812+0000 7f8fa4705640 1 -- 172.21.15.81:0/2129254955 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f9c070680 con 0x7f8f9c15a6a0 2023-11-08T16:42:12.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.813+0000 7f8fa4705640 1 -- 172.21.15.81:0/2129254955 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f9c1f33d0 con 0x7f8f9c15a6a0 2023-11-08T16:42:12.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.814+0000 7f8f8effd640 1 -- 172.21.15.81:0/2129254955 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8f94006930 con 0x7f8f9c15a6a0 2023-11-08T16:42:12.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.815+0000 7f8f8effd640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f8006ae70 0x7f8f8006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:12.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.815+0000 7f8f8effd640 1 -- 172.21.15.81:0/2129254955 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f8f940aa790 con 0x7f8f9c15a6a0 2023-11-08T16:42:12.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.815+0000 7f8fa24a1640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f8006ae70 0x7f8f8006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:12.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.816+0000 7f8fa24a1640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f8006ae70 0x7f8f8006d330 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009a00 tx=0x7f8f98008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:12.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:12.819+0000 7f8f8effd640 1 -- 172.21.15.81:0/2129254955 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8f9c1f33d0 con 0x7f8f9c15a6a0 2023-11-08T16:42:13.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.079+0000 7f8fa4705640 1 -- 172.21.15.81:0/2129254955 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]} v 0) v1 -- 0x7f8f9c1f33d0 con 0x7f8f9c15a6a0 2023-11-08T16:42:13.152 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.151+0000 7f8f8effd640 1 -- 172.21.15.81:0/2129254955 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_data_scan.TestDataScan.test_fragmented_injection"]}]=0 v595) v1 ==== 145+0+0 (secure 0 0 0) 0x7f8f9c1f33d0 con 0x7f8f9c15a6a0 2023-11-08T16:42:13.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 -- 172.21.15.81:0/2129254955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f8006ae70 msgr2=0x7f8f8006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:13.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f8006ae70 0x7f8f8006d330 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f8f98009a00 tx=0x7f8f98008040 comp rx=0 tx=0).stop 2023-11-08T16:42:13.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 -- 172.21.15.81:0/2129254955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f9c15a6a0 msgr2=0x7f8f9c13a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:13.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f9c15a6a0 0x7f8f9c13a710 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f8f94034040 tx=0x7f8f94001120 comp rx=0 tx=0).stop 2023-11-08T16:42:13.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 -- 172.21.15.81:0/2129254955 shutdown_connections 2023-11-08T16:42:13.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f8f8006ae70 0x7f8f8006d330 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:13.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f8f9c15a6a0 0x7f8f9c13a710 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:13.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8f9c06fc30 0x7f8f9c15a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:13.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 --2- 172.21.15.81:0/2129254955 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f8f88004d00 0x7f8f9c071640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:13.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 -- 172.21.15.81:0/2129254955 >> 172.21.15.81:0/2129254955 conn(0x7f8f9c06a6a0 msgr2=0x7f8f9c1420f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:13.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 -- 172.21.15.81:0/2129254955 shutdown_connections 2023-11-08T16:42:13.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:13.154+0000 7f8fa4705640 1 -- 172.21.15.81:0/2129254955 wait complete. 2023-11-08T16:42:13.175 INFO:tasks.cephfs_test_runner:That when injecting a dentry into a fragmented directory, we put it in the right fragment. ... ERROR 2023-11-08T16:42:13.176 INFO:tasks.cephfs_test_runner: 2023-11-08T16:42:13.176 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-08T16:42:13.177 INFO:tasks.cephfs_test_runner:ERROR: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-11-08T16:42:13.177 INFO:tasks.cephfs_test_runner:That when injecting a dentry into a fragmented directory, we put it in the right fragment. 2023-11-08T16:42:13.177 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T16:42:13.177 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-08T16:42:13.178 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa/tasks/cephfs/test_data_scan.py", line 560, in test_fragmented_injection 2023-11-08T16:42:13.178 INFO:tasks.cephfs_test_runner: out = self.mount_a.run_shell_payload(f"sudo cat subdir/{victim_dentry}", omit_sudo=False).stdout.getvalue().strip() 2023-11-08T16:42:13.178 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa/tasks/cephfs/mount.py", line 793, in run_shell_payload 2023-11-08T16:42:13.179 INFO:tasks.cephfs_test_runner: return self.run_shell(**kwargs) 2023-11-08T16:42:13.179 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa/tasks/cephfs/mount.py", line 784, in run_shell 2023-11-08T16:42:13.179 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-11-08T16:42:13.180 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run 2023-11-08T16:42:13.180 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-11-08T16:42:13.180 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run 2023-11-08T16:42:13.181 INFO:tasks.cephfs_test_runner: r.wait() 2023-11-08T16:42:13.181 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait 2023-11-08T16:42:13.181 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-08T16:42:13.182 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-08T16:42:13.182 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-08T16:42:13.182 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi081 with status 1: "(cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo cat subdir/21')" 2023-11-08T16:42:13.183 INFO:tasks.cephfs_test_runner: 2023-11-08T16:42:13.183 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T16:42:13.183 INFO:tasks.cephfs_test_runner:Ran 5 tests in 748.814s 2023-11-08T16:42:13.184 INFO:tasks.cephfs_test_runner: 2023-11-08T16:42:13.184 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-08T16:42:13.184 INFO:tasks.cephfs_test_runner: 2023-11-08T16:42:13.185 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-08T16:42:13.185 INFO:tasks.cephfs_test_runner:ERROR: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-11-08T16:42:13.185 INFO:tasks.cephfs_test_runner:That when injecting a dentry into a fragmented directory, we put it in the right fragment. 2023-11-08T16:42:13.185 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-08T16:42:13.186 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-08T16:42:13.186 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa/tasks/cephfs/test_data_scan.py", line 560, in test_fragmented_injection 2023-11-08T16:42:13.186 INFO:tasks.cephfs_test_runner: out = self.mount_a.run_shell_payload(f"sudo cat subdir/{victim_dentry}", omit_sudo=False).stdout.getvalue().strip() 2023-11-08T16:42:13.187 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa/tasks/cephfs/mount.py", line 793, in run_shell_payload 2023-11-08T16:42:13.187 INFO:tasks.cephfs_test_runner: return self.run_shell(**kwargs) 2023-11-08T16:42:13.187 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa/tasks/cephfs/mount.py", line 784, in run_shell 2023-11-08T16:42:13.187 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-11-08T16:42:13.188 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/remote.py", line 522, in run 2023-11-08T16:42:13.188 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-11-08T16:42:13.188 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 455, in run 2023-11-08T16:42:13.188 INFO:tasks.cephfs_test_runner: r.wait() 2023-11-08T16:42:13.189 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 161, in wait 2023-11-08T16:42:13.189 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-11-08T16:42:13.189 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-11-08T16:42:13.190 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-11-08T16:42:13.190 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi081 with status 1: "(cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo cat subdir/21')" 2023-11-08T16:42:13.190 INFO:tasks.cephfs_test_runner: 2023-11-08T16:42:13.191 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-11-08T16:42:13.258 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=52ac173d6eff4124aec04af447bcd126 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_6899cd26fceddb2fec83dc1a1349394b28c8998e/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_85bdb240f5cf2a758acc9b247951fbc8f5799bc1/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan) 2023-11-08T16:42:13.260 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-08T16:42:13.274 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-08T16:42:13.292 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-08T16:42:13.292 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:42:13.293 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:42:13.349 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:42:13.349 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2023-11-08T16:42:13.350 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:42:13.442 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054298,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-08T16:42:13.442 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:42:13.442 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:42:13.502 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:42:13.502 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-11-08T16:42:13.503 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-08T16:42:13.557 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054259,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249376k,nr_inodes=812344,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498760k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-08T16:42:13.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:13.558 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-11-08T16:42:13.610 INFO:teuthology.orchestra.run.smithi093.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-08T16:42:13.610 INFO:teuthology.orchestra.run.smithi093.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-08T16:42:13.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:13.612 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-08T16:42:13.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:13.710 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-08T16:42:13.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:13.800 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link delete ceph-brx 2023-11-08T16:42:13.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:13.913 DEBUG:teuthology.orchestra.run.smithi081:> ip netns list 2023-11-08T16:42:13.971 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-08T16:42:13.971 INFO:teuthology.orchestra.run.smithi081.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-08T16:42:13.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:13.973 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-08T16:42:14.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:14.069 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-08T16:42:14.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-08T16:42:14.160 DEBUG:teuthology.orchestra.run.smithi081:> sudo ip link delete ceph-brx 2023-11-08T16:42:14.294 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-08T16:42:14.305 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-08T16:42:14.305 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:42:14.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.496+0000 7fbf07778640 1 Processor -- start 2023-11-08T16:42:14.498 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.496+0000 7fbf07778640 1 -- start start 2023-11-08T16:42:14.499 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.498+0000 7fbf07778640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbf000b3170 0x7fbf0015b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:14.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.498+0000 7fbf07778640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf000b2310 con 0x7fbf0006c9f0 2023-11-08T16:42:14.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.498+0000 7fbf07778640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf000b25f0 con 0x7fbf000b3550 2023-11-08T16:42:14.500 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.498+0000 7fbf07778640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf000b2bb0 con 0x7fbf000b3170 2023-11-08T16:42:14.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.498+0000 7fbf05514640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fbf000b3170 0x7fbf0015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:14.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.498+0000 7fbf05d15640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fbf0006c9f0 0x7fbf00144230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54336/0 (socket says 172.21.15.81:54336) 2023-11-08T16:42:14.501 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.498+0000 7fbf05d15640 1 -- 172.21.15.81:0/1177329450 learned_addr learned my addr 172.21.15.81:0/1177329450 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:14.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.499+0000 7fbf05514640 1 -- 172.21.15.81:0/1177329450 >> v1:172.21.15.93:6789/0 conn(0x7fbf000b3550 legacy=0x7fbf000b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:14.503 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.499+0000 7fbf05514640 1 -- 172.21.15.81:0/1177329450 >> v1:172.21.15.81:6789/0 conn(0x7fbf0006c9f0 legacy=0x7fbf00144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:14.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.499+0000 7fbf05514640 1 -- 172.21.15.81:0/1177329450 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf000b28d0 con 0x7fbf000b3170 2023-11-08T16:42:14.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.499+0000 7fbef7fff640 1 -- 172.21.15.81:0/1177329450 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2969439069 0 0) 0x7fbf000b2310 con 0x7fbf0006c9f0 2023-11-08T16:42:14.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.500+0000 7fbf05514640 1 --2- 172.21.15.81:0/1177329450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000b3170 0x7fbf0015b500 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0097a0 tx=0x7fbefc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b1cb793c83e8cb7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:14.504 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.501+0000 7fbef7fff640 1 -- 172.21.15.81:0/1177329450 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc031830 con 0x7fbf000b3170 2023-11-08T16:42:14.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.501+0000 7fbef7fff640 1 -- 172.21.15.81:0/1177329450 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbefc031990 con 0x7fbf000b3170 2023-11-08T16:42:14.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.501+0000 7fbef7fff640 1 -- 172.21.15.81:0/1177329450 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbefc031d20 con 0x7fbf000b3170 2023-11-08T16:42:14.505 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.501+0000 7fbf07778640 1 -- 172.21.15.81:0/1177329450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000b3170 msgr2=0x7fbf0015b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:14.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.501+0000 7fbf07778640 1 --2- 172.21.15.81:0/1177329450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000b3170 0x7fbf0015b500 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0097a0 tx=0x7fbefc02b8d0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.501+0000 7fbf07778640 1 -- 172.21.15.81:0/1177329450 shutdown_connections 2023-11-08T16:42:14.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.501+0000 7fbf07778640 1 --2- 172.21.15.81:0/1177329450 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000b3170 0x7fbf0015b500 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.506 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.502+0000 7fbf07778640 1 -- 172.21.15.81:0/1177329450 >> 172.21.15.81:0/1177329450 conn(0x7fbf0006a6a0 msgr2=0x7fbf000b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:14.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.502+0000 7fbf07778640 1 -- 172.21.15.81:0/1177329450 shutdown_connections 2023-11-08T16:42:14.507 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.502+0000 7fbf07778640 1 -- 172.21.15.81:0/1177329450 wait complete. 2023-11-08T16:42:14.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.503+0000 7fbf07778640 1 Processor -- start 2023-11-08T16:42:14.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.503+0000 7fbf07778640 1 -- start start 2023-11-08T16:42:14.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.504+0000 7fbf07778640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf0006c9f0 0x7fbf0015a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:14.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.504+0000 7fbf07778640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000b3170 0x7fbf0006fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:14.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.504+0000 7fbf07778640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf000b3550 0x7fbf000703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:14.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.504+0000 7fbf07778640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0012c490 con 0x7fbf000b3170 2023-11-08T16:42:14.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.504+0000 7fbf07778640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0012cd30 con 0x7fbf000b3550 2023-11-08T16:42:14.511 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.504+0000 7fbf07778640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf00140cc0 con 0x7fbf0006c9f0 2023-11-08T16:42:14.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.505+0000 7fbf04d13640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000b3170 0x7fbf0006fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:14.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.505+0000 7fbf04d13640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000b3170 0x7fbf0006fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51402/0 (socket says 172.21.15.81:51402) 2023-11-08T16:42:14.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.505+0000 7fbf04d13640 1 -- 172.21.15.81:0/3905149101 learned_addr learned my addr 172.21.15.81:0/3905149101 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:14.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.505+0000 7fbf04d13640 1 -- 172.21.15.81:0/3905149101 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf0006c9f0 msgr2=0x7fbf0015a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:14.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.505+0000 7fbf04d13640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf0006c9f0 0x7fbf0015a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.505+0000 7fbf04d13640 1 -- 172.21.15.81:0/3905149101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf000b3550 msgr2=0x7fbf000703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:14.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.505+0000 7fbf04d13640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf000b3550 0x7fbf000703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.505+0000 7fbf04d13640 1 -- 172.21.15.81:0/3905149101 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc009400 con 0x7fbf000b3170 2023-11-08T16:42:14.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.505+0000 7fbf04d13640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000b3170 0x7fbf0006fe60 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fbef000b450 tx=0x7fbef000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:14.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.506+0000 7fbef5ffb640 1 -- 172.21.15.81:0/3905149101 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0011890 con 0x7fbf000b3170 2023-11-08T16:42:14.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.506+0000 7fbef5ffb640 1 -- 172.21.15.81:0/3905149101 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef00119f0 con 0x7fbf000b3170 2023-11-08T16:42:14.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.506+0000 7fbef5ffb640 1 -- 172.21.15.81:0/3905149101 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef0011da0 con 0x7fbf000b3170 2023-11-08T16:42:14.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.506+0000 7fbf07778640 1 -- 172.21.15.81:0/3905149101 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0006d000 con 0x7fbf000b3170 2023-11-08T16:42:14.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.506+0000 7fbf07778640 1 -- 172.21.15.81:0/3905149101 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0012f300 con 0x7fbf000b3170 2023-11-08T16:42:14.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.507+0000 7fbf07778640 1 -- 172.21.15.81:0/3905149101 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf00074440 con 0x7fbf000b3170 2023-11-08T16:42:14.516 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.508+0000 7fbef5ffb640 1 -- 172.21.15.81:0/3905149101 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbef001a400 con 0x7fbf000b3170 2023-11-08T16:42:14.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.509+0000 7fbef5ffb640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee406ae70 0x7fbee406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:14.517 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.510+0000 7fbef5ffb640 1 -- 172.21.15.81:0/3905149101 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fbef0089500 con 0x7fbf000b3170 2023-11-08T16:42:14.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.510+0000 7fbf05514640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee406ae70 0x7fbee406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:14.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.510+0000 7fbf05514640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee406ae70 0x7fbee406d330 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbefc036000 tx=0x7fbefc030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:14.518 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.514+0000 7fbef5ffb640 1 -- 172.21.15.81:0/3905149101 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fbef0054760 con 0x7fbf000b3170 2023-11-08T16:42:14.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.701+0000 7fbf07778640 1 -- 172.21.15.81:0/3905149101 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf0006ce80 con 0x7fbee406ae70 2023-11-08T16:42:14.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.704+0000 7fbef5ffb640 1 -- 172.21.15.81:0/3905149101 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48545 (secure 0 0 0) 0x7fbefc031b80 con 0x7fbee406ae70 2023-11-08T16:42:14.706 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:14.707 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:42:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.706+0000 7fbf07778640 1 -- 172.21.15.81:0/3905149101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee406ae70 msgr2=0x7fbee406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.706+0000 7fbf07778640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee406ae70 0x7fbee406d330 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbefc036000 tx=0x7fbefc030000 comp rx=0 tx=0).stop 2023-11-08T16:42:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.706+0000 7fbf07778640 1 -- 172.21.15.81:0/3905149101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000b3170 msgr2=0x7fbf0006fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:14.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.706+0000 7fbf07778640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000b3170 0x7fbf0006fe60 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fbef000b450 tx=0x7fbef000b910 comp rx=0 tx=0).stop 2023-11-08T16:42:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.707+0000 7fbf07778640 1 -- 172.21.15.81:0/3905149101 shutdown_connections 2023-11-08T16:42:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.707+0000 7fbf07778640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fbee406ae70 0x7fbee406d330 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.707+0000 7fbf07778640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf000b3550 0x7fbf000703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.707+0000 7fbf07778640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fbf000b3170 0x7fbf0006fe60 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.707+0000 7fbf07778640 1 --2- 172.21.15.81:0/3905149101 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fbf0006c9f0 0x7fbf0015a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.707+0000 7fbf07778640 1 -- 172.21.15.81:0/3905149101 >> 172.21.15.81:0/3905149101 conn(0x7fbf0006a6a0 msgr2=0x7fbf0015d380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:14.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.707+0000 7fbf07778640 1 -- 172.21.15.81:0/3905149101 shutdown_connections 2023-11-08T16:42:14.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.707+0000 7fbf07778640 1 -- 172.21.15.81:0/3905149101 wait complete. 2023-11-08T16:42:14.727 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":528,"stamp":"2023-11-08T16:42:12.829608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264712,"kb_used_data":15264,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710008,"statfs":{"total":773094113280,"available":772823048192,"internally_reserved":0,"allocated":15630336,"data_stored":7728915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-404392,"num_objects":-123,"num_object_clones":0,"num_object_copies":-246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-123,"num_whiteouts":0,"num_read":-1808,"num_read_kb":-1513,"num_write":-911,"num_write_kb":-891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-4,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001431"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":283,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-11-08T16:42:09.087078+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:42:09.087078+0000","last_peered":"2023-11-08T16:42:09.087078+0000","last_clean":"2023-11-08T16:42:09.087078+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:42:09.087078+0000","last_undegraded":"2023-11-08T16:42:09.087078+0000","last_fullsized":"2023-11-08T16:42:09.087078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33520,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338320,"statfs":{"total":96636764160,"available":96602439680,"internally_reserved":0,"allocated":2396160,"data_stored":1355111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0329999999999999,"15min":1.071},"min":{"1min":0.80200000000000005,"5min":0.56399999999999995,"15min":0.49299999999999999},"max":{"1min":1.5649999999999999,"5min":1.6879999999999999,"15min":3.1899999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.0900000000000001,"15min":1.1060000000000001},"min":{"1min":0.746,"5min":0.58299999999999996,"15min":0.53600000000000003},"max":{"1min":1.5660000000000001,"5min":1.6040000000000001,"15min":4.2729999999999997},"last":1.4590000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.103,"15min":1.111},"min":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.57899999999999996},"max":{"1min":1.5629999999999999,"5min":1.891,"15min":1.891},"last":1.393},{"interface":"front","average":{"1min":1.23,"5min":1.1719999999999999,"15min":1.204},"min":{"1min":0.69899999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":3.3740000000000001},"last":0.79500000000000004}]},{"osd":2,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.206},"min":{"1min":0.81799999999999995,"5min":0.70999999999999996,"15min":0.58399999999999996},"max":{"1min":1.591,"5min":1.8109999999999999,"15min":4.2249999999999996},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.228,"5min":1.252,"15min":1.224},"min":{"1min":0.67700000000000005,"5min":0.66000000000000003,"15min":0.56299999999999994},"max":{"1min":1.587,"5min":1.7010000000000001,"15min":3.4750000000000001},"last":1.5029999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.1819999999999999,"15min":1.175},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.54900000000000004},"max":{"1min":1.71,"5min":1.71,"15min":2.964},"last":1.173},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.365,"15min":1.403},"min":{"1min":1.0029999999999999,"5min":0.86899999999999999,"15min":0.58599999999999997},"max":{"1min":1.619,"5min":1.7490000000000001,"15min":4.181},"last":1.847}]},{"osd":4,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.234,"15min":1.2350000000000001},"min":{"1min":0.93899999999999995,"5min":0.80400000000000005,"15min":0.77200000000000002},"max":{"1min":1.778,"5min":1.7789999999999999,"15min":3.6669999999999998},"last":1.633},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.325,"15min":1.337},"min":{"1min":0.95599999999999996,"5min":0.872,"15min":0.59099999999999997},"max":{"1min":1.7470000000000001,"5min":2.0179999999999998,"15min":2.3730000000000002},"last":1.6910000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.415,"15min":1.3879999999999999},"min":{"1min":1.0569999999999999,"5min":0.89200000000000002,"15min":0.61799999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":3.1059999999999999},"last":1.954},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4279999999999999,"15min":1.4530000000000001},"min":{"1min":1.1000000000000001,"5min":1.0589999999999999,"15min":0.878},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.9750000000000001},"last":1.8}]},{"osd":6,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.528,"15min":1.5329999999999999},"min":{"1min":1.083,"5min":1.0800000000000001,"15min":0.94799999999999995},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":3.7530000000000001},"last":1.891},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.4770000000000001,"15min":1.486},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.73999999999999999},"max":{"1min":1.831,"5min":2.0430000000000001,"15min":6.9560000000000004},"last":1.9239999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1765376,"data_stored":770406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.093,"15min":1.0720000000000001},"min":{"1min":0.69399999999999995,"5min":0.60399999999999998,"15min":0.497},"max":{"1min":1.569,"5min":1.7230000000000001,"15min":2.5609999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.113,"5min":1.131,"15min":1.097},"min":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.56200000000000006},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.7519999999999998},"last":0.68799999999999994}]},{"osd":1,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.131,"15min":1.0880000000000001},"min":{"1min":0.63400000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.548,"5min":1.667,"15min":1.667},"last":0.75},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.159,"15min":1.137},"min":{"1min":0.72599999999999998,"5min":0.71899999999999997,"15min":0.65600000000000003},"max":{"1min":1.573,"5min":1.7290000000000001,"15min":1.97},"last":0.89200000000000002}]},{"osd":2,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.175,"15min":1.1830000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.56799999999999995},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1990000000000001,"15min":1.171},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.60499999999999998},"max":{"1min":1.639,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.0820000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2190000000000001,"15min":1.2390000000000001},"min":{"1min":0.83399999999999996,"5min":0.63200000000000001,"15min":0.58299999999999996},"max":{"1min":1.73,"5min":1.73,"15min":2.1120000000000001},"last":1.121},{"interface":"front","average":{"1min":1.331,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.88700000000000001,"5min":0.68200000000000005,"15min":0.67300000000000004},"max":{"1min":1.6930000000000001,"5min":1.7809999999999999,"15min":1.95},"last":1.5600000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3220000000000001,"15min":1.3320000000000001},"min":{"1min":0.89700000000000002,"5min":0.81799999999999995,"15min":0.65300000000000002},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":1.23},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.365,"15min":1.3859999999999999},"min":{"1min":0.93600000000000005,"5min":0.878,"15min":0.64900000000000002},"max":{"1min":1.8420000000000001,"5min":1.9590000000000001,"15min":2.7080000000000002},"last":1.6160000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3340000000000001,"15min":1.323},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.80500000000000005},"max":{"1min":1.7669999999999999,"5min":1.994,"15min":2.105},"last":1.325},{"interface":"front","average":{"1min":1.53,"5min":1.518,"15min":1.5309999999999999},"min":{"1min":1.0389999999999999,"5min":0.91100000000000003,"15min":0.78500000000000003},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.4700000000000002},"last":1.6419999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.5529999999999999},"min":{"1min":1.079,"5min":1.079,"15min":0.74299999999999999},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":3.8959999999999999},"last":1.673},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4850000000000001,"15min":1.496},"min":{"1min":1.0549999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.885,"5min":2.0870000000000002,"15min":2.4889999999999999},"last":1.595}]}]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":770310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.081,"15min":1.0720000000000001},"min":{"1min":0.70299999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.657,"5min":1.657,"15min":2.4609999999999999},"last":1.131},{"interface":"front","average":{"1min":1.052,"5min":1.097,"15min":1.1060000000000001},"min":{"1min":0.63700000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.474,"5min":1.6000000000000001,"15min":2.3639999999999999},"last":1.1859999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.236},"min":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.53000000000000003},"max":{"1min":1.554,"5min":1.649,"15min":3.8690000000000002},"last":1.073},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2330000000000001,"15min":1.244},"min":{"1min":0.77200000000000002,"5min":0.66400000000000003,"15min":0.64500000000000002},"max":{"1min":1.5589999999999999,"5min":1.8109999999999999,"15min":3.2349999999999999},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.141,"15min":1.1819999999999999},"min":{"1min":0.72299999999999998,"5min":0.66100000000000003,"15min":0.65800000000000003},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":2.6339999999999999},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.1859999999999999,"15min":1.1990000000000001},"min":{"1min":0.872,"5min":0.70499999999999996,"15min":0.65800000000000003},"max":{"1min":1.681,"5min":1.752,"15min":2.2559999999999998},"last":1.839}]},{"osd":3,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2529999999999999,"15min":1.268},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.66500000000000004},"max":{"1min":1.5089999999999999,"5min":1.6519999999999999,"15min":2.8330000000000002},"last":1.907},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2909999999999999,"15min":1.29},"min":{"1min":0.86899999999999999,"5min":0.753,"15min":0.68600000000000005},"max":{"1min":1.528,"5min":2.0470000000000002,"15min":2.5760000000000001},"last":1.5109999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.28,"15min":1.3160000000000001},"min":{"1min":0.871,"5min":0.871,"15min":0.61599999999999999},"max":{"1min":1.657,"5min":2.0019999999999998,"15min":3.9790000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.427},"min":{"1min":1.1240000000000001,"5min":1.0569999999999999,"15min":0.81499999999999995},"max":{"1min":1.591,"5min":1.8029999999999999,"15min":3.0209999999999999},"last":2.0630000000000002}]},{"osd":6,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.421,"15min":1.4099999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.752},"max":{"1min":1.6200000000000001,"5min":2.1560000000000001,"15min":2.5550000000000002},"last":1.9650000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.407,"15min":1.45},"min":{"1min":1.03,"5min":0.98499999999999999,"15min":0.93600000000000005},"max":{"1min":1.7509999999999999,"5min":1.9419999999999999,"15min":4.1189999999999998},"last":1.643}]},{"osd":7,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.502,"15min":1.5349999999999999},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0049999999999999},"max":{"1min":1.7929999999999999,"5min":2.0219999999999998,"15min":3.1160000000000001},"last":2.214},{"interface":"front","average":{"1min":1.516,"5min":1.5580000000000001,"15min":1.5509999999999999},"min":{"1min":1.22,"5min":1.075,"15min":0.96999999999999997},"max":{"1min":1.7470000000000001,"5min":2.2970000000000002,"15min":2.7050000000000001},"last":2.3340000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1744896,"data_stored":764105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.05,"15min":1.0760000000000001},"min":{"1min":0.80000000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.528,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.081,"5min":1.1080000000000001,"15min":1.091},"min":{"1min":0.78600000000000003,"5min":0.58299999999999996,"15min":0.45400000000000001},"max":{"1min":1.5680000000000001,"5min":1.772,"15min":1.772},"last":1.3700000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.1200000000000001},"min":{"1min":0.745,"5min":0.60099999999999998,"15min":0.56999999999999995},"max":{"1min":1.2849999999999999,"5min":1.51,"15min":2.0459999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.159,"15min":1.171},"min":{"1min":0.873,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.742},"last":1.401}]},{"osd":2,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.22,"15min":1.2110000000000001},"min":{"1min":0.89600000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5489999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.216,"15min":1.24},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.71699999999999997},"max":{"1min":1.524,"5min":1.7749999999999999,"15min":1.9319999999999999},"last":1.2609999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.306,"15min":1.284},"min":{"1min":0.80100000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.653,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2649999999999999,"15min":1.2709999999999999},"min":{"1min":0.82399999999999995,"5min":0.72799999999999998,"15min":0.65100000000000002},"max":{"1min":1.5660000000000001,"5min":1.8069999999999999,"15min":1.895},"last":1.1220000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3680000000000001,"15min":1.365},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.58799999999999997},"max":{"1min":1.8320000000000001,"5min":2.286,"15min":2.286},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.454,"15min":1.4330000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.61899999999999999},"max":{"1min":1.7450000000000001,"5min":2.1970000000000001,"15min":2.46},"last":1.2909999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.387,"15min":1.391},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.74299999999999999},"max":{"1min":1.6970000000000001,"5min":1.7769999999999999,"15min":2.5609999999999999},"last":1.494},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4870000000000001,"15min":1.522},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.9390000000000001,"5min":2.4780000000000002,"15min":2.6389999999999998},"last":1.554}]},{"osd":7,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5489999999999999,"15min":1.534},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.8100000000000001,"5min":2.1040000000000001,"15min":2.2639999999999998},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5509999999999999,"15min":1.5600000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8440000000000001,"5min":2.1070000000000002,"15min":2.6859999999999999},"last":1.3380000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1806336,"data_stored":770357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.82699999999999996,"15min":0.80900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.397,"5min":1.4930000000000001,"15min":1.9339999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93000000000000005,"15min":0.89800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.316},"max":{"1min":1.319,"5min":1.647,"15min":1.647},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.88600000000000001},"min":{"1min":0.48599999999999999,"5min":0.38600000000000001,"15min":0.36499999999999999},"max":{"1min":1.2729999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.318},{"interface":"front","average":{"1min":1.012,"5min":0.998,"15min":0.97299999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.378},"max":{"1min":1.52,"5min":1.5800000000000001,"15min":1.829},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.052,"15min":1.0109999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41699999999999998},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":2.0779999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.099,"15min":1.103},"min":{"1min":0.57499999999999996,"5min":0.504,"15min":0.504},"max":{"1min":1.6000000000000001,"5min":1.7270000000000001,"15min":2.4660000000000002},"last":0.46999999999999997}]},{"osd":4,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.173,"15min":1.177},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.56399999999999995},"max":{"1min":1.571,"5min":1.5720000000000001,"15min":2.25},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.24,"5min":1.278,"15min":1.268},"min":{"1min":0.76200000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.6020000000000001,"5min":2.0110000000000001,"15min":2.1829999999999998},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.3300000000000001,"15min":1.3240000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.653,"5min":2.2229999999999999,"15min":2.3999999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.339,"15min":1.3300000000000001},"min":{"1min":0.879,"5min":0.80400000000000005,"15min":0.60999999999999999},"max":{"1min":1.6399999999999999,"5min":1.887,"15min":2.593},"last":0.88500000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3720000000000001,"15min":1.373},"min":{"1min":0.81899999999999995,"5min":0.751,"15min":0.58799999999999997},"max":{"1min":1.6180000000000001,"5min":1.9810000000000001,"15min":2.9569999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.3360000000000001,"15min":1.3460000000000001},"min":{"1min":0.86299999999999999,"5min":0.83599999999999997,"15min":0.73399999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.762},"last":0.73899999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3700000000000001,"15min":1.387},"min":{"1min":0.997,"5min":0.73699999999999999,"15min":0.70799999999999996},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.8700000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.4179999999999999,"15min":1.411},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7609999999999999,"5min":2.121,"15min":2.633},"last":0.83099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1712128,"data_stored":770312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81799999999999995,"15min":0.872},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.24099999999999999},"max":{"1min":1.0880000000000001,"5min":1.335,"15min":4.5369999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91600000000000004,"15min":0.88},"min":{"1min":0.60999999999999999,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":1.546,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.69599999999999995}]},{"osd":1,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0669999999999999,"15min":1.05},"min":{"1min":0.64900000000000002,"5min":0.44900000000000001,"15min":0.379},"max":{"1min":1.6160000000000001,"5min":1.7689999999999999,"15min":4.4980000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.157,"5min":1.143,"15min":1.0629999999999999},"min":{"1min":0.63800000000000001,"5min":0.47299999999999998,"15min":0.40300000000000002},"max":{"1min":1.7190000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.83399999999999996}]},{"osd":3,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.92400000000000004,"15min":0.93600000000000005},"min":{"1min":0.47299999999999998,"5min":0.44400000000000001,"15min":0.35499999999999998},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":3.3050000000000002},"last":0.876},{"interface":"front","average":{"1min":1.099,"5min":1.0669999999999999,"15min":1.1060000000000001},"min":{"1min":0.73899999999999999,"5min":0.44800000000000001,"15min":0.42999999999999999},"max":{"1min":1.796,"5min":1.819,"15min":4.5899999999999999},"last":1.1419999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2889999999999999,"15min":1.3160000000000001},"min":{"1min":0.88100000000000001,"5min":0.76100000000000001,"15min":0.73499999999999999},"max":{"1min":1.6639999999999999,"5min":2.2160000000000002,"15min":4.9409999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2470000000000001,"15min":1.236},"min":{"1min":0.93500000000000005,"5min":0.73899999999999999,"15min":0.63100000000000001},"max":{"1min":1.6319999999999999,"5min":2.0739999999999998,"15min":2.46},"last":0.95299999999999996}]},{"osd":5,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.302,"15min":1.28},"min":{"1min":1.0009999999999999,"5min":0.84599999999999997,"15min":0.77200000000000002},"max":{"1min":1.512,"5min":1.9510000000000001,"15min":2.8530000000000002},"last":1.093},{"interface":"front","average":{"1min":1.335,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":1.032,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.77,"5min":2.5390000000000001,"15min":3.5680000000000001},"last":1.204}]},{"osd":6,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.625},"max":{"1min":1.8200000000000001,"5min":2.0859999999999999,"15min":3.536},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.395,"15min":1.423},"min":{"1min":0.97399999999999998,"5min":0.81200000000000006,"15min":0.72199999999999998},"max":{"1min":1.887,"5min":2.2719999999999998,"15min":4.7549999999999999},"last":1.486}]},{"osd":7,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3799999999999999,"15min":1.3779999999999999},"min":{"1min":0.99199999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6599999999999999,"5min":2.2930000000000001,"15min":2.9700000000000002},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.393,"15min":1.4119999999999999},"min":{"1min":0.84399999999999997,"5min":0.79600000000000004,"15min":0.70499999999999996},"max":{"1min":2.0099999999999998,"5min":2.129,"15min":3.4950000000000001},"last":1.3919999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1703936,"data_stored":770311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.77000000000000002,"15min":0.76800000000000002},"min":{"1min":0.443,"5min":0.34499999999999997,"15min":0.28899999999999998},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.542},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.875,"15min":0.93600000000000005},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":1.202,"5min":1.5429999999999999,"15min":4.0309999999999997},"last":1.258}]},{"osd":2,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.98699999999999999,"15min":1.0289999999999999},"min":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.45000000000000001},"max":{"1min":1.6020000000000001,"5min":1.7689999999999999,"15min":3.0339999999999998},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.93899999999999995,"15min":0.999},"min":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":1.5249999999999999,"5min":1.542,"15min":2.6840000000000002},"last":1.4530000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98199999999999998,"15min":1.0389999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.38900000000000001},"max":{"1min":1.4910000000000001,"5min":1.5129999999999999,"15min":4.3179999999999996},"last":1.139},{"interface":"front","average":{"1min":1.014,"5min":0.95799999999999996,"15min":1.0489999999999999},"min":{"1min":0.48699999999999999,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.4530000000000001,"5min":1.5900000000000001,"15min":8.5570000000000004},"last":1.073}]},{"osd":4,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.161,"15min":1.23},"min":{"1min":0.751,"5min":0.751,"15min":0.63300000000000001},"max":{"1min":1.804,"5min":1.907,"15min":3.1659999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.276,"5min":1.2230000000000001,"15min":1.2769999999999999},"min":{"1min":0.84899999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.659,"5min":1.823,"15min":2.734},"last":1.0209999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2410000000000001,"15min":1.421},"min":{"1min":0.874,"5min":0.72399999999999998,"15min":0.67100000000000004},"max":{"1min":1.621,"5min":1.921,"15min":8.4730000000000008},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.2509999999999999,"15min":1.4199999999999999},"min":{"1min":0.76200000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7669999999999999,"5min":1.9470000000000001,"15min":8.8040000000000003},"last":1.427}]},{"osd":6,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.28,"15min":1.3660000000000001},"min":{"1min":0.84799999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.681,"5min":1.9119999999999999,"15min":3.984},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.3100000000000001,"15min":1.3779999999999999},"min":{"1min":0.83599999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.8300000000000001,"5min":2.0449999999999999,"15min":3.2450000000000001},"last":1.1890000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.288,"15min":1.4019999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.69199999999999995},"max":{"1min":1.758,"5min":1.853,"15min":4.3609999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.369,"15min":1.4790000000000001},"min":{"1min":0.74399999999999999,"5min":0.72799999999999998,"15min":0.70499999999999996},"max":{"1min":1.974,"5min":1.974,"15min":5.3840000000000003},"last":1.554}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33912,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337928,"statfs":{"total":96636764160,"available":96602038272,"internally_reserved":0,"allocated":2797568,"data_stored":1758003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.42299999999999999,"5min":0.34000000000000002,"15min":0.29399999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.6220000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95399999999999996,"15min":0.96199999999999997},"min":{"1min":0.41099999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.5720000000000001,"5min":1.722,"15min":2.3340000000000001},"last":1.4299999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.89400000000000002,"15min":0.91500000000000004},"min":{"1min":0.48499999999999999,"5min":0.39900000000000002,"15min":0.37},"max":{"1min":1.4670000000000001,"5min":1.5860000000000001,"15min":1.762},"last":1.3420000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.92200000000000004,"15min":0.96199999999999997},"min":{"1min":0.502,"5min":0.38100000000000001,"15min":0.36799999999999999},"max":{"1min":1.4570000000000001,"5min":1.613,"15min":1.879},"last":1.2789999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.92900000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.34200000000000003},"max":{"1min":1.4510000000000001,"5min":1.72,"15min":1.728},"last":1.389},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.1539999999999999,"15min":1.177},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.7679999999999998},"last":1.25}]},{"osd":4,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1779999999999999,"15min":1.159},"min":{"1min":0.77200000000000002,"5min":0.67100000000000004,"15min":0.65300000000000002},"max":{"1min":1.429,"5min":2.0489999999999999,"15min":2.2559999999999998},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2529999999999999,"15min":1.276},"min":{"1min":0.874,"5min":0.69999999999999996,"15min":0.59399999999999997},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.867},"last":1.0489999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2709999999999999,"15min":1.264},"min":{"1min":0.84299999999999997,"5min":0.79600000000000004,"15min":0.67900000000000005},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.7280000000000002},"last":1.6899999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.234,"15min":1.2290000000000001},"min":{"1min":0.93300000000000005,"5min":0.70899999999999996,"15min":0.67600000000000005},"max":{"1min":1.5669999999999999,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.613}]},{"osd":6,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3440000000000001,"15min":1.3520000000000001},"min":{"1min":0.91700000000000004,"5min":0.80000000000000004,"15min":0.72499999999999998},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.6739999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.3260000000000001,"15min":1.345},"min":{"1min":0.89000000000000001,"5min":0.88200000000000001,"15min":0.64200000000000002},"max":{"1min":1.829,"5min":2.1280000000000001,"15min":2.8260000000000001},"last":1.5720000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.381,"15min":1.399},"min":{"1min":1.0109999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":4.5670000000000002},"last":1.645},{"interface":"front","average":{"1min":1.343,"5min":1.337,"15min":1.349},"min":{"1min":0.85399999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6080000000000001,"5min":2.6070000000000002,"15min":5.4450000000000003},"last":1.6679999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:42:14.731 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:42:14.948 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.947+0000 7f8496933640 1 Processor -- start 2023-11-08T16:42:14.949 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.947+0000 7f8496933640 1 -- start start 2023-11-08T16:42:14.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.948+0000 7f8496933640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f84900b3550 0x7f849015b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:14.951 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.948+0000 7f8496933640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f849006c9f0 0x7f8490144230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:14.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.948+0000 7f8496933640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84900b2310 con 0x7f84900b3170 2023-11-08T16:42:14.953 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.948+0000 7f8496933640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84900b2bb0 con 0x7f84900b3550 2023-11-08T16:42:14.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.948+0000 7f8496933640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84900b25f0 con 0x7f849006c9f0 2023-11-08T16:42:14.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.949+0000 7f8487fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f84900b3550 0x7f849015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:14.954 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.949+0000 7f8487fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f84900b3550 0x7f849015b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51414/0 (socket says 172.21.15.81:51414) 2023-11-08T16:42:14.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.949+0000 7f8487fff640 1 -- 172.21.15.81:0/1825330085 learned_addr learned my addr 172.21.15.81:0/1825330085 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:14.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.949+0000 7f8494ed0640 1 --2- 172.21.15.81:0/1825330085 >> v2:172.21.15.93:3300/0 conn(0x7f849006c9f0 0x7f8490144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:14.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.949+0000 7f8487fff640 1 -- 172.21.15.81:0/1825330085 >> v2:172.21.15.93:3300/0 conn(0x7f849006c9f0 msgr2=0x7f8490144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:14.955 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.949+0000 7f8487fff640 1 --2- 172.21.15.81:0/1825330085 >> v2:172.21.15.93:3300/0 conn(0x7f849006c9f0 0x7f8490144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.949+0000 7f8487fff640 1 -- 172.21.15.81:0/1825330085 >> v1:172.21.15.93:6789/0 conn(0x7f84900b3170 legacy=0x7f84900b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:14.956 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.949+0000 7f8487fff640 1 -- 172.21.15.81:0/1825330085 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84900b28d0 con 0x7f84900b3550 2023-11-08T16:42:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.950+0000 7f8487fff640 1 --2- 172.21.15.81:0/1825330085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84900b3550 0x7f849015b500 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f84900b21f0 tx=0x7f847802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f86985a8cde9eecc server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:14.959 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.951+0000 7f848f7fe640 1 -- 172.21.15.81:0/1825330085 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478031870 con 0x7f84900b3550 2023-11-08T16:42:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.951+0000 7f848f7fe640 1 -- 172.21.15.81:0/1825330085 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84780319d0 con 0x7f84900b3550 2023-11-08T16:42:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.951+0000 7f848f7fe640 1 -- 172.21.15.81:0/1825330085 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478031d60 con 0x7f84900b3550 2023-11-08T16:42:14.960 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.951+0000 7f8496933640 1 -- 172.21.15.81:0/1825330085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84900b3550 msgr2=0x7f849015b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.951+0000 7f8496933640 1 --2- 172.21.15.81:0/1825330085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84900b3550 0x7f849015b500 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f84900b21f0 tx=0x7f847802b910 comp rx=0 tx=0).stop 2023-11-08T16:42:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.951+0000 7f8496933640 1 -- 172.21.15.81:0/1825330085 shutdown_connections 2023-11-08T16:42:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.951+0000 7f8496933640 1 --2- 172.21.15.81:0/1825330085 >> v2:172.21.15.93:3300/0 conn(0x7f849006c9f0 0x7f8490144230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.961 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.951+0000 7f8496933640 1 --2- 172.21.15.81:0/1825330085 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f84900b3550 0x7f849015b500 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.951+0000 7f8496933640 1 -- 172.21.15.81:0/1825330085 >> 172.21.15.81:0/1825330085 conn(0x7f849006a6a0 msgr2=0x7f84900b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:14.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.951+0000 7f8496933640 1 -- 172.21.15.81:0/1825330085 shutdown_connections 2023-11-08T16:42:14.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.951+0000 7f8496933640 1 -- 172.21.15.81:0/1825330085 wait complete. 2023-11-08T16:42:14.962 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.953+0000 7f8496933640 1 Processor -- start 2023-11-08T16:42:14.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.953+0000 7f8496933640 1 -- start start 2023-11-08T16:42:14.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.954+0000 7f8496933640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f849006c9f0 0x7f8490159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:14.963 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.954+0000 7f8496933640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84900b3170 0x7f849013b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:14.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.954+0000 7f8496933640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84900b3550 0x7f849013b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:14.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.954+0000 7f8496933640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8490143090 con 0x7f849006c9f0 2023-11-08T16:42:14.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.954+0000 7f8496933640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84901e72f0 con 0x7f84900b3170 2023-11-08T16:42:14.964 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.954+0000 7f8496933640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84901e6530 con 0x7f84900b3550 2023-11-08T16:42:14.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.954+0000 7f848ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f849006c9f0 0x7f8490159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:14.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.954+0000 7f848ffff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f849006c9f0 0x7f8490159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51416/0 (socket says 172.21.15.81:51416) 2023-11-08T16:42:14.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.954+0000 7f848ffff640 1 -- 172.21.15.81:0/3172910842 learned_addr learned my addr 172.21.15.81:0/3172910842 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:14.965 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f848ffff640 1 -- 172.21.15.81:0/3172910842 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84900b3550 msgr2=0x7f849013b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:14.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f8494ed0640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84900b3550 0x7f849013b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:14.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f8487fff640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84900b3170 0x7f849013b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:14.966 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f848ffff640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84900b3550 0x7f849013b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f848ffff640 1 -- 172.21.15.81:0/3172910842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84900b3170 msgr2=0x7f849013b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:14.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f848ffff640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84900b3170 0x7f849013b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:14.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f848ffff640 1 -- 172.21.15.81:0/3172910842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84780094d0 con 0x7f849006c9f0 2023-11-08T16:42:14.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f848ffff640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f849006c9f0 0x7f8490159880 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f848000c410 tx=0x7f848000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:14.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f848d7fa640 1 -- 172.21.15.81:0/3172910842 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84800035f0 con 0x7f849006c9f0 2023-11-08T16:42:14.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f848d7fa640 1 -- 172.21.15.81:0/3172910842 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8480003750 con 0x7f849006c9f0 2023-11-08T16:42:14.968 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f8496933640 1 -- 172.21.15.81:0/3172910842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849006d000 con 0x7f849006c9f0 2023-11-08T16:42:14.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.955+0000 7f848d7fa640 1 -- 172.21.15.81:0/3172910842 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8480003b40 con 0x7f849006c9f0 2023-11-08T16:42:14.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.956+0000 7f8496933640 1 -- 172.21.15.81:0/3172910842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84901e6af0 con 0x7f849006c9f0 2023-11-08T16:42:14.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.957+0000 7f8496933640 1 -- 172.21.15.81:0/3172910842 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8490074440 con 0x7f849006c9f0 2023-11-08T16:42:14.969 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.958+0000 7f848d7fa640 1 -- 172.21.15.81:0/3172910842 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8480003ca0 con 0x7f849006c9f0 2023-11-08T16:42:14.970 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.959+0000 7f848d7fa640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847006ae70 0x7f847006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:14.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.959+0000 7f848d7fa640 1 -- 172.21.15.81:0/3172910842 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f8480088a00 con 0x7f849006c9f0 2023-11-08T16:42:14.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.959+0000 7f8487fff640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847006ae70 0x7f847006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:14.971 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.960+0000 7f8487fff640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847006ae70 0x7f847006d330 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8478036000 tx=0x7f847803c040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:14.972 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:14.963+0000 7f848d7fa640 1 -- 172.21.15.81:0/3172910842 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f8480053d00 con 0x7f849006c9f0 2023-11-08T16:42:15.151 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.149+0000 7f8496933640 1 -- 172.21.15.81:0/3172910842 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f849006ce80 con 0x7f847006ae70 2023-11-08T16:42:15.153 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.152+0000 7f848d7fa640 1 -- 172.21.15.81:0/3172910842 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48545 (secure 0 0 0) 0x7f8478031c40 con 0x7f847006ae70 2023-11-08T16:42:15.154 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:15.155 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:42:15.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 -- 172.21.15.81:0/3172910842 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847006ae70 msgr2=0x7f847006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847006ae70 0x7f847006d330 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f8478036000 tx=0x7f847803c040 comp rx=0 tx=0).stop 2023-11-08T16:42:15.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 -- 172.21.15.81:0/3172910842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f849006c9f0 msgr2=0x7f8490159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f849006c9f0 0x7f8490159880 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f848000c410 tx=0x7f848000c8d0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 -- 172.21.15.81:0/3172910842 shutdown_connections 2023-11-08T16:42:15.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f847006ae70 0x7f847006d330 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f84900b3550 0x7f849013b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f84900b3170 0x7f849013b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 --2- 172.21.15.81:0/3172910842 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f849006c9f0 0x7f8490159880 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 -- 172.21.15.81:0/3172910842 >> 172.21.15.81:0/3172910842 conn(0x7f849006a6a0 msgr2=0x7f849015ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:15.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 -- 172.21.15.81:0/3172910842 shutdown_connections 2023-11-08T16:42:15.161 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.155+0000 7f8496933640 1 -- 172.21.15.81:0/3172910842 wait complete. 2023-11-08T16:42:15.176 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":529,"stamp":"2023-11-08T16:42:14.829985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264712,"kb_used_data":15264,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710008,"statfs":{"total":773094113280,"available":772823048192,"internally_reserved":0,"allocated":15630336,"data_stored":7728915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-404392,"num_objects":-123,"num_object_clones":0,"num_object_copies":-246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-123,"num_whiteouts":0,"num_read":-1808,"num_read_kb":-1513,"num_write":-911,"num_write_kb":-891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-4,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001290"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":283,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-11-08T16:42:09.087078+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:42:09.087078+0000","last_peered":"2023-11-08T16:42:09.087078+0000","last_clean":"2023-11-08T16:42:09.087078+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:42:09.087078+0000","last_undegraded":"2023-11-08T16:42:09.087078+0000","last_fullsized":"2023-11-08T16:42:09.087078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33520,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338320,"statfs":{"total":96636764160,"available":96602439680,"internally_reserved":0,"allocated":2396160,"data_stored":1355111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0329999999999999,"15min":1.071},"min":{"1min":0.80200000000000005,"5min":0.56399999999999995,"15min":0.49299999999999999},"max":{"1min":1.5649999999999999,"5min":1.6879999999999999,"15min":3.1899999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.0900000000000001,"15min":1.1060000000000001},"min":{"1min":0.746,"5min":0.58299999999999996,"15min":0.53600000000000003},"max":{"1min":1.5660000000000001,"5min":1.6040000000000001,"15min":4.2729999999999997},"last":1.4590000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.103,"15min":1.111},"min":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.57899999999999996},"max":{"1min":1.5629999999999999,"5min":1.891,"15min":1.891},"last":1.393},{"interface":"front","average":{"1min":1.23,"5min":1.1719999999999999,"15min":1.204},"min":{"1min":0.69899999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":3.3740000000000001},"last":0.79500000000000004}]},{"osd":2,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.206},"min":{"1min":0.81799999999999995,"5min":0.70999999999999996,"15min":0.58399999999999996},"max":{"1min":1.591,"5min":1.8109999999999999,"15min":4.2249999999999996},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.228,"5min":1.252,"15min":1.224},"min":{"1min":0.67700000000000005,"5min":0.66000000000000003,"15min":0.56299999999999994},"max":{"1min":1.587,"5min":1.7010000000000001,"15min":3.4750000000000001},"last":1.5029999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.1819999999999999,"15min":1.175},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.54900000000000004},"max":{"1min":1.71,"5min":1.71,"15min":2.964},"last":1.173},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.365,"15min":1.403},"min":{"1min":1.0029999999999999,"5min":0.86899999999999999,"15min":0.58599999999999997},"max":{"1min":1.619,"5min":1.7490000000000001,"15min":4.181},"last":1.847}]},{"osd":4,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.234,"15min":1.2350000000000001},"min":{"1min":0.93899999999999995,"5min":0.80400000000000005,"15min":0.77200000000000002},"max":{"1min":1.778,"5min":1.7789999999999999,"15min":3.6669999999999998},"last":1.633},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.325,"15min":1.337},"min":{"1min":0.95599999999999996,"5min":0.872,"15min":0.59099999999999997},"max":{"1min":1.7470000000000001,"5min":2.0179999999999998,"15min":2.3730000000000002},"last":1.6910000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.415,"15min":1.3879999999999999},"min":{"1min":1.0569999999999999,"5min":0.89200000000000002,"15min":0.61799999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":3.1059999999999999},"last":1.954},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4279999999999999,"15min":1.4530000000000001},"min":{"1min":1.1000000000000001,"5min":1.0589999999999999,"15min":0.878},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.9750000000000001},"last":1.8}]},{"osd":6,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.528,"15min":1.5329999999999999},"min":{"1min":1.083,"5min":1.0800000000000001,"15min":0.94799999999999995},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":3.7530000000000001},"last":1.891},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.4770000000000001,"15min":1.486},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.73999999999999999},"max":{"1min":1.831,"5min":2.0430000000000001,"15min":6.9560000000000004},"last":1.9239999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1765376,"data_stored":770406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.093,"15min":1.0720000000000001},"min":{"1min":0.69399999999999995,"5min":0.60399999999999998,"15min":0.497},"max":{"1min":1.569,"5min":1.7230000000000001,"15min":2.5609999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.113,"5min":1.131,"15min":1.097},"min":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.56200000000000006},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.7519999999999998},"last":0.68799999999999994}]},{"osd":1,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.131,"15min":1.0880000000000001},"min":{"1min":0.63400000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.548,"5min":1.667,"15min":1.667},"last":0.75},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.159,"15min":1.137},"min":{"1min":0.72599999999999998,"5min":0.71899999999999997,"15min":0.65600000000000003},"max":{"1min":1.573,"5min":1.7290000000000001,"15min":1.97},"last":0.89200000000000002}]},{"osd":2,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.175,"15min":1.1830000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.56799999999999995},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1990000000000001,"15min":1.171},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.60499999999999998},"max":{"1min":1.639,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.0820000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2190000000000001,"15min":1.2390000000000001},"min":{"1min":0.83399999999999996,"5min":0.63200000000000001,"15min":0.58299999999999996},"max":{"1min":1.73,"5min":1.73,"15min":2.1120000000000001},"last":1.121},{"interface":"front","average":{"1min":1.331,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.88700000000000001,"5min":0.68200000000000005,"15min":0.67300000000000004},"max":{"1min":1.6930000000000001,"5min":1.7809999999999999,"15min":1.95},"last":1.5600000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3220000000000001,"15min":1.3320000000000001},"min":{"1min":0.89700000000000002,"5min":0.81799999999999995,"15min":0.65300000000000002},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":1.23},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.365,"15min":1.3859999999999999},"min":{"1min":0.93600000000000005,"5min":0.878,"15min":0.64900000000000002},"max":{"1min":1.8420000000000001,"5min":1.9590000000000001,"15min":2.7080000000000002},"last":1.6160000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3340000000000001,"15min":1.323},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.80500000000000005},"max":{"1min":1.7669999999999999,"5min":1.994,"15min":2.105},"last":1.325},{"interface":"front","average":{"1min":1.53,"5min":1.518,"15min":1.5309999999999999},"min":{"1min":1.0389999999999999,"5min":0.91100000000000003,"15min":0.78500000000000003},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.4700000000000002},"last":1.6419999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.5529999999999999},"min":{"1min":1.079,"5min":1.079,"15min":0.74299999999999999},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":3.8959999999999999},"last":1.673},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4850000000000001,"15min":1.496},"min":{"1min":1.0549999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.885,"5min":2.0870000000000002,"15min":2.4889999999999999},"last":1.595}]}]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":770310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.081,"15min":1.0720000000000001},"min":{"1min":0.70299999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.657,"5min":1.657,"15min":2.4609999999999999},"last":1.131},{"interface":"front","average":{"1min":1.052,"5min":1.097,"15min":1.1060000000000001},"min":{"1min":0.63700000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.474,"5min":1.6000000000000001,"15min":2.3639999999999999},"last":1.1859999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.236},"min":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.53000000000000003},"max":{"1min":1.554,"5min":1.649,"15min":3.8690000000000002},"last":1.073},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2330000000000001,"15min":1.244},"min":{"1min":0.77200000000000002,"5min":0.66400000000000003,"15min":0.64500000000000002},"max":{"1min":1.5589999999999999,"5min":1.8109999999999999,"15min":3.2349999999999999},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.141,"15min":1.1819999999999999},"min":{"1min":0.72299999999999998,"5min":0.66100000000000003,"15min":0.65800000000000003},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":2.6339999999999999},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.1859999999999999,"15min":1.1990000000000001},"min":{"1min":0.872,"5min":0.70499999999999996,"15min":0.65800000000000003},"max":{"1min":1.681,"5min":1.752,"15min":2.2559999999999998},"last":1.839}]},{"osd":3,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2529999999999999,"15min":1.268},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.66500000000000004},"max":{"1min":1.5089999999999999,"5min":1.6519999999999999,"15min":2.8330000000000002},"last":1.907},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2909999999999999,"15min":1.29},"min":{"1min":0.86899999999999999,"5min":0.753,"15min":0.68600000000000005},"max":{"1min":1.528,"5min":2.0470000000000002,"15min":2.5760000000000001},"last":1.5109999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.28,"15min":1.3160000000000001},"min":{"1min":0.871,"5min":0.871,"15min":0.61599999999999999},"max":{"1min":1.657,"5min":2.0019999999999998,"15min":3.9790000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.427},"min":{"1min":1.1240000000000001,"5min":1.0569999999999999,"15min":0.81499999999999995},"max":{"1min":1.591,"5min":1.8029999999999999,"15min":3.0209999999999999},"last":2.0630000000000002}]},{"osd":6,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.421,"15min":1.4099999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.752},"max":{"1min":1.6200000000000001,"5min":2.1560000000000001,"15min":2.5550000000000002},"last":1.9650000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.407,"15min":1.45},"min":{"1min":1.03,"5min":0.98499999999999999,"15min":0.93600000000000005},"max":{"1min":1.7509999999999999,"5min":1.9419999999999999,"15min":4.1189999999999998},"last":1.643}]},{"osd":7,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.502,"15min":1.5349999999999999},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0049999999999999},"max":{"1min":1.7929999999999999,"5min":2.0219999999999998,"15min":3.1160000000000001},"last":2.214},{"interface":"front","average":{"1min":1.516,"5min":1.5580000000000001,"15min":1.5509999999999999},"min":{"1min":1.22,"5min":1.075,"15min":0.96999999999999997},"max":{"1min":1.7470000000000001,"5min":2.2970000000000002,"15min":2.7050000000000001},"last":2.3340000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1744896,"data_stored":764105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.05,"15min":1.0760000000000001},"min":{"1min":0.80000000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.528,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.081,"5min":1.1080000000000001,"15min":1.091},"min":{"1min":0.78600000000000003,"5min":0.58299999999999996,"15min":0.45400000000000001},"max":{"1min":1.5680000000000001,"5min":1.772,"15min":1.772},"last":1.3700000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.1200000000000001},"min":{"1min":0.745,"5min":0.60099999999999998,"15min":0.56999999999999995},"max":{"1min":1.2849999999999999,"5min":1.51,"15min":2.0459999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.159,"15min":1.171},"min":{"1min":0.873,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.742},"last":1.401}]},{"osd":2,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.22,"15min":1.2110000000000001},"min":{"1min":0.89600000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5489999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.216,"15min":1.24},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.71699999999999997},"max":{"1min":1.524,"5min":1.7749999999999999,"15min":1.9319999999999999},"last":1.2609999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.306,"15min":1.284},"min":{"1min":0.80100000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.653,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2649999999999999,"15min":1.2709999999999999},"min":{"1min":0.82399999999999995,"5min":0.72799999999999998,"15min":0.65100000000000002},"max":{"1min":1.5660000000000001,"5min":1.8069999999999999,"15min":1.895},"last":1.1220000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3680000000000001,"15min":1.365},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.58799999999999997},"max":{"1min":1.8320000000000001,"5min":2.286,"15min":2.286},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.454,"15min":1.4330000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.61899999999999999},"max":{"1min":1.7450000000000001,"5min":2.1970000000000001,"15min":2.46},"last":1.2909999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.387,"15min":1.391},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.74299999999999999},"max":{"1min":1.6970000000000001,"5min":1.7769999999999999,"15min":2.5609999999999999},"last":1.494},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4870000000000001,"15min":1.522},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.9390000000000001,"5min":2.4780000000000002,"15min":2.6389999999999998},"last":1.554}]},{"osd":7,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5489999999999999,"15min":1.534},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.8100000000000001,"5min":2.1040000000000001,"15min":2.2639999999999998},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5509999999999999,"15min":1.5600000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8440000000000001,"5min":2.1070000000000002,"15min":2.6859999999999999},"last":1.3380000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1806336,"data_stored":770357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.82699999999999996,"15min":0.80900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.397,"5min":1.4930000000000001,"15min":1.9339999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93000000000000005,"15min":0.89800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.316},"max":{"1min":1.319,"5min":1.647,"15min":1.647},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.88600000000000001},"min":{"1min":0.48599999999999999,"5min":0.38600000000000001,"15min":0.36499999999999999},"max":{"1min":1.2729999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.318},{"interface":"front","average":{"1min":1.012,"5min":0.998,"15min":0.97299999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.378},"max":{"1min":1.52,"5min":1.5800000000000001,"15min":1.829},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.052,"15min":1.0109999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41699999999999998},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":2.0779999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.099,"15min":1.103},"min":{"1min":0.57499999999999996,"5min":0.504,"15min":0.504},"max":{"1min":1.6000000000000001,"5min":1.7270000000000001,"15min":2.4660000000000002},"last":0.46999999999999997}]},{"osd":4,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.173,"15min":1.177},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.56399999999999995},"max":{"1min":1.571,"5min":1.5720000000000001,"15min":2.25},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.24,"5min":1.278,"15min":1.268},"min":{"1min":0.76200000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.6020000000000001,"5min":2.0110000000000001,"15min":2.1829999999999998},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.3300000000000001,"15min":1.3240000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.653,"5min":2.2229999999999999,"15min":2.3999999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.339,"15min":1.3300000000000001},"min":{"1min":0.879,"5min":0.80400000000000005,"15min":0.60999999999999999},"max":{"1min":1.6399999999999999,"5min":1.887,"15min":2.593},"last":0.88500000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3720000000000001,"15min":1.373},"min":{"1min":0.81899999999999995,"5min":0.751,"15min":0.58799999999999997},"max":{"1min":1.6180000000000001,"5min":1.9810000000000001,"15min":2.9569999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.3360000000000001,"15min":1.3460000000000001},"min":{"1min":0.86299999999999999,"5min":0.83599999999999997,"15min":0.73399999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.762},"last":0.73899999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3700000000000001,"15min":1.387},"min":{"1min":0.997,"5min":0.73699999999999999,"15min":0.70799999999999996},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.8700000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.4179999999999999,"15min":1.411},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7609999999999999,"5min":2.121,"15min":2.633},"last":0.83099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1712128,"data_stored":770312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81799999999999995,"15min":0.872},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.24099999999999999},"max":{"1min":1.0880000000000001,"5min":1.335,"15min":4.5369999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91600000000000004,"15min":0.88},"min":{"1min":0.60999999999999999,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":1.546,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.69599999999999995}]},{"osd":1,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0669999999999999,"15min":1.05},"min":{"1min":0.64900000000000002,"5min":0.44900000000000001,"15min":0.379},"max":{"1min":1.6160000000000001,"5min":1.7689999999999999,"15min":4.4980000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.157,"5min":1.143,"15min":1.0629999999999999},"min":{"1min":0.63800000000000001,"5min":0.47299999999999998,"15min":0.40300000000000002},"max":{"1min":1.7190000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.83399999999999996}]},{"osd":3,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.92400000000000004,"15min":0.93600000000000005},"min":{"1min":0.47299999999999998,"5min":0.44400000000000001,"15min":0.35499999999999998},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":3.3050000000000002},"last":0.876},{"interface":"front","average":{"1min":1.099,"5min":1.0669999999999999,"15min":1.1060000000000001},"min":{"1min":0.73899999999999999,"5min":0.44800000000000001,"15min":0.42999999999999999},"max":{"1min":1.796,"5min":1.819,"15min":4.5899999999999999},"last":1.1419999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2889999999999999,"15min":1.3160000000000001},"min":{"1min":0.88100000000000001,"5min":0.76100000000000001,"15min":0.73499999999999999},"max":{"1min":1.6639999999999999,"5min":2.2160000000000002,"15min":4.9409999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2470000000000001,"15min":1.236},"min":{"1min":0.93500000000000005,"5min":0.73899999999999999,"15min":0.63100000000000001},"max":{"1min":1.6319999999999999,"5min":2.0739999999999998,"15min":2.46},"last":0.95299999999999996}]},{"osd":5,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.302,"15min":1.28},"min":{"1min":1.0009999999999999,"5min":0.84599999999999997,"15min":0.77200000000000002},"max":{"1min":1.512,"5min":1.9510000000000001,"15min":2.8530000000000002},"last":1.093},{"interface":"front","average":{"1min":1.335,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":1.032,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.77,"5min":2.5390000000000001,"15min":3.5680000000000001},"last":1.204}]},{"osd":6,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.625},"max":{"1min":1.8200000000000001,"5min":2.0859999999999999,"15min":3.536},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.395,"15min":1.423},"min":{"1min":0.97399999999999998,"5min":0.81200000000000006,"15min":0.72199999999999998},"max":{"1min":1.887,"5min":2.2719999999999998,"15min":4.7549999999999999},"last":1.486}]},{"osd":7,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3799999999999999,"15min":1.3779999999999999},"min":{"1min":0.99199999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6599999999999999,"5min":2.2930000000000001,"15min":2.9700000000000002},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.393,"15min":1.4119999999999999},"min":{"1min":0.84399999999999997,"5min":0.79600000000000004,"15min":0.70499999999999996},"max":{"1min":2.0099999999999998,"5min":2.129,"15min":3.4950000000000001},"last":1.3919999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1703936,"data_stored":770311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.77000000000000002,"15min":0.76800000000000002},"min":{"1min":0.443,"5min":0.34499999999999997,"15min":0.28899999999999998},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.542},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.875,"15min":0.93600000000000005},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":1.202,"5min":1.5429999999999999,"15min":4.0309999999999997},"last":1.258}]},{"osd":2,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.98699999999999999,"15min":1.0289999999999999},"min":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.45000000000000001},"max":{"1min":1.6020000000000001,"5min":1.7689999999999999,"15min":3.0339999999999998},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.93899999999999995,"15min":0.999},"min":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":1.5249999999999999,"5min":1.542,"15min":2.6840000000000002},"last":1.4530000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98199999999999998,"15min":1.0389999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.38900000000000001},"max":{"1min":1.4910000000000001,"5min":1.5129999999999999,"15min":4.3179999999999996},"last":1.139},{"interface":"front","average":{"1min":1.014,"5min":0.95799999999999996,"15min":1.0489999999999999},"min":{"1min":0.48699999999999999,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.4530000000000001,"5min":1.5900000000000001,"15min":8.5570000000000004},"last":1.073}]},{"osd":4,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.161,"15min":1.23},"min":{"1min":0.751,"5min":0.751,"15min":0.63300000000000001},"max":{"1min":1.804,"5min":1.907,"15min":3.1659999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.276,"5min":1.2230000000000001,"15min":1.2769999999999999},"min":{"1min":0.84899999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.659,"5min":1.823,"15min":2.734},"last":1.0209999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2410000000000001,"15min":1.421},"min":{"1min":0.874,"5min":0.72399999999999998,"15min":0.67100000000000004},"max":{"1min":1.621,"5min":1.921,"15min":8.4730000000000008},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.2509999999999999,"15min":1.4199999999999999},"min":{"1min":0.76200000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7669999999999999,"5min":1.9470000000000001,"15min":8.8040000000000003},"last":1.427}]},{"osd":6,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.28,"15min":1.3660000000000001},"min":{"1min":0.84799999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.681,"5min":1.9119999999999999,"15min":3.984},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.3100000000000001,"15min":1.3779999999999999},"min":{"1min":0.83599999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.8300000000000001,"5min":2.0449999999999999,"15min":3.2450000000000001},"last":1.1890000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.288,"15min":1.4019999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.69199999999999995},"max":{"1min":1.758,"5min":1.853,"15min":4.3609999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.369,"15min":1.4790000000000001},"min":{"1min":0.74399999999999999,"5min":0.72799999999999998,"15min":0.70499999999999996},"max":{"1min":1.974,"5min":1.974,"15min":5.3840000000000003},"last":1.554}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33912,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337928,"statfs":{"total":96636764160,"available":96602038272,"internally_reserved":0,"allocated":2797568,"data_stored":1758003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.42299999999999999,"5min":0.34000000000000002,"15min":0.29399999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.6220000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95399999999999996,"15min":0.96199999999999997},"min":{"1min":0.41099999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.5720000000000001,"5min":1.722,"15min":2.3340000000000001},"last":1.4299999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.89400000000000002,"15min":0.91500000000000004},"min":{"1min":0.48499999999999999,"5min":0.39900000000000002,"15min":0.37},"max":{"1min":1.4670000000000001,"5min":1.5860000000000001,"15min":1.762},"last":1.3420000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.92200000000000004,"15min":0.96199999999999997},"min":{"1min":0.502,"5min":0.38100000000000001,"15min":0.36799999999999999},"max":{"1min":1.4570000000000001,"5min":1.613,"15min":1.879},"last":1.2789999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.92900000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.34200000000000003},"max":{"1min":1.4510000000000001,"5min":1.72,"15min":1.728},"last":1.389},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.1539999999999999,"15min":1.177},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.7679999999999998},"last":1.25}]},{"osd":4,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1779999999999999,"15min":1.159},"min":{"1min":0.77200000000000002,"5min":0.67100000000000004,"15min":0.65300000000000002},"max":{"1min":1.429,"5min":2.0489999999999999,"15min":2.2559999999999998},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2529999999999999,"15min":1.276},"min":{"1min":0.874,"5min":0.69999999999999996,"15min":0.59399999999999997},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.867},"last":1.0489999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2709999999999999,"15min":1.264},"min":{"1min":0.84299999999999997,"5min":0.79600000000000004,"15min":0.67900000000000005},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.7280000000000002},"last":1.6899999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.234,"15min":1.2290000000000001},"min":{"1min":0.93300000000000005,"5min":0.70899999999999996,"15min":0.67600000000000005},"max":{"1min":1.5669999999999999,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.613}]},{"osd":6,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3440000000000001,"15min":1.3520000000000001},"min":{"1min":0.91700000000000004,"5min":0.80000000000000004,"15min":0.72499999999999998},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.6739999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.3260000000000001,"15min":1.345},"min":{"1min":0.89000000000000001,"5min":0.88200000000000001,"15min":0.64200000000000002},"max":{"1min":1.829,"5min":2.1280000000000001,"15min":2.8260000000000001},"last":1.5720000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.381,"15min":1.399},"min":{"1min":1.0109999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":4.5670000000000002},"last":1.645},{"interface":"front","average":{"1min":1.343,"5min":1.337,"15min":1.349},"min":{"1min":0.85399999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6080000000000001,"5min":2.6070000000000002,"15min":5.4450000000000003},"last":1.6679999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:42:15.178 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-08T16:42:15.179 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:42:15.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.388+0000 7f0c1eb25640 1 Processor -- start 2023-11-08T16:42:15.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.388+0000 7f0c1eb25640 1 -- start start 2023-11-08T16:42:15.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.389+0000 7f0c1eb25640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f0c180b3550 0x7f0c1815b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.390+0000 7f0c1eb25640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c180b2310 con 0x7f0c180b3170 2023-11-08T16:42:15.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.390+0000 7f0c1eb25640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c180b25f0 con 0x7f0c1806c9f0 2023-11-08T16:42:15.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.390+0000 7f0c1eb25640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c180b2bb0 con 0x7f0c180b3550 2023-11-08T16:42:15.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.390+0000 7f0c1c8c1640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f0c180b3170 0x7f0c180b1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54342/0 (socket says 172.21.15.81:54342) 2023-11-08T16:42:15.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.390+0000 7f0c1c8c1640 1 -- 172.21.15.81:0/618251461 learned_addr learned my addr 172.21.15.81:0/618251461 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:15.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.390+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3334328268 0 0) 0x7f0c180b2310 con 0x7f0c180b3170 2023-11-08T16:42:15.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.390+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c08003520 con 0x7f0c180b3170 2023-11-08T16:42:15.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.390+0000 7f0c17fff640 1 --2- 172.21.15.81:0/618251461 >> v2:172.21.15.93:3300/0 conn(0x7f0c180b3550 0x7f0c1815b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2776590215 0 0) 0x7f0c08003520 con 0x7f0c180b3170 2023-11-08T16:42:15.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 >> v2:172.21.15.93:3300/0 conn(0x7f0c180b3550 msgr2=0x7f0c1815b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:15.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 --2- 172.21.15.81:0/618251461 >> v2:172.21.15.93:3300/0 conn(0x7f0c180b3550 0x7f0c1815b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 >> v1:172.21.15.93:6789/0 conn(0x7f0c1806c9f0 legacy=0x7f0c18144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:15.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c180b28d0 con 0x7f0c180b3170 2023-11-08T16:42:15.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f0c0c001670 con 0x7f0c180b3170 2023-11-08T16:42:15.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 >> v1:172.21.15.81:6789/0 conn(0x7f0c180b3170 legacy=0x7f0c180b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 --2- 172.21.15.81:0/618251461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c08003970 0x7f0c08003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 --2- 172.21.15.81:0/618251461 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0c080047c0 0x7f0c08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 --2- 172.21.15.81:0/618251461 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c08005180 0x7f0c0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180b2bb0 con 0x7f0c08003970 2023-11-08T16:42:15.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0800c000 con 0x7f0c08005180 2023-11-08T16:42:15.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0800c1c0 con 0x7f0c080047c0 2023-11-08T16:42:15.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c1c8c1640 1 --2- 172.21.15.81:0/618251461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c08003970 0x7f0c08003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c1c8c1640 1 -- 172.21.15.81:0/618251461 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0c080047c0 msgr2=0x7f0c08004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:15.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c1c8c1640 1 --2- 172.21.15.81:0/618251461 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0c080047c0 0x7f0c08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c1c8c1640 1 -- 172.21.15.81:0/618251461 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c08005180 msgr2=0x7f0c0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:15.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c1c8c1640 1 --2- 172.21.15.81:0/618251461 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c08005180 0x7f0c0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c1c8c1640 1 -- 172.21.15.81:0/618251461 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08003520 con 0x7f0c08003970 2023-11-08T16:42:15.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.391+0000 7f0c1d0c2640 1 -- 172.21.15.81:0/618251461 reap_dead start 2023-11-08T16:42:15.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.392+0000 7f0c1c8c1640 1 --2- 172.21.15.81:0/618251461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c08003970 0x7f0c08003d70 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c210 tx=0x7f0c080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f414aef32433de server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:15.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.393+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c02cae0 con 0x7f0c08003970 2023-11-08T16:42:15.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.393+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c02cc60 con 0x7f0c08003970 2023-11-08T16:42:15.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.393+0000 7f0c177fe640 1 -- 172.21.15.81:0/618251461 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c005b10 con 0x7f0c08003970 2023-11-08T16:42:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.393+0000 7f0c1eb25640 1 -- 172.21.15.81:0/618251461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c08003970 msgr2=0x7f0c08003d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.393+0000 7f0c1eb25640 1 --2- 172.21.15.81:0/618251461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c08003970 0x7f0c08003d70 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00c210 tx=0x7f0c080057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.393+0000 7f0c1eb25640 1 -- 172.21.15.81:0/618251461 shutdown_connections 2023-11-08T16:42:15.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.393+0000 7f0c1eb25640 1 --2- 172.21.15.81:0/618251461 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c08003970 0x7f0c08003d70 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.393+0000 7f0c1eb25640 1 -- 172.21.15.81:0/618251461 >> 172.21.15.81:0/618251461 conn(0x7f0c1806a6a0 msgr2=0x7f0c180b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.393+0000 7f0c1eb25640 1 -- 172.21.15.81:0/618251461 shutdown_connections 2023-11-08T16:42:15.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.393+0000 7f0c1eb25640 1 -- 172.21.15.81:0/618251461 wait complete. 2023-11-08T16:42:15.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.395+0000 7f0c1eb25640 1 Processor -- start 2023-11-08T16:42:15.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.395+0000 7f0c1eb25640 1 -- start start 2023-11-08T16:42:15.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.396+0000 7f0c1eb25640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0c08003970 0x7f0c1814f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.396+0000 7f0c1eb25640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c1813da50 0x7f0c1813ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.396+0000 7f0c1eb25640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c180ab130 0x7f0c180ab5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.396+0000 7f0c1eb25640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180abaf0 con 0x7f0c180ab130 2023-11-08T16:42:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.396+0000 7f0c1eb25640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180a8c50 con 0x7f0c1813da50 2023-11-08T16:42:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.396+0000 7f0c1eb25640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180a8900 con 0x7f0c08003970 2023-11-08T16:42:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.397+0000 7f0c1d0c2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c180ab130 0x7f0c180ab5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.397+0000 7f0c1d0c2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c180ab130 0x7f0c180ab5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51430/0 (socket says 172.21.15.81:51430) 2023-11-08T16:42:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.397+0000 7f0c1d0c2640 1 -- 172.21.15.81:0/1306590065 learned_addr learned my addr 172.21.15.81:0/1306590065 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.397+0000 7f0c17fff640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c1813da50 0x7f0c1813ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.397+0000 7f0c1c8c1640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0c08003970 0x7f0c1814f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.397+0000 7f0c1d0c2640 1 -- 172.21.15.81:0/1306590065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0c08003970 msgr2=0x7f0c1814f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.397+0000 7f0c1d0c2640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0c08003970 0x7f0c1814f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.397+0000 7f0c1d0c2640 1 -- 172.21.15.81:0/1306590065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c1813da50 msgr2=0x7f0c1813ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.397+0000 7f0c1d0c2640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c1813da50 0x7f0c1813ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.397+0000 7f0c1d0c2640 1 -- 172.21.15.81:0/1306590065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1806d000 con 0x7f0c180ab130 2023-11-08T16:42:15.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.398+0000 7f0c1d0c2640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c180ab130 0x7f0c180ab5b0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f0c08004740 tx=0x7f0c1000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:15.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.398+0000 7f0c157fa640 1 -- 172.21.15.81:0/1306590065 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100070d0 con 0x7f0c180ab130 2023-11-08T16:42:15.412 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.398+0000 7f0c157fa640 1 -- 172.21.15.81:0/1306590065 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10007230 con 0x7f0c180ab130 2023-11-08T16:42:15.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.398+0000 7f0c157fa640 1 -- 172.21.15.81:0/1306590065 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10007620 con 0x7f0c180ab130 2023-11-08T16:42:15.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.398+0000 7f0c1eb25640 1 -- 172.21.15.81:0/1306590065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1813e4f0 con 0x7f0c180ab130 2023-11-08T16:42:15.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.398+0000 7f0c1eb25640 1 -- 172.21.15.81:0/1306590065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c18159f30 con 0x7f0c180ab130 2023-11-08T16:42:15.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.399+0000 7f0c1eb25640 1 -- 172.21.15.81:0/1306590065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1813ea10 con 0x7f0c180ab130 2023-11-08T16:42:15.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.400+0000 7f0c157fa640 1 -- 172.21.15.81:0/1306590065 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0c10007780 con 0x7f0c180ab130 2023-11-08T16:42:15.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.401+0000 7f0c157fa640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0bfc06aea0 0x7f0bfc06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.401+0000 7f0c157fa640 1 -- 172.21.15.81:0/1306590065 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f0c10088870 con 0x7f0c180ab130 2023-11-08T16:42:15.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.401+0000 7f0c1c8c1640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0bfc06aea0 0x7f0bfc06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.402+0000 7f0c1c8c1640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0bfc06aea0 0x7f0bfc06d360 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c034000 tx=0x7f0c0c009030 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:15.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.405+0000 7f0c157fa640 1 -- 172.21.15.81:0/1306590065 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f0c10053ad0 con 0x7f0c180ab130 2023-11-08T16:42:15.596 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.594+0000 7f0c1eb25640 1 -- 172.21.15.81:0/1306590065 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0c180a8ac0 con 0x7f0bfc06aea0 2023-11-08T16:42:15.601 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.600+0000 7f0c157fa640 1 -- 172.21.15.81:0/1306590065 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48545 (secure 0 0 0) 0x7f0c180a8ac0 con 0x7f0bfc06aea0 2023-11-08T16:42:15.602 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:15.603 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:42:15.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.602+0000 7f0c1eb25640 1 -- 172.21.15.81:0/1306590065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0bfc06aea0 msgr2=0x7f0bfc06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.604 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.602+0000 7f0c1eb25640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0bfc06aea0 0x7f0bfc06d360 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c034000 tx=0x7f0c0c009030 comp rx=0 tx=0).stop 2023-11-08T16:42:15.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.602+0000 7f0c1eb25640 1 -- 172.21.15.81:0/1306590065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c180ab130 msgr2=0x7f0c180ab5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.602+0000 7f0c1eb25640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c180ab130 0x7f0c180ab5b0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f0c08004740 tx=0x7f0c1000c960 comp rx=0 tx=0).stop 2023-11-08T16:42:15.605 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.603+0000 7f0c1eb25640 1 -- 172.21.15.81:0/1306590065 shutdown_connections 2023-11-08T16:42:15.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.603+0000 7f0c1eb25640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f0bfc06aea0 0x7f0bfc06d360 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.603+0000 7f0c1eb25640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f0c180ab130 0x7f0c180ab5b0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.606 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.603+0000 7f0c1eb25640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0c1813da50 0x7f0c1813ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.603+0000 7f0c1eb25640 1 --2- 172.21.15.81:0/1306590065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f0c08003970 0x7f0c1814f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.603+0000 7f0c1eb25640 1 -- 172.21.15.81:0/1306590065 >> 172.21.15.81:0/1306590065 conn(0x7f0c1806a6a0 msgr2=0x7f0c1815b590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:15.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.603+0000 7f0c1eb25640 1 -- 172.21.15.81:0/1306590065 shutdown_connections 2023-11-08T16:42:15.607 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.603+0000 7f0c1eb25640 1 -- 172.21.15.81:0/1306590065 wait complete. 2023-11-08T16:42:15.624 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":529,"stamp":"2023-11-08T16:42:14.829985+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264712,"kb_used_data":15264,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754710008,"statfs":{"total":773094113280,"available":772823048192,"internally_reserved":0,"allocated":15630336,"data_stored":7728915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-404392,"num_objects":-123,"num_object_clones":0,"num_object_copies":-246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-123,"num_whiteouts":0,"num_read":-1808,"num_read_kb":-1513,"num_write":-911,"num_write_kb":-891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-4,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001290"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":283,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-11-08T16:42:09.087078+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:42:09.087078+0000","last_peered":"2023-11-08T16:42:09.087078+0000","last_clean":"2023-11-08T16:42:09.087078+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:42:09.087078+0000","last_undegraded":"2023-11-08T16:42:09.087078+0000","last_fullsized":"2023-11-08T16:42:09.087078+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575010,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33520,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338320,"statfs":{"total":96636764160,"available":96602439680,"internally_reserved":0,"allocated":2396160,"data_stored":1355111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0329999999999999,"15min":1.071},"min":{"1min":0.80200000000000005,"5min":0.56399999999999995,"15min":0.49299999999999999},"max":{"1min":1.5649999999999999,"5min":1.6879999999999999,"15min":3.1899999999999999},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.143,"5min":1.0900000000000001,"15min":1.1060000000000001},"min":{"1min":0.746,"5min":0.58299999999999996,"15min":0.53600000000000003},"max":{"1min":1.5660000000000001,"5min":1.6040000000000001,"15min":4.2729999999999997},"last":1.4590000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.103,"15min":1.111},"min":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.57899999999999996},"max":{"1min":1.5629999999999999,"5min":1.891,"15min":1.891},"last":1.393},{"interface":"front","average":{"1min":1.23,"5min":1.1719999999999999,"15min":1.204},"min":{"1min":0.69899999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":3.3740000000000001},"last":0.79500000000000004}]},{"osd":2,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.206},"min":{"1min":0.81799999999999995,"5min":0.70999999999999996,"15min":0.58399999999999996},"max":{"1min":1.591,"5min":1.8109999999999999,"15min":4.2249999999999996},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.228,"5min":1.252,"15min":1.224},"min":{"1min":0.67700000000000005,"5min":0.66000000000000003,"15min":0.56299999999999994},"max":{"1min":1.587,"5min":1.7010000000000001,"15min":3.4750000000000001},"last":1.5029999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.1819999999999999,"15min":1.175},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.54900000000000004},"max":{"1min":1.71,"5min":1.71,"15min":2.964},"last":1.173},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.365,"15min":1.403},"min":{"1min":1.0029999999999999,"5min":0.86899999999999999,"15min":0.58599999999999997},"max":{"1min":1.619,"5min":1.7490000000000001,"15min":4.181},"last":1.847}]},{"osd":4,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.234,"15min":1.2350000000000001},"min":{"1min":0.93899999999999995,"5min":0.80400000000000005,"15min":0.77200000000000002},"max":{"1min":1.778,"5min":1.7789999999999999,"15min":3.6669999999999998},"last":1.633},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.325,"15min":1.337},"min":{"1min":0.95599999999999996,"5min":0.872,"15min":0.59099999999999997},"max":{"1min":1.7470000000000001,"5min":2.0179999999999998,"15min":2.3730000000000002},"last":1.6910000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.415,"15min":1.3879999999999999},"min":{"1min":1.0569999999999999,"5min":0.89200000000000002,"15min":0.61799999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":3.1059999999999999},"last":1.954},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4279999999999999,"15min":1.4530000000000001},"min":{"1min":1.1000000000000001,"5min":1.0589999999999999,"15min":0.878},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.9750000000000001},"last":1.8}]},{"osd":6,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.528,"15min":1.5329999999999999},"min":{"1min":1.083,"5min":1.0800000000000001,"15min":0.94799999999999995},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":3.7530000000000001},"last":1.891},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.4770000000000001,"15min":1.486},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.73999999999999999},"max":{"1min":1.831,"5min":2.0430000000000001,"15min":6.9560000000000004},"last":1.9239999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1765376,"data_stored":770406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.093,"15min":1.0720000000000001},"min":{"1min":0.69399999999999995,"5min":0.60399999999999998,"15min":0.497},"max":{"1min":1.569,"5min":1.7230000000000001,"15min":2.5609999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":1.113,"5min":1.131,"15min":1.097},"min":{"1min":0.65000000000000002,"5min":0.63100000000000001,"15min":0.56200000000000006},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.7519999999999998},"last":0.68799999999999994}]},{"osd":1,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.131,"15min":1.0880000000000001},"min":{"1min":0.63400000000000001,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.548,"5min":1.667,"15min":1.667},"last":0.75},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.159,"15min":1.137},"min":{"1min":0.72599999999999998,"5min":0.71899999999999997,"15min":0.65600000000000003},"max":{"1min":1.573,"5min":1.7290000000000001,"15min":1.97},"last":0.89200000000000002}]},{"osd":2,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.175,"15min":1.1830000000000001},"min":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.56799999999999995},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1990000000000001,"15min":1.171},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.60499999999999998},"max":{"1min":1.639,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.0820000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2290000000000001,"5min":1.2190000000000001,"15min":1.2390000000000001},"min":{"1min":0.83399999999999996,"5min":0.63200000000000001,"15min":0.58299999999999996},"max":{"1min":1.73,"5min":1.73,"15min":2.1120000000000001},"last":1.121},{"interface":"front","average":{"1min":1.331,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.88700000000000001,"5min":0.68200000000000005,"15min":0.67300000000000004},"max":{"1min":1.6930000000000001,"5min":1.7809999999999999,"15min":1.95},"last":1.5600000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.3220000000000001,"15min":1.3320000000000001},"min":{"1min":0.89700000000000002,"5min":0.81799999999999995,"15min":0.65300000000000002},"max":{"1min":2.238,"5min":2.238,"15min":2.238},"last":1.23},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.365,"15min":1.3859999999999999},"min":{"1min":0.93600000000000005,"5min":0.878,"15min":0.64900000000000002},"max":{"1min":1.8420000000000001,"5min":1.9590000000000001,"15min":2.7080000000000002},"last":1.6160000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3340000000000001,"15min":1.323},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.80500000000000005},"max":{"1min":1.7669999999999999,"5min":1.994,"15min":2.105},"last":1.325},{"interface":"front","average":{"1min":1.53,"5min":1.518,"15min":1.5309999999999999},"min":{"1min":1.0389999999999999,"5min":0.91100000000000003,"15min":0.78500000000000003},"max":{"1min":2.3490000000000002,"5min":2.3490000000000002,"15min":2.4700000000000002},"last":1.6419999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.53,"5min":1.53,"15min":1.5529999999999999},"min":{"1min":1.079,"5min":1.079,"15min":0.74299999999999999},"max":{"1min":2.6539999999999999,"5min":2.6539999999999999,"15min":3.8959999999999999},"last":1.673},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4850000000000001,"15min":1.496},"min":{"1min":1.0549999999999999,"5min":0.91900000000000004,"15min":0.91900000000000004},"max":{"1min":1.885,"5min":2.0870000000000002,"15min":2.4889999999999999},"last":1.595}]}]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":770310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.081,"15min":1.0720000000000001},"min":{"1min":0.70299999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.657,"5min":1.657,"15min":2.4609999999999999},"last":1.131},{"interface":"front","average":{"1min":1.052,"5min":1.097,"15min":1.1060000000000001},"min":{"1min":0.63700000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.474,"5min":1.6000000000000001,"15min":2.3639999999999999},"last":1.1859999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.224,"15min":1.236},"min":{"1min":0.73299999999999998,"5min":0.70599999999999996,"15min":0.53000000000000003},"max":{"1min":1.554,"5min":1.649,"15min":3.8690000000000002},"last":1.073},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.2330000000000001,"15min":1.244},"min":{"1min":0.77200000000000002,"5min":0.66400000000000003,"15min":0.64500000000000002},"max":{"1min":1.5589999999999999,"5min":1.8109999999999999,"15min":3.2349999999999999},"last":0.73999999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.141,"15min":1.1819999999999999},"min":{"1min":0.72299999999999998,"5min":0.66100000000000003,"15min":0.65800000000000003},"max":{"1min":1.7230000000000001,"5min":1.7230000000000001,"15min":2.6339999999999999},"last":1.7170000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.1859999999999999,"15min":1.1990000000000001},"min":{"1min":0.872,"5min":0.70499999999999996,"15min":0.65800000000000003},"max":{"1min":1.681,"5min":1.752,"15min":2.2559999999999998},"last":1.839}]},{"osd":3,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.2529999999999999,"15min":1.268},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.66500000000000004},"max":{"1min":1.5089999999999999,"5min":1.6519999999999999,"15min":2.8330000000000002},"last":1.907},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2909999999999999,"15min":1.29},"min":{"1min":0.86899999999999999,"5min":0.753,"15min":0.68600000000000005},"max":{"1min":1.528,"5min":2.0470000000000002,"15min":2.5760000000000001},"last":1.5109999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.28,"15min":1.3160000000000001},"min":{"1min":0.871,"5min":0.871,"15min":0.61599999999999999},"max":{"1min":1.657,"5min":2.0019999999999998,"15min":3.9790000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.3899999999999999,"15min":1.427},"min":{"1min":1.1240000000000001,"5min":1.0569999999999999,"15min":0.81499999999999995},"max":{"1min":1.591,"5min":1.8029999999999999,"15min":3.0209999999999999},"last":2.0630000000000002}]},{"osd":6,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.421,"15min":1.4099999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.752},"max":{"1min":1.6200000000000001,"5min":2.1560000000000001,"15min":2.5550000000000002},"last":1.9650000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.407,"15min":1.45},"min":{"1min":1.03,"5min":0.98499999999999999,"15min":0.93600000000000005},"max":{"1min":1.7509999999999999,"5min":1.9419999999999999,"15min":4.1189999999999998},"last":1.643}]},{"osd":7,"last update":"Wed Nov 8 16:41:17 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.502,"15min":1.5349999999999999},"min":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0049999999999999},"max":{"1min":1.7929999999999999,"5min":2.0219999999999998,"15min":3.1160000000000001},"last":2.214},{"interface":"front","average":{"1min":1.516,"5min":1.5580000000000001,"15min":1.5509999999999999},"min":{"1min":1.22,"5min":1.075,"15min":0.96999999999999997},"max":{"1min":1.7470000000000001,"5min":2.2970000000000002,"15min":2.7050000000000001},"last":2.3340000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32884,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338956,"statfs":{"total":96636764160,"available":96603090944,"internally_reserved":0,"allocated":1744896,"data_stored":764105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.05,"15min":1.0760000000000001},"min":{"1min":0.80000000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.528,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.081,"5min":1.1080000000000001,"15min":1.091},"min":{"1min":0.78600000000000003,"5min":0.58299999999999996,"15min":0.45400000000000001},"max":{"1min":1.5680000000000001,"5min":1.772,"15min":1.772},"last":1.3700000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.1000000000000001,"15min":1.1200000000000001},"min":{"1min":0.745,"5min":0.60099999999999998,"15min":0.56999999999999995},"max":{"1min":1.2849999999999999,"5min":1.51,"15min":2.0459999999999998},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.159,"15min":1.171},"min":{"1min":0.873,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.742},"last":1.401}]},{"osd":2,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.22,"15min":1.2110000000000001},"min":{"1min":0.89600000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.5489999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.216,"15min":1.24},"min":{"1min":0.83899999999999997,"5min":0.83899999999999997,"15min":0.71699999999999997},"max":{"1min":1.524,"5min":1.7749999999999999,"15min":1.9319999999999999},"last":1.2609999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.306,"15min":1.284},"min":{"1min":0.80100000000000005,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.653,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2649999999999999,"15min":1.2709999999999999},"min":{"1min":0.82399999999999995,"5min":0.72799999999999998,"15min":0.65100000000000002},"max":{"1min":1.5660000000000001,"5min":1.8069999999999999,"15min":1.895},"last":1.1220000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.4059999999999999,"5min":1.3680000000000001,"15min":1.365},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.58799999999999997},"max":{"1min":1.8320000000000001,"5min":2.286,"15min":2.286},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.403,"5min":1.454,"15min":1.4330000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.61899999999999999},"max":{"1min":1.7450000000000001,"5min":2.1970000000000001,"15min":2.46},"last":1.2909999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.387,"15min":1.391},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.74299999999999999},"max":{"1min":1.6970000000000001,"5min":1.7769999999999999,"15min":2.5609999999999999},"last":1.494},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.4870000000000001,"15min":1.522},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":1.9390000000000001,"5min":2.4780000000000002,"15min":2.6389999999999998},"last":1.554}]},{"osd":7,"last update":"Wed Nov 8 16:41:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5069999999999999,"5min":1.5489999999999999,"15min":1.534},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.8100000000000001,"5min":2.1040000000000001,"15min":2.2639999999999998},"last":1.5249999999999999},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.5509999999999999,"15min":1.5600000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8440000000000001,"5min":2.1070000000000002,"15min":2.6859999999999999},"last":1.3380000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575011,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1806336,"data_stored":770357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.82699999999999996,"15min":0.80900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.397,"5min":1.4930000000000001,"15min":1.9339999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93000000000000005,"15min":0.89800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.316},"max":{"1min":1.319,"5min":1.647,"15min":1.647},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.88600000000000001},"min":{"1min":0.48599999999999999,"5min":0.38600000000000001,"15min":0.36499999999999999},"max":{"1min":1.2729999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.318},{"interface":"front","average":{"1min":1.012,"5min":0.998,"15min":0.97299999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.378},"max":{"1min":1.52,"5min":1.5800000000000001,"15min":1.829},"last":0.51200000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.052,"15min":1.0109999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41699999999999998},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":2.0779999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.099,"15min":1.103},"min":{"1min":0.57499999999999996,"5min":0.504,"15min":0.504},"max":{"1min":1.6000000000000001,"5min":1.7270000000000001,"15min":2.4660000000000002},"last":0.46999999999999997}]},{"osd":4,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.173,"15min":1.177},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.56399999999999995},"max":{"1min":1.571,"5min":1.5720000000000001,"15min":2.25},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.24,"5min":1.278,"15min":1.268},"min":{"1min":0.76200000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.6020000000000001,"5min":2.0110000000000001,"15min":2.1829999999999998},"last":0.75600000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.3300000000000001,"15min":1.3240000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.653,"5min":2.2229999999999999,"15min":2.3999999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.355,"5min":1.339,"15min":1.3300000000000001},"min":{"1min":0.879,"5min":0.80400000000000005,"15min":0.60999999999999999},"max":{"1min":1.6399999999999999,"5min":1.887,"15min":2.593},"last":0.88500000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3720000000000001,"15min":1.373},"min":{"1min":0.81899999999999995,"5min":0.751,"15min":0.58799999999999997},"max":{"1min":1.6180000000000001,"5min":1.9810000000000001,"15min":2.9569999999999999},"last":1.0369999999999999},{"interface":"front","average":{"1min":1.407,"5min":1.3360000000000001,"15min":1.3460000000000001},"min":{"1min":0.86299999999999999,"5min":0.83599999999999997,"15min":0.73399999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.762},"last":0.73899999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3700000000000001,"15min":1.387},"min":{"1min":0.997,"5min":0.73699999999999999,"15min":0.70799999999999996},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.8700000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.4179999999999999,"15min":1.411},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7609999999999999,"5min":2.121,"15min":2.633},"last":0.83099999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32856,"kb_used_data":1672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338984,"statfs":{"total":96636764160,"available":96603119616,"internally_reserved":0,"allocated":1712128,"data_stored":770312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81799999999999995,"15min":0.872},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.24099999999999999},"max":{"1min":1.0880000000000001,"5min":1.335,"15min":4.5369999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91600000000000004,"15min":0.88},"min":{"1min":0.60999999999999999,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":1.546,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.69599999999999995}]},{"osd":1,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0669999999999999,"15min":1.05},"min":{"1min":0.64900000000000002,"5min":0.44900000000000001,"15min":0.379},"max":{"1min":1.6160000000000001,"5min":1.7689999999999999,"15min":4.4980000000000002},"last":0.78800000000000003},{"interface":"front","average":{"1min":1.157,"5min":1.143,"15min":1.0629999999999999},"min":{"1min":0.63800000000000001,"5min":0.47299999999999998,"15min":0.40300000000000002},"max":{"1min":1.7190000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.83399999999999996}]},{"osd":3,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.92400000000000004,"15min":0.93600000000000005},"min":{"1min":0.47299999999999998,"5min":0.44400000000000001,"15min":0.35499999999999998},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":3.3050000000000002},"last":0.876},{"interface":"front","average":{"1min":1.099,"5min":1.0669999999999999,"15min":1.1060000000000001},"min":{"1min":0.73899999999999999,"5min":0.44800000000000001,"15min":0.42999999999999999},"max":{"1min":1.796,"5min":1.819,"15min":4.5899999999999999},"last":1.1419999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2889999999999999,"15min":1.3160000000000001},"min":{"1min":0.88100000000000001,"5min":0.76100000000000001,"15min":0.73499999999999999},"max":{"1min":1.6639999999999999,"5min":2.2160000000000002,"15min":4.9409999999999998},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2470000000000001,"15min":1.236},"min":{"1min":0.93500000000000005,"5min":0.73899999999999999,"15min":0.63100000000000001},"max":{"1min":1.6319999999999999,"5min":2.0739999999999998,"15min":2.46},"last":0.95299999999999996}]},{"osd":5,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.302,"15min":1.28},"min":{"1min":1.0009999999999999,"5min":0.84599999999999997,"15min":0.77200000000000002},"max":{"1min":1.512,"5min":1.9510000000000001,"15min":2.8530000000000002},"last":1.093},{"interface":"front","average":{"1min":1.335,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":1.032,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.77,"5min":2.5390000000000001,"15min":3.5680000000000001},"last":1.204}]},{"osd":6,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.625},"max":{"1min":1.8200000000000001,"5min":2.0859999999999999,"15min":3.536},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.395,"15min":1.423},"min":{"1min":0.97399999999999998,"5min":0.81200000000000006,"15min":0.72199999999999998},"max":{"1min":1.887,"5min":2.2719999999999998,"15min":4.7549999999999999},"last":1.486}]},{"osd":7,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3799999999999999,"15min":1.3779999999999999},"min":{"1min":0.99199999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6599999999999999,"5min":2.2930000000000001,"15min":2.9700000000000002},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.393,"15min":1.4119999999999999},"min":{"1min":0.84399999999999997,"5min":0.79600000000000004,"15min":0.70499999999999996},"max":{"1min":2.0099999999999998,"5min":2.129,"15min":3.4950000000000001},"last":1.3919999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32848,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338992,"statfs":{"total":96636764160,"available":96603127808,"internally_reserved":0,"allocated":1703936,"data_stored":770311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.77000000000000002,"15min":0.76800000000000002},"min":{"1min":0.443,"5min":0.34499999999999997,"15min":0.28899999999999998},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.542},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.875,"15min":0.93600000000000005},"min":{"1min":0.41799999999999998,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":1.202,"5min":1.5429999999999999,"15min":4.0309999999999997},"last":1.258}]},{"osd":2,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.98699999999999999,"15min":1.0289999999999999},"min":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.45000000000000001},"max":{"1min":1.6020000000000001,"5min":1.7689999999999999,"15min":3.0339999999999998},"last":1.2889999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.93899999999999995,"15min":0.999},"min":{"1min":0.46899999999999997,"5min":0.46700000000000003,"15min":0.443},"max":{"1min":1.5249999999999999,"5min":1.542,"15min":2.6840000000000002},"last":1.4530000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97099999999999997,"5min":0.98199999999999998,"15min":1.0389999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.38900000000000001},"max":{"1min":1.4910000000000001,"5min":1.5129999999999999,"15min":4.3179999999999996},"last":1.139},{"interface":"front","average":{"1min":1.014,"5min":0.95799999999999996,"15min":1.0489999999999999},"min":{"1min":0.48699999999999999,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.4530000000000001,"5min":1.5900000000000001,"15min":8.5570000000000004},"last":1.073}]},{"osd":4,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.161,"15min":1.23},"min":{"1min":0.751,"5min":0.751,"15min":0.63300000000000001},"max":{"1min":1.804,"5min":1.907,"15min":3.1659999999999999},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.276,"5min":1.2230000000000001,"15min":1.2769999999999999},"min":{"1min":0.84899999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.659,"5min":1.823,"15min":2.734},"last":1.0209999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.2410000000000001,"15min":1.421},"min":{"1min":0.874,"5min":0.72399999999999998,"15min":0.67100000000000004},"max":{"1min":1.621,"5min":1.921,"15min":8.4730000000000008},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.2509999999999999,"15min":1.4199999999999999},"min":{"1min":0.76200000000000001,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.7669999999999999,"5min":1.9470000000000001,"15min":8.8040000000000003},"last":1.427}]},{"osd":6,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2709999999999999,"5min":1.28,"15min":1.3660000000000001},"min":{"1min":0.84799999999999998,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.681,"5min":1.9119999999999999,"15min":3.984},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.3100000000000001,"15min":1.3779999999999999},"min":{"1min":0.83599999999999997,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.8300000000000001,"5min":2.0449999999999999,"15min":3.2450000000000001},"last":1.1890000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:41:15 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.288,"15min":1.4019999999999999},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.69199999999999995},"max":{"1min":1.758,"5min":1.853,"15min":4.3609999999999998},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.369,"15min":1.4790000000000001},"min":{"1min":0.74399999999999999,"5min":0.72799999999999998,"15min":0.70499999999999996},"max":{"1min":1.974,"5min":1.974,"15min":5.3840000000000003},"last":1.554}]}]},{"osd":0,"up_from":13,"seq":55834575010,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33912,"kb_used_data":2732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337928,"statfs":{"total":96636764160,"available":96602038272,"internally_reserved":0,"allocated":2797568,"data_stored":1758003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82099999999999995,"15min":0.81999999999999995},"min":{"1min":0.42299999999999999,"5min":0.34000000000000002,"15min":0.29399999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.6220000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":0.95399999999999996,"15min":0.96199999999999997},"min":{"1min":0.41099999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.5720000000000001,"5min":1.722,"15min":2.3340000000000001},"last":1.4299999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.89400000000000002,"15min":0.91500000000000004},"min":{"1min":0.48499999999999999,"5min":0.39900000000000002,"15min":0.37},"max":{"1min":1.4670000000000001,"5min":1.5860000000000001,"15min":1.762},"last":1.3420000000000001},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.92200000000000004,"15min":0.96199999999999997},"min":{"1min":0.502,"5min":0.38100000000000001,"15min":0.36799999999999999},"max":{"1min":1.4570000000000001,"5min":1.613,"15min":1.879},"last":1.2789999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":0.92400000000000004,"5min":0.91500000000000004,"15min":0.92900000000000005},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.34200000000000003},"max":{"1min":1.4510000000000001,"5min":1.72,"15min":1.728},"last":1.389},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.1539999999999999,"15min":1.177},"min":{"1min":0.57199999999999995,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":2.1859999999999999,"5min":2.1859999999999999,"15min":2.7679999999999998},"last":1.25}]},{"osd":4,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1779999999999999,"15min":1.159},"min":{"1min":0.77200000000000002,"5min":0.67100000000000004,"15min":0.65300000000000002},"max":{"1min":1.429,"5min":2.0489999999999999,"15min":2.2559999999999998},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.2529999999999999,"15min":1.276},"min":{"1min":0.874,"5min":0.69999999999999996,"15min":0.59399999999999997},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.867},"last":1.0489999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2709999999999999,"15min":1.264},"min":{"1min":0.84299999999999997,"5min":0.79600000000000004,"15min":0.67900000000000005},"max":{"1min":2.3879999999999999,"5min":2.3879999999999999,"15min":2.7280000000000002},"last":1.6899999999999999},{"interface":"front","average":{"1min":1.2869999999999999,"5min":1.234,"15min":1.2290000000000001},"min":{"1min":0.93300000000000005,"5min":0.70899999999999996,"15min":0.67600000000000005},"max":{"1min":1.5669999999999999,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.613}]},{"osd":6,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3440000000000001,"15min":1.3520000000000001},"min":{"1min":0.91700000000000004,"5min":0.80000000000000004,"15min":0.72499999999999998},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.6739999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.3260000000000001,"15min":1.345},"min":{"1min":0.89000000000000001,"5min":0.88200000000000001,"15min":0.64200000000000002},"max":{"1min":1.829,"5min":2.1280000000000001,"15min":2.8260000000000001},"last":1.5720000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:41:13 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.381,"15min":1.399},"min":{"1min":1.0109999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.2589999999999999,"5min":2.2589999999999999,"15min":4.5670000000000002},"last":1.645},{"interface":"front","average":{"1min":1.343,"5min":1.337,"15min":1.349},"min":{"1min":0.85399999999999998,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.6080000000000001,"5min":2.6070000000000002,"15min":5.4450000000000003},"last":1.6679999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:42:15.626 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-08T16:42:15.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.837+0000 7fa7810f2640 1 Processor -- start 2023-11-08T16:42:15.839 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.838+0000 7fa7810f2640 1 -- start start 2023-11-08T16:42:15.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.839+0000 7fa7810f2640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fa77c15e0f0 0x7fa77c0b28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.839+0000 7fa7810f2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0cd1e0 con 0x7fa77c143800 2023-11-08T16:42:15.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.839+0000 7fa7810f2640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77c0cd4c0 con 0x7fa77c0cca70 2023-11-08T16:42:15.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.839+0000 7fa7810f2640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa77c0b2df0 con 0x7fa77c15e0f0 2023-11-08T16:42:15.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa77ad9d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fa77c143800 0x7fa77c15d9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54354/0 (socket says 172.21.15.81:54354) 2023-11-08T16:42:15.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa77ad9d640 1 -- 172.21.15.81:0/1481077801 learned_addr learned my addr 172.21.15.81:0/1481077801 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:15.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078666306 0 0) 0x7fa77c0cd1e0 con 0x7fa77c143800 2023-11-08T16:42:15.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa768003520 con 0x7fa77c143800 2023-11-08T16:42:15.845 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa773fff640 1 --2- 172.21.15.81:0/1481077801 >> v2:172.21.15.93:3300/0 conn(0x7fa77c15e0f0 0x7fa77c0b28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3310087056 0 0) 0x7fa768003520 con 0x7fa77c143800 2023-11-08T16:42:15.846 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 >> v2:172.21.15.93:3300/0 conn(0x7fa77c15e0f0 msgr2=0x7fa77c0b28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:15.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa77a59c640 1 --2- 172.21.15.81:0/1481077801 >> v2:172.21.15.93:3300/0 conn(0x7fa77c15e0f0 0x7fa77c0b28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 >> v1:172.21.15.93:6789/0 conn(0x7fa77c0cca70 legacy=0x7fa77c0704e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.847 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c0cd7a0 con 0x7fa77c143800 2023-11-08T16:42:15.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7fa76c001670 con 0x7fa77c143800 2023-11-08T16:42:15.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.840+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 >> v1:172.21.15.81:6789/0 conn(0x7fa77c143800 legacy=0x7fa77c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77a59c640 1 --2- 172.21.15.81:0/1481077801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa768003970 0x7fa768003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.848 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77a59c640 1 --2- 172.21.15.81:0/1481077801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7680047c0 0x7fa768004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77a59c640 1 --2- 172.21.15.81:0/1481077801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa768005180 0x7fa76800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c0b2df0 con 0x7fa768005180 2023-11-08T16:42:15.849 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76800c0d0 con 0x7fa7680047c0 2023-11-08T16:42:15.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76800c290 con 0x7fa768003970 2023-11-08T16:42:15.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77b59e640 1 --2- 172.21.15.81:0/1481077801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa768005180 0x7fa76800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.850 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa773fff640 1 --2- 172.21.15.81:0/1481077801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7680047c0 0x7fa768004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77ad9d640 1 --2- 172.21.15.81:0/1481077801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa768003970 0x7fa768003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77b59e640 1 -- 172.21.15.81:0/1481077801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa768003970 msgr2=0x7fa768003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:15.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77b59e640 1 --2- 172.21.15.81:0/1481077801 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa768003970 0x7fa768003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.851 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77b59e640 1 -- 172.21.15.81:0/1481077801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7680047c0 msgr2=0x7fa768004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:15.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77b59e640 1 --2- 172.21.15.81:0/1481077801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7680047c0 0x7fa768004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.841+0000 7fa77b59e640 1 -- 172.21.15.81:0/1481077801 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768003520 con 0x7fa768005180 2023-11-08T16:42:15.852 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.842+0000 7fa77b59e640 1 -- 172.21.15.81:0/1481077801 reap_dead start 2023-11-08T16:42:15.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.843+0000 7fa77b59e640 1 --2- 172.21.15.81:0/1481077801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa768005180 0x7fa76800b980 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fa774001e50 tx=0x7fa7680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fae7ca69239efd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:15.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.843+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774003b70 con 0x7fa768005180 2023-11-08T16:42:15.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.843+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa774003d10 con 0x7fa768005180 2023-11-08T16:42:15.853 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.843+0000 7fa77a59c640 1 -- 172.21.15.81:0/1481077801 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa774005310 con 0x7fa768005180 2023-11-08T16:42:15.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.843+0000 7fa7810f2640 1 -- 172.21.15.81:0/1481077801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa768005180 msgr2=0x7fa76800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.843+0000 7fa7810f2640 1 --2- 172.21.15.81:0/1481077801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa768005180 0x7fa76800b980 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7fa774001e50 tx=0x7fa7680057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.854 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.844+0000 7fa7810f2640 1 -- 172.21.15.81:0/1481077801 shutdown_connections 2023-11-08T16:42:15.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.844+0000 7fa7810f2640 1 --2- 172.21.15.81:0/1481077801 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa768005180 0x7fa76800b980 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.844+0000 7fa7810f2640 1 -- 172.21.15.81:0/1481077801 >> 172.21.15.81:0/1481077801 conn(0x7fa77c06a6a0 msgr2=0x7fa77c0b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:15.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.844+0000 7fa7810f2640 1 -- 172.21.15.81:0/1481077801 shutdown_connections 2023-11-08T16:42:15.855 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.844+0000 7fa7810f2640 1 -- 172.21.15.81:0/1481077801 wait complete. 2023-11-08T16:42:15.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.846+0000 7fa7810f2640 1 Processor -- start 2023-11-08T16:42:15.858 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.846+0000 7fa7810f2640 1 -- start start 2023-11-08T16:42:15.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.847+0000 7fa7810f2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa768005180 0x7fa77c1f5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.847+0000 7fa7810f2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa77c15c810 0x7fa77c15ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.859 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.847+0000 7fa7810f2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa77c1f6000 0x7fa77c1f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.847+0000 7fa7810f2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c15d380 con 0x7fa77c15c810 2023-11-08T16:42:15.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.847+0000 7fa7810f2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c1f69d0 con 0x7fa768005180 2023-11-08T16:42:15.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.847+0000 7fa7810f2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77c15bba0 con 0x7fa77c1f6000 2023-11-08T16:42:15.860 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.847+0000 7fa773fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa77c15c810 0x7fa77c15ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.847+0000 7fa773fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa77c15c810 0x7fa77c15ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51450/0 (socket says 172.21.15.81:51450) 2023-11-08T16:42:15.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.847+0000 7fa773fff640 1 -- 172.21.15.81:0/1570409808 learned_addr learned my addr 172.21.15.81:0/1570409808 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:15.861 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.847+0000 7fa77ad9d640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa768005180 0x7fa77c1f5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.847+0000 7fa77b59e640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa77c1f6000 0x7fa77c1f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.848+0000 7fa773fff640 1 -- 172.21.15.81:0/1570409808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa77c1f6000 msgr2=0x7fa77c1f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.848+0000 7fa773fff640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa77c1f6000 0x7fa77c1f6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.862 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.848+0000 7fa773fff640 1 -- 172.21.15.81:0/1570409808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa768005180 msgr2=0x7fa77c1f5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:15.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.848+0000 7fa773fff640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa768005180 0x7fa77c1f5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:15.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.848+0000 7fa773fff640 1 -- 172.21.15.81:0/1570409808 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa77c15acb0 con 0x7fa77c15c810 2023-11-08T16:42:15.863 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.848+0000 7fa773fff640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa77c15c810 0x7fa77c15ce40 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fa76400f580 tx=0x7fa76400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:15.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.848+0000 7fa7737fe640 1 -- 172.21.15.81:0/1570409808 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764013810 con 0x7fa77c15c810 2023-11-08T16:42:15.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.848+0000 7fa7737fe640 1 -- 172.21.15.81:0/1570409808 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa764013970 con 0x7fa77c15c810 2023-11-08T16:42:15.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.848+0000 7fa7737fe640 1 -- 172.21.15.81:0/1570409808 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764013ce0 con 0x7fa77c15c810 2023-11-08T16:42:15.864 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.848+0000 7fa7810f2640 1 -- 172.21.15.81:0/1570409808 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77c15af90 con 0x7fa77c15c810 2023-11-08T16:42:15.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.848+0000 7fa7810f2640 1 -- 172.21.15.81:0/1570409808 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77c15b550 con 0x7fa77c15c810 2023-11-08T16:42:15.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.850+0000 7fa7810f2640 1 -- 172.21.15.81:0/1570409808 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77c15bd50 con 0x7fa77c15c810 2023-11-08T16:42:15.865 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.850+0000 7fa7737fe640 1 -- 172.21.15.81:0/1570409808 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa76401f460 con 0x7fa77c15c810 2023-11-08T16:42:15.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.851+0000 7fa7737fe640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa75c06aea0 0x7fa75c06d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:15.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.851+0000 7fa7737fe640 1 -- 172.21.15.81:0/1570409808 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fa764089700 con 0x7fa77c15c810 2023-11-08T16:42:15.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.852+0000 7fa77ad9d640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa75c06aea0 0x7fa75c06d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:15.866 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.852+0000 7fa77ad9d640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa75c06aea0 0x7fa75c06d360 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa76c001050 tx=0x7fa76c007040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:15.867 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:15.855+0000 7fa7737fe640 1 -- 172.21.15.81:0/1570409808 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fa77c15bd50 con 0x7fa77c15c810 2023-11-08T16:42:16.044 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.042+0000 7fa7810f2640 1 -- 172.21.15.81:0/1570409808 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa77c15bd50 con 0x7fa77c15c810 2023-11-08T16:42:16.046 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.045+0000 7fa7737fe640 1 -- 172.21.15.81:0/1570409808 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+13194 (secure 0 0 0) 0x7fa77c15bd50 con 0x7fa77c15c810 2023-11-08T16:42:16.046 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:16.047 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":134,"fsid":"49b142e1-3087-4ba1-853b-235f756dae94","created":"2023-11-08T16:28:37.018840+0000","modified":"2023-11-08T16:42:12.133556+0000","last_up_change":"2023-11-08T16:28:50.812230+0000","last_in_change":"2023-11-08T16:28:41.218568+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-08T16:28:52.620854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6cdeacf5-8ee5-4738-8d05-2576d080248f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6809","nonce":1666604866}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6811","nonce":1666604866}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6815","nonce":1666604866}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":1666604866},{"type":"v1","addr":"172.21.15.81:6813","nonce":1666604866}]},"public_addr":"172.21.15.81:6809/1666604866","cluster_addr":"172.21.15.81:6811/1666604866","heartbeat_back_addr":"172.21.15.81:6815/1666604866","heartbeat_front_addr":"172.21.15.81:6813/1666604866","state":["exists","up"]},{"osd":1,"uuid":"1618ae25-61cf-4e42-b167-a75cccfd9d7b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6801","nonce":2060798739}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6803","nonce":2060798739}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6807","nonce":2060798739}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":2060798739},{"type":"v1","addr":"172.21.15.81:6805","nonce":2060798739}]},"public_addr":"172.21.15.81:6801/2060798739","cluster_addr":"172.21.15.81:6803/2060798739","heartbeat_back_addr":"172.21.15.81:6807/2060798739","heartbeat_front_addr":"172.21.15.81:6805/2060798739","state":["exists","up"]},{"osd":2,"uuid":"fc50be73-b7fa-4c89-9ac0-a7568f2525b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6825","nonce":3759311428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6827","nonce":3759311428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6831","nonce":3759311428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":3759311428},{"type":"v1","addr":"172.21.15.81:6829","nonce":3759311428}]},"public_addr":"172.21.15.81:6825/3759311428","cluster_addr":"172.21.15.81:6827/3759311428","heartbeat_back_addr":"172.21.15.81:6831/3759311428","heartbeat_front_addr":"172.21.15.81:6829/3759311428","state":["exists","up"]},{"osd":3,"uuid":"d9aad8ef-8624-40c1-9504-bd6525972f3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":129,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6817","nonce":2756229107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6819","nonce":2756229107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6823","nonce":2756229107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":2756229107},{"type":"v1","addr":"172.21.15.81:6821","nonce":2756229107}]},"public_addr":"172.21.15.81:6817/2756229107","cluster_addr":"172.21.15.81:6819/2756229107","heartbeat_back_addr":"172.21.15.81:6823/2756229107","heartbeat_front_addr":"172.21.15.81:6821/2756229107","state":["exists","up"]},{"osd":4,"uuid":"908bf7ff-953c-4d13-b343-4e552afd723e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6801","nonce":3259756279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6803","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6805","nonce":3259756279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6811","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6813","nonce":3259756279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6807","nonce":3259756279},{"type":"v1","addr":"172.21.15.93:6809","nonce":3259756279}]},"public_addr":"172.21.15.93:6801/3259756279","cluster_addr":"172.21.15.93:6805/3259756279","heartbeat_back_addr":"172.21.15.93:6813/3259756279","heartbeat_front_addr":"172.21.15.93:6809/3259756279","state":["exists","up"]},{"osd":5,"uuid":"1d1e2108-9aeb-4523-a9d1-5c9a83e2e9a3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6804","nonce":2406579932}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6808","nonce":2406579932}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6815","nonce":2406579932}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":2406579932},{"type":"v1","addr":"172.21.15.93:6812","nonce":2406579932}]},"public_addr":"172.21.15.93:6804/2406579932","cluster_addr":"172.21.15.93:6808/2406579932","heartbeat_back_addr":"172.21.15.93:6815/2406579932","heartbeat_front_addr":"172.21.15.93:6812/2406579932","state":["exists","up"]},{"osd":6,"uuid":"d8989a20-6fea-4830-aa5f-5648e5867300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6817","nonce":607960134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6819","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6821","nonce":607960134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6827","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6829","nonce":607960134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6823","nonce":607960134},{"type":"v1","addr":"172.21.15.93:6825","nonce":607960134}]},"public_addr":"172.21.15.93:6817/607960134","cluster_addr":"172.21.15.93:6821/607960134","heartbeat_back_addr":"172.21.15.93:6829/607960134","heartbeat_front_addr":"172.21.15.93:6825/607960134","state":["exists","up"]},{"osd":7,"uuid":"0d61f873-a7e7-4d54-a7bf-c81a1d0c2b50","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":125,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6820","nonce":433691658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6824","nonce":433691658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6831","nonce":433691658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":433691658},{"type":"v1","addr":"172.21.15.93:6828","nonce":433691658}]},"public_addr":"172.21.15.93:6820/433691658","cluster_addr":"172.21.15.93:6824/433691658","heartbeat_back_addr":"172.21.15.93:6831/433691658","heartbeat_front_addr":"172.21.15.93:6828/433691658","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.793438+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.583584+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.578397+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.747600+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.604525+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.626619+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.610260+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-08T16:28:46.665224+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.81:6837/1432485913":"2023-11-09T16:42:09.074914+0000","172.21.15.81:6836/1432485913":"2023-11-09T16:42:09.074914+0000","172.21.15.81:6837/296230272":"2023-11-09T16:41:49.127674+0000","172.21.15.81:6836/296230272":"2023-11-09T16:41:49.127674+0000","172.21.15.93:6835/4021510722":"2023-11-09T16:41:29.470203+0000","172.21.15.93:6833/4021510722":"2023-11-09T16:41:29.470203+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-08T16:42:16.048 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.047+0000 7fa7810f2640 1 -- 172.21.15.81:0/1570409808 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa75c06aea0 msgr2=0x7fa75c06d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:16.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.047+0000 7fa7810f2640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa75c06aea0 0x7fa75c06d360 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fa76c001050 tx=0x7fa76c007040 comp rx=0 tx=0).stop 2023-11-08T16:42:16.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.047+0000 7fa7810f2640 1 -- 172.21.15.81:0/1570409808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa77c15c810 msgr2=0x7fa77c15ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:16.049 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.047+0000 7fa7810f2640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa77c15c810 0x7fa77c15ce40 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fa76400f580 tx=0x7fa76400fa40 comp rx=0 tx=0).stop 2023-11-08T16:42:16.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.047+0000 7fa7810f2640 1 -- 172.21.15.81:0/1570409808 shutdown_connections 2023-11-08T16:42:16.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.047+0000 7fa7810f2640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fa75c06aea0 0x7fa75c06d360 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:16.050 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.047+0000 7fa7810f2640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fa77c1f6000 0x7fa77c1f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:16.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.047+0000 7fa7810f2640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fa77c15c810 0x7fa77c15ce40 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:16.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.047+0000 7fa7810f2640 1 --2- 172.21.15.81:0/1570409808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa768005180 0x7fa77c1f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:16.051 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.047+0000 7fa7810f2640 1 -- 172.21.15.81:0/1570409808 >> 172.21.15.81:0/1570409808 conn(0x7fa77c06a6a0 msgr2=0x7fa77c06b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:16.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.047+0000 7fa7810f2640 1 -- 172.21.15.81:0/1570409808 shutdown_connections 2023-11-08T16:42:16.052 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:16.048+0000 7fa7810f2640 1 -- 172.21.15.81:0/1570409808 wait complete. 2023-11-08T16:42:17.067 INFO:tasks.ceph:Scrubbing osd.0 2023-11-08T16:42:17.068 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-08T16:42:17.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.292+0000 7efeb6de9640 1 Processor -- start 2023-11-08T16:42:17.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.292+0000 7efeb6de9640 1 -- start start 2023-11-08T16:42:17.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.294+0000 7efeb6de9640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7efeb00b4810 0x7efeb0070250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.294+0000 7efeb6de9640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efeb00cb140 0x7efeb006aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.294+0000 7efeb6de9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efeb006c260 0x7efeb006c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.294+0000 7efeb6de9640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb00cbc00 con 0x7efeb00cb140 2023-11-08T16:42:17.297 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.294+0000 7efeb6de9640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb006cbf0 con 0x7efeb00b4810 2023-11-08T16:42:17.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.294+0000 7efeb6de9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb006af50 con 0x7efeb006c260 2023-11-08T16:42:17.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.294+0000 7efeb55e6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efeb00cb140 0x7efeb006aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.294+0000 7efeb55e6640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7efeb00cb140 0x7efeb006aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51456/0 (socket says 172.21.15.81:51456) 2023-11-08T16:42:17.298 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.294+0000 7efeb55e6640 1 -- 172.21.15.81:0/548184227 learned_addr learned my addr 172.21.15.81:0/548184227 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.294+0000 7efeb5de7640 1 --2- 172.21.15.81:0/548184227 >> v2:172.21.15.93:3300/0 conn(0x7efeb00b4810 0x7efeb0070250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.294+0000 7efeb65e8640 1 --2- 172.21.15.81:0/548184227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efeb006c260 0x7efeb006c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.299 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.295+0000 7efeb55e6640 1 -- 172.21.15.81:0/548184227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efeb006c260 msgr2=0x7efeb006c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:17.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.295+0000 7efeb55e6640 1 --2- 172.21.15.81:0/548184227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efeb006c260 0x7efeb006c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.295+0000 7efeb55e6640 1 -- 172.21.15.81:0/548184227 >> v2:172.21.15.93:3300/0 conn(0x7efeb00b4810 msgr2=0x7efeb0070250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:17.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.295+0000 7efeb55e6640 1 --2- 172.21.15.81:0/548184227 >> v2:172.21.15.93:3300/0 conn(0x7efeb00b4810 0x7efeb0070250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.300 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.295+0000 7efeb55e6640 1 -- 172.21.15.81:0/548184227 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb00cae80 con 0x7efeb00cb140 2023-11-08T16:42:17.303 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.296+0000 7efeb55e6640 1 --2- 172.21.15.81:0/548184227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeb00cb140 0x7efeb006aa10 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7efeb00d0160 tx=0x7efea002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc21b99e23be917c server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:17.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.296+0000 7efeb4de5640 1 -- 172.21.15.81:0/548184227 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0031870 con 0x7efeb00cb140 2023-11-08T16:42:17.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.296+0000 7efeb4de5640 1 -- 172.21.15.81:0/548184227 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea00319d0 con 0x7efeb00cb140 2023-11-08T16:42:17.304 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.296+0000 7efeb4de5640 1 -- 172.21.15.81:0/548184227 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea0031d60 con 0x7efeb00cb140 2023-11-08T16:42:17.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.297+0000 7efeb6de9640 1 -- 172.21.15.81:0/548184227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeb00cb140 msgr2=0x7efeb006aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.297+0000 7efeb6de9640 1 --2- 172.21.15.81:0/548184227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeb00cb140 0x7efeb006aa10 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7efeb00d0160 tx=0x7efea002b910 comp rx=0 tx=0).stop 2023-11-08T16:42:17.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.297+0000 7efeb6de9640 1 -- 172.21.15.81:0/548184227 shutdown_connections 2023-11-08T16:42:17.305 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.297+0000 7efeb6de9640 1 --2- 172.21.15.81:0/548184227 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efeb006c260 0x7efeb006c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.297+0000 7efeb6de9640 1 --2- 172.21.15.81:0/548184227 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeb00cb140 0x7efeb006aa10 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.297+0000 7efeb6de9640 1 --2- 172.21.15.81:0/548184227 >> v2:172.21.15.93:3300/0 conn(0x7efeb00b4810 0x7efeb0070250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.306 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.297+0000 7efeb6de9640 1 -- 172.21.15.81:0/548184227 >> 172.21.15.81:0/548184227 conn(0x7efeb00cf2e0 msgr2=0x7efeb00cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:17.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.297+0000 7efeb6de9640 1 -- 172.21.15.81:0/548184227 shutdown_connections 2023-11-08T16:42:17.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.297+0000 7efeb6de9640 1 -- 172.21.15.81:0/548184227 wait complete. 2023-11-08T16:42:17.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.299+0000 7efeb6de9640 1 Processor -- start 2023-11-08T16:42:17.307 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.299+0000 7efeb6de9640 1 -- start start 2023-11-08T16:42:17.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb6de9640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efeb006c260 0x7efeb0153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb6de9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeb00b4810 0x7efeb01f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.308 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb6de9640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00cb140 0x7efeb0156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb6de9640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00ce0c0 con 0x7efeb00b4810 2023-11-08T16:42:17.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb6de9640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00cd160 con 0x7efeb00cb140 2023-11-08T16:42:17.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb6de9640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00cd440 con 0x7efeb006c260 2023-11-08T16:42:17.309 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb55e6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeb00b4810 0x7efeb01f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb55e6640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeb00b4810 0x7efeb01f2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51466/0 (socket says 172.21.15.81:51466) 2023-11-08T16:42:17.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb55e6640 1 -- 172.21.15.81:0/1999174785 learned_addr learned my addr 172.21.15.81:0/1999174785 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:17.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb5de7640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efeb006c260 0x7efeb0153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.310 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb65e8640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00cb140 0x7efeb0156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb55e6640 1 -- 172.21.15.81:0/1999174785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efeb006c260 msgr2=0x7efeb0153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb55e6640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efeb006c260 0x7efeb0153250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.311 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb55e6640 1 -- 172.21.15.81:0/1999174785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00cb140 msgr2=0x7efeb0156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb55e6640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00cb140 0x7efeb0156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb55e6640 1 -- 172.21.15.81:0/1999174785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00094d0 con 0x7efeb00b4810 2023-11-08T16:42:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb65e8640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00cb140 0x7efeb0156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:17.312 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.301+0000 7efeb5de7640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7efeb006c260 0x7efeb0153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:17.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.302+0000 7efeb55e6640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeb00b4810 0x7efeb01f2b40 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7efea002b950 tx=0x7efea00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:17.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.302+0000 7efea67fc640 1 -- 172.21.15.81:0/1999174785 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea003ad30 con 0x7efeb00b4810 2023-11-08T16:42:17.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.302+0000 7efea67fc640 1 -- 172.21.15.81:0/1999174785 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea00443f0 con 0x7efeb00b4810 2023-11-08T16:42:17.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.302+0000 7efea67fc640 1 -- 172.21.15.81:0/1999174785 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00447a0 con 0x7efeb00b4810 2023-11-08T16:42:17.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.302+0000 7efeb6de9640 1 -- 172.21.15.81:0/1999174785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb00cd720 con 0x7efeb00b4810 2023-11-08T16:42:17.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.302+0000 7efeb6de9640 1 -- 172.21.15.81:0/1999174785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb00cdce0 con 0x7efeb00b4810 2023-11-08T16:42:17.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.303+0000 7efeb6de9640 1 -- 172.21.15.81:0/1999174785 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efeb00b4f60 con 0x7efeb00b4810 2023-11-08T16:42:17.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.304+0000 7efea67fc640 1 -- 172.21.15.81:0/1999174785 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7efea0044900 con 0x7efeb00b4810 2023-11-08T16:42:17.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.305+0000 7efea67fc640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efe9406ae70 0x7efe9406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.305+0000 7efea67fc640 1 -- 172.21.15.81:0/1999174785 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7efea002d040 con 0x7efeb00b4810 2023-11-08T16:42:17.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.305+0000 7efea67fc640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7efe94070e10 0x7efe94073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.305+0000 7efea67fc640 1 -- 172.21.15.81:0/1999174785 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe940738e0 con 0x7efe94070e10 2023-11-08T16:42:17.319 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.305+0000 7efea67fc640 1 -- 172.21.15.81:0/1999174785 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7efea00b02a0 con 0x7efeb00b4810 2023-11-08T16:42:17.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.305+0000 7efeb5de7640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efe9406ae70 0x7efe9406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.305+0000 7efeb65e8640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7efe94070e10 0x7efe94073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.320 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.306+0000 7efeb5de7640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efe9406ae70 0x7efe9406d330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efeac006fd0 tx=0x7efeac008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:17.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.306+0000 7efeb65e8640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7efe94070e10 0x7efe94073230 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:17.321 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.307+0000 7efea67fc640 1 -- 172.21.15.81:0/1999174785 <== osd.0 v2:172.21.15.81:6808/1666604866 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7efe940738e0 con 0x7efe94070e10 2023-11-08T16:42:17.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.321+0000 7efeb6de9640 1 -- 172.21.15.81:0/1999174785 --> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efeb00cb640 con 0x7efe94070e10 2023-11-08T16:42:17.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.321+0000 7efea67fc640 1 -- 172.21.15.81:0/1999174785 <== osd.0 v2:172.21.15.81:6808/1666604866 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efea8003990 con 0x7efe94070e10 2023-11-08T16:42:17.323 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:42:17.323 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T16:42:17.324 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:42:17.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.321+0000 7efeb6de9640 1 -- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7efe94070e10 msgr2=0x7efe94073230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.321+0000 7efeb6de9640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:6808/1666604866,v1:172.21.15.81:6809/1666604866] conn(0x7efe94070e10 0x7efe94073230 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.322+0000 7efeb6de9640 1 -- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efe9406ae70 msgr2=0x7efe9406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.322+0000 7efeb6de9640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7efe9406ae70 0x7efe9406d330 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7efeac006fd0 tx=0x7efeac008040 comp rx=0 tx=0).stop 2023-11-08T16:42:17.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.322+0000 7efeb6de9640 1 -- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeb00b4810 msgr2=0x7efeb01f2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.322+0000 7efeb6de9640 1 --2- 172.21.15.81:0/1999174785 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7efeb00b4810 0x7efeb01f2b40 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7efea002b950 tx=0x7efea00046c0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.322+0000 7efeb65e8640 1 -- 172.21.15.81:0/1999174785 reap_dead start 2023-11-08T16:42:17.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.322+0000 7efeb6de9640 1 -- 172.21.15.81:0/1999174785 shutdown_connections 2023-11-08T16:42:17.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.322+0000 7efeb6de9640 1 -- 172.21.15.81:0/1999174785 >> 172.21.15.81:0/1999174785 conn(0x7efeb00cf2e0 msgr2=0x7efeb00cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:17.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.322+0000 7efeb6de9640 1 -- 172.21.15.81:0/1999174785 shutdown_connections 2023-11-08T16:42:17.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.322+0000 7efeb6de9640 1 -- 172.21.15.81:0/1999174785 wait complete. 2023-11-08T16:42:17.339 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-08T16:42:17.540 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.538+0000 7f234ef1d640 1 Processor -- start 2023-11-08T16:42:17.541 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.539+0000 7f234ef1d640 1 -- start start 2023-11-08T16:42:17.542 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.540+0000 7f234ef1d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23480b43d0 0x7f23480cc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.540+0000 7f234ef1d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f23480b4c10 0x7f234815e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.540+0000 7f234ef1d640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f234815e750 con 0x7f23480b47b0 2023-11-08T16:42:17.543 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.540+0000 7f234ef1d640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23480b5020 con 0x7f23480b4c10 2023-11-08T16:42:17.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.540+0000 7f234ef1d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23480af050 con 0x7f23480b43d0 2023-11-08T16:42:17.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.540+0000 7f234d4ba640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f23480b4c10 0x7f234815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.544 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.540+0000 7f234d4ba640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f23480b4c10 0x7f234815e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51468/0 (socket says 172.21.15.81:51468) 2023-11-08T16:42:17.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.540+0000 7f234d4ba640 1 -- 172.21.15.81:0/469888251 learned_addr learned my addr 172.21.15.81:0/469888251 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:17.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.541+0000 7f234ccb9640 1 --2- 172.21.15.81:0/469888251 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23480b43d0 0x7f23480cc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.545 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.541+0000 7f234d4ba640 1 -- 172.21.15.81:0/469888251 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23480b43d0 msgr2=0x7f23480cc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:17.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.541+0000 7f234d4ba640 1 --2- 172.21.15.81:0/469888251 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23480b43d0 0x7f23480cc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.541+0000 7f234d4ba640 1 -- 172.21.15.81:0/469888251 >> v1:172.21.15.93:6789/0 conn(0x7f23480b47b0 legacy=0x7f23480af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.546 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.541+0000 7f234d4ba640 1 -- 172.21.15.81:0/469888251 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23480ccd00 con 0x7f23480b4c10 2023-11-08T16:42:17.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.542+0000 7f234d4ba640 1 --2- 172.21.15.81:0/469888251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23480b4c10 0x7f234815e210 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2340009e50 tx=0x7f234002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e152ecf9913fe5e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:17.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.543+0000 7f23477fe640 1 -- 172.21.15.81:0/469888251 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f234002cb00 con 0x7f23480b4c10 2023-11-08T16:42:17.548 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.543+0000 7f23477fe640 1 -- 172.21.15.81:0/469888251 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f234002cc60 con 0x7f23480b4c10 2023-11-08T16:42:17.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.543+0000 7f23477fe640 1 -- 172.21.15.81:0/469888251 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23400343f0 con 0x7f23480b4c10 2023-11-08T16:42:17.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.543+0000 7f234ef1d640 1 -- 172.21.15.81:0/469888251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23480b4c10 msgr2=0x7f234815e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.549 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.543+0000 7f234ef1d640 1 --2- 172.21.15.81:0/469888251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23480b4c10 0x7f234815e210 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f2340009e50 tx=0x7f234002bf60 comp rx=0 tx=0).stop 2023-11-08T16:42:17.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.544+0000 7f234ef1d640 1 -- 172.21.15.81:0/469888251 shutdown_connections 2023-11-08T16:42:17.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.544+0000 7f234ef1d640 1 --2- 172.21.15.81:0/469888251 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23480b4c10 0x7f234815e210 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.550 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.544+0000 7f234ef1d640 1 --2- 172.21.15.81:0/469888251 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23480b43d0 0x7f23480cc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.544+0000 7f234ef1d640 1 -- 172.21.15.81:0/469888251 >> 172.21.15.81:0/469888251 conn(0x7f2348071870 msgr2=0x7f2348071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:17.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.544+0000 7f234ef1d640 1 -- 172.21.15.81:0/469888251 shutdown_connections 2023-11-08T16:42:17.551 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.544+0000 7f234ef1d640 1 -- 172.21.15.81:0/469888251 wait complete. 2023-11-08T16:42:17.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.546+0000 7f234ef1d640 1 Processor -- start 2023-11-08T16:42:17.552 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.546+0000 7f234ef1d640 1 -- start start 2023-11-08T16:42:17.554 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.547+0000 7f234ef1d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23480b43d0 0x7f23480ce320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.547+0000 7f234ef1d640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23480b47b0 0x7f23480c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.547+0000 7f234ef1d640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23480b4c10 0x7f23480c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.555 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.547+0000 7f234ef1d640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f234814f760 con 0x7f23480b43d0 2023-11-08T16:42:17.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.547+0000 7f234ef1d640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23480ce900 con 0x7f23480b47b0 2023-11-08T16:42:17.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.547+0000 7f234ef1d640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23480cebe0 con 0x7f23480b4c10 2023-11-08T16:42:17.556 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.547+0000 7f234ccb9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23480b43d0 0x7f23480ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.547+0000 7f234ccb9640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23480b43d0 0x7f23480ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51470/0 (socket says 172.21.15.81:51470) 2023-11-08T16:42:17.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.547+0000 7f234ccb9640 1 -- 172.21.15.81:0/1533391672 learned_addr learned my addr 172.21.15.81:0/1533391672 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:17.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.548+0000 7f2347fff640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23480b47b0 0x7f23480c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.557 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.548+0000 7f234ccb9640 1 -- 172.21.15.81:0/1533391672 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23480b4c10 msgr2=0x7f23480c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.548+0000 7f234d4ba640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23480b4c10 0x7f23480c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.548+0000 7f234ccb9640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23480b4c10 0x7f23480c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.558 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.548+0000 7f234ccb9640 1 -- 172.21.15.81:0/1533391672 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23480b47b0 msgr2=0x7f23480c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.548+0000 7f234ccb9640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23480b47b0 0x7f23480c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.548+0000 7f234ccb9640 1 -- 172.21.15.81:0/1533391672 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2340009b30 con 0x7f23480b43d0 2023-11-08T16:42:17.559 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.548+0000 7f2347fff640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23480b47b0 0x7f23480c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:17.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.548+0000 7f234ccb9640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23480b43d0 0x7f23480ce320 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f233800d5f0 tx=0x7f233800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:17.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.548+0000 7f23457fa640 1 -- 172.21.15.81:0/1533391672 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23380125f0 con 0x7f23480b43d0 2023-11-08T16:42:17.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.549+0000 7f23457fa640 1 -- 172.21.15.81:0/1533391672 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2338012750 con 0x7f23480b43d0 2023-11-08T16:42:17.560 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.549+0000 7f234ef1d640 1 -- 172.21.15.81:0/1533391672 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23480ceec0 con 0x7f23480b43d0 2023-11-08T16:42:17.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.549+0000 7f23457fa640 1 -- 172.21.15.81:0/1533391672 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2338012b00 con 0x7f23480b43d0 2023-11-08T16:42:17.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.549+0000 7f234ef1d640 1 -- 172.21.15.81:0/1533391672 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23480cf480 con 0x7f23480b43d0 2023-11-08T16:42:17.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.550+0000 7f234ef1d640 1 -- 172.21.15.81:0/1533391672 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23480cf640 con 0x7f23480b43d0 2023-11-08T16:42:17.561 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.551+0000 7f23457fa640 1 -- 172.21.15.81:0/1533391672 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2338012c60 con 0x7f23480b43d0 2023-11-08T16:42:17.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.552+0000 7f23457fa640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f232806ae70 0x7f232806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.552+0000 7f23457fa640 1 -- 172.21.15.81:0/1533391672 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f23380898a0 con 0x7f23480b43d0 2023-11-08T16:42:17.562 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.552+0000 7f2347fff640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f232806ae70 0x7f232806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.553+0000 7f2347fff640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f232806ae70 0x7f232806d330 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f234814fa30 tx=0x7f2330009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:17.563 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.556+0000 7f23457fa640 1 -- 172.21.15.81:0/1533391672 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f2338054b90 con 0x7f23480b43d0 2023-11-08T16:42:17.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.747+0000 7f234ef1d640 1 -- 172.21.15.81:0/1533391672 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f234806a110 con 0x7f232806ae70 2023-11-08T16:42:17.749 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.748+0000 7f23457fa640 1 -- 172.21.15.81:0/1533391672 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f23300067d0 con 0x7f232806ae70 2023-11-08T16:42:17.750 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 0 to deep-scrub 2023-11-08T16:42:17.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.750+0000 7f234ef1d640 1 -- 172.21.15.81:0/1533391672 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f232806ae70 msgr2=0x7f232806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.752 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.750+0000 7f234ef1d640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f232806ae70 0x7f232806d330 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f234814fa30 tx=0x7f2330009040 comp rx=0 tx=0).stop 2023-11-08T16:42:17.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.750+0000 7f234ef1d640 1 -- 172.21.15.81:0/1533391672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23480b43d0 msgr2=0x7f23480ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.750+0000 7f234ef1d640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23480b43d0 0x7f23480ce320 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f233800d5f0 tx=0x7f233800dab0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.753 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.750+0000 7f234ef1d640 1 -- 172.21.15.81:0/1533391672 shutdown_connections 2023-11-08T16:42:17.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.750+0000 7f234ef1d640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f232806ae70 0x7f232806d330 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.750+0000 7f234ef1d640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f23480b4c10 0x7f23480c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.750+0000 7f234ef1d640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f23480b47b0 0x7f23480c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.750+0000 7f234ef1d640 1 --2- 172.21.15.81:0/1533391672 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f23480b43d0 0x7f23480ce320 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.751+0000 7f234ef1d640 1 -- 172.21.15.81:0/1533391672 >> 172.21.15.81:0/1533391672 conn(0x7f2348071870 msgr2=0x7f23480c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:17.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.751+0000 7f234ef1d640 1 -- 172.21.15.81:0/1533391672 shutdown_connections 2023-11-08T16:42:17.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.751+0000 7f234ef1d640 1 -- 172.21.15.81:0/1533391672 wait complete. 2023-11-08T16:42:17.769 INFO:tasks.ceph:Scrubbing osd.1 2023-11-08T16:42:17.770 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-08T16:42:17.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.978+0000 7f748801a640 1 Processor -- start 2023-11-08T16:42:17.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.978+0000 7f748801a640 1 -- start start 2023-11-08T16:42:17.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.979+0000 7f748801a640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f74800cdb00 0x7f74800cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.980+0000 7f748801a640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74800d1620 con 0x7f74800d2a80 2023-11-08T16:42:17.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.980+0000 7f748801a640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74800d1900 con 0x7f7480160c40 2023-11-08T16:42:17.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.980+0000 7f748801a640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74800d1fc0 con 0x7f74800cdb00 2023-11-08T16:42:17.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.980+0000 7f74855b5640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f74800d2a80 0x7f7480160530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54368/0 (socket says 172.21.15.81:54368) 2023-11-08T16:42:17.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.980+0000 7f74855b5640 1 -- 172.21.15.81:0/2527042627 learned_addr learned my addr 172.21.15.81:0/2527042627 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:17.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.980+0000 7f7485db6640 1 --2- 172.21.15.81:0/2527042627 >> v2:172.21.15.81:3300/0 conn(0x7f74800cdb00 0x7f74800cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.980+0000 7f7484db4640 1 -- 172.21.15.81:0/2527042627 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933749973 0 0) 0x7f74800d1620 con 0x7f74800d2a80 2023-11-08T16:42:17.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.980+0000 7f7484db4640 1 -- 172.21.15.81:0/2527042627 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f746c003520 con 0x7f74800d2a80 2023-11-08T16:42:17.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.981+0000 7f7485db6640 1 -- 172.21.15.81:0/2527042627 >> v1:172.21.15.93:6789/0 conn(0x7f7480160c40 legacy=0x7f74800b6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:17.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.981+0000 7f7485db6640 1 -- 172.21.15.81:0/2527042627 >> v1:172.21.15.81:6789/0 conn(0x7f74800d2a80 legacy=0x7f7480160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.981+0000 7f7484db4640 1 -- 172.21.15.81:0/2527042627 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2093087068 0 0) 0x7f746c003520 con 0x7f74800d2a80 2023-11-08T16:42:17.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.981+0000 7f7485db6640 1 -- 172.21.15.81:0/2527042627 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74800d1be0 con 0x7f74800cdb00 2023-11-08T16:42:17.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.982+0000 7f7485db6640 1 --2- 172.21.15.81:0/2527042627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800cdb00 0x7f74800cdee0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f747c0097a0 tx=0x7f747c02b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d6cb042ff9e9262 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:17.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.982+0000 7f7484db4640 1 -- 172.21.15.81:0/2527042627 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470001670 con 0x7f74800cdb00 2023-11-08T16:42:17.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.982+0000 7f7484db4640 1 -- 172.21.15.81:0/2527042627 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c02e050 con 0x7f74800cdb00 2023-11-08T16:42:17.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.982+0000 7f7484db4640 1 -- 172.21.15.81:0/2527042627 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c030710 con 0x7f74800cdb00 2023-11-08T16:42:17.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.982+0000 7f748801a640 1 -- 172.21.15.81:0/2527042627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800cdb00 msgr2=0x7f74800cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.988 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.982+0000 7f748801a640 1 --2- 172.21.15.81:0/2527042627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800cdb00 0x7f74800cdee0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f747c0097a0 tx=0x7f747c02b790 comp rx=0 tx=0).stop 2023-11-08T16:42:17.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.983+0000 7f748801a640 1 -- 172.21.15.81:0/2527042627 shutdown_connections 2023-11-08T16:42:17.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.983+0000 7f748801a640 1 --2- 172.21.15.81:0/2527042627 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800cdb00 0x7f74800cdee0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.989 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.983+0000 7f748801a640 1 -- 172.21.15.81:0/2527042627 >> 172.21.15.81:0/2527042627 conn(0x7f748006aa30 msgr2=0x7f748006ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:17.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.983+0000 7f748801a640 1 -- 172.21.15.81:0/2527042627 shutdown_connections 2023-11-08T16:42:17.990 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.983+0000 7f748801a640 1 -- 172.21.15.81:0/2527042627 wait complete. 2023-11-08T16:42:17.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.984+0000 7f748801a640 1 Processor -- start 2023-11-08T16:42:17.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.985+0000 7f748801a640 1 -- start start 2023-11-08T16:42:17.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.985+0000 7f748801a640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800cdb00 0x7f74801f2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.985+0000 7f748801a640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800d2a80 0x7f74801582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.985+0000 7f748801a640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7480160c40 0x7f74800cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:17.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.985+0000 7f748801a640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7480158b90 con 0x7f74800d2a80 2023-11-08T16:42:17.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.985+0000 7f748801a640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74801f29d0 con 0x7f7480160c40 2023-11-08T16:42:17.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.985+0000 7f748801a640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74801f6b90 con 0x7f74800cdb00 2023-11-08T16:42:17.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.986+0000 7f74855b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800d2a80 0x7f74801582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.986+0000 7f74855b5640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800d2a80 0x7f74801582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51484/0 (socket says 172.21.15.81:51484) 2023-11-08T16:42:17.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.986+0000 7f74855b5640 1 -- 172.21.15.81:0/4002281448 learned_addr learned my addr 172.21.15.81:0/4002281448 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:17.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.986+0000 7f74865b7640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7480160c40 0x7f74800cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.986+0000 7f7485db6640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800cdb00 0x7f74801f2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:17.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.986+0000 7f74855b5640 1 -- 172.21.15.81:0/4002281448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800cdb00 msgr2=0x7f74801f2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.986+0000 7f74855b5640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74800cdb00 0x7f74801f2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.986+0000 7f74855b5640 1 -- 172.21.15.81:0/4002281448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7480160c40 msgr2=0x7f74800cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:17.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.986+0000 7f74855b5640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7480160c40 0x7f74800cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:17.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.986+0000 7f74855b5640 1 -- 172.21.15.81:0/4002281448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f747c009400 con 0x7f74800d2a80 2023-11-08T16:42:17.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.986+0000 7f74855b5640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800d2a80 0x7f74801582a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f747000c750 tx=0x7f747000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:17.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.987+0000 7f74767fc640 1 -- 172.21.15.81:0/4002281448 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470005ad0 con 0x7f74800d2a80 2023-11-08T16:42:17.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.987+0000 7f74767fc640 1 -- 172.21.15.81:0/4002281448 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7470005c50 con 0x7f74800d2a80 2023-11-08T16:42:17.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.987+0000 7f748801a640 1 -- 172.21.15.81:0/4002281448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74801f6260 con 0x7f74800d2a80 2023-11-08T16:42:18.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.987+0000 7f74767fc640 1 -- 172.21.15.81:0/4002281448 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7470007070 con 0x7f74800d2a80 2023-11-08T16:42:18.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.987+0000 7f748801a640 1 -- 172.21.15.81:0/4002281448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74801f6820 con 0x7f74800d2a80 2023-11-08T16:42:18.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.988+0000 7f748801a640 1 -- 172.21.15.81:0/4002281448 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f74800ce7a0 con 0x7f74800d2a80 2023-11-08T16:42:18.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.989+0000 7f74767fc640 1 -- 172.21.15.81:0/4002281448 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f74700071d0 con 0x7f74800d2a80 2023-11-08T16:42:18.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.990+0000 7f74767fc640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f746406ae70 0x7f746406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.991+0000 7f74767fc640 1 -- 172.21.15.81:0/4002281448 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f7470088b80 con 0x7f74800d2a80 2023-11-08T16:42:18.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.991+0000 7f7485db6640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f746406ae70 0x7f746406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.991+0000 7f74767fc640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f7464070e10 0x7f7464073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.991+0000 7f74767fc640 1 -- 172.21.15.81:0/4002281448 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74640738e0 con 0x7f7464070e10 2023-11-08T16:42:18.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.991+0000 7f74767fc640 1 -- 172.21.15.81:0/4002281448 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7f747008d260 con 0x7f74800d2a80 2023-11-08T16:42:18.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.991+0000 7f74865b7640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f7464070e10 0x7f7464073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.991+0000 7f7485db6640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f746406ae70 0x7f746406d330 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f747c034000 tx=0x7f747c02f000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.992+0000 7f74865b7640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f7464070e10 0x7f7464073230 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.004 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:17.992+0000 7f74767fc640 1 -- 172.21.15.81:0/4002281448 <== osd.1 v2:172.21.15.81:6800/2060798739 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f74640738e0 con 0x7f7464070e10 2023-11-08T16:42:18.008 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.007+0000 7f748801a640 1 -- 172.21.15.81:0/4002281448 --> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f74800d23e0 con 0x7f7464070e10 2023-11-08T16:42:18.009 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.007+0000 7f74767fc640 1 -- 172.21.15.81:0/4002281448 <== osd.1 v2:172.21.15.81:6800/2060798739 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f74780019d0 con 0x7f7464070e10 2023-11-08T16:42:18.009 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:42:18.009 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T16:42:18.010 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:42:18.011 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.007+0000 7f748801a640 1 -- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f7464070e10 msgr2=0x7f7464073230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.007+0000 7f748801a640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:6800/2060798739,v1:172.21.15.81:6801/2060798739] conn(0x7f7464070e10 0x7f7464073230 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.008+0000 7f748801a640 1 -- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f746406ae70 msgr2=0x7f746406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.008+0000 7f748801a640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f746406ae70 0x7f746406d330 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f747c034000 tx=0x7f747c02f000 comp rx=0 tx=0).stop 2023-11-08T16:42:18.012 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.008+0000 7f748801a640 1 -- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800d2a80 msgr2=0x7f74801582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.008+0000 7f748801a640 1 --2- 172.21.15.81:0/4002281448 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f74800d2a80 0x7f74801582a0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f747000c750 tx=0x7f747000cc10 comp rx=0 tx=0).stop 2023-11-08T16:42:18.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.008+0000 7f74865b7640 1 -- 172.21.15.81:0/4002281448 reap_dead start 2023-11-08T16:42:18.013 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.008+0000 7f748801a640 1 -- 172.21.15.81:0/4002281448 shutdown_connections 2023-11-08T16:42:18.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.008+0000 7f748801a640 1 -- 172.21.15.81:0/4002281448 >> 172.21.15.81:0/4002281448 conn(0x7f748006aa30 msgr2=0x7f7480155510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:18.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.008+0000 7f748801a640 1 -- 172.21.15.81:0/4002281448 shutdown_connections 2023-11-08T16:42:18.014 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.008+0000 7f748801a640 1 -- 172.21.15.81:0/4002281448 wait complete. 2023-11-08T16:42:18.025 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-08T16:42:18.238 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.236+0000 7f5633856640 1 Processor -- start 2023-11-08T16:42:18.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.237+0000 7f5633856640 1 -- start start 2023-11-08T16:42:18.239 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.238+0000 7f5633856640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0ae220 0x7f562c160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.238+0000 7f5633856640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c161650 con 0x7f562c15cfc0 2023-11-08T16:42:18.240 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.238+0000 7f5633856640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562c161930 con 0x7f562c15d3a0 2023-11-08T16:42:18.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.238+0000 7f5633856640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c161c10 con 0x7f562c0ae220 2023-11-08T16:42:18.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.238+0000 7f56315f2640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f562c15cfc0 0x7f562c06fba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54378/0 (socket says 172.21.15.81:54378) 2023-11-08T16:42:18.241 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.238+0000 7f56315f2640 1 -- 172.21.15.81:0/3860234557 learned_addr learned my addr 172.21.15.81:0/3860234557 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:18.244 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.239+0000 7f5630df1640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0ae220 0x7f562c160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.239+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915267217 0 0) 0x7f562c161650 con 0x7f562c15cfc0 2023-11-08T16:42:18.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.239+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5614003520 con 0x7f562c15cfc0 2023-11-08T16:42:18.245 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.239+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2034595553 0 0) 0x7f5614003520 con 0x7f562c15cfc0 2023-11-08T16:42:18.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.239+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0ae220 msgr2=0x7f562c160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:18.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.239+0000 7f562bfff640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0ae220 0x7f562c160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.246 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.239+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 >> v1:172.21.15.93:6789/0 conn(0x7f562c15d3a0 legacy=0x7f562c0cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.239+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c0c92d0 con 0x7f562c15cfc0 2023-11-08T16:42:18.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.239+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f561c0019f0 con 0x7f562c15cfc0 2023-11-08T16:42:18.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.239+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 >> v1:172.21.15.81:6789/0 conn(0x7f562c15cfc0 legacy=0x7f562c06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.247 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f562bfff640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56140039a0 0x7f5614003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f562bfff640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 0x7f5614004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f562bfff640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0ae220 0x7f5614005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.248 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c161c10 con 0x7f56140047c0 2023-11-08T16:42:18.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561400baf0 con 0x7f56140039a0 2023-11-08T16:42:18.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561400bcb0 con 0x7f562c0ae220 2023-11-08T16:42:18.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f5630df1640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 0x7f5614004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.249 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f56315f2640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56140039a0 0x7f5614003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f5631df3640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0ae220 0x7f5614005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f5630df1640 1 -- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0ae220 msgr2=0x7f5614005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:18.250 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f5630df1640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0ae220 0x7f5614005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f5630df1640 1 -- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56140039a0 msgr2=0x7f5614003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:18.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f5630df1640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56140039a0 0x7f5614003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.240+0000 7f5630df1640 1 -- 172.21.15.81:0/3860234557 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5614003520 con 0x7f56140047c0 2023-11-08T16:42:18.251 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.241+0000 7f56315f2640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56140039a0 0x7f5614003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-08T16:42:18.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.242+0000 7f5630df1640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 0x7f5614004c40 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f562c0ae660 tx=0x7f56140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f004181b7988d2c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.242+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562002c630 con 0x7f56140047c0 2023-11-08T16:42:18.252 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.242+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f562002c790 con 0x7f56140047c0 2023-11-08T16:42:18.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.242+0000 7f562bfff640 1 -- 172.21.15.81:0/3860234557 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562002cb20 con 0x7f56140047c0 2023-11-08T16:42:18.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.243+0000 7f5633856640 1 -- 172.21.15.81:0/3860234557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 msgr2=0x7f5614004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.243+0000 7f5633856640 1 --2- 172.21.15.81:0/3860234557 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 0x7f5614004c40 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f562c0ae660 tx=0x7f56140057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.253 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.243+0000 7f5631df3640 1 -- 172.21.15.81:0/3860234557 reap_dead start 2023-11-08T16:42:18.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.243+0000 7f5633856640 1 -- 172.21.15.81:0/3860234557 shutdown_connections 2023-11-08T16:42:18.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.243+0000 7f5633856640 1 -- 172.21.15.81:0/3860234557 >> 172.21.15.81:0/3860234557 conn(0x7f562c0b0770 msgr2=0x7f562c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:18.254 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.243+0000 7f5633856640 1 -- 172.21.15.81:0/3860234557 shutdown_connections 2023-11-08T16:42:18.255 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.243+0000 7f5633856640 1 -- 172.21.15.81:0/3860234557 wait complete. 2023-11-08T16:42:18.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.245+0000 7f5633856640 1 Processor -- start 2023-11-08T16:42:18.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.245+0000 7f5633856640 1 -- start start 2023-11-08T16:42:18.258 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f5633856640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 0x7f562c15e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f5633856640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f562c15ff20 0x7f562c149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f5633856640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0b43b0 0x7f562c0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.259 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f5633856640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15fb00 con 0x7f56140047c0 2023-11-08T16:42:18.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f5633856640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562c14a390 con 0x7f562c15ff20 2023-11-08T16:42:18.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f5633856640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f562c15ee80 con 0x7f562c0b43b0 2023-11-08T16:42:18.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f56315f2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 0x7f562c15e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.260 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f56315f2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 0x7f562c15e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51508/0 (socket says 172.21.15.81:51508) 2023-11-08T16:42:18.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f56315f2640 1 -- 172.21.15.81:0/1410369396 learned_addr learned my addr 172.21.15.81:0/1410369396 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:18.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f5630df1640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f562c15ff20 0x7f562c149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.261 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f56315f2640 1 -- 172.21.15.81:0/1410369396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0b43b0 msgr2=0x7f562c0b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f5631df3640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0b43b0 0x7f562c0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f56315f2640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0b43b0 0x7f562c0b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f56315f2640 1 -- 172.21.15.81:0/1410369396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f562c15ff20 msgr2=0x7f562c149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.262 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f56315f2640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f562c15ff20 0x7f562c149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f56315f2640 1 -- 172.21.15.81:0/1410369396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f562c15f160 con 0x7f56140047c0 2023-11-08T16:42:18.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.246+0000 7f5630df1640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f562c15ff20 0x7f562c149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:18.263 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.247+0000 7f56315f2640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 0x7f562c15e8a0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f561c001330 tx=0x7f561c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.247+0000 7f5629ffb640 1 -- 172.21.15.81:0/1410369396 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c00ee50 con 0x7f56140047c0 2023-11-08T16:42:18.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.247+0000 7f5629ffb640 1 -- 172.21.15.81:0/1410369396 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c013420 con 0x7f56140047c0 2023-11-08T16:42:18.264 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.247+0000 7f5633856640 1 -- 172.21.15.81:0/1410369396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f562c15f440 con 0x7f56140047c0 2023-11-08T16:42:18.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.247+0000 7f5629ffb640 1 -- 172.21.15.81:0/1410369396 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0137b0 con 0x7f56140047c0 2023-11-08T16:42:18.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.247+0000 7f5633856640 1 -- 172.21.15.81:0/1410369396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562c0b4830 con 0x7f56140047c0 2023-11-08T16:42:18.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.248+0000 7f5633856640 1 -- 172.21.15.81:0/1410369396 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f562c15fc40 con 0x7f56140047c0 2023-11-08T16:42:18.265 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.249+0000 7f5629ffb640 1 -- 172.21.15.81:0/1410369396 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f561c013910 con 0x7f56140047c0 2023-11-08T16:42:18.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.250+0000 7f5629ffb640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f561806aea0 0x7f561806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.250+0000 7f5629ffb640 1 -- 172.21.15.81:0/1410369396 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f561c089d10 con 0x7f56140047c0 2023-11-08T16:42:18.266 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.250+0000 7f5630df1640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f561806aea0 0x7f561806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.251+0000 7f5630df1640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f561806aea0 0x7f561806d360 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5620036000 tx=0x7f5620030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.255+0000 7f5629ffb640 1 -- 172.21.15.81:0/1410369396 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f562c15fc40 con 0x7f56140047c0 2023-11-08T16:42:18.450 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.448+0000 7f5633856640 1 -- 172.21.15.81:0/1410369396 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f562c0ce1a0 con 0x7f561806aea0 2023-11-08T16:42:18.451 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.449+0000 7f5629ffb640 1 -- 172.21.15.81:0/1410369396 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f562c0ce1a0 con 0x7f561806aea0 2023-11-08T16:42:18.451 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 1 to deep-scrub 2023-11-08T16:42:18.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.452+0000 7f5633856640 1 -- 172.21.15.81:0/1410369396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f561806aea0 msgr2=0x7f561806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.452+0000 7f5633856640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f561806aea0 0x7f561806d360 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5620036000 tx=0x7f5620030000 comp rx=0 tx=0).stop 2023-11-08T16:42:18.454 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.452+0000 7f5633856640 1 -- 172.21.15.81:0/1410369396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 msgr2=0x7f562c15e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.452+0000 7f5633856640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 0x7f562c15e8a0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f561c001330 tx=0x7f561c00e530 comp rx=0 tx=0).stop 2023-11-08T16:42:18.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.452+0000 7f5633856640 1 -- 172.21.15.81:0/1410369396 shutdown_connections 2023-11-08T16:42:18.455 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.452+0000 7f5633856640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f561806aea0 0x7f561806d360 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.452+0000 7f5633856640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f562c0b43b0 0x7f562c0b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.452+0000 7f5633856640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f562c15ff20 0x7f562c149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.452+0000 7f5633856640 1 --2- 172.21.15.81:0/1410369396 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f56140047c0 0x7f562c15e8a0 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.456 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.452+0000 7f5633856640 1 -- 172.21.15.81:0/1410369396 >> 172.21.15.81:0/1410369396 conn(0x7f562c0b0770 msgr2=0x7f562c0ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:18.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.452+0000 7f5633856640 1 -- 172.21.15.81:0/1410369396 shutdown_connections 2023-11-08T16:42:18.457 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.453+0000 7f5633856640 1 -- 172.21.15.81:0/1410369396 wait complete. 2023-11-08T16:42:18.473 INFO:tasks.ceph:Scrubbing osd.2 2023-11-08T16:42:18.474 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-08T16:42:18.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.678+0000 7f945b875640 1 Processor -- start 2023-11-08T16:42:18.680 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.678+0000 7f945b875640 1 -- start start 2023-11-08T16:42:18.681 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.680+0000 7f945b875640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f945406fbe0 0x7f945415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.680+0000 7f945b875640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945415aba0 con 0x7f945406bcb0 2023-11-08T16:42:18.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.680+0000 7f945b875640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945415ae80 con 0x7f94540ccdc0 2023-11-08T16:42:18.682 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.680+0000 7f945b875640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945415b160 con 0x7f945406fbe0 2023-11-08T16:42:18.684 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.680+0000 7f9458e10640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f945406bcb0 0x7f945406ffc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54390/0 (socket says 172.21.15.81:54390) 2023-11-08T16:42:18.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.680+0000 7f9458e10640 1 -- 172.21.15.81:0/4221691952 learned_addr learned my addr 172.21.15.81:0/4221691952 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:18.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f9459611640 1 --2- 172.21.15.81:0/4221691952 >> v2:172.21.15.93:3300/0 conn(0x7f945406fbe0 0x7f945415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.685 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431622557 0 0) 0x7f945415aba0 con 0x7f945406bcb0 2023-11-08T16:42:18.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9440003520 con 0x7f945406bcb0 2023-11-08T16:42:18.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 695283007 0 0) 0x7f9440003520 con 0x7f945406bcb0 2023-11-08T16:42:18.686 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 >> v2:172.21.15.93:3300/0 conn(0x7f945406fbe0 msgr2=0x7f945415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:18.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f944bfff640 1 --2- 172.21.15.81:0/4221691952 >> v2:172.21.15.93:3300/0 conn(0x7f945406fbe0 0x7f945415a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 >> v1:172.21.15.93:6789/0 conn(0x7f94540ccdc0 legacy=0x7f94540cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94540cc8a0 con 0x7f945406bcb0 2023-11-08T16:42:18.687 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f9444001670 con 0x7f945406bcb0 2023-11-08T16:42:18.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 >> v1:172.21.15.81:6789/0 conn(0x7f945406bcb0 legacy=0x7f945406ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f944bfff640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9440003970 0x7f9440003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.688 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.681+0000 7f944bfff640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94400047c0 0x7f9440004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f944bfff640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9440005180 0x7f944000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945415b160 con 0x7f94400047c0 2023-11-08T16:42:18.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f944000c120 con 0x7f9440005180 2023-11-08T16:42:18.689 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944000e4f0 con 0x7f9440003970 2023-11-08T16:42:18.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f9458e10640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94400047c0 0x7f9440004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f9459e12640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9440005180 0x7f944000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.690 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f9458e10640 1 -- 172.21.15.81:0/4221691952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9440003970 msgr2=0x7f9440003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:18.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f9459611640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9440003970 0x7f9440003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f9458e10640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9440003970 0x7f9440003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f9458e10640 1 -- 172.21.15.81:0/4221691952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9440005180 msgr2=0x7f944000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:18.691 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f9458e10640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9440005180 0x7f944000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f9458e10640 1 -- 172.21.15.81:0/4221691952 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9440003520 con 0x7f94400047c0 2023-11-08T16:42:18.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f9459e12640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9440005180 0x7f944000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:18.692 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.682+0000 7f9459e12640 1 -- 172.21.15.81:0/4221691952 reap_dead start 2023-11-08T16:42:18.693 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.683+0000 7f9458e10640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94400047c0 0x7f9440004c40 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f944400c210 tx=0x7f94400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeb0de860101a51d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.696 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.684+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9444005ad0 con 0x7f94400047c0 2023-11-08T16:42:18.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.684+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9444005c50 con 0x7f94400047c0 2023-11-08T16:42:18.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.684+0000 7f944bfff640 1 -- 172.21.15.81:0/4221691952 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944402c8d0 con 0x7f94400047c0 2023-11-08T16:42:18.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.684+0000 7f945b875640 1 -- 172.21.15.81:0/4221691952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94400047c0 msgr2=0x7f9440004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.697 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.684+0000 7f945b875640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94400047c0 0x7f9440004c40 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f944400c210 tx=0x7f94400057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.684+0000 7f945b875640 1 -- 172.21.15.81:0/4221691952 shutdown_connections 2023-11-08T16:42:18.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.684+0000 7f945b875640 1 --2- 172.21.15.81:0/4221691952 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94400047c0 0x7f9440004c40 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.698 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.684+0000 7f945b875640 1 -- 172.21.15.81:0/4221691952 >> 172.21.15.81:0/4221691952 conn(0x7f94540cec30 msgr2=0x7f94540cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:18.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.684+0000 7f945b875640 1 -- 172.21.15.81:0/4221691952 shutdown_connections 2023-11-08T16:42:18.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.684+0000 7f945b875640 1 -- 172.21.15.81:0/4221691952 wait complete. 2023-11-08T16:42:18.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.686+0000 7f945b875640 1 Processor -- start 2023-11-08T16:42:18.699 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.686+0000 7f945b875640 1 -- start start 2023-11-08T16:42:18.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.687+0000 7f945b875640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94400047c0 0x7f945415e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.687+0000 7f945b875640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9454153e60 0x7f94540afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.700 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.687+0000 7f945b875640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94540b00c0 0x7f9454149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.687+0000 7f945b875640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94540b0540 con 0x7f94540b00c0 2023-11-08T16:42:18.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.687+0000 7f945b875640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94541539d0 con 0x7f94400047c0 2023-11-08T16:42:18.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.687+0000 7f945b875640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9454152fa0 con 0x7f9454153e60 2023-11-08T16:42:18.701 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.688+0000 7f9459e12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94540b00c0 0x7f9454149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.688+0000 7f9459e12640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94540b00c0 0x7f9454149100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51528/0 (socket says 172.21.15.81:51528) 2023-11-08T16:42:18.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.688+0000 7f9459e12640 1 -- 172.21.15.81:0/2336307825 learned_addr learned my addr 172.21.15.81:0/2336307825 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:18.702 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.688+0000 7f9459611640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94400047c0 0x7f945415e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.688+0000 7f9458e10640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9454153e60 0x7f94540afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.688+0000 7f9459e12640 1 -- 172.21.15.81:0/2336307825 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9454153e60 msgr2=0x7f94540afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.703 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.688+0000 7f9459e12640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9454153e60 0x7f94540afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.688+0000 7f9459e12640 1 -- 172.21.15.81:0/2336307825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94400047c0 msgr2=0x7f945415e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.688+0000 7f9459e12640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f94400047c0 0x7f945415e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.688+0000 7f9459e12640 1 -- 172.21.15.81:0/2336307825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9454152cc0 con 0x7f94540b00c0 2023-11-08T16:42:18.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.688+0000 7f9459e12640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94540b00c0 0x7f9454149100 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f94540cb390 tx=0x7f944c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.689+0000 7f9449ffb640 1 -- 172.21.15.81:0/2336307825 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c003d80 con 0x7f94540b00c0 2023-11-08T16:42:18.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.689+0000 7f9449ffb640 1 -- 172.21.15.81:0/2336307825 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f944c005070 con 0x7f94540b00c0 2023-11-08T16:42:18.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.689+0000 7f945b875640 1 -- 172.21.15.81:0/2336307825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9454153380 con 0x7f94540b00c0 2023-11-08T16:42:18.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.689+0000 7f9449ffb640 1 -- 172.21.15.81:0/2336307825 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944c005420 con 0x7f94540b00c0 2023-11-08T16:42:18.707 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.689+0000 7f945b875640 1 -- 172.21.15.81:0/2336307825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9454152950 con 0x7f94540b00c0 2023-11-08T16:42:18.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.690+0000 7f945b875640 1 -- 172.21.15.81:0/2336307825 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f945415b440 con 0x7f94540b00c0 2023-11-08T16:42:18.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.691+0000 7f9449ffb640 1 -- 172.21.15.81:0/2336307825 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f944c005580 con 0x7f94540b00c0 2023-11-08T16:42:18.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.692+0000 7f9449ffb640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f943806ae70 0x7f943806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.692+0000 7f9449ffb640 1 -- 172.21.15.81:0/2336307825 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f944c088c60 con 0x7f94540b00c0 2023-11-08T16:42:18.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.692+0000 7f9459611640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f943806ae70 0x7f943806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.692+0000 7f9449ffb640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f9438070e10 0x7f9438073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.692+0000 7f9449ffb640 1 -- 172.21.15.81:0/2336307825 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94380738e0 con 0x7f9438070e10 2023-11-08T16:42:18.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.692+0000 7f9449ffb640 1 -- 172.21.15.81:0/2336307825 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7f944c08d2a0 con 0x7f94540b00c0 2023-11-08T16:42:18.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.692+0000 7f9458e10640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f9438070e10 0x7f9438073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.693+0000 7f9459611640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f943806ae70 0x7f943806d330 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f94540cb160 tx=0x7f9450008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.693+0000 7f9458e10640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f9438070e10 0x7f9438073230 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.712 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.694+0000 7f9449ffb640 1 -- 172.21.15.81:0/2336307825 <== osd.2 v2:172.21.15.81:6824/3759311428 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f94380738e0 con 0x7f9438070e10 2023-11-08T16:42:18.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.708+0000 7f945b875640 1 -- 172.21.15.81:0/2336307825 --> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f9454074440 con 0x7f9438070e10 2023-11-08T16:42:18.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.708+0000 7f9449ffb640 1 -- 172.21.15.81:0/2336307825 <== osd.2 v2:172.21.15.81:6824/3759311428 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f9454074440 con 0x7f9438070e10 2023-11-08T16:42:18.714 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:42:18.714 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T16:42:18.714 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:42:18.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.709+0000 7f945b875640 1 -- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f9438070e10 msgr2=0x7f9438073230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.709+0000 7f945b875640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:6824/3759311428,v1:172.21.15.81:6825/3759311428] conn(0x7f9438070e10 0x7f9438073230 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.709+0000 7f945b875640 1 -- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f943806ae70 msgr2=0x7f943806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.709+0000 7f945b875640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f943806ae70 0x7f943806d330 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f94540cb160 tx=0x7f9450008040 comp rx=0 tx=0).stop 2023-11-08T16:42:18.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.709+0000 7f945b875640 1 -- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94540b00c0 msgr2=0x7f9454149100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.709+0000 7f945b875640 1 --2- 172.21.15.81:0/2336307825 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f94540b00c0 0x7f9454149100 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f94540cb390 tx=0x7f944c00d9d0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.709+0000 7f9459e12640 1 -- 172.21.15.81:0/2336307825 reap_dead start 2023-11-08T16:42:18.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.709+0000 7f945b875640 1 -- 172.21.15.81:0/2336307825 shutdown_connections 2023-11-08T16:42:18.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.709+0000 7f945b875640 1 -- 172.21.15.81:0/2336307825 >> 172.21.15.81:0/2336307825 conn(0x7f94540cec30 msgr2=0x7f9454070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:18.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.709+0000 7f945b875640 1 -- 172.21.15.81:0/2336307825 shutdown_connections 2023-11-08T16:42:18.719 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.709+0000 7f945b875640 1 -- 172.21.15.81:0/2336307825 wait complete. 2023-11-08T16:42:18.728 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-08T16:42:18.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.920+0000 7f16fdb2e640 1 Processor -- start 2023-11-08T16:42:18.923 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.921+0000 7f16fdb2e640 1 -- start start 2023-11-08T16:42:18.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.922+0000 7f16fdb2e640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f16f80b3da0 0x7f16f806c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.922+0000 7f16fdb2e640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f80ce130 con 0x7f16f80cc5e0 2023-11-08T16:42:18.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.922+0000 7f16fdb2e640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f81442d0 con 0x7f16f80add30 2023-11-08T16:42:18.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.922+0000 7f16fdb2e640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f8143e20 con 0x7f16f80b3da0 2023-11-08T16:42:18.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.923+0000 7f16f7fff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f16f80cc5e0 0x7f16f806ba80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54402/0 (socket says 172.21.15.81:54402) 2023-11-08T16:42:18.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.923+0000 7f16f7fff640 1 -- 172.21.15.81:0/1110130545 learned_addr learned my addr 172.21.15.81:0/1110130545 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:18.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.923+0000 7f16fd32d640 1 --2- 172.21.15.81:0/1110130545 >> v2:172.21.15.81:3300/0 conn(0x7f16f80b3da0 0x7f16f806c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.923+0000 7f16f77fe640 1 -- 172.21.15.81:0/1110130545 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596988117 0 0) 0x7f16f80ce130 con 0x7f16f80cc5e0 2023-11-08T16:42:18.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.923+0000 7f16f77fe640 1 -- 172.21.15.81:0/1110130545 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16e8003520 con 0x7f16f80cc5e0 2023-11-08T16:42:18.928 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.923+0000 7f16fd32d640 1 -- 172.21.15.81:0/1110130545 >> v1:172.21.15.93:6789/0 conn(0x7f16f80add30 legacy=0x7f16f80cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.923+0000 7f16fd32d640 1 -- 172.21.15.81:0/1110130545 >> v1:172.21.15.81:6789/0 conn(0x7f16f80cc5e0 legacy=0x7f16f806ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.923+0000 7f16fd32d640 1 -- 172.21.15.81:0/1110130545 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f80b3910 con 0x7f16f80b3da0 2023-11-08T16:42:18.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.924+0000 7f16fd32d640 1 --2- 172.21.15.81:0/1110130545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16f80b3da0 0x7f16f806c190 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009d90 tx=0x7f16f002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59ac6e652888baa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.929 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.925+0000 7f16f77fe640 1 -- 172.21.15.81:0/1110130545 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f002cd00 con 0x7f16f80b3da0 2023-11-08T16:42:18.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.925+0000 7f16f77fe640 1 -- 172.21.15.81:0/1110130545 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f002ce60 con 0x7f16f80b3da0 2023-11-08T16:42:18.930 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.925+0000 7f16f77fe640 1 -- 172.21.15.81:0/1110130545 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f0034660 con 0x7f16f80b3da0 2023-11-08T16:42:18.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.925+0000 7f16fdb2e640 1 -- 172.21.15.81:0/1110130545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16f80b3da0 msgr2=0x7f16f806c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.925+0000 7f16fdb2e640 1 --2- 172.21.15.81:0/1110130545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16f80b3da0 0x7f16f806c190 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f16f0009d90 tx=0x7f16f002bf50 comp rx=0 tx=0).stop 2023-11-08T16:42:18.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.925+0000 7f16fdb2e640 1 -- 172.21.15.81:0/1110130545 shutdown_connections 2023-11-08T16:42:18.931 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.925+0000 7f16fdb2e640 1 --2- 172.21.15.81:0/1110130545 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16f80b3da0 0x7f16f806c190 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.925+0000 7f16fdb2e640 1 -- 172.21.15.81:0/1110130545 >> 172.21.15.81:0/1110130545 conn(0x7f16f80b07c0 msgr2=0x7f16f80b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:18.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.925+0000 7f16fdb2e640 1 -- 172.21.15.81:0/1110130545 shutdown_connections 2023-11-08T16:42:18.932 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.925+0000 7f16fdb2e640 1 -- 172.21.15.81:0/1110130545 wait complete. 2023-11-08T16:42:18.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.927+0000 7f16fdb2e640 1 Processor -- start 2023-11-08T16:42:18.935 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.927+0000 7f16fdb2e640 1 -- start start 2023-11-08T16:42:18.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.928+0000 7f16fdb2e640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f16f80add30 0x7f16f815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.928+0000 7f16fdb2e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16f80b3da0 0x7f16f81f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.936 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.928+0000 7f16fdb2e640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16f80cc5e0 0x7f16f815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.928+0000 7f16fdb2e640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f813ab60 con 0x7f16f80b3da0 2023-11-08T16:42:18.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.928+0000 7f16fdb2e640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f815ae60 con 0x7f16f80cc5e0 2023-11-08T16:42:18.937 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.928+0000 7f16fdb2e640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f81f3530 con 0x7f16f80add30 2023-11-08T16:42:18.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.928+0000 7f16f7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16f80b3da0 0x7f16f81f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16f7fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16f80b3da0 0x7f16f81f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51544/0 (socket says 172.21.15.81:51544) 2023-11-08T16:42:18.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16f7fff640 1 -- 172.21.15.81:0/3435202687 learned_addr learned my addr 172.21.15.81:0/3435202687 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:18.938 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16fd32d640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16f80cc5e0 0x7f16f815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16fcb2c640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f16f80add30 0x7f16f815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16f7fff640 1 -- 172.21.15.81:0/3435202687 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f16f80add30 msgr2=0x7f16f815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16f7fff640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f16f80add30 0x7f16f815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.940 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16f7fff640 1 -- 172.21.15.81:0/3435202687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16f80cc5e0 msgr2=0x7f16f815d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16f7fff640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16f80cc5e0 0x7f16f815d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16f7fff640 1 -- 172.21.15.81:0/3435202687 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f0009a70 con 0x7f16f80b3da0 2023-11-08T16:42:18.941 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16fcb2c640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f16f80add30 0x7f16f815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:18.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16fd32d640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16f80cc5e0 0x7f16f815d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:18.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16f7fff640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16f80b3da0 0x7f16f81f2ff0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f16e400c470 tx=0x7f16e400c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.942 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.929+0000 7f16f57fa640 1 -- 172.21.15.81:0/3435202687 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4005ce0 con 0x7f16f80b3da0 2023-11-08T16:42:18.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.930+0000 7f16f57fa640 1 -- 172.21.15.81:0/3435202687 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e4005e40 con 0x7f16f80b3da0 2023-11-08T16:42:18.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.930+0000 7f16f57fa640 1 -- 172.21.15.81:0/3435202687 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e4007290 con 0x7f16f80b3da0 2023-11-08T16:42:18.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.930+0000 7f16fdb2e640 1 -- 172.21.15.81:0/3435202687 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f8139f50 con 0x7f16f80b3da0 2023-11-08T16:42:18.943 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.930+0000 7f16fdb2e640 1 -- 172.21.15.81:0/3435202687 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f813a510 con 0x7f16f80b3da0 2023-11-08T16:42:18.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.931+0000 7f16fdb2e640 1 -- 172.21.15.81:0/3435202687 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f813ad10 con 0x7f16f80b3da0 2023-11-08T16:42:18.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.932+0000 7f16f57fa640 1 -- 172.21.15.81:0/3435202687 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f16e40073f0 con 0x7f16f80b3da0 2023-11-08T16:42:18.944 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.933+0000 7f16f57fa640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16dc06ae70 0x7f16dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:18.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.933+0000 7f16f57fa640 1 -- 172.21.15.81:0/3435202687 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f16e4088940 con 0x7f16f80b3da0 2023-11-08T16:42:18.945 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.934+0000 7f16fcb2c640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16dc06ae70 0x7f16dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:18.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.934+0000 7f16fcb2c640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16dc06ae70 0x7f16dc06d330 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f16ec000f60 tx=0x7f16ec009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:18.946 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:18.938+0000 7f16f57fa640 1 -- 172.21.15.81:0/3435202687 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f16f813ad10 con 0x7f16f80b3da0 2023-11-08T16:42:19.131 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.130+0000 7f16fdb2e640 1 -- 172.21.15.81:0/3435202687 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f16f80ce360 con 0x7f16dc06ae70 2023-11-08T16:42:19.132 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.130+0000 7f16f57fa640 1 -- 172.21.15.81:0/3435202687 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f16ec0013e0 con 0x7f16dc06ae70 2023-11-08T16:42:19.132 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 2 to deep-scrub 2023-11-08T16:42:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 -- 172.21.15.81:0/3435202687 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16dc06ae70 msgr2=0x7f16dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16dc06ae70 0x7f16dc06d330 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f16ec000f60 tx=0x7f16ec009040 comp rx=0 tx=0).stop 2023-11-08T16:42:19.135 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 -- 172.21.15.81:0/3435202687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16f80b3da0 msgr2=0x7f16f81f2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16f80b3da0 0x7f16f81f2ff0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f16e400c470 tx=0x7f16e400c930 comp rx=0 tx=0).stop 2023-11-08T16:42:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 -- 172.21.15.81:0/3435202687 shutdown_connections 2023-11-08T16:42:19.136 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16dc06ae70 0x7f16dc06d330 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f16f80cc5e0 0x7f16f815d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f16f80b3da0 0x7f16f81f2ff0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.137 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 --2- 172.21.15.81:0/3435202687 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f16f80add30 0x7f16f815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 -- 172.21.15.81:0/3435202687 >> 172.21.15.81:0/3435202687 conn(0x7f16f80b07c0 msgr2=0x7f16f806ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:19.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 -- 172.21.15.81:0/3435202687 shutdown_connections 2023-11-08T16:42:19.138 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.133+0000 7f16fdb2e640 1 -- 172.21.15.81:0/3435202687 wait complete. 2023-11-08T16:42:19.152 INFO:tasks.ceph:Scrubbing osd.3 2023-11-08T16:42:19.153 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-08T16:42:19.356 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.355+0000 7f5ee6de2640 1 Processor -- start 2023-11-08T16:42:19.357 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.355+0000 7f5ee6de2640 1 -- start start 2023-11-08T16:42:19.358 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.357+0000 7f5ee6de2640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f5ee015a5c0 0x7f5ee00cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.357+0000 7f5ee6de2640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee015aba0 con 0x7f5ee006bcb0 2023-11-08T16:42:19.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.357+0000 7f5ee6de2640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ee015ae80 con 0x7f5ee006fbe0 2023-11-08T16:42:19.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.357+0000 7f5ee6de2640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ee015b160 con 0x7f5ee015a5c0 2023-11-08T16:42:19.359 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.357+0000 7f5edffff640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f5ee006bcb0 0x7f5ee00ccdc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54414/0 (socket says 172.21.15.81:54414) 2023-11-08T16:42:19.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.357+0000 7f5edffff640 1 -- 172.21.15.81:0/2195253253 learned_addr learned my addr 172.21.15.81:0/2195253253 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:19.360 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.357+0000 7f5ee537f640 1 --2- 172.21.15.81:0/2195253253 >> v2:172.21.15.81:3300/0 conn(0x7f5ee015a5c0 0x7f5ee00cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.358+0000 7f5edf7fe640 1 -- 172.21.15.81:0/2195253253 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3415846006 0 0) 0x7f5ee015aba0 con 0x7f5ee006bcb0 2023-11-08T16:42:19.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.358+0000 7f5edf7fe640 1 -- 172.21.15.81:0/2195253253 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ed0003520 con 0x7f5ee006bcb0 2023-11-08T16:42:19.362 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.358+0000 7f5ee537f640 1 -- 172.21.15.81:0/2195253253 >> v1:172.21.15.93:6789/0 conn(0x7f5ee006fbe0 legacy=0x7f5ee006ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.358+0000 7f5ee537f640 1 -- 172.21.15.81:0/2195253253 >> v1:172.21.15.81:6789/0 conn(0x7f5ee006bcb0 legacy=0x7f5ee00ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.358+0000 7f5ee537f640 1 -- 172.21.15.81:0/2195253253 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee00cc8a0 con 0x7f5ee015a5c0 2023-11-08T16:42:19.363 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.359+0000 7f5ee537f640 1 --2- 172.21.15.81:0/2195253253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ee015a5c0 0x7f5ee00cd4d0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009d90 tx=0x7f5ed802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beee4ee723461675 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:19.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.359+0000 7f5edf7fe640 1 -- 172.21.15.81:0/2195253253 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed802cc60 con 0x7f5ee015a5c0 2023-11-08T16:42:19.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.359+0000 7f5edf7fe640 1 -- 172.21.15.81:0/2195253253 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed802cdc0 con 0x7f5ee015a5c0 2023-11-08T16:42:19.364 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.359+0000 7f5edf7fe640 1 -- 172.21.15.81:0/2195253253 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8034660 con 0x7f5ee015a5c0 2023-11-08T16:42:19.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.360+0000 7f5ee6de2640 1 -- 172.21.15.81:0/2195253253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ee015a5c0 msgr2=0x7f5ee00cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.360+0000 7f5ee6de2640 1 --2- 172.21.15.81:0/2195253253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ee015a5c0 0x7f5ee00cd4d0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8009d90 tx=0x7f5ed802beb0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.360+0000 7f5ee6de2640 1 -- 172.21.15.81:0/2195253253 shutdown_connections 2023-11-08T16:42:19.365 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.360+0000 7f5ee6de2640 1 --2- 172.21.15.81:0/2195253253 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ee015a5c0 0x7f5ee00cd4d0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.360+0000 7f5ee6de2640 1 -- 172.21.15.81:0/2195253253 >> 172.21.15.81:0/2195253253 conn(0x7f5ee00cec30 msgr2=0x7f5ee00cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:19.366 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.360+0000 7f5ee6de2640 1 -- 172.21.15.81:0/2195253253 shutdown_connections 2023-11-08T16:42:19.367 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.360+0000 7f5ee6de2640 1 -- 172.21.15.81:0/2195253253 wait complete. 2023-11-08T16:42:19.369 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.362+0000 7f5ee6de2640 1 Processor -- start 2023-11-08T16:42:19.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.362+0000 7f5ee6de2640 1 -- start start 2023-11-08T16:42:19.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.363+0000 7f5ee6de2640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ee006bcb0 0x7f5ee0145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.370 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.363+0000 7f5ee6de2640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee006fbe0 0x7f5ee014d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.363+0000 7f5ee6de2640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ee015a5c0 0x7f5ee0148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.363+0000 7f5ee6de2640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee015b5c0 con 0x7f5ee006bcb0 2023-11-08T16:42:19.371 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.363+0000 7f5ee6de2640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee014d940 con 0x7f5ee006fbe0 2023-11-08T16:42:19.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.363+0000 7f5ee6de2640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee014b6d0 con 0x7f5ee015a5c0 2023-11-08T16:42:19.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.363+0000 7f5ee4b7e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ee006bcb0 0x7f5ee0145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.363+0000 7f5ee4b7e640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ee006bcb0 0x7f5ee0145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51560/0 (socket says 172.21.15.81:51560) 2023-11-08T16:42:19.372 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.363+0000 7f5ee4b7e640 1 -- 172.21.15.81:0/3054980328 learned_addr learned my addr 172.21.15.81:0/3054980328 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:19.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5edffff640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee006fbe0 0x7f5ee014d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5ee537f640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ee015a5c0 0x7f5ee0148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.373 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5ee4b7e640 1 -- 172.21.15.81:0/3054980328 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ee015a5c0 msgr2=0x7f5ee0148010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5ee4b7e640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f5ee015a5c0 0x7f5ee0148010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5ee4b7e640 1 -- 172.21.15.81:0/3054980328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee006fbe0 msgr2=0x7f5ee014d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5ee4b7e640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ee006fbe0 0x7f5ee014d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.374 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5ee4b7e640 1 -- 172.21.15.81:0/3054980328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed8009a70 con 0x7f5ee006bcb0 2023-11-08T16:42:19.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5ee4b7e640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ee006bcb0 0x7f5ee0145c10 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400c450 tx=0x7f5ed400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:19.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5edd7fa640 1 -- 172.21.15.81:0/3054980328 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40052d0 con 0x7f5ee006bcb0 2023-11-08T16:42:19.375 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5edd7fa640 1 -- 172.21.15.81:0/3054980328 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed4005430 con 0x7f5ee006bcb0 2023-11-08T16:42:19.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5ee6de2640 1 -- 172.21.15.81:0/3054980328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ee014b970 con 0x7f5ee006bcb0 2023-11-08T16:42:19.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.364+0000 7f5edd7fa640 1 -- 172.21.15.81:0/3054980328 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed4012870 con 0x7f5ee006bcb0 2023-11-08T16:42:19.376 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.365+0000 7f5ee6de2640 1 -- 172.21.15.81:0/3054980328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee014bf30 con 0x7f5ee006bcb0 2023-11-08T16:42:19.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.365+0000 7f5ee6de2640 1 -- 172.21.15.81:0/3054980328 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ee00cc3c0 con 0x7f5ee006bcb0 2023-11-08T16:42:19.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.367+0000 7f5edd7fa640 1 -- 172.21.15.81:0/3054980328 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5ed40129d0 con 0x7f5ee006bcb0 2023-11-08T16:42:19.377 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.367+0000 7f5edd7fa640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ec406aea0 0x7f5ec406d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.368+0000 7f5edd7fa640 1 -- 172.21.15.81:0/3054980328 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f5ed4088e90 con 0x7f5ee006bcb0 2023-11-08T16:42:19.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.368+0000 7f5edffff640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ec406aea0 0x7f5ec406d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.368+0000 7f5edd7fa640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7f5ec4070e40 0x7f5ec4073260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.378 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.368+0000 7f5edd7fa640 1 -- 172.21.15.81:0/3054980328 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ec4073910 con 0x7f5ec4070e40 2023-11-08T16:42:19.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.368+0000 7f5edd7fa640 1 -- 172.21.15.81:0/3054980328 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ed408b550 con 0x7f5ee006bcb0 2023-11-08T16:42:19.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.368+0000 7f5ee537f640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7f5ec4070e40 0x7f5ec4073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.379 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.368+0000 7f5edffff640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ec406aea0 0x7f5ec406d360 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5ee01f2050 tx=0x7f5ecc009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:19.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.369+0000 7f5ee537f640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7f5ec4070e40 0x7f5ec4073260 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:19.380 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.370+0000 7f5edd7fa640 1 -- 172.21.15.81:0/3054980328 <== osd.3 v2:172.21.15.81:6816/2756229107 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f5ec4073910 con 0x7f5ec4070e40 2023-11-08T16:42:19.385 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.384+0000 7f5ee6de2640 1 -- 172.21.15.81:0/3054980328 --> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f5ee015b440 con 0x7f5ec4070e40 2023-11-08T16:42:19.386 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.384+0000 7f5edd7fa640 1 -- 172.21.15.81:0/3054980328 <== osd.3 v2:172.21.15.81:6816/2756229107 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f5ed8009450 con 0x7f5ec4070e40 2023-11-08T16:42:19.386 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:42:19.386 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T16:42:19.387 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:42:19.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.385+0000 7f5ee6de2640 1 -- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7f5ec4070e40 msgr2=0x7f5ec4073260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.385+0000 7f5ee6de2640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:6816/2756229107,v1:172.21.15.81:6817/2756229107] conn(0x7f5ec4070e40 0x7f5ec4073260 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.388 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.385+0000 7f5ee6de2640 1 -- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ec406aea0 msgr2=0x7f5ec406d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.385+0000 7f5ee6de2640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f5ec406aea0 0x7f5ec406d360 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f5ee01f2050 tx=0x7f5ecc009040 comp rx=0 tx=0).stop 2023-11-08T16:42:19.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.385+0000 7f5ee6de2640 1 -- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ee006bcb0 msgr2=0x7f5ee0145c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.389 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.385+0000 7f5ee6de2640 1 --2- 172.21.15.81:0/3054980328 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f5ee006bcb0 0x7f5ee0145c10 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f5ed400c450 tx=0x7f5ed400c910 comp rx=0 tx=0).stop 2023-11-08T16:42:19.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.385+0000 7f5ee537f640 1 -- 172.21.15.81:0/3054980328 reap_dead start 2023-11-08T16:42:19.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.385+0000 7f5ee6de2640 1 -- 172.21.15.81:0/3054980328 shutdown_connections 2023-11-08T16:42:19.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.385+0000 7f5ee6de2640 1 -- 172.21.15.81:0/3054980328 >> 172.21.15.81:0/3054980328 conn(0x7f5ee00cec30 msgr2=0x7f5ee014c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:19.390 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.385+0000 7f5ee6de2640 1 -- 172.21.15.81:0/3054980328 shutdown_connections 2023-11-08T16:42:19.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.385+0000 7f5ee6de2640 1 -- 172.21.15.81:0/3054980328 wait complete. 2023-11-08T16:42:19.403 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-08T16:42:19.613 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.612+0000 7f55b0821640 1 Processor -- start 2023-11-08T16:42:19.614 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.612+0000 7f55b0821640 1 -- start start 2023-11-08T16:42:19.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.614+0000 7f55b0821640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f55ac0b50c0 0x7f55ac0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.614+0000 7f55b0821640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f55ac0d04d0 0x7f55ac0b61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.614+0000 7f55b0821640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55ac0cdf90 con 0x7f55ac1609f0 2023-11-08T16:42:19.616 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.614+0000 7f55b0821640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55ac160dd0 con 0x7f55ac0d04d0 2023-11-08T16:42:19.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.614+0000 7f55b0821640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55ac0ce7a0 con 0x7f55ac0b50c0 2023-11-08T16:42:19.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.614+0000 7f55aad9d640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f55ac0d04d0 0x7f55ac0b61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.617 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.614+0000 7f55aa59c640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f55ac1609f0 0x7f55ac0b49b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54426/0 (socket says 172.21.15.81:54426) 2023-11-08T16:42:19.618 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.614+0000 7f55aa59c640 1 -- 172.21.15.81:0/465652079 learned_addr learned my addr 172.21.15.81:0/465652079 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:19.619 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.615+0000 7f55a3fff640 1 --2- 172.21.15.81:0/465652079 >> v2:172.21.15.93:3300/0 conn(0x7f55ac0b50c0 0x7f55ac0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.615+0000 7f55aad9d640 1 -- 172.21.15.81:0/465652079 >> v2:172.21.15.93:3300/0 conn(0x7f55ac0b50c0 msgr2=0x7f55ac0cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:19.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.615+0000 7f55aad9d640 1 --2- 172.21.15.81:0/465652079 >> v2:172.21.15.93:3300/0 conn(0x7f55ac0b50c0 0x7f55ac0cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.620 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.615+0000 7f55aad9d640 1 -- 172.21.15.81:0/465652079 >> v1:172.21.15.81:6789/0 conn(0x7f55ac1609f0 legacy=0x7f55ac0b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.615+0000 7f55aad9d640 1 -- 172.21.15.81:0/465652079 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac0ce1e0 con 0x7f55ac0d04d0 2023-11-08T16:42:19.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.616+0000 7f55aad9d640 1 --2- 172.21.15.81:0/465652079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ac0d04d0 0x7f55ac0b61d0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0d3f80 tx=0x7f55a402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24ce77004ac8e2b server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:19.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.616+0000 7f55a9d9b640 1 -- 172.21.15.81:0/465652079 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a402cbe0 con 0x7f55ac0d04d0 2023-11-08T16:42:19.621 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.616+0000 7f55a9d9b640 1 -- 172.21.15.81:0/465652079 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a402cd40 con 0x7f55ac0d04d0 2023-11-08T16:42:19.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.616+0000 7f55a9d9b640 1 -- 172.21.15.81:0/465652079 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a4034660 con 0x7f55ac0d04d0 2023-11-08T16:42:19.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.617+0000 7f55b0821640 1 -- 172.21.15.81:0/465652079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ac0d04d0 msgr2=0x7f55ac0b61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.622 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.617+0000 7f55b0821640 1 --2- 172.21.15.81:0/465652079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ac0d04d0 0x7f55ac0b61d0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0d3f80 tx=0x7f55a402be30 comp rx=0 tx=0).stop 2023-11-08T16:42:19.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.617+0000 7f55b0821640 1 -- 172.21.15.81:0/465652079 shutdown_connections 2023-11-08T16:42:19.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.617+0000 7f55b0821640 1 --2- 172.21.15.81:0/465652079 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ac0d04d0 0x7f55ac0b61d0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.617+0000 7f55b0821640 1 --2- 172.21.15.81:0/465652079 >> v2:172.21.15.93:3300/0 conn(0x7f55ac0b50c0 0x7f55ac0cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.623 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.617+0000 7f55b0821640 1 -- 172.21.15.81:0/465652079 >> 172.21.15.81:0/465652079 conn(0x7f55ac0cf9b0 msgr2=0x7f55ac0cfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:19.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.617+0000 7f55b0821640 1 -- 172.21.15.81:0/465652079 shutdown_connections 2023-11-08T16:42:19.624 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.617+0000 7f55b0821640 1 -- 172.21.15.81:0/465652079 wait complete. 2023-11-08T16:42:19.626 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.619+0000 7f55b0821640 1 Processor -- start 2023-11-08T16:42:19.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.619+0000 7f55b0821640 1 -- start start 2023-11-08T16:42:19.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.620+0000 7f55b0821640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55ac0b50c0 0x7f55ac0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.627 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.620+0000 7f55b0821640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ac0d04d0 0x7f55ac14ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.620+0000 7f55b0821640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55ac1609f0 0x7f55ac0ca3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.620+0000 7f55b0821640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac0c9950 con 0x7f55ac0d04d0 2023-11-08T16:42:19.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.620+0000 7f55b0821640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac149440 con 0x7f55ac0b50c0 2023-11-08T16:42:19.628 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.620+0000 7f55b0821640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac149720 con 0x7f55ac1609f0 2023-11-08T16:42:19.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.620+0000 7f55a3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ac0d04d0 0x7f55ac14ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.620+0000 7f55a3fff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ac0d04d0 0x7f55ac14ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51582/0 (socket says 172.21.15.81:51582) 2023-11-08T16:42:19.629 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.620+0000 7f55a3fff640 1 -- 172.21.15.81:0/3495091588 learned_addr learned my addr 172.21.15.81:0/3495091588 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55aa59c640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55ac0b50c0 0x7f55ac0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55aad9d640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55ac1609f0 0x7f55ac0ca3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55a3fff640 1 -- 172.21.15.81:0/3495091588 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55ac1609f0 msgr2=0x7f55ac0ca3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.630 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55a3fff640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55ac1609f0 0x7f55ac0ca3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55a3fff640 1 -- 172.21.15.81:0/3495091588 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55ac0b50c0 msgr2=0x7f55ac0c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55a3fff640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55ac0b50c0 0x7f55ac0c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.631 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55a3fff640 1 -- 172.21.15.81:0/3495091588 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4009a70 con 0x7f55ac0d04d0 2023-11-08T16:42:19.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55a3fff640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ac0d04d0 0x7f55ac14ba00 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f559400d5f0 tx=0x7f559400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:19.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55a2ffd640 1 -- 172.21.15.81:0/3495091588 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55940125f0 con 0x7f55ac0d04d0 2023-11-08T16:42:19.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55a2ffd640 1 -- 172.21.15.81:0/3495091588 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5594012750 con 0x7f55ac0d04d0 2023-11-08T16:42:19.632 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55b0821640 1 -- 172.21.15.81:0/3495091588 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ac149a00 con 0x7f55ac0d04d0 2023-11-08T16:42:19.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.621+0000 7f55a2ffd640 1 -- 172.21.15.81:0/3495091588 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5594012b00 con 0x7f55ac0d04d0 2023-11-08T16:42:19.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.622+0000 7f55b0821640 1 -- 172.21.15.81:0/3495091588 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ac149fc0 con 0x7f55ac0d04d0 2023-11-08T16:42:19.633 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.623+0000 7f55b0821640 1 -- 172.21.15.81:0/3495091588 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ac15caa0 con 0x7f55ac0d04d0 2023-11-08T16:42:19.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.624+0000 7f55a2ffd640 1 -- 172.21.15.81:0/3495091588 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5594012c60 con 0x7f55ac0d04d0 2023-11-08T16:42:19.634 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.625+0000 7f55a2ffd640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558c06ae70 0x7f558c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:19.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.625+0000 7f55a2ffd640 1 -- 172.21.15.81:0/3495091588 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f5594089950 con 0x7f55ac0d04d0 2023-11-08T16:42:19.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.626+0000 7f55aa59c640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558c06ae70 0x7f558c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:19.635 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.626+0000 7f55aa59c640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558c06ae70 0x7f558c06d330 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f559c005880 tx=0x7f559c009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:19.636 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.630+0000 7f55a2ffd640 1 -- 172.21.15.81:0/3495091588 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f5594054bb0 con 0x7f55ac0d04d0 2023-11-08T16:42:19.824 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:42:19.825 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.824+0000 7f55b0821640 1 -- 172.21.15.81:0/3495091588 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f55ac0d26c0 con 0x7f558c06ae70 2023-11-08T16:42:19.826 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.825+0000 7f55a2ffd640 1 -- 172.21.15.81:0/3495091588 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f559c001c50 con 0x7f558c06ae70 2023-11-08T16:42:19.827 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 3 to deep-scrub 2023-11-08T16:42:19.833 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:42:19.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.827+0000 7f55b0821640 1 -- 172.21.15.81:0/3495091588 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558c06ae70 msgr2=0x7f558c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.834 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.827+0000 7f55b0821640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558c06ae70 0x7f558c06d330 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f559c005880 tx=0x7f559c009040 comp rx=0 tx=0).stop 2023-11-08T16:42:19.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.827+0000 7f55b0821640 1 -- 172.21.15.81:0/3495091588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ac0d04d0 msgr2=0x7f55ac14ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:19.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.827+0000 7f55b0821640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ac0d04d0 0x7f55ac14ba00 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f559400d5f0 tx=0x7f559400dab0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.835 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.827+0000 7f55b0821640 1 -- 172.21.15.81:0/3495091588 shutdown_connections 2023-11-08T16:42:19.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.827+0000 7f55b0821640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f558c06ae70 0x7f558c06d330 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.827+0000 7f55b0821640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f55ac1609f0 0x7f55ac0ca3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.827+0000 7f55b0821640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f55ac0d04d0 0x7f55ac14ba00 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.828+0000 7f55b0821640 1 --2- 172.21.15.81:0/3495091588 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55ac0b50c0 0x7f55ac0c9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.828+0000 7f55b0821640 1 -- 172.21.15.81:0/3495091588 >> 172.21.15.81:0/3495091588 conn(0x7f55ac0cf9b0 msgr2=0x7f55ac0d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.828+0000 7f55b0821640 1 -- 172.21.15.81:0/3495091588 shutdown_connections 2023-11-08T16:42:19.837 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:19.828+0000 7f55b0821640 1 -- 172.21.15.81:0/3495091588 wait complete. 2023-11-08T16:42:19.847 INFO:tasks.ceph:Scrubbing osd.4 2023-11-08T16:42:19.848 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-08T16:42:20.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.075+0000 7f2b39b55640 1 Processor -- start 2023-11-08T16:42:20.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.075+0000 7f2b39b55640 1 -- start start 2023-11-08T16:42:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.077+0000 7f2b39b55640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f2b3415ad40 0x7f2b340cb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.077+0000 7f2b39b55640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2b34070f30 0x7f2b34071380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.077+0000 7f2b39b55640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b340cc2e0 con 0x7f2b340718c0 2023-11-08T16:42:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.077+0000 7f2b39b55640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b3415f320 con 0x7f2b34070f30 2023-11-08T16:42:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.077+0000 7f2b39b55640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b34071cd0 con 0x7f2b3415ad40 2023-11-08T16:42:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.077+0000 7f2b33fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2b34070f30 0x7f2b34071380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.080 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.077+0000 7f2b33fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2b34070f30 0x7f2b34071380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51584/0 (socket says 172.21.15.81:51584) 2023-11-08T16:42:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.077+0000 7f2b33fff640 1 -- 172.21.15.81:0/1268033816 learned_addr learned my addr 172.21.15.81:0/1268033816 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.078+0000 7f2b33fff640 1 -- 172.21.15.81:0/1268033816 >> v2:172.21.15.93:3300/0 conn(0x7f2b3415ad40 msgr2=0x7f2b340cb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:20.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.078+0000 7f2b38b53640 1 --2- 172.21.15.81:0/1268033816 >> v2:172.21.15.93:3300/0 conn(0x7f2b3415ad40 0x7f2b340cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.078+0000 7f2b33fff640 1 --2- 172.21.15.81:0/1268033816 >> v2:172.21.15.93:3300/0 conn(0x7f2b3415ad40 0x7f2b340cb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.078+0000 7f2b33fff640 1 -- 172.21.15.81:0/1268033816 >> v1:172.21.15.93:6789/0 conn(0x7f2b340718c0 legacy=0x7f2b340cc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.078+0000 7f2b33fff640 1 -- 172.21.15.81:0/1268033816 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b3415f7b0 con 0x7f2b34070f30 2023-11-08T16:42:20.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.078+0000 7f2b38b53640 1 --2- 172.21.15.81:0/1268033816 >> v2:172.21.15.93:3300/0 conn(0x7f2b3415ad40 0x7f2b340cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.079+0000 7f2b33fff640 1 --2- 172.21.15.81:0/1268033816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b34070f30 0x7f2b34071380 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f2b3406af30 tx=0x7f2b2002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64038aa138f8245f server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.079+0000 7f2b337fe640 1 -- 172.21.15.81:0/1268033816 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b200317e0 con 0x7f2b34070f30 2023-11-08T16:42:20.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.079+0000 7f2b337fe640 1 -- 172.21.15.81:0/1268033816 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b20031940 con 0x7f2b34070f30 2023-11-08T16:42:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.079+0000 7f2b337fe640 1 -- 172.21.15.81:0/1268033816 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20031cd0 con 0x7f2b34070f30 2023-11-08T16:42:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.080+0000 7f2b39b55640 1 -- 172.21.15.81:0/1268033816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b34070f30 msgr2=0x7f2b34071380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.080+0000 7f2b39b55640 1 --2- 172.21.15.81:0/1268033816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b34070f30 0x7f2b34071380 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f2b3406af30 tx=0x7f2b2002b880 comp rx=0 tx=0).stop 2023-11-08T16:42:20.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.080+0000 7f2b39b55640 1 -- 172.21.15.81:0/1268033816 shutdown_connections 2023-11-08T16:42:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.080+0000 7f2b39b55640 1 --2- 172.21.15.81:0/1268033816 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b34070f30 0x7f2b34071380 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.080+0000 7f2b39b55640 1 --2- 172.21.15.81:0/1268033816 >> v2:172.21.15.93:3300/0 conn(0x7f2b3415ad40 0x7f2b340cb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.088 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.080+0000 7f2b39b55640 1 -- 172.21.15.81:0/1268033816 >> 172.21.15.81:0/1268033816 conn(0x7f2b340b1660 msgr2=0x7f2b340b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.080+0000 7f2b39b55640 1 -- 172.21.15.81:0/1268033816 shutdown_connections 2023-11-08T16:42:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.080+0000 7f2b39b55640 1 -- 172.21.15.81:0/1268033816 wait complete. 2023-11-08T16:42:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.081+0000 7f2b39b55640 1 Processor -- start 2023-11-08T16:42:20.089 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.082+0000 7f2b39b55640 1 -- start start 2023-11-08T16:42:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.082+0000 7f2b39b55640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b34070f30 0x7f2b3415c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.082+0000 7f2b39b55640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b340718c0 0x7f2b341f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.082+0000 7f2b39b55640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b3415ad40 0x7f2b34159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.082+0000 7f2b39b55640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340c4980 con 0x7f2b3415ad40 2023-11-08T16:42:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.082+0000 7f2b39b55640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340c4c60 con 0x7f2b340718c0 2023-11-08T16:42:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.082+0000 7f2b39b55640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b341f28e0 con 0x7f2b34070f30 2023-11-08T16:42:20.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.083+0000 7f2b39354640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b3415ad40 0x7f2b34159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.083+0000 7f2b39354640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b3415ad40 0x7f2b34159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51596/0 (socket says 172.21.15.81:51596) 2023-11-08T16:42:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.083+0000 7f2b39354640 1 -- 172.21.15.81:0/2775471394 learned_addr learned my addr 172.21.15.81:0/2775471394 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.083+0000 7f2b39354640 1 -- 172.21.15.81:0/2775471394 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b34070f30 msgr2=0x7f2b3415c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.083+0000 7f2b39354640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2b34070f30 0x7f2b3415c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.083+0000 7f2b39354640 1 -- 172.21.15.81:0/2775471394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b340718c0 msgr2=0x7f2b341f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.083+0000 7f2b33fff640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b340718c0 0x7f2b341f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.083+0000 7f2b39354640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b340718c0 0x7f2b341f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.083+0000 7f2b39354640 1 -- 172.21.15.81:0/2775471394 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b200094d0 con 0x7f2b3415ad40 2023-11-08T16:42:20.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.084+0000 7f2b39354640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b3415ad40 0x7f2b34159290 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00c780 tx=0x7f2b2c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.084+0000 7f2b317fa640 1 -- 172.21.15.81:0/2775471394 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c004b60 con 0x7f2b3415ad40 2023-11-08T16:42:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.084+0000 7f2b317fa640 1 -- 172.21.15.81:0/2775471394 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c00ce90 con 0x7f2b3415ad40 2023-11-08T16:42:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.084+0000 7f2b39b55640 1 -- 172.21.15.81:0/2775471394 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b341f2bc0 con 0x7f2b3415ad40 2023-11-08T16:42:20.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.084+0000 7f2b317fa640 1 -- 172.21.15.81:0/2775471394 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c00a850 con 0x7f2b3415ad40 2023-11-08T16:42:20.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.084+0000 7f2b39b55640 1 -- 172.21.15.81:0/2775471394 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b340c4210 con 0x7f2b3415ad40 2023-11-08T16:42:20.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.085+0000 7f2b39b55640 1 -- 172.21.15.81:0/2775471394 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2b340ca5b0 con 0x7f2b3415ad40 2023-11-08T16:42:20.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.086+0000 7f2b317fa640 1 -- 172.21.15.81:0/2775471394 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b2c00a9b0 con 0x7f2b3415ad40 2023-11-08T16:42:20.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.087+0000 7f2b317fa640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b1806ae70 0x7f2b1806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.087+0000 7f2b317fa640 1 -- 172.21.15.81:0/2775471394 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2b2c089d80 con 0x7f2b3415ad40 2023-11-08T16:42:20.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.087+0000 7f2b38b53640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b1806ae70 0x7f2b1806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.087+0000 7f2b317fa640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f2b18070e10 0x7f2b18073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.087+0000 7f2b317fa640 1 -- 172.21.15.81:0/2775471394 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2b180738e0 con 0x7f2b18070e10 2023-11-08T16:42:20.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.087+0000 7f2b317fa640 1 -- 172.21.15.81:0/2775471394 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b2c08a0f0 con 0x7f2b3415ad40 2023-11-08T16:42:20.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.088+0000 7f2b38b53640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b1806ae70 0x7f2b1806d330 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2b28006fd0 tx=0x7f2b28008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.088+0000 7f2b33fff640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f2b18070e10 0x7f2b18073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.089+0000 7f2b33fff640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f2b18070e10 0x7f2b18073230 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.091+0000 7f2b317fa640 1 -- 172.21.15.81:0/2775471394 <== osd.4 v2:172.21.15.93:6800/3259756279 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f2b180738e0 con 0x7f2b18070e10 2023-11-08T16:42:20.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.105+0000 7f2b39b55640 1 -- 172.21.15.81:0/2775471394 --> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2b340b4a10 con 0x7f2b18070e10 2023-11-08T16:42:20.107 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.106+0000 7f2b317fa640 1 -- 172.21.15.81:0/2775471394 <== osd.4 v2:172.21.15.93:6800/3259756279 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2b340b4a10 con 0x7f2b18070e10 2023-11-08T16:42:20.107 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:42:20.108 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T16:42:20.108 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:42:20.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.106+0000 7f2b39b55640 1 -- 172.21.15.81:0/2775471394 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f2b18070e10 msgr2=0x7f2b18073230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.106+0000 7f2b39b55640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.93:6800/3259756279,v1:172.21.15.93:6801/3259756279] conn(0x7f2b18070e10 0x7f2b18073230 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.106+0000 7f2b39b55640 1 -- 172.21.15.81:0/2775471394 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b1806ae70 msgr2=0x7f2b1806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.106+0000 7f2b39b55640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2b1806ae70 0x7f2b1806d330 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2b28006fd0 tx=0x7f2b28008040 comp rx=0 tx=0).stop 2023-11-08T16:42:20.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.106+0000 7f2b39b55640 1 -- 172.21.15.81:0/2775471394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b3415ad40 msgr2=0x7f2b34159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.106+0000 7f2b39b55640 1 --2- 172.21.15.81:0/2775471394 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2b3415ad40 0x7f2b34159290 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c00c780 tx=0x7f2b2c00cc40 comp rx=0 tx=0).stop 2023-11-08T16:42:20.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.106+0000 7f2b39354640 1 -- 172.21.15.81:0/2775471394 reap_dead start 2023-11-08T16:42:20.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.106+0000 7f2b39b55640 1 -- 172.21.15.81:0/2775471394 shutdown_connections 2023-11-08T16:42:20.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.106+0000 7f2b39b55640 1 -- 172.21.15.81:0/2775471394 >> 172.21.15.81:0/2775471394 conn(0x7f2b340b1660 msgr2=0x7f2b3415a030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:20.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.107+0000 7f2b39b55640 1 -- 172.21.15.81:0/2775471394 shutdown_connections 2023-11-08T16:42:20.112 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.107+0000 7f2b39b55640 1 -- 172.21.15.81:0/2775471394 wait complete. 2023-11-08T16:42:20.123 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-08T16:42:20.314 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.313+0000 7f748944b640 1 Processor -- start 2023-11-08T16:42:20.315 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.313+0000 7f748944b640 1 -- start start 2023-11-08T16:42:20.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.315+0000 7f748944b640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f748415d220 0x7f748415e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.316 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.315+0000 7f748944b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74840ccf50 0x7f748406a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.315+0000 7f748944b640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f748406ddf0 con 0x7f748406a5b0 2023-11-08T16:42:20.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.315+0000 7f748944b640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74840ccb30 con 0x7f748415d220 2023-11-08T16:42:20.317 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.315+0000 7f748944b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f748406aa20 con 0x7f74840ccf50 2023-11-08T16:42:20.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.315+0000 7f74837fe640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f748406a5b0 0x7f7484070e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:54436/0 (socket says 172.21.15.81:54436) 2023-11-08T16:42:20.318 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.315+0000 7f74837fe640 1 -- 172.21.15.81:0/1183651250 learned_addr learned my addr 172.21.15.81:0/1183651250 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:20.322 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.315+0000 7f7482ffd640 1 --2- 172.21.15.81:0/1183651250 >> v2:172.21.15.93:3300/0 conn(0x7f748415d220 0x7f748415e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.315+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398295382 0 0) 0x7f747c001490 con 0x7f748406a5b0 2023-11-08T16:42:20.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.315+0000 7f74827fc640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74840ccf50 0x7f748406a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7470003520 con 0x7f748406a5b0 2023-11-08T16:42:20.323 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1797763277 0 0) 0x7f7470003520 con 0x7f748406a5b0 2023-11-08T16:42:20.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74840ccf50 msgr2=0x7f748406a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:20.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74840ccf50 0x7f748406a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.324 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 >> v2:172.21.15.93:3300/0 conn(0x7f748415d220 msgr2=0x7f748415e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:20.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 --2- 172.21.15.81:0/1183651250 >> v2:172.21.15.93:3300/0 conn(0x7f748415d220 0x7f748415e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748415e660 con 0x7f748406a5b0 2023-11-08T16:42:20.325 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f747c001b80 con 0x7f748406a5b0 2023-11-08T16:42:20.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 >> v1:172.21.15.81:6789/0 conn(0x7f748406a5b0 legacy=0x7f7484070e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74840ccf50 0x7f7470003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.326 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 0x7f74700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7470004d00 0x7f7470005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74840ccb30 con 0x7f7470003ef0 2023-11-08T16:42:20.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748406aa20 con 0x7f7470004d00 2023-11-08T16:42:20.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747000b980 con 0x7f74840ccf50 2023-11-08T16:42:20.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.316+0000 7f74827fc640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 0x7f74700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.317+0000 7f7482ffd640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74840ccf50 0x7f7470003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.317+0000 7f74837fe640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7470004d00 0x7f7470005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.317+0000 7f74827fc640 1 -- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74840ccf50 msgr2=0x7f7470003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:20.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.317+0000 7f74827fc640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74840ccf50 0x7f7470003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.317+0000 7f74827fc640 1 -- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7470004d00 msgr2=0x7f7470005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:20.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.317+0000 7f74827fc640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7470004d00 0x7f7470005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.317+0000 7f74827fc640 1 -- 172.21.15.81:0/1183651250 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7470003520 con 0x7f7470003ef0 2023-11-08T16:42:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.317+0000 7f74837fe640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7470004d00 0x7f7470005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.318+0000 7f74827fc640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 0x7f74700047c0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f746c009ac0 tx=0x7f74700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc1819fd546c496 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.318+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c02c6b0 con 0x7f7470003ef0 2023-11-08T16:42:20.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.318+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f746c02c810 con 0x7f7470003ef0 2023-11-08T16:42:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.318+0000 7f7481ffb640 1 -- 172.21.15.81:0/1183651250 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746c02cba0 con 0x7f7470003ef0 2023-11-08T16:42:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.319+0000 7f748944b640 1 -- 172.21.15.81:0/1183651250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 msgr2=0x7f74700047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.319+0000 7f748944b640 1 --2- 172.21.15.81:0/1183651250 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 0x7f74700047c0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f746c009ac0 tx=0x7f74700057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.319+0000 7f74837fe640 1 -- 172.21.15.81:0/1183651250 reap_dead start 2023-11-08T16:42:20.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.319+0000 7f748944b640 1 -- 172.21.15.81:0/1183651250 shutdown_connections 2023-11-08T16:42:20.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.319+0000 7f748944b640 1 -- 172.21.15.81:0/1183651250 >> 172.21.15.81:0/1183651250 conn(0x7f74840b1950 msgr2=0x7f74840b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:20.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.319+0000 7f748944b640 1 -- 172.21.15.81:0/1183651250 shutdown_connections 2023-11-08T16:42:20.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.319+0000 7f748944b640 1 -- 172.21.15.81:0/1183651250 wait complete. 2023-11-08T16:42:20.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.321+0000 7f748944b640 1 Processor -- start 2023-11-08T16:42:20.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.321+0000 7f748944b640 1 -- start start 2023-11-08T16:42:20.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f748944b640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 0x7f7484144220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f748944b640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7484144840 0x7f74840b3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f748944b640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74841f1eb0 0x7f74841f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f748944b640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74841f2870 con 0x7f7470003ef0 2023-11-08T16:42:20.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f748944b640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7484159760 con 0x7f7484144840 2023-11-08T16:42:20.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f748944b640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7484159a40 con 0x7f74841f1eb0 2023-11-08T16:42:20.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f7482ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 0x7f7484144220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f7482ffd640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 0x7f7484144220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51614/0 (socket says 172.21.15.81:51614) 2023-11-08T16:42:20.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f7482ffd640 1 -- 172.21.15.81:0/1693771886 learned_addr learned my addr 172.21.15.81:0/1693771886 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:20.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f7482ffd640 1 -- 172.21.15.81:0/1693771886 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74841f1eb0 msgr2=0x7f74841f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f7482ffd640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74841f1eb0 0x7f74841f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f7482ffd640 1 -- 172.21.15.81:0/1693771886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7484144840 msgr2=0x7f74840b3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f7482ffd640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7484144840 0x7f74840b3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.322+0000 7f7482ffd640 1 -- 172.21.15.81:0/1693771886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7484159d20 con 0x7f7470003ef0 2023-11-08T16:42:20.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.323+0000 7f7482ffd640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 0x7f7484144220 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f74740116d0 tx=0x7f7474011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.323+0000 7f747b7fe640 1 -- 172.21.15.81:0/1693771886 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747400f9c0 con 0x7f7470003ef0 2023-11-08T16:42:20.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.323+0000 7f747b7fe640 1 -- 172.21.15.81:0/1693771886 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747400fb20 con 0x7f7470003ef0 2023-11-08T16:42:20.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.323+0000 7f748944b640 1 -- 172.21.15.81:0/1693771886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748415a000 con 0x7f7470003ef0 2023-11-08T16:42:20.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.323+0000 7f747b7fe640 1 -- 172.21.15.81:0/1693771886 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747400fe90 con 0x7f7470003ef0 2023-11-08T16:42:20.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.323+0000 7f748944b640 1 -- 172.21.15.81:0/1693771886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7484159410 con 0x7f7470003ef0 2023-11-08T16:42:20.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.324+0000 7f748944b640 1 -- 172.21.15.81:0/1693771886 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7484074440 con 0x7f7470003ef0 2023-11-08T16:42:20.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.325+0000 7f747b7fe640 1 -- 172.21.15.81:0/1693771886 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7474015460 con 0x7f7470003ef0 2023-11-08T16:42:20.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.326+0000 7f747b7fe640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f746406ae70 0x7f746406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.326+0000 7f747b7fe640 1 -- 172.21.15.81:0/1693771886 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f7474089d50 con 0x7f7470003ef0 2023-11-08T16:42:20.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.326+0000 7f74827fc640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f746406ae70 0x7f746406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.327+0000 7f74827fc640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f746406ae70 0x7f746406d330 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f746c036000 tx=0x7f746c02e000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.330+0000 7f747b7fe640 1 -- 172.21.15.81:0/1693771886 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f747408d030 con 0x7f7470003ef0 2023-11-08T16:42:20.509 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.507+0000 7f748944b640 1 -- 172.21.15.81:0/1693771886 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f748406db10 con 0x7f746406ae70 2023-11-08T16:42:20.510 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.508+0000 7f747b7fe640 1 -- 172.21.15.81:0/1693771886 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f746c004020 con 0x7f746406ae70 2023-11-08T16:42:20.510 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 4 to deep-scrub 2023-11-08T16:42:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.510+0000 7f748944b640 1 -- 172.21.15.81:0/1693771886 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f746406ae70 msgr2=0x7f746406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.512 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.510+0000 7f748944b640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f746406ae70 0x7f746406d330 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f746c036000 tx=0x7f746c02e000 comp rx=0 tx=0).stop 2023-11-08T16:42:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.510+0000 7f748944b640 1 -- 172.21.15.81:0/1693771886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 msgr2=0x7f7484144220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.510+0000 7f748944b640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 0x7f7484144220 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f74740116d0 tx=0x7f7474011b90 comp rx=0 tx=0).stop 2023-11-08T16:42:20.513 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.510+0000 7f748944b640 1 -- 172.21.15.81:0/1693771886 shutdown_connections 2023-11-08T16:42:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.511+0000 7f748944b640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f746406ae70 0x7f746406d330 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.511+0000 7f748944b640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f74841f1eb0 0x7f74841f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.514 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.511+0000 7f748944b640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7484144840 0x7f74840b3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.511+0000 7f748944b640 1 --2- 172.21.15.81:0/1693771886 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f7470003ef0 0x7f7484144220 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.511+0000 7f748944b640 1 -- 172.21.15.81:0/1693771886 >> 172.21.15.81:0/1693771886 conn(0x7f74840b1950 msgr2=0x7f74840cf150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.511+0000 7f748944b640 1 -- 172.21.15.81:0/1693771886 shutdown_connections 2023-11-08T16:42:20.515 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.511+0000 7f748944b640 1 -- 172.21.15.81:0/1693771886 wait complete. 2023-11-08T16:42:20.530 INFO:tasks.ceph:Scrubbing osd.5 2023-11-08T16:42:20.531 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-08T16:42:20.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.723+0000 7fe7fbfff640 1 Processor -- start 2023-11-08T16:42:20.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.723+0000 7fe7fbfff640 1 -- start start 2023-11-08T16:42:20.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.725+0000 7fe7fbfff640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fe7fc142ee0 0x7fe7fc0b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.727 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.725+0000 7fe7fbfff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe7fc1435b0 0x7fe7fc070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.725+0000 7fe7fbfff640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7fc1432c0 con 0x7fe7fc0ce8e0 2023-11-08T16:42:20.728 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.725+0000 7fe7fbfff640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc0cc620 con 0x7fe7fc1435b0 2023-11-08T16:42:20.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.725+0000 7fe7fbfff640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7fc0cc830 con 0x7fe7fc142ee0 2023-11-08T16:42:20.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.725+0000 7fe7f3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe7fc1435b0 0x7fe7fc070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.725+0000 7fe7f3fff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe7fc1435b0 0x7fe7fc070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51622/0 (socket says 172.21.15.81:51622) 2023-11-08T16:42:20.729 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.725+0000 7fe7f3fff640 1 -- 172.21.15.81:0/1662668171 learned_addr learned my addr 172.21.15.81:0/1662668171 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:20.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.726+0000 7fe7faffd640 1 --2- 172.21.15.81:0/1662668171 >> v2:172.21.15.93:3300/0 conn(0x7fe7fc142ee0 0x7fe7fc0b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.726+0000 7fe7f3fff640 1 -- 172.21.15.81:0/1662668171 >> v2:172.21.15.93:3300/0 conn(0x7fe7fc142ee0 msgr2=0x7fe7fc0b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:20.730 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.726+0000 7fe7f3fff640 1 --2- 172.21.15.81:0/1662668171 >> v2:172.21.15.93:3300/0 conn(0x7fe7fc142ee0 0x7fe7fc0b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.726+0000 7fe7f3fff640 1 -- 172.21.15.81:0/1662668171 >> v1:172.21.15.93:6789/0 conn(0x7fe7fc0ce8e0 legacy=0x7fe7fc0cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:20.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.726+0000 7fe7f3fff640 1 -- 172.21.15.81:0/1662668171 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7fc0ccb10 con 0x7fe7fc1435b0 2023-11-08T16:42:20.731 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.726+0000 7fe7faffd640 1 --2- 172.21.15.81:0/1662668171 >> v2:172.21.15.93:3300/0 conn(0x7fe7fc142ee0 0x7fe7fc0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:20.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.727+0000 7fe7f3fff640 1 --2- 172.21.15.81:0/1662668171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7fc1435b0 0x7fe7fc070b30 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc06b520 tx=0x7fe7e402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e931fac1fea73d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.734 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.727+0000 7fe7fa7fc640 1 -- 172.21.15.81:0/1662668171 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4004230 con 0x7fe7fc1435b0 2023-11-08T16:42:20.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.727+0000 7fe7fa7fc640 1 -- 172.21.15.81:0/1662668171 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7e4004390 con 0x7fe7fc1435b0 2023-11-08T16:42:20.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.727+0000 7fe7fa7fc640 1 -- 172.21.15.81:0/1662668171 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7e4003810 con 0x7fe7fc1435b0 2023-11-08T16:42:20.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.728+0000 7fe7fbfff640 1 -- 172.21.15.81:0/1662668171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7fc1435b0 msgr2=0x7fe7fc070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.735 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.728+0000 7fe7fbfff640 1 --2- 172.21.15.81:0/1662668171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7fc1435b0 0x7fe7fc070b30 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc06b520 tx=0x7fe7e402b8f0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.728+0000 7fe7fbfff640 1 -- 172.21.15.81:0/1662668171 shutdown_connections 2023-11-08T16:42:20.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.728+0000 7fe7fbfff640 1 --2- 172.21.15.81:0/1662668171 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7fc1435b0 0x7fe7fc070b30 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.736 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.728+0000 7fe7fbfff640 1 --2- 172.21.15.81:0/1662668171 >> v2:172.21.15.93:3300/0 conn(0x7fe7fc142ee0 0x7fe7fc0b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.728+0000 7fe7fbfff640 1 -- 172.21.15.81:0/1662668171 >> 172.21.15.81:0/1662668171 conn(0x7fe7fc0cb510 msgr2=0x7fe7fc0cb920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:20.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.728+0000 7fe7fbfff640 1 -- 172.21.15.81:0/1662668171 shutdown_connections 2023-11-08T16:42:20.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.728+0000 7fe7fbfff640 1 -- 172.21.15.81:0/1662668171 wait complete. 2023-11-08T16:42:20.737 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.730+0000 7fe7fbfff640 1 Processor -- start 2023-11-08T16:42:20.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.730+0000 7fe7fbfff640 1 -- start start 2023-11-08T16:42:20.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fbfff640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7fc0ce8e0 0x7fe7fc0b4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fbfff640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7fc142ee0 0x7fe7fc13bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.738 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fbfff640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7fc1435b0 0x7fe7fc158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fbfff640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc13e0d0 con 0x7fe7fc1435b0 2023-11-08T16:42:20.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fbfff640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc13d450 con 0x7fe7fc0ce8e0 2023-11-08T16:42:20.739 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fbfff640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7fc13d730 con 0x7fe7fc142ee0 2023-11-08T16:42:20.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7fc1435b0 0x7fe7fc158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fb7fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7fc1435b0 0x7fe7fc158400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:51632/0 (socket says 172.21.15.81:51632) 2023-11-08T16:42:20.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fb7fe640 1 -- 172.21.15.81:0/3212266989 learned_addr learned my addr 172.21.15.81:0/3212266989 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:20.740 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7f3fff640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7fc142ee0 0x7fe7fc13bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7faffd640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7fc0ce8e0 0x7fe7fc0b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fb7fe640 1 -- 172.21.15.81:0/3212266989 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7fc142ee0 msgr2=0x7fe7fc13bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fb7fe640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe7fc142ee0 0x7fe7fc13bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.741 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fb7fe640 1 -- 172.21.15.81:0/3212266989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7fc0ce8e0 msgr2=0x7fe7fc0b4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fb7fe640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7fc0ce8e0 0x7fe7fc0b4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.731+0000 7fe7fb7fe640 1 -- 172.21.15.81:0/3212266989 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7e4009460 con 0x7fe7fc1435b0 2023-11-08T16:42:20.742 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.732+0000 7fe7fb7fe640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7fc1435b0 0x7fe7fc158400 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400c780 tx=0x7fe7f400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.732+0000 7fe7f37fe640 1 -- 172.21.15.81:0/3212266989 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4004b60 con 0x7fe7fc1435b0 2023-11-08T16:42:20.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.732+0000 7fe7f37fe640 1 -- 172.21.15.81:0/3212266989 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f400ce90 con 0x7fe7fc1435b0 2023-11-08T16:42:20.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.732+0000 7fe7fbfff640 1 -- 172.21.15.81:0/3212266989 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7fc13da10 con 0x7fe7fc1435b0 2023-11-08T16:42:20.743 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.732+0000 7fe7f37fe640 1 -- 172.21.15.81:0/3212266989 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f400a850 con 0x7fe7fc1435b0 2023-11-08T16:42:20.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.732+0000 7fe7fbfff640 1 -- 172.21.15.81:0/3212266989 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7fc0b45a0 con 0x7fe7fc1435b0 2023-11-08T16:42:20.744 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.733+0000 7fe7fbfff640 1 -- 172.21.15.81:0/3212266989 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7fc0cd0d0 con 0x7fe7fc1435b0 2023-11-08T16:42:20.745 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.734+0000 7fe7f37fe640 1 -- 172.21.15.81:0/3212266989 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe7f400a9b0 con 0x7fe7fc1435b0 2023-11-08T16:42:20.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.735+0000 7fe7f37fe640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe7e006ae70 0x7fe7e006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.735+0000 7fe7f37fe640 1 -- 172.21.15.81:0/3212266989 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe7f40895b0 con 0x7fe7fc1435b0 2023-11-08T16:42:20.746 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.735+0000 7fe7f37fe640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7fe7e0070e10 0x7fe7e0073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.735+0000 7fe7faffd640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe7e006ae70 0x7fe7e006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.735+0000 7fe7f37fe640 1 -- 172.21.15.81:0/3212266989 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7e00738e0 con 0x7fe7e0070e10 2023-11-08T16:42:20.747 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.735+0000 7fe7f37fe640 1 -- 172.21.15.81:0/3212266989 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7f408bb90 con 0x7fe7fc1435b0 2023-11-08T16:42:20.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.736+0000 7fe7faffd640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe7e006ae70 0x7fe7e006d330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec008850 tx=0x7fe7ec008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.736+0000 7fe7f3fff640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7fe7e0070e10 0x7fe7e0073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.737+0000 7fe7f3fff640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7fe7e0070e10 0x7fe7e0073230 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.748 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.739+0000 7fe7f37fe640 1 -- 172.21.15.81:0/3212266989 <== osd.5 v2:172.21.15.93:6802/2406579932 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fe7e00738e0 con 0x7fe7e0070e10 2023-11-08T16:42:20.754 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.753+0000 7fe7fbfff640 1 -- 172.21.15.81:0/3212266989 --> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe7fc0ccdf0 con 0x7fe7e0070e10 2023-11-08T16:42:20.755 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.754+0000 7fe7f37fe640 1 -- 172.21.15.81:0/3212266989 <== osd.5 v2:172.21.15.93:6802/2406579932 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fe7e4005990 con 0x7fe7e0070e10 2023-11-08T16:42:20.756 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:42:20.756 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T16:42:20.756 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:42:20.757 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.754+0000 7fe7fbfff640 1 -- 172.21.15.81:0/3212266989 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7fe7e0070e10 msgr2=0x7fe7e0073230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.754+0000 7fe7fbfff640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.93:6802/2406579932,v1:172.21.15.93:6804/2406579932] conn(0x7fe7e0070e10 0x7fe7e0073230 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.754+0000 7fe7fbfff640 1 -- 172.21.15.81:0/3212266989 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe7e006ae70 msgr2=0x7fe7e006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.758 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.754+0000 7fe7fbfff640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe7e006ae70 0x7fe7e006d330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec008850 tx=0x7fe7ec008040 comp rx=0 tx=0).stop 2023-11-08T16:42:20.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.754+0000 7fe7fbfff640 1 -- 172.21.15.81:0/3212266989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7fc1435b0 msgr2=0x7fe7fc158400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.754+0000 7fe7fbfff640 1 --2- 172.21.15.81:0/3212266989 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe7fc1435b0 0x7fe7fc158400 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400c780 tx=0x7fe7f400cc40 comp rx=0 tx=0).stop 2023-11-08T16:42:20.759 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.755+0000 7fe7fb7fe640 1 -- 172.21.15.81:0/3212266989 reap_dead start 2023-11-08T16:42:20.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.755+0000 7fe7fbfff640 1 -- 172.21.15.81:0/3212266989 shutdown_connections 2023-11-08T16:42:20.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.755+0000 7fe7fbfff640 1 -- 172.21.15.81:0/3212266989 >> 172.21.15.81:0/3212266989 conn(0x7fe7fc0cb510 msgr2=0x7fe7fc157b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:20.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.755+0000 7fe7fbfff640 1 -- 172.21.15.81:0/3212266989 shutdown_connections 2023-11-08T16:42:20.760 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.755+0000 7fe7fbfff640 1 -- 172.21.15.81:0/3212266989 wait complete. 2023-11-08T16:42:20.771 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-08T16:42:20.974 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.972+0000 7f38ecf6c640 1 Processor -- start 2023-11-08T16:42:20.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.973+0000 7f38ecf6c640 1 -- start start 2023-11-08T16:42:20.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.974+0000 7f38ecf6c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f38e8158aa0 0x7f38e806fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.974+0000 7f38ecf6c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e8071620 con 0x7f38e80cd3c0 2023-11-08T16:42:20.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.974+0000 7f38ecf6c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38e8158e80 con 0x7f38e80cbce0 2023-11-08T16:42:20.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.974+0000 7f38ecf6c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38e8071850 con 0x7f38e8158aa0 2023-11-08T16:42:20.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.974+0000 7f38e5d9b640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f38e80cd3c0 0x7f38e80cb5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48420/0 (socket says 172.21.15.81:48420) 2023-11-08T16:42:20.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.974+0000 7f38e5d9b640 1 -- 172.21.15.81:0/3600667936 learned_addr learned my addr 172.21.15.81:0/3600667936 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:20.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e659c640 1 --2- 172.21.15.81:0/3600667936 >> v2:172.21.15.93:3300/0 conn(0x7f38e8158aa0 0x7f38e806fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.980 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781047550 0 0) 0x7f38e8071620 con 0x7f38e80cd3c0 2023-11-08T16:42:20.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d4003520 con 0x7f38e80cd3c0 2023-11-08T16:42:20.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342298868 0 0) 0x7f38d4003520 con 0x7f38e80cd3c0 2023-11-08T16:42:20.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 >> v2:172.21.15.93:3300/0 conn(0x7f38e8158aa0 msgr2=0x7f38e806fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:20.981 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 --2- 172.21.15.81:0/3600667936 >> v2:172.21.15.93:3300/0 conn(0x7f38e8158aa0 0x7f38e806fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 >> v1:172.21.15.93:6789/0 conn(0x7f38e80cbce0 legacy=0x7f38e815ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e8071b30 con 0x7f38e80cd3c0 2023-11-08T16:42:20.982 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f38d0001670 con 0x7f38e80cd3c0 2023-11-08T16:42:20.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 >> v1:172.21.15.81:6789/0 conn(0x7f38e80cd3c0 legacy=0x7f38e80cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 --2- 172.21.15.81:0/3600667936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d4003970 0x7f38d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.983 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 --2- 172.21.15.81:0/3600667936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38d40047c0 0x7f38d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 --2- 172.21.15.81:0/3600667936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 0x7f38d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e8071850 con 0x7f38d4005180 2023-11-08T16:42:20.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d400c020 con 0x7f38d40047c0 2023-11-08T16:42:20.984 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.975+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d400c1e0 con 0x7f38d4003970 2023-11-08T16:42:20.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.976+0000 7f38e6d9d640 1 --2- 172.21.15.81:0/3600667936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 0x7f38d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.976+0000 7f38e5d9b640 1 --2- 172.21.15.81:0/3600667936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38d40047c0 0x7f38d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.985 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.976+0000 7f38e659c640 1 --2- 172.21.15.81:0/3600667936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d4003970 0x7f38d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.976+0000 7f38e6d9d640 1 -- 172.21.15.81:0/3600667936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d4003970 msgr2=0x7f38d4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:20.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.976+0000 7f38e6d9d640 1 --2- 172.21.15.81:0/3600667936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38d4003970 0x7f38d4003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.986 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.976+0000 7f38e6d9d640 1 -- 172.21.15.81:0/3600667936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38d40047c0 msgr2=0x7f38d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:20.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.976+0000 7f38e6d9d640 1 --2- 172.21.15.81:0/3600667936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38d40047c0 0x7f38d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.987 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.976+0000 7f38e6d9d640 1 -- 172.21.15.81:0/3600667936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d4003520 con 0x7f38d4005180 2023-11-08T16:42:20.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.977+0000 7f38e6d9d640 1 -- 172.21.15.81:0/3600667936 reap_dead start 2023-11-08T16:42:20.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.977+0000 7f38e6d9d640 1 --2- 172.21.15.81:0/3600667936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 0x7f38d400b980 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f38e0001720 tx=0x7f38d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccf6fb05a1eefe8 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.991 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.977+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e00052d0 con 0x7f38d4005180 2023-11-08T16:42:20.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.977+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e002e050 con 0x7f38d4005180 2023-11-08T16:42:20.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.977+0000 7f38e559a640 1 -- 172.21.15.81:0/3600667936 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e00035f0 con 0x7f38d4005180 2023-11-08T16:42:20.992 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.978+0000 7f38ecf6c640 1 -- 172.21.15.81:0/3600667936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 msgr2=0x7f38d400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.978+0000 7f38ecf6c640 1 --2- 172.21.15.81:0/3600667936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 0x7f38d400b980 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f38e0001720 tx=0x7f38d40057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.978+0000 7f38ecf6c640 1 -- 172.21.15.81:0/3600667936 shutdown_connections 2023-11-08T16:42:20.993 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.978+0000 7f38ecf6c640 1 --2- 172.21.15.81:0/3600667936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 0x7f38d400b980 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.978+0000 7f38ecf6c640 1 -- 172.21.15.81:0/3600667936 >> 172.21.15.81:0/3600667936 conn(0x7f38e80b0770 msgr2=0x7f38e80b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:20.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.978+0000 7f38ecf6c640 1 -- 172.21.15.81:0/3600667936 shutdown_connections 2023-11-08T16:42:20.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.978+0000 7f38ecf6c640 1 -- 172.21.15.81:0/3600667936 wait complete. 2023-11-08T16:42:20.994 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.980+0000 7f38ecf6c640 1 Processor -- start 2023-11-08T16:42:20.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.980+0000 7f38ecf6c640 1 -- start start 2023-11-08T16:42:20.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38ecf6c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 0x7f38e814ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38ecf6c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e81e7010 0x7f38e80c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.995 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38ecf6c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38e814e6d0 0x7f38e8140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:20.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38ecf6c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80c1f70 con 0x7f38d4005180 2023-11-08T16:42:20.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38ecf6c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e80c6490 con 0x7f38e81e7010 2023-11-08T16:42:20.996 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38ecf6c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e814f530 con 0x7f38e814e6d0 2023-11-08T16:42:20.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38e659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 0x7f38e814ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38e659c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 0x7f38e814ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38238/0 (socket says 172.21.15.81:38238) 2023-11-08T16:42:20.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38e659c640 1 -- 172.21.15.81:0/2383253240 learned_addr learned my addr 172.21.15.81:0/2383253240 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:20.997 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38e5d9b640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e81e7010 0x7f38e80c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:20.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38e659c640 1 -- 172.21.15.81:0/2383253240 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38e814e6d0 msgr2=0x7f38e8140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38e659c640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38e814e6d0 0x7f38e8140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.998 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38e659c640 1 -- 172.21.15.81:0/2383253240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e81e7010 msgr2=0x7f38e80c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:20.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38e659c640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e81e7010 0x7f38e80c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:20.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.981+0000 7f38e659c640 1 -- 172.21.15.81:0/2383253240 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e81e6250 con 0x7f38d4005180 2023-11-08T16:42:20.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.982+0000 7f38e659c640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 0x7f38e814ee70 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f38d800f350 tx=0x7f38d800f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:20.999 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.982+0000 7f38deffd640 1 -- 172.21.15.81:0/2383253240 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8017600 con 0x7f38d4005180 2023-11-08T16:42:21.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.982+0000 7f38deffd640 1 -- 172.21.15.81:0/2383253240 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8011040 con 0x7f38d4005180 2023-11-08T16:42:21.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.982+0000 7f38ecf6c640 1 -- 172.21.15.81:0/2383253240 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e81e6530 con 0x7f38d4005180 2023-11-08T16:42:21.000 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.982+0000 7f38deffd640 1 -- 172.21.15.81:0/2383253240 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d80179c0 con 0x7f38d4005180 2023-11-08T16:42:21.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.982+0000 7f38ecf6c640 1 -- 172.21.15.81:0/2383253240 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e81e6af0 con 0x7f38d4005180 2023-11-08T16:42:21.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.983+0000 7f38ecf6c640 1 -- 172.21.15.81:0/2383253240 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e8074440 con 0x7f38d4005180 2023-11-08T16:42:21.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.984+0000 7f38deffd640 1 -- 172.21.15.81:0/2383253240 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f38d8017b20 con 0x7f38d4005180 2023-11-08T16:42:21.001 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.985+0000 7f38deffd640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38c806aea0 0x7f38c806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.986+0000 7f38deffd640 1 -- 172.21.15.81:0/2383253240 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f38d8089b20 con 0x7f38d4005180 2023-11-08T16:42:21.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.986+0000 7f38e5d9b640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38c806aea0 0x7f38c806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.002 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.986+0000 7f38e5d9b640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38c806aea0 0x7f38c806d360 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f38d0001200 tx=0x7f38d0001150 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:21.003 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:20.990+0000 7f38deffd640 1 -- 172.21.15.81:0/2383253240 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f38d8054e40 con 0x7f38d4005180 2023-11-08T16:42:21.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.163+0000 7f38ecf6c640 1 -- 172.21.15.81:0/2383253240 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f38e806d840 con 0x7f38c806aea0 2023-11-08T16:42:21.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.164+0000 7f38deffd640 1 -- 172.21.15.81:0/2383253240 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f38d0001ce0 con 0x7f38c806aea0 2023-11-08T16:42:21.166 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 5 to deep-scrub 2023-11-08T16:42:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.166+0000 7f38ecf6c640 1 -- 172.21.15.81:0/2383253240 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38c806aea0 msgr2=0x7f38c806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.166+0000 7f38ecf6c640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38c806aea0 0x7f38c806d360 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f38d0001200 tx=0x7f38d0001150 comp rx=0 tx=0).stop 2023-11-08T16:42:21.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.166+0000 7f38ecf6c640 1 -- 172.21.15.81:0/2383253240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 msgr2=0x7f38e814ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.166+0000 7f38ecf6c640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 0x7f38e814ee70 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f38d800f350 tx=0x7f38d800f810 comp rx=0 tx=0).stop 2023-11-08T16:42:21.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.166+0000 7f38ecf6c640 1 -- 172.21.15.81:0/2383253240 shutdown_connections 2023-11-08T16:42:21.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.166+0000 7f38ecf6c640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f38c806aea0 0x7f38c806d360 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.166+0000 7f38ecf6c640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f38e814e6d0 0x7f38e8140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.166+0000 7f38ecf6c640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f38e81e7010 0x7f38e80c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.166+0000 7f38ecf6c640 1 --2- 172.21.15.81:0/2383253240 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f38d4005180 0x7f38e814ee70 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.166+0000 7f38ecf6c640 1 -- 172.21.15.81:0/2383253240 >> 172.21.15.81:0/2383253240 conn(0x7f38e80b0770 msgr2=0x7f38e80b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:21.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.167+0000 7f38ecf6c640 1 -- 172.21.15.81:0/2383253240 shutdown_connections 2023-11-08T16:42:21.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.167+0000 7f38ecf6c640 1 -- 172.21.15.81:0/2383253240 wait complete. 2023-11-08T16:42:21.186 INFO:tasks.ceph:Scrubbing osd.6 2023-11-08T16:42:21.186 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-08T16:42:21.391 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.390+0000 7f2fed939640 1 Processor -- start 2023-11-08T16:42:21.392 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.390+0000 7f2fed939640 1 -- start start 2023-11-08T16:42:21.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.392+0000 7f2fed939640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fe806fbe0 0x7f2fe815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.394 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.392+0000 7f2fed939640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2fe806bcb0 0x7f2fe806ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.392+0000 7f2fed939640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe8070500 con 0x7f2fe815dbb0 2023-11-08T16:42:21.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.392+0000 7f2fed939640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fe815aba0 con 0x7f2fe806bcb0 2023-11-08T16:42:21.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.392+0000 7f2fed939640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe815ae80 con 0x7f2fe806fbe0 2023-11-08T16:42:21.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.392+0000 7f2fdffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2fe806bcb0 0x7f2fe806ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.392+0000 7f2fdffff640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f2fe806bcb0 0x7f2fe806ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38242/0 (socket says 172.21.15.81:38242) 2023-11-08T16:42:21.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.392+0000 7f2fdffff640 1 -- 172.21.15.81:0/3844223755 learned_addr learned my addr 172.21.15.81:0/3844223755 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.393+0000 7f2fe6ffd640 1 --2- 172.21.15.81:0/3844223755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fe806fbe0 0x7f2fe815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.393+0000 7f2fdffff640 1 -- 172.21.15.81:0/3844223755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fe806fbe0 msgr2=0x7f2fe815a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.393+0000 7f2fdffff640 1 --2- 172.21.15.81:0/3844223755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fe806fbe0 0x7f2fe815a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.397 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.393+0000 7f2fdffff640 1 -- 172.21.15.81:0/3844223755 >> v1:172.21.15.93:6789/0 conn(0x7f2fe815dbb0 legacy=0x7f2fe815e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:21.398 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.393+0000 7f2fdffff640 1 -- 172.21.15.81:0/3844223755 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe815b160 con 0x7f2fe806bcb0 2023-11-08T16:42:21.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.394+0000 7f2fdffff640 1 --2- 172.21.15.81:0/3844223755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe806bcb0 0x7f2fe806ffc0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009870 tx=0x7f2fd002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a97ccfc9de9513 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:21.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.394+0000 7f2fe67fc640 1 -- 172.21.15.81:0/3844223755 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd00317c0 con 0x7f2fe806bcb0 2023-11-08T16:42:21.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.394+0000 7f2fe67fc640 1 -- 172.21.15.81:0/3844223755 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd0031920 con 0x7f2fe806bcb0 2023-11-08T16:42:21.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.394+0000 7f2fe67fc640 1 -- 172.21.15.81:0/3844223755 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0031cb0 con 0x7f2fe806bcb0 2023-11-08T16:42:21.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.394+0000 7f2fed939640 1 -- 172.21.15.81:0/3844223755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe806bcb0 msgr2=0x7f2fe806ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.394+0000 7f2fed939640 1 --2- 172.21.15.81:0/3844223755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe806bcb0 0x7f2fe806ffc0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009870 tx=0x7f2fd002b860 comp rx=0 tx=0).stop 2023-11-08T16:42:21.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.395+0000 7f2fed939640 1 -- 172.21.15.81:0/3844223755 shutdown_connections 2023-11-08T16:42:21.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.395+0000 7f2fed939640 1 --2- 172.21.15.81:0/3844223755 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe806bcb0 0x7f2fe806ffc0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.395+0000 7f2fed939640 1 --2- 172.21.15.81:0/3844223755 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fe806fbe0 0x7f2fe815a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.395+0000 7f2fed939640 1 -- 172.21.15.81:0/3844223755 >> 172.21.15.81:0/3844223755 conn(0x7f2fe80cec30 msgr2=0x7f2fe80cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:21.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.395+0000 7f2fed939640 1 -- 172.21.15.81:0/3844223755 shutdown_connections 2023-11-08T16:42:21.404 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.395+0000 7f2fed939640 1 -- 172.21.15.81:0/3844223755 wait complete. 2023-11-08T16:42:21.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.396+0000 7f2fed939640 1 Processor -- start 2023-11-08T16:42:21.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.396+0000 7f2fed939640 1 -- start start 2023-11-08T16:42:21.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.397+0000 7f2fed939640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2fe806bcb0 0x7f2fe814d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.405 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.397+0000 7f2fed939640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fe806fbe0 0x7f2fe8154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.397+0000 7f2fed939640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe815dbb0 0x7f2fe8158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.397+0000 7f2fed939640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe8154070 con 0x7f2fe815dbb0 2023-11-08T16:42:21.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.397+0000 7f2fed939640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe8154880 con 0x7f2fe806bcb0 2023-11-08T16:42:21.406 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.397+0000 7f2fed939640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe8154ff0 con 0x7f2fe806fbe0 2023-11-08T16:42:21.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fe77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe815dbb0 0x7f2fe8158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fe77fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe815dbb0 0x7f2fe8158540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38252/0 (socket says 172.21.15.81:38252) 2023-11-08T16:42:21.407 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fe77fe640 1 -- 172.21.15.81:0/2048864936 learned_addr learned my addr 172.21.15.81:0/2048864936 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:21.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fe6ffd640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2fe806bcb0 0x7f2fe814d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fdffff640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fe806fbe0 0x7f2fe8154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.408 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fe77fe640 1 -- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fe806fbe0 msgr2=0x7f2fe8154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fe77fe640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f2fe806fbe0 0x7f2fe8154340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fe77fe640 1 -- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2fe806bcb0 msgr2=0x7f2fe814d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fe77fe640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2fe806bcb0 0x7f2fe814d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.409 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fe77fe640 1 -- 172.21.15.81:0/2048864936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd00094d0 con 0x7f2fe815dbb0 2023-11-08T16:42:21.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fe6ffd640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2fe806bcb0 0x7f2fe814d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:21.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.398+0000 7f2fe77fe640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe815dbb0 0x7f2fe8158540 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c780 tx=0x7f2fe000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:21.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.399+0000 7f2fe4ff9640 1 -- 172.21.15.81:0/2048864936 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0004b60 con 0x7f2fe815dbb0 2023-11-08T16:42:21.410 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.399+0000 7f2fe4ff9640 1 -- 172.21.15.81:0/2048864936 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe000ce90 con 0x7f2fe815dbb0 2023-11-08T16:42:21.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.399+0000 7f2fe4ff9640 1 -- 172.21.15.81:0/2048864936 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe000a850 con 0x7f2fe815dbb0 2023-11-08T16:42:21.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.399+0000 7f2fed939640 1 -- 172.21.15.81:0/2048864936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe81552d0 con 0x7f2fe815dbb0 2023-11-08T16:42:21.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.399+0000 7f2fed939640 1 -- 172.21.15.81:0/2048864936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe8155890 con 0x7f2fe815dbb0 2023-11-08T16:42:21.411 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.399+0000 7f2fed939640 1 -- 172.21.15.81:0/2048864936 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2fe815b440 con 0x7f2fe815dbb0 2023-11-08T16:42:21.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.401+0000 7f2fe4ff9640 1 -- 172.21.15.81:0/2048864936 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fe000a9b0 con 0x7f2fe815dbb0 2023-11-08T16:42:21.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.402+0000 7f2fe4ff9640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2fc806aea0 0x7f2fc806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.413 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.402+0000 7f2fe4ff9640 1 -- 172.21.15.81:0/2048864936 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f2fe0089580 con 0x7f2fe815dbb0 2023-11-08T16:42:21.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.402+0000 7f2fe6ffd640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2fc806aea0 0x7f2fc806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.402+0000 7f2fe4ff9640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f2fc8070e40 0x7f2fc8073260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.414 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.402+0000 7f2fe4ff9640 1 -- 172.21.15.81:0/2048864936 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2fc8073910 con 0x7f2fc8070e40 2023-11-08T16:42:21.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.402+0000 7f2fe4ff9640 1 -- 172.21.15.81:0/2048864936 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fe008bb60 con 0x7f2fe815dbb0 2023-11-08T16:42:21.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.402+0000 7f2fe6ffd640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2fc806aea0 0x7f2fc806d360 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8006fd0 tx=0x7f2fd8008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:21.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.403+0000 7f2fdffff640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f2fc8070e40 0x7f2fc8073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.415 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.404+0000 7f2fdffff640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f2fc8070e40 0x7f2fc8073260 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:21.416 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.406+0000 7f2fe4ff9640 1 -- 172.21.15.81:0/2048864936 <== osd.6 v2:172.21.15.93:6816/607960134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7f2fc8073910 con 0x7f2fc8070e40 2023-11-08T16:42:21.421 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.420+0000 7f2fed939640 1 -- 172.21.15.81:0/2048864936 --> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2fe80cc8a0 con 0x7f2fc8070e40 2023-11-08T16:42:21.422 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.420+0000 7f2fe4ff9640 1 -- 172.21.15.81:0/2048864936 <== osd.6 v2:172.21.15.93:6816/607960134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2fd0031b10 con 0x7f2fc8070e40 2023-11-08T16:42:21.422 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:42:21.423 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T16:42:21.423 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:42:21.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.421+0000 7f2fed939640 1 -- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f2fc8070e40 msgr2=0x7f2fc8073260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.424 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.421+0000 7f2fed939640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.93:6816/607960134,v1:172.21.15.93:6817/607960134] conn(0x7f2fc8070e40 0x7f2fc8073260 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.421+0000 7f2fed939640 1 -- 172.21.15.81:0/2048864936 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2fc806aea0 msgr2=0x7f2fc806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.421+0000 7f2fed939640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f2fc806aea0 0x7f2fc806d360 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f2fd8006fd0 tx=0x7f2fd8008040 comp rx=0 tx=0).stop 2023-11-08T16:42:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.421+0000 7f2fed939640 1 -- 172.21.15.81:0/2048864936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe815dbb0 msgr2=0x7f2fe8158540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.425 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.421+0000 7f2fed939640 1 --2- 172.21.15.81:0/2048864936 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f2fe815dbb0 0x7f2fe8158540 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f2fe000c780 tx=0x7f2fe000cc40 comp rx=0 tx=0).stop 2023-11-08T16:42:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.421+0000 7f2fe77fe640 1 -- 172.21.15.81:0/2048864936 reap_dead start 2023-11-08T16:42:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.421+0000 7f2fed939640 1 -- 172.21.15.81:0/2048864936 shutdown_connections 2023-11-08T16:42:21.426 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.421+0000 7f2fed939640 1 -- 172.21.15.81:0/2048864936 >> 172.21.15.81:0/2048864936 conn(0x7f2fe80cec30 msgr2=0x7f2fe80aebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:21.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.422+0000 7f2fed939640 1 -- 172.21.15.81:0/2048864936 shutdown_connections 2023-11-08T16:42:21.427 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.422+0000 7f2fed939640 1 -- 172.21.15.81:0/2048864936 wait complete. 2023-11-08T16:42:21.439 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-08T16:42:21.644 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.643+0000 7f75db47c640 1 Processor -- start 2023-11-08T16:42:21.645 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.643+0000 7f75db47c640 1 -- start start 2023-11-08T16:42:21.646 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.644+0000 7f75db47c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75d4158aa0 0x7f75d406fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.644+0000 7f75db47c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f75d40cd3c0 0x7f75d40cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.644+0000 7f75db47c640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f75d40cbb10 0x7f75d406bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.647 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.644+0000 7f75db47c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d40cbfa0 con 0x7f75d40cd3c0 2023-11-08T16:42:21.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.644+0000 7f75db47c640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d4071620 con 0x7f75d40cbb10 2023-11-08T16:42:21.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.644+0000 7f75db47c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d4158e80 con 0x7f75d4158aa0 2023-11-08T16:42:21.648 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.644+0000 7f75d8a17640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f75d40cd3c0 0x7f75d40cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.645+0000 7f75d8a17640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f75d40cd3c0 0x7f75d40cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38268/0 (socket says 172.21.15.81:38268) 2023-11-08T16:42:21.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.645+0000 7f75d8a17640 1 -- 172.21.15.81:0/1329216520 learned_addr learned my addr 172.21.15.81:0/1329216520 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:21.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.645+0000 7f75d9a19640 1 --2- 172.21.15.81:0/1329216520 >> v2:172.21.15.93:3300/0 conn(0x7f75d40cbb10 0x7f75d406bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.649 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.645+0000 7f75d9218640 1 --2- 172.21.15.81:0/1329216520 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75d4158aa0 0x7f75d406fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.645+0000 7f75d8a17640 1 -- 172.21.15.81:0/1329216520 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75d4158aa0 msgr2=0x7f75d406fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:21.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.645+0000 7f75d8a17640 1 --2- 172.21.15.81:0/1329216520 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75d4158aa0 0x7f75d406fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.650 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.645+0000 7f75d8a17640 1 -- 172.21.15.81:0/1329216520 >> v2:172.21.15.93:3300/0 conn(0x7f75d40cbb10 msgr2=0x7f75d406bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:21.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.645+0000 7f75d8a17640 1 --2- 172.21.15.81:0/1329216520 >> v2:172.21.15.93:3300/0 conn(0x7f75d40cbb10 0x7f75d406bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.651 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.645+0000 7f75d8a17640 1 -- 172.21.15.81:0/1329216520 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d4071850 con 0x7f75d40cd3c0 2023-11-08T16:42:21.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.646+0000 7f75d8a17640 1 --2- 172.21.15.81:0/1329216520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75d40cd3c0 0x7f75d40cb5d0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f75d40c91c0 tx=0x7f75c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e51af6e60626cda0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:21.654 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.647+0000 7f75d3fff640 1 -- 172.21.15.81:0/1329216520 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8031870 con 0x7f75d40cd3c0 2023-11-08T16:42:21.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.647+0000 7f75d3fff640 1 -- 172.21.15.81:0/1329216520 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c80319d0 con 0x7f75d40cd3c0 2023-11-08T16:42:21.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.647+0000 7f75d3fff640 1 -- 172.21.15.81:0/1329216520 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c8031d60 con 0x7f75d40cd3c0 2023-11-08T16:42:21.655 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.648+0000 7f75db47c640 1 -- 172.21.15.81:0/1329216520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75d40cd3c0 msgr2=0x7f75d40cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.648+0000 7f75db47c640 1 --2- 172.21.15.81:0/1329216520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75d40cd3c0 0x7f75d40cb5d0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f75d40c91c0 tx=0x7f75c802b910 comp rx=0 tx=0).stop 2023-11-08T16:42:21.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.648+0000 7f75db47c640 1 -- 172.21.15.81:0/1329216520 shutdown_connections 2023-11-08T16:42:21.656 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.648+0000 7f75db47c640 1 --2- 172.21.15.81:0/1329216520 >> v2:172.21.15.93:3300/0 conn(0x7f75d40cbb10 0x7f75d406bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.648+0000 7f75db47c640 1 --2- 172.21.15.81:0/1329216520 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75d40cd3c0 0x7f75d40cb5d0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.657 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.648+0000 7f75db47c640 1 --2- 172.21.15.81:0/1329216520 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75d4158aa0 0x7f75d406fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.648+0000 7f75db47c640 1 -- 172.21.15.81:0/1329216520 >> 172.21.15.81:0/1329216520 conn(0x7f75d40b0770 msgr2=0x7f75d40b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:21.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.648+0000 7f75db47c640 1 -- 172.21.15.81:0/1329216520 shutdown_connections 2023-11-08T16:42:21.658 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.648+0000 7f75db47c640 1 -- 172.21.15.81:0/1329216520 wait complete. 2023-11-08T16:42:21.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.650+0000 7f75db47c640 1 Processor -- start 2023-11-08T16:42:21.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.650+0000 7f75db47c640 1 -- start start 2023-11-08T16:42:21.659 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75db47c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75d40cbb10 0x7f75d40c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75db47c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f75d40cd3c0 0x7f75d40b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75db47c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75d4158aa0 0x7f75d40c8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.660 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75db47c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d40ca420 con 0x7f75d4158aa0 2023-11-08T16:42:21.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75db47c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d40c9810 con 0x7f75d40cd3c0 2023-11-08T16:42:21.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75db47c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d40c9af0 con 0x7f75d40cbb10 2023-11-08T16:42:21.661 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75d9a19640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75d4158aa0 0x7f75d40c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75d9a19640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75d4158aa0 0x7f75d40c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38284/0 (socket says 172.21.15.81:38284) 2023-11-08T16:42:21.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75d9a19640 1 -- 172.21.15.81:0/2773494065 learned_addr learned my addr 172.21.15.81:0/2773494065 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:21.662 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75d8a17640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f75d40cd3c0 0x7f75d40b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75d9a19640 1 -- 172.21.15.81:0/2773494065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75d40cbb10 msgr2=0x7f75d40c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75d9218640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75d40cbb10 0x7f75d40c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.663 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75d9a19640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75d40cbb10 0x7f75d40c9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75d9a19640 1 -- 172.21.15.81:0/2773494065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f75d40cd3c0 msgr2=0x7f75d40b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75d9a19640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f75d40cd3c0 0x7f75d40b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.664 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.651+0000 7f75d9a19640 1 -- 172.21.15.81:0/2773494065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c80094d0 con 0x7f75d4158aa0 2023-11-08T16:42:21.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.652+0000 7f75d9a19640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75d4158aa0 0x7f75d40c8700 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00d5d0 tx=0x7f75cc00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:21.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.652+0000 7f75d1ffb640 1 -- 172.21.15.81:0/2773494065 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc0125f0 con 0x7f75d4158aa0 2023-11-08T16:42:21.665 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.652+0000 7f75d1ffb640 1 -- 172.21.15.81:0/2773494065 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc012770 con 0x7f75d4158aa0 2023-11-08T16:42:21.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.652+0000 7f75d1ffb640 1 -- 172.21.15.81:0/2773494065 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc012b20 con 0x7f75d4158aa0 2023-11-08T16:42:21.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.652+0000 7f75db47c640 1 -- 172.21.15.81:0/2773494065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d40c9dd0 con 0x7f75d4158aa0 2023-11-08T16:42:21.666 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.652+0000 7f75db47c640 1 -- 172.21.15.81:0/2773494065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d4154970 con 0x7f75d4158aa0 2023-11-08T16:42:21.667 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.653+0000 7f75db47c640 1 -- 172.21.15.81:0/2773494065 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d40ca590 con 0x7f75d4158aa0 2023-11-08T16:42:21.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.654+0000 7f75d1ffb640 1 -- 172.21.15.81:0/2773494065 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f75cc012c80 con 0x7f75d4158aa0 2023-11-08T16:42:21.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.655+0000 7f75d1ffb640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75c006ae70 0x7f75c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:21.668 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.655+0000 7f75d1ffb640 1 -- 172.21.15.81:0/2773494065 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f75cc089930 con 0x7f75d4158aa0 2023-11-08T16:42:21.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.655+0000 7f75d9218640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75c006ae70 0x7f75c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:21.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.656+0000 7f75d9218640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75c006ae70 0x7f75c006d330 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f75d4157640 tx=0x7f75c400b040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:21.669 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.659+0000 7f75d1ffb640 1 -- 172.21.15.81:0/2773494065 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f75cc054ba0 con 0x7f75d4158aa0 2023-11-08T16:42:21.836 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.835+0000 7f75db47c640 1 -- 172.21.15.81:0/2773494065 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f75d40c9c60 con 0x7f75c006ae70 2023-11-08T16:42:21.838 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.836+0000 7f75d1ffb640 1 -- 172.21.15.81:0/2773494065 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f75d40c9c60 con 0x7f75c006ae70 2023-11-08T16:42:21.838 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 6 to deep-scrub 2023-11-08T16:42:21.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.838+0000 7f75db47c640 1 -- 172.21.15.81:0/2773494065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75c006ae70 msgr2=0x7f75c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.840 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.838+0000 7f75db47c640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75c006ae70 0x7f75c006d330 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f75d4157640 tx=0x7f75c400b040 comp rx=0 tx=0).stop 2023-11-08T16:42:21.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.838+0000 7f75db47c640 1 -- 172.21.15.81:0/2773494065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75d4158aa0 msgr2=0x7f75d40c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:21.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.838+0000 7f75db47c640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75d4158aa0 0x7f75d40c8700 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00d5d0 tx=0x7f75cc00da90 comp rx=0 tx=0).stop 2023-11-08T16:42:21.841 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.838+0000 7f75db47c640 1 -- 172.21.15.81:0/2773494065 shutdown_connections 2023-11-08T16:42:21.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.838+0000 7f75db47c640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f75c006ae70 0x7f75c006d330 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.838+0000 7f75db47c640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f75d4158aa0 0x7f75d40c8700 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.838+0000 7f75db47c640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f75d40cd3c0 0x7f75d40b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.842 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.838+0000 7f75db47c640 1 --2- 172.21.15.81:0/2773494065 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f75d40cbb10 0x7f75d40c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:21.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.838+0000 7f75db47c640 1 -- 172.21.15.81:0/2773494065 >> 172.21.15.81:0/2773494065 conn(0x7f75d40b0770 msgr2=0x7f75d40cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:21.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.839+0000 7f75db47c640 1 -- 172.21.15.81:0/2773494065 shutdown_connections 2023-11-08T16:42:21.843 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:21.839+0000 7f75db47c640 1 -- 172.21.15.81:0/2773494065 wait complete. 2023-11-08T16:42:21.858 INFO:tasks.ceph:Scrubbing osd.7 2023-11-08T16:42:21.858 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-08T16:42:22.075 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.074+0000 7fcf0c090640 1 Processor -- start 2023-11-08T16:42:22.076 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.074+0000 7fcf0c090640 1 -- start start 2023-11-08T16:42:22.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.075+0000 7fcf0c090640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf0406fbe0 0x7fcf0415a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.077 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.075+0000 7fcf0c090640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcf0406bcb0 0x7fcf0406ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.075+0000 7fcf0c090640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf04070500 con 0x7fcf0415dbb0 2023-11-08T16:42:22.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.075+0000 7fcf0c090640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf0415aba0 con 0x7fcf0406bcb0 2023-11-08T16:42:22.078 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.075+0000 7fcf0c090640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf0415ae80 con 0x7fcf0406fbe0 2023-11-08T16:42:22.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.076+0000 7fcf0962b640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fcf0406bcb0 0x7fcf0406ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.076+0000 7fcf0a62d640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fcf0415dbb0 0x7fcf0415e040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:48430/0 (socket says 172.21.15.81:48430) 2023-11-08T16:42:22.079 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.076+0000 7fcf0a62d640 1 -- 172.21.15.81:0/2517775605 learned_addr learned my addr 172.21.15.81:0/2517775605 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:22.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.076+0000 7fcf09e2c640 1 --2- 172.21.15.81:0/2517775605 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf0406fbe0 0x7fcf0415a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.081 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.076+0000 7fcf0962b640 1 -- 172.21.15.81:0/2517775605 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf0406fbe0 msgr2=0x7fcf0415a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:22.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.076+0000 7fcf0962b640 1 --2- 172.21.15.81:0/2517775605 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf0406fbe0 0x7fcf0415a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.076+0000 7fcf0962b640 1 -- 172.21.15.81:0/2517775605 >> v1:172.21.15.81:6789/0 conn(0x7fcf0415dbb0 legacy=0x7fcf0415e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.082 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.076+0000 7fcf0962b640 1 -- 172.21.15.81:0/2517775605 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0415b160 con 0x7fcf0406bcb0 2023-11-08T16:42:22.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.076+0000 7fcf08e2a640 1 -- 172.21.15.81:0/2517775605 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3556235726 0 0) 0x7fcf04070500 con 0x7fcf0415dbb0 2023-11-08T16:42:22.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.076+0000 7fcf09e2c640 1 --2- 172.21.15.81:0/2517775605 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf0406fbe0 0x7fcf0415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:22.083 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.077+0000 7fcf0962b640 1 --2- 172.21.15.81:0/2517775605 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf0406bcb0 0x7fcf0406ffc0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fcef40097a0 tx=0x7fcef402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03f36e04b5e86f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:22.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.078+0000 7fcf08e2a640 1 -- 172.21.15.81:0/2517775605 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4031780 con 0x7fcf0406bcb0 2023-11-08T16:42:22.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.078+0000 7fcf08e2a640 1 -- 172.21.15.81:0/2517775605 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef40318e0 con 0x7fcf0406bcb0 2023-11-08T16:42:22.084 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.078+0000 7fcf08e2a640 1 -- 172.21.15.81:0/2517775605 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4031c70 con 0x7fcf0406bcb0 2023-11-08T16:42:22.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.078+0000 7fcf0c090640 1 -- 172.21.15.81:0/2517775605 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf0406bcb0 msgr2=0x7fcf0406ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.078+0000 7fcf0c090640 1 --2- 172.21.15.81:0/2517775605 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf0406bcb0 0x7fcf0406ffc0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fcef40097a0 tx=0x7fcef402b820 comp rx=0 tx=0).stop 2023-11-08T16:42:22.085 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.078+0000 7fcf0c090640 1 -- 172.21.15.81:0/2517775605 shutdown_connections 2023-11-08T16:42:22.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.078+0000 7fcf0c090640 1 --2- 172.21.15.81:0/2517775605 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf0406bcb0 0x7fcf0406ffc0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.078+0000 7fcf0c090640 1 --2- 172.21.15.81:0/2517775605 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf0406fbe0 0x7fcf0415a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.078+0000 7fcf0c090640 1 -- 172.21.15.81:0/2517775605 >> 172.21.15.81:0/2517775605 conn(0x7fcf040cec30 msgr2=0x7fcf040cf040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:22.086 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.079+0000 7fcf0c090640 1 -- 172.21.15.81:0/2517775605 shutdown_connections 2023-11-08T16:42:22.087 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.079+0000 7fcf0c090640 1 -- 172.21.15.81:0/2517775605 wait complete. 2023-11-08T16:42:22.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.081+0000 7fcf0c090640 1 Processor -- start 2023-11-08T16:42:22.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.081+0000 7fcf0c090640 1 -- start start 2023-11-08T16:42:22.090 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0c090640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf0406bcb0 0x7fcf041442f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0c090640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf0406fbe0 0x7fcf040ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0c090640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf0415dbb0 0x7fcf04149410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.091 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0c090640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf04144830 con 0x7fcf0415dbb0 2023-11-08T16:42:22.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0c090640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040ae130 con 0x7fcf0406fbe0 2023-11-08T16:42:22.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0c090640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf040adb70 con 0x7fcf0406bcb0 2023-11-08T16:42:22.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0a62d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf0415dbb0 0x7fcf04149410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.092 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0a62d640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf0415dbb0 0x7fcf04149410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38304/0 (socket says 172.21.15.81:38304) 2023-11-08T16:42:22.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0a62d640 1 -- 172.21.15.81:0/399382860 learned_addr learned my addr 172.21.15.81:0/399382860 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:22.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf09e2c640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf0406bcb0 0x7fcf041442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0a62d640 1 -- 172.21.15.81:0/399382860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf0406bcb0 msgr2=0x7fcf041442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.093 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0a62d640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fcf0406bcb0 0x7fcf041442f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0a62d640 1 -- 172.21.15.81:0/399382860 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf0406fbe0 msgr2=0x7fcf040ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0a62d640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf0406fbe0 0x7fcf040ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.094 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.082+0000 7fcf0a62d640 1 -- 172.21.15.81:0/399382860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef4009400 con 0x7fcf0415dbb0 2023-11-08T16:42:22.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.083+0000 7fcf0a62d640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf0415dbb0 0x7fcf04149410 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00c7e0 tx=0x7fcefc00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:22.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.083+0000 7fcefa7fc640 1 -- 172.21.15.81:0/399382860 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc00a640 con 0x7fcf0415dbb0 2023-11-08T16:42:22.095 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.083+0000 7fcefa7fc640 1 -- 172.21.15.81:0/399382860 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcefc00a7c0 con 0x7fcf0415dbb0 2023-11-08T16:42:22.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.083+0000 7fcf0c090640 1 -- 172.21.15.81:0/399382860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf040ade50 con 0x7fcf0415dbb0 2023-11-08T16:42:22.096 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.083+0000 7fcefa7fc640 1 -- 172.21.15.81:0/399382860 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcefc00ab70 con 0x7fcf0415dbb0 2023-11-08T16:42:22.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.083+0000 7fcf0c090640 1 -- 172.21.15.81:0/399382860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf0414a880 con 0x7fcf0415dbb0 2023-11-08T16:42:22.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.084+0000 7fcf0c090640 1 -- 172.21.15.81:0/399382860 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf040cc8a0 con 0x7fcf0415dbb0 2023-11-08T16:42:22.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.085+0000 7fcefa7fc640 1 -- 172.21.15.81:0/399382860 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcefc00acd0 con 0x7fcf0415dbb0 2023-11-08T16:42:22.097 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.086+0000 7fcefa7fc640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcee806ae70 0x7fcee806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.086+0000 7fcefa7fc640 1 -- 172.21.15.81:0/399382860 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fcefc0889f0 con 0x7fcf0415dbb0 2023-11-08T16:42:22.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.086+0000 7fcefa7fc640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fcee8070e10 0x7fcee8073230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.098 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.086+0000 7fcefa7fc640 1 -- 172.21.15.81:0/399382860 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcee80738e0 con 0x7fcee8070e10 2023-11-08T16:42:22.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.086+0000 7fcefa7fc640 1 -- 172.21.15.81:0/399382860 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_get_version_reply(handle=1 version=134) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcefc08d2a0 con 0x7fcf0415dbb0 2023-11-08T16:42:22.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.086+0000 7fcf09e2c640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcee806ae70 0x7fcee806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.087+0000 7fcf0962b640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fcee8070e10 0x7fcee8073230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.099 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.087+0000 7fcf09e2c640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcee806ae70 0x7fcee806d330 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcf0414c290 tx=0x7fcf00008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:22.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.088+0000 7fcf0962b640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fcee8070e10 0x7fcee8073230 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:22.100 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.090+0000 7fcefa7fc640 1 -- 172.21.15.81:0/399382860 <== osd.7 v2:172.21.15.93:6818/433691658 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26745 (crc 0 0 0) 0x7fcee80738e0 con 0x7fcee8070e10 2023-11-08T16:42:22.105 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.104+0000 7fcf0c090640 1 -- 172.21.15.81:0/399382860 --> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcf04151330 con 0x7fcee8070e10 2023-11-08T16:42:22.106 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.105+0000 7fcefa7fc640 1 -- 172.21.15.81:0/399382860 <== osd.7 v2:172.21.15.93:6818/433691658 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fcef4031a50 con 0x7fcee8070e10 2023-11-08T16:42:22.107 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2023-11-08T16:42:22.107 INFO:teuthology.orchestra.run.smithi081.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-08T16:42:22.108 INFO:teuthology.orchestra.run.smithi081.stdout:} 2023-11-08T16:42:22.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.106+0000 7fcf0c090640 1 -- 172.21.15.81:0/399382860 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fcee8070e10 msgr2=0x7fcee8073230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.106+0000 7fcf0c090640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.93:6818/433691658,v1:172.21.15.93:6820/433691658] conn(0x7fcee8070e10 0x7fcee8073230 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.106+0000 7fcf0c090640 1 -- 172.21.15.81:0/399382860 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcee806ae70 msgr2=0x7fcee806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.109 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.106+0000 7fcf0c090640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fcee806ae70 0x7fcee806d330 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcf0414c290 tx=0x7fcf00008040 comp rx=0 tx=0).stop 2023-11-08T16:42:22.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.106+0000 7fcf0c090640 1 -- 172.21.15.81:0/399382860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf0415dbb0 msgr2=0x7fcf04149410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.106+0000 7fcf0c090640 1 --2- 172.21.15.81:0/399382860 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fcf0415dbb0 0x7fcf04149410 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fcefc00c7e0 tx=0x7fcefc00cca0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.110 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.106+0000 7fcf0a62d640 1 -- 172.21.15.81:0/399382860 reap_dead start 2023-11-08T16:42:22.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.106+0000 7fcf0c090640 1 -- 172.21.15.81:0/399382860 shutdown_connections 2023-11-08T16:42:22.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.106+0000 7fcf0c090640 1 -- 172.21.15.81:0/399382860 >> 172.21.15.81:0/399382860 conn(0x7fcf040cec30 msgr2=0x7fcf0415e000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:22.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.106+0000 7fcf0c090640 1 -- 172.21.15.81:0/399382860 shutdown_connections 2023-11-08T16:42:22.111 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.106+0000 7fcf0c090640 1 -- 172.21.15.81:0/399382860 wait complete. 2023-11-08T16:42:22.124 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-08T16:42:22.327 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.325+0000 7f1714426640 1 Processor -- start 2023-11-08T16:42:22.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.325+0000 7f1714426640 1 -- start start 2023-11-08T16:42:22.328 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.327+0000 7f1714426640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f170c0cd3c0 0x7f170c0cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.327+0000 7f1714426640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f170c0cbb10 0x7f170c15ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.327+0000 7f1714426640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f170c071620 con 0x7f170c158aa0 2023-11-08T16:42:22.329 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.327+0000 7f1714426640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f170c0cbfa0 con 0x7f170c0cd3c0 2023-11-08T16:42:22.330 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.327+0000 7f1714426640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c158e80 con 0x7f170c0cbb10 2023-11-08T16:42:22.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.328+0000 7f17119c1640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f170c0cd3c0 0x7f170c0cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.331 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.328+0000 7f17129c3640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f170c0cbb10 0x7f170c15ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.328+0000 7f17119c1640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7f170c0cd3c0 0x7f170c0cb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.81:35716/0 (socket says 172.21.15.81:35716) 2023-11-08T16:42:22.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.328+0000 7f17119c1640 1 -- 172.21.15.81:0/2917560829 learned_addr learned my addr 172.21.15.81:0/2917560829 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:22.332 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.328+0000 7f17119c1640 1 -- 172.21.15.81:0/2917560829 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f170c0cbb10 msgr2=0x7f170c15ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:22.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.328+0000 7f17119c1640 1 --2- 172.21.15.81:0/2917560829 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f170c0cbb10 0x7f170c15ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.328+0000 7f17119c1640 1 -- 172.21.15.81:0/2917560829 >> v1:172.21.15.93:6789/0 conn(0x7f170c158aa0 legacy=0x7f170c06fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.333 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.328+0000 7f17119c1640 1 -- 172.21.15.81:0/2917560829 --> v2:172.21.15.93:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c071850 con 0x7f170c0cd3c0 2023-11-08T16:42:22.334 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.330+0000 7f17119c1640 1 --2- 172.21.15.81:0/2917560829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f170c0cd3c0 0x7f170c0cb5d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009870 tx=0x7f16fc02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d27ca78ca1edeb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:22.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.331+0000 7f17111c0640 1 -- 172.21.15.81:0/2917560829 <== mon.1 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc030840 con 0x7f170c0cd3c0 2023-11-08T16:42:22.337 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.331+0000 7f17111c0640 1 -- 172.21.15.81:0/2917560829 <== mon.1 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16fc0309a0 con 0x7f170c0cd3c0 2023-11-08T16:42:22.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.331+0000 7f17111c0640 1 -- 172.21.15.81:0/2917560829 <== mon.1 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16fc030d30 con 0x7f170c0cd3c0 2023-11-08T16:42:22.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.331+0000 7f1714426640 1 -- 172.21.15.81:0/2917560829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f170c0cd3c0 msgr2=0x7f170c0cb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.338 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.331+0000 7f1714426640 1 --2- 172.21.15.81:0/2917560829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f170c0cd3c0 0x7f170c0cb5d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f16fc009870 tx=0x7f16fc02b980 comp rx=0 tx=0).stop 2023-11-08T16:42:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.331+0000 7f1714426640 1 -- 172.21.15.81:0/2917560829 shutdown_connections 2023-11-08T16:42:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.331+0000 7f1714426640 1 --2- 172.21.15.81:0/2917560829 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f170c0cbb10 0x7f170c15ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.331+0000 7f1714426640 1 --2- 172.21.15.81:0/2917560829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f170c0cd3c0 0x7f170c0cb5d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.339 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.331+0000 7f1714426640 1 -- 172.21.15.81:0/2917560829 >> 172.21.15.81:0/2917560829 conn(0x7f170c0b0770 msgr2=0x7f170c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:22.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.331+0000 7f1714426640 1 -- 172.21.15.81:0/2917560829 shutdown_connections 2023-11-08T16:42:22.340 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.331+0000 7f1714426640 1 -- 172.21.15.81:0/2917560829 wait complete. 2023-11-08T16:42:22.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.333+0000 7f1714426640 1 Processor -- start 2023-11-08T16:42:22.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.333+0000 7f1714426640 1 -- start start 2023-11-08T16:42:22.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.334+0000 7f1714426640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f170c0cbb10 0x7f170c150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.341 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.334+0000 7f1714426640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f170c0cd3c0 0x7f170c156cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.334+0000 7f1714426640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f170c158aa0 0x7f170c157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.334+0000 7f1714426640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0c3770 con 0x7f170c158aa0 2023-11-08T16:42:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.334+0000 7f1714426640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0cab60 con 0x7f170c0cbb10 2023-11-08T16:42:22.342 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.334+0000 7f1714426640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c0cae40 con 0x7f170c0cd3c0 2023-11-08T16:42:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.334+0000 7f17129c3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f170c158aa0 0x7f170c157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.334+0000 7f17129c3640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f170c158aa0 0x7f170c157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38308/0 (socket says 172.21.15.81:38308) 2023-11-08T16:42:22.343 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.334+0000 7f17129c3640 1 -- 172.21.15.81:0/376476791 learned_addr learned my addr 172.21.15.81:0/376476791 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:22.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f17121c2640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f170c0cbb10 0x7f170c150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f17119c1640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f170c0cd3c0 0x7f170c156cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.344 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f17129c3640 1 -- 172.21.15.81:0/376476791 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f170c0cd3c0 msgr2=0x7f170c156cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f17129c3640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f170c0cd3c0 0x7f170c156cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f17129c3640 1 -- 172.21.15.81:0/376476791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f170c0cbb10 msgr2=0x7f170c150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f17129c3640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f170c0cbb10 0x7f170c150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.345 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f17129c3640 1 -- 172.21.15.81:0/376476791 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc0094d0 con 0x7f170c158aa0 2023-11-08T16:42:22.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f17121c2640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f170c0cbb10 0x7f170c150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:22.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f17129c3640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f170c158aa0 0x7f170c157230 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f170400efc0 tx=0x7f170400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:22.346 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f1702ffd640 1 -- 172.21.15.81:0/376476791 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1704009ba0 con 0x7f170c158aa0 2023-11-08T16:42:22.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f1702ffd640 1 -- 172.21.15.81:0/376476791 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f170400f040 con 0x7f170c158aa0 2023-11-08T16:42:22.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f1702ffd640 1 -- 172.21.15.81:0/376476791 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17040146c0 con 0x7f170c158aa0 2023-11-08T16:42:22.347 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f1714426640 1 -- 172.21.15.81:0/376476791 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f170c0cb120 con 0x7f170c158aa0 2023-11-08T16:42:22.348 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.335+0000 7f1714426640 1 -- 172.21.15.81:0/376476791 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f170c1506d0 con 0x7f170c158aa0 2023-11-08T16:42:22.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.337+0000 7f1714426640 1 -- 172.21.15.81:0/376476791 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f170c0c3c50 con 0x7f170c158aa0 2023-11-08T16:42:22.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.337+0000 7f1702ffd640 1 -- 172.21.15.81:0/376476791 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1704014820 con 0x7f170c158aa0 2023-11-08T16:42:22.349 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.338+0000 7f1702ffd640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16f006aea0 0x7f16f006d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.338+0000 7f1702ffd640 1 -- 172.21.15.81:0/376476791 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f1704089f90 con 0x7f170c158aa0 2023-11-08T16:42:22.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.339+0000 7f17121c2640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16f006aea0 0x7f16f006d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.350 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.339+0000 7f17121c2640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16f006aea0 0x7f16f006d360 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f170c158620 tx=0x7f1708009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:22.351 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.343+0000 7f1702ffd640 1 -- 172.21.15.81:0/376476791 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f170c0c3c50 con 0x7f170c158aa0 2023-11-08T16:42:22.530 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.529+0000 7f1714426640 1 -- 172.21.15.81:0/376476791 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f170c071e10 con 0x7f16f006aea0 2023-11-08T16:42:22.531 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.529+0000 7f1702ffd640 1 -- 172.21.15.81:0/376476791 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1708001cd0 con 0x7f16f006aea0 2023-11-08T16:42:22.531 INFO:teuthology.orchestra.run.smithi081.stderr:instructed osd(s) 7 to deep-scrub 2023-11-08T16:42:22.533 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 -- 172.21.15.81:0/376476791 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16f006aea0 msgr2=0x7f16f006d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16f006aea0 0x7f16f006d360 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f170c158620 tx=0x7f1708009040 comp rx=0 tx=0).stop 2023-11-08T16:42:22.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 -- 172.21.15.81:0/376476791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f170c158aa0 msgr2=0x7f170c157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.534 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f170c158aa0 0x7f170c157230 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f170400efc0 tx=0x7f170400c490 comp rx=0 tx=0).stop 2023-11-08T16:42:22.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 -- 172.21.15.81:0/376476791 shutdown_connections 2023-11-08T16:42:22.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f16f006aea0 0x7f16f006d360 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f170c158aa0 0x7f170c157230 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.535 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f170c0cd3c0 0x7f170c156cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 --2- 172.21.15.81:0/376476791 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f170c0cbb10 0x7f170c150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 -- 172.21.15.81:0/376476791 >> 172.21.15.81:0/376476791 conn(0x7f170c0b0770 msgr2=0x7f170c0b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:22.536 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 -- 172.21.15.81:0/376476791 shutdown_connections 2023-11-08T16:42:22.537 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.532+0000 7f1714426640 1 -- 172.21.15.81:0/376476791 wait complete. 2023-11-08T16:42:22.551 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:42:22.763 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.761+0000 7f9f981d0640 1 Processor -- start 2023-11-08T16:42:22.764 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.761+0000 7f9f981d0640 1 -- start start 2023-11-08T16:42:22.766 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.763+0000 7f9f981d0640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9f900b3170 0x7f9f9015b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.763+0000 7f9f981d0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f900b3550 0x7f9f900b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.763+0000 7f9f981d0640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f900b2310 con 0x7f9f9006c9f0 2023-11-08T16:42:22.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.763+0000 7f9f981d0640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f900b2bb0 con 0x7f9f900b3170 2023-11-08T16:42:22.767 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.763+0000 7f9f981d0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900b25f0 con 0x7f9f900b3550 2023-11-08T16:42:22.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.763+0000 7f9f95f6c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9f900b3170 0x7f9f9015b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.763+0000 7f9f95f6c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7f9f900b3170 0x7f9f9015b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38318/0 (socket says 172.21.15.81:38318) 2023-11-08T16:42:22.768 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.763+0000 7f9f95f6c640 1 -- 172.21.15.81:0/3406962329 learned_addr learned my addr 172.21.15.81:0/3406962329 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:22.769 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.764+0000 7f9f95f6c640 1 -- 172.21.15.81:0/3406962329 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f900b3550 msgr2=0x7f9f900b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:22.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.764+0000 7f9f9576b640 1 --2- 172.21.15.81:0/3406962329 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f900b3550 0x7f9f900b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.764+0000 7f9f95f6c640 1 --2- 172.21.15.81:0/3406962329 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f900b3550 0x7f9f900b1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.770 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.764+0000 7f9f95f6c640 1 -- 172.21.15.81:0/3406962329 >> v1:172.21.15.93:6789/0 conn(0x7f9f9006c9f0 legacy=0x7f9f90144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:22.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.764+0000 7f9f95f6c640 1 -- 172.21.15.81:0/3406962329 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f900b28d0 con 0x7f9f900b3170 2023-11-08T16:42:22.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.764+0000 7f9f9576b640 1 --2- 172.21.15.81:0/3406962329 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f900b3550 0x7f9f900b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:22.771 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.765+0000 7f9f95f6c640 1 --2- 172.21.15.81:0/3406962329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f900b3170 0x7f9f9015b500 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009870 tx=0x7f9f8c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=574f9873f806cdb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:22.775 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.765+0000 7f9f94f6a640 1 -- 172.21.15.81:0/3406962329 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c031640 con 0x7f9f900b3170 2023-11-08T16:42:22.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.765+0000 7f9f94f6a640 1 -- 172.21.15.81:0/3406962329 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c0317a0 con 0x7f9f900b3170 2023-11-08T16:42:22.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.765+0000 7f9f94f6a640 1 -- 172.21.15.81:0/3406962329 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c031b30 con 0x7f9f900b3170 2023-11-08T16:42:22.776 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.766+0000 7f9f981d0640 1 -- 172.21.15.81:0/3406962329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f900b3170 msgr2=0x7f9f9015b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.766+0000 7f9f981d0640 1 --2- 172.21.15.81:0/3406962329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f900b3170 0x7f9f9015b500 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009870 tx=0x7f9f8c02b8f0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.766+0000 7f9f981d0640 1 -- 172.21.15.81:0/3406962329 shutdown_connections 2023-11-08T16:42:22.777 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.766+0000 7f9f981d0640 1 --2- 172.21.15.81:0/3406962329 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f900b3550 0x7f9f900b1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.766+0000 7f9f981d0640 1 --2- 172.21.15.81:0/3406962329 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f900b3170 0x7f9f9015b500 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.766+0000 7f9f981d0640 1 -- 172.21.15.81:0/3406962329 >> 172.21.15.81:0/3406962329 conn(0x7f9f9006a6a0 msgr2=0x7f9f900b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:22.778 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.766+0000 7f9f981d0640 1 -- 172.21.15.81:0/3406962329 shutdown_connections 2023-11-08T16:42:22.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.766+0000 7f9f981d0640 1 -- 172.21.15.81:0/3406962329 wait complete. 2023-11-08T16:42:22.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.768+0000 7f9f981d0640 1 Processor -- start 2023-11-08T16:42:22.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.768+0000 7f9f981d0640 1 -- start start 2023-11-08T16:42:22.779 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.769+0000 7f9f981d0640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f9006c9f0 0x7f9f90157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.769+0000 7f9f981d0640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f900b3170 0x7f9f90070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.780 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.769+0000 7f9f981d0640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f900b3550 0x7f9f9013ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.769+0000 7f9f981d0640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9013e990 con 0x7f9f9006c9f0 2023-11-08T16:42:22.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.769+0000 7f9f981d0640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9013dae0 con 0x7f9f900b3170 2023-11-08T16:42:22.781 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.769+0000 7f9f981d0640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f901580f0 con 0x7f9f900b3550 2023-11-08T16:42:22.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.769+0000 7f9f95f6c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f9006c9f0 0x7f9f90157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.782 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.769+0000 7f9f95f6c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f9006c9f0 0x7f9f90157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:38328/0 (socket says 172.21.15.81:38328) 2023-11-08T16:42:22.783 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.769+0000 7f9f95f6c640 1 -- 172.21.15.81:0/4244084740 learned_addr learned my addr 172.21.15.81:0/4244084740 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:22.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.770+0000 7f9f95f6c640 1 -- 172.21.15.81:0/4244084740 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f900b3550 msgr2=0x7f9f9013ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.784 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.770+0000 7f9f95f6c640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f900b3550 0x7f9f9013ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.770+0000 7f9f95f6c640 1 -- 172.21.15.81:0/4244084740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f900b3170 msgr2=0x7f9f90070640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.770+0000 7f9f9576b640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f900b3170 0x7f9f90070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.785 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.770+0000 7f9f95f6c640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f900b3170 0x7f9f90070640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.770+0000 7f9f95f6c640 1 -- 172.21.15.81:0/4244084740 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f8c0094d0 con 0x7f9f9006c9f0 2023-11-08T16:42:22.786 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.770+0000 7f9f95f6c640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f9006c9f0 0x7f9f90157bb0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009870 tx=0x7f9f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:22.787 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.771+0000 7f9f867fc640 1 -- 172.21.15.81:0/4244084740 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c03ac60 con 0x7f9f9006c9f0 2023-11-08T16:42:22.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.771+0000 7f9f867fc640 1 -- 172.21.15.81:0/4244084740 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8c03ade0 con 0x7f9f9006c9f0 2023-11-08T16:42:22.788 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.771+0000 7f9f867fc640 1 -- 172.21.15.81:0/4244084740 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c044630 con 0x7f9f9006c9f0 2023-11-08T16:42:22.789 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.771+0000 7f9f981d0640 1 -- 172.21.15.81:0/4244084740 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9013dd80 con 0x7f9f9006c9f0 2023-11-08T16:42:22.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.771+0000 7f9f981d0640 1 -- 172.21.15.81:0/4244084740 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9013e340 con 0x7f9f9006c9f0 2023-11-08T16:42:22.790 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.772+0000 7f9f981d0640 1 -- 172.21.15.81:0/4244084740 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f90074440 con 0x7f9f9006c9f0 2023-11-08T16:42:22.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.773+0000 7f9f867fc640 1 -- 172.21.15.81:0/4244084740 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9f8c02f050 con 0x7f9f9006c9f0 2023-11-08T16:42:22.791 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.774+0000 7f9f867fc640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f7406ae70 0x7f9f7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:22.792 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.774+0000 7f9f867fc640 1 -- 172.21.15.81:0/4244084740 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f9f8c0ab550 con 0x7f9f9006c9f0 2023-11-08T16:42:22.793 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.774+0000 7f9f9576b640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f7406ae70 0x7f9f7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:22.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.775+0000 7f9f9576b640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f7406ae70 0x7f9f7406d330 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9f90071d90 tx=0x7f9f80008040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:22.794 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.778+0000 7f9f867fc640 1 -- 172.21.15.81:0/4244084740 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f9f8c076840 con 0x7f9f9006c9f0 2023-11-08T16:42:22.967 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.965+0000 7f9f981d0640 1 -- 172.21.15.81:0/4244084740 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9f900b2020 con 0x7f9f7406ae70 2023-11-08T16:42:22.973 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.971+0000 7f9f867fc640 1 -- 172.21.15.81:0/4244084740 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48323 (secure 0 0 0) 0x7f9f8000c040 con 0x7f9f7406ae70 2023-11-08T16:42:22.974 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:22.975 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:42:22.975 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 -- 172.21.15.81:0/4244084740 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f7406ae70 msgr2=0x7f9f7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f7406ae70 0x7f9f7406d330 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9f90071d90 tx=0x7f9f80008040 comp rx=0 tx=0).stop 2023-11-08T16:42:22.976 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 -- 172.21.15.81:0/4244084740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f9006c9f0 msgr2=0x7f9f90157bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:22.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f9006c9f0 0x7f9f90157bb0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c009870 tx=0x7f9f8c0046c0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 -- 172.21.15.81:0/4244084740 shutdown_connections 2023-11-08T16:42:22.977 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f9f7406ae70 0x7f9f7406d330 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f9f900b3550 0x7f9f9013ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9f900b3170 0x7f9f90070640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 --2- 172.21.15.81:0/4244084740 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f9f9006c9f0 0x7f9f90157bb0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:22.978 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 -- 172.21.15.81:0/4244084740 >> 172.21.15.81:0/4244084740 conn(0x7f9f9006a6a0 msgr2=0x7f9f90143780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:22.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 -- 172.21.15.81:0/4244084740 shutdown_connections 2023-11-08T16:42:22.979 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:22.974+0000 7f9f981d0640 1 -- 172.21.15.81:0/4244084740 wait complete. 2023-11-08T16:42:22.994 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":533,"stamp":"2023-11-08T16:42:22.832087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263668,"kb_used_data":14216,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711052,"statfs":{"total":773094113280,"available":772824117248,"internally_reserved":0,"allocated":14557184,"data_stored":7389200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.693901"},"pg_stats":[{"pgid":"1.0","version":"17'117","reported_seq":284,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-11-08T16:42:11.143000+0000","last_change":"2023-11-08T16:28:54.815276+0000","last_active":"2023-11-08T16:42:11.143000+0000","last_peered":"2023-11-08T16:42:11.143000+0000","last_clean":"2023-11-08T16:42:11.143000+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:42:11.143000+0000","last_undegraded":"2023-11-08T16:42:11.143000+0000","last_fullsized":"2023-11-08T16:42:11.143000+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-08T16:28:52.849628+0000","last_clean_scrub_stamp":"2023-11-08T16:28:52.849628+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T03:54:25.199104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2265088,"data_stored":1366426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":1.0329999999999999,"15min":1.071},"min":{"1min":0.80200000000000005,"5min":0.56399999999999995,"15min":0.49299999999999999},"max":{"1min":1.5649999999999999,"5min":1.6879999999999999,"15min":3.1899999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.143,"5min":1.0900000000000001,"15min":1.1060000000000001},"min":{"1min":0.746,"5min":0.58299999999999996,"15min":0.53600000000000003},"max":{"1min":1.5660000000000001,"5min":1.6040000000000001,"15min":4.2729999999999997},"last":1.0680000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.103,"15min":1.111},"min":{"1min":0.72699999999999998,"5min":0.71499999999999997,"15min":0.57899999999999996},"max":{"1min":1.5629999999999999,"5min":1.891,"15min":1.891},"last":1.411},{"interface":"front","average":{"1min":1.23,"5min":1.1719999999999999,"15min":1.204},"min":{"1min":0.69899999999999995,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":3.3740000000000001},"last":0.90700000000000003}]},{"osd":2,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1419999999999999,"15min":1.206},"min":{"1min":0.81799999999999995,"5min":0.70999999999999996,"15min":0.58399999999999996},"max":{"1min":1.591,"5min":1.8109999999999999,"15min":4.2249999999999996},"last":0.76900000000000002},{"interface":"front","average":{"1min":1.228,"5min":1.252,"15min":1.224},"min":{"1min":0.67700000000000005,"5min":0.66000000000000003,"15min":0.56299999999999994},"max":{"1min":1.587,"5min":1.7010000000000001,"15min":3.4750000000000001},"last":1.3240000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.1819999999999999,"15min":1.175},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.54900000000000004},"max":{"1min":1.71,"5min":1.71,"15min":2.964},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.365,"15min":1.403},"min":{"1min":1.0029999999999999,"5min":0.86899999999999999,"15min":0.58599999999999997},"max":{"1min":1.619,"5min":1.7490000000000001,"15min":4.181},"last":1.1990000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.234,"15min":1.2350000000000001},"min":{"1min":0.93899999999999995,"5min":0.80400000000000005,"15min":0.77200000000000002},"max":{"1min":1.778,"5min":1.7789999999999999,"15min":3.6669999999999998},"last":1.371},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.325,"15min":1.337},"min":{"1min":0.95599999999999996,"5min":0.872,"15min":0.59099999999999997},"max":{"1min":1.7470000000000001,"5min":2.0179999999999998,"15min":2.3730000000000002},"last":1.5369999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.415,"15min":1.3879999999999999},"min":{"1min":1.0569999999999999,"5min":0.89200000000000002,"15min":0.61799999999999999},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":3.1059999999999999},"last":1.266},{"interface":"front","average":{"1min":1.4490000000000001,"5min":1.4279999999999999,"15min":1.4530000000000001},"min":{"1min":1.1000000000000001,"5min":1.0589999999999999,"15min":0.878},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":2.9750000000000001},"last":1.639}]},{"osd":6,"last update":"Wed Nov 8 16:41:21 2023","interfaces":[{"interface":"back","average":{"1min":1.5649999999999999,"5min":1.528,"15min":1.5329999999999999},"min":{"1min":1.083,"5min":1.0800000000000001,"15min":0.94799999999999995},"max":{"1min":1.9510000000000001,"5min":1.9510000000000001,"15min":3.7530000000000001},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.5389999999999999,"5min":1.4770000000000001,"15min":1.486},"min":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.73999999999999999},"max":{"1min":1.831,"5min":2.0430000000000001,"15min":6.9560000000000004},"last":1.5900000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0940000000000001,"15min":1.0669999999999999},"min":{"1min":0.69499999999999995,"5min":0.64500000000000002,"15min":0.497},"max":{"1min":1.3480000000000001,"5min":1.7230000000000001,"15min":2.5609999999999999},"last":1.2929999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.113,"15min":1.0880000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.56200000000000006},"max":{"1min":1.45,"5min":2.0489999999999999,"15min":2.7519999999999998},"last":0.72499999999999998}]},{"osd":1,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.1359999999999999,"15min":1.0940000000000001},"min":{"1min":0.70299999999999996,"5min":0.63400000000000001,"15min":0.60899999999999999},"max":{"1min":1.6339999999999999,"5min":1.667,"15min":1.667},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.139},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.677,"5min":1.7290000000000001,"15min":1.97},"last":1.373}]},{"osd":2,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.181,"15min":1.1759999999999999},"min":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.56799999999999995},"max":{"1min":1.5549999999999999,"5min":2.262,"15min":2.262},"last":1.022},{"interface":"front","average":{"1min":1.085,"5min":1.1879999999999999,"15min":1.1639999999999999},"min":{"1min":0.752,"5min":0.71399999999999997,"15min":0.60499999999999998},"max":{"1min":1.516,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.80800000000000005}]},{"osd":3,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.218,"15min":1.2310000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.58299999999999996},"max":{"1min":1.756,"5min":1.756,"15min":2.1120000000000001},"last":1.145},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2789999999999999,"15min":1.28},"min":{"1min":0.71499999999999997,"5min":0.70899999999999996,"15min":0.67300000000000004},"max":{"1min":1.5600000000000001,"5min":1.7809999999999999,"15min":1.95},"last":0.90100000000000002}]},{"osd":4,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3120000000000001,"15min":1.3240000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7749999999999999,"5min":2.238,"15min":2.238},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.335,"15min":1.373},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6970000000000001,"5min":1.9590000000000001,"15min":2.7080000000000002},"last":1.202}]},{"osd":5,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3049999999999999,"15min":1.3200000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.637,"5min":1.994,"15min":2.105},"last":1.095},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.5049999999999999,"15min":1.5229999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.78500000000000003},"max":{"1min":1.9259999999999999,"5min":2.3490000000000002,"15min":2.4700000000000002},"last":1.51}]},{"osd":7,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.5009999999999999,"15min":1.536},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.74299999999999999},"max":{"1min":1.8180000000000001,"5min":2.6539999999999999,"15min":3.8959999999999999},"last":1.337},{"interface":"front","average":{"1min":1.298,"5min":1.4390000000000001,"15min":1.48},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.615,"5min":2.0870000000000002,"15min":2.4889999999999999},"last":1.2509999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0900000000000001,"15min":1.071},"min":{"1min":0.64300000000000002,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.571,"5min":1.657,"15min":2.4609999999999999},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.103},"min":{"1min":0.752,"5min":0.63700000000000001,"15min":0.57999999999999996},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":2.3639999999999999},"last":0.72899999999999998}]},{"osd":1,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2430000000000001,"15min":1.2350000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.53000000000000003},"max":{"1min":1.5580000000000001,"5min":1.649,"15min":3.8690000000000002},"last":0.85199999999999998},{"interface":"front","average":{"1min":1.22,"5min":1.264,"15min":1.236},"min":{"1min":0.73999999999999999,"5min":0.68899999999999995,"15min":0.64500000000000002},"max":{"1min":1.6200000000000001,"5min":1.8109999999999999,"15min":3.2349999999999999},"last":0.94099999999999995}]},{"osd":2,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1619999999999999,"15min":1.177},"min":{"1min":0.72999999999999998,"5min":0.66100000000000003,"15min":0.65800000000000003},"max":{"1min":1.7170000000000001,"5min":1.7230000000000001,"15min":2.6339999999999999},"last":0.998},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.2070000000000001,"15min":1.204},"min":{"1min":0.92100000000000004,"5min":0.70499999999999996,"15min":0.65800000000000003},"max":{"1min":1.839,"5min":1.839,"15min":2.2559999999999998},"last":1.0589999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2490000000000001,"15min":1.2589999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.66500000000000004},"max":{"1min":1.907,"5min":1.907,"15min":2.8330000000000002},"last":1.109},{"interface":"front","average":{"1min":1.373,"5min":1.319,"15min":1.2929999999999999},"min":{"1min":1.0649999999999999,"5min":0.84599999999999997,"15min":0.68600000000000005},"max":{"1min":1.6850000000000001,"5min":2.0470000000000002,"15min":2.5760000000000001},"last":1.3320000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.61599999999999999},"max":{"1min":1.6279999999999999,"5min":2.0019999999999998,"15min":3.9790000000000001},"last":1.252},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.379,"15min":1.4159999999999999},"min":{"1min":1.0660000000000001,"5min":1.0569999999999999,"15min":0.81499999999999995},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":3.0209999999999999},"last":1.389}]},{"osd":6,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.419,"15min":1.407},"min":{"1min":1.0129999999999999,"5min":0.86699999999999999,"15min":0.752},"max":{"1min":1.9650000000000001,"5min":2.1560000000000001,"15min":2.5550000000000002},"last":1.48},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.411,"15min":1.4390000000000001},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.93600000000000005},"max":{"1min":1.7110000000000001,"5min":1.9419999999999999,"15min":4.1189999999999998},"last":1.429}]},{"osd":7,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.5269999999999999},"min":{"1min":1.222,"5min":1.0249999999999999,"15min":1.0049999999999999},"max":{"1min":2.214,"5min":2.214,"15min":3.1160000000000001},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.595,"5min":1.5660000000000001,"15min":1.552},"min":{"1min":1.2390000000000001,"5min":1.075,"15min":0.96999999999999997},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.7050000000000001},"last":1.6439999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.0960000000000001,"15min":1.0860000000000001},"min":{"1min":0.77200000000000002,"5min":0.63100000000000001,"15min":0.59499999999999997},"max":{"1min":1.917,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1439999999999999,"15min":1.099},"min":{"1min":0.68300000000000005,"5min":0.58299999999999996,"15min":0.45400000000000001},"max":{"1min":1.6859999999999999,"5min":1.772,"15min":1.772},"last":1.4790000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.101,"15min":1.119},"min":{"1min":0.71299999999999997,"5min":0.60099999999999998,"15min":0.56999999999999995},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":2.0459999999999998},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.179,"5min":1.173,"15min":1.173},"min":{"1min":0.84199999999999997,"5min":0.66800000000000004,"15min":0.59599999999999997},"max":{"1min":1.5469999999999999,"5min":1.6890000000000001,"15min":1.742},"last":1.3660000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.234,"15min":1.216},"min":{"1min":0.77200000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.6970000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.252},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.28,"15min":1.25},"min":{"1min":1.1499999999999999,"5min":0.83899999999999997,"15min":0.71699999999999997},"max":{"1min":1.782,"5min":1.782,"15min":1.9319999999999999},"last":1.2350000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.329,"15min":1.2869999999999999},"min":{"1min":0.95799999999999996,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6799999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.274,"15min":1.268},"min":{"1min":0.83199999999999996,"5min":0.82399999999999995,"15min":0.65100000000000002},"max":{"1min":1.544,"5min":1.8069999999999999,"15min":1.895},"last":0.76100000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.397,"15min":1.369},"min":{"1min":0.91000000000000003,"5min":0.69099999999999995,"15min":0.58799999999999997},"max":{"1min":2.0129999999999999,"5min":2.286,"15min":2.286},"last":1.286},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.456,"15min":1.4350000000000001},"min":{"1min":1.0629999999999999,"5min":0.64600000000000002,"15min":0.61899999999999999},"max":{"1min":1.827,"5min":2.1970000000000001,"15min":2.46},"last":1.528}]},{"osd":6,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.387,"15min":1.3839999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.74299999999999999},"max":{"1min":1.752,"5min":1.7769999999999999,"15min":2.5609999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.5169999999999999,"15min":1.522},"min":{"1min":1.111,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.2410000000000001,"5min":2.4780000000000002,"15min":2.6389999999999998},"last":1.4450000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5589999999999999,"15min":1.5369999999999999},"min":{"1min":1.159,"5min":0.754,"15min":0.754},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2639999999999998},"last":1.413},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.548,"15min":1.5509999999999999},"min":{"1min":1.1200000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8540000000000001,"5min":2.1070000000000002,"15min":2.6859999999999999},"last":1.212}]}]},{"osd":3,"up_from":13,"seq":55834575013,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.82699999999999996,"15min":0.80900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.397,"5min":1.4930000000000001,"15min":1.9339999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93000000000000005,"15min":0.89800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.316},"max":{"1min":1.319,"5min":1.647,"15min":1.647},"last":0.83899999999999997}]},{"osd":1,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.88600000000000001},"min":{"1min":0.48599999999999999,"5min":0.38600000000000001,"15min":0.36499999999999999},"max":{"1min":1.2729999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.012,"5min":0.998,"15min":0.97299999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.378},"max":{"1min":1.52,"5min":1.5800000000000001,"15min":1.829},"last":0.65100000000000002}]},{"osd":2,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.052,"15min":1.0109999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41699999999999998},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":2.0779999999999998},"last":0.78400000000000003},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.099,"15min":1.103},"min":{"1min":0.57499999999999996,"5min":0.504,"15min":0.504},"max":{"1min":1.6000000000000001,"5min":1.7270000000000001,"15min":2.4660000000000002},"last":0.875}]},{"osd":4,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.173,"15min":1.177},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.56399999999999995},"max":{"1min":1.571,"5min":1.5720000000000001,"15min":2.25},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.278,"15min":1.268},"min":{"1min":0.76200000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.6020000000000001,"5min":2.0110000000000001,"15min":2.1829999999999998},"last":1.165}]},{"osd":5,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.3300000000000001,"15min":1.3240000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.653,"5min":2.2229999999999999,"15min":2.3999999999999999},"last":1.357},{"interface":"front","average":{"1min":1.355,"5min":1.339,"15min":1.3300000000000001},"min":{"1min":0.879,"5min":0.80400000000000005,"15min":0.60999999999999999},"max":{"1min":1.6399999999999999,"5min":1.887,"15min":2.593},"last":1.4179999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3720000000000001,"15min":1.373},"min":{"1min":0.81899999999999995,"5min":0.751,"15min":0.58799999999999997},"max":{"1min":1.6180000000000001,"5min":1.9810000000000001,"15min":2.9569999999999999},"last":1.389},{"interface":"front","average":{"1min":1.407,"5min":1.3360000000000001,"15min":1.3460000000000001},"min":{"1min":0.86299999999999999,"5min":0.83599999999999997,"15min":0.73399999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.762},"last":1.306}]},{"osd":7,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3700000000000001,"15min":1.387},"min":{"1min":0.997,"5min":0.73699999999999999,"15min":0.70799999999999996},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.8700000000000001},"last":1.429},{"interface":"front","average":{"1min":1.353,"5min":1.4179999999999999,"15min":1.411},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7609999999999999,"5min":2.121,"15min":2.633},"last":1.4570000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81799999999999995,"15min":0.872},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.24099999999999999},"max":{"1min":1.0880000000000001,"5min":1.335,"15min":4.5369999999999999},"last":0.437},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91600000000000004,"15min":0.88},"min":{"1min":0.60999999999999999,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":1.546,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.70499999999999996}]},{"osd":1,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0669999999999999,"15min":1.05},"min":{"1min":0.64900000000000002,"5min":0.44900000000000001,"15min":0.379},"max":{"1min":1.6160000000000001,"5min":1.7689999999999999,"15min":4.4980000000000002},"last":0.56200000000000006},{"interface":"front","average":{"1min":1.157,"5min":1.143,"15min":1.0629999999999999},"min":{"1min":0.63800000000000001,"5min":0.47299999999999998,"15min":0.40300000000000002},"max":{"1min":1.7190000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.92400000000000004,"15min":0.93600000000000005},"min":{"1min":0.47299999999999998,"5min":0.44400000000000001,"15min":0.35499999999999998},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":3.3050000000000002},"last":0.61199999999999999},{"interface":"front","average":{"1min":1.099,"5min":1.0669999999999999,"15min":1.1060000000000001},"min":{"1min":0.73899999999999999,"5min":0.44800000000000001,"15min":0.42999999999999999},"max":{"1min":1.796,"5min":1.819,"15min":4.5899999999999999},"last":0.51900000000000002}]},{"osd":4,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2889999999999999,"15min":1.3160000000000001},"min":{"1min":0.88100000000000001,"5min":0.76100000000000001,"15min":0.73499999999999999},"max":{"1min":1.6639999999999999,"5min":2.2160000000000002,"15min":4.9409999999999998},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2470000000000001,"15min":1.236},"min":{"1min":0.93500000000000005,"5min":0.73899999999999999,"15min":0.63100000000000001},"max":{"1min":1.6319999999999999,"5min":2.0739999999999998,"15min":2.46},"last":1.048}]},{"osd":5,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.302,"15min":1.28},"min":{"1min":1.0009999999999999,"5min":0.84599999999999997,"15min":0.77200000000000002},"max":{"1min":1.512,"5min":1.9510000000000001,"15min":2.8530000000000002},"last":0.97399999999999998},{"interface":"front","average":{"1min":1.335,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":1.032,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.77,"5min":2.5390000000000001,"15min":3.5680000000000001},"last":0.86299999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.625},"max":{"1min":1.8200000000000001,"5min":2.0859999999999999,"15min":3.536},"last":1.151},{"interface":"front","average":{"1min":1.391,"5min":1.395,"15min":1.423},"min":{"1min":0.97399999999999998,"5min":0.81200000000000006,"15min":0.72199999999999998},"max":{"1min":1.887,"5min":2.2719999999999998,"15min":4.7549999999999999},"last":1.0209999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3799999999999999,"15min":1.3779999999999999},"min":{"1min":0.99199999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6599999999999999,"5min":2.2930000000000001,"15min":2.9700000000000002},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.393,"15min":1.4119999999999999},"min":{"1min":0.84399999999999997,"5min":0.79600000000000004,"15min":0.70499999999999996},"max":{"1min":2.0099999999999998,"5min":2.129,"15min":3.4950000000000001},"last":1.004}]}]},{"osd":1,"up_from":13,"seq":55834575014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.78800000000000003,"15min":0.77000000000000002},"min":{"1min":0.46100000000000002,"5min":0.34499999999999997,"15min":0.28899999999999998},"max":{"1min":1.218,"5min":1.5249999999999999,"15min":1.542},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.91900000000000004},"min":{"1min":0.502,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":1.2769999999999999,"5min":1.5429999999999999,"15min":4.0309999999999997},"last":0.95299999999999996}]},{"osd":2,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.99299999999999999,"15min":1.0189999999999999},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.45000000000000001},"max":{"1min":1.708,"5min":1.7689999999999999,"15min":3.0339999999999998},"last":1.351},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.95999999999999996,"15min":0.98899999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":2.6840000000000002},"last":1.135}]},{"osd":3,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.999,"15min":1.03},"min":{"1min":0.51500000000000001,"5min":0.39400000000000002,"15min":0.38900000000000001},"max":{"1min":1.587,"5min":1.587,"15min":4.3179999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.95499999999999996,"15min":1.0369999999999999},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.5329999999999999,"5min":1.585,"15min":8.5570000000000004},"last":1.304}]},{"osd":4,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.167,"15min":1.2130000000000001},"min":{"1min":0.86299999999999999,"5min":0.751,"15min":0.63300000000000001},"max":{"1min":1.76,"5min":1.907,"15min":3.1659999999999999},"last":1.464},{"interface":"front","average":{"1min":1.268,"5min":1.2350000000000001,"15min":1.266},"min":{"1min":0.75600000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.742,"5min":1.823,"15min":2.734},"last":1.093}]},{"osd":5,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.238,"15min":1.379},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.67100000000000004},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":8.4730000000000008},"last":1.504},{"interface":"front","average":{"1min":1.284,"5min":1.254,"15min":1.3819999999999999},"min":{"1min":0.92700000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.631,"5min":1.9470000000000001,"15min":8.8040000000000003},"last":1.5549999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.288,"15min":1.351},"min":{"1min":0.95399999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.677,"5min":1.9119999999999999,"15min":3.984},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.3080000000000001,"15min":1.3620000000000001},"min":{"1min":0.81599999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.78,"5min":2.0449999999999999,"15min":3.2450000000000001},"last":1.2350000000000001}]},{"osd":7,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3149999999999999,"15min":1.3819999999999999},"min":{"1min":0.83399999999999996,"5min":0.77000000000000002,"15min":0.69199999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":4.3609999999999998},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.351,"15min":1.452},"min":{"1min":0.89300000000000002,"5min":0.72799999999999998,"15min":0.70499999999999996},"max":{"1min":1.748,"5min":1.974,"15min":5.3840000000000003},"last":1.617}]}]},{"osd":0,"up_from":13,"seq":55834575012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33396,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338444,"statfs":{"total":96636764160,"available":96602566656,"internally_reserved":0,"allocated":2265088,"data_stored":1366426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.45400000000000001,"5min":0.34000000000000002,"15min":0.29399999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.6220000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.996,"15min":0.96699999999999997},"min":{"1min":0.54300000000000004,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.49,"5min":1.722,"15min":2.3340000000000001},"last":0.56100000000000005}]},{"osd":2,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90000000000000002,"15min":0.90700000000000003},"min":{"1min":0.41099999999999998,"5min":0.39900000000000002,"15min":0.37},"max":{"1min":1.3420000000000001,"5min":1.5860000000000001,"15min":1.762},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.92500000000000004,"15min":0.95199999999999996},"min":{"1min":0.47099999999999997,"5min":0.38100000000000001,"15min":0.36799999999999999},"max":{"1min":1.508,"5min":1.613,"15min":1.879},"last":0.52100000000000002}]},{"osd":3,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.93899999999999995,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.40000000000000002,"15min":0.34200000000000003},"max":{"1min":1.4430000000000001,"5min":1.72,"15min":1.728},"last":0.54400000000000004},{"interface":"front","average":{"1min":1.131,"5min":1.165,"15min":1.1719999999999999},"min":{"1min":0.499,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5820000000000001,"5min":2.1859999999999999,"15min":2.7679999999999998},"last":0.67700000000000005}]},{"osd":4,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1859999999999999,"15min":1.1579999999999999},"min":{"1min":0.90100000000000002,"5min":0.67100000000000004,"15min":0.65300000000000002},"max":{"1min":1.623,"5min":2.0489999999999999,"15min":2.2559999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.25,"15min":1.2669999999999999},"min":{"1min":0.83399999999999996,"5min":0.69999999999999996,"15min":0.59399999999999997},"max":{"1min":1.639,"5min":2.4809999999999999,"15min":2.867},"last":1.0109999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2909999999999999,"15min":1.266},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.67900000000000005},"max":{"1min":1.6899999999999999,"5min":2.3879999999999999,"15min":2.7280000000000002},"last":0.80500000000000005},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2470000000000001,"15min":1.226},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.613,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.996}]},{"osd":6,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3260000000000001,"15min":1.3440000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.611,"5min":2.3300000000000001,"15min":2.6739999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.325,"15min":1.3360000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.64200000000000002},"max":{"1min":1.7350000000000001,"5min":2.1280000000000001,"15min":2.8260000000000001},"last":1.03}]},{"osd":7,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.377,"15min":1.3939999999999999},"min":{"1min":0.92300000000000004,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7050000000000001,"5min":2.2589999999999999,"15min":4.5670000000000002},"last":0.83299999999999996},{"interface":"front","average":{"1min":1.319,"5min":1.339,"15min":1.3440000000000001},"min":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7749999999999999,"5min":2.6070000000000002,"15min":5.4450000000000003},"last":1.056}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:42:23.042 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-08T16:28:52.849628+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=16, tm_min=28, tm_sec=52, tm_wday=2, tm_yday=312, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=8, tm_hour=16, tm_min=42, tm_sec=16, tm_wday=2, tm_yday=312, tm_isdst=0) 2023-11-08T16:42:23.042 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-08T16:42:43.043 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-08T16:42:43.267 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.266+0000 7f47fbeec640 1 Processor -- start 2023-11-08T16:42:43.268 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.266+0000 7f47fbeec640 1 -- start start 2023-11-08T16:42:43.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.268+0000 7f47fbeec640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f415b500 0x7f47f4144230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.269 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.268+0000 7f47fbeec640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f40b2310 con 0x7f47f40b3170 2023-11-08T16:42:43.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.268+0000 7f47fbeec640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f40b25f0 con 0x7f47f40b3550 2023-11-08T16:42:43.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.268+0000 7f47fbeec640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40b2bb0 con 0x7f47f415b500 2023-11-08T16:42:43.270 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.268+0000 7f47f9c88640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7f47f40b3170 0x7f47f40b1ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40676/0 (socket says 172.21.15.81:40676) 2023-11-08T16:42:43.271 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.268+0000 7f47f9c88640 1 -- 172.21.15.81:0/3036215058 learned_addr learned my addr 172.21.15.81:0/3036215058 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:43.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.268+0000 7f47fa489640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f415b500 0x7f47f4144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1419905188 0 0) 0x7f47f40b2310 con 0x7f47f40b3170 2023-11-08T16:42:43.274 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 --> v1:172.21.15.81:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e0003520 con 0x7f47f40b3170 2023-11-08T16:42:43.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 <== mon.1 v1:172.21.15.93:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863392142 0 0) 0x7f47f40b25f0 con 0x7f47f40b3550 2023-11-08T16:42:43.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 --> v1:172.21.15.93:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47f40b2310 con 0x7f47f40b3550 2023-11-08T16:42:43.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 <== mon.0 v1:172.21.15.81:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1652193872 0 0) 0x7f47e0003520 con 0x7f47f40b3170 2023-11-08T16:42:43.275 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f415b500 msgr2=0x7f47f4144230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:43.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f415b500 0x7f47f4144230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 >> v1:172.21.15.93:6789/0 conn(0x7f47f40b3550 legacy=0x7f47f406c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.276 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 --> v1:172.21.15.81:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40b28d0 con 0x7f47f40b3170 2023-11-08T16:42:43.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 <== mon.0 v1:172.21.15.81:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 457607630 0 0) 0x7f47f0001670 con 0x7f47f40b3170 2023-11-08T16:42:43.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 >> v1:172.21.15.81:6789/0 conn(0x7f47f40b3170 legacy=0x7f47f40b1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.277 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f415b500 0x7f47e0003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47e0004090 0x7f47e00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 0x7f47e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.278 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40b2bb0 con 0x7f47e0004d00 2023-11-08T16:42:43.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47e000bb10 con 0x7f47e0004090 2023-11-08T16:42:43.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.269+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47e000bca0 con 0x7f47f415b500 2023-11-08T16:42:43.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.270+0000 7f47fa489640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 0x7f47e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.279 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.270+0000 7f47f9c88640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f415b500 0x7f47e0003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.270+0000 7f47fa489640 1 -- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f415b500 msgr2=0x7f47e0003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:43.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.270+0000 7f47fa489640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f415b500 0x7f47e0003b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.280 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.270+0000 7f47f9487640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47e0004090 0x7f47e00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.270+0000 7f47fa489640 1 -- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47e0004090 msgr2=0x7f47e00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:43.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.270+0000 7f47fa489640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47e0004090 0x7f47e00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.270+0000 7f47fa489640 1 -- 172.21.15.81:0/3036215058 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47e0003520 con 0x7f47e0004d00 2023-11-08T16:42:43.281 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.270+0000 7f47f9487640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47e0004090 0x7f47e00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:43.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.271+0000 7f47fa489640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 0x7f47e0005180 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009fc0 tx=0x7f47e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5b97001ac2bdaa server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:43.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.271+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec02cc20 con 0x7f47e0004d00 2023-11-08T16:42:43.282 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.271+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec00f040 con 0x7f47e0004d00 2023-11-08T16:42:43.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.271+0000 7f47f8c86640 1 -- 172.21.15.81:0/3036215058 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec034440 con 0x7f47e0004d00 2023-11-08T16:42:43.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.272+0000 7f47fbeec640 1 -- 172.21.15.81:0/3036215058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 msgr2=0x7f47e0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.283 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.272+0000 7f47fbeec640 1 --2- 172.21.15.81:0/3036215058 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 0x7f47e0005180 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009fc0 tx=0x7f47e00057f0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.272+0000 7f47fa489640 1 -- 172.21.15.81:0/3036215058 reap_dead start 2023-11-08T16:42:43.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.272+0000 7f47fbeec640 1 -- 172.21.15.81:0/3036215058 shutdown_connections 2023-11-08T16:42:43.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.272+0000 7f47fbeec640 1 -- 172.21.15.81:0/3036215058 >> 172.21.15.81:0/3036215058 conn(0x7f47f406a6a0 msgr2=0x7f47f40b07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:43.284 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.272+0000 7f47fbeec640 1 -- 172.21.15.81:0/3036215058 shutdown_connections 2023-11-08T16:42:43.285 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.272+0000 7f47fbeec640 1 -- 172.21.15.81:0/3036215058 wait complete. 2023-11-08T16:42:43.287 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.274+0000 7f47fbeec640 1 Processor -- start 2023-11-08T16:42:43.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.274+0000 7f47fbeec640 1 -- start start 2023-11-08T16:42:43.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.275+0000 7f47fbeec640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 0x7f47f4150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.288 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.275+0000 7f47fbeec640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47f415a290 0x7f47f40a8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.275+0000 7f47fbeec640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f40aa900 0x7f47f40aad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.275+0000 7f47fbeec640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40ab2c0 con 0x7f47e0004d00 2023-11-08T16:42:43.289 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.275+0000 7f47fbeec640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40a8d70 con 0x7f47f415a290 2023-11-08T16:42:43.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.275+0000 7f47fbeec640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f40a9050 con 0x7f47f40aa900 2023-11-08T16:42:43.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.275+0000 7f47f9c88640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 0x7f47f4150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.275+0000 7f47f9c88640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 0x7f47f4150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34666/0 (socket says 172.21.15.81:34666) 2023-11-08T16:42:43.290 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.275+0000 7f47f9c88640 1 -- 172.21.15.81:0/3907178283 learned_addr learned my addr 172.21.15.81:0/3907178283 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:43.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47f9487640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47f415a290 0x7f47f40a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47f9c88640 1 -- 172.21.15.81:0/3907178283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f40aa900 msgr2=0x7f47f40aad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.291 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47fa489640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f40aa900 0x7f47f40aad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47f9c88640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f40aa900 0x7f47f40aad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47f9c88640 1 -- 172.21.15.81:0/3907178283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47f415a290 msgr2=0x7f47f40a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47f9c88640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47f415a290 0x7f47f40a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.292 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47f9c88640 1 -- 172.21.15.81:0/3907178283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f40a9330 con 0x7f47e0004d00 2023-11-08T16:42:43.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47f9487640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47f415a290 0x7f47f40a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:43.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47f9c88640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 0x7f47f4150150 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f47f000d6c0 tx=0x7f47f000db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:43.293 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47ea7fc640 1 -- 172.21.15.81:0/3907178283 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f00072d0 con 0x7f47e0004d00 2023-11-08T16:42:43.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47ea7fc640 1 -- 172.21.15.81:0/3907178283 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f0007430 con 0x7f47e0004d00 2023-11-08T16:42:43.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47fbeec640 1 -- 172.21.15.81:0/3907178283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f40a9610 con 0x7f47e0004d00 2023-11-08T16:42:43.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.276+0000 7f47ea7fc640 1 -- 172.21.15.81:0/3907178283 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f00047e0 con 0x7f47e0004d00 2023-11-08T16:42:43.294 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.277+0000 7f47fbeec640 1 -- 172.21.15.81:0/3907178283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f40a9e10 con 0x7f47e0004d00 2023-11-08T16:42:43.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.278+0000 7f47fbeec640 1 -- 172.21.15.81:0/3907178283 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f4074440 con 0x7f47e0004d00 2023-11-08T16:42:43.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.278+0000 7f47ea7fc640 1 -- 172.21.15.81:0/3907178283 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f47f0004940 con 0x7f47e0004d00 2023-11-08T16:42:43.295 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.280+0000 7f47ea7fc640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f47d806ae70 0x7f47d806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.280+0000 7f47ea7fc640 1 -- 172.21.15.81:0/3907178283 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f47f0088fe0 con 0x7f47e0004d00 2023-11-08T16:42:43.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.280+0000 7f47f9487640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f47d806ae70 0x7f47d806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.281+0000 7f47f9487640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f47d806ae70 0x7f47d806d330 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f47f415a670 tx=0x7f47e4009040 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:43.296 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.285+0000 7f47ea7fc640 1 -- 172.21.15.81:0/3907178283 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7f47f00542d0 con 0x7f47e0004d00 2023-11-08T16:42:43.473 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.472+0000 7f47fbeec640 1 -- 172.21.15.81:0/3907178283 --> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47f40ccc10 con 0x7f47d806ae70 2023-11-08T16:42:43.477 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.476+0000 7f47ea7fc640 1 -- 172.21.15.81:0/3907178283 <== mgr.4101 v2:172.21.15.81:6832/3984324735 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48348 (secure 0 0 0) 0x7f47e400d040 con 0x7f47d806ae70 2023-11-08T16:42:43.478 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:43.479 INFO:teuthology.orchestra.run.smithi081.stderr:dumped all 2023-11-08T16:42:43.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.478+0000 7f47fbeec640 1 -- 172.21.15.81:0/3907178283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f47d806ae70 msgr2=0x7f47d806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.480 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.478+0000 7f47fbeec640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f47d806ae70 0x7f47d806d330 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f47f415a670 tx=0x7f47e4009040 comp rx=0 tx=0).stop 2023-11-08T16:42:43.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.478+0000 7f47fbeec640 1 -- 172.21.15.81:0/3907178283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 msgr2=0x7f47f4150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.478+0000 7f47fbeec640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 0x7f47f4150150 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f47f000d6c0 tx=0x7f47f000db80 comp rx=0 tx=0).stop 2023-11-08T16:42:43.481 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.479+0000 7f47fbeec640 1 -- 172.21.15.81:0/3907178283 shutdown_connections 2023-11-08T16:42:43.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.479+0000 7f47fbeec640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7f47d806ae70 0x7f47d806d330 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.479+0000 7f47fbeec640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7f47f40aa900 0x7f47f40aad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.482 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.479+0000 7f47fbeec640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47f415a290 0x7f47f40a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.479+0000 7f47fbeec640 1 --2- 172.21.15.81:0/3907178283 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7f47e0004d00 0x7f47f4150150 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.479+0000 7f47fbeec640 1 -- 172.21.15.81:0/3907178283 >> 172.21.15.81:0/3907178283 conn(0x7f47f406a6a0 msgr2=0x7f47f40ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:43.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.479+0000 7f47fbeec640 1 -- 172.21.15.81:0/3907178283 shutdown_connections 2023-11-08T16:42:43.483 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.479+0000 7f47fbeec640 1 -- 172.21.15.81:0/3907178283 wait complete. 2023-11-08T16:42:43.498 INFO:teuthology.orchestra.run.smithi081.stdout:{"pg_ready":true,"pg_map":{"version":543,"stamp":"2023-11-08T16:42:42.837873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":263656,"kb_used_data":14216,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711064,"statfs":{"total":773094113280,"available":772824129536,"internally_reserved":0,"allocated":14557184,"data_stored":7389200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003486"},"pg_stats":[{"pgid":"1.0","version":"134'119","reported_seq":295,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-11-08T16:42:22.793351+0000","last_change":"2023-11-08T16:42:22.793451+0000","last_active":"2023-11-08T16:42:22.793351+0000","last_peered":"2023-11-08T16:42:22.793351+0000","last_clean":"2023-11-08T16:42:22.793351+0000","last_became_active":"2023-11-08T16:28:54.812843+0000","last_became_peered":"2023-11-08T16:28:54.812843+0000","last_unstale":"2023-11-08T16:42:22.793351+0000","last_undegraded":"2023-11-08T16:42:22.793351+0000","last_fullsized":"2023-11-08T16:42:22.793351+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"134'119","last_scrub_stamp":"2023-11-08T16:42:22.793172+0000","last_deep_scrub":"134'119","last_deep_scrub_stamp":"2023-11-08T16:42:22.793172+0000","last_clean_scrub_stamp":"2023-11-08T16:42:22.793172+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-10T04:37:36.274207+0000","scrub_duration":0.0089067959999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33388,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338452,"statfs":{"total":96636764160,"available":96602574848,"internally_reserved":0,"allocated":2265088,"data_stored":1366426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:25 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.0429999999999999,"15min":1.071},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.49299999999999999},"max":{"1min":1.8240000000000001,"5min":1.8240000000000001,"15min":3.1899999999999999},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.0649999999999999,"5min":1.1000000000000001,"15min":1.1020000000000001},"min":{"1min":0.73399999999999999,"5min":0.58299999999999996,"15min":0.53600000000000003},"max":{"1min":1.4590000000000001,"5min":1.6040000000000001,"15min":4.2729999999999997},"last":1.3720000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:42:25 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.109,"15min":1.1120000000000001},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.57899999999999996},"max":{"1min":1.6140000000000001,"5min":1.891,"15min":1.891},"last":1.214},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1659999999999999,"15min":1.194},"min":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.58299999999999996},"max":{"1min":1.657,"5min":1.7090000000000001,"15min":3.3740000000000001},"last":0.80600000000000005}]},{"osd":2,"last update":"Wed Nov 8 16:42:25 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.1379999999999999,"15min":1.1899999999999999},"min":{"1min":0.76900000000000002,"5min":0.70999999999999996,"15min":0.58399999999999996},"max":{"1min":1.4730000000000001,"5min":1.8109999999999999,"15min":4.2249999999999996},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2629999999999999,"15min":1.2310000000000001},"min":{"1min":0.68700000000000006,"5min":0.66000000000000003,"15min":0.56299999999999994},"max":{"1min":1.738,"5min":1.738,"15min":3.4750000000000001},"last":1.2609999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:42:25 2023","interfaces":[{"interface":"back","average":{"1min":1.085,"5min":1.1779999999999999,"15min":1.1699999999999999},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.54900000000000004},"max":{"1min":1.4610000000000001,"5min":1.71,"15min":2.964},"last":1.4930000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.3560000000000001,"15min":1.3859999999999999},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.58599999999999997},"max":{"1min":1.847,"5min":1.847,"15min":4.181},"last":1.4450000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:42:25 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.244,"15min":1.2370000000000001},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.6539999999999999,"5min":1.7789999999999999,"15min":3.6669999999999998},"last":1.673},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.341,"15min":1.3340000000000001},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.59099999999999997},"max":{"1min":1.748,"5min":2.0179999999999998,"15min":2.3730000000000002},"last":1.419}]},{"osd":5,"last update":"Wed Nov 8 16:42:25 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.409,"15min":1.3879999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.954,"5min":1.954,"15min":3.1059999999999999},"last":1.613},{"interface":"front","average":{"1min":1.375,"5min":1.4279999999999999,"15min":1.446},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8480000000000001,"5min":1.8520000000000001,"15min":2.9750000000000001},"last":1.3280000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:42:25 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.528,"15min":1.5289999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.9370000000000001,"5min":1.9510000000000001,"15min":3.7530000000000001},"last":1.571},{"interface":"front","average":{"1min":1.466,"5min":1.4850000000000001,"15min":1.4850000000000001},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.974,"5min":2.0430000000000001,"15min":6.9560000000000004},"last":1.536}]}]},{"osd":6,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0940000000000001,"15min":1.0669999999999999},"min":{"1min":0.69499999999999995,"5min":0.64500000000000002,"15min":0.497},"max":{"1min":1.3480000000000001,"5min":1.7230000000000001,"15min":2.5609999999999999},"last":1.556},{"interface":"front","average":{"1min":0.95199999999999996,"5min":1.113,"15min":1.0880000000000001},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.56200000000000006},"max":{"1min":1.45,"5min":2.0489999999999999,"15min":2.7519999999999998},"last":0.91000000000000003}]},{"osd":1,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.1359999999999999,"15min":1.0940000000000001},"min":{"1min":0.70299999999999996,"5min":0.63400000000000001,"15min":0.60899999999999999},"max":{"1min":1.6339999999999999,"5min":1.667,"15min":1.667},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.177,"15min":1.139},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.677,"5min":1.7290000000000001,"15min":1.97},"last":1.3180000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.181,"15min":1.1759999999999999},"min":{"1min":0.72799999999999998,"5min":0.68300000000000005,"15min":0.56799999999999995},"max":{"1min":1.5549999999999999,"5min":2.262,"15min":2.262},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.085,"5min":1.1879999999999999,"15min":1.1639999999999999},"min":{"1min":0.752,"5min":0.71399999999999997,"15min":0.60499999999999998},"max":{"1min":1.516,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":1.099}]},{"osd":3,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.218,"15min":1.2310000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.58299999999999996},"max":{"1min":1.756,"5min":1.756,"15min":2.1120000000000001},"last":1.403},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.2789999999999999,"15min":1.28},"min":{"1min":0.71499999999999997,"5min":0.70899999999999996,"15min":0.67300000000000004},"max":{"1min":1.5600000000000001,"5min":1.7809999999999999,"15min":1.95},"last":1.2769999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.274,"5min":1.3120000000000001,"15min":1.3240000000000001},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7749999999999999,"5min":2.238,"15min":2.238},"last":1.46},{"interface":"front","average":{"1min":1.254,"5min":1.335,"15min":1.373},"min":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"max":{"1min":1.6970000000000001,"5min":1.9590000000000001,"15min":2.7080000000000002},"last":1.4219999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.2410000000000001,"5min":1.3049999999999999,"15min":1.3200000000000001},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.637,"5min":1.994,"15min":2.105},"last":1.006},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.5049999999999999,"15min":1.5229999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.78500000000000003},"max":{"1min":1.9259999999999999,"5min":2.3490000000000002,"15min":2.4700000000000002},"last":1.631}]},{"osd":7,"last update":"Wed Nov 8 16:42:13 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.5009999999999999,"15min":1.536},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.74299999999999999},"max":{"1min":1.8180000000000001,"5min":2.6539999999999999,"15min":3.8959999999999999},"last":1.4990000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.4390000000000001,"15min":1.48},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.615,"5min":2.0870000000000002,"15min":2.4889999999999999},"last":1.179}]}]},{"osd":5,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0900000000000001,"15min":1.071},"min":{"1min":0.64300000000000002,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.571,"5min":1.657,"15min":2.4609999999999999},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.097,"5min":1.1160000000000001,"15min":1.103},"min":{"1min":0.752,"5min":0.63700000000000001,"15min":0.57999999999999996},"max":{"1min":1.7849999999999999,"5min":1.7849999999999999,"15min":2.3639999999999999},"last":1.016}]},{"osd":1,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2430000000000001,"15min":1.2350000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.53000000000000003},"max":{"1min":1.5580000000000001,"5min":1.649,"15min":3.8690000000000002},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.22,"5min":1.264,"15min":1.236},"min":{"1min":0.73999999999999999,"5min":0.68899999999999995,"15min":0.64500000000000002},"max":{"1min":1.6200000000000001,"5min":1.8109999999999999,"15min":3.2349999999999999},"last":1.2050000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1619999999999999,"15min":1.177},"min":{"1min":0.72999999999999998,"5min":0.66100000000000003,"15min":0.65800000000000003},"max":{"1min":1.7170000000000001,"5min":1.7230000000000001,"15min":2.6339999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.3080000000000001,"5min":1.2070000000000001,"15min":1.204},"min":{"1min":0.92100000000000004,"5min":0.70499999999999996,"15min":0.65800000000000003},"max":{"1min":1.839,"5min":1.839,"15min":2.2559999999999998},"last":1.536}]},{"osd":3,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.2490000000000001,"15min":1.2589999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.66500000000000004},"max":{"1min":1.907,"5min":1.907,"15min":2.8330000000000002},"last":1.248},{"interface":"front","average":{"1min":1.373,"5min":1.319,"15min":1.2929999999999999},"min":{"1min":1.0649999999999999,"5min":0.84599999999999997,"15min":0.68600000000000005},"max":{"1min":1.6850000000000001,"5min":2.0470000000000002,"15min":2.5760000000000001},"last":1.3979999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2709999999999999,"15min":1.3029999999999999},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.61599999999999999},"max":{"1min":1.6279999999999999,"5min":2.0019999999999998,"15min":3.9790000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.379,"15min":1.4159999999999999},"min":{"1min":1.0660000000000001,"5min":1.0569999999999999,"15min":0.81499999999999995},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":3.0209999999999999},"last":1.4330000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.399,"5min":1.419,"15min":1.407},"min":{"1min":1.0129999999999999,"5min":0.86699999999999999,"15min":0.752},"max":{"1min":1.9650000000000001,"5min":2.1560000000000001,"15min":2.5550000000000002},"last":1.149},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.411,"15min":1.4390000000000001},"min":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.93600000000000005},"max":{"1min":1.7110000000000001,"5min":1.9419999999999999,"15min":4.1189999999999998},"last":1.087}]},{"osd":7,"last update":"Wed Nov 8 16:42:18 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.5269999999999999},"min":{"1min":1.222,"5min":1.0249999999999999,"15min":1.0049999999999999},"max":{"1min":2.214,"5min":2.214,"15min":3.1160000000000001},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.595,"5min":1.5660000000000001,"15min":1.552},"min":{"1min":1.2390000000000001,"5min":1.075,"15min":0.96999999999999997},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.7050000000000001},"last":1.292}]}]},{"osd":4,"up_from":13,"seq":55834575017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.0960000000000001,"15min":1.0860000000000001},"min":{"1min":0.77200000000000002,"5min":0.63100000000000001,"15min":0.59499999999999997},"max":{"1min":1.917,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1439999999999999,"15min":1.099},"min":{"1min":0.68300000000000005,"5min":0.58299999999999996,"15min":0.45400000000000001},"max":{"1min":1.6859999999999999,"5min":1.772,"15min":1.772},"last":1.3480000000000001}]},{"osd":1,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.101,"15min":1.119},"min":{"1min":0.71299999999999997,"5min":0.60099999999999998,"15min":0.56999999999999995},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":2.0459999999999998},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.179,"5min":1.173,"15min":1.173},"min":{"1min":0.84199999999999997,"5min":0.66800000000000004,"15min":0.59599999999999997},"max":{"1min":1.5469999999999999,"5min":1.6890000000000001,"15min":1.742},"last":1.466}]},{"osd":2,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.234,"15min":1.216},"min":{"1min":0.77200000000000002,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.6970000000000001,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":0.99199999999999999},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.28,"15min":1.25},"min":{"1min":1.1499999999999999,"5min":0.83899999999999997,"15min":0.71699999999999997},"max":{"1min":1.782,"5min":1.782,"15min":1.9319999999999999},"last":1.677}]},{"osd":3,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.329,"15min":1.2869999999999999},"min":{"1min":0.95799999999999996,"5min":0.68799999999999994,"15min":0.68799999999999994},"max":{"1min":1.6799999999999999,"5min":2.2919999999999998,"15min":2.2919999999999998},"last":1.298},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.274,"15min":1.268},"min":{"1min":0.83199999999999996,"5min":0.82399999999999995,"15min":0.65100000000000002},"max":{"1min":1.544,"5min":1.8069999999999999,"15min":1.895},"last":1.4019999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.397,"15min":1.369},"min":{"1min":0.91000000000000003,"5min":0.69099999999999995,"15min":0.58799999999999997},"max":{"1min":2.0129999999999999,"5min":2.286,"15min":2.286},"last":1.556},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.456,"15min":1.4350000000000001},"min":{"1min":1.0629999999999999,"5min":0.64600000000000002,"15min":0.61899999999999999},"max":{"1min":1.827,"5min":2.1970000000000001,"15min":2.46},"last":1.169}]},{"osd":6,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.387,"15min":1.3839999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.74299999999999999},"max":{"1min":1.752,"5min":1.7769999999999999,"15min":2.5609999999999999},"last":1.054},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.5169999999999999,"15min":1.522},"min":{"1min":1.111,"5min":0.78800000000000003,"15min":0.78800000000000003},"max":{"1min":2.2410000000000001,"5min":2.4780000000000002,"15min":2.6389999999999998},"last":1.6299999999999999}]},{"osd":7,"last update":"Wed Nov 8 16:42:14 2023","interfaces":[{"interface":"back","average":{"1min":1.55,"5min":1.5589999999999999,"15min":1.5369999999999999},"min":{"1min":1.159,"5min":0.754,"15min":0.754},"max":{"1min":2.2029999999999998,"5min":2.2029999999999998,"15min":2.2639999999999998},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.548,"15min":1.5509999999999999},"min":{"1min":1.1200000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8540000000000001,"5min":2.1070000000000002,"15min":2.6859999999999999},"last":1.532}]}]},{"osd":3,"up_from":13,"seq":55834575017,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339024,"statfs":{"total":96636764160,"available":96603160576,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.82699999999999996,"15min":0.80900000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.397,"5min":1.4930000000000001,"15min":1.9339999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93000000000000005,"15min":0.89800000000000002},"min":{"1min":0.45500000000000002,"5min":0.45000000000000001,"15min":0.316},"max":{"1min":1.319,"5min":1.647,"15min":1.647},"last":1.0269999999999999}]},{"osd":1,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.89300000000000002,"15min":0.88600000000000001},"min":{"1min":0.48599999999999999,"5min":0.38600000000000001,"15min":0.36499999999999999},"max":{"1min":1.2729999999999999,"5min":1.6120000000000001,"15min":1.6120000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.012,"5min":0.998,"15min":0.97299999999999998},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.378},"max":{"1min":1.52,"5min":1.5800000000000001,"15min":1.829},"last":1.5129999999999999}]},{"osd":2,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":1.052,"15min":1.0109999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.41699999999999998},"max":{"1min":1.8700000000000001,"5min":1.8700000000000001,"15min":2.0779999999999998},"last":1.163},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.099,"15min":1.103},"min":{"1min":0.57499999999999996,"5min":0.504,"15min":0.504},"max":{"1min":1.6000000000000001,"5min":1.7270000000000001,"15min":2.4660000000000002},"last":1.4339999999999999}]},{"osd":4,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.173,"15min":1.177},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.56399999999999995},"max":{"1min":1.571,"5min":1.5720000000000001,"15min":2.25},"last":1.319},{"interface":"front","average":{"1min":1.24,"5min":1.278,"15min":1.268},"min":{"1min":0.76200000000000001,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.6020000000000001,"5min":2.0110000000000001,"15min":2.1829999999999998},"last":1.4810000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2729999999999999,"5min":1.3300000000000001,"15min":1.3240000000000001},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.653,"5min":2.2229999999999999,"15min":2.3999999999999999},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.339,"15min":1.3300000000000001},"min":{"1min":0.879,"5min":0.80400000000000005,"15min":0.60999999999999999},"max":{"1min":1.6399999999999999,"5min":1.887,"15min":2.593},"last":1.2529999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.3720000000000001,"15min":1.373},"min":{"1min":0.81899999999999995,"5min":0.751,"15min":0.58799999999999997},"max":{"1min":1.6180000000000001,"5min":1.9810000000000001,"15min":2.9569999999999999},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.407,"5min":1.3360000000000001,"15min":1.3460000000000001},"min":{"1min":0.86299999999999999,"5min":0.83599999999999997,"15min":0.73399999999999999},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":2.762},"last":1.532}]},{"osd":7,"last update":"Wed Nov 8 16:42:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4319999999999999,"5min":1.3700000000000001,"15min":1.387},"min":{"1min":0.997,"5min":0.73699999999999999,"15min":0.70799999999999996},"max":{"1min":2.0089999999999999,"5min":2.0089999999999999,"15min":2.8700000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.353,"5min":1.4179999999999999,"15min":1.411},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.7609999999999999,"5min":2.121,"15min":2.633},"last":1.2170000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.81799999999999995,"15min":0.872},"min":{"1min":0.47099999999999997,"5min":0.39600000000000002,"15min":0.24099999999999999},"max":{"1min":1.0880000000000001,"5min":1.335,"15min":4.5369999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91600000000000004,"15min":0.88},"min":{"1min":0.60999999999999999,"5min":0.35699999999999998,"15min":0.33300000000000002},"max":{"1min":1.546,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.96399999999999997}]},{"osd":1,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0669999999999999,"15min":1.05},"min":{"1min":0.64900000000000002,"5min":0.44900000000000001,"15min":0.379},"max":{"1min":1.6160000000000001,"5min":1.7689999999999999,"15min":4.4980000000000002},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.143,"15min":1.0629999999999999},"min":{"1min":0.63800000000000001,"5min":0.47299999999999998,"15min":0.40300000000000002},"max":{"1min":1.7190000000000001,"5min":2.1909999999999998,"15min":2.1909999999999998},"last":1.3280000000000001}]},{"osd":3,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":0.92400000000000004,"15min":0.93600000000000005},"min":{"1min":0.47299999999999998,"5min":0.44400000000000001,"15min":0.35499999999999998},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":3.3050000000000002},"last":1.194},{"interface":"front","average":{"1min":1.099,"5min":1.0669999999999999,"15min":1.1060000000000001},"min":{"1min":0.73899999999999999,"5min":0.44800000000000001,"15min":0.42999999999999999},"max":{"1min":1.796,"5min":1.819,"15min":4.5899999999999999},"last":1.1040000000000001}]},{"osd":4,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2809999999999999,"5min":1.2889999999999999,"15min":1.3160000000000001},"min":{"1min":0.88100000000000001,"5min":0.76100000000000001,"15min":0.73499999999999999},"max":{"1min":1.6639999999999999,"5min":2.2160000000000002,"15min":4.9409999999999998},"last":1.377},{"interface":"front","average":{"1min":1.2509999999999999,"5min":1.2470000000000001,"15min":1.236},"min":{"1min":0.93500000000000005,"5min":0.73899999999999999,"15min":0.63100000000000001},"max":{"1min":1.6319999999999999,"5min":2.0739999999999998,"15min":2.46},"last":1.6539999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.302,"15min":1.28},"min":{"1min":1.0009999999999999,"5min":0.84599999999999997,"15min":0.77200000000000002},"max":{"1min":1.512,"5min":1.9510000000000001,"15min":2.8530000000000002},"last":1.472},{"interface":"front","average":{"1min":1.335,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":1.032,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.77,"5min":2.5390000000000001,"15min":3.5680000000000001},"last":1.4279999999999999}]},{"osd":6,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.2869999999999999,"15min":1.3149999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.625},"max":{"1min":1.8200000000000001,"5min":2.0859999999999999,"15min":3.536},"last":1.504},{"interface":"front","average":{"1min":1.391,"5min":1.395,"15min":1.423},"min":{"1min":0.97399999999999998,"5min":0.81200000000000006,"15min":0.72199999999999998},"max":{"1min":1.887,"5min":2.2719999999999998,"15min":4.7549999999999999},"last":1.272}]},{"osd":7,"last update":"Wed Nov 8 16:42:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.3799999999999999,"15min":1.3779999999999999},"min":{"1min":0.99199999999999999,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6599999999999999,"5min":2.2930000000000001,"15min":2.9700000000000002},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.393,"15min":1.4119999999999999},"min":{"1min":0.84399999999999997,"5min":0.79600000000000004,"15min":0.70499999999999996},"max":{"1min":2.0099999999999998,"5min":2.129,"15min":3.4950000000000001},"last":1.633}]}]},{"osd":1,"up_from":13,"seq":55834575018,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32812,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1671168,"data_stored":776058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.78800000000000003,"15min":0.77000000000000002},"min":{"1min":0.46100000000000002,"5min":0.34499999999999997,"15min":0.28899999999999998},"max":{"1min":1.218,"5min":1.5249999999999999,"15min":1.542},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.85999999999999999,"5min":0.88,"15min":0.91900000000000004},"min":{"1min":0.502,"5min":0.33700000000000002,"15min":0.309},"max":{"1min":1.2769999999999999,"5min":1.5429999999999999,"15min":4.0309999999999997},"last":1.2450000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.99299999999999999,"15min":1.0189999999999999},"min":{"1min":0.52200000000000002,"5min":0.48899999999999999,"15min":0.45000000000000001},"max":{"1min":1.708,"5min":1.7689999999999999,"15min":3.0339999999999998},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.0229999999999999,"5min":0.95999999999999996,"15min":0.98899999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":2.6840000000000002},"last":1.2769999999999999}]},{"osd":3,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.999,"15min":1.03},"min":{"1min":0.51500000000000001,"5min":0.39400000000000002,"15min":0.38900000000000001},"max":{"1min":1.587,"5min":1.587,"15min":4.3179999999999996},"last":1.0680000000000001},{"interface":"front","average":{"1min":1.0149999999999999,"5min":0.95499999999999996,"15min":1.0369999999999999},"min":{"1min":0.48899999999999999,"5min":0.38100000000000001,"15min":0.34999999999999998},"max":{"1min":1.5329999999999999,"5min":1.585,"15min":8.5570000000000004},"last":0.90500000000000003}]},{"osd":4,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.167,"15min":1.2130000000000001},"min":{"1min":0.86299999999999999,"5min":0.751,"15min":0.63300000000000001},"max":{"1min":1.76,"5min":1.907,"15min":3.1659999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.2350000000000001,"15min":1.266},"min":{"1min":0.75600000000000001,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.742,"5min":1.823,"15min":2.734},"last":1.3839999999999999}]},{"osd":5,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2909999999999999,"5min":1.238,"15min":1.379},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.67100000000000004},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":8.4730000000000008},"last":1.119},{"interface":"front","average":{"1min":1.284,"5min":1.254,"15min":1.3819999999999999},"min":{"1min":0.92700000000000005,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.631,"5min":1.9470000000000001,"15min":8.8040000000000003},"last":1.4770000000000001}]},{"osd":6,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.288,"15min":1.351},"min":{"1min":0.95399999999999996,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.677,"5min":1.9119999999999999,"15min":3.984},"last":1.53},{"interface":"front","average":{"1min":1.298,"5min":1.3080000000000001,"15min":1.3620000000000001},"min":{"1min":0.81599999999999995,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.78,"5min":2.0449999999999999,"15min":3.2450000000000001},"last":1.329}]},{"osd":7,"last update":"Wed Nov 8 16:42:16 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.3149999999999999,"15min":1.3819999999999999},"min":{"1min":0.83399999999999996,"5min":0.77000000000000002,"15min":0.69199999999999995},"max":{"1min":2.2309999999999999,"5min":2.2309999999999999,"15min":4.3609999999999998},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.373,"5min":1.351,"15min":1.452},"min":{"1min":0.89300000000000002,"5min":0.72799999999999998,"15min":0.70499999999999996},"max":{"1min":1.748,"5min":1.974,"15min":5.3840000000000003},"last":1.417}]}]},{"osd":0,"up_from":13,"seq":55834575016,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33388,"kb_used_data":2212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338452,"statfs":{"total":96636764160,"available":96602574848,"internally_reserved":0,"allocated":2265088,"data_stored":1366426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.88,"5min":0.83199999999999996,"15min":0.82599999999999996},"min":{"1min":0.45400000000000001,"5min":0.34000000000000002,"15min":0.29399999999999998},"max":{"1min":1.413,"5min":1.413,"15min":1.6220000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.0760000000000001,"5min":0.996,"15min":0.96699999999999997},"min":{"1min":0.54300000000000004,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.49,"5min":1.722,"15min":2.3340000000000001},"last":0.88500000000000001}]},{"osd":2,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.90000000000000002,"15min":0.90700000000000003},"min":{"1min":0.41099999999999998,"5min":0.39900000000000002,"15min":0.37},"max":{"1min":1.3420000000000001,"5min":1.5860000000000001,"15min":1.762},"last":1.3759999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.92500000000000004,"15min":0.95199999999999996},"min":{"1min":0.47099999999999997,"5min":0.38100000000000001,"15min":0.36799999999999999},"max":{"1min":1.508,"5min":1.613,"15min":1.879},"last":1.006}]},{"osd":3,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.93899999999999995,"15min":0.93000000000000005},"min":{"1min":0.433,"5min":0.40000000000000002,"15min":0.34200000000000003},"max":{"1min":1.4430000000000001,"5min":1.72,"15min":1.728},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.131,"5min":1.165,"15min":1.1719999999999999},"min":{"1min":0.499,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.5820000000000001,"5min":2.1859999999999999,"15min":2.7679999999999998},"last":1.492}]},{"osd":4,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.1859999999999999,"15min":1.1579999999999999},"min":{"1min":0.90100000000000002,"5min":0.67100000000000004,"15min":0.65300000000000002},"max":{"1min":1.623,"5min":2.0489999999999999,"15min":2.2559999999999998},"last":1.234},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.25,"15min":1.2669999999999999},"min":{"1min":0.83399999999999996,"5min":0.69999999999999996,"15min":0.59399999999999997},"max":{"1min":1.639,"5min":2.4809999999999999,"15min":2.867},"last":1.6990000000000001}]},{"osd":5,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2909999999999999,"15min":1.266},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.67900000000000005},"max":{"1min":1.6899999999999999,"5min":2.3879999999999999,"15min":2.7280000000000002},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.2170000000000001,"5min":1.2470000000000001,"15min":1.226},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.613,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":1.415}]},{"osd":6,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.3260000000000001,"15min":1.3440000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.611,"5min":2.3300000000000001,"15min":2.6739999999999999},"last":1.619},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.325,"15min":1.3360000000000001},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.64200000000000002},"max":{"1min":1.7350000000000001,"5min":2.1280000000000001,"15min":2.8260000000000001},"last":1.458}]},{"osd":7,"last update":"Wed Nov 8 16:42:15 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.377,"15min":1.3939999999999999},"min":{"1min":0.92300000000000004,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.7050000000000001,"5min":2.2589999999999999,"15min":4.5670000000000002},"last":1.74},{"interface":"front","average":{"1min":1.319,"5min":1.339,"15min":1.3440000000000001},"min":{"1min":0.76400000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.7749999999999999,"5min":2.6070000000000002,"15min":5.4450000000000003},"last":1.554}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-08T16:42:43.501 DEBUG:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-08T16:42:43.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.702+0000 7fe90fe7c640 1 Processor -- start 2023-11-08T16:42:43.704 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.703+0000 7fe90fe7c640 1 -- start start 2023-11-08T16:42:43.705 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.704+0000 7fe90fe7c640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe90815d420 0x7fe9080d10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.704+0000 7fe90fe7c640 1 -- --> v1:172.21.15.81:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9080d1a90 con 0x7fe9080b60a0 2023-11-08T16:42:43.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.704+0000 7fe90fe7c640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9080d1d70 con 0x7fe9080cf710 2023-11-08T16:42:43.706 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.704+0000 7fe90fe7c640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9080d2050 con 0x7fe90815d420 2023-11-08T16:42:43.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.705+0000 7fe90dc18640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fe90815d420 0x7fe9080d10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.705+0000 7fe90e419640 1 --1- >> v1:172.21.15.81:6789/0 conn(0x7fe9080b60a0 0x7fe9080b6510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.81:6789/0 says I am v1:172.21.15.81:40680/0 (socket says 172.21.15.81:40680) 2023-11-08T16:42:43.708 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.705+0000 7fe90e419640 1 -- 172.21.15.81:0/370247147 learned_addr learned my addr 172.21.15.81:0/370247147 (peer_addr_for_me v1:172.21.15.81:0/0) 2023-11-08T16:42:43.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.705+0000 7fe90dc18640 1 -- 172.21.15.81:0/370247147 >> v1:172.21.15.93:6789/0 conn(0x7fe9080cf710 legacy=0x7fe9080cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:43.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.705+0000 7fe90dc18640 1 -- 172.21.15.81:0/370247147 >> v1:172.21.15.81:6789/0 conn(0x7fe9080b60a0 legacy=0x7fe9080b6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.709 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.705+0000 7fe90dc18640 1 -- 172.21.15.81:0/370247147 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9080d2330 con 0x7fe90815d420 2023-11-08T16:42:43.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.705+0000 7fe90cc16640 1 -- 172.21.15.81:0/370247147 <== mon.0 v1:172.21.15.81:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513594583 0 0) 0x7fe9080d1a90 con 0x7fe9080b60a0 2023-11-08T16:42:43.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.706+0000 7fe90dc18640 1 --2- 172.21.15.81:0/370247147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe90815d420 0x7fe9080d10f0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009d90 tx=0x7fe8f802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c0b9c192c3ff03 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:43.710 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.707+0000 7fe90cc16640 1 -- 172.21.15.81:0/370247147 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f802cc60 con 0x7fe90815d420 2023-11-08T16:42:43.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.707+0000 7fe90cc16640 1 -- 172.21.15.81:0/370247147 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f802cdc0 con 0x7fe90815d420 2023-11-08T16:42:43.711 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.707+0000 7fe90cc16640 1 -- 172.21.15.81:0/370247147 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8034660 con 0x7fe90815d420 2023-11-08T16:42:43.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.707+0000 7fe90fe7c640 1 -- 172.21.15.81:0/370247147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe90815d420 msgr2=0x7fe9080d10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.713 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.707+0000 7fe90fe7c640 1 --2- 172.21.15.81:0/370247147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe90815d420 0x7fe9080d10f0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009d90 tx=0x7fe8f802beb0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.708+0000 7fe90fe7c640 1 -- 172.21.15.81:0/370247147 shutdown_connections 2023-11-08T16:42:43.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.708+0000 7fe90fe7c640 1 --2- 172.21.15.81:0/370247147 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe90815d420 0x7fe9080d10f0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.714 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.708+0000 7fe90fe7c640 1 -- 172.21.15.81:0/370247147 >> 172.21.15.81:0/370247147 conn(0x7fe9080d2f70 msgr2=0x7fe9080d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:43.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.708+0000 7fe90fe7c640 1 -- 172.21.15.81:0/370247147 shutdown_connections 2023-11-08T16:42:43.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.708+0000 7fe90fe7c640 1 -- 172.21.15.81:0/370247147 wait complete. 2023-11-08T16:42:43.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.710+0000 7fe90fe7c640 1 Processor -- start 2023-11-08T16:42:43.715 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.710+0000 7fe90fe7c640 1 -- start start 2023-11-08T16:42:43.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.711+0000 7fe90fe7c640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9080b60a0 0x7fe9080b4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.711+0000 7fe90fe7c640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9080cf710 0x7fe9081590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.711+0000 7fe90fe7c640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe90815d420 0x7fe90815ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.716 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.711+0000 7fe90fe7c640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9081580c0 con 0x7fe9080cf710 2023-11-08T16:42:43.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.711+0000 7fe90fe7c640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe908158330 con 0x7fe9080b60a0 2023-11-08T16:42:43.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.711+0000 7fe90fe7c640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe908158610 con 0x7fe90815d420 2023-11-08T16:42:43.717 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.712+0000 7fe90d417640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9080cf710 0x7fe9081590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.712+0000 7fe90d417640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9080cf710 0x7fe9081590f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34686/0 (socket says 172.21.15.81:34686) 2023-11-08T16:42:43.718 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.712+0000 7fe90d417640 1 -- 172.21.15.81:0/48748355 learned_addr learned my addr 172.21.15.81:0/48748355 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:43.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.712+0000 7fe90dc18640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9080b60a0 0x7fe9080b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.720 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.712+0000 7fe90d417640 1 -- 172.21.15.81:0/48748355 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe90815d420 msgr2=0x7fe90815ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.712+0000 7fe90d417640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe90815d420 0x7fe90815ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.712+0000 7fe90d417640 1 -- 172.21.15.81:0/48748355 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9080b60a0 msgr2=0x7fe9080b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.721 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.712+0000 7fe90d417640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9080b60a0 0x7fe9080b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.712+0000 7fe90d417640 1 -- 172.21.15.81:0/48748355 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8009a70 con 0x7fe9080cf710 2023-11-08T16:42:43.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.712+0000 7fe90dc18640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9080b60a0 0x7fe9080b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:43.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.713+0000 7fe90d417640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9080cf710 0x7fe9081590f0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00c400 tx=0x7fe8fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:43.722 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.713+0000 7fe9067fc640 1 -- 172.21.15.81:0/48748355 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc011790 con 0x7fe9080cf710 2023-11-08T16:42:43.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.713+0000 7fe9067fc640 1 -- 172.21.15.81:0/48748355 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc011930 con 0x7fe9080cf710 2023-11-08T16:42:43.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.713+0000 7fe9067fc640 1 -- 172.21.15.81:0/48748355 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc011ce0 con 0x7fe9080cf710 2023-11-08T16:42:43.723 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.713+0000 7fe90fe7c640 1 -- 172.21.15.81:0/48748355 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9081588f0 con 0x7fe9080cf710 2023-11-08T16:42:43.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.713+0000 7fe90fe7c640 1 -- 172.21.15.81:0/48748355 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe908159630 con 0x7fe9080cf710 2023-11-08T16:42:43.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.714+0000 7fe90fe7c640 1 -- 172.21.15.81:0/48748355 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe908158e10 con 0x7fe9080cf710 2023-11-08T16:42:43.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.715+0000 7fe9067fc640 1 -- 172.21.15.81:0/48748355 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe8fc01c460 con 0x7fe9080cf710 2023-11-08T16:42:43.724 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.716+0000 7fe9067fc640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe8f406ae70 0x7fe8f406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:43.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.716+0000 7fe9067fc640 1 -- 172.21.15.81:0/48748355 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe8fc0892e0 con 0x7fe9080cf710 2023-11-08T16:42:43.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.716+0000 7fe90dc18640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe8f406ae70 0x7fe8f406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:43.725 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.717+0000 7fe90dc18640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe8f406ae70 0x7fe8f406d330 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8036000 tx=0x7fe8f8030000 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:43.726 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.721+0000 7fe9067fc640 1 -- 172.21.15.81:0/48748355 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fe908158e10 con 0x7fe9080cf710 2023-11-08T16:42:43.912 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.910+0000 7fe90fe7c640 1 -- 172.21.15.81:0/48748355 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fe908158e10 con 0x7fe9080cf710 2023-11-08T16:42:43.919 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.918+0000 7fe9067fc640 1 -- 172.21.15.81:0/48748355 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v20)=0 v20) v1 ==== 123+0+0 (secure 0 0 0) 0x7fe908158e10 con 0x7fe9080cf710 2023-11-08T16:42:43.922 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.920+0000 7fe9067fc640 1 -- 172.21.15.81:0/48748355 <== mon.0 v2:172.21.15.81:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe8fc0545e0 con 0x7fe9080cf710 2023-11-08T16:42:43.924 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.922+0000 7fe90fe7c640 1 -- 172.21.15.81:0/48748355 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe8f406ae70 msgr2=0x7fe8f406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.922+0000 7fe90fe7c640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe8f406ae70 0x7fe8f406d330 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8036000 tx=0x7fe8f8030000 comp rx=0 tx=0).stop 2023-11-08T16:42:43.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.922+0000 7fe90fe7c640 1 -- 172.21.15.81:0/48748355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9080cf710 msgr2=0x7fe9081590f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:43.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.922+0000 7fe90fe7c640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9080cf710 0x7fe9081590f0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00c400 tx=0x7fe8fc00c8c0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.925 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.923+0000 7fe90fe7c640 1 -- 172.21.15.81:0/48748355 shutdown_connections 2023-11-08T16:42:43.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.923+0000 7fe90fe7c640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fe8f406ae70 0x7fe8f406d330 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.923+0000 7fe90fe7c640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fe90815d420 0x7fe90815ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.926 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.923+0000 7fe90fe7c640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fe9080cf710 0x7fe9081590f0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.923+0000 7fe90fe7c640 1 --2- 172.21.15.81:0/48748355 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe9080b60a0 0x7fe9080b4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:43.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.923+0000 7fe90fe7c640 1 -- 172.21.15.81:0/48748355 >> 172.21.15.81:0/48748355 conn(0x7fe9080d2f70 msgr2=0x7fe90815c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:43.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.923+0000 7fe90fe7c640 1 -- 172.21.15.81:0/48748355 shutdown_connections 2023-11-08T16:42:43.927 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:43.923+0000 7fe90fe7c640 1 -- 172.21.15.81:0/48748355 wait complete. 2023-11-08T16:42:43.943 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-08T16:42:43.944 DEBUG:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-08T16:42:44.154 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.153+0000 7fb739724640 1 Processor -- start 2023-11-08T16:42:44.155 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.153+0000 7fb739724640 1 -- start start 2023-11-08T16:42:44.156 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.154+0000 7fb739724640 1 --2- >> v2:172.21.15.93:3300/0 conn(0x7fb73406a2e0 0x7fb7340cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:44.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.154+0000 7fb739724640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb73406a6c0 0x7fb7340cb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:44.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.154+0000 7fb739724640 1 -- --> v1:172.21.15.93:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7340ca2a0 con 0x7fb73406ee90 2023-11-08T16:42:44.157 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.154+0000 7fb739724640 1 -- --> v2:172.21.15.81:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73406f420 con 0x7fb73406a6c0 2023-11-08T16:42:44.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.154+0000 7fb739724640 1 -- --> v2:172.21.15.93:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7340ca4f0 con 0x7fb73406a2e0 2023-11-08T16:42:44.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.154+0000 7fb7327fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb73406a6c0 0x7fb7340cb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:44.158 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.154+0000 7fb7327fc640 1 --2- >> v2:172.21.15.81:3300/0 conn(0x7fb73406a6c0 0x7fb7340cb510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34690/0 (socket says 172.21.15.81:34690) 2023-11-08T16:42:44.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.154+0000 7fb7327fc640 1 -- 172.21.15.81:0/759931677 learned_addr learned my addr 172.21.15.81:0/759931677 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:44.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.155+0000 7fb7327fc640 1 -- 172.21.15.81:0/759931677 >> v2:172.21.15.93:3300/0 conn(0x7fb73406a2e0 msgr2=0x7fb7340cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:44.159 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.155+0000 7fb732ffd640 1 --2- 172.21.15.81:0/759931677 >> v2:172.21.15.93:3300/0 conn(0x7fb73406a2e0 0x7fb7340cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:44.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.155+0000 7fb7327fc640 1 --2- 172.21.15.81:0/759931677 >> v2:172.21.15.93:3300/0 conn(0x7fb73406a2e0 0x7fb7340cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:44.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.155+0000 7fb7327fc640 1 -- 172.21.15.81:0/759931677 >> v1:172.21.15.93:6789/0 conn(0x7fb73406ee90 legacy=0x7fb7340cba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-08T16:42:44.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.155+0000 7fb7327fc640 1 -- 172.21.15.81:0/759931677 --> v2:172.21.15.81:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7340ca7d0 con 0x7fb73406a6c0 2023-11-08T16:42:44.160 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.155+0000 7fb732ffd640 1 --2- 172.21.15.81:0/759931677 >> v2:172.21.15.93:3300/0 conn(0x7fb73406a2e0 0x7fb7340cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-08T16:42:44.163 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.156+0000 7fb7327fc640 1 --2- 172.21.15.81:0/759931677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb73406a6c0 0x7fb7340cb510 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fb734061da0 tx=0x7fb71c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3bc04fa6b1045d server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:44.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.156+0000 7fb731ffb640 1 -- 172.21.15.81:0/759931677 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c031870 con 0x7fb73406a6c0 2023-11-08T16:42:44.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.156+0000 7fb731ffb640 1 -- 172.21.15.81:0/759931677 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb71c0319d0 con 0x7fb73406a6c0 2023-11-08T16:42:44.164 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.156+0000 7fb731ffb640 1 -- 172.21.15.81:0/759931677 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c031d80 con 0x7fb73406a6c0 2023-11-08T16:42:44.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.157+0000 7fb739724640 1 -- 172.21.15.81:0/759931677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb73406a6c0 msgr2=0x7fb7340cb510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:44.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.157+0000 7fb739724640 1 --2- 172.21.15.81:0/759931677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb73406a6c0 0x7fb7340cb510 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fb734061da0 tx=0x7fb71c02b910 comp rx=0 tx=0).stop 2023-11-08T16:42:44.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.157+0000 7fb739724640 1 -- 172.21.15.81:0/759931677 shutdown_connections 2023-11-08T16:42:44.165 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.157+0000 7fb739724640 1 --2- 172.21.15.81:0/759931677 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb73406a6c0 0x7fb7340cb510 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:44.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.157+0000 7fb739724640 1 --2- 172.21.15.81:0/759931677 >> v2:172.21.15.93:3300/0 conn(0x7fb73406a2e0 0x7fb7340cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:44.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.157+0000 7fb739724640 1 -- 172.21.15.81:0/759931677 >> 172.21.15.81:0/759931677 conn(0x7fb7340cc6d0 msgr2=0x7fb7340ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:44.166 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.157+0000 7fb739724640 1 -- 172.21.15.81:0/759931677 shutdown_connections 2023-11-08T16:42:44.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.157+0000 7fb739724640 1 -- 172.21.15.81:0/759931677 wait complete. 2023-11-08T16:42:44.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.159+0000 7fb739724640 1 Processor -- start 2023-11-08T16:42:44.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.159+0000 7fb739724640 1 -- start start 2023-11-08T16:42:44.167 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb739724640 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb73406a2e0 0x7fb7340c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:44.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb739724640 1 --2- >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb73406a6c0 0x7fb7340c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:44.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb739724640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb73406ee90 0x7fb7340cb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:44.168 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb739724640 1 -- --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340c6120 con 0x7fb73406ee90 2023-11-08T16:42:44.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb739724640 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340c65b0 con 0x7fb73406a2e0 2023-11-08T16:42:44.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb739724640 1 -- --> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7340c9a60 con 0x7fb73406a6c0 2023-11-08T16:42:44.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb7337fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb73406ee90 0x7fb7340cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:44.169 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb7337fe640 1 --2- >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb73406ee90 0x7fb7340cb470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.81:3300/0 says I am v2:172.21.15.81:34700/0 (socket says 172.21.15.81:34700) 2023-11-08T16:42:44.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb7337fe640 1 -- 172.21.15.81:0/2292057430 learned_addr learned my addr 172.21.15.81:0/2292057430 (peer_addr_for_me v2:172.21.15.81:0/0) 2023-11-08T16:42:44.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb7337fe640 1 -- 172.21.15.81:0/2292057430 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb73406a6c0 msgr2=0x7fb7340c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:44.170 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb7337fe640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb73406a6c0 0x7fb7340c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:44.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb7337fe640 1 -- 172.21.15.81:0/2292057430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb73406a2e0 msgr2=0x7fb7340c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:44.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.160+0000 7fb732ffd640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb73406a2e0 0x7fb7340c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:44.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.161+0000 7fb7337fe640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb73406a2e0 0x7fb7340c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:44.171 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.161+0000 7fb7337fe640 1 -- 172.21.15.81:0/2292057430 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb71c0094d0 con 0x7fb73406ee90 2023-11-08T16:42:44.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.161+0000 7fb7337fe640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb73406ee90 0x7fb7340cb470 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00c390 tx=0x7fb72c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:44.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.161+0000 7fb72b7fe640 1 -- 172.21.15.81:0/2292057430 <== mon.0 v2:172.21.15.81:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c0035f0 con 0x7fb73406ee90 2023-11-08T16:42:44.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.161+0000 7fb72b7fe640 1 -- 172.21.15.81:0/2292057430 <== mon.0 v2:172.21.15.81:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fb72c003750 con 0x7fb73406ee90 2023-11-08T16:42:44.172 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.161+0000 7fb72b7fe640 1 -- 172.21.15.81:0/2292057430 <== mon.0 v2:172.21.15.81:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c003b20 con 0x7fb73406ee90 2023-11-08T16:42:44.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.161+0000 7fb739724640 1 -- 172.21.15.81:0/2292057430 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7340c57f0 con 0x7fb73406ee90 2023-11-08T16:42:44.173 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.161+0000 7fb739724640 1 -- 172.21.15.81:0/2292057430 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7340c5db0 con 0x7fb73406ee90 2023-11-08T16:42:44.174 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.163+0000 7fb739724640 1 -- 172.21.15.81:0/2292057430 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7340c6290 con 0x7fb73406ee90 2023-11-08T16:42:44.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.163+0000 7fb72b7fe640 1 -- 172.21.15.81:0/2292057430 <== mon.0 v2:172.21.15.81:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb72c003c80 con 0x7fb73406ee90 2023-11-08T16:42:44.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.164+0000 7fb72b7fe640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb71406ae70 0x7fb71406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-08T16:42:44.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.164+0000 7fb72b7fe640 1 -- 172.21.15.81:0/2292057430 <== mon.0 v2:172.21.15.81:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fb72c088790 con 0x7fb73406ee90 2023-11-08T16:42:44.175 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.165+0000 7fb732ffd640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb71406ae70 0x7fb71406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-08T16:42:44.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.165+0000 7fb732ffd640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb71406ae70 0x7fb71406d330 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb7341e61e0 tx=0x7fb724008450 comp rx=0 tx=0).ready entity=mgr.4101 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-08T16:42:44.176 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.169+0000 7fb72b7fe640 1 -- 172.21.15.81:0/2292057430 <== mon.0 v2:172.21.15.81:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179335 (secure 0 0 0) 0x7fb72c053960 con 0x7fb73406ee90 2023-11-08T16:42:44.395 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.394+0000 7fb739724640 1 -- 172.21.15.81:0/2292057430 --> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb734074440 con 0x7fb73406ee90 2023-11-08T16:42:44.396 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.395+0000 7fb72b7fe640 1 -- 172.21.15.81:0/2292057430 <== mon.0 v2:172.21.15.81:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+3701 (secure 0 0 0) 0x7fb72c017090 con 0x7fb73406ee90 2023-11-08T16:42:44.397 INFO:teuthology.orchestra.run.smithi081.stdout: 2023-11-08T16:42:44.397 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":122,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5804,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6834/3393468607","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":3393468607},{"type":"v1","addr":"172.21.15.93:6834","nonce":3393468607}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":6445,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/329260749","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6833","nonce":329260749},{"type":"v1","addr":"172.21.15.93:6835","nonce":329260749}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":109},{"gid":8553,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6835/1914299737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6834","nonce":1914299737},{"type":"v1","addr":"172.21.15.81:6835","nonce":1914299737}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":103},{"gid":9210,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.81:6837/2092593611","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6836","nonce":2092593611},{"type":"v1","addr":"172.21.15.81:6837","nonce":2092593611}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":122}],"filesystems":[]} 2023-11-08T16:42:44.398 INFO:teuthology.orchestra.run.smithi081.stderr:dumped fsmap epoch 122 2023-11-08T16:42:44.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.397+0000 7fb739724640 1 -- 172.21.15.81:0/2292057430 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb71406ae70 msgr2=0x7fb71406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:44.399 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.397+0000 7fb739724640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb71406ae70 0x7fb71406d330 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb7341e61e0 tx=0x7fb724008450 comp rx=0 tx=0).stop 2023-11-08T16:42:44.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.397+0000 7fb739724640 1 -- 172.21.15.81:0/2292057430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb73406ee90 msgr2=0x7fb7340cb470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-08T16:42:44.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.397+0000 7fb739724640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb73406ee90 0x7fb7340cb470 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00c390 tx=0x7fb72c00c850 comp rx=0 tx=0).stop 2023-11-08T16:42:44.400 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.397+0000 7fb739724640 1 -- 172.21.15.81:0/2292057430 shutdown_connections 2023-11-08T16:42:44.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.397+0000 7fb739724640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.81:6832/3984324735,v1:172.21.15.81:6833/3984324735] conn(0x7fb71406ae70 0x7fb71406d330 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:44.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.397+0000 7fb739724640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] conn(0x7fb73406ee90 0x7fb7340cb470 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:44.401 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.398+0000 7fb739724640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.93:3301/0,v1:172.21.15.93:6790/0] conn(0x7fb73406a6c0 0x7fb7340c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:44.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.398+0000 7fb739724640 1 --2- 172.21.15.81:0/2292057430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb73406a2e0 0x7fb7340c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-08T16:42:44.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.398+0000 7fb739724640 1 -- 172.21.15.81:0/2292057430 >> 172.21.15.81:0/2292057430 conn(0x7fb7340cc6d0 msgr2=0x7fb734071400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-08T16:42:44.402 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.398+0000 7fb739724640 1 -- 172.21.15.81:0/2292057430 shutdown_connections 2023-11-08T16:42:44.403 INFO:teuthology.orchestra.run.smithi081.stderr:2023-11-08T16:42:44.398+0000 7fb739724640 1 -- 172.21.15.81:0/2292057430 wait complete. 2023-11-08T16:42:44.417 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-08T16:42:44.418 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-08T16:42:44.418 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-08T16:42:44.419 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:44.435 INFO:tasks.ceph.mds.a:Stopped 2023-11-08T16:42:44.435 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-08T16:42:44.436 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:44.496 INFO:tasks.ceph.mds.c:Stopped 2023-11-08T16:42:44.497 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-08T16:42:44.497 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:44.519 INFO:tasks.ceph.mds.b:Stopped 2023-11-08T16:42:44.520 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-08T16:42:44.520 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:44.580 INFO:tasks.ceph.mds.d:Stopped 2023-11-08T16:42:44.581 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-08T16:42:44.582 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-08T16:42:44.582 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:44.652 INFO:tasks.ceph.osd.0:Stopped 2023-11-08T16:42:44.653 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-08T16:42:44.653 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:44.711 INFO:tasks.ceph.osd.1:Stopped 2023-11-08T16:42:44.711 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-08T16:42:44.712 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:44.772 INFO:tasks.ceph.osd.2:Stopped 2023-11-08T16:42:44.772 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-08T16:42:44.773 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:44.835 INFO:tasks.ceph.osd.3:Stopped 2023-11-08T16:42:44.836 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-08T16:42:44.836 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:44.911 INFO:tasks.ceph.osd.4:Stopped 2023-11-08T16:42:44.912 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-08T16:42:44.912 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:44.971 INFO:tasks.ceph.osd.5:Stopped 2023-11-08T16:42:44.971 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-08T16:42:44.972 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:45.038 INFO:tasks.ceph.osd.6:Stopped 2023-11-08T16:42:45.038 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-08T16:42:45.039 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:45.121 INFO:tasks.ceph.osd.7:Stopped 2023-11-08T16:42:45.122 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-08T16:42:45.122 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-08T16:42:45.122 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:45.201 INFO:tasks.ceph.mgr.y:Stopped 2023-11-08T16:42:45.202 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-08T16:42:45.202 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:45.277 INFO:tasks.ceph.mgr.x:Stopped 2023-11-08T16:42:45.278 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-08T16:42:45.278 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-08T16:42:45.279 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:45.331 INFO:tasks.ceph.mon.a:Stopped 2023-11-08T16:42:45.332 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-08T16:42:45.332 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:45.364 INFO:tasks.ceph.mon.b:Stopped 2023-11-08T16:42:45.365 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-08T16:42:45.365 INFO:teuthology.orchestra.run:waiting for 300 2023-11-08T16:42:45.435 INFO:tasks.ceph.mon.c:Stopped 2023-11-08T16:42:45.436 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-08T16:42:45.436 DEBUG:teuthology.orchestra.run.smithi081:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v 'inconsistent rstat on inode' | egrep -v 'Error recovering journal' | head -n 1 2023-11-08T16:42:45.551 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:42:45.551 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-08T16:42:45.787 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:42:45.788 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-08T16:42:45.952 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:42:45.953 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-08T16:42:46.092 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:42:46.092 DEBUG:teuthology.orchestra.run.smithi081:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-08T16:42:46.262 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:42:46.263 DEBUG:teuthology.orchestra.run.smithi093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-08T16:42:46.502 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:42:46.502 DEBUG:teuthology.orchestra.run.smithi093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-08T16:42:46.653 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:42:46.654 DEBUG:teuthology.orchestra.run.smithi093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-08T16:42:46.806 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:42:46.806 DEBUG:teuthology.orchestra.run.smithi093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-08T16:42:46.974 INFO:tasks.ceph:Archiving mon data... 2023-11-08T16:42:46.976 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595/data/mon.a.tgz 2023-11-08T16:42:46.976 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2023-11-08T16:42:47.036 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.D9JTG2YrYD 2023-11-08T16:42:47.037 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.D9JTG2YrYD 2023-11-08T16:42:47.244 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.D9JTG2YrYD 2023-11-08T16:42:47.388 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.D9JTG2YrYD is 456KB 2023-11-08T16:42:47.499 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.D9JTG2YrYD 2023-11-08T16:42:47.560 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595/data/mon.b.tgz 2023-11-08T16:42:47.560 DEBUG:teuthology.orchestra.run.smithi093:> mktemp 2023-11-08T16:42:47.619 INFO:teuthology.orchestra.run.smithi093.stdout:/tmp/tmp.FGDtj30SlD 2023-11-08T16:42:47.620 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.FGDtj30SlD 2023-11-08T16:42:47.836 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 0666 /tmp/tmp.FGDtj30SlD 2023-11-08T16:42:47.993 DEBUG:teuthology.orchestra.remote:smithi093:/tmp/tmp.FGDtj30SlD is 467KB 2023-11-08T16:42:48.109 DEBUG:teuthology.orchestra.run.smithi093:> rm -fr /tmp/tmp.FGDtj30SlD 2023-11-08T16:42:48.168 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595/data/mon.c.tgz 2023-11-08T16:42:48.169 DEBUG:teuthology.orchestra.run.smithi093:> mktemp 2023-11-08T16:42:48.223 INFO:teuthology.orchestra.run.smithi093.stdout:/tmp/tmp.gA1GLZy3jf 2023-11-08T16:42:48.224 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.gA1GLZy3jf 2023-11-08T16:42:48.422 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 0666 /tmp/tmp.gA1GLZy3jf 2023-11-08T16:42:48.566 DEBUG:teuthology.orchestra.remote:smithi093:/tmp/tmp.gA1GLZy3jf is 467KB 2023-11-08T16:42:48.680 DEBUG:teuthology.orchestra.run.smithi093:> rm -fr /tmp/tmp.gA1GLZy3jf 2023-11-08T16:42:48.739 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-08T16:42:48.740 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-08T16:42:48.750 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-08T16:42:48.842 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-08T16:42:48.843 DEBUG:teuthology.orchestra.run.smithi081:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-08T16:42:48.849 DEBUG:teuthology.orchestra.run.smithi093:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-08T16:42:48.943 INFO:teuthology.orchestra.run.smithi081.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-08T16:42:48.947 INFO:teuthology.orchestra.run.smithi093.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-08T16:42:48.960 INFO:tasks.ceph:Archiving crash dumps... 2023-11-08T16:42:48.963 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595/remote/smithi081/crash 2023-11-08T16:42:48.965 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-08T16:42:49.051 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595/remote/smithi093/crash 2023-11-08T16:42:49.053 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-08T16:42:49.146 INFO:tasks.ceph:Shutting down logrotate 2023-11-08T16:42:49.146 DEBUG:teuthology.orchestra.run.smithi081:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:42:49.156 DEBUG:teuthology.orchestra.run.smithi093:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-08T16:42:49.279 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-08T16:42:49.378 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-08T16:42:49.482 INFO:tasks.ceph:Compressing logs... 2023-11-08T16:42:49.482 DEBUG:teuthology.orchestra.run.smithi081:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-08T16:42:49.491 DEBUG:teuthology.orchestra.run.smithi093:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-08T16:42:49.581 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115428.log 2023-11-08T16:42:49.582 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120291.log 2023-11-08T16:42:49.583 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111976.log 2023-11-08T16:42:49.585 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98348.log 2023-11-08T16:42:49.585 INFO:teuthology.orchestra.run.smithi081.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.115428.log.gz 2023-11-08T16:42:49.585 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120291.log.gz 2023-11-08T16:42:49.586 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111976.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.111976.log.gz 2023-11-08T16:42:49.586 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126848.log 2023-11-08T16:42:49.587 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94853.log 2023-11-08T16:42:49.588 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98348.log.gz 2023-11-08T16:42:49.588 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-08T16:42:49.589 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113156.log 2023-11-08T16:42:49.590 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126848.log.gz 2023-11-08T16:42:49.590 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111756.log 2023-11-08T16:42:49.590 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94853.log.gz 2023-11-08T16:42:49.592 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106948.log 2023-11-08T16:42:49.592 INFO:teuthology.orchestra.run.smithi081.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.113156.log.gz 2023-11-08T16:42:49.593 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102851.log 2023-11-08T16:42:49.593 INFO:teuthology.orchestra.run.smithi081.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111756.log.gz 2023-11-08T16:42:49.594 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.93798.log 2023-11-08T16:42:49.594 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87397.log 2023-11-08T16:42:49.595 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-client.3.93798.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-08T16:42:49.595 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-client.admin.87397.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-08T16:42:49.595 INFO:teuthology.orchestra.run.smithi093.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87397.log.gz 2023-11-08T16:42:49.596 INFO:teuthology.orchestra.run.smithi093.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.3.93798.log.gz 2023-11-08T16:42:49.596 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-08T16:42:49.597 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87437.log 2023-11-08T16:42:49.598 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87637.log 2023-11-08T16:42:49.600 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129549.log 2023-11-08T16:42:49.600 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106948.log.gz 2023-11-08T16:42:49.600 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108592.log 2023-11-08T16:42:49.600 INFO:teuthology.orchestra.run.smithi081.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.102851.log.gz 2023-11-08T16:42:49.601 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129549.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-08T16:42:49.601 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129549.log.gz 2023-11-08T16:42:49.601 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108592.log: gzip 88.8% -5 -- replaced with /var/log/ceph/ceph-client.admin.108592.log.gz --verbose 2023-11-08T16:42:49.602 INFO:teuthology.orchestra.run.smithi081.stderr: -- /var/log/ceph/ceph-client.admin.105888.log 2023-11-08T16:42:49.602 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102425.log 2023-11-08T16:42:49.602 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99115.log 2023-11-08T16:42:49.603 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105888.log.gz 2023-11-08T16:42:49.603 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123064.log 2023-11-08T16:42:49.603 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102425.log.gz 2023-11-08T16:42:49.606 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-client.admin.87437.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-08T16:42:49.607 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose 87.9% -- -- replaced with /var/log/ceph/ceph-client.admin.87437.log.gz /var/log/ceph/ceph-osd.7.log 2023-11-08T16:42:49.607 INFO:teuthology.orchestra.run.smithi093.stderr: 2023-11-08T16:42:49.607 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-client.admin.87637.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-08T16:42:49.607 INFO:teuthology.orchestra.run.smithi093.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87637.log.gz 2023-11-08T16:42:49.608 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87597.log 2023-11-08T16:42:49.608 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-08T16:42:49.608 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-client.admin.87597.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87597.log.gz 2023-11-08T16:42:49.608 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87557.log 2023-11-08T16:42:49.611 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121312.log 2023-11-08T16:42:49.611 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99115.log.gz 2023-11-08T16:42:49.612 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120375.log 2023-11-08T16:42:49.612 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123064.log.gz 2023-11-08T16:42:49.612 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113934.log 2023-11-08T16:42:49.612 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121312.log.gz 2023-11-08T16:42:49.613 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111225.log 2023-11-08T16:42:49.613 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120375.log.gz 2023-11-08T16:42:49.613 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113934.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113934.log.gz 2023-11-08T16:42:49.614 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106066.log 2023-11-08T16:42:49.614 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115296.log 2023-11-08T16:42:49.614 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111225.log.gz 2023-11-08T16:42:49.614 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101261.log 2023-11-08T16:42:49.615 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106066.log.gz 2023-11-08T16:42:49.615 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124622.log 2023-11-08T16:42:49.615 INFO:teuthology.orchestra.run.smithi081.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.115296.log.gz 2023-11-08T16:42:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101261.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101261.log.gz 2023-11-08T16:42:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98575.log 2023-11-08T16:42:49.616 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126121.log 2023-11-08T16:42:49.616 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124622.log.gz 2023-11-08T16:42:49.623 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114245.log 2023-11-08T16:42:49.623 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98575.log.gz 2023-11-08T16:42:49.624 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119500.log 2023-11-08T16:42:49.624 INFO:teuthology.orchestra.run.smithi081.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.126121.log.gz 2023-11-08T16:42:49.624 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114245.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114245.log.gz 2023-11-08T16:42:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128297.log 2023-11-08T16:42:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.119500.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.81321.log 2023-11-08T16:42:49.625 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119500.log.gz 2023-11-08T16:42:49.625 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97003.log 2023-11-08T16:42:49.626 INFO:teuthology.orchestra.run.smithi081.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.128297.log.gz 2023-11-08T16:42:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph.tmp-client.admin.81321.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.81321.log.gz 2023-11-08T16:42:49.626 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100399.log 2023-11-08T16:42:49.627 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126207.log 2023-11-08T16:42:49.627 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97003.log.gz 2023-11-08T16:42:49.627 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97370.log 2023-11-08T16:42:49.627 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100399.log.gz 2023-11-08T16:42:49.628 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116741.log 2023-11-08T16:42:49.628 INFO:teuthology.orchestra.run.smithi081.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.126207.log.gz 2023-11-08T16:42:49.628 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127574.log 2023-11-08T16:42:49.629 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97370.log.gz 2023-11-08T16:42:49.629 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.116741.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.116741.log.gz 2023-11-08T16:42:49.629 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108676.log 2023-11-08T16:42:49.632 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.87557.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87557.log.gz 2023-11-08T16:42:49.632 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87677.log 2023-11-08T16:42:49.632 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-client.admin.87677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.93294.log 2023-11-08T16:42:49.633 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-client.1.93294.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.87677.log.gz 2023-11-08T16:42:49.633 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-08T16:42:49.633 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87517.log 2023-11-08T16:42:49.634 INFO:teuthology.orchestra.run.smithi093.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-08T16:42:49.634 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose 90.6% -- replaced with /var/log/ceph/ceph-client.1.93294.log.gz 2023-11-08T16:42:49.634 INFO:teuthology.orchestra.run.smithi093.stderr: -- /var/log/ceph/ceph-mon.b.log 2023-11-08T16:42:49.634 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.87517.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.87517.log.gz 2023-11-08T16:42:49.635 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-08T16:42:49.635 INFO:teuthology.orchestra.run.smithi093.stderr: 92.3% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-08T16:42:49.635 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87477.log 2023-11-08T16:42:49.636 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-client.admin.87477.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87477.log.gz 2023-11-08T16:42:49.636 INFO:teuthology.orchestra.run.smithi093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-08T16:42:49.640 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126990.log 2023-11-08T16:42:49.641 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127574.log.gz 2023-11-08T16:42:49.641 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108873.log 2023-11-08T16:42:49.641 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108676.log.gz 2023-11-08T16:42:49.642 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118196.log 2023-11-08T16:42:49.642 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.126990.log.gz 2023-11-08T16:42:49.642 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127032.log 2023-11-08T16:42:49.642 INFO:teuthology.orchestra.run.smithi081.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.108873.log.gz 2023-11-08T16:42:49.643 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97800.log 2023-11-08T16:42:49.643 INFO:teuthology.orchestra.run.smithi081.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118196.log.gz 2023-11-08T16:42:49.643 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98947.log 2023-11-08T16:42:49.644 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127032.log.gz 2023-11-08T16:42:49.644 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114852.log 2023-11-08T16:42:49.644 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97800.log.gz 2023-11-08T16:42:49.644 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105718.log 2023-11-08T16:42:49.645 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98947.log.gz 2023-11-08T16:42:49.645 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98821.log 2023-11-08T16:42:49.645 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105718.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.114852.log.gz 2023-11-08T16:42:49.646 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111183.log 2023-11-08T16:42:49.646 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105718.log.gz 2023-11-08T16:42:49.646 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102140.log 2023-11-08T16:42:49.646 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98821.log.gz 2023-11-08T16:42:49.647 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96457.log 2023-11-08T16:42:49.647 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111183.log.gz 2023-11-08T16:42:49.647 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92815.log 2023-11-08T16:42:49.648 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102140.log.gz 2023-11-08T16:42:49.648 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118452.log 2023-11-08T16:42:49.648 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96457.log.gz 2023-11-08T16:42:49.648 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.92815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97045.log 2023-11-08T16:42:49.649 INFO:teuthology.orchestra.run.smithi081.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.92815.log.gz 2023-11-08T16:42:49.649 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118452.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118452.log.gz 2023-11-08T16:42:49.649 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118536.log 2023-11-08T16:42:49.653 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120107.log 2023-11-08T16:42:49.653 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97045.log.gz 2023-11-08T16:42:49.654 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122659.log 2023-11-08T16:42:49.654 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118536.log.gz 2023-11-08T16:42:49.654 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97758.log 2023-11-08T16:42:49.655 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120107.log.gz 2023-11-08T16:42:49.655 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124328.log 2023-11-08T16:42:49.655 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.122659.log.gz 2023-11-08T16:42:49.655 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115021.log 2023-11-08T16:42:49.656 INFO:teuthology.orchestra.run.smithi081.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97758.log.gz 2023-11-08T16:42:49.656 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128534.log 2023-11-08T16:42:49.656 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124328.log.gz 2023-11-08T16:42:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115021.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115021.log.gz 2023-11-08T16:42:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120802.log 2023-11-08T16:42:49.657 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120417.log 2023-11-08T16:42:49.657 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.128534.log.gz 2023-11-08T16:42:49.658 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112069.log 2023-11-08T16:42:49.658 INFO:teuthology.orchestra.run.smithi081.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.120802.log.gz 2023-11-08T16:42:49.658 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105335.log 2023-11-08T16:42:49.659 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120417.log.gz 2023-11-08T16:42:49.659 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112069.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112069.log.gz 2023-11-08T16:42:49.659 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121691.log 2023-11-08T16:42:49.663 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104365.log 2023-11-08T16:42:49.663 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105335.log.gz 2023-11-08T16:42:49.663 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114573.log 2023-11-08T16:42:49.664 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121691.log.gz 2023-11-08T16:42:49.664 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102803.log 2023-11-08T16:42:49.664 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104365.log.gz 2023-11-08T16:42:49.665 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98264.log 2023-11-08T16:42:49.665 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114573.log.gz 2023-11-08T16:42:49.665 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93323.log 2023-11-08T16:42:49.665 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98264.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.102803.log.gz 2023-11-08T16:42:49.666 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108944.log 2023-11-08T16:42:49.666 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98264.log.gz 2023-11-08T16:42:49.666 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98653.log 2023-11-08T16:42:49.667 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93323.log.gz 2023-11-08T16:42:49.667 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95573.log 2023-11-08T16:42:49.667 INFO:teuthology.orchestra.run.smithi081.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.108944.log.gz 2023-11-08T16:42:49.667 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100779.log 2023-11-08T16:42:49.668 INFO:teuthology.orchestra.run.smithi081.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98653.log.gz 2023-11-08T16:42:49.670 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.95573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129873.log 2023-11-08T16:42:49.670 INFO:teuthology.orchestra.run.smithi081.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95573.log.gz 2023-11-08T16:42:49.670 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118938.log 2023-11-08T16:42:49.671 INFO:teuthology.orchestra.run.smithi081.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.100779.log.gz 2023-11-08T16:42:49.671 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123250.log 2023-11-08T16:42:49.671 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129873.log.gz 2023-11-08T16:42:49.672 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113072.log 2023-11-08T16:42:49.672 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118938.log.gz 2023-11-08T16:42:49.672 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123250.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.123250.log.gz -5 2023-11-08T16:42:49.672 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.125665.log 2023-11-08T16:42:49.673 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111351.log 2023-11-08T16:42:49.673 INFO:teuthology.orchestra.run.smithi081.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.113072.log.gz 2023-11-08T16:42:49.675 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123773.log 2023-11-08T16:42:49.675 INFO:teuthology.orchestra.run.smithi081.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.125665.log.gz 2023-11-08T16:42:49.676 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123605.log 2023-11-08T16:42:49.676 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111351.log.gz 2023-11-08T16:42:49.676 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123773.log: gzip 87.9% -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.123773.log.gz /var/log/ceph/ceph-client.admin.129339.log 2023-11-08T16:42:49.676 INFO:teuthology.orchestra.run.smithi081.stderr: 2023-11-08T16:42:49.677 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126027.log 2023-11-08T16:42:49.677 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123605.log.gz 2023-11-08T16:42:49.677 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92987.log 2023-11-08T16:42:49.678 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.129339.log.gz 2023-11-08T16:42:49.679 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126027.log: 86.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.126027.log.gz -5 2023-11-08T16:42:49.679 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.98863.log 2023-11-08T16:42:49.680 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.92987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97968.log 2023-11-08T16:42:49.680 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92987.log.gz 2023-11-08T16:42:49.680 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117458.log 2023-11-08T16:42:49.681 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98863.log.gz 2023-11-08T16:42:49.681 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99619.log 2023-11-08T16:42:49.682 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.97968.log.gz 2023-11-08T16:42:49.683 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117164.log 2023-11-08T16:42:49.683 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117458.log.gz 2023-11-08T16:42:49.683 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106254.log 2023-11-08T16:42:49.684 INFO:teuthology.orchestra.run.smithi081.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99619.log.gz 2023-11-08T16:42:49.684 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117164.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117164.log.gz 2023-11-08T16:42:49.684 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125547.log 2023-11-08T16:42:49.685 INFO:teuthology.orchestra.run.smithi093.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.c.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-08T16:42:49.686 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121125.log 2023-11-08T16:42:49.687 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106254.log.gz 2023-11-08T16:42:49.687 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122980.log 2023-11-08T16:42:49.687 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125547.log.gz 2023-11-08T16:42:49.687 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118154.log 2023-11-08T16:42:49.688 INFO:teuthology.orchestra.run.smithi081.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.121125.log.gz 2023-11-08T16:42:49.689 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122980.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.122980.log.gz 2023-11-08T16:42:49.689 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96961.log 2023-11-08T16:42:49.689 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118154.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118154.log.gz 2023-11-08T16:42:49.690 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99493.log 2023-11-08T16:42:49.691 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115212.log 2023-11-08T16:42:49.691 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96961.log.gz 2023-11-08T16:42:49.691 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96373.log 2023-11-08T16:42:49.692 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99493.log.gz 2023-11-08T16:42:49.693 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115212.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108162.log 2023-11-08T16:42:49.693 INFO:teuthology.orchestra.run.smithi081.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.115212.log.gz 2023-11-08T16:42:49.693 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96373.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96373.log.gz 2023-11-08T16:42:49.694 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94727.log 2023-11-08T16:42:49.695 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87651.log 2023-11-08T16:42:49.695 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108162.log.gz 2023-11-08T16:42:49.696 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108321.log 2023-11-08T16:42:49.696 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94727.log.gz 2023-11-08T16:42:49.697 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.87651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125421.log 2023-11-08T16:42:49.697 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87651.log.gz 2023-11-08T16:42:49.698 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102299.log 2023-11-08T16:42:49.698 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125421.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.108321.log.gz 2023-11-08T16:42:49.699 INFO:teuthology.orchestra.run.smithi081.stderr: 89.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.125421.log.gz 2023-11-08T16:42:49.700 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.110506.log 2023-11-08T16:42:49.700 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112894.log 2023-11-08T16:42:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.110506.log: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.102299.log.gz 2023-11-08T16:42:49.701 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124242.log 2023-11-08T16:42:49.701 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.110506.log.gz 2023-11-08T16:42:49.702 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103691.log 2023-11-08T16:42:49.703 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112894.log.gz 2023-11-08T16:42:49.704 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124242.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124242.log.gzgzip 2023-11-08T16:42:49.704 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96095.log 2023-11-08T16:42:49.704 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.0.103691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127217.log 2023-11-08T16:42:49.705 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92419.log 2023-11-08T16:42:49.706 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96095.log.gz 2023-11-08T16:42:49.706 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127436.log 2023-11-08T16:42:49.707 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.92419.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.92419.log.gz 2023-11-08T16:42:49.708 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102612.log 2023-11-08T16:42:49.708 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127436.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.127217.log.gz 2023-11-08T16:42:49.709 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129591.log 2023-11-08T16:42:49.709 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.127436.log.gz 2023-11-08T16:42:49.710 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104281.log 2023-11-08T16:42:49.711 INFO:teuthology.orchestra.run.smithi081.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.102612.log.gz 2023-11-08T16:42:49.712 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117038.log 2023-11-08T16:42:49.712 INFO:teuthology.orchestra.run.smithi081.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.103691.log.gz 2023-11-08T16:42:49.712 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.129591.log.gz 2023-11-08T16:42:49.713 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104281.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104281.log.gz 2023-11-08T16:42:49.713 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126165.log 2023-11-08T16:42:49.715 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113114.log 2023-11-08T16:42:49.715 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117038.log.gz 2023-11-08T16:42:49.715 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100611.log 2023-11-08T16:42:49.715 INFO:teuthology.orchestra.run.smithi081.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.126165.log.gz 2023-11-08T16:42:49.716 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113028.log 2023-11-08T16:42:49.717 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113114.log.gz 2023-11-08T16:42:49.718 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100611.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100611.log.gz 2023-11-08T16:42:49.718 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97285.log 2023-11-08T16:42:49.718 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99157.log 2023-11-08T16:42:49.719 INFO:teuthology.orchestra.run.smithi081.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113028.log.gz 2023-11-08T16:42:49.720 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105167.log 2023-11-08T16:42:49.720 INFO:teuthology.orchestra.run.smithi081.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97285.log.gz 2023-11-08T16:42:49.721 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129465.log 2023-11-08T16:42:49.721 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99157.log.gz 2023-11-08T16:42:49.722 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105167.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105167.log.gz 2023-11-08T16:42:49.722 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92503.log 2023-11-08T16:42:49.723 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129465.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.129465.log.gz 2023-11-08T16:42:49.723 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.112027.log 2023-11-08T16:42:49.724 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.92503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121251.log 2023-11-08T16:42:49.725 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.92503.log.gz 2023-11-08T16:42:49.725 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93627.log 2023-11-08T16:42:49.725 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112027.log.gz 2023-11-08T16:42:49.726 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115799.log 2023-11-08T16:42:49.727 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120207.log 2023-11-08T16:42:49.727 INFO:teuthology.orchestra.run.smithi081.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.93627.log.gz 2023-11-08T16:42:49.728 INFO:teuthology.orchestra.run.smithi081.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.121251.log.gz 2023-11-08T16:42:49.729 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117080.log 2023-11-08T16:42:49.729 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115799.log.gz 2023-11-08T16:42:49.730 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120207.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120207.log.gz 2023-11-08T16:42:49.731 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123292.log 2023-11-08T16:42:49.731 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99883.log 2023-11-08T16:42:49.731 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117080.log.gz 2023-11-08T16:42:49.732 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108928.log 2023-11-08T16:42:49.732 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123292.log.gz 2023-11-08T16:42:49.733 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104197.log 2023-11-08T16:42:49.733 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99883.log.gz 2023-11-08T16:42:49.734 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124748.log 2023-11-08T16:42:49.735 INFO:teuthology.orchestra.run.smithi081.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.108928.log.gz 2023-11-08T16:42:49.735 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98053.log 2023-11-08T16:42:49.735 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104197.log.gz 2023-11-08T16:42:49.736 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98533.log 2023-11-08T16:42:49.737 INFO:teuthology.orchestra.run.smithi081.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.124748.log.gz 2023-11-08T16:42:49.737 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98053.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98053.log.gz 2023-11-08T16:42:49.737 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127530.log 2023-11-08T16:42:49.738 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108634.log 2023-11-08T16:42:49.738 INFO:teuthology.orchestra.run.smithi081.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98533.log.gz 2023-11-08T16:42:49.739 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124412.log 2023-11-08T16:42:49.740 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.127530.log.gz 2023-11-08T16:42:49.741 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109564.log 2023-11-08T16:42:49.741 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108634.log.gz 2023-11-08T16:42:49.742 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124412.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.124412.log.gz 2023-11-08T16:42:49.742 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124538.log 2023-11-08T16:42:49.743 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129676.log 2023-11-08T16:42:49.743 INFO:teuthology.orchestra.run.smithi081.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.109564.log.gz 2023-11-08T16:42:49.744 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117603.log 2023-11-08T16:42:49.744 INFO:teuthology.orchestra.run.smithi081.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124538.log.gz 2023-11-08T16:42:49.745 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129676.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.129676.log.gz 2023-11-08T16:42:49.745 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98010.log 2023-11-08T16:42:49.746 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96289.log 2023-11-08T16:42:49.746 INFO:teuthology.orchestra.run.smithi081.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117603.log.gz 2023-11-08T16:42:49.747 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119458.log 2023-11-08T16:42:49.748 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98010.log.gz 2023-11-08T16:42:49.748 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123394.log 2023-11-08T16:42:49.748 INFO:teuthology.orchestra.run.smithi081.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.96289.log.gz 2023-11-08T16:42:49.750 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.119458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125749.log 2023-11-08T16:42:49.750 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119458.log.gz 2023-11-08T16:42:49.750 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123394.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.123394.log.gz 2023-11-08T16:42:49.751 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126948.log 2023-11-08T16:42:49.752 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101584.log 2023-11-08T16:42:49.752 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.125749.log.gz 2023-11-08T16:42:49.752 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112111.log 2023-11-08T16:42:49.753 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.126948.log.gz 2023-11-08T16:42:49.753 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96247.log 2023-11-08T16:42:49.754 INFO:teuthology.orchestra.run.smithi081.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101584.log.gz 2023-11-08T16:42:49.754 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112111.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112111.log.gz 2023-11-08T16:42:49.755 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114403.log 2023-11-08T16:42:49.756 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.95991.log 2023-11-08T16:42:49.756 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96247.log.gz 2023-11-08T16:42:49.757 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114403.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114403.log.gz 2023-11-08T16:42:49.757 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107773.log 2023-11-08T16:42:49.758 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.2.95991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116365.log 2023-11-08T16:42:49.759 INFO:teuthology.orchestra.run.smithi081.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.2.95991.log.gz 2023-11-08T16:42:49.759 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107773.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107773.log.gz 2023-11-08T16:42:49.760 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105802.log 2023-11-08T16:42:49.761 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.116365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123437.log 2023-11-08T16:42:49.761 INFO:teuthology.orchestra.run.smithi081.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.116365.log.gz 2023-11-08T16:42:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105802.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105802.log.gz 2023-11-08T16:42:49.762 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126079.log 2023-11-08T16:42:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123437.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.123437.log.gz 2023-11-08T16:42:49.763 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100357.log 2023-11-08T16:42:49.764 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126079.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.126079.log.gz 2023-11-08T16:42:49.765 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108084.log 2023-11-08T16:42:49.765 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100357.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100357.log.gz 2023-11-08T16:42:49.766 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124580.log 2023-11-08T16:42:49.767 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108084.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108084.log.gz 2023-11-08T16:42:49.767 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117645.log 2023-11-08T16:42:49.767 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124580.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124580.log.gz 2023-11-08T16:42:49.768 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96751.log 2023-11-08T16:42:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117645.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117645.log.gz 2023-11-08T16:42:49.769 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129297.log 2023-11-08T16:42:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96751.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96751.log.gz 2023-11-08T16:42:49.770 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100177.log 2023-11-08T16:42:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129297.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.129297.log.gz 2023-11-08T16:42:49.771 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113709.log 2023-11-08T16:42:49.772 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100177.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100177.log.gz 2023-11-08T16:42:49.773 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102570.log 2023-11-08T16:42:49.773 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.0.113709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123479.log 2023-11-08T16:42:49.774 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102570.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102570.log.gz 2023-11-08T16:42:49.775 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121649.log 2023-11-08T16:42:49.776 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123479.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.123479.log.gz 2023-11-08T16:42:49.776 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93239.log 2023-11-08T16:42:49.777 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121649.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121649.log.gz 2023-11-08T16:42:49.777 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104239.log 2023-11-08T16:42:49.778 INFO:teuthology.orchestra.run.smithi081.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.113709.log.gz 2023-11-08T16:42:49.779 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129633.log 2023-11-08T16:42:49.779 INFO:teuthology.orchestra.run.smithi081.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93239.log.gz 2023-11-08T16:42:49.780 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104239.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.104239.log.gz 2023-11-08T16:42:49.780 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116657.log 2023-11-08T16:42:49.781 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129633.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.129633.log.gz 2023-11-08T16:42:49.781 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104323.log 2023-11-08T16:42:49.782 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.116657.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116657.log.gz 2023-11-08T16:42:49.782 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121539.log 2023-11-08T16:42:49.787 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120249.log 2023-11-08T16:42:49.787 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104323.log.gz 2023-11-08T16:42:49.788 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121539.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.121539.log.gz 2023-11-08T16:42:49.788 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96667.log 2023-11-08T16:42:49.788 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120249.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120249.log.gz 2023-11-08T16:42:49.788 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100653.log 2023-11-08T16:42:49.789 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104896.log 2023-11-08T16:42:49.789 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.96667.log.gz 2023-11-08T16:42:49.790 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100653.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.100653.log.gz 2023-11-08T16:42:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108825.log 2023-11-08T16:42:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104896.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104896.log.gz 2023-11-08T16:42:49.791 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93113.log 2023-11-08T16:42:49.792 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97716.log 2023-11-08T16:42:49.793 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93113.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.108825.log.gz 2023-11-08T16:42:49.794 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.93113.log.gz 2023-11-08T16:42:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116699.log 2023-11-08T16:42:49.794 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97716.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.97716.log.gz 2023-11-08T16:42:49.795 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93620.log 2023-11-08T16:42:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.116699.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.116699.log.gz 2023-11-08T16:42:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122896.log 2023-11-08T16:42:49.796 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111267.log 2023-11-08T16:42:49.797 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93620.log.gz 2023-11-08T16:42:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122896.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122896.log.gz 2023-11-08T16:42:49.798 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97926.log 2023-11-08T16:42:49.799 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111267.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111267.log.gz 2023-11-08T16:42:49.799 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108447.log 2023-11-08T16:42:49.800 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97926.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97926.log.gz 2023-11-08T16:42:49.800 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106990.log 2023-11-08T16:42:49.801 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108447.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108447.log.gz 2023-11-08T16:42:49.801 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108937.log 2023-11-08T16:42:49.802 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106990.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106990.log.gz 2023-11-08T16:42:49.802 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108923.log 2023-11-08T16:42:49.803 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106110.log 2023-11-08T16:42:49.803 INFO:teuthology.orchestra.run.smithi081.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.108937.log.gz 2023-11-08T16:42:49.804 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126890.log 2023-11-08T16:42:49.805 INFO:teuthology.orchestra.run.smithi081.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.108923.log.gz 2023-11-08T16:42:49.805 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106110.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106110.log.gz 2023-11-08T16:42:49.806 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107731.log 2023-11-08T16:42:49.806 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126890.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.126890.log.gz 2023-11-08T16:42:49.807 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102202.log 2023-11-08T16:42:49.808 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118238.log 2023-11-08T16:42:49.808 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107731.log.gz 2023-11-08T16:42:49.809 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102202.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102202.log.gz 2023-11-08T16:42:49.809 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109229.log 2023-11-08T16:42:49.810 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118238.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118238.log.gz 2023-11-08T16:42:49.810 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93960.log 2023-11-08T16:42:49.811 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128870.log 2023-11-08T16:42:49.812 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93960.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.109229.log.gz 2023-11-08T16:42:49.813 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93960.log.gz 2023-11-08T16:42:49.813 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104770.log 2023-11-08T16:42:49.814 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128870.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.128870.log.gz 2023-11-08T16:42:49.814 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93966.log 2023-11-08T16:42:49.815 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104770.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.104770.log.gz 2023-11-08T16:42:49.815 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.100307.log 2023-11-08T16:42:49.816 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93966.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93966.log.gz 2023-11-08T16:42:49.817 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93616.log 2023-11-08T16:42:49.817 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100307.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100307.log.gz 2023-11-08T16:42:49.818 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115512.log 2023-11-08T16:42:49.819 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93616.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93616.log.gz 2023-11-08T16:42:49.819 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108377.log 2023-11-08T16:42:49.820 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113366.log 2023-11-08T16:42:49.821 INFO:teuthology.orchestra.run.smithi081.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.115512.log.gz 2023-11-08T16:42:49.821 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108377.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108377.log.gz 2023-11-08T16:42:49.822 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98222.log 2023-11-08T16:42:49.823 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99241.log 2023-11-08T16:42:49.823 INFO:teuthology.orchestra.run.smithi081.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.113366.log.gz 2023-11-08T16:42:49.824 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98222.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98222.log.gz 2023-11-08T16:42:49.824 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87777.log 2023-11-08T16:42:49.825 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99241.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99241.log.gz 2023-11-08T16:42:49.825 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129999.log 2023-11-08T16:42:49.826 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.87777.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87777.log.gz 2023-11-08T16:42:49.826 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105676.log 2023-11-08T16:42:49.827 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129999.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129999.log.gz 2023-11-08T16:42:49.828 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123948.log 2023-11-08T16:42:49.829 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105676.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105676.log.gz 2023-11-08T16:42:49.829 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129957.log 2023-11-08T16:42:49.830 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123948.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.123948.log.gz 2023-11-08T16:42:49.830 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129099.log 2023-11-08T16:42:49.831 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129957.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.129957.log.gz 2023-11-08T16:42:49.831 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129830.log 2023-11-08T16:42:49.832 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129099.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129099.log.gz 2023-11-08T16:42:49.832 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114161.log 2023-11-08T16:42:49.833 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129830.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.129830.log.gz 2023-11-08T16:42:49.834 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120149.log 2023-11-08T16:42:49.834 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114161.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114161.log.gz 2023-11-08T16:42:49.835 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104602.log 2023-11-08T16:42:49.836 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120149.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120149.log.gz 2023-11-08T16:42:49.836 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128339.log 2023-11-08T16:42:49.837 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104602.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104602.log.gz 2023-11-08T16:42:49.837 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111546.log 2023-11-08T16:42:49.838 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128339.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128339.log.gz 2023-11-08T16:42:49.839 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125463.log 2023-11-08T16:42:49.839 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111546.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111546.log.gz 2023-11-08T16:42:49.840 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121402.log 2023-11-08T16:42:49.840 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125463.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.125463.log.gz 2023-11-08T16:42:49.841 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119372.log 2023-11-08T16:42:49.842 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121402.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121402.log.gz 2023-11-08T16:42:49.842 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93625.log 2023-11-08T16:42:49.843 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.119372.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119372.log.gz 2023-11-08T16:42:49.844 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93630.log 2023-11-08T16:42:49.844 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93625.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93625.log.gz 2023-11-08T16:42:49.845 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-08T16:42:49.846 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93630.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93630.log.gz 2023-11-08T16:42:49.846 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103372.log 2023-11-08T16:42:49.847 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.116226.log 2023-11-08T16:42:49.848 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.103372.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103372.log.gz 2023-11-08T16:42:49.848 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125091.log 2023-11-08T16:42:49.849 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.0.116226.log: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-08T16:42:49.850 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105514.log 2023-11-08T16:42:49.850 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.0.125091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118410.log 2023-11-08T16:42:49.851 INFO:teuthology.orchestra.run.smithi081.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.0.116226.log.gz 2023-11-08T16:42:49.852 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105514.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105514.log.gz 2023-11-08T16:42:49.852 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108224.log 2023-11-08T16:42:49.853 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118410.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118410.log.gz 2023-11-08T16:42:49.853 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96835.log 2023-11-08T16:42:49.854 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108224.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108224.log.gz 2023-11-08T16:42:49.855 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129915.log 2023-11-08T16:42:49.856 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96835.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96835.log.gz 2023-11-08T16:42:49.856 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100441.log 2023-11-08T16:42:49.857 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129915.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.129915.log.gz 2023-11-08T16:42:49.857 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119828.log 2023-11-08T16:42:49.858 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100441.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100441.log.gz 2023-11-08T16:42:49.858 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117992.log 2023-11-08T16:42:49.859 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.119828.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119828.log.gz 2023-11-08T16:42:49.860 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120928.log 2023-11-08T16:42:49.860 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117992.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117992.log.gz 2023-11-08T16:42:49.861 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99283.log 2023-11-08T16:42:49.862 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120928.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.120928.log.gz 2023-11-08T16:42:49.862 INFO:teuthology.orchestra.run.smithi081.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.93407.log 2023-11-08T16:42:49.863 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129718.log 2023-11-08T16:42:49.863 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99283.log.gz 2023-11-08T16:42:49.864 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93407.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93407.log.gz 2023-11-08T16:42:49.864 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125505.log 2023-11-08T16:42:49.865 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129718.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.129718.log.gz 2023-11-08T16:42:49.866 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101794.log 2023-11-08T16:42:49.867 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125505.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.125505.log.gz 2023-11-08T16:42:49.867 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130169.log 2023-11-08T16:42:49.868 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101794.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101794.log.gz 2023-11-08T16:42:49.868 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105209.log 2023-11-08T16:42:49.869 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.130169.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.130169.log.gz 2023-11-08T16:42:49.869 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99841.log 2023-11-08T16:42:49.870 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105209.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105209.log.gz 2023-11-08T16:42:49.870 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124496.log 2023-11-08T16:42:49.871 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99841.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99841.log.gz 2023-11-08T16:42:49.872 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116825.log 2023-11-08T16:42:49.872 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124496.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124496.log.gz 2023-11-08T16:42:49.873 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94289.log 2023-11-08T16:42:49.873 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.116825.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116825.log.gz 2023-11-08T16:42:49.874 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129381.log 2023-11-08T16:42:49.875 INFO:teuthology.orchestra.run.smithi093.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-08T16:42:49.875 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94289.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94289.log.gz 2023-11-08T16:42:49.876 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101389.log 2023-11-08T16:42:49.877 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129381.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.129381.log.gz 2023-11-08T16:42:49.877 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120760.log 2023-11-08T16:42:49.878 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101389.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101389.log.gz 2023-11-08T16:42:49.878 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93071.log 2023-11-08T16:42:49.879 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120760.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.120760.log.gz 2023-11-08T16:42:49.879 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94373.log 2023-11-08T16:42:49.880 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93071.log: gzip -5 88.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.93071.log.gz -- 2023-11-08T16:42:49.880 INFO:teuthology.orchestra.run.smithi081.stderr: /var/log/ceph/ceph-client.admin.117897.log 2023-11-08T16:42:49.881 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94373.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94373.log.gz 2023-11-08T16:42:49.881 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99367.log 2023-11-08T16:42:49.882 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117897.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117897.log.gz 2023-11-08T16:42:49.883 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100569.log 2023-11-08T16:42:49.883 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99367.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.99367.log.gz 2023-11-08T16:42:49.884 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120023.log 2023-11-08T16:42:49.885 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100569.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100569.log.gz 2023-11-08T16:42:49.885 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101978.log 2023-11-08T16:42:49.886 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120023.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120023.log.gz 2023-11-08T16:42:49.886 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93281.log 2023-11-08T16:42:49.887 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101978.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.101978.log.gz 2023-11-08T16:42:49.887 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111840.log 2023-11-08T16:42:49.888 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93281.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93281.log.gz 2023-11-08T16:42:49.888 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128702.log 2023-11-08T16:42:49.889 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111840.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111840.log.gz 2023-11-08T16:42:49.890 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99073.log 2023-11-08T16:42:49.890 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128702.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.128702.log.gz 2023-11-08T16:42:49.891 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122575.log 2023-11-08T16:42:49.892 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99073.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99073.log.gz 2023-11-08T16:42:49.892 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124664.log 2023-11-08T16:42:49.893 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122575.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.122575.log.gz 2023-11-08T16:42:49.893 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109832.log 2023-11-08T16:42:49.894 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124664.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124664.log.gz 2023-11-08T16:42:49.894 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97842.log 2023-11-08T16:42:49.895 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94685.log 2023-11-08T16:42:49.896 INFO:teuthology.orchestra.run.smithi081.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.109832.log.gz 2023-11-08T16:42:49.897 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97842.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97842.log.gz 2023-11-08T16:42:49.897 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127194.log 2023-11-08T16:42:49.898 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94685.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94685.log.gz 2023-11-08T16:42:49.898 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119981.log 2023-11-08T16:42:49.899 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127194.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.127194.log.gz 2023-11-08T16:42:49.900 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116515.log 2023-11-08T16:42:49.900 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.119981.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119981.log.gz 2023-11-08T16:42:49.901 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123190.log 2023-11-08T16:42:49.902 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.116515.log: 93.2% -- replaced with /var/log/ceph/ceph-client.0.125091.log.gz 2023-11-08T16:42:49.902 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118662.log 2023-11-08T16:42:49.902 INFO:teuthology.orchestra.run.smithi081.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.116515.log.gz 2023-11-08T16:42:49.903 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123190.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.123190.log.gz 2023-11-08T16:42:49.904 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114615.log 2023-11-08T16:42:49.904 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118662.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118662.log.gz 2023-11-08T16:42:49.905 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105251.log 2023-11-08T16:42:49.906 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114615.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114615.log.gz 2023-11-08T16:42:49.906 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99409.log 2023-11-08T16:42:49.907 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105251.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105251.log.gz 2023-11-08T16:42:49.907 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93967.log 2023-11-08T16:42:49.908 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99409.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99409.log.gz 2023-11-08T16:42:49.908 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124198.log 2023-11-08T16:42:49.909 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93967.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93967.log.gz 2023-11-08T16:42:49.910 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94247.log 2023-11-08T16:42:49.911 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115105.log 2023-11-08T16:42:49.911 INFO:teuthology.orchestra.run.smithi081.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.124198.log.gz 2023-11-08T16:42:49.912 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94247.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94247.log.gz 2023-11-08T16:42:49.912 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93618.log 2023-11-08T16:42:49.913 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115105.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115105.log.gz 2023-11-08T16:42:49.914 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123689.log 2023-11-08T16:42:49.915 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93618.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93618.log.gz 2023-11-08T16:42:49.916 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112722.log 2023-11-08T16:42:49.917 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123689.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.123689.log.gz 2023-11-08T16:42:49.917 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122173.log 2023-11-08T16:42:49.918 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112722.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112722.log.gz 2023-11-08T16:42:49.918 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128744.log 2023-11-08T16:42:49.920 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120065.log 2023-11-08T16:42:49.921 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128744.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.122173.log.gz 2023-11-08T16:42:49.921 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128744.log.gz 2023-11-08T16:42:49.922 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92731.log 2023-11-08T16:42:49.922 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120065.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120065.log.gz 2023-11-08T16:42:49.923 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116867.log 2023-11-08T16:42:49.924 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.92731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100219.log 2023-11-08T16:42:49.924 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92731.log.gz 2023-11-08T16:42:49.925 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.116867.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116867.log.gz 2023-11-08T16:42:49.925 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127380.log 2023-11-08T16:42:49.926 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100219.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100219.log.gz 2023-11-08T16:42:49.926 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114447.log 2023-11-08T16:42:49.927 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127380.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.127380.log.gz 2023-11-08T16:42:49.928 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98737.log 2023-11-08T16:42:49.929 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97554.log 2023-11-08T16:42:49.929 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114447.log.gz 2023-11-08T16:42:49.930 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98737.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98737.log.gz 2023-11-08T16:42:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99799.log 2023-11-08T16:42:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97554.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97554.log.gz 2023-11-08T16:42:49.931 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128828.log 2023-11-08T16:42:49.932 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99799.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99799.log.gz 2023-11-08T16:42:49.933 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130125.log 2023-11-08T16:42:49.934 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128828.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.128828.log.gz 2023-11-08T16:42:49.934 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101122.log 2023-11-08T16:42:49.935 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.130125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114119.log 2023-11-08T16:42:49.935 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130125.log.gz 2023-11-08T16:42:49.936 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.0.101122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111882.log 2023-11-08T16:42:49.937 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114119.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114119.log.gz 2023-11-08T16:42:49.937 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105293.log 2023-11-08T16:42:49.938 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112764.log 2023-11-08T16:42:49.939 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111882.log.gz 2023-11-08T16:42:49.939 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105293.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105293.log.gz 2023-11-08T16:42:49.940 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123148.log 2023-11-08T16:42:49.941 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112850.log 2023-11-08T16:42:49.941 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112764.log.gz 2023-11-08T16:42:49.942 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123148.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123148.log.gz 2023-11-08T16:42:49.943 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115663.log 2023-11-08T16:42:49.943 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104560.log 2023-11-08T16:42:49.944 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.112850.log.gz 2023-11-08T16:42:49.945 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115663.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115663.log.gz 2023-11-08T16:42:49.945 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94183.log 2023-11-08T16:42:49.946 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93029.log 2023-11-08T16:42:49.947 INFO:teuthology.orchestra.run.smithi081.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104560.log.gz 2023-11-08T16:42:49.948 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94183.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94183.log.gz 2023-11-08T16:42:49.948 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107647.log 2023-11-08T16:42:49.949 INFO:teuthology.orchestra.run.smithi081.stderr: 94.5% -- replaced with /var/log/ceph/ceph-client.0.101122.log.gz 2023-11-08T16:42:49.950 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93029.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94331.log 2023-11-08T16:42:49.950 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93029.log.gz 2023-11-08T16:42:49.951 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107647.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107647.log.gz 2023-11-08T16:42:49.951 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105377.log 2023-11-08T16:42:49.951 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94331.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94331.log.gz 2023-11-08T16:42:49.953 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92857.log 2023-11-08T16:42:49.953 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105377.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105377.log.gz 2023-11-08T16:42:49.953 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112153.log 2023-11-08T16:42:49.954 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.92857.log: gzip -5 --verbose 89.3% -- /var/log/ceph/ceph-client.admin.109790.log -- replaced with /var/log/ceph/ceph-client.admin.92857.log.gz 2023-11-08T16:42:49.955 INFO:teuthology.orchestra.run.smithi081.stderr: 2023-11-08T16:42:49.955 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112153.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112153.log.gz 2023-11-08T16:42:49.956 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118872.log 2023-11-08T16:42:49.957 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109790.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.109790.log.gz 2023-11-08T16:42:49.957 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115254.log 2023-11-08T16:42:49.958 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125971.log 2023-11-08T16:42:49.958 INFO:teuthology.orchestra.run.smithi081.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.118872.log.gz 2023-11-08T16:42:49.959 INFO:teuthology.orchestra.run.smithi093.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-08T16:42:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107874.log 2023-11-08T16:42:49.960 INFO:teuthology.orchestra.run.smithi081.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.115254.log.gz 2023-11-08T16:42:49.960 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125971.log: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.125971.log.gz 2023-11-08T16:42:49.961 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109746.log 2023-11-08T16:42:49.962 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107874.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107874.log.gz 2023-11-08T16:42:49.962 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99535.log 2023-11-08T16:42:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109746.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109746.log.gz 2023-11-08T16:42:49.963 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125363.log 2023-11-08T16:42:49.964 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99535.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99535.log.gz 2023-11-08T16:42:49.965 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117248.log 2023-11-08T16:42:49.965 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125363.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125363.log.gz 2023-11-08T16:42:49.965 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93947.log 2023-11-08T16:42:49.966 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117248.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117248.log.gz 2023-11-08T16:42:49.967 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128786.log 2023-11-08T16:42:49.968 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93947.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93947.log.gz 2023-11-08T16:42:49.968 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117332.log 2023-11-08T16:42:49.969 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128786.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.128786.log.gz 2023-11-08T16:42:49.969 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115841.log 2023-11-08T16:42:49.970 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117332.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117332.log.gz 2023-11-08T16:42:49.970 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-08T16:42:49.972 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113848.log 2023-11-08T16:42:49.972 INFO:teuthology.orchestra.run.smithi081.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.115841.log.gz 2023-11-08T16:42:49.972 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128660.log 2023-11-08T16:42:49.973 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113848.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113848.log.gz 2023-11-08T16:42:49.974 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101852.log 2023-11-08T16:42:49.975 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128660.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128660.log.gz 2023-11-08T16:42:49.975 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116431.log 2023-11-08T16:42:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101852.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101852.log.gz 2023-11-08T16:42:49.976 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118072.log 2023-11-08T16:42:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.116431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115573.log 2023-11-08T16:42:49.978 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.116431.log.gz 2023-11-08T16:42:49.978 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118072.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118072.log.gz 2023-11-08T16:42:49.979 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122701.log 2023-11-08T16:42:49.980 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115573.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115573.log.gz 2023-11-08T16:42:49.980 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127682.log 2023-11-08T16:42:49.981 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122701.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122701.log.gz 2023-11-08T16:42:49.981 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111462.log 2023-11-08T16:42:49.982 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127682.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.127682.log.gz 2023-11-08T16:42:49.983 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116783.log 2023-11-08T16:42:49.983 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111462.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111462.log.gz 2023-11-08T16:42:49.983 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93962.log 2023-11-08T16:42:49.984 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.116783.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116783.log.gz 2023-11-08T16:42:49.985 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109211.log 2023-11-08T16:42:49.985 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93962.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93962.log.gz 2023-11-08T16:42:49.986 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92899.log 2023-11-08T16:42:49.987 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99031.log 2023-11-08T16:42:49.987 INFO:teuthology.orchestra.run.smithi081.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.109211.log.gz 2023-11-08T16:42:49.988 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.92899.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92899.log.gz 2023-11-08T16:42:49.988 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96331.log 2023-11-08T16:42:49.989 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99031.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99031.log.gz 2023-11-08T16:42:49.990 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127488.log 2023-11-08T16:42:49.991 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96331.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96331.log.gz 2023-11-08T16:42:49.991 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108915.log 2023-11-08T16:42:49.992 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127488.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.127488.log.gz 2023-11-08T16:42:49.992 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128450.log 2023-11-08T16:42:49.993 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102654.log 2023-11-08T16:42:49.994 INFO:teuthology.orchestra.run.smithi081.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.admin.108915.log.gz 2023-11-08T16:42:49.994 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128450.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128450.log.gz 2023-11-08T16:42:49.995 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129423.log 2023-11-08T16:42:49.996 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102654.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.102654.log.gz 2023-11-08T16:42:49.996 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105760.log 2023-11-08T16:42:49.997 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129423.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.129423.log.gz 2023-11-08T16:42:49.997 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92943.log 2023-11-08T16:42:49.998 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105760.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.105760.log.gz 2023-11-08T16:42:49.999 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105982.log 2023-11-08T16:42:49.999 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.92943.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92943.log.gz 2023-11-08T16:42:50.000 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114978.log 2023-11-08T16:42:50.001 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105982.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.105982.log.gz 2023-11-08T16:42:50.001 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109222.log 2023-11-08T16:42:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114978.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114978.log.gz 2023-11-08T16:42:50.002 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106765.log 2023-11-08T16:42:50.003 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114755.log 2023-11-08T16:42:50.004 INFO:teuthology.orchestra.run.smithi081.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.109222.log.gz 2023-11-08T16:42:50.004 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.0.106765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115147.log 2023-11-08T16:42:50.005 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114755.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114755.log.gz 2023-11-08T16:42:50.006 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114287.log 2023-11-08T16:42:50.007 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103128.log 2023-11-08T16:42:50.008 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115147.log.gz 2023-11-08T16:42:50.009 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114287.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114287.log.gz 2023-11-08T16:42:50.009 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100845.log 2023-11-08T16:42:50.010 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.103128.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103128.log.gz 2023-11-08T16:42:50.011 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102486.log 2023-11-08T16:42:50.011 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115755.log 2023-11-08T16:42:50.013 INFO:teuthology.orchestra.run.smithi081.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100845.log.gz 2023-11-08T16:42:50.014 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102486.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.102486.log.gz 2023-11-08T16:42:50.014 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94601.log 2023-11-08T16:42:50.015 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115755.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115755.log.gz 2023-11-08T16:42:50.015 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105083.log 2023-11-08T16:42:50.016 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101936.log 2023-11-08T16:42:50.016 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94601.log.gz 2023-11-08T16:42:50.017 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105083.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.105083.log.gz 2023-11-08T16:42:50.018 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109216.log 2023-11-08T16:42:50.018 INFO:teuthology.orchestra.run.smithi093.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-08T16:42:50.019 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125607.log 2023-11-08T16:42:50.020 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101936.log.gz 2023-11-08T16:42:50.021 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114203.log 2023-11-08T16:42:50.021 INFO:teuthology.orchestra.run.smithi081.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.109216.log.gz 2023-11-08T16:42:50.022 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120717.log 2023-11-08T16:42:50.022 INFO:teuthology.orchestra.run.smithi081.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.125607.log.gz 2023-11-08T16:42:50.023 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114203.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114203.log.gz 2023-11-08T16:42:50.023 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104938.log 2023-11-08T16:42:50.024 INFO:teuthology.orchestra.run.smithi081.stderr:gzip/var/log/ceph/ceph-client.admin.120717.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.101626.log 2023-11-08T16:42:50.025 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120717.log.gz 2023-11-08T16:42:50.025 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104938.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104938.log.gz 2023-11-08T16:42:50.026 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97596.log 2023-11-08T16:42:50.027 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98098.log/var/log/ceph/ceph-client.admin.101626.log: 2023-11-08T16:42:50.027 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101626.log.gz 2023-11-08T16:42:50.028 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97596.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97596.log.gz 2023-11-08T16:42:50.028 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96877.log 2023-11-08T16:42:50.029 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105125.log 2023-11-08T16:42:50.030 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98098.log: /var/log/ceph/ceph-client.admin.96877.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98098.log.gz 2023-11-08T16:42:50.030 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96877.log.gz 2023-11-08T16:42:50.031 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110356.log 2023-11-08T16:42:50.031 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105419.log 2023-11-08T16:42:50.032 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105125.log.gz 2023-11-08T16:42:50.033 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.110356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112680.log 2023-11-08T16:42:50.033 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.110356.log.gz 2023-11-08T16:42:50.035 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114531.log 2023-11-08T16:42:50.035 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105419.log.gz 2023-11-08T16:42:50.036 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112680.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112680.log.gz 2023-11-08T16:42:50.036 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122533.log 2023-11-08T16:42:50.037 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100051.log 2023-11-08T16:42:50.037 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114531.log.gz 2023-11-08T16:42:50.038 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122533.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.122533.log.gz 2023-11-08T16:42:50.038 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112321.log 2023-11-08T16:42:50.040 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113432.log 2023-11-08T16:42:50.040 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100051.log: /var/log/ceph/ceph-client.admin.112321.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100051.log.gz 2023-11-08T16:42:50.041 INFO:teuthology.orchestra.run.smithi081.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112321.log.gz 2023-11-08T16:42:50.041 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100695.log 2023-11-08T16:42:50.042 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104407.log 2023-11-08T16:42:50.042 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113432.log.gz 2023-11-08T16:42:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100695.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100695.log.gz 2023-11-08T16:42:50.043 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100093.log 2023-11-08T16:42:50.044 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5/var/log/ceph/ceph-client.admin.104407.log: --verbose -- /var/log/ceph/ceph-client.admin.112279.log 2023-11-08T16:42:50.045 INFO:teuthology.orchestra.run.smithi081.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104407.log.gz 2023-11-08T16:42:50.046 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100093.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100093.log.gz 2023-11-08T16:42:50.046 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94499.log 2023-11-08T16:42:50.047 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125911.log 2023-11-08T16:42:50.047 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112279.log.gz 2023-11-08T16:42:50.048 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94499.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94499.log.gz 2023-11-08T16:42:50.048 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97328.log 2023-11-08T16:42:50.049 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117813.log 2023-11-08T16:42:50.049 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.125911.log.gz 2023-11-08T16:42:50.050 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97328.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97328.log.gz 2023-11-08T16:42:50.051 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106212.log 2023-11-08T16:42:50.051 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.126550.log 2023-11-08T16:42:50.052 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117813.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117813.log.gz 2023-11-08T16:42:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106212.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106212.log.gz 2023-11-08T16:42:50.053 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102977.log 2023-11-08T16:42:50.054 INFO:teuthology.orchestra.run.smithi081.stderr:gzip/var/log/ceph/ceph-client.0.126550.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.101347.log 2023-11-08T16:42:50.055 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111630.log 2023-11-08T16:42:50.056 INFO:teuthology.orchestra.run.smithi081.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.admin.102977.log.gz 2023-11-08T16:42:50.056 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105040.log 2023-11-08T16:42:50.057 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101347.log.gz 2023-11-08T16:42:50.058 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112237.log 2023-11-08T16:42:50.059 INFO:teuthology.orchestra.run.smithi081.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111630.log.gz 2023-11-08T16:42:50.059 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105040.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.105040.log.gz 2023-11-08T16:42:50.060 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98491.log 2023-11-08T16:42:50.060 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103980.log 2023-11-08T16:42:50.061 INFO:teuthology.orchestra.run.smithi081.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.112237.log.gz 2023-11-08T16:42:50.062 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98491.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98491.log.gz 2023-11-08T16:42:50.062 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116996.log 2023-11-08T16:42:50.063 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.103980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101305.log 2023-11-08T16:42:50.064 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.116996.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.103980.log.gz 2023-11-08T16:42:50.064 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116996.log.gz 2023-11-08T16:42:50.065 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113324.log 2023-11-08T16:42:50.065 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110422.log 2023-11-08T16:42:50.066 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101305.log.gz 2023-11-08T16:42:50.067 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113324.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113324.log.gz 2023-11-08T16:42:50.067 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127295.log 2023-11-08T16:42:50.068 INFO:teuthology.orchestra.run.smithi081.stderr: 94.3% -- replaced with /var/log/ceph/ceph-client.0.126550.log.gz 2023-11-08T16:42:50.069 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.110422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121167.log 2023-11-08T16:42:50.069 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127295.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.110422.log.gz 2023-11-08T16:42:50.070 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100737.log 2023-11-08T16:42:50.071 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124030.log 2023-11-08T16:42:50.073 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113892.log 2023-11-08T16:42:50.073 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100737.log.gz 2023-11-08T16:42:50.074 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124030.log: 92.7% -- replaced with /var/log/ceph/ceph-client.admin.121167.log.gz 2023-11-08T16:42:50.074 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127616.log 2023-11-08T16:42:50.075 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124030.log.gz 2023-11-08T16:42:50.076 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122257.log 2023-11-08T16:42:50.076 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113892.log.gz 2023-11-08T16:42:50.076 INFO:teuthology.orchestra.run.smithi081.stderr: 94.7% -- replaced with /var/log/ceph/ceph-client.0.106765.log.gz 2023-11-08T16:42:50.077 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127616.log: 93.4% -- replaced with /var/log/ceph/ceph-client.admin.127295.log.gz 2023-11-08T16:42:50.077 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113976.log 2023-11-08T16:42:50.078 INFO:teuthology.orchestra.run.smithi081.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.127616.log.gz 2023-11-08T16:42:50.078 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118322.log 2023-11-08T16:42:50.079 INFO:teuthology.orchestra.run.smithi081.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.122257.log.gz 2023-11-08T16:42:50.080 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113976.log: gzip -5 --verbose 88.7% -- -- replaced with /var/log/ceph/ceph-client.admin.113976.log.gz /var/log/ceph/ceph-client.admin.111672.log 2023-11-08T16:42:50.080 INFO:teuthology.orchestra.run.smithi081.stderr: 2023-11-08T16:42:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118322.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118322.log.gz 2023-11-08T16:42:50.081 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96499.log 2023-11-08T16:42:50.082 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115189.log 2023-11-08T16:42:50.082 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111672.log.gz 2023-11-08T16:42:50.083 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96499.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.96499.log.gz 2023-11-08T16:42:50.084 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124114.log 2023-11-08T16:42:50.084 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114329.log 2023-11-08T16:42:50.085 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115189.log.gz 2023-11-08T16:42:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124114.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.124114.log.gz 2023-11-08T16:42:50.086 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96541.log 2023-11-08T16:42:50.087 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125833.log 2023-11-08T16:42:50.087 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114329.log.gz 2023-11-08T16:42:50.088 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96541.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96541.log.gz 2023-11-08T16:42:50.089 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94997.log 2023-11-08T16:42:50.090 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125833.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-08T16:42:50.090 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.125833.log.gz 2023-11-08T16:42:50.090 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94997.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94997.log.gz 2023-11-08T16:42:50.091 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107816.log 2023-11-08T16:42:50.092 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102062.log 2023-11-08T16:42:50.093 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107816.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107816.log.gz 2023-11-08T16:42:50.093 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128213.log 2023-11-08T16:42:50.094 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102062.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102062.log.gz 2023-11-08T16:42:50.094 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126722.log 2023-11-08T16:42:50.095 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128213.log: gzip -5 88.6% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.128213.log.gz /var/log/ceph/ceph-client.admin.114908.log 2023-11-08T16:42:50.096 INFO:teuthology.orchestra.run.smithi081.stderr: 2023-11-08T16:42:50.096 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126722.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.126722.log.gz 2023-11-08T16:42:50.097 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93155.log 2023-11-08T16:42:50.098 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106296.log 2023-11-08T16:42:50.099 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114908.log.gz 2023-11-08T16:42:50.100 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93155.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93155.log.gz 2023-11-08T16:42:50.100 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129057.log 2023-11-08T16:42:50.101 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106296.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106296.log.gz 2023-11-08T16:42:50.101 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117206.log 2023-11-08T16:42:50.102 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112381.log 2023-11-08T16:42:50.103 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.129057.log.gz 2023-11-08T16:42:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117206.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117206.log.gz 2023-11-08T16:42:50.104 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125321.log 2023-11-08T16:42:50.105 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112381.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112381.log.gz 2023-11-08T16:42:50.105 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106422.log 2023-11-08T16:42:50.106 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102677.log 2023-11-08T16:42:50.107 INFO:teuthology.orchestra.run.smithi081.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.125321.log.gz 2023-11-08T16:42:50.108 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123022.log 2023-11-08T16:42:50.108 INFO:teuthology.orchestra.run.smithi081.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.106422.log.gz 2023-11-08T16:42:50.109 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102677.log: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.102677.log.gz 2023-11-08T16:42:50.109 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99451.log 2023-11-08T16:42:50.110 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127116.log 2023-11-08T16:42:50.111 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123022.log.gz 2023-11-08T16:42:50.112 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99451.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99451.log.gz 2023-11-08T16:42:50.112 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128255.log 2023-11-08T16:42:50.113 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127116.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.127116.log.gz 2023-11-08T16:42:50.113 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127074.log 2023-11-08T16:42:50.114 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113240.log 2023-11-08T16:42:50.115 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.128255.log.gz 2023-11-08T16:42:50.116 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127074.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.127074.log.gz 2023-11-08T16:42:50.116 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118366.log 2023-11-08T16:42:50.117 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113240.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113240.log.gz 2023-11-08T16:42:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126764.log 2023-11-08T16:42:50.118 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117374.log 2023-11-08T16:42:50.119 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118366.log.gz 2023-11-08T16:42:50.120 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126764.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.126764.log.gz 2023-11-08T16:42:50.121 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123106.log 2023-11-08T16:42:50.122 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117374.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.117374.log.gz -5 2023-11-08T16:42:50.122 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.130251.log 2023-11-08T16:42:50.123 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123106.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123106.log.gz 2023-11-08T16:42:50.124 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113198.log 2023-11-08T16:42:50.124 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.130251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124454.log 2023-11-08T16:42:50.125 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130251.log.gz 2023-11-08T16:42:50.126 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113198.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.113198.log.gz 2023-11-08T16:42:50.126 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125707.log 2023-11-08T16:42:50.127 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124454.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124454.log.gz 2023-11-08T16:42:50.128 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93958.log 2023-11-08T16:42:50.128 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118620.log 2023-11-08T16:42:50.129 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93958.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.125707.log.gz 2023-11-08T16:42:50.130 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93958.log.gz 2023-11-08T16:42:50.130 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94457.log 2023-11-08T16:42:50.131 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118620.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118620.log.gz 2023-11-08T16:42:50.132 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104518.log 2023-11-08T16:42:50.132 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98406.log 2023-11-08T16:42:50.133 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94457.log.gz 2023-11-08T16:42:50.134 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104518.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.104518.log.gz 2023-11-08T16:42:50.134 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108783.log 2023-11-08T16:42:50.135 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92773.log 88.8% 2023-11-08T16:42:50.136 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98406.log.gz 2023-11-08T16:42:50.137 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108783.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.108783.log.gz 2023-11-08T16:42:50.137 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130041.log 2023-11-08T16:42:50.138 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.92773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93197.log 2023-11-08T16:42:50.139 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.130041.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.92773.log.gz 2023-11-08T16:42:50.140 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.130041.log.gz 2023-11-08T16:42:50.140 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118746.log 2023-11-08T16:42:50.141 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121583.log 2023-11-08T16:42:50.141 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93197.log.gz 2023-11-08T16:42:50.142 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118746.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118746.log.gz 2023-11-08T16:42:50.143 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109247.log 2023-11-08T16:42:50.144 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122854.log 2023-11-08T16:42:50.144 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109247.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.121583.log.gz 2023-11-08T16:42:50.145 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111714.log 2023-11-08T16:42:50.146 INFO:teuthology.orchestra.run.smithi081.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.109247.log.gz 2023-11-08T16:42:50.147 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101752.log 2023-11-08T16:42:50.147 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122854.log.gz 2023-11-08T16:42:50.148 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128618.log 2023-11-08T16:42:50.148 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111714.log.gz 2023-11-08T16:42:50.149 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101752.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101752.log.gz 2023-11-08T16:42:50.150 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103262.log 2023-11-08T16:42:50.150 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128618.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.128618.log.gz 2023-11-08T16:42:50.151 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128576.log 2023-11-08T16:42:50.152 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.103262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98779.log 2023-11-08T16:42:50.152 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103262.log.gz 2023-11-08T16:42:50.153 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128576.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.128576.log.gz 2023-11-08T16:42:50.154 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94643.log 2023-11-08T16:42:50.154 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98779.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98779.log.gz 2023-11-08T16:42:50.155 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114693.log 2023-11-08T16:42:50.156 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102761.log 2023-11-08T16:42:50.156 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94643.log.gz 2023-11-08T16:42:50.157 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114693.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114693.log.gz 2023-11-08T16:42:50.158 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122617.log 2023-11-08T16:42:50.159 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102761.log: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.102761.log.gz 2023-11-08T16:42:50.159 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104728.log 2023-11-08T16:42:50.160 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111798.log 2023-11-08T16:42:50.160 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.122617.log.gz 2023-11-08T16:42:50.161 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104728.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.104728.log.gz 2023-11-08T16:42:50.162 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123563.log 2023-11-08T16:42:50.162 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111798.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111798.log.gz 2023-11-08T16:42:50.163 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121077.log 2023-11-08T16:42:50.164 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109654.log 2023-11-08T16:42:50.164 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123563.log.gz 2023-11-08T16:42:50.165 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123521.log 2023-11-08T16:42:50.166 INFO:teuthology.orchestra.run.smithi081.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.121077.log.gz 2023-11-08T16:42:50.166 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117518.log 2023-11-08T16:42:50.167 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109654.log.gz 2023-11-08T16:42:50.168 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123521.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.123521.log.gz 2023-11-08T16:42:50.168 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117416.log 2023-11-08T16:42:50.169 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117518.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.117518.log.gz 2023-11-08T16:42:50.169 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118830.log 2023-11-08T16:42:50.170 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96709.log 2023-11-08T16:42:50.171 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.117416.log.gz 2023-11-08T16:42:50.172 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118830.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.118830.log.gz 2023-11-08T16:42:50.172 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108546.log 2023-11-08T16:42:50.173 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96709.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96709.log.gz 2023-11-08T16:42:50.174 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117122.log 2023-11-08T16:42:50.174 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108546.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108546.log.gz 2023-11-08T16:42:50.175 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119416.log 2023-11-08T16:42:50.176 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117122.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117122.log.gz 2023-11-08T16:42:50.177 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108953.log 2023-11-08T16:42:50.177 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.119416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92688.log 2023-11-08T16:42:50.178 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.119416.log.gz 2023-11-08T16:42:50.179 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119939.log 2023-11-08T16:42:50.180 INFO:teuthology.orchestra.run.smithi081.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.108953.log.gz 2023-11-08T16:42:50.180 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.92688.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.92688.log.gz 2023-11-08T16:42:50.181 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104644.log 2023-11-08T16:42:50.182 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.119939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115907.log 2023-11-08T16:42:50.182 INFO:teuthology.orchestra.run.smithi081.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.119939.log.gz 2023-11-08T16:42:50.183 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104644.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104644.log.gz 2023-11-08T16:42:50.183 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98449.log 2023-11-08T16:42:50.184 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115907.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115907.log.gz 2023-11-08T16:42:50.185 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108741.log 2023-11-08T16:42:50.186 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117771.log 2023-11-08T16:42:50.186 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98449.log.gz 2023-11-08T16:42:50.187 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108741.log: gzip -5 --verbose -- 90.6% /var/log/ceph/ceph-client.admin.117855.log 2023-11-08T16:42:50.187 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108741.log.gz 2023-11-08T16:42:50.188 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117771.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117771.log.gz 2023-11-08T16:42:50.188 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-08T16:42:50.189 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95433.log 2023-11-08T16:42:50.190 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6%/var/log/ceph/ceph-mgr.y.log: -- replaced with /var/log/ceph/ceph-client.admin.117855.log.gz 2023-11-08T16:42:50.191 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115386.log 2023-11-08T16:42:50.192 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.0.95433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104854.log 2023-11-08T16:42:50.193 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115386.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.95433.log.gz 2023-11-08T16:42:50.194 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126273.log 2023-11-08T16:42:50.194 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.115386.log.gz 2023-11-08T16:42:50.195 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111504.log 2023-11-08T16:42:50.195 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104854.log.gz 2023-11-08T16:42:50.196 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97512.log 2023-11-08T16:42:50.197 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126273.log.gz 2023-11-08T16:42:50.198 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108918.log 2023-11-08T16:42:50.198 INFO:teuthology.orchestra.run.smithi081.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111504.log.gz 2023-11-08T16:42:50.199 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121209.log 2023-11-08T16:42:50.200 INFO:teuthology.orchestra.run.smithi081.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97512.log.gz 2023-11-08T16:42:50.201 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106170.log 2023-11-08T16:42:50.202 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.108918.log.gz 2023-11-08T16:42:50.202 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101894.log 2023-11-08T16:42:50.204 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99325.log 2023-11-08T16:42:50.204 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106170.log.gz 2023-11-08T16:42:50.205 INFO:teuthology.orchestra.run.smithi081.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.121209.log.gz 2023-11-08T16:42:50.206 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120591.log 2023-11-08T16:42:50.206 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101894.log.gz 2023-11-08T16:42:50.207 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98179.log 2023-11-08T16:42:50.207 INFO:teuthology.orchestra.run.smithi081.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.99325.log.gz 2023-11-08T16:42:50.208 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120993.log 2023-11-08T16:42:50.209 INFO:teuthology.orchestra.run.smithi081.stderr: 91.4%/var/log/ceph/ceph-client.admin.98179.log: -- replaced with /var/log/ceph/ceph-client.admin.120591.log.gz 2023-11-08T16:42:50.210 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124286.log 2023-11-08T16:42:50.210 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98179.log.gz 2023-11-08T16:42:50.211 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98695.log 2023-11-08T16:42:50.211 INFO:teuthology.orchestra.run.smithi081.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.120993.log.gz 2023-11-08T16:42:50.213 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.127338.log 2023-11-08T16:42:50.213 INFO:teuthology.orchestra.run.smithi081.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.124286.log.gz 2023-11-08T16:42:50.214 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123868.log 2023-11-08T16:42:50.214 INFO:teuthology.orchestra.run.smithi081.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98695.log.gz 2023-11-08T16:42:50.216 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.127338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111309.log 2023-11-08T16:42:50.217 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124814.log 2023-11-08T16:42:50.217 INFO:teuthology.orchestra.run.smithi081.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123868.log.gz 2023-11-08T16:42:50.218 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100263.log 2023-11-08T16:42:50.219 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111309.log.gz 2023-11-08T16:42:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124814.log: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.127338.log.gz 2023-11-08T16:42:50.220 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102719.log 89.1% 2023-11-08T16:42:50.220 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.124814.log.gz 2023-11-08T16:42:50.221 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107916.log 2023-11-08T16:42:50.222 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.100263.log.gz 2023-11-08T16:42:50.223 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92600.log 2023-11-08T16:42:50.223 INFO:teuthology.orchestra.run.smithi081.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.102719.log.gz 2023-11-08T16:42:50.224 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97147.log 2023-11-08T16:42:50.224 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.107916.log.gz 2023-11-08T16:42:50.225 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.92600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118494.log 2023-11-08T16:42:50.226 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92600.log.gz 2023-11-08T16:42:50.227 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106338.log 2023-11-08T16:42:50.227 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97147.log.gz 2023-11-08T16:42:50.229 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112944.log 2023-11-08T16:42:50.229 INFO:teuthology.orchestra.run.smithi081.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118494.log.gz 2023-11-08T16:42:50.230 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124370.log 2023-11-08T16:42:50.230 INFO:teuthology.orchestra.run.smithi081.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106338.log.gz 2023-11-08T16:42:50.231 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118788.log 2023-11-08T16:42:50.232 INFO:teuthology.orchestra.run.smithi081.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112944.log.gz 2023-11-08T16:42:50.233 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126806.log 2023-11-08T16:42:50.233 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124370.log.gz 2023-11-08T16:42:50.234 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118788.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118788.log.gz 2023-11-08T16:42:50.235 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103306.log 2023-11-08T16:42:50.236 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.126806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107958.log 2023-11-08T16:42:50.236 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126806.log.gz 2023-11-08T16:42:50.237 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.103306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97105.log 2023-11-08T16:42:50.238 INFO:teuthology.orchestra.run.smithi081.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.103306.log.gz 2023-11-08T16:42:50.239 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108699.log 2023-11-08T16:42:50.239 INFO:teuthology.orchestra.run.smithi081.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107958.log.gz 2023-11-08T16:42:50.240 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103220.log 2023-11-08T16:42:50.240 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97105.log.gz 2023-11-08T16:42:50.242 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111141.log 2023-11-08T16:42:50.242 INFO:teuthology.orchestra.run.smithi081.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.108699.log.gz 2023-11-08T16:42:50.243 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.103220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101668.log 2023-11-08T16:42:50.243 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103220.log.gz 2023-11-08T16:42:50.244 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111924.log 2023-11-08T16:42:50.244 INFO:teuthology.orchestra.run.smithi081.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111141.log.gz 2023-11-08T16:42:50.246 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108490.log 2023-11-08T16:42:50.246 INFO:teuthology.orchestra.run.smithi081.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101668.log.gz 2023-11-08T16:42:50.247 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93631.log 2023-11-08T16:42:50.248 INFO:teuthology.orchestra.run.smithi081.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111924.log.gz 2023-11-08T16:42:50.248 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108490.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108490.log.gz 2023-11-08T16:42:50.249 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102935.log 2023-11-08T16:42:50.251 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93631.log: gzip -5 --verbose -- 88.5% /var/log/ceph/ceph-osd.0.log 2023-11-08T16:42:50.251 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93631.log.gz 2023-11-08T16:42:50.252 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94182.log 2023-11-08T16:42:50.253 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106904.log 2023-11-08T16:42:50.254 INFO:teuthology.orchestra.run.smithi081.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.102935.log.gz 2023-11-08T16:42:50.255 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94182.log: gzip -5 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94182.log.gz 2023-11-08T16:42:50.256 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96583.log 2023-11-08T16:42:50.256 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106904.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.106904.log.gz 2023-11-08T16:42:50.257 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120333.log 2023-11-08T16:42:50.258 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96583.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96583.log.gz 2023-11-08T16:42:50.259 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109940.log 2023-11-08T16:42:50.260 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108042.log 2023-11-08T16:42:50.260 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120333.log.gz 2023-11-08T16:42:50.261 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114489.log 2023-11-08T16:42:50.262 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109940.log.gz 2023-11-08T16:42:50.263 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122743.log 2023-11-08T16:42:50.263 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108042.log.gz 2023-11-08T16:42:50.264 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.114489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98989.log 2023-11-08T16:42:50.264 INFO:teuthology.orchestra.run.smithi081.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114489.log.gz 2023-11-08T16:42:50.266 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105932.log 2023-11-08T16:42:50.266 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.122743.log.gz 2023-11-08T16:42:50.267 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125791.log 2023-11-08T16:42:50.267 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98989.log.gz 2023-11-08T16:42:50.269 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120495.log 2023-11-08T16:42:50.269 INFO:teuthology.orchestra.run.smithi081.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.105932.log.gz 2023-11-08T16:42:50.270 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.125791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123334.log 2023-11-08T16:42:50.270 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125791.log.gz 2023-11-08T16:42:50.271 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93628.log 2023-11-08T16:42:50.272 INFO:teuthology.orchestra.run.smithi081.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120495.log.gz 2023-11-08T16:42:50.273 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99925.log 2023-11-08T16:42:50.273 INFO:teuthology.orchestra.run.smithi081.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123334.log.gz 2023-11-08T16:42:50.274 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115063.log 2023-11-08T16:42:50.275 INFO:teuthology.orchestra.run.smithi081.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93628.log.gz 2023-11-08T16:42:50.276 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100135.log 2023-11-08T16:42:50.276 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99925.log.gz 2023-11-08T16:42:50.277 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94769.log 2023-11-08T16:42:50.278 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115063.log.gz 2023-11-08T16:42:50.279 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.121968.log 2023-11-08T16:42:50.279 INFO:teuthology.orchestra.run.smithi081.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.100135.log.gz 2023-11-08T16:42:50.280 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94811.log 2023-11-08T16:42:50.281 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94769.log.gz 2023-11-08T16:42:50.282 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.0.121968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107605.log 2023-11-08T16:42:50.283 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94811.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94811.log.gz 2023-11-08T16:42:50.284 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109898.log 2023-11-08T16:42:50.285 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107605.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107605.log.gz 2023-11-08T16:42:50.285 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112986.log 2023-11-08T16:42:50.286 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109898.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109898.log.gz 2023-11-08T16:42:50.288 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112986.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112986.log.gz 2023-11-08T16:42:50.289 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109237.log 2023-11-08T16:42:50.291 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98905.log 2023-11-08T16:42:50.291 INFO:teuthology.orchestra.run.smithi081.stderr: 92.1% -- replaced with /var/log/ceph/ceph-client.0.121968.log.gz 2023-11-08T16:42:50.292 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120648.log 2023-11-08T16:42:50.293 INFO:teuthology.orchestra.run.smithi081.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.109237.log.gz 2023-11-08T16:42:50.294 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.98905.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98905.log.gz -5 2023-11-08T16:42:50.294 INFO:teuthology.orchestra.run.smithi081.stderr: --verbose -- /var/log/ceph/ceph.log 2023-11-08T16:42:50.295 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112576.log 2023-11-08T16:42:50.295 INFO:teuthology.orchestra.run.smithi081.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120648.log.gz 2023-11-08T16:42:50.297 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94559.log 2023-11-08T16:42:50.298 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112576.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112576.log.gz 2023-11-08T16:42:50.299 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128171.log 2023-11-08T16:42:50.300 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94559.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.94559.log.gz 2023-11-08T16:42:50.301 INFO:teuthology.orchestra.run.smithi081.stderr: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-08T16:42:50.301 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96793.log 2023-11-08T16:42:50.302 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128171.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.128171.log.gz 2023-11-08T16:42:50.303 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104980.log 2023-11-08T16:42:50.303 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121035.log 2023-11-08T16:42:50.304 INFO:teuthology.orchestra.run.smithi081.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96793.log.gz 2023-11-08T16:42:50.305 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130083.log 2023-11-08T16:42:50.305 INFO:teuthology.orchestra.run.smithi081.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104980.log.gz 2023-11-08T16:42:50.306 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96415.log 2023-11-08T16:42:50.307 INFO:teuthology.orchestra.run.smithi081.stderr: 90.1% -- replaced with /var/log/ceph/ceph-client.admin.121035.log.gz 2023-11-08T16:42:50.308 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.130083.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130083.log.gz 2023-11-08T16:42:50.308 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115470.log 2023-11-08T16:42:50.309 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96415.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96415.log.gz 2023-11-08T16:42:50.310 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102020.log 2023-11-08T16:42:50.311 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93365.log 2023-11-08T16:42:50.313 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102020.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102020.log.gz 2023-11-08T16:42:50.314 INFO:teuthology.orchestra.run.smithi081.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.115470.log.gz 2023-11-08T16:42:50.314 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97242.log 2023-11-08T16:42:50.315 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93365.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93365.log.gz 2023-11-08T16:42:50.315 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-08T16:42:50.316 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97242.log: gzip -5 --verbose -- 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97242.log.gz 2023-11-08T16:42:50.317 INFO:teuthology.orchestra.run.smithi081.stderr: /var/log/ceph/ceph-client.admin.120951.log 2023-11-08T16:42:50.318 INFO:teuthology.orchestra.run.smithi093.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-08T16:42:50.319 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129507.log 2023-11-08T16:42:50.320 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120951.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.120951.log.gz 2023-11-08T16:42:50.320 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125934.log 2023-11-08T16:42:50.323 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129507.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101710.log/var/log/ceph/ceph-client.admin.125934.log: 2023-11-08T16:42:50.323 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.129507.log.gz 2023-11-08T16:42:50.324 INFO:teuthology.orchestra.run.smithi081.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.125934.log.gz 2023-11-08T16:42:50.325 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.101710.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.101710.log.gz 2023-11-08T16:42:50.326 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120886.log 2023-11-08T16:42:50.327 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97638.log 2023-11-08T16:42:50.329 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120886.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120886.log.gz 2023-11-08T16:42:50.330 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97638.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97638.log.gz 2023-11-08T16:42:50.331 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-08T16:42:50.332 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104812.log 2023-11-08T16:42:50.335 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.104812.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104812.log.gz 2023-11-08T16:42:50.336 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93950.log 2023-11-08T16:42:50.339 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115338.log 2023-11-08T16:42:50.340 INFO:teuthology.orchestra.run.smithi081.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.93950.log.gz 2023-11-08T16:42:50.342 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102528.log 2023-11-08T16:42:50.344 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115338.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.115338.log.gz 2023-11-08T16:42:50.347 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.102528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107689.log 2023-11-08T16:42:50.350 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.107689.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102528.log.gz 2023-11-08T16:42:50.351 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97412.log 2023-11-08T16:42:50.352 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107689.log.gz 2023-11-08T16:42:50.352 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97470.log 2023-11-08T16:42:50.356 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97470.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97470.log.gz 2023-11-08T16:42:50.356 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112195.log 2023-11-08T16:42:50.358 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97412.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97412.log.gz 2023-11-08T16:42:50.359 INFO:teuthology.orchestra.run.smithi093.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-08T16:42:50.360 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112195.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112195.log.gz 2023-11-08T16:42:50.362 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124156.log 2023-11-08T16:42:50.364 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112806.log 2023-11-08T16:42:50.365 INFO:teuthology.orchestra.run.smithi081.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.124156.log.gz 2023-11-08T16:42:50.367 INFO:teuthology.orchestra.run.smithi081.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-08T16:42:50.368 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112806.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112806.log.gz 2023-11-08T16:42:50.368 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109503.log 2023-11-08T16:42:50.369 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102893.log 2023-11-08T16:42:50.372 INFO:teuthology.orchestra.run.smithi093.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-08T16:42:50.373 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109503.log: /var/log/ceph/ceph-client.admin.102893.log: 93.0% -- replaced with /var/log/ceph/ceph-client.admin.109503.log.gz 2023-11-08T16:42:50.373 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103896.log 2023-11-08T16:42:50.374 INFO:teuthology.orchestra.run.smithi081.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.102893.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112638.log 2023-11-08T16:42:50.375 INFO:teuthology.orchestra.run.smithi081.stderr: 2023-11-08T16:42:50.376 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97884.log 2023-11-08T16:42:50.378 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.103896.log: /var/log/ceph/ceph-client.admin.112638.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112638.log.gz 2023-11-08T16:42:50.378 INFO:teuthology.orchestra.run.smithi081.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.103896.log.gz 2023-11-08T16:42:50.379 INFO:teuthology.orchestra.run.smithi093.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-08T16:42:50.380 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123731.log 2023-11-08T16:42:50.381 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.97884.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97884.log.gz 2023-11-08T16:42:50.382 INFO:teuthology.orchestra.run.smithi093.stderr: 2023-11-08T16:42:50.382 INFO:teuthology.orchestra.run.smithi093.stderr:real 0m0.837s 2023-11-08T16:42:50.383 INFO:teuthology.orchestra.run.smithi093.stderr:user 0m3.795s 2023-11-08T16:42:50.383 INFO:teuthology.orchestra.run.smithi093.stderr:sys 0m0.405s 2023-11-08T16:42:50.385 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123731.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123731.log.gz 2023-11-08T16:42:50.385 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128973.log 2023-11-08T16:42:50.385 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106380.log 2023-11-08T16:42:50.386 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128973.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.128973.log.gz 2023-11-08T16:42:50.387 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106380.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106380.log.gz 2023-11-08T16:42:50.388 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113282.log 2023-11-08T16:42:50.389 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117729.log 2023-11-08T16:42:50.391 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.113282.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113282.log.gz 2023-11-08T16:42:50.392 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117729.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117729.log.gz 2023-11-08T16:42:50.393 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93449.log 2023-11-08T16:42:50.394 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118280.log 2023-11-08T16:42:50.404 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93449.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93449.log.gz 2023-11-08T16:42:50.404 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93963.log 2023-11-08T16:42:50.405 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118280.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118280.log.gz 2023-11-08T16:42:50.405 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103414.log 2023-11-08T16:42:50.405 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.93963.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93963.log.gz 2023-11-08T16:42:50.406 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87393.log 2023-11-08T16:42:50.406 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.103414.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103414.log.gz 2023-11-08T16:42:50.406 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94895.log 2023-11-08T16:42:50.407 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.87393.log: 91.8% -- replaced with /var/log/ceph/ceph-client.admin.87393.log.gz 2023-11-08T16:42:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99577.log 2023-11-08T16:42:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94895.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.94895.log.gz 2023-11-08T16:42:50.410 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.99577.log: /var/log/ceph/ceph-client.admin.124706.log 2023-11-08T16:42:50.411 INFO:teuthology.orchestra.run.smithi081.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99577.log.gz 2023-11-08T16:42:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94415.log 2023-11-08T16:42:50.411 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124706.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124706.log.gz 2023-11-08T16:42:50.414 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103038.log 2023-11-08T16:42:50.414 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.94415.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94415.log.gz 2023-11-08T16:42:50.414 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118704.log 2023-11-08T16:42:50.415 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.103038.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103038.log.gz 2023-11-08T16:42:50.416 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118578.log 2023-11-08T16:42:50.416 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118704.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118704.log.gz 2023-11-08T16:42:50.417 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117290.log 2023-11-08T16:42:50.419 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.118578.log: 88.5%gzip -5/var/log/ceph/ceph-client.admin.117290.log: --verbose -- /var/log/ceph/ceph-client.admin.129786.log 2023-11-08T16:42:50.419 INFO:teuthology.orchestra.run.smithi081.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.117290.log.gz 2023-11-08T16:42:50.420 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.118578.log.gz 2023-11-08T16:42:50.421 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105594.log 2023-11-08T16:42:50.422 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129786.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.129786.log.gz 2023-11-08T16:42:50.423 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117687.log 2023-11-08T16:42:50.425 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105594.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105594.log.gz 2023-11-08T16:42:50.425 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100485.log 2023-11-08T16:42:50.427 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117687.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117687.log.gz 2023-11-08T16:42:50.427 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122938.log 2023-11-08T16:42:50.428 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100485.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100485.log.gz 2023-11-08T16:42:50.429 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106024.log 2023-11-08T16:42:50.430 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.122938.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.122938.log.gz 2023-11-08T16:42:50.431 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128930.log 2023-11-08T16:42:50.432 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106024.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106024.log.gz 2023-11-08T16:42:50.433 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129015.log 2023-11-08T16:42:50.434 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128930.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128930.log.gz 2023-11-08T16:42:50.435 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129255.log 2023-11-08T16:42:50.436 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129015.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.129015.log.gz 2023-11-08T16:42:50.437 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123647.log 2023-11-08T16:42:50.438 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.129255.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.129255.log.gz 2023-11-08T16:42:50.439 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110217.log 2023-11-08T16:42:50.440 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.123647.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123647.log.gz 2023-11-08T16:42:50.441 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107032.log 2023-11-08T16:42:50.443 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.0.110217.log: /var/log/ceph/ceph-client.admin.107032.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107032.log.gz 2023-11-08T16:42:50.444 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124072.log 2023-11-08T16:42:50.447 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.124072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109209.log 2023-11-08T16:42:50.448 INFO:teuthology.orchestra.run.smithi081.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.124072.log.gz 2023-11-08T16:42:50.449 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99737.log 2023-11-08T16:42:50.453 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.109209.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.109209.log.gz 2023-11-08T16:42:50.454 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128492.log 2023-11-08T16:42:50.454 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99737.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99737.log.gz 2023-11-08T16:42:50.456 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.128492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104686.log 2023-11-08T16:42:50.457 INFO:teuthology.orchestra.run.smithi081.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.128492.log.gz 2023-11-08T16:42:50.459 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96625.log 2023-11-08T16:42:50.464 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.104686.log: /var/log/ceph/ceph-client.admin.96625.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104686.log.gz 2023-11-08T16:42:50.464 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106488.log 2023-11-08T16:42:50.467 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.106488.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96625.log.gz 2023-11-08T16:42:50.468 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.119215.log 2023-11-08T16:42:50.471 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.0.119215.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99967.log 2023-11-08T16:42:50.472 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106488.log.gz 2023-11-08T16:42:50.476 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117561.log 2023-11-08T16:42:50.481 INFO:teuthology.orchestra.run.smithi081.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99967.log.gz 2023-11-08T16:42:50.482 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.117561.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117561.log.gz 2023-11-08T16:42:50.483 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100009.log 2023-11-08T16:42:50.487 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121494.log 2023-11-08T16:42:50.487 INFO:teuthology.orchestra.run.smithi081.stderr: 94.0% -- replaced with /var/log/ceph/ceph-client.0.110217.log.gz 2023-11-08T16:42:50.488 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100009.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100009.log.gz 2023-11-08T16:42:50.489 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122107.log 2023-11-08T16:42:50.492 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.121494.log: /var/log/ceph/ceph-client.admin.122107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108000.log 2023-11-08T16:42:50.493 INFO:teuthology.orchestra.run.smithi081.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121494.log.gz 2023-11-08T16:42:50.494 INFO:teuthology.orchestra.run.smithi081.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.122107.log.gz 2023-11-08T16:42:50.495 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102355.log 2023-11-08T16:42:50.500 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.108000.log: /var/log/ceph/ceph-client.admin.102355.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120844.log 2023-11-08T16:42:50.500 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108000.log.gz 2023-11-08T16:42:50.501 INFO:teuthology.orchestra.run.smithi081.stderr: 93.4% -- replaced with /var/log/ceph/ceph-client.0.119215.log.gz 2023-11-08T16:42:50.501 INFO:teuthology.orchestra.run.smithi081.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102355.log.gz 2023-11-08T16:42:50.503 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99199.log 2023-11-08T16:42:50.504 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.120844.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120844.log.gz 2023-11-08T16:42:50.505 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96137.log 2023-11-08T16:42:50.505 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.99199.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99199.log.gz 2023-11-08T16:42:50.507 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96137.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96137.log.gz 2023-11-08T16:42:50.508 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-08T16:42:50.509 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.127959.log 2023-11-08T16:42:50.514 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.0.127959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115949.log 2023-11-08T16:42:50.515 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105844.log 2023-11-08T16:42:50.517 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.115949.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115949.log.gz 2023-11-08T16:42:50.518 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98306.log 2023-11-08T16:42:50.523 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.105844.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105844.log.gz/var/log/ceph/ceph-client.admin.98306.log: 2023-11-08T16:42:50.524 INFO:teuthology.orchestra.run.smithi081.stderr: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111588.log 2023-11-08T16:42:50.528 INFO:teuthology.orchestra.run.smithi081.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98306.log.gz 2023-11-08T16:42:50.528 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96919.log 2023-11-08T16:42:50.529 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.111588.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111588.log.gz 2023-11-08T16:42:50.532 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103830.log 2023-11-08T16:42:50.533 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.96919.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96919.log.gz 2023-11-08T16:42:50.535 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100527.log 2023-11-08T16:42:50.536 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.103830.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.103830.log.gz 2023-11-08T16:42:50.540 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130209.log 2023-11-08T16:42:50.541 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.100527.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100527.log.gz 2023-11-08T16:42:50.543 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.130209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112476.log 2023-11-08T16:42:50.544 INFO:teuthology.orchestra.run.smithi081.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.130209.log.gz 2023-11-08T16:42:50.547 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/ceph-client.admin.112476.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112476.log.gz 2023-11-08T16:42:50.549 INFO:teuthology.orchestra.run.smithi081.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-08T16:42:50.551 INFO:teuthology.orchestra.run.smithi081.stderr: 94.8% -- replaced with /var/log/ceph/ceph-client.0.127959.log.gz 2023-11-08T16:42:50.608 INFO:teuthology.orchestra.run.smithi081.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-08T16:42:50.740 INFO:teuthology.orchestra.run.smithi081.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-08T16:42:50.824 INFO:teuthology.orchestra.run.smithi081.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-08T16:42:50.947 INFO:teuthology.orchestra.run.smithi081.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-08T16:42:50.977 INFO:teuthology.orchestra.run.smithi081.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-08T16:42:51.184 INFO:teuthology.orchestra.run.smithi081.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-08T16:42:51.188 INFO:teuthology.orchestra.run.smithi081.stderr: 2023-11-08T16:42:51.188 INFO:teuthology.orchestra.run.smithi081.stderr:real 0m1.649s 2023-11-08T16:42:51.189 INFO:teuthology.orchestra.run.smithi081.stderr:user 0m5.052s 2023-11-08T16:42:51.189 INFO:teuthology.orchestra.run.smithi081.stderr:sys 0m2.172s 2023-11-08T16:42:51.190 INFO:tasks.ceph:Archiving logs... 2023-11-08T16:42:51.190 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595/remote/smithi081/log 2023-11-08T16:42:51.192 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-08T16:42:52.456 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595/remote/smithi093/log 2023-11-08T16:42:52.457 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-08T16:42:53.037 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-08T16:42:53.047 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-08T16:42:53.048 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-08T16:42:53.057 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-08T16:42:53.220 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-08T16:42:53.230 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-08T16:42:53.230 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T16:42:53.239 DEBUG:teuthology.orchestra.run.smithi093:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-08T16:42:53.288 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2023-11-08T16:42:53.295 INFO:teuthology.orchestra.run.smithi093.stderr:bash: line 1: ntpq: command not found 2023-11-08T16:42:53.298 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T16:42:53.298 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2023-11-08T16:42:53.298 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 46 +718us[ +718us] +/- 70ms 2023-11-08T16:42:53.299 INFO:teuthology.orchestra.run.smithi081.stdout:^* hv02.front.sepia.ceph.com 3 7 377 47 -992us[ -962us] +/- 76ms 2023-11-08T16:42:53.299 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 48 +454us[ +484us] +/- 85ms 2023-11-08T16:42:53.299 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T16:42:53.305 INFO:teuthology.orchestra.run.smithi093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-08T16:42:53.305 INFO:teuthology.orchestra.run.smithi093.stdout:=============================================================================== 2023-11-08T16:42:53.306 INFO:teuthology.orchestra.run.smithi093.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 49 +706us[ +746us] +/- 70ms 2023-11-08T16:42:53.306 INFO:teuthology.orchestra.run.smithi093.stdout:^* hv02.front.sepia.ceph.com 3 6 377 49 -1013us[ -973us] +/- 76ms 2023-11-08T16:42:53.306 INFO:teuthology.orchestra.run.smithi093.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 49 +501us[ +501us] +/- 85ms 2023-11-08T16:42:53.307 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-08T16:42:53.308 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-08T16:42:53.317 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-08T16:42:53.318 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-08T16:42:53.327 DEBUG:teuthology.orchestra.run.smithi081:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-08T16:42:53.615 DEBUG:teuthology.orchestra.run.smithi093:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-08T16:42:53.875 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T16:42:53.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:42:53.988 DEBUG:teuthology.orchestra.run.smithi093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-08T16:42:54.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:42:54.095 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-08T16:42:54.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-08T16:42:54.115 INFO:teuthology.task.internal:Duration was 1618.960484 seconds 2023-11-08T16:42:54.116 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-08T16:42:54.125 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-08T16:42:54.125 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-08T16:42:54.135 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-08T16:42:54.298 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T16:42:54.307 INFO:teuthology.orchestra.run.smithi093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-08T16:42:54.655 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-08T16:42:54.656 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2023-11-08T16:42:54.656 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-08T16:42:54.722 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi093.front.sepia.ceph.com 2023-11-08T16:42:54.723 DEBUG:teuthology.orchestra.run.smithi093:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-08T16:42:54.801 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-08T16:42:54.801 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-08T16:42:54.811 DEBUG:teuthology.orchestra.run.smithi093:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-08T16:42:54.906 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-08T16:42:54.907 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-08T16:42:54.913 DEBUG:teuthology.orchestra.run.smithi093:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-08T16:42:55.115 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-08T16:42:55.125 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-08T16:42:55.126 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-08T16:42:55.222 DEBUG:teuthology.orchestra.run.smithi093:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-08T16:42:55.316 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-08T16:42:55.327 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-08T16:42:55.336 DEBUG:teuthology.orchestra.run.smithi093:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-08T16:42:55.442 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2023-11-08T16:42:55.446 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern = core 2023-11-08T16:42:55.499 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-08T16:42:55.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:42:55.550 DEBUG:teuthology.orchestra.run.smithi093:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-08T16:42:55.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-08T16:42:55.607 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-08T16:42:55.618 INFO:teuthology.task.internal:Transferring archived files... 2023-11-08T16:42:55.619 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595/remote/smithi081 2023-11-08T16:42:55.619 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-08T16:42:55.733 DEBUG:teuthology.misc:Transferring archived files from smithi093:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595/remote/smithi093 2023-11-08T16:42:55.734 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-08T16:42:55.842 INFO:teuthology.task.internal:Removing archive directory... 2023-11-08T16:42:55.843 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-08T16:42:55.852 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-08T16:42:55.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-08T16:42:55.919 INFO:teuthology.task.internal:Not uploading archives. 2023-11-08T16:42:55.919 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-08T16:42:55.929 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-08T16:42:55.930 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-08T16:42:55.938 DEBUG:teuthology.orchestra.run.smithi093:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-08T16:42:55.989 INFO:teuthology.orchestra.run.smithi081.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 8 16:42 /home/ubuntu/cephtest 2023-11-08T16:42:55.997 INFO:teuthology.orchestra.run.smithi093.stdout: 393249 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 8 16:42 /home/ubuntu/cephtest 2023-11-08T16:42:56.000 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-08T16:42:56.011 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-08T16:42:56.032 INFO:teuthology.nuke:Checking targets against current locks 2023-11-08T16:42:56.054 DEBUG:teuthology.nuke:shortname: smithi081 2023-11-08T16:42:56.055 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T16:42:56.078 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 16:06:48.927014', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEb/Zd1RkyJlvHItW402GsLGrVK/DxAP1UQ3Ffdb0JiiZSX55PYP8qQj1NkQP07dHaUwrlKgI7Xouph9A5trxkU='} 2023-11-08T16:42:56.081 DEBUG:teuthology.nuke:shortname: smithi093 2023-11-08T16:42:56.082 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-08T16:42:56.104 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-11-06_10:33:49-fs-wip-vshankar-testing-20231106.073650-testing-default-smithi/7449595', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-08 16:06:48.924968', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBITVhNVsh9QJgG0rtGpBJz4qGMcOK6+9C0lzvZOaEmIqMbNTLzfFp3YuUcIX2Eq0w7oJ2Tjqx/svsh6/pCfWsGY='} 2023-11-08T16:42:56.133 INFO:teuthology.orchestra.console.smithi081:Power off 2023-11-08T16:42:56.134 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-08T16:42:56.158 DEBUG:teuthology.orchestra.console.smithi081:power off output: Chassis Power Control: Down/Off 2023-11-08T16:42:56.158 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T16:42:56.174 INFO:teuthology.orchestra.console.smithi093:Power off 2023-11-08T16:42:56.175 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-08T16:42:56.185 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2023-11-08T16:42:56.198 DEBUG:teuthology.orchestra.console.smithi093:power off output: Chassis Power Control: Down/Off 2023-11-08T16:42:56.198 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T16:42:56.221 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on 2023-11-08T16:43:00.188 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T16:43:00.222 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T16:43:00.298 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2023-11-08T16:43:00.334 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on 2023-11-08T16:43:04.301 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T16:43:04.335 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-08T16:43:04.412 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off 2023-11-08T16:43:04.447 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off 2023-11-08T16:43:04.513 INFO:teuthology.orchestra.console.smithi081:Power off completed 2023-11-08T16:43:04.549 INFO:teuthology.orchestra.console.smithi093:Power off completed 2023-11-08T16:43:04.654 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/data-scan} duration: 1618.960483789444 failure_reason: 'Test failure: test_fragmented_injection (tasks.cephfs.test_data_scan.TestDataScan)' flavor: default mon.a-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 mon.b-kernel-sha1: 5b73517e0e2fa132ea79290e5fc06ea5349b2288 owner: scheduled_vshankar@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=52ac173d6eff4124aec04af447bcd126 status: fail success: false 2023-11-08T16:43:04.655 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-08T16:43:04.750 INFO:teuthology.run:FAIL